file handle count calculation

Navin_C

Hi All,

 

I have been struggling to find a relation between Open file handle count and Vertica process on each node.

How can we determine, how many files does vertica open for a query.

I know we can use resource acquisitions table, but I need to understand the logic behind these numbers.

 

Logically.

A ROS container can have many files, one file for each column of that container.

 

If a select query with only one column is executed, then it should open only the index file and file containing the column values in that ROs container as a result of partition pruning, but this is not observed in simple cases.

 

I tried querying the single column of a partitioned table with 20 values. The data was distributed on all nodes of a 3 node cluster, but the query shows file handle count as 

Node 1 : 7 (initiator)

Node 2 : 4

Node 3 : 4

 

I query two columns and the result changes

Node 1 : 9 (initiator)

Node 2 : 6

Node 3 : 6

 

Any expalnation to logic would be appreciated.

 

Thanks

eli_revach

Hi ,

Looks like from 7.X and above you no longer have pidx and fdb files , vertica do not maintain the index file in seperate file , for now on , you will see file with extention _0.gt .

 

Now for your specific use case  i see few options :

1. Mutipule ROS files per specific columns 
2. Mutipule ROS files per specific columns + temp files (Sort_Temp_<id>.dat) , the temp files are create and delete at the end of the query 

 

Very easy way to monitor which files are open is using lsof utility , you can corellect the name of the files you get from the lsof utility (try to grep the results by data location you have on the file system )  with the sal_storage_id column exists in the storage_containers table .

 

Hope you will find it useful

Thanks .