Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!

file handle count calculation

Hi All,

 

I have been struggling to find a relation between Open file handle count and Vertica process on each node.

How can we determine, how many files does vertica open for a query.

I know we can use resource acquisitions table, but I need to understand the logic behind these numbers.

 

Logically.

A ROS container can have many files, one file for each column of that container.

 

If a select query with only one column is executed, then it should open only the index file and file containing the column values in that ROs container as a result of partition pruning, but this is not observed in simple cases.

 

I tried querying the single column of a partitioned table with 20 values. The data was distributed on all nodes of a 3 node cluster, but the query shows file handle count as 

Node 1 : 7 (initiator)

Node 2 : 4

Node 3 : 4

 

I query two columns and the result changes

Node 1 : 9 (initiator)

Node 2 : 6

Node 3 : 6

 

Any expalnation to logic would be appreciated.

 

Thanks

Comments

  • Hi ,

    Looks like from 7.X and above you no longer have pidx and fdb files , vertica do not maintain the index file in seperate file , for now on , you will see file with extention _0.gt .

     

    Now for your specific use case  i see few options :

    1. Mutipule ROS files per specific columns 
    2. Mutipule ROS files per specific columns + temp files (Sort_Temp_<id>.dat) , the temp files are create and delete at the end of the query 

     

    Very easy way to monitor which files are open is using lsof utility , you can corellect the name of the files you get from the lsof utility (try to grep the results by data location you have on the file system )  with the sal_storage_id column exists in the storage_containers table .

     

    Hope you will find it useful

    Thanks .

     

     

     

     

     

     

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file

Can't find what you're looking for? Search the Vertica Documentation, Knowledge Base, or Blog for more information.