We're Moving!

The Vertica Forum is moving to a new OpenText Analytics Database (Vertica) Community.

Join us there to post discussion topics, learn about

product releases, share tips, access the blog, and much more.

Create My New Community Account Now


file handle count calculation — Vertica Forum

file handle count calculation

Navin_CNavin_C Vertica Customer

Hi All,

 

I have been struggling to find a relation between Open file handle count and Vertica process on each node.

How can we determine, how many files does vertica open for a query.

I know we can use resource acquisitions table, but I need to understand the logic behind these numbers.

 

Logically.

A ROS container can have many files, one file for each column of that container.

 

If a select query with only one column is executed, then it should open only the index file and file containing the column values in that ROs container as a result of partition pruning, but this is not observed in simple cases.

 

I tried querying the single column of a partitioned table with 20 values. The data was distributed on all nodes of a 3 node cluster, but the query shows file handle count as 

Node 1 : 7 (initiator)

Node 2 : 4

Node 3 : 4

 

I query two columns and the result changes

Node 1 : 9 (initiator)

Node 2 : 6

Node 3 : 6

 

Any expalnation to logic would be appreciated.

 

Thanks

Comments

  • Hi ,

    Looks like from 7.X and above you no longer have pidx and fdb files , vertica do not maintain the index file in seperate file , for now on , you will see file with extention _0.gt .

     

    Now for your specific use case  i see few options :

    1. Mutipule ROS files per specific columns 
    2. Mutipule ROS files per specific columns + temp files (Sort_Temp_<id>.dat) , the temp files are create and delete at the end of the query 

     

    Very easy way to monitor which files are open is using lsof utility , you can corellect the name of the files you get from the lsof utility (try to grep the results by data location you have on the file system )  with the sal_storage_id column exists in the storage_containers table .

     

    Hope you will find it useful

    Thanks .

     

     

     

     

     

     

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file