Logic behind load_streams monitoring table

K__1

I am trying to monitor our bulk loading using the load_streams table but I do not completely understand how it works. Is it supposed to be cleared/trimmed after it hits a certain point (internal variable). Whats the logic behind clearing it? Whats the max size? Is it possible to modify it? If we start loading data if its close to its storage threshold, does it clear the entire table? 

[Deleted User]

The Load stream table is a view of the data collector tables, most of the v_monitor.* tables are views of the data_collector tables so to increase the size you need to increase the size of the underline data_collector table. 

The data_collector tables retention is by size and start to purge the data when it reach the size, it removes older data, it does not truncate.  You can look at the documentation for all the functions in how to get, set or flush the data_collector data. Here is a link but you can also search for data_collector keyword and you will get the related links,

https://my.vertica.com/docs/7.0.x/HTML/index.htm#Attic/Troubleshooting/WorkingWithDataCollectionLogs...


In the load stream case is a view of the dc_execution_engine_profiles tables so you need to increase the retention policy of that table to see more data in the load_stream table. 

I hope it make sense, 

Eugenia

K__1

Yup it did. Thanks for your response. I have a followup questions. So following the view definitions and the table they reference I believe the following dc_* tables need to be given more space for our environment. 

dc_errors
dc_execution_engine_profiles
dc_projections_used
dc_requests_completed
dc_requests_issued

On looking up the view definition of execution_engine_profiles it references v_internal.vs_execution_engine_profiles. I am assuming this is an actual table and not a view. Can you please confirm?

Thanks! 

K__1

I am seeing data in the load streams table with the following stats

accepted_row_count        | 0
rejected_row_count          | 0
read_bytes                       | 0
input_file_size_bytes         | 0
parse_complete_percent |
unsorted_row_count         | 13328
sorted_row_count             | 13328
sort_complete_percent    | 100

I need a little explanation about how this is valid. Does this mean the copy failed but then the rejected_row_count is 0. Please explain. 

Thanks!