The Vertica Forum recently got a makeover! Let us know what you think by filling out this short, anonymous survey.
Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!
Selectively retaining data from data collectors
While the execution_engine_profiles data collector is very useful for capturing query metrics, the table grows large very quickly. Is there a way to selectively retain certain metrics?
More generally, can a data collector retain information in a query optimized format (regular permanent tables)?
There are several answers to this.
1. You can up the retention for the data collector tables.
2. You can send the DC data elsewhere, using Kafka: https://www.vertica.com/blog/publish-data-collector-tables-apache-kafka/
3. You can copy the DC data to a table; vertica generates example scripts on how to do this and puts them in your DataCollector directory when it starts up.
4. For the execution_engine_profiles table specifically, consider increasing the time threshold for recording queries, so that the shortest ones aren't retained. See "SaveDCEEProfileThresholdUS".