Selectively retaining data from data collectors

kxukxu Administrator

While the execution_engine_profiles data collector is very useful for capturing query metrics, the table grows large very quickly. Is there a way to selectively retain certain metrics?

More generally, can a data collector retain information in a query optimized format (regular permanent tables)?

Answers

  • ChuckBChuckB Employee

    There are several answers to this.
    1. You can up the retention for the data collector tables.
    2. You can send the DC data elsewhere, using Kafka: https://www.vertica.com/blog/publish-data-collector-tables-apache-kafka/
    3. You can copy the DC data to a table; vertica generates example scripts on how to do this and puts them in your DataCollector directory when it starts up.
    4. For the execution_engine_profiles table specifically, consider increasing the time threshold for recording queries, so that the shortest ones aren't retained. See "SaveDCEEProfileThresholdUS".

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file