Selectively retaining data from data collectors

While the execution_engine_profiles data collector is very useful for capturing query metrics, the table grows large very quickly. Is there a way to selectively retain certain metrics?

More generally, can a data collector retain information in a query optimized format (regular permanent tables)?

Answers

  • ChuckBChuckB Vertica Employee Employee

    There are several answers to this.
    1. You can up the retention for the data collector tables.
    2. You can send the DC data elsewhere, using Kafka: https://www.vertica.com/blog/publish-data-collector-tables-apache-kafka/
    3. You can copy the DC data to a table; vertica generates example scripts on how to do this and puts them in your DataCollector directory when it starts up.
    4. For the execution_engine_profiles table specifically, consider increasing the time threshold for recording queries, so that the shortest ones aren't retained. See "SaveDCEEProfileThresholdUS".

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file