Space usage in DataCollector directory

usao

The DataCollector directory in my Vertica installation is taking a significant amount of space from the catalog filesystem allocation, and they all appear to be "log" files, so I was wondering if it was possible to purge/remove any of these files?
I don't know what they are used for, and not sure if they are critical to Vertica or if that is something I can clean up regularly through a cron job...

Jim_Knicely

Hi,

You can control the size restraint (memory and disk space in kilobytes) for the specified Data Collector table on all nodes via the SET_DATA_COLLECTOR_POLICY function.

See:
https://my.vertica.com/docs/9.1.x/HTML/index.htm#Authoring/SQLReferenceManual/Functions/VerticaFunctions/DataCollection/SET_DATA_COLLECTOR_POLICY.htm

You can see all of the DC function here:
https://my.vertica.com/docs/9.1.x/HTML/index.htm#Authoring/SQLReferenceManual/Functions/VerticaFunctions/DataCollection/DataCollectorFunctions.htm

Here is a quick example of using the CLEAR_DATA_COLLECTOR function ...

[dbadmin@s18384357 DataCollector]$ du -h
132M     .
[dbadmin@s18384357 DataCollector]$ vsql -c "SELECT clear_data_collector();"
 clear_data_collector
----------------------
 CLEAR
(1 row)

[dbadmin@s18384357 DataCollector]$ du -h
2.7M    .

usao

That worked...
[dbadmin@localhost DataCollector]$ du -sh .
38.2G .
[dbadmin@localhost DataCollector]$ vsql -c "SELECT clear_data_collector();"
Password:

clear_data_collector

CLEAR
(1 row)

[dbadmin@localhost DataCollector]$ du -sh .
2.7M .

Jim_Knicely

Nice! You might want to alter the data retention policy of particular DC tables.

See:
https://my.vertica.com/docs/9.1.x/HTML/index.htm#Authoring/AdministratorsGuide/Monitoring/Vertica/ConfiguringDataRetentionPolicies.htm