Viewing Parquet Export Events More Easily
Jim_Knicely
- Select Field - Administrator
The EXPORT TO PARQUET command exports a table, columns from a table, or query results to files in the Parquet format. When you run EXPORT TO PARQUET information about the files created during the export is stored in the Vertica log. It's no fun combing through a Vertica log looking for those particular records.
Good news! As of Vertica 9.3.1, the V_MONITOR.UDX_EVENTS system table now also records the events logged during the export!
Example:
dbadmin=> SELECT * FROM test;
c1
----
1
2
3
(3 rows)
dbadmin=> EXPORT TO PARQUET(DIRECTORY='/home/dbadmin/parq/') AS SELECT * FROM test;
Rows Exported
---------------
3
(1 row)
dbadmin=> SELECT created, file, rows FROM v_monitor.udx_events WHERE udx_name = 'ParquetExport';
created | file | rows
-------------------------------+----------------------------------------------------------------------------------+------
2020-01-27 10:05:30.166202-05 | /home/dbadmin/parqhUGvipCI/ad3a94ae-v_test_db_node0002-139833020229376-0.parquet | 3
(1 row)
Helpful Links:
https://www.vertica.com/docs/9.3.x/HTML/Content/Authoring/SQLReferenceManual/Statements/EXPORTTOPARQUET.htm
https://www.vertica.com/docs/9.3.x/HTML/Content/Authoring/HadoopIntegrationGuide/NativeFormats/ExportMonitor.htm
https://www.vertica.com/docs/9.3.x/HTML/Content/Authoring/SQLReferenceManual/SystemTables/MONITOR/UDX_EVENTS.htm
Have Fun!