Options
Performance Delimiter Vs. Fcsvparser
Hi
I would like to know, why would fcsvparser taking more time to complete compared to delimiter as below:
dbadmin=> copy fs.columnar_tbl1 from '/home/shared/300MB.csv' delimiter ',' skip 1;
Rows Loaded
-------------
261607
(1 row)
Time: First fetch (1 row): 26819.648 ms.
dbadmin=> copy fs.columnar_tbl2 from '/home/shared/300MB.csv' parser fcsvparser();
Rows Loaded
-------------
261607
(1 row)
Time: First fetch (1 row): 58866.387 ms.
Many Thanks
0
Comments
Most likely because with delimiter it only parses for comas, while the fcsvparser also looks for quotes, escape characters, headers... And by default, trims values.
[Optional] Indicates whether to trim white space from header names and key values.
Default value: true
https://my.vertica.com/docs/7.2.x/HTML/Content/Authoring/FlexTables/FCSVPARSERreference.htm