The Vertica Forum recently got a makeover! Let us know what you think by filling out this short, anonymous survey.
Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!

VerticaCopyStream processing speed

We use vertica 7 community edition on 20cores 32GB physical server. When we run VerticaCopyStream from our java program stream of 6.5MB (42k rows) is processed in 42seconds. CPU and disk utilisation is about 1%. How to improve it? I need to import handreds of GB. COPY #tmp_table_name FROM LOCAL STDIN DELIMITER '|' DIRECT ENFORCELENGTH We import the data to tmp table and then merge the data to other tables. BTW merge is fast enough , it takes few milliseconds.


  • Hi ,

    How much it take when you try to load  the same  file via simple copy command from vsql ?   do your java process run on the db server or on some other server , if on other server what is the CPU and memory utilization you see their ?   

    One of the way to scale data load is to run parallel load streams  on different nodes .

  • We run our java app on different server. Import via vsql on the same node takes milliseconds. It's network problem. Thank you for the help.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file