The Vertica Forum recently got a makeover! Let us know what you think by filling out this short, anonymous survey.
Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!
VerticaCopyStream processing speed
We use vertica 7 community edition on 20cores 32GB physical server. When we run VerticaCopyStream from our java program stream of 6.5MB (42k rows) is processed in 42seconds. CPU and disk utilisation is about 1%. How to improve it? I need to import handreds of GB. COPY #tmp_table_name FROM LOCAL STDIN DELIMITER '|' DIRECT ENFORCELENGTH We import the data to tmp table and then merge the data to other tables. BTW merge is fast enough , it takes few milliseconds.
0
Comments
How much it take when you try to load the same file via simple copy command from vsql ? do your java process run on the db server or on some other server , if on other server what is the CPU and memory utilization you see their ?
One of the way to scale data load is to run parallel load streams on different nodes .