performance consideration for bulk loading

hoseiney · July 2017

We have 1 Node Vertica Database ...
how can we increase performance for loading data using COPY command in vertica?
based on our tests, in a server with 2 CPU core and 8 GB RAM, loading a json file with 1 million (900 MB size) takes 70 seconds, same file in a server with 128 GB RAM and 32 CPU core takes 30 seconds...
if we want for example loading this file takes 10 seconds what we should do ?

François_JEHL · August 2017

I think you should talk about both your IO and network throughput. Use vioperf and vnetworkperf and check the bottleneck and if it's coherent wrt to your hardware specs.

TomM · August 2017

I don't know if you'll find this useful, and in terms of improving load performance my understanding is one of your options is to try apportioned load.

https://my.vertica.com/docs/8.1.x/HTML/index.htm#Authoring/ExtendingVertica/UDx/UDL/ApportionedLoad.htm?Highlight=apportioned load

We're Moving!

Create My New Community Account Now

performance consideration for bulk loading

Answers

Leave a Comment