Multithreaded Java UDL Source

mti

Hi Vertica gurus,

 

I have written a java user defined source which I use with Vertica COPY command. It reads data from a given source and writes it into a Vertica table. This works perfectly fine. Now, the problem is that sometimes these data objects could be as huge as 500 GB. So, I was wondering if it is possible to make my UDL Source work in such a fashion that it could read data from the source in multiple chunks and write them into the Vertica table in parallel. I am not able to figure out how exactly I can open parallel connections from within my UDL Source for each input data chunk and push the data into the target table.

 

Pardon my ignorance as I am completely new to Vertica. 

 

Thank you so much for your valuable time.

Sattipandu

Do you work for HP? If yes, can you please explian how you have connected to other data sources using Java UDF and how are you bringing data to vertica staging?