CDC Replication Oracle to HP Vertica

rafael_sisweb

Hi,

I'm working on a tool that read CDC from oracle logs. This collect in a contant process all DML commands from an oracle database and persist to an staging area. On another side i have an applier that read this commands from this staging area, transform commands to correct vertica sintax, and then execute this commands on my vertica database.

Everything is working well, but some tables has a lot of changes in a small intevall, somethin around 1k commands per minute. Many of this commands are Updates and Deletes and some of this take more than 2 seconds to run, and because of that is impossible to sync this tables.

 

I'd tried tu run execute this commands in batch processes, single transactions, with and without /*+ DIRECT */ markup, but the results are always the same. Collecting something like 300 commands each 5 sec and taking more than 3 minutes to run in vertica.

 

Somebody has any tips about the best strategie to be used in this case?