obtain all value data modified or inserted from all the tables from a concrete schema hp vertica 7.1
We have developed an api rest which obtained data from schema database hp vertica 7.1using some methods developed in the api rest,
Now we want to build a batch process to execute every five minutes which obtain all the value differences happened in the database..
Is there any function or procedure or projection in hp vertica to obtain all the values inserted, modified and deleted since the last execution of the batch process?
How could we obtain the differences of the value registers of all tables of schema database between the last execution of the batch process with the actual execution of the same batch process?
We desestimate the option of doing a complete import every time because hp vertica could have a lot of data.
Thanks in advance
Best regards
Josep Maria
Comments
As I know there are no specail functions for that purpose like there is no redolog to parse all changes.
I think the only way will be to add special technical column smthng like processed_dt timestamp.
Every time you insert, update or delete (technicaly you should make logical delete not physical ) you mark processed_dt as sysdate
Than you can synchronize all changes by processed_dt.
PS
Also you can use protion namber - processed_id number instead of timestamp. All you need is a special metadata table to lookup portions ids (did you get this portion yet ot not)