Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!
How to handle duplicate key errors as REJECTED DATA during COPY
Hi,
is there a way to handle duplicate key errors during COPY the same way as parsing errors, i.e. load them into the REJECTED DATA TABLE?
Currently, I catch the exception, create a tmp table without constraints re-run the COPY statement and then INSERT the distinct rows into the original table. However, that is not possible when I want to automatically run COPY using a scheduler.
best,
Jörg
0
Leave a Comment
Can't find what you're looking for? Search the Vertica Documentation, Knowledge Base, or Blog for more information.
Answers
if you are loading json data, then there is one option. please use --parser-parameters and pass reject_on_duplicate to true and it will reject the rows with duplicate key errors.. please find the below links for the same.
https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/KafkaIntegrationGuide/UtilityOptions/LoadSpecUtilityOptions.htm?
https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/KafkaIntegrationGuide/KafkaFunctions/KafkaJSONParser.htm?Highlight=reject_on_duplicate
Thanks for the hint, but my COPY comand still throws an duplicate key error. The command I use is
Could you share the Vertica server version, exact text of the COPY error, and if possible a sample JSON that triggers the error? Please open a support case if you'd rather not share details on public forum.