Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!

where to find COPY error files in EON mode

Hi forum,
we have started our first data COPYs to an EON DB. Can you advise where the error files are located?
I have found a directory metadata/dbname/nodes/nodename/Catalog directory with to subdirs, Checkpoints and Txnlogs.
Is this the place to look for?
Please advise.
Thanks
Dieter

Comments

  • There should be a directory CopyErrorLogs in the catalog directory.
    Can you also explicitly specify the file name or table name in the copy command.
    see copy command syntax for that

  • The error logs are written to S3, not the DB node, correct?
    The metadata/dbname/nodes/nodename/Catalog in the S3 bucket (ROS location) is the catalog location?
    Thanks

  • Jim_KnicelyJim_Knicely Administrator
    edited December 2017

    Check the nodes system table for the catalog path:

    Example:

    dbadmin=> select catalog_path from nodes;
                        catalog_path
    ----------------------------------------------------
     /home/dbadmin/sfdc/v_sfdc_node0001_catalog/Catalog
    (1 row)
    

    As @skeswani said, there should be a CopyErrorLogs directory in the catalog directory.

    Example:

    [[email protected] v_sfdc_node0001_catalog]$ ls /home/dbadmin/sfdc/v_sfdc_node0001_catalog/CopyErrorLogs | head -2
    customers-data.txt-copy-from-exceptions
    customers-data.txt-copy-from-rejected-data
    

    Would it be easier to use the "rejected data table" functionality of the COPY command?

  • Thanks Jim, good to know the catalog is on the V node, NOT in S3.

  • We have a local copy of the catalog on the V node. There's also a copy in S3. But the error logs won't end up in S3 unless you ask for them to be written there. I would try
    COPY foo FROM 's3://mybucket/data/*' REJECTED DATA 's3://mybucket/loaderrors/';
    or (even cooler!) REJECTED DATA AS TABLE foo_errors;

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file

Can't find what you're looking for? Search the Vertica Documentation, Knowledge Base, or Blog for more information.