where to find COPY error files in EON mode

DieterC

Hi forum,
we have started our first data COPYs to an EON DB. Can you advise where the error files are located?
I have found a directory metadata/dbname/nodes/nodename/Catalog directory with to subdirs, Checkpoints and Txnlogs.
Is this the place to look for?
Please advise.
Thanks
Dieter

skeswani

There should be a directory CopyErrorLogs in the catalog directory.
Can you also explicitly specify the file name or table name in the copy command.
see copy command syntax for that

DieterC

The error logs are written to S3, not the DB node, correct?
The metadata/dbname/nodes/nodename/Catalog in the S3 bucket (ROS location) is the catalog location?
Thanks

Jim_Knicely

Check the nodes system table for the catalog path:

Example:

dbadmin=> select catalog_path from nodes;
                    catalog_path
----------------------------------------------------
 /home/dbadmin/sfdc/v_sfdc_node0001_catalog/Catalog
(1 row)

As @skeswani said, there should be a CopyErrorLogs directory in the catalog directory.

Example:

[dbadmin@s18384357 v_sfdc_node0001_catalog]$ ls /home/dbadmin/sfdc/v_sfdc_node0001_catalog/CopyErrorLogs | head -2
customers-data.txt-copy-from-exceptions
customers-data.txt-copy-from-rejected-data

Would it be easier to use the "rejected data table" functionality of the COPY command?

DieterC

Thanks Jim, good to know the catalog is on the V node, NOT in S3.

Ben_Vandiver

We have a local copy of the catalog on the V node. There's also a copy in S3. But the error logs won't end up in S3 unless you ask for them to be written there. I would try
COPY foo FROM 's3://mybucket/data/*' REJECTED DATA 's3://mybucket/loaderrors/';
or (even cooler!) REJECTED DATA AS TABLE foo_errors;