COPY Fails when multiple s3 files are passed

prad33p2301

Hi,

I have a copy command like this

COPY SDNSLOGS_stg FROM 
's3://raw/converted-logs/test/4cff0890-f597-4374-9671-edaaa0ce8bef.gz'
,'s3://raw/converted-logs/test/4ecdebb7-f401-47d7-a3d1-6bcce1329aab.gz'
,'s3://raw/converted-logs/test/6110cfe7-b383-4166-a0a1-8acb2de9258f.gz'

GZIP
DELIMITER '^'
RECORD TERMINATOR '▼'
REJECTED DATA AS TABLE loader_rejects100;   

When I run the COPY like above with 3 file names I am seeing that only data from the last file 6110*.gz is getting loaded while the others are rejected with errors saying **Invalid input syntax for date

There is actually no error in the first and the second file too !

I confirmed that by running COPY with just one of the file name

COPY SDNSLOGS_stg FROM 
's3://raw/converted-logs/test/4ecdebb7-f401-47d7-a3d1-6bcce1329aab.gz'
GZIP
DELIMITER '^'
RECORD TERMINATOR '▼'
REJECTED DATA AS TABLE loader_rejects101;

why am I seeing this problem when trying to copy multiple files

prad33p2301

Hi

I figured this out after reading the documentation clearly,

if there are multiple files being passed we have to mention the compression format if any for each of the files

COPY SDNSLOGS_stg FROM 
's3://raw/converted-logs/test/4cff0890-f597-4374-9671-edaaa0ce8bef.gz'  GZIP
,'s3://raw/converted-logs/test/4ecdebb7-f401-47d7-a3d1-6bcce1329aab.gz' GZIP
,'s3://raw/converted-logs/test/6110cfe7-b383-4166-a0a1-8acb2de9258f.gz'  GZIP

DELIMITER '^'
RECORD TERMINATOR '▼'
REJECTED DATA AS TABLE loader_rejects100;