Max rows per file in COPY command

VerticaQ

Hi

 

I am using COPY command to load 1 file with 27 million rows in it and i am getting an error [HY000] Unexpected protocol message: DataRow

 

when doing it on a file with 14 million rows there is no problem.

 

Is there a limit on the number of rows in a file?

eli_revach

Hi ,

Their is no limit on the amount of the rows per file , you should provide more deatils about the content of the file  the method you using the COPY command , and your vertica setup , it may for getting more info about what you trying to do 

 

Thanks 

sKwa

Hi!

 

 

Limit of cause exist: number of rows per load - 2^63 = 9,223,372,036,854,775,808 rows

http://my.vertica.com/docs/7.1.x/HTML/index.htm#Authoring/SQLReferenceManual/SystemLimits.htm

 

>> Unexpected protocol message: DataRow

Here is a problem. DataRow - is response message generated by SELECT, not by COPY.

 

Can you provide more info?

• db version
• client (vsql/JDBC/ODBC/ADO/Tableu/...???)
• file is locally or remotely
• post your COPY statement please.

VerticaQ

Hi

 

1 Vertica 7.1.2-2

2 .NET (ADO)

3 Locally

4 copy command sample:

COPY DB.SCHEMA.TABLE (col1,col2,col3)
FROM LOCAL C:\MyFile.csv UNCOMPRESSED DELIMITER ',' NULL AS 'NULL' 
ESCAPE AS '\\' ENCLOSED BY '\"' DIRECT STREAM NAME 'StreamName' 
EXCEPTIONS 'C:\exceptions' REJECTED DATA 'C:\rejections' 
ABORT ON ERROR NO COMMIT; 

 

eli_revach

Did you try to run this copy command from vsql ? 

sKwa

Hi!

 

I think that DataRow comes here to describe a rejected row and fails. Have you successfully uploaded 14M of data(without any rejection)?

 

BTW: your COPY statement is strange: exceptions+rejections and abort on error+no commit. Can you explain your logic? I can't get it.

 

Disclaimer: Im not an expert in ADO(.NET) so maybe someone else can help you, but 99.9% problem in ADO.

 

PS

http://www.akadia.com/services/dotnet_handling_row_errors.html

 

sKwa

Hi!

 

I don't know why, but my replay deleted and I will not repost it

Sorry.