bulk loading

Amit_Pandey · August 2013

Hi, I am trying to load a .txt file with nearly 33 Million integers( separated with spaces) which I am storing in a table as a single column of integers. I am using COPY with Direct option. Though on execution I am not getting any error but the loading is not happening even after hours of waiting. .txt file is nearly of 1.2 Gb. I am using community edition.

Daniel_Leybovic · August 2013

Hi! 1. You can check COPY load metrics using: Vertica functions LOAD_STREAMS system table https://my.vertica.com/docs/6.1.x/HTML/index.htm#12067.htm 2. What is exact options of copy command? What is definition of RECORD TERMINATOR ? COPY 'table name' FROM '/path/to/data.txt' DIRECT RECORD TERMINATOR ',' 3. try option ABORT ON ERROR - will output a reason for rejection. COPY 'table name' FROM '/path/to/data.txt' DIRECT RECORD TERMINATOR ',' ABORT ON ERROR 4. Check rejected and exception files or define it in COPY: COPY 'table name' FROM '/path/to/data.txt' DIRECT RECORD TERMINATOR ',' REJECTED DATA 'path' EXCEPTIONS 'path' https://my.vertica.com/docs/6.1.x/HTML/index.htm#12330.htm

bulk loading

Comments

Leave a Comment