copy s3 ( .csv file Data ) from AWS s3 to vertica table

ishangaurishangaur Registered User

I am unable to copy the data from the .csv file present on AWS s3 to vertica .
I am executing these commands on VSQL first and then copy command as below -- >

ALTER SESSION SET UDPARAMETER FOR awslib aws_region='us-west-2';
 ALTER SESSION SET UDPARAMETER FOR awslib aws_id='xxx';
ALTER SESSION SET UDPARAMETER FOR awslib aws_secret='XXX';

=> copy abc.table SOURCE S3(url='s3://s3bucket/xyz.csv') DELIMITER ',';
ERROR 5861:  Error calling process() in User Function UDSource at [src/S3.cpp:137], error code: 0, message: Client Error: The AWS Access Key Id you provided does not exist in our records.

Please help.

Comments

  • Jim_KnicelyJim_Knicely Employee, Registered User, VerticaExpert
    edited June 13

    Looks like you are attempting to configure it properly. The error "The AWS Access Key Id you provided does not exist in our records" is coming from AWS, not Vertica.

    I think you are using an invalid AWS_ID.

    I can easily duplicate the error...

    dbadmin=> ALTER SESSION SET UDPARAMETER FOR awslib aws_id='... CORRECT ID ...';
    ALTER SESSION
    
    dbadmin=> ALTER SESSION SET UDPARAMETER FOR awslib aws_secret='... CORRECT SECRET ...;
    ALTER SESSION
    
    dbadmin=> ALTER SESSION SET UDPARAMETER FOR awslib aws_region='us-east-2';
    ALTER SESSION
    
    dbadmin=> CREATE FLEX EXTERNAL TABLE ge() AS COPY SOURCE s3(bucket='s3://vertica-mybucket/*.json') PARSER fjsonparser();
    CREATE TABLE
    
    dbadmin=> SELECT name, type, height, hike_safety FROM ge;
          name      |   type   | height | hike_safety
    ----------------+----------+--------+-------------
     Arenal Volcano | volcano  | 5479   | 12.4
     Joshua         | mountain | 236172 | 100
     Everest        | mountain | 29029  | 34.1
     Mt St Helens   | volcano  | 29029  | 15.4
     Denali         | mountain | 17000  | 12.2
     Kilimanjaro    | mountain | 14000  |
     Mt Washington  | mountain |        | 50.6
    (7 rows)
    

    Now I will change to an invalid AWS_ID:

    dbadmin=> ALTER SESSION SET UDPARAMETER FOR awslib aws_id='... INCORRECT ID ...';
    ALTER SESSION
    
    dbadmin=> SELECT name, type, height, hike_safety FROM ge;
    ERROR 5861:  Error calling setup() in User Function UDSource at [/data/build-centos6/qb/buildagent/workspace/jenkins2/ReleaseBuilds/Grader/REL-9_1_0-x_grader/build/udx/supported/AWS/../../../vertica/SAL/S3UDFS/S3Helpers_internal.cpp:894], error code: 5, message: The AWS Access Key Id you provided does not exist in our records.
    

    I got the same error as you!

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file