COPY from STORAGE LOCATION works but S3EXPORT doesn't

Poslanik

Hello,

Vertica Analytic Database v9.3.0-0 nodes are on AWS EC2 instances with AmazonS3FullAccess policy.
On Vertica there is a storage location created pointing to an S3 bucket on the same AWS account.

Without setting AWS_SET_CONFIG for aws_id and aws_secret I am able to COPY from storage location, but when I want to do S3EXPORT to the same storage location I get an error:

SQL Error [5861] [VP001]: [Vertica]VJDBC ERROR: Error calling processPartition() in User Function s3export at [/data/qb_workspaces/jenkins2/ReleaseBuilds/Grader/REL-9_3_x-x_grader/build/udx/supported/AWS/src/S3.cpp:780], error code: 0, message: Unknown exception: Anonymous users cannot initiate multipart uploads. Please authenticate.

After I set AWS_SET_CONFIG aws_id and aws_secret of an AWS user with the same policy as Vertica EC2 instances, S3EXPORT finishes successfully.

Am I missing something? How come S3EXPORT function doesn't work without setting aws_id and aws_secret and COPY function works?

SruthiA

@Poslanik What you are seeing is an expected behaviour. S3EXPORT and copy from S3 are different functions... They use different configuration parameters as well. aws_id and aws_secret are to be set explicitly for S3EXPORT to work.

Jim_Knicely

Per @SruthiA's comment...

Setup ...

S3EXPORT - AWS Library S3-Compatible User-Defined Session Parameters

Poslanik

Thank you for your replies.
So I guess the only option to use S3EXPORT without exposing aws_secret is to use Vertica keychain as described in Configure Session Parameters Using Credentials Stored in a Table
Any other suggestions?

SruthiA

@Poslanik : Yes. storing the credentials in keychain table is the only option.