Export To Parquet Default Directory Permission

victoryhl7

I am using EXPORT TO PARQUET to export my table.

EXPORT TO PARQUET (directory = '/datalake/2018/201810/dim') AS SELECT * FROM dim;

I noticed that the permission for /datalake/2018/201810/dim is set to "drwx--S---" by default. Only dbadmin user can access the files inside the directory.

Is there a way to change the default permission to allow users from the same group to access the underlying parquet files?

Regards,
Victor

Jim_Knicely

Hi,

You can create a "user" storage location (where the usage type = USER)!

See:
https://www.vertica.com/docs/9.1.x/HTML/index.htm#Authoring/SQLReferenceManual/Statements/CreateLocation.htm
https://www.vertica.com/docs/9.1.x/HTML/index.htm#Authoring/AdministratorsGuide/DBUsersAndPrivileges/Privileges/StorageLocationPrivileges.htm

victoryhl7

Hi Jim,

I understand that by creating a "user" storage location, I am able to grant the read permission to other database users.

However, I am trying to share the exported parquet files with other applications outside of Vertica. Is it possible to change the default directory permission at OS level from "drwx--S---" to "drwxrwsr-x"?

Thanks!

Jim_Knicely

Hi,

There is not currently a way to change the "default" directory permissions when exporting to Parquet. There is a feature coming very soon that will let you do that if exporting to HDFS. And there is also a feature request for other file systems to be supported. I'll update the JIRA to let them know about your interest.

For now, maybe you can change the permissions yourself from vsql (if that's how you are exporting)?

Example:

dbadmin=> CREATE TABLe parq_exp (C INT);
CREATE TABLE

dbadmin=> INSERT INTO parq_exp SELECT 1;
 OUTPUT
--------
      1
(1 row)

dbadmin=> EXPORT TO PARQUET (directory = '/home/dbadmin/parq_exp') OVER () AS SELECT * FROM parq_exp;;
 Rows Exported
---------------
             1
(1 row)

dbadmin=> \! ls -ld /home/dbadmin/parq_exp
drwx------ 2 dbadmin verticadba 4096 Oct 23 09:35 /home/dbadmin/parq_exp

dbadmin=> \! ls -lrt /home/dbadmin/parq_exp/*.parquet
-rw------- 1 dbadmin verticadba 310 Oct 23 09:35 /home/dbadmin/parq_exp/a18bc913-v_test_db_node0001-140133481297664-0.parquet

dbadmin=> \! chmod 766 /home/dbadmin/parq_exp/ -R

dbadmin=> \! ls -ld /home/dbadmin/parq_exp
drwxrw-rw- 2 dbadmin verticadba 4096 Oct 23 09:35 /home/dbadmin/parq_exp

dbadmin=> \! ls -lrt /home/dbadmin/parq_exp/*.parquet
-rwxrw-rw- 1 dbadmin verticadba 310 Oct 23 09:35 /home/dbadmin/parq_exp/a18bc913-v_test_db_node0001-140133481297664-0.parquet