Export To Parquet Default Directory Permission

victoryhl7victoryhl7 Registered User
edited October 22 in Vertica Forum

I am using EXPORT TO PARQUET to export my table.

EXPORT TO PARQUET (directory = '/datalake/2018/201810/dim') AS SELECT * FROM dim;

I noticed that the permission for /datalake/2018/201810/dim is set to "drwx--S---" by default. Only dbadmin user can access the files inside the directory.

Is there a way to change the default permission to allow users from the same group to access the underlying parquet files?

Regards,
Victor

Comments

  • victoryhl7victoryhl7 Registered User

    Hi Jim,

    I understand that by creating a "user" storage location, I am able to grant the read permission to other database users.

    However, I am trying to share the exported parquet files with other applications outside of Vertica. Is it possible to change the default directory permission at OS level from "drwx--S---" to "drwxrwsr-x"?

    Thanks!

  • Jim_KnicelyJim_Knicely Administrator, Moderator, Employee, Registered User, VerticaExpert
    edited October 23

    Hi,

    There is not currently a way to change the "default" directory permissions when exporting to Parquet. There is a feature coming very soon that will let you do that if exporting to HDFS. And there is also a feature request for other file systems to be supported. I'll update the JIRA to let them know about your interest.

    For now, maybe you can change the permissions yourself from vsql (if that's how you are exporting)?

    Example:

    dbadmin=> CREATE TABLe parq_exp (C INT);
    CREATE TABLE
    
    dbadmin=> INSERT INTO parq_exp SELECT 1;
     OUTPUT
    --------
          1
    (1 row)
    
    dbadmin=> EXPORT TO PARQUET (directory = '/home/dbadmin/parq_exp') OVER () AS SELECT * FROM parq_exp;;
     Rows Exported
    ---------------
                 1
    (1 row)
    
    dbadmin=> \! ls -ld /home/dbadmin/parq_exp
    drwx------ 2 dbadmin verticadba 4096 Oct 23 09:35 /home/dbadmin/parq_exp
    
    dbadmin=> \! ls -lrt /home/dbadmin/parq_exp/*.parquet
    -rw------- 1 dbadmin verticadba 310 Oct 23 09:35 /home/dbadmin/parq_exp/a18bc913-v_test_db_node0001-140133481297664-0.parquet
    
    dbadmin=> \! chmod 766 /home/dbadmin/parq_exp/ -R
    
    dbadmin=> \! ls -ld /home/dbadmin/parq_exp
    drwxrw-rw- 2 dbadmin verticadba 4096 Oct 23 09:35 /home/dbadmin/parq_exp
    
    dbadmin=> \! ls -lrt /home/dbadmin/parq_exp/*.parquet
    -rwxrw-rw- 1 dbadmin verticadba 310 Oct 23 09:35 /home/dbadmin/parq_exp/a18bc913-v_test_db_node0001-140133481297664-0.parquet
    

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file