Accessing external files in an encrypted zone on HDFS using webhdfs

poojan

If you try to open a file in encrypted zone using curl it fails. As you can see, it is complaining that the 'hdfs' user did not have permission to run DECRYPT_EEK (even though I am using 'test_user' user).

curl -L --negotiate -u : "http://namenode1:9870/webhdfs/v1/tmp/encr/test.txt?op=OPEN"
{"RemoteException":{"exception":"AuthorizationException","javaClassName":"org.apache.hadoop.security.authorize.AuthorizationException","message":"User:hdfs not allowed to do 'DECRYPT_EEK' on 'hdp_cluster_key'"}

Solution is to remove hdfs from the hadoop.kms.blacklist.DECRYPT_EEK property from hdfs-site.xml

curl -L --negotiate -u : "http://namenode1:9870/webhdfs/v1/tmp/encr/test.txt?op=OPEN&user.name=test_user"
a|1
b|2

Vertica's webhdfs uses curl lib underneath the hood to access the webhdfs rest API so removing hdfs from blacklist will also enable Vertica to read from encrypted hdfs zone.

Refer
HWX: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_hdfs_admin_tools/content/webhdfs-hdfs-encr.html
Cloudera: https://www.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_sg_kms_security.html