Does Vertica 9.3.1 support connectivity to a Hadoop Cluster which is Knox enabled?
I am trying to establish connectivity between Vertica and a Hadoop cluster which is Knox enabled. As per the documentation have placed core-site.xml and hdfs-site.xml in a location and it shows up while executing VERIFY_HADOOP_CONF_DIR.
While trying to execute other Hadoop function like HDFS_CLUSTER_CONFIG_CHECK(), EXTERNAL_CONFIG_CHECK etc. it fails.
Tried parquet export - select EXPORT TO PARQUET(directory='hdfs:///data/dir_name') .
It fails with below error.
dbadmin=> EXPORT TO PARQUET(directory='hdfs:///data/dir_name/parquet-test')
dbadmin-> AS select dataflow_id, dataflow_load_date from schemaname.tabname limit 5;
NOTICE 8194: HDFS cluster [hdfs://xxxx/] has wire encryption enabled. Falling back to swebhdfs
HINT: You can continue to use 'hdfs://' in your queries, but they are actually using 'swebhdfs://'
ERROR 8198: Unable to verify if directory [hdfs:///data/dir_name/parquet-test/] exists due to 'Error listing directory [hdfs:///data/dir_name/parquet-test] [https://xxx.xxxx.net:xxxx/webhdfs/v1/data/dir_name/parquet-test?op=LISTSTATUS&user.name=dbadmin]: Curl Error: Couldn't connect to server
Error Details: Failed to connect to xx-xxxx-xxx.xxxx.net port xxxx: Connection refused
[https://xxxxx-xxxxxxx.xxxx.net:xxxx/webhdfs/v1/data/dir_name/parquet-test?op=LISTSTATUS&user.name=dbadmin]: Curl Error: Couldn't connect to server
Error Details: Failed to connect to xxxxx-xxxxxxx.xxxx.net port xxxxx: Connection refused
The URL formed to do the export is actually using namenode address not knox address due to which it is failing. As knox is the entry point, URL should be using knox address instead of namenode address directly.
Didn't find any documentation around if anything specific to be done in a knox enabled system.
Does someone has any idea how to get around this?
Thanks in advance.