Copy From webhdfs using Knox

jhun1321

I am trying to use knox server for Copy ...From webhdfs but getting the error message. My code below

copy 'webhdfs://knox.dev.xxx.com:8443/gateway/default/webhdfs/v1//hdfsfile' on ANY NODE ORC DIRECT;

Error Message :

ERROR 6776: Failed to glob [webhdfs://knox.dev.xxx.com:8443/gateway] because of error: [http://knox.dev.xxx.com:8443/webhdfs/v1/gateway/default/webhdfs/v1//hdfsfile'/?op=GETDELEGATIONTOKEN]: Curl Error: Server returned nothing (no headers, no data)
Error Details: Empty reply from server

I noticed that webhdfs is defaulting to "http" instead of "https" . Please advise how I can make this copy work.

Any help will be appreciated.

jhun1321

Jim_Knicely

What version of Vertica are you using?

Aren't you missing the target Vertica table in the COPY command?

You might need something like this:

copy table_name from 'webhdfs://knox.dev.xxx.com:8443/gateway/default/webhdfs/v1//hdfsfile' ON ANY NODE ORC DIRECT;

Is hdfsfile a directory? If so I think you need to use ... /hdfsfile/*

Have you tried using the hdfs scheme instead of the webhdfs scheme?

See: https://my.vertica.com/docs/8.1.x/HTML/index.htm#Authoring/HadoopIntegrationGuide/libhdfs/ReadingDirectlyFromHDFS.htm