Transfer closed with data remaining to read when using HDFS connector

Andy_He

Hi,

When copy data from HDFS to vertica using the standard HDFS connector, we encounter unexpected errors below:

COPY store_sales SOURCE Hdfs(url='http://ip-10-137-3-180.ec2.internal:50070/webhdfs/v1/user/root/relational_data/store_sales/*.dat', username='root');
ERROR 0:  Error calling process() in User Function UDSource at [src/Hdfs.cpp:105], error code: 0, message: [transfer closed with 662920330 bytes remaining to read]

What could be the problem? Thanks in advance.

  

[Deleted User]

Hi Andy,

At first glance, it sounds like your HDFS server is closing the connection before the transfer is complete.

Are there any relevant messages in your HDFS log?

Adam