Transfer closed with data remaining to read when using HDFS connector


When copy data from HDFS to vertica using the standard HDFS connector, we encounter unexpected errors below:

COPY store_sales SOURCE Hdfs(url='http://ip-10-137-3-180.ec2.internal:50070/webhdfs/v1/user/root/relational_data/store_sales/*.dat', username='root');
ERROR 0:  Error calling process() in User Function UDSource at [src/Hdfs.cpp:105], error code: 0, message: [transfer closed with 662920330 bytes remaining to read]

What could be the problem? Thanks in advance.



  • Hi Andy,

    At first glance, it sounds like your HDFS server is closing the connection before the transfer is complete.

    Are there any relevant messages in your HDFS log?


Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file