Transfer closed with data remaining to read when using HDFS connector
Hi,
When copy data from HDFS to vertica using the standard HDFS connector, we encounter unexpected errors below:
COPY store_sales SOURCE Hdfs(url='http://ip-10-137-3-180.ec2.internal:50070/webhdfs/v1/user/root/relational_data/store_sales/*.dat', username='root');
ERROR 0: Error calling process() in User Function UDSource at [src/Hdfs.cpp:105], error code: 0, message: [transfer closed with 662920330 bytes remaining to read]
What could be the problem? Thanks in advance.
When copy data from HDFS to vertica using the standard HDFS connector, we encounter unexpected errors below:
COPY store_sales SOURCE Hdfs(url='http://ip-10-137-3-180.ec2.internal:50070/webhdfs/v1/user/root/relational_data/store_sales/*.dat', username='root');
ERROR 0: Error calling process() in User Function UDSource at [src/Hdfs.cpp:105], error code: 0, message: [transfer closed with 662920330 bytes remaining to read]
What could be the problem? Thanks in advance.
0
Comments
At first glance, it sounds like your HDFS server is closing the connection before the transfer is complete.
Are there any relevant messages in your HDFS log?
Adam