Copy lzo compressed file from hdfs to Vertica using vertica hdfs connector
Can we copy lzo compressed file from hdfs to Vertica using vertica hdfs connector ?
I tried the below but looks like its not able to identify LZO
Copy abc.test Hdfs(url='http://prd101.corp.test.net:40070/webhdfs/v1/vertica/data/lzo/*', username='xyz') LZO DELIMITER ',' NULL AS '' ESCAPE AS E'\001';
If not is there any other way we can copy lzo files from hdfs to vertica?
Regards,
-Amit
I tried the below but looks like its not able to identify LZO
Copy abc.test Hdfs(url='http://prd101.corp.test.net:40070/webhdfs/v1/vertica/data/lzo/*', username='xyz') LZO DELIMITER ',' NULL AS '' ESCAPE AS E'\001';
ERROR 4856: Syntax error at or near "LZO" at character 180
LINE 1: .../vertica/data/lzo/*', username='sys_vertica_sbm') LZO DELIMI...
^
The below page says we can do it using Copy command but not sure if thats possible for Copy with hdfs connector?If not is there any other way we can copy lzo files from hdfs to vertica?
Regards,
-Amit
0
Comments
We do not currently support loading data compressed in LZO on HDFS. A feature request has been submitted to have this feature added to a future release of the product.
Regards
Rahul Choudhary
Hi Amit/Rahul,
Do we have any solution for the same now ? Or any alternative way to do the same.
Thanks,
Soumyaranjan