We're Moving!

The Vertica Forum is moving to a new OpenText Analytics Database (Vertica) Community.

Join us there to post discussion topics, learn about

product releases, share tips, access the blog, and much more.

Create My New Community Account Now


How to copy data from hive table (with partitions) to vertica table? — Vertica Forum

How to copy data from hive table (with partitions) to vertica table?

I have data in a table in hive which has a single partition (date string). I am using the COPY command in vsql to move the data from there (with webhdfs URL to hive table) to a table in vertica. I am faced with the two scenarios -
1. Vertica table has no partition column (so one column less than hive table) - When I issue COPY command, the data gets copied without issues.
2. Vertica table is also partitioned with date string column (hive and vertica have same number of columns now) - When I try to COPY in this case, all the records get rejected and copy fails. Here is the error that I get - ERROR 2006: COPY: 10000 records have been rejected

The exact COPY command that I am issuing goes like this -
COPY vertica_table SOURCE Hdfs(url='http://url:50070/webhdfs/v1/user/user/warehouse/hive_db.db/hive_table/date=20170525/*') DELIMITER E'\001' NULL AS '' NO ESCAPE REJECTMAX 10000 DIRECT;

Can someone please help on how to get the data copied with partitions?
Thanks!

Answers

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file