Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!

Loading Timestamp data from Kafka Topic

edited September 2017 in General Discussion

I am using the Vertica Kafka Integration tools to capture messages from a topic into a target table.

Now my target table stores time stamp data as INT fields, and this works fine since my messages also represent time stamps as INT (number of microseconds since unix epoch).

However, I would like to change my target table so that the time stamp field is a TIMESTAMPTZ data type. But I presume this would mean I need to tweak the COPY statement that the vertica-kafka executes to do the necessary transform.

Example (mimicked from https://my.vertica.com/docs/6.1.x/HTML/index.htm#12431.htm:

COPY t(MyTimestamp as TO_TIMESTAMP(MyTimestampInUnixEpocMicros)) FROM '/media/data/hour.csv' ABORT ON ERROR;

where

  • MyTimestamp is the TIMESTAMPTZ field in my target table
  • MyTimestampInUnixEpochMicros is the INT field in the message on the kafka topic.

How can I accomplish this?

I saw this article, but it wasn't so clear to me.

Thanks!

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file

Can't find what you're looking for? Search the Vertica Documentation, Knowledge Base, or Blog for more information.