Consuming data from kafka does not include key

I am able to consume messages from a kafka topic and store them in a flex table using a COPY from KafkaSource statement. This however does not get the key set by kafka for each message.
The messages have a simple key i.e.: i-01234567 and the message itself is a simple json with one value: { "loginAt": "2020-08-28T23:59:59Z"}
Is there a way to also get the key?


Best Answer

  • Options
    SergeBSergeB - Select Field - Employee
    Answer ✓

    Unfortunately, there is currently no mechanism to retrieve the Kafka message keys unless they're part of the message payload e.g { {"kafka_key":"i-01234567 ","loginAt": "2020-08-28T23:59:59Z"}


  • Options
    LauriPessiLauriPessi Vertica Customer

    Is this something that's going to change in the (near) future?

    It would be very beneficial to get the keys from messages - along with coordinates (partition, offset)

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file