If Vertica kafka Utilities support parsing Avro Enum type data?

In my avro schema, there are two enum type fields defined, like below.


"fields" : [ {"name" : "eventType", "type": {"name" : "EventType", "type": "enum", "symbols" : ["A", "C"]}}, {"name": "packetType", "type": {"name": "PacketType", "type": "enum", "symbols" : ["IIPMS", "SMS"]}}]


I use a kafka schedular to parse messages from kafka with KafkaAVROParser, but the values can't be persisted into db table.


Untitled picture.png


I want to know if vertica kafka utility support the Avro enum type data parsing and persistence. If no, I will change the enum type to string. If yes, why the data value can't be persisted correctly, is it because I miss some important configurations?


  • Options



    Yes, KafkaAvroParser supports Avro Enum type.

    How are you writing the messages? see: https://community.dev.hpe.com/t5/Vertica-Forum/Loading-AVRO-data-using-KAFKA-Parser/m-p/235240

    Does your schema contains siteID and alarmString so those values were also loaded using KafkaAvroParser?


    E.g. Enum type is loaded using KafkaAvroParser with the following Avro schema (it is an Avro Record containing String fields and also the Enum fileds)


    {"type":"record","name":"Users","namespace":"test","fields":[{"name":"username","type":"string"},{"name":"age","type":"int"},{"name":"address","type":"string"},{"name" : "eventType", "type": {"name" : "EventType", "type": "enum", "symbols" : ["A", "C"]}}, {"name": "packetType", "type": {"name": "PacketType", "type": "enum", "symbols" : ["IIPMS", "SMS"]}}],"doc":"Record with enums"}


    So target FlexTable contains

    SELECT maptostring(__raw__) FROM myTargetTable;
    "EventType" : "A",
    "PacketType" : "SMS",
    "__name__" : "Users",
    "address" : "Street 1 #54",
    "age" : "30",
    "username" : "John"

    (1 row)

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file