If Vertica kafka Utilities support parsing Avro Enum type data?

Jonna

In my avro schema, there are two enum type fields defined, like below.

 

"fields" : [ {"name" : "eventType", "type": {"name" : "EventType", "type": "enum", "symbols" : ["A", "C"]}}, {"name": "packetType", "type": {"name": "PacketType", "type": "enum", "symbols" : ["IIPMS", "SMS"]}}]

 

I use a kafka schedular to parse messages from kafka with KafkaAVROParser, but the values can't be persisted into db table.

 

 

I want to know if vertica kafka utility support the Avro enum type data parsing and persistence. If no, I will change the enum type to string. If yes, why the data value can't be persisted correctly, is it because I miss some important configurations?

Cornelio

Hi

 

Yes, KafkaAvroParser supports Avro Enum type.

How are you writing the messages? see: https://community.dev.hpe.com/t5/Vertica-Forum/Loading-AVRO-data-using-KAFKA-Parser/m-p/235240

Does your schema contains siteID and alarmString so those values were also loaded using KafkaAvroParser?

 

E.g. Enum type is loaded using KafkaAvroParser with the following Avro schema (it is an Avro Record containing String fields and also the Enum fileds)

 

{"type":"record","name":"Users","namespace":"test","fields":[{"name":"username","type":"string"},{"name":"age","type":"int"},{"name":"address","type":"string"},{"name" : "eventType", "type": {"name" : "EventType", "type": "enum", "symbols" : ["A", "C"]}}, {"name": "packetType", "type": {"name": "PacketType", "type": "enum", "symbols" : ["IIPMS", "SMS"]}}],"doc":"Record with enums"}

 

So target FlexTable contains

SELECT maptostring(__raw__) FROM myTargetTable;
maptostring
-----------------------------------------------------------------------------------------------------------------------------------------------------
{
"EventType" : "A",
"PacketType" : "SMS",
"__name__" : "Users",
"address" : "Street 1 #54",
"age" : "30",
"username" : "John"
}

(1 row)