how to use KafkaJSONParser in scheduler?

Moonvert

I am trying to send messages from Kafka to Vertica flex table using KafkaJSONParser. 

However JSON formatted message does not load into table as JSON format.

 

This is my vkconfig topic command using KafkaJSONParser:

./vkconfig topic --edit --conf config.properties --target flextest1.kafka --rejection-table flextest1.kafka_rej --config-schema flextest1 --topic flextest1 --parser KafkaJSONParser

 

When I editted vkconfig topic, it is not updated in the table:

kafka=> select * from flextest1.kafka_targets;
target_schema | target_table | enabled | parser | rejection_table | parser_parameters | load_method | target_columns
---------------+--------------+---------+-------------+---------------------+-------------------+-------------+----------------
flextest1 | kafka | t | KafkaParser | flextest1.kafka_rej | | TRICKLE |
(1 row)

 

I even tried to change parser value manually to KafkaJSONParser, I am unable to parse as JSON formatted.

 

JSON formatted message I tried to send from Kafka:

{"mac": "f0:cb:d1:5b:b8:0f","ipAddress": "10.255.220.12"}

 

The message goes to __raw__ column as it is, but not JSON formatted.

 

 

SruthiA

Hi,

 

    Did you launch the scheduler to load data in JSON format? What is the command you used to lauch? Can you try executing it using entire path as specified in the documentation

 

 

/opt/vertica/packages/kafka/bin/vkconfig topic --edit --target flextest1.kafka --parser KafkaJSONParser

Now query your target table to see if it has changed

 

 

Sruthi

Moonvert

Thank you Sruthi,

It worked as charm when running vkconfig via full path.