Processing Arrays with FjsonParser/KafkaJsonParser
Thank you all in advance just for reading this and any help, idea or recommendation will be highly appreciated.
Im trying to ingest data using vertica copy commands with KafkaSource. (https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/KafkaIntegrationGuide/KafkaFunctions/KafkaSource.htm?zoom_highlight=kafkasource)
My question is what is the most optimal way of processing arrays within kafka message. I am mostly interested to save data into regular table (not flex). I've tried to set VARCHAR data type but Im mostly recieving weird characters 'SOH' (Start of header ?), 'NUL' around real values. Then I got recommendation to use directly ARRAY[VARCHAR(x)] data type, but once data are saved, I can see lots of unicode codes within values.
My question is, whether there is a way how to save data properly, and possibly use EXPLODE function to multiply array elements to rows and process data further ?
Im not interested in flex tables and option to 'flatten_arrays=true' since it is now feasible from the point of design.