KAFKA SCHEDULER ERROR

AlokT

We met the following error when using Kafka scheduler in Vertica 9.3.1-12 :

message: Rolling back MB: [Vertica]VJDBC ERROR: Error calling process() in User Function UDFilter at [/data/qb_workspaces/jenkins2/ReleaseBuilds/Grader/REL-9_3_1-x_grader/build/udx/supported/StreamHelpers/StreamFilters.cpp:50], error code: 0, message: User code caused Vertica to throw exception "VIAssert(input.offset == input.size) failed"
java.sql.SQLNonTransientException: [Vertica]VJDBC ERROR: Error calling process() in User Function UDFilter at [/data/qb_workspaces/jenkins2/ReleaseBuilds/Grader/REL-9_3_1-x_grader/build/udx/supported/StreamHelpers/StreamFilters.cpp:50], error code: 0, message: User code caused Vertica to throw exception "VIAssert(input.offset == input.size) failed"
at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.dataengine.VResultSet.fetchChunk(Unknown Source)
at com.vertica.dataengine.VResultSet.initialize(Unknown Source)
at com.vertica.dataengine.VQueryExecutor.readExecuteResponse(Unknown Source)
at com.vertica.dataengine.VQueryExecutor.handleExecuteResponse(Unknown Source)
at com.vertica.dataengine.VQueryExecutor.execute(Unknown Source)
at com.vertica.jdbc.common.SPreparedStatement.executeWithParams(Unknown Source)
at com.vertica.jdbc.common.SPreparedStatement.executeUpdate(Unknown Source)
at com.vertica.solutions.kafka.scheduler.Microbatch.execute(Microbatch.java:350)
at com.vertica.solutions.kafka.scheduler.LaneWorker.run(LaneWorker.java:67)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.vertica.support.exceptions.NonTransientException: [Vertica]VJDBC ERROR: Error calling process() in User Function UDFilter at [/data/qb_workspaces/jenkins2/ReleaseBuilds/Grader/REL-9_3_1-x_grader/build/udx/supported/StreamHelpers/StreamFilters.cpp:50], error code: 0, message: User code caused Vertica to throw exception "VIAssert(input.offset == input.size) failed"
... 11 more

Current kafka scheduler settings:
vkconfig load-spec --create \
--load-spec kafka_load_specs \
--filters " FILTER KafkaInsertDelimiters(delimiter = E'\n') " \
--parser delimited \
--parser-parameters "delimiter '
'" \
--conf kafka.conf

We successfully run kafkacat for load data from topic with command below
kafkacat -C -b broker_host:9092 -t mesint_603_db -o 4387842911 -c 35698 -e

But we see one interesting moment.
After retrive error "error calling process" third kafka_scheduller attempt to load data from topic was successful.

SergeB

The error is a failure to insert the Kafka filter delimiter. Since you're successful in later tests, it's likely data related. If the problem reoccurs capture the COPY command issued generating the error and try to capture (via kafkacat) the offsets that COPY command was trying to load.