sheduler does not process new messages

aleksss55

Hello, Could you please help me with the following issue?
I have an installed vertica cluster (3 hosts) and installed apache kafka cluster on 3 other hosts. I have scheduler that reads data from kafka topic. This scheduler worked correctly but stopped processing new messages a few days ago. In vertica log I see the following error:

2020-07-07 13:00:15.087 com.vertica.solutions.kafka.scheduler.StreamCoordinator::Main  [INFO] Starting frame @ 2020-07-07 13:00:15.087
2020-07-07 13:00:28.905 com.vertica.solutions.kafka.scheduler.LaneWorker::Lane Worker 1 (MB:750001-FRAME@2020-07-07 13:00:15.087) [ERROR] Rolling back MB: [Vertica][VJDBC](8361) ERROR: NULL value detected in partition group by expression
java.sql.SQLDataException: [Vertica][VJDBC](8361) ERROR: NULL value detected in partition group by expression
        at com.vertica.util.ServerErrorData.buildException(Unknown Source)
        at com.vertica.dataengine.VResultSet.fetchChunk(Unknown Source)
        at com.vertica.dataengine.VResultSet.initialize(Unknown Source)
        at com.vertica.dataengine.VQueryExecutor.readExecuteResponse(Unknown Source)
        at com.vertica.dataengine.VQueryExecutor.handleExecuteResponse(Unknown Source)
        at com.vertica.dataengine.VQueryExecutor.execute(Unknown Source)
        at com.vertica.jdbc.common.SPreparedStatement.executeWithParams(Unknown Source)
        at com.vertica.jdbc.common.SPreparedStatement.executeUpdate(Unknown Source)
        at com.vertica.solutions.kafka.scheduler.Microbatch.execute(Microbatch.java:350)
        at com.vertica.solutions.kafka.scheduler.LaneWorker.run(LaneWorker.java:67)
        at java.lang.Thread.run(Thread.java:748)

With kafkacat, I read messages correctly (I execute the command from the host where the scheduler is running):
kafkacat -C -b 192.168.0.10:9092 -t mytopic -o -1 -e

{"replyOn":"a112e","source1":{"name":"mysource1","version":"1.0.3"},"source2":{"name":"mysource2","version":"0.8.9.3"}}

Why is that?

deepikaym

hi , can you provide more details about the issue.
-What is your vertica version?
-what is the copy command used?
-can you share your vertica.log

aleksss55

I use vertica 9.3
I don't use copy command, I use automatically copying Data From Kafka, I configured by documentation
https://www.vertica.com/docs/9.3.x/HTML/Content/Authoring/KafkaIntegrationGuide/Kafka_Job_Scheduler.htm?TocPath=Integrating with Apache Kafka|Automatically%20Copying%20Data%20From%C2%A0Kafka|_____0

• can you share your vertica.log

How to attach a log file? (I can not find how to do this)

deepikaym

you can attach vertica.log in the comment, there is an attach file option if it is too huge trim it at least up to when the error occured

aleksss55

The error began to appear after I split the table into partitions using the command:
ALTER TABLE mytable
PARTITION BY ((time)::date) GROUP BY (date_trunc('day', (time)::date)) REORGANIZE;

I tried on a new database (I also split the table into partitions) and have the same error [Vertica][VJDBC](8361) ERROR: NULL value detected in partition group by expression and scheduler does not read / write data to the table
How to split a table into partitions correctly?

Here is a piece of the error log:
2020-07-08 00:00:01.639 com.vertica.solutions.kafka.scheduler.LaneWorker::Lane Worker 1 (MB:750001-FRAME@2020-07-07 23:59:48.209) [ERROR] Rolling back MB: [Vertica]VJDBC ERROR: NULL value detected in partition group by expression
java.sql.SQLDataException: [Vertica]VJDBC ERROR: NULL value detected in partition group by expression
at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.dataengine.VResultSet.fetchChunk(Unknown Source)
at com.vertica.dataengine.VResultSet.initialize(Unknown Source)
at com.vertica.dataengine.VQueryExecutor.readExecuteResponse(Unknown Source)
at com.vertica.dataengine.VQueryExecutor.handleExecuteResponse(Unknown Source)
at com.vertica.dataengine.VQueryExecutor.execute(Unknown Source)
at com.vertica.jdbc.common.SPreparedStatement.executeWithParams(Unknown Source)
at com.vertica.jdbc.common.SPreparedStatement.executeUpdate(Unknown Source)
at com.vertica.solutions.kafka.scheduler.Microbatch.execute(Microbatch.java:350)
at com.vertica.solutions.kafka.scheduler.LaneWorker.run(LaneWorker.java:67)
at java.lang.Thread.run(Thread.java:748)

SergeB

Seems like you have some NULL values for your time column.
You can't partition on NULL see warning when creating table with your partition expression.
create table zt1 (c0 int, c1 varchar(80), time datetime) PARTITION BY ((time)::date) GROUP BY (date_trunc('day', (time)::date)) ;
WARNING 9249: Using PARTITION expression that may result in NULL values
HINT: DML statements which produce NULL values for partition expressions will roll back
CREATE TABLE
And here's your 8361 error when inserting a null value for time:
insert into zt1 values (1,'blah', null);
ERROR 8361: NULL value detected in partition group by expression