sheduler does not process new messages

Hello, Could you please help me with the following issue?
I have an installed vertica cluster (3 hosts) and installed apache kafka cluster on 3 other hosts. I have scheduler that reads data from kafka topic. This scheduler worked correctly but stopped processing new messages a few days ago. In vertica log I see the following error:

2020-07-07 13:00:15.087 com.vertica.solutions.kafka.scheduler.StreamCoordinator::Main  [INFO] Starting frame @ 2020-07-07 13:00:15.087
2020-07-07 13:00:28.905 com.vertica.solutions.kafka.scheduler.LaneWorker::Lane Worker 1 (MB:[email protected] 13:00:15.087) [ERROR] Rolling back MB: [Vertica][VJDBC](8361) ERROR: NULL value detected in partition group by expression
java.sql.SQLDataException: [Vertica][VJDBC](8361) ERROR: NULL value detected in partition group by expression
        at com.vertica.util.ServerErrorData.buildException(Unknown Source)
        at com.vertica.dataengine.VResultSet.fetchChunk(Unknown Source)
        at com.vertica.dataengine.VResultSet.initialize(Unknown Source)
        at com.vertica.dataengine.VQueryExecutor.readExecuteResponse(Unknown Source)
        at com.vertica.dataengine.VQueryExecutor.handleExecuteResponse(Unknown Source)
        at com.vertica.dataengine.VQueryExecutor.execute(Unknown Source)
        at com.vertica.jdbc.common.SPreparedStatement.executeWithParams(Unknown Source)
        at com.vertica.jdbc.common.SPreparedStatement.executeUpdate(Unknown Source)
        at com.vertica.solutions.kafka.scheduler.Microbatch.execute(Microbatch.java:350)
        at com.vertica.solutions.kafka.scheduler.LaneWorker.run(LaneWorker.java:67)
        at java.lang.Thread.run(Thread.java:748)

With kafkacat, I read messages correctly (I execute the command from the host where the scheduler is running):
kafkacat -C -b 192.168.0.10:9092 -t mytopic -o -1 -e

{"replyOn":"a112e","source1":{"name":"mysource1","version":"1.0.3"},"source2":{"name":"mysource2","version":"0.8.9.3"}}

Why is that?

Tagged:

Answers

  • deepikaymdeepikaym Employee

    hi , can you provide more details about the issue.
    -What is your vertica version?
    -what is the copy command used?
    -can you share your vertica.log

  • I use vertica 9.3
    I don't use copy command, I use automatically copying Data From Kafka, I configured by documentation
    https://www.vertica.com/docs/9.3.x/HTML/Content/Authoring/KafkaIntegrationGuide/Kafka_Job_Scheduler.htm?TocPath=Integrating with Apache Kafka|Automatically%20Copying%20Data%20From%C2%A0Kafka|_____0

    • can you share your vertica.log

    How to attach a log file? (I can not find how to do this)

  • deepikaymdeepikaym Employee

    you can attach vertica.log in the comment, there is an attach file option if it is too huge trim it at least up to when the error occured

  • The error began to appear after I split the table into partitions using the command:
    ALTER TABLE mytable
    PARTITION BY ((time)::date) GROUP BY (date_trunc('day', (time)::date)) REORGANIZE;

    I tried on a new database (I also split the table into partitions) and have the same error [Vertica][VJDBC](8361) ERROR: NULL value detected in partition group by expression and scheduler does not read / write data to the table
    How to split a table into partitions correctly?

    Here is a piece of the error log:
    2020-07-08 00:00:01.639 com.vertica.solutions.kafka.scheduler.LaneWorker::Lane Worker 1 (MB:[email protected] 23:59:48.209) [ERROR] Rolling back MB: [Vertica]VJDBC ERROR: NULL value detected in partition group by expression
    java.sql.SQLDataException: [Vertica]VJDBC ERROR: NULL value detected in partition group by expression
    at com.vertica.util.ServerErrorData.buildException(Unknown Source)
    at com.vertica.dataengine.VResultSet.fetchChunk(Unknown Source)
    at com.vertica.dataengine.VResultSet.initialize(Unknown Source)
    at com.vertica.dataengine.VQueryExecutor.readExecuteResponse(Unknown Source)
    at com.vertica.dataengine.VQueryExecutor.handleExecuteResponse(Unknown Source)
    at com.vertica.dataengine.VQueryExecutor.execute(Unknown Source)
    at com.vertica.jdbc.common.SPreparedStatement.executeWithParams(Unknown Source)
    at com.vertica.jdbc.common.SPreparedStatement.executeUpdate(Unknown Source)
    at com.vertica.solutions.kafka.scheduler.Microbatch.execute(Microbatch.java:350)
    at com.vertica.solutions.kafka.scheduler.LaneWorker.run(LaneWorker.java:67)
    at java.lang.Thread.run(Thread.java:748)

  • SergeBSergeB Employee

    Seems like you have some NULL values for your time column.
    You can't partition on NULL see warning when creating table with your partition expression.
    create table zt1 (c0 int, c1 varchar(80), time datetime) PARTITION BY ((time)::date) GROUP BY (date_trunc('day', (time)::date)) ;
    WARNING 9249: Using PARTITION expression that may result in NULL values
    HINT: DML statements which produce NULL values for partition expressions will roll back
    CREATE TABLE
    And here's your 8361 error when inserting a null value for time:
    insert into zt1 values (1,'blah', null);
    ERROR 8361: NULL value detected in partition group by expression

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file

Can't find what you're looking for? Search the Vertica Documentation, Knowledge Base, or Blog for more information.