For the Kafka job scheduler, caught exception from VDBLogger, schema "kafka_config" does not exist
When using v7.2.1, I can use --config-schema and --operator options to configure and launch Kafka job scheduler, so that the Kafka job scheduler can use the same schema and DB user as our application.
But after upgrading to v7.2.2-1, I can still use --config-schema and --operator options to configure Kafka job scheduler, but when launching the scheduler, I got the following error log:
2016-05-21 00:56:18.058 com.vertica.solutions.kafka.logger.VDBLogger::VDBLogger Thread [ERROR] Caught exception from VDBLogger, not writing logs to database. Closing connection, will try to recreate connection next time.
java.sql.SQLException: [Vertica][VJDBC](4650) ERROR: Schema "kafka_config" does not exist
at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown Source)
at com.vertica.solutions.kafka.util.CountedConnection.prepareStatement(CountedConnection.java:69)
at com.vertica.solutions.kafka.logger.VDBLogger$VDBLogRunner.run(VDBLogger.java:121)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.vertica.support.exceptions.ErrorException: [Vertica][VJDBC](4650) ERROR: Schema "kafka_config" does not exist
... 13 more
I think if --config-schema is explicitly provided, the scheduler shouldn't depend on the schema "kafka_config" again.
Even after I created an empty "kafka_config" schema, I still got another error log:
2016-05-21 01:06:38.937 com.vertica.solutions.kafka.logger.VDBLogger::VDBLogger Thread [ERROR] Caught exception from VDBLogger, not writing logs to database. Closing connection, will try to recreate connection next time.
java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](4883) ERROR: Table "kafka_config.kafka_events" does not exist
at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown Source)
at com.vertica.solutions.kafka.util.CountedConnection.prepareStatement(CountedConnection.java:69)
at com.vertica.solutions.kafka.logger.VDBLogger$VDBLogRunner.run(VDBLogger.java:121)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.vertica.support.exceptions.SyntaxErrorException: [Vertica][VJDBC](4883) ERROR: Table "kafka_config.kafka_events" does not exist
... 13 more
But the kafka_events table exists in the my specified config schema. Why it still requires kafka_config.kafka_events table?
Is this a bug? Is there a workaround? Thanks in advance!
Comments
Does anyone have suggestion for this problem?
The good news are that it seems the Kafka job scheduler can actually consume messages even though there're error messages in the log file. It seems that logs will always be writen to kafka_config.kafka_events table no matter whether --config-schema is presented.
How did you solve this problem?
I'm running into the same issue.
thanks
Anyone resolve this problem?
So many Vertica questions are unanswered here. I'm beginning to think HP abandoned this forum. Just use StackOverflow I guess.