Configuring Scheduler for SSL

benoit_frbenoit_fr Registered User

Hi,
I try to integrate data from Kafka into Vertica. Kafka topic is secured so we need to configure SSL on our scheduler.
I follow steps on documentation (https://my.vertica.com/docs/8.1.x/HTML/index.htm#Authoring/KafkaIntegrationGuide/ConfiguringSchedulerforSSL.htm?TocPath=Integrating%20with%20Apache%20Kafka|Using%20SSL%20with%20Kafka|_____3) to create truststore and keystore but when I when to launch my scheduler I have the following :

java.sql.SQLNonTransientException: com.vertica.solutions.kafka.exception.ConfigurationException: No keystore system property found: null
at com.vertica.solutions.kafka.util.SQLUtilities.getConnection(SQLUtilities.java:173)
at com.vertica.solutions.kafka.cli.CLIUtil.assertDBConnectionWorks(CLIUtil.java:40)
at com.vertica.solutions.kafka.Launcher.run(Launcher.java:128)
at com.vertica.solutions.kafka.Launcher.main(Launcher.java:258)
Caused by: com.vertica.solutions.kafka.exception.ConfigurationException: No keystore system property found: null
at com.vertica.solutions.kafka.security.KeyStoreUtil.loadStore(KeyStoreUtil.java:77)
at com.vertica.solutions.kafka.security.KeyStoreUtil.(KeyStoreUtil.java:42)
at com.vertica.solutions.kafka.util.SQLUtilities.getConnection(SQLUtilities.java:171)
... 3 more

Did I miss something ?
Regards,
Benoît

Comments

  • SruthiASruthiA Employee, Registered User, VerticaExpert ✭✭

    Can you share me the exact commands that you used to create a keystore on your scheduler? Can you share me the command you used to launch scheduler?

  • benoit_frbenoit_fr Registered User

    Hi SruthiA,

    • Create a Trust Store and and Key Store on Your Scheduler *

    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.truststore.jks -alias cacert -import -file /tmp/certif/myServer.crt
    (#)/usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.truststore.jks -alias root -import -file "/tmp/certif/root.crt"
    (#)/usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.truststore.jks -alias intermed -import -file "/tmp/certif/servers.crt"

    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias ctlvs011 -validity 365 -keyalg RSA -genkey
    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias ctlvs011 -certreq -file cert-file.tmp
    openssl x509 -req -CA /tmp/certif/myServer.crt -CAkey /tmp/certif/myServer.key -in /data/vertica/cert-file.tmp -out cert-signed.tmp -days 365 -CAcreateserial
    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias cacert -import -file /tmp/certif/myServer.crt
    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias root -trustcacerts -import -file "/tmp/certif/root.crt"
    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias intermed -trustcacerts -import -file "/tmp/certif/servers.crt"
    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias ctlvs011 -import -file /data/vertica/cert-signed.tmp

    • Launch Your Scheduler with SSL Enabled *
      /opt/vertica/packages/kafka/bin/vkconfig launch --enable-SSL true --ssl-ca-alias cacert --ssl-key-alias ctlvs011 --ssl-key-password XXXXX

    I try with and without the 2 lignes with (#)
    Keystore files are created on /data/vertica folder.

    Regards,
    Benoît

  • benoit_frbenoit_fr Registered User

    Hi SruthiA,

    • Create a Trust Store and and Key Store on Your Scheduler *

    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.truststore.jks -alias cacert -import -file /tmp/certif/myServer.crt
    (#)/usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.truststore.jks -alias root -import -file "/tmp/certif/root.crt"
    (#)/usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.truststore.jks -alias intermed -import -file "/tmp/certif/servers.crt"

    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias ctlvs011 -validity 365 -keyalg RSA -genkey
    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias ctlvs011 -certreq -file cert-file.tmp
    openssl x509 -req -CA /tmp/certif/myServer.crt -CAkey /tmp/certif/myServer.key -in /data/vertica/cert-file.tmp -out cert-signed.tmp -days 365 -CAcreateserial
    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias cacert -import -file /tmp/certif/myServer.crt
    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias root -trustcacerts -import -file "/tmp/certif/root.crt"
    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias intermed -trustcacerts -import -file "/tmp/certif/servers.crt"
    /usr/java/jdk1.8.0_66/jre/bin/keytool -keystore cq_config_aa.keystore.jks -alias ctlvs011 -import -file /data/vertica/cert-signed.tmp

    • Launch Your Scheduler with SSL Enabled *
      /opt/vertica/packages/kafka/bin/vkconfig launch --enable-SSL true --ssl-ca-alias cacert --ssl-key-alias ctlvs011 --ssl-key-password XXXXX

    I tried with and without the two lignes with (#).
    Keystore files are created on /data/vertica folder.

    Regards,
    Benoît

  • benoit_frbenoit_fr Registered User

    In addition I set JAVA_TOOL_OPTIONS with -DJVMjavax.net.ssl.trustStore, -DJVMjavax.net.ssl.keyStore and -DJVMjavax.net.ssl.keyStorePassword but still having the same error

  • benoit_frbenoit_fr Registered User

    Corrects args are -Djavax.net.ssl.trustStore, -Djavax.net.ssl.keyStore and -Djavax.net.ssl.keyStorePassword

  • SruthiASruthiA Employee, Registered User, VerticaExpert ✭✭

    Can you share me kafka scheduler logs in private message. I will take a look.

  • benoit_frbenoit_fr Registered User

    Hi SruthiA,

    I managed to configure my scheduler for SSL.
    First issue was JVM args that are wrong on Vertica documentation (-Djavax.net.ssl.trustStore, -Djavax.net.ssl.keyStore and -Djavax.net.ssl.keyStorePassword expected).
    Second, my launch command should specified config-schema and password options.

    But we still have an issue, my scheduler was created on cq_config_aa config-schema and this error appears in vkafka-sched.log

    2017-05-02 09:17:48.904 com.vertica.solutions.kafka.security.KeyStoreUtil::VDBLogger Thread [INFO] Loading KeyStores as store type: jks
    2017-05-02 09:17:48.941 com.vertica.solutions.kafka.logger.VDBLogger::VDBLogger Thread [ERROR] Caught exception from VDBLogger, not writing logs to database. Closing connection, will try to recreate connection next time.
    java.sql.SQLException: [Vertica]VJDBC ERROR: Schema "stream_config" does not exist
    at com.vertica.util.ServerErrorData.buildException(Unknown Source)
    at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
    at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
    at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
    at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
    at com.vertica.jdbc.common.SPreparedStatement.(Unknown Source)
    at com.vertica.jdbc.jdbc4.S4PreparedStatement.(Unknown Source)
    at com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.(Unknown Source)
    at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
    at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown Source)
    at com.vertica.solutions.kafka.util.CountedConnection.prepareStatement(CountedConnection.java:69)
    at com.vertica.solutions.kafka.logger.VDBLogger$VDBLogRunner.run(VDBLogger.java:121)
    at java.lang.Thread.run(Thread.java:745)
    Caused by: com.vertica.support.exceptions.ErrorException: [Vertica]VJDBC ERROR: Schema "stream_config" does not exist
    ... 13 more

    To avoid this, I create my scheduler in stream_config schema and it works, I receive my data in my target table.
    Is there a real issue using SSL with scheduler or is there a parameter that I miss about schema for keyStore/VDBLogger ?

    Regards,
    Benoît

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file