Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!

Seeing Broken pipe errors after upgrading the Vertica nodes from Linux 6 to 7.

`Caused by: java.sql.SQLRecoverableException: [Vertica]VJDBC IOException while communicating with server: java.net.SocketException: Broken pipe (Write failed).
at com.vertica.io.ProtocolStream.logAndConvertToNetworkException(Unknown Source)
at com.vertica.io.ProtocolStream.flush(Unknown Source)
at com.vertica.core.VConnection.handleError(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.jdbc.common.SStatement.executeNoParams(Unknown Source)
at com.vertica.jdbc.common.SStatement.executeUpdate(Unknown Source)
at com.xx.management.bigdata.common.vertica.VerticaTableSQL.finalizeTable(VerticaTableSQL.java:430)
at com.xx.management.bigdata.common.vertica.VerticaTableSQL.finalizeTable(VerticaTableSQL.java:379)
at com.xx.management.bigdata.common.vertica.VerticaTableSQL.finalizeTables(VerticaTableSQL.java:328)
at com.xx.management.wolfe.hadoop.crunch.job.CrunchJob.finalize(CrunchJob.java:436)
at com.xx.management.wolfe.hadoop.crunch.job.CrunchJob.execute(CrunchJob.java:390)
at com.xx.management.wolfe.hadoop.crunch.job.CrunchJob.executeJob(CrunchJob.java:220)
at com.xx.management.vertica.job.ESMJobManager.executeJob(ESMJobManager.java:88)
... 9 more
Caused by: com.vertica.support.exceptions.RecoverableException: [Vertica]VJDBC IOException while communicating with server: java.net.SocketException: Broken pipe (Write failed).
... 24 more
Caused by: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at com.vertica.io.VStream.flush(Unknown Source)
... 23 more

Build step 'Execute shell' marked build as failure
Finished: FAILURE`

Tagged:

Answers

  • Bryan_HBryan_H Administrator

    What specifically causes the exception? That is, was it on connect, or a query, or something else like a prepared statement?
    Do other clients (JDBC, vsql, etc.) still work from the same environment?
    Was the JDK upgraded also during the OS upgrade? Did any OS settings change, such as firewall or iptables?

  • We have hadoop jobs Inserting data to Vertica. If the job runs over 1 hour or more it fails with above error.

    No JDK is installed on the Vertica nodes.

    No firewalls/ACL's present on the Vertica cluster.

  • SruthiASruthiA Employee

    is it copy from HDFS which is failing?

  • @SruthiA said:
    is it copy from HDFS which is failing?

    Yes. HDFS data is inserted to Vertica.

  • how are you connecting to vertica and issuing copy from HDFS queries? is it via JDBC? If so, could you please run one long running COPY statement via vsql to see if completes successfully?

  • @SruthiA said:
    how are you connecting to vertica and issuing copy from HDFS queries? is it via JDBC? If so, could you please run one long running COPY statement via vsql to see if completes successfully?

    JDBC being used is vertica-jdbc-8.1.1-18.jar.

  • @SruthiA : Will upgrading the vertica-jdbc to higher version help??

  • @gjabraj issue looks like connection time out based on the errors. please run one long running COPY statement via vsql to see if completes successfully. it will confirm that issue is with long running jobs via JDBC. do you see any errors in vertica log when it fails with error?

  • @SruthiA : Looks like vsql does not have an issue being idle over an hour. Our next plan is to upgrade vertica-jdbc jar to latest. Please let us know otherwise.

  • @gjabraj Thank you for confirming. Please try using the latest JDBC driver and see if it helps. if there is any load balancer between vertica and your client using JDBC driver try check if that is disconnecting the idle sessions

  • edited October 4

    @SruthiA said:
    how are you connecting to vertica and issuing copy from HDFS queries? is it via JDBC? If so, could you please run one long running COPY statement via vsql to see if completes successfully?

    JDBC being used is vertica-jdbc-8.1.1-18.jar

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file

Can't find what you're looking for? Search the Vertica Documentation, Knowledge Base, or Blog for more information.