Error from JDBC - COPY: Wrong Header size
Hi,
I have stream of errors in Java app, using JDBC v 12.0.4, connecting to database v 12.0.4:
[Vertica]VJDBC ERROR: COPY: Wrong Header size 497. Expected 501
Is it a bug in JDBC driver, or it is something in app?
JDBC is not exposing API for forming header in data loading. It should be some internal JDBC bug?
Thank you
Sergey
0
Answers
Morning,
Looking at vertica log:
2023-07-06 08:37:26.039 Init Session:0x7ff4ef7db700-a0000000632a5a @v_xxx_node0001: 22V04/2767: COPY: Wrong Header size 501. Expected 505
LOCATION: parseHeader, /data/jenkins/workspace/RE-ReleaseBuilds/RE-Knuckleboom_2/server/vertica/EE/UDL/ParseFormats.h:729
My personal opinion is that data for loading are not formed correctly. Parser is not reporting error properly.
Would be very nice if developer from client driver team comment, what is the source of this error. How to find what is wrong with data.
Thank you
Sergey
Hi, I've asked client driver team to investigate. However, I know that JDBC converts certain batch loads (preparedStatement, or SQL COPY) into something like a COPY LOCAL. It would help if you could share the syntax of the command (COPY? PreparedStatement? something else?) that causes this and any sample data - particularly if there are UTF-8 or Unicode strings, or NUMERIC/DECIMAL (e.g. BigInteger, BigDecimal) types in the Java. Open a support case if necessary, or just email me if you don't want to post here.
Hi,
Thank you very much for looking.
Here is SQL that causes error:
COPY schema.table FROM local STDIN UNCOMPRESSED NATIVE EXCEPTIONS '/tmp/230705BA_0_RejectedReason' REJECTED DATA '/tmp/230705BA_0_RejectedData' AUTO
Table has 125 columns of types Boolean, Char, Integer, Timestamp, Varchar datatypes. No NUMERIC/DECIMAL.
I am very unsure about Unicode strings, that needs investigating.
There is not sample data readily available - app getting data from network, do some internal ETL, and load directly into Vertica. To get sample data, app has to be modified to produce it.
I will look on app code, how it loads - using prepared statement/bulk insert or using stream.
FYI app display interesting behaviour. It works in many instances in parallel without problems. If app throw JDBC error that we are discussing, it never recover and continue to throw same error. App is discarding connection that get error and make new connection, and this does not help - same error show up again. Restarting app - i.e. restarting JVM - fixes problem. Another thing - this JDBC error appear on loading to only one table, loads to many other tables never gets problem.
Sergey
Hi,
It appears, as you can see in COPY command, app implemented Vertica native binary format. And, based on error message, header is not being formed properly in some cases.
Yes, it takes some discussion with other person to find what is wrong. Your hint to look on COPY command helped to find what is going on.
Can you ask developers to clarify error message, make it "COPY: Native Binary format header has wrong size XXX, expected YYY". That will make it clear what is wrong, would save tons of time. Current error message is VERY confusing.
Thank you for your time
Sergey