Trouble using VerticaCopyStream with parser and Java InputStream with STDIN

nomlaran

I'm using VerticaCopyStream to read from an InputStream in Java, through a UDL parser and into a vertica table.

 

The statement that I'm passing to Vertica is generated with this code:

 

String.format("COPY %s FROM STDIN WITH PARSER %s", verticaTableName, verticaParserWithArgs);

I then use VerticaCopyStream with this code:

String copyStatement = String.format("COPY %s FROM STDIN WITH PARSER %s", verticaTableName, verticaParserWithArgs);
InputStream countingInputStream =
        new CountingInputStream(myInputStream);
copyStream = new VerticaCopyStream(verticaConnection, copyStatement);
copyStream.start();
copyStream.addStream(countingInputStream);
copyStream.execute();
copyStream.finish();

The COPY statements don't throw errors. But no rows are inserted either.

 

Looking into the logs, I see that the statement that VerticaCopyStream is actually executing is this:

 

COPY mytable FROM local STDIN WITH PARSER MyParser() returnrejected

From what I understand, "local" is only relevant when reading from a file on the file system. But, again, I'm not super familiar with the ins and outs of VerticaCopyStream.

 

And again, as you can see from the code above, I'm reading from a java buffer/stream, NOT a file.

 

Is anything obviously wrong with what I'm doing? Is VerticaCopyStream able to handle passing input streams to UDL parsers?

 

Thanks

[Deleted User]

Hi nomlaran,

 

The 'local' in the query just means that the data will be coming from the client. It can be used in conjunction with STDIN... it just means that the pipe supplying the data is on the client system, not the STDIN on the HP Vertica node.

 

Is your UDL parser working correctly? Can you use it to execute a query that loads data directly from your client system? I.e.

 

\set rejected ../except_reject/copyLocal.rejected
\set exceptions ../except_reject/copyLocal.exceptions
COPY mytable FROM local '/home/dbadmin/test.txt' with parser MyParser() rejected data :rejected exceptions :exceptions;