VerticaCopyStream appears to be hanging or waiting indefenitly

HPEHPE
in General Discussion

Hi,

 

VerticaCopyStream appears to be hanging or waiting indefenitly in start/execute method calls when using IO stream.

 

In my case I am using PipedIOStream(PipedInputStream & PipedOutputStream) to stream the data from my program to vertica continously. Interestingly, the program is working fine if I close the Out stream; so it looks like Vertica calls waits till the stream to be closed to consume data. I even tried flushing the stream; but not of any help.

 

Regards,

Abhilash R

Comments

  • victor_dileovictor_dileo

    I'll take a shot in the dark here...

     

    Experienced "hanging" behavior when COPYing into Vertica from AWS S3 via a third-party tool called S3FS.

    S3FS tries to make AWS S3 look like a conventional file system, and it succeeds but only to a degree.

     

    When doing huge loads S3FS choked from all the multi-threading, making it look superficially like COPY was hanging. To be clear, the problem was S3FS, not Vertica's COPY.

     

    Perhaps your scenario bears some resemblance to this?

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Format
SpoilerCodeQuote
Heading 2Heading 1
Emoji
:):D:(;):/:o:s:p:'(:|B):#:*<3o:)>:)
Url
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file
Home • General DiscussionComment As ...