Error "Cannot access a closed Stream" when an underlying TCP connection gets silently closed

bysza

When I open a connection to Vertica (not pooled) using the ado.net driver and execute a query, everything works fine.

However, if for any reason, the underlying tcp/ip connection is closed (eg. network issue), and I try to reuse the open Vertica connection, I receive the following error(s):

1. First try to use the connection, eg. by calling the Prepare() method of the dbcommand:
   [0] Int32 offset, Int32 count)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.BufferedStream.Flush()
   at Vertica.Data.Internal.IO.VStream.flush()
   at Vertica.Data.Internal.IO.ProtocolStream.ReadMessage()
   at: at Vertica.Data.Internal.ADO.Net.SCommand.ExecuteReader(CommandBehavior behavior)
   at Vertica.Data.Internal.ADO.Net.SCommand.ExecuteDbDataReader(CommandBehavior behavior)
   at System.Data.Common.DbCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)

2. Second try:
   Cannot access a closed Stream.
   at: at System.IO.BufferedStream.ReadByteSlow()
   at System.IO.BufferedStream.ReadByte()
   at Vertica.Data.Internal.IO.VStream.ReceiveChar()
   at Vertica.Data.Internal.IO.ProtocolStream.ReadMessage()
   at Vertica.Data.Internal.DataEngine.VResultSet.FetchChunk(Boolean clearExistingRowCache)
   at Vertica.Data.Internal.DataEngine.VResultSet.TryFinishReadingResult()
   at Vertica.Data.Internal.Core.VConnection.EnsureNotInLRS(String query)
   at Vertica.Data.Internal.DataEngine.VDataEngine.PrepareImpl(String sqlQuery, Boolean isBatch)
   at Vertica.Data.Internal.DataEngine.VDataEngine.Prepare(String sql)
   at Vertica.Data.Internal.ADO.Net.SCommand.Prepare(Boolean isExecDirect)
   at Vertica.Data.Internal.ADO.Net.SCommand.Prepare()
   at Vertica.Data.VerticaClient.VerticaCommand.Prepare()

It looks like the Vertica ado.net driver is unaware of the network condition, and tries to use the stream, even if it is unusable (eg. closed or broken).

I haven't found any method on VerticaConnection object that would help me work around that. The connection state is Open, I can create a command on it (because no network calls).

Could you help to resolve the issue?

SruthiA

@bysza: Do you have any load balancer in front of vertica?

bysza

No load balancer.
On local dev I have a single node instance in docker, remotely a full installation (still no LB). Same result.

SruthiA

can you share vertica.log and driver logs around the time of error for review?

bysza

I can get that, however, the easiest local repro would be to just open an ado.net connection to Vertica -> kill the underlying TCP connection (on windows use TcpView tool)-> execute the Prepare() method on a command that uses the connection opened at the beginning. You can setup the debug env however you like then.
Would it work for you?