The Vertica Forum recently got a makeover! Let us know what you think by filling out this short, anonymous survey.

Memory Leak in m_DatabaseMetaData member of Connection com.vertica.jdbc.VerticaConnectionImpl

We are using Vertica 6.12 jdbc driver+vertica 6.12. During our performance tests we are facing memory leak in some internal collections held by VerticaConnectionImpl.
We have a managed connection pool with maximum of 30 connections and we monitor it from the vertica side. There is no connection leak.
We noticed the following as the root cause of a memory leak in our tomact server:
  • As the application continues to run the size of the member m_DatabaseMetaData of this class (which is an ArrayList) continue to rise.
  • The elements of this collection are of type com.vertica.jdbc.SDatabaseMetaData
  • The size of the list seems unbounded and can reach  hundred thousands of elements
  • Throwing away the connection and creating a new one  solves the problem

We would like to know:

What is the purpose of these hash tables? Why are they continue to grow?

What can be done on our side to eliminate this? Can we use some connection parameter to disable this?

Can you say if 6.1.3 solves this?

We find closing the connection as a workaround in a connection pool when it is not idle an anti-pattern.


Thanks.

Comments

  • Hi Sharon,

    This array is, I believe, a cache of the results of Connection.getMetadata().  It's used/needed internally for various reasons.  Unfortunately, in currently-released versions, the cache can potentially grow very large with long-lived / reused sessions, if Connection.getMetadata() is called frequently.

    A workaround is to store and reuse the result of that function call, rather than calling it repeatedly.  Though I'm not sure if that's terribly useful for your use case...

    I don't believe this is addressed in 6.1.3.  I can't speak to unreleased changes or fixes; if this is important to you, and if you have an Enterprise license, you are welcome to contact our Support folks (through the my.vertica.com portal) for more information.

    Adam
  • Another workaround, and a common pattern when building stateful pools of any sort, would be to re-use connections for only a certain period of time or a certain number of uses.

    I understand that you don't like the idea of closing connections that are in use.  Also that overgrown caches are problematic.  But it's a common approach that should solve your problem, with no significant outward-facing disadvantage (that I'm aware of?).  It's just not elegant.
  • Hi,
    Is this relevant to ResultSet.getMetaData?
    Will we experience the same issue with many invocations of "ResultSet.getMetaData"?

    kind regards,
     - Jony

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file

Can't find what you're looking for? Search the Vertica Documentation, Knowledge Base, or Blog for more information.