CREATE LIBRARY got timeout

BHIFPBHIFP Vertica Customer

dbadmin=> CREATE LIBRARY JavaTransformFunctions AS :libTfile LANGUAGE 'JAVA';

ROLLBACK 5924: Insufficient resources to get resource from jvm pool [Timedout waiting for resource request: General cannot supply required overflow resources: Memory(KB) Exceeded: Requested = 6225920, Free = 4209727 (Limit = 103621008, Used = 99411281) (queueing threshold)]

While the machines seem healthy with enough memory remaining, although nothing really are running at the moment, so I am not sure what is using 65G per node:

Any idea why? Thanks

Tagged:

Answers

  • SruthiASruthiA Administrator
    edited September 2021

    it looks like in JVM pool already used up 99411281KB and create library is requesting about 6 GB. run the below query and check MEMORY_INUSE_KB for jvm pool. Is 99 GB really being used by the pool or not?

    select * from resource_pool_status

  • BHIFPBHIFP Vertica Customer

    what is the best mean to show you that, SruthiA?

    I have run SELECT RELEASE_JVM_MEMORY(); nothing really happened.

  • SruthiASruthiA Administrator

    Can you set plannedconcurrency to 1 for jvm pool and try create library once?

  • BHIFPBHIFP Vertica Customer
    edited September 2021

    what do you think the first 3 are?

  • SruthiASruthiA Administrator

    General pool used up almost all the memory available on 2 nodes. JVM pool has to borrow from general pool since it's memory size value is set to 0. General is not able to provide the resources requested to the jvm pool for CREATE LIBRARY Stmt

  • BHIFPBHIFP Vertica Customer

    same:

  • BHIFPBHIFP Vertica Customer

    How do I find what is using memory? In fact, I am not aware of anything running on those nodes.

  • BHIFPBHIFP Vertica Customer

    "General pool used up almost all the memory available on 2 nodes."

    htop shows less that half of all memories are used, how should I mobilize the remaining?

  • SruthiASruthiA Administrator

    run the below query to see the current queries in execution and memory acquired by them.

    select * from query_requests where is_executing;

  • BHIFPBHIFP Vertica Customer

    Thanks, SruthiA! That helped.

    There was a long running query.

    That aside, how should one utilize the remaining memory fully?

  • BHIFPBHIFP Vertica Customer

    the original statement still not running even after the kill:

    nothing is running now:

  • BHIFPBHIFP Vertica Customer

    Once I set plannedconcurrency back to AUTO from 1, it started working.
    Thanks!

  • SruthiASruthiA Administrator

    @BHIFP : good to know it is working

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file