Vertica: Cannot allocate sufficient memory for copy statement

PavanVejjuPavanVejju Vertica Customer

While using copy command in vertica, getting below error.
ERROR : Exception while merge- [Vertica]VJDBC ERROR: Cannot allocate sufficient memory for COPY statement (536870912 requested, 268435456 permitted)

Answers

  • SruthiASruthiA Administrator

    it looks like your resource pool where query is getting executed is not having enough resources. please allocate more resources. please review the below link which can help you in fine tuning the resource pools.

    https://www.vertica.com/kb/BestPracticesforManagingResourcePools/Content/BestPractices/BestPracticesforManagingResourcePools.htm

  • PavanVejjuPavanVejju Vertica Customer

    Thank you for replying
    These are the details of my pool
    dbadmin=> SELECT name, memorysize, maxmemorysize, priority, maxconcurrency FROM V_CATALOG.RESOURCE_POOLS;
    name | memorysize | maxmemorysize | priority | maxconcurrency
    ----------+------------+---------------+----------+----------------
    general | | Special: 95% | 0 |
    sysquery | 1G | | 110 |
    sysdata | 0M | 0M | |
    wosdata | 0% | 2G | |
    tm | 4G | | 105 | 7
    refresh | 0% | | -10 |
    recovery | 0% | | 107 | 7
    dbd | 0% | | 0 |
    jvm | 0% | 2G | 0 |
    blobdata | 0% | 10% | |
    metadata | 0% | | |
    (11 rows)

    Can you please let me know where exactly need to update my configurations

  • SruthiASruthiA Administrator

    can you please share me the output of the following

    select * from host_resources;

  • PavanVejjuPavanVejju Vertica Customer

    select * from host_resources;
    host_name | open_files_limit | threads_limit | core_file_limit_max_size_bytes | processor_count | processor_core_count | processor_description | opened_file_count | opened_socket_count | opened_nonfile_nonsocket_count | total_memory_bytes | total_memory_free_bytes | total_buffer_memory_bytes | total_memory_cache_bytes | total_swap_memory_bytes | total_swap_memory_free_bytes | disk_space_free_mb | disk_space_used_mb | disk_space_total_mb | system_open_files | system_max_files
    ----------------+------------------+---------------+--------------------------------+-----------------+----------------------+----------------------------------------+-------------------+---------------------+--------------------------------+--------------------+-------------------------+---------------------------+--------------------------+-------------------------+------------------------------+--------------------+--------------------+---------------------+-------------------+------------------
    98.218.112.345 | 65536 | 256876 | 0 | 1 | 12 | Intel(R) Xeon(R) E-2176G CPU @ 3.70GHz | 7 | 6 | 6 | 67401281536 | 1682550784 | 145334272 | 60176822272 | 34325131264 | 34218962944 | 588732 | 278673 | 867405 | 1120 | 6560404
    (1 row)

  • Jim_KnicelyJim_Knicely Administrator

    @PavanVejju: Can you post the results of this query?

    SELECT memory_size_kb, memory_size_actual_kb, memory_inuse_kb, queueing_threshold_kb, max_memory_size_kb, planned_concurrency, max_concurrency, query_budget_kb FROM resource_pool_status WHERE pool_name = 'general';

  • PavanVejjuPavanVejju Vertica Customer

    dbadmin=> SELECT memory_size_kb, memory_size_actual_kb, memory_inuse_kb, queueing_threshold_kb, max_memory_size_kb, planned_concurrency, max_concurrency, query_budget_kb FROM resource_pool_status WHERE pool_name = 'general';
    memory_size_kb | memory_size_actual_kb | memory_inuse_kb | queueing_threshold_kb | max_memory_size_kb | planned_concurrency | max_concurrency | query_budget_kb
    ----------------+-----------------------+-----------------+-----------------------+--------------------+---------------------+-----------------+-----------------
    55219873 | 55219873 | 0 | 52458876 | 55219873 | 12 | | 4371573
    (1 row)

  • PavanVejjuPavanVejju Vertica Customer

    Can anyone help me the same, I am facing this issue randomly

  • SruthiASruthiA Administrator

    @PavanVejju : Please open a support case and provide a scrutinize for review. so that we can analyze logs and provide recommendations accordingly.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file