Vertica: Cannot allocate sufficient memory for copy statement

PavanVejju

While using copy command in vertica, getting below error.
ERROR : Exception while merge- [Vertica]VJDBC ERROR: Cannot allocate sufficient memory for COPY statement (536870912 requested, 268435456 permitted)

SruthiA

it looks like your resource pool where query is getting executed is not having enough resources. please allocate more resources. please review the below link which can help you in fine tuning the resource pools.

https://www.vertica.com/kb/BestPracticesforManagingResourcePools/Content/BestPractices/BestPracticesforManagingResourcePools.htm

PavanVejju

Thank you for replying
These are the details of my pool
dbadmin=> SELECT name, memorysize, maxmemorysize, priority, maxconcurrency FROM V_CATALOG.RESOURCE_POOLS;
name | memorysize | maxmemorysize | priority | maxconcurrency
----------+------------+---------------+----------+----------------
general | | Special: 95% | 0 |
sysquery | 1G | | 110 |
sysdata | 0M | 0M | |
wosdata | 0% | 2G | |
tm | 4G | | 105 | 7
refresh | 0% | | -10 |
recovery | 0% | | 107 | 7
dbd | 0% | | 0 |
jvm | 0% | 2G | 0 |
blobdata | 0% | 10% | |
metadata | 0% | | |
(11 rows)

Can you please let me know where exactly need to update my configurations

SruthiA

can you please share me the output of the following

select * from host_resources;

PavanVejju

select * from host_resources;
host_name | open_files_limit | threads_limit | core_file_limit_max_size_bytes | processor_count | processor_core_count | processor_description | opened_file_count | opened_socket_count | opened_nonfile_nonsocket_count | total_memory_bytes | total_memory_free_bytes | total_buffer_memory_bytes | total_memory_cache_bytes | total_swap_memory_bytes | total_swap_memory_free_bytes | disk_space_free_mb | disk_space_used_mb | disk_space_total_mb | system_open_files | system_max_files
----------------+------------------+---------------+--------------------------------+-----------------+----------------------+----------------------------------------+-------------------+---------------------+--------------------------------+--------------------+-------------------------+---------------------------+--------------------------+-------------------------+------------------------------+--------------------+--------------------+---------------------+-------------------+------------------
98.218.112.345 | 65536 | 256876 | 0 | 1 | 12 | Intel(R) Xeon(R) E-2176G CPU @ 3.70GHz | 7 | 6 | 6 | 67401281536 | 1682550784 | 145334272 | 60176822272 | 34325131264 | 34218962944 | 588732 | 278673 | 867405 | 1120 | 6560404
(1 row)

Jim_Knicely

@PavanVejju: Can you post the results of this query?

SELECT memory_size_kb, memory_size_actual_kb, memory_inuse_kb, queueing_threshold_kb, max_memory_size_kb, planned_concurrency, max_concurrency, query_budget_kb FROM resource_pool_status WHERE pool_name = 'general';

PavanVejju

dbadmin=> SELECT memory_size_kb, memory_size_actual_kb, memory_inuse_kb, queueing_threshold_kb, max_memory_size_kb, planned_concurrency, max_concurrency, query_budget_kb FROM resource_pool_status WHERE pool_name = 'general';
memory_size_kb | memory_size_actual_kb | memory_inuse_kb | queueing_threshold_kb | max_memory_size_kb | planned_concurrency | max_concurrency | query_budget_kb
----------------+-----------------------+-----------------+-----------------------+--------------------+---------------------+-----------------+-----------------
55219873 | 55219873 | 0 | 52458876 | 55219873 | 12 | | 4371573
(1 row)

PavanVejju

Can anyone help me the same, I am facing this issue randomly

SruthiA

@PavanVejju : Please open a support case and provide a scrutinize for review. so that we can analyze logs and provide recommendations accordingly.