[Vertica][VJDBC](3587) ERROR: Insufficient resources to execute plan on pool general
I'm a new user on HP Vertica and I'm trying to load some data on it.
My server is: 6 cores and 16GB memory (around 11GB free).
When I run my load process I get the following error :
[Vertica][VJDBC](3587) ERROR: Insufficient resources to execute plan on pool general [Timedout waiting for resource request: Request exceeds limits: Memory(KB) Exceeded: Requested = 2142854, Free = 2 (Limit = 12865318, Used = 12865316)]
I thought that the problem was that VERTICA it was trying to use more memory then available, so I reduced MAXMEMORYSIZE of general pool to 10G and also to 5G. But the problem kept:
[Vertica][VJDBC](3587) ERROR: Insufficient resources to execute plan on pool general [Timedout waiting for resource request: Request exceeds limits: Memory(KB) Exceeded: Requested = 1154611, Free = 0 (Limit = 4626636, Used = 4626636)]
Changing the general pool parameters, my load process worked only when I raised planned_concurrency from 6 (AUTO) to 10.
Why do I get a memory error if i have more concurrency threads than planned_concurrency even if memory_inuse_kb is less than max_memory_size_kb?
Kleyson Rios.
My server is: 6 cores and 16GB memory (around 11GB free).
When I run my load process I get the following error :
[Vertica][VJDBC](3587) ERROR: Insufficient resources to execute plan on pool general [Timedout waiting for resource request: Request exceeds limits: Memory(KB) Exceeded: Requested = 2142854, Free = 2 (Limit = 12865318, Used = 12865316)]
I thought that the problem was that VERTICA it was trying to use more memory then available, so I reduced MAXMEMORYSIZE of general pool to 10G and also to 5G. But the problem kept:
[Vertica][VJDBC](3587) ERROR: Insufficient resources to execute plan on pool general [Timedout waiting for resource request: Request exceeds limits: Memory(KB) Exceeded: Requested = 1154611, Free = 0 (Limit = 4626636, Used = 4626636)]
Changing the general pool parameters, my load process worked only when I raised planned_concurrency from 6 (AUTO) to 10.
Why do I get a memory error if i have more concurrency threads than planned_concurrency even if memory_inuse_kb is less than max_memory_size_kb?
Kleyson Rios.
0
Comments
planned_concurrency and use for determinate the target memory allocated to each query in the pool
Memory allocated for query = memorysize/planned_concurrency , check this both parameters values on your pool
But I realized that when I have more queries (RUNNING_QUERY_COUNT) than PLANNEDCONCURRENCY my process stops to work.
If I increase PLANNEDCONCURRENCY for some number bigger than the maximum RUNNING_QUERY_COUNT needed the process works fine.
Why that happens ?
Kleyson Rios.
As i mention above the amount of allocated memory will be based on the formula above , each query get its allocation at the execution plan time (base of the formula) – before real execution , Vertica allocate the memory at the beginning (base of the formula) and adjust it (only up) and the runtime , so your query’s probably not need so much memory (this is the fact) , The only way to overcome it is to have high value for PLANNEDCONCURRENCY or have more RAM on your server .
People posting the same issue with vertica , i see the same issue with vertica execution for queries everywhere. I faced the same issue with mentioned error message in bold above.
select * from resource_pools where name = 'general';
output
45035996273718900 general true Special: 95% AUTO 0 MEDIUM 2 300 AUTO [NULL] [NULL] false
screenshot attached
Can anyone let me know what to do ?