Understanding memory usage for data load using COPY
karthik
Vertica Customer ✭
Hi Team,
I'm trying to load a table using COPY DIRECT against a vertica 8.1.1. I'd like to understand further why this query is taking about 2.5GB of memory in total?
Besides, I'd like to know whether (and if so, how much) memory is borrowed from general resource pool.
I'm using a user-defined resource pool which has a memory size of about 2GB (this is for testing purpose) with planned concurrency being 2 so that i get 1 GB as query budget. I'd like to understand what happens in the process of getting 2.5 GB (or whatever it requires as the total memory) and how to track these using system tables/vertica logs?
test_user=> profile copy public.test (about fifty columns) from local '/tmp/filename.csv' GZIP NATIVE DIRECT;
NOTICE 4788: Statement is being profiled
HINT: Select * from v_monitor.execution_engine_profiles where transaction_id=67553994421244784 and statement_id=2;
NOTICE 3557: Initiator memory for query: [on pool test_pool: 2693964 KB, minimum: 2693964 KB]
NOTICE 5077: Total memory required by query: [2693964 KB]
Rows Loaded
120206
(1 row)
Thanks,
Karthik