Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!

Understanding memory usage for data load using COPY

Hi Team,
I'm trying to load a table using COPY DIRECT against a vertica 8.1.1. I'd like to understand further why this query is taking about 2.5GB of memory in total?
Besides, I'd like to know whether (and if so, how much) memory is borrowed from general resource pool.
I'm using a user-defined resource pool which has a memory size of about 2GB (this is for testing purpose) with planned concurrency being 2 so that i get 1 GB as query budget. I'd like to understand what happens in the process of getting 2.5 GB (or whatever it requires as the total memory) and how to track these using system tables/vertica logs?

test_user=> profile copy public.test (about fifty columns) from local '/tmp/filename.csv' GZIP NATIVE DIRECT;
NOTICE 4788: Statement is being profiled
HINT: Select * from v_monitor.execution_engine_profiles where transaction_id=67553994421244784 and statement_id=2;
NOTICE 3557: Initiator memory for query: [on pool test_pool: 2693964 KB, minimum: 2693964 KB]
NOTICE 5077: Total memory required by query: [2693964 KB]

Rows Loaded

  120206

(1 row)
Thanks,
Karthik

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file

Can't find what you're looking for? Search the Vertica Documentation, Knowledge Base, or Blog for more information.