Copying 116 billion row table - Disk usage while in progress
Copied a 116 billion row table between two 8 node clusters running version 5.0 using the copy from direct command. There's one projection which is segmented and it's associated segmented buddy projection with a K-safe value of 1. The total space used on each node for the table is 346GB. While the table was copying over I was monitoring the file system usage on the nodes of the destination cluster and noticed they had increased by around 700GB, it then cleared down and began steadily increasing again. It did this a few times until the table had finished copying whereby data usage dropped to ~340GB. Could you help me understand the fluctuation in disk space used during the copy?