Vertica load procedure

Kazuki

I would like to know How Vertica load data.

 

In documentations, 

COPY :

FILE/STDIN -> WOS -> Moveout(Sort & Compress) -> ROS -> Mergeout(Bond)

COPY DIRECT :

FILE/STDIN -> ROS -> Mergeout(Sort & Compress & Bond)

(Do i understand correctly?)

 

However, i can't find when segmentation or partitioning occur.

Does anyone know that?

 

eli_revach

Hi

Small correction for the ROS load :

FILE/STDIN -> ROS (Sort & Compress )  -> Mergeout(Sort & Compress & Bond) 

 

About partition and segmentation 

Segmentation  WOS & ROS - as part of the data load the internal loader  applay the hasing function to the segmentaion keys and send it to the related executer node .

 

Partition is only applay to ROS data , so it will be applay as part of MoveOut for WOS data and by internal loader for ROS 

 

Thanks 

Kazuki

Hello eli_revach

Thank you for reply

 

• initiator node

1. copy data from file/stdin
2. segment data by segmentation keys and distribute it  to executer node

• executer node

1. sort data
2. compress data using RLE  <= WOS creation completed
3. Moveout(sort, compress, partitioning) <= ROS created
4. Mergeout(sort, compress) 

If data is loaded like above, is segmented data distributed as raw data(not compressed)?

 

eli_revach

Yes ,

initiator is responsable for loading the data from input file parse it  and segment it and send it to each one of the nodes as a raw , each one of the executer nodes (including initiator ) sorting & compress  are localy take place .

Thanks 

Kazuki

Thank you very much for your help!