Vertica load procedure

KazukiKazuki Registered User

I would like to know How Vertica load data.

 

In documentations, 

COPY :

FILE/STDIN -> WOS -> Moveout(Sort & Compress) -> ROS -> Mergeout(Bond)

COPY DIRECT :

FILE/STDIN -> ROS -> Mergeout(Sort & Compress & Bond)

(Do i understand correctly?)

 

However, i can't find when segmentation or partitioning occur.

Does anyone know that?

 

Comments

  • eli_revacheli_revach Registered User

    Hi

    Small correction for the ROS load :

    FILE/STDIN -> ROS (Sort & Compress )  -> Mergeout(Sort & Compress & Bond) 

     

    About partition and segmentation 

    Segmentation  WOS & ROS - as part of the data load the internal loader  applay the hasing function to the segmentaion keys and send it to the related executer node .

     

    Partition is only applay to ROS data , so it will be applay as part of MoveOut for WOS data and by internal loader for ROS 

     

    Thanks 

  • KazukiKazuki Registered User

    Hello eli_revach

    Thank you for reply

     

    • initiator node
      1. copy data from file/stdin
      2. segment data by segmentation keys and distribute it  to executer node
    • executer node
      1. sort data
      2. compress data using RLE  <= WOS creation completed
      3. Moveout(sort, compress, partitioning) <= ROS created
      4. Mergeout(sort, compress) 

    If data is loaded like above, is segmented data distributed as raw data(not compressed)?

     

  • eli_revacheli_revach Registered User

    Yes ,

    initiator is responsable for loading the data from input file parse it  and segment it and send it to each one of the nodes as a raw , each one of the executer nodes (including initiator ) sorting & compress  are localy take place .

    Thanks 

  • KazukiKazuki Registered User

    Thank you very much for your help!

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file