Hadoop connector

Muthu_Sundaram_1

Hadoop Connector - I believe we install this in every node of hadoop cluster?
What do we install in vertica cluster? What runs in vertica nodes to communicate with counterparts in hadoop nodes? What protocol does it use? Please provide more details regarding the communication between hadoop nodes and vertica nodes.

R_McKeon

Hello Muthu,

Yes, you install the Hadoop Connector on each node. From https://my.vertica.com/docs/6.1.x/HTML/index.htm#15504.htm:

The Hadoop Connector runs on each node in the Hadoop cluster, so the Hadoop nodes and HP Vertica nodes communicate with each other directly. Direct connections allow data to be transferred in parallel, dramatically increasing processing speed.

The information regarding how Hadoop and Vertica communicate is located at the following links:

https://my.vertica.com/docs/6.1.x/HTML/index.htm#15503.htm
https://my.vertica.com/docs/6.1.x/HTML/index.htm#15511.htm

Thanks,
Rory

Muthu_Sundaram_1

Thanks Roy. I am trying to put an architectural diagram. Hadoop connectors are installed in Hadoop nodes. What is listening in Vertica clusters? What is the counterpart in Vertica side? Are hadoop clusters pushing data to Vertica using JDBC? What kind of protocol does it use? Should number of nodes between hadoop and Vertica match? Node1 from hadoop will talk directly to Node1 in vertica? Do we need any loadbalancer in vertica side for this?