Migrating single node ( local host bind) IP address node to 3 nodes Vertica cluster
I have been using a single node (127.0.0.1) vertica cluster. Currently , I am planning to move the DB from this node to a new 3 node cluster. In fact, I already changed the IP to a new non-local IP using "admintools -t re_ip -f mapfile.txt". I changes the IP just to avoid any type of backup/restore issue , if any.
AFAIK, we can't restore the full back from the single node cluster to 3 node cluster as per the design. So, I decided to move to a brand new single node cluster ( bind to x.x.x.x having same DB name ). I restored the full backup from the source one. It worked fine.
Now I wanted to add 2 new nodes to the above single node cluster ( IP = x.x.x.x). I used below command to add 2 hosts to the cluster ( x.x.x.x node1)
/opt/vertica/sbin/update_vertica --add-host y.y.y.y, z.z.z.z --dba-user verticauser --license ~/123.dat --failure-threshold FAIL --rpm ~/vertica-9.2.1-0.x86_64.RHEL6.rpm
it works fine. I a not sure, what is the best way to verify that the host additions to cluster worked fine. The result did't through any error.
Then, I tried to add the database to the new hosts.
admintools -t db_add_node -d testdb -p novell -s y.y.y.y , z.z.z.z
Looks like it works fine as well. I believe this step must create the database directory in same location. I am not able to see all the files under catalog (e.g. /v_testdb_node0002_catalog/ ). Only spread.conf and vertica.conf files exist.
Database starts on both the nodes throws error.
*** Starting database: testdb ***
Error: the vertica process for the database is running on the following hosts:
This may be because the process has not completed previous shutdown activities. Please wait and retry again.
Database start up failed. Processes still running.
Found these errors in startup.logs on hosts:
.../v_testdb_node0002_catalog/startup.log: exception Can't open .../v_testdb_node0002_catalog/startup.logfor reading: [Errno 2] No such file or directory:
I may be doing something completely wrong. The passwordless ssh was also set correctly. This looks like the required files for database (catalog) are not getting replicated to other 2 nodes.
I will appreciate for any help to resolve this issue.