Vertica Start Issue and Spread Socket file missing after Restart

amri_raed

Hello Everyone, I have vertica 9 installed on 3 linux RHEL nodes. After a restart on the servers the database is down and can't be started. I checked the dbLog file and found this error : unable to connect via UNIX socket to /opt/vertica/spread/tmp/4803 (pid=14883): Error: No such file or directory. the file 4803 which is missing seems to be the cause of this failure. Would you please suggest any solution ? Thanks

Jim_Knicely

Hi,

Did something change with the network? Like a subnet change?

How are you restarting the DB? What command?

Note that the /opt/vertica/spread/tmp/4803 file only exists when the DB is up.

[dbadmin@SE-Sandbox-26-node1 ~]$ ps -ef | grep [4]803
dbadmin  474660      1  1 Aug05 ?        00:49:47 /opt/vertica/bin/vertica -D /home/dbadmin/test_db/v_test_db_node0001_catalog -C test_db -n v_test_db_node0001 -h 172.16.61.176 -p 5433 -P 4803 -Y ipv4

[dbadmin@SE-Sandbox-26-node1 ~]$ ls /opt/vertica/spread/tmp/4803
/opt/vertica/spread/tmp/4803

[dbadmin@SE-Sandbox-26-node1 ~]$ admintools -t stop_db -d test_db -F
Info: no password specified, using none
        Connecting to database
        Issuing shutdown command to database
Database test_db stopped successfully

[dbadmin@SE-Sandbox-26-node1 ~]$ ls /opt/vertica/spread/tmp/4803
ls: cannot access /opt/vertica/spread/tmp/4803: No such file or directory

[dbadmin@SE-Sandbox-26-node1 tmp]$ admintools -t start_db -d test_db -F
Info: no password specified, using none
        Starting nodes:
                v_test_db_node0001 (172.16.61.176)
                v_test_db_node0002 (172.16.61.177)
                v_test_db_node0003 (172.16.61.178)
        Starting Vertica on all nodes. Please wait, databases with a large catalog may take a while to initialize.
        Node Status: v_test_db_node0001: (DOWN) v_test_db_node0002: (DOWN) v_test_db_node0003: (DOWN)
        Node Status: v_test_db_node0001: (DOWN) v_test_db_node0002: (DOWN) v_test_db_node0003: (DOWN)
        Node Status: v_test_db_node0001: (DOWN) v_test_db_node0002: (DOWN) v_test_db_node0003: (DOWN)
        Node Status: v_test_db_node0001: (DOWN) v_test_db_node0002: (DOWN) v_test_db_node0003: (DOWN)
        Node Status: v_test_db_node0001: (DOWN) v_test_db_node0002: (DOWN) v_test_db_node0003: (DOWN)
        Node Status: v_test_db_node0001: (INITIALIZING) v_test_db_node0002: (INITIALIZING) v_test_db_node0003: (INITIALIZING)
        Node Status: v_test_db_node0001: (UP) v_test_db_node0002: (UP) v_test_db_node0003: (UP)
Database test_db: Startup Succeeded.  All Nodes are UP

[dbadmin@SE-Sandbox-26-node1 tmp]$ ls /opt/vertica/spread/tmp/4803
/opt/vertica/spread/tmp/4803