Vertica is not starting on Node

veerkumar

i have tried admintools -t start_db -d OSSA
Starting nodes:
v_ossa_node0001 (10.254.169.129)
Starting Vertica on all nodes. Please wait, databases with a large catalog may take a while to initialize.
Node Status: v_ossa_node0001: (DOWN)
Node Status: v_ossa_node0001: (DOWN)
Node Status: v_ossa_node0001: (DOWN)
Node Status: v_ossa_node0001: (DOWN)
Node Status: v_ossa_node0001: (DOWN)
Node Status: v_ossa_node0001: (DOWN)
Node Status: v_ossa_node0001: (DOWN)
Node Status: v_ossa_node0001: (DOWN)
Node Status: v_ossa_node0001: (DOWN)
Node Status: v_ossa_node0001: (DOWN)
It is suggested that you continue waiting.
Do you want to continue waiting? (yes/no) [yes] no
Database startup was successful, but there are some nodes
which were not started. Loss of additional nodes could cause
the database to shutdown.

also tried with force start but no luck at all.
then looked into logs files (/data/vcatalog/OSSA/v_ossa_node0001_catalog/startup.log) and found below error at last

{
"node" : "v_ossa_node0001",
"stage" : "Startup Failed, ASR Required",
"text" : "Node Dependencies:\n1 - cnt: 69\n\n1 - name: v_ossa_node0001\nNodes certainly in the cluster:\n\tNode 0(v_ossa_node0001), epoch 12986465\nFilling more nodes to satisfy node dependencies:\nData dependencies fulfilled, remaining nodes LGEs don't matter:\n--",
"timestamp" : "2020-01-23 12:28:55.462"
}

Few days ago ago it was working fine, then VMs got rebooted and since then node is not starting.
Any help would be highly appreciated.
vertica version: v8.1.1-27

Bryan_H

Hi, we strongly recommend filing a support ticket if you are able. It is possible you might lose data during a force restart.
You can roll back the database to the LGE (Last Good Epoch), but you might lose data because this will also roll back any uncommitted transactions.
To force restart:
Log in as dbadmin (or admin user)
Run admintools
Select Option 7, Advanced Menu
Select Option 1, RollBack Database to Last Good Epoch
Select any options that are presented. Enter the database password and the database will be restarted at the last successful commit.
You can also restart from LGE from command line:
admintools -t restart_db -d -e last
Be sure to verify database integrity after restart!

veerkumar

Hi Bryan,
Thanks. it helped by using the admintools command (admintools -t restart_db -d -e last) suggested by you. It Worked.
But i have one more question. when i tried this by admintools GUI.
Select Option 7, Advanced Menu
Select Option 1, RollBack Database to Last Good Epoch
I got the error as below:
Error: Invalid selection:

Thats kind of strange.

Thanks again, it started on last good epoch.