ERROR: Not all nodes came up, but not all down. Run scrutinize.

shangshang Community Edition User
edited June 2020 in General Discussion

when I create an EON mode database as below, I got this Error
to appreciate it if someone can help me fix this error.
thanks again.

[dbadmin@iZcZ ~]$ admintools -t create_db -x auth_params.conf -s 'xxx.xxx.156.44,xxx.xxx.156.45,xxx.xxx.156.46' -d VMart1 -p xxx --depot-path=/home/dbadmin/depot --shard-count=6 --communal-storage-location=s3://xxxxx-test-1 -D /home/dbadmin/data/ -c /home/dbadmin/catalog/ --depot-size 5G
Distributing changes to cluster.
Creating database VMart1
Starting bootstrap node v_vmart1_node0002 (xxx.xxx.156.45)
Starting nodes:
v_vmart1_node0002 (xxx.xxx.156.45)
Starting Vertica on all nodes. Please wait, databases with a large catalog may take a while to initialize.
Node Status: v_vmart1_node0002: (DOWN)
Node Status: v_vmart1_node0002: (DOWN)
Node Status: v_vmart1_node0002: (DOWN)
Node Status: v_vmart1_node0002: (DOWN)
Node Status: v_vmart1_node0002: (DOWN)
Node Status: v_vmart1_node0002: (UP)
Creating database nodes
Creating node v_vmart1_node0001 (host xxx.xxx.156.44)
Creating node v_vmart1_node0003 (host xxx.xxx.156.46)
Generating new configuration information
Stopping single node db before adding additional nodes.
Database shutdown complete
Starting all nodes
Start hosts = ['xxx.xxx.156.44', 'xxx.xxx.156.45', 'xxx.xxx.156.46']
Starting nodes:
v_vmart1_node0002 (xxx.xxx.156.45)
v_vmart1_node0001 (xxx.xxx.156.44)
v_vmart1_node0003 (xxx.xxx.156.46)
Starting Vertica on all nodes. Please wait, databases with a large catalog may take a while to initialize.
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
It is suggested that you continue waiting.
Do you want to continue waiting? (yes/no) [yes]
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
It is suggested that you continue waiting.
Do you want to continue waiting? (yes/no) [yes]
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
It is suggested that you continue waiting.
Do you want to continue waiting? (yes/no) [yes]
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
It is suggested that you continue waiting.
Do you want to continue waiting? (yes/no) [yes]
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
Node Status: v_vmart1_node0001: (DOWN) v_vmart1_node0002: (DOWN) v_vmart1_node0003: (DOWN)
ERROR: Not all nodes came up, but not all down. Run scrutinize.
Unable to establish vsql script connection: Unable to connect to 'VMart1'
Unable to establish client-server connection: Unable to connect to 'VMart1'
Unable to create depot storage locations (if Eon) without a client-server connection.
Unable to rebalance shards (if Eon) without a client-server connection.
Unable to set K-safety value without a client-server connection.
Unable to install default extension packages without a vsql script connection
Unable to sync database catalog (if Eon) without a client-server connection.
Database creation SQL tasks included one or more failures (see above).
Database VMart1 created successfully, some nodes may have had startup problems.

Best wishes

Answers

  • HibikiHibiki Vertica Employee Employee

    If you have still faced this issue, please give us the log files.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file