I can't remove the nodes in the cluster
Anyone know what is going on, what happens is that I want to remove some nodes that really are no longer part of the cluster but in admintools and in the MC it considers them as part of the cluster. When I try to remove them from admintools I get this error
[dbadmin@AZURE ~]$ /opt/vertica/bin/adminTools -t db_remove_node -p 'password' -d DB_name -s 172.18.220.111
Error removing node(s) from database. ['Unable to mark node \'v_db_name_node0002\' ephemeral: Severity: ROLLBACK, Message: Node "v_db_name_node0002" does not exist, Sqlstate: 42704, Routine: AlterNode, Fi le: /data/jenkins/workspace/RE-ReleaseBuilds/RE-Miner/server/vertica/Commands/DDL.cpp, Line: 7813, Error Code: 4136, SQL: VALTER NODE v_db_name_node0002 IS EPHEMERAL;\' ']
[dbadmin@AZURE ~]$ /opt/vertica/bin/adminTools -t list_allnodes
Node | Host | State | Version | DB
---------------------+----------------+-------+------------------+----------
v_db_name_node0001 | 172.18.220.135 | UP | vertica-24.2.0.1 | DB_name
v_db_name_node0002 | 172.18.220.111 | DOWN | vertica-24.2.0.1 | DB_name
v_db_name_node0003 | 172.18.220.86 | DOWN | vertica-24.2.0.1 | DB_name
as can be seen, the two nodes to be removed are in down status, the version used is 24.2.0-1
querying from vsql actually only returns a single node, which is correct
select * from nodes;
node_name | node_id | node_state | is_primary | is_readonly | node_address | node_address_family | export_address | export_address_family | catalog_path | node_type | is_ephemeral | standing_in_for | subcluster_name | sandbox | last_msg_from_node_at | node_down_since | build_info
---------------------+-------------------+------------+------------+-------------+----------------+---------------------+----------------+-----------------------+--------------------------------------------------------------+-----------+--------------+-----------------+--------------------+---------+-------------------------------+-----------------+----------------------------------------------------
v_db_name_node0001 | 45035996273704992 | UP | t | f | 172.18.220.135 | ipv4 | 172.18.220.135 | ipv4 | /data01/catalog/ModeloBI/v_db_name_node0001_catalog/Catalog | PERMANENT | f | | default_subcluster | | 2024-08-14 18:41:29.005455-05 | | v24.2.0-1-d52dc2eff25ea7e221904aded25063b05974ebab
(1 row)
what is wrong and how to fix it, it is worth mentioning that it is an Eon mode deployed in Azure.
Thanks for your answers.
Answers
Vertica should be able access node via SSH so that it can be removed. If instance no longer exists, it is tough. Please open a support case as it requires many manual steps to remove nodes.