Is there any way to promote secondary subcluster to primary subcluster?
cyun
Vertica Employee
Is there any way to promote secondary subcluster to primary subcluster when primary subcluster was down?
If there are two or more primary subclusters with the same number of nodes and specifications, I wonder if there is a way to use DML without restrictions even if a specific primary subcluster goes down.
dbadmin=> select version();
version
-------------------------------------
Vertica Analytic Database v12.0.0-0
-- Even if I have two primary subclusters, if one primary subcluster goes down, the remaining clusters(including primary subcluster) are converted to read-only mode.
dbadmin=> select subcluster_name, node_name, node_state, is_primary, is_readonly from nodes order by node_name; subcluster_name | node_name | node_state | is_primary | is_readonly -----------------+-------------------+------------+------------+------------- sc1 | v_vertdb_node0001 | UP | t | f sc1 | v_vertdb_node0002 | UP | t | f sc1 | v_vertdb_node0003 | UP | t | f sc2 | v_vertdb_node0004 | UP | t | f sc2 | v_vertdb_node0005 | UP | t | f sc2 | v_vertdb_node0006 | UP | t | f sc3 | v_vertdb_node0007 | UP | f | f sc3 | v_vertdb_node0008 | UP | f | f sc3 | v_vertdb_node0009 | UP | f | f
-- primary subcluster #2 shutdown
dbadmin=> select shutdown_subcluster('sc2');
dbadmin=> select subcluster_name, node_name, node_state, is_primary, is_readonly from nodes order by node_name;
subcluster_name | node_name | node_state | is_primary | is_readonly
-----------------+-------------------+------------+------------+-------------
sc1 | v_vertdb_node0001 | UP | t | t
sc1 | v_vertdb_node0002 | UP | t | t
sc1 | v_vertdb_node0003 | UP | t | t
sc2 | v_vertdb_node0004 | DOWN | t | t
sc2 | v_vertdb_node0005 | DOWN | t | t
sc2 | v_vertdb_node0006 | DOWN | t | t
sc3 | v_vertdb_node0007 | UP | f | t
sc3 | v_vertdb_node0008 | UP | f | t
sc3 | v_vertdb_node0009 | UP | f | t
(9 rows)
dbadmin=> create table t1(c1 int);
ERROR 10428: Transaction commit aborted since the database is currently in read-only mode
HINT: Commits will be restored when the database restores the quorum
-- If I have an additional secondary subcluster in one primary subcluster, the secondary subcluster cannot be promoted to the primary subcluster after the primary subcluster is down.
dbadmin=> select demote_subcluster_to_secondary('sc2');
demote_subcluster_to_secondary
--------------------------------
DEMOTE SUBCLUSTER TO SECONDARY
(1 row)
dbadmin=>
dbadmin=> select subcluster_name, node_name, node_state, is_primary, is_readonly from nodes order by node_name;
subcluster_name | node_name | node_state | is_primary | is_readonly
-----------------+-------------------+------------+------------+-------------
sc1 | v_vertdb_node0001 | UP | t | f
sc1 | v_vertdb_node0002 | UP | t | f
sc1 | v_vertdb_node0003 | UP | t | f
sc2 | v_vertdb_node0004 | UP | f | f
sc2 | v_vertdb_node0005 | UP | f | f
sc2 | v_vertdb_node0006 | UP | f | f
sc3 | v_vertdb_node0007 | UP | f | f
sc3 | v_vertdb_node0008 | UP | f | f
sc3 | v_vertdb_node0009 | UP | f | f
(9 rows)
dbadmin=> select shutdown_subcluster('sc1');
shutdown_subcluster
---------------------
Subcluster shutdown
dbadmin=> select subcluster_name, node_name, node_state, is_primary, is_readonly from nodes order by node_name;
subcluster_name | node_name | node_state | is_primary | is_readonly
-----------------+-------------------+------------+------------+-------------
sc1 | v_vertdb_node0001 | DOWN | t | t
sc1 | v_vertdb_node0002 | DOWN | t | t
sc1 | v_vertdb_node0003 | DOWN | t | t
sc2 | v_vertdb_node0004 | UP | f | t
sc2 | v_vertdb_node0005 | UP | f | t
sc2 | v_vertdb_node0006 | UP | f | t
sc3 | v_vertdb_node0007 | UP | f | t
sc3 | v_vertdb_node0008 | UP | f | t
sc3 | v_vertdb_node0009 | UP | f | t
(9 rows)
dbadmin=> select promote_subcluster_to_primary('sc2');
ERROR 10551: Modifying subclusters is not supported in read-only mode
HINT: The operation will be supported when the database is restored to full mode
0
Answers
All the primary nodes work together to maintain the data in the shards. even if one primary subcluster is down, cluster will go into read only mode. In read only, you cannot promote secondary subcluster to primary. it is expected behavior. for more information, please visit the below link
https://www.vertica.com/docs/12.0.x/HTML/Content/Authoring/Eon/MaintainingDataIntegrityInEonMode.htm
Thank you for your reply.
I have an additional question.
If there are 2 primary subclusters, can you explain in more detail why if one primary subcluster goes down, the remaining primary subcluster cannot maintain quorum?
There are two requirements for the database to continue normal operations: maintaining quorum, and maintaining shard coverage.
SELECT * FROM critical_subclusters; will list the EON primary subclusters whose loss would cause the database to become unsafe and force it to shutdown.
See: https://www.vertica.com/docs/12.0.x/HTML/Content/Authoring/Eon/MaintainingDataIntegrityInEonMode.htm