The Vertica Forum recently got a makeover! Let us know what you think by filling out this short, anonymous survey.
Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!
vbr -t replicate requires REBALANCE

Hello,
After we replicate one of our tables to the secondary cluster it seems that the table needs to be rebalanced.
In REBALANCE_STATUS the progress is 0% and it's take some time to finish the rebalance.
In the DDL the projections are "ALL NODES" , all nodes are UP and the clusters have the same number of nodes.
We don't know how to fix this problem,
Thank you !
Tagged:
0
Answers
Hi,
That's weird. What version of Vertica is this? Are both clusters using the same version? Can you post the output of:
SELECT export_objects('', 'table_schema.table_name');
From both the source and the target?
**CLUSTER 1
**
dbadmin=> select version();
version
Vertica Analytic Database V10.0.1-3
dbadmin=> CREATE TABLE test (id number);
CREATE TABLE
dbadmin=> insert into test values (1);
OUTPUT
1
dbadmin=>commit;
COMMIT
dbadmin=>select * from test;
id
1
dbadmin=>select export_objects('','test');
export objects
CREATE TABLE public.test
(
id numeric(38,0)
);
CREATE PROJECTION public.test /+createtype(L)/
(
id
)
AS
SELECT test.id
FROM public.test
ORDER BY test.id
SEGMENTED BY hash(test.id) ALL NODES KSAFE 1;
SELECT MARK_DESIGN_KSAFE(1);
dbadmin=> select table_name,separated_percent,transferred_percent from rebalance_status where table_name='test';
table_name | separated_percent | transferred_percent
-----------+-------------------+--------------------
(0 rows)
vbr -t replicate -c clone_TEST.ini --debug 3
**CLUSTER 2
**
dbadmin=> select version();
version
Vertica Analytic Database V10.0.1-3
dbadmin=>select * from test;
id
1
dbadmin=>select export_objects('','test');
export objects
CREATE TABLE public.test
(
id numeric(38,0)
);
CREATE PROJECTION public.test /+createtype(L)/
(
id
)
AS
SELECT test.id
FROM public.test
ORDER BY test.id
SEGMENTED BY hash(test.id) ALL NODES KSAFE 1;
SELECT MARK_DESIGN_KSAFE(1);
dbadmin=> select table_name,separated_percent,transferred_percent from rebalance_status where table_name='test';
table_name | separated_percent | transferred_percent
-----------+-------------------+--------------------
test | 0.00| 0.00
(1 row)
After you ran vbr replicate, you did not run rebalance right? Still we can see one row from your test table right?
Right, after I replicate the table I see a record in REBALANCE_STATUS (in cluster 1 I don't see since it was created in that cluster)
Can you share me the output of the following and attach the vbr log? I see that you ran it debug 3 option
select * from rebalance_status where table_name='test';
Hi SruthiA,
dbadmin=> select * from rebalance_status where table_name='test';
table_schema | table_name | separated_percent | transferred_percent | TABLE_SIZE_GB
-------------+------------+-------------------+---------------------+--------------
public | test | 0.00 | 0.00 | 0
dbadmin@hostname [~/VbrDir]
$ vbr -t replicate -c clone_TEST.ini --debug 3
Participating nodes: V_DatabaseName_node0001,V_DatabaseName_node0002,V_DatabaseName_node0003,V_DatabaseName_node0004,V_DatabaseName_node0005,V_DatabaseName_node0006,V_DatabaseName_node0007,V_DatabaseName_node0008,V_DatabaseName_node0009,V_DatabaseName_node0010,V_DatabaseName_node0011,V_DatabaseName_node0012,V_DatabaseName_node0013,V_DatabaseName_node0014,V_DatabaseName_node0015,V_DatabaseName_node0016,V_DatabaseName_node0017,V_DatabaseName_node0018
Starting replication of objects ['public.TEST'] from DatabaseName.
Snapshotting the source database.
Snapshot complete.
Copying catalog snapshot from source to destination.
Preparing destination database for replicating objects.
Prep complete, start syncing files.
Approximate bytes to copy: 0 of 164 total.
Progress: 0 out of 0 bytes (100%)
Finalizing object replication.
Complete object replication, removing snapshot from the source database.
Object replication complete!
I meant vbr log file for the replicate you have run.. It should be present in /tmp/vbr.
Is there anything specific I should look for? this is closed site so I can't attach the log.
Also I don't see any errors there.
Thanks.
@ybbhosale - What are you seeing in the REMOTE_REPLICATION_STATUS table on the source database?
Example:
Hi Jim,
dbadmin=> select * from remote_replication_status;
objects | snapshot_name | replication_point | last_replicated_time | replicated_epoch | current_epoch
------------+---------------------+--------------------------------------------+----------------------------+------------------------+-------------------
public.test |snapshot_copy_objects |snapshot_copy_objects_ACF6D.. | 2021-10-17 18:10:41+03 | 2375600 | 2375622
@lybhrwn742 - I tested replication using Vertica 10.0.1-3 between two three node clusters. I did not see any requirements for rebalancing on the target DB.
Can you run the following commands on both of your clusters?
SELECT get_node_dependencies();
SELECT COUNT(*) FROM projections WHERE NOT is_up_to_date;
Thanks!
dbadmin=> select get_node_dependencies();
000000000000000011 - cnt: 7
000000000000000110 - cnt: 56
000000000000001100 - cnt: 56
000000000000011000 - cnt: 9
000000000000110000 - cnt: 58
000000000001001000 - cnt: 2
000000000001100000 - cnt: 58
000000000011000000 - cnt: 7
000000000110000000 - cnt: 56
000000001100000000 - cnt: 56
000000011000000000 - cnt: 7
000000110000000000 - cnt: 56
000001100000000000 - cnt: 56
000010000000000001 - cnt: 56
000011000000000000 - cnt: 56
000011111111111111 - cnt: 6
000100000000000001 - cnt: 49
000100000000000010 - cnt: 49
001000000000001000 - cnt: 49
001000000000010000 - cnt: 49
010000000000100000 - cnt: 49
010000000001000000 - cnt: 49
100000000100000000 - cnt: 49
100000001000000000 - cnt: 49
111111111111111111 - cnt: 1
000000000000000001 - name: v_database_name_node0001
000000000000000010 - name: v_database_name_node0002
000000000000000100 - name: v_database_name_node0003
000000000000001000 - name: v_database_name_node0004
000000000000010000 - name: v_database_name_node0005
000000000000100000 - name: v_database_name_node0006
000000000001000000 - name: v_database_name_node0007
000000000010000000 - name: v_database_name_node0008
000000000100000000 - name: v_database_name_node0009
000000001000000000 - name: v_database_name_node0010
000000010000000000 - name: v_database_name_node0011
000000100000000000 - name: v_database_name_node0012
000001000000000000 - name: v_database_name_node0013
000010000000000000 - name: v_database_name_node0014
000100000000000000 - name: v_database_name_node0015
001000000000000000 - name: v_database_name_node0016
010000000000000000 - name: v_database_name_node0017
100000000000000000 - name: v_database_name_node0018
dbadmin=> select count(*) from projections where not is_up_to_date;
count
(1 row)
dbadmin=> select get_node_dependencies();
000000000000000011 - cnt: 2
000000000000000110 - cnt: 8
000000000000001001 - cnt: 1
000000000000001100 - cnt: 74
000000000000011000 - cnt: 8
000000000000110000 - cnt: 74
000000000001100000 - cnt: 3
000000000011000000 - cnt: 73
000000000110000000 - cnt: 74
000000001100000000 - cnt: 74
000000010000000001 - cnt: 6
000000011000000000 - cnt: 73
000000011111111111 - cnt: 6
000000100000000001 - cnt: 71
000000110000000000 - cnt: 2
000001000000000010 - cnt: 1
000001000000100000 - cnt: 71
000001000001000000 - cnt: 71
000001100000000000 - cnt: 3
000010000000000001 - cnt: 65
000010001000000000 - cnt: 1
000010010000000000 - cnt: 66
000100000001000000 - cnt: 1
000100000010000000 - cnt: 1
000100100000000000 - cnt: 65
001000000000000010 - cnt: 66
001000000000000100 - cnt: 66
010000000000000001 - cnt: 1
010000000000001000 - cnt: 65
010000000000010000 - cnt: 66
100000000000000001 - cnt: 2
100000000000000010 - cnt: 71
100000010000000000 - cnt: 1
100000100000000000 - cnt: 7
100001000000000000 - cnt: 2
100100000000000000 - cnt: 65
111111111111111111 - cnt: 1
000000000000000001 - name: v_database_name_node0004
000000000000000001 - name: v_database_name_node0001
000000000000000001 - name: v_database_name_node0002
000000000000000001 - name: v_database_name_node0003
000000000000000001 - name: v_database_name_node0005
000000000000000001 - name: v_database_name_node0006
000000000000000001 - name: v_database_name_node0007
000000000000000001 - name: v_database_name_node0008
000000000000000001 - name: v_database_name_node0009
000000000000000001 - name: v_database_name_node0010
000000000000000001 - name: v_database_name_node0011
000000000000000001 - name: v_database_name_node0013
000000000000000001 - name: v_database_name_node0014
000000000000000001 - name: v_database_name_node0015
000000000000000001 - name: v_database_name_node0016
000000000000000001 - name: v_database_name_node0017
000000000000000001 - name: v_database_name_node0018
000000000000000001 - name: v_database_name_node0012
dbadmin=> select count(*) from projections where not is_up_to_date;
count
(1 row)
Thanks.
Correction for cluster 2 -
000000000000000001 - name: v_database_name_node0004
000000000000000010 - name: v_database_name_node0001
000000000000000100 - name: v_database_name_node0002
000000000000001000 - name: v_database_name_node0003
000000000000010000 - name: v_database_name_node0005
000000000000100000 - name: v_database_name_node0006
000000000001000000 - name: v_database_name_node0007
000000000010000000 - name: v_database_name_node0008
000000000100000000 - name: v_database_name_node0009
000000001000000000 - name: v_database_name_node0010
000000010000000000 - name: v_database_name_node0011
000000100000000000 - name: v_database_name_node0013
000001000000000000 - name: v_database_name_node0014
000010000000000000 - name: v_database_name_node0015
000100000000000000 - name: v_database_name_node0016
001000000000000000 - name: v_database_name_node0017
010000000000000000 - name: v_database_name_node0018
100000000000000000 - name: v_database_name_node0012