The Vertica Forum recently got a makeover! Let us know what you think by filling out this short, anonymous survey.
Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!

vbr -t replicate requires REBALANCE

lybhrwn742lybhrwn742 Vertica Customer
in General Discussion

Hello,
After we replicate one of our tables to the secondary cluster it seems that the table needs to be rebalanced.
In REBALANCE_STATUS the progress is 0% and it's take some time to finish the rebalance.
In the DDL the projections are "ALL NODES" , all nodes are UP and the clusters have the same number of nodes.
We don't know how to fix this problem,
Thank you !

Tagged:

Answers

  • Jim_KnicelyJim_Knicely - Select Field - Administrator
    edited October 2021

    Hi,

    That's weird. What version of Vertica is this? Are both clusters using the same version? Can you post the output of:

    SELECT export_objects('', 'table_schema.table_name');

    From both the source and the target?

  • lybhrwn742lybhrwn742 Vertica Customer

    **CLUSTER 1
    **
    dbadmin=> select version();

    version

    Vertica Analytic Database V10.0.1-3

    dbadmin=> CREATE TABLE test (id number);
    CREATE TABLE
    dbadmin=> insert into test values (1);

    OUTPUT

    1

    dbadmin=>commit;
    COMMIT
    dbadmin=>select * from test;

    id

    1

    dbadmin=>select export_objects('','test');

    export objects

    CREATE TABLE public.test
    (
    id numeric(38,0)
    );

    CREATE PROJECTION public.test /+createtype(L)/
    (
    id
    )
    AS
    SELECT test.id
    FROM public.test
    ORDER BY test.id
    SEGMENTED BY hash(test.id) ALL NODES KSAFE 1;

    SELECT MARK_DESIGN_KSAFE(1);

    dbadmin=> select table_name,separated_percent,transferred_percent from rebalance_status where table_name='test';
    table_name | separated_percent | transferred_percent
    -----------+-------------------+--------------------
    (0 rows)

    vbr -t replicate -c clone_TEST.ini --debug 3

    **CLUSTER 2
    **
    dbadmin=> select version();

    version

    Vertica Analytic Database V10.0.1-3

    dbadmin=>select * from test;

    id

    1

    dbadmin=>select export_objects('','test');

    export objects

    CREATE TABLE public.test
    (
    id numeric(38,0)
    );

    CREATE PROJECTION public.test /+createtype(L)/
    (
    id
    )
    AS
    SELECT test.id
    FROM public.test
    ORDER BY test.id
    SEGMENTED BY hash(test.id) ALL NODES KSAFE 1;

    SELECT MARK_DESIGN_KSAFE(1);

    dbadmin=> select table_name,separated_percent,transferred_percent from rebalance_status where table_name='test';
    table_name | separated_percent | transferred_percent
    -----------+-------------------+--------------------
    test | 0.00| 0.00
    (1 row)

  • SruthiASruthiA Vertica Employee Employee

    After you ran vbr replicate, you did not run rebalance right? Still we can see one row from your test table right?

  • lybhrwn742lybhrwn742 Vertica Customer

    Right, after I replicate the table I see a record in REBALANCE_STATUS (in cluster 1 I don't see since it was created in that cluster)

  • SruthiASruthiA Vertica Employee Employee

    Can you share me the output of the following and attach the vbr log? I see that you ran it debug 3 option

    select * from rebalance_status where table_name='test';

  • lybhrwn742lybhrwn742 Vertica Customer

    Hi SruthiA,

    dbadmin=> select * from rebalance_status where table_name='test';
    table_schema | table_name | separated_percent | transferred_percent | TABLE_SIZE_GB
    -------------+------------+-------------------+---------------------+--------------
    public | test | 0.00 | 0.00 | 0

    dbadmin@hostname [~/VbrDir]
    $ vbr -t replicate -c clone_TEST.ini --debug 3

    Participating nodes: V_DatabaseName_node0001,V_DatabaseName_node0002,V_DatabaseName_node0003,V_DatabaseName_node0004,V_DatabaseName_node0005,V_DatabaseName_node0006,V_DatabaseName_node0007,V_DatabaseName_node0008,V_DatabaseName_node0009,V_DatabaseName_node0010,V_DatabaseName_node0011,V_DatabaseName_node0012,V_DatabaseName_node0013,V_DatabaseName_node0014,V_DatabaseName_node0015,V_DatabaseName_node0016,V_DatabaseName_node0017,V_DatabaseName_node0018
    Starting replication of objects ['public.TEST'] from DatabaseName.
    Snapshotting the source database.
    Snapshot complete.
    Copying catalog snapshot from source to destination.
    Preparing destination database for replicating objects.
    Prep complete, start syncing files.
    Approximate bytes to copy: 0 of 164 total.
    Progress: 0 out of 0 bytes (100%)
    Finalizing object replication.
    Complete object replication, removing snapshot from the source database.
    Object replication complete!

  • SruthiASruthiA Vertica Employee Employee

    I meant vbr log file for the replicate you have run.. It should be present in /tmp/vbr.

  • lybhrwn742lybhrwn742 Vertica Customer

    Is there anything specific I should look for? this is closed site so I can't attach the log.
    Also I don't see any errors there.

    Thanks.

  • Jim_KnicelyJim_Knicely - Select Field - Administrator

    @ybbhosale - What are you seeing in the REMOTE_REPLICATION_STATUS table on the source database?

    Example:

    dbadmin=> SELECT * FROM remote_replication_status;
 objects |  snapshot_name  |                replication_point                 |  last_replicated_time  | replicated_epoch | current_epoch
---------+-----------------+--------------------------------------------------+------------------------+------------------+---------------
 test    | backup_snapshot | backup_snapshot_3R7MOX4G59S5FZA3WPSWBPGKXHWCIL01 | 2021-10-13 07:34:49-04 |               19 |            20
(1 row)
  • lybhrwn742lybhrwn742 Vertica Customer

    Hi Jim,

    dbadmin=> select * from remote_replication_status;
    objects | snapshot_name | replication_point | last_replicated_time | replicated_epoch | current_epoch
    ------------+---------------------+--------------------------------------------+----------------------------+------------------------+-------------------
    public.test |snapshot_copy_objects |snapshot_copy_objects_ACF6D.. | 2021-10-17 18:10:41+03 | 2375600 | 2375622

  • Jim_KnicelyJim_Knicely - Select Field - Administrator

    @lybhrwn742 - I tested replication using Vertica 10.0.1-3 between two three node clusters. I did not see any requirements for rebalancing on the target DB.

    Can you run the following commands on both of your clusters?

    SELECT get_node_dependencies();
    SELECT COUNT(*) FROM projections WHERE NOT is_up_to_date;

    Thanks!

  • lybhrwn742lybhrwn742 Vertica Customer 
        CLUSTER 1

    dbadmin=> select get_node_dependencies();

    000000000000000011 - cnt: 7
    000000000000000110 - cnt: 56
    000000000000001100 - cnt: 56
    000000000000011000 - cnt: 9
    000000000000110000 - cnt: 58
    000000000001001000 - cnt: 2
    000000000001100000 - cnt: 58
    000000000011000000 - cnt: 7
    000000000110000000 - cnt: 56
    000000001100000000 - cnt: 56
    000000011000000000 - cnt: 7
    000000110000000000 - cnt: 56
    000001100000000000 - cnt: 56
    000010000000000001 - cnt: 56
    000011000000000000 - cnt: 56
    000011111111111111 - cnt: 6
    000100000000000001 - cnt: 49
    000100000000000010 - cnt: 49
    001000000000001000 - cnt: 49
    001000000000010000 - cnt: 49
    010000000000100000 - cnt: 49
    010000000001000000 - cnt: 49
    100000000100000000 - cnt: 49
    100000001000000000 - cnt: 49
    111111111111111111 - cnt: 1

    000000000000000001 - name: v_database_name_node0001
    000000000000000010 - name: v_database_name_node0002
    000000000000000100 - name: v_database_name_node0003
    000000000000001000 - name: v_database_name_node0004
    000000000000010000 - name: v_database_name_node0005
    000000000000100000 - name: v_database_name_node0006
    000000000001000000 - name: v_database_name_node0007
    000000000010000000 - name: v_database_name_node0008
    000000000100000000 - name: v_database_name_node0009
    000000001000000000 - name: v_database_name_node0010
    000000010000000000 - name: v_database_name_node0011
    000000100000000000 - name: v_database_name_node0012
    000001000000000000 - name: v_database_name_node0013
    000010000000000000 - name: v_database_name_node0014
    000100000000000000 - name: v_database_name_node0015
    001000000000000000 - name: v_database_name_node0016
    010000000000000000 - name: v_database_name_node0017
    100000000000000000 - name: v_database_name_node0018

    dbadmin=> select count(*) from projections where not is_up_to_date;

    count

      0

    (1 row)

    CLUSTER 2

    dbadmin=> select get_node_dependencies();

    000000000000000011 - cnt: 2
    000000000000000110 - cnt: 8
    000000000000001001 - cnt: 1
    000000000000001100 - cnt: 74
    000000000000011000 - cnt: 8
    000000000000110000 - cnt: 74
    000000000001100000 - cnt: 3
    000000000011000000 - cnt: 73
    000000000110000000 - cnt: 74
    000000001100000000 - cnt: 74
    000000010000000001 - cnt: 6
    000000011000000000 - cnt: 73
    000000011111111111 - cnt: 6
    000000100000000001 - cnt: 71
    000000110000000000 - cnt: 2
    000001000000000010 - cnt: 1
    000001000000100000 - cnt: 71
    000001000001000000 - cnt: 71
    000001100000000000 - cnt: 3
    000010000000000001 - cnt: 65
    000010001000000000 - cnt: 1
    000010010000000000 - cnt: 66
    000100000001000000 - cnt: 1
    000100000010000000 - cnt: 1
    000100100000000000 - cnt: 65
    001000000000000010 - cnt: 66
    001000000000000100 - cnt: 66
    010000000000000001 - cnt: 1
    010000000000001000 - cnt: 65
    010000000000010000 - cnt: 66
    100000000000000001 - cnt: 2
    100000000000000010 - cnt: 71
    100000010000000000 - cnt: 1
    100000100000000000 - cnt: 7
    100001000000000000 - cnt: 2
    100100000000000000 - cnt: 65
    111111111111111111 - cnt: 1

    000000000000000001 - name: v_database_name_node0004
    000000000000000001 - name: v_database_name_node0001
    000000000000000001 - name: v_database_name_node0002
    000000000000000001 - name: v_database_name_node0003
    000000000000000001 - name: v_database_name_node0005
    000000000000000001 - name: v_database_name_node0006
    000000000000000001 - name: v_database_name_node0007
    000000000000000001 - name: v_database_name_node0008
    000000000000000001 - name: v_database_name_node0009
    000000000000000001 - name: v_database_name_node0010
    000000000000000001 - name: v_database_name_node0011
    000000000000000001 - name: v_database_name_node0013
    000000000000000001 - name: v_database_name_node0014
    000000000000000001 - name: v_database_name_node0015
    000000000000000001 - name: v_database_name_node0016
    000000000000000001 - name: v_database_name_node0017
    000000000000000001 - name: v_database_name_node0018
    000000000000000001 - name: v_database_name_node0012

    dbadmin=> select count(*) from projections where not is_up_to_date;

    count

      2

    (1 row)

    Thanks.

  • lybhrwn742lybhrwn742 Vertica Customer

    Correction for cluster 2 -

    000000000000000001 - name: v_database_name_node0004
    000000000000000010 - name: v_database_name_node0001
    000000000000000100 - name: v_database_name_node0002
    000000000000001000 - name: v_database_name_node0003
    000000000000010000 - name: v_database_name_node0005
    000000000000100000 - name: v_database_name_node0006
    000000000001000000 - name: v_database_name_node0007
    000000000010000000 - name: v_database_name_node0008
    000000000100000000 - name: v_database_name_node0009
    000000001000000000 - name: v_database_name_node0010
    000000010000000000 - name: v_database_name_node0011
    000000100000000000 - name: v_database_name_node0013
    000001000000000000 - name: v_database_name_node0014
    000010000000000000 - name: v_database_name_node0015
    000100000000000000 - name: v_database_name_node0016
    001000000000000000 - name: v_database_name_node0017
    010000000000000000 - name: v_database_name_node0018
    100000000000000000 - name: v_database_name_node0012

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Format
SpoilerCodeQuote
Heading 2Heading 1
Emoji
:):D:(;):/:o:s:p:'(:|B):#:*<3o:)>:)
Url
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file
Home • General DiscussionComment As ...