need to scale my Vertica cluster UP

K_Krutiy

I need to scale my cluster UP without impact to my current PROD cluster.

I have 2 different options in mind.

Option 1.

1. snapshot current cluster

2. restore snapshot on different cluster with same node count

3. add more nodes to this new cluster

4. rebalance data

5. make new bigger cluster PROD

Option 2.

1. create new blank Vertica cluster with more nodes

2. copy data from PROD cluster to new cluster using

            EXPORT TO VERTICA db.schema.table AS SELECT * FROM schema.table

3. make new bigger cluster PROD

Any advice on which option is better/faster ?

Any other methods available ?

Thanks

Konstantine

Adrian_Oprea_1

By scaling up you mean add more nodes ?  or you what to change hosts (use better machines)?

 I would take a full backup of my cluster and add a new node to the cluster , next re-balance the data across ! 
 This does not require you to stop the database and re-balance can be done in low activity time. 

K_Krutiy

I am planning on adding more nodes.

What is important is that I am planning to build new cluster and when it is ready I will move queries to new cluster.

This way my PROD operations will be not impacted

Konstantine

Adrian_Oprea_1

I think that one approach to do this might be creating your new cluster layout(with how many nodes you what)

1.  Export you metadata and recreate it so that will comply with your chosen k-safety on the new cluster. 
2.  Create a database user on the production database(with full access on its objects/tables).
3.  Create a specific resource pool that will not influence the performance of your production database when it will be used, and give access to this pool to the created user.
4. Using the connect(connect with the new created user)+ copy from command, copy your data from one database to another.

This would be my approach.

Curtis_1

Both seem like reasonable options. My bigger question is - what do you mean by "impact".  You could just add additional nodes to your existing cluster, and rebalance. And yes - there will be impact in the sense that Vertica will work to rebalance data across the new nodes. But your Vertica cluster can remain up during this time.

K_Krutiy

When I talk about impact I mean performance of queries.

My cluster work and Vertica doing rebalancing should not impact performance. Query response time is critical for us.

This is why I want to build new cluster in parallel with working.
When new cluster ready I will do benchmarks and make sure new cluster will perform better and only ofter that move operations to new cluster.


Also I am trying to get feel of which method will take less time.
I will do it in AWS and I want to run 2 clusters in parallel for shortest possible period of time.
This way I will try to keep AWS bill as low as possible.

my cluster is 9 nodes now.
I am thinking about building 15 node cluster.

What will take less time ? rebalancing cluster or copying same data from one cluster to another ?


Thanks
Konstantine

Adrian_Oprea

I would go for rebalance insted of creating a new cluster. I think at the end of is just a problem of resoirce managemet issue. One of Vertica main part is the inline scalar growth, we should take adantage of that.

eli_revach_1

I also prefer rebalance , however you mention that you plan to add more node to get better perormence , Vertica is near linear scale when in parallel to adding node you alos incress the size of the data  with the same facor , if this is not the case you will see some imrovment but not  linear , short time quert will not be effected or may be effected very small .

 

Ben_Vandiver

rebalance transfers significantly less data than a full reshuffle.
It also is more tolerant of failure -- if interrupted, it can restart from where it left off.

rebalance can be slower if you have a massive cluster, such as going from 100 -> 200 nodes, but that's not your situation.