Mixing on-prem and cloud-based nodes in Eon Mode?

GaryGGaryG Administrator
edited April 2020 in General Discussion

Is there a concept of a hybrid cluster with subclusters on-prem and in the cloud.

Question asked at the 2020 Big Data Conference presentation Sizing and Configuring Vertica in Eon Mode for Different Use Cases given by @skeswani and @skamat .

Answers

  • Bryan_HBryan_H Vertica Employee Administrator

    Is it technically possible? Yes, I was able to set up Eon mode using on-prem storage with on-prem and cloud (AWS) subclusters.
    Will there be issues with setup and operation? Yes, I would expect a cluster of any scale would need a lot of low-latency bandwidth. You'll need an SSH tunnel with appropriate network security configuration if you use on-prem storage, which I recommend because it's cheaper to send data to the cloud than to retrieve it and also it's easier to meet data locality requirements with persistent on-prem storage and ephemeral cloud caches. You'll also need sufficient bandwidth to on-prem and cloud compute to populate caches / depots and serve any requests that miss local cache. Also you might run into issues with spread since the clusters still all need to talk to each other.
    So it might be worth it if your business case justifies it and you're willing to work out any technical hurdles with infrastructure and networking.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file