Communal Storage Locations

J_KelleyJ_Kelley ✭
in General Discussion

Why support of more communal storage locations is not on your roadmap?
Migrating your Vertica Cluster to the Cloud
@SumeetAgrawal @Chris_Daly_HPE
Question from @Jan_Soubusta @JanSoubusta

Tagged:

Answers

  • dsprogisdsprogis Employee

    Supporting additional communal storage locations is definitely on our roadmap!
    Eon currently runs on AWS, Pure Storage FlashBlade, and Minio.
    With Version 10.0, Eon will run on HDFS and Google Cloud Platform (GCP).
    You can expect to see Eon on Azure next.
    We are working with Alibaba to get Eon running on OSS.
    We are looking at CEPH which, like Minio, can be deployed on a large variety of hardware.
    And we are looking at a number of hardware specific solutions as well.
    Please post the platforms that you would be interested in seeing Eon run on.

  • Jan_SoubustaJan_Soubusta Vertica Customer ✭

    Hi David ;-)
    I thought the support for having two different communal storage location in the same database cluster.
    Esp. I am interested in the scenario when recent data are in the different S3-like (high-end) storage then archive data (low-end cheap storage) and you can archive from the recent data.

  • jheffnerjheffner Employee

    We may support a single database having multiple communal storage locations in the future, for reasons related to tiering and also due to capacity limitations of some shared storage systems. However, there is benefit in the simplicity of having all durable database storage in one place.

    One method available today for archiving infrequently-accessed data is to export to Parquet or ORC format, which can be queried as an external table. While this requires some additional automation, and queries of archived data may not perform as well, it has the added benefit that metadata is also in cool storage and does not consume catalog memory in Vertica nodes.

  • Jan_SoubustaJan_Soubusta Vertica Customer ✭

    Thanks for explanation, now I understand, why it is not on the roadmap - there is quite suitable workaround solution! ;-)
    Just, please, consider this:
    Swapping partition between two communal storages would be much less resource demanding than exporting to parquet.

  • jheffnerjheffner Employee

    Definitely! Multiple, tiered storage locations is something we've supported for some time in Enterprise Edition, and is something on the list of possible future features for communal storage. (Another motivation is capacity limitations for some shared storage systems.)

  • SruthiASruthiA Administrator

    This feature has been released and is available for use in 24.3

    https://docs.vertica.com/24.3.x/en/new-features/24.3/eon-mode/

This discussion has been closed.