Options
Eon mode/Locate commonly used data at Depot.
Kaito
Employee
My customer wants to locate commonly used data at Depot always. Especially replica/unsegmented dimension tables.
In my understanding, Depot stores data with First-In-First-Out rule. Do we have any workaround to stick these data at Depot?
Thanks in advance,
0
Comments
Hi Kaito,
The Depot is not managed with a FIFO policy - it is managed with a "Most Recently Used" policy. This means that commonly used data will be in the Depot, subject to the size of the Depot. If the Depot is sized to accommodate all commonly used data then it is unlikely that commonly used data will get evicted.
One way to ensure that commonly used data is not evicted from a Depot is to ensure that all queries access the same data. If the user wants to run other queries that will access uncommonly used data, they should do so in a different subcluster.
There is one caveat - newly loaded data might evict files even if data loading is done in a different subcluster because the current assumption is that all nodes are interested in the latest data. We plan to add an option that would not peer-fill recently loaded data - this option will help protect commonly access files from getting evicted even if they are in another subcluster.
Dave