Tuple Mover falls behind, resulting in constant growth of ROS containers
Since upgrading from 5.0.8 to 6.1.0 and 6.1.1 we are seeing constant growth in ROS containers after a period of uptime since a DB or node restart. I confirm this by querying the projection_storage table for ros_count. Initially, ROS container volumes are maintained, but over time, my queries to the projection_storage table get slower and slower and ROS volumes grow. The only resolution appears to be to run manual mergeouts using the tuple mover. This reduces the ROS count and also the query duration against the projection_storage table. However, this is only a temporary measure, as ROS containers continue to grow even after a manual mergeout. We did not witness this issue in v5.0.8. The only permanent solution is to restart vertica, upon a restart, the tuple mover reduces the ROS count back to normal levels. Are there known issues with the Tuple Mover lagging behind and causing runaway ROS growth? Could this be due to poor performance of queries against the projection_storage table as the number of ROS containers grows?