Is there a way to set a KafkaSource end condition to load data until there are no new data arrivals

olivertwsitolivertwsit Community Edition User

My goal is to manually copy data into Vertica using a KafkaSource. Apparently, I can define an end condition so that no new data arrives until a timeout period expires.
What is the timeout period and how do I set it?
With a KafkaSource, I can COPY data into Vertica over a defined period of time. Would it be possible to retrieve the Kafka message offset where this COPY statement ended so that the next round begins exactly there?


Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file