Near Real Time load into Vertica

MraoMrao
in General Discussion
I have data generated by monitoring devices and I want to load the data every 5 min. Basically, it is a trickle feed which I want to load on near real time basis.  

I am looking a best design approach to load and store the data.  Pls let me know if you have done any real time implementations/any white paper/case study.

Comments

  • Ben_VandiverBen_Vandiver Employee
    Our customer, Visible Measures, gave a great talk at our Big Data Conference 2013 on how they load >500 million events per day into a Vertica database with a 2 second latency between the event arriving at their data center and it being queryable in Vertica.  They use a number of tools, including Storm and Kestrel to stage the data into Vertica.  I dug around looking for a reference and couldn't find one.
  • MraoMrao
    Thanks for the information.  I at least now know that it was implemented at a large volume.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Format
SpoilerCodeQuote
Heading 2Heading 1
Emoji
:):D:(;):/:o:s:p:'(:|B):#:*<3o:)>:)
Url
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file
Home • General DiscussionComment As ...