Can you provide guidance on how to accomplish this?

mackpatrickmackpatrick Community Edition User

Hi, I have a significant amount of data stored in Arcsight Logger, which I haven't yet integrated with Vertica. I am interested in automatically ingesting the data into Vertica using a script. Can you provide guidance on how to accomplish this? Additionally, if scripts are required for this process, could you please advise me on how to create a script for any type of data? Thank you.

Answers

  • SruthiASruthiA Vertica Employee Administrator

    I think you can use kafka integration to automatically ingest data into vertica.

    https://docs.vertica.com/12.0.x/en/kafka-integration/

  • moshegmosheg Vertica Employee Administrator

    In addition, lacat is old and simple utility that exports CEF records from a Logger archive file.
    It prints them to stdout by design, allowing the user to redirect them to a file or pipe them into csv or redirect it all to a Vertica COPY statement.
    It is written in python (targetting 2.7 and 3.5) using only the standard python library.
    See: https://github.com/hpsec/lacat

  • SergeBSergeB - Select Field - Employee

    As mentioned, Arcsight has a built-in integration with Vertica via Kafka. Vertica also provides a built-in CEF flex parser that might match your needs. https://docs.vertica.com/12.0.x/en/sql-reference/statements/copy/parsers/fcefparser/

  • David1cDavid1c Community Edition User
    edited May 2023

    Determine the format and location of the data in ArcSight Logger. You'll need to know the file format (e.g., CSV, JSON) and where the files are stored on disk or in the ArcSight Logger database. Write a script to extract the data from ArcSight Logger and transform it into a format that can be ingested into Vertica. This script can be written in any scripting language that supports connectivity to both ArcSight Logger and Vertica, such as Python or Bash.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file