Does Vertica support parquet files???

Does vertica support parquet files???

If yes could you pls let me know which version of vertica supports parquet.

Comments

  • SruthiASruthiA Vertica Employee Employee

    Hi,

     

        It does support parquet files. Using the HCatalog Connector Interface you can read all file formats supported by Hadoop. It is supported in 7.1.2 vertica version. For more information go through the following URL

     

    http://my.vertica.com/docs/7.1.x/HTML/index.htm#Authoring/SQLOnHadoop/ConnectingVerticaToHDFS.htm?Highlight=parquet

     

     

    Sruthi

  • Hi Sruthi,

     

                Thanks for the reply. I am trying to read a parquet file and convert them to dataframes using spark and then load that dataframes into vertica database as of the link below 

     

    http://www.sparkexpert.com/2015/04/17/save-apache-spark-dataframe-to-database/

     

    As per this link i tried parquet file loading to mysql & it worked fine.

     

    But when i tried to load the same parquet file into vertica i am facing below exception

     

    Exception in thread "main" java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](5108) ERROR: Type "TEXT" does not exist
    at com.vertica.util.ServerErrorData.buildException(Unknown Source)
    at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
    at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
    at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
    at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
    at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown Source)
    at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown Source)
    at com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
    at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
    at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown Source)
    at org.apache.spark.sql.DataFrameWriter.jdbc(DataFrameWriter.scala:275)
    at org.apache.spark.sql.DataFrame.insertIntoJDBC(DataFrame.scala:1630)
    at com.sparkread.SparkVertica.JdbctoVertica.main(JdbctoVertica.java:37)
    Caused by: com.vertica.support.exceptions.SyntaxErrorException: [Vertica][VJDBC](5108) ERROR: Type "TEXT" does not exist
    ... 13 more
    15/10/14 22:46:33 INFO SparkContext: Invoking stop() from shutdown hook
    15/10/14 22:46:33 INFO SparkUI: Stopped Spark web UI at http://192.168.75.132:4040
    15/10/14 22:46:33 INFO DAGScheduler: Stopping DAGScheduler
    15/10/14 22:46:33 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
    15/10/14 22:46:33 INFO MemoryStore: MemoryStore cleared
    15/10/14 22:46:33 INFO BlockManager: BlockManager stopped
    15/10/14 22:46:33 INFO BlockManagerMaster: BlockManagerMaster stopped
    15/10/14 22:46:33 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
    15/10/14 22:46:33 INFO SparkContext: Successfully stopped SparkContext
    15/10/14 22:46:33 INFO ShutdownHookManager: Shutdown hook called
    15/10/14 22:46:33 INFO ShutdownHookManager: Deleting directory /tmp/spark-738073f0-cc61-40e5-a74a-97b003611deb

     

    Could you please help me in fixing the issue. Thanks in advance....

     

     

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file