Does Vertica support parquet files???

vivek789

Does vertica support parquet files???

If yes could you pls let me know which version of vertica supports parquet.

SruthiA

Hi,

 

    It does support parquet files. Using the HCatalog Connector Interface you can read all file formats supported by Hadoop. It is supported in 7.1.2 vertica version. For more information go through the following URL

 

http://my.vertica.com/docs/7.1.x/HTML/index.htm#Authoring/SQLOnHadoop/ConnectingVerticaToHDFS.htm?Highlight=parquet

 

 

Sruthi

vivek789

Hi Sruthi,

 

            Thanks for the reply. I am trying to read a parquet file and convert them to dataframes using spark and then load that dataframes into vertica database as of the link below 

 

http://www.sparkexpert.com/2015/04/17/save-apache-spark-dataframe-to-database/

 

As per this link i tried parquet file loading to mysql & it worked fine.

 

But when i tried to load the same parquet file into vertica i am facing below exception

 

Exception in thread "main" java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](5108) ERROR: Type "TEXT" does not exist
at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown Source)
at org.apache.spark.sql.DataFrameWriter.jdbc(DataFrameWriter.scala:275)
at org.apache.spark.sql.DataFrame.insertIntoJDBC(DataFrame.scala:1630)
at com.sparkread.SparkVertica.JdbctoVertica.main(JdbctoVertica.java:37)
Caused by: com.vertica.support.exceptions.SyntaxErrorException: [Vertica][VJDBC](5108) ERROR: Type "TEXT" does not exist
... 13 more
15/10/14 22:46:33 INFO SparkContext: Invoking stop() from shutdown hook
15/10/14 22:46:33 INFO SparkUI: Stopped Spark web UI at http://192.168.75.132:4040
15/10/14 22:46:33 INFO DAGScheduler: Stopping DAGScheduler
15/10/14 22:46:33 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
15/10/14 22:46:33 INFO MemoryStore: MemoryStore cleared
15/10/14 22:46:33 INFO BlockManager: BlockManager stopped
15/10/14 22:46:33 INFO BlockManagerMaster: BlockManagerMaster stopped
15/10/14 22:46:33 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
15/10/14 22:46:33 INFO SparkContext: Successfully stopped SparkContext
15/10/14 22:46:33 INFO ShutdownHookManager: Shutdown hook called
15/10/14 22:46:33 INFO ShutdownHookManager: Deleting directory /tmp/spark-738073f0-cc61-40e5-a74a-97b003611deb

 

Could you please help me in fixing the issue. Thanks in advance....