Options
Vertica 12 spark connector issue while saving dataframe
Hi Team,
Can you please suggest about below error while using https://repo1.maven.org/maven2/com/vertica/spark/vertica-spark/3.3.0/ connector . We are running jobs on google dataproc. Earlier with vertica 10 and connector vertica-spark2.4-3.0_scala2.12.jar it worked fine.
With respect to documentation of vertica 12 we have made changes and using specified datasource but still its not able to find that datasource specified.
spark 3.3.0
scala 2.12.18
Hadoop 3.3.6
Error message :
Failed to find data source: com.vertica.spark.datasource.VerticaSource. Please find packages at
https://spark.apache.org/third-party-projects.html
0
Answers
The README on GitHub may have more current setup details: https://github.com/vertica/spark-connector
Be sure Dataproc references the latest Maven artifact as a dependency. We would probably need a sample config and job to test.