Facing issue in Spark + HDFS to load data into Vertica tables.
We use Spark + HDFS to load data into Vertica tables. Here tables are auto generated using HDFS connector provided by Vertica. We have seen the auto generated tables are not optimized - It has Float data type, a fixed varchar length (we provide max 5000) and an un-optimized projection, which is created using default rules of Vertica.
Our need is
Define data types using HDFS Connector smartly (i.e. Do not use float and do not give default width to string columns)
how to control the data types dynamically using the same.
Please help me ...