We're Moving!

The Vertica Forum is moving to a new OpenText Analytics Database (Vertica) Community.

Join us there to post discussion topics, learn about

product releases, share tips, access the blog, and much more.

Create My New Community Account Now


vertica jdbc not working for kerberos enabled hdfs copy — Vertica Forum

vertica jdbc not working for kerberos enabled hdfs copy

i have kerberos enabled hdfs cluster and i also have enabled kerberos on my vertica cluster. If I try to copy some files from hdfs to vertica using vsql. Eg., 
vsql -k <servicename> -K <principalname> -h <hostname> -U <username> -t -A -c "copy <tablename> source hdfs(url='<webhdfsurl>', user.name='<username>')"
 it works. But when I try to do the same thing using jdbc connection, it gives 401 error while connecting to webhdfs. Is jdbc not capable enough or am I doing anything wrong.
PS: am passing kerberosservicename and principalname in jdbc and am able to connect to vertica. Am also able to execute select command and view results. The problem happens only when I try to copy from hdfs.
Any help will be greatly appreciated.

Comments

  • Hi,

    Could you please share a portion of logs with us.


    Regards

    Bhawana
  • Hi,
    here is whats happening : 
    1 : Success while doing vsql
    >$ vsql -k $SERVICE_NAME -K $PRINCIPAL_NAME -h $HOST_NAME -U $USER_NAME -t -A -c "copy $TABLE_NAME source Hdfs(url='$WEB_HDFS_URL', username='$USER_NAME')" -x

    Rows Loaded|$NUM_ROWS

    2 : Failure while doing jdbc call

    >$ java -cp ../lib/vertica-jdbc.jar:. TestCopy

    Exception in thread "main" java.sql.SQLNonTransientException: [Vertica][VJDBC](3399) ERROR: Failure in UDx RPC call InvokePlanUDL(): Error calling planUDL() in User Defined Object [Hdfs] at [src/Hdfs.cpp:307], error code: 0, message: [The requested URL returned error: 401. URL: $WEB_HDFS_URL]

    at com.vertica.util.ServerErrorData.buildException(Unknown Source)

    at com.vertica.dataengine.VQueryExecutor.executeSimpleProtocol(Unknown Source)

    at com.vertica.dataengine.VQueryExecutor.execute(Unknown Source)

    at com.vertica.jdbc.common.SStatement.executeNoParams(Unknown Source)

    at com.vertica.jdbc.common.SStatement.execute(Unknown Source)

    at TestCopy.main(TestCopy.java:28)

    Caused by: com.vertica.support.exceptions.NonTransientException: [Vertica][VJDBC](3399) ERROR: Failure in UDx RPC call InvokePlanUDL(): Error calling planUDL() in User Defined Object [Hdfs] at [src/Hdfs.cpp:307], error code: 0, message: [The requested URL returned error: 401. URL: $WEB_HDFS_URL]

    ... 6 more


    Java Code

    public class TestCopy {


      public static void main(String[] args) throws SQLException, ClassNotFoundException {

        Class.forName("com.vertica.jdbc.Driver");

        Properties props = new Properties();

        props.setProperty("user", "$USER_NAME");

        props.setProperty("KerberosServiceName", "$SERVICE_NAME");

        props.setProperty("KerberosHostName", "$PRINCIPAL_NAME");

        props.setProperty("JAASConfigName", "verticajdbc");

        VerticaConnection conn = (VerticaConnection) DriverManager.getConnection(

            "$JDBC_STRING", props);

        Statement stmt = conn.createStatement();

        boolean res = stmt.execute("copy $TABLE_NAME source Hdfs(url='$WEB_HDFS_URL', username='$USER_NAME')");

        System.out.println(res);

        stmt.close();

        conn.close();

      }

    }


    Do let me know if you need more info. Whatever error messages I provided are obtained as console output.

    ~Ashwini.
  • Prasanta_PalPrasanta_Pal - Select Field - Employee
    define all $<variable_name> in java code, could be like,

    string USER_NAME = "dbadmin";

  • Hi,
    All variables have been properly defined in the code. Has it worked for anybody? I mean, has anyone tried and succeeded in copying data from kerberos enabled  hdfs to vertica using jdbc??
  • Hi All,

     

    Can you please let me know if this work for someone? I tested the same code it is not working so i am working if the HDFS Copy can work in a kerberized environment over JDBC.

     

    Thanks for your help,

    Abdoul

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file