vertica jdbc not working for kerberos enabled hdfs copy

i have kerberos enabled hdfs cluster and i also have enabled kerberos on my vertica cluster. If I try to copy some files from hdfs to vertica using vsql. Eg., 
vsql -k <servicename> -K <principalname> -h <hostname> -U <username> -t -A -c "copy <tablename> source hdfs(url='<webhdfsurl>', user.name='<username>')"
 it works. But when I try to do the same thing using jdbc connection, it gives 401 error while connecting to webhdfs. Is jdbc not capable enough or am I doing anything wrong.
PS: am passing kerberosservicename and principalname in jdbc and am able to connect to vertica. Am also able to execute select command and view results. The problem happens only when I try to copy from hdfs.
Any help will be greatly appreciated.

Comments

  • Hi,

    Could you please share a portion of logs with us.


    Regards

    Bhawana
  • Hi,
    here is whats happening : 
    1 : Success while doing vsql
    >$ vsql -k $SERVICE_NAME -K $PRINCIPAL_NAME -h $HOST_NAME -U $USER_NAME -t -A -c "copy $TABLE_NAME source Hdfs(url='$WEB_HDFS_URL', username='$USER_NAME')" -x

    Rows Loaded|$NUM_ROWS

    2 : Failure while doing jdbc call

    >$ java -cp ../lib/vertica-jdbc.jar:. TestCopy

    Exception in thread "main" java.sql.SQLNonTransientException: [Vertica][VJDBC](3399) ERROR: Failure in UDx RPC call InvokePlanUDL(): Error calling planUDL() in User Defined Object [Hdfs] at [src/Hdfs.cpp:307], error code: 0, message: [The requested URL returned error: 401. URL: $WEB_HDFS_URL]

    at com.vertica.util.ServerErrorData.buildException(Unknown Source)

    at com.vertica.dataengine.VQueryExecutor.executeSimpleProtocol(Unknown Source)

    at com.vertica.dataengine.VQueryExecutor.execute(Unknown Source)

    at com.vertica.jdbc.common.SStatement.executeNoParams(Unknown Source)

    at com.vertica.jdbc.common.SStatement.execute(Unknown Source)

    at TestCopy.main(TestCopy.java:28)

    Caused by: com.vertica.support.exceptions.NonTransientException: [Vertica][VJDBC](3399) ERROR: Failure in UDx RPC call InvokePlanUDL(): Error calling planUDL() in User Defined Object [Hdfs] at [src/Hdfs.cpp:307], error code: 0, message: [The requested URL returned error: 401. URL: $WEB_HDFS_URL]

    ... 6 more


    Java Code

    public class TestCopy {


      public static void main(String[] args) throws SQLException, ClassNotFoundException {

        Class.forName("com.vertica.jdbc.Driver");

        Properties props = new Properties();

        props.setProperty("user", "$USER_NAME");

        props.setProperty("KerberosServiceName", "$SERVICE_NAME");

        props.setProperty("KerberosHostName", "$PRINCIPAL_NAME");

        props.setProperty("JAASConfigName", "verticajdbc");

        VerticaConnection conn = (VerticaConnection) DriverManager.getConnection(

            "$JDBC_STRING", props);

        Statement stmt = conn.createStatement();

        boolean res = stmt.execute("copy $TABLE_NAME source Hdfs(url='$WEB_HDFS_URL', username='$USER_NAME')");

        System.out.println(res);

        stmt.close();

        conn.close();

      }

    }


    Do let me know if you need more info. Whatever error messages I provided are obtained as console output.

    ~Ashwini.
  • Prasanta_PalPrasanta_Pal - Select Field - Employee
    define all $<variable_name> in java code, could be like,

    string USER_NAME = "dbadmin";

  • Hi,
    All variables have been properly defined in the code. Has it worked for anybody? I mean, has anyone tried and succeeded in copying data from kerberos enabled  hdfs to vertica using jdbc??
  • Hi All,

     

    Can you please let me know if this work for someone? I tested the same code it is not working so i am working if the HDFS Copy can work in a kerberized environment over JDBC.

     

    Thanks for your help,

    Abdoul

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file