Options

HP has shipped the wrong version of vertica-hcatalogudl.jar with the product.

knows_not_muchknows_not_much
in General Discussion

When I execute this line

 

 

create or replace library public.VHCatalogLib as 
'/opt/vertica/packages/hcat/lib2/vertica-hcatalogudl.jar' 
DEPENDS '/opt/vertica/packages/hcat/lib/*.*' language 'java';

 

 

 

I can see that the library file creates the following objects in vertica

 

 

SELECT * FROM USER_LIBRARY_MANIFEST WHERE lib_name = 'VHCatalogLib';
 schema_name |   lib_name   |      lib_oid      |                     obj_name                     |    obj_type     | arg_types | return_type
-------------+--------------+-------------------+--------------------------------------------------+-----------------+-----------+-------------
 public      | VHCatalogLib | 45035996371219906 | com.vertica.hcatalogudl.WebHCatConfLoaderFactory | Scalar Function |           | Varchar
(1 row)

 

But when you look at the code of the VERTICA HCATALOG CONNECTION install.sql it tries 3 objects

 

 

CREATE SOURCE public.VHCatSource as LANGUAGE 'java'  name 'com.vertica.hcatalogudl.HCatalogSplitsNoOpSourceFactory' library public.VHCatalogLib;
GRANT EXECUTE ON SOURCE public.VHCatSource() TO PUBLIC;
CREATE PARSER public.VHCatParser as LANGUAGE 'java'  name 'com.vertica.hcatalogudl.HCatalogSplitsParserFactory' library public.VHCatalogLib;
GRANT EXECUTE ON PARSER public.VHCatParser() TO PUBLIC;
CREATE FUNCTION get_webhcat_host as LANGUAGE 'java' name 'com.vertica.hcatalogudl.WebHCatConfLoaderFactory' library public.VHCatalogLib;

 

But as you can see clearly.... that the library file never installed the 2 objects this script is looking for

 

  1. com.vertica.hcatalogudl.HCatalogSplitsNoOpSourceFactory
  2. com.vertica.hcatalogudl.HCatalogSplitsParserFactory

This is why the internet is filled with error message 

 

1. Couldn't instantiate class com.vertica.hcatalogudl.HCatalogSplitsNoOpSourceFactory

2. Couldn't instantiate class com.vertica.hcatalogudl.HCatalogSplitsParserFactory

 

The reason is that these objects are not installed by the library but the create source and create parser commands depend on it.

 

The reason why the library doesn't contain it is because we have the wrong vertica-hcatalogudl.jar

 

 

 

Comments

  • Options
    xiaowei_zhuxiaowei_zhu

    Hi, which version of Vertica are you currently running? Can you run uninstall.sql first then run install.sql again to see if this issue can be resolved? I think the problem is for creating VHCatSource and VHCatParser the install.sql is not using CREATE OR REPLACE, which should be fixed. Thanks.

  • Options
    knows_not_muchknows_not_much

    I am running 7.1.2 I have run uninstall.sql more than 10 times.

     

    I am very sure that the vertica-hcataloggudl.jar does not have all the code which is needed by install.sql

     

    because install.sql is loading 3 extensions.. but the gudl.jar only has 1.

     

    Can you please contact the actual vertica team... let them confirm if version 7.1.2 gudl.jar and install.sql

     

     

  • Options
    kmackeenkmackeen

    Another possiblility is that you have not copied the Hadoop/Hive jars to the vertica/packages/hcat/lib directory yet. There is some good information here about how to copy and verify your jar files:

     

    http://my.vertica.com/docs/7.2.x/HTML/index.htm#Authoring/HadoopIntegrationGuide/HCatalogConnector/ConfiguringVerticaForHCatalog.htm%3FTocPath%3DIntegrating%2520with%2520Apache%2520Hadoop|Using%2520the%2520HCatalog%2520Connector|_____5

     

    I found that, if I tried to run install.sql without copying the jar files first, I received a similar message (see below). After copying, it worked as expected. I think this is just a deceptive error message.

     

    release=> \i install.sql
    vsql:install.sql:13: NOTICE 6733:  Did not find any configuration files in dependency paths /opt/vertica/packages/hcat/lib/*.*
    CREATE LIBRARY
    vsql:install.sql:16: ROLLBACK 3399:  Failure in UDx RPC call InvokeGetUdxType(): Error in User Defined Object [VHCatSource], error code: 0
    Couldn't instantiate class com.vertica.hcatalogudl.HCatalogSplitsNoOpSourceFactory

    vsql:install.sql:17: ROLLBACK 2059:  Source with specified name and parameters does not exist: VHCatSource
    vsql:install.sql:18: ROLLBACK 3399:  Failure in UDx RPC call InvokeGetUdxType(): Error in User Defined Object [VHCatParser], error code: 0
    Couldn't instantiate class com.vertica.hcatalogudl.HCatalogSplitsParserFactory

  • Options
    xiaowei_zhuxiaowei_zhu

    Hi, it does seem as Kevin described. You probably did not copy all the required jars to your lib directory. Have you run hcatUtil -v --hcatLibPath=/path/to/jars to verify that you have all the jars in place? If everything is correct, you will find as following query returns:

     

    vsql=> SELECT * FROM USER_LIBRARY_MANIFEST WHERE lib_name = 'VHCatalogLib';
    schema_name | lib_name | lib_oid | obj_name | obj_type | arg_types | return_type
    -------------+--------------+-------------------+---------------------------------------------------------+----------------------+-----------+-------------
    public | VHCatalogLib | 45035996273738062 | com.vertica.hcatalogudl.HCatalogSplitsParserFactory | Load Parser Function | |
    public | VHCatalogLib | 45035996273738062 | com.vertica.hcatalogudl.HCatalogSplitsNoOpSourceFactory | Load Source Function | |
    public | VHCatalogLib | 45035996273738062 | com.vertica.hcatalogudl.WebHCatConfLoaderFactory | Scalar Function | | Varchar
    (3 rows)

     

    Note in your result it only has one of the three, which indicates that HCatalogSplitsParserFactory and HCatalogSplitsNoOpSourceFactory is not correctly installed.

  • Options
    knows_not_muchknows_not_much

    Hadoop jars are not responsible for installing the vertica modules. ONLY the vertica-hcatalogudl.jar contains the code for the 3 modules. and they are deployed like this

     

     

    create or replace library public.VHCatalogLib as 
    '/opt/vertica/packages/hcat/lib2/vertica-hcatalogudl.jar' 
    DEPENDS '/opt/vertica/packages/hcat/lib/*.*' language 'java';

      

    Anyway I have copied all the jars and below is the output of the hcatUtil verify. It very clearly says that it has everything it needs.

     

     

     

    Summary
    -------
    Found 27 out of 27 required JARs
    Found 5 out of 5 optional Hive SerDe related JARs
    Found 2 out of 2 optional native libraries, 5 files in total
    Looking at [/opt/vertica/packages/hcat] for configuration files
    Found configuration file [cli.xml]
    Found configuration file [data-manipulation-statements.xml]
    Found configuration file [joins.xml]
    Found configuration file [reflect.xml]
    Found configuration file [index.xml]
    Found configuration file [hdfs-site.xml]
    Found configuration file [yarn-site.xml]
    Found configuration file [ssl-client.xml]
    Found configuration file [core-site.xml]
    Found configuration file [project.xml]
    Found configuration file [mapred-site.xml]
    Found configuration file [var_substitution.xml]
    Found configuration file [hive-site.xml]
    Found configuration file [working_with_bucketed_tables.xml]

     

    By looking at the output above, can you still say that I have not copied the hadoop jars and configuration files correctly?

     

     Once again, Hadoop Jars have nothing to do with the 3 modules. these are purely HP vertica stuff and therefore must be deployed by the vertica-hcatalogudl.jar. the problem is that the version of vertica-hcatalogudl.jar which is shipped with vertica 7.1.2 does not contain the 3 modules.

     

  • Options
    xiaowei_zhuxiaowei_zhu

    Those jars copied from Hadoop are required to install HCatalog connector library modules. As kevin has shown you above, without those jars you will not be able to install successfully.

     

    I noticed that your vertica-hcatalogudl.jar is under lib2 directory and the dependency jars are set in lib/ directory. Where are the copied jars exactly? I could not tell since you only did hcatUtil -v under hcat/ directory.

     

    Also, could you check UDxFencedProcessesJava.log and see what is the detailed error message?

  • Options
    knows_not_muchknows_not_much

    Here is the output of running hcatUtil

     

    ./hcatUtil -v --hcatLibPath=/opt/vertica/packages/hcat
    Looking at [/opt/vertica/packages/hcat] for Jars

    SUCCESS: Found all required Jars.

    SUCCESS: Found all optional Hive SerDe related Jars.
    SUCCESS: Found all native libraries


    Summary
    -------
    Found 27 out of 27 required JARs
    Found 5 out of 5 optional Hive SerDe related JARs
    Found 2 out of 2 optional native libraries, 5 files in total
    Looking at [/opt/vertica/packages/hcat] for configuration files
    Found configuration file [cli.xml]
    Found configuration file [data-manipulation-statements.xml]
    Found configuration file [joins.xml]
    Found configuration file [reflect.xml]
    Found configuration file [index.xml]
    Found configuration file [hdfs-site.xml]
    Found configuration file [yarn-site.xml]
    Found configuration file [ssl-client.xml]
    Found configuration file [core-site.xml]
    Found configuration file [project.xml]
    Found configuration file [mapred-site.xml]
    Found configuration file [var_substitution.xml]
    Found configuration file [hive-site.xml]
    Found configuration file [working_with_bucketed_tables.xml]

    Now do 

     

    mydb=> create or replace library public.VHCatalogLib 
    as '/opt/vertica/packages/hcat/lib/vertica-hcatalogudl.jar' 
    DEPENDS '/opt/vertica/packages/hcat/lib/*' language 'java';

    NOTICE 6564:  Found Hadoop configuration files in dependency paths. 
    If any of the configuration files is changed, 
    please re-install HCatalog connector library with modified configuration files

    CREATE LIBRARY
    mydb=>
    mydb=> SELECT * FROM USER_LIBRARY_MANIFEST WHERE lib_name = 'VHCatalogLib';
     schema_name |   lib_name   |      lib_oid      |                     obj_name                     |    obj_type     | arg_types | return_type
    -------------+--------------+-------------------+--------------------------------------------------+-----------------+-----------+-------------
     public      | VHCatalogLib | 45035996371250348 | com.vertica.hcatalogudl.WebHCatConfLoaderFactory | Scalar Function |           | Varchar
    (1 row)

     

    You see ONLY 1 module is loaded. where are the other 2. Even though I have copied everything from Hadoop (as verified above by hcatUtil) still only 1 module is loaded

     

    PLEASE NOTE THAT THE OUTPUT FOR VERSION 7.2 MIGHT BE DIFFERENT. SO YOU MUST ONLY DO TESTING WITH 7,1.2

     

    Can you do a ls on your lib directory and take the details for vertica-hcatalogudl.jar and put the output in your response?

     

    mine is 

     

    -rwxrwxr-x 1 root root 43596 Jul  3 00:55 vertica-hcatalogudl.jar

     

    I also opened the log file  sudo -u vertica vi /db/vertica/mydb/v_mydb_node0001_catalog/UDxLogs/UDxFencedProcessesJava.log

     

    and found these 3 lines for today

     

    2015-11-30 19:57:27.940 [Java-9135] 0x0a Loading VerticaClassLoader
    2015-11-30 19:57:49.365 [Java-9135] 0x0a Loading VerticaClassLoader
    2015-11-30 19:57:56.455 [Java-9135] 0x0a Loading VerticaClassLoader
    2015-11-30 20:18:21.677 [Java-9135] 0x0a Loading VerticaClassLoader

     

  • Options
    xiaowei_zhuxiaowei_zhu
    Please do hcatUtil -v with path ends with hcat/lib instead of just hcat/. And why there is a lib2 directory under hcat/?
  • Options
    knows_not_muchknows_not_much

    There is no lib2. see my last post above. (just before your reply). There is no lib2

     

    here is the output of hcatUtil with the lib in path

     

    $ ./hcatUtil -v --hcatLibPath=/opt/vertica/packages/hcat/lib/
    Looking at [/opt/vertica/packages/hcat/lib/] for Jars

    SUCCESS: Found all required Jars.

    SUCCESS: Found all optional Hive SerDe related Jars.
    SUCCESS: Found all native libraries


    Summary
    -------
    Found 27 out of 27 required JARs
    Found 5 out of 5 optional Hive SerDe related JARs
    Found 2 out of 2 optional native libraries, 5 files in total
    Looking at [/opt/vertica/packages/hcat/lib/] for configuration files
    Found configuration file [cli.xml]
    Found configuration file [data-manipulation-statements.xml]
    Found configuration file [joins.xml]
    Found configuration file [reflect.xml]
    Found configuration file [index.xml]
    Found configuration file [hdfs-site.xml]
    Found configuration file [yarn-site.xml]
    Found configuration file [ssl-client.xml]
    Found configuration file [core-site.xml]
    Found configuration file [project.xml]
    Found configuration file [mapred-site.xml]
    Found configuration file [var_substitution.xml]
    Found configuration file [hive-site.xml]
    Found configuration file [working_with_bucketed_tables.xml]

     

  • Options
    xiaowei_zhuxiaowei_zhu

    I see. Thanks for your updates. Do note that these jars are required to install those modules.

     

    Another important question is what Hadoop distro did you copy the jars from? With Vertica 7.1.2 we dropped support for some older hadoop distros.

     

    Also, are you sure the UDx log is the right one? The timestamps look wired. Usually if install fails there will be error messages show up in the UDx java log.

  • Options
    knows_not_muchknows_not_much

    I am using cloudera 5.4.1

     

    I did a search for UDXFencedProcessesJava.log

     

    [AB@machine-verticatest01 ~]$ sudo find / -name UDxFencedProcessesJava.log
    [sudo] password for AB:
    /db/vertica/mydb/v_mydb_node0001_catalog/UDxLogs/UDxFencedProcessesJava.log

     

    I am attaching the file with this post.

  • Options
    xiaowei_zhuxiaowei_zhu

    From log I found these error messages, but they are some days ago:

     

    2015-11-25 04:16:21.953 [Java-31635] 0x0a com.vertica.sdk.UdfException: Couldn't instantiate class com.vertica.hcatalogudl.HCatalogSplitsNoOpSourceFactory
    at com.vertica.udxfence.UDxExecContext.findUDXFactoryInLibrary(UDxExecContext.java:386)
    at com.vertica.udxfence.UDxExecContext.setupLanguageContext(UDxExecContext.java:459)
    at com.vertica.udxfence.UDxExecContext.setupContext(UDxExecContext.java:528)
    at com.vertica.udxfence.UDxExecContext.run(UDxExecContext.java:219)
    at java.lang.Thread.run(Thread.java:745)
    Caused by: java.lang.NoClassDefFoundError: org/apache/hcatalog/common/HCatException
    at java.lang.Class.getDeclaredConstructors0(Native Method)
    at java.lang.Class.privateGetDeclaredConstructors(Class.java:2585)
    at java.lang.Class.getConstructor0(Class.java:2885)
    at java.lang.Class.newInstance(Class.java:350)
    at com.vertica.udxfence.UDxExecContext.findUDXFactoryInLibrary(UDxExecContext.java:374)
    ... 4 more
    Caused by: java.lang.ClassNotFoundException: org.apache.hcatalog.common.HCatException
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 9 more

     

    The root cause is the HCatalog java udl is looking for "org.apache.hcatalog.common.HCatException", which is the class path before Hive 0.14.

     

    This is the old implementation of HCatalog connector. Which specific vertica version are you using? 7.1.2-x. What is the x? Can you try to upgrade to 7.1.2-3 or later and see that can resolve the issue? Check the Vertica hotfix schedule https://my.vertica.com/docs/ReleaseNotes/7.1.x/HP_Vertica_7.1.x_Release_Notes.htm and look for VER-39674 in 7.1.2-3 release note.

  • Options
    knows_not_muchknows_not_much

    It seems I am running 7.1.2-2

     

    mydb=> select version();
    version
    ------------------------------------
    Vertica Analytic Database v7.1.2-2
    (1 row)
    mydb=>

     

     

    Let me check with the team to see if we can upgrade to 7.1.2-3 

     

     

  • Options
    xiaowei_zhuxiaowei_zhu

    OK. Yes you have to upgrade to 7.1.2-3 or higher. And officially, Vertica 7.1.2 only supports CDH5.2 and CDH5.3. 7.2 adds support for CDH5.4.

  • Options
    knows_not_muchknows_not_much

    Thank you so much for your help. it makes sense to me now.

     

    I will work with my team to ensure that we use the right version of vertica.

     

    I appriciate you taking the time throughout this day to reach the root cause of this problem. I appriciate your work and thank you for it.

     

    I will revert on this thread once we have the right version. This might take some time... but I will eventually come back.

     

    Thank you so much.

  • Options
    xiaowei_zhuxiaowei_zhu

    No problem. Let me know how things go. Thanks.

  • Options
    knows_not_muchknows_not_much

    We upgraded our Vertica instance to Version 7.2 and after the upgrade the installation was succesful.

     

    vertica=> \i install.sql
    vsql:install.sql:13: NOTICE 6792: Found Hadoop configuration files in dependency paths. If any of the configuration files is changed, please re-install HCatalog connector library with modified configuration files
    CREATE LIBRARY
    CREATE SOURCE FUNCTION
    GRANT PRIVILEGE
    CREATE PARSER FUNCTION
    GRANT PRIVILEGE
    CREATE FUNCTION

     

    Thank you so much for your help!!!

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Format
SpoilerCodeQuote
Heading 2Heading 1
Emoji
:):D:(;):/:o:s:p:'(:|B):#:*<3o:)>:)
Url
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file
Home • General DiscussionComment As ...