Building FilterFunctions in Vertica 9.1.0-1 Fails

Environment:
Vertica BYOL, Amazon Linux 2.0 Version: 9.1.0-1

I was able to build the filter functions in Vertica 9.0 (though it was on CentOS), but when I attempt in Vertica 9.1.0-1 I am getting the following:

cd /opt/vertica/sdk/examples
make

g++ -I /opt/vertica/sdk/include -I HelperLibraries -g -Wall -Wno-unused-value -shared -fPIC -o /opt/vertica/sdk/examples/build/ScalarFunctions.so ScalarFunctions/.cpp /opt/vertica/sdk/include/Vertica.cpp
In file included from /opt/vertica/sdk/include/Vertica.h:75:0,
from ScalarFunctions/Add2Ints.cpp:7:
/opt/vertica/sdk/include/VerticaUDx.h: In member function ‘void Vertica::UDXObject::cancelUDX(Vertica::ServerInterface&)’:
/opt/vertica/sdk/include/VerticaUDx.h:2305:57: error: operand type ‘bool
’ is incompatible with argument 1 of ‘__sync_fetch_and_or’
if (!__sync_fetch_and_or(&(this->canceled), true)) {
^
In file included from /opt/vertica/sdk/include/Vertica.h:75:0,
from ScalarFunctions/AddAnyInts.cpp:9:
/opt/vertica/sdk/include/VerticaUDx.h: In member function ‘void Vertica::UDXObject::cancelUDX(Vertica::ServerInterface&)’:
/opt/vertica/sdk/include/VerticaUDx.h:2305:57: error: operand type ‘bool*’ is incompatible with argument 1 of ‘__sync_fetch_and_or’
if (!__sync_fetch_and_or(&(this->canceled), true)) {
^

Tagged:

Comments

  • So its complaining about the first argument type for __sync_fetch_and_or(). This code is in the VerticaUDx.h:

    void cancelUDX(ServerInterface &srvInterface) {
         if (!__sync_fetch_and_or(&(this->canceled), true)) {                                                                                                                          
         cancel(srvInterface);
      }
    }
    

    I convert the first arg to an integer and it compiles:

    void cancelUDX(ServerInterface &srvInterface) {
      int local_canceled = this->canceled !=0 ? 1 : 0;
      if (!__sync_fetch_and_or(&local_canceled, true)) {
         cancel(srvInterface);
      }
    }
    

    Don't know if this is what is expected at runtime.

  • Sergey_Cherepan_1Sergey_Cherepan_1 ✭✭✭
    edited March 2019

    That would be a very dumb fix (sorry for words)
    Goal of __sync_fetch_and_or is to extract atomic value of cancel and set it to true. Use of atomic allow not to acquire lock.
    Fix circumvent atomic operation, canceled is not being set etc. Effectively, fix is breaking code.

    gcc __sync_fetch_and_or is an extension of Intel API and accept 1,2,4 and 8 bytes integer parameter. Version of gcc >= 7 (???) do not convert bool into single-byte integral by default, and code error out on compilation.
    Correct fix would be

        if (!__sync_fetch_and_or(reinterpret_cast<unsigned char*>(&(this->canceled)), reinterpret_cast<unsigned char*>(true))) {
    

    Not to mention, Vertica developers are not testing their code against newer compilers. Vertica docs 9.2 say "need g++" to compile UDx, but then it says "some distribution have not compatible g++" and requiring to install g++ v4.8.
    Vertica is still on very old and now looking odd g++ 4.8. That is more than 10 years old.

  • edited May 2019

    Hi Sergey, Tried to use your fix it worked for overcoming the above error. We have upgraded to 9.2 and installing extension packages.

    But running into this error -

    vsql:/opt/vertica/packages/Vertica-Extension-Packages-master/strings_package/ddl/install.sql:7: ROLLBACK 6850: Library [StringsLib] built with incompatible SDK version [8.1.1]. Please rebuild with SDK version [9.2.0] and recreate the library
    vsql:/opt/vertica/packages/Vertica-Extension-Packages-master/strings_package/ddl/install.sql:8: ROLLBACK 6850: Library [StringsLib] built with incompatible SDK version [8.1.1]. Please rebuild with SDK version [9.2.0] and recreate the library
    vsql:/opt/vertica/packages/Vertica-Extension-Packages-master/strings_package/ddl/install.sql:9: ROLLBACK 6850: Library [StringsLib] built with incompatible SDK version [8.1.1]. Please rebuild with SDK

    Any pointers? Thanks!

  • Looking forward for some responses on my question above. Thanks!

  • skeswaniskeswani - Select Field - Employee

    I think the right way to avoid this error is to use gcc 4.8 as shown here
    if (!__sync_fetch_and_or(&(this->canceled), true)) {

    https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/ExtendingVertica/UDx/DevEnvironment.htm

    instead of editing the code, you can use something like this where you select the correct version of g++
    ! /usr/bin/g++-4.8 -D HAVE_LONG_LONG_INT_64 -g -Wall -Wno-unused-value -shared -fPIC -I /opt/vertica/sdk/include/ -o /tmp/Reader.so ./Reader.cpp /opt/vertica/sdk/include/Vertica.cpp

  • hink the right way to avoid this error is to use gcc 4.8 as shown here
    if (!__sync_fetch_and_or(&(this->canceled), true)) {

    https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/ExtendingVertica/UDx/DevEnvironment.htm

    instead of editing the code, you can use something like this where you select the correct version of g++
    ! /usr/bin/g++-4.8 -D HAVE_LONG_LONG_INT_64 -g -Wall -Wno-unused-value -shared -fPIC -I /opt/vertica/sdk/include/ -o /tmp/Reader.so ./Reader.cpp /opt/vertica/sdk/include/Vertica.cpp

    that's true indeed... worked for me ! :)

  • edited February 2020

    Hi Sergey, Tried to use your fix it worked for overcoming the above error. We have upgraded to 9.2 and installing extension packages.

    But running into this error -

    vsql:/opt/vertica/packages/Vertica-Extension-Packages-master/strings_package/ddl/install.sql:7: ROLLBACK 6850: Library [StringsLib] built with incompatible SDK version [8.1.1]. Please rebuild with SDK version [9.2.0] and recreate the library
    vsql:/opt/vertica/packages/Vertica-Extension-Packages-master/strings_package/ddl/install.sql:8: ROLLBACK 6850: Library built with incompatible SDK version [8.1.1]. Please rebuild with SDK version [9.2.0] and recreate the library
    vsql:/opt/vertica/packages/Vertica-Extension-Packages-master/strings_package/ddl/install.sql:9: ROLLBACK 6850: Library [StringsLib] built with incompatible SDK version [8.1.1]. Please rebuild with SDK

    Any pointers?
    it makes sense, doesn't it ?

  • SruthiASruthiA Vertica Employee Administrator

    please reinstall packages using the following command

    admintools -t install_package -d db-name -p password -P all --force-reinstall

    and then re create all your existing libraries.

  • lovelashlovelash
    edited December 2019

    I think the right way to avoid this error is to use gcc 4.8 as shown here
    if (!__sync_fetch_and_or(&(this->canceled), true)) {

    https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/ExtendingVertica/UDx/DevEnvironment.htm

    instead of editing the code, you can use something like this where you select the correct version of g++
    ! /usr/bin/g++-4.8 -D HAVE_LONG_LONG_INT_64 -g -Wall -Wno-unused-value -shared -fPIC -I /opt/vertica/sdk/include/ -o /tmp/Reader.so ./Reader.cpp /opt/vertica/sdk/include/Vertica.cpp

  • LenoyJLenoyJ - Select Field - Employee

    @smithclarkson01 you'll need to recompile your UDx's (including Vertica Extension Package) every time you upgrade major versions. See here: https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/ExtendingVertica/UDx/UpdatingUDFsAfterAServerUpgrade.htm

  • rohan123rohan123 Vertica Customer
    edited December 2019

    @lovelash said:
    I think the right way to avoid this error is to use gcc 4.8 as shown here
    if (!__sync_fetch_and_or(&(this->canceled), true)) {

    https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/ExtendingVertica/UDx/DevEnvironment.htm

    instead of editing the code, you can use something like this where you select the correct version of g++
    ! /usr/bin/g++-4.8 -D HAVE_LONG_LONG_INT_64 -g -Wall -Wno-unused-value -shared -fPIC -I /opt/vertica/sdk/include/ -o /tmp/Reader.so ./Reader.cpp /opt/vertica/sdk/include/Vertica.cpp

    Thankyou

  • scottsen11scottsen11 Community Edition User
    edited December 2019

    Vertica COPY command expects an uncompressed CSV input by default. GZIP option is available when source data files are located on the node, but not available when using the S3 function since it is based on the User-Defined Load (UDL) feature.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file