The Vertica Forum recently got a makeover! Let us know what you think by filling out this short, anonymous survey.
Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!

Why do I get "KAFKASOURCE() does not exist" Error, when loading from Kafka

joergschaberjoergschaber Vertica Customer


I have set up Kafka server with ** no permission** . When I consume data from a topic using a .NET client lib and configurw the consumer with

_config = new ConsumerConfig
                          BootstrapServers = "<server:port>",
                          GroupId = "<groupName>",
                          AutoOffsetReset = AutoOffsetReset.Earliest

I can readily consumer messages.

However, when I try to load messages from a topic (2 paritions) into Vertica using COPY with

                brokers = '<server:port>', duration = interval '10000 milliseconds', 
                group_id = '<groupName>', executionparallelism = 2) 
                PARSER KafkaJSONParser(flatten_maps=false, flatten_arrays=false,reject_on_duplicate=false)
                NO COMMIT;

I get an error Function KAFKASOURCE() does not exist, or permission is denied for KAFKASOURCE()'.
I ran

GRANT USAGE ON library kafkalib to <user>;
GRANT EXECUTE ON source KafkaSource() to <user>;;

When I use the dbadmin, the command does not return/erminate.
Even when I close the corresponding session, the session still appears in the session table.

Unsing Vertica 10.0, Any idea?


  • SruthiASruthiA Vertica Employee Employee

    Could you please try running copy from vertica node? does it still throw same error? is kafka package installed in vertica cluster?

  • joergschaberjoergschaber Vertica Customer

    Good question, I thought that is standard. How do I check?

  • SruthiASruthiA Vertica Employee Employee

    select * from user_libraries WHERE lib_name = 'KafkaLib';

  • joergschaberjoergschaber Vertica Customer

    KafkaLib is installed.

  • joergschaberjoergschaber Vertica Customer

    I ran the COPY command also from the vertica node as dbadmin. Same here, the command does not return and the session that was openeds cannot be deleted. Is there a more detialed log I can look into?

  • SruthiASruthiA Vertica Employee Employee

    so looks like it s hanging when run via vsql. vertica.log should provide more information

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file