The Vertica Forum recently got a makeover! Let us know what you think by filling out this short, anonymous survey.
Please take this survey to help us learn more about how you use third party tools. Your input is greatly appreciated!
Why do I get "KAFKASOURCE() does not exist" Error, when loading from Kafka

Hi,
I have set up Kafka server with ** no permission** . When I consume data from a topic using a .NET client lib and configurw the consumer with
_config = new ConsumerConfig { BootstrapServers = "<server:port>", GroupId = "<groupName>", AutoOffsetReset = AutoOffsetReset.Earliest };
I can readily consumer messages.
However, when I try to load messages from a topic (2 paritions) into Vertica using COPY with
COPY <schema.table> SOURCE KAFKASOURCE( stream='<topics>|0|-2,<topics>|1|-2', brokers = '<server:port>', duration = interval '10000 milliseconds', group_id = '<groupName>', executionparallelism = 2) PARSER KafkaJSONParser(flatten_maps=false, flatten_arrays=false,reject_on_duplicate=false) NO COMMIT;
I get an error Function KAFKASOURCE() does not exist, or permission is denied for KAFKASOURCE()'.
I ran
GRANT USAGE ON library kafkalib to <user>; GRANT EXECUTE ON source KafkaSource() to <user>;;
When I use the dbadmin, the command does not return/erminate.
Even when I close the corresponding session, the session still appears in the session table.
Unsing Vertica 10.0, Any idea?
0
Answers
Could you please try running copy from vertica node? does it still throw same error? is kafka package installed in vertica cluster?
Good question, I thought that is standard. How do I check?
select * from user_libraries WHERE lib_name = 'KafkaLib';
KafkaLib is installed.
I ran the COPY command also from the vertica node as dbadmin. Same here, the command does not return and the session that was openeds cannot be deleted. Is there a more detialed log I can look into?
so looks like it s hanging when run via vsql. vertica.log should provide more information