Options
Seeking any suggestions
Hello, I've noticed a potential memory leak in one of my User-Defined Extensions (UDx); the resident set size (RSS) and virtual memory size (VSZ) continuously increase for the vertica-udx-C++ process during its operation. The UDx is running in fenced mode. Despite thorough code reviews, we haven't been able to identify the exact cause. I'm seeking any suggestions or guidance you might have. Please, let's skip suggestions about using vt_alloc for now. Ideally, I'd appreciate detailed steps on how to apply AddressSanitizer (ASAN) to a UDx operating in fenced mode.
0
Answers
I'm not a C++ expert, but based on information I've gathered, here's a response to your question.
If necessary, please open a support case, for more accurate guidance.
Try to re-create the issue in a controlled environment.
Then, as the UDx runs in fenced mode, measure (and graph) the RSS/VSZ memory segments of the two processes:
Vertica and zygote, to understand which process causes the memory leak.
Consider that a continuous increase of the allocated memory is not always a sign of a memory leak
Did you monitor the Catalog size? Is it increasing as well? And, if yes, which Catalog component is increasing?
After this simple RSS/VSZ memory segment measures over time, and assuming the “guilty” is the UDx zygote process…
Then analyze the code and use tools like Valgrind to debug it.
See: valgrind.org and https://github.com/dingqiangliu/blogs/blob/master/how-to-write-debug-and-profile-vertica-udx_en.md
BTW, The FencedUDxMemoryLimitMB configuration parameter lets you create an absolute memory limit for UDxs. Any attempt by a UDx to allocate more memory than this limit results in a bad_alloc exception.
For an example of setting FencedUDxMemoryLimitMB, see How resource limits are enforced here:
https://docs.vertica.com/24.1.x/en/extending/developing-udxs/developing-with-sdk/cpp-sdk/resource-use-cpp-udxs/how-resource-limits-are-enforced