JVM heap memory error
I know this isn't entirely about Vertica, but i guess this is a good place to ask this. While performing scalability testing on a UDF using PARTITION BY we encountered an error
Error in .jcall(rp, "[D", "getDoubles", i) :
java.lang.OutOfMemoryError: Java heap space
Similar errors have appeared such as
java.lang.OutOfMemoryError: GC overhead limit exceeded
Help?
Error in .jcall(rp, "[D", "getDoubles", i) :
java.lang.OutOfMemoryError: Java heap space
Similar errors have appeared such as
java.lang.OutOfMemoryError: GC overhead limit exceeded
Help?
0
Comments
You can adjust the JVM memory settings by editing /opt/vertica/bin/mctl
I would recommend the following adjustments to start:Original Line: $JAVA_HOME/bin/java -Dvertica.home=$VERTICA_HOME -Djava.library.path=$VERTICA_HOME/lib -Dderby.system.home=/opt/vertica/mcdb/derby -Xmx2048m -Xms1024m -XX:MaxPermSize=256m -jar $VERTICA_HOME/lib/webui.war & Updated Line:$JAVA_HOME/bin/java -Dvertica.home=$VERTICA_HOME -Djava.library.path=$VERTICA_HOME/lib -Dderby.system.home=/opt/vertica/mcdb/derby –Xmx3096m -Xms1512m -XX:MaxPermSize=256m -jar $VERTICA_HOME/lib/webui.war & Once you have made the change, please restart the Management Console for the new settings to take effect.
Regards,
Bhawana
Nothing changed between tests.
I'll spare the fine detail of memory blocks and heap and stack in C# but the answer is at the link I provided When you get to large data compute happens at the disk and "reducing the process" gets you done quicker. It is better to break up the process into smaller steps if possible.
You might want to try Java 8, I am assuming you are on 7. Apparently Java 7's G1 implementation had bugs.
http://blog.mgm-tp.com/2013/12/benchmarking-g1-and-other-java-7-garbage-collectors/
http://mechanical-sympathy.blogspot.com/2013/07/java-garbage-collection-distilled.html