A couple of co-workers of mine found the following problem:
Error occurred during initialization of VM Could not reserve enough space for object heap Could not create the Java virtual machine.
It happened in a batch-job environment where we use ulimit (for all programs, Java or not), in addition to Java's heap specification with -Xms and -Xmx. And it only started happening on a test switch from Java 6 to Java 7.
Now, Java requires memory for more than just heap --- native C malloc in JNI, mmap, and so on. But it also (moreso in Java 7, which is the point of this post) needs some non-heap memory to keep track of the heap itself.
A way to measure this is the following script called trymem:
#!/bin/bash
if [ $# -ne 3 ]; then
echo "Usage: $0 {javadir} {heap MB} {extra MB}"
exit 1
fi
javadir=$1
heap_mb=$2
xtra_mb=$3
totl_mb=$[heap_mb+xtra_mb]
totl_kb=$[totl_mb*1024]
ulimit -v $totl_kb
$javadir/bin/java -cp . -Xmx${heap_mb}m -Xms${heap_mb}m MyProgram
status=$?
echo ${heap_mb}+${xtra_mb}:${status}
exit $status
where MyProgram.java is simply
public class MyProgram {
public static void main(String[] args) {
}
}
along with a second script called searchmem:
#!/bin/bash
if [ $# -ne 1 ]; then
echo "Usage: $0 {Java dir}"
exit 1
fi
javadir=$1
heap_mb=10000
while [ $heap_mb -le 60000 ]; do
echo -n "$heap_mb "
xtra_mb=100
while true; do
./trymem $javadir $heap_mb $xtra_mb 1> /dev/null 2> /dev/null
status=$?
echo -n .
if [ $status -eq 0 ]; then
echo $xtra_mb
break
elif [ $xtra_mb -gt $heap_mb ]; then
echo "> $heap_mb"
break
fi
xtra_mb=$[xtra_mb+100]
done
heap_mb=$[heap_mb+5000]
done
The idea is to set ulimit to the heap size, then keep increasing it until the test program can run without error. The difference between Java 6 and Java 7 is significant:
$ ./searchmem /usr/local/jdk/x86_64/jdk1.6.0_35 10000 ....400 15000 ....400 20000 ....400 25000 ....400 30000 .....500 35000 .....500 40000 .....500 45000 .....500 50000 .....500 55000 ......600 60000 ......600
$ ./searchmem /usr/local/jdk/x86_64/jdk1.7.0_9 10000 ........800 15000 ..........1000 20000 .............1300 25000 ..............1400 30000 ................1600 35000 ...................1900 40000 .....................2100 45000 .......................2300 50000 .........................2500 55000 ............................2800 60000 .............................2900We couldn't find a pronouncement from Oracle regarding minimum ulimit as a function of heap size. But a plot of the above numbers suggests a linear relationship, and a regression (with more densely sampled data in searchmem, stepping heap_mb by 1000 rather than 5000) shows slope 1/24 and intercept 400MB. That is, given a Java heap size in MB, divide it by 24 and add 400 to it to find Java 7's heap-management overhead.
P.S. Thanks to David Craft for syntax highlighting in Blogspot:
http://www.craftyfella.com/2010/01/syntax-highlighting-with-blogger-engine.html
