共计 10276 个字符，预计需要花费 26 分钟才能阅读完成。
序
本文主要研究一下 jvm 的 -XX:MaxDirectMemorySize
-XX:MaxDirectMemorySize
-XX:MaxDirectMemorySize=size 用于设置 New I/O(java.nio) direct-buffer allocations 的最大大小，size 的单位可以使用 k /K、m/M、g/G；如果没有设置该参数则默认值为 0，意味着 JVM 自己自动给 NIO direct-buffer allocations 选择最大大小
System.initPhase1
java.base/java/lang/System.java
public final class System {
/* Register the natives via the static initializer.
*
* VM will invoke the initializeSystemClass method to complete
* the initialization for this class separated from clinit.
* Note that to use properties set by the VM, see the constraints
* described in the initializeSystemClass method.
*/
private static native void registerNatives();
static {
registerNatives();
}
/** Don’t let anyone instantiate this class */
private System() {
}
/**
* Initialize the system class. Called after thread initialization.
*/
private static void initPhase1() {
// VM might invoke JNU_NewStringPlatform() to set those encoding
// sensitive properties (user.home, user.name, boot.class.path, etc.)
// during “props” initialization.
// The charset is initialized in System.c and does not depend on the Properties.
Map<String, String> tempProps = SystemProps.initProperties();
VersionProps.init(tempProps);
// There are certain system configurations that may be controlled by
// VM options such as the maximum amount of direct memory and
// Integer cache size used to support the object identity semantics
// of autoboxing. Typically, the library will obtain these values
// from the properties set by the VM. If the properties are for
// internal implementation use only, these properties should be
// masked from the system properties.
//
// Save a private copy of the system properties object that
// can only be accessed by the internal implementation.
VM.saveProperties(tempProps);
props = createProperties(tempProps);
StaticProperty.javaHome(); // Load StaticProperty to cache the property values
lineSeparator = props.getProperty(“line.separator”);
FileInputStream fdIn = new FileInputStream(FileDescriptor.in);
FileOutputStream fdOut = new FileOutputStream(FileDescriptor.out);
FileOutputStream fdErr = new FileOutputStream(FileDescriptor.err);
setIn0(new BufferedInputStream(fdIn));
setOut0(newPrintStream(fdOut, props.getProperty(“sun.stdout.encoding”)));
setErr0(newPrintStream(fdErr, props.getProperty(“sun.stderr.encoding”)));
// Setup Java signal handlers for HUP, TERM, and INT (where available).
Terminator.setup();
// Initialize any miscellaneous operating system settings that need to be
// set for the class libraries. Currently this is no-op everywhere except
// for Windows where the process-wide error mode is set before the java.io
// classes are used.
VM.initializeOSEnvironment();
// The main thread is not added to its thread group in the same
// way as other threads; we must do it ourselves here.
Thread current = Thread.currentThread();
current.getThreadGroup().add(current);
// register shared secrets
setJavaLangAccess();
// Subsystems that are invoked during initialization can invoke
// VM.isBooted() in order to avoid doing things that should
// wait until the VM is fully initialized. The initialization level
// is incremented from 0 to 1 here to indicate the first phase of
// initialization has completed.
// IMPORTANT: Ensure that this remains the last initialization action!
VM.initLevel(1);
}
//……
}
System 的 initPhase1 方法会调用 VM.saveProperties(tempProps) 方法来保存一份系统配置供内部实现使用；其中 tempProps 为 SystemProps.initProperties()
jvm.cpp
hotspot/share/prims/jvm.cpp
// Convert the -XX:MaxDirectMemorySize= command line flag
// to the sun.nio.MaxDirectMemorySize property.
// Do this after setting user properties to prevent people
// from setting the value with a -D option, as requested.
// Leave empty if not supplied
if (!FLAG_IS_DEFAULT(MaxDirectMemorySize)) {
char as_chars[256];
jio_snprintf(as_chars, sizeof(as_chars), JULONG_FORMAT, MaxDirectMemorySize);
Handle key_str = java_lang_String::create_from_platform_dependent_str(“sun.nio.MaxDirectMemorySize”, CHECK_NULL);
Handle value_str = java_lang_String::create_from_platform_dependent_str(as_chars, CHECK_NULL);
result_h->obj_at_put(ndx * 2, key_str());
result_h->obj_at_put(ndx * 2 + 1, value_str());
ndx++;
}
jvm.cpp 里头有一段代码用于把 -XX:MaxDirectMemorySize 命令参数转换为 key 为 sun.nio.MaxDirectMemorySize 的属性
VM.saveProperties
java.base/jdk/internal/misc/VM.java
public class VM {
// the init level when the VM is fully initialized
private static final int JAVA_LANG_SYSTEM_INITED = 1;
private static final int MODULE_SYSTEM_INITED = 2;
private static final int SYSTEM_LOADER_INITIALIZING = 3;
private static final int SYSTEM_BOOTED = 4;
private static final int SYSTEM_SHUTDOWN = 5;
// 0, 1, 2, …
private static volatile int initLevel;
private static final Object lock = new Object();
//……
// A user-settable upper limit on the maximum amount of allocatable direct
// buffer memory. This value may be changed during VM initialization if
// “java” is launched with “-XX:MaxDirectMemorySize=<size>”.
//
// The initial value of this field is arbitrary; during JRE initialization
// it will be reset to the value specified on the command line, if any,
// otherwise to Runtime.getRuntime().maxMemory().
//
private static long directMemory = 64 * 1024 * 1024;
// Returns the maximum amount of allocatable direct buffer memory.
// The directMemory variable is initialized during system initialization
// in the saveAndRemoveProperties method.
//
public static long maxDirectMemory() {
return directMemory;
}
//……
// Save a private copy of the system properties and remove
// the system properties that are not intended for public access.
//
// This method can only be invoked during system initialization.
public static void saveProperties(Map<String, String> props) {
if (initLevel() != 0)
throw new IllegalStateException(“Wrong init level”);
// only main thread is running at this time, so savedProps and
// its content will be correctly published to threads started later
if (savedProps == null) {
savedProps = props;
}
// Set the maximum amount of direct memory. This value is controlled
// by the vm option -XX:MaxDirectMemorySize=<size>.
// The maximum amount of allocatable direct buffer memory (in bytes)
// from the system property sun.nio.MaxDirectMemorySize set by the VM.
// If not set or set to -1, the max memory will be used
// The system property will be removed.
String s = props.get(“sun.nio.MaxDirectMemorySize”);
if (s == null || s.isEmpty() || s.equals(“-1”)) {
// -XX:MaxDirectMemorySize not given, take default
directMemory = Runtime.getRuntime().maxMemory();
} else {
long l = Long.parseLong(s);
if (l > -1)
directMemory = l;
}
// Check if direct buffers should be page aligned
s = props.get(“sun.nio.PageAlignDirectMemory”);
if (“true”.equals(s))
pageAlignDirectMemory = true;
}
//……
}
VM 的 saveProperties 方法读取 sun.nio.MaxDirectMemorySize 属性，如果为 null 或者是空或者是 -1，那么则设置为 Runtime.getRuntime().maxMemory()；如果有设置 MaxDirectMemorySize 且值大于 -1，那么使用该值作为 directMemory 的值；而 VM 的 maxDirectMemory 方法则返回的是 directMemory 的值
获取 maxDirectMemory 的值
实例
public BufferPoolMXBean getDirectBufferPoolMBean(){
return ManagementFactory.getPlatformMXBeans(BufferPoolMXBean.class)
.stream()
.filter(e -> e.getName().equals(“direct”))
.findFirst()
.orElseThrow();
}
public JavaNioAccess.BufferPool getNioBufferPool(){
return SharedSecrets.getJavaNioAccess().getDirectBufferPool();
}
/**
* -XX:MaxDirectMemorySize=60M
*/
@Test
public void testGetMaxDirectMemory(){
ByteBuffer.allocateDirect(25*1024*1024);
System.out.println(Runtime.getRuntime().maxMemory()/1024.0/1024.0);
System.out.println(VM.maxDirectMemory()/ 1024.0 / 1024.0);
System.out.println(getDirectBufferPoolMBean().getTotalCapacity() / 1024.0 / 1024.0);
System.out.println(getNioBufferPool().getTotalCapacity() / 1024.0 / 1024.0);
}
输出
输出结果如下：
4096.0
60.0
25.0
25.0
由于 java9 模块化之后，VM 从原来的 sun.misc.VM 变更到 java.base 模块下的 jdk.internal.misc.VM；上面代码默认是 unamed module，要使用 jdk.internal.misc.VM 就需要使用 –add-exports java.base/jdk.internal.misc=ALL-UNNAMED 将其导出到 UNNAMED，这样才可以运行
同理 java9 模块化之后，SharedSecrets 从原来的 sun.misc.SharedSecrets 变更到 java.base 模块下的 jdk.internal.access.SharedSecrets；要使用 –add-exports java.base/jdk.internal.access=ALL-UNNAMED 将其导出到 UNNAMED，这样才可以运行
从输出结果可以看出，Runtime.getRuntime().maxMemory() 输出的值是正确的，而 BufferPoolMXBean 及 JavaNioAccess.BufferPool 的 getTotalCapacity 返回的都是 directBuffer 大小，而非 max 值
使用 API 查看 directBuffer 使用情况
实例
/**
* -XX:MaxDirectMemorySize=60M
*/
@Test
public void testGetDirectMemoryUsage(){
ByteBuffer.allocateDirect(30*1024*1024);
System.out.println(getDirectBufferPoolMBean().getMemoryUsed()/ 1024.0 / 1024.0);
System.out.println(getNioBufferPool().getMemoryUsed() / 1024.0 / 1024.0);
}
输出
输出结果如下：
30.0
30.0
可以看到 BufferPoolMXBean 及 JavaNioAccess.BufferPool 的 getMemoryUsed 可以返回 directBuffer 大小
OOM
java.lang.OutOfMemoryError: Direct buffer memory
at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)
at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
如果上面的 ByteBuffer.allocateDirect 改为分配超过 60M，则运行抛出 OutOfMemoryError
使用 NMT 查看 directBuffer 使用情况
jcmd 3088 VM.native_memory scale=MB
3088:
Native Memory Tracking:
Total: reserved=5641MB, committed=399MB
– Java Heap (reserved=4096MB, committed=258MB)
(mmap: reserved=4096MB, committed=258MB)
– Class (reserved=1032MB, committed=6MB)
(classes #1609)
(instance classes #1460, array classes #149)
(mmap: reserved=1032MB, committed=5MB)
(Metadata:)
(reserved=8MB, committed=5MB)
(used=3MB)
(free=2MB)
(waste=0MB =0.00%)
(Class space:)
(reserved=1024MB, committed=1MB)
(used=0MB)
(free=0MB)
(waste=0MB =0.00%)
– Thread (reserved=18MB, committed=18MB)
(thread #18)
(stack: reserved=18MB, committed=18MB)
– Code (reserved=242MB, committed=7MB)
(mmap: reserved=242MB, committed=7MB)
– GC (reserved=203MB, committed=60MB)
(malloc=18MB #2443)
(mmap: reserved=185MB, committed=43MB)
– Internal (reserved=1MB, committed=1MB)
(malloc=1MB #1257)
– Other (reserved=30MB, committed=30MB)
(malloc=30MB #2)
– Symbol (reserved=1MB, committed=1MB)
(malloc=1MB #13745)
– Shared class space (reserved=17MB, committed=17MB)
(mmap: reserved=17MB, committed=17MB)
从 Other 部分可以看到其值跟 ByteBuffer.allocateDirect 使用的值一致，改变 ByteBuffer.allocateDirect 的值再重新查看，可以发现 Other 部分跟着改变；因而初步断定 Other 部分应该是可以反映 direct memory 的使用大小
小结
-XX:MaxDirectMemorySize=size 用于设置 New I/O(java.nio) direct-buffer allocations 的最大大小，size 的单位可以使用 k /K、m/M、g/G；如果没有设置该参数则默认值为 0，意味着 JVM 自己自动给 NIO direct-buffer allocations 选择最大大小；从代码 java.base/jdk/internal/misc/VM.java 中可以看到默认是取的 Runtime.getRuntime().maxMemory()
使用 jdk.internal.misc.VM.maxDirectMemory() 可以获取 maxDirectMemory 的值；由于 java9 模块化之后，VM 从原来的 sun.misc.VM 变更到 java.base 模块下的 jdk.internal.misc.VM；上面代码默认是 unamed module，要使用 jdk.internal.misc.VM 就需要使用 –add-exports java.base/jdk.internal.misc=ALL-UNNAMED 将其导出到 UNNAMED，这样才可以运行
BufferPoolMXBean 及 JavaNioAccess.BufferPool(通过 SharedSecrets 获取) 的 getMemoryUsed 可以获取 direct memory 的大小；其中 java9 模块化之后，SharedSecrets 从原来的 sun.misc.SharedSecrets 变更到 java.base 模块下的 jdk.internal.access.SharedSecrets；要使用 –add-exports java.base/jdk.internal.access=ALL-UNNAMED 将其导出到 UNNAMED，这样才可以运行
另外使用 NMT 也可以查看 direct memory 的使用情况，其包含在了 Other 部分
doc
Default HotSpot Maximum Direct Memory Size
JVM 源码分析之堆外内存完全解读
Java 堆外内存增长问题排查 Case
DirectByteBuffer 堆外内存溢出问题排查
Default for XX:MaxDirectMemorySize
java.lang.OutOfMemoryError: Direct buffer memory (with tcnative) #6813
VM.java MaxDirectMemorySize
Summary of understanding “-XX:MaxDirectMemorySize” setting.
native-mem-tracking.md
Java Platform, Standard Edition Tools Reference
