Details about Transparent Huge Pages, THP, Static Huge Pages, Larges Pages, Shared Memory Huge Pages

This article explains the differences between the following types of huge or large memory pages on Linux:

• Anonymous Transparent Huge Pages, or just Transparent Huge Pages, often abbreviated THP
• Shared Memory Transparent Huge Pages, or Shared Memory Huge Pages (SHMEM-THP)
• Static Huge Pages

If you are just looking for the recommended THP configuration for Zing, this detailed explanation is not needed and you can directly use the configuration explained on the following page:

https://docs.azul.com/prime/Enable-Huge-Pages

But if you are curious to read more details, for example when your system components already make use of one of those described above, then the following text may be helpful:

The standard memory pagesize on Linux is 4 kBytes, except for some arm64 systems where it is 64 kBytes, but also arm64 with 4k exists. Zing uses those by default when running with or without ZST.

In addition, Linux offers larger pages or huge pages in various sizes, but almost always 2 MBytes is their default size.

From the huge pages section, Zing can only use Shared Memory Huge Pages of 2 MBytes size for the Java Heap when running without ZST.

ZST as optional component offers its own management of huge pages for the Java Heap, internally based on the standard 4k Linux pages, and is only available for Zing, but as soon Shared Memory Transparent Huge Pages (SHMEM-THP) are available, ZST is not needed anymore if the Java Heap size is below 2800 GBytes (2.8 TBytes).

Transparent Huge Pages in general can cause system-wide Linux memory compaction pauses when Linux needs to defragment memory to acquire more huge pages. That's why for low latency situations they are often recommend to be switched off.

But when configuring them correctly, the risk for system-wide pauses is drastically reduced. The configuration recommendation on https://docs.azul.com/prime/Enable-Huge-Pages makes them only available for those few processes which explicitly request them and with that setting, they are well usable also for low latency systems and simple to manage.

The main reason for using Huge Pages is to increase throughput performance. Often improvements of 10% or even more can be observed for metrics about application throughput which would also improve the individual response time if the response duration for processing on the server is large compared to network latency. A 2nd reason to use them is to achieve a significantly quicker process start, especially relevant with Java Heap sizes of 100 GBytes or more, where a process start for a 2 TBytes heap for example will take  many minutes without SHMEM-THP. With SHMEM-THP it will be shortened by factor of more than 10.

The SHMEM-THP needed by Zing without ZST are only available on modern Linux versions, for example RHEL 8 or newer or Ubuntu 20.04 LTS or newer or Amazon Linux. One specialty is Oracle Linux 7.9 where the default setting actually provides SHMEM-THP, just when booting into the RHEL 7.9 mode they are not available there.

Static Huge Pages can also be used with Zing, but are not recommended for new installations due to their complexity in system management and SHMEM-THP or ZST are usually the better alternatives.

The recommended configuration https://docs.azul.com/prime/Enable-Huge-Pages works for Zing with and without the ZST component.