With using Zing for Cassandra you don't need to spend time in tuning the JVM Garbage Collection parameters.
Here a quick overview for switching from other JVMs to Zing on a Cassandra node:
First, after stopping the Cassandra service on the node, install Zing. The fastest way is to start on the Zing quick start page using the Zing package repository. Then change the JVM setting for Cassandra: JAVA_HOME=/opt/zing/zing-jdk8
For a test or developer system, that's all. After restarting the Cassandra node you can check with the zing-ps command if the switch to Zing was successful and a Zing JVM process is running.
For production systems, the following additional Zing settings are required:
- First, look up the -Xmx value used on this Cassandra node. It is set in the configuration file cassandra-env.sh. For a node with 32 GB RAM for example it is typically -Xmx8G.
- Change the Zing memory reservation configuration to reserve-at-config and in addition allow for more memory to be available for the file page cache as common Cassandra usage patterns are I/O intensive. By default, Zing reserve-at-config mode reserves 75% of the the system's RAM for Java and leaving only 25% for the caches and all Linux processes.
To change this, start the command sudo system-config-zing-memory, then change the settings to the following and as value Xg enter the -Xmx value from cassandra-env.sh plus an additional 2G. Example: If cassandra-env.sh is set to -Xmx8G enter 10g for Reservable memory:
** Enter reserve-at-(c)onfig or reserve-at-(l)aunch: c
** Enter (s)pecific amount or (p)ercentage of system memory [default 's']: s
** Enter amount of Reservable memory [default '16g']: Xg
Keep all other settings at the predefined default values.
The reason for entering 2G more than the used -Xmx for Cassandra is to have enough Zing memory left to start small diagnostic Zing java processes in parallel to the main Cassandra process.
- You don't need to change any JVM parameter in the cassandra-env.sh configuration. Zing is ignoring non-Zing -XX parameters.
- Disable the Cassandra GCInspector log messages as it might report long running GC phases which are of no concern when using Zing as those logged times are phases of the GC which are run in parallel to the application and don't pause the application. The log messages written to the Cassandra system.log are of the following pattern:
INFO [Service Thread] DATE GCInspector.java:284 - GPGC New GC in 300ms. GenPauseless
To disable them, add the following line to cassandra_install_location/conf/logback.xml:
<logger name="org.apache.cassandra.service.GCInspector" level="ERROR"/>
- For high performance Cassandra systems add or change in /etc/sysctl.conf or /etc/sysctl.d/zing.conf the following file system cache settings:
Those are standard Linux recommendations generally for databases to prevent spikes and waits in I/O.
- After rebooting the host and when Cassandra is running on the host again, check the following:
Is enough Zing memory available to start diagnostic java processes?
Run "/opt/zing/zing-jdk8/bin/java -version" to check if it starts.
Is Cassandra using the correct -Xmx?
The command "zing-ps" will list it as column "Xmx".
For further questions about Zing configuration or system performance, please ask Azul Support.
In addition, you can request our Azul Cassandra Healthcheck performed by one of our Field Engineers, for instance if you plan a large Cassandra deployment.
Cassandra on Zing was never affected by the following bug as a fix in Zing was included in the Zing releases around the time this bug was triggered by changed added by JDK 8u161 / 8u162: https://issues.apache.org/jira/browse/CASSANDRA-14173