Payara Server Advanced Upgrade Guide

There are three valid methods of upgrading to a new release of Payara Server Enterprise, which should work in most circumstances.

1. The first method is using the asadmin upgrade-server subcommand. This is often the go-to method.
2. The second method is backing up and restoring existing domain configurations to a new install of Payara Server Enterprise. This is recommended whenever you have a non-standard install.
3. The third method is pointing a new version of Payara Server Enterprise to the existing domain and node directories during the upgrade. This method can only be used when you are maintaining completely separate domain and node directories from the server directory.

Pre-requisites:

Before attempting any upgrade on your production environment, here's some preparation that you should do to ensure a smooth and surprise-free upgrade process.

Ensure the PATH Variable points to the JDK/bin directory.

Check that the following executables from the JDK are available on the PATH variable of the environment or terminal in which you're going to be running the upgrade process. 

java

jar

keytool

The Upgrade Tool also requires these to be present on the environment PATH during the upgrade process. This is especially important in Windows environments with Oracle JDK installed via the Oracle installer. Since the Oracle installer does not add the entire JDK/bin directory to the standard Windows PATH variable, the Upgrade Tool can fail during the installation. Thus, check the presence of these commands by running them in your environment or terminal before beginning the upgrade process.

Cygwin on Windows:

If you're using Cygwin on Windows, then check the availability of these commands inside the Cygwin terminal as well. If the commands are not available in your Cygwin terminal, then first configure your Cygwin environment to properly reflect the Windows PATH variable. Check here for more details on the Cygwin environment.

Upgrading the upgrade-server-tool to the latest version

Usually, the Upgrade Tool that ships with the latest version of Payara Server Enterprise would be the most recent one at the time of the release of the said version, however, there can be a newer version of the Upgrade Tool incorporating bug fixes and upgrades. Hence, always check and download the latest version of the Upgrade Tool from the Nexus repository > Payara Enterprise Downloadable Artifacts > Fish > Payara > Extras > Payara-Upgrade-Tool. At the time of this writing, the latest version is 1.6.

Once you have downloaded the latest version of the Upgrade Tool, copy the JAR file to

 payara5 > glassfish > lib > asadmin 

the directory inside your existing Payara Server install directory. To ensure that you overwrite the existing 

payara-upgrade-tool.jar 

file with the new file. You might have to rename the newly downloaded JAR file to overwrite it.

1. Using the upgrade-server subcommand

The simplest way to upgrade your Payara Server's installation is to simply issue the following command:

asadmin upgrade-server --username <my-username> --version <new_version>

Note: If you upgrade from Payara Server version 5.31.0 or older, you need to set the fish.payara.upgrade.repo.url system property as described in Payara Enterprise Nexus Repository changes after October 18th 2021 to point the Upgrade Tool to the new location of the download repository.

This command will upgrade all domains located in the default domain directory to the specified version. The process is seamless, but in the case, any unexpected issues arise, it is possible to automatically revert the changes with the rollback-server command. On Windows environments, the rollback-server subcommand is not available however, you can use the rollbackUpgrade.bat script instead. The script can be found in payara5/glassfish/bin/ directory.

You can only use this feature from Payara Server Enterprise release 5.24.0 onwards, however, it is possible to download the current release of the Ppgrade Tool and install it on earlier releases of Payara Server Enterprise to migrate to newer versions.

More information and details about the command are available on the documentation site.

2. Using backup-domain and restore-domain subcommands

In this method, we download a fresh copy of the latest version of Payara Enterprise and use the backup-domain and restore-domain subcommands to restore domains from the previous version to the newer version.

1. Download the latest version of Payara Enterprise from the Payara Enterprise Nexus Repository and unzip it in your desired directory.
2. Stop the domain if it's running already, since the backup-domain subcommand will only work if the domain is not running. So, for a production system or cluster, you would need to arrange for a period of downtime.
3. Issue the following command from an existing Payara Server domain.

4. asadmin backup-domain <domain_name>

5. Once the domain has been backed up you should see the resulting zip file with the following name{backupDir}/{domain_name}/<domain-name>_<YYYY_MM_DD>_v<backup-number>
6. Now head to the directory with the newer version of Payara Enterprise and issue the following command to restore the backup we created in step 5. 

7. asadmin restore-domain --filename {path_to_backup} {domain_name}

After following these steps, your host environment should be left with a directory structure similar to the following:

/opt/payara/<old_version>/glassfish/domains/myDomain
/opt/payara/<new_version>/glassfish/domains/myDomain
/opt/payara/backups/myDomain/myDomain_2021_01_12_v00001.zip

Keep in mind that this method only backs up the domain directory and its contents. Any instances you have in node directories will need to be "regenerated" by using the --sync=full the argument of the start-local-instance subcommand:

asadmin start-local-instance --sync=full myInstance

This will recreate the instance from scratch. Be aware, though, that this will not restore data stored in the instance directory which is outside of standard configuration stores like JMS file stores, for example.

3. Linking Domain Directory to Payara Install

This method requires that both the Payara Server domain and node directories are stored in a location separate from the installation directory. For example, if you download Payara Server 5.38, create a domain, and then wanted to re-use it with release 5.50, you will need to run the following commands:

/opt/payara/5.38/ > bin/asadmin create-domain --domaindir /var/domains myTempDomain
/opt/payara/5.38/ > cd ../5.50
/opt/payara/5.50/ > bin/asadmin start-domain --domaindir /var/domains myTempDomain			

This will use the 5.38 installation to create a new domain and then will start it by using the binaries present in the 5.50 installation.

Likewise, node directories can be configured in the same manner:

/opt/payara/5.38/ > bin/asadmin create-node --nodedir /var/nodes myLocalNode

In this way, the domain and node configurations are always kept separate from server binaries, and rollback is as simple as using the previous installation directory.

Linux Symlinks

If you are using Payara Server on Linux, you could take advantage of symbolic links to make life easier. When your domain directory is in a non-standard location, it means that you need to specify its location with each start/stop/restart command. If you create a symbolic link to the domain directory inside the server's domains directory, then the asadmin CLI and any other scripts that rely on the standard locations will carry on as normal.

You could even extend this further to create a symbolic link to whichever Payara Server installation is currently used in production, for example:

/opt/payara/live/5.50/....

This means that any rollback is as simple as:

1. Stopping the domain
2. Updating the symbolic link
3. Starting it again.

Without the use of symbolic links, things get a bit more manual (though still quite straightforward). The start-domain subcommand has a --domaindir argument that allows you to specify the location of a domain directory. Take the following directory structure, for example:

/opt/payara/5.38/
/opt/payara/5.39/
/opt/payara/5.50/
/opt/payara/domains/myDomain/
/opt/payara/nodes/myLocalNode/

You can start your domain with whatever version of Payara Server you want to use:

/opt/payara/5.50/bin/asadmin start-domain --domaindir /opt/payara/domains myDomain

/opt/payara/162/payara41/bin/asadmin start-domain --domaindir /opt/payara/domains myDomain

You can also start nodes in the same way:

/opt/payara/162/payara41/bin/asadmin start-local-instance --nodedir /opt/payara/nodes myLocalNode

Note: Using Symlinks is not supported on Windows environment.