Skip to content

Call for testers: the next phase of Apocrita

For much of the year we have been working on a major project to upgrade Apocrita to a new operating system. As part of the project, we have deployed a new package building tool to help us recompile all of the research applications to work on the new system. We are now calling for Apocrita users to preview and test this new system, ahead of our full roll-out, to help bring about a smoother and quicker transition.

This is an opportunity to check that your applications work on the new system, and for us to address any issues before we fully roll it out.

Notice

This blog post will receive further updates during the testing phase.

What is the new system?

The existing Apocrita cluster runs the CentOS7 Operating System, which is based off Redhat Enterprise Linux (RHEL) 7. This has reached end-of-life and we are replacing it with Rocky9, based off RHEL9. Apart from security and compliance reasons, this will provide the benefit of much newer libraries and applications.

The existing end-of-life platform is no longer able to support newer software releases - various packages are no longer working, as our community of R users have discovered recently. Our new platform will be able to offer newer versions of research applications, so this will be of particular interest to anyone wishing to get latest versions of software.

Additionally, due to recent decommissioning of older nodes, we have been able to compile the new applications against a more modern CPU architecture, providing better compatibility and performance across the whole cluster.

We have currently provisioned a new login node, ondemand service and various types of compute and GPU node available for test users. This includes our brand new vhm high memory servers which can accommodate jobs up to 3TB RAM.

How can I use the new system?

Currently we have set an access list to run jobs on the upgraded nodes. To get added to the list, please send a message in the #pilot-testing channel on our slack channel, or send us an email and we'll add you.

Although jobs can we submitted from the CentOS7 frontend nodes, you will not be able to view the list of newly-built applications from a CentOS7 node. To access the Rocky9 login node, SSH into login-test.hpc.qmul.ac.uk using your existing Apocrita username, password and SSH key. To check what applications are available, you can run module avail from this server.

We are providing a new ondemand server, ondemand-dev.hpc.qmul.ac.uk which brings new functionality, and newer versions of applications (for example a newer RStudio).

We have a range of compute nodes running Rocky9, which can be checked with nodestatus -t rocky from the login server.

To run on the new nodes, you must:

  • request to be added to the Rocky9 testers list
  • request -l rocky to ensure your job is allocated to a Rocky9 node
  • ensure your job script is loading a module provided under Rocky9

Who should try the new system?

Any user who is typically running jobs with applications loaded with module load, ondemand or conda/virtualenv users should give it a try. We have provisioned a few Rocky9 serial, parallel and GPU nodes dedicated for testing. Any issues encountered should be reported via the aforementioned slack channel, or via support ticket.

Changes for conda users

Due to some recent anaconda licence changes, we now only provide the miniforge module for using conda environments. This works in the same way as anaconda and miniconda but does not use certain problematic anaconda repositories. Miniforge is the officially supported installation method for the faster Mamba libsolver and comes with the popular conda-forge channel pre-configured as its only channel, where most of the common conda packages are found. We strongly advise not to use any existing conda environments on Rocky9 that you created previously on CentOS7. Instead, you should start with fresh environments. If you want to "migrate" old ones, export them as a YML file on a CentOS7 node and then use that same YML file to re-create the environment again on a Rocky9 node.

Photo by Chris Liverani on Unsplash.