Updated February 2026
We have updated this post with answers to some common questions.
For the lifetime of the Apocrita HPC cluster, the queue scheduler software we have been using to allocate jobs to individual compute nodes has been a variant of Grid Engine. Over the past few years, the company who own this software has changed hands a few times, and we feel that development has stagnated in terms of features, while bugs have not been resolved to our satisfaction.
On Monday 3rd November 2025, we will be implementing a change to reduce the maximum runtime for all Open OnDemand applications from 240 hours (10 days) to 24 hours.
Hybrid programming allows you to program the majority of your software in your favourite language but performance-critical parts in a faster language. With the Python package CuPy, you can program CPU code in Python and custom GPU kernel functions in CUDA. Thus, we can design our software with a familiar Python interface but run faster GPU code under the hood.
CuPy also has GPU versions of existing NumPy functions which may help transition your CPU code to the GPU without modifying your code too much. This may also help structure your GPU code with familiar NumPy functions, making it readable to many Python users.
In a previous blog, we looked at using Numba to speed up Python code by using a just-in-time (JIT) compiler and multiple cores. The speed-up is remarkable with small changes to the existing code.
In this blog post, we will continue exploring the Numba ecosystem and implement the Gauss map on the GPU, gaining further speed up while still writing Python code. We will also look at CuPy which is another way to write and run GPU code. Instead of being Pythonic, it allows hybrid programming where the GPU code is written in CUDA but executed in Python.
One key advantage of hybrid programming is writing software in your preferred language while optimising performance-critical sections in a faster language - the best of both worlds!
The majority of the cluster has now been upgraded to Rocky 9 and the remaining CentOS 7 nodes will be updated in due course. There may be some users that are still hesitant to move over, but there are a few reasons why you should.
With the major operating system upgrade from Centos 7 to Rocky 9, we want to ensure that using R, RStudio, and Open OnDemand (OOD) is as seamless as possible. This post will include new tips for a better experience, as well as a reiteration of the important or frequently forgotten old tips.
A previous blog post covered
Using R inside of Conda, but what about
if you want to use Python packages inside an existing
R or
RStudio via OnDemand session?
This is where the
reticulate R Interface to Python
comes in.
Traditionally we have recommend that users use a
Python virtualenv
or Conda environment
to manage personal package installations via pip install and mamba install
commands. But a new contender has entered the fray:
uv, an extremely fast Python package and
project manager, written in Rust.
The Apocrita cluster was upgraded from CentOS 7 to Rocky 9 recently. There are some important things Python users need to know, such as how to migrate your existing environments to Rocky 9, as well as how to tackle some common problems during the process.
For much of the year we have been working on a major project to upgrade Apocrita to a new operating system, (Rocky Linux 9, hereafter known as Rocky 9). As part of the project, we have deployed a new package building tool to help us recompile all of the research applications to work on the new system.
The majority of the cluster has now been upgraded to Rocky 9. The remaining CentOS 7 nodes will be updated in due course.