There are many strategies and tools for improving the performance of Python code, for a comprehensive treatment see High Performance Python by Gorelick and Ozsvald (institutional access is available to QM staff). However, there are some subtleties when using them in an HPC environment. More bluntly, requesting processor cores does not automatically mean your code will use them effectively, and that cannot happen if it doesn't know how many of them there are!
As the complexity of HPC applications increases, the management of memory and threading scopes becomes increasingly important. Tools like Intel Inspector are crucial in this context, to effectively identify and resolve a wide array of memory errors and thread synchronisation issues.
Modules are the centralised method of accessing different software on an HPC cluster. By using a variety of modules you can quickly and easily access different versions of applications and create work flows that suit particular projects. The modules offered on Apocrita cover a wide range of applications but there will always be situations that require something unusual or a relatively niche version of a piece of software.
The ITSR support team often receive tickets from Anaconda users concerned that creating environments and installing packages is taking quite a long time. We recently installed Miniconda as a module on Apocrita, which enables users to install packages using the Mamba libsolver.