Changes to R in August 2023
The ITSR support team have some changes to R and RStudio on Apocrita in August 2023. Some action will be required by users, so please read the below carefully.
Welcome to the QMUL HPC blog¶
New Role at QMUL: Research Community Lead
Hello world, my name is Julita Inca and I am originally from Lima, Peru. I am currently based in London, UK. My professional focus is High Performance Computing (HPC) and Linux.
Going to the HPC-SIG 2023
On Thursday, June 13, I attended the HPC-SIG meeting at the University of Bath which hosted many High Performance Computing (HPC) specialists in the UK. The event was hybrid, so there were international attendees as well as those from the UK.
Pythonic Parallel Processing for HPC: Your Gauss is as good as mine
There are many strategies and tools for improving the performance of Python code, for a comprehensive treatment see High Performance Python by Gorelick and Ozsvald (institutional access is available to QM staff). However, there are some subtleties when using them in an HPC environment. More bluntly, requesting processor cores does not automatically mean your code will use them effectively, and that cannot happen if it doesn't know how many of them there are!
Intel Inspector 2022.2 on Apocrita
As the complexity of HPC applications increases, the management of memory and threading scopes becomes increasingly important. Tools like Intel Inspector are crucial in this context, to effectively identify and resolve a wide array of memory errors and thread synchronisation issues.
Creating and Using Private Modules
Modules are the centralised method of accessing different software on an HPC cluster. By using a variety of modules you can quickly and easily access different versions of applications and create work flows that suit particular projects. The modules offered on Apocrita cover a wide range of applications but there will always be situations that require something unusual or a relatively niche version of a piece of software.
Installing Anaconda packages more quickly using Mamba
The ITSR support team often receive tickets from Anaconda users concerned that creating environments and installing packages is taking quite a long time. We recently installed Miniconda as a module on Apocrita, which enables users to install packages using the Mamba libsolver.
Modules Update December 2022
Since the last module update in December 2021, we have:
- added/moved 84 modules to production
- added 9 modules to the development environment
- deprecated 5 modules
- deleted 12 modules
Speeding Up Grep Searches
Sometimes you may find yourself needing to filter a large amount of output
using the grep command. However, grep can sometimes struggle when you try
to filter files with an incredibly large number of lines, as it loads each line
into RAM line-by-line. This can mean you can quickly exhaust even large amounts
of requested RAM. There are a few ways around this.
R Tutorial Part Two - R vs RStudio
Following up from part one of our R tutorial we'll be taking a look at the differences between R - the command-line language which can be loaded as a module and used in your Apocrita batch jobs - and Rstudio - the graphical development environment, accessed via a web server and provided via the OnDemand service.