RSE team activities up to June 2022
The RSE team in ITS Research has had a busy few years since we started sharing our work in this blog. In this post we look at some highlights of recent activity and what we have to look forward to.
Welcome to the QMUL HPC blog¶
Using job statistics to increase job performance and reduce queueing time
You may wonder why some jobs start immediately but some wait in the queue for hours or days, even if your job is quite simple. If you notice your job has been queueing for a while, you may want to consider adjusting the requested resources to reduce queueing time and reduce any potential resource wastage as the job runs. Below, we outline two useful tools for you to check the resource usage of previous jobs.
Speeding up and Parallelising R packages (using Rcpp and C++)
Polytomous variables can be used to model data that has two or more possible outcomes. For example, a survey with multiple-choice questions is polytomous. The R package, poLCA, does statistical clustering of polytomous variables. For example, grouping together survey results that are similar to each other.
Assessing code quality with the NAG Fortran compiler
The NAG Fortran compiler, like other compilers, has diagnostic capabilities which can help us write correct and portable Fortran programs. In this post we'll look at these, comparing with those of the GCC and Intel compilers, and see how the compiler can be a valuable tool when developing or maintaining Fortran code.
Using compiler options to help debug programs
Once we've written a program more advanced than our "Hello, world!" example, we're going to make mistakes. In this post, we'll look at how we can use the very compilers we're using to compile our program to pick up on some of these mistakes.
Visualising HEALPix results with Jupyter Notebook
In this tutorial we'll be showing you how to visualise HEALPix results using Jupyter Notebook in our OnDemand appliance on the Apocrita HPC cluster. We'll start with installing the required Python packages before demonstrating how to run the Healpy tutorial.
Information about running other components of HEALPix not covered in this tutorial can be found on our docs site.
Introducing Sherman Lo, RSE
Hello! I am Sherman and I have just joined the RSE team at Queen Mary. Glad to meet you all!
My background is in computational statistics and machine learning. I have completed projects in rainfall prediction, defect detection for 3D printing and Markov chains using Monte Carlo. These projects involved collaboration with various scientists, such as meteorologists, engineers and statisticians.
File Permissions
An understanding of file permissions is important to the success of computational jobs, and the security of your files.
The default settings are suitable for some, but not every use-case: without sufficient awareness, your files may be visible to people who should not be able to access them, and vice-versa.
Modules Update December 2021
Since the last module update in December 2019, we have:
- added/moved 48 modules to production
- added 20 modules to the development environment
- deprecated 12 modules
- deleted 4 modules
One year of code review club with the William Harvey Research Institute
Over the past year, researchers from QMUL's William Harvey Research Institute (WHRI) have engaged on a collaborative code review club. Through this collaborative effort the group aims to peer review the computational components of their research and provide code quality assurance to all involved researchers. Additionally, the Research Software Engineering group of ITS Research has been assisting the group with knowledge transfer and by participating in the review process.