Comparing common compression tools using real-world data

Posted on Thu 10 October 2019 in article by Simon Butcher

Compression tools can significantly reduce the amount of disk space consumed by your data. In this two-part article, we will look at the effectiveness of some compression tools on real-world data sets, make some recommendations, and perhaps persuade you that compression is worth the effort.

Continue reading

Performance testing with NVMe storage and Spectrum Scale 5

Posted on Wed 19 December 2018 in article by Peter Childs

We have recently procured 120TB of NVMe based SSD storage from E8 Storage for the Apocrita HPC Cluster. The plan is to deploy this to replace our oldest and slowest provision of scratch storage.

We are still in the final stages of deployment and we hope to make it available …

Continue reading

Python performance in containers

Posted on Mon 03 July 2017 in article by Simon Butcher

Singularity is a container solution designed for HPC. Due to the secure and simple design, it can be easily used to provide applications for use with HPC clusters where other containers, such as Docker would not be suitable.

Linux containers offer a great way to compare software performance on a …

Continue reading