New high performance storage for scratch data

We are pleased to announce a new scratch storage array that is based on fast NvME based hardware. This will hopefully make I/O related tasks much faster

Scratch space is the recommended location for temporarily storing data produced by cluster jobs. We often see jobs failing on the cluster due to users filling their home directory quota with data from job outputs, instead of using scratch space. Some users may be unaware that a P.I. can request a lab share and get the initial 1TB at no cost.

This new service replaces the old legacy /data/scratch service, which we switched off a few weeks ago, in order to re-use the same file paths. By reusing the old paths we hope to make the service as simple to use as possible.

The automatic deletion of old files is based on the same concept we have used in the auto-purging scratch space called /data/autoScratch/weekly/$USER and /data/autoScratch/monthly/$USER. We will be looking to phase these weekly/monthly services out within the next few weeks or months in favour of the faster and simpler /data/scratch/$USER file path. This is partly due to the disks these service are based on reaching end-of-life. All users should start changing their job scripts to use the new service instead. The long-term storage (home directories and lab shares) remains unchanged.

Action required

Change your job scripts to use /data/scratch/$USER path (e.g. /data/scratch/abc123) instead of /data/autoScratch. Also check for presence of any symbolic links that point to the autoScratch service.

Files placed in /data/scratch are deleted 35 days after last modification. You will be sent warning emails that files are about to be deleted after 31 days (4 days before deletion). These numbers have been selected based on available capacity and observed behaviour. The service has loose capacity-based quotas of 1TB or 1 million files, and requests to extend this will be considered on an individual basis.

Scratch is not backed up

Scratch data is not backed up and any important data should be moved to other locations as quickly as possible. Data stored here will be automatically deleted. An email is sent to you shortly before this occurs.

Please do not abuse this service by modifying timestamps of your files to make this a permanent location as it is unfair to other users. It is far better to ensure that files are correctly backed up in the right place.