File Permissions¶

An understanding of file permissions is important to the success of computational jobs, and the security of your files.

The default settings are suitable for some, but not every use-case: without sufficient awareness, your files may be visible to people who should not be able to access them, and vice-versa.

Personally Identifiable Data

We do not allow personally identifiable data to be stored on the cluster. If your data is personal please ensure it is stored correctly in line with the Data Protection policy. Please contact us if you are unsure, so we can discuss your options

About half of all the storage support requests we get are regarding file permissions. This blog post will address common issues.

Every operating system uses file permissions, and while these may differ, they all follow the same principles.

The storage on Apocrita is based on the standard POSIX file permissions of User, Group, and Other. However, this is extended with Access Control Lists (ACLs) which make it possible to grant access to multiple groups or individuals in finer detail.

Basics¶

With standard POSIX/Linux permissions, each file has three sets of permissions, relating to owner, group and everyone else (commonly known as "other"). By default, new files are owned by your own username and default group.

The default group on the cluster is qmul, although if you are an external collaborator, it might be external instead.

Each file permission for the user, group and other can be assigned any of the following:

read - user/group/other can read the file - signified by the 'r' character
write - user/group/other can write to the file - signified by the 'w' character
execute - user/group/other can run the file, or if it's a directory they can list the contents - signified by the 'x' character

These will normally be listed as a series of letters but can also be represented in octal form with a digit for each of the user, group and "other" categories:

0 - no permissions
1 - ‘—x’ : execute
2 - ‘-w-’ : write
4 - ‘r—’ : read
3 : ‘-wx’ : writable and executable
5 : ‘r-x’ : readable and executable
6 : ‘rw-’ : readable and writable
7 : ‘rwx’ : read, write and execute.

You can see the permissions of a file by using ls -l

[abc123 ~]$ ls -l script.sh
-rwxrw-r-- 1 abc123 qmul 1483838 Nov 9 2021 script.sh

This rwxrw-r-- format means:

owner has read, write and execute permission (rwx)
group has read and write permission (rw-)
everyone/other has read permission (r--)

This can be represented in octal form as 761.

Permission Denied errors

A common issue experienced by new users is receiving a permission denied error when trying to execute a script they have written. The error usually means that they need to add execute permissions to their file, rather than any other lack of user access privileges."

The default permissions are set based on the umask, which specifies the permissions which you don't want to see by default on newly created files. It can get a little complicated, since removal of a permission doesn't necessarily mean that all permission is granted when you don't remove permissions. For example, umask 077 ensures that rwx is not applied to a new file. Note that the user/owner execute permission is not added by the touch command by default though.

$ umask 077
$ mkdir dir
$ touch file
$ ls -ld dir file

drwx------ 2 abc123 qmul 4096 2021-11-20 12:05 dir
-rw------- 1 abc123 qmul 0    2021-11-20 12:05 file

Inheritance¶

Group shares have been designed to inherit group ownership from the parent directory - files created in a group share will be be limited to your group and not everyone.

A side-effect of this is that files created in your home directory and later moved into a group share (with mv) will keep the original permissions and other members of the group will not have the correct permissions. This issue is exacerbated if you create a file in a folder you moved. If you are moving files into a group share it is always best to copy them and then delete the originals for this reason.

You can temporarily change your current default group for the duration of your login session with newgrp <groupname>. This can be a time-saver when you're working with filesets owned by different groups, and using it before copying and moving files around can help you ensure that file permissions are correct.

Changing Permissions¶

It's not possible for standard users to change the ownership of a file. To claim ownership of a file owned by someone else, you can take a copy of a file - the resulting file will be owned by you. To delete the original file you will need someone who has write permissions on that file. You can always ask our team for help if you don't have permissions to perform certain tasks. Where necessary, we will seek approval from the file owner first though.

You can change the permissions on any file you own. For example chmod g-w file will revoke write access to file by the group, and chmod g+r will add group read permission.

To add owner execute permissions to file:

$ chmod u+x file
$ ls -l file
-rwrx------ 1 abc123 qmul 0    2021-11-20 12:05 file

You can also change the group ownership of a file to any other group you are also a member of, so if you're a member of multiple groups you can do a chgrp newgroup file which will change the group ownership of file from the default qmul group to newgroup

Access Control Lists¶

Access Control Lists extend the standard POSIX view of user/group/other and allow you to grant permission to additional groups and/or users. They are very flexible but can also cause more confusion.

While the standard Linux file ACL commands (such as getfacl and setfacl) may work, they can still give a limited view of what is really going on. If you really want to see what the permissions are you should use mmgetacl and mmsetacl. A detailed look at ACLs is out of scope for this article, but it's good be aware that if you have particular requirements that aren't satisfied by the standard permissions, we may be able to help.

Even More Permissions¶

The immutable flag which can be set with mmchattr -i yes means a file can't be changed without unsetting this flag first. We use this on the README file in the top directory of a shared Research group area, so that the users within the group can't edit, change or remove that file. Additionally, you may also come across sticky bit (commonly used on the tmp directory to allow everyone to write files that only the owner can delete) and setuid/setgid permissions.

Conclusion¶

It's important for all users to learn the basics of file permissions to ensure the visibility and ability to modify their data is neither too permissive, or too restrictive. Additionally, if cluster jobs fail due to permission issues, an understanding of the issues covered here will help you to troubleshoot why this occurred, and how to resolve the issue yourself, or escalating to us if administrative privileges are required.

Title image: Maxim Zhgulev on Unsplash