File permissions#
Access to files and directories is controlled by setting permissions, on a UNIX system the base permissions are set using POSIX and can be extended with Access Control Lists (ACLs).
POSIX#
Most UNIX file systems use the Portable Operating System Interface (POSIX) to control who can access files and directories. POSIX uses a combination of three groups of users, known as access classes
:
user
each user identified by a user-idgroup
a group of users identified by a group-id, on NCI groups are usually associated to a project, and occasionally to a specific roleother
the definition of this class depends on each file/directory context, as it represents anyone else who is not the user or a member of the group.
Each file or directory is owned by only one user and one group, only the owner or a system administrator can change its permissions.
There are three kinds of permissions:
read
- the permission to read a file. When set for a directory, only names of files in the directory can be listed, other details like file type, size, ownership, permissions etc. are not visible.write
- the permission to modify or remove a file. When set for a directory, it grants the ability to modify entries in the directory, including creating, deleting, and renaming files. Note that, without execute permission on a directory, the write permission does not take any effect.execute
- the permission to execute a file. When set for a directory, it is interpreted as the search permission: it grants the ability to access file contents and meta-information if its name is known, but does not list files inside the directory, unless read is set.
Permission example
This is an example of what the permissions look like when we list files using the long l
option.
ls -l
-rw-r--r-- 1 pxp581 ua8 18K May 16 2016 readme.txt
drwx------ 5 pxp581 ua8 512 May 21 2020 private/
lrwxrwxrwx 1 pxp581 ua8 Sep 13 20:45 data -> /g/data/ua8/Climdex/v1-0/
-rwxr--r-- 1 pxp581 ua8 171 Sep 9 09:18 loadimage.sh
drwxrwxrwx 1 pxp581 ua8 512 Sep 13 20:45 worldwriteable/
^ ^ ^ ^--- other permissions
| | '------- group permissions
| '---------- user permissions
'------------ file type: d=directory, l=symbolic link, - means regular file
NB When listing a directory content with ls -l
, an additional “+” character is displayed after the permissions of all files that have extended ACLs (see below for more information).
Changing permissions#
A file/directory owner is the only user that can set and change a file’s permissions and/or group (aside from a system administrator). Unix commands are:
chmod
to change permissions;chgrp
to change groupchgrp
;chown
to change owner.
Warning
chown
can only be used by an NCI system administrator. A request can be made to the NCI helpdesk and both the current and new owner should be cc-ed, occasionally NCI might also request the Lead CI of the project to authorise the transaction.
In most cases just using the POSIX permission system is sufficient to handle who can view or write a specific file. The limitations of this system are obvious when there is the need to set different levels of permissions for different groups of users.
In that case, the distinction between other
and group
might not be sufficient.
An example of this is when managing a data collection at NCI. NCI policy dictates only users in the group can view a project’s files. It is not allowed to have an other
reading permission at the main project directory level.
This means that it is not possible to have only a subset of the group users (i.e., the data managers) having writing permission.
This is where Access Control Lists (ACLs) enter into play, as with ACLs is possible to set permissions for a different group and/or user.
In the data management context this usually happens by setting a writers group which can be joined by all the managers and then using ACLs to give this group writing permission.
Access Control Lists#
Access Control Lists (ACLs) are used to impose a finer level of permissions control on files and directories.
ACLs cheat sheet#
Listing ACLs: getfacl
getfacl <file/dir>
Shows file/dir ACLs. If ACLs are set, a line like “mask::r-x” should be present.
ls -l dir
should also show a ‘+’ at the end of permissions if ACLs are set
Eg. drwxr-s—+
getfacl -h
Display help
getfacl -a somedir
Display the ACLs for files in somedir only
getfacl -d somedir
Display the default ACLs for somedir only
u
for user
g
for group
o
for others as in the usual permissions
getfacl --omit-header somedir
Will show only the actual permissions and not the header showing owner, group and flags
Setting ACLs: setfacl
setfacl -h
Display help
setfacl -m u:user1:r-x onefile
Give user1 r-x access to onefile,
-m stands for modify
setfacl -m u:user1:rwx file, g:group1:r-x
Give user1 rwx access and group1 r-x access to file
setfacl -m d:u:user1:r-x somedir
Give user1 default r-x access to somedir, any new files in somedir will inherit this permission
setfacl -R -m d:u:user1:r-x somedir
Give user1 default r-x access to somedir and recursively to all its subdirectories
setfacl -L -m d:u:user1:r-x somedir
Give user1 default r-x access to somedir following symbolic links
Removing ACLs
setfacl -x u:user1 somedir
Remove all ACLs for user1 in somedir
setfacl -b somedir
Remove all ACLs for directory somedir
setfacl -k somedir
Remove all default ACLs for directory somedir
Backing up and restoring ACLs
getfacl -R somedir > somedir_acls
Save ACLs directive for somedir to somedir_acls file
setfacl --restore=somedir_acls
Restore ACLs directives for somedir from the somedir_acls file
Warning
Only the file/directory owner can change ACLs, as for any other permissions. If a user runs this command on a directory containing some files they do not own, they will get a warning like “Operation not permitted”. These warnings can be safely ignored, if there are files that they own, the ACLs will be successfully updated for these.
Troubleshooting#
ACLs are very useful but tend to get broken easily.
Let’s see first how to interpret the getfacl
output
getfacl Download/
# file: Download/
# owner: pxp581
# group: ua8
# flags: -s-
user::rwx
user:ua8_nfs:rwx
group::r-x
mask::rwx
other::--x
default:user::rwx
default:user:pxp581:rwx
default:group::r-x
default:mask::rwx
default:other::r-x
The first three lines refer to the normal POSIX permissions, in this case we can see that pxp581 is the owner, ua8 the group and that this directory has setgid (flag: s) set. So, any new file in it will belong to ua8 too.
The next 5 lines show the POSIX permissions (user, group and other), a permission for user ua8_nfs to rwx
set by ACLs and the mask also rwx
.
What the mask
does is to set the upper boundary of what permissions can be.
It affects only the permissions of the group and/or any extended user and group added by ACLs. The mask has no affect on other or the main user permissions.
However, as the directory has setgid, set the directory other permissions act as a mask for the file other permissions. So, if a file with write permissions was copied in here, the new file will still have only r-x
permissions, as the file can’t get more permissions on ‘other’ than the directory other r-x permissions.
Order of operations
When a process requests access to a file system object, two steps are performed.
The ACL entry that most closely matches the requesting process is selected. The ACL entries are looked at in the following order: owner, named users, (owning or named) groups, others. Only the highest ranking entry determines access: i.e. if the user is the owner this will have precedence on the group permissions.
The system checks if the matching entry contains sufficient permissions to execute action.
The effective permissions are set to the permissions defined in the mode
, minus the permissions set in the current umask
. However, the umask
has no affect if a default ACL exists.
The mask entry is automatically created when needed but not provided. Its permissions are set to the union of the permissions of all entries that are in the group class, so initially the mask entry does not mask any permissions.
Here is an example of ‘broken’ ACLs:
# file: test.txt
# owner: pxp581
# group: hh5
user::rw-
group::r-x #effective:r--
group:w40:rwx #effective:rw-
group:hh5_w:rwx #effective:rw-
mask::rw-
other::r--
As some process has set the mask
to rw-
, the group will have effectively r--
instead of r-x
permissions. And similarly, the groups w40 and hh5_w ACL entry contains permissions that are disabled by the mask entry, so getfacl
adds a comment that shows the effective set of permissions granted by that entry.
If we grant again writing permissions to the group using chmod
the mask will change, and the ACLs will be again effective. The ACLs can be disabled but not altered by chmod
.
Any command that sets the mode parameter
can have unexpected effects on ACLs. Aside from setacls
and chmod
, these are usually commands used to create, move, and copy files.
mkdir
Unless otherwise specified, the mkdir
command uses a value of 0777 as the mode parameter to the mkdir
system call, which it uses for creating the new directory.
This corresponds to a+rwx,ug-s,-t
.
If this is run in a directory with ACLs set, all permissions not included in the mode parameter are removed from the corresponding ACL entries, but there is no noticeable affect because the value 0777 used for the mode parameter represents a full set of permissions.
cp
cp
will preserve the permissions of the original file, but it will change the group depending on the setgid bit of the destination folder.
As an example:
cp test.txt /g/data/ua8/.
the new test.txt file in ua8 will keep the same set of permissions of the original but with the ua8 group.
Using the archive flag -a
:
cp -a test.txt /g/data/ua8/.
the new test.txt file in ua8 will preserve the same set of permissions and group of the original file.
Adding the ACLs in the mix:
setfacl -m d:u:ua8_nfs:rwx Test
the full set of permissions for the Test folder is now:
getfacl --omit-header Test/
user::rwx
group::r-x
other::r-x
default:user::rwx
default:user:ua8_nfs:rwx
default:group::r-x
default:mask::rwx
default:other::r-x
A new test.txt file with group w40 and rwx
only for the owner pxp581 is created and then copied to Test:
cp test.txt /g/data/ua8/Test/.
The resulting permissions for the copy are:
# file: test.txt
# owner: pxp581
# group: ua8
user::rw-
user:ua8_nfs:rwx #effective:---
group::r-x #effective:---
mask::---
other::---
The group changed to ua8 as expected but the original file POSIX permissions are setting a mask to ---
which gets mapped to the ua8_nfs user and the group permissions.
Finally, in the case the file to copy has ACLs set, running cp -a
will preserve the permissions in the copy regardless that the destination has or not ACLs set.
Without the archive flag, the copy will lose all the ACLs and its permissions will be set depending on the directory settings.
mv
Move (mv) behaves as an attribute-preserving copy followed by a deletion (rm), as far as permissions are concerned.
mv test.txt /g/data/ua8/.
test.txt file will retain both permissions and group without need of specifying any mode preserving flag.
Warning
So, after moving files to a new project, it’s a good idea to run
chgrp <new-project-id>
on all files that have been moved.
Moving a file to a directory with ACLs, for example Test from the copy example:
mv test.txt /g/data/ua8/Test/.
will produce the same effective permissions, as it will retain completely the permissions and group of the original file, basically ignoring the ACLs completely.
# file: test.txt
# owner: pxp581
# group: w40
user::rw-
group::---
other::---
Finally, in the case the file to move has ACLs set, running mv
will preserve the permissions regardless of the destination ACLs set (or not set).
Warning
Moving a file across the same file system means removing its directory entry from its containing directory. The writing operation affects the directory, not the file itself, hence no write permissions on the file are necessary to move it!
rsync
rsync
will behave similarly to cp
when run without any flag. So, if the ACLs are set on the destination, the files will inherit them.
Like cp
rsync has an archive flag -a
which should preserve both permissions and group.
Differently from cp
, if ACLs are set, even when using the archive flag, the permissions and group will not be preserved. The archive
flag, as well as the perms
and group
flags, are ignored if there are ACLs.
If rsync-ing a directory with ACLs set, as Test in previous examples, to a directory with no ACLs:
rsync -r Test Test-no-acls/.
The original directory ACLs are completely ignored, the same happens for the files contained in it.
To preserve them rsync has a specific flag -A
rsync -rA Test Test-no-acls/.
this will preserve the ACLs. If run with the additional archive flag -raA
, the POSIX permissions and group will also be inherited by the new directory.
Note
The -A flag requires source and destination to have compatible ACL systems, when this is not the case a potential solution in rsync is to use the --chmod
option.
From the rsync man page (in the –perms section of the manual):
To give new files the destination-default permissions (while leaving existing files unchanged), make sure that the –perms option is off and use
--chmod=ugo=rwX
(which ensures that all non-masked bits get enabled).
NB the order of flags is important: -a, -A and so on can in fact, be used but MUST come before the –chmod flag. This tip is from ref (4).
touch
The touch command passes a mode value of 0666 to the kernel for creating the file. So, all classes will have rw-
permissions and the group and ACLs extended classes will be mapped by a rw-
mask.
Warning
Please note that if the ACLs were ignored because mv
or cp -a
were used, simply copying the file again with cp
only won’t preserve the ACLs, the file needs to be deleted first and then copied again.