File System Locking
Locking files in Linux is tricky - not because it is complex, but complicated due to the many variants.
First of all you have different kinds:
- mandatory locks are enforced by the Linux kernel and prevent other processes from opening the file while another process has an exclusive lock. This can easily deadlock the system. Linux supports this on file systems explicitly mounted with option
mand
. - advisory locks require support from all applications: each one has to implement the same mechanism to guarantee proper cooperation.
For advisory locks you have multiple types in Linux:
- BSD file lock (<man:flock(2)>)
- POSIX record lock (<man:fnctl(2)>; the simplified version <man:lockf(3)> only supporting exclusive locks)
- Open file descriptor (OFD) lock (<man:fcntl(2)>)
Basically they are incompatible with each other and your applications should agree to use only one.
Variant | Origin | NFS | range | associated | fork() |
auto released |
---|---|---|---|---|---|---|
FLOCK | BSD | no¹ | file | open FD | inherited | last FD closed |
POSIX | POSIX | yes | bytes | pid,inode | dropped | any FDs closed |
OFDLCK | Linux | yes | bytes | open FD | inherited | last FD closed |
BSD file locks
- only complete files can be locked, not parts
- NFS support may not work, only since Linux 2.6.11 (¹)
- mode change is not atomic
- inherited on
fork()
- released when last FD is
close()
d - Linux type
FLOCK
in/proc/locks
- API:
flock(fd, LOCK_{SH,EX,UN})
<man:flock(2)> - for shell scripts there is <man:flock(1)>
POSIX record lock
- very strange behavior: lock is associated with
(pid,inode)
tuple - closing any duplicated file descriptor will release all locks
fork()
ed child process does not inherit locks- does not work well with threads
- Linux type
POSIX
in/proc/locks
- API:
fcntl(fd, F_{GETLK,SETLK,SETLKW}, struct flock *)
<man:fcntl(2)> - API:
lockf(fd, F_{LOCK,TLOCK,ULOCK,TEST}, len)
<man:lockf(3)>
Open File Descriptor locks (file-private POSIX locks)
- Linux specific, only since Linux 3.15+
- released when last FD is
close()
d - thread may open files themselves and get their own locks
- Linux type
OFDLCK
in/proc/locks
- API:
fcntl(fd, F_OFD_{GETLK,SETLK,SETLKW}, struct flock *)
<man:fcntl(2)>
Python
In Python fnctl.lockf() is a wrapper using fcntl.ioctl()
directly.
It is implemented in CPython:fcntlmodule.c.
As such despite the name it does not use the implementation from Glibc:lockf.c.
Linux
The Linux kernel tracks locks in /proc/locks
by inode.
See <man:proc(5)> for the format.
Further reading
Written on July 6, 2022