File System Locking

Locking files in Linux is tricky - not because it is complex, but complicated due to the many variants.

First of all you have different kinds:

  • mandatory locks are enforced by the Linux kernel and prevent other processes from opening the file while another process has an exclusive lock. This can easily deadlock the system. Linux supports this on file systems explicitly mounted with option mand.
  • advisory locks require support from all applications: each one has to implement the same mechanism to guarantee proper cooperation.

For advisory locks you have multiple types in Linux:

  • BSD file lock (<man:flock(2)>)
  • POSIX record lock (<man:fnctl(2)>; the simplified version <man:lockf(3)> only supporting exclusive locks)
  • Open file descriptor (OFD) lock (<man:fcntl(2)>)

Basically they are incompatible with each other and your applications should agree to use only one.

Variant Origin NFS range associated fork() auto released
FLOCK BSD no¹ file open FD inherited last FD closed
POSIX POSIX yes bytes pid,inode dropped any FDs closed
OFDLCK Linux yes bytes open FD inherited last FD closed

BSD file locks

  • only complete files can be locked, not parts
  • NFS support may not work, only since Linux 2.6.11 (¹)
  • mode change is not atomic
  • inherited on fork()
  • released when last FD is close()d
  • Linux type FLOCK in /proc/locks
  • API: flock(fd, LOCK_{SH,EX,UN}) <man:flock(2)>
  • for shell scripts there is <man:flock(1)>

POSIX record lock

  • very strange behavior: lock is associated with (pid,inode) tuple
  • closing any duplicated file descriptor will release all locks
  • fork()ed child process does not inherit locks
  • does not work well with threads
  • Linux type POSIX in /proc/locks
  • API: fcntl(fd, F_{GETLK,SETLK,SETLKW}, struct flock *) <man:fcntl(2)>
  • API: lockf(fd, F_{LOCK,TLOCK,ULOCK,TEST}, len) <man:lockf(3)>

Open File Descriptor locks (file-private POSIX locks)

  • Linux specific, only since Linux 3.15+
  • released when last FD is close()d
  • thread may open files themselves and get their own locks
  • Linux type OFDLCK in /proc/locks
  • API: fcntl(fd, F_OFD_{GETLK,SETLK,SETLKW}, struct flock *) <man:fcntl(2)>

Python

In Python fnctl.lockf() is a wrapper using fcntl.ioctl() directly. It is implemented in CPython:fcntlmodule.c. As such despite the name it does not use the implementation from Glibc:lockf.c.

Linux

The Linux kernel tracks locks in /proc/locks by inode. See <man:proc(5)> for the format.

Further reading

Written on July 6, 2022