File System Locking

Locking files in Linux is tricky - not because it is complex, but complicated due to the many variants.

First of all you have different kinds:

  • mandatory locks are enforced by the Linux kernel and prevent other processes from opening the file while another process has an exclusive lock. This can easily deadlock the system. Linux supports this on file systems explicitly mounted with option mand.
  • advisory locks require support from all applications: each one has to implement the same mechanism to guarantee proper cooperation.

For advisory locks you have multiple types in Linux:

  • BSD file lock (<man:flock(2)>)
  • POSIX record lock (<man:fnctl(2)>; the simplified version <man:lockf(3)> only supporting exclusive locks)
  • Open file descriptor (OFD) lock (<man:fcntl(2)>)

Basically they are incompatible with each other and your applications should agree to use only one.

Variant Origin NFS range associated fork() auto released
BSD BSD no¹ file open FD inherited last FD closed
POSIX POSIX yes bytes pid,inode dropped any FDs closed
OFD Linux yes bytes open FD inherited last FD closed

BSD file locks

  • only complete files can be locked, not parts
  • NFS support may not work, only since Linux 2.6.11 (¹)
  • mode change is not atomic
  • inherited on fork()
  • released when last FD is closed()

POSIX record lock

  • very strange behavior: lock is associated with (pid,inode) tuple
  • closing any duplicated file descriptor will release all locks
  • fork()ed child process does not inherit locks
  • does not work well with threads

Open File Descriptor locks (file-private POSIX locks)

  • Linux specific
  • released when last FD is closed()
  • thread may open files themselves and get their own locks

Python

In Python fnctl.lockf() is a wrapper using fcntl.ioctl() directly. It is implemented in CPython:fcntlmodule.c. It does not use the implementation from Glibc:lockf.c.

Further reading

Written on July 6, 2022