Skip to content

bpf, cgroup: Don't wait for cgroup_mutex to read or free cgroup bpf state

Shakeel Butt bpf cgroups lkml sashiko-reviews 1 version Applied

Status from this message: This series was applied to bpf/bpf.git (master)

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest3 patches 2 repliesApplied 3/3

Patches in v1 All 3 applied

#Subject and filesSizeRepliesApplied
1/3bpf, cgroup: Don't read freed effective arrays in BPF_PROG_QUERYkernel/bpf/cgroup.c+8 -00 repliesApplied
2/3bpf, cgroup: Don't attach to a removed cgroupkernel/bpf/cgroup.c+13 -00 repliesApplied
3/3bpf, cgroup: Protect cgroup_bpf state with its own mutexinclude/linux/bpf-cgroup-defs.h include/linux/bpf-cgroup.h kernel/bpf/cgroup.c +1 more+102 -471 replyApplied

Reviews trailers per patch

None of the 3 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchTejun Heo
SeriesA-b
1/3
2/3
3/3

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH bpf-next 0/3] bpf, cgroup: Don't wait for cgroup_mutex to read or free cgroup bpf state

In the Meta fleet, BPF_PROG_QUERY callers, link-info readers and the cgroup bpf release work often wait minutes for cgroup_mutex. The holder is almost always a cgroup mkdir or rmdir stuck on kernfs_rwsem. In one day, hung-task reports showed this on 1,075 hosts (query), 94 hosts (link info) and 458 hosts (release work).

Patch 3 gives the cgroup bpf state its own mutex, so these paths no longer take cgroup_mutex. Attach, detach and replace still take cgroup_mutex first, because they walk the descendants.

Patches 1 and 2 fix two old races. Attach and query get the cgroup from an fd and take the lock later, without checking whether the cgroup's bpf state was released in between. An effective query then reads freed memory (patch 1). Attach leaks the program, or with cgroup storage writes to a freed cgroup later (patch 2). Patch 3 makes the attach race easy to hit, so patch 2 must come before it. Both fixes have Fixes tags.

Testing, in an x86-64 VM with lockdep, PROVE_RCU and KASAN:

- The 41 cgroup bpf selftests pass. A 60-second stress of mkdir/rmdir, attach/detach, links, queries, fdinfo, storage map free and sockets in dying cgroups shows no lockdep or KASAN reports.
- Debug-only delays reproduce both races. Without the fixes, KASAN reports use-after-free in cgroup_bpf_prog_query() and, with patch 3, in cgroup_storage_map_free(), and BPF_PROG_ATTACH leaks the program. With the fixes, attach and query return -ENOENT.
- While a mkdir holds cgroup_mutex for 5 seconds, query and link info return in under a millisecond instead of 4.6 seconds. Attach and detach still wait.
- Builds with CGROUP_BPF=n and with BPF_LSM=n.

Shakeel Butt (3):
bpf, cgroup: Don't read freed effective arrays in BPF_PROG_QUERY bpf, cgroup: Don't attach to a removed cgroup
bpf, cgroup: Protect cgroup_bpf state with its own mutex

 include/linux/bpf-cgroup-defs.h |   1 +
 include/linux/bpf-cgroup.h      |   3 +
 kernel/bpf/cgroup.c             | 158 +++++++++++++++++++++++---------
 kernel/bpf/local_storage.c      |   8 +-
 4 files changed, 123 insertions(+), 47 deletions(-)

base-commit: 0fb18ce8a2e9de830433923cf65838933d615359

Signature · 1 line
2.53.0-Meta