Skip to content

Charge loop device i/o to issuing cgroup

Dan Schatzberg cgroups linux-block linux-fsdevel linux-mm lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest4 patches 5 repliesIn review

Changes since V3 from the cover letter

* Fix race on loop device destruction and deferred worker cleanup
* Ensure charge on shmem_swapin_page works just like getpage
* Minor style changes

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4loop: Use worker per cgroup instead of kworkerdrivers/block/loop.c drivers/block/loop.h+180 -380 replies
2/4mm: support nesting memalloc_use_memcg()fs/buffer.c fs/notify/fanotify/fanotify.c fs/notify/inotify/inotify_fsnotify.c +1 more+18 -262 replies
3/4mm: Charge active memcg when no mm is setmm/memcontrol.c mm/shmem.c+10 -50 replies
4/4loop: Charge i/o to mem and blk cgdrivers/block/loop.c drivers/block/loop.h include/linux/memcontrol.h +1 more+48 -210 replies

Reviews trailers per patch

3 of 4 patches have a Reviewed-by or Acked-by.

PatchJohannes WeinerTejun HeoChris DownShakeel ButtRoman Gushchin
1/4
2/4R-b A-b
3/4A-b A-b A-b R-b
4/4A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 5 reviewers.

Read the v1 thread

Cover letter [PATCH 0/4] Charge loop device i/o to issuing cgroup

Changes since V4:

Only patches 1 and 2 have changed.

* Fixed irq lock ordering bug
* Simplified loop detach
* Added support for nesting memalloc_use_memcg

Changes since V3:

* Fix race on loop device destruction and deferred worker cleanup
* Ensure charge on shmem_swapin_page works just like getpage
* Minor style changes

Changes since V2:

* Deferred destruction of workqueue items so in the common case there is no allocation needed

Changes since V1:

* Split out and reordered patches so cgroup charging changes are separate from kworker -> workqueue change

* Add mem_css to struct loop_cmd to simplify logic

The loop device runs all i/o to the backing file on a separate kworker thread which results in all i/o being charged to the root cgroup. This allows a loop device to be used to trivially bypass resource limits and other policy. This patch series fixes this gap in accounting.

A simple script to demonstrate this behavior on cgroupv2 machine:

'''
#!/bin/bash
set -e

CGROUP=/sys/fs/cgroup/test.slice
LOOP_DEV=/dev/loop0

if [[ ! -d $CGROUP ]]
then
sudo mkdir $CGROUP
fi

grep oom_kill $CGROUP/memory.events

# Set a memory limit, write more than that limit to tmpfs -> OOM kill sudo unshare -m bash -c "
echo \$\$ > $CGROUP/cgroup.procs;
echo 0 > $CGROUP/memory.swap.max;
echo 64M > $CGROUP/memory.max;
mount -t tmpfs -o size=512m tmpfs /tmp;
dd if=/dev/zero of=/tmp/file bs=1M count=256" || true

grep oom_kill $CGROUP/memory.events

# Set a memory limit, write more than that limit through loopback

# device -> no OOM kill
sudo unshare -m bash -c "
echo \$\$ > $CGROUP/cgroup.procs;
echo 0 > $CGROUP/memory.swap.max;
echo 64M > $CGROUP/memory.max;
mount -t tmpfs -o size=512m tmpfs /tmp;
truncate -s 512m /tmp/backing_file
losetup $LOOP_DEV /tmp/backing_file
dd if=/dev/zero of=$LOOP_DEV bs=1M count=256;
losetup -D $LOOP_DEV" || true

grep oom_kill $CGROUP/memory.events
'''

Naively charging cgroups could result in priority inversions through the single kworker thread in the case where multiple cgroups are reading/writing to the same loop device. This patch series does some minor modification to the loop driver so that each cgroup can make forward progress independently to avoid this inversion.

With this patch series applied, the above script triggers OOM kills when writing through the loop device as expected.

Dan Schatzberg (4):
loop: Use worker per cgroup instead of kworker
mm: support nesting memalloc_use_memcg()
mm: Charge active memcg when no mm is set
loop: Charge i/o to mem and blk cg

 drivers/block/loop.c                 | 246 ++++++++++++++++++++++-----
 drivers/block/loop.h                 |  14 +-
 fs/buffer.c                          |   6 +-
 fs/notify/fanotify/fanotify.c        |   5 +-
 fs/notify/inotify/inotify_fsnotify.c |   5 +-
 include/linux/memcontrol.h           |   6 +
 include/linux/sched/mm.h             |  28 +--
 kernel/cgroup/cgroup.c               |   1 +
 mm/memcontrol.c                      |  11 +-
 mm/shmem.c                           |   4 +-
 10 files changed, 246 insertions(+), 80 deletions(-)
Signature · 1 line
2.24.1