Writeback Cgroup/Dirty Throttle: very small buffered write thoughput caused by writeback cgroup and dirty thottle

Hi, Tejun and Fengguang

I found that buffered write thoughput was dropped down by writeback cgroup and dirty thottle on 4.6-rc7 kernel. If I ran benchmark on the top block cgroup, the thoughput was more than 1500MB/s. If I ran benchmark on a new block cgroup, the thoughput was down to 4MB/s.

Steps to reproduce:

# mount -t cgroup2 cgroup <cgrp_mnt>
# echo "+io +memory" > <cgrp_mnt>/cgroup.subtree_control
# mkdir <cgrp_mnt>/aaa
# echo $$ > <cgrp_mnt>/aaa/cgroup.procs
# fio test.config
job0: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.2.8
Starting 1 thread
Jobs: 1 (f=1): [W(1)] [3.7% done] [0KB/4000KB/0KB /s] [0/1000/0 iops] [eta 04m:50s]

Fio configuration is:
[global]
bs=4K
direct=0
ioengine=psync
iodepth=1
directory=/mnt/ext4/tstdir0
time_based
runtime=300
group_reporting
size=16G
sync=0
max_latency=120000000
thread

[job0]
numjobs=1
rw=write

My box has 48 cores and 188GB memory, but I set
vm.dirty_background_bytes = 268435456
vm.dirty_bytes = 536870912

if I set vm.dirty_background_bytes and vm.dirty_bytes to be a large number(vm.dirty_background_bytes = 3GB, vm.dirty_bytes = 4GB), then fio thoughput would be more than 1500MB/s. and then if I reset them to the original value(the above ones), the thoughout would be down to 500MB/s.

And according my debug, I found fio sleeped for 1ms every time we dirty a page(balance dirty pages) when the thoughput was down to 4MB/s, it might be a bug of dirty throttle when we open write back cgroup, I think.

Tejun and Fengguang, please let me know what you guys think about this issue, and if you have any suggestions for possible solutions, Any input is greatly appreciated!

Thanks
Miao

Quoted 10 lines

My box has 48 cores and 188GB memory, but I set
vm.dirty_background_bytes = 268435456
vm.dirty_bytes = 536870912

if I set vm.dirty_background_bytes and vm.dirty_bytes to be a large number(vm.dirty_background_bytes = 3GB, vm.dirty_bytes = 4GB), then fio thoughput would be more than 1500MB/s. and then if I reset them to the original value(the above ones), the thoughout would be down to 500MB/s.

And according my debug, I found fio sleeped for 1ms every time we dirty a page(balance dirty pages) when the thoughput was down to 4MB/s, it might be a bug of dirty throttle when we open write back cgroup, I think.

Quoted 10 lines · Miao Xie

Quoted 10 lines

My box has 48 cores and 188GB memory, but I set
vm.dirty_background_bytes = 268435456
vm.dirty_bytes = 536870912

if I set vm.dirty_background_bytes and vm.dirty_bytes to be a large number(vm.dirty_background_bytes = 3GB, vm.dirty_bytes = 4GB), then fio thoughput would be more than 1500MB/s. and then if I reset them to the original value(the above ones), the thoughout would be down to 500MB/s.

And according my debug, I found fio sleeped for 1ms every time we dirty a page(balance dirty pages) when the thoughput was down to 4MB/s, it might be a bug of dirty throttle when we open write back cgroup, I think.

Heh, so, for cgroups, the absolute byte limits can't applied directly and converted to percentage value before being applied. You're specifying 0.27% for threshold. Unfortunately, the ratio is translated into a percentage number and 0.27% becomes 0, so your cgroups are always over limit and being throttled.

Can you please see whether the following patch fixes the issue?

Thanks.

mm/page-writeback.c +12 -9

index 999792d..a455a21 100644
@@ -369,8 +369,9 @@ static void domain_dirty_limits(struct dirty_throttle_control *dtc)
  	struct dirty_throttle_control *gdtc = mdtc_gdtc(dtc);
  	unsigned long bytes = vm_dirty_bytes;
  	unsigned long bg_bytes = dirty_background_bytes;
-	unsigned long ratio = vm_dirty_ratio;
-	unsigned long bg_ratio = dirty_background_ratio;
+	/* convert ratios to per-PAGE_SIZE for higher precision */
+	unsigned long ratio = (vm_dirty_ratio * PAGE_SIZE) / 100;
+	unsigned long bg_ratio = (dirty_background_ratio * PAGE_SIZE) / 100;
  	unsigned long thresh;
  	unsigned long bg_thresh;
  	struct task_struct *tsk;

@@ -382,26 +383,28 @@ static void domain_dirty_limits(struct dirty_throttle_control *dtc)
  		/*
  		 * The byte settings can't be applied directly to memcg
  		 * domains.  Convert them to ratios by scaling against
-		 * globally available memory.
+		 * globally available memory.  As the ratios are in
+		 * per-PAGE_SIZE, they can be obtained by dividing bytes by
+		 * pages.
  		 */
  		if (bytes)
-			ratio = min(DIV_ROUND_UP(bytes, PAGE_SIZE) * 100 /
-				    global_avail, 100UL);
+			ratio = min(DIV_ROUND_UP(bytes, global_avail),
+				    PAGE_SIZE);
  		if (bg_bytes)
-			bg_ratio = min(DIV_ROUND_UP(bg_bytes, PAGE_SIZE) * 100 /
-				       global_avail, 100UL);
+			bg_ratio = min(DIV_ROUND_UP(bg_bytes, global_avail),
+				       PAGE_SIZE);
  		bytes = bg_bytes = 0;
  	}
 
  	if (bytes)
  		thresh = DIV_ROUND_UP(bytes, PAGE_SIZE);
  	else
-		thresh = (ratio * available_memory) / 100;
+		thresh = (ratio * available_memory) / PAGE_SIZE;
 
  	if (bg_bytes)
  		bg_thresh = DIV_ROUND_UP(bg_bytes, PAGE_SIZE);
  	else
-		bg_thresh = (bg_ratio * available_memory) / 100;
+		bg_thresh = (bg_ratio * available_memory) / PAGE_SIZE;
 
  	if (bg_thresh >= thresh)
  		bg_thresh = thresh / 2;

.

Better than the kernel without patch. Now the benchmark could reach the device bandwidth after 5-8 seconds. But at the beginning, it was still very slow, and its thoughput was only 4MB/s for ~4 seconds, then it could go up in 1~3 seconds.

As vm.dirty_[background_]bytes can't be applied verbatim to multiple cgroup writeback domains, they get converted to percentages in domain_dirty_limits() and applied the same way as vm.dirty_[background]ratio. However, if the specified bytes is lower than 1% of available memory, the calculated ratios become zero and the writeback domain gets throttled constantly.

Fix it by using per-PAGE_SIZE instead of percentage for ratio calculations. Also, the updated DIV_ROUND_UP() usages now should yield 1/4096 (0.0244%) as the minimum ratio as long as the specified bytes are above zero.

As vm.dirty_[background_]bytes can't be applied verbatim to multiple cgroup writeback domains, they get converted to percentages in domain_dirty_limits() and applied the same way as vm.dirty_[background]ratio. However, if the specified bytes is lower than 1% of available memory, the calculated ratios become zero and the writeback domain gets throttled constantly.

Fix it by using per-PAGE_SIZE instead of percentage for ratio calculations. Also, the updated DIV_ROUND_UP() usages now should yield 1/4096 (0.0244%) as the minimum ratio as long as the specified bytes are above zero.

Signed-off-by Tejun Heo <[email protected]>

Reported-by Miao Xie <[email protected]>

Link http://lkml.kernel.org/g/[email protected] open here

Cc [email protected] # v4.2+

Fixes 9fc3a43e1757 ("writeback: separate out domain_dirty_limits()")