writeback: remove the internal 5% low bound on dirty_ratio

The dirty_ratio was silently limited to>= 5%. This is not a user expected behavior. Let's rip it.

It's not likely the user space will depend on the old behavior. So the risk of breaking user space is very low.

CC Jan Kara<[email protected]>

CC Neil Brown<[email protected]>

Signed-off-by Wu Fengguang<[email protected]>

The dirty_ratio was silently limited to >= 5%. This is not a user expected behavior. Let's rip it.

It's not likely the user space will depend on the old behavior. So the risk of breaking user space is very low.

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

Quoted 6 lines

The dirty_ratio was silently limited to >= 5%. This is not a user expected behavior. Let's rip it.

It's not likely the user space will depend on the old behavior. So the risk of breaking user space is very low.

Quoted 3 lines

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

Thank you.

Reviewed-by KOSAKI Motohiro <[email protected]>

Quoted 6 lines · KOSAKI Motohiro

Quoted 6 lines

The dirty_ratio was silently limited to >= 5%. This is not a user expected behavior. Let's rip it.

It's not likely the user space will depend on the old behavior. So the risk of breaking user space is very low.

Quoted 6 lines

Quoted 3 lines

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

Thank you.

Reviewed-by KOSAKI Motohiro <[email protected]>

I have tried to do this in the past, and setting this value to 0 on some machines caused the machine to come to a complete standstill with small writes to disk. It seemed there was some kind of "minimum" amount of data required by the VM before anything would make it to the disk and I never quite found out where that blockade occurred. This was some time ago (3 years ago) so I'm not sure if the problem has since been fixed in the VM since then. I suggest you do some testing with this value set to zero before approving this change.

Quoted 6 lines · KOSAKI Motohiro

Quoted 6 lines

The dirty_ratio was silently limited to >= 5%. This is not a user expected behavior. Let's rip it.

It's not likely the user space will depend on the old behavior. So the risk of breaking user space is very low.

Quoted 6 lines

Quoted 3 lines

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

Thank you.

Reviewed-by KOSAKI Motohiro <[email protected]>

I have tried to do this in the past, and setting this value to 0 on some machines caused the machine to come to a complete standstill with small writes to disk. It seemed there was some kind of "minimum" amount of data required by the VM before anything would make it to the disk and I never quite found out where that blockade occurred. This was some time ago (3 years ago) so I'm not sure if the problem has since been fixed in the VM since then. I suggest you do some testing with this value set to zero before approving this change.

Quoted 14 lines · On Fri, 20 Aug 2010 15:50:54 +1000 Con Kolivas <[email protected]> wrote:

Quoted 12 lines · KOSAKI Motohiro

Quoted 9 lines

The dirty_ratio was silently limited to >= 5%. This is not a user expected behavior. Let's rip it.

It's not likely the user space will depend on the old behavior. So the risk of breaking user space is very low.

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

Thank you.

Reviewed-by KOSAKI Motohiro <[email protected]>

Quoted 8 lines

I have tried to do this in the past, and setting this value to 0 on some machines caused the machine to come to a complete standstill with small writes to disk. It seemed there was some kind of "minimum" amount of data required by the VM before anything would make it to the disk and I never quite found out where that blockade occurred. This was some time ago (3 years ago) so I'm not sure if the problem has since been fixed in the VM since then. I suggest you do some testing with this value set to zero before approving this change.

If it is appropriate to have a lower limit, that should be imposed where the sysctl is defined in kernel/sysctl.c, not imposed after the fact where the value is used.

As we now have dirty_bytes which over-rides dirty_ratio, there is little cost in having a lower_limit for dirty_ratio - it could even stay at 5% - but it really shouldn't be silent. Writing a number below the limit to the sysctl file should fail.

Quoted 25 lines · Neil Brown

On Fri, 20 Aug 2010 15:50:54 +1000

Quoted 22 lines · Con Kolivas

Quoted 6 lines · KOSAKI Motohiro

Quoted 6 lines

The dirty_ratio was silently limited to >= 5%. This is not a user expected behavior. Let's rip it.

It's not likely the user space will depend on the old behavior. So the risk of breaking user space is very low.

Quoted 6 lines

Quoted 3 lines

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

Thank you.

Reviewed-by KOSAKI Motohiro <[email protected]>

I have tried to do this in the past, and setting this value to 0 on some machines caused the machine to come to a complete standstill with small writes to disk. It seemed there was some kind of "minimum" amount of data required by the VM before anything would make it to the disk and I never quite found out where that blockade occurred. This was some time ago (3 years ago) so I'm not sure if the problem has since been fixed in the VM since then. I suggest you do some testing with this value set to zero before approving this change.

You are right, vm.dirty_ratio=0 will block applications for ever..

Quoted 27 lines · Wu Fengguang

Quoted 25 lines · Neil Brown

On Fri, 20 Aug 2010 15:50:54 +1000

Quoted 22 lines · Con Kolivas

Quoted 6 lines · KOSAKI Motohiro

Quoted 6 lines

The dirty_ratio was silently limited to >= 5%. This is not a user expected behavior. Let's rip it.

It's not likely the user space will depend on the old behavior. So the risk of breaking user space is very low.

Quoted 6 lines

Quoted 3 lines

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

Thank you.

Reviewed-by KOSAKI Motohiro <[email protected]>

I have tried to do this in the past, and setting this value to 0 on some machines caused the machine to come to a complete standstill with small writes to disk. It seemed there was some kind of "minimum" amount of data required by the VM before anything would make it to the disk and I never quite found out where that blockade occurred. This was some time ago (3 years ago) so I'm not sure if the problem has since been fixed in the VM since then. I suggest you do some testing with this value set to zero before approving this change.

Quoted 1 line

You are right, vm.dirty_ratio=0 will block applications for ever..

Indeed. And while you shouldn't set the lower limit to zero to avoid this problem, it doesn't answer _why_ this happens. What is this "minimum write" that blocks everything, will 1% be enough, and is it hiding another real bug somewhere in the VM?

writeback: remove the internal 5% low bound on dirty_ratio

The dirty_ratio was silently limited in global_dirty_limits() to >= 5%. This is not a user expected behavior. And it's inconsistent with calc_period_shift(), which uses the plain vm_dirty_ratio value. So let's rip the internal bound.

At the same time, force a user visible low bound of 1% for the vm.dirty_ratio interface. Applications trying to write 0 will be rejected with -EINVAL. This will break user space applications if they
1) try to write 0 to vm.dirty_ratio
2) and check the return value
That is very weird combination, so the risk of breaking user space is low.

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

CC Rik van Riel <[email protected]>

CC Con Kolivas <[email protected]>

CC Peter Zijlstra <[email protected]>

CC KOSAKI Motohiro <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

---
 kernel/sysctl.c     |    2 +-
 mm/page-writeback.c |   10 ++--------
 2 files changed, 3 insertions(+), 9 deletions(-)

mm/page-writeback.c +2 -8

@@ -415,14 +415,8 @@ void global_dirty_limits(unsigned long *
 
 	if (vm_dirty_bytes)
 		dirty = DIV_ROUND_UP(vm_dirty_bytes, PAGE_SIZE);
-	else {
-		int dirty_ratio;
-
-		dirty_ratio = vm_dirty_ratio;
-		if (dirty_ratio < 5)
-			dirty_ratio = 5;
-		dirty = (dirty_ratio * available_memory) / 100;
-	}
+	else
+		dirty = (vm_dirty_ratio * available_memory) / 100;
 
 	if (dirty_background_bytes)
 		background = DIV_ROUND_UP(dirty_background_bytes, PAGE_SIZE);

kernel/sysctl.c +1 -1

@@ -1029,7 +1029,7 @@ static struct ctl_table vm_table[] = {
 		.maxlen		= sizeof(vm_dirty_ratio),
 		.mode		= 0644,
 		.proc_handler	= dirty_ratio_handler,
-		.extra1		= &zero,
+		.extra1		= &one,
 		.extra2		= &one_hundred,
 	},
 	{

Quoted 81 lines · you

Quoted 71 lines · Con Kolivas

Quoted 70 lines · you

Quoted 30 lines · Con Kolivas

Quoted 29 lines · Wu Fengguang

Quoted 27 lines · Neil Brown

On Fri, 20 Aug 2010 15:50:54 +1000

Quoted 24 lines · Con Kolivas

Quoted 6 lines · KOSAKI Motohiro

Quoted 6 lines

The dirty_ratio was silently limited to >= 5%. This is not a user expected behavior. Let's rip it.

It's not likely the user space will depend on the old behavior. So the risk of breaking user space is very low.

Quoted 7 lines

Quoted 3 lines

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

Thank you.
Reviewed-by: KOSAKI Motohiro
<[email protected]>

I have tried to do this in the past, and setting this value to 0 on some machines caused the machine to come to a complete standstill with small writes to disk. It seemed there was some kind of "minimum" amount of data required by the VM before anything would make it to the disk and I never quite found out where that blockade occurred. This was some time ago (3 years ago) so I'm not sure if the problem has since been fixed in the VM since then. I suggest you do some testing with this value set to zero before approving this change.

Quoted 6 lines

Quoted 1 line

You are right, vm.dirty_ratio=0 will block applications for ever..

Indeed. And while you shouldn't set the lower limit to zero to avoid this problem, it doesn't answer _why_ this happens. What is this "minimum write" that blocks everything, will 1% be enough, and is it hiding another real bug somewhere in the VM?

Good question.
This simple change will unblock the application even with vm_dirty_ratio=0.

# echo 0 > /proc/sys/vm/dirty_ratio
# echo 0 > /proc/sys/vm/dirty_background_ratio
# vmmon nr_dirty nr_writeback nr_unstable

nr_dirty nr_writeback nr_unstable
0 444 1369
37 37 326
0 0 37
74 772 694
0 0 19
0 0 1406
0 0 23
0 0 0
0 370 186
74 1073 1221
0 12 26
0 703 1147
37 0 999
37 37 1517
0 888 63
0 0 0
0 0 20
37 0 0
37 74 1776
0 0 8
37 629 333
0 12 19

Quoted 6 lines

Quoted 2 lines

Even with it, the 1% explicit bound still looks reasonable for me. Who will want to set it to 0%? That would destroy IO inefficient.

Thanks for your work in this area. I'll experiment with these later. There are low latency applications that would benefit with it set to zero.

It might be useful to some users. Shall we give the rope to users, heh?

Quoted 3 lines

Note that for these applications, they may well use /proc/sys/vm/dirty_bytes for more fine grained control. That interface only imposes a low limit of 2 pages.

I don't see why there needs to be a limit. Users fiddling with sysctls should know what they're messing with, and there may well be a valid use out there somewhere for it.

Quoted 82 lines · Con Kolivas

Quoted 81 lines · you

Quoted 71 lines · Con Kolivas

Quoted 70 lines · you

Quoted 30 lines · Con Kolivas

Quoted 29 lines · Wu Fengguang

Quoted 27 lines · Neil Brown

On Fri, 20 Aug 2010 15:50:54 +1000

Quoted 24 lines · Con Kolivas

Quoted 6 lines · KOSAKI Motohiro

Quoted 6 lines

The dirty_ratio was silently limited to >= 5%. This is not a user expected behavior. Let's rip it.

It's not likely the user space will depend on the old behavior. So the risk of breaking user space is very low.

Quoted 7 lines

Quoted 3 lines

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

Thank you.
Reviewed-by: KOSAKI Motohiro
<[email protected]>

I have tried to do this in the past, and setting this value to 0 on some machines caused the machine to come to a complete standstill with small writes to disk. It seemed there was some kind of "minimum" amount of data required by the VM before anything would make it to the disk and I never quite found out where that blockade occurred. This was some time ago (3 years ago) so I'm not sure if the problem has since been fixed in the VM since then. I suggest you do some testing with this value set to zero before approving this change.

Quoted 6 lines

Quoted 1 line

You are right, vm.dirty_ratio=0 will block applications for ever..

Indeed. And while you shouldn't set the lower limit to zero to avoid this problem, it doesn't answer _why_ this happens. What is this "minimum write" that blocks everything, will 1% be enough, and is it hiding another real bug somewhere in the VM?

Good question.
This simple change will unblock the application even with vm_dirty_ratio=0.

# echo 0 > /proc/sys/vm/dirty_ratio
# echo 0 > /proc/sys/vm/dirty_background_ratio
# vmmon nr_dirty nr_writeback nr_unstable

nr_dirty nr_writeback nr_unstable
0 444 1369
37 37 326
0 0 37
74 772 694
0 0 19
0 0 1406
0 0 23
0 0 0
0 370 186
74 1073 1221
0 12 26
0 703 1147
37 0 999
37 37 1517
0 888 63
0 0 0
0 0 20
37 0 0
37 74 1776
0 0 8
37 629 333
0 12 19

Quoted 6 lines

Quoted 2 lines

Even with it, the 1% explicit bound still looks reasonable for me. Who will want to set it to 0%? That would destroy IO inefficient.

Thanks for your work in this area. I'll experiment with these later. There are low latency applications that would benefit with it set to zero.

It might be useful to some users. Shall we give the rope to users, heh?

Quoted 7 lines

Quoted 3 lines

Note that for these applications, they may well use /proc/sys/vm/dirty_bytes for more fine grained control. That interface only imposes a low limit of 2 pages.

I don't see why there needs to be a limit. Users fiddling with sysctls should know what they're messing with, and there may well be a valid use out there somewhere for it.

OK, the following patch gives users the full freedom. I tested 1 single dirtier and 9 parallel dirtiers, the system remains alive, but with much slower IO throughput. Maybe not all users care IO performance in all situations?

Thanks,
Fengguang
---
writeback: remove the internal 5% low bound on dirty_ratio

The dirty_ratio was silently limited in global_dirty_limits() to >= 5%. This is not a user expected behavior. And it's inconsistent with calc_period_shift(), which uses the plain vm_dirty_ratio value.

Let's rip the internal bound.

At the same time, fix balance_dirty_pages() to work with the dirty_thresh=0 case. This allows applications to proceed when dirty+writeback pages are all cleaned.

CC Jan Kara <[email protected]>

CC Neil Brown <[email protected]>

CC Con Kolivas <[email protected]>

CC Rik van Riel <[email protected]>

CC Peter Zijlstra <[email protected]>

CC KOSAKI Motohiro <[email protected]>

Signed-off-by Wu Fengguang <[email protected]>

---
 mm/page-writeback.c |   14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)

mm/page-writeback.c +4 -10

@@ -415,14 +415,8 @@ void global_dirty_limits(unsigned long *
 
 	if (vm_dirty_bytes)
 		dirty = DIV_ROUND_UP(vm_dirty_bytes, PAGE_SIZE);
-	else {
-		int dirty_ratio;
-
-		dirty_ratio = vm_dirty_ratio;
-		if (dirty_ratio < 5)
-			dirty_ratio = 5;
-		dirty = (dirty_ratio * available_memory) / 100;
-	}
+	else
+		dirty = (vm_dirty_ratio * available_memory) / 100;
 
 	if (dirty_background_bytes)
 		background = DIV_ROUND_UP(dirty_background_bytes, PAGE_SIZE);

@@ -542,8 +536,8 @@ static void balance_dirty_pages(struct a
 		 * the last resort safeguard.
 		 */
 		dirty_exceeded =
-			(bdi_nr_reclaimable + bdi_nr_writeback >= bdi_thresh)
-			|| (nr_reclaimable + nr_writeback >= dirty_thresh);
+			(bdi_nr_reclaimable + bdi_nr_writeback > bdi_thresh)
+			|| (nr_reclaimable + nr_writeback > dirty_thresh);
 
 		if (!dirty_exceeded)
 			break;

Quoted 8 lines

writeback: remove the internal 5% low bound on dirty_ratio

The dirty_ratio was silently limited in global_dirty_limits() to >= 5%. This is not a user expected behavior. And it's inconsistent with calc_period_shift(), which uses the plain vm_dirty_ratio value.

Let's rip the internal bound.

Quoted 3 lines

At the same time, fix balance_dirty_pages() to work with the dirty_thresh=0 case. This allows applications to proceed when dirty+writeback pages are all cleaned.

And ">" fits with the name "exceeded" better than ">=" does. I think it is an aesthetic improvement as well as a functional one.

Reviewed-by NeilBrown <[email protected]>