Per file dirty limit throttling

When the total dirty pages exceed vm_dirty_ratio, the dirtier is made to do the writeback. But this dirtier may not be the one who took the system to this state. Instead, if we can track the dirty count per-file, we could throttle the dirtier of a file, when the file's dirty pages exceed a certain limit. Even though this dirtier may not be the one who dirtied the other pages of this file, it is fair to throttle this process, as it uses that file.

This patch
1. Adds dirty page accounting per-file.
2. Exports the number of pages of this file in cache and no of pages dirty via proc-fdinfo.
3. Adds a new tunable, /proc/sys/vm/file_dirty_bytes. When a files dirty data exceeds this limit, the writeback of that inode is done by the current dirtier.

This certainly will affect the throughput of certain heavy-dirtying workloads, but should help for interactive systems.

Documentation/sysctl/vm.txt +9 -0

index b606c2c..4f8bc06 100644
@@ -133,6 +133,15 @@ Setting this to zero disables periodic writeback altogether.
 
 ==============================================================
 
+file_dirty_bytes
+
+When a files total dirty data exceeds file_dirty_bytes, the current generator
+of dirty data would be made to do the writeback of dirty pages of that file.
+
+0 disables this behaviour.
+
+==============================================================
+
 drop_caches
 
 Writing to this will cause the kernel to drop clean caches, dentries and

fs/read_write.c +5 -0

index 74e3658..8881b7d 100644
@@ -16,6 +16,8 @@
 #include <linux/syscalls.h>
 #include <linux/pagemap.h>
 #include <linux/splice.h>
+#include <linux/buffer_head.h>
+#include <linux/writeback.h>
 #include "read_write.h"
 
 #include <asm/uaccess.h>

@@ -414,9 +416,19 @@ SYSCALL_DEFINE3(write, unsigned int, fd, const char __user *, buf,
 
 	file = fget_light(fd, &fput_needed);
 	if (file) {
+		struct address_space *as = file->f_mapping;
+		unsigned long file_dirty_pages;
 		loff_t pos = file_pos_read(file);
+
 		ret = vfs_write(file, buf, count, &pos);

mm/memory.c +1 -0

index 9606ceb..0961f70 100644
@@ -2873,6 +2873,7 @@ static int __do_fault(struct mm_struct *mm, struct vm_area_struct *vma,
 	struct vm_fault vmf;
 	int ret;
 	int page_mkwrite = 0;
+	unsigned long file_dirty_pages;
 
 	vmf.virtual_address = (void __user *)(address & PAGE_MASK);
 	vmf.pgoff = pgoff;

@@ -3024,6 +3025,13 @@ out:
 		/* file_update_time outside page_lock */
 		if (vma->vm_file)
 			file_update_time(vma->vm_file);

mm/page-writeback.c +2 -0

@@ -1126,6 +1137,7 @@ void account_page_dirtied(struct page *page, struct address_space *mapping)
 {
 	if (mapping_cap_account_dirty(mapping)) {
 		__inc_zone_page_state(page, NR_FILE_DIRTY);
+		mapping->nrdirty++;
 		__inc_bdi_stat(mapping->backing_dev_info, BDI_RECLAIMABLE);
 		task_dirty_inc(current);
 		task_io_account_write(PAGE_CACHE_SIZE);

@@ -1301,6 +1313,7 @@ int clear_page_dirty_for_io(struct page *page)
 		 */
 		if (TestClearPageDirty(page)) {
 			dec_zone_page_state(page, NR_FILE_DIRTY);
+			mapping->nrdirty--;
 			dec_bdi_stat(mapping->backing_dev_info,
 					BDI_RECLAIMABLE);
 			return 1;

mm/truncate.c +0 -0

index ba887bf..5846d6a 100644

Quoted 16 lines · Nikanth Karthikesan

When the total dirty pages exceed vm_dirty_ratio, the dirtier is made to do the writeback. But this dirtier may not be the one who took the system to this state. Instead, if we can track the dirty count per-file, we could throttle the dirtier of a file, when the file's dirty pages exceed a certain limit. Even though this dirtier may not be the one who dirtied the other pages of this file, it is fair to throttle this process, as it uses that file.

This patch
1. Adds dirty page accounting per-file.
2. Exports the number of pages of this file in cache and no of pages dirty via proc-fdinfo.
3. Adds a new tunable, /proc/sys/vm/file_dirty_bytes. When a files dirty data exceeds this limit, the writeback of that inode is done by the current dirtier.

Quoted 2 lines

This certainly will affect the throughput of certain heavy-dirtying workloads, but should help for interactive systems.

I'm not really charmed by this.. it adds another random variable to prod at. Nor does it really tell me why you're wanting to do this. We already have per-task invluence on the dirty limits, a task that sporadically dirties pages (your vi) will already end up with a higher dirty limit than a task that only dirties pages (your dd).

Quoted 50 lines

Documentation/sysctl/vm.txt +0 -0

index b606c2c..4f8bc06 100644
@@ -133,6 +133,15 @@ Setting this to zero disables periodic writeback

altogether.

==============================================================

Documentation/sysctl/vm.txt +3 -0

+file_dirty_bytes
+
+When a files total dirty data exceeds file_dirty_bytes, the current

generator +of dirty data would be made to do the writeback of dirty pages of that file. +
+0 disables this behaviour.

Documentation/sysctl/vm.txt +3 -0

+
+==============================================================
+
 drop_caches
 
 Writing to this will cause the kernel to drop clean caches, dentries and

fs/read_write.c +9 -0

index 74e3658..8881b7d 100644
@@ -16,6 +16,8 @@
 #include <linux/syscalls.h>
 #include <linux/pagemap.h>
 #include <linux/splice.h>
+#include <linux/buffer_head.h>
+#include <linux/writeback.h>
 #include "read_write.h"
 
 #include <asm/uaccess.h>

@@ -414,9 +416,19 @@ SYSCALL_DEFINE3(write, unsigned int, fd, const char
 __user *, buf,
 
 	file = fget_light(fd, &fput_needed);
 	if (file) {
+		struct address_space *as = file->f_mapping;
+		unsigned long file_dirty_pages;
 		loff_t pos = file_pos_read(file);
+
 		ret = vfs_write(file, buf, count, &pos);
 		file_pos_write(file, pos);
+		/* Start write-out ? */
+		if (file_dirty_bytes) {
+			file_dirty_pages = file_dirty_bytes / PAGE_SIZE;
+			if (as->nrdirty > file_dirty_pages)

fs/read_write.c SYSCALL_DEFINE3(write, unsigned int, fd, const char __user *, buf, · line 433

Quoted 5 lines

fs/read_write.c +3 -0

+				write_inode_now(as->host, 0);
+		}
+
 		fput_light(file, fput_needed);
 	}

This seems wrong, wth are you doing it here and not in the generic balance_dirty_pages thing called by set_page_dirty()?

Quoted 16 lines

mm/memory.c +1 -0

index 9606ceb..0961f70 100644
@@ -2873,6 +2873,7 @@ static int __do_fault(struct mm_struct *mm, struct
 vm_area_struct *vma, struct vm_fault vmf;
 	int ret;
 	int page_mkwrite = 0;
+	unsigned long file_dirty_pages;
 
 	vmf.virtual_address = (void __user *)(address & PAGE_MASK);
 	vmf.pgoff = pgoff;

@@ -3024,6 +3025,13 @@ out:
 		/* file_update_time outside page_lock */
 		if (vma->vm_file)
 			file_update_time(vma->vm_file);

mm/memory.c out: · line 3037

Quoted 10 lines

mm/memory.c +7 -0

+
+		/* Start write-back ? */
+		if (mapping && file_dirty_bytes) {
+			file_dirty_pages = file_dirty_bytes / PAGE_SIZE;
+			if (mapping->nrdirty > file_dirty_pages)
+				write_inode_now(mapping->host, 0);
+		}
 	} else {
 		unlock_page(vmf.page);
 		if (anon)

Idem, replicating that code at every site that can dirty a page is just wrong, hook into the regular set_page_dirty()->balance_dirty_pages() code.

Quoted 7 lines

mm/page-writeback.c +0 -0

index 20890d8..1cabd7f 100644
@@ -87,6 +87,13 @@ int vm_dirty_ratio = 20;
 unsigned long vm_dirty_bytes;
 

Quoted 6 lines

mm/page-writeback.c +1 -0

 /*
+ * When a files total dirty data exceeds file_dirty_bytes, the current

generator + * of dirty data would be made to do the writeback of dirty pages of that file. + * 0 disables this behaviour.
+ */
+unsigned long file_dirty_bytes = 0;

So you're adding a extra cacheline to dirty even though its not used by default, that seems like suckage..

Quoted 21 lines

mm/page-writeback.c +2 -0

@@ -1126,6 +1137,7 @@ void account_page_dirtied(struct page *page, struct
 address_space *mapping) {
 	if (mapping_cap_account_dirty(mapping)) {
 		__inc_zone_page_state(page, NR_FILE_DIRTY);
+		mapping->nrdirty++;
 		__inc_bdi_stat(mapping->backing_dev_info, BDI_RECLAIMABLE);
 		task_dirty_inc(current);
 		task_io_account_write(PAGE_CACHE_SIZE);

@@ -1301,6 +1313,7 @@ int clear_page_dirty_for_io(struct page *page)
 		 */
 		if (TestClearPageDirty(page)) {
 			dec_zone_page_state(page, NR_FILE_DIRTY);
+			mapping->nrdirty--;
 			dec_bdi_stat(mapping->backing_dev_info,
 					BDI_RECLAIMABLE);
 			return 1;

mm/truncate.c +0 -0

index ba887bf..5846d6a 100644

mm/truncate.c void cancel_dirty_page(struct page *page, unsigned int · line 81

Quoted 7 lines

mm/truncate.c +1 -0

@@ -75,6 +75,7 @@ void cancel_dirty_page(struct page *page, unsigned int
 account_size) struct address_space *mapping = page->mapping;
 		if (mapping && mapping_cap_account_dirty(mapping)) {
 			dec_zone_page_state(page, NR_FILE_DIRTY);
+			mapping->nrdirty--;
 			dec_bdi_stat(mapping->backing_dev_info,
 					BDI_RECLAIMABLE);
 			if (account_size)

Preferably we don't add any extra fields under tree_lock so that we can easily split it up if/when we decide to use a fine-grain locked radix tree.

Also, like mentioned, you just added a whole new cacheline to dirty.

Oh, nice. Per-task limit is an elegant solution, which should help during most of the common cases.

But I just wonder what happens, when
1. The dirtier is multiple co-operating processes
2. Some app like a shell script, that repeatedly calls dd with seek and skip? People do this for data deduplication, sparse skipping etc..
3. The app dies and comes back again. Like a VM that is rebooted, and continues writing to a disk backed by a file on the host.

Do you think, in those cases this might still be useful?

Quoted 10 lines · Nikanth Karthikesan

Oh, nice. Per-task limit is an elegant solution, which should help during most of the common cases.

But I just wonder what happens, when
1. The dirtier is multiple co-operating processes
2. Some app like a shell script, that repeatedly calls dd with seek and skip? People do this for data deduplication, sparse skipping etc..
3. The app dies and comes back again. Like a VM that is rebooted, and continues writing to a disk backed by a file on the host.

Quoted 1 line

Do you think, in those cases this might still be useful?

Those cases do indeed defeat the current per-task-limit, however I think the solution to that is to limit the amount of writeback done by each blocked process.

Jan Kara had some good ideas in that department.

Quoted 11 lines · On Tuesday 17 August 2010 13:54:35 Peter Zijlstra wrote:

Quoted 10 lines · Nikanth Karthikesan

Oh, nice. Per-task limit is an elegant solution, which should help during most of the common cases.

But I just wonder what happens, when
1. The dirtier is multiple co-operating processes
2. Some app like a shell script, that repeatedly calls dd with seek and skip? People do this for data deduplication, sparse skipping etc..
3. The app dies and comes back again. Like a VM that is rebooted, and continues writing to a disk backed by a file on the host.

Quoted 6 lines

Quoted 1 line

Do you think, in those cases this might still be useful?

Those cases do indeed defeat the current per-task-limit, however I think the solution to that is to limit the amount of writeback done by each blocked process.

Blocked on what? Sorry, I do not understand.

Quoted 19 lines · Nikanth Karthikesan

Quoted 17 lines · On Tuesday 17 August 2010 13:54:35 Peter Zijlstra wrote:

Quoted 11 lines · Nikanth Karthikesan

Oh, nice. Per-task limit is an elegant solution, which should help during most of the common cases.

But I just wonder what happens, when
1. The dirtier is multiple co-operating processes
2. Some app like a shell script, that repeatedly calls dd with seek and skip? People do this for data deduplication, sparse skipping etc..
3. The app dies and comes back again. Like a VM that is rebooted, and continues writing to a disk backed by a file on the host.

Do you think, in those cases this might still be useful?

Those cases do indeed defeat the current per-task-limit, however I think the solution to that is to limit the amount of writeback done by each blocked process.

Quoted 1 line

Blocked on what? Sorry, I do not understand.

balance_dirty_pages(), by limiting the work done there (or actually, the amount of page writeback completions you wait for -- starting IO isn't that expensive), you can also affect the time it takes, and therefore influence the impact.

There is an ongoing effort to look at per-cgroup dirty limits and I honestly think it would be nice to do it at that level first. We need it there as a part of the overall I/O controller. As a specialized need it could handle your case as well.

Quoted 4 lines · Balbir Singh

There is an ongoing effort to look at per-cgroup dirty limits and I honestly think it would be nice to do it at that level first. We need it there as a part of the overall I/O controller. As a specialized need it could handle your case as well.

Well, it would be good to isolate that to the cgroup code. Also from what I understood, the plan was to simply mark dirty inodes with a cgroup and use that from writeout_inodes() to write out inodes specifically used by that cgroup.

That is, on top of what Andrea Righi already proposed, which would provide the actual per cgroup dirty limit (although the per-bdi proportions applied to a cgroup limit aren't strictly correct, but that seems to be something you'll have to live with, a per-bdi-per-cgroup proportion would simply be accounting insanity).

That is a totally different thing than what was proposed.

Quoted 19 lines · Nikanth Karthikesan

Quoted 17 lines · On Tuesday 17 August 2010 13:54:35 Peter Zijlstra wrote:

Quoted 11 lines · Nikanth Karthikesan

Oh, nice. Per-task limit is an elegant solution, which should help during most of the common cases.

But I just wonder what happens, when
1. The dirtier is multiple co-operating processes
2. Some app like a shell script, that repeatedly calls dd with seek and skip? People do this for data deduplication, sparse skipping etc.. 3. The app dies and comes back again. Like a VM that is rebooted, and continues writing to a disk backed by a file on the host.

Quoted 1 line

Do you think, in those cases this might still be useful?

Those cases do indeed defeat the current per-task-limit, however I think the solution to that is to limit the amount of writeback done by each blocked process.

Quoted 1 line

Blocked on what? Sorry, I do not understand.

balance_dirty_pages(), by limiting the work done there (or actually, the amount of page writeback completions you wait for -- starting IO isn't that expensive), you can also affect the time it takes, and therefore influence the impact.

When the total dirty pages exceed vm_dirty_ratio, the dirtier is made to do the writeback. But this dirtier may not be the one who took the system to this state. Instead, if we can track the dirty count per-file, we could throttle the dirtier of a file, when the file's dirty pages exceed a certain limit. Even though this dirtier may not be the one who dirtied the other pages of this file, it is fair to throttle this process, as it uses that file.

This patch
1. Adds dirty page accounting per-file.
2. Exports the number of pages of this file in cache and no of pages dirty via proc-fdinfo.
3. Adds a new tunable, /proc/sys/vm/file_dirty_bytes. When a files dirty data exceeds this limit, the writeback of that inode is done by the current dirtier.

This certainly will affect the throughput of certain heavy-dirtying workloads, but should help for interactive systems.

Quoted 7 lines · Nikanth Karthikesan

When the total dirty pages exceed vm_dirty_ratio, the dirtier is made to do the writeback. But this dirtier may not be the one who took the system to this state. Instead, if we can track the dirty count per-file, we could throttle the dirtier of a file, when the file's dirty pages exceed a certain limit. Even though this dirtier may not be the one who dirtied the other pages of this file, it is fair to throttle this process, as it uses that file.

I agree with your problem description, a single program which writes a single large file can make an interactive system suck. Creating a 25+GB Blu-Ray image will often saturate the buffer space. I played with per-fd limiting during 2.5.xx development and I had an app writing 5-10GB files. While I wanted to get something to submit while the kernel was changing, I kept hitting cornet cases.

Quoted 8 lines

This patch
1. Adds dirty page accounting per-file.
2. Exports the number of pages of this file in cache and no of pages dirty via proc-fdinfo.
3. Adds a new tunable, /proc/sys/vm/file_dirty_bytes. When a files dirty data exceeds this limit, the writeback of that inode is done by the current dirtier.

I think you have this in the wrong place, can't it go in balance_dirty_pages?

Quoted 3 lines

This certainly will affect the throughput of certain heavy-dirtying workloads, but should help for interactive systems.

I found that the effect was about the same as forcing the application to use O_DIRECT, and since it was our application I could do that. Not all badly-behaved programs are open source, so that addressed my issue but not the general case.

I think you really need to track by process, not file, as you said "Even though this dirtier may not be the one who dirtied the other pages of this file..." that doesn't work, you block a process which is contributing minimally to the problem while letting the real problem process continue. Ex: a log file, with one process spewing error messages while others write a few lines/min. You have to get it right, I think.

When the total dirty pages exceed vm_dirty_ratio, the dirtier is made to do the writeback. But this dirtier may not be the one who took the system to this state. Instead, if we can track the dirty count per-file, we could throttle the dirtier of a file, when the file's dirty pages exceed a certain limit. Even though this dirtier may not be the one who dirtied the other pages of this file, it is fair to throttle this process, as it uses that file.