2.6.xx: dirty pages never being sync'd to disk?
Okay, this one's been nagging me since I first began using 2.6.xx.
My Notebook computer has 2GB of RAM, and the 2.6.xx kernel seems quite happy to leave hundreds of MB of dirty unsync'd pages laying around more or less indefinitely. This worries me, because that's a lot of data to lose should the kernel crash (which it has once quite recently) or the battery die.
/proc/sys/vm/dirty_expire_centisecs = 3000 (30 seconds) /proc/sys/vm/dirty_writeback_centisecs = 500 (5 seconds)
My understanding (please correct if wrong) is that this means that any (file data) page which is dirtied, should get flushed back to disk after 30 seconds or so.
That doesn't happen here. Hundreds of MB of dirty pages just hang around indefinitely, until I manually type "sync", at which point the hard drive gets very busy for 20 seconds or so.
What's going on?
Quoted 8 lines · Mark Lord
Okay, this one's been nagging me since I first began using 2.6.xx.
My Notebook computer has 2GB of RAM, and the 2.6.xx kernel seems quite happy to leave hundreds of MB of dirty unsync'd pages laying around more or less indefinitely. This worries me, because that's a lot of data to lose should the kernel crash (which it has once quite recently) or the battery die.
Quoted 6 lines
/proc/sys/vm/dirty_expire_centisecs= 3000 (30 seconds)/proc/sys/vm/dirty_writeback_centisecs= 500 (5 seconds)My understanding (please correct if wrong) is that this means that any (file data) page which is dirtied, should get flushed back to disk after 30 seconds or so.
do you have laptop mode enabled? That changes the behavior bigtime in this regard and makes the kernel behave quite different.
also if these are files written to by mmap, the kernel only really sees those as dirty when the mapping gets taken down (eg the propagation from the pagetable dirty bit to the per page dirty bit goes a bit lazy)
Quoted 1 line · Arjan van de Ven
On Mon, 2005-11-14 at 10:30 -0500, Mark Lord wrote:
..
Quoted 2 lines
Quoted 2 lines
My Notebook computer has 2GB of RAM, and the 2.6.xx kernel seems quite happy to leave hundreds of MB of dirty unsync'd pages laying around
..
Quoted 2 lines
Quoted 2 lines
/proc/sys/vm/dirty_expire_centisecs= 3000 (30 seconds)/proc/sys/vm/dirty_writeback_centisecs= 500 (5 seconds)
..
Quoted 2 lines
do you have laptop mode enabled? That changes the behavior bigtime in this regard and makes the kernel behave quite different.
No. Laptop-mode mostly just modifies the dirty_expire and related settings, and I have them set as shown above. But there's also this:
/proc/sys/vm/laptop_mode = 0
Quoted 2 lines
also if these are files written to by mmap, the kernel only really sees those as dirty when the mapping gets taken down
They certainly show up in the counts in /proc/meminfo under "Dirty", so I assumed that means the kernel knows they are dirty.
A simple test I do for this:
$ mkdir t $ cp /usr/src/*.bz2 t (about 400-500MB worth of kernel tar files)
In another window, I do this:
$ while (sleep 1); do echo -n "`date`: "; grep Dirty /proc/meminfo; done
And then watch the count get large, but take virtually forever to count back down to a "safe" value.
Typing "sync" causes all the Dirty pages to immediately be flushed to disk, as expected.
Here's what the monitoring of /proc/meminfo shows, on an otherwise mostly idle system after having done the big file copies noted earlier:
Mon Nov 14 10:40:22 EST 2005: Dirty: 481284 kB Mon Nov 14 10:40:23 EST 2005: Dirty: 479680 kB Mon Nov 14 10:40:24 EST 2005: Dirty: 480380 kB Mon Nov 14 10:40:25 EST 2005: Dirty: 480380 kB Mon Nov 14 10:40:26 EST 2005: Dirty: 480380 kB Mon Nov 14 10:40:27 EST 2005: Dirty: 480380 kB Mon Nov 14 10:40:28 EST 2005: Dirty: 480384 kB Mon Nov 14 10:40:29 EST 2005: Dirty: 480384 kB Mon Nov 14 10:40:30 EST 2005: Dirty: 480384 kB Mon Nov 14 10:40:31 EST 2005: Dirty: 480384 kB Mon Nov 14 10:40:32 EST 2005: Dirty: 480384 kB Mon Nov 14 10:40:33 EST 2005: Dirty: 480688 kB Mon Nov 14 10:40:34 EST 2005: Dirty: 479972 kB Mon Nov 14 10:40:35 EST 2005: Dirty: 479972 kB Mon Nov 14 10:40:36 EST 2005: Dirty: 479972 kB Mon Nov 14 10:40:37 EST 2005: Dirty: 480016 kB Mon Nov 14 10:40:38 EST 2005: Dirty: 480016 kB Mon Nov 14 10:40:39 EST 2005: Dirty: 480016 kB Mon Nov 14 10:40:40 EST 2005: Dirty: 480020 kB Mon Nov 14 10:40:41 EST 2005: Dirty: 480020 kB Mon Nov 14 10:40:42 EST 2005: Dirty: 480028 kB Mon Nov 14 10:40:43 EST 2005: Dirty: 480028 kB Mon Nov 14 10:40:44 EST 2005: Dirty: 475868 kB Mon Nov 14 10:40:45 EST 2005: Dirty: 475868 kB Mon Nov 14 10:40:46 EST 2005: Dirty: 475868 kB Mon Nov 14 10:40:47 EST 2005: Dirty: 475868 kB Mon Nov 14 10:40:48 EST 2005: Dirty: 475880 kB Mon Nov 14 10:40:49 EST 2005: Dirty: 475880 kB Mon Nov 14 10:40:50 EST 2005: Dirty: 475880 kB Mon Nov 14 10:40:51 EST 2005: Dirty: 475880 kB Mon Nov 14 10:40:52 EST 2005: Dirty: 475880 kB Mon Nov 14 10:40:53 EST 2005: Dirty: 475880 kB Mon Nov 14 10:40:54 EST 2005: Dirty: 455160 kB Mon Nov 14 10:40:55 EST 2005: Dirty: 455160 kB Mon Nov 14 10:40:57 EST 2005: Dirty: 455160 kB Mon Nov 14 10:40:58 EST 2005: Dirty: 455160 kB Mon Nov 14 10:40:59 EST 2005: Dirty: 455164 kB Mon Nov 14 10:41:00 EST 2005: Dirty: 455160 kB Mon Nov 14 10:41:01 EST 2005: Dirty: 455160 kB Mon Nov 14 10:41:02 EST 2005: Dirty: 455160 kB Mon Nov 14 10:41:03 EST 2005: Dirty: 455164 kB Mon Nov 14 10:41:04 EST 2005: Dirty: 455164 kB Mon Nov 14 10:41:05 EST 2005: Dirty: 455168 kB Mon Nov 14 10:41:06 EST 2005: Dirty: 455168 kB Mon Nov 14 10:41:07 EST 2005: Dirty: 455168 kB Mon Nov 14 10:41:08 EST 2005: Dirty: 455188 kB Mon Nov 14 10:41:09 EST 2005: Dirty: 455176 kB Mon Nov 14 10:41:10 EST 2005: Dirty: 455176 kB Mon Nov 14 10:41:11 EST 2005: Dirty: 455176 kB Mon Nov 14 10:41:12 EST 2005: Dirty: 455176 kB Mon Nov 14 10:41:13 EST 2005: Dirty: 455180 kB Mon Nov 14 10:41:14 EST 2005: Dirty: 450972 kB Mon Nov 14 10:41:15 EST 2005: Dirty: 450972 kB Mon Nov 14 10:41:16 EST 2005: Dirty: 450972 kB Mon Nov 14 10:41:17 EST 2005: Dirty: 450972 kB Mon Nov 14 10:41:18 EST 2005: Dirty: 451016 kB Mon Nov 14 10:41:19 EST 2005: Dirty: 430336 kB Mon Nov 14 10:41:20 EST 2005: Dirty: 430336 kB Mon Nov 14 10:41:21 EST 2005: Dirty: 430336 kB Mon Nov 14 10:41:22 EST 2005: Dirty: 430336 kB Mon Nov 14 10:41:23 EST 2005: Dirty: 430348 kB Mon Nov 14 10:41:24 EST 2005: Dirty: 430348 kB Mon Nov 14 10:41:25 EST 2005: Dirty: 430348 kB Mon Nov 14 10:41:26 EST 2005: Dirty: 430348 kB Mon Nov 14 10:41:27 EST 2005: Dirty: 430348 kB Mon Nov 14 10:41:28 EST 2005: Dirty: 430356 kB Mon Nov 14 10:41:29 EST 2005: Dirty: 430352 kB Mon Nov 14 10:41:30 EST 2005: Dirty: 430352 kB Mon Nov 14 10:41:31 EST 2005: Dirty: 430352 kB Mon Nov 14 10:41:32 EST 2005: Dirty: 430352 kB Mon Nov 14 10:41:33 EST 2005: Dirty: 430356 kB Mon Nov 14 10:41:34 EST 2005: Dirty: 430356 kB Mon Nov 14 10:41:35 EST 2005: Dirty: 430356 kB Mon Nov 14 10:41:36 EST 2005: Dirty: 430356 kB Mon Nov 14 10:41:37 EST 2005: Dirty: 430368 kB Mon Nov 14 10:41:38 EST 2005: Dirty: 430364 kB Mon Nov 14 10:41:39 EST 2005: Dirty: 430360 kB Mon Nov 14 10:41:40 EST 2005: Dirty: 430364 kB Mon Nov 14 10:41:41 EST 2005: Dirty: 430364 kB Mon Nov 14 10:41:42 EST 2005: Dirty: 430368 kB Mon Nov 14 10:41:43 EST 2005: Dirty: 430368 kB Mon Nov 14 10:41:44 EST 2005: Dirty: 405552 kB Mon Nov 14 10:41:45 EST 2005: Dirty: 405552 kB Mon Nov 14 10:41:46 EST 2005: Dirty: 405552 kB Mon Nov 14 10:41:47 EST 2005: Dirty: 405552 kB Mon Nov 14 10:41:48 EST 2005: Dirty: 405556 kB Mon Nov 14 10:41:49 EST 2005: Dirty: 405548 kB Mon Nov 14 10:41:50 EST 2005: Dirty: 405548 kB Mon Nov 14 10:41:51 EST 2005: Dirty: 405548 kB Mon Nov 14 10:41:52 EST 2005: Dirty: 405548 kB Mon Nov 14 10:41:53 EST 2005: Dirty: 405552 kB Mon Nov 14 10:41:54 EST 2005: Dirty: 405492 kB Mon Nov 14 10:41:55 EST 2005: Dirty: 405492 kB Mon Nov 14 10:41:56 EST 2005: Dirty: 405492 kB Mon Nov 14 10:41:57 EST 2005: Dirty: 405524 kB Mon Nov 14 10:41:58 EST 2005: Dirty: 405528 kB Mon Nov 14 10:41:59 EST 2005: Dirty: 405524 kB Mon Nov 14 10:42:00 EST 2005: Dirty: 405524 kB Mon Nov 14 10:42:01 EST 2005: Dirty: 405524 kB Mon Nov 14 10:42:02 EST 2005: Dirty: 405536 kB Mon Nov 14 10:42:03 EST 2005: Dirty: 405536 kB Mon Nov 14 10:42:04 EST 2005: Dirty: 405536 kB Mon Nov 14 10:42:05 EST 2005: Dirty: 405536 kB Mon Nov 14 10:42:06 EST 2005: Dirty: 405536 kB Mon Nov 14 10:42:07 EST 2005: Dirty: 405536 kB Mon Nov 14 10:42:08 EST 2005: Dirty: 405536 kB Mon Nov 14 10:42:10 EST 2005: Dirty: 405532 kB Mon Nov 14 10:42:11 EST 2005: Dirty: 405532 kB Mon Nov 14 10:42:12 EST 2005: Dirty: 405532 kB Mon Nov 14 10:42:13 EST 2005: Dirty: 405532 kB Mon Nov 14 10:42:14 EST 2005: Dirty: 405544 kB Mon Nov 14 10:42:15 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:16 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:17 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:18 EST 2005: Dirty: 380680 kB Mon Nov 14 10:42:19 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:20 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:21 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:22 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:23 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:24 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:25 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:26 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:27 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:28 EST 2005: Dirty: 380680 kB Mon Nov 14 10:42:29 EST 2005: Dirty: 380668 kB Mon Nov 14 10:42:30 EST 2005: Dirty: 380668 kB Mon Nov 14 10:42:31 EST 2005: Dirty: 380668 kB Mon Nov 14 10:42:32 EST 2005: Dirty: 380668 kB Mon Nov 14 10:42:33 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:34 EST 2005: Dirty: 380628 kB Mon Nov 14 10:42:35 EST 2005: Dirty: 380628 kB Mon Nov 14 10:42:36 EST 2005: Dirty: 380628 kB Mon Nov 14 10:42:37 EST 2005: Dirty: 380632 kB Mon Nov 14 10:42:38 EST 2005: Dirty: 380672 kB Mon Nov 14 10:42:39 EST 2005: Dirty: 380672 kB Mon Nov 14 10:42:40 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:41 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:42 EST 2005: Dirty: 380676 kB Mon Nov 14 10:42:43 EST 2005: Dirty: 380684 kB Mon Nov 14 10:42:44 EST 2005: Dirty: 362476 kB Mon Nov 14 10:42:45 EST 2005: Dirty: 362476 kB Mon Nov 14 10:42:46 EST 2005: Dirty: 362476 kB Mon Nov 14 10:42:47 EST 2005: Dirty: 362476 kB Mon Nov 14 10:42:48 EST 2005: Dirty: 362476 kB Mon Nov 14 10:42:49 EST 2005: Dirty: 358340 kB Mon Nov 14 10:42:50 EST 2005: Dirty: 358340 kB Mon Nov 14 10:42:51 EST 2005: Dirty: 358340 kB Mon Nov 14 10:42:52 EST 2005: Dirty: 358340 kB Mon Nov 14 10:42:53 EST 2005: Dirty: 358340 kB Mon Nov 14 10:42:54 EST 2005: Dirty: 358340 kB Mon Nov 14 10:42:55 EST 2005: Dirty: 358340 kB Mon Nov 14 10:42:56 EST 2005: Dirty: 358340 kB Mon Nov 14 10:42:57 EST 2005: Dirty: 358340 kB Mon Nov 14 10:42:58 EST 2005: Dirty: 358344 kB Mon Nov 14 10:42:59 EST 2005: Dirty: 358344 kB Mon Nov 14 10:43:00 EST 2005: Dirty: 358344 kB Mon Nov 14 10:43:01 EST 2005: Dirty: 358344 kB Mon Nov 14 10:43:02 EST 2005: Dirty: 358352 kB Mon Nov 14 10:43:03 EST 2005: Dirty: 358352 kB Mon Nov 14 10:43:04 EST 2005: Dirty: 358348 kB Mon Nov 14 10:43:05 EST 2005: Dirty: 358348 kB Mon Nov 14 10:43:06 EST 2005: Dirty: 358348 kB Mon Nov 14 10:43:07 EST 2005: Dirty: 358348 kB Mon Nov 14 10:43:08 EST 2005: Dirty: 358352 kB Mon Nov 14 10:43:09 EST 2005: Dirty: 358340 kB Mon Nov 14 10:43:10 EST 2005: Dirty: 358340 kB Mon Nov 14 10:43:11 EST 2005: Dirty: 358340 kB Mon Nov 14 10:43:12 EST 2005: Dirty: 358340 kB Mon Nov 14 10:43:13 EST 2005: Dirty: 358344 kB Mon Nov 14 10:43:14 EST 2005: Dirty: 341716 kB Mon Nov 14 10:43:15 EST 2005: Dirty: 341716 kB Mon Nov 14 10:43:16 EST 2005: Dirty: 341716 kB Mon Nov 14 10:43:17 EST 2005: Dirty: 341756 kB Mon Nov 14 10:43:18 EST 2005: Dirty: 341756 kB Mon Nov 14 10:43:19 EST 2005: Dirty: 341748 kB Mon Nov 14 10:43:21 EST 2005: Dirty: 341748 kB Mon Nov 14 10:43:22 EST 2005: Dirty: 341748 kB Mon Nov 14 10:43:23 EST 2005: Dirty: 341752 kB Mon Nov 14 10:43:24 EST 2005: Dirty: 341752 kB Mon Nov 14 10:43:25 EST 2005: Dirty: 338268 kB Mon Nov 14 10:43:26 EST 2005: Dirty: 338268 kB Mon Nov 14 10:43:27 EST 2005: Dirty: 338268 kB Mon Nov 14 10:43:28 EST 2005: Dirty: 338276 kB Mon Nov 14 10:43:29 EST 2005: Dirty: 338268 kB Mon Nov 14 10:43:30 EST 2005: Dirty: 338268 kB Mon Nov 14 10:43:31 EST 2005: Dirty: 338268 kB Mon Nov 14 10:43:32 EST 2005: Dirty: 338268 kB Mon Nov 14 10:43:33 EST 2005: Dirty: 338272 kB Mon Nov 14 10:43:34 EST 2005: Dirty: 338272 kB Mon Nov 14 10:43:35 EST 2005: Dirty: 338272 kB Mon Nov 14 10:43:36 EST 2005: Dirty: 338272 kB Mon Nov 14 10:43:37 EST 2005: Dirty: 338276 kB Mon Nov 14 10:43:38 EST 2005: Dirty: 338280 kB Mon Nov 14 10:43:39 EST 2005: Dirty: 338276 kB Mon Nov 14 10:43:40 EST 2005: Dirty: 338280 kB Mon Nov 14 10:43:41 EST 2005: Dirty: 338280 kB Mon Nov 14 10:43:42 EST 2005: Dirty: 338280 kB Mon Nov 14 10:43:43 EST 2005: Dirty: 338288 kB Mon Nov 14 10:43:44 EST 2005: Dirty: 321708 kB Mon Nov 14 10:43:45 EST 2005: Dirty: 321708 kB Mon Nov 14 10:43:46 EST 2005: Dirty: 321708 kB Mon Nov 14 10:43:47 EST 2005: Dirty: 321708 kB Mon Nov 14 10:43:48 EST 2005: Dirty: 321708 kB Mon Nov 14 10:43:49 EST 2005: Dirty: 321704 kB Mon Nov 14 10:43:50 EST 2005: Dirty: 321704 kB Mon Nov 14 10:43:51 EST 2005: Dirty: 321704 kB Mon Nov 14 10:43:52 EST 2005: Dirty: 321704 kB Mon Nov 14 10:43:53 EST 2005: Dirty: 321708 kB Mon Nov 14 10:43:54 EST 2005: Dirty: 321656 kB Mon Nov 14 10:43:55 EST 2005: Dirty: 321656 kB Mon Nov 14 10:43:56 EST 2005: Dirty: 321656 kB Mon Nov 14 10:43:57 EST 2005: Dirty: 321656 kB Mon Nov 14 10:43:58 EST 2005: Dirty: 321688 kB Mon Nov 14 10:43:59 EST 2005: Dirty: 321684 kB Mon Nov 14 10:44:00 EST 2005: Dirty: 321684 kB Mon Nov 14 10:44:01 EST 2005: Dirty: 321684 kB Mon Nov 14 10:44:02 EST 2005: Dirty: 321696 kB Mon Nov 14 10:44:03 EST 2005: Dirty: 321696 kB Mon Nov 14 10:44:04 EST 2005: Dirty: 321696 kB Mon Nov 14 10:44:05 EST 2005: Dirty: 321696 kB Mon Nov 14 10:44:06 EST 2005: Dirty: 321696 kB Mon Nov 14 10:44:07 EST 2005: Dirty: 321696 kB Mon Nov 14 10:44:08 EST 2005: Dirty: 321696 kB Mon Nov 14 10:44:09 EST 2005: Dirty: 321692 kB Mon Nov 14 10:44:10 EST 2005: Dirty: 321692 kB Mon Nov 14 10:44:11 EST 2005: Dirty: 321692 kB Mon Nov 14 10:44:12 EST 2005: Dirty: 321692 kB Mon Nov 14 10:44:13 EST 2005: Dirty: 321692 kB Mon Nov 14 10:44:14 EST 2005: Dirty: 317604 kB Mon Nov 14 10:44:15 EST 2005: Dirty: 317604 kB Mon Nov 14 10:44:16 EST 2005: Dirty: 317608 kB Mon Nov 14 10:44:17 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:18 EST 2005: Dirty: 317616 kB Mon Nov 14 10:44:19 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:20 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:21 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:22 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:23 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:24 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:25 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:26 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:27 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:28 EST 2005: Dirty: 317616 kB Mon Nov 14 10:44:29 EST 2005: Dirty: 317608 kB Mon Nov 14 10:44:30 EST 2005: Dirty: 317608 kB Mon Nov 14 10:44:32 EST 2005: Dirty: 317608 kB Mon Nov 14 10:44:33 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:34 EST 2005: Dirty: 317612 kB Mon Nov 14 10:44:35 EST 2005: Dirty: 317564 kB Mon Nov 14 10:44:36 EST 2005: Dirty: 317564 kB Mon Nov 14 10:44:37 EST 2005: Dirty: 317568 kB Mon Nov 14 10:44:38 EST 2005: Dirty: 317608 kB Mon Nov 14 10:44:39 EST 2005: Dirty: 317616 kB
Here are all of the values from /proc/sys/vm/
0 block_dump
10 dirty_background_ratio
3000 dirty_expire_centisecs
40 dirty_ratio
500 dirty_writeback_centisecs
0 laptop_mode
0 legacy_va_layout
256 32 lowmem_reserve_ratio
65536 max_map_count
3831 min_free_kbytes
2 nr_pdflush_threads
0 overcommit_memory
50 overcommit_ratio
3 page-cluster
60 swappiness
300 swap_token_timeout
100 vfs_cache_pressureQuoted 39 lines · Mark Lord
Quoted 1 line · Arjan van de Ven
On Mon, 2005-11-14 at 10:30 -0500, Mark Lord wrote:
..
Quoted 2 lines
Quoted 2 lines
My Notebook computer has 2GB of RAM, and the 2.6.xx kernel seems quite happy to leave hundreds of MB of dirty unsync'd pages laying around
..
Quoted 2 lines
Quoted 2 lines
/proc/sys/vm/dirty_expire_centisecs= 3000 (30 seconds)/proc/sys/vm/dirty_writeback_centisecs= 500 (5 seconds)..
Quoted 2 lines
do you have laptop mode enabled? That changes the behavior bigtime in this regard and makes the kernel behave quite different.
No. Laptop-mode mostly just modifies the
dirty_expireand related settings, and I have them set as shown above. But there's also this:
/proc/sys/vm/laptop_mode= 0Quoted 2 lines
also if these are files written to by mmap, the kernel only really sees those as dirty when the mapping gets taken down
They certainly show up in the counts in
/proc/meminfounder "Dirty", so I assumed that means the kernel knows they are dirty.A simple test I do for this:
$ mkdir t $ cp /usr/src/*.bz2 t (about 400-500MB worth of kernel tar files)In another window, I do this:
$ while (sleep 1); do echo -n "`date`: "; grep Dirty /proc/meminfo; doneAnd then watch the count get large, but take virtually forever to count back down to a "safe" value.
Typing "sync" causes all the Dirty pages to immediately be flushed to disk, as expected.
Quoted 6 lines
Here's what the monitoring of
/proc/meminfoshows, on an otherwise mostly idle system after having done the big file copies noted earlier:Mon Nov 14 10:40:22 EST 2005: Dirty: 481284 kB
Mon Nov 14 10:40:23 EST 2005: Dirty: 479680 kB
Interesting. Since you have a very easy to reproduce case - can you write a program to do posix_fadvise(POSIX_FADV_DONTNEED) on those files in directory "t" and see what happens ?
Thanks,
Badari
Quoted 4 lines · Badari Pulavarty
Interesting. Since you have a very easy to reproduce case - can you write a program to do
posix_fadvise(POSIX_FADV_DONTNEED)on those files in directory "t" and see what happens ?
Sure. It appears to cause an immediate "sync" of the file data to disk (lots of drive activity for a few seconds), and the Dirty count from /proc/meminfo reduces correspondingly.
Oddly enough, the Dirty count didn't go all the way down, though. Doing a "sync" or a second run of the program afterwards does get the count down to zero immediately, without any significant I/O.
Strange.
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
int main (int argc, char *argv[])
{
while (--argc > 0) {
int fd = open(*++argv, O_RDWR);
if (fd == -1) {
perror(*argv);
} else {
int posix_fadvise(int, off_t, off_t, int);
const int POSIX_FADV_DONTNEED = 4;
int rc = posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED);
if (rc == -1)
perror(*argv);
close(fd);
}
}
return 0;
}Ahh... there's the magic line I needed:
#define _GNU_SOURCE
Another intriguing observation:
If I do my file copy test from files that are already cached, then the data usually gets committed to disk more or less right away.
If the copy is done from uncached files, then the data usually does not get committed to disk until I "sync" or run the posix_fadvise test program on it.
Anyone else seeing this? 2.6.15-rc1, x86, 2GB
Quoted 5 lines · Mark Lord
Another intriguing observation:
If I do my file copy test from files that are already cached, then the data usually gets committed to disk more or less right away.
And even in that case, /proc/meminfo still shows the memory as "Dirty", until a "sync" is done. Why is that?