how do file-mapped (mmapped) pages become dirty?
Hi,
When a process mmaps a file, how does the kernel know the memory has been written to (and hence the page is dirty)? Is this done by setting the protected flag, and when the memory is first written to it's set to dirty? What function is responsible for this setting? And when will the page be written back to disk (i.e. where's the flusher located)?
Thanks for any help,
Amir.
Quoted 4 lines · Amir Hermelin
Hi,
When a process mmaps a file, how does the kernel know the memory has been written to (and hence the page is dirty)? Is this done by setting the protected flag, and when the memory is first written to it's set to dirty?
No need on a i386. The cpu sets the page dirty in the pagetables when it is written to. It doesn't matter what kind of page it is.
Quoted 3 lines
What function is responsible for this setting? And when will the page be written back to disk (i.e. where's the flusher located)?
When there's memory pressure, or a sync.
Helge Hafting
Quoted 6 lines · Helge Hafting
Quoted 3 lines · Amir Hermelin
What function is responsible for this setting? And when will the page be written back to disk (i.e. where's the flusher located)?
When there's memory pressure, or a sync.
Note however that you need an msync() -- fsync() and fdatasync() do not catch changes to mmapped pages.
Chris
Signature · 4 lines
Chris Friesen | MailStop: 043/33/F10 Nortel Networks | work: (613) 765-0557 3500 Carling Avenue | fax: (613) 765-2986 Nepean, ON K2H 8E9 Canada | email: [email protected]
Quoted 6 lines · Chris Friesen
Quoted 5 lines · Helge Hafting
Quoted 3 lines · Amir Hermelin
What function is responsible for this setting? And when will the page be written back to disk (i.e. where's the flusher located)?
Quoted 7 lines
Quoted 1 line
When there's memory pressure, or a sync.
Note however that you need an
msync()--fsync()andfdatasync()do not catch changes to mmapped pages.Chris
Sure they do. fsync() will sync the whole file, regardless of whether or not it's been mapped. msync() allows you to sync only a specific portion and control how that portion is handled with some flags.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (797.90 BogoMips).
Note 96.31% of all statistics are fiction.
Quoted 9 lines · Richard B. Johnson
Quoted 2 lines · Chris Friesen
Note however that you need an
msync()--fsync()andfdatasync()do not catch changes to mmapped pages.Sure they do.
fsync()will sync the whole file, regardless of whether or not it's been mapped.msync()allows you to sync only a specific portion and control how that portion is handled with some flags.
According to Rik van Riel and HPA, fsync() does not check page tables for dirty bits. This is confirmed by my testing, in which msync() takes significantly longer than fsync() for the same file (in a ramdisk). msync() on the 3 pages dirtied took 39 usec, while fsync() on the whole 200KB file took 12 usec.
Also, some googling leads to to believe that if the file is mmapped and you close() it, it won't get synced.
Chris
Signature · 4 lines
Chris Friesen | MailStop: 043/33/F10 Nortel Networks | work: (613) 765-0557 3500 Carling Avenue | fax: (613) 765-2986 Nepean, ON K2H 8E9 Canada | email: [email protected]
Quoted 3 lines · Amir Hermelin
Hi,
When a process mmaps a file, how does the kernel know the memory has been written to (and hence the page is dirty)? Is this done by setting the
This is automatically done by the CPU (no overhead) with Intel CPUs.
Quoted 4 lines
protected flag, and when the memory is first written to it's set to dirty? What function is responsible for this setting? And when will the page be written back to disk (i.e. where's the flusher located)?
If you start to run low on memory, the disk buffer(s) may be flushed to the hardware. Otherwise, you need an explicit fsync() or a close().
Any reads by others will always show updated changes. You don't need to do anything special.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (797.90 BogoMips).
Note 96.31% of all statistics are fiction.
In article <006901c39d50$0b1313d0$2501a8c0@CARTMAN>, Amir Hermelin <[email protected]> wrote:
| When a process mmaps a file, how does the kernel know the memory has been | written to (and hence the page is dirty)? Is this done by setting the | protected flag, and when the memory is first written to it's set to dirty? | What function is responsible for this setting? And when will the page be | written back to disk (i.e. where's the flusher located)?
At least on x86, the CPU sets the dirty bit on write, although once upon a time less capable CPUs did it the way you suggest. That said, I think copy on write is still done the way you suggest, but look at the code if you really care. Or wait for someone to tell me I'm wrong ;-)
Signature · 3 lines
bill davidsen <[email protected]> CTO, TMR Associates, Inc Doing interesting things with little computers since 1979.