msync() more expensive than fsync()?
I have some code that runs on a ramdisk-based filesystem. Through a special device it mmaps a section of memory that is persistant over reboots, to be used for logging. In order to guarantee that the logs were flushed to the memory area, I assume that I need to use some kind of sync operation. This is where things get a bit odd.
I did some testing with relatively small messages, 50 bytes or so, with results as follows:
Without any explicit flushing it takes 8 usec to log a message.
If I msync() only the pages that were touched in writing (usually 3 pages) it takes 39 usecs to log a message.
If I fsync() the entire file (200KB) it takes 12 usec to log a message.
Why the additional cost for msync()? I would have thought it would be faster since it is explicitely for mmapped memory areas. As a side note, the difference is even more extreme if a file is used on a disk-backed filesystem.
The kernel was 2.4.18.
Chris
Signature · 4 lines
Chris Friesen | MailStop: 043/33/F10 Nortel Networks | work: (613) 765-0557 3500 Carling Avenue | fax: (613) 765-2986 Nepean, ON K2H 8E9 Canada | email: [email protected]
Quoted 6 lines · Chris Friesen
If I
msync()only the pages that were touched in writing (usually 3 pages) it takes 39 usecs to log a message.If I
fsync()the entire file (200KB) it takes 12 usec to log a message.Why the additional cost for
msync()?
If you only write into the file through an mmap()d area, then I guess that fsync() is a NOP since it doesn't check the page tables for dirty bits, while msync() does check the page tables.
This also means that fsync() won't see any dirty bits as long as they're still only in the page tables, and consequently won't write anything to disk, while msync() will write.
Note that I don't remember the details 100%, but IIRC it was something like this.
Followup to: <[email protected]> By author: Chris Friesen <[email protected]> In newsgroup: linux.dev.kernel
Quoted 12 lines
Without any explicit flushing it takes 8 usec to log a message.
If I
msync()only the pages that were touched in writing (usually 3 pages) it takes 39 usecs to log a message.If I
fsync()the entire file (200KB) it takes 12 usec to log a message.Why the additional cost for
msync()? I would have thought it would be faster since it is explicitely for mmapped memory areas. As a side note, the difference is even more extreme if a file is used on a disk-backed filesystem.
Because fsync() does less work. In particular, fsync() doesn't do the work you want: unlike msync() it doesn't guarantee that memory maps are consistent with the filesystem.
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64