Skip to contentSkip to conversation

mtime updates for mmapped files.

DISCUSSION Rogier Wolff 5 participants 6 messages Jan 16, 2008

lore
Share & tools
lore
#0 lore raw

Hi,

I wrote a small app yesterday that updates a file by mmapping the file (RW), changing the thing around, and then exiting.

This did not trigger a change in the mtime of the file. Thus rsync didn't pick up that the file had changed.

I understand that tracking every change to a RW mmapped file is costly, and thus unfeasable, but shouldn't then the close cause a mtime update?

The server where this happened is running 2.6.21, so my apologies if this has already been corrected.

Roger.

Signature · 6 lines
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2600998 **
**    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. 
Does it sit on the couch all day? Is it unemployed? Please be specific! 
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ
#1 lore raw
Quoted 9 lines · Rogier Wolff

Hi,

I wrote a small app yesterday that updates a file by mmapping the file (RW), changing the thing around, and then exiting.

This did not trigger a change in the mtime of the file. Thus rsync didn't pick up that the file had changed.

Quoted 6 lines

I understand that tracking every change to a RW mmapped file is costly, and thus unfeasable, but shouldn't then the close cause a mtime update?

The server where this happened is running 2.6.21, so my apologies if this has already been corrected.

This is an old bug, don't know if it was fixed or not. Please see
http://lkml.org/lkml/2006/5/17/138

Quoted 16 lines

Roger.

--
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2600998 **
** Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. Does it sit on the couch all day? Is it unemployed? Please be specific! Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ --
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#2 lore raw
Quoted 13 lines
Quoted 13 lines

Hi,

I wrote a small app yesterday that updates a file by mmapping the file (RW), changing the thing around, and then exiting.

This did not trigger a change in the mtime of the file. Thus rsync didn't pick up that the file had changed.

I understand that tracking every change to a RW mmapped file is costly, and thus unfeasable, but shouldn't then the close cause a mtime update?

Quoted 5 lines
Quoted 2 lines

The server where this happened is running 2.6.21, so my apologies if this has already been corrected.

This is an old bug, don't know if it was fixed or not. Please see
http://lkml.org/lkml/2006/5/17/138

It's not fixed yet, but Anton Salikhmetov is currently working on it:

http://lkml.org/lkml/2008/1/15/202

Miklos

#3 lore raw
In reply to Dave Young
Quoted 14 lines · Dave Young
Quoted 13 lines · Rogier Wolff

Hi,

I wrote a small app yesterday that updates a file by mmapping the file (RW), changing the thing around, and then exiting.

This did not trigger a change in the mtime of the file. Thus rsync didn't pick up that the file had changed.

I understand that tracking every change to a RW mmapped file is costly, and thus unfeasable, but shouldn't then the close cause a mtime update?

Quoted 5 lines
Quoted 2 lines

The server where this happened is running 2.6.21, so my apologies if this has already been corrected.

This is an old bug, don't know if it was fixed or not. Please see
http://lkml.org/lkml/2006/5/17/138

If it was fixed, we'd not be having the currently ongoing long thread about how to fix it. :)

See also:
http://lkml.org/lkml/2007/2/21/242
http://lkml.org/lkml/2008/1/7/234

(Note the last is the start of the *current* thread...)

#4 lore raw
In reply to Rogier Wolff

2008/1/16, Rogier Wolff <[email protected]>:

Quoted 9 lines

Hi,

I wrote a small app yesterday that updates a file by mmapping the file (RW), changing the thing around, and then exiting.

This did not trigger a change in the mtime of the file. Thus rsync didn't pick up that the file had changed.

Quoted 6 lines

I understand that tracking every change to a RW mmapped file is costly, and thus unfeasable, but shouldn't then the close cause a mtime update?

The server where this happened is running 2.6.21, so my apologies if this has already been corrected.

Unfortunately, this issue has not been fully fixed yet.

My last attempt (http://lkml.org/lkml/2008/1/15/202) to solve this problem has a couple of drawbacks:

1) calling a possibly sleeping function from atomic context - I've already corrected this;

2) there's a very special case with retouching the memory-mapped data after a call to msync() with MS_ASYNC.

I'm still working on the latter case, but I guess that I have found a solution.

If you badly need a quick fix, I can send my working unreleased patch to you. I reckon that your particular problem will be fixed by this patch. Let me know if you want that.

However, if your application calls msync() with the MS_ASYNC flag, it's better to wait a little bit more - I'll release the next version of my solution
shortly.

Quoted 16 lines

Roger.

--
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2600998 **
** Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. Does it sit on the couch all day? Is it unemployed? Please be specific! Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ --
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#5 lore raw
Quoted 11 lines · Anton Salikhmetov

Unfortunately, this issue has not been fully fixed yet.

My last attempt (http://lkml.org/lkml/2008/1/15/202) to solve this problem has a couple of drawbacks:

1) calling a possibly sleeping function from atomic context - I've already corrected this;

2) there's a very special case with retouching the memory-mapped data after a call to msync() with MS_ASYNC.

Quoted 6 lines

I'm still working on the latter case, but I guess that I have found a solution.

If you badly need a quick fix, I can send my working unreleased patch to you. I reckon that your particular problem will be fixed by this patch. Let me know if you want that.

No need. Thanks for the offer.

Quoted 4 lines

However, if your application calls msync() with the MS_ASYNC flag, it's better to wait a little bit more - I'll release the next version of my solution
shortly.

My application calls "exit" (*). My workaround was to type

touch <destfile>

to get things to work. In my situation, I was worried about the more general case, and not really about my personal situation.

I could integrate the "touch <destfile>" into my application, but I think it is the OS's duty to do this for me. This would be the easiest non-manual fix for me.

Roger.

(*) So please check that:
- mmap file
- change mmapped area
- exit
also works (modifies the timestamp).

Signature · 6 lines
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2600998 **
**    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. 
Does it sit on the couch all day? Is it unemployed? Please be specific! 
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ