Skip to contentSkip to conversation

Force client page cache invalidation?

DISCUSSION ralph gonzalez 3 participants 4 messages Feb 24, 2004 – Feb 26, 2004

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

We tend to run multiple applications on a single client, which all mmap (usually readonly) the same file on an nfs-mounted volume. If the file changes on the server, then we would like clients to see these changes upon closing/reopening this file.

On Solaris (which we are moving from), this would work fine. However Linux 2.4.22-10mdk (nfs utils 1.0.5) seems to be unwilling to reload the file cache *when another application already has the file mmapped*.

Therefore each time the file is editted on the server, we have to force all applications mapping this file (on the client in question) to close the file *simultaneously*, and then allow them to reopen the mapping.

*Question*: Instead, is there a way to force the client to reload the file cache (exclusively for this file)? (So when we subsequently close/open the file in each application asynchronously, they see the correct pages.) Maybe there's a simple option or utility I'm missing?

A separate problem is that the file attributes are cached, so the file cache is only reloaded if we force the client to reread the file attributes whenever the file is editted on the server. This we accomplish by the trick of locking/unlocking the file immediately after opening it, to avoid disabling mount-level attribute caching via the noac option.

Thanks for any suggestions!

Signature · 13 lines
Ralph Gonzalez
[email protected]



-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
#1 lore raw

På ty , 24/02/2004 klokka 10:30, skreiv ralph gonzalez:

Quoted 1 line

We tend to run multiple applications on a single client, which all mmap

(usually readonly) the same file on an nfs-mounted volume.

Quoted 1 line

If the file changes on the server, then we would like clients to see

these changes upon closing/reopening this file.

Quoted 2 lines

On Solaris (which we are moving from), this would work fine. However

Linux 2.4.22-10mdk (nfs utils 1.0.5) seems to be unwilling to reload the file cache *when another application already has the file mmapped*.

Which is quite sane semantics for mmap().

Quoted 1 line

Therefore each time the file is editted on the server, we have to force

all applications mapping this file (on the client in question) to close the file *simultaneously*, and then allow them to reopen the mapping.

Quoted 2 lines

*Question*: Instead, is there a way to force the client to reload the

file cache (exclusively for this file)? (So when we subsequently close/open the file in each application asynchronously, they see the correct pages.) Maybe there's a simple option or utility I'm missing?

Linux does not support this sort of kludge and is unlikely to do so for the near future. The VM simply cannot support it.

In any case, the fact is that strict POSIX mmap() semantics assume exclusive access to the file on the server while it is mapped on a given client. If you're not providing that, then you are going to be prone to some nasty races whether or not the cache invalidation works like Solaris does.

Cheers,
Trond

-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now. Build and deploy apps & Web services for Linux with a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#2 lore raw
Quoted 2 lines

Linux does not support this sort of kludge and is unlikely to do so for the near future. The VM simply cannot support it.

What about using fadvise(POSIX_FADV_DONTNEED) - shouldn't this do the trick? Or does it refuse to invalidate mapped pages?

Olaf

Signature · 13 lines
Olaf Kirch     |  Stop wasting entropy - start using predictable
[email protected]   |  tempfile names today!
---------------+ 


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
#3 lore raw

På to , 26/02/2004 klokka 00:57, skreiv Olaf Kirch:

Quoted 1 line
Quoted 1 line

Linux does not support this sort of kludge and is unlikely to do so

for

Quoted 4 lines
Quoted 1 line

the near future. The VM simply cannot support it.

What about using fadvise(POSIX_FADV_DONTNEED) - shouldn't this do the trick? Or does it refuse to invalidate mapped pages?

fadvise(POSIX_FADV_DONTNEED) uses the same interface that NFS does to invalidate pages (invalidate_mapping_pages()), so it has the same
weaknesses: if you have unmapped those pages since whenever it was NFS tried to invalidate them, then it will work, but there is no way for the user to guarantee that it will.
Moreover, fadvise() is only available under Linux 2.6.x...

I did, however, see an encouraging posting by Daniel Phillips on the linux-kernel list yesterday, which indicates he is working on this problem. See the thread about "Distributed mmap API".

Cheers,
Trond

-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now. Build and deploy apps & Web services for Linux with a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs