nfs cache bug (when server delete the file ,nfs client can read file also)

DISCUSSION fanchaoting 3 participants 7 messages Nov 23, 2012 – Nov 26, 2012

Subsystems (2)

NFS client NFS server

lore

Share & tools

lore

fanchaoting

author

2012-11-23 04:49 UTC #0 lore raw

Hi,everyone. I found a big bug abount nfs cache.

when server delete the file ,nfs client can read file also.

the following is the reproduce.

ip: 192.168.0.19 nfs-client
ip: 192.168.0.20 nfs-client
ip: 192.168.0.21 nfs-server

############################################################################

/usr/bin/ssh -n 192.168.0.21 service nfs start
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.19 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.20 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.21 "rm -rf /nfsroot; mkdir -p /nfsroot"
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -au
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -i -o insecure,no_root_squash,rw,fsid=0 *:/nfsroot

/usr/bin/ssh -n 192.168.0.19 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
/usr/bin/ssh -n 192.168.0.20 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)

/usr/bin/ssh -n 192.168.0.19 umount /nfsroot /usr/bin/ssh -n 192.168.0.20 umount /nfsroot
cmd="echo \"hello world\" > /nfsroot/tmpfile" /usr/bin/ssh -n 192.168.0.21 $cmd
/usr/bin/ssh -n 192.168.0.21 mkdir /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.21 touch /nfsroot/tmpdir/tmpdfile /usr/bin/ssh -n 192.168.0.19 mount -t nfs4 192.168.0.21:/ /nfsroot /usr/bin/ssh -n 192.168.0.20 mount -t nfs4 192.168.0.21:/ /nfsroot /usr/bin/ssh -n 192.168.0.21 cat /nfsroot/tmpfile /usr/bin/ssh -n 192.168.0.21 ls -l /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile /usr/bin/ssh -n 192.168.0.19 ls -l /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile /usr/bin/ssh -n 192.168.0.20 ls -l /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile > /dev/null /usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile > /dev/null /usr/bin/ssh -n 192.168.0.21 rm -rf /nfsroot/tmpfile echo -e "sleep 60~~~~~~~~\n"
sleep 60

#############################################################################

last: In 192.168.0.19 I do:

#cat /nfsroot/tmpfile <--the nfs server delete the file,but nfs client can read the file
hello world

I think when the nfs server delete the file ,
the server should notice the nfs client,
but the upstream kernel does't this.

Myklebust, Trond

2012-11-23 05:11 UTC #1 lore raw

T24gRnJpLCAyMDEyLTExLTIzIGF0IDEyOjE4ICswODAwLCBmYW5jaGFvdGluZyB3cm90ZToNCj4g SGksZXZlcnlvbmUuIEkgZm91bmQgYSBiaWcgYnVnIGFib3VudCBuZnMgY2FjaGUuDQo+IA0KPiB3 aGVuIHNlcnZlciBkZWxldGUgdGhlIGZpbGUgLG5mcyBjbGllbnQgY2FuIHJlYWQgZmlsZSBhbHNv Lg0KPiANCj4gdGhlIGZvbGxvd2luZyBpcyB0aGUgcmVwcm9kdWNlLg0KPiANCj4gaXA6IDE5Mi4x NjguMC4xOSAgbmZzLWNsaWVudA0KPiBpcDogMTkyLjE2OC4wLjIwICBuZnMtY2xpZW50DQo+IGlw OiAxOTIuMTY4LjAuMjEgIG5mcy1zZXJ2ZXINCj4gDQo+ICMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMNCj4g DQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgc2VydmljZSBuZnMgc3RhcnQNCj4gL3Vz ci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMSAvdXNyL3NiaW4vcnBjLmlkbWFwZA0KPiAvdXNyL2Jp bi9zc2ggLW4gMTkyLjE2OC4wLjE5IC91c3Ivc2Jpbi9ycGMuaWRtYXBkDQo+IC91c3IvYmluL3Nz aCAtbiAxOTIuMTY4LjAuMjAgL3Vzci9zYmluL3JwYy5pZG1hcGQNCj4gL3Vzci9iaW4vc3NoIC1u IDE5Mi4xNjguMC4yMSAicm0gLXJmIC9uZnNyb290OyBta2RpciAtcCAvbmZzcm9vdCINCj4gL3Vz ci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMSAvdXNyL3NiaW4vZXhwb3J0ZnMgLWF1DQo+IC91c3Iv YmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgL3Vzci9zYmluL2V4cG9ydGZzIC1pIC1vIGluc2VjdXJl LG5vX3Jvb3Rfc3F1YXNoLHJ3LGZzaWQ9MCAqOi9uZnNyb290DQo+IC91c3IvYmluL3NzaCAtbiAx OTIuMTY4LjAuMTkgdGVzdCAtZCAvbmZzcm9vdCB8fCAocm0gLXJmIC9uZnNyb290OyBta2RpciAt cCAvbmZzcm9vdCkNCj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMCB0ZXN0IC1kIC9uZnNy b290IHx8IChybSAtcmYgL25mc3Jvb3Q7IG1rZGlyIC1wIC9uZnNyb290KQ0KPiANCj4gL3Vzci9i aW4vc3NoIC1uIDE5Mi4xNjguMC4xOSB1bW91bnQgL25mc3Jvb3QNCj4gL3Vzci9iaW4vc3NoIC1u IDE5Mi4xNjguMC4yMCB1bW91bnQgL25mc3Jvb3QNCj4gY21kPSJlY2hvIFwiaGVsbG8gd29ybGRc IiA+IC9uZnNyb290L3RtcGZpbGUiDQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgJGNt ZA0KPiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIG1rZGlyIC9uZnNyb290L3RtcGRpcg0K PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIHRvdWNoIC9uZnNyb290L3RtcGRpci90bXBk ZmlsZQ0KPiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjE5IG1vdW50IC10IG5mczQgMTkyLjE2 OC4wLjIxOi8gL25mc3Jvb3QNCj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMCBtb3VudCAt dCBuZnM0IDE5Mi4xNjguMC4yMTovIC9uZnNyb290DQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4 LjAuMjEgY2F0ICAvbmZzcm9vdC90bXBmaWxlDQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAu MjEgbHMgLWwgL25mc3Jvb3QvdG1wZGlyDQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMTkg Y2F0ICAvbmZzcm9vdC90bXBmaWxlDQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMTkgbHMg LWwgIC9uZnNyb290L3RtcGRpcg0KPiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIwIGNhdCAg L25mc3Jvb3QvdG1wZmlsZQ0KPiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIwIGxzIC1sICAv bmZzcm9vdC90bXBkaXINCj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4xOSBjYXQgL25mc3Jv b3QvdG1wZmlsZSA+IC9kZXYvbnVsbA0KPiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIwIGNh dCAvbmZzcm9vdC90bXBmaWxlID4gL2Rldi9udWxsDQo+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4 LjAuMjEgcm0gLXJmIC9uZnNyb290L3RtcGZpbGUNCj4gZWNobyAtZSAic2xlZXAgNjB+fn5+fn5+ flxuIg0KPiBzbGVlcCA2MA0KPiANCj4gIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMNCj4gDQo+IGxhc3Q6 IEluIDE5Mi4xNjguMC4xOSBJIGRvOg0KPiANCj4gI2NhdCAvbmZzcm9vdC90bXBmaWxlICAgICAg ICAgICA8LS10aGUgbmZzIHNlcnZlciBkZWxldGUgdGhlIGZpbGUsYnV0IG5mcyBjbGllbnQgY2Fu IHJlYWQgdGhlIGZpbGUNCj4gIGhlbGxvIHdvcmxkDQo+IA0KPiANCj4gSSB0aGluayAgd2hlbiB0 aGUgbmZzIHNlcnZlciBkZWxldGUgdGhlIGZpbGUgLA0KPiB0aGUgIHNlcnZlciBzaG91bGQgbm90 aWNlIHRoZSBuZnMgY2xpZW50LA0KPiBidXQgdGhlIHVwc3RyZWFtIGtlcm5lbCBkb2VzJ3QgdGhp cy4NCg0KU28gaXMgdGhpcyBhIHByb2JsZW0gd2l0aCB0aGUgY2xpZW50IG9yIHRoZSBzZXJ2ZXI/ IEluIG90aGVyIHdvcmRzLCBpZg0KeW91IHVzZSBhIGRpZmZlcmVudCBzZXJ2ZXIvY2xpZW50IGNv bWJpbmF0aW9uLCBkbyB5b3Ugc2VlIGEgZGlmZmVyZW50DQpyZXN1bHQ/DQoNCldoaWNoIGtlcm5l bHMgYXJlIHlvdSB1c2luZyBpbiB5b3VyIHRlc3RzIGZvciB0aGUgY2xpZW50IGFuZCBzZXJ2ZXI/ DQoNCi0tIA0KVHJvbmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWluZXINCg0K TmV0QXBwDQpUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0Kd3d3Lm5ldGFwcC5jb20NCg==

fanchaoting

author

2012-11-23 05:17 UTC #2 lore raw

Quoted 51 lines · Trond

Quoted 50 lines · fanchaoting
Hi,everyone. I found a big bug abount nfs cache.
when server delete the file ,nfs client can read file also.
the following is the reproduce.
ip: 192.168.0.19 nfs-client
ip: 192.168.0.20 nfs-client
ip: 192.168.0.21 nfs-server
############################################################################
/usr/bin/ssh -n 192.168.0.21 service nfs start
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.19 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.20 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.21 "rm -rf /nfsroot; mkdir -p /nfsroot"
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -au
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -i -o insecure,no_root_squash,rw,fsid=0 *:/nfsroot
/usr/bin/ssh -n 192.168.0.19 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
/usr/bin/ssh -n 192.168.0.20 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
/usr/bin/ssh -n 192.168.0.19 umount /nfsroot /usr/bin/ssh -n 192.168.0.20 umount /nfsroot
cmd="echo \"hello world\" > /nfsroot/tmpfile" /usr/bin/ssh -n 192.168.0.21 $cmd
/usr/bin/ssh -n 192.168.0.21 mkdir /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.21 touch /nfsroot/tmpdir/tmpdfile /usr/bin/ssh -n 192.168.0.19 mount -t nfs4 192.168.0.21:/ /nfsroot /usr/bin/ssh -n 192.168.0.20 mount -t nfs4 192.168.0.21:/ /nfsroot /usr/bin/ssh -n 192.168.0.21 cat /nfsroot/tmpfile /usr/bin/ssh -n 192.168.0.21 ls -l /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile /usr/bin/ssh -n 192.168.0.19 ls -l /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile /usr/bin/ssh -n 192.168.0.20 ls -l /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile > /dev/null /usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile > /dev/null /usr/bin/ssh -n 192.168.0.21 rm -rf /nfsroot/tmpfile echo -e "sleep 60~~~~~~~~\n"
sleep 60
#############################################################################
last: In 192.168.0.19 I do:
#cat /nfsroot/tmpfile <--the nfs server delete the file,but nfs client can read the file
hello world

Quoted 8 lines

Quoted 3 lines
I think when the nfs server delete the file ,
the server should notice the nfs client,
but the upstream kernel does't this.
So is this a problem with the client or the server? In other words, if you use a different server/client combination, do you see a different result?

I think that the server has the problem .when the server deletes the file , it should notice the client immediately.

Quoted 2 lines

Which kernels are you using in your tests for the client and server?

I found the problem in kernel 3.7.0-rc6

Myklebust, Trond

2012-11-23 05:36 UTC #3 lore raw

T24gRnJpLCAyMDEyLTExLTIzIGF0IDEzOjIzICswODAwLCBmYW5jaGFvdGluZyB3cm90ZToNCj4g TXlrbGVidXN0LCBUcm9uZCDlhpnpgZM6DQo+ID4gT24gRnJpLCAyMDEyLTExLTIzIGF0IDEyOjE4 ICswODAwLCBmYW5jaGFvdGluZyB3cm90ZToNCj4gPj4gSGksZXZlcnlvbmUuIEkgZm91bmQgYSBi aWcgYnVnIGFib3VudCBuZnMgY2FjaGUuDQo+ID4+DQo+ID4+IHdoZW4gc2VydmVyIGRlbGV0ZSB0 aGUgZmlsZSAsbmZzIGNsaWVudCBjYW4gcmVhZCBmaWxlIGFsc28uDQo+ID4+DQo+ID4+IHRoZSBm b2xsb3dpbmcgaXMgdGhlIHJlcHJvZHVjZS4NCj4gPj4NCj4gPj4gaXA6IDE5Mi4xNjguMC4xOSAg bmZzLWNsaWVudA0KPiA+PiBpcDogMTkyLjE2OC4wLjIwICBuZnMtY2xpZW50DQo+ID4+IGlwOiAx OTIuMTY4LjAuMjEgIG5mcy1zZXJ2ZXINCj4gPj4NCj4gPj4gIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIw0K PiA+Pg0KPiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIHNlcnZpY2UgbmZzIHN0YXJ0 DQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgL3Vzci9zYmluL3JwYy5pZG1hcGQN Cj4gPj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4xOSAvdXNyL3NiaW4vcnBjLmlkbWFwZA0K PiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIwIC91c3Ivc2Jpbi9ycGMuaWRtYXBkDQo+ ID4+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgInJtIC1yZiAvbmZzcm9vdDsgbWtkaXIg LXAgL25mc3Jvb3QiDQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgL3Vzci9zYmlu L2V4cG9ydGZzIC1hdQ0KPiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIC91c3Ivc2Jp bi9leHBvcnRmcyAtaSAtbyBpbnNlY3VyZSxub19yb290X3NxdWFzaCxydyxmc2lkPTAgKjovbmZz cm9vdA0KPiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjE5IHRlc3QgLWQgL25mc3Jvb3Qg fHwgKHJtIC1yZiAvbmZzcm9vdDsgbWtkaXIgLXAgL25mc3Jvb3QpDQo+ID4+IC91c3IvYmluL3Nz aCAtbiAxOTIuMTY4LjAuMjAgdGVzdCAtZCAvbmZzcm9vdCB8fCAocm0gLXJmIC9uZnNyb290OyBt a2RpciAtcCAvbmZzcm9vdCkNCj4gPj4NCj4gPj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4x OSB1bW91bnQgL25mc3Jvb3QNCj4gPj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMCB1bW91 bnQgL25mc3Jvb3QNCj4gPj4gY21kPSJlY2hvIFwiaGVsbG8gd29ybGRcIiA+IC9uZnNyb290L3Rt cGZpbGUiDQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgJGNtZA0KPiA+PiAvdXNy L2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIG1rZGlyIC9uZnNyb290L3RtcGRpcg0KPiA+PiAvdXNy L2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjIxIHRvdWNoIC9uZnNyb290L3RtcGRpci90bXBkZmlsZQ0K PiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTkyLjE2OC4wLjE5IG1vdW50IC10IG5mczQgMTkyLjE2OC4w LjIxOi8gL25mc3Jvb3QNCj4gPj4gL3Vzci9iaW4vc3NoIC1uIDE5Mi4xNjguMC4yMCBtb3VudCAt dCBuZnM0IDE5Mi4xNjguMC4yMTovIC9uZnNyb290DQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIu MTY4LjAuMjEgY2F0ICAvbmZzcm9vdC90bXBmaWxlDQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIu MTY4LjAuMjEgbHMgLWwgL25mc3Jvb3QvdG1wZGlyDQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIu MTY4LjAuMTkgY2F0ICAvbmZzcm9vdC90bXBmaWxlDQo+ID4+IC91c3IvYmluL3NzaCAtbiAxOTIu MTY4LjAuMTkgbHMgLWwgIC9uZnNyb290L3RtcGRpcg0KPiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTky LjE2OC4wLjIwIGNhdCAgL25mc3Jvb3QvdG1wZmlsZQ0KPiA+PiAvdXNyL2Jpbi9zc2ggLW4gMTky LjE2OC4wLjIwIGxzIC1sICAvbmZzcm9vdC90bXBkaXINCj4gPj4gL3Vzci9iaW4vc3NoIC1uIDE5 Mi4xNjguMC4xOSBjYXQgL25mc3Jvb3QvdG1wZmlsZSA+IC9kZXYvbnVsbA0KPiA+PiAvdXNyL2Jp bi9zc2ggLW4gMTkyLjE2OC4wLjIwIGNhdCAvbmZzcm9vdC90bXBmaWxlID4gL2Rldi9udWxsDQo+ ID4+IC91c3IvYmluL3NzaCAtbiAxOTIuMTY4LjAuMjEgcm0gLXJmIC9uZnNyb290L3RtcGZpbGUN Cj4gPj4gZWNobyAtZSAic2xlZXAgNjB+fn5+fn5+flxuIg0KPiA+PiBzbGVlcCA2MA0KPiA+Pg0K PiA+PiAjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIw0KPiA+Pg0KPiA+PiBsYXN0OiBJbiAxOTIuMTY4LjAu MTkgSSBkbzoNCj4gPj4NCj4gPj4gI2NhdCAvbmZzcm9vdC90bXBmaWxlICAgICAgICAgICA8LS10 aGUgbmZzIHNlcnZlciBkZWxldGUgdGhlIGZpbGUsYnV0IG5mcyBjbGllbnQgY2FuIHJlYWQgdGhl IGZpbGUNCj4gPj4gIGhlbGxvIHdvcmxkDQo+ID4+DQo+ID4+DQo+ID4+IEkgdGhpbmsgIHdoZW4g dGhlIG5mcyBzZXJ2ZXIgZGVsZXRlIHRoZSBmaWxlICwNCj4gPj4gdGhlICBzZXJ2ZXIgc2hvdWxk IG5vdGljZSB0aGUgbmZzIGNsaWVudCwNCj4gPj4gYnV0IHRoZSB1cHN0cmVhbSBrZXJuZWwgZG9l cyd0IHRoaXMuDQo+ID4gDQo+ID4gU28gaXMgdGhpcyBhIHByb2JsZW0gd2l0aCB0aGUgY2xpZW50 IG9yIHRoZSBzZXJ2ZXI/IEluIG90aGVyIHdvcmRzLCBpZg0KPiA+IHlvdSB1c2UgYSBkaWZmZXJl bnQgc2VydmVyL2NsaWVudCBjb21iaW5hdGlvbiwgZG8geW91IHNlZSBhIGRpZmZlcmVudA0KPiA+ IHJlc3VsdD8NCj4gPiANCj4gDQo+IEkgdGhpbmsgdGhhdCB0aGUgc2VydmVyIGhhcyB0aGUgcHJv YmxlbSAud2hlbiB0aGUgc2VydmVyIGRlbGV0ZXMgdGhlIGZpbGUgLA0KPiBpdCBzaG91bGQgbm90 aWNlIHRoZSBjbGllbnQgaW1tZWRpYXRlbHkuDQoNClRoZXJlIGlzIG5vIG5vdGlmaWNhdGlvbiBt ZWNoYW5pc20gaW4gTkZTOyBvbiBvcGVuKCksIHRoZSBjbGllbnQgaXMNCnN1cHBvc2VkIHRvIHJl dmFsaWRhdGUgaXRzIGNhY2hlZCBpbmZvcm1hdGlvbiBhbmQgdGhlIHNlcnZlciBpcyBzdXBwb3Nl ZA0KdG8gcmV0dXJuIGFuIEVTVEFMRSBlcnJvciBpZiB0aGUgZmlsZWhhbmRsZSBpcyBubyBsb25n ZXIgdmFsaWQuIEVpdGhlcg0Kb25lIG9mIHRoZXNlIDIgbWVjaGFuaXNtcyAoY2xpZW50IHJldmFs aWRhdGlvbiBvciBzZXJ2ZXIgcmVwbHkpIGNvdWxkIGJlDQpnb2luZyB3cm9uZyBoZXJlLCB3aGlj aCBpcyB3aHkgSSdtIGFza2luZy4NCg0KPiA+IFdoaWNoIGtlcm5lbHMgYXJlIHlvdSB1c2luZyBp biB5b3VyIHRlc3RzIGZvciB0aGUgY2xpZW50IGFuZCBzZXJ2ZXI/DQo+ID4gDQo+IA0KPiBJIGZv dW5kIHRoZSBwcm9ibGVtIGluIGtlcm5lbCAgMy43LjAtcmM2DQoNCk9LLiBEbyB5b3UgaGF2ZSBh IG5vbi1MaW51eCBzZXJ2ZXIgb3IgY2xpZW50IGF2YWlsYWJsZSB0aGF0IHlvdSBjYW4gdXNlDQpm b3IgdGVzdGluZz8gQWx0ZXJuYXRpdmVseSwgY291bGQgeW91IHVzZSB3aXJlc2hhcmsgdG8gY2Fw dHVyZSBhIGR1bXAgb2YNCnRoZSBORlMgdHJhZmZpYyBiZXR3ZWVuIHRoZSBjbGllbnQgYW5kIHNl cnZlcj8NCg0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5l cg0KDQpOZXRBcHANClRyb25kLk15a2xlYnVzdEBuZXRhcHAuY29tDQp3d3cubmV0YXBwLmNvbQ0K

fanchaoting

author

2012-11-23 05:53 UTC #4 lore raw

Quoted 60 lines · Trond

Quoted 59 lines · fanchaoting
Quoted 58 lines · Trond
Quoted 46 lines · fanchaoting
Hi,everyone. I found a big bug abount nfs cache.
when server delete the file ,nfs client can read file also.
the following is the reproduce.
ip: 192.168.0.19 nfs-client
ip: 192.168.0.20 nfs-client
ip: 192.168.0.21 nfs-server
############################################################################
/usr/bin/ssh -n 192.168.0.21 service nfs start
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.19 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.20 /usr/sbin/rpc.idmapd
/usr/bin/ssh -n 192.168.0.21 "rm -rf /nfsroot; mkdir -p /nfsroot"
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -au
/usr/bin/ssh -n 192.168.0.21 /usr/sbin/exportfs -i -o insecure,no_root_squash,rw,fsid=0 *:/nfsroot
/usr/bin/ssh -n 192.168.0.19 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
/usr/bin/ssh -n 192.168.0.20 test -d /nfsroot || (rm -rf /nfsroot; mkdir -p /nfsroot)
/usr/bin/ssh -n 192.168.0.19 umount /nfsroot /usr/bin/ssh -n 192.168.0.20 umount /nfsroot
cmd="echo \"hello world\" > /nfsroot/tmpfile" /usr/bin/ssh -n 192.168.0.21 $cmd
/usr/bin/ssh -n 192.168.0.21 mkdir /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.21 touch /nfsroot/tmpdir/tmpdfile /usr/bin/ssh -n 192.168.0.19 mount -t nfs4 192.168.0.21:/ /nfsroot /usr/bin/ssh -n 192.168.0.20 mount -t nfs4 192.168.0.21:/ /nfsroot /usr/bin/ssh -n 192.168.0.21 cat /nfsroot/tmpfile /usr/bin/ssh -n 192.168.0.21 ls -l /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile /usr/bin/ssh -n 192.168.0.19 ls -l /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile /usr/bin/ssh -n 192.168.0.20 ls -l /nfsroot/tmpdir /usr/bin/ssh -n 192.168.0.19 cat /nfsroot/tmpfile > /dev/null /usr/bin/ssh -n 192.168.0.20 cat /nfsroot/tmpfile > /dev/null /usr/bin/ssh -n 192.168.0.21 rm -rf /nfsroot/tmpfile echo -e "sleep 60~~~~~~~~\n"
sleep 60
#############################################################################
last: In 192.168.0.19 I do:
Quoted 7 lines
#cat /nfsroot/tmpfile <--the nfs server delete the file,but nfs client can read the file
hello world
I think when the nfs server delete the file ,
the server should notice the nfs client,
but the upstream kernel does't this.
So is this a problem with the client or the server? In other words, if you use a different server/client combination, do you see a different result?

Quoted 8 lines

Quoted 2 lines
I think that the server has the problem .when the server deletes the file , it should notice the client immediately.
There is no notification mechanism in NFS; on open(), the client is supposed to revalidate its cached information and the server is supposed to return an ESTALE error if the filehandle is no longer valid. Either one of these 2 mechanisms (client revalidation or server reply) could be going wrong here, which is why I'm asking.

I found J. Bruce Fields's patch(break delegations on unlink) maybe solve this problem . But I did't found it in the upstream kernel.

I think when the server delete the file, it should reply a DELEGRETURN to the nfs client.

Quoted 8 lines

Quoted 4 lines
Quoted 3 lines
Which kernels are you using in your tests for the client and server?
I found the problem in kernel 3.7.0-rc6
OK. Do you have a non-Linux server or client available that you can use for testing? Alternatively, could you use wireshark to capture a dump of the NFS traffic between the client and server?

In the wireshark, I found the right traffic should have open operation, but now i can't find the open operation.

I found the function nfs4_open_prepare has problem , in the old kernel , it can run

...snip...

  rpc_call_start(task);

...snip...

Quoted 1 line

[email protected]

2012-11-23 17:20 UTC #5 lore raw

Quoted 14 lines · fanchaoting

Quoted 13 lines · Trond
Quoted 11 lines · fanchaoting
Quoted 8 lines · Trond
Quoted 3 lines · fanchaoting
I think when the nfs server delete the file ,
the server should notice the nfs client,
but the upstream kernel does't this.
So is this a problem with the client or the server? In other words, if you use a different server/client combination, do you see a different result?
I think that the server has the problem .when the server deletes the file , it should notice the client immediately.

Quoted 8 lines

Quoted 5 lines
There is no notification mechanism in NFS; on open(), the client is supposed to revalidate its cached information and the server is supposed to return an ESTALE error if the filehandle is no longer valid. Either one of these 2 mechanisms (client revalidation or server reply) could be going wrong here, which is why I'm asking.
I found J. Bruce Fields's patch(break delegations on unlink) maybe solve this problem . But I did't found it in the upstream kernel.

Yes, still working on getting that merged. Next step is probably to rebase it on top of jlayton's ESTALE patches, assuming those will go in first. In theory they could make 3.8 but that may be optimistic.

--b.

fanchaoting

author

2012-11-26 05:34 UTC #6 lore raw

Quoted 19 lines · [email protected] 写道:

Quoted 18 lines · fanchaoting
Quoted 17 lines · Trond
Quoted 11 lines · fanchaoting
Quoted 8 lines · Trond
Quoted 3 lines · fanchaoting
I think when the nfs server delete the file ,
the server should notice the nfs client,
but the upstream kernel does't this.
So is this a problem with the client or the server? In other words, if you use a different server/client combination, do you see a different result?
I think that the server has the problem .when the server deletes the file , it should notice the client immediately.
There is no notification mechanism in NFS; on open(), the client is supposed to revalidate its cached information and the server is supposed to return an ESTALE error if the filehandle is no longer valid. Either one of these 2 mechanisms (client revalidation or server reply) could be going wrong here, which is why I'm asking.

Quoted 7 lines

Quoted 2 lines
I found J. Bruce Fields's patch(break delegations on unlink) maybe solve this problem . But I did't found it in the upstream kernel.
Yes, still working on getting that merged. Next step is probably to rebase it on top of jlayton's ESTALE patches, assuming those will go in first. In theory they could make 3.8 but that may be optimistic.

Yes, when I applyed the patch, this problem was solved. I found when server deleted the file , it can return "DELEGRETURN" to the nfs client.

Quoted 7 lines

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html