NFS mounts being mounted read-only depending upon mount order???
Subsystems (2)
All,
I'm having problems with the latest version of FC5 which contains kernel 2.6.18 - the problem is that NFS mounts of read-write file-systems are being mounted read-only, sometimes.
Whether a mount is mounted read-only or read-write seems to very depending upon what other directories were mounted from the same server and with what options.
Take the following file-systems exported from my
Linux Server server:
root@polyslo:/proc/fs/nfs# cat /etc/exports
/disk1/dir_ro *(ro,no_root_squash,sync,wdelay,no_subtree_check)
/disk1/dir_rw *(rw,no_root_squash,sync,wdelay,no_subtree_check)
root@polyslo:/proc/fs/nfs# showmount -e 10.20.20.48
Export list for 10.20.20.48:
/disk1/dir_rw *
/disk1/dir_ro *
root@polyslo:/proc/fs/nfs#If I now do the following mounts (in this order):
# Mount initial directory read-only - this is intentional mount -t nfs -o ro 10.20.20.48:/disk1/dir_ro /tmp/dir_ro
# Attempt to mount second directory read-write mount -t nfs -o rw 10.20.20.48:/disk1/dir_rw /tmp/dir_rw
# Now I attempt to touch a file in the rw directory:
[root@vt-1 tmp]# touch /tmp/dir_rw/hello
touch: cannot touch `/tmp/dir_rw/hello': Read-only file system
# And if I take a look in the /proc/mounts directory - I see # that the directory was truly mounted read-only
[root@vt-1 tmp]# grep 10.20.20.48 /proc/mounts
10.20.20.48:/disk1/dir_ro /tmp/dir_ro nfs ro,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0 10.20.20.48:/disk1/dir_rw /tmp/dir_rw nfs ro,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0 [root@vt-1 tmp]#
The above seems wrong to me - can someone clarify what's going on. I know that this is new behavior that was not true in the past. My RHEL_4U4 server running 2.6.9 kernel does not have this problem.
Notice that if I change the order of the mounts I get different behavior:
# Initially mount the read-write directory:
mount -t nfs -o rw 10.20.20.48:/disk1/dir_rw /tmp/dir_rw
# Then mount the read-only directory:
mount -t nfs -o ro 10.20.20.48:/disk1/dir_ro /tmp/dir_ro
# Now the directories are mounted as I wish, I can write to the # read-write mount:
[root@vt-1 tmp]# touch /tmp/dir_rw/hello
[root@vt-1 tmp]#
# And the read-only directory is read-only:
[root@vt-1 tmp]# touch /tmp/dir_ro/hello
touch: cannot touch `/tmp/dir_ro/hello': Read-only file system [root@vt-1 tmp]#
# And the /proc/mounts file reflects the states I expect
[root@vt-1 tmp]# grep 10.20.20.48 /proc/mounts
10.20.20.48:/disk1/dir_rw /tmp/dir_rw nfs rw,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0 10.20.20.48:/disk1/dir_ro /tmp/dir_ro nfs rw,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0 [root@vt-1 tmp]#
Note that previously the order did not matter and the permissions always reflected the permissions of the individual mount requests.
If someone could provide insights on what's going on above that would be great. This is a artificial test case - but it's actually hitting code we have in production and will be a significant issue for us.
Thanx much!
_Mike_
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
All,
Any insights on this?
I did find the following post from Trond which appears
http://lkml.org/lkml/2006/10/18/264
Which seems to imply that this is a 'feature' and not a 'bug'? Is that true - or have I missed something?
Thanx,
_Mike_
Quoted 83 lines · Michael Walker
All,
I'm having problems with the latest version of FC5 which contains kernel 2.6.18 - the problem is that NFS mounts of read-write file-systems are being mounted read-only, sometimes.
Whether a mount is mounted read-only or read-write seems to very depending upon what other directories were mounted from the same server and with what options.
Take the following file-systems exported from my
Linux Server server:root@polyslo:/proc/fs/nfs# cat /etc/exports /disk1/dir_ro *(ro,no_root_squash,sync,wdelay,no_subtree_check) /disk1/dir_rw *(rw,no_root_squash,sync,wdelay,no_subtree_check) root@polyslo:/proc/fs/nfs# showmount -e 10.20.20.48 Export list for 10.20.20.48: /disk1/dir_rw * /disk1/dir_ro * root@polyslo:/proc/fs/nfs#If I now do the following mounts (in this order):
# Mount initial directory read-only - this is intentional mount -t nfs -o ro 10.20.20.48:/disk1/dir_ro
/tmp/dir_ro# Attempt to mount second directory read-write mount -t nfs -o rw 10.20.20.48:/disk1/dir_rw
/tmp/dir_rw# Now I attempt to touch a file in the rw directory:
[root@vt-1 tmp]# touch/tmp/dir_rw/hello
touch: cannot touch `/tmp/dir_rw/hello': Read-only file system# And if I take a look in the
/proc/mountsdirectory - I see# that the directory was truly mounted read-only[root@vt-1 tmp]# grep 10.20.20.48 /proc/mounts 10.20.20.48:/disk1/dir_ro /tmp/dir_ro nfsro,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
10.20.20.48:/disk1/dir_rw/tmp/dir_rwnfs
ro,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
[root@vt-1 tmp]#The above seems wrong to me - can someone clarify what's going on. I know that this is new behavior that was not true in the past. My
RHEL_4U4server running 2.6.9 kernel does not have this problem.Notice that if I change the order of the mounts I get different behavior:
# Initially mount the read-write directory:
mount -t nfs -o rw 10.20.20.48:/disk1/dir_rw/tmp/dir_rw# Then mount the read-only directory:
mount -t nfs -o ro 10.20.20.48:/disk1/dir_ro/tmp/dir_ro# Now the directories are mounted as I wish, I can write to the
# read-write mount: [root@vt-1 tmp]# touch /tmp/dir_rw/hello [root@vt-1 tmp]## And the read-only directory is read-only:
[root@vt-1 tmp]# touch/tmp/dir_ro/hello
touch: cannot touch `/tmp/dir_ro/hello': Read-only file system [root@vt-1 tmp]## And the /proc/mounts file reflects the states I expect [root@vt-1 tmp]# grep 10.20.20.48 /proc/mounts 10.20.20.48:/disk1/dir_rw /tmp/dir_rw nfsrw,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
10.20.20.48:/disk1/dir_ro/tmp/dir_ronfs
rw,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
[root@vt-1 tmp]#Note that previously the order did not matter and the permissions always reflected the permissions of the individual mount requests.
Quoted 8 lines
If someone could provide insights on what's going on above that would be great. This is a artificial test case - but it's actually hitting code we have in production and will be a significant issue for us.
Thanx much!
_Mike_
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 11 lines · Michael Walker
All,
Any insights on this?
I did find the following post from Trond which appears
http://lkml.org/lkml/2006/10/18/264Which seems to imply that this is a 'feature' and not a 'bug'? Is that true - or have I missed something?
That is quite correct (thanks for taking the time to search the list archives). For various reasons, we're trying to tighten the NFS cache consistency. That means that practices that were previously allowed (such as mounting different directories from the same filesystem using different mount options) have been restricted.
The ability to have the same filesystem mounted both read-only and read-write will hopefully soon be allowed again, but it needs to be fixed at the VFS level. The patches to do so have circulated for quite some time now, but have not been merged due to disagreements over how the user interface for 'mount --bind' should be in this case.
Cheers
Trond
Quoted 98 lines
Thanx,
_Mike_Quoted 83 lines · Michael Walker
All,
I'm having problems with the latest version of FC5 which contains kernel 2.6.18 - the problem is that NFS mounts of read-write file-systems are being mounted read-only, sometimes.
Whether a mount is mounted read-only or read-write seems to very depending upon what other directories were mounted from the same server and with what options.
Take the following file-systems exported from my
Linux Server server:root@polyslo:/proc/fs/nfs# cat /etc/exports /disk1/dir_ro *(ro,no_root_squash,sync,wdelay,no_subtree_check) /disk1/dir_rw *(rw,no_root_squash,sync,wdelay,no_subtree_check) root@polyslo:/proc/fs/nfs# showmount -e 10.20.20.48 Export list for 10.20.20.48: /disk1/dir_rw * /disk1/dir_ro * root@polyslo:/proc/fs/nfs#If I now do the following mounts (in this order):
# Mount initial directory read-only - this is intentional mount -t nfs -o ro 10.20.20.48:/disk1/dir_ro
/tmp/dir_ro# Attempt to mount second directory read-write mount -t nfs -o rw 10.20.20.48:/disk1/dir_rw
/tmp/dir_rw# Now I attempt to touch a file in the rw directory:
[root@vt-1 tmp]# touch/tmp/dir_rw/hello
touch: cannot touch `/tmp/dir_rw/hello': Read-only file system# And if I take a look in the
/proc/mountsdirectory - I see# that the directory was truly mounted read-only[root@vt-1 tmp]# grep 10.20.20.48 /proc/mounts 10.20.20.48:/disk1/dir_ro /tmp/dir_ro nfsro,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
10.20.20.48:/disk1/dir_rw/tmp/dir_rwnfs
ro,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
[root@vt-1 tmp]#The above seems wrong to me - can someone clarify what's going on. I know that this is new behavior that was not true in the past. My
RHEL_4U4server running 2.6.9 kernel does not have this problem.Notice that if I change the order of the mounts I get different behavior:
# Initially mount the read-write directory:
mount -t nfs -o rw 10.20.20.48:/disk1/dir_rw/tmp/dir_rw# Then mount the read-only directory:
mount -t nfs -o ro 10.20.20.48:/disk1/dir_ro/tmp/dir_ro# Now the directories are mounted as I wish, I can write to the
# read-write mount: [root@vt-1 tmp]# touch /tmp/dir_rw/hello [root@vt-1 tmp]## And the read-only directory is read-only:
[root@vt-1 tmp]# touch/tmp/dir_ro/hello
touch: cannot touch `/tmp/dir_ro/hello': Read-only file system [root@vt-1 tmp]## And the /proc/mounts file reflects the states I expect [root@vt-1 tmp]# grep 10.20.20.48 /proc/mounts 10.20.20.48:/disk1/dir_rw /tmp/dir_rw nfsrw,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
10.20.20.48:/disk1/dir_ro/tmp/dir_ronfs
rw,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
[root@vt-1 tmp]#Note that previously the order did not matter and the permissions always reflected the permissions of the individual mount requests.
Quoted 8 lines
If someone could provide insights on what's going on above that would be great. This is a artificial test case - but it's actually hitting code we have in production and will be a significant issue for us.
Thanx much!
_Mike_
Quoted 8 lines
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
feature - Wow - I'm surprised part of this could be integrated without having the full solution put together.
At the moment we have a diskless NFS system setup
for booting our systems. We have the root as read-only and other mount points mounted read-write (/usr, /var, ...)
With this feature integrated we have to make sure our read-only and our read-write directories are in different physical partitions on the server? Wow - in my mind this is a drastic change to integrate without having the full solution present? Previously the only thing we had to worry about on the server was how the NFS exports were setup.
Can you point me to where the required 'mount --bind' fixes are being discussed? I haven't been able to find the thread in the 'lkml' (not that it isn't there somewhere).
Without the matching 'mount --bind' fixes this is a very serious regression for us?
Thanx for info.
_Mike_
Quoted 23 lines · Trond Myklebust
Quoted 10 lines · Michael Walker
All,
Any insights on this?
I did find the following post from Trond which appears
http://lkml.org/lkml/2006/10/18/264Which seems to imply that this is a 'feature' and not a 'bug'? Is that true - or have I missed something?
That is quite correct (thanks for taking the time to search the list archives). For various reasons, we're trying to tighten the NFS cache consistency. That means that practices that were previously allowed (such as mounting different directories from the same filesystem using different mount options) have been restricted.
The ability to have the same filesystem mounted both read-only and read-write will hopefully soon be allowed again, but it needs to be fixed at the VFS level. The patches to do so have circulated for quite some time now, but have not been merged due to disagreements over how the user interface for 'mount --bind' should be in this case.
Quoted 107 lines
Cheers
TrondQuoted 104 lines
Thanx,
_Mike_Quoted 82 lines · Michael Walker
All,
I'm having problems with the latest version of FC5 which contains kernel 2.6.18 - the problem is that NFS mounts of read-write file-systems are being mounted read-only, sometimes.
Whether a mount is mounted read-only or read-write seems to very depending upon what other directories were mounted from the same server and with what options.
Take the following file-systems exported from my
Linux Server server:root@polyslo:/proc/fs/nfs# cat /etc/exports /disk1/dir_ro *(ro,no_root_squash,sync,wdelay,no_subtree_check) /disk1/dir_rw *(rw,no_root_squash,sync,wdelay,no_subtree_check) root@polyslo:/proc/fs/nfs# showmount -e 10.20.20.48 Export list for 10.20.20.48: /disk1/dir_rw * /disk1/dir_ro * root@polyslo:/proc/fs/nfs#If I now do the following mounts (in this order):
# Mount initial directory read-only - this is intentional mount -t nfs -o ro 10.20.20.48:/disk1/dir_ro
/tmp/dir_ro# Attempt to mount second directory read-write mount -t nfs -o rw 10.20.20.48:/disk1/dir_rw
/tmp/dir_rw# Now I attempt to touch a file in the rw directory:
[root@vt-1 tmp]# touch/tmp/dir_rw/hello
touch: cannot touch `/tmp/dir_rw/hello': Read-only file system# And if I take a look in the
/proc/mountsdirectory - I see# that the directory was truly mounted read-only[root@vt-1 tmp]# grep 10.20.20.48 /proc/mounts 10.20.20.48:/disk1/dir_ro /tmp/dir_ro nfsro,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
10.20.20.48:/disk1/dir_rw/tmp/dir_rwnfs
ro,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
[root@vt-1 tmp]#The above seems wrong to me - can someone clarify what's going on. I know that this is new behavior that was not true in the past. My
RHEL_4U4server running 2.6.9 kernel does not have this problem.Notice that if I change the order of the mounts I get different behavior:
# Initially mount the read-write directory:
mount -t nfs -o rw 10.20.20.48:/disk1/dir_rw/tmp/dir_rw# Then mount the read-only directory:
mount -t nfs -o ro 10.20.20.48:/disk1/dir_ro/tmp/dir_ro# Now the directories are mounted as I wish, I can write to the
# read-write mount: [root@vt-1 tmp]# touch /tmp/dir_rw/hello [root@vt-1 tmp]## And the read-only directory is read-only:
[root@vt-1 tmp]# touch/tmp/dir_ro/hello
touch: cannot touch `/tmp/dir_ro/hello': Read-only file system [root@vt-1 tmp]## And the /proc/mounts file reflects the states I expect [root@vt-1 tmp]# grep 10.20.20.48 /proc/mounts 10.20.20.48:/disk1/dir_rw /tmp/dir_rw nfsrw,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
10.20.20.48:/disk1/dir_ro/tmp/dir_ronfs
rw,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=10.20.20.48 0 0
[root@vt-1 tmp]#Note that previously the order did not matter and the permissions always reflected the permissions of the individual mount requests.
Quoted 8 lines
If someone could provide insights on what's going on above that would be great. This is a artificial test case - but it's actually hitting code we have in production and will be a significant issue for us.
Thanx much!
_Mike_
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 9 lines · Michael Walker
feature - Wow - I'm surprised part of this could be integrated without having the full solution put together.
At the moment we have a diskless NFS system setup
for booting our systems. We have the root as read-only and other mount points mounted read-write (/usr, /var, ...)
Quoted 6 lines
With this feature integrated we have to make sure our read-only and our read-write directories are in different physical partitions on the server? Wow - in my mind this is a drastic change to integrate without having the full solution present? Previously the only thing we had to worry about on the server was how the NFS exports were setup.
There _are_ workarounds available. You can use the 'fsid' export option to tell knfsd to export the directories as if they were different filesystems.
For instance if I have
/export/home 141.211.133.0/24(rw,sync,no_subtree_check,fsid=0)
/export/home/trondmy 141.211.133.0/24(rw,sync,no_subtree_check,fsid=3)in /etc/exports on the server, then I can do
mount -t nfs -oro server:/export/home /mnt/foo mount -t nfs -orw server:/export/home/trondmy /mnt/bar
with no trouble. From 'cat /proc/mounts'
server:/export/home /mnt/foo nfs rw,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=server 0 0 server:/export/home/trondmy /mnt/bar nfs ro,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=server 0 0
as expected.
Quoted 6 lines
Can you point me to where the required 'mount --bind' fixes are being discussed? I haven't been able to find the thread in the 'lkml' (not that it isn't there somewhere).
Without the matching 'mount --bind' fixes this is a very serious regression for us?
I believe the most recent patchsets can be found from
http://marc.theaimsgroup.com/?l=linux-kernel&m=115447687730218&w=2
Cheers,
Trond
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
Quoted 16 lines · Trond Myklebust
On Mon, 2006-11-20 at 13:22 -0800, Michael Walker wrote:
There _are_ workarounds available. You can use the 'fsid' export option to tell knfsd to export the directories as if they were different filesystems.
For instance if I have
/export/home 141.211.133.0/24(rw,sync,no_subtree_check,fsid=0) /export/home/trondmy 141.211.133.0/24(rw,sync,no_subtree_check,fsid=3)in
/etc/exportson the server, then I can domount -t nfs -oro server:/export/home
/mnt/foomount -t nfs -orw server:/export/home/trondmy/mnt/bar
Quoted 6 lines
with no trouble. From 'cat
/proc/mounts'server:/export/home
/mnt/foonfs rw,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=server 0 0 server:/export/home/trondmy/mnt/barnfs ro,vers=3,rsize=32768,wsize=32768,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=server 0 0as expected.
Oh - promising.
So - would I need to make the fsid unique for each mount - or could I just divide them into two pools. 'fsid=1' for all 'read-only' exported filesystems and 'fsid=2' for all read-write exported filesystems?
We could possibly make something like this work.
I could possibly generate unique fsid's for each export - but if the above simple logic would be sufficient - that would be easiest.
Quoted 5 lines
Quoted 5 lines
Can you point me to where the required 'mount --bind' fixes are being discussed? I haven't been able to find the thread in the 'lkml' (not that it isn't there somewhere).
Quoted 7 lines
Quoted 2 lines
Without the matching 'mount --bind' fixes this is a very serious regression for us?
I believe the most recent patchsets can be found from
http://marc.theaimsgroup.com/?l=linux-kernel&m=115447687730218&w=2
Thanx for the pointer - I'll track this thread down....
_Mike_
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs