re-exporting NFS-mounted dir over NFS
Subsystems (2)
I am in the process of setting up an "NFS-proxy", a machine which mounts directories from an NetApp-filer using NFS3, and should re-export them using NFS4 (with Kerberos).
But apparently nfsd doesn't support re-exporting NFS-mounted dirs:
# mount | grep home netapp.example.com:/vol/home/schumar on /srv/nfs4/home/schumar type nfs # cat /etc/exports /srv/nfs4/home/schumar gss/krb5(rw,sync,fsid=0,secure,no_subtree_check) # exportfs -r exportfs: Warning: /srv/nfs4/home/schumar does not support NFS export.
I traced this back to linux-2.6.25.1/fs/nfsd/export.c, where it says, starting in line 386:
if (!inode->i_sb->s_export_op ||
!inode->i_sb->s_export_op->fh_to_dentry) {
dprintk("exp_export: export of invalid fs type.\n");
return -EINVAL;
}(and a quick look in fs/nfs/super.c confirmed that the nfs-client never sets an export_op)
Is there a technical reason for this (i.e. is it simply theoretically impossible to re-export an NFS-mount), or is my use-case so strange that nobody has ever needed this until now, and thus it just wasn't coded? Or am I just stupid/blind? (wouldn't be the first time :)
Thanks in advance,
Signature · 3 lines
Infineon Technologies IT-Services GmbH Martin.Schuster1-d0qZbvYSIPpWk0Htik3J/[email protected] Lakeside B05, 9020 Klagenfurt, Austria Martin Schuster FB: LG Klagenfurt, FN 246787y +43 5 1777 3517
Quoted 23 lines · Martin Schuster (IFKL IT OS DSM CD)
I am in the process of setting up an "NFS-proxy", a machine which mounts directories from an NetApp-filer using NFS3, and should re-export them using NFS4 (with Kerberos).
But apparently nfsd doesn't support re-exporting NFS-mounted dirs:
# mount | grep home netapp.example.com:/vol/home/schumar on /srv/nfs4/home/schumar type nfs # cat /etc/exports /srv/nfs4/home/schumar gss/krb5(rw,sync,fsid=0,secure,no_subtree_check) # exportfs -r exportfs: Warning: /srv/nfs4/home/schumar does not support NFS export.I traced this back to
linux-2.6.25.1/fs/nfsd/export.c, where it says, starting in line 386:if (!inode->i_sb->s_export_op || !inode->i_sb->s_export_op->fh_to_dentry) { dprintk("exp_export: export of invalid fs type.\n"); return -EINVAL; }(and a quick look in
fs/nfs/super.cconfirmed that the nfs-client never sets anexport_op)
Quoted 7 lines
Is there a technical reason for this (i.e. is it simply theoretically impossible to re-export an NFS-mount), or is my use-case so strange that nobody has ever needed this until now, and thus it just wasn't coded? Or am I just stupid/blind? (wouldn't be the first time :)
Thanks in advance,
Is the real goal to be able to export the files using krb5 authentication or the use of NFSv4?
If the former, then why not just export the files from the NetApp using Kerberos?
If the latter, then I suspect that it won't provide much, if any, benefit. It would still be limited to the NFSv3 semantics of the file system.
Thanx...
ps
Thanks for your thoughts about this.
Quoted 3 lines · Peter Staubach
Is the real goal to be able to export the files using krb5 authentication or the use of NFSv4?
Both, I fear.
Quoted 7 lines
If the former, then why not just export the files from the NetApp using Kerberos?
If the latter, then I suspect that it won't provide much, if any, benefit. It would still be limited to the NFSv3 semantics of the file system.
The current NFS4-support in NetApps OnTap is afaik quite new, so our filer administrator doesn't want to enable it in the near future; he prefers waiting until the issues that are likely to come up are solved before allowing it on a productive machine.
But mounting directly from the filer using NFS3+Kerberos would allow the following attack vector, as the clients are in an unsecure network (i.e. could get root access on their machines):
User mounts an directory using his Kerberos-credentials User gets root, then changes w/o password to another user User can now read the files of that other user, as the NFS3-server
doesn't check the permissions
(at least, that's how I understood the difference between NFS3
and NFS4 -- please correct me if I'm wrong)
So my question still is: Is re-exporting an NFS-mount technically impossible, or does it just need some coding to get it working?
Thanks in advance,
Signature · 3 lines
Infineon Technologies IT-Services GmbH Martin.Schuster1-d0qZbvYSIPpWk0Htik3J/[email protected] Lakeside B05, 9020 Klagenfurt, Austria Martin Schuster FB: LG Klagenfurt, FN 246787y +43 5 1777 3517
Quoted 32 lines · Martin Schuster (IFKL IT OS DSM CD)
Thanks for your thoughts about this.
Quoted 4 lines · Peter Staubach
Is the real goal to be able to export the files using krb5 authentication or the use of NFSv4?
Both, I fear.
Quoted 8 lines
If the former, then why not just export the files from the NetApp using Kerberos?
If the latter, then I suspect that it won't provide much, if any, benefit. It would still be limited to the NFSv3 semantics of the file system.
The current NFS4-support in NetApps OnTap is afaik quite new, so our filer administrator doesn't want to enable it in the near future; he prefers waiting until the issues that are likely to come up are solved before allowing it on a productive machine.
But mounting directly from the filer using NFS3+Kerberos would allow the following attack vector, as the clients are in an unsecure network (i.e. could get root access on their machines):
User mounts an directory using his Kerberos-credentials User gets root, then changes w/o password to another user User can now read the files of that other user, as the NFS3-server
doesn't check the permissions
Quoted 4 lines
(at least, that's how I understood the difference between NFS3
and NFS4 -- please correct me if I'm wrong)
Ahh, no. All versions of the NFS servers check permissions on each and every file access. Even NFSv2. NFSv3 and NFSv4 support an ACCESS protocol operation which allows the client to ask the server for which file access permissions that the user would be allowed to have.
When a file system is exported using krb5, then all file accesses must be made with the right kerberos credential or access will be denied. The attack that you described, while working for AUTH_SYS, does not work for RPCSEC_GSS with krb5, no matter which version of the NFS protocol that you are using.
Quoted 2 lines
So my question still is: Is re-exporting an NFS-mount technically impossible, or does it just need some coding to get it working?
It may be technically possible, in some situations, but is not something that is always possible or has ever been supported. It could easily be used to thwart security. A compromised client, which was allowed access to file systems from the server, could then re-export those file systems to other clients which should not have been allowed access.
----
The bottom line is that 1) I don't think that the NFSv4 implementation from NetApp is as bad as feared and 2) that using NFSv3 with krb5 should be as secure as NFSv4 with krb5. Give either or both a shot. I think that you will be pleasantly surprised.
ps
Quoted 11 lines · Martin Schuster (IFKL IT OS DSM CD)
Thanks for your thoughts about this.
Quoted 3 lines · Peter Staubach
Is the real goal to be able to export the files using krb5 authentication or the use of NFSv4?
Both, I fear.
Quoted 3 lines
If the former, then why not just export the files from the NetApp using Kerberos?
Quoted 8 lines
Quoted 4 lines
If the latter, then I suspect that it won't provide much, if any, benefit. It would still be limited to the NFSv3 semantics of the file system.
The current NFS4-support in NetApps OnTap is afaik quite new, so our filer administrator doesn't want to enable it in the near future; he prefers waiting until the issues that are likely to come up are solved before allowing it on a productive machine.
The NFSv4 server in OnTAP is several years old, actually. There were some problems with it in the 6.5 and 7.0 time frame, but if your filer is running a current OnTAP release (7.2.2, I think, is the most recent GD release) you should be fine. The "early adopter period" is long over for NFSv4 support in OnTAP.
Mounting the filer directly will definitely be more secure (and perform better) than going through an NFSv3 gateway. I think Peter has covered most of the details already.
Quoted 2 lines
So my question still is: Is re-exporting an NFS-mount technically impossible, or does it just need some coding to get it working?
Aside from the fact that it is not advisable to do, I don't think it is technically possible on Linux with the kernel NFS server. NFS-exportable file systems must have special hooks to construct NFS file handles from on-disk inodes correctly, and I don't think the Linux NFS client provides those hooks. You might have better luck using the user-space NFS server on your gateway system, but it has plenty of known issues.
Signature · 1 line
Chuck Lever
Quoted 20 lines · Martin Schuster (IFKL IT OS DSM CD)
I am in the process of setting up an "NFS-proxy", a machine which mounts directories from an NetApp-filer using NFS3, and should re-export them using NFS4 (with Kerberos).
But apparently nfsd doesn't support re-exporting NFS-mounted dirs:
# mount | grep home netapp.example.com:/vol/home/schumar on /srv/nfs4/home/schumar type nfs # cat /etc/exports /srv/nfs4/home/schumar gss/krb5(rw,sync,fsid=0,secure,no_subtree_check) # exportfs -r exportfs: Warning: /srv/nfs4/home/schumar does not support NFS export.I traced this back to
linux-2.6.25.1/fs/nfsd/export.c, where it says, starting in line 386:if (!inode->i_sb->s_export_op || !inode->i_sb->s_export_op->fh_to_dentry) { dprintk("exp_export: export of invalid fs type.\n"); return -EINVAL; }
Quoted 6 lines
(and a quick look in
fs/nfs/super.cconfirmed that the nfs-client never sets anexport_op)Is there a technical reason for this (i.e. is it simply theoretically impossible to re-export an NFS-mount), or is my use-case so strange that nobody has ever needed this until now, and thus it just wasn't coded?
It's certainly not trivial to implement re-export. It's probably also possible, at least in theory, but would be difficult (and probably wouldn't work terribly well). So, like Peter Staubach says, I'd try to talk the server administrator into turning on krb5 on the filer.
--b.
Quoted 11 lines
Or am I just stupid/blind? (wouldn't be the first time :)
Thanks in advance,
Signature · 7 lines
Infineon Technologies IT-Services GmbH Martin.Schuster1-d0qZbvYSIPpWk0Htik3J/[email protected] Lakeside B05, 9020 Klagenfurt, Austria Martin Schuster FB: LG Klagenfurt, FN 246787y +43 5 1777 3517 -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 3 lines · Peter Staubach
The bottom line is that 1) I don't think that the NFSv4 implementation from NetApp is as bad as feared and 2) that using NFSv3 with krb5 should be as secure as NFSv4 with krb5.
That's true for the protocol itself, though depending on the threat you're worried about, the fact that NFSv4 allows the equivalent of mount calls to be done with krb5 security (and thus thwarts spoofing of the replies) may be an advantage for NFSv4.
--b.
Quoted 2 lines
Give either or both a shot. I think that you will be pleasantly surprised.