Skip to contentSkip to conversation

Problems with POSIX ACL <=> NFSv4 ACL mapping in mainline

DISCUSSION Andreas Gruenbacher 4 participants 8 messages Jun 23, 2006 – Jun 26, 2006

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hello,

we have a problem with the POSIX ACL <=> NFSv4 ACL mapping as currently implemented in NFSv4 in mainline: it is implemented on the server side only, which leads to interesting problems. I'll try to explain.

POSIX ACLs [1] and NFSv4 ACLs [2] are based on very different concepts, and so they don't map onto each other very well. Marius and others are proposing a mapping between the two formats [3], and an implementation of that mapping seems to have found its way into the mainline nfsv4 implementation. (Note that I didn't actually verify the implemented mapping actually reflects draft [3].)

The mapping from POSIX ACLs to NFSv4 ACLs and back is implemented only on the server side only right now: POSIX ACLs on the server are presented as NFSv4 ACLs over the network, and as "system.nfs4_acl" extended attributes to user-space on the client. This is no problem when moving or copying files between NFSv4 mounts, but other filesystems on the client don't know how to deal with "system.nfs4_acl" attributes, and so preserving those attributes will inevitably fail. What's really bad is that this will even fail for filesystems that support POSIX ACLs.

The CITI has patches that try to work around this in libacl by trying both POSIX ACLs and "system.nfs4_acl" attributes, and converting between the two if necessary. This approach has a number of disadvantages: (1) applications that don't use libacl will still fail, (2) always trying both formats without knowing the properties of the underlying filesystem will always be slow, (3) it exposes a deficiency of NFSv4 instead of trying to hide it.

In addition, I could imagine that we'll optionally support local NFSv4-style ACLs on specific file systems somewhen in the future. (Samba would benefit greatly from that for example; Solaris already can do that.) Not being able to tell the one kind of ACLs from the other in a world where we may have both sounds like a recipe for disaster to me.

I could imagine the following solutions to this problem:

* Don't map between POSIX ACLs and NFSv4 ACLs at all, and use a different protocol for POSIX ACLs and for NFSv4 ACLs instead (e.g., version 4 of the NFSACL protocol extension). The protocol extension is not currently implemented in Solaris because current Solaris supports NFSv4 ACLs natively instead of POSIX ACLs, and so they don't seem to have as strong a motivation to keep POSIX ACLs working well as we currently do. The NFSACL protocol has the huge advantage of supporting POSIX ACLs pretty well, and so a lot of the overhead and complexity of doing the NFSv4 mapping would go away though.

* Map back from NFSv4 ACLs to POSIX ACLs on the client side, so that POSIX ACLs on the server will appear as POSIX ACLs on the client. This would take away all the mess we currently have in user-space and limit the mapping problems to the NFSv4 protocol, where it really occurs. We could define an additional NFsv4 attribute that the server could use to declare if it supports POSIX ACLs only, so that the client won't have to try to convert from NFSv4 to POSIX ACLs for servers that have native NFSv4 ACLs.

* Go with the user-space solution, and implement the mapping in libacl. This solution is ugly, slow, and adds complexity to an unrelated piece of software. Some applications will break, and there is no hope that we'll ever get rid of this complex hack in the future, for as long as libacl will still exist. Yuck!

* Don't support POSIX ACLs over NFSv4 at all.

What do you think?

Thanks,
Andreas

[1] POSIX 1003.1e/2c draft 17, withdrawn,
http://wt.xpilot.org/publications/posix.1e/download.html

[2] Spencer Shepler et al.: Network File System (NFS) version 4 Protocol, section 5.11. Access Control Lists, http://www.ietf.org/rfc/rfc3530.txt

[3] Marius Aamodt Eriksen et al.: Mapping Between NFSv4 and Posix Draft ACLs,
http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-acl-mapping-04.txt

Signature · 10 lines
Andreas Gruenbacher <[email protected]>
Novell / SUSE Labs

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
#1 lore raw
Quoted 5 lines · Andreas Gruenbacher

they don't map onto each other very well. Marius and others are proposing a mapping between the two formats [3], and an implementation of that mapping seems to have found its way into the mainline nfsv4 implementation. (Note that I didn't actually verify the implemented mapping actually reflects draft [3].)

It doesn't; the implementation is based on the previous version. The newer version requires the server to perform a (possibly lossy) mapping of virtually any NFSv4 ACL, whereas the previous (currently implemented) version only accepted a very small subset of possible NFSv4 ACLs.

Quoted 4 lines

The CITI has patches that try to work around this in libacl by trying both POSIX ACLs and "system.nfs4_acl" attributes, and converting between the two if necessary. This approach has a number of disadvantages: (1) applications that don't use libacl will still fail

Are there particular examples of such applications that you're thinking of?

Quoted 2 lines

(2) always trying both formats without knowing the properties of the underlying filesystem will always be slow,

How much slower? I'd expect the cost of one system call (which just immediately returns -ENOTSUPP) to be insignificant.

Quoted 5 lines

In addition, I could imagine that we'll optionally support local NFSv4-style ACLs on specific file systems somewhen in the future. (Samba would benefit greatly from that for example; Solaris already can do that.) Not being able to tell the one kind of ACLs from the other in a world where we may have both sounds like a recipe for disaster to me.

I'm not following you. Currently nfsv4 and posix acls are being presented to userspace via different extended attributes, so you can tell one type from the other.

Quoted 10 lines

I could imagine the following solutions to this problem:

* Don't map between POSIX ACLs and NFSv4 ACLs at all, and use a different protocol for POSIX ACLs and for NFSv4 ACLs instead (e.g., version 4 of the NFSACL protocol extension). The protocol extension is not currently implemented in Solaris because current Solaris supports NFSv4 ACLs natively instead of POSIX ACLs, and so they don't seem to have as strong a motivation to keep POSIX ACLs working well as we currently do. The NFSACL protocol has the huge advantage of supporting POSIX ACLs pretty well, and so a lot of the overhead and complexity of doing the NFSv4 mapping would go away though.

If you wanted to support *both* POSIX and NFSv4 ACLs, then you'd still have the same problem. Unless you expect it to be possible to set both types of ACLs on the same file independently, and make the kernel enforce the AND of the two, or something like that. I don't think that would be helpful.

Quoted 3 lines

* Map back from NFSv4 ACLs to POSIX ACLs on the client side, so that POSIX ACLs on the server will appear as POSIX ACLs on the client. This would take away all the mess we currently have in user-space

And put it in the kernel instead? We originally did exactly that, and Trond argued that we should do this in userspace instead. I still think that was the right decision.

I'm not satisfied with the current situation either. But I don't think Samba or NFSv4 will ever be completely happy until we provide good support for NFSv4/Windows-style ACLs. We can't rip out POSIX ACLs in the process, so we're going to have to support both. I don't see how to support both well without mapping between them. And if we've got to do this complicated mapping, let's do it in userspace whenever possible.

--b.

#2 lore raw
Quoted 6 lines · J. Bruce Fields
Quoted 4 lines

The CITI has patches that try to work around this in libacl by trying both POSIX ACLs and "system.nfs4_acl" attributes, and converting between the two if necessary. This approach has a number of disadvantages: (1) applications that don't use libacl will still fail

Are there particular examples of such applications that you're thinking of?

Anything that simply copies xattrs from a file on NFSv4 to a file on another filesystem fails. The specific case I ran into was that I didn't apply the CITI libacl patches because I am unconvinced about the general approach, and I didn't feel well with adding the whole mess to libacl late in the SLES10 game, let alone testing the code. The result were errors when copying files with cp -p (which preserves xattrs).

I much dislike adding a whole lot of complexity to libacl for a specific corner case only because of the NFSv4 to POSIX ACL mapping. This mapping should really be hidden from applications wherever possible (and only applications that specifically deal with implementing the mapping should have to deal with it).

Quoted 5 lines
Quoted 2 lines

(2) always trying both formats without knowing the properties of the underlying filesystem will always be slow,

How much slower? I'd expect the cost of one system call (which just immediately returns -ENOTSUPP) to be insignificant.

The time overhead probably is low, yes. It's still a mess, and it will fail if ever we'll have a file system that understands both forms of ACLs. (How to resolve such conflicts is another question of course.)

Quoted 9 lines
Quoted 5 lines

In addition, I could imagine that we'll optionally support local NFSv4-style ACLs on specific file systems somewhen in the future. (Samba would benefit greatly from that for example; Solaris already can do that.) Not being able to tell the one kind of ACLs from the other in a world where we may have both sounds like a recipe for disaster to me.

I'm not following you. Currently nfsv4 and posix acls are being presented to userspace via different extended attributes, so you can tell one type from the other.

Except on NFSv4, we see POSIX ACLs represented as "system.nfs4_acl". I'd be much happier to see POSIX ACLs represented as POSIX ACLs, and NFSv4 ACLs and NFSv4 ACLs.

Quoted 12 lines
Quoted 11 lines

I could imagine the following solutions to this problem:

* Don't map between POSIX ACLs and NFSv4 ACLs at all, and use a different protocol for POSIX ACLs and for NFSv4 ACLs instead (e.g., version 4 of the NFSACL protocol extension). The protocol extension is not currently implemented in Solaris because current Solaris supports NFSv4 ACLs natively instead of POSIX ACLs, and so they don't seem to have as strong a motivation to keep POSIX ACLs working well as we currently do. The NFSACL protocol has the huge advantage of supporting POSIX ACLs pretty well, and so a lot of the overhead and complexity of doing the NFSv4 mapping would go away though.

Quoted 4 lines

If you wanted to support *both* POSIX and NFSv4 ACLs, then you'd still have the same problem. Unless you expect it to be possible to set both types of ACLs on the same file independently, and make the kernel enforce the AND of the two, or something like that. I don't think that would be helpful.

Both at the same time probably is not a good idea, agreed. Perhaps storing an NFSv4 ACL when NFSv4 ACL tools are used and POSIX ACLs when POSIX ACL tools are used might benting version 4 of the NFSACL side-band protocol and dumping the NFSv4 ACL <=> POSIX ACL mapping in nfsd would completely get rid of no-op (hopefully) rounds of mappings. That would mean less complexity in the kernel (we already have version 3 NFSACL), and complexity in user spaceecome an option, or some such.

Quoted 5 lines
Quoted 3 lines

* Map back from NFSv4 ACLs to POSIX ACLs on the client side, so that POSIX ACLs on the server will appear as POSIX ACLs on the client. This would take away all the mess we currently have in user-space

And put it in the kernel instead?

Yes.

Quoted 2 lines

We originally did exactly that, and Trond argued that we should do this in userspace instead. I still think that was the right decision.

Things would look different with native NFSv4 ACLs as on Solaris, but we are not in this situation.

Quoted 6 lines

I'm not satisfied with the current situation either. But I don't think Samba or NFSv4 will ever be completely happy until we provide good support for NFSv4/Windows-style ACLs. We can't rip out POSIX ACLs in the process, so we're going to have to support both. I don't see how to support both well without mapping between them. And if we've got to do this complicated mapping, let's do it in userspace whenever possible.

Implementing version 4 of the NFSACL side-band protocol and dumping the NFSv4 ACL <=> POSIX ACL mapping in nfsd would completely get rid of no-op (hopefully) rounds of mappings. That would mean less complexity in the kernel (we already have version 3 NFSACL, and version 4 is essentially the same).

Andreas

#3 lore raw
Quoted 2 lines · Andreas Gruenbacher

I much dislike adding a whole lot of complexity to libacl for a specific corner case only because of the NFSv4 to POSIX ACL mapping.

That's certainly understandable, but...

Quoted 3 lines

This mapping should really be hidden from applications wherever possible (and only applications that specifically deal with implementing the mapping should have to deal with it).

... short of putting the mapping into the kernel, putting it in libacl seems the closest we can come to hiding it from applications. Maybe there are other ways we could connect libacl with the NFSv4 mapping code that would minimize the impact on libacl?

The current libacl patches need some work in any case.

Quoted 9 lines
Quoted 5 lines
Quoted 2 lines

(2) always trying both formats without knowing the properties of the underlying filesystem will always be slow,

How much slower? I'd expect the cost of one system call (which just immediately returns -ENOTSUPP) to be insignificant.

The time overhead probably is low, yes. It's still a mess, and it will fail if ever we'll have a file system that understands both forms of ACLs. (How to resolve such conflicts is another question of course.)

You've referred several times to the case of a file system that understand both forms of ACLs. How are you imagining that would work? That's a problem that forces some hard choices, and without knowing how you're thinking of making those choices, I find it hard to judge whether the current system would help or hurt that case.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

In addition, I could imagine that we'll optionally support local NFSv4-style ACLs on specific file systems somewhen in the future. (Samba would benefit greatly from that for example; Solaris already can do that.) Not being able to tell the one kind of ACLs from the other in a world where we may have both sounds like a recipe for disaster to me.

Quoted 7 lines
Quoted 3 lines

I'm not following you. Currently nfsv4 and posix acls are being presented to userspace via different extended attributes, so you can tell one type from the other.

Except on NFSv4, we see POSIX ACLs represented as "system.nfs4_acl". I'd be much happier to see POSIX ACLs represented as POSIX ACLs, and NFSv4 ACLs and NFSv4 ACLs.

The stuff stored in "system.nfs4_acl" clearly represents an NFSv4 ACL, not a POSIX ACL, so I'm not sure what you mean here.

Quoted 3 lines

Both at the same time probably is not a good idea, agreed. Perhaps storing an NFSv4 ACL when NFSv4 ACL tools are used and POSIX ACLs when POSIX ACL tools are used might benting version 4 of the NFSACL side-band protocol

(I think there was some email-editing mistake here?)

If what you're proposing is storing *either* NFSv4 ACLs, *or* POSIX ACLs, but never both, that could work. It has the problem that if two users using the two different types of ACLs ever operate on the same file, then they won't be able to see each others ACLs, and setting one type will completely lose the other.

If we'd been willing to go for that kind of solution from the start, we could have tossed mode bits and chosen ACLs that were simpler than both NFSv4 and POSIX ACLs. But backwards compatibility seems too strong a requirement....

Quoted 4 lines

Implementing version 4 of the NFSACL side-band protocol and dumping the NFSv4 ACL <=> POSIX ACL mapping in nfsd would completely get rid of no-op (hopefully) rounds of mappings. That would mean less complexity in the kernel (we already have version 3 NFSACL, and version 4 is essentially the same).

I'm not as familiar with the NFSACL protocol as I should be. It can't accept NFSv4 filehandles, though, can it?

--b.

#4 lore raw
In reply to Andreas Gruenbacher
Quoted 1 line · Andreas Gruenbacher

The result were errors when copying files with cp -p (which preserves xattrs).

Is that true of "cp" in general, or only SuSE "cp"? We get problems with Lustre when root is copying files, because SuSE cp is incorrectly trying to copy all of the EAs by default. If I have an EA that is specific to a file I don't necessarily want it to be copied.

It would probably be preferrable that cp only copy EAs that it understands.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#5 lore raw
Quoted 5 lines · Andreas Dilger
Quoted 2 lines · Andreas Gruenbacher

The result were errors when copying files with cp -p (which preserves xattrs).

Is that true of "cp" in general, or only SuSE "cp"?

It's our cp only right now.

Quoted 4 lines

We get problems with
Lustre when root is copying files, because SuSE cp is incorrectly trying to copy all of the EAs by default. If I have an EA that is specific to a file I don't necessarily want it to be copied.

There's a blacklist in libattr which we could add Lustre attributes to. A blacklist somewhere in /etc would be even better though I guess.

Quoted 1 line

It would probably be preferrable that cp only copy EAs that it understands.

What to do with unknown attributes? Warn? Err out?

Andreas

#6 lore raw
Quoted 7 lines · Andreas Gruenbacher
Quoted 5 lines · Andreas Dilger
Quoted 2 lines · Andreas Gruenbacher

The result were errors when copying files with cp -p (which preserves xattrs).

Is that true of "cp" in general, or only SuSE "cp"?

Quoted 8 lines

It's our cp only right now.

Quoted 4 lines

We get problems with
Lustre when root is copying files, because SuSE cp is incorrectly trying to copy all of the EAs by default. If I have an EA that is specific to a file I don't necessarily want it to be copied.

There's a blacklist in libattr which we could add Lustre attributes to.

I figured as much after digging through the SLES-patched cp, I just hadn't gotten as far as grabbing the libattr source and seeing what that was. Is this blacklist specific to the SLES libattr, or is this in the released version?

Quoted 1 line

A blacklist somewhere in /etc would be even better though I guess.

Definitely, yes. Given that libselinx gropes around in a several files already, adding one more won't hurt. Something like /etc/libattr.conf?

Quoted 3 lines
Quoted 1 line

It would probably be preferrable that cp only copy EAs that it understands.

What to do with unknown attributes? Warn? Err out?

Warn once, maybe with pointer to /etc/libattr.conf for adding to include or exclude list. Given that copying file attributes isn't guaranteed in any case, we can probably safely live without copying everything.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

#7 lore raw
In reply to Andreas Gruenbacher
Quoted 2 lines · Andreas Gruenbacher

In addition, I could imagine that we'll optionally support local NFSv4-style ACLs on specific file systems somewhen in the future.

Most likely not. Let's not spread the NFSv4 ACL cancer more than absolutely nessecary.

Quoted 8 lines

* Don't map between POSIX ACLs and NFSv4 ACLs at all, and use a different protocol for POSIX ACLs and for NFSv4 ACLs instead (e.g., version 4 of the NFSACL protocol extension). The protocol extension is not currently implemented in Solaris because current Solaris supports NFSv4 ACLs natively instead of POSIX ACLs, and so they don't seem to have as strong a motivation to keep POSIX ACLs working well as we currently do. The NFSACL protocol has the huge advantage of supporting POSIX ACLs pretty well, and so a lot of the overhead and complexity of doing the NFSv4 mapping would go away though.

We definitly should support NFSACL for Linux to Linux deployments to avoid all that mess, yes.

Quoted 7 lines

* Map back from NFSv4 ACLs to POSIX ACLs on the client side, so that POSIX ACLs on the server will appear as POSIX ACLs on the client. This would take away all the mess we currently have in user-space and limit the mapping problems to the NFSv4 protocol, where it really occurs. We could define an additional NFsv4 attribute that the server could use to declare if it supports POSIX ACLs only, so that the client won't have to try to convert from NFSv4 to POSIX ACLs for servers that have native NFSv4 ACLs.

That's probably useful if we want to support non-Linux NFSv4 servers.