Skip to contentSkip to conversation

nfs vs xfstests 193

PATCH In review Christoph Hellwig 3 participants 12 messages Nov 6, 2013 – Dec 12, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I've noticed that xfstests 193 fails when run over NFS talking to an XFS-based Linux server. The test checks that we behave correctly vs Posix 1003.1 for the various operations that end up in ->setattr.

Without the no_root_squash export flag we're not even able to run something resembling the test as we get permission problems all through it, see the first attachment for details.

With the root_squash export op we fail to clear the setuid/setgid bits in various truncate and chown subtests, see the second attachment for details.

root/xfstests/results//generic/193.out.bad renamed from generic/193.out +7 -7
@@ -35,26 +35,26 @@
 after:  -rw-r-Sr--
 with user exec perm
 before: -rwsr-Sr--
-after:  -rwxr-Sr--
+after:  -rwsr-Sr--
 with group exec perm
 before: -rwSr-sr--
-after:  -rw-r-xr--
+after:  -rwSr-sr--
 with user+group exec perm
 before: -rwsr-sr--
-after:  -rwxr-xr--
+after:  -rwsr-sr--
 check that suid/sgid bits are cleared after successful truncate...
 with no exec perm
 before: -rwSr-Sr--
-after:  -rw-r-Sr--
+after:  -rwSr-Sr--
 with user exec perm
 before: -rwsr-Sr--
-after:  -rwxr-Sr--
+after:  -rwsr-Sr--
 with group exec perm
 before: -rwSr-sr--
-after:  -rw-r-xr--
+after:  -rwSr-sr--
 with user+group exec perm
 before: -rwsr-sr--
-after:  -rwxr-xr--
+after:  -rwsr-sr--
 
 testing ATTR_*TIMES_SET
 
root/xfstests/results//generic/193.out.bad renamed from generic/193.out +80 -25

143 lines not on this page · raw message

#1 lore raw
Quoted 11 lines · Christoph Hellwig

I've noticed that xfstests 193 fails when run over NFS talking to an XFS-based Linux server. The test checks that we behave correctly vs Posix 1003.1 for the various operations that end up in ->setattr.

Without the no_root_squash export flag we're not even able to run something resembling the test as we get permission problems all through it, see the first attachment for details.

With the root_squash export op we fail to clear the setuid/setgid bits in various truncate and chown subtests, see the second attachment for details.

Hi!

I've come across the same issue. But NFS server is backed by ext4 file system in my environment.

The test case quotes POSIX:

"If the specified file is a regular file, one or more of the S_IXUSR, S_IXGRP, or S_IXOTH bits of the
file mode are set, and the process has appropriate privileges, it is implementation-defined whether the set-user-ID and set-group-ID bits are altered."

So the difference that what we have now:

between nfs:
~# touch file; chmod ug+s file; chmod u+x file; ls -l file; chown root file; ls -l file; rm -f file -rwsr-Sr-- 1 root root 0 Dec 6 04:49 file
-rwsr-Sr-- 1 root root 0 Dec 6 04:49 file

and ext3, ext4, xfs, btrfs:
~# touch file; chmod ug+s file; chmod u+x file; ls -l file; chown root file; ls -l file; rm -f file -rwsr-Sr-- 1 root root 0 Dec 6 04:49 file
-rwxr-Sr-- 1 root root 0 Dec 6 04:49 file

is not a violation of this POSIX statement. But It's just an "implementation-defined" behaviour.

I suppose that the difference raises because of this part of code in fs/nfsd/vfs.c:

        /* Revoke setuid/setgid on chown */
        if (!S_ISDIR(inode->i_mode) &&
            (((iap->ia_valid & ATTR_UID) && !uid_eq(iap->ia_uid, 
inode->i_uid)) ||
             ((iap->ia_valid & ATTR_GID) && !gid_eq(iap->ia_gid, 
inode->i_gid)))) {
                iap->ia_valid |= ATTR_KILL_PRIV;
                if (iap->ia_valid & ATTR_MODE) {
                        /* we're setting mode too, just clear the s*id 
bits */
                        iap->ia_mode &= ~S_ISUID;
                        if (iap->ia_mode & S_IXGRP)
                                iap->ia_mode &= ~S_ISGID;
                } else {
                        /* set ATTR_KILL_* bits and let VFS handle it */
                        iap->ia_valid |= (ATTR_KILL_SUID | ATTR_KILL_SGID);
                }
        }

uid_eq() and gid_eq() checkings allow removal of s*id bits only if the owner/group of the file is changed during chown().

I.e. on nfs:
~# touch file; chmod ug+s file; chmod u+x file; ls -l file; chown bin file; ls -l file; rm -f file -rwsr-Sr-- 1 root root 0 Dec 6 05:02 file
-rwxr-Sr-- 1 bin root 0 Dec 6 05:02 file

Is it acceptable to change NFS kernel server behaviour by removal of !uid_eq(iap->ia_uid, inode->i_uid) and !gid_eq(iap->ia_gid, inode->i_gid) from the condition above?

Just to make the behaviour more consistent between NFS and other "local" file systems as It was done by commit https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=0953e620de0538cbd081f1b45126f6098112a598

Thank you!

#3 lore raw
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Stanislav Kholmanskikh

Just to make the behaviour more consistent between NFS and other
"local" file systems as It was done by
commit https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=0953e620de0538cbd081f1b45126f6098112a598

Seems like we got others in line with XFS behavior.

But, not having tested the behavior, it looks like fs/open.c has a simlar !S_ISDIR() check. Where's that behavior implemented?

Quoted 1 line

I'd prefer to have NFS follow this as well.

Huh. Sachin, do you remember if there was any other motivation behind that patch?

--b.

#4 lore raw
Quoted 8 lines · bfields
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Stanislav Kholmanskikh

Just to make the behaviour more consistent between NFS and other
"local" file systems as It was done by
commit https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=0953e620de0538cbd081f1b45126f6098112a598

Seems like we got others in line with XFS behavior.

Quoted 7 lines

But, not having tested the behavior, it looks like fs/open.c has a simlar !S_ISDIR() check. Where's that behavior implemented?

Quoted 1 line

I'd prefer to have NFS follow this as well.

Huh. Sachin, do you remember if there was any other motivation behind that patch?

Never mind, I see, the complaint is about the case where the id's don't change, not about the directory case. So Sachin's 0953e620de0538cbd081f1b45126f6098112a598 doesn't actually have anything to do with this.

I'm fine with removing the id comparisons and changing the nfsd behavior to match local filesystems.

--b.

#5 lore raw
Quoted 17 lines · J. Bruce Fields
Quoted 8 lines · bfields
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Stanislav Kholmanskikh

Just to make the behaviour more consistent between NFS and other
"local" file systems as It was done by
commit https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=0953e620de0538cbd081f1b45126f6098112a598

Seems like we got others in line with XFS behavior.

Quoted 7 lines

But, not having tested the behavior, it looks like fs/open.c has a simlar !S_ISDIR() check. Where's that behavior implemented?

Quoted 1 line

I'd prefer to have NFS follow this as well.

Huh. Sachin, do you remember if there was any other motivation behind that patch?

Quoted 7 lines

Never mind, I see, the complaint is about the case where the id's don't change, not about the directory case. So Sachin's 0953e620de0538cbd081f1b45126f6098112a598 doesn't actually have anything to do with this.

I'm fine with removing the id comparisons and changing the nfsd behavior to match local filesystems.

Great.

I will try to produce a patch for this.

Quoted 8 lines

--b.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#6 lore raw

PATCH 1/1 nfsd: revoking of suid/sgid bits after chown() in a consistent way

In reply to J. Bruce Fields

There is an inconsistency in the handling of SUID/SGID file bits after chown() between NFS and other local file systems.

Local file systems (for example, ext3, ext4, xfs, btrfs) revoke SUID/SGID bits after chown() on a regular file even if the owner/group of the file has not been changed:

~# touch file; chmod ug+s file; chmod u+x file
~# ls -l file
-rwsr-Sr-- 1 root root 0 Dec 6 04:49 file
~# chown root file; ls -l file
-rwxr-Sr-- 1 root root 0 Dec 6 04:49 file

but NFS doesn't do that:

~# touch file; chmod ug+s file; chmod u+x file
~# ls -l file
-rwsr-Sr-- 1 root root 0 Dec 6 04:49 file
~# chown root file; ls -l file
-rwsr-Sr-- 1 root root 0 Dec 6 04:49 file

NFS does that only if the owner/group has been changed:

~# touch file; chmod ug+s file; chmod u+x file
~# ls -l file
-rwsr-Sr-- 1 root root 0 Dec 6 05:02 file
~# chown bin file; ls -l file
-rwxr-Sr-- 1 bin root 0 Dec 6 05:02 file

See: http://pubs.opengroup.org/onlinepubs/9699919799/functions/chown.html

"If the specified file is a regular file, one or more of the S_IXUSR, S_IXGRP, or S_IXOTH bits of the file mode are set, and the process has appropriate privileges, it is implementation-defined whether the set-user-ID and set-group-ID bits are altered."

So both variants are acceptable by POSIX.

This patch makes NFS to behave like local file systems.

---
 fs/nfsd/vfs.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
fs/nfsd/vfs.c +1 -2
index 72cb28e..8226991 100644
@@ -348,8 +348,7 @@ nfsd_sanitize_attrs(struct inode *inode, struct iattr *iap)
 
 	/* Revoke setuid/setgid on chown */
 	if (!S_ISDIR(inode->i_mode) &&
-	    (((iap->ia_valid & ATTR_UID) && !uid_eq(iap->ia_uid, inode->i_uid)) ||
-	     ((iap->ia_valid & ATTR_GID) && !gid_eq(iap->ia_gid, inode->i_gid)))) {
+	    ((iap->ia_valid & ATTR_UID) || (iap->ia_valid & ATTR_GID))) {
 		iap->ia_valid |= ATTR_KILL_PRIV;
 		if (iap->ia_valid & ATTR_MODE) {
 			/* we're setting mode too, just clear the s*id bits */
Signature · 1 line
1.7.1
#7 lore raw

Re: [PATCH] nfsd: revoking of suid/sgid bits after chown() in a consistent way

On 12/11/2013 02:16 PM, Stanislav Kholmanskikh wrote:
[cut off]

Quoted 3 lines

This patch makes NFS to behave like local file systems.

[cut off]

This patch allows to run generic/193 without any issues with NFSv3.

With NFSv4 generic/193 fails (but with the other issues, which existed even before the patch).

generic/193 expects that suid/sgid bits are cleared after the file truncation:

touch file
chown fsgqa:fsgqa file
chmod u+s file
echo 'xyz' > file
ls -l file
su fsgqa -c 'echo > file'
ls -l file

With ext4 (for example), we have expectable results:
-rwSr--r-- 1 fsgqa fsgqa 4 Dec 11 05:21 file -rw-r--r-- 1 fsgqa fsgqa 1 Dec 11 05:22 file

With NFSv3 as well:
-rwSr--r-- 1 fsgqa fsgqa 4 Dec 11 05:24 file
-rw-r--r-- 1 fsgqa fsgqa 1 Dec 11 05:25 file

But with NFSv4 the bits are not cleared:
-rwSr--r-- 1 fsgqa fsgqa 1 Dec 11 05:19 file
-rwSr--r-- 1 fsgqa fsgqa 1 Dec 11 05:21 file

'echo > file' issues:

open("file", O_WRONLY|O_CREAT|O_TRUNC, 0666)

Can it be because of design differences between NFSv3 and NFSv4?

Thank you.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#8 lore raw

Re: [PATCH] nfsd: revoking of suid/sgid bits after chown() in a consistent way

Quoted 37 lines · Stanislav Kholmanskikh

On 12/11/2013 02:16 PM, Stanislav Kholmanskikh wrote:
[cut off]

Quoted 3 lines

This patch makes NFS to behave like local file systems.

[cut off]

This patch allows to run generic/193 without any issues with NFSv3.

With NFSv4 generic/193 fails (but with the other issues, which existed even before the patch).

generic/193 expects that suid/sgid bits are cleared after the file truncation:

touch file
chown fsgqa:fsgqa file
chmod u+s file
echo 'xyz' > file
ls -l file
su fsgqa -c 'echo > file'
ls -l file

With ext4 (for example), we have expectable results:

(quoted diff) +0 -2
-rwSr--r-- 1 fsgqa fsgqa 4 Dec 11 05:21 file
-rw-r--r-- 1 fsgqa fsgqa 1 Dec 11 05:22 file

With NFSv3 as well:

(quoted diff) +0 -2
-rwSr--r-- 1 fsgqa fsgqa 4 Dec 11 05:24 file
-rw-r--r-- 1 fsgqa fsgqa 1 Dec 11 05:25 file

But with NFSv4 the bits are not cleared:

(quoted diff) +0 -2
-rwSr--r-- 1 fsgqa fsgqa 1 Dec 11 05:19 file
-rwSr--r-- 1 fsgqa fsgqa 1 Dec 11 05:21 file
Quoted 5 lines

'echo > file' issues:

open("file", O_WRONLY|O_CREAT|O_TRUNC, 0666)

Can it be because of design differences between NFSv3 and NFSv4?

In the v3 case I'd expect the open O_TRUNC to result in a SETATTR rpc, in the v4 case an OPEN rpc. Both result in a call to nfsd_setattr, though I only see nfsd_setattr turning off the SUID/SGID bits in the chown case. Are you sure it isn't the subsequent write that clears those bits?

But looks to me like nfsd_vfs_write (used in both v3 & v4 cases) clears suid & guid, so I still don't see it.

--b.

#9 lore raw

Re: [PATCH] nfsd: revoking of suid/sgid bits after chown() in a consistent way

Quoted 5 lines · J. Bruce Fields

In the v3 case I'd expect the open O_TRUNC to result in a SETATTR rpc, in the v4 case an OPEN rpc. Both result in a call to nfsd_setattr, though I only see nfsd_setattr turning off the SUID/SGID bits in the chown case. Are you sure it isn't the subsequent write that clears those bits?

We've traditionally cleared the suid bits for O_TRUNC for local filesystem, although this is more a convention than a real security need. It would still be good if NFSv4 would follow the general semantics.

#10 lore raw

Re: [PATCH] nfsd: revoking of suid/sgid bits after chown() in a consistent way

In reply to J. Bruce Fields
Quoted 42 lines · J. Bruce Fields
Quoted 41 lines · Stanislav Kholmanskikh

On 12/11/2013 02:16 PM, Stanislav Kholmanskikh wrote:
[cut off]

Quoted 3 lines

This patch makes NFS to behave like local file systems.

[cut off]

This patch allows to run generic/193 without any issues with NFSv3.

With NFSv4 generic/193 fails (but with the other issues, which existed even before the patch).

generic/193 expects that suid/sgid bits are cleared after the file truncation:

touch file
chown fsgqa:fsgqa file
chmod u+s file
echo 'xyz' > file
ls -l file
su fsgqa -c 'echo > file'
ls -l file

With ext4 (for example), we have expectable results:

(quoted diff) +0 -2
-rwSr--r-- 1 fsgqa fsgqa 4 Dec 11 05:21 file
-rw-r--r-- 1 fsgqa fsgqa 1 Dec 11 05:22 file

With NFSv3 as well:

(quoted diff) +0 -2
-rwSr--r-- 1 fsgqa fsgqa 4 Dec 11 05:24 file
-rw-r--r-- 1 fsgqa fsgqa 1 Dec 11 05:25 file

But with NFSv4 the bits are not cleared:

(quoted diff) +0 -2
-rwSr--r-- 1 fsgqa fsgqa 1 Dec 11 05:19 file
-rwSr--r-- 1 fsgqa fsgqa 1 Dec 11 05:21 file

'echo > file' issues:

open("file", O_WRONLY|O_CREAT|O_TRUNC, 0666)

Quoted 7 lines
Quoted 1 line

Can it be because of design differences between NFSv3 and NFSv4?

In the v3 case I'd expect the open O_TRUNC to result in a SETATTR rpc, in the v4 case an OPEN rpc. Both result in a call to nfsd_setattr, though I only see nfsd_setattr turning off the SUID/SGID bits in the chown case. Are you sure it isn't the subsequent write that clears those bits?

Actually, in the above test script I occasionally swapped positions of "echo 'xyz' > file" and "chmod u+s file". Of course, chmod should be after the writing. Sorry.

But here we are:

rm -f file; touch file
chown fsgqa:fsgqa file
echo 'xyz' > file
chmod u+s file
ls -l file
su fsgqa -c 'echo -n > file' # open(O_TRUNC), close() ls -l file
su fsgqa -c 'echo > file' # open(O_TRUNC), write("\n"), close()
ls -l file

With NFSv3 suid is cleared after write:
-rwSr--r-- 1 fsgqa fsgqa 4 Dec 12 05:24 file
-rwSr--r-- 1 fsgqa fsgqa 0 Dec 12 05:24 file
-rw-r--r-- 1 fsgqa fsgqa 1 Dec 12 05:25 file

With NFSv4 suid is not cleared after write("\n"):
-rwSr--r-- 1 fsgqa fsgqa 4 Dec 12 05:26 file
-rwSr--r-- 1 fsgqa fsgqa 0 Dec 12 05:26 file
-rwSr--r-- 1 fsgqa fsgqa 1 Dec 12 05:27 file

but if we issue "su fsgqa -c 'echo -n b > file'", we will clear it:
-rw-r--r-- 1 fsgqa fsgqa 1 Dec 12 05:28 file

So if "file" is a file on NFSv4:
-rwSr--r-- 1 fsgqa fsgqa 4 Dec 12 05:26 file
and we do:

fd = open(file, O_WRONLY);

then write(fd, "\n", 1) will not clear suid bit, but write(fd, "b", 1) will do.

With ext4 suid is cleared after open(O_TRUNC):
-rwSr--r-- 1 fsgqa fsgqa 4 Dec 12 05:29 file
-rw-r--r-- 1 fsgqa fsgqa 0 Dec 12 05:29 file
-rw-r--r-- 1 fsgqa fsgqa 1 Dec 12 05:30 file

Execution of (via 'su fsgqa -c ...'):
fd = open("file", O_WRONLY);
close(fd);
if "file" is on ext4 file system and has suid bit on will not clear suid bit.

Execution of (via 'su fsgqa -c ...'):
fd = open("file", O_WRONLY);
write(fd, "a", 1);
close(fd);
if "file" is on ext4 file system and has suid bit on will clear suid bit.

To conclude:
1. With NFS suid is not cleared after open(O_TRUNC)
This may be solved by addition of ATTR_SIZE handling in nfsd_setattr (i.e nfsd_sanitize_attrs). Right?

2. NFSv4 treats "\n" on write() specially.
No ideas by the moment.

Quoted 6 lines

But looks to me like nfsd_vfs_write (used in both v3 & v4 cases) clears suid & guid, so I still don't see it.

--b.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#11 lore raw

Re: [PATCH] nfsd: revoking of suid/sgid bits after chown() in a consistent way

In reply to Stanislav Kholmanskikh

Thanks, applying for 3.14.--b.

Quoted 63 lines · Stanislav Kholmanskikh

There is an inconsistency in the handling of SUID/SGID file bits after chown() between NFS and other local file systems.

Local file systems (for example, ext3, ext4, xfs, btrfs) revoke SUID/SGID bits after chown() on a regular file even if the owner/group of the file has not been changed:

~# touch file; chmod ug+s file; chmod u+x file
~# ls -l file
-rwsr-Sr-- 1 root root 0 Dec 6 04:49 file
~# chown root file; ls -l file
-rwxr-Sr-- 1 root root 0 Dec 6 04:49 file

but NFS doesn't do that:

~# touch file; chmod ug+s file; chmod u+x file
~# ls -l file
-rwsr-Sr-- 1 root root 0 Dec 6 04:49 file
~# chown root file; ls -l file
-rwsr-Sr-- 1 root root 0 Dec 6 04:49 file

NFS does that only if the owner/group has been changed:

~# touch file; chmod ug+s file; chmod u+x file
~# ls -l file
-rwsr-Sr-- 1 root root 0 Dec 6 05:02 file
~# chown bin file; ls -l file
-rwxr-Sr-- 1 bin root 0 Dec 6 05:02 file

See: http://pubs.opengroup.org/onlinepubs/9699919799/functions/chown.html

"If the specified file is a regular file, one or more of the S_IXUSR, S_IXGRP, or S_IXOTH bits of the file mode are set, and the process has appropriate privileges, it is implementation-defined whether the set-user-ID and set-group-ID bits are altered."

So both variants are acceptable by POSIX.

This patch makes NFS to behave like local file systems.

---
 fs/nfsd/vfs.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)
fs/nfsd/vfs.c +1 -2
index 72cb28e..8226991 100644
@@ -348,8 +348,7 @@ nfsd_sanitize_attrs(struct inode *inode, struct iattr *iap)
 
 	/* Revoke setuid/setgid on chown */
 	if (!S_ISDIR(inode->i_mode) &&
-	    (((iap->ia_valid & ATTR_UID) && !uid_eq(iap->ia_uid, inode->i_uid)) ||
-	     ((iap->ia_valid & ATTR_GID) && !gid_eq(iap->ia_gid, inode->i_gid)))) {
+	    ((iap->ia_valid & ATTR_UID) || (iap->ia_valid & ATTR_GID))) {
 		iap->ia_valid |= ATTR_KILL_PRIV;
 		if (iap->ia_valid & ATTR_MODE) {
 			/* we're setting mode too, just clear the s*id bits */
Signature · 1 line
1.7.1