Skip to contentSkip to conversation

CAP(abilities) and NFS mounted storage

DISCUSSION Sander Smeenk 3 participants 7 messages Oct 13, 2015

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

I'm struggling with a permission issue involving NFS-mounted storage and a certain set of capabilities set by cap_set_flags(). The behaviour differs for local storage versus NFS mounted storage.

I have this structure on local storage (spinning disks):

| # namei -l /opt/home/sites/t/test/dir/structure/.htaccess
| f: /opt/home/sites/t/test/dir/structure/.htaccess
| drwxr-xr-x root      root                 /
| drwxr-xr-x root      root                 opt
| drwxr-x--x root      root                 home
| drwx--x--x root      root                 sites
| drwx--x--x root      root                 t
| drwx------ http-test http-linux_http-test test
| drwx--x--x http-test http-linux_http-test dir
| drwx------ http-test http-linux_http-test structure
| -rw------- http-test http-linux_http-test .htaccess

And this same structure on NFS-mounted storage:

| # namei -l /mnt/home/sites/t/test/dir/structure/.htaccess
| f: /mnt/home/sites/t/test/dir/structure/.htaccess
| drwxr-xr-x root      root                 /
| drwxr-xr-x root      root                 mnt
| drwxr-x--x root      root                 home
| drwx--x--x root      root                 sites
| drwx--x--x root      root                 t
| drwx------ http-test http-linux_http-test test
| drwx--x--x http-test http-linux_http-test dir
| drwx------ http-test http-linux_http-test structure
| -rw------- http-test http-linux_http-test .htaccess

The NFS server is a NetApp filer (-sec=sys,rw=clientip,root=clientip). I tried this with a Linux server too (rw,no_root_squash,no_subtree_check). The client is always a Linux machine (rw,vers=3,tcp,bg).

I made a little C program to illustrate the issue. It drops privileges to www-data and tries to access the file specified with a certain set of capabilties[*].

This works for local storage, fails on NFS:

LOCAL:

| # ./capset /opt/home/sites/t/test/dir/structure/.htaccess
| euid:33 uid:33 egid:33 gid:33
| Process capabilities: = cap_dac_read_search,cap_setgid,cap_setuid,cap_sys_nice+ep;
| Access: success!

NFS:

| # ./capset /mnt/home/sites/t/test/dir/structure/.htaccess
| euid:33 uid:33 egid:33 gid:33
| Process capabilities: = cap_dac_read_search,cap_setgid,cap_setuid,cap_sys_nice+ep;
| Access: error (13): Permission denied

The source for capset can be seen pasted at https://8n1.org/10831/12f0
Lines >=42

I've experimented with different capabilties, but CAP_DAC_OVERRIDE is not enough. I'd very much like to hear if it is possible for this to work on NFS like it does on local storage.

Any ideas?

Thanks in advance.
-Sndr.

[*] This issue popped up since Apache module 'mpm_itk' started using libcap capabilties to further enhance the security. The capabilties set was taken from mpm_itk source to 'prove the point' w/o the entire Apache setup.

Signature · 3 lines
| I wish i was a glow worm, a glow worm's never glum.
| How can you be unhappy when the sun shines out your bum!
| 4096R/20CC6CD2 - 6D40 1A20 B9AA 87D4 84C7  FBD6 F3A9 9442 20CC 6CD2
#1 lore raw
Quoted 54 lines · Sander Smeenk

Hi,

I'm struggling with a permission issue involving NFS-mounted storage and a certain set of capabilities set by cap_set_flags(). The behaviour differs for local storage versus NFS mounted storage.

I have this structure on local storage (spinning disks):

| # namei -l /opt/home/sites/t/test/dir/structure/.htaccess
| f: /opt/home/sites/t/test/dir/structure/.htaccess
| drwxr-xr-x root      root                 /
| drwxr-xr-x root      root                 opt
| drwxr-x--x root      root                 home
| drwx--x--x root      root                 sites
| drwx--x--x root      root                 t
| drwx------ http-test http-linux_http-test test
| drwx--x--x http-test http-linux_http-test dir
| drwx------ http-test http-linux_http-test structure
| -rw------- http-test http-linux_http-test .htaccess

And this same structure on NFS-mounted storage:

| # namei -l /mnt/home/sites/t/test/dir/structure/.htaccess
| f: /mnt/home/sites/t/test/dir/structure/.htaccess
| drwxr-xr-x root      root                 /
| drwxr-xr-x root      root                 mnt
| drwxr-x--x root      root                 home
| drwx--x--x root      root                 sites
| drwx--x--x root      root                 t
| drwx------ http-test http-linux_http-test test
| drwx--x--x http-test http-linux_http-test dir
| drwx------ http-test http-linux_http-test structure
| -rw------- http-test http-linux_http-test .htaccess

The NFS server is a NetApp filer (-sec=sys,rw=clientip,root=clientip). I tried this with a Linux server too (rw,no_root_squash,no_subtree_check). The client is always a Linux machine (rw,vers=3,tcp,bg).

I made a little C program to illustrate the issue. It drops privileges to www-data and tries to access the file specified with a certain set of capabilties[*].

This works for local storage, fails on NFS:

LOCAL:

| # ./capset /opt/home/sites/t/test/dir/structure/.htaccess
| euid:33 uid:33 egid:33 gid:33
| Process capabilities: = cap_dac_read_search,cap_setgid,cap_setuid,cap_sys_nice+ep;
| Access: success!

NFS:

| # ./capset /mnt/home/sites/t/test/dir/structure/.htaccess
| euid:33 uid:33 egid:33 gid:33
| Process capabilities: = cap_dac_read_search,cap_setgid,cap_setuid,cap_sys_nice+ep;
| Access: error (13): Permission denied
Quoted 6 lines

The source for capset can be seen pasted at https://8n1.org/10831/12f0
Lines >=42

I've experimented with different capabilties, but CAP_DAC_OVERRIDE is not enough. I'd very much like to hear if it is possible for this to work on NFS like it does on local storage.

This will not work on NFS. The server, which enforces permissions, has no way to know what capabilities your process has on the client.

Cheers
Trond

#2 lore raw
Quoted 5 lines · Trond Myklebust
Quoted 3 lines

I've experimented with different capabilties, but CAP_DAC_OVERRIDE is not enough. I'd very much like to hear if it is possible for this to work on NFS like it does on local storage.

This will not work on NFS. The server, which enforces permissions, has no way to know what capabilities your process has on the client.

Thanks. I feared this answer. But i understand that the NFS-server cant know if the process on the NFS-client has CAP_DAC_READ_SEARCH capabilities set.

Would setfsuid() help anything in this case? Or is it just a big no-go?

-Sndr.

Signature · 2 lines
| Daylight savings time - why are they saving it and where do they keep it?
| 4096R/20CC6CD2 - 6D40 1A20 B9AA 87D4 84C7  FBD6 F3A9 9442 20CC 6CD2
#3 lore raw
Quoted 5 lines · Sander Smeenk
Quoted 3 lines · Trond Myklebust
Quoted 3 lines

I've experimented with different capabilties, but CAP_DAC_OVERRIDE is not enough. I'd very much like to hear if it is possible for this to work on NFS like it does on local storage.

Quoted 9 lines
Quoted 2 lines

This will not work on NFS. The server, which enforces permissions, has no way to know what capabilities your process has on the client.

Thanks. I feared this answer. But i understand that the NFS-server cant know if the process on the NFS-client has CAP_DAC_READ_SEARCH capabilities set.

Would setfsuid() help anything in this case? Or is it just a big no-go?

Are you looking for something like labeled NFS that supports capabilities? I think Redhat7 has SElinux labeled NFS support.

Quoted 8 lines

-Sndr.
--
| Daylight savings time - why are they saving it and where do they keep it? | 4096R/20CC6CD2 - 6D40 1A20 B9AA 87D4 84C7 FBD6 F3A9 9442 20CC 6CD2 --
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
Quoted 6 lines · Olga Kornievskaia
Quoted 5 lines · Sander Smeenk
Quoted 3 lines · Trond Myklebust
Quoted 3 lines

I've experimented with different capabilties, but CAP_DAC_OVERRIDE is not enough. I'd very much like to hear if it is possible for this to work on NFS like it does on local storage.

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

This will not work on NFS. The server, which enforces permissions, has no way to know what capabilities your process has on the client.

Thanks. I feared this answer. But i understand that the NFS-server cant know if the process on the NFS-client has CAP_DAC_READ_SEARCH capabilities set.

Would setfsuid() help anything in this case? Or is it just a big no-go?

setfsuid() would allow you to set a user up with root privileges on the fs. That's better than giving overall root privileges, but is still a risk, since a user could use it to overwrite /etc/passwd etc.

Quoted 3 lines

Are you looking for something like labeled NFS that supports capabilities? I think Redhat7 has SElinux labeled NFS support.

The labeled NFS implementation is client enforced and confers no extra privileges on the server. Plus (please correct me if I'm wrong), I believe NetApp has yet to announce support for labeled NFS in OnTAP.

#5 lore raw
Quoted 5 lines · Trond Myklebust
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

I've experimented with different capabilties, but CAP_DAC_OVERRIDE is not enough. I'd very much like to hear if it is possible for this to work on NFS like it does on local storage.

This will not work on NFS. The server, which enforces permissions, has no way to know what capabilities your process has on the client.

Quoted 8 lines
Quoted 4 lines
Quoted 4 lines

Thanks. I feared this answer. But i understand that the NFS-server cant know if the process on the NFS-client has CAP_DAC_READ_SEARCH capabilities set.
Would setfsuid() help anything in this case? Or is it just a big no-go?

setfsuid() would allow you to set a user up with root privileges on the fs. That's better than giving overall root privileges, but is still a risk, since a user could use it to overwrite /etc/passwd etc.

I'm not going to run anything i'm trying in a production environment yet. ;)

But, should setfsuid() work with NFS-mounted filesystems too? 'Cause i tried, but it didn't seem to change much (w/ caps set). Might be i need more capabilities for that to work. Hmm.

Quoted 6 lines
Quoted 2 lines

Are you looking for something like labeled NFS that supports capabilities? I think Redhat7 has SElinux labeled NFS support.

The labeled NFS implementation is client enforced and confers no extra privileges on the server. Plus (please correct me if I'm wrong), I believe NetApp has yet to announce support for labeled NFS in OnTAP.

Yes. This is not really an option. ;)

Regards,
-Sndr.

Signature · 2 lines
| Spend some time trying hard not to think about giraffes.
| 4096R/20CC6CD2 - 6D40 1A20 B9AA 87D4 84C7  FBD6 F3A9 9442 20CC 6CD2
#6 lore raw
In reply to Olga Kornievskaia
Quoted 5 lines · Olga Kornievskaia
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

I've experimented with different capabilties, but CAP_DAC_OVERRIDE is not enough. I'd very much like to hear if it is possible for this to work on NFS like it does on local storage.

This will not work on NFS. The server, which enforces permissions, has no way to know what capabilities your process has on the client.

Quoted 7 lines
Quoted 4 lines

Thanks. I feared this answer. But i understand that the NFS-server cant know if the process on the NFS-client has CAP_DAC_READ_SEARCH capabilities set.
Would setfsuid() help anything in this case? Or is it just a big no-go?

Are you looking for something like labeled NFS that supports capabilities? I think Redhat7 has SElinux labeled NFS support.

SELinux might be a bit too much for our situation. ;)

I'm looking for an answer to who exactly controls filesystem access permission checks when dealing with files on an NFS mount, and perhaps an answer to why these permission checks seem to differ for files on NFS mounts and files on locally attached storage w/ strict capabilities set.

Regards,
-Sndr.

Signature · 2 lines
| Have you ever imagined a world with no hypothetical situations?
| 4096R/20CC6CD2 - 6D40 1A20 B9AA 87D4 84C7  FBD6 F3A9 9442 20CC 6CD2