fuse: Implement FUSE_HANDLE_KILLPRIV_V2 and enable SB_NOSEC
Subsystems (2)
v3 cover fuse: Implement FUSE_HANDLE_KILLPRIV_V2 and enable SB_NOSEC
Hi All,
Please find attached V3 of the patches to enable SB_NOSEC for fuse. I posted V1 and V2 here.
v2:
https://lore.kernel.org/linux-fsdevel/[email protected]/
v1:
https://lore.kernel.org/linux-fsdevel/[email protected]/
Changes since v2:
- Based on Miklos's feedback, dropped a patch where we send ATTR_MODE as that's racy. To help the case of writeback_cache with killpriv_v2, I fallback to a synchronous WRITE if suid/sgid is set on file.
I have generated these patches on top of.
https://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git/log/?h=for-nex
+t
I have taken care of feedback from last round. For the case of random write peformance has jumped from 50MB/s to 250MB/s. So I am really looking forward to these changes so that fuse/virtiofs performance can be improved for direct random writes.
Thanks
Vivek
Vivek Goyal (6):
fuse: Introduce the notion of FUSE_HANDLE_KILLPRIV_V2
fuse: Set FUSE_WRITE_KILL_PRIV in cached write path
fuse: setattr should set FATTR_KILL_PRIV upon size change
fuse: Don't send ATTR_MODE to kill suid/sgid for handle_killpriv_v2
fuse: Add a flag FUSE_OPEN_KILL_PRIV for open() request
fuse: Support SB_NOSEC flag to improve direct write performance
fs/fuse/dir.c | 4 +++- fs/fuse/file.c | 16 +++++++++++++++- fs/fuse/fuse_i.h | 6 ++++++ fs/fuse/inode.c | 17 ++++++++++++++++- include/uapi/linux/fuse.h | 18 +++++++++++++++++- 5 files changed, 57 insertions(+), 4 deletions(-)
Signature · 1 line
2.25.4
PATCH v3 1/6 fuse: Introduce the notion of FUSE_HANDLE_KILLPRIV_V2
We already have FUSE_HANDLE_KILLPRIV flag that says that file server will remove suid/sgid/caps on truncate/chown/write. But that's little different from what Linux VFS implements.
To be consistent with Linux VFS behavior what we want is.
- caps are always cleared on chown/write/truncate
- suid is always cleared on chown, while for truncate/write it is cleared only if caller does not have CAP_FSETID.
- sgid is always cleared on chown, while for truncate/write it is cleared only if caller does not have CAP_FSETID as well as file has group execute permission.
As previous flag did not provide above semantics. Implement a V2 of the protocol with above said constraints.
Server does not know if caller has CAP_FSETID or not. So for the case of write()/truncate(), client will send information in special flag to indicate whether to kill priviliges or not. These changes are in subsequent patches.
FUSE_HANDLE_KILLPRIV_V2 relies on WRITE being sent to server to clear suid/sgid/security.capability. But with ->writeback_cache, WRITES are cached in guest. So it is not recommended to use FUSE_HANDLE_KILLPRIV_V2 and writeback_cache together. Though it probably might be good enough for lot of use cases.
--- fs/fuse/fuse_i.h | 6 ++++++ fs/fuse/inode.c | 5 ++++- include/uapi/linux/fuse.h | 7 +++++++ 3 files changed, 17 insertions(+), 1 deletion(-)
fs/fuse/fuse_i.h +6 -0
@@ -631,6 +631,12 @@ struct fuse_conn {/* show legacy mount options */ unsigned int legacy_opts_show:1; +/** fs kills suid/sgid/cap on write/chown/trunc. suid is +killed on write/trunc only if caller did not have CAP_FSETID. +sgid is killed on write/truncate only if caller did not have +CAP_FSETID as well as file has group execute permission. */ + unsigned handle_killpriv_v2:1; +/* * The following bitfields are only for optimization purposes * and hence races in setting them will not cause malfunction
fs/fuse/inode.c +4 -1
@@ -993,6 +993,8 @@ static void process_init_reply(struct fuse_conn *fc, struct fuse_args *args, !fuse_dax_check_alignment(fc, arg->map_alignment)) { ok = false; } +if (arg->flags &FUSE_HANDLE_KILLPRIV_V2 ) + fc->handle_killpriv_v2 =1 ; }else { ra_pages = fc->max_read /PAGE_SIZE ; fc->no_lock =1 ;
@@ -1035,7 +1037,8 @@ void fuse_send_init(struct fuse_conn *fc)FUSE_WRITEBACK_CACHE |FUSE_NO_OPEN_SUPPORT |FUSE_PARALLEL_DIROPS |FUSE_HANDLE_KILLPRIV |FUSE_POSIX_ACL |FUSE_ABORT_ERROR |FUSE_MAX_PAGES |FUSE_CACHE_SYMLINKS | -FUSE_NO_OPENDIR_SUPPORT |FUSE_EXPLICIT_INVAL_DATA ; +FUSE_NO_OPENDIR_SUPPORT |FUSE_EXPLICIT_INVAL_DATA | +FUSE_HANDLE_KILLPRIV_V2 ;#ifdef CONFIG_FUSE_DAX if (fc->dax) ia->in.flags |=FUSE_MAP_ALIGNMENT ;
include/uapi/linux/fuse.h +7 -0
@@ -172,6 +172,7 @@* - add FUSE_WRITE_KILL_PRIV flag * - add FUSE_SETUPMAPPING and FUSE_REMOVEMAPPING * - add map_alignment to fuse_init_out, add FUSE_MAP_ALIGNMENT flag +* - add FUSE_HANDLE_KILLPRIV_V2 */ #ifndef _LINUX_FUSE_H
@@ -316,6 +317,11 @@ struct fuse_file_lock {* FUSE_MAP_ALIGNMENT: init_out.map_alignment contains log2(byte alignment) for * foffset and moffset fields in struct * fuse_setupmapping_out and fuse_removemapping_one. +* FUSE_HANDLE_KILLPRIV_V2: fs kills suid/sgid/cap on write/chown/trunc. +* Upon write/truncate suid/sgid is only killed if caller +* does not have CAP_FSETID. Additionally upon +* write/truncate sgid is killed only if file has group +* execute permission. (Same as Linux VFS behavior). */ #define FUSE_ASYNC_READ (1 << 0)#define FUSE_POSIX_LOCKS (1 << 1)
@@ -344,6 +350,7 @@ struct fuse_file_lock {#define FUSE_NO_OPENDIR_SUPPORT (1 << 24)#define FUSE_EXPLICIT_INVAL_DATA (1 << 25)#define FUSE_MAP_ALIGNMENT (1 << 26) +#define FUSE_HANDLE_KILLPRIV_V2 (1 << 27)/** * CUSE INIT request/reply flags
Signature · 1 line
2.25.4
PATCH v3 2/6 fuse: Set FUSE_WRITE_KILL_PRIV in cached write path
In reply to Vivek GoyalWith HANDLE_KILLPRIV_V2, server will need to kill suid/sgid if caller does not have CAP_FSETID. We already have a flag FUSE_WRITE_KILL_PRIV in WRITE request and we already set it in direct I/O path.
To make it work in cached write path also, start setting FUSE_WRITE_KILL_PRIV in this path too.
Set it only if fc->handle_killpriv_v2 is set. Otherwise client is responsible for kill suid/sgid.
In case of direct I/O we set FUSE_WRITE_KILL_PRIV unconditionally because we do't call file_remove_privs() in that path (with cache=none option).
--- fs/fuse/file.c | 2 ++ 1 file changed, 2 insertions(+)
fs/fuse/file.c +2 -0
@@ -1095,6 +1095,8 @@ static ssize_t fuse_send_write_pages(struct fuse_io_args *ia,fuse_write_args_fill (ia, ff, pos, count); ia->write.in.flags =fuse_write_flags (iocb); +if (fc->handle_killpriv_v2 && !capable (CAP_FSETID )) + ia->write.in.write_flags |=FUSE_WRITE_KILL_PRIV ; err =fuse_simple_request (fc, &ap->args);if (!err && ia->write.out.size > count)
Signature · 1 line
2.25.4
PATCH v3 3/6 fuse: setattr should set FATTR_KILL_PRIV upon size change
In reply to Vivek GoyalIf fc->handle_killpriv_v2 is enabled, we expect file server to clear suid/sgid/security.capbility upon chown/truncate/write as appropriate.
Upon truncate (ATTR_SIZE), suid/sgid is cleared only if caller does not have CAP_FSETID. File server does not know whether caller has CAP_FSETID or not. Hence set FATTR_KILL_PRIV upon truncate to let file server know that caller does not have CAP_FSETID and it should kill suid/sgid as appropriate.
We don't have to send this information for chown (ATTR_UID/ATTR_GID) as that always clears suid/sgid irrespective of capabilities of calling process.
--- fs/fuse/dir.c | 2 ++ include/uapi/linux/fuse.h | 1 + 2 files changed, 3 insertions(+)
fs/fuse/dir.c +2 -0
@@ -1575,6 +1575,8 @@ int fuse_do_setattr(struct dentry *dentry, struct iattr *attr,/* For mandatory locking in truncate */ inarg.valid |=FATTR_LOCKOWNER ; inarg.lock_owner =fuse_lock_owner_id (fc, current->files); +if (fc->handle_killpriv_v2 && !capable (CAP_FSETID )) + inarg.valid |=FATTR_KILL_PRIV ; }fuse_setattr_fill (fc, &args, inode, &inarg, &outarg); err =fuse_simple_request (fc, &args);
include/uapi/linux/fuse.h +1 -0
@@ -269,6 +269,7 @@ struct fuse_file_lock {#define FATTR_MTIME_NOW (1 << 8)#define FATTR_LOCKOWNER (1 << 9)#define FATTR_CTIME (1 << 10) +#define FATTR_KILL_PRIV (1 << 14)/* Matches ATTR_KILL_PRIV */ /** * Flags returned by the OPEN request
Signature · 1 line
2.25.4
Miklos Szeredi
maintainerRe: [PATCH v3 3/6] fuse: setattr should set FATTR_KILL_PRIV upon size change
Quoted 10 lines · Vivek Goyal
If
fc->handle_killpriv_v2is enabled, we expect file server to clear suid/sgid/security.capbility upon chown/truncate/write as appropriate.Upon truncate (
ATTR_SIZE), suid/sgid is cleared only if caller does not haveCAP_FSETID. File server does not know whether caller hasCAP_FSETIDor not. Hence setFATTR_KILL_PRIVupon truncate to let file server know that caller does not haveCAP_FSETIDand it should kill suid/sgid as appropriate.
Quoted 3 lines
We don't have to send this information for chown (ATTR_UID/ATTR_GID) as that always clears suid/sgid irrespective of capabilities of calling process.
I'm undecided on this. Would it hurt to set it on chown? That
might make the logic in some servers simpler, no?
What would be the drawback of setting FATTR_KILL_PRIV for chown as well?
Thanks,
Miklos
Re: [PATCH v3 3/6] fuse: setattr should set FATTR_KILL_PRIV upon size change
Quoted 11 lines · Miklos Szeredi
Quoted 10 lines · Vivek Goyal
If
fc->handle_killpriv_v2is enabled, we expect file server to clear suid/sgid/security.capbility upon chown/truncate/write as appropriate.Upon truncate (
ATTR_SIZE), suid/sgid is cleared only if caller does not haveCAP_FSETID. File server does not know whether caller hasCAP_FSETIDor not. Hence setFATTR_KILL_PRIVupon truncate to let file server know that caller does not haveCAP_FSETIDand it should kill suid/sgid as appropriate.
Quoted 8 lines
Quoted 3 lines
We don't have to send this information for chown (ATTR_UID/ATTR_GID) as that always clears suid/sgid irrespective of capabilities of calling process.
I'm undecided on this. Would it hurt to set it on chown? That
might make the logic in some servers simpler, no?What would be the drawback of setting
FATTR_KILL_PRIVfor chown as well?
Hi Miklos,
Thinking loud.
So these are the rules we expect from VFS point of view.
- caps are always cleared on chown/write/truncate
- suid is always cleared on chown, while for truncate/write it is cleared only if caller does not have CAP_FSETID.
- sgid is always cleared on chown, while for truncate/write it is cleared only if caller does not have CAP_FSETID as well as file has group execute permission.
Quoted 1 line
From server point of view, these rules become.
- caps are always cleared on chown/write/truncate
- suid is always cleared on chown, while for truncate/write it is cleared only if client set appropriate flag.
- For truncate, this flag will either be FUSE_OPEN_KILL_PRIV or FATTR_KILL_PRIV.
- For write, FUSE_WRITE_KILL_PRIV will be set.
- sgid is always cleared on chown, while for truncate/write it is cleared only if caller has set a flag as well as file has group execute permission.
- For truncate, this flag will either be FUSE_OPEN_KILL_PRIV or FATTR_KILL_PRIV.
- For write, FUSE_WRITE_KILL_PRIV will be set.
Above rules assumes that chown() will always clear caps/suid/sgid and server does not have to rely on any flags.
I think it does not hurt to start passing FATTR_KILL_PRIV for chown() as well. In that case, server will always clear caps on chown but clear suid/sgid only if FATTR_KILL_PRIV is set. (Which will always be set).
So anything is fine. We just need to document it well. I think I will write it very clearly in qemu patch depending on what goes in kernel.
Thanks
Vivek
Miklos Szeredi
maintainerRe: [PATCH v3 3/6] fuse: setattr should set FATTR_KILL_PRIV upon size change
Quoted 4 lines · Vivek Goyal
I think it does not hurt to start passing
FATTR_KILL_PRIVforchown()as well. In that case, server will always clear caps on chown but clear suid/sgid only ifFATTR_KILL_PRIVis set. (Which will always be set).
Okay.
More thoughts for FUSE_HANDLE_KILLPRIV_V2:
- clear "security.capability" on write, truncate and chown unconditionally
- clear suid/sgid if
o setattr has FATTR_SIZE and FATTR_KILL_PRIV
o setattr has FATTR_UID or FATTR_GID
o open has O_TRUNC and FUSE_OPEN_KILL_PRIV
o write has FUSE_WRITE_KILL_PRIV
Kernel has:ATTR_KILL_PRIV -> clear "security.capability"ATTR_KILL_SUID -> clear S_ISUIDATTR_KILL_SGID -> clear S_ISGID if executable
Fuse has:
FUSE_*KILL_PRIV -> clear S_ISUID and S_ISGID if executable
So the fuse meaning of FUSE_*KILL_PRIV has a complementary meaning to that of ATTR_KILL_PRIV, which is somewhat confusing. Also "PRIV" implies all privileges, including "security.capability" but the fuse ones relate to suid/sgid only.
How about FUSE_*KILL_SUIDGID (FUSE_WRITE_KILL_SUIDGID being an alias for FUSE_WRITE_KILL_PRIV)?
Thanks,
Miklos
Quoted 7 lines
So anything is fine. We just need to document it well. I think I will write it very clearly in qemu patch depending on what goes in kernel.
Thanks
Vivek
Re: [Virtio-fs] [PATCH v3 3/6] fuse: setattr should set FATTR_KILL_PRIV upon size change
I suggest default settings meet this test: A Fedora vm booted directly via virtio FS (with virtiofs as root) running with SE enabled, with xattr 'on' and the host fs xattr capable like btrfs (Fedora default) -- does not generate SE linux denials or 'attribute not settable' complaints. In other words -- 'it just works'. At least in the default fedora at this time that is not the case.
HC
Quoted 50 lines · Miklos Szeredi
Quoted 4 lines · Vivek Goyal
I think it does not hurt to start passing
FATTR_KILL_PRIVforchown()as well. In that case, server will always clear caps on chown but clear suid/sgid only ifFATTR_KILL_PRIVis set. (Which will always be set).Okay.
More thoughts for
FUSE_HANDLE_KILLPRIV_V2:- clear "security.capability" on write, truncate and chown unconditionally
- clear suid/sgid if
o setattr hasFATTR_SIZEandFATTR_KILL_PRIV
o setattr hasFATTR_UIDorFATTR_GID
o open hasO_TRUNCandFUSE_OPEN_KILL_PRIV
o write hasFUSE_WRITE_KILL_PRIVKernel has:
ATTR_KILL_PRIV-> clear "security.capability"ATTR_KILL_SUID-> clearS_ISUIDATTR_KILL_SGID-> clearS_ISGIDif executableFuse has:
FUSE_*KILL_PRIV -> clearS_ISUIDandS_ISGIDif executableSo the fuse meaning of FUSE_*KILL_PRIV has a complementary meaning to that of
ATTR_KILL_PRIV, which is somewhat confusing. Also "PRIV" implies all privileges, including "security.capability" but the fuse ones relate to suid/sgid only.How about FUSE_*KILL_SUIDGID (
FUSE_WRITE_KILL_SUIDGIDbeing an alias forFUSE_WRITE_KILL_PRIV)?Thanks,
MiklosQuoted 6 lines
So anything is fine. We just need to document it well. I think I will write it very clearly in qemu patch depending on what goes in kernel.
Thanks
Vivek_______________________________________________
Virtio-fs mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virtio-fs
Miklos Szeredi
maintainerRe: [PATCH v3 3/6] fuse: setattr should set FATTR_KILL_PRIV upon size change
In reply to Miklos SzerediQuoted 2 lines · Miklos Szeredi
How about FUSE_*KILL_SUIDGID (
FUSE_WRITE_KILL_SUIDGIDbeing an alias forFUSE_WRITE_KILL_PRIV)?
Series pushed to #for-next with these changes. Please take a look and test.
Thanks,
Miklos
Re: [PATCH v3 3/6] fuse: setattr should set FATTR_KILL_PRIV upon size change
Quoted 6 lines · Miklos Szeredi
Quoted 2 lines · Miklos Szeredi
How about FUSE_*KILL_SUIDGID (
FUSE_WRITE_KILL_SUIDGIDbeing an alias forFUSE_WRITE_KILL_PRIV)?Series pushed to #for-next with these changes. Please take a look and test.
Thanks Miklos. I looked at the patches quickly and they look good. I am now fixing qemu virtiofsd side and will test.
Thanks
Vivek
Re: [PATCH v3 3/6] fuse: setattr should set FATTR_KILL_PRIV upon size change
In reply to Miklos SzerediQuoted 26 lines · Miklos Szeredi
Quoted 4 lines · Vivek Goyal
I think it does not hurt to start passing
FATTR_KILL_PRIVforchown()as well. In that case, server will always clear caps on chown but clear suid/sgid only ifFATTR_KILL_PRIVis set. (Which will always be set).Okay.
More thoughts for
FUSE_HANDLE_KILLPRIV_V2:- clear "security.capability" on write, truncate and chown unconditionally
- clear suid/sgid if
o setattr hasFATTR_SIZEandFATTR_KILL_PRIV
o setattr hasFATTR_UIDorFATTR_GID
o open hasO_TRUNCandFUSE_OPEN_KILL_PRIV
o write hasFUSE_WRITE_KILL_PRIVKernel has:
ATTR_KILL_PRIV-> clear "security.capability"ATTR_KILL_SUID-> clearS_ISUIDATTR_KILL_SGID-> clearS_ISGIDif executableFuse has:
FUSE_*KILL_PRIV -> clearS_ISUIDandS_ISGIDif executable
Quoted 7 lines
So the fuse meaning of FUSE_*KILL_PRIV has a complementary meaning to that of
ATTR_KILL_PRIV, which is somewhat confusing. Also "PRIV" implies all privileges, including "security.capability" but the fuse ones relate to suid/sgid only.How about FUSE_*KILL_SUIDGID (
FUSE_WRITE_KILL_SUIDGIDbeing an alias forFUSE_WRITE_KILL_PRIV)?
Hi Miklos,
Renaming FUSE_*KILL_PRIV to FUSE_*KILL_SUIDSGID sounds good. For a breif moment I was also thinking that these FUSE_*KILL_PRIV and and ATTR_KILL_PRIV are not exactly mapping. Glad you caught it and made the situation better.
Thanks
Vivek
PATCH v3 4/6 fuse: Don't send ATTR_MODE to kill suid/sgid for handle_killpriv_v2
In reply to Vivek GoyalIf client does a write() on a suid/sgid file, VFS will first call fuse_setattr() with ATTR_KILL_S[UG]ID set. This requires sending setattr to file server with ATTR_MODE set to kill suid/sgid. But to do that client needs to know latest mode otherwise it is racy.
To reduce the race window, current code first call fuse_do_getattr() to get latest ->i_mode and then resets suid/sgid bits and sends rest to server with setattr(ATTR_MODE). This does not reduce the race completely but narrows race window significantly.
With fc->handle_killpriv_v2 enabled, it should be possible to remove this race completely. Do not kill suid/sgid with ATTR_MODE at all. It will be killed by server when WRITE request is sent to server soon. This is similar to fc->handle_killpriv logic. V2 is just more refined version of protocol. Hence this patch does not send ATTR_MODE to kill suid/sgid if fc->handle_killpriv_v2 is enabled.
This creates an issue if fc->writeback_cache is enabled. In that case WRITE can be cached in guest and server might not see WRITE request and hence will not kill suid/sgid. Miklos suggested that in such cases, we should fallback to a writethrough WRITE instead and that will generate WRITE request and kill suid/sgid. This patch implements that too.
But this relies on client seeing the suid/sgid set. If another client sets suid/sgid and this client does not see it immideately, then we will not fallback to writethrough WRITE. So this is one limitation with both fc->handle_killpriv_v2 and fc->writeback_cache enabled. Both the options are not fully compatible. But might be good enough for many use cases.
Note: I am not checking whether security.capability is set or not when
falling back to writethrough path. if suid/sgid is not set and only security.capability is set, that will be taken care of by file_remove_privs() call in ->writeback_cache path.
--- fs/fuse/dir.c | 2 +- fs/fuse/file.c | 9 ++++++++- 2 files changed, 9 insertions(+), 2 deletions(-)
fs/fuse/dir.c +1 -1
@@ -1664,7 +1664,7 @@ static int fuse_setattr(struct dentry *entry, struct iattr *attr)* * This should be done on write(), truncate() and chown(). */ -if (!fc->handle_killpriv) { +if (!fc->handle_killpriv && !fc->handle_killpriv_v2) {/* * ia_mode calculation may have used stale i_mode. * Refresh and recalculate.
fs/fuse/file.c +8 -1
@@ -1260,17 +1260,24 @@ static ssize_t fuse_cache_write_iter(struct kiocb *iocb, struct iov_iter *from)ssize_t written_buffered =0 ;struct inode *inode = mapping->host;ssize_t err; +struct fuse_conn *fc =get_fuse_conn (inode);loff_t endbyte =0 ; -if (get_fuse_conn (inode)->writeback_cache) { +if (fc->writeback_cache) {/* Update size (EOF optimization) and mode (SUID clearing) */ err =fuse_update_attributes (mapping->host, file);if (err)return err; +if (fc->handle_killpriv_v2 && +should_remove_suid (file_dentry (file))) { + goto writethrough; + } +return generic_file_write_iter (iocb, from); } +writethrough:inode_lock (inode);/* We can write back this queue in page reclaim */
Signature · 1 line
2.25.4
PATCH v3 5/6 fuse: Add a flag FUSE_OPEN_KILL_PRIV for open() request
In reply to Vivek GoyalWith FUSE_HANDLE_KILLPRIV_V2 support, server will need to kill suid/sgid/security.capability on open(O_TRUNC), if server supports FUSE_ATOMIC_O_TRUNC.
But server needs to kill suid/sgid only if caller does not have CAP_FSETID. Given server does not have this information, client needs to send this info to server.
So add a flag FUSE_OPEN_KILL_PRIV to fuse_open_in request which tells server to kill suid/sgid(only if group execute is set).
--- fs/fuse/file.c | 5 +++++ include/uapi/linux/fuse.h | 10 +++++++++- 2 files changed, 14 insertions(+), 1 deletion(-)
fs/fuse/file.c +5 -0
@@ -42,6 +42,11 @@ static int fuse_send_open(struct fuse_conn *fc, u64 nodeid, struct file *file, inarg.flags = file->f_flags & ~(O_CREAT |O_EXCL |O_NOCTTY );if (!fc->atomic_o_trunc) inarg.flags &= ~O_TRUNC ; + +if (fc->handle_killpriv_v2 && (inarg.flags &O_TRUNC ) && + !capable (CAP_FSETID )) + inarg.open_flags |=FUSE_OPEN_KILL_PRIV ; + args.opcode = opcode; args.nodeid = nodeid; args.in_numargs =1 ;
include/uapi/linux/fuse.h +9 -1
@@ -173,6 +173,7 @@* - add FUSE_SETUPMAPPING and FUSE_REMOVEMAPPING * - add map_alignment to fuse_init_out, add FUSE_MAP_ALIGNMENT flag * - add FUSE_HANDLE_KILLPRIV_V2 +* - add FUSE_OPEN_KILL_PRIV */ #ifndef _LINUX_FUSE_H
@@ -427,6 +428,13 @@ struct fuse_file_lock {*/ #define FUSE_FSYNC_FDATASYNC (1 << 0) +/** +* Open flags +* FUSE_OPEN_KILL_PRIV: Kill suid/sgid/security.capability. sgid is cleared +* only if file has group execute permission. +*/ +#define FUSE_OPEN_KILL_PRIV (1 << 0) +enum fuse_opcode {FUSE_LOOKUP =1 ,FUSE_FORGET =2 ,/* no reply */
@@ -588,7 +596,7 @@ struct fuse_setattr_in {struct fuse_open_in {uint32_t flags; -uint32_t unused; +uint32_t open_flags; };struct fuse_create_in {
Signature · 1 line
2.25.4
Miklos Szeredi
maintainerRe: [PATCH v3 5/6] fuse: Add a flag FUSE_OPEN_KILL_PRIV for open() request
Quoted 11 lines · Vivek Goyal
With
FUSE_HANDLE_KILLPRIV_V2support, server will need to kill suid/sgid/security.capability onopen(O_TRUNC), if server supportsFUSE_ATOMIC_O_TRUNC.But server needs to kill suid/sgid only if caller does not have
CAP_FSETID. Given server does not have this information, client needs to send this info to server.So add a flag
FUSE_OPEN_KILL_PRIVtofuse_open_inrequest which tells server to kill suid/sgid(only if group execute is set).
This is needed for FUSE_CREATE as well (which may act as a normal open in case the file exists, and no O_EXCL was specified), right?
I can edit the patch, if you agree.
Thanks,
Miklos
Re: [PATCH v3 5/6] fuse: Add a flag FUSE_OPEN_KILL_PRIV for open() request
Quoted 10 lines · Miklos Szeredi
Quoted 9 lines · Vivek Goyal
With
FUSE_HANDLE_KILLPRIV_V2support, server will need to kill suid/sgid/security.capability onopen(O_TRUNC), if server supportsFUSE_ATOMIC_O_TRUNC.But server needs to kill suid/sgid only if caller does not have
CAP_FSETID. Given server does not have this information, client needs to send this info to server.
Quoted 5 lines
Quoted 2 lines
So add a flag
FUSE_OPEN_KILL_PRIVtofuse_open_inrequest which tells server to kill suid/sgid(only if group execute is set).This is needed for
FUSE_CREATEas well (which may act as a normal open in case the file exists, and noO_EXCLwas specified), right?
Hi Miklos,
IIUC, In current code we seem to use FUSE_CREATE only if file does not exist. If file exists, then we probably will take FUSE_OPEN path.
Are you concerned about future proofing where somebody decides to use FUSE_CREATE for create + open on a file which exists. If yes, I agree that patching FUSE_CREATE makes sense.
Quoted 2 lines
I can edit the patch, if you agree.
Please do.
Thanks
Vivek
Miklos Szeredi
maintainerRe: [PATCH v3 5/6] fuse: Add a flag FUSE_OPEN_KILL_PRIV for open() request
Quoted 15 lines · Vivek Goyal
Quoted 13 lines · Miklos Szeredi
Quoted 11 lines · Vivek Goyal
With
FUSE_HANDLE_KILLPRIV_V2support, server will need to kill suid/sgid/security.capability onopen(O_TRUNC), if server supportsFUSE_ATOMIC_O_TRUNC.But server needs to kill suid/sgid only if caller does not have
CAP_FSETID. Given server does not have this information, client needs to send this info to server.So add a flag
FUSE_OPEN_KILL_PRIVtofuse_open_inrequest which tells server to kill suid/sgid(only if group execute is set).
Quoted 7 lines
Quoted 2 lines
This is needed for
FUSE_CREATEas well (which may act as a normal open in case the file exists, and noO_EXCLwas specified), right?Hi Miklos,
IIUC, In current code we seem to use
FUSE_CREATEonly if file does not exist. If file exists, then we probably will takeFUSE_OPENpath.
That's true if the cache is up to date, one important point for FUSE_CREATE is that it works atomically even if the cache is stale. So if cache is negative and we send a FUSE_CREATE it may still open an *existing* file, and we want to do suid/caps clearing in that case also, no?
Thanks,
Miklos
Re: [PATCH v3 5/6] fuse: Add a flag FUSE_OPEN_KILL_PRIV for open() request
Quoted 21 lines · Miklos Szeredi
Quoted 20 lines · Vivek Goyal
Quoted 10 lines · Miklos Szeredi
Quoted 9 lines · Vivek Goyal
With
FUSE_HANDLE_KILLPRIV_V2support, server will need to kill suid/sgid/security.capability onopen(O_TRUNC), if server supportsFUSE_ATOMIC_O_TRUNC.But server needs to kill suid/sgid only if caller does not have
CAP_FSETID. Given server does not have this information, client needs to send this info to server.Quoted 5 lines
Quoted 2 lines
So add a flag
FUSE_OPEN_KILL_PRIVtofuse_open_inrequest which tells server to kill suid/sgid(only if group execute is set).This is needed for
FUSE_CREATEas well (which may act as a normal open in case the file exists, and noO_EXCLwas specified), right?Hi Miklos,
Quoted 8 lines
Quoted 2 lines
IIUC, In current code we seem to use
FUSE_CREATEonly if file does not exist. If file exists, then we probably will takeFUSE_OPENpath.That's true if the cache is up to date, one important point for
FUSE_CREATEis that it works atomically even if the cache is stale. So if cache is negative and we send aFUSE_CREATEit may still open an *existing* file, and we want to do suid/caps clearing in that case also, no?
Yes, makes sense. This can happen in a race condition also where fuse_lookup_name() gets a negative dentry and then another client creates file (with setuid/setgid/caps) set. Now fuse_create_open() is called without O_EXCL and in that case we should remove setuid/setgid/caps as needed.
So yes, please make modifications accordingly for FUSE_CREATE. If you want me to do make those changes, please let me know.
Thanks
Vivek
PATCH v3 6/6 fuse: Support SB_NOSEC flag to improve direct write performance
In reply to Vivek Goyalvirtiofs can be slow with small writes if xattr are enabled and we are doing cached writes (No direct I/). Ganesh Mahalingam noticed this here.
https://github.com/kata-containers/runtime/issues/2815
Some debugging showed that that file_remove_privs() is called in cached write path on every write. And everytime it calls security_inode_need_killpriv() which results in call to __vfs_getxattr(XATTR_NAME_CAPS). And this goes to file server to fetch xattr. This extra round trip for every write slows down writes tremendously.
Normally to avoid paying this penalty on every write, vfs has the notion of caching this information in inode (S_NOSEC). So vfs sets S_NOSEC, if filesystem opted for it using super block flag SB_NOSEC. And S_NOSEC is cleared when setuid/setgid bit is set or when security xattr is set on inode so that next time a write happens, we check inode again for clearing setuid/setgid bits as well clear any security.capability xattr.
This seems to work well for local file systems but for remote file systems it is possible that VFS does not have full picture and a different client sets setuid/setgid bit or security.capability xattr on file and that means VFS information about S_NOSEC on another client will be stale. So for remote filesystems SB_NOSEC was disabled by default.
commit 9e1f1de02c2275d7172e18dc4e7c2065777611bf
Author: Al Viro <[email protected]> Date: Fri Jun 3 18:24:58 2011 -0400
more conservative S_NOSEC handling
That commit mentioned that these filesystems can still make use of SB_NOSEC as long as they clear S_NOSEC when they are refreshing inode attriutes from server.
So this patch tries to enable SB_NOSEC on fuse (regular fuse as well as virtiofs). And clear SB_NOSEC when we are refreshing inode attributes.
This is enabled only if server supports FUSE_HANDLE_KILLPRIV_V2. This says that server will clear setuid/setgid/security.capability on chown/truncate/write as apporpriate.
This should provide tighter coherency because now suid/sgid/security.capability will be cleared even if fuse client cache has not seen these attrs.
Basic idea is that fuse client will trigger suid/sgid/security.capability clearing based on its attr cache. But even if cache has gone stale, it is fine because FUSE_HANDLE_KILLPRIV_V2 will make sure WRITE clear suid/sgid/security.capability.
We make this change only if server supports FUSE_HANDLE_KILLPRIV_V2. This should make sure that existing filesystems which might be relying on seucurity.capability always being queried from server are not impacted.
This tighter coherency relies on WRITE showing up on server (and not being cached in guest). So writeback_cache mode will not provide that tight coherency and it is not recommended to use two together. Having said that it might work reasonably well for lot of use cases.
This change improves random write performance very significantly. I am running virtiofsd with cache=auto and following fio command.
fio --ioengine=libaio --direct=1 --name=test --filename=/mnt/virtiofs/random_read_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randwrite
Before this patch I get around 50MB/s and after the patch I get around 250MB/s bandwidth. So improvement is very significant.
--- fs/fuse/inode.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
fs/fuse/inode.c +13 -1
@@ -201,6 +201,16 @@ void fuse_change_attributes_common(struct inode *inode, struct fuse_attr *attr, inode->i_mode &= ~S_ISVTX ; fi->orig_ino = attr->ino; + +/* +* We are refreshing inode data and it is possible that another +* client set suid/sgid or security.capability xattr. So clear +* S_NOSEC. Ideally, we could have cleared it only if suid/sgid +* was set or if security.capability xattr was set. But we don't +* know if security.capability has been set or not. So clear it +* anyway. Its less efficient but should is safe. +*/ + inode->i_flags &= ~S_NOSEC ; } void fuse_change_attributes(struct inode *inode, struct fuse_attr *attr,
@@ -993,8 +1003,10 @@ static void process_init_reply(struct fuse_conn *fc, struct fuse_args *args, !fuse_dax_check_alignment(fc, arg->map_alignment)) { ok = false; } -if (arg->flags &FUSE_HANDLE_KILLPRIV_V2 ) +if (arg->flags &FUSE_HANDLE_KILLPRIV_V2 ) { fc->handle_killpriv_v2 =1 ; + fc->sb->s_flags |=SB_NOSEC ; + } }else { ra_pages = fc->max_read /PAGE_SIZE ; fc->no_lock =1 ;
Signature · 1 line
2.25.4