vfs.git part 2
Subsystems (5)
Al Viro
maintainer Assorted f_pos race fixes, making do_splice_direct() safe to
call with i_mutex on parent, O_TMPFILE support, Jeff's locks.c series, ->d_hash/->d_compare calling conventions changes from Linus, misc stuff all over the place. Please, pull fromgit://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git for-linus
Shortlog:
Al Viro (46):
pcm_native: switch to fdget()/fdput()
do_last(): fix missing checks for LAST_BIND case
lift file_*_write out of do_splice_from()
lift file_*_write out of do_splice_direct()
allow build_open_flags() to return an error
[O_TMPFILE] it's still short a few helpers, but infrastructure should be OK now... allow the temp files created by open() to be linked to
ext3 ->tmpfile() support
udf: provide ->tmpfile()
don't call file_pos_write() if vfs_{read,write}{,v}() fails
ncpfs: don't bother with EBUSY on removal of busy directories proc_fill_cache(): kill pointless check
proc_pid_readdir(): stop wanking with proc_fill_cache() for /proc/self proc_fill_cache(): just make instantiate_t return int proc_fill_cache(): clean up, get rid of pointless find_inode_number() use
coda: don't bother with find_inode_number()
kill find_inode_number()
more open-coded file_inode() calls
comedi: quit wanking with FASYNC in ->release()
fanotify: quit wanking with FASYNC in ->release()
btrfs: more open-coded file_inode()
fuse: another open-coded file_inode()
ecryptfs: switch ecryptfs_decode_and_decrypt_filename() from dentry to sb new helper: fixed_size_llseek()
mtdchar: switch to fixed_size_llseek()
zorro: switch to fixed_size_llseek()
bna: switch to fixed_size_llseek()
eisa_eeprom: switch to fixed_size_llseek()
vc: switch to fixed_size_llseek()
fnic: switch to fixed_size_llseek()
bfa: switch to fixed_size_llseek()
wlcore: use *ppos, not file->f_pos
ps3flash: switch to generic_file_llseek_size()
constify rw_verify_area()
splice: lift checks from do_splice_from() into callers
lpfc: switch to fixed_size_llseek()
isapnp: switch to fixed_size_llseek()
pci/proc: switch to fixed_size_llseek()
ubi/cdev: switch to fixed_size_llseek()
proc_powerpc: switch to fixed_size_llseek()
tile-srom: switch to fixed_size_llseek()
cpqphp_sysfs: switch to fixed_size_llseek()
block_dev: switch to fixed_size_llseek()
lseek_execute() doesn't need an inode passed to it
ext4: ->tmpfile() support
Document ->tmpfile()
Dan Carpenter (1):
minix: bug widening a binary "not" operation
David Howells (2):
Replace a bunch of file->dentry->d_inode refs with file_inode()
SELinux: Institute file_path_has_perm()
Jeff Layton (12):
locks: drop the unused filp argument to posix_unblock_lock
cifs: use posix_unblock_lock instead of locks_delete_block
locks: make generic_add_lease and generic_delete_lease static
locks: comment cleanups and clarifications
locks: make "added" in __posix_lock_file a bool
locks: encapsulate the fl_link list handling
locks: protect most of the file_lock handling with i_lock
locks: avoid taking global lock if possible when waking up blocked waiters
locks: convert fl_link to a hlist_node
locks: turn the blocked_list into a hashtable
locks: add a new "lm_owner_key" lock operation
locks: give the blocked_hash its own spinlock
Jie Liu (1):
vfs: export lseek_execute() to modules
Linus Torvalds (1):
Don't pass inode to ->d_hash() and ->d_compare()
Diffstat:
Documentation/filesystems/Locking | 35 ++- Documentation/filesystems/vfs.txt | 24 +- arch/alpha/include/uapi/asm/fcntl.h | 1 + arch/arc/kernel/troubleshoot.c | 2 +- arch/parisc/include/uapi/asm/fcntl.h | 1 + arch/powerpc/kernel/proc_powerpc.c | 20 +-- arch/sparc/include/uapi/asm/fcntl.h | 1 + drivers/char/ps3flash.c | 28 +-- drivers/char/tile-srom.c | 28 +-- drivers/mtd/mtdchar.c | 20 +-- drivers/mtd/ubi/cdev.c | 26 +-- drivers/net/ethernet/brocade/bna/bnad_debugfs.c | 22 +-- drivers/net/wireless/ti/wlcore/debugfs.c | 4 +- drivers/parisc/eisa_eeprom.c | 15 +- drivers/pci/hotplug/cpqphp_sysfs.c | 22 +-- drivers/pci/proc.c | 23 +-- drivers/pnp/isapnp/proc.c | 22 +-- drivers/scsi/bfa/bfad_debugfs.c | 28 +-- drivers/scsi/fnic/fnic_debugfs.c | 16 +-- drivers/scsi/lpfc/lpfc_debugfs.c | 18 +-- drivers/staging/android/logger.c | 2 +- drivers/staging/comedi/comedi_fops.c | 3 - drivers/tty/vt/vc_screen.c | 17 +-- drivers/zorro/proc.c | 22 +-- fs/adfs/dir.c | 6 +- fs/affs/namei.c | 26 +-- fs/afs/flock.c | 7 +- fs/aio.c | 2 + fs/block_dev.c | 23 +-- fs/btrfs/file.c | 15 +-- fs/btrfs/ioctl.c | 8 +- fs/ceph/file.c | 11 +- fs/ceph/locks.c | 2 +- fs/ceph/mds_client.c | 8 +- fs/cifs/cifsfs.c | 2 +- fs/cifs/dir.c | 9 +- fs/cifs/file.c | 15 +- fs/coda/dir.c | 8 +- fs/dcache.c | 66 ++---- fs/ecryptfs/crypto.c | 5 +- fs/ecryptfs/ecryptfs_kernel.h | 2 +- fs/ecryptfs/file.c | 9 +- fs/ecryptfs/inode.c | 2 +- fs/efivarfs/super.c | 9 +- fs/exec.c | 10 +- fs/ext2/namei.c | 24 ++ fs/ext3/namei.c | 47 ++++- fs/ext4/file.c | 24 +-- fs/ext4/namei.c | 47 ++++- fs/fat/namei_msdos.c | 6 +- fs/fat/namei_vfat.c | 12 +- fs/file_table.c | 2 +- fs/fuse/file.c | 3 +- fs/gfs2/dentry.c | 3 +- fs/gfs2/file.c | 2 +- fs/hfs/hfs_fs.h | 7 +- fs/hfs/string.c | 6 +- fs/hfsplus/hfsplus_fs.h | 7 +- fs/hfsplus/unicode.c | 7 +- fs/hpfs/dentry.c | 7 +- fs/inode.c | 4 +- fs/internal.h | 6 +- fs/isofs/inode.c | 48 ++--- fs/isofs/namei.c | 3 +- fs/jfs/namei.c | 7 +- fs/lockd/svclock.c | 14 +- fs/lockd/svcsubs.c | 12 +- fs/locks.c | 281 ++++++++++++++++------- fs/minix/dir.c | 2 +- fs/minix/namei.c | 13 + fs/namei.c | 113 +++++++--- fs/ncpfs/dir.c | 45 ++-- fs/ncpfs/inode.c | 4 + fs/nfs/delegation.c | 10 +- fs/nfs/nfs4state.c | 8 +- fs/nfsd/nfs4state.c | 8 +- fs/notify/fanotify/fanotify_user.c | 3 - fs/ocfs2/file.c | 12 +- fs/open.c | 63 +++-- fs/proc/base.c | 105 ++++----- fs/proc/fd.c | 18 +- fs/proc/internal.h | 2 +- fs/proc/namespaces.c | 13 +- fs/proc/proc_sysctl.c | 7 +- fs/read_write.c | 65 ++++-- fs/splice.c | 38 ++-- fs/sysv/namei.c | 3 +- fs/udf/namei.c | 24 ++ fs/xfs/xfs_file.c | 6 +- include/linux/dcache.h | 11 +- include/linux/fs.h | 49 +++-- include/linux/fsnotify.h | 8 +- include/uapi/asm-generic/fcntl.h | 4 + mm/shmem.c | 37 +++- net/sunrpc/auth_gss/svcauth_gss.c | 4 +- security/integrity/ima/ima_main.c | 2 +- security/selinux/hooks.c | 24 ++- sound/core/pcm_native.c | 40 ++-- 98 files changed, 979 insertions(+), 986 deletions(-)
Quoted 5 lines · Al Viro
Assorted
f_posrace fixes, makingdo_splice_direct()safe to
call withi_mutexon parent,O_TMPFILEsupport, Jeff'slocks.cseries, ->d_hash/->d_compare calling conventions changes from Linus, misc stuff all over the place. Please, pull fromgit://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.gitfor-linus
Btw, the new O_TMPFILE support seems to have a fundamental interface
problem: old kernels ignore that flag, and happily implement totally different semantics than the intended new ones.
So with a *new* kernel, if the filesystem doesn't support O_TMPFILE, you get an ENOTSUPP error, and you can fall back on whatever old tmpfile logic you had.
But with an *old* kernel, O_TMPFILE will just silently be ignored as an unrecognized flag, and things won't work. If you do
fd = open("dirname", O_CREAT | O_TMPFILE | O_RDWR, 0666);it may be that it ends up acting as a "create file at specified directory path" instead of what the user *meant* for it to do, which was "create unnamed temporary file in the specified directory".
This seems to make the feature actively dangerous. You can't just try to use it and have a fallback, because that "try to use it" phase may incorrectly succeed.
Yes, you can force things to not work on old systems by having a slash at the end of the directory name, but if you ever forget that, you'll end up with the above problem.
Am I missing something?
Linus
Al Viro
maintainerQuoted 28 lines · Linus Torvalds
Quoted 5 lines · Al Viro
Assorted
f_posrace fixes, makingdo_splice_direct()safe to
call withi_mutexon parent,O_TMPFILEsupport, Jeff'slocks.cseries, ->d_hash/->d_compare calling conventions changes from Linus, misc stuff all over the place. Please, pull fromgit://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.gitfor-linusBtw, the new
O_TMPFILEsupport seems to have a fundamental interface
problem: old kernels ignore that flag, and happily implement totally different semantics than the intended new ones.So with a *new* kernel, if the filesystem doesn't support
O_TMPFILE, you get an ENOTSUPP error, and you can fall back on whatever old tmpfile logic you had.But with an *old* kernel,
O_TMPFILEwill just silently be ignored as an unrecognized flag, and things won't work. If you dofd = open("dirname", O_CREAT | O_TMPFILE | O_RDWR, 0666);it may be that it ends up acting as a "create file at specified directory path" instead of what the user *meant* for it to do, which was "create unnamed temporary file in the specified directory".
This seems to make the feature actively dangerous. You can't just try to use it and have a fallback, because that "try to use it" phase may incorrectly succeed.
Quoted 5 lines
Yes, you can force things to not work on old systems by having a slash at the end of the directory name, but if you ever forget that, you'll end up with the above problem.
Am I missing something?
It's slightly less painful than that - if dirname exists, the old kernels will fail; O_CREAT for existing directory means an error. So in practice you can use it safely. I'm not too happy about that situation, but I hadn't been able to come up with anything better, short of a new syscall that would duplicate openat(2), but reject unknown values in flags argument from the very beginning ;-/ Which is what we probably should've done with openat(2) itself, but it's too late for that now...
Al Viro
maintainerQuoted 7 lines · Al Viro
It's slightly less painful than that - if dirname exists, the old kernels will fail;
O_CREATfor existing directory means an error. So in practice you can use it safely. I'm not too happy about that situation, but I hadn't been able to come up with anything better, short of a new syscall that would duplicate openat(2), but reject unknown values in flags argument from the very beginning ;-/ Which is what we probably should've done with openat(2) itself, but it's too late for that now...
FWIW, it might make sense to do sys_openat2() that would really validate the 'flags' argument. The question is, what checks do we want to do? The situation for open(2) is complicated by OSF, HPUX and Solaris compat. AFAICS, we have the following picture:
* two bits are used to encode O_RDONLY/O_WRONLY/O_RDWR/open-for-ioctl Any combination is allowed. Ignored with O_PATH.
* O_CLOEXEC - orthogonal to everything else
* O_NOATIME, O_APPEND, O_DIRECT, O_NONBLOCK, O_NDELAY - more or less arch-independent (O_NDELAY may or may not coincide with O_NONBLOCK, on parisc O_NOATIME is the same value as O_INVISIBLE, whatever that is). Can be changed by fcntl(), probably shouldn't be allowed with O_PATH (right now we just ignore those in that case). For that matter, F_SETFL makes no sense for O_PATH descriptors. O_APPEND makes no sense for O_RDONLY opens; might be better to leave that as-is, though.
* O_DIRECTORY - shouldn't be allowed with O_TMPFILE. Requires either O_RDONLY or O_PATH (in which case both lower bits are ignored anyway).
* O_NOFOLLOW - orthogonal to everything else
* O_NOCTTY, O_LARGEFILE - probably shouldn't be allowed with O_PATH (currently ignored in that case).
* O_CREAT - makes no sense with O_PATH. O_TMPFILE currently demands it, but that's just an attempt to get it fail on old kernels more reliably.
* O_TRUNC - makes no sense with O_PATH. Pointless with O_TMPFILE.
* O_EXCL - $DEITY-awful mess. Combination with O_CREAT is probably the most regular part; however, it is usable without O_CREAT for some devices, with varying semantics. I'd mapped "can't link tmpfile in place" semantics on that one (with O_CREAT | O_TMPFILE).
* O_SYNC and things around it - arch-dependent mess.
* O_BLKSEEK, etc. - weird flags from HPUX et.al.; ignored by the kernel, OSF fortunately has only O_SYNC/O_DSYNC near that pile and that's the only one we might realistically care about - Solaris compat is gone and HPUX one is not much better off.
IMO we should fail on any unknown bits if we go that way, and probably tighten O_PATH side of things as well. It's still not too late - we can just add sys_openat2() and make sys_openat() ignore O_TMPFILE completely. Do you want to go that way? If so, let's decide what validation will be done and spell it out explicitly...
Quoted 6 lines · Al Viro
Quoted 5 lines · Linus Torvalds
But with an *old* kernel,
O_TMPFILEwill just silently be ignored as an unrecognized flag, and things won't work. If you dofd = open("dirname", O_CREAT | O_TMPFILE | O_RDWR, 0666);
Quoted 7 lines
Quoted 4 lines
it may be that it ends up acting as a "create file at specified directory path" instead of what the user *meant* for it to do, which was "create unnamed temporary file in the specified directory".
It's slightly less painful than that - if dirname exists, the old kernels will fail;
O_CREATfor existing directory means an error.
But isn't the problem the case where dirname does not exist? I.e., the application has to make sure that "/some/where" exists and is a directory before open("/some/where", O_CREAT | O_TMPFILE | O_RDWR, 0666) can be relied upon to fail on kernels not recognizing O_TMPFILE, instead of just creating "where" in "/some".
Just thinking out loud, and please tell me to shut up if it doesn't make
sense: The documentation for O_DIRECTORY seems to imply that one could require O_DIRECTORY to be given when using O_TMPFILE. The "If pathname is not a directory, cause the open to fail" certainly seems to make sense when O_TMPFILE is used, and older kernels should complain when seeing the O_CREAT|O_DIRECTORY combination. It is a hack, though.
Rasmus
Al Viro
maintainerQuoted 6 lines · Rasmus Villemoes
But isn't the problem the case where dirname does not exist? I.e., the application has to make sure that "/some/where" exists and is a directory before
open("/some/where", O_CREAT | O_TMPFILE | O_RDWR, 0666)can be relied upon to fail on kernels not recognizingO_TMPFILE, instead of just creating "where" in "/some".
Quoted 6 lines
Just thinking out loud, and please tell me to shut up if it doesn't make
sense: The documentation forO_DIRECTORYseems to imply that one could requireO_DIRECTORYto be given when usingO_TMPFILE. The "If pathname is not a directory, cause the open to fail" certainly seems to make sense whenO_TMPFILEis used, and older kernels should complain when seeing the O_CREAT|O_DIRECTORY combination. It is a hack, though.
They should, but they won't ;-/ It's the same problem - we do *not* validate the flags argument. We'll get to do_last(), hit lookup_open(), which will create the sucker and go to finish_open_created. Which is past the logics checking for LOOKUP_DIRECTORY trying to return a non-directory and it would've been too late to fail anyway - the file has already been created. IOW, O_DIRECTORY is ignored when O_CREAT is present *and* file didn't exist already. In that case we almost certainly can treat that as a bug (i.e. start failing open() on O_CREAT | O_DIRECTORY in all cases - I'd be _very_ surprised if somebody called open() with such combination of flags), but that doesn't help with older kernels...
Quoted 8 lines · Al Viro
Quoted 6 lines · Rasmus Villemoes
But isn't the problem the case where dirname does not exist? I.e., the application has to make sure that "/some/where" exists and is a directory before
open("/some/where", O_CREAT | O_TMPFILE | O_RDWR, 0666)can be relied upon to fail on kernels not recognizingO_TMPFILE, instead of just creating "where" in "/some".
Quoted 8 lines
Quoted 6 lines
Just thinking out loud, and please tell me to shut up if it doesn't make
sense: The documentation forO_DIRECTORYseems to imply that one could requireO_DIRECTORYto be given when usingO_TMPFILE. The "If pathname is not a directory, cause the open to fail" certainly seems to make sense whenO_TMPFILEis used, and older kernels should complain when seeing the O_CREAT|O_DIRECTORY combination. It is a hack, though.They should, but they won't ;-/
I see; I should test before I post, but...
Quoted 11 lines
It's the same problem - we do *not* validate the flags argument. We'll get to
do_last(), hitlookup_open(), which will create the sucker and go tofinish_open_created. Which is past the logics checking forLOOKUP_DIRECTORYtrying to return a non-directory and it would've been too late to fail anyway - the file has already been created. IOW,O_DIRECTORYis ignored whenO_CREATis present *and* file didn't exist already. In that case we almost certainly can treat that as a bug (i.e. start failingopen()onO_CREAT|O_DIRECTORYin all cases - I'd be _very_ surprised if somebody calledopen()with such combination of flags), but that doesn't help with older kernels...
... it seems that if one then omits O_CREAT, things work out ok, as long as one uses O_RDWR (which is the only sane thing to do with O_TMPFILE, I guess):
open("/tmp/test/dir", O_DIRECTORY | O_RDWR, 0666) -> -1; Is a directory open("/tmp/test/dir", O_DIRECTORY | O_RDONLY, 0666) -> 3; Success open("/tmp/test/file", O_DIRECTORY | O_RDWR, 0666) -> -1; Not a directory open("/tmp/test/link_to_file", O_DIRECTORY | O_RDWR, 0666) -> -1; Not a directory open("/tmp/test/link_to_nowhere", O_DIRECTORY | O_RDWR, 0666) -> -1; No such file or directory open("/tmp/test/link_to_dir", O_DIRECTORY | O_RDWR, 0666) -> -1; Is a directory open("/tmp/test/link_to_dir", O_DIRECTORY | O_RDONLY, 0666) -> 3; Success
open("/tmp/test/link_to_dir", O_NOFOLLOW | O_DIRECTORY | O_RDWR, 0666) -> -1; Too many levels of symbolic links
open("/tmp/test/link_to_dir", O_NOFOLLOW | O_DIRECTORY | O_RDONLY, 0666) -> -1; Too many levels of symbolic links(The above flags are what an old kernel would effectively see with or without O_TMPFILE present, I suppose.)
How about simply making O_TMPFILE == O_DIRECTORY | O_RDWR | O_TMPFILE_INTERNAL, and letting the correct use be
open("/some/dir", O_TMPFILE) [with or without a mode argument]
Using O_DIRECTORY when we don't want to open a directory, and omitting O_CREAT when we do want to create something new, is somewhat counter-intuitive, but I think this would solve the problem with old kernels.
Rasmus
Al Viro
maintainerQuoted 43 lines · Rasmus Villemoes
Quoted 8 lines
Quoted 6 lines
Just thinking out loud, and please tell me to shut up if it doesn't make
sense: The documentation forO_DIRECTORYseems to imply that one could requireO_DIRECTORYto be given when usingO_TMPFILE. The "If pathname is not a directory, cause the open to fail" certainly seems to make sense whenO_TMPFILEis used, and older kernels should complain when seeing the O_CREAT|O_DIRECTORY combination. It is a hack, though.They should, but they won't ;-/
I see; I should test before I post, but...
Quoted 11 lines
It's the same problem - we do *not* validate the flags argument. We'll get to
do_last(), hitlookup_open(), which will create the sucker and go tofinish_open_created. Which is past the logics checking forLOOKUP_DIRECTORYtrying to return a non-directory and it would've been too late to fail anyway - the file has already been created. IOW,O_DIRECTORYis ignored whenO_CREATis present *and* file didn't exist already. In that case we almost certainly can treat that as a bug (i.e. start failingopen()onO_CREAT|O_DIRECTORYin all cases - I'd be _very_ surprised if somebody calledopen()with such combination of flags), but that doesn't help with older kernels...... it seems that if one then omits
O_CREAT, things work out ok, as long as one usesO_RDWR(which is the only sane thing to do withO_TMPFILE, I guess):
open("/tmp/test/dir", O_DIRECTORY | O_RDWR, 0666)-> -1; Is a directoryopen("/tmp/test/dir", O_DIRECTORY | O_RDONLY, 0666)-> 3; Successopen("/tmp/test/file", O_DIRECTORY | O_RDWR, 0666)-> -1; Not a directoryopen("/tmp/test/link_to_file", O_DIRECTORY | O_RDWR, 0666)-> -1; Not a directoryopen("/tmp/test/link_to_nowhere", O_DIRECTORY | O_RDWR, 0666)-> -1; No such file or directoryopen("/tmp/test/link_to_dir", O_DIRECTORY | O_RDWR, 0666)-> -1; Is a directoryopen("/tmp/test/link_to_dir", O_DIRECTORY | O_RDONLY, 0666)-> 3; Successopen("/tmp/test/link_to_dir", O_NOFOLLOW | O_DIRECTORY | O_RDWR, 0666) -> -1; Too many levels of symbolic links open("/tmp/test/link_to_dir", O_NOFOLLOW | O_DIRECTORY | O_RDONLY, 0666) -> -1; Too many levels of symbolic links(The above flags are what an old kernel would effectively see with or without
O_TMPFILEpresent, I suppose.)How about simply making
O_TMPFILE==O_DIRECTORY|O_RDWR|O_TMPFILE_INTERNAL, and letting the correct use be
Quoted 6 lines
open("/some/dir", O_TMPFILE)[with or without a mode argument]Using
O_DIRECTORYwhen we don't want to open a directory, and omittingO_CREATwhen we do want to create something new, is somewhat counter-intuitive, but I think this would solve the problem with old kernels.
Hrm... I can't say I like it, but it's almost OK; the only problem here is the bug fixed by commit bc77daa78 - on some of the old kernels (including 3.10, BTW) we used to allow opening /proc/self/fd/0 with O_DIRECTORY|O_RDWR ;-/
Said that, I think it's more tolerable than the kludge I came up with - one would need to pass it a procfs symlink as argument to hit that. Linus, your opinion?
Al Viro
maintainerQuoted 5 lines · Al Viro
Quoted 4 lines
Using
O_DIRECTORYwhen we don't want to open a directory, and omittingO_CREATwhen we do want to create something new, is somewhat counter-intuitive, but I think this would solve the problem with old kernels.
Quoted 7 lines
Hrm... I can't say I like it, but it's almost OK; the only problem here is the bug fixed by commit
bc77daa78- on some of the old kernels (including 3.10, BTW) we used to allow opening/proc/self/fd/0with O_DIRECTORY|O_RDWR ;-/Said that, I think it's more tolerable than the kludge I came up with - one would need to pass it a procfs symlink as argument to hit that. Linus, your opinion?
I mean something like this:
Safer ABI for O_TMPFILE
[suggested by Rasmus Villemoes] make O_DIRECTORY | O_RDWR part of O_TMPFILE; that will fail on old kernels in a lot more cases than what I came up with.
---
arch/alpha/include/uapi/asm/fcntl.h +1 -1
@@ -32,7 +32,7 @@#define O_SYNC (__O_SYNC|O_DSYNC)#define O_PATH 040000000 -#define O_TMPFILE 0100000000 +#define __O_TMPFILE 0100000000#define F_GETLK 7#define F_SETLK 8
arch/parisc/include/uapi/asm/fcntl.h +1 -1
@@ -20,7 +20,7 @@#define O_INVISIBLE 004000000/* invisible I/O, for DMAPI/XDSM */ #define O_PATH 020000000 -#define O_TMPFILE 040000000 +#define __O_TMPFILE 040000000#define F_GETLK64 8#define F_SETLK64 9
arch/sparc/include/uapi/asm/fcntl.h +1 -1
@@ -35,7 +35,7 @@#define O_SYNC (__O_SYNC|O_DSYNC)#define O_PATH 0x1000000 -#define O_TMPFILE 0x2000000 +#define __O_TMPFILE 0x2000000#define F_GETOWN 5/* for sockets. */ #define F_SETOWN 6/* for sockets. */
fs/namei.c +1 -1
@@ -2977,7 +2977,7 @@ static struct file *path_openat(int dfd, struct filename *pathname, file->f_flags = op->open_flag; -if (unlikely (file->f_flags &O_TMPFILE )) { +if (unlikely (file->f_flags & __O_TMPFILE)) { error =do_tmpfile (dfd, pathname, nd, flags, op, file, &opened); goto out; }
fs/open.c +2 -2
@@ -840,8 +840,8 @@ static inline int build_open_flags(int flags, umode_t mode, struct open_flags *oif (flags & __O_SYNC) flags |=O_DSYNC ; -if (flags &O_TMPFILE ) { -if (!(flags &O_CREAT )) +if (flags & __O_TMPFILE) { +if ((flags &O_TMPFILE ) !=O_TMPFILE )return -EINVAL ; acc_mode =MAY_OPEN |ACC_MODE (flags); }else if (flags &O_PATH ) {
include/uapi/asm-generic/fcntl.h +5 -2
@@ -84,10 +84,13 @@#define O_PATH 010000000#endif -#ifndef O_TMPFILE -#define O_TMPFILE 020000000 +#ifndef __O_TMPFILE +#define __O_TMPFILE 020000000#endif +/* a horrid kludge trying to make sure that this will fail on old kernels */ +#define O_TMPFILE (__O_TMPFILE | O_DIRECTORY | O_RDWR) +#ifndef O_NDELAY #define O_NDELAY O_NONBLOCK#endif
Quoted 7 lines · Al Viro
Quoted 6 lines · Rasmus Villemoes
How about simply making
O_TMPFILE==O_DIRECTORY|O_RDWR|O_TMPFILE_INTERNAL, and letting the correct use be
open("/some/dir", O_TMPFILE)[with or without a mode argument]
Quoted 7 lines
Hrm... I can't say I like it, but it's almost OK; the only problem here is the bug fixed by commit
bc77daa78- on some of the old kernels (including 3.10, BTW) we used to allow opening/proc/self/fd/0with O_DIRECTORY|O_RDWR ;-/Said that, I think it's more tolerable than the kludge I came up with - one would need to pass it a procfs symlink as argument to hit that. Linus, your opinion?
I think I like it. Because we really shouldn't rely on "the directory already exists", since it's actually quite possible that it doesn't. Sure, things like /tmp and /usr/tmp we can generally rely on, but mkstemp() and friends are often done using TMPDIR etc, so for a O_TMPFILE we really shouldn't assume that the directory is some long-term stable and reliable thing.
My only suggestion is that we *enforce* that O_DIRECTORY is set, and that O_CREAT is not set (the latter is the reverse of what we do now), so that we don't get programs that "happen" to work on older kernels (the /proc bug thing I think we can ignore - at least it makes the possibility of accidental problems much *much* less).
That said, I'm not sure about O_RDWR. There are ways to possibly turn an fd into a new path, so I could imagine O_WRONLY being useful ("create a temporary file, fill in the content, do fdatasync, then atomically make it appear in the filesystem with a linkat() system call").
I'd actually want to at least bring up again the possible requirement that the pathname argument to O_TEMPFILE must end in a '/'. It would be an easy check to add, and then we could actually drop the whole O_DIRECTORY flag, and O_CREAT becomes a non-issue too. The only downside of that is that it might be very inconvenient for user mode (eg if user-mode just wants to use the TMPDIR environment variable directly), so it might well make for a "better" patch for the kernel, but be much worse as an ABI issue.
As to the mode argument: we should encourage people to have it, since the inode *may* become visible afterwards. See above (can you do linkat() to just turn an fd into a name? I didn't really check, but I think you can do it as a "link(/proc/sef/fd/..)" thing regardless).
Linus
Al Viro
maintainerQuoted 4 lines · Linus Torvalds
As to the mode argument: we should encourage people to have it, since the inode *may* become visible afterwards. See above (can you do
linkat()to just turn an fd into a name? I didn't really check, but I think you can do it as a "link(/proc/sef/fd/..)" thing regardless).
link() won't do it, linkat() with the right arguments will (link(2) doesn't follow symlinks on the oldname). It's a bit more complicated, though:
* vfs_link() will refuse to link with link count that went down to 0
(always had, and that's a damn good idea for security reasons)
* with O_TMPFILE, vfs_link() will act as usual *if* you give open()O_EXCL as well - files created that way cannot be linked in, ever. They stay with link count 0 and they die as soon as they are closed. Again, that's the behaviour we'd have with O_CREAT open + unlink. tmpfile(3) ought to call it that way.
* O_TMPFILE _without_ O_EXCL == you may link it in once you've
decided that you've set it up enough. In that case you are supposed to open it, write whatever you want to write, fchmod/fchown/fsetxattr/etc. and use linkat() to put it in place. Or use vfs_link() if you are doing that kernel-side.
Quoted 7 lines · Al Viro
I mean something like this:
Safer ABI for
O_TMPFILE[suggested by Rasmus Villemoes] make
O_DIRECTORY|O_RDWRpart ofO_TMPFILE; that will fail on old kernels in a lot more cases than what I came up with.
So see earlier about why I'm not convinced about O_RDWR. But even if we really want that (and it might be better to start off too narrow than accept anything else) your patch tests those bits the wrong way (any O_RDWR test should be done using the O_ACCMODE mask, not using the O_RDWR value itself as a mask)
Also, to make sure that the "no preexisting file" case fails, I still think you should also verify that O_CREAT is not set. Otherwise Rasmus' case
open("/tmp/test/link_to_nowhere", O_DIRECTORY | O_RDWR, 0666) ->-1; No such file or directory
can work, and the case that rasmus didn't have at all (non-existent pathname) also just silently succeeds by creating a file instead of the expected directory..
So you could have something like
#define O_TMPFILE (__O_TMPFILE | O_DIRECTORY | O_RDWR) #define O_TMPFILE_MASK (__O_TMPFILE | O_DIRECTORY | O_CREAT | O_ACCMODE)
and then use
if ((flags & O_TMPFILE_MASK) != O_TMPFILE)
return -ENOTSUPP;
or whatever.
Hmm?
Linus
Al Viro
maintainerQuoted 11 lines · Linus Torvalds
So you could have something like
#define O_TMPFILE (__O_TMPFILE | O_DIRECTORY | O_RDWR) #define O_TMPFILE_MASK (__O_TMPFILE | O_DIRECTORY | O_CREAT | O_ACCMODE)and then use
if ((flags & O_TMPFILE_MASK) != O_TMPFILE) return -ENOTSUPP;or whatever.
Point... I'd rather use EINVAL, though. FWIW, we might want to add openat2() at some later point, with proper validation of arguments; I really don't want to think of the kludges we'll need the next time we add an open flag...
Safer ABI for O_TMPFILE
[suggested by Rasmus Villemoes] make O_DIRECTORY | O_RDWR part of O_TMPFILE; that will fail on old kernels in a lot more cases than what I came up with. And make sure O_CREAT doesn't get there...
---
arch/alpha/include/uapi/asm/fcntl.h +1 -1
@@ -32,7 +32,7 @@#define O_SYNC (__O_SYNC|O_DSYNC)#define O_PATH 040000000 -#define O_TMPFILE 0100000000 +#define __O_TMPFILE 0100000000#define F_GETLK 7#define F_SETLK 8
arch/parisc/include/uapi/asm/fcntl.h +1 -1
@@ -20,7 +20,7 @@#define O_INVISIBLE 004000000/* invisible I/O, for DMAPI/XDSM */ #define O_PATH 020000000 -#define O_TMPFILE 040000000 +#define __O_TMPFILE 040000000#define F_GETLK64 8#define F_SETLK64 9
arch/sparc/include/uapi/asm/fcntl.h +1 -1
@@ -35,7 +35,7 @@#define O_SYNC (__O_SYNC|O_DSYNC)#define O_PATH 0x1000000 -#define O_TMPFILE 0x2000000 +#define __O_TMPFILE 0x2000000#define F_GETOWN 5/* for sockets. */ #define F_SETOWN 6/* for sockets. */
fs/namei.c +1 -1
@@ -2977,7 +2977,7 @@ static struct file *path_openat(int dfd, struct filename *pathname, file->f_flags = op->open_flag; -if (unlikely (file->f_flags &O_TMPFILE )) { +if (unlikely (file->f_flags & __O_TMPFILE)) { error =do_tmpfile (dfd, pathname, nd, flags, op, file, &opened); goto out; }
fs/open.c +2 -2
@@ -840,8 +840,8 @@ static inline int build_open_flags(int flags, umode_t mode, struct open_flags *oif (flags & __O_SYNC) flags |=O_DSYNC ; -if (flags &O_TMPFILE ) { -if (!(flags &O_CREAT )) +if (flags & __O_TMPFILE) { +if ((flags &O_TMPFILE_MASK ) !=O_TMPFILE )return -EINVAL ; acc_mode =MAY_OPEN |ACC_MODE (flags); }else if (flags &O_PATH ) {
include/uapi/asm-generic/fcntl.h +6 -2
@@ -84,10 +84,14 @@#define O_PATH 010000000#endif -#ifndef O_TMPFILE -#define O_TMPFILE 020000000 +#ifndef __O_TMPFILE +#define __O_TMPFILE 020000000#endif +/* a horrid kludge trying to make sure that this will fail on old kernels */ +#define O_TMPFILE (__O_TMPFILE | O_DIRECTORY | O_RDWR) +#define O_TMPFILE_MASK (__O_TMPFILE | O_DIRECTORY | O_CREAT | O_ACCMODE) +#ifndef O_NDELAY #define O_NDELAY O_NONBLOCK#endif
Quoted 10 lines · Linus Torvalds
Quoted 3 lines · Al Viro
[suggested by Rasmus Villemoes] make
O_DIRECTORY|O_RDWRpart ofO_TMPFILE; that will fail on old kernels in a lot more cases than what I came up with.So see earlier about why I'm not convinced about
O_RDWR. But even if we really want that (and it might be better to start off too narrow than accept anything else) your patch tests those bits the wrong way (anyO_RDWRtest should be done using theO_ACCMODEmask, not using theO_RDWRvalue itself as a mask)
On further thought, I think I'll retract the suggestion to include O_RDWR in O_TMPFILE: If that is done, I don't see how one can ever allow O_WRONLY functionality without breaking the ABI (or introducing O_TMPFILE_WRONLY). So I suggest O_TMPFILE == O_DIRECTORY|__O_TMPFILE, and correct use is open(path, O_TMPFILE|O_RDWR, mode).
Also, O_TMPFILE without O_RDWR (or O_WRONLY) should then give -EINVAL.
Pros:
The access mode is explicit
Easy to allow O_TMPFILE|O_WRONLY in the future (or now?) Static code checkers complaining about the lack of O_{RDONLY,RDWR,WRONLY} in open() (?)
Cons:
We catch one fewer case on older kernels, but not one which is likely to occur in real programs: On old kernels, O_TMPFILE|O_RDONLY, or equivalently O_TMPFILE, may give a valid file descriptor (in the case where path does name a directory). However: Anyone writing a program using O_TMPFILE probably at least tests on a kernel knowing about that, and so would be given the -EINVAL error, and the documentation can just spell out that O_TMPFILE is not compatible with O_RDONLY.
Rasmus