vfs: don't bother clearing close_on_exec bit for unused fds
RFC 1/1 vfs: don't bother clearing close_on_exec bit for unused fds
In fc90888d07b8 (vfs: conditionally clear close-on-exec flag) a conditional was added to __clear_close_on_exec to avoid dirtying a cache line in the common case where the bit is already clear. However, AFAICT, we don't rely on the close_on_exec bit being clear for unused fds, except as an optimization in do_close_on_exec(); if I haven't missed anything, __{set,clear}_close_on_exec is always called when a new fd is allocated. At the expense of also reading through ->open_fds in do_close_on_exec(), we can avoid accessing the close_on_exec bitmap altogether in close(), which I think is a reasonable trade-off.
The conditional added in the commit above still makes sense to avoid the dirtying on the allocation paths, but I also think it might make sense in __set_close_on_exec: I suppose any given app handling a non-trivial amount of fds uses O_CLOEXEC for either almost none or almost all of them.
---
I'm sure I've missed something, hence the RFC. But if not, there's probably also a few memsets which become redundant. And the __set_close_on_exec part should probably be its own patch...
fs/file.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
fs/file.c +3 -3
@@ -231,7 +231,8 @@ repeat:static inline void __set_close_on_exec (int fd,struct fdtable *fdt) { -__set_bit (fd, fdt->close_on_exec); +if (!test_bit (fd, fdt->close_on_exec)) +__set_bit (fd, fdt->close_on_exec); }static inline void __clear_close_on_exec (int fd,struct fdtable *fdt)
@@ -644,7 +645,6 @@ int __close_fd(struct files_struct *files, unsigned fd)if (!file) goto out_unlock;rcu_assign_pointer (fdt->fd[fd],NULL ); -__clear_close_on_exec (fd, fdt);__put_unused_fd (files, fd);spin_unlock (&files->file_lock);return filp_close (file, files);
@@ -667,7 +667,7 @@ void do_close_on_exec(struct files_struct *files) fdt =files_fdtable (files);if (fd >= fdt->max_fds)break ; - set = fdt->close_on_exec[i]; + set = fdt->close_on_exec[i] & fdt->open_fds[i];if (!set)continue ; fdt->close_on_exec[i] =0 ;
Signature · 1 line
2.6.1
Quoted 4 lines · Rasmus Villemoes
I'm sure I've missed something, hence the RFC. But if not, there's probably also a few memsets which become redundant. And the
__set_close_on_execpart should probably be its own patch...
The patch looks fine to me. I'm not sure the __set_close_on_exec part even makes sense, because if you set that bit, it usually really *is* clear before, so testing it beforehand is just pointless. And if somebody really keeps setting the bit, they are doing something stupid anyway..
So I have nothing against the patch, but I do wonder how much it matters. If there isn't a noticeable performance win, I'd almost rather just keep the close-on-exec bitmap up-to-date. Hmm?
Linus
Quoted 7 lines · Linus Torvalds
Quoted 4 lines · Rasmus Villemoes
I'm sure I've missed something, hence the RFC. But if not, there's probably also a few memsets which become redundant. And the
__set_close_on_execpart should probably be its own patch...
Quoted 5 lines
The patch looks fine to me. I'm not sure the
__set_close_on_execpart even makes sense, because if you set that bit, it usually really *is* clear before, so testing it beforehand is just pointless. And if somebody really keeps setting the bit, they are doing something stupid anyway..
So that's true for the lifetime of a single fd where no-one of course does fcntl(fd, FD_CLOEXEC) more than once. But the scenario I was thinking of was when fds get recycled. open(, O_CLOEXEC) => 5, close(5), open(, O_CLOEXEC) => 5; in that case, letting the close_on_exec bit keep its value avoids dirtying the cache line on all subsequent allocations of fd 5 (for example, had Eric's app been using *_CLOEXEC for all its open's, socket's etc. there wouldn't have been any gain by adding the conditional to __clear_close_on_exec, but I'd expect to see a similar gain by doing the symmetric thing). Again, this is assuming that almost all fd allocations either do or do not apply CLOEXEC - after a while, ->close_on_exec would reach a steady-state where no bits get flipped anymore.
The "usually really *is* clear" only holds when we do "bother clearing close_on_exec bit for unused fds", which is what I suggest we don't :-)
I don't think either state of the bit in close_on_exec is more or less 'up-to-date' when its buddy in open_fds is not set.
Rasmus
fs/file.c void do_close_on_exec(struct files_struct *files) · line 673
Quoted 9 lines · Rasmus Villemoes
fs/file.c +1
-1@@ -667,7 +667,7 @@ void do_close_on_exec(struct files_struct *files) fdt =files_fdtable (files);if (fd >= fdt->max_fds)break ; - set = fdt->close_on_exec[i]; + set = fdt->close_on_exec[i] & fdt->open_fds[i];if (!set)continue ; fdt->close_on_exec[i] =0 ;
If you don't bother, why leaving this final fdt->close_on_exec[i] = 0 ?
Quoted 13 lines · Eric Dumazet
Review void do_close_on_exec(struct files_struct *files) · line 673
Quoted 9 lines · Rasmus Villemoes
(quoted diff) +1
-1@@ -667,7 +667,7 @@ void do_close_on_exec(struct files_struct *files) fdt = files_fdtable(files); if (fd >= fdt->max_fds) break; - set = fdt->close_on_exec[i]; + set = fdt->close_on_exec[i] & fdt->open_fds[i]; if (!set) continue; fdt->close_on_exec[i] = 0;If you don't bother, why leaving this final fdt->close_on_exec[i] = 0 ?
Thanks, it should go, along with the mentioned memsets. Updated patch below.
Reading dup_fd() I'm even more convinced that we're not relying on any particular value for close_on_exec bits for unused fds. After
/*
* The fd may be claimed in the fd bitmap but not yet
* instantiated in the files array if a sibling thread
* is partway through open(). So make sure that this
* fd is available to the new process.
*/
we only __clear_open_fd(), so the close_on_exec bit may be left set in the new process.
From: Rasmus Villemoes <[email protected]>
Date: Tue, 3 Nov 2015 09:43:53 +0100
Subject: [PATCH] vfs: don't bother clearing close_on_exec bit for unused fds
In fc90888d07b8 (vfs: conditionally clear close-on-exec flag) a conditional was added to __clear_close_on_exec to avoid dirtying a cache line in the common case where the bit is already clear. However, AFAICT, we don't rely on the close_on_exec bit being clear for unused fds, except as an optimization in do_close_on_exec(); if I haven't missed anything, __{set,clear}_close_on_exec is always called when a new fd is allocated. At the expense of also reading through ->open_fds in do_close_on_exec(), we can avoid accessing the close_on_exec bitmap altogether in close(), which I think is a reasonable trade-off.
The conditional added in the commit above still makes sense to avoid the dirtying on the allocation paths, but I also think it might make sense in __set_close_on_exec: I suppose any given app handling a non-trivial amount of fds uses O_CLOEXEC for either almost none or almost all of them, so after a while one would reach a sort of steady-state where bits in ->close_on_exec are almost never flipped.
--- fs/file.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-)
fs/file.c +3 -6
@@ -79,7 +79,6 @@ static void copy_fdtable(struct fdtable *nfdt, struct fdtable *ofdt)memcpy (nfdt->open_fds, ofdt->open_fds, cpy);memset ((char *)(nfdt->open_fds) + cpy,0 , set);memcpy (nfdt->close_on_exec, ofdt->close_on_exec, cpy); -memset ((char *)(nfdt->close_on_exec) + cpy,0 , set); cpy =BITBIT_SIZE (ofdt->max_fds); set =BITBIT_SIZE (nfdt->max_fds) - cpy;
@@ -231,7 +230,8 @@ repeat:static inline void __set_close_on_exec (int fd,struct fdtable *fdt) { -__set_bit (fd, fdt->close_on_exec); +if (!test_bit (fd, fdt->close_on_exec)) +__set_bit (fd, fdt->close_on_exec); }static inline void __clear_close_on_exec (int fd,struct fdtable *fdt)
@@ -369,7 +369,6 @@ struct files_struct *dup_fd(struct files_struct *oldf, int *errorp)int start = open_files /BITS_PER_LONG ;memset (&new_fdt->open_fds[start],0 , left); -memset (&new_fdt->close_on_exec[start],0 , left); }rcu_assign_pointer (newf->fdt, new_fdt);
@@ -644,7 +643,6 @@ int __close_fd(struct files_struct *files, unsigned fd)if (!file) goto out_unlock;rcu_assign_pointer (fdt->fd[fd],NULL ); -__clear_close_on_exec (fd, fdt);__put_unused_fd (files, fd);spin_unlock (&files->file_lock);return filp_close (file, files);
@@ -667,10 +665,9 @@ void do_close_on_exec(struct files_struct *files) fdt =files_fdtable (files);if (fd >= fdt->max_fds)break ; - set = fdt->close_on_exec[i]; + set = fdt->close_on_exec[i] & fdt->open_fds[i];if (!set)continue ; - fdt->close_on_exec[i] =0 ;for ( ; set ; fd++, set >>=1 ) {struct file *file;if (!(set &1 ))
Signature · 1 line
2.6.1
fs/file.c void do_close_on_exec(struct files_struct *files) · line 670
Quoted 8 lines · Rasmus Villemoes
fs/file.c +1
-1@@ -667,10 +665,9 @@ void do_close_on_exec(struct files_struct *files) fdt =files_fdtable (files);if (fd >= fdt->max_fds)break ; - set = fdt->close_on_exec[i]; + set = fdt->close_on_exec[i] & fdt->open_fds[i];if (!set)continue ;
Many processes have a big hole at the end of fdt->open_fds[], due to the fact that max_fds is rounded to a power of two.
It makes sense to avoid bringing in cpu caches the close_on_exec[] part.
set = fdt->open_fds[i];
if (!set)
continue; set &= fdt->close_on_exec[i]; if (!set) continue;
Not sure if this is a net win due to branch prediction...