osdep: add a qemu_close_all_open_fd() helper
PATCH v5 1/1 osdep: add a qemu_close_all_open_fd() helper
Since commit 03e471c41d8b ("qemu_init: increase NOFILE soft limit on POSIX"), the maximum number of file descriptors that can be opened are raised to nofile.rlim_max. On recent debian distro, this yield a maximum of 1073741816 file descriptors. Now, when forking to start qemu-bridge-helper, this actually calls close() on the full possible file descriptor range (more precisely [3 - sysconf(_SC_OPEN_MAX)]) which takes a considerable amount of time. In order to reduce that time, factorize existing code to close all open files descriptors in a new qemu_close_all_open_fd() function. This function uses various methods to close all the open file descriptors ranging from the most efficient one to the least one. It also accepts an ordered array of file descriptors that should not be closed since this is required by the callers that calls it after forking. Since this function is not used for Win32, do not implement it to force an error at link time if used.
---
v5:
- Move qemu_close_all_open_fd() to oslib-posix.c since it does not compile on windows and is not even used on it.
- v4: https://lore.kernel.org/qemu-devel/[email protected]/
v4:
- Add a comment saying that qemu_close_all_open_fd() can take a NULL skip array and nskip == 0
- Added an assert in qemu_close_all_open_fd() to check for skip/nskip parameters
- Fix spurious tabs instead of spaces
- Applied checkpatch
- v3: https://lore.kernel.org/qemu-devel/[email protected]/
v3:
- Use STD*_FILENO defines instead of raw values
- Fix indentation of close_all_fds_after_fork()
- Check for nksip in fallback code
- Check for path starting with a '.' in qemu_close_all_open_fd_proc()
- Use unsigned for cur_skip
- Move ifdefs inside close_fds functions rather than redefining them
- Remove uneeded 'if(nskip)' test
- Add comments to close_range version
- Reduce range of skip fd as we find them in
- v2: https://lore.kernel.org/qemu-devel/[email protected]/
v2:
- Factorize async_teardown.c close_fds implementation as well as tap.c ones
- Apply checkpatch
- v1: https://lore.kernel.org/qemu-devel/[email protected]/
--- include/qemu/osdep.h | 9 +++ net/tap.c | 33 +++++----- system/async-teardown.c | 37 +----------- util/oslib-posix.c | 131 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 160 insertions(+), 50 deletions(-)
include/qemu/osdep.h +9 -0
@@ -757,6 +757,15 @@ static inline void qemu_reset_optind(void)int qemu_fdatasync (int fd); +/** +* Close all open file descriptors except the ones supplied in the @skip array +* +* @skip: ordered array of distinct file descriptors that should not be closed +* if any, or NULL. +* @nskip: number of entries in the @skip array or 0 if @skip is NULL. +*/ +void qemu_close_all_open_fd (const int *skip,unsigned int nskip); +/** * Sync changes made to the memory mapped file back to the backing * storage. For POSIX compliant systems this will fallback
net/tap.c +19 -14
@@ -385,6 +385,23 @@ static TAPState *net_tap_fd_init(NetClientState *peer,return s; } +static void close_all_fds_after_fork (int excluded_fd) +{ +const int skip_fd[] = {STDIN_FILENO ,STDOUT_FILENO ,STDERR_FILENO , + excluded_fd}; +unsigned int nskip =ARRAY_SIZE (skip_fd); + +/* +* skip_fd must be an ordered array of distinct fds, exclude +* excluded_fd if already included in the [STDIN_FILENO - STDERR_FILENO] +* range +*/ +if (excluded_fd <=STDERR_FILENO ) { + nskip--; + } +qemu_close_all_open_fd (skip_fd, nskip); +} +static void launch_script (const char *setup_script,const char *ifname,int fd,Error **errp) {
@@ -400,13 +417,7 @@ static void launch_script(const char *setup_script, const char *ifname,return ; }if (pid ==0 ) { -int open_max =sysconf (_SC_OPEN_MAX), i; - -for (i =3 ; i < open_max; i++) { -if (i != fd) { -close (i); - } - } +close_all_fds_after_fork (fd); parg = args; *parg++ = (char *)setup_script; *parg++ = (char *)ifname;
@@ -490,17 +501,11 @@ static int net_bridge_run_helper(const char *helper, const char *bridge,return -1 ; }if (pid ==0 ) { -int open_max =sysconf (_SC_OPEN_MAX), i;char *fd_buf =NULL ;char *br_buf =NULL ;char *helper_cmd =NULL ; -for (i =3 ; i < open_max; i++) { -if (i != sv[1 ]) { -close (i); - } - } - +close_all_fds_after_fork (sv[1 ]); fd_buf =g_strdup_printf ("%s%d" ,"--fd=" , sv[1 ]);if (strrchr (helper,' ' ) ||strrchr (helper,'\t' )) {
system/async-teardown.c +1 -36
@@ -26,40 +26,6 @@static pid_t the_ppid; -/* -* Close all open file descriptors. -*/ -static void close_all_open_fd (void ) -{ -struct dirent *de; -int fd, dfd; -DIR *dir; - -#ifdef CONFIG_CLOSE_RANGE -int r =close_range (0 , ~0U ,0 ); -if (!r) { -/* Success, no need to try other ways. */ -return ; - } -#endif - - dir =opendir ("/proc/self/fd" ); -if (!dir) { -/* If /proc is not mounted, there is nothing that can be done. */ -return ; - } -/* Avoid closing the directory. */ - dfd =dirfd (dir); - -for (de =readdir (dir); de; de =readdir (dir)) { - fd =atoi (de->d_name); -if (fd != dfd) { -close (fd); - } - } -closedir (dir); -} -static void hup_handler (int signal) {/* Check every second if this process has been reparented. */
@@ -85,9 +51,8 @@ static int async_teardown_fn(void *arg)/* * Close all file descriptors that might have been inherited from the * main qemu process when doing clone, needed to make libvirt happy. -* Not using close_range for increased compatibility with older kernels. */ -close_all_open_fd (); +qemu_close_all_open_fd (NULL ,0 );/* Set up a handler for SIGHUP and unblock SIGHUP. */ sigaction (SIGHUP , &sa,NULL );
util/oslib-posix.c +131 -0
134 lines not on this page · raw message
Signature · 1 line
2.45.2
Hi Clément,
Quoted 49 lines · Clément Léger
Since commit
03e471c41d8b("qemu_init: increase NOFILE soft limit on POSIX"), the maximum number of file descriptors that can be opened are raised to nofile.rlim_max. On recent debian distro, this yield a maximum of 1073741816 file descriptors. Now, when forking to start qemu-bridge-helper, this actually callsclose()on the full possible file descriptor range (more precisely [3 -sysconf(_SC_OPEN_MAX)]) which takes a considerable amount of time. In order to reduce that time, factorize existing code to close all open files descriptors in a newqemu_close_all_open_fd()function. This function uses various methods to close all the open file descriptors ranging from the most efficient one to the least one. It also accepts an ordered array of file descriptors that should not be closed since this is required by the callers that calls it after forking. Since this function is not used for Win32, do not implement it to force an error at link time if used.Signed-off-by Clément Léger <[email protected]>Reviewed-by Richard Henderson <[email protected]>---
v5:
- Moveqemu_close_all_open_fd()tooslib-posix.csince it does not compile on windows and is not even used on it.
- v4: https://lore.kernel.org/qemu-devel/[email protected]/v4:
- Add a comment saying thatqemu_close_all_open_fd()can take a NULL skiparray and nskip == 0- Added an assert in
qemu_close_all_open_fd()to check for skip/nskip parameters
- Fix spurious tabs instead of spaces
- Applied checkpatch
- v3: https://lore.kernel.org/qemu-devel/[email protected]/v3:
- Use STD*_FILENO defines instead of raw values
- Fix indentation ofclose_all_fds_after_fork()
- Check for nksip in fallback code
- Check for path starting with a '.' inqemu_close_all_open_fd_proc()
- Use unsigned forcur_skip
- Move ifdefs insideclose_fdsfunctions rather than redefining them
- Remove uneeded 'if(nskip)' test
- Add comments toclose_rangeversion
- Reduce range of skip fd as we find them in
- v2: https://lore.kernel.org/qemu-devel/[email protected]/v2:
- Factorizeasync_teardown.cclose_fdsimplementation as well astap.cones
- Apply checkpatch
- v1: https://lore.kernel.org/qemu-devel/[email protected]/
Quoted 6 lines
--- include/qemu/osdep.h | 9 +++ net/tap.c | 33 +++++----- system/async-teardown.c | 37 +----------- util/oslib-posix.c | 131 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 160 insertions(+), 50 deletions(-)
I'm getting this error on msys2, not sure if related:
WARNING: Failed to terminate process: 1 error occurred:
* failed to attach the runner process to the console of its parent
process: The handle is invalid.
I find your patch hard to review. Do you mind splitting as trivial changes? Something like:
- Expose close_all_open_fd() renamed as qemu_close_all_open_fd()
- Rework qemu_close_all_open_fd()
- Factor close_all_fds_after_fork() in net/tap.c
- Use qemu_close_all_open_fd() in net/tap.c
Thanks,
Phil.
Quoted 71 lines · Philippe Mathieu-Daudé
Hi Clément,
Quoted 55 lines · Clément Léger
Since commit
03e471c41d8b("qemu_init: increase NOFILE soft limit on POSIX"), the maximum number of file descriptors that can be opened are raised to nofile.rlim_max. On recent debian distro, this yield a maximum of 1073741816 file descriptors. Now, when forking to start qemu-bridge-helper, this actually callsclose()on the full possible file descriptor range (more precisely [3 -sysconf(_SC_OPEN_MAX)]) which takes a considerable amount of time. In order to reduce that time, factorize existing code to close all open files descriptors in a newqemu_close_all_open_fd()function. This function uses various methods to close all the open file descriptors ranging from the most efficient one to the least one. It also accepts an ordered array of file descriptors that should not be closed since this is required by the callers that calls it after forking. Since this function is not used for Win32, do not implement it to force an error at link time if used.Signed-off-by Clément Léger <[email protected]>Reviewed-by Richard Henderson <[email protected]>---
v5:
- Moveqemu_close_all_open_fd()tooslib-posix.csince it does not compile on windows and is not even used on it.
- v4:
https://lore.kernel.org/qemu-devel/[email protected]/v4:
- Add a comment saying thatqemu_close_all_open_fd()can take a NULL skip
array and nskip == 0
- Added an assert inqemu_close_all_open_fd()to check for skip/nskip parameters
- Fix spurious tabs instead of spaces
- Applied checkpatch
- v3:
https://lore.kernel.org/qemu-devel/[email protected]/v3:
- Use STD*_FILENO defines instead of raw values
- Fix indentation ofclose_all_fds_after_fork()
- Check for nksip in fallback code
- Check for path starting with a '.' inqemu_close_all_open_fd_proc()
- Use unsigned forcur_skip
- Move ifdefs insideclose_fdsfunctions rather than redefining them
- Remove uneeded 'if(nskip)' test
- Add comments toclose_rangeversion
- Reduce range of skip fd as we find them in
- v2:
https://lore.kernel.org/qemu-devel/[email protected]/v2:
- Factorizeasync_teardown.cclose_fdsimplementation as well astap.cones
- Apply checkpatch
- v1:
https://lore.kernel.org/qemu-devel/[email protected]/Quoted 6 lines
---
include/qemu/osdep.h | 9 +++
net/tap.c | 33 +++++-----system/async-teardown.c| 37 +----------- util/oslib-posix.c | 131 ++++++++++++++++++++++++++++++++++++++++4 files changed, 160 insertions(+), 50 deletions(-)I'm getting this error on msys2, not sure if related:
WARNING: Failed to terminate process: 1 error occurred:
* failed to attach the runner process to the console of its parent
process: The handle is invalid.
Quoted 7 lines
I find your patch hard to review. Do you mind splitting as trivial changes? Something like:
- Expose
close_all_open_fd()renamed asqemu_close_all_open_fd()
- Reworkqemu_close_all_open_fd()
- Factorclose_all_fds_after_fork()innet/tap.c
- Useqemu_close_all_open_fd()innet/tap.c
Yes sure, I'll do that.
Clément
Quoted 4 lines
Thanks,
Phil.
Quoted 77 lines · Clément Léger
Quoted 74 lines · Philippe Mathieu-Daudé
Hi Clément,
Quoted 55 lines · Clément Léger
Since commit
03e471c41d8b("qemu_init: increase NOFILE soft limit on POSIX"), the maximum number of file descriptors that can be opened are raised to nofile.rlim_max. On recent debian distro, this yield a maximum of 1073741816 file descriptors. Now, when forking to start qemu-bridge-helper, this actually callsclose()on the full possible file descriptor range (more precisely [3 -sysconf(_SC_OPEN_MAX)]) which takes a considerable amount of time. In order to reduce that time, factorize existing code to close all open files descriptors in a newqemu_close_all_open_fd()function. This function uses various methods to close all the open file descriptors ranging from the most efficient one to the least one. It also accepts an ordered array of file descriptors that should not be closed since this is required by the callers that calls it after forking. Since this function is not used for Win32, do not implement it to force an error at link time if used.Signed-off-by Clément Léger <[email protected]>Reviewed-by Richard Henderson <[email protected]>---
v5:
- Moveqemu_close_all_open_fd()tooslib-posix.csince it does not compile on windows and is not even used on it.
- v4:
https://lore.kernel.org/qemu-devel/[email protected]/v4:
- Add a comment saying thatqemu_close_all_open_fd()can take a NULL
skip
array and nskip == 0
- Added an assert inqemu_close_all_open_fd()to check for skip/nskip parameters
- Fix spurious tabs instead of spaces
- Applied checkpatch
- v3:
https://lore.kernel.org/qemu-devel/[email protected]/v3:
- Use STD*_FILENO defines instead of raw values
- Fix indentation ofclose_all_fds_after_fork()
- Check for nksip in fallback code
- Check for path starting with a '.' inqemu_close_all_open_fd_proc()
- Use unsigned forcur_skip
- Move ifdefs insideclose_fdsfunctions rather than redefining them
- Remove uneeded 'if(nskip)' test
- Add comments toclose_rangeversion
- Reduce range of skip fd as we find them in
- v2:
https://lore.kernel.org/qemu-devel/[email protected]/v2:
- Factorizeasync_teardown.cclose_fdsimplementation as well astap.cones
- Apply checkpatch
- v1:
https://lore.kernel.org/qemu-devel/[email protected]/Quoted 6 lines
--- include/qemu/osdep.h | 9 +++ net/tap.c | 33 +++++----- system/async-teardown.c | 37 +----------- util/oslib-posix.c | 131 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 160 insertions(+), 50 deletions(-)I'm getting this error on msys2, not sure if related:
WARNING: Failed to terminate process: 1 error occurred:
* failed to attach the runner process to the console of its parent
process: The handle is invalid.I find your patch hard to review. Do you mind splitting as trivial changes? Something like:
Quoted 6 lines
Quoted 4 lines
- Expose
close_all_open_fd()renamed asqemu_close_all_open_fd()
- Reworkqemu_close_all_open_fd()
- Factorclose_all_fds_after_fork()innet/tap.c
- Useqemu_close_all_open_fd()innet/tap.cYes sure, I'll do that.
If you're making updates, I think you should drop the linux ifdef for /proc/self/fd. This is also present on Solaris. Importantly, it'll compile on all POSIX systems, whether or not /proc is available.
r~
Quoted 88 lines · Richard Henderson
Quoted 87 lines · Clément Léger
Quoted 83 lines · Philippe Mathieu-Daudé
Hi Clément,
Quoted 59 lines · Clément Léger
Since commit
03e471c41d8b("qemu_init: increase NOFILE soft limit on POSIX"), the maximum number of file descriptors that can be opened are raised to nofile.rlim_max. On recent debian distro, this yield a maximum
of 1073741816 file descriptors. Now, when forking to start qemu-bridge-helper, this actually callsclose()on the full possible file
descriptor range (more precisely [3 -sysconf(_SC_OPEN_MAX)]) which takes a considerable amount of time. In order to reduce that time, factorize existing code to close all open files descriptors in a newqemu_close_all_open_fd()function. This function uses various methods to close all the open file descriptors ranging from the most efficient one to the least one. It also accepts an ordered array of file descriptors that should not be closed since this is required by the callers that calls it after forking. Since this function is not used for Win32, do not implement it to force an error at link time if used.Signed-off-by Clément Léger <[email protected]>Reviewed-by Richard Henderson <[email protected]>---
v5:
- Moveqemu_close_all_open_fd()tooslib-posix.csince it does not compile on windows and is not even used on it.
- v4:
https://lore.kernel.org/qemu-devel/[email protected]/v4:
- Add a comment saying thatqemu_close_all_open_fd()can take a NULL skip
array and nskip == 0
- Added an assert inqemu_close_all_open_fd()to check for skip/nskip
parameters
- Fix spurious tabs instead of spaces
- Applied checkpatch
- v3:
https://lore.kernel.org/qemu-devel/[email protected]/v3:
- Use STD*_FILENO defines instead of raw values
- Fix indentation ofclose_all_fds_after_fork()
- Check for nksip in fallback code
- Check for path starting with a '.' inqemu_close_all_open_fd_proc()
- Use unsigned forcur_skip
- Move ifdefs insideclose_fdsfunctions rather than redefining them
- Remove uneeded 'if(nskip)' test
- Add comments toclose_rangeversion
- Reduce range of skip fd as we find them in
- v2:
https://lore.kernel.org/qemu-devel/[email protected]/v2:
- Factorizeasync_teardown.cclose_fdsimplementation as well astap.cones
- Apply checkpatch
- v1:
https://lore.kernel.org/qemu-devel/[email protected]/Quoted 7 lines
---
include/qemu/osdep.h | 9 +++
net/tap.c | 33 +++++-----
system/async-teardown.c| 37 +-----------
util/oslib-posix.c | 131
++++++++++++++++++++++++++++++++++++++++4 files changed, 160 insertions(+), 50 deletions(-)I'm getting this error on msys2, not sure if related:
WARNING: Failed to terminate process: 1 error occurred:
* failed to attach the runner process to the console of its parent
process: The handle is invalid.I find your patch hard to review. Do you mind splitting as trivial changes? Something like:
- Expose
close_all_open_fd()renamed asqemu_close_all_open_fd()
- Reworkqemu_close_all_open_fd()
- Factorclose_all_fds_after_fork()innet/tap.c
- Useqemu_close_all_open_fd()innet/tap.c
Quoted 5 lines
Quoted 1 line
Yes sure, I'll do that.
If you're making updates, I think you should drop the linux ifdef for /proc/self/fd. This is also present on Solaris. Importantly, it'll compile on all POSIX systems, whether or not /proc is available.
Acked
Thanks,
Clément
Quoted 3 lines
r~