vfork(2) fails after unshare(CLONE_NEWTIME) (was: [Bug 215769] man 2 vfork() does not document corner case when PID == 1)

https://bugzilla.kernel.org/show_bug.cgi?id=215769

--- Comment #3 from Коренберг Марк ([email protected]) ---
Hi,
I appreciate depth of information validation. Actually, you are right. vfork() DOES work with pid=1 processes. I figured out the cause in my case. In order to reproduce -- add unshare(CLONE_NEWTIME) just before vfork(). Now, I don't know if it's a bug in vfork() or in fork(). Yes, both are clone() actually.

In any case, they should either both give EINVAL or both don't fail. But it's definitely bug in the kernel around CLONE_NEWTIME.

https://bugzilla.kernel.org/show_bug.cgi?id=215769

--- Comment #4 from Коренберг Марк ([email protected]) --- #define _GNU_SOURCE 1
#include <stdio.h>
#include <sched.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <err.h>

#ifndef CLONE_NEWTIME
#define CLONE_NEWTIME 0x00000080
#endif

int main (void)
{
if (unshare (CLONE_NEWTIME)) err (EXIT_FAILURE, "UNSHARE_NEWTIME");

pid_t pid;
switch (pid=vfork ())
{
case 0:

    _exit(0);
  case -1:
    err(EXIT_FAILURE, "vfork BUG");
  default:
    waitpid(pid, NULL, 0);

}
return 0;
}

[Added some kernel CCs that may know what's going on]

Hi,

Quoted 12 lines · On 3/31/22 09:53, [email protected] wrote:

https://bugzilla.kernel.org/show_bug.cgi?id=215769

--- Comment #3 from Коренберг Марк ([email protected]) ---
Hi,
I appreciate depth of information validation. Actually, you are right. vfork() DOES work with pid=1 processes. I figured out the cause in my case. In order to reproduce -- add unshare(CLONE_NEWTIME) just before vfork(). Now, I don't know if it's a bug in vfork() or in fork(). Yes, both are clone() actually.

In any case, they should either both give EINVAL or both don't fail. But it's definitely bug in the kernel around CLONE_NEWTIME.

Quoted 24 lines · On 3/31/22 10:12, [email protected] wrote:

https://bugzilla.kernel.org/show_bug.cgi?id=215769

--- Comment #4 from Коренберг Марк ([email protected]) --- #define _GNU_SOURCE 1
#include <stdio.h>
#include <sched.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <err.h>

#ifndef CLONE_NEWTIME
#define CLONE_NEWTIME 0x00000080
#endif

int main (void)
{
if (unshare (CLONE_NEWTIME)) err (EXIT_FAILURE, "UNSHARE_NEWTIME");

pid_t pid;
switch (pid=vfork ())
{
case 0:

Quoted 9 lines

    _exit(0);
  case -1:
    err(EXIT_FAILURE, "vfork BUG");
  default:
    waitpid(pid, NULL, 0);

}
return 0;
}

I could reproduce it with the following code. I tried syscall(SYS_vfork) to make sure it's not a problem in the libc wrapper, and to make sure I do call vfork(2). If I replace vfork(2) with fork(2), I don't get the error.

$ cat vfork.c
#define _GNU_SOURCE
#include <err.h>
#include <linux/sched.h>
#include <sched.h>
#include <signal.h>
#include <stdlib.h>
#include <sys/syscall.h>
#include <unistd.h>

int main(void)

{
	pid_t pid;

	if (unshare(CLONE_NEWTIME) == -1)
		err(EXIT_FAILURE, "unshare(2)");
	if (signal(SIGCHLD, SIG_IGN) == SIG_ERR)
		err(EXIT_FAILURE, "sigaction(2)");
	pid = syscall(SYS_vfork);
	switch (pid) {
	case 0:
		errx(EXIT_SUCCESS, "Grandchild exiting normally.");
	case -1:
		/* If we got here, the report is confirmed. */
		err(EXIT_FAILURE, "vfork(2)");
	default:
		errx(EXIT_SUCCESS, "Child exiting normally.");
	}
}

$ cc -Wall -Wextra -Werror vfork.c
$ sudo ./a.out
a.out: vfork(2): Invalid argument

$ grep_syscall_def vfork
kernel/fork.c:2711:
SYSCALL_DEFINE0(vfork)
{
	struct kernel_clone_args args = {
		.flags		= CLONE_VFORK | CLONE_VM,
		.exit_signal	= SIGCHLD,
	};

	return kernel_clone(&args);
}

Maybe someone in the kernel can send some patch for the clone(2) and/or vfork(2) manual pages that explains the reason (if it's intended).

Quoted 1 line · Alejandro Colomar (man-pages)

[Added some kernel CCs that may know what's going on]

Quoted 2 lines

Maybe someone in the kernel can send some patch for the clone(2) and/or vfork(2) manual pages that explains the reason (if it's intended).

Hey Alejandro,

I won't be able to send a patch very soon but I can at least explain why you see EINVAL. :)

This is intended.

vfork() suspends the parent process and the child process will share the same vm as the parent process. If the child process is in a new time namespace different from its parent process it is not allowed to be in the same threadgroup or share virtual memory with the parent process. That's why you see EINVAL.

Note, the unshare(CLONE_NEWTIME) call will _not_ cause the calling process to be moved into a different time namespace. Only the newly created child process will be after a subsequent fork()/vfork()/clone()/clone3()...

The semantics are equivalent to that of CLONE_NEWPID in this regard. You can see this via /proc/<pid>/ns/ where you see two entries for pid namespaces and also two entries for time namespaces:

* CLONE_NEWTIME
* /proc/<pid>/ns/time // current time namespace
* /proc/<pid>/ns/time_for_children // time namespace for the new child process

If during fork:

parent_process->time != parent_process->time_for_children

and either CLONE_VM or CLONE_THREAD is set you see EINVAL.

You can thus replicate the same error via:

unshare(CLONE_NEWTIME)

and a

clone() or clone3() call with CLONE_VM or CLONE_THREAD.

Hey, Christian!

Quoted 2 lines · Christian Brauner

Quoted 1 line · Alejandro Colomar (man-pages)

[Added some kernel CCs that may know what's going on]

[...]

Quoted 7 lines

Quoted 2 lines

Maybe someone in the kernel can send some patch for the clone(2) and/or vfork(2) manual pages that explains the reason (if it's intended).

Hey Alejandro,

I won't be able to send a patch very soon but I can at least explain why you see EINVAL. :)

Don't hurry, we're not planning to release any soon :)

Quoted 8 lines

This is intended.

vfork() suspends the parent process and the child process will share the same vm as the parent process. If the child process is in a new time namespace different from its parent process it is not allowed to be in the same threadgroup or share virtual memory with the parent process. That's why you see EINVAL.

That makes a lot of sense to me.

Quoted 6 lines

Note, the unshare(CLONE_NEWTIME) call will _not_ cause the calling process to be moved into a different time namespace. Only the newly created child process will be after a subsequent fork()/vfork()/clone()/clone3()...

Quoted 7 lines

The semantics are equivalent to that of CLONE_NEWPID in this regard. You can see this via /proc/<pid>/ns/ where you see two entries for pid namespaces and also two entries for time namespaces:

* CLONE_NEWTIME
* /proc/<pid>/ns/time // current time namespace
* /proc/<pid>/ns/time_for_children // time namespace for the new child process

Also makes sense. Michael taught me that a few weeks ago :)

This also triggers some doubt: will the same problem happen with CLONE_NEWPID since it also moves the child into a new ns (in this case a PID one)? See test program below.

Quoted 8 lines

If during fork:

parent_process->time != parent_process->time_for_children

and either CLONE_VM or CLONE_THREAD is set you see EINVAL.

Quoted 7 lines

You can thus replicate the same error via:

unshare(CLONE_NEWTIME)

and a

clone() or clone3() call with CLONE_VM or CLONE_THREAD.

So, to test my doubts, I wrote this similar program (and also similar programs where only the CLONE_NEW* flag was changed, one with CLONE_NEWTIME, and one with CLONE_NEWNS)):

$ cat vfork_newpid.c
#define _GNU_SOURCE
#include <err.h>
#include <errno.h>
#include <linux/sched.h>
#include <sched.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/syscall.h>
#include <unistd.h>

static char *const child_argv[] = {
	"print_pid",
	NULL
};

static char *const child_envp[] = {
NULL
};

int
main(void)

{
	pid_t pid;

	printf("%s: PID: %ld\n", program_invocation_short_name, (long) getpid());

	if (unshare(CLONE_NEWPID) == -1)
		err(EXIT_FAILURE, "unshare(2)");
	if (signal(SIGCHLD, SIG_IGN) == SIG_ERR)
		err(EXIT_FAILURE, "signal(2)");

	pid = syscall(SYS_vfork);
	//pid = vfork();  // This behaves differently.
	switch (pid) {
	case 0:
		execve("/home/alx/tmp/print_pid", child_argv, child_envp);
		err(EXIT_SUCCESS, "PID %jd exiting after execve(2)",
		    (long) getpid());
	case -1:
		err(EXIT_FAILURE, "vfork(2)");
	default:
		errx(EXIT_SUCCESS, "Parent exiting after vfork(2).");
	}
}

$ cat print_pid.c
#include <err.h>
#include <stdlib.h>
#include <unistd.h>

int
main(void)

{
	errx(EXIT_SUCCESS, "PID %jd exiting.", (long) getpid());
}

$ cc -Wall -Wextra -Werror -o print_pid print_pid.c
$ cc -Wall -Wextra -Werror -o vfork_newpid vfork_newpid.c
$
$
$ sudo ./vfork_newpid
vfork_newpid: PID: 8479
vfork_newpid: PID 8479 exiting after execve(2): Success
print_pid: PID 1 exiting.
$
$
$ sudo ./vfork_newtime
vfork_newtime: PID: 8484
vfork_newtime: vfork(2): Invalid argument
$
$
$ sudo ./vfork_newns
vfork_newns: PID: 8486
vfork_newns: PID 8486 exiting after execve(2): Success
print_pid: PID 8487 exiting.

The first thing I noted is that usage of vfork(2) differs considerably from fork(2), and that's something that's not clear by reading the manual page. It sais that the parent process is suspended until the child calls execve(2), but I expected it to mean that vfork(2) doesn't return to the parent until that happened, but was otherwise transparent. I was wrong and my tests showed me that.

I was going to propose an example program for the manual page, when I decided to try a slightly different thing: call vfork() instead of syscall(SYS_vfork); that changed the behavior to the same one as with fork(2) (i.e., the parent resumes after vfork(2) returns the PID of the child.

Is that also intended? I couldn't find the glibc wrapper source code, so I don't know what is glibc doing here, but I straced the processes, and they're all calling vfork(), so the behavior should be consistent; it's quite weird. I'm very confused at this point.

I'm also wondering why it's okay to have processes in different PID ns share the same vm, but I guess that's implementation details that I don't need to care that much.

$ sudo ./vfork_newpid
vfork_newpid: PID: 8479
vfork_newpid: PID 8479 exiting after execve(2): Success
print_pid: PID 1 exiting. 

Quoted 4 lines

$ sudo ./vfork_newpid
vfork_newpid: PID: 8479
vfork_newpid: PID 8479 exiting after execve(2): Success
print_pid: PID 1 exiting. 

I definitely think this is a kernel (or glibc) bug. execve(2) is supposed to _never_ return 0 (and errno 0). I submitted a new bug to discuss it.

Please see <https://bugzilla.kernel.org/show_bug.cgi?id=215813>

It's not clear if this is valid. The syscall function in glibc does not protect the on-stack return address against overwriting, so it can't be used to call SYS_vfork on x86.

Can you reproduce this with a true inline syscall, or the glibc vfork function (which protects the return address)?