Skip to contentSkip to conversation

rseq: Implement KTLS prototype for x86-64

RFC PATCH In review Mathieu Desnoyers 3 participants 8 messages Sep 25, 2020 – Oct 29, 2020

Subsystems (7)
x86 Kselftest MM core Scheduler +3 more subsystems
lore
Share & tools
lore

Mathieu Desnoyers

author maintainer
#0 lore raw

RFC PATCH 1/2 rseq: Implement KTLS prototype for x86-64

Upstreaming efforts aiming to integrate rseq support into glibc led to interesting discussions, where we identified a clear need to extend the size of the per-thread structure shared between kernel and user-space (struct rseq). This is something that is not possible with the current rseq ABI. The fact that the current non-extensible rseq kernel ABI would also prevent glibc's ABI to be extended prevents its integration into glibc.

Discussions with glibc maintainers led to the following design, which we are calling "Kernel Thread Local Storage" or KTLS:

- at glibc library init:
- glibc queries the size and alignment of the KTLS area supported by the kernel,
- glibc reserves the memory area required by the kernel for main thread,
- glibc registers the offset from thread pointer where the KTLS area will be placed for all threads belonging to the threads group which are created with clone3 CLONE_RSEQ_KTLS,
- at nptl thread creation:
- glibc reserves the memory area required by the kernel,
- application/libraries can query glibc for the offset/size of the KTLS area, and offset from the thread pointer to access that area.

The basic idea is that the kernel UAPI will define the layout of that per-thread area, and glibc only deals with its allocation.

It should fulfill the extensibility needs for many extensions which can benefit from a per-thread shared memory area between kernel and user-space, e.g.:

- Exposing:
- current NUMA node number,
- current cpu number,
- current user id, group id, thread id, process id, process group id, ...
- signal block flag (fast-path for signal blocking),
- pretty much anything a vDSO can be used for, without requiring the function call.

This patch implements a crude prototype of extensible rseq ABI to show what interfaces we need to expose. It's currently wired up using rseq flags, turning rseq into a system call multiplexer, which was easy for a prototype.

It should be ready for some bikeshedding on how that ABI should look like. Is this extension using flags OK, or should this appear as new system calls instead ?

Cc Carlos O'Donell <[email protected]>
Cc "Florian Weimer <[email protected]>
Cc Peter Zijlstra (Intel) <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Boqun Feng <[email protected]>
---
 arch/x86/kernel/process_64.c |   1 +
 include/linux/sched.h        | 122 +-----------------------
 include/linux/sched/signal.h | 150 +++++++++++++++++++++++++++++
 include/uapi/linux/rseq.h    |  16 +++-
 include/uapi/linux/sched.h   |   1 +
 kernel/fork.c                |  14 ++-
 kernel/rseq.c                | 177 +++++++++++++++++++++++++++--------
 7 files changed, 321 insertions(+), 160 deletions(-)
arch/x86/kernel/process_64.c +1 -0
index 9a97415b2139..f0c822a78d01 100644
@@ -670,6 +670,7 @@ long do_arch_prctl_64(struct task_struct *task, int option, unsigned long arg2)
 			task->thread.fsindex = 0;
 			x86_fsbase_write_task(task, arg2);
 		}
+		rseq_set_thread_pointer(task, arg2);
 		preempt_enable();
 		break;
 	}
include/linux/sched.h +2 -120

140 lines not on this page · raw message

include/linux/sched/signal.h +150 -0

160 lines not on this page · raw message

include/uapi/linux/rseq.h +15 -1
index 9a402fdb60e9..f0fa947d6344 100644
@@ -18,8 +18,22 @@ enum rseq_cpu_id_state {
 	RSEQ_CPU_ID_REGISTRATION_FAILED		= -2,
 };
 
+struct rseq_ktls_layout {
+	__u32 size;
+	__u32 alignment;
+	__u64 offset;	/* 0 if not set. */
+};
+
+struct rseq_ktls_offset {
+	__s64 offset;
+};
+
 enum rseq_flags {
-	RSEQ_FLAG_UNREGISTER = (1 << 0),
+	RSEQ_FLAG_UNREGISTER		= (1 << 0),
+	RSEQ_FLAG_GET_KTLS_LAYOUT	= (1 << 1),	/* Get ktls size, alignment, offset from thread pointer (if set). */
+	RSEQ_FLAG_SET_KTLS_OFFSET	= (1 << 2),	/* Set ktls offset from thread pointer for process. */
+	RSEQ_FLAG_SET_SIG		= (1 << 3),	/* Set rseq signature for process. */
+	RSEQ_FLAG_SET_KTLS_THREAD	= (1 << 4),	/* Set ktls enabled for thread (same as done by CLONE_RSEQ_KTLS). */
 };
 
 enum rseq_cs_flags_bit {
include/uapi/linux/sched.h +1 -0
index 3bac0a8ceab2..96a88899c9b4 100644
@@ -36,6 +36,7 @@
 /* Flags for the clone3() syscall. */
 #define CLONE_CLEAR_SIGHAND 0x100000000ULL /* Clear any signal handler and reset to SIG_DFL. */
 #define CLONE_INTO_CGROUP 0x200000000ULL /* Clone into a specific cgroup given the right permissions. */
+#define CLONE_RSEQ_KTLS 0x400000000ULL /* Child has rseq ktls area. */
 
 /*
  * cloning flags intersect with CSIGNAL so can be used with unshare and clone3
kernel/fork.c +12 -2
index efc5493203ae..8a68601e4384 100644
@@ -1604,6 +1604,12 @@ static int copy_signal(unsigned long clone_flags, struct task_struct *tsk)
 	sig->oom_score_adj = current->signal->oom_score_adj;
 	sig->oom_score_adj_min = current->signal->oom_score_adj_min;
 
+#ifdef CONFIG_RSEQ
+	sig->rseq_ktls_offset = current->signal->rseq_ktls_offset;
+	sig->rseq_has_sig = current->signal->rseq_has_sig;
+	sig->rseq_sig = current->signal->rseq_sig;
+#endif
+
 	mutex_init(&sig->cred_guard_mutex);
 	mutex_init(&sig->exec_update_mutex);
 
@@ -2237,7 +2243,7 @@ static __latent_entropy struct task_struct *copy_process(
 	 */
 	copy_seccomp(p);
 
-	rseq_fork(p, clone_flags);
+	rseq_fork(p, clone_flags, args->tls);
 
 	/* Don't start children in a dying pid namespace */
 	if (unlikely(!(ns_of_pid(pid)->pid_allocated & PIDNS_ADDING))) {
@@ -2712,7 +2718,7 @@ static bool clone3_args_valid(struct kernel_clone_args *kargs)
 {
 	/* Verify that no unknown flags are passed along. */
 	if (kargs->flags &
-	    ~(CLONE_LEGACY_FLAGS | CLONE_CLEAR_SIGHAND | CLONE_INTO_CGROUP))
+	    ~(CLONE_LEGACY_FLAGS | CLONE_CLEAR_SIGHAND | CLONE_INTO_CGROUP | CLONE_RSEQ_KTLS))
 		return false;
 
 	/*
@@ -2730,6 +2736,10 @@ static bool clone3_args_valid(struct kernel_clone_args *kargs)
 	    kargs->exit_signal)
 		return false;
 
+	if ((kargs->flags & CLONE_RSEQ_KTLS) &&
+	    (!(kargs->flags & CLONE_THREAD) || !(kargs->flags & CLONE_SETTLS)))
+		return false;
+
 	if (!clone3_stack_valid(kargs))
 		return false;
 
kernel/rseq.c +140 -37

223 lines not on this page · raw message

Signature · 1 line
2.17.1
author maintainer
#1 lore raw

RFC PATCH 2/2 selftests/rseq: Adapt x86-64 rseq selftest to rseq KTLS prototype

The rseq KTLS ABI only requires a single SET_KTLS_OFFSET system call at library init for the entire thread group. There is no more need for per-thread registration.

The only architecture-specific part of this patch is rseq_get_thread_pointer, which is only implemented for x86-64 so far. Other architectures can rely on __builtin_thread_pointer(), but it is unfortunately unimplemented by gcc for at least x86-32 and x86-64 at the moment.

This is a minimal change to the rseq selftests which keeps using a fixed-size __rseq_abi TLS inital-exec variable in user-space, but use the rseq KTLS ABI for registration to the kernel.

In order to facilitate prototyping without requiring an updated glibc, there is one per-thread operation which is still performed right after thread creation: RSEQ_FLAG_SET_KTLS_THREAD. It sets the rseq_ktls flag to true in the current task struct. This is meant to be performed by glibc through use of clone3 CLONE_RSEQ_KTLS.

Cc Carlos O'Donell <[email protected]>
Cc "Florian Weimer <[email protected]>
Cc Peter Zijlstra (Intel) <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Boqun Feng <[email protected]>
---
 tools/testing/selftests/rseq/rseq-x86.h |   8 ++
 tools/testing/selftests/rseq/rseq.c     | 101 ++++++++----------------
 tools/testing/selftests/rseq/rseq.h     |   2 +-
 3 files changed, 44 insertions(+), 67 deletions(-)
tools/testing/selftests/rseq/rseq-x86.h +8 -0
index b2da6004fe30..e959d3fb1dea 100644
@@ -28,6 +28,14 @@
 
 #ifdef __x86_64__
 
+static inline void *rseq_get_thread_pointer(void)
+{
+	void *p;
+
+	asm ("mov %%fs:0, %0" : "=r" (p));
+	return p;
+}
+
 #define rseq_smp_mb()	\
 	__asm__ __volatile__ ("lock; addl $0,-128(%%rsp)" ::: "memory", "cc")
 #define rseq_smp_rmb()	rseq_barrier()
tools/testing/selftests/rseq/rseq.c +35 -66

136 lines not on this page · raw message

tools/testing/selftests/rseq/rseq.h +1 -1
index 3f63eb362b92..3c4fad7be4f7 100644
@@ -43,7 +43,7 @@
 #define RSEQ_INJECT_FAILED
 #endif
 
-extern __thread volatile struct rseq __rseq_abi;
+extern __thread __attribute__((tls_model("initial-exec"))) struct rseq __rseq_abi;
 extern int __rseq_handled;
 
 #define rseq_likely(x)		__builtin_expect(!!(x), 1)
Signature · 1 line
2.17.1
#2 lore raw
In reply to Mathieu Desnoyers

* Mathieu Desnoyers:

Quoted 11 lines

Upstreaming efforts aiming to integrate rseq support into glibc led to interesting discussions, where we identified a clear need to extend the size of the per-thread structure shared between kernel and user-space (struct rseq). This is something that is not possible with the current rseq ABI. The fact that the current non-extensible rseq kernel ABI would also prevent glibc's ABI to be extended prevents its integration into glibc.

Discussions with glibc maintainers led to the following design, which we are calling "Kernel Thread Local Storage" or KTLS:

Quoted 12 lines

- at glibc library init:
- glibc queries the size and alignment of the KTLS area supported by the kernel,
- glibc reserves the memory area required by the kernel for main thread,
- glibc registers the offset from thread pointer where the KTLS area will be placed for all threads belonging to the threads group which are created with clone3 CLONE_RSEQ_KTLS,
- at nptl thread creation:
- glibc reserves the memory area required by the kernel,
- application/libraries can query glibc for the offset/size of the KTLS area, and offset from the thread pointer to access that area.

One remaining challenge see is that we want to use vDSO functions to abstract away the exact layout of the KTLS area. For example, there are various implementation strategies for getuid optimizations, some of them exposing a shared struct cred in a thread group, and others not doing that.

The vDSO has access to the thread pointer because it's ABI (something that we recently (and quite conveniently) clarified for x86). What it does not know is the offset of the KTLS area from the thread pointer. In the original rseq implementation, this offset could vary from thread to thread in a process, although the submitted glibc implementation did not use this level of flexibility and the offset is constant. The vDSO is not relocated by the run-time dynamic loader, so it can't use ELF TLS data.

Furthermore, not all threads in a thread group may have an associated KTLS area. In a potential glibc implementation, only the threads created by pthread_create would have it; threads created directly using clone would lack it (and would not even run with a correctly set up userspace TCB).

So we have a bootstrap issue here that needs to be solved, I think.

In most cases, I would not be too eager to bypass the vDSO completely, and having the kernel expose a data-only interface. I could perhaps make an exception for the current TID because that's so convenient to use in mutex implementations, and errno. With the latter, we could directly expose the vDSO implementation to applications, assuming that we agree that the vDSO will not fail with ENOSYS to request fallback to the system call, but will itself perform the system call.

Thanks,
Florian

Signature · 3 lines
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
author maintainer
#3 lore raw
Quoted 32 lines · ----- On Sep 28, 2020, at 11:13 AM, Florian Weimer [email protected] wrote:

* Mathieu Desnoyers:

Quoted 11 lines

Upstreaming efforts aiming to integrate rseq support into glibc led to interesting discussions, where we identified a clear need to extend the size of the per-thread structure shared between kernel and user-space (struct rseq). This is something that is not possible with the current rseq ABI. The fact that the current non-extensible rseq kernel ABI would also prevent glibc's ABI to be extended prevents its integration into glibc.

Discussions with glibc maintainers led to the following design, which we are calling "Kernel Thread Local Storage" or KTLS:

Quoted 12 lines

- at glibc library init:
- glibc queries the size and alignment of the KTLS area supported by the kernel,
- glibc reserves the memory area required by the kernel for main thread,
- glibc registers the offset from thread pointer where the KTLS area will be placed for all threads belonging to the threads group which are created with clone3 CLONE_RSEQ_KTLS,
- at nptl thread creation:
- glibc reserves the memory area required by the kernel,
- application/libraries can query glibc for the offset/size of the KTLS area, and offset from the thread pointer to access that area.

One remaining challenge see is that we want to use vDSO functions to abstract away the exact layout of the KTLS area. For example, there are various implementation strategies for getuid optimizations, some of them exposing a shared struct cred in a thread group, and others not doing that.

Quoted 8 lines

The vDSO has access to the thread pointer because it's ABI (something that we recently (and quite conveniently) clarified for x86). What it does not know is the offset of the KTLS area from the thread pointer. In the original rseq implementation, this offset could vary from thread to thread in a process, although the submitted glibc implementation did not use this level of flexibility and the offset is constant. The vDSO is not relocated by the run-time dynamic loader, so it can't use ELF TLS data.

In the context of this prototype, the KTLS offset is the same for all threads belonging to a thread group.

Quoted 6 lines

Furthermore, not all threads in a thread group may have an associated KTLS area. In a potential glibc implementation, only the threads created by pthread_create would have it; threads created directly using clone would lack it (and would not even run with a correctly set up userspace TCB).

Right.

Quoted 2 lines

So we have a bootstrap issue here that needs to be solved, I think.

The one thing I'm not sure about is whether the vDSO interface is indeed superior to KTLS, or if it is just the model we are used to.

AFAIU, the current use-cases for vDSO is that an application calls into glibc, which then calls the vDSO function exposed by the kernel. I wonder whether the vDSO indirection is really needed if we typically have a glibc function used as indirection ? For an end user, what is the benefit of vDSO over accessing KTLS data directly from glibc ?

If we decide that using KTLS from a vDSO function is indeed a requirement, then, as you point out, the thread_pointer is available as ABI, but we miss the KTLS offset.

Some ideas on how we could solve this: we could either make the KTLS offset part of the ABI (fixed offset), or save the offset near the thread pointer at a location that would become ABI. It would have to be already populated with something which can help detect the case where a vDSO is called from a thread which does not populate KTLS though. Is that even remotely doable ?

Quoted 5 lines

In most cases, I would not be too eager to bypass the vDSO completely, and having the kernel expose a data-only interface. I could perhaps make an exception for the current TID because that's so convenient to use in mutex implementations, and errno.

Indeed, using a KTLS field to store errno is another use-case I forgot to mention. That would make life easier for errno handling in vDSO as well.

Quoted 4 lines

With the latter, we could
directly expose the vDSO implementation to applications, assuming that we agree that the vDSO will not fail with ENOSYS to request fallback to the system call, but will itself perform the system call.

We should not forget the fields needed by rseq as well: the rseq_cs pointer and the cpu_id fields need to be accessed directly from the rseq critical section, without function call. Those use-cases require that applications and library can know the KTLS offset and size and use those fields directly. That being said, there are certainly plenty of use-cases where it makes sense to use the KTLS data through a vDSO, and only expose the vDSO interface, if the cost of the extra vDSO call indirection is not prohibitive.

Thanks,

Mathieu

Quoted 7 lines

Thanks,
Florian
--
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#4 lore raw

* Mathieu Desnoyers:

Quoted 10 lines
Quoted 1 line

So we have a bootstrap issue here that needs to be solved, I think.

The one thing I'm not sure about is whether the vDSO interface is indeed superior to KTLS, or if it is just the model we are used to.

AFAIU, the current use-cases for vDSO is that an application calls into glibc, which then calls the vDSO function exposed by the kernel. I wonder whether the vDSO indirection is really needed if we typically have a glibc function used as indirection ? For an end user, what is the benefit of vDSO over accessing KTLS data directly from glibc ?

I think the kernel can only reasonably maintain a single userspace data structure. It's not reasonable to update several versions of the data structure in parallel.

This means that glibc would have to support multiple kernel data structures, and users might lose userspace acceleration after a kernel update, until they update glibc as well. The glibc update should be ABI-compatible, but someone would still have to backport it, apply it to container images, etc.

What's worse, the glibc code would be quite hard to test because we would have to keep around multiple kernel versions to exercise all the different data structure variants.

In contrast, the vDSO code always matches the userspace data structures, is always updated at the same time, and tested together. That looks like a clear win to me.

Quoted 10 lines

If we decide that using KTLS from a vDSO function is indeed a requirement, then, as you point out, the thread_pointer is available as ABI, but we miss the KTLS offset.

Some ideas on how we could solve this: we could either make the KTLS offset part of the ABI (fixed offset), or save the offset near the thread pointer at a location that would become ABI. It would have to be already populated with something which can help detect the case where a vDSO is called from a thread which does not populate KTLS though. Is that even remotely doable ?

I don't know.

We could decide that these accelerated system calls must only be called with a valid TCB. That's unavoidable if the vDSO sets errno directly, so it's perhaps not a big loss. It's also backwards-compatible because existing TCB-less code won't know about those new vDSO entrypoints. Calling into glibc from a TCB-less thread has always been undefined. TCB-less code would have to make direct, non-vDSO system calls, as today.

For discovering the KTLS offset, a per-process page at a fixed offset from the vDSO code (i.e., what real shared objects already do for global data) could store this offset. This way, we could entirely avoid an ABI dependency.

We'll see what will break once we have the correct TID after vfork. 8-> glibc currently supports malloc-after-vfork as an extension, and a lot of software depends on it (OpenJDK, for example).

Quoted 10 lines
Quoted 4 lines

With the latter, we could
directly expose the vDSO implementation to applications, assuming that we agree that the vDSO will not fail with ENOSYS to request fallback to the system call, but will itself perform the system call.

We should not forget the fields needed by rseq as well: the rseq_cs pointer and the cpu_id fields need to be accessed directly from the rseq critical section, without function call. Those use-cases require that applications and library can know the KTLS offset and size and use those fields directly.

Yes, but those offsets could be queried using a function from the vDSO (or using a glibc interface, to simplify linking).

Thanks,
Florian

Signature · 3 lines
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
author maintainer
#5 lore raw
Quoted 13 lines · ----- On Sep 29, 2020, at 4:13 AM, Florian Weimer [email protected] wrote:

* Mathieu Desnoyers:

Quoted 10 lines
Quoted 1 line

So we have a bootstrap issue here that needs to be solved, I think.

The one thing I'm not sure about is whether the vDSO interface is indeed superior to KTLS, or if it is just the model we are used to.

AFAIU, the current use-cases for vDSO is that an application calls into glibc, which then calls the vDSO function exposed by the kernel. I wonder whether the vDSO indirection is really needed if we typically have a glibc function used as indirection ? For an end user, what is the benefit of vDSO over accessing KTLS data directly from glibc ?

Quoted 3 lines

I think the kernel can only reasonably maintain a single userspace data structure. It's not reasonable to update several versions of the data structure in parallel.

I disagree with your statement. Considering that the kernel needs to keep ABI compatibility for whatever it exposes to user-space, claiming that it should never update several versions of data structures exposed to user-space in parallel means that once a data structure is exposed to user-space as ABI in a certain way, it can never ever change in the future, even if we find a better way to do things.

It makes more sense to allow multiple data structures to be updated in parallel until older ones become deprecated/unused/irrelevant, at which point those can be configured out at build time and eventually phased out after years of deprecation. Having the ability to update multiple data structures in user-space with replicated information is IMHO necessary to allow creation of new/better accelerated ABIs.

Quoted 6 lines

This means that glibc would have to support multiple kernel data structures, and users might lose userspace acceleration after a kernel update, until they update glibc as well. The glibc update should be ABI-compatible, but someone would still have to backport it, apply it to container images, etc.

No. If the kernel ever exposes a data structure to user-space as ABI, then it needs to stay there, and not break userspace. Hence the need to duplicate information provided to user-space if need be, so we can move on to better ABIs without breaking the old ones.

Quoted 8 lines

What's worse, the glibc code would be quite hard to test because we would have to keep around multiple kernel versions to exercise all the different data structure variants.

In contrast, the vDSO code always matches the userspace data structures, is always updated at the same time, and tested together. That looks like a clear win to me.

For cases where the overhead of vDSO is not an issue, I agree that it makes things tidier than directly accessing a data structure. The documentation of the ABI becomes much simpler as well.

Quoted 21 lines
Quoted 11 lines

If we decide that using KTLS from a vDSO function is indeed a requirement, then, as you point out, the thread_pointer is available as ABI, but we miss the KTLS offset.

Some ideas on how we could solve this: we could either make the KTLS offset part of the ABI (fixed offset), or save the offset near the thread pointer at a location that would become ABI. It would have to be already populated with something which can help detect the case where a vDSO is called from a thread which does not populate KTLS though. Is that even remotely doable ?

I don't know.

We could decide that these accelerated system calls must only be called with a valid TCB. That's unavoidable if the vDSO sets errno directly, so it's perhaps not a big loss. It's also backwards-compatible because existing TCB-less code won't know about those new vDSO entrypoints. Calling into glibc from a TCB-less thread has always been undefined. TCB-less code would have to make direct, non-vDSO system calls, as today.

Quoted 4 lines

For discovering the KTLS offset, a per-process page at a fixed offset from the vDSO code (i.e., what real shared objects already do for global data) could store this offset. This way, we could entirely avoid an ABI dependency.

Or as Andy mentioned, we would simply pass the ktls offset as argument to the vDSO ? It seems simple enough. Would it fit all our use-cases including errno ?

Quoted 4 lines

We'll see what will break once we have the correct TID after vfork. 8-> glibc currently supports malloc-after-vfork as an extension, and a lot of software depends on it (OpenJDK, for example).

I am not sure to see how that is related to ktls ?

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

With the latter, we could
directly expose the vDSO implementation to applications, assuming that we agree that the vDSO will not fail with ENOSYS to request fallback to the system call, but will itself perform the system call.

Quoted 8 lines
Quoted 5 lines

We should not forget the fields needed by rseq as well: the rseq_cs pointer and the cpu_id fields need to be accessed directly from the rseq critical section, without function call. Those use-cases require that applications and library can know the KTLS offset and size and use those fields directly.

Yes, but those offsets could be queried using a function from the vDSO (or using a glibc interface, to simplify linking).

Good point!

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#6 lore raw

* Mathieu Desnoyers:

Quoted 19 lines
Quoted 13 lines · ----- On Sep 29, 2020, at 4:13 AM, Florian Weimer [email protected] wrote:

* Mathieu Desnoyers:

Quoted 10 lines
Quoted 1 line

So we have a bootstrap issue here that needs to be solved, I think.

The one thing I'm not sure about is whether the vDSO interface is indeed superior to KTLS, or if it is just the model we are used to.

AFAIU, the current use-cases for vDSO is that an application calls into glibc, which then calls the vDSO function exposed by the kernel. I wonder whether the vDSO indirection is really needed if we typically have a glibc function used as indirection ? For an end user, what is the benefit of vDSO over accessing KTLS data directly from glibc ?

Quoted 3 lines

I think the kernel can only reasonably maintain a single userspace data structure. It's not reasonable to update several versions of the data structure in parallel.

Quoted 6 lines

I disagree with your statement. Considering that the kernel needs to keep ABI compatibility for whatever it exposes to user-space, claiming that it should never update several versions of data structures exposed to user-space in parallel means that once a data structure is exposed to user-space as ABI in a certain way, it can never ever change in the future, even if we find a better way to do things.

I think it's possible to put data into userspace without making it ABI. Think about the init_module system call. The module blob comes from userspace, but its (deeper) internal structure does not have a stable ABI. Similar for many BPF use cases.

If the internal KTLS blob structure turns into ABI, including the parts that need to be updated on context switch, each versioning change has a performance impact.

Quoted 10 lines
Quoted 5 lines

This means that glibc would have to support multiple kernel data structures, and users might lose userspace acceleration after a kernel update, until they update glibc as well. The glibc update should be ABI-compatible, but someone would still have to backport it, apply it to container images, etc.

No. If the kernel ever exposes a data structure to user-space as ABI, then it needs to stay there, and not break userspace. Hence the need to duplicate information provided to user-space if need be, so we can move on to better ABIs without breaking the old ones.

It can expose the data as an opaque blob.

Quoted 3 lines

Or as Andy mentioned, we would simply pass the ktls offset as argument to the vDSO ? It seems simple enough. Would it fit all our use-cases including errno ?

That would work, yes. It's neat, but it won't give you a way to provide traditional syscall wrappers directly from the vDSO.

Quoted 5 lines
Quoted 3 lines

We'll see what will break once we have the correct TID after vfork. 8-> glibc currently supports malloc-after-vfork as an extension, and a lot of software depends on it (OpenJDK, for example).

I am not sure to see how that is related to ktls ?

The mutex implementation could switch to the KTLS TID because it always correct. But then locking in a vfork'ed subprocess would no longer look like locking from the parent thread because the TID would be different.

Thanks,
Florian

Signature · 3 lines
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
#7 lore raw
In reply to Florian Weimer
Quoted 33 lines · Florian Weimer

* Mathieu Desnoyers:

Quoted 11 lines

Upstreaming efforts aiming to integrate rseq support into glibc led to interesting discussions, where we identified a clear need to extend the size of the per-thread structure shared between kernel and user-space (struct rseq). This is something that is not possible with the current rseq ABI. The fact that the current non-extensible rseq kernel ABI would also prevent glibc's ABI to be extended prevents its integration into glibc.

Discussions with glibc maintainers led to the following design, which we are calling "Kernel Thread Local Storage" or KTLS:

Quoted 12 lines

- at glibc library init:
- glibc queries the size and alignment of the KTLS area supported by the kernel,
- glibc reserves the memory area required by the kernel for main thread,
- glibc registers the offset from thread pointer where the KTLS area will be placed for all threads belonging to the threads group which are created with clone3 CLONE_RSEQ_KTLS,
- at nptl thread creation:
- glibc reserves the memory area required by the kernel,
- application/libraries can query glibc for the offset/size of the KTLS area, and offset from the thread pointer to access that area.

One remaining challenge see is that we want to use vDSO functions to abstract away the exact layout of the KTLS area. For example, there are various implementation strategies for getuid optimizations, some of them exposing a shared struct cred in a thread group, and others not doing that.

Quoted 8 lines

The vDSO has access to the thread pointer because it's ABI (something that we recently (and quite conveniently) clarified for x86). What it does not know is the offset of the KTLS area from the thread pointer. In the original rseq implementation, this offset could vary from thread to thread in a process, although the submitted glibc implementation did not use this level of flexibility and the offset is constant. The vDSO is not relocated by the run-time dynamic loader, so it can't use ELF TLS data.

I assume that, by "thread pointer", you mean the pointer stored in GSBASE on x86_32, FSBASE on x86_64, and elsewhere on other architectures?

The vDSO has done pretty well so far having the vDSO not touch FS, GS, or their bases at all. If we want to change that, I would be very nervous about doing so in existing vDSO functions. Regardless of anything an ABI document might say and anything that existing or previous glibc versions may or may not have done, there are plenty of bizarre programs out there that don't really respect the psABI document. Go and various not-ready-for-prime-time-but-released-anyway Bionic branches come to mind. So we would need to tread very, very carefully.

One way to side-step much of this would be to make the interface explicit:

long __vdso_do_whatever(void *ktls_ptr, ...);

Sadly, on x86, actually generating the ktls ptr is bit nasty due to the fact that lea %fs:(offset) doesn't do what one might have liked it to do. I suppose this could also be:

long __vdso_do_whatever(unsigned long ktls_offset);

which will generate quite nice code on x86_64. I can't speak for the asm capabilities of other architectures.

What I *don't* want to do is to accidentally repeat anything like the %gs:0x28 mess we have with the stack cookie on x86_32. (The stack cookie is, in kernel code, in a completely nonsensical location. I'm quite surprised that any of the maintainers ever accepted the current stack cookie implementation. I assume there's some history there, but I don't know it. The end result is a festering mess in the x86_32 kernel code that only persists because no one cares quite enough about x86_32 to fix it.) We obviously won't end up with precisely the same type of mistake here, but a mis-step here certainly does have the possibility of promoting an unfortunate-in-hindsight design decision in glibc and/or psABI to something that every other x86_64 Linux software stack has to copy to be compatible with the vDSO.

As for errno itself, with all due respect to those who designed errno before I was born, IMO it was a mistake. Why exactly should the vDSO know about errno?