Skip to contentSkip to conversation

rseq/membarrier: add MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ

PATCH Superseded Peter Oskolkov 4 participants 12 messages Aug 11, 2020 – Aug 25, 2020

Subsystems (5)

v3 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/2 rseq/membarrier: add MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ

This patchset is based on Google-internal RSEQ work done by Paul Turner and Andrew Hunter.

When working with per-CPU RSEQ-based memory allocations, it is sometimes important to make sure that a global memory location is no longer accessed from RSEQ critical sections. For example, there can be two per-CPU lists, one is "active" and accessed per-CPU, while another one is inactive and worked on asynchronously "off CPU" (e.g. garbage collection is performed). Then at some point the two lists are swapped, and a fast RCU-like mechanism is required to make sure that the previously active list is no longer accessed.

This patch introduces such a mechanism: in short, membarrier() syscall issues an IPI to a CPU, restarting a potentially active RSEQ critical section on the CPU.

v1->v2:
- removed the ability to IPI all CPUs in a single sycall;
- use task->mm rather than task->group_leader to identify tasks belonging to the same process.
v2->v3:
- re-added the ability to IPI all CPUs in a single syscall;
- integrated with membarrier_private_expedited() to make sure only CPUs running tasks with the same mm as the current task are interrupted;
- also added MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ;
- flags in membarrier_private_expedited are never actually bit flags but always distinct values (i.e. never two flags are combined), so I modified bit testing to full equation comparison for simplicity (otherwise the code needs to work when several bits are set, for example).

The second patch in the patchset adds a selftest of this feature.

---
 include/linux/sched/mm.h        |  3 ++
 include/uapi/linux/membarrier.h | 24 ++++++++++++++
 kernel/sched/membarrier.c       | 55 ++++++++++++++++++++++++++++-----
 3 files changed, 74 insertions(+), 8 deletions(-)
include/linux/sched/mm.h +3 -0
index 6be66f52a2ad..477d5526e053 100644
@@ -352,10 +352,13 @@ enum {
 	MEMBARRIER_STATE_GLOBAL_EXPEDITED			= (1U << 3),
 	MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_READY	= (1U << 4),
 	MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE		= (1U << 5),
+	MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ_READY		= (1U << 6),
+	MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ			= (1U << 7),
 };
 
 enum {
 	MEMBARRIER_FLAG_SYNC_CORE	= (1U << 0),
+	MEMBARRIER_FLAG_RSEQ		= (1U << 1),
 };
 
 #ifdef CONFIG_ARCH_HAS_MEMBARRIER_CALLBACKS
include/uapi/linux/membarrier.h +24 -0
index 5891d7614c8c..f6bafe0676ba 100644
@@ -114,6 +114,28 @@
  *                          If this command is not implemented by an
  *                          architecture, -EINVAL is returned.
  *                          Returns 0 on success.
+ * @MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ:
+ *                          In addition to provide memory ordering
+ *                          guarantees described in
+ *                          MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE,
+ *                          ensure the caller thread, upon return from
+ *                          system call, that all its running thread
+ *                          siblings have any currently running rseq
+ *                          critical sections restarted if @flags
+ *                          parameter is -1; if @flags parameter is >= 0,
+ *                          then this operation is performed only
+ *                          on CPU indicated by @flags. If this command is
+ *                          not implemented by an architecture, -EINVAL
+ *                          is returned. A process needs to register its
+ *                          intent to use the private expedited rseq
+ *                          command prior to using it, otherwise
+ *                          this command returns -EPERM.
+ * @MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ:
+ *                          Register the process intent to use
+ *                          MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ.
+ *                          If this command is not implemented by an
+ *                          architecture, -EINVAL is returned.
+ *                          Returns 0 on success.
  * @MEMBARRIER_CMD_SHARED:
  *                          Alias to MEMBARRIER_CMD_GLOBAL. Provided for
  *                          header backward compatibility.
@@ -131,6 +153,8 @@ enum membarrier_cmd {
 	MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED		= (1 << 4),
 	MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE		= (1 << 5),
 	MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE	= (1 << 6),
+	MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ			= (1 << 7),
+	MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ		= (1 << 8),
 
 	/* Alias for header backward compatibility. */
 	MEMBARRIER_CMD_SHARED			= MEMBARRIER_CMD_GLOBAL,
kernel/sched/membarrier.c +47 -8

135 lines not on this page · raw message

Signature · 1 line
2.28.0.236.gb10cc79966-goog
#1 lore raw

PATCH v3 2/2 rseq/selftests: test MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ

Based on Google-internal RSEQ work done by
Paul Turner and Andrew Hunter.

This patch adds a selftest for MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ. The test quite often fails without the previous patch in this patchset, but consistently passes with it.

v3: added rseq_offset_deref_addv() to x86_64 to make the test
more explicit; on other architectures I kept using existing rseq_cmpeqv_cmpeqv_storev() as I have no easy way to test there. Added a comment explaining why the test works this way.

---
 .../selftests/rseq/basic_percpu_ops_test.c    | 196 ++++++++++++++++++
 tools/testing/selftests/rseq/rseq-x86.h       |  55 +++++
 2 files changed, 251 insertions(+)
tools/testing/selftests/rseq/basic_percpu_ops_test.c +196 -0
index eb3f6db36d36..c9784a3d19fb 100644
@@ -3,16 +3,22 @@
 #include <assert.h>
 #include <pthread.h>
 #include <sched.h>
+#include <stdatomic.h>
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <stddef.h>
+#include <syscall.h>
+#include <unistd.h>
 
 #include "rseq.h"
 
 #define ARRAY_SIZE(arr)	(sizeof(arr) / sizeof((arr)[0]))
 
+#define MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ	(1<<7)
+#define MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ	(1<<8)
+
 struct percpu_lock_entry {
 	intptr_t v;
 } __attribute__((aligned(128)));
@@ -289,6 +295,194 @@ void test_percpu_list(void)
 	assert(sum == expected_sum);
 }
 
+struct test_membarrier_thread_args {
+	int stop;
+	intptr_t percpu_list_ptr;
+};
+
+/* Worker threads modify data in their "active" percpu lists. */
+void *test_membarrier_worker_thread(void *arg)
+{
+	struct test_membarrier_thread_args *args =
+		(struct test_membarrier_thread_args *)arg;
+	const int iters = 10 * 1000 * 1000;
+	int i;
+
+	if (rseq_register_current_thread()) {
+		fprintf(stderr, "Error: rseq_register_current_thread(...) failed(%d): %s\n",
+			errno, strerror(errno));
+		abort();
+	}
+
+	/* Wait for initialization. */
+	while (!atomic_load(&args->percpu_list_ptr)) {}
+
+	for (i = 0; i < iters; ++i) {
+		int ret;
+
+		do {
+			int cpu = rseq_cpu_start();
+#if defined(__x86_64__)
+			/* For x86_64, we have rseq_offset_deref_addv. */
+			ret = rseq_offset_deref_addv(&args->percpu_list_ptr,
+							128 * cpu, 1, cpu);
+#else
+			/*
+			 *  For other architectures, we rely on the fact that
+			 *  the manager thread keeps list_ptr alive, so we can
+			 *  use rseq_cmpeqv_cmpeqv_storev to make sure
+			 *  list_ptr we got outside of rseq cs is still
+			 *  "active".
+			 */
+			struct percpu_list *list_ptr = (struct percpu_list *)
+					atomic_load(&args->percpu_list_ptr);
+
+			struct percpu_list_node *node = list_ptr->c[cpu].head;
+			const intptr_t prev = node->data;
+
+			ret = rseq_cmpeqv_cmpeqv_storev(&node->data, prev,
+					&args->percpu_list_ptr,
+					(intptr_t)list_ptr, prev + 1, cpu);
+#endif
+		} while (rseq_unlikely(ret));
+	}
+
+	if (rseq_unregister_current_thread()) {
+		fprintf(stderr, "Error: rseq_unregister_current_thread(...) failed(%d): %s\n",
+			errno, strerror(errno));
+		abort();
+	}
+	return NULL;
+}
+
+void test_membarrier_init_percpu_list(struct percpu_list *list)
+{
+	int i;
+
+	memset(list, 0, sizeof(*list));
+	for (i = 0; i < CPU_SETSIZE; i++) {
+		struct percpu_list_node *node;
+
+		node = malloc(sizeof(*node));
+		assert(node);
+		node->data = 0;
+		node->next = NULL;
+		list->c[i].head = node;
+	}
+}
+
+void test_membarrier_free_percpu_list(struct percpu_list *list)
+{
+	int i;
+
+	for (i = 0; i < CPU_SETSIZE; i++)
+		free(list->c[i].head);
+}
+
+static int sys_membarrier(int cmd, int flags)
+{
+	return syscall(__NR_membarrier, cmd, flags);
+}
+
+/*
+ * The manager thread swaps per-cpu lists that worker threads see,
+ * and validates that there are no unexpected modifications.
+ */
+void *test_membarrier_manager_thread(void *arg)
+{
+	struct test_membarrier_thread_args *args =
+		(struct test_membarrier_thread_args *)arg;
+	struct percpu_list list_a, list_b;
+	intptr_t expect_a = 0, expect_b = 0;
+	int cpu_a = 0, cpu_b = 0;
+
+	if (rseq_register_current_thread()) {
+		fprintf(stderr, "Error: rseq_register_current_thread(...) failed(%d): %s\n",
+			errno, strerror(errno));
+		abort();
+	}
+
+	/* Init lists. */
+	test_membarrier_init_percpu_list(&list_a);
+	test_membarrier_init_percpu_list(&list_b);
+
+	atomic_store(&args->percpu_list_ptr, (intptr_t)&list_a);
+
+	while (!atomic_load(&args->stop)) {
+		/* list_a is "active". */
+		cpu_a = rand() % CPU_SETSIZE;
+		/*
+		 * As list_b is "inactive", we should never see changes
+		 * to list_b.
+		 */
+		if (expect_b != atomic_load(&list_b.c[cpu_b].head->data)) {
+			fprintf(stderr, "Membarrier test failed\n");
+			abort();
+		}
+
+		/* Make list_b "active". */
+		atomic_store(&args->percpu_list_ptr, (intptr_t)&list_b);
+		sys_membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ, cpu_a);
+		/*
+		 * Cpu A should now only modify list_b, so we values
+		 * in list_a should be stable.
+		 */
+		expect_a = atomic_load(&list_a.c[cpu_a].head->data);
+
+		cpu_b = rand() % CPU_SETSIZE;
+		/*
+		 * As list_a is "inactive", we should never see changes
+		 * to list_a.
+		 */
+		if (expect_a != atomic_load(&list_a.c[cpu_a].head->data)) {
+			fprintf(stderr, "Membarrier test failed\n");
+			abort();
+		}
+
+		/* Make list_a "active". */
+		atomic_store(&args->percpu_list_ptr, (intptr_t)&list_a);
+		sys_membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ, cpu_b);
+		/* Remember a value from list_b. */
+		expect_b = atomic_load(&list_b.c[cpu_b].head->data);
+	}
+
+	test_membarrier_free_percpu_list(&list_a);
+	test_membarrier_free_percpu_list(&list_b);
+
+	if (rseq_unregister_current_thread()) {
+		fprintf(stderr, "Error: rseq_unregister_current_thread(...) failed(%d): %s\n",
+			errno, strerror(errno));
+		abort();
+	}
+	return NULL;
+}
+
+/* Test MEMBARRIER_CMD_PRIVATE_RESTART_RSEQ_ON_CPU membarrier command. */
+void test_membarrier(void)
+{
+	struct test_membarrier_thread_args thread_args;
+	pthread_t worker_threads[CPU_SETSIZE];
+	pthread_t manager_thread;
+	int i;
+
+	sys_membarrier(MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ, 0);
+
+	thread_args.stop = 0;
+	thread_args.percpu_list_ptr = 0;
+	pthread_create(&manager_thread, NULL,
+		       test_membarrier_manager_thread, &thread_args);
+
+	for (i = 0; i < CPU_SETSIZE; i++)
+		pthread_create(&worker_threads[i], NULL,
+		       test_membarrier_worker_thread, &thread_args);
+
+	for (i = 0; i < CPU_SETSIZE; i++)
+		pthread_join(worker_threads[i], NULL);
+
+	atomic_store(&thread_args.stop, 1);
+	pthread_join(manager_thread, NULL);
+}
+
 int main(int argc, char **argv)
 {
 	if (rseq_register_current_thread()) {
@@ -300,6 +494,8 @@ int main(int argc, char **argv)
 	test_percpu_spinlock();
 	printf("percpu_list\n");
 	test_percpu_list();
+	printf("membarrier\n");
+	test_membarrier();
 	if (rseq_unregister_current_thread()) {
 		fprintf(stderr, "Error: rseq_unregister_current_thread(...) failed(%d): %s\n",
 			errno, strerror(errno));
tools/testing/selftests/rseq/rseq-x86.h +55 -0

61 lines not on this page · raw message

Signature · 1 line
2.28.0.236.gb10cc79966-goog
#2 lore raw

Re: [PATCH 2/2 v3] rseq/selftests: test MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ

Quoted 33 lines · ----- On Aug 10, 2020, at 8:09 PM, Peter Oskolkov [email protected] wrote:

Based on Google-internal RSEQ work done by
Paul Turner and Andrew Hunter.

This patch adds a selftest for MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ. The test quite often fails without the previous patch in this patchset, but consistently passes with it.

v3: added rseq_offset_deref_addv() to x86_64 to make the test
more explicit; on other architectures I kept using existing rseq_cmpeqv_cmpeqv_storev() as I have no easy way to test there. Added a comment explaining why the test works this way.

---
.../selftests/rseq/basic_percpu_ops_test.c | 196 ++++++++++++++++++
tools/testing/selftests/rseq/rseq-x86.h | 55 +++++

2 files changed, 251 insertions(+)
tools/testing/selftests/rseq/basic_percpu_ops_test.c +1 -0
index eb3f6db36d36..c9784a3d19fb 100644
@@ -3,16 +3,22 @@
 #include <assert.h>
 #include <pthread.h>
 #include <sched.h>
+#include <stdatomic.h>
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <stddef.h>

tools/testing/selftests/rseq/basic_percpu_ops_test.c line 21

Quoted 10 lines
tools/testing/selftests/rseq/basic_percpu_ops_test.c +5 -0
+#include <syscall.h>
+#include <unistd.h>
 
 #include "rseq.h"
 
 #define ARRAY_SIZE(arr)	(sizeof(arr) / sizeof((arr)[0]))
 
+#define MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ	(1<<7)
+#define MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ	(1<<8)
+

No need to define membarrier commands here if we include <linux/membarrier.h>

Quoted 46 lines
tools/testing/selftests/rseq/basic_percpu_ops_test.c +39 -0
 struct percpu_lock_entry {
 	intptr_t v;
 } __attribute__((aligned(128)));
@@ -289,6 +295,194 @@ void test_percpu_list(void)
 	assert(sum == expected_sum);
 }
 
+struct test_membarrier_thread_args {
+	int stop;
+	intptr_t percpu_list_ptr;
+};
+
+/* Worker threads modify data in their "active" percpu lists. */
+void *test_membarrier_worker_thread(void *arg)
+{
+	struct test_membarrier_thread_args *args =
+		(struct test_membarrier_thread_args *)arg;
+	const int iters = 10 * 1000 * 1000;
+	int i;
+
+	if (rseq_register_current_thread()) {
+		fprintf(stderr, "Error: rseq_register_current_thread(...) failed(%d): %s\n",
+			errno, strerror(errno));
+		abort();
+	}
+
+	/* Wait for initialization. */
+	while (!atomic_load(&args->percpu_list_ptr)) {}
+
+	for (i = 0; i < iters; ++i) {
+		int ret;
+
+		do {
+			int cpu = rseq_cpu_start();
+#if defined(__x86_64__)
+			/* For x86_64, we have rseq_offset_deref_addv. */
+			ret = rseq_offset_deref_addv(&args->percpu_list_ptr,
+							128 * cpu, 1, cpu);
+#else
+			/*
+			 *  For other architectures, we rely on the fact that
+			 *  the manager thread keeps list_ptr alive, so we can
+			 *  use rseq_cmpeqv_cmpeqv_storev to make sure
+			 *  list_ptr we got outside of rseq cs is still
+			 *  "active".
+			 */

tools/testing/selftests/rseq/basic_percpu_ops_test.c void test_percpu_list(void) · line 346

Quoted 10 lines
tools/testing/selftests/rseq/basic_percpu_ops_test.c +10 -0
+			struct percpu_list *list_ptr = (struct percpu_list *)
+					atomic_load(&args->percpu_list_ptr);
+
+			struct percpu_list_node *node = list_ptr->c[cpu].head;
+			const intptr_t prev = node->data;
+
+			ret = rseq_cmpeqv_cmpeqv_storev(&node->data, prev,
+					&args->percpu_list_ptr,
+					(intptr_t)list_ptr, prev + 1, cpu);
+#endif

Please don't special-case the implementation of a test per architecture.

We should instead "skip" (or even fail) the test on architectures that do not support this, as an incentive for architecture maintainers to implement the missing APIs in the test.

One way to do this would be to define RSEQ_ARCH_HAS_OFFSET_DEREF_ADDV in the architecture header, and skip the test if the define is not present.

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
maintainer
#3 lore raw
In reply to Peter Oskolkov

Review line 37

Quoted 4 lines · Peter Oskolkov
(quoted diff) +0 -0
@@ -27,6 +35,12 @@
 
 static void ipi_mb(void *info)
 {

The #ifdef wants to behere, otherwise you'll get a compile warning for !RSEQ builds.

Review line 45

Quoted 8 lines
(quoted diff) +6 -0
+	int *flags = info;
+
+#ifdef CONFIG_RSEQ
+	if (flags && (*flags == MEMBARRIER_FLAG_RSEQ))
+		rseq_preempt(current);
+#endif
 	smp_mb();	/* IPIs should be serializing but paranoid. */
 }

But yes, this looks much better.

Mathieu did mention a few other points that I didn't see addressed:

- he didn't like abusing the @flags syscall argument for a CPUid;
- he wondered if we should support SYNC_CORE + RSEQ.

Not sure we can easily change the syscall at this point, but the latter point could be addressed with something like this.

---

kernel/sched/membarrier.c +22 -6
Index: linux-2.6/kernel/sched/membarrier.c
===================================================================
@@ -374,8 +374,26 @@ static int membarrier_register_private_e
  */
 SYSCALL_DEFINE2(membarrier, int, cmd, int, flags)
 {
+	int cflags = 0, int cpuid = -1;
+
 	if (unlikely(flags) && cmd != MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)
 		return -EINVAL;
+
+	if (cmd & (MEMBARRIER_CMD_PRIVATE_EXPEDITED |
+		   MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE |
+		   MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)) {
+
+		if (cmd & MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)
+			cflags |= MEMBARRIER_FLAG_RSEQ;
+
+		if (cmd & MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE) {
+			cflags |= MEMBARRIER_FLAG_SYNC_CORE;
+			cpuid = flags;
+		}
+
+		cmd = MEMBARRIER_CMD_PRIVATE_EXPEDITED;
+	}
+
 	switch (cmd) {
 	case MEMBARRIER_CMD_QUERY:
 	{
@@ -396,18 +414,16 @@ SYSCALL_DEFINE2(membarrier, int, cmd, in
 		return membarrier_global_expedited();
 	case MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED:
 		return membarrier_register_global_expedited();
-	case MEMBARRIER_CMD_PRIVATE_EXPEDITED:
-		return membarrier_private_expedited(0, -1);
 	case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED:
 		return membarrier_register_private_expedited(0);
-	case MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE:
-		return membarrier_private_expedited(MEMBARRIER_FLAG_SYNC_CORE, -1);
 	case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE:
 		return membarrier_register_private_expedited(MEMBARRIER_FLAG_SYNC_CORE);
-	case MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ:
-		return membarrier_private_expedited(MEMBARRIER_FLAG_RSEQ, flags);
 	case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ:
 		return membarrier_register_private_expedited(MEMBARRIER_FLAG_RSEQ);
+
+	case MEMBARRIER_CMD_PRIVATE_EXPEDITED:
+		return membarrier_private_expedited(cflags, cpuid);
+
 	default:
 		return -EINVAL;
 	}
maintainer
#4 lore raw
Quoted 11 lines · Peter Zijlstra

SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) {

kernel/sched/membarrier.c +7 -0
+	int cflags = 0, int cpuid = -1;
+
 	if (unlikely(flags) && cmd != MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)
 		return -EINVAL;
+
+	if (cmd & (MEMBARRIER_CMD_PRIVATE_EXPEDITED |
+		   MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE |
+		   MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)) {
+

kernel/sched/membarrier.c static int membarrier_register_private_e · line 395

Quoted 10 lines
kernel/sched/membarrier.c +10 -0
+		if (cmd & MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)
+			cflags |= MEMBARRIER_FLAG_RSEQ;
+
+		if (cmd & MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE) {
+			cflags |= MEMBARRIER_FLAG_SYNC_CORE;
+			cpuid = flags;
+		}
+
+		cmd = MEMBARRIER_CMD_PRIVATE_EXPEDITED;
+	}

This of course fails to check if other bits are set, and it goes really 'funny' if you use cpuid != -1.

That all needs a little more thought.

#5 lore raw
In reply to Peter Zijlstra
Quoted 9 lines · Peter Zijlstra

Review line 37

Quoted 4 lines · Peter Oskolkov
(quoted diff) +0 -0
@@ -27,6 +35,12 @@
 
 static void ipi_mb(void *info)
 {

The #ifdef wants to behere, otherwise you'll get a compile warning for !RSEQ builds.

Ack. Will do in the next version - for now waiting for the rest to be worked out.

[...]

Quoted 4 lines

Mathieu did mention a few other points that I didn't see addressed:

- he didn't like abusing the @flags syscall argument for a CPUid;

@flags is not used now; maybe just rename it to something more generic? @param? Or @options? Or maybe more specific, like @cpu_id?

Quoted 1 line

- he wondered if we should support SYNC_CORE + RSEQ.

It seems to me that CMD_PRIVATE_EXPEDITED_RSEQ is basically CMD_PRIVATE_EXPEDITED_SYNC_CORE with the extra "restart RSEQ CSs" behavior. Am I missing something? If not, what is the point of complicating the code as suggested below? Maybe just renaming CMD_PRIVATE_EXPEDITED_RSEQ to CMD_PRIVATE_EXPEDITED_SYNC_CORE_RSEQ will do?

Quoted 61 lines

Not sure we can easily change the syscall at this point, but the latter point could be addressed with something like this.

---
Index: linux-2.6/kernel/sched/membarrier.c
===================================================================

kernel/sched/membarrier.c +22 -6
@@ -374,8 +374,26 @@ static int membarrier_register_private_e
  */
 SYSCALL_DEFINE2(membarrier, int, cmd, int, flags)
 {
+       int cflags = 0, int cpuid = -1;
+
        if (unlikely(flags) && cmd != MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)
                return -EINVAL;
+
+       if (cmd & (MEMBARRIER_CMD_PRIVATE_EXPEDITED |
+                  MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE |
+                  MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)) {
+
+               if (cmd & MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)
+                       cflags |= MEMBARRIER_FLAG_RSEQ;
+
+               if (cmd & MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE) {
+                       cflags |= MEMBARRIER_FLAG_SYNC_CORE;
+                       cpuid = flags;
+               }
+
+               cmd = MEMBARRIER_CMD_PRIVATE_EXPEDITED;
+       }
+
        switch (cmd) {
        case MEMBARRIER_CMD_QUERY:
        {
@@ -396,18 +414,16 @@ SYSCALL_DEFINE2(membarrier, int, cmd, in
                return membarrier_global_expedited();
        case MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED:
                return membarrier_register_global_expedited();
-       case MEMBARRIER_CMD_PRIVATE_EXPEDITED:
-               return membarrier_private_expedited(0, -1);
        case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED:
                return membarrier_register_private_expedited(0);
-       case MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE:
-               return membarrier_private_expedited(MEMBARRIER_FLAG_SYNC_CORE, -1);
        case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE:
                return membarrier_register_private_expedited(MEMBARRIER_FLAG_SYNC_CORE);
-       case MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ:
-               return membarrier_private_expedited(MEMBARRIER_FLAG_RSEQ, flags);
        case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ:
                return membarrier_register_private_expedited(MEMBARRIER_FLAG_RSEQ);
+
+       case MEMBARRIER_CMD_PRIVATE_EXPEDITED:
+               return membarrier_private_expedited(cflags, cpuid);
+
        default:
                return -EINVAL;
        }
#6 lore raw
Quoted 17 lines · ----- On Aug 11, 2020, at 5:08 PM, Peter Oskolkov [email protected] wrote:
Quoted 9 lines · Peter Zijlstra

Review line 37

Quoted 4 lines · Peter Oskolkov
(quoted diff) +0 -0
@@ -27,6 +35,12 @@
 
 static void ipi_mb(void *info)
 {

The #ifdef wants to behere, otherwise you'll get a compile warning for !RSEQ builds.

Ack. Will do in the next version - for now waiting for the rest to be worked out.

[...]

Quoted 1 line
Quoted 6 lines
Quoted 3 lines

Mathieu did mention a few other points that I didn't see addressed:

- he didn't like abusing the @flags syscall argument for a CPUid;

@flags is not used now; maybe just rename it to something more generic? @param? Or @options? Or maybe more specific, like @cpu_id?

"flags" is there to allow extensibility without requiring to add new membarrier commands for every change. Even though it is not used now, I don't think re-purposing it is a good idea. What is wrong with just adding an additional "cpu" parameter to the system call ?

A "flags" parameter is very common for system calls. I don't see why we should change its name, especially given it is already exposed and documented as "flags" in man pages.

Quoted 6 lines
Quoted 2 lines

- he wondered if we should support SYNC_CORE + RSEQ.

It seems to me that CMD_PRIVATE_EXPEDITED_RSEQ is basically CMD_PRIVATE_EXPEDITED_SYNC_CORE with the extra "restart RSEQ CSs" behavior. Am I missing something?

No. The "sync-core" is about doing context synchronization for JIts, and is not implemented on all architectures today. RSEQ however is available on a wider range of architectures.

Quoted 4 lines

If not, what is the point of
complicating the code as suggested below? Maybe just renaming CMD_PRIVATE_EXPEDITED_RSEQ to CMD_PRIVATE_EXPEDITED_SYNC_CORE_RSEQ will do?

We basically have the following feature matrix:

- private / global
- expedited / non-expedited
- sync-core / non-sync-core
- rseq-fence / non-rseq-fence

For a total of about 16 combinations in total if we want to support them all.

We can continue to add separate commands for new combinations, but if we want to allow them to be combined, using flags rather than adding extra commands would have the advantage of keeping the number of commands manageable.

However, if there is no actual use-case for combining a membarrier sync-core and a membarrier rseq-fence, then it limits the number of commands and maybe then it's acceptable to add the rseq-fence as a separate membarrier command.

I prefer to have this discussion now rather than once we get to the point of having 40 membarrier commands for all possible combinations.

Thanks,

Mathieu

Quoted 63 lines
Quoted 63 lines

Not sure we can easily change the syscall at this point, but the latter point could be addressed with something like this.

---
Index: linux-2.6/kernel/sched/membarrier.c
===================================================================

kernel/sched/membarrier.c +22 -7
@@ -374,8 +374,26 @@ static int membarrier_register_private_e
  */
 SYSCALL_DEFINE2(membarrier, int, cmd, int, flags)
 {
+       int cflags = 0, int cpuid = -1;
+
        if (unlikely(flags) && cmd != MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)
                return -EINVAL;
+
+       if (cmd & (MEMBARRIER_CMD_PRIVATE_EXPEDITED |
+                  MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE |
+                  MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)) {
+
+               if (cmd & MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ)
+                       cflags |= MEMBARRIER_FLAG_RSEQ;
+
+               if (cmd & MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE) {
+                       cflags |= MEMBARRIER_FLAG_SYNC_CORE;
+                       cpuid = flags;
+               }
+
+               cmd = MEMBARRIER_CMD_PRIVATE_EXPEDITED;
+       }
+
        switch (cmd) {
        case MEMBARRIER_CMD_QUERY:
        {
@@ -396,18 +414,16 @@ SYSCALL_DEFINE2(membarrier, int, cmd, in
                return membarrier_global_expedited();
        case MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED:
                return membarrier_register_global_expedited();
-       case MEMBARRIER_CMD_PRIVATE_EXPEDITED:
-               return membarrier_private_expedited(0, -1);
        case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED:
                return membarrier_register_private_expedited(0);
-       case MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE:
-               return membarrier_private_expedited(MEMBARRIER_FLAG_SYNC_CORE,
-1);
        case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE:
                return membarrier_register_private_expedited(MEMBARRIER_FLAG_SYNC_CORE);
-       case MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ:
-               return membarrier_private_expedited(MEMBARRIER_FLAG_RSEQ,
 flags);
        case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ:
                return membarrier_register_private_expedited(MEMBARRIER_FLAG_RSEQ);
+
+       case MEMBARRIER_CMD_PRIVATE_EXPEDITED:
+               return membarrier_private_expedited(cflags, cpuid);
+
        default:
                return -EINVAL;
        }
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#7 lore raw

On Wed, Aug 12, 2020 at 11:30 AM Mathieu Desnoyers <[email protected]> wrote:

[...]

Quoted 4 lines

"flags" is there to allow extensibility without requiring to add new membarrier commands for every change. Even though it is not used now, I don't think re-purposing it is a good idea. What is wrong with just adding an additional "cpu" parameter to the system call ?

Can we do that? I thought adding an additional parameter means adding another syscall (ABI => parameter types/count cannot change?)

Quoted 4 lines

A "flags" parameter is very common for system calls. I don't see why we should change its name, especially given it is already exposed and documented as "flags" in man pages.

[...]

Quoted 15 lines

We basically have the following feature matrix:

- private / global
- expedited / non-expedited
- sync-core / non-sync-core
- rseq-fence / non-rseq-fence

For a total of about 16 combinations in total if we want to support them all.

We can continue to add separate commands for new combinations, but if we want to allow them to be combined, using flags rather than adding extra commands would have the advantage of keeping the number of commands manageable.

Quoted 6 lines

However, if there is no actual use-case for combining a membarrier sync-core and a membarrier rseq-fence, then it limits the number of commands and maybe then it's acceptable to add the rseq-fence as a separate membarrier command.

I prefer to have this discussion now rather than once we get to the point of having 40 membarrier commands for all possible combinations.

All commands are currently distinct bits, but are treated as separate commands. One way of doing what you suggest is to allow some commands to be bitwise-ORed.

So, for example, the user could call

membarrier(CMD_PRIVATE_EXPEDITED_SYNC_CORE | CMD_PRIVATE_EXPEDITED_RSEQ, cpu_id)

Is this what you have in mind?

[...]

Thanks,
Peter

#8 lore raw
Quoted 5 lines · ----- On Aug 12, 2020, at 2:48 PM, Peter Oskolkov [email protected] wrote:

On Wed, Aug 12, 2020 at 11:30 AM Mathieu Desnoyers <[email protected]> wrote:

[...]

Quoted 7 lines
Quoted 4 lines

"flags" is there to allow extensibility without requiring to add new membarrier commands for every change. Even though it is not used now, I don't think re-purposing it is a good idea. What is wrong with just adding an additional "cpu" parameter to the system call ?

Can we do that? I thought adding an additional parameter means adding another syscall (ABI => parameter types/count cannot change?)

I was under the impression that adding parameters to a system call for new flags (or commands) was not an issue. One example is the clone system call which expects the ctid argument if the CLONE_CHILD_CLEARTID flag is set. But maybe it was OK at some earlier point in time, but it's not OK anymore ? (CCing linux-arch to ask for advice)

Quoted 23 lines
Quoted 5 lines

A "flags" parameter is very common for system calls. I don't see why we should change its name, especially given it is already exposed and documented as "flags" in man pages.

[...]

Quoted 15 lines

We basically have the following feature matrix:

- private / global
- expedited / non-expedited
- sync-core / non-sync-core
- rseq-fence / non-rseq-fence

For a total of about 16 combinations in total if we want to support them all.

We can continue to add separate commands for new combinations, but if we want to allow them to be combined, using flags rather than adding extra commands would have the advantage of keeping the number of commands manageable.

Quoted 8 lines
Quoted 6 lines

However, if there is no actual use-case for combining a membarrier sync-core and a membarrier rseq-fence, then it limits the number of commands and maybe then it's acceptable to add the rseq-fence as a separate membarrier command.

I prefer to have this discussion now rather than once we get to the point of having 40 membarrier commands for all possible combinations.

All commands are currently distinct bits, but are treated as separate commands.

Indeed! I forgot about that. It was done so we can return a mask of supported commands with the MEMBARRIER_CMD_QUERY for feature discoverability. Those were never meant to be OR'd though, because then it's hard for user-space to discover what are the allowed command combinations.

Quoted 7 lines

One way of doing what you suggest is to allow some commands to be bitwise-ORed.

So, for example, the user could call

membarrier(CMD_PRIVATE_EXPEDITED_SYNC_CORE | CMD_PRIVATE_EXPEDITED_RSEQ, cpu_id)

Is this what you have in mind?

Not really. This would not take care of the fact that we would end up multiplying the number of commands as we allow combinations. E.g. if we ever want to have RSEQ work in private and global, and in non-expedited and expedited, we end up needing:

- CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ
- CMD_PRIVATE_EXPEDITED_RSEQ
- CMD_PRIVATE_RSEQ
- CMD_REGISTER_GLOBAL_EXPEDITED_RSEQ
- CMD_GLOBAL_EXPEDITED_RSEQ
- CMD_GLOBAL_RSEQ

The only thing we would save by OR'ing it with the SYNC_CORE command is the additional list:

- CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ_SYNC_CORE
- CMD_PRIVATE_EXPEDITED_RSEQ_SYNC_CORE
- CMD_PRIVATE_RSEQ_SYNC_CORE
- CMD_REGISTER_GLOBAL_EXPEDITED_RSEQ_SYNC_CORE
- CMD_GLOBAL_EXPEDITED_RSEQ_SYNC_CORE
- CMD_GLOBAL_RSEQ_SYNC_CORE

But unless we receive feedback that doing a membarrier with RSEQ+sync_core all in one go is a significant use-case, I am tempted to leave out that scenario for now. If we go for new commands, this means we could add (for private-expedited-rseq):

- MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ,
- MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ,

I do however have use-cases for using RSEQ across shared memory (between processes). Not currently for a rseq-fence, but for rseq acting as per-cpu atomic operations. If I ever end up needing rseq-fence across shared memory, that would result in the following new commands:

- MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED_RSEQ,
- MEMBARRIER_CMD_GLOBAL_EXPEDITED_RSEQ,

The remaining open question is whether it would be OK to define a new membarrier flag=MEMBARRIER_FLAG_CPU, which would expect an additional @cpu parameter.

Thanks,

Mathieu

Quoted 5 lines

[...]

Thanks,
Peter

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#9 lore raw
Quoted 1 line · Mathieu Desnoyers

[...]

Quoted 42 lines
Quoted 8 lines

One way of doing what you suggest is to allow some commands to be bitwise-ORed.

So, for example, the user could call

membarrier(CMD_PRIVATE_EXPEDITED_SYNC_CORE | CMD_PRIVATE_EXPEDITED_RSEQ, cpu_id)

Is this what you have in mind?

Not really. This would not take care of the fact that we would end up multiplying the number of commands as we allow combinations. E.g. if we ever want to have RSEQ work in private and global, and in non-expedited and expedited, we end up needing:

- CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ
- CMD_PRIVATE_EXPEDITED_RSEQ
- CMD_PRIVATE_RSEQ
- CMD_REGISTER_GLOBAL_EXPEDITED_RSEQ
- CMD_GLOBAL_EXPEDITED_RSEQ
- CMD_GLOBAL_RSEQ

The only thing we would save by OR'ing it with the SYNC_CORE command is the additional list:

- CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ_SYNC_CORE
- CMD_PRIVATE_EXPEDITED_RSEQ_SYNC_CORE
- CMD_PRIVATE_RSEQ_SYNC_CORE
- CMD_REGISTER_GLOBAL_EXPEDITED_RSEQ_SYNC_CORE
- CMD_GLOBAL_EXPEDITED_RSEQ_SYNC_CORE
- CMD_GLOBAL_RSEQ_SYNC_CORE

But unless we receive feedback that doing a membarrier with RSEQ+sync_core all in one go is a significant use-case, I am tempted to leave out that scenario for now. If we go for new commands, this means we could add (for private-expedited-rseq):

- MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ,
- MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ,

I do however have use-cases for using RSEQ across shared memory (between processes). Not currently for a rseq-fence, but for rseq acting as per-cpu atomic operations. If I ever end up needing rseq-fence across shared memory, that would result in the following new commands:

Quoted 6 lines

- MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED_RSEQ,
- MEMBARRIER_CMD_GLOBAL_EXPEDITED_RSEQ,

The remaining open question is whether it would be OK to define a new membarrier flag=MEMBARRIER_FLAG_CPU, which would expect an additional @cpu parameter.

Hi Mathieu,

I do not think there is any reason to wait for additional feedback, so I believe we should finalize the API/ABI.

I see two issues to resolve:
1: how to combine commands:
- you do not like adding new commands that are combinations of existing ones;
- you do not like ORing.
=> I'm not sure what other options we have here?

2: should @flags be repurposed for cpu_id, or MEMBARRIER_FLAG_CPU
added with a new syscall parameter.
=> I'm still not sure a new parameter can be cleanly added, but I can try
it in the next patchset if you prefer it this way.

Please let me know your thoughts.

Thanks,
Peter

#10 lore raw
Quoted 61 lines · ----- On Aug 20, 2020, at 1:42 PM, Peter Oskolkov [email protected] wrote:
Quoted 1 line · Mathieu Desnoyers

[...]

Quoted 48 lines
Quoted 8 lines

One way of doing what you suggest is to allow some commands to be bitwise-ORed.

So, for example, the user could call

membarrier(CMD_PRIVATE_EXPEDITED_SYNC_CORE | CMD_PRIVATE_EXPEDITED_RSEQ, cpu_id)

Is this what you have in mind?

Not really. This would not take care of the fact that we would end up multiplying
the number of commands as we allow combinations. E.g. if we ever want to have RSEQ
work in private and global, and in non-expedited and expedited, we end up needing:

- CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ
- CMD_PRIVATE_EXPEDITED_RSEQ
- CMD_PRIVATE_RSEQ
- CMD_REGISTER_GLOBAL_EXPEDITED_RSEQ
- CMD_GLOBAL_EXPEDITED_RSEQ
- CMD_GLOBAL_RSEQ

The only thing we would save by OR'ing it with the SYNC_CORE command is the additional
list:

- CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ_SYNC_CORE
- CMD_PRIVATE_EXPEDITED_RSEQ_SYNC_CORE
- CMD_PRIVATE_RSEQ_SYNC_CORE
- CMD_REGISTER_GLOBAL_EXPEDITED_RSEQ_SYNC_CORE
- CMD_GLOBAL_EXPEDITED_RSEQ_SYNC_CORE
- CMD_GLOBAL_RSEQ_SYNC_CORE

But unless we receive feedback that doing a membarrier with RSEQ+sync_core all in
one go is a significant use-case, I am tempted to leave out that scenario for now.
If we go for new commands, this means we could add (for private-expedited-rseq):

- MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ,
- MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ,

I do however have use-cases for using RSEQ across shared memory (between processes). Not currently for a rseq-fence, but for rseq acting as per-cpu atomic operations. If I ever end up needing rseq-fence across shared memory, that would result in the following new commands:

Quoted 6 lines

- MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED_RSEQ,
- MEMBARRIER_CMD_GLOBAL_EXPEDITED_RSEQ,

The remaining open question is whether it would be OK to define a new membarrier flag=MEMBARRIER_FLAG_CPU, which would expect an additional @cpu parameter.

Hi Mathieu,

Quoted 8 lines

I do not think there is any reason to wait for additional feedback, so I believe we should finalize the API/ABI.

I see two issues to resolve:
1: how to combine commands:
- you do not like adding new commands that are combinations of existing ones;
- you do not like ORing.
=> I'm not sure what other options we have here?

Concretely speaking, let's just add a new membarrier command for the use-case at hand. All other ways of doing things we have discussed are tricky to expose in a way that is discoverable by user-space through the QUERY command. (using a flag, or OR'ing many commands together)

Quoted 5 lines

2: should @flags be repurposed for cpu_id, or MEMBARRIER_FLAG_CPU
added with a new syscall parameter.
=> I'm still not sure a new parameter can be cleanly added, but I can try
it in the next patchset if you prefer it this way.

Yes please, it's easy to implement and we'll quickly see if anyone yells. If it turns out to be a bad idea, you can always blame me. ;-)

In summary:

- We add 2 new membarrier commands:
- MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ
- MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ

- We reserve a membarrier flag:

enum membarrier_flag {
MEMBARRIER_FLAG_CPU = (1 << 0),
}

So for CMD_PRIVATE_EXPEDITED_RSEQ, if flags & MEMBARRIER_FLAG_CPU is true, then we expect the additional "int cpu" parameter (3rd parameter). Else the cpu parameter is unused.

Are you OK with this approach ?

Thanks,

Mathieu

Quoted 5 lines

Please let me know your thoughts.

Thanks,
Peter

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#11 lore raw
Quoted 1 line · Mathieu Desnoyers

[...]

Quoted 27 lines

Concretely speaking, let's just add a new membarrier command for the use-case at hand. All other ways of doing things we have discussed are tricky to expose in a way that is discoverable by user-space through the QUERY command. (using a flag, or OR'ing many commands together)

Quoted 5 lines

2: should @flags be repurposed for cpu_id, or MEMBARRIER_FLAG_CPU
added with a new syscall parameter.
=> I'm still not sure a new parameter can be cleanly added, but I can try
it in the next patchset if you prefer it this way.

Yes please, it's easy to implement and we'll quickly see if anyone yells. If it turns out to be a bad idea, you can always blame me. ;-)

In summary:

- We add 2 new membarrier commands:
- MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ
- MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ

- We reserve a membarrier flag:

enum membarrier_flag {
MEMBARRIER_FLAG_CPU = (1 << 0),
}

Quoted 5 lines

So for CMD_PRIVATE_EXPEDITED_RSEQ, if flags & MEMBARRIER_FLAG_CPU is true, then we expect the additional "int cpu" parameter (3rd parameter). Else the cpu parameter is unused.

Are you OK with this approach ?

Yes, thanks for looking into this. I'll send a v4 later this week.

Thanks,
Peter

[...]