sched/membarrier, selftests: Introduce MEMBARRIER_CMD_GET_REGISTRATIONS
Subsystems (3)
cover sched/membarrier, selftests: Introduce MEMBARRIER_CMD_GET_REGISTRATIONS
This change provides a method to query previously issued registrations. It's needed for CRIU (checkpoint/restore in userspace). Before this change we had to issue private membarrier commands during checkpoint - if they succeeded, they must have been registered. Unfortunately global membarrier succeeds even on unregistered processes, so there was no way to tell if MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED had been issued or not.
CRIU is run after the process has been frozen with ptrace, so we don't have to worry too much about the result of running this command in parallel with registration commands.
Michal Clapinski (2):
sched/membarrier: Introduce MEMBARRIER_CMD_GET_REGISTRATIONS selftests/membarrier: Test MEMBARRIER_CMD_GET_REGISTRATIONS
include/uapi/linux/membarrier.h | 4 ++ kernel/sched/membarrier.c | 39 ++++++++++++++++++- .../membarrier/membarrier_test_impl.h | 33 ++++++++++++++++ .../membarrier/membarrier_test_multi_thread.c | 2 +- .../membarrier_test_single_thread.c | 6 ++- 5 files changed, 81 insertions(+), 3 deletions(-)
Signature · 1 line
2.39.0.rc0.267.gcb52ba06e7-goog
PATCH 1/2 sched/membarrier: Introduce MEMBARRIER_CMD_GET_REGISTRATIONS
Provide a method to query previously issued registrations.
--- include/uapi/linux/membarrier.h | 4 ++++ kernel/sched/membarrier.c | 39 ++++++++++++++++++++++++++++++++- 2 files changed, 42 insertions(+), 1 deletion(-)
include/uapi/linux/membarrier.h +4 -0
@@ -137,6 +137,9 @@* @MEMBARRIER_CMD_SHARED: * Alias to MEMBARRIER_CMD_GLOBAL. Provided for * header backward compatibility. +* @MEMBARRIER_CMD_GET_REGISTRATIONS: +* Returns a bitmask of previously issued +* registration commands. * * Command to be passed to the membarrier system call. The commands need to * be a single bit each, except for MEMBARRIER_CMD_QUERY which is assigned to
@@ -153,6 +156,7 @@ enum membarrier_cmd {MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE = (1 <<6 ),MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ = (1 <<7 ),MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ = (1 <<8 ), +MEMBARRIER_CMD_GET_REGISTRATIONS = (1 <<9 ),/* Alias for header backward compatibility. */ MEMBARRIER_CMD_SHARED = MEMBARRIER_CMD_GLOBAL,
kernel/sched/membarrier.c +38 -1
@@ -159,7 +159,8 @@ | MEMBARRIER_CMD_PRIVATE_EXPEDITED \ | MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED \ | MEMBARRIER_PRIVATE_EXPEDITED_SYNC_CORE_BITMASK \ - | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK) + | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK \ + | MEMBARRIER_CMD_GET_REGISTRATIONS)static void ipi_mb (void *info) {
@@ -540,6 +541,40 @@ static int membarrier_register_private_expedited(int flags)return 0 ; } +static int membarrier_get_registrations (void ) +{ +struct task_struct *p = current; +struct mm_struct *mm = p->mm; +int registrations_mask =0 , membarrier_state, i; +static const int states[] = { +MEMBARRIER_STATE_GLOBAL_EXPEDITED | +MEMBARRIER_STATE_GLOBAL_EXPEDITED_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ_READY + }; +static const int registration_cmds[] = { +MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ + }; +BUILD_BUG_ON (ARRAY_SIZE (states) !=ARRAY_SIZE (registration_cmds)); + + membarrier_state =atomic_read (&mm->membarrier_state); +for (i =0 ; i <ARRAY_SIZE (states); ++i) { +if (membarrier_state & states[i]) { + registrations_mask |= registration_cmds[i]; + membarrier_state &= ~states[i]; + } + } +WARN_ON_ONCE (membarrier_state !=0 ); +return registrations_mask; +} +/** * sys_membarrier - issue memory barriers on a set of threads * @cmd: Takes command values defined in enum membarrier_cmd.
@@ -623,6 +658,8 @@ SYSCALL_DEFINE3(membarrier, int, cmd, unsigned int, flags, int, cpu_id)return membarrier_private_expedited (MEMBARRIER_FLAG_RSEQ , cpu_id);case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ :return membarrier_register_private_expedited (MEMBARRIER_FLAG_RSEQ ); +case MEMBARRIER_CMD_GET_REGISTRATIONS : +return membarrier_get_registrations ();default :return -EINVAL ; }
Signature · 1 line
2.39.0.rc0.267.gcb52ba06e7-goog
Mathieu Desnoyers
maintainerRe: [PATCH 1/2] sched/membarrier: Introduce MEMBARRIER_CMD_GET_REGISTRATIONS
Quoted 46 lines · Michal Clapinski
Provide a method to query previously issued registrations.
Signed-off-by Michal Clapinski <[email protected]>--- include/uapi/linux/membarrier.h | 4 ++++ kernel/sched/membarrier.c | 39 ++++++++++++++++++++++++++++++++- 2 files changed, 42 insertions(+), 1 deletion(-)include/uapi/linux/membarrier.h +4
-0@@ -137,6 +137,9 @@* @MEMBARRIER_CMD_SHARED: * Alias to MEMBARRIER_CMD_GLOBAL. Provided for * header backward compatibility. +* @MEMBARRIER_CMD_GET_REGISTRATIONS: +* Returns a bitmask of previously issued +* registration commands. * * Command to be passed to the membarrier system call. The commands need to * be a single bit each, except for MEMBARRIER_CMD_QUERY which is assigned to @@ -153,6 +156,7 @@ enum membarrier_cmd {MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE = (1 <<6 ),MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ = (1 <<7 ),MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ = (1 <<8 ), +MEMBARRIER_CMD_GET_REGISTRATIONS = (1 <<9 ),/* Alias for header backward compatibility. */ MEMBARRIER_CMD_SHARED = MEMBARRIER_CMD_GLOBAL,kernel/sched/membarrier.c +2
-1@@ -159,7 +159,8 @@ | MEMBARRIER_CMD_PRIVATE_EXPEDITED \ | MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED \ | MEMBARRIER_PRIVATE_EXPEDITED_SYNC_CORE_BITMASK \ - | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK) + | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK \ + | MEMBARRIER_CMD_GET_REGISTRATIONS)static void ipi_mb (void *info) {@@ -540,6 +541,40 @@ static int membarrier_register_private_expedited(int flags)return 0 ;
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 551
Quoted 10 lines
kernel/sched/membarrier.c +8
-0} +static int membarrier_get_registrations (void ) +{ +struct task_struct *p = current; +struct mm_struct *mm = p->mm; +int registrations_mask =0 , membarrier_state, i; +static const int states[] = { +MEMBARRIER_STATE_GLOBAL_EXPEDITED | +MEMBARRIER_STATE_GLOBAL_EXPEDITED_READY ,
What is the purpose of checking for the _READY state flag as well here ?
Quoted 8 lines
kernel/sched/membarrier.c +8
-0+MEMBARRIER_STATE_PRIVATE_EXPEDITED | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ_READY + }; + static const int registration_cmds[] = {
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 569
Quoted 10 lines
kernel/sched/membarrier.c +10
-0+MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ + }; +BUILD_BUG_ON (ARRAY_SIZE (states) !=ARRAY_SIZE (registration_cmds)); + + membarrier_state =atomic_read (&mm->membarrier_state); +for (i =0 ; i <ARRAY_SIZE (states); ++i) { +if (membarrier_state & states[i]) {
The mask will match if either of the flags to match are set. Is that your intent ?
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 571
Quoted 2 lines
kernel/sched/membarrier.c +2
-0+ registrations_mask |= registration_cmds[i]; + membarrier_state &= ~states[i];
So I understand that those _READY flags are there purely for making sure we clear the membarrier_state for validation that they have all been checked with the following WARN_ON_ONCE(). Am I on the right track ?
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 574
Quoted 3 lines
kernel/sched/membarrier.c +3
-0+ } + } +WARN_ON_ONCE (membarrier_state !=0 );
Thanks,
Mathieu
Quoted 15 lines
kernel/sched/membarrier.c +5
-0+return registrations_mask; +} +/** * sys_membarrier - issue memory barriers on a set of threads * @cmd: Takes command values defined in enum membarrier_cmd. @@ -623,6 +658,8 @@ SYSCALL_DEFINE3(membarrier, int, cmd, unsigned int, flags, int, cpu_id)return membarrier_private_expedited (MEMBARRIER_FLAG_RSEQ , cpu_id);case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ :return membarrier_register_private_expedited (MEMBARRIER_FLAG_RSEQ ); +case MEMBARRIER_CMD_GET_REGISTRATIONS : +return membarrier_get_registrations ();default :return -EINVAL ; }
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com
Re: [PATCH 1/2] sched/membarrier: Introduce MEMBARRIER_CMD_GET_REGISTRATIONS
Quoted 21 lines · Mathieu Desnoyers
Quoted 19 lines · Michal Clapinski
Provide a method to query previously issued registrations.
Signed-off-by Michal Clapinski <[email protected]>--- include/uapi/linux/membarrier.h | 4 ++++ kernel/sched/membarrier.c | 39 ++++++++++++++++++++++++++++++++- 2 files changed, 42 insertions(+), 1 deletion(-)include/uapi/linux/membarrier.h +3
-0@@ -137,6 +137,9 @@* @MEMBARRIER_CMD_SHARED: * Alias to MEMBARRIER_CMD_GLOBAL. Provided for * header backward compatibility. +* @MEMBARRIER_CMD_GET_REGISTRATIONS: +* Returns a bitmask of previously issued +* registration commands.
Quoted 8 lines
include/uapi/linux/membarrier.h enum membarrier_cmd { · line 159
Quoted 8 lines
include/uapi/linux/membarrier.h +1
-0* * Command to be passed to the membarrier system call. The commands need to * be a single bit each, except for MEMBARRIER_CMD_QUERY which is assigned to @@ -153,6 +156,7 @@ enum membarrier_cmd {MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE = (1 <<6 ),MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ = (1 <<7 ),MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ = (1 <<8 ), + MEMBARRIER_CMD_GET_REGISTRATIONS = (1 << 9),
Btw. I could do this as a flag to MEMBARRIER_CMD_QUERY instead of a separate command. Would that be preferable?
Quoted 21 lines
Quoted 21 lines
include/uapi/linux/membarrier.h +0
-0/* Alias for header backward compatibility. */ MEMBARRIER_CMD_SHARED = MEMBARRIER_CMD_GLOBAL,kernel/sched/membarrier.c +2
-1@@ -159,7 +159,8 @@ | MEMBARRIER_CMD_PRIVATE_EXPEDITED \ | MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED \ | MEMBARRIER_PRIVATE_EXPEDITED_SYNC_CORE_BITMASK \ - | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK) + | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK \ + | MEMBARRIER_CMD_GET_REGISTRATIONS)static void ipi_mb (void *info) {@@ -540,6 +541,40 @@ static int membarrier_register_private_expedited(int flags)return 0 ; }
Quoted 10 lines
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 551
Quoted 8 lines
kernel/sched/membarrier.c +8
-0+static int membarrier_get_registrations (void ) +{ +struct task_struct *p = current; +struct mm_struct *mm = p->mm; +int registrations_mask =0 , membarrier_state, i; +static const int states[] = { +MEMBARRIER_STATE_GLOBAL_EXPEDITED | +MEMBARRIER_STATE_GLOBAL_EXPEDITED_READY ,What is the purpose of checking for the
_READYstate flag as well here ?
Answered below.
Quoted 21 lines
Quoted 10 lines
kernel/sched/membarrier.c +8
-0+MEMBARRIER_STATE_PRIVATE_EXPEDITED | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ_READY + }; + static const int registration_cmds[] = {kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 569
Quoted 10 lines
kernel/sched/membarrier.c +10
-0+MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ + }; +BUILD_BUG_ON (ARRAY_SIZE (states) !=ARRAY_SIZE (registration_cmds)); + + membarrier_state =atomic_read (&mm->membarrier_state); +for (i =0 ; i <ARRAY_SIZE (states); ++i) { +if (membarrier_state & states[i]) {
Quoted 2 lines
The mask will match if either of the flags to match are set. Is that your intent ?
Kind of, it was just the easiest to write. As explained in the cover letter, I don't really care much about the result of this while the process is running. And when the process is frozen, either state and state_ready are set or none of them.
Quoted 8 lines
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 571
Quoted 4 lines
kernel/sched/membarrier.c +2
-0+ registrations_mask |= registration_cmds[i]; + membarrier_state &= ~states[i];So I understand that those
_READYflags are there purely for making sure we clear themembarrier_statefor validation that they have all been checked with the followingWARN_ON_ONCE(). Am I on the right track ?
Yes, exactly. It wastes time but I'm worried about people adding new states and not updating this function. A suggestion on how to do this better (especially at compile time) would be greatly appreciated.
Quoted 30 lines
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 574
Quoted 4 lines
kernel/sched/membarrier.c +3
-0+ } + } +WARN_ON_ONCE (membarrier_state !=0 );Thanks,
Mathieu
Quoted 7 lines
kernel/sched/membarrier.c +3
-0+return registrations_mask; +} +/** * sys_membarrier - issue memory barriers on a set of threads * @cmd: Takes command values defined in enum membarrier_cmd. kernel/sched/membarrier.c SYSCALL_DEFINE3(membarrier, int, cmd, unsigned int, flags, int, cpu_id) · line 665
Quoted 8 lines
kernel/sched/membarrier.c +2
-0@@ -623,6 +658,8 @@ SYSCALL_DEFINE3(membarrier, int, cmd, unsigned int, flags, int, cpu_id)return membarrier_private_expedited (MEMBARRIER_FLAG_RSEQ , cpu_id);case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ :return membarrier_register_private_expedited (MEMBARRIER_FLAG_RSEQ ); +case MEMBARRIER_CMD_GET_REGISTRATIONS : +return membarrier_get_registrations ();default :return -EINVAL ; }--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
Re: [PATCH 1/2] sched/membarrier: Introduce MEMBARRIER_CMD_GET_REGISTRATIONS
Quoted 107 lines · Michał Cłapiński
Quoted 21 lines · Mathieu Desnoyers
Quoted 19 lines · Michal Clapinski
Provide a method to query previously issued registrations.
Signed-off-by Michal Clapinski <[email protected]>--- include/uapi/linux/membarrier.h | 4 ++++ kernel/sched/membarrier.c | 39 ++++++++++++++++++++++++++++++++- 2 files changed, 42 insertions(+), 1 deletion(-)include/uapi/linux/membarrier.h +3
-0@@ -137,6 +137,9 @@* @MEMBARRIER_CMD_SHARED: * Alias to MEMBARRIER_CMD_GLOBAL. Provided for * header backward compatibility. +* @MEMBARRIER_CMD_GET_REGISTRATIONS: +* Returns a bitmask of previously issued +* registration commands. Quoted 8 lines
include/uapi/linux/membarrier.h enum membarrier_cmd { · line 159
Quoted 8 lines
include/uapi/linux/membarrier.h +1
-0* * Command to be passed to the membarrier system call. The commands need to * be a single bit each, except for MEMBARRIER_CMD_QUERY which is assigned to @@ -153,6 +156,7 @@ enum membarrier_cmd {MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE = (1 <<6 ),MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ = (1 <<7 ),MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ = (1 <<8 ), + MEMBARRIER_CMD_GET_REGISTRATIONS = (1 << 9),Btw. I could do this as a flag to
MEMBARRIER_CMD_QUERYinstead of a separate command. Would that be preferable?Quoted 21 lines
Quoted 21 lines
include/uapi/linux/membarrier.h +0
-0/* Alias for header backward compatibility. */ MEMBARRIER_CMD_SHARED = MEMBARRIER_CMD_GLOBAL,kernel/sched/membarrier.c +2
-1@@ -159,7 +159,8 @@ | MEMBARRIER_CMD_PRIVATE_EXPEDITED \ | MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED \ | MEMBARRIER_PRIVATE_EXPEDITED_SYNC_CORE_BITMASK \ - | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK) + | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK \ + | MEMBARRIER_CMD_GET_REGISTRATIONS)static void ipi_mb (void *info) {@@ -540,6 +541,40 @@ static int membarrier_register_private_expedited(int flags)return 0 ; }Quoted 10 lines
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 551
Quoted 8 lines
kernel/sched/membarrier.c +8
-0+static int membarrier_get_registrations (void ) +{ +struct task_struct *p = current; +struct mm_struct *mm = p->mm; +int registrations_mask =0 , membarrier_state, i; +static const int states[] = { +MEMBARRIER_STATE_GLOBAL_EXPEDITED | +MEMBARRIER_STATE_GLOBAL_EXPEDITED_READY ,What is the purpose of checking for the
_READYstate flag as well here ?Answered below.
Quoted 21 lines
Quoted 10 lines
kernel/sched/membarrier.c +8
-0+MEMBARRIER_STATE_PRIVATE_EXPEDITED | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ_READY + }; + static const int registration_cmds[] = {kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 569
Quoted 10 lines
kernel/sched/membarrier.c +10
-0+MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ + }; +BUILD_BUG_ON (ARRAY_SIZE (states) !=ARRAY_SIZE (registration_cmds)); + + membarrier_state =atomic_read (&mm->membarrier_state); +for (i =0 ; i <ARRAY_SIZE (states); ++i) { +if (membarrier_state & states[i]) {Quoted 2 lines
The mask will match if either of the flags to match are set. Is that your intent ?
Kind of, it was just the easiest to write. As explained in the cover letter, I don't really care much about the result of this while the process is running. And when the process is frozen, either state and
state_readyare set or none of them.Quoted 5 lines
Quoted 4 lines
kernel/sched/membarrier.c +2
-0+ registrations_mask |= registration_cmds[i]; + membarrier_state &= ~states[i];
Quoted 39 lines
Quoted 3 lines
So I understand that those
_READYflags are there purely for making sure we clear themembarrier_statefor validation that they have all been checked with the followingWARN_ON_ONCE(). Am I on the right track ?Yes, exactly. It wastes time but I'm worried about people adding new states and not updating this function. A suggestion on how to do this better (especially at compile time) would be greatly appreciated.
Quoted 30 lines
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 574
Quoted 4 lines
kernel/sched/membarrier.c +3
-0+ } + } +WARN_ON_ONCE (membarrier_state !=0 );Thanks,
Mathieu
Quoted 7 lines
kernel/sched/membarrier.c +3
-0+return registrations_mask; +} +/** * sys_membarrier - issue memory barriers on a set of threads * @cmd: Takes command values defined in enum membarrier_cmd. kernel/sched/membarrier.c SYSCALL_DEFINE3(membarrier, int, cmd, unsigned int, flags, int, cpu_id) · line 665
Quoted 8 lines
kernel/sched/membarrier.c +2
-0@@ -623,6 +658,8 @@ SYSCALL_DEFINE3(membarrier, int, cmd, unsigned int, flags, int, cpu_id)return membarrier_private_expedited (MEMBARRIER_FLAG_RSEQ , cpu_id);case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ :return membarrier_register_private_expedited (MEMBARRIER_FLAG_RSEQ ); +case MEMBARRIER_CMD_GET_REGISTRATIONS : +return membarrier_get_registrations ();default :return -EINVAL ; }--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
Hi Mathieu,
is there anything more you need from my side?
Mathieu Desnoyers
maintainerRe: [PATCH 1/2] sched/membarrier: Introduce MEMBARRIER_CMD_GET_REGISTRATIONS
Quoted 28 lines · Michał Cłapiński
Quoted 27 lines · Michał Cłapiński
Quoted 24 lines · Mathieu Desnoyers
Quoted 22 lines · Michal Clapinski
Provide a method to query previously issued registrations.
Signed-off-by Michal Clapinski <[email protected]>--- include/uapi/linux/membarrier.h | 4 ++++ kernel/sched/membarrier.c | 39 ++++++++++++++++++++++++++++++++- 2 files changed, 42 insertions(+), 1 deletion(-)include/uapi/linux/membarrier.h +3
-0@@ -137,6 +137,9 @@* @MEMBARRIER_CMD_SHARED: * Alias to MEMBARRIER_CMD_GLOBAL. Provided for * header backward compatibility. +* @MEMBARRIER_CMD_GET_REGISTRATIONS: +* Returns a bitmask of previously issued +* registration commands. * * Command to be passed to the membarrier system call. The commands need to * be a single bit each, except for MEMBARRIER_CMD_QUERY which is assigned to
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
include/uapi/linux/membarrier.h enum membarrier_cmd { · line 159
Quoted 5 lines
include/uapi/linux/membarrier.h +1
-0@@ -153,6 +156,7 @@ enum membarrier_cmd {MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE = (1 <<6 ),MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ = (1 <<7 ),MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ = (1 <<8 ), + MEMBARRIER_CMD_GET_REGISTRATIONS = (1 << 9),Btw. I could do this as a flag to
MEMBARRIER_CMD_QUERYinstead of a separate command. Would that be preferable?
I do not think that would be better, no. We can keep it with GET_REGISTRATIONS.
Quoted 58 lines
Quoted 58 lines
Quoted 23 lines
Quoted 23 lines
include/uapi/linux/membarrier.h +0
-0/* Alias for header backward compatibility. */ MEMBARRIER_CMD_SHARED = MEMBARRIER_CMD_GLOBAL,kernel/sched/membarrier.c +2
-1@@ -159,7 +159,8 @@ | MEMBARRIER_CMD_PRIVATE_EXPEDITED \ | MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED \ | MEMBARRIER_PRIVATE_EXPEDITED_SYNC_CORE_BITMASK \ - | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK) + | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK \ + | MEMBARRIER_CMD_GET_REGISTRATIONS)static void ipi_mb (void *info) {@@ -540,6 +541,40 @@ static int membarrier_register_private_expedited(int flags)return 0 ; }Quoted 10 lines
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 551
Quoted 8 lines
kernel/sched/membarrier.c +8
-0+static int membarrier_get_registrations (void ) +{ +struct task_struct *p = current; +struct mm_struct *mm = p->mm; +int registrations_mask =0 , membarrier_state, i; +static const int states[] = { +MEMBARRIER_STATE_GLOBAL_EXPEDITED | +MEMBARRIER_STATE_GLOBAL_EXPEDITED_READY ,What is the purpose of checking for the
_READYstate flag as well here ?Answered below.
Quoted 21 lines
Quoted 20 lines
kernel/sched/membarrier.c +18
-0+MEMBARRIER_STATE_PRIVATE_EXPEDITED | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ_READY + }; +static const int registration_cmds[] = { +MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ + }; +BUILD_BUG_ON (ARRAY_SIZE (states) !=ARRAY_SIZE (registration_cmds)); + + membarrier_state =atomic_read (&mm->membarrier_state); +for (i =0 ; i <ARRAY_SIZE (states); ++i) { +if (membarrier_state & states[i]) {
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
The mask will match if either of the flags to match are set. Is that your intent ?
Kind of, it was just the easiest to write. As explained in the cover letter, I don't really care much about the result of this while the process is running. And when the process is frozen, either state and
state_readyare set or none of them.
OK
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
kernel/sched/membarrier.c +2
-0+ registrations_mask |= registration_cmds[i]; + membarrier_state &= ~states[i];
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
So I understand that those
_READYflags are there purely for making sure we clear themembarrier_statefor validation that they have all been checked with the followingWARN_ON_ONCE(). Am I on the right track ?Yes, exactly. It wastes time but I'm worried about people adding new states and not updating this function. A suggestion on how to do this better (especially at compile time) would be greatly appreciated.
Although it's not a fast-path, so let's keep it this way for now.
Quoted 33 lines
Quoted 24 lines
Quoted 24 lines
kernel/sched/membarrier.c static int membarrier_register_private_expedited(int flags) · line 574
Quoted 6 lines
kernel/sched/membarrier.c +3
-0+ } + } +WARN_ON_ONCE (membarrier_state !=0 );Thanks,
Mathieu
Quoted 13 lines
kernel/sched/membarrier.c +5
-0+return registrations_mask; +} +/** * sys_membarrier - issue memory barriers on a set of threads * @cmd: Takes command values defined in enum membarrier_cmd. @@ -623,6 +658,8 @@ SYSCALL_DEFINE3(membarrier, int, cmd, unsigned int, flags, int, cpu_id)return membarrier_private_expedited (MEMBARRIER_FLAG_RSEQ , cpu_id);case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ :return membarrier_register_private_expedited (MEMBARRIER_FLAG_RSEQ ); +case MEMBARRIER_CMD_GET_REGISTRATIONS : +return membarrier_get_registrations ();default :Quoted 8 lines
Quoted 8 lines
kernel/sched/membarrier.c SYSCALL_DEFINE3(membarrier, int, cmd, unsigned int, flags, int, cpu_id) · line 665
Quoted 2 lines
kernel/sched/membarrier.c +0
-0return -EINVAL ; }--
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
Quoted 2 lines
Hi Mathieu,
is there anything more you need from my side?
No, I think those patches are ok.
Thanks,
Mathieu
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com
[tip: sched/core] sched/membarrier: Introduce MEMBARRIER_CMD_GET_REGISTRATIONS
In reply to Michal ClapinskiThe following commit has been merged into the sched/core branch of tip:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 544a4f2ecd45f9d6ed78d207583f39130ad40349 Gitweb: https://git.kernel.org/tip/544a4f2ecd45f9d6ed78d207583f39130ad40349 Author: Michal Clapinski <[email protected]> AuthorDate: Wed, 07 Dec 2022 17:43:37 +01:00 Committer: Ingo Molnar <[email protected]> CommitterDate: Sat, 07 Jan 2023 11:29:29 +01:00
sched/membarrier: Introduce MEMBARRIER_CMD_GET_REGISTRATIONS
Provide a method to query previously issued registrations.
--- include/uapi/linux/membarrier.h | 4 +++- kernel/sched/membarrier.c | 39 +++++++++++++++++++++++++++++++- 2 files changed, 42 insertions(+), 1 deletion(-)
include/uapi/linux/membarrier.h +4 -0
@@ -137,6 +137,9 @@* @MEMBARRIER_CMD_SHARED: * Alias to MEMBARRIER_CMD_GLOBAL. Provided for * header backward compatibility. +* @MEMBARRIER_CMD_GET_REGISTRATIONS: +* Returns a bitmask of previously issued +* registration commands. * * Command to be passed to the membarrier system call. The commands need to * be a single bit each, except for MEMBARRIER_CMD_QUERY which is assigned to
@@ -153,6 +156,7 @@ enum membarrier_cmd {MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE = (1 <<6 ),MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ = (1 <<7 ),MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ = (1 <<8 ), +MEMBARRIER_CMD_GET_REGISTRATIONS = (1 <<9 ),/* Alias for header backward compatibility. */ MEMBARRIER_CMD_SHARED = MEMBARRIER_CMD_GLOBAL,
kernel/sched/membarrier.c +38 -1
@@ -159,7 +159,8 @@ | MEMBARRIER_CMD_PRIVATE_EXPEDITED \ | MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED \ | MEMBARRIER_PRIVATE_EXPEDITED_SYNC_CORE_BITMASK \ - | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK) + | MEMBARRIER_PRIVATE_EXPEDITED_RSEQ_BITMASK \ + | MEMBARRIER_CMD_GET_REGISTRATIONS)static void ipi_mb (void *info) {
@@ -540,6 +541,40 @@ static int membarrier_register_private_expedited(int flags)return 0 ; } +static int membarrier_get_registrations (void ) +{ +struct task_struct *p = current; +struct mm_struct *mm = p->mm; +int registrations_mask =0 , membarrier_state, i; +static const int states[] = { +MEMBARRIER_STATE_GLOBAL_EXPEDITED | +MEMBARRIER_STATE_GLOBAL_EXPEDITED_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_READY , +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ | +MEMBARRIER_STATE_PRIVATE_EXPEDITED_RSEQ_READY + }; +static const int registration_cmds[] = { +MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE , +MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ + }; +BUILD_BUG_ON (ARRAY_SIZE (states) !=ARRAY_SIZE (registration_cmds)); + + membarrier_state =atomic_read (&mm->membarrier_state); +for (i =0 ; i <ARRAY_SIZE (states); ++i) { +if (membarrier_state & states[i]) { + registrations_mask |= registration_cmds[i]; + membarrier_state &= ~states[i]; + } + } +WARN_ON_ONCE (membarrier_state !=0 ); +return registrations_mask; +} +/** * sys_membarrier - issue memory barriers on a set of threads * @cmd: Takes command values defined in enum membarrier_cmd.
@@ -623,6 +658,8 @@ SYSCALL_DEFINE3(membarrier, int, cmd, unsigned int, flags, int, cpu_id)return membarrier_private_expedited (MEMBARRIER_FLAG_RSEQ , cpu_id);case MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_RSEQ :return membarrier_register_private_expedited (MEMBARRIER_FLAG_RSEQ ); +case MEMBARRIER_CMD_GET_REGISTRATIONS : +return membarrier_get_registrations ();default :return -EINVAL ; }
PATCH 2/2 selftests/membarrier: Test MEMBARRIER_CMD_GET_REGISTRATIONS
In reply to Michal ClapinskiKeep track of previously issued registrations and compare the result with MEMBARRIER_CMD_GET_REGISTRATIONS return value.
--- .../membarrier/membarrier_test_impl.h | 33 +++++++++++++++++++ .../membarrier/membarrier_test_multi_thread.c | 2 +- .../membarrier_test_single_thread.c | 6 +++- 3 files changed, 39 insertions(+), 2 deletions(-)
tools/testing/selftests/membarrier/membarrier_test_impl.h +33 -0
@@ -9,11 +9,38 @@#include "../kselftest.h" +static int registrations; +static int sys_membarrier (int cmd,int flags) {return syscall (__NR_membarrier, cmd, flags); } +static int test_membarrier_get_registrations (int cmd) +{ +int ret, flags =0 ; +const char *test_name = +"sys membarrier MEMBARRIER_CMD_GET_REGISTRATIONS" ; + + registrations |= cmd; + + ret =sys_membarrier (MEMBARRIER_CMD_GET_REGISTRATIONS ,0 ); +if (ret <0 ) { +ksft_exit_fail_msg ( +"%s test: flags = %d, errno = %d\n" , + test_name, flags, errno); + }else if (ret != registrations) { +ksft_exit_fail_msg ( +"%s test: flags = %d, ret = %d, registrations = %d\n" , + test_name, flags, ret, registrations); + } +ksft_test_result_pass ( +"%s test: flags = %d, ret = %d, registrations = %d\n" , + test_name, flags, ret, registrations); + +return 0 ; +} +static int test_membarrier_cmd_fail (void ) {int cmd =-1 , flags =0 ;
@@ -113,6 +140,8 @@ static int test_membarrier_register_private_expedited_success(void)ksft_test_result_pass ("%s test: flags = %d\n" , test_name, flags); + +test_membarrier_get_registrations (cmd);return 0 ; }
@@ -170,6 +199,8 @@ static int test_membarrier_register_private_expedited_sync_core_success(void)ksft_test_result_pass ("%s test: flags = %d\n" , test_name, flags); + +test_membarrier_get_registrations (cmd);return 0 ; }
@@ -204,6 +235,8 @@ static int test_membarrier_register_global_expedited_success(void)ksft_test_result_pass ("%s test: flags = %d\n" , test_name, flags); + +test_membarrier_get_registrations (cmd);return 0 ; }
tools/testing/selftests/membarrier/membarrier_test_multi_thread.c +1 -1
@@ -62,7 +62,7 @@ static int test_mt_membarrier(void)int main (int argc,char **argv) {ksft_print_header (); -ksft_set_plan (13 ); +ksft_set_plan (16 );test_membarrier_query ();
tools/testing/selftests/membarrier/membarrier_test_single_thread.c +5 -1
@@ -12,7 +12,9 @@int main (int argc,char **argv) {ksft_print_header (); -ksft_set_plan (13 ); +ksft_set_plan (18 ); + +test_membarrier_get_registrations (/*cmd=*/ 0 );test_membarrier_query ();
@@ -20,5 +22,7 @@ int main(int argc, char **argv)test_membarrier_success (); +test_membarrier_get_registrations (/*cmd=*/ 0 ); +return ksft_exit_pass (); }
Signature · 1 line
2.39.0.rc0.267.gcb52ba06e7-goog
[tip: sched/core] selftests/membarrier: Test MEMBARRIER_CMD_GET_REGISTRATIONS
The following commit has been merged into the sched/core branch of tip:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: d74f87f37672e71457bfcc14eca5eeb1d61b6438 Gitweb: https://git.kernel.org/tip/d74f87f37672e71457bfcc14eca5eeb1d61b6438 Author: Michal Clapinski <[email protected]> AuthorDate: Wed, 07 Dec 2022 17:43:38 +01:00 Committer: Ingo Molnar <[email protected]> CommitterDate: Sat, 07 Jan 2023 11:29:29 +01:00
selftests/membarrier: Test MEMBARRIER_CMD_GET_REGISTRATIONS
Keep track of previously issued registrations and compare the result with MEMBARRIER_CMD_GET_REGISTRATIONS return value.
--- tools/testing/selftests/membarrier/membarrier_test_impl.h | 33 +++++++++++++++++++++++++++++++++ tools/testing/selftests/membarrier/membarrier_test_multi_thread.c | 2 +- tools/testing/selftests/membarrier/membarrier_test_single_thread.c | 6 +++++- 3 files changed, 39 insertions(+), 2 deletions(-)
tools/testing/selftests/membarrier/membarrier_test_impl.h +33 -0
@@ -9,11 +9,38 @@#include "../kselftest.h" +static int registrations; +static int sys_membarrier (int cmd,int flags) {return syscall (__NR_membarrier, cmd, flags); } +static int test_membarrier_get_registrations (int cmd) +{ +int ret, flags =0 ; +const char *test_name = +"sys membarrier MEMBARRIER_CMD_GET_REGISTRATIONS" ; + + registrations |= cmd; + + ret =sys_membarrier (MEMBARRIER_CMD_GET_REGISTRATIONS ,0 ); +if (ret <0 ) { +ksft_exit_fail_msg ( +"%s test: flags = %d, errno = %d\n" , + test_name, flags, errno); + }else if (ret != registrations) { +ksft_exit_fail_msg ( +"%s test: flags = %d, ret = %d, registrations = %d\n" , + test_name, flags, ret, registrations); + } +ksft_test_result_pass ( +"%s test: flags = %d, ret = %d, registrations = %d\n" , + test_name, flags, ret, registrations); + +return 0 ; +} +static int test_membarrier_cmd_fail (void ) {int cmd =-1 , flags =0 ;
@@ -113,6 +140,8 @@ static int test_membarrier_register_private_expedited_success(void)ksft_test_result_pass ("%s test: flags = %d\n" , test_name, flags); + +test_membarrier_get_registrations (cmd);return 0 ; }
@@ -170,6 +199,8 @@ static int test_membarrier_register_private_expedited_sync_core_success(void)ksft_test_result_pass ("%s test: flags = %d\n" , test_name, flags); + +test_membarrier_get_registrations (cmd);return 0 ; }
@@ -204,6 +235,8 @@ static int test_membarrier_register_global_expedited_success(void)ksft_test_result_pass ("%s test: flags = %d\n" , test_name, flags); + +test_membarrier_get_registrations (cmd);return 0 ; }
tools/testing/selftests/membarrier/membarrier_test_multi_thread.c +1 -1
@@ -62,7 +62,7 @@ static int test_mt_membarrier(void)int main (int argc,char **argv) {ksft_print_header (); -ksft_set_plan (13 ); +ksft_set_plan (16 );test_membarrier_query ();
tools/testing/selftests/membarrier/membarrier_test_single_thread.c +5 -1
@@ -12,7 +12,9 @@int main (int argc,char **argv) {ksft_print_header (); -ksft_set_plan (13 ); +ksft_set_plan (18 ); + +test_membarrier_get_registrations (/*cmd=*/ 0 );test_membarrier_query ();
@@ -20,5 +22,7 @@ int main(int argc, char **argv)test_membarrier_success (); +test_membarrier_get_registrations (/*cmd=*/ 0 ); +return ksft_exit_pass (); }
Quoted 10 lines · Michal Clapinski
This change provides a method to query previously issued registrations. It's needed for CRIU (checkpoint/restore in userspace). Before this change we had to issue private membarrier commands during checkpoint - if they succeeded, they must have been registered. Unfortunately global membarrier succeeds even on unregistered processes, so there was no way to tell if
MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITEDhad been issued or not.CRIU is run after the process has been frozen with ptrace, so we don't have to worry too much about the result of running this command in parallel with registration commands.
Peter, Paul, I'm OK with the proposed changes. Should we route this through sched/core from the tip tree ?
For both patches:
Thanks,
Mathieu
Quoted 12 lines
Michal Clapinski (2):
sched/membarrier: IntroduceMEMBARRIER_CMD_GET_REGISTRATIONSselftests/membarrier: TestMEMBARRIER_CMD_GET_REGISTRATIONSinclude/uapi/linux/membarrier.h | 4 ++ kernel/sched/membarrier.c | 39 ++++++++++++++++++- .../membarrier/membarrier_test_impl.h | 33 ++++++++++++++++ .../membarrier/membarrier_test_multi_thread.c | 2 +- .../membarrier_test_single_thread.c | 6 ++- 5 files changed, 81 insertions(+), 3 deletions(-)
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com
Quoted 12 lines · Mathieu Desnoyers
Quoted 10 lines · Michal Clapinski
This change provides a method to query previously issued registrations. It's needed for CRIU (checkpoint/restore in userspace). Before this change we had to issue private membarrier commands during checkpoint - if they succeeded, they must have been registered. Unfortunately global membarrier succeeds even on unregistered processes, so there was no way to tell if
MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITEDhad been issued or not.CRIU is run after the process has been frozen with ptrace, so we don't have to worry too much about the result of running this command in parallel with registration commands.
Quoted 6 lines
Peter, Paul, I'm OK with the proposed changes. Should we route this through sched/core from the tip tree ?
For both patches:
Also for both patches:
Quoted 22 lines
Thanks,
Mathieu
Quoted 12 lines
Michal Clapinski (2):
sched/membarrier: IntroduceMEMBARRIER_CMD_GET_REGISTRATIONSselftests/membarrier: TestMEMBARRIER_CMD_GET_REGISTRATIONSinclude/uapi/linux/membarrier.h | 4 ++ kernel/sched/membarrier.c | 39 ++++++++++++++++++- .../membarrier/membarrier_test_impl.h | 33 ++++++++++++++++ .../membarrier/membarrier_test_multi_thread.c | 2 +- .../membarrier_test_single_thread.c | 6 ++- 5 files changed, 81 insertions(+), 3 deletions(-)Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com