Selective mitigation for trusted userspace
Subsystems (6)
Pawan Gupta
reviewercover Selective mitigation for trusted userspace
Hi,
This is an experimental series exploring the feasibility of selectively applying CPU vulnerability mitigations on a per-process basis. The motivation behind this work is to address the performance degradation experienced by trusted user-space applications due to system-wide CPU mitigations.
Currently, the mitigations are applied universally across the system, without discrimination between trusted and untrusted user-space processes. This results in a performance penalty for all applications, regardless of their trustworthiness. The proposed solution aims to provide a mechanism for system administrators to explicitly mark certain applications as trusted, allowing them to bypass these mitigations and regain lost performance.
The series introduces a new cgroup attribute and a separate kernel entry/exit path that can be used to selectively disable CPU mitigations for processes that are deemed trustworthy by the system administrator. This approach provides a tool to the administrator who understands the security implications and is aware of trustworthiness of the applications that they care.
The rationale for choosing the cgroup interface over other potential interfaces, such as LSMs, is cgroup's inherent support for core scheduling. Core scheduling allows the grouping of tasks such that they are scheduled to run on the same cores. By leveraging core scheduling, we can minimize the performance overhead caused by the MSR writes during context switching between trusted and untrusted processes. With the end goal being trusted and untrusted processes run on separate cores, enhancing the security.
Patch 1 adds the unmitigated entry/exit path. Patch 2 provides a cgroup knob to bypass CPU mitigations.
This series is lightly tested. Feedback and discussion are welcome.
TODO:
- Add CONFIG_MITIGATION_PER_PROCESS
- Add support for skipping other mitigations like RSB filling.
- Update suspend/resume paths to handle the new entry/exit path.
- Should child processes inherit the parent's unmitigated status?
- Add documentation.
---
Pawan Gupta (2):
x86/entry_64: Add a separate unmitigated entry/exit path cpu/bugs: cgroup: Add a cgroup knob to bypass CPU mitigations
arch/x86/entry/entry_64.S | 66 +++++++++++++++++++++++++++++++++------- arch/x86/include/asm/proto.h | 15 ++++++--- arch/x86/include/asm/ptrace.h | 15 ++++++--- arch/x86/include/asm/switch_to.h | 10 ++++++ arch/x86/kernel/cpu/bugs.c | 21 +++++++++++++ arch/x86/kernel/cpu/common.c | 2 +- include/linux/cgroup-defs.h | 3 ++ kernel/cgroup/cgroup.c | 42 +++++++++++++++++++++++++ kernel/sched/core.c | 2 +- 9 files changed, 155 insertions(+), 21 deletions(-)
---
base-commit: 98f7e32f20d28ec452afb208f9cffc08448a2652
change-id: 20240919-selective-mitigation-6d02c4bbb72b
Signature · 2 lines
Thanks, Pawan
Pawan Gupta
reviewerRFC PATCH 1/2 x86/entry_64: Add a separate unmitigated entry/exit path
CPU mitigations are deployed system-wide, but usually not all of the userspace is malicious. Yet, they suffer from the performance impact of the mitigations. This all or nothing approach is due to lack of a way for kernel to know which userspace can be trusted and which cannot.
For scenarios where an admin can decide which processes to trust, an interface to tell the kernel to possibly skip the mitigation would be useful.
In preparation for kernel to be able to selectively apply mitigation per-process add a separate kernel entry/exit path that skips the mitigations.
--- arch/x86/entry/entry_64.S | 66 +++++++++++++++++++++++++++++++++++-------- arch/x86/include/asm/proto.h | 15 +++++++--- arch/x86/include/asm/ptrace.h | 15 +++++++--- arch/x86/kernel/cpu/common.c | 2 +- 4 files changed, 78 insertions(+), 20 deletions(-)
arch/x86/entry/entry_64.S +55 -11
@@ -84,7 +84,7 @@ * with them due to bugs in both AMD and Intel CPUs. */ -SYM_CODE_START(entry_SYSCALL_64) +.macro __entry_SYSCALL_64 mitigated=0 UNWIND_HINT_ENTRY ENDBR
@@ -94,7 +94,12 @@ SYM_CODE_START(entry_SYSCALL_64) SWITCH_TO_KERNEL_CR3 scratch_reg=%rsp movq PER_CPU_VAR(pcpu_hot + X86_top_of_stack), %rsp -SYM_INNER_LABEL(entry_SYSCALL_64_safe_stack, SYM_L_GLOBAL) +.if \mitigated +SYM_INNER_LABEL(entry_SYSCALL_64_safe_stack_mitigated, SYM_L_GLOBAL) +.else +SYM_INNER_LABEL(entry_SYSCALL_64_safe_stack_unmitigated, SYM_L_GLOBAL) +.endif + ANNOTATE_NOENDBR /* Construct struct pt_regs on stack */
@@ -103,7 +108,11 @@ SYM_INNER_LABEL(entry_SYSCALL_64_safe_stack, SYM_L_GLOBAL) pushq %r11 /* pt_regs->flags */ pushq $__USER_CS /* pt_regs->cs */ pushq %rcx /* pt_regs->ip */ + +.if \mitigated SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL) +.endif + pushq %rax /* pt_regs->orig_ax */ PUSH_AND_CLEAR_REGS rax=$-ENOSYS
@@ -113,10 +122,12 @@ SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL) /* Sign extend the lower 32bit as syscall numbers are treated as int */ movslq %eax, %rsi +.if \mitigated /* clobbers %rax, make sure it is after saving the syscall nr */ IBRS_ENTER UNTRAIN_RET CLEAR_BRANCH_HISTORY +.endif call do_syscall_64 /* returns with IRQs disabled */
@@ -127,15 +138,26 @@ SYM_INNER_LABEL(entry_SYSCALL_64_after_hwframe, SYM_L_GLOBAL) * In the Xen PV case we must use iret anyway. */ - ALTERNATIVE "testb %al, %al; jz swapgs_restore_regs_and_return_to_usermode", \ - "jmp swapgs_restore_regs_and_return_to_usermode", X86_FEATURE_XENPV +.if \mitigated + push %rax + IBRS_EXIT + CLEAR_CPU_BUFFERS + pop %rax +.endif + + ALTERNATIVE "testb %al, %al; jz swapgs_restore_regs_and_return_to_usermode_from_syscall", \ + "jmp swapgs_restore_regs_and_return_to_usermode_from_syscall", X86_FEATURE_XENPV /* * We win! This label is here just for ease of understanding * perf profiles. Nothing jumps here. */ -syscall_return_via_sysret: - IBRS_EXIT +.if \mitigated +syscall_return_via_sysret_mitigated: +.else +syscall_return_via_sysret_unmitigated: +.endif + POP_REGS pop_rdi=0 /*
@@ -159,15 +181,36 @@ syscall_return_via_sysret: popq %rdi popq %rsp -SYM_INNER_LABEL(entry_SYSRETQ_unsafe_stack, SYM_L_GLOBAL) + +.if \mitigated +SYM_INNER_LABEL(entry_SYSRETQ_unsafe_stack_mitigated, SYM_L_GLOBAL) +.else +SYM_INNER_LABEL(entry_SYSRETQ_unsafe_stack_unmitigated, SYM_L_GLOBAL) +.endif + ANNOTATE_NOENDBR swapgs - CLEAR_CPU_BUFFERS + +.if \mitigated +SYM_INNER_LABEL(entry_SYSRETQ_end_mitigated, SYM_L_GLOBAL) +.else +SYM_INNER_LABEL(entry_SYSRETQ_end_unmitigated, SYM_L_GLOBAL) +.endif sysretq -SYM_INNER_LABEL(entry_SYSRETQ_end, SYM_L_GLOBAL) + +.endm /* __entry_SYSCALL_64 */ + +SYM_CODE_START(entry_SYSCALL_64_unmitigated) + __entry_SYSCALL_64 mitigated=0 ANNOTATE_NOENDBR int3 -SYM_CODE_END(entry_SYSCALL_64) +SYM_CODE_END(entry_SYSCALL_64_unmitigated) + +SYM_CODE_START(entry_SYSCALL_64_mitigated) + __entry_SYSCALL_64 mitigated=1 + ANNOTATE_NOENDBR + int3 +SYM_CODE_END(entry_SYSCALL_64_mitigated) /* * %rdi: prev task
@@ -559,6 +602,8 @@ __irqentry_text_end: SYM_CODE_START_LOCAL(common_interrupt_return) SYM_INNER_LABEL(swapgs_restore_regs_and_return_to_usermode, SYM_L_GLOBAL) IBRS_EXIT + CLEAR_CPU_BUFFERS +SYM_INNER_LABEL(swapgs_restore_regs_and_return_to_usermode_from_syscall, SYM_L_GLOBAL) #ifdef CONFIG_XEN_PV ALTERNATIVE "", "jmp xenpv_restore_regs_and_return_to_usermode", X86_FEATURE_XENPV #endif
@@ -573,7 +618,6 @@ SYM_INNER_LABEL(swapgs_restore_regs_and_return_to_usermode, SYM_L_GLOBAL) .Lswapgs_and_iret: swapgs - CLEAR_CPU_BUFFERS /* Assert that the IRET frame indicates user mode. */ testb $3, 8(%rsp) jnz .Lnative_iret
arch/x86/include/asm/proto.h +11 -4
@@ -11,10 +11,17 @@ struct task_struct;void syscall_init (void );#ifdef CONFIG_X86_64 -void entry_SYSCALL_64 (void ); -void entry_SYSCALL_64_safe_stack (void ); -void entry_SYSRETQ_unsafe_stack (void ); -void entry_SYSRETQ_end (void ); + +void entry_SYSCALL_64_unmitigated (void ); +void entry_SYSCALL_64_safe_stack_unmitigated (void ); +void entry_SYSRETQ_unsafe_stack_unmitigated (void ); +void entry_SYSRETQ_end_unmitigated (void ); + +void entry_SYSCALL_64_mitigated (void ); +void entry_SYSCALL_64_safe_stack_mitigated (void ); +void entry_SYSRETQ_unsafe_stack_mitigated (void ); +void entry_SYSRETQ_end_mitigated (void ); +long do_arch_prctl_64 (struct task_struct *task,int option,unsigned long arg2);#endif
arch/x86/include/asm/ptrace.h +11 -4
@@ -261,11 +261,18 @@ static inline bool any_64bit_mode(struct pt_regs *regs) static __always_inline bool ip_within_syscall_gap(struct pt_regs *regs) { -bool ret = (regs->ip >= (unsigned long )entry_SYSCALL_64 && - regs->ip < (unsigned long )entry_SYSCALL_64_safe_stack); +bool ret = (regs->ip >= (unsigned long )entry_SYSCALL_64_unmitigated && + regs->ip < (unsigned long )entry_SYSCALL_64_safe_stack_unmitigated); + + ret = ret || (regs->ip >= (unsigned long )entry_SYSRETQ_unsafe_stack_unmitigated && + regs->ip < (unsigned long )entry_SYSRETQ_end_unmitigated); + + ret = ret || (regs->ip >= (unsigned long )entry_SYSCALL_64_mitigated && + regs->ip < (unsigned long )entry_SYSCALL_64_safe_stack_mitigated); + + ret = ret || (regs->ip >= (unsigned long )entry_SYSRETQ_unsafe_stack_mitigated && + regs->ip < (unsigned long )entry_SYSRETQ_end_mitigated); - ret = ret || (regs->ip >= (unsigned long )entry_SYSRETQ_unsafe_stack && - regs->ip < (unsigned long )entry_SYSRETQ_end);#ifdef CONFIG_IA32_EMULATION ret = ret || (regs->ip >= (unsigned long )entry_SYSCALL_compat && regs->ip < (unsigned long )entry_SYSCALL_compat_safe_stack);
arch/x86/kernel/cpu/common.c +1 -1
@@ -2026,7 +2026,7 @@ static void wrmsrl_cstar(unsigned long val)static inline void idt_syscall_init (void ) { -wrmsrl (MSR_LSTAR , (unsigned long )entry_SYSCALL_64); +wrmsrl (MSR_LSTAR , (unsigned long )entry_SYSCALL_64_unmitigated);if (ia32_enabled ()) {wrmsrl_cstar ((unsigned long )entry_SYSCALL_compat);
Signature · 1 line
2.34.1
kernel test robot
build botRe: [PATCH RFC 1/2] x86/entry_64: Add a separate unmitigated entry/exit path
kernel test robot noticed the following build warnings:
Hi Pawan,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:
[auto build test WARNING on 98f7e32f20d28ec452afb208f9cffc08448a2652]
url: https://github.com/intel-lab-lkp/linux/commits/Pawan-Gupta/x86-entry_64-Add-a-separate-unmitigated-entry-exit-path/20240920-055414 base: 98f7e32f20d28ec452afb208f9cffc08448a2652 patch link: https://lore.kernel.org/r/20240919-selective-mitigation-v1-1-1846cf41895e%40linux.intel.com patch subject: [PATCH RFC 1/2] x86/entry_64: Add a separate unmitigated entry/exit path config: x86_64-rhel-8.3 (https://download.01.org/0day-ci/archive/20240920/[email protected]/config) compiler: gcc-12 (Debian 12.2.0-14) 12.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240920/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> vmlinux.o: warning: objtool: __sev_es_ist_enter+0x35: relocation to !ENDBR: entry_SYSCALL_64_unmitigated+0xf7 >> vmlinux.o: warning: objtool: vc_switch_off_ist+0x9f: relocation to !ENDBR: entry_SYSCALL_64_unmitigated+0xf7
objdump-func vmlinux.o __sev_es_ist_enter:
0000 0000000000000410 <__sev_es_ist_enter>:
0000 410: f3 0f 1e fa endbr64 0004 414: 65 48 8b 0d 00 00 00 00 mov %gs:0x0(%rip),%rcx # 41c <__sev_es_ist_enter+0xc> 418: R_X86_64_PC32 cpu_tss_rw+0x40 000c 41c: 48 8b 87 98 00 00 00 mov 0x98(%rdi),%rax 0013 423: f6 87 88 00 00 00 03 testb $0x3,0x88(%rdi) 001a 42a: 75 22 jne 44e <__sev_es_ist_enter+0x3e> 001c 42c: 48 8b 97 80 00 00 00 mov 0x80(%rdi),%rdx 0023 433: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 436: R_X86_64_32S entry_SYSCALL_64_unmitigated 002a 43a: 73 2a jae 466 <__sev_es_ist_enter+0x56> 002c 43c: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 43f: R_X86_64_32S entry_SYSRETQ_unsafe_stack_unmitigated 0033 443: 72 33 jb 478 <__sev_es_ist_enter+0x68> 0035 445: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 448: R_X86_64_32S entry_SYSRETQ_end_unmitigated 003c 44c: 73 2a jae 478 <__sev_es_ist_enter+0x68> 003e 44e: 48 89 c8 mov %rcx,%rax 0041 451: 48 8d 50 f8 lea -0x8(%rax),%rdx 0045 455: 48 89 48 f8 mov %rcx,-0x8(%rax) 0049 459: 65 48 89 15 00 00 00 00 mov %rdx,%gs:0x0(%rip) # 461 <__sev_es_ist_enter+0x51> 45d: R_X86_64_PC32 cpu_tss_rw+0x40 0051 461: e9 00 00 00 00 jmp 466 <__sev_es_ist_enter+0x56> 462: R_X86_64_PLT32 __x86_return_thunk-0x4 0056 466: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 469: R_X86_64_32S entry_SYSCALL_64_safe_stack_unmitigated 005d 46d: 72 df jb 44e <__sev_es_ist_enter+0x3e> 005f 46f: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 472: R_X86_64_32S entry_SYSRETQ_unsafe_stack_unmitigated 0066 476: 73 cd jae 445 <__sev_es_ist_enter+0x35> 0068 478: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 47b: R_X86_64_32S entry_SYSCALL_64_mitigated 006f 47f: 72 09 jb 48a <__sev_es_ist_enter+0x7a> 0071 481: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 484: R_X86_64_32S entry_SYSCALL_64_safe_stack_mitigated 0078 488: 72 c4 jb 44e <__sev_es_ist_enter+0x3e> 007a 48a: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 48d: R_X86_64_32S entry_SYSRETQ_unsafe_stack_mitigated 0081 491: 72 09 jb 49c <__sev_es_ist_enter+0x8c> 0083 493: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 496: R_X86_64_32S entry_SYSRETQ_end_mitigated 008a 49a: 72 b2 jb 44e <__sev_es_ist_enter+0x3e> 008c 49c: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 49f: R_X86_64_32S entry_SYSCALL_compat 0093 4a3: 73 39 jae 4de <__sev_es_ist_enter+0xce> 0095 4a5: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 4a8: R_X86_64_32S entry_SYSRETL_compat_unsafe_stack 009c 4ac: 72 09 jb 4b7 <__sev_es_ist_enter+0xa7> 009e 4ae: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 4b1: R_X86_64_32S entry_SYSRETL_compat_end 00a5 4b5: 72 97 jb 44e <__sev_es_ist_enter+0x3e> 00a7 4b7: 65 48 8b 15 00 00 00 00 mov %gs:0x0(%rip),%rdx # 4bf <__sev_es_ist_enter+0xaf> 4bb: R_X86_64_PC32 cea_exception_stacks-0x4 00af 4bf: 48 8d b2 00 d0 00 00 lea 0xd000(%rdx),%rsi 00b6 4c6: 48 39 f0 cmp %rsi,%rax 00b9 4c9: 72 83 jb 44e <__sev_es_ist_enter+0x3e> 00bb 4cb: 48 81 c2 00 f0 00 00 add $0xf000,%rdx 00c2 4d2: 48 39 d0 cmp %rdx,%rax 00c5 4d5: 48 0f 43 c1 cmovae %rcx,%rax 00c9 4d9: e9 73 ff ff ff jmp 451 <__sev_es_ist_enter+0x41> 00ce 4de: 48 81 fa 00 00 00 00 cmp $0x0,%rdx 4e1: R_X86_64_32S entry_SYSCALL_compat_safe_stack 00d5 4e5: 0f 82 63 ff ff ff jb 44e <__sev_es_ist_enter+0x3e> 00db 4eb: eb b8 jmp 4a5 <__sev_es_ist_enter+0x95> 00dd 4ed: 66 66 2e 0f 1f 84 00 00 00 00 00 data16 cs nopw 0x0(%rax,%rax,1) 00e8 4f8: 0f 1f 84 00 00 00 00 00 nopl 0x0(%rax,%rax,1)
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Re: [PATCH RFC 1/2] x86/entry_64: Add a separate unmitigated entry/exit path
In reply to Pawan GuptaQuoted 9 lines · Pawan Gupta
CPU mitigations are deployed system-wide, but usually not all of the userspace is malicious. Yet, they suffer from the performance impact of the mitigations. This all or nothing approach is due to lack of a way for kernel to know which userspace can be trusted and which cannot.
For scenarios where an admin can decide which processes to trust, an interface to tell the kernel to possibly skip the mitigation would be useful.
Quoted 6 lines
In preparation for kernel to be able to selectively apply mitigation per-process add a separate kernel entry/exit path that skips the mitigations.
Originally-by Josh Poimboeuf <[email protected]>Signed-off-by Pawan Gupta <[email protected]>
For the current patch, not all x86 CPU vulnerability mitigations can be disabled. Maybe we should list the subset of mitigations that can be disabled.
Cheers,
Longman
Pawan Gupta
reviewerRe: [PATCH RFC 1/2] x86/entry_64: Add a separate unmitigated entry/exit path
Quoted 15 lines · Waiman Long
Quoted 13 lines · Pawan Gupta
CPU mitigations are deployed system-wide, but usually not all of the userspace is malicious. Yet, they suffer from the performance impact of the mitigations. This all or nothing approach is due to lack of a way for kernel to know which userspace can be trusted and which cannot.
For scenarios where an admin can decide which processes to trust, an interface to tell the kernel to possibly skip the mitigation would be useful.
In preparation for kernel to be able to selectively apply mitigation per-process add a separate kernel entry/exit path that skips the mitigations.
Quoted 6 lines
Quoted 2 lines
Originally-by Josh Poimboeuf <[email protected]>Signed-off-by Pawan Gupta <[email protected]>For the current patch, not all x86 CPU vulnerability mitigations can be disabled. Maybe we should list the subset of mitigations that can be disabled.
Yes, will update that mitigations that can be bypassed are BHI, VERW, Retbleed-IBRS, Retbleed-unret and IBPB.
Meltdown, Spectre-v1, eIBRS, GDS, SRBDS, retpoline and rethunk stays enabled.
Pawan Gupta
reviewerRFC PATCH 2/2 cpu/bugs: cgroup: Add a cgroup knob to bypass CPU mitigations
In reply to Pawan GuptaFor cases where an admin wanting to bypass CPU mitigations for a specific workload that they trust. Add a cgroup attribute "cpu.skip_mitigation" that can only be set by a privileged user. When set, the CPU mitigations are bypassed for all tasks in that cgroup.
Before setting this knob, the admin should be aware of possible security risks like confused deputy attack on trusted interpreters, JIT engine, etc.
--- arch/x86/include/asm/switch_to.h | 10 ++++++++++ arch/x86/kernel/cpu/bugs.c | 21 ++++++++++++++++++++ include/linux/cgroup-defs.h | 3 +++ kernel/cgroup/cgroup.c | 42 ++++++++++++++++++++++++++++++++++++++++ kernel/sched/core.c | 2 +- 5 files changed, 77 insertions(+), 1 deletion(-)
arch/x86/include/asm/switch_to.h +10 -0
@@ -46,6 +46,16 @@ struct fork_frame {struct pt_regs regs; }; +extern inline void cpu_mitigation_skip (struct task_struct *prev,struct task_struct *next); + +#define prepare_arch_switch prepare_arch_switch + +static inline void prepare_arch_switch (struct task_struct *prev, +struct task_struct *next) +{ +cpu_mitigation_skip (prev, next); +} +#define switch_to (prev, next, last) \ do { \ ((last) = __switch_to_asm((prev), (next))); \
arch/x86/kernel/cpu/bugs.c +21 -0
@@ -128,6 +128,27 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush);DEFINE_STATIC_KEY_FALSE (mmio_stale_data_clear);EXPORT_SYMBOL_GPL (mmio_stale_data_clear); +inline void cpu_mitigation_skip (struct task_struct *prev, +struct task_struct *next) +{ +bool prev_skip = false, next_skip = false; + +if (prev->mm) + prev_skip =task_dfl_cgroup (prev)->cpu_skip_mitigation; +if (next->mm) + next_skip =task_dfl_cgroup (next)->cpu_skip_mitigation; + +if (!prev_skip && !next_skip) +return ; +if (prev_skip == next_skip) +return ; + +if (next_skip) +wrmsrl (MSR_LSTAR , (unsigned long )entry_SYSCALL_64_unmitigated); +else +wrmsrl (MSR_LSTAR , (unsigned long )entry_SYSCALL_64_mitigated); +} +void __init cpu_select_mitigations (void ) {/*
include/linux/cgroup-defs.h +3 -0
@@ -546,6 +546,9 @@ struct cgroup {struct bpf_local_storage __rcu *bpf_cgrp_storage;#endif +/* Used to bypass the CPU mitigations for tasks in a cgroup */ +bool cpu_skip_mitigation; +/* All ancestors including self */ struct cgroup *ancestors[]; };
kernel/cgroup/cgroup.c +42 -0
60 lines not on this page · raw message
kernel/sched/core.c +1 -1
@@ -4994,7 +4994,7 @@ prepare_task_switch(struct rq *rq, struct task_struct *prev,fire_sched_out_preempt_notifiers (prev, next);kmap_local_sched_out ();prepare_task (next); -prepare_arch_switch (next); +prepare_arch_switch (prev, next); }/**
Signature · 1 line
2.34.1
Re: [PATCH RFC 2/2] cpu/bugs: cgroup: Add a cgroup knob to bypass CPU mitigations
Quoted 51 lines · Pawan Gupta
For cases where an admin wanting to bypass CPU mitigations for a specific workload that they trust. Add a cgroup attribute "cpu.skip_mitigation" that can only be set by a privileged user. When set, the CPU mitigations are bypassed for all tasks in that cgroup.
Before setting this knob, the admin should be aware of possible security risks like confused deputy attack on trusted interpreters, JIT engine, etc.
Signed-off-by Pawan Gupta <[email protected]>--- arch/x86/include/asm/switch_to.h | 10 ++++++++++ arch/x86/kernel/cpu/bugs.c | 21 ++++++++++++++++++++ include/linux/cgroup-defs.h | 3 +++ kernel/cgroup/cgroup.c | 42 ++++++++++++++++++++++++++++++++++++++++ kernel/sched/core.c | 2 +- 5 files changed, 77 insertions(+), 1 deletion(-)arch/x86/include/asm/switch_to.h +10
-0@@ -46,6 +46,16 @@ struct fork_frame {struct pt_regs regs; }; +extern inline void cpu_mitigation_skip (struct task_struct *prev,struct task_struct *next); + +#define prepare_arch_switch prepare_arch_switch + +static inline void prepare_arch_switch (struct task_struct *prev, +struct task_struct *next) +{ +cpu_mitigation_skip (prev, next); +} +#define switch_to (prev, next, last) \ do { \ ((last) = __switch_to_asm((prev), (next))); \arch/x86/kernel/cpu/bugs.c +4
-0@@ -128,6 +128,27 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush);DEFINE_STATIC_KEY_FALSE (mmio_stale_data_clear);EXPORT_SYMBOL_GPL (mmio_stale_data_clear); +inline void cpu_mitigation_skip (struct task_struct *prev, +struct task_struct *next) +{ +bool prev_skip = false, next_skip = false;
arch/x86/kernel/cpu/bugs.c DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush); · line 144
Quoted 10 lines
arch/x86/kernel/cpu/bugs.c +10
-0+ +if (prev->mm) + prev_skip =task_dfl_cgroup (prev)->cpu_skip_mitigation; +if (next->mm) + next_skip =task_dfl_cgroup (next)->cpu_skip_mitigation; + +if (!prev_skip && !next_skip) +return ; +if (prev_skip == next_skip) +return ;
I believe the first (!prev_skip && !next_skip) check is redundant.
Quoted 80 lines
arch/x86/kernel/cpu/bugs.c +7
-0+ +if (next_skip) +wrmsrl (MSR_LSTAR , (unsigned long )entry_SYSCALL_64_unmitigated); +else +wrmsrl (MSR_LSTAR , (unsigned long )entry_SYSCALL_64_mitigated); +} +void __init cpu_select_mitigations (void ) {/* include/linux/cgroup-defs.h +3
-0@@ -546,6 +546,9 @@ struct cgroup {struct bpf_local_storage __rcu *bpf_cgrp_storage;#endif +/* Used to bypass the CPU mitigations for tasks in a cgroup */ +bool cpu_skip_mitigation; +/* All ancestors including self */ struct cgroup *ancestors[]; };kernel/cgroup/cgroup.c +36
-0@@ -2045,6 +2045,7 @@ static void init_cgroup_housekeeping(struct cgroup *cgrp) cgrp->dom_cgrp = cgrp; cgrp->max_descendants =INT_MAX ; cgrp->max_depth =INT_MAX ; + cgrp->cpu_skip_mitigation =0 ;INIT_LIST_HEAD (&cgrp->rstat_css_list);prev_cputime_init (&cgrp->prev_cputime);@@ -3751,6 +3752,41 @@ static int cpu_stat_show(struct seq_file *seq, void *v)return ret; } +static int cpu_skip_mitigation_show (struct seq_file *seq,void *v) +{ +struct cgroup *cgrp =seq_css (seq)->cgroup; +int ret =0 ; + +seq_printf (seq,"%d\n" , cgrp->cpu_skip_mitigation); + +return ret; +} + +static ssize_t cgroup_skip_mitigation_write (struct kernfs_open_file *of, +char *buf,size_t nbytes, +loff_t off) +{ +struct cgroup *cgrp = of->kn->parent->priv; +struct cgroup_file_ctx *ctx = of->priv; +u64 skip_mitigation; +int ret; + +/* Only privileged user in init namespace is allowed to set skip_mitigation */ +if ((ctx->ns != &init_cgroup_ns) || !capable (CAP_SYS_ADMIN )) +return -EPERM ; + + ret =kstrtoull (buf,0 , &skip_mitigation); +if (ret) +return -EINVAL ; + +if (skip_mitigation >1 ) +return -EINVAL ; + + cgrp->cpu_skip_mitigation = skip_mitigation; + +return nbytes; +} +static int cpu_local_stat_show (struct seq_file *seq,void *v) {struct cgroup __maybe_unused *cgrp =seq_css (seq)->cgroup;@@ -5290,6 +5326,12 @@ static struct cftype cgroup_base_files[] = { .name = "cpu.stat.local",
kernel/cgroup/cgroup.c static struct cftype cgroup_base_files[] = { · line 5336
Quoted 10 lines
kernel/cgroup/cgroup.c +6
-0.seq_show = cpu_local_stat_show, }, + { + .name = "cpu.skip_mitigation", + .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = cpu_skip_mitigation_show, + .write = cgroup_skip_mitigation_write, + }, { }/* terminate */ };
Since this control knob is effective only for x86_64, should we enable this only for this architecture? However, cgroup never has architecture specific control knob like that before, it will be the first if we decide to do that.
Quoted 7 lines
kernel/sched/core.c +0
-0
kernel/sched/core.c prepare_task_switch(struct rq *rq, struct task_struct *prev, · line 5000
Quoted 8 lines
kernel/sched/core.c +1
-1@@ -4994,7 +4994,7 @@ prepare_task_switch(struct rq *rq, struct task_struct *prev,fire_sched_out_preempt_notifiers (prev, next);kmap_local_sched_out ();prepare_task (next); -prepare_arch_switch (next); +prepare_arch_switch (prev, next); }/**
The sparc and m68k arches have their own prepare_arch_switch() calls. So you will need to modify those places as well.
Cheers,
Longman
Pawan Gupta
reviewerRe: [PATCH RFC 2/2] cpu/bugs: cgroup: Add a cgroup knob to bypass CPU mitigations
Quoted 54 lines · Waiman Long
Quoted 52 lines · Pawan Gupta
For cases where an admin wanting to bypass CPU mitigations for a specific workload that they trust. Add a cgroup attribute "cpu.skip_mitigation" that can only be set by a privileged user. When set, the CPU mitigations are bypassed for all tasks in that cgroup.
Before setting this knob, the admin should be aware of possible security risks like confused deputy attack on trusted interpreters, JIT engine, etc.
Signed-off-by Pawan Gupta <[email protected]>--- arch/x86/include/asm/switch_to.h | 10 ++++++++++ arch/x86/kernel/cpu/bugs.c | 21 ++++++++++++++++++++ include/linux/cgroup-defs.h | 3 +++ kernel/cgroup/cgroup.c | 42 ++++++++++++++++++++++++++++++++++++++++ kernel/sched/core.c | 2 +- 5 files changed, 77 insertions(+), 1 deletion(-)arch/x86/include/asm/switch_to.h +10
-0@@ -46,6 +46,16 @@ struct fork_frame {struct pt_regs regs; }; +extern inline void cpu_mitigation_skip (struct task_struct *prev,struct task_struct *next); + +#define prepare_arch_switch prepare_arch_switch + +static inline void prepare_arch_switch (struct task_struct *prev, +struct task_struct *next) +{ +cpu_mitigation_skip (prev, next); +} +#define switch_to (prev, next, last) \ do { \ ((last) = __switch_to_asm((prev), (next))); \arch/x86/kernel/cpu/bugs.c +7
-0@@ -128,6 +128,27 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush);DEFINE_STATIC_KEY_FALSE (mmio_stale_data_clear);EXPORT_SYMBOL_GPL (mmio_stale_data_clear); +inline void cpu_mitigation_skip (struct task_struct *prev, +struct task_struct *next) +{ +bool prev_skip = false, next_skip = false; + +if (prev->mm) + prev_skip =task_dfl_cgroup (prev)->cpu_skip_mitigation;
Quoted 8 lines
arch/x86/kernel/cpu/bugs.c DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush); · line 144
Quoted 7 lines
arch/x86/kernel/cpu/bugs.c +7
-0+if (next->mm) + next_skip =task_dfl_cgroup (next)->cpu_skip_mitigation; + +if (!prev_skip && !next_skip) +return ; +if (prev_skip == next_skip) +return ;I believe the first (!prev_skip && !next_skip) check is redundant.
Agh, you are right. I will remove it.
Quoted 80 lines
Quoted 80 lines
arch/x86/kernel/cpu/bugs.c +6
-0+if (next_skip) +wrmsrl (MSR_LSTAR , (unsigned long )entry_SYSCALL_64_unmitigated); +else +wrmsrl (MSR_LSTAR , (unsigned long )entry_SYSCALL_64_mitigated); +} +void __init cpu_select_mitigations (void ) {/* include/linux/cgroup-defs.h +3
-0@@ -546,6 +546,9 @@ struct cgroup {struct bpf_local_storage __rcu *bpf_cgrp_storage;#endif +/* Used to bypass the CPU mitigations for tasks in a cgroup */ +bool cpu_skip_mitigation; +/* All ancestors including self */ struct cgroup *ancestors[]; };kernel/cgroup/cgroup.c +38
-0@@ -2045,6 +2045,7 @@ static void init_cgroup_housekeeping(struct cgroup *cgrp) cgrp->dom_cgrp = cgrp; cgrp->max_descendants =INT_MAX ; cgrp->max_depth =INT_MAX ; + cgrp->cpu_skip_mitigation =0 ;INIT_LIST_HEAD (&cgrp->rstat_css_list);prev_cputime_init (&cgrp->prev_cputime);@@ -3751,6 +3752,41 @@ static int cpu_stat_show(struct seq_file *seq, void *v)return ret; } +static int cpu_skip_mitigation_show (struct seq_file *seq,void *v) +{ +struct cgroup *cgrp =seq_css (seq)->cgroup; +int ret =0 ; + +seq_printf (seq,"%d\n" , cgrp->cpu_skip_mitigation); + +return ret; +} + +static ssize_t cgroup_skip_mitigation_write (struct kernfs_open_file *of, +char *buf,size_t nbytes, +loff_t off) +{ +struct cgroup *cgrp = of->kn->parent->priv; +struct cgroup_file_ctx *ctx = of->priv; +u64 skip_mitigation; +int ret; + +/* Only privileged user in init namespace is allowed to set skip_mitigation */ +if ((ctx->ns != &init_cgroup_ns) || !capable (CAP_SYS_ADMIN )) +return -EPERM ; + + ret =kstrtoull (buf,0 , &skip_mitigation); +if (ret) +return -EINVAL ; + +if (skip_mitigation >1 ) +return -EINVAL ; + + cgrp->cpu_skip_mitigation = skip_mitigation; + +return nbytes; +} +static int cpu_local_stat_show (struct seq_file *seq,void *v) {struct cgroup __maybe_unused *cgrp =seq_css (seq)->cgroup;@@ -5290,6 +5326,12 @@ static struct cftype cgroup_base_files[] = { .name = "cpu.stat.local", .seq_show = cpu_local_stat_show, }, + { + .name = "cpu.skip_mitigation",
Quoted 8 lines
kernel/cgroup/cgroup.c static struct cftype cgroup_base_files[] = { · line 5336
Quoted 6 lines
kernel/cgroup/cgroup.c +4
-0+ .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = cpu_skip_mitigation_show, + .write = cgroup_skip_mitigation_write, + }, { }/* terminate */ };Since this control knob is effective only for
x86_64, should we enable this only for this architecture?
This should be under a CONFIG option that depends on the architecture selected. I don't see a reason why it will not be useful for other archs.
Quoted 2 lines
However, cgroup never has architecture specific control knob like that before, it will be the first if we decide to do that.
Hmm, maybe then it could be universally available, but only effective on archs using it, and NOP on others? Other than performance, this knob should not have any userspace visible side effects.
Quoted 7 lines
Quoted 7 lines
kernel/sched/core.c +0
-0@@ -4994,7 +4994,7 @@ prepare_task_switch(struct rq *rq, struct task_struct *prev,fire_sched_out_preempt_notifiers (prev, next);kmap_local_sched_out ();
Quoted 8 lines
kernel/sched/core.c prepare_task_switch(struct rq *rq, struct task_struct *prev,
Quoted 6 lines
kernel/sched/core.c +1
-1prepare_task (next); -prepare_arch_switch (next); +prepare_arch_switch (prev, next); }/** The sparc and m68k arches have their own
prepare_arch_switch()calls. So you will need to modify those places as well.
Will do. Thanks for the review.
Re: [PATCH RFC 2/2] cpu/bugs: cgroup: Add a cgroup knob to bypass CPU mitigations
Quoted 17 lines · Pawan Gupta
Quoted 9 lines
Quoted 9 lines
static int cpu_local_stat_show(struct seq_file *seq, void *v) { struct cgroup __maybe_unused *cgrp = seq_css(seq)->cgroup;kernel/cgroup/cgroup.c +2
-0@@ -5290,6 +5326,12 @@ static struct cftype cgroup_base_files[] = { .name = "cpu.stat.local", .seq_show = cpu_local_stat_show, }, + { + .name = "cpu.skip_mitigation",Quoted 8 lines
kernel/cgroup/cgroup.c static struct cftype cgroup_base_files[] = { · line 5336
Quoted 6 lines
kernel/cgroup/cgroup.c +4
-0+ .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = cpu_skip_mitigation_show, + .write = cgroup_skip_mitigation_write, + }, { }/* terminate */ };Since this control knob is effective only for
x86_64, should we enable this only for this architecture?
Quoted 2 lines
This should be under a CONFIG option that depends on the architecture selected. I don't see a reason why it will not be useful for other archs.
Using a CONFIG option looks fine to me. I just want to make sure that arches that don't support it won't have a useless control knob show up.
Cheers,
Longman
kernel test robot
build botRe: [PATCH RFC 2/2] cpu/bugs: cgroup: Add a cgroup knob to bypass CPU mitigations
In reply to Pawan Guptakernel test robot noticed the following build errors:
Hi Pawan,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:
[auto build test ERROR on 98f7e32f20d28ec452afb208f9cffc08448a2652]
url: https://github.com/intel-lab-lkp/linux/commits/Pawan-Gupta/x86-entry_64-Add-a-separate-unmitigated-entry-exit-path/20240920-055414 base: 98f7e32f20d28ec452afb208f9cffc08448a2652 patch link: https://lore.kernel.org/r/20240919-selective-mitigation-v1-2-1846cf41895e%40linux.intel.com patch subject: [PATCH RFC 2/2] cpu/bugs: cgroup: Add a cgroup knob to bypass CPU mitigations config: i386-defconfig (https://download.01.org/0day-ci/archive/20240921/[email protected]/config) compiler: clang version 18.1.8 (https://github.com/llvm/llvm-project 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240921/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> arch/x86/kernel/cpu/bugs.c:147:36: error: use of undeclared identifier 'entry_SYSCALL_64_unmitigated'
147 | wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64_unmitigated);
| ^
>> arch/x86/kernel/cpu/bugs.c:149:36: error: use of undeclared identifier 'entry_SYSCALL_64_mitigated'
149 | wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64_mitigated);
| ^2 errors generated.
vim +/entry_SYSCALL_64_unmitigated +147 arch/x86/kernel/cpu/bugs.c
130
131 inline void cpu_mitigation_skip(struct task_struct *prev,
132 struct task_struct *next)
133 {
134 bool prev_skip = false, next_skip = false;
135
136 if (prev->mm)
137 prev_skip = task_dfl_cgroup(prev)->cpu_skip_mitigation;
138 if (next->mm)
139 next_skip = task_dfl_cgroup(next)->cpu_skip_mitigation;
140
141 if (!prev_skip && !next_skip)
142 return;
143 if (prev_skip == next_skip)
144 return;
145
146 if (next_skip)
> 147 wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64_unmitigated); 148 else > 149 wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64_mitigated);
150 }
151
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH RFC 2/2] cpu/bugs: cgroup: Add a cgroup knob to bypass CPU mitigations
In reply to Pawan Guptakernel test robot noticed the following build errors:
Hi Pawan,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:
[auto build test ERROR on 98f7e32f20d28ec452afb208f9cffc08448a2652]
url: https://github.com/intel-lab-lkp/linux/commits/Pawan-Gupta/x86-entry_64-Add-a-separate-unmitigated-entry-exit-path/20240920-055414 base: 98f7e32f20d28ec452afb208f9cffc08448a2652 patch link: https://lore.kernel.org/r/20240919-selective-mitigation-v1-2-1846cf41895e%40linux.intel.com patch subject: [PATCH RFC 2/2] cpu/bugs: cgroup: Add a cgroup knob to bypass CPU mitigations config: i386-allnoconfig (https://download.01.org/0day-ci/archive/20240921/[email protected]/config) compiler: gcc-12 (Debian 12.2.0-14) 12.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240921/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
arch/x86/kernel/cpu/bugs.c: In function 'cpu_mitigation_skip':
>> arch/x86/kernel/cpu/bugs.c:137:29: error: implicit declaration of function 'task_dfl_cgroup'; did you mean 'task_get_cgroup1'? [-Werror=implicit-function-declaration]
137 | prev_skip = task_dfl_cgroup(prev)->cpu_skip_mitigation;
| ^~~~~~~~~~~~~~~
| task_get_cgroup1
>> arch/x86/kernel/cpu/bugs.c:137:50: error: invalid type argument of '->' (have 'int')
137 | prev_skip = task_dfl_cgroup(prev)->cpu_skip_mitigation;
| ^~
arch/x86/kernel/cpu/bugs.c:139:50: error: invalid type argument of '->' (have 'int')
139 | next_skip = task_dfl_cgroup(next)->cpu_skip_mitigation;
| ^~
arch/x86/kernel/cpu/bugs.c:147:50: error: 'entry_SYSCALL_64_unmitigated' undeclared (first use in this function)
147 | wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64_unmitigated);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
arch/x86/kernel/cpu/bugs.c:147:50: note: each undeclared identifier is reported only once for each function it appears in
arch/x86/kernel/cpu/bugs.c:149:50: error: 'entry_SYSCALL_64_mitigated' undeclared (first use in this function); did you mean 'entry_SYSCALL_compat'?
149 | wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64_mitigated);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
| entry_SYSCALL_compatcc1: some warnings being treated as errors
vim +137 arch/x86/kernel/cpu/bugs.c
130
131 inline void cpu_mitigation_skip(struct task_struct *prev,
132 struct task_struct *next)
133 {
134 bool prev_skip = false, next_skip = false;
135
136 if (prev->mm)
> 137 prev_skip = task_dfl_cgroup(prev)->cpu_skip_mitigation;
138 if (next->mm)
139 next_skip = task_dfl_cgroup(next)->cpu_skip_mitigation;
140
141 if (!prev_skip && !next_skip)
142 return;
143 if (prev_skip == next_skip)
144 return;
145
146 if (next_skip)
147 wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64_unmitigated);
148 else
149 wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64_mitigated);
150 }
151
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Michal Koutný
maintainerHello.
Quoted 5 lines · Pawan Gupta
This is an experimental series exploring the feasibility of selectively applying CPU vulnerability mitigations on a per-process basis. The motivation behind this work is to address the performance degradation experienced by trusted user-space applications due to system-wide CPU mitigations.
This is an interesting idea (like an extension of core scheduling).
Quoted 2 lines
The rationale for choosing the cgroup interface over other potential interfaces, such as LSMs, is cgroup's inherent support for core scheduling.
You don't list prctl (and process inheritance) interface here.
Quoted 2 lines
Core scheduling allows the grouping of tasks such that they are scheduled to run on the same cores.
And that is actually the way how core scheduling is implemented AFAICS -- cookie creation and passing via prctls.
Thus I don't find the implementation via a cgroup attribute ideal.
(I'd also say that cgroups are more organization/resource domains but not so much security domains.)
Quoted 1 line
- Should child processes inherit the parent's unmitigated status?
Assuming turning off mitigations is a a privileged operation, the fork could preserve it. It would be upon parent to clear it up properly before handing over execution to a child (cf e.g. dropping uid=0).
HTH,
Michal
Pawan Gupta
reviewerQuoted 9 lines · Michal Koutný
Hello.
Quoted 5 lines · Pawan Gupta
This is an experimental series exploring the feasibility of selectively applying CPU vulnerability mitigations on a per-process basis. The motivation behind this work is to address the performance degradation experienced by trusted user-space applications due to system-wide CPU mitigations.
Quoted 6 lines
This is an interesting idea (like an extension of core scheduling).
Quoted 2 lines
The rationale for choosing the cgroup interface over other potential interfaces, such as LSMs, is cgroup's inherent support for core scheduling.
You don't list prctl (and process inheritance) interface here.
Apologies for the oversight. prctl is indeed the interface that Core scheduling uses to group tasks. Cgroup was the initially proposed interface but that later got changed to prctl.
Quoted 6 lines
Quoted 2 lines
Core scheduling allows the grouping of tasks such that they are scheduled to run on the same cores.
And that is actually the way how core scheduling is implemented AFAICS -- cookie creation and passing via prctls.
Thus I don't find the implementation via a cgroup attribute ideal.
Problem with prctl is that a process can't set its own skip-mitigation status, unless it is privileged. So a privileged process has to spawn the workload that needs to be unmitigated. This is not ideal for an application that wants to selectively apply mitigations.
Quoted 2 lines
(I'd also say that cgroups are more organization/resource domains but not so much security domains.)
Agree.
Quoted 5 lines
Quoted 1 line
- Should child processes inherit the parent's unmitigated status?
Assuming turning off mitigations is a a privileged operation, the fork could preserve it. It would be upon parent to clear it up properly before handing over execution to a child (cf e.g. dropping uid=0).
IIUC, skip-mitigation should be a capability that can be inherited by child processes, or can be set for an executable file. If we want to leverage Core scheduling, it will be callenging to have all trusted processes in the system have the same cookie. But, all child processes of a trusted process would inherit the skip-mitigation status and same cookie, so they can be co-sccheduled.