Fixes to L1D flushing (on top of linux-next and
Subsystems (5)
cover Fixes to L1D flushing (on top of linux-next and
There were concerns with the current interface (checking for the state of SMT of the core at opt-in time [1]). These patches change the interface to send a SIGBUS to the task if
a. It opts into L1D flushing
b. Runs on a core with SMT enabled
There are also documentation fixes. I've tested the patches at my end, but would appreciate a broad review of the change in functionality and/or any recommended doc updates.
[1] http://lkml.iu.edu/hypermail/linux/kernel/2009.3/09310.html
[2] http://lkml.iu.edu/hypermail/linux/kernel/2009.3/09613.html
NOTE: For testing these patches, specifically the SIGBUS behaviour, there needs to be contention on the CPU where the task that opts into L1D flushing is running to see the SIGBUS being sent to it (the deterministic bit is that if there is scope of data leak the task will get killed)
Balbir Singh (3):
x86/mm: change l1d flush runtime prctl behaviour
Documentation: Update the new SIGBUS behaviour for tasks Documentation/l1d_flush: Fix up warning with labels
.../admin-guide/hw-vuln/l1d_flush.rst | 12 +++--- arch/Kconfig | 4 ++ arch/x86/Kconfig | 1 + arch/x86/mm/tlb.c | 37 +++++++++++-------- include/linux/sched.h | 10 +++++ 5 files changed, 42 insertions(+), 22 deletions(-)
Signature · 1 line
2.17.1
PATCH 1/3 x86/mm: change l1d flush runtime prctl behaviour
Detection of task affinities at API opt-in time is not the best approach, the approach is to kill the task if it runs on a SMT enable core. This is better than not flushing the L1D cache when the task switches from a non-SMT core to an SMT enabled core.
---
To be applied on top of tip commit idb6724f118d44606fddde391ba7527526b3cad211
arch/Kconfig | 4 ++++ arch/x86/Kconfig | 1 + arch/x86/mm/tlb.c | 37 +++++++++++++++++++++---------------- include/linux/sched.h | 10 ++++++++++ 4 files changed, 36 insertions(+), 16 deletions(-)
arch/Kconfig +4 -0
@@ -311,6 +311,10 @@ config ARCH_32BIT_OFF_T still support 32-bit off_t. This option is enabled for all such architectures explicitly. +config ARCH_HAS_PARANOID_L1D_FLUSH + bool + default n + config HAVE_ASM_MODVERSIONS bool help
arch/x86/Kconfig +1 -0
@@ -103,6 +103,7 @@ config X86 select ARCH_WANTS_DYNAMIC_TASK_STRUCT select ARCH_WANT_HUGE_PMD_SHARE select ARCH_WANTS_THP_SWAP if X86_64 + select ARCH_HAS_PARANOID_L1D_FLUSH select BUILDTIME_TABLE_SORT select CLKEVT_I8253 select CLOCKSOURCE_VALIDATE_LAST_CYCLE
arch/x86/mm/tlb.c +21 -16
@@ -315,31 +315,18 @@ EXPORT_SYMBOL_GPL(leave_mm);int enable_l1d_flush_for_task (struct task_struct *tsk) { -int cpu, ret =0 , i; -/* * Do not enable L1D_FLUSH_OUT if * b. The CPU is not affected by the L1TF bug * c. The CPU does not have L1D FLUSH feature support -* c. The task's affinity is on cores with SMT on. */ if (!boot_cpu_has_bug (X86_BUG_L1TF ) || - !static_cpu_has (X86_FEATURE_FLUSH_L1D )) + !boot_cpu_has (X86_FEATURE_FLUSH_L1D ))return -EINVAL ; - cpu =get_cpu (); - -for_each_cpu (i, &tsk->cpus_mask) { -if (cpu_data (i).smt_active == true) { -put_cpu (); -return -EINVAL ; - } - } -set_ti_thread_flag (&tsk->thread_info,TIF_SPEC_L1D_FLUSH ); -put_cpu (); -return ret; +return 0 ; }int disable_l1d_flush_for_task (struct task_struct *tsk)
@@ -358,13 +345,31 @@ void switch_mm(struct mm_struct *prev, struct mm_struct *next,local_irq_restore (flags); } +/* +* Sent to a task that opts into L1D flushing via the prctl interface +* but ends up running on an SMT enabled core. +*/ +static void l1d_flush_kill (struct callback_head *ch) +{ +force_sig (SIGBUS ); +} +static inline unsigned long mm_mangle_tif_spec_bits (struct task_struct *next) {unsigned long next_tif =task_thread_info (next)->flags;unsigned long spec_bits = (next_tif >>TIF_SPEC_IB ) &LAST_USER_MM_SPEC_MASK ; +unsigned long next_mm;BUILD_BUG_ON (TIF_SPEC_L1D_FLUSH !=TIF_SPEC_IB +1 ); -return (unsigned long )next->mm | spec_bits; + next_mm = (unsigned long )next->mm | spec_bits; + +if ((next_mm &LAST_USER_MM_L1D_FLUSH ) &&this_cpu_read (cpu_info.smt_active)) { +clear_ti_thread_flag (&next->thread_info,TIF_SPEC_L1D_FLUSH ); + next->l1d_flush_kill.func = l1d_flush_kill; +task_work_add (next, &next->l1d_flush_kill, true); + } + +return next_mm; }static void cond_mitigation (struct task_struct *next)
include/linux/sched.h +10 -0
@@ -1338,6 +1338,16 @@ struct task_struct {unsigned long getblk_bh_state;#endif +#ifdef CONFIG_ARCH_HAS_PARANOID_L1D_FLUSH +/* +* If L1D flush is supported on mm context switch +* then we use this callback head to queue kill work +* to kill tasks that are not running on SMT disabled +* cores +*/ +struct callback_head l1d_flush_kill; +#endif +/* * New fields for task_struct should be added above here, so that * they are included in the randomized portion of task_struct.
Signature · 1 line
2.17.1
PATCH 2/3 Documentation: Update the new SIGBUS behaviour for tasks
In reply to Balbir SinghUpdate the documentation to mention that a SIGBUS will be sent to tasks that opt-into L1D flushing and execute on non-SMT cores.
---
To be applied on top of tip commit id767d46ab566dd489733666efe48732d523c8c332
Documentation/admin-guide/hw-vuln/l1d_flush.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
Documentation/admin-guide/hw-vuln/l1d_flush.rst +4 -4
@@ -63,8 +63,8 @@ This can be addressed by controlled placement of processes on physical CPU cores or by disabling SMT. See the relevant chapter in the L1TF mitigation document: :ref:`Documentation/admin-guide/hw-vuln/l1tf.rst <smt_control>`. -**NOTE** : Checks have been added to ensure that the prctl API associated +**NOTE** : Checks have been added to ensure that the L1D flush associated with the opt-in will work only when the task affinity of the task opting -in, is limited to cores running in non-SMT mode. The same checks are made -when L1D is flushed. Changing the affinity after opting in, would result -in flushes not working on cores that are in non-SMT mode. +in, is limited to cores running in non-SMT mode. Changing the affinity after +opting in, would result in the task getting a SIGBUS when it executes on +the flush is needed and the task is executing on the non-SMT core.
Signature · 1 line
2.17.1
PATCH 3/3 Documentation/l1d_flush: Fix up warning with labels
In reply to Balbir SinghAdd a label to spec_set_ctrl to remove the build warning.
---
To be applied on top of tip commit id767d46ab566dd489733666efe48732d523c8c332
Documentation/admin-guide/hw-vuln/l1d_flush.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/admin-guide/hw-vuln/l1d_flush.rst +2 -2
@@ -22,8 +22,8 @@ mechanism Usage Guidelines ---------------- -Please see document: :ref:`Documentation/userspace-api/spec_ctrl.rst` for -details. +Please see document: :ref:`Documentation/userspace-api/spec_ctrl.rst +<set_spec_ctrl>` for details. **NOTE**: The feature is disabled by default, applications need to specifically opt into the feature to enable it.
Signature · 1 line
2.17.1