powerpc/tm: Fix FP/VMX unavailable exceptions inside a transaction
PATCH 1/3 powerpc/tm: Fix FP/VMX unavailable exceptions inside a transaction
From: Gustavo Romero <[email protected]>
When we take an FP unavailable exception in a transaction we have to account for the hardware FP TM checkpointed registers being incorrect. In this case for this process we know the current and checkpointed FP registers must be the same (since FP wasn't used inside the transaction) hence in the thread_struct we copy the current FP registers to the checkpointed ones.
This copy is done in tm_reclaim_thread(). We use thread->ckpt_regs.msr to determine if FP was on when in userspace. thread->ckpt_regs.msr represents the state of the MSR when exiting userspace. This is setup by check_if_tm_restore_required().
Unfortunatley there is an optimisation in giveup_all() which returns early if tsk->thread.regs->msr (via local variable usermsr) has FP=VEC=VSX=SPE=0. This optimisation means that check_if_tm_restore_required() is not called and hence thread->ckpt_regs.msr is not updated and will contain an old value.
This can happen if due to load_fp=255 we start a userspace process with MSR FP=1 and then we are context switched out. In this case thread->ckpt_regs.msr will contain FP=1. If that same process is then context switched in and load_fp overflows, MSR will have FP=0. If that process now enters a transaction and does an FP instruction, the FP unavailable will not update thread->ckpt_regs.msr (the bug) and MSR FP=1 will be retained in thread->ckpt_regs.msr. tm_reclaim_thread() will then not perform the required memcpy and the checkpointed FP regs in the thread struct will contain the wrong values.
The code path for this happening is:
Userspace: Kernel
Start userspace
with MSR FP/VEC/VSX/SPE=0 TM=1
< -----
...
tbegin
bne
fp instruction
FP unavailable
---- >
fp_unavailable_tm()
tm_reclaim_current()
tm_reclaim_thread()
giveup_all()
return early since FP/VMX/VSX=0
/* ckpt MSR not updated (Incorrect) */
tm_reclaim()
/* thread_struct ckpt FP regs contain junk (OK) */
/* Sees ckpt MSR FP=1 (Incorrect) */
no memcpy() performed
/* thread_struct ckpt FP regs not fixed (Incorrect) */
tm_recheckpoint()
/* Put junk in hardware checkpoint FP regs */
....
< -----
Return to userspace
with MSR TM=1 FP=1
with junk in the FP TM checkpoint
TM rollback
reads FP junkThis is a data integrity problem for the current process as the FP registers are corrupted. It's also a security problem as the FP registers from one process may be leaked to another.
This patch moves up check_if_tm_restore_required() in giveup_all() to ensure thread->ckpt_regs.msr is updated correctly.
A simple testcase to replicate this will be posted to tools/testing/selftests/powerpc/tm/tm-poison.c
Similarly for VMX.
This fixes CVE-2019-15030.
--- arch/powerpc/kernel/process.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/powerpc/kernel/process.c +2 -1
@@ -497,13 +497,14 @@ void giveup_all(struct task_struct *tsk)if (!tsk->thread.regs)return ; +check_if_tm_restore_required (tsk); + usermsr = tsk->thread.regs->msr;if ((usermsr & msr_all_available) ==0 )return ;msr_check_and_set (msr_all_available); -check_if_tm_restore_required (tsk);WARN_ON ((usermsr &MSR_VSX ) && !((usermsr &MSR_FP ) && (usermsr &MSR_VEC )));
Signature · 1 line
2.21.0
PATCH 2/3 powerpc/tm: Fix restoring FP/VMX facility incorrectly on interrupts
From: Gustavo Romero <[email protected]>
When in userspace and MSR FP=0 the hardware FP state is unrelated to the current process. This is extended for transactions where if tbegin is run with FP=0, the hardware checkpoint FP state will also be unrelated to the current process. Due to this, we need to ensure this hardware checkpoint is updated with the correct state before we enable FP for this process.
Unfortunately we get this wrong when returning to a process from a hardware interrupt. A process that starts a transaction with FP=0 can take an interrupt. When the kernel returns back to that process, we change to FP=1 but with hardware checkpoint FP state not updated. If this transaction is then rolled back, the FP registers now contain the wrong state.
The process looks like this:
Userspace: Kernel
Start userspace
with MSR FP=0 TM=1
< -----
...
tbegin
bne
Hardware interrupt
---- >
<do_IRQ...>
....
ret_from_except
restore_math()
/* sees FP=0 */
restore_fp()
tm_active_with_fp()
/* sees FP=1 (Incorrect) */
load_fp_state()
FP = 0 -> 1
< -----
Return to userspace
with MSR TM=1 FP=1
with junk in the FP TM checkpoint TM rollback
reads FP junk
When returning from the hardware exception, tm_active_with_fp() is incorrectly making restore_fp() call load_fp_state() which is setting FP=1.
The fix is to remove tm_active_with_fp().
tm_active_with_fp() is attempting to handle the case where FP state has been changed inside a transaction. In this case the checkpointed and transactional FP state is different and hence we must restore the FP state (ie. we can't do lazy FP restore inside a transaction that's used FP). It's safe to remove tm_active_with_fp() as this case is handled by restore_tm_state(). restore_tm_state() detects if FP has been using inside a transaction and will set load_fp and call restore_math() to ensure the FP state (checkpoint and transaction) is restored.
This is a data integrity problem for the current process as the FP registers are corrupted. It's also a security problem as the FP registers from one process may be leaked to another.
Similarly for VMX.
A simple testcase to replicate this will be posted to tools/testing/selftests/powerpc/tm/tm-poison.c
This fixes CVE-2019-15031.
--- arch/powerpc/kernel/process.c | 18 ++---------------- 1 file changed, 2 insertions(+), 16 deletions(-)
arch/powerpc/kernel/process.c +2 -16
@@ -101,21 +101,8 @@ static void check_if_tm_restore_required(struct task_struct *tsk) } } -static bool tm_active_with_fp (struct task_struct *tsk) -{ -return MSR_TM_ACTIVE (tsk->thread.regs->msr) && - (tsk->thread.ckpt_regs.msr &MSR_FP ); -} - -static bool tm_active_with_altivec (struct task_struct *tsk) -{ -return MSR_TM_ACTIVE (tsk->thread.regs->msr) && - (tsk->thread.ckpt_regs.msr &MSR_VEC ); -}#else static inline void check_if_tm_restore_required (struct task_struct *tsk) { } -static inline bool tm_active_with_fp (struct task_struct *tsk) {return false; } -static inline bool tm_active_with_altivec (struct task_struct *tsk) {return false; }#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */ bool strict_msr_control;
@@ -252,7 +239,7 @@ EXPORT_SYMBOL(enable_kernel_fp);static int restore_fp (struct task_struct *tsk) { -if (tsk->thread.load_fp ||tm_active_with_fp (tsk)) { +if (tsk->thread.load_fp) {load_fp_state (¤t->thread.fp_state); current->thread.load_fp++;return 1 ;
@@ -334,8 +321,7 @@ EXPORT_SYMBOL_GPL(flush_altivec_to_thread);static int restore_altivec (struct task_struct *tsk) { -if (cpu_has_feature (CPU_FTR_ALTIVEC ) && - (tsk->thread.load_vec ||tm_active_with_altivec (tsk))) { +if (cpu_has_feature (CPU_FTR_ALTIVEC ) && (tsk->thread.load_vec)) {load_vr_state (&tsk->thread.vr_state); tsk->thread.used_vr =1 ; tsk->thread.load_vec++;
Signature · 1 line
2.21.0
PATCH 3/3 powerpc/tm: Add tm-poison test
In reply to Michael NeulingFrom: Gustavo Romero <[email protected]>
Add TM selftest to check if FP or VEC register values from one process can leak into another process when both run on the same CPU.
This tests for CVE-2019-15030 and CVE-2019-15031.
--- tools/testing/selftests/powerpc/tm/.gitignore | 1 + tools/testing/selftests/powerpc/tm/Makefile | 2 +- .../testing/selftests/powerpc/tm/tm-poison.c | 180 ++++++++++++++++++ 3 files changed, 182 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/powerpc/tm/tm-poison.c
tools/testing/selftests/powerpc/tm/.gitignore +1 -0
@@ -17,3 +17,4 @@ tm-vmx-unavail tm-unavailable tm-trap tm-sigreturn +tm-poison
tools/testing/selftests/powerpc/tm/Makefile +1 -1
@@ -5,7 +5,7 @@ SIGNAL_CONTEXT_CHK_TESTS := tm-signal-context-chk-gpr tm-signal-context-chk-fpu TEST_GEN_PROGS := tm-resched-dscr tm-syscall tm-signal-msr-resv tm-signal-stack \ tm-vmxcopy tm-fork tm-tar tm-tmspr tm-vmx-unavail tm-unavailable tm-trap \ $(SIGNAL_CONTEXT_CHK_TESTS) tm-sigreturn tm-signal-sigreturn-nt \ - tm-signal-context-force-tm + tm-signal-context-force-tm tm-poison top_srcdir = ../../../../.. include ../../lib.mk
tools/testing/selftests/powerpc/tm/tm-poison.c new file +180 -0
180 lines not on this page · raw message
Signature · 1 line
2.21.0
Michael Ellerman
reviewerRe: [PATCH 3/3] powerpc/tm: Add tm-poison test
Quoted 7 lines · Michael Neuling
From: Gustavo Romero <[email protected]>
Add TM selftest to check if FP or VEC register values from one process can leak into another process when both run on the same CPU.
This tests for CVE-2019-15030 and CVE-2019-15031.
Quoted 8 lines
Signed-off-by Gustavo Romero <[email protected]>Signed-off-by Michael Neuling <[email protected]>--- tools/testing/selftests/powerpc/tm/.gitignore | 1 + tools/testing/selftests/powerpc/tm/Makefile | 2 +- .../testing/selftests/powerpc/tm/tm-poison.c | 180 ++++++++++++++++++ 3 files changed, 182 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/powerpc/tm/tm-poison.c
This doesn't build on 64-bit big endian:
tm-poison.c: In function 'tm_poison_test':
tm-poison.c:118:10: error: format '%lx' expects argument of type 'long unsigned int', but argument 2 has type 'uint64_t {aka long long unsigned int}' [-Werror=format=]
printf("Unknown value %#lx leaked into f31!\n", unknown);
^
tm-poison.c:166:10: error: format '%lx' expects argument of type 'long unsigned int', but argument 2 has type 'uint64_t {aka long long unsigned int}' [-Werror=format=]
printf("Unknown value %#lx leaked into vr31!\n", unknown);
^cheers
Re: [PATCH 3/3] powerpc/tm: Add tm-poison test
Hi Michael,
Quoted 19 lines · Michael Ellerman
Quoted 7 lines · Michael Neuling
From: Gustavo Romero <[email protected]>
Add TM selftest to check if FP or VEC register values from one process can leak into another process when both run on the same CPU.
This tests for CVE-2019-15030 and CVE-2019-15031.
Quoted 8 lines
Signed-off-by Gustavo Romero <[email protected]>Signed-off-by Michael Neuling <[email protected]>--- tools/testing/selftests/powerpc/tm/.gitignore | 1 + tools/testing/selftests/powerpc/tm/Makefile | 2 +- .../testing/selftests/powerpc/tm/tm-poison.c | 180 ++++++++++++++++++ 3 files changed, 182 insertions(+), 1 deletion(-)create mode 100644
tools/testing/selftests/powerpc/tm/tm-poison.cThis doesn't build on 64-bit big endian:
Quoted 7 lines
tm-poison.c: In function 'tm_poison_test':
tm-poison.c:118:10: error: format '%lx' expects argument of type 'long unsigned int', but argument 2 has type 'uint64_t{aka long long unsigned int}' [-Werror=format=]printf("Unknown value %#lx leaked into f31!\n", unknown); ^ tm-poison.c:166:10: error: format '%lx' expects argument of type 'long unsigned int', but argument 2 has type 'uint64_t {aka long long unsigned int}' [-Werror=format=] printf("Unknown value %#lx leaked into vr31!\n", unknown); ^
Sorry. I sent a v2 addressing it. Now I tested the fix against Travis CI.
Thank you.
Cheers,
Gustavo
PATCH v2 1/3 powerpc/tm: Fix FP/VMX unavailable exceptions inside a transaction
In reply to Michael NeulingFrom: Gustavo Romero <[email protected]>
When we take an FP unavailable exception in a transaction we have to account for the hardware FP TM checkpointed registers being incorrect. In this case for this process we know the current and checkpointed FP registers must be the same (since FP wasn't used inside the transaction) hence in the thread_struct we copy the current FP registers to the checkpointed ones.
This copy is done in tm_reclaim_thread(). We use thread->ckpt_regs.msr to determine if FP was on when in userspace. thread->ckpt_regs.msr represents the state of the MSR when exiting userspace. This is setup by check_if_tm_restore_required().
Unfortunatley there is an optimisation in giveup_all() which returns early if tsk->thread.regs->msr (via local variable usermsr) has FP=VEC=VSX=SPE=0. This optimisation means that check_if_tm_restore_required() is not called and hence thread->ckpt_regs.msr is not updated and will contain an old value.
This can happen if due to load_fp=255 we start a userspace process with MSR FP=1 and then we are context switched out. In this case thread->ckpt_regs.msr will contain FP=1. If that same process is then context switched in and load_fp overflows, MSR will have FP=0. If that process now enters a transaction and does an FP instruction, the FP unavailable will not update thread->ckpt_regs.msr (the bug) and MSR FP=1 will be retained in thread->ckpt_regs.msr. tm_reclaim_thread() will then not perform the required memcpy and the checkpointed FP regs in the thread struct will contain the wrong values.
The code path for this happening is:
Userspace: Kernel
Start userspace
with MSR FP/VEC/VSX/SPE=0 TM=1
< -----
...
tbegin
bne
fp instruction
FP unavailable
---- >
fp_unavailable_tm()
tm_reclaim_current()
tm_reclaim_thread()
giveup_all()
return early since FP/VMX/VSX=0
/* ckpt MSR not updated (Incorrect) */
tm_reclaim()
/* thread_struct ckpt FP regs contain junk (OK) */
/* Sees ckpt MSR FP=1 (Incorrect) */
no memcpy() performed
/* thread_struct ckpt FP regs not fixed (Incorrect) */
tm_recheckpoint()
/* Put junk in hardware checkpoint FP regs */
....
< -----
Return to userspace
with MSR TM=1 FP=1
with junk in the FP TM checkpoint
TM rollback
reads FP junkThis is a data integrity problem for the current process as the FP registers are corrupted. It's also a security problem as the FP registers from one process may be leaked to another.
This patch moves up check_if_tm_restore_required() in giveup_all() to ensure thread->ckpt_regs.msr is updated correctly.
A simple testcase to replicate this will be posted to tools/testing/selftests/powerpc/tm/tm-poison.c
Similarly for VMX.
This fixes CVE-2019-15030.
--- arch/powerpc/kernel/process.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/powerpc/kernel/process.c +2 -1
@@ -497,13 +497,14 @@ void giveup_all(struct task_struct *tsk)if (!tsk->thread.regs)return ; +check_if_tm_restore_required (tsk); + usermsr = tsk->thread.regs->msr;if ((usermsr & msr_all_available) ==0 )return ;msr_check_and_set (msr_all_available); -check_if_tm_restore_required (tsk);WARN_ON ((usermsr &MSR_VSX ) && !((usermsr &MSR_FP ) && (usermsr &MSR_VEC )));
Signature · 1 line
2.17.2
PATCH v2 2/3 powerpc/tm: Fix restoring FP/VMX facility incorrectly on interrupts
From: Gustavo Romero <[email protected]>
When in userspace and MSR FP=0 the hardware FP state is unrelated to the current process. This is extended for transactions where if tbegin is run with FP=0, the hardware checkpoint FP state will also be unrelated to the current process. Due to this, we need to ensure this hardware checkpoint is updated with the correct state before we enable FP for this process.
Unfortunately we get this wrong when returning to a process from a hardware interrupt. A process that starts a transaction with FP=0 can take an interrupt. When the kernel returns back to that process, we change to FP=1 but with hardware checkpoint FP state not updated. If this transaction is then rolled back, the FP registers now contain the wrong state.
The process looks like this:
Userspace: Kernel
Start userspace
with MSR FP=0 TM=1
< -----
...
tbegin
bne
Hardware interrupt
---- >
<do_IRQ...>
....
ret_from_except
restore_math()
/* sees FP=0 */
restore_fp()
tm_active_with_fp()
/* sees FP=1 (Incorrect) */
load_fp_state()
FP = 0 -> 1
< -----
Return to userspace
with MSR TM=1 FP=1
with junk in the FP TM checkpoint TM rollback
reads FP junk
When returning from the hardware exception, tm_active_with_fp() is incorrectly making restore_fp() call load_fp_state() which is setting FP=1.
The fix is to remove tm_active_with_fp().
tm_active_with_fp() is attempting to handle the case where FP state has been changed inside a transaction. In this case the checkpointed and transactional FP state is different and hence we must restore the FP state (ie. we can't do lazy FP restore inside a transaction that's used FP). It's safe to remove tm_active_with_fp() as this case is handled by restore_tm_state(). restore_tm_state() detects if FP has been using inside a transaction and will set load_fp and call restore_math() to ensure the FP state (checkpoint and transaction) is restored.
This is a data integrity problem for the current process as the FP registers are corrupted. It's also a security problem as the FP registers from one process may be leaked to another.
Similarly for VMX.
A simple testcase to replicate this will be posted to tools/testing/selftests/powerpc/tm/tm-poison.c
This fixes CVE-2019-15031.
--- arch/powerpc/kernel/process.c | 18 ++---------------- 1 file changed, 2 insertions(+), 16 deletions(-)
arch/powerpc/kernel/process.c +2 -16
@@ -101,21 +101,8 @@ static void check_if_tm_restore_required(struct task_struct *tsk) } } -static bool tm_active_with_fp (struct task_struct *tsk) -{ -return MSR_TM_ACTIVE (tsk->thread.regs->msr) && - (tsk->thread.ckpt_regs.msr &MSR_FP ); -} - -static bool tm_active_with_altivec (struct task_struct *tsk) -{ -return MSR_TM_ACTIVE (tsk->thread.regs->msr) && - (tsk->thread.ckpt_regs.msr &MSR_VEC ); -}#else static inline void check_if_tm_restore_required (struct task_struct *tsk) { } -static inline bool tm_active_with_fp (struct task_struct *tsk) {return false; } -static inline bool tm_active_with_altivec (struct task_struct *tsk) {return false; }#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */ bool strict_msr_control;
@@ -252,7 +239,7 @@ EXPORT_SYMBOL(enable_kernel_fp);static int restore_fp (struct task_struct *tsk) { -if (tsk->thread.load_fp ||tm_active_with_fp (tsk)) { +if (tsk->thread.load_fp) {load_fp_state (¤t->thread.fp_state); current->thread.load_fp++;return 1 ;
@@ -334,8 +321,7 @@ EXPORT_SYMBOL_GPL(flush_altivec_to_thread);static int restore_altivec (struct task_struct *tsk) { -if (cpu_has_feature (CPU_FTR_ALTIVEC ) && - (tsk->thread.load_vec ||tm_active_with_altivec (tsk))) { +if (cpu_has_feature (CPU_FTR_ALTIVEC ) && (tsk->thread.load_vec)) {load_vr_state (&tsk->thread.vr_state); tsk->thread.used_vr =1 ; tsk->thread.load_vec++;
Signature · 1 line
2.17.2
Michael Ellerman
AppliedRe: [PATCH v2 2/3] powerpc/tm: Fix restoring FP/VMX facility incorrectly on interrupts
Quoted 8 lines · gromero
From: Gustavo Romero <[email protected]>
When in userspace and MSR FP=0 the hardware FP state is unrelated to the current process. This is extended for transactions where if tbegin is run with FP=0, the hardware checkpoint FP state will also be unrelated to the current process. Due to this, we need to ensure this hardware checkpoint is updated with the correct state before we enable FP for this process.
...
Quoted 7 lines
This fixes CVE-2019-15031.
Fixes a7771176b439 ("powerpc: Don't enable FP/Altivec if not checkpointed")Cc [email protected] # 4.15+Signed-off-by Gustavo Romero <[email protected]>Signed-off-by Michael Neuling <[email protected]>
Applied to powerpc fixes, thanks.
https://git.kernel.org/powerpc/c/a8318c13e79badb92bc6640704a64cc022a6eb97
cheers
PATCH v2 3/3 powerpc/tm: Add tm-poison test
In reply to gromeroFrom: Gustavo Romero <[email protected]>
Add TM selftest to check if FP or VEC register values from one process can leak into another process when both run on the same CPU.
--- tools/testing/selftests/powerpc/tm/.gitignore | 1 + tools/testing/selftests/powerpc/tm/Makefile | 2 +- .../testing/selftests/powerpc/tm/tm-poison.c | 179 ++++++++++++++++++ 3 files changed, 181 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/powerpc/tm/tm-poison.c
tools/testing/selftests/powerpc/tm/.gitignore +1 -0
@@ -17,3 +17,4 @@ tm-vmx-unavail tm-unavailable tm-trap tm-sigreturn +tm-poison
tools/testing/selftests/powerpc/tm/Makefile +1 -1
@@ -5,7 +5,7 @@ SIGNAL_CONTEXT_CHK_TESTS := tm-signal-context-chk-gpr tm-signal-context-chk-fpu TEST_GEN_PROGS := tm-resched-dscr tm-syscall tm-signal-msr-resv tm-signal-stack \ tm-vmxcopy tm-fork tm-tar tm-tmspr tm-vmx-unavail tm-unavailable tm-trap \ $(SIGNAL_CONTEXT_CHK_TESTS) tm-sigreturn tm-signal-sigreturn-nt \ - tm-signal-context-force-tm + tm-signal-context-force-tm tm-poison top_srcdir = ../../../../.. include ../../lib.mk
tools/testing/selftests/powerpc/tm/tm-poison.c new file +179 -0
179 lines not on this page · raw message
Signature · 1 line
2.17.2
Michael Ellerman
AppliedRe: [PATCH v2 3/3] powerpc/tm: Add tm-poison test
Quoted 7 lines · gromero
From: Gustavo Romero <[email protected]>
Add TM selftest to check if FP or VEC register values from one process can leak into another process when both run on the same CPU.
Signed-off-by Gustavo Romero <[email protected]>Signed-off-by Michael Neuling <[email protected]>
Applied to powerpc fixes, thanks.
https://git.kernel.org/powerpc/c/a003365cab64b0f7988ac3ccb1da895ce0bece5e
cheers
Michael Ellerman
AppliedQuoted 8 lines · gromero
From: Gustavo Romero <[email protected]>
When we take an FP unavailable exception in a transaction we have to account for the hardware FP TM checkpointed registers being incorrect. In this case for this process we know the current and checkpointed FP registers must be the same (since FP wasn't used inside the transaction) hence in the
thread_structwe copy the current FP registers to the checkpointed ones.
...
Quoted 7 lines
This fixes CVE-2019-15030.
Fixes f48e91e87e67 ("powerpc/tm: Fix FP and VMX register corruption")Cc [email protected] # 4.12+Signed-off-by Gustavo Romero <[email protected]>Signed-off-by Michael Neuling <[email protected]>
Applied to powerpc fixes, thanks.
https://git.kernel.org/powerpc/c/8205d5d98ef7f155de211f5e2eb6ca03d95a5a60
cheers