powerpc: Avoid code patching freed init sections
PATCH v2 1/1 powerpc: Avoid code patching freed init sections
This stops us from doing code patching in init sections after they've been freed.
In this chain:
kvm_guest_init() ->
kvm_use_magic_page() ->
fault_in_pages_readable() ->
__get_user() ->
__get_user_nocheck() ->
barrier_nospec();We have a code patching location at barrier_nospec() and kvm_guest_init() is an init function. This whole chain gets inlined, so when we free the init section (hence kvm_guest_init()), this code goes away and hence should no longer be patched.
We seen this as userspace memory corruption when using a memory checker while doing partition migration testing on powervm (this starts the code patching post migration via /sys/kernel/mobility/migration). In theory, it could also happen when using /sys/kernel/debug/powerpc/barrier_nospec.
With this patch there is a small change of a race if we code patch between the init section being freed and setting SYSTEM_RUNNING (in kernel_init()) but that seems like an impractical time and small window for any code patching to occur.
---
For stable I've marked this as v4.13+ since that's when we refactored code-patching.c but it could go back even further than that. In reality though, I think we can only hit this since the first spectre/meltdown changes.
v2:
Print when we skip an address
--- arch/powerpc/lib/code-patching.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
arch/powerpc/lib/code-patching.c +22 -0
@@ -23,11 +23,33 @@#include <asm/code-patching.h> #include <asm/setup.h> + +static inline bool in_init_section (unsigned int *patch_addr) +{ +if (patch_addr < (unsigned int *)__init_begin) +return false; +if (patch_addr >= (unsigned int *)__init_end) +return false; +return true; +} + +static inline bool init_freed (void ) +{ +return (system_state >=SYSTEM_RUNNING ); +} +static int __patch_instruction (unsigned int *exec_addr,unsigned int instr,unsigned int *patch_addr) {int err; +/* Make sure we aren't patching a freed init section */ +if (in_init_section (patch_addr) &&init_freed ()) { +printk (KERN_DEBUG "Skipping init section patching addr: 0x%lx\n" , + (unsigned long )patch_addr); +return 0 ; + } +__put_user_size (instr, patch_addr,4 , err);if (err)return err;
Signature · 1 line
2.17.1
Le 12/09/2018 à 07:20, Michael Neuling a écrit :
Quoted 47 lines
This stops us from doing code patching in init sections after they've been freed.
In this chain:
kvm_guest_init() -> kvm_use_magic_page() -> fault_in_pages_readable() -> __get_user() -> __get_user_nocheck() -> barrier_nospec();We have a code patching location at
barrier_nospec()andkvm_guest_init()is an init function. This whole chain gets inlined, so when we free the init section (hencekvm_guest_init()), this code goes away and hence should no longer be patched.We seen this as userspace memory corruption when using a memory checker while doing partition migration testing on powervm (this starts the code patching post migration via
/sys/kernel/mobility/migration). In theory, it could also happen when using/sys/kernel/debug/powerpc/barrier_nospec.With this patch there is a small change of a race if we code patch between the init section being freed and setting
SYSTEM_RUNNING(inkernel_init()) but that seems like an impractical time and small window for any code patching to occur.cc [email protected] # 4.13+Signed-off-by Michael Neuling <[email protected]>---
For stable I've marked this as v4.13+ since that's when we refactoredcode-patching.cbut it could go back even further than that. In reality though, I think we can only hit this since the first spectre/meltdown changes.v2:
Print when we skip an address--- arch/powerpc/lib/code-patching.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)arch/powerpc/lib/code-patching.c +0
-0
arch/powerpc/lib/code-patching.c line 26
Quoted 4 lines
arch/powerpc/lib/code-patching.c +1
-0@@ -23,11 +23,33 @@#include <asm/code-patching.h> #include <asm/setup.h> +
This blank line is not needed
arch/powerpc/lib/code-patching.c line 34
Quoted 8 lines
arch/powerpc/lib/code-patching.c +8
-0+static inline bool in_init_section (unsigned int *patch_addr) +{ +if (patch_addr < (unsigned int *)__init_begin) +return false; +if (patch_addr >= (unsigned int *)__init_end) +return false; +return true; +}
Can we use the existing function init_section_contains() instead of this new function ?
arch/powerpc/lib/code-patching.c line 40
Quoted 6 lines
arch/powerpc/lib/code-patching.c +6
-0+ +static inline bool init_freed (void ) +{ +return (system_state >=SYSTEM_RUNNING ); +} +
I would call this function differently, for instance init_is_finished(), because as you mentionned it doesn't exactly mean that init memory is freed.
arch/powerpc/lib/code-patching.c line 47
Quoted 7 lines
arch/powerpc/lib/code-patching.c +2
-0static int __patch_instruction (unsigned int *exec_addr,unsigned int instr,unsigned int *patch_addr) {int err; +/* Make sure we aren't patching a freed init section */ +if (in_init_section (patch_addr) &&init_freed ()) {
The test must be done on exec_addr, not on patch_addr, as patch_addr is the address where the instruction as been remapped RW for allowing its modification.
Also I think it should be tested the other way round, because the init_freed() is a simpler test which will be false most of the time once the system is running so it should be checked first.
arch/powerpc/lib/code-patching.c line 48
Quoted 1 line
arch/powerpc/lib/code-patching.c +1
-0+ printk(KERN_DEBUG "Skipping init section patching addr: 0x%lx\n",
Maybe use pr_debug() instead.
arch/powerpc/lib/code-patching.c line 49
Quoted 1 line
arch/powerpc/lib/code-patching.c +1
-0+ (unsigned long)patch_addr);
Please align second line as per Codying style.
arch/powerpc/lib/code-patching.c line 55
Quoted 7 lines
arch/powerpc/lib/code-patching.c +3
-0+return 0 ; + } +__put_user_size (instr, patch_addr,4 , err);if (err)return err;
I think it would be better to put this verification in patch_instruction() instead, to avoid RW mapping/unmapping the instruction to patch when we are not going to do the patching.
Christophe
Quoted 10 lines
arch/powerpc/lib/code-patching.c line 26
Quoted 8 lines
arch/powerpc/lib/code-patching.c +1
-0@@ -23,11 +23,33 @@#include <asm/code-patching.h> #include <asm/setup.h> =20 +=20
This blank line is not needed
Ack
Quoted 11 lines
=20
arch/powerpc/lib/code-patching.c line 34
Quoted 8 lines
arch/powerpc/lib/code-patching.c +8
-0+static inline bool in_init_section (unsigned int *patch_addr) +{ +if (patch_addr < (unsigned int *)__init_begin) +return false; + if (patch_addr >=3D (unsigned int *)__init_end) +return false; +return true; +}=20
Can we use the existing functioninit_section_contains()instead of this=
=20
Quoted 1 line
new function ?
Nice, I was looking for something like that...=20
Quoted 8 lines
arch/powerpc/lib/code-patching.c line 40
Quoted 6 lines
arch/powerpc/lib/code-patching.c +6
-0+ +static inline bool init_freed (void ) +{ + return (system_state >=3D SYSTEM_RUNNING); +} +=20
I would call this function differently, for instance init_is_finished(),=
=20
Quoted 1 line
because as you mentionned it doesn't exactly mean that init memory is fre=
ed.
Talking to Nick and mpe offline I think we are going to have to add a flag = when
we free init mem rather than doing what we have now since what we have now = has a
potential race. That change will eliminate the function entirely.
Quoted 10 lines
arch/powerpc/lib/code-patching.c line 47
Quoted 8 lines
arch/powerpc/lib/code-patching.c +2
-0static int __patch_instruction (unsigned int *exec_addr,unsigned int instr,unsigned int *patch_addr) {int err; =20 + /* Make sure we aren't patching a freed init section */ +if (in_init_section (patch_addr) &&init_freed ()) {=20
The test must be done onexec_addr, not onpatch_addr, aspatch_addris=
=20
Quoted 1 line
the address where the instruction as been remapped RW for allowing its=
=20
Quoted 1 line
modification.
Thanks for the catch
Quoted 2 lines
Also I think it should be tested the other way round, because the=20
init_freed()is a simpler test which will be false most of the time once=
=20
Quoted 1 line
the system is running so it should be checked first.
ok, I'll change.
Quoted 4 lines
arch/powerpc/lib/code-patching.c line 50
Quoted 2 lines
arch/powerpc/lib/code-patching.c +1
-0+ printk(KERN_DEBUG "Skipping init section patching addr: 0x%lx\n",=20
Maybe usepr_debug()instead.
Sure.
Quoted 4 lines
=20
arch/powerpc/lib/code-patching.c line 49
Quoted 1 line
arch/powerpc/lib/code-patching.c +1
-0+ (unsigned long)patch_addr);=20
Please align second line as per Codying style.
Sorry I can't see what's wrong. You're (or Cody :-P) going to have to spell=
it
this out for me...
Quoted 12 lines
=20
Quoted 7 lines
arch/powerpc/lib/code-patching.c +3
-0+return 0 ; + } +__put_user_size (instr, patch_addr,4 , err);if (err)
return err;
=20=20
I think it would be better to put this verification in=20patch_instruction()instead, to avoid RW mapping/unmapping the=20 instruction to patch when we are not going to do the patching.
If we do it there then we miss the raw_patch_intruction case.
IMHO I don't think we need to optimise this rare and non-critical path.=20
Mikey
Quoted 71 lines · Michael Neuling
Quoted 10 lines
arch/powerpc/lib/code-patching.c line 26
Quoted 8 lines
arch/powerpc/lib/code-patching.c +1
-0@@ -23,11 +23,33 @@#include <asm/code-patching.h> #include <asm/setup.h> +This blank line is not needed
Ack
Quoted 12 lines
arch/powerpc/lib/code-patching.c line 34
Quoted 9 lines
arch/powerpc/lib/code-patching.c +8
-0+static inline bool in_init_section (unsigned int *patch_addr) +{ +if (patch_addr < (unsigned int *)__init_begin) +return false; +if (patch_addr >= (unsigned int *)__init_end) +return false; +return true; +}Can we use the existing function
init_section_contains()instead of this new function ?Nice, I was looking for something like that...
Quoted 9 lines
arch/powerpc/lib/code-patching.c line 40
Quoted 6 lines
arch/powerpc/lib/code-patching.c +6
-0+ +static inline bool init_freed (void ) +{ +return (system_state >=SYSTEM_RUNNING ); +} +I would call this function differently, for instance
init_is_finished(), because as you mentionned it doesn't exactly mean that init memory is freed.Talking to Nick and mpe offline I think we are going to have to add a flag when we free init mem rather than doing what we have now since what we have now has a potential race. That change will eliminate the function entirely.
Quoted 6 lines
Quoted 6 lines
arch/powerpc/lib/code-patching.c +0
-0static int __patch_instruction (unsigned int *exec_addr,unsigned int instr,unsigned int *patch_addr) {int err;Quoted 6 lines
arch/powerpc/lib/code-patching.c line 47
Quoted 2 lines
arch/powerpc/lib/code-patching.c +2
-0+/* Make sure we aren't patching a freed init section */ +if (in_init_section (patch_addr) &&init_freed ()) {The test must be done on
exec_addr, not onpatch_addr, aspatch_addris the address where the instruction as been remapped RW for allowing its modification.Thanks for the catch
Quoted 3 lines
Also I think it should be tested the other way round, because the
init_freed()is a simpler test which will be false most of the time once the system is running so it should be checked first.ok, I'll change.
Quoted 4 lines
arch/powerpc/lib/code-patching.c line 50
Quoted 2 lines
arch/powerpc/lib/code-patching.c +1
-0+ printk(KERN_DEBUG "Skipping init section patching addr: 0x%lx\n",Maybe use
pr_debug()instead.Sure.
Quoted 1 line
Quoted 6 lines
Quoted 3 lines
arch/powerpc/lib/code-patching.c line 49
Quoted 1 line
arch/powerpc/lib/code-patching.c +1
-0+ (unsigned long)patch_addr);Please align second line as per Codying style.
Sorry I can't see what's wrong. You're (or Cody :-P) going to have to spell it this out for me...
I suspect that the suggestion is the opening parenthesis of "(unsigned long)" should sit directly under the "K" of "KERN_DEBUG". I'm pretty sure Documentation/process/coding-style.rst is very adamant that all identation is always 8 characters and spaces should never be used, but there still seems to be a lot of places/suggestions that argument lists that spill over multiple lines should be space indented to align with the very first argument at the top level. So, I guess I'm not sure what the desire is here. Although moving to pr_debug might fit it to a single line anyways. ;)
-Tyrel
Quoted 20 lines
Quoted 13 lines
Quoted 9 lines
arch/powerpc/lib/code-patching.c +3
-0+return 0 ; + } +__put_user_size (instr, patch_addr,4 , err);if (err)
return err;I think it would be better to put this verification in
patch_instruction()instead, to avoid RW mapping/unmapping the instruction to patch when we are not going to do the patching.If we do it there then we miss the
raw_patch_intructioncase.IMHO I don't think we need to optimise this rare and non-critical path.
Mikey
Le 13/09/2018 à 03:21, Tyrel Datwyler a écrit :
Quoted 11 lines · Michael Neuling
Quoted 8 lines
Quoted 5 lines
arch/powerpc/lib/code-patching.c line 49
Quoted 3 lines
arch/powerpc/lib/code-patching.c +1
-0+ (unsigned long)patch_addr);Please align second line as per Codying style.
Sorry I can't see what's wrong. You're (or Cody :-P) going to have to spell it this out for me...
I suspect that the suggestion is the opening parenthesis of "(unsigned long)" should sit directly under the "K" of "
KERN_DEBUG". I'm pretty sureDocumentation/process/coding-style.rstis very adamant that all identation is always 8 characters and spaces should never be used, but there still seems to be a lot of places/suggestions that argument lists that spill over multiple lines should be space indented to align with the very first argument at the top level. So, I guess I'm not sure what the desire is here. Although moving topr_debugmight fit it to a single line anyways. ;)
It is exactly that, as reported by checkpatch, look at https://patchwork.ozlabs.org/patch/968850/
Christophe
Quoted 15 lines · Christophe LEROY
=20
Le 13/09/2018 =C3=A0 03:21, Tyrel Datwyler a =C3=A9crit :Quoted 13 lines · Michael Neuling
Quoted 9 lines
=20
Quoted 4 lines
=20
arch/powerpc/lib/code-patching.c line 49
Quoted 1 line
arch/powerpc/lib/code-patching.c +1
-0+ (unsigned long)patch_addr);=20
Please align second line as per Codying style.=20
Sorry I can't see what's wrong. You're (or Cody :-P) going to have to spell it
this out for me...=20
I suspect that the suggestion is the opening parenthesis of "(unsigned long)" should sit directly under the "K" of "KERN_DEBUG". I'm pretty su=
re
Quoted 1 line
Quoted 1 line
Documentation/process/coding-style.rstis very adamant that all identat=
ion
Quoted 2 lines
Quoted 2 lines
is always 8 characters and spaces should never be used, but there still seems to be a lot of places/suggestions that argument lists that spill =
over
Quoted 2 lines
Quoted 2 lines
multiple lines should be space indented to align with the very first argument at the top level. So, I guess I'm not sure what the desire is =
here.
Quoted 4 lines
Quoted 1 line
Although moving to
pr_debugmight fit it to a single line anyways. ;)=20
It is exactly that, as reported by checkpatch, look at=20
https://patchwork.ozlabs.org/patch/968850/
Sweet... looks like v3 is clean
https://patchwork.ozlabs.org/patch/969241/
Mikey
Le 13/09/2018 à 02:36, Michael Neuling a écrit :
Quoted 91 lines
Quoted 10 lines
arch/powerpc/lib/code-patching.c line 26
Quoted 8 lines
arch/powerpc/lib/code-patching.c +1
-0@@ -23,11 +23,33 @@#include <asm/code-patching.h> #include <asm/setup.h> +This blank line is not needed
Ack
Quoted 12 lines
arch/powerpc/lib/code-patching.c line 34
Quoted 9 lines
arch/powerpc/lib/code-patching.c +8
-0+static inline bool in_init_section (unsigned int *patch_addr) +{ +if (patch_addr < (unsigned int *)__init_begin) +return false; +if (patch_addr >= (unsigned int *)__init_end) +return false; +return true; +}Can we use the existing function
init_section_contains()instead of this new function ?Nice, I was looking for something like that...
Quoted 9 lines
arch/powerpc/lib/code-patching.c line 40
Quoted 6 lines
arch/powerpc/lib/code-patching.c +6
-0+ +static inline bool init_freed (void ) +{ +return (system_state >=SYSTEM_RUNNING ); +} +I would call this function differently, for instance
init_is_finished(), because as you mentionned it doesn't exactly mean that init memory is freed.Talking to Nick and mpe offline I think we are going to have to add a flag when we free init mem rather than doing what we have now since what we have now has a potential race. That change will eliminate the function entirely.
Quoted 6 lines
Quoted 6 lines
arch/powerpc/lib/code-patching.c +0
-0static int __patch_instruction (unsigned int *exec_addr,unsigned int instr,unsigned int *patch_addr) {int err;Quoted 6 lines
arch/powerpc/lib/code-patching.c line 47
Quoted 2 lines
arch/powerpc/lib/code-patching.c +2
-0+/* Make sure we aren't patching a freed init section */ +if (in_init_section (patch_addr) &&init_freed ()) {The test must be done on
exec_addr, not onpatch_addr, aspatch_addris the address where the instruction as been remapped RW for allowing its modification.Thanks for the catch
Quoted 3 lines
Also I think it should be tested the other way round, because the
init_freed()is a simpler test which will be false most of the time once the system is running so it should be checked first.ok, I'll change.
Quoted 4 lines
arch/powerpc/lib/code-patching.c line 50
Quoted 2 lines
arch/powerpc/lib/code-patching.c +1
-0+ printk(KERN_DEBUG "Skipping init section patching addr: 0x%lx\n",Maybe use
pr_debug()instead.Sure.
Quoted 4 lines
arch/powerpc/lib/code-patching.c line 49
Quoted 2 lines
arch/powerpc/lib/code-patching.c +1
-0+ (unsigned long)patch_addr);Please align second line as per Codying style.
Sorry I can't see what's wrong. You're (or Cody :-P) going to have to spell it this out for me...
Quoted 12 lines
Quoted 8 lines
arch/powerpc/lib/code-patching.c +3
-0+return 0 ; + } +__put_user_size (instr, patch_addr,4 , err);if (err) return err;I think it would be better to put this verification in
patch_instruction()instead, to avoid RW mapping/unmapping the instruction to patch when we are not going to do the patching.
Quoted 1 line
If we do it there then we miss the
raw_patch_intructioncase.
raw_patch_instruction() can only be used during init. Once kernel memory has been marked readonly, raw_patch_instruction() cannot be used anymore. And mark_readonly() is called immediately after free_initmem()
Christophe
Quoted 5 lines
IMHO I don't think we need to optimise this rare and non-critical path.
Mikey