Skip to contentSkip to conversation

powerpc: Avoid code patching freed init sections

PATCH Superseded Michael Neuling 3 participants 7 messages Sep 12, 2018 – Sep 13, 2018 PowerPC

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 powerpc: Avoid code patching freed init sections

This stops us from doing code patching in init sections after they've been freed.

In this chain:
kvm_guest_init() ->

    kvm_use_magic_page() ->
      fault_in_pages_readable() ->
	 __get_user() ->
	   __get_user_nocheck() ->
	     barrier_nospec();

We have a code patching location at barrier_nospec() and kvm_guest_init() is an init function. This whole chain gets inlined, so when we free the init section (hence kvm_guest_init()), this code goes away and hence should no longer be patched.

We seen this as userspace memory corruption when using a memory checker while doing partition migration testing on powervm (this starts the code patching post migration via /sys/kernel/mobility/migration). In theory, it could also happen when using /sys/kernel/debug/powerpc/barrier_nospec.

With this patch there is a small change of a race if we code patch between the init section being freed and setting SYSTEM_RUNNING (in kernel_init()) but that seems like an impractical time and small window for any code patching to occur.

---
For stable I've marked this as v4.13+ since that's when we refactored code-patching.c but it could go back even further than that. In reality though, I think we can only hit this since the first spectre/meltdown changes.

v2:
Print when we skip an address

---
 arch/powerpc/lib/code-patching.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
arch/powerpc/lib/code-patching.c +22 -0
index 850f3b8f4d..68254e7f17 100644
@@ -23,11 +23,33 @@
 #include <asm/code-patching.h>
 #include <asm/setup.h>
 
+
+static inline bool in_init_section(unsigned int *patch_addr)
+{
+	if (patch_addr < (unsigned int *)__init_begin)
+		return false;
+	if (patch_addr >= (unsigned int *)__init_end)
+		return false;
+	return true;
+}
+
+static inline bool init_freed(void)
+{
+	return (system_state >= SYSTEM_RUNNING);
+}
+
 static int __patch_instruction(unsigned int *exec_addr, unsigned int instr,
 			       unsigned int *patch_addr)
 {
 	int err;
 
+	/* Make sure we aren't patching a freed init section */
+	if (in_init_section(patch_addr) && init_freed()) {
+		printk(KERN_DEBUG "Skipping init section patching addr: 0x%lx\n",
+			(unsigned long)patch_addr);
+		return 0;
+	}
+
 	__put_user_size(instr, patch_addr, 4, err);
 	if (err)
 		return err;
Signature · 1 line
2.17.1
#1 lore raw

Le 12/09/2018 à 07:20, Michael Neuling a écrit :

Quoted 47 lines

This stops us from doing code patching in init sections after they've been freed.

In this chain:

   kvm_guest_init() ->
     kvm_use_magic_page() ->
       fault_in_pages_readable() ->
	 __get_user() ->
	   __get_user_nocheck() ->
	     barrier_nospec();

We have a code patching location at barrier_nospec() and kvm_guest_init() is an init function. This whole chain gets inlined, so when we free the init section (hence kvm_guest_init()), this code goes away and hence should no longer be patched.

We seen this as userspace memory corruption when using a memory checker while doing partition migration testing on powervm (this starts the code patching post migration via /sys/kernel/mobility/migration). In theory, it could also happen when using /sys/kernel/debug/powerpc/barrier_nospec.

With this patch there is a small change of a race if we code patch between the init section being freed and setting SYSTEM_RUNNING (in kernel_init()) but that seems like an impractical time and small window for any code patching to occur.

---
For stable I've marked this as v4.13+ since that's when we refactored code-patching.c but it could go back even further than that. In reality though, I think we can only hit this since the first spectre/meltdown changes.

v2:
Print when we skip an address

---
  arch/powerpc/lib/code-patching.c | 22 ++++++++++++++++++++++
  1 file changed, 22 insertions(+)
arch/powerpc/lib/code-patching.c +0 -0
index 850f3b8f4d..68254e7f17 100644

arch/powerpc/lib/code-patching.c line 26

Quoted 4 lines
arch/powerpc/lib/code-patching.c +1 -0
@@ -23,11 +23,33 @@
  #include <asm/code-patching.h>
  #include <asm/setup.h>
 
+

This blank line is not needed

arch/powerpc/lib/code-patching.c line 34

Quoted 8 lines
arch/powerpc/lib/code-patching.c +8 -0
+static inline bool in_init_section(unsigned int *patch_addr)
+{
+	if (patch_addr < (unsigned int *)__init_begin)
+		return false;
+	if (patch_addr >= (unsigned int *)__init_end)
+		return false;
+	return true;
+}

Can we use the existing function init_section_contains() instead of this new function ?

arch/powerpc/lib/code-patching.c line 40

Quoted 6 lines
arch/powerpc/lib/code-patching.c +6 -0
+
+static inline bool init_freed(void)
+{
+	return (system_state >= SYSTEM_RUNNING);
+}
+

I would call this function differently, for instance init_is_finished(), because as you mentionned it doesn't exactly mean that init memory is freed.

arch/powerpc/lib/code-patching.c line 47

Quoted 7 lines
arch/powerpc/lib/code-patching.c +2 -0
  static int __patch_instruction(unsigned int *exec_addr, unsigned int instr,
  			       unsigned int *patch_addr)
  {
  	int err;
 
+	/* Make sure we aren't patching a freed init section */
+	if (in_init_section(patch_addr) && init_freed()) {

The test must be done on exec_addr, not on patch_addr, as patch_addr is the address where the instruction as been remapped RW for allowing its modification.

Also I think it should be tested the other way round, because the init_freed() is a simpler test which will be false most of the time once the system is running so it should be checked first.

arch/powerpc/lib/code-patching.c line 48

Quoted 1 line
arch/powerpc/lib/code-patching.c +1 -0
+		printk(KERN_DEBUG "Skipping init section patching addr: 0x%lx\n",

Maybe use pr_debug() instead.

arch/powerpc/lib/code-patching.c line 49

Quoted 1 line
arch/powerpc/lib/code-patching.c +1 -0
+			(unsigned long)patch_addr);

Please align second line as per Codying style.

arch/powerpc/lib/code-patching.c line 55

Quoted 7 lines
arch/powerpc/lib/code-patching.c +3 -0
+		return 0;
+	}
+
  	__put_user_size(instr, patch_addr, 4, err);
  	if (err)
  		return err;

I think it would be better to put this verification in patch_instruction() instead, to avoid RW mapping/unmapping the instruction to patch when we are not going to do the patching.

Christophe

#2 lore raw
Quoted 10 lines

arch/powerpc/lib/code-patching.c line 26

Quoted 8 lines
arch/powerpc/lib/code-patching.c +1 -0
@@ -23,11 +23,33 @@
  #include <asm/code-patching.h>
  #include <asm/setup.h>
 =20
+

=20
This blank line is not needed

Ack

Quoted 11 lines

=20

arch/powerpc/lib/code-patching.c line 34

Quoted 8 lines
arch/powerpc/lib/code-patching.c +8 -0
+static inline bool in_init_section(unsigned int *patch_addr)
+{
+	if (patch_addr < (unsigned int *)__init_begin)
+		return false;
+	if (patch_addr >=3D (unsigned int *)__init_end)
+		return false;
+	return true;
+}

=20
Can we use the existing function init_section_contains() instead of this=

=20

Quoted 1 line

new function ?

Nice, I was looking for something like that...=20

Quoted 8 lines

arch/powerpc/lib/code-patching.c line 40

Quoted 6 lines
arch/powerpc/lib/code-patching.c +6 -0
+
+static inline bool init_freed(void)
+{
+	return (system_state >=3D SYSTEM_RUNNING);
+}
+

=20
I would call this function differently, for instance init_is_finished(),=

=20

Quoted 1 line

because as you mentionned it doesn't exactly mean that init memory is fre=

ed.

Talking to Nick and mpe offline I think we are going to have to add a flag = when
we free init mem rather than doing what we have now since what we have now = has a
potential race. That change will eliminate the function entirely.

Quoted 10 lines

arch/powerpc/lib/code-patching.c line 47

Quoted 8 lines
arch/powerpc/lib/code-patching.c +2 -0
  static int __patch_instruction(unsigned int *exec_addr, unsigned int
 instr,
  			       unsigned int *patch_addr)
  {
  	int err;
 =20
+	/* Make sure we aren't patching a freed init section */
+	if (in_init_section(patch_addr) && init_freed()) {

=20
The test must be done on exec_addr, not on patch_addr, as patch_addr is=

=20

Quoted 1 line

the address where the instruction as been remapped RW for allowing its=

=20

Quoted 1 line

modification.

Thanks for the catch

Quoted 2 lines

Also I think it should be tested the other way round, because the=20 init_freed() is a simpler test which will be false most of the time once=

=20

Quoted 1 line

the system is running so it should be checked first.

ok, I'll change.

Quoted 4 lines

arch/powerpc/lib/code-patching.c line 50

Quoted 2 lines
arch/powerpc/lib/code-patching.c +1 -0
+		printk(KERN_DEBUG "Skipping init section patching addr:
 0x%lx\n",

=20
Maybe use pr_debug() instead.

Sure.

Quoted 4 lines

=20

arch/powerpc/lib/code-patching.c line 49

Quoted 1 line
arch/powerpc/lib/code-patching.c +1 -0
+			(unsigned long)patch_addr);

=20
Please align second line as per Codying style.

Sorry I can't see what's wrong. You're (or Cody :-P) going to have to spell=
it
this out for me...

Quoted 12 lines

=20

Quoted 7 lines
arch/powerpc/lib/code-patching.c +3 -0
+		return 0;
+	}
+
  	__put_user_size(instr, patch_addr, 4, err);

if (err)
return err;
=20

=20
I think it would be better to put this verification in=20 patch_instruction() instead, to avoid RW mapping/unmapping the=20 instruction to patch when we are not going to do the patching.

If we do it there then we miss the raw_patch_intruction case.

IMHO I don't think we need to optimise this rare and non-critical path.=20

Mikey

#3 lore raw
Quoted 71 lines · Michael Neuling
Quoted 10 lines

arch/powerpc/lib/code-patching.c line 26

Quoted 8 lines
arch/powerpc/lib/code-patching.c +1 -0
@@ -23,11 +23,33 @@
  #include <asm/code-patching.h>
  #include <asm/setup.h>
 
+

This blank line is not needed

Ack

Quoted 12 lines

arch/powerpc/lib/code-patching.c line 34

Quoted 9 lines
arch/powerpc/lib/code-patching.c +8 -0
+static inline bool in_init_section(unsigned int *patch_addr)
+{
+	if (patch_addr < (unsigned int *)__init_begin)
+		return false;
+	if (patch_addr >= (unsigned int *)__init_end)
+		return false;
+	return true;
+}

Can we use the existing function init_section_contains() instead of this new function ?

Nice, I was looking for something like that...

Quoted 9 lines

arch/powerpc/lib/code-patching.c line 40

Quoted 6 lines
arch/powerpc/lib/code-patching.c +6 -0
+
+static inline bool init_freed(void)
+{
+	return (system_state >= SYSTEM_RUNNING);
+}
+

I would call this function differently, for instance init_is_finished(), because as you mentionned it doesn't exactly mean that init memory is freed.

Talking to Nick and mpe offline I think we are going to have to add a flag when we free init mem rather than doing what we have now since what we have now has a potential race. That change will eliminate the function entirely.

Quoted 6 lines
Quoted 6 lines
arch/powerpc/lib/code-patching.c +0 -0
  static int __patch_instruction(unsigned int *exec_addr, unsigned int
 instr,
  			       unsigned int *patch_addr)
  {
  	int err;
 
Quoted 6 lines

arch/powerpc/lib/code-patching.c line 47

Quoted 2 lines
arch/powerpc/lib/code-patching.c +2 -0
+	/* Make sure we aren't patching a freed init section */
+	if (in_init_section(patch_addr) && init_freed()) {

The test must be done on exec_addr, not on patch_addr, as patch_addr is the address where the instruction as been remapped RW for allowing its modification.

Thanks for the catch

Quoted 3 lines

Also I think it should be tested the other way round, because the init_freed() is a simpler test which will be false most of the time once the system is running so it should be checked first.

ok, I'll change.

Quoted 4 lines

arch/powerpc/lib/code-patching.c line 50

Quoted 2 lines
arch/powerpc/lib/code-patching.c +1 -0
+		printk(KERN_DEBUG "Skipping init section patching addr:
 0x%lx\n",

Maybe use pr_debug() instead.

Sure.

Quoted 1 line
Quoted 6 lines
Quoted 3 lines

arch/powerpc/lib/code-patching.c line 49

Quoted 1 line
arch/powerpc/lib/code-patching.c +1 -0
+			(unsigned long)patch_addr);

Please align second line as per Codying style.

Sorry I can't see what's wrong. You're (or Cody :-P) going to have to spell it this out for me...

I suspect that the suggestion is the opening parenthesis of "(unsigned long)" should sit directly under the "K" of "KERN_DEBUG". I'm pretty sure Documentation/process/coding-style.rst is very adamant that all identation is always 8 characters and spaces should never be used, but there still seems to be a lot of places/suggestions that argument lists that spill over multiple lines should be space indented to align with the very first argument at the top level. So, I guess I'm not sure what the desire is here. Although moving to pr_debug might fit it to a single line anyways. ;)

-Tyrel

Quoted 20 lines
Quoted 13 lines
Quoted 9 lines
arch/powerpc/lib/code-patching.c +3 -0
+		return 0;
+	}
+
  	__put_user_size(instr, patch_addr, 4, err);

if (err)
return err;

I think it would be better to put this verification in patch_instruction() instead, to avoid RW mapping/unmapping the instruction to patch when we are not going to do the patching.

If we do it there then we miss the raw_patch_intruction case.

IMHO I don't think we need to optimise this rare and non-critical path.

Mikey

#4 lore raw

Le 13/09/2018 à 03:21, Tyrel Datwyler a écrit :

Quoted 11 lines · Michael Neuling
Quoted 8 lines
Quoted 5 lines

arch/powerpc/lib/code-patching.c line 49

Quoted 3 lines
arch/powerpc/lib/code-patching.c +1 -0
+			(unsigned long)patch_addr);

Please align second line as per Codying style.

Sorry I can't see what's wrong. You're (or Cody :-P) going to have to spell it this out for me...

I suspect that the suggestion is the opening parenthesis of "(unsigned long)" should sit directly under the "K" of "KERN_DEBUG". I'm pretty sure Documentation/process/coding-style.rst is very adamant that all identation is always 8 characters and spaces should never be used, but there still seems to be a lot of places/suggestions that argument lists that spill over multiple lines should be space indented to align with the very first argument at the top level. So, I guess I'm not sure what the desire is here. Although moving to pr_debug might fit it to a single line anyways. ;)

It is exactly that, as reported by checkpatch, look at https://patchwork.ozlabs.org/patch/968850/

Christophe

#5 lore raw
Quoted 15 lines · Christophe LEROY

=20
Le 13/09/2018 =C3=A0 03:21, Tyrel Datwyler a =C3=A9crit :

Quoted 13 lines · Michael Neuling
Quoted 9 lines

=20

Quoted 4 lines

=20

arch/powerpc/lib/code-patching.c line 49

Quoted 1 line
arch/powerpc/lib/code-patching.c +1 -0
+			(unsigned long)patch_addr);

=20
Please align second line as per Codying style.

=20
Sorry I can't see what's wrong. You're (or Cody :-P) going to have to spell it
this out for me...

=20
I suspect that the suggestion is the opening parenthesis of "(unsigned long)" should sit directly under the "K" of "KERN_DEBUG". I'm pretty su=

re

Quoted 1 line
Quoted 1 line

Documentation/process/coding-style.rst is very adamant that all identat=

ion

Quoted 2 lines
Quoted 2 lines

is always 8 characters and spaces should never be used, but there still seems to be a lot of places/suggestions that argument lists that spill =

over

Quoted 2 lines
Quoted 2 lines

multiple lines should be space indented to align with the very first argument at the top level. So, I guess I'm not sure what the desire is =

here.

Quoted 4 lines
Quoted 1 line

Although moving to pr_debug might fit it to a single line anyways. ;)

=20
It is exactly that, as reported by checkpatch, look at=20
https://patchwork.ozlabs.org/patch/968850/

Sweet... looks like v3 is clean

https://patchwork.ozlabs.org/patch/969241/

Mikey

#6 lore raw
In reply to Michael Neuling

Le 13/09/2018 à 02:36, Michael Neuling a écrit :

Quoted 91 lines
Quoted 10 lines

arch/powerpc/lib/code-patching.c line 26

Quoted 8 lines
arch/powerpc/lib/code-patching.c +1 -0
@@ -23,11 +23,33 @@
   #include <asm/code-patching.h>
   #include <asm/setup.h>
 
+

This blank line is not needed

Ack

Quoted 12 lines

arch/powerpc/lib/code-patching.c line 34

Quoted 9 lines
arch/powerpc/lib/code-patching.c +8 -0
+static inline bool in_init_section(unsigned int *patch_addr)
+{
+	if (patch_addr < (unsigned int *)__init_begin)
+		return false;
+	if (patch_addr >= (unsigned int *)__init_end)
+		return false;
+	return true;
+}

Can we use the existing function init_section_contains() instead of this new function ?

Nice, I was looking for something like that...

Quoted 9 lines

arch/powerpc/lib/code-patching.c line 40

Quoted 6 lines
arch/powerpc/lib/code-patching.c +6 -0
+
+static inline bool init_freed(void)
+{
+	return (system_state >= SYSTEM_RUNNING);
+}
+

I would call this function differently, for instance init_is_finished(), because as you mentionned it doesn't exactly mean that init memory is freed.

Talking to Nick and mpe offline I think we are going to have to add a flag when we free init mem rather than doing what we have now since what we have now has a potential race. That change will eliminate the function entirely.

Quoted 6 lines
Quoted 6 lines
arch/powerpc/lib/code-patching.c +0 -0
   static int __patch_instruction(unsigned int *exec_addr, unsigned int
 instr,
   			       unsigned int *patch_addr)
   {
   	int err;
 
Quoted 6 lines

arch/powerpc/lib/code-patching.c line 47

Quoted 2 lines
arch/powerpc/lib/code-patching.c +2 -0
+	/* Make sure we aren't patching a freed init section */
+	if (in_init_section(patch_addr) && init_freed()) {

The test must be done on exec_addr, not on patch_addr, as patch_addr is the address where the instruction as been remapped RW for allowing its modification.

Thanks for the catch

Quoted 3 lines

Also I think it should be tested the other way round, because the init_freed() is a simpler test which will be false most of the time once the system is running so it should be checked first.

ok, I'll change.

Quoted 4 lines

arch/powerpc/lib/code-patching.c line 50

Quoted 2 lines
arch/powerpc/lib/code-patching.c +1 -0
+		printk(KERN_DEBUG "Skipping init section patching addr:
 0x%lx\n",

Maybe use pr_debug() instead.

Sure.

Quoted 4 lines

arch/powerpc/lib/code-patching.c line 49

Quoted 2 lines
arch/powerpc/lib/code-patching.c +1 -0
+			(unsigned long)patch_addr);

Please align second line as per Codying style.

Sorry I can't see what's wrong. You're (or Cody :-P) going to have to spell it this out for me...

Quoted 12 lines
Quoted 8 lines
arch/powerpc/lib/code-patching.c +3 -0
+		return 0;
+	}
+
   	__put_user_size(instr, patch_addr, 4, err);
   	if (err)
   		return err;

I think it would be better to put this verification in patch_instruction() instead, to avoid RW mapping/unmapping the instruction to patch when we are not going to do the patching.

Quoted 1 line

If we do it there then we miss the raw_patch_intruction case.

raw_patch_instruction() can only be used during init. Once kernel memory has been marked readonly, raw_patch_instruction() cannot be used anymore. And mark_readonly() is called immediately after free_initmem()

Christophe

Quoted 5 lines

IMHO I don't think we need to optimise this rare and non-critical path.

Mikey