Skip to contentSkip to conversation

x86/static_call: Fix __static_call_fixup()

PATCH Superseded Peter Zijlstra 4 participants 9 messages Aug 15, 2023 – Aug 17, 2023

Subsystems (2)
lore
Share & tools
lore

Peter Zijlstra

author maintainer
#0 lore raw

PATCH 1/1 x86/static_call: Fix __static_call_fixup()

Christian reported spurious module crashes after some of Song's module memory layout patches.

Turns out that if the very last instruction on the very last page of the module is a 'JMP __x86_return_thunk' then __static_call_fixup() will trip a fault and dies.

And while the module rework made this slightly more likely to happen, it's always been possible.

Fixes ee88d363d156 ("x86,static_call: Use alternative RET encoding")
---
 arch/x86/kernel/static_call.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
arch/x86/kernel/static_call.c +10 -0
index b70670a98597..2e67512d7104 100644
@@ -186,6 +186,16 @@ EXPORT_SYMBOL_GPL(arch_static_call_transform);
  */
 bool __static_call_fixup(void *tramp, u8 op, void *dest)
 {
+	/*
+	 * Not all .return_sites are a static_call trampoline (most are not).
+	 * Check if the next 3 bytes are still kernel text, if not, then this
+	 * definitely is not a trampoline and we need not worry further.
+	 *
+	 * This avoids the memcmp() below tripping over pagefaults etc..
+	 */
+	if (!kernel_text_address(tramp+7))
+		return false;
+
 	if (memcmp(tramp+5, tramp_ud, 3)) {
 		/* Not a trampoline site, not our problem. */
 		return false;
author maintainer
#1 lore raw
Quoted 2 lines · Peter Zijlstra

Christian reported spurious module crashes after some of Song's module

To clarify: module-load.

Obviously I shouldn't be writing Changelogs after 1am :-)

Quoted 37 lines

memory layout patches.

Turns out that if the very last instruction on the very last page of the module is a 'JMP __x86_return_thunk' then __static_call_fixup() will trip a fault and dies.

And while the module rework made this slightly more likely to happen, it's always been possible.

Fixes ee88d363d156 ("x86,static_call: Use alternative RET encoding")
---
 arch/x86/kernel/static_call.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
arch/x86/kernel/static_call.c +10 -0
index b70670a98597..2e67512d7104 100644
@@ -186,6 +186,16 @@ EXPORT_SYMBOL_GPL(arch_static_call_transform);
  */
 bool __static_call_fixup(void *tramp, u8 op, void *dest)
 {
+	/*
+	 * Not all .return_sites are a static_call trampoline (most are not).
+	 * Check if the next 3 bytes are still kernel text, if not, then this
+	 * definitely is not a trampoline and we need not worry further.
+	 *
+	 * This avoids the memcmp() below tripping over pagefaults etc..
+	 */
+	if (!kernel_text_address(tramp+7))
+		return false;
+
 	if (memcmp(tramp+5, tramp_ud, 3)) {
 		/* Not a trampoline site, not our problem. */
 		return false;
maintainer
#2 lore raw
In reply to Peter Zijlstra

arch/x86/kernel/static_call.c EXPORT_SYMBOL_GPL(arch_static_call_transform); · line 191

Quoted 5 lines · Peter Zijlstra

bool __static_call_fixup(void *tramp, u8 op, void *dest) {

arch/x86/kernel/static_call.c +3 -0
+	/*
+	 * Not all .return_sites are a static_call trampoline (most are not).
+	 * Check if the next 3 bytes are still kernel text, if not, then this

s/3/7 ?

arch/x86/kernel/static_call.c EXPORT_SYMBOL_GPL(arch_static_call_transform); · line 201

Quoted 10 lines
arch/x86/kernel/static_call.c +7 -0
+	 * definitely is not a trampoline and we need not worry further.
+	 *
+	 * This avoids the memcmp() below tripping over pagefaults etc..
+	 */
+	if (!kernel_text_address(tramp+7))
+		return false;
+
 	if (memcmp(tramp+5, tramp_ud, 3)) {
 		/* Not a trampoline site, not our problem. */
 		return false;

kernel_text_address() can be quite heavyweight to call in a loop during module loading. Maybe that doesn't matter much. But it would be a lot faster to only call kernel_text_address() if tramp+7 is on the next page.

Signature · 1 line
Josh
author maintainer
#3 lore raw
Quoted 8 lines · Josh Poimboeuf

arch/x86/kernel/static_call.c EXPORT_SYMBOL_GPL(arch_static_call_transform); · line 191

Quoted 5 lines · Peter Zijlstra

bool __static_call_fixup(void *tramp, u8 op, void *dest) {

arch/x86/kernel/static_call.c +3 -0
+	/*
+	 * Not all .return_sites are a static_call trampoline (most are not).
+	 * Check if the next 3 bytes are still kernel text, if not, then this

s/3/7 ?

Right, so what I meant was the 3 bytes after the return, which is 5+3, but yeah, that can be said better.

Quoted 16 lines

arch/x86/kernel/static_call.c EXPORT_SYMBOL_GPL(arch_static_call_transform); · line 201

Quoted 11 lines
arch/x86/kernel/static_call.c +7 -0
+	 * definitely is not a trampoline and we need not worry further.
+	 *
+	 * This avoids the memcmp() below tripping over pagefaults etc..
+	 */
+	if (!kernel_text_address(tramp+7))
+		return false;
+
 	if (memcmp(tramp+5, tramp_ud, 3)) {
 		/* Not a trampoline site, not our problem. */
 		return false;

kernel_text_address() can be quite heavyweight to call in a loop during module loading. Maybe that doesn't matter much. But it would be a lot faster to only call kernel_text_address() if tramp+7 is on the next page.

Oh, right, in those few configs where it doesn't use the tree. Sure can do.

#4 lore raw
In reply to Peter Zijlstra
Quoted 6 lines · On Wed, 16 Aug 2023 01:08:09 +0200 Peter Zijlstra <[email protected]> wrote:
arch/x86/kernel/static_call.c +0 -0
index b70670a98597..2e67512d7104 100644
@@ -186,6 +186,16 @@ EXPORT_SYMBOL_GPL(arch_static_call_transform);
  */

arch/x86/kernel/static_call.c EXPORT_SYMBOL_GPL(arch_static_call_transform); · line 196

Quoted 10 lines
arch/x86/kernel/static_call.c +8 -0
 bool __static_call_fixup(void *tramp, u8 op, void *dest)
 {
+	/*
+	 * Not all .return_sites are a static_call trampoline (most are not).
+	 * Check if the next 3 bytes are still kernel text, if not, then this
+	 * definitely is not a trampoline and we need not worry further.
+	 *
+	 * This avoids the memcmp() below tripping over pagefaults etc..
+	 */
+	if (!kernel_text_address(tramp+7))

The comment says "next 3 bytes" and the test is "tramp+7". Why the magic 7 number?

If the tramp is 5 bytes, shouldn't it be +8?

-- Steve

Quoted 5 lines
arch/x86/kernel/static_call.c +2 -0
+		return false;
+
 	if (memcmp(tramp+5, tramp_ud, 3)) {
 		/* Not a trampoline site, not our problem. */
 		return false;
author maintainer
#5 lore raw
Quoted 23 lines · Steven Rostedt
Quoted 6 lines · On Wed, 16 Aug 2023 01:08:09 +0200 Peter Zijlstra <[email protected]> wrote:
arch/x86/kernel/static_call.c +0 -0
index b70670a98597..2e67512d7104 100644
@@ -186,6 +186,16 @@ EXPORT_SYMBOL_GPL(arch_static_call_transform);
  */

arch/x86/kernel/static_call.c EXPORT_SYMBOL_GPL(arch_static_call_transform); · line 196

Quoted 10 lines
arch/x86/kernel/static_call.c +8 -0
 bool __static_call_fixup(void *tramp, u8 op, void *dest)
 {
+	/*
+	 * Not all .return_sites are a static_call trampoline (most are not).
+	 * Check if the next 3 bytes are still kernel text, if not, then this
+	 * definitely is not a trampoline and we need not worry further.
+	 *
+	 * This avoids the memcmp() below tripping over pagefaults etc..
+	 */
+	if (!kernel_text_address(tramp+7))

The comment says "next 3 bytes" and the test is "tramp+7". Why the magic 7 number?

Quoted 1 line

If the tramp is 5 bytes, shouldn't it be +8?

0 based, 7 is the last of the 8 bytes. +8 would be one beyond.

author maintainer
#6 lore raw

PATCH v2 1/1 x86/static_call: Fix __static_call_fixup()

In reply to Peter Zijlstra

Christian reported spurious module load crashes after some of Song's module memory layout patches.

Turns out that if the very last instruction on the very last page of the module is a 'JMP __x86_return_thunk' then __static_call_fixup() will trip a fault and die.

And while the module rework made this slightly more likely to happen, it's always been possible.

Fixes ee88d363d156 ("x86,static_call: Use alternative RET encoding")
---
 arch/x86/kernel/static_call.c |   13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/x86/kernel/static_call.c +13 -0
@@ -186,6 +186,19 @@ EXPORT_SYMBOL_GPL(arch_static_call_trans
  */
 bool __static_call_fixup(void *tramp, u8 op, void *dest)
 {
+	unsigned long addr = (unsigned long)tramp;
+	/*
+	 * Not all .return_sites are a static_call trampoline (most are not).
+	 * Check if the 3 bytes after the return are still kernel text, if not,
+	 * then this definitely is not a trampoline and we need not worry
+	 * further.
+	 *
+	 * This avoids the memcmp() below tripping over pagefaults etc..
+	 */
+	if (((addr >> PAGE_SHIFT) != ((addr + 7) >> PAGE_SHIFT)) &&
+	    !kernel_text_address(addr + 7))
+		return false;
+
 	if (memcmp(tramp+5, tramp_ud, 3)) {
 		/* Not a trampoline site, not our problem. */
 		return false;
maintainer Acked-by on v2 1/1
#7 lore raw
Quoted 8 lines · Peter Zijlstra

Christian reported spurious module load crashes after some of Song's module memory layout patches.

Turns out that if the very last instruction on the very last page of the module is a 'JMP __x86_return_thunk' then __static_call_fixup() will trip a fault and die.

Quoted 6 lines

And while the module rework made this slightly more likely to happen, it's always been possible.

Fixes ee88d363d156 ("x86,static_call: Use alternative RET encoding")
Signature · 1 line
Josh
merge bot Applied
#8 lore raw
In reply to Peter Zijlstra
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     54097309620ef0dc2d7083783dc521c6a5fef957
Gitweb:        https://git.kernel.org/tip/54097309620ef0dc2d7083783dc521c6a5fef957
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Wed, 16 Aug 2023 12:44:19 +02:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Thu, 17 Aug 2023 13:24:09 +02:00

x86/static_call: Fix __static_call_fixup()

Christian reported spurious module load crashes after some of Song's module memory layout patches.

Turns out that if the very last instruction on the very last page of the module is a 'JMP __x86_return_thunk' then __static_call_fixup() will trip a fault and die.

And while the module rework made this slightly more likely to happen, it's always been possible.

---
 arch/x86/kernel/static_call.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/x86/kernel/static_call.c +13 -0
index b70670a..77a9316 100644
@@ -186,6 +186,19 @@ EXPORT_SYMBOL_GPL(arch_static_call_transform);
  */
 bool __static_call_fixup(void *tramp, u8 op, void *dest)
 {
+	unsigned long addr = (unsigned long)tramp;
+	/*
+	 * Not all .return_sites are a static_call trampoline (most are not).
+	 * Check if the 3 bytes after the return are still kernel text, if not,
+	 * then this definitely is not a trampoline and we need not worry
+	 * further.
+	 *
+	 * This avoids the memcmp() below tripping over pagefaults etc..
+	 */
+	if (((addr >> PAGE_SHIFT) != ((addr + 7) >> PAGE_SHIFT)) &&
+	    !kernel_text_address(addr + 7))
+		return false;
+
 	if (memcmp(tramp+5, tramp_ud, 3)) {
 		/* Not a trampoline site, not our problem. */
 		return false;