irq_pipeline: Account for stage migration across faults
PATCH 2/2 irq_pipeline: Account for stage migration across faults
From: Jan Kiszka <[email protected]>
We need to unstall the inband stage when we entered for a fault over OOB and unstalled and then migrated to inband. So far we kept the inband stage stalled, causing a state corruption this way.
Introduce an additional state to irqentry_info for this, tracking the stalling state on OOB entry.
---
Note: Tested only as backport to 5.10-x86 so far.
include/linux/entry-common.h | 3 ++- kernel/entry/common.c | 10 +++++++--- 2 files changed, 9 insertions(+), 4 deletions(-)
include/linux/entry-common.h +2 -1
@@ -402,7 +402,8 @@ void irqentry_exit_to_user_mode(struct pt_regs *regs);enum irqentry_info {IRQENTRY_INBAND_UNSTALLED =0 ,IRQENTRY_INBAND_STALLED , -IRQENTRY_OOB , +IRQENTRY_OOB_UNSTALLED , +IRQENTRY_OOB_STALLED , };#ifndef irqentry_state
kernel/entry/common.c +7 -3
@@ -410,7 +410,8 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs)#ifdef CONFIG_IRQ_PIPELINE if (running_oob ()) { - ret.stage_info =IRQENTRY_OOB ; + ret.stage_info =test_oob_stall () ?IRQENTRY_OOB_STALLED : +IRQENTRY_OOB_UNSTALLED ;return ret; }#endif
@@ -577,9 +578,12 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state)/* * If pipelining interrupts, clear the in-band stall bit if * irqentry_enter() raised it in order to mirror the hardware -* state. +* state. Also clear it when we where coming from OOB_UNSTALLED, +* thus went through a migration that was caused by taking, e.g., +* a fault. */ -if (state.stage_info ==IRQENTRY_INBAND_UNSTALLED ) +if (state.stage_info ==IRQENTRY_INBAND_UNSTALLED || + state.stage_info ==IRQENTRY_OOB_UNSTALLED )unstall_inband ();#endif return ;
--
2.26.2
Quoted 51 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
We need to unstall the inband stage when we entered for a fault over OOB and unstalled and then migrated to inband. So far we kept the inband stage stalled, causing a state corruption this way.
Introduce an additional state to
irqentry_infofor this, tracking the stalling state on OOB entry.Signed-off-by Jan Kiszka <[email protected]>---
Note: Tested only as backport to 5.10-x86 so far.
include/linux/entry-common.h | 3 ++- kernel/entry/common.c | 10 +++++++--- 2 files changed, 9 insertions(+), 4 deletions(-)include/linux/entry-common.h +2
-1@@ -402,7 +402,8 @@ void irqentry_exit_to_user_mode(struct pt_regs *regs);enum irqentry_info {IRQENTRY_INBAND_UNSTALLED =0 ,IRQENTRY_INBAND_STALLED , -IRQENTRY_OOB , +IRQENTRY_OOB_UNSTALLED , +IRQENTRY_OOB_STALLED , };#ifndef irqentry_statekernel/entry/common.c +2
-2@@ -410,7 +410,8 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs)#ifdef CONFIG_IRQ_PIPELINE if (running_oob ()) { - ret.stage_info =IRQENTRY_OOB ; + ret.stage_info =test_oob_stall () ?IRQENTRY_OOB_STALLED : +IRQENTRY_OOB_UNSTALLED ;return ret; }#endif @@ -577,9 +578,12 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state)/* * If pipelining interrupts, clear the in-band stall bit if * irqentry_enter() raised it in order to mirror the hardware -* state.
kernel/entry/common.c noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state) · line 589
Quoted 10 lines
kernel/entry/common.c +5
-1+* state. Also clear it when we where coming from OOB_UNSTALLED, +* thus went through a migration that was caused by taking, e.g., +* a fault. */ -if (state.stage_info ==IRQENTRY_INBAND_UNSTALLED ) +if (state.stage_info ==IRQENTRY_INBAND_UNSTALLED || + state.stage_info ==IRQENTRY_OOB_UNSTALLED )unstall_inband ();#endif return ;
The logic of your previous patch was correct, this one is not. IRQENTRY_OOB_UNSTALLED is useless and should not be used in this context. We have all the guarantees and information we need at the exit point above: we are running in-band, so if IRQENTRY_OOB was set on entry, we must have downgraded the current execution stage in the meantime.
The stall bit for the oob stage would be almost always cleared, except when/if a synchronous fault is taken while the pipeline is entirely locked, e.g. hard_spin_lock+irqsave held then oops, which should never happen unless the code is doing something silly. In this case, we would still want the in-band stage to be unstalled on exit after the faulting task is demoted from a mere logic standpoint, although without much hope for the kernel sanity afterwards. Typically, the oob stage must not be stalled when raw_copy_to/from_user() is invoked. IOW, this new state is not conveying any valuable information.
Please resend a new iteration of your first patch, dropping the redundant check for running_inband(), this one would be fine.
Signature · 1 line
Philippe.
Quoted 82 lines · Philippe Gerum via Xenomai
Quoted 51 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
We need to unstall the inband stage when we entered for a fault over OOB and unstalled and then migrated to inband. So far we kept the inband stage stalled, causing a state corruption this way.
Introduce an additional state to
irqentry_infofor this, tracking the stalling state on OOB entry.Signed-off-by Jan Kiszka <[email protected]>---
Note: Tested only as backport to 5.10-x86 so far.
include/linux/entry-common.h | 3 ++- kernel/entry/common.c | 10 +++++++--- 2 files changed, 9 insertions(+), 4 deletions(-)include/linux/entry-common.h +2
-1@@ -402,7 +402,8 @@ void irqentry_exit_to_user_mode(struct pt_regs *regs);enum irqentry_info {IRQENTRY_INBAND_UNSTALLED =0 ,IRQENTRY_INBAND_STALLED , -IRQENTRY_OOB , +IRQENTRY_OOB_UNSTALLED , +IRQENTRY_OOB_STALLED , };#ifndef irqentry_statekernel/entry/common.c +2
-2@@ -410,7 +410,8 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs)#ifdef CONFIG_IRQ_PIPELINE if (running_oob ()) { - ret.stage_info =IRQENTRY_OOB ; + ret.stage_info =test_oob_stall () ?IRQENTRY_OOB_STALLED : +IRQENTRY_OOB_UNSTALLED ;return ret; }#endif @@ -577,9 +578,12 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state)/* * If pipelining interrupts, clear the in-band stall bit if * irqentry_enter() raised it in order to mirror the hardware -* state. kernel/entry/common.c noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state) · line 589
Quoted 10 lines
kernel/entry/common.c +5
-1+* state. Also clear it when we where coming from OOB_UNSTALLED, +* thus went through a migration that was caused by taking, e.g., +* a fault. */ -if (state.stage_info ==IRQENTRY_INBAND_UNSTALLED ) +if (state.stage_info ==IRQENTRY_INBAND_UNSTALLED || + state.stage_info ==IRQENTRY_OOB_UNSTALLED )unstall_inband ();#endif return ;The logic of your previous patch was correct, this one is not.
IRQENTRY_OOB_UNSTALLEDis useless and should not be used in this context. We have all the guarantees and information we need at the exit point above: we are running in-band, so ifIRQENTRY_OOBwas set on entry, we must have downgraded the current execution stage in the meantime.The stall bit for the oob stage would be almost always cleared, except when/if a synchronous fault is taken while the pipeline is entirely locked, e.g. hard_spin_lock+irqsave held then oops, which should never happen unless the code is doing something silly. In this case, we would still want the in-band stage to be unstalled on exit after the faulting task is demoted from a mere logic standpoint, although without much hope for the kernel sanity afterwards. Typically, the oob stage must not be stalled when raw_copy_to/from_user() is invoked. IOW, this new state is not conveying any valuable information.
Quoted 3 lines
Please resend a new iteration of your first patch, dropping the redundant check for
running_inband(), this one would be fine.
I would prefer adding a check that detects the folded case (OOB stalled on entry, inband on exit) and issue a warning. Or is this caught and reported elsewhere already?
Jan
Quoted 83 lines · Jan Kiszka
Quoted 82 lines · Philippe Gerum via Xenomai
Quoted 51 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
We need to unstall the inband stage when we entered for a fault over OOB and unstalled and then migrated to inband. So far we kept the inband stage stalled, causing a state corruption this way.
Introduce an additional state to
irqentry_infofor this, tracking the stalling state on OOB entry.Signed-off-by Jan Kiszka <[email protected]>---
Note: Tested only as backport to 5.10-x86 so far.
include/linux/entry-common.h | 3 ++- kernel/entry/common.c | 10 +++++++--- 2 files changed, 9 insertions(+), 4 deletions(-)include/linux/entry-common.h +2
-1@@ -402,7 +402,8 @@ void irqentry_exit_to_user_mode(struct pt_regs *regs);enum irqentry_info {IRQENTRY_INBAND_UNSTALLED =0 ,IRQENTRY_INBAND_STALLED , -IRQENTRY_OOB , +IRQENTRY_OOB_UNSTALLED , +IRQENTRY_OOB_STALLED , };#ifndef irqentry_statekernel/entry/common.c +2
-2@@ -410,7 +410,8 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs)#ifdef CONFIG_IRQ_PIPELINE if (running_oob ()) { - ret.stage_info =IRQENTRY_OOB ; + ret.stage_info =test_oob_stall () ?IRQENTRY_OOB_STALLED : +IRQENTRY_OOB_UNSTALLED ;return ret; }#endif @@ -577,9 +578,12 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state)/* * If pipelining interrupts, clear the in-band stall bit if * irqentry_enter() raised it in order to mirror the hardware -* state. kernel/entry/common.c noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state) · line 589
Quoted 10 lines
kernel/entry/common.c +5
-1+* state. Also clear it when we where coming from OOB_UNSTALLED, +* thus went through a migration that was caused by taking, e.g., +* a fault. */ -if (state.stage_info ==IRQENTRY_INBAND_UNSTALLED ) +if (state.stage_info ==IRQENTRY_INBAND_UNSTALLED || + state.stage_info ==IRQENTRY_OOB_UNSTALLED )unstall_inband ();#endif return ;The logic of your previous patch was correct, this one is not.
IRQENTRY_OOB_UNSTALLEDis useless and should not be used in this context. We have all the guarantees and information we need at the exit point above: we are running in-band, so ifIRQENTRY_OOBwas set on entry, we must have downgraded the current execution stage in the meantime.The stall bit for the oob stage would be almost always cleared, except when/if a synchronous fault is taken while the pipeline is entirely locked, e.g. hard_spin_lock+irqsave held then oops, which should never happen unless the code is doing something silly. In this case, we would still want the in-band stage to be unstalled on exit after the faulting task is demoted from a mere logic standpoint, although without much hope for the kernel sanity afterwards. Typically, the oob stage must not be stalled when raw_copy_to/from_user() is invoked. IOW, this new state is not conveying any valuable information.
Quoted 8 lines
Quoted 3 lines
Please resend a new iteration of your first patch, dropping the redundant check for
running_inband(), this one would be fine.I would prefer adding a check that detects the folded case (OOB stalled on entry, inband on exit) and issue a warning. Or is this caught and reported elsewhere already?
That case is superseded by the one I just described: taking a trap over the oob stage when it is stalled is in and of itself a sign of deep breakage (i.e. when the stall bit is set for the oob stage, not only when irqs are hard disabled in the CPU). We don't have to wait for a transition to the in-band stage to determine that. Therefore, the following snippet would be enough to trap the issue.
kernel/entry/common.c +1 -0
@@ -410,6 +410,7 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs)#ifdef CONFIG_IRQ_PIPELINE if (running_oob ()) { +WARN_ON_ONCE (irq_pipeline_debug () &&oob_irqs_disabled ()); ret.stage_info =IRQENTRY_OOB_ENTRY ;return ret; }
Signature · 1 line
Philippe.