tcg/ppc: Fix race in goto_tb implementation
PATCH 1/1 tcg/ppc: Fix race in goto_tb implementation
Commit 20b6643324 ("tcg/ppc: Reorg goto_tb implementation") modified goto_tb to ensure only a single instruction was patched to prevent incorrect behaviour if a thread was in the middle of multiple instructions when they were replaced. However this introduced a race between loading the jmp target into TCG_REG_TB and patching and executing the direct branch.
The relevent part of the goto_tb implementation:
ld TCG_REG_TB, TARGET_ADDR_LOCATION(TCG_REG_TB)
patch_location:
mtctr TCG_REG_TB
bctr
tb_target_set_jmp_target() will replace 'patch_location' with a direct branch if the target is in range. The direct branch now relies on TCG_REG_TB being set up correctly by the ld. Prior to this commit multiple instructions were patched in for the direct branch case; these instructions would initalise TCG_REG_TB to the same value as the branch target.
Imagine the following sequence:
1) Thread A is executing the goto_tb sequence and loads the jmp target into TCG_REG_TB.
2) Thread B updates the jmp target address and calls tb_target_set_jmp_target(). This patches a new direct branch into the goto_tb sequence.
3) Thread A executes the newly patched direct branch. The value in TCG_REG_TB still contains the old jmp target.
TCG_REG_TB MUST contain the translation block's tc.ptr. Execution will eventually crash after performing memory accesses generated from a faulty value in TCG_REG_TB.
This presents as segfaults or illegal instruction exceptions.
Do not revert commit 20b6643324 as it did fix a different race condition. Instead remove the direct branch optimization and always use indirect branches.
The direct branch optimization can be re-added later with a race free sequence.
Gitlab issue: https://gitlab.com/qemu-project/qemu/-/issues/1726
--- tcg/ppc/tcg-target.c.inc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tcg/ppc/tcg-target.c.inc +4 -4
@@ -2533,11 +2533,10 @@ static void tcg_out_goto_tb(TCGContext *s, int which) ptrdiff_t offset = tcg_tbrel_diff(s, (void *)ptr); tcg_out_mem_long(s, LD, LDX, TCG_REG_TB, TCG_REG_TB, offset); - /* Direct branch will be patched by tb_target_set_jmp_target. */ + /* TODO: Use direct branches when possible. */ set_jmp_insn_offset(s, which); tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR); - /* When branch is out of range, fall through to indirect. */ tcg_out32(s, BCCTR | BO_ALWAYS); /* For the unlinked case, need to reset TCG_REG_TB. */
@@ -2565,10 +2564,11 @@ void tb_target_set_jmp_target(const TranslationBlock *tb, int n, intptr_t diff = addr - jmp_rx; tcg_insn_unit insn; + if (USE_REG_TB) + return; + if (in_range_b(diff)) { insn = B | (diff & 0x3fffffc); - } else if (USE_REG_TB) { - insn = MTSPR | RS(TCG_REG_TB) | CTR; } else { insn = NOP; }
Signature · 1 line
2.39.3
Quoted 46 lines · Jordan Niethe
Commit
20b6643324("tcg/ppc: Reorggoto_tbimplementation") modifiedgoto_tbto ensure only a single instruction was patched to prevent incorrect behaviour if a thread was in the middle of multiple instructions when they were replaced. However this introduced a race between loading the jmp target intoTCG_REG_TBand patching and executing the direct branch.The relevent part of the
goto_tbimplementation:ld
TCG_REG_TB,TARGET_ADDR_LOCATION(TCG_REG_TB)
patch_location:
mtctrTCG_REG_TB
bctr
tb_target_set_jmp_target()will replace 'patch_location' with a direct branch if the target is in range. The direct branch now relies onTCG_REG_TBbeing set up correctly by the ld. Prior to this commit multiple instructions were patched in for the direct branch case; these instructions would initaliseTCG_REG_TBto the same value as the branch target.Imagine the following sequence:
1) Thread A is executing the
goto_tbsequence and loads the jmp target intoTCG_REG_TB.2) Thread B updates the jmp target address and calls
tb_target_set_jmp_target(). This patches a new direct branch into thegoto_tbsequence.3) Thread A executes the newly patched direct branch. The value in
TCG_REG_TBstill contains the old jmp target.
TCG_REG_TBMUST contain the translation block's tc.ptr. Execution will eventually crash after performing memory accesses generated from a faulty value inTCG_REG_TB.This presents as segfaults or illegal instruction exceptions.
Do not revert commit
20b6643324as it did fix a different race condition. Instead remove the direct branch optimization and always use indirect branches.The direct branch optimization can be re-added later with a race free sequence.
Quoted 7 lines
Gitlab issue: https://gitlab.com/qemu-project/qemu/-/issues/1726
Fixes 20b6643324 ("tcg/ppc: Reorg goto_tb implementation")Reported-by Anushree Mathur <[email protected]>Co-developed-by Benjamin Gray <[email protected]>Signed-off-by Jordan Niethe <[email protected]>
I tested with -smp 2/4/8/16 and it works fine with all.
Thanks,
Anushree-Mathur
Quoted 35 lines
--- tcg/ppc/tcg-target.c.inc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)tcg/ppc/tcg-target.c.inc +4
-4@@ -2533,11 +2533,10 @@ static void tcg_out_goto_tb(TCGContext *s, int which) ptrdiff_t offset = tcg_tbrel_diff(s, (void *)ptr); tcg_out_mem_long(s, LD, LDX, TCG_REG_TB, TCG_REG_TB, offset); - /* Direct branch will be patched by tb_target_set_jmp_target. */ + /* TODO: Use direct branches when possible. */ set_jmp_insn_offset(s, which); tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR); - /* When branch is out of range, fall through to indirect. */ tcg_out32(s, BCCTR | BO_ALWAYS); /* For the unlinked case, need to reset TCG_REG_TB. */@@ -2565,10 +2564,11 @@ void tb_target_set_jmp_target(const TranslationBlock *tb, int n, intptr_t diff = addr - jmp_rx; tcg_insn_unit insn; + if (USE_REG_TB) + return; + if (in_range_b(diff)) { insn = B | (diff & 0x3fffffc); - } else if (USE_REG_TB) { - insn = MTSPR | RS(TCG_REG_TB) | CTR; } else { insn = NOP; }
Quoted 75 lines · Jordan Niethe
Commit
20b6643324("tcg/ppc: Reorggoto_tbimplementation") modifiedgoto_tbto ensure only a single instruction was patched to prevent incorrect behaviour if a thread was in the middle of multiple instructions when they were replaced. However this introduced a race between loading the jmp target intoTCG_REG_TBand patching and executing the direct branch.The relevent part of the
goto_tbimplementation:ld
TCG_REG_TB,TARGET_ADDR_LOCATION(TCG_REG_TB)
patch_location:
mtctrTCG_REG_TB
bctr
tb_target_set_jmp_target()will replace 'patch_location' with a direct branch if the target is in range. The direct branch now relies onTCG_REG_TBbeing set up correctly by the ld. Prior to this commit multiple instructions were patched in for the direct branch case; these instructions would initaliseTCG_REG_TBto the same value as the branch target.Imagine the following sequence:
1) Thread A is executing the
goto_tbsequence and loads the jmp target intoTCG_REG_TB.2) Thread B updates the jmp target address and calls
tb_target_set_jmp_target(). This patches a new direct branch into thegoto_tbsequence.3) Thread A executes the newly patched direct branch. The value in
TCG_REG_TBstill contains the old jmp target.
TCG_REG_TBMUST contain the translation block's tc.ptr. Execution will eventually crash after performing memory accesses generated from a faulty value inTCG_REG_TB.This presents as segfaults or illegal instruction exceptions.
Do not revert commit
20b6643324as it did fix a different race condition. Instead remove the direct branch optimization and always use indirect branches.The direct branch optimization can be re-added later with a race free sequence.
Gitlab issue: https://gitlab.com/qemu-project/qemu/-/issues/1726
Fixes 20b6643324 ("tcg/ppc: Reorg goto_tb implementation")Reported-by Anushree Mathur <[email protected]>Co-developed-by Benjamin Gray <[email protected]>Signed-off-by Jordan Niethe <[email protected]>--- tcg/ppc/tcg-target.c.inc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)tcg/ppc/tcg-target.c.inc +1
-2@@ -2533,11 +2533,10 @@ static void tcg_out_goto_tb(TCGContext *s, int which) ptrdiff_t offset = tcg_tbrel_diff(s, (void *)ptr); tcg_out_mem_long(s, LD, LDX, TCG_REG_TB, TCG_REG_TB, offset); - /* Direct branch will be patched by tb_target_set_jmp_target. */ + /* TODO: Use direct branches when possible. */ set_jmp_insn_offset(s, which); tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR); - /* When branch is out of range, fall through to indirect. */ tcg_out32(s, BCCTR | BO_ALWAYS); /* For the unlinked case, need to reset TCG_REG_TB. */
tcg/ppc/tcg-target.c.inc void tb_target_set_jmp_target(const TranslationBlock *tb, int n, · line 2569
Quoted 6 lines
tcg/ppc/tcg-target.c.inc +3
-0@@ -2565,10 +2564,11 @@ void tb_target_set_jmp_target(const TranslationBlock *tb, int n, intptr_t diff = addr - jmp_rx; tcg_insn_unit insn; + if (USE_REG_TB) + return; +
Braces. Otherwise,
This is an excellent reason to add support for power10, and its pc-relative references. We can disable REG_TB for that case.
r~
Quoted 40 lines · Richard Henderson
Quoted 39 lines · Jordan Niethe
Commit
20b6643324("tcg/ppc: Reorggoto_tbimplementation") modifiedgoto_tbto ensure only a single instruction was patched to prevent incorrect behaviour if a thread was in the middle of multiple instructions when they were replaced. However this introduced a race between loading the jmp target intoTCG_REG_TBand patching and executing the direct branch.The relevent part of the
goto_tbimplementation:ld
TCG_REG_TB,TARGET_ADDR_LOCATION(TCG_REG_TB)patch_location:
mtctrTCG_REG_TB
bctr
tb_target_set_jmp_target()will replace 'patch_location' with a direct branch if the target is in range. The direct branch now relies onTCG_REG_TBbeing set up correctly by the ld. Prior to this commit multiple instructions were patched in for the direct branch case; these instructions would initaliseTCG_REG_TBto the same value as the branch target.Imagine the following sequence:
1) Thread A is executing the
goto_tbsequence and loads the jmp target intoTCG_REG_TB.2) Thread B updates the jmp target address and calls
tb_target_set_jmp_target(). This patches a new direct branch into thegoto_tbsequence.3) Thread A executes the newly patched direct branch. The value in
TCG_REG_TBstill contains the old jmp target.
TCG_REG_TBMUST contain the translation block's tc.ptr. Execution will eventually crash after performing memory accesses generated from a faulty value inTCG_REG_TB.This presents as segfaults or illegal instruction exceptions.
Quoted 8 lines
Quoted 8 lines
Do not revert commit
20b6643324as it did fix a different race condition. Instead remove the direct branch optimization and always use indirect branches.The direct branch optimization can be re-added later with a race free sequence.
Gitlab issue: https://gitlab.com/qemu-project/qemu/-/issues/1726
Resolves: ...
Please send a v2 with updated tags. Run scripts/checkpatch.pl also.
Thanks,
C.
Quoted 39 lines
Quoted 25 lines
Fixes 20b6643324 ("tcg/ppc: Reorg goto_tb implementation")Reported-by Anushree Mathur <[email protected]>Co-developed-by Benjamin Gray <[email protected]>Signed-off-by Jordan Niethe <[email protected]>---
tcg/ppc/tcg-target.c.inc | 8 ++++----1 file changed, 4 insertions(+), 4 deletions(-)tcg/ppc/tcg-target.c.inc +1
-2@@ -2533,11 +2533,10 @@ static void tcg_out_goto_tb(TCGContext *s, int which) ptrdiff_t offset = tcg_tbrel_diff(s, (void *)ptr); tcg_out_mem_long(s, LD, LDX, TCG_REG_TB, TCG_REG_TB, offset); - /* Direct branch will be patched by tb_target_set_jmp_target. */ + /* TODO: Use direct branches when possible. */ set_jmp_insn_offset(s, which); tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR); - /* When branch is out of range, fall through to indirect. */ tcg_out32(s, BCCTR | BO_ALWAYS); /* For the unlinked case, need to reset TCG_REG_TB. */tcg/ppc/tcg-target.c.inc void tb_target_set_jmp_target(const TranslationBlock *tb, int n, · line 2568
Quoted 5 lines
tcg/ppc/tcg-target.c.inc +3
-0@@ -2565,10 +2564,11 @@ void tb_target_set_jmp_target(const TranslationBlock *tb, int n, intptr_t diff = addr - jmp_rx; tcg_insn_unit insn; + if (USE_REG_TB) + return; +Braces. Otherwise,
Reviewed-by Richard Henderson <[email protected]>This is an excellent reason to add support for power10, and its pc-relative references. We can disable
REG_TBfor that case.r~
Quoted 39 lines · Jordan Niethe
Commit
20b6643324("tcg/ppc: Reorggoto_tbimplementation") modifiedgoto_tbto ensure only a single instruction was patched to prevent incorrect behaviour if a thread was in the middle of multiple instructions when they were replaced. However this introduced a race between loading the jmp target intoTCG_REG_TBand patching and executing the direct branch.The relevent part of the
goto_tbimplementation:ld
TCG_REG_TB,TARGET_ADDR_LOCATION(TCG_REG_TB)
patch_location:
mtctrTCG_REG_TB
bctr
tb_target_set_jmp_target()will replace 'patch_location' with a direct branch if the target is in range. The direct branch now relies onTCG_REG_TBbeing set up correctly by the ld. Prior to this commit multiple instructions were patched in for the direct branch case; these instructions would initaliseTCG_REG_TBto the same value as the branch target.Imagine the following sequence:
1) Thread A is executing the
goto_tbsequence and loads the jmp target intoTCG_REG_TB.2) Thread B updates the jmp target address and calls
tb_target_set_jmp_target(). This patches a new direct branch into thegoto_tbsequence.3) Thread A executes the newly patched direct branch. The value in
TCG_REG_TBstill contains the old jmp target.
TCG_REG_TBMUST contain the translation block's tc.ptr. Execution will eventually crash after performing memory accesses generated from a faulty value inTCG_REG_TB.This presents as segfaults or illegal instruction exceptions.
Quoted 8 lines
Do not revert commit
20b6643324as it did fix a different race condition. Instead remove the direct branch optimization and always use indirect branches.The direct branch optimization can be re-added later with a race free sequence.
Gitlab issue: https://gitlab.com/qemu-project/qemu/-/issues/1726
I confirm this fixes the issue hit by debian as well, - 30 runs in a row already with this patch and counting, while before it failed almost reliable on first try, sometimes on 2nd.
(this is max I can do, as I don't know tcg at all :) )
Thank you very much for the fix!
/mjt
Quoted 9 lines
tcg/ppc/tcg-target.c.inc +3
-0@@ -2565,10 +2564,11 @@ void tb_target_set_jmp_target(const TranslationBlock *tb, int n, intptr_t diff = addr - jmp_rx; tcg_insn_unit insn; + if (USE_REG_TB) + return; + if (in_range_b(diff)) { insn = B | (diff & 0x3fffffc);
On Mon, Jul 17, 2023 at 11:24 AM Jordan Niethe <[email protected]> wrote:
[snip]
Quoted 4 lines
Reported-by Anushree Mathur <[email protected]>Co-developed-by Benjamin Gray <[email protected]>Signed-off-by Jordan Niethe <[email protected]>
Sorry, should also have:
[snip]