tcg: defer tb_flush when initial region alloc fails
PATCH 1/1 tcg: defer tb_flush when initial region alloc fails
Under MTTCG, busy vCPUs can temporarily exhaust the fixed code-gen region pool by holding multiple regions. A newly registered vCPU thread that then fails tcg_region_initial_alloc() currently hits g_assert and aborts QEMU — seen during vCPU hotplug on ppc64 TCG (virt-type=qemu) with a large maxcpus.
Instead of asserting, mark the context with tb_flush_pending. The first tcg_tb_alloc() returns NULL so the existing tb_gen_code path queues a tb_flush; tcg_region_reset_all() then repacks the pool and the new thread obtains a region on retry.
This mirrors the graceful exhaustion handling already used by the runtime tcg_region_alloc() path.
--- include/tcg/tcg.h | 9 +++++++++ tcg/region.c | 15 ++++++++++++++- tcg/tcg.c | 12 ++++++++++++ 3 files changed, 35 insertions(+), 1 deletion(-)
include/tcg/tcg.h +9 -0
@@ -381,6 +381,15 @@ struct TCGContext {/* Threshold to flush the translated code buffer. */ void *code_gen_highwater; +/* +* Set when this context (a newly registered vCPU thread) could not be +* given an initial code-gen region because the region pool was +* momentarily exhausted by other busy vCPUs. Rather than aborting, the +* thread defers a tb_flush: the next tcg_tb_alloc() forces a flush + +* retry, which resets the region pool and reclaims slack for this thread. +*/ +bool tb_flush_pending; +/* Track which vCPU triggers events */ CPUState *cpu;/* *_trans */
tcg/region.c +14 -1
@@ -393,7 +393,20 @@ bool tcg_region_alloc(TCGContext *s)static void tcg_region_initial_alloc__locked (TCGContext *s) {bool err =tcg_region_alloc__locked (s); -g_assert (!err); + +/* +* The region pool can be momentarily exhausted when many busy vCPUs have +* each grabbed several regions during translation. A newly started vCPU +* thread that cannot get its initial region must not abort QEMU: instead +* flag a deferred tb_flush. The thread's first tcg_tb_alloc() will force a +* flush + retry (tcg_region_reset_all() resets region.current and +* re-packs the live contexts), which reclaims that slack and assigns this +* context a region. This mirrors the graceful runtime path in +* tcg_tb_alloc()/tcg_region_alloc(). +*/ +if (err) { + s->tb_flush_pending = true; + } }void tcg_region_initial_alloc (TCGContext *s)
tcg/tcg.c +12 -0
@@ -1830,6 +1830,18 @@ TranslationBlock *tcg_tb_alloc(TCGContext *s)TranslationBlock *tb;void *next; +/* +* A newly started vCPU thread that could not obtain its initial region +* (region pool momentarily exhausted) deferred a flush. Trigger it now: +* returning NULL makes the caller queue a tb_flush and retry, which +* resets the region pool and gives this context a valid region. Do this +* before touching code_gen_ptr, which is not yet valid for this context. +*/ +if (unlikely (s->tb_flush_pending)) { + s->tb_flush_pending = false; +return NULL ; + } + retry: tb = (void *)ROUND_UP ((uintptr_t )s->code_gen_ptr, align); next = (void *)ROUND_UP ((uintptr_t )(tb +1 ), align);
Signature · 1 line
2.54.0
Hi Richard, Chinmay,
Could you please share your review comments on this patch?
Regards,
Yogesh Vyas
Quoted 105 lines · On Wed, 5 Aug, 2026, 22:45 Yogesh Vyas, <[email protected]> wrote:
Under MTTCG, busy vCPUs can temporarily exhaust the fixed code-gen region pool by holding multiple regions. A newly registered vCPU thread that then fails
tcg_region_initial_alloc()currently hitsg_assertand aborts QEMU — seen during vCPU hotplug on ppc64 TCG (virt-type=qemu) with a large maxcpus.
Instead of asserting, mark the context withtb_flush_pending. The firsttcg_tb_alloc()returns NULL so the existingtb_gen_codepath queues atb_flush;tcg_region_reset_all()then repacks the pool and the new thread obtains a region on retry.
This mirrors the graceful exhaustion handling already used by the runtimetcg_region_alloc()path.Reported-by Anushree Mathur <[email protected]>Signed-off-by Yogesh Vyas <[email protected]>--- include/tcg/tcg.h | 9 +++++++++ tcg/region.c | 15 ++++++++++++++- tcg/tcg.c | 12 ++++++++++++ 3 files changed, 35 insertions(+), 1 deletion(-)include/tcg/tcg.h +4
-0@@ -381,6 +381,15 @@ struct TCGContext {/* Threshold to flush the translated code buffer. */ void *code_gen_highwater; +/* +* Set when this context (a newly registered vCPU thread) could not be +* given an initial code-gen region because the region pool was +* momentarily exhausted by other busy vCPUs. Rather than aborting, the
include/tcg/tcg.h +2
-0+* thread defers a tb_flush: the next tcg_tb_alloc() forces a flush + +* retry, which resets the region pool and reclaims slack for this thread.
include/tcg/tcg.h +3
-0+*/ +bool tb_flush_pending; +/* Track which vCPU triggers events */ CPUState *cpu;/* *_trans */ tcg/region.c +3
-1@@ -393,7 +393,20 @@ bool tcg_region_alloc(TCGContext *s)static void tcg_region_initial_alloc__locked (TCGContext *s) {bool err =tcg_region_alloc__locked (s); -g_assert (!err); + +/* +* The region pool can be momentarily exhausted when many busy vCPUs have
+ * each grabbed several regions during translation. A newly started
vCPU
+ * thread that cannot get its initial region must not abort QEMU:
instead
+ * flag a deferredtb_flush. The thread's firsttcg_tb_alloc()will
force atcg/region.c +2
-0+* flush + retry (tcg_region_reset_all() resets region.current and +* re-packs the live contexts), which reclaims that slack and assigns this
tcg/region.c +6
-0+* context a region. This mirrors the graceful runtime path in +* tcg_tb_alloc()/tcg_region_alloc(). +*/ +if (err) { + s->tb_flush_pending = true; + } }void tcg_region_initial_alloc (TCGContext *s)tcg/tcg.c +2
-0@@ -1830,6 +1830,18 @@ TranslationBlock *tcg_tb_alloc(TCGContext *s)TranslationBlock *tb;void *next; +/* +* A newly started vCPU thread that could not obtain its initial region
+ * (region pool momentarily exhausted) deferred a flush. Trigger it
now:tcg/tcg.c +2
-0+* returning NULL makes the caller queue a tb_flush and retry, which +* resets the region pool and gives this context a valid region. Do this
+ * before touchingcode_gen_ptr, which is not yet valid for this
context.tcg/tcg.c +6
-1+*/ +if (unlikely (s->tb_flush_pending)) { + s->tb_flush_pending = false; +return NULL ; + } + retry: tb = (void *)ROUND_UP ((uintptr_t )s->code_gen_ptr, align); next = (void *)ROUND_UP ((uintptr_t )(tb +1 ), align); --2.54.0
Hi Yogesh,
Thanks for coming up with this solution ! It looks good to me. I had a doubt though.
After tb_flush_pending is set, tcg_region_reset_all() allocates new region to all the contexts. Is there a possibility of region not being allocated here as well and the loop repeating ?
Thanks,
Chinmay
Quoted 90 lines · Yogesh Vyas
Under MTTCG, busy vCPUs can temporarily exhaust the fixed code-gen region pool by holding multiple regions. A newly registered vCPU thread that then fails
tcg_region_initial_alloc()currently hitsg_assertand aborts QEMU — seen during vCPU hotplug on ppc64 TCG (virt-type=qemu) with a large maxcpus.
Instead of asserting, mark the context withtb_flush_pending. The firsttcg_tb_alloc()returns NULL so the existingtb_gen_codepath queues atb_flush;tcg_region_reset_all()then repacks the pool and the new thread obtains a region on retry.
This mirrors the graceful exhaustion handling already used by the runtimetcg_region_alloc()path.Reported-by Anushree Mathur <[email protected]>Signed-off-by Yogesh Vyas <[email protected]>--- include/tcg/tcg.h | 9 +++++++++ tcg/region.c | 15 ++++++++++++++- tcg/tcg.c | 12 ++++++++++++ 3 files changed, 35 insertions(+), 1 deletion(-)include/tcg/tcg.h +9
-0@@ -381,6 +381,15 @@ struct TCGContext {/* Threshold to flush the translated code buffer. */ void *code_gen_highwater; +/* +* Set when this context (a newly registered vCPU thread) could not be +* given an initial code-gen region because the region pool was +* momentarily exhausted by other busy vCPUs. Rather than aborting, the +* thread defers a tb_flush: the next tcg_tb_alloc() forces a flush + +* retry, which resets the region pool and reclaims slack for this thread. +*/ +bool tb_flush_pending; +/* Track which vCPU triggers events */ CPUState *cpu;/* *_trans */ tcg/region.c +14
-1@@ -393,7 +393,20 @@ bool tcg_region_alloc(TCGContext *s)static void tcg_region_initial_alloc__locked (TCGContext *s) {bool err =tcg_region_alloc__locked (s); -g_assert (!err); + +/* +* The region pool can be momentarily exhausted when many busy vCPUs have +* each grabbed several regions during translation. A newly started vCPU +* thread that cannot get its initial region must not abort QEMU: instead +* flag a deferred tb_flush. The thread's first tcg_tb_alloc() will force a +* flush + retry (tcg_region_reset_all() resets region.current and +* re-packs the live contexts), which reclaims that slack and assigns this +* context a region. This mirrors the graceful runtime path in +* tcg_tb_alloc()/tcg_region_alloc(). +*/ +if (err) { + s->tb_flush_pending = true; + } }void tcg_region_initial_alloc (TCGContext *s)tcg/tcg.c +12
-0@@ -1830,6 +1830,18 @@ TranslationBlock *tcg_tb_alloc(TCGContext *s)TranslationBlock *tb;void *next; +/* +* A newly started vCPU thread that could not obtain its initial region +* (region pool momentarily exhausted) deferred a flush. Trigger it now: +* returning NULL makes the caller queue a tb_flush and retry, which +* resets the region pool and gives this context a valid region. Do this +* before touching code_gen_ptr, which is not yet valid for this context. +*/ +if (unlikely (s->tb_flush_pending)) { + s->tb_flush_pending = false; +return NULL ; + } + retry: tb = (void *)ROUND_UP ((uintptr_t )s->code_gen_ptr, align); next = (void *)ROUND_UP ((uintptr_t )(tb +1 ), align);
Hi Chinmay,
Thank you for your response.
No, It appears tcg_region_reset_all() always succeeds in giving at least one region to each live context as n_ctxs <= region.n, and the hotplug path does one flush and retry, therefore should not cause a repeating alloc failure.
Regards,
Yogesh Vyas
Quoted 116 lines · On Mon, 10 Aug, 2026, 16:27 Chinmay Rath, <[email protected]> wrote:
Hi Yogesh,
Thanks for coming up with this solution ! It looks good to me. I had a doubt though.
Aftertb_flush_pendingis set,tcg_region_reset_all()allocates new region to all the contexts. Is there a possibility of region not being allocated here as well and the loop repeating ?Thanks,
ChinmayQuoted 26 lines · Yogesh Vyas
Under MTTCG, busy vCPUs can temporarily exhaust the fixed code-gen region pool by holding multiple regions. A newly registered vCPU thread that then fails
tcg_region_initial_alloc()currently hitsg_assertand aborts QEMU — seen during vCPU hotplug on ppc64 TCG (virt-type=qemu) with a large maxcpus.
Instead of asserting, mark the context withtb_flush_pending. The firsttcg_tb_alloc()returns NULL so the existingtb_gen_codepath queues atb_flush;tcg_region_reset_all()then repacks the pool and the new thread obtains a region on retry.
This mirrors the graceful exhaustion handling already used by the runtimetcg_region_alloc()path.Reported-by Anushree Mathur <[email protected]>Signed-off-by Yogesh Vyas <[email protected]>--- include/tcg/tcg.h | 9 +++++++++ tcg/region.c | 15 ++++++++++++++- tcg/tcg.c | 12 ++++++++++++ 3 files changed, 35 insertions(+), 1 deletion(-)include/tcg/tcg.h +0
-0include/tcg/tcg.h struct TCGContext { · line 385
Quoted 5 lines
include/tcg/tcg.h +2
-0@@ -381,6 +381,15 @@ struct TCGContext {/* Threshold to flush the translated code buffer. */ void *code_gen_highwater; +/* +* Set when this context (a newly registered vCPU thread) could not be
include/tcg/tcg.h struct TCGContext { · line 387
Quoted 2 lines
include/tcg/tcg.h +2
-0+* given an initial code-gen region because the region pool was +* momentarily exhausted by other busy vCPUs. Rather than aborting, the
include/tcg/tcg.h struct TCGContext { · line 388
Quoted 1 line
include/tcg/tcg.h +1
-0+* thread defers a tb_flush: the next tcg_tb_alloc() forces a flush +
include/tcg/tcg.h struct TCGContext { · line 389
Quoted 1 line
include/tcg/tcg.h +1
-0+* retry, which resets the region pool and reclaims slack for this thread.
Quoted 11 lines
include/tcg/tcg.h +3
-0+*/ +bool tb_flush_pending; +/* Track which vCPU triggers events */ CPUState *cpu;/* *_trans */ tcg/region.c +0
-0tcg/region.c bool tcg_region_alloc(TCGContext *s) · line 398
Quoted 7 lines
tcg/region.c +3
-1@@ -393,7 +393,20 @@ bool tcg_region_alloc(TCGContext *s)static void tcg_region_initial_alloc__locked (TCGContext *s) {bool err =tcg_region_alloc__locked (s); -g_assert (!err); + +/* +* The region pool can be momentarily exhausted when many busy vCPUs have
tcg/region.c bool tcg_region_alloc(TCGContext *s) · line 399
Quoted 1 line
tcg/region.c +1
-0+* each grabbed several regions during translation. A newly started vCPU
tcg/region.c bool tcg_region_alloc(TCGContext *s) · line 400
Quoted 1 line
tcg/region.c +1
-0+* thread that cannot get its initial region must not abort QEMU: instead
tcg/region.c bool tcg_region_alloc(TCGContext *s) · line 401
Quoted 1 line
tcg/region.c +1
-0+* flag a deferred tb_flush. The thread's first tcg_tb_alloc() will force a
tcg/region.c bool tcg_region_alloc(TCGContext *s) · line 403
Quoted 2 lines
tcg/region.c +2
-0+* flush + retry (tcg_region_reset_all() resets region.current and +* re-packs the live contexts), which reclaims that slack and assigns this
Quoted 14 lines
tcg/region.c +6
-0+* context a region. This mirrors the graceful runtime path in +* tcg_tb_alloc()/tcg_region_alloc(). +*/ +if (err) { + s->tb_flush_pending = true; + } }void tcg_region_initial_alloc (TCGContext *s)tcg/tcg.c +0
-0tcg/tcg.c TranslationBlock *tcg_tb_alloc(TCGContext *s) · line 1834
Quoted 5 lines
tcg/tcg.c +2
-0@@ -1830,6 +1830,18 @@ TranslationBlock *tcg_tb_alloc(TCGContext *s)TranslationBlock *tb;void *next; +/* +* A newly started vCPU thread that could not obtain its initial region
tcg/tcg.c TranslationBlock *tcg_tb_alloc(TCGContext *s) · line 1835
Quoted 1 line
tcg/tcg.c +1
-0+* (region pool momentarily exhausted) deferred a flush. Trigger it now:
tcg/tcg.c TranslationBlock *tcg_tb_alloc(TCGContext *s) · line 1837
Quoted 2 lines
tcg/tcg.c +2
-0+* returning NULL makes the caller queue a tb_flush and retry, which +* resets the region pool and gives this context a valid region. Do this
tcg/tcg.c TranslationBlock *tcg_tb_alloc(TCGContext *s) · line 1838
Quoted 1 line
tcg/tcg.c +1
-0+* before touching code_gen_ptr, which is not yet valid for this context.
Quoted 9 lines
tcg/tcg.c +6
-0+*/ +if (unlikely (s->tb_flush_pending)) { + s->tb_flush_pending = false; +return NULL ; + } + retry: tb = (void *)ROUND_UP ((uintptr_t )s->code_gen_ptr, align); next = (void *)ROUND_UP ((uintptr_t )(tb +1 ), align);