accel: Only include qdev-realized vCPUs in global &cpus_queue
v2 cover accel: Only include qdev-realized vCPUs in global &cpus_queue
Changelog:
* drop wire/unwire hooks patches
* drop unrealize related patches
* include fixed up patches from
[PATCH 0/6] tcg: fix qemu crash when add assert_cpu_is_self() is enabled
and cleanups related to cpu->created check
https://patchew.org/QEMU/[email protected]/
as it's related to the topic (well, modulo bsd cleanup)
* CI mostly green modulo rust failure on Ubuntu
https://gitlab.com/imammedo/qemu/-/pipelines/1660855467
The goal of this series is to expose vCPUs in a stable state to the accelerators, in particular the QDev 'REALIZED' step.
To do this we split out cpu_index assignment into a separate step, and move call cpu_list_add() to the end of CPU realize stage.
I expect these changes to allow CPUState::cpu_index clarifications and simplifications, but this will be addressed (and commented) in a separate series.
As result, the series also
* fix regression intoroduced by
30933c4fb4f3d ("tcg/cputlb: remove other-cpu capability from TLB flushing")
for deatials see 'tcg:tlb: use tcg_debug_assert() in assert_cpu_is_self()'
* drops no longer needed workaround 'cpu->check' due to vCPU being exposed too early in cpus_queue.
Igor Mammedov (7):
bsd-user: drop not longer used target_reset_cpu()
loongarch: reset vcpu after it's created
m68k: reset vcpu after it's created
tcg:tlb: use tcg_debug_assert() in assert_cpu_is_self() Revert "tcg/cputlb: remove other-cpu capability from TLB flushing"
tcg: drop cpu->created check
cpus: expose only realized vCPUs to global &cpus_queue
Philippe Mathieu-Daudé (3):
accel/tcg: Simplify use of &first_cpu in rr_cpu_thread_fn() accel/kvm: Assert vCPU is created when calling kvm_dirty_ring_reap*() accel/kvm: Remove unreachable assertion in kvm_dirty_ring_reap*()
bsd-user/aarch64/target_arch_cpu.h | 5 --- bsd-user/arm/target_arch_cpu.h | 4 --- bsd-user/i386/target_arch_cpu.h | 5 --- bsd-user/riscv/target_arch_cpu.h | 4 --- bsd-user/x86_64/target_arch_cpu.h | 5 --- include/hw/core/cpu.h | 6 ++++ accel/kvm/kvm-all.c | 9 ------ accel/tcg/cputlb.c | 49 +++++++++++++++++++++--------- accel/tcg/tcg-accel-ops-rr.c | 13 +++++--- cpu-common.c | 23 ++++++++------ cpu-target.c | 2 +- hw/core/cpu-common.c | 2 ++ target/loongarch/cpu.c | 2 +- target/m68k/cpu.c | 2 +- 14 files changed, 68 insertions(+), 63 deletions(-)
Signature · 1 line
2.43.0
PATCH v2 1/10 bsd-user: drop not longer used target_reset_cpu()
target_reset_cpu() static inlines have no user,
remove them.
---
CC: Warner Losh <[email protected]>
CC: Kyle Evans <[email protected]>
--- bsd-user/aarch64/target_arch_cpu.h | 5 ----- bsd-user/arm/target_arch_cpu.h | 4 ---- bsd-user/i386/target_arch_cpu.h | 5 ----- bsd-user/riscv/target_arch_cpu.h | 4 ---- bsd-user/x86_64/target_arch_cpu.h | 5 ----- 5 files changed, 23 deletions(-)
bsd-user/aarch64/target_arch_cpu.h +0 -5
@@ -181,9 +181,4 @@ static inline void target_cpu_clone_regs(CPUARMState *env, target_ulong newsp) pstate_write(env, 0); } -static inline void target_cpu_reset (CPUArchState *env) -{ -} - -#endif /* TARGET_ARCH_CPU_H */
bsd-user/arm/target_arch_cpu.h +0 -4
@@ -206,8 +206,4 @@ static inline void target_cpu_clone_regs(CPUARMState *env, target_ulong newsp) env->regs[0] = 0; } -static inline void target_cpu_reset (CPUArchState *env) -{ -} -#endif /* TARGET_ARCH_CPU_H */
bsd-user/i386/target_arch_cpu.h +0 -5
@@ -194,9 +194,4 @@ static inline void target_cpu_clone_regs(CPUX86State *env, target_ulong newsp) env->regs[R_EAX] = 0; } -static inline void target_cpu_reset (CPUArchState *env) -{ -cpu_reset (env_cpu (env)); -} -#endif /* TARGET_ARCH_CPU_H */
bsd-user/riscv/target_arch_cpu.h +0 -4
@@ -141,8 +141,4 @@ static inline void target_cpu_clone_regs(CPURISCVState *env, target_ulong newsp) env->gpr[xT0] = 0; } -static inline void target_cpu_reset (CPUArchState *env) -{ -} -#endif /* TARGET_ARCH_CPU_H */
bsd-user/x86_64/target_arch_cpu.h +0 -5
@@ -169,9 +169,4 @@ static inline void target_cpu_clone_regs(CPUX86State *env, target_ulong newsp) env->regs[R_EAX] = 0; } -static inline void target_cpu_reset (CPUArchState *env) -{ -cpu_reset (env_cpu (env)); -} -#endif /* TARGET_ARCH_CPU_H */
Signature · 1 line
2.43.0
Re: [PATCH v2 01/10] bsd-user: drop not longer used target_reset_cpu()
Quoted 7 lines · Igor Mammedov
target_reset_cpu()static inlines have no user,
remove them.Signed-off-by Igor Mammedov <[email protected]>---
CC: Warner Losh <[email protected]>
CC: Kyle Evans <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Re: [PATCH v2 01/10] bsd-user: drop not longer used target_reset_cpu()
In reply to Igor MammedovQuoted 7 lines · Igor Mammedov
target_reset_cpu()static inlines have no user,
remove them.Signed-off-by Igor Mammedov <[email protected]>---
CC: Warner Losh <[email protected]>
CC: Kyle Evans <[email protected]>
Quoted 8 lines
--- bsd-user/aarch64/target_arch_cpu.h | 5 ----- bsd-user/arm/target_arch_cpu.h | 4 ---- bsd-user/i386/target_arch_cpu.h | 5 ----- bsd-user/riscv/target_arch_cpu.h | 4 ---- bsd-user/x86_64/target_arch_cpu.h | 5 ----- 5 files changed, 23 deletions(-)
Warner
Quoted 82 lines
bsd-user/aarch64/target_arch_cpu.h +0
-5@@ -181,9 +181,4 @@ static inline void target_cpu_clone_regs(CPUARMState *env, target_ulong newsp)pstate_write (env,0 ); } -static inline void target_cpu_reset (CPUArchState *env) -{ -} - -#endif /*
TARGET_ARCH_CPU_H*/bsd-user/arm/target_arch_cpu.h +0
-4@@ -206,8 +206,4 @@ static inline void target_cpu_clone_regs(CPUARMState *env, target_ulong newsp) env->regs[0 ] =0 ; } -static inline void target_cpu_reset (CPUArchState *env) -{ -} -#endif /*
TARGET_ARCH_CPU_H*/bsd-user/i386/target_arch_cpu.h +0
-5@@ -194,9 +194,4 @@ static inline void target_cpu_clone_regs(CPUX86State *env, target_ulong newsp) env->regs[R_EAX ] =0 ; } -static inline void target_cpu_reset (CPUArchState *env) -{ -cpu_reset (env_cpu (env)); -} -#endif /*
TARGET_ARCH_CPU_H*/bsd-user/riscv/target_arch_cpu.h +0
-4@@ -141,8 +141,4 @@ static inline void target_cpu_clone_regs(CPURISCVState *env, target_ulong newsp) env->gpr[xT0] =0 ; } -static inline void target_cpu_reset (CPUArchState *env) -{ -} -#endif /*
TARGET_ARCH_CPU_H*/bsd-user/x86_64/target_arch_cpu.h +0
-5@@ -169,9 +169,4 @@ static inline void target_cpu_clone_regs(CPUX86State *env, target_ulong newsp) env->regs[R_EAX ] =0 ; } -static inline void target_cpu_reset (CPUArchState *env) -{ -cpu_reset (env_cpu (env)); -} -#endif /*
TARGET_ARCH_CPU_H*/
--
2.43.0
PATCH v2 2/10 loongarch: reset vcpu after it's created
In reply to Igor MammedovReseting vcpu before its thread is created, caused various issues in the past for other targets. It doesn't cause issues for loongarch at the moment but to be consistent with the rest of targets, move reset during realize time after qemu_init_vcpu().
That basically prevents reset being run when when vCPU is in incositent state (i.e. accelerator hasn't initialized vCPU yet).
---
CC: Song Gao <[email protected]>
--- target/loongarch/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
target/loongarch/cpu.c +1 -1
@@ -634,8 +634,8 @@ static void loongarch_cpu_realizefn(DeviceState *dev, Error **errp)loongarch_cpu_register_gdb_regs_for_features (cs); -cpu_reset (cs);qemu_init_vcpu (cs); +cpu_reset (cs); lacc->parent_realize (dev, errp); }
Signature · 1 line
2.43.0
Re: [PATCH v2 02/10] loongarch: reset vcpu after it's created
Quoted 21 lines · Igor Mammedov
Reseting vcpu before its thread is created, caused various issues in the past for other targets. It doesn't cause issues for loongarch at the moment but to be consistent with the rest of targets, move reset during realize time after
qemu_init_vcpu().That basically prevents reset being run when when vCPU is in incositent state (i.e. accelerator hasn't initialized vCPU yet).
Signed-off-by Igor Mammedov <[email protected]>---
CC: Song Gao <[email protected]>--- target/loongarch/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)target/loongarch/cpu.c +0
-0
target/loongarch/cpu.c static void loongarch_cpu_realizefn(DeviceState *dev, Error **errp) · line 641
Quoted 9 lines
target/loongarch/cpu.c +1
-1@@ -634,8 +634,8 @@ static void loongarch_cpu_realizefn(DeviceState *dev, Error **errp)loongarch_cpu_register_gdb_regs_for_features (cs); -cpu_reset (cs);qemu_init_vcpu (cs); +cpu_reset (cs); lacc->parent_realize (dev, errp); }
yes, that is actually a problem in kvm mode. vCPU thread should be created and kvm_fd for the vCPU need valid before cpu_reset().
Re: [PATCH v2 02/10] loongarch: reset vcpu after it's created
In reply to Igor MammedovQuoted 6 lines · Igor Mammedov
Reseting vcpu before its thread is created, caused various issues in the past for other targets. It doesn't cause issues for loongarch at the moment but to be consistent with the rest of targets, move reset during realize time after
qemu_init_vcpu().That basically prevents reset being run when when vCPU is in incositent state
"inconsistent"
Quoted 3 lines
(i.e. accelerator hasn't initialized vCPU yet).
Signed-off-by Igor Mammedov <[email protected]>
Quoted 20 lines
---
CC: Song Gao <[email protected]>--- target/loongarch/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)target/loongarch/cpu.c +1
-1@@ -634,8 +634,8 @@ static void loongarch_cpu_realizefn(DeviceState *dev, Error **errp)loongarch_cpu_register_gdb_regs_for_features (cs); -cpu_reset (cs);qemu_init_vcpu (cs); +cpu_reset (cs); lacc->parent_realize (dev, errp); }
PATCH v2 3/10 m68k: reset vcpu after it's created
In reply to Igor MammedovReseting vcpu before its thread is created, caused various issues in the past for other targets. It doesn't cause issues for m68k at the moment but to be consistent with the rest of targets, move reset during realize time after qemu_init_vcpu().
That basically prevents reset being run when when vCPU is in incositent state (i.e. accelerator hasn't initialized vCPU yet).
---
CC: Laurent Vivier <[email protected]>
--- target/m68k/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
target/m68k/cpu.c +1 -1
@@ -334,8 +334,8 @@ static void m68k_cpu_realizefn(DeviceState *dev, Error **errp)m68k_cpu_init_gdb (cpu); -cpu_reset (cs);qemu_init_vcpu (cs); +cpu_reset (cs); mcc->parent_realize (dev, errp); }
Signature · 1 line
2.43.0
Re: [PATCH v2 03/10] m68k: reset vcpu after it's created
Quoted 9 lines · Igor Mammedov
Reseting vcpu before its thread is created, caused various issues in the past for other targets. It doesn't cause issues for m68k at the moment but to be consistent with the rest of targets, move reset during realize time after
qemu_init_vcpu().That basically prevents reset being run when when vCPU is in incositent state (i.e. accelerator hasn't initialized vCPU yet).
Signed-off-by Igor Mammedov <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Re: [PATCH v2 03/10] m68k: reset vcpu after it's created
In reply to Igor MammedovQuoted 6 lines · Igor Mammedov
Reseting vcpu before its thread is created, caused various issues in the past for other targets. It doesn't cause issues for m68k at the moment but to be consistent with the rest of targets, move reset during realize time after
qemu_init_vcpu().That basically prevents reset being run when when vCPU is in incositent state
(typo)
Quoted 3 lines
(i.e. accelerator hasn't initialized vCPU yet).
Signed-off-by Igor Mammedov <[email protected]>
Quoted 5 lines
---
CC: Laurent Vivier <[email protected]>--- target/m68k/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
PATCH v2 4/10 tcg:tlb: use tcg_debug_assert() in assert_cpu_is_self()
In reply to Igor Mammedovthat will enable assert_cpu_is_self when QEMU is configured with
--enable-debug
without need for manual patching DEBUG_TLB_GATE define.
Need to manually path DEBUG_TLB_GATE define to enable assert, let regression caused by [1] creep in unnoticed.
1) 30933c4fb4f3d ("tcg/cputlb: remove other-cpu capability from TLB flushing")
---
v2: revert DEBUG_TLB_GATE/DEBUG_TLB_LOG_GATE to 0 as it used to be
--- accel/tcg/cputlb.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
accel/tcg/cputlb.c +2 -5
@@ -74,11 +74,8 @@ } \ } while (0) -#define assert_cpu_is_self (cpu) do { \ - if (DEBUG_TLB_GATE) { \ - g_assert(!(cpu)->created || qemu_cpu_is_self(cpu)); \ - } \ - } while (0) +#define assert_cpu_is_self (cpu) \ + tcg_debug_assert(!(cpu)->created || qemu_cpu_is_self(cpu))/* run_on_cpu_data.target_ptr should always be big enough for a * vaddr even on 32 bit builds
Signature · 1 line
2.43.0
Re: [PATCH v2 04/10] tcg:tlb: use tcg_debug_assert() in assert_cpu_is_self()
Quoted 10 lines · Igor Mammedov
that will enable
assert_cpu_is_selfwhen QEMU is configured with
--enable-debug
without need for manual patchingDEBUG_TLB_GATEdefine.Need to manually path
DEBUG_TLB_GATEdefine to enable assert, let regression caused by [1] creep in unnoticed.1)
30933c4fb4f3d("tcg/cputlb: remove other-cpu capability from TLB flushing")Signed-off-by Igor Mammedov <[email protected]>Suggested-by Alex Bennée <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
PATCH v2 5/10 Revert "tcg/cputlb: remove other-cpu capability from TLB flushing"
In reply to Igor Mammedov1)
This reverts commit 30933c4fb4f3df95ae44c4c3c86a5df049852c01.
("tcg/cputlb: remove other-cpu capability from TLB flushing")
The commit caused a regression which went unnoticed due to affected being disabled by default (DEBUG_TLB_GATE 0) Previous patch switched to using tcg_debug_assert() so that at least on debug builds assert_cpu_is_self() path would be exercised.
And that lead to exposing regression introduced by [1] with abort during tests. to reproduce:
$ configure --target-list=x86_64-softmmu --enable-debug $ make && ./qemu-system-x86_64
accel/tcg/cputlb.c:419: tlb_flush_by_mmuidx:
Assertion `!(cpu)->created || qemu_cpu_is_self(cpu)' failed.
which is triggered by usage outside of cpu thread:
x86_cpu_new -> ... ->
x86_cpu_realizefn -> cpu_reset -> ... ->
tcg_cpu_reset_holdDrop offending commit for now, until a propper fix that doesn't break 'make check' is available.
PS:
fixup g_memdup() checkpatch error s/g_memdup/g_memdup2/
---
I'll leave it upto TCG folz to fix it up propperly.
--- accel/tcg/cputlb.c | 42 +++++++++++++++++++++++++++++++++--------- 1 file changed, 33 insertions(+), 9 deletions(-)
accel/tcg/cputlb.c +33 -9
@@ -416,9 +416,12 @@ void tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap) {tlb_debug ("mmu_idx: 0x%" PRIx16"\n" , idxmap); -assert_cpu_is_self (cpu); - -tlb_flush_by_mmuidx_async_work (cpu,RUN_ON_CPU_HOST_INT (idxmap)); +if (cpu->created && !qemu_cpu_is_self (cpu)) { +async_run_on_cpu (cpu, tlb_flush_by_mmuidx_async_work, +RUN_ON_CPU_HOST_INT (idxmap)); + }else { +tlb_flush_by_mmuidx_async_work (cpu,RUN_ON_CPU_HOST_INT (idxmap)); + } }void tlb_flush (CPUState *cpu)
@@ -607,12 +610,28 @@ void tlb_flush_page_by_mmuidx(CPUState *cpu, vaddr addr, uint16_t idxmap) {tlb_debug ("addr: %016" VADDR_PRIx" mmu_idx:%" PRIx16"\n" , addr, idxmap); -assert_cpu_is_self (cpu); -/* This should already be page aligned */ addr &=TARGET_PAGE_MASK ; -tlb_flush_page_by_mmuidx_async_0 (cpu, addr, idxmap); +if (qemu_cpu_is_self (cpu)) { +tlb_flush_page_by_mmuidx_async_0 (cpu, addr, idxmap); + }else if (idxmap <TARGET_PAGE_SIZE ) { +/* +* Most targets have only a few mmu_idx. In the case where +* we can stuff idxmap into the low TARGET_PAGE_BITS, avoid +* allocating memory for this operation. +*/ +async_run_on_cpu (cpu, tlb_flush_page_by_mmuidx_async_1, +RUN_ON_CPU_TARGET_PTR (addr | idxmap)); + }else { +TLBFlushPageByMMUIdxData *d =g_new (TLBFlushPageByMMUIdxData,1 ); + +/* Otherwise allocate a structure, freed by the worker. */ + d->addr = addr; + d->idxmap = idxmap; +async_run_on_cpu (cpu, tlb_flush_page_by_mmuidx_async_2, +RUN_ON_CPU_HOST_PTR (d)); + } }void tlb_flush_page (CPUState *cpu,vaddr addr)
@@ -775,8 +794,6 @@ void tlb_flush_range_by_mmuidx(CPUState *cpu, vaddr addr, {TLBFlushRangeData d; -assert_cpu_is_self (cpu); -/* * If all bits are significant, and len is small, * this devolves to tlb_flush_page.
@@ -797,7 +814,14 @@ void tlb_flush_range_by_mmuidx(CPUState *cpu, vaddr addr, d.idxmap = idxmap; d.bits = bits; -tlb_flush_range_by_mmuidx_async_0 (cpu, d); +if (qemu_cpu_is_self (cpu)) { +tlb_flush_range_by_mmuidx_async_0 (cpu, d); + }else { +/* Otherwise allocate a structure, freed by the worker. */ +TLBFlushRangeData *p =g_memdup2 (&d,sizeof (d)); +async_run_on_cpu (cpu, tlb_flush_range_by_mmuidx_async_1, +RUN_ON_CPU_HOST_PTR (p)); + } } void tlb_flush_page_bits_by_mmuidx(CPUState *cpu, vaddr addr,
Signature · 1 line
2.43.0
Re: [PATCH v2 05/10] Revert "tcg/cputlb: remove other-cpu capability from TLB flushing"
Quoted 14 lines · Igor Mammedov
1)
This reverts commit30933c4fb4f3df95ae44c4c3c86a5df049852c01.
("tcg/cputlb: remove other-cpu capability from TLB flushing")The commit caused a regression which went unnoticed due to affected being disabled by default (
DEBUG_TLB_GATE0) Previous patch switched to usingtcg_debug_assert()so that at least on debug buildsassert_cpu_is_self()path would be exercised.And that lead to exposing regression introduced by [1] with abort during tests. to reproduce:
$ configure --target-list=x86_64-softmmu --enable-debug $ make && ./qemu-system-x86_64
Quoted 7 lines
accel/tcg/cputlb.c:419:tlb_flush_by_mmuidx:Assertion `!(cpu)->created || qemu_cpu_is_self(cpu)' failed.which is triggered by usage outside of cpu thread:
x86_cpu_new -> ... -> x86_cpu_realizefn -> cpu_reset -> ... -> tcg_cpu_reset_hold
If the reset case is the only one I don't think we need to revert the rest of the changes as only tlb_flush needs calling. How about something like:
--8<---------------cut here---------------start------------->8---
cputlb: introduce tlb_flush_other_cpu for reset use
The commit 30933c4fb4 (tcg/cputlb: remove other-cpu capability from TLB flushing) introduced a regression that only shows up when --enable-debug-tcg is used. The main use case of tlb_flush outside of the current_cpu context is for handling reset and CPU creation. Rather than revert the commit introduce a new helper and tweak the documentation to make it clear where it should be used.
3 files changed, 26 insertions(+), 5 deletions(-) include/exec/exec-all.h | 20 ++++++++++++++++---- accel/tcg/cputlb.c | 9 +++++++++ accel/tcg/tcg-accel-ops.c | 2 +-
modified include/exec/exec-all.h
@@ -64,12 +64,24 @@ void tlb_flush_page_all_cpus_synced(CPUState *src, vaddr addr);
* tlb_flush:
* @cpu: CPU whose TLB should be flushed
*
- * Flush the entire TLB for the specified CPU. Most CPU architectures
- * allow the implementation to drop entries from the TLB at any time
- * so this is generally safe. If more selective flushing is required
- * use one of the other functions for efficiency.
+ * Flush the entire TLB for the specified current CPU.
+ *
+ * Most CPU architectures allow the implementation to drop entries
+ * from the TLB at any time so this is generally safe. If more
+ * selective flushing is required use one of the other functions for
+ * efficiency.
*/
void tlb_flush(CPUState *cpu);
+/**
+ * tlb_flush_other_cpu:
+ * @cpu: CPU whose TLB should be flushed
+ *
+ * Flush the entire TLB for a specified CPU. For cross vCPU flushes
+ * you shuld be using a more selective function. This is really only
+ * used for flushing CPUs being reset from outside their current
+ * context.
+ */
+void tlb_flush_other_cpu(CPUState *cpu);
/**
* tlb_flush_all_cpus_synced:
* @cpu: src CPU of the flush
modified accel/tcg/cputlb.c
@@ -414,6 +414,15 @@ void tlb_flush(CPUState *cpu)
tlb_flush_by_mmuidx(cpu, ALL_MMUIDX_BITS);
}
+void tlb_flush_other_cpu(CPUState *cpu)
+{
+ g_assert(!qemu_cpu_is_self(cpu));
+
+ async_run_on_cpu(cpu,
+ tlb_flush_by_mmuidx_async_work,
+ RUN_ON_CPU_HOST_INT(ALL_MMUIDX_BITS));
+}
+
void tlb_flush_by_mmuidx_all_cpus_synced(CPUState *src_cpu, uint16_t idxmap)
{
const run_on_cpu_func fn = tlb_flush_by_mmuidx_async_work;
modified accel/tcg/tcg-accel-ops.c
@@ -85,7 +85,7 @@ static void tcg_cpu_reset_hold(CPUState *cpu)
{
tcg_flush_jmp_cache(cpu);- tlb_flush(cpu);
+ tlb_flush_other_cpu(cpu);
}
--8<---------------cut here---------------end--------------->8---
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Re: [PATCH v2 05/10] Revert "tcg/cputlb: remove other-cpu capability from TLB flushing"
Quoted 24 lines · On Tue, 25 Feb 2025 12:42:24 +0000 Alex Bennée <[email protected]> wrote:
Quoted 14 lines · Igor Mammedov
1)
This reverts commit30933c4fb4f3df95ae44c4c3c86a5df049852c01.
("tcg/cputlb: remove other-cpu capability from TLB flushing")The commit caused a regression which went unnoticed due to affected being disabled by default (
DEBUG_TLB_GATE0) Previous patch switched to usingtcg_debug_assert()so that at least on debug buildsassert_cpu_is_self()path would be exercised.And that lead to exposing regression introduced by [1] with abort during tests. to reproduce:
$ configure --target-list=x86_64-softmmu --enable-debug $ make && ./qemu-system-x86_64Quoted 7 lines
accel/tcg/cputlb.c:419:tlb_flush_by_mmuidx:Assertion `!(cpu)->created || qemu_cpu_is_self(cpu)' failed.which is triggered by usage outside of cpu thread:
x86_cpu_new -> ... -> x86_cpu_realizefn -> cpu_reset -> ... -> tcg_cpu_reset_hold
Quoted 6 lines
If the reset case is the only one I don't think we need to revert the rest of the changes as only
tlb_flushneeds calling. How about something like:--8<---------------cut here---------------start------------->8---
cputlb: introducetlb_flush_other_cpufor reset use
while that works for my reproducer it's not sufficient, 'make check' is some tests fails anyway
ex:
10/378 qemu:qtest+qtest-x86_64 / qtest-x86_64/ahci-test ERROR 13.47s killed by signal 6 SIGABRT stderr:
qemu-system-x86_64: qemu/accel/tcg/cputlb.c:419: tlb_flush_by_mmuidx: Assertion `qemu_cpu_is_self(cpu)' failed. Broken pipeqemu/tests/qtest/libqtest.c:208: kill_qemu() detected QEMU death from signal 6 (Aborted) (core dumped)
Quoted 73 lines
The commit
30933c4fb4(tcg/cputlb: remove other-cpu capability from TLB flushing) introduced a regression that only shows up when --enable-debug-tcg is used. The main use case oftlb_flushoutside of thecurrent_cpucontext is for handling reset and CPU creation. Rather than revert the commit introduce a new helper and tweak the documentation to make it clear where it should be used.Signed-off-by Alex Bennée <[email protected]>3 files changed, 26 insertions(+), 5 deletions(-) include/exec/exec-all.h | 20 ++++++++++++++++---- accel/tcg/cputlb.c | 9 +++++++++ accel/tcg/tcg-accel-ops.c | 2 +-modified
include/exec/exec-all.h(quoted diff) +16
-4@@ -64,12 +64,24 @@ void tlb_flush_page_all_cpus_synced(CPUState *src, vaddr addr); * tlb_flush: * @cpu: CPU whose TLB should be flushed * - * Flush the entire TLB for the specified CPU. Most CPU architectures - * allow the implementation to drop entries from the TLB at any time - * so this is generally safe. If more selective flushing is required - * use one of the other functions for efficiency. + * Flush the entire TLB for the specified current CPU. + * + * Most CPU architectures allow the implementation to drop entries + * from the TLB at any time so this is generally safe. If more + * selective flushing is required use one of the other functions for + * efficiency. */ void tlb_flush(CPUState *cpu); +/** + * tlb_flush_other_cpu: + * @cpu: CPU whose TLB should be flushed + * + * Flush the entire TLB for a specified CPU. For cross vCPU flushes + * you shuld be using a more selective function. This is really only + * used for flushing CPUs being reset from outside their current + * context. + */ +void tlb_flush_other_cpu(CPUState *cpu); /** * tlb_flush_all_cpus_synced: * @cpu: src CPU of the flushmodified
accel/tcg/cputlb.c(quoted diff) +9
-0@@ -414,6 +414,15 @@ void tlb_flush(CPUState *cpu) tlb_flush_by_mmuidx(cpu, ALL_MMUIDX_BITS); } +void tlb_flush_other_cpu(CPUState *cpu) +{ + g_assert(!qemu_cpu_is_self(cpu)); + + async_run_on_cpu(cpu, + tlb_flush_by_mmuidx_async_work, + RUN_ON_CPU_HOST_INT(ALL_MMUIDX_BITS)); +} + void tlb_flush_by_mmuidx_all_cpus_synced(CPUState *src_cpu, uint16_t idxmap) { const run_on_cpu_func fn = tlb_flush_by_mmuidx_async_work;modified
accel/tcg/tcg-accel-ops.c(quoted diff) +1
-2@@ -85,7 +85,7 @@ static void tcg_cpu_reset_hold(CPUState *cpu) { tcg_flush_jmp_cache(cpu); - tlb_flush(cpu); + tlb_flush_other_cpu(cpu); } --8<---------------cut here---------------end--------------->8---
Re: [PATCH v2 05/10] Revert "tcg/cputlb: remove other-cpu capability from TLB flushing"
Quoted 37 lines · On Tue, 25 Feb 2025 18:19:03 +0100 Igor Mammedov <[email protected]> wrote:
Quoted 24 lines · On Tue, 25 Feb 2025 12:42:24 +0000 Alex Bennée <[email protected]> wrote:
Quoted 14 lines · Igor Mammedov
1)
This reverts commit30933c4fb4f3df95ae44c4c3c86a5df049852c01.
("tcg/cputlb: remove other-cpu capability from TLB flushing")The commit caused a regression which went unnoticed due to affected being disabled by default (
DEBUG_TLB_GATE0) Previous patch switched to usingtcg_debug_assert()so that at least on debug buildsassert_cpu_is_self()path would be exercised.And that lead to exposing regression introduced by [1] with abort during tests. to reproduce:
$ configure --target-list=x86_64-softmmu --enable-debug $ make && ./qemu-system-x86_64Quoted 7 lines
accel/tcg/cputlb.c:419:tlb_flush_by_mmuidx:Assertion `!(cpu)->created || qemu_cpu_is_self(cpu)' failed.which is triggered by usage outside of cpu thread:
x86_cpu_new -> ... -> x86_cpu_realizefn -> cpu_reset -> ... -> tcg_cpu_reset_holdQuoted 6 lines
If the reset case is the only one I don't think we need to revert the rest of the changes as only
tlb_flushneeds calling. How about something like:--8<---------------cut here---------------start------------->8---
cputlb: introducetlb_flush_other_cpufor reset usewhile that works for my reproducer it's not sufficient, 'make check' is some tests fails anyway
Quoted 7 lines
ex:
10/378 qemu:qtest+qtest-x86_64 / qtest-x86_64/ahci-test ERROR 13.47s killed by signal 6 SIGABRT stderr:
qemu-system-x86_64:qemu/accel/tcg/cputlb.c:419:tlb_flush_by_mmuidx: Assertion `qemu_cpu_is_self(cpu)' failed. Broken pipeqemu/tests/qtest/libqtest.c:208:kill_qemu()detected QEMU death from signal 6 (Aborted) (core dumped)
here is v3 rebased on top of the patch
https://gitlab.com/imammedo/qemu/-/commits/qemu_cpu_cond_v3?ref_type=heads
it seems that reset path is not the only one that relied on 'cpu->created' workaround
Quoted 77 lines
Quoted 76 lines
The commit
30933c4fb4(tcg/cputlb: remove other-cpu capability from TLB flushing) introduced a regression that only shows up when --enable-debug-tcg is used. The main use case oftlb_flushoutside of thecurrent_cpucontext is for handling reset and CPU creation. Rather than revert the commit introduce a new helper and tweak the documentation to make it clear where it should be used.Signed-off-by Alex Bennée <[email protected]>3 files changed, 26 insertions(+), 5 deletions(-) include/exec/exec-all.h | 20 ++++++++++++++++---- accel/tcg/cputlb.c | 9 +++++++++ accel/tcg/tcg-accel-ops.c | 2 +-modified
include/exec/exec-all.h(quoted diff) +16
-4@@ -64,12 +64,24 @@ void tlb_flush_page_all_cpus_synced(CPUState *src, vaddr addr); * tlb_flush: * @cpu: CPU whose TLB should be flushed * - * Flush the entire TLB for the specified CPU. Most CPU architectures - * allow the implementation to drop entries from the TLB at any time - * so this is generally safe. If more selective flushing is required - * use one of the other functions for efficiency. + * Flush the entire TLB for the specified current CPU. + * + * Most CPU architectures allow the implementation to drop entries + * from the TLB at any time so this is generally safe. If more + * selective flushing is required use one of the other functions for + * efficiency. */ void tlb_flush(CPUState *cpu); +/** + * tlb_flush_other_cpu: + * @cpu: CPU whose TLB should be flushed + * + * Flush the entire TLB for a specified CPU. For cross vCPU flushes + * you shuld be using a more selective function. This is really only + * used for flushing CPUs being reset from outside their current + * context. + */ +void tlb_flush_other_cpu(CPUState *cpu); /** * tlb_flush_all_cpus_synced: * @cpu: src CPU of the flushmodified
accel/tcg/cputlb.c(quoted diff) +9
-0@@ -414,6 +414,15 @@ void tlb_flush(CPUState *cpu) tlb_flush_by_mmuidx(cpu, ALL_MMUIDX_BITS); } +void tlb_flush_other_cpu(CPUState *cpu) +{ + g_assert(!qemu_cpu_is_self(cpu)); + + async_run_on_cpu(cpu, + tlb_flush_by_mmuidx_async_work, + RUN_ON_CPU_HOST_INT(ALL_MMUIDX_BITS)); +} + void tlb_flush_by_mmuidx_all_cpus_synced(CPUState *src_cpu, uint16_t idxmap) { const run_on_cpu_func fn = tlb_flush_by_mmuidx_async_work;modified
accel/tcg/tcg-accel-ops.c(quoted diff) +1
-2@@ -85,7 +85,7 @@ static void tcg_cpu_reset_hold(CPUState *cpu) { tcg_flush_jmp_cache(cpu); - tlb_flush(cpu); + tlb_flush_other_cpu(cpu); } --8<---------------cut here---------------end--------------->8---
PATCH v2 6/10 tcg: drop cpu->created check
In reply to Igor Mammedovprevious commits fixed 2 remaining cases where vcpu might have had 'cpu->created == false' during 1st vcpu reset (at realize time) that leads to call chain
tcg_cpu_reset_hold() => tlb_flush_by_mmuidx()
remove not need anymore check, with cpu->created always being true.
---
CC: Nicholas Piggin <[email protected]>
--- accel/tcg/cputlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
accel/tcg/cputlb.c +2 -2
@@ -75,7 +75,7 @@ } while (0)#define assert_cpu_is_self (cpu) \ - tcg_debug_assert(!(cpu)->created || qemu_cpu_is_self(cpu)) + tcg_debug_assert(qemu_cpu_is_self(cpu))/* run_on_cpu_data.target_ptr should always be big enough for a * vaddr even on 32 bit builds
@@ -416,7 +416,7 @@ void tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap) {tlb_debug ("mmu_idx: 0x%" PRIx16"\n" , idxmap); -if (cpu->created && !qemu_cpu_is_self (cpu)) { +if (!qemu_cpu_is_self (cpu)) {async_run_on_cpu (cpu, tlb_flush_by_mmuidx_async_work,RUN_ON_CPU_HOST_INT (idxmap)); }else {
Signature · 1 line
2.43.0
PATCH v2 7/10 accel/tcg: Simplify use of &first_cpu in rr_cpu_thread_fn()
In reply to Igor MammedovFrom: Philippe Mathieu-Daudé <[email protected]>
Let vCPUs wait for themselves being ready first, then other ones. This allows the first thread to starts without the global vcpu queue (thus &first_cpu) being populated.
--- accel/tcg/tcg-accel-ops-rr.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
accel/tcg/tcg-accel-ops-rr.c +8 -5
@@ -197,16 +197,19 @@ static void *rr_cpu_thread_fn(void *arg)qemu_guest_random_seed_thread_part2 (cpu->random_seed);/* wait for initial kick-off after machine start */ -while (first_cpu->stopped) { -qemu_cond_wait_bql (first_cpu->halt_cond); +while (cpu->stopped) { +CPUState *iter_cpu; + +qemu_cond_wait_bql (cpu->halt_cond);/* process any pending work */ -CPU_FOREACH (cpu) { - current_cpu = cpu; -qemu_wait_io_event_common (cpu); +CPU_FOREACH (iter_cpu) { + current_cpu = iter_cpu; +qemu_wait_io_event_common (iter_cpu); } } +g_assert (first_cpu);rr_start_kick_timer (); cpu = first_cpu;
Signature · 1 line
2.43.0
Re: [PATCH v2 07/10] accel/tcg: Simplify use of &first_cpu in rr_cpu_thread_fn()
Quoted 8 lines · Igor Mammedov
From: Philippe Mathieu-Daudé <[email protected]>
Let vCPUs wait for themselves being ready first, then other ones. This allows the first thread to starts without the global vcpu queue (thus
&first_cpu) being populated.Signed-off-by Philippe Mathieu-Daudé <[email protected]>Signed-off-by Igor Mammedov <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
PATCH v2 8/10 cpus: expose only realized vCPUs to global &cpus_queue
In reply to Igor Mammedovcpu_list_add() was doing 2 distinct things:
- assign some index to vCPU
- add unrealized (thus in inconsistent state) vCPU to &cpus_queue
Code using CPU_FOREACH() macro would iterate over possibly unrealized vCPUs, often dealt with special casing.
Instead of working around of vCPU existence in cpus_queue, split out cpu_index assignment from cpu_list_add(), and move the later to the end of realize stage, right before vCPU is let run.
---
CC: Yanan Wang <[email protected]>
CC: Zhao Liu <[email protected]>
--- include/hw/core/cpu.h | 6 ++++++ cpu-common.c | 23 ++++++++++++++--------- cpu-target.c | 2 +- hw/core/cpu-common.c | 2 ++ 4 files changed, 23 insertions(+), 10 deletions(-)
include/hw/core/cpu.h +6 -0
@@ -750,6 +750,12 @@ bool cpu_virtio_is_big_endian(CPUState *cpu);#endif /* CONFIG_USER_ONLY */ +/** +* cpu_auto_assign_cpu_index: +* @cpu: The CPU to be assigned a cpu_index +*/ +void cpu_auto_assign_cpu_index (CPUState *cpu); +/** * cpu_list_add: * @cpu: The CPU to be added to the list of CPUs.
cpu-common.c +14 -9
@@ -71,15 +71,7 @@ int cpu_get_free_index(void)return max_cpu_index; } -CPUTailQ cpus_queue =QTAILQ_HEAD_INITIALIZER (cpus_queue); -static unsigned int cpu_list_generation_id; - -unsigned int cpu_list_generation_id_get (void ) -{ -return cpu_list_generation_id; -} - -void cpu_list_add (CPUState *cpu) +void cpu_auto_assign_cpu_index (CPUState *cpu) {static bool cpu_index_auto_assigned;
@@ -91,6 +83,19 @@ void cpu_list_add(CPUState *cpu) }else {assert (!cpu_index_auto_assigned); } +} + +CPUTailQ cpus_queue =QTAILQ_HEAD_INITIALIZER (cpus_queue); +static unsigned int cpu_list_generation_id; + +unsigned int cpu_list_generation_id_get (void ) +{ +return cpu_list_generation_id; +} + +void cpu_list_add (CPUState *cpu) +{ +QEMU_LOCK_GUARD (&qemu_cpu_list_lock);QTAILQ_INSERT_TAIL_RCU (&cpus_queue, cpu, node); cpu_list_generation_id++; }
cpu-target.c +1 -1
@@ -142,7 +142,7 @@ bool cpu_exec_realizefn(CPUState *cpu, Error **errp) }/* Wait until cpu initialization complete before exposing cpu. */ -cpu_list_add (cpu); +cpu_auto_assign_cpu_index (cpu);#ifdef CONFIG_USER_ONLY assert(qdev_get_vmsd(DEVICE(cpu)) == NULL ||
hw/core/cpu-common.c +2 -0
@@ -211,6 +211,8 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp) } } +cpu_list_add (cpu); +if (dev->hotplugged) {cpu_synchronize_post_init (cpu);cpu_resume (cpu);
Signature · 1 line
2.43.0
Re: [PATCH v2 08/10] cpus: expose only realized vCPUs to global &cpus_queue
Quoted 9 lines · Igor Mammedov
cpu_list_add()was doing 2 distinct things:
- assign some index to vCPU
- add unrealized (thus in inconsistent state) vCPU to&cpus_queueCode using
CPU_FOREACH()macro would iterate over possibly unrealized vCPUs, often dealt with special casing.Instead of working around of vCPU existence in
cpus_queue, split outcpu_indexassignment fromcpu_list_add(),
Better split 2 distinct changes in 2 patches for clarity.
Quoted 98 lines
and move the later to the end of realize stage, right before vCPU is let run.
Signed-off-by Igor Mammedov <[email protected]>---
CC: Yanan Wang <[email protected]>
CC: Zhao Liu <[email protected]>--- include/hw/core/cpu.h | 6 ++++++ cpu-common.c | 23 ++++++++++++++--------- cpu-target.c | 2 +- hw/core/cpu-common.c | 2 ++ 4 files changed, 23 insertions(+), 10 deletions(-)include/hw/core/cpu.h +6
-0@@ -750,6 +750,12 @@ bool cpu_virtio_is_big_endian(CPUState *cpu);#endif /* CONFIG_USER_ONLY */ +/** +* cpu_auto_assign_cpu_index: +* @cpu: The CPU to be assigned a cpu_index +*/ +void cpu_auto_assign_cpu_index (CPUState *cpu); +/** * cpu_list_add: * @cpu: The CPU to be added to the list of CPUs. cpu-common.c +14
-9@@ -71,15 +71,7 @@ int cpu_get_free_index(void)return max_cpu_index; } -CPUTailQ cpus_queue =QTAILQ_HEAD_INITIALIZER (cpus_queue); -static unsigned int cpu_list_generation_id; - -unsigned int cpu_list_generation_id_get (void ) -{ -return cpu_list_generation_id; -} - -void cpu_list_add (CPUState *cpu) +void cpu_auto_assign_cpu_index (CPUState *cpu) {static bool cpu_index_auto_assigned;@@ -91,6 +83,19 @@ void cpu_list_add(CPUState *cpu) }else {assert (!cpu_index_auto_assigned); } +} + +CPUTailQ cpus_queue =QTAILQ_HEAD_INITIALIZER (cpus_queue); +static unsigned int cpu_list_generation_id; + +unsigned int cpu_list_generation_id_get (void ) +{ +return cpu_list_generation_id; +} + +void cpu_list_add (CPUState *cpu) +{ +QEMU_LOCK_GUARD (&qemu_cpu_list_lock);QTAILQ_INSERT_TAIL_RCU (&cpus_queue, cpu, node); cpu_list_generation_id++; }cpu-target.c +1
-1@@ -142,7 +142,7 @@ bool cpu_exec_realizefn(CPUState *cpu, Error **errp) }/* Wait until cpu initialization complete before exposing cpu. */ -cpu_list_add (cpu); +cpu_auto_assign_cpu_index (cpu);#ifdef CONFIG_USER_ONLY assert(qdev_get_vmsd(DEVICE(cpu)) == NULL ||hw/core/cpu-common.c +2
-0@@ -211,6 +211,8 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp) } } +cpu_list_add (cpu); +if (dev->hotplugged) {cpu_synchronize_post_init (cpu);cpu_resume (cpu);
Re: [PATCH v2 08/10] cpus: expose only realized vCPUs to global &cpus_queue
Quoted 5 lines · On Wed, 26 Feb 2025 08:16:52 +0100 Philippe Mathieu-Daudé <[email protected]>…
Quoted 4 lines · Igor Mammedov
cpu_list_add()was doing 2 distinct things:
- assign some index to vCPU
- add unrealized (thus in inconsistent state) vCPU to&cpus_queue
Quoted 7 lines
Quoted 5 lines
Code using
CPU_FOREACH()macro would iterate over possibly unrealized vCPUs, often dealt with special casing.Instead of working around of vCPU existence in
cpus_queue, split outcpu_indexassignment fromcpu_list_add(),Better split 2 distinct changes in 2 patches for clarity.
Will do it later, once folks decide how to fix broken TCG reset path.
do you mean:
#1 - introduce cpu_auto_assign_cpu_index()
#2 - move cpu_list_add() to later stage but keep cpu_auto_assign_cpu_index()
where it's now?
Quoted 100 lines
Quoted 99 lines
and move the later to the end of realize stage, right before vCPU is let run.
Signed-off-by Igor Mammedov <[email protected]>---
CC: Yanan Wang <[email protected]>
CC: Zhao Liu <[email protected]>--- include/hw/core/cpu.h | 6 ++++++ cpu-common.c | 23 ++++++++++++++--------- cpu-target.c | 2 +- hw/core/cpu-common.c | 2 ++ 4 files changed, 23 insertions(+), 10 deletions(-)include/hw/core/cpu.h +6
-0@@ -750,6 +750,12 @@ bool cpu_virtio_is_big_endian(CPUState *cpu);#endif /* CONFIG_USER_ONLY */ +/** +* cpu_auto_assign_cpu_index: +* @cpu: The CPU to be assigned a cpu_index +*/ +void cpu_auto_assign_cpu_index (CPUState *cpu); +/** * cpu_list_add: * @cpu: The CPU to be added to the list of CPUs. cpu-common.c +14
-9@@ -71,15 +71,7 @@ int cpu_get_free_index(void)return max_cpu_index; } -CPUTailQ cpus_queue =QTAILQ_HEAD_INITIALIZER (cpus_queue); -static unsigned int cpu_list_generation_id; - -unsigned int cpu_list_generation_id_get (void ) -{ -return cpu_list_generation_id; -} - -void cpu_list_add (CPUState *cpu) +void cpu_auto_assign_cpu_index (CPUState *cpu) {static bool cpu_index_auto_assigned;@@ -91,6 +83,19 @@ void cpu_list_add(CPUState *cpu) }else {assert (!cpu_index_auto_assigned); } +} + +CPUTailQ cpus_queue =QTAILQ_HEAD_INITIALIZER (cpus_queue); +static unsigned int cpu_list_generation_id; + +unsigned int cpu_list_generation_id_get (void ) +{ +return cpu_list_generation_id; +} + +void cpu_list_add (CPUState *cpu) +{ +QEMU_LOCK_GUARD (&qemu_cpu_list_lock);QTAILQ_INSERT_TAIL_RCU (&cpus_queue, cpu, node); cpu_list_generation_id++; }cpu-target.c +1
-1@@ -142,7 +142,7 @@ bool cpu_exec_realizefn(CPUState *cpu, Error **errp) }/* Wait until cpu initialization complete before exposing cpu. */ -cpu_list_add (cpu); +cpu_auto_assign_cpu_index (cpu);#ifdef CONFIG_USER_ONLY assert(qdev_get_vmsd(DEVICE(cpu)) == NULL ||hw/core/cpu-common.c +2
-0@@ -211,6 +211,8 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp) } } +cpu_list_add (cpu); +if (dev->hotplugged) {cpu_synchronize_post_init (cpu);cpu_resume (cpu);
Re: [PATCH v2 08/10] cpus: expose only realized vCPUs to global &cpus_queue
Quoted 17 lines · Igor Mammedov
Quoted 5 lines · On Wed, 26 Feb 2025 08:16:52 +0100 Philippe Mathieu-Daudé <[email protected]>…
Quoted 4 lines · Igor Mammedov
cpu_list_add()was doing 2 distinct things:
- assign some index to vCPU
- add unrealized (thus in inconsistent state) vCPU to&cpus_queueQuoted 7 lines
Quoted 5 lines
Code using
CPU_FOREACH()macro would iterate over possibly unrealized vCPUs, often dealt with special casing.Instead of working around of vCPU existence in
cpus_queue, split outcpu_indexassignment fromcpu_list_add(),Better split 2 distinct changes in 2 patches for clarity.
Quoted 6 lines
Will do it later, once folks decide how to fix broken TCG reset path.
do you mean:
#1 - introducecpu_auto_assign_cpu_index()
#2 - movecpu_list_add()to later stage but keepcpu_auto_assign_cpu_index()
where it's now?
Exactly, thanks!
PATCH v2 9/10 accel/kvm: Assert vCPU is created when calling kvm_dirty_ring_reap*()
In reply to Igor MammedovFrom: Philippe Mathieu-Daudé <[email protected]>
Previous commits made sure vCPUs are realized before accelerators (such KVM) use them. Ensure that by asserting the vCPU is created, no need to return.
For more context, see commit 56adee407fc ("kvm: dirty-ring: Fix race with vcpu creation").
---
CC: [email protected]
CC: [email protected]
--- accel/kvm/kvm-all.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
accel/kvm/kvm-all.c +3 -5
@@ -831,13 +831,11 @@ static uint32_t kvm_dirty_ring_reap_one(KVMState *s, CPUState *cpu)uint32_t count =0 , fetch = cpu->kvm_fetch_index;/* -* It's possible that we race with vcpu creation code where the vcpu is +* It's not possible that we race with vcpu creation code where the vcpu is * put onto the vcpus list but not yet initialized the dirty ring -* structures. If so, skip it. +* structures. */ -if (!cpu->created) { -return 0 ; - } +assert (cpu->created);assert (dirty_gfns && ring_size);trace_kvm_dirty_ring_reap_vcpu (cpu->cpu_index);
Signature · 1 line
2.43.0
PATCH v2 10/10 accel/kvm: Remove unreachable assertion in kvm_dirty_ring_reap*()
In reply to Igor MammedovFrom: Philippe Mathieu-Daudé <[email protected]>
Previous commit passed all our CI tests, this assertion being never triggered. Remove it as dead code.
---
CC: [email protected]
--- accel/kvm/kvm-all.c | 7 ------- 1 file changed, 7 deletions(-)
accel/kvm/kvm-all.c +0 -7
@@ -830,13 +830,6 @@ static uint32_t kvm_dirty_ring_reap_one(KVMState *s, CPUState *cpu)uint32_t ring_size = s->kvm_dirty_ring_size;uint32_t count =0 , fetch = cpu->kvm_fetch_index; -/* -* It's not possible that we race with vcpu creation code where the vcpu is -* put onto the vcpus list but not yet initialized the dirty ring -* structures. -*/ -assert (cpu->created); -assert (dirty_gfns && ring_size);trace_kvm_dirty_ring_reap_vcpu (cpu->cpu_index);
Signature · 1 line
2.43.0
Quoted 21 lines · On Fri, 7 Feb 2025 17:20:38 +0100 Igor Mammedov <[email protected]> wrote:
Changelog:
* drop wire/unwire hooks patches
* drop unrealize related patches
* include fixed up patches from
[PATCH 0/6] tcg: fix qemu crash when addassert_cpu_is_self()is enabled
and cleanups related tocpu->createdcheck
https://patchew.org/QEMU/[email protected]/
as it's related to the topic (well, modulo bsd cleanup)
* CI mostly green modulo rust failure on Ubuntuhttps://gitlab.com/imammedo/qemu/-/pipelines/1660855467The goal of this series is to expose vCPUs in a stable state to the accelerators, in particular the QDev 'REALIZED' step.
To do this we split out
cpu_indexassignment into a separate step, and move callcpu_list_add()to the end of CPU realize stage.I expect these changes to allow
CPUState::cpu_indexclarifications and simplifications, but this will be addressed (and commented) in a separate series.
Quoted 6 lines
As result, the series also
* fix regression intoroduced by
30933c4fb4f3d("tcg/cputlb: remove other-cpu capability from TLB flushing")
for deatials see 'tcg:tlb: usetcg_debug_assert()inassert_cpu_is_self()'
* drops no longer needed workaround 'cpu->check' due to vCPU being exposed too early incpus_queue.
Richard,
gentle ping,
can you pick it up is it looks reasonable.
Quoted 35 lines
CC Paolo Bonzini <[email protected]>CC Richard Henderson <[email protected]>CC "Philippe Mathieu-Daudé" <[email protected]>CC Alex Bennée <[email protected]>Igor Mammedov (7):
bsd-user: drop not longer usedtarget_reset_cpu()
loongarch: reset vcpu after it's created
m68k: reset vcpu after it's created
tcg:tlb: usetcg_debug_assert()inassert_cpu_is_self()Revert "tcg/cputlb: remove other-cpu capability from TLB flushing"
tcg: dropcpu->createdcheck
cpus: expose only realized vCPUs to global&cpus_queuePhilippe Mathieu-Daudé (3):
accel/tcg: Simplify use of&first_cpuinrr_cpu_thread_fn()accel/kvm: Assert vCPU is created when calling kvm_dirty_ring_reap*() accel/kvm: Remove unreachable assertion in kvm_dirty_ring_reap*()bsd-user/aarch64/target_arch_cpu.h | 5 --- bsd-user/arm/target_arch_cpu.h | 4 --- bsd-user/i386/target_arch_cpu.h | 5 --- bsd-user/riscv/target_arch_cpu.h | 4 --- bsd-user/x86_64/target_arch_cpu.h | 5 --- include/hw/core/cpu.h | 6 ++++ accel/kvm/kvm-all.c | 9 ------ accel/tcg/cputlb.c | 49 +++++++++++++++++++++--------- accel/tcg/tcg-accel-ops-rr.c | 13 +++++--- cpu-common.c | 23 ++++++++------ cpu-target.c | 2 +- hw/core/cpu-common.c | 2 ++ target/loongarch/cpu.c | 2 +- target/m68k/cpu.c | 2 +- 14 files changed, 68 insertions(+), 63 deletions(-)