Skip to contentSkip to conversation

accel: Do not set CPUState::can_do_io in non-TCG accels

PATCH In review Philippe Mathieu-Daudé 3 participants 7 messages Nov 29, 2023 – Jan 17, 2024 KVM

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 accel: Do not set CPUState::can_do_io in non-TCG accels

'can_do_io' is specific to TCG. Having it set in non-TCG code is confusing, so remove it from QTest / HVF / KVM.

---
 accel/dummy-cpus.c        | 1 -
 accel/hvf/hvf-accel-ops.c | 1 -
 accel/kvm/kvm-accel-ops.c | 1 -
 3 files changed, 3 deletions(-)
accel/dummy-cpus.c +0 -1
index b75c919ac3..1005ec6f56 100644
@@ -27,7 +27,6 @@ static void *dummy_cpu_thread_fn(void *arg)
     qemu_mutex_lock_iothread();
     qemu_thread_get_self(cpu->thread);
     cpu->thread_id = qemu_get_thread_id();
-    cpu->neg.can_do_io = true;
     current_cpu = cpu;
 
 #ifndef _WIN32
accel/hvf/hvf-accel-ops.c +0 -1
index abe7adf7ee..2bba54cf70 100644
@@ -428,7 +428,6 @@ static void *hvf_cpu_thread_fn(void *arg)
     qemu_thread_get_self(cpu->thread);
 
     cpu->thread_id = qemu_get_thread_id();
-    cpu->neg.can_do_io = true;
     current_cpu = cpu;
 
     hvf_init_vcpu(cpu);
accel/kvm/kvm-accel-ops.c +0 -1
index 6195150a0b..f273f415db 100644
@@ -36,7 +36,6 @@ static void *kvm_vcpu_thread_fn(void *arg)
     qemu_mutex_lock_iothread();
     qemu_thread_get_self(cpu->thread);
     cpu->thread_id = qemu_get_thread_id();
-    cpu->neg.can_do_io = true;
     current_cpu = cpu;
 
     r = kvm_init_vcpu(cpu, &error_fatal);
Signature · 1 line
2.41.0
#1 lore raw

Hi Philippe,

took a quick look with

grep -R can_do_io

and this seems to be in include/hw/core/cpu.h as well as cpu-common.c,

maybe there is more meat to address to fully solve this?

Before we had stuff for reset in cpu-common.c under a if (tcg_enabled()) {
}

but now we have cpu_exec_reset_hold(),
should the implementation for tcg of cpu_exec_reset_hold() do that (and potentially other tcg-specific non-arch-specific cpu variables we might need)?

If can_do_io is TCG-specific, maybe the whole field existence / visibility can be conditioned on TCG actually being at least compiled-in? This might help find problems of the field being used in the wrong context, by virtue of getting an error when compiling with --disable-tcg for example.

Ciao,

Claudio

Quoted 46 lines · Philippe Mathieu-Daudé

'can_do_io' is specific to TCG. Having it set in non-TCG code is confusing, so remove it from QTest / HVF / KVM.

---
 accel/dummy-cpus.c        | 1 -
 accel/hvf/hvf-accel-ops.c | 1 -
 accel/kvm/kvm-accel-ops.c | 1 -
 3 files changed, 3 deletions(-)
accel/dummy-cpus.c +0 -1
index b75c919ac3..1005ec6f56 100644
@@ -27,7 +27,6 @@ static void *dummy_cpu_thread_fn(void *arg)
     qemu_mutex_lock_iothread();
     qemu_thread_get_self(cpu->thread);
     cpu->thread_id = qemu_get_thread_id();
-    cpu->neg.can_do_io = true;
     current_cpu = cpu;
 
 #ifndef _WIN32
accel/hvf/hvf-accel-ops.c +0 -1
index abe7adf7ee..2bba54cf70 100644
@@ -428,7 +428,6 @@ static void *hvf_cpu_thread_fn(void *arg)
     qemu_thread_get_self(cpu->thread);
 
     cpu->thread_id = qemu_get_thread_id();
-    cpu->neg.can_do_io = true;
     current_cpu = cpu;
 
     hvf_init_vcpu(cpu);
accel/kvm/kvm-accel-ops.c +0 -1
index 6195150a0b..f273f415db 100644
@@ -36,7 +36,6 @@ static void *kvm_vcpu_thread_fn(void *arg)
     qemu_mutex_lock_iothread();
     qemu_thread_get_self(cpu->thread);
     cpu->thread_id = qemu_get_thread_id();
-    cpu->neg.can_do_io = true;
     current_cpu = cpu;
 
     r = kvm_init_vcpu(cpu, &error_fatal);
#2 lore raw

Hi Claudio,

Quoted 8 lines · Claudio Fontana

Hi Philippe,

took a quick look with

grep -R can_do_io

and this seems to be in include/hw/core/cpu.h as well as cpu-common.c,

Quoted 8 lines

maybe there is more meat to address to fully solve this?

Before we had stuff for reset in cpu-common.c under a if (tcg_enabled()) {
}

but now we have cpu_exec_reset_hold(),
should the implementation for tcg of cpu_exec_reset_hold() do that (and potentially other tcg-specific non-arch-specific cpu variables we might need)?

Later we eventually get there:

accel/tcg/tcg-accel-ops.c +3 -0
index 9b038b1af5..e2c5cf97dc 100644
@@ -89,6 +89,9 @@ static void tcg_cpu_reset_hold(CPUState *cpu)
 
     cpu->accel->icount_extra = 0;
     cpu->accel->mem_io_pc = 0;
+
+    qatomic_set(&cpu->neg.icount_decr.u32, 0);
+    cpu->neg.can_do_io = true;
 }
 

My branch is huge, I'm trying to split it, maybe I shouldn't have sent this single non-TCG patch out of it. I'll Cc you.

Quoted 6 lines

If can_do_io is TCG-specific, maybe the whole field existence / visibility can be conditioned on TCG actually being at least compiled-in? This might help find problems of the field being used in the wrong context, by virtue of getting an error when compiling with --disable-tcg for example.

Ciao,

Claudio

#3 lore raw
Quoted 35 lines · Philippe Mathieu-Daudé

Hi Claudio,

Quoted 8 lines · Claudio Fontana

Hi Philippe,

took a quick look with

grep -R can_do_io

and this seems to be in include/hw/core/cpu.h as well as cpu-common.c,

Quoted 8 lines

maybe there is more meat to address to fully solve this?

Before we had stuff for reset in cpu-common.c under a if (tcg_enabled()) {
}

but now we have cpu_exec_reset_hold(),
should the implementation for tcg of cpu_exec_reset_hold() do that (and potentially other tcg-specific non-arch-specific cpu variables we might need)?

Later we eventually get there:

accel/tcg/tcg-accel-ops.c +3 -0
index 9b038b1af5..e2c5cf97dc 100644
@@ -89,6 +89,9 @@ static void tcg_cpu_reset_hold(CPUState *cpu)
 
      cpu->accel->icount_extra = 0;
      cpu->accel->mem_io_pc = 0;
+
+    qatomic_set(&cpu->neg.icount_decr.u32, 0);
+    cpu->neg.can_do_io = true;
  }
 
Quoted 2 lines

My branch is huge, I'm trying to split it, maybe I shouldn't have sent this single non-TCG patch out of it. I'll Cc you.

Thanks and congrats for the rework effort there!

Ciao,

Claudio

Quoted 8 lines
Quoted 7 lines

If can_do_io is TCG-specific, maybe the whole field existence / visibility can be conditioned on TCG actually being at least compiled-in? This might help find problems of the field being used in the wrong context, by virtue of getting an error when compiling with --disable-tcg for example.

Ciao,

Claudio

Reviewed-by on v1 1/1
#4 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 15 lines · Philippe Mathieu-Daudé

'can_do_io' is specific to TCG. Having it set in non-TCG code is confusing, so remove it from QTest / HVF / KVM.

---
  accel/dummy-cpus.c        | 1 -
  accel/hvf/hvf-accel-ops.c | 1 -
  accel/kvm/kvm-accel-ops.c | 1 -
  3 files changed, 3 deletions(-)
accel/dummy-cpus.c +0 -0
index b75c919ac3..1005ec6f56 100644

accel/dummy-cpus.c static void *dummy_cpu_thread_fn(void *arg) · line 30

Quoted 5 lines
accel/dummy-cpus.c +0 -1
@@ -27,7 +27,6 @@ static void *dummy_cpu_thread_fn(void *arg)
      qemu_mutex_lock_iothread();
      qemu_thread_get_self(cpu->thread);
      cpu->thread_id = qemu_get_thread_id();
-    cpu->neg.can_do_io = true;
      current_cpu = cpu;

I expect this is ok...

When I was moving this variable around just recently, 464dacf6, I wondered about these other settings, and I wondered if they used to be required for implementing some i/o on behalf of hw accel. Something that has now been factored out to e.g. kvm_handle_io, which now uses address_space_rw directly.

I see 3 reads of can_do_io: accel/tcg/{cputlb.c, icount-common.c} and system/watchpoint.c.
The final one is nested within replay_running_debug(), which implies icount and tcg.

r~

Quoted 26 lines
accel/dummy-cpus.c +0 -0
  #ifndef _WIN32
accel/hvf/hvf-accel-ops.c +0 -1
index abe7adf7ee..2bba54cf70 100644
@@ -428,7 +428,6 @@ static void *hvf_cpu_thread_fn(void *arg)
      qemu_thread_get_self(cpu->thread);
 
      cpu->thread_id = qemu_get_thread_id();
-    cpu->neg.can_do_io = true;
      current_cpu = cpu;
 
      hvf_init_vcpu(cpu);
accel/kvm/kvm-accel-ops.c +0 -1
index 6195150a0b..f273f415db 100644
@@ -36,7 +36,6 @@ static void *kvm_vcpu_thread_fn(void *arg)
      qemu_mutex_lock_iothread();
      qemu_thread_get_self(cpu->thread);
      cpu->thread_id = qemu_get_thread_id();
-    cpu->neg.can_do_io = true;
      current_cpu = cpu;
 
      r = kvm_init_vcpu(cpu, &error_fatal);
#5 lore raw
Quoted 24 lines · Richard Henderson
Quoted 15 lines · Philippe Mathieu-Daudé

'can_do_io' is specific to TCG. Having it set in non-TCG code is confusing, so remove it from QTest / HVF / KVM.

---
  accel/dummy-cpus.c        | 1 -
  accel/hvf/hvf-accel-ops.c | 1 -
  accel/kvm/kvm-accel-ops.c | 1 -

  3 files changed, 3 deletions(-)
accel/dummy-cpus.c +0 -0
index b75c919ac3..1005ec6f56 100644

accel/dummy-cpus.c static void *dummy_cpu_thread_fn(void *arg) · line 30

Quoted 5 lines
accel/dummy-cpus.c +0 -1
@@ -27,7 +27,6 @@ static void *dummy_cpu_thread_fn(void *arg)
       qemu_mutex_lock_iothread();
       qemu_thread_get_self(cpu->thread);
       cpu->thread_id = qemu_get_thread_id();
-    cpu->neg.can_do_io = true;
       current_cpu = cpu;

I expect this is ok...

Quoted 1 line

When I was moving this variable around just recently, 464dacf6, I wondered about these other settings, and I wondered if they used to be required for implementing some i/o on behalf of hw accel.  Something that has now been factored out to e.g. kvm_handle_io, which now uses address_space_rw directly.

It was added by mistake in commit 99df7dce8a ("cpu: Move can_do_io field from CPU_COMMON to CPUState", 2013) to cpu_common_reset() and commit 626cf8f4c6 ("icount: set can_do_io outside TB execution", 2014), then moved in commits 57038a92bb ("cpus: extract out kvm-specific code to accel/kvm"), b86f59c715 ("accel: replace struct CpusAccel with AccelOpsClass") and the one you mentioned, 464dacf609 ("accel/tcg: Move can_do_io to CPUNegativeOffsetState").

The culprit is 626cf8f4c6 I guess, so maybe:

Fixes 626cf8f4c6 ("icount: set can_do_io outside TB execution")
Quoted 3 lines

I see 3 reads of can_do_io: accel/tcg/{cputlb.c, icount-common.c} and system/watchpoint.c.  The final one is nested within replay_running_debug(), which implies icount and tcg.

Thanks!