Skip to contentSkip to conversation

qemu-system-i386: Could not install MSR_CORE_THREAD_COUNT handler: Success

DISCUSSION Vitaly Chikunov 2 participants 7 messages Dec 30, 2022 – Dec 31, 2022 KVM

lore
Share & tools
lore
#0 lore raw

Hi,

QEMU 7.2.0 when run on 32-bit x86 architecture fails with:

i586$ qemu-system-i386 -enable-kvm
qemu-system-i386: Could not install MSR_CORE_THREAD_COUNT handler: Success i586$ qemu-system-x86_64 -enable-kvm
qemu-system-x86_64: Could not install MSR_CORE_THREAD_COUNT handler: Success

Minimal reproducer is `qemu-system-i386 -enable-kvm'. And this only happens on x86 (linux32 personality and binaries on x86_64 host):

i586$ file /usr/bin/qemu-system-i386
/usr/bin/qemu-system-i386: ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, BuildID[sha1]=0ba1d953bcb7a691014255954f060ff404c8df90, for GNU/Linux 3.2.0, stripped i586$ /usr/bin/qemu-system-i386 --version
QEMU emulator version 7.2.0 (qemu-7.2.0-alt1)
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

Thanks,

#1 lore raw

Hi Vitaly,

This is a kvm kernel bug and should be fixed with the latest stable releases. Which kernel version are you running?

Thanks,

Alex

Quoted 22 lines

Am 30.12.2022 um 15:30 schrieb Vitaly Chikunov <[email protected]>:

Hi,

QEMU 7.2.0 when run on 32-bit x86 architecture fails with:

i586$ qemu-system-i386 -enable-kvm
qemu-system-i386: Could not install MSR_CORE_THREAD_COUNT handler: Success i586$ qemu-system-x86_64 -enable-kvm
qemu-system-x86_64: Could not install MSR_CORE_THREAD_COUNT handler: Success

Minimal reproducer is `qemu-system-i386 -enable-kvm'. And this only happens on x86 (linux32 personality and binaries on x86_64 host):

i586$ file /usr/bin/qemu-system-i386
/usr/bin/qemu-system-i386: ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, BuildID[sha1]=0ba1d953bcb7a691014255954f060ff404c8df90, for GNU/Linux 3.2.0, stripped i586$ /usr/bin/qemu-system-i386 --version
QEMU emulator version 7.2.0 (qemu-7.2.0-alt1)
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

Thanks,

#2 lore raw

Alexander,

Quoted 3 lines · Alexander Graf

Hi Vitaly,

This is a kvm kernel bug and should be fixed with the latest stable releases. Which kernel version are you running?

This is on latest v6.0 stable - 6.0.15.

Maybe there could be workaround for such situations? (Or maybe it's possible to make this error non-fatal?) We use qemu+kvm for testing and now we cannot test on x86.

Thanks,

Quoted 29 lines

Thanks,

Alex

Quoted 22 lines

Am 30.12.2022 um 15:30 schrieb Vitaly Chikunov <[email protected]>:

Hi,

QEMU 7.2.0 when run on 32-bit x86 architecture fails with:

i586$ qemu-system-i386 -enable-kvm
qemu-system-i386: Could not install MSR_CORE_THREAD_COUNT handler: Success i586$ qemu-system-x86_64 -enable-kvm
qemu-system-x86_64: Could not install MSR_CORE_THREAD_COUNT handler: Success

Minimal reproducer is `qemu-system-i386 -enable-kvm'. And this only happens on x86 (linux32 personality and binaries on x86_64 host):

i586$ file /usr/bin/qemu-system-i386
/usr/bin/qemu-system-i386: ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, BuildID[sha1]=0ba1d953bcb7a691014255954f060ff404c8df90, for GNU/Linux 3.2.0, stripped i586$ /usr/bin/qemu-system-i386 --version
QEMU emulator version 7.2.0 (qemu-7.2.0-alt1)
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

Thanks,

#3 lore raw

Hi Vitaly,

Quoted 11 lines · Vitaly Chikunov

Alexander,

Quoted 3 lines · Alexander Graf

Hi Vitaly,

This is a kvm kernel bug and should be fixed with the latest stable releases. Which kernel version are you running?

This is on latest v6.0 stable - 6.0.15.

Maybe there could be workaround for such situations? (Or maybe it's possible to make this error non-fatal?) We use qemu+kvm for testing and now we cannot test on x86.

I'm confused what's going wrong for you. I tried to reproduce the issue locally, but am unable to:

$ uname -a
Linux server 6.0.15-default #1 SMP PREEMPT_DYNAMIC Sat Dec 31 07:52:52 CET 2022 x86_64 x86_64 x86_64 GNU/Linux
$ linux32 chroot .
$ uname -a
Linux server 6.0.15-default #1 SMP PREEMPT_DYNAMIC Sat Dec 31 07:52:52 CET 2022 i686 GNU/Linux
$ cd qemu
$ file ./build/qemu-system-i386
./build/qemu-system-i386: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=f75e20572be5c604c121de4497397665c168aa4c, with debug_info, not stripped
$ ./build/qemu-system-i386 --version
QEMU emulator version 7.2.0 (v7.2.0-dirty)
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers
$ ./build/qemu-system-i386 -nographic -enable-kvm
SeaBIOS (version rel-1.16.1-0-g3208b098f51a-prebuilt.qemu.org)
[...]

Can you please double check whether your host kernel version is 6.0.15? Please paste the output of "uname -a".

Alex

#4 lore raw

Alexander,

Quoted 34 lines · Alexander Graf
Quoted 8 lines · Vitaly Chikunov
Quoted 2 lines · Alexander Graf

This is a kvm kernel bug and should be fixed with the latest stable releases. Which kernel version are you running?

This is on latest v6.0 stable - 6.0.15.

Maybe there could be workaround for such situations? (Or maybe it's possible to make this error non-fatal?) We use qemu+kvm for testing and now we cannot test on x86.

I'm confused what's going wrong for you. I tried to reproduce the issue locally, but am unable to:

$ uname -a
Linux server 6.0.15-default #1 SMP PREEMPT_DYNAMIC Sat Dec 31 07:52:52 CET
2022 x86_64 x86_64 x86_64 GNU/Linux
$ linux32 chroot .
$ uname -a
Linux server 6.0.15-default #1 SMP PREEMPT_DYNAMIC Sat Dec 31 07:52:52 CET
2022 i686 GNU/Linux
$ cd qemu
$ file ./build/qemu-system-i386
./build/qemu-system-i386: ELF 32-bit LSB shared object, Intel 80386, version
1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux
3.2.0, BuildID[sha1]=f75e20572be5c604c121de4497397665c168aa4c, with
debug_info, not stripped
$ ./build/qemu-system-i386 --version
QEMU emulator version 7.2.0 (v7.2.0-dirty)
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers
$ ./build/qemu-system-i386 -nographic -enable-kvm
SeaBIOS (version rel-1.16.1-0-g3208b098f51a-prebuilt.qemu.org)
[...]
Quoted 2 lines

Can you please double check whether your host kernel version is 6.0.15? Please paste the output of "uname -a".

Excuse me, I'm incorrectly reported kernel version I tried to boot instead of host one. Host kernels are quite old, 5.15.59 and even 5.17.15 -- where failure is occurring.

I just tested on 5.15.85 and there is no failure.

  builder@i586:/.in$ uname -a
  Linux localhost.localdomain 5.15.85-std-def-alt1 #1 SMP Wed Dec 21 21:14:40 UTC 2022 i686 GNU/Linux
  builder@i586:/.in$ qemu-system-i386 -nographic -enable-kvm
  SeaBIOS (version 1.16.1-alt1)

Perhaps, one of solutions it to reboot our build fleet to newer kernels. [This maybe hard, though, since special builder node image should be created and reboot shall be coordinated through all systems, in compare, updating QEMU would be easier since chroot is created on every build].

Thanks for your helpful replies!

Thanks,

Quoted 3 lines

Alex

#5 lore raw

Hi Vitaly,

Quoted 35 lines · Vitaly Chikunov

Alexander,

Quoted 32 lines · Alexander Graf
Quoted 7 lines · Vitaly Chikunov
Quoted 1 line · Alexander Graf

This is a kvm kernel bug and should be fixed with the latest stable releases. Which kernel version are you running?

This is on latest v6.0 stable - 6.0.15.

Maybe there could be workaround for such situations? (Or maybe it's possible to make this error non-fatal?) We use qemu+kvm for testing and now we cannot test on x86.

I'm confused what's going wrong for you. I tried to reproduce the issue locally, but am unable to:

$ uname -a
Linux server 6.0.15-default #1 SMP PREEMPT_DYNAMIC Sat Dec 31 07:52:52 CET
2022 x86_64 x86_64 x86_64 GNU/Linux
$ linux32 chroot .
$ uname -a
Linux server 6.0.15-default #1 SMP PREEMPT_DYNAMIC Sat Dec 31 07:52:52 CET
2022 i686 GNU/Linux
$ cd qemu
$ file ./build/qemu-system-i386
./build/qemu-system-i386: ELF 32-bit LSB shared object, Intel 80386, version
1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux
3.2.0, BuildID[sha1]=f75e20572be5c604c121de4497397665c168aa4c, with
debug_info, not stripped
$ ./build/qemu-system-i386 --version
QEMU emulator version 7.2.0 (v7.2.0-dirty)
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers
$ ./build/qemu-system-i386 -nographic -enable-kvm
SeaBIOS (version rel-1.16.1-0-g3208b098f51a-prebuilt.qemu.org)
[...]
Quoted 7 lines
Quoted 2 lines

Can you please double check whether your host kernel version is 6.0.15? Please paste the output of "uname -a".

Excuse me, I'm incorrectly reported kernel version I tried to boot instead of host one. Host kernels are quite old, 5.15.59 and even 5.17.15 -- where failure is occurring.

I just tested on 5.15.85 and there is no failure.

Awesome, great to hear :). That means everything works as expected at least.

Quoted 9 lines
   builder@i586:/.in$ uname -a
   Linux localhost.localdomain 5.15.85-std-def-alt1 #1 SMP Wed Dec 21 21:14:40 UTC 2022 i686 GNU/Linux
   builder@i586:/.in$ qemu-system-i386 -nographic -enable-kvm
   SeaBIOS (version 1.16.1-alt1)

Perhaps, one of solutions it to reboot our build fleet to newer kernels. [This maybe hard, though, since special builder node image should be created and reboot shall be coordinated through all systems, in compare, updating QEMU would be easier since chroot is created on every build].

I understand that it may be slightly painful to update your build fleet, but given this is a genuine kernel bug that has a fix available upstream and it only happens on niche corner cases (i386 QEMU on x86-64 Linux kernels with the bug) that I doubt anyone will use in production, I'd prefer we keep the QEMU logic as is :).

In the meanwhile, while you're patching the build fleet, you can apply the patch below as part of your build process to ensure you don't fail due to the kernel bug. Just make sure to remove it again as soon as you're done with the fleet update :).

target/i386/kvm/kvm.c +0 -0
index a213209379..b9396bc7a6 100644
@@ -2632,7 +2632,11 @@ int kvm_arch_init(MachineState *ms, KVMState *s)

                 return ret;
             }
     }
+#ifdef __x86_64__
     if (kvm_vm_check_extension(s, KVM_CAP_X86_USER_SPACE_MSR)) { +#else
+    if (0) {
+#endif
         bool r;

         ret = kvm_vm_enable_cap(s, KVM_CAP_X86_USER_SPACE_MSR, 0,

Alex

#6 lore raw

Alexander,

Quoted 63 lines · Alexander Graf
Quoted 33 lines · Vitaly Chikunov
Quoted 32 lines · Alexander Graf
Quoted 7 lines · Vitaly Chikunov
Quoted 1 line · Alexander Graf

This is a kvm kernel bug and should be fixed with the latest stable releases. Which kernel version are you running?

This is on latest v6.0 stable - 6.0.15.

Maybe there could be workaround for such situations? (Or maybe it's possible to make this error non-fatal?) We use qemu+kvm for testing and now we cannot test on x86.

I'm confused what's going wrong for you. I tried to reproduce the issue locally, but am unable to:

$ uname -a
Linux server 6.0.15-default #1 SMP PREEMPT_DYNAMIC Sat Dec 31 07:52:52 CET
2022 x86_64 x86_64 x86_64 GNU/Linux
$ linux32 chroot .
$ uname -a
Linux server 6.0.15-default #1 SMP PREEMPT_DYNAMIC Sat Dec 31 07:52:52 CET
2022 i686 GNU/Linux
$ cd qemu
$ file ./build/qemu-system-i386
./build/qemu-system-i386: ELF 32-bit LSB shared object, Intel 80386, version
1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux
3.2.0, BuildID[sha1]=f75e20572be5c604c121de4497397665c168aa4c, with
debug_info, not stripped
$ ./build/qemu-system-i386 --version
QEMU emulator version 7.2.0 (v7.2.0-dirty)
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers
$ ./build/qemu-system-i386 -nographic -enable-kvm
SeaBIOS (version rel-1.16.1-0-g3208b098f51a-prebuilt.qemu.org)
[...]
Quoted 7 lines
Quoted 2 lines

Can you please double check whether your host kernel version is 6.0.15? Please paste the output of "uname -a".

Excuse me, I'm incorrectly reported kernel version I tried to boot instead of host one. Host kernels are quite old, 5.15.59 and even 5.17.15 -- where failure is occurring.

I just tested on 5.15.85 and there is no failure.

Awesome, great to hear :). That means everything works as expected at least.

Quoted 9 lines
   builder@i586:/.in$ uname -a
   Linux localhost.localdomain 5.15.85-std-def-alt1 #1 SMP Wed Dec 21 21:14:40 UTC 2022 i686 GNU/Linux
   builder@i586:/.in$ qemu-system-i386 -nographic -enable-kvm
   SeaBIOS (version 1.16.1-alt1)

Perhaps, one of solutions it to reboot our build fleet to newer kernels. [This maybe hard, though, since special builder node image should be created and reboot shall be coordinated through all systems, in compare, updating QEMU would be easier since chroot is created on every build].

I understand that it may be slightly painful to update your build fleet, but given this is a genuine kernel bug that has a fix available upstream and it only happens on niche corner cases (i386 QEMU on x86-64 Linux kernels with the bug) that I doubt anyone will use in production, I'd prefer we keep the QEMU logic as is :).

Quoted 4 lines

In the meanwhile, while you're patching the build fleet, you can apply the patch below as part of your build process to ensure you don't fail due to the kernel bug. Just make sure to remove it again as soon as you're done with the fleet update :).

Thanks for the suggestions.
Best wishes for the New Year!

Vitaly,

Quoted 20 lines
target/i386/kvm/kvm.c +4 -0
index a213209379..b9396bc7a6 100644
@@ -2632,7 +2632,11 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
                  return ret;
              }
      }
+#ifdef __x86_64__
      if (kvm_vm_check_extension(s, KVM_CAP_X86_USER_SPACE_MSR)) {
+#else
+    if (0) {
+#endif
          bool r;
 
          ret = kvm_vm_enable_cap(s, KVM_CAP_X86_USER_SPACE_MSR, 0,

Alex