Skip to contentSkip to conversation

qemu will core dump with "-smp 254, sockets=2, cores=3, threads=2"

PATCH In review lijun 4 participants 5 messages Dec 12, 2013 – Dec 16, 2013

lore
Share & tools
lore

lijun

author
#0 lore raw

PATCH 1/1 qemu will core dump with "-smp 254, sockets=2, cores=3, threads=2"

Hi all,

when set "-smp" more than 160, qemu will give the following warning:
Warning: Number of SMP cpus requested (161) exceeds the recommended cpus supported by KVM (160)
As the above warning, when set "-smp 160,sockets=2,cores=3,threads=2", but find that apic_id(hw/i386/acpi-build.c) is 259 not 159 and id(hw/acpi/piix4.c) is 259 not 159.

As the above warning, when set "-smp 254,sockets=2,cores=3,threads=2", but find that apic_id(hw/i386/acpi-build.c) is 513 not 253 and id(hw/acpi/piix4.c) is 513 not 253.

Based on above reasons, we have two methods to fix this issue.
1, Delete "assert(apic_id <= MAX_CPUMASK_BITS)" in file "hw/i386/acpi-build.c" and delete "g_assert((id / 8) < PIIX4_PROC_LEN)" in file "hw/acpi/piix4.c". 2, Detect the values of "sockets,cores,threads" when get them from command line. And modify smp_parse function in file vl.c to do some restrictions on these parameters when boot qemu.

I will submit the code patch later.

Best Regards,
Jun Li

author
#1 lore raw

Hi all,
As qemu core dump cause by "sockets=2,cores=3,threads=2", so add
this patch to check whether cores and threads is a power of 2.
The following is the realization of apicid_from_topo_ids function
in file target-i386/topology.h. It uses shift to get the values of pkg_id and core_id. nr_cores and nr_threads is related to this shift.
static inline apic_id_t apicid_from_topo_ids(unsigned nr_cores,
unsigned nr_threads,
unsigned pkg_id,
unsigned core_id,
unsigned smt_id)
{
return (pkg_id << apicid_pkg_offset(nr_cores, nr_threads)) |
(core_id << apicid_core_offset(nr_cores, nr_threads)) |
smt_id;
}
----
So should add a check for smp_cores and smp_threads in smp_parse function in file vl.c. Check whether smp_cores and smp_threads is a power of 2, so nr_cores and nr_threads is a power of 2. When return from apicid_from_topo_ids function, apic_id and id could get the correct values(apic_id is in file "hw/i386/acpi-build.c" and id is in file "hw/acpi/piix4.c") .

Without this check for smp_cores and smp_threads, specify "-smp 160,sockets=2,cores=3,threads=2", qemu will core dump too.

vl.c 2013-12-15 00:40:31.653800907 +0800 renamed from vl.c 2013-12-14 23:46:58.991076467 +0800 +19 -0
@@ -1384,6 +1384,19 @@
     },
 };
 
+/**
+ * This function will return whether @num is power of 2.
+ *
+ * Returns: 1 indicate @num is power of 2, 0 indicate @num is not.
+ */
+static int is_2_power(int num)
+{
+    if (num < 0 || num > 256)
+        return 1;
+
+    return !(num & (num - 1));
+}
+
 static void smp_parse(QemuOpts *opts)
 {
     if (opts) {
@@ -1418,6 +1431,12 @@
 
     }
 
+    /* check whether smp_cores and smp_threads is a power of 2 */
+    if (!is_2_power(smp_cores) || !is_2_power(smp_threads)) {
+        smp_cores = 1;
+        smp_threads = 1;
+    }
+
     if (max_cpus == 0) {
         max_cpus = smp_cpus;
     }

Best Regards,
Jun Li

#2 lore raw
Quoted 15 lines · lijun

Hi all,
As qemu core dump cause by "sockets=2,cores=3,threads=2", so add this
patch to check whether cores and threads is a power of 2.
The following is the realization of apicid_from_topo_ids function in
file target-i386/topology.h. It uses shift to get the values of pkg_id and core_id. nr_cores and nr_threads is related to this shift. static inline apic_id_t apicid_from_topo_ids(unsigned nr_cores,
unsigned nr_threads, unsigned pkg_id, unsigned core_id, unsigned smt_id)
{
return (pkg_id << apicid_pkg_offset(nr_cores, nr_threads)) |
(core_id << apicid_core_offset(nr_cores, nr_threads)) | smt_id;

Quoted 8 lines

}
----
So should add a check for smp_cores and smp_threads in smp_parse function in file vl.c. Check whether smp_cores and smp_threads is a power of 2, so nr_cores and nr_threads is a power of 2. When return from apicid_from_topo_ids function, apic_id and id could get the correct values(apic_id is in file "hw/i386/acpi-build.c" and id is in file "hw/acpi/piix4.c") .

vl.c is cross platform code so I'm not sure it's the right place to make this check. x86 ACPI may not be able to handle a non-power-of-2 number of cores in a socket, but it's a v alid ARM SMP configuration, for instance.

thanks
-- PMM

#3 lore raw
In reply to lijun
Quoted 6 lines · lijun

Hi all,
As qemu core dump cause by "sockets=2,cores=3,threads=2", so add
this patch to check whether cores and threads is a power of 2.

vl.c 2013-12-15 00:40:31.653800907 +0800 renamed from vl.c 2013-12-14 23:46:58.991076467 +0800 +2 -0
+/**
+ * This function will return whether @num is power of 2.

vl.c 2013-12-15 00:40:31.653800907 +0800 line 1398

Quoted 10 lines
vl.c 2013-12-15 00:40:31.653800907 +0800 renamed from vl.c 2013-12-14 23:46:58.991076467 +0800 +10 -0
+ *
+ * Returns: 1 indicate @num is power of 2, 0 indicate @num is not.
+ */
+static int is_2_power(int num)
+{
+    if (num < 0 || num > 256)
+        return 1;
+
+    return !(num & (num - 1));
+}

Please don't reinvent qemu-common.h's is_power_of_2. Furthermore, your function is more than just a power-of-2 check, it is also a range check.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#4 lore raw
In reply to lijun
Quoted 10 lines · lijun

Hi all,

when set "-smp" more than 160, qemu will give the following warning:
Warning: Number of SMP cpus requested (161) exceeds the recommended cpus supported by KVM (160)
As the above warning, when set "-smp 160,sockets=2,cores=3,threads=2", but find that apic_id(hw/i386/acpi-build.c) is 259 not 159 and id(hw/acpi/piix4.c) is 259 not 159.

Quoted 4 lines

As the above warning, when set "-smp 254,sockets=2,cores=3,threads=2", but find that apic_id(hw/i386/acpi-build.c) is 513 not 253 and id(hw/acpi/piix4.c) is 513 not 253.

"-smp 254,sockets=2,cores=3,threads=2" is invalid because you can't fit 254 VCPUs in 2 cores having 2*3 threads each. Setting both sockets and cores makes QEMU ignore the "threads" value and set it to smp/(cores*sockets) (42).

(42 is also an invalid value because two (42*3)-core can have only 252 VCPUs, but that's another bug.)

Anyway, your crash should be also reproducible if you simply run: "-smp 254,sockets=2,cores=3".

But in that case, the APIC ID is right because:

* With threads=42, we need 6 bits for thread ID
* With cores=3, we need 2 bits for core ID
* Bit offset of core ID is 6
* Bit offset of socket ID is 6+2 = 8
* CPU index #253 will thread #1 on core #0 on socket #2 (253 = 2*42*3 + 0*3 + 1)
* APIC ID for socket #2 core #0 thread #1 is:
(2<<8) | (0<<6) | 1 = 513

What we need to do to avoid this crash is to reject configurations where apic_id(max_cpus-1) or apic_id(smp_cpus-1) is too large. I believe this is what you mean on item 2 below (except that you will need to do that outside vl.c because the restriction is x86-specific)

Quoted 14 lines

Based on above reasons, we have two methods to fix this issue. 1, Delete "assert(apic_id <= MAX_CPUMASK_BITS)" in file "hw/i386/acpi-build.c" and delete "g_assert((id / 8) < PIIX4_PROC_LEN)" in file "hw/acpi/piix4.c".
2, Detect the values of "sockets,cores,threads" when get them from command line. And modify smp_parse function in file vl.c to do some restrictions on these parameters when boot qemu.

I will submit the code patch later.

Best Regards,
Jun Li

Signature · 1 line
Eduardo