Skip to contentSkip to conversation

Reinstate ability to use Qemu on pre-SSE4.1 x86 hosts

PATCH In review Alexander Monakov 4 participants 30 messages Jun 12, 2024 – Jun 23, 2024 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

cover Reinstate ability to use Qemu on pre-SSE4.1 x86 hosts

Hello,

I'm sending straightforward reverts to recent patches that bumped minimum required x86 instruction set to SSE4.2. The older chips did not stop working, and people still test and use new software on older hardware:
https://sourceware.org/bugzilla/show_bug.cgi?id=31867

Considering the very minor gains from the baseline raise, I'm honestly not sure why it happened. It seems better to let distributions handle that.

Alexander Monakov (5):
Revert "host/i386: assume presence of POPCNT"
Revert "host/i386: assume presence of SSSE3"
Revert "host/i386: assume presence of SSE2"
Revert "host/i386: assume presence of CMOV"
Revert "meson: assume x86-64-v2 baseline ISA"

 host/include/i386/host/cpuinfo.h |  3 +++
 meson.build                      | 10 +++-------
 tcg/i386/tcg-target.c.inc        | 15 ++++++++++++++-
 tcg/i386/tcg-target.h            |  5 +++--
 util/bufferiszero.c              |  4 ++--
 util/cpuinfo-i386.c              |  7 +++++--
 6 files changed, 30 insertions(+), 14 deletions(-)
Signature · 1 line
2.32.0
#1 lore raw

PATCH 1/5 Revert "host/i386: assume presence of POPCNT"

This reverts commit 45ccdbcb24baf99667997fac5cf60318e5e7db51.

Revert in preparation to rolling back x86_64-v2 ISA requirement.

---
 host/include/i386/host/cpuinfo.h | 1 +
 tcg/i386/tcg-target.h            | 5 +++--
 util/cpuinfo-i386.c              | 1 +
 3 files changed, 5 insertions(+), 2 deletions(-)
host/include/i386/host/cpuinfo.h +1 -0
index c1e94d75..72f6fad6 100644
@@ -11,6 +11,7 @@
 #define CPUINFO_ALWAYS          (1u << 0)  /* so cpuinfo is nonzero */
 #define CPUINFO_MOVBE           (1u << 2)
 #define CPUINFO_LZCNT           (1u << 3)
+#define CPUINFO_POPCNT          (1u << 4)
 #define CPUINFO_BMI1            (1u << 5)
 #define CPUINFO_BMI2            (1u << 6)
 #define CPUINFO_AVX1            (1u << 9)
tcg/i386/tcg-target.h +3 -2
index ecc69827..2f67a97e 100644
@@ -111,6 +111,7 @@ typedef enum {
 #endif
 
 #define have_bmi1         (cpuinfo & CPUINFO_BMI1)
+#define have_popcnt       (cpuinfo & CPUINFO_POPCNT)
 #define have_avx1         (cpuinfo & CPUINFO_AVX1)
 #define have_avx2         (cpuinfo & CPUINFO_AVX2)
 #define have_movbe        (cpuinfo & CPUINFO_MOVBE)
@@ -142,7 +143,7 @@ typedef enum {
 #define TCG_TARGET_HAS_nor_i32          0
 #define TCG_TARGET_HAS_clz_i32          1
 #define TCG_TARGET_HAS_ctz_i32          1
-#define TCG_TARGET_HAS_ctpop_i32        1
+#define TCG_TARGET_HAS_ctpop_i32        have_popcnt
 #define TCG_TARGET_HAS_deposit_i32      1
 #define TCG_TARGET_HAS_extract_i32      1
 #define TCG_TARGET_HAS_sextract_i32     1
@@ -177,7 +178,7 @@ typedef enum {
 #define TCG_TARGET_HAS_nor_i64          0
 #define TCG_TARGET_HAS_clz_i64          1
 #define TCG_TARGET_HAS_ctz_i64          1
-#define TCG_TARGET_HAS_ctpop_i64        1
+#define TCG_TARGET_HAS_ctpop_i64        have_popcnt
 #define TCG_TARGET_HAS_deposit_i64      1
 #define TCG_TARGET_HAS_extract_i64      1
 #define TCG_TARGET_HAS_sextract_i64     0
util/cpuinfo-i386.c +1 -0
index 8f2694d8..6d474a62 100644
@@ -35,6 +35,7 @@ unsigned __attribute__((constructor)) cpuinfo_init(void)
         __cpuid(1, a, b, c, d);
 
         info |= (c & bit_MOVBE ? CPUINFO_MOVBE : 0);
+        info |= (c & bit_POPCNT ? CPUINFO_POPCNT : 0);
         info |= (c & bit_PCLMUL ? CPUINFO_PCLMUL : 0);
 
         /* NOTE: our AES support requires SSSE3 (PSHUFB) as well. */
Signature · 1 line
2.32.0
#2 lore raw

PATCH 2/5 Revert "host/i386: assume presence of SSSE3"

In reply to Alexander Monakov

This reverts commit 433cd6d94a8256af70a5200f236dc8047c3c1468.

Revert in preparation to rolling back x86_64-v2 ISA requirement.

---
 util/cpuinfo-i386.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
util/cpuinfo-i386.c +2 -2
index 6d474a62..ca74ef04 100644
@@ -38,8 +38,8 @@ unsigned __attribute__((constructor)) cpuinfo_init(void)
         info |= (c & bit_POPCNT ? CPUINFO_POPCNT : 0);
         info |= (c & bit_PCLMUL ? CPUINFO_PCLMUL : 0);
 
-        /* NOTE: our AES support requires SSSE3 (PSHUFB) as well. */
-        info |= (c & bit_AES) ? CPUINFO_AES : 0;
+        /* Our AES support requires PSHUFB as well. */
+        info |= ((c & bit_AES) && (c & bit_SSSE3) ? CPUINFO_AES : 0);
 
         /* For AVX features, we must check available and usable. */
         if ((c & bit_AVX) && (c & bit_OSXSAVE)) {
Signature · 1 line
2.32.0
#3 lore raw

PATCH 3/5 Revert "host/i386: assume presence of SSE2"

In reply to Alexander Monakov

This reverts commit b18236897ca15c3db1506d8edb9a191dfe51429c.

Revert in preparation to rolling back x86_64-v2 ISA requirement.

---
 host/include/i386/host/cpuinfo.h | 1 +
 util/bufferiszero.c              | 4 ++--
 util/cpuinfo-i386.c              | 1 +
 3 files changed, 4 insertions(+), 2 deletions(-)
host/include/i386/host/cpuinfo.h +1 -0
index 72f6fad6..81771733 100644
@@ -14,6 +14,7 @@
 #define CPUINFO_POPCNT          (1u << 4)
 #define CPUINFO_BMI1            (1u << 5)
 #define CPUINFO_BMI2            (1u << 6)
+#define CPUINFO_SSE2            (1u << 7)
 #define CPUINFO_AVX1            (1u << 9)
 #define CPUINFO_AVX2            (1u << 10)
 #define CPUINFO_AVX512F         (1u << 11)
util/bufferiszero.c +2 -2
index 11c080e0..74864f7b 100644
@@ -188,14 +188,14 @@ static biz_accel_fn const accel_table[] = {
 
 static unsigned best_accel(void)
 {
-#ifdef CONFIG_AVX2_OPT
     unsigned info = cpuinfo_init();
 
+#ifdef CONFIG_AVX2_OPT
     if (info & CPUINFO_AVX2) {
         return 2;
     }
 #endif
-    return 1;
+    return info & CPUINFO_SSE2 ? 1 : 0;
 }
 
 #elif defined(__aarch64__) && defined(__ARM_NEON)
util/cpuinfo-i386.c +1 -0
index ca74ef04..90f92a42 100644
@@ -34,6 +34,7 @@ unsigned __attribute__((constructor)) cpuinfo_init(void)
     if (max >= 1) {
         __cpuid(1, a, b, c, d);
 
+        info |= (d & bit_SSE2 ? CPUINFO_SSE2 : 0);
         info |= (c & bit_MOVBE ? CPUINFO_MOVBE : 0);
         info |= (c & bit_POPCNT ? CPUINFO_POPCNT : 0);
         info |= (c & bit_PCLMUL ? CPUINFO_PCLMUL : 0);
Signature · 1 line
2.32.0
#4 lore raw

PATCH 4/5 Revert "host/i386: assume presence of CMOV"

In reply to Alexander Monakov

This reverts commit e68e97ce55b3d17af22dd62c3b3dc72f761b0862.

Revert in preparation to rolling back x86_64-v2 ISA requirement.

---
 host/include/i386/host/cpuinfo.h |  1 +
 tcg/i386/tcg-target.c.inc        | 15 ++++++++++++++-
 util/cpuinfo-i386.c              |  1 +
 3 files changed, 16 insertions(+), 1 deletion(-)
host/include/i386/host/cpuinfo.h +1 -0
index 81771733..9386c749 100644
@@ -9,6 +9,7 @@
 /* Digested version of <cpuid.h> */
 
 #define CPUINFO_ALWAYS          (1u << 0)  /* so cpuinfo is nonzero */
+#define CPUINFO_CMOV            (1u << 1)
 #define CPUINFO_MOVBE           (1u << 2)
 #define CPUINFO_LZCNT           (1u << 3)
 #define CPUINFO_POPCNT          (1u << 4)
tcg/i386/tcg-target.c.inc +14 -1
index 9a54ef7f..59235b4f 100644
@@ -157,6 +157,12 @@ static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot)
 #define SOFTMMU_RESERVE_REGS \
     (tcg_use_softmmu ? (1 << TCG_REG_L0) | (1 << TCG_REG_L1) : 0)
 
+/* For 64-bit, we always know that CMOV is available.  */
+#if TCG_TARGET_REG_BITS == 64
+# define have_cmov      true
+#else
+# define have_cmov      (cpuinfo & CPUINFO_CMOV)
+#endif
 #define have_bmi2       (cpuinfo & CPUINFO_BMI2)
 #define have_lzcnt      (cpuinfo & CPUINFO_LZCNT)
 
@@ -1809,7 +1815,14 @@ static void tcg_out_setcond2(TCGContext *s, const TCGArg *args,
 static void tcg_out_cmov(TCGContext *s, int jcc, int rexw,
                          TCGReg dest, TCGReg v1)
 {
-    tcg_out_modrm(s, OPC_CMOVCC | jcc | rexw, dest, v1);
+    if (have_cmov) {
+        tcg_out_modrm(s, OPC_CMOVCC | jcc | rexw, dest, v1);
+    } else {
+        TCGLabel *over = gen_new_label();
+        tcg_out_jxx(s, jcc ^ 1, over, 1);
+        tcg_out_mov(s, TCG_TYPE_I32, dest, v1);
+        tcg_out_label(s, over);
+    }
 }
 
 static void tcg_out_movcond(TCGContext *s, int rexw, TCGCond cond,
util/cpuinfo-i386.c +1 -0
index 90f92a42..18ab747a 100644
@@ -34,6 +34,7 @@ unsigned __attribute__((constructor)) cpuinfo_init(void)
     if (max >= 1) {
         __cpuid(1, a, b, c, d);
 
+        info |= (d & bit_CMOV ? CPUINFO_CMOV : 0);
         info |= (d & bit_SSE2 ? CPUINFO_SSE2 : 0);
         info |= (c & bit_MOVBE ? CPUINFO_MOVBE : 0);
         info |= (c & bit_POPCNT ? CPUINFO_POPCNT : 0);
Signature · 1 line
2.32.0
#5 lore raw

PATCH 5/5 Revert "meson: assume x86-64-v2 baseline ISA"

In reply to Alexander Monakov

This reverts commit 294ac64e459aca023f43441651d860980c9784f1.

Reinstate the ability to use Qemu on x86 hosts that do not meet x86_64-v2 ISA baseline.

---
 meson.build | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)
meson.build +3 -7
index ec59effc..49962cce 100644
@@ -336,13 +336,9 @@ if host_arch == 'i386' and not cc.links('''
   qemu_common_flags = ['-march=i486'] + qemu_common_flags
 endif
 
-# Assume x86-64-v2 (minus CMPXCHG16B for 32-bit code)
-if host_arch == 'i386'
-  qemu_common_flags = ['-mfpmath=sse'] + qemu_common_flags
-endif
-if host_arch in ['i386', 'x86_64']
-  qemu_common_flags = ['-mpopcnt', '-msse4.2'] + qemu_common_flags
-endif
+# ??? Only extremely old AMD cpus do not have cmpxchg16b.
+# If we truly care, we should simply detect this case at
+# runtime and generate the fallback to serial emulation.
 if host_arch == 'x86_64'
   qemu_common_flags = ['-mcx16'] + qemu_common_flags
 endif
Signature · 1 line
2.32.0
#6 lore raw
In reply to Alexander Monakov
Quoted 9 lines · Alexander Monakov

Hello,

I'm sending straightforward reverts to recent patches that bumped minimum required x86 instruction set to SSE4.2. The older chips did not stop working, and people still test and use new software on older hardware:
https://sourceware.org/bugzilla/show_bug.cgi?id=31867

Considering the very minor gains from the baseline raise, I'm honestly not sure why it happened. It seems better to let distributions handle that.

Indeed distros are opinionated about the x86_64 baseline they want to target.

While RHEL-9 switched to a x86_64-v2 baseline, Fedora has repeatedly rejected the idea of moving to an x86_64-v2 baseline, wanting to retain full backwards compat. So this assumption in QEMU is preventing the distros from satisfying their chosen build target goals.

Quoted 7 lines

Alexander Monakov (5):
Revert "host/i386: assume presence of POPCNT"
Revert "host/i386: assume presence of SSSE3"
Revert "host/i386: assume presence of SSE2"
Revert "host/i386: assume presence of CMOV"
Revert "meson: assume x86-64-v2 baseline ISA"

Quoted 12 lines
 host/include/i386/host/cpuinfo.h |  3 +++
 meson.build                      | 10 +++-------
 tcg/i386/tcg-target.c.inc        | 15 ++++++++++++++-
 tcg/i386/tcg-target.h            |  5 +++--
 util/bufferiszero.c              |  4 ++--
 util/cpuinfo-i386.c              |  7 +++++--
 6 files changed, 30 insertions(+), 14 deletions(-)
Signature · 1 line
2.32.0

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#7 lore raw
Quoted 12 lines · Daniel P. Berrangé
Quoted 9 lines · Alexander Monakov

Hello,

I'm sending straightforward reverts to recent patches that bumped minimum required x86 instruction set to SSE4.2. The older chips did not stop working, and people still test and use new software on older hardware:
https://sourceware.org/bugzilla/show_bug.cgi?id=31867

Considering the very minor gains from the baseline raise, I'm honestly not sure why it happened. It seems better to let distributions handle that.

Quoted 7 lines

Indeed distros are opinionated about the x86_64 baseline they want to target.

While RHEL-9 switched to a x86_64-v2 baseline, Fedora has repeatedly rejected the idea of moving to an x86_64-v2 baseline, wanting to retain full backwards compat. So this assumption in QEMU is preventing the distros from satisfying their chosen build target goals.

I didn't do this because of RHEL9, I did it because it's silly that QEMU cannot use POPCNT and has to waste 2% of the L1 d-cache to compute the x86 parity flag (and POPCNT was introduced at the same time as SSE4.2).

Intel x86_64-v2 processors have been around for about 15 years, AMD for a little less (2011). I'd rather hear from users about the usecases for running QEMU on such old processors before reverting, as this does not get in the way of booting/installing distros on old machines. Unless QEMU is run from within the installation media, which it isn't, requiring a particular processor family does not prevent Fedora from being installable on pre-v2 processors.

Paolo

#8 lore raw
Quoted 4 lines · Paolo Bonzini

I didn't do this because of RHEL9, I did it because it's silly that QEMU cannot use POPCNT and has to waste 2% of the L1 d-cache to compute the x86 parity flag (and POPCNT was introduced at the same time as SSE4.2).

From looking at that POPCNT patch I understood that Qemu detects presence of POPCNT at runtime and will only use the fallback when POPCNT is unavailable. Did I misunderstand?

Alexander

#9 lore raw
Quoted 9 lines · Alexander Monakov
Quoted 4 lines · Paolo Bonzini

I didn't do this because of RHEL9, I did it because it's silly that QEMU cannot use POPCNT and has to waste 2% of the L1 d-cache to compute the x86 parity flag (and POPCNT was introduced at the same time as SSE4.2).

From looking at that POPCNT patch I understood that Qemu detects presence of POPCNT at runtime and will only use the fallback when POPCNT is unavailable. Did I misunderstand?

-mpopcnt allows GCC to generate the POPCNT instruction for helper code. Right now we have code like this in target/i386/tcg/cc_helper_template.h:

    pf = parity_table[(uint8_t)dst];

and it could be instead something like

#if defined __i386__ || defined __x86_64__ || defined __s390x__|| defined __riscv_zbb
static inline unsigned int compute_pf(uint8_t x)

{
    return __builtin_parity(x) * CC_P;
}
#else
extern const uint8_t parity_table[256];
static inline unsigned int compute_pf(uint8_t x)
{
    return parity_table[x];
}

#endif

The code generated for __builtin_parity, if you don't have it available in hardware, is pretty bad.

Paolo

#10 lore raw
Quoted 15 lines · Paolo Bonzini
Quoted 9 lines · Alexander Monakov
Quoted 4 lines · Paolo Bonzini

I didn't do this because of RHEL9, I did it because it's silly that QEMU cannot use POPCNT and has to waste 2% of the L1 d-cache to compute the x86 parity flag (and POPCNT was introduced at the same time as SSE4.2).

From looking at that POPCNT patch I understood that Qemu detects presence of POPCNT at runtime and will only use the fallback when POPCNT is unavailable. Did I misunderstand?

-mpopcnt allows GCC to generate the POPCNT instruction for helper code. Right now we have code like this in target/i386/tcg/cc_helper_template.h:

Quoted 6 lines
    pf = parity_table[(uint8_t)dst];

and it could be instead something like

#if defined __i386__ || defined __x86_64__ || defined __s390x__|| defined __riscv_zbb

GCC also predefines __POPCNT__ when -mpopcnt is active, so that would be available for ifdef testing like above, but...

Quoted 10 lines

static inline unsigned int compute_pf(uint8_t x)

{
    return __builtin_parity(x) * CC_P;
}
#else
extern const uint8_t parity_table[256];
static inline unsigned int compute_pf(uint8_t x)
{
    return parity_table[x];
}
Quoted 4 lines

#endif

The code generated for __builtin_parity, if you don't have it available in hardware, is pretty bad.

On x86 parity _is_ available in baseline ISA, no? Here's what gcc-14 generates:

        xor     eax, eax
        test    dil, dil
        setnp   al
        sal     eax, 2

and with -mpopcnt:

        movsx   eax, dil
        popcnt  eax, eax
        and     eax, 1
        sal     eax, 2

Alexander

#11 lore raw
Quoted 44 lines · Alexander Monakov
Quoted 15 lines · Paolo Bonzini
Quoted 9 lines · Alexander Monakov
Quoted 4 lines · Paolo Bonzini

I didn't do this because of RHEL9, I did it because it's silly that QEMU cannot use POPCNT and has to waste 2% of the L1 d-cache to compute the x86 parity flag (and POPCNT was introduced at the same time as SSE4.2).

From looking at that POPCNT patch I understood that Qemu detects presence of POPCNT at runtime and will only use the fallback when POPCNT is unavailable. Did I misunderstand?

-mpopcnt allows GCC to generate the POPCNT instruction for helper code. Right now we have code like this in target/i386/tcg/cc_helper_template.h:

Quoted 6 lines
    pf = parity_table[(uint8_t)dst];

and it could be instead something like

#if defined __i386__ || defined __x86_64__ || defined __s390x__|| defined __riscv_zbb

GCC also predefines __POPCNT__ when -mpopcnt is active, so that would be available for ifdef testing like above, but...

Quoted 10 lines

static inline unsigned int compute_pf(uint8_t x)

{
    return __builtin_parity(x) * CC_P;
}
#else
extern const uint8_t parity_table[256];
static inline unsigned int compute_pf(uint8_t x)
{
    return parity_table[x];
}
Quoted 4 lines

#endif

The code generated for __builtin_parity, if you don't have it available in hardware, is pretty bad.

Quoted 6 lines

On x86 parity _is_ available in baseline ISA, no? Here's what gcc-14 generates:

        xor     eax, eax
        test    dil, dil
        setnp   al
        sal     eax, 2

Ahah, nice. :) I'm pretty sure that, when I tested "pf = (__builtin_popcount(x) & 1) * 4;", it was generating a call to __builtin_popcountsi2.

Still - for something that has a code generator, there _is_ a cost in supporting old CPUs, so I'd rather avoid reverting this. The glibc bug that you linked is very different not just because it affected 32-bit installation media, but also because it was a bug rather than intentional.

Since you are reporting this issue, how did you find out / what broke for you?

Paolo

#12 lore raw
Quoted 3 lines · Paolo Bonzini

Ahah, nice. :) I'm pretty sure that, when I tested "pf = (__builtin_popcount(x) & 1) * 4;", it was generating a call to __builtin_popcountsi2.

Why write '__builtin_popcount(x) & 1' when you can write '__builtin_parity(x)' in the first place?

Quoted 7 lines

Still - for something that has a code generator, there _is_ a cost in supporting old CPUs, so I'd rather avoid reverting this. The glibc bug that you linked is very different not just because it affected 32-bit installation media, but also because it was a bug rather than intentional.

Since you are reporting this issue, how did you find out / what broke for you?

I found out from the mailing list. My Core2-based desktop would be affected.

Last but not the least, I'm sympathetic to the efforts of my distro maintainers, who I imagine would be put in an uncomfortable position by this change.

Alexander

#13 lore raw
Quoted 10 lines · Alexander Monakov
Quoted 3 lines · Paolo Bonzini

Ahah, nice. :) I'm pretty sure that, when I tested "pf = (__builtin_popcount(x) & 1) * 4;", it was generating a call to __builtin_popcountsi2.

Why write '__builtin_popcount(x) & 1' when you can write '__builtin_parity(x)' in the first place?

I don't remember. :) Anhow, probably I will add __builtin_parity() to include/qemu/host-utils.h and some kind of #ifdef HAVE_FAST_CTPOP. Thanks.

Quoted 9 lines
Quoted 7 lines

Still - for something that has a code generator, there _is_ a cost in supporting old CPUs, so I'd rather avoid reverting this. The glibc bug that you linked is very different not just because it affected 32-bit installation media, but also because it was a bug rather than intentional.

Since you are reporting this issue, how did you find out / what broke for you?

I found out from the mailing list. My Core2-based desktop would be affected.

Do you run QEMU on it? With KVM or TCG?

Paolo

#14 lore raw
Quoted 3 lines · Paolo Bonzini
Quoted 1 line

I found out from the mailing list. My Core2-based desktop would be affected.

Do you run QEMU on it? With KVM or TCG?

Excuse me? Are you going to ask for SSH access to ensure my computer really exists and is in working order?

Can you tell me why you never commented on buffer_is_zero improvements, where v1 was sent in October? Just trying to understand how you care for 2% of L1D use but could be ok with those kinds of speedups be dropped on the floor.

Alexander

#15 lore raw
Quoted 7 lines · Alexander Monakov
Quoted 3 lines · Paolo Bonzini
Quoted 1 line

I found out from the mailing list. My Core2-based desktop would be affected.

Do you run QEMU on it? With KVM or TCG?

Excuse me? Are you going to ask for SSH access to ensure my computer really exists and is in working order?

Come on. The thing is, I'm not debating the existence of computers that don't have x86_64-v2, but I *am* debating the usefulness of making QEMU run on them and any extra information can be interesting.

Quoted 3 lines

Can you tell me why you never commented on buffer_is_zero improvements, where v1 was sent in October? Just trying to understand how you care for 2% of L1D use but could be ok with those kinds of speedups be dropped on the floor.

I'm not sure if there is any overlap in the scenarios where buffer_is_zero performance matters, and x86 emulation. People can care about thing A but not thing B. If there's anything that you think I can help reviewing, feel free to let me know offlist.

Paolo

#16 lore raw
Quoted 4 lines · Paolo Bonzini
Quoted 3 lines · Alexander Monakov
Quoted 2 lines · Paolo Bonzini
Quoted 1 line

I found out from the mailing list. My Core2-based desktop would be affected.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Do you run QEMU on it? With KVM or TCG?

Excuse me? Are you going to ask for SSH access to ensure my computer really exists and is in working order?

Come on. The thing is, I'm not debating the existence of computers that don't have x86_64-v2, but I *am* debating the usefulness of making QEMU run on them and any extra information can be interesting.

I think it will be useful to me, with KVM and TCG both.

Quoted 8 lines
Quoted 3 lines

Can you tell me why you never commented on buffer_is_zero improvements, where v1 was sent in October? Just trying to understand how you care for 2% of L1D use but could be ok with those kinds of speedups be dropped on the floor.

I'm not sure if there is any overlap in the scenarios where buffer_is_zero performance matters, and x86 emulation. People can care about thing A but not thing B. If there's anything that you think I can help reviewing, feel free to let me know offlist.

In that case I would've appreciated an early indication you're not interested, making Cc'ing you on followups unnecessary.

Alexander

#17 lore raw
In reply to Paolo Bonzini
Quoted 26 lines · Paolo Bonzini
Quoted 12 lines · Daniel P. Berrangé
Quoted 9 lines · Alexander Monakov

Hello,

I'm sending straightforward reverts to recent patches that bumped minimum required x86 instruction set to SSE4.2. The older chips did not stop working, and people still test and use new software on older hardware:
https://sourceware.org/bugzilla/show_bug.cgi?id=31867

Considering the very minor gains from the baseline raise, I'm honestly not sure why it happened. It seems better to let distributions handle that.

Quoted 7 lines

Indeed distros are opinionated about the x86_64 baseline they want to target.

While RHEL-9 switched to a x86_64-v2 baseline, Fedora has repeatedly rejected the idea of moving to an x86_64-v2 baseline, wanting to retain full backwards compat. So this assumption in QEMU is preventing the distros from satisfying their chosen build target goals.

I didn't do this because of RHEL9, I did it because it's silly that QEMU cannot use POPCNT and has to waste 2% of the L1 d-cache to compute the x86 parity flag (and POPCNT was introduced at the same time as SSE4.2).

Quoted 7 lines

Intel x86_64-v2 processors have been around for about 15 years, AMD for a little less (2011). I'd rather hear from users about the usecases for running QEMU on such old processors before reverting, as this does not get in the way of booting/installing distros on old machines. Unless QEMU is run from within the installation media, which it isn't, requiring a particular processor family does not prevent Fedora from being installable on pre-v2 processors.

This isn't anything to do with the distro installer. The use case is that the distro wants all its software to be able to run on the x86_64 baseline it has chosen to build with.

If we want to use POPCNT in the TCG code, can we not do a runtime check and selectively build pieces of code with __attribute__((target("popcnt"))), as we've done historically for the bufferiszero.c code, rather than changing the entire QEMU baseline ?

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#18 lore raw
Quoted 3 lines · Daniel P. Berrangé

This isn't anything to do with the distro installer. The use case is that the distro wants all its software to be able to run on the x86_64 baseline it has chosen to build with.

Sure, and they can patch the packages if their wish is not shared by upstream. Alternatively they can live with the fact that not all users will be able to use all packages, which is probably already the case. Or drop QEMU, I guess. Has FeSCO ever expressed how strict they are and which of the three options they'd pick?

Either way, this only affects either the QEMU maintainers for the distro, or the users of QEMU. It's only if the installation media used QEMU, that this change would be actively blocking usage of the distro on old processors.

Quoted 4 lines

If we want to use POPCNT in the TCG code, can we not do a runtime check and selectively build pieces of code with __attribute__((target("popcnt"))), as we've done historically for the bufferiszero.c code, rather than changing the entire QEMU baseline ?

bufferiszero.c has a very quick check in front of the indirect call and runs for several hundred clock cycles, so the tradeoff is different there.

I guess that, because these helpers are called by TCG, you wouldn't pay the price of the indirect call. However, adding all this infrastructure for 13-15 year old CPUs is not very enthralling.

Paolo

#19 lore raw
Quoted 8 lines · Paolo Bonzini
Quoted 3 lines · Daniel P. Berrangé

This isn't anything to do with the distro installer. The use case is that the distro wants all its software to be able to run on the x86_64 baseline it has chosen to build with.

Sure, and they can patch the packages if their wish is not shared by upstream. Alternatively they can live with the fact that not all users will be able to use all packages, which is probably already the case.

Yep, there's almost certainly scientific packages that have done optimizations in their builds. QEMU is slightly more special though because it is classed as a "critical path" package for the distro. Even the QEMU linux-user pieces are now critical path, since they're leveraged by docker & podman for running foreign arch containers.

Quoted 2 lines

Or drop QEMU, I guess. Has FeSCO ever expressed how strict they are and which of the three options they'd pick?

I don't know - i'm going to raise this question to find out if there's any guidance.

Quoted 4 lines

Either way, this only affects either the QEMU maintainers for the distro, or the users of QEMU. It's only if the installation media used QEMU, that this change would be actively blocking usage of the distro on old processors.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#20 lore raw
Quoted 17 lines · Daniel P. Berrangé
Quoted 8 lines · Paolo Bonzini
Quoted 3 lines · Daniel P. Berrangé

This isn't anything to do with the distro installer. The use case is that the distro wants all its software to be able to run on the x86_64 baseline it has chosen to build with.

Sure, and they can patch the packages if their wish is not shared by upstream. Alternatively they can live with the fact that not all users will be able to use all packages, which is probably already the case.

Yep, there's almost certainly scientific packages that have done optimizations in their builds. QEMU is slightly more special though because it is classed as a "critical path" package for the distro. Even the QEMU linux-user pieces are now critical path, since they're leveraged by docker & podman for running foreign arch containers.

Quoted 5 lines
Quoted 2 lines

Or drop QEMU, I guess. Has FeSCO ever expressed how strict they are and which of the three options they'd pick?

I don't know - i'm going to raise this question to find out if there's any guidance.

I learnt that FESCo approved a surprisingly loose rule saying

"Libraries packaged in Fedora may require ISA extensions,
however any packaged application must not crash on any officially supported architecture, either by providing a generic fallback implementation OR by cleanly exiting when the requisite hardware support is unavailable."

This might suggest we could put a runtime feature check in main(), print a warning and then exit(1), however, QEMU has alot of code that is triggered from ELF constructors. If we're building the entire of QEMU codebase with extra features enabled, I worry that the constructors could potentially cause a illegal instruction crash before main() runs ?

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#21 lore raw
Quoted 6 lines · Daniel P. Berrangé

This might suggest we could put a runtime feature check in main(), print a warning and then exit(1), however, QEMU has alot of code that is triggered from ELF constructors. If we're building the entire of QEMU codebase with extra features enabled, I worry that the constructors could potentially cause a illegal instruction crash before main() runs ?

And I learnt that one can simply add -mneeded to the compiler command line to achieve that, at least on glibc systems:

$ gcc f.c -mneeded -mpopcnt
$ qemu-x86_64 -cpu core2duo ./a.out
./a.out: CPU ISA level is lower than required
$ qemu-x86_64 ./a.out
1234
$ gcc f.c -mneeded
$ qemu-x86_64 -cpu core2duo ./a.out
1234

Using "readelf -n" on the executable unveils the magic:

Displaying notes found in: .note.gnu.property

  Owner                Data size     Description
  GNU                  0x00000030    NT_GNU_PROPERTY_TYPE_0

Properties: x86 ISA needed: x86-64-baseline, x86-64-v2
x86 feature used: x86
x86 ISA used:

I'm actually amazed. :)

Paolo

#22 lore raw
In reply to Daniel P. Berrangé
Quoted 8 lines · Daniel P. Berrangé

I learnt that FESCo approved a surprisingly loose rule saying

"Libraries packaged in Fedora may require ISA extensions,
however any packaged application must not crash on any officially supported architecture, either by providing a generic fallback implementation OR by cleanly exiting when the requisite hardware support is unavailable."

Quoted 6 lines

This might suggest we could put a runtime feature check in main(), print a warning and then exit(1), however, QEMU has alot of code that is triggered from ELF constructors. If we're building the entire of QEMU codebase with extra features enabled, I worry that the constructors could potentially cause a illegal instruction crash before main() runs ?

Are you literally suggesting to find a solution that satisfies the letter of Fedora rules, and not what's good for the spirit of a wider community.

Alexander

#23 lore raw
Quoted 18 lines · Alexander Monakov
Quoted 8 lines · Daniel P. Berrangé

I learnt that FESCo approved a surprisingly loose rule saying

"Libraries packaged in Fedora may require ISA extensions,
however any packaged application must not crash on any officially supported architecture, either by providing a generic fallback implementation OR by cleanly exiting when the requisite hardware support is unavailable."

Quoted 6 lines

This might suggest we could put a runtime feature check in main(), print a warning and then exit(1), however, QEMU has alot of code that is triggered from ELF constructors. If we're building the entire of QEMU codebase with extra features enabled, I worry that the constructors could potentially cause a illegal instruction crash before main() runs ?

Quoted 2 lines

Are you literally suggesting to find a solution that satisfies the letter of Fedora rules, and not what's good for the spirit of a wider community.

I'm interested in exploring what the options are. Personally I still think QEMU ought to maintain compat with the original x86_64 ABI, since very few distros have moved to requiring -v2, but if that doesn't happen I want to understand the implications for Fedora since that's where I'm a maintainer.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#24 lore raw
In reply to Daniel P. Berrangé
Quoted 24 lines · Daniel P. Berrangé
Quoted 17 lines · Daniel P. Berrangé
Quoted 8 lines · Paolo Bonzini
Quoted 3 lines · Daniel P. Berrangé

This isn't anything to do with the distro installer. The use case is that the distro wants all its software to be able to run on the x86_64 baseline it has chosen to build with.

Sure, and they can patch the packages if their wish is not shared by upstream. Alternatively they can live with the fact that not all users will be able to use all packages, which is probably already the case.

Yep, there's almost certainly scientific packages that have done optimizations in their builds. QEMU is slightly more special though because it is classed as a "critical path" package for the distro. Even the QEMU linux-user pieces are now critical path, since they're leveraged by docker & podman for running foreign arch containers.

Quoted 5 lines
Quoted 2 lines

Or drop QEMU, I guess. Has FeSCO ever expressed how strict they are and which of the three options they'd pick?

I don't know - i'm going to raise this question to find out if there's any guidance.

Quoted 8 lines

I learnt that FESCo approved a surprisingly loose rule saying

"Libraries packaged in Fedora may require ISA extensions,
however any packaged application must not crash on any officially supported architecture, either by providing a generic fallback implementation OR by cleanly exiting when the requisite hardware support is unavailable."

..snip..

I queried the looseness of this wording, and it is suggested it wasn't intended to apply to existing packages, just newly added ones. By that interpretation it wouldn't be valid for QEMU, and we'd be pushed towards the revert downstream, to retain a runtime check for the feature. I really hate the idea of keeping a revert of these patches downstream though, as it would be an indefinite rebase headache.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#25 lore raw
In reply to Paolo Bonzini
Quoted 6 lines · Paolo Bonzini
Quoted 4 lines · Daniel P. Berrangé

If we want to use POPCNT in the TCG code, can we not do a runtime check and selectively build pieces of code with __attribute__((target("popcnt"))), as we've done historically for the bufferiszero.c code, rather than changing the entire QEMU baseline ?

Quoted 7 lines

bufferiszero.c has a very quick check in front of the indirect call and runs for several hundred clock cycles, so the tradeoff is different there.

I guess that, because these helpers are called by TCG, you wouldn't pay the price of the indirect call. However, adding all this infrastructure for 13-15 year old CPUs is not very enthralling.

Ah, so the distinction is that the old code had a runtime check on 'have_popcnt' (and similar), where as now that check is eliminated at compile time, since the condition is a constant.

Rather than re-introducing a runtime check again for everyone, could we make it a configure time argument whether to assume x86_64-v2 ? So those who are happy with a increased baseline can achieve the maximum performance with all checks eliminated at compile time, while still allowing the tradeoff of a dynamic check for those who prefer compatibility over peak perfr ?

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#26 lore raw
Quoted 6 lines · Daniel P. Berrangé
Quoted 3 lines

I guess that, because these helpers are called by TCG, you wouldn't pay the price of the indirect call. However, adding all this infrastructure for 13-15 year old CPUs is not very enthralling.

Rather than re-introducing a runtime check again for everyone, could we make it a configure time argument whether to assume x86_64-v2 ?

Fair enough, I'll work on it.

Paolo

#27 lore raw
In reply to Paolo Bonzini

Hello,

Quoted 4 lines · Paolo Bonzini

I didn't do this because of RHEL9, I did it because it's silly that QEMU cannot use POPCNT and has to waste 2% of the L1 d-cache to compute the x86 parity flag (and POPCNT was introduced at the same time as SSE4.2).

I do not see where the 2% figure is coming from: even considering that the 256-byte LUT may take an extra cache line due to misalignment, 320 bytes is still less than 1% of 32KB L1D size.

More importantly, the way this comment is phrased made me think that Qemu eagerly computes PF. But the comment in target/i386/cpu.h is saying that all flags are computed in an on-demand manner. Considering that software pretty much never uses PF, why would the parity table be resident in L1D? As far as I can see, the cost is rather a cache miss and perhaps a TLB miss when PF is computed (mostly when EFLAGS are accessed all together on context switches I think).

Is there something I'm not seeing?

Thanks.
Alexander

#28 lore raw
Quoted 21 lines · Alexander Monakov

Hello,

Quoted 4 lines · Paolo Bonzini

I didn't do this because of RHEL9, I did it because it's silly that QEMU cannot use POPCNT and has to waste 2% of the L1 d-cache to compute the x86 parity flag (and POPCNT was introduced at the same time as SSE4.2).

I do not see where the 2% figure is coming from: even considering that the 256-byte LUT may take an extra cache line due to misalignment, 320 bytes is still less than 1% of 32KB L1D size.

More importantly, the way this comment is phrased made me think that Qemu eagerly computes PF. But the comment in target/i386/cpu.h is saying that all flags are computed in an on-demand manner. Considering that software pretty much never uses PF, why would the parity table be resident in L1D? As far as I can see, the cost is rather a cache miss and perhaps a TLB miss when PF is computed (mostly when EFLAGS are accessed all together on context switches I think).

Quoted 1 line

Is there something I'm not seeing?

We delay flags computation until they're needed (since flags are often overwritten by the very next instruction), but when we do, we compute all of the flags. So PF is computed at that point, even if PF itself will never be read.

r~

#29 lore raw
In reply to Daniel P. Berrangé
Quoted 11 lines · Daniel P. Berrangé
Quoted 9 lines · Alexander Monakov

Hello,

I'm sending straightforward reverts to recent patches that bumped minimum required x86 instruction set to SSE4.2. The older chips did not stop working, and people still test and use new software on older hardware:
https://sourceware.org/bugzilla/show_bug.cgi?id=31867

Considering the very minor gains from the baseline raise, I'm honestly not sure why it happened. It seems better to let distributions handle that.

Quoted 7 lines

Indeed distros are opinionated about the x86_64 baseline they want to target.

While RHEL-9 switched to a x86_64-v2 baseline, Fedora has repeatedly rejected the idea of moving to an x86_64-v2 baseline, wanting to retain full backwards compat. So this assumption in QEMU is preventing the distros from satisfying their chosen build target goals.

So, to make sure I parsed that correctly, you're in support of the reverts?

Alexander