Skip to contentSkip to conversation

target/i386: Update EPYC CPU models for Cache property, RAS, SVM feature and add EPYC-Turin CPU model

PATCH Applied Babu Moger 5 participants 13 messages May 8, 2025 – Jun 16, 2025 KVM

lore
Share & tools
lore

Babu Moger

author
#0 lore raw

v7 cover target/i386: Update EPYC CPU models for Cache property, RAS, SVM feature and add EPYC-Turin CPU model

Following changes are implemented in this series.

1. Fixed the cache(L2,L3) property details in all the EPYC models.
2. Add RAS feature bits (SUCCOR, McaOverflowRecov) on all EPYC models
3. Add missing SVM feature bits required for nested guests on all EPYC models
4. Add the missing feature bit fs-gs-base-ns(WRMSR to {FS,GS,KERNEL_G}S_BASE is non-serializing). This bit is added in EPYC-Genoa and EPYC-Turin models.
5. Add RAS, SVM, fs-gs-base-ns and perfmon-v2 on EPYC-Genoa and EPYC-Turin models.
6. Add support for EPYC-Turin.
(Add all the above feature bits and few additional bits movdiri, movdir64b,
avx512-vp2intersect, avx-vnni, prefetchi, sbpb, ibpb-brtype, srso-user-kernel-no).

---
v7: Rebased on top latest 57b6f8d07f14 (upstream/master) Merge tag 'pull-target-arm-20250506'
Added new feature bit PREFETCHI. KVM support for the bit is added recently.
https://github.com/kvm-x86/linux/commit/d88bb2ded2ef
Paolo, These patches have been pending for a while. Please consider merging when you get a chance.

v6: Initialized the boolean feature bits to true where applicable.
Added Reviewed-by tag from Zhao.

v5: Add EPYC-Turin CPU model
Dropped ERAPS and RAPSIZE bits from EPYC-Turin models as kernel support for these bits are not done yet. Users can still use the options +eraps,+rapsize to test these featers.
Add Reviewed-by tag from Maksim for the patches already reviewed.

v4: Some of the patches in v3 are already merged. Posting the rest of the patches.
Dropped EPYC-Turin model for now. Will post them later.
Added SVM feature bit as discussed in
https://lore.kernel.org/kvm/[email protected]/
Fixed the cache property details as discussed in
https://lore.kernel.org/kvm/[email protected]/
Thanks to Maksim and Paolo for their feedback.

v3: Added SBPB, IBPB_BRTYPE, SRSO_USER_KERNEL_NO, ERAPS and RAPSIZE bits
to EPYC-Turin.
Added new patch(1) to fix a minor typo.

v2: Fixed couple of typos.
Added Reviewed-by tag from Zhao.
Rebased on top of 6d00c6f98256 ("Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging")

Previous revisions:
v6: https://lore.kernel.org/kvm/[email protected]/
v5: https://lore.kernel.org/kvm/[email protected]/
v4: https://lore.kernel.org/kvm/[email protected]/
v3: https://lore.kernel.org/kvm/[email protected]/
v2: https://lore.kernel.org/kvm/[email protected]/
v1: https://lore.kernel.org/qemu-devel/[email protected]/

Babu Moger (6):
target/i386: Update EPYC CPU model for Cache property, RAS, SVM
feature bits
target/i386: Update EPYC-Rome CPU model for Cache property, RAS, SVM
feature bits
target/i386: Update EPYC-Milan CPU model for Cache property, RAS, SVM
feature bits
target/i386: Add couple of feature bits in CPUID_Fn80000021_EAX target/i386: Update EPYC-Genoa for Cache property, perfmon-v2, RAS and
SVM feature bits
target/i386: Add support for EPYC-Turin model

 target/i386/cpu.c | 439 +++++++++++++++++++++++++++++++++++++++++++++-
 target/i386/cpu.h |   4 +
 2 files changed, 441 insertions(+), 2 deletions(-)
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH v7 1/6 target/i386: Update EPYC CPU model for Cache property, RAS, SVM feature bits

Found that some of the cache properties are not set correctly for EPYC models.

l1d_cache.no_invd_sharing should not be true. l1i_cache.no_invd_sharing should not be true.

L2.self_init should be true.
L2.inclusive should be true.

L3.inclusive should not be true.
L3.no_invd_sharing should be true.

Fix the cache properties.

Also add the missing RAS and SVM features bits on AMD EPYC CPU models. The SVM feature bits are used in nested guests.

succor : Software uncorrectable error containment and recovery capability.
overflow-recov : MCA overflow recovery support.
lbrv : LBR virtualization
tsc-scale : MSR based TSC rate control
vmcb-clean : VMCB clean bits
flushbyasid : Flush by ASID
pause-filter : Pause intercept filter
pfthreshold : PAUSE filter threshold
v-vmsave-vmload : Virtualized VMLOAD and VMSAVE
vgif : Virtualized GIF

---
 target/i386/cpu.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 73 insertions(+)
target/i386/cpu.c +73 -0
index 6f21d5ed22..49d3ae8aac 100644
@@ -2185,6 +2185,60 @@ static CPUCaches epyc_v4_cache_info = {
     },
 };
 
+static CPUCaches epyc_v5_cache_info = {
+    .l1d_cache = &(CPUCacheInfo) {
+        .type = DATA_CACHE,
+        .level = 1,
+        .size = 32 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 64,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l1i_cache = &(CPUCacheInfo) {
+        .type = INSTRUCTION_CACHE,
+        .level = 1,
+        .size = 64 * KiB,
+        .line_size = 64,
+        .associativity = 4,
+        .partitions = 1,
+        .sets = 256,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l2_cache = &(CPUCacheInfo) {
+        .type = UNIFIED_CACHE,
+        .level = 2,
+        .size = 512 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 1024,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .inclusive = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l3_cache = &(CPUCacheInfo) {
+        .type = UNIFIED_CACHE,
+        .level = 3,
+        .size = 8 * MiB,
+        .line_size = 64,
+        .associativity = 16,
+        .partitions = 1,
+        .sets = 8192,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .no_invd_sharing = true,
+        .complex_indexing = false,
+        .share_level = CPU_TOPOLOGY_LEVEL_DIE,
+    },
+};
+
 static const CPUCaches epyc_rome_cache_info = {
     .l1d_cache = &(CPUCacheInfo) {
         .type = DATA_CACHE,
@@ -5212,6 +5266,25 @@ static const X86CPUDefinition builtin_x86_defs[] = {
                 },
                 .cache_info = &epyc_v4_cache_info
             },
+            {
+                .version = 5,
+                .props = (PropValue[]) {
+                    { "overflow-recov", "on" },
+                    { "succor", "on" },
+                    { "lbrv", "on" },
+                    { "tsc-scale", "on" },
+                    { "vmcb-clean", "on" },
+                    { "flushbyasid", "on" },
+                    { "pause-filter", "on" },
+                    { "pfthreshold", "on" },
+                    { "v-vmsave-vmload", "on" },
+                    { "vgif", "on" },
+                    { "model-id",
+                      "AMD EPYC-v5 Processor" },
+                    { /* end of list */ }
+                },
+                .cache_info = &epyc_v5_cache_info
+            },
             { /* end of list */ }
         }
     },
Signature · 1 line
2.34.1
author
#2 lore raw

PATCH v7 2/6 target/i386: Update EPYC-Rome CPU model for Cache property, RAS, SVM feature bits

In reply to Babu Moger

Found that some of the cache properties are not set correctly for EPYC models.

l1d_cache.no_invd_sharing should not be true.
l1i_cache.no_invd_sharing should not be true.

L2.self_init should be true.
L2.inclusive should be true.

L3.inclusive should not be true.
L3.no_invd_sharing should be true.

Fix these cache properties.

Also add the missing RAS and SVM features bits on AMD EPYC-Rome. The SVM feature bits are used in nested guests.

succor : Software uncorrectable error containment and recovery capability.
overflow-recov : MCA overflow recovery support.
lbrv : LBR virtualization
tsc-scale : MSR based TSC rate control
vmcb-clean : VMCB clean bits
flushbyasid : Flush by ASID
pause-filter : Pause intercept filter
pfthreshold : PAUSE filter threshold
v-vmsave-vmload : Virtualized VMLOAD and VMSAVE
vgif : Virtualized GIF

---
 target/i386/cpu.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 73 insertions(+)
target/i386/cpu.c +73 -0
index 49d3ae8aac..3f64293ba5 100644
@@ -2347,6 +2347,60 @@ static const CPUCaches epyc_rome_v3_cache_info = {
     },
 };
 
+static const CPUCaches epyc_rome_v5_cache_info = {
+    .l1d_cache = &(CPUCacheInfo) {
+        .type = DATA_CACHE,
+        .level = 1,
+        .size = 32 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 64,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l1i_cache = &(CPUCacheInfo) {
+        .type = INSTRUCTION_CACHE,
+        .level = 1,
+        .size = 32 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 64,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l2_cache = &(CPUCacheInfo) {
+        .type = UNIFIED_CACHE,
+        .level = 2,
+        .size = 512 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 1024,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .inclusive = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l3_cache = &(CPUCacheInfo) {
+        .type = UNIFIED_CACHE,
+        .level = 3,
+        .size = 16 * MiB,
+        .line_size = 64,
+        .associativity = 16,
+        .partitions = 1,
+        .sets = 16384,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .no_invd_sharing = true,
+        .complex_indexing = false,
+        .share_level = CPU_TOPOLOGY_LEVEL_DIE,
+    },
+};
+
 static const CPUCaches epyc_milan_cache_info = {
     .l1d_cache = &(CPUCacheInfo) {
         .type = DATA_CACHE,
@@ -5423,6 +5477,25 @@ static const X86CPUDefinition builtin_x86_defs[] = {
                     { /* end of list */ }
                 },
             },
+            {
+                .version = 5,
+                .props = (PropValue[]) {
+                    { "overflow-recov", "on" },
+                    { "succor", "on" },
+                    { "lbrv", "on" },
+                    { "tsc-scale", "on" },
+                    { "vmcb-clean", "on" },
+                    { "flushbyasid", "on" },
+                    { "pause-filter", "on" },
+                    { "pfthreshold", "on" },
+                    { "v-vmsave-vmload", "on" },
+                    { "vgif", "on" },
+                    { "model-id",
+                      "AMD EPYC-Rome-v5 Processor" },
+                    { /* end of list */ }
+                },
+                .cache_info = &epyc_rome_v5_cache_info
+            },
             { /* end of list */ }
         }
     },
Signature · 1 line
2.34.1
author
#3 lore raw

PATCH v7 3/6 target/i386: Update EPYC-Milan CPU model for Cache property, RAS, SVM feature bits

In reply to Babu Moger

Found that some of the cache properties are not set correctly for EPYC models. l1d_cache.no_invd_sharing should not be true. l1i_cache.no_invd_sharing should not be true.

L2.self_init should be true.
L2.inclusive should be true.

L3.inclusive should not be true.
L3.no_invd_sharing should be true.

Fix these cache properties.

Also add the missing RAS and SVM features bits on AMD EPYC-Milan model. The SVM feature bits are used in nested guests.

succor : Software uncorrectable error containment and recovery capability.
overflow-recov : MCA overflow recovery support.
lbrv : LBR virtualization
tsc-scale : MSR based TSC rate control
vmcb-clean : VMCB clean bits
flushbyasid : Flush by ASID
pause-filter : Pause intercept filter
pfthreshold : PAUSE filter threshold
v-vmsave-vmload : Virtualized VMLOAD and VMSAVE
vgif : Virtualized GIF

---
 target/i386/cpu.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 73 insertions(+)
target/i386/cpu.c +73 -0
index 3f64293ba5..98fad3a2f9 100644
@@ -2509,6 +2509,60 @@ static const CPUCaches epyc_milan_v2_cache_info = {
     },
 };
 
+static const CPUCaches epyc_milan_v3_cache_info = {
+    .l1d_cache = &(CPUCacheInfo) {
+        .type = DATA_CACHE,
+        .level = 1,
+        .size = 32 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 64,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l1i_cache = &(CPUCacheInfo) {
+        .type = INSTRUCTION_CACHE,
+        .level = 1,
+        .size = 32 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 64,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l2_cache = &(CPUCacheInfo) {
+        .type = UNIFIED_CACHE,
+        .level = 2,
+        .size = 512 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 1024,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .inclusive = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l3_cache = &(CPUCacheInfo) {
+        .type = UNIFIED_CACHE,
+        .level = 3,
+        .size = 32 * MiB,
+        .line_size = 64,
+        .associativity = 16,
+        .partitions = 1,
+        .sets = 32768,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .no_invd_sharing = true,
+        .complex_indexing = false,
+        .share_level = CPU_TOPOLOGY_LEVEL_DIE,
+    },
+};
+
 static const CPUCaches epyc_genoa_cache_info = {
     .l1d_cache = &(CPUCacheInfo) {
         .type = DATA_CACHE,
@@ -5571,6 +5625,25 @@ static const X86CPUDefinition builtin_x86_defs[] = {
                 },
                 .cache_info = &epyc_milan_v2_cache_info
             },
+            {
+                .version = 3,
+                .props = (PropValue[]) {
+                    { "overflow-recov", "on" },
+                    { "succor", "on" },
+                    { "lbrv", "on" },
+                    { "tsc-scale", "on" },
+                    { "vmcb-clean", "on" },
+                    { "flushbyasid", "on" },
+                    { "pause-filter", "on" },
+                    { "pfthreshold", "on" },
+                    { "v-vmsave-vmload", "on" },
+                    { "vgif", "on" },
+                    { "model-id",
+                      "AMD EPYC-Milan-v3 Processor" },
+                    { /* end of list */ }
+                },
+                .cache_info = &epyc_milan_v3_cache_info
+            },
             { /* end of list */ }
         }
     },
Signature · 1 line
2.34.1
author
#4 lore raw

PATCH v7 4/6 target/i386: Add couple of feature bits in CPUID_Fn80000021_EAX

In reply to Babu Moger

Add CPUID bit indicates that a WRMSR to MSR_FS_BASE, MSR_GS_BASE, or MSR_KERNEL_GS_BASE is non-serializing amd PREFETCHI that the indicates support for IC prefetch.

CPUID_Fn80000021_EAX

Bit    Feature description
20     Indicates support for IC prefetch.
1      FsGsKernelGsBaseNonSerializing.

WRMSR to FS_BASE, GS_BASE and KernelGSbase are non-serializing.

---
 target/i386/cpu.c | 4 ++--
 target/i386/cpu.h | 4 ++++
 2 files changed, 6 insertions(+), 2 deletions(-)
target/i386/cpu.c +2 -2
index 98fad3a2f9..741be0eaa8 100644
@@ -1239,12 +1239,12 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
     [FEAT_8000_0021_EAX] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
-            "no-nested-data-bp", NULL, "lfence-always-serializing", NULL,
+            "no-nested-data-bp", "fs-gs-base-ns", "lfence-always-serializing", NULL,
             NULL, NULL, "null-sel-clr-base", NULL,
             "auto-ibrs", NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
-            NULL, NULL, NULL, NULL,
+            "prefetchi", NULL, NULL, NULL,
             "eraps", NULL, NULL, "sbpb",
             "ibpb-brtype", "srso-no", "srso-user-kernel-no", NULL,
         },
target/i386/cpu.h +4 -0
index 4f8ed8868e..d251e32ae9 100644
@@ -1070,12 +1070,16 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 
 /* Processor ignores nested data breakpoints */
 #define CPUID_8000_0021_EAX_NO_NESTED_DATA_BP            (1U << 0)
+/* WRMSR to FS_BASE, GS_BASE, or KERNEL_GS_BASE is non-serializing */
+#define CPUID_8000_0021_EAX_FS_GS_BASE_NS                (1U << 1)
 /* LFENCE is always serializing */
 #define CPUID_8000_0021_EAX_LFENCE_ALWAYS_SERIALIZING    (1U << 2)
 /* Null Selector Clears Base */
 #define CPUID_8000_0021_EAX_NULL_SEL_CLR_BASE            (1U << 6)
 /* Automatic IBRS */
 #define CPUID_8000_0021_EAX_AUTO_IBRS                    (1U << 8)
+/* Indicates support for IC prefetch */
+#define CPUID_8000_0021_EAX_PREFETCHI                    (1U << 20)
 /* Enhanced Return Address Predictor Scurity */
 #define CPUID_8000_0021_EAX_ERAPS                        (1U << 24)
 /* Selective Branch Predictor Barrier */
Signature · 1 line
2.34.1
#5 lore raw

Re: [PATCH v7 4/6] target/i386: Add couple of feature bits in CPUID_Fn80000021_EAX

Quoted 8 lines · * Babu Moger ([email protected]) wrote:

Add CPUID bit indicates that a WRMSR to MSR_FS_BASE, MSR_GS_BASE, or MSR_KERNEL_GS_BASE is non-serializing amd PREFETCHI that the indicates support for IC prefetch.

CPUID_Fn80000021_EAX

Bit    Feature description
20     Indicates support for IC prefetch.
1      FsGsKernelGsBaseNonSerializing.

I'm curious about this:
a) Is this new CPUs are non-serialising on that write?
b) If so, what happens if you run existing kernels/firmware on them?
c) Bonus migration question; what happens if you live migrate from a host that claims to be serialising to one that has the extra non-serialising flag but is disabled in the emulated CPU model.

Dave

Quoted 55 lines

WRMSR to FS_BASE, GS_BASE and KernelGSbase are non-serializing.

---
 target/i386/cpu.c | 4 ++--
 target/i386/cpu.h | 4 ++++
 2 files changed, 6 insertions(+), 2 deletions(-)
target/i386/cpu.c +2 -2
index 98fad3a2f9..741be0eaa8 100644
@@ -1239,12 +1239,12 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
     [FEAT_8000_0021_EAX] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
-            "no-nested-data-bp", NULL, "lfence-always-serializing", NULL,
+            "no-nested-data-bp", "fs-gs-base-ns", "lfence-always-serializing", NULL,
             NULL, NULL, "null-sel-clr-base", NULL,
             "auto-ibrs", NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
             NULL, NULL, NULL, NULL,
-            NULL, NULL, NULL, NULL,
+            "prefetchi", NULL, NULL, NULL,
             "eraps", NULL, NULL, "sbpb",
             "ibpb-brtype", "srso-no", "srso-user-kernel-no", NULL,
         },
target/i386/cpu.h +4 -0
index 4f8ed8868e..d251e32ae9 100644
@@ -1070,12 +1070,16 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 
 /* Processor ignores nested data breakpoints */
 #define CPUID_8000_0021_EAX_NO_NESTED_DATA_BP            (1U << 0)
+/* WRMSR to FS_BASE, GS_BASE, or KERNEL_GS_BASE is non-serializing */
+#define CPUID_8000_0021_EAX_FS_GS_BASE_NS                (1U << 1)
 /* LFENCE is always serializing */
 #define CPUID_8000_0021_EAX_LFENCE_ALWAYS_SERIALIZING    (1U << 2)
 /* Null Selector Clears Base */
 #define CPUID_8000_0021_EAX_NULL_SEL_CLR_BASE            (1U << 6)
 /* Automatic IBRS */
 #define CPUID_8000_0021_EAX_AUTO_IBRS                    (1U << 8)
+/* Indicates support for IC prefetch */
+#define CPUID_8000_0021_EAX_PREFETCHI                    (1U << 20)
 /* Enhanced Return Address Predictor Scurity */
 #define CPUID_8000_0021_EAX_ERAPS                        (1U << 24)
 /* Selective Branch Predictor Barrier */
Signature · 1 line
2.34.1
Signature · 4 lines
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    |       Running GNU/Linux       | Happy  \ 
\        dave @ treblig.org |                               | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/
#6 lore raw

Re: [PATCH v7 4/6] target/i386: Add couple of feature bits in CPUID_Fn80000021_EAX

Hi Dave,

Quoted 10 lines · Dr. David Alan Gilbert
Quoted 8 lines · * Babu Moger ([email protected]) wrote:

Add CPUID bit indicates that a WRMSR to MSR_FS_BASE, MSR_GS_BASE, or MSR_KERNEL_GS_BASE is non-serializing amd PREFETCHI that the indicates support for IC prefetch.

CPUID_Fn80000021_EAX

Bit    Feature description
20     Indicates support for IC prefetch.
1      FsGsKernelGsBaseNonSerializing.
Quoted 6 lines

I'm curious about this:
a) Is this new CPUs are non-serialising on that write?
b) If so, what happens if you run existing kernels/firmware on them?
c) Bonus migration question; what happens if you live migrate from a host that claims to be serialising to one that has the extra non-serialising flag but is disabled in the emulated CPU model.

Good question. After looking at the AMD64 Architecture Programmer’s Manual again, these writes have always been non-serializing. Behavior has not changed. We're just reporting it through CPUID now. This information likely isn’t being used anywhere. Let me know if you have any questions.

Signature · 2 lines
Thanks
Babu Moger
#7 lore raw

Re: [PATCH v7 4/6] target/i386: Add couple of feature bits in CPUID_Fn80000021_EAX

Quoted 20 lines · * Moger, Babu ([email protected]) wrote:

Hi Dave,

Quoted 10 lines · Dr. David Alan Gilbert
Quoted 8 lines · * Babu Moger ([email protected]) wrote:

Add CPUID bit indicates that a WRMSR to MSR_FS_BASE, MSR_GS_BASE, or MSR_KERNEL_GS_BASE is non-serializing amd PREFETCHI that the indicates support for IC prefetch.

CPUID_Fn80000021_EAX

Bit    Feature description
20     Indicates support for IC prefetch.
1      FsGsKernelGsBaseNonSerializing.
Quoted 6 lines

I'm curious about this:
a) Is this new CPUs are non-serialising on that write?
b) If so, what happens if you run existing kernels/firmware on them?
c) Bonus migration question; what happens if you live migrate from a host that claims to be serialising to one that has the extra non-serialising flag but is disabled in the emulated CPU model.

Quoted 3 lines

Good question. After looking at the AMD64 Architecture Programmer’s Manual again, these writes have always been non-serializing. Behavior has not changed.

Ah OK, then nothing to worry about.

Quoted 2 lines

We're just reporting it through CPUID now. This information likely isn’t being used anywhere.

Seems curious to add it then!

Quoted 1 line

Let me know if you have any questions.

No, thanks for the reply.

Dave

Quoted 3 lines
Signature · 2 lines
Thanks
Babu Moger
Signature · 4 lines
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    |       Running GNU/Linux       | Happy  \ 
\        dave @ treblig.org |                               | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/
author
#8 lore raw

PATCH v7 5/6 target/i386: Update EPYC-Genoa for Cache property, perfmon-v2, RAS and SVM feature bits

In reply to Babu Moger

Found that some of the cache properties are not set correctly for EPYC models. l1d_cache.no_invd_sharing should not be true. l1i_cache.no_invd_sharing should not be true.

L2.self_init should be true.
L2.inclusive should be true.

L3.inclusive should not be true.
L3.no_invd_sharing should be true.

Fix these cache properties.

Also add the missing RAS and SVM features bits on AMD EPYC-Genoa model. The SVM feature bits are used in nested guests.

perfmon-v2 : Allow guests to make use of the PerfMonV2 features.
succor : Software uncorrectable error containment and recovery capability.
overflow-recov : MCA overflow recovery support.
lbrv : LBR virtualization
tsc-scale : MSR based TSC rate control
vmcb-clean : VMCB clean bits
flushbyasid : Flush by ASID
pause-filter : Pause intercept filter
pfthreshold : PAUSE filter threshold
v-vmsave-vmload: Virtualized VMLOAD and VMSAVE
vgif : Virtualized GIF
fs-gs-base-ns : WRMSR to {FS,GS,KERNEL_GS}_BASE is non-serializing

The feature details are available in APM listed below [1].
[1] AMD64 Architecture Programmer's Manual Volume 2: System Programming Publication # 24593 Revision 3.41.

---
 target/i386/cpu.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 78 insertions(+)
target/i386/cpu.c +78 -0
index 741be0eaa8..8384ad6eff 100644
@@ -2617,6 +2617,59 @@ static const CPUCaches epyc_genoa_cache_info = {
     },
 };
 
+static const CPUCaches epyc_genoa_v2_cache_info = {
+    .l1d_cache = &(CPUCacheInfo) {
+        .type = DATA_CACHE,
+        .level = 1,
+        .size = 32 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 64,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l1i_cache = &(CPUCacheInfo) {
+        .type = INSTRUCTION_CACHE,
+        .level = 1,
+        .size = 32 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 64,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l2_cache = &(CPUCacheInfo) {
+        .type = UNIFIED_CACHE,
+        .level = 2,
+        .size = 1 * MiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 2048,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .inclusive = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l3_cache = &(CPUCacheInfo) {
+        .type = UNIFIED_CACHE,
+        .level = 3,
+        .size = 32 * MiB,
+        .line_size = 64,
+        .associativity = 16,
+        .partitions = 1,
+        .sets = 32768,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .no_invd_sharing = true,
+        .complex_indexing = false,
+        .share_level = CPU_TOPOLOGY_LEVEL_DIE,
+    },
+};
 /* The following VMX features are not supported by KVM and are left out in the
  * CPU definitions:
  *
@@ -5718,6 +5771,31 @@ static const X86CPUDefinition builtin_x86_defs[] = {
         .xlevel = 0x80000022,
         .model_id = "AMD EPYC-Genoa Processor",
         .cache_info = &epyc_genoa_cache_info,
+        .versions = (X86CPUVersionDefinition[]) {
+            { .version = 1 },
+            {
+                .version = 2,
+                .props = (PropValue[]) {
+                    { "overflow-recov", "on" },
+                    { "succor", "on" },
+                    { "lbrv", "on" },
+                    { "tsc-scale", "on" },
+                    { "vmcb-clean", "on" },
+                    { "flushbyasid", "on" },
+                    { "pause-filter", "on" },
+                    { "pfthreshold", "on" },
+                    { "v-vmsave-vmload", "on" },
+                    { "vgif", "on" },
+                    { "fs-gs-base-ns", "on" },
+                    { "perfmon-v2", "on" },
+                    { "model-id",
+                      "AMD EPYC-Genoa-v2 Processor" },
+                    { /* end of list */ }
+                },
+                .cache_info = &epyc_genoa_v2_cache_info
+            },
+            { /* end of list */ }
+        }
     },
     {
         .name = "YongFeng",
Signature · 1 line
2.34.1
author
#9 lore raw

PATCH v7 6/6 target/i386: Add support for EPYC-Turin model

In reply to Babu Moger

Add the support for AMD EPYC zen 5 processors (EPYC-Turin).

Add the following new feature bits on top of the feature bits from the previous generation EPYC models.

movdiri : Move Doubleword as Direct Store Instruction
movdir64b : Move 64 Bytes as Direct Store Instruction
avx512-vp2intersect : AVX512 Vector Pair Intersection to a Pair
of Mask Register
avx-vnni : AVX VNNI Instruction
prefetchi : Indicates support for IC prefetch
sbpb : Selective Branch Predictor Barrier
ibpb-brtype : IBPB includes branch type prediction flushing
srso-user-kernel-no : Not vulnerable to SRSO at the user-kernel boundary

---
 target/i386/cpu.c | 138 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 138 insertions(+)
target/i386/cpu.c +138 -0
index 8384ad6eff..247dcdbc34 100644
@@ -2670,6 +2670,61 @@ static const CPUCaches epyc_genoa_v2_cache_info = {
         .share_level = CPU_TOPOLOGY_LEVEL_DIE,
     },
 };
+
+static const CPUCaches epyc_turin_cache_info = {
+    .l1d_cache = &(CPUCacheInfo) {
+        .type = DATA_CACHE,
+        .level = 1,
+        .size = 48 * KiB,
+        .line_size = 64,
+        .associativity = 12,
+        .partitions = 1,
+        .sets = 64,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l1i_cache = &(CPUCacheInfo) {
+        .type = INSTRUCTION_CACHE,
+        .level = 1,
+        .size = 32 * KiB,
+        .line_size = 64,
+        .associativity = 8,
+        .partitions = 1,
+        .sets = 64,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l2_cache = &(CPUCacheInfo) {
+        .type = UNIFIED_CACHE,
+        .level = 2,
+        .size = 1 * MiB,
+        .line_size = 64,
+        .associativity = 16,
+        .partitions = 1,
+        .sets = 1024,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .inclusive = true,
+        .share_level = CPU_TOPOLOGY_LEVEL_CORE,
+    },
+    .l3_cache = &(CPUCacheInfo) {
+        .type = UNIFIED_CACHE,
+        .level = 3,
+        .size = 32 * MiB,
+        .line_size = 64,
+        .associativity = 16,
+        .partitions = 1,
+        .sets = 32768,
+        .lines_per_tag = 1,
+        .self_init = true,
+        .no_invd_sharing = true,
+        .complex_indexing = false,
+        .share_level = CPU_TOPOLOGY_LEVEL_DIE,
+    },
+};
+
 /* The following VMX features are not supported by KVM and are left out in the
  * CPU definitions:
  *
@@ -5933,6 +5988,89 @@ static const X86CPUDefinition builtin_x86_defs[] = {
             { /* end of list */ }
         }
     },
+    {
+        .name = "EPYC-Turin",
+        .level = 0xd,
+        .vendor = CPUID_VENDOR_AMD,
+        .family = 26,
+        .model = 0,
+        .stepping = 0,
+        .features[FEAT_1_ECX] =
+            CPUID_EXT_RDRAND | CPUID_EXT_F16C | CPUID_EXT_AVX |
+            CPUID_EXT_XSAVE | CPUID_EXT_AES |  CPUID_EXT_POPCNT |
+            CPUID_EXT_MOVBE | CPUID_EXT_SSE42 | CPUID_EXT_SSE41 |
+            CPUID_EXT_PCID | CPUID_EXT_CX16 | CPUID_EXT_FMA |
+            CPUID_EXT_SSSE3 | CPUID_EXT_MONITOR | CPUID_EXT_PCLMULQDQ |
+            CPUID_EXT_SSE3,
+        .features[FEAT_1_EDX] =
+            CPUID_SSE2 | CPUID_SSE | CPUID_FXSR | CPUID_MMX | CPUID_CLFLUSH |
+            CPUID_PSE36 | CPUID_PAT | CPUID_CMOV | CPUID_MCA | CPUID_PGE |
+            CPUID_MTRR | CPUID_SEP | CPUID_APIC | CPUID_CX8 | CPUID_MCE |
+            CPUID_PAE | CPUID_MSR | CPUID_TSC | CPUID_PSE | CPUID_DE |
+            CPUID_VME | CPUID_FP87,
+        .features[FEAT_6_EAX] =
+            CPUID_6_EAX_ARAT,
+        .features[FEAT_7_0_EBX] =
+            CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_BMI1 | CPUID_7_0_EBX_AVX2 |
+            CPUID_7_0_EBX_SMEP | CPUID_7_0_EBX_BMI2 | CPUID_7_0_EBX_ERMS |
+            CPUID_7_0_EBX_INVPCID | CPUID_7_0_EBX_AVX512F |
+            CPUID_7_0_EBX_AVX512DQ | CPUID_7_0_EBX_RDSEED | CPUID_7_0_EBX_ADX |
+            CPUID_7_0_EBX_SMAP | CPUID_7_0_EBX_AVX512IFMA |
+            CPUID_7_0_EBX_CLFLUSHOPT | CPUID_7_0_EBX_CLWB |
+            CPUID_7_0_EBX_AVX512CD | CPUID_7_0_EBX_SHA_NI |
+            CPUID_7_0_EBX_AVX512BW | CPUID_7_0_EBX_AVX512VL,
+        .features[FEAT_7_0_ECX] =
+            CPUID_7_0_ECX_AVX512_VBMI | CPUID_7_0_ECX_UMIP | CPUID_7_0_ECX_PKU |
+            CPUID_7_0_ECX_AVX512_VBMI2 | CPUID_7_0_ECX_GFNI |
+            CPUID_7_0_ECX_VAES | CPUID_7_0_ECX_VPCLMULQDQ |
+            CPUID_7_0_ECX_AVX512VNNI | CPUID_7_0_ECX_AVX512BITALG |
+            CPUID_7_0_ECX_AVX512_VPOPCNTDQ | CPUID_7_0_ECX_LA57 |
+            CPUID_7_0_ECX_RDPID | CPUID_7_0_ECX_MOVDIRI |
+            CPUID_7_0_ECX_MOVDIR64B,
+        .features[FEAT_7_0_EDX] =
+            CPUID_7_0_EDX_FSRM | CPUID_7_0_EDX_AVX512_VP2INTERSECT,
+        .features[FEAT_7_1_EAX] =
+            CPUID_7_1_EAX_AVX_VNNI | CPUID_7_1_EAX_AVX512_BF16,
+        .features[FEAT_8000_0001_ECX] =
+            CPUID_EXT3_OSVW | CPUID_EXT3_3DNOWPREFETCH |
+            CPUID_EXT3_MISALIGNSSE | CPUID_EXT3_SSE4A | CPUID_EXT3_ABM |
+            CPUID_EXT3_CR8LEG | CPUID_EXT3_SVM | CPUID_EXT3_LAHF_LM |
+            CPUID_EXT3_TOPOEXT | CPUID_EXT3_PERFCORE,
+        .features[FEAT_8000_0001_EDX] =
+            CPUID_EXT2_LM | CPUID_EXT2_RDTSCP | CPUID_EXT2_PDPE1GB |
+            CPUID_EXT2_FFXSR | CPUID_EXT2_MMXEXT | CPUID_EXT2_NX |
+            CPUID_EXT2_SYSCALL,
+        .features[FEAT_8000_0007_EBX] =
+            CPUID_8000_0007_EBX_OVERFLOW_RECOV | CPUID_8000_0007_EBX_SUCCOR,
+        .features[FEAT_8000_0008_EBX] =
+            CPUID_8000_0008_EBX_CLZERO | CPUID_8000_0008_EBX_XSAVEERPTR |
+            CPUID_8000_0008_EBX_WBNOINVD | CPUID_8000_0008_EBX_IBPB |
+            CPUID_8000_0008_EBX_IBRS | CPUID_8000_0008_EBX_STIBP |
+            CPUID_8000_0008_EBX_STIBP_ALWAYS_ON |
+            CPUID_8000_0008_EBX_AMD_SSBD | CPUID_8000_0008_EBX_AMD_PSFD,
+        .features[FEAT_8000_0021_EAX] =
+            CPUID_8000_0021_EAX_NO_NESTED_DATA_BP |
+            CPUID_8000_0021_EAX_FS_GS_BASE_NS |
+            CPUID_8000_0021_EAX_LFENCE_ALWAYS_SERIALIZING |
+            CPUID_8000_0021_EAX_NULL_SEL_CLR_BASE |
+            CPUID_8000_0021_EAX_AUTO_IBRS | CPUID_8000_0021_EAX_PREFETCHI |
+            CPUID_8000_0021_EAX_SBPB | CPUID_8000_0021_EAX_IBPB_BRTYPE |
+            CPUID_8000_0021_EAX_SRSO_USER_KERNEL_NO,
+        .features[FEAT_8000_0022_EAX] =
+            CPUID_8000_0022_EAX_PERFMON_V2,
+        .features[FEAT_XSAVE] =
+            CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC |
+            CPUID_XSAVE_XGETBV1 | CPUID_XSAVE_XSAVES,
+        .features[FEAT_SVM] =
+            CPUID_SVM_NPT | CPUID_SVM_LBRV | CPUID_SVM_NRIPSAVE |
+            CPUID_SVM_TSCSCALE | CPUID_SVM_VMCBCLEAN | CPUID_SVM_FLUSHASID |
+            CPUID_SVM_PAUSEFILTER | CPUID_SVM_PFTHRESHOLD |
+            CPUID_SVM_V_VMSAVE_VMLOAD | CPUID_SVM_VGIF |
+            CPUID_SVM_VNMI | CPUID_SVM_SVME_ADDR_CHK,
+        .xlevel = 0x80000022,
+        .model_id = "AMD EPYC-Turin Processor",
+        .cache_info = &epyc_turin_cache_info,
+    },
 };
 
 /*
Signature · 1 line
2.34.1
Reviewed-by on the v7 series
#10 lore raw
In reply to Babu Moger
Quoted 69 lines · Babu Moger

Following changes are implemented in this series.

1. Fixed the cache(L2,L3) property details in all the EPYC models.
2. Add RAS feature bits (SUCCOR, McaOverflowRecov) on all EPYC models
3. Add missing SVM feature bits required for nested guests on all EPYC models
4. Add the missing feature bit fs-gs-base-ns(WRMSR to {FS,GS,KERNEL_G}S_BASE is non-serializing). This bit is added in EPYC-Genoa and EPYC-Turin models.
5. Add RAS, SVM, fs-gs-base-ns and perfmon-v2 on EPYC-Genoa and EPYC-Turin models.
6. Add support for EPYC-Turin.
(Add all the above feature bits and few additional bits movdiri, movdir64b,
avx512-vp2intersect, avx-vnni, prefetchi, sbpb, ibpb-brtype, srso-user-kernel-no).

---
v7: Rebased on top latest 57b6f8d07f14 (upstream/master) Merge tag 'pull-target-arm-20250506'
Added new feature bit PREFETCHI. KVM support for the bit is added recently.
https://github.com/kvm-x86/linux/commit/d88bb2ded2ef
Paolo, These patches have been pending for a while. Please consider merging when you get a chance.

v6: Initialized the boolean feature bits to true where applicable.
Added Reviewed-by tag from Zhao.

v5: Add EPYC-Turin CPU model
Dropped ERAPS and RAPSIZE bits from EPYC-Turin models as kernel support for these bits are not done yet. Users can still use the options +eraps,+rapsize to test these featers.
Add Reviewed-by tag from Maksim for the patches already reviewed.

v4: Some of the patches in v3 are already merged. Posting the rest of the patches.
Dropped EPYC-Turin model for now. Will post them later.
Added SVM feature bit as discussed in
https://lore.kernel.org/kvm/[email protected]/
Fixed the cache property details as discussed in
https://lore.kernel.org/kvm/[email protected]/
Thanks to Maksim and Paolo for their feedback.

v3: Added SBPB, IBPB_BRTYPE, SRSO_USER_KERNEL_NO, ERAPS and RAPSIZE bits
to EPYC-Turin.
Added new patch(1) to fix a minor typo.

v2: Fixed couple of typos.
Added Reviewed-by tag from Zhao.
Rebased on top of 6d00c6f98256 ("Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging")

Previous revisions:
v6: https://lore.kernel.org/kvm/[email protected]/
v5: https://lore.kernel.org/kvm/[email protected]/
v4: https://lore.kernel.org/kvm/[email protected]/
v3: https://lore.kernel.org/kvm/[email protected]/
v2: https://lore.kernel.org/kvm/[email protected]/
v1: https://lore.kernel.org/qemu-devel/[email protected]/

Babu Moger (6):
target/i386: Update EPYC CPU model for Cache property, RAS, SVM
feature bits
target/i386: Update EPYC-Rome CPU model for Cache property, RAS, SVM
feature bits
target/i386: Update EPYC-Milan CPU model for Cache property, RAS, SVM
feature bits
target/i386: Add couple of feature bits in CPUID_Fn80000021_EAX target/i386: Update EPYC-Genoa for Cache property, perfmon-v2, RAS and
SVM feature bits
target/i386: Add support for EPYC-Turin model

Quoted 3 lines

target/i386/cpu.c | 439 +++++++++++++++++++++++++++++++++++++++++++++-
target/i386/cpu.h | 4 +

2 files changed, 441 insertions(+), 2 deletions(-)

Hey Babu and Paolo,
Is there anything outstanding on this series? I didn’t see any further comments.

Anyhow, I did step thru this patch by patch, LGTM:

Reviewed-by Jon Kohler <[email protected] <mailto:[email protected]>>
maintainer Applied
#11 lore raw
In reply to Babu Moger
Quoted 12 lines · Babu Moger

Following changes are implemented in this series.

1. Fixed the cache(L2,L3) property details in all the EPYC models.
2. Add RAS feature bits (SUCCOR, McaOverflowRecov) on all EPYC models
3. Add missing SVM feature bits required for nested guests on all EPYC models
4. Add the missing feature bit fs-gs-base-ns(WRMSR to {FS,GS,KERNEL_G}S_BASE is non-serializing). This bit is added in EPYC-Genoa and EPYC-Turin models.
5. Add RAS, SVM, fs-gs-base-ns and perfmon-v2 on EPYC-Genoa and EPYC-Turin models.
6. Add support for EPYC-Turin.
(Add all the above feature bits and few additional bits movdiri, movdir64b,
avx512-vp2intersect, avx-vnni, prefetchi, sbpb, ibpb-brtype, srso-user-kernel-no).

Queued, thanks.

Paolo

Quoted 57 lines

---
v7: Rebased on top latest 57b6f8d07f14 (upstream/master) Merge tag 'pull-target-arm-20250506'
Added new feature bit PREFETCHI. KVM support for the bit is added recently.
https://github.com/kvm-x86/linux/commit/d88bb2ded2ef
Paolo, These patches have been pending for a while. Please consider merging when you get a chance.

v6: Initialized the boolean feature bits to true where applicable.
Added Reviewed-by tag from Zhao.

v5: Add EPYC-Turin CPU model
Dropped ERAPS and RAPSIZE bits from EPYC-Turin models as kernel support for these bits are not done yet. Users can still use the options +eraps,+rapsize to test these featers.
Add Reviewed-by tag from Maksim for the patches already reviewed.

v4: Some of the patches in v3 are already merged. Posting the rest of the patches.
Dropped EPYC-Turin model for now. Will post them later.
Added SVM feature bit as discussed in
https://lore.kernel.org/kvm/[email protected]/
Fixed the cache property details as discussed in
https://lore.kernel.org/kvm/[email protected]/
Thanks to Maksim and Paolo for their feedback.

v3: Added SBPB, IBPB_BRTYPE, SRSO_USER_KERNEL_NO, ERAPS and RAPSIZE bits
to EPYC-Turin.
Added new patch(1) to fix a minor typo.

v2: Fixed couple of typos.
Added Reviewed-by tag from Zhao.
Rebased on top of 6d00c6f98256 ("Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging")

Previous revisions:
v6: https://lore.kernel.org/kvm/[email protected]/
v5: https://lore.kernel.org/kvm/[email protected]/
v4: https://lore.kernel.org/kvm/[email protected]/
v3: https://lore.kernel.org/kvm/[email protected]/
v2: https://lore.kernel.org/kvm/[email protected]/
v1: https://lore.kernel.org/qemu-devel/[email protected]/

Babu Moger (6):
target/i386: Update EPYC CPU model for Cache property, RAS, SVM
feature bits
target/i386: Update EPYC-Rome CPU model for Cache property, RAS, SVM
feature bits
target/i386: Update EPYC-Milan CPU model for Cache property, RAS, SVM
feature bits
target/i386: Add couple of feature bits in CPUID_Fn80000021_EAX target/i386: Update EPYC-Genoa for Cache property, perfmon-v2, RAS and
SVM feature bits
target/i386: Add support for EPYC-Turin model

  target/i386/cpu.c | 439 +++++++++++++++++++++++++++++++++++++++++++++-
  target/i386/cpu.h |   4 +
  2 files changed, 441 insertions(+), 2 deletions(-)
#12 lore raw
Quoted 14 lines · Paolo Bonzini
Quoted 12 lines · Babu Moger

Following changes are implemented in this series.

1. Fixed the cache(L2,L3) property details in all the EPYC models.
2. Add RAS feature bits (SUCCOR, McaOverflowRecov) on all EPYC models
3. Add missing SVM feature bits required for nested guests on all EPYC models
4. Add the missing feature bit fs-gs-base-ns(WRMSR to {FS,GS,KERNEL_G} S_BASE is non-serializing). This bit is added in EPYC-Genoa and EPYC-Turin models.
5. Add RAS, SVM, fs-gs-base-ns and perfmon-v2 on EPYC-Genoa and EPYC- Turin models.
6. Add support for EPYC-Turin.
    (Add all the above feature bits and few additional bits movdiri, movdir64b, avx512-vp2intersect, avx-vnni, prefetchi, sbpb, ibpb-brtype, srso-user-kernel-no).

Quoted 2 lines

Queued, thanks.

Thank you
- Babu