Skip to contentSkip to conversation

KVM: s390: Add support for 2G hugepages

PATCH Superseded Claudio Imbrenda 3 participants 9 messages Jun 3 – Jun 8

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Claudio Imbrenda

author maintainer
#0 lore raw

cover KVM: s390: Add support for 2G hugepages

Add support for 2G hugepages.

In theory, since kernel 7.0, userspace could already use 2G hugepages to back guest memory, but KVM would use 4k or 1M pages in gmap to actually map the guest.

This series allows KVM on s390 to actually use 2G hugepages in gmap to map guest memory that is backed by 2G hugepages in userspace.

Nested guests remain possible.

Claudio Imbrenda (3):
KVM: s390: Add module parameter to fence 2G hugepages
KVM: s390: Add capability to support 2G hugepages
KVM: s390: Allow for 2G hugepages

 arch/s390/kvm/gmap.c     | 24 +++++++++++++++++++++++-
 arch/s390/kvm/kvm-s390.c | 38 +++++++++++++++++++++++++++++++++++++-
 arch/s390/kvm/pv.c       |  1 +
 include/uapi/linux/kvm.h |  1 +
 4 files changed, 62 insertions(+), 2 deletions(-)
Signature · 1 line
2.54.0
author maintainer
#1 lore raw

PATCH 1/3 KVM: s390: Add module parameter to fence 2G hugepages

Add the hpage_2g module parameter to KVM to allow enabling or disabling 2G hugepages in KVM.

If hpage_2g is enabled but hpage is not enabled, print a message and disable hpage_2g.

Opportunistically fix the comment for the hpage module parameter.

---
 arch/s390/kvm/kvm-s390.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
arch/s390/kvm/kvm-s390.c +11 -1
index ffb20a64d328..801a622691b6 100644
@@ -204,11 +204,16 @@ static int nested;
 module_param(nested, int, S_IRUGO);
 MODULE_PARM_DESC(nested, "Nested virtualization support");
 
-/* allow 1m huge page guest backing, if !nested */
+/* allow 1m huge page guest backing */
 static int hpage;
 module_param(hpage, int, 0444);
 MODULE_PARM_DESC(hpage, "1m huge page backing support");
 
+/* allow 2g huge page guest backing */
+static int hpage_2g;
+module_param(hpage_2g, int, 0444);
+MODULE_PARM_DESC(hpage_2g, "2g huge page backing support");
+
 /* maximum percentage of steal time for polling.  >100 is treated like 100 */
 static u8 halt_poll_max_steal = 10;
 module_param(halt_poll_max_steal, byte, 0644);
@@ -5820,6 +5825,11 @@ static int __init kvm_s390_init(void)
 		return -ENODEV;
 	}
 
+	if (hpage_2g && !hpage) {
+		hpage_2g = 0;
+		pr_info("Disabling 2G hugepage support, since 1M hugepage support is not enabled.\n");
+	}
+
 	for (i = 0; i < 16; i++)
 		kvm_s390_fac_base[i] |=
 			stfle_fac_list[i] & nonhyp_mask(i);
Signature · 1 line
2.54.0
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH v1 1/3] KVM: s390: Add module parameter to fence 2G hugepages

Quoted 9 lines · Claudio Imbrenda

Add the hpage_2g module parameter to KVM to allow enabling or disabling 2G hugepages in KVM.

If hpage_2g is enabled but hpage is not enabled, print a message and disable hpage_2g.

Opportunistically fix the comment for the hpage module parameter.

author maintainer
#3 lore raw

PATCH 2/3 KVM: s390: Add capability to support 2G hugepages

In reply to Claudio Imbrenda

Add KVM_CAP_S390_HPAGE_2G to signal to userspace that 2G hugepages may be used to back the guest; restrictions apply similar to 1M hugepages.

Enable the (for now still ignored) GMAP_FLAG_ALLOW_HPAGE_2G flag for the guest gmap, and propagate / disable it as necessary.

---
 arch/s390/kvm/gmap.c     |  5 +++++
 arch/s390/kvm/kvm-s390.c | 26 ++++++++++++++++++++++++++
 arch/s390/kvm/pv.c       |  1 +
 include/uapi/linux/kvm.h |  1 +
 4 files changed, 33 insertions(+)
arch/s390/kvm/gmap.c +5 -0
index 52d55ddea8d4..c9e348149ba1 100644
@@ -105,6 +105,11 @@ static void gmap_add_child(struct gmap *parent, struct gmap *child)
 	else
 		clear_bit(GMAP_FLAG_ALLOW_HPAGE_1M, &child->flags);
 
+	if (test_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &parent->flags))
+		set_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &child->flags);
+	else
+		clear_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &child->flags);
+
 	if (kvm_is_ucontrol(parent->kvm))
 		clear_bit(GMAP_FLAG_OWNS_PAGETABLES, &child->flags);
 	list_add(&child->list, &parent->children);
arch/s390/kvm/kvm-s390.c +26 -0
index 801a622691b6..616d1db7c0d9 100644
@@ -645,6 +645,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		if (hpage && !(kvm && kvm_is_ucontrol(kvm)))
 			r = 1;
 		break;
+	case KVM_CAP_S390_HPAGE_2G:
+		r = 0;
+		if (hpage_2g && !(kvm && kvm_is_ucontrol(kvm)))
+			r = 1;
+		break;
 	case KVM_CAP_S390_MEM_OP:
 		r = MEM_OP_MAX_SIZE;
 		break;
@@ -901,6 +906,27 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 		VM_EVENT(kvm, 3, "ENABLE: CAP_S390_HPAGE %s",
 			 r ? "(not available)" : "(success)");
 		break;
+	case KVM_CAP_S390_HPAGE_2G:
+		mutex_lock(&kvm->lock);
+		if (kvm->created_vcpus) {
+			r = -EBUSY;
+		} else if (!hpage_2g || kvm->arch.use_cmma || kvm_is_ucontrol(kvm)) {
+			r = -EINVAL;
+		} else {
+			r = 0;
+			set_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &kvm->arch.gmap->flags);
+			/*
+			 * We might have to create fake 4k page
+			 * tables. To avoid that the hardware works on
+			 * stale PGSTEs, we emulate these instructions.
+			 */
+			kvm->arch.use_skf = 0;
+			kvm->arch.use_pfmfi = 0;
+		}
+		mutex_unlock(&kvm->lock);
+		VM_EVENT(kvm, 3, "ENABLE: CAP_S390_HPAGE_2G %s",
+			 r ? "(not available)" : "(success)");
+		break;
 	case KVM_CAP_S390_USER_STSI:
 		VM_EVENT(kvm, 3, "%s", "ENABLE: CAP_S390_USER_STSI");
 		kvm->arch.user_stsi = 1;
arch/s390/kvm/pv.c +1 -0
index 4b865e75351c..0f818149e644 100644
@@ -741,6 +741,7 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc)
 	uvcb.flags.ap_instr_intr = kvm->arch.model.uv_feat_guest.ap_intr;
 
 	clear_bit(GMAP_FLAG_ALLOW_HPAGE_1M, &kvm->arch.gmap->flags);
+	clear_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &kvm->arch.gmap->flags);
 	gmap_split_huge_pages(kvm->arch.gmap);
 
 	cc = uv_call_sched(0, (u64)&uvcb);
include/uapi/linux/kvm.h +1 -0
index 6c8afa2047bf..419011097fa8 100644
@@ -996,6 +996,7 @@ struct kvm_enable_cap {
 #define KVM_CAP_S390_USER_OPEREXEC 246
 #define KVM_CAP_S390_KEYOP 247
 #define KVM_CAP_S390_VSIE_ESAMODE 248
+#define KVM_CAP_S390_HPAGE_2G 249
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;
Signature · 1 line
2.54.0
#4 lore raw

Re: [PATCH v1 2/3] KVM: s390: Add capability to support 2G hugepages

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

New issues:
- [Low] The new capability KVM_CAP_S390_HPAGE_2G is completely undocumented in the KVM API documentation.
- [Low] The new KVM capability is missing from the tools/ UAPI header.

Pre-existing issues:
- [High] Userspace can bypass the mutual exclusivity between CMMA and hugepages by enabling CMMA after hugepages have been enabled.
- [High] Userspace can bypass the architectural restriction that Protected VMs (PV) must use 4K pages by enabling hugepages after the VM is already in protected mode. --

commit 43482c954e0933d74bf56361955e125830f3a3fe
Author: Claudio Imbrenda <[email protected]>

KVM: s390: Add capability to support 2G hugepages

This patch adds the KVM_CAP_S390_HPAGE_2G capability to allow 2G hugepages to back the guest. It enables the GMAP_FLAG_ALLOW_HPAGE_2G flag for the guest gmap and propagates it appropriately.

Quoted 4 lines
arch/s390/kvm/kvm-s390.c +0 -0
index 801a622691b6a..616d1db7c0d9e 100644

[ ... ]

arch/s390/kvm/kvm-s390.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 913

Quoted 9 lines
arch/s390/kvm/kvm-s390.c +5 -0
@@ -901,6 +906,27 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 		VM_EVENT(kvm, 3, "ENABLE: CAP_S390_HPAGE %s",
 			 r ? "(not available)" : "(success)");
 		break;
+	case KVM_CAP_S390_HPAGE_2G:
+		mutex_lock(&kvm->lock);
+		if (kvm->created_vcpus) {
+			r = -EBUSY;
+		} else if (!hpage_2g || kvm->arch.use_cmma || kvm_is_ucontrol(kvm)) {

[Severity: High]
This is a pre-existing issue, but while this capability enablement checks if VCPUs are created, could userspace bypass the Protected VM restriction by enabling PV first and then enabling hugepages before creating VCPUs?

In kvm_s390_pv_init_vm(), the flags are cleared:

    clear_bit(GMAP_FLAG_ALLOW_HPAGE_1M, &kvm->arch.gmap->flags);
    clear_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &kvm->arch.gmap->flags);

Since !kvm_s390_pv_is_protected(kvm) is not checked during the capability enablement, does this allow KVM to map unsupported 1M/2G hugepages into a Protected VM's address space?

[Severity: High]
This is also a pre-existing issue, but while the hugepage enablement checks kvm->arch.use_cmma to enforce mutual exclusivity, does the CMMA enablement handler lack the reciprocal check?

In kvm_s390_set_mem_control():

    kvm->arch.use_cmma = 1;
    /* Not compatible with cmma. */
    kvm->arch.use_pfmfi = 0;
    ret = 0;

Could userspace enable hugepages first, and then enable CMMA, causing CMMA dirty tracking operations to silently skip huge pages since they are not handled by the dat_walk_ops .pte_entry?

Quoted 4 lines
arch/s390/kvm/kvm-s390.c +4 -0
+			r = -EINVAL;
+		} else {
+			r = 0;
+			set_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &kvm->arch.gmap->flags);

[ ... ]

Quoted 4 lines
include/uapi/linux/kvm.h +0 -0
index 6c8afa2047bf3..419011097fa8e 100644

[ ... ]

include/uapi/linux/kvm.h struct kvm_enable_cap { · line 999

Quoted 5 lines
include/uapi/linux/kvm.h +1 -0
@@ -996,6 +996,7 @@ struct kvm_enable_cap {
 #define KVM_CAP_S390_USER_OPEREXEC 246
 #define KVM_CAP_S390_KEYOP 247
 #define KVM_CAP_S390_VSIE_ESAMODE 248
+#define KVM_CAP_S390_HPAGE_2G 249

[Severity: Low]
Should this new capability be documented in Documentation/virt/kvm/api.rst to explain its usage, side effects, and mutual exclusivity with CMMA and ucontrol VMs?

[Severity: Low]
Does this capability addition also need to be synchronized to tools/include/uapi/linux/kvm.h so that userspace tools and selftests compiled from the kernel tree have the definition available?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Reviewed-by on v1 2/3
#5 lore raw

Re: [PATCH v1 2/3] KVM: s390: Add capability to support 2G hugepages

In reply to Claudio Imbrenda
Quoted 7 lines · Claudio Imbrenda

Add KVM_CAP_S390_HPAGE_2G to signal to userspace that 2G hugepages may be used to back the guest; restrictions apply similar to 1M hugepages.

Enable the (for now still ignored) GMAP_FLAG_ALLOW_HPAGE_2G flag for the guest gmap, and propagate / disable it as necessary.

author maintainer
#6 lore raw

PATCH 3/3 KVM: s390: Allow for 2G hugepages

In reply to Claudio Imbrenda

Change gmap_2g_allowed() to perform the necessary checks to allow for 2G hugepages to be used, instead of returning false. The GMAP_FLAG_ALLOW_HPAGE_2G gmap flag is now taken into account.

Also add appropriate kerneldoc comments.

---
 arch/s390/kvm/gmap.c | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)
arch/s390/kvm/gmap.c +18 -1
index c9e348149ba1..c5a8132eaa22 100644
@@ -635,10 +635,27 @@ int gmap_try_fixup_minor(struct gmap *gmap, struct guest_fault *fault)
 	return rc;
 }
 
+/**
+ * gmap_2g_allowed() - Check whether a 2G hugepage is allowed.
+ * @gmap: The gmap of the guest.
+ * @f: Describes the fault that is being resolved.
+ * @slot: The memslot the faulting address belongs to.
+ *
+ * The function checks whether the GMAP_FLAG_ALLOW_HPAGE_2G flag is set for
+ * @gmap, whether the offset of the address in the 2G virtual frame is the
+ * same as the offset in the physical 2G frame, and finally whether the whole
+ * 2G page would fit in the given memslot.
+ *
+ * Return: true if a 2G hugepage is allowed to back the faulting address, false
+ *         otherwise.
+ */
 static inline bool gmap_2g_allowed(struct gmap *gmap, struct guest_fault *f,
 				   struct kvm_memory_slot *slot)
 {
-	return false;
+	return test_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &gmap->flags) &&
+	       !((f->gfn ^ f->pfn) & ~_REGION3_FR_MASK) &&
+	       slot->base_gfn <= ALIGN_DOWN(f->gfn, _PAGES_PER_REGION3) &&
+	       slot->base_gfn + slot->npages >= ALIGN(f->gfn + 1, _PAGES_PER_REGION3);
 }
 
 /**
Signature · 1 line
2.54.0
#7 lore raw

Re: [PATCH v1 3/3] KVM: s390: Allow for 2G hugepages

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [Critical] Enabling 2G hugepages exposes a hardcoded _SEGMENT_MASK in _crste_fc1, injecting invalid entries into hardware page tables.

Pre-existing issues:
- [High] Concurrent page faults can insert 1M or 2G huge pages into a PV guest gmap while KVM_PV_ENABLE runs, because gmap_split_huge_pages and gmap_link both acquire mmu_lock in read mode without sufficient serialization. --

commit 8d8980fda5d60990dd59ce7e857350fb4c8cedbd
Author: Claudio Imbrenda <[email protected]>

KVM: s390: Allow for 2G hugepages

Change gmap_2g_allowed() to perform the necessary checks to allow for 2G hugepages to be used, instead of unconditionally returning false. The GMAP_FLAG_ALLOW_HPAGE_2G flag is now taken into account when evaluating whether a 2G hugepage can back the faulting address.

Quoted 4 lines
arch/s390/kvm/gmap.c +0 -0
index c9e348149ba12..c5a8132eaa22b 100644

[ ... ]

Quoted 13 lines
arch/s390/kvm/gmap.c +9 -0
@@ -635,10 +635,27 @@ int gmap_try_fixup_minor(struct gmap *gmap, struct guest_fault *fault)
 	return rc;
 }
 
+/**
+ * gmap_2g_allowed() - Check whether a 2G hugepage is allowed.
+ * @gmap: The gmap of the guest.
+ * @f: Describes the fault that is being resolved.
+ * @slot: The memslot the faulting address belongs to.
+ *
+ * The function checks whether the GMAP_FLAG_ALLOW_HPAGE_2G flag is set for
+ * @gmap, whether the offset of the address in the 2G virtual frame is the
+ * same as the offset in the physical 2G frame, and finally whether the whole

arch/s390/kvm/gmap.c int gmap_try_fixup_minor(struct gmap *gmap, struct guest_fault *fault) · line 655

Quoted 10 lines
arch/s390/kvm/gmap.c +6 -1
+ * 2G page would fit in the given memslot.
+ *
+ * Return: true if a 2G hugepage is allowed to back the faulting address, false
+ *         otherwise.
+ */
 static inline bool gmap_2g_allowed(struct gmap *gmap, struct guest_fault *f,
 				   struct kvm_memory_slot *slot)
 {
-	return false;
+	return test_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &gmap->flags) &&

[Severity: Critical]
Enabling 2G hugepages here exposes a hardcoded _SEGMENT_MASK in _crste_fc1(). When 2G hugepages proceed to _gmap_link(), it calls _crste_fc1() with an unaligned f->pfn and a Region-3 table type.

arch/s390/kvm/dat.h:_crste_fc1() {
    union crste res = { .val = PFN_PHYS(pfn) & _SEGMENT_MASK };
    ...
}

The constructor hardcodes _SEGMENT_MASK, which fails to zero out bits 20-30 for Region-3 hugepages. For a Region-3 large page entry, the hardware requires these bits to be strictly zero. Writing the unaligned 2MB-to-2GB offset into these reserved bits causes the hardware MMU to trigger a Translation Specification Exception when it walks the page table.

Should _crste_fc1() be updated to use _REGION3_MASK for Region-3 entries to prevent guest crashes?

[Severity: High]
This isn't a bug introduced by this patch, but adding 2G hugepages to this path exacerbates a preexisting race with PV initialization.

Concurrent page faults can insert huge pages into a PV guest gmap while kvm_s390_pv_init_vm() runs. The initialization code clears the flags and calls gmap_split_huge_pages():

arch/s390/kvm/pv.c:kvm_s390_pv_init_vm() {
    ...
    clear_bit(GMAP_FLAG_ALLOW_HPAGE_1M, &kvm->arch.gmap->flags);
    clear_bit(GMAP_FLAG_ALLOW_HPAGE_2G, &kvm->arch.gmap->flags);
    gmap_split_huge_pages(kvm->arch.gmap);
    ...
}

However, gmap_split_huge_pages() only takes a read lock on kvm->mmu_lock. If a vCPU thread evaluates test_bit() in gmap_2g_allowed() to true before the flag is cleared, and is then preempted, the splitting function can walk past the faulting GFN.

When the vCPU thread resumes, it can atomically insert the 2G huge page into the gmap via _gmap_link(), as that also only holds mmu_lock in read mode.

Does this race allow a PV guest to map a hugepage and break memory isolation invariants?

Quoted 4 lines
arch/s390/kvm/gmap.c +3 -0
+	       !((f->gfn ^ f->pfn) & ~_REGION3_FR_MASK) &&
+	       slot->base_gfn <= ALIGN_DOWN(f->gfn, _PAGES_PER_REGION3) &&
+	       slot->base_gfn + slot->npages >= ALIGN(f->gfn + 1, _PAGES_PER_REGION3);
 }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Reviewed-by on v1 3/3
#8 lore raw

Re: [PATCH v1 3/3] KVM: s390: Allow for 2G hugepages

In reply to Claudio Imbrenda
Quoted 7 lines · Claudio Imbrenda

Change gmap_2g_allowed() to perform the necessary checks to allow for 2G hugepages to be used, instead of returning false. The GMAP_FLAG_ALLOW_HPAGE_2G gmap flag is now taken into account.

Also add appropriate kerneldoc comments.