Skip to contentSkip to conversation

powerpc/XIVE: SVM: share the event-queue page with the Hypervisor.

PATCH Superseded Ram Pai 6 participants 10 messages Mar 26, 2020 – Jun 9, 2020 PowerPC

lore
Share & tools
lore

Ram Pai

author
#0 lore raw

PATCH v2 1/1 powerpc/XIVE: SVM: share the event-queue page with the Hypervisor.

XIVE interrupt controller use an Event Queue (EQ) to enqueue event notifications when an exception occurs. The EQ is a single memory page provided by the O/S defining a circular buffer, one per server and priority couple.

On baremetal, the EQ page is configured with an OPAL call. On pseries, an extra hop is necessary and the guest OS uses the hcall H_INT_SET_QUEUE_CONFIG to configure the XIVE interrupt controller.

The XIVE controller being Hypervisor privileged, it will not be allowed to enqueue event notifications for a Secure VM unless the EQ pages are shared by the Secure VM.

Hypervisor/Ultravisor still requires support for the TIMA and ESB page fault handlers. Until this is complete, QEMU can use the emulated XIVE device for Secure VMs, option "kernel_irqchip=off" on the QEMU pseries machine.

Cc Michael Ellerman <[email protected]>
Cc Thiago Jung Bauermann <[email protected]>
Cc Michael Anderson <[email protected]>
Cc Sukadev Bhattiprolu <[email protected]>
Cc Alexey Kardashevskiy <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Cedric Le Goater <[email protected]>
Cc David Gibson <[email protected]>

v2: better description of the patch from Cedric.

---
 arch/powerpc/sysdev/xive/spapr.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/powerpc/sysdev/xive/spapr.c +7 -0
index 55dc61c..608b52f 100644
@@ -26,6 +26,8 @@
 #include <asm/xive.h>
 #include <asm/xive-regs.h>
 #include <asm/hvcall.h>
+#include <asm/svm.h>
+#include <asm/ultravisor.h>
 
 #include "xive-internal.h"
 
@@ -501,6 +503,9 @@ static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio,
 		rc = -EIO;
 	} else {
 		q->qpage = qpage;
+		if (is_secure_guest())
+			uv_share_page(PHYS_PFN(qpage_phys),
+					1 << xive_alloc_order(order));
 	}
 fail:
 	return rc;
@@ -534,6 +539,8 @@ static void xive_spapr_cleanup_queue(unsigned int cpu, struct xive_cpu *xc,
 		       hw_cpu, prio);
 
 	alloc_order = xive_alloc_order(xive_queue_shift);
+	if (is_secure_guest())
+		uv_unshare_page(PHYS_PFN(__pa(q->qpage)), 1 << alloc_order);
 	free_pages((unsigned long)q->qpage, alloc_order);
 	q->qpage = NULL;
 }
Signature · 1 line
1.8.3.1
Reviewed-by on v2 1/1
#1 lore raw
Quoted 1 line · Ram Pai

XIVE interrupt controller use an Event Queue (EQ) to enqueue event

The XIVE interrupt controller uses ... (my bad)

Quoted 17 lines

notifications when an exception occurs. The EQ is a single memory page provided by the O/S defining a circular buffer, one per server and priority couple.

On baremetal, the EQ page is configured with an OPAL call. On pseries, an extra hop is necessary and the guest OS uses the hcall H_INT_SET_QUEUE_CONFIG to configure the XIVE interrupt controller.

The XIVE controller being Hypervisor privileged, it will not be allowed to enqueue event notifications for a Secure VM unless the EQ pages are shared by the Secure VM.

Hypervisor/Ultravisor still requires support for the TIMA and ESB page fault handlers. Until this is complete, QEMU can use the emulated XIVE device for Secure VMs, option "kernel_irqchip=off" on the QEMU pseries machine.

Quoted 10 lines
Cc Michael Ellerman <[email protected]>
Cc Thiago Jung Bauermann <[email protected]>
Cc Michael Anderson <[email protected]>
Cc Sukadev Bhattiprolu <[email protected]>
Cc Alexey Kardashevskiy <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Cedric Le Goater <[email protected]>

[email protected] is insecure. Please use [email protected].

Quoted 2 lines
Cc David Gibson <[email protected]>

Thanks,

C.

Quoted 39 lines

v2: better description of the patch from Cedric.

---
 arch/powerpc/sysdev/xive/spapr.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/powerpc/sysdev/xive/spapr.c +7 -0
index 55dc61c..608b52f 100644
@@ -26,6 +26,8 @@
 #include <asm/xive.h>
 #include <asm/xive-regs.h>
 #include <asm/hvcall.h>
+#include <asm/svm.h>
+#include <asm/ultravisor.h>
 
 #include "xive-internal.h"
 
@@ -501,6 +503,9 @@ static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio,
 		rc = -EIO;
 	} else {
 		q->qpage = qpage;
+		if (is_secure_guest())
+			uv_share_page(PHYS_PFN(qpage_phys),
+					1 << xive_alloc_order(order));
 	}
 fail:
 	return rc;
@@ -534,6 +539,8 @@ static void xive_spapr_cleanup_queue(unsigned int cpu, struct xive_cpu *xc,
 		       hw_cpu, prio);
 
 	alloc_order = xive_alloc_order(xive_queue_shift);
+	if (is_secure_guest())
+		uv_unshare_page(PHYS_PFN(__pa(q->qpage)), 1 << alloc_order);
 	free_pages((unsigned long)q->qpage, alloc_order);
 	q->qpage = NULL;
 }
Reviewed-by on v2 1/1
#2 lore raw
In reply to Ram Pai
Quoted 31 lines · On Thu, 26 Mar 2020 01:38:47 -0700 Ram Pai <[email protected]> wrote:

XIVE interrupt controller use an Event Queue (EQ) to enqueue event notifications when an exception occurs. The EQ is a single memory page provided by the O/S defining a circular buffer, one per server and priority couple.

On baremetal, the EQ page is configured with an OPAL call. On pseries, an extra hop is necessary and the guest OS uses the hcall H_INT_SET_QUEUE_CONFIG to configure the XIVE interrupt controller.

The XIVE controller being Hypervisor privileged, it will not be allowed to enqueue event notifications for a Secure VM unless the EQ pages are shared by the Secure VM.

Hypervisor/Ultravisor still requires support for the TIMA and ESB page fault handlers. Until this is complete, QEMU can use the emulated XIVE device for Secure VMs, option "kernel_irqchip=off" on the QEMU pseries machine.

Cc Michael Ellerman <[email protected]>
Cc Thiago Jung Bauermann <[email protected]>
Cc Michael Anderson <[email protected]>
Cc Sukadev Bhattiprolu <[email protected]>
Cc Alexey Kardashevskiy <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Cedric Le Goater <[email protected]>
Cc David Gibson <[email protected]>
Quoted 2 lines

v2: better description of the patch from Cedric.
---

Quoted 35 lines
 arch/powerpc/sysdev/xive/spapr.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/powerpc/sysdev/xive/spapr.c +7 -0
index 55dc61c..608b52f 100644
@@ -26,6 +26,8 @@
 #include <asm/xive.h>
 #include <asm/xive-regs.h>
 #include <asm/hvcall.h>
+#include <asm/svm.h>
+#include <asm/ultravisor.h>
 
 #include "xive-internal.h"
 
@@ -501,6 +503,9 @@ static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio,
 		rc = -EIO;
 	} else {
 		q->qpage = qpage;
+		if (is_secure_guest())
+			uv_share_page(PHYS_PFN(qpage_phys),
+					1 << xive_alloc_order(order));
 	}
 fail:
 	return rc;
@@ -534,6 +539,8 @@ static void xive_spapr_cleanup_queue(unsigned int cpu, struct xive_cpu *xc,
 		       hw_cpu, prio);
 
 	alloc_order = xive_alloc_order(xive_queue_shift);
+	if (is_secure_guest())
+		uv_unshare_page(PHYS_PFN(__pa(q->qpage)), 1 << alloc_order);
 	free_pages((unsigned long)q->qpage, alloc_order);
 	q->qpage = NULL;
 }
#3 lore raw
In reply to Ram Pai

Hi Ram,

Quoted 14 lines · Ram Pai
arch/powerpc/sysdev/xive/spapr.c +2 -0
index 55dc61c..608b52f 100644
@@ -26,6 +26,8 @@
 #include <asm/xive.h>
 #include <asm/xive-regs.h>
 #include <asm/hvcall.h>
+#include <asm/svm.h>
+#include <asm/ultravisor.h>
 
 #include "xive-internal.h"
 

arch/powerpc/sysdev/xive/spapr.c static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio, · line 508

Quoted 6 lines
arch/powerpc/sysdev/xive/spapr.c +3 -0
@@ -501,6 +503,9 @@ static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio,
 		rc = -EIO;
 	} else {
 		q->qpage = qpage;
+		if (is_secure_guest())
+			uv_share_page(PHYS_PFN(qpage_phys),
+					1 << xive_alloc_order(order));

If I understand this correctly, you're passing the number of bytes of the queue to uv_share_page(), but that ultracall expects the number of pages to be shared.

arch/powerpc/sysdev/xive/spapr.c static void xive_spapr_cleanup_queue(unsigned int cpu, struct xive_cpu *xc, · line 546

Quoted 12 lines
arch/powerpc/sysdev/xive/spapr.c +2 -0
 	}
 fail:
 	return rc;
@@ -534,6 +539,8 @@ static void xive_spapr_cleanup_queue(unsigned int cpu, struct xive_cpu *xc,
 		       hw_cpu, prio);
 
 	alloc_order = xive_alloc_order(xive_queue_shift);
+	if (is_secure_guest())
+		uv_unshare_page(PHYS_PFN(__pa(q->qpage)), 1 << alloc_order);
 	free_pages((unsigned long)q->qpage, alloc_order);
 	q->qpage = NULL;
 }

Same problem here.

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
author
#4 lore raw
Quoted 18 lines · Thiago Jung Bauermann

Hi Ram,

Quoted 13 lines · Ram Pai
arch/powerpc/sysdev/xive/spapr.c +2 -0
index 55dc61c..608b52f 100644
@@ -26,6 +26,8 @@
 #include <asm/xive.h>
 #include <asm/xive-regs.h>
 #include <asm/hvcall.h>
+#include <asm/svm.h>
+#include <asm/ultravisor.h>
 
 #include "xive-internal.h"
 
Quoted 11 lines

arch/powerpc/sysdev/xive/spapr.c static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio, · line 508

Quoted 7 lines
arch/powerpc/sysdev/xive/spapr.c +3 -0
@@ -501,6 +503,9 @@ static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio,
 		rc = -EIO;
 	} else {
 		q->qpage = qpage;
+		if (is_secure_guest())
+			uv_share_page(PHYS_PFN(qpage_phys),
+					1 << xive_alloc_order(order));

If I understand this correctly, you're passing the number of bytes of the queue to uv_share_page(), but that ultracall expects the number of pages to be shared.

static inline u32 xive_alloc_order(u32 queue_shift)

{
        return (queue_shift > PAGE_SHIFT) ? (queue_shift - PAGE_SHIFT) : 0;
}

xive_alloc_order(order) returns the order of PAGE_SIZE pages. Hence the value passed to uv_shared_pages is the number of pages, and not the number of bytes.

BTW: I did verify through testing that it was indeed passing 1 page to the uv_share_page().

RP

#5 lore raw
Quoted 37 lines · Ram Pai
Quoted 18 lines · Thiago Jung Bauermann

Hi Ram,

Quoted 13 lines · Ram Pai
arch/powerpc/sysdev/xive/spapr.c +2 -0
index 55dc61c..608b52f 100644
@@ -26,6 +26,8 @@
 #include <asm/xive.h>
 #include <asm/xive-regs.h>
 #include <asm/hvcall.h>
+#include <asm/svm.h>
+#include <asm/ultravisor.h>
 
 #include "xive-internal.h"
 
Quoted 11 lines

arch/powerpc/sysdev/xive/spapr.c static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio, · line 508

Quoted 7 lines
arch/powerpc/sysdev/xive/spapr.c +3 -0
@@ -501,6 +503,9 @@ static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio,
 		rc = -EIO;
 	} else {
 		q->qpage = qpage;
+		if (is_secure_guest())
+			uv_share_page(PHYS_PFN(qpage_phys),
+					1 << xive_alloc_order(order));

If I understand this correctly, you're passing the number of bytes of the queue to uv_share_page(), but that ultracall expects the number of pages to be shared.

static inline u32 xive_alloc_order(u32 queue_shift)

{
        return (queue_shift > PAGE_SHIFT) ? (queue_shift - PAGE_SHIFT) : 0;
}
Quoted 6 lines

xive_alloc_order(order) returns the order of PAGE_SIZE pages. Hence the value passed to uv_shared_pages is the number of pages, and not the number of bytes.

BTW: I did verify through testing that it was indeed passing 1 page to the uv_share_page().

Ah, my mistake. I misunderstood the code. Sorry for the noise and thanks for clarifying.

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
author
#6 lore raw

PATCH v3 1/1 powerpc/XIVE: SVM: share the event-queue page with the Hypervisor.

In reply to Ram Pai

From 10ea2eaf492ca3f22f67a5a63a2b7865e45299ad Mon Sep 17 00:00:00 2001
From: Ram Pai <[email protected]>
Date: Mon, 24 Feb 2020 01:09:48 -0500
Subject: [PATCH v3] powerpc/XIVE: SVM: share the event-queue page with the
Hypervisor.

XIVE interrupt controller uses an Event Queue (EQ) to enqueue event notifications when an exception occurs. The EQ is a single memory page provided by the O/S defining a circular buffer, one per server and priority couple.

On baremetal, the EQ page is configured with an OPAL call. On pseries, an extra hop is necessary and the guest OS uses the hcall H_INT_SET_QUEUE_CONFIG to configure the XIVE interrupt controller.

The XIVE controller being Hypervisor privileged, it will not be allowed to enqueue event notifications for a Secure VM unless the EQ pages are shared by the Secure VM.

Hypervisor/Ultravisor still requires support for the TIMA and ESB page fault handlers. Until this is complete, QEMU can use the emulated XIVE device for Secure VMs, option "kernel_irqchip=off" on the QEMU pseries machine.

Cc Michael Ellerman <[email protected]>
Cc Thiago Jung Bauermann <[email protected]>
Cc Michael Anderson <[email protected]>
Cc Sukadev Bhattiprolu <[email protected]>
Cc Alexey Kardashevskiy <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc David Gibson <[email protected]>

v3: fix a minor semantics in description.
and added reviewed-by from Cedric and Greg.
v2: better description of the patch from Cedric.

---
 arch/powerpc/sysdev/xive/spapr.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/powerpc/sysdev/xive/spapr.c +7 -0
index 55dc61c..608b52f 100644
@@ -26,6 +26,8 @@
 #include <asm/xive.h>
 #include <asm/xive-regs.h>
 #include <asm/hvcall.h>
+#include <asm/svm.h>
+#include <asm/ultravisor.h>
 
 #include "xive-internal.h"
 
@@ -501,6 +503,9 @@ static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio,
 		rc = -EIO;
 	} else {
 		q->qpage = qpage;
+		if (is_secure_guest())
+			uv_share_page(PHYS_PFN(qpage_phys),
+					1 << xive_alloc_order(order));
 	}
 fail:
 	return rc;
@@ -534,6 +539,8 @@ static void xive_spapr_cleanup_queue(unsigned int cpu, struct xive_cpu *xc,
 		       hw_cpu, prio);
 
 	alloc_order = xive_alloc_order(xive_queue_shift);
+	if (is_secure_guest())
+		uv_unshare_page(PHYS_PFN(__pa(q->qpage)), 1 << alloc_order);
 	free_pages((unsigned long)q->qpage, alloc_order);
 	q->qpage = NULL;
 }
Signature · 1 line
1.8.3.1
author
#7 lore raw

PATCH v3 1/1 powerpc/XIVE: SVM: share the event-queue page with the Hypervisor.

XIVE interrupt controller uses an Event Queue (EQ) to enqueue event notifications when an exception occurs. The EQ is a single memory page provided by the O/S defining a circular buffer, one per server and priority couple.

On baremetal, the EQ page is configured with an OPAL call. On pseries, an extra hop is necessary and the guest OS uses the hcall H_INT_SET_QUEUE_CONFIG to configure the XIVE interrupt controller.

The XIVE controller being Hypervisor privileged, it will not be allowed to enqueue event notifications for a Secure VM unless the EQ pages are shared by the Secure VM.

Hypervisor/Ultravisor still requires support for the TIMA and ESB page fault handlers. Until this is complete, QEMU can use the emulated XIVE device for Secure VMs, option "kernel_irqchip=off" on the QEMU pseries machine.

Cc Michael Ellerman <[email protected]>
Cc Thiago Jung Bauermann <[email protected]>
Cc Michael Anderson <[email protected]>
Cc Sukadev Bhattiprolu <[email protected]>
Cc Alexey Kardashevskiy <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc David Gibson <[email protected]>

v3: fix a minor semantics in description.
and added reviewed-by from Cedric and Greg.
v2: better description of the patch from Cedric.

---
 arch/powerpc/sysdev/xive/spapr.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/powerpc/sysdev/xive/spapr.c +7 -0
index 55dc61c..608b52f 100644
@@ -26,6 +26,8 @@
 #include <asm/xive.h>
 #include <asm/xive-regs.h>
 #include <asm/hvcall.h>
+#include <asm/svm.h>
+#include <asm/ultravisor.h>
 
 #include "xive-internal.h"
 
@@ -501,6 +503,9 @@ static int xive_spapr_configure_queue(u32 target, struct xive_q *q, u8 prio,
 		rc = -EIO;
 	} else {
 		q->qpage = qpage;
+		if (is_secure_guest())
+			uv_share_page(PHYS_PFN(qpage_phys),
+					1 << xive_alloc_order(order));
 	}
 fail:
 	return rc;
@@ -534,6 +539,8 @@ static void xive_spapr_cleanup_queue(unsigned int cpu, struct xive_cpu *xc,
 		       hw_cpu, prio);
 
 	alloc_order = xive_alloc_order(xive_queue_shift);
+	if (is_secure_guest())
+		uv_unshare_page(PHYS_PFN(__pa(q->qpage)), 1 << alloc_order);
 	free_pages((unsigned long)q->qpage, alloc_order);
 	q->qpage = NULL;
 }
Signature · 1 line
1.8.3.1
#8 lore raw
Quoted 31 lines · Ram Pai

XIVE interrupt controller uses an Event Queue (EQ) to enqueue event notifications when an exception occurs. The EQ is a single memory page provided by the O/S defining a circular buffer, one per server and priority couple.

On baremetal, the EQ page is configured with an OPAL call. On pseries, an extra hop is necessary and the guest OS uses the hcall H_INT_SET_QUEUE_CONFIG to configure the XIVE interrupt controller.

The XIVE controller being Hypervisor privileged, it will not be allowed to enqueue event notifications for a Secure VM unless the EQ pages are shared by the Secure VM.

Hypervisor/Ultravisor still requires support for the TIMA and ESB page fault handlers. Until this is complete, QEMU can use the emulated XIVE device for Secure VMs, option "kernel_irqchip=off" on the QEMU pseries machine.

Cc Michael Ellerman <[email protected]>
Cc Thiago Jung Bauermann <[email protected]>
Cc Michael Anderson <[email protected]>
Cc Sukadev Bhattiprolu <[email protected]>
Cc Alexey Kardashevskiy <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc David Gibson <[email protected]>
Quoted 4 lines

v3: fix a minor semantics in description.
and added reviewed-by from Cedric and Greg.
v2: better description of the patch from Cedric.
---

Please put the change history after the '---' break in future please, I had to fix this up manually.

cheers

#9 lore raw
In reply to Ram Pai
Quoted 6 lines · Ram Pai
Quoted 1 line

From 10ea2eaf492ca3f22f67a5a63a2b7865e45299ad Mon Sep 17 00:00:00 2001

From: Ram Pai <[email protected]>
Date: Mon, 24 Feb 2020 01:09:48 -0500
Subject: [PATCH v3] powerpc/XIVE: SVM: share the event-queue page with the
Hypervisor.

Quoted 6 lines

XIVE interrupt controller uses an Event Queue (EQ) to enqueue event notifications when an exception occurs. The EQ is a single memory page provided by the O/S defining a circular buffer, one per server and priority couple.

[...]

Applied to powerpc/next.

[1/1] powerpc/xive: Share the event-queue page with the Hypervisor.

      https://git.kernel.org/powerpc/c/094235222d41d68d35de18170058d94a96a82628

cheers