s390x/vfio-ccw: Channel Path Handling [KVM]

PATCH Applied Eric Farman 2 participants 17 messages May 5, 2020 – May 18, 2020

Subsystems (6)

Documentation KVM s390 VFIO driver +2 more subsystems

lore

Share & tools

lore

Eric Farman

author maintainer

2020-05-05 12:27 UTC #0 lore raw

v4 cover s390x/vfio-ccw: Channel Path Handling [KVM]

Here is a new pass at the channel-path handling code for vfio-ccw. Changes from previous versions are recorded in git notes for each patch. Patches 5 through 7 got swizzled a little bit, in order to better compartmentalize the code they define. Basically, the IRQ definitions were moved from patch 7 to 5, and then patch 6 was placed ahead of patch 5.

I have put Conny's r-b's on patches 1, 3, 4, (new) 5, and 8, and believe I have addressed all comments from v3, with two exceptions:

Quoted 2 lines

I'm wondering if we should make this [vfio_ccw_schib_region_{write,release}] callback optional (not in this patch).

I have that implemented on top of this series, and will send later as part of a larger cleanup series.

Quoted 3 lines

One thing though that keeps coming up: do we need any kind of serialization? Can there be any confusion from concurrent reads from userspace, or are we sure that we always provide consistent data?

I _think_ this is in good shape, though as suggested another set of eyeballs would be nice. There is still a problem on the main interrupt/FSM path, which I'm not attempting to address here.

With this code plus the corresponding QEMU series (posted momentarily) applied I am able to configure off/on a CHPID (for example, by issuing "chchp -c 0/1 xx" on the host), and the guest is able to see both the events and reflect the updated path masks in its structures.

v3: https://lore.kernel.org/kvm/[email protected]/
v2: https://lore.kernel.org/kvm/[email protected]/
v1: https://lore.kernel.org/kvm/[email protected]/

Eric Farman (3):
vfio-ccw: Refactor the unregister of the async regions
vfio-ccw: Refactor IRQ handlers
vfio-ccw: Add trace for CRW event

Farhan Ali (5):
vfio-ccw: Introduce new helper functions to free/destroy regions
vfio-ccw: Register a chp_event callback for vfio-ccw
vfio-ccw: Introduce a new schib region
vfio-ccw: Introduce a new CRW region
vfio-ccw: Wire up the CRW irq and CRW region

 Documentation/s390/vfio-ccw.rst     |  38 ++++++-
 drivers/s390/cio/Makefile           |   2 +-
 drivers/s390/cio/vfio_ccw_chp.c     | 148 +++++++++++++++++++++++++
 drivers/s390/cio/vfio_ccw_drv.c     | 165 ++++++++++++++++++++++++++--
 drivers/s390/cio/vfio_ccw_ops.c     |  65 ++++++++---
 drivers/s390/cio/vfio_ccw_private.h |  16 +++
 drivers/s390/cio/vfio_ccw_trace.c   |   1 +
 drivers/s390/cio/vfio_ccw_trace.h   |  30 +++++
 include/uapi/linux/vfio.h           |   3 +
 include/uapi/linux/vfio_ccw.h       |  18 +++
 10 files changed, 458 insertions(+), 28 deletions(-)
 create mode 100644 drivers/s390/cio/vfio_ccw_chp.c

Signature · 1 line

2.17.1

Eric Farman

author maintainer

2020-05-05 12:27 UTC #1 lore raw

PATCH v4 1/8 vfio-ccw: Introduce new helper functions to free/destroy regions

From: Farhan Ali <[email protected]>

Consolidate some of the cleanup code for the regions, so that as more are added we reduce code duplication.

Signed-off-by Farhan Ali <[email protected]>

Signed-off-by Eric Farman <[email protected]>

Reviewed-by Cornelia Huck <[email protected]>

---

Notes:
v1->v2:
- Add Conny's r-b

v0->v1: [EF]
- Commit message

 drivers/s390/cio/vfio_ccw_drv.c | 28 ++++++++++++++++++----------
 1 file changed, 18 insertions(+), 10 deletions(-)

drivers/s390/cio/vfio_ccw_drv.c +18 ~~-10~~

index 339a6bc0339b..8715c1c2f1e1 100644
@@ -116,6 +116,14 @@ static void vfio_ccw_sch_irq(struct subchannel *sch)
 	vfio_ccw_fsm_event(private, VFIO_CCW_EVENT_INTERRUPT);
 }
 
+static void vfio_ccw_free_regions(struct vfio_ccw_private *private)
+{
+	if (private->cmd_region)
+		kmem_cache_free(vfio_ccw_cmd_region, private->cmd_region);
+	if (private->io_region)
+		kmem_cache_free(vfio_ccw_io_region, private->io_region);
+}
+
 static int vfio_ccw_sch_probe(struct subchannel *sch)
 {
 	struct pmcw *pmcw = &sch->schib.pmcw;

@@ -181,10 +189,7 @@ static int vfio_ccw_sch_probe(struct subchannel *sch)
 	cio_disable_subchannel(sch);
 out_free:
 	dev_set_drvdata(&sch->dev, NULL);
-	if (private->cmd_region)
-		kmem_cache_free(vfio_ccw_cmd_region, private->cmd_region);
-	if (private->io_region)
-		kmem_cache_free(vfio_ccw_io_region, private->io_region);
+	vfio_ccw_free_regions(private);
 	kfree(private->cp.guest_cp);
 	kfree(private);
 	return ret;

@@ -200,8 +205,7 @@ static int vfio_ccw_sch_remove(struct subchannel *sch)
 
 	dev_set_drvdata(&sch->dev, NULL);
 
-	kmem_cache_free(vfio_ccw_cmd_region, private->cmd_region);
-	kmem_cache_free(vfio_ccw_io_region, private->io_region);
+	vfio_ccw_free_regions(private);
 	kfree(private->cp.guest_cp);
 	kfree(private);

@@ -304,6 +308,12 @@ static void vfio_ccw_debug_exit(void)
 	debug_unregister(vfio_ccw_debug_trace_id);
 }
 
+static void vfio_ccw_destroy_regions(void)
+{
+	kmem_cache_destroy(vfio_ccw_cmd_region);
+	kmem_cache_destroy(vfio_ccw_io_region);
+}
+
 static int __init vfio_ccw_sch_init(void)
 {
 	int ret;

@@ -346,8 +356,7 @@ static int __init vfio_ccw_sch_init(void)
 	return ret;
 
 out_err:
-	kmem_cache_destroy(vfio_ccw_cmd_region);
-	kmem_cache_destroy(vfio_ccw_io_region);
+	vfio_ccw_destroy_regions();
 	destroy_workqueue(vfio_ccw_work_q);
 	vfio_ccw_debug_exit();
 	return ret;

@@ -357,8 +366,7 @@ static void __exit vfio_ccw_sch_exit(void)
 {
 	css_driver_unregister(&vfio_ccw_sch_driver);
 	isc_unregister(VFIO_CCW_ISC);
-	kmem_cache_destroy(vfio_ccw_io_region);
-	kmem_cache_destroy(vfio_ccw_cmd_region);
+	vfio_ccw_destroy_regions();
 	destroy_workqueue(vfio_ccw_work_q);
 	vfio_ccw_debug_exit();
 }

Signature · 1 line

2.17.1

Eric Farman

author maintainer

2020-05-05 12:27 UTC #2 lore raw

PATCH v4 2/8 vfio-ccw: Register a chp_event callback for vfio-ccw

In reply to Eric Farman

From: Farhan Ali <[email protected]>

Signed-off-by Farhan Ali <[email protected]>

Signed-off-by Eric Farman <[email protected]>

---

Notes:
v3->v4:
- Check schib.lpum before calling cio_cancel_halt_clear [CH]

v2->v3:
- Add a call to cio_cancel_halt_clear() for CHP_VARY_OFF [CH]

v1->v2:
- Move s390dbf before cio_update_schib() call [CH]

v0->v1: [EF]
- Add s390dbf trace

 drivers/s390/cio/vfio_ccw_drv.c | 47 +++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

drivers/s390/cio/vfio_ccw_drv.c +47 -0

index 8715c1c2f1e1..fb1275a7d1f5 100644
@@ -19,6 +19,7 @@
 
 #include <asm/isc.h>
 
+#include "chp.h"
 #include "ioasm.h"
 #include "css.h"
 #include "vfio_ccw_private.h"

@@ -262,6 +263,51 @@ static int vfio_ccw_sch_event(struct subchannel *sch, int process)
 	return rc;
 }
 
+static int vfio_ccw_chp_event(struct subchannel *sch,
+			      struct chp_link *link, int event)
+{
+	struct vfio_ccw_private *private = dev_get_drvdata(&sch->dev);
+	int mask = chp_ssd_get_mask(&sch->ssd_info, link);
+	int retry = 255;
+
+	if (!private || !mask)
+		return 0;
+
+	VFIO_CCW_MSG_EVENT(2, "%pUl (%x.%x.%04x): mask=0x%x event=%d\n",
+			   mdev_uuid(private->mdev), sch->schid.cssid,
+			   sch->schid.ssid, sch->schid.sch_no,
+			   mask, event);
+
+	if (cio_update_schib(sch))
+		return -ENODEV;
+
+	switch (event) {
+	case CHP_VARY_OFF:
+		/* Path logically turned off */
+		sch->opm &= ~mask;
+		sch->lpm &= ~mask;
+		if (sch->schib.pmcw.lpum & mask)
+			cio_cancel_halt_clear(sch, &retry);
+		break;
+	case CHP_OFFLINE:
+		/* Path is gone */
+		if (sch->schib.pmcw.lpum & mask)
+			cio_cancel_halt_clear(sch, &retry);
+		break;
+	case CHP_VARY_ON:
+		/* Path logically turned on */
+		sch->opm |= mask;
+		sch->lpm |= mask;
+		break;
+	case CHP_ONLINE:
+		/* Path became available */
+		sch->lpm |= mask & sch->opm;
+		break;
+	}
+
+	return 0;
+}
+
 static struct css_device_id vfio_ccw_sch_ids[] = {
 	{ .match_flags = 0x1, .type = SUBCHANNEL_TYPE_IO, },
 	{ /* end of list */ },

@@ -279,6 +325,7 @@ static struct css_driver vfio_ccw_sch_driver = {
 	.remove = vfio_ccw_sch_remove,
 	.shutdown = vfio_ccw_sch_shutdown,
 	.sch_event = vfio_ccw_sch_event,
+	.chp_event = vfio_ccw_chp_event,
 };
 
 static int __init vfio_ccw_debug_init(void)

Signature · 1 line

2.17.1

Cornelia Huck

Reviewed-by on v4 2/8

2020-05-06 13:18 UTC #3 lore raw

Re: [PATCH v4 2/8] vfio-ccw: Register a chp_event callback for vfio-ccw

Quoted 16 lines · On Tue, 5 May 2020 14:27:39 +0200 Eric Farman <[email protected]> wrote:

From: Farhan Ali <[email protected]>
Register the chp_event callback to receive channel path related events for the subchannels managed by vfio-ccw.
Signed-off-by Farhan Ali <[email protected]>
Signed-off-by Eric Farman <[email protected]>
---
Notes:
v3->v4:
- Check schib.lpum before calling cio_cancel_halt_clear [CH]
v2->v3:
- Add a call to cio_cancel_halt_clear() for CHP_VARY_OFF [CH]

Quoted 9 lines

v1->v2:
- Move s390dbf before cio_update_schib() call [CH]
v0->v1: [EF]
- Add s390dbf trace
 drivers/s390/cio/vfio_ccw_drv.c | 47 +++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

Reviewed-by Cornelia Huck <[email protected]>

Eric Farman

author maintainer

2020-05-05 12:30 UTC #4 lore raw

PATCH v4 3/8 vfio-ccw: Refactor the unregister of the async regions

In reply to Eric Farman

This is mostly for the purposes of a later patch, since we'll need to do the same thing later.

While we are at it, move the resulting function call to ahead of the unregistering of the IOMMU notifier, so that it's done in the reverse order of how it was created.

Signed-off-by Eric Farman <[email protected]>

Reviewed-by Cornelia Huck <[email protected]>

---

Notes:
v1->v2:
- Add Conny's r-b

 drivers/s390/cio/vfio_ccw_ops.c     | 20 ++++++++++++--------
 drivers/s390/cio/vfio_ccw_private.h |  1 +
 2 files changed, 13 insertions(+), 8 deletions(-)

drivers/s390/cio/vfio_ccw_ops.c +12 -8

index f0d71ab77c50..d4fc84b8867f 100644
@@ -181,7 +181,6 @@ static void vfio_ccw_mdev_release(struct mdev_device *mdev)
 {
 	struct vfio_ccw_private *private =
 		dev_get_drvdata(mdev_parent_dev(mdev));
-	int i;
 
 	if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
 	    (private->state != VFIO_CCW_STATE_STANDBY)) {

@@ -191,15 +190,9 @@ static void vfio_ccw_mdev_release(struct mdev_device *mdev)
 	}
 
 	cp_free(&private->cp);
+	vfio_ccw_unregister_dev_regions(private);
 	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
 				 &private->nb);
-
-	for (i = 0; i < private->num_regions; i++)
-		private->region[i].ops->release(private, &private->region[i]);
-
-	private->num_regions = 0;
-	kfree(private->region);
-	private->region = NULL;
 }
 
 static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private *private,

@@ -482,6 +475,17 @@ int vfio_ccw_register_dev_region(struct vfio_ccw_private *private,
 	return 0;
 }
 
+void vfio_ccw_unregister_dev_regions(struct vfio_ccw_private *private)
+{
+	int i;
+
+	for (i = 0; i < private->num_regions; i++)
+		private->region[i].ops->release(private, &private->region[i]);
+	private->num_regions = 0;
+	kfree(private->region);
+	private->region = NULL;
+}
+
 static ssize_t vfio_ccw_mdev_ioctl(struct mdev_device *mdev,
 				   unsigned int cmd,
 				   unsigned long arg)

drivers/s390/cio/vfio_ccw_private.h +1 -0

index 9b9bb4982972..ce3834159d98 100644
@@ -53,6 +53,7 @@ int vfio_ccw_register_dev_region(struct vfio_ccw_private *private,
 				 unsigned int subtype,
 				 const struct vfio_ccw_regops *ops,
 				 size_t size, u32 flags, void *data);
+void vfio_ccw_unregister_dev_regions(struct vfio_ccw_private *private);
 
 int vfio_ccw_register_async_dev_regions(struct vfio_ccw_private *private);

Signature · 1 line

2.17.1

Eric Farman

author maintainer

2020-05-05 12:27 UTC #5 lore raw

PATCH v4 4/8 vfio-ccw: Introduce a new schib region

In reply to Eric Farman

From: Farhan Ali <[email protected]>

The schib region can be used by userspace to get the subchannel- information block (SCHIB) for the passthrough subchannel. This can be useful to get information such as channel path information via the SCHIB.PMCW fields.

Signed-off-by Farhan Ali <[email protected]>

Signed-off-by Eric Farman <[email protected]>

Reviewed-by Cornelia Huck <[email protected]>

---

Notes:
v3->v4:
- Added Conny's r-b

v2->v3:
- Updated Copyright year and Authors [CH]
- Mention that schib region triggers a STSCH [CH]

v1->v2:
- Add new region info to Documentation/s390/vfio-ccw.rst [CH]
- Add a block comment to struct ccw_schib_region [CH]

v0->v1: [EF]
- Clean up checkpatch (#include, whitespace) errors
- Remove unnecessary includes from vfio_ccw_chp.c
- Add ret=-ENOMEM in error path for new region
- Add call to vfio_ccw_unregister_dev_regions() during error exit path of vfio_ccw_mdev_open()
- New info on the module prologue
- Reorder cleanup of regions

 Documentation/s390/vfio-ccw.rst     | 19 +++++++-
 drivers/s390/cio/Makefile           |  2 +-
 drivers/s390/cio/vfio_ccw_chp.c     | 76 +++++++++++++++++++++++++++++
 drivers/s390/cio/vfio_ccw_drv.c     | 20 ++++++++
 drivers/s390/cio/vfio_ccw_ops.c     | 14 +++++-
 drivers/s390/cio/vfio_ccw_private.h |  3 ++
 include/uapi/linux/vfio.h           |  1 +
 include/uapi/linux/vfio_ccw.h       | 10 ++++
 8 files changed, 141 insertions(+), 4 deletions(-)
 create mode 100644 drivers/s390/cio/vfio_ccw_chp.c

Documentation/s390/vfio-ccw.rst +18 -1

index fca9c4f5bd9c..98832d95f395 100644
@@ -231,6 +231,22 @@ This region is exposed via region type VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD.
 
 Currently, CLEAR SUBCHANNEL and HALT SUBCHANNEL use this region.
 
+vfio-ccw schib region
+---------------------
+
+The vfio-ccw schib region is used to return Subchannel-Information
+Block (SCHIB) data to userspace::
+
+  struct ccw_schib_region {
+  #define SCHIB_AREA_SIZE 52
+         __u8 schib_area[SCHIB_AREA_SIZE];
+  } __packed;
+
+This region is exposed via region type VFIO_REGION_SUBTYPE_CCW_SCHIB.
+
+Reading this region triggers a STORE SUBCHANNEL to be issued to the
+associated hardware.
+
 vfio-ccw operation details
 --------------------------

@@ -333,7 +349,8 @@ through DASD/ECKD device online in a guest now and use it as a block
 device.
 
 The current code allows the guest to start channel programs via
-START SUBCHANNEL, and to issue HALT SUBCHANNEL and CLEAR SUBCHANNEL.
+START SUBCHANNEL, and to issue HALT SUBCHANNEL, CLEAR SUBCHANNEL,
+and STORE SUBCHANNEL.
 
 vfio-ccw supports classic (command mode) channel I/O only. Transport
 mode (HPF) is not supported.

drivers/s390/cio/Makefile +1 -1

index 23eae4188876..a9235f111e79 100644
@@ -21,5 +21,5 @@ qdio-objs := qdio_main.o qdio_thinint.o qdio_debug.o qdio_setup.o
 obj-$(CONFIG_QDIO) += qdio.o
 
 vfio_ccw-objs += vfio_ccw_drv.o vfio_ccw_cp.o vfio_ccw_ops.o vfio_ccw_fsm.o \
-	vfio_ccw_async.o vfio_ccw_trace.o
+	vfio_ccw_async.o vfio_ccw_trace.o vfio_ccw_chp.o
 obj-$(CONFIG_VFIO_CCW) += vfio_ccw.o

drivers/s390/cio/vfio_ccw_chp.c new file +76 -0

76 lines not on this page · raw message

drivers/s390/cio/vfio_ccw_drv.c +20 -0

index fb1275a7d1f5..7aeff42f370d 100644
@@ -27,6 +27,7 @@
 struct workqueue_struct *vfio_ccw_work_q;
 static struct kmem_cache *vfio_ccw_io_region;
 static struct kmem_cache *vfio_ccw_cmd_region;
+static struct kmem_cache *vfio_ccw_schib_region;
 
 debug_info_t *vfio_ccw_debug_msg_id;
 debug_info_t *vfio_ccw_debug_trace_id;

@@ -119,6 +120,8 @@ static void vfio_ccw_sch_irq(struct subchannel *sch)
 
 static void vfio_ccw_free_regions(struct vfio_ccw_private *private)
 {
+	if (private->schib_region)
+		kmem_cache_free(vfio_ccw_schib_region, private->schib_region);
 	if (private->cmd_region)
 		kmem_cache_free(vfio_ccw_cmd_region, private->cmd_region);
 	if (private->io_region)

@@ -156,6 +159,12 @@ static int vfio_ccw_sch_probe(struct subchannel *sch)
 	if (!private->cmd_region)
 		goto out_free;
 
+	private->schib_region = kmem_cache_zalloc(vfio_ccw_schib_region,
+						  GFP_KERNEL | GFP_DMA);
+
+	if (!private->schib_region)
+		goto out_free;
+
 	private->sch = sch;
 	dev_set_drvdata(&sch->dev, private);
 	mutex_init(&private->io_mutex);

@@ -357,6 +366,7 @@ static void vfio_ccw_debug_exit(void)
 
 static void vfio_ccw_destroy_regions(void)
 {
+	kmem_cache_destroy(vfio_ccw_schib_region);
 	kmem_cache_destroy(vfio_ccw_cmd_region);
 	kmem_cache_destroy(vfio_ccw_io_region);
 }

@@ -393,6 +403,16 @@ static int __init vfio_ccw_sch_init(void)
 		goto out_err;
 	}
 
+	vfio_ccw_schib_region = kmem_cache_create_usercopy("vfio_ccw_schib_region",
+					sizeof(struct ccw_schib_region), 0,
+					SLAB_ACCOUNT, 0,
+					sizeof(struct ccw_schib_region), NULL);
+
+	if (!vfio_ccw_schib_region) {
+		ret = -ENOMEM;
+		goto out_err;
+	}
+
 	isc_register(VFIO_CCW_ISC);
 	ret = css_driver_register(&vfio_ccw_sch_driver);
 	if (ret) {

drivers/s390/cio/vfio_ccw_ops.c +12 -2

index d4fc84b8867f..22988d67b6bb 100644
@@ -172,8 +172,18 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
 
 	ret = vfio_ccw_register_async_dev_regions(private);
 	if (ret)
-		vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
-					 &private->nb);
+		goto out_unregister;
+
+	ret = vfio_ccw_register_schib_dev_regions(private);
+	if (ret)
+		goto out_unregister;
+
+	return ret;
+
+out_unregister:
+	vfio_ccw_unregister_dev_regions(private);
+	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+				 &private->nb);
 	return ret;
 }

drivers/s390/cio/vfio_ccw_private.h +3 -0

index ce3834159d98..d6601a8adf13 100644
@@ -56,6 +56,7 @@ int vfio_ccw_register_dev_region(struct vfio_ccw_private *private,
 void vfio_ccw_unregister_dev_regions(struct vfio_ccw_private *private);
 
 int vfio_ccw_register_async_dev_regions(struct vfio_ccw_private *private);
+int vfio_ccw_register_schib_dev_regions(struct vfio_ccw_private *private);
 
 /**
  * struct vfio_ccw_private

@@ -69,6 +70,7 @@ int vfio_ccw_register_async_dev_regions(struct vfio_ccw_private *private);
  * @io_mutex: protect against concurrent update of I/O regions
  * @region: additional regions for other subchannel operations
  * @cmd_region: MMIO region for asynchronous I/O commands other than START
+ * @schib_region: MMIO region for SCHIB information
  * @num_regions: number of additional regions
  * @cp: channel program for the current I/O operation
  * @irb: irb info received from interrupt

@@ -87,6 +89,7 @@ struct vfio_ccw_private {
 	struct mutex		io_mutex;
 	struct vfio_ccw_region *region;
 	struct ccw_cmd_region	*cmd_region;
+	struct ccw_schib_region *schib_region;
 	int num_regions;
 
 	struct channel_program	cp;

include/uapi/linux/vfio.h +1 -0

index 015516bcfaa3..7a1abbd889bd 100644
@@ -378,6 +378,7 @@ struct vfio_region_gfx_edid {
 
 /* sub-types for VFIO_REGION_TYPE_CCW */
 #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD	(1)
+#define VFIO_REGION_SUBTYPE_CCW_SCHIB		(2)
 
 /*
  * The MSIX mappable capability informs that MSIX data of a BAR can be mmapped

include/uapi/linux/vfio_ccw.h +10 -0

index cbecbf0cd54f..758bf214898d 100644
@@ -34,4 +34,14 @@ struct ccw_cmd_region {
 	__u32 ret_code;
 } __packed;
 
+/*
+ * Used for processing commands that read the subchannel-information block
+ * Reading this region triggers a stsch() to hardware
+ * Note: this is controlled by a capability
+ */
+struct ccw_schib_region {
+#define SCHIB_AREA_SIZE 52
+	__u8 schib_area[SCHIB_AREA_SIZE];
+} __packed;
+
 #endif

Signature · 1 line

2.17.1

Eric Farman

author maintainer

2020-05-05 12:27 UTC #6 lore raw

PATCH v4 5/8 vfio-ccw: Refactor IRQ handlers

In reply to Eric Farman

To simplify future expansion.

Signed-off-by Eric Farman <[email protected]>

Reviewed-by Cornelia Huck <[email protected]>

---

Notes:
v1->v2:
- Add Conny's r-b

 drivers/s390/cio/vfio_ccw_ops.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

drivers/s390/cio/vfio_ccw_ops.c +17 -6

index 22988d67b6bb..c3a74ab7bb86 100644
@@ -387,17 +387,21 @@ static int vfio_ccw_mdev_get_region_info(struct vfio_region_info *info,
 
 static int vfio_ccw_mdev_get_irq_info(struct vfio_irq_info *info)
 {
-	if (info->index != VFIO_CCW_IO_IRQ_INDEX)
+	switch (info->index) {
+	case VFIO_CCW_IO_IRQ_INDEX:
+		info->count = 1;
+		info->flags = VFIO_IRQ_INFO_EVENTFD;
+		break;
+	default:
 		return -EINVAL;
-
-	info->count = 1;
-	info->flags = VFIO_IRQ_INFO_EVENTFD;
+	}
 
 	return 0;
 }
 
 static int vfio_ccw_mdev_set_irqs(struct mdev_device *mdev,
 				  uint32_t flags,
+				  uint32_t index,
 				  void __user *data)
 {
 	struct vfio_ccw_private *private;

@@ -407,7 +411,14 @@ static int vfio_ccw_mdev_set_irqs(struct mdev_device *mdev,
 		return -EINVAL;
 
 	private = dev_get_drvdata(mdev_parent_dev(mdev));
-	ctx = &private->io_trigger;
+
+	switch (index) {
+	case VFIO_CCW_IO_IRQ_INDEX:
+		ctx = &private->io_trigger;
+		break;
+	default:
+		return -EINVAL;
+	}
 
 	switch (flags & VFIO_IRQ_SET_DATA_TYPE_MASK) {
 	case VFIO_IRQ_SET_DATA_NONE:

@@ -579,7 +590,7 @@ static ssize_t vfio_ccw_mdev_ioctl(struct mdev_device *mdev,
 			return ret;
 
 		data = (void __user *)(arg + minsz);
-		return vfio_ccw_mdev_set_irqs(mdev, hdr.flags, data);
+		return vfio_ccw_mdev_set_irqs(mdev, hdr.flags, hdr.index, data);
 	}
 	case VFIO_DEVICE_RESET:
 		return vfio_ccw_mdev_reset(mdev);

Signature · 1 line

2.17.1

Eric Farman

author maintainer

2020-05-05 12:27 UTC #7 lore raw

PATCH v4 6/8 vfio-ccw: Introduce a new CRW region

In reply to Eric Farman

From: Farhan Ali <[email protected]>

This region provides a mechanism to pass a Channel Report Word that affect vfio-ccw devices, and needs to be passed to the guest for its awareness and/or processing.

The base driver (see crw_collect_info()) provides space for two CRWs, as a subchannel event may have two CRWs chained together (one for the ssid, one for the subchannel). As vfio-ccw will deal with everything at the subchannel level, provide space for a single CRW to be transferred in one shot.

Signed-off-by Farhan Ali <[email protected]>

Signed-off-by Eric Farman <[email protected]>

---

Notes:
v3->v4:
- Move crw_trigger and CRW_IRQ from later patch to here [EF]
- Cleanup descriptions of sizeof CRW region in doc, commentary, and commit description [CH]
- Clear crw when finished [CH]

v2->v3:
- Remove "if list-empty" check, since there's no list yet [EF]
- Reduce the CRW region to one fullword, instead of two [CH]

v1->v2:
- Add new region info to Documentation/s390/vfio-ccw.rst [CH]
- Add a block comment to struct ccw_crw_region [CH]

v0->v1: [EF]
- Clean up checkpatch (whitespace) errors
- Add ret=-ENOMEM in error path for new region
- Add io_mutex for region read (originally in last patch)
- Change crw1/crw2 to crw0/crw1
- Reorder cleanup of regions

 Documentation/s390/vfio-ccw.rst     | 19 ++++++++++
 drivers/s390/cio/vfio_ccw_chp.c     | 55 +++++++++++++++++++++++++++++
 drivers/s390/cio/vfio_ccw_drv.c     | 20 +++++++++++
 drivers/s390/cio/vfio_ccw_ops.c     |  8 +++++
 drivers/s390/cio/vfio_ccw_private.h |  4 +++
 include/uapi/linux/vfio.h           |  2 ++
 include/uapi/linux/vfio_ccw.h       |  8 +++++
 7 files changed, 116 insertions(+)

Documentation/s390/vfio-ccw.rst +19 -0

index 98832d95f395..da3a9e22663d 100644
@@ -247,6 +247,25 @@ This region is exposed via region type VFIO_REGION_SUBTYPE_CCW_SCHIB.
 Reading this region triggers a STORE SUBCHANNEL to be issued to the
 associated hardware.
 
+vfio-ccw crw region
+---------------------
+
+The vfio-ccw crw region is used to return Channel Report Word (CRW)
+data to userspace::
+
+  struct ccw_crw_region {
+         __u32 crw;
+  } __packed;
+
+This region is exposed via region type VFIO_REGION_SUBTYPE_CCW_CRW.
+
+Reading this region returns a CRW if one that is relevant for this
+subchannel (e.g. one reporting changes in channel path state) is
+pending, or all zeroes if not. If multiple CRWs are pending (including
+possibly chained CRWs), reading this region again will return the next
+one, until no more CRWs are pending and zeroes are returned. This is
+similar to how STORE CHANNEL REPORT WORD works.
+
 vfio-ccw operation details
 --------------------------

drivers/s390/cio/vfio_ccw_chp.c +55 -0

58 lines not on this page · raw message

drivers/s390/cio/vfio_ccw_drv.c +20 -0

index 7aeff42f370d..e4deae6fd525 100644
@@ -28,6 +28,7 @@ struct workqueue_struct *vfio_ccw_work_q;
 static struct kmem_cache *vfio_ccw_io_region;
 static struct kmem_cache *vfio_ccw_cmd_region;
 static struct kmem_cache *vfio_ccw_schib_region;
+static struct kmem_cache *vfio_ccw_crw_region;
 
 debug_info_t *vfio_ccw_debug_msg_id;
 debug_info_t *vfio_ccw_debug_trace_id;

@@ -120,6 +121,8 @@ static void vfio_ccw_sch_irq(struct subchannel *sch)
 
 static void vfio_ccw_free_regions(struct vfio_ccw_private *private)
 {
+	if (private->crw_region)
+		kmem_cache_free(vfio_ccw_crw_region, private->crw_region);
 	if (private->schib_region)
 		kmem_cache_free(vfio_ccw_schib_region, private->schib_region);
 	if (private->cmd_region)

@@ -165,6 +168,12 @@ static int vfio_ccw_sch_probe(struct subchannel *sch)
 	if (!private->schib_region)
 		goto out_free;
 
+	private->crw_region = kmem_cache_zalloc(vfio_ccw_crw_region,
+						GFP_KERNEL | GFP_DMA);
+
+	if (!private->crw_region)
+		goto out_free;
+
 	private->sch = sch;
 	dev_set_drvdata(&sch->dev, private);
 	mutex_init(&private->io_mutex);

@@ -366,6 +375,7 @@ static void vfio_ccw_debug_exit(void)
 
 static void vfio_ccw_destroy_regions(void)
 {
+	kmem_cache_destroy(vfio_ccw_crw_region);
 	kmem_cache_destroy(vfio_ccw_schib_region);
 	kmem_cache_destroy(vfio_ccw_cmd_region);
 	kmem_cache_destroy(vfio_ccw_io_region);

@@ -413,6 +423,16 @@ static int __init vfio_ccw_sch_init(void)
 		goto out_err;
 	}
 
+	vfio_ccw_crw_region = kmem_cache_create_usercopy("vfio_ccw_crw_region",
+					sizeof(struct ccw_crw_region), 0,
+					SLAB_ACCOUNT, 0,
+					sizeof(struct ccw_crw_region), NULL);
+
+	if (!vfio_ccw_crw_region) {
+		ret = -ENOMEM;
+		goto out_err;
+	}
+
 	isc_register(VFIO_CCW_ISC);
 	ret = css_driver_register(&vfio_ccw_sch_driver);
 	if (ret) {

drivers/s390/cio/vfio_ccw_ops.c +8 -0

index c3a74ab7bb86..8b3ed5b45277 100644
@@ -178,6 +178,10 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
 	if (ret)
 		goto out_unregister;
 
+	ret = vfio_ccw_register_crw_dev_regions(private);
+	if (ret)
+		goto out_unregister;
+
 	return ret;
 
 out_unregister:

@@ -389,6 +393,7 @@ static int vfio_ccw_mdev_get_irq_info(struct vfio_irq_info *info)
 {
 	switch (info->index) {
 	case VFIO_CCW_IO_IRQ_INDEX:
+	case VFIO_CCW_CRW_IRQ_INDEX:
 		info->count = 1;
 		info->flags = VFIO_IRQ_INFO_EVENTFD;
 		break;

@@ -416,6 +421,9 @@ static int vfio_ccw_mdev_set_irqs(struct mdev_device *mdev,
 	case VFIO_CCW_IO_IRQ_INDEX:
 		ctx = &private->io_trigger;
 		break;
+	case VFIO_CCW_CRW_IRQ_INDEX:
+		ctx = &private->crw_trigger;
+		break;
 	default:
 		return -EINVAL;
 	}

drivers/s390/cio/vfio_ccw_private.h +4 -0

index d6601a8adf13..97131b4df0b9 100644
@@ -57,6 +57,7 @@ void vfio_ccw_unregister_dev_regions(struct vfio_ccw_private *private);
 
 int vfio_ccw_register_async_dev_regions(struct vfio_ccw_private *private);
 int vfio_ccw_register_schib_dev_regions(struct vfio_ccw_private *private);
+int vfio_ccw_register_crw_dev_regions(struct vfio_ccw_private *private);
 
 /**
  * struct vfio_ccw_private

@@ -71,6 +72,7 @@ int vfio_ccw_register_schib_dev_regions(struct vfio_ccw_private *private);
  * @region: additional regions for other subchannel operations
  * @cmd_region: MMIO region for asynchronous I/O commands other than START
  * @schib_region: MMIO region for SCHIB information
+ * @crw_region: MMIO region for getting channel report words
  * @num_regions: number of additional regions
  * @cp: channel program for the current I/O operation
  * @irb: irb info received from interrupt

@@ -90,6 +92,7 @@ struct vfio_ccw_private {
 	struct vfio_ccw_region *region;
 	struct ccw_cmd_region	*cmd_region;
 	struct ccw_schib_region *schib_region;
+	struct ccw_crw_region	*crw_region;
 	int num_regions;
 
 	struct channel_program	cp;

@@ -97,6 +100,7 @@ struct vfio_ccw_private {
 	union scsw		scsw;
 
 	struct eventfd_ctx	*io_trigger;
+	struct eventfd_ctx	*crw_trigger;
 	struct work_struct	io_work;
 } __aligned(8);

include/uapi/linux/vfio.h +2 -0

index 7a1abbd889bd..907758cf6d60 100644
@@ -379,6 +379,7 @@ struct vfio_region_gfx_edid {
 /* sub-types for VFIO_REGION_TYPE_CCW */
 #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD	(1)
 #define VFIO_REGION_SUBTYPE_CCW_SCHIB		(2)
+#define VFIO_REGION_SUBTYPE_CCW_CRW		(3)
 
 /*
  * The MSIX mappable capability informs that MSIX data of a BAR can be mmapped

@@ -578,6 +579,7 @@ enum {
 
 enum {
 	VFIO_CCW_IO_IRQ_INDEX,
+	VFIO_CCW_CRW_IRQ_INDEX,
 	VFIO_CCW_NUM_IRQS
 };

include/uapi/linux/vfio_ccw.h +8 -0

index 758bf214898d..cff5076586df 100644
@@ -44,4 +44,12 @@ struct ccw_schib_region {
 	__u8 schib_area[SCHIB_AREA_SIZE];
 } __packed;
 
+/*
+ * Used for returning a Channel Report Word to userspace.
+ * Note: this is controlled by a capability
+ */
+struct ccw_crw_region {
+	__u32 crw;
+} __packed;
+
 #endif

Signature · 1 line

2.17.1

Cornelia Huck

Reviewed-by on v4 6/8

2020-05-06 13:24 UTC #8 lore raw

Re: [PATCH v4 6/8] vfio-ccw: Introduce a new CRW region

Quoted 38 lines · On Tue, 5 May 2020 14:27:43 +0200 Eric Farman <[email protected]> wrote:

From: Farhan Ali <[email protected]>
This region provides a mechanism to pass a Channel Report Word that affect vfio-ccw devices, and needs to be passed to the guest for its awareness and/or processing.
The base driver (see crw_collect_info()) provides space for two CRWs, as a subchannel event may have two CRWs chained together (one for the ssid, one for the subchannel). As vfio-ccw will deal with everything at the subchannel level, provide space for a single CRW to be transferred in one shot.
Signed-off-by Farhan Ali <[email protected]>
Signed-off-by Eric Farman <[email protected]>
---
Notes:
v3->v4:
- Move crw_trigger and CRW_IRQ from later patch to here [EF]
- Cleanup descriptions of sizeof CRW region in doc, commentary, and commit description [CH]
- Clear crw when finished [CH]
v2->v3:
- Remove "if list-empty" check, since there's no list yet [EF]
- Reduce the CRW region to one fullword, instead of two [CH]
v1->v2:
- Add new region info to Documentation/s390/vfio-ccw.rst [CH]
- Add a block comment to struct ccw_crw_region [CH]
v0->v1: [EF]
- Clean up checkpatch (whitespace) errors
- Add ret=-ENOMEM in error path for new region
- Add io_mutex for region read (originally in last patch)
- Change crw1/crw2 to crw0/crw1
- Reorder cleanup of regions

Quoted 9 lines

 Documentation/s390/vfio-ccw.rst     | 19 ++++++++++
 drivers/s390/cio/vfio_ccw_chp.c     | 55 +++++++++++++++++++++++++++++
 drivers/s390/cio/vfio_ccw_drv.c     | 20 +++++++++++
 drivers/s390/cio/vfio_ccw_ops.c     |  8 +++++
 drivers/s390/cio/vfio_ccw_private.h |  4 +++
 include/uapi/linux/vfio.h           |  2 ++
 include/uapi/linux/vfio_ccw.h       |  8 +++++
 7 files changed, 116 insertions(+)

Reviewed-by Cornelia Huck <[email protected]>

Eric Farman

author maintainer

2020-05-05 12:31 UTC #9 lore raw

PATCH v4 7/8 vfio-ccw: Wire up the CRW irq and CRW region

In reply to Eric Farman

From: Farhan Ali <[email protected]>

Use the IRQ to notify userspace that there is a CRW pending in the region, related to path-availability changes on the passthrough subchannel.

Signed-off-by Farhan Ali <[email protected]>

Signed-off-by Eric Farman <[email protected]>

---

Notes:
v3->v4:
- s/vfio_ccw_alloc_crw()/vfio_ccw_queue_crw()/ [CH]
- Remove cssid from crw that is built [CH]

v2->v3:
- Refactor vfio_ccw_alloc_crw() to accept rsc, erc, and rsid fields of a CRW as input [CH]
- Copy the right amount of CRWs to the crw_region [EF]
- Use sizeof(target) for the memcpy, rather than sizeof(source) [EF]
- Ensure the CRW region is empty if no CRW is present [EF/CH]
- Refactor how data goes from private-to-region-to-user [CH]
- Reduce the number of CRWs from two to one [CH]
- s/vc_crw/crw/ [EF]

v1->v2:
- Remove extraneous 0x0 in crw.rsid assignment [CH]
- Refactor the building/queueing of a crw into its own routine [EF]

v0->v1: [EF]
- Place the non-refactoring changes from the previous patch here
- Clean up checkpatch (whitespace) errors
- s/chp_crw/crw/
- Move acquire/release of io_mutex in vfio_ccw_crw_region_read() into patch that introduces that region
- Remove duplicate include from vfio_ccw_drv.c
- Reorder include in vfio_ccw_private.h

 drivers/s390/cio/vfio_ccw_chp.c     | 17 ++++++++++
 drivers/s390/cio/vfio_ccw_drv.c     | 49 +++++++++++++++++++++++++++++
 drivers/s390/cio/vfio_ccw_private.h |  8 +++++
 3 files changed, 74 insertions(+)

drivers/s390/cio/vfio_ccw_chp.c +17 -0

index 37ea344a4d72..876f6ade51cc 100644
@@ -82,14 +82,24 @@ static ssize_t vfio_ccw_crw_region_read(struct vfio_ccw_private *private,
 	unsigned int i = VFIO_CCW_OFFSET_TO_INDEX(*ppos) - VFIO_CCW_NUM_REGIONS;
 	loff_t pos = *ppos & VFIO_CCW_OFFSET_MASK;
 	struct ccw_crw_region *region;
+	struct vfio_ccw_crw *crw;
 	int ret;
 
 	if (pos + count > sizeof(*region))
 		return -EINVAL;
 
+	crw = list_first_entry_or_null(&private->crw,
+				       struct vfio_ccw_crw, next);
+
+	if (crw)
+		list_del(&crw->next);
+
 	mutex_lock(&private->io_mutex);
 	region = private->region[i].data;
 
+	if (crw)
+		memcpy(&region->crw, &crw->crw, sizeof(region->crw));
+
 	if (copy_to_user(buf, (void *)region + pos, count))
 		ret = -EFAULT;
 	else

@@ -98,6 +108,13 @@ static ssize_t vfio_ccw_crw_region_read(struct vfio_ccw_private *private,
 	region->crw = 0;
 
 	mutex_unlock(&private->io_mutex);
+
+	kfree(crw);
+
+	/* Notify the guest if more CRWs are on our queue */
+	if (!list_empty(&private->crw) && private->crw_trigger)
+		eventfd_signal(private->crw_trigger, 1);
+
 	return ret;
 }

drivers/s390/cio/vfio_ccw_drv.c +49 -0

89 lines not on this page · raw message

drivers/s390/cio/vfio_ccw_private.h +8 -0

index 97131b4df0b9..8723156b29ea 100644
@@ -17,6 +17,7 @@
 #include <linux/eventfd.h>
 #include <linux/workqueue.h>
 #include <linux/vfio_ccw.h>
+#include <asm/crw.h>
 #include <asm/debug.h>
 
 #include "css.h"

@@ -59,6 +60,11 @@ int vfio_ccw_register_async_dev_regions(struct vfio_ccw_private *private);
 int vfio_ccw_register_schib_dev_regions(struct vfio_ccw_private *private);
 int vfio_ccw_register_crw_dev_regions(struct vfio_ccw_private *private);
 
+struct vfio_ccw_crw {
+	struct list_head	next;
+	struct crw		crw;
+};
+
 /**
  * struct vfio_ccw_private
  * @sch: pointer to the subchannel

@@ -98,10 +104,12 @@ struct vfio_ccw_private {
 	struct channel_program	cp;
 	struct irb		irb;
 	union scsw		scsw;
+	struct list_head	crw;
 
 	struct eventfd_ctx	*io_trigger;
 	struct eventfd_ctx	*crw_trigger;
 	struct work_struct	io_work;
+	struct work_struct	crw_work;
 } __aligned(8);
 
 extern int vfio_ccw_mdev_reg(struct subchannel *sch);

Signature · 1 line

2.17.1

Cornelia Huck

Reviewed-by on v4 7/8

2020-05-06 13:53 UTC #10 lore raw

Re: [PATCH v4 7/8] vfio-ccw: Wire up the CRW irq and CRW region

Quoted 38 lines · On Tue, 5 May 2020 14:27:44 +0200 Eric Farman <[email protected]> wrote:

From: Farhan Ali <[email protected]>
Use the IRQ to notify userspace that there is a CRW pending in the region, related to path-availability changes on the passthrough subchannel.
Signed-off-by Farhan Ali <[email protected]>
Signed-off-by Eric Farman <[email protected]>
---
Notes:
v3->v4:
- s/vfio_ccw_alloc_crw()/vfio_ccw_queue_crw()/ [CH]
- Remove cssid from crw that is built [CH]
v2->v3:
- Refactor vfio_ccw_alloc_crw() to accept rsc, erc, and rsid fields of a CRW as input [CH]
- Copy the right amount of CRWs to the crw_region [EF]
- Use sizeof(target) for the memcpy, rather than sizeof(source) [EF]
- Ensure the CRW region is empty if no CRW is present [EF/CH]
- Refactor how data goes from private-to-region-to-user [CH]
- Reduce the number of CRWs from two to one [CH]
- s/vc_crw/crw/ [EF]
v1->v2:
- Remove extraneous 0x0 in crw.rsid assignment [CH]
- Refactor the building/queueing of a crw into its own routine [EF]
v0->v1: [EF]
- Place the non-refactoring changes from the previous patch here
- Clean up checkpatch (whitespace) errors
- s/chp_crw/crw/
- Move acquire/release of io_mutex in vfio_ccw_crw_region_read() into patch that introduces that region
- Remove duplicate include from vfio_ccw_drv.c
- Reorder include in vfio_ccw_private.h

Quoted 4 lines

 drivers/s390/cio/vfio_ccw_chp.c     | 17 ++++++++++
 drivers/s390/cio/vfio_ccw_drv.c     | 49 +++++++++++++++++++++++++++++
 drivers/s390/cio/vfio_ccw_private.h |  8 +++++
 3 files changed, 74 insertions(+)

Reviewed-by Cornelia Huck <[email protected]>

Eric Farman

author maintainer

2020-05-05 12:27 UTC #11 lore raw

PATCH v4 8/8 vfio-ccw: Add trace for CRW event

In reply to Eric Farman

Since CRW events are (should be) rare, let's put a trace in that routine too.

Signed-off-by Eric Farman <[email protected]>

Reviewed-by Cornelia Huck <[email protected]>

---

Notes:
v3->v4:
- Added Conny's r-b

 drivers/s390/cio/vfio_ccw_drv.c   |  1 +
 drivers/s390/cio/vfio_ccw_trace.c |  1 +
 drivers/s390/cio/vfio_ccw_trace.h | 30 ++++++++++++++++++++++++++++++
 3 files changed, 32 insertions(+)

drivers/s390/cio/vfio_ccw_drv.c +1 -0

index 9144360851ed..8c625b530035 100644
@@ -336,6 +336,7 @@ static int vfio_ccw_chp_event(struct subchannel *sch,
 	if (!private || !mask)
 		return 0;
 
+	trace_vfio_ccw_chp_event(private->sch->schid, mask, event);
 	VFIO_CCW_MSG_EVENT(2, "%pUl (%x.%x.%04x): mask=0x%x event=%d\n",
 			   mdev_uuid(private->mdev), sch->schid.cssid,
 			   sch->schid.ssid, sch->schid.sch_no,

drivers/s390/cio/vfio_ccw_trace.c +1 -0

index 8c671d2519f6..4a0205905afc 100644
@@ -9,6 +9,7 @@
 #define CREATE_TRACE_POINTS
 #include "vfio_ccw_trace.h"
 
+EXPORT_TRACEPOINT_SYMBOL(vfio_ccw_chp_event);
 EXPORT_TRACEPOINT_SYMBOL(vfio_ccw_fsm_async_request);
 EXPORT_TRACEPOINT_SYMBOL(vfio_ccw_fsm_event);
 EXPORT_TRACEPOINT_SYMBOL(vfio_ccw_fsm_io_request);

drivers/s390/cio/vfio_ccw_trace.h +30 -0

index f5d31887d413..62fb30598d47 100644
@@ -17,6 +17,36 @@
 
 #include <linux/tracepoint.h>
 
+TRACE_EVENT(vfio_ccw_chp_event,
+	TP_PROTO(struct subchannel_id schid,
+		 int mask,
+		 int event),
+	TP_ARGS(schid, mask, event),
+
+	TP_STRUCT__entry(
+		__field(u8, cssid)
+		__field(u8, ssid)
+		__field(u16, sch_no)
+		__field(int, mask)
+		__field(int, event)
+	),
+
+	TP_fast_assign(
+		__entry->cssid = schid.cssid;
+		__entry->ssid = schid.ssid;
+		__entry->sch_no = schid.sch_no;
+		__entry->mask = mask;
+		__entry->event = event;
+	),
+
+	TP_printk("schid=%x.%x.%04x mask=0x%x event=%d",
+		  __entry->cssid,
+		  __entry->ssid,
+		  __entry->sch_no,
+		  __entry->mask,
+		  __entry->event)
+);
+
 TRACE_EVENT(vfio_ccw_fsm_async_request,
 	TP_PROTO(struct subchannel_id schid,
 		 int command,

Signature · 1 line

2.17.1

Cornelia Huck

2020-05-05 12:56 UTC #12 lore raw

In reply to Eric Farman

Quoted 7 lines · On Tue, 5 May 2020 14:27:37 +0200 Eric Farman <[email protected]> wrote:

Here is a new pass at the channel-path handling code for vfio-ccw. Changes from previous versions are recorded in git notes for each patch. Patches 5 through 7 got swizzled a little bit, in order to better compartmentalize the code they define. Basically, the IRQ definitions were moved from patch 7 to 5, and then patch 6 was placed ahead of patch 5.

Quoted 8 lines

I have put Conny's r-b's on patches 1, 3, 4, (new) 5, and 8, and believe I have addressed all comments from v3, with two exceptions:
Quoted 2 lines
I'm wondering if we should make this [vfio_ccw_schib_region_{write,release}] callback optional (not in this patch).
I have that implemented on top of this series, and will send later as part of a larger cleanup series.

Good, sounds reasonable.

Quoted 8 lines

Quoted 4 lines
One thing though that keeps coming up: do we need any kind of serialization? Can there be any confusion from concurrent reads from userspace, or are we sure that we always provide consistent data?
I _think_ this is in good shape, though as suggested another set of eyeballs would be nice. There is still a problem on the main interrupt/FSM path, which I'm not attempting to address here.

I'll try to think about it some more.

Quoted 35 lines

With this code plus the corresponding QEMU series (posted momentarily) applied I am able to configure off/on a CHPID (for example, by issuing "chchp -c 0/1 xx" on the host), and the guest is able to see both the events and reflect the updated path masks in its structures.
v3: https://lore.kernel.org/kvm/[email protected]/
v2: https://lore.kernel.org/kvm/[email protected]/
v1: https://lore.kernel.org/kvm/[email protected]/
Eric Farman (3):
vfio-ccw: Refactor the unregister of the async regions
vfio-ccw: Refactor IRQ handlers
vfio-ccw: Add trace for CRW event
Farhan Ali (5):
vfio-ccw: Introduce new helper functions to free/destroy regions
vfio-ccw: Register a chp_event callback for vfio-ccw
vfio-ccw: Introduce a new schib region
vfio-ccw: Introduce a new CRW region
vfio-ccw: Wire up the CRW irq and CRW region
 Documentation/s390/vfio-ccw.rst     |  38 ++++++-
 drivers/s390/cio/Makefile           |   2 +-
 drivers/s390/cio/vfio_ccw_chp.c     | 148 +++++++++++++++++++++++++
 drivers/s390/cio/vfio_ccw_drv.c     | 165 ++++++++++++++++++++++++++--
 drivers/s390/cio/vfio_ccw_ops.c     |  65 ++++++++---
 drivers/s390/cio/vfio_ccw_private.h |  16 +++
 drivers/s390/cio/vfio_ccw_trace.c   |   1 +
 drivers/s390/cio/vfio_ccw_trace.h   |  30 +++++
 include/uapi/linux/vfio.h           |   3 +
 include/uapi/linux/vfio_ccw.h       |  18 +++
 10 files changed, 458 insertions(+), 28 deletions(-)
 create mode 100644 drivers/s390/cio/vfio_ccw_chp.c

Eric Farman

author maintainer

2020-05-05 13:00 UTC #13 lore raw

Quoted 26 lines · Cornelia Huck

Quoted 7 lines · On Tue, 5 May 2020 14:27:37 +0200 Eric Farman <[email protected]> wrote:
Here is a new pass at the channel-path handling code for vfio-ccw. Changes from previous versions are recorded in git notes for each patch. Patches 5 through 7 got swizzled a little bit, in order to better compartmentalize the code they define. Basically, the IRQ definitions were moved from patch 7 to 5, and then patch 6 was placed ahead of patch 5.
Quoted 8 lines
I have put Conny's r-b's on patches 1, 3, 4, (new) 5, and 8, and believe I have addressed all comments from v3, with two exceptions:
Quoted 2 lines
I'm wondering if we should make this [vfio_ccw_schib_region_{write,release}] callback optional (not in this patch).
I have that implemented on top of this series, and will send later as part of a larger cleanup series.
Good, sounds reasonable.
Quoted 5 lines
Quoted 4 lines
One thing though that keeps coming up: do we need any kind of serialization? Can there be any confusion from concurrent reads from userspace, or are we sure that we always provide consistent data?

Quoted 5 lines

Quoted 3 lines
I _think_ this is in good shape, though as suggested another set of eyeballs would be nice. There is still a problem on the main interrupt/FSM path, which I'm not attempting to address here.
I'll try to think about it some more.

Re: interrupt/FSM, I now have two separate patches that each straighten things out on their own. And a handful of debug patches that probably only make things worse. :) I'll get one/both of those meaningful patches sent to the list so we can have that discussion separately from this code.

Quoted 37 lines

Quoted 36 lines
With this code plus the corresponding QEMU series (posted momentarily) applied I am able to configure off/on a CHPID (for example, by issuing "chchp -c 0/1 xx" on the host), and the guest is able to see both the events and reflect the updated path masks in its structures.
v3: https://lore.kernel.org/kvm/[email protected]/
v2: https://lore.kernel.org/kvm/[email protected]/
v1: https://lore.kernel.org/kvm/[email protected]/
Eric Farman (3):
vfio-ccw: Refactor the unregister of the async regions
vfio-ccw: Refactor IRQ handlers
vfio-ccw: Add trace for CRW event
Farhan Ali (5):
vfio-ccw: Introduce new helper functions to free/destroy regions
vfio-ccw: Register a chp_event callback for vfio-ccw
vfio-ccw: Introduce a new schib region
vfio-ccw: Introduce a new CRW region
vfio-ccw: Wire up the CRW irq and CRW region
 Documentation/s390/vfio-ccw.rst     |  38 ++++++-
 drivers/s390/cio/Makefile           |   2 +-
 drivers/s390/cio/vfio_ccw_chp.c     | 148 +++++++++++++++++++++++++
 drivers/s390/cio/vfio_ccw_drv.c     | 165 ++++++++++++++++++++++++++--
 drivers/s390/cio/vfio_ccw_ops.c     |  65 ++++++++---
 drivers/s390/cio/vfio_ccw_private.h |  16 +++
 drivers/s390/cio/vfio_ccw_trace.c   |   1 +
 drivers/s390/cio/vfio_ccw_trace.h   |  30 +++++
 include/uapi/linux/vfio.h           |   3 +
 include/uapi/linux/vfio_ccw.h       |  18 +++
 10 files changed, 458 insertions(+), 28 deletions(-)
 create mode 100644 drivers/s390/cio/vfio_ccw_chp.c

Cornelia Huck

2020-05-06 13:59 UTC #14 lore raw

Quoted 9 lines · On Tue, 5 May 2020 09:00:20 -0400 Eric Farman <[email protected]> wrote:

Quoted 8 lines · Cornelia Huck
Quoted 5 lines · On Tue, 5 May 2020 14:27:37 +0200 Eric Farman <[email protected]> wrote:
Quoted 4 lines
One thing though that keeps coming up: do we need any kind of serialization? Can there be any confusion from concurrent reads from userspace, or are we sure that we always provide consistent data?

Quoted 5 lines

Quoted 5 lines
Quoted 3 lines
I _think_ this is in good shape, though as suggested another set of eyeballs would be nice. There is still a problem on the main interrupt/FSM path, which I'm not attempting to address here.
I'll try to think about it some more.

I've convinced myself that the patches do not add any new races on top of what already might be lurking in the existing code.

Quoted 6 lines

Re: interrupt/FSM, I now have two separate patches that each straighten things out on their own. And a handful of debug patches that probably only make things worse. :) I'll get one/both of those meaningful patches sent to the list so we can have that discussion separately from this code.

Yes, let's do that on top.

I think that the series looks good now, but I'd still like to see someone else give it a quick look before I queue it.

Cornelia Huck

Applied

2020-05-18 10:06 UTC #15 lore raw

In reply to Eric Farman

Quoted 45 lines · On Tue, 5 May 2020 14:27:37 +0200 Eric Farman <[email protected]> wrote:

Here is a new pass at the channel-path handling code for vfio-ccw. Changes from previous versions are recorded in git notes for each patch. Patches 5 through 7 got swizzled a little bit, in order to better compartmentalize the code they define. Basically, the IRQ definitions were moved from patch 7 to 5, and then patch 6 was placed ahead of patch 5.
I have put Conny's r-b's on patches 1, 3, 4, (new) 5, and 8, and believe I have addressed all comments from v3, with two exceptions:
Quoted 2 lines
I'm wondering if we should make this [vfio_ccw_schib_region_{write,release}] callback optional (not in this patch).
I have that implemented on top of this series, and will send later as part of a larger cleanup series.
Quoted 3 lines
One thing though that keeps coming up: do we need any kind of serialization? Can there be any confusion from concurrent reads from userspace, or are we sure that we always provide consistent data?
I _think_ this is in good shape, though as suggested another set of eyeballs would be nice. There is still a problem on the main interrupt/FSM path, which I'm not attempting to address here.
With this code plus the corresponding QEMU series (posted momentarily) applied I am able to configure off/on a CHPID (for example, by issuing "chchp -c 0/1 xx" on the host), and the guest is able to see both the events and reflect the updated path masks in its structures.
v3: https://lore.kernel.org/kvm/[email protected]/
v2: https://lore.kernel.org/kvm/[email protected]/
v1: https://lore.kernel.org/kvm/[email protected]/
Eric Farman (3):
vfio-ccw: Refactor the unregister of the async regions
vfio-ccw: Refactor IRQ handlers
vfio-ccw: Add trace for CRW event
Farhan Ali (5):
vfio-ccw: Introduce new helper functions to free/destroy regions
vfio-ccw: Register a chp_event callback for vfio-ccw
vfio-ccw: Introduce a new schib region
vfio-ccw: Introduce a new CRW region
vfio-ccw: Wire up the CRW irq and CRW region

Quoted 13 lines

 Documentation/s390/vfio-ccw.rst     |  38 ++++++-
 drivers/s390/cio/Makefile           |   2 +-
 drivers/s390/cio/vfio_ccw_chp.c     | 148 +++++++++++++++++++++++++
 drivers/s390/cio/vfio_ccw_drv.c     | 165 ++++++++++++++++++++++++++--
 drivers/s390/cio/vfio_ccw_ops.c     |  65 ++++++++---
 drivers/s390/cio/vfio_ccw_private.h |  16 +++
 drivers/s390/cio/vfio_ccw_trace.c   |   1 +
 drivers/s390/cio/vfio_ccw_trace.h   |  30 +++++
 include/uapi/linux/vfio.h           |   3 +
 include/uapi/linux/vfio_ccw.h       |  18 +++
 10 files changed, 458 insertions(+), 28 deletions(-)
 create mode 100644 drivers/s390/cio/vfio_ccw_chp.c

Thanks, applied.

The documentation needed a bit of fiddling (please double-check), and I think we want to document error codes for the schib/crw regions as well. I can do that if I find time, but I'd also happily merge a patch.

Eric Farman

author maintainer

2020-05-18 10:14 UTC #16 lore raw

Quoted 62 lines · Cornelia Huck

Quoted 45 lines · On Tue, 5 May 2020 14:27:37 +0200 Eric Farman <[email protected]> wrote:
Here is a new pass at the channel-path handling code for vfio-ccw. Changes from previous versions are recorded in git notes for each patch. Patches 5 through 7 got swizzled a little bit, in order to better compartmentalize the code they define. Basically, the IRQ definitions were moved from patch 7 to 5, and then patch 6 was placed ahead of patch 5.
I have put Conny's r-b's on patches 1, 3, 4, (new) 5, and 8, and believe I have addressed all comments from v3, with two exceptions:
Quoted 2 lines
I'm wondering if we should make this [vfio_ccw_schib_region_{write,release}] callback optional (not in this patch).
I have that implemented on top of this series, and will send later as part of a larger cleanup series.
Quoted 3 lines
One thing though that keeps coming up: do we need any kind of serialization? Can there be any confusion from concurrent reads from userspace, or are we sure that we always provide consistent data?
I _think_ this is in good shape, though as suggested another set of eyeballs would be nice. There is still a problem on the main interrupt/FSM path, which I'm not attempting to address here.
With this code plus the corresponding QEMU series (posted momentarily) applied I am able to configure off/on a CHPID (for example, by issuing "chchp -c 0/1 xx" on the host), and the guest is able to see both the events and reflect the updated path masks in its structures.
v3: https://lore.kernel.org/kvm/[email protected]/
v2: https://lore.kernel.org/kvm/[email protected]/
v1: https://lore.kernel.org/kvm/[email protected]/
Eric Farman (3):
vfio-ccw: Refactor the unregister of the async regions
vfio-ccw: Refactor IRQ handlers
vfio-ccw: Add trace for CRW event
Farhan Ali (5):
vfio-ccw: Introduce new helper functions to free/destroy regions
vfio-ccw: Register a chp_event callback for vfio-ccw
vfio-ccw: Introduce a new schib region
vfio-ccw: Introduce a new CRW region
vfio-ccw: Wire up the CRW irq and CRW region
Quoted 13 lines
 Documentation/s390/vfio-ccw.rst     |  38 ++++++-
 drivers/s390/cio/Makefile           |   2 +-
 drivers/s390/cio/vfio_ccw_chp.c     | 148 +++++++++++++++++++++++++
 drivers/s390/cio/vfio_ccw_drv.c     | 165 ++++++++++++++++++++++++++--
 drivers/s390/cio/vfio_ccw_ops.c     |  65 ++++++++---
 drivers/s390/cio/vfio_ccw_private.h |  16 +++
 drivers/s390/cio/vfio_ccw_trace.c   |   1 +
 drivers/s390/cio/vfio_ccw_trace.h   |  30 +++++
 include/uapi/linux/vfio.h           |   3 +
 include/uapi/linux/vfio_ccw.h       |  18 +++
 10 files changed, 458 insertions(+), 28 deletions(-)
 create mode 100644 drivers/s390/cio/vfio_ccw_chp.c

Quoted 6 lines

Thanks, applied.
The documentation needed a bit of fiddling (please double-check), and I think we want to document error codes for the schib/crw regions as well. I can do that if I find time, but I'd also happily merge a patch.

Thank you!

I have a start for the error code documentation here. I'll finish it up in next day or two and doublecheck the fit of this one.