Skip to contentSkip to conversation

New state handling for VFIO CCW

PATCH In review Pierre Morel 3 participants 9 messages May 6, 2019 – May 16, 2019

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover New state handling for VFIO CCW

Hi,

I did not integrate all my patches for state handling like I had before but just two patches which seems interresting to me:

- The first one allows the device ti be used only when a guest is currently using it.
Otherwise the device is in NOT_OPER state

- The second rework the sch_event callback: AFAIU we can not consider that the event moves the device in IDLE state. I think we better let it as it is currently.

Regards,
Pierre

Pierre Morel (2):
vfio-ccw: Set subchannel state STANDBY on open
vfio-ccw: rework sch_event

 drivers/s390/cio/vfio_ccw_drv.c     | 21 ++-------------------
 drivers/s390/cio/vfio_ccw_fsm.c     |  7 +------
 drivers/s390/cio/vfio_ccw_ops.c     | 36 ++++++++++++++++++------------------
 drivers/s390/cio/vfio_ccw_private.h |  1 -
 4 files changed, 21 insertions(+), 44 deletions(-)
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/2 vfio-ccw: Set subchannel state STANDBY on open

When no guest is associated with the mediated device, i.e. the mediated device is not opened, the state of the mediated device is VFIO_CCW_STATE_NOT_OPER.

The subchannel enablement and the according setting to the VFIO_CCW_STATE_STANDBY state should only be done when all parts of the VFIO mediated device have been initialized i.e. after the mediated device has been successfully opened.

Let's stay in VFIO_CCW_STATE_NOT_OPER until the mediated device has been opened.

When the mediated device is closed, disable the sub channel by calling vfio_ccw_sch_quiesce() no reset needs to be done the mediated devce will be enable on next open.

---
 drivers/s390/cio/vfio_ccw_drv.c | 10 +---------
 drivers/s390/cio/vfio_ccw_ops.c | 36 ++++++++++++++++++------------------
 2 files changed, 19 insertions(+), 27 deletions(-)
drivers/s390/cio/vfio_ccw_drv.c +1 -9
index ee8767f..a95b6c7 100644
@@ -143,26 +143,18 @@ static int vfio_ccw_sch_probe(struct subchannel *sch)
 	dev_set_drvdata(&sch->dev, private);
 	mutex_init(&private->io_mutex);
 
-	spin_lock_irq(sch->lock);
 	private->state = VFIO_CCW_STATE_NOT_OPER;
 	sch->isc = VFIO_CCW_ISC;
-	ret = cio_enable_subchannel(sch, (u32)(unsigned long)sch);
-	spin_unlock_irq(sch->lock);
-	if (ret)
-		goto out_free;
 
 	INIT_WORK(&private->io_work, vfio_ccw_sch_io_todo);
 	atomic_set(&private->avail, 1);
-	private->state = VFIO_CCW_STATE_STANDBY;
 
 	ret = vfio_ccw_mdev_reg(sch);
 	if (ret)
-		goto out_disable;
+		goto out_free;
 
 	return 0;
 
-out_disable:
-	cio_disable_subchannel(sch);
 out_free:
 	dev_set_drvdata(&sch->dev, NULL);
 	if (private->cmd_region)
drivers/s390/cio/vfio_ccw_ops.c +18 -18
index 5eb6111..497419c 100644
@@ -115,14 +115,10 @@ static int vfio_ccw_mdev_create(struct kobject *kobj, struct mdev_device *mdev)
 	struct vfio_ccw_private *private =
 		dev_get_drvdata(mdev_parent_dev(mdev));
 
-	if (private->state == VFIO_CCW_STATE_NOT_OPER)
-		return -ENODEV;
-
 	if (atomic_dec_if_positive(&private->avail) < 0)
 		return -EPERM;
 
 	private->mdev = mdev;
-	private->state = VFIO_CCW_STATE_IDLE;
 
 	return 0;
 }
@@ -132,12 +128,7 @@ static int vfio_ccw_mdev_remove(struct mdev_device *mdev)
 	struct vfio_ccw_private *private =
 		dev_get_drvdata(mdev_parent_dev(mdev));
 
-	if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
-	    (private->state != VFIO_CCW_STATE_STANDBY)) {
-		if (!vfio_ccw_sch_quiesce(private->sch))
-			private->state = VFIO_CCW_STATE_STANDBY;
-		/* The state will be NOT_OPER on error. */
-	}
+	vfio_ccw_sch_quiesce(private->sch);
 
 	cp_free(&private->cp);
 	private->mdev = NULL;
@@ -151,6 +142,7 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
 	struct vfio_ccw_private *private =
 		dev_get_drvdata(mdev_parent_dev(mdev));
 	unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
+	struct subchannel *sch = private->sch;
 	int ret;
 
 	private->nb.notifier_call = vfio_ccw_mdev_notifier;
@@ -165,6 +157,20 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
 		vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
 					 &private->nb);
 	return ret;
+
+	spin_lock_irq(private->sch->lock);
+	if (cio_enable_subchannel(sch, (u32)(unsigned long)sch))
+		goto error;
+
+	private->state = VFIO_CCW_STATE_STANDBY;
+	spin_unlock_irq(sch->lock);
+	return 0;
+
+error:
+	spin_unlock_irq(sch->lock);
+	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+				 &private->nb);
+	return -EFAULT;
 }
 
 static void vfio_ccw_mdev_release(struct mdev_device *mdev)
@@ -173,20 +179,14 @@ static void vfio_ccw_mdev_release(struct mdev_device *mdev)
 		dev_get_drvdata(mdev_parent_dev(mdev));
 	int i;
 
-	if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
-	    (private->state != VFIO_CCW_STATE_STANDBY)) {
-		if (!vfio_ccw_mdev_reset(mdev))
-			private->state = VFIO_CCW_STATE_STANDBY;
-		/* The state will be NOT_OPER on error. */
-	}
-
-	cp_free(&private->cp);
+	vfio_ccw_sch_quiesce(private->sch);
 	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
 				 &private->nb);
 
 	for (i = 0; i < private->num_regions; i++)
 		private->region[i].ops->release(private, &private->region[i]);
 
+	cp_free(&private->cp);
 	private->num_regions = 0;
 	kfree(private->region);
 	private->region = NULL;
Signature · 1 line
2.7.4
#2 lore raw

Re: [PATCH v1 1/2] vfio-ccw: Set subchannel state STANDBY on open

Quoted 9 lines · Pierre Morel

When no guest is associated with the mediated device, i.e. the mediated device is not opened, the state of the mediated device is VFIO_CCW_STATE_NOT_OPER.

The subchannel enablement and the according setting to the VFIO_CCW_STATE_STANDBY state should only be done when all parts of the VFIO mediated device have been initialized i.e. after the mediated device has been successfully opened.

Quoted 6 lines

Let's stay in VFIO_CCW_STATE_NOT_OPER until the mediated device has been opened.

When the mediated device is closed, disable the sub channel by calling vfio_ccw_sch_quiesce() no reset needs to be done the mediated devce will be enable on next open.

s/devce/device

Quoted 81 lines
---
  drivers/s390/cio/vfio_ccw_drv.c | 10 +---------
  drivers/s390/cio/vfio_ccw_ops.c | 36 ++++++++++++++++++------------------
  2 files changed, 19 insertions(+), 27 deletions(-)
drivers/s390/cio/vfio_ccw_drv.c +1 -9
index ee8767f..a95b6c7 100644
@@ -143,26 +143,18 @@ static int vfio_ccw_sch_probe(struct subchannel *sch)
  	dev_set_drvdata(&sch->dev, private);
  	mutex_init(&private->io_mutex);
 
-	spin_lock_irq(sch->lock);
  	private->state = VFIO_CCW_STATE_NOT_OPER;
  	sch->isc = VFIO_CCW_ISC;
-	ret = cio_enable_subchannel(sch, (u32)(unsigned long)sch);
-	spin_unlock_irq(sch->lock);
-	if (ret)
-		goto out_free;
 
  	INIT_WORK(&private->io_work, vfio_ccw_sch_io_todo);
  	atomic_set(&private->avail, 1);
-	private->state = VFIO_CCW_STATE_STANDBY;
 
  	ret = vfio_ccw_mdev_reg(sch);
  	if (ret)
-		goto out_disable;
+		goto out_free;
 
  	return 0;
 
-out_disable:
-	cio_disable_subchannel(sch);
  out_free:
  	dev_set_drvdata(&sch->dev, NULL);
  	if (private->cmd_region)
drivers/s390/cio/vfio_ccw_ops.c +2 -10
index 5eb6111..497419c 100644
@@ -115,14 +115,10 @@ static int vfio_ccw_mdev_create(struct kobject *kobj, struct mdev_device *mdev)
  	struct vfio_ccw_private *private =
  		dev_get_drvdata(mdev_parent_dev(mdev));
 
-	if (private->state == VFIO_CCW_STATE_NOT_OPER)
-		return -ENODEV;
-
  	if (atomic_dec_if_positive(&private->avail) < 0)
  		return -EPERM;
 
  	private->mdev = mdev;
-	private->state = VFIO_CCW_STATE_IDLE;
 
  	return 0;
  }
@@ -132,12 +128,7 @@ static int vfio_ccw_mdev_remove(struct mdev_device *mdev)
  	struct vfio_ccw_private *private =
  		dev_get_drvdata(mdev_parent_dev(mdev));
 
-	if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
-	    (private->state != VFIO_CCW_STATE_STANDBY)) {
-		if (!vfio_ccw_sch_quiesce(private->sch))
-			private->state = VFIO_CCW_STATE_STANDBY;
-		/* The state will be NOT_OPER on error. */
-	}
+	vfio_ccw_sch_quiesce(private->sch);
 
  	cp_free(&private->cp);
  	private->mdev = NULL;
@@ -151,6 +142,7 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
  	struct vfio_ccw_private *private =
  		dev_get_drvdata(mdev_parent_dev(mdev));
  	unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
+	struct subchannel *sch = private->sch;
  	int ret;
 
  	private->nb.notifier_call = vfio_ccw_mdev_notifier;

drivers/s390/cio/vfio_ccw_ops.c static int vfio_ccw_mdev_open(struct mdev_device *mdev) · line 165

Quoted 9 lines
drivers/s390/cio/vfio_ccw_ops.c +6 -0
@@ -165,6 +157,20 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
  		vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
  					 &private->nb);
  	return ret;
+
+	spin_lock_irq(private->sch->lock);
+	if (cio_enable_subchannel(sch, (u32)(unsigned long)sch))
+		goto error;
+
+	private->state = VFIO_CCW_STATE_STANDBY;

I don't think we should set the state to STANDBY here, because with just this patch applied, any VFIO_CCW_EVENT_IO_REQ will return an error (due to fsm_io_error).

It might be safe to set it to IDLE in this patch.

Quoted 35 lines
drivers/s390/cio/vfio_ccw_ops.c +10 -8
+	spin_unlock_irq(sch->lock);
+	return 0;
+
+error:
+	spin_unlock_irq(sch->lock);
+	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+				 &private->nb);
+	return -EFAULT;
  }
 
  static void vfio_ccw_mdev_release(struct mdev_device *mdev)
@@ -173,20 +179,14 @@ static void vfio_ccw_mdev_release(struct mdev_device *mdev)
  		dev_get_drvdata(mdev_parent_dev(mdev));
  	int i;
 
-	if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
-	    (private->state != VFIO_CCW_STATE_STANDBY)) {
-		if (!vfio_ccw_mdev_reset(mdev))
-			private->state = VFIO_CCW_STATE_STANDBY;
-		/* The state will be NOT_OPER on error. */
-	}
-
-	cp_free(&private->cp);
+	vfio_ccw_sch_quiesce(private->sch);
  	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
  				 &private->nb);
 
  	for (i = 0; i < private->num_regions; i++)
  		private->region[i].ops->release(private, &private->region[i]);
 
+	cp_free(&private->cp);
  	private->num_regions = 0;
  	kfree(private->region);
  	private->region = NULL;
#3 lore raw

Re: [PATCH v1 1/2] vfio-ccw: Set subchannel state STANDBY on open

Quoted 26 lines · On Tue, 7 May 2019 15:44:54 -0400 Farhan Ali <[email protected]> wrote:
Quoted 9 lines · Pierre Morel

When no guest is associated with the mediated device, i.e. the mediated device is not opened, the state of the mediated device is VFIO_CCW_STATE_NOT_OPER.

The subchannel enablement and the according setting to the VFIO_CCW_STATE_STANDBY state should only be done when all parts of the VFIO mediated device have been initialized i.e. after the mediated device has been successfully opened.

Quoted 6 lines

Let's stay in VFIO_CCW_STATE_NOT_OPER until the mediated device has been opened.

When the mediated device is closed, disable the sub channel by calling vfio_ccw_sch_quiesce() no reset needs to be done the mediated devce will be enable on next open.

s/devce/device

Quoted 7 lines
---
  drivers/s390/cio/vfio_ccw_drv.c | 10 +---------
  drivers/s390/cio/vfio_ccw_ops.c | 36 ++++++++++++++++++------------------
  2 files changed, 19 insertions(+), 27 deletions(-)

(...)

Quoted 40 lines
Quoted 40 lines
drivers/s390/cio/vfio_ccw_ops.c +2 -10
index 5eb6111..497419c 100644
@@ -115,14 +115,10 @@ static int vfio_ccw_mdev_create(struct kobject *kobj, struct mdev_device *mdev)
  	struct vfio_ccw_private *private =
  		dev_get_drvdata(mdev_parent_dev(mdev));
 
-	if (private->state == VFIO_CCW_STATE_NOT_OPER)
-		return -ENODEV;
-
  	if (atomic_dec_if_positive(&private->avail) < 0)
  		return -EPERM;
 
  	private->mdev = mdev;
-	private->state = VFIO_CCW_STATE_IDLE;
 
  	return 0;
  }
@@ -132,12 +128,7 @@ static int vfio_ccw_mdev_remove(struct mdev_device *mdev)
  	struct vfio_ccw_private *private =
  		dev_get_drvdata(mdev_parent_dev(mdev));
 
-	if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
-	    (private->state != VFIO_CCW_STATE_STANDBY)) {
-		if (!vfio_ccw_sch_quiesce(private->sch))
-			private->state = VFIO_CCW_STATE_STANDBY;
-		/* The state will be NOT_OPER on error. */
-	}
+	vfio_ccw_sch_quiesce(private->sch);
 
  	cp_free(&private->cp);
  	private->mdev = NULL;
@@ -151,6 +142,7 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
  	struct vfio_ccw_private *private =
  		dev_get_drvdata(mdev_parent_dev(mdev));
  	unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
+	struct subchannel *sch = private->sch;
  	int ret;
 
Quoted 5 lines

drivers/s390/cio/vfio_ccw_ops.c static int vfio_ccw_mdev_open(struct mdev_device *mdev) · line 159

Quoted 5 lines
drivers/s390/cio/vfio_ccw_ops.c +0 -0
  	private->nb.notifier_call = vfio_ccw_mdev_notifier;
@@ -165,6 +157,20 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
  		vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
  					 &private->nb);
  	return ret;

I think this "return ret;" needs to go into the if branch above it; otherwise, the code below won't be reached :)

Quoted 11 lines

drivers/s390/cio/vfio_ccw_ops.c static int vfio_ccw_mdev_open(struct mdev_device *mdev) · line 165

Quoted 6 lines
drivers/s390/cio/vfio_ccw_ops.c +6 -0
+
+	spin_lock_irq(private->sch->lock);
+	if (cio_enable_subchannel(sch, (u32)(unsigned long)sch))
+		goto error;
+
+	private->state = VFIO_CCW_STATE_STANDBY;  

I don't think we should set the state to STANDBY here, because with just this patch applied, any VFIO_CCW_EVENT_IO_REQ will return an error (due to fsm_io_error).

Quoted 1 line

It might be safe to set it to IDLE in this patch.

Agreed, this should be IDLE; otherwise, I don't see how a device might move into IDLE state?

(That change happens in the next patch anyway.)

Quoted 29 lines
Quoted 29 lines
drivers/s390/cio/vfio_ccw_ops.c +9 -8
+	spin_unlock_irq(sch->lock);
+	return 0;
+
+error:
+	spin_unlock_irq(sch->lock);
+	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+				 &private->nb);
+	return -EFAULT;
  }
 
  static void vfio_ccw_mdev_release(struct mdev_device *mdev)
@@ -173,20 +179,14 @@ static void vfio_ccw_mdev_release(struct mdev_device *mdev)
  		dev_get_drvdata(mdev_parent_dev(mdev));
  	int i;
 
-	if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
-	    (private->state != VFIO_CCW_STATE_STANDBY)) {
-		if (!vfio_ccw_mdev_reset(mdev))
-			private->state = VFIO_CCW_STATE_STANDBY;
-		/* The state will be NOT_OPER on error. */
-	}
-
-	cp_free(&private->cp);
+	vfio_ccw_sch_quiesce(private->sch);
  	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
  				 &private->nb);
 
Quoted 4 lines

drivers/s390/cio/vfio_ccw_ops.c static void vfio_ccw_mdev_release(struct mdev_device *mdev) · line 189

Quoted 4 lines
drivers/s390/cio/vfio_ccw_ops.c +1 -0
  	for (i = 0; i < private->num_regions; i++)
  		private->region[i].ops->release(private, &private->region[i]);
 
+	cp_free(&private->cp);

I'm wondering why this cp_free is moved -- there should not be any activity related to it after quiesce, should there?

Quoted 5 lines
Quoted 4 lines
drivers/s390/cio/vfio_ccw_ops.c +0 -0
  	private->num_regions = 0;
  	kfree(private->region);
  	private->region = NULL;
#4 lore raw

Re: [PATCH v1 1/2] vfio-ccw: Set subchannel state STANDBY on open

Quoted 126 lines · Cornelia Huck
Quoted 26 lines · On Tue, 7 May 2019 15:44:54 -0400 Farhan Ali <[email protected]> wrote:
Quoted 9 lines · Pierre Morel

When no guest is associated with the mediated device, i.e. the mediated device is not opened, the state of the mediated device is VFIO_CCW_STATE_NOT_OPER.

The subchannel enablement and the according setting to the VFIO_CCW_STATE_STANDBY state should only be done when all parts of the VFIO mediated device have been initialized i.e. after the mediated device has been successfully opened.

Quoted 6 lines

Let's stay in VFIO_CCW_STATE_NOT_OPER until the mediated device has been opened.

When the mediated device is closed, disable the sub channel by calling vfio_ccw_sch_quiesce() no reset needs to be done the mediated devce will be enable on next open.

s/devce/device

Quoted 7 lines
---
   drivers/s390/cio/vfio_ccw_drv.c | 10 +---------
   drivers/s390/cio/vfio_ccw_ops.c | 36 ++++++++++++++++++------------------
   2 files changed, 19 insertions(+), 27 deletions(-)

(...)

Quoted 40 lines
Quoted 40 lines
drivers/s390/cio/vfio_ccw_ops.c +2 -10
index 5eb6111..497419c 100644
@@ -115,14 +115,10 @@ static int vfio_ccw_mdev_create(struct kobject *kobj, struct mdev_device *mdev)
   	struct vfio_ccw_private *private =
   		dev_get_drvdata(mdev_parent_dev(mdev));
 
-	if (private->state == VFIO_CCW_STATE_NOT_OPER)
-		return -ENODEV;
-
   	if (atomic_dec_if_positive(&private->avail) < 0)
   		return -EPERM;
 
   	private->mdev = mdev;
-	private->state = VFIO_CCW_STATE_IDLE;
 
   	return 0;
   }
@@ -132,12 +128,7 @@ static int vfio_ccw_mdev_remove(struct mdev_device *mdev)
   	struct vfio_ccw_private *private =
   		dev_get_drvdata(mdev_parent_dev(mdev));
 
-	if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
-	    (private->state != VFIO_CCW_STATE_STANDBY)) {
-		if (!vfio_ccw_sch_quiesce(private->sch))
-			private->state = VFIO_CCW_STATE_STANDBY;
-		/* The state will be NOT_OPER on error. */
-	}
+	vfio_ccw_sch_quiesce(private->sch);
 
   	cp_free(&private->cp);
   	private->mdev = NULL;
@@ -151,6 +142,7 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
   	struct vfio_ccw_private *private =
   		dev_get_drvdata(mdev_parent_dev(mdev));
   	unsigned long events = VFIO_IOMMU_NOTIFY_DMA_UNMAP;
+	struct subchannel *sch = private->sch;
   	int ret;
 
Quoted 5 lines

drivers/s390/cio/vfio_ccw_ops.c static int vfio_ccw_mdev_open(struct mdev_device *mdev) · line 159

Quoted 5 lines
drivers/s390/cio/vfio_ccw_ops.c +0 -0
   	private->nb.notifier_call = vfio_ccw_mdev_notifier;
@@ -165,6 +157,20 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
   		vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
   					 &private->nb);
   	return ret;

I think this "return ret;" needs to go into the if branch above it; otherwise, the code below won't be reached :)

Quoted 11 lines

drivers/s390/cio/vfio_ccw_ops.c static int vfio_ccw_mdev_open(struct mdev_device *mdev) · line 165

Quoted 6 lines
drivers/s390/cio/vfio_ccw_ops.c +6 -0
+
+	spin_lock_irq(private->sch->lock);
+	if (cio_enable_subchannel(sch, (u32)(unsigned long)sch))
+		goto error;
+
+	private->state = VFIO_CCW_STATE_STANDBY;

I don't think we should set the state to STANDBY here, because with just this patch applied, any VFIO_CCW_EVENT_IO_REQ will return an error (due to fsm_io_error).

Quoted 1 line

It might be safe to set it to IDLE in this patch.

Agreed, this should be IDLE; otherwise, I don't see how a device might move into IDLE state?

(That change happens in the next patch anyway.)

Quoted 29 lines
Quoted 29 lines
drivers/s390/cio/vfio_ccw_ops.c +9 -8
+	spin_unlock_irq(sch->lock);
+	return 0;
+
+error:
+	spin_unlock_irq(sch->lock);
+	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
+				 &private->nb);
+	return -EFAULT;
   }
 
   static void vfio_ccw_mdev_release(struct mdev_device *mdev)
@@ -173,20 +179,14 @@ static void vfio_ccw_mdev_release(struct mdev_device *mdev)
   		dev_get_drvdata(mdev_parent_dev(mdev));
   	int i;
 
-	if ((private->state != VFIO_CCW_STATE_NOT_OPER) &&
-	    (private->state != VFIO_CCW_STATE_STANDBY)) {
-		if (!vfio_ccw_mdev_reset(mdev))
-			private->state = VFIO_CCW_STATE_STANDBY;
-		/* The state will be NOT_OPER on error. */
-	}
-
-	cp_free(&private->cp);
+	vfio_ccw_sch_quiesce(private->sch);
   	vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY,
   				 &private->nb);
 
Quoted 7 lines
Quoted 4 lines

drivers/s390/cio/vfio_ccw_ops.c static void vfio_ccw_mdev_release(struct mdev_device *mdev) · line 189

Quoted 4 lines
drivers/s390/cio/vfio_ccw_ops.c +1 -0
   	for (i = 0; i < private->num_regions; i++)
   		private->region[i].ops->release(private, &private->region[i]);
 
+	cp_free(&private->cp);

I'm wondering why this cp_free is moved -- there should not be any activity related to it after quiesce, should there?

Yes there should.
I will let it where it was.

Quoted 7 lines
Quoted 6 lines
Quoted 5 lines
drivers/s390/cio/vfio_ccw_ops.c +0 -0
   	private->num_regions = 0;
   	kfree(private->region);
   	private->region = NULL;
Signature · 2 lines
Pierre Morel
Linux/KVM/QEMU in Böblingen - Germany
#5 lore raw

PATCH 2/2 vfio-ccw: rework sch_event

In reply to Pierre Morel

Set the mediated device as non operational when the subchannel went non operational.
Otherwise keep the current state.

Since we removed the last use of VFIO_CCW_STATE_STANDBY remove this state from the state machine.

---
 drivers/s390/cio/vfio_ccw_drv.c     | 11 +----------
 drivers/s390/cio/vfio_ccw_fsm.c     |  7 +------
 drivers/s390/cio/vfio_ccw_ops.c     |  2 +-
 drivers/s390/cio/vfio_ccw_private.h |  1 -
 4 files changed, 3 insertions(+), 18 deletions(-)
drivers/s390/cio/vfio_ccw_drv.c +1 -10
index a95b6c7..2f6140d5 100644
@@ -210,17 +210,8 @@ static int vfio_ccw_sch_event(struct subchannel *sch, int process)
 	if (work_pending(&sch->todo_work))
 		goto out_unlock;
 
-	if (cio_update_schib(sch)) {
+	if (cio_update_schib(sch))
 		vfio_ccw_fsm_event(private, VFIO_CCW_EVENT_NOT_OPER);
-		rc = 0;
-		goto out_unlock;
-	}
-
-	private = dev_get_drvdata(&sch->dev);
-	if (private->state == VFIO_CCW_STATE_NOT_OPER) {
-		private->state = private->mdev ? VFIO_CCW_STATE_IDLE :
-				 VFIO_CCW_STATE_STANDBY;
-	}
 	rc = 0;
 
 out_unlock:
drivers/s390/cio/vfio_ccw_fsm.c +1 -6
index 49d9d3d..a6524ca 100644
@@ -88,6 +88,7 @@ static int fsm_do_halt(struct vfio_ccw_private *private)
 
 	/* Issue "Halt Subchannel" */
 	ccode = hsch(sch->schid);
+	pr_warn("ccode = hsch(sch->schid);\n");
 
 	switch (ccode) {
 	case 0:
@@ -326,12 +327,6 @@ fsm_func_t *vfio_ccw_jumptable[NR_VFIO_CCW_STATES][NR_VFIO_CCW_EVENTS] = {
 		[VFIO_CCW_EVENT_ASYNC_REQ]	= fsm_async_error,
 		[VFIO_CCW_EVENT_INTERRUPT]	= fsm_disabled_irq,
 	},
-	[VFIO_CCW_STATE_STANDBY] = {
-		[VFIO_CCW_EVENT_NOT_OPER]	= fsm_notoper,
-		[VFIO_CCW_EVENT_IO_REQ]		= fsm_io_error,
-		[VFIO_CCW_EVENT_ASYNC_REQ]	= fsm_async_error,
-		[VFIO_CCW_EVENT_INTERRUPT]	= fsm_irq,
-	},
 	[VFIO_CCW_STATE_IDLE] = {
 		[VFIO_CCW_EVENT_NOT_OPER]	= fsm_notoper,
 		[VFIO_CCW_EVENT_IO_REQ]		= fsm_io_request,
drivers/s390/cio/vfio_ccw_ops.c +1 -1
index 497419c..35445ca 100644
@@ -162,7 +162,7 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
 	if (cio_enable_subchannel(sch, (u32)(unsigned long)sch))
 		goto error;
 
-	private->state = VFIO_CCW_STATE_STANDBY;
+	private->state = VFIO_CCW_STATE_IDLE;
 	spin_unlock_irq(sch->lock);
 	return 0;
 
drivers/s390/cio/vfio_ccw_private.h +0 -1
index f1092c3..ece6a75 100644
@@ -105,7 +105,6 @@ extern int vfio_ccw_sch_quiesce(struct subchannel *sch);
  */
 enum vfio_ccw_state {
 	VFIO_CCW_STATE_NOT_OPER,
-	VFIO_CCW_STATE_STANDBY,
 	VFIO_CCW_STATE_IDLE,
 	VFIO_CCW_STATE_CP_PROCESSING,
 	VFIO_CCW_STATE_CP_PENDING,
Signature · 1 line
2.7.4
#6 lore raw

Re: [PATCH v1 2/2] vfio-ccw: rework sch_event

Quoted 43 lines · Pierre Morel

Set the mediated device as non operational when the subchannel went non operational.
Otherwise keep the current state.

Since we removed the last use of VFIO_CCW_STATE_STANDBY remove this state from the state machine.

---
  drivers/s390/cio/vfio_ccw_drv.c     | 11 +----------
  drivers/s390/cio/vfio_ccw_fsm.c     |  7 +------
  drivers/s390/cio/vfio_ccw_ops.c     |  2 +-
  drivers/s390/cio/vfio_ccw_private.h |  1 -
  4 files changed, 3 insertions(+), 18 deletions(-)
drivers/s390/cio/vfio_ccw_drv.c +1 -10
index a95b6c7..2f6140d5 100644
@@ -210,17 +210,8 @@ static int vfio_ccw_sch_event(struct subchannel *sch, int process)
  	if (work_pending(&sch->todo_work))
  		goto out_unlock;
 
-	if (cio_update_schib(sch)) {
+	if (cio_update_schib(sch))
  		vfio_ccw_fsm_event(private, VFIO_CCW_EVENT_NOT_OPER);
-		rc = 0;
-		goto out_unlock;
-	}
-
-	private = dev_get_drvdata(&sch->dev);
-	if (private->state == VFIO_CCW_STATE_NOT_OPER) {
-		private->state = private->mdev ? VFIO_CCW_STATE_IDLE :
-				 VFIO_CCW_STATE_STANDBY;
-	}
  	rc = 0;
 
  out_unlock:
drivers/s390/cio/vfio_ccw_fsm.c +0 -0
index 49d9d3d..a6524ca 100644

drivers/s390/cio/vfio_ccw_fsm.c static int fsm_do_halt(struct vfio_ccw_private *private) · line 91

Quoted 4 lines
drivers/s390/cio/vfio_ccw_fsm.c +1 -0
@@ -88,6 +88,7 @@ static int fsm_do_halt(struct vfio_ccw_private *private)
 
  	/* Issue "Halt Subchannel" */
  	ccode = hsch(sch->schid);
+	pr_warn("ccode = hsch(sch->schid);\n");

I am not sure what's the purpose of logging this? Was it for your debugging purpose?

Quoted 42 lines
drivers/s390/cio/vfio_ccw_fsm.c +0 -6
  	switch (ccode) {
  	case 0:
@@ -326,12 +327,6 @@ fsm_func_t *vfio_ccw_jumptable[NR_VFIO_CCW_STATES][NR_VFIO_CCW_EVENTS] = {
  		[VFIO_CCW_EVENT_ASYNC_REQ]	= fsm_async_error,
  		[VFIO_CCW_EVENT_INTERRUPT]	= fsm_disabled_irq,
  	},
-	[VFIO_CCW_STATE_STANDBY] = {
-		[VFIO_CCW_EVENT_NOT_OPER]	= fsm_notoper,
-		[VFIO_CCW_EVENT_IO_REQ]		= fsm_io_error,
-		[VFIO_CCW_EVENT_ASYNC_REQ]	= fsm_async_error,
-		[VFIO_CCW_EVENT_INTERRUPT]	= fsm_irq,
-	},
  	[VFIO_CCW_STATE_IDLE] = {
  		[VFIO_CCW_EVENT_NOT_OPER]	= fsm_notoper,
  		[VFIO_CCW_EVENT_IO_REQ]		= fsm_io_request,
drivers/s390/cio/vfio_ccw_ops.c +1 -1
index 497419c..35445ca 100644
@@ -162,7 +162,7 @@ static int vfio_ccw_mdev_open(struct mdev_device *mdev)
  	if (cio_enable_subchannel(sch, (u32)(unsigned long)sch))
  		goto error;
 
-	private->state = VFIO_CCW_STATE_STANDBY;
+	private->state = VFIO_CCW_STATE_IDLE;
  	spin_unlock_irq(sch->lock);
  	return 0;
 
drivers/s390/cio/vfio_ccw_private.h +0 -1
index f1092c3..ece6a75 100644
@@ -105,7 +105,6 @@ extern int vfio_ccw_sch_quiesce(struct subchannel *sch);
   */
  enum vfio_ccw_state {
  	VFIO_CCW_STATE_NOT_OPER,
-	VFIO_CCW_STATE_STANDBY,
  	VFIO_CCW_STATE_IDLE,
  	VFIO_CCW_STATE_CP_PROCESSING,
  	VFIO_CCW_STATE_CP_PENDING,
#7 lore raw
In reply to Pierre Morel
Quoted 5 lines · On Mon, 6 May 2019 15:11:08 +0200 Pierre Morel <[email protected]> wrote:

Hi,

I did not integrate all my patches for state handling like I had before but just two patches which seems interresting to me:

Quoted 7 lines

- The first one allows the device ti be used only when a guest is currently using it.
Otherwise the device is in NOT_OPER state

- The second rework the sch_event callback: AFAIU we can not consider that the event moves the device in IDLE state. I think we better let it as it is currently.

I agree with the direction of this patch series.

Quoted 14 lines

Regards,
Pierre

Pierre Morel (2):
vfio-ccw: Set subchannel state STANDBY on open
vfio-ccw: rework sch_event

 drivers/s390/cio/vfio_ccw_drv.c     | 21 ++-------------------
 drivers/s390/cio/vfio_ccw_fsm.c     |  7 +------
 drivers/s390/cio/vfio_ccw_ops.c     | 36 ++++++++++++++++++------------------
 drivers/s390/cio/vfio_ccw_private.h |  1 -
 4 files changed, 21 insertions(+), 44 deletions(-)
#8 lore raw
Quoted 12 lines · Cornelia Huck
Quoted 9 lines · On Mon, 6 May 2019 15:11:08 +0200 Pierre Morel <[email protected]> wrote:

Hi,

I did not integrate all my patches for state handling like I had before but just two patches which seems interresting to me:

- The first one allows the device ti be used only when a guest is currently using it.
Otherwise the device is in NOT_OPER state

Quoted 21 lines
Quoted 3 lines

- The second rework the sch_event callback: AFAIU we can not consider that the event moves the device in IDLE state. I think we better let it as it is currently.

I agree with the direction of this patch series.

Quoted 14 lines

Regards,
Pierre

Pierre Morel (2):
vfio-ccw: Set subchannel state STANDBY on open
vfio-ccw: rework sch_event

  drivers/s390/cio/vfio_ccw_drv.c     | 21 ++-------------------
  drivers/s390/cio/vfio_ccw_fsm.c     |  7 +------
  drivers/s390/cio/vfio_ccw_ops.c     | 36 ++++++++++++++++++------------------
  drivers/s390/cio/vfio_ccw_private.h |  1 -
  4 files changed, 21 insertions(+), 44 deletions(-)

Thanks,
I will post a v2 with the corrections seen by you and Farhan,

Regards,
Pierre

Signature · 2 lines
Pierre Morel
Linux/KVM/QEMU in Böblingen - Germany