Skip to contentSkip to conversation

vfio-ccw: Fix garbage sense data on I/O error

PATCH In review Eric Farman 3 participants 9 messages Jun 10, 2021 – Jun 11, 2021

lore
Share & tools
lore
#0 lore raw

cover vfio-ccw: Fix garbage sense data on I/O error

Hi Conny,

Per our offline discussion, here's a fix for the error when a guest issues "dasdfmt -M quick". It basically reverts commit 334e76850bbb ("vfio/ccw: update sense data if a unit check is pending") and modifies the check that builds sense data in the TSCH handler.

I opted to NOT disable PMCW.CSENSE, because doing so prevents vfio-ccw devices from coming online at all (didn't pursue deep enough to explain why). Turning it off in reaction to a unit check (in this now-reverted codepath) works, but only because of the corresponding PMCW.CSENSE check in the TSCH code.

I don't know if anything is needed for the (unaltered) ECW data that commit b498484ed49a ("s390x/css: sense data endianness") addressed for the copied sense_data bytes, but figure we can use this as a starting point. Thoughts?

Eric Farman (1):
vfio-ccw: Keep passthrough sense data intact

 hw/s390x/css.c | 3 ++-
 hw/vfio/ccw.c  | 6 ------
 2 files changed, 2 insertions(+), 7 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/1 vfio-ccw: Keep passthrough sense data intact

For virtual devices, there is space for sense data to be built and later copied into the IRB's ECW space once a TSCH is handled.

For passthrough devices, the IRB is passed up from hardware. There might already be sense data in the ECW, in which case it would be unusual to overwrite the IRB ESW/ECW data with itself.

In either case, if there isn't sense data, then an explicit SENSE operation might be needed from the guest driver. Regardless, fabricating sense data out of zeros seems like it would result in unexpected behavior.

Let's remove the copying of the ECW/sense data in the vfio-ccw driver, and adapt the check in the TSCH handler to look for non-zero data in the local sense data before building a sense data response in the IRB.

This fixes a "dasdfmt -M quick" command issued within a guest.

---
 hw/s390x/css.c | 3 ++-
 hw/vfio/ccw.c  | 6 ------
 2 files changed, 2 insertions(+), 7 deletions(-)
hw/s390x/css.c +2 -1
index bed46f5ec3..29234daa27 100644
@@ -1661,7 +1661,8 @@ int css_do_tsch_get_irb(SubchDev *sch, IRB *target_irb, int *irb_len)
         }
         /* If a unit check is pending, copy sense data. */
         if ((schib->scsw.dstat & SCSW_DSTAT_UNIT_CHECK) &&
-            (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE)) {
+            (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE) &&
+            (sch->sense_data[0] != 0)) {
             int i;
 
             irb.scsw.flags |= SCSW_FLAGS_MASK_ESWF | SCSW_FLAGS_MASK_ECTL;
hw/vfio/ccw.c +0 -6
index 139a3d9d1b..a4dc4acb34 100644
@@ -371,12 +371,6 @@ static void vfio_ccw_io_notifier_handler(void *opaque)
     copy_scsw_to_guest(&s, &irb.scsw);
     schib->scsw = s;
 
-    /* If a uint check is pending, copy sense data. */
-    if ((schib->scsw.dstat & SCSW_DSTAT_UNIT_CHECK) &&
-        (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE)) {
-        memcpy(sch->sense_data, irb.ecw, sizeof(irb.ecw));
-    }
-
 read_err:
     css_inject_io_interrupt(sch);
 }
Signature · 1 line
2.25.1
#6 lore raw
In reply to Eric Farman
Quoted 6 lines · Eric Farman

Hi Conny,

Per our offline discussion, here's a fix for the error when a guest issues "dasdfmt -M quick". It basically reverts commit 334e76850bbb ("vfio/ccw: update sense data if a unit check is pending") and modifies the check that builds sense data in the TSCH handler.

Should it includes a Fixes: tag then?

Quoted 19 lines

I opted to NOT disable PMCW.CSENSE, because doing so prevents vfio-ccw devices from coming online at all (didn't pursue deep enough to explain why). Turning it off in reaction to a unit check (in this now-reverted codepath) works, but only because of the corresponding PMCW.CSENSE check in the TSCH code.

I don't know if anything is needed for the (unaltered) ECW data that commit b498484ed49a ("s390x/css: sense data endianness") addressed for the copied sense_data bytes, but figure we can use this as a starting point. Thoughts?

Eric Farman (1):
vfio-ccw: Keep passthrough sense data intact

  hw/s390x/css.c | 3 ++-
  hw/vfio/ccw.c  | 6 ------
  2 files changed, 2 insertions(+), 7 deletions(-)
#7 lore raw
Quoted 9 lines · Matthew Rosato
Quoted 6 lines · Eric Farman

Hi Conny,

Per our offline discussion, here's a fix for the error when a guest issues "dasdfmt -M quick". It basically reverts commit 334e76850bbb ("vfio/ccw: update sense data if a unit check is pending") and modifies the check that builds sense data in the TSCH handler.

Should it includes a Fixes: tag then?

Yeah, probably. I'll fix it locally so it's prepped for a v2.

Quoted 19 lines
Quoted 19 lines

I opted to NOT disable PMCW.CSENSE, because doing so prevents vfio-ccw devices from coming online at all (didn't pursue deep enough to explain why). Turning it off in reaction to a unit check (in this now-reverted codepath) works, but only because of the corresponding PMCW.CSENSE check in the TSCH code.

I don't know if anything is needed for the (unaltered) ECW data that commit b498484ed49a ("s390x/css: sense data endianness") addressed for the copied sense_data bytes, but figure we can use this as a starting point. Thoughts?

Eric Farman (1):
vfio-ccw: Keep passthrough sense data intact

  hw/s390x/css.c | 3 ++-
  hw/vfio/ccw.c  | 6 ------
  2 files changed, 2 insertions(+), 7 deletions(-)
#8 lore raw
In reply to Eric Farman
Quoted 7 lines · Eric Farman

Hi Conny,

Per our offline discussion, here's a fix for the error when a guest issues "dasdfmt -M quick". It basically reverts commit 334e76850bbb ("vfio/ccw: update sense data if a unit check is pending") and modifies the check that builds sense data in the TSCH handler.

Quoted 5 lines

I opted to NOT disable PMCW.CSENSE, because doing so prevents vfio-ccw devices from coming online at all (didn't pursue deep enough to explain why). Turning it off in reaction to a unit check (in this now-reverted codepath) works, but only because of the corresponding PMCW.CSENSE check in the TSCH code.

It's a bit puzzling why fencing it off in msch doesn't work; but maybe it would not be the right thing to do anyway, if we can support some concurrent sense operations. I'm still reading the PoP.

Quoted 13 lines

I don't know if anything is needed for the (unaltered) ECW data that commit b498484ed49a ("s390x/css: sense data endianness") addressed for the copied sense_data bytes, but figure we can use this as a starting point. Thoughts?

Eric Farman (1):
vfio-ccw: Keep passthrough sense data intact

 hw/s390x/css.c | 3 ++-
 hw/vfio/ccw.c  | 6 ------
 2 files changed, 2 insertions(+), 7 deletions(-)
#2 lore raw

Re: [PATCH 1/1] vfio-ccw: Keep passthrough sense data intact

In reply to Eric Farman
Quoted 44 lines · Eric Farman

For virtual devices, there is space for sense data to be built and later copied into the IRB's ECW space once a TSCH is handled.

For passthrough devices, the IRB is passed up from hardware. There might already be sense data in the ECW, in which case it would be unusual to overwrite the IRB ESW/ECW data with itself.

In either case, if there isn't sense data, then an explicit SENSE operation might be needed from the guest driver. Regardless, fabricating sense data out of zeros seems like it would result in unexpected behavior.

Let's remove the copying of the ECW/sense data in the vfio-ccw driver, and adapt the check in the TSCH handler to look for non-zero data in the local sense data before building a sense data response in the IRB.

This fixes a "dasdfmt -M quick" command issued within a guest.

---
 hw/s390x/css.c | 3 ++-
 hw/vfio/ccw.c  | 6 ------
 2 files changed, 2 insertions(+), 7 deletions(-)
hw/s390x/css.c +2 -1
index bed46f5ec3..29234daa27 100644
@@ -1661,7 +1661,8 @@ int css_do_tsch_get_irb(SubchDev *sch, IRB *target_irb, int *irb_len)
         }
         /* If a unit check is pending, copy sense data. */
         if ((schib->scsw.dstat & SCSW_DSTAT_UNIT_CHECK) &&
-            (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE)) {
+            (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE) &&
+            (sch->sense_data[0] != 0)) {
             int i;
 
             irb.scsw.flags |= SCSW_FLAGS_MASK_ESWF | SCSW_FLAGS_MASK_ECTL;
hw/vfio/ccw.c +0 -0
index 139a3d9d1b..a4dc4acb34 100644

hw/vfio/ccw.c static void vfio_ccw_io_notifier_handler(void *opaque) · line 376

Quoted 6 lines
hw/vfio/ccw.c +0 -3
@@ -371,12 +371,6 @@ static void vfio_ccw_io_notifier_handler(void *opaque)
     copy_scsw_to_guest(&s, &irb.scsw);
     schib->scsw = s;
 
-    /* If a uint check is pending, copy sense data. */
-    if ((schib->scsw.dstat & SCSW_DSTAT_UNIT_CHECK) &&
-        (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE)) {

If I'm reading the PoP correctly, turning on concurrent sense only means that we may have sense data already available, but not that it's guaranteed. Would it be enough to look at the relevant bit in the erw and only copy sense data if it is actually set (here and/or above)?

Quoted 6 lines
hw/vfio/ccw.c +0 -3
-        memcpy(sch->sense_data, irb.ecw, sizeof(irb.ecw));
-    }
-
 read_err:
     css_inject_io_interrupt(sch);
 }
#3 lore raw

Re: [PATCH 1/1] vfio-ccw: Keep passthrough sense data intact

Quoted 14 lines · Cornelia Huck
Quoted 13 lines · Eric Farman
hw/s390x/css.c +2 -1
index bed46f5ec3..29234daa27 100644
@@ -1661,7 +1661,8 @@ int css_do_tsch_get_irb(SubchDev *sch, IRB *target_irb, int *irb_len)
         }
         /* If a unit check is pending, copy sense data. */
         if ((schib->scsw.dstat & SCSW_DSTAT_UNIT_CHECK) &&
-            (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE)) {
+            (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE) &&
+            (sch->sense_data[0] != 0)) {
             int i;
 
Quoted 2 lines
Quoted 2 lines
hw/s390x/css.c +0 -0
             irb.scsw.flags |= SCSW_FLAGS_MASK_ESWF |
             SCSW_FLAGS_MASK_ECTL;

This function is where we build the esw/ecw...

Quoted 11 lines

hw/vfio/ccw.c static void vfio_ccw_io_notifier_handler(void *opaque) · line 376

Quoted 11 lines
hw/vfio/ccw.c +0 -3
index 139a3d9d1b..a4dc4acb34 100644
@@ -371,12 +371,6 @@ static void vfio_ccw_io_notifier_handler(void *opaque)
     copy_scsw_to_guest(&s, &irb.scsw);
     schib->scsw = s;
 
-    /* If a uint check is pending, copy sense data. */
-    if ((schib->scsw.dstat & SCSW_DSTAT_UNIT_CHECK) &&
-        (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE)) {

...and here we actually do have the esw/ecw provided by the hardware.

Quoted 5 lines

If I'm reading the PoP correctly, turning on concurrent sense only means that we may have sense data already available, but not that it's guaranteed. Would it be enough to look at the relevant bit in the erw and only copy sense data if it is actually set (here and/or above)?

Maybe the root of the problem is that we actually try to build the esw ourselves? If we copy it from the irb received by the hardware, we should already have the correct data, I think.

Quoted 7 lines
Quoted 7 lines
hw/vfio/ccw.c +0 -3
-        memcpy(sch->sense_data, irb.ecw, sizeof(irb.ecw));
-    }
-
 read_err:
     css_inject_io_interrupt(sch);
 }
#4 lore raw

Re: [PATCH 1/1] vfio-ccw: Keep passthrough sense data intact

Quoted 31 lines · Cornelia Huck
Quoted 15 lines · Cornelia Huck
Quoted 14 lines · Eric Farman
hw/s390x/css.c +2 -1
index bed46f5ec3..29234daa27 100644
@@ -1661,7 +1661,8 @@ int css_do_tsch_get_irb(SubchDev *sch, IRB
 *target_irb, int *irb_len)
         }
         /* If a unit check is pending, copy sense data. */
         if ((schib->scsw.dstat & SCSW_DSTAT_UNIT_CHECK) &&
-            (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE)) {
+            (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE) &&
+            (sch->sense_data[0] != 0)) {
             int i;
 
Quoted 2 lines
Quoted 2 lines
             irb.scsw.flags |= SCSW_FLAGS_MASK_ESWF |
             SCSW_FLAGS_MASK_ECTL;

This function is where we build the esw/ecw...

Quoted 9 lines
Quoted 9 lines
hw/vfio/ccw.c +0 -0
index 139a3d9d1b..a4dc4acb34 100644
@@ -371,12 +371,6 @@ static void
 vfio_ccw_io_notifier_handler(void *opaque)
     copy_scsw_to_guest(&s, &irb.scsw);
     schib->scsw = s;
 
Quoted 10 lines
Quoted 3 lines

hw/vfio/ccw.c static void vfio_ccw_io_notifier_handler(void *opaque) · line 376

Quoted 3 lines
hw/vfio/ccw.c +0 -3
-    /* If a uint check is pending, copy sense data. */
-    if ((schib->scsw.dstat & SCSW_DSTAT_UNIT_CHECK) &&
-        (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE)) {

...and here we actually do have the esw/ecw provided by the hardware.

Quoted 4 lines

If I'm reading the PoP correctly, turning on concurrent sense only means
that we may have sense data already available, but not that it's guaranteed.

Agreed.

Quoted 2 lines
Quoted 2 lines

Would it be enough to look at the relevant bit in the erw
and only copy sense data if it is actually set (here and/or above)?

Do we have the hardware ERW in the css_do_tsch routine?

Oh, but we have SCSW, and POPS says if ERW.S is set, SCSW.E is set. So that would make this a pretty simple change then.

Quoted 5 lines

Maybe the root of the problem is that we actually try to build the esw
ourselves? If we copy it from the irb received by the hardware, we should already have the correct data, I think.

Yeah, that's part of the problem. As you note above, the PMCW.CSENSE bit only says if concurrent sense is possible, not that it was actually stored in the IRB.

I (mistakenly) thought that removing this hunk would get the whole IRB copied over, but I see now that css_do_tsch_get_irb() only copies the SCSW, and builds the ESW/ECW based off sch->sense_data.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
hw/vfio/ccw.c +0 -3
-        memcpy(sch->sense_data, irb.ecw, sizeof(irb.ecw));
-    }
-
 read_err:
     css_inject_io_interrupt(sch);

}

#5 lore raw

Re: [PATCH 1/1] vfio-ccw: Keep passthrough sense data intact

Quoted 45 lines · Eric Farman
Quoted 31 lines · Cornelia Huck
Quoted 15 lines · Cornelia Huck
Quoted 14 lines · Eric Farman
hw/s390x/css.c +2 -1
index bed46f5ec3..29234daa27 100644
@@ -1661,7 +1661,8 @@ int css_do_tsch_get_irb(SubchDev *sch, IRB
 *target_irb, int *irb_len)
         }
         /* If a unit check is pending, copy sense data. */
         if ((schib->scsw.dstat & SCSW_DSTAT_UNIT_CHECK) &&
-            (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE)) {
+            (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE) &&
+            (sch->sense_data[0] != 0)) {
             int i;
 
Quoted 2 lines
Quoted 2 lines
             irb.scsw.flags |= SCSW_FLAGS_MASK_ESWF |
             SCSW_FLAGS_MASK_ECTL;

This function is where we build the esw/ecw...

Quoted 9 lines
Quoted 9 lines
hw/vfio/ccw.c +0 -0
index 139a3d9d1b..a4dc4acb34 100644
@@ -371,12 +371,6 @@ static void
 vfio_ccw_io_notifier_handler(void *opaque)
     copy_scsw_to_guest(&s, &irb.scsw);
     schib->scsw = s;
 
Quoted 10 lines
Quoted 3 lines

hw/vfio/ccw.c static void vfio_ccw_io_notifier_handler(void *opaque) · line 376

Quoted 3 lines
hw/vfio/ccw.c +0 -3
-    /* If a uint check is pending, copy sense data. */
-    if ((schib->scsw.dstat & SCSW_DSTAT_UNIT_CHECK) &&
-        (schib->pmcw.chars & PMCW_CHARS_MASK_CSENSE)) {

...and here we actually do have the esw/ecw provided by the hardware.

Quoted 4 lines

If I'm reading the PoP correctly, turning on concurrent sense only means
that we may have sense data already available, but not that it's guaranteed.

Agreed.

Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

Would it be enough to look at the relevant bit in the erw
and only copy sense data if it is actually set (here and/or above)?

Do we have the hardware ERW in the css_do_tsch routine?

Oh, but we have SCSW, and POPS says if ERW.S is set, SCSW.E is set. So that would make this a pretty simple change then.

Nod, that looks good.

Quoted 7 lines
Quoted 6 lines

Maybe the root of the problem is that we actually try to build the esw
ourselves? If we copy it from the irb received by the hardware, we should already have the correct data, I think.

Quoted 7 lines

Yeah, that's part of the problem. As you note above, the PMCW.CSENSE bit only says if concurrent sense is possible, not that it was actually stored in the IRB.

I (mistakenly) thought that removing this hunk would get the whole IRB copied over, but I see now that css_do_tsch_get_irb() only copies the SCSW, and builds the ESW/ECW based off sch->sense_data.

Might be a good idea to go over what we pass through vs. what we emulate for vfio-ccw devices, in case we have more conditions like this. We probably should not overwrite information that we can just move guestward.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
hw/vfio/ccw.c +0 -3
-        memcpy(sch->sense_data, irb.ecw, sizeof(irb.ecw));
-    }
-
 read_err:
     css_inject_io_interrupt(sch);

}