vfio-ccw: Fix garbage sense data on I/O error
cover vfio-ccw: Fix garbage sense data on I/O error
Hi Conny,
Per our offline discussion, here's a fix for the error when a guest issues "dasdfmt -M quick". It basically reverts commit 334e76850bbb ("vfio/ccw: update sense data if a unit check is pending") and modifies the check that builds sense data in the TSCH handler.
I opted to NOT disable PMCW.CSENSE, because doing so prevents vfio-ccw devices from coming online at all (didn't pursue deep enough to explain why). Turning it off in reaction to a unit check (in this now-reverted codepath) works, but only because of the corresponding PMCW.CSENSE check in the TSCH code.
I don't know if anything is needed for the (unaltered) ECW data that commit b498484ed49a ("s390x/css: sense data endianness") addressed for the copied sense_data bytes, but figure we can use this as a starting point. Thoughts?
Eric Farman (1):
vfio-ccw: Keep passthrough sense data intact
hw/s390x/css.c | 3 ++- hw/vfio/ccw.c | 6 ------ 2 files changed, 2 insertions(+), 7 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/1 vfio-ccw: Keep passthrough sense data intact
For virtual devices, there is space for sense data to be built and later copied into the IRB's ECW space once a TSCH is handled.
For passthrough devices, the IRB is passed up from hardware. There might already be sense data in the ECW, in which case it would be unusual to overwrite the IRB ESW/ECW data with itself.
In either case, if there isn't sense data, then an explicit SENSE operation might be needed from the guest driver. Regardless, fabricating sense data out of zeros seems like it would result in unexpected behavior.
Let's remove the copying of the ECW/sense data in the vfio-ccw driver, and adapt the check in the TSCH handler to look for non-zero data in the local sense data before building a sense data response in the IRB.
This fixes a "dasdfmt -M quick" command issued within a guest.
--- hw/s390x/css.c | 3 ++- hw/vfio/ccw.c | 6 ------ 2 files changed, 2 insertions(+), 7 deletions(-)
hw/s390x/css.c +2 -1
@@ -1661,7 +1661,8 @@ int css_do_tsch_get_irb(SubchDev *sch, IRB *target_irb, int *irb_len) }/* If a unit check is pending, copy sense data. */ if ((schib->scsw.dstat &SCSW_DSTAT_UNIT_CHECK ) && - (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE )) { + (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE ) && + (sch->sense_data[0 ] !=0 )) {int i; irb.scsw.flags |=SCSW_FLAGS_MASK_ESWF |SCSW_FLAGS_MASK_ECTL ;
hw/vfio/ccw.c +0 -6
@@ -371,12 +371,6 @@ static void vfio_ccw_io_notifier_handler(void *opaque)copy_scsw_to_guest (&s, &irb.scsw); schib->scsw = s; -/* If a uint check is pending, copy sense data. */ -if ((schib->scsw.dstat &SCSW_DSTAT_UNIT_CHECK ) && - (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE )) { -memcpy (sch->sense_data, irb.ecw,sizeof (irb.ecw)); - } - read_err:css_inject_io_interrupt (sch); }
Signature · 1 line
2.25.1
Quoted 6 lines · Eric Farman
Hi Conny,
Per our offline discussion, here's a fix for the error when a guest issues "dasdfmt -M quick". It basically reverts commit
334e76850bbb("vfio/ccw: update sense data if a unit check is pending") and modifies the check that builds sense data in the TSCH handler.
Should it includes a Fixes: tag then?
Quoted 19 lines
I opted to NOT disable PMCW.CSENSE, because doing so prevents vfio-ccw devices from coming online at all (didn't pursue deep enough to explain why). Turning it off in reaction to a unit check (in this now-reverted codepath) works, but only because of the corresponding PMCW.CSENSE check in the TSCH code.
I don't know if anything is needed for the (unaltered) ECW data that commit
b498484ed49a("s390x/css: sense data endianness") addressed for the copiedsense_databytes, but figure we can use this as a starting point. Thoughts?Eric Farman (1):
vfio-ccw: Keep passthrough sense data intacthw/s390x/css.c | 3 ++- hw/vfio/ccw.c | 6 ------ 2 files changed, 2 insertions(+), 7 deletions(-)
Quoted 9 lines · Matthew Rosato
Quoted 6 lines · Eric Farman
Hi Conny,
Per our offline discussion, here's a fix for the error when a guest issues "dasdfmt -M quick". It basically reverts commit
334e76850bbb("vfio/ccw: update sense data if a unit check is pending") and modifies the check that builds sense data in the TSCH handler.Should it includes a Fixes: tag then?
Yeah, probably. I'll fix it locally so it's prepped for a v2.
Quoted 19 lines
Quoted 19 lines
I opted to NOT disable PMCW.CSENSE, because doing so prevents vfio-ccw devices from coming online at all (didn't pursue deep enough to explain why). Turning it off in reaction to a unit check (in this now-reverted codepath) works, but only because of the corresponding PMCW.CSENSE check in the TSCH code.
I don't know if anything is needed for the (unaltered) ECW data that commit
b498484ed49a("s390x/css: sense data endianness") addressed for the copiedsense_databytes, but figure we can use this as a starting point. Thoughts?Eric Farman (1):
vfio-ccw: Keep passthrough sense data intacthw/s390x/css.c | 3 ++- hw/vfio/ccw.c | 6 ------ 2 files changed, 2 insertions(+), 7 deletions(-)
Quoted 7 lines · Eric Farman
Hi Conny,
Per our offline discussion, here's a fix for the error when a guest issues "dasdfmt -M quick". It basically reverts commit
334e76850bbb("vfio/ccw: update sense data if a unit check is pending") and modifies the check that builds sense data in the TSCH handler.
Quoted 5 lines
I opted to NOT disable PMCW.CSENSE, because doing so prevents vfio-ccw devices from coming online at all (didn't pursue deep enough to explain why). Turning it off in reaction to a unit check (in this now-reverted codepath) works, but only because of the corresponding PMCW.CSENSE check in the TSCH code.
It's a bit puzzling why fencing it off in msch doesn't work; but maybe it would not be the right thing to do anyway, if we can support some concurrent sense operations. I'm still reading the PoP.
Quoted 13 lines
I don't know if anything is needed for the (unaltered) ECW data that commit
b498484ed49a("s390x/css: sense data endianness") addressed for the copiedsense_databytes, but figure we can use this as a starting point. Thoughts?Eric Farman (1):
vfio-ccw: Keep passthrough sense data intacthw/s390x/css.c | 3 ++- hw/vfio/ccw.c | 6 ------ 2 files changed, 2 insertions(+), 7 deletions(-)
Re: [PATCH 1/1] vfio-ccw: Keep passthrough sense data intact
In reply to Eric FarmanQuoted 44 lines · Eric Farman
For virtual devices, there is space for sense data to be built and later copied into the IRB's ECW space once a TSCH is handled.
For passthrough devices, the IRB is passed up from hardware. There might already be sense data in the ECW, in which case it would be unusual to overwrite the IRB ESW/ECW data with itself.
In either case, if there isn't sense data, then an explicit SENSE operation might be needed from the guest driver. Regardless, fabricating sense data out of zeros seems like it would result in unexpected behavior.
Let's remove the copying of the ECW/sense data in the vfio-ccw driver, and adapt the check in the TSCH handler to look for non-zero data in the local sense data before building a sense data response in the IRB.
This fixes a "dasdfmt -M quick" command issued within a guest.
Signed-off-by Eric Farman <[email protected]>--- hw/s390x/css.c | 3 ++- hw/vfio/ccw.c | 6 ------ 2 files changed, 2 insertions(+), 7 deletions(-)hw/s390x/css.c +2
-1@@ -1661,7 +1661,8 @@ int css_do_tsch_get_irb(SubchDev *sch, IRB *target_irb, int *irb_len) }/* If a unit check is pending, copy sense data. */ if ((schib->scsw.dstat &SCSW_DSTAT_UNIT_CHECK ) && - (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE )) { + (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE ) && + (sch->sense_data[0 ] !=0 )) {int i; irb.scsw.flags |=SCSW_FLAGS_MASK_ESWF |SCSW_FLAGS_MASK_ECTL ;hw/vfio/ccw.c +0
-0
hw/vfio/ccw.c static void vfio_ccw_io_notifier_handler(void *opaque) · line 376
Quoted 6 lines
hw/vfio/ccw.c +0
-3@@ -371,12 +371,6 @@ static void vfio_ccw_io_notifier_handler(void *opaque)copy_scsw_to_guest (&s, &irb.scsw); schib->scsw = s; -/* If a uint check is pending, copy sense data. */ -if ((schib->scsw.dstat &SCSW_DSTAT_UNIT_CHECK ) && - (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE )) {
If I'm reading the PoP correctly, turning on concurrent sense only means that we may have sense data already available, but not that it's guaranteed. Would it be enough to look at the relevant bit in the erw and only copy sense data if it is actually set (here and/or above)?
Quoted 6 lines
hw/vfio/ccw.c +0
-3-memcpy (sch->sense_data, irb.ecw,sizeof (irb.ecw)); - } - read_err:css_inject_io_interrupt (sch); }
Re: [PATCH 1/1] vfio-ccw: Keep passthrough sense data intact
Quoted 14 lines · Cornelia Huck
Quoted 13 lines · Eric Farman
hw/s390x/css.c +2
-1@@ -1661,7 +1661,8 @@ int css_do_tsch_get_irb(SubchDev *sch, IRB *target_irb, int *irb_len) }/* If a unit check is pending, copy sense data. */ if ((schib->scsw.dstat &SCSW_DSTAT_UNIT_CHECK ) && - (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE )) { + (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE ) && + (sch->sense_data[0 ] !=0 )) {int i;
Quoted 2 lines
Quoted 2 lines
hw/s390x/css.c +0
-0irb.scsw.flags |= SCSW_FLAGS_MASK_ESWF |SCSW_FLAGS_MASK_ECTL;
This function is where we build the esw/ecw...
Quoted 11 lines
hw/vfio/ccw.c static void vfio_ccw_io_notifier_handler(void *opaque) · line 376
Quoted 11 lines
hw/vfio/ccw.c +0
-3@@ -371,12 +371,6 @@ static void vfio_ccw_io_notifier_handler(void *opaque)copy_scsw_to_guest (&s, &irb.scsw); schib->scsw = s; -/* If a uint check is pending, copy sense data. */ -if ((schib->scsw.dstat &SCSW_DSTAT_UNIT_CHECK ) && - (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE )) {
...and here we actually do have the esw/ecw provided by the hardware.
Quoted 5 lines
If I'm reading the PoP correctly, turning on concurrent sense only means that we may have sense data already available, but not that it's guaranteed. Would it be enough to look at the relevant bit in the erw and only copy sense data if it is actually set (here and/or above)?
Maybe the root of the problem is that we actually try to build the esw ourselves? If we copy it from the irb received by the hardware, we should already have the correct data, I think.
Quoted 7 lines
Quoted 7 lines
hw/vfio/ccw.c +0
-3-memcpy (sch->sense_data, irb.ecw,sizeof (irb.ecw)); - } - read_err:css_inject_io_interrupt (sch); }
Re: [PATCH 1/1] vfio-ccw: Keep passthrough sense data intact
Quoted 31 lines · Cornelia Huck
Quoted 15 lines · Cornelia Huck
Quoted 14 lines · Eric Farman
hw/s390x/css.c +2
-1@@ -1661,7 +1661,8 @@ int css_do_tsch_get_irb(SubchDev *sch, IRB *target_irb, int *irb_len) }/* If a unit check is pending, copy sense data. */ if ((schib->scsw.dstat &SCSW_DSTAT_UNIT_CHECK ) && - (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE )) { + (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE ) && + (sch->sense_data[0 ] !=0 )) {int i;Quoted 2 lines
Quoted 2 lines
irb.scsw.flags |= SCSW_FLAGS_MASK_ESWF | SCSW_FLAGS_MASK_ECTL;This function is where we build the esw/ecw...
Quoted 9 lines
Quoted 9 lines
hw/vfio/ccw.c +0
-0@@ -371,12 +371,6 @@ static void vfio_ccw_io_notifier_handler(void *opaque)copy_scsw_to_guest (&s, &irb.scsw); schib->scsw = s;
Quoted 10 lines
Quoted 3 lines
hw/vfio/ccw.c static void vfio_ccw_io_notifier_handler(void *opaque) · line 376
Quoted 3 lines
hw/vfio/ccw.c +0
-3-/* If a uint check is pending, copy sense data. */ -if ((schib->scsw.dstat &SCSW_DSTAT_UNIT_CHECK ) && - (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE )) {...and here we actually do have the esw/ecw provided by the hardware.
Quoted 4 lines
If I'm reading the PoP correctly, turning on concurrent sense only means
that we may have sense data already available, but not that it's guaranteed.
Agreed.
Quoted 2 lines
Quoted 2 lines
Would it be enough to look at the relevant bit in the erw
and only copy sense data if it is actually set (here and/or above)?
Do we have the hardware ERW in the css_do_tsch routine?
Oh, but we have SCSW, and POPS says if ERW.S is set, SCSW.E is set. So that would make this a pretty simple change then.
Quoted 5 lines
Maybe the root of the problem is that we actually try to build the esw
ourselves? If we copy it from the irb received by the hardware, we should already have the correct data, I think.
Yeah, that's part of the problem. As you note above, the PMCW.CSENSE bit only says if concurrent sense is possible, not that it was actually stored in the IRB.
I (mistakenly) thought that removing this hunk would get the whole IRB copied over, but I see now that css_do_tsch_get_irb() only copies the SCSW, and builds the ESW/ECW based off sch->sense_data.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
hw/vfio/ccw.c +0
-3-memcpy (sch->sense_data, irb.ecw,sizeof (irb.ecw)); - } - read_err:css_inject_io_interrupt (sch);}
Re: [PATCH 1/1] vfio-ccw: Keep passthrough sense data intact
Quoted 45 lines · Eric Farman
Quoted 31 lines · Cornelia Huck
Quoted 15 lines · Cornelia Huck
Quoted 14 lines · Eric Farman
hw/s390x/css.c +2
-1@@ -1661,7 +1661,8 @@ int css_do_tsch_get_irb(SubchDev *sch, IRB *target_irb, int *irb_len) }/* If a unit check is pending, copy sense data. */ if ((schib->scsw.dstat &SCSW_DSTAT_UNIT_CHECK ) && - (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE )) { + (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE ) && + (sch->sense_data[0 ] !=0 )) {int i;Quoted 2 lines
Quoted 2 lines
irb.scsw.flags |= SCSW_FLAGS_MASK_ESWF | SCSW_FLAGS_MASK_ECTL;This function is where we build the esw/ecw...
Quoted 9 lines
Quoted 9 lines
hw/vfio/ccw.c +0
-0@@ -371,12 +371,6 @@ static void vfio_ccw_io_notifier_handler(void *opaque)copy_scsw_to_guest (&s, &irb.scsw); schib->scsw = s;Quoted 10 lines
Quoted 3 lines
hw/vfio/ccw.c static void vfio_ccw_io_notifier_handler(void *opaque) · line 376
Quoted 3 lines
hw/vfio/ccw.c +0
-3-/* If a uint check is pending, copy sense data. */ -if ((schib->scsw.dstat &SCSW_DSTAT_UNIT_CHECK ) && - (schib->pmcw.chars &PMCW_CHARS_MASK_CSENSE )) {...and here we actually do have the esw/ecw provided by the hardware.
Quoted 4 lines
If I'm reading the PoP correctly, turning on concurrent sense only means
that we may have sense data already available, but not that it's guaranteed.Agreed.
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Would it be enough to look at the relevant bit in the erw
and only copy sense data if it is actually set (here and/or above)?Do we have the hardware ERW in the
css_do_tschroutine?Oh, but we have SCSW, and POPS says if
ERW.Sis set, SCSW.E is set. So that would make this a pretty simple change then.
Nod, that looks good.
Quoted 7 lines
Quoted 6 lines
Maybe the root of the problem is that we actually try to build the esw
ourselves? If we copy it from the irb received by the hardware, we should already have the correct data, I think.
Quoted 7 lines
Yeah, that's part of the problem. As you note above, the PMCW.CSENSE bit only says if concurrent sense is possible, not that it was actually stored in the IRB.
I (mistakenly) thought that removing this hunk would get the whole IRB copied over, but I see now that
css_do_tsch_get_irb()only copies the SCSW, and builds the ESW/ECW based offsch->sense_data.
Might be a good idea to go over what we pass through vs. what we emulate for vfio-ccw devices, in case we have more conditions like this. We probably should not overwrite information that we can just move guestward.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
hw/vfio/ccw.c +0
-3-memcpy (sch->sense_data, irb.ecw,sizeof (irb.ecw)); - } - read_err:css_inject_io_interrupt (sch);}