s390x/vfio-ccw: Channel Path Handling [QEMU]
v4 cover s390x/vfio-ccw: Channel Path Handling [QEMU]
Here is a new pass at the channel-path handling code for vfio-ccw, to take advantage of the corresponding kernel patches posted here:
https://lore.kernel.org/kvm/[email protected]/
I have all the main comments from v3 addressed, though I have a couple of additional patches that do some further cleanups (like, a generic callback for stsch) that weren't baked enough to include here. They're working fine, but need some cleanups (e.g., fixups vs standalone patches) before they're ready for submission. Just wanted to have the basic QEMU code to go with the new KVM series.
Changes are listed in git notes for the individual patches.
v3: https://lore.kernel.org/qemu-devel/[email protected]/
v2: https://lore.kernel.org/qemu-devel/[email protected]/
v1: https://lore.kernel.org/qemu-devel/[email protected]/
Eric Farman (3):
vfio-ccw: Refactor cleanup of regions
vfio-ccw: Refactor ccw irq handler
s390x/css: Refactor the css_queue_crw() routine
Farhan Ali (3):
linux-headers: update
vfio-ccw: Add support for the schib region
vfio-ccw: Add support for the CRW region and IRQ
hw/s390x/css.c | 57 ++++++--- hw/s390x/s390-ccw.c | 21 ++++ hw/vfio/ccw.c | 208 +++++++++++++++++++++++++++++---- include/hw/s390x/css.h | 4 +- include/hw/s390x/s390-ccw.h | 1 + linux-headers/linux/vfio.h | 3 + linux-headers/linux/vfio_ccw.h | 18 +++ target/s390x/ioinst.c | 3 +- 8 files changed, 274 insertions(+), 41 deletions(-)
Signature · 1 line
2.17.1
PATCH v4 1/6 linux-headers: update
From: Farhan Ali <[email protected]>
---
Notes:
v3->v4: [EF]
- Re-ran 4 May 2020 (based on kernel tag v5.7-rc4)
v2->v3: [EF]
- Re-ran 16 April 2020 (based on kernel tag v5.6, and limited to bits interesting to this series)
v1->v2: [EF]
- Re-ran 3 February 2020 (based on kernel tag v5.5)
v0->v1: [EF]
- Run scripts/update-linux-headers.sh properly, but do not add resulting changes to linux-headers/asm-mips/
linux-headers/linux/vfio.h | 3 +++ linux-headers/linux/vfio_ccw.h | 18 ++++++++++++++++++ 2 files changed, 21 insertions(+)
linux-headers/linux/vfio.h +3 -0
@@ -378,6 +378,8 @@ struct vfio_region_gfx_edid {/* sub-types for VFIO_REGION_TYPE_CCW */ #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD (1) +#define VFIO_REGION_SUBTYPE_CCW_SCHIB (2) +#define VFIO_REGION_SUBTYPE_CCW_CRW (3)/* * The MSIX mappable capability informs that MSIX data of a BAR can be mmapped
@@ -577,6 +579,7 @@ enum {enum {VFIO_CCW_IO_IRQ_INDEX , +VFIO_CCW_CRW_IRQ_INDEX ,VFIO_CCW_NUM_IRQS };
linux-headers/linux/vfio_ccw.h +18 -0
@@ -34,4 +34,22 @@ struct ccw_cmd_region {__u32 ret_code; } __attribute__((packed)); +/* +* Used for processing commands that read the subchannel-information block +* Reading this region triggers a stsch() to hardware +* Note: this is controlled by a capability +*/ +struct ccw_schib_region { +#define SCHIB_AREA_SIZE 52 +__u8 schib_area[SCHIB_AREA_SIZE ]; +} __attribute__((packed)); + +/* +* Used for returning a Channel Report Word to userspace. +* Note: this is controlled by a capability +*/ +struct ccw_crw_region { +__u32 crw; +} __attribute__((packed)); +#endif
Signature · 1 line
2.17.1
PATCH v4 2/6 vfio-ccw: Refactor cleanup of regions
In reply to Eric FarmanWhile we're at it, add a g_free() for the async_cmd_region that is the last thing currently created. g_free() knows how to handle NULL pointers, so this makes it easier to remember what cleanups need to be performed when new regions are added.
---
Notes:
v1-v2:
- Add Conny's r-b
hw/vfio/ccw.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
hw/vfio/ccw.c +9 -5
@@ -370,8 +370,7 @@ static void vfio_ccw_get_region(VFIOCCWDevice *vcdev, Error **errp) vcdev->io_region_size = info->size;if (sizeof (*vcdev->io_region) != vcdev->io_region_size) {error_setg (errp,"vfio: Unexpected size of the I/O region" ); -g_free (info); -return ; + goto out_err; } vcdev->io_region_offset = info->offset;
@@ -384,15 +383,20 @@ static void vfio_ccw_get_region(VFIOCCWDevice *vcdev, Error **errp) vcdev->async_cmd_region_size = info->size;if (sizeof (*vcdev->async_cmd_region) != vcdev->async_cmd_region_size) {error_setg (errp,"vfio: Unexpected size of the async cmd region" ); -g_free (vcdev->io_region); -g_free (info); -return ; + goto out_err; } vcdev->async_cmd_region_offset = info->offset; vcdev->async_cmd_region =g_malloc0 (info->size); }g_free (info); +return ; + +out_err: +g_free (vcdev->async_cmd_region); +g_free (vcdev->io_region); +g_free (info); +return ; }static void vfio_ccw_put_region (VFIOCCWDevice *vcdev)
Signature · 1 line
2.17.1
PATCH v4 3/6 vfio-ccw: Add support for the schib region
In reply to Eric FarmanFrom: Farhan Ali <[email protected]>
The schib region can be used to obtain the latest SCHIB from the host passthrough subchannel. Since the guest SCHIB is virtualized, we currently only update the path related information so that the guest is aware of any path related changes when it issues the 'stsch' instruction.
---
Notes:
v1->v2:
- Remove silly variable intialization, and add a block comment, to css_do_stsch() [CH]
- Add a TODO statement to s390_ccw_store(), for myself to sort out while we go over kernel code more closely [CH/EF]
- In vfio_ccw_handle_store(),
- Set schib pointer once region is determined to be non-NULL [CH]
- Return cc=0 if pread() fails, and log an error [CH]
v0->v1: [EF]
- Change various incarnations of "update chp status" to "handle_store", to reflect the STSCH instruction that will drive this code
- Remove temporary variable for casting/testing purposes in s390_ccw_store(), and add a block comment of WHY its there.
- Add a few comments to vfio_ccw_handle_store()
hw/s390x/css.c | 13 ++++++-- hw/s390x/s390-ccw.c | 21 +++++++++++++ hw/vfio/ccw.c | 63 +++++++++++++++++++++++++++++++++++++ include/hw/s390x/css.h | 3 +- include/hw/s390x/s390-ccw.h | 1 + target/s390x/ioinst.c | 3 +- 6 files changed, 99 insertions(+), 5 deletions(-)
hw/s390x/css.c +11 -2
@@ -1335,11 +1335,20 @@ static void copy_schib_to_guest(SCHIB *dest, const SCHIB *src) } } -int css_do_stsch (SubchDev *sch,SCHIB *schib) +IOInstEnding css_do_stsch (SubchDev *sch,SCHIB *schib) { +int ret; + +/* +* For some subchannels, we may want to update parts of +* the schib (e.g., update path masks from the host device +* for passthrough subchannels). +*/ + ret =s390_ccw_store (sch); +/* Use current status. */ copy_schib_to_guest (schib, &sch->curr_status); -return 0 ; +return ret; }static void copy_pmcw_from_guest (PMCW *dest,const PMCW *src)
hw/s390x/s390-ccw.c +21 -0
@@ -51,6 +51,27 @@ int s390_ccw_clear(SubchDev *sch)return cdc->handle_clear (sch); } +IOInstEnding s390_ccw_store (SubchDev *sch) +{ +S390CCWDeviceClass *cdc =NULL ; +int ret =IOINST_CC_EXPECTED ; + +/* +* This code is called for both virtual and passthrough devices, +* but only applies to to the latter. This ugly check makes that +* distinction for us. +*/ +if (object_dynamic_cast (OBJECT (sch->driver_data),TYPE_S390_CCW )) { + cdc =S390_CCW_DEVICE_GET_CLASS (sch->driver_data); + } + +if (cdc && cdc->handle_store) { + ret = cdc->handle_store (sch); + } + +return ret; +} +static void s390_ccw_get_dev_info (S390CCWDevice *cdev,char *sysfsdev,Error **errp)
hw/vfio/ccw.c +63 -0
97 lines not on this page · raw message
include/hw/s390x/css.h +2 -1
@@ -218,6 +218,7 @@ IOInstEnding do_subchannel_work_passthrough(SubchDev *sub);int s390_ccw_halt (SubchDev *sch);int s390_ccw_clear (SubchDev *sch); +IOInstEnding s390_ccw_store (SubchDev *sch); typedef enum { CSS_IO_ADAPTER_VIRTIO = 0,
@@ -242,7 +243,7 @@ SubchDev *css_find_subch(uint8_t m, uint8_t cssid, uint8_t ssid, uint16_t schid);bool css_subch_visible (SubchDev *sch);void css_conditional_io_interrupt (SubchDev *sch); -int css_do_stsch (SubchDev *sch,SCHIB *schib); +IOInstEnding css_do_stsch (SubchDev *sch,SCHIB *schib);bool css_schid_final (int m,uint8_t cssid,uint8_t ssid,uint16_t schid);IOInstEnding css_do_msch (SubchDev *sch,const SCHIB *schib);IOInstEnding css_do_xsch (SubchDev *sch);
include/hw/s390x/s390-ccw.h +1 -0
@@ -37,6 +37,7 @@ typedef struct S390CCWDeviceClass {IOInstEnding (*handle_request) (SubchDev *sch);int (*handle_halt) (SubchDev *sch);int (*handle_clear) (SubchDev *sch); + IOInstEnding (*handle_store) (SubchDev *sch); } S390CCWDeviceClass;#endif
target/s390x/ioinst.c +1 -2
@@ -292,8 +292,7 @@ void ioinst_handle_stsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, sch =css_find_subch (m, cssid, ssid, schid);if (sch) {if (css_subch_visible (sch)) { -css_do_stsch (sch, &schib); - cc =0 ; + cc =css_do_stsch (sch, &schib); }else {/* Indicate no more subchannels in this css/ss */ cc =3 ;
Signature · 1 line
2.17.1
Re: [PATCH v4 3/6] vfio-ccw: Add support for the schib region
Quoted 38 lines · On Tue, 5 May 2020 14:57:54 +0200 Eric Farman <[email protected]> wrote:
From: Farhan Ali <[email protected]>
The schib region can be used to obtain the latest SCHIB from the host passthrough subchannel. Since the guest SCHIB is virtualized, we currently only update the path related information so that the guest is aware of any path related changes when it issues the 'stsch' instruction.
Signed-off-by Farhan Ali <[email protected]>Signed-off-by Eric Farman <[email protected]>---
Notes:
v1->v2:
- Remove silly variable intialization, and add a block comment, tocss_do_stsch()[CH]
- Add a TODO statement tos390_ccw_store(), for myself to sort out while we go over kernel code more closely [CH/EF]
- Invfio_ccw_handle_store(),
- Set schib pointer once region is determined to be non-NULL [CH]
- Return cc=0 ifpread()fails, and log an error [CH]
v0->v1: [EF]
- Change various incarnations of "update chp status" to "handle_store", to reflect the STSCH instruction that will drive this code
- Remove temporary variable for casting/testing purposes ins390_ccw_store(), and add a block comment of WHY its there.
- Add a few comments tovfio_ccw_handle_store()hw/s390x/css.c | 13 ++++++-- hw/s390x/s390-ccw.c | 21 +++++++++++++ hw/vfio/ccw.c | 63 +++++++++++++++++++++++++++++++++++++ include/hw/s390x/css.h | 3 +- include/hw/s390x/s390-ccw.h | 1 + target/s390x/ioinst.c | 3 +- 6 files changed, 99 insertions(+), 5 deletions(-)
(...)
Quoted 14 lines
hw/vfio/ccw.c +13
-0+static IOInstEnding vfio_ccw_handle_store (SubchDev *sch) +{ +S390CCWDevice *cdev = sch->driver_data; +VFIOCCWDevice *vcdev =DO_UPCAST (VFIOCCWDevice, cdev, cdev); +SCHIB *schib = &sch->curr_status; +struct ccw_schib_region *region = vcdev->schib_region; +SCHIB *s; +int ret; + +/* schib region not available so nothing else to do */ +if (!region) { +return IOINST_CC_EXPECTED ; + }
hw/vfio/ccw.c again: · line 151
Quoted 10 lines
hw/vfio/ccw.c +10
-0+ +memset (region,0 ,sizeof (*region)); + ret =pread (vcdev->vdev.fd, region, vcdev->schib_region_size, + vcdev->schib_region_offset); + +if (ret ==-1 ) { +/* +* Device is probably damaged, but store subchannel does not +* have a nonzero cc defined for this scenario. Log an error, +* and presume things are otherwise fine.
One thing we might do is set the device number valid bit to 0 in our local copy and inject a (subchannel) crw. Maybe as a patch on top later.
Quoted 12 lines
hw/vfio/ccw.c +12
-0+*/ +error_report ("vfio-ccw: store region read failed with errno=%d" , errno); +return IOINST_CC_EXPECTED ; + } + +/* +* Selectively copy path-related bits of the SCHIB, +* rather than copying the entire struct. +*/ + s = (SCHIB *)region->schib_area; + schib->pmcw.pnom = s->pmcw.pnom; + schib->pmcw.lpum = s->pmcw.lpum;
hw/vfio/ccw.c again: · line 173
Quoted 10 lines
hw/vfio/ccw.c +10
-0+ schib->pmcw.pam = s->pmcw.pam; + schib->pmcw.pom = s->pmcw.pom; + +if (s->scsw.flags &SCSW_FLAGS_MASK_PNO ) { + schib->scsw.flags |=SCSW_FLAGS_MASK_PNO ; + } + +return IOINST_CC_EXPECTED ; +} +
PATCH v4 4/6 vfio-ccw: Refactor ccw irq handler
In reply to Eric FarmanMake it easier to add new ones in the future.
---
Notes:
v2->v3:
- Added Conny's r-b
v1->v2:
- Make irq parameter unsigned [CH]
- Remove extraneous %m from error_report calls [CH]
hw/vfio/ccw.c | 58 +++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 16 deletions(-)
hw/vfio/ccw.c +42 -16
@@ -331,22 +331,36 @@ read_err:css_inject_io_interrupt (sch); } -static void vfio_ccw_register_io_notifier (VFIOCCWDevice *vcdev,Error **errp) +static void vfio_ccw_register_irq_notifier (VFIOCCWDevice *vcdev, +unsigned int irq, +Error **errp) {VFIODevice *vdev = &vcdev->vdev;struct vfio_irq_info *irq_info;size_t argsz;int fd; +EventNotifier *notifier; +IOHandler *fd_read; + +switch (irq) { +case VFIO_CCW_IO_IRQ_INDEX : + notifier = &vcdev->io_notifier; + fd_read = vfio_ccw_io_notifier_handler; +break ; +default : +error_setg (errp,"vfio: Unsupported device irq(%d)" , irq); +return ; + } -if (vdev->num_irqs <VFIO_CCW_IO_IRQ_INDEX +1 ) { -error_setg (errp,"vfio: unexpected number of io irqs %u" , +if (vdev->num_irqs < irq +1 ) { +error_setg (errp,"vfio: unexpected number of irqs %u" , vdev->num_irqs);return ; } argsz =sizeof (*irq_info); irq_info =g_malloc0 (argsz); - irq_info->index =VFIO_CCW_IO_IRQ_INDEX ; + irq_info->index = irq; irq_info->argsz = argsz;if (ioctl (vdev->fd,VFIO_DEVICE_GET_IRQ_INFO , irq_info) <0 || irq_info->count <1 ) {
@@ -354,37 +368,49 @@ static void vfio_ccw_register_io_notifier(VFIOCCWDevice *vcdev, Error **errp) goto out_free_info; } -if (event_notifier_init (&vcdev->io_notifier,0 )) { +if (event_notifier_init (notifier,0 )) {error_setg_errno (errp, errno, -"vfio: Unable to init event notifier for IO" ); +"vfio: Unable to init event notifier for irq (%d)" , + irq); goto out_free_info; } - fd =event_notifier_get_fd (&vcdev->io_notifier); -qemu_set_fd_handler (fd, vfio_ccw_io_notifier_handler,NULL , vcdev); + fd =event_notifier_get_fd (notifier); +qemu_set_fd_handler (fd, fd_read,NULL , vcdev); -if (vfio_set_irq_signaling (vdev,VFIO_CCW_IO_IRQ_INDEX ,0 , +if (vfio_set_irq_signaling (vdev, irq,0 ,VFIO_IRQ_SET_ACTION_TRIGGER , fd, errp)) {qemu_set_fd_handler (fd,NULL ,NULL , vcdev); -event_notifier_cleanup (&vcdev->io_notifier); +event_notifier_cleanup (notifier); } out_free_info:g_free (irq_info); } -static void vfio_ccw_unregister_io_notifier (VFIOCCWDevice *vcdev) +static void vfio_ccw_unregister_irq_notifier (VFIOCCWDevice *vcdev, +unsigned int irq) {Error *err =NULL ; +EventNotifier *notifier; + +switch (irq) { +case VFIO_CCW_IO_IRQ_INDEX : + notifier = &vcdev->io_notifier; +break ; +default : +error_report ("vfio: Unsupported device irq(%d)" , irq); +return ; + } -if (vfio_set_irq_signaling (&vcdev->vdev,VFIO_CCW_IO_IRQ_INDEX ,0 , +if (vfio_set_irq_signaling (&vcdev->vdev, irq,0 ,VFIO_IRQ_SET_ACTION_TRIGGER ,-1 , &err)) {error_reportf_err (err,VFIO_MSG_PREFIX , vcdev->vdev.name); } -qemu_set_fd_handler (event_notifier_get_fd (&vcdev->io_notifier), +qemu_set_fd_handler (event_notifier_get_fd (notifier),NULL ,NULL , vcdev); -event_notifier_cleanup (&vcdev->io_notifier); +event_notifier_cleanup (notifier); }static void vfio_ccw_get_region (VFIOCCWDevice *vcdev,Error **errp)
@@ -572,7 +598,7 @@ static void vfio_ccw_realize(DeviceState *dev, Error **errp) goto out_region_err; } -vfio_ccw_register_io_notifier (vcdev, &err); +vfio_ccw_register_irq_notifier (vcdev,VFIO_CCW_IO_IRQ_INDEX , &err);if (err) { goto out_notifier_err; }
@@ -601,7 +627,7 @@ static void vfio_ccw_unrealize(DeviceState *dev, Error **errp)S390CCWDeviceClass *cdc =S390_CCW_DEVICE_GET_CLASS (cdev);VFIOGroup *group = vcdev->vdev.group; -vfio_ccw_unregister_io_notifier (vcdev); +vfio_ccw_unregister_irq_notifier (vcdev,VFIO_CCW_IO_IRQ_INDEX );vfio_ccw_put_region (vcdev);vfio_ccw_put_device (vcdev);vfio_put_group (group);
Signature · 1 line
2.17.1
PATCH v4 5/6 s390x/css: Refactor the css_queue_crw() routine
In reply to Eric FarmanWe have a use case (vfio-ccw) where a CRW is already built and ready to use. Rather than teasing out the components just to reassemble it later, let's rework this code so we can queue a fully-qualified CRW directly.
---
Notes:
v3->v4:
- s/css_queue_crw_cont/css_crw_add_to_queue/ [CH]
hw/s390x/css.c | 44 ++++++++++++++++++++++++++++-------------- include/hw/s390x/css.h | 1 + 2 files changed, 30 insertions(+), 15 deletions(-)
hw/s390x/css.c +29 -15
@@ -2170,30 +2170,23 @@ void css_subch_assign(uint8_t cssid, uint8_t ssid, uint16_t schid, } } -void css_queue_crw (uint8_t rsc,uint8_t erc,int solicited, -int chain,uint16_t rsid) +void css_crw_add_to_queue (CRW crw) {CrwContainer *crw_cont; -trace_css_crw (rsc, erc, rsid, chain ?"(chained)" :"" ); +trace_css_crw ((crw.flags &CRW_FLAGS_MASK_RSC ) >>8 , + crw.flags &CRW_FLAGS_MASK_ERC , + crw.rsid, + (crw.flags &CRW_FLAGS_MASK_C ) ?"(chained)" :"" ); +/* TODO: Maybe use a static crw pool? */ crw_cont =g_try_new0 (CrwContainer,1 );if (!crw_cont) { channel_subsys.crws_lost = true;return ; } - crw_cont->crw.flags = (rsc <<8 ) | erc; -if (solicited) { - crw_cont->crw.flags |=CRW_FLAGS_MASK_S ; - } -if (chain) { - crw_cont->crw.flags |=CRW_FLAGS_MASK_C ; - } - crw_cont->crw.rsid = rsid; -if (channel_subsys.crws_lost) { - crw_cont->crw.flags |=CRW_FLAGS_MASK_R ; - channel_subsys.crws_lost = false; - } + + crw_cont->crw = crw;QTAILQ_INSERT_TAIL (&channel_subsys.pending_crws, crw_cont, sibling);
@@ -2204,6 +2197,27 @@ void css_queue_crw(uint8_t rsc, uint8_t erc, int solicited, } } +void css_queue_crw (uint8_t rsc,uint8_t erc,int solicited, +int chain,uint16_t rsid) +{ +CRW crw; + + crw.flags = (rsc <<8 ) | erc; +if (solicited) { + crw.flags |=CRW_FLAGS_MASK_S ; + } +if (chain) { + crw.flags |=CRW_FLAGS_MASK_C ; + } + crw.rsid = rsid; +if (channel_subsys.crws_lost) { + crw.flags |=CRW_FLAGS_MASK_R ; + channel_subsys.crws_lost = false; + } + +css_crw_add_to_queue (crw); +} +void css_generate_sch_crws (uint8_t cssid,uint8_t ssid,uint16_t schid,int hotplugged,int add) {
include/hw/s390x/css.h +1 -0
@@ -205,6 +205,7 @@ void copy_scsw_to_guest(SCSW *dest, const SCSW *src);void css_inject_io_interrupt (SubchDev *sch);void css_reset (void );void css_reset_sch (SubchDev *sch); +void css_crw_add_to_queue (CRW crw);void css_queue_crw (uint8_t rsc,uint8_t erc,int solicited,int chain,uint16_t rsid); void css_generate_sch_crws(uint8_t cssid, uint8_t ssid, uint16_t schid,
Signature · 1 line
2.17.1
Re: [PATCH v4 5/6] s390x/css: Refactor the css_queue_crw() routine
Quoted 8 lines · On Tue, 5 May 2020 14:57:56 +0200 Eric Farman <[email protected]> wrote:
We have a use case (vfio-ccw) where a CRW is already built and ready to use. Rather than teasing out the components just to reassemble it later, let's rework this code so we can queue a fully-qualified CRW directly.
Signed-off-by Eric Farman <[email protected]>---
Quoted 7 lines
Notes:
v3->v4:
- s/css_queue_crw_cont/css_crw_add_to_queue/ [CH]hw/s390x/css.c | 44 ++++++++++++++++++++++++++++-------------- include/hw/s390x/css.h | 1 + 2 files changed, 30 insertions(+), 15 deletions(-)
PATCH v4 6/6 vfio-ccw: Add support for the CRW region and IRQ
In reply to Eric FarmanFrom: Farhan Ali <[email protected]>
The crw region can be used to obtain information about Channel Report Words (CRW) from vfio-ccw driver.
Currently only channel-path related CRWs are passed to QEMU from vfio-ccw driver.
---
Notes:
v3->v4:
- Merge region patch into IRQ patch [CH]
- Rework the testing/clearing notifier and reading region [CH]
v2->v3:
- Remove "size==0" check in CRW notifier [CH]
- Remove intermediate rsc/erc variables, use css_queue_crw_cont() [CH]
- s/crw0/crw/ [CH]
v1->v2:
- Add a loop to continually read region while data is present, queueing CRWs as found [CH]
v0->v1: [EF]
- Fixed copy/paste error in error message (s/schib/CRW)
- Check vcdev->crw_region before registering the irq, in case host kernel does not have matching support
- Split the refactoring changes to an earlier (new) patch (and don't remove the "num_irqs" check in the register routine, but adjust it to the check the input variable)
- Don't revert the cool vfio_set_irq_signaling() stuff
- Unregister CRW IRQ before IO IRQ in unrealize
- s/crw1/crw0/
hw/vfio/ccw.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)
hw/vfio/ccw.c +73 -0
@@ -44,7 +44,11 @@ struct VFIOCCWDevice {uint64_t schib_region_size;uint64_t schib_region_offset;struct ccw_schib_region *schib_region; +uint64_t crw_region_size; +uint64_t crw_region_offset; +struct ccw_crw_region *crw_region;EventNotifier io_notifier; +EventNotifier crw_notifier;bool force_orb_pfch;bool warned_orb_pfch; };
@@ -261,6 +265,44 @@ static void vfio_ccw_reset(DeviceState *dev)ioctl (vcdev->vdev.fd,VFIO_DEVICE_RESET ); } +static void vfio_ccw_crw_read (VFIOCCWDevice *vcdev) +{ +struct ccw_crw_region *region = vcdev->crw_region; +CRW crw; +int size; + +/* Keep reading CRWs as long as data is returned */ +do { +memset (region,0 ,sizeof (*region)); + size =pread (vcdev->vdev.fd, region, vcdev->crw_region_size, + vcdev->crw_region_offset); + +if (size ==-1 ) { +error_report ("vfio-ccw: Read crw region failed with errno=%d" , + errno); +break ; + } + +if (region->crw ==0 ) { +/* No more CRWs to queue */ +break ; + } + +memcpy (&crw, ®ion->crw,sizeof (CRW )); + +css_crw_add_to_queue (crw); + }while (1 ); +} + +static void vfio_ccw_crw_notifier_handler (void *opaque) +{ +VFIOCCWDevice *vcdev = opaque; + +while (event_notifier_test_and_clear (&vcdev->crw_notifier)) { +vfio_ccw_crw_read (vcdev); + } +} +static void vfio_ccw_io_notifier_handler (void *opaque) {VFIOCCWDevice *vcdev = opaque;
@@ -347,6 +389,10 @@ static void vfio_ccw_register_irq_notifier(VFIOCCWDevice *vcdev, notifier = &vcdev->io_notifier; fd_read = vfio_ccw_io_notifier_handler;break ; +case VFIO_CCW_CRW_IRQ_INDEX : + notifier = &vcdev->crw_notifier; + fd_read = vfio_ccw_crw_notifier_handler; +break ;default :error_setg (errp,"vfio: Unsupported device irq(%d)" , irq);return ;
@@ -398,6 +444,9 @@ static void vfio_ccw_unregister_irq_notifier(VFIOCCWDevice *vcdev,case VFIO_CCW_IO_IRQ_INDEX : notifier = &vcdev->io_notifier;break ; +case VFIO_CCW_CRW_IRQ_INDEX : + notifier = &vcdev->crw_notifier; +break ;default :error_report ("vfio: Unsupported device irq(%d)" , irq);return ;
@@ -475,10 +524,24 @@ static void vfio_ccw_get_region(VFIOCCWDevice *vcdev, Error **errp) vcdev->schib_region =g_malloc (info->size); } + ret =vfio_get_dev_region_info (vdev,VFIO_REGION_TYPE_CCW , +VFIO_REGION_SUBTYPE_CCW_CRW , &info); + +if (!ret) { + vcdev->crw_region_size = info->size; +if (sizeof (*vcdev->crw_region) != vcdev->crw_region_size) { +error_setg (errp,"vfio: Unexpected size of the CRW region" ); + goto out_err; + } + vcdev->crw_region_offset = info->offset; + vcdev->crw_region =g_malloc (info->size); + } +g_free (info);return ; out_err: +g_free (vcdev->crw_region);g_free (vcdev->schib_region);g_free (vcdev->async_cmd_region);g_free (vcdev->io_region);
@@ -488,6 +551,7 @@ out_err:static void vfio_ccw_put_region (VFIOCCWDevice *vcdev) { +g_free (vcdev->crw_region);g_free (vcdev->schib_region);g_free (vcdev->async_cmd_region);g_free (vcdev->io_region);
@@ -603,6 +667,14 @@ static void vfio_ccw_realize(DeviceState *dev, Error **errp) goto out_notifier_err; } +if (vcdev->crw_region) { +vfio_ccw_register_irq_notifier (vcdev,VFIO_CCW_CRW_IRQ_INDEX , &err); +if (err) { +vfio_ccw_unregister_irq_notifier (vcdev,VFIO_CCW_IO_IRQ_INDEX ); + goto out_notifier_err; + } + } +return ; out_notifier_err:
@@ -627,6 +699,7 @@ static void vfio_ccw_unrealize(DeviceState *dev, Error **errp)S390CCWDeviceClass *cdc =S390_CCW_DEVICE_GET_CLASS (cdev);VFIOGroup *group = vcdev->vdev.group; +vfio_ccw_unregister_irq_notifier (vcdev,VFIO_CCW_CRW_IRQ_INDEX );vfio_ccw_unregister_irq_notifier (vcdev,VFIO_CCW_IO_IRQ_INDEX );vfio_ccw_put_region (vcdev);vfio_ccw_put_device (vcdev);
Signature · 1 line
2.17.1
Re: [PATCH v4 6/6] vfio-ccw: Add support for the CRW region and IRQ
Quoted 37 lines · On Tue, 5 May 2020 14:57:57 +0200 Eric Farman <[email protected]> wrote:
From: Farhan Ali <[email protected]>
The crw region can be used to obtain information about Channel Report Words (CRW) from vfio-ccw driver.
Currently only channel-path related CRWs are passed to QEMU from vfio-ccw driver.
Signed-off-by Farhan Ali <[email protected]>Signed-off-by Eric Farman <[email protected]>---
Notes:
v3->v4:
- Merge region patch into IRQ patch [CH]
- Rework the testing/clearing notifier and reading region [CH]
v2->v3:
- Remove "size==0" check in CRW notifier [CH]
- Remove intermediate rsc/erc variables, usecss_queue_crw_cont()[CH]
- s/crw0/crw/ [CH]
v1->v2:
- Add a loop to continually read region while data is present, queueing CRWs as found [CH]
v0->v1: [EF]
- Fixed copy/paste error in error message (s/schib/CRW)
- Checkvcdev->crw_regionbefore registering the irq, in case host kernel does not have matching support
- Split the refactoring changes to an earlier (new) patch (and don't remove the "num_irqs" check in the register routine, but adjust it to the check the input variable)
- Don't revert the coolvfio_set_irq_signaling()stuff
- Unregister CRW IRQ before IO IRQ in unrealize
- s/crw1/crw0/
Quoted 3 lines
hw/vfio/ccw.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)
Quoted 5 lines · On Tue, 5 May 2020 14:57:51 +0200 Eric Farman <[email protected]> wrote:
Here is a new pass at the channel-path handling code for vfio-ccw, to take advantage of the corresponding kernel patches posted here:
Quoted 7 lines
I have all the main comments from v3 addressed, though I have a couple of additional patches that do some further cleanups (like, a generic callback for stsch) that weren't baked enough to include here. They're working fine, but need some cleanups (e.g., fixups vs standalone patches) before they're ready for submission. Just wanted to have the basic QEMU code to go with the new KVM series.
This also looks good to me now, cleanup on top sounds good.
Would not mind some further review here, either :) This would need to wait for a proper header update before queuing to s390-next, anyway.
Quoted 29 lines · On Tue, 5 May 2020 14:57:51 +0200 Eric Farman <[email protected]> wrote:
Here is a new pass at the channel-path handling code for vfio-ccw, to take advantage of the corresponding kernel patches posted here:
https://lore.kernel.org/kvm/[email protected]/
I have all the main comments from v3 addressed, though I have a couple of additional patches that do some further cleanups (like, a generic callback for stsch) that weren't baked enough to include here. They're working fine, but need some cleanups (e.g., fixups vs standalone patches) before they're ready for submission. Just wanted to have the basic QEMU code to go with the new KVM series.
Changes are listed in git notes for the individual patches.
v3: https://lore.kernel.org/qemu-devel/[email protected]/
v2: https://lore.kernel.org/qemu-devel/[email protected]/
v1: https://lore.kernel.org/qemu-devel/[email protected]/Eric Farman (3):
vfio-ccw: Refactor cleanup of regions
vfio-ccw: Refactor ccw irq handler
s390x/css: Refactor thecss_queue_crw()routineFarhan Ali (3):
linux-headers: update
vfio-ccw: Add support for the schib region
vfio-ccw: Add support for the CRW region and IRQ
Quoted 10 lines
hw/s390x/css.c | 57 ++++++--- hw/s390x/s390-ccw.c | 21 ++++ hw/vfio/ccw.c | 208 +++++++++++++++++++++++++++++---- include/hw/s390x/css.h | 4 +- include/hw/s390x/s390-ccw.h | 1 + linux-headers/linux/vfio.h | 3 + linux-headers/linux/vfio_ccw.h | 18 +++ target/s390x/ioinst.c | 3 +- 8 files changed, 274 insertions(+), 41 deletions(-)
Thanks, replaced patch 1 with a proper headers update against 5.8-rc1 and applied.