Report vfio-ap configuration changes
v2 cover Report vfio-ap configuration changes
Changelog:
v2:
- removed warnings that weren't needed
- added unregister function
- removed whitelines
- changed variable names for consistency
- removed rc variable and returning 1 or 0 outright
- reversed logics for if statements
- using g_free() instead of free()
- replaced hardcoded numeric values by defining them with #define in the header
--------------------------------------------------------------------------
This patch series creates and registers a handler that is called when userspace is notified by the kernel that a guest's AP configuration has changed. The handler in turn notifies the guest that its AP configuration has changed. This allows the guest to immediately respond to AP configuration changes rather than relying on polling or some other inefficient mechanism for detecting config changes.
Rorie Reyes (5):
linux-headers: NOTFORMERGE - placeholder uapi updates for AP config
change
hw/vfio/ap: notification handler for AP config changed event hw/vfio/ap: store object indicating AP config changed in a queue hw/vfio/ap: Storing event information for an AP configuration change
event
s390: implementing CHSC SEI for AP config change
hw/vfio/ap.c | 79 ++++++++++++++++++++++++++++++++++++ include/hw/s390x/ap-bridge.h | 22 ++++++++++ linux-headers/linux/vfio.h | 1 + target/s390x/ioinst.c | 11 ++++- 4 files changed, 111 insertions(+), 2 deletions(-)
Signature · 1 line
2.39.5 (Apple Git-154)
RFC PATCH v2 1/5 linux-headers: NOTFORMERGE - placeholder uapi updates for AP config change
This patch adds enumeration constant VFIO_AP_CFG_CHG_IRQ_INDEX to specify an IRQ index for signaling that a change has been made to the guest's AP configuration. This is a placeholder for QEMU patches that use this value since it is a linux-headers update which includes changes that aren't merged into the kernel. Linux-headers patches should be generated using scripts/update-linux-headers.sh.
--- linux-headers/linux/vfio.h | 1 + 1 file changed, 1 insertion(+)
linux-headers/linux/vfio.h +1 -0
@@ -671,6 +671,7 @@ enum {*/ enum {VFIO_AP_REQ_IRQ_INDEX , +VFIO_AP_CFG_CHG_IRQ_INDEX ,VFIO_AP_NUM_IRQS };
Signature · 1 line
2.39.5 (Apple Git-154)
Re: [RFC PATCH v2 1/5] linux-headers: NOTFORMERGE - placeholder uapi updates for AP config change
Hello Rorie,
Quoted 17 lines · Rorie Reyes
This patch adds enumeration constant
VFIO_AP_CFG_CHG_IRQ_INDEXto specify an IRQ index for signaling that a change has been made to the guest's AP configuration. This is a placeholder for QEMU patches that use this value since it is a linux-headers update which includes changes that aren't merged into the kernel. Linux-headers patches should be generated usingscripts/update-linux-headers.sh.Signed-off-by Rorie Reyes <[email protected]>--- linux-headers/linux/vfio.h | 1 + 1 file changed, 1 insertion(+)linux-headers/linux/vfio.h +0
-0
linux-headers/linux/vfio.h enum { · line 677
Quoted 7 lines
linux-headers/linux/vfio.h +1
-0@@ -671,6 +671,7 @@ enum {*/ enum {VFIO_AP_REQ_IRQ_INDEX , +VFIO_AP_CFG_CHG_IRQ_INDEX ,VFIO_AP_NUM_IRQS };
Are the kernel changes planned for 6.14 ?
FYI, QEMU 10.0 hard freeze is scheduled for 2025-03-18 which is approximately when 6.14-rc7 will be released.
Thanks,
C.
Re: [RFC PATCH v2 1/5] linux-headers: NOTFORMERGE - placeholder uapi updates for AP config change
Quoted 9 lines · Cédric Le Goater
Are the kernel changes planned for 6.14 ?
FYI, QEMU 10.0 hard freeze is scheduled for 2025-03-18 which is approximately when 6.14-rc7 will be released.
Thanks,
C.
Vasily - Since you applied my kernel patches already ([RFC PATCH v2] s390/vfio-ap: Notify userspace that guest's AP config changed when mdev removed), would you be able to answer Cedric's
a question?
Cedric - I have some changes I need to make based on Tony's review comments from Friday, but I'll wait to post it once you get an answer for your question. Sorry I took a while to respond. I had to
make some changes for my kernel code this past month.
Re: [RFC PATCH v2 1/5] linux-headers: NOTFORMERGE - placeholder uapi updates for AP config change
Hello Rorie,
Quoted 12 lines · Rorie Reyes
Quoted 9 lines · Cédric Le Goater
Are the kernel changes planned for 6.14 ?
FYI, QEMU 10.0 hard freeze is scheduled for 2025-03-18 which is approximately when 6.14-rc7 will be released.
Thanks,
C.
Quoted 8 lines
Vasily - Since you applied my kernel patches already ([RFC PATCH v2] s390/vfio-ap: Notify userspace that guest's AP config changed when mdev removed), would you be able to answer Cedric's
a question?
Cedric - I have some changes I need to make based on Tony's review comments from Friday, but I'll wait to post it once you get an answer for your question. Sorry I took a while to respond. I had to
make some changes for my kernel code this past month.
AFAICT, the changes were merged in the s390 sub-maintainer tree, 07d89045bffea30ef08b902c2441a3329e44f29d, and they haven't reached Linux master yet. Seems a bit late for the 6.14 cycle now.
linux-headers in QEMU are in sync with version 6.14-rc3 and that should the last update before QEMU 10.0 soft-freeze tomorrow.
Most likely, this series should now target the next cycles:
Linux 6.15 and QEMU 10.1. We have time.
Thanks,
C.
Re: [RFC PATCH v2 1/5] linux-headers: NOTFORMERGE - placeholder uapi updates for AP config change
Quoted 33 lines · Cédric Le Goater
Hello Rorie,
Quoted 12 lines · Rorie Reyes
Quoted 9 lines · Cédric Le Goater
Are the kernel changes planned for 6.14 ?
FYI, QEMU 10.0 hard freeze is scheduled for 2025-03-18 which is approximately when 6.14-rc7 will be released.
Thanks,
C.
Quoted 8 lines
Vasily - Since you applied my kernel patches already ([RFC PATCH v2] s390/vfio-ap: Notify userspace that guest's AP config changed when mdev removed), would you be able to answer Cedric's
a question?
Cedric - I have some changes I need to make based on Tony's review comments from Friday, but I'll wait to post it once you get an answer for your question. Sorry I took a while to respond. I had to
make some changes for my kernel code this past month.
AFAICT, the changes were merged in the s390 sub-maintainer tree,
07d89045bffea30ef08b902c2441a3329e44f29d, and they haven't reached Linux master yet. Seems a bit late for the 6.14 cycle now.linux-headers in QEMU are in sync with version 6.14-rc3 and that should the last update before QEMU 10.0 soft-freeze tomorrow.
Quoted 7 lines
Most likely, this series should now target the next cycles:
Linux 6.15 and QEMU 10.1. We have time.Thanks,
C.
Thanks Cedric! I'll post the v3 patch from Tony's review
Re: [RFC PATCH v2 1/5] linux-headers: NOTFORMERGE - placeholder uapi updates for AP config change
In reply to Rorie ReyesQuoted 10 lines · Rorie Reyes
Quoted 8 lines · Cédric Le Goater
Are the kernel changes planned for 6.14 ?
FYI, QEMU 10.0 hard freeze is scheduled for 2025-03-18 which is approximately when 6.14-rc7 will be released.
Quoted 7 lines
Quoted 3 lines
Thanks,
C.
Vasily - Since you applied my kernel patches already ([RFC PATCH v2] s390/vfio-ap: Notify userspace that guest's AP config changed when mdev removed), would you be able to answer Cedric's
We are at v6.14-rc6. The changes are queued for the 6.15 merge window.
https://lore.kernel.org/r/[email protected]
https://lore.kernel.org/r/[email protected]
Re: [RFC PATCH v2 1/5] linux-headers: NOTFORMERGE - placeholder uapi updates for AP config change
Quoted 12 lines · Vasily Gorbik
Quoted 11 lines · Rorie Reyes
Quoted 9 lines · Cédric Le Goater
Are the kernel changes planned for 6.14 ?
FYI, QEMU 10.0 hard freeze is scheduled for 2025-03-18 which is approximately when 6.14-rc7 will be released.
Thanks,
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
C.
Vasily - Since you applied my kernel patches already ([RFC PATCH v2] s390/vfio-ap: Notify userspace that guest's AP config changed when mdev removed), would you be able to answer Cedric's
We are at v6.14-rc6. The changes are queued for the 6.15 merge window.
https://lore.kernel.org/r/[email protected]
https://lore.kernel.org/r/[email protected]
Thanks Vasily!
RFC PATCH v2 2/5 hw/vfio/ap: notification handler for AP config changed event
In reply to Rorie ReyesRegister an event notifier handler to process AP configuration change events by queuing the event and generating a CRW to let the guest know its AP configuration has changed
--- hw/vfio/ap.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)
hw/vfio/ap.c +29 -0
@@ -18,6 +18,7 @@#include "hw/vfio/vfio-common.h" #include "system/iommufd.h" #include "hw/s390x/ap-device.h" +#include "hw/s390x/css.h" #include "qemu/error-report.h" #include "qemu/event_notifier.h" #include "qemu/main-loop.h"
@@ -37,6 +38,7 @@ struct VFIOAPDevice {APDevice apdev;VFIODevice vdev;EventNotifier req_notifier; +EventNotifier cfg_notifier; };OBJECT_DECLARE_SIMPLE_TYPE (VFIOAPDevice,VFIO_AP_DEVICE )
@@ -70,6 +72,16 @@ static void vfio_ap_req_notifier_handler(void *opaque) } } +static void vfio_ap_cfg_chg_notifier_handler (void *opaque) +{ +VFIOAPDevice *vapdev = opaque; + +if (!event_notifier_test_and_clear (&vapdev->cfg_notifier)) { +css_generate_css_crws (0 ); + } + +} +static bool vfio_ap_register_irq_notifier (VFIOAPDevice *vapdev,unsigned int irq,Error **errp) {
@@ -85,6 +97,10 @@ static bool vfio_ap_register_irq_notifier(VFIOAPDevice *vapdev, notifier = &vapdev->req_notifier; fd_read = vfio_ap_req_notifier_handler;break ; +case VFIO_AP_CFG_CHG_IRQ_INDEX : + notifier = &vapdev->cfg_notifier; + fd_read = vfio_ap_cfg_chg_notifier_handler; +break ;default :error_setg (errp,"vfio: Unsupported device irq(%d)" , irq);return false;
@@ -136,6 +152,9 @@ static void vfio_ap_unregister_irq_notifier(VFIOAPDevice *vapdev,case VFIO_AP_REQ_IRQ_INDEX : notifier = &vapdev->req_notifier;break ; +case VFIO_AP_CFG_CHG_IRQ_INDEX : + notifier = &vapdev->cfg_notifier; +break ;default :error_report ("vfio: Unsupported device irq(%d)" , irq);return ;
@@ -175,6 +194,15 @@ static void vfio_ap_realize(DeviceState *dev, Error **errp)warn_report_err (err); } +if (!vfio_ap_register_irq_notifier (vapdev,VFIO_AP_CFG_CHG_IRQ_INDEX , &err)) + { +/* +* Report this error, but do not make it a failing condition. +* Lack of this IRQ in the host does not prevent normal operation. +*/ +warn_report_err (err); + } +return ; error:
@@ -187,6 +215,7 @@ static void vfio_ap_unrealize(DeviceState *dev)VFIOAPDevice *vapdev =VFIO_AP_DEVICE (dev);vfio_ap_unregister_irq_notifier (vapdev,VFIO_AP_REQ_IRQ_INDEX ); +vfio_ap_unregister_irq_notifier (vapdev,VFIO_AP_CFG_CHG_IRQ_INDEX );vfio_detach_device (&vapdev->vdev);g_free (vapdev->vdev.name); }
Signature · 1 line
2.39.5 (Apple Git-154)
Re: [RFC PATCH v2 2/5] hw/vfio/ap: notification handler for AP config changed event
Quoted 30 lines · Rorie Reyes
Register an event notifier handler to process AP configuration change events by queuing the event and generating a CRW to let the guest know its AP configuration has changed
Signed-off-by Rorie Reyes <[email protected]>--- hw/vfio/ap.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)hw/vfio/ap.c +2
-0@@ -18,6 +18,7 @@#include "hw/vfio/vfio-common.h" #include "system/iommufd.h" #include "hw/s390x/ap-device.h" +#include "hw/s390x/css.h" #include "qemu/error-report.h" #include "qemu/event_notifier.h" #include "qemu/main-loop.h" @@ -37,6 +38,7 @@ struct VFIOAPDevice {APDevice apdev;VFIODevice vdev;EventNotifier req_notifier; +EventNotifier cfg_notifier; };OBJECT_DECLARE_SIMPLE_TYPE (VFIOAPDevice,VFIO_AP_DEVICE )
hw/vfio/ap.c static void vfio_ap_req_notifier_handler(void *opaque) · line 80
Quoted 9 lines
hw/vfio/ap.c +6
-0@@ -70,6 +72,16 @@ static void vfio_ap_req_notifier_handler(void *opaque) } } +static void vfio_ap_cfg_chg_notifier_handler (void *opaque) +{ +VFIOAPDevice *vapdev = opaque; + +if (!event_notifier_test_and_clear (&vapdev->cfg_notifier)) { +css_generate_css_crws (0 );
In patch 3/5, you move the css_generate_css_crws() function outside of this if block and replace it with a return here. That is the right thing to do, but you should do it here in this patch.
Quoted 52 lines
hw/vfio/ap.c +21
-0+ } + +} +static bool vfio_ap_register_irq_notifier (VFIOAPDevice *vapdev,unsigned int irq,Error **errp) {@@ -85,6 +97,10 @@ static bool vfio_ap_register_irq_notifier(VFIOAPDevice *vapdev, notifier = &vapdev->req_notifier; fd_read = vfio_ap_req_notifier_handler;break ; +case VFIO_AP_CFG_CHG_IRQ_INDEX : + notifier = &vapdev->cfg_notifier; + fd_read = vfio_ap_cfg_chg_notifier_handler; +break ;default :error_setg (errp,"vfio: Unsupported device irq(%d)" , irq);return false;@@ -136,6 +152,9 @@ static void vfio_ap_unregister_irq_notifier(VFIOAPDevice *vapdev,case VFIO_AP_REQ_IRQ_INDEX : notifier = &vapdev->req_notifier;break ; +case VFIO_AP_CFG_CHG_IRQ_INDEX : + notifier = &vapdev->cfg_notifier; +break ;default :error_report ("vfio: Unsupported device irq(%d)" , irq);return ;@@ -175,6 +194,15 @@ static void vfio_ap_realize(DeviceState *dev, Error **errp)warn_report_err (err); } +if (!vfio_ap_register_irq_notifier (vapdev,VFIO_AP_CFG_CHG_IRQ_INDEX , &err)) + { +/* +* Report this error, but do not make it a failing condition. +* Lack of this IRQ in the host does not prevent normal operation. +*/ +warn_report_err (err); + } +return ; error:@@ -187,6 +215,7 @@ static void vfio_ap_unrealize(DeviceState *dev)VFIOAPDevice *vapdev =VFIO_AP_DEVICE (dev);vfio_ap_unregister_irq_notifier (vapdev,VFIO_AP_REQ_IRQ_INDEX ); +vfio_ap_unregister_irq_notifier (vapdev,VFIO_AP_CFG_CHG_IRQ_INDEX );vfio_detach_device (&vapdev->vdev);g_free (vapdev->vdev.name); }
RFC PATCH v2 3/5 hw/vfio/ap: store object indicating AP config changed in a queue
In reply to Rorie ReyesCreates an object indicating that an AP configuration change event has been received and stores it in a queue. These objects will later be used to store event information for an AP configuration change when the CHSC instruction is intercepted.
--- hw/vfio/ap.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
hw/vfio/ap.c +13 -1
@@ -41,6 +41,13 @@ struct VFIOAPDevice {EventNotifier cfg_notifier; }; +typedef struct APConfigChgEvent { +QTAILQ_ENTRY (APConfigChgEvent) next; +}APConfigChgEvent ; + +QTAILQ_HEAD(, APConfigChgEvent) cfg_chg_events = +QTAILQ_HEAD_INITIALIZER (cfg_chg_events); + OBJECT_DECLARE_SIMPLE_TYPE(VFIOAPDevice, VFIO_AP_DEVICE)static void vfio_ap_compute_needs_reset (VFIODevice *vdev)
@@ -75,11 +82,16 @@ static void vfio_ap_req_notifier_handler(void *opaque)static void vfio_ap_cfg_chg_notifier_handler (void *opaque) {VFIOAPDevice *vapdev = opaque; +APConfigChgEvent *cfg_chg_event =g_new0 (APConfigChgEvent,1 );if (!event_notifier_test_and_clear (&vapdev->cfg_notifier)) { -css_generate_css_crws (0 ); +return ; } +QTAILQ_INSERT_TAIL (&cfg_chg_events, cfg_chg_event, next); + +css_generate_css_crws (0 ); + } static bool vfio_ap_register_irq_notifier(VFIOAPDevice *vapdev,
Signature · 1 line
2.39.5 (Apple Git-154)
Re: [RFC PATCH v2 3/5] hw/vfio/ap: store object indicating AP config changed in a queue
Quoted 29 lines · Rorie Reyes
Creates an object indicating that an AP configuration change event has been received and stores it in a queue. These objects will later be used to store event information for an AP configuration change when the CHSC instruction is intercepted.
Signed-off-by Rorie Reyes <[email protected]>--- hw/vfio/ap.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)hw/vfio/ap.c +7
-0@@ -41,6 +41,13 @@ struct VFIOAPDevice {EventNotifier cfg_notifier; }; +typedef struct APConfigChgEvent { +QTAILQ_ENTRY (APConfigChgEvent) next; +}APConfigChgEvent ; + +QTAILQ_HEAD(, APConfigChgEvent) cfg_chg_events = +QTAILQ_HEAD_INITIALIZER (cfg_chg_events); + OBJECT_DECLARE_SIMPLE_TYPE(VFIOAPDevice, VFIO_AP_DEVICE)static void vfio_ap_compute_needs_reset (VFIODevice *vdev)
hw/vfio/ap.c static void vfio_ap_req_notifier_handler(void *opaque) · line 88
Quoted 8 lines
hw/vfio/ap.c +2
-1@@ -75,11 +82,16 @@ static void vfio_ap_req_notifier_handler(void *opaque)static void vfio_ap_cfg_chg_notifier_handler (void *opaque) {VFIOAPDevice *vapdev = opaque; +APConfigChgEvent *cfg_chg_event =g_new0 (APConfigChgEvent,1 );if (!event_notifier_test_and_clear (&vapdev->cfg_notifier)) { -css_generate_css_crws (0 ); +return ;
As I stated in my review of patch 2/5, the change above should have been done there.
hw/vfio/ap.c static void vfio_ap_req_notifier_handler(void *opaque) · line 93
Quoted 5 lines
hw/vfio/ap.c +3
-0} +QTAILQ_INSERT_TAIL (&cfg_chg_events, cfg_chg_event, next); + +css_generate_css_crws (0 );
Ditto.
Quoted 4 lines
hw/vfio/ap.c +1
-0+ } static bool vfio_ap_register_irq_notifier(VFIOAPDevice *vapdev,
RFC PATCH v2 4/5 hw/vfio/ap: Storing event information for an AP configuration change event
In reply to Rorie ReyesThese functions can be invoked by the function that handles interception of the CHSC SEI instruction for requests indicating the accessibility of one or more adjunct processors has changed.
--- hw/vfio/ap.c | 38 ++++++++++++++++++++++++++++++++++++ include/hw/s390x/ap-bridge.h | 22 +++++++++++++++++++++ 2 files changed, 60 insertions(+)
hw/vfio/ap.c +38 -0
@@ -94,6 +94,44 @@ static void vfio_ap_cfg_chg_notifier_handler(void *opaque) } +int ap_chsc_sei_nt0_get_event (void *res) +{ +APConfigChgEvent *cfg_chg_event =QTAILQ_FIRST (&cfg_chg_events); +ChscSeiNt0Res *nt0_res = (ChscSeiNt0Res *)res; + +memset (nt0_res,0 ,sizeof (*nt0_res)); + +if (!cfg_chg_event) { +return 1 ; + } + +QTAILQ_REMOVE (&cfg_chg_events, cfg_chg_event, next); +g_free (cfg_chg_event); + +/* +* If there are any AP configuration change events in the queue, +* indicate to the caller that there is pending event info in +* the response block +*/ +if (ap_chsc_sei_nt0_have_event ()) { + nt0_res->flags |=PENDING_EVENT_INFO_BITMASK ; + } + + nt0_res->length =sizeof (ChscSeiNt0Res); + nt0_res->code =NT0_RES_RESPONSE_CODE ; + nt0_res->nt =NT0_RES_NT_DEFAULT ; + nt0_res->rs =NT0_RES_RS_AP_CHANGE ; + nt0_res->cc =NT0_RES_CC_AP_CHANGE ; + +return 0 ; + +} + +int ap_chsc_sei_nt0_have_event (void ) +{ +return !QTAILQ_EMPTY (&cfg_chg_events); +} +static bool vfio_ap_register_irq_notifier (VFIOAPDevice *vapdev,unsigned int irq,Error **errp) {
include/hw/s390x/ap-bridge.h +22 -0
@@ -16,4 +16,26 @@void s390_init_ap (void ); +typedef struct ChscSeiNt0Res { +uint16_t length; +uint16_t code; +uint8_t reserved1; +uint16_t reserved2; +uint8_t nt; +#define PENDING_EVENT_INFO_BITMASK 0x80; +uint8_t flags; +uint8_t reserved3; +uint8_t rs; +uint8_t cc; +} QEMU_PACKED ChscSeiNt0Res; + +#define NT0_RES_RESPONSE_CODE 1; +#define NT0_RES_NT_DEFAULT 0; +#define NT0_RES_RS_AP_CHANGE 5; +#define NT0_RES_CC_AP_CHANGE 3; + +int ap_chsc_sei_nt0_get_event (void *res); + +int ap_chsc_sei_nt0_have_event (void ); +#endif
Signature · 1 line
2.39.5 (Apple Git-154)
RFC PATCH v2 5/5 s390: implementing CHSC SEI for AP config change
In reply to Rorie ReyesHandle interception of the CHSC SEI instruction for requests indicating the guest's AP configuration has changed.
--- target/s390x/ioinst.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
target/s390x/ioinst.c +9 -2
@@ -17,6 +17,7 @@#include "trace.h" #include "hw/s390x/s390-pci-bus.h" #include "target/s390x/kvm/pv.h" +#include "hw/s390x/ap-bridge.h" /* All I/O instructions but chsc use the s format */ static uint64_t get_address_from_regs(CPUS390XState *env, uint32_t ipb,
@@ -573,13 +574,19 @@ out:static int chsc_sei_nt0_get_event (void *res) { -/* no events yet */ +if (s390_has_feat (S390_FEAT_AP )) { +return ap_chsc_sei_nt0_get_event (res); + } +return 1 ; }static int chsc_sei_nt0_have_event (void ) { -/* no events yet */ +if (s390_has_feat (S390_FEAT_AP )) { +return ap_chsc_sei_nt0_have_event (); + } +return 0 ; }
Signature · 1 line
2.39.5 (Apple Git-154)