Skip to contentSkip to conversation

Report vfio-ap configuration changes

RFC PATCH Applied Rorie Reyes 4 participants 13 messages Jun 9, 2025 – Jun 30, 2025 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

v13 cover Report vfio-ap configuration changes

Changelog:
v13:
- added lock to 'vfio_ap_cfg_chg_notifier_handler' in patch 2
- added RBs for patch 2 and 3 from Tony

v12:
- adding locks to 'ap_chsc_sei_nt0_have_event' and 'ap_chsc_sei_nt0_get_event'

v11:
- reverted return type to int for 'ap_chsc_sei_nt0_get_event'
- files reflected are 'ap.c', 'ap-bridge'h, and 'ap-stub.c'
- using defined variables to represent return 0 and 1 to reflect logical sense
- update documentation for 'ap_chsc_sei_nt0_get_event' to reflect return types

v10:
- Added reviewed by for patch 4 by Tony and Cedric
- review needed for patch 2 and 3
- changed 'cfg_chg_events' to a static variable
- locked 'cfg_chg_events' using QemuMutex
- removed ';' at the end of the definition for NT0
- returning bools for 'ap_chsc_sei_nt0_get_event' and 'ap_chsc_sei_nt0_have_event'
- updated the header file that contains 'ap_chsc_sei_nt0_get_event' and 'ap_chsc_sei_nt0_have_event' to a bool function
- added documentation explaining the returning bool functions
- whitespace clean up

v9:
- added SPDX licensing to newly created file 'hw/s390x/ap-stub.c'

v8:
- fixed windows cross-compile build
- moved /hw/vfio/ap-stub.c to /hw/s390x/ap-stub.c
- updated the use of stub file to MAINTAINERS to reflect new location
- removed if_false for 'CONFIG_VFIO_AP' statement from /hw/vfio/meson.build
- added if_false for 'CONFIG_VFIO_AP' to use ap-stub.c in /hw/s390x/meson.build
- all those changes still address '--without-default-devices' issue from v5

v7:
- Dropped initial commit for linux-header file vfio.h since I created two new commits to address the changes made in v6
- Moved patches 6 and 7 to the beginning of the series after dropping the first patch
- Because I dropped the initial commit for linux-header file vfio.h, I had to add
VFIO_AP_CFG_CHG_IRQ_INDEX
- Resyncing latest to v6.15-rc3
- Still need Thomas Huth's review of v5 changes for patch 6/6

v6:
- Updating the update-linux-headers script to address kernel commit change 8a14
- Update headers to retrieve uapi information for vfio-ap for update to Linux v6.15-rc1
- Still need Thomas Huth's review of v5 changes for patch 7/7 (see below)

v5:
- configuring using the '--without-default-devices' fails when building the code
- created a stub file for functions ap_chsc_sei_nt0_get_event and ap_chsc_sei_nt0_have_event
- add if_false for 'CONFIG_VFIO_AP' use ap-stub.c in meson.build
- add the use of the stub file to MAINTAINERS since it's a new file

v4:
- allocating cfg_chg_event before inserting into the queue
- calling nt0_have_event in if loop to check if there are any elemenets in the queue, then calling QTAILQ_FIRST when the check passes
- moving memset() after the check

v3:
- changes that were made to patch 3/5 should have been made in patch 2/5

v2:
- removed warnings that weren't needed
- added unregister function
- removed whitelines
- changed variable names for consistency
- removed rc variable and returning 1 or 0 outright
- reversed logics for if statements
- using g_free() instead of free()
- replaced hardcoded numeric values by defining them with #define in the header

--------------------------------------------------------------------------

Rorie Reyes (4):
hw/vfio/ap: notification handler for AP config changed event hw/vfio/ap: store object indicating AP config changed in a queue hw/vfio/ap: Storing event information for an AP configuration change
event
s390: implementing CHSC SEI for AP config change

 MAINTAINERS                  |  1 +
 hw/s390x/ap-stub.c           | 21 +++++++++
 hw/s390x/meson.build         |  1 +
 hw/vfio/ap.c                 | 88 ++++++++++++++++++++++++++++++++++++
 include/hw/s390x/ap-bridge.h | 39 ++++++++++++++++
 target/s390x/ioinst.c        | 11 ++++-
 6 files changed, 159 insertions(+), 2 deletions(-)
 create mode 100644 hw/s390x/ap-stub.c
Signature · 1 line
2.48.1
#1 lore raw

RFC PATCH v13 1/4 hw/vfio/ap: notification handler for AP config changed event

Register an event notifier handler to process AP configuration change events by queuing the event and generating a CRW to let the guest know its AP configuration has changed

---
 hw/vfio/ap.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
hw/vfio/ap.c +31 -0
index 785c0a0197..93c74ebedb 100644
@@ -18,6 +18,7 @@
 #include "hw/vfio/vfio-device.h"
 #include "system/iommufd.h"
 #include "hw/s390x/ap-device.h"
+#include "hw/s390x/css.h"
 #include "qemu/error-report.h"
 #include "qemu/event_notifier.h"
 #include "qemu/main-loop.h"
@@ -37,6 +38,7 @@ struct VFIOAPDevice {
     APDevice apdev;
     VFIODevice vdev;
     EventNotifier req_notifier;
+    EventNotifier cfg_notifier;
 };
 
 OBJECT_DECLARE_SIMPLE_TYPE(VFIOAPDevice, VFIO_AP_DEVICE)
@@ -70,6 +72,18 @@ static void vfio_ap_req_notifier_handler(void *opaque)
     }
 }
 
+static void vfio_ap_cfg_chg_notifier_handler(void *opaque)
+{
+    VFIOAPDevice *vapdev = opaque;
+
+    if (!event_notifier_test_and_clear(&vapdev->cfg_notifier)) {
+        return;
+    }
+
+    css_generate_css_crws(0);
+
+}
+
 static bool vfio_ap_register_irq_notifier(VFIOAPDevice *vapdev,
                                           unsigned int irq, Error **errp)
 {
@@ -85,6 +99,10 @@ static bool vfio_ap_register_irq_notifier(VFIOAPDevice *vapdev,
         notifier = &vapdev->req_notifier;
         fd_read = vfio_ap_req_notifier_handler;
         break;
+    case VFIO_AP_CFG_CHG_IRQ_INDEX:
+        notifier = &vapdev->cfg_notifier;
+        fd_read = vfio_ap_cfg_chg_notifier_handler;
+        break;
     default:
         error_setg(errp, "vfio: Unsupported device irq(%d)", irq);
         return false;
@@ -137,6 +155,9 @@ static void vfio_ap_unregister_irq_notifier(VFIOAPDevice *vapdev,
     case VFIO_AP_REQ_IRQ_INDEX:
         notifier = &vapdev->req_notifier;
         break;
+    case VFIO_AP_CFG_CHG_IRQ_INDEX:
+        notifier = &vapdev->cfg_notifier;
+        break;
     default:
         error_report("vfio: Unsupported device irq(%d)", irq);
         return;
@@ -176,6 +197,15 @@ static void vfio_ap_realize(DeviceState *dev, Error **errp)
         warn_report_err(err);
     }
 
+    if (!vfio_ap_register_irq_notifier(vapdev, VFIO_AP_CFG_CHG_IRQ_INDEX, &err))
+    {
+        /*
+         * Report this error, but do not make it a failing condition.
+         * Lack of this IRQ in the host does not prevent normal operation.
+         */
+        warn_report_err(err);
+    }
+
     return;
 
 error:
@@ -188,6 +218,7 @@ static void vfio_ap_unrealize(DeviceState *dev)
     VFIOAPDevice *vapdev = VFIO_AP_DEVICE(dev);
 
     vfio_ap_unregister_irq_notifier(vapdev, VFIO_AP_REQ_IRQ_INDEX);
+    vfio_ap_unregister_irq_notifier(vapdev, VFIO_AP_CFG_CHG_IRQ_INDEX);
     vfio_device_detach(&vapdev->vdev);
     g_free(vapdev->vdev.name);
 }
Signature · 1 line
2.48.1
#2 lore raw

RFC PATCH v13 2/4 hw/vfio/ap: store object indicating AP config changed in a queue

In reply to Rorie Reyes

Creates an object indicating that an AP configuration change event has been received and stores it in a queue. These objects will later be used to store event information for an AP configuration change when the CHSC instruction is intercepted.

---
 hw/vfio/ap.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
hw/vfio/ap.c +17 -0
index 93c74ebedb..681fd4a4f1 100644
@@ -21,6 +21,7 @@
 #include "hw/s390x/css.h"
 #include "qemu/error-report.h"
 #include "qemu/event_notifier.h"
+#include "qemu/lockable.h"
 #include "qemu/main-loop.h"
 #include "qemu/module.h"
 #include "qemu/option.h"
@@ -41,6 +42,15 @@ struct VFIOAPDevice {
     EventNotifier cfg_notifier;
 };
 
+typedef struct APConfigChgEvent {
+    QTAILQ_ENTRY(APConfigChgEvent) next;
+} APConfigChgEvent;
+
+static QTAILQ_HEAD(, APConfigChgEvent) cfg_chg_events =
+    QTAILQ_HEAD_INITIALIZER(cfg_chg_events);
+
+static QemuMutex cfg_chg_events_lock;
+
 OBJECT_DECLARE_SIMPLE_TYPE(VFIOAPDevice, VFIO_AP_DEVICE)
 
 static void vfio_ap_compute_needs_reset(VFIODevice *vdev)
@@ -74,12 +84,19 @@ static void vfio_ap_req_notifier_handler(void *opaque)
 
 static void vfio_ap_cfg_chg_notifier_handler(void *opaque)
 {
+    APConfigChgEvent *cfg_chg_event;
     VFIOAPDevice *vapdev = opaque;
 
     if (!event_notifier_test_and_clear(&vapdev->cfg_notifier)) {
         return;
     }
 
+    cfg_chg_event = g_new0(APConfigChgEvent, 1);
+
+    WITH_QEMU_LOCK_GUARD(&cfg_chg_events_lock) {
+        QTAILQ_INSERT_TAIL(&cfg_chg_events, cfg_chg_event, next);
+    }
+
     css_generate_css_crws(0);
 
 }
Signature · 1 line
2.48.1
#3 lore raw

RFC PATCH v13 3/4 hw/vfio/ap: Storing event information for an AP configuration change event

In reply to Rorie Reyes

These functions can be invoked by the function that handles interception of the CHSC SEI instruction for requests indicating the accessibility of one or more adjunct processors has changed.

---
 hw/vfio/ap.c                 | 40 ++++++++++++++++++++++++++++++++++++
 include/hw/s390x/ap-bridge.h | 39 +++++++++++++++++++++++++++++++++++
 2 files changed, 79 insertions(+)
hw/vfio/ap.c +40 -0
index 681fd4a4f1..874e0d1eaf 100644
@@ -10,6 +10,7 @@
  * directory.
  */
 
+#include <stdbool.h>
 #include "qemu/osdep.h"
 #include CONFIG_DEVICES /* CONFIG_IOMMUFD */
 #include <linux/vfio.h>
@@ -101,6 +102,38 @@ static void vfio_ap_cfg_chg_notifier_handler(void *opaque)
 
 }
 
+int ap_chsc_sei_nt0_get_event(void *res)
+{
+    ChscSeiNt0Res *nt0_res  = (ChscSeiNt0Res *)res;
+    APConfigChgEvent *cfg_chg_event;
+
+    WITH_QEMU_LOCK_GUARD(&cfg_chg_events_lock) {
+        if (QTAILQ_EMPTY(&cfg_chg_events)) {
+            return EVENT_INFORMATION_NOT_STORED;
+        }
+
+        cfg_chg_event = QTAILQ_FIRST(&cfg_chg_events);
+        QTAILQ_REMOVE(&cfg_chg_events, cfg_chg_event, next);
+    }
+
+    memset(nt0_res, 0, sizeof(*nt0_res));
+    g_free(cfg_chg_event);
+    nt0_res->flags |= PENDING_EVENT_INFO_BITMASK;
+    nt0_res->length = sizeof(ChscSeiNt0Res);
+    nt0_res->code = NT0_RES_RESPONSE_CODE;
+    nt0_res->nt = NT0_RES_NT_DEFAULT;
+    nt0_res->rs = NT0_RES_RS_AP_CHANGE;
+    nt0_res->cc = NT0_RES_CC_AP_CHANGE;
+
+    return EVENT_INFORMATION_STORED;
+}
+
+bool ap_chsc_sei_nt0_have_event(void)
+{
+    QEMU_LOCK_GUARD(&cfg_chg_events_lock);
+    return !QTAILQ_EMPTY(&cfg_chg_events);
+}
+
 static bool vfio_ap_register_irq_notifier(VFIOAPDevice *vapdev,
                                           unsigned int irq, Error **errp)
 {
@@ -197,6 +230,13 @@ static void vfio_ap_realize(DeviceState *dev, Error **errp)
     VFIOAPDevice *vapdev = VFIO_AP_DEVICE(dev);
     VFIODevice *vbasedev = &vapdev->vdev;
 
+    static bool lock_initialized;
+
+    if (!lock_initialized) {
+        qemu_mutex_init(&cfg_chg_events_lock);
+        lock_initialized = true;
+    }
+
     if (!vfio_device_get_name(vbasedev, errp)) {
         return;
     }
include/hw/s390x/ap-bridge.h +39 -0
index 470e439a98..7efc52928d 100644
@@ -16,4 +16,43 @@
 
 void s390_init_ap(void);
 
+typedef struct ChscSeiNt0Res {
+    uint16_t length;
+    uint16_t code;
+    uint8_t reserved1;
+    uint16_t reserved2;
+    uint8_t nt;
+#define PENDING_EVENT_INFO_BITMASK 0x80;
+    uint8_t flags;
+    uint8_t reserved3;
+    uint8_t rs;
+    uint8_t cc;
+} QEMU_PACKED ChscSeiNt0Res;
+
+#define NT0_RES_RESPONSE_CODE 1
+#define NT0_RES_NT_DEFAULT    0
+#define NT0_RES_RS_AP_CHANGE  5
+#define NT0_RES_CC_AP_CHANGE  3
+
+#define EVENT_INFORMATION_NOT_STORED 1
+#define EVENT_INFORMATION_STORED     0
+
+/**
+ * ap_chsc_sei_nt0_get_event - Retrieve the next pending AP config
+ * change event
+ * @res: Pointer to a ChscSeiNt0Res struct to be filled with event
+ * data
+ *
+ * This function checks for any pending AP config change events and,
+ * if present, populates the provided response structure with the
+ * appropriate SEI NT0 fields.
+ *
+ * Return:
+ *   EVENT_INFORMATION_STORED - An event was available and written to @res
+ *   EVENT_INFORMATION_NOT_STORED - No event was available
+ */
+int ap_chsc_sei_nt0_get_event(void *res);
+
+bool ap_chsc_sei_nt0_have_event(void);
+
 #endif
Signature · 1 line
2.48.1
#4 lore raw

Re: [RFC PATCH v13 3/4] hw/vfio/ap: Storing event information for an AP configuration change event

Quoted 63 lines · Rorie Reyes

These functions can be invoked by the function that handles interception of the CHSC SEI instruction for requests indicating the accessibility of one or more adjunct processors has changed.

---
  hw/vfio/ap.c                 | 40 ++++++++++++++++++++++++++++++++++++
  include/hw/s390x/ap-bridge.h | 39 +++++++++++++++++++++++++++++++++++
  2 files changed, 79 insertions(+)
hw/vfio/ap.c +33 -0
index 681fd4a4f1..874e0d1eaf 100644
@@ -10,6 +10,7 @@
   * directory.
   */
 
+#include <stdbool.h>
  #include "qemu/osdep.h"
  #include CONFIG_DEVICES /* CONFIG_IOMMUFD */
  #include <linux/vfio.h>
@@ -101,6 +102,38 @@ static void vfio_ap_cfg_chg_notifier_handler(void *opaque)
 
  }
 
+int ap_chsc_sei_nt0_get_event(void *res)
+{
+    ChscSeiNt0Res *nt0_res  = (ChscSeiNt0Res *)res;
+    APConfigChgEvent *cfg_chg_event;
+
+    WITH_QEMU_LOCK_GUARD(&cfg_chg_events_lock) {
+        if (QTAILQ_EMPTY(&cfg_chg_events)) {
+            return EVENT_INFORMATION_NOT_STORED;
+        }
+
+        cfg_chg_event = QTAILQ_FIRST(&cfg_chg_events);
+        QTAILQ_REMOVE(&cfg_chg_events, cfg_chg_event, next);
+    }
+
+    memset(nt0_res, 0, sizeof(*nt0_res));
+    g_free(cfg_chg_event);
+    nt0_res->flags |= PENDING_EVENT_INFO_BITMASK;
+    nt0_res->length = sizeof(ChscSeiNt0Res);
+    nt0_res->code = NT0_RES_RESPONSE_CODE;
+    nt0_res->nt = NT0_RES_NT_DEFAULT;
+    nt0_res->rs = NT0_RES_RS_AP_CHANGE;
+    nt0_res->cc = NT0_RES_CC_AP_CHANGE;
+
+    return EVENT_INFORMATION_STORED;
+}
+
+bool ap_chsc_sei_nt0_have_event(void)
+{
+    QEMU_LOCK_GUARD(&cfg_chg_events_lock);
+    return !QTAILQ_EMPTY(&cfg_chg_events);
+}
+
  static bool vfio_ap_register_irq_notifier(VFIOAPDevice *vapdev,
                                            unsigned int irq, Error **errp)
  {

hw/vfio/ap.c static void vfio_ap_realize(DeviceState *dev, Error **errp) · line 239

Quoted 10 lines
hw/vfio/ap.c +7 -0
@@ -197,6 +230,13 @@ static void vfio_ap_realize(DeviceState *dev, Error **errp)
      VFIOAPDevice *vapdev = VFIO_AP_DEVICE(dev);
      VFIODevice *vbasedev = &vapdev->vdev;
 
+    static bool lock_initialized;
+
+    if (!lock_initialized) {
+        qemu_mutex_init(&cfg_chg_events_lock);
+        lock_initialized = true;
+    }
+

cfg_chg_events_lock could be initialized in a __constructor__ routine.

Thanks,

C.

Quoted 51 lines
hw/vfio/ap.c +0 -0
      if (!vfio_device_get_name(vbasedev, errp)) {
          return;
      }
include/hw/s390x/ap-bridge.h +39 -0
index 470e439a98..7efc52928d 100644
@@ -16,4 +16,43 @@
 
  void s390_init_ap(void);
 
+typedef struct ChscSeiNt0Res {
+    uint16_t length;
+    uint16_t code;
+    uint8_t reserved1;
+    uint16_t reserved2;
+    uint8_t nt;
+#define PENDING_EVENT_INFO_BITMASK 0x80;
+    uint8_t flags;
+    uint8_t reserved3;
+    uint8_t rs;
+    uint8_t cc;
+} QEMU_PACKED ChscSeiNt0Res;
+
+#define NT0_RES_RESPONSE_CODE 1
+#define NT0_RES_NT_DEFAULT    0
+#define NT0_RES_RS_AP_CHANGE  5
+#define NT0_RES_CC_AP_CHANGE  3
+
+#define EVENT_INFORMATION_NOT_STORED 1
+#define EVENT_INFORMATION_STORED     0
+
+/**
+ * ap_chsc_sei_nt0_get_event - Retrieve the next pending AP config
+ * change event
+ * @res: Pointer to a ChscSeiNt0Res struct to be filled with event
+ * data
+ *
+ * This function checks for any pending AP config change events and,
+ * if present, populates the provided response structure with the
+ * appropriate SEI NT0 fields.
+ *
+ * Return:
+ *   EVENT_INFORMATION_STORED - An event was available and written to @res
+ *   EVENT_INFORMATION_NOT_STORED - No event was available
+ */
+int ap_chsc_sei_nt0_get_event(void *res);
+
+bool ap_chsc_sei_nt0_have_event(void);
+
  #endif
#5 lore raw

RFC PATCH v13 4/4 s390: implementing CHSC SEI for AP config change

In reply to Rorie Reyes

Handle interception of the CHSC SEI instruction for requests indicating the guest's AP configuration has changed.

If configuring --without-default-devices, hw/s390x/ap-stub.c was created to handle such circumstance. Also added the following to hw/s390x/meson.build if CONFIG_VFIO_AP is false, it will use the stub file.

---
 MAINTAINERS           |  1 +
 hw/s390x/ap-stub.c    | 21 +++++++++++++++++++++
 hw/s390x/meson.build  |  1 +
 target/s390x/ioinst.c | 11 +++++++++--
 4 files changed, 32 insertions(+), 2 deletions(-)
 create mode 100644 hw/s390x/ap-stub.c
MAINTAINERS +1 -0
index aa6763077e..1e84bfeaee 100644
@@ -112,6 +112,7 @@ F: hw/intc/s390_flic.c
 F: hw/intc/s390_flic_kvm.c
 F: hw/s390x/
 F: hw/vfio/ap.c
+F: hw/s390x/ap-stub.c
 F: hw/vfio/ccw.c
 F: hw/watchdog/wdt_diag288.c
 F: include/hw/s390x/
hw/s390x/ap-stub.c new file +21 -0
new file mode 100644
index 0000000000..001fe5f8b0
@@ -0,0 +1,21 @@
+/*
+ * VFIO based AP matrix device assignment
+ *
+ * Copyright 2025 IBM Corp.
+ * Author(s): Rorie Reyes <[email protected]>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/s390x/ap-bridge.h"
+
+int ap_chsc_sei_nt0_get_event(void *res)
+{
+    return EVENT_INFORMATION_NOT_STORED;
+}
+
+bool ap_chsc_sei_nt0_have_event(void)
+{
+    return false;
+}
hw/s390x/meson.build +1 -0
index 3bbebfd817..99cbcbd7d6 100644
@@ -33,6 +33,7 @@ s390x_ss.add(when: 'CONFIG_S390_CCW_VIRTIO', if_true: files(
 ))
 s390x_ss.add(when: 'CONFIG_TERMINAL3270', if_true: files('3270-ccw.c'))
 s390x_ss.add(when: 'CONFIG_VFIO', if_true: files('s390-pci-vfio.c'))
+s390x_ss.add(when: 'CONFIG_VFIO_AP', if_false: files('ap-stub.c'))
 
 virtio_ss = ss.source_set()
 virtio_ss.add(files('virtio-ccw.c'))
target/s390x/ioinst.c +9 -2
index fe62ba5b06..2320dd4c12 100644
@@ -18,6 +18,7 @@
 #include "trace.h"
 #include "hw/s390x/s390-pci-bus.h"
 #include "target/s390x/kvm/pv.h"
+#include "hw/s390x/ap-bridge.h"
 
 /* All I/O instructions but chsc use the s format */
 static uint64_t get_address_from_regs(CPUS390XState *env, uint32_t ipb,
@@ -574,13 +575,19 @@ out:
 
 static int chsc_sei_nt0_get_event(void *res)
 {
-    /* no events yet */
+    if (s390_has_feat(S390_FEAT_AP)) {
+        return ap_chsc_sei_nt0_get_event(res);
+    }
+
     return 1;
 }
 
 static int chsc_sei_nt0_have_event(void)
 {
-    /* no events yet */
+    if (s390_has_feat(S390_FEAT_AP)) {
+        return ap_chsc_sei_nt0_have_event();
+    }
+
     return 0;
 }
 
Signature · 1 line
2.48.1
#6 lore raw

Re: [RFC PATCH v13 4/4] s390: implementing CHSC SEI for AP config change

Quoted 11 lines · Rorie Reyes

Handle interception of the CHSC SEI instruction for requests indicating the guest's AP configuration has changed.

If configuring --without-default-devices, hw/s390x/ap-stub.c was created to handle such circumstance. Also added the following to hw/s390x/meson.build if CONFIG_VFIO_AP is false, it will use the stub file.

FYI, this patch (or some part of this series) breaks hotplug for PCI devices on s390x. I verified it breaks with both PCI passthrough and virtio-pci via virsh attach-device.

I get greeted with the following message:
qemu-system-s390x: ../util/qemu-thread-posix.c:92: qemu_mutex_lock_impl: Assertion `mutex->initialized' failed.

2025-06-26 14:12:23.031+0000: shutting down, reason=crashed

As a quick test, reverting this patch resolves the issue.

Another test shows that if I have a vfio-ap device attached to my guest before I try the zPCI hotplug, everything works fine (because vfio_ap_realize was driven -- see more info below).

I note in the failing call chain that in response to the hotplug event for a zPCI device we end up calling into hw/vfio/ap.c and getting some locks which seems wrong:

(gdb) bt

#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x000003ffa85ad636 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x000003ffa8553c20 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x000003ffa8533e3c in __GI_abort () at abort.c:79
#4  0x000003ffa854ac9a in __assert_fail_base
    (fmt=<optimized out>, assertion=assertion@entry=0x2aa3fa76414 "mutex->initialized", file=file@entry=0x2aa3fa763f8 "../util/qemu-thread-posix.c", line=line@entry=92, function=function@entry=0x2aa3fa7647a <__PRETTY_FUNCTION__.23> "qemu_mutex_lock_impl") at assert.c:94
#5  0x000003ffa854acf4 in __assert_fail
    (assertion=0x2aa3fa76414 "mutex->initialized", file=0x2aa3fa763f8 "../util/qemu-thread-posix.c", line=92, function=0x2aa3fa7647a <__PRETTY_FUNCTION__.23> "qemu_mutex_lock_impl") at assert.c:103
#6  0x000002aa3f83ef02 in qemu_mutex_lock_impl (mutex=0x2aa3fed9e08 <cfg_chg_events_lock>, file=0x2aa3f9e653e "/usr/src/qemu/include/qemu/lockable.h", line=56) at ../util/qemu-thread-posix.c:92
#7  0x000002aa3f4c3a70 in qemu_lockable_mutex_lock (x=0x2aa3fed9e08 <cfg_chg_events_lock>) at /usr/src/qemu/include/qemu/lockable.h:56
#8  0x000002aa3f4c3b00 in qemu_lockable_lock (x=0x3fd076f7730) at /usr/src/qemu/include/qemu/lockable.h:110
#9  0x000002aa3f4c3b82 in qemu_lockable_auto_lock (x=0x3fd076f7730) at /usr/src/qemu/include/qemu/lockable.h:120

#10 0x000002aa3f4c3f90 in ap_chsc_sei_nt0_get_event (res=0x3fd076f79b8) at ../hw/vfio/ap.c:110 ^^^^^^^^
#11 0x000002aa3f3a59ea in chsc_sei_nt0_get_event (res=0x3fd076f79b8) at ../target/s390x/ioinst.c:579 #12 0x000002aa3f3a5b76 in ioinst_handle_chsc_sei (req=0x3fd076f79a8, res=0x3fd076f79b8) at ../target/s390x/ioinst.c:620 #13 0x000002aa3f3a6062 in ioinst_handle_chsc (cpu=0x2aa40cc9430, ipb=2097152, ra=0) at ../target/s390x/ioinst.c:708 #14 0x000002aa3f4647d2 in handle_b2 (cpu=0x2aa40cc9430, run=0x3ffa4180000, ipa1=95 '_') at ../target/s390x/kvm/kvm.c:1218 #15 0x000002aa3f465db2 in handle_instruction (cpu=0x2aa40cc9430, run=0x3ffa4180000) at ../target/s390x/kvm/kvm.c:1627 #16 0x000002aa3f4661da in handle_intercept (cpu=0x2aa40cc9430) at ../target/s390x/kvm/kvm.c:1709
#17 0x000002aa3f466c10 in kvm_arch_handle_exit (cs=0x2aa40cc9430, run=0x3ffa4180000) at ../target/s390x/kvm/kvm.c:1914 #18 0x000002aa3f523c98 in kvm_cpu_exec (cpu=0x2aa40cc9430) at ../accel/kvm/kvm-all.c:3294
#19 0x000002aa3f527f52 in kvm_vcpu_thread_fn (arg=0x2aa40cc9430) at ../accel/kvm/kvm-accel-ops.c:51 #20 0x000002aa3f840002 in qemu_thread_start (args=0x2aa40ccd090) at ../util/qemu-thread-posix.c:393 #21 0x000003ffa85ab622 in start_thread (arg=0x3fd076fe8c0) at pthread_create.c:448
#22 0x000003ffa862b390 in thread_start () at ../sysdeps/unix/sysv/linux/s390/s390-64/clone3.S:71

Note that PCI devices were already using SEI before this series (e.g. see hw/s390x/s390-pci-bus.c:pci_chsc_sei_nt2_get_event()); I think the problem here is that nt0 events are looked for implicitly in the shared handler (ioinst_handle_chsc_sei()) which is fine -- but you cannot go acquiring locks from ap.c that might have never been initialized (cfg_chg_events_lock which is only initialized in vfio_ap_realize()); that's why we ultimately crash.

AFAICT this portion of the handler including mutex should be moved out of ap.c into chsc_sei_nt0_get_event(). When vfio_ap is not in use, we can't call into it. Can't you also build without VFIO_AP? I didn't try it but that sure seems like it would cause issues too.

I guess the mutex and list would then need to be part of something common to s390x like S390CcwMachineState.

Alternatively, you could leave those in vfio_ap but would need some kind of indicator that vfio_ap is "armed" for these kinds of events and only bother calling into vfio_ap to look at cfg_chg_events when that indicator has been set (e.g. vfio_ap_realize has been called once. And it actually looks like you already do something like this via lock_initialized in ap.c). Otherwise, we should always return EVENT_INFORMATION_NOT_STORED like the old code did (via return 1).

Or maybe someone else has a better idea.

Quoted 93 lines
---
 MAINTAINERS           |  1 +
 hw/s390x/ap-stub.c    | 21 +++++++++++++++++++++
 hw/s390x/meson.build  |  1 +
 target/s390x/ioinst.c | 11 +++++++++--
 4 files changed, 32 insertions(+), 2 deletions(-)
 create mode 100644 hw/s390x/ap-stub.c
MAINTAINERS +1 -0
index aa6763077e..1e84bfeaee 100644
@@ -112,6 +112,7 @@ F: hw/intc/s390_flic.c
 F: hw/intc/s390_flic_kvm.c
 F: hw/s390x/
 F: hw/vfio/ap.c
+F: hw/s390x/ap-stub.c
 F: hw/vfio/ccw.c
 F: hw/watchdog/wdt_diag288.c
 F: include/hw/s390x/
hw/s390x/ap-stub.c new file +21 -0
new file mode 100644
index 0000000000..001fe5f8b0
@@ -0,0 +1,21 @@
+/*
+ * VFIO based AP matrix device assignment
+ *
+ * Copyright 2025 IBM Corp.
+ * Author(s): Rorie Reyes <[email protected]>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/s390x/ap-bridge.h"
+
+int ap_chsc_sei_nt0_get_event(void *res)
+{
+    return EVENT_INFORMATION_NOT_STORED;
+}
+
+bool ap_chsc_sei_nt0_have_event(void)
+{
+    return false;
+}
hw/s390x/meson.build +1 -0
index 3bbebfd817..99cbcbd7d6 100644
@@ -33,6 +33,7 @@ s390x_ss.add(when: 'CONFIG_S390_CCW_VIRTIO', if_true: files(
 ))
 s390x_ss.add(when: 'CONFIG_TERMINAL3270', if_true: files('3270-ccw.c'))
 s390x_ss.add(when: 'CONFIG_VFIO', if_true: files('s390-pci-vfio.c'))
+s390x_ss.add(when: 'CONFIG_VFIO_AP', if_false: files('ap-stub.c'))
 
 virtio_ss = ss.source_set()
 virtio_ss.add(files('virtio-ccw.c'))
target/s390x/ioinst.c +9 -2
index fe62ba5b06..2320dd4c12 100644
@@ -18,6 +18,7 @@
 #include "trace.h"
 #include "hw/s390x/s390-pci-bus.h"
 #include "target/s390x/kvm/pv.h"
+#include "hw/s390x/ap-bridge.h"
 
 /* All I/O instructions but chsc use the s format */
 static uint64_t get_address_from_regs(CPUS390XState *env, uint32_t ipb,
@@ -574,13 +575,19 @@ out:
 
 static int chsc_sei_nt0_get_event(void *res)
 {
-    /* no events yet */
+    if (s390_has_feat(S390_FEAT_AP)) {
+        return ap_chsc_sei_nt0_get_event(res);
+    }
+
     return 1;
 }
 
 static int chsc_sei_nt0_have_event(void)
 {
-    /* no events yet */
+    if (s390_has_feat(S390_FEAT_AP)) {
+        return ap_chsc_sei_nt0_have_event();
+    }
+
     return 0;
 }
 
#7 lore raw

Re: [RFC PATCH v13 4/4] s390: implementing CHSC SEI for AP config change

Quoted 53 lines · Matthew Rosato
Quoted 11 lines · Rorie Reyes

Handle interception of the CHSC SEI instruction for requests indicating the guest's AP configuration has changed.

If configuring --without-default-devices, hw/s390x/ap-stub.c was created to handle such circumstance. Also added the following to hw/s390x/meson.build if CONFIG_VFIO_AP is false, it will use the stub file.

FYI, this patch (or some part of this series) breaks hotplug for PCI devices on s390x. I verified it breaks with both PCI passthrough and virtio-pci via virsh attach-device.

I get greeted with the following message:
qemu-system-s390x: ../util/qemu-thread-posix.c:92: qemu_mutex_lock_impl: Assertion `mutex->initialized' failed.

2025-06-26 14:12:23.031+0000: shutting down, reason=crashed

As a quick test, reverting this patch resolves the issue.

Another test shows that if I have a vfio-ap device attached to my guest before I try the zPCI hotplug, everything works fine (because vfio_ap_realize was driven -- see more info below).

I note in the failing call chain that in response to the hotplug event for a zPCI device we end up calling into hw/vfio/ap.c and getting some locks which seems wrong:

(gdb) bt

#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x000003ffa85ad636 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x000003ffa8553c20 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x000003ffa8533e3c in __GI_abort () at abort.c:79
#4  0x000003ffa854ac9a in __assert_fail_base
     (fmt=<optimized out>, assertion=assertion@entry=0x2aa3fa76414 "mutex->initialized", file=file@entry=0x2aa3fa763f8 "../util/qemu-thread-posix.c", line=line@entry=92, function=function@entry=0x2aa3fa7647a <__PRETTY_FUNCTION__.23> "qemu_mutex_lock_impl") at assert.c:94
#5  0x000003ffa854acf4 in __assert_fail
     (assertion=0x2aa3fa76414 "mutex->initialized", file=0x2aa3fa763f8 "../util/qemu-thread-posix.c", line=92, function=0x2aa3fa7647a <__PRETTY_FUNCTION__.23> "qemu_mutex_lock_impl") at assert.c:103
#6  0x000002aa3f83ef02 in qemu_mutex_lock_impl (mutex=0x2aa3fed9e08 <cfg_chg_events_lock>, file=0x2aa3f9e653e "/usr/src/qemu/include/qemu/lockable.h", line=56) at ../util/qemu-thread-posix.c:92
#7  0x000002aa3f4c3a70 in qemu_lockable_mutex_lock (x=0x2aa3fed9e08 <cfg_chg_events_lock>) at /usr/src/qemu/include/qemu/lockable.h:56
#8  0x000002aa3f4c3b00 in qemu_lockable_lock (x=0x3fd076f7730) at /usr/src/qemu/include/qemu/lockable.h:110
#9  0x000002aa3f4c3b82 in qemu_lockable_auto_lock (x=0x3fd076f7730) at /usr/src/qemu/include/qemu/lockable.h:120

#10 0x000002aa3f4c3f90 in ap_chsc_sei_nt0_get_event (res=0x3fd076f79b8) at ../hw/vfio/ap.c:110 ^^^^^^^^
#11 0x000002aa3f3a59ea in chsc_sei_nt0_get_event (res=0x3fd076f79b8) at ../target/s390x/ioinst.c:579 #12 0x000002aa3f3a5b76 in ioinst_handle_chsc_sei (req=0x3fd076f79a8, res=0x3fd076f79b8) at ../target/s390x/ioinst.c:620 #13 0x000002aa3f3a6062 in ioinst_handle_chsc (cpu=0x2aa40cc9430, ipb=2097152, ra=0) at ../target/s390x/ioinst.c:708 #14 0x000002aa3f4647d2 in handle_b2 (cpu=0x2aa40cc9430, run=0x3ffa4180000, ipa1=95 '_') at ../target/s390x/kvm/kvm.c:1218 #15 0x000002aa3f465db2 in handle_instruction (cpu=0x2aa40cc9430, run=0x3ffa4180000) at ../target/s390x/kvm/kvm.c:1627 #16 0x000002aa3f4661da in handle_intercept (cpu=0x2aa40cc9430) at ../target/s390x/kvm/kvm.c:1709
#17 0x000002aa3f466c10 in kvm_arch_handle_exit (cs=0x2aa40cc9430, run=0x3ffa4180000) at ../target/s390x/kvm/kvm.c:1914 #18 0x000002aa3f523c98 in kvm_cpu_exec (cpu=0x2aa40cc9430) at ../accel/kvm/kvm-all.c:3294
#19 0x000002aa3f527f52 in kvm_vcpu_thread_fn (arg=0x2aa40cc9430) at ../accel/kvm/kvm-accel-ops.c:51 #20 0x000002aa3f840002 in qemu_thread_start (args=0x2aa40ccd090) at ../util/qemu-thread-posix.c:393 #21 0x000003ffa85ab622 in start_thread (arg=0x3fd076fe8c0) at pthread_create.c:448
#22 0x000003ffa862b390 in thread_start () at ../sysdeps/unix/sysv/linux/s390/s390-64/clone3.S:71

Quoted 1 line

Note that PCI devices were already using SEI before this series (e.g. see hw/s390x/s390-pci-bus.c:pci_chsc_sei_nt2_get_event()); I think the problem here is that nt0 events are looked for implicitly in the shared handler (ioinst_handle_chsc_sei()) which is fine -- but you cannot go acquiring locks from ap.c that might have never been initialized (cfg_chg_events_lock which is only initialized in vfio_ap_realize()); that's why we ultimately crash.

The following check is done in both the chsc_sei_nt0_get_event and chsc_sei_nt0_have_event functions before calling into the AP versions of those functions:

    if (s390_has_feat(S390_FEAT_AP)) {
        return ap_chsc_sei_nt0_get_event(res);
    }

Given that the call to AP is made, it looks like AP is installed on the guest. So it seems the reason that the vfio_ap_realize() function does not get called is because a mediated device is not passed through to the guest. This check alone, therefore, does not protect us against making a call into the ap_chsc_sei_nt0_get/have_event() functions which results in the problem you are seeing.

Quoted 2 lines

AFAICT this portion of the handler including mutex should be moved out of ap.c into chsc_sei_nt0_get_event(). When vfio_ap is not in use, we can't call into it. Can't you also build without VFIO_AP? I didn't try it but that sure seems like it would cause issues too.

If that were the case, I think the checks I mentioned above would be sufficient because the S390_FEAT_AP feature could not be set.

Quoted 8 lines

I guess the mutex and list would then need to be part of something common to s390x like S390CcwMachineState.

Alternatively, you could leave those in vfio_ap but would need some kind of indicator that vfio_ap is "armed" for these kinds of events and only bother calling into vfio_ap to look at cfg_chg_events when that indicator has been set (e.g. vfio_ap_realize has been called once. And it actually looks like you already do something like this via lock_initialized in ap.c). Otherwise, we should always return EVENT_INFORMATION_NOT_STORED like the old code did (via return 1).

Or maybe someone else has a better idea.

What about moving lock initialization to the vfio_ap_class_init() function?

Quoted 95 lines
Quoted 95 lines

---
  MAINTAINERS           |  1 +
  hw/s390x/ap-stub.c    | 21 +++++++++++++++++++++
  hw/s390x/meson.build  |  1 +
  target/s390x/ioinst.c | 11 +++++++++--
  4 files changed, 32 insertions(+), 2 deletions(-)

create mode 100644 hw/s390x/ap-stub.c

MAINTAINERS +1 -0
index aa6763077e..1e84bfeaee 100644
@@ -112,6 +112,7 @@ F: hw/intc/s390_flic.c
  F: hw/intc/s390_flic_kvm.c
  F: hw/s390x/
  F: hw/vfio/ap.c
+F: hw/s390x/ap-stub.c
  F: hw/vfio/ccw.c
  F: hw/watchdog/wdt_diag288.c
  F: include/hw/s390x/
hw/s390x/ap-stub.c new file +21 -0
new file mode 100644
index 0000000000..001fe5f8b0
@@ -0,0 +1,21 @@
+/*
+ * VFIO based AP matrix device assignment
+ *
+ * Copyright 2025 IBM Corp.
+ * Author(s): Rorie Reyes <[email protected]>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/s390x/ap-bridge.h"
+
+int ap_chsc_sei_nt0_get_event(void *res)
+{
+    return EVENT_INFORMATION_NOT_STORED;
+}
+
+bool ap_chsc_sei_nt0_have_event(void)
+{
+    return false;
+}
hw/s390x/meson.build +1 -0
index 3bbebfd817..99cbcbd7d6 100644
@@ -33,6 +33,7 @@ s390x_ss.add(when: 'CONFIG_S390_CCW_VIRTIO', if_true: files(
  ))
  s390x_ss.add(when: 'CONFIG_TERMINAL3270', if_true: files('3270-ccw.c'))
  s390x_ss.add(when: 'CONFIG_VFIO', if_true: files('s390-pci-vfio.c'))
+s390x_ss.add(when: 'CONFIG_VFIO_AP', if_false: files('ap-stub.c'))
 
  virtio_ss = ss.source_set()
  virtio_ss.add(files('virtio-ccw.c'))
target/s390x/ioinst.c +9 -2
index fe62ba5b06..2320dd4c12 100644
@@ -18,6 +18,7 @@
  #include "trace.h"
  #include "hw/s390x/s390-pci-bus.h"
  #include "target/s390x/kvm/pv.h"
+#include "hw/s390x/ap-bridge.h"
 
  /* All I/O instructions but chsc use the s format */
  static uint64_t get_address_from_regs(CPUS390XState *env, uint32_t ipb,
@@ -574,13 +575,19 @@ out:
 
  static int chsc_sei_nt0_get_event(void *res)
  {
-    /* no events yet */
+    if (s390_has_feat(S390_FEAT_AP)) {
+        return ap_chsc_sei_nt0_get_event(res);
+    }
+
      return 1;
  }
 
  static int chsc_sei_nt0_have_event(void)
  {
-    /* no events yet */
+    if (s390_has_feat(S390_FEAT_AP)) {
+        return ap_chsc_sei_nt0_have_event();
+    }
+
      return 0;
  }
 
#8 lore raw

Re: [RFC PATCH v13 4/4] s390: implementing CHSC SEI for AP config change

Quoted 7 lines
Quoted 3 lines

AFAICT this portion of the handler including mutex should be moved out of ap.c into chsc_sei_nt0_get_event().  When vfio_ap is not in use, we can't call into it.  Can't you also build without VFIO_AP?  I didn't try it but that sure seems like it would cause issues too.

If that were the case, I think the checks I mentioned above would be sufficient because the S390_FEAT_AP feature could not be set.

OK, I tested it quick (remove VFIO_AP from config) and that appears to be the case; this also restores zPCI hotplug capability since we won't call info hw/vfio/ap.c/.

Quoted 3 lines

What about moving lock initialization to the vfio_ap_class_init() function?

I tried a quick hack of that too and it appears to work, at least for the zPCI regression (note: I did not test the AP portion).

This approach is probably not the way I would have designed it initially... But it does seem to be the simplest thing to do from a fix perspective.

Can you or Rorie prepare a proper patch w/ a fixes tag and send (please add me to CC). Also please test the change against the vfio-ap environment; you can also test zPCI but I would also like to run some tests.

#9 lore raw

Re: [RFC PATCH v13 4/4] s390: implementing CHSC SEI for AP config change

Quoted 6 lines · Matthew Rosato
Quoted 4 lines
Quoted 1 line

AFAICT this portion of the handler including mutex should be moved out of ap.c into chsc_sei_nt0_get_event().  When vfio_ap is not in use, we can't call into it.  Can't you also build without VFIO_AP?  I didn't try it but that sure seems like it would cause issues too.

If that were the case, I think the checks I mentioned above would be sufficient because the S390_FEAT_AP feature could not be set.

OK, I tested it quick (remove VFIO_AP from config) and that appears to be the case; this also restores zPCI hotplug capability since we won't call info hw/vfio/ap.c/.

Quoted 7 lines
Quoted 2 lines

What about moving lock initialization to the vfio_ap_class_init() function?

I tried a quick hack of that too and it appears to work, at least for the zPCI regression (note: I did not test the AP portion).

This approach is probably not the way I would have designed it initially... But it does seem to be the simplest thing to do from a fix perspective.

Can you or Rorie prepare a proper patch w/ a fixes tag and send (please add me to CC). Also please test the change against the vfio-ap environment; you can also test zPCI but I would also like to run some tests.

I can make those changes

#10 lore raw

Re: [RFC PATCH v13 4/4] s390: implementing CHSC SEI for AP config change

In reply to Matthew Rosato

Sure

Quoted 13 lines · Matthew Rosato
Quoted 4 lines
Quoted 1 line

AFAICT this portion of the handler including mutex should be moved out of ap.c into chsc_sei_nt0_get_event().  When vfio_ap is not in use, we can't call into it.  Can't you also build without VFIO_AP?  I didn't try it but that sure seems like it would cause issues too.

If that were the case, I think the checks I mentioned above would be sufficient because the S390_FEAT_AP feature could not be set.

OK, I tested it quick (remove VFIO_AP from config) and that appears to be the case; this also restores zPCI hotplug capability since we won't call info hw/vfio/ap.c/.

Quoted 2 lines

What about moving lock initialization to the vfio_ap_class_init() function?

I tried a quick hack of that too and it appears to work, at least for the zPCI regression (note: I did not test the AP portion).

This approach is probably not the way I would have designed it initially... But it does seem to be the simplest thing to do from a fix perspective.

Can you or Rorie prepare a proper patch w/ a fixes tag and send (please add me to CC). Also please test the change against the vfio-ap environment; you can also test zPCI but I would also like to run some tests.

#11 lore raw

Re: [RFC PATCH v13 4/4] s390: implementing CHSC SEI for AP config change

In reply to Matthew Rosato
Quoted 9 lines · Matthew Rosato
Quoted 8 lines · Rorie Reyes

Handle interception of the CHSC SEI instruction for requests indicating the guest's AP configuration has changed.

If configuring --without-default-devices, hw/s390x/ap-stub.c was created to handle such circumstance. Also added the following to hw/s390x/meson.build if CONFIG_VFIO_AP is false, it will use the stub file.

Quoted 6 lines
Quoted 3 lines

FYI, this patch (or some part of this series) breaks hotplug for PCI devices on s390x. I verified it breaks with both PCI passthrough and virtio-pci via virsh attach-device.

I was unaware of 639ff87a1a ("attribute constructor for cfg_chg_events_lock") which apparently was already sitting in vfio-next and merged to master the day after I reported this issue.

FWIW, 639ff87a1a resolves the issue by ensuring the mutex_init happens even if there is no vfio-ap device attached so long as vfio_ap is configured/built (and I like the use of the constructor).

I also now see the stub functions in hw/s390x/ap-stub.c that will be called if vfio_ap is not configured.

So: my concerns are resolved. Thanks!

#12 lore raw
In reply to Rorie Reyes
Quoted 89 lines · Rorie Reyes

Changelog:
v13:
- added lock to 'vfio_ap_cfg_chg_notifier_handler' in patch 2
- added RBs for patch 2 and 3 from Tony

v12:
- adding locks to 'ap_chsc_sei_nt0_have_event' and 'ap_chsc_sei_nt0_get_event'

v11:
- reverted return type to int for 'ap_chsc_sei_nt0_get_event'
- files reflected are 'ap.c', 'ap-bridge'h, and 'ap-stub.c'
- using defined variables to represent return 0 and 1 to reflect logical sense
- update documentation for 'ap_chsc_sei_nt0_get_event' to reflect return types

v10:
- Added reviewed by for patch 4 by Tony and Cedric
- review needed for patch 2 and 3
- changed 'cfg_chg_events' to a static variable
- locked 'cfg_chg_events' using QemuMutex
- removed ';' at the end of the definition for NT0
- returning bools for 'ap_chsc_sei_nt0_get_event' and 'ap_chsc_sei_nt0_have_event'
- updated the header file that contains 'ap_chsc_sei_nt0_get_event' and 'ap_chsc_sei_nt0_have_event' to a bool function
- added documentation explaining the returning bool functions
- whitespace clean up

v9:
- added SPDX licensing to newly created file 'hw/s390x/ap-stub.c'

v8:
- fixed windows cross-compile build
- moved /hw/vfio/ap-stub.c to /hw/s390x/ap-stub.c
- updated the use of stub file to MAINTAINERS to reflect new location
- removed if_false for 'CONFIG_VFIO_AP' statement from /hw/vfio/meson.build
- added if_false for 'CONFIG_VFIO_AP' to use ap-stub.c in /hw/s390x/meson.build
- all those changes still address '--without-default-devices' issue from v5

v7:
- Dropped initial commit for linux-header file vfio.h since I created two new commits to address the changes made in v6
- Moved patches 6 and 7 to the beginning of the series after dropping the first patch
- Because I dropped the initial commit for linux-header file vfio.h, I had to add
VFIO_AP_CFG_CHG_IRQ_INDEX
- Resyncing latest to v6.15-rc3
- Still need Thomas Huth's review of v5 changes for patch 6/6

v6:
- Updating the update-linux-headers script to address kernel commit change 8a14
- Update headers to retrieve uapi information for vfio-ap for update to Linux v6.15-rc1
- Still need Thomas Huth's review of v5 changes for patch 7/7 (see below)

v5:
- configuring using the '--without-default-devices' fails when building the code
- created a stub file for functions ap_chsc_sei_nt0_get_event and ap_chsc_sei_nt0_have_event
- add if_false for 'CONFIG_VFIO_AP' use ap-stub.c in meson.build
- add the use of the stub file to MAINTAINERS since it's a new file

v4:
- allocating cfg_chg_event before inserting into the queue
- calling nt0_have_event in if loop to check if there are any elemenets in the queue, then calling QTAILQ_FIRST when the check passes
- moving memset() after the check

v3:
- changes that were made to patch 3/5 should have been made in patch 2/5

v2:
- removed warnings that weren't needed
- added unregister function
- removed whitelines
- changed variable names for consistency
- removed rc variable and returning 1 or 0 outright
- reversed logics for if statements
- using g_free() instead of free()
- replaced hardcoded numeric values by defining them with #define in the header

--------------------------------------------------------------------------

Rorie Reyes (4):
hw/vfio/ap: notification handler for AP config changed event hw/vfio/ap: store object indicating AP config changed in a queue hw/vfio/ap: Storing event information for an AP configuration change
event
s390: implementing CHSC SEI for AP config change

Quoted 9 lines
  MAINTAINERS                  |  1 +
  hw/s390x/ap-stub.c           | 21 +++++++++
  hw/s390x/meson.build         |  1 +
  hw/vfio/ap.c                 | 88 ++++++++++++++++++++++++++++++++++++
  include/hw/s390x/ap-bridge.h | 39 ++++++++++++++++
  target/s390x/ioinst.c        | 11 ++++-
  6 files changed, 159 insertions(+), 2 deletions(-)

create mode 100644 hw/s390x/ap-stub.c

Applied to vfio-next.

Thanks,

C.