DEVICE_UNPLUG_ERROR QAPI event
v2 cover DEVICE_UNPLUG_ERROR QAPI event
Hi,
This is the v2 of a series that started with 2 events, DEVICE_UNPLUG_ERROR and DEVICE_NOT_DELETED [1]. After discussions in v1 we reached the conclussion that the DEVICE_NOT_DELETED wasn't doing much of anything. It was an event that was trying to say 'I think something happen, but I'm not sure', forcing the QAPI listener to inspect the guest itself to see what went wrong, or just wait for some sort of internal timeout (as Libvirt will do) and fail the operation regardless.
During this period between v1 and this v2 the PowerPC kernel was changed to add a reliable error report mechanism in the device_removal path of CPUs, which in turn gave QEMU the opportunity to do the same. This made the DEVICE_UNPLUG_ERROR more relevant because now we can report CPU and DIMM hotunplug errors.
changes from v1:
- former patches 1 and 2: dropped
- patch 1 (former 3): changed the version to '6.1'
- patch 2 (former 4): add a DEVICE_UNPLUG_ERROR event in the device unplug error path of CPUs and DIMMs
[1] v1 link: https://lists.gnu.org/archive/html/qemu-devel/2021-03/msg04682.html
Daniel Henrique Barboza (2):
qapi/machine.json: add DEVICE_UNPLUG_ERROR QAPI event
spapr: use DEVICE_UNPLUG_ERROR to report unplug errors
hw/ppc/spapr.c | 2 +- hw/ppc/spapr_drc.c | 15 +++++++++------ qapi/machine.json | 23 +++++++++++++++++++++++ 3 files changed, 33 insertions(+), 7 deletions(-)
Signature · 1 line
2.31.1
PATCH v2 1/2 qapi/machine.json: add DEVICE_UNPLUG_ERROR QAPI event
At this moment we only provide one event to report a hotunplug error, MEM_UNPLUG_ERROR. As of Linux kernel 5.12 and QEMU 6.0.0, the pseries machine is now able to report unplug errors for other device types, such as CPUs.
Instead of creating a (device_type)_UNPLUG_ERROR for each new device, create a generic DEVICE_UNPLUG_ERROR event that can be used by all unplug errors in the future.
--- qapi/machine.json | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
qapi/machine.json +23 -0
@@ -1274,3 +1274,26 @@ ## { 'event': 'MEM_UNPLUG_ERROR', 'data': { 'device': 'str', 'msg': 'str' } } + +## +# @DEVICE_UNPLUG_ERROR: +# +# Emitted when a device hot unplug error occurs. +# +# @device: device name +# +# @msg: Informative message +# +# Since: 6.1 +# +# Example: +# +# <- { "event": "DEVICE_UNPLUG_ERROR" +# "data": { "device": "dimm1", +# "msg": "Memory hotunplug rejected by the guest for device dimm1" +# }, +# "timestamp": { "seconds": 1615570772, "microseconds": 202844 } } +# +## +{ 'event': 'DEVICE_UNPLUG_ERROR', + 'data': { 'device': 'str', 'msg': 'str' } }
Signature · 1 line
2.31.1
Re: [PATCH v2 1/2] qapi/machine.json: add DEVICE_UNPLUG_ERROR QAPI event
Quoted 10 lines · Daniel Henrique Barboza
At this moment we only provide one event to report a hotunplug error,
MEM_UNPLUG_ERROR. As of Linux kernel 5.12 and QEMU 6.0.0, the pseries machine is now able to report unplug errors for other device types, such as CPUs.Instead of creating a (device_type)_UNPLUG_ERROR for each new device, create a generic
DEVICE_UNPLUG_ERRORevent that can be used by all unplug errors in the future.Signed-off-by Daniel Henrique Barboza <[email protected]>
Quoted 35 lines
--- qapi/machine.json | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)qapi/machine.json +23
-0@@ -1274,3 +1274,26 @@ ## { 'event': 'MEM_UNPLUG_ERROR', 'data': { 'device': 'str', 'msg': 'str' } } + +## +# @DEVICE_UNPLUG_ERROR: +# +# Emitted when a device hot unplug error occurs. +# +# @device: device name +# +# @msg: Informative message +# +# Since: 6.1 +# +# Example: +# +# <- { "event": "DEVICE_UNPLUG_ERROR" +# "data": { "device": "dimm1", +# "msg": "Memory hotunplug rejected by the guest for device dimm1" +# }, +# "timestamp": { "seconds": 1615570772, "microseconds": 202844 } } +# +## +{ 'event': 'DEVICE_UNPLUG_ERROR', + 'data': { 'device': 'str', 'msg': 'str' } }
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Re: [PATCH v2 1/2] qapi/machine.json: add DEVICE_UNPLUG_ERROR QAPI event
Quoted 6 lines · David Gibson
Quoted 5 lines · Daniel Henrique Barboza
At this moment we only provide one event to report a hotunplug error,
MEM_UNPLUG_ERROR. As of Linux kernel 5.12 and QEMU 6.0.0, the pseries machine is now able to report unplug errors for other device types, such as CPUs.
Quoted 7 lines
Quoted 5 lines
Instead of creating a (device_type)_UNPLUG_ERROR for each new device, create a generic
DEVICE_UNPLUG_ERRORevent that can be used by all unplug errors in the future.Signed-off-by Daniel Henrique Barboza <[email protected]>Reviewed-by David Gibson <[email protected]>
Markus, I'm happy to take this through my tree if that's convenient for you, but I'd like to get an ack.
Quoted 37 lines
Quoted 36 lines
--- qapi/machine.json | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)qapi/machine.json +23
-0@@ -1274,3 +1274,26 @@ ## { 'event': 'MEM_UNPLUG_ERROR', 'data': { 'device': 'str', 'msg': 'str' } } + +## +# @DEVICE_UNPLUG_ERROR: +# +# Emitted when a device hot unplug error occurs. +# +# @device: device name +# +# @msg: Informative message +# +# Since: 6.1 +# +# Example: +# +# <- { "event": "DEVICE_UNPLUG_ERROR" +# "data": { "device": "dimm1", +# "msg": "Memory hotunplug rejected by the guest for device dimm1" +# }, +# "timestamp": { "seconds": 1615570772, "microseconds": 202844 } } +# +## +{ 'event': 'DEVICE_UNPLUG_ERROR', + 'data': { 'device': 'str', 'msg': 'str' } }
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Eric Blake
Reviewed-byRe: [PATCH v2 1/2] qapi/machine.json: add DEVICE_UNPLUG_ERROR QAPI event
Quoted 11 lines · David Gibson
Quoted 10 lines · David Gibson
Quoted 9 lines · Daniel Henrique Barboza
At this moment we only provide one event to report a hotunplug error,
MEM_UNPLUG_ERROR. As of Linux kernel 5.12 and QEMU 6.0.0, the pseries machine is now able to report unplug errors for other device types, such as CPUs.Instead of creating a (device_type)_UNPLUG_ERROR for each new device, create a generic
DEVICE_UNPLUG_ERRORevent that can be used by all unplug errors in the future.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Daniel Henrique Barboza <[email protected]>Reviewed-by David Gibson <[email protected]>Markus, I'm happy to take this through my tree if that's convenient for you, but I'd like to get an ack.
I'm not Markus, but if you're happy with my interface review,
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [PATCH v2 1/2] qapi/machine.json: add DEVICE_UNPLUG_ERROR QAPI event
In reply to Daniel Henrique BarbozaQuoted 35 lines · Daniel Henrique Barboza
At this moment we only provide one event to report a hotunplug error,
MEM_UNPLUG_ERROR. As of Linux kernel 5.12 and QEMU 6.0.0, the pseries machine is now able to report unplug errors for other device types, such as CPUs.Instead of creating a (device_type)_UNPLUG_ERROR for each new device, create a generic
DEVICE_UNPLUG_ERRORevent that can be used by all unplug errors in the future.Signed-off-by Daniel Henrique Barboza <[email protected]>--- qapi/machine.json | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)qapi/machine.json +13
-0@@ -1274,3 +1274,26 @@ ## { 'event': 'MEM_UNPLUG_ERROR', 'data': { 'device': 'str', 'msg': 'str' } } + +## +# @DEVICE_UNPLUG_ERROR: +# +# Emitted when a device hot unplug error occurs. +# +# @device: device name +# +# @msg: Informative message +# +# Since: 6.1 +# +# Example:
qapi/machine.json line 1299
Quoted 10 lines
qapi/machine.json +10
-0+# +# <- { "event": "DEVICE_UNPLUG_ERROR" +# "data": { "device": "dimm1", +# "msg": "Memory hotunplug rejected by the guest for device dimm1" +# }, +# "timestamp": { "seconds": 1615570772, "microseconds": 202844 } } +# +## +{ 'event': 'DEVICE_UNPLUG_ERROR', + 'data': { 'device': 'str', 'msg': 'str' } }
Missing: update of device_add's doc comment in qdev.json:
# Notes: When this command completes, the device may not be removed from the
# guest. Hot removal is an operation that requires guest cooperation.
# This command merely requests that the guest begin the hot removal
# process. Completion of the device removal process is signaled with a
# DEVICE_DELETED event. Guest reset will automatically complete removal
# for all devices.
This sure could use some polish.
If I understand things correctly, we're aiming for the following device unplug protocol:
Unplug the device with device_del (or possibly equivalent)
If we know we can't unplug the device, fail immediately. Also emit DEVICE_UNPLUG_ERROR.
If possible, unplug the device synchronously and succeed. Also emit DEVICE_DELETED.
Else, initiate unplug and succeed.
When unplug finishes, emit either DEVICE_DELETED or DEVICE_UNPLUG_ERROR.
For some machines and devices, unplug may never finish.
Correct?
Any particular reason for not putting event DEVICE_UNPLUG_ERROR next to DEVICE_DELETED in qdev.json?
Re: [PATCH v2 1/2] qapi/machine.json: add DEVICE_UNPLUG_ERROR QAPI event
Quoted 50 lines · Markus Armbruster
Quoted 35 lines · Daniel Henrique Barboza
At this moment we only provide one event to report a hotunplug error,
MEM_UNPLUG_ERROR. As of Linux kernel 5.12 and QEMU 6.0.0, the pseries machine is now able to report unplug errors for other device types, such as CPUs.Instead of creating a (device_type)_UNPLUG_ERROR for each new device, create a generic
DEVICE_UNPLUG_ERRORevent that can be used by all unplug errors in the future.Signed-off-by Daniel Henrique Barboza <[email protected]>--- qapi/machine.json | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)qapi/machine.json +13
-0@@ -1274,3 +1274,26 @@ ## { 'event': 'MEM_UNPLUG_ERROR', 'data': { 'device': 'str', 'msg': 'str' } } + +## +# @DEVICE_UNPLUG_ERROR: +# +# Emitted when a device hot unplug error occurs. +# +# @device: device name +# +# @msg: Informative message +# +# Since: 6.1 +# +# Example:qapi/machine.json line 1299
Quoted 10 lines
qapi/machine.json +10
-0+# +# <- { "event": "DEVICE_UNPLUG_ERROR" +# "data": { "device": "dimm1", +# "msg": "Memory hotunplug rejected by the guest for device dimm1" +# }, +# "timestamp": { "seconds": 1615570772, "microseconds": 202844 } } +# +## +{ 'event': 'DEVICE_UNPLUG_ERROR', + 'data': { 'device': 'str', 'msg': 'str' } }Missing: update of device_add's doc comment in qdev.json:
Quoted 6 lines
# Notes: When this command completes, the device may not be removed from the
# guest. Hot removal is an operation that requires guest cooperation.
# This command merely requests that the guest begin the hot removal
# process. Completion of the device removal process is signaled with a
#DEVICE_DELETEDevent. Guest reset will automatically complete removal
# for all devices.
Ok
Quoted 5 lines
This sure could use some polish.
If I understand things correctly, we're aiming for the following device unplug protocol:
One thing to note is that DEVICE_UNPLUG_ERROR isn't guaranteed to be send for every hotunplug error. The event depends on machine/architecture support to detect a guest side error.
Quoted 5 lines
Unplug the device with
device_del(or possibly equivalent)If we know we can't unplug the device, fail immediately. Also emit
DEVICE_UNPLUG_ERROR.
I haven't predicted to use this event in those cases as well, although it seems reasonable to do so now that you mentioned it.
Quoted 8 lines
If possible, unplug the device synchronously and succeed. Also emit
DEVICE_DELETED.Else, initiate unplug and succeed.
When unplug finishes, emit either
DEVICE_DELETEDorDEVICE_UNPLUG_ERROR.
Since there's no 100% guarantee that DEVICE_UNPLUG_ERROR will be emitted for guest side errors, the wording here would be
"When unplug finishes, emit DEVICE_DELETED. A DEVICE_UNPLUG_ERROR can be emitted if a guest side error was detected"
Quoted 7 lines
For some machines and devices, unplug may never finish.
Correct?
Any particular reason for not putting event
DEVICE_UNPLUG_ERRORnext toDEVICE_DELETEDin qdev.json?
Not really. I looked where MEM_UNPLUG_ERROR was declared and put it right after it. I can change it to qdev.json near DEVICE_DELETED.
Thanks,
Daniel
Quoted 2 lines
Re: [PATCH v2 1/2] qapi/machine.json: add DEVICE_UNPLUG_ERROR QAPI event
Quoted 61 lines · Daniel Henrique Barboza
Quoted 50 lines · Markus Armbruster
Quoted 35 lines · Daniel Henrique Barboza
At this moment we only provide one event to report a hotunplug error,
MEM_UNPLUG_ERROR. As of Linux kernel 5.12 and QEMU 6.0.0, the pseries machine is now able to report unplug errors for other device types, such as CPUs.Instead of creating a (device_type)_UNPLUG_ERROR for each new device, create a generic
DEVICE_UNPLUG_ERRORevent that can be used by all unplug errors in the future.Signed-off-by Daniel Henrique Barboza <[email protected]>--- qapi/machine.json | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)qapi/machine.json +13
-0@@ -1274,3 +1274,26 @@ ## { 'event': 'MEM_UNPLUG_ERROR', 'data': { 'device': 'str', 'msg': 'str' } } + +## +# @DEVICE_UNPLUG_ERROR: +# +# Emitted when a device hot unplug error occurs. +# +# @device: device name +# +# @msg: Informative message +# +# Since: 6.1 +# +# Example:qapi/machine.json line 1299
Quoted 10 lines
qapi/machine.json +10
-0+# +# <- { "event": "DEVICE_UNPLUG_ERROR" +# "data": { "device": "dimm1", +# "msg": "Memory hotunplug rejected by the guest for device dimm1" +# }, +# "timestamp": { "seconds": 1615570772, "microseconds": 202844 } } +# +## +{ 'event': 'DEVICE_UNPLUG_ERROR', + 'data': { 'device': 'str', 'msg': 'str' } }Missing: update of device_add's doc comment in qdev.json:
Quoted 6 lines
# Notes: When this command completes, the device may not be removed from the
# guest. Hot removal is an operation that requires guest cooperation.
# This command merely requests that the guest begin the hot removal
# process. Completion of the device removal process is signaled with a
#DEVICE_DELETEDevent. Guest reset will automatically complete removal
# for all devices.Ok
Quoted 1 line
Quoted 8 lines
Quoted 4 lines
This sure could use some polish.
If I understand things correctly, we're aiming for the following device unplug protocol:
One thing to note is that
DEVICE_UNPLUG_ERRORisn't guaranteed to be send for every hotunplug error. The event depends on machine/architecture support to detect a guest side error.
Yes. I tried to provide for that in my description of the protocol.
Quoted 9 lines
Quoted 5 lines
Unplug the device with
device_del(or possibly equivalent)If we know we can't unplug the device, fail immediately. Also emit
DEVICE_UNPLUG_ERROR.I haven't predicted to use this event in those cases as well, although it seems reasonable to do so now that you mentioned it.
I think this is a matter of taste. For what it's worth, we do emit DEVICE_DELETED on immediate success (see next paragraph).
Emitting DEVICE_DELETED always lets QMP clients track device deletion even when it's triggered by something else. Feature.
No such tracking is needed for unplug failure.
Quoted 5 lines
Quoted 5 lines
If possible, unplug the device synchronously and succeed. Also emit
DEVICE_DELETED.Else, initiate unplug and succeed.
Quoted 8 lines
Quoted 2 lines
When unplug finishes, emit either
DEVICE_DELETEDorDEVICE_UNPLUG_ERROR.Since there's no 100% guarantee that
DEVICE_UNPLUG_ERRORwill be emitted for guest side errors, the wording here would be"When unplug finishes, emit
DEVICE_DELETED. ADEVICE_UNPLUG_ERRORcan be emitted if a guest side error was detected"
My assumption is that QEMU can detect asynchronous success reliably, but not asynchronous failure.
From QEMU's point of view, "asynchronous unplug has not finished" is indistinguishable from "asynchronous unplug finished unsuccessfully, but we can't detect it". So I lumped these two cases together in the next paragraph:
Quoted 1 line
Quoted 1 line
For some machines and devices, unplug may never finish.
My goal is a clear description of the state machine as it can be observed in QMP.
Quoted 8 lines
Quoted 4 lines
Correct?
Any particular reason for not putting event
DEVICE_UNPLUG_ERRORnext toDEVICE_DELETEDin qdev.json?Not really. I looked where
MEM_UNPLUG_ERRORwas declared and put it right after it. I can change it to qdev.json nearDEVICE_DELETED.
Yes, please.
PATCH v2 2/2 spapr: use DEVICE_UNPLUG_ERROR to report unplug errors
In reply to Daniel Henrique BarbozaLinux Kernel 5.12 is now unisolating CPU DRCs in the device_removal error path, signalling that the hotunplug process wasn't successful. This allow us to send a DEVICE_UNPLUG_ERROR in drc_unisolate_logical() to signal this error to the management layer.
We also have another error path in spapr_memory_unplug_rollback() for configured LMB DRCs. Kernels older than 5.13 will not unisolate the LMBs in the hotunplug error path, but it will reconfigure them. Let's send the DEVICE_UNPLUG_ERROR event in that code path as well to cover the case of older kernels.
--- hw/ppc/spapr.c | 2 +- hw/ppc/spapr_drc.c | 15 +++++++++------ 2 files changed, 10 insertions(+), 7 deletions(-)
hw/ppc/spapr.c +1 -1
@@ -3639,7 +3639,7 @@ void spapr_memory_unplug_rollback(SpaprMachineState *spapr, DeviceState *dev)*/ qapi_error =g_strdup_printf ("Memory hotunplug rejected by the guest " "for device %s" , dev->id); -qapi_event_send_mem_unplug_error (dev->id, qapi_error); +qapi_event_send_device_unplug_error (dev->id, qapi_error); }/* Callback to be called during DRC release. */
hw/ppc/spapr_drc.c +9 -6
@@ -17,6 +17,8 @@#include "hw/ppc/spapr_drc.h" #include "qom/object.h" #include "migration/vmstate.h" +#include "qapi/error.h" +#include "qapi/qapi-events-machine.h" #include "qapi/visitor.h" #include "qemu/error-report.h" #include "hw/ppc/spapr.h" /* for RTAS return codes */
@@ -160,6 +162,10 @@ static uint32_t drc_unisolate_logical(SpaprDrc *drc)* means that the kernel is refusing the removal. */ if (drc->unplug_requested && drc->dev) { +const char qapi_error_fmt[] ="Device hotunplug rejected by the " +"guest for device %s" ; + g_autofree char *qapi_error = NULL; +if (spapr_drc_type (drc) ==SPAPR_DR_CONNECTOR_TYPE_LMB ) { spapr =SPAPR_MACHINE (qdev_get_machine ());
@@ -167,13 +173,10 @@ static uint32_t drc_unisolate_logical(SpaprDrc *drc) } drc->unplug_requested = false; -error_report ("Device hotunplug rejected by the guest " -"for device %s" , drc->dev->id); +error_report (qapi_error_fmt, drc->dev->id); -/* -* TODO: send a QAPI DEVICE_UNPLUG_ERROR event when -* it is implemented. -*/ + qapi_error =g_strdup_printf (qapi_error_fmt, drc->dev->id); +qapi_event_send_device_unplug_error (drc->dev->id, qapi_error); }return RTAS_OUT_SUCCESS ;/* Nothing to do */
Signature · 1 line
2.31.1
Re: [PATCH v2 2/2] spapr: use DEVICE_UNPLUG_ERROR to report unplug errors
Quoted 5 lines · Daniel Henrique Barboza
Linux Kernel 5.12 is now unisolating CPU DRCs in the
device_removalerror path, signalling that the hotunplug process wasn't successful. This allow us to send aDEVICE_UNPLUG_ERRORindrc_unisolate_logical()to signal this error to the management layer.
Quoted 7 lines
We also have another error path in
spapr_memory_unplug_rollback()for configured LMB DRCs. Kernels older than 5.13 will not unisolate the LMBs in the hotunplug error path, but it will reconfigure them. Let's send theDEVICE_UNPLUG_ERRORevent in that code path as well to cover the case of older kernels.Signed-off-by Daniel Henrique Barboza <[email protected]>
Quoted 59 lines
--- hw/ppc/spapr.c | 2 +- hw/ppc/spapr_drc.c | 15 +++++++++------ 2 files changed, 10 insertions(+), 7 deletions(-)hw/ppc/spapr.c +1
-1@@ -3639,7 +3639,7 @@ void spapr_memory_unplug_rollback(SpaprMachineState *spapr, DeviceState *dev)*/ qapi_error =g_strdup_printf ("Memory hotunplug rejected by the guest " "for device %s" , dev->id); -qapi_event_send_mem_unplug_error (dev->id, qapi_error); +qapi_event_send_device_unplug_error (dev->id, qapi_error); }/* Callback to be called during DRC release. */ hw/ppc/spapr_drc.c +9
-6@@ -17,6 +17,8 @@#include "hw/ppc/spapr_drc.h" #include "qom/object.h" #include "migration/vmstate.h" +#include "qapi/error.h" +#include "qapi/qapi-events-machine.h" #include "qapi/visitor.h" #include "qemu/error-report.h" #include "hw/ppc/spapr.h" /* for RTAS return codes */ @@ -160,6 +162,10 @@ static uint32_t drc_unisolate_logical(SpaprDrc *drc)* means that the kernel is refusing the removal. */ if (drc->unplug_requested && drc->dev) { +const char qapi_error_fmt[] ="Device hotunplug rejected by the " +"guest for device %s" ; + g_autofree char *qapi_error = NULL; +if (spapr_drc_type (drc) ==SPAPR_DR_CONNECTOR_TYPE_LMB ) { spapr =SPAPR_MACHINE (qdev_get_machine ());@@ -167,13 +173,10 @@ static uint32_t drc_unisolate_logical(SpaprDrc *drc) } drc->unplug_requested = false; -error_report ("Device hotunplug rejected by the guest " -"for device %s" , drc->dev->id); +error_report (qapi_error_fmt, drc->dev->id); -/* -* TODO: send a QAPI DEVICE_UNPLUG_ERROR event when -* it is implemented. -*/ + qapi_error =g_strdup_printf (qapi_error_fmt, drc->dev->id); +qapi_event_send_device_unplug_error (drc->dev->id, qapi_error); }return RTAS_OUT_SUCCESS ;/* Nothing to do */
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Re: [PATCH v2 2/2] spapr: use DEVICE_UNPLUG_ERROR to report unplug errors
In reply to Daniel Henrique BarbozaQuoted 22 lines · Daniel Henrique Barboza
Linux Kernel 5.12 is now unisolating CPU DRCs in the
device_removalerror path, signalling that the hotunplug process wasn't successful. This allow us to send aDEVICE_UNPLUG_ERRORindrc_unisolate_logical()to signal this error to the management layer.We also have another error path in
spapr_memory_unplug_rollback()for configured LMB DRCs. Kernels older than 5.13 will not unisolate the LMBs in the hotunplug error path, but it will reconfigure them. Let's send theDEVICE_UNPLUG_ERRORevent in that code path as well to cover the case of older kernels.Signed-off-by Daniel Henrique Barboza <[email protected]>--- hw/ppc/spapr.c | 2 +- hw/ppc/spapr_drc.c | 15 +++++++++------ 2 files changed, 10 insertions(+), 7 deletions(-)hw/ppc/spapr.c +0
-0
hw/ppc/spapr.c void spapr_memory_unplug_rollback(SpaprMachineState *spapr, DeviceState *dev) · line 3642
Quoted 5 lines
hw/ppc/spapr.c +1
-1@@ -3639,7 +3639,7 @@ void spapr_memory_unplug_rollback(SpaprMachineState *spapr, DeviceState *dev)*/ qapi_error =g_strdup_printf ("Memory hotunplug rejected by the guest " "for device %s" , dev->id); -qapi_event_send_mem_unplug_error (dev->id, qapi_error); +qapi_event_send_device_unplug_error (dev->id, qapi_error);
Incompatible change: we now emit DEVICE_UNPLUG_ERROR instead of MEM_UNPLUG_ERROR. Intentional?
If yes, we need a release note.
To avoid the incompatible, we can emit both, and deprecate MEM_UNPLUG_ERROR.
What about the MEM_UNPLUG_ERROR in acpi_memory_hotplug_write()?
Quoted 34 lines
hw/ppc/spapr.c +0
-0}/* Callback to be called during DRC release. */ hw/ppc/spapr_drc.c +7
-2@@ -17,6 +17,8 @@#include "hw/ppc/spapr_drc.h" #include "qom/object.h" #include "migration/vmstate.h" +#include "qapi/error.h" +#include "qapi/qapi-events-machine.h" #include "qapi/visitor.h" #include "qemu/error-report.h" #include "hw/ppc/spapr.h" /* for RTAS return codes */ @@ -160,6 +162,10 @@ static uint32_t drc_unisolate_logical(SpaprDrc *drc)* means that the kernel is refusing the removal. */ if (drc->unplug_requested && drc->dev) { +const char qapi_error_fmt[] ="Device hotunplug rejected by the " +"guest for device %s" ; + g_autofree char *qapi_error = NULL; +if (spapr_drc_type (drc) ==SPAPR_DR_CONNECTOR_TYPE_LMB ) { spapr =SPAPR_MACHINE (qdev_get_machine ());@@ -167,13 +173,10 @@ static uint32_t drc_unisolate_logical(SpaprDrc *drc) } drc->unplug_requested = false; -error_report ("Device hotunplug rejected by the guest " -"for device %s" , drc->dev->id); +error_report (qapi_error_fmt, drc->dev->id);
hw/ppc/spapr_drc.c static uint32_t drc_unisolate_logical(SpaprDrc *drc) · line 182
Quoted 10 lines
hw/ppc/spapr_drc.c +2
-4-/* -* TODO: send a QAPI DEVICE_UNPLUG_ERROR event when -* it is implemented. -*/ + qapi_error =g_strdup_printf (qapi_error_fmt, drc->dev->id); +qapi_event_send_device_unplug_error (drc->dev->id, qapi_error); }return RTAS_OUT_SUCCESS ;/* Nothing to do */
Reporting both to stderr and QMP is odd. Can you describe a use case where the report to stderr is useful?
Re: [PATCH v2 2/2] spapr: use DEVICE_UNPLUG_ERROR to report unplug errors
Quoted 33 lines · Markus Armbruster
Quoted 22 lines · Daniel Henrique Barboza
Linux Kernel 5.12 is now unisolating CPU DRCs in the
device_removalerror path, signalling that the hotunplug process wasn't successful. This allow us to send aDEVICE_UNPLUG_ERRORindrc_unisolate_logical()to signal this error to the management layer.We also have another error path in
spapr_memory_unplug_rollback()for configured LMB DRCs. Kernels older than 5.13 will not unisolate the LMBs in the hotunplug error path, but it will reconfigure them. Let's send theDEVICE_UNPLUG_ERRORevent in that code path as well to cover the case of older kernels.Signed-off-by Daniel Henrique Barboza <[email protected]>--- hw/ppc/spapr.c | 2 +- hw/ppc/spapr_drc.c | 15 +++++++++------ 2 files changed, 10 insertions(+), 7 deletions(-)hw/ppc/spapr.c +0
-0hw/ppc/spapr.c void spapr_memory_unplug_rollback(SpaprMachineState *spapr, DeviceState *dev) · line 3642
Quoted 5 lines
hw/ppc/spapr.c +1
-1@@ -3639,7 +3639,7 @@ void spapr_memory_unplug_rollback(SpaprMachineState *spapr, DeviceState *dev)*/ qapi_error =g_strdup_printf ("Memory hotunplug rejected by the guest " "for device %s" , dev->id); -qapi_event_send_mem_unplug_error (dev->id, qapi_error); +qapi_event_send_device_unplug_error (dev->id, qapi_error);Incompatible change: we now emit
DEVICE_UNPLUG_ERRORinstead ofMEM_UNPLUG_ERROR. Intentional?
Quoted 6 lines
If yes, we need a release note.
To avoid the incompatible, we can emit both, and deprecate
MEM_UNPLUG_ERROR.What about the
MEM_UNPLUG_ERRORinacpi_memory_hotplug_write()?
I'll emit DEVICE_UNPLUG_ERROR together with all MEM_UNPLUG_ERROR instances. Then we can deprecate MEM_UNPLUG_ERROR.
By the way, how do I mark MEM_UNPLUG_ERROR as deprecated? I see examples of command line options being documented as deprecated in docs/system/deprecated.rst and some deprecated QOM/QDEV properties are marked as deprecated directly in their .json files, but I didn't find any case where a whole event is deprecated. Would something like this be adequate?
$ git diff
qapi/machine.json +4 -0
@@ -1261,6 +1261,10 @@ # # @msg: Informative message # +# +# @deprecated: Starting in 6.1 this event has been replaced by +# DEVICE_UNPLUG_ERROR. +# # Since: 2.4 # # Example:
Thanks,
Daniel
Quoted 49 lines
Quoted 35 lines
hw/ppc/spapr.c +0
-0}/* Callback to be called during DRC release. */ hw/ppc/spapr_drc.c +7
-2@@ -17,6 +17,8 @@#include "hw/ppc/spapr_drc.h" #include "qom/object.h" #include "migration/vmstate.h" +#include "qapi/error.h" +#include "qapi/qapi-events-machine.h" #include "qapi/visitor.h" #include "qemu/error-report.h" #include "hw/ppc/spapr.h" /* for RTAS return codes */ @@ -160,6 +162,10 @@ static uint32_t drc_unisolate_logical(SpaprDrc *drc)* means that the kernel is refusing the removal. */ if (drc->unplug_requested && drc->dev) { +const char qapi_error_fmt[] ="Device hotunplug rejected by the " +"guest for device %s" ; + g_autofree char *qapi_error = NULL; +if (spapr_drc_type (drc) ==SPAPR_DR_CONNECTOR_TYPE_LMB ) { spapr =SPAPR_MACHINE (qdev_get_machine ());@@ -167,13 +173,10 @@ static uint32_t drc_unisolate_logical(SpaprDrc *drc) } drc->unplug_requested = false; -error_report ("Device hotunplug rejected by the guest " -"for device %s" , drc->dev->id); +error_report (qapi_error_fmt, drc->dev->id);hw/ppc/spapr_drc.c static uint32_t drc_unisolate_logical(SpaprDrc *drc) · line 182
Quoted 10 lines
hw/ppc/spapr_drc.c +2
-4-/* -* TODO: send a QAPI DEVICE_UNPLUG_ERROR event when -* it is implemented. -*/ + qapi_error =g_strdup_printf (qapi_error_fmt, drc->dev->id); +qapi_event_send_device_unplug_error (drc->dev->id, qapi_error); }return RTAS_OUT_SUCCESS ;/* Nothing to do */ Reporting both to stderr and QMP is odd. Can you describe a use case where the report to stderr is useful?
Re: [PATCH v2 2/2] spapr: use DEVICE_UNPLUG_ERROR to report unplug errors
Quoted 13 lines · Daniel Henrique Barboza
Quoted 9 lines
Incompatible change: we now emit
DEVICE_UNPLUG_ERRORinstead ofMEM_UNPLUG_ERROR. Intentional?If yes, we need a release note.
To avoid the incompatible, we can emit both, and deprecate
MEM_UNPLUG_ERROR.What about the
MEM_UNPLUG_ERRORinacpi_memory_hotplug_write()?I'll emit
DEVICE_UNPLUG_ERRORtogether with allMEM_UNPLUG_ERRORinstances. Then we can deprecateMEM_UNPLUG_ERROR.
Quoted 6 lines
By the way, how do I mark
MEM_UNPLUG_ERRORas deprecated? I see examples of command line options being documented as deprecated indocs/system/deprecated.rstand some deprecated QOM/QDEV properties are marked as deprecated directly in their .json files, but I didn't find any case where a whole event is deprecated. Would something like this be adequate?
Almost. That documents the deprecation for readers, but you also need to mark it for viewing by machine code...
Quoted 8 lines
$ git diffqapi/machine.json +0
-0
qapi/machine.json line 1270
Quoted 10 lines
qapi/machine.json +4
-0@@ -1261,6 +1261,10 @@ # # @msg: Informative message # +# +# @deprecated: Starting in 6.1 this event has been replaced by +# DEVICE_UNPLUG_ERROR. +# # Since: 2.4 # # Example:
...do that by adding 'features':['deprecated'] to the QAPI event definition.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Quoted 17 lines · Daniel Henrique Barboza
Hi,
This is the v2 of a series that started with 2 events,
DEVICE_UNPLUG_ERRORandDEVICE_NOT_DELETED[1]. After discussions in v1 we reached the conclussion that theDEVICE_NOT_DELETEDwasn't doing much of anything. It was an event that was trying to say 'I think something happen, but I'm not sure', forcing the QAPI listener to inspect the guest itself to see what went wrong, or just wait for some sort of internal timeout (as Libvirt will do) and fail the operation regardless.During this period between v1 and this v2 the PowerPC kernel was changed to add a reliable error report mechanism in the
device_removalpath of CPUs, which in turn gave QEMU the opportunity to do the same. This made theDEVICE_UNPLUG_ERRORmore relevant because now we can report CPU and DIMM hotunplug errors.
Quoted 8 lines
changes from v1:
- former patches 1 and 2: dropped
- patch 1 (former 3): changed the version to '6.1'
- patch 2 (former 4): add aDEVICE_UNPLUG_ERRORevent in the device unplug error path of CPUs and DIMMs[1] v1 link:
https://lists.gnu.org/archive/html/qemu-devel/2021-03/msg04682.html
It would be nice to add a patch making x86 also issue the new error format (as well as the old) on memory hot unplug errors.
Quoted 12 lines
Daniel Henrique Barboza (2):
qapi/machine.json: addDEVICE_UNPLUG_ERRORQAPI event
spapr: useDEVICE_UNPLUG_ERRORto report unplug errorshw/ppc/spapr.c | 2 +- hw/ppc/spapr_drc.c | 15 +++++++++------ qapi/machine.json | 23 +++++++++++++++++++++++ 3 files changed, 33 insertions(+), 7 deletions(-)
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson