PCI: liveupdate: PCI core support for Live Update

MAINTAINERS +2 -0

index c9b7b6f9828e..94af31837375 100644
@@ -20555,6 +20555,18 @@ L:	[email protected]
 S:	Supported
 F:	Documentation/PCI/pci-error-recovery.rst
 
+PCI LIVE UPDATE
+M:	Bjorn Helgaas <[email protected]>

MAINTAINERS +0 -0

 PCI MSI DRIVER FOR ALTERA MSI IP
 L:	[email protected]
 S:	Orphan

Hi David,

On Thu, Apr 23 2026, David Matlack wrote:
[...]

Quoted 10 lines

MAINTAINERS +2 -0

index c9b7b6f9828e..94af31837375 100644
@@ -20555,6 +20555,18 @@ L:	[email protected]
 S:	Supported
 F:	Documentation/PCI/pci-error-recovery.rst
 
+PCI LIVE UPDATE
+M:	Bjorn Helgaas <[email protected]>

MAINTAINERS L: [email protected] · line 20569

Quoted 10 lines

MAINTAINERS +10 -0

+M:	David Matlack <[email protected]>
+L:	[email protected]
+S:	Supported
+Q:	https://patchwork.kernel.org/project/linux-pci/list/
+B:	https://bugzilla.kernel.org
+C:	irc://irc.oftc.net/linux-pci
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git
+F:	drivers/pci/liveupdate.c
+F:	include/linux/kho/abi/pci.h
+

Can we please also add these files under the "LIVE UPDATE" entry. The code here concerns both live update and PCI. We can figure out the maintenance details as we go along, but I think the live update maintainers should at least get all the patches for PCI live update.

Perhaps also add the kexec list here? We plan to use it to maintain the LUO patches, and adding it will make sure we get the patches in case someone updates the file list here but forgets to update it in the LIVE UPDATE entry.

Quoted 12 lines · Pratyush Yadav

Hi David,

On Thu, Apr 23 2026, David Matlack wrote:
[...]

Quoted 8 lines

MAINTAINERS +0 -0

index c9b7b6f9828e..94af31837375 100644
@@ -20555,6 +20555,18 @@ L:	[email protected]
 S:	Supported
 F:	Documentation/PCI/pci-error-recovery.rst
 

Quoted 7 lines

Quoted 7 lines

MAINTAINERS +7 -0

+PCI LIVE UPDATE
+M:	Bjorn Helgaas <[email protected]>
+M:	David Matlack <[email protected]>
+L:	[email protected]
+S:	Supported
+Q:	https://patchwork.kernel.org/project/linux-pci/list/
+B:	https://bugzilla.kernel.org

Quoted 8 lines

MAINTAINERS L: [email protected] · line 20569

Quoted 5 lines

MAINTAINERS +5 -0

+C:	irc://irc.oftc.net/linux-pci
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git
+F:	drivers/pci/liveupdate.c
+F:	include/linux/kho/abi/pci.h
+

Can we please also add these files under the "LIVE UPDATE" entry. The code here concerns both live update and PCI.

Quoted 3 lines

We can figure out the
maintenance details as we go along, but I think the live update maintainers should at least get all the patches for PCI live update.

Quoted 5 lines

Perhaps also add the kexec list here? We plan to use it to maintain the LUO patches, and adding it will make sure we get the patches in case someone updates the file list here but forgets to update it in the LIVE UPDATE entry.

+1

These files should also be added to the Live Update entry, and the kexec mailing list should be included.

Changes specific to Live Update should be routed through the liveupdate/linux.git tree, while generic PCI changes should go through pci/pci.git. In either case, if liveupdate.c or abi/pci.h are modified, acks are required from the Live Update group.

Quoted 13 lines · Pasha Tatashin

Quoted 12 lines · Pratyush Yadav

Hi David,

On Thu, Apr 23 2026, David Matlack wrote:
[...]

Quoted 8 lines

MAINTAINERS +0 -0

index c9b7b6f9828e..94af31837375 100644
@@ -20555,6 +20555,18 @@ L:	[email protected]
 S:	Supported
 F:	Documentation/PCI/pci-error-recovery.rst
 

Quoted 15 lines

Quoted 7 lines

Quoted 7 lines

MAINTAINERS +7 -0

+PCI LIVE UPDATE
+M:	Bjorn Helgaas <[email protected]>
+M:	David Matlack <[email protected]>
+L:	[email protected]
+S:	Supported
+Q:	https://patchwork.kernel.org/project/linux-pci/list/
+B:	https://bugzilla.kernel.org

Quoted 8 lines

MAINTAINERS L: [email protected] · line 20569

Quoted 5 lines

MAINTAINERS +5 -0

+C:	irc://irc.oftc.net/linux-pci
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git
+F:	drivers/pci/liveupdate.c
+F:	include/linux/kho/abi/pci.h
+

Can we please also add these files under the "LIVE UPDATE" entry. The code here concerns both live update and PCI.

Covering that intersection of Live Update and PCI was exactly my intention with introducing this new PCI LIVE UPDATE entry. This ensures we have maintenance coverage with knowledge of that intersection.

Quoted 3 lines

Quoted 3 lines

We can figure out the
maintenance details as we go along, but I think the live update maintainers should at least get all the patches for PCI live update.

Would adding kexec@ here be sufficient or do you want to be CC'd directly?

If you want to be CC'd directly do you think makes more sense to add the Live Update maintainers as Reviewers under PCI LIVE UPDATE, or add drivers/pci/liveupdate.c under LIVE UPDATE?

Quoted 8 lines

Quoted 5 lines

Perhaps also add the kexec list here? We plan to use it to maintain the LUO patches, and adding it will make sure we get the patches in case someone updates the file list here but forgets to update it in the LIVE UPDATE entry.

+1

Quoted 7 lines

These files should also be added to the Live Update entry, and the kexec mailing list should be included.

Changes specific to Live Update should be routed through the liveupdate/linux.git tree, while generic PCI changes should go through pci/pci.git. In either case, if liveupdate.c or abi/pci.h are modified, acks are required from the Live Update group.

Do you want to merge changes to drivers/pci/liveupdate.c through the live update tree or PCI tree? We should probably decide now. I was assuming the PCI tree since its part of PCI core.

As we project this out there are going to be users of the Live Update API across different parts of the kernel: PCI core, IOMMU core, IOMMU drivers, VFIO core, VFIO PCI drivers, and KVM. I don't think it will scale to take all that code through the live update tree.

Quoted 61 lines · David Matlack

Quoted 13 lines · Pasha Tatashin

Quoted 12 lines · Pratyush Yadav

Hi David,

On Thu, Apr 23 2026, David Matlack wrote:
[...]

Quoted 8 lines

MAINTAINERS +0 -0

index c9b7b6f9828e..94af31837375 100644
@@ -20555,6 +20555,18 @@ L:	[email protected]
 S:	Supported
 F:	Documentation/PCI/pci-error-recovery.rst
 

Quoted 15 lines

Quoted 7 lines

Quoted 7 lines

MAINTAINERS +7 -0

+PCI LIVE UPDATE
+M:	Bjorn Helgaas <[email protected]>
+M:	David Matlack <[email protected]>
+L:	[email protected]
+S:	Supported
+Q:	https://patchwork.kernel.org/project/linux-pci/list/
+B:	https://bugzilla.kernel.org

Quoted 8 lines

MAINTAINERS L: [email protected] · line 20569

Quoted 5 lines

MAINTAINERS +5 -0

+C:	irc://irc.oftc.net/linux-pci
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git
+F:	drivers/pci/liveupdate.c
+F:	include/linux/kho/abi/pci.h
+

Can we please also add these files under the "LIVE UPDATE" entry. The code here concerns both live update and PCI.

Covering that intersection of Live Update and PCI was exactly my intention with introducing this new PCI LIVE UPDATE entry. This ensures we have maintenance coverage with knowledge of that intersection.

Quoted 3 lines

Quoted 3 lines

We can figure out the
maintenance details as we go along, but I think the live update maintainers should at least get all the patches for PCI live update.

Would adding kexec@ here be sufficient or do you want to be CC'd directly?

If you want to be CC'd directly do you think makes more sense to add the Live Update maintainers as Reviewers under PCI LIVE UPDATE, or add drivers/pci/liveupdate.c under LIVE UPDATE?

Quoted 8 lines

Quoted 5 lines

Perhaps also add the kexec list here? We plan to use it to maintain the LUO patches, and adding it will make sure we get the patches in case someone updates the file list here but forgets to update it in the LIVE UPDATE entry.

+1

Quoted 7 lines

These files should also be added to the Live Update entry, and the kexec mailing list should be included.

Changes specific to Live Update should be routed through the liveupdate/linux.git tree, while generic PCI changes should go through pci/pci.git. In either case, if liveupdate.c or abi/pci.h are modified, acks are required from the Live Update group.

Quoted 8 lines

Do you want to merge changes to drivers/pci/liveupdate.c through the live update tree or PCI tree? We should probably decide now. I was assuming the PCI tree since its part of PCI core.

As we project this out there are going to be users of the Live Update API across different parts of the kernel: PCI core, IOMMU core, IOMMU drivers, VFIO core, VFIO PCI drivers, and KVM. I don't think it will scale to take all that code through the live update tree.

All Live-Update-specific changes should go through the liveupdate tree. The liveupdate tree is the only Linux tree that will cover full Live Update regression testing, and it contains reviewers and maintainers who know the details of the Live Update process, its lifecycle, and its requirements.

The request we are hearing from other subsystem maintainers is that they want to make sure Live Update is isolated enough not to make their lives harder. This means reducing the number of conflicts, the maintenance burden, and testing responsibilities.

Therefore, the "PCI LIVE UPDATE" entry should specify you as a maintainer, "[email protected]" as the list to which all LU changes should be CC'd, and "liveupdate/linux.git" as the git tree against which changes should be applied.

It should also include "[email protected]" so the PCI maintainers are CC'd. In case there are larger changes that touch core PCI and liveupdate.c/abi, we can ACK them, ensuring we are aware of incoming conflicts during the current or next merge cycle.

It should also specify the members of LU group so we can stage the changes.

This is the way we agreed to handle kexec changes: Baoquan He is the maintainer, and without his Reviewed-by tag, we won't take changes to kexec. This is the approach we follow with MM for KHO changes to memblock and memfd preservation, as well as the upcoming hugetlb/guestmemfd preservation.

This is also the approach we should continue using when adding LUO support to other components like PCI, VFIO, IOMMU, and KVM. It keeps life easier for the core component maintainers and ensures we do not regress LU by staging everything in the same tree and sending LU merge requests from a single tree.

Quoted 36 lines · Pasha Tatashin

Quoted 35 lines · David Matlack

Quoted 11 lines · Pasha Tatashin

Quoted 10 lines · Pratyush Yadav

On Thu, Apr 23 2026, David Matlack wrote:
[...]

Quoted 8 lines

MAINTAINERS +0 -0

index c9b7b6f9828e..94af31837375 100644
@@ -20555,6 +20555,18 @@ L:	[email protected]
 S:	Supported
 F:	Documentation/PCI/pci-error-recovery.rst
 

Quoted 15 lines

Quoted 7 lines

Quoted 7 lines

MAINTAINERS +7 -0

+PCI LIVE UPDATE
+M:	Bjorn Helgaas <[email protected]>
+M:	David Matlack <[email protected]>
+L:	[email protected]
+S:	Supported
+Q:	https://patchwork.kernel.org/project/linux-pci/list/
+B:	https://bugzilla.kernel.org

Quoted 8 lines

MAINTAINERS L: [email protected] · line 20569

Quoted 5 lines

MAINTAINERS +5 -0

+C:	irc://irc.oftc.net/linux-pci
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git
+F:	drivers/pci/liveupdate.c
+F:	include/linux/kho/abi/pci.h
+

Can we please also add these files under the "LIVE UPDATE" entry. The code here concerns both live update and PCI.

Covering that intersection of Live Update and PCI was exactly my intention with introducing this new PCI LIVE UPDATE entry. This ensures we have maintenance coverage with knowledge of that intersection.

Quoted 3 lines

Quoted 3 lines

We can figure out the
maintenance details as we go along, but I think the live update maintainers should at least get all the patches for PCI live update.

...

Quoted 11 lines

This is the way we agreed to handle kexec changes: Baoquan He is the maintainer, and without his Reviewed-by tag, we won't take changes to kexec. This is the approach we follow with MM for KHO changes to memblock and memfd preservation, as well as the upcoming hugetlb/guestmemfd preservation.

This is also the approach we should continue using when adding LUO support to other components like PCI, VFIO, IOMMU, and KVM. It keeps life easier for the core component maintainers and ensures we do not regress LU by staging everything in the same tree and sending LU merge requests from a single tree.

Ok it sounds like we are aligned on keeping drivers/pci/liveupdate.c, include/linux/kho/abi/pci.h, and Documentation/PCI/liveupdate.rst in the PCI LIVE UPDATE entry and not duplicating them in the LIVE UPDATE entry.

I think the only open question is what tree to use for the PCI LIVE UPDATE entry, PCI tree or Live Update tree. You are proposing the Live Update tree.

Bjorn are you ok with that approach?

Quoted 22 lines · David Matlack

Quoted 11 lines · Pasha Tatashin

This is the way we agreed to handle kexec changes: Baoquan He is the maintainer, and without his Reviewed-by tag, we won't take changes to kexec. This is the approach we follow with MM for KHO changes to memblock and memfd preservation, as well as the upcoming hugetlb/guestmemfd preservation.

This is also the approach we should continue using when adding LUO support to other components like PCI, VFIO, IOMMU, and KVM. It keeps life easier for the core component maintainers and ensures we do not regress LU by staging everything in the same tree and sending LU merge requests from a single tree.

Ok it sounds like we are aligned on keeping drivers/pci/liveupdate.c, include/linux/kho/abi/pci.h, and Documentation/PCI/liveupdate.rst in the PCI LIVE UPDATE entry and not duplicating them in the LIVE UPDATE entry.

I think the only open question is what tree to use for the PCI LIVE UPDATE entry, PCI tree or Live Update tree. You are proposing the Live Update tree.

Quoted 1 line

Bjorn are you ok with that approach?

Yes, I think that makes sense, at least to start. In early days, it would probably be a headache to coordinate LU things across multiple trees.

Set up a File-Lifecycle-Bound (FLB) handler for the PCI core to enable it to participate in the preservation of PCI devices across Live Update. Essentially, this commit enables the PCI core to allocate a struct (struct pci_ser) and preserve it across a Live Update whenever at least one device is preserved.

Preserving PCI devices across Live Update is built on top of the Live Update Orchestrator's (LUO) support for file preservation. Drivers are expected to expose a file to userspace to represent a single PCI device and support preservation of that file. This is itended primarily to support preservation of PCI devices bound to VFIO drivers.

MAINTAINERS +9 -0

+M:	David Matlack <[email protected]>
+L:	[email protected]
+S:	Supported
+Q:	https://patchwork.kernel.org/project/linux-pci/list/
+B:	https://bugzilla.kernel.org
+C:	irc://irc.oftc.net/linux-pci
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git
+F:	drivers/pci/liveupdate.c
+F:	include/linux/kho/abi/pci.h

MAINTAINERS L: [email protected] · line 20559

Quoted 7 lines

+++ b/MAINTAINERS

MAINTAINERS +2 -0

@@ -20555,6 +20555,18 @@ L:   [email protected]
 S:   Supported
 F:   Documentation/PCI/pci-error-recovery.rst
 
+PCI LIVE UPDATE
+M:   Bjorn Helgaas <[email protected]>

I don't think I want to be the primary contact here. I'll see patches regardless since they're on linux-pci.

include/linux/kho/abi/pci.h new file +11 -0

+} __packed;
+
+/**
+ * struct pci_ser - PCI Subsystem Live Update State
+ *
+ * This struct tracks state about all devices that are being preserved across
+ * a Live Update for the next kernel.
+ *
+ * @max_nr_devices: The length of the devices[] flexible array.
+ * @nr_devices: The number of devices that were preserved.
+ * @devices: Flexible array of pci_dev_ser structs for each device.

drivers/pci/liveupdate.c line 84

Quoted 6 lines · David Matlack

drivers/pci/liveupdate.c new file +6 -0

+	pr_debug("Preserving struct pci_ser with room for %u devices\n",
+		 max_nr_devices);
+
+	ser = kho_alloc_preserve(size);
+	if (IS_ERR(ser))
+		return PTR_ERR(ser);

Should there be a similar pr_debug() in case of failure to denote that above "Preserving ..." message didn't finish, or, maybe just print one pr_debug() after the error check above?

include/linux/kho/abi/pci.h line 38

Quoted 12 lines

include/linux/kho/abi/pci.h new file +11 -0

+/**
+ * struct pci_dev_ser - Serialized state about a single PCI device.
+ *
+ * @domain: The device's PCI domain number (segment).
+ * @bdf: The device's PCI bus, device, and function number.
+ * @reserved: Reserved (to naturally align struct pci_dev_ser).
+ */
+struct pci_dev_ser {
+	u32 domain;
+	u16 bdf;
+	u16 reserved;

Should this be renamed to 'u8 __padding[2];' instead? This will allow to just change the array length based on the need (0, 1, 2, 3).

Quoted 17 lines

include/linux/kho/abi/pci.h new file +17 -0

+} __packed;
+
+/**
+ * struct pci_ser - PCI Subsystem Live Update State
+ *
+ * This struct tracks state about all devices that are being preserved across
+ * a Live Update for the next kernel.
+ *
+ * @max_nr_devices: The length of the devices[] flexible array.
+ * @nr_devices: The number of devices that were preserved.
+ * @devices: Flexible array of pci_dev_ser structs for each device.
+ */
+struct pci_ser {
+	u32 max_nr_devices;
+	u32 nr_devices;
+	struct pci_dev_ser devices[];
+} __packed;

include/linux/kho/abi/pci.h line 59

Quoted 4 lines

include/linux/kho/abi/pci.h new file +4 -0

+
+/* Ensure all elements of devices[] are naturally aligned. */
+static_assert(offsetof(struct pci_ser, devices) % sizeof(unsigned long) == 0);
+static_assert(sizeof(struct pci_dev_ser) % sizeof(unsigned long) == 0);

Nit: Maybe move this assert to be near to the definition of this struct, easier to find it when editing the struct vs finding it later during build.

Quoted 11 lines · Vipin Sharma

drivers/pci/liveupdate.c line 84

Quoted 6 lines · David Matlack

drivers/pci/liveupdate.c new file +6 -0

+	pr_debug("Preserving struct pci_ser with room for %u devices\n",
+		 max_nr_devices);
+
+	ser = kho_alloc_preserve(size);
+	if (IS_ERR(ser))
+		return PTR_ERR(ser);

Should there be a similar pr_debug() in case of failure to denote that above "Preserving ..." message didn't finish, or, maybe just print one pr_debug() after the error check above?

Hm... I guess there could always be more pr_debug()s but I don't want to instrument every error path. I could move it to the success path but I don't see how that makes it any better.

Quoted 15 lines

include/linux/kho/abi/pci.h line 38

Quoted 12 lines

include/linux/kho/abi/pci.h new file +11 -0

+/**
+ * struct pci_dev_ser - Serialized state about a single PCI device.
+ *
+ * @domain: The device's PCI domain number (segment).
+ * @bdf: The device's PCI bus, device, and function number.
+ * @reserved: Reserved (to naturally align struct pci_dev_ser).
+ */
+struct pci_dev_ser {
+	u32 domain;
+	u16 bdf;
+	u16 reserved;

Should this be renamed to 'u8 __padding[2];' instead? This will allow to just change the array length based on the need (0, 1, 2, 3).

Sorry I'm not following what you mean here. What is the reason to rename this field and change it to an array?

Quoted 17 lines

Quoted 17 lines

include/linux/kho/abi/pci.h new file +17 -0

+} __packed;
+
+/**
+ * struct pci_ser - PCI Subsystem Live Update State
+ *
+ * This struct tracks state about all devices that are being preserved across
+ * a Live Update for the next kernel.
+ *
+ * @max_nr_devices: The length of the devices[] flexible array.
+ * @nr_devices: The number of devices that were preserved.
+ * @devices: Flexible array of pci_dev_ser structs for each device.
+ */
+struct pci_ser {
+	u32 max_nr_devices;
+	u32 nr_devices;
+	struct pci_dev_ser devices[];
+} __packed;

Quoted 8 lines

include/linux/kho/abi/pci.h line 59

Quoted 4 lines

include/linux/kho/abi/pci.h new file +4 -0

+
+/* Ensure all elements of devices[] are naturally aligned. */
+static_assert(offsetof(struct pci_ser, devices) % sizeof(unsigned long) == 0);
+static_assert(sizeof(struct pci_dev_ser) % sizeof(unsigned long) == 0);

Nit: Maybe move this assert to be near to the definition of this struct, easier to find it when editing the struct vs finding it later during build.

The combination of these 2 asserts is what guarantees that every element of the devices[] array are naturally aligned, that's why I put them together here.

I can move it up though if you think it's better.

Quoted 13 lines · David Matlack

Quoted 11 lines · Vipin Sharma

drivers/pci/liveupdate.c line 84

Quoted 6 lines · David Matlack

drivers/pci/liveupdate.c new file +6 -0

+	pr_debug("Preserving struct pci_ser with room for %u devices\n",
+		 max_nr_devices);
+
+	ser = kho_alloc_preserve(size);
+	if (IS_ERR(ser))
+		return PTR_ERR(ser);

Should there be a similar pr_debug() in case of failure to denote that above "Preserving ..." message didn't finish, or, maybe just print one pr_debug() after the error check above?

Quoted 4 lines

Hm... I guess there could always be more pr_debug()s but I don't want to instrument every error path. I could move it to the success path but I don't see how that makes it any better.

In current way, it is printing that it is preserving pci_ser, but there is no indication did it succeed or not in logs. If we are printing the logs then may be complete picture will be to know what is the action and its result.

I think moving to success path or printing again based on the failure provides assurance of what happened. If this gets printed in happy path, then we will know it succeeded in preserving that struct on kho. Absence means it didn't.

We can also remove pr_debug(), if this is of no value.

Quoted 17 lines

Quoted 16 lines

include/linux/kho/abi/pci.h line 38

Quoted 13 lines

include/linux/kho/abi/pci.h new file +11 -0

+/**
+ * struct pci_dev_ser - Serialized state about a single PCI device.
+ *
+ * @domain: The device's PCI domain number (segment).
+ * @bdf: The device's PCI bus, device, and function number.
+ * @reserved: Reserved (to naturally align struct pci_dev_ser).
+ */
+struct pci_dev_ser {
+	u32 domain;
+	u16 bdf;
+	u16 reserved;

Should this be renamed to 'u8 __padding[2];' instead? This will allow to just change the array length based on the need (0, 1, 2, 3).

Quoted 3 lines

Sorry I'm not following what you mean here. What is the reason to rename this field and change it to an array?

Having a padding explicitly tells there is a requirement of being aligned. Reserved sounds more like don't use this u16.

If someone add more field down the line say u8, then to make struct size aligned they will need to add another u8, u16, u32, and name those fields padding or reserved. IMO, having a u8 array named padding makes it easier to just change array length as per the need.

Add APIs to allow drivers to notify the PCI core of which devices are being preserved across a Live Update for the next kernel, i.e. "outgoing" devices.

Drivers must notify the PCI core when devices are preserved so that the PCI core can update its FLB data (struct pci_ser) and track the list of outgoing devices. pci_liveupdate_preserve() notifies the PCI core that a device must be preserved across Live Update. pci_liveupdate_unpreserve() reverses this (cancels the preservation of the device).

This tracking ensures the PCI core is fully aware of which devices may need special handling during shutdown and kexec, and so that it can be handed off to the next kernel.

Signed-off-by David Matlack <[email protected]>

---
 drivers/pci/liveupdate.c    | 101
++++++++++++++++++++++++++++++++++++ include/linux/kho/abi/pci.h |
7 +-- include/linux/pci.h         |  26 ++++++++++
 3 files changed, 131 insertions(+), 3 deletions(-)

drivers/pci/liveupdate.c +5 -0

index d4fa61625d56..2dd8daa2f17c 100644
@@ -43,6 +43,26 @@
  *
  *  * ``pci_liveupdate_register_flb(driver_file_handler)``
  *  * ``pci_liveupdate_unregister_flb(driver_file_handler)``
+ *
+ * Device Tracking
+ * ===============
+ *
+ * Drivers must notify the PCI core when specific devices are

preserved or
+ * unpreserved with the following APIs:
+ *
+ * * ``pci_liveupdate_preserve(pci_dev)``
+ * * ``pci_liveupdate_unpreserve(pci_dev)``
+ *
+ * This allows the PCI core to keep it's FLB data (struct pci_ser) up to date
+ * with the list of **outgoing** preserved devices for the next kernel.
+ *
+ * Restrictions
+ * ============
+ *
+ * The PCI core enforces the following restrictions on which devices can be
+ * preserved. These may be relaxed in the future:
+ *
+ * * The device cannot be a Virtual Function (VF).
*/

#define pr_fmt(fmt) "PCI: liveupdate: " fmt

drivers/pci/liveupdate.c +9 -0

@@ -57,6 +77,8 @@
 #include <linux/pci.h>
 #include <linux/sort.h>
 
+static DEFINE_MUTEX(pci_flb_outgoing_lock);
+
 static int pci_flb_preserve(struct liveupdate_flb_op_args *args)
 {
 	struct pci_dev *dev = NULL;

@@ -124,6 +146,85 @@ static struct liveupdate_flb pci_liveupdate_flb
 = { .compatible = PCI_LUO_FLB_COMPATIBLE,
 };
 
+int pci_liveupdate_preserve(struct pci_dev *dev)
+{
+	struct pci_ser *ser;
+	int i, ret;
+
+	guard(mutex)(&pci_flb_outgoing_lock);
+

drivers/pci/liveupdate.c +10 -0

+
+	if (dev->liveupdate_outgoing)
+		return -EBUSY;
+
+	if (ser->nr_devices == ser->max_nr_devices)
+		return -ENOSPC;
+
+	for (i = 0; i < ser->max_nr_devices; i++) {
+		/*
+		 * Start searching at index ser->nr_devices. This

should result

drivers/pci/liveupdate.c +1 -0

+		 * in a constant time search under expected

conditions (devices

drivers/pci/liveupdate.c +3 -0

+		 * are not getting unpreserved).
+		 */
+		int index = (ser->nr_devices + i) %

ser->max_nr_devices;

drivers/pci/liveupdate.c +6 -0

+		struct pci_dev_ser *dev_ser = &ser->devices[index];
+
+		if (dev_ser->refcount)
+			continue;
+
+		pci_info(dev, "Device will be preserved across next

Live Update\n");

drivers/pci/liveupdate.c +23 -0

+		ser->nr_devices++;
+
+		dev_ser->domain = pci_domain_nr(dev->bus);
+		dev_ser->bdf = pci_dev_id(dev);
+		dev_ser->refcount = 1;
+
+		dev->liveupdate_outgoing = dev_ser;
+		return 0;
+	}
+
+	return -ENOSPC;
+}
+EXPORT_SYMBOL_GPL(pci_liveupdate_preserve);
+
+void pci_liveupdate_unpreserve(struct pci_dev *dev)
+{
+	struct pci_dev_ser *dev_ser;
+	struct pci_ser *ser = NULL;
+	int ret;
+
+	guard(mutex)(&pci_flb_outgoing_lock);
+
+	ret = liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void

**)&ser); +

drivers/pci/liveupdate.c +2 -0

+	if (ret || !ser) {
+		pci_warn(dev, "Cannot unpreserve device without

outgoing Live Update state\n");

drivers/pci/liveupdate.c +7 -0

+		return;
+
+	}
+
+	dev_ser = dev->liveupdate_outgoing;
+	if (!dev_ser) {
+		pci_warn(dev, "Cannot unpreserve device that is not

preserved\n");

drivers/pci/liveupdate.c +4 -0

+		return;
+	}
+
+	pci_info(dev, "Device will no longer be preserved across

next Live Update\n");

drivers/pci/liveupdate.c +6 -0

+	ser->nr_devices--;
+	memset(dev_ser, 0, sizeof(*dev_ser));
+	dev->liveupdate_outgoing = NULL;
+}
+EXPORT_SYMBOL_GPL(pci_liveupdate_unpreserve);
+
 int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh)
 {
 	pr_debug("Registering file handler \"%s\"\n",

fh->compatible); diff --git a/include/linux/kho/abi/pci.h b/include/linux/kho/abi/pci.h index 5c0e92588c00..5b4c8d9e462c 100644

include/linux/kho/abi/pci.h +0 -0

@@ -23,19 +23,20 @@
  * incrementing the version number in the PCI_LUO_FLB_COMPATIBLE

string. */

include/linux/kho/abi/pci.h +2 -2

-#define PCI_LUO_FLB_COMPATIBLE "pci-v1"
+#define PCI_LUO_FLB_COMPATIBLE "pci-v2"
 
 /**
  * struct pci_dev_ser - Serialized state about a single PCI device.
  *
  * @domain: The device's PCI domain number (segment).
  * @bdf: The device's PCI bus, device, and function number.
- * @reserved: Reserved (to naturally align struct pci_dev_ser).
+ * @refcount: Reference count used by the PCI core to keep track of

whether it
+ * is done using a device's struct pci_dev_ser.
*/
struct pci_dev_ser {

include/linux/kho/abi/pci.h +1 -1

 	u32 domain;
 	u16 bdf;
-	u16 reserved;
+	u16 refcount;
 } __packed;
 
 /**

include/linux/pci.h +3 -0

index d70080babd52..eb94cbd8ab9d 100644
@@ -41,6 +41,7 @@
 #include <linux/msi_api.h>
 #include <uapi/linux/pci.h>
 #include <linux/liveupdate.h>
+#include <linux/kho/abi/pci.h>
 
 #include <linux/pci_ids.h>
 

@@ -594,6 +595,9 @@ struct pci_dev {
 	u8		tph_mode;	/* TPH mode */
 	u8		tph_req_type;	/* TPH requester type
 */ #endif
+#ifdef CONFIG_PCI_LIVEUPDATE
+	struct pci_dev_ser *liveupdate_outgoing; /* State preserved

for next kernel */ +#endif
};

static inline struct pci_dev *pci_physfn(struct pci_dev *dev)

include/linux/pci.h +6 -0

@@ -2880,6 +2884,14 @@ void pci_uevent_ers(struct pci_dev *pdev, enum
 pci_ers_result err_type); #ifdef CONFIG_PCI_LIVEUPDATE
 int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh);
 void pci_liveupdate_unregister_flb(struct liveupdate_file_handler
 *fh); +
+int pci_liveupdate_preserve(struct pci_dev *dev);
+void pci_liveupdate_unpreserve(struct pci_dev *dev);
+
+static inline struct pci_dev_ser *pci_liveupdate_outgoing(struct
 pci_dev *dev) +{
+	return dev->liveupdate_outgoing;
+}
 #else
 static inline int pci_liveupdate_register_flb(struct

liveupdate_file_handler *fh) {

include/linux/pci.h +0 -0

@@ -2889,6 +2901,20 @@ static inline int

pci_liveupdate_register_flb(struct liveupdate_file_handler *fh static inline void pci_liveupdate_unregister_flb(struct liveupdate_file_handler *fh) { }

include/linux/pci.h +11 -0

+
+static inline int pci_liveupdate_preserve(struct pci_dev *dev)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline void pci_liveupdate_unpreserve(struct pci_dev *dev)
+{
+}
+
+static inline struct pci_dev_ser *pci_liveupdate_outgoing(struct

pci_dev *dev) +{

include/linux/pci.h +2 -0

+	return NULL;
+}
 #endif
 
 #endif /* LINUX_PCI_H */

Quoted 10 lines

drivers/pci/liveupdate.c +8 -0

+int pci_liveupdate_preserve(struct pci_dev *dev)
+{
+     struct pci_ser *ser;
+     int i, ret;
+
+     guard(mutex)(&pci_flb_outgoing_lock);
+
+     ret = liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void

**)&ser);
+ if (ret)

drivers/pci/liveupdate.c static struct liveupdate_flb pci_liveupdate_flb = { · line 164

Quoted 7 lines

drivers/pci/liveupdate.c +7 -0

+             return ret;
+
+     if (!ser)
+             return -ENOENT;
+
+     if (dev->is_virtfn)
+             return -EINVAL;

This can be left outside the mutex?

Add APIs to allow drivers to notify the PCI core of which devices are being preserved across a Live Update for the next kernel, i.e. "outgoing" devices.

+++ b/drivers/pci/liveupdate.c

drivers/pci/liveupdate.c +2 -0

@@ -43,6 +43,26 @@
  *
  *  * ``pci_liveupdate_register_flb(driver_file_handler)``
  *  * ``pci_liveupdate_unregister_flb(driver_file_handler)``
+ *
+ * Device Tracking

Add APIs to allow drivers to notify the PCI core of which devices are being preserved across a Live Update for the next kernel, i.e. "outgoing" devices.

Drivers must notify the PCI core when devices are preserved so that the PCI core can update its FLB data (struct pci_ser) and track the list of outgoing devices. pci_liveupdate_preserve() notifies the PCI core that a device must be preserved across Live Update. pci_liveupdate_unpreserve() reverses this (cancels the preservation of the device).

This tracking ensures the PCI core is fully aware of which devices may need special handling during shutdown and kexec, and so that it can be handed off to the next kernel.

Signed-off-by David Matlack <[email protected]>

---

drivers/pci/liveupdate.c    | 101 ++++++++++++++++++++++++++++++++++++
include/linux/kho/abi/pci.h |   7 +--
include/linux/pci.h         |  26 ++++++++++
3 files changed, 131 insertions(+), 3 deletions(-)

drivers/pci/liveupdate.c +43 -0

index d4fa61625d56..2dd8daa2f17c 100644
@@ -43,6 +43,26 @@
 *
 *  * ``pci_liveupdate_register_flb(driver_file_handler)``
 *  * ``pci_liveupdate_unregister_flb(driver_file_handler)``
+ *
+ * Device Tracking
+ * ===============
+ *
+ * Drivers must notify the PCI core when specific devices are preserved or
+ * unpreserved with the following APIs:
+ *
+ *  * ``pci_liveupdate_preserve(pci_dev)``
+ *  * ``pci_liveupdate_unpreserve(pci_dev)``
+ *
+ * This allows the PCI core to keep it's FLB data (struct pci_ser) up to date
+ * with the list of **outgoing** preserved devices for the next kernel.
+ *
+ * Restrictions
+ * ============
+ *
+ * The PCI core enforces the following restrictions on which devices can be
+ * preserved. These may be relaxed in the future:
+ *
+ *  * The device cannot be a Virtual Function (VF).
 */
 
 #define pr_fmt(fmt) "PCI: liveupdate: " fmt

@@ -57,6 +77,8 @@
 #include <linux/pci.h>
 #include <linux/sort.h>
 
+static DEFINE_MUTEX(pci_flb_outgoing_lock);
+
 static int pci_flb_preserve(struct liveupdate_flb_op_args *args)
 {
 	struct pci_dev *dev = NULL;

@@ -124,6 +146,85 @@ static struct liveupdate_flb pci_liveupdate_flb = {
 	.compatible = PCI_LUO_FLB_COMPATIBLE,
 };
 
+int pci_liveupdate_preserve(struct pci_dev *dev)
+{
+	struct pci_ser *ser;
+	int i, ret;
+
+	guard(mutex)(&pci_flb_outgoing_lock);
+
+	ret = liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser);
+	if (ret)
+		return ret;
+
+	if (!ser)
+		return -ENOENT;
+
+	if (dev->is_virtfn)
+		return -EINVAL;
+
+	if (dev->liveupdate_outgoing)
+		return -EBUSY;
+
+	if (ser->nr_devices == ser->max_nr_devices)

drivers/pci/liveupdate.c +48 -0

+
+		if (dev_ser->refcount)
+			continue;
+
+		pci_info(dev, "Device will be preserved across next Live Update\n");
+		ser->nr_devices++;
+
+		dev_ser->domain = pci_domain_nr(dev->bus);
+		dev_ser->bdf = pci_dev_id(dev);
+		dev_ser->refcount = 1;
+
+		dev->liveupdate_outgoing = dev_ser;
+		return 0;
+	}
+
+	return -ENOSPC;
+}
+EXPORT_SYMBOL_GPL(pci_liveupdate_preserve);
+
+void pci_liveupdate_unpreserve(struct pci_dev *dev)
+{
+	struct pci_dev_ser *dev_ser;
+	struct pci_ser *ser = NULL;
+	int ret;
+
+	guard(mutex)(&pci_flb_outgoing_lock);
+
+	ret = liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser);
+
+	if (ret || !ser) {
+		pci_warn(dev, "Cannot unpreserve device without outgoing Live Update state\n");
+		return;
+
+	}
+
+	dev_ser = dev->liveupdate_outgoing;
+	if (!dev_ser) {
+		pci_warn(dev, "Cannot unpreserve device that is not preserved\n");
+		return;
+	}
+
+	pci_info(dev, "Device will no longer be preserved across next Live Update\n");
+	ser->nr_devices--;
+	memset(dev_ser, 0, sizeof(*dev_ser));
+	dev->liveupdate_outgoing = NULL;
+}
+EXPORT_SYMBOL_GPL(pci_liveupdate_unpreserve);
+
 int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh)
 {
 	pr_debug("Registering file handler \"%s\"\n", fh->compatible);

include/linux/kho/abi/pci.h +4 -3

index 5c0e92588c00..5b4c8d9e462c 100644
@@ -23,19 +23,20 @@
 * incrementing the version number in the PCI_LUO_FLB_COMPATIBLE string.
 */
 
-#define PCI_LUO_FLB_COMPATIBLE "pci-v1"
+#define PCI_LUO_FLB_COMPATIBLE "pci-v2"
 
 /**
 * struct pci_dev_ser - Serialized state about a single PCI device.
 *
 * @domain: The device's PCI domain number (segment).
 * @bdf: The device's PCI bus, device, and function number.
- * @reserved: Reserved (to naturally align struct pci_dev_ser).
+ * @refcount: Reference count used by the PCI core to keep track of whether it
+ *            is done using a device's struct pci_dev_ser.
 */
 struct pci_dev_ser {
 	u32 domain;
 	u16 bdf;
-	u16 reserved;
+	u16 refcount;
 } __packed;
 
 /**

include/linux/pci.h +4 -0

index d70080babd52..eb94cbd8ab9d 100644
@@ -41,6 +41,7 @@
 #include <linux/msi_api.h>
 #include <uapi/linux/pci.h>
 #include <linux/liveupdate.h>
+#include <linux/kho/abi/pci.h>
 
 #include <linux/pci_ids.h>
 

@@ -594,6 +595,9 @@ struct pci_dev {
 	u8		tph_mode;	/* TPH mode */
 	u8		tph_req_type;	/* TPH requester type */
 #endif
+#ifdef CONFIG_PCI_LIVEUPDATE
+	struct pci_dev_ser *liveupdate_outgoing; /* State preserved for next kernel */
+#endif
 };
 
 static inline struct pci_dev *pci_physfn(struct pci_dev *dev)

@@ -2880,6 +2884,14 @@ void pci_uevent_ers(struct pci_dev *pdev, enum  pci_ers_result err_type);
 #ifdef CONFIG_PCI_LIVEUPDATE

include/linux/pci.h +14 -0

 #else
 static inline int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh)
 {

@@ -2889,6 +2901,20 @@ static inline int pci_liveupdate_register_flb(struct liveupdate_file_handler *fh
 static inline void pci_liveupdate_unregister_flb(struct liveupdate_file_handler *fh)
 {
 }
+
+static inline int pci_liveupdate_preserve(struct pci_dev *dev)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline void pci_liveupdate_unpreserve(struct pci_dev *dev)
+{
+}
+
+static inline struct pci_dev_ser *pci_liveupdate_outgoing(struct pci_dev *dev)
+{
+	return NULL;
+}
 #endif
 
 #endif /* LINUX_PCI_H */

Signature · 1 line

2.54.0.rc2.544.gc7ae2d5bb8-goog

Quoted 32 lines · David Matlack

Add APIs to allow drivers to notify the PCI core of which devices are being preserved across a Live Update for the next kernel, i.e. "outgoing" devices.

Drivers must notify the PCI core when devices are preserved so that the PCI core can update its FLB data (struct pci_ser) and track the list of outgoing devices. pci_liveupdate_preserve() notifies the PCI core that a device must be preserved across Live Update. pci_liveupdate_unpreserve() reverses this (cancels the preservation of the device).

This tracking ensures the PCI core is fully aware of which devices may need special handling during shutdown and kexec, and so that it can be handed off to the next kernel.

Signed-off-by David Matlack <[email protected]>

---

drivers/pci/liveupdate.c    | 101 ++++++++++++++++++++++++++++++++++++
include/linux/kho/abi/pci.h |   7 +--
include/linux/pci.h         |  26 ++++++++++
3 files changed, 131 insertions(+), 3 deletions(-)

include/linux/kho/abi/pci.h +1 -1

index 5c0e92588c00..5b4c8d9e462c 100644
@@ -23,19 +23,20 @@
 * incrementing the version number in the PCI_LUO_FLB_COMPATIBLE string.
 */
 
-#define PCI_LUO_FLB_COMPATIBLE "pci-v1"
+#define PCI_LUO_FLB_COMPATIBLE "pci-v2"
 

include/linux/kho/abi/pci.h line 34

Quoted 8 lines

include/linux/kho/abi/pci.h +2 -1

 /**
 * struct pci_dev_ser - Serialized state about a single PCI device.
 *
 * @domain: The device's PCI domain number (segment).
 * @bdf: The device's PCI bus, device, and function number.
- * @reserved: Reserved (to naturally align struct pci_dev_ser).
+ * @refcount: Reference count used by the PCI core to keep track of whether it
+ *            is done using a device's struct pci_dev_ser.

include/linux/kho/abi/pci.h line 42

Quoted 9 lines

include/linux/kho/abi/pci.h +1 -1

 */
 struct pci_dev_ser {
 	u32 domain;
 	u16 bdf;
-	u16 reserved;
+	u16 refcount;
 } __packed;
 
 /**

Quoted 33 lines · Samiullah Khawaja

Quoted 32 lines · David Matlack

Add APIs to allow drivers to notify the PCI core of which devices are being preserved across a Live Update for the next kernel, i.e. "outgoing" devices.

Drivers must notify the PCI core when devices are preserved so that the PCI core can update its FLB data (struct pci_ser) and track the list of outgoing devices. pci_liveupdate_preserve() notifies the PCI core that a device must be preserved across Live Update. pci_liveupdate_unpreserve() reverses this (cancels the preservation of the device).

This tracking ensures the PCI core is fully aware of which devices may need special handling during shutdown and kexec, and so that it can be handed off to the next kernel.

Signed-off-by David Matlack <[email protected]>

---

drivers/pci/liveupdate.c    | 101 ++++++++++++++++++++++++++++++++++++
include/linux/kho/abi/pci.h |   7 +--
include/linux/pci.h         |  26 ++++++++++
3 files changed, 131 insertions(+), 3 deletions(-)

include/linux/kho/abi/pci.h +1 -1

index 5c0e92588c00..5b4c8d9e462c 100644
@@ -23,19 +23,20 @@
 * incrementing the version number in the PCI_LUO_FLB_COMPATIBLE string.
 */
 
-#define PCI_LUO_FLB_COMPATIBLE "pci-v1"
+#define PCI_LUO_FLB_COMPATIBLE "pci-v2"
 

Quoted 8 lines

include/linux/kho/abi/pci.h line 34

Quoted 8 lines

include/linux/kho/abi/pci.h +2 -1

 /**
 * struct pci_dev_ser - Serialized state about a single PCI device.
 *
 * @domain: The device's PCI domain number (segment).
 * @bdf: The device's PCI bus, device, and function number.
- * @reserved: Reserved (to naturally align struct pci_dev_ser).
+ * @refcount: Reference count used by the PCI core to keep track of whether it
+ *            is done using a device's struct pci_dev_ser.

nit: I was wondering why is this named refcount and why it is needed. It seems you are using it to track the preservation dependency between parent devices and leaf. Maybe add a comment to clarify that.

drivers/pci/liveupdate.c static struct liveupdate_flb pci_liveupdate_flb = { · line 179

Quoted 8 lines · David Matlack

drivers/pci/liveupdate.c +8 -0

+	for (i = 0; i < ser->max_nr_devices; i++) {
+		/*
+		 * Start searching at index ser->nr_devices. This should result
+		 * in a constant time search under expected conditions (devices
+		 * are not getting unpreserved).
+		 */
+		int index = (ser->nr_devices + i) % ser->max_nr_devices;
+		struct pci_dev_ser *dev_ser = &ser->devices[index];

nit: Maybe we can move this logic in a separate function as down the road when we expand this to add VFs and Hotpluggable devices, this might change significantly? It's good if it is self-contained.

include/linux/pci.h void pci_uevent_ers(struct pci_dev *pdev, enum pci_ers_result err_type); · line 2894

Quoted 4 lines

include/linux/pci.h +4 -0

+static inline struct pci_dev_ser *pci_liveupdate_outgoing(struct pci_dev *dev)
+{
+	return dev->liveupdate_outgoing;
+}

Is this expected to be called under the outgoing lock?

Quoted 14 lines · Samiullah Khawaja

drivers/pci/liveupdate.c static struct liveupdate_flb pci_liveupdate_flb = { · line 179

Quoted 8 lines · David Matlack

drivers/pci/liveupdate.c +8 -0

+	for (i = 0; i < ser->max_nr_devices; i++) {
+		/*
+		 * Start searching at index ser->nr_devices. This should result
+		 * in a constant time search under expected conditions (devices
+		 * are not getting unpreserved).
+		 */
+		int index = (ser->nr_devices + i) % ser->max_nr_devices;
+		struct pci_dev_ser *dev_ser = &ser->devices[index];

nit: Maybe we can move this logic in a separate function as down the road when we expand this to add VFs and Hotpluggable devices, this might change significantly? It's good if it is self-contained.

Did you mean to leave this comment on pci_flb_preserve() where it decides how many devices to allocate room for?

Quoted 7 lines

include/linux/pci.h void pci_uevent_ers(struct pci_dev *pdev, enum pci_ers_result err_type); · line 2894

Quoted 5 lines

include/linux/pci.h +4 -0

+static inline struct pci_dev_ser *pci_liveupdate_outgoing(struct pci_dev *dev)
+{
+	return dev->liveupdate_outgoing;
+}

Is this expected to be called under the outgoing lock?

For now this API is only used during shutdown, at which point userspace should have already been stopped so drivers should not be changing the preservation status of an outgoing device. So I don't think this needs to be under the outgoing lock, but it would be nice to have some more explicit synchronization.

Quoted 16 lines · David Matlack

Quoted 14 lines · Samiullah Khawaja

drivers/pci/liveupdate.c static struct liveupdate_flb pci_liveupdate_flb = { · line 179

Quoted 8 lines · David Matlack

drivers/pci/liveupdate.c +8 -0

+  for (i = 0; i < ser->max_nr_devices; i++) {
+          /*
+           * Start searching at index ser->nr_devices. This should result
+           * in a constant time search under expected conditions (devices
+           * are not getting unpreserved).
+           */
+          int index = (ser->nr_devices + i) % ser->max_nr_devices;
+          struct pci_dev_ser *dev_ser = &ser->devices[index];

nit: Maybe we can move this logic in a separate function as down the road when we expand this to add VFs and Hotpluggable devices, this might change significantly? It's good if it is self-contained.

Quoted 2 lines

Did you mean to leave this comment on pci_flb_preserve() where it decides how many devices to allocate room for?

I was talking about this one, as I think depending on the scheme we take this might change significantly. Just a nit, you can ignore it.

Quoted 8 lines

Quoted 7 lines

include/linux/pci.h void pci_uevent_ers(struct pci_dev *pdev, enum pci_ers_result err_type); · line 2894

Quoted 5 lines

include/linux/pci.h +4 -0

+static inline struct pci_dev_ser *pci_liveupdate_outgoing(struct pci_dev *dev)
+{
+  return dev->liveupdate_outgoing;
+}

Is this expected to be called under the outgoing lock?

Quoted 5 lines

For now this API is only used during shutdown, at which point userspace should have already been stopped so drivers should not be changing the preservation status of an outgoing device. So I don't think this needs to be under the outgoing lock, but it would be nice to have some more explicit synchronization.

Ok that makes sense. I have similar cases in my series, but maybe we can add kdoc regarding these stating in which context this is expected to be used?

drivers/pci/liveupdate.c +37 -0

+int pci_liveupdate_preserve(struct pci_dev *dev)
+{
+	struct pci_ser *ser;
+	int i, ret;
+
+	guard(mutex)(&pci_flb_outgoing_lock);
+
+	ret = liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser);
+	if (ret)
+		return ret;
+
+	if (!ser)
+		return -ENOENT;
+
+	if (dev->is_virtfn)
+		return -EINVAL;
+
+	if (dev->liveupdate_outgoing)
+		return -EBUSY;
+
+	if (ser->nr_devices == ser->max_nr_devices)
+		return -ENOSPC;
+
+	for (i = 0; i < ser->max_nr_devices; i++) {
+		/*
+		 * Start searching at index ser->nr_devices. This should result
+		 * in a constant time search under expected conditions (devices
+		 * are not getting unpreserved).
+		 */
+		int index = (ser->nr_devices + i) % ser->max_nr_devices;
+		struct pci_dev_ser *dev_ser = &ser->devices[index];
+
+		if (dev_ser->refcount)
+			continue;
+
+		pci_info(dev, "Device will be preserved across next Live Update\n");
+		ser->nr_devices++;

Quoted 43 lines · David Matlack

drivers/pci/liveupdate.c +43 -0

+int pci_liveupdate_preserve(struct pci_dev *dev)
+{
+     struct pci_ser *ser;
+     int i, ret;
+
+     guard(mutex)(&pci_flb_outgoing_lock);
+
+     ret = liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser);
+     if (ret)
+             return ret;
+
+     if (!ser)
+             return -ENOENT;
+
+     if (dev->is_virtfn)
+             return -EINVAL;
+
+     if (dev->liveupdate_outgoing)
+             return -EBUSY;
+
+     if (ser->nr_devices == ser->max_nr_devices)
+             return -ENOSPC;
+
+     for (i = 0; i < ser->max_nr_devices; i++) {
+             /*
+              * Start searching at index ser->nr_devices. This should result
+              * in a constant time search under expected conditions (devices
+              * are not getting unpreserved).
+              */
+             int index = (ser->nr_devices + i) % ser->max_nr_devices;
+             struct pci_dev_ser *dev_ser = &ser->devices[index];
+
+             if (dev_ser->refcount)
+                     continue;
+
+             pci_info(dev, "Device will be preserved across next Live Update\n");
+             ser->nr_devices++;
+
+             dev_ser->domain = pci_domain_nr(dev->bus);
+             dev_ser->bdf = pci_dev_id(dev);
+             dev_ser->refcount = 1;
+
+             dev->liveupdate_outgoing = dev_ser;

drivers/pci/liveupdate.c static struct liveupdate_flb pci_liveupdate_flb = { · line 195

Quoted 4 lines

drivers/pci/liveupdate.c +4 -0

+             return 0;
+     }
+
+     return -ENOSPC;

Since it is executing under a mutex, and we already failed
'if (ser->nr_devices == ser->max_nr_devices) check above, will we ever reach here and return -ENOSPC?

Quoted 8 lines

include/linux/kho/abi/pci.h +0 -0

index 5c0e92588c00..5b4c8d9e462c 100644
@@ -23,19 +23,20 @@
  * incrementing the version number in the PCI_LUO_FLB_COMPATIBLE string.
  */
 

include/linux/kho/abi/pci.h line 26

Quoted 2 lines

include/linux/kho/abi/pci.h +1 -1

-#define PCI_LUO_FLB_COMPATIBLE "pci-v1"
+#define PCI_LUO_FLB_COMPATIBLE "pci-v2"

Just curious, why did we change the version here?

It's not like just
previous patch is working enough to perform a live update. As the config is experimental, can't we just keep it PCI-v1 for the whole series?

Quoted 45 lines · Vipin Sharma

Quoted 43 lines · David Matlack

drivers/pci/liveupdate.c +43 -0

+int pci_liveupdate_preserve(struct pci_dev *dev)
+{
+     struct pci_ser *ser;
+     int i, ret;
+
+     guard(mutex)(&pci_flb_outgoing_lock);
+
+     ret = liveupdate_flb_get_outgoing(&pci_liveupdate_flb, (void **)&ser);
+     if (ret)
+             return ret;
+
+     if (!ser)
+             return -ENOENT;
+
+     if (dev->is_virtfn)
+             return -EINVAL;
+
+     if (dev->liveupdate_outgoing)
+             return -EBUSY;
+
+     if (ser->nr_devices == ser->max_nr_devices)
+             return -ENOSPC;
+
+     for (i = 0; i < ser->max_nr_devices; i++) {
+             /*
+              * Start searching at index ser->nr_devices. This should result
+              * in a constant time search under expected conditions (devices
+              * are not getting unpreserved).
+              */
+             int index = (ser->nr_devices + i) % ser->max_nr_devices;
+             struct pci_dev_ser *dev_ser = &ser->devices[index];
+
+             if (dev_ser->refcount)
+                     continue;
+
+             pci_info(dev, "Device will be preserved across next Live Update\n");
+             ser->nr_devices++;
+
+             dev_ser->domain = pci_domain_nr(dev->bus);
+             dev_ser->bdf = pci_dev_id(dev);
+             dev_ser->refcount = 1;
+
+             dev->liveupdate_outgoing = dev_ser;

Quoted 8 lines

drivers/pci/liveupdate.c static struct liveupdate_flb pci_liveupdate_flb = { · line 195

Quoted 4 lines

drivers/pci/liveupdate.c +4 -0

+             return 0;
+     }
+
+     return -ENOSPC;

Since it is executing under a mutex, and we already failed
'if (ser->nr_devices == ser->max_nr_devices) check above, will we ever reach here and return -ENOSPC?

Yeah I wouldn't expect to ever reach here.

Quoted 9 lines

Quoted 9 lines

include/linux/kho/abi/pci.h +0 -0

index 5c0e92588c00..5b4c8d9e462c 100644
@@ -23,19 +23,20 @@
  * incrementing the version number in the PCI_LUO_FLB_COMPATIBLE string.
  */
 

Quoted 4 lines

include/linux/kho/abi/pci.h line 26

Quoted 2 lines

include/linux/kho/abi/pci.h +1 -1

-#define PCI_LUO_FLB_COMPATIBLE "pci-v1"
+#define PCI_LUO_FLB_COMPATIBLE "pci-v2"

Just curious, why did we change the version here?

Because a field in struct pci_dev_ser changed.

Quoted 3 lines

It's not like just
previous patch is working enough to perform a live update. As the config is experimental, can't we just keep it PCI-v1 for the whole series?

What is the benefit of keeping "pci-v1"?

I think it makes sense to follow the rule we set which is to update the compatibility string in any commit that changes the ABI.

Quoted 38 lines · David Matlack

Quoted 36 lines · Vipin Sharma

Quoted 20 lines · David Matlack

drivers/pci/liveupdate.c +19 -0

+int pci_liveupdate_preserve(struct pci_dev *dev)
+{
 
+     if (ser->nr_devices == ser->max_nr_devices)
+             return -ENOSPC;
+
+     for (i = 0; i < ser->max_nr_devices; i++) {
+             /*
+              * Start searching at index ser->nr_devices. This should result
+              * in a constant time search under expected conditions (devices
+              * are not getting unpreserved).
+              */
+             int index = (ser->nr_devices + i) % ser->max_nr_devices;
+             struct pci_dev_ser *dev_ser = &ser->devices[index];
+
+             if (dev_ser->refcount)
+                     continue;
+
+             pci_info(dev, "Device will be preserved across next Live Update\n");
+             ser->nr_devices++;

drivers/pci/liveupdate.c static struct liveupdate_flb pci_liveupdate_flb = { · line 195

Quoted 10 lines

drivers/pci/liveupdate.c +10 -0

+
+             dev_ser->domain = pci_domain_nr(dev->bus);
+             dev_ser->bdf = pci_dev_id(dev);
+             dev_ser->refcount = 1;
+
+             dev->liveupdate_outgoing = dev_ser;
+             return 0;
+     }
+
+     return -ENOSPC;

Since it is executing under a mutex, and we already failed
'if (ser->nr_devices == ser->max_nr_devices) check above, will we ever reach here and return -ENOSPC?

Quoted 1 line

Yeah I wouldn't expect to ever reach here.

Will you be removing it or want to keep it just in case scenario?

Quoted 39 lines · Vipin Sharma

Quoted 38 lines · David Matlack

Quoted 36 lines · Vipin Sharma

Quoted 20 lines · David Matlack

drivers/pci/liveupdate.c +19 -0

+int pci_liveupdate_preserve(struct pci_dev *dev)
+{
 
+     if (ser->nr_devices == ser->max_nr_devices)
+             return -ENOSPC;
+
+     for (i = 0; i < ser->max_nr_devices; i++) {
+             /*
+              * Start searching at index ser->nr_devices. This should result
+              * in a constant time search under expected conditions (devices
+              * are not getting unpreserved).
+              */
+             int index = (ser->nr_devices + i) % ser->max_nr_devices;
+             struct pci_dev_ser *dev_ser = &ser->devices[index];
+
+             if (dev_ser->refcount)
+                     continue;
+
+             pci_info(dev, "Device will be preserved across next Live Update\n");
+             ser->nr_devices++;

drivers/pci/liveupdate.c static struct liveupdate_flb pci_liveupdate_flb = { · line 195

Quoted 10 lines

drivers/pci/liveupdate.c +10 -0

+
+             dev_ser->domain = pci_domain_nr(dev->bus);
+             dev_ser->bdf = pci_dev_id(dev);
+             dev_ser->refcount = 1;
+
+             dev->liveupdate_outgoing = dev_ser;
+             return 0;
+     }
+
+     return -ENOSPC;

Since it is executing under a mutex, and we already failed
'if (ser->nr_devices == ser->max_nr_devices) check above, will we ever reach here and return -ENOSPC?

Quoted 3 lines

Quoted 1 line

Yeah I wouldn't expect to ever reach here.

Will you be removing it or want to keep it just in case scenario?

I'm not sure how I would remove it. The code doesn't not compile without a return.

Quoted 42 lines · David Matlack

Quoted 41 lines · Vipin Sharma

Quoted 39 lines · David Matlack

Quoted 28 lines · Vipin Sharma

Quoted 26 lines · David Matlack

drivers/pci/liveupdate.c +25 -0

+int pci_liveupdate_preserve(struct pci_dev *dev)
+{
 
+     if (ser->nr_devices == ser->max_nr_devices)
+             return -ENOSPC;
+
+     for (i = 0; i < ser->max_nr_devices; i++) {
+             /*
+              * Start searching at index ser->nr_devices. This should result
+              * in a constant time search under expected conditions (devices
+              * are not getting unpreserved).
+              */
+             int index = (ser->nr_devices + i) % ser->max_nr_devices;
+             struct pci_dev_ser *dev_ser = &ser->devices[index];
+
+             if (dev_ser->refcount)
+                     continue;
+
+             pci_info(dev, "Device will be preserved across next Live Update\n");
+             ser->nr_devices++;
+
+             dev_ser->domain = pci_domain_nr(dev->bus);
+             dev_ser->bdf = pci_dev_id(dev);
+             dev_ser->refcount = 1;
+
+             dev->liveupdate_outgoing = dev_ser;

Quoted 8 lines

drivers/pci/liveupdate.c static struct liveupdate_flb pci_liveupdate_flb = { · line 195

Quoted 4 lines

drivers/pci/liveupdate.c +4 -0

+             return 0;
+     }
+
+     return -ENOSPC;

Since it is executing under a mutex, and we already failed
'if (ser->nr_devices == ser->max_nr_devices) check above, will we ever reach here and return -ENOSPC?

Yeah I wouldn't expect to ever reach here.

Quoted 4 lines

Quoted 1 line

Will you be removing it or want to keep it just in case scenario?

I'm not sure how I would remove it. The code doesn't not compile without a return.

Oh, I meant, inside loop a break and outside just return 0, that way it won't be a dead code.

During PCI enumeration, the previous kernel might have passed state about devices that were preserved across kexec. The PCI core needs to fetch this state to identify which devices are "incoming" and require special handling.

+++ b/drivers/pci/liveupdate.c

drivers/pci/liveupdate.c +3 -0

+void pci_liveupdate_finish(struct pci_dev *dev)
+{
+	if (!dev->liveupdate_incoming) {

Inherit bus numbers from the previous kernel during a Live Update when one or more PCI devices are being preserved.

During a Live Update, preserved devices must be allowed to continue performing memory transactions so the kernel cannot change the fabric topology, including bus numbers, since that would require disabling and flushing any memory transactions first.

To keep things simple, inherit the secondary and subordinate bus numbers on all bridges if any PCI devices were preserved (i.e. even bridges without any downstream endpoints that were preserved). This avoids accidentally assigning a bridge a new window that overlaps with a preserved device that is downstream of a different bridge.

If a bridge is enumerated with a broken topology or has no bus numbers set during a Live Update, refuse to assign it new bus numbers and refuse to enumerate devices below it. This is a safety measure to prevent topology conflicts.

Require that CONFIG_CARDBUS is not enabled to enable CONFIG_PCI_LIVEUPDATE since inheriting bus numbers on PCI-to-CardBus bridges requires additional work but is not a priority at the moment.

Signed-off-by David Matlack <[email protected]>

---
 .../admin-guide/kernel-parameters.txt         |  6 +++-
 drivers/pci/Kconfig                           |  2 +-
 drivers/pci/liveupdate.c                      | 28
+++++++++++++++++++ drivers/pci/probe.c                           |
21 +++++++++++--- include/linux/pci.h                           |  1 +
 5 files changed, 52 insertions(+), 6 deletions(-)

(quoted diff) +0 -0

b/Documentation/admin-guide/kernel-parameters.txt index cf3807641d89..f412a4b77fb7 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5156,7 +5156,11 @@ Kernel parameters explicitly which ones they are.

Documentation/admin-guide/kernel-parameters.txt +1 -1

 		assign-busses	[X86] Always assign all PCI bus
 				numbers ourselves, overriding
-				whatever the firmware may have done.
+				whatever the firmware may have done.

Ignored

(quoted diff) +1 -0

+				during a Live Update, where the

kernel must

(quoted diff) +1 -0

+				inherit the PCI topology (including

bus numbers)

Documentation/admin-guide/kernel-parameters.txt +2 -0

+				to avoid interrupting ongoing memory
+				transactions of preserved devices.
 		usepirqmask	[X86] Honor the possible IRQ mask

stored in the BIOS $PIR table. This is needed on
some systems with broken BIOSes,
notably diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 08398cbe970c..6ef457ff9d08 100644

drivers/pci/Kconfig +1 -1

@@ -330,7 +330,7 @@ config VGA_ARB_MAX_GPUS
 
 config PCI_LIVEUPDATE
 	bool "PCI Live Update Support (EXPERIMENTAL)"
-	depends on PCI && LIVEUPDATE
+	depends on PCI && LIVEUPDATE && !CARDBUS
 	help
 	  Enable PCI core support for preserving PCI devices across
 Live Update. This, in combination with support in a device's driver,

drivers/pci/liveupdate.c +0 -0

index c0a30d16d9b8..cf8cff134a75 100644
@@ -93,6 +93,19 @@
  * bound to the correct driver. i.e. The PCI core does not protect

against a
* device getting preserved by driver A in the outgoing kernel and
then getting
* bound to driver B in the incoming kernel.
+ *
+ * BDF Stability
+ * =============
+ *
+ * The PCI core guarantees that incoming preserved devices can be identified by
+ * the same bus, device, and function numbers as prior to kexec. To accomplish
+ * this, the PCI core always inherits the secondary and subordinate bus numbers
+ * assigned to bridges during enumeration, rather than assigning new ones (the
+ * PCI core assumes that the previous kernel established a sane topology).
+ *
+ * If a misconfigured or unconfigured bridge is encountered during enumeration
+ * while there are incoming preserved devices, it's secondary and subordinate
+ * bus numbers will be cleared and devices below it will not be enumerated. */

#define pr_fmt(fmt) "PCI: liveupdate: " fmt

drivers/pci/liveupdate.c +2 -0

@@ -354,6 +367,21 @@ void pci_liveupdate_setup_device(struct pci_dev
 *dev) if (!xa)
 		return;
 
+	/*
+	 * During a Live Update, preserved devices are allowed to

continue

drivers/pci/liveupdate.c +1 -0

+	 * performing memory transactions. The kernel must not

change the fabric

drivers/pci/liveupdate.c +1 -0

+	 * topology, including bus numbers, since that would require

disabling

drivers/pci/liveupdate.c +3 -0

+	 * and flushing any memory transactions first.
+	 *
+	 * To keep things simple, inherit the secondary and

subordinate bus

drivers/pci/liveupdate.c +1 -0

+	 * numbers on _all_ bridges if _any_ PCI devices were

preserved (i.e.

drivers/pci/liveupdate.c +1 -0

+	 * even bridges without any downstream endpoints that were

preserved).

drivers/pci/liveupdate.c +1 -0

+	 * This avoids accidentally assigning a bridge a new window

that

drivers/pci/liveupdate.c +1 -0

+	 * overlaps with a preserved device that is downstream of a

different

key = pci_ser_xa_key(pci_domain_nr(dev->bus),
pci_dev_id(dev)); dev_ser = xa_load(xa, key);

drivers/pci/probe.c +8 -0

index 938a28e4a7a0..fa26f4170add 100644
@@ -1374,6 +1374,14 @@ bool pci_ea_fixed_busnrs(struct pci_dev *dev,
 u8 *sec, u8 *sub) return true;
 }
 
+static bool pci_should_assign_new_buses(struct pci_dev *dev)
+{
+	if (dev->liveupdate_inherit_buses)
+		return false;
+
+	return pcibios_assign_all_busses();
+}
+
 /*
  * pci_scan_bridge_extend() - Scan buses behind a bridge
  * @bus: Parent bus the bridge is on

@@ -1401,6 +1409,7 @@ static int pci_scan_bridge_extend(struct

pci_bus *bus, struct pci_dev *dev, int max, unsigned int available_buses, int pass)
{

drivers/pci/probe.c +1 -0

+	const bool assign_new_buses =

pci_should_assign_new_buses(dev); struct pci_bus *child;

drivers/pci/probe.c +2 -4

 	u32 buses;
 	u16 bctl;

@@ -1453,8 +1462,7 @@ static int pci_scan_bridge_extend(struct
 pci_bus *bus, struct pci_dev *dev, goto out;
 	}
 
-	if ((secondary || subordinate) &&
-	    !pcibios_assign_all_busses() && !broken) {
+	if ((secondary || subordinate) && !assign_new_buses &&
 !broken) { unsigned int cmax, buses;
 
 		/*

@@ -1496,8 +1504,7 @@ static int pci_scan_bridge_extend(struct
 pci_bus *bus, struct pci_dev *dev,
 		 * do in the second pass.
 		 */
 		if (!pass) {
-			if (pcibios_assign_all_busses() || broken)
-
+			if (assign_new_buses || broken)
 				/*
 				 * Temporarily disable forwarding of

the
* configuration cycles on all
bridges in @@ -1511,6 +1518,12 @@ static int pci_scan_bridge_extend(struct pci_bus *bus, struct pci_dev *dev, goto out; }

drivers/pci/probe.c +2 -0

+		if (dev->liveupdate_inherit_buses) {
+			pci_err(dev, "Cannot reconfigure bridge

during Live Update!\n");

drivers/pci/probe.c +1 -0

+			pci_err(dev, "Downstream devices will not be

enumerated!\n");

drivers/pci/probe.c +3 -0

+			goto out;
+		}
+
 		/* Clear errors */
 		pci_write_config_word(dev, PCI_STATUS, 0xffff);

include/linux/pci.h +0 -0

index dd6b26ca9462..9a602b322e3c 100644
@@ -511,6 +511,7 @@ struct pci_dev {
 	unsigned int	rom_bar_overlap:1;	/* ROM BAR

disable broken */ unsigned int rom_attr_enabled:1; /*
Display of ROM attribute enabled? */ unsigned int
non_mappable_bars:1; /* BARs can't be mapped to user-space */

include/linux/pci.h +1 -0

+	unsigned int	liveupdate_inherit_buses:1; /* Inherit

bus numbers due to Live Update */ pci_dev_flags_t dev_flags;
atomic_t enable_cnt; /* pci_enable_device has
been called */

Quoted 36 lines · On Thu, 23 Apr 2026 21:23:09 +0000 David Matlack <[email protected]> wrote:

To keep things simple, inherit the secondary and subordinate bus numbers on all bridges if any PCI devices were preserved (i.e. even bridges without any downstream endpoints that were preserved). This avoids accidentally assigning a bridge a new window that overlaps with a preserved device that is downstream of a different bridge.

If a bridge is enumerated with a broken topology or has no bus numbers set during a Live Update, refuse to assign it new bus numbers and refuse to enumerate devices below it. This is a safety measure to prevent topology conflicts.

Require that CONFIG_CARDBUS is not enabled to enable CONFIG_PCI_LIVEUPDATE since inheriting bus numbers on PCI-to-CardBus bridges requires additional work but is not a priority at the moment.

Signed-off-by David Matlack <[email protected]>

(quoted diff) +2 -0

+	/*
+	 * During a Live Update, preserved devices are allowed to

continue

(quoted diff) +1 -0

+	 * performing memory transactions. The kernel must not

change the fabric

(quoted diff) +1 -0

+	 * topology, including bus numbers, since that would require

disabling

drivers/pci/liveupdate.c +3 -0

+	 * and flushing any memory transactions first.
+	 *
+	 * To keep things simple, inherit the secondary and

subordinate bus

(quoted diff) +1 -0

+	 * numbers on _all_ bridges if _any_ PCI devices were

preserved (i.e.

(quoted diff) +1 -0

+	 * even bridges without any downstream endpoints that were

preserved).

(quoted diff) +1 -0

+	 * This avoids accidentally assigning a bridge a new window

that

(quoted diff) +1 -0

+	 * overlaps with a preserved device that is downstream of a

different

drivers/pci/liveupdate.c void pci_liveupdate_setup_device(struct pci_dev *dev) · line 384

Quoted 4 lines

drivers/pci/liveupdate.c +4 -0

+	 * bridge.
+	 */
+	dev->liveupdate_inherit_buses = true;
+

This flag never gets cleared after the incoming kernel boot up, what if the user does a manual rescan via sysfs? i.e.

# echo 1 > /sys/bus/pci/rescan
pcibios_assign_all_busses() will never gets called for this device, and
may hit this
	if (dev->liveupdate_inherit_buses) {
		pci_err(dev, "Cannot reconfigure bridge during
		Live Update!\n");

So, maybe clear it in pci_liveupdate_finish()?

Quoted 45 lines · Jacob Pan

Quoted 36 lines · On Thu, 23 Apr 2026 21:23:09 +0000 David Matlack <[email protected]> wrote:

To keep things simple, inherit the secondary and subordinate bus numbers on all bridges if any PCI devices were preserved (i.e. even bridges without any downstream endpoints that were preserved). This avoids accidentally assigning a bridge a new window that overlaps with a preserved device that is downstream of a different bridge.

If a bridge is enumerated with a broken topology or has no bus numbers set during a Live Update, refuse to assign it new bus numbers and refuse to enumerate devices below it. This is a safety measure to prevent topology conflicts.

Require that CONFIG_CARDBUS is not enabled to enable CONFIG_PCI_LIVEUPDATE since inheriting bus numbers on PCI-to-CardBus bridges requires additional work but is not a priority at the moment.

Signed-off-by David Matlack <[email protected]>

(quoted diff) +2 -0

+	/*
+	 * During a Live Update, preserved devices are allowed to

continue

(quoted diff) +1 -0

+	 * performing memory transactions. The kernel must not

change the fabric

(quoted diff) +1 -0

+	 * topology, including bus numbers, since that would require

disabling

(quoted diff) +3 -0

+	 * and flushing any memory transactions first.
+	 *
+	 * To keep things simple, inherit the secondary and

subordinate bus

(quoted diff) +1 -0

+	 * numbers on _all_ bridges if _any_ PCI devices were

preserved (i.e.

(quoted diff) +1 -0

+	 * even bridges without any downstream endpoints that were

preserved).

(quoted diff) +1 -0

+	 * This avoids accidentally assigning a bridge a new window

that

(quoted diff) +1 -0

+	 * overlaps with a preserved device that is downstream of a

different

Review

Quoted 4 lines

(quoted diff) +4 -0

+	 * bridge.
+	 */
+	dev->liveupdate_inherit_buses = true;
+

This flag never gets cleared after the incoming kernel boot up, what if the user does a manual rescan via sysfs? i.e.

Quoted 8 lines

# echo 1 > /sys/bus/pci/rescan
pcibios_assign_all_busses() will never gets called for this device, and
may hit this
	if (dev->liveupdate_inherit_buses) {
		pci_err(dev, "Cannot reconfigure bridge during
		Live Update!\n");

So, maybe clear it in pci_liveupdate_finish()?

I think we can allo wa rescan to assign new bus numbers once all devices go through pci_liveupdate_finish() by clearing dev->liveupdate_inherit_buses on all devices in pci_flb_finish(). We would need to hold pci_rescan_remove_lock to avoid this racing with such a rescan.

Now that you bring up /sys/bus/pci/rescan... I think we also need to set dev->liveupdate_inherit_buses in the outgoing kernel, to avoid bus numbers changing on outgoing preserved devices. pci_flb_preserve() should take pci_rescan_remove_lock and set dev->liveupdate_inherit_buses on all devices, and pci_flb_unpreserve() should do the opposite.

If we did all then then /sys/bus/pci/rescan can work like normal as long as no devices are preserved (incoming or outgoing). If any devices are preserved then dev->liveupdate_inherit_buses gets set to prevent bus numbers from changing during a possible rescan.

Quoted 70 lines · David Matlack

Quoted 47 lines · Jacob Pan

Quoted 38 lines · On Thu, 23 Apr 2026 21:23:09 +0000 David Matlack <[email protected]> wrote:

To keep things simple, inherit the secondary and subordinate bus numbers on all bridges if any PCI devices were preserved (i.e. even bridges without any downstream endpoints that were preserved). This avoids accidentally assigning a bridge a new window that overlaps with a preserved device that is downstream of a different bridge.

If a bridge is enumerated with a broken topology or has no bus numbers set during a Live Update, refuse to assign it new bus numbers and refuse to enumerate devices below it. This is a safety measure to prevent topology conflicts.

Require that CONFIG_CARDBUS is not enabled to enable CONFIG_PCI_LIVEUPDATE since inheriting bus numbers on PCI-to-CardBus bridges requires additional work but is not a priority at the moment.

Signed-off-by David Matlack <[email protected]>

(quoted diff) +2 -0

+	/*
+	 * During a Live Update, preserved devices are allowed

to continue

(quoted diff) +1 -0

+	 * performing memory transactions. The kernel must not

change the fabric

(quoted diff) +1 -0

+	 * topology, including bus numbers, since that would

require disabling

(quoted diff) +3 -0

+	 * and flushing any memory transactions first.
+	 *
+	 * To keep things simple, inherit the secondary and

subordinate bus

(quoted diff) +1 -0

+	 * numbers on _all_ bridges if _any_ PCI devices were

preserved (i.e.

(quoted diff) +1 -0

+	 * even bridges without any downstream endpoints that

were preserved).

(quoted diff) +1 -0

+	 * This avoids accidentally assigning a bridge a new

window that

(quoted diff) +1 -0

+	 * overlaps with a preserved device that is downstream

of a different

drivers/pci/liveupdate.c line 136

Quoted 4 lines

drivers/pci/liveupdate.c +4 -0

+	 * bridge.
+	 */
+	dev->liveupdate_inherit_buses = true;
+  

This flag never gets cleared after the incoming kernel boot up, what if the user does a manual rescan via sysfs? i.e.

Quoted 8 lines

# echo 1 > /sys/bus/pci/rescan
pcibios_assign_all_busses() will never gets called for this
device, and may hit this
	if (dev->liveupdate_inherit_buses) {
		pci_err(dev, "Cannot reconfigure bridge during
		Live Update!\n");

So, maybe clear it in pci_liveupdate_finish()?

I think we can allo wa rescan to assign new bus numbers once all devices go through pci_liveupdate_finish() by clearing dev->liveupdate_inherit_buses on all devices in pci_flb_finish(). We would need to hold pci_rescan_remove_lock to avoid this racing with such a rescan.

Now that you bring up /sys/bus/pci/rescan... I think we also need to set dev->liveupdate_inherit_buses in the outgoing kernel, to avoid bus numbers changing on outgoing preserved devices. pci_flb_preserve() should take pci_rescan_remove_lock and set dev->liveupdate_inherit_buses on all devices, and pci_flb_unpreserve() should do the opposite.

Quoted 4 lines

If we did all then then /sys/bus/pci/rescan can work like normal as long as no devices are preserved (incoming or outgoing). If any devices are preserved then dev->liveupdate_inherit_buses gets set to prevent bus numbers from changing during a possible rescan.

Something like this? This is a diff applied on top of this commit.

drivers/pci/liveupdate.c +2 -0

index fead478e8a04..b1b0a5b1a5df 100644
@@ -120,6 +120,20 @@
 
 #include "pci.h"
 
+/*
+ * During a Live Update, preserved devices are allowed to continue

performing
+ * memory transactions. The kernel must not change the fabric topology,
+ * including bus numbers, since that would require disabling and flushing any
+ * memory transactions first.
+ *
+ * To keep things simple, inherit the secondary and subordinate bus numbers on
+ * _all_ bridges if _any_ PCI devices are preserved (i.e. even bridges without
+ * any downstream endpoints that were preserved). This avoids accidentally
+ * assigning a bridge a new window that overlaps with a preserved device that is
+ * downstream of a different bridge.
+ */

drivers/pci/liveupdate.c +14 -0

+static atomic_t inherit_buses;
+
 struct pci_flb_outgoing {
 	/* The pci_ser struct to be passed to the next kernel */
 	struct pci_ser *ser;

@@ -141,6 +155,29 @@ static unsigned long pci_ser_xa_key(unsigned
 long domain, unsigned long bdf) return domain << 16 | bdf;
 }
 
+bool pci_liveupdate_inherit_buses(void)
+{
+	return atomic_read(&inherit_buses);
+}
+
+static void pci_set_liveupdate_inherit_buses(bool enable)
+{
+	/* Ensure updates to inherit_buses do not race with rescans
 */
+	pci_lock_rescan_remove();
+
+	/*
+	 * Increment/decrement instead of setting directly to

true/false so that

drivers/pci/liveupdate.c +1 -0

+	 * pci_liveupdate_inherit_buses() returns true if any device

is outgoing

drivers/pci/liveupdate.c +18 -2

+	 * preserved or incoming preserved.
+	 */
+	if (enable)
+		atomic_inc(&inherit_buses);
+	else
+		atomic_dec(&inherit_buses);
+
+	pci_unlock_rescan_remove();
+}
+
 static int pci_flb_preserve(struct liveupdate_flb_op_args *args)
 {
 	struct pci_flb_outgoing *outgoing;

@@ -180,6 +217,8 @@ static int pci_flb_preserve(struct
 liveupdate_flb_op_args *args) 
 	args->obj = outgoing;
 	args->data = virt_to_phys(outgoing->ser);
+
+	pci_set_liveupdate_inherit_buses(true);
 	return 0;
 }
 

@@ -187,6 +226,8 @@ static void pci_flb_unpreserve(struct
 liveupdate_flb_op_args *args) {
 	struct pci_flb_outgoing *outgoing = args->obj;
 
+	pci_set_liveupdate_inherit_buses(false);
+
 	pr_debug("Unpreserving struct pci_ser\n");
 	WARN_ON_ONCE(outgoing->ser->nr_devices);
 	kho_unpreserve_free(outgoing->ser);

@@ -223,6 +264,8 @@ static int pci_flb_retrieve(struct
 liveupdate_flb_op_args *args) }
 
 	args->obj = incoming;
+
+	pci_set_liveupdate_inherit_buses(true);
 	return 0;
 }
 

@@ -230,6 +273,8 @@ static void pci_flb_finish(struct
 liveupdate_flb_op_args *args) {
 	struct pci_flb_incoming *incoming = args->obj;
 
+	pci_set_liveupdate_inherit_buses(false);
+
 	xa_destroy(&incoming->xa);
 	kho_restore_free(incoming->ser);
 	kfree(incoming);

@@ -385,21 +430,6 @@ void pci_liveupdate_setup_device(struct pci_dev
 *dev) if (!xa)
 		return;
 
-	/*
-	 * During a Live Update, preserved devices are allowed to

continue

drivers/pci/liveupdate.c +0 -1

-	 * performing memory transactions. The kernel must not

change the fabric

drivers/pci/liveupdate.c +0 -1

-	 * topology, including bus numbers, since that would require

disabling

drivers/pci/liveupdate.c +0 -3

-	 * and flushing any memory transactions first.
-	 *
-	 * To keep things simple, inherit the secondary and

subordinate bus

drivers/pci/liveupdate.c +0 -1

-	 * numbers on _all_ bridges if _any_ PCI devices were

preserved (i.e.

drivers/pci/liveupdate.c +0 -1

-	 * even bridges without any downstream endpoints that were

preserved).

drivers/pci/liveupdate.c +0 -1

-	 * This avoids accidentally assigning a bridge a new window

that

drivers/pci/liveupdate.c +0 -1

-	 * overlaps with a preserved device that is downstream of a

different

drivers/pci/liveupdate.c +0 -4

-	 * bridge.
-	 */
-	dev->liveupdate_inherit_buses = true;
-
 	key = pci_ser_xa_key(pci_domain_nr(dev->bus),

pci_dev_id(dev)); dev_ser = xa_load(xa, key);

drivers/pci/pci.h +0 -0

index 09bab39738d7..abd8379b99cf 100644
@@ -1442,6 +1442,7 @@ static inline int pci_msix_write_tph_tag(struct

pci_dev *pdev, unsigned int inde #ifdef CONFIG_PCI_LIVEUPDATE
void pci_liveupdate_setup_device(struct pci_dev *dev); void pci_liveupdate_cleanup_device(struct pci_dev *dev);

drivers/pci/pci.h +1 -0

+bool pci_liveupdate_inherit_buses(void);
 #else
 static inline void pci_liveupdate_setup_device(struct pci_dev *dev)
 {

@@ -1450,6 +1451,11 @@ static inline void

pci_liveupdate_setup_device(struct pci_dev *dev) static inline void pci_liveupdate_cleanup_device(struct pci_dev *dev) {
}

drivers/pci/pci.h +5 -0

+
+static inline bool pci_liveupdate_inherit_buses(void)
+{
+	return false;
+}
 #endif
 
 #endif /* DRIVERS_PCI_H */

drivers/pci/probe.c +2 -2

index fa26f4170add..f94fa1fc76cc 100644
@@ -1374,9 +1374,9 @@ bool pci_ea_fixed_busnrs(struct pci_dev *dev,
 u8 *sec, u8 *sub) return true;
 }
 
-static bool pci_should_assign_new_buses(struct pci_dev *dev)
+static bool pci_should_assign_new_buses(void)
 {
-	if (dev->liveupdate_inherit_buses)
+	if (pci_liveupdate_inherit_buses())
 		return false;
 
 	return pcibios_assign_all_busses();

@@ -1409,7 +1409,7 @@ static int pci_scan_bridge_extend(struct

pci_bus *bus, struct pci_dev *dev, int max, unsigned int available_buses, int pass)
{

drivers/pci/probe.c +0 -1

-	const bool assign_new_buses =

pci_should_assign_new_buses(dev);

drivers/pci/probe.c +2 -1

+	const bool assign_new_buses = pci_should_assign_new_buses();
 	struct pci_bus *child;
 	u32 buses;
 	u16 bctl;

@@ -1518,7 +1518,7 @@ static int pci_scan_bridge_extend(struct
 pci_bus *bus, struct pci_dev *dev, goto out;
 		}
 
-		if (dev->liveupdate_inherit_buses) {
+		if (pci_liveupdate_inherit_buses()) {
 			pci_err(dev, "Cannot reconfigure bridge

during Live Update!\n"); pci_err(dev, "Downstream devices will not be enumerated!\n"); goto out;

include/linux/pci.h +0 -0

index 9a602b322e3c..dd6b26ca9462 100644
@@ -511,7 +511,6 @@ struct pci_dev {
 	unsigned int	rom_bar_overlap:1;	/* ROM BAR

disable broken */ unsigned int rom_attr_enabled:1; /*
Display of ROM attribute enabled? */ unsigned int
non_mappable_bars:1; /* BARs can't be mapped to user-space */

include/linux/pci.h +0 -1

-	unsigned int	liveupdate_inherit_buses:1; /* Inherit

bus numbers due to Live Update */ pci_dev_flags_t dev_flags;
atomic_t enable_cnt; /* pci_enable_device has
been called */