PCI: hv: Support host bridge probing on ARM64
Subsystems (5)
v5 cover PCI: hv: Support host bridge probing on ARM64
Hi Bjorn, Arnd and Marc,
This is the v5 for the preparation of virtual PCI support on Hyper-V ARM64, Previous versions:
v1: https://lore.kernel.org/lkml/[email protected]/
v2: https://lore.kernel.org/lkml/[email protected]/
v3: https://lore.kernel.org/lkml/[email protected]/
v4: https://lore.kernel.org/lkml/[email protected]/
Changes since last version:
* Rebase to 5.14-rc2
* Wording changes (Capitalization and more explanation on why) as
suggested by Bjorn.
* Split the patch #3 in previous into two patches as suggested by
Bjorn.
The basic problem we need to resolve is that ARM64 is an arch with PCI_DOMAINS_GENERIC=y, so the bus sysdata is pci_config_window. However, Hyper-V PCI provides a paravirtualized PCI interface, so there is no actual pci_config_window for a PCI host bridge, so no information can be retrieve from the pci_config_window of a Hyper-V virtual PCI bus. Also there is no corresponding ACPI device for the Hyper-V PCI root bridge, which introduces a special case when trying to find the ACPI device from the sysdata (see patch #3).
With this patchset, we could enable the virtual PCI on Hyper-V ARM64 guest with other code under development.
Comments and suggestions are welcome.
Regards,
Boqun
Arnd Bergmann (1):
PCI: hv: Generify PCI probing
Boqun Feng (7):
PCI: Introduce domain_nr in pci_host_bridge
PCI: Support populating MSI domains of root buses via bridges
arm64: PCI: Restructure pcibios_root_bridge_prepare()
arm64: PCI: Support root bridge preparation for Hyper-V
PCI: hv: Set ->domain_nr of pci_host_bridge at probing time
PCI: hv: Set up MSI domain at bridge probing time
PCI: hv: Turn on the host bridge probing on ARM64
arch/arm64/kernel/pci.c | 29 +++++++--- drivers/pci/controller/pci-hyperv.c | 86 +++++++++++++++++------------ drivers/pci/probe.c | 12 +++- include/linux/pci.h | 10 ++++ 4 files changed, 92 insertions(+), 45 deletions(-)
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
RFC v5 1/8 PCI: Introduce domain_nr in pci_host_bridge
Currently we retrieve the PCI domain number of the host bridge from the bus sysdata (or pci_config_window if PCI_DOMAINS_GENERIC=y). Actually we have the information at PCI host bridge probing time, and it makes sense that we store it into pci_host_bridge. One benefit of doing so is the requirement for supporting PCI on Hyper-V for ARM64, because the host bridge of Hyper-V doesn't have pci_config_window, whereas ARM64 is a PCI_DOMAINS_GENERIC=y arch, so we cannot retrieve the PCI domain number from pci_config_window on ARM64 Hyper-V guest.
As the preparation for ARM64 Hyper-V PCI support, we introduce the domain_nr in pci_host_bridge and a sentinel value to allow drivers to set domain numbers properly at probing time. Currently CONFIG_PCI_DOMAINS_GENERIC=y archs are only users of this newly-introduced field.
--- drivers/pci/probe.c | 6 +++++- include/linux/pci.h | 10 ++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-)
drivers/pci/probe.c +5 -1
@@ -594,6 +594,7 @@ static void pci_init_host_bridge(struct pci_host_bridge *bridge) bridge->native_pme =1 ; bridge->native_ltr =1 ; bridge->native_dpc =1 ; + bridge->domain_nr =PCI_DOMAIN_NR_NOT_SET ;device_initialize (&bridge->dev); }
@@ -898,7 +899,10 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge) bus->ops = bridge->ops; bus->number = bus->busn_res.start = bridge->busnr;#ifdef CONFIG_PCI_DOMAINS_GENERIC - bus->domain_nr =pci_bus_find_domain_nr (bus, parent); +if (bridge->domain_nr ==PCI_DOMAIN_NR_NOT_SET ) + bus->domain_nr =pci_bus_find_domain_nr (bus, parent); +else + bus->domain_nr = bridge->domain_nr;#endif b =pci_find_bus (pci_domain_nr (bus), bridge->busnr);
include/linux/pci.h +10 -0
@@ -526,6 +526,15 @@ static inline int pci_channel_offline(struct pci_dev *pdev)return (pdev->error_state != pci_channel_io_normal); } +/* +* Currently in ACPI spec, for each PCI host bridge, PCI Segment Group number is +* limited to a 16-bit value, therefore (int)-1 is not a valid PCI domain number, +* and can be used as a sentinel value indicating >domain_nr is not set by the +* driver (and CONFIG_PCI_DOMAINS_GENERIC=y archs will set it with +* pci_bus_find_domain_nr()). +*/ +#define PCI_DOMAIN_NR_NOT_SET (-1) +struct pci_host_bridge {struct device dev;struct pci_bus *bus;/* Root bus */
@@ -533,6 +542,7 @@ struct pci_host_bridge {struct pci_ops *child_ops;void *sysdata;int busnr; +int domain_nr;struct list_head windows;/* resource_entry */ struct list_head dma_ranges;/* dma ranges resource list */ u8 (*swizzle_irq)(struct pci_dev *,u8 *);/* Platform IRQ swizzler */
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [RFC v5 1/8] PCI: Introduce domain_nr in pci_host_bridge
Quoted 9 lines · Boqun Feng
Currently we retrieve the PCI domain number of the host bridge from the bus sysdata (or
pci_config_windowif PCI_DOMAINS_GENERIC=y). Actually we have the information at PCI host bridge probing time, and it makes sense that we store it intopci_host_bridge. One benefit of doing so is the requirement for supporting PCI on Hyper-V for ARM64, because the host bridge of Hyper-V doesn't havepci_config_window, whereas ARM64 is a PCI_DOMAINS_GENERIC=y arch, so we cannot retrieve the PCI domain number frompci_config_windowon ARM64 Hyper-V guest.
Quoted 7 lines
As the preparation for ARM64 Hyper-V PCI support, we introduce the
domain_nrinpci_host_bridgeand a sentinel value to allow drivers to set domain numbers properly at probing time. CurrentlyCONFIG_PCI_DOMAINS_GENERIC=yarchs are only users of this newly-introduced field.Signed-off-by Boqun Feng <[email protected]>
Once all the issues are ironed out, Lorenzo should probably merge this since it's primarily Hyper-V stuff, but I'm OK with this part:
But fix the comment issue below.
Quoted 34 lines
--- drivers/pci/probe.c | 6 +++++- include/linux/pci.h | 10 ++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-)drivers/pci/probe.c +5
-1@@ -594,6 +594,7 @@ static void pci_init_host_bridge(struct pci_host_bridge *bridge) bridge->native_pme =1 ; bridge->native_ltr =1 ; bridge->native_dpc =1 ; + bridge->domain_nr =PCI_DOMAIN_NR_NOT_SET ;device_initialize (&bridge->dev); }@@ -898,7 +899,10 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge) bus->ops = bridge->ops; bus->number = bus->busn_res.start = bridge->busnr;#ifdef CONFIG_PCI_DOMAINS_GENERIC - bus->domain_nr =pci_bus_find_domain_nr (bus, parent); +if (bridge->domain_nr ==PCI_DOMAIN_NR_NOT_SET ) + bus->domain_nr =pci_bus_find_domain_nr (bus, parent); +else + bus->domain_nr = bridge->domain_nr;#endif b =pci_find_bus (pci_domain_nr (bus), bridge->busnr);include/linux/pci.h +0
-0
include/linux/pci.h static inline int pci_channel_offline(struct pci_dev *pdev) · line 535
Quoted 10 lines
include/linux/pci.h +7
-0@@ -526,6 +526,15 @@ static inline int pci_channel_offline(struct pci_dev *pdev)return (pdev->error_state != pci_channel_io_normal); } +/* +* Currently in ACPI spec, for each PCI host bridge, PCI Segment Group number is +* limited to a 16-bit value, therefore (int)-1 is not a valid PCI domain number, +* and can be used as a sentinel value indicating >domain_nr is not set by the +* driver (and CONFIG_PCI_DOMAINS_GENERIC=y archs will set it with +* pci_bus_find_domain_nr()). +*/
Fix comment indentation (follow style of other comments in this file).
s/>domain_nr/->domain_nr/
Quoted 16 lines
include/linux/pci.h +3
-0+#define PCI_DOMAIN_NR_NOT_SET (-1) +struct pci_host_bridge {struct device dev;struct pci_bus *bus;/* Root bus */ @@ -533,6 +542,7 @@ struct pci_host_bridge {struct pci_ops *child_ops;void *sysdata;int busnr; +int domain_nr;struct list_head windows;/* resource_entry */ struct list_head dma_ranges;/* dma ranges resource list */ u8 (*swizzle_irq)(struct pci_dev *,u8 *);/* Platform IRQ swizzler */ Signature · 1 line
2.30.2
Re: [RFC v5 1/8] PCI: Introduce domain_nr in pci_host_bridge
Quoted 16 lines · Bjorn Helgaas
Quoted 15 lines · Boqun Feng
Currently we retrieve the PCI domain number of the host bridge from the bus sysdata (or
pci_config_windowif PCI_DOMAINS_GENERIC=y). Actually we have the information at PCI host bridge probing time, and it makes sense that we store it intopci_host_bridge. One benefit of doing so is the requirement for supporting PCI on Hyper-V for ARM64, because the host bridge of Hyper-V doesn't havepci_config_window, whereas ARM64 is a PCI_DOMAINS_GENERIC=y arch, so we cannot retrieve the PCI domain number frompci_config_windowon ARM64 Hyper-V guest.As the preparation for ARM64 Hyper-V PCI support, we introduce the
domain_nrinpci_host_bridgeand a sentinel value to allow drivers to set domain numbers properly at probing time. CurrentlyCONFIG_PCI_DOMAINS_GENERIC=yarchs are only users of this newly-introduced field.
Quoted 7 lines
Quoted 1 line
Signed-off-by Boqun Feng <[email protected]>Once all the issues are ironed out, Lorenzo should probably merge this since it's primarily Hyper-V stuff, but I'm OK with this part:
Acked-by Bjorn Helgaas <[email protected]>
Thanks!
Quoted 2 lines
But fix the comment issue below.
Just send a v5.1 for this patch with the comment fixed and your Acked-by.
Regards,
Boqun
Quoted 65 lines
Quoted 34 lines
--- drivers/pci/probe.c | 6 +++++- include/linux/pci.h | 10 ++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-)drivers/pci/probe.c +5
-1@@ -594,6 +594,7 @@ static void pci_init_host_bridge(struct pci_host_bridge *bridge) bridge->native_pme =1 ; bridge->native_ltr =1 ; bridge->native_dpc =1 ; + bridge->domain_nr =PCI_DOMAIN_NR_NOT_SET ;device_initialize (&bridge->dev); }@@ -898,7 +899,10 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge) bus->ops = bridge->ops; bus->number = bus->busn_res.start = bridge->busnr;#ifdef CONFIG_PCI_DOMAINS_GENERIC - bus->domain_nr =pci_bus_find_domain_nr (bus, parent); +if (bridge->domain_nr ==PCI_DOMAIN_NR_NOT_SET ) + bus->domain_nr =pci_bus_find_domain_nr (bus, parent); +else + bus->domain_nr = bridge->domain_nr;#endif b =pci_find_bus (pci_domain_nr (bus), bridge->busnr);include/linux/pci.h +0
-0include/linux/pci.h static inline int pci_channel_offline(struct pci_dev *pdev) · line 535
Quoted 10 lines
include/linux/pci.h +7
-0@@ -526,6 +526,15 @@ static inline int pci_channel_offline(struct pci_dev *pdev)return (pdev->error_state != pci_channel_io_normal); } +/* +* Currently in ACPI spec, for each PCI host bridge, PCI Segment Group number is +* limited to a 16-bit value, therefore (int)-1 is not a valid PCI domain number, +* and can be used as a sentinel value indicating >domain_nr is not set by the +* driver (and CONFIG_PCI_DOMAINS_GENERIC=y archs will set it with +* pci_bus_find_domain_nr()). +*/ Fix comment indentation (follow style of other comments in this file).
s/>domain_nr/->domain_nr/
Quoted 16 lines
include/linux/pci.h +3
-0+#define PCI_DOMAIN_NR_NOT_SET (-1) +struct pci_host_bridge {struct device dev;struct pci_bus *bus;/* Root bus */ @@ -533,6 +542,7 @@ struct pci_host_bridge {struct pci_ops *child_ops;void *sysdata;int busnr; +int domain_nr;struct list_head windows;/* resource_entry */ struct list_head dma_ranges;/* dma ranges resource list */ u8 (*swizzle_irq)(struct pci_dev *,u8 *);/* Platform IRQ swizzler */ Signature · 1 line
2.30.2
RFC v5 2/8 PCI: Support populating MSI domains of root buses via bridges
In reply to Boqun FengCurrently, at probing time, the MSI domains of root buses are populated if either the information of MSI domain is available from firmware (DT or ACPI), or arch-specific sysdata is used to pass the fwnode of the MSI domain. These two conditions don't cover all, e.g. Hyper-V virtual PCI on ARM64, which doesn't have the MSI information in the firmware and couldn't use arch-specific sysdata because running on an architecture with PCI_DOMAINS_GENERIC=y.
To support populating MSI domains of the root buses at the probing when neither of the above condition is true, the ->msi_domain of the corresponding bridge device is used: in pci_host_bridge_msi_domain(), which should return the MSI domain of the root bus, the ->msi_domain of the corresponding bridge is fetched first as a potential value of the MSI domain of the root bus.
In order to use the approach to populate MSI domains, the driver needs to dev_set_msi_domain() on the bridge before calling pci_register_host_bridge(), and makes sure GENERIC_MSI_IRQ_DOMAIN=y.
Another advantage of this new approach is providing an arch-independent way to populate MSI domains, which allows sharing the driver code as much as possible between architectures.
--- drivers/pci/probe.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/pci/probe.c +5 -1
@@ -829,11 +829,15 @@ static struct irq_domain *pci_host_bridge_msi_domain(struct pci_bus *bus) {struct irq_domain *d; +/* If the host bridge driver sets a MSI domain of the bridge, use it */ + d =dev_get_msi_domain (bus->bridge); +/* * Any firmware interface that can resolve the msi_domain * should be called from here. */ - d =pci_host_bridge_of_msi_domain (bus); +if (!d) + d =pci_host_bridge_of_msi_domain (bus);if (!d) d =pci_host_bridge_acpi_msi_domain (bus);
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
RFC v5 3/8 arm64: PCI: Restructure pcibios_root_bridge_prepare()
In reply to Boqun FengRestructure the pcibios_root_bridge_prepare() as the preparation for supporting cases when no real ACPI device is related to the PCI host bridge.
No functional change.
--- arch/arm64/kernel/pci.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
arch/arm64/kernel/pci.c +12 -7
@@ -82,14 +82,19 @@ int acpi_pci_bus_find_domain_nr(struct pci_bus *bus)int pcibios_root_bridge_prepare (struct pci_host_bridge *bridge) { -if (!acpi_disabled) { -struct pci_config_window *cfg = bridge->bus->sysdata; -struct acpi_device *adev =to_acpi_device (cfg->parent); -struct device *bus_dev = &bridge->bus->dev; +struct pci_config_window *cfg; +struct acpi_device *adev; +struct device *bus_dev; -ACPI_COMPANION_SET (&bridge->dev, adev); -set_dev_node (bus_dev,acpi_get_node (acpi_device_handle (adev))); - } +if (acpi_disabled) +return 0 ; + + cfg = bridge->bus->sysdata; + adev =to_acpi_device (cfg->parent); + bus_dev = &bridge->bus->dev; + +ACPI_COMPANION_SET (&bridge->dev, adev); +set_dev_node (bus_dev,acpi_get_node (acpi_device_handle (adev)));return 0 ; }
Signature · 1 line
2.30.2
RFC v5 4/8 arm64: PCI: Support root bridge preparation for Hyper-V
In reply to Boqun FengCurrently at root bridge preparation, the corresponding ACPI device will be set as the companion, however for a Hyper-V virtual PCI root bridge, there is no corresponding ACPI device, because a Hyper-V virtual PCI root bridge is discovered via VMBus rather than ACPI table. In order to support this, we need to make pcibios_root_bridge_prepare() work with cfg->parent being NULL.
Use a NULL pointer as the ACPI device if there is no corresponding ACPI device, and this is fine because: 1) ACPI_COMPANION_SET() can work with the second parameter being NULL, 2) semantically, if a NULL pointer is set via ACPI_COMPANION_SET(), ACPI_COMPANION() (the read API for this field) will return NULL, and since ACPI_COMPANION() may return NULL, so users must have handled the cases where it returns NULL, and 3) since there is no corresponding ACPI device, it would be wrong to use any other value here.
--- arch/arm64/kernel/pci.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
arch/arm64/kernel/pci.c +11 -1
@@ -90,7 +90,17 @@ int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)return 0 ; cfg = bridge->bus->sysdata; - adev =to_acpi_device (cfg->parent); + +/* +* On Hyper-V there is no corresponding ACPI device for a root bridge, +* therefore ->parent is set as NULL by the driver. And set 'adev' as +* NULL in this case because there is no proper ACPI device. +*/ +if (!cfg->parent) + adev =NULL ; +else + adev =to_acpi_device (cfg->parent); + bus_dev = &bridge->bus->dev;ACPI_COMPANION_SET (&bridge->dev, adev);
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
RFC v5 5/8 PCI: hv: Generify PCI probing
In reply to Boqun FengFrom: Arnd Bergmann <[email protected]>
In order to support ARM64 Hyper-V PCI, we need to set up the bridge at probing time because ARM64 is a PCI_DOMAIN_GENERIC=y arch and we don't have pci_config_window (ARM64 sysdata) for a PCI root bus on Hyper-V, so it's impossible to retrieve the information (e.g. PCI domains, MSI domains) from bus sysdata on ARM64 after creation.
Originally in create_root_hv_pci_bus(), pci_create_root_bus() is used to create the root bus and the corresponding bridge based on x86 sysdata. Now we create a bridge first and then call pci_scan_root_bus_bridge(), which allows us to do the necessary set-ups for the bridge.
--- drivers/pci/controller/pci-hyperv.c | 57 +++++++++++++++-------------- 1 file changed, 30 insertions(+), 27 deletions(-)
drivers/pci/controller/pci-hyperv.c +30 -27
@@ -449,6 +449,7 @@ enum hv_pcibus_state {struct hv_pcibus_device {struct pci_sysdata sysdata; +struct pci_host_bridge *bridge;/* Protocol version negotiated with the host */ enum pci_protocol_version_t protocol_version;enum hv_pcibus_state state;
@@ -464,8 +465,6 @@ struct hv_pcibus_device {spinlock_t device_list_lock;/* Protect lists below */ void __iomem *cfg_addr; -struct list_head resources_for_children; -struct list_head children;struct list_head dr_list;
@@ -1797,7 +1796,7 @@ static void hv_pci_assign_slots(struct hv_pcibus_device *hbus) slot_nr =PCI_SLOT (wslot_to_devfn (hpdev->desc.win_slot.slot));snprintf (name,SLOT_NAME_SIZE ,"%u" , hpdev->desc.ser); - hpdev->pci_slot =pci_create_slot (hbus->pci_bus, slot_nr, + hpdev->pci_slot =pci_create_slot (hbus->bridge->bus, slot_nr, name,NULL );if (IS_ERR (hpdev->pci_slot)) {pr_warn ("pci_create slot %s failed\n" , name);
@@ -1827,7 +1826,7 @@ static void hv_pci_remove_slots(struct hv_pcibus_device *hbus)static void hv_pci_assign_numa_node (struct hv_pcibus_device *hbus) {struct pci_dev *dev; -struct pci_bus *bus = hbus->pci_bus; +struct pci_bus *bus = hbus->bridge->bus;struct hv_pci_dev *hv_dev;list_for_each_entry (dev, &bus->devices, bus_list) {
@@ -1850,21 +1849,22 @@ static void hv_pci_assign_numa_node(struct hv_pcibus_device *hbus)*/ static int create_root_hv_pci_bus (struct hv_pcibus_device *hbus) { -/* Register the device */ - hbus->pci_bus =pci_create_root_bus (&hbus->hdev->device, -0 ,/* bus number is always zero */ - &hv_pcifront_ops, - &hbus->sysdata, - &hbus->resources_for_children); -if (!hbus->pci_bus) -return -ENODEV ; +int error; +struct pci_host_bridge *bridge = hbus->bridge; + + bridge->dev.parent = &hbus->hdev->device; + bridge->sysdata = &hbus->sysdata; + bridge->ops = &hv_pcifront_ops; + + error =pci_scan_root_bus_bridge (bridge); +if (error) +return error;pci_lock_rescan_remove (); -pci_scan_child_bus (hbus->pci_bus);hv_pci_assign_numa_node (hbus); -pci_bus_assign_resources (hbus->pci_bus); +pci_bus_assign_resources (bridge->bus);hv_pci_assign_slots (hbus); -pci_bus_add_devices (hbus->pci_bus); +pci_bus_add_devices (bridge->bus);pci_unlock_rescan_remove (); hbus->state = hv_pcibus_installed;return 0 ;
@@ -2127,7 +2127,7 @@ static void pci_devices_present_work(struct work_struct *work)* because there may have been changes. */ pci_lock_rescan_remove (); -pci_scan_child_bus (hbus->pci_bus); +pci_scan_child_bus (hbus->bridge->bus);hv_pci_assign_numa_node (hbus);hv_pci_assign_slots (hbus);pci_unlock_rescan_remove ();
@@ -2295,8 +2295,8 @@ static void hv_eject_device_work(struct work_struct *work)/* * Ejection can come before or after the PCI bus has been set up, so * attempt to find it and tear down the bus state, if it exists. This -* must be done without constructs like pci_domain_nr(hbus->pci_bus) -* because hbus->pci_bus may not exist yet. +* must be done without constructs like pci_domain_nr(hbus->bridge->bus) +* because hbus->bridge->bus may not exist yet. */ wslot =wslot_to_devfn (hpdev->desc.win_slot.slot); pdev =pci_get_domain_bus_and_slot (hbus->sysdata.domain,0 , wslot);
@@ -2662,8 +2662,7 @@ static int hv_pci_allocate_bridge_windows(struct hv_pcibus_device *hbus)/* Modify this resource to become a bridge window. */ hbus->low_mmio_res->flags |=IORESOURCE_WINDOW ; hbus->low_mmio_res->flags &= ~IORESOURCE_BUSY ; -pci_add_resource (&hbus->resources_for_children, - hbus->low_mmio_res); +pci_add_resource (&hbus->bridge->windows, hbus->low_mmio_res); }if (hbus->high_mmio_space) {
@@ -2682,8 +2681,7 @@ static int hv_pci_allocate_bridge_windows(struct hv_pcibus_device *hbus)/* Modify this resource to become a bridge window. */ hbus->high_mmio_res->flags |=IORESOURCE_WINDOW ; hbus->high_mmio_res->flags &= ~IORESOURCE_BUSY ; -pci_add_resource (&hbus->resources_for_children, - hbus->high_mmio_res); +pci_add_resource (&hbus->bridge->windows, hbus->high_mmio_res); }return 0 ;
@@ -3002,6 +3000,7 @@ static void hv_put_dom_num(u16 dom)static int hv_pci_probe (struct hv_device *hdev,const struct hv_vmbus_device_id *dev_id) { +struct pci_host_bridge *bridge;struct hv_pcibus_device *hbus;u16 dom_req, dom;char *name;
@@ -3014,6 +3013,10 @@ static int hv_pci_probe(struct hv_device *hdev,*/ BUILD_BUG_ON (sizeof (*hbus) >HV_HYP_PAGE_SIZE ); + bridge =devm_pci_alloc_host_bridge (&hdev->device,0 ); +if (!bridge) +return -ENOMEM ; +/* * With the recent 59bb47985c1d ("mm, sl[aou]b: guarantee natural * alignment for kmalloc(power-of-two)"), kzalloc() is able to allocate
@@ -3035,6 +3038,8 @@ static int hv_pci_probe(struct hv_device *hdev, hbus =kzalloc (HV_HYP_PAGE_SIZE ,GFP_KERNEL );if (!hbus)return -ENOMEM ; + + hbus->bridge = bridge; hbus->state = hv_pcibus_init; hbus->wslot_res_allocated =-1 ;
@@ -3071,7 +3076,6 @@ static int hv_pci_probe(struct hv_device *hdev, hbus->hdev = hdev;INIT_LIST_HEAD (&hbus->children);INIT_LIST_HEAD (&hbus->dr_list); -INIT_LIST_HEAD (&hbus->resources_for_children);spin_lock_init (&hbus->config_lock);spin_lock_init (&hbus->device_list_lock);spin_lock_init (&hbus->retarget_msi_interrupt_lock);
@@ -3295,9 +3299,9 @@ static int hv_pci_remove(struct hv_device *hdev)/* Remove the bus from PCI's point of view. */ pci_lock_rescan_remove (); -pci_stop_root_bus (hbus->pci_bus); +pci_stop_root_bus (hbus->bridge->bus);hv_pci_remove_slots (hbus); -pci_remove_root_bus (hbus->pci_bus); +pci_remove_root_bus (hbus->bridge->bus);pci_unlock_rescan_remove (); }
@@ -3307,7 +3311,6 @@ static int hv_pci_remove(struct hv_device *hdev)iounmap (hbus->cfg_addr);hv_free_config_window (hbus); -pci_free_resource_list (&hbus->resources_for_children);hv_pci_free_bridge_windows (hbus);irq_domain_remove (hbus->irq_domain);irq_domain_free_fwnode (hbus->sysdata.fwnode);
@@ -3390,7 +3393,7 @@ static int hv_pci_restore_msi_msg(struct pci_dev *pdev, void *arg)*/ static void hv_pci_restore_msi_state (struct hv_pcibus_device *hbus) { -pci_walk_bus (hbus->pci_bus, hv_pci_restore_msi_msg,NULL ); +pci_walk_bus (hbus->bridge->bus, hv_pci_restore_msi_msg,NULL ); }static int hv_pci_resume (struct hv_device *hdev)
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
RFC v5 6/8 PCI: hv: Set ->domain_nr of pci_host_bridge at probing time
In reply to Boqun FengNo functional change, just store and maintain the PCI domain number in the ->domain_nr of pci_host_bridge. Note that we still need to keep the copy of domain number in x86-specific pci_sysdata, because x86 is not a PCI_DOMAINS_GENERIC=y architecture, so the ->domain_nr of pci_host_bridge doesn't work for it yet.
--- drivers/pci/controller/pci-hyperv.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/pci/controller/pci-hyperv.c +5 -4
@@ -2299,7 +2299,7 @@ static void hv_eject_device_work(struct work_struct *work)* because hbus->bridge->bus may not exist yet. */ wslot =wslot_to_devfn (hpdev->desc.win_slot.slot); - pdev =pci_get_domain_bus_and_slot (hbus->sysdata.domain,0 , wslot); + pdev =pci_get_domain_bus_and_slot (hbus->bridge->domain_nr,0 , wslot);if (pdev) {pci_lock_rescan_remove ();pci_stop_and_remove_bus_device (pdev);
@@ -3071,6 +3071,7 @@ static int hv_pci_probe(struct hv_device *hdev,"PCI dom# 0x%hx has collision, using 0x%hx" , dom_req, dom); + hbus->bridge->domain_nr = dom; hbus->sysdata.domain = dom; hbus->hdev = hdev;
@@ -3080,7 +3081,7 @@ static int hv_pci_probe(struct hv_device *hdev,spin_lock_init (&hbus->device_list_lock);spin_lock_init (&hbus->retarget_msi_interrupt_lock); hbus->wq =alloc_ordered_workqueue ("hv_pci_%x" ,0 , - hbus->sysdata.domain); + hbus->bridge->domain_nr);if (!hbus->wq) { ret = -ENOMEM ; goto free_dom;
@@ -3207,7 +3208,7 @@ static int hv_pci_probe(struct hv_device *hdev, destroy_wq:destroy_workqueue (hbus->wq); free_dom: -hv_put_dom_num (hbus->sysdata.domain); +hv_put_dom_num (hbus->bridge->domain_nr); free_bus:kfree (hbus);return ret;
@@ -3315,7 +3316,7 @@ static int hv_pci_remove(struct hv_device *hdev)irq_domain_remove (hbus->irq_domain);irq_domain_free_fwnode (hbus->sysdata.fwnode); -hv_put_dom_num (hbus->sysdata.domain); +hv_put_dom_num (hbus->bridge->domain_nr);kfree (hbus);return ret;
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
RFC v5 7/8 PCI: hv: Set up MSI domain at bridge probing time
In reply to Boqun FengSince PCI_HYPERV depends on PCI_MSI_IRQ_DOMAIN which selects GENERIC_MSI_IRQ_DOMAIN, we can use dev_set_msi_domain() to set up the MSI domain at probing time, and this works for both x86 and ARM64.
Therefore use it as the preparation for ARM64 Hyper-V PCI support.
As a result, no longer need to maintain ->fwnode in x86 specific pci_sysdata, and make hv_pcibus_device own it instead.
--- drivers/pci/controller/pci-hyperv.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
drivers/pci/controller/pci-hyperv.c +8 -5
@@ -450,6 +450,7 @@ enum hv_pcibus_state {struct hv_pcibus_device {struct pci_sysdata sysdata;struct pci_host_bridge *bridge; +struct fwnode_handle *fwnode;/* Protocol version negotiated with the host */ enum pci_protocol_version_t protocol_version;enum hv_pcibus_state state;
@@ -1565,7 +1566,7 @@ static int hv_pcie_init_irq_domain(struct hv_pcibus_device *hbus) hbus->msi_info.handler = handle_edge_irq; hbus->msi_info.handler_name ="edge" ; hbus->msi_info.data = hbus; - hbus->irq_domain =pci_msi_create_irq_domain (hbus->sysdata.fwnode, + hbus->irq_domain =pci_msi_create_irq_domain (hbus->fwnode, &hbus->msi_info, x86_vector_domain);if (!hbus->irq_domain) {
@@ -1574,6 +1575,8 @@ static int hv_pcie_init_irq_domain(struct hv_pcibus_device *hbus)return -ENODEV ; } +dev_set_msi_domain (&hbus->bridge->dev, hbus->irq_domain); +return 0 ; }
@@ -3118,9 +3121,9 @@ static int hv_pci_probe(struct hv_device *hdev, goto unmap; } - hbus->sysdata.fwnode =irq_domain_alloc_named_fwnode (name); + hbus->fwnode =irq_domain_alloc_named_fwnode (name);kfree (name); -if (!hbus->sysdata.fwnode) { +if (!hbus->fwnode) { ret = -ENOMEM ; goto unmap; }
@@ -3198,7 +3201,7 @@ static int hv_pci_probe(struct hv_device *hdev, free_irq_domain:irq_domain_remove (hbus->irq_domain); free_fwnode: -irq_domain_free_fwnode (hbus->sysdata.fwnode); +irq_domain_free_fwnode (hbus->fwnode); unmap:iounmap (hbus->cfg_addr); free_config:
@@ -3314,7 +3317,7 @@ static int hv_pci_remove(struct hv_device *hdev)hv_free_config_window (hbus);hv_pci_free_bridge_windows (hbus);irq_domain_remove (hbus->irq_domain); -irq_domain_free_fwnode (hbus->sysdata.fwnode); +irq_domain_free_fwnode (hbus->fwnode);hv_put_dom_num (hbus->bridge->domain_nr);
Signature · 1 line
2.30.2
RFC v5 8/8 PCI: hv: Turn on the host bridge probing on ARM64
In reply to Boqun FengNow we have everything we need, just provide a proper sysdata type for the bus to use on ARM64 and everything else works.
--- drivers/pci/controller/pci-hyperv.c | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/pci/controller/pci-hyperv.c +7 -0
@@ -40,6 +40,7 @@#include <linux/kernel.h> #include <linux/module.h> #include <linux/pci.h> +#include <linux/pci-ecam.h> #include <linux/delay.h> #include <linux/semaphore.h> #include <linux/irqdomain.h>
@@ -448,7 +449,11 @@ enum hv_pcibus_state { };struct hv_pcibus_device { +#ifdef CONFIG_X86 struct pci_sysdata sysdata; +#elif defined(CONFIG_ARM64 ) +struct pci_config_window sysdata; +#endif struct pci_host_bridge *bridge;struct fwnode_handle *fwnode;/* Protocol version negotiated with the host */
@@ -3075,7 +3080,9 @@ static int hv_pci_probe(struct hv_device *hdev, dom_req, dom); hbus->bridge->domain_nr = dom; +#ifdef CONFIG_X86 hbus->sysdata.domain = dom; +#endif hbus->hdev = hdev;INIT_LIST_HEAD (&hbus->children);
Signature · 7 lines
2.30.2 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [RFC v5 8/8] PCI: hv: Turn on the host bridge probing on ARM64
Quoted 22 lines · Boqun Feng
Now we have everything we need, just provide a proper sysdata type for the bus to use on ARM64 and everything else works.
Signed-off-by Boqun Feng <[email protected]>--- drivers/pci/controller/pci-hyperv.c | 7 +++++++ 1 file changed, 7 insertions(+)drivers/pci/controller/pci-hyperv.c +1
-0@@ -40,6 +40,7 @@#include <linux/kernel.h> #include <linux/module.h> #include <linux/pci.h> +#include <linux/pci-ecam.h> #include <linux/delay.h> #include <linux/semaphore.h> #include <linux/irqdomain.h>
drivers/pci/controller/pci-hyperv.c enum hv_pcibus_state { · line 456
Quoted 8 lines
drivers/pci/controller/pci-hyperv.c +4
-0@@ -448,7 +449,11 @@ enum hv_pcibus_state { };struct hv_pcibus_device { +#ifdef CONFIG_X86 struct pci_sysdata sysdata; +#elif defined(CONFIG_ARM64 ) +struct pci_config_window sysdata; +#endif
Am I the only one who find this rather odd? Nothing ever populates this data structure on arm64, and its only purpose seems to serve as an anchor to retrieve the hbus via container_of().
If that's indeed the case, I'd rather see an arch-specific to_hbus() helper that uses another (preexisting) field as the anchor for arm64.
Thanks,
M.
Signature · 6 lines
Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [RFC v5 8/8] PCI: hv: Turn on the host bridge probing on ARM64
Quoted 26 lines · Marc Zyngier
Quoted 24 lines · Boqun Feng
Now we have everything we need, just provide a proper sysdata type for the bus to use on ARM64 and everything else works.
Signed-off-by Boqun Feng <[email protected]>--- drivers/pci/controller/pci-hyperv.c | 7 +++++++ 1 file changed, 7 insertions(+)drivers/pci/controller/pci-hyperv.c +1
-0@@ -40,6 +40,7 @@#include <linux/kernel.h> #include <linux/module.h> #include <linux/pci.h> +#include <linux/pci-ecam.h> #include <linux/delay.h> #include <linux/semaphore.h> #include <linux/irqdomain.h> @@ -448,7 +449,11 @@ enum hv_pcibus_state { };
Quoted 11 lines
drivers/pci/controller/pci-hyperv.c enum hv_pcibus_state { · line 456
Quoted 6 lines
drivers/pci/controller/pci-hyperv.c +4
-0struct hv_pcibus_device { +#ifdef CONFIG_X86 struct pci_sysdata sysdata; +#elif defined(CONFIG_ARM64 ) +struct pci_config_window sysdata; +#endif Am I the only one who find this rather odd? Nothing ever populates this data structure on arm64, and its only purpose seems to serve as an anchor to retrieve the hbus via
container_of().
This field will also be used as the ->sysdata of pci_bus and pci_host_bridge, and some of the PCI core code touches. Although I made this field as all zeroed and make sure PCI core can handle (patch #4).
Quoted 3 lines
If that's indeed the case, I'd rather see an arch-specific
to_hbus()helper that uses another (preexisting) field as the anchor for arm64.
I did a quick look, but I didn't find another field works: the field needs to be placed inside hv_pcibus_device and the address can be retrieved via pci_bus. I'm open to any suggestion in case that I missed something.
Regards,
Boqun
Quoted 6 lines
Thanks,
M.Signature · 1 line
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [RFC v5 8/8] PCI: hv: Turn on the host bridge probing on ARM64
Quoted 40 lines · Boqun Feng
Quoted 26 lines · Marc Zyngier
Quoted 24 lines · Boqun Feng
Now we have everything we need, just provide a proper sysdata type for the bus to use on ARM64 and everything else works.
Signed-off-by Boqun Feng <[email protected]>--- drivers/pci/controller/pci-hyperv.c | 7 +++++++ 1 file changed, 7 insertions(+)drivers/pci/controller/pci-hyperv.c +1
-0@@ -40,6 +40,7 @@#include <linux/kernel.h> #include <linux/module.h> #include <linux/pci.h> +#include <linux/pci-ecam.h> #include <linux/delay.h> #include <linux/semaphore.h> #include <linux/irqdomain.h> @@ -448,7 +449,11 @@ enum hv_pcibus_state { };Quoted 11 lines
drivers/pci/controller/pci-hyperv.c enum hv_pcibus_state { · line 456
Quoted 6 lines
drivers/pci/controller/pci-hyperv.c +4
-0struct hv_pcibus_device { +#ifdef CONFIG_X86 struct pci_sysdata sysdata; +#elif defined(CONFIG_ARM64 ) +struct pci_config_window sysdata; +#endif Am I the only one who find this rather odd? Nothing ever populates this data structure on arm64, and its only purpose seems to serve as an anchor to retrieve the hbus via
container_of().
Quoted 3 lines
This field will also be used as the ->sysdata of
pci_busandpci_host_bridge, and some of the PCI core code touches. Although I made this field as all zeroed and make sure PCI core can handle (patch #4).
Huh, I see. I missed this particular nugget. This is so convoluted...
Quoted 8 lines
Quoted 3 lines
If that's indeed the case, I'd rather see an arch-specific
to_hbus()helper that uses another (preexisting) field as the anchor for arm64.I did a quick look, but I didn't find another field works: the field needs to be placed inside
hv_pcibus_deviceand the address can be retrieved viapci_bus. I'm open to any suggestion in case that I missed something.
No, the above pretty much kills my suggestion.
Thanks for the explanation,
M.
Signature · 6 lines
Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel