iommu/amd: Add multiple PCI segments support
Subsystems (2)
cover iommu/amd: Add multiple PCI segments support
Newer AMD systems can support multiple PCI segments, where each segment contains one or more IOMMU instances. However, an IOMMU instance can only support a single PCI segment.
Current code assumes a system contains only one PCI segment (segment 0) and creates global data structures such as device table, rlookup table, etc.
This series introduces per-PCI-segment data structure, which contains device table, alias table, etc. For each PCI segment, all IOMMUs share the same data structure. The series also makes necessary code adjustment and logging enhancements. Finally it removes global data structures like device table, alias table, etc.
In case of system w/ single PCI segment (e.g. PCI segment ID is zero), IOMMU driver allocates one PCI segment data structure, which will be shared by all IOMMUs.
Patch 1 Updates struct iommu_dev_data defination.
Patch 2 - 13 introduce new PCI segment structure and allocate per data structures, and introduce the amd_iommu.pci_seg pointer to point to the corresponded pci_segment structure. Also, we have introduced a helper function rlookup_amd_iommu() to reverse-lookup each iommu for a particular device.
Patch 14 - 29 adopt to per PCI segment data structure and removes global data structure.
Patch 30 fixes flushing logic to flush upto last_bdf.
Patch 31 - 37 convert usages of 16-bit PCI device ID to include 16-bit segment ID.
RFC patchset : https://lore.kernel.org/linux-iommu/[email protected]/T/#t
Changes in RFC -> v1:
- Rebased patches on top of iommu/next tree.
- Update struct iommu_dev_data defination
- Updated few log message to print segment ID
- Fix smatch warnings
Regards,
Vasant
Suravee Suthikulpanit (21):
iommu/amd: Introduce per PCI segment device table
iommu/amd: Introduce per PCI segment rlookup table
iommu/amd: Introduce per PCI segment old_dev_tbl_cpy
iommu/amd: Introduce per PCI segment alias_table
iommu/amd: Convert to use rlookup_amd_iommu helper function
iommu/amd: Update irq_remapping_alloc to use IOMMU lookup helper function iommu/amd: Introduce struct amd_ir_data.iommu
iommu/amd: Update amd_irte_ops functions
iommu/amd: Update alloc_irq_table and alloc_irq_index
iommu/amd: Update set_dte_entry and clear_dte_entry
iommu/amd: Update iommu_ignore_device
iommu/amd: Update dump_dte_entry
iommu/amd: Update set_dte_irq_entry
iommu/amd: Update (un)init_device_table_dma()
iommu/amd: Update set_dev_entry_bit() and get_dev_entry_bit() iommu/amd: Remove global amd_iommu_dev_table
iommu/amd: Remove global amd_iommu_alias_table
iommu/amd: Introduce get_device_sbdf_id() helper function
iommu/amd: Include PCI segment ID when initialize IOMMU
iommu/amd: Specify PCI segment ID when getting pci device
iommu/amd: Add PCI segment support for ivrs_ioapic, ivrs_hpet, ivrs_acpihid commands
Vasant Hegde (16):
iommu/amd: Update struct iommu_dev_data defination iommu/amd: Introduce pci segment structure
iommu/amd: Introduce per PCI segment irq_lookup_table iommu/amd: Introduce per PCI segment dev_data_list iommu/amd: Introduce per PCI segment unity map list iommu/amd: Introduce per PCI segment last_bdf iommu/amd: Introduce per PCI segment device table size iommu/amd: Introduce per PCI segment alias table size iommu/amd: Introduce per PCI segment rlookup table size iommu/amd: Convert to use per PCI segment irq_lookup_table iommu/amd: Convert to use per PCI segment rlookup_table iommu/amd: Remove global amd_iommu_last_bdf
iommu/amd: Flush upto last_bdf only
iommu/amd: Print PCI segment ID in error log messages iommu/amd: Update device_state structure to include PCI seg ID iommu/amd: Update amd_iommu_fault structure to include PCI seg ID
.../admin-guide/kernel-parameters.txt | 34 +- drivers/iommu/amd/amd_iommu.h | 13 +- drivers/iommu/amd/amd_iommu_types.h | 127 +++- drivers/iommu/amd/init.c | 683 +++++++++++------- drivers/iommu/amd/iommu.c | 540 ++++++++------ drivers/iommu/amd/iommu_v2.c | 67 +- drivers/iommu/amd/quirks.c | 4 +- 7 files changed, 884 insertions(+), 584 deletions(-)
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 1/37 iommu/amd: Update struct iommu_dev_data defination
struct iommu_dev_data contains member "pdev" to point to pci_dev. This is valid for only PCI devices and for other devices this will be NULL. This causes unnecessary "pdev != NULL" check at various places.
Replace "struct pci_dev" member with "struct device" and use to_pci_dev() to get pci device reference as needed. Also adjust setup_aliases() and clone_aliases() function.
No functional change intended.
--- drivers/iommu/amd/amd_iommu_types.h | 2 +- drivers/iommu/amd/iommu.c | 27 +++++++++++++++------------ 2 files changed, 16 insertions(+), 13 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +1 -1
@@ -685,7 +685,7 @@ struct iommu_dev_data {struct list_head list;/* For domain->dev_list */ struct llist_node dev_data_list;/* For global dev_data_list */ struct protection_domain *domain;/* Domain the device is bound to */ -struct pci_dev *pdev; +struct device *dev;u16 devid;/* PCI Device ID */ bool iommu_v2;/* Device can make use of IOMMUv2 */ struct {
drivers/iommu/amd/iommu.c +15 -12
@@ -188,8 +188,10 @@ static int clone_alias(struct pci_dev *pdev, u16 alias, void *data)return 0 ; } -static void clone_aliases (struct pci_dev *pdev) +static void clone_aliases (struct device *dev) { +struct pci_dev *pdev =to_pci_dev (dev); +if (!pdev)return ;
@@ -203,14 +205,14 @@ static void clone_aliases(struct pci_dev *pdev)pci_for_each_dma_alias (pdev, clone_alias,NULL ); } -static struct pci_dev *setup_aliases (struct device *dev) +static void setup_aliases (struct device *dev) {struct pci_dev *pdev =to_pci_dev (dev);u16 ivrs_alias;/* For ACPI HID devices, there are no aliases */ if (!dev_is_pci (dev)) -return NULL ; +return ;/* * Add the IVRS alias to the pci aliases if it is on the same
@@ -221,9 +223,7 @@ static struct pci_dev *setup_aliases(struct device *dev) PCI_BUS_NUM(ivrs_alias) == pdev->bus->number)pci_add_dma_alias (pdev, ivrs_alias &0xff ,1 ); -clone_aliases (pdev); - -return pdev; +clone_aliases (dev); }static struct iommu_dev_data *find_dev_data (u16 devid)
@@ -331,7 +331,8 @@ static int iommu_init_device(struct device *dev)if (!dev_data)return -ENOMEM ; - dev_data->pdev =setup_aliases (dev); + dev_data->dev = dev; +setup_aliases (dev);/* * By default we use passthrough mode for IOMMUv2 capable device.
@@ -1232,13 +1233,15 @@ static int device_flush_dte_alias(struct pci_dev *pdev, u16 alias, void *data)static int device_flush_dte (struct iommu_dev_data *dev_data) {struct amd_iommu *iommu; +struct pci_dev *pdev;u16 alias;int ret; iommu = amd_iommu_rlookup_table[dev_data->devid]; -if (dev_data->pdev) - ret =pci_for_each_dma_alias (dev_data->pdev, + pdev =to_pci_dev (dev_data->dev); +if (pdev) + ret =pci_for_each_dma_alias (pdev, device_flush_dte_alias, iommu);else ret =iommu_flush_dte (iommu, dev_data->devid);
@@ -1561,7 +1564,7 @@ static void do_attach(struct iommu_dev_data *dev_data,/* Update device table */ set_dte_entry (dev_data->devid, domain, ats, dev_data->iommu_v2); -clone_aliases (dev_data->pdev); +clone_aliases (dev_data->dev);device_flush_dte (dev_data); }
@@ -1577,7 +1580,7 @@ static void do_detach(struct iommu_dev_data *dev_data) dev_data->domain =NULL ;list_del (&dev_data->list);clear_dte_entry (dev_data->devid); -clone_aliases (dev_data->pdev); +clone_aliases (dev_data->dev);/* Flush the DTE entry */ device_flush_dte (dev_data);
@@ -1818,7 +1821,7 @@ static void update_device_table(struct protection_domain *domain)list_for_each_entry (dev_data, &domain->dev_list, list) {set_dte_entry (dev_data->devid, domain, dev_data->ats.enabled, dev_data->iommu_v2); -clone_aliases (dev_data->pdev); +clone_aliases (dev_data->dev); } }
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 2/37 iommu/amd: Introduce pci segment structure
In reply to Vasant Hegde via iommuNewer AMD systems can support multiple PCI segments, where each segment contains one or more IOMMU instances. However, an IOMMU instance can only support a single PCI segment.
Current code assumes that system contains only one pci segment (segment 0) and creates global data structures such as device table, rlookup table, etc.
Introducing per PCI segment data structure, which contains segment specific data structures. This will eventually replace the global data structures.
Also update amd_iommu->pci_seg variable to point to PCI segment structure instead of PCI segment ID.
--- drivers/iommu/amd/amd_iommu_types.h | 23 ++++++++++++++- drivers/iommu/amd/init.c | 46 ++++++++++++++++++++++++++++- 2 files changed, 67 insertions(+), 2 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +22 -1
@@ -452,6 +452,11 @@ extern bool amd_iommu_irq_remap;/* kmem_cache to get tables with 128 byte alignement */ extern struct kmem_cache *amd_iommu_irq_cache; +/* Make iterating over all pci segment easier */ +#define for_each_pci_segment (pci_seg) \ + list_for_each_entry((pci_seg), &amd_iommu_pci_seg_list, list) +#define for_each_pci_segment_safe (pci_seg, next) \ + list_for_each_entry_safe((pci_seg), (next), &amd_iommu_pci_seg_list, list)/* * Make iterating over all IOMMUs easier */
@@ -526,6 +531,16 @@ struct protection_domain {unsigned dev_iommu[MAX_IOMMUS ];/* per-IOMMU reference count */ }; +/* +* This structure contains information about one PCI segment in the system. +*/ +struct amd_iommu_pci_seg { +struct list_head list; + +/* PCI segment number */ +u16 id; +}; +/* * Structure where we save information about one hardware AMD IOMMU in the * system.
@@ -577,7 +592,7 @@ struct amd_iommu {u16 cap_ptr;/* pci domain of this IOMMU */ -u16 pci_seg; +struct amd_iommu_pci_seg *pci_seg;/* start of exclusion range of that IOMMU */ u64 exclusion_start;
@@ -705,6 +720,12 @@ extern struct list_head ioapic_map;extern struct list_head hpet_map;extern struct list_head acpihid_map; +/* +* List with all PCI segments in the system. This list is not locked because +* it is only written at driver initialization time +*/ +extern struct list_head amd_iommu_pci_seg_list; +/* * List with all IOMMUs in the system. This list is not locked because it is * only written and read at driver initialization or suspend time
drivers/iommu/amd/init.c +45 -1
78 lines not on this page · raw message
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 3/37 iommu/amd: Introduce per PCI segment device table
In reply to Vasant Hegde via iommuFrom: Suravee Suthikulpanit <[email protected]>
Introduce per PCI segment device table. All IOMMUs within the segment will share this device table. This will replace global device table i.e. amd_iommu_dev_table.
Also introduce helper function to get the device table for the given IOMMU.
--- drivers/iommu/amd/amd_iommu.h | 1 + drivers/iommu/amd/amd_iommu_types.h | 10 ++++++++++ drivers/iommu/amd/init.c | 26 ++++++++++++++++++++++++-- drivers/iommu/amd/iommu.c | 12 ++++++++++++ 4 files changed, 47 insertions(+), 2 deletions(-)
drivers/iommu/amd/amd_iommu.h +1 -0
@@ -128,4 +128,5 @@ static inline void amd_iommu_apply_ivrs_quirks(void) { }extern void amd_iommu_domain_set_pgtable (struct protection_domain *domain,u64 *root,int mode); +extern struct dev_table_entry *get_dev_table (struct amd_iommu *iommu);#endif
drivers/iommu/amd/amd_iommu_types.h +10 -0
@@ -539,6 +539,16 @@ struct amd_iommu_pci_seg {/* PCI segment number */ u16 id; + +/* +* device table virtual address +* +* Pointer to the per PCI segment device table. +* It is indexed by the PCI device id or the HT unit id and contains +* information about the domain the device belongs to as well as the +* page table root pointer. +*/ +struct dev_table_entry *dev_table; };/*
drivers/iommu/amd/init.c +24 -2
@@ -640,11 +640,29 @@ static int __init find_last_devid_acpi(struct acpi_table_header *table)* * The following functions belong to the code path which parses the ACPI table * the second time. In this ACPI parsing iteration we allocate IOMMU specific -* data structures, initialize the device/alias/rlookup table and also -* basically initialize the hardware. +* data structures, initialize the per PCI segment device/alias/rlookup table +* and also basically initialize the hardware. * ****************************************************************************/ +/* Allocate per PCI segment device table */ +static inline int __init alloc_dev_table (struct amd_iommu_pci_seg *pci_seg) +{ + pci_seg->dev_table = (void *)__get_free_pages (GFP_KERNEL | __GFP_ZERO |GFP_DMA32 , +get_order (dev_table_size)); +if (!pci_seg->dev_table) +return -ENOMEM ; + +return 0 ; +} + +static inline void free_dev_table (struct amd_iommu_pci_seg *pci_seg) +{ +free_pages ((unsigned long )pci_seg->dev_table, +get_order (dev_table_size)); + pci_seg->dev_table =NULL ; +} +/* * Allocates the command buffer. This buffer is per AMD IOMMU. We can * write commands to that buffer later and the IOMMU will execute them
@@ -1469,6 +1487,9 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id) pci_seg->id = id;list_add_tail (&pci_seg->list, &amd_iommu_pci_seg_list); +if (alloc_dev_table (pci_seg)) +return NULL ; +return pci_seg; }
@@ -1490,6 +1511,7 @@ static void __init free_pci_segment(void)for_each_pci_segment_safe (pci_seg, next) {list_del (&pci_seg->list); +free_dev_table (pci_seg);kfree (pci_seg); } }
drivers/iommu/amd/iommu.c +12 -0
@@ -134,6 +134,18 @@ static inline int get_device_id(struct device *dev)return devid; } +struct dev_table_entry *get_dev_table (struct amd_iommu *iommu) +{ +struct dev_table_entry *dev_table; +struct amd_iommu_pci_seg *pci_seg = iommu->pci_seg; + +BUG_ON (pci_seg ==NULL ); + dev_table = pci_seg->dev_table; +BUG_ON (dev_table ==NULL ); + +return dev_table; +} +static struct protection_domain *to_pdomain (struct iommu_domain *dom) { return container_of(dom, struct protection_domain, domain);
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 4/37 iommu/amd: Introduce per PCI segment rlookup table
In reply to Vasant Hegde via iommuFrom: Suravee Suthikulpanit <[email protected]>
This will replace global rlookup table (amd_iommu_rlookup_table). Also add helper functions to set/get rlookup table for the given device.
--- drivers/iommu/amd/amd_iommu.h | 1 + drivers/iommu/amd/amd_iommu_types.h | 8 ++++++ drivers/iommu/amd/init.c | 23 +++++++++++++++ drivers/iommu/amd/iommu.c | 44 +++++++++++++++++++++++++++++ 4 files changed, 76 insertions(+)
drivers/iommu/amd/amd_iommu.h +1 -0
@@ -19,6 +19,7 @@ extern int amd_iommu_init_devices(void);extern void amd_iommu_uninit_devices (void );extern void amd_iommu_init_notifier (void );extern int amd_iommu_init_api (void ); +extern void amd_iommu_set_rlookup_table (struct amd_iommu *iommu,u16 devid);#ifdef CONFIG_AMD_IOMMU_DEBUGFS void amd_iommu_debugfs_setup (struct amd_iommu *iommu);
drivers/iommu/amd/amd_iommu_types.h +8 -0
@@ -486,6 +486,7 @@ struct amd_iommu_fault { }; +struct amd_iommu ;struct iommu_domain ;struct irq_domain ;struct amd_irte_ops ;
@@ -549,6 +550,13 @@ struct amd_iommu_pci_seg {* page table root pointer. */ struct dev_table_entry *dev_table; + +/* +* The rlookup iommu table is used to find the IOMMU which is +* responsible for a specific device. It is indexed by the PCI +* device id. +*/ +struct amd_iommu **rlookup_table; };/*
drivers/iommu/amd/init.c +23 -0
@@ -663,6 +663,26 @@ static inline void free_dev_table(struct amd_iommu_pci_seg *pci_seg) pci_seg->dev_table =NULL ; } +/* Allocate per PCI segment IOMMU rlookup table. */ +static inline int __init alloc_rlookup_table (struct amd_iommu_pci_seg *pci_seg) +{ + pci_seg->rlookup_table = (void *)__get_free_pages ( +GFP_KERNEL | __GFP_ZERO, +get_order (rlookup_table_size)); +if (pci_seg->rlookup_table ==NULL ) +return -ENOMEM ; + +return 0 ; +} + +static inline void free_rlookup_table (struct amd_iommu_pci_seg *pci_seg) +{ +free_pages ((unsigned long )pci_seg->rlookup_table, +get_order (rlookup_table_size)); + pci_seg->rlookup_table =NULL ; +} + +/* * Allocates the command buffer. This buffer is per AMD IOMMU. We can * write commands to that buffer later and the IOMMU will execute them
@@ -1489,6 +1509,8 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id)if (alloc_dev_table (pci_seg))return NULL ; +if (alloc_rlookup_table (pci_seg)) +return NULL ;return pci_seg; }
@@ -1511,6 +1533,7 @@ static void __init free_pci_segment(void)for_each_pci_segment_safe (pci_seg, next) {list_del (&pci_seg->list); +free_rlookup_table (pci_seg);free_dev_table (pci_seg);kfree (pci_seg); }
drivers/iommu/amd/iommu.c +44 -0
50 lines not on this page · raw message
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 5/37 iommu/amd: Introduce per PCI segment irq_lookup_table
In reply to Vasant Hegde via iommuThis will replace global irq lookup table (irq_lookup_table).
--- drivers/iommu/amd/amd_iommu_types.h | 6 ++++++ drivers/iommu/amd/init.c | 27 +++++++++++++++++++++++++++ 2 files changed, 33 insertions(+)
drivers/iommu/amd/amd_iommu_types.h +6 -0
@@ -557,6 +557,12 @@ struct amd_iommu_pci_seg {* device id. */ struct amd_iommu **rlookup_table; + +/* +* This table is used to find the irq remapping table for a given +* device id quickly. +*/ +struct irq_remap_table **irq_lookup_table; };/*
drivers/iommu/amd/init.c +27 -0
@@ -682,6 +682,26 @@ static inline void free_rlookup_table(struct amd_iommu_pci_seg *pci_seg) pci_seg->rlookup_table =NULL ; } +static inline int __init alloc_irq_lookup_table (struct amd_iommu_pci_seg *pci_seg) +{ + pci_seg->irq_lookup_table = (void *)__get_free_pages ( +GFP_KERNEL | __GFP_ZERO, +get_order (rlookup_table_size)); +kmemleak_alloc (pci_seg->irq_lookup_table, + rlookup_table_size,1 ,GFP_KERNEL ); +if (pci_seg->irq_lookup_table ==NULL ) +return -ENOMEM ; + +return 0 ; +} + +static inline void free_irq_lookup_table (struct amd_iommu_pci_seg *pci_seg) +{ +kmemleak_free (pci_seg->irq_lookup_table); +free_pages ((unsigned long )pci_seg->irq_lookup_table, +get_order (rlookup_table_size)); + pci_seg->irq_lookup_table =NULL ; +}/* * Allocates the command buffer. This buffer is per AMD IOMMU. We can
@@ -1533,6 +1553,7 @@ static void __init free_pci_segment(void)for_each_pci_segment_safe (pci_seg, next) {list_del (&pci_seg->list); +free_irq_lookup_table (pci_seg);free_rlookup_table (pci_seg);free_dev_table (pci_seg);kfree (pci_seg);
@@ -2896,6 +2917,7 @@ static int __init early_amd_iommu_init(void) amd_iommu_irq_remap =check_ioapic_information ();if (amd_iommu_irq_remap) { +struct amd_iommu_pci_seg *pci_seg;/* * Interrupt remapping enabled, create kmem_cache for the * remapping tables.
@@ -2912,6 +2934,11 @@ static int __init early_amd_iommu_init(void)if (!amd_iommu_irq_cache) goto out; +for_each_pci_segment (pci_seg) { +if (alloc_irq_lookup_table (pci_seg)) + goto out; + } + irq_lookup_table = (void *)__get_free_pages (GFP_KERNEL | __GFP_ZERO,get_order (rlookup_table_size));
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 6/37 iommu/amd: Introduce per PCI segment dev_data_list
In reply to Vasant Hegde via iommuThis will replace global dev_data_list.
--- drivers/iommu/amd/amd_iommu_types.h | 3 +++ drivers/iommu/amd/init.c | 1 + drivers/iommu/amd/iommu.c | 21 ++++++++++----------- 3 files changed, 14 insertions(+), 11 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +3 -0
@@ -538,6 +538,9 @@ struct protection_domain {struct amd_iommu_pci_seg {struct list_head list; +/* List of all available dev_data structures */ +struct llist_head dev_data_list; +/* PCI segment number */ u16 id;
drivers/iommu/amd/init.c +1 -0
@@ -1525,6 +1525,7 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id)return NULL ; pci_seg->id = id; +init_llist_head (&pci_seg->dev_data_list);list_add_tail (&pci_seg->list, &amd_iommu_pci_seg_list);if (alloc_dev_table (pci_seg))
drivers/iommu/amd/iommu.c +10 -11
@@ -62,9 +62,6 @@static DEFINE_SPINLOCK (pd_bitmap_lock); -/* List of all available dev_data structures */ -static LLIST_HEAD (dev_data_list); -LIST_HEAD (ioapic_map);LIST_HEAD (hpet_map);LIST_HEAD (acpihid_map);
@@ -195,9 +192,10 @@ static struct protection_domain *to_pdomain(struct iommu_domain *dom) return container_of(dom, struct protection_domain, domain); } -static struct iommu_dev_data *alloc_dev_data (u16 devid) +static struct iommu_dev_data *alloc_dev_data (struct amd_iommu *iommu,u16 devid) {struct iommu_dev_data *dev_data; +struct amd_iommu_pci_seg *pci_seg = iommu->pci_seg; dev_data =kzalloc (sizeof (*dev_data),GFP_KERNEL );if (!dev_data)
@@ -207,19 +205,20 @@ static struct iommu_dev_data *alloc_dev_data(u16 devid) dev_data->devid = devid;ratelimit_default_init (&dev_data->rs); -llist_add (&dev_data->dev_data_list, &dev_data_list); +llist_add (&dev_data->dev_data_list, &pci_seg->dev_data_list);return dev_data; } -static struct iommu_dev_data *search_dev_data (u16 devid) +static struct iommu_dev_data *search_dev_data (struct amd_iommu *iommu,u16 devid) {struct iommu_dev_data *dev_data;struct llist_node *node; +struct amd_iommu_pci_seg *pci_seg = iommu->pci_seg; -if (llist_empty (&dev_data_list)) +if (llist_empty (&pci_seg->dev_data_list))return NULL ; - node = dev_data_list.first; + node = pci_seg->dev_data_list.first;llist_for_each_entry (dev_data, node, dev_data_list) {if (dev_data->devid == devid)return dev_data;
@@ -287,10 +286,10 @@ static struct iommu_dev_data *find_dev_data(u16 devid)struct iommu_dev_data *dev_data;struct amd_iommu *iommu = amd_iommu_rlookup_table[devid]; - dev_data =search_dev_data (devid); + dev_data =search_dev_data (iommu, devid);if (dev_data ==NULL ) { - dev_data =alloc_dev_data (devid); + dev_data =alloc_dev_data (iommu, devid);if (!dev_data)return NULL ;
@@ -3461,7 +3460,7 @@ static int amd_ir_set_vcpu_affinity(struct irq_data *data, void *vcpu_info)struct vcpu_data *vcpu_pi_info = pi_data->vcpu_data;struct amd_ir_data *ir_data = data->chip_data;struct irq_2_irte *irte_info = &ir_data->irq_2_irte; -struct iommu_dev_data *dev_data =search_dev_data (irte_info->devid); +struct iommu_dev_data *dev_data =search_dev_data (NULL , irte_info->devid);/* Note: * This device has never been set up for guest mode.
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 7/37 iommu/amd: Introduce per PCI segment old_dev_tbl_cpy
In reply to Vasant Hegde via iommuFrom: Suravee Suthikulpanit <[email protected]>
It will remove global old_dev_tbl_cpy. Also update copy_device_table() copy device table for all PCI segments.
--- drivers/iommu/amd/amd_iommu_types.h | 6 ++ drivers/iommu/amd/init.c | 109 ++++++++++++++++------------ 2 files changed, 70 insertions(+), 45 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +6 -0
@@ -566,6 +566,12 @@ struct amd_iommu_pci_seg {* device id quickly. */ struct irq_remap_table **irq_lookup_table; + +/* +* Pointer to a device table which the content of old device table +* will be copied to. It's only be used in kdump kernel. +*/ +struct dev_table_entry *old_dev_tbl_cpy; };/*
drivers/iommu/amd/init.c +64 -45
183 lines not on this page · raw message
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 8/37 iommu/amd: Introduce per PCI segment alias_table
In reply to Vasant Hegde via iommuFrom: Suravee Suthikulpanit <[email protected]>
This will replace global alias table (amd_iommu_alias_table).
--- drivers/iommu/amd/amd_iommu_types.h | 7 +++++ drivers/iommu/amd/init.c | 41 ++++++++++++++++++++++------- drivers/iommu/amd/iommu.c | 41 ++++++++++++++++++----------- 3 files changed, 64 insertions(+), 25 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +7 -0
@@ -572,6 +572,13 @@ struct amd_iommu_pci_seg {* will be copied to. It's only be used in kdump kernel. */ struct dev_table_entry *old_dev_tbl_cpy; + +/* +* The alias table is a driver specific data structure which contains the +* mappings of the PCI device ids to the actual requestor ids on the IOMMU. +* More than one device can share the same requestor id. +*/ +u16 *alias_table; };/*
drivers/iommu/amd/init.c +32 -9
89 lines not on this page · raw message
drivers/iommu/amd/iommu.c +25 -16
151 lines not on this page · raw message
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 9/37 iommu/amd: Introduce per PCI segment unity map list
In reply to Vasant Hegde via iommuNewer AMD systems can support multiple PCI segments. In order to support multiple PCI segments IVMD table in IVRS structure is enhanced to include pci segment id. Update ivmd_header structure to include "pci_seg".
Also introduce per PCI segment unity map list. It will replace global amd_iommu_unity_map list.
Note that we have used "reserved" field in IVMD table to include "pci_seg id" which was set to zero. It will take care of backward compatibility (new kernel will work fine on older systems).
--- drivers/iommu/amd/amd_iommu_types.h | 13 +++++++------ drivers/iommu/amd/init.c | 30 +++++++++++++++++++---------- drivers/iommu/amd/iommu.c | 8 +++++++- 3 files changed, 34 insertions(+), 17 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +7 -6
@@ -579,6 +579,13 @@ struct amd_iommu_pci_seg {* More than one device can share the same requestor id. */ u16 *alias_table; + +/* +* A list of required unity mappings we find in ACPI. It is not locked +* because as runtime it is only read. It is created at ACPI table +* parsing time. +*/ +struct list_head unity_map; };/*
@@ -805,12 +812,6 @@ struct unity_map_entry {int prot; }; -/* -* List of all unity mappings. It is not locked because as runtime it is only -* read. It is created at ACPI table parsing time. -*/ -extern struct list_head amd_iommu_unity_map; -/* * Data structures for device handling */
drivers/iommu/amd/init.c +20 -10
@@ -142,7 +142,8 @@ struct ivmd_header {u16 length;u16 devid;u16 aux; -u64 resv; +u16 pci_seg; +u8 resv[6 ];u64 range_start;u64 range_length; } __attribute__((packed));
@@ -162,8 +163,6 @@ static int amd_iommu_target_ivhd_type;u16 amd_iommu_last_bdf;/* largest PCI device id we have to handle */ -LIST_HEAD (amd_iommu_unity_map);/* a list of required unity mappings -we find in ACPI */ LIST_HEAD (amd_iommu_pci_seg_list);/* list of all PCI segments */ LIST_HEAD (amd_iommu_list);/* list of all AMD IOMMUs in the
@@ -1562,6 +1561,7 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id) pci_seg->id = id;init_llist_head (&pci_seg->dev_data_list); +INIT_LIST_HEAD (&pci_seg->unity_map);list_add_tail (&pci_seg->list, &amd_iommu_pci_seg_list);if (alloc_dev_table (pci_seg))
@@ -2397,10 +2397,13 @@ static int iommu_init_irq(struct amd_iommu *iommu)static void __init free_unity_maps (void ) {struct unity_map_entry *entry, *next; +struct amd_iommu_pci_seg *p, *pci_seg; -list_for_each_entry_safe (entry, next, &amd_iommu_unity_map, list) { -list_del (&entry->list); -kfree (entry); +for_each_pci_segment_safe (pci_seg, p) { +list_for_each_entry_safe (entry, next, &pci_seg->unity_map, list) { +list_del (&entry->list); +kfree (entry); + } } }
@@ -2408,8 +2411,13 @@ static void __init free_unity_maps(void)static int __init init_unity_map_range (struct ivmd_header *m) {struct unity_map_entry *e =NULL ; +struct amd_iommu_pci_seg *pci_seg;char *s; + pci_seg =get_pci_segment (m->pci_seg); +if (pci_seg ==NULL ) +return -ENOMEM ; + e =kzalloc (sizeof (*e),GFP_KERNEL );if (e ==NULL )return -ENOMEM ;
@@ -2447,14 +2455,16 @@ static int __init init_unity_map_range(struct ivmd_header *m)if (m->flags &IVMD_FLAG_EXCL_RANGE ) e->prot = (IVMD_FLAG_IW |IVMD_FLAG_IR ) >>1 ; -DUMP_printk ("%s devid_start: %02x:%02x.%x devid_end: %02x:%02x.%x" -" range_start: %016llx range_end: %016llx flags: %x\n" , s, +DUMP_printk ("%s devid_start: %04x:%02x:%02x.%x devid_end: " +"%04x:%02x:%02x.%x range_start: %016llx range_end: %016llx" +" flags: %x\n" , s, m->pci_seg,PCI_BUS_NUM (e->devid_start),PCI_SLOT (e->devid_start), -PCI_FUNC (e->devid_start),PCI_BUS_NUM (e->devid_end), +PCI_FUNC (e->devid_start), m->pci_seg, +PCI_BUS_NUM (e->devid_end),PCI_SLOT (e->devid_end),PCI_FUNC (e->devid_end), e->address_start, e->address_end, m->flags); -list_add_tail (&e->list, &amd_iommu_unity_map); +list_add_tail (&e->list, &pci_seg->unity_map);return 0 ; }
drivers/iommu/amd/iommu.c +7 -1
@@ -2239,13 +2239,19 @@ static void amd_iommu_get_resv_regions(struct device *dev, {struct iommu_resv_region *region;struct unity_map_entry *entry; +struct amd_iommu *iommu; +struct amd_iommu_pci_seg *pci_seg;int devid; devid =get_device_id (dev);if (devid <0 )return ; + iommu =rlookup_amd_iommu (dev); +if (!iommu) +return ; + pci_seg = iommu->pci_seg; -list_for_each_entry (entry, &amd_iommu_unity_map, list) { +list_for_each_entry (entry, &pci_seg->unity_map, list) {int type, prot =0 ;size_t length;
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 10/37 iommu/amd: Introduce per PCI segment last_bdf
In reply to Vasant Hegde via iommuCurrent code uses global "amd_iommu_last_bdf" to track the last bdf supported by the system. This value is used for various memory allocation, device data flushing, etc.
Introduce per PCI segment last_bdf which will be used to track last bdf supported by the given PCI segment and use this value for all per segment memory allocations. Eventually it will replace global "amd_iommu_last_bdf".
--- drivers/iommu/amd/amd_iommu_types.h | 3 ++ drivers/iommu/amd/init.c | 68 ++++++++++++++++++----------- 2 files changed, 45 insertions(+), 26 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +3 -0
@@ -544,6 +544,9 @@ struct amd_iommu_pci_seg {/* PCI segment number */ u16 id; +/* Largest PCI device id we expect translation requests for */ +u16 last_bdf; +/* * device table virtual address *
drivers/iommu/amd/init.c +42 -26
176 lines not on this page · raw message
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 11/37 iommu/amd: Introduce per PCI segment device table size
In reply to Vasant Hegde via iommuWith multiple pci segment support, number of BDF supported by each segment may differ. Hence introduce per segment device table size which depends on last_bdf. This will replace global "device_table_size" variable.
--- drivers/iommu/amd/amd_iommu_types.h | 3 +++ drivers/iommu/amd/init.c | 18 ++++++++++-------- 2 files changed, 13 insertions(+), 8 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +3 -0
@@ -547,6 +547,9 @@ struct amd_iommu_pci_seg {/* Largest PCI device id we expect translation requests for */ u16 last_bdf; +/* Size of the device table */ +u32 dev_table_size; +/* * device table virtual address *
drivers/iommu/amd/init.c +10 -8
@@ -414,6 +414,7 @@ static void iommu_set_cwwb_range(struct amd_iommu *iommu)static void iommu_set_device_table (struct amd_iommu *iommu) {u64 entry; +u32 dev_table_size = iommu->pci_seg->dev_table_size;BUG_ON (iommu->mmio_base ==NULL );
@@ -651,7 +652,7 @@ static int __init find_last_devid_acpi(struct acpi_table_header *table, u16 pci_static inline int __init alloc_dev_table (struct amd_iommu_pci_seg *pci_seg) { pci_seg->dev_table = (void *)__get_free_pages (GFP_KERNEL | __GFP_ZERO |GFP_DMA32 , -get_order (dev_table_size)); +get_order (pci_seg->dev_table_size));if (!pci_seg->dev_table)return -ENOMEM ;
@@ -661,7 +662,7 @@ static inline int __init alloc_dev_table(struct amd_iommu_pci_seg *pci_seg)static inline void free_dev_table (struct amd_iommu_pci_seg *pci_seg) {free_pages ((unsigned long )pci_seg->dev_table, -get_order (dev_table_size)); +get_order (pci_seg->dev_table_size)); pci_seg->dev_table =NULL ; }
@@ -1034,7 +1035,7 @@ static bool __copy_device_table(struct amd_iommu *iommu) entry = (((u64 ) hi) <<32 ) + lo; old_devtb_size = ((entry & ~PAGE_MASK ) +1 ) <<12 ; -if (old_devtb_size != dev_table_size) { +if (old_devtb_size != pci_seg->dev_table_size) {pr_err ("The device table size of IOMMU:%d is not expected!\n" , iommu->index);return false;
@@ -1053,15 +1054,15 @@ static bool __copy_device_table(struct amd_iommu *iommu) } old_devtb = (cc_platform_has (CC_ATTR_HOST_MEM_ENCRYPT ) &&is_kdump_kernel ()) ? (__force void *)ioremap_encrypted (old_devtb_phys, - dev_table_size) - :memremap (old_devtb_phys, dev_table_size,MEMREMAP_WB ); + pci_seg->dev_table_size) + :memremap (old_devtb_phys, pci_seg->dev_table_size,MEMREMAP_WB );if (!old_devtb)return false; gfp_flag =GFP_KERNEL | __GFP_ZERO |GFP_DMA32 ; pci_seg->old_dev_tbl_cpy = (void *)__get_free_pages (gfp_flag, -get_order (dev_table_size)); +get_order (pci_seg->dev_table_size));if (pci_seg->old_dev_tbl_cpy ==NULL ) {pr_err ("Failed to allocate memory for copying old device table!\n" );memunmap (old_devtb);
@@ -1580,6 +1581,7 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id, pci_seg->last_bdf = last_bdf;DUMP_printk ("PCI segment : 0x%0x, last bdf : 0x%04x\n" , id, last_bdf); + pci_seg->dev_table_size =tbl_size (DEV_TABLE_ENTRY_SIZE ); pci_seg->id = id;init_llist_head (&pci_seg->dev_data_list);
@@ -2675,7 +2677,7 @@ static void early_enable_iommus(void)for_each_pci_segment (pci_seg) {if (pci_seg->old_dev_tbl_cpy !=NULL ) {free_pages ((unsigned long )pci_seg->old_dev_tbl_cpy, -get_order (dev_table_size)); +get_order (pci_seg->dev_table_size)); pci_seg->old_dev_tbl_cpy =NULL ; } }
@@ -2689,7 +2691,7 @@ static void early_enable_iommus(void)for_each_pci_segment (pci_seg) {free_pages ((unsigned long )pci_seg->dev_table, -get_order (dev_table_size)); +get_order (pci_seg->dev_table_size)); pci_seg->dev_table = pci_seg->old_dev_tbl_cpy; }
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 12/37 iommu/amd: Introduce per PCI segment alias table size
In reply to Vasant Hegde via iommuIt will replace global "alias_table_size" variable.
--- drivers/iommu/amd/amd_iommu_types.h | 3 +++ drivers/iommu/amd/init.c | 5 +++-- 2 files changed, 6 insertions(+), 2 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +3 -0
@@ -550,6 +550,9 @@ struct amd_iommu_pci_seg {/* Size of the device table */ u32 dev_table_size; +/* Size of the alias table */ +u32 alias_table_size; +/* * device table virtual address *
drivers/iommu/amd/init.c +3 -2
@@ -711,7 +711,7 @@ static int __init alloc_alias_table(struct amd_iommu_pci_seg *pci_seg)int i; pci_seg->alias_table = (void *)__get_free_pages (GFP_KERNEL , -get_order (alias_table_size)); +get_order (pci_seg->alias_table_size));if (!pci_seg->alias_table)return -ENOMEM ;
@@ -727,7 +727,7 @@ static int __init alloc_alias_table(struct amd_iommu_pci_seg *pci_seg)static void __init free_alias_table (struct amd_iommu_pci_seg *pci_seg) {free_pages ((unsigned long )pci_seg->alias_table, -get_order (alias_table_size)); +get_order (pci_seg->alias_table_size)); pci_seg->alias_table =NULL ; }
@@ -1582,6 +1582,7 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id, pci_seg->last_bdf = last_bdf;DUMP_printk ("PCI segment : 0x%0x, last bdf : 0x%04x\n" , id, last_bdf); pci_seg->dev_table_size =tbl_size (DEV_TABLE_ENTRY_SIZE ); + pci_seg->alias_table_size =tbl_size (ALIAS_TABLE_ENTRY_SIZE ); pci_seg->id = id;init_llist_head (&pci_seg->dev_data_list);
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 13/37 iommu/amd: Introduce per PCI segment rlookup table size
In reply to Vasant Hegde via iommuIt will replace global "rlookup_table_size" variable.
--- drivers/iommu/amd/amd_iommu_types.h | 3 +++ drivers/iommu/amd/init.c | 11 ++++++----- 2 files changed, 9 insertions(+), 5 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +3 -0
@@ -553,6 +553,9 @@ struct amd_iommu_pci_seg {/* Size of the alias table */ u32 alias_table_size; +/* Size of the rlookup table */ +u32 rlookup_table_size; +/* * device table virtual address *
drivers/iommu/amd/init.c +6 -5
@@ -671,7 +671,7 @@ static inline int __init alloc_rlookup_table(struct amd_iommu_pci_seg *pci_seg) { pci_seg->rlookup_table = (void *)__get_free_pages (GFP_KERNEL | __GFP_ZERO, -get_order (rlookup_table_size)); +get_order (pci_seg->rlookup_table_size));if (pci_seg->rlookup_table ==NULL )return -ENOMEM ;
@@ -681,7 +681,7 @@ static inline int __init alloc_rlookup_table(struct amd_iommu_pci_seg *pci_seg)static inline void free_rlookup_table (struct amd_iommu_pci_seg *pci_seg) {free_pages ((unsigned long )pci_seg->rlookup_table, -get_order (rlookup_table_size)); +get_order (pci_seg->rlookup_table_size)); pci_seg->rlookup_table =NULL ; }
@@ -689,9 +689,9 @@ static inline int __init alloc_irq_lookup_table(struct amd_iommu_pci_seg *pci_se { pci_seg->irq_lookup_table = (void *)__get_free_pages (GFP_KERNEL | __GFP_ZERO, -get_order (rlookup_table_size)); +get_order (pci_seg->rlookup_table_size));kmemleak_alloc (pci_seg->irq_lookup_table, - rlookup_table_size,1 ,GFP_KERNEL ); + pci_seg->rlookup_table_size,1 ,GFP_KERNEL );if (pci_seg->irq_lookup_table ==NULL )return -ENOMEM ;
@@ -702,7 +702,7 @@ static inline void free_irq_lookup_table(struct amd_iommu_pci_seg *pci_seg) {kmemleak_free (pci_seg->irq_lookup_table);free_pages ((unsigned long )pci_seg->irq_lookup_table, -get_order (rlookup_table_size)); +get_order (pci_seg->rlookup_table_size)); pci_seg->irq_lookup_table =NULL ; }
@@ -1583,6 +1583,7 @@ static struct amd_iommu_pci_seg *__init alloc_pci_segment(u16 id,DUMP_printk ("PCI segment : 0x%0x, last bdf : 0x%04x\n" , id, last_bdf); pci_seg->dev_table_size =tbl_size (DEV_TABLE_ENTRY_SIZE ); pci_seg->alias_table_size =tbl_size (ALIAS_TABLE_ENTRY_SIZE ); + pci_seg->rlookup_table_size =tbl_size (RLOOKUP_TABLE_ENTRY_SIZE ); pci_seg->id = id;init_llist_head (&pci_seg->dev_data_list);
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 14/37 iommu/amd: Convert to use per PCI segment irq_lookup_table
In reply to Vasant Hegde via iommuThen, remove the global irq_lookup_table.
--- drivers/iommu/amd/amd_iommu_types.h | 2 -- drivers/iommu/amd/init.c | 19 --------------- drivers/iommu/amd/iommu.c | 36 ++++++++++++++++++----------- 3 files changed, 23 insertions(+), 34 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +0 -2
@@ -444,8 +444,6 @@ struct irq_remap_table {u32 *table; }; -extern struct irq_remap_table **irq_lookup_table; -/* Interrupt remapping feature used? */ extern bool amd_iommu_irq_remap;
drivers/iommu/amd/init.c +0 -19
@@ -206,12 +206,6 @@ u16 *amd_iommu_alias_table;*/ struct amd_iommu **amd_iommu_rlookup_table; -/* -* This table is used to find the irq remapping table for a given device id -* quickly. -*/ -struct irq_remap_table **irq_lookup_table; -/* * AMD IOMMU allows up to 2^16 different protection domains. This is a bitmap * to know which ones are already in use.
@@ -2786,11 +2780,6 @@ static struct syscore_ops amd_iommu_syscore_ops = {static void __init free_iommu_resources (void ) { -kmemleak_free (irq_lookup_table); -free_pages ((unsigned long )irq_lookup_table, -get_order (rlookup_table_size)); - irq_lookup_table =NULL ; -kmem_cache_destroy (amd_iommu_irq_cache); amd_iommu_irq_cache =NULL ;
@@ -3011,14 +3000,6 @@ static int __init early_amd_iommu_init(void)if (alloc_irq_lookup_table (pci_seg)) goto out; } - - irq_lookup_table = (void *)__get_free_pages ( -GFP_KERNEL | __GFP_ZERO, -get_order (rlookup_table_size)); -kmemleak_alloc (irq_lookup_table, rlookup_table_size, -1 ,GFP_KERNEL ); -if (!irq_lookup_table) - goto out; } ret =init_memory_definitions (ivrs_base);
drivers/iommu/amd/iommu.c +23 -13
@@ -2727,16 +2727,18 @@ static void set_dte_irq_entry(u16 devid, struct irq_remap_table *table) amd_iommu_dev_table[devid].data[2 ] = dte; } -static struct irq_remap_table *get_irq_table (u16 devid) +static struct irq_remap_table *get_irq_table (struct amd_iommu *iommu,u16 devid) {struct irq_remap_table *table; +struct amd_iommu_pci_seg *pci_seg = iommu->pci_seg;if (WARN_ONCE (!amd_iommu_rlookup_table[devid],"%s: no iommu for devid %x\n" , __func__, devid))return NULL ; - table = irq_lookup_table[devid]; -if (WARN_ONCE (!table,"%s: no table for devid %x\n" , __func__, devid)) + table = pci_seg->irq_lookup_table[devid]; +if (WARN_ONCE (!table,"%s: no table for devid %x:%x\n" , + __func__, pci_seg->id, devid))return NULL ;return table;
@@ -2769,7 +2771,9 @@ static struct irq_remap_table *__alloc_irq_table(void)static void set_remap_table_entry (struct amd_iommu *iommu,u16 devid,struct irq_remap_table *table) { - irq_lookup_table[devid] = table; +struct amd_iommu_pci_seg *pci_seg = iommu->pci_seg; + + pci_seg->irq_lookup_table[devid] = table;set_dte_irq_entry (devid, table);iommu_flush_dte (iommu, devid); }
@@ -2778,8 +2782,14 @@ static int set_remap_table_entry_alias(struct pci_dev *pdev, u16 alias, void *data) {struct irq_remap_table *table = data; +struct amd_iommu_pci_seg *pci_seg; +struct amd_iommu *iommu =rlookup_amd_iommu (&pdev->dev); - irq_lookup_table[alias] = table; +if (!iommu) +return -EINVAL ; + + pci_seg = iommu->pci_seg; + pci_seg->irq_lookup_table[alias] = table;set_dte_irq_entry (alias, table);iommu_flush_dte (amd_iommu_rlookup_table[alias], alias);
@@ -2803,12 +2813,12 @@ static struct irq_remap_table *alloc_irq_table(u16 devid, struct pci_dev *pdev) goto out_unlock; pci_seg = iommu->pci_seg; - table = irq_lookup_table[devid]; + table = pci_seg->irq_lookup_table[devid];if (table) goto out_unlock; alias = pci_seg->alias_table[devid]; - table = irq_lookup_table[alias]; + table = pci_seg->irq_lookup_table[alias];if (table) {set_remap_table_entry (iommu, devid, table); goto out_wait;
@@ -2822,11 +2832,11 @@ static struct irq_remap_table *alloc_irq_table(u16 devid, struct pci_dev *pdev)spin_lock_irqsave (&iommu_table_lock, flags); - table = irq_lookup_table[devid]; + table = pci_seg->irq_lookup_table[devid];if (table) goto out_unlock; - table = irq_lookup_table[alias]; + table = pci_seg->irq_lookup_table[alias];if (table) {set_remap_table_entry (iommu, devid, table); goto out_wait;
@@ -2920,7 +2930,7 @@ static int modify_irte_ga(u16 devid, int index, struct irte_ga *irte,if (iommu ==NULL )return -EINVAL ; - table =get_irq_table (devid); + table =get_irq_table (iommu, devid);if (!table)return -ENOMEM ;
@@ -2961,7 +2971,7 @@ static int modify_irte(u16 devid, int index, union irte *irte)if (iommu ==NULL )return -EINVAL ; - table =get_irq_table (devid); + table =get_irq_table (iommu, devid);if (!table)return -ENOMEM ;
@@ -2985,7 +2995,7 @@ static void free_irte(u16 devid, int index)if (iommu ==NULL )return ; - table =get_irq_table (devid); + table =get_irq_table (iommu, devid);if (!table)return ;
@@ -3620,7 +3630,7 @@ int amd_iommu_update_ga(int cpu, bool is_run, void *data)if (!iommu)return -ENODEV ; - table =get_irq_table (devid); + table =get_irq_table (iommu, devid);if (!table)return -ENODEV ;
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 15/37 iommu/amd: Convert to use rlookup_amd_iommu helper function
In reply to Vasant Hegde via iommuFrom: Suravee Suthikulpanit <[email protected]>
Use rlookup_amd_iommu() helper function which will give per PCI segment rlookup_table.
--- drivers/iommu/amd/iommu.c | 64 +++++++++++++++++++++++---------------- 1 file changed, 38 insertions(+), 26 deletions(-)
drivers/iommu/amd/iommu.c +38 -26
@@ -229,13 +229,17 @@ static struct iommu_dev_data *search_dev_data(struct amd_iommu *iommu, u16 devidstatic int clone_alias (struct pci_dev *pdev,u16 alias,void *data) { +struct amd_iommu *iommu;u16 devid =pci_dev_id (pdev);if (devid == alias)return 0 ; - amd_iommu_rlookup_table[alias] = - amd_iommu_rlookup_table[devid]; + iommu =rlookup_amd_iommu (&pdev->dev); +if (!iommu) +return 0 ; + +amd_iommu_set_rlookup_table (iommu, alias);memcpy (amd_iommu_dev_table[alias].data, amd_iommu_dev_table[devid].data,sizeof (amd_iommu_dev_table[alias].data));
@@ -365,7 +369,7 @@ static bool check_device(struct device *dev)if (devid > amd_iommu_last_bdf)return false; -if (amd_iommu_rlookup_table[devid] ==NULL ) +if (rlookup_amd_iommu (dev) ==NULL )return false;return true;
@@ -1269,7 +1273,9 @@ static int device_flush_iotlb(struct iommu_dev_data *dev_data,int qdep; qdep = dev_data->ats.qdep; - iommu = amd_iommu_rlookup_table[dev_data->devid]; + iommu =rlookup_amd_iommu (dev_data->dev); +if (!iommu) +return -EINVAL ;build_inv_iotlb_pages (&cmd, dev_data->devid, qdep, address, size);
@@ -1294,7 +1300,9 @@ static int device_flush_dte(struct iommu_dev_data *dev_data)u16 alias;int ret; - iommu = amd_iommu_rlookup_table[dev_data->devid]; + iommu =rlookup_amd_iommu (dev_data->dev); +if (!iommu) +return -EINVAL ; pdev =to_pci_dev (dev_data->dev);if (pdev)
@@ -1522,8 +1530,8 @@ static void free_gcr3_table(struct protection_domain *domain)free_page ((unsigned long )domain->gcr3_tbl); } -static void set_dte_entry (u16 devid,struct protection_domain *domain, -bool ats,bool ppr) +static void set_dte_entry (struct amd_iommu *iommu,u16 devid, +struct protection_domain *domain,bool ats,bool ppr) {u64 pte_root =0 ;u64 flags =0 ;
@@ -1542,8 +1550,6 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, flags |=DTE_FLAG_IOTLB ;if (ppr) { -struct amd_iommu *iommu = amd_iommu_rlookup_table[devid]; -if (iommu_feature (iommu,FEATURE_EPHSUP )) pte_root |=1ULL <<DEV_ENTRY_PPR ; }
@@ -1587,8 +1593,6 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain,* entries for the old domain ID that is being overwritten */ if (old_domid) { -struct amd_iommu *iommu = amd_iommu_rlookup_table[devid]; -amd_iommu_flush_tlb_domid (iommu, old_domid); } }
@@ -1608,7 +1612,9 @@ static void do_attach(struct iommu_dev_data *dev_data,struct amd_iommu *iommu;bool ats; - iommu = amd_iommu_rlookup_table[dev_data->devid]; + iommu =rlookup_amd_iommu (dev_data->dev); +if (!iommu) +return ; ats = dev_data->ats.enabled;/* Update data structures */
@@ -1620,7 +1626,7 @@ static void do_attach(struct iommu_dev_data *dev_data, domain->dev_cnt +=1 ;/* Update device table */ -set_dte_entry (dev_data->devid, domain, +set_dte_entry (iommu, dev_data->devid, domain, ats, dev_data->iommu_v2);clone_aliases (iommu, dev_data->dev);
@@ -1632,7 +1638,9 @@ static void do_detach(struct iommu_dev_data *dev_data)struct protection_domain *domain = dev_data->domain;struct amd_iommu *iommu; - iommu = amd_iommu_rlookup_table[dev_data->devid]; + iommu =rlookup_amd_iommu (dev_data->dev); +if (!iommu) +return ;/* Update data structures */ dev_data->domain =NULL ;
@@ -1810,13 +1818,14 @@ static struct iommu_device *amd_iommu_probe_device(struct device *dev) {struct iommu_device *iommu_dev;struct amd_iommu *iommu; -int ret, devid; +int ret;if (!check_device (dev))return ERR_PTR (-ENODEV ); - devid =get_device_id (dev); - iommu = amd_iommu_rlookup_table[devid]; + iommu =rlookup_amd_iommu (dev); +if (!iommu) +return ERR_PTR (-ENODEV );if (dev_iommu_priv_get (dev))return &iommu->iommu;
@@ -1846,13 +1855,14 @@ static void amd_iommu_probe_finalize(struct device *dev)static void amd_iommu_release_device (struct device *dev) { -int devid =get_device_id (dev);struct amd_iommu *iommu;if (!check_device (dev))return ; - iommu = amd_iommu_rlookup_table[devid]; + iommu =rlookup_amd_iommu (dev); +if (!iommu) +return ;amd_iommu_uninit_device (dev);iommu_completion_wait (iommu);
@@ -1881,7 +1891,7 @@ static void update_device_table(struct protection_domain *domain)if (!iommu)continue ; -set_dte_entry (dev_data->devid, domain, +set_dte_entry (iommu, dev_data->devid, domain, dev_data->ats.enabled, dev_data->iommu_v2);clone_aliases (iommu, dev_data->dev); }
@@ -2076,7 +2086,6 @@ static void amd_iommu_detach_device(struct iommu_domain *dom, struct device *dev) {struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); -int devid =get_device_id (dev);struct amd_iommu *iommu;if (!check_device (dev))
@@ -2085,7 +2094,7 @@ static void amd_iommu_detach_device(struct iommu_domain *dom,if (dev_data->domain !=NULL )detach_device (dev); - iommu = amd_iommu_rlookup_table[devid]; + iommu =rlookup_amd_iommu (dev);if (!iommu)return ;
@@ -2112,7 +2121,7 @@ static int amd_iommu_attach_device(struct iommu_domain *dom, dev_data =dev_iommu_priv_get (dev); dev_data->defer_attach = false; - iommu = amd_iommu_rlookup_table[dev_data->devid]; + iommu =rlookup_amd_iommu (dev);if (!iommu)return -EINVAL ;
@@ -2488,8 +2497,9 @@ static int __flush_pasid(struct protection_domain *domain, u32 pasid,continue ; qdep = dev_data->ats.qdep; - iommu = amd_iommu_rlookup_table[dev_data->devid]; - + iommu =rlookup_amd_iommu (dev_data->dev); +if (!iommu) +continue ;build_inv_iotlb_pasid (&cmd, dev_data->devid, pasid, qdep, address, size);
@@ -2651,7 +2661,9 @@ int amd_iommu_complete_ppr(struct pci_dev *pdev, u32 pasid,struct iommu_cmd cmd; dev_data =dev_iommu_priv_get (&pdev->dev); - iommu = amd_iommu_rlookup_table[dev_data->devid]; + iommu =rlookup_amd_iommu (&pdev->dev); +if (!iommu) +return -ENODEV ;build_complete_ppr (&cmd, dev_data->devid, pasid, status, tag, dev_data->pri_tlp);
Signature · 6 lines
2.27.0 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Quoted 4 lines · Vasant Hegde via iommu
Newer AMD systems can support multiple PCI segments, where each segment contains one or more IOMMU instances. However, an IOMMU instance can only support a single PCI segment.
Hi,
Please ignore this series. Looks like I had network glitch and git didn't send entire series. I have resent series.
-Vasant
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu