Remove default DMA window before creating DDW
v4 cover Remove default DMA window before creating DDW
There are some devices in which a hypervisor may only allow 1 DMA window to exist at a time, and in those cases, a DDW is never created to them, since the default DMA window keeps using this resource.
LoPAR recommends this procedure:
1. Remove the default DMA window,
2. Query for which configs the DDW can be created,
3. Create a DDW.
Patch #1:
Create defines for outputs of ibm,ddw-applicable, so it's easier to identify them.
Patch #2:
- After LoPAR level 2.8, there is an extension that can make ibm,query-pe-dma-windows to have 6 outputs instead of 5. This changes the order of the outputs, and that can cause some trouble.
- query_ddw() was updated to check how many outputs the ibm,query-pe-dma-windows is supposed to have, update the rtas_call() and deal correctly with the outputs in both cases.
- This patch looks somehow unrelated to the series, but it can avoid future problems on DDW creation.
Patch #3 moves the window-removing code from remove_ddw() to remove_dma_window(), creating a way to delete any DMA window, so it can be used to delete the default DMA window.
Patch #4 makes use of the remove_dma_window() from patch #3 to remove the default DMA window before query_ddw(). It also implements a new rtas call to recover the default DMA window, in case anything fails after it was removed, and a DDW couldn't be created.
Patch #5 moves the part of iommu_table_free() that does struct iommu_table cleaning into iommu_table_clean, so we can invoke it separately in patch #6.
Patch #6:
Instead of destroying the created DDW if it doesn't map the whole partition, make use of it instead of the default DMA window as it improves performance. Also, update the iommu_table and re-generate the pools.
Patch #7:
Does some renaming of 'direct window' to 'dma window', given the DDW created can now be also used in indirect mapping if direct mapping is not available.
All patches were tested into an LPAR with an Ethernet VF:
4005:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4 Virtual Function]
Patch #6 It was tested with a 64GB DDW which did not map the whole partition (128G). Performance improvement noticed by using the DDW instead of the default DMA window:
64 thread write throughput: +203.0%
64 thread read throughput: +17.5%
1 thread write throughput: +20.5%
1 thread read throughput: +3.43%
Average write latency: -23.0% Average read latency: -2.26%
---
Changes since v3:
- Introduces new patch #5, to prepare for an important change in #6
- struct iommu_table was not being updated, so include a way to do this
in patch #6.
- Improved patch #4 based in a suggestion from Alexey, to make code
more easily understandable
- v3 link: http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=187348&state=%2A&archive=both
Changes since v2:
- Change the way ibm,ddw-extensions is accessed, using a proper function
instead of doing this inline everytime it's used.
- Remove previous patch #6, as it doesn't look like it would be useful.
- Add new patch, for changing names from direct* to dma*, as indirect
mapping can be used from now on.
- Fix some typos, corrects some define usage.
- v2 link: http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=185433&state=%2A&archive=both
Changes since v1:
- Add defines for ibm,ddw-applicable and ibm,ddw-extensions outputs
- Merge aux function query_ddw_out_sz() into query_ddw()
- Merge reset_dma_window() patch (prev. #2) into remove default DMA
window patch (#4).
- Keep device_node *np name instead of using pdn in remove_*()
- Rename 'device_node *pdn' into 'parent' in new functions
- Rename dfl_win to default_win
- Only remove the default DMA window if there is no window available
in first query.
- Check if default DMA window can be restored before removing it.
- Fix 'unitialized use' (found by travis mpe:ci-test)
- New patches #5 and #6
- v1 link: http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=184420&state=%2A&archive=both
Special thanks for Alexey Kardashevskiy, Brian King and Oliver O'Halloran for the feedback provided!
Leonardo Bras (7):
powerpc/pseries/iommu: Create defines for operations in
ibm,ddw-applicable
powerpc/pseries/iommu: Update call to ibm,query-pe-dma-windows powerpc/pseries/iommu: Move window-removing part of remove_ddw into
remove_dma_window
powerpc/pseries/iommu: Remove default DMA window before creating DDW powerpc/iommu: Move iommu_table cleaning routine to iommu_table_clean powerpc/pseries/iommu: Make use of DDW even if it does not map the
partition
powerpc/pseries/iommu: Rename "direct window" to "dma window"
arch/powerpc/include/asm/iommu.h | 3 + arch/powerpc/kernel/iommu.c | 45 ++- arch/powerpc/platforms/pseries/iommu.c | 380 ++++++++++++++++++------- 3 files changed, 313 insertions(+), 115 deletions(-)
Signature · 1 line
2.25.4
PATCH v4 1/7 powerpc/pseries/iommu: Create defines for operations in ibm,ddw-applicable
Create defines to help handling ibm,ddw-applicable values, avoiding confusion about the index of given operations.
--- arch/powerpc/platforms/pseries/iommu.c | 43 ++++++++++++++++---------- 1 file changed, 26 insertions(+), 17 deletions(-)
arch/powerpc/platforms/pseries/iommu.c +26 -17
@@ -39,6 +39,14 @@#include "pseries.h" +enum { +DDW_QUERY_PE_DMA_WIN =0 , +DDW_CREATE_PE_DMA_WIN =1 , +DDW_REMOVE_PE_DMA_WIN =2 , + +DDW_APPLICABLE_SIZE +}; +static struct iommu_table_group *iommu_pseries_alloc_group (int node) {struct iommu_table_group *table_group;
@@ -771,12 +779,12 @@ static void remove_ddw(struct device_node *np, bool remove_prop) {struct dynamic_dma_window_prop *dwp;struct property *win64; -u32 ddw_avail[3 ]; +u32 ddw_avail[DDW_APPLICABLE_SIZE ];u64 liobn;int ret =0 ; ret =of_property_read_u32_array (np,"ibm,ddw-applicable" , - &ddw_avail[0 ],3 ); + &ddw_avail[0 ],DDW_APPLICABLE_SIZE ); win64 =of_find_property (np,DIRECT64_PROPNAME ,NULL );if (!win64)
@@ -798,15 +806,15 @@ static void remove_ddw(struct device_node *np, bool remove_prop)pr_debug ("%pOF successfully cleared tces in window.\n" , np); - ret =rtas_call (ddw_avail[2 ],1 ,1 ,NULL , liobn); + ret =rtas_call (ddw_avail[DDW_REMOVE_PE_DMA_WIN ],1 ,1 ,NULL , liobn);if (ret)pr_warn ("%pOF: failed to remove direct window: rtas returned " "%d to ibm,remove-pe-dma-window(%x) %llx\n" , - np, ret, ddw_avail[2 ], liobn); + np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN ], liobn);else pr_debug ("%pOF: successfully removed direct window: rtas returned " "%d to ibm,remove-pe-dma-window(%x) %llx\n" , - np, ret, ddw_avail[2 ], liobn); + np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN ], liobn); delprop:if (remove_prop)
@@ -889,11 +897,11 @@ static int query_ddw(struct pci_dev *dev, const u32 *ddw_avail, buid = pdn->phb->buid; cfg_addr = ((pdn->busno <<16 ) | (pdn->devfn <<8 )); - ret =rtas_call (ddw_avail[0 ],3 ,5 , (u32 *)query, - cfg_addr,BUID_HI (buid),BUID_LO (buid)); + ret =rtas_call (ddw_avail[DDW_QUERY_PE_DMA_WIN ],3 ,5 , (u32 *)query, + cfg_addr,BUID_HI (buid),BUID_LO (buid));dev_info (&dev->dev,"ibm,query-pe-dma-windows(%x) %x %x %x" -" returned %d\n" , ddw_avail[0 ], cfg_addr,BUID_HI (buid), -BUID_LO (buid), ret); +" returned %d\n" , ddw_avail[DDW_QUERY_PE_DMA_WIN ], cfg_addr, +BUID_HI (buid),BUID_LO (buid), ret);return ret; }
@@ -920,15 +928,16 @@ static int create_ddw(struct pci_dev *dev, const u32 *ddw_avail,do {/* extra outputs are LIOBN and dma-addr (hi, lo) */ - ret =rtas_call (ddw_avail[1 ],5 ,4 , (u32 *)create, - cfg_addr,BUID_HI (buid),BUID_LO (buid), - page_shift, window_shift); + ret =rtas_call (ddw_avail[DDW_CREATE_PE_DMA_WIN ],5 ,4 , + (u32 *)create, cfg_addr,BUID_HI (buid), +BUID_LO (buid), page_shift, window_shift); }while (rtas_busy_delay (ret));dev_info (&dev->dev,"ibm,create-pe-dma-window(%x) %x %x %x %x %x returned %d " -"(liobn = 0x%x starting addr = %x %x)\n" , ddw_avail[1 ], - cfg_addr,BUID_HI (buid),BUID_LO (buid), page_shift, - window_shift, ret, create->liobn, create->addr_hi, create->addr_lo); +"(liobn = 0x%x starting addr = %x %x)\n" , + ddw_avail[DDW_CREATE_PE_DMA_WIN ], cfg_addr,BUID_HI (buid), +BUID_LO (buid), page_shift, window_shift, ret, create->liobn, + create->addr_hi, create->addr_lo);return ret; }
@@ -996,7 +1005,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)int page_shift;u64 dma_addr, max_addr;struct device_node *dn; -u32 ddw_avail[3 ]; +u32 ddw_avail[DDW_APPLICABLE_SIZE ];struct direct_window *window;struct property *win64;struct dynamic_dma_window_prop *ddwprop;
@@ -1029,7 +1038,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)* the property is actually in the parent, not the PE */ ret =of_property_read_u32_array (pdn,"ibm,ddw-applicable" , - &ddw_avail[0 ],3 ); + &ddw_avail[0 ],DDW_APPLICABLE_SIZE );if (ret) goto out_failed;
Signature · 1 line
2.25.4
Re: [PATCH v4 1/7] powerpc/pseries/iommu: Create defines for operations in ibm, ddw-applicable
Quoted 139 lines · Leonardo Bras
Create defines to help handling ibm,ddw-applicable values, avoiding confusion about the index of given operations.
Signed-off-by Leonardo Bras <[email protected]>--- arch/powerpc/platforms/pseries/iommu.c | 43 ++++++++++++++++------ ---- 1 file changed, 26 insertions(+), 17 deletions(-)arch/powerpc/platforms/pseries/iommu.c +10
-2@@ -39,6 +39,14 @@#include "pseries.h" +enum { +DDW_QUERY_PE_DMA_WIN =0 , +DDW_CREATE_PE_DMA_WIN =1 , +DDW_REMOVE_PE_DMA_WIN =2 , + +DDW_APPLICABLE_SIZE +}; +static struct iommu_table_group *iommu_pseries_alloc_group (int node) {struct iommu_table_group *table_group;@@ -771,12 +779,12 @@ static void remove_ddw(struct device_node *np, bool remove_prop) {struct dynamic_dma_window_prop *dwp;struct property *win64; -u32 ddw_avail[3 ]; +u32 ddw_avail[DDW_APPLICABLE_SIZE ];u64 liobn;int ret =0 ; ret =of_property_read_u32_array (np,"ibm,ddw-applicable" , - &ddw_avail[0 ],3 ); + &ddw_avail[0 ],DDW_APPLICABLE_SIZE );arch/powerpc/platforms/pseries/iommu.c +1
-1win64 =of_find_property (np,DIRECT64_PROPNAME ,NULL );if (!win64)@@ -798,15 +806,15 @@ static void remove_ddw(struct device_node *np, bool remove_prop) pr_debug("%pOF successfully cleared tces in window.\n", np); - ret =rtas_call (ddw_avail[2 ],1 ,1 ,NULL , liobn); + ret =rtas_call (ddw_avail[DDW_REMOVE_PE_DMA_WIN ],1 ,1 ,NULL , liobn);if (ret) pr_warn("%pOF: failed to remove direct window: rtasreturned "
arch/powerpc/platforms/pseries/iommu.c +1
-1"%d to ibm,remove-pe-dma-window(%x) %llx\n" , - np, ret, ddw_avail[2], liobn); + np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN],liobn);
arch/powerpc/platforms/pseries/iommu.c +0
-0else pr_debug("%pOF: successfully removed direct window:rtas returned "
arch/powerpc/platforms/pseries/iommu.c +1
-1"%d to ibm,remove-pe-dma-window(%x) %llx\n" , - np, ret, ddw_avail[2], liobn); + np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN],liobn);
delprop:
arch/powerpc/platforms/pseries/iommu.c +3
-4if (remove_prop)@@ -889,11 +897,11 @@ static int query_ddw(struct pci_dev *dev, constu32 *ddw_avail, buid = pdn->phb->buid; cfg_addr = ((pdn->busno <<16 ) | (pdn->devfn <<8 )); - ret =rtas_call (ddw_avail[0 ],3 ,5 , (u32 *)query, - cfg_addr,BUID_HI (buid),BUID_LO (buid)); + ret =rtas_call (ddw_avail[DDW_QUERY_PE_DMA_WIN ],3 ,5 , (u32 *)query, + cfg_addr,BUID_HI (buid),BUID_LO (buid)); dev_info(&dev->dev, "ibm,query-pe-dma-windows(%x) %x %x %x" -" returned %d\n" , ddw_avail[0 ], cfg_addr, BUID_HI(buid), -BUID_LO (buid), ret); + " returned %d\n", ddw_avail[DDW_QUERY_PE_DMA_WIN],
cfg_addr,arch/powerpc/platforms/pseries/iommu.c +4
-3+ BUID_HI(buid), BUID_LO(buid), ret);return ret; }@@ -920,15 +928,16 @@ static int create_ddw(struct pci_dev *dev,const u32 *ddw_avail, do {/* extra outputs are LIOBN and dma-addr (hi, lo) */ - ret =rtas_call (ddw_avail[1 ],5 ,4 , (u32 *)create, - cfg_addr,BUID_HI (buid),BUID_LO (buid), - page_shift, window_shift); + ret =rtas_call (ddw_avail[DDW_CREATE_PE_DMA_WIN ],5 ,4 , + (u32 *)create, cfg_addr,BUID_HI (buid), +BUID_LO (buid), page_shift, window_shift); }while (rtas_busy_delay (ret)); dev_info(&dev->dev, "ibm,create-pe-dma-window(%x) %x %x %x %x %x returned%d "
arch/powerpc/platforms/pseries/iommu.c +0
-3-"(liobn = 0x%x starting addr = %x %x)\n" , ddw_avail[1 ], - cfg_addr,BUID_HI (buid),BUID_LO (buid), page_shift, - window_shift, ret, create->liobn, create->addr_hi,
create->addr_lo);arch/powerpc/platforms/pseries/iommu.c +2
-0+"(liobn = 0x%x starting addr = %x %x)\n" , + ddw_avail[DDW_CREATE_PE_DMA_WIN], cfg_addr,
BUID_HI(buid),arch/powerpc/platforms/pseries/iommu.c +1
-0+ BUID_LO(buid), page_shift, window_shift, ret, create-Quoted 1 line
liobn,
arch/powerpc/platforms/pseries/iommu.c +2
-1+ create->addr_hi, create->addr_lo);return ret; }@@ -996,7 +1005,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)int page_shift;u64 dma_addr, max_addr;struct device_node *dn; -u32 ddw_avail[3 ]; +u32 ddw_avail[DDW_APPLICABLE_SIZE ];struct direct_window *window;struct property *win64;arch/powerpc/platforms/pseries/iommu.c +1
-1struct dynamic_dma_window_prop *ddwprop;@@ -1029,7 +1038,7 @@ static u64 enable_ddw(struct pci_dev *dev,struct device_node *pdn) * the property is actually in the parent, not the PE */ ret =of_property_read_u32_array (pdn,"ibm,ddw-applicable" , - &ddw_avail[0 ],3 ); + &ddw_avail[0 ],DDW_APPLICABLE_SIZE );if (ret)
Quoted 2 lines
goto
out_failed;
Re: [PATCH v4 1/7] powerpc/pseries/iommu: Create defines for operations in ibm, ddw-applicable
In reply to Leonardo BrasQuoted 139 lines · Leonardo Bras
Create defines to help handling ibm,ddw-applicable values, avoiding confusion about the index of given operations.
Signed-off-by Leonardo Bras <[email protected]>--- arch/powerpc/platforms/pseries/iommu.c | 43 ++++++++++++++++------ ---- 1 file changed, 26 insertions(+), 17 deletions(-)arch/powerpc/platforms/pseries/iommu.c +10
-2@@ -39,6 +39,14 @@#include "pseries.h" +enum { +DDW_QUERY_PE_DMA_WIN =0 , +DDW_CREATE_PE_DMA_WIN =1 , +DDW_REMOVE_PE_DMA_WIN =2 , + +DDW_APPLICABLE_SIZE +}; +static struct iommu_table_group *iommu_pseries_alloc_group (int node) {struct iommu_table_group *table_group;@@ -771,12 +779,12 @@ static void remove_ddw(struct device_node *np, bool remove_prop) {struct dynamic_dma_window_prop *dwp;struct property *win64; -u32 ddw_avail[3 ]; +u32 ddw_avail[DDW_APPLICABLE_SIZE ];u64 liobn;int ret =0 ; ret =of_property_read_u32_array (np,"ibm,ddw-applicable" , - &ddw_avail[0 ],3 ); + &ddw_avail[0 ],DDW_APPLICABLE_SIZE );arch/powerpc/platforms/pseries/iommu.c +1
-1win64 =of_find_property (np,DIRECT64_PROPNAME ,NULL );if (!win64)@@ -798,15 +806,15 @@ static void remove_ddw(struct device_node *np, bool remove_prop) pr_debug("%pOF successfully cleared tces in window.\n", np); - ret =rtas_call (ddw_avail[2 ],1 ,1 ,NULL , liobn); + ret =rtas_call (ddw_avail[DDW_REMOVE_PE_DMA_WIN ],1 ,1 ,NULL , liobn);if (ret) pr_warn("%pOF: failed to remove direct window: rtasreturned "
arch/powerpc/platforms/pseries/iommu.c +1
-1"%d to ibm,remove-pe-dma-window(%x) %llx\n" , - np, ret, ddw_avail[2], liobn); + np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN],liobn);
arch/powerpc/platforms/pseries/iommu.c +0
-0else pr_debug("%pOF: successfully removed direct window:rtas returned "
arch/powerpc/platforms/pseries/iommu.c +1
-1"%d to ibm,remove-pe-dma-window(%x) %llx\n" , - np, ret, ddw_avail[2], liobn); + np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN],liobn);
delprop:
arch/powerpc/platforms/pseries/iommu.c +3
-4if (remove_prop)@@ -889,11 +897,11 @@ static int query_ddw(struct pci_dev *dev, constu32 *ddw_avail, buid = pdn->phb->buid; cfg_addr = ((pdn->busno <<16 ) | (pdn->devfn <<8 )); - ret =rtas_call (ddw_avail[0 ],3 ,5 , (u32 *)query, - cfg_addr,BUID_HI (buid),BUID_LO (buid)); + ret =rtas_call (ddw_avail[DDW_QUERY_PE_DMA_WIN ],3 ,5 , (u32 *)query, + cfg_addr,BUID_HI (buid),BUID_LO (buid)); dev_info(&dev->dev, "ibm,query-pe-dma-windows(%x) %x %x %x" -" returned %d\n" , ddw_avail[0 ], cfg_addr, BUID_HI(buid), -BUID_LO (buid), ret); + " returned %d\n", ddw_avail[DDW_QUERY_PE_DMA_WIN],
cfg_addr,arch/powerpc/platforms/pseries/iommu.c +4
-3+ BUID_HI(buid), BUID_LO(buid), ret);return ret; }@@ -920,15 +928,16 @@ static int create_ddw(struct pci_dev *dev,const u32 *ddw_avail, do {/* extra outputs are LIOBN and dma-addr (hi, lo) */ - ret =rtas_call (ddw_avail[1 ],5 ,4 , (u32 *)create, - cfg_addr,BUID_HI (buid),BUID_LO (buid), - page_shift, window_shift); + ret =rtas_call (ddw_avail[DDW_CREATE_PE_DMA_WIN ],5 ,4 , + (u32 *)create, cfg_addr,BUID_HI (buid), +BUID_LO (buid), page_shift, window_shift); }while (rtas_busy_delay (ret)); dev_info(&dev->dev, "ibm,create-pe-dma-window(%x) %x %x %x %x %x returned%d "
arch/powerpc/platforms/pseries/iommu.c +0
-3-"(liobn = 0x%x starting addr = %x %x)\n" , ddw_avail[1 ], - cfg_addr,BUID_HI (buid),BUID_LO (buid), page_shift, - window_shift, ret, create->liobn, create->addr_hi,
create->addr_lo);arch/powerpc/platforms/pseries/iommu.c +2
-0+"(liobn = 0x%x starting addr = %x %x)\n" , + ddw_avail[DDW_CREATE_PE_DMA_WIN], cfg_addr,
BUID_HI(buid),arch/powerpc/platforms/pseries/iommu.c +1
-0+ BUID_LO(buid), page_shift, window_shift, ret, create-Quoted 1 line
liobn,
arch/powerpc/platforms/pseries/iommu.c +2
-1+ create->addr_hi, create->addr_lo);return ret; }@@ -996,7 +1005,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)int page_shift;u64 dma_addr, max_addr;struct device_node *dn; -u32 ddw_avail[3 ]; +u32 ddw_avail[DDW_APPLICABLE_SIZE ];struct direct_window *window;struct property *win64;arch/powerpc/platforms/pseries/iommu.c +1
-1struct dynamic_dma_window_prop *ddwprop;@@ -1029,7 +1038,7 @@ static u64 enable_ddw(struct pci_dev *dev,struct device_node *pdn) * the property is actually in the parent, not the PE */ ret =of_property_read_u32_array (pdn,"ibm,ddw-applicable" , - &ddw_avail[0 ],3 ); + &ddw_avail[0 ],DDW_APPLICABLE_SIZE );if (ret)
Quoted 2 lines
goto
out_failed;
PATCH v4 2/7 powerpc/pseries/iommu: Update call to ibm,query-pe-dma-windows
In reply to Leonardo BrasFrom LoPAR level 2.8, "ibm,ddw-extensions" index 3 can make the number of outputs from "ibm,query-pe-dma-windows" go from 5 to 6.
This change of output size is meant to expand the address size of largest_available_block PE TCE from 32-bit to 64-bit, which ends up shifting page_size and migration_capable.
This ends up requiring the update of ddw_query_response->largest_available_block from u32 to u64, and manually assigning the values from the buffer into this struct, according to output size.
Also, a routine was created for helping reading the ddw extensions as suggested by LoPAR: First reading the size of the extension array from index 0, checking if the property exists, and then returning it's value.
--- arch/powerpc/platforms/pseries/iommu.c | 91 +++++++++++++++++++++++--- 1 file changed, 81 insertions(+), 10 deletions(-)
arch/powerpc/platforms/pseries/iommu.c +81 -10
@@ -47,6 +47,12 @@ enum { DDW_APPLICABLE_SIZE }; +enum { +DDW_EXT_SIZE =0 , +DDW_EXT_RESET_DMA_WIN =1 , +DDW_EXT_QUERY_OUT_SIZE =2 +}; +static struct iommu_table_group *iommu_pseries_alloc_group (int node) {struct iommu_table_group *table_group;
@@ -342,7 +348,7 @@ struct direct_window {/* Dynamic DMA Window support */ struct ddw_query_response {u32 windows_available; -u32 largest_available_block; +u64 largest_available_block;u32 page_size;u32 migration_capable; };
@@ -877,14 +883,62 @@ static int find_existing_ddw_windows(void) }machine_arch_initcall (pseries, find_existing_ddw_windows); +/** +* ddw_read_ext - Get the value of an DDW extension +* @np: device node from which the extension value is to be read. +* @extnum: index number of the extension. +* @value: pointer to return value, modified when extension is available. +* +* Checks if "ibm,ddw-extensions" exists for this node, and get the value +* on index 'extnum'. +* It can be used only to check if a property exists, passing value == NULL. +* +* Returns: +* 0 if extension successfully read +* -EINVAL if the "ibm,ddw-extensions" does not exist, +* -ENODATA if "ibm,ddw-extensions" does not have a value, and +* -EOVERFLOW if "ibm,ddw-extensions" does not contain this extension. +*/ +static inline int ddw_read_ext (const struct device_node *np,int extnum, +u32 *value) +{ +static const char propname[] ="ibm,ddw-extensions" ; +u32 count; +int ret; + + ret =of_property_read_u32_index (np, propname,DDW_EXT_SIZE , &count); +if (ret) +return ret; + +if (count < extnum) +return -EOVERFLOW ; + +if (!value) + value = &count; + +return of_property_read_u32_index (np, propname, extnum, value); +} +static int query_ddw (struct pci_dev *dev,const u32 *ddw_avail, -struct ddw_query_response *query) +struct ddw_query_response *query, +struct device_node *parent) {struct device_node *dn;struct pci_dn *pdn; -u32 cfg_addr; +u32 cfg_addr, ext_query, query_out[5 ];u64 buid; -int ret; +int ret, out_sz; + +/* +* From LoPAR level 2.8, "ibm,ddw-extensions" index 3 can rule how many +* output parameters ibm,query-pe-dma-windows will have, ranging from +* 5 to 6. +*/ + ret =ddw_read_ext (parent,DDW_EXT_QUERY_OUT_SIZE , &ext_query); +if (!ret && ext_query ==1 ) + out_sz =6 ; +else + out_sz =5 ;/* * Get the config address and phb buid of the PE window.
@@ -897,11 +951,28 @@ static int query_ddw(struct pci_dev *dev, const u32 *ddw_avail, buid = pdn->phb->buid; cfg_addr = ((pdn->busno <<16 ) | (pdn->devfn <<8 )); - ret =rtas_call (ddw_avail[DDW_QUERY_PE_DMA_WIN ],3 ,5 , (u32 *)query, + ret =rtas_call (ddw_avail[DDW_QUERY_PE_DMA_WIN ],3 , out_sz, query_out, cfg_addr,BUID_HI (buid),BUID_LO (buid)); -dev_info (&dev->dev,"ibm,query-pe-dma-windows(%x) %x %x %x" -" returned %d\n" , ddw_avail[DDW_QUERY_PE_DMA_WIN ], cfg_addr, -BUID_HI (buid),BUID_LO (buid), ret); +dev_info (&dev->dev,"ibm,query-pe-dma-windows(%x) %x %x %x returned %d\n" , + ddw_avail[DDW_QUERY_PE_DMA_WIN ], cfg_addr,BUID_HI (buid), +BUID_LO (buid), ret); + +switch (out_sz) { +case 5 : + query->windows_available = query_out[0 ]; + query->largest_available_block = query_out[1 ]; + query->page_size = query_out[2 ]; + query->migration_capable = query_out[3 ]; +break ; +case 6 : + query->windows_available = query_out[0 ]; + query->largest_available_block = ((u64 )query_out[1 ] <<32 ) | + query_out[2 ]; + query->page_size = query_out[3 ]; + query->migration_capable = query_out[4 ]; +break ; + } +return ret; }
@@ -1049,7 +1120,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)* of page sizes: supported and supported for migrate-dma. */ dn =pci_device_to_OF_node (dev); - ret =query_ddw (dev, ddw_avail, &query); + ret =query_ddw (dev, ddw_avail, &query, pdn);if (ret !=0 ) goto out_failed;
@@ -1077,7 +1148,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)/* check largest block * page size > max memory hotplug addr */ max_addr =ddw_memory_hotplug_max ();if (query.largest_available_block < (max_addr >> page_shift)) { -dev_dbg (&dev->dev,"can't map partition max 0x%llx with %u " +dev_dbg (&dev->dev,"can't map partition max 0x%llx with %llu " "%llu-sized pages\n" , max_addr, query.largest_available_block,1ULL << page_shift); goto out_failed;
Signature · 1 line
2.25.4
Re: [PATCH v4 2/7] powerpc/pseries/iommu: Update call to ibm, query-pe-dma-windows
Quoted 190 lines · Leonardo Bras
Quoted 2 lines
From LoPAR level 2.8, "ibm,ddw-extensions" index 3 can make the number of
outputs from "ibm,query-pe-dma-windows" go from 5 to 6.
This change of output size is meant to expand the address size of
largest_available_blockPE TCE from 32-bit to 64-bit, which ends up shiftingpage_sizeandmigration_capable.This ends up requiring the update of
ddw_query_response->largest_available_blockfrom u32 to u64, and manually
assigning the values from the buffer into this struct, according to output size.Also, a routine was created for helping reading the ddw extensions as suggested by LoPAR: First reading the size of the extension array from
index 0, checking if the property exists, and then returning it's value.Signed-off-by Leonardo Bras <[email protected]>--- arch/powerpc/platforms/pseries/iommu.c | 91 +++++++++++++++++++++++- -- 1 file changed, 81 insertions(+), 10 deletions(-)arch/powerpc/platforms/pseries/iommu.c +10
-1@@ -47,6 +47,12 @@ enum { DDW_APPLICABLE_SIZE }; +enum { +DDW_EXT_SIZE =0 , +DDW_EXT_RESET_DMA_WIN =1 , +DDW_EXT_QUERY_OUT_SIZE =2 +}; +static struct iommu_table_group *iommu_pseries_alloc_group (int node) {struct iommu_table_group *table_group;@@ -342,7 +348,7 @@ struct direct_window {/* Dynamic DMA Window support */ struct ddw_query_response {u32 windows_available; -u32 largest_available_block; +u64 largest_available_block;u32 page_size;u32 migration_capable; };@@ -877,14 +883,62 @@ static int find_existing_ddw_windows(void) }machine_arch_initcall (pseries, find_existing_ddw_windows); +/** +* ddw_read_ext - Get the value of an DDW extension +* @np: device node from which the extension value is to be read.
+ * @extnum: index number of the extension.
+ * @value: pointer to return value, modified when extension is
available.
+ *
+ * Checks if "ibm,ddw-extensions" exists for this node, and get the value
+ * on index 'extnum'.
+ * It can be used only to check if a property exists, passing value == NULL.
+ *
+ * Returns:
+ * 0 if extension successfully read
+ *-EINVALif the "ibm,ddw-extensions" does not exist,
+ *-ENODATAif "ibm,ddw-extensions" does not have a value, and
+ *-EOVERFLOWif "ibm,ddw-extensions" does not contain this
extension.
+ */
+static inline int ddw_read_ext(const structdevice_node*np, int extnum,arch/powerpc/platforms/pseries/iommu.c +7
-0+ u32 *value) +{ +static const char propname[] ="ibm,ddw-extensions" ; +u32 count; +int ret; + + ret = of_property_read_u32_index(np, propname, DDW_EXT_SIZE,&count);
arch/powerpc/platforms/pseries/iommu.c +19
-3+if (ret) +return ret; + +if (count < extnum) +return -EOVERFLOW ; + +if (!value) + value = &count; + +return of_property_read_u32_index (np, propname, extnum, value); +} +static int query_ddw (struct pci_dev *dev,const u32 *ddw_avail, -struct ddw_query_response *query) +struct ddw_query_response *query, +struct device_node *parent) {struct device_node *dn;struct pci_dn *pdn; -u32 cfg_addr; +u32 cfg_addr, ext_query, query_out[5 ];u64 buid; -int ret; +int ret, out_sz; + +/* +* From LoPAR level 2.8, "ibm,ddw-extensions" index 3 can rule how many
arch/powerpc/platforms/pseries/iommu.c +1
-0+* output parameters ibm,query-pe-dma-windows will have, ranging from
arch/powerpc/platforms/pseries/iommu.c +9
-4+* 5 to 6. +*/ + ret =ddw_read_ext (parent,DDW_EXT_QUERY_OUT_SIZE , &ext_query); +if (!ret && ext_query ==1 ) + out_sz =6 ; +else + out_sz =5 ;/* * Get the config address and phb buid of the PE window. @@ -897,11 +951,28 @@ static int query_ddw(struct pci_dev *dev, constu32 *ddw_avail, buid = pdn->phb->buid; cfg_addr = ((pdn->busno <<16 ) | (pdn->devfn <<8 )); - ret =rtas_call (ddw_avail[DDW_QUERY_PE_DMA_WIN ],3 ,5 , (u32 *)query, + ret =rtas_call (ddw_avail[DDW_QUERY_PE_DMA_WIN ],3 , out_sz, query_out, cfg_addr,BUID_HI (buid),BUID_LO (buid)); -dev_info (&dev->dev,"ibm,query-pe-dma-windows(%x) %x %x %x" -" returned %d\n" , ddw_avail[DDW_QUERY_PE_DMA_WIN ], cfg_addr, -BUID_HI (buid),BUID_LO (buid), ret); + dev_info(&dev->dev, "ibm,query-pe-dma-windows(%x) %x %x %xreturned %d\n",
arch/powerpc/platforms/pseries/iommu.c +1
-0+ ddw_avail[DDW_QUERY_PE_DMA_WIN], cfg_addr,
BUID_HI(buid),arch/powerpc/platforms/pseries/iommu.c +12
-0+ BUID_LO(buid), ret); + +switch (out_sz) { +case 5 : + query->windows_available = query_out[0 ]; + query->largest_available_block = query_out[1 ]; + query->page_size = query_out[2 ]; + query->migration_capable = query_out[3 ]; +break ; +case 6 : + query->windows_available = query_out[0 ]; + query->largest_available_block = ((u64)query_out[1] <<32) |
arch/powerpc/platforms/pseries/iommu.c +7
-2+ query_out[2 ]; + query->page_size = query_out[3 ]; + query->migration_capable = query_out[4 ]; +break ; + } +return ret; }@@ -1049,7 +1120,7 @@ static u64 enable_ddw(struct pci_dev *dev,struct device_node *pdn) * of page sizes: supported and supported for migrate-dma. */ dn =pci_device_to_OF_node (dev); - ret =query_ddw (dev, ddw_avail, &query); + ret =query_ddw (dev, ddw_avail, &query, pdn);if (ret !=0 ) goto out_failed;@@ -1077,7 +1148,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)/* check largest block * page size > max memory hotplug addr */ max_addr =ddw_memory_hotplug_max ();if (query.largest_available_block < (max_addr >> page_shift)) { - dev_dbg(&dev->dev, "can't map partition max 0x%llx with%u "
arch/powerpc/platforms/pseries/iommu.c +1
-0+ dev_dbg(&dev->dev, "can't map partition max 0x%llx with%llu "
"%llu-sized pages\n",max_addr, query.largest_available_block,
arch/powerpc/platforms/pseries/iommu.c static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) · line 1154
Quoted 2 lines
arch/powerpc/platforms/pseries/iommu.c +0
-01ULL << page_shift); goto out_failed;
PATCH v4 3/7 powerpc/pseries/iommu: Move window-removing part of remove_ddw into remove_dma_window
In reply to Leonardo BrasMove the window-removing part of remove_ddw into a new function (remove_dma_window), so it can be used to remove other DMA windows.
It's useful for removing DMA windows that don't create DIRECT64_PROPNAME property, like the default DMA window from the device, which uses "ibm,dma-window".
--- arch/powerpc/platforms/pseries/iommu.c | 45 +++++++++++++++----------- 1 file changed, 27 insertions(+), 18 deletions(-)
arch/powerpc/platforms/pseries/iommu.c +27 -18
@@ -781,25 +781,14 @@ static int __init disable_ddw_setup(char *str)early_param ("disable_ddw" , disable_ddw_setup); -static void remove_ddw (struct device_node *np,bool remove_prop) +static void remove_dma_window (struct device_node *np,u32 *ddw_avail, +struct property *win) {struct dynamic_dma_window_prop *dwp; -struct property *win64; -u32 ddw_avail[DDW_APPLICABLE_SIZE ];u64 liobn; -int ret =0 ; - - ret =of_property_read_u32_array (np,"ibm,ddw-applicable" , - &ddw_avail[0 ],DDW_APPLICABLE_SIZE ); - - win64 =of_find_property (np,DIRECT64_PROPNAME ,NULL ); -if (!win64) -return ; - -if (ret || win64->length <sizeof (*dwp)) - goto delprop; +int ret; - dwp = win64->value; + dwp = win->value; liobn = (u64 )be32_to_cpu (dwp->liobn);/* clear the whole window, note the arg is in kernel pages */
@@ -821,10 +810,30 @@ static void remove_ddw(struct device_node *np, bool remove_prop)pr_debug ("%pOF: successfully removed direct window: rtas returned " "%d to ibm,remove-pe-dma-window(%x) %llx\n" , np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN ], liobn); +} + +static void remove_ddw (struct device_node *np,bool remove_prop) +{ +struct property *win; +u32 ddw_avail[DDW_APPLICABLE_SIZE ]; +int ret =0 ; + + ret =of_property_read_u32_array (np,"ibm,ddw-applicable" , + &ddw_avail[0 ],DDW_APPLICABLE_SIZE ); +if (ret) +return ; + + win =of_find_property (np,DIRECT64_PROPNAME ,NULL ); +if (!win) +return ; + +if (win->length >=sizeof (struct dynamic_dma_window_prop )) +remove_dma_window (np, ddw_avail, win); + +if (!remove_prop) +return ; -delprop: -if (remove_prop) - ret =of_remove_property (np, win64); + ret =of_remove_property (np, win);if (ret)pr_warn ("%pOF: failed to remove direct window property: %d\n" , np, ret);
Signature · 1 line
2.25.4
Re: [PATCH v4 3/7] powerpc/pseries/iommu: Move window-removing part of remove_ddw into remove_dma_window
Quoted 88 lines · Leonardo Bras
Move the window-removing part of
remove_ddwinto a new function (remove_dma_window), so it can be used to remove other DMA windows.It's useful for removing DMA windows that don't create
DIRECT64_PROPNAME
property, like the default DMA window from the device, which uses "ibm,dma-window".Signed-off-by Leonardo Bras <[email protected]>--- arch/powerpc/platforms/pseries/iommu.c | 45 +++++++++++++++--------- -- 1 file changed, 27 insertions(+), 18 deletions(-)arch/powerpc/platforms/pseries/iommu.c +4
-15@@ -781,25 +781,14 @@ static int __init disable_ddw_setup(char *str)early_param ("disable_ddw" , disable_ddw_setup); -static void remove_ddw(struct device_node *np, bool remove_prop) +static void remove_dma_window (struct device_node *np,u32 *ddw_avail, +struct property *win) {struct dynamic_dma_window_prop *dwp; -struct property *win64; -u32 ddw_avail[DDW_APPLICABLE_SIZE ];u64 liobn; -int ret =0 ; - - ret =of_property_read_u32_array (np,"ibm,ddw-applicable" , - &ddw_avail[0 ], DDW_APPLICABLE_SIZE); - - win64 =of_find_property (np,DIRECT64_PROPNAME ,NULL ); -if (!win64) -return ; - -if (ret || win64->length <sizeof (*dwp)) - goto delprop; +int ret; - dwp = win64->value; + dwp = win->value; liobn = (u64 )be32_to_cpu (dwp->liobn);arch/powerpc/platforms/pseries/iommu.c +0
-0/* clear the whole window, note the arg is in kernel pages */ @@ -821,10 +810,30 @@ static void remove_ddw(struct device_node *np, bool remove_prop) pr_debug("%pOF: successfully removed direct window:rtas returned "
arch/powerpc/platforms/pseries/iommu.c +0
-0"%d to ibm,remove-pe-dma-window(%x) %llx\n" , np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN],liobn);
arch/powerpc/platforms/pseries/iommu.c +10
-0+} + +static void remove_ddw (struct device_node *np,bool remove_prop) +{ +struct property *win; +u32 ddw_avail[DDW_APPLICABLE_SIZE ]; +int ret =0 ; + + ret = of_property_read_u32_array(np, "ibm,ddw-applicable", + &ddw_avail[0],
DDW_APPLICABLE_SIZE);arch/powerpc/platforms/pseries/iommu.c +13
-3+if (ret) +return ; + + win =of_find_property (np,DIRECT64_PROPNAME ,NULL ); +if (!win) +return ; + +if (win->length >=sizeof (struct dynamic_dma_window_prop )) +remove_dma_window (np, ddw_avail, win); + +if (!remove_prop) +return ; -delprop: -if (remove_prop) - ret =of_remove_property (np, win64); + ret =of_remove_property (np, win);if (ret) pr_warn("%pOF: failed to remove direct window property:
Quoted 2 lines
%d\n",
np, ret);
PATCH v4 4/7 powerpc/pseries/iommu: Remove default DMA window before creating DDW
In reply to Leonardo BrasOn LoPAR "DMA Window Manipulation Calls", it's recommended to remove the default DMA window for the device, before attempting to configure a DDW, in order to make the maximum resources available for the next DDW to be created.
This is a requirement for using DDW on devices in which hypervisor allows only one DMA window.
If setting up a new DDW fails anywhere after the removal of this default DMA window, it's needed to restore the default DMA window. For this, an implementation of ibm,reset-pe-dma-windows rtas call is needed:
Platforms supporting the DDW option starting with LoPAR level 2.7 implement ibm,ddw-extensions. The first extension available (index 2) carries the token for ibm,reset-pe-dma-windows rtas call, which is used to restore the default DMA window for a device, if it has been deleted.
It does so by resetting the TCE table allocation for the PE to it's boot time value, available in "ibm,dma-window" device tree node.
--- arch/powerpc/platforms/pseries/iommu.c | 73 +++++++++++++++++++++++--- 1 file changed, 66 insertions(+), 7 deletions(-)
arch/powerpc/platforms/pseries/iommu.c +66 -7
@@ -1066,6 +1066,38 @@ static phys_addr_t ddw_memory_hotplug_max(void)return max_addr; } +/* +* Platforms supporting the DDW option starting with LoPAR level 2.7 implement +* ibm,ddw-extensions, which carries the rtas token for +* ibm,reset-pe-dma-windows. +* That rtas-call can be used to restore the default DMA window for the device. +*/ +static void reset_dma_window (struct pci_dev *dev,struct device_node *par_dn) +{ +int ret; +u32 cfg_addr, reset_dma_win; +u64 buid; +struct device_node *dn; +struct pci_dn *pdn; + + ret =ddw_read_ext (par_dn,DDW_EXT_RESET_DMA_WIN , &reset_dma_win); +if (ret) +return ; + + dn =pci_device_to_OF_node (dev); + pdn =PCI_DN (dn); + buid = pdn->phb->buid; + cfg_addr = ((pdn->busno <<16 ) | (pdn->devfn <<8 )); + + ret =rtas_call (reset_dma_win,3 ,1 ,NULL , cfg_addr,BUID_HI (buid), +BUID_LO (buid)); +if (ret) +dev_info (&dev->dev, +"ibm,reset-pe-dma-windows(%x) %x %x %x returned %d " , + reset_dma_win, cfg_addr,BUID_HI (buid),BUID_LO (buid), + ret); +} +/* * If the PE supports dynamic dma windows, and there is space for a table * that can map all pages in a linear offset, then setup such a table,
@@ -1090,6 +1122,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)struct property *win64;struct dynamic_dma_window_prop *ddwprop;struct failed_ddw_pdn *fpdn; +bool default_win_removed = false;mutex_lock (&direct_window_init_mutex);
@@ -1133,14 +1166,38 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)if (ret !=0 ) goto out_failed; +/* +* If there is no window available, remove the default DMA window, +* if it's present. This will make all the resources available to the +* new DDW window. +* If anything fails after this, we need to restore it, so also check +* for extensions presence. +*/ if (query.windows_available ==0 ) { -/* -* no additional windows are available for this device. -* We might be able to reallocate the existing window, -* trading in for a larger page size. -*/ -dev_dbg (&dev->dev,"no free dynamic windows" ); - goto out_failed; +struct property *default_win; +int reset_win_ext; + + default_win =of_find_property (pdn,"ibm,dma-window" ,NULL ); +if (!default_win) + goto out_failed; + + reset_win_ext =ddw_read_ext (pdn,DDW_EXT_RESET_DMA_WIN ,NULL ); +if (reset_win_ext) + goto out_failed; + +remove_dma_window (pdn, ddw_avail, default_win); + default_win_removed = true; + +/* Query again, to check if the window is available */ + ret =query_ddw (dev, ddw_avail, &query, pdn); +if (ret !=0 ) + goto out_failed; + +if (query.windows_available ==0 ) { +/* no windows are available for this device. */ +dev_dbg (&dev->dev,"no free dynamic windows" ); + goto out_failed; + } }if (query.page_size &4 ) { page_shift =24 ;/* 16MB */
@@ -1231,6 +1288,8 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)kfree (win64); out_failed: +if (default_win_removed) +reset_dma_window (dev, pdn); fpdn =kzalloc (sizeof (*fpdn),GFP_KERNEL );if (!fpdn)
Signature · 1 line
2.25.4
Re: [PATCH v4 4/7] powerpc/pseries/iommu: Remove default DMA window before creating DDW
Quoted 157 lines · Leonardo Bras
On LoPAR "DMA Window Manipulation Calls", it's recommended to remove the
default DMA window for the device, before attempting to configure a DDW,
in order to make the maximum resources available for the next DDW to be
created.This is a requirement for using DDW on devices in which hypervisor allows only one DMA window.
If setting up a new DDW fails anywhere after the removal of this default DMA window, it's needed to restore the default DMA window. For this, an implementation of ibm,reset-pe-dma-windows rtas call is needed:
Platforms supporting the DDW option starting with LoPAR level 2.7 implement
ibm,ddw-extensions. The first extension available (index 2) carries the
token for ibm,reset-pe-dma-windows rtas call, which is used to restore
the default DMA window for a device, if it has been deleted.It does so by resetting the TCE table allocation for the PE to it's boot time value, available in "ibm,dma-window" device tree node.
Signed-off-by Leonardo Bras <[email protected]>--- arch/powerpc/platforms/pseries/iommu.c | 73 +++++++++++++++++++++++- -- 1 file changed, 66 insertions(+), 7 deletions(-)arch/powerpc/platforms/pseries/iommu.c +2
-0@@ -1066,6 +1066,38 @@ static phys_addr_tddw_memory_hotplug_max (void ) return max_addr; } +/* +* Platforms supporting the DDW option starting with LoPAR level 2.7 implement
+ * ibm,ddw-extensions, which carries the rtas token for
+ * ibm,reset-pe-dma-windows.
+ * That rtas-call can be used to restore the default DMA window for the device.
+ */
+static voidreset_dma_window(struct pci_dev *dev, struct device_node *par_dn)arch/powerpc/platforms/pseries/iommu.c +8
-0+{ +int ret; +u32 cfg_addr, reset_dma_win; +u64 buid; +struct device_node *dn; +struct pci_dn *pdn; + + ret = ddw_read_ext(par_dn, DDW_EXT_RESET_DMA_WIN,
&reset_dma_win);arch/powerpc/platforms/pseries/iommu.c +9
-0+if (ret) +return ; + + dn =pci_device_to_OF_node (dev); + pdn =PCI_DN (dn); + buid = pdn->phb->buid; + cfg_addr = ((pdn->busno <<16 ) | (pdn->devfn <<8 )); + + ret = rtas_call(reset_dma_win, 3, 1, NULL, cfg_addr,
BUID_HI(buid),arch/powerpc/platforms/pseries/iommu.c +4
-0+ BUID_LO(buid)); +if (ret) + dev_info(&dev->dev, + "ibm,reset-pe-dma-windows(%x) %x %x %xreturned %d ",
arch/powerpc/platforms/pseries/iommu.c +1
-0+ reset_dma_win, cfg_addr, BUID_HI(buid),
BUID_LO(buid),arch/powerpc/platforms/pseries/iommu.c +3
-0+ ret); +} +/* * If the PE supports dynamic dma windows, and there is space for a table
* that can map all pages in a linear offset, then setup such a
table,arch/powerpc/platforms/pseries/iommu.c +4
-0@@ -1090,6 +1122,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)struct property *win64;struct dynamic_dma_window_prop *ddwprop;struct failed_ddw_pdn *fpdn; +bool default_win_removed = false;mutex_lock (&direct_window_init_mutex);@@ -1133,14 +1166,38 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)if (ret !=0 ) goto out_failed; +/* +* If there is no window available, remove the default DMA window, +* if it's present. This will make all the resources available to the
arch/powerpc/platforms/pseries/iommu.c +2
-0+* new DDW window. +* If anything fails after this, we need to restore it, so also check
arch/powerpc/platforms/pseries/iommu.c +6
-7+* for extensions presence. +*/ if (query.windows_available ==0 ) { -/* -* no additional windows are available for this device. -* We might be able to reallocate the existing window, -* trading in for a larger page size. -*/ -dev_dbg (&dev->dev,"no free dynamic windows" ); - goto out_failed; +struct property *default_win; +int reset_win_ext; + + default_win = of_find_property(pdn, "ibm,dma-window",NULL);
arch/powerpc/platforms/pseries/iommu.c +4
-0+if (!default_win) + goto out_failed; + + reset_win_ext = ddw_read_ext(pdn,
DDW_EXT_RESET_DMA_WIN, NULL);arch/powerpc/platforms/pseries/iommu.c +18
-0+if (reset_win_ext) + goto out_failed; + +remove_dma_window (pdn, ddw_avail, default_win); + default_win_removed = true; + +/* Query again, to check if the window is available */ + ret =query_ddw (dev, ddw_avail, &query, pdn); +if (ret !=0 ) + goto out_failed; + +if (query.windows_available ==0 ) { +/* no windows are available for this device. */ +dev_dbg (&dev->dev,"no free dynamic windows" ); + goto out_failed; + } }if (query.page_size &4 ) { page_shift =24 ;/* 16MB */ @@ -1231,6 +1288,8 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)kfree (win64 ); out_failed: +if (default_win_removed) +reset_dma_window (dev, pdn); fpdn =kzalloc (sizeof (*fpdn),GFP_KERNEL );
Quoted 1 line
if (!fpdn)
PATCH v4 5/7 powerpc/iommu: Move iommu_table cleaning routine to iommu_table_clean
In reply to Leonardo BrasMove the part of iommu_table_free() that does struct iommu_table cleaning into iommu_table_clean, so we can invoke it separately.
This new function is useful for cleaning struct iommu_table before initializing it again with a new DMA window, without having it freed and allocated again.
--- arch/powerpc/kernel/iommu.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-)
arch/powerpc/kernel/iommu.c +18 -12
@@ -735,21 +735,10 @@ struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid,return tbl; } -static void iommu_table_free (struct kref *kref) +static void iommu_table_clean (struct iommu_table *tbl) {unsigned long bitmap_sz;unsigned int order; -struct iommu_table *tbl; - - tbl = container_of(kref, struct iommu_table, it_kref); - -if (tbl->it_ops->free) - tbl->it_ops->free (tbl); - -if (!tbl->it_map) { -kfree (tbl); -return ; - }iommu_table_release_pages (tbl);
@@ -763,6 +752,23 @@ static void iommu_table_free(struct kref *kref)/* free bitmap */ order =get_order (bitmap_sz);free_pages ((unsigned long ) tbl->it_map, order); +} + +static void iommu_table_free (struct kref *kref) +{ +struct iommu_table *tbl; + + tbl = container_of(kref, struct iommu_table, it_kref); + +if (tbl->it_ops->free) + tbl->it_ops->free (tbl); + +if (!tbl->it_map) { +kfree (tbl); +return ; + } + +iommu_table_clean (tbl);/* free table */ kfree (tbl);
Signature · 1 line
2.25.4
Re: [PATCH v4 5/7] powerpc/iommu: Move iommu_table cleaning routine to iommu_table_clean
Quoted 17 lines · Leonardo Bras
Move the part of
iommu_table_free()that does structiommu_tablecleaning intoiommu_table_clean, so we can invoke it separately.This new function is useful for cleaning struct
iommu_tablebefore initializing it again with a new DMA window, without having it freed and allocated again.Signed-off-by Leonardo Bras <[email protected]>--- arch/powerpc/kernel/iommu.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-)arch/powerpc/kernel/iommu.c +0
-0
arch/powerpc/kernel/iommu.c struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid, · line 738
Quoted 5 lines
arch/powerpc/kernel/iommu.c +1
-1@@ -735,21 +735,10 @@ struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid,return tbl; } -static void iommu_table_free (struct kref *kref) +static void iommu_table_clean (struct iommu_table *tbl)
iommu_table_free() + iommu_init_table() + set_iommu_table_base() should work too, why new helper?
There is also iommu_table_clear() which does a different thing so you need a better name.
Second, iommu_table_free() would print a warning if any IOMMU page is in use and it would be ok as we would only see this when hot-unplugging a PE because we always kept the default window.
Btw you must be seeing these warnings now every time you create DDW with these patches as at least the first page is reserved, do not you?
Since we are replacing a table for a device which is still in the system, we should not try messing with its DMA if it already has mappings so the warning should become an error preventing DDW. It is rather hard to trigger in practice but I could hack a driver to ask for 32bit DMA mask first, map few pages and then ask for 64bit DMA mask, it is not illegal, I think. So this needs a new helper - "bool iommu_table_in_use(tbl)" - to use in enable_ddw(). Or I am overthinking this?... Thanks,
Quoted 42 lines
arch/powerpc/kernel/iommu.c +17
-11{unsigned long bitmap_sz;unsigned int order; -struct iommu_table *tbl; - - tbl = container_of(kref, struct iommu_table, it_kref); - -if (tbl->it_ops->free) - tbl->it_ops->free (tbl); - -if (!tbl->it_map) { -kfree (tbl); -return ; - }iommu_table_release_pages (tbl);@@ -763,6 +752,23 @@ static void iommu_table_free(struct kref *kref)/* free bitmap */ order =get_order (bitmap_sz);free_pages ((unsigned long ) tbl->it_map, order); +} + +static void iommu_table_free (struct kref *kref) +{ +struct iommu_table *tbl; + + tbl = container_of(kref, struct iommu_table, it_kref); + +if (tbl->it_ops->free) + tbl->it_ops->free (tbl); + +if (!tbl->it_map) { +kfree (tbl); +return ; + } + +iommu_table_clean (tbl);/* free table */ kfree (tbl);
Signature · 1 line
Alexey
Re: [PATCH v4 5/7] powerpc/iommu: Move iommu_table cleaning routine to iommu_table_clean
Quoted 22 lines · Alexey Kardashevskiy
Quoted 20 lines · Leonardo Bras
Move the part of
iommu_table_free()that does structiommu_tablecleaning intoiommu_table_clean, so we can invoke it separately.This new function is useful for cleaning struct
iommu_tablebefore initializing it again with a new DMA window, without having it freed and allocated again.Signed-off-by Leonardo Bras <[email protected]>--- arch/powerpc/kernel/iommu.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-)arch/powerpc/kernel/iommu.c +0
-0@@ -735,21 +735,10 @@ struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid,return tbl; }
Quoted 5 lines
arch/powerpc/kernel/iommu.c struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid, · line 738
Quoted 2 lines
arch/powerpc/kernel/iommu.c +1
-1-static void iommu_table_free (struct kref *kref) +static void iommu_table_clean (struct iommu_table *tbl)
iommu_table_free()+iommu_init_table()+set_iommu_table_base()should work too, why new helper?
iommu_table_free() also frees the tbl, which would need allocate it again (new address) and to fill it up again, unnecessarily. I think it's a better approach to only change what is needed.
Quoted 2 lines
There is also
iommu_table_clear()which does a different thing so you need a better name.
I agree.
I had not noticed this other function before sending the patchset. What would be a better name though? __iommu_table_free()?
Quoted 5 lines
Second,
iommu_table_free
use and it would be ok as we would only see this when hot-unplugging a PE because we always kept the default window.
Btw you must be seeing these warnings now every time you create DDW with these patches as at least the first page is reserved, do not you?
It does not print a warning.
I noticed other warnings, but not this one from iommu_table_free():
/* verify that table contains no entries */
if (!bitmap_empty(tbl->it_ma
p, tbl->it_size))
pr_warn("%s: Unexpected TCEs\n", __func__);Before that, iommu_table_release_pages(tbl) is supposed to clear the bitmap, so this only tests for a tce that is created in this short period.
Quoted 8 lines
Since we are replacing a table for a device which is still in the system, we should not try messing with its DMA if it already has mappings so the warning should become an error preventing DDW. It is rather hard to trigger in practice but I could hack a driver to ask for 32bit DMA mask first, map few pages and then ask for 64bit DMA mask, it is not illegal, I think. So this needs a new helper - "bool
iommu_table_in_use(tbl)" - to use inenable_ddw(). Or I am overthinking this?... Thanks,
As of today, there seems to be nothing like that happening in the driver I am testing.
I spoke to Brian King on slack, and he mentioned that at the point DDW is created there should be no allocations in place.
But I suppose some driver could try to do this.
Maybe a better approach would be removing the mapping only if the default window is removed (at the end of enable_ddw, as an else to resetting the default DMA window), and having a way to add more mappings to those pools. But this last part doesn't look so simple, and it would be better to understand if it's necessary investing work in this.
What do you think?
Best regards,
Re: [PATCH v4 5/7] powerpc/iommu: Move iommu_table cleaning routine to iommu_table_clean
Quoted 65 lines · Leonardo Bras
Quoted 22 lines · Alexey Kardashevskiy
Quoted 20 lines · Leonardo Bras
Move the part of
iommu_table_free()that does structiommu_tablecleaning intoiommu_table_clean, so we can invoke it separately.This new function is useful for cleaning struct
iommu_tablebefore initializing it again with a new DMA window, without having it freed and allocated again.Signed-off-by Leonardo Bras <[email protected]>--- arch/powerpc/kernel/iommu.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-)arch/powerpc/kernel/iommu.c +0
-0@@ -735,21 +735,10 @@ struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid,return tbl; }Quoted 5 lines
arch/powerpc/kernel/iommu.c struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid, · line 738
Quoted 2 lines
arch/powerpc/kernel/iommu.c +1
-1-static void iommu_table_free (struct kref *kref) +static void iommu_table_clean (struct iommu_table *tbl)
iommu_table_free()+iommu_init_table()+set_iommu_table_base()should work too, why new helper?
iommu_table_free()also frees the tbl, which would need allocate it again (new address) and to fill it up again, unnecessarily. I think it's a better approach to only change what is needed.Quoted 2 lines
There is also
iommu_table_clear()which does a different thing so you need a better name.I agree.
I had not noticed this other function before sending the patchset. What would be a better name though?__iommu_table_free()?Quoted 5 lines
Second,
iommu_table_free
use and it would be ok as we would only see this when hot-unplugging a PE because we always kept the default window.
Btw you must be seeing these warnings now every time you create DDW with these patches as at least the first page is reserved, do not you?It does not print a warning.
I noticed other warnings, but not this one fromiommu_table_free():
/* verify that table contains no entries */
if (!bitmap_empty(tbl->it_ma
p,tbl->it_size))pr_warn("%s: Unexpected TCEs\n", __func__);Before that,
iommu_table_release_pages(tbl)is supposed to clear the bitmap, so this only tests for a tce that is created in this short period.Quoted 8 lines
Since we are replacing a table for a device which is still in the system, we should not try messing with its DMA if it already has mappings so the warning should become an error preventing DDW. It is rather hard to trigger in practice but I could hack a driver to ask for 32bit DMA mask first, map few pages and then ask for 64bit DMA mask, it is not illegal, I think. So this needs a new helper - "bool
iommu_table_in_use(tbl)" - to use inenable_ddw(). Or I am overthinking this?... Thanks,
Quoted 4 lines
As of today, there seems to be nothing like that happening in the driver I am testing.
I spoke to Brian King on slack, and he mentioned that at the point DDW is created there should be no allocations in place.
I think there are a couple of scenarios here. One is where there is a DMA allocation prior to a call to set the DMA mask. Second scenario is if the driver makes multiple calls to set the DMA mask. I would argue that a properly written driver should tell the IOMMU subsystem what DMA mask it supports prior to allocating DMA memroy. Documentation/core-api/dma-api-howto.rst should describe what is legal and what is not.
It might be reasonable to declare its not allowed to allocate DMA memory and then later change the DMA mask and clearly call this out in the documentation if its not already.
-Brian
Signature · 3 lines
Brian King Power Linux I/O IBM Linux Technology Center
Re: [PATCH v4 5/7] powerpc/iommu: Move iommu_table cleaning routine to iommu_table_clean
Quoted 13 lines · Brian King
Quoted 5 lines
As of today, there seems to be nothing like that happening in the driver I am testing.
I spoke to Brian King on slack, and he mentioned that at the point DDW is created there should be no allocations in place.I think there are a couple of scenarios here. One is where there is a DMA allocation prior to a call to set the DMA mask. Second scenario is if the driver makes multiple calls to set the DMA mask. I would argue that a properly written driver should tell the IOMMU subsystem what DMA mask it supports prior to allocating DMA memroy.
Documentation/core-api/dma-api-howto.rstshould describe what is legal and what is not.
Quoted 5 lines
It might be reasonable to declare its not allowed to allocate DMA memory and then later change the DMA mask and clearly call this out in the documentation if its not already.
-Brian
Thank you for the feedback Brian!
That makes sense to me. I will try to have this in mind for the next patchset.
Best regards,
Re: [PATCH v4 5/7] powerpc/iommu: Move iommu_table cleaning routine to iommu_table_clean
In reply to Leonardo BrasQuoted 23 lines · Leonardo Bras
Quoted 22 lines · Alexey Kardashevskiy
Quoted 20 lines · Leonardo Bras
Move the part of
iommu_table_free()that does structiommu_tablecleaning intoiommu_table_clean, so we can invoke it separately.This new function is useful for cleaning struct
iommu_tablebefore initializing it again with a new DMA window, without having it freed and allocated again.Signed-off-by Leonardo Bras <[email protected]>--- arch/powerpc/kernel/iommu.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-)arch/powerpc/kernel/iommu.c +0
-0@@ -735,21 +735,10 @@ struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid,return tbl; }
Quoted 8 lines
Quoted 5 lines
arch/powerpc/kernel/iommu.c struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid, · line 738
Quoted 2 lines
arch/powerpc/kernel/iommu.c +1
-1-static void iommu_table_free (struct kref *kref) +static void iommu_table_clean (struct iommu_table *tbl)
iommu_table_free()+iommu_init_table()+set_iommu_table_base()should work too, why new helper?
iommu_table_free()also frees the tbl, which would need allocate it again (new address) and to fill it up again, unnecessarily.
It is a new table in fact, everything is new there. You are only saving kfree+kzalloc which does not seem a huge win.
Also, iommu_table_update() simply assumes 64bit window by passing res_start=res_end=0 to iommu_init_table() which is not horribly robust either. Yeah, I know, iommu_init_table() is always called with zeroes in pseries but this is somewhat ok as those tables are from the device tree and those windows don't overlap with 32bit MMIO but under KVM they will (well, if we hack QEMU to advertise a single window).
I suggest removing iommu_pseries_table_update() from 6/7 and do iommu_table_free() + iommu_init_table() + set_iommu_table_base() with a WARN_ON(pdev->dev.archdata.dma_offset>=SZ_4G), may be even do this all in enable_ddw() where we know for sure if it is 1:1 mapping or just a big window.
Out of curiosity - what page sizes does pHyp advertise in "query"?
Quoted 9 lines
I think it's a better approach to only change what is needed.
Quoted 2 lines
There is also
iommu_table_clear()which does a different thing so you need a better name.I agree.
I had not noticed this other function before sending the patchset. What would be a better name though?__iommu_table_free()?
Quoted 8 lines
Quoted 5 lines
Second,
iommu_table_free
use and it would be ok as we would only see this when hot-unplugging a PE because we always kept the default window.
Btw you must be seeing these warnings now every time you create DDW with these patches as at least the first page is reserved, do not you?It does not print a warning.
I noticed other warnings,
And what are these?
Quoted 8 lines
but not this one from
iommu_table_free():
/* verify that table contains no entries */
if (!bitmap_empty(tbl->it_ma
p,tbl->it_size))pr_warn("%s: Unexpected TCEs\n", __func__);Before that,
iommu_table_release_pages(tbl)is supposed to clear the bitmap, so this only tests for a tce that is created in this short period.
iommu_table_release_pages() only clears reserved pages - page 0 (just a protection against NULL DMA pointers) and 32bit MMIO (these should not be set for 64bit window). The "%s: Unexpected TCEs\n" is what checks for actual mapped TCEs.
Quoted 9 lines
Quoted 8 lines
Since we are replacing a table for a device which is still in the system, we should not try messing with its DMA if it already has mappings so the warning should become an error preventing DDW. It is rather hard to trigger in practice but I could hack a driver to ask for 32bit DMA mask first, map few pages and then ask for 64bit DMA mask, it is not illegal, I think. So this needs a new helper - "bool
iommu_table_in_use(tbl)" - to use inenable_ddw(). Or I am overthinking this?... Thanks,
Quoted 4 lines
As of today, there seems to be nothing like that happening in the driver I am testing.
I spoke to Brian King on slack, and he mentioned that at the point DDW is created there should be no allocations in place.
Correct, there should not be. But it is also not a huge effort to fall back if there are.
Quoted 10 lines
But I suppose some driver could try to do this.
Maybe a better approach would be removing the mapping only if the default window is removed (at the end of
enable_ddw, as an else to resetting the default DMA window), and having a way to add more mappings to those pools. But this last part doesn't look so simple, and it would be better to understand if it's necessary investing work in this.What do you think?
Add iommu_table_in_use(tbl) and fail DDW if that says "yes".
Signature · 1 line
Alexey
Re: [PATCH v4 5/7] powerpc/iommu: Move iommu_table cleaning routine to iommu_table_clean
Quoted 43 lines · Alexey Kardashevskiy
Quoted 22 lines · Leonardo Bras
Quoted 21 lines · Alexey Kardashevskiy
Quoted 20 lines · Leonardo Bras
Move the part of
iommu_table_free()that does structiommu_tablecleaning intoiommu_table_clean, so we can invoke it separately.This new function is useful for cleaning struct
iommu_tablebefore initializing it again with a new DMA window, without having it freed and allocated again.Signed-off-by Leonardo Bras <[email protected]>--- arch/powerpc/kernel/iommu.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-)arch/powerpc/kernel/iommu.c +0
-0@@ -735,21 +735,10 @@ struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid,return tbl; }Quoted 8 lines
Quoted 5 lines
arch/powerpc/kernel/iommu.c struct iommu_table *iommu_init_table(struct iommu_table *tbl, int nid, · line 738
Quoted 2 lines
arch/powerpc/kernel/iommu.c +1
-1-static void iommu_table_free (struct kref *kref) +static void iommu_table_clean (struct iommu_table *tbl)
iommu_table_free()+iommu_init_table()+set_iommu_table_base()should work too, why new helper?
iommu_table_free()also frees the tbl, which would need allocate it again (new address) and to fill it up again, unnecessarily.It is a new table in fact, everything is new there. You are only saving kfree+kzalloc which does not seem a huge win.
Also,
iommu_table_update()simply assumes 64bit window by passing res_start=res_end=0 toiommu_init_table()which is not horribly robust either. Yeah, I know,iommu_init_table()is always called with zeroes in pseries but this is somewhat ok as those tables are from the device tree and those windows don't overlap with 32bit MMIO but under KVM they will (well, if we hack QEMU to advertise a single window).
Quoted 5 lines
I suggest removing
iommu_pseries_table_update()from 6/7 and doiommu_table_free()+iommu_init_table()+set_iommu_table_base()with aWARN_ON(pdev->dev.archdata.dma_offset>=SZ_4G), may be even do this all inenable_ddw()where we know for sure if it is 1:1 mapping or just a big window.
Sure, I have yet to understand the full impact of this change, but I will implement this and give it a try.
Quoted 2 lines
Out of curiosity - what page sizes does pHyp advertise in "query"?
64kB (page shift 0x10)
Quoted 17 lines
Quoted 17 lines
I think it's a better approach to only change what is needed.
Quoted 2 lines
There is also
iommu_table_clear()which does a different thing so you need a better name.I agree.
I had not noticed this other function before sending the patchset. What would be a better name though?__iommu_table_free()?Quoted 5 lines
Second,
iommu_table_free
use and it would be ok as we would only see this when hot-unplugging a PE because we always kept the default window.
Btw you must be seeing these warnings now every time you create DDW with these patches as at least the first page is reserved, do not you?
Quoted 4 lines
Quoted 2 lines
It does not print a warning.
I noticed other warnings,And what are these?
tce_freemulti_pSeriesLP: plpar_tce_stuff failed
[...]
It's regarding the change in pagesize.
Some places have the tceshift hardcoded as 12, tce_freemulti_pSeriesLP is one of them, and that is causing some errors.
I wrote a patch fixing this, and I will include it in the next series.
Quoted 7 lines
Quoted 7 lines
but not this one from
iommu_table_free():
/* verify that table contains no entries */
if (!bitmap_empty(tbl->it_ma
p,tbl->it_size))pr_warn("%s: Unexpected TCEs\n", __func__);
Quoted 8 lines
Quoted 2 lines
Before that,
iommu_table_release_pages(tbl)is supposed to clear the bitmap, so this only tests for a tce that is created in this short period.
iommu_table_release_pages()only clears reserved pages - page 0 (just a protection against NULL DMA pointers) and 32bit MMIO (these should not be set for 64bit window). The "%s: Unexpected TCEs\n" is what checks for actual mapped TCEs.
Oh, I haven't noticed that. Thanks for pointing!
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Since we are replacing a table for a device which is still in the system, we should not try messing with its DMA if it already has mappings so the warning should become an error preventing DDW. It is rather hard to trigger in practice but I could hack a driver to ask for 32bit DMA mask first, map few pages and then ask for 64bit DMA mask, it is not illegal, I think. So this needs a new helper - "bool
iommu_table_in_use(tbl)" - to use inenable_ddw(). Or I am overthinking this?... Thanks,
Quoted 7 lines
Quoted 4 lines
As of today, there seems to be nothing like that happening in the driver I am testing.
I spoke to Brian King on slack, and he mentioned that at the point DDW is created there should be no allocations in place.Correct, there should not be. But it is also not a huge effort to fall back if there are.
True.
Quoted 10 lines
Quoted 10 lines
But I suppose some driver could try to do this.
Maybe a better approach would be removing the mapping only if the default window is removed (at the end of
enable_ddw, as an else to resetting the default DMA window), and having a way to add more mappings to those pools. But this last part doesn't look so simple, and it would be better to understand if it's necessary investing work in this.
Quoted 3 lines
Quoted 1 line
What do you think?
Add
iommu_table_in_use(tbl)and fail DDW if that says "yes".
Seems good, I will include that on the next patchset.
Still, I will try to implement that more complex approach in a future patchset, as it may come to be useful.
Thank you for the feedback!
PATCH v4 6/7 powerpc/pseries/iommu: Make use of DDW even if it does not map the partition
In reply to Leonardo BrasAs of today, if the biggest DDW that can be created can't map the whole partition, it's creation is skipped and the default DMA window "ibm,dma-window" is used instead.
Usually this DDW is bigger than the default DMA window, and it performs better, so it would be nice to use it instead.
The DDW created will be used for direct mapping by default. If it's not available, indirect mapping will be used instead.
In this case, it's necessary to update the iommu_table so iommu_alloc() can use the DDW created. For this, iommu_table_update() is called after a enable_ddw() when direct DMA is not available.
As there will never have both direct and indirect mappings at the same time, the same property name can be used for the created DDW.
So renaming
define DIRECT64_PROPNAME "linux,direct64-ddr-window-info" to
define DMA64_PROPNAME "linux,dma64-ddr-window-info" looks the right thing to do.
--- arch/powerpc/include/asm/iommu.h | 3 ++ arch/powerpc/kernel/iommu.c | 15 +++++++++ arch/powerpc/platforms/pseries/iommu.c | 46 +++++++++++++++++++------- 3 files changed, 52 insertions(+), 12 deletions(-)
arch/powerpc/include/asm/iommu.h +3 -0
@@ -154,6 +154,9 @@ extern int iommu_tce_table_put(struct iommu_table *tbl);*/ extern struct iommu_table *iommu_init_table (struct iommu_table *tbl,int nid,unsigned long res_start,unsigned long res_end); +void iommu_table_update (struct iommu_table *tbl,int nid,unsigned long liobn, +unsigned long win_addr,unsigned long page_shift, +unsigned long window_shift);#define IOMMU_TABLE_GROUP_MAX_TABLES 2
arch/powerpc/kernel/iommu.c +15 -0
@@ -774,6 +774,21 @@ static void iommu_table_free(struct kref *kref)kfree (tbl); } +void iommu_table_update (struct iommu_table *tbl,int nid,unsigned long liobn, +unsigned long win_addr,unsigned long page_shift, +unsigned long window_shift) +{ +iommu_table_clean (tbl); + +/* Update tlb with values from ddw */ + tbl->it_index = liobn; + tbl->it_offset = win_addr >> page_shift; + tbl->it_page_shift = page_shift; + tbl->it_size =1 << (window_shift - page_shift); + +iommu_init_table (tbl, nid,0 ,0 ); +} +struct iommu_table *iommu_tce_table_get (struct iommu_table *tbl) {if (kref_get_unless_zero (&tbl->it_kref))
arch/powerpc/platforms/pseries/iommu.c +34 -12
98 lines not on this page · raw message
Signature · 1 line
2.25.4
Re: [PATCH v4 6/7] powerpc/pseries/iommu: Make use of DDW even if it does not map the partition
Quoted 20 lines · Leonardo Bras
arch/powerpc/platforms/pseries/iommu.c +16
-0+static void iommu_pseries_table_update (struct pci_dev *dev, +struct device_node *pdn) +{ +const struct dynamic_dma_window_prop *ddw; +struct pci_dn *pci; +int len; + + ddw =of_get_property (pdn,DMA64_PROPNAME , &len); +if (!ddw || len <sizeof (struct dynamic_dma_window_prop )) +return ; + +iommu_table_update (pci->table_group->tables[0 ], pci->phb->node, + ddw->liobn, ddw->dma_base, ddw->tce_shift, + ddw->window_shift); +} +static void pci_dma_dev_setup_pSeriesLP (struct pci_dev *dev) {struct device_node *pdn, *dn;
arch/powerpc/platforms/pseries/iommu.c static bool iommu_bypass_supported_pSeriesLP(struct pci_dev *pdev, u64 dma_mask) · line 1408
Quoted 6 lines
arch/powerpc/platforms/pseries/iommu.c +1
-0@@ -1382,6 +1403,7 @@ static bool iommu_bypass_supported_pSeriesLP(struct pci_dev *pdev, u64 dma_mask) pdev->dev.archdata.dma_offset =enable_ddw (pdev, pdn);if (pdev->dev.archdata.dma_offset)return true; +iommu_pseries_table_update (pdev, pdn); }
Noticed a bug in this one: pci is not getting assigned. My bad, there must have been a merge error.
Also, I will refactor the function to make use of pdn only, as I can do pci = PCI_DN(pdn) (I think it's better this way).
Sorry for the buggy patch.
Best regards,
PATCH v4 7/7 powerpc/pseries/iommu: Rename "direct window" to "dma window"
In reply to Leonardo BrasA previous change introduced the usage of DDW as a bigger indirect DMA mapping when the DDW available size does not map the whole partition.
As most of the code that manipulates direct mappings was reused for indirect mappings, it's necessary to rename all names and debug/info messages to reflect that it can be used for both kinds of mapping.
Also, defines DEFAULT_DMA_WIN as "ibm,dma-window" to document that it's the name of the default DMA window.
Those changes are not supposed to change how the code works in any way, just adjust naming.
--- arch/powerpc/platforms/pseries/iommu.c | 100 +++++++++++++------------ 1 file changed, 52 insertions(+), 48 deletions(-)
arch/powerpc/platforms/pseries/iommu.c +52 -48
@@ -339,7 +339,7 @@ struct dynamic_dma_window_prop {__be32 window_shift;/* ilog2(tce_window_size) */ }; -struct direct_window { +struct dma_win {struct device_node *device;const struct dynamic_dma_window_prop *prop;struct list_head list;
@@ -359,12 +359,13 @@ struct ddw_create_response {u32 addr_lo; }; -static LIST_HEAD (direct_window_list); +static LIST_HEAD (dma_win_list);/* prevents races between memory on/offline and window creation */ -static DEFINE_SPINLOCK (direct_window_list_lock); +static DEFINE_SPINLOCK (dma_win_list_lock);/* protects initializing window twice for same device */ -static DEFINE_MUTEX (direct_window_init_mutex); +static DEFINE_MUTEX (dma_win_init_mutex);#define DMA64_PROPNAME "linux,dma64-ddr-window-info" +#define DEFAULT_DMA_WIN "ibm,dma-window"static int tce_clearrange_multi_pSeriesLP (unsigned long start_pfn,unsigned long num_pfn,const void *arg)
@@ -697,15 +698,18 @@ static void pci_dma_bus_setup_pSeriesLP(struct pci_bus *bus)pr_debug ("pci_dma_bus_setup_pSeriesLP: setting up bus %pOF\n" , dn); -/* Find nearest ibm,dma-window, walking up the device tree */ +/* +* Find nearest ibm,dma-window (default DMA window), walking up the +* device tree +*/ for (pdn = dn; pdn !=NULL ; pdn = pdn->parent) { - dma_window =of_get_property (pdn,"ibm,dma-window" ,NULL ); + dma_window =of_get_property (pdn,DEFAULT_DMA_WIN ,NULL );if (dma_window !=NULL )break ; }if (dma_window ==NULL ) { -pr_debug (" no ibm,dma-window property !\n" ); +pr_debug (" no %s property !\n" ,DEFAULT_DMA_WIN );return ; }
@@ -803,11 +807,11 @@ static void remove_dma_window(struct device_node *np, u32 *ddw_avail, ret =rtas_call (ddw_avail[DDW_REMOVE_PE_DMA_WIN ],1 ,1 ,NULL , liobn);if (ret) -pr_warn ("%pOF: failed to remove direct window: rtas returned " +pr_warn ("%pOF: failed to remove dma window: rtas returned " "%d to ibm,remove-pe-dma-window(%x) %llx\n" , np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN ], liobn);else -pr_debug ("%pOF: successfully removed direct window: rtas returned " +pr_debug ("%pOF: successfully removed dma window: rtas returned " "%d to ibm,remove-pe-dma-window(%x) %llx\n" , np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN ], liobn); }
@@ -835,26 +839,26 @@ static void remove_ddw(struct device_node *np, bool remove_prop) ret =of_remove_property (np, win);if (ret) -pr_warn ("%pOF: failed to remove direct window property: %d\n" , +pr_warn ("%pOF: failed to remove dma window property: %d\n" , np, ret); }static u64 find_existing_ddw (struct device_node *pdn) { -struct direct_window *window; -const struct dynamic_dma_window_prop *direct64; +struct dma_win *window; +const struct dynamic_dma_window_prop *dma64;u64 dma_addr =0 ; -spin_lock (&direct_window_list_lock); +spin_lock (&dma_win_list_lock);/* check if we already created a window and dupe that config if so */ -list_for_each_entry (window, &direct_window_list, list) { +list_for_each_entry (window, &dma_win_list, list) {if (window->device == pdn) { - direct64 = window->prop; - dma_addr =be64_to_cpu (direct64->dma_base); + dma64 = window->prop; + dma_addr =be64_to_cpu (dma64->dma_base);break ; } } -spin_unlock (&direct_window_list_lock); +spin_unlock (&dma_win_list_lock);return dma_addr; }
@@ -863,15 +867,15 @@ static int find_existing_ddw_windows(void) {int len;struct device_node *pdn; -struct direct_window *window; -const struct dynamic_dma_window_prop *direct64; +struct dma_win *window; +const struct dynamic_dma_window_prop *dma64;if (!firmware_has_feature (FW_FEATURE_LPAR ))return 0 ;for_each_node_with_property (pdn,DMA64_PROPNAME ) { - direct64 =of_get_property (pdn,DMA64_PROPNAME , &len); -if (!direct64) + dma64 =of_get_property (pdn,DMA64_PROPNAME , &len); +if (!dma64)continue ; window =kzalloc (sizeof (*window),GFP_KERNEL );
@@ -882,10 +886,10 @@ static int find_existing_ddw_windows(void) } window->device = pdn; - window->prop = direct64; -spin_lock (&direct_window_list_lock); -list_add (&window->list, &direct_window_list); -spin_unlock (&direct_window_list_lock); + window->prop = dma64; +spin_lock (&dma_win_list_lock); +list_add (&window->list, &dma_win_list); +spin_unlock (&dma_win_list_lock); }return 0 ;
@@ -1118,13 +1122,13 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)u64 dma_addr, max_addr;struct device_node *dn;u32 ddw_avail[DDW_APPLICABLE_SIZE ]; -struct direct_window *window; +struct dma_win *window;struct property *win64;struct dynamic_dma_window_prop *ddwprop;struct failed_ddw_pdn *fpdn;bool default_win_removed = false; -mutex_lock (&direct_window_init_mutex); +mutex_lock (&dma_win_init_mutex); dma_addr =find_existing_ddw (pdn);if (dma_addr !=0 )
@@ -1177,7 +1181,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)struct property *default_win;int reset_win_ext; - default_win =of_find_property (pdn,"ibm,dma-window" ,NULL ); + default_win =of_find_property (pdn,DEFAULT_DMA_WIN ,NULL );if (!default_win) goto out_failed;
@@ -1206,8 +1210,8 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) }else if (query.page_size &1 ) { page_shift =12 ;/* 4kB */ }else { -dev_dbg (&dev->dev,"no supported direct page size in mask %x" , - query.page_size); +dev_dbg (&dev->dev,"no supported page size in mask %x" , + query.page_size); goto out_failed; }
@@ -1258,7 +1262,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) ret =walk_system_ram_range (0 ,memblock_end_of_DRAM () >>PAGE_SHIFT , win64->value, tce_setrange_multi_pSeriesLP_walk);if (ret) { -dev_info (&dev->dev,"failed to map direct window for %pOF: %d\n" , +dev_info (&dev->dev,"failed to map DMA window for %pOF: %d\n" , dn, ret); goto out_free_window; }
@@ -1272,9 +1276,9 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) window->device = pdn; window->prop = ddwprop; -spin_lock (&direct_window_list_lock); -list_add (&window->list, &direct_window_list); -spin_unlock (&direct_window_list_lock); +spin_lock (&dma_win_list_lock); +list_add (&window->list, &dma_win_list); +spin_unlock (&dma_win_list_lock);/* Only returns the dma_addr if DDW maps the whole partition */ if (len ==order_base_2 (max_addr))
@@ -1303,7 +1307,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)list_add (&fpdn->list, &failed_ddw_pdn_list); out_unlock: -mutex_unlock (&direct_window_init_mutex); +mutex_unlock (&dma_win_init_mutex);return dma_addr; }
@@ -1343,7 +1347,7 @@ static void pci_dma_dev_setup_pSeriesLP(struct pci_dev *dev)for (pdn = dn; pdn &&PCI_DN (pdn) && !PCI_DN (pdn)->table_group; pdn = pdn->parent) { - dma_window =of_get_property (pdn,"ibm,dma-window" ,NULL ); + dma_window =of_get_property (pdn,DEFAULT_DMA_WIN ,NULL );if (dma_window)break ; }
@@ -1394,7 +1398,7 @@ static bool iommu_bypass_supported_pSeriesLP(struct pci_dev *pdev, u64 dma_mask)*/ for (pdn = dn; pdn &&PCI_DN (pdn) && !PCI_DN (pdn)->table_group; pdn = pdn->parent) { - dma_window =of_get_property (pdn,"ibm,dma-window" ,NULL ); + dma_window =of_get_property (pdn,DEFAULT_DMA_WIN ,NULL );if (dma_window)break ; }
@@ -1412,29 +1416,29 @@ static bool iommu_bypass_supported_pSeriesLP(struct pci_dev *pdev, u64 dma_mask)static int iommu_mem_notifier (struct notifier_block *nb,unsigned long action,void *data) { -struct direct_window *window; +struct dma_win *window;struct memory_notify *arg = data;int ret =0 ;switch (action) {case MEM_GOING_ONLINE : -spin_lock (&direct_window_list_lock); -list_for_each_entry (window, &direct_window_list, list) { +spin_lock (&dma_win_list_lock); +list_for_each_entry (window, &dma_win_list, list) { ret |=tce_setrange_multi_pSeriesLP (arg->start_pfn, arg->nr_pages, window->prop);/* XXX log error */ } -spin_unlock (&direct_window_list_lock); +spin_unlock (&dma_win_list_lock);break ;case MEM_CANCEL_ONLINE :case MEM_OFFLINE : -spin_lock (&direct_window_list_lock); -list_for_each_entry (window, &direct_window_list, list) { +spin_lock (&dma_win_list_lock); +list_for_each_entry (window, &dma_win_list, list) { ret |=tce_clearrange_multi_pSeriesLP (arg->start_pfn, arg->nr_pages, window->prop);/* XXX log error */ } -spin_unlock (&direct_window_list_lock); +spin_unlock (&dma_win_list_lock);break ;default :break ;
@@ -1455,7 +1459,7 @@ static int iommu_reconfig_notifier(struct notifier_block *nb, unsigned long actistruct of_reconfig_data *rd = data;struct device_node *np = rd->dn;struct pci_dn *pci =PCI_DN (np); -struct direct_window *window; +struct dma_win *window;switch (action) {case OF_RECONFIG_DETACH_NODE :
@@ -1471,15 +1475,15 @@ static int iommu_reconfig_notifier(struct notifier_block *nb, unsigned long actiiommu_pseries_free_group (pci->table_group, np->full_name); -spin_lock (&direct_window_list_lock); -list_for_each_entry (window, &direct_window_list, list) { +spin_lock (&dma_win_list_lock); +list_for_each_entry (window, &dma_win_list, list) {if (window->device == np) {list_del (&window->list);kfree (window);break ; } } -spin_unlock (&direct_window_list_lock); +spin_unlock (&dma_win_list_lock);break ;default : err =NOTIFY_DONE ;
Signature · 1 line
2.25.4