Skip to content

DDW + Indirect Mapping

Leonardo Bras linuxppc-dev lkml 6 versions – Applied

Status from this message: Applied to powerpc/next.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v6 Latest11 patches 10 repliesApplied
  2. v511 patches 25 repliesSuperseded
  3. v411 patches 13 repliesSuperseded
Earlier versions · 3
VersionPostedPatchesRepliesStatus
v311 patches5 repliesSuperseded
v214 patches25 repliesSuperseded
v110 patches41 repliesSuperseded

Changes in v6 from the cover letter

- Reviews from Frederic Barrat
- 02/11 : memset bitmap only if tbl not in use
- 06/11 : remove_ddw() is not used in enable_ddw() error path anymore
New helpers were created for that.
- 10/11 : There was a typo, but got replaced due to 06/11 fix.
v5 Link: http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=253799&state=%2A&archive=both

Patches in v6 Applied

#Subject and filesSizeRepliesApplied
1/11powerpc/pseries/iommu: Replace hard-coded page shiftarch/powerpc/include/asm/tce.h arch/powerpc/platforms/pseries/iommu.c+23 -240 replies
2/11powerpc/kernel/iommu: Add new iommu_table_in_use() helperarch/powerpc/include/asm/iommu.h arch/powerpc/kernel/iommu.c+32 -301 reply
3/11powerpc/pseries/iommu: Add iommu_pseries_alloc_table() helperarch/powerpc/platforms/pseries/iommu.c+14 -110 replies
4/11powerpc/pseries/iommu: Add ddw_list_new_entry() helperarch/powerpc/platforms/pseries/iommu.c+21 -110 replies
5/11powerpc/pseries/iommu: Allow DDW windows starting at 0x00arch/powerpc/platforms/pseries/iommu.c+18 -180 replies
6/11powerpc/pseries/iommu: Add ddw_property_create() and refactor enable_ddw()arch/powerpc/platforms/pseries/iommu.c+84 -451 reply
7/11powerpc/pseries/iommu: Reorganize iommu_table_setparms*() with new helperarch/powerpc/platforms/pseries/iommu.c+38 -340 replies
8/11powerpc/pseries/iommu: Update remove_dma_window() to accept property namearch/powerpc/platforms/pseries/iommu.c+10 -81 reply
9/11powerpc/pseries/iommu: Find existing DDW with given property namearch/powerpc/platforms/pseries/iommu.c+15 -100 replies
10/11powerpc/pseries/iommu: Make use of DDW for indirect mappingarch/powerpc/platforms/pseries/iommu.c+74 -152 replies
11/11powerpc/pseries/iommu: Rename "direct window" to "dma window"arch/powerpc/platforms/pseries/iommu.c+45 -420 replies

Reviews trailers per patch

All 11 patches have a Reviewed-by or Acked-by.

PatchAlexey KardashevskiyFrederic Barrat
1/11R-b R-b
2/11R-b R-b
3/11R-b R-b
4/11R-b R-b
5/11R-b R-b
6/11R-b R-b
7/11R-b
8/11R-b R-b
9/11R-b R-b
10/11R-b
11/11R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v6 thread

Cover letter [PATCH v6 00/11] DDW + Indirect Mapping

So far it's assumed possible to map the guest RAM 1:1 to the bus, which works with a small number of devices. SRIOV changes it as the user can configure hundreds VFs and since phyp preallocates TCEs and does not allow IOMMU pages bigger than 64K, it has to limit the number of TCEs per a PE to limit waste of physical pages.

As of today, if the assumed direct mapping is not possible, DDW creation is skipped and the default DMA window "ibm,dma-window" is used instead.

Using the DDW instead of the default DMA window may allow to expand the amount of memory that can be DMA-mapped, given the number of pages (TCEs) may stay the same (or increase) and the default DMA window offers only 4k-pages while DDW may offer larger pages (4k, 64k, 16M ...).

Patch #1 replaces hard-coded 4K page size with a variable containing the correct page size for the window.

Patch #2 introduces iommu_table_in_use(), and replace manual bit-field checking where it's used. It will be used for aborting enable_ddw() if there is any current iommu allocation and we are trying single window indirect mapping.

Patch #3 introduces iommu_pseries_alloc_table() that will be helpful when indirect mapping needs to replace the iommu_table.

Patch #4 adds helpers for adding DDWs in the list.

Patch #5 refactors enable_ddw() so it returns if direct mapping is possible, instead of DMA offset. It helps for next patches on indirect DMA mapping and also allows DMA windows starting at 0x00.

Patch #6 bring new helper to simplify enable_ddw(), allowing some reorganization for introducing indirect mapping DDW.

Patch #7 adds new helper _iommu_table_setparms() and use it in other *setparams*() to fill iommu_table. It will also be used for creating a new iommu_table for indirect mapping.

Patch #8 updates remove_dma_window() to accept different property names, so we can introduce a new property for indirect mapping.

Patch #9 extracts find_existing_ddw_windows() into find_existing_ddw_windows_named(), and calls it by it's property name. This will be useful when the property for indirect mapping is created, so we can search the device-tree for both properties.

Patch #10:
Instead of destroying the created DDW if it doesn't map the whole partition, make use of it instead of the default DMA window as it improves performance. Also, update the iommu_table and re-generate the pools. It introduces a new property name for DDW with indirect DMA mapping.

Patch #11:
Does some renaming of 'direct window' to 'dma window', given the DDW created can now be also used in indirect mapping if direct mapping is not available.

All patches were tested into an LPAR with an virtio-net interface that allows default DMA window and DDW to coexist.

Changes since v5:
- Reviews from Frederic Barrat
- 02/11 : memset bitmap only if tbl not in use
- 06/11 : remove_ddw() is not used in enable_ddw() error path anymore
New helpers were created for that.
- 10/11 : There was a typo, but got replaced due to 06/11 fix.
v5 Link: http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=253799&state=%2A&archive=both

Changes since v4:
- Solve conflicts with new upstream versions
- Avoid unecessary code moving by doing variable declaration before definition
- Rename _iommu_table_setparms to iommu_table_setparms_common and changed base parameter from unsigned long to void* in order to avoid unecessary casting.
- Fix breaking case for existing direct-mapping.
- Fix IORESOURCE_MEM bound issue
- Move new tbl to pci->table_group->tables[1] instead of replacing [0]
v4 Link: https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=241597&state=%2A&archive=both

Changes since v3:
- Fixed inverted free order at ddw_property_create()
- Updated goto tag naming
v3 Link: https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=240287&state=%2A&archive=both

Changes since v2:
- Some patches got removed from the series and sent by themselves,
- New tbl created for DDW + indirect mapping reserves MMIO32 space,
- Improved reserved area algorithm,
- Improved commit messages,
- Removed define for default DMA window prop name,
- Avoided some unnecessary renaming,
- Removed some unnecessary empty lines,
- Changed some code moving to forward declarations.
v2 Link: http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=201210&state=%2A&archive=both

Leonardo Bras (11):
powerpc/pseries/iommu: Replace hard-coded page shift powerpc/kernel/iommu: Add new iommu_table_in_use() helper powerpc/pseries/iommu: Add iommu_pseries_alloc_table() helper powerpc/pseries/iommu: Add ddw_list_new_entry() helper powerpc/pseries/iommu: Allow DDW windows starting at 0x00 powerpc/pseries/iommu: Add ddw_property_create() and refactor
enable_ddw()
powerpc/pseries/iommu: Reorganize iommu_table_setparms*() with new
helper
powerpc/pseries/iommu: Update remove_dma_window() to accept property
name
powerpc/pseries/iommu: Find existing DDW with given property name powerpc/pseries/iommu: Make use of DDW for indirect mapping powerpc/pseries/iommu: Rename "direct window" to "dma window"

 arch/powerpc/include/asm/iommu.h       |   1 +
 arch/powerpc/include/asm/tce.h         |   8 -
 arch/powerpc/kernel/iommu.c            |  65 ++--
 arch/powerpc/platforms/pseries/iommu.c | 481 +++++++++++++++----------
 4 files changed, 330 insertions(+), 225 deletions(-)
Signature · 1 line
2.32.0