Skip to content

iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC

Mostafa Saleh linux-doc linux-iommu linux-mm lkml 7 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v7 Latest5 patches 4 repliesIn review
  2. v64 patches 19 repliesApplied
  3. v54 patches 19 repliesSuperseded
Earlier versions · 4
VersionPostedPatchesRepliesStatus
v44 patches9 repliesSuperseded
v34 patches11 repliesSuperseded
v24 patches13 repliesSuperseded
v1 RFC4 patches7 repliesSuperseded

Changes in v7 from the cover letter

v6: https://lore.kernel.org/linux-iommu/[email protected]/
- Add a new function page_ext_get_phys() and use it instead of pfn_valid() + phys_to_page().
- Drop R-bs on patch 4.

Patches in v7 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/5mm/page_ext: Add page_ext_get_phys()include/linux/page_ext.h mm/page_ext.c+29 -00 replies
2/5iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOCDocumentation/admin-guide/kernel-parameters.txt drivers/iommu/Kconfig drivers/iommu/Makefile +3 more+82 -00 replies
3/5iommu: Add calls for IOMMU_DEBUG_PAGEALLOCdrivers/iommu/iommu-debug-pagealloc.c drivers/iommu/iommu-priv.h drivers/iommu/iommu.c +1 more+96 -20 replies
4/5iommu: debug-pagealloc: Track IOMMU pagesdrivers/iommu/iommu-debug-pagealloc.c+86 -00 replies
5/5iommu: debug-pagealloc: Check mapped/unmapped kernel memorydrivers/iommu/iommu-debug-pagealloc.c include/linux/iommu-debug-pagealloc.h include/linux/mm.h+42 -00 replies

Reviews trailers per patch

3 of 5 patches have a Reviewed-by or Acked-by.

PatchDavid Hildenbrand (Red Hat)Pranjal ShrivastavaLu BaoluSamiullah Khawaja
1/5
2/5A-b R-b R-b
3/5R-b R-b R-b
4/5
5/5R-b R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.

Read the v7 thread

Cover letter [PATCH v7 0/5] iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC

Overview
--------
This patch series introduces a new debugging feature, IOMMU_DEBUG_PAGEALLOC, designed to catch DMA use-after-free bugs and IOMMU mapping leaks from buggy drivers.

The kernel has powerful sanitizers like KASAN and DEBUG_PAGEALLOC for catching CPU-side memory corruption. However, there is limited runtime sanitization for DMA mappings managed by the IOMMU. A buggy driver can free a page while it is still mapped for DMA, leading to memory corruption or use-after-free vulnerabilities when that page is reallocated and used for a different purpose.

Inspired by DEBUG_PAGEALLOC, this sanitizer tracks IOMMU mappings on a per-page basis, as it’s not possible to unmap the pages, because it requires to lock and walk all domains on every kernel free, instead we rely on page_ext to add an IOMMU-specific mapping reference count for each page.
And on each page allocated/freed from the kernel we simply check the count and WARN if it is not zero, and dumping page owner information if enabled.

Concurrency
-----------
By design this check is racy where one caller can map pages just after the check, which can lead to false negatives.
In my opinion this is acceptable for sanitizers (for ex KCSAN have that property).
Otherwise we have to implement locks in iommu_map/unmap for all domains which is not favourable even for a debug feature.
The sanitizer only guarantees that the refcount itself doesn’t get corrupted using atomics. And there are no false positives.

CPU vs IOMMU Page Size
----------------------
IOMMUs can use different page sizes and which can be non-homogeneous; not even all of them have the same page size.

To solve this, the refcount is always incremented and decremented in units of the smallest page size supported by the IOMMU domain. This ensures the accounting remains consistent regardless of the size of the map or unmap operation, otherwise double counting can happen.

Testing & Performance
---------------------
Earlier versions were tested on Morello with Arm64 + SMMUv3 This version was tested and benchmarked on Lenovo IdeaCentre X Gen 10 Snapdragon.

With dma_map_benchmark, tested on both SMMUv3 and SMMUv3, where:

CONFIG refers to "CONFIG_IOMMU_DEBUG_PAGEALLOC" cmdline refers to "iommu.debug_pagealloc"
Numbers are (map latency)/(unmap latency), lower is better.

SMMUv3:
echo dma_map_benchmark > /sys/bus/pci/devices/0005\:01\:00.0/driver_override echo 0005:01:00.0 > /sys/bus/pci/devices/0005\:01\:00.0/driver/unbind echo 0005:01:00.0 > /sys/bus/pci/drivers/dma_map_benchmark/bind ./dma_map_benchmark -t $threads -g $nr_pages

			CONFIG=n    CONFIG=y    CONFIG=y
			            cmdline=0   cmdline=1
4K - 1 thread		0.0/1.8     0.0/1.8     0.1/1.9

4K - 8 threads 0.2/6.2 0.2/6.1 0.3/6.8
1M - 1 thread 0.5/54.3 0.5/54.4 6.7/74.4
1M - 8 threads 1.2/339.7 1.1/340.9 10.5/331.7

SMMUv2:
echo dma_map_benchmark > /sys/bus/platform/devices/ac0000.geniqup/driver_override echo ac0000.geniqup > /sys/bus/platform/devices/ac0000.geniqup/driver/unbind echo ac0000.geniqup > /sys/bus/platform/drivers/dma_map_benchmark/bind ./dma_map_benchmark -t $threads -g $nr_pages

			CONFIG=n    CONFIG=y    CONFIG=y
			            cmdline=0   cmdline=1
4K - 1 thread		0.6/3.9     0.6/3.9     0.7/4.0

4K - 8 threads 1.0/19.6 1.0/19.7 1.1/19.9
1M - 1 thread 61.8/372.0 61.8/374.9 68.1/396.7
1M - 8 threads 89.4/1470.5 89.9/1362.9 100.3/1373.3

Changes in v7:
v6: https://lore.kernel.org/linux-iommu/[email protected]/
- Add a new function page_ext_get_phys() and use it instead of pfn_valid() + phys_to_page().
- Drop R-bs on patch 4.

Changes in v6:
v5: https://lore.kernel.org/linux-iommu/[email protected]/
- Remove wrong logic for handling unmap with bigger size than requested and warn instead.
- Collect R-bs and Acked-by

Changes in v5:
v4: https://lore.kernel.org/all/[email protected]/
- Fix typo in comment
- Collect Baolu R-bs

Main changes in v4:
v3: https://lore.kernel.org/all/[email protected]/
- Update the kernel parameter format in docs based on Randy feedback
- Update commit subjects
- Add IOMMU only functions in iommu-priv.h based on Baolu feedback

Main changes in v3: (Most of them addressing Will comments)
v2: https://lore.kernel.org/linux-iommu/[email protected]/
- Reword the Kconfig help
- Use unmap_begin/end instead of unmap/remap
- Use relaxed accessors when refcounting
- Fix a bug with checking the returned address from iova_to_phys
- Add more hardening checks (overflow)
- Add more debug info on assertions (dump_page_owner())
- Handle cases where unmap returns larger size as the core code seems to tolerate that.
- Drop Tested-by tags from Qinxin as the code logic changed

Main changes in v2:
v1: https://lore.kernel.org/linux-iommu/[email protected]/
- Address Jörg comments about #ifdefs and static keys
- Reword the Kconfig help
- Drop RFC
- Collect t-b from Qinxin
- Minor cleanups

Mostafa Saleh (5):
mm/page_ext: Add page_ext_get_phys()
iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC
iommu: Add calls for IOMMU_DEBUG_PAGEALLOC
iommu: debug-pagealloc: Track IOMMU pages
iommu: debug-pagealloc: Check mapped/unmapped kernel memory

 .../admin-guide/kernel-parameters.txt         |   9 +
 drivers/iommu/Kconfig                         |  19 ++
 drivers/iommu/Makefile                        |   1 +
 drivers/iommu/iommu-debug-pagealloc.c         | 169 ++++++++++++++++++
 drivers/iommu/iommu-priv.h                    |  58 ++++++
 drivers/iommu/iommu.c                         |  11 +-
 include/linux/iommu-debug-pagealloc.h         |  32 ++++
 include/linux/mm.h                            |   5 +
 include/linux/page_ext.h                      |   6 +
 mm/page_ext.c                                 |  27 +++
 10 files changed, 335 insertions(+), 2 deletions(-)
 create mode 100644 drivers/iommu/iommu-debug-pagealloc.c
 create mode 100644 include/linux/iommu-debug-pagealloc.h
Signature · 1 line
2.52.0.457.g6b5491de43-goog