iommu/riscv: Add hardware dirty tracking for second-stage domains
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v7 Latest10 patches 9 repliesIn review
- v610 patches 3 repliesSuperseded
- v5 RFC10 patches 13 repliesSuperseded
Earlier versions · 4
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v4 RFC | 10 patches | 15 replies | Superseded | |
| v3 RFC | 9 of 10 patches | 4 replies | Superseded | |
| v2 RFC | 10 patches | 6 replies | Superseded | |
| v1 RFC | 11 patches | 18 replies | Superseded |
Changes in v7 from the cover letter
- Rebased on top of v7.4-rc1 (6628e7d4c6b2).
- Return -EINVAL instead of -ENODEV when the IOMMU does not support the iohgatp mode of a second-stage domain, following the attach_dev errno semantics and allowing callers such as QEMU to try another domain or allocate a new compatible one.
- Assign dirty_ops only when IOMMU_HWPT_ALLOC_DIRTY_TRACKING is requested, and reject attaching a dirty-tracking domain to an IOMMU without AMO_HWAD with -EINVAL. (Per Gong Shuai)
- Link to v6:
https://lore.kernel.org/linux-iommu/[email protected]/
Patches in v7 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/10 | iommu/generic_pt: Fix the RISC-V supported feature maskdrivers/ | +2 | 0 replies | |
| 2/10 | iommupt: Add RISC-V Second-stage (iohgatp) page table supportdrivers/ drivers/ include/ +1 more | +102 | 0 replies | |
| 3/10 | iommupt: Add RISC-V dirty tracking PTE opsdrivers/ | +49 | 0 replies | |
| 4/10 | iommu/riscv: use data structure instead of individual valuesdrivers/ | +17 | 0 replies | |
| 5/10 | iommu/riscv: support GSCID and GVMA invalidation commanddrivers/ drivers/ | +58 | 2 replies | |
| 6/10 | RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V archarch/ | +2 | 2 replies | |
| 7/10 | iommu/riscv: Add domain_alloc_paging_flags for second-stage domaindrivers/ | +84 | 0 replies | |
| 8/10 | iommu/riscv: Pre-enable GADE for second-stage domainsdrivers/ | +4 | 0 replies | |
| 9/10 | iommu/riscv: Add dirty tracking support for second-stage domainsdrivers/ | +30 | 2 replies | |
| 10/10 | iommu/riscv: Add IOTINVAL.GVMA invalidationsdrivers/ | +15 | 0 replies |
Reviews trailers per patch
9 of 10 patches have a Reviewed-by or Acked-by.
| Patch | Fangyu Yu | Nutty Liu | Jason Gunthorpe |
|---|---|---|---|
| 1/10 | T-b | R-b | R-b |
| 2/10 | R-b | ||
| 3/10 | R-b | ||
| 4/10 | R-b | R-b | |
| 5/10 | R-b | ||
| 6/10 | |||
| 7/10 | R-b | ||
| 8/10 | R-b | ||
| 9/10 | R-b | ||
| 10/10 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.
Cover letter [PATCH v7 00/10] iommu/riscv: Add hardware dirty tracking for second-stage domains
From: Fangyu Yu <[email protected]>
The RISC-V IOMMU architecture defines an AMO_HWAD capability (Hardware Access/Dirty update) that allows the IOMMU to atomically set the A/D bits in second-stage PTEs on DMA access. When DC.tc.GADE is asserted, the IOMMU autonomously sets D on the first write to a page mapped by an iohgatp domain. This series wires that capability up to the iommufd dirty-tracking interface (IOMMU_HWPT_SET_DIRTY_TRACKING / IOMMU_HWPT_GET_DIRTY_BITMAP) and reports IOMMU_CAP_DIRTY_TRACKING.
Design notes
============
* The feature is scoped to second-stage (iohgatp) domains only; these are the domains created for KVM / VFIO device pass-through when userspace allocates an HWPT with IOMMU_HWPT_ALLOC_NEST_PARENT or IOMMU_HWPT_ALLOC_DIRTY_TRACKING. First-stage (iosatp) domains are not touched by this series.
* The page-table side plugs into the existing generic_pt dirty hook framework (amdv1 / vtdss style). RISC-V adds the three required PTE ops - is_write_dirty / make_write_clean / make_write_dirty.
Testing
=======
* Test on QEMU RISC-V, a nvme and an e1000e device was passed through to an L2 guest via vfio-pci + iommufd.
* generic_pt KUnit: the existing test_dirty case now runs and passes for the RISC-V 64-bit format.
Follow-up work
==============
* Build a dedicated end-to-end test case that drives the full flow (HWPT_ALLOC with DIRTY_TRACKING -> attach -> IOAS_MAP -> generate real DMA -> SET_DIRTY_TRACKING -> GET_DIRTY_BITMAP -> verify bitmap against expected IOVA footprint) so that the behaviour can be regression-tested beyond the KUnit PTE-level coverage.
LLM-based coding assistants were used for code exploration, patch review, test execution, and drafting/editing messages and this cover letter. I reviewed and finalized all resulting code and text.
This series functionally depends on [1], which has been posted but is not yet in mainline.
---
Changes in v7:
- Rebased on top of v7.4-rc1 (6628e7d4c6b2).
- Return -EINVAL instead of -ENODEV when the IOMMU does not support the iohgatp mode of a second-stage domain, following the attach_dev errno semantics and allowing callers such as QEMU to try another domain or allocate a new compatible one.
- Assign dirty_ops only when IOMMU_HWPT_ALLOC_DIRTY_TRACKING is requested, and reject attaching a dirty-tracking domain to an IOMMU without AMO_HWAD with -EINVAL. (Per Gong Shuai)
- Link to v6:
https://lore.kernel.org/linux-iommu/[email protected]/
Changes in v6:
- Drop the RFC tag.
- Rebased on top of mainline (390febb5805d).
- Carry the unmerged prerequisite [2] as patch 1.
- Drop "iommu/riscv: report iommu capabilities"; mainline commit 3069e8f3e1e9 ("iommu/riscv: Report cache coherency capability") already adds riscv_iommu_capable().
- domain_alloc_paging_flags: drop the redundant flags pre-check and supported_flags, and split the per-stage setup into riscv_iommu_init_domain_s1() / riscv_iommu_init_domain_s2(). (Per Jason)
- Link to v5:
https://lore.kernel.org/linux-iommu/[email protected]/
Changes in v5:
- Propagate the GADE bit from the new device context into the hardware DC in riscv_iommu_iodir_update(), so the pre-enabled GADE actually reaches the hardware. (Per Gong Shuai)
- Issue a global IOTINVAL.GVMA in riscv_iommu_iodir_set_mode(), so that second-stage translations cached before a DDTP mode change (e.g. by a previous kernel after kexec) are flushed as the specification requires. (Per Tan Chi)
- Rebased on top of v7.3, with [1] and [2] applied.
- Link to v4:
https://lore.kernel.org/linux-riscv/[email protected]/
Changes in v4 (Jason's suggestions):
- Rebased the series on top of [1] and [2].
- Drop the RISC-V-specific pt_num_items_lg2() top-level special case.
- Validate page-table address widths by translation stage: accept only Sv39/Sv48/Sv57 widths for first-stage tables and only Sv39x4/Sv48x4/ Sv57x4 widths for second-stage tables.
- Replace the aliased first-stage/second-stage MODE union with independent FSC/IOSATP and IOHGATP MODE fields.
- Add pt_dirty_supported() to restrict RISC-V generic_pt dirty tracking to second-stage tables.
- Link to v3:
https://lore.kernel.org/linux-riscv/[email protected]/
Changes in v3 (Andrew Jones's suggestions):
- Rebased the series on top of Andrew Jones' generic_pt RISC-V supported feature mask fix, which adds PT_FEAT_RISCV_SVPBMT to the supported feature set.
- Added PT_FEAT_RISCV_S2 to the RISC-V generic_pt supported feature mask and KUnit feature matrix.
- Kept PT_FEAT_SIGN_EXTEND only for first-stage RISC-V KUnit configs; second stage configs now use PT_FEAT_RISCV_S2 without sign-extension semantics.
- Split RISC-V IOMMU capability checks into first-stage FSC and second-stage IOHGATP helpers.
- Updated second-stage paging domain setup to use GPA widths 41/50/59 and to avoid PT_FEAT_SIGN_EXTEND.
- Link to v2:
https://lore.kernel.org/linux-riscv/[email protected]/
Changes in v2 (Jason's suggestions):
- Introduced a single PT_FEAT_RISCV_S2: second-stage selection is driven purely by this feature bit.
- Switched from dynamic DC.tc.GADE toggling to static pre-enable.
- domain_alloc_paging_flags: follow the switch/case design from other drivers.
- Drop IOMMU_CAP_DEFERRED_FLUSH in riscv_iommu_capable.
- Remove the .hw_info-related patch.
- Link to v1:
https://lore.kernel.org/linux-riscv/[email protected]/
[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-iommu/[email protected]/
Andrew Jones (1):
iommu/generic_pt: Fix the RISC-V supported feature mask
Fangyu Yu (6):
iommupt: Add RISC-V Second-stage (iohgatp) page table support
iommupt: Add RISC-V dirty tracking PTE ops
iommu/riscv: Add domain_alloc_paging_flags for second-stage domain iommu/riscv: Pre-enable GADE for second-stage domains iommu/riscv: Add dirty tracking support for second-stage domains iommu/riscv: Add IOTINVAL.GVMA invalidations
Tomasz Jeznach (1):
RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch
Zong Li (2):
iommu/riscv: use data structure instead of individual values iommu/riscv: support GSCID and GVMA invalidation command
arch/riscv/kvm/Kconfig | 2 + drivers/iommu/generic_pt/fmt/iommu_riscv64.c | 3 +- drivers/iommu/generic_pt/fmt/riscv.h | 158 ++++++++++-- drivers/iommu/riscv/iommu-bits.h | 7 + drivers/iommu/riscv/iommu.c | 244 +++++++++++++++---- include/linux/generic_pt/common.h | 4 + include/linux/generic_pt/iommu.h | 11 + 7 files changed, 359 insertions(+), 70 deletions(-)
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
Signature · 7 lines
2.50.1 -- kvm-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kvm-riscv