iommu/riscv: Add hardware dirty tracking for second-stage domains
Subsystems (6)
cover iommu/riscv: Add hardware dirty tracking for second-stage domains
From: Fangyu Yu <[email protected]>
The RISC-V IOMMU architecture defines an AMO_HWAD capability (Hardware Access/Dirty update) that allows the IOMMU to atomically set the A/D bits in second-stage PTEs on DMA access. When DC.tc.GADE is asserted, the IOMMU autonomously sets D on the first write to a page mapped by an iohgatp domain. This series wires that capability up to the iommufd dirty-tracking interface (IOMMU_HWPT_SET_DIRTY_TRACKING / IOMMU_HWPT_GET_DIRTY_BITMAP) and reports IOMMU_CAP_DIRTY_TRACKING.
Design notes
------------
* The feature is scoped to second-stage (iohgatp) domains only; these are the domains created for KVM / VFIO device pass-through when userspace allocates an HWPT with IOMMU_HWPT_ALLOC_NEST_PARENT or IOMMU_HWPT_ALLOC_DIRTY_TRACKING. First-stage (iosatp) domains are not touched by this series.
* The page-table side plugs into the existing generic_pt dirty hook framework (amdv1 / vtdss style). RISC-V adds the three required PTE ops – is_write_dirty / make_write_clean / make_write_dirty.
Testing
-------
* Test on QEMU RISC-V, a virtio-net and an e1000e device was passed through to an L2 guest via vfio-pci + iommufd.
* generic_pt KUnit: the existing test_dirty case now runs and passes for the RISC-V 64-bit format.
Follow-up work
--------------
* Build a dedicated end-to-end test case that drives the full flow (HWPT_ALLOC with DIRTY_TRACKING -> attach -> IOAS_MAP -> generate real DMA -> SET_DIRTY_TRACKING -> GET_DIRTY_BITMAP -> verify bitmap against expected IOVA footprint) so that the behaviour can be regression-tested beyond the KUnit PTE-level coverage.
* If possible, rebase and retest on top of the updated "iommu irqbypass" patchset.
Fangyu Yu (6):
iommupt: Add RISC-V Second-stage (iohgatp) page table support iommu/riscv: Add domain_alloc_paging_flags for second-stage domain
iommupt: Don't preset D when RISC-V IOMMU dirty tracking on iommu/riscv: Add dirty tracking support for second-stage domains iommu/riscv: Add IOTINVAL.GVMA after updating DDT/PDT entries
iommupt: Add RISC-V dirty tracking PTE ops
Tomasz Jeznach (2):
iommu/riscv: report iommu capabilities
RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch
Zong Li (3):
iommu/riscv: use data structure instead of individual values iommu/riscv: support GSCID and GVMA invalidation command iommu/riscv: support nested iommu for getting iommu hardware
information
arch/riscv/kvm/Kconfig | 2 + drivers/iommu/generic_pt/fmt/riscv.h | 120 ++++++++++++- drivers/iommu/riscv/iommu-bits.h | 7 + drivers/iommu/riscv/iommu.c | 247 +++++++++++++++++++++++---- include/linux/generic_pt/common.h | 13 ++ include/linux/generic_pt/iommu.h | 17 +- include/uapi/linux/iommufd.h | 18 ++ 7 files changed, 383 insertions(+), 41 deletions(-)
Signature · 1 line
2.50.1
RFC PATCH 1/11 iommupt: Add RISC-V Second-stage (iohgatp) page table support
From: Fangyu Yu <[email protected]>
Add support for Sv39x4/Sv48x4/Sv57x4 Second-stage page tables used by the RISC-V IOMMU iohgatp register. The x4 root page table is 16 KiB instead of the usual 4 KiB, covering 2 extra GPA bits (hw_max_vasz_lg2 = 41/50/59).
--- drivers/iommu/generic_pt/fmt/riscv.h | 64 +++++++++++++++++++++++++--- include/linux/generic_pt/common.h | 5 +++ include/linux/generic_pt/iommu.h | 17 +++++++- 3 files changed, 80 insertions(+), 6 deletions(-)
drivers/iommu/generic_pt/fmt/riscv.h +59 -5
@@ -37,7 +37,16 @@ enum {PT_MAX_OUTPUT_ADDRESS_LG2 =34 ,PT_MAX_TOP_LEVEL =1 , #else -PT_MAX_VA_ADDRESS_LG2 =57 , +/* +* PT_MAX_VA_ADDRESS_LG2 is the upper bound accepted by the generic +* pt_iommu_init() range check. It must cover both first-stage and +* second-stage (G-stage) modes: +* +* First-stage (fsc/iosatp): Sv39=39, Sv48=48, Sv57=57 +* Second-stage (iohgatp): Sv39x4=41, Sv48x4=50, Sv57x4=59 +* +*/ +PT_MAX_VA_ADDRESS_LG2 =59 ,PT_MAX_OUTPUT_ADDRESS_LG2 =56 , PT_MAX_TOP_LEVEL = 4, #endif
@@ -124,6 +133,14 @@ riscvpt_entry_num_contig_lg2(const struct pt_state *pts)static inline unsigned int riscvpt_num_items_lg2 (const struct pt_state *pts) { +/* +* Second-stage (iohgatp) root page tables have 4x the usual number of +* entries (2048 = 2^11 instead of 512 = 2^9) to cover the 2 extra GPA +* bits in Sv39x4/Sv48x4/Sv57x4. Only the root (top) level is +* enlarged; all other levels remain at the standard 9-bit index width. +*/ +if (to_riscvpt (pts)->second_stage && pts->level == pts->range->top_level) +return PT_TABLEMEM_LG2SZ -ilog2 (sizeof (u64)) +2 ;return PT_TABLEMEM_LG2SZ -ilog2 (sizeof (u64)); }#define pt_num_items_lg2 riscvpt_num_items_lg2
@@ -254,6 +271,7 @@ riscvpt_iommu_fmt_init(struct pt_iommu_riscv_64 *iommu_table,struct pt_riscv *table = &iommu_table->riscv_64pt;switch (cfg->common.hw_max_vasz_lg2) { +/* First-stage (fsc/iosatp): Sv39 / Sv48 / Sv57 */ case 39 :pt_top_set_level (&table->common,2 );break ;
@@ -263,6 +281,22 @@ riscvpt_iommu_fmt_init(struct pt_iommu_riscv_64 *iommu_table,case 57 :pt_top_set_level (&table->common,4 );break ; +/* +* Second-stage (iohgatp): Sv39x4 / Sv48x4 / Sv57x4. +* The top level is the same as for the first-stage counterpart. +*/ +case 41 : +pt_top_set_level (&table->common,2 ); + table->second_stage = true; +break ; +case 50 : +pt_top_set_level (&table->common,3 ); + table->second_stage = true; +break ; +case 59 : +pt_top_set_level (&table->common,4 ); + table->second_stage = true; +break ;default :return -EINVAL ; }
@@ -283,10 +317,17 @@ riscvpt_iommu_fmt_hw_info(struct pt_iommu_riscv_64 *table,PT_WARN_ON (top_phys & ~PT_TOP_PHYS_MASK );/* -* See Table 3. Encodings of iosatp.MODE field" for DC.tx.SXL = 0: -* 8 = Sv39 = top level 2 -* 9 = Sv38 = top level 3 -* 10 = Sv57 = top level 4 +* Both first-stage (fsc/iosatp) and second-stage (iohgatp) share the +* same MODE numeric values for a given top level: +* top_level 2 -> MODE 8 (Sv39 / Sv39x4) +* top_level 3 -> MODE 9 (Sv48 / Sv48x4) +* top_level 4 -> MODE 10 (Sv57 / Sv57x4) +* +* The union members fsc_iosatp_mode and iohgatp_mode occupy the same +* byte; the caller selects the appropriate name based on domain type. +* +* See "Table 3. Encodings of iosatp.MODE field" (DC.tc.SXL = 0) and +* "Table 2. Encoding of iohgatp.MODE field" in the RISC-V IOMMU spec. */ info->fsc_iosatp_mode = top_range->top_level +6 ; }
@@ -294,6 +335,7 @@ riscvpt_iommu_fmt_hw_info(struct pt_iommu_riscv_64 *table,#if defined(GENERIC_PT_KUNIT )static const struct pt_iommu_riscv_64_cfg riscv_64_kunit_fmt_cfgs[] = { +/* First-stage (fsc/iosatp): Sv39 / Sv48 / Sv57 */ [0 ] = { .common.features =BIT (PT_FEAT_RISCV_SVNAPOT_64K ), .common.hw_max_oasz_lg2 =56 , .common.hw_max_vasz_lg2 = 39 },
@@ -303,6 +345,18 @@ static const struct pt_iommu_riscv_64_cfg riscv_64_kunit_fmt_cfgs[] = { [2 ] = { .common.features =BIT (PT_FEAT_RISCV_SVNAPOT_64K ), .common.hw_max_oasz_lg2 =56 , .common.hw_max_vasz_lg2 =57 }, +/* +* Second-stage (iohgatp): Sv39x4 / Sv48x4 / Sv57x4. + */ + [3 ] = { .common.features =BIT (PT_FEAT_RISCV_SVNAPOT_64K ), + .common.hw_max_oasz_lg2 =56 , + .common.hw_max_vasz_lg2 = 41 }, + [4 ] = { .common.features =0 , + .common.hw_max_oasz_lg2 =56 , + .common.hw_max_vasz_lg2 = 50 }, + [5 ] = { .common.features =BIT (PT_FEAT_RISCV_SVNAPOT_64K ), + .common.hw_max_oasz_lg2 =56 , + .common.hw_max_vasz_lg2 = 59 }, };#define kunit_fmt_cfgs riscv_64_kunit_fmt_cfgsenum {
include/linux/generic_pt/common.h +5 -0
@@ -181,6 +181,11 @@ struct pt_riscv_32 {struct pt_riscv_64 {struct pt_common common; +/* +* True when this table is used for second-stage / iohgatp +* address translation. +*/ +bool second_stage; };enum {
include/linux/generic_pt/iommu.h +16 -1
@@ -328,7 +328,22 @@ struct pt_iommu_riscv_64_cfg {struct pt_iommu_riscv_64_hw_info {u64 ppn; -u8 fsc_iosatp_mode; +union { +/* +* First-stage (fsc/iosatp) MODE encoding: +* 8 = Sv39, 9 = Sv48, 10 = Sv57 +* Used to program DC.fsc.iosatp.MODE. +*/ +u8 fsc_iosatp_mode; +/* +* Second-stage (iohgatp) MODE encoding: +* 8 = Sv39x4, 9 = Sv48x4, 10 = Sv57x4 +* Used to program DC.iohgatp.MODE. +* The numeric values are identical to fsc_iosatp_mode; +* the caller selects the interpretation based on domain type. +*/ +u8 iohgatp_mode; + }; };IOMMU_FORMAT (riscv_64, riscv_64pt);
Signature · 1 line
2.50.1
Re: [RFC PATCH 01/11] iommupt: Add RISC-V Second-stage (iohgatp) page table support
drivers/iommu/generic_pt/fmt/riscv.h riscvpt_iommu_fmt_init(struct pt_iommu_riscv_64 *iommu_table, · line 291
Quoted 12 lines · On Tue, Apr 28, 2026 at 09:13:49PM +0800, [email protected] wrote:
drivers/iommu/generic_pt/fmt/riscv.h +8
-0@@ -263,6 +281,22 @@ riscvpt_iommu_fmt_init(struct pt_iommu_riscv_64 *iommu_table,case 57 :pt_top_set_level (&table->common,4 );break ; +/* +* Second-stage (iohgatp): Sv39x4 / Sv48x4 / Sv57x4. +* The top level is the same as for the first-stage counterpart. +*/ +case 41 : +pt_top_set_level (&table->common,2 ); + table->second_stage = true; +break ;
Second stage needs to be an explicit PT_FEAT not implicitly deduced based on the vasz.
Jason
Re: Re: [RFC PATCH 01/11] iommupt: Add RISC-V Second-stage (iohgatp) page table support
Quoted 7 lines
Quoted 7 lines
drivers/iommu/generic_pt/fmt/riscv.h +3
-0@@ -263,6 +281,22 @@ riscvpt_iommu_fmt_init(struct pt_iommu_riscv_64 *iommu_table,case 57 :pt_top_set_level (&table->common,4 );break ; +/* +* Second-stage (iohgatp): Sv39x4 / Sv48x4 / Sv57x4. +* The top level is the same as for the first-stage counterpart.
Quoted 8 lines
drivers/iommu/generic_pt/fmt/riscv.h riscvpt_iommu_fmt_init(struct pt_iommu_riscv_64 *iommu_table, · line 291
Quoted 5 lines
drivers/iommu/generic_pt/fmt/riscv.h +5
-0+*/ +case 41 : +pt_top_set_level (&table->common,2 ); + table->second_stage = true; +break ;Second stage needs to be an explicit
PT_FEATnot implicitly deduced based on the vasz.
Agreed. I will add an explicit PT_FEAT_RISCV_SECOND_STAGE flag and stop deriving second-stage semantics from vasz.
Thanks,
Fangyu
Quoted 2 lines
Jason
Re: Re: [RFC PATCH 01/11] iommupt: Add RISC-V Second-stage (iohgatp) page table support
Quoted 16 lines · On Wed, Apr 29, 2026 at 09:06:50AM +0800, [email protected] wrote:
Quoted 7 lines
Quoted 7 lines
drivers/iommu/generic_pt/fmt/riscv.h +3
-0@@ -263,6 +281,22 @@ riscvpt_iommu_fmt_init(struct pt_iommu_riscv_64 *iommu_table,case 57 :pt_top_set_level (&table->common,4 );break ; +/* +* Second-stage (iohgatp): Sv39x4 / Sv48x4 / Sv57x4. +* The top level is the same as for the first-stage counterpart. Quoted 8 lines
drivers/iommu/generic_pt/fmt/riscv.h riscvpt_iommu_fmt_init(struct pt_iommu_riscv_64 *iommu_table, · line 291
Quoted 5 lines
drivers/iommu/generic_pt/fmt/riscv.h +5
-0+*/ +case 41 : +pt_top_set_level (&table->common,2 ); + table->second_stage = true; +break ;Second stage needs to be an explicit
PT_FEATnot implicitly deduced based on the vasz.
Quoted 2 lines
Agreed. I will add an explicit
PT_FEAT_RISCV_SECOND_STAGEflag and stop deriving second-stage semantics from vasz.
PT_FEAT_RISCV_S2 would match what I have for ARM
Jason
Re: Re: Re: [RFC PATCH 01/11] iommupt: Add RISC-V Second-stage (iohgatp) page table support
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
drivers/iommu/generic_pt/fmt/riscv.h riscvpt_iommu_fmt_init(struct pt_iommu_riscv_64 *iommu_table, · line 291
Quoted 12 lines
drivers/iommu/generic_pt/fmt/riscv.h +8
-0@@ -263,6 +281,22 @@ riscvpt_iommu_fmt_init(struct pt_iommu_riscv_64 *iommu_table,case 57 :pt_top_set_level (&table->common,4 );break ; +/* +* Second-stage (iohgatp): Sv39x4 / Sv48x4 / Sv57x4. +* The top level is the same as for the first-stage counterpart. +*/ +case 41 : +pt_top_set_level (&table->common,2 ); + table->second_stage = true; +break ;Second stage needs to be an explicit
PT_FEATnot implicitly deduced based on the vasz.
Quoted 5 lines
Quoted 2 lines
Agreed. I will add an explicit
PT_FEAT_RISCV_SECOND_STAGEflag and stop deriving second-stage semantics from vasz.
PT_FEAT_RISCV_S2would match what I have for ARM
Thanks for the suggestion, I’ll use PT_FEAT_RISCV_S2 to match the ARM naming.
Fangyu
Quoted 1 line
Jason
RFC PATCH 2/11 iommu/riscv: report iommu capabilities
In reply to [email protected]From: Tomasz Jeznach <[email protected]>
Report RISC-V IOMMU capabilities required by VFIO subsystem to enable PCIe device assignment.
--- drivers/iommu/riscv/iommu.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
drivers/iommu/riscv/iommu.c +12 -0
@@ -1336,6 +1336,17 @@ static struct iommu_group *riscv_iommu_device_group(struct device *dev)return generic_device_group (dev); } +static bool riscv_iommu_capable (struct device *dev,enum iommu_cap cap) +{ +switch (cap) { +case IOMMU_CAP_CACHE_COHERENCY : +case IOMMU_CAP_DEFERRED_FLUSH : +return true; +default : +return false; + } +} +static int riscv_iommu_of_xlate (struct device *dev,const struct of_phandle_args *args) {return iommu_fwspec_add_ids (dev, args->args,1 );
@@ -1397,6 +1408,7 @@ static void riscv_iommu_release_device(struct device *dev)static const struct iommu_ops riscv_iommu_ops = { .of_xlate = riscv_iommu_of_xlate, + .capable = riscv_iommu_capable, .identity_domain = &riscv_iommu_identity_domain, .blocked_domain = &riscv_iommu_blocking_domain, .release_domain = &riscv_iommu_blocking_domain,
Signature · 1 line
2.50.1
Re: [RFC PATCH 02/11] iommu/riscv: report iommu capabilities
drivers/iommu/riscv/iommu.c static struct iommu_group *riscv_iommu_device_group(struct device *dev) · line 1343
Quoted 5 lines · On Tue, Apr 28, 2026 at 09:13:50PM +0800, [email protected] wrote:
drivers/iommu/riscv/iommu.c +5
-0+static bool riscv_iommu_capable (struct device *dev,enum iommu_cap cap) +{ +switch (cap) { +case IOMMU_CAP_CACHE_COHERENCY : +case IOMMU_CAP_DEFERRED_FLUSH :
IOMMU_CAP_DEFERRED_FLUSH is not needed in v7.1
Jason
Re: Re: [RFC PATCH 02/11] iommu/riscv: report iommu capabilities
Quoted 7 lines
drivers/iommu/riscv/iommu.c static struct iommu_group *riscv_iommu_device_group(struct device *dev) · line 1343
Quoted 5 lines
drivers/iommu/riscv/iommu.c +5
-0+static bool riscv_iommu_capable (struct device *dev,enum iommu_cap cap) +{ +switch (cap) { +case IOMMU_CAP_CACHE_COHERENCY : +case IOMMU_CAP_DEFERRED_FLUSH :
IOMMU_CAP_DEFERRED_FLUSHis not needed in v7.1
Thanks, I will drop IOMMU_CAP_DEFERRED_FLUSH.
Quoted 2 lines
Jason
RFC PATCH 3/11 iommu/riscv: use data structure instead of individual values
In reply to [email protected]From: Zong Li <[email protected]>
The parameter will be increased when we need to set up more bit fields in the device context. Use a data structure to wrap them up.
--- drivers/iommu/riscv/iommu.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-)
drivers/iommu/riscv/iommu.c +17 -10
@@ -1077,7 +1077,7 @@ static void riscv_iommu_iodir_iotinval(struct riscv_iommu_device *iommu,* interim translation faults. */ static void riscv_iommu_iodir_update (struct riscv_iommu_device *iommu, -struct device *dev,u64 fsc,u64 ta) +struct device *dev,struct riscv_iommu_dc *new_dc) {struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (dev);struct riscv_iommu_dc *dc;
@@ -1116,10 +1116,10 @@ static void riscv_iommu_iodir_update(struct riscv_iommu_device *iommu,for (i =0 ; i < fwspec->num_ids; i++) { dc =riscv_iommu_get_dc (iommu, fwspec->ids[i]); tc =READ_ONCE (dc->tc); - tc |= ta &RISCV_IOMMU_DC_TC_V ; + tc |= new_dc->ta &RISCV_IOMMU_DC_TC_V ; -WRITE_ONCE (dc->fsc, fsc); -WRITE_ONCE (dc->ta, ta &RISCV_IOMMU_PC_TA_PSCID ); +WRITE_ONCE (dc->fsc, new_dc->fsc); +WRITE_ONCE (dc->ta, new_dc->ta &RISCV_IOMMU_PC_TA_PSCID );/* Update device context, write TC.V as the last step. */ dma_wmb ();WRITE_ONCE (dc->tc, tc);
@@ -1205,22 +1205,22 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,struct riscv_iommu_device *iommu =dev_to_iommu (dev);struct riscv_iommu_info *info =dev_iommu_priv_get (dev);struct pt_iommu_riscv_64_hw_info pt_info; -u64 fsc, ta; +struct riscv_iommu_dc dc = {0 };pt_iommu_riscv_64_hw_info (&domain->riscvpt, &pt_info);if (!riscv_iommu_pt_supported (iommu, pt_info.fsc_iosatp_mode))return -ENODEV ; - fsc =FIELD_PREP (RISCV_IOMMU_PC_FSC_MODE , pt_info.fsc_iosatp_mode) | + dc.fsc =FIELD_PREP (RISCV_IOMMU_PC_FSC_MODE , pt_info.fsc_iosatp_mode) |FIELD_PREP (RISCV_IOMMU_PC_FSC_PPN , pt_info.ppn); - ta =FIELD_PREP (RISCV_IOMMU_PC_TA_PSCID , domain->pscid) | + dc.ta =FIELD_PREP (RISCV_IOMMU_PC_TA_PSCID , domain->pscid) |RISCV_IOMMU_PC_TA_V ;if (riscv_iommu_bond_link (domain, dev))return -ENOMEM ; -riscv_iommu_iodir_update (iommu, dev, fsc, ta); +riscv_iommu_iodir_update (iommu, dev, &dc);riscv_iommu_bond_unlink (info->domain, dev); info->domain = domain;
@@ -1292,9 +1292,12 @@ static int riscv_iommu_attach_blocking_domain(struct iommu_domain *iommu_domain, {struct riscv_iommu_device *iommu =dev_to_iommu (dev);struct riscv_iommu_info *info =dev_iommu_priv_get (dev); +struct riscv_iommu_dc dc = {0 }; + + dc.fsc =RISCV_IOMMU_FSC_BARE ;/* Make device context invalid, translation requests will fault w/ #258 */ -riscv_iommu_iodir_update (iommu, dev,RISCV_IOMMU_FSC_BARE ,0 ); +riscv_iommu_iodir_update (iommu, dev, &dc);riscv_iommu_bond_unlink (info->domain, dev); info->domain =NULL ;
@@ -1314,8 +1317,12 @@ static int riscv_iommu_attach_identity_domain(struct iommu_domain *iommu_domain, {struct riscv_iommu_device *iommu =dev_to_iommu (dev);struct riscv_iommu_info *info =dev_iommu_priv_get (dev); +struct riscv_iommu_dc dc = {0 }; + + dc.fsc =RISCV_IOMMU_FSC_BARE ; + dc.ta =RISCV_IOMMU_PC_TA_V ; -riscv_iommu_iodir_update (iommu, dev,RISCV_IOMMU_FSC_BARE ,RISCV_IOMMU_PC_TA_V ); +riscv_iommu_iodir_update (iommu, dev, &dc);riscv_iommu_bond_unlink (info->domain, dev); info->domain =NULL ;
Signature · 1 line
2.50.1
RFC PATCH 4/11 iommu/riscv: support GSCID and GVMA invalidation command
In reply to [email protected]From: Zong Li <[email protected]>
This patch adds a ID Allocator for GSCID and a wrap for setting up GSCID in IOTLB invalidation command.
Set up iohgatp to enable second stage table and flush stage-2 table if the GSCID is set.
The GSCID of domain should be freed when release domain. GSCID will be allocated for parent domain in nested IOMMU process.
--- drivers/iommu/riscv/iommu-bits.h | 7 +++++++ drivers/iommu/riscv/iommu.c | 32 ++++++++++++++++++++++++++------ 2 files changed, 33 insertions(+), 6 deletions(-)
drivers/iommu/riscv/iommu-bits.h +7 -0
@@ -716,6 +716,13 @@ static inline void riscv_iommu_cmd_inval_vma(struct riscv_iommu_command *cmd) cmd->dword1 =0 ; } +static inline void riscv_iommu_cmd_inval_gvma (struct riscv_iommu_command *cmd) +{ + cmd->dword0 =FIELD_PREP (RISCV_IOMMU_CMD_OPCODE ,RISCV_IOMMU_CMD_IOTINVAL_OPCODE ) | +FIELD_PREP (RISCV_IOMMU_CMD_FUNC ,RISCV_IOMMU_CMD_IOTINVAL_FUNC_GVMA ); + cmd->dword1 =0 ; +} +static inline void riscv_iommu_cmd_inval_set_addr (struct riscv_iommu_command *cmd,u64 addr) {
drivers/iommu/riscv/iommu.c +26 -6
@@ -48,6 +48,10 @@static DEFINE_IDA (riscv_iommu_pscids);#define RISCV_IOMMU_MAX_PSCID (BIT(20) - 1) +/* IOMMU GSCID allocation namespace. */ +static DEFINE_IDA (riscv_iommu_gscids); +#define RISCV_IOMMU_MAX_GSCID (BIT(16) - 1) +/* Device resource-managed allocations */ struct riscv_iommu_devres {void *addr;
@@ -819,6 +823,7 @@ struct riscv_iommu_domain {struct list_head bonds;spinlock_t lock;/* protect bonds list updates. */ int pscid; +int gscid; }; PT_IOMMU_CHECK_DOMAIN(struct riscv_iommu_domain, riscvpt.iommu, domain);
@@ -967,15 +972,20 @@ static void riscv_iommu_iotlb_inval(struct riscv_iommu_domain *domain,/* * IOTLB invalidation request can be safely omitted if already sent -* to the IOMMU for the same PSCID, and with domain->bonds list +* to the IOMMU for the same PSCID/GSCID, and with domain->bonds list * arranged based on the device's IOMMU, it's sufficient to check * last device the invalidation was sent to. */ if (iommu == prev)continue ; -riscv_iommu_cmd_inval_vma (&cmd); -riscv_iommu_cmd_inval_set_pscid (&cmd, domain->pscid); +if (domain->gscid) { +riscv_iommu_cmd_inval_gvma (&cmd); +riscv_iommu_cmd_inval_set_gscid (&cmd, domain->gscid); + }else { +riscv_iommu_cmd_inval_vma (&cmd); +riscv_iommu_cmd_inval_set_pscid (&cmd, domain->pscid); + }if (end - start <RISCV_IOMMU_IOTLB_INVAL_LIMIT -1 ) {unsigned long iova = start;
@@ -1120,6 +1130,7 @@ static void riscv_iommu_iodir_update(struct riscv_iommu_device *iommu,WRITE_ONCE (dc->fsc, new_dc->fsc);WRITE_ONCE (dc->ta, new_dc->ta &RISCV_IOMMU_PC_TA_PSCID ); +WRITE_ONCE (dc->iohgatp, new_dc->iohgatp);/* Update device context, write TC.V as the last step. */ dma_wmb ();WRITE_ONCE (dc->tc, tc);
@@ -1175,8 +1186,10 @@ static void riscv_iommu_free_paging_domain(struct iommu_domain *iommu_domain)WARN_ON (!list_empty (&domain->bonds)); -if ((int )domain->pscid >0 ) +if (domain->pscid >0 )ida_free (&riscv_iommu_pscids, domain->pscid); +if (domain->gscid >0 ) +ida_free (&riscv_iommu_gscids, domain->gscid);pt_iommu_deinit (&domain->riscvpt.iommu);kfree (domain);
@@ -1212,8 +1225,15 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,if (!riscv_iommu_pt_supported (iommu, pt_info.fsc_iosatp_mode))return -ENODEV ; - dc.fsc =FIELD_PREP (RISCV_IOMMU_PC_FSC_MODE , pt_info.fsc_iosatp_mode) | -FIELD_PREP (RISCV_IOMMU_PC_FSC_PPN , pt_info.ppn); +if (domain->gscid) { + dc.iohgatp =FIELD_PREP (RISCV_IOMMU_DC_IOHGATP_MODE , pt_info.iohgatp_mode) | +FIELD_PREP (RISCV_IOMMU_DC_IOHGATP_GSCID , domain->gscid) | +FIELD_PREP (RISCV_IOMMU_DC_IOHGATP_PPN , pt_info.ppn); + }else { + dc.fsc =FIELD_PREP (RISCV_IOMMU_PC_FSC_MODE , pt_info.fsc_iosatp_mode) | +FIELD_PREP (RISCV_IOMMU_PC_FSC_PPN , pt_info.ppn); + } + dc.ta =FIELD_PREP (RISCV_IOMMU_PC_TA_PSCID , domain->pscid) |RISCV_IOMMU_PC_TA_V ;
Signature · 1 line
2.50.1
RFC PATCH 5/11 RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch
In reply to [email protected]From: Tomasz Jeznach <[email protected]>
Enable KVM/VFIO support on RISC-V architecture.
--- arch/riscv/kvm/Kconfig | 2 ++ 1 file changed, 2 insertions(+)
arch/riscv/kvm/Kconfig +2 -0
@@ -30,8 +30,10 @@ config KVM select KVM_GENERIC_HARDWARE_ENABLING select KVM_MMIO select VIRT_XFER_TO_GUEST_WORK + select KVM_VFIO select SCHED_INFO select GUEST_PERF_EVENTS if PERF_EVENTS + select SRCU help Support hosting virtualized guest machines.
Signature · 1 line
2.50.1
RFC PATCH 6/11 iommu/riscv: Add domain_alloc_paging_flags for second-stage domain
In reply to [email protected]From: Fangyu Yu <[email protected]>
Replace .domain_alloc_paging with .domain_alloc_paging_flags so callers can pass allocation flags to select the appropriate page-table type.
When IOMMU_HWPT_ALLOC_NEST_PARENT or IOMMU_HWPT_ALLOC_DIRTY_TRACKING is set in @flags, allocate a second-stage (iohgatp) domain.
When @flags is 0 the behaviour is identical to the previousdomain_alloc_paging: first-stage (iosatp) domain.
--- drivers/iommu/riscv/iommu.c | 66 ++++++++++++++++++++++++++++--------- 1 file changed, 51 insertions(+), 15 deletions(-)
drivers/iommu/riscv/iommu.c +51 -15
@@ -1255,23 +1255,50 @@ static const struct iommu_domain_ops riscv_iommu_paging_domain_ops = { .flush_iotlb_all = riscv_iommu_iotlb_flush_all, }; -static struct iommu_domain *riscv_iommu_alloc_paging_domain (struct device *dev) +static struct iommu_domain *riscv_iommu_domain_alloc_paging_flags ( +struct device *dev,u32 flags, +const struct iommu_user_data *user_data) { +const bool second_stage = flags & + (IOMMU_HWPT_ALLOC_NEST_PARENT |IOMMU_HWPT_ALLOC_DIRTY_TRACKING );struct pt_iommu_riscv_64_cfg cfg = {};struct riscv_iommu_domain *domain;struct riscv_iommu_device *iommu;int ret; +if (user_data) +return ERR_PTR (-EOPNOTSUPP ); + iommu =dev_to_iommu (dev); -if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV57 ) { - cfg.common.hw_max_vasz_lg2 =57 ; - }else if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV48 ) { - cfg.common.hw_max_vasz_lg2 =48 ; - }else if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV39 ) { - cfg.common.hw_max_vasz_lg2 =39 ; + +if (second_stage) { +/* +* Second-stage (iohgatp) page table for KVM VFIO device +* pass-through and dirty tracking. The GPA space is 2 bits +* wider than the corresponding first-stage VA space (x4 root +* page table), so hw_max_vasz_lg2 values are 41/50/59. +*/ +if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV57X4 ) { + cfg.common.hw_max_vasz_lg2 =59 ; + }else if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV48X4 ) { + cfg.common.hw_max_vasz_lg2 =50 ; + }else if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV39X4 ) { + cfg.common.hw_max_vasz_lg2 =41 ; + }else { +dev_err (dev,"cannot find supported second-stage page table mode\n" ); +return ERR_PTR (-ENODEV ); + } }else { -dev_err (dev,"cannot find supported page table mode\n" ); -return ERR_PTR (-ENODEV ); +if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV57 ) { + cfg.common.hw_max_vasz_lg2 =57 ; + }else if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV48 ) { + cfg.common.hw_max_vasz_lg2 =48 ; + }else if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV39 ) { + cfg.common.hw_max_vasz_lg2 =39 ; + }else { +dev_err (dev,"cannot find supported page table mode\n" ); +return ERR_PTR (-ENODEV ); + } } cfg.common.hw_max_oasz_lg2 =56 ;
@@ -1291,11 +1318,20 @@ static struct iommu_domain *riscv_iommu_alloc_paging_domain(struct device *dev) domain->riscvpt.iommu.nid =dev_to_node (iommu->dev); domain->domain.ops = &riscv_iommu_paging_domain_ops; - domain->pscid =ida_alloc_range (&riscv_iommu_pscids,1 , -RISCV_IOMMU_MAX_PSCID ,GFP_KERNEL ); -if (domain->pscid <0 ) { -riscv_iommu_free_paging_domain (&domain->domain); -return ERR_PTR (-ENOMEM ); +if (second_stage) { + domain->gscid =ida_alloc_range (&riscv_iommu_gscids,1 , +RISCV_IOMMU_MAX_GSCID ,GFP_KERNEL ); +if (domain->gscid <0 ) { +riscv_iommu_free_paging_domain (&domain->domain); +return ERR_PTR (-ENOMEM ); + } + }else { + domain->pscid =ida_alloc_range (&riscv_iommu_pscids,1 , +RISCV_IOMMU_MAX_PSCID ,GFP_KERNEL ); +if (domain->pscid <0 ) { +riscv_iommu_free_paging_domain (&domain->domain); +return ERR_PTR (-ENOMEM ); + } } ret =pt_iommu_riscv_64_init (&domain->riscvpt, &cfg,GFP_KERNEL );
@@ -1439,7 +1475,7 @@ static const struct iommu_ops riscv_iommu_ops = { .identity_domain = &riscv_iommu_identity_domain, .blocked_domain = &riscv_iommu_blocking_domain, .release_domain = &riscv_iommu_blocking_domain, - .domain_alloc_paging = riscv_iommu_alloc_paging_domain, + .domain_alloc_paging_flags = riscv_iommu_domain_alloc_paging_flags, .device_group = riscv_iommu_device_group, .probe_device = riscv_iommu_probe_device, .release_device = riscv_iommu_release_device,
Signature · 1 line
2.50.1
Re: [RFC PATCH 06/11] iommu/riscv: Add domain_alloc_paging_flags for second-stage domain
Quoted 21 lines · On Tue, Apr 28, 2026 at 09:13:54PM +0800, [email protected] wrote:
From: Fangyu Yu <[email protected]>
Replace .domain_alloc_paging with .domain_alloc_paging_flags so callers can pass allocation flags to select the appropriate page-table type.
When
IOMMU_HWPT_ALLOC_NEST_PARENTorIOMMU_HWPT_ALLOC_DIRTY_TRACKINGis set in @flags, allocate a second-stage (iohgatp) domain.When @flags is 0 the behaviour is identical to the previous
domain_alloc_paging: first-stage (iosatp) domain.Signed-off-by Fangyu Yu <[email protected]>--- drivers/iommu/riscv/iommu.c | 66 ++++++++++++++++++++++++++++--------- 1 file changed, 51 insertions(+), 15 deletions(-)drivers/iommu/riscv/iommu.c +0
-0
drivers/iommu/riscv/iommu.c static const struct iommu_domain_ops riscv_iommu_paging_domain_ops = { · line 1263
Quoted 10 lines
drivers/iommu/riscv/iommu.c +5
-1@@ -1255,23 +1255,50 @@ static const struct iommu_domain_ops riscv_iommu_paging_domain_ops = { .flush_iotlb_all = riscv_iommu_iotlb_flush_all, }; -static struct iommu_domain *riscv_iommu_alloc_paging_domain (struct device *dev) +static struct iommu_domain *riscv_iommu_domain_alloc_paging_flags ( +struct device *dev,u32 flags, +const struct iommu_user_data *user_data) { +const bool second_stage = flags & + (IOMMU_HWPT_ALLOC_NEST_PARENT |IOMMU_HWPT_ALLOC_DIRTY_TRACKING );
This isn't the right logic, you should follow the switch/case design from other drivers.
Quoted 22 lines
drivers/iommu/riscv/iommu.c +10
-6struct pt_iommu_riscv_64_cfg cfg = {};struct riscv_iommu_domain *domain;struct riscv_iommu_device *iommu;int ret; +if (user_data) +return ERR_PTR (-EOPNOTSUPP ); + iommu =dev_to_iommu (dev); -if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV57 ) { - cfg.common.hw_max_vasz_lg2 =57 ; - }else if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV48 ) { - cfg.common.hw_max_vasz_lg2 =48 ; - }else if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV39 ) { - cfg.common.hw_max_vasz_lg2 =39 ; + +if (second_stage) { +/* +* Second-stage (iohgatp) page table for KVM VFIO device +* pass-through and dirty tracking. The GPA space is 2 bits +* wider than the corresponding first-stage VA space (x4 root +* page table), so hw_max_vasz_lg2 values are 41/50/59.
drivers/iommu/riscv/iommu.c static const struct iommu_domain_ops riscv_iommu_paging_domain_ops = { · line 1289
Quoted 10 lines
drivers/iommu/riscv/iommu.c +10
-0+*/ +if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV57X4 ) { + cfg.common.hw_max_vasz_lg2 =59 ; + }else if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV48X4 ) { + cfg.common.hw_max_vasz_lg2 =50 ; + }else if (iommu->caps &RISCV_IOMMU_CAPABILITIES_SV39X4 ) { + cfg.common.hw_max_vasz_lg2 =41 ; + }else { +dev_err (dev,"cannot find supported second-stage page table mode\n" ); +return ERR_PTR (-ENODEV );
Do not make log messages for failing system calls.
Jason
Re: Re: [RFC PATCH 06/11] iommu/riscv: Add domain_alloc_paging_flags for second-stage domain
Understood. I will rework the flags handling to follow the switch/case pattern used by other drivers, and remove the dev_err() on the failure path.
Thanks,
Fangyu
Quoted 1 line
Jason
RFC PATCH 7/11 iommupt: Don't preset D when RISC-V IOMMU dirty tracking on
In reply to [email protected]From: Fangyu Yu <[email protected]>
When mapping writable pages, the RISC-V format code currently pre-sets the PTE D bit unconditionally.
If hardware dirty tracking is active (DC.tc.GADE set), the IOMMU sets D autonomously on the first write. Pre-setting D makes every new mapping appear dirty immediately and breaks dirty tracking.
Introduce PT_FEAT_RISCV_DIRTY_TRACKING_ACTIVE and, when set, leave D cleared for new writable mappings so hardware can capture the first write. Keep pre-setting D when dirty tracking is inactive.
Only meaningful for second-stage (iohgatp) page tables.
--- drivers/iommu/generic_pt/fmt/riscv.h | 13 +++++++++++-- include/linux/generic_pt/common.h | 8 ++++++++ 2 files changed, 19 insertions(+), 2 deletions(-)
drivers/iommu/generic_pt/fmt/riscv.h +11 -2
@@ -248,8 +248,17 @@ static inline int riscvpt_iommu_set_prot(struct pt_common *common,u64 pte; pte =RISCVPT_A |RISCVPT_U ; -if (iommu_prot &IOMMU_WRITE ) - pte |=RISCVPT_W |RISCVPT_R |RISCVPT_D ; +if (iommu_prot &IOMMU_WRITE ) { + pte |=RISCVPT_W |RISCVPT_R ; +/* +* When hardware dirty tracking is active (GADE set), the IOMMU +* sets the D bit autonomously on the first write access. +* +*/ +if (!(common->features & +BIT (PT_FEAT_RISCV_DIRTY_TRACKING_ACTIVE ))) + pte |=RISCVPT_D ; + }if (iommu_prot &IOMMU_READ ) pte |=RISCVPT_R ;if (!(iommu_prot &IOMMU_NOEXEC ))
include/linux/generic_pt/common.h +8 -0
@@ -193,6 +193,14 @@ enum {* Support the 64k contiguous page size following the Svnapot extension. */ PT_FEAT_RISCV_SVNAPOT_64K =PT_FEAT_FMT_START , +/* +* Hardware dirty tracking is currently active: DC.tc.GADE is set and +* the IOMMU will set the D bit in PTEs autonomously on write access. +* When this flag is set, new mappings must not pre-set the D bit so +* that every write is correctly captured by hardware. +* Only meaningful for second-stage (iohgatp) page tables. +*/ + PT_FEAT_RISCV_DIRTY_TRACKING_ACTIVE, };
Signature · 1 line
2.50.1
Re: [RFC PATCH 07/11] iommupt: Don't preset D when RISC-V IOMMU dirty tracking on
Quoted 9 lines · On Tue, Apr 28, 2026 at 09:13:55PM +0800, [email protected] wrote:
From: Fangyu Yu <[email protected]>
When mapping writable pages, the RISC-V format code currently pre-sets the PTE D bit unconditionally.
If hardware dirty tracking is active (DC.tc.GADE set), the IOMMU sets D autonomously on the first write. Pre-setting D makes every new mapping appear dirty immediately and breaks dirty tracking.
Quoted 5 lines
Introduce
PT_FEAT_RISCV_DIRTY_TRACKING_ACTIVEand, when set, leave D cleared for new writable mappings so hardware can capture the first write. Keep pre-setting D when dirty tracking is inactive.Only meaningful for second-stage (iohgatp) page tables.
You shouldn't need anything like this, the D bit is managed by the iommufd core appropriately. It *should* start out pre-set as that is faster when not tacking. Only once dirty tracking is started does D get cleared. User space is supposed to assume that everything is dirty prior to its first D clear.
Jason
Re: Re: [RFC PATCH 07/11] iommupt: Don't preset D when RISC-V IOMMU dirty tracking on
Quoted 13 lines
Quoted 13 lines
From: Fangyu Yu <[email protected]>
When mapping writable pages, the RISC-V format code currently pre-sets the PTE D bit unconditionally.
If hardware dirty tracking is active (DC.tc.GADE set), the IOMMU sets D autonomously on the first write. Pre-setting D makes every new mapping appear dirty immediately and breaks dirty tracking.
Introduce
PT_FEAT_RISCV_DIRTY_TRACKING_ACTIVEand, when set, leave D cleared for new writable mappings so hardware can capture the first write. Keep pre-setting D when dirty tracking is inactive.
Quoted 8 lines
Quoted 1 line
Only meaningful for second-stage (iohgatp) page tables.
You shouldn't need anything like this, the D bit is managed by the iommufd core appropriately. It *should* start out pre-set as that is faster when not tacking. Only once dirty tracking is started does D get cleared. User space is supposed to assume that everything is dirty prior to its first D clear.
Thanks, that makes sense. I will drop PT_FEAT_RISCV_DIRTY_TRACKING_ACTIVE and rely on the iommufd core to manage D bit.
Fangyu
Quoted 1 line
Jason
RFC PATCH 8/11 iommu/riscv: Add dirty tracking support for second-stage domains
In reply to [email protected]From: Fangyu Yu <[email protected]>
Add hardware dirty tracking support for second-stage (iohgatp) domains used in KVM VFIO device pass-through.
The RISC-V IOMMU can automatically set the dirty bit in PTEs on write access when DC.tc.GADE is set and the hardware has AMO_HWAD capability. Wire this up to the iommufd dirty tracking interface:
- riscv_iommu_set_dirty_tracking(): Walks all bonds of the domain and sets or clears DC.tc.GADE in each device context entry.
- riscv_iommu_dirty_ops: Exposes set_dirty_tracking and the generic page-table read_and_clear_dirty via IOMMU_PT_DIRTY_OPS(riscv_64).
- domain_alloc_paging_flags: Assigns dirty_ops to second-stage domains when AMO_HWAD is advertised in hardware capabilities.
- riscv_iommu_capable: Reports IOMMU_CAP_DIRTY_TRACKING when AMO_HWAD is present.
--- drivers/iommu/riscv/iommu.c | 84 +++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+)
drivers/iommu/riscv/iommu.c +84 -0
@@ -1247,6 +1247,84 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,return 0 ; } +/* +* Enable or disable hardware A/D bit updates (GADE) in the device context for +* all devices attached to a second-stage domain. When dirty tracking is +* enabled the IOMMU hardware will set the dirty bit in PTEs on write access, +* making them visible to read_and_clear_dirty(). +*/ +static int riscv_iommu_set_dirty_tracking (struct iommu_domain *iommu_domain, +bool enable) +{ +struct riscv_iommu_domain *domain =iommu_domain_to_riscv (iommu_domain); +struct riscv_iommu_bond *bond; +struct riscv_iommu_device *iommu, *prev; +struct riscv_iommu_dc *dc; +struct iommu_fwspec *fwspec; +struct riscv_iommu_command cmd; +u64 tc; +int i; + +rcu_read_lock (); + +list_for_each_entry_rcu (bond, &domain->bonds, list) { + iommu =dev_to_iommu (bond->dev); + fwspec =dev_iommu_fwspec_get (bond->dev); + +for (i =0 ; i < fwspec->num_ids; i++) { + dc =riscv_iommu_get_dc (iommu, fwspec->ids[i]); + tc =READ_ONCE (dc->tc); +if (!(tc &RISCV_IOMMU_DC_TC_V )) +continue ; + +if (enable) + tc |=RISCV_IOMMU_DC_TC_GADE ; +else + tc &= ~RISCV_IOMMU_DC_TC_GADE ; +WRITE_ONCE (dc->tc, tc); + +/* Invalidate cached device context entry */ +riscv_iommu_cmd_iodir_inval_ddt (&cmd); +riscv_iommu_cmd_iodir_set_did (&cmd, fwspec->ids[i]); +riscv_iommu_cmd_send (iommu, &cmd); +riscv_iommu_iodir_iotinval (iommu, false, dc->iohgatp, dc,NULL ); + } + } + + prev =NULL ; +list_for_each_entry_rcu (bond, &domain->bonds, list) { + iommu =dev_to_iommu (bond->dev); +if (iommu == prev) +continue ; + +riscv_iommu_cmd_sync (iommu,RISCV_IOMMU_IOTINVAL_TIMEOUT ); + prev = iommu; + } + +rcu_read_unlock (); + +/* +* Reflect the active dirty-tracking state in the page table feature +* flags. When active, riscvpt_iommu_set_prot() will leave D=0 in +* new mappings so that the hardware can set it on the first write, +* providing accurate per-page dirty information. When inactive, +* new mappings get D=1 to avoid write faults on a D=0 PTE. +*/ +if (enable) + domain->riscvpt.riscv_64pt.common.features |= +BIT (PT_FEAT_RISCV_DIRTY_TRACKING_ACTIVE ); +else + domain->riscvpt.riscv_64pt.common.features &= + ~BIT (PT_FEAT_RISCV_DIRTY_TRACKING_ACTIVE ); + +return 0 ; +} + +static const struct iommu_dirty_ops riscv_iommu_dirty_ops = { +IOMMU_PT_DIRTY_OPS (riscv_64), + .set_dirty_tracking = riscv_iommu_set_dirty_tracking, +}; +static const struct iommu_domain_ops riscv_iommu_paging_domain_ops = {IOMMU_PT_DOMAIN_OPS (riscv_64), .attach_dev = riscv_iommu_attach_paging_domain,
@@ -1325,6 +1403,8 @@ static struct iommu_domain *riscv_iommu_domain_alloc_paging_flags(riscv_iommu_free_paging_domain (&domain->domain);return ERR_PTR (-ENOMEM ); } +if (iommu->caps &RISCV_IOMMU_CAPABILITIES_AMO_HWAD ) + domain->domain.dirty_ops = &riscv_iommu_dirty_ops; }else { domain->pscid =ida_alloc_range (&riscv_iommu_pscids,1 ,RISCV_IOMMU_MAX_PSCID ,GFP_KERNEL );
@@ -1401,10 +1481,14 @@ static struct iommu_group *riscv_iommu_device_group(struct device *dev)static bool riscv_iommu_capable (struct device *dev,enum iommu_cap cap) { +struct riscv_iommu_device *iommu =dev_to_iommu (dev); +switch (cap) {case IOMMU_CAP_CACHE_COHERENCY :case IOMMU_CAP_DEFERRED_FLUSH :return true; +case IOMMU_CAP_DIRTY_TRACKING : +return !!(iommu->caps &RISCV_IOMMU_CAPABILITIES_AMO_HWAD );default :return false; }
Signature · 1 line
2.50.1
Re: [RFC PATCH 08/11] iommu/riscv: Add dirty tracking support for second-stage domains
Quoted 29 lines · On Tue, Apr 28, 2026 at 09:13:56PM +0800, [email protected] wrote:
drivers/iommu/riscv/iommu.c +25
-0@@ -1247,6 +1247,84 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,return 0 ; } +/* +* Enable or disable hardware A/D bit updates (GADE) in the device context for +* all devices attached to a second-stage domain. When dirty tracking is +* enabled the IOMMU hardware will set the dirty bit in PTEs on write access, +* making them visible to read_and_clear_dirty(). +*/ +static int riscv_iommu_set_dirty_tracking (struct iommu_domain *iommu_domain, +bool enable) +{ +struct riscv_iommu_domain *domain =iommu_domain_to_riscv (iommu_domain); +struct riscv_iommu_bond *bond; +struct riscv_iommu_device *iommu, *prev; +struct riscv_iommu_dc *dc; +struct iommu_fwspec *fwspec; +struct riscv_iommu_command cmd; +u64 tc; +int i; + +rcu_read_lock (); + +list_for_each_entry_rcu (bond, &domain->bonds, list) { + iommu =dev_to_iommu (bond->dev); + fwspec =dev_iommu_fwspec_get (bond->dev); + +for (i =0 ; i < fwspec->num_ids; i++) {
drivers/iommu/riscv/iommu.c static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain, · line 1284
Quoted 10 lines
drivers/iommu/riscv/iommu.c +10
-0+ dc =riscv_iommu_get_dc (iommu, fwspec->ids[i]); + tc =READ_ONCE (dc->tc); +if (!(tc &RISCV_IOMMU_DC_TC_V )) +continue ; + +if (enable) + tc |=RISCV_IOMMU_DC_TC_GADE ; +else + tc &= ~RISCV_IOMMU_DC_TC_GADE ; +WRITE_ONCE (dc->tc, tc);
I'm pretty sure you don't need to do this. Just preset GADE when ever a S2 domain is attached, rely on the pre-set D to avoid any HW cost and you are fine. No need to change it dynamically unless something is reall weird about riscv.
Jason
Re: Re: [RFC PATCH 08/11] iommu/riscv: Add dirty tracking support for second-stage domains
Quoted 36 lines
Quoted 36 lines
drivers/iommu/riscv/iommu.c +32
-0@@ -1247,6 +1247,84 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,return 0 ; } +/* +* Enable or disable hardware A/D bit updates (GADE) in the device context for +* all devices attached to a second-stage domain. When dirty tracking is +* enabled the IOMMU hardware will set the dirty bit in PTEs on write access, +* making them visible to read_and_clear_dirty(). +*/ +static int riscv_iommu_set_dirty_tracking (struct iommu_domain *iommu_domain, +bool enable) +{ +struct riscv_iommu_domain *domain =iommu_domain_to_riscv (iommu_domain); +struct riscv_iommu_bond *bond; +struct riscv_iommu_device *iommu, *prev; +struct riscv_iommu_dc *dc; +struct iommu_fwspec *fwspec; +struct riscv_iommu_command cmd; +u64 tc; +int i; + +rcu_read_lock (); + +list_for_each_entry_rcu (bond, &domain->bonds, list) { + iommu =dev_to_iommu (bond->dev); + fwspec =dev_iommu_fwspec_get (bond->dev); + +for (i =0 ; i < fwspec->num_ids; i++) { + dc =riscv_iommu_get_dc (iommu, fwspec->ids[i]); + tc =READ_ONCE (dc->tc); +if (!(tc &RISCV_IOMMU_DC_TC_V )) +continue ; + +if (enable) + tc |=RISCV_IOMMU_DC_TC_GADE ;
Quoted 9 lines
drivers/iommu/riscv/iommu.c static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain, · line 1284
Quoted 3 lines
drivers/iommu/riscv/iommu.c +3
-0+else + tc &= ~RISCV_IOMMU_DC_TC_GADE ; +WRITE_ONCE (dc->tc, tc);I'm pretty sure you don't need to do this. Just preset GADE when ever a S2 domain is attached, rely on the pre-set D to avoid any HW cost and you are fine. No need to change it dynamically unless something is reall weird about riscv.
Thanks, that’s a good suggestion. I will follow that approach: preset GADE on second-stage domain attach and rely on the core-managed D-bit behavior.
Fangyu
Quoted 1 line
Jason
RFC PATCH 9/11 iommu/riscv: Add IOTINVAL.GVMA after updating DDT/PDT entries
In reply to [email protected]From: Fangyu Yu <[email protected]>
Previously, only IOTINVAL.VMA was issued, which is insufficient for second-stage address translation consistency.
--- drivers/iommu/riscv/iommu.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/iommu/riscv/iommu.c +6 -3
@@ -1065,12 +1065,15 @@ static void riscv_iommu_iodir_iotinval(struct riscv_iommu_device *iommu,/* * else: IOTINVAL.VMA with GV=1,AV=PSCV=0,and * GSCID=DC.iohgatp.GSCID -* +*/ +riscv_iommu_cmd_send (iommu, &cmd); +/* * IOTINVAL.GVMA with GV=1,AV=0,and * GSCID=DC.iohgatp.GSCID -* TODO: For now, the Second-Stage feature have not yet been merged, -* also issue IOTINVAL.GVMA once second-stage support is merged. */ +riscv_iommu_cmd_inval_gvma (&cmd); +riscv_iommu_cmd_inval_set_gscid (&cmd, +FIELD_GET (RISCV_IOMMU_DC_IOHGATP_GSCID , iohgatp)); }riscv_iommu_cmd_send (iommu, &cmd); }
Signature · 1 line
2.50.1
RFC PATCH 10/11 iommupt: Add RISC-V dirty tracking PTE ops
In reply to [email protected]From: Fangyu Yu <[email protected]>
Implement the three dirty-tracking hooks required by the generic page table framework for the RISC-V format:
pt_entry_is_write_dirty():
Check the D bit (bit 7) in the PTE.
pt_entry_make_write_clean():
Clear the D bit across the full contiguous range.
pt_entry_make_write_dirty():
Atomically set D via try_cmpxchg64() on a single PTE.
--- drivers/iommu/generic_pt/fmt/riscv.h | 43 ++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
drivers/iommu/generic_pt/fmt/riscv.h +43 -0
@@ -222,6 +222,49 @@ static inline void riscvpt_attr_from_entry(const struct pt_state *pts, }#define pt_attr_from_entry riscvpt_attr_from_entry +/* +* Dirty tracking: RISC-V PTEs use D (bit 7) as the hardware dirty bit. +* When Svnapot 64K is active a leaf entry spans 16 consecutive PTEs; we +* must check / clear all of them so that no dirty indication is lost. +*/ +static inline bool riscvpt_entry_is_write_dirty (const struct pt_state *pts) +{ +unsigned int num_contig_lg2 =riscvpt_entry_num_contig_lg2 (pts); +const pt_riscv_entry_t *tablep = +pt_cur_table (pts, pt_riscv_entry_t) + +log2_set_mod (pts->index,0 , num_contig_lg2); +const pt_riscv_entry_t *end = tablep +log2_to_int (num_contig_lg2); + +for (; tablep != end; tablep++) +if (READ_ONCE (*tablep) &RISCVPT_D ) +return true; +return false; +} +#define pt_entry_is_write_dirty riscvpt_entry_is_write_dirty + +static inline void riscvpt_entry_make_write_clean (struct pt_state *pts) +{ +unsigned int num_contig_lg2 =riscvpt_entry_num_contig_lg2 (pts); +pt_riscv_entry_t *tablep = +pt_cur_table (pts, pt_riscv_entry_t) + +log2_set_mod (pts->index,0 , num_contig_lg2); +pt_riscv_entry_t *end = tablep +log2_to_int (num_contig_lg2); + +for (; tablep != end; tablep++) +WRITE_ONCE (*tablep,READ_ONCE (*tablep) & ~(pt_riscv_entry_t )RISCVPT_D ); +} +#define pt_entry_make_write_clean riscvpt_entry_make_write_clean + +static inline bool riscvpt_entry_make_write_dirty (struct pt_state *pts) +{ +pt_riscv_entry_t *tablep = +pt_cur_table (pts, pt_riscv_entry_t) + pts->index; +pt_riscv_entry_t new = pts->entry |RISCVPT_D ; + +return try_cmpxchg64 (tablep, &pts->entry, new); +} +#define pt_entry_make_write_dirty riscvpt_entry_make_write_dirty +/* --- iommu */ #include <linux/generic_pt/iommu.h> #include <linux/iommu.h>
Signature · 1 line
2.50.1
Re: [RFC PATCH 10/11] iommupt: Add RISC-V dirty tracking PTE ops
Quoted 11 lines · On Tue, Apr 28, 2026 at 09:13:58PM +0800, [email protected] wrote:
From: Fangyu Yu <[email protected]>
Implement the three dirty-tracking hooks required by the generic page table framework for the RISC-V format:
pt_entry_is_write_dirty():
Check the D bit (bit 7) in the PTE.
pt_entry_make_write_clean():
Clear the D bit across the full contiguous range.
Quoted 7 lines
pt_entry_make_write_dirty():
Atomically set D viatry_cmpxchg64()on a single PTE.Signed-off-by Fangyu Yu <[email protected]>--- drivers/iommu/generic_pt/fmt/riscv.h | 43 ++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
This patch should probably go earlier in your series, before adding the alloc_paging flags at least.
Jason
Re: Re: [RFC PATCH 10/11] iommupt: Add RISC-V dirty tracking PTE ops
Quoted 14 lines
Quoted 14 lines
From: Fangyu Yu <[email protected]>
Implement the three dirty-tracking hooks required by the generic page table framework for the RISC-V format:
pt_entry_is_write_dirty():
Check the D bit (bit 7) in the PTE.
pt_entry_make_write_clean():
Clear the D bit across the full contiguous range.
pt_entry_make_write_dirty():
Atomically set D viatry_cmpxchg64()on a single PTE.
Quoted 8 lines
Quoted 4 lines
Signed-off-by Fangyu Yu <[email protected]>--- drivers/iommu/generic_pt/fmt/riscv.h | 43 ++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)This patch should probably go earlier in your series, before adding the
alloc_pagingflags at least.
Agreed. I will reorder the series and place this patch earlier.
Thanks,
Fangyu
Quoted 1 line
Jason
RFC PATCH 11/11 iommu/riscv: support nested iommu for getting iommu hardware information
In reply to [email protected]From: Zong Li <[email protected]>
This patch implements .hw_info operation and the related data structures for passing the IOMMU hardware capabilities for iommufd.
--- drivers/iommu/riscv/iommu.c | 19 +++++++++++++++++++ include/uapi/linux/iommufd.h | 18 ++++++++++++++++++ 2 files changed, 37 insertions(+)
drivers/iommu/riscv/iommu.c +19 -0
@@ -1556,8 +1556,27 @@ static void riscv_iommu_release_device(struct device *dev)kfree_rcu_mightsleep (info); } +static void *riscv_iommu_hw_info (struct device *dev,u32 *length,u32 *type) +{ +struct riscv_iommu_device *iommu =dev_to_iommu (dev); +struct iommu_hw_info_riscv_iommu *info; + + info =kzalloc_obj (*info); +if (!info) +return ERR_PTR (-ENOMEM ); + + info->capability = iommu->caps; + info->fctl =riscv_iommu_readl (iommu,RISCV_IOMMU_REG_FCTL ); + + *length =sizeof (*info); + *type =IOMMU_HW_INFO_TYPE_RISCV_IOMMU ; + +return info; +} +static const struct iommu_ops riscv_iommu_ops = { .of_xlate = riscv_iommu_of_xlate, + .hw_info = riscv_iommu_hw_info, .capable = riscv_iommu_capable, .identity_domain = &riscv_iommu_identity_domain, .blocked_domain = &riscv_iommu_blocking_domain,
include/uapi/linux/iommufd.h +18 -0
@@ -660,6 +660,22 @@ struct iommu_hw_info_amd {__aligned_u64 efr2; }; +/** +* struct iommu_hw_info_riscv_iommu - RISCV IOMMU hardware information +* +* @capability: Value of RISC-V IOMMU capability register defined in +* RISC-V IOMMU spec section 5.3 IOMMU capabilities +* @fctl: Value of RISC-V IOMMU feature control register defined in +* RISC-V IOMMU spec section 5.4 Features-control register +* +* Don't advertise ATS support to the guest because driver doesn't support it. +*/ +struct iommu_hw_info_riscv_iommu { +__aligned_u64 capability; +__u32 fctl; +__u32 __reserved; +}; +/** * enum iommu_hw_info_type - IOMMU Hardware Info Types * @IOMMU_HW_INFO_TYPE_NONE: Output by the drivers that do not report hardware
@@ -670,6 +686,7 @@ struct iommu_hw_info_amd {* @IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV: NVIDIA Tegra241 CMDQV (extension for ARM * SMMUv3) info type * @IOMMU_HW_INFO_TYPE_AMD: AMD IOMMU info type +* @IOMMU_HW_INFO_TYPE_RISCV_IOMMU: RISC-V iommu info type */ enum iommu_hw_info_type {IOMMU_HW_INFO_TYPE_NONE =0 ,
@@ -678,6 +695,7 @@ enum iommu_hw_info_type {IOMMU_HW_INFO_TYPE_ARM_SMMUV3 =2 ,IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV =3 ,IOMMU_HW_INFO_TYPE_AMD =4 , + IOMMU_HW_INFO_TYPE_RISCV_IOMMU = 5, };/**
Signature · 1 line
2.50.1
Re: [RFC PATCH 11/11] iommu/riscv: support nested iommu for getting iommu hardware information
Quoted 5 lines · On Tue, Apr 28, 2026 at 09:13:59PM +0800, [email protected] wrote:
From: Zong Li <[email protected]>
This patch implements .hw_info operation and the related data structures for passing the IOMMU hardware capabilities for iommufd.
Quoted 7 lines
Signed-off-by Zong Li <[email protected]>Reviewed-by Jason Gunthorpe <[email protected]>Signed-off-by Fangyu Yu <[email protected]>--- drivers/iommu/riscv/iommu.c | 19 +++++++++++++++++++ include/uapi/linux/iommufd.h | 18 ++++++++++++++++++ 2 files changed, 37 insertions(+)
This has nothing to do with dirty tracking. It should go with a series introducing viommu.
Jason
Re: Re: [RFC PATCH 11/11] iommu/riscv: support nested iommu for getting iommu hardware information
Quoted 13 lines
Quoted 5 lines
From: Zong Li <[email protected]>
This patch implements .hw_info operation and the related data structures for passing the IOMMU hardware capabilities for iommufd.
Quoted 7 lines
Signed-off-by Zong Li <[email protected]>Reviewed-by Jason Gunthorpe <[email protected]>Signed-off-by Fangyu Yu <[email protected]>--- drivers/iommu/riscv/iommu.c | 19 +++++++++++++++++++ include/uapi/linux/iommufd.h | 18 ++++++++++++++++++ 2 files changed, 37 insertions(+)
Quoted 3 lines
This has nothing to do with dirty tracking. It should go with a series introducing viommu.
Thanks for pointing that out.
I added the .hw_info-related patch because, during passthrough testing with a VM, I observed that QEMU calls iommufd_get_hw_info, so I initially thought it was required. However, it appears that the .hw_info implementation is not necessary for this series. I will remove the .hw_info-related patch from my dirty-tracking series in a follow-up revision.
Thanks,
Fangyu
Quoted 1 line
Jason
Quoted 40 lines · On Tue, Apr 28, 2026 at 09:13:48PM +0800, [email protected] wrote:
From: Fangyu Yu <[email protected]>
The RISC-V IOMMU architecture defines an
AMO_HWADcapability (Hardware Access/Dirty update) that allows the IOMMU to atomically set the A/D bits in second-stage PTEs on DMA access. When DC.tc.GADE is asserted, the IOMMU autonomously sets D on the first write to a page mapped by an iohgatp domain. This series wires that capability up to the iommufd dirty-tracking interface (IOMMU_HWPT_SET_DIRTY_TRACKING/IOMMU_HWPT_GET_DIRTY_BITMAP) and reportsIOMMU_CAP_DIRTY_TRACKING.Design notes
------------* The feature is scoped to second-stage (iohgatp) domains only; these are the domains created for KVM / VFIO device pass-through when userspace allocates an HWPT with
IOMMU_HWPT_ALLOC_NEST_PARENTorIOMMU_HWPT_ALLOC_DIRTY_TRACKING. First-stage (iosatp) domains are not touched by this series.* The page-table side plugs into the existing
generic_ptdirty hook framework (amdv1 / vtdss style). RISC-V adds the three required PTE ops –is_write_dirty/make_write_clean/make_write_dirty.Testing
-------* Test on QEMU RISC-V, a virtio-net and an e1000e device was passed through to an L2 guest via vfio-pci + iommufd.
*
generic_ptKUnit: the existingtest_dirtycase now runs and passes for the RISC-V 64-bit format.Follow-up work
--------------
* Build a dedicated end-to-end test case that drives the full flow (HWPT_ALLOCwithDIRTY_TRACKING-> attach ->IOAS_MAP-> generate real DMA ->SET_DIRTY_TRACKING->GET_DIRTY_BITMAP-> verify bitmap against expected IOVA footprint) so that the behaviour can be regression-tested beyond the KUnit PTE-level coverage.
Quoted 2 lines
* If possible, rebase and retest on top of the updated "iommu irqbypass" patchset.
Thanks for this series! I was starting to go down a similar road myself in order to limit irqbypass to IOMMU_HWPT_ALLOC_NEST_PARENT domains since I wasn't happy with other approaches, e.g. continuing to use s-stage, but activating g-stage too with identity mappings since the MSI table can't be activated otherwise. Or, simply using g-stage instead of s-stage in order to get the MSI table enabled. In the end, I think the best is to require nested for irqbypass and this series will provide a good base for that.
I'll rebase irqbypass on this series and test it out.
Thanks,
drew
Quoted 51 lines
Quoted 40 lines
From: Fangyu Yu <[email protected]>
The RISC-V IOMMU architecture defines an
AMO_HWADcapability (Hardware Access/Dirty update) that allows the IOMMU to atomically set the A/D bits in second-stage PTEs on DMA access. When DC.tc.GADE is asserted, the IOMMU autonomously sets D on the first write to a page mapped by an iohgatp domain. This series wires that capability up to the iommufd dirty-tracking interface (IOMMU_HWPT_SET_DIRTY_TRACKING/IOMMU_HWPT_GET_DIRTY_BITMAP) and reportsIOMMU_CAP_DIRTY_TRACKING.Design notes
------------* The feature is scoped to second-stage (iohgatp) domains only; these are the domains created for KVM / VFIO device pass-through when userspace allocates an HWPT with
IOMMU_HWPT_ALLOC_NEST_PARENTorIOMMU_HWPT_ALLOC_DIRTY_TRACKING. First-stage (iosatp) domains are not touched by this series.* The page-table side plugs into the existing
generic_ptdirty hook framework (amdv1 / vtdss style). RISC-V adds the three required PTE ops –is_write_dirty/make_write_clean/make_write_dirty.Testing
-------* Test on QEMU RISC-V, a virtio-net and an e1000e device was passed through to an L2 guest via vfio-pci + iommufd.
*
generic_ptKUnit: the existingtest_dirtycase now runs and passes for the RISC-V 64-bit format.Follow-up work
--------------
* Build a dedicated end-to-end test case that drives the full flow (HWPT_ALLOCwithDIRTY_TRACKING-> attach ->IOAS_MAP-> generate real DMA ->SET_DIRTY_TRACKING->GET_DIRTY_BITMAP-> verify bitmap against expected IOVA footprint) so that the behaviour can be regression-tested beyond the KUnit PTE-level coverage.Quoted 2 lines
* If possible, rebase and retest on top of the updated "iommu irqbypass" patchset.
Thanks for this series! I was starting to go down a similar road myself in order to limit irqbypass to
IOMMU_HWPT_ALLOC_NEST_PARENTdomains since I wasn't happy with other approaches, e.g. continuing to use s-stage, but activating g-stage too with identity mappings since the MSI table can't be activated otherwise. Or, simply using g-stage instead of s-stage in order to get the MSI table enabled. In the end, I think the best is to require nested for irqbypass and this series will provide a good base for that.
Quoted 2 lines
I'll rebase irqbypass on this series and test it out.
Thanks for the feedback. Jason has provided some helpful suggestions on this series, and I am in the process of updating it. I expect to send out a new version in the coming days.
Fangyu
Quoted 2 lines
Thanks,
drew