Skip to contentSkip to conversation

iommu: Permit modular builds of io-pgtable drivers

RFC PATCH In review Isaac J. Manjarres 6 participants 21 messages Dec 22, 2020 – Jan 19, 2021

Subsystems (7)
lore
Share & tools
lore
#1 lore raw

PATCH v2 1/7 iommu/io-pgtable: Introduce dynamic io-pgtable fmt registration

In reply to Isaac J. Manjarres

The io-pgtable code constructs an array of init functions for each page table format at compile time. This is not ideal, as this increases the footprint of the io-pgtable code, as well as prevents io-pgtable formats from being built as kernel modules.

In preparation for modularizing the io-pgtable formats, switch to a dynamic registration scheme, where each io-pgtable format can register their init functions with the io-pgtable code at boot or module insertion time.

---
 drivers/iommu/io-pgtable-arm-v7s.c | 34 +++++++++++++-
 drivers/iommu/io-pgtable-arm.c     | 90 ++++++++++++++++++++++++++----------
 drivers/iommu/io-pgtable.c         | 94 ++++++++++++++++++++++++++++++++------
 include/linux/io-pgtable.h         | 51 +++++++++++++--------
 4 files changed, 209 insertions(+), 60 deletions(-)
drivers/iommu/io-pgtable-arm-v7s.c +32 -2
index 1d92ac9..89aad2f 100644
@@ -28,6 +28,7 @@
 #include <linux/iommu.h>
 #include <linux/kernel.h>
 #include <linux/kmemleak.h>
+#include <linux/module.h>
 #include <linux/sizes.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
@@ -835,7 +836,8 @@ static struct io_pgtable *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg,
 	return NULL;
 }
 
-struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
+static struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
+	.fmt	= ARM_V7S,
 	.alloc	= arm_v7s_alloc_pgtable,
 	.free	= arm_v7s_free_pgtable,
 };
@@ -982,5 +984,33 @@ static int __init arm_v7s_do_selftests(void)
 	pr_info("self test ok\n");
 	return 0;
 }
-subsys_initcall(arm_v7s_do_selftests);
+#else
+static int arm_v7s_do_selftests(void)
+{
+	return 0;
+}
 #endif
+
+static int __init arm_v7s_init(void)
+{
+	int ret;
+
+	ret = io_pgtable_ops_register(&io_pgtable_arm_v7s_init_fns);
+	if (ret < 0) {
+		pr_err("Failed to register ARM V7S format\n");
+		return ret;
+	}
+
+	ret = arm_v7s_do_selftests();
+	if (ret < 0)
+		io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
+
+	return ret;
+}
+core_initcall(arm_v7s_init);
+
+static void __exit arm_v7s_exit(void)
+{
+	io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
+}
+module_exit(arm_v7s_exit);
drivers/iommu/io-pgtable-arm.c +66 -24

106 lines not on this page · raw message

drivers/iommu/io-pgtable.c +79 -15

124 lines not on this page · raw message

include/linux/io-pgtable.h +32 -19

61 lines not on this page · raw message

Signature · 7 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#15 lore raw

PATCH v2 5/7 drm/msm: Add dependency on io-pgtable-arm format module

In reply to Isaac J. Manjarres

The MSM DRM driver depends on the availability of the ARM LPAE io-pgtable format code to work properly. In preparation for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable-arm format module is loaded before loading the MSM DRM driver module.

---
 drivers/gpu/drm/msm/msm_drv.c | 1 +
 1 file changed, 1 insertion(+)
drivers/gpu/drm/msm/msm_drv.c +1 -0
index 535a026..8be3506 100644
@@ -1369,3 +1369,4 @@ module_exit(msm_drm_unregister);
 MODULE_AUTHOR("Rob Clark <[email protected]");
 MODULE_DESCRIPTION("MSM DRM Driver");
 MODULE_LICENSE("GPL");
+MODULE_SOFTDEP("pre: io-pgtable-arm");
Signature · 7 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#0 lore raw

v2 cover iommu: Permit modular builds of io-pgtable drivers

The goal of the Generic Kernel Image (GKI) effort is to have a common kernel image that works across multiple Android devices. This involves generating a kernel image that has core features integrated into it, while SoC specific functionality can be added to the kernel for the device as a module.

Along with modularizing IOMMU drivers, this also means building the io-pgtable code as modules, which allows for SoC vendors to only include the io-pgtable implementations that they use. For example, GKI for arm64 must include support for both the IOMMU ARM LPAE/V7S formats at the moment. Having the code for both formats as modules allows SoC vendors to only provide the page table format that they use, along with their IOMMU driver.

Main changes since v1:

1) Retain io-pgtable.c as part of the core kernel

The patches are split into 4 parts:

1) Modularizing io-pgtable-arm[-v7s].c, while leaving the io-pgtable.c code as part of the core kernel, requires removing the references to the ARM LPAE and ARM V7S io-pgtable init functions, and using a dynamic method for formats to register their io-pgtable init functions.

The reason for defining an io_pgtable_init_fns_node structure is to not have the data structures used to store the init functions seep into the io-pgtable fmt drivers. Doing so allows for changing the internal data structure used to keep track of the init functions, without impacting the client data structures.

2) Taking references to the io-pgtable format drivers to ensure that they cannot be unloaded while in use.

3) Adding pre MODULE_SOFTDEP() dependencies to drivers in the kernel that are tristate, and invoke [alloc/free]_io_pgtable_ops(). This makes it so that the io-pgtable format drivers are loaded before the driver that needs them.

4) Changing the Kconfig options for the ARM LPAE nad ARM V7S to tristate.

Thanks in advance for the feedback,

Isaac J. Manjarres

Isaac J. Manjarres (7):
iommu/io-pgtable: Introduce dynamic io-pgtable fmt registration iommu/io-pgtable: Add refcounting for io-pgtable format modules iommu/arm-smmu: Add dependency on io-pgtable format modules iommu/arm-smmu-v3: Add dependency on io-pgtable-arm format module drm/msm: Add dependency on io-pgtable-arm format module drm/panfrost: Add dependency on io-pgtable-arm format module iommu/io-pgtable-arm: Allow building modular io-pgtable fmts

 drivers/gpu/drm/msm/msm_drv.c               |   1 +
 drivers/gpu/drm/panfrost/panfrost_drv.c     |   1 +
 drivers/iommu/Kconfig                       |   4 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |   1 +
 drivers/iommu/arm/arm-smmu/arm-smmu.c       |   1 +
 drivers/iommu/io-pgtable-arm-v7s.c          |  37 +++++++++-
 drivers/iommu/io-pgtable-arm.c              |  97 +++++++++++++++++++-------
 drivers/iommu/io-pgtable.c                  | 104 +++++++++++++++++++++++-----
 include/linux/io-pgtable.h                  |  53 +++++++++-----
 9 files changed, 236 insertions(+), 63 deletions(-)
Signature · 7 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#8 lore raw

PATCH v2 2/7 iommu/io-pgtable: Add refcounting for io-pgtable format modules

In preparation for modularizing io-pgtable formats, add support for reference counting the io-pgtable format modules to ensure that the modules are not unloaded while they are in use.

---
 drivers/iommu/io-pgtable-arm-v7s.c |  1 +
 drivers/iommu/io-pgtable-arm.c     |  5 +++++
 drivers/iommu/io-pgtable.c         | 12 ++++++++++--
 include/linux/io-pgtable.h         |  2 ++
 4 files changed, 18 insertions(+), 2 deletions(-)
drivers/iommu/io-pgtable-arm-v7s.c +1 -0
index 89aad2f..a5cb755a 100644
@@ -840,6 +840,7 @@ static struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
 	.fmt	= ARM_V7S,
 	.alloc	= arm_v7s_alloc_pgtable,
 	.free	= arm_v7s_free_pgtable,
+	.owner	= THIS_MODULE,
 };
 
 #ifdef CONFIG_IOMMU_IO_PGTABLE_ARMV7S_SELFTEST
drivers/iommu/io-pgtable-arm.c +5 -0
index ff0ea2f..e8b1e34 100644
@@ -1049,26 +1049,31 @@ static struct io_pgtable_init_fns io_pgtable_arm_lpae_init_fns[] = {
 		.fmt	= ARM_32_LPAE_S1,
 		.alloc	= arm_32_lpae_alloc_pgtable_s1,
 		.free	= arm_lpae_free_pgtable,
+		.owner	= THIS_MODULE,
 	},
 	{
 		.fmt	= ARM_32_LPAE_S2,
 		.alloc	= arm_32_lpae_alloc_pgtable_s2,
 		.free	= arm_lpae_free_pgtable,
+		.owner	= THIS_MODULE,
 	},
 	{
 		.fmt	= ARM_64_LPAE_S1,
 		.alloc	= arm_64_lpae_alloc_pgtable_s1,
 		.free	= arm_lpae_free_pgtable,
+		.owner	= THIS_MODULE,
 	},
 	{
 		.fmt	= ARM_64_LPAE_S2,
 		.alloc	= arm_64_lpae_alloc_pgtable_s2,
 		.free	= arm_lpae_free_pgtable,
+		.owner	= THIS_MODULE,
 	},
 	{
 		.fmt	= ARM_MALI_LPAE,
 		.alloc	= arm_mali_lpae_alloc_pgtable,
 		.free	= arm_lpae_free_pgtable,
+		.owner	= THIS_MODULE,
 	},
 };
 
drivers/iommu/io-pgtable.c +10 -2
index 2c6eb2e..cc83542 100644
@@ -10,6 +10,7 @@
 #include <linux/bug.h>
 #include <linux/io-pgtable.h>
 #include <linux/kernel.h>
+#include <linux/module.h>
 #include <linux/rwlock.h>
 #include <linux/slab.h>
 #include <linux/types.h>
@@ -52,9 +53,14 @@ struct io_pgtable_ops *alloc_io_pgtable_ops(enum io_pgtable_fmt fmt,
 	if (!fns)
 		return NULL;
 
+	if (!try_module_get(fns->owner))
+		return NULL;
+
 	iop = fns->alloc(cfg, cookie);
-	if (!iop)
+	if (!iop) {
+		module_put(fns->owner);
 		return NULL;
+	}
 
 	iop->fmt	= fmt;
 	iop->cookie	= cookie;
@@ -79,8 +85,10 @@ void free_io_pgtable_ops(struct io_pgtable_ops *ops)
 	iop = io_pgtable_ops_to_pgtable(ops);
 	io_pgtable_tlb_flush_all(iop);
 	fns = io_pgtable_get_init_fns(iop->fmt);
-	if (fns)
+	if (fns) {
 		fns->free(iop);
+		module_put(fns->owner);
+	}
 }
 EXPORT_SYMBOL_GPL(free_io_pgtable_ops);
 
include/linux/io-pgtable.h +2 -0
index 45b367ce..a03b262 100644
@@ -169,11 +169,13 @@ struct io_pgtable_ops {
  * @fmt:   The page table format.
  * @alloc: Allocate a set of page tables described by cfg.
  * @free:  Free the page tables associated with iop.
+ * @owner: Driver module providing these ops.
  */
 struct io_pgtable_init_fns {
 	enum io_pgtable_fmt fmt;
 	struct io_pgtable *(*alloc)(struct io_pgtable_cfg *cfg, void *cookie);
 	void (*free)(struct io_pgtable *iop);
+	struct module *owner;
 };
 
 /**
Signature · 7 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#14 lore raw

PATCH v2 4/7 iommu/arm-smmu-v3: Add dependency on io-pgtable-arm format module

In reply to Isaac J. Manjarres

The SMMUv3 driver depends on the availability of the ARM LPAE io-pgtable format code to work properly. In preparation for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable-arm format module is loaded before loading the ARM SMMUv3 driver module.

---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 1 +
 1 file changed, 1 insertion(+)
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +1 -0
index 8ca7415..c498ac8 100644
@@ -3650,3 +3650,4 @@ MODULE_DESCRIPTION("IOMMU API for ARM architected SMMUv3 implementations");
 MODULE_AUTHOR("Will Deacon <[email protected]>");
 MODULE_ALIAS("platform:arm-smmu-v3");
 MODULE_LICENSE("GPL v2");
+MODULE_SOFTDEP("pre: io-pgtable-arm");
Signature · 7 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#19 lore raw

PATCH v2 6/7 drm/panfrost: Add dependency on io-pgtable-arm format module

In reply to Isaac J. Manjarres

The Panfrost DRM driver depends on the availability of the ARM LPAE io-pgtable format code to work properly. In preparation for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable-arm format module is loaded before loading the Panfrost DRM driver module.

---
 drivers/gpu/drm/panfrost/panfrost_drv.c | 1 +
 1 file changed, 1 insertion(+)
drivers/gpu/drm/panfrost/panfrost_drv.c +1 -0
index 83a461b..7294622 100644
@@ -704,3 +704,4 @@ module_platform_driver(panfrost_driver);
 MODULE_AUTHOR("Panfrost Project Developers");
 MODULE_DESCRIPTION("Panfrost DRM Driver");
 MODULE_LICENSE("GPL v2");
+MODULE_SOFTDEP("pre: io-pgtable-arm");
Signature · 7 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#20 lore raw

PATCH v2 7/7 iommu/io-pgtable-arm: Allow building modular io-pgtable fmts

In reply to Isaac J. Manjarres

Now that everything is in place for modular io-pgtable formats, allow the ARM LPAE and ARMV7S io-pgtable formats to be built as modules.

---
 drivers/iommu/Kconfig              | 4 ++--
 drivers/iommu/io-pgtable-arm-v7s.c | 2 ++
 drivers/iommu/io-pgtable-arm.c     | 2 ++
 3 files changed, 6 insertions(+), 2 deletions(-)
drivers/iommu/Kconfig +2 -2
index 192ef8f..7e4f44f 100644
@@ -30,7 +30,7 @@ config IOMMU_IO_PGTABLE
 	bool
 
 config IOMMU_IO_PGTABLE_LPAE
-	bool "ARMv7/v8 Long Descriptor Format"
+	tristate "ARMv7/v8 Long Descriptor Format"
 	select IOMMU_IO_PGTABLE
 	depends on ARM || ARM64 || (COMPILE_TEST && !GENERIC_ATOMIC64)
 	help
@@ -49,7 +49,7 @@ config IOMMU_IO_PGTABLE_LPAE_SELFTEST
 	  If unsure, say N here.
 
 config IOMMU_IO_PGTABLE_ARMV7S
-	bool "ARMv7/v8 Short Descriptor Format"
+	tristate "ARMv7/v8 Short Descriptor Format"
 	select IOMMU_IO_PGTABLE
 	depends on ARM || ARM64 || COMPILE_TEST
 	help
drivers/iommu/io-pgtable-arm-v7s.c +2 -0
index a5cb755a..9d9f08f 100644
@@ -1015,3 +1015,5 @@ static void __exit arm_v7s_exit(void)
 	io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
 }
 module_exit(arm_v7s_exit);
+
+MODULE_LICENSE("GPL v2");
drivers/iommu/io-pgtable-arm.c +2 -0
index e8b1e34..e0de4ad 100644
@@ -1299,3 +1299,5 @@ static void __exit arm_lpae_exit(void)
 		io_pgtable_ops_unregister(&io_pgtable_arm_lpae_init_fns[i]);
 }
 module_exit(arm_lpae_exit);
+
+MODULE_LICENSE("GPL v2");
Signature · 7 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#9 lore raw

PATCH v2 3/7 iommu/arm-smmu: Add dependency on io-pgtable format modules

In reply to Isaac J. Manjarres

The SMMU driver depends on the availability of the ARM LPAE and ARM V7S io-pgtable format code to work properly. In preparation for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable format modules are loaded before loading the ARM SMMU driver module.

---
 drivers/iommu/arm/arm-smmu/arm-smmu.c | 1 +
 1 file changed, 1 insertion(+)
drivers/iommu/arm/arm-smmu/arm-smmu.c +1 -0
index d8c6bfd..a72649f 100644
@@ -2351,3 +2351,4 @@ MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU implementations");
 MODULE_AUTHOR("Will Deacon <[email protected]>");
 MODULE_ALIAS("platform:arm-smmu");
 MODULE_LICENSE("GPL v2");
+MODULE_SOFTDEP("pre: io-pgtable-arm io-pgtable-arm-v7s");
Signature · 7 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
reviewer
#10 lore raw

Re: [PATCH v2 3/7] iommu/arm-smmu: Add dependency on io-pgtable format modules

Quoted 2 lines · Isaac J. Manjarres

The SMMU driver depends on the availability of the ARM LPAE and ARM V7S io-pgtable format code to work properly. In preparation

Nit: we don't really depend on v7s - we *can* use it if it's available, address constraints are suitable, and the SMMU implementation actually supports it (many don't), but we can still quite happily not use it even so. LPAE is mandatory in the architecture so that's our only hard requirement, embodied in the kconfig select.

This does mean there may technically still be a corner case involving ARM_SMMU=y and IO_PGTABLE_ARM_V7S=m, but at worst it's now a runtime failure rather than a build error, so unless and until anyone demonstrates that it actually matters I don't feel particularly inclined to give it much thought.

Robin.

Quoted 15 lines

for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable format modules are loaded before loading the ARM SMMU driver module.

---
  drivers/iommu/arm/arm-smmu/arm-smmu.c | 1 +
  1 file changed, 1 insertion(+)
drivers/iommu/arm/arm-smmu/arm-smmu.c +0 -0
index d8c6bfd..a72649f 100644

drivers/iommu/arm/arm-smmu/arm-smmu.c MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU implementations"); · line 2354

Quoted 4 lines
drivers/iommu/arm/arm-smmu/arm-smmu.c +1 -0
@@ -2351,3 +2351,4 @@ MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU implementations");
  MODULE_AUTHOR("Will Deacon <[email protected]>");
  MODULE_ALIAS("platform:arm-smmu");
  MODULE_LICENSE("GPL v2");
+MODULE_SOFTDEP("pre: io-pgtable-arm io-pgtable-arm-v7s");

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

reviewer
#2 lore raw

Re: [PATCH v2 1/7] iommu/io-pgtable: Introduce dynamic io-pgtable fmt registration

In reply to Isaac J. Manjarres
Quoted 49 lines · Isaac J. Manjarres

The io-pgtable code constructs an array of init functions for each page table format at compile time. This is not ideal, as this increases the footprint of the io-pgtable code, as well as prevents io-pgtable formats from being built as kernel modules.

In preparation for modularizing the io-pgtable formats, switch to a dynamic registration scheme, where each io-pgtable format can register their init functions with the io-pgtable code at boot or module insertion time.

---
  drivers/iommu/io-pgtable-arm-v7s.c | 34 +++++++++++++-
  drivers/iommu/io-pgtable-arm.c     | 90 ++++++++++++++++++++++++++----------
  drivers/iommu/io-pgtable.c         | 94 ++++++++++++++++++++++++++++++++------
  include/linux/io-pgtable.h         | 51 +++++++++++++--------
  4 files changed, 209 insertions(+), 60 deletions(-)
drivers/iommu/io-pgtable-arm-v7s.c +7 -2
index 1d92ac9..89aad2f 100644
@@ -28,6 +28,7 @@
  #include <linux/iommu.h>
  #include <linux/kernel.h>
  #include <linux/kmemleak.h>
+#include <linux/module.h>
  #include <linux/sizes.h>
  #include <linux/slab.h>
  #include <linux/spinlock.h>
@@ -835,7 +836,8 @@ static struct io_pgtable *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg,
  	return NULL;
  }
 
-struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
+static struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
+	.fmt	= ARM_V7S,
  	.alloc	= arm_v7s_alloc_pgtable,
  	.free	= arm_v7s_free_pgtable,
  };
@@ -982,5 +984,33 @@ static int __init arm_v7s_do_selftests(void)
  	pr_info("self test ok\n");
  	return 0;
  }
-subsys_initcall(arm_v7s_do_selftests);
+#else
+static int arm_v7s_do_selftests(void)
+{
+	return 0;

drivers/iommu/io-pgtable-arm-v7s.c static int __init arm_v7s_do_selftests(void) · line 1000

Quoted 10 lines
drivers/iommu/io-pgtable-arm-v7s.c +9 -0
+}
  #endif
+
+static int __init arm_v7s_init(void)
+{
+	int ret;
+
+	ret = io_pgtable_ops_register(&io_pgtable_arm_v7s_init_fns);
+	if (ret < 0) {
+		pr_err("Failed to register ARM V7S format\n");

Super-nit: I think "v7s" should probably be lowercase there. Also general consistency WRT to showing the error code and whether or not to abbreviate "format" would be nice.

Quoted 106 lines
drivers/iommu/io-pgtable-arm-v7s.c +16 -0
+		return ret;
+	}
+
+	ret = arm_v7s_do_selftests();
+	if (ret < 0)
+		io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
+
+	return ret;
+}
+core_initcall(arm_v7s_init);
+
+static void __exit arm_v7s_exit(void)
+{
+	io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
+}
+module_exit(arm_v7s_exit);
drivers/iommu/io-pgtable-arm.c +43 -24
index 87def58..ff0ea2f 100644
@@ -13,6 +13,7 @@
  #include <linux/bitops.h>
  #include <linux/io-pgtable.h>
  #include <linux/kernel.h>
+#include <linux/module.h>
  #include <linux/sizes.h>
  #include <linux/slab.h>
  #include <linux/types.h>
@@ -1043,29 +1044,32 @@ arm_mali_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie)
  	return NULL;
  }
 
-struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns = {
-	.alloc	= arm_64_lpae_alloc_pgtable_s1,
-	.free	= arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns = {
-	.alloc	= arm_64_lpae_alloc_pgtable_s2,
-	.free	= arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s1_init_fns = {
-	.alloc	= arm_32_lpae_alloc_pgtable_s1,
-	.free	= arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s2_init_fns = {
-	.alloc	= arm_32_lpae_alloc_pgtable_s2,
-	.free	= arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns = {
-	.alloc	= arm_mali_lpae_alloc_pgtable,
-	.free	= arm_lpae_free_pgtable,
+static struct io_pgtable_init_fns io_pgtable_arm_lpae_init_fns[] = {
+	{
+		.fmt	= ARM_32_LPAE_S1,
+		.alloc	= arm_32_lpae_alloc_pgtable_s1,
+		.free	= arm_lpae_free_pgtable,
+	},
+	{
+		.fmt	= ARM_32_LPAE_S2,
+		.alloc	= arm_32_lpae_alloc_pgtable_s2,
+		.free	= arm_lpae_free_pgtable,
+	},
+	{
+		.fmt	= ARM_64_LPAE_S1,
+		.alloc	= arm_64_lpae_alloc_pgtable_s1,
+		.free	= arm_lpae_free_pgtable,
+	},
+	{
+		.fmt	= ARM_64_LPAE_S2,
+		.alloc	= arm_64_lpae_alloc_pgtable_s2,
+		.free	= arm_lpae_free_pgtable,
+	},
+	{
+		.fmt	= ARM_MALI_LPAE,
+		.alloc	= arm_mali_lpae_alloc_pgtable,
+		.free	= arm_lpae_free_pgtable,
+	},
  };
 
  #ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE_SELFTEST
@@ -1250,5 +1254,43 @@ static int __init arm_lpae_do_selftests(void)
  	pr_info("selftest: completed with %d PASS %d FAIL\n", pass, fail);
  	return fail ? -EFAULT : 0;
  }
-subsys_initcall(arm_lpae_do_selftests);
+#else
+static int __init arm_lpae_do_selftests(void)
+{
+	return 0;
+}
  #endif
+
+static int __init arm_lpae_init(void)
+{
+	int ret, i;
+
+	for (i = 0; i < ARRAY_SIZE(io_pgtable_arm_lpae_init_fns); i++) {
+		ret = io_pgtable_ops_register(&io_pgtable_arm_lpae_init_fns[i]);
+		if (ret < 0) {
+			pr_err("Failed to register ARM LPAE fmt: %d\n");
+			goto err_io_pgtable_register;
+		}

drivers/iommu/io-pgtable-arm.c static int __init arm_lpae_do_selftests(void) · line 1283

Quoted 10 lines
drivers/iommu/io-pgtable-arm.c +10 -0
+	}
+
+	ret = arm_lpae_do_selftests();
+	if (ret < 0)
+		goto err_io_pgtable_register;
+
+	return 0;
+
+err_io_pgtable_register:
+	for (i = i - 1; i >= 0; i--)

Personally I find "while (i--)" a bit clearer for this kind of unwinding, but maybe post-decrement isn't to everyone's taste.

Quoted 50 lines
drivers/iommu/io-pgtable-arm.c +13 -0
+		io_pgtable_ops_unregister(&io_pgtable_arm_lpae_init_fns[i]);
+	return ret;
+}
+core_initcall(arm_lpae_init);
+
+static void __exit arm_lpae_exit(void)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(io_pgtable_arm_lpae_init_fns); i++)
+		io_pgtable_ops_unregister(&io_pgtable_arm_lpae_init_fns[i]);
+}
+module_exit(arm_lpae_exit);
drivers/iommu/io-pgtable.c +13 -12
index 94394c8..2c6eb2e 100644
@@ -10,33 +10,45 @@
  #include <linux/bug.h>
  #include <linux/io-pgtable.h>
  #include <linux/kernel.h>
+#include <linux/rwlock.h>
+#include <linux/slab.h>
  #include <linux/types.h>
 
-static const struct io_pgtable_init_fns *
-io_pgtable_init_table[IO_PGTABLE_NUM_FMTS] = {
-#ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE
-	[ARM_32_LPAE_S1] = &io_pgtable_arm_32_lpae_s1_init_fns,
-	[ARM_32_LPAE_S2] = &io_pgtable_arm_32_lpae_s2_init_fns,
-	[ARM_64_LPAE_S1] = &io_pgtable_arm_64_lpae_s1_init_fns,
-	[ARM_64_LPAE_S2] = &io_pgtable_arm_64_lpae_s2_init_fns,
-	[ARM_MALI_LPAE] = &io_pgtable_arm_mali_lpae_init_fns,
-#endif
-#ifdef CONFIG_IOMMU_IO_PGTABLE_ARMV7S
-	[ARM_V7S] = &io_pgtable_arm_v7s_init_fns,
-#endif
+struct io_pgtable_init_fns_node {
+	struct io_pgtable_init_fns *fns;
+	struct list_head list;
  };
 
+static LIST_HEAD(io_pgtable_init_fns_list);
+static DEFINE_RWLOCK(io_pgtable_init_fns_list_lock);
+
+static struct io_pgtable_init_fns *io_pgtable_get_init_fns(enum io_pgtable_fmt fmt)
+{
+	struct io_pgtable_init_fns_node *iter;
+	struct io_pgtable_init_fns *fns = NULL;
+

drivers/iommu/io-pgtable.c line 39

Quoted 10 lines
drivers/iommu/io-pgtable.c +10 -0
+	read_lock(&io_pgtable_init_fns_list_lock);
+	list_for_each_entry(iter, &io_pgtable_init_fns_list, list)
+		if (iter->fns->fmt == fmt) {
+			fns = iter->fns;
+			break;
+		}
+	read_unlock(&io_pgtable_init_fns_list_lock);
+
+	return fns;
+}

I think it would be a lot easier to stick with a simple array indexed by enum - that way you can just set/clear/test entries without needing to worry about locking. Basically just remove the const and the initialisers from the existing one ;)

(and if you think you're concerned about memory, consider that just the list head plus lock is already half the size of the table)

Other than that, I think this all looks pretty promising - I'd suggest sending a non-RFC after rc1 so that it gets everyone's proper attention.

Thanks,
Robin.

Quoted 141 lines
drivers/iommu/io-pgtable.c +56 -3
+
  struct io_pgtable_ops *alloc_io_pgtable_ops(enum io_pgtable_fmt fmt,
  					    struct io_pgtable_cfg *cfg,
  					    void *cookie)
  {
  	struct io_pgtable *iop;
-	const struct io_pgtable_init_fns *fns;
+	struct io_pgtable_init_fns *fns;
 
  	if (fmt >= IO_PGTABLE_NUM_FMTS)
  		return NULL;
 
-	fns = io_pgtable_init_table[fmt];
+	fns = io_pgtable_get_init_fns(fmt);
  	if (!fns)
  		return NULL;
 
@@ -59,12 +71,64 @@ EXPORT_SYMBOL_GPL(alloc_io_pgtable_ops);
  void free_io_pgtable_ops(struct io_pgtable_ops *ops)
  {
  	struct io_pgtable *iop;
+	struct io_pgtable_init_fns *fns;
 
  	if (!ops)
  		return;
 
  	iop = io_pgtable_ops_to_pgtable(ops);
  	io_pgtable_tlb_flush_all(iop);
-	io_pgtable_init_table[iop->fmt]->free(iop);
+	fns = io_pgtable_get_init_fns(iop->fmt);
+	if (fns)
+		fns->free(iop);
  }
  EXPORT_SYMBOL_GPL(free_io_pgtable_ops);
+
+int io_pgtable_ops_register(struct io_pgtable_init_fns *init_fns)
+{
+	struct io_pgtable_init_fns_node *iter, *fns_node;
+	int ret = 0;
+
+	if (!init_fns || init_fns->fmt >= IO_PGTABLE_NUM_FMTS ||
+	    !init_fns->alloc || !init_fns->free)
+		return -EINVAL;
+
+	fns_node = kzalloc(sizeof(*fns_node), GFP_KERNEL);
+	if (!fns_node)
+		return -ENOMEM;
+
+	write_lock(&io_pgtable_init_fns_list_lock);
+	list_for_each_entry(iter, &io_pgtable_init_fns_list, list)
+		if (iter->fns->fmt == init_fns->fmt) {
+			ret = -EEXIST;
+			kfree(fns_node);
+			break;
+		}
+
+	if (!ret) {
+		fns_node->fns = init_fns;
+		INIT_LIST_HEAD(&fns_node->list);
+		list_add_tail(&fns_node->list, &io_pgtable_init_fns_list);
+	}
+	write_unlock(&io_pgtable_init_fns_list_lock);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(io_pgtable_ops_register);
+
+void io_pgtable_ops_unregister(struct io_pgtable_init_fns *init_fns)
+{
+	struct io_pgtable_init_fns_node *iter, *tmp;
+
+	if (!init_fns)
+		return;
+
+	write_lock(&io_pgtable_init_fns_list_lock);
+	list_for_each_entry_safe(iter, tmp, &io_pgtable_init_fns_list, list)
+		if (iter->fns == init_fns) {
+			list_del(&iter->list);
+			kfree(iter);
+			break;
+		}
+	write_unlock(&io_pgtable_init_fns_list_lock);
+}
+EXPORT_SYMBOL_GPL(io_pgtable_ops_unregister);
include/linux/io-pgtable.h +32 -10
index ea727eb..45b367ce 100644
@@ -163,6 +163,38 @@ struct io_pgtable_ops {
  };
 
  /**
+ * struct io_pgtable_init_fns - Alloc/free a set of page tables for a
+ *                              particular format.
+ *
+ * @fmt:   The page table format.
+ * @alloc: Allocate a set of page tables described by cfg.
+ * @free:  Free the page tables associated with iop.
+ */
+struct io_pgtable_init_fns {
+	enum io_pgtable_fmt fmt;
+	struct io_pgtable *(*alloc)(struct io_pgtable_cfg *cfg, void *cookie);
+	void (*free)(struct io_pgtable *iop);
+};
+
+/**
+ * io_pgtable_ops_register() - Register the page table routines for a page table
+ *                             format.
+ *
+ * @init_fns: The functions for allocating and freeing the page tables of
+ *            a particular format.
+ */
+int io_pgtable_ops_register(struct io_pgtable_init_fns *init_fns);
+
+/**
+ * io_pgtable_ops_unregister() - Unregister the page table routines for a page
+ *                               table format.
+ *
+ * @init_fns: The functions for allocating and freeing the page tables of
+ *            a particular format.
+ */
+void io_pgtable_ops_unregister(struct io_pgtable_init_fns *init_fns);
+
+/**
   * alloc_io_pgtable_ops() - Allocate a page table allocator for use by an IOMMU.
   *
   * @fmt:    The page table format.
@@ -233,23 +265,4 @@ io_pgtable_tlb_add_page(struct io_pgtable *iop,
  		iop->cfg.tlb->tlb_add_page(gather, iova, granule, iop->cookie);
  }
 
-/**
- * struct io_pgtable_init_fns - Alloc/free a set of page tables for a
- *                              particular format.
- *
- * @alloc: Allocate a set of page tables described by cfg.
- * @free:  Free the page tables associated with iop.
- */
-struct io_pgtable_init_fns {
-	struct io_pgtable *(*alloc)(struct io_pgtable_cfg *cfg, void *cookie);
-	void (*free)(struct io_pgtable *iop);

include/linux/io-pgtable.h io_pgtable_tlb_add_page(struct io_pgtable *iop, · line 268

Quoted 10 lines
include/linux/io-pgtable.h +0 -9
-};
-
-extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s1_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s2_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns;
-
  #endif /* __IO_PGTABLE_H */

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#11 lore raw

Re: [PATCH v2 3/7] iommu/arm-smmu: Add dependency on io-pgtable format modules

In reply to Robin Murphy
Quoted 11 lines · Robin Murphy
Quoted 2 lines · Isaac J. Manjarres

The SMMU driver depends on the availability of the ARM LPAE and ARM V7S io-pgtable format code to work properly. In preparation

Nit: we don't really depend on v7s - we *can* use it if it's available, address constraints are suitable, and the SMMU implementation actually supports it (many don't), but we can still quite happily not use it even so. LPAE is mandatory in the architecture so that's our only hard requirement, embodied in the kconfig select.

Quoted 8 lines

This does mean there may technically still be a corner case involving ARM_SMMU=y and IO_PGTABLE_ARM_V7S=m, but at worst it's now a runtime failure rather than a build error, so unless and until anyone demonstrates that it actually matters I don't feel particularly inclined to give it much thought.

Robin.

Okay, I'll fix up the commit message, as well as the code, so that it only depends on io-pgtable-arm.

Thanks,
Isaac

Quoted 9 lines
Quoted 9 lines

for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable format modules are loaded before loading the ARM SMMU driver module.

---
  drivers/iommu/arm/arm-smmu/arm-smmu.c | 1 +
  1 file changed, 1 insertion(+)
Quoted 16 lines

drivers/iommu/arm/arm-smmu/arm-smmu.c MODULE_DESCRIPTION("IOMMU API for ARM · line 2354

Quoted 11 lines
drivers/iommu/arm/arm-smmu/arm-smmu.c +1 -0
index d8c6bfd..a72649f 100644
@@ -2351,3 +2351,4 @@ MODULE_DESCRIPTION("IOMMU API for ARM 
 architected SMMU implementations");
  MODULE_AUTHOR("Will Deacon <[email protected]>");
  MODULE_ALIAS("platform:arm-smmu");
  MODULE_LICENSE("GPL v2");
+MODULE_SOFTDEP("pre: io-pgtable-arm io-pgtable-arm-v7s");

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#3 lore raw

Re: [PATCH v2 1/7] iommu/io-pgtable: Introduce dynamic io-pgtable fmt registration

In reply to Robin Murphy
Quoted 58 lines · Robin Murphy
Quoted 57 lines · Isaac J. Manjarres

The io-pgtable code constructs an array of init functions for each page table format at compile time. This is not ideal, as this increases the footprint of the io-pgtable code, as well as prevents io-pgtable formats from being built as kernel modules.

In preparation for modularizing the io-pgtable formats, switch to a dynamic registration scheme, where each io-pgtable format can register their init functions with the io-pgtable code at boot or module insertion time.

---
  drivers/iommu/io-pgtable-arm-v7s.c | 34 +++++++++++++-
  drivers/iommu/io-pgtable-arm.c     | 90 
++++++++++++++++++++++++++----------
  drivers/iommu/io-pgtable.c         | 94 
++++++++++++++++++++++++++++++++------
  include/linux/io-pgtable.h         | 51 +++++++++++++--------
  4 files changed, 209 insertions(+), 60 deletions(-)
drivers/iommu/io-pgtable-arm-v7s.c +12 -1
index 1d92ac9..89aad2f 100644
@@ -28,6 +28,7 @@
  #include <linux/iommu.h>
  #include <linux/kernel.h>
  #include <linux/kmemleak.h>
+#include <linux/module.h>
  #include <linux/sizes.h>
  #include <linux/slab.h>
  #include <linux/spinlock.h>
@@ -835,7 +836,8 @@ static struct io_pgtable 
 *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg,
  	return NULL;
  }
  -struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
+static struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
+	.fmt	= ARM_V7S,
  	.alloc	= arm_v7s_alloc_pgtable,
  	.free	= arm_v7s_free_pgtable,
  };
@@ -982,5 +984,33 @@ static int __init arm_v7s_do_selftests(void)
  	pr_info("self test ok\n");
  	return 0;
  }
-subsys_initcall(arm_v7s_do_selftests);
+#else
+static int arm_v7s_do_selftests(void)
+{
+	return 0;
+}
  #endif
+
+static int __init arm_v7s_init(void)
+{
+	int ret;
Quoted 9 lines

drivers/iommu/io-pgtable-arm-v7s.c static int __init arm_v7s_do_selftests(void) · line 1000

Quoted 4 lines
drivers/iommu/io-pgtable-arm-v7s.c +4 -0
+
+	ret = io_pgtable_ops_register(&io_pgtable_arm_v7s_init_fns);
+	if (ret < 0) {
+		pr_err("Failed to register ARM V7S format\n");

Super-nit: I think "v7s" should probably be lowercase there. Also general consistency WRT to showing the error code and whether or not to abbreviate "format" would be nice.

Ok, I can fix this accordingly.

Quoted 110 lines
Quoted 110 lines
drivers/iommu/io-pgtable-arm-v7s.c +16 -0
+		return ret;
+	}
+
+	ret = arm_v7s_do_selftests();
+	if (ret < 0)
+		io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
+
+	return ret;
+}
+core_initcall(arm_v7s_init);
+
+static void __exit arm_v7s_exit(void)
+{
+	io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
+}
+module_exit(arm_v7s_exit);
drivers/iommu/io-pgtable-arm.c +48 -23
index 87def58..ff0ea2f 100644
@@ -13,6 +13,7 @@
  #include <linux/bitops.h>
  #include <linux/io-pgtable.h>
  #include <linux/kernel.h>
+#include <linux/module.h>
  #include <linux/sizes.h>
  #include <linux/slab.h>
  #include <linux/types.h>
@@ -1043,29 +1044,32 @@ arm_mali_lpae_alloc_pgtable(struct 
 io_pgtable_cfg *cfg, void *cookie)
  	return NULL;
  }
  -struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns = {
-	.alloc	= arm_64_lpae_alloc_pgtable_s1,
-	.free	= arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns = {
-	.alloc	= arm_64_lpae_alloc_pgtable_s2,
-	.free	= arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s1_init_fns = {
-	.alloc	= arm_32_lpae_alloc_pgtable_s1,
-	.free	= arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s2_init_fns = {
-	.alloc	= arm_32_lpae_alloc_pgtable_s2,
-	.free	= arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns = {
-	.alloc	= arm_mali_lpae_alloc_pgtable,
-	.free	= arm_lpae_free_pgtable,
+static struct io_pgtable_init_fns io_pgtable_arm_lpae_init_fns[] = {
+	{
+		.fmt	= ARM_32_LPAE_S1,
+		.alloc	= arm_32_lpae_alloc_pgtable_s1,
+		.free	= arm_lpae_free_pgtable,
+	},
+	{
+		.fmt	= ARM_32_LPAE_S2,
+		.alloc	= arm_32_lpae_alloc_pgtable_s2,
+		.free	= arm_lpae_free_pgtable,
+	},
+	{
+		.fmt	= ARM_64_LPAE_S1,
+		.alloc	= arm_64_lpae_alloc_pgtable_s1,
+		.free	= arm_lpae_free_pgtable,
+	},
+	{
+		.fmt	= ARM_64_LPAE_S2,
+		.alloc	= arm_64_lpae_alloc_pgtable_s2,
+		.free	= arm_lpae_free_pgtable,
+	},
+	{
+		.fmt	= ARM_MALI_LPAE,
+		.alloc	= arm_mali_lpae_alloc_pgtable,
+		.free	= arm_lpae_free_pgtable,
+	},
  };
    #ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE_SELFTEST
@@ -1250,5 +1254,43 @@ static int __init arm_lpae_do_selftests(void)
  	pr_info("selftest: completed with %d PASS %d FAIL\n", pass, fail);
  	return fail ? -EFAULT : 0;
  }
-subsys_initcall(arm_lpae_do_selftests);
+#else
+static int __init arm_lpae_do_selftests(void)
+{
+	return 0;
+}
  #endif
+
+static int __init arm_lpae_init(void)
+{
+	int ret, i;
+
+	for (i = 0; i < ARRAY_SIZE(io_pgtable_arm_lpae_init_fns); i++) {
+		ret = io_pgtable_ops_register(&io_pgtable_arm_lpae_init_fns[i]);
+		if (ret < 0) {
+			pr_err("Failed to register ARM LPAE fmt: %d\n");
+			goto err_io_pgtable_register;
+		}
+	}
+
+	ret = arm_lpae_do_selftests();
+	if (ret < 0)
+		goto err_io_pgtable_register;
Quoted 9 lines

drivers/iommu/io-pgtable-arm.c static int __init arm_lpae_do_selftests(void) · line 1283

Quoted 5 lines
drivers/iommu/io-pgtable-arm.c +5 -0
+
+	return 0;
+
+err_io_pgtable_register:
+	for (i = i - 1; i >= 0; i--)

Personally I find "while (i--)" a bit clearer for this kind of unwinding, but maybe post-decrement isn't to everyone's taste.

I'm not particularly attached to the current approach, so a while loop is okay with me :).

Quoted 68 lines
Quoted 49 lines
drivers/iommu/io-pgtable-arm.c +13 -0
+		io_pgtable_ops_unregister(&io_pgtable_arm_lpae_init_fns[i]);
+	return ret;
+}
+core_initcall(arm_lpae_init);
+
+static void __exit arm_lpae_exit(void)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(io_pgtable_arm_lpae_init_fns); i++)
+		io_pgtable_ops_unregister(&io_pgtable_arm_lpae_init_fns[i]);
+}
+module_exit(arm_lpae_exit);
drivers/iommu/io-pgtable.c +12 -11
index 94394c8..2c6eb2e 100644
@@ -10,33 +10,45 @@
  #include <linux/bug.h>
  #include <linux/io-pgtable.h>
  #include <linux/kernel.h>
+#include <linux/rwlock.h>
+#include <linux/slab.h>
  #include <linux/types.h>
  -static const struct io_pgtable_init_fns *
-io_pgtable_init_table[IO_PGTABLE_NUM_FMTS] = {
-#ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE
-	[ARM_32_LPAE_S1] = &io_pgtable_arm_32_lpae_s1_init_fns,
-	[ARM_32_LPAE_S2] = &io_pgtable_arm_32_lpae_s2_init_fns,
-	[ARM_64_LPAE_S1] = &io_pgtable_arm_64_lpae_s1_init_fns,
-	[ARM_64_LPAE_S2] = &io_pgtable_arm_64_lpae_s2_init_fns,
-	[ARM_MALI_LPAE] = &io_pgtable_arm_mali_lpae_init_fns,
-#endif
-#ifdef CONFIG_IOMMU_IO_PGTABLE_ARMV7S
-	[ARM_V7S] = &io_pgtable_arm_v7s_init_fns,
-#endif
+struct io_pgtable_init_fns_node {
+	struct io_pgtable_init_fns *fns;
+	struct list_head list;
  };
  +static LIST_HEAD(io_pgtable_init_fns_list);
+static DEFINE_RWLOCK(io_pgtable_init_fns_list_lock);
+
+static struct io_pgtable_init_fns *io_pgtable_get_init_fns(enum 
 io_pgtable_fmt fmt)
+{
+	struct io_pgtable_init_fns_node *iter;
+	struct io_pgtable_init_fns *fns = NULL;
+

drivers/iommu/io-pgtable.c line 39

Quoted 10 lines
drivers/iommu/io-pgtable.c +10 -0
+	read_lock(&io_pgtable_init_fns_list_lock);
+	list_for_each_entry(iter, &io_pgtable_init_fns_list, list)
+		if (iter->fns->fmt == fmt) {
+			fns = iter->fns;
+			break;
+		}
+	read_unlock(&io_pgtable_init_fns_list_lock);
+
+	return fns;
+}

I think it would be a lot easier to stick with a simple array indexed by enum - that way you can just set/clear/test entries without needing to worry about locking. Basically just remove the const and the initialisers from the existing one ;)

(and if you think you're concerned about memory, consider that just the list head plus lock is already half the size of the table)

Quoted 7 lines

Other than that, I think this all looks pretty promising - I'd suggest sending a non-RFC after rc1 so that it gets everyone's proper attention.

Thanks,
Robin.

Thanks for all of the feedback! With respect to the comment about keeping the current array, don't we need some sort of lock to protect access to the data structure (e.g. concurrent access if adding two ops for the same format)? Or do you mean that we should use atomic operations to handle this cleanly?

Thanks,
Isaac

Quoted 148 lines
Quoted 148 lines
drivers/iommu/io-pgtable.c +56 -2
+
  struct io_pgtable_ops *alloc_io_pgtable_ops(enum io_pgtable_fmt fmt,
  					    struct io_pgtable_cfg *cfg,
  					    void *cookie)
  {
  	struct io_pgtable *iop;
-	const struct io_pgtable_init_fns *fns;
+	struct io_pgtable_init_fns *fns;
    	if (fmt >= IO_PGTABLE_NUM_FMTS)
  		return NULL;
  -	fns = io_pgtable_init_table[fmt];
+	fns = io_pgtable_get_init_fns(fmt);
  	if (!fns)
  		return NULL;
  @@ -59,12 +71,64 @@ EXPORT_SYMBOL_GPL(alloc_io_pgtable_ops);
  void free_io_pgtable_ops(struct io_pgtable_ops *ops)
  {
  	struct io_pgtable *iop;
+	struct io_pgtable_init_fns *fns;
    	if (!ops)
  		return;
    	iop = io_pgtable_ops_to_pgtable(ops);
  	io_pgtable_tlb_flush_all(iop);
-	io_pgtable_init_table[iop->fmt]->free(iop);
+	fns = io_pgtable_get_init_fns(iop->fmt);
+	if (fns)
+		fns->free(iop);
  }
  EXPORT_SYMBOL_GPL(free_io_pgtable_ops);
+
+int io_pgtable_ops_register(struct io_pgtable_init_fns *init_fns)
+{
+	struct io_pgtable_init_fns_node *iter, *fns_node;
+	int ret = 0;
+
+	if (!init_fns || init_fns->fmt >= IO_PGTABLE_NUM_FMTS ||
+	    !init_fns->alloc || !init_fns->free)
+		return -EINVAL;
+
+	fns_node = kzalloc(sizeof(*fns_node), GFP_KERNEL);
+	if (!fns_node)
+		return -ENOMEM;
+
+	write_lock(&io_pgtable_init_fns_list_lock);
+	list_for_each_entry(iter, &io_pgtable_init_fns_list, list)
+		if (iter->fns->fmt == init_fns->fmt) {
+			ret = -EEXIST;
+			kfree(fns_node);
+			break;
+		}
+
+	if (!ret) {
+		fns_node->fns = init_fns;
+		INIT_LIST_HEAD(&fns_node->list);
+		list_add_tail(&fns_node->list, &io_pgtable_init_fns_list);
+	}
+	write_unlock(&io_pgtable_init_fns_list_lock);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(io_pgtable_ops_register);
+
+void io_pgtable_ops_unregister(struct io_pgtable_init_fns *init_fns)
+{
+	struct io_pgtable_init_fns_node *iter, *tmp;
+
+	if (!init_fns)
+		return;
+
+	write_lock(&io_pgtable_init_fns_list_lock);
+	list_for_each_entry_safe(iter, tmp, &io_pgtable_init_fns_list, list)
+		if (iter->fns == init_fns) {
+			list_del(&iter->list);
+			kfree(iter);
+			break;
+		}
+	write_unlock(&io_pgtable_init_fns_list_lock);
+}
+EXPORT_SYMBOL_GPL(io_pgtable_ops_unregister);
include/linux/io-pgtable.h +15 -0
index ea727eb..45b367ce 100644
@@ -163,6 +163,38 @@ struct io_pgtable_ops {
  };
    /**
+ * struct io_pgtable_init_fns - Alloc/free a set of page tables for a
+ *                              particular format.
+ *
+ * @fmt:   The page table format.
+ * @alloc: Allocate a set of page tables described by cfg.
+ * @free:  Free the page tables associated with iop.
+ */
+struct io_pgtable_init_fns {
+	enum io_pgtable_fmt fmt;
+	struct io_pgtable *(*alloc)(struct io_pgtable_cfg *cfg, void 
 *cookie);
+	void (*free)(struct io_pgtable *iop);
+};
+
+/**
+ * io_pgtable_ops_register() - Register the page table routines for a 

page table
+ * format.
+ *
+ * @init_fns: The functions for allocating and freeing the page tables of
+ * a particular format.
+ */

include/linux/io-pgtable.h +4 -0
+int io_pgtable_ops_register(struct io_pgtable_init_fns *init_fns);
+
+/**
+ * io_pgtable_ops_unregister() - Unregister the page table routines 

for a page
+ * table format.
+ *
+ * @init_fns: The functions for allocating and freeing the page tables of
+ * a particular format.
+ */

include/linux/io-pgtable.h +3 -0
+void io_pgtable_ops_unregister(struct io_pgtable_init_fns *init_fns);
+
+/**
   * alloc_io_pgtable_ops() - Allocate a page table allocator for use 

by an IOMMU.
*
* @fmt: The page table format.

include/linux/io-pgtable.h +0 -17
@@ -233,23 +265,4 @@ io_pgtable_tlb_add_page(struct io_pgtable *iop,
  		iop->cfg.tlb->tlb_add_page(gather, iova, granule, iop->cookie);
  }
  -/**
- * struct io_pgtable_init_fns - Alloc/free a set of page tables for a
- *                              particular format.
- *
- * @alloc: Allocate a set of page tables described by cfg.
- * @free:  Free the page tables associated with iop.
- */
-struct io_pgtable_init_fns {
-	struct io_pgtable *(*alloc)(struct io_pgtable_cfg *cfg, void 
 *cookie);
-	void (*free)(struct io_pgtable *iop);
-};
-
-extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s1_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s2_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns;
Quoted 8 lines

include/linux/io-pgtable.h io_pgtable_tlb_add_page(struct io_pgtable *iop,

Quoted 3 lines
include/linux/io-pgtable.h +0 -1
-
  #endif /* __IO_PGTABLE_H */

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

reviewer
#12 lore raw

Re: [PATCH v2 3/7] iommu/arm-smmu: Add dependency on io-pgtable format modules

In reply to [email protected]
Quoted 18 lines · On 2020-12-22 19:49, [email protected] wrote:
Quoted 17 lines · Robin Murphy
Quoted 2 lines · Isaac J. Manjarres

The SMMU driver depends on the availability of the ARM LPAE and ARM V7S io-pgtable format code to work properly. In preparation

Nit: we don't really depend on v7s - we *can* use it if it's available, address constraints are suitable, and the SMMU implementation actually supports it (many don't), but we can still quite happily not use it even so. LPAE is mandatory in the architecture so that's our only hard requirement, embodied in the kconfig select.

This does mean there may technically still be a corner case involving ARM_SMMU=y and IO_PGTABLE_ARM_V7S=m, but at worst it's now a runtime failure rather than a build error, so unless and until anyone demonstrates that it actually matters I don't feel particularly inclined to give it much thought.

Quoted 4 lines
Quoted 2 lines

Robin.

Okay, I'll fix up the commit message, as well as the code, so that it only depends on io-pgtable-arm.

Well, IIUC it would make sense to keep the softdep for when the v7s module *is* present; I just wanted to clarify that it's more of a nice-to-have rather than a necessity.

Robin.

Quoted 27 lines

Thanks,
Isaac

Quoted 25 lines
Quoted 17 lines

for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable format modules are loaded before loading the ARM SMMU driver module.

---
  drivers/iommu/arm/arm-smmu/arm-smmu.c | 1 +

  1 file changed, 1 insertion(+)
drivers/iommu/arm/arm-smmu/arm-smmu.c +0 -0
index d8c6bfd..a72649f 100644
@@ -2351,3 +2351,4 @@ MODULE_DESCRIPTION("IOMMU API for ARM 
 architected SMMU implementations");
   MODULE_AUTHOR("Will Deacon <[email protected]>");
   MODULE_ALIAS("platform:arm-smmu");
Quoted 3 lines

  MODULE_LICENSE("GPL v2");
+MODULE_SOFTDEP("pre: io-pgtable-arm io-pgtable-arm-v7s");

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

reviewer
#4 lore raw

Re: [PATCH v2 1/7] iommu/io-pgtable: Introduce dynamic io-pgtable fmt registration

In reply to [email protected]
Quoted 265 lines · On 2020-12-22 19:54, [email protected] wrote:
Quoted 58 lines · Robin Murphy
Quoted 57 lines · Isaac J. Manjarres

The io-pgtable code constructs an array of init functions for each page table format at compile time. This is not ideal, as this increases the footprint of the io-pgtable code, as well as prevents io-pgtable formats from being built as kernel modules.

In preparation for modularizing the io-pgtable formats, switch to a dynamic registration scheme, where each io-pgtable format can register their init functions with the io-pgtable code at boot or module insertion time.

---
  drivers/iommu/io-pgtable-arm-v7s.c | 34 +++++++++++++- drivers/iommu/io-pgtable-arm.c     | 90
++++++++++++++++++++++++++----------
  drivers/iommu/io-pgtable.c         | 94
++++++++++++++++++++++++++++++++------
  include/linux/io-pgtable.h         | 51 +++++++++++++--------

  4 files changed, 209 insertions(+), 60 deletions(-)
drivers/iommu/io-pgtable-arm-v7s.c +3 -0
index 1d92ac9..89aad2f 100644
@@ -28,6 +28,7 @@
   #include <linux/iommu.h>
   #include <linux/kernel.h>
   #include <linux/kmemleak.h>
+#include <linux/module.h>
   #include <linux/sizes.h>
   #include <linux/slab.h>
   #include <linux/spinlock.h>
@@ -835,7 +836,8 @@ static struct io_pgtable 
 *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg,
       return NULL;
   }
   -struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
+static struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
+    .fmt    = ARM_V7S,
       .alloc    = arm_v7s_alloc_pgtable,
       .free    = arm_v7s_free_pgtable,

  };

drivers/iommu/io-pgtable-arm-v7s.c +9 -1
@@ -982,5 +984,33 @@ static int __init arm_v7s_do_selftests(void)
       pr_info("self test ok\n");
       return 0;
   }
-subsys_initcall(arm_v7s_do_selftests);
+#else
+static int arm_v7s_do_selftests(void)
+{
+    return 0;
+}
   #endif
+
+static int __init arm_v7s_init(void)
+{
+    int ret;
Quoted 9 lines

drivers/iommu/io-pgtable-arm-v7s.c static int __init arm_v7s_do_selftests(void) · line 1000

Quoted 4 lines
drivers/iommu/io-pgtable-arm-v7s.c +4 -0
+
+    ret = io_pgtable_ops_register(&io_pgtable_arm_v7s_init_fns);
+    if (ret < 0) {
+        pr_err("Failed to register ARM V7S format\n");

Super-nit: I think "v7s" should probably be lowercase there. Also general consistency WRT to showing the error code and whether or not to abbreviate "format" would be nice.

Ok, I can fix this accordingly.

Quoted 111 lines
Quoted 111 lines
drivers/iommu/io-pgtable-arm-v7s.c +16 -0
+        return ret;
+    }
+
+    ret = arm_v7s_do_selftests();
+    if (ret < 0)
+        io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
+
+    return ret;
+}
+core_initcall(arm_v7s_init);
+
+static void __exit arm_v7s_exit(void)
+{
+    io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
+}
+module_exit(arm_v7s_exit);
drivers/iommu/io-pgtable-arm.c +39 -23
index 87def58..ff0ea2f 100644
@@ -13,6 +13,7 @@
   #include <linux/bitops.h>
   #include <linux/io-pgtable.h>
   #include <linux/kernel.h>
+#include <linux/module.h>
   #include <linux/sizes.h>
   #include <linux/slab.h>
   #include <linux/types.h>
@@ -1043,29 +1044,32 @@ arm_mali_lpae_alloc_pgtable(struct 
 io_pgtable_cfg *cfg, void *cookie)
       return NULL;
   }
   -struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns = {
-    .alloc    = arm_64_lpae_alloc_pgtable_s1,
-    .free    = arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns = {
-    .alloc    = arm_64_lpae_alloc_pgtable_s2,
-    .free    = arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s1_init_fns = {
-    .alloc    = arm_32_lpae_alloc_pgtable_s1,
-    .free    = arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s2_init_fns = {
-    .alloc    = arm_32_lpae_alloc_pgtable_s2,
-    .free    = arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns = {
-    .alloc    = arm_mali_lpae_alloc_pgtable,
-    .free    = arm_lpae_free_pgtable,
+static struct io_pgtable_init_fns io_pgtable_arm_lpae_init_fns[] = {
+    {
+        .fmt    = ARM_32_LPAE_S1,
+        .alloc    = arm_32_lpae_alloc_pgtable_s1,
+        .free    = arm_lpae_free_pgtable,
+    },
+    {
+        .fmt    = ARM_32_LPAE_S2,
+        .alloc    = arm_32_lpae_alloc_pgtable_s2,
+        .free    = arm_lpae_free_pgtable,
+    },
+    {
+        .fmt    = ARM_64_LPAE_S1,
+        .alloc    = arm_64_lpae_alloc_pgtable_s1,
+        .free    = arm_lpae_free_pgtable,
+    },
+    {
+        .fmt    = ARM_64_LPAE_S2,
+        .alloc    = arm_64_lpae_alloc_pgtable_s2,
+        .free    = arm_lpae_free_pgtable,
+    },
+    {
+        .fmt    = ARM_MALI_LPAE,
+        .alloc    = arm_mali_lpae_alloc_pgtable,
+        .free    = arm_lpae_free_pgtable,
+    },
   };
     #ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE_SELFTEST
@@ -1250,5 +1254,43 @@ static int __init arm_lpae_do_selftests(void)
       pr_info("selftest: completed with %d PASS %d FAIL\n", pass, fail);
       return fail ? -EFAULT : 0;
   }
-subsys_initcall(arm_lpae_do_selftests);
+#else
+static int __init arm_lpae_do_selftests(void)
+{
+    return 0;
+}
   #endif
+
+static int __init arm_lpae_init(void)
+{
+    int ret, i;
+
+    for (i = 0; i < ARRAY_SIZE(io_pgtable_arm_lpae_init_fns); i++) {
+        ret = 

io_pgtable_ops_register(&io_pgtable_arm_lpae_init_fns[i]); +        if (ret < 0) {
+            pr_err("Failed to register ARM LPAE fmt: %d\n"); +            goto err_io_pgtable_register;
+        }
+    }
+
+    ret = arm_lpae_do_selftests();
+    if (ret < 0)
+        goto err_io_pgtable_register;

Quoted 9 lines

drivers/iommu/io-pgtable-arm.c static int __init arm_lpae_do_selftests(void) · line 1283

Quoted 5 lines
drivers/iommu/io-pgtable-arm.c +5 -0
+
+    return 0;
+
+err_io_pgtable_register:
+    for (i = i - 1; i >= 0; i--)

Personally I find "while (i--)" a bit clearer for this kind of unwinding, but maybe post-decrement isn't to everyone's taste.

I'm not particularly attached to the current approach, so a while loop is okay with me :).

Quoted 72 lines
Quoted 49 lines

+        io_pgtable_ops_unregister(&io_pgtable_arm_lpae_init_fns[i]); +    return ret;

drivers/iommu/io-pgtable-arm.c +11 -0
+}
+core_initcall(arm_lpae_init);
+
+static void __exit arm_lpae_exit(void)
+{
+    int i;
+
+    for (i = 0; i < ARRAY_SIZE(io_pgtable_arm_lpae_init_fns); i++)
+        io_pgtable_ops_unregister(&io_pgtable_arm_lpae_init_fns[i]);
+}
+module_exit(arm_lpae_exit);
drivers/iommu/io-pgtable.c +2 -0
index 94394c8..2c6eb2e 100644
@@ -10,33 +10,45 @@
   #include <linux/bug.h>
   #include <linux/io-pgtable.h>
   #include <linux/kernel.h>
+#include <linux/rwlock.h>
+#include <linux/slab.h>
   #include <linux/types.h>

  -static const struct io_pgtable_init_fns *

drivers/iommu/io-pgtable.c +3 -11
-io_pgtable_init_table[IO_PGTABLE_NUM_FMTS] = {
-#ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE
-    [ARM_32_LPAE_S1] = &io_pgtable_arm_32_lpae_s1_init_fns,
-    [ARM_32_LPAE_S2] = &io_pgtable_arm_32_lpae_s2_init_fns,
-    [ARM_64_LPAE_S1] = &io_pgtable_arm_64_lpae_s1_init_fns,
-    [ARM_64_LPAE_S2] = &io_pgtable_arm_64_lpae_s2_init_fns,
-    [ARM_MALI_LPAE] = &io_pgtable_arm_mali_lpae_init_fns,
-#endif
-#ifdef CONFIG_IOMMU_IO_PGTABLE_ARMV7S
-    [ARM_V7S] = &io_pgtable_arm_v7s_init_fns,
-#endif
+struct io_pgtable_init_fns_node {
+    struct io_pgtable_init_fns *fns;
+    struct list_head list;

  };
  +static LIST_HEAD(io_pgtable_init_fns_list);

drivers/iommu/io-pgtable.c +3 -0
+static DEFINE_RWLOCK(io_pgtable_init_fns_list_lock);
+
+static struct io_pgtable_init_fns *io_pgtable_get_init_fns(enum 

io_pgtable_fmt fmt)

drivers/iommu/io-pgtable.c +4 -0
+{
+    struct io_pgtable_init_fns_node *iter;
+    struct io_pgtable_init_fns *fns = NULL;
+

drivers/iommu/io-pgtable.c line 39

Quoted 10 lines
drivers/iommu/io-pgtable.c +10 -0
+    read_lock(&io_pgtable_init_fns_list_lock);
+    list_for_each_entry(iter, &io_pgtable_init_fns_list, list)
+        if (iter->fns->fmt == fmt) {
+            fns = iter->fns;
+            break;
+        }
+    read_unlock(&io_pgtable_init_fns_list_lock);
+
+    return fns;
+}

I think it would be a lot easier to stick with a simple array indexed by enum - that way you can just set/clear/test entries without needing to worry about locking. Basically just remove the const and the initialisers from the existing one ;)

(and if you think you're concerned about memory, consider that just the list head plus lock is already half the size of the table)

Other than that, I think this all looks pretty promising - I'd suggest sending a non-RFC after rc1 so that it gets everyone's proper attention.

Quoted 8 lines
Quoted 3 lines

Thanks,
Robin.

Thanks for all of the feedback! With respect to the comment about keeping the current array, don't we need some sort of lock to protect access to the data structure (e.g. concurrent access if adding two ops for the same format)? Or do you mean that we should use atomic operations to handle this cleanly?

I don't think any races are realistically possible - built-in initcalls should run sequentially, while module init should be serialised by the module loader lock, so registrations should never be able to race against each other. And if there could be a race between registering a format and somebody trying to use it, then something's fundamentally wrong anyway - sure we could use READ_ONCE/WRITE_ONCE for accessing the table, but that would be basically be admitting that we've failed at the primary purpose of making sure formats are loaded *before* the users that depend on them.

With a directly-indexed table there's no contention between looking up one format and registering another unrelated format, unlike with a list.

Robin.

Quoted 159 lines

Thanks,
Isaac

Quoted 148 lines
Quoted 148 lines

+
  struct io_pgtable_ops *alloc_io_pgtable_ops(enum io_pgtable_fmt fmt, struct io_pgtable_cfg *cfg,
                          void *cookie)
  {
      struct io_pgtable *iop;
-    const struct io_pgtable_init_fns *fns;
+    struct io_pgtable_init_fns *fns;
        if (fmt >= IO_PGTABLE_NUM_FMTS)
          return NULL;
  -    fns = io_pgtable_init_table[fmt];
+    fns = io_pgtable_get_init_fns(fmt);
      if (!fns)
          return NULL;
  @@ -59,12 +71,64 @@ EXPORT_SYMBOL_GPL(alloc_io_pgtable_ops); void free_io_pgtable_ops(struct io_pgtable_ops *ops)
  {
      struct io_pgtable *iop;
+    struct io_pgtable_init_fns *fns;
        if (!ops)
          return;
        iop = io_pgtable_ops_to_pgtable(ops); io_pgtable_tlb_flush_all(iop);
-    io_pgtable_init_table[iop->fmt]->free(iop);
+    fns = io_pgtable_get_init_fns(iop->fmt);
+    if (fns)
+        fns->free(iop);
  }
  EXPORT_SYMBOL_GPL(free_io_pgtable_ops);

drivers/iommu/io-pgtable.c +41 -0
+
+int io_pgtable_ops_register(struct io_pgtable_init_fns *init_fns)
+{
+    struct io_pgtable_init_fns_node *iter, *fns_node;
+    int ret = 0;
+
+    if (!init_fns || init_fns->fmt >= IO_PGTABLE_NUM_FMTS ||
+        !init_fns->alloc || !init_fns->free)
+        return -EINVAL;
+
+    fns_node = kzalloc(sizeof(*fns_node), GFP_KERNEL);
+    if (!fns_node)
+        return -ENOMEM;
+
+    write_lock(&io_pgtable_init_fns_list_lock);
+    list_for_each_entry(iter, &io_pgtable_init_fns_list, list)
+        if (iter->fns->fmt == init_fns->fmt) {
+            ret = -EEXIST;
+            kfree(fns_node);
+            break;
+        }
+
+    if (!ret) {
+        fns_node->fns = init_fns;
+        INIT_LIST_HEAD(&fns_node->list);
+        list_add_tail(&fns_node->list, &io_pgtable_init_fns_list);
+    }
+    write_unlock(&io_pgtable_init_fns_list_lock);
+    return ret;
+}
+EXPORT_SYMBOL_GPL(io_pgtable_ops_register);
+
+void io_pgtable_ops_unregister(struct io_pgtable_init_fns *init_fns)
+{
+    struct io_pgtable_init_fns_node *iter, *tmp;
+
+    if (!init_fns)
+        return;
+
+    write_lock(&io_pgtable_init_fns_list_lock);
+    list_for_each_entry_safe(iter, tmp, &io_pgtable_init_fns_list, 

list)
+        if (iter->fns == init_fns) {
+            list_del(&iter->list);
+            kfree(iter);
+            break;
+        }
+    write_unlock(&io_pgtable_init_fns_list_lock);

drivers/iommu/io-pgtable.c +2 -0
+}
+EXPORT_SYMBOL_GPL(io_pgtable_ops_unregister);
include/linux/io-pgtable.h +15 -0
index ea727eb..45b367ce 100644
@@ -163,6 +163,38 @@ struct io_pgtable_ops {
   };
     /**
+ * struct io_pgtable_init_fns - Alloc/free a set of page tables for a
+ *                              particular format.
+ *
+ * @fmt:   The page table format.
+ * @alloc: Allocate a set of page tables described by cfg.
+ * @free:  Free the page tables associated with iop.
+ */
+struct io_pgtable_init_fns {
+    enum io_pgtable_fmt fmt;
+    struct io_pgtable *(*alloc)(struct io_pgtable_cfg *cfg, void 
 *cookie);
+    void (*free)(struct io_pgtable *iop);
+};
+
+/**
+ * io_pgtable_ops_register() - Register the page table routines for 

a page table
+ *                             format.
+ *
+ * @init_fns: The functions for allocating and freeing the page tables of
+ *            a particular format.
+ */

include/linux/io-pgtable.h +4 -0
+int io_pgtable_ops_register(struct io_pgtable_init_fns *init_fns);
+
+/**
+ * io_pgtable_ops_unregister() - Unregister the page table routines 

for a page
+ *                               table format.
+ *
+ * @init_fns: The functions for allocating and freeing the page tables of
+ *            a particular format.
+ */

include/linux/io-pgtable.h +3 -0
+void io_pgtable_ops_unregister(struct io_pgtable_init_fns *init_fns);
+
+/**

   * alloc_io_pgtable_ops() - Allocate a page table allocator for use by an IOMMU. *
   * @fmt:    The page table format.

include/linux/io-pgtable.h +0 -0
@@ -233,23 +265,4 @@ io_pgtable_tlb_add_page(struct io_pgtable *iop,
           iop->cfg.tlb->tlb_add_page(gather, iova, granule, iop->cookie);
   }

  -/**
- * struct io_pgtable_init_fns - Alloc/free a set of page tables for a
- *                              particular format.
- *
- * @alloc: Allocate a set of page tables described by cfg.
- * @free:  Free the page tables associated with iop.
- */
-struct io_pgtable_init_fns {
-    struct io_pgtable *(*alloc)(struct io_pgtable_cfg *cfg, void *cookie);
-    void (*free)(struct io_pgtable *iop);

include/linux/io-pgtable.h +0 -8
-};
-
-extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s1_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s2_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns;
-extern struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns;
Quoted 8 lines
Quoted 3 lines

-
  #endif /* __IO_PGTABLE_H */

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Quoted 4 lines

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#13 lore raw

Re: [PATCH v2 3/7] iommu/arm-smmu: Add dependency on io-pgtable format modules

In reply to Robin Murphy
Quoted 21 lines · Robin Murphy
Quoted 20 lines · On 2020-12-22 19:49, [email protected] wrote:
Quoted 19 lines · Robin Murphy
Quoted 2 lines · Isaac J. Manjarres

The SMMU driver depends on the availability of the ARM LPAE and ARM V7S io-pgtable format code to work properly. In preparation

Nit: we don't really depend on v7s - we *can* use it if it's available, address constraints are suitable, and the SMMU implementation actually supports it (many don't), but we can still quite happily not use it even so. LPAE is mandatory in the architecture so that's our only hard requirement, embodied in the kconfig select.

This does mean there may technically still be a corner case involving ARM_SMMU=y and IO_PGTABLE_ARM_V7S=m, but at worst it's now a runtime failure rather than a build error, so unless and until anyone demonstrates that it actually matters I don't feel particularly inclined to give it much thought.

Robin.

Quoted 9 lines
Quoted 2 lines

Okay, I'll fix up the commit message, as well as the code, so that it only depends on io-pgtable-arm.

Well, IIUC it would make sense to keep the softdep for when the v7s module *is* present; I just wanted to clarify that it's more of a nice-to-have rather than a necessity.

Robin.

Understood, I will keep it there and reword the commit msg. I just tried it out in an environment where the io-pgtable-arm-v7s module isn't present, and I didn't see any
warnings or error messages, and the SMMU driver module was loaded properly,
so yes, it's good to have it.

Thanks,
Isaac

Quoted 11 lines
Quoted 11 lines

Thanks,
Isaac

Quoted 9 lines
Quoted 9 lines

for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable format modules are loaded before loading the ARM SMMU driver module.

---
  drivers/iommu/arm/arm-smmu/arm-smmu.c | 1 +

  1 file changed, 1 insertion(+)
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/iommu/arm/arm-smmu/arm-smmu.c +0 -0
index d8c6bfd..a72649f 100644
@@ -2351,3 +2351,4 @@ MODULE_DESCRIPTION("IOMMU API for ARM 
 architected SMMU implementations");
   MODULE_AUTHOR("Will Deacon <[email protected]>");
   MODULE_ALIAS("platform:arm-smmu");
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

  MODULE_LICENSE("GPL v2");
+MODULE_SOFTDEP("pre: io-pgtable-arm io-pgtable-arm-v7s");

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#5 lore raw

Re: [PATCH v2 1/7] iommu/io-pgtable: Introduce dynamic io-pgtable fmt registration

In reply to Robin Murphy
Quoted 289 lines · Robin Murphy
Quoted 269 lines · On 2020-12-22 19:54, [email protected] wrote:
Quoted 58 lines · Robin Murphy
Quoted 57 lines · Isaac J. Manjarres

The io-pgtable code constructs an array of init functions for each page table format at compile time. This is not ideal, as this increases the footprint of the io-pgtable code, as well as prevents io-pgtable formats from being built as kernel modules.

In preparation for modularizing the io-pgtable formats, switch to a dynamic registration scheme, where each io-pgtable format can register their init functions with the io-pgtable code at boot or module insertion time.

---
  drivers/iommu/io-pgtable-arm-v7s.c | 34 +++++++++++++- drivers/iommu/io-pgtable-arm.c     | 90
++++++++++++++++++++++++++----------
  drivers/iommu/io-pgtable.c         | 94
++++++++++++++++++++++++++++++++------
  include/linux/io-pgtable.h         | 51 +++++++++++++--------

  4 files changed, 209 insertions(+), 60 deletions(-)
drivers/iommu/io-pgtable-arm-v7s.c +3 -0
index 1d92ac9..89aad2f 100644
@@ -28,6 +28,7 @@
   #include <linux/iommu.h>
   #include <linux/kernel.h>
   #include <linux/kmemleak.h>
+#include <linux/module.h>
   #include <linux/sizes.h>
   #include <linux/slab.h>
   #include <linux/spinlock.h>
@@ -835,7 +836,8 @@ static struct io_pgtable 
 *arm_v7s_alloc_pgtable(struct io_pgtable_cfg *cfg,
       return NULL;
   }
   -struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
+static struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns = {
+    .fmt    = ARM_V7S,
       .alloc    = arm_v7s_alloc_pgtable,
       .free    = arm_v7s_free_pgtable,

  };

drivers/iommu/io-pgtable-arm-v7s.c +9 -1
@@ -982,5 +984,33 @@ static int __init arm_v7s_do_selftests(void)
       pr_info("self test ok\n");
       return 0;
   }
-subsys_initcall(arm_v7s_do_selftests);
+#else
+static int arm_v7s_do_selftests(void)
+{
+    return 0;
+}
   #endif
+
+static int __init arm_v7s_init(void)
+{
+    int ret;
Quoted 9 lines

drivers/iommu/io-pgtable-arm-v7s.c static int __init arm_v7s_do_selftests(void) · line 1000

Quoted 4 lines
drivers/iommu/io-pgtable-arm-v7s.c +4 -0
+
+    ret = io_pgtable_ops_register(&io_pgtable_arm_v7s_init_fns);
+    if (ret < 0) {
+        pr_err("Failed to register ARM V7S format\n");

Super-nit: I think "v7s" should probably be lowercase there. Also general consistency WRT to showing the error code and whether or not to abbreviate "format" would be nice.

Ok, I can fix this accordingly.

Quoted 113 lines
Quoted 113 lines
drivers/iommu/io-pgtable-arm-v7s.c +16 -0
+        return ret;
+    }
+
+    ret = arm_v7s_do_selftests();
+    if (ret < 0)
+        io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
+
+    return ret;
+}
+core_initcall(arm_v7s_init);
+
+static void __exit arm_v7s_exit(void)
+{
+    io_pgtable_ops_unregister(&io_pgtable_arm_v7s_init_fns);
+}
+module_exit(arm_v7s_exit);
drivers/iommu/io-pgtable-arm.c +27 -22
index 87def58..ff0ea2f 100644
@@ -13,6 +13,7 @@
   #include <linux/bitops.h>
   #include <linux/io-pgtable.h>
   #include <linux/kernel.h>
+#include <linux/module.h>
   #include <linux/sizes.h>
   #include <linux/slab.h>
   #include <linux/types.h>
@@ -1043,29 +1044,32 @@ arm_mali_lpae_alloc_pgtable(struct 
 io_pgtable_cfg *cfg, void *cookie)
       return NULL;
   }
   -struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns = {
-    .alloc    = arm_64_lpae_alloc_pgtable_s1,
-    .free    = arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns = {
-    .alloc    = arm_64_lpae_alloc_pgtable_s2,
-    .free    = arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s1_init_fns = {
-    .alloc    = arm_32_lpae_alloc_pgtable_s1,
-    .free    = arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s2_init_fns = {
-    .alloc    = arm_32_lpae_alloc_pgtable_s2,
-    .free    = arm_lpae_free_pgtable,
-};
-
-struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns = {
-    .alloc    = arm_mali_lpae_alloc_pgtable,
-    .free    = arm_lpae_free_pgtable,
+static struct io_pgtable_init_fns io_pgtable_arm_lpae_init_fns[] = 
 {
+    {
+        .fmt    = ARM_32_LPAE_S1,
+        .alloc    = arm_32_lpae_alloc_pgtable_s1,
+        .free    = arm_lpae_free_pgtable,
+    },
+    {
+        .fmt    = ARM_32_LPAE_S2,
+        .alloc    = arm_32_lpae_alloc_pgtable_s2,
+        .free    = arm_lpae_free_pgtable,
+    },
+    {
+        .fmt    = ARM_64_LPAE_S1,
+        .alloc    = arm_64_lpae_alloc_pgtable_s1,
+        .free    = arm_lpae_free_pgtable,
+    },
+    {
+        .fmt    = ARM_64_LPAE_S2,
+        .alloc    = arm_64_lpae_alloc_pgtable_s2,
+        .free    = arm_lpae_free_pgtable,
+    },
+    {
+        .fmt    = ARM_MALI_LPAE,
+        .alloc    = arm_mali_lpae_alloc_pgtable,
+        .free    = arm_lpae_free_pgtable,
+    },
   };

    #ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE_SELFTEST

drivers/iommu/io-pgtable-arm.c +11 -1
@@ -1250,5 +1254,43 @@ static int __init arm_lpae_do_selftests(void)
       pr_info("selftest: completed with %d PASS %d FAIL\n", pass, fail);
       return fail ? -EFAULT : 0;
   }
-subsys_initcall(arm_lpae_do_selftests);
+#else
+static int __init arm_lpae_do_selftests(void)
+{
+    return 0;
+}
   #endif
+
+static int __init arm_lpae_init(void)
+{
+    int ret, i;
+
+    for (i = 0; i < ARRAY_SIZE(io_pgtable_arm_lpae_init_fns); i++) 

{
+        ret = io_pgtable_ops_register(&io_pgtable_arm_lpae_init_fns[i]); +        if (ret < 0) {
+            pr_err("Failed to register ARM LPAE fmt: %d\n"); +            goto err_io_pgtable_register;
+        }
+    }
+
+    ret = arm_lpae_do_selftests();
+    if (ret < 0)
+        goto err_io_pgtable_register;

Quoted 9 lines

drivers/iommu/io-pgtable-arm.c static int __init arm_lpae_do_selftests(void) · line 1283

Quoted 5 lines
drivers/iommu/io-pgtable-arm.c +5 -0
+
+    return 0;
+
+err_io_pgtable_register:
+    for (i = i - 1; i >= 0; i--)

Personally I find "while (i--)" a bit clearer for this kind of unwinding, but maybe post-decrement isn't to everyone's taste.

I'm not particularly attached to the current approach, so a while loop is okay with me :).

Quoted 74 lines
Quoted 51 lines

+ io_pgtable_ops_unregister(&io_pgtable_arm_lpae_init_fns[i]); +    return ret;

drivers/iommu/io-pgtable-arm.c +9 -0
+}
+core_initcall(arm_lpae_init);
+
+static void __exit arm_lpae_exit(void)
+{
+    int i;
+
+    for (i = 0; i < ARRAY_SIZE(io_pgtable_arm_lpae_init_fns); i++)
+        

io_pgtable_ops_unregister(&io_pgtable_arm_lpae_init_fns[i]);

drivers/iommu/io-pgtable-arm.c +2 -0
+}
+module_exit(arm_lpae_exit);
drivers/iommu/io-pgtable.c +2 -0
index 94394c8..2c6eb2e 100644
@@ -10,33 +10,45 @@
   #include <linux/bug.h>
   #include <linux/io-pgtable.h>
   #include <linux/kernel.h>
+#include <linux/rwlock.h>
+#include <linux/slab.h>
   #include <linux/types.h>

  -static const struct io_pgtable_init_fns *

drivers/iommu/io-pgtable.c +3 -11
-io_pgtable_init_table[IO_PGTABLE_NUM_FMTS] = {
-#ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE
-    [ARM_32_LPAE_S1] = &io_pgtable_arm_32_lpae_s1_init_fns,
-    [ARM_32_LPAE_S2] = &io_pgtable_arm_32_lpae_s2_init_fns,
-    [ARM_64_LPAE_S1] = &io_pgtable_arm_64_lpae_s1_init_fns,
-    [ARM_64_LPAE_S2] = &io_pgtable_arm_64_lpae_s2_init_fns,
-    [ARM_MALI_LPAE] = &io_pgtable_arm_mali_lpae_init_fns,
-#endif
-#ifdef CONFIG_IOMMU_IO_PGTABLE_ARMV7S
-    [ARM_V7S] = &io_pgtable_arm_v7s_init_fns,
-#endif
+struct io_pgtable_init_fns_node {
+    struct io_pgtable_init_fns *fns;
+    struct list_head list;

  };
  +static LIST_HEAD(io_pgtable_init_fns_list);

drivers/iommu/io-pgtable.c +3 -0
+static DEFINE_RWLOCK(io_pgtable_init_fns_list_lock);
+
+static struct io_pgtable_init_fns *io_pgtable_get_init_fns(enum 

io_pgtable_fmt fmt)

drivers/iommu/io-pgtable.c +4 -0
+{
+    struct io_pgtable_init_fns_node *iter;
+    struct io_pgtable_init_fns *fns = NULL;
+

drivers/iommu/io-pgtable.c line 39

Quoted 10 lines
drivers/iommu/io-pgtable.c +10 -0
+    read_lock(&io_pgtable_init_fns_list_lock);
+    list_for_each_entry(iter, &io_pgtable_init_fns_list, list)
+        if (iter->fns->fmt == fmt) {
+            fns = iter->fns;
+            break;
+        }
+    read_unlock(&io_pgtable_init_fns_list_lock);
+
+    return fns;
+}

I think it would be a lot easier to stick with a simple array indexed by enum - that way you can just set/clear/test entries without needing to worry about locking. Basically just remove the const and the initialisers from the existing one ;)

(and if you think you're concerned about memory, consider that just the list head plus lock is already half the size of the table)

Other than that, I think this all looks pretty promising - I'd suggest sending a non-RFC after rc1 so that it gets everyone's proper attention.

Quoted 8 lines
Quoted 3 lines

Thanks,
Robin.

Thanks for all of the feedback! With respect to the comment about keeping the current array, don't we need some sort of lock to protect access to the data structure (e.g. concurrent access if adding two ops for the same format)? Or do you mean that we should use atomic operations to handle this cleanly?

I don't think any races are realistically possible - built-in initcalls should run sequentially, while module init should be serialised by the module loader lock, so registrations should never be able to race against each other. And if there could be a race between registering a format and somebody trying to use it, then something's fundamentally wrong anyway - sure we could use READ_ONCE/WRITE_ONCE for accessing the table, but that would be basically be admitting that we've failed at the primary purpose of making sure formats are loaded *before* the users that depend on them.

Quoted 6 lines

With a directly-indexed table there's no contention between looking up one format and registering another unrelated format, unlike with a list.

Robin.

Got it. I'll stick with the array as suggested. Thanks again for the feedback. I'll be sending the version with your suggestions when 5.11-rc1 comes out as a non-RFC.

--Isaac

Quoted 168 lines
Quoted 168 lines

Thanks,
Isaac

Quoted 165 lines
Quoted 156 lines

+
  struct io_pgtable_ops *alloc_io_pgtable_ops(enum io_pgtable_fmt fmt, struct io_pgtable_cfg *cfg,
                          void *cookie)
  {
      struct io_pgtable *iop;
-    const struct io_pgtable_init_fns *fns;
+    struct io_pgtable_init_fns *fns;
        if (fmt >= IO_PGTABLE_NUM_FMTS)
          return NULL;
  -    fns = io_pgtable_init_table[fmt];
+    fns = io_pgtable_get_init_fns(fmt);
      if (!fns)
          return NULL;
  @@ -59,12 +71,64 @@ EXPORT_SYMBOL_GPL(alloc_io_pgtable_ops); void free_io_pgtable_ops(struct io_pgtable_ops *ops)
  {
      struct io_pgtable *iop;
+    struct io_pgtable_init_fns *fns;
        if (!ops)
          return;
        iop = io_pgtable_ops_to_pgtable(ops); io_pgtable_tlb_flush_all(iop);
-    io_pgtable_init_table[iop->fmt]->free(iop);
+    fns = io_pgtable_get_init_fns(iop->fmt);
+    if (fns)
+        fns->free(iop);
  }
  EXPORT_SYMBOL_GPL(free_io_pgtable_ops);

drivers/iommu/io-pgtable.c +33 -0
+
+int io_pgtable_ops_register(struct io_pgtable_init_fns *init_fns)
+{
+    struct io_pgtable_init_fns_node *iter, *fns_node;
+    int ret = 0;
+
+    if (!init_fns || init_fns->fmt >= IO_PGTABLE_NUM_FMTS ||
+        !init_fns->alloc || !init_fns->free)
+        return -EINVAL;
+
+    fns_node = kzalloc(sizeof(*fns_node), GFP_KERNEL);
+    if (!fns_node)
+        return -ENOMEM;
+
+    write_lock(&io_pgtable_init_fns_list_lock);
+    list_for_each_entry(iter, &io_pgtable_init_fns_list, list)
+        if (iter->fns->fmt == init_fns->fmt) {
+            ret = -EEXIST;
+            kfree(fns_node);
+            break;
+        }
+
+    if (!ret) {
+        fns_node->fns = init_fns;
+        INIT_LIST_HEAD(&fns_node->list);
+        list_add_tail(&fns_node->list, &io_pgtable_init_fns_list);
+    }
+    write_unlock(&io_pgtable_init_fns_list_lock);
+    return ret;
+}
+EXPORT_SYMBOL_GPL(io_pgtable_ops_register);
+
+void io_pgtable_ops_unregister(struct io_pgtable_init_fns 

*init_fns)

drivers/iommu/io-pgtable.c +8 -0
+{
+    struct io_pgtable_init_fns_node *iter, *tmp;
+
+    if (!init_fns)
+        return;
+
+    write_lock(&io_pgtable_init_fns_list_lock);
+    list_for_each_entry_safe(iter, tmp, &io_pgtable_init_fns_list, 

list)
+        if (iter->fns == init_fns) {
+            list_del(&iter->list);
+            kfree(iter);
+            break;
+        }
+    write_unlock(&io_pgtable_init_fns_list_lock);

drivers/iommu/io-pgtable.c +2 -0
+}
+EXPORT_SYMBOL_GPL(io_pgtable_ops_unregister);
include/linux/io-pgtable.h +1 -0
index ea727eb..45b367ce 100644
@@ -163,6 +163,38 @@ struct io_pgtable_ops {
   };
     /**
+ * struct io_pgtable_init_fns - Alloc/free a set of page tables for 

a
+ *                              particular format.
+ *
+ * @fmt:   The page table format.
+ * @alloc: Allocate a set of page tables described by cfg.
+ * @free:  Free the page tables associated with iop.
+ */
+struct io_pgtable_init_fns {
+    enum io_pgtable_fmt fmt;
+    struct io_pgtable *(*alloc)(struct io_pgtable_cfg *cfg, void *cookie);
+    void (*free)(struct io_pgtable *iop);

include/linux/io-pgtable.h +4 -0
+};
+
+/**
+ * io_pgtable_ops_register() - Register the page table routines for 

a page table
+ *                             format.
+ *
+ * @init_fns: The functions for allocating and freeing the page tables of
+ *            a particular format.
+ */

include/linux/io-pgtable.h +4 -0
+int io_pgtable_ops_register(struct io_pgtable_init_fns *init_fns);
+
+/**
+ * io_pgtable_ops_unregister() - Unregister the page table routines 

for a page
+ *                               table format.
+ *
+ * @init_fns: The functions for allocating and freeing the page tables of
+ *            a particular format.
+ */
+void io_pgtable_ops_unregister(struct io_pgtable_init_fns *init_fns);

include/linux/io-pgtable.h +2 -0
+
+/**

   * alloc_io_pgtable_ops() - Allocate a page table allocator for use by an IOMMU. *
   * @fmt:    The page table format.

include/linux/io-pgtable.h +0 -0
@@ -233,23 +265,4 @@ io_pgtable_tlb_add_page(struct io_pgtable *iop,
           iop->cfg.tlb->tlb_add_page(gather, iova, granule, iop->cookie);
   }

  -/**
- * struct io_pgtable_init_fns - Alloc/free a set of page tables for a
- *                              particular format.
- *
- * @alloc: Allocate a set of page tables described by cfg.
- * @free:  Free the page tables associated with iop.
- */
-struct io_pgtable_init_fns {
-    struct io_pgtable *(*alloc)(struct io_pgtable_cfg *cfg, void *cookie);
-    void (*free)(struct io_pgtable *iop);

include/linux/io-pgtable.h +0 -3
-};
-
-extern struct io_pgtable_init_fns 

io_pgtable_arm_32_lpae_s1_init_fns;
-extern struct io_pgtable_init_fns
io_pgtable_arm_32_lpae_s2_init_fns;
-extern struct io_pgtable_init_fns
io_pgtable_arm_64_lpae_s1_init_fns;
-extern struct io_pgtable_init_fns
io_pgtable_arm_64_lpae_s2_init_fns;

include/linux/io-pgtable.h +0 -2
-extern struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns;
-extern struct io_pgtable_init_fns 
Quoted 4 lines

io_pgtable_arm_mali_lpae_init_fns;
-
  #endif /* __IO_PGTABLE_H */

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Quoted 4 lines
Quoted 4 lines

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#6 lore raw

Re: [PATCH v2 1/7] iommu/io-pgtable: Introduce dynamic io-pgtable fmt registration

In reply to Isaac J. Manjarres

Hi Isaac,

Quoted 23 lines · Isaac J. Manjarres

The io-pgtable code constructs an array of init functions for each page table format at compile time. This is not ideal, as this increases the footprint of the io-pgtable code, as well as prevents io-pgtable formats from being built as kernel modules.

In preparation for modularizing the io-pgtable formats, switch to a dynamic registration scheme, where each io-pgtable format can register their init functions with the io-pgtable code at boot or module insertion time.

---
  drivers/iommu/io-pgtable-arm-v7s.c | 34 +++++++++++++-
  drivers/iommu/io-pgtable-arm.c     | 90 ++++++++++++++++++++++++++----------
  drivers/iommu/io-pgtable.c         | 94 ++++++++++++++++++++++++++++++++------
  include/linux/io-pgtable.h         | 51 +++++++++++++--------
  4 files changed, 209 insertions(+), 60 deletions(-)
drivers/iommu/io-pgtable-arm-v7s.c +0 -0
index 1d92ac9..89aad2f 100644
@@ -28,6 +28,7 @@

[..]

drivers/iommu/io-pgtable-arm-v7s.c

Quoted 8 lines
drivers/iommu/io-pgtable-arm-v7s.c +8 -0
+static int __init arm_lpae_init(void)
+{
+	int ret, i;
+
+	for (i = 0; i < ARRAY_SIZE(io_pgtable_arm_lpae_init_fns); i++) {
+		ret = io_pgtable_ops_register(&io_pgtable_arm_lpae_init_fns[i]);
+		if (ret < 0) {
+			pr_err("Failed to register ARM LPAE fmt: %d\n");

I guess we want to print the format here?

Thanks,
Georgi

#7 lore raw

Re: [PATCH v2 1/7] iommu/io-pgtable: Introduce dynamic io-pgtable fmt registration

In reply to Isaac J. Manjarres
Thank you for the patch! Perhaps something to improve:

Hi "Isaac,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on iommu/next]
[also build test WARNING on soc/for-next xlnx/master linus/master v5.10 next-20201223] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Isaac-J-Manjarres/iommu-Permit-modular-builds-of-io-pgtable-drivers/20201222-085121
base:   https://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git next
config: alpha-randconfig-r023-20201221 (attached as .config)
compiler: alpha-linux-gcc (GCC) 9.3.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/9cc3cfc5f79e5c9072aea2218dd9080227933caa
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Isaac-J-Manjarres/iommu-Permit-modular-builds-of-io-pgtable-drivers/20201222-085121 git checkout 9cc3cfc5f79e5c9072aea2218dd9080227933caa
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=alpha

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

In file included from include/linux/printk.h:7,
from include/linux/kernel.h:16, from include/asm-generic/bug.h:20, from arch/alpha/include/asm/bug.h:23, from include/linux/bug.h:5,
from include/linux/scatterlist.h:7, from include/linux/iommu.h:10, from include/linux/io-pgtable.h:6, from drivers/iommu/io-pgtable-arm.c:14:
drivers/iommu/io-pgtable-arm.c: In function 'arm_lpae_init':

>> include/linux/kern_levels.h:5:18: warning: format '%d' expects a matching 'int' argument [-Wformat=]
       5 | #define KERN_SOH "\001"  /* ASCII Start Of Header */
         |                  ^~~~~~
   include/linux/kern_levels.h:11:18: note: in expansion of macro 'KERN_SOH'
      11 | #define KERN_ERR KERN_SOH "3" /* error conditions */
         |                  ^~~~~~~~
   include/linux/printk.h:343:9: note: in expansion of macro 'KERN_ERR'
     343 |  printk(KERN_ERR pr_fmt(fmt), ##__VA_ARGS__)
         |         ^~~~~~~~
   drivers/iommu/io-pgtable-arm.c:1257:4: note: in expansion of macro 'pr_err'
    1257 |    pr_err("Failed to register ARM LPAE fmt: %d\n");
         |    ^~~~~~
   drivers/iommu/io-pgtable-arm.c:1257:46: note: format string is defined here
    1257 |    pr_err("Failed to register ARM LPAE fmt: %d\n");
         |                                             ~^
         |                                              |
         |                                              int

vim +5 include/linux/kern_levels.h

314ba3520e513a7 Joe Perches 2012-07-30 4
04d2c8c83d0e3ac Joe Perches 2012-07-30 @5 #define KERN_SOH "\001" /* ASCII Start Of Header */
04d2c8c83d0e3ac Joe Perches 2012-07-30 6 #define KERN_SOH_ASCII '\001'
04d2c8c83d0e3ac Joe Perches 2012-07-30 7

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#16 lore raw

Re: [PATCH v2 5/7] drm/msm: Add dependency on io-pgtable-arm format module

In reply to Isaac J. Manjarres
Quoted 8 lines · Isaac J. Manjarres

The MSM DRM driver depends on the availability of the ARM LPAE io-pgtable format code to work properly. In preparation for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable-arm format module is loaded before loading the MSM DRM driver module.

Thanks, I've queued this up locally

BR,
-R

Quoted 17 lines
---
 drivers/gpu/drm/msm/msm_drv.c | 1 +
 1 file changed, 1 insertion(+)
drivers/gpu/drm/msm/msm_drv.c +1 -1
index 535a026..8be3506 100644
@@ -1369,3 +1369,4 @@ module_exit(msm_drm_unregister);
 MODULE_AUTHOR("Rob Clark <[email protected]");
 MODULE_DESCRIPTION("MSM DRM Driver");
 MODULE_LICENSE("GPL");
+MODULE_SOFTDEP("pre: io-pgtable-arm");
--

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project

Quoted 4 lines

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

maintainer
#17 lore raw

Re: [PATCH v2 5/7] drm/msm: Add dependency on io-pgtable-arm format module

Quoted 9 lines · Rob Clark
Quoted 7 lines · Isaac J. Manjarres

The MSM DRM driver depends on the availability of the ARM LPAE io-pgtable format code to work properly. In preparation for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable-arm format module is loaded before loading the MSM DRM driver module.

Quoted 3 lines
Quoted 1 line

Thanks, I've queued this up locally

I don't plan to make the io-pgtable code modular, so please drop this patch.

https://lore.kernel.org/r/20210106123428.GA1798@willie-the-truck

Will
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#18 lore raw

Re: [PATCH v2 5/7] drm/msm: Add dependency on io-pgtable-arm format module

Quoted 11 lines · Will Deacon
Quoted 9 lines · Rob Clark
Quoted 7 lines · Isaac J. Manjarres

The MSM DRM driver depends on the availability of the ARM LPAE io-pgtable format code to work properly. In preparation for having the io-pgtable formats as modules, add a "pre" dependency with MODULE_SOFTDEP() to ensure that the io-pgtable-arm format module is loaded before loading the MSM DRM driver module.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

Thanks, I've queued this up locally

I don't plan to make the io-pgtable code modular, so please drop this patch.

https://lore.kernel.org/r/20210106123428.GA1798@willie-the-truck

Ok, done. Thanks

BR,
-R