Skip to contentSkip to conversation

Forbid static SRCU use in modules

RFC PATCH In review Paul E. McKenney 6 participants 47 messages Apr 2, 2019 – Apr 9, 2019

Subsystems (11)
lore
Share & tools
lore
#0 lore raw

cover Forbid static SRCU use in modules

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

This series consist of the following:

1. Dynamically allocate dax_srcu.

2. Dynamically allocate drm_unplug_srcu.

3. Dynamically allocate kfd_processes_srcu.

These build and have been subjected to 0day testing, but might also need testing by someone having the requisite hardware.

Thanx, Paul

------------------------------------------------------------------------

 drivers/dax/super.c                        |   10 +++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c |    5 +++
 drivers/gpu/drm/amd/amdkfd/kfd_process.c   |    2 -
 drivers/gpu/drm/drm_drv.c                  |    8 ++++
 include/linux/srcutree.h                   |   19 +++++++++--
 kernel/rcu/rcuperf.c                       |   40 +++++++++++++++++++-----
 kernel/rcu/rcutorture.c                    |   48 +++++++++++++++++++++--------
 7 files changed, 105 insertions(+), 27 deletions(-)
#4 lore raw

RFC PATCH 2/4 drivers/gpu/drm: Dynamically allocate drm_unplug_srcu

Having DEFINE_SRCU() or DEFINE_STATIC_SRCU() in a loadable module requires that the size of the reserved region be increased, which is not something we really want to be doing. This commit therefore removes the DEFINE_STATIC_SRCU() from drivers/gpu/drm/drm_drv.c in favor of defining drm_unplug_srcu as a simple srcu_struct, initializing it in drm_core_init(), and cleaning it up in drm_core_exit().

In this particular case, drm_unplug_srcu is statically allocated and thus guaranteed to be initially zero. Unfortunately, cleanup_srcu_struct() cannot rely on this in the general case (which includes dynamically allocated srcu_struct structures), and therefore cannot tell whether it is being called on something that has been passed to init_srcu_struct(). Thus the code added to drm_core_init() is a bit non-standard.

Cc Maarten Lankhorst <[email protected]>
Cc Maxime Ripard <[email protected]>
Cc Sean Paul <[email protected]>
Cc David Airlie <[email protected]>
Cc Daniel Vetter <[email protected]>
Cc Tejun Heo <[email protected]>
---
 drivers/gpu/drm/drm_drv.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/gpu/drm/drm_drv.c +7 -1
index 381581b01d48..ce4582af814d 100644
@@ -77,7 +77,7 @@ static bool drm_core_init_complete = false;
 
 static struct dentry *drm_debugfs_root;
 
-DEFINE_STATIC_SRCU(drm_unplug_srcu);
+static struct srcu_struct drm_unplug_srcu;
 
 /*
  * DRM Minors
@@ -958,12 +958,18 @@ static void drm_core_exit(void)
 	drm_sysfs_destroy();
 	idr_destroy(&drm_minors_idr);
 	drm_connector_ida_destroy();
+	cleanup_srcu_struct(&drm_unplug_srcu);
 }
 
 static int __init drm_core_init(void)
 {
 	int ret;
 
+	ret = init_srcu_struct(&drm_unplug_srcu);
+	if (ret) {
+		DRM_ERROR("Cannot create DRM class: %d\n", ret);
+		return ret; /* cannot cleanup after failed srcu_struct init. */
+	}
 	drm_connector_ida_init();
 	idr_init(&drm_minors_idr);
 
Signature · 1 line
2.17.1
#6 lore raw

RFC PATCH 3/4 drivers/gpu/drm/amd: Dynamically allocate kfd_processes_srcu

In reply to Paul E. McKenney

Having DEFINE_SRCU() or DEFINE_STATIC_SRCU() in a loadable module requires that the size of the reserved region be increased, which is not something we really want to be doing. This commit therefore removes the DEFINE_STATIC_SRCU() from drivers/gpu/drm/amd/amdkfd/kfd_process.c in favor of defining kfd_processes_srcu as a simple srcu_struct, initializing it in amdgpu_amdkfd_init(), and cleaning it up in amdgpu_amdkfd_fini().

Cc Oded Gabbay <[email protected]>
Cc Alex Deucher <[email protected]>
Cc "Christian König" <[email protected]
Cc "David (ChunMing) Zhou" <[email protected]>
Cc David Airlie <[email protected]>
Cc Daniel Vetter <[email protected]>
Cc Tejun Heo <[email protected]>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 5 +++++
 drivers/gpu/drm/amd/amdkfd/kfd_process.c   | 2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +5 -0
index fe1d7368c1e6..eadb20dee867 100644
@@ -28,6 +28,8 @@
 #include <linux/module.h>
 #include <linux/dma-buf.h>
 
+extern struct srcu_struct kfd_processes_srcu;
+
 static const unsigned int compute_vmid_bitmap = 0xFF00;
 
 /* Total memory size in system memory and all GPU VRAM. Used to
@@ -40,6 +42,8 @@ int amdgpu_amdkfd_init(void)
 	struct sysinfo si;
 	int ret;
 
+	ret = init_srcu_struct(&kfd_processes_srcu);
+	WARN_ON(ret);
 	si_meminfo(&si);
 	amdgpu_amdkfd_total_mem_size = si.totalram - si.totalhigh;
 	amdgpu_amdkfd_total_mem_size *= si.mem_unit;
@@ -57,6 +61,7 @@ int amdgpu_amdkfd_init(void)
 void amdgpu_amdkfd_fini(void)
 {
 	kgd2kfd_exit();
+	cleanup_srcu_struct(&kfd_processes_srcu);
 }
 
 void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdkfd/kfd_process.c +1 -1
index 4bdae78bab8e..98b694068b8a 100644
@@ -47,7 +47,7 @@ struct mm_struct;
 DEFINE_HASHTABLE(kfd_processes_table, KFD_PROCESS_TABLE_SIZE);
 static DEFINE_MUTEX(kfd_processes_mutex);
 
-DEFINE_SRCU(kfd_processes_srcu);
+struct srcu_struct kfd_processes_srcu;
 
 /* For process termination handling */
 static struct workqueue_struct *kfd_process_wq;
Signature · 1 line
2.17.1
#1 lore raw

RFC PATCH 1/4 dax/super: Dynamically allocate dax_srcu

In reply to Paul E. McKenney

Having DEFINE_SRCU() or DEFINE_STATIC_SRCU() in a loadable module requires that the size of the reserved region be increased, which is not something we really want to be doing. This commit therefore removes the DEFINE_STATIC_SRCU() from drivers/dax/super.c in favor of defining dax_srcu as a simple srcu_struct, initializing it in dax_core_init(), and cleaning it up in dax_core_exit().

---
 drivers/dax/super.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
drivers/dax/super.c +9 -1
index 0a339b85133e..3b152db30c6b 100644
@@ -25,7 +25,7 @@
 #include "dax-private.h"
 
 static dev_t dax_devt;
-DEFINE_STATIC_SRCU(dax_srcu);
+static struct srcu_struct dax_srcu;
 static struct vfsmount *dax_mnt;
 static DEFINE_IDA(dax_minor_ida);
 static struct kmem_cache *dax_cache __read_mostly;
@@ -665,6 +665,10 @@ static int __init dax_core_init(void)
 {
 	int rc;
 
+	rc = init_srcu_struct(&dax_srcu);
+	if (rc)
+		goto err_srcu;
+
 	rc = dax_fs_init();
 	if (rc)
 		return rc;
@@ -678,10 +682,13 @@ static int __init dax_core_init(void)
 		goto err_bus;
 	return 0;
 
+
 err_bus:
 	unregister_chrdev_region(dax_devt, MINORMASK+1);
 err_chrdev:
 	dax_fs_exit();
+	cleanup_srcu_struct(&dax_srcu);
+err_srcu:
 	return 0;
 }
 
@@ -690,6 +697,7 @@ static void __exit dax_core_exit(void)
 	unregister_chrdev_region(dax_devt, MINORMASK+1);
 	ida_destroy(&dax_minor_ida);
 	dax_fs_exit();
+	cleanup_srcu_struct(&dax_srcu);
 }
 
 MODULE_AUTHOR("Intel Corporation");
Signature · 6 lines
2.17.1

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
#9 lore raw

RFC PATCH 4/4 rcu: Forbid DEFINE{,_STATIC}_SRCU() from modules

In reply to Paul E. McKenney

Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

This commit enforces this advice by refusing to define DEFINE_SRCU() and DEFINE_STATIC_SRCU() within loadable modules.

---
 include/linux/srcutree.h | 19 ++++++++++++----
 kernel/rcu/rcuperf.c     | 40 ++++++++++++++++++++++++++-------
 kernel/rcu/rcutorture.c  | 48 ++++++++++++++++++++++++++++++----------
 3 files changed, 83 insertions(+), 24 deletions(-)
include/linux/srcutree.h +15 -4
index 7f7c8c050f63..ac5ea1c72e97 100644
@@ -105,6 +105,15 @@ struct srcu_struct {
  * Define and initialize a srcu struct at build time.
  * Do -not- call init_srcu_struct() nor cleanup_srcu_struct() on it.
  *
+ * Build-time srcu_struct definition is not allowed in modules because
+ * otherwise it is necessary to increase the size of the reserved region
+ * each time a DEFINE_SRCU() or DEFINE_STATIC_SRCU() are added to a
+ * kernel module.  Kernel modules should instead declare an srcu_struct
+ * and then invoke init_srcu_struct() from their module_init function and
+ * cleanup_srcu_struct() from their module_exit function.  Note that modules
+ * using call_srcu() will also need to invoke srcu_barrier() from their
+ * module_exit function.
+ *
  * Note that although DEFINE_STATIC_SRCU() hides the name from other
  * files, the per-CPU variable rules nevertheless require that the
  * chosen name be globally unique.  These rules also prohibit use of
@@ -120,11 +129,13 @@ struct srcu_struct {
  *
  * See include/linux/percpu-defs.h for the rules on per-CPU variables.
  */
-#define __DEFINE_SRCU(name, is_static)					\
-	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);\
+#ifndef MODULE
+#  define __DEFINE_SRCU(name, is_static)				\
+	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);	\
 	is_static struct srcu_struct name = __SRCU_STRUCT_INIT(name, name##_srcu_data)
-#define DEFINE_SRCU(name)		__DEFINE_SRCU(name, /* not static */)
-#define DEFINE_STATIC_SRCU(name)	__DEFINE_SRCU(name, static)
+#  define DEFINE_SRCU(name)		__DEFINE_SRCU(name, /* not static */)
+#  define DEFINE_STATIC_SRCU(name)	__DEFINE_SRCU(name, static)
+#endif
 
 void synchronize_srcu_expedited(struct srcu_struct *ssp);
 void srcu_barrier(struct srcu_struct *ssp);
kernel/rcu/rcuperf.c +32 -8
index 7a6890b23c5f..8b24b1c929b0 100644
@@ -139,6 +139,7 @@ struct rcu_perf_ops {
 	void (*sync)(void);
 	void (*exp_sync)(void);
 	const char *name;
+	const char *altname;
 };
 
 static struct rcu_perf_ops *cur_ops;
@@ -186,8 +187,16 @@ static struct rcu_perf_ops rcu_ops = {
  * Definitions for srcu perf testing.
  */
 
+static struct srcu_struct *srcu_ctlp;
+
+#ifndef MODULE
 DEFINE_STATIC_SRCU(srcu_ctl_perf);
-static struct srcu_struct *srcu_ctlp = &srcu_ctl_perf;
+
+static void srcu_sync_perf_init(void)
+{
+	srcu_ctlp = &srcu_ctl_perf;
+}
+#endif
 
 static int srcu_perf_read_lock(void) __acquires(srcu_ctlp)
 {
@@ -224,9 +233,10 @@ static void srcu_perf_synchronize_expedited(void)
 	synchronize_srcu_expedited(srcu_ctlp);
 }
 
+#ifndef MODULE
 static struct rcu_perf_ops srcu_ops = {
 	.ptype		= SRCU_FLAVOR,
-	.init		= rcu_sync_perf_init,
+	.init		= srcu_sync_perf_init,
 	.readlock	= srcu_perf_read_lock,
 	.readunlock	= srcu_perf_read_unlock,
 	.get_gp_seq	= srcu_perf_completed,
@@ -238,24 +248,28 @@ static struct rcu_perf_ops srcu_ops = {
 	.exp_sync	= srcu_perf_synchronize_expedited,
 	.name		= "srcu"
 };
+#define SRCU_OPS (&srcu_ops)
+#else /* #ifndef MODULE */
+#define SRCU_OPS NULL
+#endif /* #else #ifndef MODULE */
 
 static struct srcu_struct srcud;
 
-static void srcu_sync_perf_init(void)
+static void srcud_sync_perf_init(void)
 {
 	srcu_ctlp = &srcud;
 	init_srcu_struct(srcu_ctlp);
 }
 
-static void srcu_sync_perf_cleanup(void)
+static void srcud_sync_perf_cleanup(void)
 {
 	cleanup_srcu_struct(srcu_ctlp);
 }
 
 static struct rcu_perf_ops srcud_ops = {
 	.ptype		= SRCU_FLAVOR,
-	.init		= srcu_sync_perf_init,
-	.cleanup	= srcu_sync_perf_cleanup,
+	.init		= srcud_sync_perf_init,
+	.cleanup	= srcud_sync_perf_cleanup,
 	.readlock	= srcu_perf_read_lock,
 	.readunlock	= srcu_perf_read_unlock,
 	.get_gp_seq	= srcu_perf_completed,
@@ -265,7 +279,10 @@ static struct rcu_perf_ops srcud_ops = {
 	.gp_barrier	= srcu_rcu_barrier,
 	.sync		= srcu_perf_synchronize,
 	.exp_sync	= srcu_perf_synchronize_expedited,
-	.name		= "srcud"
+	.name		= "srcud",
+#ifndef MODULE
+	.altname	= "srcu" /* Avoid breaking kbuild test robot. */
+#endif
 };
 
 /*
@@ -598,7 +615,7 @@ rcu_perf_init(void)
 	long i;
 	int firsterr = 0;
 	static struct rcu_perf_ops *perf_ops[] = {
-		&rcu_ops, &srcu_ops, &srcud_ops, &tasks_ops,
+		&rcu_ops, SRCU_OPS, &srcud_ops, &tasks_ops,
 	};
 
 	if (!torture_init_begin(perf_type, verbose))
@@ -607,8 +624,15 @@ rcu_perf_init(void)
 	/* Process args and tell the world that the perf'er is on the job. */
 	for (i = 0; i < ARRAY_SIZE(perf_ops); i++) {
 		cur_ops = perf_ops[i];
+		if (!cur_ops)
+			continue;
 		if (strcmp(perf_type, cur_ops->name) == 0)
 			break;
+		if (cur_ops->altname &&
+		    strcmp(perf_type, cur_ops->altname) == 0) {
+			pr_alert("rcu-perf: substituting perf type: \"%s\" for \"%s\"\n", cur_ops->name, perf_type);
+			break;
+		}
 	}
 	if (i == ARRAY_SIZE(perf_ops)) {
 		pr_alert("rcu-perf: invalid perf type: \"%s\"\n", perf_type);
kernel/rcu/rcutorture.c +36 -12

114 lines not on this page · raw message

Signature · 1 line
2.17.1
#10 lore raw
In reply to Paul E. McKenney
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck [email protected] wrote:

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Thanks,

Mathieu

Quoted 24 lines

This series consist of the following:

1. Dynamically allocate dax_srcu.

2. Dynamically allocate drm_unplug_srcu.

3. Dynamically allocate kfd_processes_srcu.

These build and have been subjected to 0day testing, but might also need testing by someone having the requisite hardware.

Thanx, Paul

------------------------------------------------------------------------

drivers/dax/super.c                        |   10 +++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c |    5 +++
drivers/gpu/drm/amd/amdkfd/kfd_process.c   |    2 -
drivers/gpu/drm/drm_drv.c                  |    8 ++++
include/linux/srcutree.h                   |   19 +++++++++--
kernel/rcu/rcuperf.c                       |   40 +++++++++++++++++++-----
kernel/rcu/rcutorture.c                    |   48 +++++++++++++++++++++--------
 7 files changed, 105 insertions(+), 27 deletions(-)
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#11 lore raw
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck [email protected] wrote:

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

Thanx, Paul

Quoted 35 lines

Thanks,

Mathieu

Quoted 24 lines

This series consist of the following:

1. Dynamically allocate dax_srcu.

2. Dynamically allocate drm_unplug_srcu.

3. Dynamically allocate kfd_processes_srcu.

These build and have been subjected to 0day testing, but might also need testing by someone having the requisite hardware.

Thanx, Paul

------------------------------------------------------------------------

drivers/dax/super.c                        |   10 +++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c |    5 +++
drivers/gpu/drm/amd/amdkfd/kfd_process.c   |    2 -
drivers/gpu/drm/drm_drv.c                  |    8 ++++
include/linux/srcutree.h                   |   19 +++++++++--
kernel/rcu/rcuperf.c                       |   40 +++++++++++++++++++-----
kernel/rcu/rcutorture.c                    |   48 +++++++++++++++++++++--------
 7 files changed, 105 insertions(+), 27 deletions(-)
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#12 lore raw
Quoted 20 lines · ----- On Apr 2, 2019, at 11:23 AM, paulmck…
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck…

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

I was more thinking that under #ifdef MODULE, both DEFINE_SRCU and DEFINE_STATIC_SRCU could append data in a dedicated section. module.c would additionally lookup that section on module load, and deal with those statically defined SRCU entries as if they were dynamically allocated ones. It would of course cleanup those resources on module unload.

Am I missing some subtlety there ?

Thanks,

Mathieu

Quoted 37 lines

Thanx, Paul

Quoted 34 lines

Thanks,

Mathieu

Quoted 16 lines

This series consist of the following:

1. Dynamically allocate dax_srcu.

2. Dynamically allocate drm_unplug_srcu.

3. Dynamically allocate kfd_processes_srcu.

These build and have been subjected to 0day testing, but might also need testing by someone having the requisite hardware.

Thanx, Paul

------------------------------------------------------------------------

Quoted 8 lines
drivers/dax/super.c                        |   10 +++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c |    5 +++
drivers/gpu/drm/amd/amdkfd/kfd_process.c   |    2 -
drivers/gpu/drm/drm_drv.c                  |    8 ++++
include/linux/srcutree.h                   |   19 +++++++++--
kernel/rcu/rcuperf.c                       |   40 +++++++++++++++++++-----
kernel/rcu/rcutorture.c                    |   48 +++++++++++++++++++++--------
 7 files changed, 105 insertions(+), 27 deletions(-)

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
Reviewed-by on v1 2/4
#5 lore raw

Re: [PATCH RFC tip/core/rcu 2/4] drivers/gpu/drm: Dynamically allocate drm_unplug_srcu

In reply to Paul E. McKenney
Quoted 14 lines · Paul E. McKenney

Having DEFINE_SRCU() or DEFINE_STATIC_SRCU() in a loadable module requires that the size of the reserved region be increased, which is not something we really want to be doing. This commit therefore removes the DEFINE_STATIC_SRCU() from drivers/gpu/drm/drm_drv.c in favor of defining drm_unplug_srcu as a simple srcu_struct, initializing it in drm_core_init(), and cleaning it up in drm_core_exit().

In this particular case, drm_unplug_srcu is statically allocated and thus guaranteed to be initially zero. Unfortunately, cleanup_srcu_struct() cannot rely on this in the general case (which includes dynamically allocated srcu_struct structures), and therefore cannot tell whether it is being called on something that has been passed to init_srcu_struct(). Thus the code added to drm_core_init() is a bit non-standard.

Quoted 9 lines
Cc Maarten Lankhorst <[email protected]>
Cc Maxime Ripard <[email protected]>
Cc Sean Paul <[email protected]>
Cc David Airlie <[email protected]>
Cc Daniel Vetter <[email protected]>
Cc Tejun Heo <[email protected]>

Seems reasonable (but I don't have the full patch series as context).

Any preferences as to how this should be merged? Upfront ack for merging through rcu trees if that helps you.
-Daniel

Quoted 39 lines
---
 drivers/gpu/drm/drm_drv.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/gpu/drm/drm_drv.c +7 -1
index 381581b01d48..ce4582af814d 100644
@@ -77,7 +77,7 @@ static bool drm_core_init_complete = false;
 
 static struct dentry *drm_debugfs_root;
 
-DEFINE_STATIC_SRCU(drm_unplug_srcu);
+static struct srcu_struct drm_unplug_srcu;
 
 /*
  * DRM Minors
@@ -958,12 +958,18 @@ static void drm_core_exit(void)
 	drm_sysfs_destroy();
 	idr_destroy(&drm_minors_idr);
 	drm_connector_ida_destroy();
+	cleanup_srcu_struct(&drm_unplug_srcu);
 }
 
 static int __init drm_core_init(void)
 {
 	int ret;
 
+	ret = init_srcu_struct(&drm_unplug_srcu);
+	if (ret) {
+		DRM_ERROR("Cannot create DRM class: %d\n", ret);
+		return ret; /* cannot cleanup after failed srcu_struct init. */
+	}
 	drm_connector_ida_init();
 	idr_init(&drm_minors_idr);
 
Signature · 1 line
2.17.1
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
maintainer
#7 lore raw

Re: [PATCH RFC tip/core/rcu 3/4] drivers/gpu/drm/amd: Dynamically allocate kfd_processes_srcu

In reply to Paul E. McKenney
Quoted 38 lines · Paul E. McKenney

Having DEFINE_SRCU() or DEFINE_STATIC_SRCU() in a loadable module requires that the size of the reserved region be increased, which is not something we really want to be doing. This commit therefore removes the DEFINE_STATIC_SRCU() from drivers/gpu/drm/amd/amdkfd/kfd_process.c in favor of defining kfd_processes_srcu as a simple srcu_struct, initializing it in amdgpu_amdkfd_init(), and cleaning it up in amdgpu_amdkfd_fini().

Cc Oded Gabbay <[email protected]>
Cc Alex Deucher <[email protected]>
Cc "Christian König" <[email protected]
Cc "David (ChunMing) Zhou" <[email protected]>
Cc David Airlie <[email protected]>
Cc Daniel Vetter <[email protected]>
Cc Tejun Heo <[email protected]>
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 5 +++++
  drivers/gpu/drm/amd/amdkfd/kfd_process.c   | 2 +-
  2 files changed, 6 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +2 -0
index fe1d7368c1e6..eadb20dee867 100644
@@ -28,6 +28,8 @@
  #include <linux/module.h>
  #include <linux/dma-buf.h>
 
+extern struct srcu_struct kfd_processes_srcu;
+
  static const unsigned int compute_vmid_bitmap = 0xFF00;
 
  /* Total memory size in system memory and all GPU VRAM. Used to

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c int amdgpu_amdkfd_init(void) · line 46

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +2 -0
@@ -40,6 +42,8 @@ int amdgpu_amdkfd_init(void)
  	struct sysinfo si;
  	int ret;
 
+	ret = init_srcu_struct(&kfd_processes_srcu);
+	WARN_ON(ret);

kfd_processes_srcu only exists if kfd_process.c is compiled in. That depends on CONFIG_HSA_AMD. So this should at least move into #ifdef a few lines below.

However, it would be cleaner to move this initialization into kfd_init in kfd_module.c, or better yet, into kfd_process_create_wq in kfd_process.c. Then kfd_process_create_wq should be renamed to something more generic, such as kfd_process_init.

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c int amdgpu_amdkfd_init(void) · line 64

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1 -0
  	si_meminfo(&si);
  	amdgpu_amdkfd_total_mem_size = si.totalram - si.totalhigh;
  	amdgpu_amdkfd_total_mem_size *= si.mem_unit;
@@ -57,6 +61,7 @@ int amdgpu_amdkfd_init(void)
  void amdgpu_amdkfd_fini(void)
  {
  	kgd2kfd_exit();
+	cleanup_srcu_struct(&kfd_processes_srcu);

Similarly, this would be cleaner in kfd_exit in kfd_module.c or kfd_process_destroy_wq in kfd_process.c, with that function similarly renamed to kfd_process_fini.

I'm attaching a revised patch. It's only compile tested.

Regards,
  Felix

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0 -0
  }
 
  void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdkfd/kfd_process.c +0 -0
index 4bdae78bab8e..98b694068b8a 100644

drivers/gpu/drm/amd/amdkfd/kfd_process.c struct mm_struct; · line 53

Quoted 8 lines
drivers/gpu/drm/amd/amdkfd/kfd_process.c +1 -1
@@ -47,7 +47,7 @@ struct mm_struct;
  DEFINE_HASHTABLE(kfd_processes_table, KFD_PROCESS_TABLE_SIZE);
  static DEFINE_MUTEX(kfd_processes_mutex);
 
-DEFINE_SRCU(kfd_processes_srcu);
+struct srcu_struct kfd_processes_srcu;
 
  /* For process termination handling */
  static struct workqueue_struct *kfd_process_wq;

From 5857a9aa63957a5755ff81ae5c46533bca408c12 Mon Sep 17 00:00:00 2001
From: "Paul E. McKenney" <[email protected]>
Date: Tue, 2 Apr 2019 07:29:32 -0700
Subject: [PATCH 1/1] drivers/gpu/drm/amd: Dynamically allocate
kfd_processes_srcu v2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Having DEFINE_SRCU() or DEFINE_STATIC_SRCU() in a loadable module requires that the size of the reserved region be increased, which is not something we really want to be doing. This commit therefore removes the DEFINE_STATIC_SRCU() from drivers/gpu/drm/amd/amdkfd/kfd_process.c in favor of defining kfd_processes_srcu as a simple srcu_struct, initializing it in kfd_process_init(), and cleaning it up in kfd_process_fini().

v2 (Felix Kuehling): Move srcu init and cleanup into kfd_process.c

Tested-by (v1): kbuild test robot <[email protected]>
Cc: Oded Gabbay <[email protected]>
Cc: Alex Deucher <[email protected]>
Cc: "Christian König" <[email protected]>
Cc: "David (ChunMing) Zhou" <[email protected]>
Cc: David Airlie <[email protected]>
Cc: Daniel Vetter <[email protected]>
Cc: Tejun Heo <[email protected]>
Cc: <[email protected]>
Cc: <[email protected]>

---
 drivers/gpu/drm/amd/amdkfd/kfd_module.c  |  8 ++++----
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h    |  4 ++--
 drivers/gpu/drm/amd/amdkfd/kfd_process.c | 19 ++++++++++++-------
 3 files changed, 18 insertions(+), 13 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_module.c +4 -4
index 932007e..e8e2c15 100644
@@ -52,15 +52,15 @@ static int kfd_init(void)
 	if (err < 0)
 		goto err_topology;
 
-	err = kfd_process_create_wq();
+	err = kfd_process_init();
 	if (err < 0)
-		goto err_create_wq;
+		goto err_process;
 
 	kfd_debugfs_init();
 
 	return 0;
 
-err_create_wq:
+err_process:
 	kfd_topology_shutdown();
 err_topology:
 	kfd_chardev_exit();
@@ -71,7 +71,7 @@ static int kfd_init(void)
 static void kfd_exit(void)
 {
 	kfd_debugfs_fini();
-	kfd_process_destroy_wq();
+	kfd_process_fini();
 	kfd_topology_shutdown();
 	kfd_chardev_exit();
 }
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +2 -2
index 9e02309..9dd187c 100644
@@ -708,8 +708,8 @@ struct amdkfd_ioctl_desc {
 };
 bool kfd_dev_is_large_bar(struct kfd_dev *dev);
 
-int kfd_process_create_wq(void);
-void kfd_process_destroy_wq(void);
+int kfd_process_init(void);
+void kfd_process_fini(void);
 struct kfd_process *kfd_create_process(struct file *filep);
 struct kfd_process *kfd_get_process(const struct task_struct *);
 struct kfd_process *kfd_lookup_process_by_pasid(unsigned int pasid);
drivers/gpu/drm/amd/amdkfd/kfd_process.c +12 -7
index 4bdae78..f810911 100644
@@ -47,7 +47,7 @@ struct mm_struct;
 DEFINE_HASHTABLE(kfd_processes_table, KFD_PROCESS_TABLE_SIZE);
 static DEFINE_MUTEX(kfd_processes_mutex);
 
-DEFINE_SRCU(kfd_processes_srcu);
+struct srcu_struct kfd_processes_srcu;
 
 /* For process termination handling */
 static struct workqueue_struct *kfd_process_wq;
@@ -69,22 +69,25 @@ static void evict_process_worker(struct work_struct *work);
 static void restore_process_worker(struct work_struct *work);
 
 
-int kfd_process_create_wq(void)
+int kfd_process_init(void)
 {
+	int ret;
+
 	if (!kfd_process_wq)
 		kfd_process_wq = alloc_workqueue("kfd_process_wq", 0, 0);
 	if (!kfd_restore_wq)
 		kfd_restore_wq = alloc_ordered_workqueue("kfd_restore_wq", 0);
 
-	if (!kfd_process_wq || !kfd_restore_wq) {
-		kfd_process_destroy_wq();
+	if (!kfd_process_wq || !kfd_restore_wq)
 		return -ENOMEM;
-	}
 
-	return 0;
+	ret = init_srcu_struct(&kfd_processes_srcu);
+	WARN_ON(ret);
+
+	return ret;
 }
 
-void kfd_process_destroy_wq(void)
+void kfd_process_fini(void)
 {
 	if (kfd_process_wq) {
 		destroy_workqueue(kfd_process_wq);
@@ -94,6 +97,8 @@ void kfd_process_destroy_wq(void)
 		destroy_workqueue(kfd_restore_wq);
 		kfd_restore_wq = NULL;
 	}
+
+	cleanup_srcu_struct(&kfd_processes_srcu);
 }
 
 static void kfd_process_free_gpuvm(struct kgd_mem *mem,
Signature · 1 line
2.7.4
#45 lore raw
In reply to Paul E. McKenney
Quoted 20 lines · Paul E. McKenney
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck…

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

Hi Paul,

Which 'reserved region' are you referring to? Isn't this region also used for non-module cases in which case the same problem applies to non-modules?

thanks!

- Joel

#46 lore raw
Quoted 27 lines · Joel Fernandes
Quoted 26 lines · Paul E. McKenney
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck [email protected] wrote:

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Quoted 7 lines
Quoted 1 line

Or am I missing a trick here?

Hi Paul,

Which 'reserved region' are you referring to? Isn't this region also used for non-module cases in which case the same problem applies to non-modules?

The percpu/module reservation discussed in this thread:

https://lore.kernel.org/lkml/[email protected]/T/#mbcacf60ddc0b3fd6e831a3ea71efc90da359a3bf

For non-modules, global per-CPU variables are statically allocated. For modules, they must be dynamically allocated at modprobe time, and their size is set by PERCPU_MODULE_RESERVE.

Thanx, Paul

#13 lore raw
In reply to Mathieu Desnoyers
Quoted 30 lines · Mathieu Desnoyers
Quoted 20 lines · ----- On Apr 2, 2019, at 11:23 AM, paulmck [email protected] wrote:
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck [email protected] wrote:

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

Quoted 8 lines

I was more thinking that under #ifdef MODULE, both DEFINE_SRCU and DEFINE_STATIC_SRCU could append data in a dedicated section. module.c would additionally lookup that section on module load, and deal with those statically defined SRCU entries as if they were dynamically allocated ones. It would of course cleanup those resources on module unload.

Am I missing some subtlety there ?

If I understand you correctly, that is actually what is already done. The size of this dedicated section is currently set by PERCPU_MODULE_RESERVE, and the additions of DEFINE{_STATIC}_SRCU() in modules was requiring that this to be increased frequently. That led to a request that something be done, in turn leading to this patch series.

I don't see a way around this short of changing module loading to do alloc_percpu() and then updating the relocation based on this result. Which would admittedly be far more convenient. I was assuming that this would be difficult due to varying CPU offsets or the like.

But if it can be done reasonably, it would be quite a bit nicer than forcing dynamic allocation in cases where it is not otherwise needed.

Thanx, Paul

Quoted 48 lines

Thanks,

Mathieu

Quoted 37 lines

Thanx, Paul

Quoted 34 lines

Thanks,

Mathieu

Quoted 16 lines

This series consist of the following:

1. Dynamically allocate dax_srcu.

2. Dynamically allocate drm_unplug_srcu.

3. Dynamically allocate kfd_processes_srcu.

These build and have been subjected to 0day testing, but might also need testing by someone having the requisite hardware.

Thanx, Paul

------------------------------------------------------------------------

Quoted 8 lines
drivers/dax/super.c                        |   10 +++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c |    5 +++
drivers/gpu/drm/amd/amdkfd/kfd_process.c   |    2 -
drivers/gpu/drm/drm_drv.c                  |    8 ++++
include/linux/srcutree.h                   |   19 +++++++++--
kernel/rcu/rcuperf.c                       |   40 +++++++++++++++++++-----
kernel/rcu/rcutorture.c                    |   48 +++++++++++++++++++++--------
 7 files changed, 105 insertions(+), 27 deletions(-)

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#14 lore raw
Quoted 38 lines · ----- On Apr 3, 2019, at 9:32 AM, paulmck [email protected] wrote:
Quoted 37 lines · Mathieu Desnoyers
Quoted 20 lines · ----- On Apr 2, 2019, at 11:23 AM, paulmck [email protected] wrote:
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck [email protected] wrote:

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

I was more thinking that under #ifdef MODULE, both DEFINE_SRCU and DEFINE_STATIC_SRCU could append data in a dedicated section. module.c would additionally lookup that section on module load, and deal with those statically defined SRCU entries as if they were dynamically allocated ones. It would of course cleanup those resources on module unload.

Quoted 7 lines
Quoted 1 line

Am I missing some subtlety there ?

If I understand you correctly, that is actually what is already done. The size of this dedicated section is currently set by PERCPU_MODULE_RESERVE, and the additions of DEFINE{_STATIC}_SRCU() in modules was requiring that this to be increased frequently. That led to a request that something be done, in turn leading to this patch series.

I think we are not expressing quite the same idea.

AFAIU, yours is to have DEFINE*_SRCU directly define per-cpu data within modules, which ends up using percpu module reserved memory.

My idea is to make DEFINE*_SRCU have a different behavior under #ifdef MODULE. It could emit a _global variable_ (_not_ per-cpu) within a new section. That section would then be used by module init/exit code to figure out what "srcu descriptors" are present in the modules. It would therefore rely on dynamic allocation for those, therefore removing the need to involve the percpu module reserved pool at all.

Quoted 8 lines

I don't see a way around this short of changing module loading to do alloc_percpu() and then updating the relocation based on this result. Which would admittedly be far more convenient. I was assuming that this would be difficult due to varying CPU offsets or the like.

But if it can be done reasonably, it would be quite a bit nicer than forcing dynamic allocation in cases where it is not otherwise needed.

Hopefully my explanation above helps clear out what I have in mind.

You can find similar tricks performed by include/linux/tracepoint.h:

#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)

{
        return offset_to_ptr(p);
}
#define __TRACEPOINT_ENTRY(name)                                        \
        asm("   .section \"__tracepoints_ptrs\", \"a\"          \n"     \
            "   .balign 4                                       \n"     \
            "   .long   __tracepoint_" #name " - .              \n"     \
            "   .previous                                       \n")
#else
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
{
        return *p;
}
#define __TRACEPOINT_ENTRY(name)                                         \
        static tracepoint_ptr_t __tracepoint_ptr_##name __used           \
        __attribute__((section("__tracepoints_ptrs"))) =                 \
                &__tracepoint_##name

#endif

[...]

#define DEFINE_TRACE_FN(name, reg, unreg)                                \
        static const char __tpstrtab_##name[]                            \
        __attribute__((section("__tracepoints_strings"))) = #name;       \
        struct tracepoint __tracepoint_##name                            \
        __attribute__((section("__tracepoints"), used)) =                \
                { __tpstrtab_##name, STATIC_KEY_INIT_FALSE, reg, unreg, NULL };\
        __TRACEPOINT_ENTRY(name);

And kernel/module.c:

find_module_sections():

#ifdef CONFIG_TRACEPOINTS
        mod->tracepoints_ptrs = section_objs(info, "__tracepoints_ptrs",
                                             sizeof(*mod->tracepoints_ptrs),
                                             &mod->num_tracepoints);

#endif

And kernel/tracepoint.c:tracepoint_module_notify() for the module coming/going notifier.

Basically you would want to have your own structure within your own section of the module which describes the srcu domain, and have a module coming/going notifier responsible for dynamically allocating the srcu domain on "coming", and doing a srcu barrier and cleanup the domain on "going".

Thanks,

Mathieu

Quoted 50 lines

Thanx, Paul

Quoted 47 lines

Thanks,

Mathieu

Quoted 37 lines

Thanx, Paul

Quoted 34 lines

Thanks,

Mathieu

Quoted 16 lines

This series consist of the following:

1. Dynamically allocate dax_srcu.

2. Dynamically allocate drm_unplug_srcu.

3. Dynamically allocate kfd_processes_srcu.

These build and have been subjected to 0day testing, but might also need testing by someone having the requisite hardware.

Thanx, Paul

------------------------------------------------------------------------

Quoted 8 lines
drivers/dax/super.c                        |   10 +++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c |    5 +++
drivers/gpu/drm/amd/amdkfd/kfd_process.c   |    2 -
drivers/gpu/drm/drm_drv.c                  |    8 ++++
include/linux/srcutree.h                   |   19 +++++++++--
kernel/rcu/rcuperf.c                       |   40 +++++++++++++++++++-----
kernel/rcu/rcutorture.c                    |   48 +++++++++++++++++++++--------
 7 files changed, 105 insertions(+), 27 deletions(-)

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#15 lore raw
Quoted 116 lines · Mathieu Desnoyers
Quoted 38 lines · ----- On Apr 3, 2019, at 9:32 AM, paulmck…
Quoted 37 lines · Mathieu Desnoyers
Quoted 20 lines · ----- On Apr 2, 2019, at 11:23 AM, paulmck…
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck…

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

I was more thinking that under #ifdef MODULE, both DEFINE_SRCU and DEFINE_STATIC_SRCU could append data in a dedicated section. module.c would additionally lookup that section on module load, and deal with those statically defined SRCU entries as if they were dynamically allocated ones. It would of course cleanup those resources on module unload.

Quoted 7 lines
Quoted 1 line

Am I missing some subtlety there ?

If I understand you correctly, that is actually what is already done. The size of this dedicated section is currently set by PERCPU_MODULE_RESERVE, and the additions of DEFINE{_STATIC}_SRCU() in modules was requiring that this to be increased frequently. That led to a request that something be done, in turn leading to this patch series.

I think we are not expressing quite the same idea.

AFAIU, yours is to have DEFINE*_SRCU directly define per-cpu data within modules, which ends up using percpu module reserved memory.

My idea is to make DEFINE*_SRCU have a different behavior under #ifdef MODULE. It could emit a _global variable_ (_not_ per-cpu) within a new section. That section would then be used by module init/exit code to figure out what "srcu descriptors" are present in the modules. It would therefore rely on dynamic allocation for those, therefore removing the need to involve the percpu module reserved pool at all.

Quoted 8 lines

I don't see a way around this short of changing module loading to do alloc_percpu() and then updating the relocation based on this result. Which would admittedly be far more convenient. I was assuming that this would be difficult due to varying CPU offsets or the like.

But if it can be done reasonably, it would be quite a bit nicer than forcing dynamic allocation in cases where it is not otherwise needed.

Hopefully my explanation above helps clear out what I have in mind.

You can find similar tricks performed by include/linux/tracepoint.h:

#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)

{
        return offset_to_ptr(p);
}
#define __TRACEPOINT_ENTRY(name)                                        \
        asm("   .section \"__tracepoints_ptrs\", \"a\"          \n"     \
            "   .balign 4                                       \n"     \
            "   .long   __tracepoint_" #name " - .              \n"     \
            "   .previous                                       \n")
#else
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
{
        return *p;
}
#define __TRACEPOINT_ENTRY(name)                                         \
        static tracepoint_ptr_t __tracepoint_ptr_##name __used           \
        __attribute__((section("__tracepoints_ptrs"))) =                 \
                &__tracepoint_##name

#endif

[...]

#define DEFINE_TRACE_FN(name, reg, unreg)                                \
        static const char __tpstrtab_##name[]                            \
        __attribute__((section("__tracepoints_strings"))) = #name;       \
        struct tracepoint __tracepoint_##name                            \
        __attribute__((section("__tracepoints"), used)) =                \
                { __tpstrtab_##name, STATIC_KEY_INIT_FALSE, reg, unreg, NULL };\
        __TRACEPOINT_ENTRY(name);

And kernel/module.c:

find_module_sections():

#ifdef CONFIG_TRACEPOINTS
        mod->tracepoints_ptrs = section_objs(info, "__tracepoints_ptrs",
                                             sizeof(*mod->tracepoints_ptrs),
                                             &mod->num_tracepoints);

#endif

Quoted 7 lines

And kernel/tracepoint.c:tracepoint_module_notify() for the module coming/going notifier.

Basically you would want to have your own structure within your own section of the module which describes the srcu domain, and have a module coming/going notifier responsible for dynamically allocating the srcu domain on "coming", and doing a srcu barrier and cleanup the domain on "going".

Ah, sounds like an excellent approach! I will give it a shot, thank you!

Thanx, Paul

Quoted 61 lines

Thanks,

Mathieu

Quoted 50 lines

Thanx, Paul

Quoted 47 lines

Thanks,

Mathieu

Quoted 37 lines

Thanx, Paul

Quoted 34 lines

Thanks,

Mathieu

Quoted 16 lines

This series consist of the following:

1. Dynamically allocate dax_srcu.

2. Dynamically allocate drm_unplug_srcu.

3. Dynamically allocate kfd_processes_srcu.

These build and have been subjected to 0day testing, but might also need testing by someone having the requisite hardware.

Thanx, Paul

------------------------------------------------------------------------

Quoted 8 lines
drivers/dax/super.c                        |   10 +++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c |    5 +++
drivers/gpu/drm/amd/amdkfd/kfd_process.c   |    2 -
drivers/gpu/drm/drm_drv.c                  |    8 ++++
include/linux/srcutree.h                   |   19 +++++++++--
kernel/rcu/rcuperf.c                       |   40 +++++++++++++++++++-----
kernel/rcu/rcutorture.c                    |   48 +++++++++++++++++++++--------
 7 files changed, 105 insertions(+), 27 deletions(-)

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
Reviewed-by on v1 1/4
#2 lore raw

Re: [PATCH RFC tip/core/rcu 1/4] dax/super: Dynamically allocate dax_srcu

In reply to Paul E. McKenney
Quoted 10 lines · Paul E. McKenney

Having DEFINE_SRCU() or DEFINE_STATIC_SRCU() in a loadable module requires that the size of the reserved region be increased, which is not something we really want to be doing. This commit therefore removes the DEFINE_STATIC_SRCU() from drivers/dax/super.c in favor of defining dax_srcu as a simple srcu_struct, initializing it in dax_core_init(), and cleaning it up in dax_core_exit().

Looks good to me.

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#16 lore raw
In reply to Paul E. McKenney
Quoted 120 lines · Paul E. McKenney
Quoted 119 lines · Mathieu Desnoyers
Quoted 38 lines · ----- On Apr 3, 2019, at 9:32 AM, paulmck [email protected] wrote:
Quoted 37 lines · Mathieu Desnoyers
Quoted 20 lines · ----- On Apr 2, 2019, at 11:23 AM, paulmck [email protected] wrote:
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck [email protected] wrote:

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

I was more thinking that under #ifdef MODULE, both DEFINE_SRCU and DEFINE_STATIC_SRCU could append data in a dedicated section. module.c would additionally lookup that section on module load, and deal with those statically defined SRCU entries as if they were dynamically allocated ones. It would of course cleanup those resources on module unload.

Quoted 7 lines
Quoted 1 line

Am I missing some subtlety there ?

If I understand you correctly, that is actually what is already done. The size of this dedicated section is currently set by PERCPU_MODULE_RESERVE, and the additions of DEFINE{_STATIC}_SRCU() in modules was requiring that this to be increased frequently. That led to a request that something be done, in turn leading to this patch series.

I think we are not expressing quite the same idea.

AFAIU, yours is to have DEFINE*_SRCU directly define per-cpu data within modules, which ends up using percpu module reserved memory.

My idea is to make DEFINE*_SRCU have a different behavior under #ifdef MODULE. It could emit a _global variable_ (_not_ per-cpu) within a new section. That section would then be used by module init/exit code to figure out what "srcu descriptors" are present in the modules. It would therefore rely on dynamic allocation for those, therefore removing the need to involve the percpu module reserved pool at all.

Quoted 8 lines

I don't see a way around this short of changing module loading to do alloc_percpu() and then updating the relocation based on this result. Which would admittedly be far more convenient. I was assuming that this would be difficult due to varying CPU offsets or the like.

But if it can be done reasonably, it would be quite a bit nicer than forcing dynamic allocation in cases where it is not otherwise needed.

Hopefully my explanation above helps clear out what I have in mind.

You can find similar tricks performed by include/linux/tracepoint.h:

#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)

{
        return offset_to_ptr(p);
}
#define __TRACEPOINT_ENTRY(name)                                        \
        asm("   .section \"__tracepoints_ptrs\", \"a\"          \n"     \
            "   .balign 4                                       \n"     \
            "   .long   __tracepoint_" #name " - .              \n"     \
            "   .previous                                       \n")
#else
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
{
        return *p;
}
#define __TRACEPOINT_ENTRY(name)                                         \
        static tracepoint_ptr_t __tracepoint_ptr_##name __used           \
        __attribute__((section("__tracepoints_ptrs"))) =                 \
                &__tracepoint_##name

#endif

[...]

#define DEFINE_TRACE_FN(name, reg, unreg)                                \
        static const char __tpstrtab_##name[]                            \
        __attribute__((section("__tracepoints_strings"))) = #name;       \
        struct tracepoint __tracepoint_##name                            \
        __attribute__((section("__tracepoints"), used)) =                \
                { __tpstrtab_##name, STATIC_KEY_INIT_FALSE, reg, unreg, NULL };\
        __TRACEPOINT_ENTRY(name);

And kernel/module.c:

find_module_sections():

#ifdef CONFIG_TRACEPOINTS
        mod->tracepoints_ptrs = section_objs(info, "__tracepoints_ptrs",
                                             sizeof(*mod->tracepoints_ptrs),
                                             &mod->num_tracepoints);

#endif

And kernel/tracepoint.c:tracepoint_module_notify() for the module coming/going notifier.

Quoted 6 lines
Quoted 4 lines

Basically you would want to have your own structure within your own section of the module which describes the srcu domain, and have a module coming/going notifier responsible for dynamically allocating the srcu domain on "coming", and doing a srcu barrier and cleanup the domain on "going".

Ah, sounds like an excellent approach! I will give it a shot, thank you!

I agree with the idea as well. It is nice that tracepoints work with modules so well, a feature that many folks use for debugging ;-)

thanks!

- Joel

author Applied
#3 lore raw

Re: [PATCH RFC tip/core/rcu 1/4] dax/super: Dynamically allocate dax_srcu

In reply to Dan Williams
Quoted 9 lines · Dan Williams
Quoted 8 lines · Paul E. McKenney

Having DEFINE_SRCU() or DEFINE_STATIC_SRCU() in a loadable module requires that the size of the reserved region be increased, which is not something we really want to be doing. This commit therefore removes the DEFINE_STATIC_SRCU() from drivers/dax/super.c in favor of defining dax_srcu as a simple srcu_struct, initializing it in dax_core_init(), and cleaning it up in dax_core_exit().

Quoted 6 lines
Quoted 2 lines

Looks good to me.

Applied, thank you!

Thanx, Paul

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#8 lore raw

Re: [PATCH RFC tip/core/rcu 3/4] drivers/gpu/drm/amd: Dynamically allocate kfd_processes_srcu

In reply to Kuehling, Felix
Quoted 68 lines · Felix
Quoted 38 lines · Paul E. McKenney

Having DEFINE_SRCU() or DEFINE_STATIC_SRCU() in a loadable module requires that the size of the reserved region be increased, which is not something we really want to be doing. This commit therefore removes the DEFINE_STATIC_SRCU() from drivers/gpu/drm/amd/amdkfd/kfd_process.c in favor of defining kfd_processes_srcu as a simple srcu_struct, initializing it in amdgpu_amdkfd_init(), and cleaning it up in amdgpu_amdkfd_fini().

Cc Oded Gabbay <[email protected]>
Cc Alex Deucher <[email protected]>
Cc "Christian König" <[email protected]
Cc "David (ChunMing) Zhou" <[email protected]>
Cc David Airlie <[email protected]>
Cc Daniel Vetter <[email protected]>
Cc Tejun Heo <[email protected]>
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 5 +++++
  drivers/gpu/drm/amd/amdkfd/kfd_process.c   | 2 +-
  2 files changed, 6 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +2 -0
index fe1d7368c1e6..eadb20dee867 100644
@@ -28,6 +28,8 @@
  #include <linux/module.h>
  #include <linux/dma-buf.h>
 
+extern struct srcu_struct kfd_processes_srcu;
+
  static const unsigned int compute_vmid_bitmap = 0xFF00;
 
  /* Total memory size in system memory and all GPU VRAM. Used to

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c int amdgpu_amdkfd_init(void) · line 46

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +2 -0
@@ -40,6 +42,8 @@ int amdgpu_amdkfd_init(void)
  	struct sysinfo si;
  	int ret;
 
+	ret = init_srcu_struct(&kfd_processes_srcu);
+	WARN_ON(ret);

kfd_processes_srcu only exists if kfd_process.c is compiled in. That depends on CONFIG_HSA_AMD. So this should at least move into #ifdef a few lines below.

However, it would be cleaner to move this initialization into kfd_init in kfd_module.c, or better yet, into kfd_process_create_wq in kfd_process.c. Then kfd_process_create_wq should be renamed to something more generic, such as kfd_process_init.

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c int amdgpu_amdkfd_init(void) · line 64

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +1 -0
  	si_meminfo(&si);
  	amdgpu_amdkfd_total_mem_size = si.totalram - si.totalhigh;
  	amdgpu_amdkfd_total_mem_size *= si.mem_unit;
@@ -57,6 +61,7 @@ int amdgpu_amdkfd_init(void)
  void amdgpu_amdkfd_fini(void)
  {
  	kgd2kfd_exit();
+	cleanup_srcu_struct(&kfd_processes_srcu);

Similarly, this would be cleaner in kfd_exit in kfd_module.c or kfd_process_destroy_wq in kfd_process.c, with that function similarly renamed to kfd_process_fini.

Quoted 1 line

I'm attaching a revised patch. It's only compile tested.

Thank you, Felix! I have reverted my original and applied your revised version.

Thanx, Paul

Quoted 162 lines

Regards,
  Felix

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +0 -0
  }
 
  void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdkfd/kfd_process.c +0 -0
index 4bdae78bab8e..98b694068b8a 100644

drivers/gpu/drm/amd/amdkfd/kfd_process.c struct mm_struct; · line 53

Quoted 8 lines
drivers/gpu/drm/amd/amdkfd/kfd_process.c +1 -1
@@ -47,7 +47,7 @@ struct mm_struct;
  DEFINE_HASHTABLE(kfd_processes_table, KFD_PROCESS_TABLE_SIZE);
  static DEFINE_MUTEX(kfd_processes_mutex);
 
-DEFINE_SRCU(kfd_processes_srcu);
+struct srcu_struct kfd_processes_srcu;
 
  /* For process termination handling */
  static struct workqueue_struct *kfd_process_wq;

From 5857a9aa63957a5755ff81ae5c46533bca408c12 Mon Sep 17 00:00:00 2001
From: "Paul E. McKenney" <[email protected]>
Date: Tue, 2 Apr 2019 07:29:32 -0700
Subject: [PATCH 1/1] drivers/gpu/drm/amd: Dynamically allocate
kfd_processes_srcu v2
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Having DEFINE_SRCU() or DEFINE_STATIC_SRCU() in a loadable module requires that the size of the reserved region be increased, which is not something we really want to be doing. This commit therefore removes the DEFINE_STATIC_SRCU() from drivers/gpu/drm/amd/amdkfd/kfd_process.c in favor of defining kfd_processes_srcu as a simple srcu_struct, initializing it in kfd_process_init(), and cleaning it up in kfd_process_fini().

v2 (Felix Kuehling): Move srcu init and cleanup into kfd_process.c

Tested-by (v1): kbuild test robot <[email protected]>
Cc: Oded Gabbay <[email protected]>
Cc: Alex Deucher <[email protected]>
Cc: "Christian K??nig" <[email protected]>
Cc: "David (ChunMing) Zhou" <[email protected]>
Cc: David Airlie <[email protected]>
Cc: Daniel Vetter <[email protected]>
Cc: Tejun Heo <[email protected]>
Cc: <[email protected]>
Cc: <[email protected]>

---
 drivers/gpu/drm/amd/amdkfd/kfd_module.c  |  8 ++++----
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h    |  4 ++--
 drivers/gpu/drm/amd/amdkfd/kfd_process.c | 19 ++++++++++++-------
 3 files changed, 18 insertions(+), 13 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_module.c +4 -4
index 932007e..e8e2c15 100644
@@ -52,15 +52,15 @@ static int kfd_init(void)
 	if (err < 0)
 		goto err_topology;
 
-	err = kfd_process_create_wq();
+	err = kfd_process_init();
 	if (err < 0)
-		goto err_create_wq;
+		goto err_process;
 
 	kfd_debugfs_init();
 
 	return 0;
 
-err_create_wq:
+err_process:
 	kfd_topology_shutdown();
 err_topology:
 	kfd_chardev_exit();
@@ -71,7 +71,7 @@ static int kfd_init(void)
 static void kfd_exit(void)
 {
 	kfd_debugfs_fini();
-	kfd_process_destroy_wq();
+	kfd_process_fini();
 	kfd_topology_shutdown();
 	kfd_chardev_exit();
 }
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +2 -2
index 9e02309..9dd187c 100644
@@ -708,8 +708,8 @@ struct amdkfd_ioctl_desc {
 };
 bool kfd_dev_is_large_bar(struct kfd_dev *dev);
 
-int kfd_process_create_wq(void);
-void kfd_process_destroy_wq(void);
+int kfd_process_init(void);
+void kfd_process_fini(void);
 struct kfd_process *kfd_create_process(struct file *filep);
 struct kfd_process *kfd_get_process(const struct task_struct *);
 struct kfd_process *kfd_lookup_process_by_pasid(unsigned int pasid);
drivers/gpu/drm/amd/amdkfd/kfd_process.c +12 -7
index 4bdae78..f810911 100644
@@ -47,7 +47,7 @@ struct mm_struct;
 DEFINE_HASHTABLE(kfd_processes_table, KFD_PROCESS_TABLE_SIZE);
 static DEFINE_MUTEX(kfd_processes_mutex);
 
-DEFINE_SRCU(kfd_processes_srcu);
+struct srcu_struct kfd_processes_srcu;
 
 /* For process termination handling */
 static struct workqueue_struct *kfd_process_wq;
@@ -69,22 +69,25 @@ static void evict_process_worker(struct work_struct *work);
 static void restore_process_worker(struct work_struct *work);
 
 
-int kfd_process_create_wq(void)
+int kfd_process_init(void)
 {
+	int ret;
+
 	if (!kfd_process_wq)
 		kfd_process_wq = alloc_workqueue("kfd_process_wq", 0, 0);
 	if (!kfd_restore_wq)
 		kfd_restore_wq = alloc_ordered_workqueue("kfd_restore_wq", 0);
 
-	if (!kfd_process_wq || !kfd_restore_wq) {
-		kfd_process_destroy_wq();
+	if (!kfd_process_wq || !kfd_restore_wq)
 		return -ENOMEM;
-	}
 
-	return 0;
+	ret = init_srcu_struct(&kfd_processes_srcu);
+	WARN_ON(ret);
+
+	return ret;
 }
 
-void kfd_process_destroy_wq(void)
+void kfd_process_fini(void)
 {
 	if (kfd_process_wq) {
 		destroy_workqueue(kfd_process_wq);
@@ -94,6 +97,8 @@ void kfd_process_destroy_wq(void)
 		destroy_workqueue(kfd_restore_wq);
 		kfd_restore_wq = NULL;
 	}
+
+	cleanup_srcu_struct(&kfd_processes_srcu);
 }
 
 static void kfd_process_free_gpuvm(struct kgd_mem *mem,
Signature · 1 line
2.7.4
#17 lore raw
In reply to Paul E. McKenney
Quoted 120 lines · Paul E. McKenney
Quoted 119 lines · Mathieu Desnoyers
Quoted 38 lines · ----- On Apr 3, 2019, at 9:32 AM, paulmck…
Quoted 37 lines · Mathieu Desnoyers
Quoted 20 lines · ----- On Apr 2, 2019, at 11:23 AM, paulmck…
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck…

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

I was more thinking that under #ifdef MODULE, both DEFINE_SRCU and DEFINE_STATIC_SRCU could append data in a dedicated section. module.c would additionally lookup that section on module load, and deal with those statically defined SRCU entries as if they were dynamically allocated ones. It would of course cleanup those resources on module unload.

Quoted 7 lines
Quoted 1 line

Am I missing some subtlety there ?

If I understand you correctly, that is actually what is already done. The size of this dedicated section is currently set by PERCPU_MODULE_RESERVE, and the additions of DEFINE{_STATIC}_SRCU() in modules was requiring that this to be increased frequently. That led to a request that something be done, in turn leading to this patch series.

I think we are not expressing quite the same idea.

AFAIU, yours is to have DEFINE*_SRCU directly define per-cpu data within modules, which ends up using percpu module reserved memory.

My idea is to make DEFINE*_SRCU have a different behavior under #ifdef MODULE. It could emit a _global variable_ (_not_ per-cpu) within a new section. That section would then be used by module init/exit code to figure out what "srcu descriptors" are present in the modules. It would therefore rely on dynamic allocation for those, therefore removing the need to involve the percpu module reserved pool at all.

Quoted 8 lines

I don't see a way around this short of changing module loading to do alloc_percpu() and then updating the relocation based on this result. Which would admittedly be far more convenient. I was assuming that this would be difficult due to varying CPU offsets or the like.

But if it can be done reasonably, it would be quite a bit nicer than forcing dynamic allocation in cases where it is not otherwise needed.

Hopefully my explanation above helps clear out what I have in mind.

You can find similar tricks performed by include/linux/tracepoint.h:

#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)

{
        return offset_to_ptr(p);
}
#define __TRACEPOINT_ENTRY(name)                                        \
        asm("   .section \"__tracepoints_ptrs\", \"a\"          \n"     \
            "   .balign 4                                       \n"     \
            "   .long   __tracepoint_" #name " - .              \n"     \
            "   .previous                                       \n")
#else
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
{
        return *p;
}
#define __TRACEPOINT_ENTRY(name)                                         \
        static tracepoint_ptr_t __tracepoint_ptr_##name __used           \
        __attribute__((section("__tracepoints_ptrs"))) =                 \
                &__tracepoint_##name

#endif

[...]

#define DEFINE_TRACE_FN(name, reg, unreg)                                \
        static const char __tpstrtab_##name[]                            \
        __attribute__((section("__tracepoints_strings"))) = #name;       \
        struct tracepoint __tracepoint_##name                            \
        __attribute__((section("__tracepoints"), used)) =                \
                { __tpstrtab_##name, STATIC_KEY_INIT_FALSE, reg, unreg, NULL };\
        __TRACEPOINT_ENTRY(name);

And kernel/module.c:

find_module_sections():

#ifdef CONFIG_TRACEPOINTS
        mod->tracepoints_ptrs = section_objs(info, "__tracepoints_ptrs",
                                             sizeof(*mod->tracepoints_ptrs),
                                             &mod->num_tracepoints);

#endif

And kernel/tracepoint.c:tracepoint_module_notify() for the module coming/going notifier.

Quoted 6 lines
Quoted 4 lines

Basically you would want to have your own structure within your own section of the module which describes the srcu domain, and have a module coming/going notifier responsible for dynamically allocating the srcu domain on "coming", and doing a srcu barrier and cleanup the domain on "going".

Ah, sounds like an excellent approach! I will give it a shot, thank you!

Please see below for an untested shot.

The original commits posted in this series are still available within the -srcu tree at branch srcunomod.2019.04.05a. Yes, I am a digital packrat. Why do you ask?

Thoughts? Or more accurately, given that this is the first time I have used linker sections, what did I mess up?

Thanx, Paul

------------------------------------------------------------------------

commit e24a0dab1414c563bb96bcb28d5963c9df18b1e8

Author: Paul E. McKenney <[email protected]>
Date:   Fri Apr 5 16:15:00 2019 -0700

srcu: Allocate per-CPU data for DEFINE_SRCU() in modules

Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires that the size of the reserved region be increased, which is not something we want to be doing all that often. One approach would be to require that loadable modules define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. However, this is more than a bit user unfriendly.

This commit therefore creates an ___srcu_struct_ptrs linker section, and pointers to srcu_struct structures created by DEFINE_SRCU() and DEFINE_STATIC_SRCU() within a module are placed into that module's ___srcu_struct_ptrs section. The required init_srcu_struct() and cleanup_srcu_struct() functions are then automatically invoked as needed when that module is loaded and unloaded, thus allowing modules to continue to use DEFINE_SRCU() and DEFINE_STATIC_SRCU() while avoiding the need to increase the size of the reserved region.

Many of the algorithms and some of the code was cheerfully cherry-picked from other code making use of linker sections, perhaps most notably from tracepoints. All bugs are nevertheless the sole property of the author.

include/asm-generic/vmlinux.lds.h +4 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
+		__end___srcu_struct = .;				\
 	}								\
 									\
 	.rodata1          : AT(ADDR(.rodata1) - LOAD_OFFSET) {		\
include/linux/module.h +5 -0
index 5bf5dcd91009..921443a026dd 100644
@@ -21,6 +21,7 @@
 #include <linux/rbtree_latch.h>
 #include <linux/error-injection.h>
 #include <linux/tracepoint-defs.h>
+#include <linux/srcu.h>
 
 #include <linux/percpu.h>
 #include <asm/module.h>
@@ -448,6 +449,10 @@ struct module {
 	unsigned int num_tracepoints;
 	tracepoint_ptr_t *tracepoints_ptrs;
 #endif
+#ifdef CONFIG_TREE_SRCU
+	unsigned int num_srcu_structs;
+	struct srcu_struct **srcu_struct_ptrs;
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	unsigned int num_bpf_raw_events;
 	struct bpf_raw_event_map *bpf_raw_events;
include/linux/srcutree.h +11 -3
index 7f7c8c050f63..93685a9f3b4c 100644
@@ -120,9 +120,17 @@ struct srcu_struct {
  *
  * See include/linux/percpu-defs.h for the rules on per-CPU variables.
  */
-#define __DEFINE_SRCU(name, is_static)					\
-	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);\
-	is_static struct srcu_struct name = __SRCU_STRUCT_INIT(name, name##_srcu_data)
+#ifdef MODULE
+# define __DEFINE_SRCU(name, is_static)					\
+	is_static struct srcu_struct name;				\
+	struct srcu_struct *__srcu_struct_##name			\
+		__attribute__((section("___srcu_struct_ptrs"))) = &name
+#else
+# define __DEFINE_SRCU(name, is_static)					\
+	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);	\
+	is_static struct srcu_struct name =				\
+		__SRCU_STRUCT_INIT(name, name##_srcu_data)
+#endif
 #define DEFINE_SRCU(name)		__DEFINE_SRCU(name, /* not static */)
 #define DEFINE_STATIC_SRCU(name)	__DEFINE_SRCU(name, static)
 
kernel/module.c +5 -0
index 0b9aa8ab89f0..524da609c884 100644
@@ -3093,6 +3093,11 @@ static int find_module_sections(struct module *mod, struct load_info *info)
 					     sizeof(*mod->tracepoints_ptrs),
 					     &mod->num_tracepoints);
 #endif
+#ifdef CONFIG_TREE_SRCU
+	mod->srcu_struct_ptrs = section_objs(info, "___srcu_struct_ptrs",
+					     sizeof(*mod->srcu_struct_ptrs),
+					     &mod->num_srcu_structs);
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	mod->bpf_raw_events = section_objs(info, "__bpf_raw_tp_map",
 					   sizeof(*mod->bpf_raw_events),
kernel/rcu/srcutree.c +67 -0

70 lines not on this page · raw message

#18 lore raw
Quoted 214 lines · Paul E. McKenney
Quoted 120 lines · Paul E. McKenney
Quoted 119 lines · Mathieu Desnoyers
Quoted 38 lines · ----- On Apr 3, 2019, at 9:32 AM, paulmck [email protected] wrote:
Quoted 37 lines · Mathieu Desnoyers
Quoted 20 lines · ----- On Apr 2, 2019, at 11:23 AM, paulmck [email protected] wrote:
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck [email protected] wrote:

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

I was more thinking that under #ifdef MODULE, both DEFINE_SRCU and DEFINE_STATIC_SRCU could append data in a dedicated section. module.c would additionally lookup that section on module load, and deal with those statically defined SRCU entries as if they were dynamically allocated ones. It would of course cleanup those resources on module unload.

Quoted 7 lines
Quoted 1 line

Am I missing some subtlety there ?

If I understand you correctly, that is actually what is already done. The size of this dedicated section is currently set by PERCPU_MODULE_RESERVE, and the additions of DEFINE{_STATIC}_SRCU() in modules was requiring that this to be increased frequently. That led to a request that something be done, in turn leading to this patch series.

I think we are not expressing quite the same idea.

AFAIU, yours is to have DEFINE*_SRCU directly define per-cpu data within modules, which ends up using percpu module reserved memory.

My idea is to make DEFINE*_SRCU have a different behavior under #ifdef MODULE. It could emit a _global variable_ (_not_ per-cpu) within a new section. That section would then be used by module init/exit code to figure out what "srcu descriptors" are present in the modules. It would therefore rely on dynamic allocation for those, therefore removing the need to involve the percpu module reserved pool at all.

Quoted 8 lines

I don't see a way around this short of changing module loading to do alloc_percpu() and then updating the relocation based on this result. Which would admittedly be far more convenient. I was assuming that this would be difficult due to varying CPU offsets or the like.

But if it can be done reasonably, it would be quite a bit nicer than forcing dynamic allocation in cases where it is not otherwise needed.

Hopefully my explanation above helps clear out what I have in mind.

You can find similar tricks performed by include/linux/tracepoint.h:

#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)

{
        return offset_to_ptr(p);
}
#define __TRACEPOINT_ENTRY(name)                                        \
        asm("   .section \"__tracepoints_ptrs\", \"a\"          \n"     \
            "   .balign 4                                       \n"     \
            "   .long   __tracepoint_" #name " - .              \n"     \
            "   .previous                                       \n")
#else
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
{
        return *p;
}
#define __TRACEPOINT_ENTRY(name)                                         \
        static tracepoint_ptr_t __tracepoint_ptr_##name __used           \
        __attribute__((section("__tracepoints_ptrs"))) =                 \
                &__tracepoint_##name

#endif

[...]

#define DEFINE_TRACE_FN(name, reg, unreg)                                \
        static const char __tpstrtab_##name[]                            \
        __attribute__((section("__tracepoints_strings"))) = #name;       \
        struct tracepoint __tracepoint_##name                            \
        __attribute__((section("__tracepoints"), used)) =                \
                { __tpstrtab_##name, STATIC_KEY_INIT_FALSE, reg, unreg, NULL };\
        __TRACEPOINT_ENTRY(name);

And kernel/module.c:

find_module_sections():

#ifdef CONFIG_TRACEPOINTS
        mod->tracepoints_ptrs = section_objs(info, "__tracepoints_ptrs",
                                             sizeof(*mod->tracepoints_ptrs),
                                             &mod->num_tracepoints);

#endif

And kernel/tracepoint.c:tracepoint_module_notify() for the module coming/going notifier.

Quoted 6 lines
Quoted 4 lines

Basically you would want to have your own structure within your own section of the module which describes the srcu domain, and have a module coming/going notifier responsible for dynamically allocating the srcu domain on "coming", and doing a srcu barrier and cleanup the domain on "going".

Ah, sounds like an excellent approach! I will give it a shot, thank you!

Please see below for an untested shot.

The original commits posted in this series are still available within the -srcu tree at branch srcunomod.2019.04.05a. Yes, I am a digital packrat. Why do you ask?

Thoughts? Or more accurately, given that this is the first time I have used linker sections, what did I mess up?

Thanx, Paul

------------------------------------------------------------------------

commit e24a0dab1414c563bb96bcb28d5963c9df18b1e8

Author: Paul E. McKenney <[email protected]>
Date:   Fri Apr 5 16:15:00 2019 -0700

srcu: Allocate per-CPU data for DEFINE_SRCU() in modules

Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires that the size of the reserved region be increased, which is not something we want to be doing all that often. One approach would be to require that loadable modules define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. However, this is more than a bit user unfriendly.

This commit therefore creates an ___srcu_struct_ptrs linker section, and pointers to srcu_struct structures created by DEFINE_SRCU() and DEFINE_STATIC_SRCU() within a module are placed into that module's ___srcu_struct_ptrs section. The required init_srcu_struct() and cleanup_srcu_struct() functions are then automatically invoked as needed when that module is loaded and unloaded, thus allowing modules to continue to use DEFINE_SRCU() and DEFINE_STATIC_SRCU() while avoiding the need to increase the size of the reserved region.

Many of the algorithms and some of the code was cheerfully cherry-picked from other code making use of linker sections, perhaps most notably from tracepoints. All bugs are nevertheless the sole property of the author.

include/asm-generic/vmlinux.lds.h +4 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
+		__end___srcu_struct = .;				\
 	}								\
 									\
 	.rodata1          : AT(ADDR(.rodata1) - LOAD_OFFSET) {		\
include/linux/module.h +5 -0
index 5bf5dcd91009..921443a026dd 100644
@@ -21,6 +21,7 @@
 #include <linux/rbtree_latch.h>
 #include <linux/error-injection.h>
 #include <linux/tracepoint-defs.h>
+#include <linux/srcu.h>
 
 #include <linux/percpu.h>
 #include <asm/module.h>
@@ -448,6 +449,10 @@ struct module {
 	unsigned int num_tracepoints;
 	tracepoint_ptr_t *tracepoints_ptrs;
 #endif
+#ifdef CONFIG_TREE_SRCU
+	unsigned int num_srcu_structs;
+	struct srcu_struct **srcu_struct_ptrs;
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	unsigned int num_bpf_raw_events;
 	struct bpf_raw_event_map *bpf_raw_events;
include/linux/srcutree.h +0 -0
index 7f7c8c050f63..93685a9f3b4c 100644
@@ -120,9 +120,17 @@ struct srcu_struct {
  *

include/linux/srcutree.h struct srcu_struct { · line 127

Quoted 10 lines
include/linux/srcutree.h +5 -3
  * See include/linux/percpu-defs.h for the rules on per-CPU variables.
  */
-#define __DEFINE_SRCU(name, is_static)					\
-	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);\
-	is_static struct srcu_struct name = __SRCU_STRUCT_INIT(name, name##_srcu_data)
+#ifdef MODULE
+# define __DEFINE_SRCU(name, is_static)					\
+	is_static struct srcu_struct name;				\
+	struct srcu_struct *__srcu_struct_##name			\
+		__attribute__((section("___srcu_struct_ptrs"))) = &name

This can be replaced by:
__section("__srcu_struct_ptrs") = &name;

Quoted 69 lines
include/linux/srcutree.h +6 -0
+#else
+# define __DEFINE_SRCU(name, is_static)					\
+	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);	\
+	is_static struct srcu_struct name =				\
+		__SRCU_STRUCT_INIT(name, name##_srcu_data)
+#endif
 #define DEFINE_SRCU(name)		__DEFINE_SRCU(name, /* not static */)
 #define DEFINE_STATIC_SRCU(name)	__DEFINE_SRCU(name, static)
 
kernel/module.c +5 -0
index 0b9aa8ab89f0..524da609c884 100644
@@ -3093,6 +3093,11 @@ static int find_module_sections(struct module *mod, struct load_info *info)
 					     sizeof(*mod->tracepoints_ptrs),
 					     &mod->num_tracepoints);
 #endif
+#ifdef CONFIG_TREE_SRCU
+	mod->srcu_struct_ptrs = section_objs(info, "___srcu_struct_ptrs",
+					     sizeof(*mod->srcu_struct_ptrs),
+					     &mod->num_srcu_structs);
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	mod->bpf_raw_events = section_objs(info, "__bpf_raw_tp_map",
 					   sizeof(*mod->bpf_raw_events),
kernel/rcu/srcutree.c +36 -0
index 9b761e546de8..ac0f6f0a8916 100644
@@ -1310,3 +1310,70 @@ void __init srcu_init(void)
 		queue_work(rcu_gp_wq, &ssp->work.work);
 	}
 }
+
+#ifdef CONFIG_MODULES
+
+/* Initialize any global-scope srcu_struct structures used by this module. */
+static int srcu_module_coming(struct module *mod)
+{
+	int i;
+	struct srcu_struct **sspp = mod->srcu_struct_ptrs;
+	int ret;
+
+	for (i = 0; i < mod->num_srcu_structs; i++) {
+		ret = init_srcu_struct(*(sspp++));
+		if (WARN_ON_ONCE(ret))
+			return ret;
+	}
+	return 0;
+}
+
+/* Clean up any global-scope srcu_struct structures used by this module. */
+static void srcu_module_going(struct module *mod)
+{
+	int i;
+	struct srcu_struct **sspp = mod->srcu_struct_ptrs;
+
+	for (i = 0; i < mod->num_srcu_structs; i++)
+		cleanup_srcu_struct(*(sspp++));
+}
+
+/* Handle one module, either coming or going. */
+static int srcu_module_notify(struct notifier_block *self,
+			      unsigned long val, void *data)
+{
+	struct module *mod = data;
+	int ret = 0;
+
+	switch (val) {

kernel/rcu/srcutree.c void __init srcu_init(void) · line 1358

Quoted 10 lines
kernel/rcu/srcutree.c +10 -0
+	case MODULE_STATE_COMING:
+		ret = srcu_module_coming(mod);
+		break;
+	case MODULE_STATE_LIVE:
+		break;
+	case MODULE_STATE_GOING:
+		srcu_module_going(mod);
+		break;
+	case MODULE_STATE_UNFORMED:
+		break;

The unused cases can be put under default or even clubbed thus saving a line but if you prefer to keep them.like this that is fine.

It looks good to me but I will test it out more later today. Thanks!

- Joel

Quoted 21 lines
kernel/rcu/srcutree.c +20 -0
+	return ret;
+}
+
+static struct notifier_block srcu_module_nb = {
+	.notifier_call = srcu_module_notify,
+	.priority = 0,
+};
+
+static __init int init_srcu_module_notifier(void)
+{
+	int ret;
+
+	ret = register_module_notifier(&srcu_module_nb);
+	if (ret)
+		pr_warn("Failed to register srcu module notifier\n");
+	return ret;
+}
+late_initcall(init_srcu_module_notifier);
+
+#endif /* #ifdef CONFIG_MODULES */
#20 lore raw
In reply to Paul E. McKenney
Quoted 175 lines · Paul E. McKenney
Quoted 120 lines · Paul E. McKenney
Quoted 119 lines · Mathieu Desnoyers
Quoted 38 lines · ----- On Apr 3, 2019, at 9:32 AM, paulmck…
Quoted 37 lines · Mathieu Desnoyers
Quoted 20 lines · ----- On Apr 2, 2019, at 11:23 AM, paulmck…
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck…

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

I was more thinking that under #ifdef MODULE, both DEFINE_SRCU and DEFINE_STATIC_SRCU could append data in a dedicated section. module.c would additionally lookup that section on module load, and deal with those statically defined SRCU entries as if they were dynamically allocated ones. It would of course cleanup those resources on module unload.

Quoted 7 lines
Quoted 1 line

Am I missing some subtlety there ?

If I understand you correctly, that is actually what is already done. The size of this dedicated section is currently set by PERCPU_MODULE_RESERVE, and the additions of DEFINE{_STATIC}_SRCU() in modules was requiring that this to be increased frequently. That led to a request that something be done, in turn leading to this patch series.

I think we are not expressing quite the same idea.

AFAIU, yours is to have DEFINE*_SRCU directly define per-cpu data within modules, which ends up using percpu module reserved memory.

My idea is to make DEFINE*_SRCU have a different behavior under #ifdef MODULE. It could emit a _global variable_ (_not_ per-cpu) within a new section. That section would then be used by module init/exit code to figure out what "srcu descriptors" are present in the modules. It would therefore rely on dynamic allocation for those, therefore removing the need to involve the percpu module reserved pool at all.

Quoted 8 lines

I don't see a way around this short of changing module loading to do alloc_percpu() and then updating the relocation based on this result. Which would admittedly be far more convenient. I was assuming that this would be difficult due to varying CPU offsets or the like.

But if it can be done reasonably, it would be quite a bit nicer than forcing dynamic allocation in cases where it is not otherwise needed.

Hopefully my explanation above helps clear out what I have in mind.

You can find similar tricks performed by include/linux/tracepoint.h:

#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)

{
        return offset_to_ptr(p);
}
#define __TRACEPOINT_ENTRY(name)                                        \
        asm("   .section \"__tracepoints_ptrs\", \"a\"          \n"     \
            "   .balign 4                                       \n"     \
            "   .long   __tracepoint_" #name " - .              \n"     \
            "   .previous                                       \n")
#else
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
{
        return *p;
}
#define __TRACEPOINT_ENTRY(name)                                         \
        static tracepoint_ptr_t __tracepoint_ptr_##name __used           \
        __attribute__((section("__tracepoints_ptrs"))) =                 \
                &__tracepoint_##name

#endif

[...]

#define DEFINE_TRACE_FN(name, reg, unreg)                                \
        static const char __tpstrtab_##name[]                            \
        __attribute__((section("__tracepoints_strings"))) = #name;       \
        struct tracepoint __tracepoint_##name                            \
        __attribute__((section("__tracepoints"), used)) =                \
                { __tpstrtab_##name, STATIC_KEY_INIT_FALSE, reg, unreg, NULL };\
        __TRACEPOINT_ENTRY(name);

And kernel/module.c:

find_module_sections():

#ifdef CONFIG_TRACEPOINTS
        mod->tracepoints_ptrs = section_objs(info, "__tracepoints_ptrs",
                                             sizeof(*mod->tracepoints_ptrs),
                                             &mod->num_tracepoints);

#endif

And kernel/tracepoint.c:tracepoint_module_notify() for the module coming/going notifier.

Quoted 6 lines
Quoted 4 lines

Basically you would want to have your own structure within your own section of the module which describes the srcu domain, and have a module coming/going notifier responsible for dynamically allocating the srcu domain on "coming", and doing a srcu barrier and cleanup the domain on "going".

Ah, sounds like an excellent approach! I will give it a shot, thank you!

Please see below for an untested shot.

The original commits posted in this series are still available within the -srcu tree at branch srcunomod.2019.04.05a. Yes, I am a digital packrat. Why do you ask?

Thoughts? Or more accurately, given that this is the first time I have used linker sections, what did I mess up?

Thanx, Paul

------------------------------------------------------------------------

commit e24a0dab1414c563bb96bcb28d5963c9df18b1e8

Author: Paul E. McKenney <[email protected]>
Date:   Fri Apr 5 16:15:00 2019 -0700

srcu: Allocate per-CPU data for DEFINE_SRCU() in modules

Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires that the size of the reserved region be increased, which is not something we want to be doing all that often. One approach would be to require that loadable modules define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. However, this is more than a bit user unfriendly.

This commit therefore creates an ___srcu_struct_ptrs linker section, and pointers to srcu_struct structures created by DEFINE_SRCU() and DEFINE_STATIC_SRCU() within a module are placed into that module's ___srcu_struct_ptrs section. The required init_srcu_struct() and cleanup_srcu_struct() functions are then automatically invoked as needed when that module is loaded and unloaded, thus allowing modules to continue to use DEFINE_SRCU() and DEFINE_STATIC_SRCU() while avoiding the need to increase the size of the reserved region.

Many of the algorithms and some of the code was cheerfully cherry-picked from other code making use of linker sections, perhaps most notably from tracepoints. All bugs are nevertheless the sole property of the author.

include/asm-generic/vmlinux.lds.h +0 -0
index f8f6f04c4453..c2d919a1566e 100644

include/asm-generic/vmlinux.lds.h line 345

Quoted 8 lines
include/asm-generic/vmlinux.lds.h +4 -0
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Thanks!

---8<-----------------------

Quoted 1 line

From 369ad090f706ce8e1facdd18eb10828b5f7e2b72 Mon Sep 17 00:00:00 2001

From: "Joel Fernandes (Google)" <joel-QYYGw3jwrUn5owFQY34kdNi2O/[email protected]>
Date: Sat, 6 Apr 2019 18:57:17 -0400
Subject: [PATCH] srcu: Remove unused vmlinux srcu linker entries

The SRCU for modules optimization introduced vmlinux linker entries which is unused since it applies only to the built-in vmlinux. So remove it to prevent any space usage due to the 8 byte alignment.

Tested with SRCU torture_type and rcutorture.

---
 include/asm-generic/vmlinux.lds.h | 4 ----
 1 file changed, 4 deletions(-)
include/asm-generic/vmlinux.lds.h +0 -4
index c2d919a1566e..f8f6f04c4453 100644
@@ -338,10 +338,6 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
-		. = ALIGN(8);						\
-		__start___srcu_struct = .;				\
-		*(___srcu_struct_ptrs)					\
-		__end___srcu_struct = .;				\
 	}								\
 									\
 	.rodata1          : AT(ADDR(.rodata1) - LOAD_OFFSET) {		\
Signature · 1 line
2.21.0.392.gf8f6787159e-goog
#21 lore raw
Quoted 182 lines · Joel Fernandes
Quoted 181 lines · Paul E. McKenney
Quoted 120 lines · Paul E. McKenney
Quoted 119 lines · Mathieu Desnoyers
Quoted 38 lines · ----- On Apr 3, 2019, at 9:32 AM, paulmck [email protected] wrote:
Quoted 37 lines · Mathieu Desnoyers
Quoted 20 lines · ----- On Apr 2, 2019, at 11:23 AM, paulmck [email protected] wrote:
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck [email protected] wrote:

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

I was more thinking that under #ifdef MODULE, both DEFINE_SRCU and DEFINE_STATIC_SRCU could append data in a dedicated section. module.c would additionally lookup that section on module load, and deal with those statically defined SRCU entries as if they were dynamically allocated ones. It would of course cleanup those resources on module unload.

Quoted 7 lines
Quoted 1 line

Am I missing some subtlety there ?

If I understand you correctly, that is actually what is already done. The size of this dedicated section is currently set by PERCPU_MODULE_RESERVE, and the additions of DEFINE{_STATIC}_SRCU() in modules was requiring that this to be increased frequently. That led to a request that something be done, in turn leading to this patch series.

I think we are not expressing quite the same idea.

AFAIU, yours is to have DEFINE*_SRCU directly define per-cpu data within modules, which ends up using percpu module reserved memory.

My idea is to make DEFINE*_SRCU have a different behavior under #ifdef MODULE. It could emit a _global variable_ (_not_ per-cpu) within a new section. That section would then be used by module init/exit code to figure out what "srcu descriptors" are present in the modules. It would therefore rely on dynamic allocation for those, therefore removing the need to involve the percpu module reserved pool at all.

Quoted 8 lines

I don't see a way around this short of changing module loading to do alloc_percpu() and then updating the relocation based on this result. Which would admittedly be far more convenient. I was assuming that this would be difficult due to varying CPU offsets or the like.

But if it can be done reasonably, it would be quite a bit nicer than forcing dynamic allocation in cases where it is not otherwise needed.

Hopefully my explanation above helps clear out what I have in mind.

You can find similar tricks performed by include/linux/tracepoint.h:

#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)

{
        return offset_to_ptr(p);
}
#define __TRACEPOINT_ENTRY(name)                                        \
        asm("   .section \"__tracepoints_ptrs\", \"a\"          \n"     \
            "   .balign 4                                       \n"     \
            "   .long   __tracepoint_" #name " - .              \n"     \
            "   .previous                                       \n")
#else
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
{
        return *p;
}
#define __TRACEPOINT_ENTRY(name)                                         \
        static tracepoint_ptr_t __tracepoint_ptr_##name __used           \
        __attribute__((section("__tracepoints_ptrs"))) =                 \
                &__tracepoint_##name

#endif

[...]

#define DEFINE_TRACE_FN(name, reg, unreg)                                \
        static const char __tpstrtab_##name[]                            \
        __attribute__((section("__tracepoints_strings"))) = #name;       \
        struct tracepoint __tracepoint_##name                            \
        __attribute__((section("__tracepoints"), used)) =                \
                { __tpstrtab_##name, STATIC_KEY_INIT_FALSE, reg, unreg, NULL };\
        __TRACEPOINT_ENTRY(name);

And kernel/module.c:

find_module_sections():

#ifdef CONFIG_TRACEPOINTS
        mod->tracepoints_ptrs = section_objs(info, "__tracepoints_ptrs",
                                             sizeof(*mod->tracepoints_ptrs),
                                             &mod->num_tracepoints);

#endif

And kernel/tracepoint.c:tracepoint_module_notify() for the module coming/going notifier.

Quoted 6 lines
Quoted 4 lines

Basically you would want to have your own structure within your own section of the module which describes the srcu domain, and have a module coming/going notifier responsible for dynamically allocating the srcu domain on "coming", and doing a srcu barrier and cleanup the domain on "going".

Ah, sounds like an excellent approach! I will give it a shot, thank you!

Please see below for an untested shot.

The original commits posted in this series are still available within the -srcu tree at branch srcunomod.2019.04.05a. Yes, I am a digital packrat. Why do you ask?

Thoughts? Or more accurately, given that this is the first time I have used linker sections, what did I mess up?

Thanx, Paul

------------------------------------------------------------------------

commit e24a0dab1414c563bb96bcb28d5963c9df18b1e8

Author: Paul E. McKenney <[email protected]>
Date:   Fri Apr 5 16:15:00 2019 -0700

srcu: Allocate per-CPU data for DEFINE_SRCU() in modules

Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires that the size of the reserved region be increased, which is not something we want to be doing all that often. One approach would be to require that loadable modules define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. However, this is more than a bit user unfriendly.

This commit therefore creates an ___srcu_struct_ptrs linker section, and pointers to srcu_struct structures created by DEFINE_SRCU() and DEFINE_STATIC_SRCU() within a module are placed into that module's ___srcu_struct_ptrs section. The required init_srcu_struct() and cleanup_srcu_struct() functions are then automatically invoked as needed when that module is loaded and unloaded, thus allowing modules to continue to use DEFINE_SRCU() and DEFINE_STATIC_SRCU() while avoiding the need to increase the size of the reserved region.

Many of the algorithms and some of the code was cheerfully cherry-picked from other code making use of linker sections, perhaps most notably from tracepoints. All bugs are nevertheless the sole property of the author.

include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

Thanx,Paul

Quoted 40 lines

Thanks!

---8<-----------------------

Quoted 1 line

From 369ad090f706ce8e1facdd18eb10828b5f7e2b72 Mon Sep 17 00:00:00 2001

From: "Joel Fernandes (Google)" <[email protected]>
Date: Sat, 6 Apr 2019 18:57:17 -0400
Subject: [PATCH] srcu: Remove unused vmlinux srcu linker entries

The SRCU for modules optimization introduced vmlinux linker entries which is unused since it applies only to the built-in vmlinux. So remove it to prevent any space usage due to the 8 byte alignment.

Tested with SRCU torture_type and rcutorture.

---
 include/asm-generic/vmlinux.lds.h | 4 ----
 1 file changed, 4 deletions(-)
include/asm-generic/vmlinux.lds.h +0 -4
index c2d919a1566e..f8f6f04c4453 100644
@@ -338,10 +338,6 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
-		. = ALIGN(8);						\
-		__start___srcu_struct = .;				\
-		*(___srcu_struct_ptrs)					\
-		__end___srcu_struct = .;				\
 	}								\
 									\
 	.rodata1          : AT(ADDR(.rodata1) - LOAD_OFFSET) {		\
Signature · 1 line
2.21.0.392.gf8f6787159e-goog
#19 lore raw
In reply to Joel Fernandes
Quoted 220 lines · Joel Fernandes
Quoted 219 lines · Paul E. McKenney
Quoted 120 lines · Paul E. McKenney
Quoted 119 lines · Mathieu Desnoyers
Quoted 38 lines · ----- On Apr 3, 2019, at 9:32 AM, paulmck…
Quoted 37 lines · Mathieu Desnoyers
Quoted 20 lines · ----- On Apr 2, 2019, at 11:23 AM, paulmck…
Quoted 14 lines · Mathieu Desnoyers
Quoted 11 lines · ----- On Apr 2, 2019, at 10:28 AM, paulmck…

Hello!

This series prohibits use of DEFINE_SRCU() and DEFINE_STATIC_SRCU() by loadable modules. The reason for this prohibition is the fact that using these two macros within modules requires that the size of the reserved region be increased, which is not something we want to be doing all that often. Instead, loadable modules should define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. Note that modules using call_srcu() will also need to invoke srcu_barrier() from their module_exit function.

Quoted 4 lines

This arbitrary API limitation seems weird.

Isn't there a way to allow modules to use DEFINE_SRCU and DEFINE_STATIC_SRCU while implementing them with dynamic allocation under the hood ?

Quoted 7 lines

Although call_srcu() already has initialization hooks, some would also be required in srcu_read_lock(), and I am concerned about adding memory allocation at that point, especially given the possibility of memory-allocation failure. And the possibility that the first srcu_read_lock() happens in an interrupt handler or similar.

Or am I missing a trick here?

I was more thinking that under #ifdef MODULE, both DEFINE_SRCU and DEFINE_STATIC_SRCU could append data in a dedicated section. module.c would additionally lookup that section on module load, and deal with those statically defined SRCU entries as if they were dynamically allocated ones. It would of course cleanup those resources on module unload.

Quoted 7 lines
Quoted 1 line

Am I missing some subtlety there ?

If I understand you correctly, that is actually what is already done. The size of this dedicated section is currently set by PERCPU_MODULE_RESERVE, and the additions of DEFINE{_STATIC}_SRCU() in modules was requiring that this to be increased frequently. That led to a request that something be done, in turn leading to this patch series.

I think we are not expressing quite the same idea.

AFAIU, yours is to have DEFINE*_SRCU directly define per-cpu data within modules, which ends up using percpu module reserved memory.

My idea is to make DEFINE*_SRCU have a different behavior under #ifdef MODULE. It could emit a _global variable_ (_not_ per-cpu) within a new section. That section would then be used by module init/exit code to figure out what "srcu descriptors" are present in the modules. It would therefore rely on dynamic allocation for those, therefore removing the need to involve the percpu module reserved pool at all.

Quoted 8 lines

I don't see a way around this short of changing module loading to do alloc_percpu() and then updating the relocation based on this result. Which would admittedly be far more convenient. I was assuming that this would be difficult due to varying CPU offsets or the like.

But if it can be done reasonably, it would be quite a bit nicer than forcing dynamic allocation in cases where it is not otherwise needed.

Hopefully my explanation above helps clear out what I have in mind.

You can find similar tricks performed by include/linux/tracepoint.h:

#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)

{
        return offset_to_ptr(p);
}
#define __TRACEPOINT_ENTRY(name)                                        \
        asm("   .section \"__tracepoints_ptrs\", \"a\"          \n"     \
            "   .balign 4                                       \n"     \
            "   .long   __tracepoint_" #name " - .              \n"     \
            "   .previous                                       \n")
#else
static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
{
        return *p;
}
#define __TRACEPOINT_ENTRY(name)                                         \
        static tracepoint_ptr_t __tracepoint_ptr_##name __used           \
        __attribute__((section("__tracepoints_ptrs"))) =                 \
                &__tracepoint_##name

#endif

[...]

#define DEFINE_TRACE_FN(name, reg, unreg)                                \
        static const char __tpstrtab_##name[]                            \
        __attribute__((section("__tracepoints_strings"))) = #name;       \
        struct tracepoint __tracepoint_##name                            \
        __attribute__((section("__tracepoints"), used)) =                \
                { __tpstrtab_##name, STATIC_KEY_INIT_FALSE, reg, unreg, NULL };\
        __TRACEPOINT_ENTRY(name);

And kernel/module.c:

find_module_sections():

#ifdef CONFIG_TRACEPOINTS
        mod->tracepoints_ptrs = section_objs(info, "__tracepoints_ptrs",
                                             sizeof(*mod->tracepoints_ptrs),
                                             &mod->num_tracepoints);

#endif

And kernel/tracepoint.c:tracepoint_module_notify() for the module coming/going notifier.

Quoted 6 lines
Quoted 4 lines

Basically you would want to have your own structure within your own section of the module which describes the srcu domain, and have a module coming/going notifier responsible for dynamically allocating the srcu domain on "coming", and doing a srcu barrier and cleanup the domain on "going".

Ah, sounds like an excellent approach! I will give it a shot, thank you!

Please see below for an untested shot.

The original commits posted in this series are still available within the -srcu tree at branch srcunomod.2019.04.05a. Yes, I am a digital packrat. Why do you ask?

Thoughts? Or more accurately, given that this is the first time I have used linker sections, what did I mess up?

Thanx, Paul

------------------------------------------------------------------------

commit e24a0dab1414c563bb96bcb28d5963c9df18b1e8

Author: Paul E. McKenney <[email protected]>
Date:   Fri Apr 5 16:15:00 2019 -0700

srcu: Allocate per-CPU data for DEFINE_SRCU() in modules

Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires that the size of the reserved region be increased, which is not something we want to be doing all that often. One approach would be to require that loadable modules define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. However, this is more than a bit user unfriendly.

This commit therefore creates an ___srcu_struct_ptrs linker section, and pointers to srcu_struct structures created by DEFINE_SRCU() and DEFINE_STATIC_SRCU() within a module are placed into that module's ___srcu_struct_ptrs section. The required init_srcu_struct() and cleanup_srcu_struct() functions are then automatically invoked as needed when that module is loaded and unloaded, thus allowing modules to continue to use DEFINE_SRCU() and DEFINE_STATIC_SRCU() while avoiding the need to increase the size of the reserved region.

Many of the algorithms and some of the code was cheerfully cherry-picked from other code making use of linker sections, perhaps most notably from tracepoints. All bugs are nevertheless the sole property of the author.

include/asm-generic/vmlinux.lds.h +4 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
+		__end___srcu_struct = .;				\
 	}								\
 									\
 	.rodata1          : AT(ADDR(.rodata1) - LOAD_OFFSET) {		\
include/linux/module.h +5 -0
index 5bf5dcd91009..921443a026dd 100644
@@ -21,6 +21,7 @@
 #include <linux/rbtree_latch.h>
 #include <linux/error-injection.h>
 #include <linux/tracepoint-defs.h>
+#include <linux/srcu.h>
 
 #include <linux/percpu.h>
 #include <asm/module.h>
@@ -448,6 +449,10 @@ struct module {
 	unsigned int num_tracepoints;
 	tracepoint_ptr_t *tracepoints_ptrs;
 #endif
+#ifdef CONFIG_TREE_SRCU
+	unsigned int num_srcu_structs;
+	struct srcu_struct **srcu_struct_ptrs;
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	unsigned int num_bpf_raw_events;
 	struct bpf_raw_event_map *bpf_raw_events;
include/linux/srcutree.h +0 -3
index 7f7c8c050f63..93685a9f3b4c 100644
@@ -120,9 +120,17 @@ struct srcu_struct {
  *
  * See include/linux/percpu-defs.h for the rules on per-CPU variables.
  */
-#define __DEFINE_SRCU(name, is_static)					\
-	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);\
-	is_static struct srcu_struct name = __SRCU_STRUCT_INIT(name, name##_srcu_data)
Quoted 8 lines

include/linux/srcutree.h struct srcu_struct { · line 127

Quoted 5 lines
include/linux/srcutree.h +5 -0
+#ifdef MODULE
+# define __DEFINE_SRCU(name, is_static)					\
+	is_static struct srcu_struct name;				\
+	struct srcu_struct *__srcu_struct_##name			\
+		__attribute__((section("___srcu_struct_ptrs"))) = &name

This can be replaced by:
__section("__srcu_struct_ptrs") = &name;

That is way easier on the eyes! Thank you, will fix.

Quoted 74 lines
Quoted 74 lines
include/linux/srcutree.h +6 -0
+#else
+# define __DEFINE_SRCU(name, is_static)					\
+	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);	\
+	is_static struct srcu_struct name =				\
+		__SRCU_STRUCT_INIT(name, name##_srcu_data)
+#endif
 #define DEFINE_SRCU(name)		__DEFINE_SRCU(name, /* not static */)
 #define DEFINE_STATIC_SRCU(name)	__DEFINE_SRCU(name, static)
 
kernel/module.c +5 -0
index 0b9aa8ab89f0..524da609c884 100644
@@ -3093,6 +3093,11 @@ static int find_module_sections(struct module *mod, struct load_info *info)
 					     sizeof(*mod->tracepoints_ptrs),
 					     &mod->num_tracepoints);
 #endif
+#ifdef CONFIG_TREE_SRCU
+	mod->srcu_struct_ptrs = section_objs(info, "___srcu_struct_ptrs",
+					     sizeof(*mod->srcu_struct_ptrs),
+					     &mod->num_srcu_structs);
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	mod->bpf_raw_events = section_objs(info, "__bpf_raw_tp_map",
 					   sizeof(*mod->bpf_raw_events),
kernel/rcu/srcutree.c +41 -0
index 9b761e546de8..ac0f6f0a8916 100644
@@ -1310,3 +1310,70 @@ void __init srcu_init(void)
 		queue_work(rcu_gp_wq, &ssp->work.work);
 	}
 }
+
+#ifdef CONFIG_MODULES
+
+/* Initialize any global-scope srcu_struct structures used by this module. */
+static int srcu_module_coming(struct module *mod)
+{
+	int i;
+	struct srcu_struct **sspp = mod->srcu_struct_ptrs;
+	int ret;
+
+	for (i = 0; i < mod->num_srcu_structs; i++) {
+		ret = init_srcu_struct(*(sspp++));
+		if (WARN_ON_ONCE(ret))
+			return ret;
+	}
+	return 0;
+}
+
+/* Clean up any global-scope srcu_struct structures used by this module. */
+static void srcu_module_going(struct module *mod)
+{
+	int i;
+	struct srcu_struct **sspp = mod->srcu_struct_ptrs;
+
+	for (i = 0; i < mod->num_srcu_structs; i++)
+		cleanup_srcu_struct(*(sspp++));
+}
+
+/* Handle one module, either coming or going. */
+static int srcu_module_notify(struct notifier_block *self,
+			      unsigned long val, void *data)
+{
+	struct module *mod = data;
+	int ret = 0;
+
+	switch (val) {
+	case MODULE_STATE_COMING:
+		ret = srcu_module_coming(mod);
+		break;
+	case MODULE_STATE_LIVE:
+		break;
Quoted 8 lines

kernel/rcu/srcutree.c void __init srcu_init(void) · line 1358

Quoted 5 lines
kernel/rcu/srcutree.c +5 -0
+	case MODULE_STATE_GOING:
+		srcu_module_going(mod);
+		break;
+	case MODULE_STATE_UNFORMED:
+		break;

The unused cases can be put under default or even clubbed thus saving a line but if you prefer to keep them.like this that is fine.

Well, saving two lines is saving two lines, so will apply.

Quoted 1 line

It looks good to me but I will test it out more later today. Thanks!

Thank you!

Thanx, Paul

Quoted 25 lines

- Joel

Quoted 21 lines
kernel/rcu/srcutree.c +20 -0
+	return ret;
+}
+
+static struct notifier_block srcu_module_nb = {
+	.notifier_call = srcu_module_notify,
+	.priority = 0,
+};
+
+static __init int init_srcu_module_notifier(void)
+{
+	int ret;
+
+	ret = register_module_notifier(&srcu_module_nb);
+	if (ret)
+		pr_warn("Failed to register srcu module notifier\n");
+	return ret;
+}
+late_initcall(init_srcu_module_notifier);
+
+#endif /* #ifdef CONFIG_MODULES */
#22 lore raw
In reply to Paul E. McKenney
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 20 lines
Quoted 11 lines
Quoted 11 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

And may I have your Tested-by?

Thanx, Paul

------------------------------------------------------------------------

commit a365bb5f6eafb220a1448674054b05c250829313

Author: Paul E. McKenney <[email protected]>
Date:   Fri Apr 5 16:15:00 2019 -0700

srcu: Allocate per-CPU data for DEFINE_SRCU() in modules

Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires that the size of the reserved region be increased, which is not something we want to be doing all that often. One approach would be to require that loadable modules define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. However, this is more than a bit user unfriendly.

This commit therefore creates an ___srcu_struct_ptrs linker section, and pointers to srcu_struct structures created by DEFINE_SRCU() and DEFINE_STATIC_SRCU() within a module are placed into that module's ___srcu_struct_ptrs section. The required init_srcu_struct() and cleanup_srcu_struct() functions are then automatically invoked as needed when that module is loaded and unloaded, thus allowing modules to continue to use DEFINE_SRCU() and DEFINE_STATIC_SRCU() while avoiding the need to increase the size of the reserved region.

Many of the algorithms and some of the code was cheerfully cherry-picked from other code making use of linker sections, perhaps most notably from tracepoints. All bugs are nevertheless the sole property of the author.

[ paulmck: Use __section() and use "default" in srcu_module_notify()'s
"switch" statement as suggested by Joel Fernandes. ]

include/asm-generic/vmlinux.lds.h +4 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
+		__end___srcu_struct = .;				\
 	}								\
 									\
 	.rodata1          : AT(ADDR(.rodata1) - LOAD_OFFSET) {		\
include/linux/module.h +5 -0
index 5bf5dcd91009..921443a026dd 100644
@@ -21,6 +21,7 @@
 #include <linux/rbtree_latch.h>
 #include <linux/error-injection.h>
 #include <linux/tracepoint-defs.h>
+#include <linux/srcu.h>
 
 #include <linux/percpu.h>
 #include <asm/module.h>
@@ -448,6 +449,10 @@ struct module {
 	unsigned int num_tracepoints;
 	tracepoint_ptr_t *tracepoints_ptrs;
 #endif
+#ifdef CONFIG_TREE_SRCU
+	unsigned int num_srcu_structs;
+	struct srcu_struct **srcu_struct_ptrs;
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	unsigned int num_bpf_raw_events;
 	struct bpf_raw_event_map *bpf_raw_events;
include/linux/srcutree.h +11 -3
index 7f7c8c050f63..8af1824c46a8 100644
@@ -120,9 +120,17 @@ struct srcu_struct {
  *
  * See include/linux/percpu-defs.h for the rules on per-CPU variables.
  */
-#define __DEFINE_SRCU(name, is_static)					\
-	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);\
-	is_static struct srcu_struct name = __SRCU_STRUCT_INIT(name, name##_srcu_data)
+#ifdef MODULE
+# define __DEFINE_SRCU(name, is_static)					\
+	is_static struct srcu_struct name;				\
+	struct srcu_struct *__srcu_struct_##name			\
+		__section("___srcu_struct_ptrs") = &name
+#else
+# define __DEFINE_SRCU(name, is_static)					\
+	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);	\
+	is_static struct srcu_struct name =				\
+		__SRCU_STRUCT_INIT(name, name##_srcu_data)
+#endif
 #define DEFINE_SRCU(name)		__DEFINE_SRCU(name, /* not static */)
 #define DEFINE_STATIC_SRCU(name)	__DEFINE_SRCU(name, static)
 
kernel/module.c +5 -0
index 0b9aa8ab89f0..524da609c884 100644
@@ -3093,6 +3093,11 @@ static int find_module_sections(struct module *mod, struct load_info *info)
 					     sizeof(*mod->tracepoints_ptrs),
 					     &mod->num_tracepoints);
 #endif
+#ifdef CONFIG_TREE_SRCU
+	mod->srcu_struct_ptrs = section_objs(info, "___srcu_struct_ptrs",
+					     sizeof(*mod->srcu_struct_ptrs),
+					     &mod->num_srcu_structs);
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	mod->bpf_raw_events = section_objs(info, "__bpf_raw_tp_map",
 					   sizeof(*mod->bpf_raw_events),
kernel/rcu/srcutree.c +65 -0

68 lines not on this page · raw message

#23 lore raw
Quoted 28 lines · Paul E. McKenney
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 23 lines
Quoted 11 lines
Quoted 11 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Good point, given that otherwise FORTRAN named common blocks would not work.

Quoted 8 lines
Quoted 3 lines

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Actually the vmlinux.lds.h file is unused for module building. For ex, if you delete include/asm-generic/vmlinux.lds.h , then you can still build rcutorture.ko. Did I miss something obvious? In that case the vmlinux.lds.h are not needed since the __section annotations automatically place the srcu structs in a separate section.

Let me know if you would like me to send a patch separately, or if the appended patch for the same in my previous email suffices.

Quoted 1 line

And may I have your Tested-by?

Absolutely, please do and thanks!

- Joel

Quoted 188 lines

Thanx, Paul

------------------------------------------------------------------------

commit a365bb5f6eafb220a1448674054b05c250829313

Author: Paul E. McKenney <[email protected]>
Date:   Fri Apr 5 16:15:00 2019 -0700

srcu: Allocate per-CPU data for DEFINE_SRCU() in modules

Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires that the size of the reserved region be increased, which is not something we want to be doing all that often. One approach would be to require that loadable modules define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. However, this is more than a bit user unfriendly.

This commit therefore creates an ___srcu_struct_ptrs linker section, and pointers to srcu_struct structures created by DEFINE_SRCU() and DEFINE_STATIC_SRCU() within a module are placed into that module's ___srcu_struct_ptrs section. The required init_srcu_struct() and cleanup_srcu_struct() functions are then automatically invoked as needed when that module is loaded and unloaded, thus allowing modules to continue to use DEFINE_SRCU() and DEFINE_STATIC_SRCU() while avoiding the need to increase the size of the reserved region.

Many of the algorithms and some of the code was cheerfully cherry-picked from other code making use of linker sections, perhaps most notably from tracepoints. All bugs are nevertheless the sole property of the author.

[ paulmck: Use __section() and use "default" in srcu_module_notify()'s
"switch" statement as suggested by Joel Fernandes. ]

include/asm-generic/vmlinux.lds.h +4 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
+		__end___srcu_struct = .;				\
 	}								\
 									\
 	.rodata1          : AT(ADDR(.rodata1) - LOAD_OFFSET) {		\
include/linux/module.h +5 -0
index 5bf5dcd91009..921443a026dd 100644
@@ -21,6 +21,7 @@
 #include <linux/rbtree_latch.h>
 #include <linux/error-injection.h>
 #include <linux/tracepoint-defs.h>
+#include <linux/srcu.h>
 
 #include <linux/percpu.h>
 #include <asm/module.h>
@@ -448,6 +449,10 @@ struct module {
 	unsigned int num_tracepoints;
 	tracepoint_ptr_t *tracepoints_ptrs;
 #endif
+#ifdef CONFIG_TREE_SRCU
+	unsigned int num_srcu_structs;
+	struct srcu_struct **srcu_struct_ptrs;
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	unsigned int num_bpf_raw_events;
 	struct bpf_raw_event_map *bpf_raw_events;
include/linux/srcutree.h +11 -3
index 7f7c8c050f63..8af1824c46a8 100644
@@ -120,9 +120,17 @@ struct srcu_struct {
  *
  * See include/linux/percpu-defs.h for the rules on per-CPU variables.
  */
-#define __DEFINE_SRCU(name, is_static)					\
-	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);\
-	is_static struct srcu_struct name = __SRCU_STRUCT_INIT(name, name##_srcu_data)
+#ifdef MODULE
+# define __DEFINE_SRCU(name, is_static)					\
+	is_static struct srcu_struct name;				\
+	struct srcu_struct *__srcu_struct_##name			\
+		__section("___srcu_struct_ptrs") = &name
+#else
+# define __DEFINE_SRCU(name, is_static)					\
+	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);	\
+	is_static struct srcu_struct name =				\
+		__SRCU_STRUCT_INIT(name, name##_srcu_data)
+#endif
 #define DEFINE_SRCU(name)		__DEFINE_SRCU(name, /* not static */)
 #define DEFINE_STATIC_SRCU(name)	__DEFINE_SRCU(name, static)
 
kernel/module.c +5 -0
index 0b9aa8ab89f0..524da609c884 100644
@@ -3093,6 +3093,11 @@ static int find_module_sections(struct module *mod, struct load_info *info)
 					     sizeof(*mod->tracepoints_ptrs),
 					     &mod->num_tracepoints);
 #endif
+#ifdef CONFIG_TREE_SRCU
+	mod->srcu_struct_ptrs = section_objs(info, "___srcu_struct_ptrs",
+					     sizeof(*mod->srcu_struct_ptrs),
+					     &mod->num_srcu_structs);
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	mod->bpf_raw_events = section_objs(info, "__bpf_raw_tp_map",
 					   sizeof(*mod->bpf_raw_events),
kernel/rcu/srcutree.c +65 -0
index 9b761e546de8..2ded2614a2f4 100644
@@ -1310,3 +1310,68 @@ void __init srcu_init(void)
 		queue_work(rcu_gp_wq, &ssp->work.work);
 	}
 }
+
+#ifdef CONFIG_MODULES
+
+/* Initialize any global-scope srcu_struct structures used by this module. */
+static int srcu_module_coming(struct module *mod)
+{
+	int i;
+	struct srcu_struct **sspp = mod->srcu_struct_ptrs;
+	int ret;
+
+	for (i = 0; i < mod->num_srcu_structs; i++) {
+		ret = init_srcu_struct(*(sspp++));
+		if (WARN_ON_ONCE(ret))
+			return ret;
+	}
+	return 0;
+}
+
+/* Clean up any global-scope srcu_struct structures used by this module. */
+static void srcu_module_going(struct module *mod)
+{
+	int i;
+	struct srcu_struct **sspp = mod->srcu_struct_ptrs;
+
+	for (i = 0; i < mod->num_srcu_structs; i++)
+		cleanup_srcu_struct(*(sspp++));
+}
+
+/* Handle one module, either coming or going. */
+static int srcu_module_notify(struct notifier_block *self,
+			      unsigned long val, void *data)
+{
+	struct module *mod = data;
+	int ret = 0;
+
+	switch (val) {
+	case MODULE_STATE_COMING:
+		ret = srcu_module_coming(mod);
+		break;
+	case MODULE_STATE_GOING:
+		srcu_module_going(mod);
+		break;
+	default:
+		break;
+	}
+	return ret;
+}
+
+static struct notifier_block srcu_module_nb = {
+	.notifier_call = srcu_module_notify,
+	.priority = 0,
+};
+
+static __init int init_srcu_module_notifier(void)
+{
+	int ret;
+
+	ret = register_module_notifier(&srcu_module_nb);
+	if (ret)
+		pr_warn("Failed to register srcu module notifier\n");
+	return ret;
+}
+late_initcall(init_srcu_module_notifier);
+
+#endif /* #ifdef CONFIG_MODULES */
#24 lore raw
Quoted 37 lines · Joel Fernandes
Quoted 28 lines · Paul E. McKenney
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 23 lines
Quoted 11 lines
Quoted 11 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Good point, given that otherwise FORTRAN named common blocks would not work.

Quoted 7 lines
Quoted 3 lines

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 5 lines

Actually the vmlinux.lds.h file is unused for module building. For ex, if you delete include/asm-generic/vmlinux.lds.h , then you can still build rcutorture.ko. Did I miss something obvious? In that case the vmlinux.lds.h are not needed since the __section annotations automatically place the srcu structs in a separate section.

Hard to argue given that I just deleted include/asm-generic/vmlinux.lds.h, touched kernel/rcu/rcutorture.c, and rebuilt the corresponding .ko without errors. ;-)

Hmmm... Is there some way to place a section into a read-only page, for example, tagged onto the text section for that module? That would get rid of a class of bugs, if nothing else.

Quoted 2 lines

Let me know if you would like me to send a patch separately, or if the appended patch for the same in my previous email suffices.

Please do resend as a formal patch with the above in the commit log. I doubt that I am the only one needing a bit of module-build education! And thank you for providing that education, by the way!

Quoted 3 lines
Quoted 1 line

And may I have your Tested-by?

Absolutely, please do and thanks!

Done, and thank you for giving it a go!

Thanx, Paul

Quoted 192 lines

- Joel

Quoted 188 lines

Thanx, Paul

------------------------------------------------------------------------

commit a365bb5f6eafb220a1448674054b05c250829313

Author: Paul E. McKenney <[email protected]>
Date:   Fri Apr 5 16:15:00 2019 -0700

srcu: Allocate per-CPU data for DEFINE_SRCU() in modules

Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires that the size of the reserved region be increased, which is not something we want to be doing all that often. One approach would be to require that loadable modules define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. However, this is more than a bit user unfriendly.

This commit therefore creates an ___srcu_struct_ptrs linker section, and pointers to srcu_struct structures created by DEFINE_SRCU() and DEFINE_STATIC_SRCU() within a module are placed into that module's ___srcu_struct_ptrs section. The required init_srcu_struct() and cleanup_srcu_struct() functions are then automatically invoked as needed when that module is loaded and unloaded, thus allowing modules to continue to use DEFINE_SRCU() and DEFINE_STATIC_SRCU() while avoiding the need to increase the size of the reserved region.

Many of the algorithms and some of the code was cheerfully cherry-picked from other code making use of linker sections, perhaps most notably from tracepoints. All bugs are nevertheless the sole property of the author.

[ paulmck: Use __section() and use "default" in srcu_module_notify()'s
"switch" statement as suggested by Joel Fernandes. ]

include/asm-generic/vmlinux.lds.h +4 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
+		__end___srcu_struct = .;				\
 	}								\
 									\
 	.rodata1          : AT(ADDR(.rodata1) - LOAD_OFFSET) {		\
include/linux/module.h +5 -0
index 5bf5dcd91009..921443a026dd 100644
@@ -21,6 +21,7 @@
 #include <linux/rbtree_latch.h>
 #include <linux/error-injection.h>
 #include <linux/tracepoint-defs.h>
+#include <linux/srcu.h>
 
 #include <linux/percpu.h>
 #include <asm/module.h>
@@ -448,6 +449,10 @@ struct module {
 	unsigned int num_tracepoints;
 	tracepoint_ptr_t *tracepoints_ptrs;
 #endif
+#ifdef CONFIG_TREE_SRCU
+	unsigned int num_srcu_structs;
+	struct srcu_struct **srcu_struct_ptrs;
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	unsigned int num_bpf_raw_events;
 	struct bpf_raw_event_map *bpf_raw_events;
include/linux/srcutree.h +11 -3
index 7f7c8c050f63..8af1824c46a8 100644
@@ -120,9 +120,17 @@ struct srcu_struct {
  *
  * See include/linux/percpu-defs.h for the rules on per-CPU variables.
  */
-#define __DEFINE_SRCU(name, is_static)					\
-	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);\
-	is_static struct srcu_struct name = __SRCU_STRUCT_INIT(name, name##_srcu_data)
+#ifdef MODULE
+# define __DEFINE_SRCU(name, is_static)					\
+	is_static struct srcu_struct name;				\
+	struct srcu_struct *__srcu_struct_##name			\
+		__section("___srcu_struct_ptrs") = &name
+#else
+# define __DEFINE_SRCU(name, is_static)					\
+	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);	\
+	is_static struct srcu_struct name =				\
+		__SRCU_STRUCT_INIT(name, name##_srcu_data)
+#endif
 #define DEFINE_SRCU(name)		__DEFINE_SRCU(name, /* not static */)
 #define DEFINE_STATIC_SRCU(name)	__DEFINE_SRCU(name, static)
 
kernel/module.c +5 -0
index 0b9aa8ab89f0..524da609c884 100644
@@ -3093,6 +3093,11 @@ static int find_module_sections(struct module *mod, struct load_info *info)
 					     sizeof(*mod->tracepoints_ptrs),
 					     &mod->num_tracepoints);
 #endif
+#ifdef CONFIG_TREE_SRCU
+	mod->srcu_struct_ptrs = section_objs(info, "___srcu_struct_ptrs",
+					     sizeof(*mod->srcu_struct_ptrs),
+					     &mod->num_srcu_structs);
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	mod->bpf_raw_events = section_objs(info, "__bpf_raw_tp_map",
 					   sizeof(*mod->bpf_raw_events),
kernel/rcu/srcutree.c +65 -0
index 9b761e546de8..2ded2614a2f4 100644
@@ -1310,3 +1310,68 @@ void __init srcu_init(void)
 		queue_work(rcu_gp_wq, &ssp->work.work);
 	}
 }
+
+#ifdef CONFIG_MODULES
+
+/* Initialize any global-scope srcu_struct structures used by this module. */
+static int srcu_module_coming(struct module *mod)
+{
+	int i;
+	struct srcu_struct **sspp = mod->srcu_struct_ptrs;
+	int ret;
+
+	for (i = 0; i < mod->num_srcu_structs; i++) {
+		ret = init_srcu_struct(*(sspp++));
+		if (WARN_ON_ONCE(ret))
+			return ret;
+	}
+	return 0;
+}
+
+/* Clean up any global-scope srcu_struct structures used by this module. */
+static void srcu_module_going(struct module *mod)
+{
+	int i;
+	struct srcu_struct **sspp = mod->srcu_struct_ptrs;
+
+	for (i = 0; i < mod->num_srcu_structs; i++)
+		cleanup_srcu_struct(*(sspp++));
+}
+
+/* Handle one module, either coming or going. */
+static int srcu_module_notify(struct notifier_block *self,
+			      unsigned long val, void *data)
+{
+	struct module *mod = data;
+	int ret = 0;
+
+	switch (val) {
+	case MODULE_STATE_COMING:
+		ret = srcu_module_coming(mod);
+		break;
+	case MODULE_STATE_GOING:
+		srcu_module_going(mod);
+		break;
+	default:
+		break;
+	}
+	return ret;
+}
+
+static struct notifier_block srcu_module_nb = {
+	.notifier_call = srcu_module_notify,
+	.priority = 0,
+};
+
+static __init int init_srcu_module_notifier(void)
+{
+	int ret;
+
+	ret = register_module_notifier(&srcu_module_nb);
+	if (ret)
+		pr_warn("Failed to register srcu module notifier\n");
+	return ret;
+}
+late_initcall(init_srcu_module_notifier);
+
+#endif /* #ifdef CONFIG_MODULES */
#27 lore raw
In reply to Paul E. McKenney
Quoted 33 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck…
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

Quoted 5 lines

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#28 lore raw
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck…
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

thanks.

Quoted 8 lines

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#29 lore raw
Quoted 42 lines · ----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google…
Quoted 41 lines · Mathieu Desnoyers
Quoted 38 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck…
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

And may I have your Tested-by?

Quoted 8 lines
Quoted 5 lines

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#30 lore raw
Quoted 51 lines · Mathieu Desnoyers
Quoted 48 lines · ----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google…
Quoted 46 lines · Mathieu Desnoyers
Quoted 33 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

Quoted 5 lines

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

Quoted 7 lines
Quoted 2 lines

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

thanks!

Quoted 9 lines

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#25 lore raw
In reply to Paul E. McKenney
Quoted 44 lines · Paul E. McKenney
Quoted 37 lines · Joel Fernandes
Quoted 28 lines · Paul E. McKenney
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 23 lines
Quoted 11 lines
Quoted 11 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Good point, given that otherwise FORTRAN named common blocks would not work.

Quoted 7 lines
Quoted 3 lines

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 5 lines

Actually the vmlinux.lds.h file is unused for module building. For ex, if you delete include/asm-generic/vmlinux.lds.h , then you can still build rcutorture.ko. Did I miss something obvious? In that case the vmlinux.lds.h are not needed since the __section annotations automatically place the srcu structs in a separate section.

Quoted 7 lines

Hard to argue given that I just deleted include/asm-generic/vmlinux.lds.h, touched kernel/rcu/rcutorture.c, and rebuilt the corresponding .ko without errors. ;-)

Hmmm... Is there some way to place a section into a read-only page, for example, tagged onto the text section for that module? That would get rid of a class of bugs, if nothing else.

Strictly speaking, the array of pointers in the new srcu section are fixed up at runtime because the srcu_struct(s) they point to can be loaded at a dynamic location in memory. The srcu_struct(s) are themselves in the .bss section of the module and their locations depend on where the .bss section of the module is loaded in memory at load time.

I agree that after such relocation fixups are done, there is no reason to keep the array-of-pointers section readable but unfortunately I couldn't figure a way out to make them read-only post the relocations.

I copied Jessica Yu who maintains module loading for any input. Jessica, as a summary, we are trying to create a custom ELF section of srcu_struct pointers in kernels modules, and then make the module loader do SRCU initialization from structs pointed to by this section. The srcu_struct themselves are defined on the .bss section. Is there any way we can make this pointer array section read-only *after* the relocation fixups of the array are completed?

Quoted 6 lines
Quoted 2 lines

Let me know if you would like me to send a patch separately, or if the appended patch for the same in my previous email suffices.

Please do resend as a formal patch with the above in the commit log. I doubt that I am the only one needing a bit of module-build education! And thank you for providing that education, by the way!

Sounds great, I will go ahead and send out a patch in the morning for this part.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

And may I have your Tested-by?

Absolutely, please do and thanks!

Done, and thank you for giving it a go!

You're very welcome. thanks,

- Joel

#31 lore raw
In reply to Joel Fernandes
Quoted 55 lines · Joel Fernandes
Quoted 54 lines · Mathieu Desnoyers
Quoted 50 lines · ----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google…
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

So if an srcu_barrier() is needed, the module developer needs to supply it.

Thanx, Paul

#26 lore raw
In reply to Joel Fernandes
Quoted 73 lines · Joel Fernandes
Quoted 44 lines · Paul E. McKenney
Quoted 37 lines · Joel Fernandes
Quoted 28 lines · Paul E. McKenney
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 23 lines
Quoted 11 lines
Quoted 11 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Good point, given that otherwise FORTRAN named common blocks would not work.

Quoted 7 lines
Quoted 3 lines

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 5 lines

Actually the vmlinux.lds.h file is unused for module building. For ex, if you delete include/asm-generic/vmlinux.lds.h , then you can still build rcutorture.ko. Did I miss something obvious? In that case the vmlinux.lds.h are not needed since the __section annotations automatically place the srcu structs in a separate section.

Quoted 7 lines

Hard to argue given that I just deleted include/asm-generic/vmlinux.lds.h, touched kernel/rcu/rcutorture.c, and rebuilt the corresponding .ko without errors. ;-)

Hmmm... Is there some way to place a section into a read-only page, for example, tagged onto the text section for that module? That would get rid of a class of bugs, if nothing else.

Strictly speaking, the array of pointers in the new srcu section are fixed up at runtime because the srcu_struct(s) they point to can be loaded at a dynamic location in memory. The srcu_struct(s) are themselves in the .bss section of the module and their locations depend on where the .bss section of the module is loaded in memory at load time.

I agree that after such relocation fixups are done, there is no reason to keep the array-of-pointers section readable but unfortunately I couldn't figure a way out to make them read-only post the relocations.

I copied Jessica Yu who maintains module loading for any input. Jessica, as a summary, we are trying to create a custom ELF section of srcu_struct pointers in kernels modules, and then make the module loader do SRCU initialization from structs pointed to by this section. The srcu_struct themselves are defined on the .bss section. Is there any way we can make this pointer array section read-only *after* the relocation fixups of the array are completed?

Quoted 3 lines
Quoted 2 lines

Let me know if you would like me to send a patch separately, or if the appended patch for the same in my previous email suffices.

Quoted 6 lines
Quoted 3 lines

Please do resend as a formal patch with the above in the commit log. I doubt that I am the only one needing a bit of module-build education! And thank you for providing that education, by the way!

Sounds great, I will go ahead and send out a patch in the morning for this part.

Sounds good on both counts!

Thanx, Paul

Quoted 10 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

And may I have your Tested-by?

Absolutely, please do and thanks!

Done, and thank you for giving it a go!

You're very welcome. thanks,

- Joel

#32 lore raw
In reply to Paul E. McKenney
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck [email protected] wrote:
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google [email protected]

Quoted 50 lines · wrote:
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#33 lore raw
Quoted 83 lines · Mathieu Desnoyers
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck [email protected] wrote:
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google [email protected]

Quoted 50 lines · wrote:
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

Quoted 4 lines

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

We do seem to be talking past each other. ;-)

This has nothing to do with the order of events at module-unload time.

So please let me try again.

If a given srcu_struct in a module never has call_srcu() invoked, there is no need to invoke rcu_barrier() at any time, whether at module-unload time or not. Adding rcu_barrier() in this case adds overhead and latency for no good reason.

If a given srcu_struct in a module does have at least one call_srcu() invoked, it is already that module's responsibility to make sure that the code sticks around long enough for the callback to be invoked.

This means that correct SRCU users that invoke call_srcu() already have srcu_barrier() at module-unload time. Incorrect SRCU users, with reasonable probability, now get a WARN_ON() at module-unload time, with the per-CPU state getting leaked. Before this change, they would (also with reasonable probability) instead get an instruction-fetch fault when the SRCU callback was invoked after the completion of the module unload. Furthermore, in all cases where they would previously have gotten the instruction-fetch fault, they now get the WARN_ON(), like this:

	if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
		return; /* Forgot srcu_barrier(), so just leak it! */

So this change already represents an improvement in usability.

Thanx, Paul

#34 lore raw
Quoted 91 lines · ----- On Apr 8, 2019, at 10:22 AM, paulmck [email protected] wrote:
Quoted 83 lines · Mathieu Desnoyers
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck [email protected] wrote:
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google [email protected]

Quoted 50 lines · wrote:
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

Quoted 4 lines

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

We do seem to be talking past each other. ;-)

Quoted 8 lines

This has nothing to do with the order of events at module-unload time.

So please let me try again.

If a given srcu_struct in a module never has call_srcu() invoked, there is no need to invoke rcu_barrier() at any time, whether at module-unload time or not. Adding rcu_barrier() in this case adds overhead and latency for no good reason.

Not if we invoke srcu_barrier() for that specific domain. If call_srcu was never invoked for a srcu domain, I don't see why srcu_barrier() should be more expensive than a simple check that the domain does not have any srcu work queued.

Quoted 4 lines

If a given srcu_struct in a module does have at least one call_srcu() invoked, it is already that module's responsibility to make sure that the code sticks around long enough for the callback to be invoked.

I understand that when users do explicit dynamic allocation/cleanup of srcu domains, they indeed need to take care of doing explicit srcu_barrier(). However, if they do static definition of srcu domains, it would be nice if we can handle the barriers under the hood.

Quoted 10 lines

This means that correct SRCU users that invoke call_srcu() already have srcu_barrier() at module-unload time. Incorrect SRCU users, with reasonable probability, now get a WARN_ON() at module-unload time, with the per-CPU state getting leaked. Before this change, they would (also with reasonable probability) instead get an instruction-fetch fault when the SRCU callback was invoked after the completion of the module unload. Furthermore, in all cases where they would previously have gotten the instruction-fetch fault, they now get the WARN_ON(), like this:

Quoted 4 lines
	if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
		return; /* Forgot srcu_barrier(), so just leak it! */

So this change already represents an improvement in usability.

Considering that we can do a srcu_barrier() for the specific domain, and that it should add no noticeable overhead if there is no queued callbacks, I don't see a good reason for leaving the srcu_barrier invocation to the user rather than implicitly doing it from the module going notifier.

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#35 lore raw
Quoted 102 lines · Mathieu Desnoyers
Quoted 91 lines · ----- On Apr 8, 2019, at 10:22 AM, paulmck [email protected] wrote:
Quoted 83 lines · Mathieu Desnoyers
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck [email protected] wrote:
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google [email protected]

Quoted 50 lines · wrote:
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
include/asm-generic/vmlinux.lds.h +3 -0
index f8f6f04c4453..c2d919a1566e 100644
@@ -338,6 +338,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
Quoted 8 lines

include/asm-generic/vmlinux.lds.h line 345

Quoted 2 lines
include/asm-generic/vmlinux.lds.h +1 -0
+		__end___srcu_struct = .;				\
 	}								\

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

Quoted 4 lines

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

We do seem to be talking past each other. ;-)

Quoted 8 lines

This has nothing to do with the order of events at module-unload time.

So please let me try again.

If a given srcu_struct in a module never has call_srcu() invoked, there is no need to invoke rcu_barrier() at any time, whether at module-unload time or not. Adding rcu_barrier() in this case adds overhead and latency for no good reason.

Quoted 4 lines

Not if we invoke srcu_barrier() for that specific domain. If call_srcu was never invoked for a srcu domain, I don't see why srcu_barrier() should be more expensive than a simple check that the domain does not have any srcu work queued.

But that simple check does involve a cache miss for each possible CPU (not just each online CPU), so it is non-trivial, especially on large systems.

Quoted 8 lines
Quoted 3 lines

If a given srcu_struct in a module does have at least one call_srcu() invoked, it is already that module's responsibility to make sure that the code sticks around long enough for the callback to be invoked.

I understand that when users do explicit dynamic allocation/cleanup of srcu domains, they indeed need to take care of doing explicit srcu_barrier(). However, if they do static definition of srcu domains, it would be nice if we can handle the barriers under the hood.

All else being equal, of course. But...

Quoted 12 lines
Quoted 12 lines

This means that correct SRCU users that invoke call_srcu() already have srcu_barrier() at module-unload time. Incorrect SRCU users, with reasonable probability, now get a WARN_ON() at module-unload time, with the per-CPU state getting leaked. Before this change, they would (also with reasonable probability) instead get an instruction-fetch fault when the SRCU callback was invoked after the completion of the module unload. Furthermore, in all cases where they would previously have gotten the instruction-fetch fault, they now get the WARN_ON(), like this:

	if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
		return; /* Forgot srcu_barrier(), so just leak it! */
Quoted 7 lines
Quoted 1 line

So this change already represents an improvement in usability.

Considering that we can do a srcu_barrier() for the specific domain, and that it should add no noticeable overhead if there is no queued callbacks, I don't see a good reason for leaving the srcu_barrier invocation to the user rather than implicitly doing it from the module going notifier.

Now, I could automatically add an indicator of whether or not a call_srcu() had happened, but then again, that would either add a call_srcu() scalability bottleneck or again require a scan of all possible CPUs... to figure out if it was necessary to scan all possible CPUs.

Or is scanning all possible CPUs down in the noise in this case? Or am I missing a trick that would reduce the overhead?

Thanx, Paul

#36 lore raw
Quoted 142 lines · ----- On Apr 8, 2019, at 11:46 AM, paulmck [email protected] wrote:
Quoted 102 lines · Mathieu Desnoyers
Quoted 91 lines · ----- On Apr 8, 2019, at 10:22 AM, paulmck [email protected] wrote:
Quoted 83 lines · Mathieu Desnoyers
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck [email protected] wrote:
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google [email protected]

Quoted 50 lines · wrote:
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines

diff --git a/include/asm-generic/vmlinux.lds.h
b/include/asm-generic/vmlinux.lds.h
index f8f6f04c4453..c2d919a1566e 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -338,6 +338,10 @@
KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
__stop___tracepoints_ptrs = .; \
*(__tracepoints_strings)/* Tracepoints: strings */ \
+ . = ALIGN(8); \
+ __start___srcu_struct = .; \
+ *(___srcu_struct_ptrs) \

Quoted 8 lines
Quoted 2 lines

+ __end___srcu_struct = .; \
} \

This vmlinux linker modification is not needed. I tested without it and srcu torture works fine with rcutorture built as a module. Putting further prints in kernel/module.c verified that the kernel is able to find the srcu structs just fine. You could squash the below patch into this one or apply it on top of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

Quoted 4 lines

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

We do seem to be talking past each other. ;-)

Quoted 8 lines

This has nothing to do with the order of events at module-unload time.

So please let me try again.

If a given srcu_struct in a module never has call_srcu() invoked, there is no need to invoke rcu_barrier() at any time, whether at module-unload time or not. Adding rcu_barrier() in this case adds overhead and latency for no good reason.

Quoted 4 lines

Not if we invoke srcu_barrier() for that specific domain. If call_srcu was never invoked for a srcu domain, I don't see why srcu_barrier() should be more expensive than a simple check that the domain does not have any srcu work queued.

But that simple check does involve a cache miss for each possible CPU (not just each online CPU), so it is non-trivial, especially on large systems.

Quoted 8 lines
Quoted 3 lines

If a given srcu_struct in a module does have at least one call_srcu() invoked, it is already that module's responsibility to make sure that the code sticks around long enough for the callback to be invoked.

I understand that when users do explicit dynamic allocation/cleanup of srcu domains, they indeed need to take care of doing explicit srcu_barrier(). However, if they do static definition of srcu domains, it would be nice if we can handle the barriers under the hood.

All else being equal, of course. But...

Quoted 12 lines
Quoted 12 lines

This means that correct SRCU users that invoke call_srcu() already have srcu_barrier() at module-unload time. Incorrect SRCU users, with reasonable probability, now get a WARN_ON() at module-unload time, with the per-CPU state getting leaked. Before this change, they would (also with reasonable probability) instead get an instruction-fetch fault when the SRCU callback was invoked after the completion of the module unload. Furthermore, in all cases where they would previously have gotten the instruction-fetch fault, they now get the WARN_ON(), like this:

	if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
		return; /* Forgot srcu_barrier(), so just leak it! */
Quoted 7 lines
Quoted 1 line

So this change already represents an improvement in usability.

Considering that we can do a srcu_barrier() for the specific domain, and that it should add no noticeable overhead if there is no queued callbacks, I don't see a good reason for leaving the srcu_barrier invocation to the user rather than implicitly doing it from the module going notifier.

Quoted 7 lines

Now, I could automatically add an indicator of whether or not a call_srcu() had happened, but then again, that would either add a call_srcu() scalability bottleneck or again require a scan of all possible CPUs... to figure out if it was necessary to scan all possible CPUs.

Or is scanning all possible CPUs down in the noise in this case? Or am I missing a trick that would reduce the overhead?

Module unloading implicitly does a synchronize_rcu (for RCU-sched), and a stop_machine. So I would be tempted to say that overhead of iteration over all CPUs might not matter that much considering the rest.

About notifying that a call_srcu has happened for the srcu domain in a scalable fashion, let's see... We could have a flag "call_srcu_used" for each call_srcu domain. Whenever call_srcu is invoked, it would load that flag. It sets it on first use.

The idea here is to only use that flag when srcu_barrier is performed right before the srcu domain cleanup (it could become part of that cleanup). Else, using it in all srcu_barrier() might be tricky, because we may then need to add memory barriers or locking to the call_srcu fast-path, which is an overhead we try to avoid.

However, if we only use that flag as part of the srcu domain cleanup, it's already prohibited to invoke call_srcu concurrently with the cleanup of the same domain, so I don't think we would need any memory barriers in call_srcu.

Thoughts ?

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#37 lore raw
Quoted 167 lines · Mathieu Desnoyers
Quoted 142 lines · ----- On Apr 8, 2019, at 11:46 AM, paulmck [email protected] wrote:
Quoted 102 lines · Mathieu Desnoyers
Quoted 91 lines · ----- On Apr 8, 2019, at 10:22 AM, paulmck [email protected] wrote:
Quoted 83 lines · Mathieu Desnoyers
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck [email protected] wrote:
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google [email protected]

Quoted 50 lines · wrote:
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 21 lines
Quoted 20 lines

diff --git a/include/asm-generic/vmlinux.lds.h
b/include/asm-generic/vmlinux.lds.h
index f8f6f04c4453..c2d919a1566e 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -338,6 +338,10 @@
KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
__stop___tracepoints_ptrs = .; \
*(__tracepoints_strings)/* Tracepoints: strings */ \
+ . = ALIGN(8); \
+ __start___srcu_struct = .; \
+ *(___srcu_struct_ptrs) \
+ __end___srcu_struct = .; \
} \

This vmlinux linker modification is not needed. I tested without it and srcu
torture works fine with rcutorture built as a module. Putting further prints
in kernel/module.c verified that the kernel is able to find the srcu structs
just fine. You could squash the below patch into this one or apply it on top
of the dev branch.

Quoted 6 lines

Good point, given that otherwise FORTRAN named common blocks would not work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION() macro that it can be mapped read-only? Or am I suffering from excessive optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

Quoted 4 lines

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

We do seem to be talking past each other. ;-)

Quoted 8 lines

This has nothing to do with the order of events at module-unload time.

So please let me try again.

If a given srcu_struct in a module never has call_srcu() invoked, there is no need to invoke rcu_barrier() at any time, whether at module-unload time or not. Adding rcu_barrier() in this case adds overhead and latency for no good reason.

Quoted 4 lines

Not if we invoke srcu_barrier() for that specific domain. If call_srcu was never invoked for a srcu domain, I don't see why srcu_barrier() should be more expensive than a simple check that the domain does not have any srcu work queued.

But that simple check does involve a cache miss for each possible CPU (not just each online CPU), so it is non-trivial, especially on large systems.

Quoted 8 lines
Quoted 3 lines

If a given srcu_struct in a module does have at least one call_srcu() invoked, it is already that module's responsibility to make sure that the code sticks around long enough for the callback to be invoked.

I understand that when users do explicit dynamic allocation/cleanup of srcu domains, they indeed need to take care of doing explicit srcu_barrier(). However, if they do static definition of srcu domains, it would be nice if we can handle the barriers under the hood.

All else being equal, of course. But...

Quoted 12 lines
Quoted 12 lines

This means that correct SRCU users that invoke call_srcu() already have srcu_barrier() at module-unload time. Incorrect SRCU users, with reasonable probability, now get a WARN_ON() at module-unload time, with the per-CPU state getting leaked. Before this change, they would (also with reasonable probability) instead get an instruction-fetch fault when the SRCU callback was invoked after the completion of the module unload. Furthermore, in all cases where they would previously have gotten the instruction-fetch fault, they now get the WARN_ON(), like this:

	if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
		return; /* Forgot srcu_barrier(), so just leak it! */
Quoted 7 lines
Quoted 1 line

So this change already represents an improvement in usability.

Considering that we can do a srcu_barrier() for the specific domain, and that it should add no noticeable overhead if there is no queued callbacks, I don't see a good reason for leaving the srcu_barrier invocation to the user rather than implicitly doing it from the module going notifier.

Quoted 7 lines

Now, I could automatically add an indicator of whether or not a call_srcu() had happened, but then again, that would either add a call_srcu() scalability bottleneck or again require a scan of all possible CPUs... to figure out if it was necessary to scan all possible CPUs.

Or is scanning all possible CPUs down in the noise in this case? Or am I missing a trick that would reduce the overhead?

Module unloading implicitly does a synchronize_rcu (for RCU-sched), and a stop_machine. So I would be tempted to say that overhead of iteration over all CPUs might not matter that much considering the rest.

About notifying that a call_srcu has happened for the srcu domain in a scalable fashion, let's see... We could have a flag "call_srcu_used" for each call_srcu domain. Whenever call_srcu is invoked, it would load that flag. It sets it on first use.

The idea here is to only use that flag when srcu_barrier is performed right before the srcu domain cleanup (it could become part of that cleanup). Else, using it in all srcu_barrier() might be tricky, because we may then need to add memory barriers or locking to the call_srcu fast-path, which is an overhead we try to avoid.

Quoted 4 lines

However, if we only use that flag as part of the srcu domain cleanup, it's already prohibited to invoke call_srcu concurrently with the cleanup of the same domain, so I don't think we would need any memory barriers in call_srcu.

About the last part of your email, it seems to that if after call_srcu has returned, if the module could be unloaded on some other CPU - then it would need to see the flag stored by the preceding call_srcu, so I believe there would be a memory barrier between the two opreations (call_srcu and module unload).

Also about doing the unconditional srcu_barrier, since a module could be unloaded at any time - don't all SRCU using modules need to invoke srcu_barrier() during their clean up anyway so we are incurring the barrier overhead anyway? Or, am I missing a design pattern here? It seems to me rcutorture module definitely calls srcu_barrier() before it is unloaded.

thanks,

- Joel

Quoted 10 lines

Thoughts ?

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#38 lore raw
Quoted 173 lines · ----- On Apr 9, 2019, at 11:40 AM, Joel Fernandes, Google…
Quoted 167 lines · Mathieu Desnoyers
Quoted 142 lines · ----- On Apr 8, 2019, at 11:46 AM, paulmck…
Quoted 102 lines · Mathieu Desnoyers
Quoted 91 lines · ----- On Apr 8, 2019, at 10:22 AM, paulmck…
Quoted 83 lines · Mathieu Desnoyers
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck…
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google joel-QYYGw3jwrUn5owFQY34kdNi2O/[email protected]

Quoted 50 lines · wrote:
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck…
Quoted 1 line · Paul E. McKenney

On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

Quoted 27 lines

diff --git a/include/asm-generic/vmlinux.lds.h
b/include/asm-generic/vmlinux.lds.h
index f8f6f04c4453..c2d919a1566e 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -338,6 +338,10 @@
KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
__stop___tracepoints_ptrs = .; \
*(__tracepoints_strings)/* Tracepoints: strings */ \
+ . = ALIGN(8); \
+ __start___srcu_struct = .; \
+ *(___srcu_struct_ptrs) \
+ __end___srcu_struct = .; \
} \

This vmlinux linker modification is not needed. I tested without it and srcu
torture works fine with rcutorture built as a module. Putting further prints
in kernel/module.c verified that the kernel is able to find the srcu structs
just fine. You could squash the below patch into this one or apply it on top
of the dev branch.

Good point, given that otherwise FORTRAN named common blocks would not
work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION()
macro that it can be mapped read-only? Or am I suffering from excessive
optimism?

And to answer the other question, in the case where I am suffering from excessive optimism, it should be a separate commit. Please see below for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

Quoted 4 lines

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

We do seem to be talking past each other. ;-)

Quoted 8 lines

This has nothing to do with the order of events at module-unload time.

So please let me try again.

If a given srcu_struct in a module never has call_srcu() invoked, there is no need to invoke rcu_barrier() at any time, whether at module-unload time or not. Adding rcu_barrier() in this case adds overhead and latency for no good reason.

Quoted 4 lines

Not if we invoke srcu_barrier() for that specific domain. If call_srcu was never invoked for a srcu domain, I don't see why srcu_barrier() should be more expensive than a simple check that the domain does not have any srcu work queued.

But that simple check does involve a cache miss for each possible CPU (not just each online CPU), so it is non-trivial, especially on large systems.

Quoted 8 lines
Quoted 3 lines

If a given srcu_struct in a module does have at least one call_srcu() invoked, it is already that module's responsibility to make sure that the code sticks around long enough for the callback to be invoked.

I understand that when users do explicit dynamic allocation/cleanup of srcu domains, they indeed need to take care of doing explicit srcu_barrier(). However, if they do static definition of srcu domains, it would be nice if we can handle the barriers under the hood.

All else being equal, of course. But...

Quoted 12 lines
Quoted 12 lines

This means that correct SRCU users that invoke call_srcu() already have srcu_barrier() at module-unload time. Incorrect SRCU users, with reasonable probability, now get a WARN_ON() at module-unload time, with the per-CPU state getting leaked. Before this change, they would (also with reasonable probability) instead get an instruction-fetch fault when the SRCU callback was invoked after the completion of the module unload. Furthermore, in all cases where they would previously have gotten the instruction-fetch fault, they now get the WARN_ON(), like this:

	if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
		return; /* Forgot srcu_barrier(), so just leak it! */
Quoted 7 lines
Quoted 1 line

So this change already represents an improvement in usability.

Considering that we can do a srcu_barrier() for the specific domain, and that it should add no noticeable overhead if there is no queued callbacks, I don't see a good reason for leaving the srcu_barrier invocation to the user rather than implicitly doing it from the module going notifier.

Quoted 7 lines

Now, I could automatically add an indicator of whether or not a call_srcu() had happened, but then again, that would either add a call_srcu() scalability bottleneck or again require a scan of all possible CPUs... to figure out if it was necessary to scan all possible CPUs.

Or is scanning all possible CPUs down in the noise in this case? Or am I missing a trick that would reduce the overhead?

Module unloading implicitly does a synchronize_rcu (for RCU-sched), and a stop_machine. So I would be tempted to say that overhead of iteration over all CPUs might not matter that much considering the rest.

About notifying that a call_srcu has happened for the srcu domain in a scalable fashion, let's see... We could have a flag "call_srcu_used" for each call_srcu domain. Whenever call_srcu is invoked, it would load that flag. It sets it on first use.

The idea here is to only use that flag when srcu_barrier is performed right before the srcu domain cleanup (it could become part of that cleanup). Else, using it in all srcu_barrier() might be tricky, because we may then need to add memory barriers or locking to the call_srcu fast-path, which is an overhead we try to avoid.

Quoted 4 lines

However, if we only use that flag as part of the srcu domain cleanup, it's already prohibited to invoke call_srcu concurrently with the cleanup of the same domain, so I don't think we would need any memory barriers in call_srcu.

Quoted 5 lines

About the last part of your email, it seems to that if after call_srcu has returned, if the module could be unloaded on some other CPU - then it would need to see the flag stored by the preceding call_srcu, so I believe there would be a memory barrier between the two opreations (call_srcu and module unload).

In order for the module unload not to race against module execution, it needs to happen after the call_srcu in a way that is already ordered by other means, else module unload races against the module code.

Quoted 6 lines

Also about doing the unconditional srcu_barrier, since a module could be unloaded at any time - don't all SRCU using modules need to invoke srcu_barrier() during their clean up anyway so we are incurring the barrier overhead anyway? Or, am I missing a design pattern here? It seems to me rcutorture module definitely calls srcu_barrier() before it is unloaded.

I think a valid approach which is even simpler might be: if a module statically defines a SRCU domain, it should be expected to use it. So adding a srcu_barrier() to its module going notifier should not hurt. The rare case where a module defines a static SRCU domain *and* does not actually use it with call_srcu() does not seem that usual, and not worth optimizing for.

Thoughts ?

Thanks,

Mathieu

Quoted 15 lines

thanks,

- Joel

Quoted 10 lines

Thoughts ?

Thanks,

Mathieu

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#39 lore raw
Quoted 181 lines · Mathieu Desnoyers
Quoted 173 lines · ----- On Apr 9, 2019, at 11:40 AM, Joel Fernandes, Google…
Quoted 167 lines · Mathieu Desnoyers
Quoted 142 lines · ----- On Apr 8, 2019, at 11:46 AM, paulmck [email protected] wrote:
Quoted 102 lines · Mathieu Desnoyers
Quoted 91 lines · ----- On Apr 8, 2019, at 10:22 AM, paulmck [email protected] wrote:
Quoted 83 lines · Mathieu Desnoyers
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck [email protected] wrote:
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google [email protected]

Quoted 50 lines · wrote:
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:

On Sun, Apr 07, 2019 at 06:39:41AM -0700, Paul E. McKenney wrote:
On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

diff --git a/include/asm-generic/vmlinux.lds.h
b/include/asm-generic/vmlinux.lds.h
index f8f6f04c4453..c2d919a1566e 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -338,6 +338,10 @@
KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
__stop___tracepoints_ptrs = .; \
*(__tracepoints_strings)/* Tracepoints: strings */ \
+ . = ALIGN(8); \
+ __start___srcu_struct = .; \
+ *(___srcu_struct_ptrs) \
+ __end___srcu_struct = .; \
} \

This vmlinux linker modification is not needed. I tested without it and srcu
torture works fine with rcutorture built as a module. Putting further prints
in kernel/module.c verified that the kernel is able to find the srcu structs
just fine. You could squash the below patch into this one or apply it on top
of the dev branch.

Good point, given that otherwise FORTRAN named common blocks would not
work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION()
macro that it can be mapped read-only? Or am I suffering from excessive
optimism?

And to answer the other question, in the case where I am suffering from
excessive optimism, it should be a separate commit. Please see below
for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

Quoted 4 lines

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

We do seem to be talking past each other. ;-)

Quoted 8 lines

This has nothing to do with the order of events at module-unload time.

So please let me try again.

If a given srcu_struct in a module never has call_srcu() invoked, there is no need to invoke rcu_barrier() at any time, whether at module-unload time or not. Adding rcu_barrier() in this case adds overhead and latency for no good reason.

Quoted 4 lines

Not if we invoke srcu_barrier() for that specific domain. If call_srcu was never invoked for a srcu domain, I don't see why srcu_barrier() should be more expensive than a simple check that the domain does not have any srcu work queued.

But that simple check does involve a cache miss for each possible CPU (not just each online CPU), so it is non-trivial, especially on large systems.

Quoted 8 lines
Quoted 3 lines

If a given srcu_struct in a module does have at least one call_srcu() invoked, it is already that module's responsibility to make sure that the code sticks around long enough for the callback to be invoked.

I understand that when users do explicit dynamic allocation/cleanup of srcu domains, they indeed need to take care of doing explicit srcu_barrier(). However, if they do static definition of srcu domains, it would be nice if we can handle the barriers under the hood.

All else being equal, of course. But...

Quoted 12 lines
Quoted 12 lines

This means that correct SRCU users that invoke call_srcu() already have srcu_barrier() at module-unload time. Incorrect SRCU users, with reasonable probability, now get a WARN_ON() at module-unload time, with the per-CPU state getting leaked. Before this change, they would (also with reasonable probability) instead get an instruction-fetch fault when the SRCU callback was invoked after the completion of the module unload. Furthermore, in all cases where they would previously have gotten the instruction-fetch fault, they now get the WARN_ON(), like this:

	if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
		return; /* Forgot srcu_barrier(), so just leak it! */
Quoted 7 lines
Quoted 1 line

So this change already represents an improvement in usability.

Considering that we can do a srcu_barrier() for the specific domain, and that it should add no noticeable overhead if there is no queued callbacks, I don't see a good reason for leaving the srcu_barrier invocation to the user rather than implicitly doing it from the module going notifier.

Quoted 7 lines

Now, I could automatically add an indicator of whether or not a call_srcu() had happened, but then again, that would either add a call_srcu() scalability bottleneck or again require a scan of all possible CPUs... to figure out if it was necessary to scan all possible CPUs.

Or is scanning all possible CPUs down in the noise in this case? Or am I missing a trick that would reduce the overhead?

Module unloading implicitly does a synchronize_rcu (for RCU-sched), and a stop_machine. So I would be tempted to say that overhead of iteration over all CPUs might not matter that much considering the rest.

About notifying that a call_srcu has happened for the srcu domain in a scalable fashion, let's see... We could have a flag "call_srcu_used" for each call_srcu domain. Whenever call_srcu is invoked, it would load that flag. It sets it on first use.

The idea here is to only use that flag when srcu_barrier is performed right before the srcu domain cleanup (it could become part of that cleanup). Else, using it in all srcu_barrier() might be tricky, because we may then need to add memory barriers or locking to the call_srcu fast-path, which is an overhead we try to avoid.

Quoted 4 lines

However, if we only use that flag as part of the srcu domain cleanup, it's already prohibited to invoke call_srcu concurrently with the cleanup of the same domain, so I don't think we would need any memory barriers in call_srcu.

Quoted 5 lines

About the last part of your email, it seems to that if after call_srcu has returned, if the module could be unloaded on some other CPU - then it would need to see the flag stored by the preceding call_srcu, so I believe there would be a memory barrier between the two opreations (call_srcu and module unload).

Quoted 3 lines

In order for the module unload not to race against module execution, it needs to happen after the call_srcu in a way that is already ordered by other means, else module unload races against the module code.

Agreed.

Quoted 11 lines
Quoted 5 lines

Also about doing the unconditional srcu_barrier, since a module could be unloaded at any time - don't all SRCU using modules need to invoke srcu_barrier() during their clean up anyway so we are incurring the barrier overhead anyway? Or, am I missing a design pattern here? It seems to me rcutorture module definitely calls srcu_barrier() before it is unloaded.

I think a valid approach which is even simpler might be: if a module statically defines a SRCU domain, it should be expected to use it. So adding a srcu_barrier() to its module going notifier should not hurt. The rare case where a module defines a static SRCU domain *and* does not actually use it with call_srcu() does not seem that usual, and not worth optimizing for.

That sounds like a reasonable approach to me.

thanks,

- Joel

#40 lore raw
In reply to Mathieu Desnoyers
Quoted 192 lines · Mathieu Desnoyers
Quoted 173 lines · ----- On Apr 9, 2019, at 11:40 AM, Joel Fernandes, Google…
Quoted 167 lines · Mathieu Desnoyers
Quoted 142 lines · ----- On Apr 8, 2019, at 11:46 AM, paulmck [email protected] wrote:
Quoted 102 lines · Mathieu Desnoyers
Quoted 91 lines · ----- On Apr 8, 2019, at 10:22 AM, paulmck [email protected] wrote:
Quoted 83 lines · Mathieu Desnoyers
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck [email protected] wrote:
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google [email protected]

Quoted 50 lines · wrote:
Quoted 39 lines · Mathieu Desnoyers
Quoted 37 lines · ----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:

On Sun, Apr 07, 2019 at 06:39:41AM -0700, Paul E. McKenney wrote:
On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

diff --git a/include/asm-generic/vmlinux.lds.h
b/include/asm-generic/vmlinux.lds.h
index f8f6f04c4453..c2d919a1566e 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -338,6 +338,10 @@
KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
__stop___tracepoints_ptrs = .; \
*(__tracepoints_strings)/* Tracepoints: strings */ \
+ . = ALIGN(8); \
+ __start___srcu_struct = .; \
+ *(___srcu_struct_ptrs) \
+ __end___srcu_struct = .; \
} \

This vmlinux linker modification is not needed. I tested without it and srcu
torture works fine with rcutorture built as a module. Putting further prints
in kernel/module.c verified that the kernel is able to find the srcu structs
just fine. You could squash the below patch into this one or apply it on top
of the dev branch.

Good point, given that otherwise FORTRAN named common blocks would not
work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION()
macro that it can be mapped read-only? Or am I suffering from excessive
optimism?

And to answer the other question, in the case where I am suffering from
excessive optimism, it should be a separate commit. Please see below
for the updated original commit thus far.

Quoted 7 lines
Quoted 1 line

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going notifier end up acting as a barrier first before freeing the memory ? If not, is it explicitly stated that a barrier must be issued before module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

Quoted 4 lines

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

We do seem to be talking past each other. ;-)

Quoted 8 lines

This has nothing to do with the order of events at module-unload time.

So please let me try again.

If a given srcu_struct in a module never has call_srcu() invoked, there is no need to invoke rcu_barrier() at any time, whether at module-unload time or not. Adding rcu_barrier() in this case adds overhead and latency for no good reason.

Quoted 4 lines

Not if we invoke srcu_barrier() for that specific domain. If call_srcu was never invoked for a srcu domain, I don't see why srcu_barrier() should be more expensive than a simple check that the domain does not have any srcu work queued.

But that simple check does involve a cache miss for each possible CPU (not just each online CPU), so it is non-trivial, especially on large systems.

Quoted 8 lines
Quoted 3 lines

If a given srcu_struct in a module does have at least one call_srcu() invoked, it is already that module's responsibility to make sure that the code sticks around long enough for the callback to be invoked.

I understand that when users do explicit dynamic allocation/cleanup of srcu domains, they indeed need to take care of doing explicit srcu_barrier(). However, if they do static definition of srcu domains, it would be nice if we can handle the barriers under the hood.

All else being equal, of course. But...

Quoted 12 lines
Quoted 12 lines

This means that correct SRCU users that invoke call_srcu() already have srcu_barrier() at module-unload time. Incorrect SRCU users, with reasonable probability, now get a WARN_ON() at module-unload time, with the per-CPU state getting leaked. Before this change, they would (also with reasonable probability) instead get an instruction-fetch fault when the SRCU callback was invoked after the completion of the module unload. Furthermore, in all cases where they would previously have gotten the instruction-fetch fault, they now get the WARN_ON(), like this:

	if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
		return; /* Forgot srcu_barrier(), so just leak it! */
Quoted 7 lines
Quoted 1 line

So this change already represents an improvement in usability.

Considering that we can do a srcu_barrier() for the specific domain, and that it should add no noticeable overhead if there is no queued callbacks, I don't see a good reason for leaving the srcu_barrier invocation to the user rather than implicitly doing it from the module going notifier.

Quoted 7 lines

Now, I could automatically add an indicator of whether or not a call_srcu() had happened, but then again, that would either add a call_srcu() scalability bottleneck or again require a scan of all possible CPUs... to figure out if it was necessary to scan all possible CPUs.

Or is scanning all possible CPUs down in the noise in this case? Or am I missing a trick that would reduce the overhead?

Module unloading implicitly does a synchronize_rcu (for RCU-sched), and a stop_machine. So I would be tempted to say that overhead of iteration over all CPUs might not matter that much considering the rest.

About notifying that a call_srcu has happened for the srcu domain in a scalable fashion, let's see... We could have a flag "call_srcu_used" for each call_srcu domain. Whenever call_srcu is invoked, it would load that flag. It sets it on first use.

The idea here is to only use that flag when srcu_barrier is performed right before the srcu domain cleanup (it could become part of that cleanup). Else, using it in all srcu_barrier() might be tricky, because we may then need to add memory barriers or locking to the call_srcu fast-path, which is an overhead we try to avoid.

Quoted 4 lines

However, if we only use that flag as part of the srcu domain cleanup, it's already prohibited to invoke call_srcu concurrently with the cleanup of the same domain, so I don't think we would need any memory barriers in call_srcu.

Quoted 5 lines

About the last part of your email, it seems to that if after call_srcu has returned, if the module could be unloaded on some other CPU - then it would need to see the flag stored by the preceding call_srcu, so I believe there would be a memory barrier between the two opreations (call_srcu and module unload).

In order for the module unload not to race against module execution, it needs to happen after the call_srcu in a way that is already ordered by other means, else module unload races against the module code.

Quoted 6 lines

Also about doing the unconditional srcu_barrier, since a module could be unloaded at any time - don't all SRCU using modules need to invoke srcu_barrier() during their clean up anyway so we are incurring the barrier overhead anyway? Or, am I missing a design pattern here? It seems to me rcutorture module definitely calls srcu_barrier() before it is unloaded.

Quoted 7 lines

I think a valid approach which is even simpler might be: if a module statically defines a SRCU domain, it should be expected to use it. So adding a srcu_barrier() to its module going notifier should not hurt. The rare case where a module defines a static SRCU domain *and* does not actually use it with call_srcu() does not seem that usual, and not worth optimizing for.

Thoughts ?

Most SRCU users use only synchronize_srcu(), and don't ever use call_srcu(). Which is not too surprising given that call_srcu() showed up late in the game.

But something still bothers me about this, and I am not yet sure what. One thing that seems to reduce anxiety somewhat is doing the srcu_barrier() on all calls to cleanup_srcu_struct() rather than just those invoked from the modules infrastructure, but I don't see why at the moment.

Thanx, Paul

Quoted 26 lines

Thanks,

Mathieu

Quoted 15 lines

thanks,

- Joel

Quoted 10 lines

Thoughts ?

Thanks,

Mathieu

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#41 lore raw
Quoted 208 lines · ----- On Apr 9, 2019, at 12:40 PM, paulmck…
Quoted 201 lines · Mathieu Desnoyers

----- On Apr 9, 2019, at 11:40 AM, Joel Fernandes, Google joel-QYYGw3jwrUn5owFQY34kdNi2O/[email protected]

Quoted 173 lines · wrote:
Quoted 167 lines · Mathieu Desnoyers
Quoted 142 lines · ----- On Apr 8, 2019, at 11:46 AM, paulmck…
Quoted 102 lines · Mathieu Desnoyers
Quoted 91 lines · ----- On Apr 8, 2019, at 10:22 AM, paulmck…
Quoted 83 lines · Mathieu Desnoyers
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck…
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google joel-QYYGw3jwrUn5owFQY34kdNi2O/[email protected]

Quoted 50 lines · wrote:
Quoted 46 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:

On Sun, Apr 07, 2019 at 06:39:41AM -0700, Paul E. McKenney wrote:
On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

diff --git a/include/asm-generic/vmlinux.lds.h
b/include/asm-generic/vmlinux.lds.h
index f8f6f04c4453..c2d919a1566e 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -338,6 +338,10 @@
KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
__stop___tracepoints_ptrs = .; \
*(__tracepoints_strings)/* Tracepoints: strings */ \
+ . = ALIGN(8); \
+ __start___srcu_struct = .; \
+ *(___srcu_struct_ptrs) \
+ __end___srcu_struct = .; \
} \

This vmlinux linker modification is not needed. I tested without it and srcu
torture works fine with rcutorture built as a module. Putting further prints
in kernel/module.c verified that the kernel is able to find the srcu structs
just fine. You could squash the below patch into this one or apply it on top
of the dev branch.

Good point, given that otherwise FORTRAN named common blocks would not
work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION()
macro that it can be mapped read-only? Or am I suffering from excessive
optimism?

And to answer the other question, in the case where I am suffering from
excessive optimism, it should be a separate commit. Please see below
for the updated original commit thus far.

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going
notifier end up acting as a barrier first before freeing the memory ?
If not, is it explicitly stated that a barrier must be issued before
module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

Quoted 4 lines

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

We do seem to be talking past each other. ;-)

Quoted 8 lines

This has nothing to do with the order of events at module-unload time.

So please let me try again.

If a given srcu_struct in a module never has call_srcu() invoked, there is no need to invoke rcu_barrier() at any time, whether at module-unload time or not. Adding rcu_barrier() in this case adds overhead and latency for no good reason.

Quoted 4 lines

Not if we invoke srcu_barrier() for that specific domain. If call_srcu was never invoked for a srcu domain, I don't see why srcu_barrier() should be more expensive than a simple check that the domain does not have any srcu work queued.

But that simple check does involve a cache miss for each possible CPU (not just each online CPU), so it is non-trivial, especially on large systems.

Quoted 8 lines
Quoted 3 lines

If a given srcu_struct in a module does have at least one call_srcu() invoked, it is already that module's responsibility to make sure that the code sticks around long enough for the callback to be invoked.

I understand that when users do explicit dynamic allocation/cleanup of srcu domains, they indeed need to take care of doing explicit srcu_barrier(). However, if they do static definition of srcu domains, it would be nice if we can handle the barriers under the hood.

All else being equal, of course. But...

Quoted 12 lines
Quoted 12 lines

This means that correct SRCU users that invoke call_srcu() already have srcu_barrier() at module-unload time. Incorrect SRCU users, with reasonable probability, now get a WARN_ON() at module-unload time, with the per-CPU state getting leaked. Before this change, they would (also with reasonable probability) instead get an instruction-fetch fault when the SRCU callback was invoked after the completion of the module unload. Furthermore, in all cases where they would previously have gotten the instruction-fetch fault, they now get the WARN_ON(), like this:

	if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
		return; /* Forgot srcu_barrier(), so just leak it! */
Quoted 7 lines
Quoted 1 line

So this change already represents an improvement in usability.

Considering that we can do a srcu_barrier() for the specific domain, and that it should add no noticeable overhead if there is no queued callbacks, I don't see a good reason for leaving the srcu_barrier invocation to the user rather than implicitly doing it from the module going notifier.

Quoted 7 lines

Now, I could automatically add an indicator of whether or not a call_srcu() had happened, but then again, that would either add a call_srcu() scalability bottleneck or again require a scan of all possible CPUs... to figure out if it was necessary to scan all possible CPUs.

Or is scanning all possible CPUs down in the noise in this case? Or am I missing a trick that would reduce the overhead?

Module unloading implicitly does a synchronize_rcu (for RCU-sched), and a stop_machine. So I would be tempted to say that overhead of iteration over all CPUs might not matter that much considering the rest.

About notifying that a call_srcu has happened for the srcu domain in a scalable fashion, let's see... We could have a flag "call_srcu_used" for each call_srcu domain. Whenever call_srcu is invoked, it would load that flag. It sets it on first use.

The idea here is to only use that flag when srcu_barrier is performed right before the srcu domain cleanup (it could become part of that cleanup). Else, using it in all srcu_barrier() might be tricky, because we may then need to add memory barriers or locking to the call_srcu fast-path, which is an overhead we try to avoid.

Quoted 4 lines

However, if we only use that flag as part of the srcu domain cleanup, it's already prohibited to invoke call_srcu concurrently with the cleanup of the same domain, so I don't think we would need any memory barriers in call_srcu.

Quoted 5 lines

About the last part of your email, it seems to that if after call_srcu has returned, if the module could be unloaded on some other CPU - then it would need to see the flag stored by the preceding call_srcu, so I believe there would be a memory barrier between the two opreations (call_srcu and module unload).

In order for the module unload not to race against module execution, it needs to happen after the call_srcu in a way that is already ordered by other means, else module unload races against the module code.

Quoted 6 lines

Also about doing the unconditional srcu_barrier, since a module could be unloaded at any time - don't all SRCU using modules need to invoke srcu_barrier() during their clean up anyway so we are incurring the barrier overhead anyway? Or, am I missing a design pattern here? It seems to me rcutorture module definitely calls srcu_barrier() before it is unloaded.

I think a valid approach which is even simpler might be: if a module statically defines a SRCU domain, it should be expected to use it. So adding a srcu_barrier()
to its module going notifier should not hurt. The rare case where a module defines
a static SRCU domain *and* does not actually use it with call_srcu() does not seem that usual, and not worth optimizing for.

Quoted 1 line

Thoughts ?

Most SRCU users use only synchronize_srcu(), and don't ever use call_srcu(). Which is not too surprising given that call_srcu() showed up late in the game.

Quoted 5 lines

But something still bothers me about this, and I am not yet sure what. One thing that seems to reduce anxiety somewhat is doing the srcu_barrier() on all calls to cleanup_srcu_struct() rather than just those invoked from the modules infrastructure, but I don't see why at the moment.

Indeed, providing similar guarantees for the dynamic allocation case would be nice.

The one thing that is making me anxious here is use-cases where users would decide to chain their call_srcu(). Then they would need as many srcu_barrier() as chain hops. This would be a valid reason for leaving invocation of srcu_barrier() to the user and not hide it under the hood.

Thoughts ?

Thanks,

Mathieu

Quoted 28 lines

Thanx, Paul

Quoted 25 lines

Thanks,

Mathieu

Quoted 15 lines

thanks,

- Joel

Quoted 10 lines

Thoughts ?

Thanks,

Mathieu

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#42 lore raw
Quoted 219 lines · Mathieu Desnoyers
Quoted 208 lines · ----- On Apr 9, 2019, at 12:40 PM, paulmck [email protected] wrote:
Quoted 201 lines · Mathieu Desnoyers

----- On Apr 9, 2019, at 11:40 AM, Joel Fernandes, Google [email protected]

Quoted 173 lines · wrote:
Quoted 167 lines · Mathieu Desnoyers
Quoted 142 lines · ----- On Apr 8, 2019, at 11:46 AM, paulmck [email protected] wrote:
Quoted 102 lines · Mathieu Desnoyers
Quoted 91 lines · ----- On Apr 8, 2019, at 10:22 AM, paulmck [email protected] wrote:
Quoted 83 lines · Mathieu Desnoyers
Quoted 72 lines · ----- On Apr 7, 2019, at 10:27 PM, paulmck [email protected] wrote:
Quoted 56 lines · Joel Fernandes
Quoted 55 lines · Mathieu Desnoyers

----- On Apr 7, 2019, at 3:32 PM, Joel Fernandes, Google [email protected]

Quoted 50 lines · wrote:

On Sun, Apr 07, 2019 at 03:26:16PM -0400, Mathieu Desnoyers wrote:
----- On Apr 7, 2019, at 9:59 AM, paulmck [email protected] wrote:

On Sun, Apr 07, 2019 at 06:39:41AM -0700, Paul E. McKenney wrote:
On Sat, Apr 06, 2019 at 07:06:13PM -0400, Joel Fernandes wrote:

[ . . . ]

diff --git a/include/asm-generic/vmlinux.lds.h
b/include/asm-generic/vmlinux.lds.h
index f8f6f04c4453..c2d919a1566e 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -338,6 +338,10 @@
KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
__stop___tracepoints_ptrs = .; \
*(__tracepoints_strings)/* Tracepoints: strings */ \
+ . = ALIGN(8); \
+ __start___srcu_struct = .; \
+ *(___srcu_struct_ptrs) \
+ __end___srcu_struct = .; \
} \

This vmlinux linker modification is not needed. I tested without it and srcu
torture works fine with rcutorture built as a module. Putting further prints
in kernel/module.c verified that the kernel is able to find the srcu structs
just fine. You could squash the below patch into this one or apply it on top
of the dev branch.

Good point, given that otherwise FORTRAN named common blocks would not
work.

But isn't one advantage of leaving that stuff in the RO_DATA_SECTION()
macro that it can be mapped read-only? Or am I suffering from excessive
optimism?

And to answer the other question, in the case where I am suffering from
excessive optimism, it should be a separate commit. Please see below
for the updated original commit thus far.

And may I have your Tested-by?

Just to confirm: does the cleanup performed in the modules going
notifier end up acting as a barrier first before freeing the memory ?
If not, is it explicitly stated that a barrier must be issued before
module unload ?

You mean rcu_barrier? It is mentioned in the documentation that this is the
responsibility of the module writer to prevent delays for all modules.

Quoted 7 lines
Quoted 4 lines

It's a srcu barrier yes. Considering it would be a barrier specific to the srcu domain within that module, I don't see how it would cause delays for "all" modules if we implicitly issue the barrier on module unload. What am I missing ?

Yes you are right. I thought of this after I just sent my email. I think it makes sense for srcu case to do and could avoid a class of bugs.

If there are call_srcu() callbacks outstanding, the module writer still needs the srcu_barrier() because otherwise callbacks arrive after the module text has gone, which will be disappoint the CPU when it tries fetching instructions that are no longer mapped. If there are no call_srcu() callbacks from that module, then there is no need for srcu_barrier() either way.

Quoted 2 lines

So if an srcu_barrier() is needed, the module developer needs to supply it.

When you say "callbacks arrive after the module text has gone", I think you assume that free_module() is invoked before the MODULE_STATE_GOING notifiers are called. But it's done in the opposite order: going notifiers are called first, and then free_module() is invoked.

Quoted 4 lines

So AFAIU it would be safe to issue the srcu_barrier() from the module going notifier.

Or am I missing something ?

We do seem to be talking past each other. ;-)

Quoted 8 lines

This has nothing to do with the order of events at module-unload time.

So please let me try again.

If a given srcu_struct in a module never has call_srcu() invoked, there is no need to invoke rcu_barrier() at any time, whether at module-unload time or not. Adding rcu_barrier() in this case adds overhead and latency for no good reason.

Quoted 4 lines

Not if we invoke srcu_barrier() for that specific domain. If call_srcu was never invoked for a srcu domain, I don't see why srcu_barrier() should be more expensive than a simple check that the domain does not have any srcu work queued.

But that simple check does involve a cache miss for each possible CPU (not just each online CPU), so it is non-trivial, especially on large systems.

Quoted 8 lines
Quoted 3 lines

If a given srcu_struct in a module does have at least one call_srcu() invoked, it is already that module's responsibility to make sure that the code sticks around long enough for the callback to be invoked.

I understand that when users do explicit dynamic allocation/cleanup of srcu domains, they indeed need to take care of doing explicit srcu_barrier(). However, if they do static definition of srcu domains, it would be nice if we can handle the barriers under the hood.

All else being equal, of course. But...

Quoted 12 lines
Quoted 12 lines

This means that correct SRCU users that invoke call_srcu() already have srcu_barrier() at module-unload time. Incorrect SRCU users, with reasonable probability, now get a WARN_ON() at module-unload time, with the per-CPU state getting leaked. Before this change, they would (also with reasonable probability) instead get an instruction-fetch fault when the SRCU callback was invoked after the completion of the module unload. Furthermore, in all cases where they would previously have gotten the instruction-fetch fault, they now get the WARN_ON(), like this:

	if (WARN_ON(rcu_segcblist_n_cbs(&sdp->srcu_cblist)))
		return; /* Forgot srcu_barrier(), so just leak it! */
Quoted 7 lines
Quoted 1 line

So this change already represents an improvement in usability.

Considering that we can do a srcu_barrier() for the specific domain, and that it should add no noticeable overhead if there is no queued callbacks, I don't see a good reason for leaving the srcu_barrier invocation to the user rather than implicitly doing it from the module going notifier.

Quoted 7 lines

Now, I could automatically add an indicator of whether or not a call_srcu() had happened, but then again, that would either add a call_srcu() scalability bottleneck or again require a scan of all possible CPUs... to figure out if it was necessary to scan all possible CPUs.

Or is scanning all possible CPUs down in the noise in this case? Or am I missing a trick that would reduce the overhead?

Module unloading implicitly does a synchronize_rcu (for RCU-sched), and a stop_machine. So I would be tempted to say that overhead of iteration over all CPUs might not matter that much considering the rest.

About notifying that a call_srcu has happened for the srcu domain in a scalable fashion, let's see... We could have a flag "call_srcu_used" for each call_srcu domain. Whenever call_srcu is invoked, it would load that flag. It sets it on first use.

The idea here is to only use that flag when srcu_barrier is performed right before the srcu domain cleanup (it could become part of that cleanup). Else, using it in all srcu_barrier() might be tricky, because we may then need to add memory barriers or locking to the call_srcu fast-path, which is an overhead we try to avoid.

Quoted 4 lines

However, if we only use that flag as part of the srcu domain cleanup, it's already prohibited to invoke call_srcu concurrently with the cleanup of the same domain, so I don't think we would need any memory barriers in call_srcu.

Quoted 5 lines

About the last part of your email, it seems to that if after call_srcu has returned, if the module could be unloaded on some other CPU - then it would need to see the flag stored by the preceding call_srcu, so I believe there would be a memory barrier between the two opreations (call_srcu and module unload).

In order for the module unload not to race against module execution, it needs to happen after the call_srcu in a way that is already ordered by other means, else module unload races against the module code.

Quoted 6 lines

Also about doing the unconditional srcu_barrier, since a module could be unloaded at any time - don't all SRCU using modules need to invoke srcu_barrier() during their clean up anyway so we are incurring the barrier overhead anyway? Or, am I missing a design pattern here? It seems to me rcutorture module definitely calls srcu_barrier() before it is unloaded.

I think a valid approach which is even simpler might be: if a module statically defines a SRCU domain, it should be expected to use it. So adding a srcu_barrier()
to its module going notifier should not hurt. The rare case where a module defines
a static SRCU domain *and* does not actually use it with call_srcu() does not seem that usual, and not worth optimizing for.

Quoted 1 line

Thoughts ?

Most SRCU users use only synchronize_srcu(), and don't ever use call_srcu(). Which is not too surprising given that call_srcu() showed up late in the game.

Quoted 5 lines

But something still bothers me about this, and I am not yet sure what. One thing that seems to reduce anxiety somewhat is doing the srcu_barrier() on all calls to cleanup_srcu_struct() rather than just those invoked from the modules infrastructure, but I don't see why at the moment.

Indeed, providing similar guarantees for the dynamic allocation case would be nice.

Quoted 7 lines

The one thing that is making me anxious here is use-cases where users would decide to chain their call_srcu(). Then they would need as many srcu_barrier() as chain hops. This would be a valid reason for leaving invocation of srcu_barrier() to the user and not hide it under the hood.

Thoughts ?

The current state is not horrible, so my thought would be to give it some time to see if better thoughts arise.

Either way, cleanup_srcu_struct() keeps its current checks for callbacks still being in flight, which is why I believe that the current state is not horrible. ;-)

Thanx, Paul

#43 lore raw

----- On Apr 9, 2019, at 1:55 PM, paulmck [email protected] wrote:
[...]

Quoted 6 lines

The current state is not horrible, so my thought would be to give it some time to see if better thoughts arise.

Either way, cleanup_srcu_struct() keeps its current checks for callbacks still being in flight, which is why I believe that the current state is not horrible. ;-)

In that case, I think the comment above cleanup_srcu_struct_quiesced() in include/linux/srcu.h needs to be updated to cover situations where API users statically define a SRCU domain in a module and intend to unload that module.

Given that we end up doing the allocation/cleanup under the hood, the API users don't interact with init_srcu_struct() nor cleanup_srcu_struct(), so it's not obvious that this comment also applies to them.

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#44 lore raw
Quoted 9 lines · Mathieu Desnoyers

----- On Apr 9, 2019, at 1:55 PM, paulmck [email protected] wrote:
[...]

Quoted 6 lines

The current state is not horrible, so my thought would be to give it some time to see if better thoughts arise.

Either way, cleanup_srcu_struct() keeps its current checks for callbacks still being in flight, which is why I believe that the current state is not horrible. ;-)

Quoted 8 lines

In that case, I think the comment above cleanup_srcu_struct_quiesced() in include/linux/srcu.h needs to be updated to cover situations where API users statically define a SRCU domain in a module and intend to unload that module.

Given that we end up doing the allocation/cleanup under the hood, the API users don't interact with init_srcu_struct() nor cleanup_srcu_struct(), so it's not obvious that this comment also applies to them.

Actually, it turned out that cleanup_srcu_struct_quiesced() is extremely hard to use correctly, and maybe even impossible to use correctly. So cleanup_srcu_struct_quiesced has been eliminated in current -rcu.

Thanx, Paul

Quoted 9 lines

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com