Skip to contentSkip to conversation

SRCU updates for v5.3

PATCH New Paul E. McKenney 1 participant 5 messages May 30, 2019

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover SRCU updates for v5.3

Hello!

This series provides SRCU updates:

1. Allocate per-CPU data for DEFINE_SRCU() in modules.

2. Remove unused vmlinux srcu linker entries, courtesy of Joel Fernandes.

3. Make srcu_struct ptr array as read-only, courtesy of Joel Fernandes.

4. Make __call_srcu static, courtesy of Jiang Biao.

Thanx, Paul

------------------------------------------------------------------------

 include/asm-generic/vmlinux.lds.h |    8 ++--
 include/linux/module.h            |    5 ++
 include/linux/srcutree.h          |   16 ++++++--
 kernel/module.c                   |    5 ++
 kernel/rcu/srcutree.c             |   69 ++++++++++++++++++++++++++++++++++++--
 5 files changed, 93 insertions(+), 10 deletions(-)
#3 lore raw

PATCH 3/4 module: Make srcu_struct ptr array as read-only

From: "Joel Fernandes (Google)" <[email protected]>

Since commit title ("srcu: Allocate per-CPU data for DEFINE_SRCU() in modules"), modules that call DEFINE_{STATIC,}SRCU will have a new array of srcu_struct pointers, which is used by srcu code to initialize and clean up these structures and save valuable per-cpu reserved space.

There is no reason for this array of pointers to be writable, and can cause security or other hidden bugs. Mark these are read-only after the module init has completed.

Tested with the following diff to ensure array not writable:

(diff is a bit reduced to avoid patch command getting confused)
a/kernel/module.c
b/kernel/module.c
-3506,6 +3506,14 static noinline int do_init_module [snip]
rcu_assign_pointer(mod->kallsyms, &mod->core_kallsyms);
#endif
module_enable_ro(mod, true);
+
+ if (mod->srcu_struct_ptrs) {
+ // Check if srcu_struct_ptrs access is possible
+ char x = *(char *)mod->srcu_struct_ptrs;
+ *(char *)mod->srcu_struct_ptrs = 0;
+ *(char *)mod->srcu_struct_ptrs = x;
+ }
+
mod_tree_remove_init(mod); disable_ro_nx(&mod->init_layout); module_arch_freeing_init(mod);

---
 include/linux/srcutree.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/srcutree.h +1 -1
index 8af1824c46a8..9cfcc8a756ae 100644
@@ -123,7 +123,7 @@ struct srcu_struct {
 #ifdef MODULE
 # define __DEFINE_SRCU(name, is_static)					\
 	is_static struct srcu_struct name;				\
-	struct srcu_struct *__srcu_struct_##name			\
+	struct srcu_struct * const __srcu_struct_##name			\
 		__section("___srcu_struct_ptrs") = &name
 #else
 # define __DEFINE_SRCU(name, is_static)					\
Signature · 1 line
2.17.1
#4 lore raw

PATCH 4/4 rcu: Make __call_srcu static

In reply to Paul E. McKenney

From: Jiang Biao <[email protected]>

Because __call_srcu() is not used outside kernel/rcu/srcutree.c, this commit makes it static.

---
 kernel/rcu/srcutree.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/rcu/srcutree.c +2 -2
index 2ded2614a2f4..cf0e886314f2 100644
@@ -831,8 +831,8 @@ static void srcu_leak_callback(struct rcu_head *rhp)
  * srcu_read_lock(), and srcu_read_unlock() that are all passed the same
  * srcu_struct structure.
  */
-void __call_srcu(struct srcu_struct *ssp, struct rcu_head *rhp,
-		 rcu_callback_t func, bool do_norm)
+static void __call_srcu(struct srcu_struct *ssp, struct rcu_head *rhp,
+			rcu_callback_t func, bool do_norm)
 {
 	unsigned long flags;
 	int idx;
Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/4 srcu: Remove unused vmlinux srcu linker entries

In reply to Paul E. McKenney

From: "Joel Fernandes (Google)" <[email protected]>

The SRCU for modules optimization (commit title "srcu: Allocate per-CPU data for DEFINE_SRCU() in modules") introduced vmlinux linker entries which is unused since it applies only to the built-in vmlinux. So remove it to prevent any space usage due to the 8 byte alignment it added. vmlinux.lds.h has no effect on module loading and is not used for building the module object, so the changes were not needed in the first place since the optimization is specific to modules.

Tested with SRCU torture_type and rcutorture. Put prints in module loader to confirm it is able to find and initialize the srcu structures.

---
 include/asm-generic/vmlinux.lds.h | 4 ----
 1 file changed, 4 deletions(-)
include/asm-generic/vmlinux.lds.h +0 -4
index ba1ad39468fc..088987e9a3ea 100644
@@ -337,10 +337,6 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
-		. = ALIGN(8);						\
-		__start___srcu_struct = .;				\
-		*(___srcu_struct_ptrs)					\
-		__end___srcu_struct = .;				\
 	}								\
 									\
 	.rodata1          : AT(ADDR(.rodata1) - LOAD_OFFSET) {		\
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/4 srcu: Allocate per-CPU data for DEFINE_SRCU() in modules

In reply to Paul E. McKenney

Adding DEFINE_SRCU() or DEFINE_STATIC_SRCU() to a loadable module requires that the size of the reserved region be increased, which is not something we want to be doing all that often. One approach would be to require that loadable modules define an srcu_struct and invoke init_srcu_struct() from their module_init function and cleanup_srcu_struct() from their module_exit function. However, this is more than a bit user unfriendly.

This commit therefore creates an ___srcu_struct_ptrs linker section, and pointers to srcu_struct structures created by DEFINE_SRCU() and DEFINE_STATIC_SRCU() within a module are placed into that module's ___srcu_struct_ptrs section. The required init_srcu_struct() and cleanup_srcu_struct() functions are then automatically invoked as needed when that module is loaded and unloaded, thus allowing modules to continue to use DEFINE_SRCU() and DEFINE_STATIC_SRCU() while avoiding the need to increase the size of the reserved region.

Many of the algorithms and some of the code was cheerfully cherry-picked from other code making use of linker sections, perhaps most notably from tracepoints. All bugs are nevertheless the sole property of the author.

[ paulmck: Use __section() and use "default" in srcu_module_notify()'s
"switch" statement as suggested by Joel Fernandes. ]

---
 include/asm-generic/vmlinux.lds.h |  4 ++
 include/linux/module.h            |  5 +++
 include/linux/srcutree.h          | 14 +++++--
 kernel/module.c                   |  5 +++
 kernel/rcu/srcutree.c             | 65 +++++++++++++++++++++++++++++++
 5 files changed, 90 insertions(+), 3 deletions(-)
include/asm-generic/vmlinux.lds.h +4 -0
index 088987e9a3ea..ba1ad39468fc 100644
@@ -337,6 +337,10 @@
 		KEEP(*(__tracepoints_ptrs)) /* Tracepoints: pointer array */ \
 		__stop___tracepoints_ptrs = .;				\
 		*(__tracepoints_strings)/* Tracepoints: strings */	\
+		. = ALIGN(8);						\
+		__start___srcu_struct = .;				\
+		*(___srcu_struct_ptrs)					\
+		__end___srcu_struct = .;				\
 	}								\
 									\
 	.rodata1          : AT(ADDR(.rodata1) - LOAD_OFFSET) {		\
include/linux/module.h +5 -0
index 188998d3dca9..1455812dd325 100644
@@ -21,6 +21,7 @@
 #include <linux/rbtree_latch.h>
 #include <linux/error-injection.h>
 #include <linux/tracepoint-defs.h>
+#include <linux/srcu.h>
 
 #include <linux/percpu.h>
 #include <asm/module.h>
@@ -450,6 +451,10 @@ struct module {
 	unsigned int num_tracepoints;
 	tracepoint_ptr_t *tracepoints_ptrs;
 #endif
+#ifdef CONFIG_TREE_SRCU
+	unsigned int num_srcu_structs;
+	struct srcu_struct **srcu_struct_ptrs;
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	unsigned int num_bpf_raw_events;
 	struct bpf_raw_event_map *bpf_raw_events;
include/linux/srcutree.h +11 -3
index 7f7c8c050f63..8af1824c46a8 100644
@@ -120,9 +120,17 @@ struct srcu_struct {
  *
  * See include/linux/percpu-defs.h for the rules on per-CPU variables.
  */
-#define __DEFINE_SRCU(name, is_static)					\
-	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);\
-	is_static struct srcu_struct name = __SRCU_STRUCT_INIT(name, name##_srcu_data)
+#ifdef MODULE
+# define __DEFINE_SRCU(name, is_static)					\
+	is_static struct srcu_struct name;				\
+	struct srcu_struct *__srcu_struct_##name			\
+		__section("___srcu_struct_ptrs") = &name
+#else
+# define __DEFINE_SRCU(name, is_static)					\
+	static DEFINE_PER_CPU(struct srcu_data, name##_srcu_data);	\
+	is_static struct srcu_struct name =				\
+		__SRCU_STRUCT_INIT(name, name##_srcu_data)
+#endif
 #define DEFINE_SRCU(name)		__DEFINE_SRCU(name, /* not static */)
 #define DEFINE_STATIC_SRCU(name)	__DEFINE_SRCU(name, static)
 
kernel/module.c +5 -0
index 6e6712b3aaf5..c79a53b629b6 100644
@@ -3095,6 +3095,11 @@ static int find_module_sections(struct module *mod, struct load_info *info)
 					     sizeof(*mod->tracepoints_ptrs),
 					     &mod->num_tracepoints);
 #endif
+#ifdef CONFIG_TREE_SRCU
+	mod->srcu_struct_ptrs = section_objs(info, "___srcu_struct_ptrs",
+					     sizeof(*mod->srcu_struct_ptrs),
+					     &mod->num_srcu_structs);
+#endif
 #ifdef CONFIG_BPF_EVENTS
 	mod->bpf_raw_events = section_objs(info, "__bpf_raw_tp_map",
 					   sizeof(*mod->bpf_raw_events),
kernel/rcu/srcutree.c +65 -0

68 lines not on this page · raw message

Signature · 1 line
2.17.1