Skip to contentSkip to conversation

Programmatic nestable expedited grace periods

PATCH In review Paul E. McKenney 5 participants 29 messages Feb 20, 2015 – Feb 22, 2015

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Programmatic nestable expedited grace periods

Hello!

This series, possibly for v3.21, contains changes that allow in-kernel code to specify that all subsequent synchronous grace-period primitives (synchronize_rcu() and friends) be expedited. New rcu_expedite_gp() and rcu_unexpedite_gp() primitives enable and disable expediting, and these may be nested. Note that the rcu_expedited boot/sysfs variable, if non-zero, causes expediting to happen regardless of calls to rcu_expedite_gp().

Because one of the use cases for these primitives is to expedite grace periods during the in-kernel portion of boot, a new Kconfig parameter named CONFIG_RCU_EXPEDITE_BOOT causes the kernel to act as if rcu_expedite_gp() was called very early in boot. At the end of boot (presumably just before init is spawned), a call to rcu_end_inkernel_boot() will provide the matching rcu_unexpedite_gp() if required.

The patches in this series are as follows:

1. Add rcu_expedite_gp() and rcu_unexpedite_gp() functions.

2. Add rcutorture tests for rcu_expedite_gp() and rcu_unexpedite_gp().

3. Change open-coded access to the rcu_expedited variable to
instead use a new rcu_gp_is_expedited() function.

4. Add the CONFIG_RCU_EXPEDITE_BOOT Kconfig parameter and
the rcu_end_inkernel_boot() function.

This passes light rcutorture testing.

Thanx, Paul

------------------------------------------------------------------------

 b/include/linux/rcupdate.h |   21 ++++++++++++++++
 b/init/Kconfig             |   13 +++++++++
 b/kernel/rcu/rcutorture.c  |   24 ++++++++++++++++++
 b/kernel/rcu/srcu.c        |    2 -
 b/kernel/rcu/tree.c        |    9 +++---
 b/kernel/rcu/tree_plugin.h |    2 -
 b/kernel/rcu/update.c      |   59 ++++++++++++++++++++++++++++++++++++++++++++-
 7 files changed, 123 insertions(+), 7 deletions(-)
#4 lore raw

RFC PATCH v2 4/4 rcu: Add Kconfig option to expedite grace periods during boot

In reply to Paul E. McKenney

From: "Paul E. McKenney" <[email protected]>

This commit adds a CONFIG_RCU_EXPEDITE_BOOT Kconfig parameter that emulates a very early boot rcu_expedite_gp(). A late-boot call to rcu_end_inkernel_boot() will provide the corresponding rcu_unexpedite_gp(). The late-boot call to rcu_end_inkernel_boot() should be made just before init is spawned.

---
 include/linux/rcupdate.h |  1 +
 init/Kconfig             | 13 +++++++++++++
 kernel/rcu/update.c      | 11 ++++++++++-
 3 files changed, 24 insertions(+), 1 deletion(-)
include/linux/rcupdate.h +1 -0
index 0189ac088c67..573a5afd5ed8 100644
@@ -287,6 +287,7 @@ static inline int rcu_preempt_depth(void)
 
 /* Internal to kernel */
 void rcu_init(void);
+void rcu_end_inkernel_boot(void);
 void rcu_sched_qs(void);
 void rcu_bh_qs(void);
 void rcu_check_callbacks(int user);
init/Kconfig +13 -0
index 1354ac09b516..ece7e9380867 100644
@@ -791,6 +791,19 @@ config RCU_NOCB_CPU_ALL
 
 endchoice
 
+config RCU_EXPEDITE_BOOT
+	bool
+	default n
+	help
+	  This option enables expedited grace periods at boot time,
+	  as if rcu_expedite_gp() had been invoked early in boot.
+	  The corresponding rcu_unexpedite_gp() is invoked from
+	  rcu_end_inkernel_boot(), which is intended to be invoked
+	  at the end of the kernel-only boot sequence, just before
+	  init is exec'ed.
+
+	  Accept the default if unsure.
+
 endmenu # "RCU Subsystem"
 
 config BUILD_BIN2C
kernel/rcu/update.c +10 -1
index 44a74c24936a..1f133350da01 100644
@@ -64,7 +64,8 @@ module_param(rcu_expedited, int, 0);
 
 #ifndef CONFIG_TINY_RCU
 
-static atomic_t rcu_expedited_nesting;
+static atomic_t rcu_expedited_nesting =
+	ATOMIC_INIT(IS_ENABLED(CONFIG_RCU_EXPEDITE_BOOT) ? 1 : 0);
 
 /*
  * Should normal grace-period primitives be expedited?  Intended for
@@ -109,6 +110,14 @@ EXPORT_SYMBOL_GPL(rcu_unexpedite_gp);
 
 #endif /* #ifndef CONFIG_TINY_RCU */
 
+/*
+ * Inform RCU of the end of the in-kernel boot sequence.
+ */
+void rcu_end_inkernel_boot(void)
+{
+	if (IS_ENABLED(CONFIG_RCU_EXPEDITE_BOOT))
+		rcu_unexpedite_gp();
+}
 
 #ifdef CONFIG_PREEMPT_RCU
 
Signature · 1 line
1.8.1.5
#3 lore raw

RFC PATCH v2 3/4 rcu: Update from rcu_expedited variable to rcu_gp_is_expedited()

In reply to Paul E. McKenney

From: "Paul E. McKenney" <[email protected]>

This commit updates open-coded tests of the rcu_expedited variable to instead use rcu_gp_is_expedited().

---
 kernel/rcu/srcu.c        | 2 +-
 kernel/rcu/tree.c        | 9 +++++----
 kernel/rcu/tree_plugin.h | 2 +-
 3 files changed, 7 insertions(+), 6 deletions(-)
kernel/rcu/srcu.c +1 -1
index 81f53b504c18..cad76e76b4e7 100644
@@ -490,7 +490,7 @@ static void __synchronize_srcu(struct srcu_struct *sp, int trycount)
  */
 void synchronize_srcu(struct srcu_struct *sp)
 {
-	__synchronize_srcu(sp, rcu_expedited
+	__synchronize_srcu(sp, rcu_gp_is_expedited()
 			   ? SYNCHRONIZE_SRCU_EXP_TRYCOUNT
 			   : SYNCHRONIZE_SRCU_TRYCOUNT);
 }
kernel/rcu/tree.c +5 -4
index 715e7bb35852..5fb9888af6d5 100644
@@ -3064,7 +3064,7 @@ void synchronize_sched(void)
 			   "Illegal synchronize_sched() in RCU-sched read-side critical section");
 	if (rcu_blocking_is_gp())
 		return;
-	if (rcu_expedited)
+	if (rcu_gp_is_expedited())
 		synchronize_sched_expedited();
 	else
 		wait_rcu_gp(call_rcu_sched);
@@ -3091,7 +3091,7 @@ void synchronize_rcu_bh(void)
 			   "Illegal synchronize_rcu_bh() in RCU-bh read-side critical section");
 	if (rcu_blocking_is_gp())
 		return;
-	if (rcu_expedited)
+	if (rcu_gp_is_expedited())
 		synchronize_rcu_bh_expedited();
 	else
 		wait_rcu_gp(call_rcu_bh);
@@ -3783,11 +3783,12 @@ static int rcu_pm_notify(struct notifier_block *self,
 	case PM_HIBERNATION_PREPARE:
 	case PM_SUSPEND_PREPARE:
 		if (nr_cpu_ids <= 256) /* Expediting bad for large systems. */
-			rcu_expedited = 1;
+			rcu_expedite_gp();
 		break;
 	case PM_POST_HIBERNATION:
 	case PM_POST_SUSPEND:
-		rcu_expedited = 0;
+		if (nr_cpu_ids <= 256) /* Expediting bad for large systems. */
+			rcu_unexpedite_gp();
 		break;
 	default:
 		break;
kernel/rcu/tree_plugin.h +1 -1
index 65629cc3c6cf..d11612ff3daa 100644
@@ -572,7 +572,7 @@ void synchronize_rcu(void)
 			   "Illegal synchronize_rcu() in RCU read-side critical section");
 	if (!rcu_scheduler_active)
 		return;
-	if (rcu_expedited)
+	if (rcu_gp_is_expedited())
 		synchronize_rcu_expedited();
 	else
 		wait_rcu_gp(call_rcu);
Signature · 1 line
1.8.1.5
#1 lore raw

RFC PATCH v2 1/4 rcu: Provide rcu_expedite_gp() and rcu_unexpedite_gp()

In reply to Paul E. McKenney

From: "Paul E. McKenney" <[email protected]>

Currently, expediting of normal synchronous grace-period primitives (synchronize_rcu() and friends) is controlled by the rcu_expedited() boot/sysfs parameter. This works well, but does not handle nesting. This commit therefore provides rcu_expedite_gp() to enable expediting and rcu_unexpedite_gp() to cancel a prior rcu_expedite_gp(), both of which support nesting.

---
 include/linux/rcupdate.h | 20 ++++++++++++++++++++
 kernel/rcu/update.c      | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 68 insertions(+)
include/linux/rcupdate.h +20 -0
index 9d3d0e1d0766..0189ac088c67 100644
@@ -48,6 +48,26 @@
 
 extern int rcu_expedited; /* for sysctl */
 
+#ifdef CONFIG_TINY_RCU
+/* Tiny RCU doesn't expedite, as its purpose in life is instead to be tiny. */
+static inline bool rcu_gp_is_expedited(void)  /* Internal RCU use. */
+{
+	return false;
+}
+
+static inline void rcu_expedite_gp(void)
+{
+}
+
+static inline void rcu_unexpedite_gp(void)
+{
+}
+#else /* #ifdef CONFIG_TINY_RCU */
+bool rcu_gp_is_expedited(void);  /* Internal RCU use. */
+void rcu_expedite_gp(void);
+void rcu_unexpedite_gp(void);
+#endif /* #else #ifdef CONFIG_TINY_RCU */
+
 enum rcutorture_type {
 	RCU_FLAVOR,
 	RCU_BH_FLAVOR,
kernel/rcu/update.c +48 -0

54 lines not on this page · raw message

Signature · 1 line
1.8.1.5
#2 lore raw

RFC PATCH v2 2/4 rcu: Add rcu_expedite_gp() and rcu_unexpedite_gp() to rcutorture

From: "Paul E. McKenney" <[email protected]>

---
 kernel/rcu/rcutorture.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
kernel/rcu/rcutorture.c +24 -0
index 30d42aa55d83..3a60815ce97c 100644
@@ -853,6 +853,8 @@ rcu_torture_fqs(void *arg)
 static int
 rcu_torture_writer(void *arg)
 {
+	bool can_expedite = !rcu_gp_is_expedited();
+	int expediting = 0;
 	unsigned long gp_snap;
 	bool gp_cond1 = gp_cond, gp_exp1 = gp_exp, gp_normal1 = gp_normal;
 	bool gp_sync1 = gp_sync;
@@ -865,6 +867,12 @@ rcu_torture_writer(void *arg)
 	int nsynctypes = 0;
 
 	VERBOSE_TOROUT_STRING("rcu_torture_writer task started");
+	pr_alert("%s" TORTURE_FLAG
+		 " Grace periods expedited from boot/sysfs for %s,\n",
+		 torture_type, cur_ops->name);
+	pr_alert("%s" TORTURE_FLAG
+		 " Testing of dynamic grace-period expediting diabled.\n",
+		 torture_type);
 
 	/* Initialize synctype[] array.  If none set, take default. */
 	if (!gp_cond1 && !gp_exp1 && !gp_normal1 && !gp_sync)
@@ -949,9 +957,25 @@ rcu_torture_writer(void *arg)
 			}
 		}
 		rcutorture_record_progress(++rcu_torture_current_version);
+		/* Cycle through nesting levels of rcu_expedite_gp() calls. */
+		if (can_expedite && torture_random(&rand)) {
+			WARN_ON_ONCE(expediting == 0 && rcu_gp_is_expedited());
+			if (expediting >= 0)
+				rcu_expedite_gp();
+			else
+				rcu_unexpedite_gp();
+			if (++expediting > 3)
+				expediting = -expediting;
+		}
 		rcu_torture_writer_state = RTWS_STUTTER;
 		stutter_wait("rcu_torture_writer");
 	} while (!torture_must_stop());
+	/* Reset expediting back to unexpedited. */
+	if (expediting > 0)
+		expediting = -expediting;
+	while (can_expedite && expediting++ < 0)
+		rcu_unexpedite_gp();
+	WARN_ON_ONCE(can_expedite && rcu_gp_is_expedited());
 	rcu_torture_writer_state = RTWS_STOPPING;
 	torture_kthread_stopping("rcu_torture_writer");
 	return 0;
Signature · 1 line
1.8.1.5
#5 lore raw
In reply to Paul E. McKenney
Quoted 10 lines · Paul E. McKenney

Hello!

This series, possibly for v3.21, contains changes that allow in-kernel code to specify that all subsequent synchronous grace-period primitives (synchronize_rcu() and friends) be expedited. New rcu_expedite_gp() and rcu_unexpedite_gp() primitives enable and disable expediting, and these may be nested. Note that the rcu_expedited boot/sysfs variable, if non-zero, causes expediting to happen regardless of calls to rcu_expedite_gp().

Quoted 7 lines

Because one of the use cases for these primitives is to expedite grace periods during the in-kernel portion of boot, a new Kconfig parameter named CONFIG_RCU_EXPEDITE_BOOT causes the kernel to act as if rcu_expedite_gp() was called very early in boot. At the end of boot (presumably just before init is spawned), a call to rcu_end_inkernel_boot() will provide the matching rcu_unexpedite_gp() if required.

So I though we wanted to get rid / limit the expedited stuff because its IPI happy, and here its spreading.

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

#6 lore raw
Quoted 19 lines · Peter Zijlstra
Quoted 10 lines · Paul E. McKenney

Hello!

This series, possibly for v3.21, contains changes that allow in-kernel code to specify that all subsequent synchronous grace-period primitives (synchronize_rcu() and friends) be expedited. New rcu_expedite_gp() and rcu_unexpedite_gp() primitives enable and disable expediting, and these may be nested. Note that the rcu_expedited boot/sysfs variable, if non-zero, causes expediting to happen regardless of calls to rcu_expedite_gp().

Quoted 7 lines

Because one of the use cases for these primitives is to expedite grace periods during the in-kernel portion of boot, a new Kconfig parameter named CONFIG_RCU_EXPEDITE_BOOT causes the kernel to act as if rcu_expedite_gp() was called very early in boot. At the end of boot (presumably just before init is spawned), a call to rcu_end_inkernel_boot() will provide the matching rcu_unexpedite_gp() if required.

Quoted 2 lines

So I though we wanted to get rid / limit the expedited stuff because its IPI happy, and here its spreading.

Well, at least it no longer IPIs idle CPUs. ;-)

And this is during boot, when a few extra IPIs should not be a big deal.

Quoted 3 lines

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

The report I heard was that it provided 10-15% faster boot times.

Thanx, Paul

#7 lore raw
Quoted 7 lines · Paul E. McKenney
Quoted 2 lines · Peter Zijlstra

So I though we wanted to get rid / limit the expedited stuff because its IPI happy, and here its spreading.

Well, at least it no longer IPIs idle CPUs. ;-)

And this is during boot, when a few extra IPIs should not be a big deal.

Well the one application now is during boot; but you expose the interface for all to use, and therefore someone will.

Quoted 5 lines
Quoted 3 lines

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

The report I heard was that it provided 10-15% faster boot times.

That's not insignificant; got more details? I think we should really look at why people are using the sync primitives.

#8 lore raw
Quoted 11 lines · Peter Zijlstra
Quoted 7 lines · Paul E. McKenney
Quoted 2 lines · Peter Zijlstra

So I though we wanted to get rid / limit the expedited stuff because its IPI happy, and here its spreading.

Well, at least it no longer IPIs idle CPUs. ;-)

And this is during boot, when a few extra IPIs should not be a big deal.

Well the one application now is during boot; but you expose the interface for all to use, and therefore someone will.

I could make rcu_expedite_gp() and rcu_unexpedite_gp() be static, I suppose. Except that I need to test them with rcutorture. I suppose I could put the declaration in rcutorture.c, but then 0day will tell me to made them static. :-/

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

The report I heard was that it provided 10-15% faster boot times.

That's not insignificant; got more details? I think we should really look at why people are using the sync primitives.

I must defer to the people who took the exact measurements.

But yes, once I have that info, I should add it to the commit log.

Thanx, Paul

#9 lore raw
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

The report I heard was that it provided 10-15% faster boot times.

That's not insignificant; got more details? I think we should really look at why people are using the sync primitives.

I must defer to the people who took the exact measurements.

But yes, once I have that info, I should add it to the commit log.

so the two most obvious cases are

Registering sysrq keys ... even when the old key code had no handler (have a patch pending for this)

registering idle handlers
(this is more tricky, it's very obvious abuse but the fix is less clear)

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

#10 lore raw
Quoted 2 lines · Arjan van de Ven

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

So my objections are twofold:

- I object to fast expedites in principle; they spray IPIs across the system, so ideally we'd not have them at all, therefore also not at boot.

Because as soon as the option exists, people will use it for other things too.

- The proposed interface is very much exposed to everybody who wants it; this again is wide open to (ab)use.

Once it exists people will start to use, and before you know it we'll always have that fast counter incremented and we're in IPI hell. Most likely because someone was lazy and it seemed like a quick fix for some stupid code.

And esp. in bootup code you can special case a lot of stuff; there's limited concurrency esp. because userspace it not there yet. So we might not actually need those sync calls.

#11 lore raw
Quoted 4 lines · Peter Zijlstra
Quoted 2 lines · Arjan van de Ven

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

Quoted 8 lines

So my objections are twofold:

- I object to fast expedites in principle; they spray IPIs across the system, so ideally we'd not have them at all, therefore also not at boot.

Because as soon as the option exists, people will use it for other things too.

the option exists today in sysfs and kernel parameter...

Quoted 4 lines

And esp. in bootup code you can special case a lot of stuff; there's limited concurrency esp. because userspace it not there yet. So we might not actually need those sync calls.

yeah I am going down that angle as well absolutely.
but there are cases that may well be legit (or are 5 function calls deep into common code)

#17 lore raw
In reply to Arjan van de Ven
Quoted 15 lines · Arjan van de Ven
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

The report I heard was that it provided 10-15% faster boot times.

That's not insignificant; got more details? I think we should really look at why people are using the sync primitives.

I must defer to the people who took the exact measurements.

But yes, once I have that info, I should add it to the commit log.

so the two most obvious cases are

Quoted 8 lines

Registering sysrq keys ... even when the old key code had no handler (have a patch pending for this)

registering idle handlers
(this is more tricky, it's very obvious abuse but the fix is less clear)

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

It would be good to have before-and-after measurements of actual boot time. Are these numbers available?

Thanx, Paul

#18 lore raw
Quoted 19 lines · Paul E. McKenney
Quoted 18 lines · Arjan van de Ven
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

The report I heard was that it provided 10-15% faster boot times.

That's not insignificant; got more details? I think we should really look at why people are using the sync primitives.

I must defer to the people who took the exact measurements.

But yes, once I have that info, I should add it to the commit log.

so the two most obvious cases are

Registering sysrq keys ... even when the old key code had no handler (have a patch pending for this)

Quoted 8 lines
Quoted 5 lines

registering idle handlers
(this is more tricky, it's very obvious abuse but the fix is less clear)

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

It would be good to have before-and-after measurements of actual boot time. Are these numbers available?

I'll make you pretty graphs when I get home from collab summit, which should be later today

#14 lore raw
In reply to Peter Zijlstra
Quoted 9 lines · Peter Zijlstra
Quoted 2 lines · Arjan van de Ven

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

So my objections are twofold:

- I object to fast expedites in principle; they spray IPIs across the system, so ideally we'd not have them at all, therefore also not at boot.

There are only a few uses of expedited grace periods, despite their having been in the kernel for some years. So people do seem to be exercising appropriate restraint here.

Quoted 10 lines

Because as soon as the option exists, people will use it for other things too.

- The proposed interface is very much exposed to everybody who wants it; this again is wide open to (ab)use.

Once it exists people will start to use, and before you know it we'll always have that fast counter incremented and we're in IPI hell. Most likely because someone was lazy and it seemed like a quick fix for some stupid code.

I suppose that another way to keep it private would be to have the declaration in both update.c and rcutorture.c. This would mean that no other file could invoke it, and should keep 0day happy. It would mean that the declarations would be duplicated, but worse things could happen.

Quoted 3 lines

And esp. in bootup code you can special case a lot of stuff; there's limited concurrency esp. because userspace it not there yet. So we might not actually need those sync calls.

I expect that some could be rewritten, but it might not work well for code common to boot and to runtime.

Thanx, Paul

#19 lore raw
In reply to Arjan van de Ven
Quoted 23 lines · Arjan van de Ven
Quoted 22 lines · Paul E. McKenney
Quoted 21 lines · Arjan van de Ven
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

The report I heard was that it provided 10-15% faster boot times.

That's not insignificant; got more details? I think we should really look at why people are using the sync primitives.

I must defer to the people who took the exact measurements.

But yes, once I have that info, I should add it to the commit log.

so the two most obvious cases are

Registering sysrq keys ... even when the old key code had no handler (have a patch pending for this)

registering idle handlers
(this is more tricky, it's very obvious abuse but the fix is less clear)

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

It would be good to have before-and-after measurements of actual boot time. Are these numbers available?

I'll make you pretty graphs when I get home from collab summit, which should be later today

Very good, looking forward to seeing them. ;-)

Thanx, Paul

maintainer
#26 lore raw
In reply to Peter Zijlstra
Quoted 10 lines · Peter Zijlstra
Quoted 6 lines · Paul E. McKenney
Quoted 3 lines · Peter Zijlstra

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

The report I heard was that it provided 10-15% faster boot times.

That's not insignificant; got more details? I think we should really look at why people are using the sync primitives.

Paul, what do you think about adding a compile-time debug option to synchronize_rcu() that causes it to capture the time on entry and exit and print the duration together with the file:line of the caller? Similar to initcall_debug, but for blocking calls to synchronize_rcu(). Put that together with initcall_debug, and you'd have a pretty good idea of where that holds up boot.

We do want early boot to run as asynchronously as possible, and to avoid having later bits of boot waiting on a synchronize_rcu from earlier bits of boot. Switching a caller over to call_rcu() doesn't actually help if it still has to finish a grace period before it can allow later bits to run. Ideally, we ought to be able to work out the "depth" of boot in grace-periods.

Has anyone wired initcall_debug up to a bootchart-like graph?

- Josh Triplett

#27 lore raw

----- Original Message -----

Quoted 28 lines

From: "Josh Triplett" <[email protected]>
To: "Peter Zijlstra" <[email protected]>
Cc: "Paul E. McKenney" <[email protected]>, [email protected], [email protected], [email protected], [email protected], [email protected], "mathieu desnoyers" <[email protected]>, [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "bobby prani" <[email protected]>
Sent: Saturday, February 21, 2015 1:04:28 AM
Subject: Re: [PATCH tip/core/rcu 0/4] Programmatic nestable expedited grace periods

Quoted 10 lines · Peter Zijlstra
Quoted 6 lines · Paul E. McKenney
Quoted 3 lines · Peter Zijlstra

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

The report I heard was that it provided 10-15% faster boot times.

That's not insignificant; got more details? I think we should really look at why people are using the sync primitives.

Paul, what do you think about adding a compile-time debug option to synchronize_rcu() that causes it to capture the time on entry and exit and print the duration together with the file:line of the caller? Similar to initcall_debug, but for blocking calls to synchronize_rcu(). Put that together with initcall_debug, and you'd have a pretty good idea of where that holds up boot.

Quoted 8 lines

We do want early boot to run as asynchronously as possible, and to avoid having later bits of boot waiting on a synchronize_rcu from earlier bits of boot. Switching a caller over to call_rcu() doesn't actually help if it still has to finish a grace period before it can allow later bits to run. Ideally, we ought to be able to work out the "depth" of boot in grace-periods.

Has anyone wired initcall_debug up to a bootchart-like graph?

The information about begin/end of synchronize_rcu, as well as begin/end of rcu_barrier() seems to be very relevant here. This should perhaps be covered tracepoints ? Isn't it already ?

Thanks,

Mathieu

Quoted 3 lines

- Josh Triplett

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#20 lore raw
In reply to Paul E. McKenney
Quoted 6 lines
Quoted 3 lines

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

It would be good to have before-and-after measurements of actual boot time. Are these numbers available?

To show the boot time, I'm using the timestamp of the "Write protecting" line, that's pretty much the last thing we print prior to ring 3 execution.

A kernel with default RCU behavior (inside KVM, only virtual devices) looks like this:

[    0.038724] Write protecting the kernel read-only data: 10240k

a kernel with expedited RCU (using the command line option, so that I don't have to recompile between measurements and thus am completely oranges-to-oranges)

[    0.031768] Write protecting the kernel read-only data: 10240k

which, in percentage, is an 18% improvement.

#12 lore raw
In reply to Arjan van de Ven
Quoted 7 lines · Arjan van de Ven
Quoted 6 lines · Peter Zijlstra
Quoted 2 lines · Arjan van de Ven

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

So my objections are twofold:

Quoted 8 lines
Quoted 6 lines

- I object to fast expedites in principle; they spray IPIs across the system, so ideally we'd not have them at all, therefore also not at boot.

Because as soon as the option exists, people will use it for other things too.

the option exists today in sysfs and kernel parameter...

Yeah, Paul and me have been having this argument for a while now ;-)

Quoted 6 lines
Quoted 3 lines

And esp. in bootup code you can special case a lot of stuff; there's limited concurrency esp. because userspace it not there yet. So we might not actually need those sync calls.

yeah I am going down that angle as well absolutely.
but there are cases that may well be legit (or are 5 function calls deep into common code)

Good ;-)

#15 lore raw
In reply to Paul E. McKenney
Quoted 7 lines · Paul E. McKenney
Quoted 6 lines · Peter Zijlstra
Quoted 2 lines · Arjan van de Ven

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

So my objections are twofold:

Quoted 7 lines
Quoted 3 lines

- I object to fast expedites in principle; they spray IPIs across the system, so ideally we'd not have them at all, therefore also not at boot.

There are only a few uses of expedited grace periods, despite their having been in the kernel for some years. So people do seem to be exercising appropriate restraint here.

Or people just don't know about it :-)

Quoted 11 lines
Quoted 10 lines

Because as soon as the option exists, people will use it for other things too.

- The proposed interface is very much exposed to everybody who wants it; this again is wide open to (ab)use.

Once it exists people will start to use, and before you know it we'll always have that fast counter incremented and we're in IPI hell. Most likely because someone was lazy and it seemed like a quick fix for some stupid code.

Quoted 4 lines

I suppose that another way to keep it private would be to have the declaration in both update.c and rcutorture.c. This would mean that no other file could invoke it, and should keep 0day happy. It would mean that the declarations would be duplicated, but worse things could happen.

Why do you need it for rcu torture? That can call the regular expedited call to exercise those rcu paths, right?

That would allow you to use system_state < SYSTEM_RUNNING if you really wanted to do this without exposing any interface for this.

#16 lore raw
Quoted 12 lines · Peter Zijlstra
Quoted 11 lines · Paul E. McKenney
Quoted 9 lines · Peter Zijlstra
Quoted 2 lines · Arjan van de Ven

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

So my objections are twofold:

- I object to fast expedites in principle; they spray IPIs across the system, so ideally we'd not have them at all, therefore also not at boot.

Quoted 5 lines
Quoted 3 lines

There are only a few uses of expedited grace periods, despite their having been in the kernel for some years. So people do seem to be exercising appropriate restraint here.

Or people just don't know about it :-)

"Ignorance: The #1 contributor to appropriate restraint!" ;-)

Quoted 11 lines
Quoted 11 lines
Quoted 10 lines

Because as soon as the option exists, people will use it for other things too.

- The proposed interface is very much exposed to everybody who wants it; this again is wide open to (ab)use.

Once it exists people will start to use, and before you know it we'll always have that fast counter incremented and we're in IPI hell. Most likely because someone was lazy and it seemed like a quick fix for some stupid code.

Quoted 7 lines
Quoted 4 lines

I suppose that another way to keep it private would be to have the declaration in both update.c and rcutorture.c. This would mean that no other file could invoke it, and should keep 0day happy. It would mean that the declarations would be duplicated, but worse things could happen.

Why do you need it for rcu torture? That can call the regular expedited call to exercise those rcu paths, right?

Yes, but not the ability to turn expediting on and off in the normal path.

Quoted 2 lines

That would allow you to use system_state < SYSTEM_RUNNING if you really wanted to do this without exposing any interface for this.

This decision is not up to RCU. Something else must tell RCU whether or not and when to treat normal grace periods as expedited.

Thanx, Paul

#21 lore raw
In reply to Arjan van de Ven
Quoted 13 lines · Arjan van de Ven
Quoted 6 lines
Quoted 3 lines

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

It would be good to have before-and-after measurements of actual boot time. Are these numbers available?

To show the boot time, I'm using the timestamp of the "Write protecting" line, that's pretty much the last thing we print prior to ring 3 execution.

A kernel with default RCU behavior (inside KVM, only virtual devices) looks like this:

Quoted 8 lines
[    0.038724] Write protecting the kernel read-only data: 10240k

a kernel with expedited RCU (using the command line option, so that I don't have to recompile between measurements and thus am completely oranges-to-oranges)

[    0.031768] Write protecting the kernel read-only data: 10240k

which, in percentage, is an 18% improvement.

Thank you, will repost with this info.

Thanx, Paul

#28 lore raw
In reply to Mathieu Desnoyers
Quoted 36 lines · Mathieu Desnoyers

----- Original Message -----

Quoted 35 lines

From: "Josh Triplett" <[email protected]>
To: "Peter Zijlstra" <[email protected]>
Cc: "Paul E. McKenney" <[email protected]>, [email protected], [email protected], [email protected], [email protected], [email protected], "mathieu desnoyers" <[email protected]>, [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "bobby prani" <[email protected]>
Sent: Saturday, February 21, 2015 1:04:28 AM
Subject: Re: [PATCH tip/core/rcu 0/4] Programmatic nestable expedited grace periods

Quoted 10 lines · Peter Zijlstra
Quoted 6 lines · Paul E. McKenney
Quoted 3 lines · Peter Zijlstra

Does it really make a machine boot much faster? Why are people using synchronous gp primitives if they care about speed? Should we not fix that instead?

The report I heard was that it provided 10-15% faster boot times.

That's not insignificant; got more details? I think we should really look at why people are using the sync primitives.

Paul, what do you think about adding a compile-time debug option to synchronize_rcu() that causes it to capture the time on entry and exit and print the duration together with the file:line of the caller? Similar to initcall_debug, but for blocking calls to synchronize_rcu(). Put that together with initcall_debug, and you'd have a pretty good idea of where that holds up boot.

We do want early boot to run as asynchronously as possible, and to avoid having later bits of boot waiting on a synchronize_rcu from earlier bits of boot. Switching a caller over to call_rcu() doesn't actually help if it still has to finish a grace period before it can allow later bits to run. Ideally, we ought to be able to work out the "depth" of boot in grace-periods.

Quoted 5 lines
Quoted 1 line

Has anyone wired initcall_debug up to a bootchart-like graph?

The information about begin/end of synchronize_rcu, as well as begin/end of rcu_barrier() seems to be very relevant here. This should perhaps be covered tracepoints ? Isn't it already ?

Good points, but they did measure this somehow. Wouldn't some ftrace magic get this result?

Thanx, Paul

Quoted 13 lines

Thanks,

Mathieu

Quoted 3 lines

- Josh Triplett

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#13 lore raw
In reply to Peter Zijlstra
Quoted 12 lines · Peter Zijlstra
Quoted 11 lines · Arjan van de Ven
Quoted 10 lines · Peter Zijlstra
Quoted 2 lines · Arjan van de Ven

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

So my objections are twofold:

- I object to fast expedites in principle; they spray IPIs across the system, so ideally we'd not have them at all, therefore also not at boot.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

Because as soon as the option exists, people will use it for other things too.

the option exists today in sysfs and kernel parameter...

Yeah, Paul and me have been having this argument for a while now ;-)

Indeed we have. ;-)

And if expedited grace periods start causing latency issues in real-world workloads, I will address those issues.

In the meantime, one of the nice things about NO_HZ_FULL is that synchronize_sched_expedited() avoids IPIing CPUs having a single runnable task that is running in nohz_full mode. ;-)

Thanx, Paul

Quoted 9 lines
Quoted 6 lines
Quoted 3 lines

And esp. in bootup code you can special case a lot of stuff; there's limited concurrency esp. because userspace it not there yet. So we might not actually need those sync calls.

yeah I am going down that angle as well absolutely.
but there are cases that may well be legit (or are 5 function calls deep into common code)

Good ;-)

maintainer
#22 lore raw
In reply to Arjan van de Ven
Quoted 9 lines · Arjan van de Ven
Quoted 6 lines
Quoted 3 lines

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

It would be good to have before-and-after measurements of actual boot time. Are these numbers available?

To show the boot time, I'm using the timestamp of the "Write protecting" line, that's pretty much the last thing we print prior to ring 3 execution.

That's a little sad; we ought to be write-protecting kernel read-only data as *early* as possible.

Quoted 10 lines

A kernel with default RCU behavior (inside KVM, only virtual devices) looks like this:

[    0.038724] Write protecting the kernel read-only data: 10240k

a kernel with expedited RCU (using the command line option, so that I don't have to recompile between measurements and thus am completely oranges-to-oranges)

[    0.031768] Write protecting the kernel read-only data: 10240k

which, in percentage, is an 18% improvement.

Nice improvement, but that suggests that we're spending far too much time waiting on RCU grace periods at boot time.

- Josh Triplett

#23 lore raw
Quoted 21 lines · Josh Triplett
Quoted 9 lines · Arjan van de Ven
Quoted 6 lines
Quoted 3 lines

there's a few others as well that I'm chasing down...
.. but the flip side, prior to running ring 3 code, why NOT do fast expedites?

It would be good to have before-and-after measurements of actual boot time. Are these numbers available?

To show the boot time, I'm using the timestamp of the "Write protecting" line, that's pretty much the last thing we print prior to ring 3 execution.

That's a little sad; we ought to be write-protecting kernel read-only data as *early* as possible.

Quoted 7 lines

A kernel with default RCU behavior (inside KVM, only virtual devices) looks like this:

[    0.038724] Write protecting the kernel read-only data: 10240k

a kernel with expedited RCU (using the command line option, so that I don't have to recompile between measurements and thus am completely oranges-to-oranges)

Quoted 6 lines
Quoted 3 lines
[    0.031768] Write protecting the kernel read-only data: 10240k

which, in percentage, is an 18% improvement.

Nice improvement, but that suggests that we're spending far too much time waiting on RCU grace periods at boot time.

Let's see... 0.038724-0.031768=0.006956, or about seven milliseconds. This might be as many as ten grace periods, but is more likely to be about two of them. Of course, this counts only the grace periods after the scheduler starts, as those prior to scheduler start are no-ops, courtesy of your single-CPU optimization.

So, how many grace periods between scheduler start and init spawning do you feel would be appropriate?

Thanx, Paul

#24 lore raw
In reply to Josh Triplett
Quoted 6 lines
Quoted 3 lines

To show the boot time, I'm using the timestamp of the "Write protecting" line, that's pretty much the last thing we print prior to ring 3 execution.

That's a little sad; we ought to be write-protecting kernel read-only data as *early* as possible.

well... if you are compromised before the first ring 3 instruction...
.... you have a slightly bigger problem than where in the kernel we write protect things.

maintainer
#25 lore raw
Quoted 8 lines · Arjan van de Ven
Quoted 5 lines
Quoted 2 lines

To show the boot time, I'm using the timestamp of the "Write protecting" line, that's pretty much the last thing we print prior to ring 3 execution.

That's a little sad; we ought to be write-protecting kernel read-only data as *early* as possible.

well... if you are compromised before the first ring 3 instruction...
.... you have a slightly bigger problem than where in the kernel we write protect things.

Definitely not talking about malicious compromise here; malicious code could just remove the write protection. However, write-protecting kernel read-only data also protects against a class of bugs.

- Josh Triplett