Skip to contentSkip to conversation

rcu: Make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled

PATCH Superseded Zqiang 3 participants 12 messages Oct 19, 2022 – Oct 20, 2022 RCU

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Zqiang

author
#0 lore raw

PATCH 1/1 rcu: Make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled

Currently, regardless of whether the CONFIG_RCU_LAZY is enabled, invoke the call_rcu() is always lazy, it also means that when CONFIG_RCU_LAZY is disabled, invoke the call_rcu_flush() is also lazy. therefore, this commit make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled.

---
 kernel/rcu/tree.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +7 -1
index abc615808b6e..97ef602da3d5 100644
@@ -2839,7 +2839,6 @@ void call_rcu_flush(struct rcu_head *head, rcu_callback_t func)
 	return __call_rcu_common(head, func, false);
 }
 EXPORT_SYMBOL_GPL(call_rcu_flush);
-#endif
 
 /**
  * call_rcu() - Queue an RCU callback for invocation after a grace period.
@@ -2890,6 +2889,13 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
 	return __call_rcu_common(head, func, true);
 }
 EXPORT_SYMBOL_GPL(call_rcu);
+#else
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+	return __call_rcu_common(head, func, false);
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif
 
 /* Maximum number of jiffies to wait before draining a batch. */
 #define KFREE_DRAIN_JIFFIES (5 * HZ)
Signature · 1 line
2.25.1
#1 lore raw
Quoted 28 lines · Zqiang

Currently, regardless of whether the CONFIG_RCU_LAZY is enabled, invoke the call_rcu() is always lazy, it also means that when CONFIG_RCU_LAZY is disabled, invoke the call_rcu_flush() is also lazy. therefore, this commit make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled.

---
kernel/rcu/tree.c | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +0 -1
index abc615808b6e..97ef602da3d5 100644
@@ -2839,7 +2839,6 @@ void call_rcu_flush(struct rcu_head *head, rcu_callback_t func)
   return __call_rcu_common(head, func, false);
 }
 EXPORT_SYMBOL_GPL(call_rcu_flush);
-#endif
 
 /**
 * call_rcu() - Queue an RCU callback for invocation after a grace period.

kernel/rcu/tree.c void call_rcu(struct rcu_head *head, rcu_callback_t func) · line 2895

Quoted 7 lines
kernel/rcu/tree.c +4 -0
@@ -2890,6 +2889,13 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
   return __call_rcu_common(head, func, true);
 }
 EXPORT_SYMBOL_GPL(call_rcu);
+#else
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+    return __call_rcu_common(head, func, false);

Thanks. Instead of adding new function, you can also pass IS_ENABLED(CONFIG…) to the existing function of the same name.

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

Thanks.

- Joel

Quoted 9 lines
kernel/rcu/tree.c +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif
 
 /* Maximum number of jiffies to wait before draining a batch. */
 #define KFREE_DRAIN_JIFFIES (5 * HZ)
Signature · 1 line
2.25.1
#2 lore raw
Quoted 42 lines · Joel Fernandes



Quoted 26 lines · Zqiang

Currently, regardless of whether the CONFIG_RCU_LAZY is enabled, invoke the call_rcu() is always lazy, it also means that when CONFIG_RCU_LAZY is disabled, invoke the call_rcu_flush() is also lazy. therefore, this commit make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled.

---
kernel/rcu/tree.c | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +0 -1
index abc615808b6e..97ef602da3d5 100644
@@ -2839,7 +2839,6 @@ void call_rcu_flush(struct rcu_head *head, rcu_callback_t func)
  return __call_rcu_common(head, func, false);
 }
 EXPORT_SYMBOL_GPL(call_rcu_flush);
-#endif
 
 /**
 * call_rcu() - Queue an RCU callback for invocation after a grace period.

kernel/rcu/tree.c void call_rcu(struct rcu_head *head, rcu_callback_t func) · line 2895

Quoted 7 lines
kernel/rcu/tree.c +4 -0
@@ -2890,6 +2889,13 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
  return __call_rcu_common(head, func, true);
 }
 EXPORT_SYMBOL_GPL(call_rcu);
+#else
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+    return __call_rcu_common(head, func, false);

Thanks. Instead of adding new function, you can also pass IS_ENABLED(CONFIG…) to the existing function of the same name.

Quoted 3 lines

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

On the other hand, what I’d want is to nuke the config option altogether or make it default y, we want to catch issues sooner than later.

Thanks.

Quoted 15 lines

Thanks.

- Joel

Quoted 9 lines
kernel/rcu/tree.c +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif
 
 /* Maximum number of jiffies to wait before draining a batch. */
 #define KFREE_DRAIN_JIFFIES (5 * HZ)
Signature · 1 line
2.25.1
maintainer
#3 lore raw
Quoted 8 lines · Joel Fernandes
Quoted 8 lines · Joel Fernandes
Quoted 7 lines · Zqiang

Currently, regardless of whether the CONFIG_RCU_LAZY is enabled, invoke the call_rcu() is always lazy, it also means that when CONFIG_RCU_LAZY is disabled, invoke the call_rcu_flush() is also lazy. therefore, this commit make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled.

First, good eyes! Thank you for spotting this!!!

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

---
kernel/rcu/tree.c | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +0 -1
index abc615808b6e..97ef602da3d5 100644
@@ -2839,7 +2839,6 @@ void call_rcu_flush(struct rcu_head *head, rcu_callback_t func)
  return __call_rcu_common(head, func, false);
 }
 EXPORT_SYMBOL_GPL(call_rcu_flush);
-#endif
 
Quoted 12 lines
Quoted 4 lines
Quoted 4 lines
kernel/rcu/tree.c +0 -0
 /**
 * call_rcu() - Queue an RCU callback for invocation after a grace period.
@@ -2890,6 +2889,13 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
  return __call_rcu_common(head, func, true);
Quoted 8 lines

kernel/rcu/tree.c void call_rcu(struct rcu_head *head, rcu_callback_t func) · line 2895

Quoted 6 lines
kernel/rcu/tree.c +4 -0
 }
 EXPORT_SYMBOL_GPL(call_rcu);
+#else
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+    return __call_rcu_common(head, func, false);

Thanks. Instead of adding new function, you can also pass IS_ENABLED(CONFIG…) to the existing function of the same name.

I do like this approach better -- less code, more obvious what is going on.

Quoted 5 lines
Quoted 3 lines

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

On the other hand, what I’d want is to nuke the config option altogether or make it default y, we want to catch issues sooner than later.

That might be what we do at some point, but one thing at a time. Let's not penalize innocent bystanders, at least not just yet.

I do very strongly encourage the ChromeOS and Android folks to test this very severely, however.

Thanx, Paul

Quoted 17 lines

Thanks.

Quoted 15 lines

Thanks.

- Joel

Quoted 9 lines
kernel/rcu/tree.c +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif
 
 /* Maximum number of jiffies to wait before draining a batch. */
 #define KFREE_DRAIN_JIFFIES (5 * HZ)
Signature · 1 line
2.25.1
#4 lore raw
Quoted 43 lines · Paul E. McKenney
Quoted 37 lines · Joel Fernandes
Quoted 8 lines · Joel Fernandes
Quoted 7 lines
Quoted 7 lines · Zqiang

Currently, regardless of whether the CONFIG_RCU_LAZY is enabled, invoke the call_rcu() is always lazy, it also means that when CONFIG_RCU_LAZY is disabled, invoke the call_rcu_flush() is also lazy. therefore, this commit make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled.

First, good eyes! Thank you for spotting this!!

Quoted 27 lines
Quoted 15 lines
Quoted 15 lines

---
kernel/rcu/tree.c | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +0 -1
index abc615808b6e..97ef602da3d5 100644
@@ -2839,7 +2839,6 @@ void call_rcu_flush(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, false);
 }
 EXPORT_SYMBOL_GPL(call_rcu_flush);
-#endif
 
Quoted 10 lines

kernel/rcu/tree.c void call_rcu(struct rcu_head *head, rcu_callback_t func) · line 2895

Quoted 10 lines
kernel/rcu/tree.c +4 -0
 /**
 * call_rcu() - Queue an RCU callback for invocation after a grace period.
@@ -2890,6 +2889,13 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, true);
 }
 EXPORT_SYMBOL_GPL(call_rcu);
+#else
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+    return __call_rcu_common(head, func, false);

Thanks. Instead of adding new function, you can also pass IS_ENABLED(CONFIG…) to the existing function of the same name.

Quoted 1 line

I do like this approach better -- less code, more obvious what is going on.

Sounds good. Zqiang, do you mind updating your patch along these lines? That way you get the proper attribution.

More comments below:

Quoted 9 lines
Quoted 6 lines
Quoted 4 lines

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

On the other hand, what I’d want is to nuke the config option altogether or make it default y, we want to catch issues sooner than later.

That might be what we do at some point, but one thing at a time. Let's not penalize innocent bystanders, at least not just yet.

It’s a trade off, I thought that’s why we wanted to have the binary search stuff. If no one reports issue on Linux-next, then that code won’t be put to use in the near future at least.

Quoted 2 lines

I do very strongly encourage the ChromeOS and Android folks to test this very severely, however.

Agreed. Yes that will happen, though I have to make a note for Android folks other than Vlad, to backports these (and enable the config option), carefully! Especially on pre-5.15 kernels. Luckily I had to do this (not so trivial) exercise myself.

Thanks!

- Joel

Quoted 20 lines

Thanx, Paul

Quoted 17 lines

Thanks.

Quoted 15 lines

Thanks.

- Joel

Quoted 9 lines
kernel/rcu/tree.c +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif

/* Maximum number of jiffies to wait before draining a batch. */ #define KFREE_DRAIN_JIFFIES (5 * HZ)

Signature · 1 line
2.25.1
#5 lore raw
Quoted 41 lines · Paul E. McKenney
Quoted 37 lines · Joel Fernandes
Quoted 8 lines · Joel Fernandes
Quoted 7 lines
Quoted 7 lines · Zqiang

Currently, regardless of whether the CONFIG_RCU_LAZY is enabled, invoke the call_rcu() is always lazy, it also means that when CONFIG_RCU_LAZY is disabled, invoke the call_rcu_flush() is also lazy. therefore, this commit make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled.

First, good eyes! Thank you for spotting this!!

Quoted 27 lines
Quoted 15 lines
Quoted 15 lines

---
kernel/rcu/tree.c | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +0 -1
index abc615808b6e..97ef602da3d5 100644
@@ -2839,7 +2839,6 @@ void call_rcu_flush(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, false);
 }
 EXPORT_SYMBOL_GPL(call_rcu_flush);
-#endif
 
Quoted 10 lines

kernel/rcu/tree.c void call_rcu(struct rcu_head *head, rcu_callback_t func) · line 2895

Quoted 10 lines
kernel/rcu/tree.c +4 -0
 /**
 * call_rcu() - Queue an RCU callback for invocation after a grace period.
@@ -2890,6 +2889,13 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, true);
 }
 EXPORT_SYMBOL_GPL(call_rcu);
+#else
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+    return __call_rcu_common(head, func, false);

Thanks. Instead of adding new function, you can also pass IS_ENABLED(CONFIG…) to the existing function of the same name.

Quoted 4 lines

I do like this approach better -- less code, more obvious what is going on.

Sounds good. Zqiang, do you mind updating your patch along these lines? That way you get the proper attribution.

Thanks Joel and Paul review, I will update my patch and resend.

Thanks
Zqiang

Quoted 42 lines

More comments below:

Quoted 5 lines
Quoted 3 lines

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

On the other hand, what I’d want is to nuke the config option altogether or make it default y, we want to catch issues sooner than later.

That might be what we do at some point, but one thing at a time. Let's not penalize innocent bystanders, at least not just yet.

It’s a trade off, I thought that’s why we wanted to have the binary search stuff. If no one reports issue on Linux-next, then that code won’t be put to use in the near future at least.

I do very strongly encourage the ChromeOS and Android folks to test this very severely, however.

Agreed. Yes that will happen, though I have to make a note for Android folks other than Vlad, to backports these (and enable the config option), carefully! Especially on pre-5.15 kernels. Luckily I had to do this (not so trivial) exercise myself.

Thanks!

- Joel

Thanx, Paul

Quoted 17 lines

Thanks.

Quoted 15 lines

Thanks.

- Joel

Quoted 9 lines
kernel/rcu/tree.c +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif

/* Maximum number of jiffies to wait before draining a batch. */ #define KFREE_DRAIN_JIFFIES (5 * HZ)

Signature · 1 line
2.25.1
maintainer
#6 lore raw
In reply to Joel Fernandes
Quoted 51 lines · Joel Fernandes
Quoted 43 lines · Paul E. McKenney
Quoted 37 lines · Joel Fernandes
Quoted 8 lines · Joel Fernandes
Quoted 7 lines
Quoted 7 lines · Zqiang

Currently, regardless of whether the CONFIG_RCU_LAZY is enabled, invoke the call_rcu() is always lazy, it also means that when CONFIG_RCU_LAZY is disabled, invoke the call_rcu_flush() is also lazy. therefore, this commit make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled.

First, good eyes! Thank you for spotting this!!

Quoted 27 lines
Quoted 15 lines
Quoted 15 lines

---
kernel/rcu/tree.c | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +0 -1
index abc615808b6e..97ef602da3d5 100644
@@ -2839,7 +2839,6 @@ void call_rcu_flush(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, false);
 }
 EXPORT_SYMBOL_GPL(call_rcu_flush);
-#endif
 
Quoted 10 lines

kernel/rcu/tree.c void call_rcu(struct rcu_head *head, rcu_callback_t func) · line 2895

Quoted 10 lines
kernel/rcu/tree.c +4 -0
 /**
 * call_rcu() - Queue an RCU callback for invocation after a grace period.
@@ -2890,6 +2889,13 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, true);
 }
 EXPORT_SYMBOL_GPL(call_rcu);
+#else
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+    return __call_rcu_common(head, func, false);

Thanks. Instead of adding new function, you can also pass IS_ENABLED(CONFIG…) to the existing function of the same name.

Quoted 1 line

I do like this approach better -- less code, more obvious what is going on.

Sounds good. Zqiang, do you mind updating your patch along these lines? That way you get the proper attribution.

More comments below:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

On the other hand, what I’d want is to nuke the config option altogether or make it default y, we want to catch issues sooner than later.

That might be what we do at some point, but one thing at a time. Let's not penalize innocent bystanders, at least not just yet.

It’s a trade off, I thought that’s why we wanted to have the binary search stuff. If no one reports issue on Linux-next, then that code won’t be put to use in the near future at least.

Well, not to put too fine a point on it, but we currently really are exposing -next to lazy call_rcu(). ;-)

Quoted 4 lines
Quoted 2 lines

I do very strongly encourage the ChromeOS and Android folks to test this very severely, however.

Agreed. Yes that will happen, though I have to make a note for Android folks other than Vlad, to backports these (and enable the config option), carefully! Especially on pre-5.15 kernels. Luckily I had to do this (not so trivial) exercise myself.

And this is another situation in which the binary search stuff may prove extremely useful.

Thanx, Paul

Quoted 24 lines

Thanks!

- Joel

Quoted 20 lines

Thanx, Paul

Quoted 17 lines

Thanks.

Quoted 15 lines

Thanks.

- Joel

Quoted 9 lines
kernel/rcu/tree.c +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif

/* Maximum number of jiffies to wait before draining a batch. */ #define KFREE_DRAIN_JIFFIES (5 * HZ)

Signature · 1 line
2.25.1
#7 lore raw
Quoted 13 lines · Paul E. McKenney
Quoted 8 lines · Joel Fernandes
Quoted 8 lines
Quoted 3 lines · Paul E. McKenney
Quoted 3 lines · Joel Fernandes
Quoted 3 lines · Joel Fernandes
Quoted 2 lines
Quoted 2 lines · Zqiang
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Currently, regardless of whether the CONFIG_RCU_LAZY is enabled, invoke the call_rcu() is always lazy, it also means that when CONFIG_RCU_LAZY is disabled, invoke the call_rcu_flush() is also lazy. therefore, this commit make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled.

First, good eyes! Thank you for spotting this!!

Indeed.

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
Quoted 27 lines
Quoted 15 lines
Quoted 15 lines

---
kernel/rcu/tree.c | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +0 -1
index abc615808b6e..97ef602da3d5 100644
@@ -2839,7 +2839,6 @@ void call_rcu_flush(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, false);
 }
 EXPORT_SYMBOL_GPL(call_rcu_flush);
-#endif
 
Quoted 10 lines

kernel/rcu/tree.c void call_rcu(struct rcu_head *head, rcu_callback_t func) · line 2895

Quoted 10 lines
kernel/rcu/tree.c +4 -0
 /**
 * call_rcu() - Queue an RCU callback for invocation after a grace period.
@@ -2890,6 +2889,13 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, true);
 }
 EXPORT_SYMBOL_GPL(call_rcu);
+#else
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+    return __call_rcu_common(head, func, false);

Thanks. Instead of adding new function, you can also pass IS_ENABLED(CONFIG…) to the existing function of the same name.

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

I do like this approach better -- less code, more obvious what is going on.

Sounds good. Zqiang, do you mind updating your patch along these lines? That way you get the proper attribution.

Acked that patch.

Quoted 8 lines
Quoted 8 lines

More comments below:

Quoted 7 lines
Quoted 6 lines
Quoted 4 lines

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

On the other hand, what I’d want is to nuke the config option altogether or make it default y, we want to catch issues sooner than later.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

That might be what we do at some point, but one thing at a time. Let's not penalize innocent bystanders, at least not just yet.

It’s a trade off, I thought that’s why we wanted to have the binary search stuff. If no one reports issue on Linux-next, then that code won’t be put to use in the near future at least.

Well, not to put too fine a point on it, but we currently really are exposing -next to lazy call_rcu(). ;-)

This is true. I think I assumed nobody will enable a default off config option but I probably meant a smaller percentage will.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

I do very strongly encourage the ChromeOS and Android folks to test this very severely, however.

Agreed. Yes that will happen, though I have to make a note for Android folks other than Vlad, to backports these (and enable the config option), carefully! Especially on pre-5.15 kernels. Luckily I had to do this (not so trivial) exercise myself.

And this is another situation in which the binary search stuff may prove extremely useful.

Agreed. Thanks. Very least I owe per-rdp splitting of the hashtable, to that code. Steven and me talked today that probably the hashtable can go into the rcu_segcblist itself, and protect it by the nocb lock.

Thanks,

- Joel

Quoted 26 lines

Thanx, Paul

Quoted 24 lines

Thanks!

- Joel

Quoted 20 lines

Thanx, Paul

Quoted 17 lines

Thanks.

Quoted 15 lines

Thanks.

- Joel

Quoted 9 lines
kernel/rcu/tree.c +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif

/* Maximum number of jiffies to wait before draining a batch. */ #define KFREE_DRAIN_JIFFIES (5 * HZ)

Signature · 1 line
2.25.1
maintainer
#8 lore raw
Quoted 76 lines · Joel Fernandes
Quoted 11 lines · Paul E. McKenney
Quoted 8 lines · Joel Fernandes
Quoted 8 lines
Quoted 3 lines · Paul E. McKenney
Quoted 3 lines · Joel Fernandes
Quoted 3 lines · Joel Fernandes
Quoted 2 lines
Quoted 2 lines · Zqiang
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Currently, regardless of whether the CONFIG_RCU_LAZY is enabled, invoke the call_rcu() is always lazy, it also means that when CONFIG_RCU_LAZY is disabled, invoke the call_rcu_flush() is also lazy. therefore, this commit make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled.

First, good eyes! Thank you for spotting this!!

Indeed.

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
Quoted 27 lines
Quoted 15 lines
Quoted 15 lines

---
kernel/rcu/tree.c | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +0 -1
index abc615808b6e..97ef602da3d5 100644
@@ -2839,7 +2839,6 @@ void call_rcu_flush(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, false);
 }
 EXPORT_SYMBOL_GPL(call_rcu_flush);
-#endif
 
Quoted 10 lines

kernel/rcu/tree.c void call_rcu(struct rcu_head *head, rcu_callback_t func) · line 2895

Quoted 10 lines
kernel/rcu/tree.c +4 -0
 /**
 * call_rcu() - Queue an RCU callback for invocation after a grace period.
@@ -2890,6 +2889,13 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, true);
 }
 EXPORT_SYMBOL_GPL(call_rcu);
+#else
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+    return __call_rcu_common(head, func, false);

Thanks. Instead of adding new function, you can also pass IS_ENABLED(CONFIG…) to the existing function of the same name.

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

I do like this approach better -- less code, more obvious what is going on.

Sounds good. Zqiang, do you mind updating your patch along these lines? That way you get the proper attribution.

Acked that patch.

Quoted 8 lines
Quoted 8 lines

More comments below:

Quoted 7 lines
Quoted 6 lines
Quoted 4 lines

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

On the other hand, what I’d want is to nuke the config option altogether or make it default y, we want to catch issues sooner than later.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

That might be what we do at some point, but one thing at a time. Let's not penalize innocent bystanders, at least not just yet.

It’s a trade off, I thought that’s why we wanted to have the binary search stuff. If no one reports issue on Linux-next, then that code won’t be put to use in the near future at least.

Well, not to put too fine a point on it, but we currently really are exposing -next to lazy call_rcu(). ;-)

This is true. I think I assumed nobody will enable a default off config option but I probably meant a smaller percentage will.

Quoted 3 lines
Quoted 3 lines
Quoted 2 lines

I do very strongly encourage the ChromeOS and Android folks to test this very severely, however.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

Agreed. Yes that will happen, though I have to make a note for Android folks other than Vlad, to backports these (and enable the config option), carefully! Especially on pre-5.15 kernels. Luckily I had to do this (not so trivial) exercise myself.

And this is another situation in which the binary search stuff may prove extremely useful.

Agreed. Thanks. Very least I owe per-rdp splitting of the hashtable, to that code. Steven and me talked today that probably the hashtable can go into the rcu_segcblist itself, and protect it by the nocb lock.

I have to ask...

How does this fit in with CPU-hotplug and callback migration?

More to the point, what events would cause us to decide that this is required? For example, shouldn't we give your current binary-search code at least a few chances to save the day?

Thanx, Paul

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
(quoted diff) +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif

/* Maximum number of jiffies to wait before draining a batch. */ #define KFREE_DRAIN_JIFFIES (5 * HZ)

Signature · 1 line
2.25.1
#9 lore raw
Quoted 83 lines · Paul E. McKenney
Quoted 78 lines · Joel Fernandes
Quoted 11 lines
Quoted 1 line · Paul E. McKenney
Quoted 8 lines · Joel Fernandes
Quoted 8 lines
Quoted 3 lines · Paul E. McKenney
Quoted 3 lines · Joel Fernandes
Quoted 3 lines · Joel Fernandes
Quoted 2 lines
Quoted 2 lines · Zqiang
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Currently, regardless of whether the CONFIG_RCU_LAZY is enabled, invoke the call_rcu() is always lazy, it also means that when CONFIG_RCU_LAZY is disabled, invoke the call_rcu_flush() is also lazy. therefore, this commit make call_rcu() lazy only when CONFIG_RCU_LAZY is enabled.

First, good eyes! Thank you for spotting this!!

Indeed.

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
Quoted 27 lines
Quoted 15 lines
Quoted 15 lines

---
kernel/rcu/tree.c | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +0 -1
index abc615808b6e..97ef602da3d5 100644
@@ -2839,7 +2839,6 @@ void call_rcu_flush(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, false);
 }
 EXPORT_SYMBOL_GPL(call_rcu_flush);
-#endif
 
Quoted 10 lines

kernel/rcu/tree.c void call_rcu(struct rcu_head *head, rcu_callback_t func) · line 2895

Quoted 10 lines
kernel/rcu/tree.c +4 -0
 /**
 * call_rcu() - Queue an RCU callback for invocation after a grace period.
@@ -2890,6 +2889,13 @@ void call_rcu(struct rcu_head *head, rcu_callback_t func)
 return __call_rcu_common(head, func, true);
 }
 EXPORT_SYMBOL_GPL(call_rcu);
+#else
+void call_rcu(struct rcu_head *head, rcu_callback_t func)
+{
+    return __call_rcu_common(head, func, false);

Thanks. Instead of adding new function, you can also pass IS_ENABLED(CONFIG…) to the existing function of the same name.

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

I do like this approach better -- less code, more obvious what is going on.

Sounds good. Zqiang, do you mind updating your patch along these lines? That way you get the proper attribution.

Acked that patch.

Quoted 8 lines
Quoted 8 lines

More comments below:

Quoted 7 lines
Quoted 6 lines
Quoted 4 lines

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

On the other hand, what I’d want is to nuke the config option altogether or make it default y, we want to catch issues sooner than later.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

That might be what we do at some point, but one thing at a time. Let's not penalize innocent bystanders, at least not just yet.

It’s a trade off, I thought that’s why we wanted to have the binary search stuff. If no one reports issue on Linux-next, then that code won’t be put to use in the near future at least.

Well, not to put too fine a point on it, but we currently really are exposing -next to lazy call_rcu(). ;-)

This is true. I think I assumed nobody will enable a default off config option but I probably meant a smaller percentage will.

Quoted 5 lines
Quoted 4 lines
Quoted 2 lines

I do very strongly encourage the ChromeOS and Android folks to test this very severely, however.

Agreed. Yes that will happen, though I have to make a note for Android folks other than Vlad, to backports these (and enable the config option), carefully! Especially on pre-5.15 kernels. Luckily I had to do this (not so trivial) exercise myself.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

And this is another situation in which the binary search stuff may prove extremely useful.

Agreed. Thanks. Very least I owe per-rdp splitting of the hashtable, to that code. Steven and me talked today that probably the hashtable can go into the rcu_segcblist itself, and protect it by the nocb lock.

I have to ask...

How does this fit in with CPU-hotplug and callback migration?

Yes it will require change and I already thought of that, have to update the hashtable on all such events.

Quoted 3 lines

More to the point, what events would cause us to decide that this is required? For example, shouldn't we give your current binary-search code at least a few chances to save the day?

Totally, if you’re taking the patch as is, I would be very happy. And I’ll continue to improve it with the above. But I was not sure yet if you’re taking it.

I think it’s a worthwhile to take it for mainline in the current state and I’ll also add more data about callbacks to it in future (queuing time of callback, etc) — basically all the stuff I wanted to add to rcu_head.

One reason for the above proposal is I also want to keep it turned on in production, and the current solution cannot be, due to the global locking and is not expected to be kept on in production. But is still a worthwhile addition for debug kernels IMO.

Thanks,

- Joel

Quoted 11 lines

Thanx, Paul

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
(quoted diff) +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif

/* Maximum number of jiffies to wait before draining a batch. */ #define KFREE_DRAIN_JIFFIES (5 * HZ)

Signature · 1 line
2.25.1
#10 lore raw
Quoted 42 lines · Joel Fernandes
Quoted 24 lines
Quoted 24 lines
Quoted 9 lines
Quoted 9 lines

More comments below:

Quoted 7 lines
Quoted 6 lines
Quoted 4 lines

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

On the other hand, what I’d want is to nuke the config option altogether or make it default y, we want to catch issues sooner than later.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

That might be what we do at some point, but one thing at a time. Let's not penalize innocent bystanders, at least not just yet.

It’s a trade off, I thought that’s why we wanted to have the binary search stuff. If no one reports issue on Linux-next, then that code won’t be put to use in the near future at least.

Well, not to put too fine a point on it, but we currently really are exposing -next to lazy call_rcu(). ;-)

This is true. I think I assumed nobody will enable a default off config option but I probably meant a smaller percentage will.

Quoted 5 lines
Quoted 4 lines
Quoted 2 lines

I do very strongly encourage the ChromeOS and Android folks to test this very severely, however.

Agreed. Yes that will happen, though I have to make a note for Android folks other than Vlad, to backports these (and enable the config option), carefully! Especially on pre-5.15 kernels. Luckily I had to do this (not so trivial) exercise myself.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

And this is another situation in which the binary search stuff may prove extremely useful.

Agreed. Thanks. Very least I owe per-rdp splitting of the hashtable, to that code. Steven and me talked today that probably the hashtable can go into the rcu_segcblist itself, and protect it by the nocb lock.

I have to ask...

How does this fit in with CPU-hotplug and callback migration?

Yes it will require change and I already thought of that, have to update the hashtable on all such events.

Quoted 3 lines

More to the point, what events would cause us to decide that this is required? For example, shouldn't we give your current binary-search code at least a few chances to save the day?

Quoted 5 lines

Totally, if you’re taking the patch as is, I would be very happy. And I’ll continue to improve it with the above. But I was not sure yet if you’re taking it.

I think it’s a worthwhile to take it for mainline in the current state and I’ll also add more data about callbacks to it in future (queuing time of callback, etc) — basically all the stuff I wanted to add to rcu_head.

One reason for the above proposal is I also want to keep it turned on in production, and the current solution cannot be, due to the global locking and is not expected to be kept on in production. But is still a worthwhile addition for debug kernels IMO.

I realized while talking to Steve that the hashtable has to be per CPU if we are to store more than a lazy flag, such as queuing timestamps. This is because you can have multiple callbacks of the same function pointer queued on multiple CPUs. So you can have multiple timestamps to store. Same thing if we stored automata. It’s per callback instance, not per callback function.

Thanks,

- Joel

Quoted 17 lines

Thanks,

- Joel

Quoted 11 lines

Thanx, Paul

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
(quoted diff) +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif

/* Maximum number of jiffies to wait before draining a batch. */ #define KFREE_DRAIN_JIFFIES (5 * HZ)

Signature · 1 line
2.25.1
maintainer
#11 lore raw
Quoted 42 lines · Joel Fernandes
Quoted 42 lines · Joel Fernandes
Quoted 24 lines
Quoted 24 lines
Quoted 9 lines
Quoted 9 lines

More comments below:

Quoted 7 lines
Quoted 6 lines
Quoted 4 lines

Looks like though I made every one test the patch without having to enable the config option ;-). Hey, I’m a half glass full kind of guy, why do you ask?

Paul, I’ll take a closer look once I’m at the desk, but would you prefer to squash a diff into the existing patch, or want a new patch altogether?

On the other hand, what I’d want is to nuke the config option altogether or make it default y, we want to catch issues sooner than later.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

That might be what we do at some point, but one thing at a time. Let's not penalize innocent bystanders, at least not just yet.

It’s a trade off, I thought that’s why we wanted to have the binary search stuff. If no one reports issue on Linux-next, then that code won’t be put to use in the near future at least.

Well, not to put too fine a point on it, but we currently really are exposing -next to lazy call_rcu(). ;-)

This is true. I think I assumed nobody will enable a default off config option but I probably meant a smaller percentage will.

Quoted 5 lines
Quoted 4 lines
Quoted 2 lines

I do very strongly encourage the ChromeOS and Android folks to test this very severely, however.

Agreed. Yes that will happen, though I have to make a note for Android folks other than Vlad, to backports these (and enable the config option), carefully! Especially on pre-5.15 kernels. Luckily I had to do this (not so trivial) exercise myself.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

And this is another situation in which the binary search stuff may prove extremely useful.

Agreed. Thanks. Very least I owe per-rdp splitting of the hashtable, to that code. Steven and me talked today that probably the hashtable can go into the rcu_segcblist itself, and protect it by the nocb lock.

I have to ask...

How does this fit in with CPU-hotplug and callback migration?

Yes it will require change and I already thought of that, have to update the hashtable on all such events.

Quoted 3 lines

More to the point, what events would cause us to decide that this is required? For example, shouldn't we give your current binary-search code at least a few chances to save the day?

Quoted 7 lines
Quoted 5 lines

Totally, if you’re taking the patch as is, I would be very happy. And I’ll continue to improve it with the above. But I was not sure yet if you’re taking it.

I think it’s a worthwhile to take it for mainline in the current state and I’ll also add more data about callbacks to it in future (queuing time of callback, etc) — basically all the stuff I wanted to add to rcu_head.

One reason for the above proposal is I also want to keep it turned on in production, and the current solution cannot be, due to the global locking and is not expected to be kept on in production. But is still a worthwhile addition for debug kernels IMO.

I realized while talking to Steve that the hashtable has to be per CPU if we are to store more than a lazy flag, such as queuing timestamps. This is because you can have multiple callbacks of the same function pointer queued on multiple CPUs. So you can have multiple timestamps to store. Same thing if we stored automata. It’s per callback instance, not per callback function.

Agreed, to be useful, this must be per callback instance.

Thanx, Paul

Quoted 22 lines

Thanks,

- Joel

Quoted 17 lines

Thanks,

- Joel

Quoted 11 lines

Thanx, Paul

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
(quoted diff) +3 -0
+}
+EXPORT_SYMBOL_GPL(call_rcu);
+#endif

/* Maximum number of jiffies to wait before draining a batch. */ #define KFREE_DRAIN_JIFFIES (5 * HZ)

Signature · 1 line
2.25.1