Skip to contentSkip to conversation

rust: sync: optimize rust symbol generation for CondVar

PATCH Applied Kunwu Chan 4 participants 8 messages Mar 24, 2025 – Jul 3, 2025

Subsystems (2)
lore
Share & tools
lore

Kunwu Chan

author
#0 lore raw

PATCH v2 1/1 rust: sync: optimize rust symbol generation for CondVar

From: Kunwu Chan <[email protected]>

When build the kernel using the llvm-18.1.3-rust-1.85.0-x86_64 with ARCH=arm64, the following symbols are generated:

$nm vmlinux | grep ' _R'.*CondVar | rustfilt
... T <kernel::sync::condvar::CondVar>::notify_all
... T <kernel::sync::condvar::CondVar>::notify_one
... T <kernel::sync::condvar::CondVar>::notify_sync
... T <kernel::sync::condvar::CondVar>::new::{closure#0}::{closure#0}::panic_cold_explicit ... T <kernel::sync::condvar::CondVar>::new::{closure#0}::{closure#0}::panic_cold_explicit ... T <kernel::sync::poll::PollCondVar>::new::{closure#0}::{closure#0}::panic_cold_explicit ... T <kernel::sync::poll::PollCondVar as core::ops::drop::Drop>::drop

These notify_* symbols are trivial wrappers around the C functions __wake_up and __wake_up_sync. It doesn't make sense to go through a trivial wrapper for these functions, so mark them inline.

---
Changes in v2:
- Remove '#[inline]' for notify()
- Reword commit msg
- v1 link: https://lore.kernel.org/rust-for-linux/[email protected]

---
 rust/kernel/sync/condvar.rs | 3 +++
 1 file changed, 3 insertions(+)
rust/kernel/sync/condvar.rs +3 -0
index caebf03f553b..c6ec64295c9f 100644
@@ -216,6 +216,7 @@ fn notify(&self, count: c_int) {
     /// This method behaves like `notify_one`, except that it hints to the scheduler that the
     /// current thread is about to go to sleep, so it should schedule the target thread on the same
     /// CPU.
+    #[inline]
     pub fn notify_sync(&self) {
         // SAFETY: `wait_queue_head` points to valid memory.
         unsafe { bindings::__wake_up_sync(self.wait_queue_head.get(), TASK_NORMAL) };
@@ -225,6 +226,7 @@ pub fn notify_sync(&self) {
     ///
     /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
     /// completely (as opposed to automatically waking up the next waiter).
+    #[inline]
     pub fn notify_one(&self) {
         self.notify(1);
     }
@@ -233,6 +235,7 @@ pub fn notify_one(&self) {
     ///
     /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
     /// completely (as opposed to automatically waking up the next waiter).
+    #[inline]
     pub fn notify_all(&self) {
         self.notify(0);
     }
Signature · 1 line
2.43.0
#1 lore raw
Quoted 3 lines · Kunwu Chan

Nit: I typically use Link after Suggested-by, to mimic what the docs require about Reported-by with Link. (No need to resend a new version just for this :)

Quoted 1 line

- Remove '#[inline]' for notify()

It is good that the commit matches the log now, though I wonder if in the future we may want the #[inline] for notify anyway, even if LLVM does that one on its own today.

Cheers,
Miguel

#2 lore raw
Quoted 9 lines · Miguel Ojeda
Quoted 3 lines · Kunwu Chan

Nit: I typically use Link after Suggested-by, to mimic what the docs require about Reported-by with Link. (No need to resend a new version just for this :)

Quoted 6 lines
Quoted 1 line

- Remove '#[inline]' for notify()

It is good that the commit matches the log now, though I wonder if in the future we may want the #[inline] for notify anyway, even if LLVM does that one on its own today.

IMO, inlining should be mostly decided by compilers instead of programmers, so if we don't have evidences that compilers need some guidance, we shouldn't introduce the #[inline] attribute.

Maybe Kunwu has an example saying otherwise?

Regards,
Boqun

Quoted 2 lines

Cheers,
Miguel

author
#3 lore raw
In reply to Miguel Ojeda
Quoted 6 lines · Miguel Ojeda
Quoted 2 lines · Kunwu Chan

Nit: I typically use Link after Suggested-by, to mimic what the docs require about Reported-by with Link. (No need to resend a new version just for this :)

Thanks for the kind reminder, I'll follow this next time

Quoted 4 lines
Quoted 1 line

- Remove '#[inline]' for notify()

It is good that the commit matches the log now, though I wonder if in the future we may want the #[inline] for notify anyway, even if LLVM does that one on its own today.

Now, the default '-Copt-level' is 2 when define 'CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE', and in  KBUILD_RUSTFLAGS the  '-Ccodegen-units' is 1.

So if the config not change, the result should be the same.

And the actualy compile steps for rust/kernel.o is as following:

make -f ./scripts/Makefile.build obj=rust
# RUSTC L rust/kernel.o
OBJTREE=/media/kunwuchan/68d922f3-5dde-45b0-af93-c45ba81d85ef/linux-next rustc --edition=2021 -Zbinary_dep_depinfo=y -Astable_features -Dnon_ascii_idents -Dunsafe_op_in_unsafe_fn -Wmissing_docs -Wrust_2018_idioms -Wunreachable_pub -Wclippy::all -Wclippy::ignored_unit_patterns -Wclippy::mut_mut -Wclippy::needless_bitwise_bool -Wclippy::needless_continue -Aclippy::needless_lifetimes -Wclippy::no_mangle_with_rust_abi -Wclippy::undocumented_unsafe_blocks -Wclippy::unnecessary_safety_comment -Wclippy::unnecessary_safety_doc -Wrustdoc::missing_crate_level_docs -Wrustdoc::unescaped_backticks -Cpanic=abort -Cembed-bitcode=n -Clto=n -Cforce-unwind-tables=n -Ccodegen-units=1 -Csymbol-mangling-version=v0 -Crelocation-model=static -Zfunction-sections=n -Wclippy::float_arithmetic --target=aarch64-unknown-none-softfloat -Cforce-unwind-tables=n -Zbranch-protection=bti,pac-ret -Copt-level=2 -Cdebug-assertions=y -Coverflow-checks=y -Cforce-frame-pointers=y -Cdebuginfo=1 @./include/generated/rustc_cfg --extern ffi --extern pin_init --extern build_error --extern macros --extern bindings --extern uapi --emit=dep-info=rust/.kernel.o.d --emit=obj=rust/kernel.o --emit=metadata=rust/libkernel.rmeta --crate-type rlib -L./rust --crate-name kernel rust/kernel/lib.rs --sysroot=/dev/null # EXPORTS rust/exports_kernel_generated.h
  llvm-nm -p --defined-only rust/kernel.o | awk '$2~/(T|R|D|B)/ && $3!~/__cfi/ && $3!~/__odr_asan/ { printf "EXPORT_SYMBOL_RUST_GPL(%s);
",$3 }' > rust/exports_kernel_generated.h

Quoted 2 lines

Cheers,
Miguel

Signature · 2 lines
Thanks,
  Kunwu.Chan(Tao.Chan)
#4 lore raw
Quoted 2 lines · Kunwu Chan

Thanks for the kind reminder, I'll follow this next time

You're welcome!

Quoted 5 lines

Now, the default '-Copt-level' is 2 when define 'CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE',
and in KBUILD_RUSTFLAGS the '-Ccodegen-units' is 1.

So if the config not change, the result should be the same.

I don't think that is true, because there are a few rustc versions as well as LLVM ones that we support, so how inlining happens (at both rustc and LLVM levels) may change even if the configuration is essentially the same.

And, of course, then there are other non-compiler-related kernel config options (i.e. not compiler flags, but other stuff, like cfgs) that influence which and how gets emitted and thus the inlining decisions.

Eventually we should have pure GCC builds too, which is yet another factor...

To be clear, I agree with Boqun that having to write #[inline] everywhere is not great. Rust 1.75 already started to tag small functions as #[inline] to try to prevent the proliferation of the attribute, which is good (i.e. which is what triggered the message in Compiler Explorer).

By the way, one difference is whether it is pub -- the notify() isn't (unlike the others), but if it were, then we would have needed the #[inline], from a quick test.

Thanks!

Cheers,
Miguel

author
#5 lore raw
Quoted 20 lines · Miguel Ojeda
Quoted 1 line · Kunwu Chan

Thanks for the kind reminder, I'll follow this next time

You're welcome!

Quoted 5 lines

Now, the default '-Copt-level' is 2 when define 'CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE',
and in KBUILD_RUSTFLAGS the '-Ccodegen-units' is 1.

So if the config not change, the result should be the same.

I don't think that is true, because there are a few rustc versions as well as LLVM ones that we support, so how inlining happens (at both rustc and LLVM levels) may change even if the configuration is essentially the same.

And, of course, then there are other non-compiler-related kernel config options (i.e. not compiler flags, but other stuff, like cfgs) that influence which and how gets emitted and thus the inlining decisions.
Eventually we should have pure GCC builds too, which is yet another factor...

Quoted 5 lines

To be clear, I agree with Boqun that having to write #[inline] everywhere is not great. Rust 1.75 already started to tag small functions as #[inline] to try to prevent the proliferation of the attribute, which is good (i.e. which is what triggered the message in Compiler Explorer).

Thanks for the detailed reply.

Sure on the one hand, the decisions is a result of many factors,

on the other hand the rustc and llvm is rapidly developing.

It's a complicated thing.

Quoted 3 lines

By the way, one difference is whether it is pub -- the notify() isn't (unlike the others), but if it were, then we would have needed the #[inline], from a quick test.

That is a easy way to judge. I learned.

Quoted 4 lines

Thanks!

Cheers,
Miguel

Signature · 2 lines
Thanks,
  Kunwu.Chan(Tao.Chan)
#6 lore raw
In reply to Kunwu Chan
Quoted 18 lines · Kunwu Chan

From: Kunwu Chan <[email protected]>

When build the kernel using the llvm-18.1.3-rust-1.85.0-x86_64 with ARCH=arm64, the following symbols are generated:

$nm vmlinux | grep ' _R'.*CondVar | rustfilt
... T <kernel::sync::condvar::CondVar>::notify_all
... T <kernel::sync::condvar::CondVar>::notify_one
... T <kernel::sync::condvar::CondVar>::notify_sync
... T <kernel::sync::condvar::CondVar>::new::{closure#0}::{closure#0}::panic_cold_explicit ... T <kernel::sync::condvar::CondVar>::new::{closure#0}::{closure#0}::panic_cold_explicit ... T <kernel::sync::poll::PollCondVar>::new::{closure#0}::{closure#0}::panic_cold_explicit ... T <kernel::sync::poll::PollCondVar as core::ops::drop::Drop>::drop

These notify_* symbols are trivial wrappers around the C functions __wake_up and __wake_up_sync. It doesn't make sense to go through a trivial wrapper for these functions, so mark them inline.

Quoted 8 lines

I'm taking this via tip with the version as it is (i.e. no #[inline] for notify()), but will reorder the tags. Thanks!

Regards,
Boqun

Quoted 41 lines

---
Changes in v2:
- Remove '#[inline]' for notify()
- Reword commit msg
- v1 link: https://lore.kernel.org/rust-for-linux/[email protected]

---
 rust/kernel/sync/condvar.rs | 3 +++
 1 file changed, 3 insertions(+)
rust/kernel/sync/condvar.rs +3 -0
index caebf03f553b..c6ec64295c9f 100644
@@ -216,6 +216,7 @@ fn notify(&self, count: c_int) {
     /// This method behaves like `notify_one`, except that it hints to the scheduler that the
     /// current thread is about to go to sleep, so it should schedule the target thread on the same
     /// CPU.
+    #[inline]
     pub fn notify_sync(&self) {
         // SAFETY: `wait_queue_head` points to valid memory.
         unsafe { bindings::__wake_up_sync(self.wait_queue_head.get(), TASK_NORMAL) };
@@ -225,6 +226,7 @@ pub fn notify_sync(&self) {
     ///
     /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
     /// completely (as opposed to automatically waking up the next waiter).
+    #[inline]
     pub fn notify_one(&self) {
         self.notify(1);
     }
@@ -233,6 +235,7 @@ pub fn notify_one(&self) {
     ///
     /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
     /// completely (as opposed to automatically waking up the next waiter).
+    #[inline]
     pub fn notify_all(&self) {
         self.notify(0);
     }
Signature · 1 line
2.43.0
merge bot Applied
#7 lore raw

[tip: sched/core] rust: sync: Mark CondVar::notify_*() inline

In reply to Kunwu Chan
The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     3f9ebeba9878679bb43ee2db7d50a4691f55e3a5
Gitweb:        https://git.kernel.org/tip/3f9ebeba9878679bb43ee2db7d50a4691f55e3a5
Author:        Kunwu Chan <[email protected]>
AuthorDate:    Mon, 24 Mar 2025 14:18:34 +08:00
Committer:     Boqun Feng <[email protected]>
CommitterDate: Tue, 24 Jun 2025 10:23:48 -07:00

rust: sync: Mark CondVar::notify_*() inline

When build the kernel using the llvm-18.1.3-rust-1.85.0-x86_64 with ARCH=arm64, the following symbols are generated:

$nm vmlinux | grep ' _R'.*CondVar | rustfilt ... T <kernel::sync::condvar::CondVar>::notify_all ... T <kernel::sync::condvar::CondVar>::notify_one ... T <kernel::sync::condvar::CondVar>::notify_sync
...

These notify_*() symbols are trivial wrappers around the C functions __wake_up() and __wake_up_sync(). It doesn't make sense to go through a trivial wrapper for these functions, so mark them inline.

[boqun: Reword the commit title for consistency and reformat the commit log.]

---
 rust/kernel/sync/condvar.rs | 3 +++
 1 file changed, 3 insertions(+)
rust/kernel/sync/condvar.rs +3 -0
index caebf03..c6ec642 100644
@@ -216,6 +216,7 @@ impl CondVar {
     /// This method behaves like `notify_one`, except that it hints to the scheduler that the
     /// current thread is about to go to sleep, so it should schedule the target thread on the same
     /// CPU.
+    #[inline]
     pub fn notify_sync(&self) {
         // SAFETY: `wait_queue_head` points to valid memory.
         unsafe { bindings::__wake_up_sync(self.wait_queue_head.get(), TASK_NORMAL) };
@@ -225,6 +226,7 @@ impl CondVar {
     ///
     /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
     /// completely (as opposed to automatically waking up the next waiter).
+    #[inline]
     pub fn notify_one(&self) {
         self.notify(1);
     }
@@ -233,6 +235,7 @@ impl CondVar {
     ///
     /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
     /// completely (as opposed to automatically waking up the next waiter).
+    #[inline]
     pub fn notify_all(&self) {
         self.notify(0);
     }