`scripts/rust_is_available.sh` improvements

From: Masahiro Yamada <[email protected]>

The -v option is passed when this script is invoked from Makefile, but not when invoked from Kconfig.

As you can see in scripts/Kconfig.include, the 'success' macro suppresses stdout and stderr anyway, so this script does not need to be quiet.

Signed-off-by Masahiro Yamada <[email protected]>

Reviewed-by Miguel Ojeda <[email protected]>

Tested-by Miguel Ojeda <[email protected]>

Reviewed-by Nathan Chancellor <[email protected]>

Link https://lore.kernel.org/r/[email protected] open here

[ Reworded prefix to match the others in the patch series. ]

Signed-off-by Miguel Ojeda <[email protected]>

---

From: Russell Currey <[email protected]>

rust_is_available.sh uses cc-version.sh to identify which C compiler is in use, as scripts/Kconfig.include does. cc-version.sh isn't designed to be able to handle multiple arguments in one variable, i.e. "ccache clang". Its invocation in rust_is_available.sh quotes "$CC", which makes $1 == "ccache clang" instead of the intended $1 == ccache & $2 == clang.

cc-version.sh could also be changed to handle having "ccache clang" as one argument, but it only has the one consumer upstream, making it simpler to fix the caller here.

Signed-off-by Russell Currey <[email protected]>

Fixes 78521f3399ab ("scripts: add `rust_is_available.sh`")

Link https://github.com/Rust-for-Linux/linux/pull/873

[ Reworded title prefix and reflow line to 75 columns. ]

Signed-off-by Miguel Ojeda <[email protected]>

---
[...]

From: Russell Currey <[email protected]>

rust_is_available.sh uses cc-version.sh to identify which C compiler is in use, as scripts/Kconfig.include does. cc-version.sh isn't designed to be able to handle multiple arguments in one variable, i.e. "ccache clang". Its invocation in rust_is_available.sh quotes "$CC", which makes $1 == "ccache clang" instead of the intended $1 == ccache & $2 == clang.

cc-version.sh could also be changed to handle having "ccache clang" as one argument, but it only has the one consumer upstream, making it simpler to fix the caller here.

Signed-off-by Russell Currey <[email protected]>

Fixes 78521f3399ab ("scripts: add `rust_is_available.sh`")

Link https://github.com/Rust-for-Linux/linux/pull/873

[ Reworded title prefix and reflow line to 75 columns. ]

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

scripts/rust_is_available.sh +2 -2

index f43a010eaf30..0c9be438e4cd 100755
@@ -113,10 +113,10 @@ fi
 #
 # In the future, we might be able to perform a full version check, see
 # https://github.com/rust-lang/rust-bindgen/issues/2138.
-cc_name=$($(dirname $0)/cc-version.sh "$CC" | cut -f1 -d' ')
+cc_name=$($(dirname $0)/cc-version.sh $CC | cut -f1 -d' ')
 if [ "$cc_name" = Clang ]; then
 	clang_version=$( \
-		LC_ALL=C "$CC" --version 2>/dev/null \
+		LC_ALL=C $CC --version 2>/dev/null \
 			| sed -nE '1s:.*version ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 	)
 	if [ "$clang_version" != "$bindgen_libclang_version" ]; then

Signature · 1 line

2.41.0

Sometimes users need to tweak the finding process of libclang for bindgen via the clang-sys-provided environment variables.

Thus add a paragraph to the setting up guide, including a reference to clang-sys's relevant documentation.

Link https://lore.kernel.org/rust-for-linux/CAKwvOdm5JT4wbdQQYuW+RT07rCi6whGBM2iUAyg8A1CmLXG6Nw@mail.gmail.com/ open here

Reviewed-by Nick Desaulniers <[email protected]>

Signed-off-by Miguel Ojeda <[email protected]>

---
[...]

Sometimes users need to tweak the finding process of libclang for bindgen via the clang-sys-provided environment variables.

Thus add a paragraph to the setting up guide, including a reference to clang-sys's relevant documentation.

Link https://lore.kernel.org/rust-for-linux/CAKwvOdm5JT4wbdQQYuW+RT07rCi6whGBM2iUAyg8A1CmLXG6Nw@mail.gmail.com/ open here

Reviewed-by Nick Desaulniers <[email protected]>

Signed-off-by Miguel Ojeda <[email protected]>

---
 Documentation/rust/quick-start.rst | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

Documentation/rust/quick-start.rst +17 -0

index 13b7744b1e27..a635be69e062 100644
@@ -100,6 +100,23 @@ Install it via (note that this will download and build the tool from source)::
 
 	cargo install --locked --version $(scripts/min-tool-version.sh bindgen) bindgen
 
+``bindgen`` needs to find a suitable ``libclang`` in order to work. If it is
+not found (or a different ``libclang`` than the one found should be used),
+the process can be tweaked using the environment variables understood by
+``clang-sys`` (the Rust bindings crate that ``bindgen`` uses to access
+``libclang``):
+
+* ``LLVM_CONFIG_PATH`` can be pointed to an ``llvm-config`` executable.
+
+* Or ``LIBCLANG_PATH`` can be pointed to a ``libclang`` shared library
+  or to the directory containing it.
+
+* Or ``CLANG_PATH`` can be pointed to a ``clang`` executable.
+
+For details, please see ``clang-sys``'s documentation at:
+
+	https://github.com/KyleMayes/clang-sys#environment-variables
+
 
 Requirements: Developing
 ------------------------

Signature · 1 line

2.41.0

People trying out the Rust support in the kernel may get warnings and errors from scripts/rust_is_available.sh from the rustavailable target or the build step.

Some of those users may be following the Quick Start guide, but others may not (likely those getting warnings from the build step instead of the target).

While the messages are fairly clear on what the problem is, it may not be clear how to solve the particular issue, especially for those not aware of the documentation.

We could add all sorts of details on the script for each one, but it is better to point users to the documentation instead, where it is easily readable in different formats. It also avoids duplication.

Thus add a reference to the documentation whenever the script fails or there is at least a warning.

Reviewed-by Finn Behrens <[email protected]>

Signed-off-by Miguel Ojeda <[email protected]>

---
[...]

People trying out the Rust support in the kernel may get warnings and errors from scripts/rust_is_available.sh from the rustavailable target or the build step.

Some of those users may be following the Quick Start guide, but others may not (likely those getting warnings from the build step instead of the target).

While the messages are fairly clear on what the problem is, it may not be clear how to solve the particular issue, especially for those not aware of the documentation.

We could add all sorts of details on the script for each one, but it is better to point users to the documentation instead, where it is easily readable in different formats. It also avoids duplication.

Thus add a reference to the documentation whenever the script fails or there is at least a warning.

Reviewed-by Finn Behrens <[email protected]>

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

scripts/rust_is_available.sh +17 -0

index 0c9be438e4cd..6b8131d5b547 100755
@@ -19,6 +19,20 @@ get_canonical_version()
 	echo $((100000 * $1 + 100 * $2 + $3))
 }
 
+# Print a reference to the Quick Start guide in the documentation.
+print_docs_reference()
+{
+	echo >&2 "***"
+	echo >&2 "*** Please see Documentation/rust/quick-start.rst for details"
+	echo >&2 "*** on how to set up the Rust support."
+	echo >&2 "***"
+}
+
+# If the script fails for any reason, or if there was any warning, then
+# print a reference to the documentation on exit.
+warning=0
+trap 'if [ $? -ne 0 ] || [ $warning -ne 0 ]; then print_docs_reference; fi' EXIT
+
 # Check that the Rust compiler exists.
 if ! command -v "$RUSTC" >/dev/null; then
 	echo >&2 "***"

@@ -60,6 +74,7 @@ if [ "$rust_compiler_cversion" -gt "$rust_compiler_min_cversion" ]; then
 	echo >&2 "***   Your version:     $rust_compiler_version"
 	echo >&2 "***   Expected version: $rust_compiler_min_version"
 	echo >&2 "***"
+	warning=1
 fi
 
 # Check that the Rust bindings generator is suitable.

@@ -87,6 +102,7 @@ if [ "$rust_bindings_generator_cversion" -gt "$rust_bindings_generator_min_cvers
 	echo >&2 "***   Your version:     $rust_bindings_generator_version"
 	echo >&2 "***   Expected version: $rust_bindings_generator_min_version"
 	echo >&2 "***"
+	warning=1
 fi
 
 # Check that the `libclang` used by the Rust bindings generator is suitable.

@@ -126,6 +142,7 @@ if [ "$cc_name" = Clang ]; then
 		echo >&2 "***   libclang version: $bindgen_libclang_version"
 		echo >&2 "***   Clang version:    $clang_version"
 		echo >&2 "***"
+		warning=1
 	fi
 fi
 

Signature · 1 line

2.41.0

People trying out the Rust support in the kernel may get warnings and errors from scripts/rust_is_available.sh from the rustavailable target or the build step.

Some of those users may be following the Quick Start guide, but others may not (likely those getting warnings from the build step instead of the target).

While the messages are fairly clear on what the problem is, it may not be clear how to solve the particular issue, especially for those not aware of the documentation.

We could add all sorts of details on the script for each one, but it is better to point users to the documentation instead, where it is easily readable in different formats. It also avoids duplication.

Thus add a reference to the documentation whenever the script fails or there is at least a warning.

Reviewed-by Finn Behrens <[email protected]>

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

scripts/rust_is_available.sh +3 -0

index 0c9be438e4cd..6b8131d5b547 100755
@@ -19,6 +19,20 @@ get_canonical_version()
        echo $((100000 * $1 + 100 * $2 + $3))
 }
 
+# Print a reference to the Quick Start guide in the documentation.
+print_docs_reference()
+{

scripts/rust_is_available.sh +4 -1

+
 # Check that the Rust compiler exists.
 if ! command -v "$RUSTC" >/dev/null; then
        echo >&2 "***"

@@ -60,6 +74,7 @@ if [ "$rust_compiler_cversion" -gt "$rust_compiler_min_cversion" ]; then
        echo >&2 "***   Your version:     $rust_compiler_version"
        echo >&2 "***   Expected version: $rust_compiler_min_version"
        echo >&2 "***"
+       warning=1
 fi
 
 # Check that the Rust bindings generator is suitable.

@@ -87,6 +102,7 @@ if [ "$rust_bindings_generator_cversion" -gt "$rust_bindings_generator_min_cvers
        echo >&2 "***   Your version:     $rust_bindings_generator_version"
        echo >&2 "***   Expected version: $rust_bindings_generator_min_version"
        echo >&2 "***"
+       warning=1
 fi
 
 # Check that the `libclang` used by the Rust bindings generator is suitable.

@@ -126,6 +142,7 @@ if [ "$cc_name" = Clang ]; then
                echo >&2 "***   libclang version: $bindgen_libclang_version"
                echo >&2 "***   Clang version:    $clang_version"
                echo >&2 "***"
+               warning=1
        fi
 fi
 
--

2.41.0

scripts/rust_is_available.sh calls bindgen with a special header in order to check whether the libclang version in use is suitable.

However, the invocation itself may fail if, for instance, bindgen cannot locate libclang. This is fine for Kconfig (since the script will still fail and therefore disable Rust as it should), but it is pretty confusing for users of the rustavailable target given the error will be unrelated:

     ./scripts/rust_is_available.sh: 21: arithmetic expression: expecting primary: "100000 *  + 100 *  + "
     make: *** [Makefile:1816: rustavailable] Error 2

Instead, run the bindgen invocation independently in a previous step, saving its output and return code. If it fails, then show the user a proper error message. Otherwise, continue as usual with the saved output.

Since the previous patch we show a reference to the docs, and the docs now explain how bindgen looks for libclang, thus the error message can leverage the documentation, avoiding duplication here (and making users aware of the setup guide in the documentation).

Reported-by Nick Desaulniers <[email protected]>

Link https://lore.kernel.org/rust-for-linux/CAKwvOdm5JT4wbdQQYuW+RT07rCi6whGBM2iUAyg8A1CmLXG6Nw@mail.gmail.com/ open here

Reported-by François Valenduc <[email protected]>

Closes https://github.com/Rust-for-Linux/linux/issues/934

Reported-by Alexandru Radovici <[email protected]>

Closes https://github.com/Rust-for-Linux/linux/pull/921

Reported-by Matthew Leach <[email protected]>

Closes https://lore.kernel.org/rust-for-linux/[email protected]/ open here

Fixes 78521f3399ab ("scripts: add `rust_is_available.sh`")

Signed-off-by Miguel Ojeda <[email protected]>

---
[...]

scripts/rust_is_available.sh calls bindgen with a special header in order to check whether the libclang version in use is suitable.

However, the invocation itself may fail if, for instance, bindgen cannot locate libclang. This is fine for Kconfig (since the script will still fail and therefore disable Rust as it should), but it is pretty confusing for users of the rustavailable target given the error will be unrelated:

    ./scripts/rust_is_available.sh: 21: arithmetic expression: expecting primary: "100000 *  + 100 *  + "
    make: *** [Makefile:1816: rustavailable] Error 2

Instead, run the bindgen invocation independently in a previous step, saving its output and return code. If it fails, then show the user a proper error message. Otherwise, continue as usual with the saved output.

Since the previous patch we show a reference to the docs, and the docs now explain how bindgen looks for libclang, thus the error message can leverage the documentation, avoiding duplication here (and making users aware of the setup guide in the documentation).

Reported-by Nick Desaulniers <[email protected]>

Link https://lore.kernel.org/rust-for-linux/CAKwvOdm5JT4wbdQQYuW+RT07rCi6whGBM2iUAyg8A1CmLXG6Nw@mail.gmail.com/ open here

Reported-by François Valenduc <[email protected]>

Closes https://github.com/Rust-for-Linux/linux/issues/934

Reported-by Alexandru Radovici <[email protected]>

Closes https://github.com/Rust-for-Linux/linux/pull/921

Reported-by Matthew Leach <[email protected]>

Closes https://lore.kernel.org/rust-for-linux/[email protected]/ open here

Fixes 78521f3399ab ("scripts: add `rust_is_available.sh`")

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

scripts/rust_is_available.sh +21 -1

index 6b8131d5b547..1bdff4472cbe 100755
@@ -106,8 +106,28 @@ if [ "$rust_bindings_generator_cversion" -gt "$rust_bindings_generator_min_cvers
 fi
 
 # Check that the `libclang` used by the Rust bindings generator is suitable.
+#
+# In order to do that, first invoke `bindgen` to get the `libclang` version
+# found by `bindgen`. This step may already fail if, for instance, `libclang`
+# is not found, thus inform the user in such a case.
+bindgen_libclang_output=$( \
+	LC_ALL=C "$BINDGEN" $(dirname $0)/rust_is_available_bindgen_libclang.h 2>&1 >/dev/null
+) || bindgen_libclang_code=$?
+if [ -n "$bindgen_libclang_code" ]; then
+	echo >&2 "***"
+	echo >&2 "*** Running '$BINDGEN' to check the libclang version (used by the Rust"
+	echo >&2 "*** bindings generator) failed with code $bindgen_libclang_code. This may be caused by"
+	echo >&2 "*** a failure to locate libclang. See output and docs below for details:"
+	echo >&2 "***"
+	echo >&2 "$bindgen_libclang_output"
+	echo >&2 "***"
+	exit 1
+fi
+
+# `bindgen` returned successfully, thus use the output to check that the version
+# of the `libclang` found by the Rust bindings generator is suitable.
 bindgen_libclang_version=$( \
-	LC_ALL=C "$BINDGEN" $(dirname $0)/rust_is_available_bindgen_libclang.h 2>&1 >/dev/null \
+	echo "$bindgen_libclang_output" \
 		| grep -F 'clang version ' \
 		| grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
 		| head -n 1 \

Signature · 1 line

2.41.0

scripts/rust_is_available.sh calls bindgen with a special header in order to check whether the libclang version in use is suitable.

However, the invocation itself may fail if, for instance, bindgen cannot locate libclang. This is fine for Kconfig (since the script will still fail and therefore disable Rust as it should), but it is pretty confusing for users of the rustavailable target given the error will be unrelated:

    ./scripts/rust_is_available.sh: 21: arithmetic expression: expecting primary: "100000 *  + 100 *  + "
    make: *** [Makefile:1816: rustavailable] Error 2

Instead, run the bindgen invocation independently in a previous step, saving its output and return code. If it fails, then show the user a proper error message. Otherwise, continue as usual with the saved output.

Since the previous patch we show a reference to the docs, and the docs now explain how bindgen looks for libclang, thus the error message can leverage the documentation, avoiding duplication here (and making users aware of the setup guide in the documentation).

Reported-by Nick Desaulniers <[email protected]>

Link https://lore.kernel.org/rust-for-linux/CAKwvOdm5JT4wbdQQYuW+RT07rCi6whGBM2iUAyg8A1CmLXG6Nw@mail.gmail.com/ open here

Reported-by François Valenduc <[email protected]>

Closes https://github.com/Rust-for-Linux/linux/issues/934

Reported-by Alexandru Radovici <[email protected]>

Closes https://github.com/Rust-for-Linux/linux/pull/921

Reported-by Matthew Leach <[email protected]>

Closes https://lore.kernel.org/rust-for-linux/[email protected]/ open here

Fixes 78521f3399ab ("scripts: add `rust_is_available.sh`")

Signed-off-by Miguel Ojeda <[email protected]>

---

 scripts/rust_is_available.sh | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

scripts/rust_is_available.sh +0 -0

index 6b8131d5b547..1bdff4472cbe 100755

scripts/rust_is_available.sh +15 -2

+) || bindgen_libclang_code=$?
+if [ -n "$bindgen_libclang_code" ]; then
+       echo >&2 "***"
+       echo >&2 "*** Running '$BINDGEN' to check the libclang version (used by the Rust"
+       echo >&2 "*** bindings generator) failed with code $bindgen_libclang_code. This may be caused by"
+       echo >&2 "*** a failure to locate libclang. See output and docs below for details:"
+       echo >&2 "***"
+       echo >&2 "$bindgen_libclang_output"
+       echo >&2 "***"
+       exit 1
+fi
+
+# `bindgen` returned successfully, thus use the output to check that the version
+# of the `libclang` found by the Rust bindings generator is suitable.
 bindgen_libclang_version=$( \
-       LC_ALL=C "$BINDGEN" $(dirname $0)/rust_is_available_bindgen_libclang.h 2>&1 >/dev/null \
+       echo "$bindgen_libclang_output" \
                | grep -F 'clang version ' \
                | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
                | head -n 1 \
--

2.41.0

Sometimes [1] users may attempt to setup the Rust support by checking what Kbuild does and they end up finding out about scripts/rust_is_available.sh. Inevitably, they run the script directly, but unless they setup the required variables, the result of the script is not meaningful.

We could add some defaults to the variables, but that could be confusing for those that may override the defaults (compared to their kernel builds), and $CC would not be a simple default in any case.

Therefore, instead, explicitly check whether the expected variables are set ($RUSTC, $BINDGEN and $CC). If not, print an explanation about the fact that the script is meant to be called from Kbuild, since that is the most likely cause for the variables not being set.

Link https://lore.kernel.org/oe-kbuild-all/[email protected]/ [1] open here

Signed-off-by Miguel Ojeda <[email protected]>

---
[...]

Sometimes [1] users may attempt to setup the Rust support by checking what Kbuild does and they end up finding out about scripts/rust_is_available.sh. Inevitably, they run the script directly, but unless they setup the required variables, the result of the script is not meaningful.

We could add some defaults to the variables, but that could be confusing for those that may override the defaults (compared to their kernel builds), and $CC would not be a simple default in any case.

Therefore, instead, explicitly check whether the expected variables are set ($RUSTC, $BINDGEN and $CC). If not, print an explanation about the fact that the script is meant to be called from Kbuild, since that is the most likely cause for the variables not being set.

Link https://lore.kernel.org/oe-kbuild-all/[email protected]/ [1] open here

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

scripts/rust_is_available.sh +29 -0

index 1bdff4472cbe..7e0368babe64 100755
@@ -28,11 +28,40 @@ print_docs_reference()
 	echo >&2 "***"
 }
 
+# Print an explanation about the fact that the script is meant to be called from Kbuild.
+print_kbuild_explanation()
+{
+	echo >&2 "***"
+	echo >&2 "*** This script is intended to be called from Kbuild."
+	echo >&2 "*** Please use the 'rustavailable' target to call it instead."
+	echo >&2 "*** Otherwise, the results may not be meaningful."
+	exit 1
+}
+
 # If the script fails for any reason, or if there was any warning, then
 # print a reference to the documentation on exit.
 warning=0
 trap 'if [ $? -ne 0 ] || [ $warning -ne 0 ]; then print_docs_reference; fi' EXIT
 
+# Check that the expected environment variables are set.
+if [ -z "${RUSTC+x}" ]; then
+	echo >&2 "***"
+	echo >&2 "*** Environment variable 'RUSTC' is not set."
+	print_kbuild_explanation
+fi
+
+if [ -z "${BINDGEN+x}" ]; then
+	echo >&2 "***"
+	echo >&2 "*** Environment variable 'BINDGEN' is not set."
+	print_kbuild_explanation
+fi
+
+if [ -z "${CC+x}" ]; then
+	echo >&2 "***"
+	echo >&2 "*** Environment variable 'CC' is not set."
+	print_kbuild_explanation
+fi
+
 # Check that the Rust compiler exists.
 if ! command -v "$RUSTC" >/dev/null; then
 	echo >&2 "***"

Signature · 1 line

2.41.0

Sometimes [1] users may attempt to setup the Rust support by checking what Kbuild does and they end up finding out about scripts/rust_is_available.sh. Inevitably, they run the script directly, but unless they setup the required variables, the result of the script is not meaningful.

We could add some defaults to the variables, but that could be confusing for those that may override the defaults (compared to their kernel builds), and $CC would not be a simple default in any case.

Therefore, instead, explicitly check whether the expected variables are set ($RUSTC, $BINDGEN and $CC). If not, print an explanation about the fact that the script is meant to be called from Kbuild, since that is the most likely cause for the variables not being set.

Link https://lore.kernel.org/oe-kbuild-all/[email protected]/ [1] open here

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

scripts/rust_is_available.sh +10 -0

index 1bdff4472cbe..7e0368babe64 100755
@@ -28,11 +28,40 @@ print_docs_reference()
        echo >&2 "***"
 }
 
+# Print an explanation about the fact that the script is meant to be called from Kbuild.
+print_kbuild_explanation()
+{
+       echo >&2 "***"
+       echo >&2 "*** This script is intended to be called from Kbuild."
+       echo >&2 "*** Please use the 'rustavailable' target to call it instead."
+       echo >&2 "*** Otherwise, the results may not be meaningful."
+       exit 1
+}
+
 # If the script fails for any reason, or if there was any warning, then

scripts/rust_is_available.sh +13 -1

+
+if [ -z "${BINDGEN+x}" ]; then
+       echo >&2 "***"
+       echo >&2 "*** Environment variable 'BINDGEN' is not set."
+       print_kbuild_explanation
+fi
+
+if [ -z "${CC+x}" ]; then
+       echo >&2 "***"
+       echo >&2 "*** Environment variable 'CC' is not set."
+       print_kbuild_explanation
+fi
+
 # Check that the Rust compiler exists.
 if ! command -v "$RUSTC" >/dev/null; then
        echo >&2 "***"
--

2.41.0

bindgen's output for libclang's version check contains paths, which in turn may contain strings that look like version numbers [1][2]:

     .../6.1.0-dev/.../rust_is_available_bindgen_libclang.h:2:9: warning: clang version 11.1.0  [-W#pragma-messages], err: false

which the script will pick up as the version instead of the latter.

It is also the case that versions may appear after the actual version (e.g. distribution's version text), which was the reason behind head [3]:

     .../rust-is-available-bindgen-libclang.h:2:9: warning: clang version 13.0.0 (Fedora 13.0.0-3.fc35) [-W#pragma-messages], err: false

Thus instead ask for a match after the clang version string.

Reported-by Jordan Isaacs <[email protected]>

Closes https://github.com/Rust-for-Linux/linux/issues/942 [1]

Reported-by Ethan D. Twardy <[email protected]>

Closes https://lore.kernel.org/rust-for-linux/[email protected]/ [2] open here

Reported-by Tiago Lam <[email protected]>

Closes https://github.com/Rust-for-Linux/linux/pull/789 [3]

Fixes 78521f3399ab ("scripts: add `rust_is_available.sh`")

Signed-off-by Miguel Ojeda <[email protected]>

---
[...]

bindgen's output for libclang's version check contains paths, which in turn may contain strings that look like version numbers [1][2]:

    .../6.1.0-dev/.../rust_is_available_bindgen_libclang.h:2:9: warning: clang version 11.1.0  [-W#pragma-messages], err: false

which the script will pick up as the version instead of the latter.

It is also the case that versions may appear after the actual version (e.g. distribution's version text), which was the reason behind head [3]:

    .../rust-is-available-bindgen-libclang.h:2:9: warning: clang version 13.0.0 (Fedora 13.0.0-3.fc35) [-W#pragma-messages], err: false

Thus instead ask for a match after the clang version string.

Reported-by Jordan Isaacs <[email protected]>

Closes https://github.com/Rust-for-Linux/linux/issues/942 [1]

Reported-by Ethan D. Twardy <[email protected]>

Closes https://lore.kernel.org/rust-for-linux/[email protected]/ [2] open here

Reported-by Tiago Lam <[email protected]>

Closes https://github.com/Rust-for-Linux/linux/pull/789 [3]

Fixes 78521f3399ab ("scripts: add `rust_is_available.sh`")

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

scripts/rust_is_available.sh +1 -3

index 7e0368babe64..810691af66eb 100755
@@ -157,9 +157,7 @@ fi
 # of the `libclang` found by the Rust bindings generator is suitable.
 bindgen_libclang_version=$( \
 	echo "$bindgen_libclang_output" \
-		| grep -F 'clang version ' \
-		| grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
-		| head -n 1 \
+		| sed -nE 's:.*clang version ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 )
 bindgen_libclang_min_version=$($min_tool_version llvm)
 bindgen_libclang_cversion=$(get_canonical_version $bindgen_libclang_version)

Signature · 1 line

2.41.0

---
 scripts/rust_is_available.sh | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

scripts/rust_is_available.sh +0 -0

index 7e0368babe64..810691af66eb 100755

In order to match the version string, sed is used in a couple cases, and grep and head in a couple others.

Make the script more consistent and easier to understand by using the same method, sed, for all of them.

This makes the version matching also a bit more strict for the changed cases, since the strings rustc and bindgen will now be required, which should be fine since rustc complains if one attempts to call it with another program name, and bindgen uses a hardcoded string.

In addition, clarify why one of the existing sed commands does not provide an address like the others.

Signed-off-by Miguel Ojeda <[email protected]>

---
[...]

In order to match the version string, sed is used in a couple cases, and grep and head in a couple others.

Make the script more consistent and easier to understand by using the same method, sed, for all of them.

This makes the version matching also a bit more strict for the changed cases, since the strings rustc and bindgen will now be required, which should be fine since rustc complains if one attempts to call it with another program name, and bindgen uses a hardcoded string.

In addition, clarify why one of the existing sed commands does not provide an address like the others.

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

scripts/rust_is_available.sh +5 -4

index 810691af66eb..b7e0781fdea9 100755
@@ -83,8 +83,7 @@ fi
 # Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
 rust_compiler_version=$( \
 	LC_ALL=C "$RUSTC" --version 2>/dev/null \
-		| head -n 1 \
-		| grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
+		| sed -nE '1s:.*rustc ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 )
 rust_compiler_min_version=$($min_tool_version rustc)
 rust_compiler_cversion=$(get_canonical_version $rust_compiler_version)

@@ -111,8 +110,7 @@ fi
 # Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
 rust_bindings_generator_version=$( \
 	LC_ALL=C "$BINDGEN" --version 2>/dev/null \
-		| head -n 1 \
-		| grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
+		| sed -nE '1s:.*bindgen ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 )
 rust_bindings_generator_min_version=$($min_tool_version bindgen)
 rust_bindings_generator_cversion=$(get_canonical_version $rust_bindings_generator_version)

@@ -155,6 +153,9 @@ fi
 
 # `bindgen` returned successfully, thus use the output to check that the version
 # of the `libclang` found by the Rust bindings generator is suitable.
+#
+# Unlike other version checks, note that this one does not necessarily appear
+# in the first line of the output, thus no `sed` address is provided.
 bindgen_libclang_version=$( \
 	echo "$bindgen_libclang_output" \
 		| sed -nE 's:.*clang version ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'

Signature · 1 line

2.41.0

In order to match the version string, sed is used in a couple cases, and grep and head in a couple others.

Make the script more consistent and easier to understand by using the same method, sed, for all of them.

This makes the version matching also a bit more strict for the changed cases, since the strings rustc and bindgen will now be required, which should be fine since rustc complains if one attempts to call it with another program name, and bindgen uses a hardcoded string.

In addition, clarify why one of the existing sed commands does not provide an address like the others.

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

scripts/rust_is_available.sh +5 -5

index 810691af66eb..b7e0781fdea9 100755
@@ -83,8 +83,7 @@ fi
 # Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
 rust_compiler_version=$( \
        LC_ALL=C "$RUSTC" --version 2>/dev/null \
-               | head -n 1 \
-               | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
+               | sed -nE '1s:.*rustc ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 )
 rust_compiler_min_version=$($min_tool_version rustc)
 rust_compiler_cversion=$(get_canonical_version $rust_compiler_version)

@@ -111,8 +110,7 @@ fi
 # Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
 rust_bindings_generator_version=$( \
        LC_ALL=C "$BINDGEN" --version 2>/dev/null \
-               | head -n 1 \
-               | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
+               | sed -nE '1s:.*bindgen ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 )
 rust_bindings_generator_min_version=$($min_tool_version bindgen)
 rust_bindings_generator_cversion=$(get_canonical_version $rust_bindings_generator_version)

@@ -155,6 +153,9 @@ fi
 
 # `bindgen` returned successfully, thus use the output to check that the version
 # of the `libclang` found by the Rust bindings generator is suitable.
+#
+# Unlike other version checks, note that this one does not necessarily appear
+# in the first line of the output, thus no `sed` address is provided.
 bindgen_libclang_version=$( \
        echo "$bindgen_libclang_output" \
                | sed -nE 's:.*clang version ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
--

2.41.0

The script already checks if $RUSTC and $BINDGEN exists via command, but the environment variables may point to a non-executable file, or the programs may fail for some other reason. While the script successfully exits with a failure as it should, the error given can be quite confusing depending on the shell and the behavior of its command. For instance, with dash:

     $ RUSTC=./mm BINDGEN=bindgen CC=clang scripts/rust_is_available.sh
     scripts/rust_is_available.sh: 19: arithmetic expression: expecting primary: "100000 *  + 100 *  + "

Thus detect failure exit codes when calling $RUSTC and $BINDGEN and print a better message, in a similar way to what we do when extracting the libclang version found by bindgen.

Link https://lore.kernel.org/rust-for-linux/CAK7LNAQYk6s11MASRHW6oxtkqF00EJVqhHOP=5rynWt-QDUsXw@mail.gmail.com/ open here

Signed-off-by Miguel Ojeda <[email protected]>

---
[...]

The script already checks if $RUSTC and $BINDGEN exists via command, but the environment variables may point to a non-executable file, or the programs may fail for some other reason. While the script successfully exits with a failure as it should, the error given can be quite confusing depending on the shell and the behavior of its command. For instance, with dash:

    $ RUSTC=./mm BINDGEN=bindgen CC=clang scripts/rust_is_available.sh
    scripts/rust_is_available.sh: 19: arithmetic expression: expecting primary: "100000 *  + 100 *  + "

Thus detect failure exit codes when calling $RUSTC and $BINDGEN and print a better message, in a similar way to what we do when extracting the libclang version found by bindgen.

Link https://lore.kernel.org/rust-for-linux/CAK7LNAQYk6s11MASRHW6oxtkqF00EJVqhHOP=5rynWt-QDUsXw@mail.gmail.com/ open here

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 28 ++++++++++++++++++++++++++--
 1 file changed, 26 insertions(+), 2 deletions(-)

scripts/rust_is_available.sh +26 -2

index b7e0781fdea9..da8296cd9b8d 100755
@@ -81,8 +81,20 @@ fi
 # Check that the Rust compiler version is suitable.
 #
 # Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
+rust_compiler_output=$( \
+	LC_ALL=C "$RUSTC" --version 2>/dev/null
+) || rust_compiler_code=$?
+if [ -n "$rust_compiler_code" ]; then
+	echo >&2 "***"
+	echo >&2 "*** Running '$RUSTC' to check the Rust compiler version failed with"
+	echo >&2 "*** code $rust_compiler_code. See output and docs below for details:"
+	echo >&2 "***"
+	echo >&2 "$rust_compiler_output"
+	echo >&2 "***"
+	exit 1
+fi
 rust_compiler_version=$( \
-	LC_ALL=C "$RUSTC" --version 2>/dev/null \
+	echo "$rust_compiler_output" \
 		| sed -nE '1s:.*rustc ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 )
 rust_compiler_min_version=$($min_tool_version rustc)

@@ -108,8 +120,20 @@ fi
 # Check that the Rust bindings generator is suitable.
 #
 # Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
+rust_bindings_generator_output=$( \
+	LC_ALL=C "$BINDGEN" --version 2>/dev/null
+) || rust_bindings_generator_code=$?
+if [ -n "$rust_bindings_generator_code" ]; then
+	echo >&2 "***"
+	echo >&2 "*** Running '$BINDGEN' to check the Rust bindings generator version failed with"
+	echo >&2 "*** code $rust_bindings_generator_code. See output and docs below for details:"
+	echo >&2 "***"
+	echo >&2 "$rust_bindings_generator_output"
+	echo >&2 "***"
+	exit 1
+fi
 rust_bindings_generator_version=$( \
-	LC_ALL=C "$BINDGEN" --version 2>/dev/null \
+	echo "$rust_bindings_generator_output" \
 		| sed -nE '1s:.*bindgen ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 )
 rust_bindings_generator_min_version=$($min_tool_version bindgen)

Signature · 1 line

2.41.0

The script already checks for $RUSTC and $BINDGEN existing and exiting without failure. However, one may still pass an unexpected binary that does not output what the later parsing expects. The script still successfully reports a failure as expected, but the error is confusing. For instance:

     $ RUSTC=true BINDGEN=bindgen CC=clang scripts/rust_is_available.sh
     scripts/rust_is_available.sh: 19: arithmetic expression: expecting primary: "100000 *  + 100 *  + "
     ***
     *** Please see Documentation/rust/quick-start.rst for details
     *** on how to set up the Rust support.
     ***

Thus add an explicit check and a proper message for unexpected output from the called command.

Similarly, do so for the libclang version parsing, too.

Link https://lore.kernel.org/rust-for-linux/CAK7LNAQYk6s11MASRHW6oxtkqF00EJVqhHOP=5rynWt-QDUsXw@mail.gmail.com/ open here

Signed-off-by Miguel Ojeda <[email protected]>

---
[...]

The script already checks for $RUSTC and $BINDGEN existing and exiting without failure. However, one may still pass an unexpected binary that does not output what the later parsing expects. The script still successfully reports a failure as expected, but the error is confusing. For instance:

    $ RUSTC=true BINDGEN=bindgen CC=clang scripts/rust_is_available.sh
    scripts/rust_is_available.sh: 19: arithmetic expression: expecting primary: "100000 *  + 100 *  + "
    ***
    *** Please see Documentation/rust/quick-start.rst for details
    *** on how to set up the Rust support.
    ***

Thus add an explicit check and a proper message for unexpected output from the called command.

Similarly, do so for the libclang version parsing, too.

Link https://lore.kernel.org/rust-for-linux/CAK7LNAQYk6s11MASRHW6oxtkqF00EJVqhHOP=5rynWt-QDUsXw@mail.gmail.com/ open here

Signed-off-by Miguel Ojeda <[email protected]>

---
 scripts/rust_is_available.sh | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

scripts/rust_is_available.sh +28 -0

index da8296cd9b8d..117018946b57 100755
@@ -97,6 +97,15 @@ rust_compiler_version=$( \
 	echo "$rust_compiler_output" \
 		| sed -nE '1s:.*rustc ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 )
+if [ -z "$rust_compiler_version" ]; then
+	echo >&2 "***"
+	echo >&2 "*** Running '$RUSTC' to check the Rust compiler version did not return"
+	echo >&2 "*** an expected output. See output and docs below for details:"
+	echo >&2 "***"
+	echo >&2 "$rust_compiler_output"
+	echo >&2 "***"
+	exit 1
+fi
 rust_compiler_min_version=$($min_tool_version rustc)
 rust_compiler_cversion=$(get_canonical_version $rust_compiler_version)
 rust_compiler_min_cversion=$(get_canonical_version $rust_compiler_min_version)

@@ -136,6 +145,15 @@ rust_bindings_generator_version=$( \
 	echo "$rust_bindings_generator_output" \
 		| sed -nE '1s:.*bindgen ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 )
+if [ -z "$rust_bindings_generator_version" ]; then
+	echo >&2 "***"
+	echo >&2 "*** Running '$BINDGEN' to check the bindings generator version did not return"
+	echo >&2 "*** an expected output. See output and docs below for details:"
+	echo >&2 "***"
+	echo >&2 "$rust_bindings_generator_output"
+	echo >&2 "***"
+	exit 1
+fi
 rust_bindings_generator_min_version=$($min_tool_version bindgen)
 rust_bindings_generator_cversion=$(get_canonical_version $rust_bindings_generator_version)
 rust_bindings_generator_min_cversion=$(get_canonical_version $rust_bindings_generator_min_version)

@@ -184,6 +202,16 @@ bindgen_libclang_version=$( \
 	echo "$bindgen_libclang_output" \
 		| sed -nE 's:.*clang version ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
 )
+if [ -z "$bindgen_libclang_version" ]; then
+	echo >&2 "***"
+	echo >&2 "*** Running '$BINDGEN' to check the libclang version (used by the Rust"
+	echo >&2 "*** bindings generator) did not return an expected output. See output"
+	echo >&2 "*** and docs below for details:"
+	echo >&2 "***"
+	echo >&2 "$bindgen_libclang_output"
+	echo >&2 "***"
+	exit 1
+fi
 bindgen_libclang_min_version=$($min_tool_version llvm)
 bindgen_libclang_cversion=$(get_canonical_version $bindgen_libclang_version)
 bindgen_libclang_min_cversion=$(get_canonical_version $bindgen_libclang_min_version)

Signature · 1 line

2.41.0

The rust_is_available.sh script runs for everybody compiling the kernel, even if not using Rust. Therefore, it is important to ensure that the script is correct to avoid breaking people's compilation.

In addition, the script needs to be able to handle a set of subtle cases, including parsing version strings of different tools.

Therefore, maintenance of this script can be greatly eased with a set of tests.

Thus add a test suite to cover hopefully most of the setups that the script may encounter in the wild. Extra setups can be easily added later on if missing.

The script currently covers all the branches of the shell script, including several ways in which they may be entered.

Python is used for this script, since the script under test does not depend on Rust, thus hopefully making it easier for others to use if the need arises.

Signed-off-by Miguel Ojeda <[email protected]>

---
[...]

This is the patch series to improve scripts/rust_is_available.sh.

The major addition in v2 is the test suite in the last commit. I added it because I wanted to have a proper way to test any further changes to it (such as the suggested set -- idea to avoid forking by Masahiro), and so that adding new checks was easier to justify too (i.e. vs. the added complexity).

In addition, there are also a few new checks in the script, to cover for even some more cases, which hopefully make problematic setups easier to identify and solve by users building the kernel. For instance, running the script externally gives:

    $ scripts/rust_is_available.sh
    ***
    *** Environment variable 'RUSTC' is not set.
    ***
    *** This script is intended to be called from Kbuild.
    *** Please use the 'rustavailable' target to call it instead.
    *** Otherwise, the results may not be meaningful.
    ***
    *** Please see Documentation/rust/quick-start.rst for details
    *** on how to set up the Rust support.
    ***

I also changed it to avoid setting -e as Masahiro suggested. Similarly, I now check for $RUSTC, $BINDGEN and $CC, instead of $MAKEFLAGS, as he also suggested (but I gave it their own error message rather than use the ${CC?: is not set} approach. This goes in line with the reasons outlined above, i.e. trying to give users a clear error of what step exactly failed).

In the test suite I included previously problematic compiler version strings we got reports for. The test suite covers all current branches in the script, and we should keep it that way in the future.

The patch series also include Masahiro's patch to remove the -v option, as well as Russell's patch for supporting multiple arguments in $CC.

All in all, this should solve all the issues we got so far (unless I have missed something) and improve things further with the new checks plus the test suite to hopefully have an easier time in the future.

Testers for this one are appreciated, especially if you have uncommon or custom setups for building the kernel.

This could go through either the Kbuild or the Rust tree.

Masahiro Yamada (1):
kbuild: rust_is_available: remove -v option

Miguel Ojeda (9):
docs: rust: add paragraph about finding a suitable libclang
kbuild: rust_is_available: print docs reference
kbuild: rust_is_available: add check for bindgen invocation
kbuild: rust_is_available: check that environment variables are set
kbuild: rust_is_available: fix confusion when a version appears in the
path
kbuild: rust_is_available: normalize version matching
kbuild: rust_is_available: handle failures calling $RUSTC/$BINDGEN
kbuild: rust_is_available: check that output looks as expected
kbuild: rust_is_available: add test suite

Russell Currey (1):
kbuild: rust_is_available: fix version check when CC has multiple
arguments

 Documentation/rust/quick-start.rst |  17 ++
 Makefile                           |   4 +-
 scripts/rust_is_available.sh       | 233 +++++++++++++------
 scripts/rust_is_available_test.py  | 346 +++++++++++++++++++++++++++++
 4 files changed, 532 insertions(+), 68 deletions(-)
 create mode 100755 scripts/rust_is_available_test.py

base-commit: 858fd168a95c5b9669aac8db6c14a9aeab446375 --
2.41.0

This is the patch series to improve scripts/rust_is_available.sh.

The major addition in v2 is the test suite in the last commit. I added it because I wanted to have a proper way to test any further changes to it (such as the suggested set -- idea to avoid forking by Masahiro), and so that adding new checks was easier to justify too (i.e. vs. the added complexity).

In addition, there are also a few new checks in the script, to cover for even some more cases, which hopefully make problematic setups easier to identify and solve by users building the kernel. For instance, running the script externally gives:

    $ scripts/rust_is_available.sh
    ***
    *** Environment variable 'RUSTC' is not set.
    ***
    *** This script is intended to be called from Kbuild.
    *** Please use the 'rustavailable' target to call it instead.
    *** Otherwise, the results may not be meaningful.
    ***
    *** Please see Documentation/rust/quick-start.rst for details
    *** on how to set up the Rust support.
    ***

I also changed it to avoid setting -e as Masahiro suggested. Similarly, I now check for $RUSTC, $BINDGEN and $CC, instead of $MAKEFLAGS, as he also suggested (but I gave it their own error message rather than use the ${CC?: is not set} approach. This goes in line with the reasons outlined above, i.e. trying to give users a clear error of what step exactly failed).

In the test suite I included previously problematic compiler version strings we got reports for. The test suite covers all current branches in the script, and we should keep it that way in the future.

The patch series also include Masahiro's patch to remove the -v option, as well as Russell's patch for supporting multiple arguments in $CC.

All in all, this should solve all the issues we got so far (unless I have missed something) and improve things further with the new checks plus the test suite to hopefully have an easier time in the future.

Testers for this one are appreciated, especially if you have uncommon or custom setups for building the kernel.

This could go through either the Kbuild or the Rust tree.

This is the patch series to improve scripts/rust_is_available.sh.

The major addition in v2 is the test suite in the last commit. I added it because I wanted to have a proper way to test any further changes to it (such as the suggested set -- idea to avoid forking by Masahiro), and so that adding new checks was easier to justify too (i.e. vs. the added complexity).

In addition, there are also a few new checks in the script, to cover for even some more cases, which hopefully make problematic setups easier to identify and solve by users building the kernel. For instance, running the script externally gives:

    $ scripts/rust_is_available.sh
    ***
    *** Environment variable 'RUSTC' is not set.
    ***
    *** This script is intended to be called from Kbuild.
    *** Please use the 'rustavailable' target to call it instead.
    *** Otherwise, the results may not be meaningful.
    ***
    *** Please see Documentation/rust/quick-start.rst for details
    *** on how to set up the Rust support.
    ***

I also changed it to avoid setting -e as Masahiro suggested. Similarly, I now check for $RUSTC, $BINDGEN and $CC, instead of $MAKEFLAGS, as he also suggested (but I gave it their own error message rather than use the ${CC?: is not set} approach. This goes in line with the reasons outlined above, i.e. trying to give users a clear error of what step exactly failed).

In the test suite I included previously problematic compiler version strings we got reports for. The test suite covers all current branches in the script, and we should keep it that way in the future.

The patch series also include Masahiro's patch to remove the -v option, as well as Russell's patch for supporting multiple arguments in $CC.

All in all, this should solve all the issues we got so far (unless I have missed something) and improve things further with the new checks plus the test suite to hopefully have an easier time in the future.

Testers for this one are appreciated, especially if you have uncommon or custom setups for building the kernel.

This could go through either the Kbuild or the Rust tree.