selftests/nolibc: always rebuild the sysroot when running a test
Subsystems (2)
Willy Tarreau
maintainerPATCH 1/1 selftests/nolibc: always rebuild the sysroot when running a test
Paul and myself got trapped a few times by not seeing the effects of applying a patch to the nolibc source code until a "make clean" was issued in the nolibc directory. It's particularly annoying when trying to confirm that a proposed patch really solves a problem (or that reverting it reintroduces the problem).
The reason for the sysroot not being rebuilt was that it can be quite slow. But in fact it's only slow after a "make clean" issued at the kernel's topdir, because it's the main "make headers" that can take a tens of seconds; as long as "usr/include" still contains headers, the "headers_install" phase is only a quick "rsync", and rebuilding the whole nolibc sysroot takes a bit less than one second, which is perfectly acceptable for a test, even more once the time lost caused by misleading results if factored in.
This patch marks the sysroot target as phony and starts by clearing the previous sysroot for the current architecture before reinstalling it. Thanks to this, applying a patch to nolibc makes the effect immediately visible to "make nolibc-test":
$ time make -j -C tools/testing/selftests/nolibc nolibc-test
make: Entering directory '/k/tools/testing/selftests/nolibc'
MKDIR sysroot/x86/include
make[1]: Entering directory '/k/tools/include/nolibc'
make[2]: Entering directory '/k'
make[2]: Leaving directory '/k'
make[2]: Entering directory '/k'
INSTALL /k/tools/testing/selftests/nolibc/sysroot/sysroot/include
make[2]: Leaving directory '/k'
make[1]: Leaving directory '/k/tools/include/nolibc'
CC nolibc-test
make: Leaving directory '/k/tools/testing/selftests/nolibc'real 0m0.869s user 0m0.716s sys 0m0.149s
--- tools/testing/selftests/nolibc/Makefile | 3 +++ 1 file changed, 3 insertions(+)
tools/testing/selftests/nolibc/Makefile +3 -0
@@ -95,6 +95,7 @@ all: run sysroot: sysroot/$(ARCH)/include sysroot/$(ARCH)/include: + $(Q)rm -rf sysroot/$(ARCH) sysroot/sysroot $(QUIET_MKDIR)mkdir -p sysroot $(Q)$(MAKE) -C ../../../include/nolibc ARCH=$(ARCH) OUTPUT=$(CURDIR)/sysroot/ headers_standalone $(Q)mv sysroot/sysroot sysroot/$(ARCH)
@@ -133,3 +134,5 @@ clean: $(Q)rm -rf initramfs $(call QUIET_CLEAN, run.out) $(Q)rm -rf run.out + +.PHONY: sysroot/$(ARCH)/include
Signature · 1 line
2.35.3
Quoted 33 lines · Willy Tarreau
Paul and myself got trapped a few times by not seeing the effects of applying a patch to the nolibc source code until a "make clean" was issued in the nolibc directory. It's particularly annoying when trying to confirm that a proposed patch really solves a problem (or that reverting it reintroduces the problem).
The reason for the sysroot not being rebuilt was that it can be quite slow. But in fact it's only slow after a "make clean" issued at the kernel's topdir, because it's the main "make headers" that can take a tens of seconds; as long as "usr/include" still contains headers, the "
headers_install" phase is only a quick "rsync", and rebuilding the whole nolibc sysroot takes a bit less than one second, which is perfectly acceptable for a test, even more once the time lost caused by misleading results if factored in.This patch marks the sysroot target as phony and starts by clearing the previous sysroot for the current architecture before reinstalling it. Thanks to this, applying a patch to nolibc makes the effect immediately visible to "make nolibc-test":
$ time make -j -C tools/testing/selftests/nolibc nolibc-test make: Entering directory '/k/tools/testing/selftests/nolibc' MKDIR sysroot/x86/include make[1]: Entering directory '/k/tools/include/nolibc' make[2]: Entering directory '/k' make[2]: Leaving directory '/k' make[2]: Entering directory '/k' INSTALL /k/tools/testing/selftests/nolibc/sysroot/sysroot/include make[2]: Leaving directory '/k' make[1]: Leaving directory '/k/tools/include/nolibc' CC nolibc-test make: Leaving directory '/k/tools/testing/selftests/nolibc'
Quoted 7 lines
real 0m0.869s user 0m0.716s sys 0m0.149sCc "Paul E. McKenney" <[email protected]>Signed-off-by Willy Tarreau <[email protected]>
Works like a champ with reverting and unreverting c9388e0c1c6c ("tools/nolibc/string: Fix memcmp() implementation"), thank you!!!
I have queued this. I expect to push this into the next merge window, thus avoiding the need to document the need for "make clean" in my pull request. ;-)
Thanx, Paul
Quoted 25 lines
--- tools/testing/selftests/nolibc/Makefile | 3 +++ 1 file changed, 3 insertions(+)tools/testing/selftests/nolibc/Makefile +3
-0@@ -95,6 +95,7 @@ all: run sysroot: sysroot/$(ARCH)/include sysroot/$(ARCH)/include: + $(Q)rm -rf sysroot/$(ARCH) sysroot/sysroot $(QUIET_MKDIR)mkdir -p sysroot $(Q)$(MAKE) -C ../../../include/nolibc ARCH=$(ARCH) OUTPUT=$(CURDIR)/sysroot/ headers_standalone $(Q)mv sysroot/sysroot sysroot/$(ARCH)@@ -133,3 +134,5 @@ clean: $(Q)rm -rf initramfs $(call QUIET_CLEAN, run.out) $(Q)rm -rf run.out + +.PHONY: sysroot/$(ARCH)/includeSignature · 1 line
2.35.3
Willy Tarreau
maintainerQuoted 3 lines · Paul E. McKenney
On Wed, Oct 26, 2022 at 07:45:08AM +0200, Willy Tarreau wrote:
Works like a champ with reverting and unrevertingc9388e0c1c6c("tools/nolibc/string: Fixmemcmp()implementation"), thank you!!!
Thanks for the feedback, and glad it suits your needs as well. I hope that it will progressively encourage a few of us to enhance it with more tests.
Quoted 3 lines
I have queued this. I expect to push this into the next merge window, thus avoiding the need to document the need for "make clean" in my pull request. ;-)
Stupid question, is it really worth postponing it, considering that we've just introduced this series right now ? I mean, if the current usage is confusing, it's probably better to propose the fix before 6.1-final ? It's not a new feature here but rather a fix for a recently introduced one, thus I think it could be part of the next fix series. Rest assured I don't want to put a mess into your patch workflow, just asking :-)
Thanks!
Willy
Quoted 8 lines · Willy Tarreau
Quoted 3 lines · Paul E. McKenney
On Wed, Oct 26, 2022 at 07:45:08AM +0200, Willy Tarreau wrote:
Works like a champ with reverting and unrevertingc9388e0c1c6c("tools/nolibc/string: Fixmemcmp()implementation"), thank you!!!Thanks for the feedback, and glad it suits your needs as well. I hope that it will progressively encourage a few of us to enhance it with more tests.
Here is hoping! ;-)
Quoted 11 lines
Quoted 3 lines
I have queued this. I expect to push this into the next merge window, thus avoiding the need to document the need for "make clean" in my pull request. ;-)
Stupid question, is it really worth postponing it, considering that we've just introduced this series right now ? I mean, if the current usage is confusing, it's probably better to propose the fix before 6.1-final ? It's not a new feature here but rather a fix for a recently introduced one, thus I think it could be part of the next fix series. Rest assured I don't want to put a mess into your patch workflow, just asking :-)
You lost me here.
My intent is to push these nolicb patches into the upcoming v6.2 merge window:
2318a710bffbd tools/nolibc: Fix missing strlen() definition and infinite loop with gcc-12 6937b8de8f1c3 tools/nolibc/string: Fix memcmp() implementation e1bbfe393c900 selftests/nolibc: Add 7 tests for memcmp()3f2c1c45a3a9a selftests/nolibc: Always rebuild the sysroot when running a test
I didn't see the problem until I queued the third patch (e1bbfe393c900), and it is still in -rcu, not in v6.1.
What am I missing here?
Thanx, Paul
Willy Tarreau
maintainerQuoted 12 lines · Paul E. McKenney
Quoted 11 lines
Quoted 3 lines
I have queued this. I expect to push this into the next merge window, thus avoiding the need to document the need for "make clean" in my pull request. ;-)
Stupid question, is it really worth postponing it, considering that we've just introduced this series right now ? I mean, if the current usage is confusing, it's probably better to propose the fix before 6.1-final ? It's not a new feature here but rather a fix for a recently introduced one, thus I think it could be part of the next fix series. Rest assured I don't want to put a mess into your patch workflow, just asking :-)
Quoted 1 line
You lost me here.
Ah sorry!
Quoted 8 lines
My intent is to push these nolicb patches into the upcoming v6.2 merge window:
2318a710bffbdtools/nolibc: Fix missingstrlen()definition and infinite loop with gcc-126937b8de8f1c3tools/nolibc/string: Fixmemcmp()implementatione1bbfe393c900selftests/nolibc: Add 7 tests formemcmp()3f2c1c45a3a9aselftests/nolibc: Always rebuild the sysroot when running a test
Quoted 4 lines
I didn't see the problem until I queued the third patch (
e1bbfe393c900), and it is still in -rcu, not in v6.1.What am I missing here?
I thought that since some of them are fixes, they would be pushed during 6.1-rc so that we don't release 6.1 with known defects. For example Rasmus' fix for memcmp() or the strlen() fix would IMHO make sense for this release since we're aware of the bugs and we have the fixes. The 3rd one is indeed an addition and in no way a fix and it can easily wait for 6.2. The 4th one is more of a usability fix but I agree that for this last one it's debatable, I was mostly seeing this as a possiility to avoid causing needless confusion.
Hoping this clarifies my initial question.
Thanks,
Willy
Quoted 39 lines · Willy Tarreau
Quoted 12 lines · Paul E. McKenney
Quoted 11 lines
Quoted 3 lines
I have queued this. I expect to push this into the next merge window, thus avoiding the need to document the need for "make clean" in my pull request. ;-)
Stupid question, is it really worth postponing it, considering that we've just introduced this series right now ? I mean, if the current usage is confusing, it's probably better to propose the fix before 6.1-final ? It's not a new feature here but rather a fix for a recently introduced one, thus I think it could be part of the next fix series. Rest assured I don't want to put a mess into your patch workflow, just asking :-)
Quoted 1 line
You lost me here.
Ah sorry!
Quoted 8 lines
My intent is to push these nolicb patches into the upcoming v6.2 merge window:
2318a710bffbdtools/nolibc: Fix missingstrlen()definition and infinite loop with gcc-126937b8de8f1c3tools/nolibc/string: Fixmemcmp()implementatione1bbfe393c900selftests/nolibc: Add 7 tests formemcmp()3f2c1c45a3a9aselftests/nolibc: Always rebuild the sysroot when running a testQuoted 4 lines
I didn't see the problem until I queued the third patch (
e1bbfe393c900), and it is still in -rcu, not in v6.1.What am I missing here?
I thought that since some of them are fixes, they would be pushed during 6.1-rc so that we don't release 6.1 with known defects. For example Rasmus' fix for
memcmp()or thestrlen()fix would IMHO make sense for this release since we're aware of the bugs and we have the fixes. The 3rd one is indeed an addition and in no way a fix and it can easily wait for 6.2. The 4th one is more of a usability fix but I agree that for this last one it's debatable, I was mostly seeing this as a possiility to avoid causing needless confusion.
Quoted 1 line
Hoping this clarifies my initial question.
Very much so, thank you!
I was not considering the bug fixed by the first two patches to be serious, my mistake, apologies for my misclassification.
Given that background, I would rebase these two, test them, and send off a pull request, probably early next week.
2318a710bffbd tools/nolibc: Fix missing strlen() definition and infinite loop with gcc-12 6937b8de8f1c3 tools/nolibc/string: Fix memcmp() implementation
I would push the other two commits into the upcoming merge window.
Or might the discussion between you and Rasmus result in changes to either of those first two commits? If so, I should of course wait for that discussion to resolve.
Thanx, Paul
Willy Tarreau
maintainerQuoted 22 lines · Paul E. McKenney
Quoted 22 lines
Quoted 8 lines
My intent is to push these nolicb patches into the upcoming v6.2 merge window:
2318a710bffbdtools/nolibc: Fix missingstrlen()definition and infinite loop with gcc-126937b8de8f1c3tools/nolibc/string: Fixmemcmp()implementatione1bbfe393c900selftests/nolibc: Add 7 tests formemcmp()3f2c1c45a3a9aselftests/nolibc: Always rebuild the sysroot when running a testQuoted 4 lines
I didn't see the problem until I queued the third patch (
e1bbfe393c900), and it is still in -rcu, not in v6.1.What am I missing here?
I thought that since some of them are fixes, they would be pushed during 6.1-rc so that we don't release 6.1 with known defects. For example Rasmus' fix for
memcmp()or thestrlen()fix would IMHO make sense for this release since we're aware of the bugs and we have the fixes. The 3rd one is indeed an addition and in no way a fix and it can easily wait for 6.2. The 4th one is more of a usability fix but I agree that for this last one it's debatable, I was mostly seeing this as a possiility to avoid causing needless confusion.
Quoted 6 lines
Quoted 1 line
Hoping this clarifies my initial question.
Very much so, thank you!
I was not considering the bug fixed by the first two patches to be serious, my mistake, apologies for my misclassification.
No worries, I wasn't probably clear upfront about the purpose.
Quoted 5 lines
Given that background, I would rebase these two, test them, and send off a pull request, probably early next week.
2318a710bffbdtools/nolibc: Fix missingstrlen()definition and infinite loop with gcc-126937b8de8f1c3tools/nolibc/string: Fixmemcmp()implementation
Perfect, thank you!
Quoted 1 line
I would push the other two commits into the upcoming merge window.
OK!
Quoted 3 lines
Or might the discussion between you and Rasmus result in changes to either of those first two commits? If so, I should of course wait for that discussion to resolve.
We'll see, but in any case it would just be a minor detail, but I'll give you a quick response so that you don't have to deal with multiple versions of the patch, we all know that it's painful.
Thanks!
Willy
Quoted 44 lines · Willy Tarreau
Quoted 22 lines · Paul E. McKenney
Quoted 22 lines
Quoted 8 lines
My intent is to push these nolicb patches into the upcoming v6.2 merge window:
2318a710bffbdtools/nolibc: Fix missingstrlen()definition and infinite loop with gcc-126937b8de8f1c3tools/nolibc/string: Fixmemcmp()implementatione1bbfe393c900selftests/nolibc: Add 7 tests formemcmp()3f2c1c45a3a9aselftests/nolibc: Always rebuild the sysroot when running a testQuoted 4 lines
I didn't see the problem until I queued the third patch (
e1bbfe393c900), and it is still in -rcu, not in v6.1.What am I missing here?
I thought that since some of them are fixes, they would be pushed during 6.1-rc so that we don't release 6.1 with known defects. For example Rasmus' fix for
memcmp()or thestrlen()fix would IMHO make sense for this release since we're aware of the bugs and we have the fixes. The 3rd one is indeed an addition and in no way a fix and it can easily wait for 6.2. The 4th one is more of a usability fix but I agree that for this last one it's debatable, I was mostly seeing this as a possiility to avoid causing needless confusion.Quoted 6 lines
Quoted 1 line
Hoping this clarifies my initial question.
Very much so, thank you!
I was not considering the bug fixed by the first two patches to be serious, my mistake, apologies for my misclassification.
No worries, I wasn't probably clear upfront about the purpose.
Quoted 5 lines
Given that background, I would rebase these two, test them, and send off a pull request, probably early next week.
2318a710bffbdtools/nolibc: Fix missingstrlen()definition and infinite loop with gcc-126937b8de8f1c3tools/nolibc/string: Fixmemcmp()implementationPerfect, thank you!
Quoted 1 line
I would push the other two commits into the upcoming merge window.
OK!
Quoted 7 lines
Quoted 3 lines
Or might the discussion between you and Rasmus result in changes to either of those first two commits? If so, I should of course wait for that discussion to resolve.
We'll see, but in any case it would just be a minor detail, but I'll give you a quick response so that you don't have to deal with multiple versions of the patch, we all know that it's painful.
If I don't hear otherwise from you by the end of tomorrow (Friday), Pacific Time, I will rebase those two patches in preparation for sending a pull request for the regression. I will of course run the pull-message text past you before sending the pull request.
Thanx, Paul
Willy Tarreau
maintainerHi Paul,
Quoted 8 lines · Paul E. McKenney
Quoted 3 lines
We'll see, but in any case it would just be a minor detail, but I'll give you a quick response so that you don't have to deal with multiple versions of the patch, we all know that it's painful.
If I don't hear otherwise from you by the end of tomorrow (Friday), Pacific Time, I will rebase those two patches in preparation for sending a pull request for the regression. I will of course run the pull-message text past you before sending the pull request.
No news on this front, so feel free to pick what you already have.
Thank you!
Willy
Quoted 7 lines · Willy Tarreau
Hi Paul,
Quoted 4 lines · Paul E. McKenney
Quoted 3 lines
We'll see, but in any case it would just be a minor detail, but I'll give you a quick response so that you don't have to deal with multiple versions of the patch, we all know that it's painful.
Quoted 6 lines
Quoted 4 lines
If I don't hear otherwise from you by the end of tomorrow (Friday), Pacific Time, I will rebase those two patches in preparation for sending a pull request for the regression. I will of course run the pull-message text past you before sending the pull request.
No news on this front, so feel free to pick what you already have.
Very good, thank you!
I currently expect to send something like the pull request shown below early next week.
Thoughts?
Thanx, Paul
------------------------------------------------------------------------
Hello, Linus,
This pull request fixes a couple of nolibc string-function bugs noted by kernel test robot, Rasmus Villemoes, Willy Tarreau.
The following changes since commit 9abf2313adc1ca1b6180c508c25f22f9395cc780:
Linux 6.1-rc1 (2022-10-16 15:36:24 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git tags/nolibc-urgent.2022.10.28a
for you to fetch changes up to b3f4f51ea68a495f8a5956064c33dce711a2df91:
tools/nolibc/string: Fix memcmp() implementation (2022-10-28 15:07:02 -0700)
----------------------------------------------------------------
Urgent nolibc pull request for v6.1
This pull request contains a couple of commits that fix string-function bugs introduced by:
96980b833a21 ("tools/nolibc/string: do not use __builtin_strlen() at -O0") 66b6f755ad45 ("rcutorture: Import a copy of nolibc")
These appeared in v5.19 and v5.0, respectively, but it would be good to get these fixes in sooner rather than later. Commits providing the corresponding tests are in -rcu and I expect to submit them into the upcoming v6.2 merge window.
----------------------------------------------------------------
Rasmus Villemoes (1):
tools/nolibc/string: Fix memcmp() implementation
Willy Tarreau (1):
tools/nolibc: Fix missing strlen() definition and infinite loop with gcc-12
tools/include/nolibc/string.h | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
Willy Tarreau
maintainerQuoted 4 lines · Paul E. McKenney
I currently expect to send something like the pull request shown below early next week.
Thoughts?
That's perfect, thank you Paul!
Willy