Skip to contentSkip to conversation

selftests/nolibc: small simplification of test development phase

PATCH In review Willy Tarreau 3 participants 21 messages Jan 21, 2023 – Jan 23, 2023

Subsystems (2)
lore
Share & tools
lore

Willy Tarreau

author maintainer
#0 lore raw

cover selftests/nolibc: small simplification of test development phase

Hello Paul,

while developing and testing the recent changes for errno/environ/auxv, I found that I wasn't relying on the kernel that much and that I was mostly using qemu in userland only with my local kernel.

I figured that it was more convenient for this purpose than rebuilding an initramfs and kernel for a quick test, and decided to make this approach easier to use for everyone by adding a "run-user" target to the Makefile to do exactly this. E.g:

Native build:

  $ time make -C tools/testing/selftests/nolibc run-user
  ...
  make: Entering directory '/g/public/linux/master/tools/testing/selftests/nolibc'
    MKDIR   sysroot/x86/include
  make[1]: Entering directory '/g/public/linux/master/tools/include/nolibc'
  make[2]: Entering directory '/g/public/linux/master'
  make[2]: Leaving directory '/g/public/linux/master'
  make[2]: Entering directory '/g/public/linux/master'
    INSTALL /g/public/linux/master/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/g/public/linux/master'
  make[1]: Leaving directory '/g/public/linux/master/tools/include/nolibc'
    CC      nolibc-test
  18 chroot_root = -1 EPERM               [FAIL]
  43 link_dir = -1 EACCES  != (-1 EPERM)  [FAIL]
  See all results in /g/public/linux/master/tools/testing/selftests/nolibc/run.out
  make: Leaving directory '/g/public/linux/master/tools/testing/selftests/nolibc'
  real    0m0.966s
  user    0m0.731s
  sys     0m0.164s

Cross build:

  $ time make -C tools/testing/selftests/nolibc run-user ARCH=s390 CROSS_COMPILE=/f/tc/nolibc/gcc-11.3.0-nolibc/s390-linux/bin/s390-linux-
  make: Entering directory '/g/public/linux/master/tools/testing/selftests/nolibc'
    MKDIR   sysroot/s390/include
  make[1]: Entering directory '/g/public/linux/master/tools/include/nolibc'
  make[2]: Entering directory '/g/public/linux/master'
  make[2]: Leaving directory '/g/public/linux/master'
  make[2]: Entering directory '/g/public/linux/master'
    INSTALL /g/public/linux/master/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/g/public/linux/master'
  make[1]: Leaving directory '/g/public/linux/master/tools/include/nolibc'
    CC      nolibc-test
  18 chroot_root = -1 EPERM               [FAIL]
  43 link_dir = -1 EACCES  != (-1 EPERM)  [FAIL]
  See all results in /g/public/linux/master/tools/testing/selftests/nolibc/run.out
  make: Leaving directory '/g/public/linux/master/tools/testing/selftests/nolibc'
  real    0m1.014s
  user    0m0.732s
  sys     0m0.183s

In addition, the "x86_64" value for ARCH= is now supported as I got caught too many times with it not working in this subdir while it's used for the rest of the kernel ("x86" is used instead as coming from subarch.include). Generally you don't type it as x86_64 probably is the native build for most users, but when you start to test toolchains it's a different thing.

There's no matter of urgency for these patches, they're just a bit of user-friendly stuff. As such, if you're fine with stacking them on top of what you already have for 6.3, that will be great, otherwise they can easily wait.

Thank you!
Willy

[CCing Ammar who could benefit from this]

---
Willy Tarreau (2):
selftests/nolibc: support "x86_64" for arch name selftests/nolibc: add a "run-user" target to test the program in user
land

 tools/testing/selftests/nolibc/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)
Signature · 1 line
2.17.5
author maintainer
#1 lore raw

PATCH 1/2 selftests/nolibc: support "x86_64" for arch name

Building the kernel with ARCH=x86_64 is perfectly supported, but the nolibc-test only supports "x86", which causes errors when reusing existing build environment. Let's permit it to be used as well.

---
 tools/testing/selftests/nolibc/Makefile | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/nolibc/Makefile +4 -0
index 2bf613ee363d..423598045ff1 100644
@@ -14,6 +14,7 @@ endif
 
 # kernel image names by architecture
 IMAGE_i386    = arch/x86/boot/bzImage
+IMAGE_x86_64  = arch/x86/boot/bzImage
 IMAGE_x86     = arch/x86/boot/bzImage
 IMAGE_arm64   = arch/arm64/boot/Image
 IMAGE_arm     = arch/arm/boot/zImage
@@ -25,6 +26,7 @@ IMAGE_NAME    = $(notdir $(IMAGE))
 
 # default kernel configurations that appear to be usable
 DEFCONFIG_i386    = defconfig
+DEFCONFIG_x86_64  = defconfig
 DEFCONFIG_x86     = defconfig
 DEFCONFIG_arm64   = defconfig
 DEFCONFIG_arm     = multi_v7_defconfig
@@ -38,6 +40,7 @@ TEST =
 
 # QEMU_ARCH: arch names used by qemu
 QEMU_ARCH_i386    = i386
+QEMU_ARCH_x86_64  = x86_64
 QEMU_ARCH_x86     = x86_64
 QEMU_ARCH_arm64   = aarch64
 QEMU_ARCH_arm     = arm
@@ -48,6 +51,7 @@ QEMU_ARCH         = $(QEMU_ARCH_$(ARCH))
 
 # QEMU_ARGS : some arch-specific args to pass to qemu
 QEMU_ARGS_i386    = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
+QEMU_ARGS_x86_64  = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_x86     = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_arm64   = -M virt -cpu cortex-a53 -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_arm     = -M virt -append "panic=-1 $(TEST:%=NOLIBC_TEST=%)"
Signature · 1 line
2.17.5
author maintainer
#2 lore raw

PATCH 2/2 selftests/nolibc: add a "run-user" target to test the program in user land

In reply to Willy Tarreau

I found that when developing tests I'm not that often building kernels, and instead I'm often using the much quicker QEMU Linux emulator instead of the system emulator. It is sufficient to test startup code, stdlib code and syscall calling convention. As such it is expected that other users will value this, so let's add a "run-user" target that immediately executes the prebuilt executable with the user's privilege (thus some tests may fail due to insufficient permissions or missing features in the local kernel).

Now running a userland test is as simple as issuing:

make ARCH=xxx CROSS_COMPILE=xxx run-user

---
 tools/testing/selftests/nolibc/Makefile | 6 ++++++
 1 file changed, 6 insertions(+)
tools/testing/selftests/nolibc/Makefile +6 -0
index 423598045ff1..8fe61d3e3cce 100644
@@ -80,6 +80,7 @@ help:
 	@echo "  help         this help"
 	@echo "  sysroot      create the nolibc sysroot here (uses \$$ARCH)"
 	@echo "  nolibc-test  build the executable (uses \$$CC and \$$CROSS_COMPILE)"
+	@echo "  run-user     runs the executable under QEMU (uses \$$ARCH, \$$TEST)"
 	@echo "  initramfs    prepare the initramfs with nolibc-test"
 	@echo "  defconfig    create a fresh new default config (uses \$$ARCH)"
 	@echo "  kernel       (re)build the kernel with the initramfs (uses \$$ARCH)"
@@ -113,6 +114,11 @@ nolibc-test: nolibc-test.c sysroot/$(ARCH)/include
 	$(QUIET_CC)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ \
 	  -nostdlib -static -Isysroot/$(ARCH)/include $< -lgcc
 
+# qemu user-land test
+run-user: nolibc-test
+	$(Q)qemu-$(QEMU_ARCH) ./nolibc-test > "$(CURDIR)/run.out" || :
+	$(Q)grep -w FAIL "$(CURDIR)/run.out" && echo "See all results in $(CURDIR)/run.out" || echo "$$(grep -c ^[0-9].*OK $(CURDIR)/run.out) test(s) passed."
+
 initramfs: nolibc-test
 	$(QUIET_MKDIR)mkdir -p initramfs
 	$(call QUIET_INSTALL, initramfs/init)
Signature · 1 line
2.17.5
#3 lore raw
In reply to Willy Tarreau
Quoted 60 lines · Willy Tarreau

Hello Paul,

while developing and testing the recent changes for errno/environ/auxv, I found that I wasn't relying on the kernel that much and that I was mostly using qemu in userland only with my local kernel.

I figured that it was more convenient for this purpose than rebuilding an initramfs and kernel for a quick test, and decided to make this approach easier to use for everyone by adding a "run-user" target to the Makefile to do exactly this. E.g:

Native build:

  $ time make -C tools/testing/selftests/nolibc run-user
  ...
  make: Entering directory '/g/public/linux/master/tools/testing/selftests/nolibc'
    MKDIR   sysroot/x86/include
  make[1]: Entering directory '/g/public/linux/master/tools/include/nolibc'
  make[2]: Entering directory '/g/public/linux/master'
  make[2]: Leaving directory '/g/public/linux/master'
  make[2]: Entering directory '/g/public/linux/master'
    INSTALL /g/public/linux/master/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/g/public/linux/master'
  make[1]: Leaving directory '/g/public/linux/master/tools/include/nolibc'
    CC      nolibc-test
  18 chroot_root = -1 EPERM               [FAIL]
  43 link_dir = -1 EACCES  != (-1 EPERM)  [FAIL]
  See all results in /g/public/linux/master/tools/testing/selftests/nolibc/run.out
  make: Leaving directory '/g/public/linux/master/tools/testing/selftests/nolibc'
  real    0m0.966s
  user    0m0.731s
  sys     0m0.164s

Cross build:

  $ time make -C tools/testing/selftests/nolibc run-user ARCH=s390 CROSS_COMPILE=/f/tc/nolibc/gcc-11.3.0-nolibc/s390-linux/bin/s390-linux-
  make: Entering directory '/g/public/linux/master/tools/testing/selftests/nolibc'
    MKDIR   sysroot/s390/include
  make[1]: Entering directory '/g/public/linux/master/tools/include/nolibc'
  make[2]: Entering directory '/g/public/linux/master'
  make[2]: Leaving directory '/g/public/linux/master'
  make[2]: Entering directory '/g/public/linux/master'
    INSTALL /g/public/linux/master/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/g/public/linux/master'
  make[1]: Leaving directory '/g/public/linux/master/tools/include/nolibc'
    CC      nolibc-test
  18 chroot_root = -1 EPERM               [FAIL]
  43 link_dir = -1 EACCES  != (-1 EPERM)  [FAIL]
  See all results in /g/public/linux/master/tools/testing/selftests/nolibc/run.out
  make: Leaving directory '/g/public/linux/master/tools/testing/selftests/nolibc'
  real    0m1.014s
  user    0m0.732s
  sys     0m0.183s

In addition, the "x86_64" value for ARCH= is now supported as I got caught too many times with it not working in this subdir while it's used for the rest of the kernel ("x86" is used instead as coming from subarch.include). Generally you don't type it as x86_64 probably is the native build for most users, but when you start to test toolchains it's a different thing.

Quoted 7 lines

There's no matter of urgency for these patches, they're just a bit of user-friendly stuff. As such, if you're fine with stacking them on top of what you already have for 6.3, that will be great, otherwise they can easily wait.

Thank you!
Willy

Nice, thank you!

I have these placed on top of the -rcu "dev" branch initially for further review and testing. If things go well over the next week or so, I will set it up for the upcoming merge window.

One dependency is of course qemu-x86_64, so in the meantime I will figure out where I get that from. ;-)

Thanx, Paul

Quoted 14 lines

[CCing Ammar who could benefit from this]

---
Willy Tarreau (2):
selftests/nolibc: support "x86_64" for arch name selftests/nolibc: add a "run-user" target to test the program in user
land

 tools/testing/selftests/nolibc/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)
Signature · 1 line
2.17.5
author maintainer
#4 lore raw
Quoted 5 lines · Paul E. McKenney
Quoted 5 lines

There's no matter of urgency for these patches, they're just a bit of user-friendly stuff. As such, if you're fine with stacking them on top of what you already have for 6.3, that will be great, otherwise they can easily wait.

Quoted 8 lines
Quoted 2 lines

Thank you!
Willy

Nice, thank you!

I have these placed on top of the -rcu "dev" branch initially for further review and testing. If things go well over the next week or so, I will set it up for the upcoming merge window.

Thanks!

Quoted 2 lines

One dependency is of course qemu-x86_64, so in the meantime I will figure out where I get that from. ;-)

I build it from time to time from the sources, it's not that long and normally doesn't reserve me any surprises. But if you have it for other platforms it's likely that you have it for most platforms as well, including this one.

Best regards,
Willy

#5 lore raw
Quoted 17 lines · Willy Tarreau
Quoted 5 lines · Paul E. McKenney
Quoted 5 lines

There's no matter of urgency for these patches, they're just a bit of user-friendly stuff. As such, if you're fine with stacking them on top of what you already have for 6.3, that will be great, otherwise they can easily wait.

Quoted 8 lines
Quoted 2 lines

Thank you!
Willy

Nice, thank you!

I have these placed on top of the -rcu "dev" branch initially for further review and testing. If things go well over the next week or so, I will set it up for the upcoming merge window.

Thanks!

Quoted 7 lines
Quoted 2 lines

One dependency is of course qemu-x86_64, so in the meantime I will figure out where I get that from. ;-)

I build it from time to time from the sources, it's not that long and normally doesn't reserve me any surprises. But if you have it for other platforms it's likely that you have it for most platforms as well, including this one.

And building from sources proved to be reasonably easy, so the test now passes for me. My initial thought of putting qemu-x86_64 into my ~/bin directory fails the sudo test, but putting it into /usr/bin works fine.

Thank you for the hints!

Should I add a sentence to the commit log noting the potential need to build qemu from the git repo and to install qemu-x86_64, give or take what architecture one is running?

Thanx, Paul

author maintainer
#6 lore raw

Hello Paul,

Quoted 4 lines · Paul E. McKenney

And building from sources proved to be reasonably easy, so the test now passes for me. My initial thought of putting qemu-x86_64 into my ~/bin directory fails the sudo test, but putting it into /usr/bin works fine.

Great!

Quoted 5 lines

Thank you for the hints!

Should I add a sentence to the commit log noting the potential need to build qemu from the git repo and to install qemu-x86_64, give or take what architecture one is running?

Well, I've always had all the variants for all supported archs and didn't know that sometimes only part of them could be installed. I've used and tested qemu-{i386,x86_64,arm,aarch64,mips,s390x,riscv64} with this with success, and all of them are built by default for me. Thus I'm not seeing a good reason for making a special case of x86_64. Or maybe I'm missing the point ?

Thanks,
Willy

#7 lore raw
Quoted 16 lines · Willy Tarreau

Hello Paul,

Quoted 4 lines · Paul E. McKenney

And building from sources proved to be reasonably easy, so the test now passes for me. My initial thought of putting qemu-x86_64 into my ~/bin directory fails the sudo test, but putting it into /usr/bin works fine.

Great!

Quoted 5 lines

Thank you for the hints!

Should I add a sentence to the commit log noting the potential need to build qemu from the git repo and to install qemu-x86_64, give or take what architecture one is running?

Quoted 6 lines

Well, I've always had all the variants for all supported archs and didn't know that sometimes only part of them could be installed. I've used and tested qemu-{i386,x86_64,arm,aarch64,mips,s390x,riscv64} with this with success, and all of them are built by default for me. Thus I'm not seeing a good reason for making a special case of x86_64. Or maybe I'm missing the point ?

Fair point, and yes, I am showing the x86-centricity of my test environment. ;-) This might also apply to non-x86 distro setups, but I have no idea either way.

I would be OK leaving it as is and responding to problems if and when they actually occur. But you are quite correct, if we do add some sort of informative "Just build qemu!" diagnostic, it should be arch-independent.

Thanx, Paul

#8 lore raw
In reply to Willy Tarreau
Quoted 8 lines · Willy Tarreau
Quoted 2 lines · Paul E. McKenney

One dependency is of course qemu-x86_64, so in the meantime I will figure out where I get that from. ;-)

I build it from time to time from the sources, it's not that long and normally doesn't reserve me any surprises. But if you have it for other platforms it's likely that you have it for most platforms as well, including this one.

Willy,

Is there a way to make it work for the default qemu installation? Or maybe it's a mandatory requirement to build qemu from the source?

I use the qemu that comes from Ubuntu apt. I have "qemu-system-x86_64", but no "qemu-x86_64". So, something like this...

  $ which qemu-x86_64
  $ echo $?
  1
  $ which qemu-system-x86_64
  /usr/bin/qemu-system-x86_64
  $ time make run-user
    MKDIR   sysroot/x86/include
  make[1]: Entering directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
    INSTALL /home/ammarfaizi2/work/linux.work.cc/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[1]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
    CC      nolibc-test
  /bin/sh: 1: qemu-x86_64: not found
  0 test(s) passed.
  real    0m0.506s
  user    0m0.393s
  sys     0m0.086s

It would be great if we can avoid building qemu from the source. But if not, let's go with that.

Thanks!

Signature · 1 line
Ammar Faizi
#9 lore raw
Quoted 38 lines · Ammar Faizi
Quoted 8 lines · Willy Tarreau
Quoted 2 lines · Paul E. McKenney

One dependency is of course qemu-x86_64, so in the meantime I will figure out where I get that from. ;-)

I build it from time to time from the sources, it's not that long and normally doesn't reserve me any surprises. But if you have it for other platforms it's likely that you have it for most platforms as well, including this one.

Willy,

Is there a way to make it work for the default qemu installation? Or maybe it's a mandatory requirement to build qemu from the source?

I use the qemu that comes from Ubuntu apt. I have "qemu-system-x86_64", but no "qemu-x86_64". So, something like this...

  $ which qemu-x86_64
  $ echo $?
  1
  $ which qemu-system-x86_64
  /usr/bin/qemu-system-x86_64
  $ time make run-user
    MKDIR   sysroot/x86/include
  make[1]: Entering directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
    INSTALL /home/ammarfaizi2/work/linux.work.cc/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[1]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
    CC      nolibc-test
  /bin/sh: 1: qemu-x86_64: not found
  0 test(s) passed.
Quoted 6 lines
  real    0m0.506s
  user    0m0.393s
  sys     0m0.086s

It would be great if we can avoid building qemu from the source. But if not, let's go with that.

I had no problems building it from source.

One possible enhancement would be to make the error message tell the user to build from source, but I will let you guys debate the merits and drawbacks of that approach.

Thanx, Paul

author maintainer
#10 lore raw
In reply to Ammar Faizi

Hi Ammar,

Quoted 6 lines · Ammar Faizi

Is there a way to make it work for the default qemu installation? Or maybe it's a mandatory requirement to build qemu from the source?

I use the qemu that comes from Ubuntu apt. I have "qemu-system-x86_64", but no "qemu-x86_64". So, something like this...

Quoted 6 lines
  $ which qemu-x86_64
  $ echo $?
  1
  $ which qemu-system-x86_64
  /usr/bin/qemu-system-x86_64

Ah now I think I understand Paul's question. I didn't know that the userland version was not always provided. I've always had both side by side.

Quoted 2 lines

It would be great if we can avoid building qemu from the source. But if not, let's go with that.

As Paul indicated, it's really trouble-free and I think I've only done that since the very first day I started to use QEMU, reason why I probably never noticed that not everything was packaged.

Then at least to respond to Paul, it could make sense to add a note that on some distros the userland version might not always be provided and might require a pretty simple rebuild of QEMU.

Thanks!
Willy

#11 lore raw
Quoted 7 lines · Willy Tarreau

Ah now I think I understand Paul's question. I didn't know that the userland version was not always provided. I've always had both side by side.

Quoted 2 lines

It would be great if we can avoid building qemu from the source. But if not, let's go with that.

Quoted 7 lines

As Paul indicated, it's really trouble-free and I think I've only done that since the very first day I started to use QEMU, reason why I probably never noticed that not everything was packaged.

Then at least to respond to Paul, it could make sense to add a note that on some distros the userland version might not always be provided and might require a pretty simple rebuild of QEMU.

Ah well...

I figured it out. It turned qemu-user is a different package. So I have "qemu-system" installed, but not "qemu-user".

Without building from source, just do this on Ubuntu:

  $ sudo apt-get install qemu-user -y
  ...
  Preparing to unpack .../qemu-user_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
  Unpacking qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
  Selecting previously unselected package qemu-user-binfmt.
  Preparing to unpack .../qemu-user-binfmt_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
  Unpacking qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
  Setting up qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
  Setting up qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
  Processing triggers for man-db (2.10.2-1) ...
  $ which qemu-x86_64
  /usr/bin/qemu-x86_64
  $ sudo make run-user
  MKDIR   sysroot/x86/include
  make[1]: Entering directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
    INSTALL /home/ammarfaizi2/work/linux.work.cc/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[1]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
    CC      nolibc-test
  83 test(s) passed.

Sorry for that. I didn't know that they come from different packages. It works fine for me now.

Signature · 1 line
Ammar Faizi
author maintainer
#12 lore raw
Quoted 16 lines · Ammar Faizi

I figured it out. It turned qemu-user is a different package. So I have "qemu-system" installed, but not "qemu-user".

Without building from source, just do this on Ubuntu:

  $ sudo apt-get install qemu-user -y
  ...
  Preparing to unpack .../qemu-user_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
  Unpacking qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
  Selecting previously unselected package qemu-user-binfmt.
  Preparing to unpack .../qemu-user-binfmt_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
  Unpacking qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
  Setting up qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
  Setting up qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
  Processing triggers for man-db (2.10.2-1) ...
Quoted 2 lines
  $ which qemu-x86_64
  /usr/bin/qemu-x86_64

Ah cool! I looked for something like this on a remote ubuntu system but failed to figure it out by myself. Do you also have the other archs with it ?

Quoted 11 lines
  $ sudo make run-user
  MKDIR   sysroot/x86/include
  make[1]: Entering directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
    INSTALL /home/ammarfaizi2/work/linux.work.cc/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[1]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
    CC      nolibc-test
  83 test(s) passed.

Great. And without sudo you should see two of them fail. I intend to implement geteuid() and condition their result to it, that will be sufficient for most cases. I'd rather avoid seeing users run test programs under sudo actually, since they're run on the local system :-/

Quoted 2 lines

Sorry for that. I didn't know that they come from different packages. It works fine for me now.

No, don't be sorry, quite the opposite, you taught us something useful!

Thank you!
Willy

#13 lore raw
Quoted 3 lines · Willy Tarreau

Ah cool! I looked for something like this on a remote ubuntu system but failed to figure it out by myself. Do you also have the other archs with it ?

Yes, they come with many architectures.

$ ls /usr/bin/qemu-* | cat
/usr/bin/qemu-aarch64
/usr/bin/qemu-aarch64_be
/usr/bin/qemu-alpha
/usr/bin/qemu-arm
/usr/bin/qemu-armeb
/usr/bin/qemu-cris
/usr/bin/qemu-hexagon
/usr/bin/qemu-hppa
/usr/bin/qemu-i386
/usr/bin/qemu-img
/usr/bin/qemu-io
/usr/bin/qemu-m68k
/usr/bin/qemu-microblaze
/usr/bin/qemu-microblazeel
/usr/bin/qemu-mips
/usr/bin/qemu-mips64
/usr/bin/qemu-mips64el
/usr/bin/qemu-mipsel
/usr/bin/qemu-mipsn32
/usr/bin/qemu-mipsn32el
/usr/bin/qemu-nbd
/usr/bin/qemu-nios2
/usr/bin/qemu-or1k
/usr/bin/qemu-ppc
/usr/bin/qemu-ppc64
/usr/bin/qemu-ppc64le
/usr/bin/qemu-pr-helper
/usr/bin/qemu-riscv32
/usr/bin/qemu-riscv64
/usr/bin/qemu-s390x
/usr/bin/qemu-sh4
/usr/bin/qemu-sh4eb
/usr/bin/qemu-sparc
/usr/bin/qemu-sparc32plus
/usr/bin/qemu-sparc64
/usr/bin/qemu-storage-daemon
/usr/bin/qemu-system-i386
/usr/bin/qemu-system-x86_64
/usr/bin/qemu-system-x86_64-microvm
/usr/bin/qemu-system-x86_64-spice
/usr/bin/qemu-x86_64
/usr/bin/qemu-xtensa
/usr/bin/qemu-xtensaeb
Quoted 4 lines

Great. And without sudo you should see two of them fail. I intend to implement geteuid() and condition their result to it, that will be sufficient for most cases. I'd rather avoid seeing users run test programs under sudo actually, since they're run on the local system :-/

I see.

Signature · 1 line
Ammar Faizi
#14 lore raw
In reply to Ammar Faizi
Quoted 49 lines · Ammar Faizi
Quoted 7 lines · Willy Tarreau

Ah now I think I understand Paul's question. I didn't know that the userland version was not always provided. I've always had both side by side.

Quoted 2 lines

It would be great if we can avoid building qemu from the source. But if not, let's go with that.

Quoted 7 lines

As Paul indicated, it's really trouble-free and I think I've only done that since the very first day I started to use QEMU, reason why I probably never noticed that not everything was packaged.

Then at least to respond to Paul, it could make sense to add a note that on some distros the userland version might not always be provided and might require a pretty simple rebuild of QEMU.

Ah well...

I figured it out. It turned qemu-user is a different package. So I have "qemu-system" installed, but not "qemu-user".

Without building from source, just do this on Ubuntu:

  $ sudo apt-get install qemu-user -y
  ...
  Preparing to unpack .../qemu-user_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
  Unpacking qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
  Selecting previously unselected package qemu-user-binfmt.
  Preparing to unpack .../qemu-user-binfmt_1%3a6.2+dfsg-2ubuntu6.6_amd64.deb ...
  Unpacking qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
  Setting up qemu-user (1:6.2+dfsg-2ubuntu6.6) ...
  Setting up qemu-user-binfmt (1:6.2+dfsg-2ubuntu6.6) ...
  Processing triggers for man-db (2.10.2-1) ...
  $ which qemu-x86_64
  /usr/bin/qemu-x86_64
  $ sudo make run-user
  MKDIR   sysroot/x86/include
  make[1]: Entering directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[2]: Entering directory '/home/ammarfaizi2/work/linux.work.cc'
    INSTALL /home/ammarfaizi2/work/linux.work.cc/tools/testing/selftests/nolibc/sysroot/sysroot/include
  make[2]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc'
  make[1]: Leaving directory '/home/ammarfaizi2/work/linux.work.cc/tools/include/nolibc'
    CC      nolibc-test
  83 test(s) passed.
Quoted 2 lines

Sorry for that. I didn't know that they come from different packages. It works fine for me now.

I looked for that, but didn't find it, so thank you!

(Yes, I should have used dpkg, but I was lazy.)

Except that when I install Ubuntu 20.04's version, I get this:

------------------------------------------------------------------------

$ sudo make run-user
  MKDIR   sysroot/x86/include
make[1]: Entering directory '/home/git/linux-rcu/tools/include/nolibc'
make[2]: Entering directory '/home/git/linux-rcu'
make[2]: Leaving directory '/home/git/linux-rcu'
make[2]: Entering directory '/home/git/linux-rcu'
  INSTALL /home/git/linux-rcu/tools/testing/selftests/nolibc/sysroot/sysroot/include
make[2]: Leaving directory '/home/git/linux-rcu'
make[1]: Leaving directory '/home/git/linux-rcu/tools/include/nolibc'
  CC      nolibc-test
32 gettimeofday_null = -1 EFAULT        [FAIL]
See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out

------------------------------------------------------------------------

I have attached run.out.

In contrast, with my hand-built qemu-x86_64, all tests passed.

This might be just a version-related bug, but figured I should let you guys know.

Thanx, Paul

Running test 'syscall'

0 getpid = 2270744                       [OK]
1 getppid = 2270743                      [OK]
3 gettid = 2270744                       [OK]
5 getpgid_self = 2270584                 [OK]
6 getpgid_bad = -1 ESRCH                 [OK]
7 kill_0 = 0                             [OK]
8 kill_CONT = 0                          [OK]
9 kill_BADPID = -1 ESRCH                 [OK]
10 sbrk = 0                              [OK]
11 brk = 0                               [OK]
12 chdir_root = 0                        [OK]
13 chdir_dot = 0                         [OK]
14 chdir_blah = -1 ENOENT                [OK]
15 chmod_net = 0                         [OK]
16 chmod_self = -1 EPERM                 [OK]
17 chown_self = -1 EPERM                 [OK]
18 chroot_root = 0                       [OK]
19 chroot_blah = -1 ENOENT               [OK]
20 chroot_exe = -1 ENOTDIR               [OK]
21 close_m1 = -1 EBADF                   [OK]
22 close_dup = 0                         [OK]
23 dup_0 = 3                             [OK]
24 dup_m1 = -1 EBADF                     [OK]
25 dup2_0 = 100                          [OK]
26 dup2_m1 = -1 EBADF                    [OK]
27 dup3_0 = 100                          [OK]
28 dup3_m1 = -1 EBADF                    [OK]
29 execve_root = -1 EACCES               [OK]
30 getdents64_root = 672                 [OK]
31 getdents64_null = -1 ENOTDIR          [OK]
32 gettimeofday_null = -1 EFAULT        [FAIL]
34 gettimeofday_bad1 = -1 EFAULT         [OK]
35 gettimeofday_bad2 = -1 EFAULT         [OK]
36 gettimeofday_bad2 = -1 EFAULT         [OK]
38 getpagesize = 0                       [OK]
39 ioctl_tiocinq = 0                     [OK]
40 ioctl_tiocinq = 0                     [OK]
41 link_root1 = -1 EEXIST                [OK]
42 link_blah = -1 ENOENT                 [OK]
43 link_dir = -1 EPERM                   [OK]
44 link_cross = -1 EXDEV                 [OK]
45 lseek_m1 = -1 EBADF                   [OK]
46 lseek_0 = -1 ESPIPE                   [OK]
47 mkdir_root = -1 EEXIST                [OK]
48 open_tty = 3                          [OK]
49 open_blah = -1 ENOENT                 [OK]
50 poll_null = 0                         [OK]
51 poll_stdout = 1                       [OK]
52 poll_fault = -1 EFAULT                [OK]
53 read_badf = -1 EBADF                  [OK]
54 sched_yield = 0                       [OK]
55 select_null = 0                       [OK]
56 select_stdout = 1                     [OK]
57 select_fault = -1 EFAULT              [OK]
58 stat_blah = -1 ENOENT                 [OK]
59 stat_fault = -1 EFAULT                [OK]
60 symlink_root = -1 EEXIST              [OK]
61 unlink_root = -1 EISDIR               [OK]
62 unlink_blah = -1 ENOENT               [OK]
63 wait_child = -1 ECHILD                [OK]
64 waitpid_min = -1 ESRCH                [OK]
65 waitpid_child = -1 ECHILD             [OK]
66 write_badf = -1 EBADF                 [OK]
67 write_zero = 0                        [OK]

Errors during this test: 1

Running test 'stdlib'

0 getenv_TERM = <xterm>                  [OK]
1 getenv_blah = <(null)>                 [OK]
2 setcmp_blah_blah = 0                   [OK]
3 setcmp_blah_blah2 = -50                [OK]
4 setncmp_blah_blah = 0                  [OK]
5 setncmp_blah_blah4 = 0                 [OK]
6 setncmp_blah_blah5 = -53               [OK]
7 setncmp_blah_blah6 = -54               [OK]
8 strchr_foobar_o = <oobar>              [OK]
9 strchr_foobar_z = <(null)>             [OK]
10 strrchr_foobar_o = <obar>             [OK]
11 strrchr_foobar_z = <(null)>           [OK]
12 memcmp_20_20 = 0                      [OK]
13 memcmp_20_60 = -64                    [OK]
14 memcmp_60_20 = 64                     [OK]
15 memcmp_20_e0 = -192                   [OK]
16 memcmp_e0_20 = 192                    [OK]
17 memcmp_80_e0 = -96                    [OK]
18 memcmp_e0_80 = 96                     [OK]

Errors during this test: 0

Total number of errors: 1
Exiting with status 1

#15 lore raw
Quoted 21 lines · Paul E. McKenney

I looked for that, but didn't find it, so thank you!

(Yes, I should have used dpkg, but I was lazy.)

Except that when I install Ubuntu 20.04's version, I get this:

------------------------------------------------------------------------

$ sudo make run-user
  MKDIR   sysroot/x86/include
make[1]: Entering directory '/home/git/linux-rcu/tools/include/nolibc'
make[2]: Entering directory '/home/git/linux-rcu'
make[2]: Leaving directory '/home/git/linux-rcu'
make[2]: Entering directory '/home/git/linux-rcu'
  INSTALL /home/git/linux-rcu/tools/testing/selftests/nolibc/sysroot/sysroot/include
make[2]: Leaving directory '/home/git/linux-rcu'
make[1]: Leaving directory '/home/git/linux-rcu/tools/include/nolibc'
  CC      nolibc-test
32 gettimeofday_null = -1 EFAULT        [FAIL]
See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out
Quoted 8 lines

------------------------------------------------------------------------

I have attached run.out.

In contrast, with my hand-built qemu-x86_64, all tests passed.

This might be just a version-related bug, but figured I should let you guys know.

This is an interesting bug.

I'm a bit reluctant to say that this is a qemu bug. But I can't reproduce it on my machine. I use qemu that comes from Ubuntu 22.04.

FWIW, my qemu version is:

   $ qemu-x86_64 -version
   qemu-x86_64 version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.6)
   Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

I'll take a look at that gettimeofday() code and see if we have a bug in it. In meantime, could you send your qemu version info?

Signature · 1 line
Ammar Faizi
#16 lore raw
Quoted 36 lines · Ammar Faizi
Quoted 21 lines · Paul E. McKenney

I looked for that, but didn't find it, so thank you!

(Yes, I should have used dpkg, but I was lazy.)

Except that when I install Ubuntu 20.04's version, I get this:

------------------------------------------------------------------------

$ sudo make run-user
  MKDIR   sysroot/x86/include
make[1]: Entering directory '/home/git/linux-rcu/tools/include/nolibc'
make[2]: Entering directory '/home/git/linux-rcu'
make[2]: Leaving directory '/home/git/linux-rcu'
make[2]: Entering directory '/home/git/linux-rcu'
  INSTALL /home/git/linux-rcu/tools/testing/selftests/nolibc/sysroot/sysroot/include
make[2]: Leaving directory '/home/git/linux-rcu'
make[1]: Leaving directory '/home/git/linux-rcu/tools/include/nolibc'
  CC      nolibc-test
32 gettimeofday_null = -1 EFAULT        [FAIL]
See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out
Quoted 8 lines

------------------------------------------------------------------------

I have attached run.out.

In contrast, with my hand-built qemu-x86_64, all tests passed.

This might be just a version-related bug, but figured I should let you guys know.

This is an interesting bug.

I'm a bit reluctant to say that this is a qemu bug. But I can't reproduce it on my machine. I use qemu that comes from Ubuntu 22.04.

Quoted 8 lines

FWIW, my qemu version is:

   $ qemu-x86_64 -version
   qemu-x86_64 version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.6)
   Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

I'll take a look at that gettimeofday() code and see if we have a bug in it. In meantime, could you send your qemu version info?

Thank you for looking into this, and here you go:

    $ qemu-x86_64 -version
    qemu-x86_64 version 4.2.1 (Debian 1:4.2-3ubuntu6.24)
    Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

The version that I built by hand (which passed all tests) is as follows:

    $ /home/git/qemu/build/qemu-x86_64 -version
    qemu-x86_64 version 7.2.0 (v7.2.0)
    Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

Thanx, Paul

author maintainer
#17 lore raw
In reply to Paul E. McKenney
Quoted 17 lines · Paul E. McKenney

Except that when I install Ubuntu 20.04's version, I get this:

------------------------------------------------------------------------

$ sudo make run-user
  MKDIR   sysroot/x86/include
make[1]: Entering directory '/home/git/linux-rcu/tools/include/nolibc'
make[2]: Entering directory '/home/git/linux-rcu'
make[2]: Leaving directory '/home/git/linux-rcu'
make[2]: Entering directory '/home/git/linux-rcu'
  INSTALL /home/git/linux-rcu/tools/testing/selftests/nolibc/sysroot/sysroot/include
make[2]: Leaving directory '/home/git/linux-rcu'
make[1]: Leaving directory '/home/git/linux-rcu/tools/include/nolibc'
  CC      nolibc-test
32 gettimeofday_null = -1 EFAULT        [FAIL]
See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out
Quoted 8 lines

------------------------------------------------------------------------

I have attached run.out.

In contrast, with my hand-built qemu-x86_64, all tests passed.

This might be just a version-related bug, but figured I should let you guys know.

Interesting. Maybe something differs in the way it passes expectedly invalid pointers to some syscalls. Keep in mind that it's using your local kernel also, that could make a difference. I'm not that much keen on trying to investigate that one to be honest, given that this user mode is really meant to ease the life of test developers like Ammar and myself who just want to focus on the correctness of the test they're adding and not that much on the validity of the test itself in this context. I suggest we keep this one in mind without putting too much effort on it for now.

Thanks!
Willy

#18 lore raw
Quoted 27 lines · Willy Tarreau
Quoted 17 lines · Paul E. McKenney

Except that when I install Ubuntu 20.04's version, I get this:

------------------------------------------------------------------------

$ sudo make run-user
  MKDIR   sysroot/x86/include
make[1]: Entering directory '/home/git/linux-rcu/tools/include/nolibc'
make[2]: Entering directory '/home/git/linux-rcu'
make[2]: Leaving directory '/home/git/linux-rcu'
make[2]: Entering directory '/home/git/linux-rcu'
  INSTALL /home/git/linux-rcu/tools/testing/selftests/nolibc/sysroot/sysroot/include
make[2]: Leaving directory '/home/git/linux-rcu'
make[1]: Leaving directory '/home/git/linux-rcu/tools/include/nolibc'
  CC      nolibc-test
32 gettimeofday_null = -1 EFAULT        [FAIL]
See all results in /home/git/linux-rcu/tools/testing/selftests/nolibc/run.out
Quoted 8 lines

------------------------------------------------------------------------

I have attached run.out.

In contrast, with my hand-built qemu-x86_64, all tests passed.

This might be just a version-related bug, but figured I should let you guys know.

Quoted 9 lines

Interesting. Maybe something differs in the way it passes expectedly invalid pointers to some syscalls. Keep in mind that it's using your local kernel also, that could make a difference. I'm not that much keen on trying to investigate that one to be honest, given that this user mode is really meant to ease the life of test developers like Ammar and myself who just want to focus on the correctness of the test they're adding and not that much on the validity of the test itself in this context. I suggest we keep this one in mind without putting too much effort on it for now.

Indeed, it is easy for me to remove qemu-user and re-install my hand-built version. In fact, I just now did this and verified that everything now passes. ;-)

Thanx, Paul

Tested-by
#19 lore raw
In reply to Paul E. McKenney
Quoted 8 lines · Paul E. McKenney

Nice, thank you!

I have these placed on top of the -rcu "dev" branch initially for further review and testing. If things go well over the next week or so, I will set it up for the upcoming merge window.

One dependency is of course qemu-x86_64, so in the meantime I will figure out where I get that from. ;-)

Thank you all, I'll base my work on top of the -rcu "dev" branch.

Signature · 1 line
Ammar Faizi
#20 lore raw
Quoted 7 lines · Ammar Faizi
Quoted 6 lines · Paul E. McKenney

Nice, thank you!

I have these placed on top of the -rcu "dev" branch initially for further review and testing. If things go well over the next week or so, I will set it up for the upcoming merge window.

Quoted 6 lines
Quoted 2 lines

One dependency is of course qemu-x86_64, so in the meantime I will figure out where I get that from. ;-)

Thank you all, I'll base my work on top of the -rcu "dev" branch.

I will apply this on my next rebase, thank you!

Thanx, Paul