Skip to contentSkip to conversation

OpenBSD fixes

PATCH In review Philippe Mathieu-Daudé 5 participants 14 messages Jan 25, 2019 – Jan 28, 2019

lore
Share & tools
lore
#0 lore raw

cover OpenBSD fixes

Fixes I encountered while trying to run QEMU test suite on OpenBSD. More to come, but please review.

Regards,

Phil.

Philippe Mathieu-Daudé (3):
configure: Disable W^X on OpenBSD
XXX oslib-posix: Ignore fcntl("/dev/null", F_SETFL, O_NONBLOCK)
failure
WIP tests/vm: Run tests on OpenBSD

 configure          | 11 +++++++++++
 tests/vm/openbsd   |  4 +---
 util/oslib-posix.c |  2 ++
 3 files changed, 14 insertions(+), 3 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/3 configure: Disable W^X on OpenBSD

Since OpenBSD 6.0 [1], W^X is enforced by default [2]. TCG requires WX access. Disable W^X if it is available. This fixes:

  # lm32-softmmu/qemu-system-lm32
  Could not allocate dynamic translator buffer
  # sysctl kern.wxabort=1
  kern.wxabort: 0 -> 1
  # lm32-softmmu/qemu-system-lm32
  mmap: Not supported
  Abort trap (core dumped)
  # gdb -q lm32-softmmu/qemu-system-lm32 qemu-system-lm32.core
  (gdb) bt
  #0  0x000017e3c156c50a in _thread_sys___syscall () at {standard input}:5
  #1  0x000017e3c15e5d7a in *_libc_mmap (addr=Variable "addr" is not available.) at /usr/src/lib/libc/sys/mmap.c:47
  #2  0x000017e17d9abc8b in alloc_code_gen_buffer () at /usr/src/qemu/accel/tcg/translate-all.c:1064
  #3  0x000017e17d9abd04 in code_gen_alloc (tb_size=0) at /usr/src/qemu/accel/tcg/translate-all.c:1112
  #4  0x000017e17d9abe81 in tcg_exec_init (tb_size=0) at /usr/src/qemu/accel/tcg/translate-all.c:1149
  #5  0x000017e17d9897e9 in tcg_init (ms=0x17e45e456800) at /usr/src/qemu/accel/tcg/tcg-all.c:66
  #6  0x000017e17d9891b8 in accel_init_machine (acc=0x17e3c3f50800, ms=0x17e45e456800) at /usr/src/qemu/accel/accel.c:63
  #7  0x000017e17d989312 in configure_accelerator (ms=0x17e45e456800, progname=0x7f7fffff07b0 "lm32-softmmu/qemu-system-lm32") at /usr/src/qemu/accel/accel.c:111
  #8  0x000017e17d9d8616 in main (argc=1, argv=0x7f7fffff06b8, envp=0x7f7fffff06c8) at vl.c:4325

[1] https://www.openbsd.org/faq/upgrade60.html
[2] https://undeadly.org/cgi?action=article&sid=20160527203200

---
 configure | 11 +++++++++++
 1 file changed, 11 insertions(+)
configure +11 -0
index b18281c61f..f6acc028a7 100755
@@ -5795,6 +5795,17 @@ if test "$mingw32" = "yes" ; then
     done
 fi
 
+# Disable W^X if available
+if test "$tcg" = "yes" -a "$targetos" = "OpenBSD"; then
+    cat > $TMPC <<EOF
+    int main(void) { return 0; }
+EOF
+    wx_ldflags="-Wl,-z,wxneeded"
+    if compile_prog "" "$wx_ldflags"; then
+        QEMU_LDFLAGS="$QEMU_LDFLAGS -Wl,-z,wxneeded"
+    fi
+fi
+
 qemu_confdir=$sysconfdir$confsuffix
 qemu_moddir=$libdir$confsuffix
 qemu_datadir=$datadir$confsuffix
Signature · 1 line
2.20.1
#2 lore raw

Re: [Qemu-devel] [PATCH 1/3] configure: Disable W^X on OpenBSD

Quoted 37 lines · Philippe Mathieu-Daudé

Since OpenBSD 6.0 [1], W^X is enforced by default [2]. TCG requires WX access. Disable W^X if it is available. This fixes:

  # lm32-softmmu/qemu-system-lm32
  Could not allocate dynamic translator buffer
  # sysctl kern.wxabort=1
  kern.wxabort: 0 -> 1
  # lm32-softmmu/qemu-system-lm32
  mmap: Not supported
  Abort trap (core dumped)
  # gdb -q lm32-softmmu/qemu-system-lm32 qemu-system-lm32.core
  (gdb) bt
  #0  0x000017e3c156c50a in _thread_sys___syscall () at {standard input}:5
  #1  0x000017e3c15e5d7a in *_libc_mmap (addr=Variable "addr" is not available.) at /usr/src/lib/libc/sys/mmap.c:47
  #2  0x000017e17d9abc8b in alloc_code_gen_buffer () at /usr/src/qemu/accel/tcg/translate-all.c:1064
  #3  0x000017e17d9abd04 in code_gen_alloc (tb_size=0) at /usr/src/qemu/accel/tcg/translate-all.c:1112
  #4  0x000017e17d9abe81 in tcg_exec_init (tb_size=0) at /usr/src/qemu/accel/tcg/translate-all.c:1149
  #5  0x000017e17d9897e9 in tcg_init (ms=0x17e45e456800) at /usr/src/qemu/accel/tcg/tcg-all.c:66
  #6  0x000017e17d9891b8 in accel_init_machine (acc=0x17e3c3f50800, ms=0x17e45e456800) at /usr/src/qemu/accel/accel.c:63
  #7  0x000017e17d989312 in configure_accelerator (ms=0x17e45e456800, progname=0x7f7fffff07b0 "lm32-softmmu/qemu-system-lm32") at /usr/src/qemu/accel/accel.c:111
  #8  0x000017e17d9d8616 in main (argc=1, argv=0x7f7fffff06b8, envp=0x7f7fffff06c8) at vl.c:4325

[1] https://www.openbsd.org/faq/upgrade60.html
[2] https://undeadly.org/cgi?action=article&sid=20160527203200

---
 configure | 11 +++++++++++
 1 file changed, 11 insertions(+)
configure +0 -0
index b18281c61f..f6acc028a7 100755

configure if test "$mingw32" = "yes" ; then · line 5798

Quoted 4 lines
configure +1 -0
@@ -5795,6 +5795,17 @@ if test "$mingw32" = "yes" ; then
     done
 fi
 
+# Disable W^X if available

I'd like to suggest to mention OpenBSD in the comment.

configure if test "$mingw32" = "yes" ; then · line 5805

Quoted 7 lines
configure +7 -0
+if test "$tcg" = "yes" -a "$targetos" = "OpenBSD"; then
+    cat > $TMPC <<EOF
+    int main(void) { return 0; }
+EOF
+    wx_ldflags="-Wl,-z,wxneeded"
+    if compile_prog "" "$wx_ldflags"; then
+        QEMU_LDFLAGS="$QEMU_LDFLAGS -Wl,-z,wxneeded"

Why do you introduce the wx_ldflags variable above, just to use it one time? I'd suggest to either use it in the QEMU_LDFLAGS line, too, or to get rid of the variable completely and always use -Wl,-z,wxneeded directly.

Thomas

Quoted 6 lines
configure +3 -0
+    fi
+fi
+
 qemu_confdir=$sysconfdir$confsuffix
 qemu_moddir=$libdir$confsuffix
 qemu_datadir=$datadir$confsuffix
#3 lore raw

Re: [Qemu-devel] [PATCH 1/3] configure: Disable W^X on OpenBSD

Quoted 41 lines · Thomas Huth
Quoted 40 lines · Philippe Mathieu-Daudé

Since OpenBSD 6.0 [1], W^X is enforced by default [2]. TCG requires WX access. Disable W^X if it is available. This fixes:

  # lm32-softmmu/qemu-system-lm32
  Could not allocate dynamic translator buffer
  # sysctl kern.wxabort=1
  kern.wxabort: 0 -> 1
  # lm32-softmmu/qemu-system-lm32
  mmap: Not supported
  Abort trap (core dumped)
  # gdb -q lm32-softmmu/qemu-system-lm32 qemu-system-lm32.core
  (gdb) bt
  #0  0x000017e3c156c50a in _thread_sys___syscall () at {standard input}:5
  #1  0x000017e3c15e5d7a in *_libc_mmap (addr=Variable "addr" is not available.) at /usr/src/lib/libc/sys/mmap.c:47
  #2  0x000017e17d9abc8b in alloc_code_gen_buffer () at /usr/src/qemu/accel/tcg/translate-all.c:1064
  #3  0x000017e17d9abd04 in code_gen_alloc (tb_size=0) at /usr/src/qemu/accel/tcg/translate-all.c:1112
  #4  0x000017e17d9abe81 in tcg_exec_init (tb_size=0) at /usr/src/qemu/accel/tcg/translate-all.c:1149
  #5  0x000017e17d9897e9 in tcg_init (ms=0x17e45e456800) at /usr/src/qemu/accel/tcg/tcg-all.c:66
  #6  0x000017e17d9891b8 in accel_init_machine (acc=0x17e3c3f50800, ms=0x17e45e456800) at /usr/src/qemu/accel/accel.c:63
  #7  0x000017e17d989312 in configure_accelerator (ms=0x17e45e456800, progname=0x7f7fffff07b0 "lm32-softmmu/qemu-system-lm32") at /usr/src/qemu/accel/accel.c:111
  #8  0x000017e17d9d8616 in main (argc=1, argv=0x7f7fffff06b8, envp=0x7f7fffff06c8) at vl.c:4325

[1] https://www.openbsd.org/faq/upgrade60.html
[2] https://undeadly.org/cgi?action=article&sid=20160527203200

---
 configure | 11 +++++++++++
 1 file changed, 11 insertions(+)
configure +0 -0
index b18281c61f..f6acc028a7 100755
@@ -5795,6 +5795,17 @@ if test "$mingw32" = "yes" ; then
     done
 fi
 
Quoted 3 lines

configure if test "$mingw32" = "yes" ; then · line 5798

Quoted 1 line
configure +1 -0
+# Disable W^X if available

I'd like to suggest to mention OpenBSD in the comment.

Good idea.

Quoted 11 lines

configure if test "$mingw32" = "yes" ; then · line 5805

Quoted 7 lines
configure +7 -0
+if test "$tcg" = "yes" -a "$targetos" = "OpenBSD"; then
+    cat > $TMPC <<EOF
+    int main(void) { return 0; }
+EOF
+    wx_ldflags="-Wl,-z,wxneeded"
+    if compile_prog "" "$wx_ldflags"; then
+        QEMU_LDFLAGS="$QEMU_LDFLAGS -Wl,-z,wxneeded"

Why do you introduce the wx_ldflags variable above, just to use it one time? I'd suggest to either use it in the QEMU_LDFLAGS line, too, or to get rid of the variable completely and always use -Wl,-z,wxneeded directly.

I did not notice :) I'll remove the variable to avoid creating variables used once.

Thanks!

Phil.

Quoted 9 lines

Thomas

Quoted 6 lines
configure +3 -0
+    fi
+fi
+
 qemu_confdir=$sysconfdir$confsuffix
 qemu_moddir=$libdir$confsuffix
 qemu_datadir=$datadir$confsuffix
#4 lore raw

PATCH 2/3 XXX oslib-posix: Ignore fcntl("/dev/null", F_SETFL, O_NONBLOCK) failure

In reply to Philippe Mathieu-Daudé

Previous to OpenBSD 6.3 [1], fcntl(F_SETFL) is not permitted on memory devices. Do not assert fcntl failures on OpenBSD.
This fixes:

  $ lm32-softmmu/qemu-system-lm32
  assertion "f != -1" failed: file "util/oslib-posix.c", line 247, function "qemu_set_nonblock"
  Abort trap (core dumped)

[1] The fix seems https://github.com/openbsd/src/commit/c2a35b387f9d3c
"fcntl(F_SETFL) invokes the FIONBIO and FIOASYNC ioctls internally, so the memory devices (/dev/null, /dev/zero, etc) need to permit them."

---
 util/oslib-posix.c | 2 ++
 1 file changed, 2 insertions(+)
util/oslib-posix.c +2 -0
index 4ce1ba9ca4..064c3ae2f7 100644
@@ -244,7 +244,9 @@ void qemu_set_nonblock(int fd)
     f = fcntl(fd, F_GETFL);
     assert(f != -1);
     f = fcntl(fd, F_SETFL, f | O_NONBLOCK);
+#ifndef __OpenBSD__
     assert(f != -1);
+#endif
 }
 
 int socket_set_fast_reuse(int fd)
Signature · 1 line
2.20.1
#5 lore raw

Re: [Qemu-devel] [PATCH 2/3] XXX oslib-posix: Ignore fcntl("/dev/null", F_SETFL, O_NONBLOCK) failure

Is the XXX in the subject meant to go on permanent record?

#6 lore raw

Re: [Qemu-devel] [PATCH 2/3] XXX oslib-posix: Ignore fcntl("/dev/null", F_SETFL, O_NONBLOCK) failure

Quoted 1 line · Markus Armbruster

Is the XXX in the subject meant to go on permanent record?

The original plan was to rename it as NOTFORMERGE before sending but I forgot (late Friday). So the whole content of this patch isn't meant to go on perm record ;)

#7 lore raw

Re: [Qemu-devel] [PATCH 2/3] XXX oslib-posix: Ignore fcntl("/dev/null", F_SETFL, O_NONBLOCK) failure

In reply to Philippe Mathieu-Daudé
Quoted 11 lines · Philippe Mathieu-Daudé

Previous to OpenBSD 6.3 [1], fcntl(F_SETFL) is not permitted on memory devices. Do not assert fcntl failures on OpenBSD.
This fixes:

  $ lm32-softmmu/qemu-system-lm32
  assertion "f != -1" failed: file "util/oslib-posix.c", line 247, function "qemu_set_nonblock"
  Abort trap (core dumped)

[1] The fix seems https://github.com/openbsd/src/commit/c2a35b387f9d3c
"fcntl(F_SETFL) invokes the FIONBIO and FIOASYNC ioctls internally, so the memory devices (/dev/null, /dev/zero, etc) need to permit them."

I assume set_nonblock is called on more than just these special devices? Is there anyway to check this on OpenBSD or is it just an anonymous fd at this point?

Quoted 11 lines
---
 util/oslib-posix.c | 2 ++
 1 file changed, 2 insertions(+)
util/oslib-posix.c +0 -0
index 4ce1ba9ca4..064c3ae2f7 100644

util/oslib-posix.c void qemu_set_nonblock(int fd) · line 252

Quoted 9 lines
util/oslib-posix.c +2 -0
@@ -244,7 +244,9 @@ void qemu_set_nonblock(int fd)
     f = fcntl(fd, F_GETFL);
     assert(f != -1);
     f = fcntl(fd, F_SETFL, f | O_NONBLOCK);
+#ifndef __OpenBSD__
     assert(f != -1);
+#endif
 }
 
 int socket_set_fast_reuse(int fd)

--
Alex Bennée

#8 lore raw

Re: [Qemu-devel] [PATCH 2/3] XXX oslib-posix: Ignore fcntl("/dev/null", F_SETFL, O_NONBLOCK) failure

Quoted 11 lines · Alex Bennée
Quoted 8 lines · Philippe Mathieu-Daudé

Previous to OpenBSD 6.3 [1], fcntl(F_SETFL) is not permitted on memory devices. Do not assert fcntl failures on OpenBSD.
This fixes:

  $ lm32-softmmu/qemu-system-lm32
  assertion "f != -1" failed: file "util/oslib-posix.c", line 247, function "qemu_set_nonblock"
  Abort trap (core dumped)
Quoted 7 lines
Quoted 3 lines

[1] The fix seems https://github.com/openbsd/src/commit/c2a35b387f9d3c
"fcntl(F_SETFL) invokes the FIONBIO and FIOASYNC ioctls internally, so the memory devices (/dev/null, /dev/zero, etc) need to permit them."

I assume set_nonblock is called on more than just these special devices? Is there anyway to check this on OpenBSD or is it just an anonymous fd at this point?

I'll let an OpenBSD expert to answer that.

I forgot to comment, the assert() was added one month ago (da93b82079d), Michael said we can also revert this change.

Quoted 25 lines
Quoted 11 lines
---
 util/oslib-posix.c | 2 ++
 1 file changed, 2 insertions(+)
util/oslib-posix.c +0 -0
index 4ce1ba9ca4..064c3ae2f7 100644

util/oslib-posix.c void qemu_set_nonblock(int fd) · line 252

Quoted 9 lines
util/oslib-posix.c +2 -0
@@ -244,7 +244,9 @@ void qemu_set_nonblock(int fd)
     f = fcntl(fd, F_GETFL);
     assert(f != -1);
     f = fcntl(fd, F_SETFL, f | O_NONBLOCK);
+#ifndef __OpenBSD__
     assert(f != -1);
+#endif
 }
 
 int socket_set_fast_reuse(int fd)

--
Alex Bennée

#9 lore raw

Re: [Qemu-devel] [PATCH 2/3] XXX oslib-posix: Ignore fcntl("/dev/null", F_SETFL, O_NONBLOCK) failure

In reply to Alex Bennée
Quoted 8 lines · Alex Bennée
Quoted 4 lines

[1] The fix seems https://github.com/openbsd/src/commit/c2a35b387f9d3c
"fcntl(F_SETFL) invokes the FIONBIO and FIOASYNC ioctls internally, so the memory devices (/dev/null, /dev/zero, etc) need to permit them."

I assume set_nonblock is called on more than just these special devices? Is there anyway to check this on OpenBSD or is it just an anonymous fd at this point?

Perhaps on OpenBSD we should just assert that we don't get EBADF?

Paolo

#10 lore raw

Re: [Qemu-devel] [PATCH 2/3] XXX oslib-posix: Ignore fcntl("/dev/null", F_SETFL, O_NONBLOCK) failure

Quoted 11 lines · Paolo Bonzini
Quoted 8 lines · Alex Bennée
Quoted 4 lines

[1] The fix seems https://github.com/openbsd/src/commit/c2a35b387f9d3c
"fcntl(F_SETFL) invokes the FIONBIO and FIOASYNC ioctls internally, so the memory devices (/dev/null, /dev/zero, etc) need to permit them."

I assume set_nonblock is called on more than just these special devices? Is there anyway to check this on OpenBSD or is it just an anonymous fd at this point?

Perhaps on OpenBSD we should just assert that we don't get EBADF?

We get ENODEV for "not a memory device":

19 ENODEV Operation not supported by device.
An attempt was made to apply an inappropriate function to a device, for example, trying to read a write-only device such as a printer.

I'll respin with your suggestion.

Thanks!

Phil.

#11 lore raw

PATCH 3/3 WIP tests/vm: Run tests on OpenBSD

In reply to Philippe Mathieu-Daudé
---
 tests/vm/openbsd | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
tests/vm/openbsd +1 -3
index cfe0572c59..de907dd21c 100755
@@ -25,9 +25,7 @@ class OpenBSDVM(basevm.BaseVM):
         cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
         tar -xf /dev/rsd1c;
         ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
-        gmake --output-sync -j{jobs} {verbose};
-        # XXX: "gmake check" seems to always hang or fail
-        #gmake --output-sync -j{jobs} check {verbose};
+        gmake --output-sync -j{jobs} check {verbose};
     """
 
     def build_image(self, img):
Signature · 1 line
2.20.1
#12 lore raw

Re: [Qemu-devel] [PATCH 3/3] WIP tests/vm: Run tests on OpenBSD

Quoted 11 lines · Philippe Mathieu-Daudé
---
 tests/vm/openbsd | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
tests/vm/openbsd +0 -0
index cfe0572c59..de907dd21c 100755

tests/vm/openbsd class OpenBSDVM(basevm.BaseVM): · line 31

Quoted 10 lines
tests/vm/openbsd +1 -3
@@ -25,9 +25,7 @@ class OpenBSDVM(basevm.BaseVM):
         cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
         tar -xf /dev/rsd1c;
         ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
-        gmake --output-sync -j{jobs} {verbose};
-        # XXX: "gmake check" seems to always hang or fail
-        #gmake --output-sync -j{jobs} check {verbose};
+        gmake --output-sync -j{jobs} check {verbose};
     """
 
     def build_image(self, img):

Please add a proper patch description and remove the WIP from the subject.

Thanks,
Thomas

#13 lore raw

Re: [Qemu-devel] [PATCH 3/3] WIP tests/vm: Run tests on OpenBSD

Quoted 20 lines · Thomas Huth
Quoted 19 lines · Philippe Mathieu-Daudé
---
 tests/vm/openbsd | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
tests/vm/openbsd +1 -3
index cfe0572c59..de907dd21c 100755
@@ -25,9 +25,7 @@ class OpenBSDVM(basevm.BaseVM):
         cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
         tar -xf /dev/rsd1c;
         ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
-        gmake --output-sync -j{jobs} {verbose};
-        # XXX: "gmake check" seems to always hang or fail
-        #gmake --output-sync -j{jobs} check {verbose};
+        gmake --output-sync -j{jobs} check {verbose};
     """
 
Quoted 4 lines

tests/vm/openbsd class OpenBSDVM(basevm.BaseVM): · line 31

Quoted 2 lines
tests/vm/openbsd +0 -0
     def build_image(self, img):

Please add a proper patch description and remove the WIP from the subject.

Oops :/ I thought about renaming this also NOTFORMERGE.

The description could be:

OpenBSD tests are still failing, but this patch help to test the improvement of the previous 2 patches (disable W^X, disable assert(fcntl)).