Skip to contentSkip to conversation

kvm-unit-tests: Build changes for illumos.

PATCH In review Dan Cross 4 participants 31 messages May 12, 2022 – Jun 2, 2022 1 message not in the archive KVM

lore
Share & tools
lore

Dan Cross

author
#0 lore raw

PATCH 1/1 kvm-unit-tests: Build changes for illumos.

We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.

This change modifies the build infrastructure to allow building on illumos; I have also tested it on Linux. The required changes were pretty minimal: the most invasive was switching from using the C compiler as a linker driver to simply invoking the linker directly in two places. This allows us to easily use gold instead of the Solaris linker.

---
 configure           | 5 +++--
 x86/Makefile.common | 6 +++---
 2 files changed, 6 insertions(+), 5 deletions(-)
configure +3 -2
index 86c3095..7193811 100755
@@ -15,6 +15,7 @@ objdump=objdump
 ar=ar
 addr2line=addr2line
 arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/')
+os=$(uname -s)
 host=$arch
 cross_prefix=
 endian=""
@@ -317,9 +318,9 @@ EOF
   rm -f lib-test.{o,S}
 fi
 
-# require enhanced getopt
+# require enhanced getopt everywhere except illumos
 getopt -T > /dev/null
-if [ $? -ne 4 ]; then
+if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then
     echo "Enhanced getopt is not available, add it to your PATH?"
     exit 1
 fi
x86/Makefile.common +3 -3
index b903988..0a0f7b9 100644
@@ -62,7 +62,7 @@ else
 .PRECIOUS: %.elf %.o
 
 %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o)
-	$(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \
+	$(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \
 		$(filter %.o, $^) $(FLATLIBS)
 	@chmod a-x $@
 
@@ -98,8 +98,8 @@ test_cases: $(tests-common) $(tests)
 $(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib
 
 $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o
-	$(CC) -m32 -nostdlib -o $@ -Wl,-m,elf_i386 \
-	      -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^
+	$(LD) -m elf_i386 -nostdlib -o $@ \
+	      -T $(SRCDIR)/$(TEST_DIR)/realmode.lds $^
 
 $(TEST_DIR)/realmode.o: bits = $(if $(call cc-option,-m16,""),16,32)
 
Signature · 1 line
2.36.1
#1 lore raw
Quoted 6 lines · Dan Cross

We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.

Quoted 7 lines

This change modifies the build infrastructure to allow building on illumos; I have also tested it on Linux. The required changes were pretty minimal: the most invasive was switching from using the C compiler as a linker driver to simply invoking the linker directly in two places. This allows us to easily use gold instead of the Solaris linker.

Can you please split this into two patches? One for the $(CC) => $(LD) change, and one for the getopt thing. The switch to $(LD) in particular could be valuable irrespective of using a non-Linux OS.

Quoted 19 lines
---
 configure           | 5 +++--
 x86/Makefile.common | 6 +++---
 2 files changed, 6 insertions(+), 5 deletions(-)
configure +1 -0
index 86c3095..7193811 100755
@@ -15,6 +15,7 @@ objdump=objdump
 ar=ar
 addr2line=addr2line
 arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/')
+os=$(uname -s)
 host=$arch
 cross_prefix=
 endian=""

configure EOF · line 323

Quoted 8 lines
configure +2 -2
@@ -317,9 +318,9 @@ EOF
   rm -f lib-test.{o,S}
 fi
 
-# require enhanced getopt
+# require enhanced getopt everywhere except illumos
 getopt -T > /dev/null
-if [ $? -ne 4 ]; then
+if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then

Presumably whatever "enhanced" features are being used are supported by illumos, so rather than check the OS, why not improve the probing?

Quoted 3 lines
configure +0 -0
     echo "Enhanced getopt is not available, add it to your PATH?"
     exit 1
 fi
author
#2 lore raw

cover kvm-unit-tests: Build changes to support illumos.

We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.

This patch series modifies the build infrastructure to allow building on illumos; I have also tested these changes on Linux. The required changes were pretty minimal.

Dan Cross (2):
kvm-unit-tests: invoke $LD explicitly in
kvm-unit-tests: configure changes for illumos.

 configure           | 5 +++--
 x86/Makefile.common | 6 +++---
 2 files changed, 6 insertions(+), 5 deletions(-)
Signature · 1 line
2.31.1
author
#3 lore raw

PATCH 1/2 kvm-unit-tests: invoke $LD explicitly in

This change modifies x86/Makefile.common to allow building on illumos; I have also tested it on Linux. This switches from using the C compiler as a linker driver to simply invoking the linker directly in two places, allowing us to easily use gold instead of the Solaris linker.

---
 x86/Makefile.common | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
x86/Makefile.common +3 -3
index b903988..0a0f7b9 100644
@@ -62,7 +62,7 @@ else
 .PRECIOUS: %.elf %.o
 
 %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o)
-	$(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \
+	$(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \
 		$(filter %.o, $^) $(FLATLIBS)
 	@chmod a-x $@
 
@@ -98,8 +98,8 @@ test_cases: $(tests-common) $(tests)
 $(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib
 
 $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o
-	$(CC) -m32 -nostdlib -o $@ -Wl,-m,elf_i386 \
-	      -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^
+	$(LD) -m elf_i386 -nostdlib -o $@ \
+	      -T $(SRCDIR)/$(TEST_DIR)/realmode.lds $^
 
 $(TEST_DIR)/realmode.o: bits = $(if $(call cc-option,-m16,""),16,32)
 
Signature · 1 line
2.31.1
author
#4 lore raw

PATCH 2/2 kvm-unit-tests: configure changes for illumos.

In reply to Dan Cross

This change modifies the configure script to run under illumos by not probing for, getopt -T (illumos getopt supports the required functionality, but exits with a different return status when invoked with -T).

---
 configure | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
configure +3 -2
index 86c3095..7193811 100755
@@ -15,6 +15,7 @@ objdump=objdump
 ar=ar
 addr2line=addr2line
 arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/')
+os=$(uname -s)
 host=$arch
 cross_prefix=
 endian=""
@@ -317,9 +318,9 @@ EOF
   rm -f lib-test.{o,S}
 fi
 
-# require enhanced getopt
+# require enhanced getopt everywhere except illumos
 getopt -T > /dev/null
-if [ $? -ne 4 ]; then
+if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then
     echo "Enhanced getopt is not available, add it to your PATH?"
     exit 1
 fi
Signature · 1 line
2.31.1
author
#31 lore raw

PATCH 1/1 kvm-unit-tests: Build changes for illumos.

In reply to a message not in the archive

[Sean, sorry for the dup; I forgot to switch to plaintext mode]

Quoted 15 lines · Sean Christopherson
Quoted 6 lines · Dan Cross

We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.

Quoted 7 lines

This change modifies the build infrastructure to allow building on illumos; I have also tested it on Linux. The required changes were pretty minimal: the most invasive was switching from using the C compiler as a linker driver to simply invoking the linker directly in two places. This allows us to easily use gold instead of the Solaris linker.

Quoted 3 lines

Can you please split this into two patches? One for the $(CC) => $(LD) change, and one for the getopt thing. The switch to $(LD) in particular could be valuable irrespective of using a non-Linux OS.

Done.

Quoted 9 lines

[snip]

configure EOF · line 323

Quoted 4 lines
configure +2 -1
+# require enhanced getopt everywhere except illumos
 getopt -T > /dev/null
-if [ $? -ne 4 ]; then
+if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then

Presumably whatever "enhanced" features are being used are supported by illumos, so rather than check the OS, why not improve the probing?

Actually, it looks like I've got some egg on my face here. It appears that getopt
is not used in configure beyond this check; presumably this exists to avoid surprises
when running run_tests.sh, which does use extended getopt (for long options) and
repeats this same test. I had not noticed since configure ran successfully, and we are
not presently using run_tests.sh on illumos as we use a rather different VMM than
qemu. That patch should probably be dropped (or changed to remove the getopt check
in configure entirely, if folks are fine deferring needing to have it to the invocation of
run_tests.sh...). Or I just convince folks to provide a GNU getopt package on illumos.
Let me know what is preferred.

- Dan C.

#5 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

In reply to Dan Cross

Adding the official KUT maintainers, they undoubtedly know more about the getopt stuff than me.

Quoted 1 line · Dan Cross

This change modifies the configure script to run under illumos

Nit, use imperative mood. KUT follows the kernel's rules/guidelines for the most part. From Linux's Documentation/process/submitting-patches.rst:

Describe your changes in imperative mood, e.g. "make xyzzy do frotz" instead of "[This patch] makes xyzzy do frotz" or "[I] changed xyzzy to do frotz", as if you are giving orders to the codebase to change its behaviour.

E.g.

Exempt illumos, which reports itself as SunOS, from the getopt -T check
for enhanced getopt. blah blah blah...

Quoted 22 lines

by not probing for, getopt -T (illumos getopt supports the required functionality, but exits with a different return status when invoked with -T).

---
 configure | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
configure +1 -0
index 86c3095..7193811 100755
@@ -15,6 +15,7 @@ objdump=objdump
 ar=ar
 addr2line=addr2line
 arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/')
+os=$(uname -s)
 host=$arch
 cross_prefix=
 endian=""

configure EOF · line 323

Quoted 8 lines
configure +2 -2
@@ -317,9 +318,9 @@ EOF
   rm -f lib-test.{o,S}
 fi
 
-# require enhanced getopt
+# require enhanced getopt everywhere except illumos
 getopt -T > /dev/null
-if [ $? -ne 4 ]; then
+if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then

What does illumos return for getopt -T? Unless it's a direct collision with the "old" getopt, why not check for illumos' return? The SunOS check could be kept (or not). E.g. IMO this is much more self-documenting (though does $? get clobbered by the check? I'm terrible at shell scripts...).

if [ $? -ne 4 ] && [ "$os" != "SunOS" || $? -ne 666 ]; then

Test if your getopt(1) is this enhanced version or an old version. This generates no output, and sets the error status to 4. Other implementations of getopt(1), and this version if the environment variable GETOPT_COMPATIBLE is set, will return '--' and error status 0.

Quoted 6 lines
configure +0 -0
     echo "Enhanced getopt is not available, add it to your PATH?"
     exit 1
 fi
Signature · 1 line
2.31.1
#6 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

Quoted 45 lines · Sean Christopherson

Adding the official KUT maintainers, they undoubtedly know more about the getopt stuff than me.

Quoted 1 line · Dan Cross

This change modifies the configure script to run under illumos

Nit, use imperative mood. KUT follows the kernel's rules/guidelines for the most part. From Linux's Documentation/process/submitting-patches.rst:

Describe your changes in imperative mood, e.g. "make xyzzy do frotz" instead of "[This patch] makes xyzzy do frotz" or "[I] changed xyzzy to do frotz", as if you are giving orders to the codebase to change its behaviour.

E.g.

Exempt illumos, which reports itself as SunOS, from the getopt -T check
for enhanced getopt. blah blah blah...

Quoted 25 lines

by not probing for, getopt -T (illumos getopt supports the required functionality, but exits with a different return status when invoked with -T).

---
  configure | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)
configure +1 -0
index 86c3095..7193811 100755
@@ -15,6 +15,7 @@ objdump=objdump
  ar=ar
  addr2line=addr2line
  arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/')
+os=$(uname -s)
  host=$arch
  cross_prefix=
  endian=""
@@ -317,9 +318,9 @@ EOF
    rm -f lib-test.{o,S}
  fi
 
Quoted 10 lines

configure EOF · line 323

Quoted 5 lines
configure +2 -2
-# require enhanced getopt
+# require enhanced getopt everywhere except illumos
  getopt -T > /dev/null
-if [ $? -ne 4 ]; then
+if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then

What does illumos return for getopt -T? Unless it's a direct collision with the "old" getopt, why not check for illumos' return? The SunOS check could be kept (or not). E.g. IMO this is much more self-documenting (though does $? get clobbered by the check? I'm terrible at shell scripts...).

According to https://illumos.org/man/1/getopt :

NOTES

getopt will not be supported in the next major release.
...

So even if we apply this fix now, this will likely break soon again. Is there another solution to this problem?

Thomas

author
#7 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

Quoted 5 lines · Thomas Huth
Quoted 1 line · Sean Christopherson

[snip]

According to https://illumos.org/man/1/getopt :

Quoted 7 lines

NOTES

getopt will not be supported in the next major release.
...

So even if we apply this fix now, this will likely break soon again. Is there another solution to this problem?

I wouldn't put too much stock into that; that note came from Solaris and has been in the man page for 30 years or more [*]. I think there are too many shell scripts in the wild using getopt for it to ever be removed. Indeed, if anything this highlights something to clean up on the illumos side by removing that from the man page.

- Dan C.

[*] e.g. https://www.freebsd.org/cgi/man.cgi?query=getopt&apropos=0&sektion=0&manpath=SunOS+5.5.1&arch=default&format=html
from 1992!

author
#9 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

In reply to Sean Christopherson
Quoted 2 lines · Sean Christopherson

Adding the official KUT maintainers, they undoubtedly know more about the getopt stuff than me.

Thanks.

Quoted 6 lines · Dan Cross
Quoted 1 line

This change modifies the configure script to run under illumos

Nit, use imperative mood. KUT follows the kernel's rules/guidelines for the most part. From Linux's Documentation/process/submitting-patches.rst:
[snip]

Done locally, but see below.

Quoted 16 lines
Quoted 16 lines
configure +1 -0
index 86c3095..7193811 100755
@@ -15,6 +15,7 @@ objdump=objdump
 ar=ar
 addr2line=addr2line
 arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/')
+os=$(uname -s)
 host=$arch
 cross_prefix=
 endian=""
@@ -317,9 +318,9 @@ EOF
   rm -f lib-test.{o,S}
 fi
 
Quoted 7 lines

configure EOF · line 323

Quoted 5 lines
configure +2 -2
-# require enhanced getopt
+# require enhanced getopt everywhere except illumos
 getopt -T > /dev/null
-if [ $? -ne 4 ]; then
+if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then

What does illumos return for getopt -T?

Sadly, it returns "0". I was wrong in my earlier explorations because I did not realize that configure does not use getopt aside from that one check, which is repeated in run_tests.sh.

I would argue that the most straight-forward way to deal with this is to just remove the check for "getopt" from "configure", which doesn't otherwise use "getopt". The only place it is used is in run_tests.sh, which is unlikely to be used directly for illumos, and repeats the check anyway.

Quoted 2 lines

Unless it's a direct collision with
the "old" getopt, why not check for illumos' return?

It collides with "legacy" getopt. :-(

Quoted 3 lines

The SunOS check could be
kept (or not). E.g. IMO this is much more self-documenting (though does $? get clobbered by the check? I'm terrible at shell scripts...).

I think it is more or less moot now, but "$?" does get clobbered by the check.

Quoted 6 lines

if [ $? -ne 4 ] && [ "$os" != "SunOS" || $? -ne 666 ]; then

Test if your getopt(1) is this enhanced version or an old version. This generates no output, and sets the error status to 4. Other implementations of getopt(1), and this version if the environment variable GETOPT_COMPATIBLE is set, will return '--' and error status 0.

Sigh. That's precisely what the illumos version does. :-(

But it's perhaps worth pointing out that getopt isn't used by configure aside from just that check. Does it, perhaps, make more sense just to remove it from configure entirely?

- Dan C.

#8 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

In reply to Dan Cross
Quoted 8 lines · Dan Cross
Quoted 7 lines · Thomas Huth
Quoted 1 line · Sean Christopherson

[snip]

According to https://illumos.org/man/1/getopt :

NOTES

Quoted 8 lines
Quoted 5 lines

getopt will not be supported in the next major release.
...

So even if we apply this fix now, this will likely break soon again. Is there another solution to this problem?

I wouldn't put too much stock into that; that note came from Solaris and has been in the man page for 30 years or more [*].

:-)

Quoted 4 lines

I think there
are too many shell scripts in the wild using getopt for it to ever be removed. Indeed, if anything this highlights something to clean up on the illumos side by removing that from the man page.

Yup, cleaning up that man page sounds like a good idea then, indeed.

Thomas

#10 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

In reply to Dan Cross
Quoted 1 line · Dan Cross

On Fri, May 13, 2022 at 10:35 AM Sean Christopherson <[email protected]> wrote:

...

Quoted 28 lines
Quoted 16 lines
Quoted 16 lines
configure +1 -0
index 86c3095..7193811 100755
@@ -15,6 +15,7 @@ objdump=objdump
  ar=ar
  addr2line=addr2line
  arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/')
+os=$(uname -s)
  host=$arch
  cross_prefix=
  endian=""
@@ -317,9 +318,9 @@ EOF
    rm -f lib-test.{o,S}
  fi
 
Quoted 7 lines

configure EOF · line 323

Quoted 5 lines
configure +2 -2
-# require enhanced getopt
+# require enhanced getopt everywhere except illumos
  getopt -T > /dev/null
-if [ $? -ne 4 ]; then
+if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then

What does illumos return for getopt -T?

Sadly, it returns "0". I was wrong in my earlier explorations because I did not realize that configure does not use getopt aside from that one check, which is repeated in run_tests.sh.

Quoted 5 lines

I would argue that the most straight-forward way to deal with this is to just remove the check for "getopt" from "configure", which doesn't otherwise use "getopt". The only place it is used is in run_tests.sh, which is unlikely to be used directly for illumos, and repeats the check anyway.

Fine for me if we remove the check from configure, or turn it into a warning instead ("Enhanced getopt is not available, you won't be able to use the run_tests.sh script" or so).

Thomas

#11 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

Quoted 37 lines · Thomas Huth
Quoted 1 line · Dan Cross

On Fri, May 13, 2022 at 10:35 AM Sean Christopherson <[email protected]> wrote:

...

Quoted 28 lines
Quoted 16 lines
Quoted 16 lines
configure +1 -0
index 86c3095..7193811 100755
@@ -15,6 +15,7 @@ objdump=objdump
  ar=ar
  addr2line=addr2line
  arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/')
+os=$(uname -s)
  host=$arch
  cross_prefix=
  endian=""
@@ -317,9 +318,9 @@ EOF
    rm -f lib-test.{o,S}
  fi
 
Quoted 7 lines

configure EOF · line 323

Quoted 5 lines
configure +2 -2
-# require enhanced getopt
+# require enhanced getopt everywhere except illumos
  getopt -T > /dev/null
-if [ $? -ne 4 ]; then
+if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then

What does illumos return for getopt -T?

Sadly, it returns "0". I was wrong in my earlier explorations because I did not realize that configure does not use getopt aside from that one check, which is repeated in run_tests.sh.

Quoted 5 lines

I would argue that the most straight-forward way to deal with this is to just remove the check for "getopt" from "configure", which doesn't otherwise use "getopt". The only place it is used is in run_tests.sh, which is unlikely to be used directly for illumos, and repeats the check anyway.

Quoted 4 lines

Fine for me if we remove the check from configure, or turn it into a warning instead ("Enhanced getopt is not available, you won't be able to use the run_tests.sh script" or so).

Ack for simply changing the configure error to a warning for now. Ideally we'd limit the dependencies this project has though. So maybe rewriting the run_tests.sh command line parser without getopt would be the better thing to do.

Thanks,
drew

author
#12 lore raw

cover kvm-unit-tests: Build changes to support illumos.

We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.

This patch series modifies the build infrastructure to allow building on illumos; the changes were minimal. I have also tested these changes on Linux to ensure no regressions.

Dan Cross (2):
kvm-unit-tests: invoke $LD explicitly in
kvm-unit-tests: configure changes for illumos.

 configure           | 5 +++--
 x86/Makefile.common | 6 +++---
 2 files changed, 6 insertions(+), 5 deletions(-)
Signature · 1 line
2.31.1
author
#24 lore raw

PATCH 2/2 kvm-unit-tests: configure changes for illumos.

Warn, don't fail, if the check for getopt -T fails in the configure script.

Aside from this check, configure does not use getopt, so don't fail to run if getopt -T doesn't indicate support for the extended Linux version. Getopt is only used in run_tests.sh, which tests for extended getopt anyway, but emit a warning here.

---
 configure | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
configure +3 -3
index 23085da..5b7daac 100755
@@ -318,11 +318,11 @@ EOF
   rm -f lib-test.{o,S}
 fi
 
-# require enhanced getopt
+# warn if enhanced getopt is unavailable
 getopt -T > /dev/null
 if [ $? -ne 4 ]; then
-    echo "Enhanced getopt is not available, add it to your PATH?"
-    exit 1
+    echo "Without enhanced getopt you won't be able to use run_tests.sh."
+    echo "Add it to your PATH?"
 fi
 
 # Are we in a separate build tree? If so, link the Makefile
Signature · 1 line
2.31.2
author
#13 lore raw

PATCH 1/2 kvm-unit-tests: invoke $LD explicitly in

In reply to Dan Cross

Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver.

This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.

---
 x86/Makefile.common | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
x86/Makefile.common +3 -3
index b903988..0a0f7b9 100644
@@ -62,7 +62,7 @@ else
 .PRECIOUS: %.elf %.o
 
 %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o)
-	$(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \
+	$(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \
 		$(filter %.o, $^) $(FLATLIBS)
 	@chmod a-x $@
 
@@ -98,8 +98,8 @@ test_cases: $(tests-common) $(tests)
 $(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib
 
 $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o
-	$(CC) -m32 -nostdlib -o $@ -Wl,-m,elf_i386 \
-	      -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^
+	$(LD) -m elf_i386 -nostdlib -o $@ \
+	      -T $(SRCDIR)/$(TEST_DIR)/realmode.lds $^
 
 $(TEST_DIR)/realmode.o: bits = $(if $(call cc-option,-m16,""),16,32)
 
Signature · 1 line
2.31.2
author
#29 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

In reply to Andrew Jones
Quoted 10 lines · Andrew Jones
Quoted 4 lines · Thomas Huth

[snip]
Fine for me if we remove the check from configure, or turn it into a warning instead ("Enhanced getopt is not available, you won't be able to use the run_tests.sh script" or so).

Ack for simply changing the configure error to a warning for now. Ideally we'd limit the dependencies this project has though. So maybe rewriting the run_tests.sh command line parser without getopt would be the better thing to do.

Sounds good. New patches inbound. Thank you!

- Dan C.

#14 lore raw

Re: [PATCH 1/2] kvm-unit-tests: invoke $LD explicitly in

In reply to Dan Cross
Quoted 25 lines · Dan Cross

Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver.

This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.

---
 x86/Makefile.common | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
x86/Makefile.common +1 -1
index b903988..0a0f7b9 100644
@@ -62,7 +62,7 @@ else
 .PRECIOUS: %.elf %.o
 
 %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o)
-	$(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \
+	$(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \
 		$(filter %.o, $^) $(FLATLIBS)
 	@chmod a-x $@
 

x86/Makefile.common test_cases: $(tests-common) $(tests) · line 102

Quoted 7 lines
x86/Makefile.common +2 -2
@@ -98,8 +98,8 @@ test_cases: $(tests-common) $(tests)
 $(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib
 
 $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o
-	$(CC) -m32 -nostdlib -o $@ -Wl,-m,elf_i386 \
-	      -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^
+	$(LD) -m elf_i386 -nostdlib -o $@ \
+	      -T $(SRCDIR)/$(TEST_DIR)/realmode.lds $^

AFAICT, this is functionally a nop, and both 32-bit and 64-bit are happy, so,

#25 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

In reply to Dan Cross

In the future, please tag each new posting with a version number, e.g. no version number for the first posting, than v2, v3, v4, etc... for each subsequent posting. It took me a while to figure what was what.

Quoted 7 lines · Dan Cross

Warn, don't fail, if the check for getopt -T fails in the configure script.

Aside from this check, configure does not use getopt, so don't fail to run if getopt -T doesn't indicate support for the extended Linux version. Getopt is only used in run_tests.sh, which tests for extended getopt anyway, but emit a warning here.

Why not simply move the check to run_tests.sh? I can't imaging it's performance sensitive, and I doubt I'm the only one that builds tests on one system and runs them on a completely different system.

author
#26 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

Quoted 3 lines · Sean Christopherson

In the future, please tag each new posting with a version number, e.g. no version number for the first posting, than v2, v3, v4, etc... for each subsequent posting. It took me a while to figure what was what.

Ah, sorry about that. Sure thing.

Quoted 12 lines · Dan Cross
Quoted 7 lines

Warn, don't fail, if the check for getopt -T fails in the configure script.

Aside from this check, configure does not use getopt, so don't fail to run if getopt -T doesn't indicate support for the extended Linux version. Getopt is only used in run_tests.sh, which tests for extended getopt anyway, but emit a warning here.

Why not simply move the check to run_tests.sh? I can't imaging it's performance sensitive, and I doubt I'm the only one that builds tests on one system and runs them on a completely different system.

run_tests.sh already has the test. Changing it to a warning here was at the suggestion of Thomas and Drew.

- Dan C.

#27 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

Quoted 7 lines · Dan Cross
Quoted 3 lines · Sean Christopherson

Why not simply move the check to run_tests.sh? I can't imaging it's performance sensitive, and I doubt I'm the only one that builds tests on one system and runs them on a completely different system.

run_tests.sh already has the test. Changing it to a warning here was at the suggestion of Thomas and Drew.

Ah, hence the earlier comment about removing the check entirely. Either works for me, thanks!

author
#28 lore raw

Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.

Quoted 11 lines · Sean Christopherson
Quoted 7 lines · Dan Cross
Quoted 3 lines · Sean Christopherson

Why not simply move the check to run_tests.sh? I can't imaging it's performance sensitive, and I doubt I'm the only one that builds tests on one system and runs them on a completely different system.

run_tests.sh already has the test. Changing it to a warning here was at the suggestion of Thomas and Drew.

Ah, hence the earlier comment about removing the check entirely. Either works for me, thanks!

Thanks for the helpful comments on the review!

- Dan C.

#15 lore raw

Re: [PATCH 1/2] kvm-unit-tests: invoke $LD explicitly in

In reply to Dan Cross
Quoted 16 lines · Dan Cross

Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver.

This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.

---
  x86/Makefile.common | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
x86/Makefile.common +0 -0
index b903988..0a0f7b9 100644

x86/Makefile.common else · line 67

Quoted 7 lines
x86/Makefile.common +1 -1
@@ -62,7 +62,7 @@ else
  .PRECIOUS: %.elf %.o
 
  %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o)
-	$(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \
+	$(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \
  		$(filter %.o, $^) $(FLATLIBS)
  	@chmod a-x $@

Hi,

something seems to be missing here - this is failing our 32-bit CI job:

https://gitlab.com/thuth/kvm-unit-tests/-/jobs/2531237708

ld -T /builds/thuth/kvm-unit-tests/x86/flat.lds -nostdlib -o x86/taskswitch2.elf \
x86/taskswitch2.o x86/cstart.o lib/libcflat.a
ld: i386 architecture of input file `x86/taskswitch.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `x86/cstart.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(argv.o)' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(printf.o)' is incompatible with i386:x86-64 output
...

You can find the job definition in the .gitlab-ci.yml file (it's basically just about running "configure" with --arch=i386).

Thomas

author
#16 lore raw

Re: [PATCH 1/2] kvm-unit-tests: invoke $LD explicitly in

Quoted 41 lines · Thomas Huth
Quoted 16 lines · Dan Cross

Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver.

This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.

---
  x86/Makefile.common | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
x86/Makefile.common +0 -0
index b903988..0a0f7b9 100644

x86/Makefile.common else · line 67

Quoted 7 lines
x86/Makefile.common +1 -1
@@ -62,7 +62,7 @@ else
  .PRECIOUS: %.elf %.o
 
  %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o)
-     $(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \
+     $(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \
              $(filter %.o, $^) $(FLATLIBS)
      @chmod a-x $@

Hi,

something seems to be missing here - this is failing our 32-bit CI job:

https://gitlab.com/thuth/kvm-unit-tests/-/jobs/2531237708

ld -T /builds/thuth/kvm-unit-tests/x86/flat.lds -nostdlib -o x86/taskswitch2.elf \
x86/taskswitch2.o x86/cstart.o lib/libcflat.a
ld: i386 architecture of input file `x86/taskswitch.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `x86/cstart.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(argv.o)' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(printf.o)' is incompatible with i386:x86-64 output
...

Quoted 2 lines

You can find the job definition in the .gitlab-ci.yml file (it's basically just about running "configure" with --arch=i386).

Thanks. I think the easiest way to fix this is to plumb an argument through to the linker that expands to -m elf_i386 on 32-bit, and possibly, -m elf_x86_64 on 64-bit. The architecture specific Makefiles set the ldarch variable, and that doesn't seem used anywhere, but I see that's set to match the string accepted by the linker scripts/objcopy, not something acceptable to -m. However, one can add something to LDARGS, but I see that that's set to include the contents of CFLAGS, which means it includes flags that are not directly consumable by the linker itself.

I think the simplest way forward is to introduce a new variable in x86/Makefile.i386 and x86/Makefile.x86_64 and refer to that in x86/Makefile.common; possibly LDEXTRA? I've got an updated patch here that does that, and it seems to work (building under both illumos and arch locally), but before I send up another patchset, let me know if that sounds acceptable?

- Dan C.

#17 lore raw

Re: [PATCH 1/2] kvm-unit-tests: invoke $LD explicitly in

Quoted 56 lines · Dan Cross
Quoted 41 lines · Thomas Huth
Quoted 16 lines · Dan Cross

Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver.

This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.

---
  x86/Makefile.common | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
x86/Makefile.common +0 -0
index b903988..0a0f7b9 100644

x86/Makefile.common else · line 67

Quoted 7 lines
x86/Makefile.common +1 -1
@@ -62,7 +62,7 @@ else
  .PRECIOUS: %.elf %.o
 
  %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o)
-     $(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \
+     $(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \
              $(filter %.o, $^) $(FLATLIBS)
      @chmod a-x $@

Hi,

something seems to be missing here - this is failing our 32-bit CI job:

https://gitlab.com/thuth/kvm-unit-tests/-/jobs/2531237708

ld -T /builds/thuth/kvm-unit-tests/x86/flat.lds -nostdlib -o x86/taskswitch2.elf \
x86/taskswitch2.o x86/cstart.o lib/libcflat.a
ld: i386 architecture of input file `x86/taskswitch.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `x86/cstart.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(argv.o)' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(printf.o)' is incompatible with i386:x86-64 output
...

Quoted 2 lines

You can find the job definition in the .gitlab-ci.yml file (it's basically just about running "configure" with --arch=i386).

Thanks. I think the easiest way to fix this is to plumb an argument through to the linker that expands to -m elf_i386 on 32-bit, and possibly, -m elf_x86_64 on 64-bit. The architecture specific Makefiles set the ldarch variable, and that doesn't seem used anywhere, but I see that's set to match the string accepted by the linker scripts/objcopy, not something acceptable to -m. However, one can add something to LDARGS, but I see that that's set to include the contents of CFLAGS, which means it includes flags that are not directly consumable by the linker itself.

Quoted 3 lines

I think the simplest way forward is to introduce a new variable in x86/Makefile.i386 and x86/Makefile.x86_64 and refer to that in x86/Makefile.common; possibly LDEXTRA?

We do that for arm, but call that argument arch_LDFLAGS.

Quoted 4 lines

I've got an updated patch here that does that, and it seems to work (building under both illumos and arch locally), but before I send up another patchset, let me know if that sounds acceptable?

It does to me.

Thanks,
drew

author
#18 lore raw

Re: [PATCH 1/2] kvm-unit-tests: invoke $LD explicitly in

Quoted 12 lines · Andrew Jones
Quoted 11 lines · Dan Cross

Thanks. I think the easiest way to fix this is to plumb an argument through to the linker that expands to -m elf_i386 on 32-bit, and possibly, -m elf_x86_64 on 64-bit. The architecture specific Makefiles set the ldarch variable, and that doesn't seem used anywhere, but I see that's set to match the string accepted by the linker scripts/objcopy, not something acceptable to -m. However, one can add something to LDARGS, but I see that that's set to include the contents of CFLAGS, which means it includes flags that are not directly consumable by the linker itself.

Quoted 5 lines
Quoted 3 lines

I think the simplest way forward is to introduce a new variable in x86/Makefile.i386 and x86/Makefile.x86_64 and refer to that in x86/Makefile.common; possibly LDEXTRA?

We do that for arm, but call that argument arch_LDFLAGS.

Ah cool; I see the pattern now. Okay, I've modified x86 to follow ARM in this regard and tested locally.

Quoted 9 lines
Quoted 4 lines

I've got an updated patch here that does that, and it seems to work (building under both illumos and arch locally), but before I send up another patchset, let me know if that sounds acceptable?

It does to me.

Thanks,
drew

Great. Next patch rev inbound.

- Dan C.

author
#19 lore raw

v4 cover kvm-unit-tests: Build changes to support illumos.

We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.

This patch series modifies the build infrastructure to allow building on illumos; the changes were minimal. I have also tested these changes on Linux to ensure no regressions.

Dan Cross (2):
kvm-unit-tests: configure changes for illumos.
kvm-unit-tests: invoke $LD explicitly

 configure           | 6 +++---
 x86/Makefile.common | 9 +++++----
 x86/Makefile.i386   | 1 +
 x86/Makefile.x86_64 | 2 ++
 4 files changed, 11 insertions(+), 7 deletions(-)
Signature · 1 line
2.36.1
author
#20 lore raw

PATCH v4 1/2 kvm-unit-tests: configure changes for illumos.

Warn, don't fail, if the check for getopt -T fails in the configure script.

Aside from this check, configure does not use getopt, so don't fail to run if getopt -T doesn't indicate support for the extended Linux version. Getopt is only used in run_tests.sh, which tests for extended getopt anyway, but emit a warning here.

---
 configure | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
configure +3 -3
index 23085da..5b7daac 100755
@@ -318,11 +318,11 @@ EOF
   rm -f lib-test.{o,S}
 fi
 
-# require enhanced getopt
+# warn if enhanced getopt is unavailable
 getopt -T > /dev/null
 if [ $? -ne 4 ]; then
-    echo "Enhanced getopt is not available, add it to your PATH?"
-    exit 1
+    echo "Without enhanced getopt you won't be able to use run_tests.sh."
+    echo "Add it to your PATH?"
 fi
 
 # Are we in a separate build tree? If so, link the Makefile
Signature · 1 line
2.36.1
author
#21 lore raw

PATCH v4 2/2 kvm-unit-tests: invoke $LD explicitly

In reply to Dan Cross

Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver. Plumb LDFLAGS through to the linker.

This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.

---
 x86/Makefile.common | 9 +++++----
 x86/Makefile.i386   | 1 +
 x86/Makefile.x86_64 | 2 ++
 3 files changed, 8 insertions(+), 4 deletions(-)
x86/Makefile.common +5 -4
index b903988..a600c72 100644
@@ -30,7 +30,7 @@ endif
 
 OBJDIRS += lib/x86
 
-$(libcflat): LDFLAGS += -nostdlib
+$(libcflat): LDFLAGS += -nostdlib $(arch_LDFLAGS)
 $(libcflat): CFLAGS += -ffreestanding -I $(SRCDIR)/lib -I lib
 
 COMMON_CFLAGS += -m$(bits)
@@ -61,8 +61,9 @@ else
 # We want to keep intermediate file: %.elf and %.o
 .PRECIOUS: %.elf %.o
 
+%.elf: LDFLAGS = -nostdlib $(arch_LDFLAGS)
 %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o)
-	$(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \
+	$(LD) $(LDFLAGS) -T $(SRCDIR)/x86/flat.lds -o $@ \
 		$(filter %.o, $^) $(FLATLIBS)
 	@chmod a-x $@
 
@@ -98,8 +99,8 @@ test_cases: $(tests-common) $(tests)
 $(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib
 
 $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o
-	$(CC) -m32 -nostdlib -o $@ -Wl,-m,elf_i386 \
-	      -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^
+	$(LD) -m elf_i386 -nostdlib -o $@ \
+	      -T $(SRCDIR)/$(TEST_DIR)/realmode.lds $^
 
 $(TEST_DIR)/realmode.o: bits = $(if $(call cc-option,-m16,""),16,32)
 
x86/Makefile.i386 +1 -0
index 7d19d55..0a845e6 100644
@@ -4,6 +4,7 @@ ldarch = elf32-i386
 exe = flat
 bin = elf
 COMMON_CFLAGS += -mno-sse -mno-sse2
+arch_LDFLAGS = -m elf_i386
 
 cflatobjs += lib/x86/setjmp32.o lib/ldiv32.o
 
x86/Makefile.x86_64 +2 -0
index f18c1e2..e19284a 100644
@@ -6,9 +6,11 @@ exe = efi
 bin = so
 FORMAT = efi-app-x86_64
 cstart.o = $(TEST_DIR)/efi/efistart64.o
+arch_LDFLAGS = ''
 else
 exe = flat
 bin = elf
+arch_LDFLAGS = -m elf_x86_64
 endif
 
 fcf_protection_full := $(call cc-option, -fcf-protection=full,)
Signature · 1 line
2.36.1
#22 lore raw

Re: [PATCH v4 0/2] kvm-unit-tests: Build changes to support illumos.

In reply to Dan Cross
Quoted 9 lines · Dan Cross

We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.

This patch series modifies the build infrastructure to allow building on illumos; the changes were minimal. I have also tested these changes on Linux to ensure no regressions.

Thanks, this version survived the CI, so I've applied it now.

Thomas

author
#23 lore raw

Re: [PATCH v4 0/2] kvm-unit-tests: Build changes to support illumos.

Quoted 7 lines · Thomas Huth
Quoted 6 lines · Dan Cross

We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.

Quoted 5 lines
Quoted 3 lines

This patch series modifies the build infrastructure to allow building on illumos; the changes were minimal. I have also tested these changes on Linux to ensure no regressions.

Thanks, this version survived the CI, so I've applied it now.

Thank you very much!

- Dan C.