kvm-unit-tests: Build changes for illumos.
PATCH 1/1 kvm-unit-tests: Build changes for illumos.
We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.
This change modifies the build infrastructure to allow building on illumos; I have also tested it on Linux. The required changes were pretty minimal: the most invasive was switching from using the C compiler as a linker driver to simply invoking the linker directly in two places. This allows us to easily use gold instead of the Solaris linker.
--- configure | 5 +++-- x86/Makefile.common | 6 +++--- 2 files changed, 6 insertions(+), 5 deletions(-)
configure +3 -2
@@ -15,6 +15,7 @@ objdump=objdump ar=ar addr2line=addr2line arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') +os=$(uname -s) host=$arch cross_prefix= endian=""
@@ -317,9 +318,9 @@ EOF rm -f lib-test.{o,S} fi -# require enhanced getopt +# require enhanced getopt everywhere except illumos getopt -T > /dev/null -if [ $? -ne 4 ]; then +if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then echo "Enhanced getopt is not available, add it to your PATH?" exit 1 fi
x86/Makefile.common +3 -3
@@ -62,7 +62,7 @@ else .PRECIOUS: %.elf %.o %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o) - $(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \ + $(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \ $(filter %.o, $^) $(FLATLIBS) @chmod a-x $@
@@ -98,8 +98,8 @@ test_cases: $(tests-common) $(tests) $(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o - $(CC) -m32 -nostdlib -o $@ -Wl,-m,elf_i386 \ - -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^ + $(LD) -m elf_i386 -nostdlib -o $@ \ + -T $(SRCDIR)/$(TEST_DIR)/realmode.lds $^ $(TEST_DIR)/realmode.o: bits = $(if $(call cc-option,-m16,""),16,32)
Signature · 1 line
2.36.1
Sean Christopherson
reviewerQuoted 6 lines · Dan Cross
We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.
Quoted 7 lines
This change modifies the build infrastructure to allow building on illumos; I have also tested it on Linux. The required changes were pretty minimal: the most invasive was switching from using the C compiler as a linker driver to simply invoking the linker directly in two places. This allows us to easily use gold instead of the Solaris linker.
Can you please split this into two patches? One for the $(CC) => $(LD) change, and one for the getopt thing. The switch to $(LD) in particular could be valuable irrespective of using a non-Linux OS.
Quoted 19 lines
Signed-off-by Dan Cross <[email protected]>--- configure | 5 +++-- x86/Makefile.common | 6 +++--- 2 files changed, 6 insertions(+), 5 deletions(-)configure +1
-0@@ -15,6 +15,7 @@ objdump=objdump ar=ar addr2line=addr2line arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') +os=$(uname -s) host=$arch cross_prefix= endian=""
configure EOF · line 323
Quoted 8 lines
configure +2
-2@@ -317,9 +318,9 @@ EOF rm -f lib-test.{o,S} fi -# require enhanced getopt +# require enhanced getopt everywhere except illumos getopt -T > /dev/null -if [ $? -ne 4 ]; then +if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then
Presumably whatever "enhanced" features are being used are supported by illumos, so rather than check the OS, why not improve the probing?
Quoted 3 lines
configure +0
-0echo "Enhanced getopt is not available, add it to your PATH?" exit 1 fi
cover kvm-unit-tests: Build changes to support illumos.
We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.
This patch series modifies the build infrastructure to allow building on illumos; I have also tested these changes on Linux. The required changes were pretty minimal.
Dan Cross (2):
kvm-unit-tests: invoke $LD explicitly in
kvm-unit-tests: configure changes for illumos.
configure | 5 +++-- x86/Makefile.common | 6 +++--- 2 files changed, 6 insertions(+), 5 deletions(-)
Signature · 1 line
2.31.1
PATCH 1/2 kvm-unit-tests: invoke $LD explicitly in
This change modifies x86/Makefile.common to allow building on illumos; I have also tested it on Linux. This switches from using the C compiler as a linker driver to simply invoking the linker directly in two places, allowing us to easily use gold instead of the Solaris linker.
--- x86/Makefile.common | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
x86/Makefile.common +3 -3
@@ -62,7 +62,7 @@ else .PRECIOUS: %.elf %.o %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o) - $(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \ + $(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \ $(filter %.o, $^) $(FLATLIBS) @chmod a-x $@
@@ -98,8 +98,8 @@ test_cases: $(tests-common) $(tests) $(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o - $(CC) -m32 -nostdlib -o $@ -Wl,-m,elf_i386 \ - -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^ + $(LD) -m elf_i386 -nostdlib -o $@ \ + -T $(SRCDIR)/$(TEST_DIR)/realmode.lds $^ $(TEST_DIR)/realmode.o: bits = $(if $(call cc-option,-m16,""),16,32)
Signature · 1 line
2.31.1
PATCH 2/2 kvm-unit-tests: configure changes for illumos.
In reply to Dan CrossThis change modifies the configure script to run under illumos by not probing for, getopt -T (illumos getopt supports the required functionality, but exits with a different return status when invoked with -T).
--- configure | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
configure +3 -2
@@ -15,6 +15,7 @@ objdump=objdump ar=ar addr2line=addr2line arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') +os=$(uname -s) host=$arch cross_prefix= endian=""
@@ -317,9 +318,9 @@ EOF rm -f lib-test.{o,S} fi -# require enhanced getopt +# require enhanced getopt everywhere except illumos getopt -T > /dev/null -if [ $? -ne 4 ]; then +if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then echo "Enhanced getopt is not available, add it to your PATH?" exit 1 fi
Signature · 1 line
2.31.1
Sean Christopherson
reviewerRe: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
Adding the official KUT maintainers, they undoubtedly know more about the getopt stuff than me.
Quoted 1 line · Dan Cross
This change modifies the
configurescript to run under illumos
Nit, use imperative mood. KUT follows the kernel's rules/guidelines for the most part. From Linux's Documentation/process/submitting-patches.rst:
Describe your changes in imperative mood, e.g. "make xyzzy do frotz" instead of "[This patch] makes xyzzy do frotz" or "[I] changed xyzzy to do frotz", as if you are giving orders to the codebase to change its behaviour.
E.g.
Exempt illumos, which reports itself as SunOS, from the getopt -T check
for enhanced getopt. blah blah blah...
Quoted 22 lines
by not probing for,
getopt -T(illumosgetoptsupports the required functionality, but exits with a different return status when invoked with-T).Signed-off-by Dan Cross <[email protected]>--- configure | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)configure +1
-0@@ -15,6 +15,7 @@ objdump=objdump ar=ar addr2line=addr2line arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') +os=$(uname -s) host=$arch cross_prefix= endian=""
configure EOF · line 323
Quoted 8 lines
configure +2
-2@@ -317,9 +318,9 @@ EOF rm -f lib-test.{o,S} fi -# require enhanced getopt +# require enhanced getopt everywhere except illumos getopt -T > /dev/null -if [ $? -ne 4 ]; then +if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; then
What does illumos return for getopt -T? Unless it's a direct collision with the "old" getopt, why not check for illumos' return? The SunOS check could be kept (or not). E.g. IMO this is much more self-documenting (though does $? get clobbered by the check? I'm terrible at shell scripts...).
if [ $? -ne 4 ] && [ "$os" != "SunOS" || $? -ne 666 ]; then
Test if your getopt(1) is this enhanced version or an old version. This generates no output, and sets the error status to 4. Other implementations of getopt(1), and this version if the environment variable GETOPT_COMPATIBLE is set, will return '--' and error status 0.
Quoted 6 lines
configure +0
-0echo "Enhanced getopt is not available, add it to your PATH?" exit 1 fiSignature · 1 line
2.31.1
Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
Quoted 45 lines · Sean Christopherson
Adding the official KUT maintainers, they undoubtedly know more about the getopt stuff than me.
Quoted 1 line · Dan Cross
This change modifies the
configurescript to run under illumosNit, use imperative mood. KUT follows the kernel's rules/guidelines for the most part. From Linux's
Documentation/process/submitting-patches.rst:Describe your changes in imperative mood, e.g. "make xyzzy do frotz" instead of "[This patch] makes xyzzy do frotz" or "[I] changed xyzzy to do frotz", as if you are giving orders to the codebase to change its behaviour.
E.g.
Exempt illumos, which reports itself as SunOS, from the
getopt -Tcheck
for enhanced getopt. blah blah blah...Quoted 25 lines
by not probing for,
getopt -T(illumosgetoptsupports the required functionality, but exits with a different return status when invoked with-T).Signed-off-by Dan Cross <[email protected]>--- configure | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)configure +1
-0@@ -15,6 +15,7 @@ objdump=objdump ar=ar addr2line=addr2line arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') +os=$(uname -s) host=$arch cross_prefix= endian=""@@ -317,9 +318,9 @@ EOF rm -f lib-test.{o,S} fi
Quoted 10 lines
configure EOF · line 323
Quoted 5 lines
configure +2
-2-# require enhanced getopt +# require enhanced getopt everywhere except illumos getopt -T > /dev/null -if [ $? -ne 4 ]; then +if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; thenWhat does illumos return for
getopt -T? Unless it's a direct collision with the "old" getopt, why not check for illumos' return? The SunOS check could be kept (or not). E.g. IMO this is much more self-documenting (though does $? get clobbered by the check? I'm terrible at shell scripts...).
According to https://illumos.org/man/1/getopt :
NOTES
getopt will not be supported in the next major release.
...
So even if we apply this fix now, this will likely break soon again. Is there another solution to this problem?
Thomas
Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
Quoted 5 lines · Thomas Huth
Quoted 1 line · Sean Christopherson
[snip]
According to https://illumos.org/man/1/getopt :
Quoted 7 lines
NOTES
getopt will not be supported in the next major release.
...So even if we apply this fix now, this will likely break soon again. Is there another solution to this problem?
I wouldn't put too much stock into that; that note came from Solaris and has been in the man page for 30 years or more [*]. I think there are too many shell scripts in the wild using getopt for it to ever be removed. Indeed, if anything this highlights something to clean up on the illumos side by removing that from the man page.
- Dan C.
[*] e.g. https://www.freebsd.org/cgi/man.cgi?query=getopt&apropos=0&sektion=0&manpath=SunOS+5.5.1&arch=default&format=html
from 1992!
Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
Quoted 8 lines · Dan Cross
Quoted 7 lines · Thomas Huth
Quoted 1 line · Sean Christopherson
[snip]
According to https://illumos.org/man/1/getopt :
NOTES
Quoted 8 lines
Quoted 5 lines
getopt will not be supported in the next major release.
...So even if we apply this fix now, this will likely break soon again. Is there another solution to this problem?
I wouldn't put too much stock into that; that note came from Solaris and has been in the man page for 30 years or more [*].
:-)
Quoted 4 lines
I think there
are too many shell scripts in the wild usinggetoptfor it to ever be removed. Indeed, if anything this highlights something to clean up on the illumos side by removing that from the man page.
Yup, cleaning up that man page sounds like a good idea then, indeed.
Thomas
Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
In reply to Sean ChristophersonQuoted 2 lines · Sean Christopherson
Adding the official KUT maintainers, they undoubtedly know more about the getopt stuff than me.
Thanks.
Quoted 6 lines · Dan Cross
Quoted 1 line
This change modifies the
configurescript to run under illumosNit, use imperative mood. KUT follows the kernel's rules/guidelines for the most part. From Linux's
Documentation/process/submitting-patches.rst:
[snip]
Done locally, but see below.
Quoted 16 lines
Quoted 16 lines
configure +1
-0@@ -15,6 +15,7 @@ objdump=objdump ar=ar addr2line=addr2line arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') +os=$(uname -s) host=$arch cross_prefix= endian=""@@ -317,9 +318,9 @@ EOF rm -f lib-test.{o,S} fi
Quoted 7 lines
configure EOF · line 323
Quoted 5 lines
configure +2
-2-# require enhanced getopt +# require enhanced getopt everywhere except illumos getopt -T > /dev/null -if [ $? -ne 4 ]; then +if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; thenWhat does illumos return for
getopt -T?
Sadly, it returns "0". I was wrong in my earlier explorations because I did not realize that configure does not use getopt aside from that one check, which is repeated in run_tests.sh.
I would argue that the most straight-forward way to deal with this is to just remove the check for "getopt" from "configure", which doesn't otherwise use "getopt". The only place it is used is in run_tests.sh, which is unlikely to be used directly for illumos, and repeats the check anyway.
Quoted 2 lines
Unless it's a direct collision with
the "old" getopt, why not check for illumos' return?
It collides with "legacy" getopt. :-(
Quoted 3 lines
The SunOS check could be
kept (or not). E.g. IMO this is much more self-documenting (though does $? get clobbered by the check? I'm terrible at shell scripts...).
I think it is more or less moot now, but "$?" does get clobbered by the check.
Quoted 6 lines
if [ $? -ne 4 ] && [ "$os" != "SunOS" || $? -ne 666 ]; then
Test if your getopt(1) is this enhanced version or an old version. This generates no output, and sets the error status to 4. Other implementations of getopt(1), and this version if the environment variable
GETOPT_COMPATIBLEis set, will return '--' and error status 0.
Sigh. That's precisely what the illumos version does. :-(
But it's perhaps worth pointing out that getopt isn't used by configure aside from just that check. Does it, perhaps, make more sense just to remove it from configure entirely?
- Dan C.
Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
Quoted 1 line · Dan Cross
On Fri, May 13, 2022 at 10:35 AM Sean Christopherson <[email protected]> wrote:
...
Quoted 28 lines
Quoted 16 lines
Quoted 16 lines
configure +1
-0@@ -15,6 +15,7 @@ objdump=objdump ar=ar addr2line=addr2line arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') +os=$(uname -s) host=$arch cross_prefix= endian=""@@ -317,9 +318,9 @@ EOF rm -f lib-test.{o,S} fiQuoted 7 lines
configure EOF · line 323
Quoted 5 lines
configure +2
-2-# require enhanced getopt +# require enhanced getopt everywhere except illumos getopt -T > /dev/null -if [ $? -ne 4 ]; then +if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; thenWhat does illumos return for
getopt -T?Sadly, it returns "0". I was wrong in my earlier explorations because I did not realize that
configuredoes not usegetoptaside from that one check, which is repeated inrun_tests.sh.
Quoted 5 lines
I would argue that the most straight-forward way to deal with this is to just remove the check for "getopt" from "configure", which doesn't otherwise use "getopt". The only place it is used is in
run_tests.sh, which is unlikely to be used directly for illumos, and repeats the check anyway.
Fine for me if we remove the check from configure, or turn it into a warning instead ("Enhanced getopt is not available, you won't be able to use the run_tests.sh script" or so).
Thomas
Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
Quoted 37 lines · Thomas Huth
Quoted 1 line · Dan Cross
On Fri, May 13, 2022 at 10:35 AM Sean Christopherson <[email protected]> wrote:
...
Quoted 28 lines
Quoted 16 lines
Quoted 16 lines
configure +1
-0@@ -15,6 +15,7 @@ objdump=objdump ar=ar addr2line=addr2line arch=$(uname -m | sed -e 's/i.86/i386/;s/arm64/aarch64/;s/arm.*/arm/;s/ppc64.*/ppc64/') +os=$(uname -s) host=$arch cross_prefix= endian=""@@ -317,9 +318,9 @@ EOF rm -f lib-test.{o,S} fiQuoted 7 lines
configure EOF · line 323
Quoted 5 lines
configure +2
-2-# require enhanced getopt +# require enhanced getopt everywhere except illumos getopt -T > /dev/null -if [ $? -ne 4 ]; then +if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; thenWhat does illumos return for
getopt -T?Sadly, it returns "0". I was wrong in my earlier explorations because I did not realize that
configuredoes not usegetoptaside from that one check, which is repeated inrun_tests.sh.Quoted 5 lines
I would argue that the most straight-forward way to deal with this is to just remove the check for "getopt" from "configure", which doesn't otherwise use "getopt". The only place it is used is in
run_tests.sh, which is unlikely to be used directly for illumos, and repeats the check anyway.
Quoted 4 lines
Fine for me if we remove the check from configure, or turn it into a warning instead ("Enhanced getopt is not available, you won't be able to use the
run_tests.shscript" or so).
Ack for simply changing the configure error to a warning for now. Ideally we'd limit the dependencies this project has though. So maybe rewriting the run_tests.sh command line parser without getopt would be the better thing to do.
Thanks,
drew
cover kvm-unit-tests: Build changes to support illumos.
We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.
This patch series modifies the build infrastructure to allow building on illumos; the changes were minimal. I have also tested these changes on Linux to ensure no regressions.
Dan Cross (2):
kvm-unit-tests: invoke $LD explicitly in
kvm-unit-tests: configure changes for illumos.
configure | 5 +++-- x86/Makefile.common | 6 +++--- 2 files changed, 6 insertions(+), 5 deletions(-)
Signature · 1 line
2.31.1
PATCH 1/2 kvm-unit-tests: invoke $LD explicitly in
Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver.
This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.
--- x86/Makefile.common | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
x86/Makefile.common +3 -3
@@ -62,7 +62,7 @@ else .PRECIOUS: %.elf %.o %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o) - $(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \ + $(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \ $(filter %.o, $^) $(FLATLIBS) @chmod a-x $@
@@ -98,8 +98,8 @@ test_cases: $(tests-common) $(tests) $(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o - $(CC) -m32 -nostdlib -o $@ -Wl,-m,elf_i386 \ - -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^ + $(LD) -m elf_i386 -nostdlib -o $@ \ + -T $(SRCDIR)/$(TEST_DIR)/realmode.lds $^ $(TEST_DIR)/realmode.o: bits = $(if $(call cc-option,-m16,""),16,32)
Signature · 1 line
2.31.2
Sean Christopherson
reviewerRe: [PATCH 1/2] kvm-unit-tests: invoke $LD explicitly in
Quoted 25 lines · Dan Cross
Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver.
This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.
Signed-off-by Dan Cross <[email protected]>--- x86/Makefile.common | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)x86/Makefile.common +1
-1@@ -62,7 +62,7 @@ else .PRECIOUS: %.elf %.o %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o) - $(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \ + $(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \ $(filter %.o, $^) $(FLATLIBS) @chmod a-x $@
x86/Makefile.common test_cases: $(tests-common) $(tests) · line 102
Quoted 7 lines
x86/Makefile.common +2
-2@@ -98,8 +98,8 @@ test_cases: $(tests-common) $(tests) $(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o - $(CC) -m32 -nostdlib -o $@ -Wl,-m,elf_i386 \ - -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^ + $(LD) -m elf_i386 -nostdlib -o $@ \ + -T $(SRCDIR)/$(TEST_DIR)/realmode.lds $^
AFAICT, this is functionally a nop, and both 32-bit and 64-bit are happy, so,
Re: [PATCH 1/2] kvm-unit-tests: invoke $LD explicitly in
In reply to Dan CrossQuoted 16 lines · Dan Cross
Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver.
This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.
Signed-off-by Dan Cross <[email protected]>--- x86/Makefile.common | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)x86/Makefile.common +0
-0
x86/Makefile.common else · line 67
Quoted 7 lines
x86/Makefile.common +1
-1@@ -62,7 +62,7 @@ else .PRECIOUS: %.elf %.o %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o) - $(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \ + $(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \ $(filter %.o, $^) $(FLATLIBS) @chmod a-x $@
Hi,
something seems to be missing here - this is failing our 32-bit CI job:
https://gitlab.com/thuth/kvm-unit-tests/-/jobs/2531237708
ld -T /builds/thuth/kvm-unit-tests/x86/flat.lds -nostdlib -o x86/taskswitch2.elf \
x86/taskswitch2.o x86/cstart.o lib/libcflat.a
ld: i386 architecture of input file `x86/taskswitch.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `x86/cstart.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(argv.o)' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(printf.o)' is incompatible with i386:x86-64 output
...
You can find the job definition in the .gitlab-ci.yml file (it's basically just about running "configure" with --arch=i386).
Thomas
Re: [PATCH 1/2] kvm-unit-tests: invoke $LD explicitly in
Quoted 41 lines · Thomas Huth
Quoted 16 lines · Dan Cross
Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver.
This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.
Signed-off-by Dan Cross <[email protected]>--- x86/Makefile.common | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)x86/Makefile.common +0
-0x86/Makefile.common else · line 67
Quoted 7 lines
x86/Makefile.common +1
-1@@ -62,7 +62,7 @@ else .PRECIOUS: %.elf %.o %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o) - $(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \ + $(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \ $(filter %.o, $^) $(FLATLIBS) @chmod a-x $@Hi,
something seems to be missing here - this is failing our 32-bit CI job:
https://gitlab.com/thuth/kvm-unit-tests/-/jobs/2531237708
ld -T /builds/thuth/kvm-unit-tests/x86/flat.lds -nostdlib -o x86/taskswitch2.elf \
x86/taskswitch2.ox86/cstart.olib/libcflat.a
ld: i386 architecture of input file `x86/taskswitch.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `x86/cstart.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(argv.o)' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(printf.o)' is incompatible with i386:x86-64 output
...
Quoted 2 lines
You can find the job definition in the .gitlab-ci.yml file (it's basically just about running "configure" with --arch=i386).
Thanks. I think the easiest way to fix this is to plumb an argument through to the linker that expands to -m elf_i386 on 32-bit, and possibly, -m elf_x86_64 on 64-bit. The architecture specific Makefiles set the ldarch variable, and that doesn't seem used anywhere, but I see that's set to match the string accepted by the linker scripts/objcopy, not something acceptable to -m. However, one can add something to LDARGS, but I see that that's set to include the contents of CFLAGS, which means it includes flags that are not directly consumable by the linker itself.
I think the simplest way forward is to introduce a new variable in x86/Makefile.i386 and x86/Makefile.x86_64 and refer to that in x86/Makefile.common; possibly LDEXTRA? I've got an updated patch here that does that, and it seems to work (building under both illumos and arch locally), but before I send up another patchset, let me know if that sounds acceptable?
- Dan C.
Re: [PATCH 1/2] kvm-unit-tests: invoke $LD explicitly in
Quoted 56 lines · Dan Cross
Quoted 41 lines · Thomas Huth
Quoted 16 lines · Dan Cross
Change x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver.
This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.
Signed-off-by Dan Cross <[email protected]>--- x86/Makefile.common | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)x86/Makefile.common +0
-0x86/Makefile.common else · line 67
Quoted 7 lines
x86/Makefile.common +1
-1@@ -62,7 +62,7 @@ else .PRECIOUS: %.elf %.o %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o) - $(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \ + $(LD) -T $(SRCDIR)/x86/flat.lds -nostdlib -o $@ \ $(filter %.o, $^) $(FLATLIBS) @chmod a-x $@Hi,
something seems to be missing here - this is failing our 32-bit CI job:
https://gitlab.com/thuth/kvm-unit-tests/-/jobs/2531237708
ld -T /builds/thuth/kvm-unit-tests/x86/flat.lds -nostdlib -o x86/taskswitch2.elf \
x86/taskswitch2.ox86/cstart.olib/libcflat.a
ld: i386 architecture of input file `x86/taskswitch.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `x86/cstart.o' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(argv.o)' is incompatible with i386:x86-64 output
ld: i386 architecture of input file `lib/libcflat.a(printf.o)' is incompatible with i386:x86-64 output
...Quoted 2 lines
You can find the job definition in the .gitlab-ci.yml file (it's basically just about running "configure" with --arch=i386).
Thanks. I think the easiest way to fix this is to plumb an argument through to the linker that expands to
-m elf_i386on 32-bit, and possibly,-m elf_x86_64on 64-bit. The architecture specific Makefiles set theldarchvariable, and that doesn't seem used anywhere, but I see that's set to match the string accepted by the linkerscripts/objcopy, not something acceptable to-m. However, one can add something toLDARGS, but I see that that's set to include the contents of CFLAGS, which means it includes flags that are not directly consumable by the linker itself.
Quoted 3 lines
I think the simplest way forward is to introduce a new variable in x86/Makefile.i386 and x86/Makefile.x86_64 and refer to that in x86/Makefile.common; possibly
LDEXTRA?
We do that for arm, but call that argument arch_LDFLAGS.
Quoted 4 lines
I've got an updated patch here that does that, and it seems to work (building under both illumos and arch locally), but before I send up another patchset, let me know if that sounds acceptable?
It does to me.
Thanks,
drew
Re: [PATCH 1/2] kvm-unit-tests: invoke $LD explicitly in
Quoted 12 lines · Andrew Jones
Quoted 11 lines · Dan Cross
Thanks. I think the easiest way to fix this is to plumb an argument through to the linker that expands to
-m elf_i386on 32-bit, and possibly,-m elf_x86_64on 64-bit. The architecture specific Makefiles set theldarchvariable, and that doesn't seem used anywhere, but I see that's set to match the string accepted by the linkerscripts/objcopy, not something acceptable to-m. However, one can add something toLDARGS, but I see that that's set to include the contents of CFLAGS, which means it includes flags that are not directly consumable by the linker itself.
Quoted 5 lines
Quoted 3 lines
I think the simplest way forward is to introduce a new variable in x86/Makefile.i386 and x86/Makefile.x86_64 and refer to that in x86/Makefile.common; possibly
LDEXTRA?We do that for arm, but call that argument
arch_LDFLAGS.
Ah cool; I see the pattern now. Okay, I've modified x86 to follow ARM in this regard and tested locally.
Quoted 9 lines
Quoted 4 lines
I've got an updated patch here that does that, and it seems to work (building under both illumos and arch locally), but before I send up another patchset, let me know if that sounds acceptable?
It does to me.
Thanks,
drew
Great. Next patch rev inbound.
- Dan C.
v4 cover kvm-unit-tests: Build changes to support illumos.
We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.
This patch series modifies the build infrastructure to allow building on illumos; the changes were minimal. I have also tested these changes on Linux to ensure no regressions.
Dan Cross (2):
kvm-unit-tests: configure changes for illumos.
kvm-unit-tests: invoke $LD explicitly
configure | 6 +++--- x86/Makefile.common | 9 +++++---- x86/Makefile.i386 | 1 + x86/Makefile.x86_64 | 2 ++ 4 files changed, 11 insertions(+), 7 deletions(-)
Signature · 1 line
2.36.1
PATCH v4 1/2 kvm-unit-tests: configure changes for illumos.
Warn, don't fail, if the check for getopt -T fails in the configure script.
Aside from this check, configure does not use getopt, so don't fail to run if getopt -T doesn't indicate support for the extended Linux version. Getopt is only used in run_tests.sh, which tests for extended getopt anyway, but emit a warning here.
--- configure | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
configure +3 -3
@@ -318,11 +318,11 @@ EOF rm -f lib-test.{o,S} fi -# require enhanced getopt +# warn if enhanced getopt is unavailable getopt -T > /dev/null if [ $? -ne 4 ]; then - echo "Enhanced getopt is not available, add it to your PATH?" - exit 1 + echo "Without enhanced getopt you won't be able to use run_tests.sh." + echo "Add it to your PATH?" fi # Are we in a separate build tree? If so, link the Makefile
Signature · 1 line
2.36.1
PATCH v4 2/2 kvm-unit-tests: invoke $LD explicitly
In reply to Dan CrossChange x86/Makefile.common to invoke the linker directly instead of using the C compiler as a linker driver. Plumb LDFLAGS through to the linker.
This supports building on illumos, allowing the user to use gold instead of the Solaris linker. Tested on Linux and illumos.
--- x86/Makefile.common | 9 +++++---- x86/Makefile.i386 | 1 + x86/Makefile.x86_64 | 2 ++ 3 files changed, 8 insertions(+), 4 deletions(-)
x86/Makefile.common +5 -4
@@ -30,7 +30,7 @@ endif OBJDIRS += lib/x86 -$(libcflat): LDFLAGS += -nostdlib +$(libcflat): LDFLAGS += -nostdlib $(arch_LDFLAGS) $(libcflat): CFLAGS += -ffreestanding -I $(SRCDIR)/lib -I lib COMMON_CFLAGS += -m$(bits)
@@ -61,8 +61,9 @@ else # We want to keep intermediate file: %.elf and %.o .PRECIOUS: %.elf %.o +%.elf: LDFLAGS = -nostdlib $(arch_LDFLAGS) %.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o) - $(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \ + $(LD) $(LDFLAGS) -T $(SRCDIR)/x86/flat.lds -o $@ \ $(filter %.o, $^) $(FLATLIBS) @chmod a-x $@
@@ -98,8 +99,8 @@ test_cases: $(tests-common) $(tests) $(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o - $(CC) -m32 -nostdlib -o $@ -Wl,-m,elf_i386 \ - -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^ + $(LD) -m elf_i386 -nostdlib -o $@ \ + -T $(SRCDIR)/$(TEST_DIR)/realmode.lds $^ $(TEST_DIR)/realmode.o: bits = $(if $(call cc-option,-m16,""),16,32)
x86/Makefile.i386 +1 -0
@@ -4,6 +4,7 @@ ldarch = elf32-i386 exe = flat bin = elf COMMON_CFLAGS += -mno-sse -mno-sse2 +arch_LDFLAGS = -m elf_i386 cflatobjs += lib/x86/setjmp32.o lib/ldiv32.o
x86/Makefile.x86_64 +2 -0
@@ -6,9 +6,11 @@ exe = efi bin = so FORMAT = efi-app-x86_64 cstart.o = $(TEST_DIR)/efi/efistart64.o +arch_LDFLAGS = '' else exe = flat bin = elf +arch_LDFLAGS = -m elf_x86_64 endif fcf_protection_full := $(call cc-option, -fcf-protection=full,)
Signature · 1 line
2.36.1
Re: [PATCH v4 0/2] kvm-unit-tests: Build changes to support illumos.
In reply to Dan CrossQuoted 9 lines · Dan Cross
We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.
This patch series modifies the build infrastructure to allow building on illumos; the changes were minimal. I have also tested these changes on Linux to ensure no regressions.
Thanks, this version survived the CI, so I've applied it now.
Thomas
Re: [PATCH v4 0/2] kvm-unit-tests: Build changes to support illumos.
Quoted 7 lines · Thomas Huth
Quoted 6 lines · Dan Cross
We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.
Quoted 5 lines
Quoted 3 lines
This patch series modifies the build infrastructure to allow building on illumos; the changes were minimal. I have also tested these changes on Linux to ensure no regressions.
Thanks, this version survived the CI, so I've applied it now.
Thank you very much!
- Dan C.
PATCH 2/2 kvm-unit-tests: configure changes for illumos.
In reply to Dan CrossWarn, don't fail, if the check for getopt -T fails in the configure script.
Aside from this check, configure does not use getopt, so don't fail to run if getopt -T doesn't indicate support for the extended Linux version. Getopt is only used in run_tests.sh, which tests for extended getopt anyway, but emit a warning here.
--- configure | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
configure +3 -3
@@ -318,11 +318,11 @@ EOF rm -f lib-test.{o,S} fi -# require enhanced getopt +# warn if enhanced getopt is unavailable getopt -T > /dev/null if [ $? -ne 4 ]; then - echo "Enhanced getopt is not available, add it to your PATH?" - exit 1 + echo "Without enhanced getopt you won't be able to use run_tests.sh." + echo "Add it to your PATH?" fi # Are we in a separate build tree? If so, link the Makefile
Signature · 1 line
2.31.2
Sean Christopherson
reviewerRe: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
In the future, please tag each new posting with a version number, e.g. no version number for the first posting, than v2, v3, v4, etc... for each subsequent posting. It took me a while to figure what was what.
Quoted 7 lines · Dan Cross
Warn, don't fail, if the check for
getopt -Tfails in theconfigurescript.Aside from this check,
configuredoes not usegetopt, so don't fail to run ifgetopt -Tdoesn't indicate support for the extended Linux version. Getopt is only used inrun_tests.sh, which tests for extended getopt anyway, but emit a warning here.
Why not simply move the check to run_tests.sh? I can't imaging it's performance sensitive, and I doubt I'm the only one that builds tests on one system and runs them on a completely different system.
Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
Quoted 3 lines · Sean Christopherson
In the future, please tag each new posting with a version number, e.g. no version number for the first posting, than v2, v3, v4, etc... for each subsequent posting. It took me a while to figure what was what.
Ah, sorry about that. Sure thing.
Quoted 12 lines · Dan Cross
Quoted 7 lines
Warn, don't fail, if the check for
getopt -Tfails in theconfigurescript.Aside from this check,
configuredoes not usegetopt, so don't fail to run ifgetopt -Tdoesn't indicate support for the extended Linux version. Getopt is only used inrun_tests.sh, which tests for extended getopt anyway, but emit a warning here.Why not simply move the check to
run_tests.sh? I can't imaging it's performance sensitive, and I doubt I'm the only one that builds tests on one system and runs them on a completely different system.
run_tests.sh already has the test. Changing it to a warning here was at the suggestion of Thomas and Drew.
- Dan C.
Sean Christopherson
reviewerRe: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
Quoted 7 lines · Dan Cross
Quoted 3 lines · Sean Christopherson
Why not simply move the check to
run_tests.sh? I can't imaging it's performance sensitive, and I doubt I'm the only one that builds tests on one system and runs them on a completely different system.
run_tests.shalready has the test. Changing it to a warning here was at the suggestion of Thomas and Drew.
Ah, hence the earlier comment about removing the check entirely. Either works for me, thanks!
Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
Quoted 11 lines · Sean Christopherson
Quoted 7 lines · Dan Cross
Quoted 3 lines · Sean Christopherson
Why not simply move the check to
run_tests.sh? I can't imaging it's performance sensitive, and I doubt I'm the only one that builds tests on one system and runs them on a completely different system.
run_tests.shalready has the test. Changing it to a warning here was at the suggestion of Thomas and Drew.Ah, hence the earlier comment about removing the check entirely. Either works for me, thanks!
Thanks for the helpful comments on the review!
- Dan C.
Re: [PATCH 2/2] kvm-unit-tests: configure changes for illumos.
In reply to Andrew JonesQuoted 10 lines · Andrew Jones
Quoted 4 lines · Thomas Huth
[snip]
Fine for me if we remove the check from configure, or turn it into a warning instead ("Enhanced getopt is not available, you won't be able to use therun_tests.shscript" or so).Ack for simply changing the configure error to a warning for now. Ideally we'd limit the dependencies this project has though. So maybe rewriting the
run_tests.shcommand line parser without getopt would be the better thing to do.
Sounds good. New patches inbound. Thank you!
- Dan C.
A message that is not in the archive (sent off-list or not yet received): CAA9fzEGdi0k8bkyXQwvt6gFd-gwHNNFF7A89U4DhtGHjKqe4AQ@mail.gmail.com
PATCH 1/1 kvm-unit-tests: Build changes for illumos.
[Sean, sorry for the dup; I forgot to switch to plaintext mode]
Quoted 15 lines · Sean Christopherson
Quoted 6 lines · Dan Cross
We have begun using kvm-unit-tests to test Bhyve under illumos. We started by cross-compiling the tests on Linux and transfering the binary artifacts to illumos machines, but it proved more convenient to build them directly on illumos.
Quoted 7 lines
This change modifies the build infrastructure to allow building on illumos; I have also tested it on Linux. The required changes were pretty minimal: the most invasive was switching from using the C compiler as a linker driver to simply invoking the linker directly in two places. This allows us to easily use gold instead of the Solaris linker.
Quoted 3 lines
Can you please split this into two patches? One for the $(CC) => $(LD) change, and one for the getopt thing. The switch to $(LD) in particular could be valuable irrespective of using a non-Linux OS.
Done.
Quoted 9 lines
[snip]
configure EOF · line 323
Quoted 4 lines
configure +2
-1+# require enhanced getopt everywhere except illumos getopt -T > /dev/null -if [ $? -ne 4 ]; then +if [ $? -ne 4 ] && [ "$os" != "SunOS" ]; thenPresumably whatever "enhanced" features are being used are supported by illumos, so rather than check the OS, why not improve the probing?
Actually, it looks like I've got some egg on my face here. It appears that getopt
is not used in configure beyond this check; presumably this exists to avoid surprises
when running run_tests.sh, which does use extended getopt (for long options) and
repeats this same test. I had not noticed since configure ran successfully, and we are
not presently using run_tests.sh on illumos as we use a rather different VMM than
qemu. That patch should probably be dropped (or changed to remove the getopt check
in configure entirely, if folks are fine deferring needing to have it to the invocation ofrun_tests.sh...). Or I just convince folks to provide a GNU getopt package on illumos.
Let me know what is preferred.
- Dan C.