Skip to contentSkip to conversation

minios: minor buildsystem fixes

PATCH In review Christoph Egger 5 participants 15 messages Jul 15, 2009 – Jul 16, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 minios: minor buildsystem fixes

Hi!

Attached patch changes MiniOS.mk and tools/Rules.mk use the OS settings we build on.

Signature · 6 lines
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
config/MiniOS.mk +1 -1
diff -r 323ae92f774b config/MiniOS.mk
@@ -1,4 +1,4 @@
-include $(XEN_ROOT)/config/StdGNU.mk
+include $(XEN_ROOT)/config/$(shell uname -s).mk
 include $(XEN_ROOT)/extras/mini-os/Config.mk
 CFLAGS += $(DEF_CFLAGS) $(ARCH_CFLAGS)
 CPPFLAGS += $(DEF_CPPFLAGS) $(ARCH_CPPFLAGS) $(extra_incl)
tools/Rules.mk +4 -1
diff -r 323ae92f774b tools/Rules.mk
@@ -33,7 +33,10 @@ CFLAGS += -D__XEN_TOOLS__
 CFLAGS += -MMD -MF .$(@F).d
 DEPS = .*.d
 
-ifneq ($(XEN_OS),NetBSD)
+# On NetBSD LFS is always on.
+# We can't use $(XEN_OS) or we will miss MiniOS
+# when we build MiniOS on NetBSD.
+ifneq ($(shell uname -s),NetBSD)
 # Enable implicit LFS support *and* explicit LFS names.
 CFLAGS  += $(shell getconf LFS_CFLAGS)
 CFLAGS  += -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw
Quoted 4 lines · Christoph Egger

Attached patch changes MiniOS.mk and tools/Rules.mk use the OS settings we build on.

This may need some thought. Using XEN_OS in some places and open coding 'uname -s' in others is just not nice and is fragile.

Since stubdom is not being built in its own chroot, does it make sense for it to override XEN_OS at all? Should we keep that as 'build host OS'? The change around LFS in tools/Rules.mk could be indicative of a deeper issue -- information about arbitrary (non-GNU) host environments leaking into binaries which are actually targetting GNUish minios. Can the current approach work safely for NetBSD, Solaris, ... build hosts?

So I throw this one to Stefano/Samuel. ;-)

-- Keir

#2 lore raw
Quoted 9 lines · On Wednesday 15 July 2009 12:40:22 Keir Fraser wrote:
Quoted 4 lines · Christoph Egger

Attached patch changes MiniOS.mk and tools/Rules.mk use the OS settings we build on.

This may need some thought. Using XEN_OS in some places and open coding 'uname -s' in others is just not nice and is fragile.

Quoted 8 lines

Since stubdom is not being built in its own chroot, does it make sense for it to override XEN_OS at all? Should we keep that as 'build host OS'? The change around LFS in tools/Rules.mk could be indicative of a deeper issue -- information about arbitrary (non-GNU) host environments leaking into binaries which are actually targetting GNUish minios. Can the current approach work safely for NetBSD, Solaris, ... build hosts?

So I throw this one to Stefano/Samuel. ;-)

The current approach works for Linux only. NetBSD's base system is designed for cross-building. My roadmap is to change the way how to build mini-os/stubdom fundamentally:

The new infrastructure I have in mind will allow to choose if you use newlib, libz, lwip and pciutils or use the native libs from the base system.

Linux build will default to first, NetBSD build will default to latter one.

NetBSD libc doesn't have linux integer types, that's why mini-os has been moved to C99 integer types. The mini-os headers need some re-structuring to not conflict with the native libc, namely if you want a mini-os header you explicitely have to type <mini-os/header.h>

Christoph

Signature · 6 lines
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
#3 lore raw
Quoted 10 lines · Christoph Egger

The current approach works for Linux only. NetBSD's base system is designed for cross-building. My roadmap is to change the way how to build mini-os/stubdom fundamentally:

The new infrastructure I have in mind will allow to choose if you use newlib, libz, lwip and pciutils or use the native libs from the base system.

Linux build will default to first, NetBSD build will default to latter one.

I like the idea, could you please explain with a little bit more details how the crossbuilding works on NetBSD?
Would it be possible to use NetBSD libc instead of newlib even on Linux?

However Keir is right about XEN_OS: the stubdom Makefile shouldn't change it but should only set CONFIG_MiniOS instead.

#4 lore raw
Quoted 4 lines · Stefano Stabellini
Quoted 4 lines

The new infrastructure I have in mind will allow to choose if you use newlib, libz, lwip and pciutils or use the native libs from the base system.

Quoted 8 lines
Quoted 1 line

Linux build will default to first, NetBSD build will default to latter one.

I like the idea, could you please explain with a little bit more details how the crossbuilding works on NetBSD?
Would it be possible to use NetBSD libc instead of newlib even on Linux?

However Keir is right about XEN_OS: the stubdom Makefile shouldn't change it but should only set CONFIG_MiniOS instead.

If the distinction that Christoph's patch introduced *were* to be required, I would do it as XEN_COMPILE_OS and XEN_TARGET_OS replacing XEN_OS. Arguably it would be clearer to rename existing XEN_OS to the former anyway?

-- Keir

#5 lore raw
Quoted 13 lines · Keir Fraser
Quoted 10 lines · Stefano Stabellini
Quoted 5 lines

The new infrastructure I have in mind will allow to choose if you use newlib, libz, lwip and pciutils or use the native libs from the base system.

Linux build will default to first, NetBSD build will default to latter one.

I like the idea, could you please explain with a little bit more details how the crossbuilding works on NetBSD?
Would it be possible to use NetBSD libc instead of newlib even on Linux?

Quoted 7 lines
Quoted 2 lines

However Keir is right about XEN_OS: the stubdom Makefile shouldn't change it but should only set CONFIG_MiniOS instead.

If the distinction that Christoph's patch introduced *were* to be required, I would do it as XEN_COMPILE_OS and XEN_TARGET_OS replacing XEN_OS. Arguably it would be clearer to rename existing XEN_OS to the former anyway?

I agree on that, XEN_OS as it is now is misleading.

#6 lore raw
Quoted 6 lines · Stefano Stabellini
Quoted 4 lines

If the distinction that Christoph's patch introduced *were* to be required, I would do it as XEN_COMPILE_OS and XEN_TARGET_OS replacing XEN_OS. Arguably it would be clearer to rename existing XEN_OS to the former anyway?

I agree on that, XEN_OS as it is now is misleading.

I changed my mind after all that. Obviously it is rather entrenched in tools/ right now that the compile OS is also the target OS. Renaming XEN_OS to one or the other gives the misleading impression we can cross-compile across operating systems, which we can't.

I think your plan to leave XEN_OS alone and just define CONFIG_MiniOS -- and then hack makefiles a bit where necessary to explicitly take note of the latter -- makes most sense.

-- Keir

#7 lore raw
In reply to Stefano Stabellini
Quoted 10 lines · On Wednesday 15 July 2009 14:35:37 Stefano Stabellini wrote:
Quoted 9 lines · Christoph Egger

The current approach works for Linux only. NetBSD's base system is designed for cross-building. My roadmap is to change the way how to build mini-os/stubdom fundamentally:

The new infrastructure I have in mind will allow to choose if you use newlib, libz, lwip and pciutils or use the native libs from the base system.

Quoted 5 lines
Quoted 2 lines

Linux build will default to first, NetBSD build will default to latter one.

I like the idea, could you please explain with a little bit more details how the crossbuilding works on NetBSD?

The headers are in /usr/include. The arch-dependent headers are in /usr/include/<arch>/. These are automatically included by the standard headers.

The native libs are in /lib and in /usr/lib/. The libs for the other architectures are in /usr/lib/<arch>/.

On x86, it is enough to specify -m64 or -m32 to tell gcc and its linker if you want to build a binary for amd64 or i386.

If you want to build for other archs, you build a toolchain first which is very easy with the build.sh script.

Quoted 1 line

Would it be possible to use NetBSD libc instead of newlib even on Linux?

It's not just to take the libc sources. You need the whole infrastructure around it which would let you very likely end up in a half new linux distribution. I don't think, this is what you want. :)

BTW: I don't mind if you want to port Xen to a VAX :-D

Christoph

Signature · 6 lines
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
#8 lore raw
In reply to Christoph Egger

Christoph Egger writes ("[Xen-devel] [PATCH] minios: minor buildsystem fixes"):

Quoted 2 lines

Attached patch changes MiniOS.mk and tools/Rules.mk use the OS settings we build on.

I don't think that's right. The minios cross-build should use the Xen cross build machinery. That machinery isn't very good but it is the same on every host.

Christoph Egger writes ("Re: [Xen-devel] [PATCH] minios: minor buildsystem fixes"):

Quoted 1 line

The current approach works for Linux only.

Really ? What's wrong with it for NetBSD ?

Quoted 1 line

NetBSD's base system is designed for cross-building.

I don't think we should be trying to use the host operating system's cross-build support.

Quoted 3 lines

The new infrastructure I have in mind will allow to choose if you use newlib, libz, lwip and pciutils or use the native libs from the base system.

And certainly not the host operating system's libraries for our cross-build !

Quoted 1 line

Linux build will default to first, NetBSD build will default to latter one.

In particular I think this is quite wrong.

Quoted 5 lines

NetBSD libc doesn't have linux integer types, that's why mini-os has been moved to C99 integer types. The mini-os headers need some re-structuring to not conflict with the native libc, namely if you want a mini-os header you explicitely have to type <mini-os/header.h>

I agree that it would be good to replace the minios libc. The current libc is horrid. The BSD libc is very nice (and I have found it a joy to port to a new embedded target, in the past).

But if we change to a new libc this should be done by always using the new libc (BSD-derived, but not necessarily NetBSD; FreeBSD is another choice) even on Linux hosts.

Ian.

#9 lore raw
Quoted 3 lines · Ian Jackson
Quoted 1 line

Linux build will default to first, NetBSD build will default to latter one.

In particular I think this is quite wrong.

After giving it more thoughts, I agree with Ian.

Quoted 7 lines
Quoted 6 lines

NetBSD libc doesn't have linux integer types, that's why mini-os has been moved to C99 integer types. The mini-os headers need some re-structuring to not conflict with the native libc, namely if you want a mini-os header you explicitely have to type <mini-os/header.h>

Quoted 8 lines

I agree that it would be good to replace the minios libc. The current libc is horrid. The BSD libc is very nice (and I have found it a joy to port to a new embedded target, in the past).

But if we change to a new libc this should be done by always using the new libc (BSD-derived, but not necessarily NetBSD; FreeBSD is another choice) even on Linux hosts.

In the process of making qemu stubdoms work, we certainly found many shortcomings in newlib, so I welcome the idea of switching to a different libc, especially if we want to make minios a good environment to run other applications.
But maintaining a cross-compile environment is hard enough, we don't want to maintain two of them, so the new libc should compile on Linux hosts too.

#10 lore raw
In reply to Ian Jackson

Hello,

Quoted 6 lines · Ian Jackson, le Wed 15 Jul 2009 17:03:36 +0100, a écrit :
Quoted 3 lines

The new infrastructure I have in mind will allow to choose if you use newlib, libz, lwip and pciutils or use the native libs from the base system.

And certainly not the host operating system's libraries for our cross-build !

Yes, that's asking for problems.

Quoted 9 lines
Quoted 5 lines

NetBSD libc doesn't have linux integer types, that's why mini-os has been moved to C99 integer types. The mini-os headers need some re-structuring to not conflict with the native libc, namely if you want a mini-os header you explicitely have to type <mini-os/header.h>

I agree that it would be good to replace the minios libc. The current libc is horrid. The BSD libc is very nice (and I have found it a joy to port to a new embedded target, in the past).

That should be fine yes. I don't remember precisely the reasons why I chose newlib. I didn't know the BSD libc had embedded targets too.

Quoted 2 lines

But if we change to a new libc this should be done by always using the new libc even on Linux hosts.

Yes. We should avoid having to fix several libcs.

Samuel

#11 lore raw
In reply to Ian Jackson
Quoted 1 line · On Wednesday 15 July 2009 18:03:36 Ian Jackson wrote:

Christoph Egger writes ("[Xen-devel] [PATCH] minios: minor buildsystem

fixes"):

Quoted 6 lines
Quoted 2 lines

Attached patch changes MiniOS.mk and tools/Rules.mk use the OS settings we build on.

I don't think that's right. The minios cross-build should use the Xen cross build machinery. That machinery isn't very good but it is the same on every host.

It's not the same on every host due to lack of proper abstraction.

Quoted 1 line

Christoph Egger writes ("Re: [Xen-devel] [PATCH] minios: minor buildsystem

fixes"):

Quoted 3 lines
Quoted 1 line

The current approach works for Linux only.

Really ? What's wrong with it for NetBSD ?

Assumptions which don't apply to NetBSD.

Quoted 4 lines
Quoted 1 line

NetBSD's base system is designed for cross-building.

I don't think we should be trying to use the host operating system's cross-build support.

Please explain why not.
It's a question of having a proper abstraction.

Quoted 6 lines
Quoted 3 lines

The new infrastructure I have in mind will allow to choose if you use newlib, libz, lwip and pciutils or use the native libs from the base system.

And certainly not the host operating system's libraries for our cross-build !

Please explain why not.
It's a question of having a proper abstraction.

Quoted 4 lines
Quoted 2 lines

Linux build will default to first, NetBSD build will default to latter one.

In particular I think this is quite wrong.

Please explain why.

Quoted 6 lines
Quoted 5 lines

NetBSD libc doesn't have linux integer types, that's why mini-os has been moved to C99 integer types. The mini-os headers need some re-structuring to not conflict with the native libc, namely if you want a mini-os header you explicitely have to type <mini-os/header.h>

Quoted 7 lines

I agree that it would be good to replace the minios libc. The current libc is horrid. The BSD libc is very nice (and I have found it a joy to port to a new embedded target, in the past).

But if we change to a new libc this should be done by always using the new libc (BSD-derived, but not necessarily NetBSD; FreeBSD is another choice) even on Linux hosts.

I think, let's first find a proper abstraction and then make the decision what is best for which host.

Christoph

Signature · 6 lines
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Thomas M. McCoy, Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
#12 lore raw
Quoted 5 lines · Christoph Egger, le Thu 16 Jul 2009 14:17:20 +0200, a écrit :
Quoted 2 lines

And certainly not the host operating system's libraries for our cross-build !

Please explain why not.
It's a question of having a proper abstraction.

It's a problem of ABI. For instance, there is no way to implement x86_64's red zone without adding a kernel/user separation in MiniOS ; libcs usually assume that the red zone is available, thus some patching to newlib for instance.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

Linux build will default to first, NetBSD build will default to latter one.

In particular I think this is quite wrong.

Please explain why.

It's asking for having to fix issues coming from both linux' case and NetBSD's case, while using the same libc all the time would bring just one case.

Samuel

#13 lore raw
In reply to Christoph Egger

Christoph Egger writes ("Re: [Xen-devel] [PATCH] minios: minor buildsystem fixes"):

Quoted 4 lines · On Wednesday 15 July 2009 18:03:36 Ian Jackson wrote:
Quoted 1 line

Really ? What's wrong with it for NetBSD ?

Assumptions which don't apply to NetBSD.

Specifically.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

NetBSD's base system is designed for cross-building.

I don't think we should be trying to use the host operating system's cross-build support.

Please explain why not.
It's a question of having a proper abstraction.

No, it's a question of trying to run two parallel cross build arrangements.

Quoted 2 lines

I think, let's first find a proper abstraction and then make the decision what is best for which host.

It does not make sense that the results of a cross-build (such as which libc is used) should depend on the host.

Ian.

#14 lore raw
In reply to Christoph Egger
Quoted 6 lines · Christoph Egger

Hi!

Attached patch changes MiniOS.mk and tools/Rules.mk use the OS settings we build on.

Looking at the original patch of this thread, I realize that probably none of this is actually needed:

config/MiniOS.mk line 4

Quoted 9 lines

diff -r 323ae92f774b config/MiniOS.mk

config/MiniOS.mk +1 -1
@@ -1,4 +1,4 @@
-include $(XEN_ROOT)/config/StdGNU.mk
+include $(XEN_ROOT)/config/$(shell uname -s).mk
 include $(XEN_ROOT)/extras/mini-os/Config.mk
 CFLAGS += $(DEF_CFLAGS) $(ARCH_CFLAGS)
 CPPFLAGS += $(DEF_CPPFLAGS) $(ARCH_CPPFLAGS) $(extra_incl)

This is not needed because the only things that NetBSD.mk adds more than StdGNU.mk are not needed in the stubdom case.

Quoted 5 lines

diff -r 323ae92f774b tools/Rules.mk

tools/Rules.mk +0 -0
@@ -33,7 +33,10 @@ CFLAGS += -D__XEN_TOOLS__
 CFLAGS += -MMD -MF .$(@F).d

tools/Rules.mk CFLAGS += -D__XEN_TOOLS__ · line 42

Quoted 10 lines
tools/Rules.mk +4 -1
 DEPS = .*.d
 
-ifneq ($(XEN_OS),NetBSD)
+# On NetBSD LFS is always on.
+# We can't use $(XEN_OS) or we will miss MiniOS
+# when we build MiniOS on NetBSD.
+ifneq ($(shell uname -s),NetBSD)
 # Enable implicit LFS support *and* explicit LFS names.
 CFLAGS  += $(shell getconf LFS_CFLAGS)
 CFLAGS  += -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE

Given that you are still using GNU gcc on NetBSD, even if LFS is already enabled by default, how is this an error?
Does it actually prevent you from compiling successfully qemu?

If you really need to disable those flags you should do that based on getconf responses.