From: Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= Subject: Re: [PATCH]: Implement bzip2 and LZMA loaders / xen-unstable stubdom pvgrub support Date: Tue, 1 Sep 2009 22:08:55 +0300 Message-ID: <20090901190855.GC31123@reaktio.net> References: <20090821201200.GW19938@edu.joroinen.fi> <20090821212230.GX19938@edu.joroinen.fi> <20090825152319.GB19938@edu.joroinen.fi> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="SkvwRMAIpAhPCcCJ" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <20090825152319.GB19938@edu.joroinen.fi> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Stefano Stabellini Cc: Chris Lalancette , Gerd Hoffmann , "xen-devel@lists.xensource.com" , Keir Fraser List-Id: xen-devel@lists.xenproject.org --SkvwRMAIpAhPCcCJ Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 25, 2009 at 06:23:19PM +0300, Pasi K=E4rkk=E4inen wrote: > On Mon, Aug 24, 2009 at 03:04:00PM +0100, Stefano Stabellini wrote: > > On Fri, 21 Aug 2009, Pasi K=E4rkk=E4inen wrote: > > > On Fri, Aug 21, 2009 at 11:12:00PM +0300, Pasi K=E4rkk=E4inen wrote= : > > > > On Fri, Aug 21, 2009 at 11:13:23AM +0100, Keir Fraser wrote: > > > > > On 21/08/2009 10:54, "Keir Fraser" = wrote: > > > > >=20 > > > > > >> Good point. pvgrub should support these aswell. > > > > > >>=20 > > > > > >> Keir: Would you like to add those libs to stubdom? > > > > > >>=20 > > > > > >> (Note the lzma.h is coming from that "xz" package..) > > > > > >=20 > > > > > > No. :-) But feel free to make a patch! You'll just need to mo= dify > > > > > > libxc/Makefile too to force those libs on instead of off, as = well as > > > > > > download and build the libs of course. > > > > >=20 > > > > > Actually I just checked in c/s 20105 to get rid of the minios-s= pecific hack > > > > > in that Makefile. So all you need to do is patch to download an= d build the > > > > > libs. If you make a patch, I will arrange for the lib tarballs = to be hosted > > > > > on xenbits and adjust the patch URLs appropriately. > > > > >=20 > > > >=20 > > > > I can now see why you didn't want to do that.. :) I managed to ge= t the libs to=20 > > > > compile, but now I'm having other problems.. > > > >=20 > > > > "make stubdom" gets to the point where it compiles ioemu-stubdom,= but it fails=20 > > > > to link because xen-unstable.hg/stubdom/mini-os-x86_32-ioemu/mini= -os.o has=20 > > > > undefined references in xc_try_bzip2_decode and xc_try_lzma_decod= e functions..=20 > > > >=20 > > > > I guess only pvgrub would need to have this stuff in? I've been t= rying to > > > > figure out how the makefile magic works, but haven't really under= stood it yet. > > > >=20 > > > > Any tips how to make ioemu-stubdom compile/link without including= bzip2/lzma stuff into it?=20 > > > >=20 > > >=20 > > > Actually I guess I could add the missing libs to extras/mini-os/Mak= efile to > > > APP_LDLIBS to get them included into ioemu-stubdom? > > >=20 > > > Is that the right way to do it? > > >=20 > >=20 > > bzlib and lzma are libxc dependencies now, so firstly you have to add > > the two libraries to the stubdom build system, take a look at > > pciutils in stubdom/Makefile, that is a good example of how a new > > library is added. >=20 > This is done, and works. libbz2.a and liblzma.a end up in > stubdom/cross-root-i686/i686-xen-elf/lib/ directory on my 32bit host. >=20 > > Then you also need to add them both at least to the libxc target, tak= e a > > look at zlib (and cross-zlib), that should be close to what you need. >=20 > Done. >=20 > > Finally you need to tweak the libxc Makefile for the stubdom case so > > that the two libraries are correctly added to zlib-options. > >=20 >=20 > So you mean xen-unstable/tools/libxc/Makefile here? This part? >=20 > ifeq ($(CONFIG_MiniOS),y) > zlib-options =3D >=20 >=20 >=20 > The problem I'm seeing now is "ioemu" target from xen-unstable/stubdom/= Makefile fails to > link: >=20 > ld -nostdlib -L/root/tem/xen-unstable.hg/stubdom/cross-root-i686/i686-x= en-elf/lib -m elf_i386 -T arch/x86/minios-x86_32.lds > /root/tem/xen-unstable.hg/stubdom/mini-os-x86_32-ioemu/mini-os.o -o /r= oot/tem/xen-unstable.hg/stubdom/mini-os-x86_32-ioemu/mini-os > ld: warning: section `.bss' type changed to PROGBITS=20 > /root/tem/xen-unstable.hg/stubdom/mini-os-x86_32-ioemu/mini-os.o: In fu= nction `xc_try_bzip2_decode': > /root/tem/xen-unstable.hg/stubdom/libxc-x86_32/xc_dom_bzimageloader.c:4= 2: undefined reference to `BZ2_bzDecompressInit' > /root/tem/xen-unstable.hg/stubdom/libxc-x86_32/xc_dom_bzimageloader.c:1= 05: undefined reference to `BZ2_bzDecompressEnd' > /root/tem/xen-unstable.hg/stubdom/libxc-x86_32/xc_dom_bzimageloader.c:6= 9: undefined reference to `BZ2_bzDecompress' > /root/tem/xen-unstable.hg/stubdom/libxc-x86_32/xc_dom_bzimageloader.c:1= 05: undefined reference to `BZ2_bzDecompressEnd' > /root/tem/xen-unstable.hg/stubdom/mini-os-x86_32-ioemu/mini-os.o: In fu= nction `xc_try_lzma_decode': > /root/tem/xen-unstable.hg/stubdom/libxc-x86_32/xc_dom_bzimageloader.c:1= 58: undefined reference to `lzma_alone_decoder' > /root/tem/xen-unstable.hg/stubdom/libxc-x86_32/xc_dom_bzimageloader.c:2= 53: undefined reference to `lzma_end' > /root/tem/xen-unstable.hg/stubdom/libxc-x86_32/xc_dom_bzimageloader.c:1= 85: undefined reference to `lzma_code' > /root/tem/xen-unstable.hg/stubdom/libxc-x86_32/xc_dom_bzimageloader.c:2= 53: undefined reference to `lzma_end' > make[1]: *** > [/root/tem/xen-unstable.hg/stubdom/mini-os-x86_32-ioemu/mini-os] Error = 1 > make[1]: Leaving directory `/root/tem/xen-unstable.hg/extras/mini-os' > make: *** [ioemu-stubdom] Error 2 >=20 > So basicly I should add libbz2.a and liblzma.a to the list of files to = link, > but I'm a bit lost where this is happening. Should I edit > xen-unstable/extras/mini-os/Makefile to add those libs or somewhere els= e?=20 >=20 > I tried adding those libs to xen-unstable/stubdom/Makefile to "ioemu" T= ARGET_LDFLAGS, but > that didn't seem to help. >=20 > All tips welcome :) >=20 Attached are my current work-in-progress patches. It still doesn't link p= roperly..=20 Stefano: Would you like to take a look? The patches are against current x= en-unstable. I'm getting weird linking error about stdin/stdout/stderr missing..=20 libc (newlib) is definitely linked in, so I'm not sure what's happening..= =20 ld -nostdlib -L/root/xen-unstable.hg/stubdom/cross-root-i686/i686-xen-elf= /lib -m elf_i386 -T arch/x86/minios-x86_32.lds /root/xen-unstable.hg/stubdom/mini-os-x86_32-ioemu/mini-os.o -o /root/xen= -unstable.hg/stubdom/mini-os-x86_32-ioemu/mini-os xen-unstable.hg/stubdom/mini-os-x86_32-ioemu/mini-os.o: In function `bzop= en_or_bzdopen': xen-unstable.hg/stubdom/bzip2-x86_32/bzlib.c:1411: undefined reference to= `__ctype_b_loc' xen-unstable.hg/stubdom/bzip2-x86_32/bzlib.c:1447: undefined reference to= `stdin' xen-unstable.hg/stubdom/bzip2-x86_32/bzlib.c:1447: undefined reference to= `stdout' ... In the earlier step mini-os.o was definitely linked against -lc ..=20 "strings /path/mini-os.o | grep -i stdin" doesn't give anything though..=20 I wonder why the stuff doesn't get included there. -- Pasi --SkvwRMAIpAhPCcCJ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="stubdom-Makefile-bzip2-lzma-support-v1.patch" --- xen-unstable.hg/stubdom/Makefile.backup.orig 2009-09-01 18:30:06.000000000 +0300 +++ xen-unstable.hg/stubdom/Makefile 2009-09-01 20:32:20.000000000 +0300 @@ -11,6 +11,14 @@ ZLIB_URL=$(XEN_EXTFILES_URL) ZLIB_VERSION=1.2.3 +#BZIP2_URL?=http://www.bzip.org/1.0.5/bzip2-1.0.5.tar.gz +BZIP2_URL=http://www.bzip.org/1.0.5 +BZIP2_VERSION=1.0.5 + +#LZMA_URL?=http://tukaani.org/xz/xz-4.999.9beta.tar.gz +LZMA_URL=http://tukaani.org/xz +LZMA_VERSION=4.999.9beta + #LIBPCI_URL?=http://www.kernel.org/pub/software/utils/pciutils LIBPCI_URL?=$(XEN_EXTFILES_URL) LIBPCI_VERSION=2.2.9 @@ -134,6 +142,51 @@ $(MAKE) libz.a && \ $(MAKE) install ) +############# +# Cross-bzip2 +############# + +bzip2-$(BZIP2_VERSION).tar.gz: + $(WGET) $(BZIP2_URL)/$@ + +bzip2-$(XEN_TARGET_ARCH): bzip2-$(BZIP2_VERSION).tar.gz + tar xzf $< + mv bzip2-$(BZIP2_VERSION) $@ + +BZIP2_STAMPFILE=$(CROSS_ROOT)/$(GNU_TARGET_ARCH)-xen-elf/lib/libbz2.a +.PHONY: cross-bzip2 +cross-bzip2: $(BZIP2_STAMPFILE) +$(BZIP2_STAMPFILE): bzip2-$(XEN_TARGET_ARCH) $(NEWLIB_STAMPFILE) + ( cd $< && \ + CFLAGS="$(TARGET_CPPFLAGS) $(TARGET_CFLAGS)" CC=$(CC) $(MAKE) && \ + $(INSTALL_DATA) libbz2.a $(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/lib && \ + $(INSTALL_DATA) bzlib.h $(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/include && \ + $(INSTALL_DATA) bzlib_private.h $(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/include ) + + +############ +# Cross-lzma +############ + +xz-$(LZMA_VERSION).tar.gz: + $(WGET) $(LZMA_URL)/$@ + +xz-$(XEN_TARGET_ARCH): xz-$(LZMA_VERSION).tar.gz + tar xzf $< + mv xz-$(LZMA_VERSION) $@ + +LZMA_STAMPFILE=$(CROSS_ROOT)/$(GNU_TARGET_ARCH)-xen-elf/lib/liblzma.a +.PHONY: cross-lzma +cross-lzma: $(LZMA_STAMPFILE) +$(LZMA_STAMPFILE): xz-$(XEN_TARGET_ARCH) $(NEWLIB_STAMPFILE) + ( cd $< && \ + CFLAGS="$(TARGET_CPPFLAGS) $(TARGET_CFLAGS)" CC=$(CC) ./configure --prefix=$(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf --enable-shared=no && \ + cd src/liblzma && $(MAKE) && cd ../../ && \ + $(INSTALL_DATA) src/liblzma/.libs/liblzma.a $(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/lib && \ + $(INSTALL_DATA) src/liblzma/api/lzma.h $(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/include && \ + cp -r src/liblzma/api/lzma $(CROSS_PREFIX)/$(GNU_TARGET_ARCH)-xen-elf/include/ ) + + ############## # Cross-libpci ############## @@ -210,7 +263,7 @@ ####### .PHONY: $(CROSS_ROOT) -$(CROSS_ROOT): cross-newlib cross-zlib cross-libpci +$(CROSS_ROOT): cross-newlib cross-zlib cross-bzip2 cross-lzma cross-libpci $(XEN_ROOT)/tools/ioemu-dir: $(MAKE) -C $(XEN_ROOT)/tools ioemu-dir-find @@ -265,15 +318,15 @@ .PHONY: libxc libxc: libxc-$(XEN_TARGET_ARCH)/libxenctrl.a libxc-$(XEN_TARGET_ARCH)/libxenguest.a -libxc-$(XEN_TARGET_ARCH)/libxenctrl.a libxc-$(XEN_TARGET_ARCH)/libxenguest.a:: cross-zlib - CPPFLAGS="$(TARGET_CPPFLAGS)" CFLAGS="$(TARGET_CFLAGS)" $(MAKE) -C libxc-$(XEN_TARGET_ARCH) +libxc-$(XEN_TARGET_ARCH)/libxenctrl.a libxc-$(XEN_TARGET_ARCH)/libxenguest.a:: cross-zlib cross-bzip2 cross-lzma + CPPFLAGS="$(TARGET_CPPFLAGS) -DHAVE_BZLIB -DHAVE_LZMA" CFLAGS="$(TARGET_CFLAGS) -DHAVE_BZLIB -DHAVE_LZMA" $(MAKE) -C libxc-$(XEN_TARGET_ARCH) ####### # ioemu ####### .PHONY: ioemu -ioemu: cross-zlib cross-libpci libxc +ioemu: cross-zlib cross-bzip2 cross-lzma cross-libpci libxc [ -f ioemu/config-host.mak ] || \ ( $(absolutify_xen_root); \ $(buildmakevars2shellvars); \ @@ -339,7 +392,7 @@ .PHONY: pv-grub pv-grub: mini-os-$(XEN_TARGET_ARCH)-grub libxc grub - DEF_CPPFLAGS="$(TARGET_CPPFLAGS)" DEF_CFLAGS="-DCONFIG_GRUB $(TARGET_CFLAGS)" DEF_LDFLAGS="$(TARGET_LDFLAGS)" $(MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a + DEF_CPPFLAGS="$(TARGET_CPPFLAGS) -DHAVE_BZLIB -DHAVE_LZMA" DEF_CFLAGS="-DCONFIG_GRUB $(TARGET_CFLAGS) -DHAVE_BZLIB -DHAVE_LZMA" DEF_LDFLAGS="$(TARGET_LDFLAGS) -lbz2 -llzma" $(MAKE) -C $(MINI_OS) OBJ_DIR=$(CURDIR)/$< APP_OBJS=$(CURDIR)/grub-$(XEN_TARGET_ARCH)/main.a ######### # install --SkvwRMAIpAhPCcCJ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="extras-minios-Makefile-bzip2-lzma-support-v1.patch" --- xen-unstable.hg/extras/mini-os/Makefile.backup.orig 2009-09-01 18:30:06.000000000 +0300 +++ xen-unstable.hg/extras/mini-os/Makefile 2009-09-01 21:43:14.000000000 +0300 @@ -83,6 +83,8 @@ APP_LDLIBS += -lpci APP_LDLIBS += -lz APP_LDLIBS += -lm +APP_LDLIBS += -lbz2 +APP_LDLIBS += -llzma LDLIBS += -lc endif --SkvwRMAIpAhPCcCJ Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --SkvwRMAIpAhPCcCJ--