Skip to contentSkip to conversation

icedtea6-native: task compile fails with `unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or …`

DISCUSSION Paul Menzel 2 participants 7 messages May 22, 2011 – May 25, 2011

lore
Share & tools
lore
#0 lore raw

Dear OE folks,

even with the latest patches send to the list [1].

Importing binaries from component CORBA_DIST
( cd /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
Archive: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
note: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period. make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9

Running the command directly works fine though.

        $ /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
        Archive:  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
          inflating: lib/orb.idl             
          inflating: lib/ir.idl

Thanks,

Paul

[1] http://patches.openembedded.org/patch/4621/

#1 lore raw

Hi,
I'll look into this.

Regards,
Robert

Am 22.05.2011 23:24, schrieb Paul Menzel:

Quoted 37 lines

Dear OE folks,

even with the latest patches send to the list [1].

Importing binaries from component CORBA_DIST
( cd /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
Archive: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
note: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period. make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9

Running the command directly works fine though.

        $ /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
        Archive:  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
          inflating: lib/orb.idl             
          inflating: lib/ir.idl

Thanks,

Paul

[1] http://patches.openembedded.org/patch/4621/

_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

#2 lore raw
In reply to Paul Menzel

Hi,

Quoted 10 lines

Importing binaries from component CORBA_DIST
( cd /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
Archive: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
note: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period. make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9

Did you notice that somehow the makefile tries to unpack the file '/oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip'. I smell some shell-quoting problem or similar.

Regards,
Robert

#3 lore raw

Dear Robert,

Quoted 10 lines · Robert Schuster
Quoted 10 lines

Importing binaries from component CORBA_DIST
( cd /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
Archive: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
note: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period. make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9

Quoted 2 lines

Did you notice that somehow the makefile tries to unpack the file '/oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip'.

It looks strange. But I thought that is some behavior of unzip to look into some default paths.

Quoted 1 line

I smell some shell-quoting problem or similar.

I found the following parts but do not know how this can fail.

        /oe/build-minimal-eglibc/minimal-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0$ grep -Ri "Importing binaries" *
        icedtea6-1.7.10/openjdk/jdk/make/common/internal/ImportComponents.gmk:  $(ECHO) "Importing binaries from component $1"; \
        icedtea6-1.7.10/openjdk-ecj/jdk/make/common/internal/ImportComponents.gmk:  $(ECHO) "Importing binaries from component $1"; \
        openjdk-src-dir/jdk/make/common/internal/ImportComponents.gmk:  $(ECHO) "Importing binaries from component $1"; \
        temp/log.do_compile:Importing binaries from component CORBA_DIST
        temp/log.do_compile.22501:Importing binaries from component CORBA_DIST

looking at there is

        # Importing optional component bin files (for install image)
        define import-one-binaries
        @if [ "$($1)" != "" -a -f $($1)/$(BIN_ZIP_FILE) ] ; then \
          $(ECHO) "Importing binaries from component $1"; \
          $(call Unzipper,$2,$($1)/$(BIN_ZIP_FILE)); \
        fi
        endef

and the function/method Unzipper is defined as follows.

        # Unzip zip file $2 into directory $1 (if $2 exists)
        #   Warning: $2 must be absolute path not relative
        define Unzipper
        ( \
          $(MKDIR) -p $1; \
          $(ECHO) "( $(CD) $1 && $(UNZIP) -o $2 )"; \
          ( $(CD) $1 && $(UNZIP) -o $2 ) \
        )
        endef

I do not see where this fails. By the way, to use the native unzip I have the following in my recipe (as send to the list [2]).

        EXTRA_OEMAKE = ' \
                        ZIPEXE="${STAGING_BINDIR_NATIVE}/zip" \
                        UNZIP="${STAGING_BINDIR_NATIVE}/unzip" \
                       '

Thanks,

Paul

[1] icedtea6-1.7.10/openjdk/jdk/make/common/internal/ImportComponents.gmk
[2] http://patches.openembedded.org/patch/4621/

#4 lore raw

Hi Paul,

Quoted 11 lines
        # Unzip zip file $2 into directory $1 (if $2 exists)
        #   Warning: $2 must be absolute path not relative
        define Unzipper
        ( \
          $(MKDIR) -p $1; \
          $(ECHO) "( $(CD) $1 && $(UNZIP) -o $2 )"; \
          ( $(CD) $1 && $(UNZIP) -o $2 ) \
        )
        endef

I do not see where this fails. By the way, to use the native unzip I have the following in my recipe (as send to the list [2]).

Wow, this is really weird! There is a file openjdk-ecj/jdk/make/common/shared/Defs-utils.gmk containing all the tool variable definitions - including UNZIP. So I removed the EXTRA_OEMAKE part for UNZIP and added the path to our native unzip executable directly into Defs-utils.gmk and guess what. That works!

This is really weird because the 'Unzipper' command actually echos its arguments and there $2 is printed perfectly. But when it is accessed a second time in the actual call it is wrong.

/me puzzled ....

Regards,
Robert

#5 lore raw
In reply to Paul Menzel

Hi Paul,
greetings from the GDB cave ....

I now what is going wrong with unzip and ... whoa this is so weird ...

Am 22.05.2011 23:24, schrieb Paul Menzel:

Quoted 11 lines

Importing binaries from component CORBA_DIST
( cd /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
Archive: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
note: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period. make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9

Quoted 6 lines

Running the command directly works fine though.

        $ /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
        Archive:  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
          inflating: lib/orb.idl             
          inflating: lib/ir.idl

The thing is. You are tweaking the IcedTea build by declaring the makefile variable UNZIP through the command-line. For programs that are called from this make process the variables turn into environment variables ...
And this is where the shit hits the fan: As it turns out Info-Zip's unzip understand an arcane way of declaring the file to be unzipped. Through an environment variable called: UNZIP (!!!)

To find this out I did nothing less than running an OE-called unzip and a manually called unzip process run under gdb and compare what they do in the beginning step by step.

Here a shorter explanation of what I mean:

UNZIP=xyz /home/rschus-oe/panda/tmp/sysroots/x86_64-linux/usr/bin/unzip blablafoo
unzip: cannot find or open xyz, xyz.zip or xyz.ZIP.

So apparently UNZIP is in our case set to the zip file executable already to please the openjdk build.

I don't have an immediate idea how to fix it properly. I'd vote for patching our unzip not to use that variable (It still has another one called UNZIP_OPTS for the same use).

Last but not least: Our unzip recipe sucks. It does not enforce OE build flags and does not prevent that the called makefile strips the binary. There are also newer version of unzip out there (ours is from 2005). I'll plan on fixing all of the above.

Regards,
Robert

#6 lore raw

Dear Robert,

Am Mittwoch, den 25.05.2011, 20:00 +0200 schrieb Robert Schuster:

Quoted 30 lines

greetings from the GDB cave ....

I now what is going wrong with unzip and ... whoa this is so weird ...

Am 22.05.2011 23:24, schrieb Paul Menzel:

Quoted 11 lines

Importing binaries from component CORBA_DIST
( cd /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86 && /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip )
Archive: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
note: /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip may be a plain executable, not an archive
unzip: cannot find zipfile directory in one of /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip or /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.zip, and cannot find /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip.ZIP, period. make[4]: *** [/oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/tmp/java/components_imported] Error 9

Quoted 6 lines

Running the command directly works fine though.

        $ /oe/build-angstrom-next/angstrom-dev/sysroots/i686-linux/usr/bin/unzip -o /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
        Archive:  /oe/build-angstrom-next/angstrom-dev/work/i686-linux/icedtea6-native-1.7.10-r4.0/icedtea6-1.7.10/openjdk-ecj/build/linux-x86/corba/dist/lib/bin.zip
          inflating: lib/orb.idl             
          inflating: lib/ir.idl

The thing is. You are tweaking the IcedTea build by declaring the makefile variable UNZIP through the command-line. For programs that are called from this make process the variables turn into environment variables ...
And this is where the shit hits the fan: As it turns out Info-Zip's unzip understand an arcane way of declaring the file to be unzipped. Through an environment variable called: UNZIP (!!!)

Quoted 3 lines

To find this out I did nothing less than running an OE-called unzip and a manually called unzip process run under gdb and compare what they do in the beginning step by step.

Great analysis. I could not have done it and only used gdb to create a backtrace for the developers.

Quoted 6 lines

Here a shorter explanation of what I mean:

UNZIP=xyz /home/rschus-oe/panda/tmp/sysroots/x86_64-linux/usr/bin/unzip blablafoo
unzip: cannot find or open xyz, xyz.zip or xyz.ZIP.

Quoted 6 lines

So apparently UNZIP is in our case set to the zip file executable already to please the openjdk build.

I don't have an immediate idea how to fix it properly. I'd vote for patching our unzip not to use that variable (It still has another one called UNZIP_OPTS for the same use).

Well as far as I understand the manual, UNZIP_OPTS should be used for VMS, but could also be used for other operating systems.

Actually I would think to solve this, by patching OpenJDK to use another variable and report that upstream. I suggest for example UNZIP_EXE for unzip executable.

I do not know how if the unzip developers have decided on this issue to rename remove that variable and how many programs use this feature.

Quoted 4 lines

Last but not least: Our unzip recipe sucks. It does not enforce OE build flags and does not prevent that the called makefile strips the binary. There are also newer version of unzip out there (ours is from 2005). I'll plan on fixing all of the above.

That would be awesome.

I am sorry for opening Pandora’s box.

Thanks,

Paul