Skip to contentSkip to conversation

inherit setuptools

DISCUSSION Trevor Woerner 2 participants 4 messages Mar 22, 2014 – Mar 25, 2014

lore
Share & tools
lore
#0 lore raw

Hi,

I'm trying to write a recipe to add a native python application to the sysroot so it can be used while building an image.

If I build and install the python app "by hand" I get the following install layout:

.
|-- bin
| `-- bmaptool
`-- lib

    `-- python2.7
        `-- site-packages
            |-- bmap_tools-3.2-py2.7.egg
            |-- easy-install.pth
            |-- site.py
            `-- site.pyc

When I use my recipe I get:

.
|-- bin
| `-- bmaptool
`-- lib

    `-- python2.7
        `-- site-packages
            |-- bmap_tools-3.2-py2.7.egg-info
            |   |-- PKG-INFO
            |   |-- SOURCES.txt
            |   |-- dependency_links.txt
            |   `-- top_level.txt
            `-- bmaptools
                |-- BmapCopy.py
                |-- BmapCopy.pyc
                |-- BmapCreate.py
                |-- BmapCreate.pyc
                |-- BmapHelpers.py
                |-- BmapHelpers.pyc
                |-- Filemap.py
                |-- Filemap.pyc
                |-- TransRead.py
                |-- TransRead.pyc
                |-- __init__.py
                `-- __init__.pyc

When my build tries to invoke the bmaptool it fails because:

| Traceback (most recent call last):
| File
"/SSD/build/fsl/build-wandboard-dual/tmp/sysroots/x86_64-linux/usr/bin/bmaptool", line 65, in <module>
| from bmaptools import BmapCreate, BmapCopy, BmapHelpers, TransRead
| ImportError: No module named bmaptools

Should I ignore "setuptools" and write my own install task? Or is there some quick and easy fix for this?

#1 lore raw
Quoted 6 lines · Trevor Woerner

| Traceback (most recent call last):
| File
"/SSD/build/fsl/build-wandboard-dual/tmp/sysroots/x86_64-linux/usr/bin/bmaptool", line 65, in <module>
| from bmaptools import BmapCreate, BmapCopy, BmapHelpers, TransRead
| ImportError: No module named bmaptools

That's interesting because as can be seen in your listing, there *is* a module called bmaptools. Does bmap do anything "clever" (read:
stupid) before it does that import?

And FWIW, I didn't get much out of bmap when I tried it because of how our images were constructed: all of the sparse segments disappeared when a whole-disk image was created. I got a much better win by using USB 3: around 10 seconds to write a gigabyte+ image to a stick...

Ross

#2 lore raw
Quoted 10 lines · Ross
Quoted 6 lines · Trevor Woerner

| Traceback (most recent call last):
| File
"/SSD/build/fsl/build-wandboard-dual/tmp/sysroots/x86_64-linux/usr/bin/bmaptool", line 65, in <module>
| from bmaptools import BmapCreate, BmapCopy, BmapHelpers, TransRead
| ImportError: No module named bmaptools

That's interesting because as can be seen in your listing, there *is* a module called bmaptools. Does bmap do anything "clever" (read:
stupid) before it does that import?

Oh wow, you're right. The "bmaptools" python script which is installed "by-hand" simply does:

    #!/usr/bin/python
    # EASY-INSTALL-SCRIPT: 'bmap-tools==3.2','bmaptool'
    __requires__ = 'bmap-tools==3.2'
    import pkg_resources
    pkg_resources.run_script('bmap-tools==3.2', 'bmaptool')

Whereas the one installed by using OE looks more like a regular python script. It's starts with:

    #!/usr/bin/env python
    [...snip...]
    VERSION = "3.2"

    import argparse
    import sys
    import os
    import stat
    import time
    import logging
    import tempfile
    import traceback
    import shutil
    import io
    from bmaptools import BmapCreate, BmapCopy, BmapHelpers, TransRead

    class NamedFile(object):
    [...snip...]

And if my understanding of "inherit setuptools" is correct, this class explicitly circumvents the use of easy-install. (?)

Quoted 4 lines

And FWIW, I didn't get much out of bmap when I tried it because of how our images were constructed: all of the sparse segments disappeared when a whole-disk image was created. I got a much better win by using USB 3: around 10 seconds to write a gigabyte+ image to a stick...

Interesting.

By default an image created by OE will be just slightly larger than what is required. So a core-image-minimal will be just ~80MB, most of the empty space is removed, and the image can be dd'ed onto a microSD card quickly. But unless I partition the device (for example, 4GB microSD card) myself, the resulting image will only see ~80MB of disk space. When I force the build to generate a 4GB image (to load into a 4GB microSD card and have the remaining space available) using dd takes almost 50 minutes. Using bmap to load the same image onto the same 4GB card takes under 15 minutes.

So yes, there's no win if you're just burning the image that comes out of the build, because these images only contain the useful parts. But the images which come out of the build only create a partition that is slightly larger than the used space, leaving the rest of the device unusable (unless you partition it yourself).

Or is there some trick I'm missing?

#3 lore raw
In reply to Burton, Ross

Hi Ross,

Quoted 3 lines · Ross

And FWIW, I didn't get much out of bmap when I tried it because of how our images were constructed: all of the sparse segments disappeared when a whole-disk image was created.

I have expanded a little bit on this issue here:
http://twoerner.blogspot.ca/2014/03/using-bmaptool-to-create-memory-card.html

and I see better what you mean about sparse segments. Using bmaptool, theoretically, the same core-image-minimal should take the same ~20 seconds to write whether the image is 80MB or 4GB. The fact these are significantly different suggests there might be room for improvement in the way our images are created?

Still, I was able to go from 50 minutes to 14 when writing the 4GB image using bmaptool versus using dd, so that's an improvement none the less.

Quoted 2 lines

I got a much better win by using USB 3: around 10 seconds to write a gigabyte+ image to a stick... Ross

Thanks for the tip!