Skip to contentSkip to conversation

FatELF patches...

DISCUSSION Eric Windisch 3 participants 5 messages Nov 3, 2009 – Nov 10, 2009

lore
Share & tools
lore
#0 lore raw

First, I apologize if this message gets top-posted or otherwise improperly threaded, as I'm not currently a subscriber to the list (I can no longer handle the daily traffic). I politely ask that I be CC'ed on any replies.

In response to Alan's request for a FatELF use-case, I'll submit two of my own.

I have customers which operate low-memory x86 virtual machine instances. Until recently, these ran with as little as 64MB of RAM. Many customers have chosen 32-bit distributions for these systems, but would like the flexibility of scaling beyond 4GB of memory. These customers would like the choice of migrating to 64-bit without having to reinstall their distribution.

Furthermore, I'm involved in several "cloud computing" initiatives, including interoperability efforts. There has been discussion of assuring portability of virtual machine images across varying infrastructure services. I could see how FatELF could be part of a solution to this problem, enabling a single image to function against host services running a variety of architectures.

As for negatives: I'm running ZFS which now supports deduplication, so this might potentially eliminate my own concerns in regard to storage. Eventually, Btrfs will provide this capability under Linux directly. The networking isn't much of an issue either, as I have my own mirrors for the popular distributions. While this isn't the typical end-user environment, it might be a typical environment for companies facing the unique problems FatELF solves.

I concede that there are a number of ways that solutions to these problems might be implemented, and FatELF binaries might not be the optimal solution. Regardless, I do feel that use cases do exist, even if there are questions and concerns about the implementation.

Signature · 2 lines
Regards,
Eric Windisch
#1 lore raw
Quoted 2 lines · Eric Windisch

First, I apologize if this message gets top-posted or otherwise improperly threaded, as I'm not currently a subscriber to the list (I

Given proper references:-headers, the mail should have threaded properly.

Quoted 2 lines

can no longer handle the daily traffic). I politely ask that I be CC'ed on any replies.

Which raises the question why you didn't cc: anyone in the first place.

Quoted 4 lines

In response to Alan's request for a FatELF use-case, I'll submit two of my own.

I have customers which operate low-memory x86 virtual machine instances.

Low resource environments (be it embedded or not) are probably the last who wants (or even can handle) such "bloat by design".
The question in that world is not "how can I make it run on more architectures" but "how can I get rid of run-time code as soon as possible".

Quoted 5 lines

Until recently, these ran with as little as 64MB of RAM. Many customers have chosen 32-bit distributions for these systems, but would like the flexibility of scaling beyond 4GB of memory. These customers would like the choice of migrating to 64-bit without having to reinstall their distribution.

Just install a 64bit kernel (and leave the user-space intact). A 64bit kernel can run 32bit binaries.

Quoted 2 lines

Furthermore, I'm involved in several "cloud computing" initiatives, including interoperability efforts. There has been discussion of

The better solution is probably to agree on pseudo-machine-code (like e.g. JVM, parrot, or whatever) with good interpreters/JIT-compilers which focus more on security and how to validate potentially hostile programs than anything else.

Quoted 4 lines

assuring portability of virtual machine images across varying infrastructure services. I could see how FatELF could be part of a solution to this problem, enabling a single image to function against host services running a variety of architectures.

Let's hope that the n versions in a given FatElf image actually are instances of the same source.

[....]

Quoted 4 lines

I concede that there are a number of ways that solutions to these problems might be implemented, and FatELF binaries might not be the optimal solution. Regardless, I do feel that use cases do exist, even if there are questions and concerns about the implementation.

The obvious drawbacks are:
- Even if disk space is cheap, the vast amount is a problem for mirroring that stuff.
- Fat-Binaries (ab)use more Internet bandwidth. Hell, Fedora/RedHat got delta-RPMS working (just?) for this reason.
- Fat-Binaries (ab)use much more memory and I/O bandwidth - loading code for n architectures and throw n-1 of it away doesn't sound very sound.
- Compiling+linking for n architectures needs n-1 cross-compilers installed and working.
- Compiling+linking for n architectures needs much more *time* than for 1 (n times or so).
Guess what people/developers did first on the old NeXT machines: They disable the default "build for all architectures" as it speeded things up.
Even if the expected development setup is "build for local only", at least packagers and regression testers won't have the luxury of that.

The only remotely useful benefit in the long run I can imagine is: The permanent cross-compiling will make AC_TRY_RUN() go away. Or at least the alternatives are applicable without reading the generated configure.sh (and config.log) to guess how to tell the script some details.
But that isn't really worth it - as we are living without it for long.

Bernd

Signature · 3 lines
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services
#2 lore raw
Quoted 5 lines · Bernd Petrovitsch

The only remotely useful benefit in the long run I can imagine is: The permanent cross-compiling will make AC_TRY_RUN() go away. Or at least the alternatives are applicable without reading the generated configure.sh (and config.log) to guess how to tell the script some details.

hmm, that could be the real killer argument - evolutionarily sort out the guys who're too dumb to write proper buildscripts ;-)

cu

Signature · 8 lines
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------
#3 lore raw
Quoted 10 lines · Enrico Weigelt
Quoted 5 lines · Bernd Petrovitsch

The only remotely useful benefit in the long run I can imagine is: The permanent cross-compiling will make AC_TRY_RUN() go away. Or at least the alternatives are applicable without reading the generated configure.sh (and config.log) to guess how to tell the script some details.

hmm, that could be the real killer argument - evolutionarily sort out the guys who're too dumb to write proper buildscripts ;-)

Obviously your irony detector triggered;-)

Bernd

Signature · 3 lines
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services
#4 lore raw
In reply to Eric Windisch

* Eric Windisch <[email protected]> wrote:

Hi,

Quoted 6 lines

I have customers which operate low-memory x86 virtual machine instances. Until recently, these ran with as little as 64MB of RAM. Many customers have chosen 32-bit distributions for these systems, but would like the flexibility of scaling beyond 4GB of memory. These customers would like the choice of migrating to 64-bit without having to reinstall their distribution.

Assuming that are reasonably critical production systems, you wont get around a specially tailored distro/package manager (where anybody already did all the vast amount of testing of the upgrade process) or do it all manually. Nevertheless you'll (at least temporarily) need an multilib system or jails.

I don't see where FatELF will give you special help here.

Quoted 6 lines

Furthermore, I'm involved in several "cloud computing" initiatives, including interoperability efforts. There has been discussion of assuring portability of virtual machine images across varying infrastructure services. I could see how FatELF could be part of a solution to this problem, enabling a single image to function against host services running a variety of architectures.

Drop that idea. Better create images for each target platform. Let an automated build system handle that. (if you need one, feel free to contact me off-list).

You want to migrate a running VM to a different arch ? Forget it. You won't come around processor emulation, better use some VP like Java, LLVM, etc.

cu

Signature · 8 lines
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------