RTLD_DEEPBIND and uClibc

Hello everyone

Yesterday evening I managed to compile cairo-dock-core and cairo-dock-plugins
version 3.1.2 for buildroot 2012.08.

It's a nice, lightweight (made with C), Mac OSX feeling application dock.

The problem is that cairo-dock-core has the following line in it's code:

pCairoDockModule->handle = dlopen (pCairoDockModule->cSoFilePath, RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);

And uClibc does not have RTLD_DEEPBIND. Removing it will allow cairo-dock to compile but the result is
that cairo-dock plugins won't work.

Is there any workaround to this RTLD_DEEPBIND stuff ?

Regards
Stefan

Dear Stefan Fr?berg,

Strange, from a quick look, it seems that it is used to get a single symbol, called "pre_init" from each plugin. I don't see how pre_init may conflict with some other global symbol, which is what RTLD_DEEPBIND is all about. But I've only done a 5 minutes look at cairo-dock code, so I might have missed the point.

Since you're using a big stuff like X.org, why do you persist in using uClibc? Using glibc or eglibc would make a very small size difference, and would avoid the hassle of all those small, but annoying, uClibc limitations.

Best regards,

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Hi Thomas

17.1.2013 1:00, Thomas Petazzoni kirjoitti:

Well, it's not *that* big.

I mean, yeah, I have Xorg but besides that I only have Fluxbox as my environment
and I just wanted to have much more feature richer & "cooler" application launcher than what
Fluxbox offers (that is, none).

As a matter of fact, my biggest uClibc-compiled application is not Xorg but firefox 18.0 (patch coming soon)

And IMHO those limitations are not really uClibc's fault at all.

It's because glibc offers stuff like (execinfo.h, mcheck.h, RTLD_DEEPBIND etc...) which are not listed in any standard (POSIX or otherwise)
and which people happily use in their own coding projects while being unaware that they are not standard comforming
and hence not easily portable.
Im sure you noticed this when making that elfutils package Thomas. I had as hellish experience as you did when doing
the initial build of that thing. All because the author of that software happened to be Ulrich Drepped, the father of glibc,
so no wonder that it's code was littered with (non-standard) glibc-stuff.

Also, it's a good learning experience for me while porting stuff to uClibc

Regards
Stefan

P.S:

I found out that they started using RTLD_DEEPBIND in cairo-dock-core 2.4.0 version.
So I will try 2.3.0 and report my findings.

Dear Stefan Fr?berg,

It's not that big, but compared it to the size difference between uClibc and glibc, and you'll see that switching to glibc is reasonable.

My perception is that uClibc is very relevant for non-MMU systems or small systems (having limited quantity of flash/RAM), and on those systems, we are not going to run a full-blown X.org stack with a window manager and so on.

Whenever you start having X.org on a system and a window manager, most likely you have hundreds of megabytes of flash, and therefore, the size difference between uClibc and glibc (probably somewhere between 1 and 1.5 MB) isn't that much of a problem anymore.

Indeed. And I agree that fixing those compatibility problems in elfutils is important, because using perf on low-end systems is useful. However, getting cairo-dock to work with uClibc seems less important to me, but...

... of course, for learning purposes, it's always useful!

Best regards,

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Hi Thomas

17.1.2013 10:31, Thomas Petazzoni kirjoitti:

Eh... I think it's more than 1 - 1.5 MB :-)

http://www.etalabs.net/compare_libcs.html

In that comparison it's eglibc (the smaller brother of glibc) vs. other, more lighter c-libraries.
And even then it's 1.5 MB or more vs uClibc's 336 KB with .a files and 2 MB or more vs. 520 KB.

Even tought in my case, Im targeting to running my system with low-end, ancient PC desktop's (2000 or earlier),
so it's not really exactly comparable with todays embedded systems, it's still a huge saving of space when
you start thinking about all the lib's and binaries that go to normal desktop system.

In a related note, my *full* Live-CD system is now uncompressed to 520 MB (194 MB, if using squashfs xz-compressed image with loopdevice), and the absolute minimum runtime RAM requirement for starting it (Xorg,fluxbox, firefox, etc...) is
now about 120 MB (bigger part of that goes to tmp files and other stuff that are automatically created in initramfs from which it runs directly).

I don't think I could have done that with glibc or eglibc ...

And I haven't even started using GCC 4.7.2 Linktime optimizations yet. which I expect to cut about 10 - 20 KB per binary away
and finally UPX compression of selected few binaries and libs.

And if even *that* is not enought then I could consider maybe switching to musl ... maybe ...

So all in all, Im very happy with buildroot and uClibc and not going to turn back to glibc and bloat

Regards
Stefan

I've been following musl for a while - it really might be worth giving a go. They're aiming to be more lightweight than Glibc, while more functional than uclibc. Plus some musl patches for buildroot and crosstool-ng would be kind of awesome ;)

Alex

Hello Alex!

17.1.2013 18:18, Alex Bradbury kirjoitti:

Is it true that musl supports (at least most) of the glibc ABI ? (my biggest worry so far when using uClibc is that I encounter an application that is provided as glibc-only
binary and no source code provided.)

What about it's configuration ? Does it have nice kconf style "make menuconfig" like uClibc or what ?

Stefan

Yes, this is one of the goals. There was a big discussion on the musl mailing list <http://dir.gmane.org/gmane.linux.lib.musl.general> in December. One of the aims is glibc ABI compatibility, with applications like Opera being discussed on the list. Development is *very* active and Rich Felker (lead dev) seems to respond very quickly to issues or bug reports.

In fact, I've fallen behind following the musl mailing list the last couple of months but see a recent posts that indicate there has been good glibc ABI progress
<http://article.gmane.org/gmane.linux.lib.musl.general/2549>. Of course many of the proprietary apps are likely to use C++, and the C++ ABI is...fiddly.

I don't *think* so, but could be wrong. Still, it aims to be substantially lighterweight than glibc, while hopefully providing more performant implementations of functions than either glibc or uclibc.

Alex

17.1.2013 19:09, Alex Bradbury kirjoitti:

Thanks Alex!
I will keep watching that list.

Stefan

Alex> On 17 January 2013 13:50, Stefan Fr?berg

Alex> I've been following musl for a while - it really might be worth giving Alex> a go. They're aiming to be more lightweight than Glibc, while more Alex> functional than uclibc. Plus some musl patches for buildroot and Alex> crosstool-ng would be kind of awesome ;)

I would certainly be open for patches adding musl support in buildroot.

Bye, Peter Korsgaard

Stefan, All,

The full set of libraries from a full-blown eglibc, for an ARM gnueabihf target, is:
Unstripped Stripped

Only eglibc libs      3.6MiB         2.3MiB
With gcc's libs       8.8MiB         3.0MiB

And for a relatively complete uClibc for x86_64 target:
Unstripped Stripped

Only eglibc libs      2.0MiB         1.0MiB
With gcc's libs       6.9MiB         1.9MiB

(gcc libs: libstdc++ and libgcc_s)
(These are not the same architecture, but I went for the toolchains I have locally)

So, typically, speaking only about the C library libs, the gain would be around ~1.5MiB.

How large is your Xorg stack?
Then, compare that 1.5MiB to the size of your Xorg stack. What's the ratio?

If you're 1.5MiB short of space, then probably Xorg is not the solution for your platform.

Of course, switching to uClibc as a learning experience is very intreresting. But no more. ;-)

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Hi Yann

17.1.2013 19:44, Yann E. MORIN kirjoitti:

Here's the contents of my target /lib

*yhteens? 1,4M*
drwxr-xr-x. 6 root root 3,0K 17.1. 15:32 . drwxr-xr-x. 20 root root 1,0K 13.1. 17:35 .. drwxr-xr-x. 64 root root 10K 17.1. 15:31 firmware -rwxr-xr-x. 1 root root 29K 17.1. 15:32 ld-uClibc-0.9.33.2.so
lrwxrwxrwx. 1 root root 21 7.1. 00:27 ld-uClibc.so.0 ->
ld-uClibc-0.9.33.2.so

lrwxrwxrwx.  1 root root   17  8.1. 20:20 libblkid.so.1 -> libblkid.so.1.1.0
-rwxr-xr-x.  1 root root 152K 17.1. 15:31 libblkid.so.1.1.0
-rwxr-xr-x.  1 root root  25K 17.1. 15:32 libcrypt-0.9.33.2.so
lrwxrwxrwx.  1 root root   20  7.1. 00:27 libcrypt.so.0 ->
libcrypt-0.9.33.2.so
lrwxrwxrwx.  1 root root   21  7.1. 00:27 libc.so.0 -> libuClibc-0.9.33.2.so
-rwxr-xr-x.  1 root root  13K 17.1. 15:32 libdl-0.9.33.2.so
lrwxrwxrwx.  1 root root   17  7.1. 00:27 libdl.so.0 -> libdl-0.9.33.2.so
lrwxrwxrwx.  1 root root   13  7.1. 00:44 libgcc_s.so -> libgcc_s.so.1
-rw-r--r--.  1 root root  89K 17.1. 15:32 libgcc_s.so.1
-rwxr-xr-x.  1 root root  49K 17.1. 15:31 libm-0.9.33.2.so
lrwxrwxrwx.  1 root root   17  8.1. 20:20 libmount.so.1 -> libmount.so.1.1.0
-rwxr-xr-x.  1 root root 152K 17.1. 15:31 libmount.so.1.1.0
lrwxrwxrwx.  1 root root   16  7.1. 00:27 libm.so.0 -> libm-0.9.33.2.so
-rwxr-xr-x.  1 root root 4,5K 17.1. 15:32 libnsl-0.9.33.2.so
lrwxrwxrwx.  1 root root   18  7.1. 00:27 libnsl.so.0 -> libnsl-0.9.33.2.so
lrwxrwxrwx.  1 root root   17  7.1. 20:47 libpamc.so -> libpamc.so.0.82.1
lrwxrwxrwx.  1 root root   17  7.1. 20:47 libpamc.so.0 -> libpamc.so.0.82.1
-rwxr-xr-x.  1 root root 9,6K 17.1. 15:32 libpamc.so.0.82.1
lrwxrwxrwx.  1 root root   21  7.1. 20:47 libpam_misc.so ->
libpam_misc.so.0.82.0
lrwxrwxrwx.  1 root root   21  7.1. 20:47 libpam_misc.so.0 ->
libpam_misc.so.0.82.0
-rwxr-xr-x.  1 root root 8,1K 17.1. 15:32 libpam_misc.so.0.82.0
lrwxrwxrwx.  1 root root   16  7.1. 20:47 libpam.so -> libpam.so.0.83.1
lrwxrwxrwx.  1 root root   16  7.1. 20:47 libpam.so.0 -> libpam.so.0.83.1
-rwxr-xr-x.  1 root root  45K 17.1. 15:31 libpam.so.0.83.1
-rwxr-xr-x.  1 root root  74K 17.1. 15:31 libpthread-0.9.33.2.so
lrwxrwxrwx.  1 root root   22  7.1. 00:27 libpthread.so.0 ->

libpthread-0.9.33.2.so
-rwxr-xr-x. 1 root root 4,5K 17.1. 15:32 libresolv-0.9.33.2.so
lrwxrwxrwx. 1 root root 21 7.1. 00:27 libresolv.so.0 ->
libresolv-0.9.33.2.so

-rwxr-xr-x.  1 root root  13K 17.1. 15:31 librt-0.9.33.2.so
lrwxrwxrwx.  1 root root   17  7.1. 00:27 librt.so.0 -> librt-0.9.33.2.so
-rwxr-xr-x.  1 root root 690K 17.1. 15:31 libuClibc-0.9.33.2.so
-rwxr-xr-x.  1 root root 4,7K 17.1. 15:31 libutil-0.9.33.2.so
lrwxrwxrwx.  1 root root   19  7.1. 00:27 libutil.so.0 ->
libutil-0.9.33.2.so
lrwxrwxrwx.  1 root root   16  8.1. 20:20 libuuid.so.1 -> libuuid.so.1.3.0
-rwxr-xr-x.  1 root root  14K 17.1. 15:31 libuuid.so.1.3.0
drwxr-xr-x.  3 root root 1,0K  9.1. 04:01 modules
drwxr-xr-x.  3 root root 5,0K 17.1. 15:31 security
drwxr-xr-x.  3 root root 1,0K  8.1. 22:41 udev

As you can see from the top of the line the total sum of uClibc + *all the other stuff (pam etc..)* is just
1.4 MB (the size of libuClibc-0.9.33.2.so itself is just 690 KB)

So uClibc wins this round :-)

You mean how much Xorg stuff from buildroot I have installed into my target system???
Short answer: Full
Well, almost, left only xterm (replaced with rxvt-unicode which lighter), twm (replaced with fluxbox)
and xeyes (totally useless application) out.

BTW, I forgot to mention in my message to Thomas that that 520 MB uncompressed Live-CD stuff does
also contain the full linux 3.3.1 kernel and *all* the drivers compiled as modules
(Live-CD without that would be useless when running it in myriad x86 machines with allkinds of hardware)

Regards
Stefan

Stefan, All,

[--SNIP--]

uClibc == libuClibc.so + libutil.so + ld-uClibc.so + librt.so + libnsl.so
+ libm.so + libdl.so + libcrypt.so

Which totals around 905KiB in your case. Just about the number I gave you in my previous mail (mine was exactly 1004KiB). And the eglibc numbers are for the same set of libraries (plus all the libnss libs uClibc does not have).

Yes, in absolute numbers, uClibc *is* smaller than (e)glibc. We *never* said it was bigger.

What Thomas and I are saying, is that you gained about 1.5MiB on your 520MiB filesystem. That's about a 0.29% gain. Was that worth the hassle? (Beside learning, of course.)

At one point, one must must leave the keyboard alone, cross one's arms, look deeply at the screen, and just ask oneself: "Well, where am I heading, now?"

- if the point is to prove how smaller a complete system can be by using uClibc instead of (e)glibc, then the answer is "marginaly smaller", and "no, I should not bother suferring like in Hell"

- if the point is to prove that building a complete system using uClibc instead of (e)glibc, then the answer is "doable, with some gotchas in the corners", and "no, I should not suffer like in Hell"

- if the point is to learn, then yes, go on, learn; this is of course a very valid and respectful reason. But, still, the answer is "Whoa, I learned quite a bit!" and "no, I should not bother suferring like in Hell".

Well, 0.29% smaller... Damn! That's some kind of an achievment! ;-)

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Stefan, All,

Forgot: + libresolv.so + libpthread.so (still 905KiB already accounted for those libs)

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

17.1.2013 22:38, Yann E. MORIN kirjoitti:

My gentoo box with similar configuration is 1500 MB. So yeah I think it was worth the hassle :-)

Regards
Stefan

Stefan, All,

No, you're not comparing comparable things.

Of course, I can tell you that my Debian Squeeze is much bigger than my busybox-based router, but that's not the same thing.

Rebuild your LiveCD with (e)glibc, and compare the sizes. Then you will notice that the size with eglibc is marginaly bigger than with uClibc. *That* is what we are saying.

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

17.1.2013 23:17, Yann E. MORIN kirjoitti:

Actually in my case it is. The only difference between my gentoo box and buildroot version
is that gentoo uses bash and coreutils instead busybox.

And why I would want to make my system fatter again ? Hmmmh ??? Marginal or not

No, I will take my system as far as I can go with all the optimizations I still have left and
then maybe start considering musl (Thanks Alex)

Regards
Stefan