Skip to contentSkip to conversation

klibc - another libc?

DISCUSSION Michael Tokarev 5 participants 16 messages Jun 7, 2006 – Jun 13, 2006

lore
Share & tools
lore
#0 lore raw

After several mentions of klibc recently, I want to ask a question.

I understand all the kernel-mode cleanups -- moving initialization from kernel to user space is a very good thing.

But the question really is: why yet another libc? We already have dietlibc, uclibc, glibc, now klibc... With modern kernel, initramfs will very probably contain quite some programs linked with glibc (modprobe/insmod, mdadm/lvm, etc; I highly suggest putting some minimal text editor like nvi there too, for rescue purposes) -- so why not have an option to use whatever libc is available on the host platform?

In the other words, kinit/ipconfig/nfsmount/etc stuff is ok, no questions. But the libc itself -- what for?

And another related question: why not dietlibc which is already here, for quite long time?

Thanks.

/mjt

#1 lore raw
Followup to:  <[email protected]>
By author:    Michael Tokarev <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 6 lines

After several mentions of klibc recently, I want to ask a question.

I understand all the kernel-mode cleanups -- moving initialization from kernel to user space is a very good thing.

Quoted 8 lines

But the question really is: why yet another libc? We already have dietlibc, uclibc, glibc, now klibc... With modern kernel, initramfs will very probably contain quite some programs linked with glibc (modprobe/insmod, mdadm/lvm, etc; I highly suggest putting some minimal text editor like nvi there too, for rescue purposes) -- so why not have an option to use whatever libc is available on the host platform?

You have that option just fine; if you build your own initramfs you can do whatever you want.

Quoted 2 lines

In the other words, kinit/ipconfig/nfsmount/etc stuff is ok, no questions. But the libc itself -- what for?

To be able to *require* it, which means it can't significantly bloat the total size of the kernel image. klibc binaries are *extremely* small. Static kinit is only a few tens of kilobytes, a lot of which is zlib.

Quoted 2 lines

And another related question: why not dietlibc which is already here, for quite long time?

- Bigger by an order of magnitude
- License issues
- Platform support
- Speed of portability (klibc is fully portable to a new platform in an afternoon)
- Additional issues which you can find if look through the archives of this list

-hpa

#2 lore raw

Hi,

Quoted 4 lines · H. Peter Anvin

To be able to *require* it, which means it can't significantly bloat the total size of the kernel image. klibc binaries are *extremely* small. Static kinit is only a few tens of kilobytes, a lot of which is zlib.

Every project starts small and has the annoying tendency to grow. That still doesn't answer, why it has to be distributed with the kernel, just install the thing somewhere under /lib and Kbuild can link to it. The point is that it contains nothing kernel specific and doesn't has to be rebult with every new kernel.

bye, Roman

#3 lore raw
Followup to:  <[email protected]>
By author:    Roman Zippel <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 10 lines

Hi,

Quoted 4 lines · H. Peter Anvin

To be able to *require* it, which means it can't significantly bloat the total size of the kernel image. klibc binaries are *extremely* small. Static kinit is only a few tens of kilobytes, a lot of which is zlib.

Quoted 6 lines

Every project starts small and has the annoying tendency to grow. That still doesn't answer, why it has to be distributed with the kernel, just install the thing somewhere under /lib and Kbuild can link to it. The point is that it contains nothing kernel specific and doesn't has to be rebult with every new kernel.

Actually, that isn't quite true. One of the ways klibc is kept small is by not guaranteeing a stable ABI... and not having compatibility support for older kernels. This is one of the kinds of luxuries that bundling offers.

Does it make bundling mandatory? Not really. In fact, people have been using klibc in its standalone form for years. However, I believe there would be a lot of resistace to have the kernel tarball have outside dependencies on anything but the most basic build tools.

-hpa

#4 lore raw

Hi,

Quoted 10 lines · H. Peter Anvin
Quoted 5 lines

That still doesn't answer, why it has to be distributed with the kernel, just install the thing somewhere under /lib and Kbuild can link to it. The point is that it contains nothing kernel specific and doesn't has to be rebult with every new kernel.

Actually, that isn't quite true. One of the ways klibc is kept small is by not guaranteeing a stable ABI... and not having compatibility support for older kernels. This is one of the kinds of luxuries that bundling offers.

It's indeed more of a luxury than a necessity.
How often does that really change? Bundling it with the kernel may actually encourage some developers to be less careful regarding compatibility. We already have enough problems with this as it is.

Quoted 4 lines

Does it make bundling mandatory? Not really. In fact, people have been using klibc in its standalone form for years. However, I believe there would be a lot of resistace to have the kernel tarball have outside dependencies on anything but the most basic build tools.

If you wouldn't remove all old init code at once it would still be possible to build a kernel this way. Why are you making it mandatory? Why don't you leave it optional for a while and only gradually remove the old code? This way distributions/users can experiment with it regarding their current initrd/boot setups.

Why shouldn't klibc be part of the basic build tools? I asked this already
earlier: where do you draw the line regarding duplication? Are you going to duplicate every single tool, which might be needed to build the kernel only to reduce outside dependencies? IMO that's illusory for more complex setups anyway. Let's take booting from raid, in this case you need to install mdadm anyway, which could also provide an initramfs version. This way the setup tools can be generated from the same source, which reduces duplication and maintenance overhead.

Just to be clear here, I really appreciate the work you've done, but I'm not exactly comfortable with merging a huge patch, which completely changes the boot sequence at once, without any clear plan of what's coming next. It would be a lot less problematic if the transition would be more gradually, which IMO is very well possible. Usually it's a requirement to split large patches, why should klibc be special?

bye, Roman

#5 lore raw
Followup to:  <[email protected]>
By author:    Roman Zippel <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 8 lines
Quoted 5 lines

Actually, that isn't quite true. One of the ways klibc is kept small is by not guaranteeing a stable ABI... and not having compatibility support for older kernels. This is one of the kinds of luxuries that bundling offers.

It's indeed more of a luxury than a necessity.
How often does that really change?

A lot more often than you'd think.

Quoted 5 lines

If you wouldn't remove all old init code at once it would still be possible to build a kernel this way. Why are you making it mandatory? Why don't you leave it optional for a while and only gradually remove the old code? This way distributions/users can experiment with it regarding their current initrd/boot setups.

Linus vetoed that option years ago. Anyway, the standalone version of klibc has been available for almost three years, and has been used by at least Debian for their initramfs code for a considerable amount of time. It's not like it's an untested piece of code.

Quoted 2 lines

Why shouldn't klibc be part of the basic build tools? I asked this already
earlier: where do you draw the line regarding duplication?

Judgement call. I'm not sure I have done the best possible job, either, but you have to call it somewhere.

Quoted 3 lines

Are you going to duplicate every single tool, which might be needed to build the kernel only to reduce outside dependencies? IMO that's illusory for more complex setups anyway.

Hyperbole.

FWIW, I've worked in environments where you pull a single source tree and get the compilers as well as the code you build them with. It is *wonderful* for reproducibility, but I don't think that's going to happen with respect to Linux, and noone is calling for that.

Quoted 4 lines

Let's take booting from raid, in this case you need to install
mdadm anyway, which could also provide an initramfs version. This way the setup tools can be generated from the same source, which reduces duplication and maintenance overhead.

You don't need mdadm to boot from RAID. kinit handles it just fine. At the same time, I do intend to continue to maintain the standalone klibc, which can be used to produce external binary trees.

Quoted 6 lines

Just to be clear here, I really appreciate the work you've done, but I'm not exactly comfortable with merging a huge patch, which completely changes the boot sequence at once, without any clear plan of what's coming next. It would be a lot less problematic if the transition would be more gradually, which IMO is very well possible. Usually it's a requirement to split large patches, why should klibc be special?

When I asked Andrew how he'd like me to pass him the code, he said he was happy pulling my git tree. This saved time for both of us, and made it easier and quicker for me to integrate fixes. You can do so too, which will get you a full history of the development, in about 1,400 pieces.

-hpa

#6 lore raw

H. Peter Anvin wrote:
[]

Quoted 6 lines
Quoted 4 lines

Let's take booting from raid, in this case you need to install
mdadm anyway, which could also provide an initramfs version. This way the setup tools can be generated from the same source, which reduces duplication and maintenance overhead.

You don't need mdadm to boot from RAID. kinit handles it just fine.

You *do* need mdadm to boot from RAID. Unless you rely on broken, obsolete, "don't use" in-kernel raid autodetection code (which, in this case, will be moved from kernel space into kinit). There are many reasons why raid autodetection in its current form is not a good idea, all goes to simple "unreliable" definition - there where many discussions about this already.

Well ok, mdadm/Assemble.c can be merged into kinit instead of current stuff present there, and adopted somehow. Until when, mdadm IS necessary.

Ok, the next question may be 'and what about lvm?', or dm, or whatever else.. Md autodetection code has been in kernel for a long time, while lvm/dm/etc stuff wasn't. So there IS a difference... ;)

But I see a reason for kinit *now*, in its current form - it's compatibility. Later on, maybe the whole stuff can be removed entirely, to rely on external tools for booting. Existing mkinitrd/mkinitramfs/ etc solutions works, they're being improved all the time, and they don't use kinit.

Did I get it all right? :)

Thanks.

/mjt

#7 lore raw
Quoted 16 lines · Michael Tokarev

You *do* need mdadm to boot from RAID. Unless you rely on broken, obsolete, "don't use" in-kernel raid autodetection code (which, in this case, will be moved from kernel space into kinit). There are many reasons why raid autodetection in its current form is not a good idea, all goes to simple "unreliable" definition - there where many discussions about this already.

Well ok, mdadm/Assemble.c can be merged into kinit instead of current stuff present there, and adopted somehow. Until when, mdadm IS necessary.

Ok, the next question may be 'and what about lvm?', or dm, or whatever else.. Md autodetection code has been in kernel for a long time, while lvm/dm/etc stuff wasn't. So there IS a difference... ;)

Quoted 8 lines

But I see a reason for kinit *now*, in its current form - it's compatibility. Later on, maybe the whole stuff can be removed entirely, to rely on external tools for booting. Existing mkinitrd/mkinitramfs/ etc solutions works, they're being improved all the time, and they don't use kinit.

Did I get it all right? :)

Pretty much. :)

However, this also is a good basis to point out that one of the big advantages with klibc is that you *can* share code with standalone userspace tools with relative ease. For example, porting mdadm/Assemble.c to be a tool -- be it standalone or integrated into kinit -- is a pretty trivial project, whereas trying to integrate code like that into the kernel proper is a major task.

As you can also notice, a number of modules of kinit are also available as standalone programs (and, in fact, I have already gotten requests to break out additional modules in the same way.) This means that that code is available to distributions to use in their initramfs solutions, without having to take the kinit package. Finally, unlike the current code, kinit can be wrapped, as is, from a script.

-hpa

#8 lore raw
In reply to Michael Tokarev

Hi,

Quoted 3 lines · Michael Tokarev

But I see a reason for kinit *now*, in its current form - it's compatibility. Later on, maybe the whole stuff can be removed entirely, to rely on external tools for booting.

The compatibility code is already in the kernel, just don't call it if e.g. there's a /kinit in initramfs. We can add the hooks to the kernel to pull whatever you want into the initramfs and with time we can remove the old code.
Why create new (temporary?) compatibility code, if the current code is working just fine?

bye, Roman

#9 lore raw
Quoted 9 lines · Roman Zippel

Hi,

Quoted 3 lines · Michael Tokarev

But I see a reason for kinit *now*, in its current form - it's compatibility. Later on, maybe the whole stuff can be removed entirely, to rely on external tools for booting.

The compatibility code is already in the kernel, just don't call it if

Isn't kinit/klibc intended to *replace* that whole code in the kernel?

Quoted 5 lines

e.g. there's a /kinit in initramfs. We can add the hooks to the kernel to pull whatever you want into the initramfs and with time we can remove the old code.
Why create new (temporary?) compatibility code, if the current code is working just fine?

That was partly my question (or something I had in mind but didn't ask).

Well. It's interesting.

Embedded folks are using uclibc or dietlibc - i see no reason for another "more small" libc for booting, regular tools (linked against uc or diet) can be used just fine.

Big systems will usually require some "advanced" stuff like that same mdadm or lvm - and again, I don't see why regular glibc can't be used (wich has an advandage - no need to have two versions of the toolset, linked with klibc and glibc).

The only real reason I see in klibc is to be able to remove all that 'kinit' code from kernel but still be compatible - so that I will be able to download linux-2.7, make oldconfig all install, and it Will Just Work (in case I had no initrd/initramfs) - new userspace kinit will take care of everything removed from kernel code.

In the other words, i see the whole klibc thing as a temporary compatibility solution.

Did I miss something?

And yes, it's a PITA to have two sets of all the system utilities - I'd not go this route. Also, since klibc will have compatibility problems with kernel API changes, such a 'parallel toolset' needs to be recompiled on a regular basis, which means not two toolsets, but many of them (one 'main', and several 'small for boot', for different kernel versions, linked against different klibc-XXX.so).

The only problem I see with using glibc in initramfs is -- systems with small amount of memory, where full-blown glibc with all the required tools will just not fit into ramfs/tmpfs. Not "embedded" systems (like boxed routers for example - they're already using something much more compact than glibc). Are there such systems out there still?

/mjt

#10 lore raw
Quoted 3 lines · Michael Tokarev

Embedded folks are using uclibc or dietlibc - i see no reason for another "more small" libc for booting, regular tools (linked against uc or diet) can be used just fine.

Then they can just use uclibc or dietlibc. There is nothing that prevents you from providing everything fr the initramfs. klibc is the default choice so the kernel can provide a functional set of programs. Without klibc it would be almost impossible to factor out all the code that ought to run in userspace but are part of the ekrnel because there were no other way to do it.

And klibc being temporary is some odd argument. Why should it be temporary. Are there other alternatives when bundling a 'libc' with the kernel?

Sam

#11 lore raw
In reply to Michael Tokarev

Hi,

Quoted 7 lines · Michael Tokarev
Quoted 5 lines
Quoted 3 lines

But I see a reason for kinit *now*, in its current form - it's compatibility. Later on, maybe the whole stuff can be removed entirely, to rely on external tools for booting.

The compatibility code is already in the kernel, just don't call it if

Isn't kinit/klibc intended to *replace* that whole code in the kernel?

Well, that's the basic idea, but slowly I'm wondering whether anyone has really figured out how to do this.
Being able to produce small binaries is part of it, but it somehow also has to work together with the rest of system, e.g. how can be configured (by the user, distribution, ...) or how can it be extended. We need to know where we are going, otherwise it's hard to know whether we're on the right path.

bye, Roman

#12 lore raw
In reply to H. Peter Anvin

Hi,

Quoted 9 lines · H. Peter Anvin
Quoted 7 lines
Quoted 4 lines

Actually, that isn't quite true. One of the ways klibc is kept small is by not guaranteeing a stable ABI... and not having compatibility support for older kernels. This is one of the kinds of luxuries that bundling offers.

It's indeed more of a luxury than a necessity.
How often does that really change?

A lot more often than you'd think.

Often enough?

Quoted 7 lines
Quoted 5 lines

If you wouldn't remove all old init code at once it would still be possible to build a kernel this way. Why are you making it mandatory? Why don't you leave it optional for a while and only gradually remove the old code? This way distributions/users can experiment with it regarding their current initrd/boot setups.

Linus vetoed that option years ago.

Name dropping is of course always impressive - scares little kids and all. Could you please provide more info, what exactly he vetoed?

Quoted 4 lines

Anyway, the standalone version of
klibc has been available for almost three years, and has been used by at least Debian for their initramfs code for a considerable amount of time. It's not like it's an untested piece of code.

I didn't say that, the point is if e.g. Debian is already distributing it, why do we need another copy?

Quoted 5 lines
Quoted 2 lines

Why shouldn't klibc be part of the basic build tools? I asked this already
earlier: where do you draw the line regarding duplication?

Judgement call. I'm not sure I have done the best possible job, either, but you have to call it somewhere.

Well for now you pretty much just moved code, that was in the kernel before. What I'm trying to find out is what is coming next. How does e.g. udev or modules fit into the picture?

Quoted 5 lines
Quoted 3 lines

Are you going to duplicate every single tool, which might be needed to build the kernel only to reduce outside dependencies? IMO that's illusory for more complex setups anyway.

Hyperbole.

Why?

Quoted 8 lines
Quoted 4 lines

Let's take booting from raid, in this case you need to install
mdadm anyway, which could also provide an initramfs version. This way the setup tools can be generated from the same source, which reduces duplication and maintenance overhead.

You don't need mdadm to boot from RAID. kinit handles it just fine. At the same time, I do intend to continue to maintain the standalone klibc, which can be used to produce external binary trees.

So why can't we use that external copy? mdadm is only example, there are more - please explain your plans. How will they be integrated?

Quoted 7 lines
Quoted 6 lines

Just to be clear here, I really appreciate the work you've done, but I'm not exactly comfortable with merging a huge patch, which completely changes the boot sequence at once, without any clear plan of what's coming next. It would be a lot less problematic if the transition would be more gradually, which IMO is very well possible. Usually it's a requirement to split large patches, why should klibc be special?

Quoted 5 lines

When I asked Andrew how he'd like me to pass him the code, he said he was happy pulling my git tree. This saved time for both of us, and made it easier and quicker for me to integrate fixes. You can do so too, which will get you a full history of the development, in about 1,400 pieces.

For -mm that's fine, but do you seriously expect it to get merged like that. Again, what makes klibc so special, that it doesn't have to follow standard rules?

Please be more specific, you are avoiding any clear answer. I'm really trying to understand, why you're doing it this way and what speaks against the alternatives and what are the next steps?

bye, Roman

#13 lore raw
Quoted 3 lines

Well for now you pretty much just moved code, that was in the kernel before. What I'm trying to find out is what is coming next. How does e.g. udev or modules fit into the picture?

udev and module-init-tools fits nicely with the kernel. I never have understood this 'keep-everyhig-separate' mantra. Just see how many people had troubles with missing module-init-tools or people having troubles with non-backward compatible udev.

Quoted 3 lines

For -mm that's fine, but do you seriously expect it to get merged like that. Again, what makes klibc so special, that it doesn't have to follow standard rules?

So part of what you ask for is a set of incremental patches that shows all the kernal modifications?
That should be doable with some effort but unfortunately I'm not up to the task.

Sam

#14 lore raw

Hi,

Quoted 7 lines · Sam Ravnborg
Quoted 3 lines

Well for now you pretty much just moved code, that was in the kernel before. What I'm trying to find out is what is coming next. How does e.g. udev or modules fit into the picture?

udev and module-init-tools fits nicely with the kernel. I never have understood this 'keep-everyhig-separate' mantra. Just see how many people had troubles with missing module-init-tools or people having troubles with non-backward compatible udev.

It just means that we suck at producing stable kernel interfaces. Moving everything into the kernel doesn't magically solve problem, it only shifts the problem.
You still had to define the boundaries - how does it interface with the rest of the system. This is the part I'm currently trying to figure out regarding klibc.

Quoted 5 lines
Quoted 3 lines

For -mm that's fine, but do you seriously expect it to get merged like that. Again, what makes klibc so special, that it doesn't have to follow standard rules?

So part of what you ask for is a set of incremental patches that shows all the kernal modifications?

Well, it's common practice. If we start making exceptions, everyone wants one, so it IMO would need a very, very good reason to merge this as is.

bye, Roman

#15 lore raw
In reply to Roman Zippel
Quoted 8 lines · Roman Zippel
Quoted 6 lines · H. Peter Anvin
Quoted 5 lines

If you wouldn't remove all old init code at once it would still be possible to build a kernel this way. Why are you making it mandatory? Why don't you leave it optional for a while and only gradually remove the old code? This way distributions/users can experiment with it regarding their current initrd/boot setups.

Quoted 4 lines
Quoted 1 line

Linus vetoed that option years ago.

Name dropping is of course always impressive - scares little kids and all. Could you please provide more info, what exactly he vetoed?

His rule was no code was to be added to the kernel code unless it already had a user. So there would be no adding code hoping a user would appear.

If I'm wrong, I'll likely be corrected... :-)

-Paul