klibc development release

Okay, I'm starting to have enough guts about this to release for testing...

klibc is a tiny C library subset intended to be integrated into the kernel source tree and being used for initramfs stuff. Thus, initramfs+rootfs can be used to move things that are currently in kernel space, such as ip autoconfiguration or nfsroot (in fact, mounting root in general) into user space.

I would particularly appreciate portability comments, since I'm flying blind on non-i386 machines (anyone want to send me hardware?), although any bug reports would be appreciated.

ftp://ftp.kernel.org/pub/linux/libs/klibc/klibc.tar.gz

I haven't bothered putting a version number on it, since it changes quite often. I have also published the CVS repository in the directory above.

-hpa

P.S. I'm aware gregkh already started a klibc project; I have discussed it with him and we have agreed my project supercedes his.

<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<[email protected]>

I just tried compiling for 64 bit s390x, this is what I found so far:

- in include/stdint.h, the definitions for *int_fast*_t conflict with the 64 bit ones

- mmap.c does not compile because _syscall6 is defined only for i386, alpha, cris, mips(64) and parisc but not for the other ten architectures.

- syscalls/time.o does not compile because __NR_time is not defined for alpha, ia64 and s390x

- making klibc cross compilable with 'make CROSS_COMPILE=prefix-' would be a nice and trivial addition

- compile time warnings: see below

more later when I actually manage to run something

	Arnd <><

testvsnp.c: In function `main':
testvsnp.c:14: warning: int format, different type arg (arg 3) testvsnp.c:17: warning: int format, different type arg (arg 3) testvsnp.c:20: warning: int format, different type arg (arg 3) testvsnp.c:23: warning: int format, different type arg (arg 3) testvsnp.c:31: warning: int format, different type arg (arg 3) testvsnp.c:36: warning: int format, different type arg (arg 3) testvsnp.c:39: warning: int format, different type arg (arg 3) testvsnp.c:42: warning: int format, different type arg (arg 3) testvsnp.c:45: warning: int format, different type arg (arg 3) testvsnp.c:48: warning: int format, different type arg (arg 3) testvsnp.c:51: warning: int format, different type arg (arg 3) testvsnp.c:54: warning: int format, different type arg (arg 3) testvsnp.c:57: warning: int format, different type arg (arg 3) testvsnp.c:60: warning: int format, different type arg (arg 3) testvsnp.c:63: warning: int format, different type arg (arg 3) testvsnp.c:66: warning: int format, different type arg (arg 3) testvsnp.c:69: warning: int format, different type arg (arg 3) testvsnp.c:72: warning: int format, different type arg (arg 3) testvsnp.c:75: warning: int format, different type arg (arg 3) testvsnp.c:79: warning: int format, different type arg (arg 3) testvsnp.c:83: warning: int format, different type arg (arg 3) testvsnp.c:87: warning: int format, different type arg (arg 3) testvsnp.c:90: warning: int format, different type arg (arg 3) testvsnp.c:93: warning: int format, different type arg (arg 3) testvsnp.c:96: warning: int format, different type arg (arg 3) testvsnp.c:99: warning: int format, different type arg (arg 3) testvsnp.c:102: warning: int format, different type arg (arg 3) testvsnp.c:105: warning: int format, different type arg (arg 3) testvsnp.c:108: warning: int format, different type arg (arg 3) testvsnp.c:111: warning: int format, different type arg (arg 3)
vfprintf.c: In function `vfprintf':
vfprintf.c:26: warning: cast from pointer to integer of different size fputs.c: In function `fputs':
fputs.c:15: warning: cast from pointer to integer of different size calloc.c: In function `calloc':
calloc.c:16: warning: type mismatch in implicit declaration for built-in function memset' vfprintf.c: In function vfprintf':
vfprintf.c:26: warning: cast from pointer to integer of different size fputs.c: In function `fputs':
fputs.c:15: warning: cast from pointer to integer of different size calloc.c: In function `calloc':

some regression data from running the test programs on s390 and s390x:

test 31 bit 64 bit
-----------------------------------------------------
microhello ok ok
minihello ok ok
hello ok ok
getenvtest wrong output segfault in strncmp
getopttest wrong output (getenv) segfault from getenv/strncmp
malloctest mmap not implemented mmap not implemented
testvsnp ok wrong output

You could say you are 50% there ;-)

For the autogenerated system call objects, a quick grep in include/asm-*/unistd showed some more that appear to be missing on some platforms:

alpha: alarm, fdatasync, getegid, geteuid, getgid, getpid, getppid, getuid, nice,
pause, umount2, utime, time
ia64: alarm, fork, getpgrp, nice, pause, umount2, utime, vfork, time
mips64: pivot_root, select, vfork
mips: vfork
parisc: select, umount
x86_64: nice, umount
sparc: setresgid, setresuid
s390x: time
s390: select
m68k: pivot_root

Hmf... some of these seem to be outright omissions (pivot_root() and umount2() especially), and probably indicate bugs or that the stock kernel isn't up to date anymore.

I can see umount() being missing (as in "use umount2()"). time() I guess can be emulated with gettimeofday(), but select() being missing is rather odd (is this a "emulate using poll()" philosophy? Barf...)

If the maintainers of the above ports could give me a pointer to how the above system calls are expected to be done I'd appreciate it. I'll probably look through the glibc sources for hints in the meantime.

-hpa

HPA> Hmf... some of these seem to be outright omissions HPA> (pivot_root() and umount2() especially), and probably indicate HPA> bugs or that the stock kernel isn't up to date anymore.

HPA> I can see umount() being missing (as in "use umount2()").

Alpha calls umount2() "oldumount"; ia64 never had a one-argument version of umount(), so there is no point creating legacy (and the naming is inconsistent anyhow...).

	--david

The gratuitous inconsistencies between platforms is something that is currently driving me up the wall. I'm starting to think the NetBSD people have the right idea: when you add a system call on NetBSD, you only have to add it in one place and it becomes available on all the platforms they support. Of course, you can provide a custom implementation for any one platform, but the idea is to keep as much of the code generic as possible...

-hpa

...

History... Initially I thought you are describing something
different from Linux model (after all, different platforms have different ABIs for syscalls, leading to different call-vector tables, etc.) but no, I see no difference from this description.

How NetBSD handles the issue, I don't know. One interpretation
of what you say is that when a new architecture is added to NetBSD, it will instantly inherit the entire historical set of syscalls, including the obsolete ones.

As long as old calls are supported, the kernel needs to know how to handle a call serving some thing 10 years ago.. ( e.g. "oldumount"). Ok, some have been revoked, but not everything of the very old and obsolete stuff.

How this reflects to klibc ? What I understand of klibc,
it can track very closely the kernel in question. Introducing new improved syscall to do XYZ obsoliting ABC, whimblam, you replace it. No need to carry code supporting any other version of kernel than for what the stuff is made for. ("Small is beautifull", and all that...)

glibc folks are (to an extreme pain) supporting kernels 2.0 (if not from before) to current bleeding edge, and emulating all fancy dancing available with new system services by doing some weird gimmicks..

Desire to support running of "native UNIX" binaries means emulating their syscalls. Initially Linux ran MINIX binaries, then came SCO binaries, and Alpha running some of OSF/1 binaries...

/Matti Aarnio

What NetBSD does is that they treat the system call mechanism as a special kind of function call... like an RPC stub kind of thing. Just like it's not visible to the C programmer how a particular architecture is implementing a function call, on NetBSD all one really needs to know is the system call number (which is architecture independent!) and its prototype, and you can generate both user-side and kernel-side stubs for this system call, ending up with it calling a function in the generic kernel code (sys_mmap(), or whatever.)

It seems to me that the big losers of the Linux model are the smaller architectures, since most people aren't going to be able to know when they're breaking something on, for example, alpha, cris or s390, and they're certainly not going to know how to add something to them.

-hpa

The glibc folks care about binary compatibility. klibc doesn't need to care about such things, and therefore need not care about such baggage.

-Erik

--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

Exactly.

-hpa

netbsd puts all syscall code not needed by the current release under a per-version ifdef. A new port starting at, say 1.4, will never have this enabled (unless it has binary emulations that need parts of the old syscalls)

Sure, but that still means the *current* ABI is consistent between platforms.

-hpa

yes, that's why I like the way they do it. It doesn't really impose any overhead and is much, much cleaner.

100% in agreement.

-hpa

Amen brother. That would be great! But I'm not holding
my breath waiting to see it,

-Erik

--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

You mean alpha calls umount2() "umount" and umount() "oldumount"?

-hpa

HPA> You mean alpha calls umount2() "umount" and umount() HPA> "oldumount"?

Yes.

	--david

Cool; I have added this. I have uploaded the new version as klibc-0.1.tar.gz to get the version number thing going...

-hpa

"H. Peter Anvin" <[email protected]> writes:

|> I would particularly appreciate portability comments, since I'm flying |> blind on non-i386 machines (anyone want to send me hardware?),

You don't need hardware, just use a cross-compiler.

Andreas.

Andreas Schwab, SuSE Labs, [email protected]
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

I'd like to actually be able to test something. The cross-compiler helps somewhat, of course, but doesn't let me test anything.

-hpa

Could I link 4-clause BSD source against this? (the GPL is incompatible with the 4-clause BSD license)

I could test on PowerPC, but wouldn't be able to tell you if I'm testing the latest code or not. You don't need to get creative with the version number; an integer is fine.

I'm planning to release this under a BSD-like license, such as 3-clause BSD, MIT or the X license. I'm still looking at each of those.

Good point.

-hpa

What's wrong with LGPL? I thought libraries were what it was originally intended for. (Is 4 clause BSD incompatable with LGPL?)

Yeah, I know stallman's decided to hate. I'm sure he'd be happy to know that stance encourages stuff to be released BSD-ish instead. :)

Rob

klibc is static-only. So for all practical purposes LGPL would be every bit as viral as GPV itself.

You say that as if it were a bad thing.

(And technically incorrect, if you also provide .o files so that the end user can relink as they desire.)

That aside for the moment, isn't the plan to pull stuff that's currently GPL out of the kernel and link against this lib anyway?

Second point - if it goes into the kernel source tree as suggested, but with a 'copycenter' license, we can expect to have the nVidia problem but worse.

Making it GPL shouldn't be a big deal. It is intended to be a small amount of code, after all. And I'd hate to get into a situation where people started shipping their magic 'make the hardware work' bits as closed replacements for the early boot code.

 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

I do. I have no problems with people choosing whatever license they prefer for their work and I have no problems with using GPL when I'm working on projects that are already under it, but it's not the license I would choose for my work in cases when I have a choice.

As for the "make the hardware work" code, there's nothing to stop people from doing that _NOW_. I'm not too fond of that, but as long as we are talking about userland code it
* will have to use normal system calls
* will not have to link against any particular library, no matter
what we provide
* will be up to those who write it and those who decide to use it.

We are talking about libc. _Nothing_ in that code couldn't be reimplemented by any half-competent programmer. It's a textbook stuff. Those who don't like GPL would be trivially able to reimplement all these functions in their own code anyway. End of story. Whatever license is chosen, it won't prevent people from putting their code under any license they like.

There is a crucial difference from the situation with nVidia, Veritrash and the rest of let's-bugger-the-kernel team. _They_ want more than using syscalls from user mode - they want an access to guts of the kernel and that's a very different can of worms. And _that_ I have problems with. A lot. Especially when they expect us to abstain from changes of kernel internals that might break their junk and when they whine when such changes are done.

People do have a right to put their code under whatever license they like. Now, _I_ won't use the stuff I don't have a source for unless I have exceptionally good reason to believe that authors of that stuff are among the few percents of programmers who *can* find their arse without outside help. But that has nothing to do with licensing or any moral considerations and everything to the fact that I know what kind of crap most of the software is.

Remind me why we'd want this in the kernel source tree when we don't even have modutils there? Or a real bootloader? There is no requirement that the kernel must be able to build a bootable image with ip autoconf and nfsroot, etc. without using external tools. A minimal 'parse command line to mount root and call real init' might be nice, but mostly as an example, like the example watchdog code.

I think a better way to go is to simply roll an initbootutils.tar.gz, point to it in the kernel CHANGES, etc., start pulling stuff into it, and people will catch on in no time.

 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

It's optional: CONFIG_MODULES=n

Also optional: cat bzImage > /dev/fd0 (ugly but still works)

How about partition detection? When initramfs goes in that's one of the things they're threatening to move to userspace. Also lots of the hardware detection and setup (ACPI, hotplug style PCI probing...)

If that sort of stuff goes into userspace, you may not be able to boot ANY linux system without it.

See "partition detection in userspace". How minimal is "minimal"?

Possibly it could be gotten to work before being split off? And then there's the question of upgrading your kernel without upgrading the initbootutils: if this never happens, what exactly is the benefit of it being in another tarball? (I.E. how tight is the binding gonna be? I certainly don't know yet...)

Rob

For some configurations. Part of this boot in userspace deal was more reliance on modules and late initialization.

On some machines. I don't have any with floppies. Making a bootable CDROM without a real userspace is kindof silly. And I can't do a whole heck of a lot with that floppy unless I've got some other packages around anyway.

Yes, yes. The whole point of this exercise is to move such things out of the kernel. The mechanics of booting can (and should) be made entirely independent of the kernel tree. LILO, grub, etc., do this on one side, bootutils.tgz should do this on the other. The kernel's job here is to get some form of userspace running and say 'not my job'. Having an 'official' bootutils tied to the kernel effectively keeps it kernel policy and will discourage people from rolling their own alternatives.

Big deal. For most practical purposes, you already need a bunch of other packages to do anything useful.

Hello world, really. Given that kernels can't boot off IDE without a bootloader, partition detection isn't going to get it closer to self-booting.

It ought not be any more tightly bound than regular libc. Isn't that the point? If it still depends on non-generic services in the kernel, then we haven't succeeded in pulling it all the way into userspace.

 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

...

Klibc is a neat hack, but makes little sense. In the end we wind up with _more_ code to maintain, not less. The boot process becomes more complicated too. A microkernel by any other name still smells. :-) Not all of us have 2 GHz boxes BTW.

Moving partition code out of the kernel is just begging for bugs and limited functionality. The EVMS people have the right idea. Does anyone else remember the user-space isapnp disaster? I do. Users everywhere were screaming "my sound card won't work".

"Regular" libc being glibc? Which still supports 2.0 kernels by sacrificing chickens in the preprocessor, takes an insanely long time to compile (and with an invocation that's just a touch more complicated than "configure; make; make install"), and statically links "hello world" to 1.6 megabytes?

The system's libc is the glue between userspace and kernel space. Every time the kernel comes up with new functionality (thread groups, 32 bit uids, 64 bit file access...), userspace can't access it until libc gets up to speed. Pretending your libc isn't related to the kernel at all and can easily port to HP-UX and netbsd is just a recipe for bloat if you ask me. Sometimes, being too generic is a BAD thing.

Personally, I question the klibc work a bit because it's reinventing the wheel with projects like uclibc already mostly there. But Al pointed out the license issue with static linking against LGPL libraries, and in any case a stripped down minimalist libc as a reference implementation for others to build on is probably a good thing. My interest in the libc area is a bit off to one side: I'm not fond of glibc and am looking to replace it in my own system, but it hasn't made it to the top of my to-do list yet. (Dietlibc is straight GPL: it can't even be the dynamic replacement for glibc in a real world linux distribution. HPA suggested I look at newlibc, which I've added to my to-do list).

a lot more understandable. A lot more people understand userspace than understand kernel space, and a LOT more people are competent to hack userspace than kernel space. (100 times as many people, at LEAST. Even pinning down the order of magnitude depends on your definition of "competent", on both sides...)

A microkernel has multiple processes committing incest (ahem, "cooperating") spending all their time passing messages to each other, racing, and deadlocking on their own attempts at communication and synchronization.

Initramfs has a bunch of userspace processes initializing the system, but they mostly run one after the other in series and don't have to communicate with each other any more than the components of your average shell script. Basically, it takes the concept of init scripts and puts some of them before "init". That isn't microkernel thinking, shell scripts are bog standard unix philosophy.

My 133 mhz Dell laptop recently died, and got replaced by a 366 mhz toshiba. Woo, blazing speed. My home desktop's still a pentium pro 180. (I LIKE developing on slow machines: it makes you notice inefficient code and care about fixing it. I have a 700 mhz athlon at work. It's a file server.)

I'm kind of surprised people think an ACPI interpreter is going to depend on completely generic userspace services, but I suppose it takes all kinds. (I can't say I'm an expert on ACPI, but from what I understand neither is anybody else, including Intel.) And of course if you write devicefs so you can expose your device tree largely for power management purposes, and then retroactively call it a standard userspace service so you don't feel guilty when your ACPI interpreter interacts with it... I'm all for it, it's a better API overall, but there's a bit of putting Descartes before the horse in there somewhere saying ACPI in userspace depends on generic services that are generic and exposed to userspace in part because you wanted to put ACPI in userspace long before the infrastructure to do so existed... :)

The external pcmcia add-on package wasn't much fun either, and that required building against your kernel source directory...

Define "out of the kernel". Looking at it one way, the nvidia driver is "out of kernel", even though it's a driver. Looking at it another way, menuconfig is in-kernel. It's in the tarball...

Doing stuff in process context can be very nice. We have plenty of hybrid kernel/userspace things already: anything that shows up in ps with square brackets around it. Initramfs puts more of the init-time stuff in process context, and the real REASON for doing this is again: more people grok userspace than grok an environment with spinlocks and preemption counts, where scheduling and swapping are not your friend, your stack is tiny, it's difficult to access the filesystem, you have to worry about what operations might conceivably block, reference counting and race conditions abound, you have to care about the difference between a virtual address and a physical one and then there's highmem and mapping to a PCI bus address...

There's simply a lot less to screw up in userspace.

That said, decoupling the reference implementation of the initramfs boot scripts from the kernel tarball is a seperate issue. Thinking "we don't have to maintain that stuff, when it breaks somebody else will fix it" sounds fun, but is not always the world's most effective strategy. If it stops being broken on a regular basis by new development, fine. But I for one would prefer to wait for it to stabilize BEFORE trying to fork it off too far.

Of course it's not my call...

Rob

Keep in mind that you _can_ staticly link even closed source stuff against an LGPL library... You just need to "Accompany the work with a written offer, valid for at least three years" to provide users with an object file that can be re-linked against newer versions of the LGPL library. Or simply provide the .o file. See the LGPL Section 6...

As far as I know, uClibc is the only library that is able to replace glibc for real world linux distributions... And I've looked long and hard (which was why I ended up making uClibc),

-Erik

--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

This, of course, is not a goal for klibc at all. I looked at uclibc, dietlibc, and newlib before starting klibc. klibc is meant to be *tiny* first of all, and is very much not designed to be able to compile arbitrary programs.

-hpa

Since klibc is meant for compiling programs tightly bound to the kernel in initramfs, such as partition scanning, NFS root mounting, module loading etc., I wonder what the problem with even a GPLed klibc is?

Surely all the programs that its intended to be used with will be GPLed, perhaps even part of the kernel source tree?

(Not that I mind at all. I expect I will be using klibc's simplicity and 3-clause BSD license in my employer's proprietary product soon enough..)

-- Jamie