Driver Core: devtmpfs - kernel-maintained tmpfs-based /dev

drivers/base/core.c +2 -0

@@ -929,6 +929,8 @@ int device_add(struct device *dev)
 		error = device_create_sys_dev_entry(dev);
 		if (error)
 			goto devtattrError;
+
+		devtmpfs_create_node(dev);
 	}
 
 	error = device_add_class_symlinks(dev);

drivers/base/devtmpfs.c new file +0 -0

+int devtmpfs_create_node(struct device *dev)

+int devtmpfs_delete_node(struct device *dev)

Quoted 12 lines · Greg KH

drivers/base/core.c +2 -0

@@ -929,6 +929,8 @@ int device_add(struct device *dev)
 		error = device_create_sys_dev_entry(dev);
 		if (error)
 			goto devtattrError;
+
+		devtmpfs_create_node(dev);
 	}
 
 	error = device_add_class_symlinks(dev);

[...]

Quoted 2 lines

drivers/base/devtmpfs.c new file +0 -0

[...]

Quoted 1 line

+int devtmpfs_create_node(struct device *dev)

Returns an int that's never checked. Is this intentional?

Quoted 1 line

+int devtmpfs_delete_node(struct device *dev)

Similarly.

Here's the devtmpfs patch again. For .32 it's a simple and clean patch.

What tree did you find with devtmpfs in it? It was pretty clearly rejected when it came up.

so just to state the obvious: this code is not needed to boot fast. It is mostly a workaround for having a bad initrd; if you don't use an initrd, or if you use an initrd that's made with the right device nodes in it already, you really just don't need this.

I would much rather that you just fix your initrd... than to put this sort of thing into the kernel....

Eric has shown that just making the nodes is 0.06 seconds with todays sysfs interface, and there is room for improvement,

Quoted 5 lines

option for us. And still, nobody will be forced
to use it, it's entirely optional. For our systems, we decided to do it that way, and we ship it already in the distro, and if there are no substantial problems coming up, which we don't expect, we will continue using it.

Yes but you are asking all of us to maintain it. Forever in perpetuity. A better case needs to be made than you have already shipped the code.

Quoted 1 line · On Wed, 5 Aug 2009 10:15:13 -0700 Greg KH <[email protected]> wrote:

Here's the devtmpfs patch again. For .32 it's a simple and clean patch.

You seem to have reposted it rather than anyone addressing any of the complaints people made about ?

To quote Christoph

Quoted 2 lines

What tree did you find with devtmpfs in it? It was pretty clearly rejected when it came up.

and Arjan (who is doing about the fastest Linux boot anyone has and has better data than anyone else)

Quoted 7 lines

so just to state the obvious: this code is not needed to boot fast. It is mostly a workaround for having a bad initrd; if you don't use an initrd, or if you use an initrd that's made with the right device nodes in it already, you really just don't need this.

I would much rather that you just fix your initrd... than to put this sort of thing into the kernel....

plus

Quoted 2 lines

Eric has shown that just making the nodes is 0.06 seconds with todays sysfs interface, and there is room for improvement,

and Erik Biederman

Quoted 8 lines

Quoted 5 lines

option for us. And still, nobody will be forced
to use it, it's entirely optional. For our systems, we decided to do it that way, and we ship it already in the distro, and if there are no substantial problems coming up, which we don't expect, we will continue using it.

Yes but you are asking all of us to maintain it. Forever in perpetuity. A better case needs to be made than you have already shipped the code.

Also can you confirm the SELinux issues raised by Stephen Smalley and David Quigley were fixed and resolved.

From what I recall, yes, they were. I'm guessing the Red Hat boot tests

Quoted 12 lines · Alan Cox

Quoted 1 line · On Wed, 5 Aug 2009 10:15:13 -0700 Greg KH <[email protected]> wrote:

Here's the devtmpfs patch again. For .32 it's a simple and clean patch.

You seem to have reposted it rather than anyone addressing any of the complaints people made about ?

To quote Christoph

Quoted 2 lines

What tree did you find with devtmpfs in it? It was pretty clearly rejected when it came up.

That's not anything to address, what do you want me to say, "maple"? :)

Quoted 2 lines

and Arjan (who is doing about the fastest Linux boot anyone has and has better data than anyone else)

We did this work based on Arjan's work. It solves a problem that Moblin has as well, which is why Novell is using it for their Moblin-based images they are shipping. It makes things go faster, with a simpler userspace codepath, and we have the numbers as have been previously posted.

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

And then there's the great reason Kay previously stated, "if sysfs has all the names, numbers, and device properties, why would we need complex userspace to have the nodes?"

Quoted 5 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

If you use mount by label or UUID, you still need udev (or other tools)

in the initrd to find the right disk, correct? And for distros that want to support that, does this really reduce the amount of code in the initrd?

Quoted 7 lines · Andi Kleen

Quoted 5 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

Quoted 4 lines

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

If you use mount by label or UUID, you still need udev (or other tools) in the initrd to find the right disk, correct?

And for distros that want to support that, does this really reduce the amount of code in the initrd?

Quoted 6 lines · David Dillow

Quoted 5 lines · Andi Kleen

Quoted 4 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

Quoted 5 lines

Quoted 3 lines

As a initrd less kernel user I can really only agree: getting rid
of the udev-in-initrd requirement would be a big step forward
in usability. Typically I always have to pre populate a on disk /dev manually first to get my kernels to boot.

If you use mount by label or UUID, you still need udev (or other tools) in the initrd to find the right disk, correct?

Yes, you would.

Quoted 2 lines · David Dillow

If you use mount by label or UUID, you still need udev (or other tools) in the initrd to find the right disk, correct?

Yes, you would.

Quoted 2 lines

And for distros that want to support that, does this really reduce the amount of code in the initrd?

Yes it does, see the code in the Novell Moblin images for an example of how to use devtmpfs with udev for how to do this.

Quoted 9 lines · Greg KH

Quoted 2 lines · David Dillow

If you use mount by label or UUID, you still need udev (or other tools) in the initrd to find the right disk, correct?

Yes, you would.

Quoted 2 lines

And for distros that want to support that, does this really reduce the amount of code in the initrd?

Quoted 2 lines

Yes it does, see the code in the Novell Moblin images for an example of how to use devtmpfs with udev for how to do this.

I followed up with Greg offline to make sure I was looking at the right thing, but the upshot is that Moblin does not use an initrd and devtmpfs does not in any way reduce the amount of code for those distros that want to support more than the root=/dev/blah syntax.

Given Eric and Arjan's numbers about the time it takes to populate /dev from /sys -- pointing the speed problem squarely at udev or the ruleset -- I don't see much of a win for devtmpfs other than avoiding a static /dev to make init=/bin/sh work.

You cannot rely on it for a hotplug disk such as USB, as it doesn't do the root=LABEL=usbroot that you'd want to do since the location is not stable.

Quoted 4 lines · David Dillow

I followed up with Greg offline to make sure I was looking at the right thing, but the upshot is that Moblin does not use an initrd and devtmpfs does not in any way reduce the amount of code for those distros that want to support more than the root=/dev/blah syntax.

Like Scott points out, this option now allows distros to use blkid directly instead of udev if they want to implement things like this without udev.

Quoted 4 lines

Given Eric and Arjan's numbers about the time it takes to populate /dev from /sys -- pointing the speed problem squarely at udev or the ruleset -- I don't see much of a win for devtmpfs other than avoiding a static /dev to make init=/bin/sh work.

That's a very big win right there, don't you think?

Also, embedded and
rescue disks also would like it for this very reason.

Quoted 3 lines

You cannot rely on it for a hotplug disk such as USB, as it doesn't do the root=LABEL=usbroot that you'd want to do since the location is not stable.

Again, not all the world requires such labels.

No, not really. It isn't hard to make a static /dev, or a rescue initrd for the cases with dynamic device numbers.

Quoted 2 lines

No, not really. It isn't hard to make a static /dev, or a rescue initrd for the cases with dynamic device numbers.

There's a world between strictly controlled embedded and fully general distributions.

I want a dynamic /dev, but a fast one that doesn't need initrd or slows down booting.

This patch seems to be a good way to get that.

Quoted 2 lines

No, not really. It isn't hard to make a static /dev, or a rescue initrd for the cases with dynamic device numbers.

There's a world between strictly controlled embedded and fully general distributions.

I want a dynamic /dev, but a fast one that doesn't need initrd or slows down booting.

Quoted 5 lines · Andi Kleen

Quoted 2 lines

No, not really. It isn't hard to make a static /dev, or a rescue initrd for the cases with dynamic device numbers.

There's a world between strictly controlled embedded and fully general distributions.

Sure, and I've acknowledged that. But it doesn't mean this needs a kernel solution.

Quoted 2 lines

I want a dynamic /dev, but a fast one that doesn't need initrd or slows down booting.

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

Quoted 7 lines · David Dillow

Quoted 5 lines · Andi Kleen

Quoted 2 lines

No, not really. It isn't hard to make a static /dev, or a rescue initrd for the cases with dynamic device numbers.

There's a world between strictly controlled embedded and fully general distributions.

Quoted 8 lines

Sure, and I've acknowledged that. But it doesn't mean this needs a kernel solution.

Quoted 2 lines

I want a dynamic /dev, but a fast one that doesn't need initrd or slows down booting.

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

Their program only handles it for a reconstruction of /dev based on sysfs one time at boot. It does not handle things that are added or discovered by the system after that, you need udev for that.

So it's a great hack for boot time stuff, but not a complete /dev management replacement

Quoted 2 lines · David Dillow

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

Their program only handles it for a reconstruction of /dev based on sysfs one time at boot. It does not handle things that are added or discovered by the system after that, you need udev for that.

So it's a great hack for boot time stuff, but not a complete /dev management replacement like this code can be for numerous systems.

Quoted 4 lines · Greg KH

Quoted 2 lines · David Dillow

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

Quoted 6 lines

Their program only handles it for a reconstruction of /dev based on sysfs one time at boot. It does not handle things that are added or discovered by the system after that, you need udev for that.

So it's a great hack for boot time stuff, but not a complete /dev management replacement like this code can be for numerous systems.

What systems would those be?

Quoted 11 lines · David Dillow

Quoted 10 lines · Greg KH

Quoted 3 lines · David Dillow

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

Their program only handles it for a reconstruction of /dev based on sysfs one time at boot. It does not handle things that are added or discovered by the system after that, you need udev for that.

Quoted 4 lines

Quoted 2 lines

So it's a great hack for boot time stuff, but not a complete /dev management replacement like this code can be for numerous systems.

What systems would those be?

Rescue disks, Embedded systems with no local users, servers with no local users, etc. Basically anything that you are only root on, and don't care about group permissions.

Quoted 15 lines · On Wed, 12 Aug 2009 06:44:40 -0700 Greg KH <[email protected]> wrote:

Quoted 14 lines · David Dillow

Quoted 12 lines · Greg KH

Quoted 3 lines · David Dillow

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

Their program only handles it for a reconstruction of /dev based on sysfs one time at boot. It does not handle things that are added or discovered by the system after that, you need udev for that.

So it's a great hack for boot time stuff, but not a complete /dev management replacement like this code can be for numerous systems.

Quoted 5 lines

Quoted 1 line

What systems would those be?

Rescue disks, Embedded systems with no local users, servers with no local users, etc. Basically anything that you are only root on, and don't care about group permissions.

you're speaking for devtmpfs; the tool we use actually does do group and owners.

Quoted 9 lines · David Dillow

Quoted 8 lines · Greg KH

Quoted 2 lines · David Dillow

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

Their program only handles it for a reconstruction of /dev based on sysfs one time at boot. It does not handle things that are added or discovered by the system after that, you need udev for that.

Quoted 4 lines

Quoted 2 lines

So it's a great hack for boot time stuff, but not a complete /dev management replacement like this code can be for numerous systems.

What systems would those be?

Rescue disks, Embedded systems with no local users, servers with no local users, etc. Basically anything that you are only root on, and don't care about group permissions.

Quoted 15 lines · Greg KH

Quoted 13 lines · David Dillow

Quoted 10 lines · Greg KH

Quoted 2 lines · David Dillow

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

Their program only handles it for a reconstruction of /dev based on sysfs one time at boot. It does not handle things that are added or discovered by the system after that, you need udev for that.

So it's a great hack for boot time stuff, but not a complete /dev management replacement like this code can be for numerous systems.

What systems would those be?

Quoted 3 lines

Rescue disks, Embedded systems with no local users, servers with no local users, etc. Basically anything that you are only root on, and don't care about group permissions.

We've discussed those, though, and I think most of those fall under either having full control of their static environment, or would have something to watch for hotplug events -- udev or a replacement -- and so would have the ability to create the node from user space.

Rescue disks may be an exception, but they only need a minimal static /dev and then run a tool to get the initial setup. If you are expecting an expert to use them, then either running the tool by hand or as a post-modprobe rule should catch updates. They can also run udev (or a replacement) with stripped rules if you want to make it friendlier to non-expert users.

Quoted 6 lines · David Dillow

Rescue disks may be an exception, but they only need a minimal static /dev and then run a tool to get the initial setup. If you are expecting an expert to use them, then either running the tool by hand or as a post-modprobe rule should catch updates. They can also run udev (or a replacement) with stripped rules if you want to make it friendlier to non-expert users.

Ok, so don't use devtmpfs on your rescue disk, let the rest of us have a choice :)

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

Quoted 2 lines

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

That would likely need initrd at least on the distro I use which uses device files very early.

Quoted 2 lines

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

That would likely need initrd at least on the distro I use which uses device files very early.

Quoted 3 lines · On Wed, 12 Aug 2009 09:31:52 +0200 Andi Kleen <[email protected]> wrote:

Quoted 2 lines

So use Eric/Arjan's program that does it in 60ms -- you get a dynamic /dev, no initrd, fast boot, and no kernel changes required.

Quoted 2 lines

That would likely need initrd at least on the distro I use which uses device files very early.

nah just need to call it early in /etc/rc.sysinit.

initrd is a read herring; distros don't need an initrd (well except fedora who ship borken with a /dev without even /dev/console)

Quoted 11 lines · Andi Kleen

Quoted 5 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

If you use mount by label or UUID, you still need udev (or other tools) in the initrd to find the right disk, correct? And for distros that want to support that, does this really reduce the amount of code in the initrd?

Quoted 4 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

Quoted 10 lines · Andi Kleen

Quoted 4 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

Oh good, I thought I was the only one doing that.

The reason I don't like udev is that it's just to slow; something like a 5-10s delay on each boot.  No idea why it should be so slow,

but it's probably
probing the kernel for all available devices at boot, when it could be much quicker by probing for the device on access.

Quoted 12 lines · Al Boldi

Quoted 10 lines · Andi Kleen

Quoted 4 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

Quoted 4 lines

Oh good, I thought I was the only one doing that.

The reason I don't like udev is that it's just to slow; something like a 5-10s delay on each boot. No idea why it should be so slow,

Because you setup is broken, I guess.

Quoted 3 lines

but it's probably
probing the kernel for all available devices at boot, when it could be much quicker by probing for the device on access.

It takes more like 0.5 - 0.7 seconds on a usual setup.

Quoted 20 lines · Kay Sievers

Quoted 12 lines · Al Boldi

Quoted 10 lines · Andi Kleen

Quoted 4 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

Quoted 4 lines

Oh good, I thought I was the only one doing that.

The reason I don't like udev is that it's just to slow; something like a 5-10s delay on each boot.  No idea why it should be so slow,

Because you setup is broken, I guess.

Quoted 5 lines

Quoted 3 lines

but it's probably
probing the kernel for all available devices at boot, when it could be much quicker by probing for the device on access.

It takes more like 0.5 - 0.7 seconds on a usual setup.

I don't know about your setup, but all dists I used (fedora, mandriva, ubuntu, debian) hang every boot for 5-10s.

Quoted 10 lines · Andi Kleen

Quoted 4 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

Oh good, I thought I was the only one doing that.

The reason I don't like udev is that it's just to slow; something like a 5-10s delay on each boot. No idea why it should be so slow, but it's probably probing the kernel for all available devices at boot, when it could be much quicker by probing for the device on access.

Quoted 14 lines · Al Boldi

Quoted 10 lines · Andi Kleen

Quoted 4 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

Oh good, I thought I was the only one doing that.

Quoted 4 lines

The reason I don't like udev is that it's just to slow; something like a 5-10s delay on each boot. No idea why it should be so slow, but it's probably probing the kernel for all available devices at boot, when it could be much quicker by probing for the device on access.

Like Kay stated, this sounds like a misconfiguration of your distro's udev setup, as the ones I use (openSUSE and Gentoo) do not have this problem at all.

Quoted 20 lines · Greg KH

Quoted 18 lines · Al Boldi

Quoted 10 lines · Andi Kleen

Quoted 4 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

Oh good, I thought I was the only one doing that.

The reason I don't like udev is that it's just to slow; something like a 5-10s delay on each boot. No idea why it should be so slow, but it's probably probing the kernel for all available devices at boot, when it could be much quicker by probing for the device on access.

Quoted 3 lines

Like Kay stated, this sounds like a misconfiguration of your distro's udev setup, as the ones I use (openSUSE and Gentoo) do not have this problem at all.

Maybe they are using the same trick as Ubuntu and Debian, as they run udev in the background to hide the slowness. Both Fedora and Mandriva run udev in the foreground where the slowness is visible.

So really, if devtmpfs compares to udev speeds then this just looks like a devfs comeback. Remember, devfs was really slow.

Quoted 2 lines · Al Boldi

So really, if devtmpfs compares to udev speeds then this just looks like a devfs comeback. Remember, devfs was really slow.

Again, there is no "speed" for devtmpfs on its own, the device nodes just appear when the devices are added to the kernel, the speed of that depends on the device discovery within the kernel, nothing else.

Quoted 7 lines · Greg KH

Quoted 2 lines · Al Boldi

So really, if devtmpfs compares to udev speeds then this just looks like a devfs comeback. Remember, devfs was really slow.

Again, there is no "speed" for devtmpfs on its own, the device nodes just appear when the devices are added to the kernel, the speed of that depends on the device discovery within the kernel, nothing else.

So on bootup this would mean a lot of discovery.

I think we could get some big speedup, by just dumping the possible non-realized device list on bootup, and then just refine it on physical access. This could make devtmpfs an acceptable replacement to static /dev.

Quoted 5 lines · Al Boldi

Quoted 4 lines · Greg KH

Quoted 2 lines · Al Boldi

So really, if devtmpfs compares to udev speeds then this just looks like a devfs comeback. Remember, devfs was really slow.

Quoted 5 lines

Quoted 3 lines

Again, there is no "speed" for devtmpfs on its own, the device nodes just appear when the devices are added to the kernel, the speed of that depends on the device discovery within the kernel, nothing else.

So on bootup this would mean a lot of discovery.

Yes, all of this happens within the kernel, like normal. What are you getting at?

Quoted 4 lines

I think we could get some big speedup, by just dumping the possible non-realized device list on bootup, and then just refine it on physical access. This could make devtmpfs an acceptable replacement to static /dev.

Um, that's exactly what devtmpfs does, it creates the nodes based on the fact that the devices were physically (or virtually for some devices) discovered and registered with the kernel. This happens at the same time the existing uevents are generated and sent out to userspace.

Quoted 10 lines · Greg KH

Quoted 9 lines · Al Boldi

Quoted 7 lines · Greg KH

Quoted 2 lines · Al Boldi

So really, if devtmpfs compares to udev speeds then this just looks like a devfs comeback. Remember, devfs was really slow.

Again, there is no "speed" for devtmpfs on its own, the device nodes just appear when the devices are added to the kernel, the speed of that depends on the device discovery within the kernel, nothing else.

Quoted 4 lines

Quoted 1 line

So on bootup this would mean a lot of discovery.

Yes, all of this happens within the kernel, like normal. What are you getting at?

I am getting at boot delay. A lot of discovery means a lot of delay. A lot of delay means people will stick with static /dev.

Quoted 5 lines

Quoted 4 lines

I think we could get some big speedup, by just dumping the possible non-realized device list on bootup, and then just refine it on physical access. This could make devtmpfs an acceptable replacement to static /dev.

Quoted 4 lines

Um, that's exactly what devtmpfs does, it creates the nodes based on the fact that the devices were physically (or virtually for some devices) discovered and registered with the kernel. This happens at the same time the existing uevents are generated and sent out to userspace.

The question is, how fast can devtmpfs get the device list from the kernel on bootup? How much faster than udev? How much slower than static /dev?

It's much faster than udev, and is equivalent to a static /dev with the exception that the group and permission settings that you are used to. udev then needs to come along and make those settings, but that's so frickin fast it's amazing.

So, test it out and see if you want to verify this.

Quoted 4 lines · Greg KH

It's much faster than udev, and is equivalent to a static /dev with the exception that the group and permission settings that you are used to. udev then needs to come along and make those settings, but that's so frickin fast it's amazing.

Sounds great!

Quoted 1 line

So, test it out and see if you want to verify this.

Ok, can you post a patch against .30?

Quoted 2 lines · Al Boldi

The question is, how fast can devtmpfs get the device list from the kernel on bootup? How much faster than udev? How much slower than static /dev?

It's much faster than udev, and is equivalent to a static /dev with the exception that the group and permission settings that you are used to. udev then needs to come along and make those settings, but that's so frickin fast it's amazing.

Quoted 5 lines · Greg KH

Quoted 2 lines · Al Boldi

The question is, how fast can devtmpfs get the device list from the kernel on bootup?  How much faster than udev?  How much slower than static /dev?

Quoted 4 lines

It's much faster than udev, and is equivalent to a static /dev with the exception that the group and permission settings that you are used to. udev then needs to come along and make those settings, but that's so frickin fast it's amazing.

Earlier in the thread you indicated a 0.5sec speedup over udev.  Is that really considered "much faster"?

I do agree that it makes sense to do this, but more from an elegance view than a performance one.

Quoted 9 lines · Greg KH

Quoted 3 lines · Al Boldi

The question is, how fast can devtmpfs get the device list from the kernel on bootup? How much faster than udev? How much slower than static /dev?

It's much faster than udev, and is equivalent to a static /dev with the exception that the group and permission settings that you are used to. udev then needs to come along and make those settings, but that's so frickin fast it's amazing.

Earlier in the thread you indicated a 0.5sec speedup over udev. Is that really considered "much faster"?

I do agree that it makes sense to do this, but more from an elegance view than a performance one.

Quoted 11 lines · Chris Friesen

Quoted 9 lines · Greg KH

Quoted 3 lines · Al Boldi

The question is, how fast can devtmpfs get the device list from the kernel on bootup? How much faster than udev? How much slower than static /dev?

It's much faster than udev, and is equivalent to a static /dev with the exception that the group and permission settings that you are used to. udev then needs to come along and make those settings, but that's so frickin fast it's amazing.

Quoted 5 lines

Earlier in the thread you indicated a 0.5sec speedup over udev. Is that really considered "much faster"?

I do agree that it makes sense to do this, but more from an elegance view than a performance one.

It's definitely more elegant, but at what cost?

For devtmpfs to be a realistic replacement for static /dev, it has to be comparable to static /dev in both speed and size.

WRT speed, there should be no slowdown and it should be just as fast as a "tar -xp < dev.tar".

WRT size, it should not be dependent on hotplug, and instead offer hotplug as an option.

Quoted 2 lines · Al Boldi

For devtmpfs to be a realistic replacement for static /dev, it has to be comparable to static /dev in both speed and size.

Since when is this requirement necessary? You want something for free in both speed and size? Well, you got it in speed, but not size, it will take up memory that is swapable, and a tiny ammount of non-swapable kernel memory for the code.

Quoted 2 lines

WRT speed, there should be no slowdown and it should be just as fast as a "tar -xp < dev.tar".

Again, where is this requirement coming from?

Have you timed devtmpfs?

Quoted 2 lines

WRT size, it should not be dependent on hotplug, and instead offer hotplug as an option.

Um, again, who made up such a requirement? Are you running systems today with CONFIG_HOTPLUG disabled? If so, how well is that working for you?

Quoted 8 lines · Greg KH

Quoted 2 lines · Al Boldi

For devtmpfs to be a realistic replacement for static /dev, it has to be comparable to static /dev in both speed and size.

Since when is this requirement necessary? You want something for free in both speed and size? Well, you got it in speed, but not size, it will take up memory that is swapable, and a tiny ammount of non-swapable kernel memory for the code.

Not so tiny when you count in the hotplug dependency.

Quoted 6 lines

Quoted 2 lines

WRT speed, there should be no slowdown and it should be just as fast as a "tar -xp < dev.tar".

Again, where is this requirement coming from?

Have you timed devtmpfs?

Not yet, I am still waiting for the latest patch against .30.

Quoted 6 lines

Quoted 2 lines

WRT size, it should not be dependent on hotplug, and instead offer hotplug as an option.

Um, again, who made up such a requirement? Are you running systems today with CONFIG_HOTPLUG disabled? If so, how well is that working for you?

It's working out quite well. I don't like hotplug, it's too slow. I always turn it and module-autoloading off, to achieve a much more responsive system.

Quoted 5 lines · Al Boldi

Quoted 4 lines · Greg KH

Quoted 2 lines · Al Boldi

For devtmpfs to be a realistic replacement for static /dev, it has to be comparable to static /dev in both speed and size.

Quoted 6 lines

Quoted 4 lines

Since when is this requirement necessary? You want something for free in both speed and size? Well, you got it in speed, but not size, it will take up memory that is swapable, and a tiny ammount of non-swapable kernel memory for the code.

Not so tiny when you count in the hotplug dependency.

devtmpfs does not rely on hotplug at all.

Quoted 11 lines · Greg KH

Quoted 10 lines · Al Boldi

Quoted 8 lines · Greg KH

Quoted 2 lines · Al Boldi

For devtmpfs to be a realistic replacement for static /dev, it has to be comparable to static /dev in both speed and size.

Since when is this requirement necessary? You want something for free in both speed and size? Well, you got it in speed, but not size, it will take up memory that is swapable, and a tiny ammount of non-swapable kernel memory for the code.

Quoted 3 lines

Quoted 1 line

Not so tiny when you count in the hotplug dependency.

devtmpfs does not rely on hotplug at all.

Are you sure?

This is from the patch of this thread:

Quoted 3 lines

(quoted diff) +3 -0

+config DEVTMPFS
+       bool "Create a kernel maintained /dev tmpfs (EXPERIMENTAL)"
+       depends on HOTPLUG && SHMEM && TMPFS

Quoted 12 lines · Al Boldi

Quoted 11 lines · Greg KH

Quoted 10 lines · Al Boldi

Quoted 8 lines · Greg KH

Quoted 2 lines · Al Boldi

For devtmpfs to be a realistic replacement for static /dev, it has to be comparable to static /dev in both speed and size.

Since when is this requirement necessary? You want something for free in both speed and size? Well, you got it in speed, but not size, it will take up memory that is swapable, and a tiny ammount of non-swapable kernel memory for the code.

Quoted 10 lines

Quoted 3 lines

Quoted 1 line

Not so tiny when you count in the hotplug dependency.

devtmpfs does not rely on hotplug at all.

Are you sure?

This is from the patch of this thread:

Quoted 3 lines

(quoted diff) +3 -0

+config DEVTMPFS
+       bool "Create a kernel maintained /dev tmpfs (EXPERIMENTAL)"
+       depends on HOTPLUG && SHMEM && TMPFS

Ah, you're right.

Ok then, since you don't run CONFIG_HOTPLUG kernels, why are you worried about this code at all?

confused,

Quoted 29 lines · Greg KH

Quoted 12 lines · Al Boldi

Quoted 11 lines · Greg KH

Quoted 10 lines · Al Boldi

Quoted 8 lines · Greg KH

Quoted 2 lines · Al Boldi

For devtmpfs to be a realistic replacement for static /dev, it has to be comparable to static /dev in both speed and size.

Since when is this requirement necessary? You want something for free in both speed and size? Well, you got it in speed, but not size, it will take up memory that is swapable, and a tiny ammount of non-swapable kernel memory for the code.

Quoted 10 lines

Quoted 3 lines

Quoted 1 line

Not so tiny when you count in the hotplug dependency.

devtmpfs does not rely on hotplug at all.

Are you sure?

This is from the patch of this thread:

Quoted 3 lines

(quoted diff) +3 -0

+config DEVTMPFS
+       bool "Create a kernel maintained /dev tmpfs (EXPERIMENTAL)"
+       depends on HOTPLUG && SHMEM && TMPFS

Ah, you're right.

Ok then, since you don't run CONFIG_HOTPLUG kernels, why are you worried about this code at all?

Quoted 1 line

confused,

Linux used to be lean and mean which made it fun to work with, and which made the switch from the competition easy. Nowadays I see a lot of bloat going into the kernel which may indicate that Linux is starting to run out of steam.

devtmpfs seems bloaty due to the hotplug dependency.

Quoted 4 lines · Al Boldi

Linux used to be lean and mean which made it fun to work with, and which made the switch from the competition easy. Nowadays I see a lot of bloat going into the kernel which may indicate that Linux is starting to run out of steam.

That doesn't seem to make sense, if more development is happening, and our number of contributors, different companies, and rate of change is increasing, how are we "running out of steam"?

What specific development number is proof of us slowing down? I see nothing but the exact, and extreme, opposite thing happening.

Quoted 1 line

devtmpfs seems bloaty due to the hotplug dependency.

Would you use it if we fix it to remove this dependancy?

So what really is your objection here? That we did not let devtmpfs work within a CONFIG_HOTPLUG=n type system?

Or that devtmpfs works with
the existing CONFIG_HOTPLUG=y systems (i.e. 99% of the world)?

Quoted 9 lines · Greg KH

Quoted 4 lines · Al Boldi

Linux used to be lean and mean which made it fun to work with, and which made the switch from the competition easy. Nowadays I see a lot of bloat going into the kernel which may indicate that Linux is starting to run out of steam.

That doesn't seem to make sense, if more development is happening, and our number of contributors, different companies, and rate of change is increasing, how are we "running out of steam"?

Running out of "quality steam". Quantity can never replace Quality.

Quoted 2 lines

What specific development number is proof of us slowing down? I see nothing but the exact, and extreme, opposite thing happening.

The increasing number of regressions are probably cause for concern.

Quoted 3 lines

Quoted 1 line

devtmpfs seems bloaty due to the hotplug dependency.

Would you use it if we fix it to remove this dependancy?

Yes, if it also compares to static /dev in terms of speed.

Quoted 2 lines

So what really is your objection here? That we did not let devtmpfs work within a CONFIG_HOTPLUG=n type system?

Yes.

Quoted 2 lines

Or that devtmpfs works with
the existing CONFIG_HOTPLUG=y systems (i.e. 99% of the world)?

And that should stay an option.

Again, I don't really believe that you use a CONFIG_HOTPLUG=n system in the real world, so barring a patch from you or anyone else, I'll leave it as-is.

Quoted 3 lines · Greg KH

Again, I don't really believe that you use a CONFIG_HOTPLUG=n system in the real world, so barring a patch from you or anyone else, I'll leave it as-is.

That seems a bit harsh. There are lots of embedded devices that don't need hotplug.

You seem to have missed the previous statements from Al about this topic which resulted in him ignoring my previous questions about this topic, and hence, this statement from me.

Maybe they are using the same trick as Ubuntu and Debian, as they run udev in the background to hide the slowness. Both Fedora and Mandriva run udev in the foreground where the slowness is visible.

Quoted 3 lines · Al Boldi

Maybe they are using the same trick as Ubuntu and Debian, as they run udev in the background to hide the slowness. Both Fedora and Mandriva run udev in the foreground where the slowness is visible.

Ubuntu does not run udev in the background.

Quoted 5 lines · Scott James Remnant

Quoted 3 lines · Al Boldi

Maybe they are using the same trick as Ubuntu and Debian, as they run udev in the background to hide the slowness. Both Fedora and Mandriva run udev in the foreground where the slowness is visible.

Quoted 1 line

Ubuntu does not run udev in the background.

I put a date timestamp at the beginning and end of the start command of init.d/udev, and when you stop and then start init.d/udev it shows a ~5s delay. At boot I don't see this delay, but instead see the initial date stamp, then a flurry of parallel activity, then the final date stamp, which shows the ~5s delay.

Quoted 17 lines · Greg KH

Quoted 16 lines · Al Boldi

Quoted 9 lines · Andi Kleen

Quoted 4 lines · Greg KH

It makes the userspace boot process much simpler and easier to maintain, as well as providing a way to handle rescue disks and images trivially, and it makes the kernel _less_ dependant on the early userspace bootup scripts.

As a initrd less kernel user I can really only agree: getting rid of the udev-in-initrd requirement would be a big step forward in usability. Typically I always have to pre populate
a on disk /dev manually first to get my kernels to boot.

Oh good, I thought I was the only one doing that.

The reason I don't like udev is that it's just to slow; something like a 5-10s delay on each boot. No idea why it should be so slow, but it's probably probing the kernel for all available devices at boot, when it could be much quicker by probing for the device on access.

Quoted 3 lines

Like Kay stated, this sounds like a misconfiguration of your distro's udev setup, as the ones I use (openSUSE and Gentoo) do not have this problem at all.

Maybe they are using the same trick as Ubuntu and Debian, as they run udev in the background to hide the slowness. Both Fedora and Mandriva run udev in the foreground where the slowness is visible.

So really, if devtmpfs compares to udev speeds then this just looks like a devfs comeback. Remember, devfs was really slow.

Thanks!

--
Al

Quoted 2 lines

and Arjan (who is doing about the fastest Linux boot anyone has and has better data than anyone else)

We did this work based on Arjan's work. It solves a problem that Moblin has as well, which is why Novell is using it for their Moblin-based images they are shipping. It makes things go faster, with a simpler userspace codepath, and we have the numbers as have been previously posted.

Quoted 4 lines

plus

Quoted 2 lines

Eric has shown that just making the nodes is 0.06 seconds with todays sysfs interface, and there is room for improvement,

A 0.06 speedup is still a speedup, right? :)

Seriously, we saw much more than that, about 0.5 what I last remember, Kay?

Here's the devtmpfs patch again. For .32 it's a simple and clean patch. It's been tested and agreed by three major distros that this is a good idea. SuSE has been shipping this in their kernels for a while now with no problems, and actual speedups measured on their boot times. Gentoo also has been testing it, but we haven't gotten a tested-by: line from them yet, hopefully that will happen soon.

A number of embedded distros have also privately said they would be using this patch, I really don't understand why they don't publically want to state this, but oh well...

It's been in linux-next for many months now, with no reported regressions at all as well.

thanks,

greg k-h

--------------

From: Kay Sievers <[email protected]>

Devtmpfs lets the kernel create a tmpfs instance called devtmpfs very early at kernel initialization, before any driver-core device is registered. Every device with a major/minor will provide a device node in devtmpfs.

Devtmpfs can be changed and altered by userspace at any time, and in any way needed - just like today's udev-mounted tmpfs. Unmodified udev versions will run just fine on top of it, and will recognize an already existing kernel-created device node and use it. The default node permissions are root:root 0600. Proper permissions and user/group ownership, meaningful symlinks, all other policy still needs to be applied by userspace.

Quoted 35 lines · Greg KH

Here's the devtmpfs patch again. For .32 it's a simple and clean patch. It's been tested and agreed by three major distros that this is a good idea. SuSE has been shipping this in their kernels for a while now with no problems, and actual speedups measured on their boot times. Gentoo also has been testing it, but we haven't gotten a tested-by: line from them yet, hopefully that will happen soon.

A number of embedded distros have also privately said they would be using this patch, I really don't understand why they don't publically want to state this, but oh well...

It's been in linux-next for many months now, with no reported regressions at all as well.

thanks,

greg k-h

--------------

From: Kay Sievers <[email protected]>

Devtmpfs lets the kernel create a tmpfs instance called devtmpfs very early at kernel initialization, before any driver-core device is registered. Every device with a major/minor will provide a device node in devtmpfs.

Devtmpfs can be changed and altered by userspace at any time, and in any way needed - just like today's udev-mounted tmpfs. Unmodified udev versions will run just fine on top of it, and will recognize an already existing kernel-created device node and use it. The default node permissions are root:root 0600. Proper permissions and user/group ownership, meaningful symlinks, all other policy still needs to be applied by userspace.

drivers/base/devtmpfs.c line 104

Quoted 10 lines

drivers/base/devtmpfs.c new file +10 -0

+	err = vfs_path_lookup(dev_mnt->mnt_root, dev_mnt,
+			      path, LOOKUP_PARENT, &nd);
+	if (err == 0) {
+		struct dentry *dentry;
+
+		/* create directory right away */
+		dentry = lookup_create(&nd, 1);
+		if (!IS_ERR(dentry)) {
+			err = vfs_mkdir(nd.path.dentry->d_inode,
+					dentry, 0775);

Is there a typo here? I think the mode should be 0755. 0755 is used
below, and that does fit better with the 0600 mode for device nodes.

Quoted 5 lines · Alan Jenkins

Quoted 2 lines

+                   err = vfs_mkdir(nd.path.dentry->d_inode,
+                                   dentry, 0775);

Is there a typo here?  I think the mode should be 0755.   0755 is used below, and that does fit better with the 0600 mode for device nodes.

Yeah, I think you are correct.  Kay, any objection to me making this change?

Quoted 5 lines · Greg KH

Quoted 3 lines · Alan Jenkins

Quoted 2 lines

+                   err = vfs_mkdir(nd.path.dentry->d_inode,
+                                   dentry, 0775);

Quoted 5 lines

Quoted 2 lines

Is there a typo here?  I think the mode should be 0755.   0755 is used below, and that does fit better with the 0600 mode for device nodes.

Yeah, I think you are correct.  Kay, any objection to me making this change?

No, sure no objection. It sounds right to use 0755. I was just waiting if more things to fix come up. :)

Hi!

drivers/base/devtmpfs.c line 5

Quoted 8 lines

drivers/base/devtmpfs.c new file +5 -0

@@ -0,0 +1,367 @@
+/*
+ * devtmpfs - kernel-maintained tmpfs-based /dev
+ *
+ * Copyright (C) 2009, Kay Sievers <[email protected]>
+ *

I'd expect GPL notice here...

drivers/base/devtmpfs.c line 31

Quoted 5 lines

drivers/base/devtmpfs.c new file +5 -0

+#if defined CONFIG_DEVTMPFS_MOUNT
+static int dev_mount = 1;
+#else
+static int dev_mount;
+#endif

A bit too many config options?

drivers/base/devtmpfs.c line 58

Quoted 7 lines

drivers/base/devtmpfs.c new file +7 -0

+#ifdef CONFIG_BLOCK
+static inline int is_blockdev(struct device *dev)
+{
+	return dev->class == &block_class;
+}
+#else
+static inline int is_blockdev(struct device *dev) { return 0; }

Should this go to header somewhere?

Quoted 12 lines · Pavel Machek

Hi!

drivers/base/devtmpfs.c line 5

Quoted 8 lines

drivers/base/devtmpfs.c new file +5 -0

@@ -0,0 +1,367 @@
+/*
+ * devtmpfs - kernel-maintained tmpfs-based /dev
+ *
+ * Copyright (C) 2009, Kay Sievers <[email protected]>
+ *

I'd expect GPL notice here...

But it's not needed, the overall license of the kernel covers it.

Quoted 8 lines

drivers/base/devtmpfs.c line 31

Quoted 5 lines

drivers/base/devtmpfs.c new file +5 -0

+#if defined CONFIG_DEVTMPFS_MOUNT
+static int dev_mount = 1;
+#else
+static int dev_mount;
+#endif

A bit too many config options?

It's only 1 option, is that 1 too many for you?

Quoted 9 lines

Quoted 8 lines

drivers/base/devtmpfs.c line 31

Quoted 5 lines

drivers/base/devtmpfs.c new file +5 -0

+#if defined CONFIG_DEVTMPFS_MOUNT
+static int dev_mount = 1;
+#else
+static int dev_mount;
+#endif

A bit too many config options?

Quoted 1 line

It's only 1 option, is that 1 too many for you?

It is two, and this option duplicates  commandline option. Maybe its better to use config_cmdline at that point?

Here's the devtmpfs patch again.

Quoted 1 line · Greg KH

Here's the devtmpfs patch again.

As an embedded distro maintainer, I'd really like such an approach. With ptxdist, we can today build the complete rootfs for an embedded system without root permissions - just if we want to nfs-root boot the boards during development, it doesn't work, because you have to create /dev/console, /dev/null and /dev/zero and that works only for root.

On slow embedded systems (i.e. ARM 200 MHz), note that boot times are often much slower than on full-blown x86. So any speed improvement is welcome here as well.

Quoted 1 line · Greg KH

Here's the devtmpfs patch again.

As an embedded distro maintainer, I'd really like such an approach. With ptxdist, we can today build the complete rootfs for an embedded system without root permissions - just if we want to nfs-root boot the boards during development, it doesn't work, because you have to create /dev/console, /dev/null and /dev/zero and that works only for root.

Quoted 3 lines · Robert Schwebel

Quoted 1 line · Greg KH

Here's the devtmpfs patch again.

Quoted 5 lines

As an embedded distro maintainer, I'd really like such an approach. With ptxdist, we can today build the complete rootfs for an embedded system without root permissions - just if we want to nfs-root boot the boards during development, it doesn't work, because you have to create /dev/console, /dev/null and /dev/zero and that works only for root.

Have you tried genext2fs ? It lets you create a filesystem image complete with device nodes as a normal user.

Here's the devtmpfs patch again.  For .32 it's a simple and clean patch.

2009/8/6 Greg KH <[email protected]>:

Quoted 1 line

Here's the devtmpfs patch again.  For .32 it's a simple and clean patch.

I also prefer devtmpfs.

Base on devtmpfs, we can remove polling for being ready of some asynchronous root device(usb storage, mmc, ....) easily before mounting rootfs.

      http://marc.info/?l=linux-kernel&m=124955163908015&w=2
      http://marc.info/?l=linux-kernel&m=124955166508077&w=2