Where is /dev/console created when using devtmpfs?
I'm currently running with devtmpfs only but am looking into using mdev so I spent some time looking at my device usage and for the life of me I can't find where /dev/console gets created during the build process. Looking at the root filesystem when it's not running shows this in /dev
drwxrwxr-x 2 root root 4096 Mar 1 15:26 pts
lrwxrwxrwx 1 root root 10 Mar 1 15:26 log -> ../tmp/log
crw--w--w- 1 root root 5, 1 Mar 22 21:52 console
and I see that pts and log come from the skeleton but I can't find console. I'm running an x86_64 system using the default vga console if that matters. I'm obviously missing something but I can't find a mknod or mkdevs for it.
Where is /dev/console created when using devtmpfs?
Thanks,
Steve
Quoted 1 line · Steve Kenton
I'm currently running with devtmpfs only but am looking into using mdev
As the original author of mdev and the guy who did:
http://lkml.iu.edu/hypermail/linux/kernel/1306.3/04204.html
I should probably poke my nose in here. :)
Quoted 2 lines
so I spent some time looking at my device usage and for the life of me I can't find where
/dev/consolegets created during the build process.
The console PID 1 inherits at process launch is the /dev/console in initramfs. When you don't have initramfs, the init/noinitramfs.c code literally does:
err = sys_mkdir((const char __user __force *) "/dev", 0755);
if (err < 0)
goto out;
err = sys_mknod((const char __user __force *) "/dev/console",
S_IFCHR | S_IRUSR | S_IWUSR,
new_encode_dev(MKDEV(5, 1)));And then init/main.c does:
/* Open the /dev/console on the rootfs, this should never fail */
if (sys_open((const char __user *) "/dev/console", O_RDWR, 0) < 0)
printk(KERN_WARNING "Warning: unable to open aninitial console.\n");
(void) sys_dup(0);
(void) sys_dup(0);The devtmpfs mount happens later in the boot, and in fact I have a pending todo item to submit a patch to make the kernel DEVTMPFS_MOUNT config option actually _apply_ to initramfs. (Last I checked it would only mount devtmpfs on /dev in the fallback root that root= switches you to when you didn't have an executable /init in initramfs. I should make it so that if it's about to exec /init out of rootfs, it does the above mknod /dev trick (ignore failure if it's already there) and then mounts devtmpfs on it (and if _that_ fails some crazy person probably made a file called /dev and they get to keep the pieces).
Quoted 7 lines
Looking at the root filesystem when it's not running shows this in /dev
drwxrwxr-x 2 root root 4096 Mar 1 15:26 pts
lrwxrwxrwx 1 root root 10 Mar 1 15:26 log -> ../tmp/log
crw--w--w- 1 root root 5, 1 Mar 22 21:52 consoleand I see that pts and log come from the skeleton but I can't find console.
Possibly I'm answering the wrong question.
devtmpfs is a synthetic filesystem, like /proc or /sys. You're not talking to any sort of backing store when you mount it, instead the driver is hallucinating contents for you to read and write from on the fly, and when you interact with it you talk to functions in the driver that send/receive some data they just made up to humor your request.
I wrote a thing about the four different types of filesystems a while back:
http://landley.livejournal.com/52326.html
Quoted 4 lines
I'm running an
x86_64system using the default vga console if that matters. I'm obviously missing something but I can't find a mknod or mkdevs for it.Where is
/dev/consolecreated when using devtmpfs?
The same place /proc/mounts is created on procfs?
The name "console" comes from drivers/tty/tty_io.c:
consdev = device_create(tty_class, NULL, MKDEV(TTYAUX_MAJOR, 1), NULL,
"console");
That sticks an entry into some kernel linked list (or possibly tree), which is interpreted by devtmpfs to give you a console device. It's also interpreted by sysfs to give you a "console" directory under /sys/class/tty. Both filesystem drivers are parsing the same kernel internal data to produce different types of output. They do so on the fly, when you open the relevant directory and list its contents.
Quoted 3 lines
Thanks,
Steve
Rob
Quoted 8 lines · Rob Landley
Quoted 1 line · Steve Kenton
I'm currently running with devtmpfs only but am looking into using mdev
As the original author of mdev and the guy who did:
http://lkml.iu.edu/hypermail/linux/kernel/1306.3/04204.html
I should probably poke my nose in here. :)
Thank you very much for taking the time!
Quoted 7 lines
Quoted 3 lines
so I spent some time looking at my device usage and for the life of me I can't find where
/dev/consolegets created during the build process.The console PID 1 inherits at process launch is the
/dev/consolein initramfs. When you don't have initramfs, theinit/noinitramfs.ccode literally does:
Yes, this is my configuration - initramfs not linked into kernel by buildroot # BR2_TARGET_ROOTFS_INITRAMFS is not set
Quoted 9 lines
err = sys_mkdir((const char __user __force *) "/dev", 0755);
if (err < 0)
goto out;
err = sys_mknod((const char __user __force *) "/dev/console",
S_IFCHR | S_IRUSR | S_IWUSR,
new_encode_dev(MKDEV(5, 1)));Quoted 6 lines
And then
init/main.cdoes:/* Open the /dev/console on the rootfs, this should never fail */ if (sys_open((const char __user *) "/dev/console", O_RDWR, 0) < 0) printk(KERN_WARNING "Warning: unable to open aninitial console.\n");
If I understand correctly all of which occurs on rootfs in ram and does not touch whatever disk based filesystem may eventually get mounted on /?
Quoted 9 lines
(void) sys_dup(0); (void) sys_dup(0);The devtmpfs mount happens later in the boot, and in fact I have a pending todo item to submit a patch to make the kernel
DEVTMPFS_MOUNTconfig option actually _apply_ to initramfs. (Last I checked it would only mount devtmpfs on /dev in the fallback root that root= switches you to when you didn't have an executable /init in initramfs. I should
Apparently still the case, at least buildroot 2015.02 supplied this as /init:
#!/bin/sh
# devtmpfs does not get automounted for initramfs /bin/mount -t devtmpfs devtmpfs /dev exec 0</dev/console exec 1>/dev/console exec 2>/dev/console exec /sbin/init $*
Quoted 4 lines
make it so that if it's about to exec /init out of rootfs, it does the above mknod /dev trick (ignore failure if it's already there) and then mounts devtmpfs on it (and if _that_ fails some crazy person probably made a file called /dev and they get to keep the pieces).
grin
Quoted 10 lines
Quoted 8 lines
Looking at the root filesystem when it's not running shows this in /dev
drwxrwxr-x 2 root root 4096 Mar 1 15:26 pts
lrwxrwxrwx 1 root root 10 Mar 1 15:26 log -> ../tmp/log
crw--w--w- 1 root root 5, 1 Mar 22 21:52 consoleand I see that pts and log come from the skeleton but I can't find console.
Possibly I'm answering the wrong question.
Since I'm trying to understand what's going on it's a very good answer and I've read it multiple time while perusing the kernel code, but yes I was originally wondering where /dev/console was created on the target filesystem as part of the buildroot make process.
Quoted 6 lines
devtmpfs is a synthetic filesystem, like /proc or /sys. You're not talking to any sort of backing store when you mount it, instead the driver is hallucinating contents for you to read and write from on the fly, and when you interact with it you talk to functions in the driver that send/receive some data they just made up to humor your request.
Again if I under stand correctly the contents of the on-disk /dev on the root filesystem in my case (initramfs not linked into kernel) are unused and normally inaccessible because of devtmpfs mount on top?
Quoted 4 lines
I wrote a thing about the four different types of filesystems a while back:
Another very good read - thanks for that too
Quoted 5 lines
Quoted 5 lines
I'm running an
x86_64system using the default vga console if that matters. I'm obviously missing something but I can't find a mknod or mkdevs for it.Where is
/dev/consolecreated when using devtmpfs?
I was thinking in terms of the buildroot "use devtmpfs only" as opposed with mdev, eudev, or systemd
Quoted 7 lines
The same place
/proc/mountsis created on procfs?The name "console" comes from
drivers/tty/tty_io.c:consdev = device_create(tty_class, NULL,
MKDEV(TTYAUX_MAJOR, 1), NULL,
"console");
OK
Quoted 7 lines
That sticks an entry into some kernel linked list (or possibly tree), which is interpreted by devtmpfs to give you a console device. It's also interpreted by sysfs to give you a "console" directory under
/sys/class/tty. Both filesystem drivers are parsing the same kernel internal data to produce different types of output. They do so on the fly, when you open the relevant directory and list its contents.
Thanks again for your time and detailed explaination.
Steve
Quoted 7 lines
Quoted 4 lines
Thanks,
Steve
Rob
Quoted 2 lines · Steve Kenton
Quoted 1 line · Rob Landley
On Wed, Mar 25, 2015 at 12:44 PM, Steve Kenton <[email protected]> wrote:
[snip]
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
so I spent some time looking at my device usage and for the life of me I can't find where
/dev/consolegets created during the build process.The console PID 1 inherits at process launch is the
/dev/consolein initramfs. When you don't have initramfs, theinit/noinitramfs.ccode literally does:Yes, this is my configuration - initramfs not linked into kernel by buildroot #
BR2_TARGET_ROOTFS_INITRAMFSis not set
Just to be clear "initramfs not linked into kernel" is BR2_TARGET_ROOTFS_CPIO,
"initramfs linked into the kernel" is BR2_TARGET_ROOTFS_INITRAMFS. But for the purpose of the contents of the rootfs, that doesn't make a difference.
But so in your case, Steve, I believe you _do_ have an initramfs, i.e. you have
selected BR2_TARGET_ROOTFS_CPIO. Otherwise /dev/console wouldn't have been created :-).
Quoted 9 lines
Quoted 9 lines
err = sys_mkdir((const char __user __force *) "/dev", 0755); if (err < 0) goto out; err = sys_mknod((const char __user __force *) "/dev/console", S_IFCHR | S_IRUSR | S_IWUSR, new_encode_dev(MKDEV(5, 1)));
Note again that this stuff is only done when there is no initramfs. In other
words, when there is an initramfs, this initramfs must contain /dev/console or you have no console.
That's why buildroot adds it explicitly in fs/cpio/cpio.mk:
PACKAGES_PERMISSIONS_TABLE += /dev/console c 622 0 0 5 1 - - -$(sep)
Quoted 9 lines
Quoted 7 lines
And then
init/main.cdoes:/* Open the /dev/console on the rootfs, this should never fail */ if (sys_open((const char __user *) "/dev/console", O_RDWR, 0) < 0) printk(KERN_WARNING "Warning: unable to open aninitial console.\n");
If I understand correctly all of which occurs on rootfs in ram and does not touch whatever disk based filesystem may eventually get mounted on /?
Indeed.
[snip]
Quoted 9 lines
Quoted 6 lines
devtmpfs is a synthetic filesystem, like /proc or /sys. You're not talking to any sort of backing store when you mount it, instead the driver is hallucinating contents for you to read and write from on the fly, and when you interact with it you talk to functions in the driver that send/receive some data they just made up to humor your request.
Again if I under stand correctly the contents of the on-disk /dev on the root filesystem in my case (initramfs not linked into kernel) are unused and normally inaccessible because of devtmpfs mount on top?
Indeed, its contents becomes inaccessible as soon as devtmpfs is mounted on
top. In case of initramfs, that is when /init is executed and it is mounted explicitly; in case of non-initramfs, it is mounted just after / is mounted and before /sbin/init is executed (Rob, correct me if I'm wrong).
So with initramfs, you will have two versions of /dev/console: one real
mknod'ed one in the initramfs itself, and a virtual one in the devtmpfs. After devtmpfs is mounted, the one in initramfs is no longer accessible.
Quoted 8 lines
Quoted 5 lines
I wrote a thing about the four different types of filesystems a while back:
Another very good read - thanks for that too
Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
I'm running an
x86_64system using the default vga console if that matters. I'm obviously missing something but I can't find a mknod or mkdevs for it.Where is
/dev/consolecreated when using devtmpfs?I was thinking in terms of the buildroot "use devtmpfs only" as opposed with mdev, eudev, or systemd
Actually, all four of these use devtmpfs. The only thing that mdev, eudev, and
systemd-udev add is the possibility to do more processing (e.g. creating symlinks) when the device is created. But the kernel itself will always create the device node in devtmpfs.
Regards,
Arnout
[snip]
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
Quoted 13 lines · Arnout Vandecappelle
Quoted 2 lines · Steve Kenton
Quoted 1 line · Rob Landley
On Wed, Mar 25, 2015 at 12:44 PM, Steve Kenton <[email protected]> wrote:
[snip]
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
so I spent some time looking at my device usage and for the life of me I can't find where
/dev/consolegets created during the build process.The console PID 1 inherits at process launch is the
/dev/consolein initramfs. When you don't have initramfs, theinit/noinitramfs.ccode literally does:Yes, this is my configuration - initramfs not linked into kernel by buildroot #
BR2_TARGET_ROOTFS_INITRAMFSis not set
Quoted 7 lines
Just to be clear "initramfs not linked into kernel" is
BR2_TARGET_ROOTFS_CPIO,
"initramfs linked into the kernel" isBR2_TARGET_ROOTFS_INITRAMFS. But for the purpose of the contents of the rootfs, that doesn't make a difference.But so in your case, Steve, I believe you _do_ have an initramfs, i.e. you have
selectedBR2_TARGET_ROOTFS_CPIO. Otherwise/dev/consolewouldn't have been created :-).
I've been known to be confused once or twice :-)
Yes, BR2_TARGET_ROOTFS_CPIO was still explicitly set in my config from previous testing and experimentation with initramfs even though BR2_TARGET_ROOTFS_INITRAMFS was not set. Since BR2_TARGET_ROOTFS_INITRAMFS selects BR2_TARGET_ROOTFS_CPIO I must have been confused :-( So even though I am currently using the tar version of the rootfs there is indeed also a cpio version getting created and that process is where /dev/console comes from in my on-disk rootfs. It should disappear when I clear the unneeded BR2_TARGET_ROOTFS_CPIO.
.../buildroot-2015.02/output/images$ ls -l
total 548980
-rw-rw-r-- 1 steve steve 5896640 Mar 26 22:44 bzImage
-rw-rw-r-- 1 steve steve 139560 Mar 22 19:59 grub.img
-rw-rw-r-- 1 steve steve 183162368 Mar 26 22:44 rootfs.cpio -rw-rw-r-- 1 steve steve 213949440 Mar 26 22:44 rootfs.ext2
lrwxrwxrwx 1 steve steve 11 Mar 26 22:44 rootfs.ext4 -> rootfs.ext2
-rw-rw-r-- 1 steve steve 185610240 Mar 26 22:44 rootfs.tar
drwxr-xr-x 2 steve steve 4096 Mar 22 20:01 syslinux
Oops!
.../dev$ ls -l
total 4
crw--w--w- 1 root root 5, 1 Mar 25 22:54 console
lrwxrwxrwx 1 root root 10 Mar 1 15:26 log -> ../tmp/log
drwxrwxr-x 2 root root 4096 Mar 1 15:26 pts
And it makes sense now.
Quoted 11 lines
Quoted 10 lines
Quoted 10 lines
err = sys_mkdir((const char __user __force *) "/dev", 0755); if (err < 0) goto out; err = sys_mknod((const char __user __force *) "/dev/console", S_IFCHR | S_IRUSR | S_IWUSR, new_encode_dev(MKDEV(5, 1)));
Quoted 7 lines
Note again that this stuff is only done when there is no initramfs. In other
words, when there is an initramfs, this initramfs must contain/dev/consoleor you have no console.That's why buildroot adds it explicitly in
fs/cpio/cpio.mk:
PACKAGES_PERMISSIONS_TABLE+=/dev/consolec 622 0 0 5 1 - - -$(sep)
Ahhh! and that answers my original question of where the on-disk /dev/console come from.
Thanks everyone for the information.
Rob, my 2 cents worth says that todo item of yours is a good idea.
Steve
[snip]
[Buildroot] Something odd about /dev/console creation
In reply to Arnout VandecappelleIt looks like there is something odd going on with the creation of /dev/console There also seems to be some unintended mixing of creation of the cpio filesystem vs installing it as an initramfs as controlled by these settings.
BR2_TARGET_ROOTFS_CPIOBR2_TARGET_ROOTFS_INITRAMFS
Which can result in /init being created and populated with the script for an initramfs boot when in fact it is not needed if BR2_TARGET_ROOTFS_CPIO os set and BR2_TARGET_ROOTFS_INITRAMFS is not. It does not seem to hurt anything but is odd to see.
# Anyway, back to /dev/console apparently always being created in the on-disk filesystem. # As far as I can tell this line always gets executed during any buildroot make
# independent of the above settings for actually creating a cpio filesystem or initramfs ./fs/cpio/cpio.mk:PACKAGES_PERMISSIONS_TABLE += /dev/console c 622 0 0 5 1 - - -$(sep)
# Which means that this line:./fs/common.mk: printf '$$(subst $$(sep),\n,$$(PACKAGES_PERMISSIONS_TABLE))' >> $$(FULL_DEVICE_TABLE)
# will always add /dev/console to _device_table.txt for creation during the build (example from my usual build) printf ' /bin/busybox f 4755 0 0 - - - - -\n /usr/share/udhcpc/default.script f 755 0 0 - - - - -\n \ # split for email readability /dev/console c 622 0 0 5 1 - - -\n' >> /data/bcdist/buildroot-2015.02/output/build/_device_table.txt
# I got lost in the maze of twisty windy make rules, all different, so I
# added a syntactically invalid 'puke here' to the fs/cpio/cpio.mk as shown here PACKAGES_PERMISSIONS_TABLE += /dev/console c 622 0 0 5 1 - - -$(sep) puke here # Then did 'make defconfig; make -d' in a clean buildroot tree and generated this output when it died # which definitely shows it's being read, even when no kernel is being built. ... Reading makefile 'fs/cpio/cpio.mk' (search path) (no ~ expansion)... fs/cpio/cpio.mk:25: *** missing separator (did you mean TAB instead of 8 spaces?). Stop.
# Without the 'puke here' the minimal build results in a dev directory containing these files:
# .../buildroot-2015.02$ ls -l /output/target/dev
total 4
-rw--w--w- 1 steve steve 0 Mar 28 14:32 console
lrwxrwxrwx 1 steve steve 10 Mar 1 15:26 log -> ../tmp/log
drwxrwxr-x 2 steve steve 4096 Mar 1 15:26 pts
I'm not sure how to proceed from here but I don't think the current situation is 'correct'
Steve Kenton
Quoted 13 lines · Arnout Vandecappelle
Quoted 2 lines · Steve Kenton
Quoted 1 line · Rob Landley
On Wed, Mar 25, 2015 at 12:44 PM, Steve Kenton <[email protected]> wrote:
[snip]
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
so I spent some time looking at my device usage and for the life of me I can't find where
/dev/consolegets created during the build process.The console PID 1 inherits at process launch is the
/dev/consolein initramfs. When you don't have initramfs, theinit/noinitramfs.ccode literally does:Yes, this is my configuration - initramfs not linked into kernel by buildroot #
BR2_TARGET_ROOTFS_INITRAMFSis not set
Quoted 7 lines
Just to be clear "initramfs not linked into kernel" is
BR2_TARGET_ROOTFS_CPIO,
"initramfs linked into the kernel" isBR2_TARGET_ROOTFS_INITRAMFS. But for the purpose of the contents of the rootfs, that doesn't make a difference.But so in your case, Steve, I believe you _do_ have an initramfs, i.e. you have
selectedBR2_TARGET_ROOTFS_CPIO. Otherwise/dev/consolewouldn't have been created :-).
I note that these are buildroot symbols and I was talking about kernel config symbols, so you may have to trace through the build process a bit to see what the kernel .config actually winds up with and thus what happens at boot time. But it looks like you covered that.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
devtmpfs is a synthetic filesystem, like /proc or /sys. You're not talking to any sort of backing store when you mount it, instead the driver is hallucinating contents for you to read and write from on the fly, and when you interact with it you talk to functions in the driver that send/receive some data they just made up to humor your request.
Quoted 8 lines
Quoted 3 lines
Again if I under stand correctly the contents of the on-disk /dev on the root filesystem in my case (initramfs not linked into kernel) are unused and normally inaccessible because of devtmpfs mount on top?
Indeed, its contents becomes inaccessible as soon as devtmpfs is mounted on
top. In case of initramfs, that is when /init is executed and it is mounted explicitly; in case of non-initramfs, it is mounted just after / is mounted and before/sbin/initis executed (Rob, correct me if I'm wrong).
I mentioned I've been meaning to send a patch to make initramfs honor the kernel's auto-mount-devtmpfs config option. Last I checked it was only happening for root=/dev/hda style mounts, not for initramfs mounts. (So your /init script had to mount devtmpfs, I don't think the kernel does it for you on initramfs. But it's quite possible somebody else fixed it in the last year or so and I didn't notice. I've been off doing toybox and banging on sh2 nommu stuff.)
Either way, if you mount another filesystem on top of /dev, the original contents of /dev are inaccessable until you unmount the filesystem blocking it. This is pretty much true of the contents of any directory you mount stuff on: the mount renders the original contents inaccessable. (Nobody said the directory you mounted on was empty...)
(Ok, that's not _quite_ true. You can mount a filesystem on $PWD but still access the old contents through "." until you cd out of it. A new traversal from root wouldn't get you _back_ there, but your current position still links to the otherwise inaccessable directory. And yes, back in the day people hacking into 1980's unix systems would hide their files from the sysadmin by overmounting them. Each process has "/" and "." as two process-local symlinks indicating your current chroot inode and your current chdir inode, all paths start from one or the other. This leads to a fun trick for breaking out of chroots where you make a subdirectory, call chroot(2) to point / into that subdirectory but _not_ cd into it, then chroot ../../../.. until you stop moving, then chroot / to get access to the full tree. There's only one chroot symlink you see, and the test for "can't cd .. past it" is an == test which won't trigger if your cwd is currently _above_ it.)
Sorry, digression. :)
Quoted 3 lines
So with initramfs, you will have two versions of
/dev/console: one real
mknod'ed one in the initramfs itself, and a virtual one in the devtmpfs. After devtmpfs is mounted, the one in initramfs is no longer accessible.
Neither is "real" or "virtual". Both are device nodes with "block/char", "major", and "minor" telling the system which driver to talk to you when you open it.
(If you loopback mount a squashfs that contains device nodes, is that not a "real" filesystem? How about if rhw dilw you loopback mounted it from is on a network mount?)
Rob