/linuxrc query
Hi,
I have a initrd working, a /linuxrc on it that runs and executes. My question for the commands after pivot_root which works like a charm, thanks to initrd.txt,
what does redirecting stdin\stdout\stderr to dev/console achieve? I thought since the root is now the "new" root, dev/console will be used automatically? Also, why chroot, why not call init directly? #exec chroot . sbin/init 3 <dev/console >dev/console 2>&1
Since the above never returns, what follows in not freed. Does this mean I have around 4-6 mb of ram being used up unnecessarily? Any solution?
#umount /initrd
#blockdev --flushbufs /dev/ram0 # /dev/rd/0 if using devfs
Thanks and Regards
Amit
Quoted 2 lines · Amit D Chaudhary
what does redirecting stdin\stdout\stderr to dev/console achieve? I thought since the root is now the "new" root, dev/console will be used automatically?
No, you would continue using the file descriptors which are already open, i.e. on /dev/console on the old root.
Quoted 1 line
Also, why chroot, why not call init directly?
To make sure the root of the current process is indeed changed. pivot_root currently forces a chroot on all processes (except the ones that have explicitly moved out of /) in order to move all the kernel threads too, but this is not a nice solution. Once a better solution is implemented for the kernel threads, we might drop the forced chroot, and then the explicit chroot here becomes important.
Quoted 1 line
Since the above never returns, what follows in not freed.
You can run them later, e.g. /etc/rc.d/rc.local
Or, if you needs the space immediately, make "what-follows" a script than first frees them, and then exec's init.
- Werner
Signature · 3 lines
_________________________________________________________________________ / Werner Almesberger, ICA, EPFL, CH [email protected] / /_IN_N_032__Tel_+41_21_693_6621__Fax_+41_21_693_6610_____________________/
Hi,
Thanks for the response. PSB,
Quoted 4 lines · Werner Almesberger
Amit D Chaudhary wrote:
No, you would continue using the file descriptors which are already open, i.e. on
/dev/consoleon the old root.
So, makes sense. And the child process that follow will use now the new fd's.
Quoted 9 lines
Quoted 1 line
Also, why chroot, why not call init directly?
To make sure the root of the current process is indeed changed.
pivot_rootcurrently forces a chroot on all processes (except the ones that have explicitly moved out of /) in order to move all the kernel threads too, but this is not a nice solution. Once a better solution is implemented for the kernel threads, we might drop the forced chroot, and then the explicit chroot here becomes important.
So, it is not a requirement currently but it is useful to have the script not dependent on the current pivot_root implementation.
Quoted 3 lines
You can run them later, e.g.
/etc/rc.d/rc.local
Or, if you needs the space immediately, make "what-follows" a script than first frees them, and then exec's init.
Sure will put in a script that does it. I had left it in /linuxrc as I thought that's what initrd.txt suggested one to do. But other information in the initrd.txt mentions otherwise, hence the query here.
I am assuming umount and thereby blockdev after pivot_script and before "chroot . init ..." don't make sense as files(dev/console among others) are\might still be in use.
Best Regards
Amit
Hi,
Also as a note, what we are doing is keeping our rootfs on flash as a tar.gz and reading it and mounting it on a ramfs in the /linuxrc before doing a pivot_root. To summarize, pivot_root has been a life saver as the earlier real_root_dev might not have been useful in this case. Not using the ramfs limits for now, will do soon.
Thanks
Amit
Quoted 28 lines · Werner Almesberger
Quoted 1 line · Amit D Chaudhary
what does redirecting stdin\stdout\stderr to dev/console achieve? I thought since the root is now the "new" root, dev/console will be used automatically?
No, you would continue using the file descriptors which are already open, i.e. on
/dev/consoleon the old root.Quoted 1 line
Also, why chroot, why not call init directly?
To make sure the root of the current process is indeed changed.
pivot_rootcurrently forces a chroot on all processes (except the ones that have explicitly moved out of /) in order to move all the kernel threads too, but this is not a nice solution. Once a better solution is implemented for the kernel threads, we might drop the forced chroot, and then the explicit chroot here becomes important.Quoted 1 line
Since the above never returns, what follows in not freed.
You can run them later, e.g.
/etc/rc.d/rc.local
Or, if you needs the space immediately, make "what-follows" a script than first frees them, and then exec's init.- Werner
Quoted 2 lines · Amit D Chaudhary
So, it is not a requirement currently but it is useful to have the script not dependent on the current
pivot_rootimplementation.
Yes. Also note that the relative path for dev/console works in either case, while /dev/console would fail without the implied chroot in pivot_root.
Quoted 2 lines
But other information in the
initrd.txt mentions otherwise, hence the query here.
Hmm, sounds like a bug. Where did you find this ?
Quoted 3 lines
I am assuming umount and thereby blockdev after
pivot_scriptand before "chroot . init ..." don't make sense as files(dev/console among others) are\might still be in use.
Exactly. They's in use in any case until you close and re-open the console.
- Werner
Signature · 3 lines
_________________________________________________________________________ / Werner Almesberger, ICA, EPFL, CH [email protected] / /_IN_N_032__Tel_+41_21_693_6621__Fax_+41_21_693_6610_____________________/
Quoted 2 lines · Amit D Chaudhary
To summarize,
pivot_roothas been a life saver as the earlierreal_root_devmight not have been useful in this case.
The whole old change_root mechanism with real_root_dev is best forgotten quickly ;-) It's also completely helpless as soon as you fire off some kernel threads that don't call exit_fs.
Quoted 1 line
Not using the ramfs limits for now, will do soon.
BTW, if you can't free the RAM disk, you may have to apply
http://icawww1.epfl.ch/~almesber/patches/rdfree
- Werner
Signature · 3 lines
_________________________________________________________________________ / Werner Almesberger, ICA, EPFL, CH [email protected] / /_IN_N_032__Tel_+41_21_693_6621__Fax_+41_21_693_6610_____________________/
Quoted 5 lines · [email protected] said:
Also as a note, what we are doing is keeping our rootfs on flash as a
tar.gz and reading it and mounting it on a ramfs in the /linuxrc before doing apivot_root. To summarize,pivot_roothas been a life saver as the earlierreal_root_devmight not have been useful in this case. Not using the ramfs limits for now, will do soon.
If you're concerned about memory usage - why untar the whole of your root filesystem into a ramfs? My preferred solution is to just mount the root filesystem directly from the flash as cramfs (or JFFS2), with symlinks into a ramfs for appropriate parts like /tmp and /var.
I suppose the best option is actually to union-mount the ramfs over the root, rather than mucking about with symlinks. I just haven't got round to doing that yet.
--
dwmw2
Quoted 5 lines · Werner Almesberger
Quoted 1 line · Amit D Chaudhary
But other information in the initrd.txt mentions otherwise, hence the query here.
Hmm, sounds like a bug. Where did you find this ?
I quote from the version in linux-2.4.2-ac22
"
Now, the initrd can be unmounted and the memory allocated by the RAM disk can be freed:
# umount /initrd # blockdev --flushbufs /dev/ram0 # /dev/rd/0 if using devfs "
I guess I assumed the above has to be put in /linuxrc as it follows other commands which do go in the /linuxrc. The assumption is not quite correct as I have since found out.
Regards
Amit
RAMFS, CRAMFS and JFFS2(was Re: /linuxrc query)
In reply to Amit D ChaudharyHi David,
I did consider CRAMFS and JFFS2 when it was announced on the mtd list. Conserving flash over system ram is more relevant. Our reasons are below:
RAMFS v/s CRAMFS
1. RAMFS is just more stable in terms of less complexity, less bugs reported over the time, etc.
2. RAMFS is a fairly robust filesystem and all features required as far as I can tell.
RAMFS v/s ext2 on a ramdisk
1. Fixed size. Results in overallocation or more dangerously overshooting the size decided earlier during mke2fs.
RAMFS v/s JFFS2
1. JFFS2 was announced around a month ago and few utils were still in development, hence it was not there for the first development cycle.
2. Just offhand, joining 100 small files (say in /etc) and then compressing into a one file over these 100 files in a JFFS2 fs seems to give better overall space usage and probably less flash wear during modifications.
3. JFFS2 does has some compelling reasons, more robustness ofcourse with JFFS2 over above approach, flash wearing(though would require sufficient free space on that mtd partition), etc. Plan to compare these later.
I might be wrong and hence would welcome any suggestions.
Regards
Amit
Quoted 18 lines · David Woodhouse
Quoted 5 lines · [email protected] said:
Also as a note, what we are doing is keeping our rootfs on flash as a
tar.gz and reading it and mounting it on a ramfs in the /linuxrc before doing apivot_root. To summarize,pivot_roothas been a life saver as the earlierreal_root_devmight not have been useful in this case. Not using the ramfs limits for now, will do soon.If you're concerned about memory usage - why untar the whole of your root filesystem into a ramfs? My preferred solution is to just mount the root filesystem directly from the flash as cramfs (or JFFS2), with symlinks into a ramfs for appropriate parts like /tmp and /var.
I suppose the best option is actually to union-mount the ramfs over the root, rather than mucking about with symlinks. I just haven't got round to doing that yet.
--
dwmw2
Re: RAMFS, CRAMFS and JFFS2(was Re: /linuxrc query)
Quoted 10 lines · Amit D Chaudhary
Hi David,
I did consider CRAMFS and JFFS2 when it was announced on the mtd list. Conserving flash over system ram is more relevant. Our reasons are below:
RAMFS v/s CRAMFS
1. RAMFS is just more stable in terms of less complexity, less bugs reported over the time, etc.
2. RAMFS is a fairly robust filesystem and all features required as far as I can tell.
I'm not aware of any bugs being found in cramfs recently - unless you wanted to use it on Alpha (or anything else where PAGE_SIZE != the hard-coded 4096 in mkcramfs.c).
I wouldn't avoid it for those reasons - although if you're _really_ short of flash space, the same argument applies as for JFFS2 - a single compression stream (tar.gz) will be smaller than compressing individual pages like JFFS2 and cramfs do.
Quoted 1 line
I might be wrong and hence would welcome any suggestions.
Given your stated constraints - you're very short of flash and don't care too much about the RAM you use, you've may have made the same choice I would have done.
Bearing in mind that you have to take into account the overhead of the initrd which does the untarring - what's the total size of the initrd + tarball on the flash, and what size would the corresponding cramfs be?
If you could fit your root filesystem into a cramfs on the flash, I'd do that instead and use ramfs for the parts which need to be writeable.
Signature · 1 line
dwmw2
Re: CRAMFS
Quoted 4 lines · David Woodhouse
Quoted 4 lines
1. RAMFS is just more stable in terms of less complexity, less bugs reported over the time, etc.
2. RAMFS is a fairly robust filesystem and all features required as far as I can tell.
Ok, ramfs is really simple, but heck, cramfs is not much more complex :) It's as simple a flash-filesystem as you can get.
I don't know why the comparision is made though, they are used for two completely different things... ramfs is for temporary file storage, cramfs is for immutable files stored on flash. Each by itself is quite optimal for what it's designed for, isn't it ?
Quoted 3 lines
I'm not aware of any bugs being found in cramfs recently - unless you wanted to use it on Alpha (or anything else where
PAGE_SIZE!= the hard-coded 4096 inmkcramfs.c).
I committed a patch that disappeared that added the choice of page size (trivial yes :), we have PAGE_SIZE == 8192 on our systems. Works fine.
Quoted 4 lines
I wouldn't avoid it for those reasons - although if you're _really_ short of flash space, the same argument applies as for JFFS2 - a single compression stream (tar.gz) will be smaller than compressing individual pages like JFFS2 and cramfs do.
Here are some results from a quite mixed filesystem:
[bjornw@godzilla linux]$ ls -l cram*
-rw-r--r-- 1 bjornw users 1179648 Mar 23 22:38 cram32768 -rw-r--r-- 1 bjornw users 1282048 Mar 23 22:38 cram4096 -rw-r--r-- 1 bjornw users 1220608 Mar 23 22:38 cram8192
(the numbers correspond to blocksize)
There's not any big difference here.
With bigger files though, the difference get larger. YMMV.
Notice that you can change cramfs so it uses a blocksize that is bigger than PAGE_SIZE, of course, if it really is necessary. You'll get worse performance at run-time though since you need to cache the page and hope for read-ahead or similar (you can stuff the pages in the page-cache even if they are not requested for example).
-BW
Re: CRAMFS
Quoted 4 lines
I don't know why the comparision is made though, they are used for two completely different things... ramfs is for temporary file storage, cramfs is for immutable files stored on flash. Each by itself is quite optimal for what it's designed for, isn't it ?
Exactly. My mistake earlier to assume cramfs was "compressed ramfs"! ;-)
I should compare it to the tar.gz option and JFFS2. Will do in the next evaluation.
This will be more of a replace initrd+custom /linuxrc with a CRAMFS-based rootfs on a flash device assuming CRAMFS can be directly read by kernel\init for getting the rootfs. Ditto for JFFS2
Also, the platform is PPC, IBM 405GP to be precise.
Regards
Amit