ramdisk/tmpfs/ramfs/memfs ?
Hi,
I'm working on an embedded system here which has no harddisk. So, I can't swap to disk and need to have /var & /tmp in RAM. I'm confused between the various options for in RAM file- systems. At the moment I've created a ramdisk and made an ext2 partition in it (which is compressed as I applied the e2compr patch), which is working fine. Anyway questions:
1. I presume the kernel is clever enough to not cache any files from these filesystems? Would it ever need to?
2. Is tmpfs is basically swap and /tmp together in a ramdisk? The advantage being you need to reserve less RAM for both together than seperately?
3. If I've no backing store (harddisk?) is there any advantage of using tmpfs instead of ramfs? Also does tmpfs need a backing store?
5. Can you set size limits on ramfs/tmpfs/memfs?
6. Is a ramdisk resizable like the others. If so, do you have to delete/recreate or umount/resize a fs (e.g. ext2) every time it's resized? Do ramfs/tmpfs/memfs do this transparently? Are ramdisks resizable in kernel 2.2?
7. What's memfs?
8. Is there a way I can get transparent compression like I now have using a ramdisk+ext2+e2compr with ramfs et al?
9. Apart from this transparent compression, is there any other functionality ext2 would have over ramfs for e.g, for /tmp & /var? Also would ramfs have less/more speed over ext2?
thanks,
Padraig.
Quoted 6 lines · Padraig Brady
I'm working on an embedded system here which has no harddisk. So, I can't swap to disk and need to have /var & /tmp in RAM. I'm confused between the various options for in RAM file- systems. At the moment I've created a ramdisk and made an ext2 partition in it (which is compressed as I applied the e2compr patch), which is working fine. Anyway questions:
Ouch.. yes you had to do stuff like that in the old days but it's very cumbersome and inefficient compared to ramfs for what you're trying to do.
Quoted 2 lines
1. I presume the kernel is clever enough to not cache any files from these filesystems? Would it ever need to?
You always need to "cache" pages read. Because a page is the smallest possible granularity for the MMU, and a block-based filesystem does not need to be page-aligned, so it's impossible to do it otherwise in a general way.
Quoted 3 lines
3. If I've no backing store (harddisk?) is there any advantage of using tmpfs instead of ramfs? Also does tmpfs need a backing store?
I don't know what tmpfs does actually, but if it is like you suggest (a ramfs that can be swapped out ?) then you don't need it obviously (since you don't have any swap).
ramfs simply inserts any files written into the kernels cache and tells it not to forget it. it can't get much more simple than that.
Quoted 1 line
5. Can you set size limits on ramfs/tmpfs/memfs?
i don't think you can set a limit in the current ramfs implementation but it would not be particularly difficult to make it work I think
Quoted 4 lines
6. Is a ramdisk resizable like the others. If so, do you have to delete/recreate or umount/resize a fs (e.g. ext2) every time it's resized? Do ramfs/tmpfs/memfs do this transparently? Are ramdisks resizable in kernel 2.2?
ramfs does not need any "resizing" because there is no filesystem behind it. there is only the actual file data and metadata in the cache itself. if you delete a file, it disapperas, if you create a new one new pages are brought in.
Quoted 3 lines
7. What's memfs?
8. Is there a way I can get transparent compression like I now have using a ramdisk+ext2+e2compr with ramfs et al?
you could try using jffs2 on a RAM-simulated MTD partition. i think that would work but i have not tried it..
Quoted 3 lines
9. Apart from this transparent compression, is there any other functionality ext2 would have over ramfs for e.g, for /tmp & /var? Also would ramfs have less/more speed over ext2?
ramfs has all the bells and whistles you need except size limiting. and obviously its faster than simulating a harddisk in ram and using ext2 on it..
-bw
Quoted 2 lines · [email protected] said:
you could try using jffs2 on a RAM-simulated MTD partition. i think
that would work but i have not tried it..
It works. Most of the early testing and development was done on it. It wouldn't give you dynamic sizing like ramfs though.
It would be nice to have a version of ramfs which compresses pages into a separate backing store when they're unused. Shame somebody nicked the name 'cramfs' for something else, really :)
But I'm confused. Padraig, if you have no backing store, where do the initial contents of your root filesystem come from?
--
dwmw2
Quoted 7 lines
Quoted 2 lines
you could try using jffs2 on a RAM-simulated MTD partition. i think
that would work but i have not tried it..It works. Most of the early testing and development was done on it. It wouldn't give you dynamic sizing like ramfs though.
It would be nice to have a version of ramfs which compresses pages
into a
Quoted 1 line
separate backing store when they're unused. Shame somebody nicked the
name
great... especially for my boot/rootdisks on an 8/16MB system which start
swapon right in the /linuxrc
Quoted 1 line
'cramfs' for something else, really :)
This should be names cromfs as IIRC it isn't writable.
Quoted 3 lines
But I'm confused. Padraig, if you have no backing store, where do the initial contents of your root filesystem come from?
Netboot? Floppies? (as for me)
-mirabilos
Thanks for all the replies!
Would anyone have any objections to me writing a Documentation/ramdisks.txt or Documentation/filesystems/ram.txt as this stuff is very hard to find info on?
Anyway seems like ramfs is the way to go with the limits patch applied. I loose compression, but gain as there is no data duplication in RAM, and also I don't need to preallocate space for a partition. If I understand correctly ramfs just points to the file data which are pages in the cache marked not to be uncached. Doh! is ramfs supported in 2.2?
dwmw2 mentioned, for ramfs, it would be useful to compress unused pages (to someplace in RAM would be very good for my application), but I guess this would add a lot of complexity.
btw I get my initial root filesystem from a compact flash that can be accessed just like a hardisk. It's writeable also like a harddisk, but we boot with it readonly, and only mount it rw if we want to save config or whatever. We definitely wouldn't swap to it as it has limited erase/write cycles. The filesystem is compressed ext2.
I don't have swap so don't need tmpfs, but could probably use it anyway without a backing store? Anyway why was ramfs created if tmpfs existed, unless tmpfs requires backing store? They both seem to have been written around the same time?
As for using JFFS2 + MTD ramdisk intead of ext2+e2compr+ramdisk is not an option as the only advantage would be journalling, you still can't resize. IMHO JFFS is only required where you have flash without an IDE interface.
cheers,
Padraig.
Quoted 68 lines · Bjorn Wesen
Quoted 6 lines · Padraig Brady
I'm working on an embedded system here which has no harddisk. So, I can't swap to disk and need to have /var & /tmp in RAM. I'm confused between the various options for in RAM file- systems. At the moment I've created a ramdisk and made an ext2 partition in it (which is compressed as I applied the e2compr patch), which is working fine. Anyway questions:
Ouch.. yes you had to do stuff like that in the old days but it's very cumbersome and inefficient compared to ramfs for what you're trying to do.
Quoted 2 lines
1. I presume the kernel is clever enough to not cache any files from these filesystems? Would it ever need to?
You always need to "cache" pages read. Because a page is the smallest possible granularity for the MMU, and a block-based filesystem does not need to be page-aligned, so it's impossible to do it otherwise in a general way.
Quoted 3 lines
3. If I've no backing store (harddisk?) is there any advantage of using tmpfs instead of ramfs? Also does tmpfs need a backing store?
I don't know what tmpfs does actually, but if it is like you suggest (a ramfs that can be swapped out ?) then you don't need it obviously (since you don't have any swap).
ramfs simply inserts any files written into the kernels cache and tells it not to forget it. it can't get much more simple than that.
Quoted 1 line
5. Can you set size limits on ramfs/tmpfs/memfs?
i don't think you can set a limit in the current ramfs implementation but it would not be particularly difficult to make it work I think
Quoted 4 lines
6. Is a ramdisk resizable like the others. If so, do you have to delete/recreate or umount/resize a fs (e.g. ext2) every time it's resized? Do ramfs/tmpfs/memfs do this transparently? Are ramdisks resizable in kernel 2.2?
ramfs does not need any "resizing" because there is no filesystem behind it. there is only the actual file data and metadata in the cache itself. if you delete a file, it disapperas, if you create a new one new pages are brought in.
Quoted 3 lines
7. What's memfs?
8. Is there a way I can get transparent compression like I now have using a ramdisk+ext2+e2compr with ramfs et al?you could try using jffs2 on a RAM-simulated MTD partition. i think that would work but i have not tried it..
Quoted 3 lines
9. Apart from this transparent compression, is there any other functionality ext2 would have over ramfs for e.g, for /tmp & /var? Also would ramfs have less/more speed over ext2?
ramfs has all the bells and whistles you need except size limiting. and obviously its faster than simulating a harddisk in ram and using ext2 on it..
(quoted diff) +0
-2-bw -To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 2 lines · Padraig Brady
for a partition. If I understand correctly ramfs just points to the file data which are pages in the cache marked not to be
It does not even do that - as of 2.4, the VFS in the kernel also knows how to cache a filestructure itself. It's in the dentry-cache. So ramfs just provides the thin mapping between VFS operations and the VFS caches (dentries, inodes, pages) like any other 2.4 filesystem - with the difference that ramfs does not need to know anything about actually transferring the cache entries to a backing store (a physical filesystem).
Take a look at fs/ramfs/inode.c, it's just some hundred odd lines of code and worth reading to find out more about how 2.4's VFS works.
Quoted 1 line
uncached. Doh! is ramfs supported in 2.2?
Don't think so, for the above reason.
-BW
Quoted 5 lines · [email protected] said:
btw I get my initial root filesystem from a compact flash that can be accessed just like a hardisk. It's writeable also like a harddisk, but we boot with it readonly, and only mount it rw if we want to save config or whatever. We definitely wouldn't swap to it as it has limited erase/write cycles. The filesystem is compressed ext2.
Why copy it into RAM? Why not use cramfs and either turn the writable directories into symlinks into a ramfs which you create at boot time, or union-mount a ramfs over the top of it?
Quoted 4 lines · [email protected] said:
As for using JFFS2 + MTD ramdisk intead of ext2+e2compr+ramdisk is not an option as the only advantage would be journalling, you still can't resize. IMHO JFFS is only required where you have flash without an IDE interface.
True. We are currently lacking a compact, compressing, journalling filesystem for use on block devices. It's been suggested that we could make JFFS2 work on them, by making a fake MTD device which uses a block device as backing store. Nobody's yet shown me the code though :)
--
dwmw2
Quoted 11 lines · David Woodhouse
Quoted 5 lines · [email protected] said:
btw I get my initial root filesystem from a compact flash that can be accessed just like a hardisk. It's writeable also like a harddisk, but we boot with it readonly, and only mount it rw if we want to save config or whatever. We definitely wouldn't swap to it as it has limited erase/write cycles. The filesystem is compressed ext2.
Why copy it into RAM? Why not use cramfs and either turn the writable directories into symlinks into a ramfs which you create at boot time, or union-mount a ramfs over the top of it?
? I never said I copied it into RAM. The ramfs is just used for /tmp & /var
which are symlinks on the flash. I didn't know you could read/write cramfs
like ext2/JFFS2? I still want to be able to update/create/delete files like a normal ext2 partition. Oh I see the confusion, sorry my fault, when
I said the root filesystem is compressed ext2, I meant it's ext2 with chattr +c done on all files (the e2compr patch implmenents it). By the way why isn't e2compr a standard part of the kernel. I've have no problems
at all with it.
I didn't know about union-mounting, seems useful.
As for whether to use tmpfs/ramfs I think they do basically the same thing
only the implementation is different. tmpfs uses shared mem and so is probably more portable than ramfs which it tightly coupled with VFS. I think I'll use ramfs (with limits patch) so.
Quoted 10 lines
Quoted 4 lines · [email protected] said:
As for using JFFS2 + MTD ramdisk intead of ext2+e2compr+ramdisk is not an option as the only advantage would be journalling, you still can't resize. IMHO JFFS is only required where you have flash without an IDE interface.
True. We are currently lacking a compact, compressing, journalling filesystem for use on block devices. It's been suggested that we could make JFFS2 work on them, by making a fake MTD device which uses a block device as backing store. Nobody's yet shown me the code though :)
How much more efficent is JFFS than say ext3+e3compr, wrt:
logic size/logic speed/RAM requirements/filesystem structure size.
Quoted 2 lines
--
dwmw2
Padraig.
Quoted 2 lines · Padraig Brady
How much more efficent is JFFS than say ext3+e3compr, wrt:
logic size/logic speed/RAM requirements/filesystem structure size.
JFFS doesn't have to use the FTL layer between its block device and the flash - that's a lot already !
Xav
Quoted 3 lines · David Woodhouse
Why copy it into RAM? Why not use cramfs and either turn the writable directories into symlinks into a ramfs which you create at boot time, or union-mount a ramfs over the top of it?
^^^^^^^^^^^
I didn't think we had union-mounting support... does it exist and I've somehow missed it?
regards,
David
Signature · 3 lines
David L. Parsley Network Administrator Roanoke College
Quoted 2 lines · [email protected] said:
I didn't think we had union-mounting support... does it exist and
I've somehow missed it?
I think you need Al Viro's namespace stuff for it to work properly.
--
dwmw2
Quoted 1 line
Quoted 1 line
btw I get my initial root filesystem from a compact flash that can
be
Quoted 1 line
Quoted 1 line
accessed just like a hardisk. It's writeable also like a harddisk,
but
Quoted 6 lines
Quoted 3 lines
we boot with it readonly, and only mount it rw if we want to save config or whatever. We definitely wouldn't swap to it as it has limited erase/write cycles. The filesystem is compressed ext2.
Why copy it into RAM? Why not use cramfs and either turn the writable directories into symlinks into a ramfs which you create at boot time,
or
Quoted 1 line
union-mount a ramfs over the top of it?
Hey! SOunds great! How to do it?
I tried to # mount --bind the directories but they do not show me both entries. I have romfs root and tmpfs.
-mirabilos
Hi Padraig,
Quoted 2 lines · Padraig Brady
I don't have swap so don't need tmpfs, but could probably use it anyway without a backing store?
Yes, it does not need backing store.
Quoted 3 lines
Anyway why was ramfs created if tmpfs existed, unless tmpfs requires backing store? They both seem to have been written around the same time?
- shm fs was written as a specialized fs to implement POSIX shared memory based on SYSV shm.
- ramfs was introduced shortly after shm fs and was meant as a programming example for a minimal virtual filesystem.
- Later shm fs was redone to use the same methods like ramfs but still was only useable for shared memory.
- After the release of 2.4.0, I extended shm fs to support read/write and thus be tmpfs and since then it can replace ramfs.
Greetings
Christoph
Quoted 10 lines · David Woodhouse
Quoted 3 lines · [email protected] said:
you could try using jffs2 on a RAM-simulated MTD partition. i think that would work but i have not tried it..
It works. Most of the early testing and development was done on it. It wouldn't give you dynamic sizing like ramfs though.
It would be nice to have a version of ramfs which compresses pages into a separate backing store when they're unused. Shame somebody nicked the name 'cramfs' for something else, really :)
Hmm must look into getting ramfs working with
http://linuxcompressed.sourceforge.net/
seems like the best of both worlds.
Padraig
Followup to: <[email protected]> By author: Bjorn Wesen <[email protected]> In newsgroup: linux.dev.kernel
Quoted 6 lines
Quoted 2 lines
5. Can you set size limits on ramfs/tmpfs/memfs?
i don't think you can set a limit in the current ramfs implementation but it would not be particularly difficult to make it work I think
It's a little more painful than you'd think for the simple reason that ramfs currently contains no space accounting whatsoever, which probably is a bad thing. It definitely gave me some serious pause when I was working on SuperRescue 1.3, since I had no way of reasonably judging how big my ramfs actually was. The only way I could get a reasonable idea was rebooting with various mem= parameters.
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt
Quoted 9 lines · Bjorn Wesen
Quoted 3 lines · Padraig Brady
3. If I've no backing store (harddisk?) is there any advantage of using tmpfs instead of ramfs? Also does tmpfs need a backing store?
I don't know what tmpfs does actually, but if it is like you suggest (a ramfs that can be swapped out ?) then you don't need it obviously (since you don't have any swap).
Quoted 7 lines
ramfs simply inserts any files written into the kernels cache and tells it not to forget it. it can't get much more simple than that.
Quoted 1 line
5. Can you set size limits on ramfs/tmpfs/memfs?
i don't think you can set a limit in the current ramfs implementation but it would not be particularly difficult to make it work I think
tmpfs is basically ramfs with limits.
Signature · 1 line
marko
Quoted 1 line · H. Peter Anvin
Quoted 1 line
Quoted 1 line
5. Can you set size limits on ramfs/tmpfs/memfs?
Yes, there is a patch for this.
Quoted 9 lines
Quoted 2 lines
i don't think you can set a limit in the current ramfs implementation but it would not be particularly difficult to make it work I think
It's a little more painful than you'd think for the simple reason that ramfs currently contains no space accounting whatsoever, which probably is a bad thing. It definitely gave me some serious pause when I was working on SuperRescue 1.3, since I had no way of reasonably judging how big my ramfs actually was. The only way I could get a reasonable idea was rebooting with various mem= parameters.
The patched variant gives to all of it. Even several kinds of limits (inodes, dentries, ram pages).
I use this patch in production.
The ac-Kernels all have this patch included, which are sometimes more stable anyway these day. ;-)
HTH
Regards
Ingo Oeser
Signature · 2 lines
10.+11.03.2001 - 3. Chemnitzer LinuxTag <http://www.tu-chemnitz.de/linux/tag>
<<<<<<<<<<<< been there and had much fun >>>>>>>>>>>>Followup to: <[email protected]> By author: Marko Kreen <[email protected]> In newsgroup: linux.dev.kernel
Quoted 3 lines
tmpfs is basically ramfs with limits.
... and swappable.
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt
Hi Padraig,
Quoted 3 lines · Padraig Brady
2. Is tmpfs is basically swap and /tmp together in a ramdisk? The advantage being you need to reserve less RAM for both together than seperately?
tmpfs is ramfs+swap+limits. It is not using ramdisks and is not related to them.
Quoted 3 lines
3. If I've no backing store (harddisk?) is there any advantage of using tmpfs instead of ramfs? Also does tmpfs need a backing store?
Probably yes, since you spare a little bit kernel memory. most of tmpfs is unconditionally in the kernel for shared mappings. So the actual CONFIG_TMPFS only adds some small functions to the kernel to export this to usre space.
Greetings
Christoph
Quoted 6 lines
Quoted 2 lines
tmpfs is basically ramfs with limits.
... and swappable.
-hpa
Hmmm and what's shmfs? Precedessor of tmpfs?
I even cant remember which one I use for /tmp ;-)
-mirabilos
Quoted 8 lines · On Fri, 27 Apr 2001, [email protected] wrote:
Quoted 6 lines
Quoted 2 lines
tmpfs is basically ramfs with limits.
... and swappable.
-hpa
Hmmm and what's shmfs? Precedessor of tmpfs?
Yes.
Quoted 1 line
I even cant remember which one I use for /tmp ;-)
You can mount tmpfs also with type "shm" for compatibility. Type "shm" will be marked as obsolete in 2.5
Greetings
Christoph