What File System supports Application XIP
Hi there,
We are developing embedded Linux system. Performance is our consideration. We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in RAM, and run in XIP (eXecute In Place) manners.
I know that Cramfs has supported Application XIP. Is there any other FS that also supports it? Ramdisk? Ramfs? Romfs?
Thanks and regards,
Colin
Quoted 3 lines · colin
Hi there,
We are developing embedded Linux system. Performance is our consideration. We hope some applications can run as fast as possible,
well ramfs by definition is XIP :)
but I guess the filesystem comes from flash somewhere at which point jffs2 with compression might be a better choice; if you have enough ram then the apps run from the pagecache anyway and compression keeps you from transfering too much data from the slower flash. It's not XIP but I don't think you really want XIP...
Hi,
How does ramfs offer application XIP ability?
I mean, when the ramfs image is mounted and the application in it is executed,
"exec", which is called by sh, should first copy every section of the application to RAM and then jump to the text section.
How do I avoid the stage copying text section to RAM?
Thanks and regards,
Colin
----- Original Message -----
From: "Arjan van de Ven" <[email protected]>
To: "colin" <[email protected]>
Cc: <[email protected]>
Sent: Thursday, September 09, 2004 5:18 PM
Subject: Re: What File System supports Application XIP
Quoted 3 lines · colin
Quoted 2 lines
Hi there,
We are developing embedded Linux system. Performance is our
consideration.
Quoted 9 lines
Quoted 1 line
We hope some applications can run as fast as possible,
well ramfs by definition is XIP :)
but I guess the filesystem comes from flash somewhere at which point jffs2 with compression might be a better choice; if you have enough ram then the apps run from the pagecache anyway and compression keeps you from transfering too much data from the slower flash. It's not XIP but I don't think you really want XIP...
Quoted 7 lines · colin
Hi,
How does ramfs offer application XIP ability?
I mean, when the ramfs image is mounted and the application in it is executed,
"exec", which is called by sh, should first copy every section of the application to RAM and then jump to the text section.
How do I avoid the stage copying text section to RAM?
this is not how linux works. programs execute directly from the pagecache without copy.
Quoted 11 lines · Arjan van de Ven
Quoted 7 lines · colin
Hi,
How does ramfs offer application XIP ability?
I mean, when the ramfs image is mounted and the application in it is executed,
"exec", which is called by sh, should first copy every section of the application to RAM and then jump to the text section.
How do I avoid the stage copying text section to RAM?this is not how linux works. programs execute directly from the pagecache without copy.
Most other filesystems populate the pagecache with I/O, presumably. In the case of a ramfs, is the page mapped directly from the fs into the pagecache without a copy?
=============================
Tim Bird
Architecture Group Co-Chair, CE Linux Forum
Senior Staff Engineer, Sony Electronics
E-mail: [email protected]
=============================
Quoted 4 lines
Most other filesystems populate the pagecache with I/O, presumably. In the case of a ramfs, is the page mapped directly from the fs into the pagecache without a copy?
actually ramfs exclusively lives in the pagecache.
Quoted 3 lines · Tim Bird
Most other filesystems populate the pagecache with I/O, presumably. In the case of a ramfs, is the page mapped directly from the fs into the pagecache without a copy?
ramfs _is_ pagecache.
it is cool like that.
Robert Love
Hi Robert,
Sorry. I donot quite understand. My English is poor... :-( Do you mean there wonot be another duplicated text section in RAM when application in Ramfs is executed?
Regards,
Colin
----- Original Message -----
From: "Robert Love" <[email protected]>
To: "Tim Bird" <[email protected]>
Cc: <[email protected]>; "colin" <[email protected]>; <[email protected]>
Sent: Friday, September 10, 2004 1:38 AM
Subject: [*垃圾郵件*] Re: What File System supports Application XIP
Quoted 14 lines · Tim Bird
Quoted 4 lines
Most other filesystems populate the pagecache with I/O, presumably. In the case of a ramfs, is the page mapped directly from the fs into the pagecache without a copy?
ramfs _is_ pagecache.
it is cool like that.
Robert Love
Quoted 3 lines · colin
Sorry. I donot quite understand. My English is poor... :-( Do you mean there wonot be another duplicated text section in RAM when application in Ramfs is executed?
This is correct, Colin.
Ramfs is essentially an in-memory filesystem implemented in the page cache.
Robert Love
Quoted 8 lines · colin
Hi there,
We are developing embedded Linux system. Performance is our consideration. We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in RAM, and run in XIP (eXecute In Place) manners.
I know that Cramfs has supported Application XIP. Is there any other FS that also supports it? Ramdisk? Ramfs? Romfs?
Obvisously cramfs can not support XIP, because the "in-place" image is compressed (unless you have a processor that can execute compressed code :)
AFAIK only tmpfs supports XIP because it works on a higher level without using block devices underneath. Ramdisks are simply RAM block devices that behave like any other block device.
You can have a compressed image in flash (for instance), decompress everything into a tmpfs and execute from there.
I'm not sure, however, that this will be such a performance gain.
If you use cramfs (for instance) then the kernel will uncompress and run only the pages that are needed, and they will be cached in page cache so that they will be available again when needed. This way you only waste the RAM you actually need, and can still drop old pages if the application needs more RAM.
Just my two cents,
Signature · 4 lines
Paulo Marques - www.grupopie.com To err is human, but to really foul things up requires a computer. Farmers' Almanac, 1978
Quoted 10 lines · Paulo Marques
Quoted 8 lines · colin
Hi there,
We are developing embedded Linux system. Performance is our consideration. We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in RAM, and run in XIP (eXecute In Place) manners.
I know that Cramfs has supported Application XIP. Is there any other FS that also supports it? Ramdisk? Ramfs? Romfs?
Quoted 3 lines
Obvisously cramfs can not support XIP, because the "in-place" image is compressed (unless you have a processor that can execute compressed code :)
Actually there are hacks floating around which do let you use XIP with cramfs -- obviously you have to dispense with compression for the files you want to access that way.
You won't gain at runtime by using XIP though. Your code and data will end up in RAM _whatever_ file system you use, and you'll be running from page cache.
You may get a _slightly_ faster startup time after reboot if you use XIP from flash, because it doesn't have to be loaded into RAM first. But that comes at the cost of making it all a low slower during normal operation -- it's a lot slower to fetch icache lines from flash than it is from RAM.
Arjan's right -- you almost certainly don't really want XIP.
Signature · 1 line
dwmw2
Quoted 11 lines · David Woodhouse
Quoted 10 lines · Paulo Marques
Quoted 8 lines · colin
Hi there,
We are developing embedded Linux system. Performance is our consideration. We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in RAM, and run in XIP (eXecute In Place) manners.
I know that Cramfs has supported Application XIP. Is there any other FS that also supports it? Ramdisk? Ramfs? Romfs?
Quoted 7 lines
Quoted 3 lines
Obvisously cramfs can not support XIP, because the "in-place" image is compressed (unless you have a processor that can execute compressed code :)
Actually there are hacks floating around which do let you use XIP with cramfs -- obviously you have to dispense with compression for the files you want to access that way.
The patches I've seen require setting the CRAMFS_LINEAR option, to turn on linear addressing for cramfs, and CRAMFS_LINEAR_XIP. The result of these is to dispense with compression.
Quoted 4 lines
You won't gain at runtime by using XIP though. Your code and data will end up in RAM _whatever_ file system you use, and you'll be running from page cache.
Really? I thought that when you XIPed from flash, the page was mapped directly back to the flash memory address. There shouldn't be a RAM copy of the page at all.
Quoted 11 lines · Greg Ungerer (uClinux maintainer) once
[Application XIP provides a] win of keeping the application code in flash even when that is shared. Can make a difference on small memory systems.
It also helps alleviate the contiguous memory problem (or memory fragmentation if you prefer) when you don't have an MMU. We need to be able to allocate a big enough contigous memory region to load the text into. Can be a problem on systems that have been running for a while and free memory is fragmented. If the application can be run XIP from flash then you at least don't need to worry about that. (This is a very real problem on small RAM systems).
In the CE Linux Forum, we've been investigating Application XIP for two purposes: reduction in RAM footprint, and faster startup time. However, those both come at the sacrifice of runtime performance (as David says below). Running XIP from a RAM-based filesystem, as originally proposed, might solve the performance problem, but it wouldn't improve space utilization, and since you have to populate the RAM from somewhere on startup anyway, I don't think it will improve your bootup time. It _might_ improve per-application startup time, once the system is running, but I think a regular RAM disk suffices for that.
Quoted 5 lines
You may get a _slightly_ faster startup time after reboot if you use XIP from flash, because it doesn't have to be loaded into RAM first. But that comes at the cost of making it all a low slower during normal operation -- it's a lot slower to fetch icache lines from flash than it is from RAM.
FYI - Here are some rough numbers:
Time to run shell script which starts TinyX X server and "xsetroot -solid red", then shuts down:
First invocation: Non-XIP 3.195 seconds, XIP 2.035 seconds Second invocation: Non-XIP 1.744 seconds, XIP 1.765 seconds
I think this was on a 133 MHz PPC, but I'm not positive. In both cases the filesystem was in flash. Note that once the application pages are in RAM in the page cache, the Non-XIP case beats the XIP case (probably due to the penalty to access flash memory).
So the only performance win is on the first invocation of the application.
I'm just now starting to put together a wiki page of information about Application XIP, for those interested. (It has nothing now, so don't get excited.) But contributions are welcome at:
http://tree.celinuxforum.org/pubwiki/moin.cgi/ApplicationXIP
(Maybe when someone googles this message a year from now, there will be something interesting there... ;-)
=============================
Tim Bird
Architecture Group Co-Chair, CE Linux Forum
Senior Staff Engineer, Sony Electronics
E-mail: [email protected]
=============================
Tim Bird wrote:
...
Quoted 3 lines
The patches I've seen require setting the
CRAMFS_LINEARoption, to turn on linear addressing for cramfs, andCRAMFS_LINEAR_XIP. The result of these is to dispense with compression.
Compression is skipped for the XIP files, which are typically marked via the sticky bit. You'll also need a version of mkcramfs that creates the image without compressing those files.
...
Quoted 9 lines
FYI - Here are some rough numbers:
Time to run shell script which starts TinyX X server and "xsetroot -solid red", then shuts down:First invocation: Non-XIP 3.195 seconds, XIP 2.035 seconds Second invocation: Non-XIP 1.744 seconds, XIP 1.765 seconds
I think this was on a 133 MHz PPC, but I'm not positive. In both cases the filesystem was in flash.
It was measured on a 168MHz ARM 925T TI OMAP 1510.
Others' advice that "you probably don't want XIP" is true in most cases.
But in producing a battery-operated product with certain requirements
for performance, power savings (due to reduced RAM requirements), startup time (depending on the platform and software stack the difference can be significant), etc. XIP is an option chosen by some CE designers, who are willing to accept the performance penalty on a product that will still run adequately for its intended uses. It would be interesting to see an in-depth analysis of these topics on an actual Linux-based product such as a cell phone. There are, of course, a number of ways to address all these issues, not just XIP...
Signature · 2 lines
Todd Poynor MontaVista Software
Re: ¡@[*©U§£¶l¥ó*] Re: What File System supports Application XIP
In reply to Paulo MarquesHi Paulo,
Cramfs seems to support it now.
At the beginning, I also think it's impossible to support Application XIP in Cramfs.
http://www.denx.de/twiki/publish/DULG/ConfigureLinuxForXIP.html
Regards,
Colin
----- Original Message -----
From: "Paulo Marques" <[email protected]>
To: "colin" <[email protected]>
Cc: <[email protected]>
Sent: Thursday, September 09, 2004 5:19 PM
Subject: ¡@[*©U§£¶l¥ó*] Re: What File System supports Application XIP
Quoted 4 lines
Quoted 3 lines · colin
Hi there,
We are developing embedded Linux system. Performance is our
consideration.
Quoted 4 lines
Quoted 4 lines
We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in RAM, and run in XIP (eXecute In Place) manners.
I know that Cramfs has supported Application XIP. Is there any other FS
that
Quoted 29 lines
Quoted 1 line
also supports it? Ramdisk? Ramfs? Romfs?
Obvisously cramfs can not support XIP, because the "in-place" image is compressed (unless you have a processor that can execute compressed code :)
AFAIK only tmpfs supports XIP because it works on a higher level without using block devices underneath. Ramdisks are simply RAM block devices that behave like any other block device.
You can have a compressed image in flash (for instance), decompress everything into a tmpfs and execute from there.
I'm not sure, however, that this will be such a performance gain.
If you use cramfs (for instance) then the kernel will uncompress and run only the pages that are needed, and they will be cached in page cache so that they will be available again when needed. This way you only waste the RAM you actually need, and can still drop old pages if the application needs more RAM.
Just my two cents,
Signature · 4 lines
Paulo Marques - www.grupopie.com To err is human, but to really foul things up requires a computer. Farmers' Almanac, 1978
Quoted 2 lines · colin
I know that Cramfs has supported Application XIP. Is there any other FS that also supports it? Ramdisk? Ramfs? Romfs?
On http://linuxvm.org/Patches/, you can find a file system called xip2fs, that uses an ext2 read-only fs for XIP. The code there works only if the backing memory is a zSeries DCSS memory segment, but it should be fairly easy to port to some other low-level memory provider.
Arnd <><