Skip to contentSkip to conversation

What File System supports Application XIP

DISCUSSION colin 8 participants 15 messages Sep 9, 2004 – Sep 11, 2004

lore
Share & tools
lore

colin

author
#0 lore raw

Hi there,
We are developing embedded Linux system. Performance is our consideration. We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in RAM, and run in XIP (eXecute In Place) manners.
I know that Cramfs has supported Application XIP. Is there any other FS that also supports it? Ramdisk? Ramfs? Romfs?

Thanks and regards,
Colin

#1 lore raw
Quoted 3 lines · colin

Hi there,
We are developing embedded Linux system. Performance is our consideration. We hope some applications can run as fast as possible,

well ramfs by definition is XIP :)

but I guess the filesystem comes from flash somewhere at which point jffs2 with compression might be a better choice; if you have enough ram then the apps run from the pagecache anyway and compression keeps you from transfering too much data from the slower flash. It's not XIP but I don't think you really want XIP...

author
#2 lore raw

Hi,
How does ramfs offer application XIP ability?
I mean, when the ramfs image is mounted and the application in it is executed,
"exec", which is called by sh, should first copy every section of the application to RAM and then jump to the text section.
How do I avoid the stage copying text section to RAM?

Thanks and regards,
Colin

----- Original Message -----
From: "Arjan van de Ven" <[email protected]>
To: "colin" <[email protected]>
Cc: <[email protected]>
Sent: Thursday, September 09, 2004 5:18 PM
Subject: Re: What File System supports Application XIP

Quoted 3 lines · colin
Quoted 2 lines

Hi there,
We are developing embedded Linux system. Performance is our

consideration.

Quoted 9 lines
Quoted 1 line

We hope some applications can run as fast as possible,

well ramfs by definition is XIP :)

but I guess the filesystem comes from flash somewhere at which point jffs2 with compression might be a better choice; if you have enough ram then the apps run from the pagecache anyway and compression keeps you from transfering too much data from the slower flash. It's not XIP but I don't think you really want XIP...

#3 lore raw
Quoted 7 lines · colin

Hi,
How does ramfs offer application XIP ability?
I mean, when the ramfs image is mounted and the application in it is executed,
"exec", which is called by sh, should first copy every section of the application to RAM and then jump to the text section.
How do I avoid the stage copying text section to RAM?

this is not how linux works. programs execute directly from the pagecache without copy.

#4 lore raw
Quoted 11 lines · Arjan van de Ven
Quoted 7 lines · colin

Hi,
How does ramfs offer application XIP ability?
I mean, when the ramfs image is mounted and the application in it is executed,
"exec", which is called by sh, should first copy every section of the application to RAM and then jump to the text section.
How do I avoid the stage copying text section to RAM?

this is not how linux works. programs execute directly from the pagecache without copy.

Most other filesystems populate the pagecache with I/O, presumably. In the case of a ramfs, is the page mapped directly from the fs into the pagecache without a copy?

=============================
Tim Bird
Architecture Group Co-Chair, CE Linux Forum
Senior Staff Engineer, Sony Electronics
E-mail: [email protected]
=============================

#5 lore raw
Quoted 4 lines

Most other filesystems populate the pagecache with I/O, presumably. In the case of a ramfs, is the page mapped directly from the fs into the pagecache without a copy?

actually ramfs exclusively lives in the pagecache.

#6 lore raw
In reply to Tim Bird
Quoted 3 lines · Tim Bird

Most other filesystems populate the pagecache with I/O, presumably. In the case of a ramfs, is the page mapped directly from the fs into the pagecache without a copy?

ramfs _is_ pagecache.

it is cool like that.

Robert Love

author
#7 lore raw

Hi Robert,
Sorry. I donot quite understand. My English is poor... :-( Do you mean there wonot be another duplicated text section in RAM when application in Ramfs is executed?

Regards,
Colin

----- Original Message -----
From: "Robert Love" <[email protected]>
To: "Tim Bird" <[email protected]>
Cc: <[email protected]>; "colin" <[email protected]>; <[email protected]>
Sent: Friday, September 10, 2004 1:38 AM
Subject:  [*垃圾郵件*] Re: What File System supports Application XIP

Quoted 14 lines · Tim Bird
Quoted 4 lines

Most other filesystems populate the pagecache with I/O, presumably. In the case of a ramfs, is the page mapped directly from the fs into the pagecache without a copy?

ramfs _is_ pagecache.

it is cool like that.

Robert Love

#8 lore raw
Quoted 3 lines · colin

Sorry. I donot quite understand. My English is poor... :-( Do you mean there wonot be another duplicated text section in RAM when application in Ramfs is executed?

This is correct, Colin.

Ramfs is essentially an in-memory filesystem implemented in the page cache.

Robert Love

#9 lore raw
In reply to colin
Quoted 8 lines · colin

Hi there,
We are developing embedded Linux system. Performance is our consideration. We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in RAM, and run in XIP (eXecute In Place) manners.
I know that Cramfs has supported Application XIP. Is there any other FS that also supports it? Ramdisk? Ramfs? Romfs?

Obvisously cramfs can not support XIP, because the "in-place" image is compressed (unless you have a processor that can execute compressed code :)

AFAIK only tmpfs supports XIP because it works on a higher level without using block devices underneath. Ramdisks are simply RAM block devices that behave like any other block device.

You can have a compressed image in flash (for instance), decompress everything into a tmpfs and execute from there.

I'm not sure, however, that this will be such a performance gain.

If you use cramfs (for instance) then the kernel will uncompress and run only the pages that are needed, and they will be cached in page cache so that they will be available again when needed. This way you only waste the RAM you actually need, and can still drop old pages if the application needs more RAM.

Just my two cents,

Signature · 4 lines
Paulo Marques - www.grupopie.com

To err is human, but to really foul things up requires a computer.
Farmers' Almanac, 1978
#10 lore raw
Quoted 10 lines · Paulo Marques
Quoted 8 lines · colin

Hi there,
We are developing embedded Linux system. Performance is our consideration. We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in RAM, and run in XIP (eXecute In Place) manners.
I know that Cramfs has supported Application XIP. Is there any other FS that also supports it? Ramdisk? Ramfs? Romfs?

Quoted 3 lines

Obvisously cramfs can not support XIP, because the "in-place" image is compressed (unless you have a processor that can execute compressed code :)

Actually there are hacks floating around which do let you use XIP with cramfs -- obviously you have to dispense with compression for the files you want to access that way.

You won't gain at runtime by using XIP though. Your code and data will end up in RAM _whatever_ file system you use, and you'll be running from page cache.

You may get a _slightly_ faster startup time after reboot if you use XIP from flash, because it doesn't have to be loaded into RAM first. But that comes at the cost of making it all a low slower during normal operation -- it's a lot slower to fetch icache lines from flash than it is from RAM.

Arjan's right -- you almost certainly don't really want XIP.

Signature · 1 line
dwmw2
#11 lore raw
Quoted 11 lines · David Woodhouse
Quoted 10 lines · Paulo Marques
Quoted 8 lines · colin

Hi there,
We are developing embedded Linux system. Performance is our consideration. We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in RAM, and run in XIP (eXecute In Place) manners.
I know that Cramfs has supported Application XIP. Is there any other FS that also supports it? Ramdisk? Ramfs? Romfs?

Quoted 7 lines
Quoted 3 lines

Obvisously cramfs can not support XIP, because the "in-place" image is compressed (unless you have a processor that can execute compressed code :)

Actually there are hacks floating around which do let you use XIP with cramfs -- obviously you have to dispense with compression for the files you want to access that way.

The patches I've seen require setting the CRAMFS_LINEAR option, to turn on linear addressing for cramfs, and CRAMFS_LINEAR_XIP. The result of these is to dispense with compression.

Quoted 4 lines

You won't gain at runtime by using XIP though. Your code and data will end up in RAM _whatever_ file system you use, and you'll be running from page cache.

Really? I thought that when you XIPed from flash, the page was mapped directly back to the flash memory address. There shouldn't be a RAM copy of the page at all.

Quoted 11 lines · Greg Ungerer (uClinux maintainer) once

[Application XIP provides a] win of keeping the application code in flash even when that is shared. Can make a difference on small memory systems.

It also helps alleviate the contiguous memory problem (or memory fragmentation if you prefer) when you don't have an MMU. We need to be able to allocate a big enough contigous memory region to load the text into. Can be a problem on systems that have been running for a while and free memory is fragmented. If the application can be run XIP from flash then you at least don't need to worry about that. (This is a very real problem on small RAM systems).

In the CE Linux Forum, we've been investigating Application XIP for two purposes: reduction in RAM footprint, and faster startup time. However, those both come at the sacrifice of runtime performance (as David says below). Running XIP from a RAM-based filesystem, as originally proposed, might solve the performance problem, but it wouldn't improve space utilization, and since you have to populate the RAM from somewhere on startup anyway, I don't think it will improve your bootup time. It _might_ improve per-application startup time, once the system is running, but I think a regular RAM disk suffices for that.

Quoted 5 lines

You may get a _slightly_ faster startup time after reboot if you use XIP from flash, because it doesn't have to be loaded into RAM first. But that comes at the cost of making it all a low slower during normal operation -- it's a lot slower to fetch icache lines from flash than it is from RAM.

FYI - Here are some rough numbers:
Time to run shell script which starts TinyX X server and "xsetroot -solid red", then shuts down:

First invocation: Non-XIP 3.195 seconds, XIP 2.035 seconds Second invocation: Non-XIP 1.744 seconds, XIP 1.765 seconds

I think this was on a 133 MHz PPC, but I'm not positive. In both cases the filesystem was in flash. Note that once the application pages are in RAM in the page cache, the Non-XIP case beats the XIP case (probably due to the penalty to access flash memory).

So the only performance win is on the first invocation of the application.

I'm just now starting to put together a wiki page of information about Application XIP, for those interested. (It has nothing now, so don't get excited.) But contributions are welcome at:
http://tree.celinuxforum.org/pubwiki/moin.cgi/ApplicationXIP
(Maybe when someone googles this message a year from now, there will be something interesting there... ;-)

=============================
Tim Bird
Architecture Group Co-Chair, CE Linux Forum
Senior Staff Engineer, Sony Electronics
E-mail: [email protected]
=============================

#12 lore raw

Tim Bird wrote:
...

Quoted 3 lines

The patches I've seen require setting the CRAMFS_LINEAR option, to turn on linear addressing for cramfs, and CRAMFS_LINEAR_XIP. The result of these is to dispense with compression.

Compression is skipped for the XIP files, which are typically marked via the sticky bit. You'll also need a version of mkcramfs that creates the image without compressing those files.

...

Quoted 9 lines

FYI - Here are some rough numbers:
Time to run shell script which starts TinyX X server and "xsetroot -solid red", then shuts down:

First invocation: Non-XIP 3.195 seconds, XIP 2.035 seconds Second invocation: Non-XIP 1.744 seconds, XIP 1.765 seconds

I think this was on a 133 MHz PPC, but I'm not positive. In both cases the filesystem was in flash.

It was measured on a 168MHz ARM 925T TI OMAP 1510.

Others' advice that "you probably don't want XIP" is true in most cases.
But in producing a battery-operated product with certain requirements
for performance, power savings (due to reduced RAM requirements), startup time (depending on the platform and software stack the difference can be significant), etc. XIP is an option chosen by some CE designers, who are willing to accept the performance penalty on a product that will still run adequately for its intended uses. It would be interesting to see an in-depth analysis of these topics on an actual Linux-based product such as a cell phone. There are, of course, a number of ways to address all these issues, not just XIP...

Signature · 2 lines
Todd Poynor
MontaVista Software
author
#13 lore raw

Re: ¡@[*©U§£¶l¥ó*] Re: What File System supports Application XIP

In reply to Paulo Marques

Hi Paulo,
Cramfs seems to support it now.
At the beginning, I also think it's impossible to support Application XIP in Cramfs.
http://www.denx.de/twiki/publish/DULG/ConfigureLinuxForXIP.html

Regards,
Colin

----- Original Message -----
From: "Paulo Marques" <[email protected]>
To: "colin" <[email protected]>
Cc: <[email protected]>
Sent: Thursday, September 09, 2004 5:19 PM
Subject: ¡@[*©U§£¶l¥ó*] Re: What File System supports Application XIP

Quoted 4 lines
Quoted 3 lines · colin

Hi there,
We are developing embedded Linux system. Performance is our

consideration.

Quoted 4 lines
Quoted 4 lines

We hope some applications can run as fast as possible,
and are think if they can be put in a filesystem image, which resides in RAM, and run in XIP (eXecute In Place) manners.
I know that Cramfs has supported Application XIP. Is there any other FS

that

Quoted 29 lines
Quoted 1 line

also supports it? Ramdisk? Ramfs? Romfs?

Obvisously cramfs can not support XIP, because the "in-place" image is compressed (unless you have a processor that can execute compressed code :)

AFAIK only tmpfs supports XIP because it works on a higher level without using block devices underneath. Ramdisks are simply RAM block devices that behave like any other block device.

You can have a compressed image in flash (for instance), decompress everything into a tmpfs and execute from there.

I'm not sure, however, that this will be such a performance gain.

If you use cramfs (for instance) then the kernel will uncompress and run only the pages that are needed, and they will be cached in page cache so that they will be available again when needed. This way you only waste the RAM you actually need, and can still drop old pages if the application needs more RAM.

Just my two cents,

Signature · 4 lines
Paulo Marques - www.grupopie.com

To err is human, but to really foul things up requires a computer.
Farmers' Almanac, 1978
#14 lore raw
In reply to colin
Quoted 2 lines · colin

I know that Cramfs has supported Application XIP. Is there any other FS that also supports it? Ramdisk? Ramfs? Romfs?

On http://linuxvm.org/Patches/, you can find a file system called xip2fs, that uses an ext2 read-only fs for XIP. The code there works only if the backing memory is a zSeries DCSS memory segment, but it should be fairly easy to port to some other low-level memory provider.

	Arnd <><