Linux processes, tempfs and programs
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
This is not really PPC specific, but it is an embedded PPC system that I'm working with, so...
My embedded system is structured such that the main user-mode processes that are being run are downloaded and executed on demand. I'm currently downloading the executable to an ext3 system on the CompactFlash, but there is really no reason to use non-volatile memory so I'm thinking to download to a tempfs directory and execute from there.
But if I do that, I want to remove the program from the directory after I start it, so that the file does not take up ram space. Will that actually work? I'm using exec(2) to execute the program file wherever it is downloaded. Will a subsequent unlink of the file have a result, or will the file continue to take up space as backing store for the executable?
- --
Steve Williams "The woods are lovely, dark and deep. steve at icarus.com But I have promises to keep, http://www.icarus.com and lines to code before I sleep, http://www.picturel.com And lines to code before I sleep."
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFCFSrcrPt1Sc2b3ikRApXCAKDJpcfBMIGj7aOQ46hh19o0eOV2XgCgnprQ rIAecQJg+VpdFbo+gWXwWKk=
=p475
-----END PGP SIGNATURE-----
Quoted 7 lines · Stephen Williams
My embedded system is structured such that the main user-mode processes that are being run are downloaded and executed on demand. I'm currently downloading the executable to an ext3 system on the CompactFlash, but there is really no reason to use non-volatile memory so I'm thinking to download to a tempfs directory and execute from there.
Quoted 6 lines
But if I do that, I want to remove the program from the directory after I start it, so that the file does not take up ram space. Will that actually work? I'm using exec(2) to execute the program file wherever it is downloaded. Will a subsequent unlink of the file have a result, or will the file continue to take up space as backing store for the executable?
I think unlink will remove the file from directory (so you won't be able to see it with ls), but it will still continue to to take space - you're right it will be used as backing store, at least for read-only segments, which can be discarded if memory is tight. Even if you mlock all executable in memory, I think there will be still at least one reference to this file, which will prevent freeing tmpfs memory.
--
Eugene
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Eugene Surovegin wrote:
| On Thu, Feb 17, 2005 at 03:38:05PM -0800, Stephen Williams wrote: |>But if I do that, I want to remove the program from the directory |>after I start it, so that the file does not take up ram space. Will |>that actually work? I'm using exec(2) to execute the program file |>wherever it is downloaded. Will a subsequent unlink of the file |>have a result, or will the file continue to take up space as |>backing store for the executable? | | | I think unlink will remove the file from directory (so you won't be | able to see it with ls), but it will still continue to to take space - | you're right it will be used as backing store, at least for read-only | segments, which can be discarded if memory is tight. Even if you mlock | all executable in memory, I think there will be still at least one | reference to this file, which will prevent freeing tmpfs memory.
Which is what I thought, and why I didn't do it that way in the first place:-( That's 128+ Kbytes I'd rather have holding image data:-(((
I've got shared libraries on the CF disk, it's fine (preferable) if it pages out of them, but I don't want the program itself to reside anywhere but in its memory image. (And I don't want to go writing to the CF disk except for upgrades or sys admin stuff.)
- --
Steve Williams "The woods are lovely, dark and deep. steve at icarus.com But I have promises to keep, http://www.icarus.com and lines to code before I sleep, http://www.picturel.com And lines to code before I sleep."
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFCFTxvrPt1Sc2b3ikRAuy4AKCidKG/LYPDo9thwZag9S7JXLfSRACfUen6 N8ICW/n6BCR6OgJ5NMlW99Q=
=Fjct
-----END PGP SIGNATURE-----
Quoted 15 lines · Eugene Surovegin
Quoted 7 lines · Stephen Williams
My embedded system is structured such that the main user-mode processes that are being run are downloaded and executed on demand. I'm currently downloading the executable to an ext3 system on the CompactFlash, but there is really no reason to use non-volatile memory so I'm thinking to download to a tempfs directory and execute from there.
Quoted 6 lines
But if I do that, I want to remove the program from the directory after I start it, so that the file does not take up ram space. Will that actually work? I'm using exec(2) to execute the program file wherever it is downloaded. Will a subsequent unlink of the file have a result, or will the file continue to take up space as backing store for the executable?
Quoted 6 lines
I think unlink will remove the file from directory (so you won't be able to see it with ls), but it will still continue to to take space - you're right it will be used as backing store, at least for read-only segments, which can be discarded if memory is tight. Even if you mlock all executable in memory, I think there will be still at least one reference to this file, which will prevent freeing tmpfs memory.
A little correction, according to tmpfs doc, it lives completely in page cache, so I think memory is not wasted for unmodified sections of the loaded file (e.g. a second copy, when file is executed and loaded into user-space, isn't being made).
But as usual, make some measurements first :)
--
Eugene.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Eugene Surovegin wrote:
| On Thu, Feb 17, 2005 at 04:43:27PM -0800, Eugene Surovegin wrote: | |>On Thu, Feb 17, 2005 at 03:38:05PM -0800, Stephen Williams wrote:
|>>But if I do that, I want to remove the program from the directory |>>after I start it, so that the file does not take up ram space. Will |>>that actually work? I'm using exec(2) to execute the program file |>>wherever it is downloaded. Will a subsequent unlink of the file |>>have a result, or will the file continue to take up space as |>>backing store for the executable?
|>I think unlink will remove the file from directory (so you won't be |>able to see it with ls), but it will still continue to to take space - |>you're right it will be used as backing store, at least for read-only |>segments, which can be discarded if memory is tight. Even if you mlock |>all executable in memory, I think there will be still at least one |>reference to this file, which will prevent freeing tmpfs memory. | | | A little correction, according to tmpfs doc, it lives completely in | page cache, so I think memory is not wasted for unmodified sections of | the loaded file (e.g. a second copy, when file is executed and loaded | into user-space, isn't being made). | | But as usual, make some measurements first :)
So far as I can see in my simplistic tests, this still takes twice as much memory as leaving the executable on a disk somewhere. This seems to be an inevitable consequence of execve on Linux 2.4.x. That's a bummer. Any other suggestions?
- --
Steve Williams "The woods are lovely, dark and deep. steve at icarus.com But I have promises to keep, http://www.icarus.com and lines to code before I sleep, http://www.picturel.com And lines to code before I sleep."
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFCFjrjrPt1Sc2b3ikRAjwvAKCogiZ5i+OHdsGskhsfC97+jxl/ygCgqAql P77LUWxMW87T7ReXVg27fj8=
=j9KI
-----END PGP SIGNATURE-----