PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

What does tmpfs have to do with ram size? Its swappable. This _might_ be useful for ramfs but for tmpfs, IMHO, its not a good idea.

Ed Tomlinson

The default size of a tmpfs filesystem is 50% of RAM. That can be overridden by setting the size explicitly, CaT is offering percent instead. Which is nice, and neatly done.

But I do agree with you that it seems a bit strange, not to take swap into account at all. That's one reason I never bothered to add the feature CaT proposes before. Just imagine you keep swap at some % of your RAM, then it'll all scale together.

Hugh

Basically:

tmpfs size < ram size + swap size

now, lets say a fair bit of your ram dies. you can't repolace it but the box will manage to run with your new ram size. if the tmpfs size is static then you get

tmpfs size > new ram size + swap size

and so any process can fill up your ram+swap by writing to tmpfs when you use tmpfs for /tmp.

By having the tmpfs size be a function of your ram size (as it is by default at 50%) you wont get that. Currently my tmpfs size is 63%. If half my ram dies I can still happily use my laptop without any fiddling because its new size will by 63% of the new ram size (so it'll be around 80MB rather then 160MB). Now, it does mean my /tmp is smaller but everything is still functional and it's bigger then my root partition, which I'd rather nto be actively writing to and only has 20MB free by design.

"Other countries of course, bear the same risk. But there's no doubt his
hatred is mainly directed at us. After all this is the guy who tried to
kill my dad."
        - George W. Bush Jr, Leader of the United States Regime
          September 26, 2002 (from a political fundraiser in Houston, Texas)

Hi Ed,

I agree and I think if you add this option it should adjust to a percentage of (ram + swap). With this it would be a really nice improvement.
I even had patches for this but to do it efficently you would need to add some hooks to swapon and swapoff.

Greetings
Christoph

You surprise me, Christoph, I'd expected you to approve of CaT's.

If tmpfs already defaulted to 50% of ram+swap, then I'd agree with you. But it has all along been in terms of RAM, so I think it's better to continue in that way. (We could add options to allow +swap in too, but I'm not terribly interested.)

If people really wanted their tmpfs pages to go out to disk, I think they'd be choosing a more sophisticated filesystem to manage that:
swap is a vital overflow area for tmpfs, not its home.

Hugh

Hi Hugh,

look back into your mail store. I said this pretty early.

No, I always thought the 50% RAM was a terrible hack. And IMHO it is one of the biggest usability gaps of tmpfs.

I definitely want them to go there. It is a very sophisticated mechinsm because by adding swap on _any_ storage I can grow my tmpfs instances even temporarily.

If you now were able to take advantage of additional swap automatically administration would be a no brainer. Also distributions could add much saner defaults for /dev/shm or even use it for /tmp.

Greetings
Christoph

I've never had an issue with it. I just made sure it was large enough.

-- wli

Sounds like what you want is a dynamically resizing tmpfs based on the amount of memory (ram+swap) available. That's a much bigger goose to fry I believe.

Now, even if the percentile patch took into account swap, you'd still need to remount tmpfs in order to get it to take into account of any swap you add on the fly.

I use it for /tmp now just fine. :) It's sized at 63% of 256MB of RAM.

"Other countries of course, bear the same risk. But there's no doubt his
hatred is mainly directed at us. After all this is the guy who tried to
kill my dad."
        - George W. Bush Jr, Leader of the United States Regime
          September 26, 2002 (from a political fundraiser in Houston, Texas)

No, that's why I said you would need hooks into swapon and swapoff. Then it would adjust on the fly. Else it's useless from the usability point of view. With these hooks it's easy to do.

And I have set it to 400MB on a 256MB box just fine ;-) How could you do these two setup generally without knowing your hardware. 20MB tmpfs on a 40MB machine can be a desaster btw.

Greetings
Christoph

Can you show me the easy part with this setup?
256MB RAM
512MB swap
50% tmpfs (384MB)
fill tmpfs completely
swapoff -a

Jörn

It's just what we asked for, but not what we want!
-- anonymous

Don't blame Christoph for that, one of these days I'll face up to my responsibilities and make swapoff fail (probably get itself OOM-killed) instead of having everything else OOM-killed.

Hugh

Ack. If you do that, I'd have to agree with Christoph.

Jörn

Homo Sapiens is a goal, not a description.
-- unknown

Hi Hugh,

The hooks for the accounting would solve this easily: the swapoff hook would realize that there is not enough space left for the tmpfs instance and simply return an error. So swapoff would fail. We would not even stress the vm to swapin all pages until realizingthat there is not enough memory left.

Greetings
Christoph

You're supposing that it's tmpfs causing this problem: not at all, that's just an easy way to show it. Take away tmpfs, and swapoff is still liable to hang, with other tasks OOMing (even when strict no-overcommit accounting: I've not yet added the check it needs).

Hugh

Hi Hugh,

No, I know tmpfs is not faulty :-)

But the hook would give tmpfs the chance to say stop before the mm layer even tries to do something and where it actually fails to do it right.

Greetings
Christoph

# swapoff -a
swapoff: device is busy

# umount /tmpfs
# swapoff -a
#

... in an ideal world, of course.

Regards
Henning

Dipl.-Inf. (Univ.) Henning P. Schmiedehausen          INTERMETA GmbH
[email protected]        +49 9131 50 654 0   http://www.intermeta.de/

Java, perl, Solaris, Linux, xSP Consulting, Web Services 
freelance consultant -- Jakarta Turbine Development  -- hero for hire

Ahh. Missed that. Sorry.

Yes. That'd solve that issue. You'd have it resize itself to your fixed % of the new totals.

:)

If you want to take swap into account you still can't but then this patch changes nothing for you in either case. You'd still specify 400MB or (now) 156%.

It can be, depending on what you do with it, obviously. That's not an argument against this and I'm not actually against it myself. I just want the current patch in cos at least it's a step in the right direction, it's simple and it ups the usability of tmpfs (and it stops me from having to patch my kernel each and every time ;).

Personally I'd like a flag to set wether or not to take it into account because it introduces a 2nd potential variable number into the equation and I don't want my setting depending on swap suddenly dropping off Gods green, somewhat polluted and messy, earth. I'd be in the same boat I was in before the patch.

"Other countries of course, bear the same risk. But there's no doubt his
hatred is mainly directed at us. After all this is the guy who tried to
kill my dad."
        - George W. Bush Jr, Leader of the United States Regime
          September 26, 2002 (from a political fundraiser in Houston, Texas)

Well, I got bored.

size=<ram>%+<swap>%

So size=100%+50% will set the tmpfs size at 100% the size of your ram
+ 50% the size of your swap.

It's just an idea. It's still simple and gives you what you seem to want (in part... there's no dynamic changes on swapon/off) and what I want (the ability to set the size off just real ram).

It seems like a decent compromise so I'm sure someone'll hate it. :)

Martin's distress was in contrast to the bitter satisfaction of some
of his fellow marines as they surveyed the scene. "The Iraqis are sick
people and we are the chemotherapy," said Corporal Ryan Dupre. "I am
starting to hate this country. Wait till I get hold of a friggin' Iraqi.
No, I won't get hold of one. I'll just kill him."
	- http://www.informationclearinghouse.info/article2479.htm

@@ -34,6 +34,7 @@
 #include <linux/writeback.h>
 #include <linux/vfs.h>
 #include <asm/uaccess.h>
+#include <asm/div64.h>
 
 /* This magic number is used in glibc for posix shared memory */
 #define TMPFS_MAGIC	0x01021994

@@ -1630,6 +1631,25 @@
 		if (!strcmp(this_char,"size")) {
 			unsigned long long size;
 			size = memparse(value,&rest);
+			if (*rest == '%') {
+				size <<= PAGE_SHIFT;
+				size *= totalram_pages;
+				do_div(size, 100);
+				rest++;
+				if(*rest == '+') {
+					unsigned long long swpsize;
+					rest++;
+					value = rest;
+					swpsize = memparse(value,&rest);
+					if (*rest == '%') {
+						swpsize <<= PAGE_SHIFT;
+						swpsize *= nr_swap_pages;
+						do_div(swpsize, 100);
+						rest++;
+					}
+					size+=swpsize;
+				}
+			}
 			if (*rest)
 				goto bad_val;
 			*blocks = size >> PAGE_CACHE_SHIFT;

@@ -1695,7 +1715,6 @@
 	uid_t uid = current->fsuid;
 	gid_t gid = current->fsgid;
 	struct shmem_sb_info *sbinfo;
-	struct sysinfo si;
 	int err = -ENOMEM;
 
 	sbinfo = kmalloc(sizeof(struct shmem_sb_info), GFP_KERNEL);

@@ -1708,8 +1727,7 @@
 	 * Per default we only allow half of the physical ram per
 	 * tmpfs instance
 	 */
-	si_meminfo(&si);
-	blocks = inodes = si.totalram / 2;
+	blocks = inodes = totalram_pages / 2;
 
 #ifdef CONFIG_TMPFS
 	if (shmem_parse_options(data, &mode, &uid, &gid, &blocks, &inodes)) {

Which means we have two orthogonal problems to deal with. Adding the percent option and "fixing" it to take swap into account.

Not completely orthogonal, though, since the "fix" actually changes semantics and might break some peoples setups. Take 3x50% of RAM and twice the RAM as swap, for example.

Jörn

They laughed at Galileo.  They laughed at Copernicus.  They laughed at
Columbus. But remember, they also laughed at Bozo the Clown.
-- unknown

This (using swap as part of the tmpfs type system) is what happens on a Sun. I was disappointed (surprised even) in the Linux implementations because mounting a truly temporary /tmp was what I wanted it for.

I would like to see a tmpfs (swapfs?) that did presume that files not in use (lately?) would migrate out of my valuable RAM and onto the super-cheap swap device.

IMHO of course.

Rob.

-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Christoph Rohland
Sent: Tuesday, April 01, 2003 8:28 AM
To: [email protected]
Cc: CaT; [email protected]; Hugh Dickins
Subject: Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

Hi Ed,

I agree and I think if you add this option it should adjust to a percentage of (ram + swap). With this it would be a really nice improvement.
I even had patches for this but to do it efficently you would need to add some hooks to swapon and swapoff.

Greetings
Christoph

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

The Linux implementation of tmpfs _does_ use swap:
tmpfs data pages go out to swap under memory pressure.

hmm, I think, on reflection, that I knew that because of having encountered the swapoff problem after putting a bunch of stuff in a tmpfs...

Regardless,

Since the tmpfs occupies both ram and swap, then having the absolute size limit based only on ram seems odd.

IMHO of course 8-)

Rob.

-----Original Message-----
From: Hugh Dickins [mailto:[email protected]]
Sent: Thursday, April 03, 2003 12:35 AM
To: Robert White
Cc: Christoph Rohland; [email protected]; CaT; [email protected]
Subject: RE: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

Sun.

The Linux implementation of tmpfs _does_ use swap:
tmpfs data pages go out to swap under memory pressure.

use

swap