Skip to contentSkip to conversation

Random "File size limit exceeded" under 2.4

DISCUSSION Derek Glidden 4 participants 4 messages Dec 5, 2001 – Dec 15, 2001

lore
Share & tools
lore
#0 lore raw

I've been experiencing random and occasional encounters with "File size limit exceeded" errors under 2.4 kernels when trying to make filesystems. Unfortunately, there's no real rhyme or reason when this occurs, except that - usually - if I recompile a kernel or do a big rsync on a box that isn't giving me these problems, and then try to mkfs something, I'll encounter it. And if I'm unable to mkfs a device because of this error - usually - if I reboot the box, it goes away.

Note that it has *nothing* to do with the device I'm trying to mkfs being over 2GB! Once I start experiencing this, I've had *any* device that I try to mkfs error out with SIGXFSZ until I rebooted. (Usually, of course. I hate bugs that aren't consistent....)

I know I've seen this with at least stock 2.4.10, 2.4.13 and 2.4.14 and also 2.4.10 and 2.4.14 with XFS patches. There may have been other earlier versions in there were I ran into it but ignored it or overlooked it. All our boxes here at work are more-or-less stock Debian 2.2 with upgraded modutils to support running 2.4 kernels.

I don't have any particular correlation between hardware except that all of them have had IDE drives in them so my only real suspicion is some bizarrity in the IDE drivers.

I've got a machine that's experiencing this problem right now and I've done "strace" on both mke2fs and mkreiserfs to see what the results would be and it's a little wierd and beyond my interpretation capabilities. Here are the last few lines from both of those straces:

mke2fs.strace:

lseek(4, 2015346688, SEEK_SET) = 2015346688
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32768
lseek(4, 2015379456, SEEK_SET) = 2015379456
write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
write(1, "\10\10\10\10\10", 5) = 5
write(1, "16/72", 5) = 5
_llseek(4, 18446744071562084352, [2147500032], SEEK_SET) = 0 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = -1 EFBIG (File too large)
--- SIGXFSZ (File size limit exceeded) ---
+++ killed by SIGXFSZ +++

mkreiserfs.strace:

_llseek(4, 1744830464, [1744830464], SEEK_SET) = 0 write(4, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
_llseek(4, 1879048192, [1879048192], SEEK_SET) = 0 write(4, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
_llseek(4, 2013265920, [2013265920], SEEK_SET) = 0 write(4, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 4096
_llseek(4, 18446744071562067968, [2147483648], SEEK_SET) = 0 write(4, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = -1 EFBIG (File too large)
--- SIGXFSZ (File size limit exceeded) ---
+++ killed by SIGXFSZ +++

And this is the output of "sfdisk -l" for that box:

# sfdisk -l

Disk /dev/hda: 1655 cylinders, 255 heads, 63 sectors/track Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls   #blocks   Id  System
/dev/hda1          0+     32      33-   265072   83  Linux
/dev/hda2         33      98      66    530145   82  Linux swap
/dev/hda3         99     164      66    530145   83  Linux
/dev/hda4        165    1654    1490  11968425    5  Extended
/dev/hda5        165+    413     249-  2000092   83  Linux
/dev/hda6        414+    479      66-   530144+  83  Linux
/dev/hda7        480+   1654    1175-  9438187   83  Linux

Now it looks to me that in both cases, the mkfs is trying to seek well beyond the end of the device for some reason. Except that the drive isn't anything unusual - it's a 13GB Maxtor 91360U4 and the IDE interface is reported as "VIA vt82c596a (rev 09) IDE UDMA33 controller on pci00:07.1" which the VIA line of controllers has generally been very well supported under Linux, in my experience. Plus, as I said, I've seen this on a number of boxen with different drives and IDE controllers in them, so I don't see it being particular to a specific drive or controller.

Signature · 8 lines
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
#!/usr/bin/perl -w
$_='while(read+STDIN,$_,2048){$a=29;$b=73;$c=142;$t=255;@t=map
{$_%16or$t^=$c^=($m=(11,10,116,100,11,122,20,100)[$_/16%8])&110;
$t^=(72,@z=(64,72,$a^=12*($_%16-2?0:$m&17)),$b^=$_%64?12:0,@z)
[$_%8]}(16..271);if((@a=unx"C*",$_)[20]&48){$h=5;$_=unxb24,join
"",@b=map{xB8,unxb8,chr($_^$a[--$h+84])}@ARGV;s/...$/1$&/;$d=
unxV,xb25,$_;$e=256|(ord$b[4])<<9|ord$b[3];$d=$d>>8^($f=$t&($d
Quoted 1 line
Quoted 1 line

12^$d>>4^$d^$d/8))<<17,$e=$e>>8^($t&($g=($q=$e>>14&7^$e)^$q*

8^$q<<6))<<9,$_=$t[$_]^(($h>>=8)+=$f+(~$g&$t))for@a[128..$#a]} print+x"C*",@a}';s/x/pack+/g;eval

usage: qrpff 153 2 8 105 225 < /mnt/dvd/VOB_FILENAME \
| extract_mpeg2 | mpeg2dec -

http://www.cs.cmu.edu/~dst/DeCSS/Gallery/
http://www.eff.org/ http://www.anti-dmca.org/

#1 lore raw
Quoted 4 lines · Derek Glidden

I've been experiencing random and occasional encounters with "File size limit exceeded" errors under 2.4 kernels when trying to make filesystems.

I don't know if anyone has come forth to fix this yet.

Apparently it's something to do with your shell setting rlimits, and block devices are (bogusly) honouring those settings.

The word is that if you log in as root' at the login prompt, rather than using su', the problem goes away.

#2 lore raw
In reply to Derek Glidden

In article <[email protected]>,

Quoted 11 lines · Andrew Morton
Quoted 4 lines · Derek Glidden

I've been experiencing random and occasional encounters with "File size limit exceeded" errors under 2.4 kernels when trying to make filesystems.

I don't know if anyone has come forth to fix this yet.

Apparently it's something to do with your shell setting rlimits, and block devices are (bogusly) honouring those settings.

Perhaps the old app is calling sys_old_getrlimit() from linux/kernel/sys.c. It truncates rlimits to 0x7FFFFFFF if it's bigger than that. 0x7FFFFFFF used to be the old RLIM_INFINITY in 2.2 [actually, ((long)(~0UL>>1))]. In 2.4, RLIM_INFINITY is (~0UL).

So if you call sys_setrlimit() with the old RLIM_INFINITY from 2.2 OR with the result from sys_old_getrlimit(), then the new limit will be 0x7FFFFFFF instead of unlimited.

Looks like someone forgot to implement sys_old_setrlimit(), which would have been the right thing to do.

Now all we can do is to hack sys_setrlimit and let it translate 0x7FFFFFFF to RLIM_INFINITY.

The following untested and uncompiled patch might do it, or not...

kernel/sys.c renamed from kernel/sys.c.orig +10 -0
@@ -1120,6 +1120,16 @@
 		return -EINVAL;
 	if(copy_from_user(&new_rlim, rlim, sizeof(*rlim)))
 		return -EFAULT;
+#if !defined(__ia64__)
+	/*
+	 * 	In 2.2, RLIMIT_INFINITY was defined as ((long)(~0UL>>1)).
+	 * 	Reckognize it and translate it to the new RLIMIT_INFINITY.
+	 */
+	if ((long)new_rlim.rlim_cur == ((long)(~0UL>>1)))
+		new_rlim.rlim_cur = RLIMIT_INFINITY;
+	if ((long)new_rlim.rlim_max == ((long)(~0UL>>1)))
+		new_rlim.rlim_max = RLIMIT_INFINITY;
+#endif
 	old_rlim = current->rlim + resource;
 	if (((new_rlim.rlim_cur > old_rlim->rlim_max) ||
 	     (new_rlim.rlim_max > old_rlim->rlim_max)) &&

Mike.

Signature · 2 lines
"Only two things are infinite, the universe and human stupidity,
 and I'm not sure about the former" -- Albert Einstein.
#3 lore raw
In reply to Derek Glidden

FWIW, I've been using this over-engineered, under-tested command line utility to run e2fsck on our older distributions:

http://web.irridia.com/info/linux/unflimit.c

It works like "nice" or "time", but sets the proper RLIM_INFINITY filesize limit for the inheriting subshell. For example:

unflimit e2fsck /dev/sda3

The boot-time fscks work fine, but in multiuser I'm extremely lucky to get it to work without this.

The kernel patch below would work, I expect, but I think it's not necessarily the Right Thing To Do. If Andrew is right, is anyone looking at why block devices (or fsck?) are honoring the filesize limit?

Thanks,

Signature · 2 lines
Ken.
[email protected]

On Wed, Dec 05, 2001 at 10:33:19PM +0000, Miquel van Smoorenburg wrote:

| In article <[email protected]>,
| Andrew Morton  <[email protected]> wrote:
| >Derek Glidden wrote:
| >> 
| >> I've been experiencing random and occasional encounters with "File size
| >> limit exceeded" errors under 2.4 kernels when trying to make
| >> filesystems.
| >
| >I don't know if anyone has come forth to fix this yet.
| >
| >Apparently it's something to do with your shell setting
| >rlimits, and block devices are (bogusly) honouring those
| >settings.
| 
| Perhaps the old app is calling sys_old_getrlimit() from
| linux/kernel/sys.c. It truncates rlimits to 0x7FFFFFFF
| if it's bigger than that. 0x7FFFFFFF used to be the old
| RLIM_INFINITY in 2.2 [actually, ((long)(~0UL>>1))]. In
| 2.4, RLIM_INFINITY is (~0UL).
| 
| So if you call sys_setrlimit() with the old RLIM_INFINITY from 2.2
| OR with the result from sys_old_getrlimit(), then the new limit
| will be 0x7FFFFFFF instead of unlimited.
| 
| Looks like someone forgot to implement sys_old_setrlimit(),
| which would have been the right thing to do.
| 
| Now all we can do is to hack sys_setrlimit and let it translate
| 0x7FFFFFFF to RLIM_INFINITY.
| 
| The following untested and uncompiled patch might do it, or not...
| 
| --- linux-2.4.17-pre2/kernel/sys.c.orig	Tue Sep 18 23:10:43 2001
| +++ linux-2.4.17-pre2/kernel/sys.c	Wed Dec  5 23:30:50 2001
| @@ -1120,6 +1120,16 @@
|  		return -EINVAL;
|  	if(copy_from_user(&new_rlim, rlim, sizeof(*rlim)))
|  		return -EFAULT;
| +#if !defined(__ia64__)
| +	/*
| +	 * 	In 2.2, RLIMIT_INFINITY was defined as ((long)(~0UL>>1)).
| +	 * 	Reckognize it and translate it to the new RLIMIT_INFINITY.
| +	 */
| +	if ((long)new_rlim.rlim_cur == ((long)(~0UL>>1)))
| +		new_rlim.rlim_cur = RLIMIT_INFINITY;
| +	if ((long)new_rlim.rlim_max == ((long)(~0UL>>1)))
| +		new_rlim.rlim_max = RLIMIT_INFINITY;
| +#endif
|  	old_rlim = current->rlim + resource;
|  	if (((new_rlim.rlim_cur > old_rlim->rlim_max) ||
|  	     (new_rlim.rlim_max > old_rlim->rlim_max)) &&
| 
| Mike.
| -- 
| "Only two things are infinite, the universe and human stupidity,
|  and I'm not sure about the former" -- Albert Einstein.
| 
| -
| To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
| the body of a message to [email protected]
| More majordomo info at  http://vger.kernel.org/majordomo-info.html
| Please read the FAQ at  http://www.tux.org/lkml/