Skip to contentSkip to conversation

Use of SIGXFSZ outside of soft limits

DISCUSSION Micah Cowan 2 participants 4 messages Apr 10, 2007 – May 20, 2007

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hello,

I would like to object to a non-standard and, IMHO, ill-advised application of SIGXFSZ to non-rlimit-related filesize limits, and request that its use be minimized as much as possible. SUSv3 does not sanction its use in any situation other than that of exceeding the "soft file size limit for the process". While, of course, we are free to do as we like, and ignore the standards when it is advisable to do so, in this case, it seems fairly clear that sending an unexpected signal whose default action is to dump core, instead of the standard-approved behavior of returning an error signal and setting errno to EFBIG, is a move in the wrong direction. I'd say it's the wrong thing to do for resource-limited file sizes as well, but that at least is standard-mandated, and I guess we're pretty much stuck with it.

The problem, currently, is that users are encountering issues when they use cp or mv, etc, to transfer large files from one filesystem to another (for example, from ext3 to vfat). If the file size is greater than 4g, cp will abort with SIGXFSZ. This is annoying to users, particularly if there was more than one source file for the command (a hierarchical copy, for instance). In such a case, they would normally expect failure for the one problem source file, followed by continued processing of the remaining source files.

Now, of course, the coreutils guys could block or handle SIGXFSZ, which is exactly what I proposed when I was under my initial impression that the OS had license to send SIGXFSZ under these circumstances. Also, users could wrap the binaries with their own program that first blocks the signal, allowing it to get an EFBIG error instead.

The problem is, that in order to avoid abnormal termination, every program that ever uses write() would need to trap SIGXFSZ: "fixing" cp, mv, dd and install would only help alleviate the problem for the most "likely" programs to encounter the issue.

Furthermore, in addition to being (IMO) the wrong thing to do from a practical usability standpoint, it is also the wrong thing to do from a standards-conformance standpoint. If one looks at what SUSv3 has to say about it at http://www.opengroup.org/onlinepubs/000095399/functions/write.html :

<<<
If a write() requests that more bytes be written than there is room for (for example, [XSI] [Option Start] the process' file size limit or [Option End] the physical end of a medium), only as many bytes as there is room for shall be written. For example, suppose there is space for 20 bytes more in a file before reaching a limit. A write of 512 bytes will return 20. The next write of a non-zero number of bytes would give a failure return (except as noted below).

Quoted 1 line
Quoted 1 line
Quoted 1 line

Among the exceptions "noted below" is:

<<<
[XSI] [Option Start] If the request would cause the file size to exceed the soft file size limit for the process and there is no room for any bytes to be written, the request shall fail and the implementation shall generate the SIGXFSZ signal for the thread. [Option End]

Quoted 1 line
Quoted 1 line
Quoted 1 line

but no exception is made for file size limits other than "soft... process" limits; thus the requirement that the "next write of a non-zero number of bytes would give a failure return" still stands.

I would therefore propose that we eliminate the signalling of SIGXFSZ in all cases other than the case of the requested write causing a file's size to exceed limits set by setrlimit(RLIMIT_FSIZE, ...). I'd be happy to submit a patch to this effect if there is agreement that this is the right thing to do.

( Brief discussion with coreutils folks: http://lists.gnu.org/archive/html/bug-coreutils/2007-04/msg00070.html )

Signature · 3 lines
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer...
http://micah.cowan.name/
#1 lore raw
Quoted 5 lines

[XSI] [Option Start] If the request would cause the file size to exceed the soft file size limit for the process and there is no room for any bytes to be written, the request shall fail and the implementation shall generate the SIGXFSZ signal for the thread. [Option End]

Quoted 1 line
Quoted 1 line
Quoted 1 line

This all depends which document and version you review. AIX for example has or had the same behaviour as Linux which comes from the Large File Summit and indeed our implementation was carefully tested to pass the test suite of the time.

SUSv3 seems to subsume the older LFS standards, and has adjusted them somewhat in the merging so there may well be a good case for normalizing our behaviour to match SUSv3. Run some tests and send patches.

Alan

#2 lore raw
Quoted 8 lines · Alan Cox
Quoted 2 lines

[XSI] [Option Start] If the request would cause the file size to exceed the soft file size limit for the process and there is no room for any bytes to be written, the request shall fail and the implementation shall generate the SIGXFSZ signal for the thread. [Option End]

Quoted 1 line
Quoted 1 line
Quoted 1 line

This all depends which document and version you review. AIX for example has or had the same behaviour as Linux which comes from the Large File Summit and indeed our implementation was carefully tested to pass the test suite of the time.

Quoted 5 lines

SUSv3 seems to subsume the older LFS standards, and has adjusted them somewhat in the merging so there may well be a good case for normalizing our behaviour to match SUSv3. Run some tests and send patches.

Alan

Thanks very much for this response, Alan.

I kind of suspected it might be something like this. I'm relieved to know that the original reasons for signaling that on other cases may no longer apply.

I'll plan to be back with patches, then! :)

-Micah

#3 lore raw
Quoted 29 lines · Micah Cowan
Quoted 12 lines · Alan Cox
Quoted 6 lines

[XSI] [Option Start] If the request would cause the file size to exceed the soft file size limit for the process and there is no room for any bytes to be written, the request shall fail and the implementation shall generate the SIGXFSZ signal for the thread. [Option End]

Quoted 1 line
Quoted 1 line
Quoted 1 line

This all depends which document and version you review. AIX for example has or had the same behaviour as Linux which comes from the Large File Summit and indeed our implementation was carefully tested to pass the test suite of the time.

Quoted 5 lines

SUSv3 seems to subsume the older LFS standards, and has adjusted them somewhat in the merging so there may well be a good case for normalizing our behaviour to match SUSv3. Run some tests and send patches.

Alan

Thanks very much for this response, Alan.

I kind of suspected it might be something like this. I'm relieved to know that the original reasons for signaling that on other cases may no longer apply.

I'll plan to be back with patches, then! :)

(quoted diff) +0 -2
-Micah
-
Quoted 4 lines

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Sorry it's taken this long. The patch seems to work well, and the changes are _quite_ trivial.

fs/ncpfs/file.c +0 -2
diff -ru linux-2.6.20.6-orig/fs/ncpfs/file.c linux-2.6.20.6/fs/ncpfs/file.c
@@ -203,7 +203,6 @@
 
 	if (pos + count > MAX_NON_LFS && !(file->f_flags&O_LARGEFILE)) {
 		if (pos >= MAX_NON_LFS) {
-			send_sig(SIGXFSZ, current, 0);
 			return -EFBIG;
 		}
 		if (count > MAX_NON_LFS - (u32)pos) {
@@ -212,7 +211,6 @@
 	}
 	if (pos >= inode->i_sb->s_maxbytes) {
 		if (count || pos > inode->i_sb->s_maxbytes) {
-			send_sig(SIGXFSZ, current, 0);
 			return -EFBIG;
 		}
 	}
fs/reiserfs/file.c +0 -1
diff -ru linux-2.6.20.6-orig/fs/reiserfs/file.c linux-2.6.20.6/fs/reiserfs/file.c
@@ -1323,7 +1323,6 @@
 	if (get_inode_item_key_version (inode) == KEY_FORMAT_3_5 &&
 	    *ppos + count > MAX_NON_LFS) {
 		if (*ppos >= MAX_NON_LFS) {
-			send_sig(SIGXFSZ, current, 0);
 			return -EFBIG;
 		}
 		if (count > MAX_NON_LFS - (unsigned long)*ppos)
mm/filemap.c +0 -2
diff -ru linux-2.6.20.6-orig/mm/filemap.c linux-2.6.20.6/mm/filemap.c
@@ -1971,7 +1971,6 @@
 	if (unlikely(*pos + *count > MAX_NON_LFS &&
 				!(file->f_flags & O_LARGEFILE))) {
 		if (*pos >= MAX_NON_LFS) {
-			send_sig(SIGXFSZ, current, 0);
 			return -EFBIG;
 		}
 		if (*count > MAX_NON_LFS - (unsigned long)*pos) {
@@ -1989,7 +1988,6 @@
 	if (likely(!isblk)) {
 		if (unlikely(*pos >= inode->i_sb->s_maxbytes)) {
 			if (*count || *pos > inode->i_sb->s_maxbytes) {
-				send_sig(SIGXFSZ, current, 0);
 				return -EFBIG;
 			}
 			/* zero-length writes at ->s_maxbytes are OK */