mmap()ing a size-extended file on a 100% full tmpfs
Hello,
If you try to run the following program with /dev/shm being 100% full, it will be terminated by a SIGBUS in memset():
"""
#define _GNU_SOURCE
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <stdio.h>
#include <fcntl.h>
#include <sys/mman.h>
int main(void)
{
int fd = shm_open("segg", O_CREAT|O_RDWR, 0666);
printf("fd: %d\n", fd);
printf("truncate: %d\n", ftruncate(fd, 1024*1024));
// errno = posix_fallocate(fd, 0, 1024*1024);
// printf("falloc: %s\n", strerror(errno));
void *ptr = mmap(NULL, 1024*1024, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
printf("ptr: %p\n", ptr);
memset(ptr, 0, 1024*1024);
return 0;
}"""
On a similarly full ext2 file system memset() completes successfully (though I'm not sure whether it made through it by mere chance).
So the probelm is that the program may not know at all what the underlying file system is, and in case of tmpfs it may be terminated for a completely unexpected reason.
A portable solution could be to [posix_]fallocate() the file before trying to mmap() it. That works (except that perhaps tmpfs can deallocate memory if it's under pressure).
Alternatively I could imagine such an ftruncate() implementation for tmpfs, which would incorporate fallocate()ion.
In combination with this mmap() could refuse the operation if insufficient backing store is available. Ie. it would return MAP_FAILED if the programmer didn't call ftruncate() which would include fallocate().
The wayland developers faced the same problem last year:
http://lists.freedesktop.org/archives/wayland-devel/2013-October/011501.html
My opinion is that either ftruncate() or mmap() should return an error. What do you think?
Signature · 7 lines
adam -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 6 lines · Adam Endrodi
Hello,
If you try to run the following program with
/dev/shmbeing 100% full, it will be terminated by a SIGBUS inmemset():
Yes.
Quoted 24 lines
"""
#define_GNU_SOURCE
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <stdio.h>
#include <fcntl.h>
#include <sys/mman.h>int main(void)
{ int fd = shm_open("segg", O_CREAT|O_RDWR, 0666); printf("fd: %d\n", fd); printf("truncate: %d\n", ftruncate(fd, 1024*1024)); // errno = posix_fallocate(fd, 0, 1024*1024); // printf("falloc: %s\n", strerror(errno)); void *ptr = mmap(NULL, 1024*1024, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0); printf("ptr: %p\n", ptr); memset(ptr, 0, 1024*1024); return 0; }
Quoted 4 lines
"""
On a similarly full ext2 file system
memset()completes successfully (though I'm not sure whether it made through it by mere chance).
Well, I changed your "shm_open" above to "open", and ran from a full ext2 root, and from a full ext4 root: in each case got the same SIGBUS myself.
I wonder if you freed the space in your /dev/shm, and then ran the program from a full ext2 root, but forgot that you were using shm_open(), which was creating the file over in /dev/shm again.
Quoted 4 lines
So the probelm is that the program may not know at all what the underlying file system is, and in case of tmpfs it may be terminated for a completely unexpected reason.
It is normal to SIGBUS on an mmap'ed region when the filesystem cannot provide backing store - usually because it has run out of space.
Quoted 4 lines
A portable solution could be to [posix_]fallocate() the file before trying to
mmap()it. That works (except that perhaps tmpfs can deallocate memory if it's under pressure).
Yes, please do use fallocate (or posix_fallocate) for that:
tmpfs will not deallocate what has been allocated.
Quoted 3 lines
Alternatively I could imagine such an
ftruncate()implementation for tmpfs, which would incorporate fallocate()ion.
No, ftruncate and fallocate have different semantics: fallocate is the right one to use if you want to allocate the space in advance.
Quoted 4 lines
In combination with this
mmap()could refuse the operation if insufficient backing store is available. Ie. it would returnMAP_FAILEDif the programmer didn't callftruncate()which would includefallocate().
That would be a non-standard change in behaviour for mmap, and would make some accepted uses of sparse (holey) files impossible.
Quoted 3 lines
The wayland developers faced the same problem last year:
http://lists.freedesktop.org/archives/wayland-devel/2013-October/011501.html
Thanks for the link: which concludes that a fallocate is what's needed.
Quoted 3 lines
My opinion is that either
ftruncate()ormmap()should return an error. What do you think?
I agree that SIGBUS can often be an unwelcome surprise, but disagree that we should give ftruncate or mmap different behaviour on tmpfs.
Hugh
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>