bade page state while calling munmap() for kmalloc'ed UIO memory
Hi,
I'm currently analysing a problem similar to some mmap() issue reported in the past: https://lkml.org/lkml/2010/7/11/140
So, what I'm trying to do is mapping some physically continuous memory (allocated by kmalloc) to userspace, using a trivial UIO driver (the idea is that a device can directly DMA to that buffer):
[...]
#define MEM_SIZE (4 * PAGE_SIZE)
addr = kmalloc(MEM_SIZE, GFP_KERNEL)
[...]
info.mem[0].addr = (unsigned long) addr;
info.mem[0].internal_addr = addr;
info.mem[0].size = MEM_SIZE;
info.mem[0].memtype = UIO_MEM_LOGICAL;
[...]
ret = uio_register_device(&pdev->dev, &info);
Userspace maps that memory range and writes its contents to a file:
[...]
fd = open("/dev/uio0", O_RDWR);
if (fd < 0) {
perror("Can't open UIO device\n");
exit(1);
}mem_map = mmap(NULL, MAP_SIZE, PROT_READ | PROT_WRITE,
MAP_PRIVATE, fd, 0);
if(mem_map == MAP_FAILED) {
perror("Can't map UIO memory\n");
ret = -ENOMEM;
goto out_file;
}[...]bytes_written = write(fd_file, mem_map, MAP_SIZE)
[...]
munmap(mem_map);
So, what happens is (I'm currently testing with 3.0.3 on ARM VersatilePB): When I do the munmap(), I run into the following error:
BUG: Bad page state in process uio_test pfn:078ed
page:c0409154 count:0 mapcount:0 mapping: (null) index:0x0
page flags: 0x284(referenced|slab|arch_1)
[<c0033e50>] (unwind_backtrace+0x0/0xe4) from [<c0079938>] (bad_page+0xcc/0xf8) [<c0079938>] (bad_page+0xcc/0xf8) from [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) from [<c007a778>] (free_hot_cold_page+0x20/0x18c) [<c007a778>] (free_hot_cold_page+0x20/0x18c) from [<c008ccb4>] (unmap_vmas+0x338/0x564) [<c008ccb4>] (unmap_vmas+0x338/0x564) from [<c008f0f4>] (unmap_region+0xa4/0x1e0) [<c008f0f4>] (unmap_region+0xa4/0x1e0) from [<c0090428>] (do_munmap+0x20c/0x274) [<c0090428>] (do_munmap+0x20c/0x274) from [<c00904cc>] (sys_munmap+0x3c/0x50) [<c00904cc>] (sys_munmap+0x3c/0x50) from [<c002e680>] (ret_fast_syscall+0x0/0x2c) Disabling lock debugging due to kernel taint
BUG: Bad page state in process uio_test pfn:078ee
page:c0409178 count:0 mapcount:0 mapping: (null) index:0x0
page flags: 0x284(referenced|slab|arch_1)
[<c0033e50>] (unwind_backtrace+0x0/0xe4) from [<c0079938>] (bad_page+0xcc/0xf8) [<c0079938>] (bad_page+0xcc/0xf8) from [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) from [<c007a778>] (free_hot_cold_page+0x20/0x18c) [<c007a778>] (free_hot_cold_page+0x20/0x18c) from [<c008ccb4>] (unmap_vmas+0x338/0x564) [<c008ccb4>] (unmap_vmas+0x338/0x564) from [<c008f0f4>] (unmap_region+0xa4/0x1e0) [<c008f0f4>] (unmap_region+0xa4/0x1e0) from [<c0090428>] (do_munmap+0x20c/0x274) [<c0090428>] (do_munmap+0x20c/0x274) from [<c00904cc>] (sys_munmap+0x3c/0x50) [<c00904cc>] (sys_munmap+0x3c/0x50) from [<c002e680>] (ret_fast_syscall+0x0/0x2c)
BUG: Bad page state in process uio_test pfn:078ef
page:c040919c count:0 mapcount:0 mapping: (null) index:0x0
page flags: 0x284(referenced|slab|arch_1)
[<c0033e50>] (unwind_backtrace+0x0/0xe4) from [<c0079938>] (bad_page+0xcc/0xf8) [<c0079938>] (bad_page+0xcc/0xf8) from [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) from [<c007a778>] (free_hot_cold_page+0x20/0x18c) [<c007a778>] (free_hot_cold_page+0x20/0x18c) from [<c008ccb4>] (unmap_vmas+0x338/0x564) [<c008ccb4>] (unmap_vmas+0x338/0x564) from [<c008f0f4>] (unmap_region+0xa4/0x1e0) [<c008f0f4>] (unmap_region+0xa4/0x1e0) from [<c0090428>] (do_munmap+0x20c/0x274) [<c0090428>] (do_munmap+0x20c/0x274) from [<c00904cc>] (sys_munmap+0x3c/0x50) [<c00904cc>] (sys_munmap+0x3c/0x50) from [<c002e680>] (ret_fast_syscall+0x0/0x2c)
This happens for every page except the first one. If I change the code and just touch the first page, everything's working fine. As soon as I touch one of the other pages, I can see the "bad page state error" for that page. The kernel is currently built using CONFIG_SLAB (my .config is based on the versatile_defconfig); if I change to CONFIG_SLUB, munmap() seems to be happy and I can't see the "bad page state" error.
Any idea what might be wrong here? Am I missing something obvious? (I've prepared some brown paperbags for that case ;-))
Thanks,
Jan
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
On Mon, Aug 29, 2011 at 05:05:47PM +0200, Jan Altenberg wrote:
[Since we got no reply on linux-mm, I added lkml and Andrew to Cc: (mm doesn't seem to have a maintainer...)]
Quoted 4 lines
Hi,
I'm currently analysing a problem similar to some
mmap()issue reported in the past: https://lkml.org/lkml/2010/7/11/140
The arch there was microblaze, and you are working on arm. That means the problem appears on at least to archs.
Quoted 35 lines
So, what I'm trying to do is mapping some physically continuous memory (allocated by kmalloc) to userspace, using a trivial UIO driver (the idea is that a device can directly DMA to that buffer):
[...]
#defineMEM_SIZE(4 *PAGE_SIZE)addr =
kmalloc(MEM_SIZE, GFP_KERNEL)
[...]
info.mem[0].addr = (unsigned long) addr;
info.mem[0].internal_addr = addr;
info.mem[0].size =MEM_SIZE;
info.mem[0].memtype =UIO_MEM_LOGICAL;
[...]
ret =uio_register_device(&pdev->dev, &info);Userspace maps that memory range and writes its contents to a file:
[...]
fd =
open("/dev/uio0", O_RDWR);if (fd < 0) { perror("Can't open UIO device\n"); exit(1); }
mem_map= mmap(NULL,MAP_SIZE,PROT_READ|PROT_WRITE,MAP_PRIVATE, fd, 0);if(mem_map == MAP_FAILED) { perror("Can't map UIO memory\n"); ret = -ENOMEM; goto out_file; }
Quoted 7 lines
[...]
bytes_written=write(fd_file, mem_map, MAP_SIZE)
[...]
munmap(mem_map);From my point of view (I've got Jan's full test case code), this
is a completely correct UIO use case.
Quoted 28 lines
So, what happens is (I'm currently testing with 3.0.3 on ARM VersatilePB): When I do the
munmap(), I run into the following error:BUG: Bad page state in process uio_test pfn:078edpage:c0409154 count:0 mapcount:0 mapping: (null) index:0x0
page flags: 0x284(referenced|slab|arch_1)
[<c0033e50>] (unwind_backtrace+0x0/0xe4) from [<c0079938>] (bad_page+0xcc/0xf8) [<c0079938>] (bad_page+0xcc/0xf8) from [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) from [<c007a778>] (free_hot_cold_page+0x20/0x18c) [<c007a778>] (free_hot_cold_page+0x20/0x18c) from [<c008ccb4>] (unmap_vmas+0x338/0x564) [<c008ccb4>] (unmap_vmas+0x338/0x564) from [<c008f0f4>] (unmap_region+0xa4/0x1e0) [<c008f0f4>] (unmap_region+0xa4/0x1e0) from [<c0090428>] (do_munmap+0x20c/0x274) [<c0090428>] (do_munmap+0x20c/0x274) from [<c00904cc>] (sys_munmap+0x3c/0x50) [<c00904cc>] (sys_munmap+0x3c/0x50) from [<c002e680>] (ret_fast_syscall+0x0/0x2c) Disabling lock debugging due to kernel taintBUG: Bad page state in process uio_test pfn:078eepage:c0409178 count:0 mapcount:0 mapping: (null) index:0x0
page flags: 0x284(referenced|slab|arch_1)
[<c0033e50>] (unwind_backtrace+0x0/0xe4) from [<c0079938>] (bad_page+0xcc/0xf8) [<c0079938>] (bad_page+0xcc/0xf8) from [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) from [<c007a778>] (free_hot_cold_page+0x20/0x18c) [<c007a778>] (free_hot_cold_page+0x20/0x18c) from [<c008ccb4>] (unmap_vmas+0x338/0x564) [<c008ccb4>] (unmap_vmas+0x338/0x564) from [<c008f0f4>] (unmap_region+0xa4/0x1e0) [<c008f0f4>] (unmap_region+0xa4/0x1e0) from [<c0090428>] (do_munmap+0x20c/0x274) [<c0090428>] (do_munmap+0x20c/0x274) from [<c00904cc>] (sys_munmap+0x3c/0x50) [<c00904cc>] (sys_munmap+0x3c/0x50) from [<c002e680>] (ret_fast_syscall+0x0/0x2c)BUG: Bad page state in process uio_test pfn:078ef
Quoted 10 lines
page:c040919c count:0 mapcount:0 mapping: (null) index:0x0
page flags: 0x284(referenced|slab|arch_1)
[<c0033e50>] (unwind_backtrace+0x0/0xe4) from [<c0079938>] (bad_page+0xcc/0xf8) [<c0079938>] (bad_page+0xcc/0xf8) from [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) [<c007a5f0>] (free_pages_prepare+0x6c/0xcc) from [<c007a778>] (free_hot_cold_page+0x20/0x18c) [<c007a778>] (free_hot_cold_page+0x20/0x18c) from [<c008ccb4>] (unmap_vmas+0x338/0x564) [<c008ccb4>] (unmap_vmas+0x338/0x564) from [<c008f0f4>] (unmap_region+0xa4/0x1e0) [<c008f0f4>] (unmap_region+0xa4/0x1e0) from [<c0090428>] (do_munmap+0x20c/0x274) [<c0090428>] (do_munmap+0x20c/0x274) from [<c00904cc>] (sys_munmap+0x3c/0x50) [<c00904cc>] (sys_munmap+0x3c/0x50) from [<c002e680>] (ret_fast_syscall+0x0/0x2c)
Quite strange that memory that could be mapped with mmap() cannot be unmapped with munmap().
Quoted 2 lines
This happens for every page except the first one.
...which is the next strange thing.
Quoted 4 lines
If I change the code
and just touch the first page, everything's working fine. As soon as I touch one of the other pages, I can see the "bad page state error" for that page.
The pages are mapped when you access them through the UIO core page fault handler.
Quoted 3 lines
The kernel is currently built using
CONFIG_SLAB(my .config is based on theversatile_defconfig); if I change toCONFIG_SLUB,munmap()seems to be happy and I can't see the "bad page state" error.
That is more than strange, that points to some things going really wrong.
Quoted 7 lines
Any idea what might be wrong here? Am I missing something obvious? (I've prepared some brown paperbags for that case ;-))
Thanks,
Jan
Thanks, Jan, for reporting this!
Hans
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerQuoted 52 lines · On Wed, 31 Aug 2011 11:58:25 +0200 "Hans J. Koch" <[email protected]> wrote:
On Mon, Aug 29, 2011 at 05:05:47PM +0200, Jan Altenberg wrote:
[Since we got no reply on linux-mm, I added lkml and Andrew to Cc: (mm doesn't seem to have a maintainer...)]
Quoted 4 lines
Hi,
I'm currently analysing a problem similar to some
mmap()issue reported in the past: https://lkml.org/lkml/2010/7/11/140The arch there was microblaze, and you are working on arm. That means the problem appears on at least to archs.
Quoted 39 lines
So, what I'm trying to do is mapping some physically continuous memory (allocated by kmalloc) to userspace, using a trivial UIO driver (the idea is that a device can directly DMA to that buffer):
[...]
#defineMEM_SIZE(4 *PAGE_SIZE)addr =
kmalloc(MEM_SIZE, GFP_KERNEL)
[...]
info.mem[0].addr = (unsigned long) addr;
info.mem[0].internal_addr = addr;
info.mem[0].size =MEM_SIZE;
info.mem[0].memtype =UIO_MEM_LOGICAL;
[...]
ret =uio_register_device(&pdev->dev, &info);Userspace maps that memory range and writes its contents to a file:
[...]
fd =
open("/dev/uio0", O_RDWR);if (fd < 0) { perror("Can't open UIO device\n"); exit(1); }
mem_map= mmap(NULL,MAP_SIZE,PROT_READ|PROT_WRITE,MAP_PRIVATE, fd, 0);if(mem_map == MAP_FAILED) { perror("Can't map UIO memory\n"); ret = -ENOMEM; goto out_file; }[...]
bytes_written=write(fd_file, mem_map, MAP_SIZE)
[...]
Quoted 12 lines
Quoted 3 lines
munmap(mem_map);From my point of view (I've got Jan's full test case code), this
is a completely correct UIO use case.
Quoted 7 lines
So, what happens is (I'm currently testing with 3.0.3 on ARM VersatilePB): When I do the
munmap(), I run into the following error:BUG: Bad page state in process uio_test pfn:078edpage:c0409154 count:0 mapcount:0 mapping: (null) index:0x0 page flags: 0x284(referenced|slab|arch_1)
PG_slab is set. The kernel is complaining because a page which was allocated via kmalloc/kmem_cache_alloc was directly passed to the page allocator for freeing. It should have been passed to kfree().
Presumably the uio driver expects that its memory was allocated via alloc_pages(), not via kmalloc().
Quoted 68 lines · Andrew Morton
Quoted 52 lines · On Wed, 31 Aug 2011 11:58:25 +0200 "Hans J. Koch" <[email protected]> wrote:
On Mon, Aug 29, 2011 at 05:05:47PM +0200, Jan Altenberg wrote:
[Since we got no reply on linux-mm, I added lkml and Andrew to Cc: (mm doesn't seem to have a maintainer...)]
Quoted 4 lines
Hi,
I'm currently analysing a problem similar to some
mmap()issue reported in the past: https://lkml.org/lkml/2010/7/11/140The arch there was microblaze, and you are working on arm. That means the problem appears on at least to archs.
Quoted 39 lines
So, what I'm trying to do is mapping some physically continuous memory (allocated by kmalloc) to userspace, using a trivial UIO driver (the idea is that a device can directly DMA to that buffer):
[...]
#defineMEM_SIZE(4 *PAGE_SIZE)addr =
kmalloc(MEM_SIZE, GFP_KERNEL)
[...]
info.mem[0].addr = (unsigned long) addr;
info.mem[0].internal_addr = addr;
info.mem[0].size =MEM_SIZE;
info.mem[0].memtype =UIO_MEM_LOGICAL;
[...]
ret =uio_register_device(&pdev->dev, &info);Userspace maps that memory range and writes its contents to a file:
[...]
fd =
open("/dev/uio0", O_RDWR);if (fd < 0) { perror("Can't open UIO device\n"); exit(1); }
mem_map= mmap(NULL,MAP_SIZE,PROT_READ|PROT_WRITE,MAP_PRIVATE, fd, 0);if(mem_map == MAP_FAILED) { perror("Can't map UIO memory\n"); ret = -ENOMEM; goto out_file; }[...]
bytes_written=write(fd_file, mem_map, MAP_SIZE)
[...]Quoted 12 lines
Quoted 3 lines
munmap(mem_map);From my point of view (I've got Jan's full test case code), this
is a completely correct UIO use case.
Quoted 7 lines
So, what happens is (I'm currently testing with 3.0.3 on ARM VersatilePB): When I do the
munmap(), I run into the following error:BUG: Bad page state in process uio_test pfn:078edpage:c0409154 count:0 mapcount:0 mapping: (null) index:0x0 page flags: 0x284(referenced|slab|arch_1)
Quoted 6 lines
PG_slabis set. The kernel is complaining because a page which was allocated via kmalloc/kmem_cache_alloc was directly passed to the page allocator for freeing. It should have been passed tokfree().Presumably the uio driver expects that its memory was allocated via
alloc_pages(), not viakmalloc().
Thanks for that hint. I'll check and will update UIO documentation accordingly if necessary.
BUT the following problems are still threatening my mental health:
If userspace gets a valid and working pointer from mmap(), it is entitled to expect munmap() to work on that pointer, too, isn't it? Shouldn't mmap() fail in such a case?
The kernel's behavior should be the same, no matter which SLAB or SLUB is chosen.
Or am I following wrong philosophies?
Thanks,
Hans
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>