Short question about the mmap method

Hi lkml readers,

I have a question regarding drivers implementing the mmap and nopage methods. In some references I've read that pages in kernel allocated memory (either allocated with kmalloc, vmalloc or __get_free_pages) should be set to reserved (mem_map_reserve or set_bit(PG_reserved, page->flags) before they can be mmap'ed to guarantee that they can't be swapped out. Is this true ?

The reason I ask is that I have a test driver that allocates some pages with vmalloc(), reserves them with mem_map_reserve(), and uses the "nopage" method to give a userspace app access to them. When the userpace app accesses the page, the "nopage" function is invoked as expected and the page->count is incremented (by the nopage function). When this application exits the page->count should have been decremented by the kernel, but it seems like it doesn't since the page is reserved and this causes a giant memleak when the driver is unloaded since the pages are not put back to the free list unless page->count is zero (I do mem_map_unreserve before vfree). If I avoid using mem_map_reserve (and _unreserve) the page count is 1 at the time I unload the driver and everything goes fine.

The kernel I'm using is RedHat 7.2 2.4.9-12

One more question:

When getting the "struct page" for pages allocated with vmalloc() I use the same method as drivers/media/video/bttv-driver.c (checking the page table). Somewhere (I think Rubini) I read that the init_mm.page_table_lock should be held before checking the page table. Is this true, or can it safely be done without doing that (the bttv-driver.c doesn't) ?

Any ideas & comments are appreciated.

Thanks,

  Steffen Persvold   | Scalable Linux Systems |   Try out the world's best   
 mailto:[email protected]  |  http://www.scali.com  | performing MPI implementation:
Tel: (+47) 2262 8950 |   Olaf Helsets vei 6   |      - ScaMPI 1.12.2 -         
Fax: (+47) 2262 8951 |   N0621 Oslo, NORWAY   | >300MBytes/s and <4uS latency

Uttered "Steffen Persvold" <[email protected]>, spoke thus:

[kv]malloc memory is _never_ subject to paging and can be mmap'ed with a vengeance without resorting to mucking about with marking pages or the like.

You're working too hard ;-)

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- + -- -- -- -- -- -- -- -- -- --

Tommy Reynolds                               | mailto: <[email protected]>
Red Hat, Inc., Embedded Development Services | Phone:  +1.256.704.9286
307 Wynn Drive NW, Huntsville, AL 35805 USA  | FAX:    +1.256.837.3839
Senior Software Developer                    | Mobile: +1.919.641.2923

OK, thanks. But I found out that if you want to use remap_page_range on kmalloc'ed memory you need to set the reserve bit first. Without it, it just doesn't work. When using the nopage method no reserving is necessary.

What about my question regarding locking the mm spinlock table before traversing the page table (for vmalloc'ed memory). Any ideas there ?

Regards,

  Steffen Persvold   | Scalable Linux Systems |   Try out the world's best   
 mailto:[email protected]  |  http://www.scali.com  | performing MPI implementation:
Tel: (+47) 2262 8950 |   Olaf Helsets vei 6   |      - ScaMPI 1.12.2 -         
Fax: (+47) 2262 8951 |   N0621 Oslo, NORWAY   | >300MBytes/s and <4uS latency

Another thing, when allocating memory with vmalloc, how can I be sure that the pages I get is adressable within 4GB (i.e I wan't to call pci_map_sg on this buffer for my 32bit PCI device without having to use bounce buffers ) ? On systems with less that 4GB of physical memory there's no problem, but what happens if you have more (lets say an IA64 server with 16GB of RAM) and don't have an IOMMU (like alpha and sparc) ?

I noticed a vmalloc_32 in linux/vmalloc.h (the comment says "32bit PA addressable pages - eg for PCI 32bit devices"), but is that one platform independent (I see that it is only using GFP_KERNEL, while vmalloc is using GFP_KERNEL | __GFP_HIGHMEM) ? This issue goes for __get_free_pages too I guess.

Regards,

  Steffen Persvold   | Scalable Linux Systems |   Try out the world's best   
 mailto:[email protected]  |  http://www.scali.com  | performing MPI implementation:
Tel: (+47) 2262 8950 |   Olaf Helsets vei 6   |      - ScaMPI 1.12.2 -         
Fax: (+47) 2262 8951 |   N0621 Oslo, NORWAY   | >300MBytes/s and <4uS latency

Hmm, it helps looking back at old threads (which I actually was involved in).

What ever happened to Jens Axboe's "zone_dma32" patch ? Why wasn't it included in the main 2.4.6 kernel tree (seemed like a good idea to me) ?

Regards,

  Steffen Persvold   | Scalable Linux Systems |   Try out the world's best   
 mailto:[email protected]  |  http://www.scali.com  | performing MPI implementation:
Tel: (+47) 2262 8950 |   Olaf Helsets vei 6   |      - ScaMPI 1.12.2 -         
Fax: (+47) 2262 8951 |   N0621 Oslo, NORWAY   | >300MBytes/s and <4uS latency

Hi,

You must only set PG_reserved or only increment the page count. If you increment the page count, the kernel will try to free memory by removing the page from the process and reread the page later by calling the nopage function again.
If you have the pages allocated all the time anyway, it's better to set PG_reserved, but these pages can only be freed again when all files are closed. You probably also want to set VM_RESERVED in vma->vm_flags, so the kernel won't even look at these pages, while scanning the process for freeable memory.

Currently this lock isn't needed, since noone will remove or change this mapping until you call vfree().

bye, Roman