get_user_pages() and shared memory question
Hi,
Is it possible for a page of memory that's been "grabbed" with get_user_pages() to ever be allocated to another process? I'm assuming the answer is no, but I have a specific case I want to ask about.
Let's say an application allocates some shared memory, and then calls into a driver which calls get_user_pages(). The driver exits without releasing the pages, so they now have a reference count on them. Then the application deallocates the shared memory. At this point, the virtual addresses disappear, and no process owns them, but the pages still have a reference count.
Another process now tries to allocate a shared memory buffer. Is there any way that this new buffer can contain those pages that were grabbed with get_user_pages() (i.e. that already have a reference count)?
Until 2.6.7, there was a bug in the VM where a page that was grabbed with get_user_pages() could be swapped out. Those of you familar with the OpenIB work know what I'm talking about. Would that bug affect anything I'm talking about?
Signature · 7 lines
Timur Tabi Staff Software Engineer [email protected] One thing a Southern boy will never say is, "I don't think duct tape will fix it." -- Ed Smylie, NASA engineer for Apollo 13
Timur> Hi, Is it possible for a page of memory that's been Timur> "grabbed" with get_user_pages() to ever be allocated to Timur> another process? I'm assuming the answer is no, but I have Timur> a specific case I want to ask about.
Not to the best of my knowledge, although you should probably read the code to convince yourself.
Timur> Until 2.6.7, there was a bug in the VM where a page that Timur> was grabbed with get_user_pages() could be swapped out. Timur> Those of you familar with the OpenIB work know what I'm Timur> talking about. Would that bug affect anything I'm talking Timur> about?
This isn't what the bug caused. What could happen was that the swapper could unmap a page from a process's virtual memory map before it noticed that the page had an elevated reference count. The page wouldn't get swapped out, but when the process caused a page fault to bring the virtual address back, it would get a different piece of physical memory.
- R.
Quoted 12 lines · Timur Tabi
Is it possible for a page of memory that's been "grabbed" with
get_user_pages()to ever be allocated to another process? I'm assuming the answer is no, but I have a specific case I want to ask about.Let's say an application allocates some shared memory, and then calls into a driver which calls
get_user_pages(). The driver exits without releasing the pages, so they now have a reference count on them. Then the application deallocates the shared memory. At this point, the virtual addresses disappear, and no process owns them, but the pages still have a reference count.
Quoted 3 lines
Another process now tries to allocate a shared memory buffer. Is there any way that this new buffer can contain those pages that were grabbed with
get_user_pages()(i.e. that already have a reference count)?
It depends on what you mean by allocate and deallocate. If the second process is attaching the same shared memory segment as the first process had attached, then yes, its buffer will contain those very pages which the driver erroneously failed to release.
Quoted 4 lines
Until 2.6.7, there was a bug in the VM where a page that was grabbed with
get_user_pages()could be swapped out. Those of you familar with the OpenIB work know what I'm talking about. Would that bug affect anything I'm talking about?
No. That was a bug peculiar to anonymous memory,
whereas shared memory is treated like file cache.
Hugh
Quoted 4 lines · Hugh Dickins
It depends on what you mean by allocate and deallocate. If the second process is attaching the same shared memory segment as the first process had attached, then yes, its buffer will contain those very pages which the driver erroneously failed to release.
No, I'm talking about when the first process completely destroys the shared memory segment so that it no longer exists. No processes are attached to it, and any attempt to attach to it results in an error, because it doesn't exist.
In this case, when a process creates a new memory segment, I just want to know whether the pages with a non-zero refcount (because of the get_user_pages() call) can ever be used in a new shared memory segment.
I'm assuming the answer is no, because that would defeat the purpose of refcount (right?).
I've been looking at the code and reading books on the VM, but I get lost easily. It
appears that the function which allocates a page is shmem_alloc_page(), which calls alloc_page() to do the actual work. If that's correct, is it possible for alloc_page() to return a page that has been previously "claimed" by get_user_pages()? I'm looking at __alloc_pages(), and I don't see any calls to page_count(), so I guess there's some other mechanism (either in get_user_pages() or in the way the VM works) that prevents this possibility. However, I'm getting dangerously close to my limit of understanding the Linux VM.
Thanks for replying to my message. I really appreciate the help in understanding the Linux VM.
Signature · 7 lines
Timur Tabi Staff Software Engineer [email protected] One thing a Southern boy will never say is, "I don't think duct tape will fix it." -- Ed Smylie, NASA engineer for Apollo 13
Quoted 7 lines · Timur Tabi
In this case, when a process creates a new memory segment, I just want to know whether the pages with a non-zero refcount (because of the
get_user_pages()call) can ever be used in a new shared memory segment.I'm assuming the answer is no, because that would defeat the purpose of refcount (right?).
Exactly right.
Hugh
Quoted 3 lines · Timur Tabi
Let's say an application allocates some shared memory, and then calls into a driver which calls
get_user_pages(). The driver exits without releasing the pages, so they now have a reference count on them.
Preventing the driver from doing this would probably be the right solution here... If the driver called get_user_pages, it is its responsibility to release the pages.
Brice
Quoted 3 lines · Brice Goglin
Preventing the driver from doing this would probably be the right solution here... If the driver called
get_user_pages, it is its responsibility to release the pages.
The driver does release the pages, but only when asked to do so. If the process dies, then the driver automatically cleans up, but otherwise how is the driver to know that the memory is no longer needed?
Perhaps you mean that the driver should release the pages before it exits. Unfortunately, that defeats the purpose of calling get_user_pages() in the first place. The driver needs to pin the application's buffers so that the subsequent DMA operations work. This driver supports an RDMA adapter that transfer network data directly to the application's buffers.
You're probably now thinking, "Well, why doesn't the driver just allocate the buffers on behalf of the app?" There are two reasons why we can't do that. One, the app may need have gigabytes of memory for the RDMA operations. Two, the APIs we need to support allow the app to allocate memory any way it sees fit.
Signature · 7 lines
Timur Tabi Staff Software Engineer [email protected] One thing a Southern boy will never say is, "I don't think duct tape will fix it." -- Ed Smylie, NASA engineer for Apollo 13