Skip to contentSkip to conversation

put_page on transparent huge page leaks?

DISCUSSION Jay Cornwall 2 participants 4 messages Feb 21, 2014 – Feb 22, 2014

lore
Share & tools
lore
#0 lore raw

Hi,

I'm tracking a possible memory leak in iommu/amd. The driver uses this logic to fault a page in response to a PRI from a device:

npages = get_user_pages(fault->state->task, fault->state->mm,

                        fault->address, 1, write, 0, &page, NULL);

if (npages == 1)

        put_page(page);
else
        ...

This works correctly when get_user_pages returns a 4KB page. When transparent huge pages are enabled any 2MB page returned by this call appears to leak on process exit. The non-cached memory usage stays elevated by the set of faulted 2MB pages. This behavior is not observed when the exception handler demand faults 2MB pages.

I notice there is a difference in reference count between the 4KB/2MB paths.

get_user_pages (4KB): page_count()=3, page_mapcount()=1
put_page (4KB): page_count()=2, page_mapcount()=1

get_user_pages (2MB): page_count()=3, page_mapcount()=1
put_page (2MB): page_count()=3, page_mapcount()=0

I'm concerned that the driver appears to be holding a reference count after put_page(). Am I interpreting this observation correctly?

#1 lore raw
Quoted 21 lines · Jay Cornwall

Hi,

I'm tracking a possible memory leak in iommu/amd. The driver uses this logic to fault a page in response to a PRI from a device:

npages = get_user_pages(fault->state->task, fault->state->mm,

                        fault->address, 1, write, 0, &page, NULL);

if (npages == 1)

        put_page(page);
else
        ...

This works correctly when get_user_pages returns a 4KB page. When transparent huge pages are enabled any 2MB page returned by this call appears to leak on process exit. The non-cached memory usage stays elevated by the set of faulted 2MB pages. This behavior is not observed when the exception handler demand faults 2MB pages.

I notice there is a difference in reference count between the 4KB/2MB paths.

Quoted 8 lines

get_user_pages (4KB): page_count()=3, page_mapcount()=1
put_page (4KB): page_count()=2, page_mapcount()=1

get_user_pages (2MB): page_count()=3, page_mapcount()=1
put_page (2MB): page_count()=3, page_mapcount()=0

I'm concerned that the driver appears to be holding a reference count after put_page(). Am I interpreting this observation correctly?

Could you show output of dump_page() on 2M pages for both points?

My guess is that your page is PageTail(). Refcounting for tail pages is
different: on get_page() we increase *->_mapcount* of tail and increase ->_count of relevant head page. ->_count of tail pages should always be zero, but it's 3 in your case which is odd.

BTW, I don't see where you take mmap_sem in drivers/iommu/amd_iommu_v2.c, which is required for gup. Do I miss something?

Signature · 1 line
 Kirill A. Shutemov
#2 lore raw
Quoted 12 lines · Kirill A. Shutemov
Quoted 11 lines · Jay Cornwall

I'm tracking a possible memory leak in iommu/amd. The driver uses this logic to fault a page in response to a PRI from a device:

npages = get_user_pages(fault->state->task, fault->state->mm,

                        fault->address, 1, write, 0, &page, NULL);

if (npages == 1)

        put_page(page);
else
        ...
Quoted 7 lines
Quoted 5 lines

This works correctly when get_user_pages returns a 4KB page. When transparent huge pages are enabled any 2MB page returned by this call appears to leak on process exit. The non-cached memory usage stays elevated by the set of faulted 2MB pages. This behavior is not observed when the exception handler demand faults 2MB pages.

Could you show output of dump_page() on 2M pages for both points?

get_user_pages():
page:ffffea000ffa00c0 count:0 mapcount:1 mapping: (null)
index:0x0
page flags: 0x2fffe0000008000(tail)
// page_count(page)=3 (head page)
put_page():
page:ffffea000ffa00c0 count:0 mapcount:0 mapping: (null)
index:0x0
page flags: 0x2fffe0000008000(tail)
// page_count(page)=3 (head page)

Repeat on the same page.

get_user_pages():
page:ffffea000ffa00c0 count:0 mapcount:1 mapping: (null)
index:0x0
page flags: 0x2fffe0000008000(tail)
// page_count(page)=4 (head page)
put_page():
page:ffffea000ffa00c0 count:0 mapcount:0 mapping: (null)
index:0x0
page flags: 0x2fffe0000008000(tail)
// page_count(page)=4 (head page)

The head page appears to be leaking a reference. There is *no leak* if the driver faults the head page directly.

Quoted 4 lines

My guess is that your page is PageTail(). Refcounting for tail pages is
different: on get_page() we increase *->_mapcount* of tail and increase ->_count of relevant head page. ->_count of tail pages should always be zero, but it's 3 in your case which is odd.

That's correct, this is a tail page. page_count() references the head page:

  static inline int page_count(struct page *page)
  {
          return atomic_read(&compound_head(page)->_count);
  }
Quoted 2 lines

BTW, I don't see where you take mmap_sem in drivers/iommu/amd_iommu_v2.c, which is required for gup. Do I miss something?

You're right. I have a patch applied on my local branch to fix this.

#3 lore raw
Quoted 7 lines · Jay Cornwall
Quoted 3 lines · Kirill A. Shutemov
Quoted 2 lines · Jay Cornwall

I'm tracking a possible memory leak in iommu/amd. The driver uses this logic to fault a page in response to a PRI from a device:

The head page appears to be leaking a reference. There is *no leak* if the driver faults the head page directly.

My apologies, this was fixed somewhere in the patch series: "mm: tail page refcounting optimization for slab and hugetlbfs" (Jan 22nd).

Our merge was slightly older than I'd thought.