Skip to contentSkip to conversation

THP handling with driver compound page on fault

DISCUSSION Minchan Kim 2 participants 7 messages Jan 30, 2021 Memory management

lore
Share & tools
lore
#0 lore raw

Hi Folks,

A custom driver overrides (vm_operations_struct.fault) and map their compound page(__GFP_COMP) to page table on userprocess on demand.

Here problem is

page fault occur:

__do_fault

	vma->vm_ops->fault <- driver returns their compould page 

finish_fault
alloc_set_pte
PageTransCompound(page)
do_set_pmd
map the page as THP page

Since it's not a THP page, system goes random crash after a while due to conflict of VM THP handling with the driver compound page handling. Even though it happens in old kernel with backporting THP pagecache, I think it's still valid on recent kernel.

What's the best way to fix it? Driver don't do that or VM need to deal with it?

Thank you.

#1 lore raw
Quoted 2 lines · Minchan Kim

A custom driver overrides (vm_operations_struct.fault) and map their compound page(__GFP_COMP) to page table on userprocess on demand.

You're looking to backport:

commit d01ac3c35214ce362f50cada37cb7bab8c801896

Author: Matthew Wilcox (Oracle) <[email protected]>
Date:   Thu Oct 15 20:05:26 2020 -0700

mm/memory: remove page fault assumption of compound page size

#2 lore raw

Hi Mattew,

Quoted 4 lines · Matthew Wilcox
Quoted 2 lines · Minchan Kim

A custom driver overrides (vm_operations_struct.fault) and map their compound page(__GFP_COMP) to page table on userprocess on demand.

Quoted 8 lines

You're looking to backport:

commit d01ac3c35214ce362f50cada37cb7bab8c801896

Author: Matthew Wilcox (Oracle) <[email protected]>
Date:   Thu Oct 15 20:05:26 2020 -0700

mm/memory: remove page fault assumption of compound page size

I guess you meant the check below.

	if (compound_order(page) != HPAGE_PMD_ORDER)

What happens if driver allcoated HPAGE_PMD_ORDER size?

#3 lore raw
Quoted 13 lines · Minchan Kim

Hi Mattew,

Quoted 10 lines · Matthew Wilcox
Quoted 2 lines · Minchan Kim

A custom driver overrides (vm_operations_struct.fault) and map their compound page(__GFP_COMP) to page table on userprocess on demand.

You're looking to backport:

commit d01ac3c35214ce362f50cada37cb7bab8c801896

Author: Matthew Wilcox (Oracle) <[email protected]>
Date:   Thu Oct 15 20:05:26 2020 -0700
Quoted 8 lines
Quoted 2 lines

mm/memory: remove page fault assumption of compound page size

I guess you meant the check below.

	if (compound_order(page) != HPAGE_PMD_ORDER)

What happens if driver allcoated HPAGE_PMD_ORDER size?

... then it should be fine to map it with a PMD entry?

#4 lore raw
Quoted 17 lines · Matthew Wilcox
Quoted 16 lines · Minchan Kim

Hi Mattew,

Quoted 12 lines · Matthew Wilcox
Quoted 2 lines · Minchan Kim

A custom driver overrides (vm_operations_struct.fault) and map their compound page(__GFP_COMP) to page table on userprocess on demand.

You're looking to backport:

commit d01ac3c35214ce362f50cada37cb7bab8c801896

Author: Matthew Wilcox (Oracle) <[email protected]>
Date:   Thu Oct 15 20:05:26 2020 -0700

mm/memory: remove page fault assumption of compound page size

Quoted 7 lines
Quoted 5 lines

I guess you meant the check below.

	if (compound_order(page) != HPAGE_PMD_ORDER)

What happens if driver allcoated HPAGE_PMD_ORDER size?

... then it should be fine to map it with a PMD entry?

I don't think so because following logic assumes it's THP page, for example, page_add_file_rmap, count_vm_event(THP_FILE_MAPPED) in do_set_pmd.

Isn't it bug?

#5 lore raw
Quoted 24 lines · Minchan Kim
Quoted 23 lines · Matthew Wilcox
Quoted 21 lines · Minchan Kim

Hi Mattew,

Quoted 4 lines · Matthew Wilcox
Quoted 2 lines · Minchan Kim

A custom driver overrides (vm_operations_struct.fault) and map their compound page(__GFP_COMP) to page table on userprocess on demand.

Quoted 8 lines

You're looking to backport:

commit d01ac3c35214ce362f50cada37cb7bab8c801896

Author: Matthew Wilcox (Oracle) <[email protected]>
Date:   Thu Oct 15 20:05:26 2020 -0700

mm/memory: remove page fault assumption of compound page size

I guess you meant the check below.

	if (compound_order(page) != HPAGE_PMD_ORDER)

What happens if driver allcoated HPAGE_PMD_ORDER size?

Quoted 7 lines
Quoted 1 line

... then it should be fine to map it with a PMD entry?

I don't think so because following logic assumes it's THP page, for example, page_add_file_rmap, count_vm_event(THP_FILE_MAPPED) in do_set_pmd.

Isn't it bug?

Clearly nobody's actually tested it, but I think in principle, if a device driver returns a PMD sized page (and it's appropriately aligned), it should be mapped with a PMD entry.

#6 lore raw
Quoted 31 lines · Matthew Wilcox
Quoted 30 lines · Minchan Kim
Quoted 17 lines · Matthew Wilcox
Quoted 16 lines · Minchan Kim

Hi Mattew,

Quoted 12 lines · Matthew Wilcox
Quoted 2 lines · Minchan Kim

A custom driver overrides (vm_operations_struct.fault) and map their compound page(__GFP_COMP) to page table on userprocess on demand.

You're looking to backport:

commit d01ac3c35214ce362f50cada37cb7bab8c801896

Author: Matthew Wilcox (Oracle) <[email protected]>
Date:   Thu Oct 15 20:05:26 2020 -0700

mm/memory: remove page fault assumption of compound page size

Quoted 7 lines
Quoted 5 lines

I guess you meant the check below.

	if (compound_order(page) != HPAGE_PMD_ORDER)

What happens if driver allcoated HPAGE_PMD_ORDER size?

... then it should be fine to map it with a PMD entry?

I don't think so because following logic assumes it's THP page, for example, page_add_file_rmap, count_vm_event(THP_FILE_MAPPED) in do_set_pmd.

Quoted 5 lines
Quoted 1 line

Isn't it bug?

Clearly nobody's actually tested it, but I think in principle, if a device driver returns a PMD sized page (and it's appropriately aligned), it should be mapped with a PMD entry.

Okay, then you mean d01ac3c35214ce362f50cada37cb7bab8c801896 will fix the corruption for every-order driver compound page mapping.

I have no idea it's worth to solve THP_FILE_MAPPED misaccounting problem but it doesn't matter for my case, at least.

Thanks for the quick answer, Matthew.