mm: Allow GFP_IOFS for page_cache_read page cache allocation
Subsystems (5)
PATCH 1/1 mm: Allow GFP_IOFS for page_cache_read page cache allocation
From: Michal Hocko <[email protected]>
page_cache_read has been historically using page_cache_alloc_cold to allocate a new page. This means that mapping_gfp_mask is used as the base for the gfp_mask. Many filesystems are setting this mask to GFP_NOFS to prevent from fs recursion issues. page_cache_read is called from the vm_operations_struct::fault() context during the page fault. This context doesn't need the reclaim protection normally.
ceph and ocfs2 which call filemap_fault from their fault handlers seem to be OK because they are not taking any fs lock before invoking generic implementation. xfs which takes XFS_MMAPLOCK_SHARED is safe from the reclaim recursion POV because this lock serializes truncate and punch hole with the page faults and it doesn't get involved in the reclaim.
There is simply no reason to deliberately use a weaker allocation context when a __GFP_FS | __GFP_IO can be used. The GFP_NOFS protection might be even harmful. There is a push to fail GFP_NOFS allocations rather than loop within allocator indefinitely with a very limited reclaim ability. Once we start failing those requests the OOM killer might be triggered prematurely because the page cache allocation failure is propagated up the page fault path and end up in pagefault_out_of_memory.
We cannot play with mapping_gfp_mask directly because that would be racy wrt. parallel page faults and it might interfere with other users who really rely on NOFS semantic from the stored gfp_mask. The mask is also inode proper so it would even be a layering violation. What we can do instead is to push the gfp_mask into struct vm_fault and allow fs layer to overwrite it should the callback need to be called with a different allocation context.
Initialize the default to (mapping_gfp_mask | __GFP_FS | __GFP_IO) because this should be safe from the page fault path normally. Why do we care about mapping_gfp_mask at all then? Because this doesn't hold only reclaim protection flags but it also might contain zone and movability restrictions (GFP_DMA32, __GFP_MOVABLE and others) so we have to respect those.
---
Hi,
this has been posted previously as a part of larger GFP_NOFS related
patch set (http://lkml.kernel.org/r/1438768284-30927-1-git-send-email-mhocko%40kernel.org)
but I think it makes sense to discuss it even out of that scope.
I would like to hear FS and other MM people about the proposed interface. Using mapping_gfp_mask blindly doesn't sound good to me and vm_fault looks like a proper channel to communicate between MM and FS layers.
Comments? Are there any better ideas?
include/linux/mm.h | 4 ++++ mm/filemap.c | 9 ++++----- mm/memory.c | 17 +++++++++++++++++ 3 files changed, 25 insertions(+), 5 deletions(-)
include/linux/mm.h +4 -0
@@ -225,10 +225,14 @@ extern pgprot_t protection_map[16];* ->fault function. The vma's ->fault is responsible for returning a bitmask * of VM_FAULT_xxx flags that give details about how the fault was handled. * +* MM layer fills up gfp_mask for page allocations but fault handler might +* alter it if its implementation requires a different allocation context. +* * pgoff should be used in favour of virtual_address, if possible. */ struct vm_fault {unsigned int flags;/* FAULT_FLAG_xxx flags */ +gfp_t gfp_mask;/* gfp mask to be used for allocations */ pgoff_t pgoff;/* Logical page offset based on vma */ void __user *virtual_address;/* Faulting virtual address */
mm/filemap.c +4 -5
@@ -1815,19 +1815,18 @@ EXPORT_SYMBOL(generic_file_read_iter);* This adds the requested page to the page cache if it isn't already there, * and schedules an I/O to read in its contents from disk. */ -static int page_cache_read (struct file *file,pgoff_t offset) +static int page_cache_read (struct file *file,pgoff_t offset,gfp_t gfp_mask) {struct address_space *mapping = file->f_mapping;struct page *page;int ret; do { - page = page_cache_alloc_cold(mapping); + page = __page_cache_alloc(gfp_mask|__GFP_COLD); if (!page) return -ENOMEM; - ret = add_to_page_cache_lru(page, mapping, offset, - mapping_gfp_constraint(mapping, GFP_KERNEL)); + ret = add_to_page_cache_lru(page, mapping, offset, gfp_mask & GFP_KERNEL); if (ret == 0) ret = mapping->a_ops->readpage(file, page); else if (ret == -EEXIST)
@@ -2008,7 +2007,7 @@ int filemap_fault(struct vm_area_struct *vma, struct vm_fault *vmf)* We're only likely to ever get here if MADV_RANDOM is in * effect. */ - error =page_cache_read (file, offset); + error =page_cache_read (file, offset, vmf->gfp_mask);/* * The page we want has now been added to the page cache.
mm/memory.c +17 -0
@@ -1943,6 +1943,20 @@ static inline void cow_user_page(struct page *dst, struct page *src, unsigned locopy_user_highpage (dst, src, va, vma); } +static gfp_t __get_fault_gfp_mask (struct vm_area_struct *vma) +{ +struct file *vm_file = vma->vm_file; + +if (vm_file) +return mapping_gfp_mask (vm_file->f_mapping) | __GFP_FS | __GFP_IO; + +/* +* Special mappings (e.g. VDSO) do not have any file so fake +* a default GFP_KERNEL for them. +*/ +return GFP_KERNEL ; +} +/* * Notify the address space that the page is about to become writable so that * it can prohibit this or wait for the page to get into an appropriate state.
@@ -1958,6 +1972,7 @@ static int do_page_mkwrite(struct vm_area_struct *vma, struct page *page, vmf.virtual_address = (void __user *)(address &PAGE_MASK ); vmf.pgoff = page->index; vmf.flags =FAULT_FLAG_WRITE |FAULT_FLAG_MKWRITE ; + vmf.gfp_mask =__get_fault_gfp_mask (vma); vmf.page = page; vmf.cow_page =NULL ;
@@ -2762,6 +2777,7 @@ static int __do_fault(struct vm_area_struct *vma, unsigned long address, vmf.pgoff = pgoff; vmf.flags = flags; vmf.page =NULL ; + vmf.gfp_mask =__get_fault_gfp_mask (vma); vmf.cow_page = cow_page; ret = vma->vm_ops->fault (vma, &vmf);
@@ -2928,6 +2944,7 @@ static void do_fault_around(struct vm_area_struct *vma, unsigned long address, vmf.pgoff = pgoff; vmf.max_pgoff = max_pgoff; vmf.flags = flags; + vmf.gfp_mask =__get_fault_gfp_mask (vma); vma->vm_ops->map_pages (vma, &vmf); }
Signature · 1 line
2.6.2
Quoted 49 lines · On Wed 11-11-15 15:13:53, [email protected] wrote:
From: Michal Hocko <[email protected]>
page_cache_readhas been historically usingpage_cache_alloc_coldto allocate a new page. This means thatmapping_gfp_maskis used as the base for thegfp_mask. Many filesystems are setting this mask toGFP_NOFSto prevent from fs recursion issues.page_cache_readis called from thevm_operations_struct::fault()context during the page fault. This context doesn't need the reclaim protection normally.ceph and ocfs2 which call
filemap_faultfrom their fault handlers seem to be OK because they are not taking any fs lock before invoking generic implementation. xfs which takesXFS_MMAPLOCK_SHAREDis safe from the reclaim recursion POV because this lock serializes truncate and punch hole with the page faults and it doesn't get involved in the reclaim.There is simply no reason to deliberately use a weaker allocation context when a
__GFP_FS|__GFP_IOcan be used. TheGFP_NOFSprotection might be even harmful. There is a push to failGFP_NOFSallocations rather than loop within allocator indefinitely with a very limited reclaim ability. Once we start failing those requests the OOM killer might be triggered prematurely because the page cache allocation failure is propagated up the page fault path and end up inpagefault_out_of_memory.We cannot play with
mapping_gfp_maskdirectly because that would be racy wrt. parallel page faults and it might interfere with other users who really rely on NOFS semantic from the storedgfp_mask. The mask is also inode proper so it would even be a layering violation. What we can do instead is to push thegfp_maskinto structvm_faultand allow fs layer to overwrite it should the callback need to be called with a different allocation context.Initialize the default to (
mapping_gfp_mask|__GFP_FS|__GFP_IO) because this should be safe from the page fault path normally. Why do we care aboutmapping_gfp_maskat all then? Because this doesn't hold only reclaim protection flags but it also might contain zone and movability restrictions (GFP_DMA32,__GFP_MOVABLEand others) so we have to respect those.Reported-by Tetsuo Handa <[email protected]>Signed-off-by Michal Hocko <[email protected]>---
Hi,
this has been posted previously as a part of largerGFP_NOFSrelated
patch set (http://lkml.kernel.org/r/1438768284-30927-1-git-send-email-mhocko%40kernel.org)
but I think it makes sense to discuss it even out of that scope.
Quoted 5 lines
I would like to hear FS and other MM people about the proposed interface. Using
mapping_gfp_maskblindly doesn't sound good to me andvm_faultlooks like a proper channel to communicate between MM and FS layers.Comments? Are there any better ideas?
Makes sense to me and the filesystems I know should be fine with this (famous last words ;). Feel free to add:
Honza
Signature · 8 lines
Jan Kara <[email protected]> SUSE Labs, CR -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 54 lines · Jan Kara
Quoted 53 lines · On Wed 11-11-15 15:13:53, [email protected] wrote:
From: Michal Hocko <[email protected]>
page_cache_readhas been historically usingpage_cache_alloc_coldto allocate a new page. This means thatmapping_gfp_maskis used as the base for thegfp_mask. Many filesystems are setting this mask toGFP_NOFSto prevent from fs recursion issues.page_cache_readis called from thevm_operations_struct::fault()context during the page fault. This context doesn't need the reclaim protection normally.ceph and ocfs2 which call
filemap_faultfrom their fault handlers seem to be OK because they are not taking any fs lock before invoking generic implementation. xfs which takesXFS_MMAPLOCK_SHAREDis safe from the reclaim recursion POV because this lock serializes truncate and punch hole with the page faults and it doesn't get involved in the reclaim.There is simply no reason to deliberately use a weaker allocation context when a
__GFP_FS|__GFP_IOcan be used. TheGFP_NOFSprotection might be even harmful. There is a push to failGFP_NOFSallocations rather than loop within allocator indefinitely with a very limited reclaim ability. Once we start failing those requests the OOM killer might be triggered prematurely because the page cache allocation failure is propagated up the page fault path and end up inpagefault_out_of_memory.We cannot play with
mapping_gfp_maskdirectly because that would be racy wrt. parallel page faults and it might interfere with other users who really rely on NOFS semantic from the storedgfp_mask. The mask is also inode proper so it would even be a layering violation. What we can do instead is to push thegfp_maskinto structvm_faultand allow fs layer to overwrite it should the callback need to be called with a different allocation context.Initialize the default to (
mapping_gfp_mask|__GFP_FS|__GFP_IO) because this should be safe from the page fault path normally. Why do we care aboutmapping_gfp_maskat all then? Because this doesn't hold only reclaim protection flags but it also might contain zone and movability restrictions (GFP_DMA32,__GFP_MOVABLEand others) so we have to respect those.Reported-by Tetsuo Handa <[email protected]>Signed-off-by Michal Hocko <[email protected]>---
Hi,
this has been posted previously as a part of largerGFP_NOFSrelated
patch set (http://lkml.kernel.org/r/1438768284-30927-1-git-send-email-mhocko%40kernel.org)
but I think it makes sense to discuss it even out of that scope.I would like to hear FS and other MM people about the proposed interface. Using
mapping_gfp_maskblindly doesn't sound good to me andvm_faultlooks like a proper channel to communicate between MM and FS layers.
Quoted 6 lines
Quoted 1 line
Comments? Are there any better ideas?
Makes sense to me and the filesystems I know should be fine with this (famous last words ;). Feel free to add:
Acked-by Jan Kara <[email protected]>
Thanks a lot! Are there any objections from other fs/mm people?
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Vlastimil Babka
Acked-byQuoted 12 lines · Michal Hocko
Quoted 11 lines · Jan Kara
Quoted 10 lines · On Wed 11-11-15 15:13:53, [email protected] wrote:
Hi,
this has been posted previously as a part of largerGFP_NOFSrelated
patch set (http://lkml.kernel.org/r/1438768284-30927-1-git-send-email-mhocko%40kernel.org)
but I think it makes sense to discuss it even out of that scope.I would like to hear FS and other MM people about the proposed interface. Using
mapping_gfp_maskblindly doesn't sound good to me andvm_faultlooks like a proper channel to communicate between MM and FS layers.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Comments? Are there any better ideas?
Makes sense to me and the filesystems I know should be fine with this (famous last words ;). Feel free to add:
Acked-by Jan Kara <[email protected]>Thanks a lot! Are there any objections from other fs/mm people?
Please replace "GFP_IOFS" in the subject, as the "flag" has been removed recently. Otherwise
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 18 lines · Vlastimil Babka
Quoted 17 lines · Michal Hocko
Quoted 16 lines · Jan Kara
Quoted 11 lines · On Wed 11-11-15 15:13:53, [email protected] wrote:
Hi,
this has been posted previously as a part of largerGFP_NOFSrelated
patch set (http://lkml.kernel.org/r/1438768284-30927-1-git-send-email-mhocko%40kernel.org)
but I think it makes sense to discuss it even out of that scope.I would like to hear FS and other MM people about the proposed interface. Using
mapping_gfp_maskblindly doesn't sound good to me andvm_faultlooks like a proper channel to communicate between MM and FS layers.Comments? Are there any better ideas?
Makes sense to me and the filesystems I know should be fine with this (famous last words ;). Feel free to add:
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Acked-by Jan Kara <[email protected]>Thanks a lot! Are there any objections from other fs/mm people?
Please replace "
GFP_IOFS" in the subject, as the "flag" has been removed recently. Otherwise
Done.
mm: Allow GFP_{FS,IO} for page_cache_read page cache allocation
Quoted 1 line
Acked-by Vlastimil Babka <[email protected]>
Thanks
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>