btrfs: Convert kmap/memset/kunmap to memzero_user()
Subsystems (5)
cover btrfs: Convert kmap/memset/kunmap to memzero_user()
From: Ira Weiny <[email protected]>
Previously this was submitted to convert to zero_user()[1]. zero_user() is not the same as memzero_user() and in fact some zero_user() calls may be better off as memzero_user(). Regardless it was incorrect to convert btrfs to zero_user().
This series corrects this by lifting memzero_user(), converting it to kmap_local_page(), and then using it in btrfs.
Thanks,
Ira
[1] https://lore.kernel.org/lkml/[email protected]/
Ira Weiny (3):
iov_iter: Lift memzero_page() to highmem.h mm/highmem: Convert memzero_page() to kmap_local_page()
btrfs: Use memzero_page() instead of open coded kmap pattern
fs/btrfs/compression.c | 5 +---- fs/btrfs/extent_io.c | 22 ++++------------------ fs/btrfs/inode.c | 33 ++++++++++----------------------- fs/btrfs/reflink.c | 6 +----- fs/btrfs/zlib.c | 5 +---- fs/btrfs/zstd.c | 5 +---- include/linux/highmem.h | 7 +++++++ lib/iov_iter.c | 8 +------- 8 files changed, 26 insertions(+), 65 deletions(-)
Signature · 1 line
2.28.0.rc0.12.gb6a658bd00c9
PATCH 1/3 iov_iter: Lift memzero_page() to highmem.h
From: Ira Weiny <[email protected]>
memzero_page() can replace the kmap/memset/kunmap pattern in other places in the code. While zero_user() has the same interface it is not the same call and its use should be limited and some of those calls may be better converted from zero_user() to memzero_page().[1] But that is not addressed in this series.
Lift memzero_page() to highmem.
To: Andrew Morton <[email protected]>
[1] https://lore.kernel.org/lkml/CAHk-=wijdojzo56FzYqE5TOYw2Vws7ik3LEMGj9SPQaJJ+Z73Q@mail.gmail.com/
--- include/linux/highmem.h | 7 +++++++ lib/iov_iter.c | 8 +------- 2 files changed, 8 insertions(+), 7 deletions(-)
include/linux/highmem.h +7 -0
@@ -332,4 +332,11 @@ static inline void memcpy_to_page(struct page *page, size_t offset, kunmap_local(to); } +static inline void memzero_page (struct page *page,size_t offset,size_t len) +{ +char *addr =kmap_atomic (page); +memset (addr + offset,0 , len); +kunmap_atomic (addr); +} +#endif /* _LINUX_HIGHMEM_H */
lib/iov_iter.c +1 -7
@@ -5,6 +5,7 @@#include <linux/fault-inject-usercopy.h> #include <linux/uio.h> #include <linux/pagemap.h> +#include <linux/highmem.h> #include <linux/slab.h> #include <linux/vmalloc.h> #include <linux/splice.h>
@@ -464,13 +465,6 @@ void iov_iter_init(struct iov_iter *i, unsigned int direction, }EXPORT_SYMBOL (iov_iter_init); -static void memzero_page (struct page *page,size_t offset,size_t len) -{ -char *addr =kmap_atomic (page); -memset (addr + offset,0 , len); -kunmap_atomic (addr); -} -static inline bool allocated (struct pipe_buffer *buf) {return buf->ops == &default_pipe_buf_ops;
Signature · 1 line
2.28.0.rc0.12.gb6a658bd00c9
PATCH 2/3 mm/highmem: Convert memzero_page() to kmap_local_page()
In reply to [email protected]From: Ira Weiny <[email protected]>
The memset() does not need to be performed atomically. Use kmap_local_page() which will improved performance for this call.
--- include/linux/highmem.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/highmem.h +2 -2
@@ -334,9 +334,9 @@ static inline void memcpy_to_page(struct page *page, size_t offset,static inline void memzero_page (struct page *page,size_t offset,size_t len) { -char *addr =kmap_atomic (page); +char *addr =kmap_local_page (page);memset (addr + offset,0 , len); -kunmap_atomic (addr); +kunmap_local (addr); }#endif /* _LINUX_HIGHMEM_H */
Signature · 1 line
2.28.0.rc0.12.gb6a658bd00c9
PATCH 3/3 btrfs: Use memzero_page() instead of open coded kmap pattern
In reply to [email protected]From: Ira Weiny <[email protected]>
There are many places where kmap/memset/kunmap patterns occur.
Use the newly lifted memzero_page() to eliminate direct uses of kmap and leverage the new core functions use of kmap_local_page().
The development of this patch was aided by the following coccinelle script:
// <smpl>
// SPDX-License-Identifier: GPL-2.0-only
// Find kmap/memset/kunmap pattern and replace with memset*page calls //
// NOTE: Offsets and other expressions may be more complex than what the script // will automatically generate. Therefore a catchall rule is provided to find // the pattern which then must be evaluated by hand.
//
// Confidence: Low
// Copyright: (C) 2021 Intel Corporation
// URL: http://coccinelle.lip6.fr/
// Comments:
// Options:
//
// Then the memset pattern
//
@ memset_rule1 @
expression page, V, L, Off;
identifier ptr;
type VP;
@@
(
-VP ptr = kmap(page);
|
-ptr = kmap(page);
|
-VP ptr = kmap_atomic(page);
|
-ptr = kmap_atomic(page);
)
<+...
(
-memset(ptr, 0, L);
+memzero_page(page, 0, L);
|
-memset(ptr + Off, 0, L);
+memzero_page(page, Off, L);
|
-memset(ptr, V, L);
+memset_page(page, V, 0, L);
|
-memset(ptr + Off, V, L);
+memset_page(page, V, Off, L);
)
...+>
(
-kunmap(page);
|
-kunmap_atomic(ptr);
)
// Remove any pointers left unused
@
depends on memset_rule1
@
identifier memset_rule1.ptr;
type VP, VP1;
@@
-VP ptr; ... when != ptr;
? VP1 ptr;
//
// Catch all
//
@ memset_rule2 @
expression page;
identifier ptr;
expression GenTo, GenSize, GenValue;
type VP;
@@
(
-VP ptr = kmap(page);
|
-ptr = kmap(page);
|
-VP ptr = kmap_atomic(page);
|
-ptr = kmap_atomic(page);
)
<+...
(
//
// Some call sites have complex expressions within the memset/memcpy // The follow are catch alls which need to be evaluated by hand. //
-memset(GenTo, 0, GenSize);
+memzero_pageExtra(page, GenTo, GenSize);
|
-memset(GenTo, GenValue, GenSize); +memset_pageExtra(page, GenValue, GenTo, GenSize);
)
...+>
(
-kunmap(page);
|
-kunmap_atomic(ptr);
)
// Remove any pointers left unused
@
depends on memset_rule2
@
identifier memset_rule2.ptr;
type VP, VP1;
@@
-VP ptr; ... when != ptr;
? VP1 ptr;
// </smpl>
---
Changes from v2:
Refactor to use memzero_page() per Linus
Changes from v1:
Update commit message per David https://lore.kernel.org/lkml/[email protected]/ --- fs/btrfs/compression.c | 5 +---- fs/btrfs/extent_io.c | 22 ++++------------------ fs/btrfs/inode.c | 33 ++++++++++----------------------- fs/btrfs/reflink.c | 6 +----- fs/btrfs/zlib.c | 5 +---- fs/btrfs/zstd.c | 5 +---- 6 files changed, 18 insertions(+), 58 deletions(-)
fs/btrfs/compression.c +1 -4
@@ -586,16 +586,13 @@ static noinline int add_ra_bio_pages(struct inode *inode,free_extent_map (em);if (page->index == end_index) { -char *userpage;size_t zero_offset =offset_in_page (isize);if (zero_offset) {int zeros; zeros =PAGE_SIZE - zero_offset; - userpage =kmap_atomic (page); -memset (userpage + zero_offset,0 , zeros); +memzero_page (page, zero_offset, zeros);flush_dcache_page (page); -kunmap_atomic (userpage); } }
fs/btrfs/extent_io.c +4 -18
@@ -3381,15 +3381,12 @@ int btrfs_do_readpage(struct page *page, struct extent_map **em_cached, }if (page->index == last_byte >>PAGE_SHIFT ) { -char *userpage;size_t zero_offset =offset_in_page (last_byte);if (zero_offset) { iosize =PAGE_SIZE - zero_offset; - userpage =kmap_atomic (page); -memset (userpage + zero_offset,0 , iosize); +memzero_page (page, zero_offset, iosize);flush_dcache_page (page); -kunmap_atomic (userpage); } }begin_page_read (fs_info, page);
@@ -3398,14 +3395,11 @@ int btrfs_do_readpage(struct page *page, struct extent_map **em_cached,u64 disk_bytenr;if (cur >= last_byte) { -char *userpage;struct extent_state *cached =NULL ; iosize =PAGE_SIZE - pg_offset; - userpage =kmap_atomic (page); -memset (userpage + pg_offset,0 , iosize); +memzero_page (page, pg_offset, iosize);flush_dcache_page (page); -kunmap_atomic (userpage);set_extent_uptodate (tree, cur, cur + iosize -1 , &cached,GFP_NOFS ); unlock_extent_cached(tree, cur,
@@ -3488,13 +3482,10 @@ int btrfs_do_readpage(struct page *page, struct extent_map **em_cached,/* we've found a hole, just zero and go on */ if (block_start ==EXTENT_MAP_HOLE ) { -char *userpage;struct extent_state *cached =NULL ; - userpage =kmap_atomic (page); -memset (userpage + pg_offset,0 , iosize); +memzero_page (page, pg_offset, iosize);flush_dcache_page (page); -kunmap_atomic (userpage);set_extent_uptodate (tree, cur, cur + iosize -1 , &cached,GFP_NOFS );
@@ -3805,12 +3796,7 @@ static int __extent_writepage(struct page *page, struct writeback_control *wbc, }if (page->index == end_index) { -char *userpage; - - userpage =kmap_atomic (page); -memset (userpage + pg_offset,0 , -PAGE_SIZE - pg_offset); -kunmap_atomic (userpage); +memzero_page (page, pg_offset,PAGE_SIZE - pg_offset);flush_dcache_page (page); }
fs/btrfs/inode.c +10 -23
@@ -641,17 +641,12 @@ static noinline int compress_file_range(struct async_chunk *async_chunk)if (!ret) {unsigned long offset =offset_in_page (total_compressed);struct page *page = pages[nr_pages -1 ]; -char *kaddr;/* zero the tail end of the last page, we might be * sending it down to disk */ -if (offset) { - kaddr =kmap_atomic (page); -memset (kaddr + offset,0 , -PAGE_SIZE - offset); -kunmap_atomic (kaddr); - } +if (offset) +memzero_page (page, offset,PAGE_SIZE - offset); will_compress =1 ; } }
@@ -4829,7 +4824,6 @@ int btrfs_truncate_block(struct btrfs_inode *inode, loff_t from, loff_t len,struct btrfs_ordered_extent *ordered;struct extent_state *cached_state =NULL ;struct extent_changeset *data_reserved =NULL ; -char *kaddr;bool only_release_metadata = false;u32 blocksize = fs_info->sectorsize;pgoff_t index = from >>PAGE_SHIFT ;
@@ -4921,15 +4915,13 @@ int btrfs_truncate_block(struct btrfs_inode *inode, loff_t from, loff_t len,if (offset != blocksize) {if (!len) len = blocksize - offset; - kaddr =kmap (page);if (front) -memset (kaddr + (block_start -page_offset (page)), -0 , offset); +memzero_page (page, (block_start -page_offset (page)), + offset);else -memset (kaddr + (block_start -page_offset (page)) + offset, -0 , len); +memzero_page (page, (block_start -page_offset (page)) + offset, + len);flush_dcache_page (page); -kunmap (page); }ClearPageChecked (page);set_page_dirty (page);
@@ -6828,11 +6820,9 @@ static noinline int uncompress_inline(struct btrfs_path *path,* cover that region here. */ -if (max_size + pg_offset <PAGE_SIZE ) { -char *map =kmap (page); -memset (map + pg_offset + max_size,0 ,PAGE_SIZE - max_size - pg_offset); -kunmap (page); - } +if (max_size + pg_offset <PAGE_SIZE ) +memzero_page (page, pg_offset + max_size, +PAGE_SIZE - max_size - pg_offset);kfree (tmp);return ret; }
@@ -8498,7 +8488,6 @@ vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf)struct btrfs_ordered_extent *ordered;struct extent_state *cached_state =NULL ;struct extent_changeset *data_reserved =NULL ; -char *kaddr;unsigned long zero_start;loff_t size;vm_fault_t ret;
@@ -8610,10 +8599,8 @@ vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf) zero_start =PAGE_SIZE ;if (zero_start !=PAGE_SIZE ) { - kaddr =kmap (page); -memset (kaddr + zero_start,0 ,PAGE_SIZE - zero_start); +memzero_page (page, zero_start,PAGE_SIZE - zero_start);flush_dcache_page (page); -kunmap (page); }ClearPageChecked (page);set_page_dirty (page);
fs/btrfs/reflink.c +1 -5
@@ -129,12 +129,8 @@ static int copy_inline_to_page(struct btrfs_inode *inode,* So what's in the range [500, 4095] corresponds to zeroes. */ if (datal < block_size) { -char *map; - - map =kmap (page); -memset (map + datal,0 , block_size - datal); +memzero_page (page, datal, block_size - datal);flush_dcache_page (page); -kunmap (page); }SetPageUptodate (page);
fs/btrfs/zlib.c +1 -4
@@ -375,7 +375,6 @@ int zlib_decompress(struct list_head *ws, unsigned char *data_in,unsigned long bytes_left;unsigned long total_out =0 ;unsigned long pg_offset =0 ; -char *kaddr; destlen = min_t(unsigned long, destlen, PAGE_SIZE); bytes_left = destlen;
@@ -455,9 +454,7 @@ int zlib_decompress(struct list_head *ws, unsigned char *data_in,* end of the inline extent (destlen) to the end of the page */ if (pg_offset < destlen) { - kaddr =kmap_atomic (dest_page); -memset (kaddr + pg_offset,0 , destlen - pg_offset); -kunmap_atomic (kaddr); +memzero_page (dest_page, pg_offset, destlen - pg_offset); }return ret; }
fs/btrfs/zstd.c +1 -4
@@ -631,7 +631,6 @@ int zstd_decompress(struct list_head *ws, unsigned char *data_in,size_t ret2;unsigned long total_out =0 ;unsigned long pg_offset =0 ; -char *kaddr; stream =ZSTD_initDStream (ZSTD_BTRFS_MAX_INPUT , workspace->mem, workspace->size);
@@ -696,9 +695,7 @@ int zstd_decompress(struct list_head *ws, unsigned char *data_in, ret =0 ; finish:if (pg_offset < destlen) { - kaddr =kmap_atomic (dest_page); -memset (kaddr + pg_offset,0 , destlen - pg_offset); -kunmap_atomic (kaddr); +memzero_page (dest_page, pg_offset, destlen - pg_offset); }return ret; }
Signature · 1 line
2.28.0.rc0.12.gb6a658bd00c9
Re: [PATCH 3/3] btrfs: Use memzero_page() instead of open coded kmap pattern
Quoted 121 lines · On Tue, Mar 09, 2021 at 01:21:37PM -0800, [email protected] wrote:
From: Ira Weiny <[email protected]>
There are many places where kmap/memset/kunmap patterns occur.
Use the newly lifted
memzero_page()to eliminate direct uses of kmap and leverage the new core functions use ofkmap_local_page().The development of this patch was aided by the following coccinelle script:
// <smpl>
// SPDX-License-Identifier: GPL-2.0-only
// Find kmap/memset/kunmap pattern and replace with memset*page calls //
// NOTE: Offsets and other expressions may be more complex than what the script // will automatically generate. Therefore a catchall rule is provided to find // the pattern which then must be evaluated by hand.
//
// Confidence: Low
// Copyright: (C) 2021 Intel Corporation
// URL: http://coccinelle.lip6.fr/
// Comments:
// Options://
// Then the memset pattern
//
@memset_rule1@
expression page, V, L, Off;
identifier ptr;
type VP;
@@(
-VP ptr = kmap(page);
|
-ptr = kmap(page);
|
-VP ptr =kmap_atomic(page);
|
-ptr =kmap_atomic(page);
)
<+...
((quoted diff) +1
-1-memset(ptr, 0, L); +memzero_page(page, 0, L);|
(quoted diff) +1
-1-memset(ptr + Off, 0, L); +memzero_page(page, Off, L);|
(quoted diff) +1
-1-memset(ptr, V, L); +memset_page(page, V, 0, L);|
(quoted diff) +1
-1-memset(ptr + Off, V, L); +memset_page(page, V, Off, L);)
...+>
(
-kunmap(page);
|
-kunmap_atomic(ptr);
)// Remove any pointers left unused
@
depends onmemset_rule1
@
identifier memset_rule1.ptr;
type VP, VP1;
@@-VP ptr; ... when != ptr;? VP1 ptr;
//
// Catch all
//
@memset_rule2@
expression page;
identifier ptr;
expression GenTo, GenSize, GenValue;
type VP;
@@(
-VP ptr = kmap(page);
|
-ptr = kmap(page);
|
-VP ptr =kmap_atomic(page);
|
-ptr =kmap_atomic(page);
)
<+...
(
//
// Some call sites have complex expressions within the memset/memcpy // The follow are catch alls which need to be evaluated by hand. //(quoted diff) +1
-1-memset(GenTo, 0, GenSize); +memzero_pageExtra(page, GenTo, GenSize);|
(quoted diff) +1
-1-memset(GenTo, GenValue, GenSize); +memset_pageExtra(page, GenValue, GenTo, GenSize);)
...+>
(
-kunmap(page);
|
-kunmap_atomic(ptr);
)// Remove any pointers left unused
@
depends onmemset_rule2
@
identifier memset_rule2.ptr;
type VP, VP1;
@@
Quoted 7 lines
-VP ptr; ... when != ptr;? VP1 ptr;
// </smpl>
Signed-off-by Ira Weiny <[email protected]>
Quoted 7 lines · On Tue, 9 Mar 2021 13:21:34 -0800 [email protected] wrote:
Previously this was submitted to convert to zero_user()[1].
zero_user()is not the same asmemzero_user()and in fact somezero_user()calls may be better off asmemzero_user(). Regardless it was incorrect to convert btrfs tozero_user().This series corrects this by lifting
memzero_user(), converting it tokmap_local_page(), and then using it in btrfs.
This impacts btrfs more than MM. I suggest the btrfs developers grab it, with my
Quoted 7 lines · Andrew Morton
Quoted 5 lines · On Tue, 9 Mar 2021 13:21:34 -0800 [email protected] wrote:
Previously this was submitted to convert to zero_user()[1].
zero_user()is not the same asmemzero_user()and in fact somezero_user()calls may be better off asmemzero_user(). Regardless it was incorrect to convert btrfs tozero_user().
Quoted 5 lines
Quoted 2 lines
This series corrects this by lifting
memzero_user(), converting it tokmap_local_page(), and then using it in btrfs.This impacts btrfs more than MM. I suggest the btrfs developers grab it, with my
I thought David wanted you to take these this time?
"I can play the messenger again but now it seems a round of review is needed and with some testing it'll be possible in some -rc. At that point you may take the patches via the mm tree, unless Linus is ok with a late pull."
-- https://lore.kernel.org/lkml/[email protected]/
But reading that again I'm not sure what he meant.
David?
Ira
Quoted 3 lines
Acked-by Andrew Morton <[email protected]>
Quoted 16 lines · Ira Weiny
Quoted 7 lines · Andrew Morton
Quoted 5 lines · On Tue, 9 Mar 2021 13:21:34 -0800 [email protected] wrote:
Previously this was submitted to convert to zero_user()[1].
zero_user()is not the same asmemzero_user()and in fact somezero_user()calls may be better off asmemzero_user(). Regardless it was incorrect to convert btrfs tozero_user().Quoted 5 lines
Quoted 2 lines
This series corrects this by lifting
memzero_user(), converting it tokmap_local_page(), and then using it in btrfs.This impacts btrfs more than MM. I suggest the btrfs developers grab it, with my
I thought David wanted you to take these this time?
Quoted 7 lines
"I can play the messenger again but now it seems a round of review is needed and with some testing it'll be possible in some -rc. At that point you may take the patches via the mm tree, unless Linus is ok with a late pull."
-- https://lore.kernel.org/lkml/[email protected]/But reading that again I'm not sure what he meant.
As Linus had some objections I was not sure it was still feasible for the merge window, but this is now sorted. This new patchset does further changes in MM and the btrfs part is a straightforward cleanup. I've noticed Andrew added the patches to his queue which I'd prefer so I've added my reviewed-by to the third patch. Thanks.