btrfs: use kvmalloc() for allocations that can be larger than page size

The @src_copy buffer utilized inside overwrite_item() can be as large as the nodesize.

For an existing btrfs with 64KiB nodesize, it means there is a high chance to fail the kmalloc() call if there is not enough physically contiguous pages.

Meanwhile there is really no need for such physically contiguous pages, as we only use that buffer to compare the content of the item.

Use kvmalloc() to replace the kmalloc() call.
For most cases that kvmalloc() call will be easily fulfilled by regular kmalloc(), but for really large items and large nodes, kvmalloc() will have a much higher chance to get memory allocated.

Signed-off-by Qu Wenruo <[email protected]>

---
 fs/btrfs/tree-log.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

fs/btrfs/tree-log.c +2 -3

index cfb0b0e9e124..7fb476b19eb4 100644
@@ -503,7 +503,7 @@ static int overwrite_item(struct walk_control *wc)
                        btrfs_release_path(wc->subvol_path);
                        return 0;
                }
-               src_copy = kmalloc(item_size, GFP_NOFS);
+               src_copy = kvmalloc(item_size, GFP_NOFS);
                if (!src_copy) {
                        btrfs_abort_log_replay(wc, -ENOMEM,
                               "failed to allocate memory for log leaf item");

@@ -514,7 +514,7 @@ static int overwrite_item(struct walk_control *wc)
                dst_ptr = btrfs_item_ptr_offset(dst_eb, dst_slot);
                ret = memcmp_extent_buffer(dst_eb, src_copy, dst_ptr, item_size);
 
-               kfree(src_copy);
+               kvfree(src_copy);
                /*
                 * they have the same contents, just return, this saves
                 * us from cowing blocks in the destination tree and doing
--

2.55.0

Although btrfs doesn't support inlined extents larger than PAGE_SIZE for bs > ps cases, it's still possible for the experimental bs > ps support to mount a btrfs created on a system with a much larger page size, thus can still hit an inlined extent that is way larger than the current page size.

E.g. a compressed inline extent which has 32K compressed size, is created on 64K page sized arm64 with 64K sectorsize, then mounted on x86_64 with the experimental bs > ps support.

In that case, when reading the compressed inline extent, we need to allocate a buffer that is the same size as the compressed inline extent (32K).

That kmalloc() call will request physically contiguous memory for that 32K allocation, and if the system has a very fragmented memory space, such allocation can fail.

But there is really no reason that we require such buffer to be physically contiguous, so change it to kvmalloc() to reduce the chance of allocation failure for bs > ps cases.

And for all bs <= ps cases, the kvmalloc() call will just be fulfilled by kmalloc() so this will not bring any change to the most common cases. Only bs > ps will get the benefit of less memory allocation failure.

Signed-off-by Qu Wenruo <[email protected]>

---
 fs/btrfs/inode.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

fs/btrfs/inode.c +2 -3

index 59e92908c6c5..a85a7c561cf8 100644
@@ -7133,7 +7133,7 @@ static noinline int uncompress_inline(struct btrfs_path *path,
        compress_type = btrfs_file_extent_compression(leaf, item);
        max_size = btrfs_file_extent_ram_bytes(leaf, item);
        inline_size = btrfs_file_extent_inline_item_len(leaf, path->slots[0]);
-       tmp = kmalloc(inline_size, GFP_NOFS);
+       tmp = kvmalloc(inline_size, GFP_NOFS);
        if (!tmp)
                return -ENOMEM;
        ptr = btrfs_file_extent_inline_start(item);

@@ -7154,7 +7154,7 @@ static noinline int uncompress_inline(struct btrfs_path *path,
 
        if (max_size < blocksize)
                folio_zero_range(folio, max_size, blocksize - max_size);
-       kfree(tmp);
+       kvfree(tmp);
        return ret;
 }
 
--

2.55.0

With the experimental bs > ps support, the workspace buffer for both zlib and zstd can be as large as 64K, and on 4K page sized systems such kmalloc() calls have a much higher chance to fail, as that requires physically contiguous memory to fulfill such allocation.

The same also applies to S390's hardware accelerated path, which requires a buffer size of 4 pages.

Meanwhile lzo is already using kvmalloc() for its buffer, and there is no special requirement for any physically contiguous memory anyway.

So change the zlib and zstd workspace buffer allocation to use kvmalloc() to reduce the chance of memory allocation failure.

Signed-off-by Qu Wenruo <[email protected]>

---
 fs/btrfs/zlib.c | 10 +++++-----
 fs/btrfs/zstd.c |  4 ++--
 2 files changed, 7 insertions(+), 7 deletions(-)

fs/btrfs/zlib.c +5 -5

index 486b52db583e..e995d0b1452b 100644
@@ -49,7 +49,7 @@ void zlib_free_workspace(struct list_head *ws)
        struct workspace *workspace = list_entry(ws, struct workspace, list);
 
        kvfree(workspace->strm.workspace);
-       kfree(workspace->buf);
+       kvfree(workspace->buf);
        kfree(workspace);
 }
 

@@ -84,13 +84,13 @@ struct list_head *zlib_alloc_workspace(struct btrfs_fs_info *fs_info, unsigned i
        workspace->level = level;
        workspace->buf = NULL;
        if (need_special_buffer(fs_info)) {
-               workspace->buf = kmalloc(ZLIB_DFLTCC_BUF_SIZE,
-                                        __GFP_NOMEMALLOC | __GFP_NORETRY |
-                                        __GFP_NOWARN | GFP_NOIO);
+               workspace->buf = kvmalloc(ZLIB_DFLTCC_BUF_SIZE,
+                                         __GFP_NOMEMALLOC | __GFP_NORETRY |
+                                         __GFP_NOWARN | GFP_NOIO);
                workspace->buf_size = ZLIB_DFLTCC_BUF_SIZE;
        }
        if (!workspace->buf) {
-               workspace->buf = kmalloc(fs_info->sectorsize, GFP_KERNEL);
+               workspace->buf = kvmalloc(fs_info->sectorsize, GFP_KERNEL);
                workspace->buf_size = fs_info->sectorsize;
        }
        if (!workspace->strm.workspace || !workspace->buf)

fs/btrfs/zstd.c +2 -3

index 280ac5273438..cc92d0b1b948 100644
@@ -373,7 +373,7 @@ void zstd_free_workspace(struct list_head *ws)
        struct workspace *workspace = list_entry(ws, struct workspace, list);
 
        kvfree(workspace->mem);
-       kfree(workspace->buf);
+       kvfree(workspace->buf);
        kfree(workspace);
 }
 

@@ -391,7 +391,7 @@ struct list_head *zstd_alloc_workspace(struct btrfs_fs_info *fs_info, int level)
        workspace->req_level = level;
        workspace->last_used = jiffies;
        workspace->mem = kvmalloc(workspace->size, GFP_KERNEL | __GFP_NOWARN);
-       workspace->buf = kmalloc(fs_info->sectorsize, GFP_KERNEL);
+       workspace->buf = kvmalloc(fs_info->sectorsize, GFP_KERNEL);
        if (!workspace->mem || !workspace->buf)
                goto fail;
 
--

2.55.0

With the experimental bs > ps support, btrfs can hit several kmalloc() call sites requesting physically contiguous memory that are larger than a page, sometimes such allocation can be as large as 64K.

This means those kmalloc() has a much higher chance to fail if the system's available memory is very fragmented on 4K page sized systems.

And there is already a report that with 64K nodesize, even bs == ps we can still hit such kmalloc() failure.
(https://lore.kernel.org/linux-btrfs/[email protected]/)

But we do not really require any physically contiguous memory for a lot of kmalloc() call sites, we just want a buffer to save some data temporarily.

So change those possible larger than page size kmalloc() to kvmalloc() to reduce the chance of memory allocation failure.

And since kvmalloc() falls back to kmalloc() if the allocation is not larger than a page, it means for most cases it's still calling kmalloc(), not affecting the most common cases anyway.

The first patch is a special case that affects the existing bs == ps cases already.
The second one only affects bs > ps cases.
The third one mostly affects bs > ps cases, but can still improve S390's hardware accelerated path with 4K page size.

Qu Wenruo (3):
btrfs: use kvmalloc() for overwrite_item()
btrfs: use kvmalloc() for uncompress_inline()
btrfs: use kvmalloc() to allocate compression workspace buffer

  fs/btrfs/inode.c    |  4 ++--
  fs/btrfs/tree-log.c |  4 ++--
  fs/btrfs/zlib.c     | 10 +++++-----
  fs/btrfs/zstd.c     |  4 ++--
  4 files changed, 11 insertions(+), 11 deletions(-)

With the experimental bs > ps support, btrfs can hit several kmalloc() call sites requesting physically contiguous memory that are larger than a page, sometimes such allocation can be as large as 64K.

This means those kmalloc() has a much higher chance to fail if the system's available memory is very fragmented on 4K page sized systems.

And there is already a report that with 64K nodesize, even bs == ps we can still hit such kmalloc() failure.
(https://lore.kernel.org/linux-btrfs/[email protected]/)

But we do not really require any physically contiguous memory for a lot of kmalloc() call sites, we just want a buffer to save some data temporarily.

So change those possible larger than page size kmalloc() to kvmalloc() to reduce the chance of memory allocation failure.

And since kvmalloc() falls back to kmalloc() if the allocation is not larger than a page, it means for most cases it's still calling kmalloc(), not affecting the most common cases anyway.

The first patch is a special case that affects the existing bs == ps cases already.
The second one only affects bs > ps cases.
The third one mostly affects bs > ps cases, but can still improve S390's hardware accelerated path with 4K page size.