Preamture OOM fixes
cover Preamture OOM fixes
David reported that following the async cow path refactoring that got into misc-next he started seeing premature OOM kills due to large page allocations (order 4). This was due to now the whole async context (alongisde ancillary structures, describing chunks) being allocated in a single kmalloc call. The first patch of this series aims to fix that by converting the kmalloc call to kvmalloc. This will have virtually no perfromance impact for small writes and for larger ones will fallback to vmalloc.
However, while testing the first patch I observed BUG_ONs in the async csum calculation path to trigger. This is clearly caused by the fact that the first patch introduces a fairly large memory allocation. Nevertheless, having large allocations in the csum path without a fallback to some sort of order 0 page allocation or at least graceful handling of memory errors is gross. Patch 2 aims to rectify this by switching the allocation in btrfs_csum_one_bio to using kvmalloc, giving greated chance for success. Even if patch 1 is ignored nothing prevents the csum calculation to trigger the bug on depending on the memory usage/fragmentation of the server so IMO it's beneficial either ways.
Nikolay Borisov (2):
btrfs: Use kvmalloc for allocating compressed path context
btrfs: Switch memory allocations in async csum calculation path to
kvmalloc
fs/btrfs/file-item.c | 15 +++++++++++---- fs/btrfs/inode.c | 9 +++++++-- fs/btrfs/ordered-data.c | 3 ++- 3 files changed, 20 insertions(+), 7 deletions(-)
Signature · 1 line
2.17.1
PATCH 1/2 btrfs: Use kvmalloc for allocating compressed path context
Recent refactoring of cow_file_range_async means it's now possible to request a rather large physically contiguous memory via kmalloc. The size is dependent on the number of 512k chunks that the compressed range consists of. David reported multiple OOM messages on such large allocations. Fix it by switching to using kvmalloc.
--- fs/btrfs/inode.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
fs/btrfs/inode.c +7 -2
@@ -28,6 +28,7 @@#include <linux/magic.h> #include <linux/iversion.h> #include <linux/swap.h> +#include <linux/sched/mm.h> #include <asm/unaligned.h> #include "ctree.h" #include "disk-io.h"
@@ -1172,7 +1173,7 @@ static noinline void async_cow_free(struct btrfs_work *work)* async_chunk's, freeing it ensures the whole array has been freed. */ if (atomic_dec_and_test (async_chunk->pending)) -kfree (async_chunk->pending); +kvfree (async_chunk->pending); } static int cow_file_range_async(struct inode *inode, struct page *locked_page,
@@ -1188,6 +1189,7 @@ static int cow_file_range_async(struct inode *inode, struct page *locked_page,u64 num_chunks =DIV_ROUND_UP (end - start,SZ_512K );int i;bool should_compress; +unsigned nofs_flag;unlock_extent (&BTRFS_I (inode)->io_tree, start, end);
@@ -1199,7 +1201,10 @@ static int cow_file_range_async(struct inode *inode, struct page *locked_page, should_compress = true; } - ctx =kmalloc (struct_size (ctx, chunks, num_chunks),GFP_NOFS ); + nofs_flag =memalloc_nofs_save (); + ctx =kvmalloc (struct_size (ctx, chunks, num_chunks),GFP_KERNEL ); +memalloc_nofs_restore (nofs_flag); +if (!ctx) {unsigned clear_bits =EXTENT_LOCKED |EXTENT_DELALLOC | EXTENT_DELALLOC_NEW | EXTENT_DEFRAG |
Signature · 1 line
2.17.1
PATCH 2/2 btrfs: Switch memory allocations in async csum calculation path to kvmalloc
In reply to Nikolay BorisovFor performance reasons checksum calculation on pending writes is performed in asynchronous context. This requires doing potentially large order memory allocations (in my testing I've seen up to order 6 or 512k). This could put quite a strain on the slab allocator since it's not guaranteed such a large, phisically contiguous memory allocation can succeed - allocation could fail because of memory fragmentation in addition to exhaustion. To add insult to injury, the code path in question can't handle allocation failure gracefully, instead it just BUGs.
This patch tries to alleviate the issue by switching the allocation from using kmalloc to using kvmalloc. For small writes this is unlikely to have any visible effect since kmalloc will still satisfy allocation requests. For larger requests the code will just fallback to vmalloc.
--- fs/btrfs/file-item.c | 15 +++++++++++---- fs/btrfs/ordered-data.c | 3 ++- 2 files changed, 13 insertions(+), 5 deletions(-)
fs/btrfs/file-item.c +11 -4
@@ -7,6 +7,7 @@#include <linux/slab.h> #include <linux/pagemap.h> #include <linux/highmem.h> +#include <linux/sched/mm.h> #include "ctree.h" #include "disk-io.h" #include "transaction.h"
@@ -427,9 +428,13 @@ blk_status_t btrfs_csum_one_bio(struct inode *inode, struct bio *bio,unsigned long this_sum_bytes =0 ;int i;u64 offset; +unsigned nofs_flag; + + nofs_flag =memalloc_nofs_save (); + sums =kvmalloc (btrfs_ordered_sum_size (fs_info, bio->bi_iter.bi_size), +GFP_KERNEL ); +memalloc_nofs_restore (nofs_flag); - sums =kzalloc (btrfs_ordered_sum_size (fs_info, bio->bi_iter.bi_size), -GFP_NOFS );if (!sums)return BLK_STS_RESOURCE ;
@@ -469,8 +474,10 @@ blk_status_t btrfs_csum_one_bio(struct inode *inode, struct bio *bio, bytes_left = bio->bi_iter.bi_size - total_bytes; - sums =kzalloc (btrfs_ordered_sum_size (fs_info, bytes_left), -GFP_NOFS ); + nofs_flag =memalloc_nofs_save (); + sums =kzalloc (btrfs_ordered_sum_size (fs_info, + bytes_left),GFP_KERNEL ); +memalloc_nofs_restore (nofs_flag);BUG_ON (!sums);/* -ENOMEM */ sums->len = bytes_left; ordered = btrfs_lookup_ordered_extent(inode,
fs/btrfs/ordered-data.c +2 -1
@@ -6,6 +6,7 @@#include <linux/slab.h> #include <linux/blkdev.h> #include <linux/writeback.h> +#include <linux/sched/mm.h> #include "ctree.h" #include "transaction.h" #include "btrfs_inode.h"
@@ -441,7 +442,7 @@ void btrfs_put_ordered_extent(struct btrfs_ordered_extent *entry) cur = entry->list.next; sum = list_entry(cur, struct btrfs_ordered_sum, list);list_del (&sum->list); -kfree (sum); +kvfree (sum); }kmem_cache_free (btrfs_ordered_extent_cache, entry); }
Signature · 1 line
2.17.1
Quoted 8 lines · Nikolay Borisov
David reported that following the async cow path refactoring that got into misc-next he started seeing premature OOM kills due to large page allocations (order 4). This was due to now the whole async context (alongisde ancillary structures, describing chunks) being allocated in a single kmalloc call. The first patch of this series aims to fix that by converting the kmalloc call to kvmalloc. This will have virtually no perfromance impact for small writes and for larger ones will fallback to vmalloc.
Quoted 9 lines
However, while testing the first patch I observed
BUG_ONsin the async csum calculation path to trigger. This is clearly caused by the fact that the first patch introduces a fairly large memory allocation. Nevertheless, having large allocations in the csum path without a fallback to some sort of order 0 page allocation or at least graceful handling of memory errors is gross. Patch 2 aims to rectify this by switching the allocation inbtrfs_csum_one_bioto using kvmalloc, giving greated chance for success. Even if patch 1 is ignored nothing prevents the csum calculation to trigger the bug on depending on the memory usage/fragmentation of the server so IMO it's beneficial either ways.
For the record: the patches that introduced the preallocation + followup cleanups have been removed from misc-next until we're sure that the supposed change in IO submission path does not introduce further surprises.
The potential performance impact is about to be evaluated. The error not-handling currently depends on the kmalloc behaviour that does not fail for small allocations, so current code is safer for users.
The preallocation provides a nicer way to handle errors early and fails gracefully, unfortunately this is out of the safe zone of kmalloc so this introduces a new problem.
The io submission path is probably the biggest and worst BUG_ON error handling misuse, but also the hardest one to fix. We'll get there, but as the prealloc patches showed we need to be careful.