btrfs: do more graceful error/warning for 32bit kernel
PATCH v2 1/1 btrfs: do more graceful error/warning for 32bit kernel
Due to the pagecache limit of 32bit systems, btrfs can't access metadata at or beyond (ULONG_MAX + 1) << PAGE_SHIFT.
This is 16T for 4K page size while 256T for 64K page size.
And unlike other fses, btrfs uses internally mapped u64 address space for all of its metadata, this is more tricky than other fses.
Users can have a fs which doesn't have metadata beyond the boundary at mount time, but later balance can cause btrfs to create metadata beyond the boundary.
And modification to MM layer is unrealistic just for such minor use case.
To address such problem, this patch will introduce the following checks, much like how XFS handles such problem:
- Mount time rejection
This will reject any fs which has metadata chunk at or beyond the boundary.
- Mount time early warning
If there is any metadata chunk beyond 5/8 of the boundary, we do an early warning and hope the end user will see it.
- Runtime extent buffer rejection
If we're going to allocate an extent buffer at or beyond the boundary, reject such request with -EOVERFLOW.
This is definitely going to cause problems like transaction abort, but we have no better ways.
- Runtime extent buffer early warning
If an extent buffer beyond 5/8 of the max file size is allocated, do an early warning.
Above error/warning message will only be outputted once for each fs to reduce dmesg flood.
---
Changelog:
v2:
- Rebased to latest misc-next and add reviewed-by tag
--- fs/btrfs/ctree.h | 18 +++++++++++++++ fs/btrfs/extent_io.c | 12 ++++++++++ fs/btrfs/super.c | 26 ++++++++++++++++++++++ fs/btrfs/volumes.c | 53 ++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 107 insertions(+), 2 deletions(-)
fs/btrfs/ctree.h +18 -0
@@ -565,6 +565,12 @@ enum {/* Indicate whether there are any tree modification log users */ BTRFS_FS_TREE_MOD_LOG_USERS, + +#if BITS_PER_LONG ==32 +/* Indicate if we have error/warn message outputted for 32bit system */ + BTRFS_FS_32BIT_ERROR, + BTRFS_FS_32BIT_WARN, +#endif };/*
@@ -3392,6 +3398,18 @@ static inline void assertfail(const char *expr, const char* file, int line) { }#define ASSERT (expr) (void)(expr)#endif +#if BITS_PER_LONG ==32 +#define BTRFS_32BIT_MAX_FILE_SIZE (((u64)ULONG_MAX + 1) << PAGE_SHIFT) +/* +* The warning threshold is 5/8 of the max file size. +* +* For 4K page size it should be 10T, for 64K it would 160T. +*/ +#define BTRFS_32BIT_EARLY_WARN_THRESHOLD (BTRFS_32BIT_MAX_FILE_SIZE * 5 / 8) +void btrfs_warn_32bit_limit (struct btrfs_fs_info *fs_info); +void btrfs_err_32bit_limit (struct btrfs_fs_info *fs_info); +#endif +/* * Get the correct offset inside the page of extent buffer. *
fs/btrfs/extent_io.c +12 -0
@@ -5594,6 +5594,18 @@ struct extent_buffer *alloc_extent_buffer(struct btrfs_fs_info *fs_info,return ERR_PTR (-EINVAL ); } +#if BITS_PER_LONG ==32 +if (start >=MAX_LFS_FILESIZE ) { +btrfs_err (fs_info, +"extent buffer %llu is beyond 32bit page cache limit" , + start); +btrfs_err_32bit_limit (fs_info); +return ERR_PTR (-EOVERFLOW ); + } +if (start >=BTRFS_32BIT_EARLY_WARN_THRESHOLD ) +btrfs_warn_32bit_limit (fs_info); +#endif +if (fs_info->sectorsize <PAGE_SIZE &&offset_in_page (start) + len >PAGE_SIZE ) { btrfs_err(fs_info,
fs/btrfs/super.c +26 -0
@@ -252,6 +252,32 @@ void __cold btrfs_printk(const struct btrfs_fs_info *fs_info, const char *fmt, . }#endif +#if BITS_PER_LONG ==32 +void __cold btrfs_warn_32bit_limit (struct btrfs_fs_info *fs_info) +{ +if (!test_and_set_bit (BTRFS_FS_32BIT_WARN , &fs_info->flags)) { +btrfs_warn (fs_info,"btrfs is reaching 32bit kernel limit." ); +btrfs_warn (fs_info, +"due to 32bit page cache limit, btrfs can't access metadata at or beyond %lluT." , +BTRFS_32BIT_MAX_FILE_SIZE >>40 ); +btrfs_warn (fs_info, +"please consider upgrade to 64bit kernel/hardware." ); + } +} + +void __cold btrfs_err_32bit_limit (struct btrfs_fs_info *fs_info) +{ +if (!test_and_set_bit (BTRFS_FS_32BIT_ERROR , &fs_info->flags)) { +btrfs_err (fs_info,"btrfs reached 32bit kernel limit." ); +btrfs_err (fs_info, +"due to 32bit page cache limit, btrfs can't access metadata at or beyond %lluT." , +BTRFS_32BIT_MAX_FILE_SIZE >>40 ); +btrfs_err (fs_info, +"please consider upgrade to 64bit kernel/hardware." ); + } +} +#endif +/* * We only mark the transaction aborted and then set the file system read-only. * This will prevent new transactions from starting or trying to join this
fs/btrfs/volumes.c +51 -2
81 lines not on this page · raw message
Signature · 1 line
2.31.1
Quoted 35 lines · Qu Wenruo
Due to the pagecache limit of 32bit systems, btrfs can't access metadata at or beyond (
ULONG_MAX+ 1) <<PAGE_SHIFT.
This is 16T for 4K page size while 256T for 64K page size.And unlike other fses, btrfs uses internally mapped u64 address space for all of its metadata, this is more tricky than other fses.
Users can have a fs which doesn't have metadata beyond the boundary at mount time, but later balance can cause btrfs to create metadata beyond the boundary.
And modification to MM layer is unrealistic just for such minor use case.
To address such problem, this patch will introduce the following checks, much like how XFS handles such problem:
- Mount time rejection
This will reject any fs which has metadata chunk at or beyond the boundary.- Mount time early warning
If there is any metadata chunk beyond 5/8 of the boundary, we do an early warning and hope the end user will see it.- Runtime extent buffer rejection
If we're going to allocate an extent buffer at or beyond the boundary, reject such request with-EOVERFLOW.
This is definitely going to cause problems like transaction abort, but we have no better ways.- Runtime extent buffer early warning
If an extent buffer beyond 5/8 of the max file size is allocated, do an early warning.
Quoted 6 lines
Above error/warning message will only be outputted once for each fs to reduce dmesg flood.
Reported-by Erik Jensen <[email protected]>Signed-off-by Qu Wenruo <[email protected]>Reviewed-by Josef Bacik <[email protected]>
I've added some references and updated wording what's the problem. Patch moved from topic branch to misc-next, thanks.