ext4: fix NULL pointer dereference in ext4_mb_load_buddy_gfp()

Hello,

We have hit a kernel NULL pointer dereference in ext4_mb_load_buddy_gfp() twice in one day on a freshly installed server. Both crashes are identical (same RIP and offset) and were captured by kdump. Patch 1 fixes it; this cover letter contains the full crash data and the analysis.

Environment
-----------

Kernel: 6.1.192-1.el9.elrepo.x86_64 (vanilla linux-6.1.y built by ELRepo;
the affected code is identical in current mainline)

Hardware:    ASUSTeK RS500-E9-PS4 (2 sockets, 64 GiB RAM)
Filesystem:  ext4 on LVM on md-raid1, 120 GiB, block size 4096,
             features: has_journal ext_attr resize_inode dir_index filetype
             extent 64bit flex_bg sparse_super large_file huge_file
             dir_nlink extra_isize metadata_csum
Mount:       rw,relatime
Workload:    fresh OS install followed by heavy package installation and

database restore (lots of writes and page cache churn); both crashes happened in background writeback (kworker), not tied to any specific userspace task.

Crash 1 (kdump at 19:26:27, 4683s after boot)
---------------------------------------------

[ 4683.607255] BUG: kernel NULL pointer dereference, address: 0000000000000000
[ 4683.619530] #PF: supervisor read access in kernel mode
[ 4683.624757] #PF: error_code(0x0000) - not-present page
[ 4683.627387] Oops: 0000 [#1] PREEMPT SMP NOPTI
[ 4683.631834] CPU: 51 PID: 137989 Comm: kworker/u161:1 Kdump: loaded Tainted: G            E      6.1.192-1.el9.elrepo.x86_64 #1
[ 4683.652787] Workqueue: writeback wb_workfn (flush-253:1)
[ 4683.658194] RIP: 0010:ext4_mb_load_buddy_gfp+0xc0/0x3a0 [ext4]
[ 4683.664141] Code: 48 63 f0 89 54 24 14 ba 01 00 00 00 48 89 74 24 08 e8 44 49 ec f5 48 8b 74 24 08 48 3d 00 f0 ff ff 49 89 c7 0f 87 32 01 00 00 <48> 8b 00 a8 04 0f 84 1c 01 00 00 49 8b 07 a8 01 0f 85 11 01 00 00
[ 4683.688284] RAX: 0000000000000000 RBX: 00000000000000a2 RCX: 0000000000000022

[ 4683.765356] Call Trace:
[ 4683.767559]  <TASK>
[ 4683.773060]  ext4_mb_regular_allocator+0x1ba/0x600 [ext4]
[ 4683.777952]  ? ext4_find_extent+0x378/0x440 [ext4]
[ 4683.782848]  ext4_mb_new_blocks+0x3a9/0x530 [ext4]
[ 4683.787823]  ext4_ext_map_blocks+0x670/0x8a0 [ext4]
[ 4683.792460]  ext4_map_blocks+0x22b/0x5d0 [ext4]
[ 4683.797441]  mpage_map_one_extent+0x60/0x170 [ext4]
[ 4683.797441]  mpage_map_and_submit_extent+0x76/0x300 [ext4]
[ 4683.803020]  ext4_writepages+0x6f8/0x860 [ext4]
[ 4683.818105]  do_writepages+0xcc/0x1d0
[ 4683.821867]  __writeback_single_inode+0x41/0x270
[ 4683.826575]  writeback_sb_inodes+0x209/0x4c0
[ 4683.830937]  __writeback_inodes_wb+0x4c/0xe0
[ 4683.835300]  wb_writeback+0x1d7/0x2d0
[ 4683.839058]  wb_do_writeback+0x22a/0x2b0
[ 4683.843080]  wb_workfn+0x5d/0x250

Crash 2 (kdump at 20:19:39, 3005s after the following boot) is identical:
same RIP (ext4_mb_load_buddy_gfp+0xc0/0x3a0), same workqueue, same instruction sequence, RAX=0, different block group (RBX=0xbe vs 0xa2) and CPU. (The only difference in the Code dump is the relative call displacement - see the note under the disassembly below.)

Analysis
--------

The faulting instruction sequence decodes to (bytes from the Crash 1 dump above; in the Crash 2 dump the call displacement differs - the call to __filemap_get_folio is stored as an R_X86_64_PLT32 relocation against the vmlinux symbol in the module and is resolved at module load time, so its encoded value depends on the kernel and module load addresses of a given boot, while the instruction sequence is identical):

    e8 44 49 ec f5        call  __filemap_get_folio      ; fast path
    48 8b 74 24 08        mov   rsi, [rsp+0x8]
    48 3d 00 f0 ff ff     cmp   rax, 0xfffff000
    49 89 c7              mov   r15, rax
    0f 87 32 01 00 00     ja    err                      ; IS_ERR() only
    <48 8b 00>            mov   rax, [rax]               ; RAX == NULL -> #PF
    a8 04                 test  al, 0x4                  ; PG_uptodate
    0f 84 ...             jz    slow_path
    49 8b 07              mov   rax, [r15]
    a8 01                 test  al, 0x1                  ; PG_locked
    0f 85 ...             jnz   slow_path

This is the fast path in ext4_mb_load_buddy_gfp():

    folio = __filemap_get_folio(inode->i_mapping, pnum, FGP_ACCESSED, 0);
    if (IS_ERR(folio) || !folio_test_uptodate(folio) || folio_test_locked(folio)) {
            ...
    }

__filemap_get_folio() without FGP_CREAT legitimately returns NULL when the folio is not in the page cache ("if (!folio) goto no_page;" in mm/filemap.c). The fast-path condition only checks IS_ERR(folio) before dereferencing the folio, so a NULL return results in a NULL deref.

The ext4_mb_init_group() call at the top of the function is skipped for groups that are already initialized (EXT4_MB_GRP_NEED_INIT not set), so nothing guarantees the buddy cache folio is still in the page cache at this point: the folios of the s_buddy_cache inode are clean and can be reclaimed under memory pressure. The folio conversion series dropped the explicit NULL check that the old find_get_page_flags()-based code had ("page == NULL ||" was the first condition) in both lookups: the bitmap lookup was converted by commit 99b150d84e49 ("ext4: convert bd_bitmap_page to bd_bitmap_folio") and the buddy lookup by commit 5eea586b47f0 ("ext4: convert bd_buddy_page to bd_buddy_folio"), which is where the regression comes from.

The error-path fix from commit c6a6c9694aad ("ext4: fix error pointer dereference in ext4_mb_load_buddy_gfp()") does not cover this case.

The patch switches the fast-path conditions (and the two folio_put() guards) to IS_ERR_OR_NULL(), so a NULL folio falls back to the FGP_CREAT path which re-creates it.

Deterministic reproducer
------------------------

Once the eviction mechanism was understood, the crash turned out to be trivially reproducible on a test VM (same 6.1.192 kernel, stock ext4 module):

    # fallocate -l 1G /root/t.img
    # mkfs.ext4 -F /root/t.img
    # mount -o loop /root/t.img /mnt/t
    # dd if=/dev/urandom of=/mnt/t/f1 bs=1M count=200
    # sync
    # echo 3 > /proc/sys/vm/drop_caches
    # dd if=/dev/urandom of=/mnt/t/f2 bs=1M count=200

The drop_caches evicts the clean s_buddy_cache folios, and the next block allocation in the writeback worker hits the NULL fast path within seconds:

    BUG: kernel NULL pointer dereference, address: 0000000000000000
    CPU: 0 PID: 446 Comm: kworker/u4:4 Kdump: loaded
    Workqueue: writeback wb_workfn (flush-7:0)
    RIP: 0010:ext4_mb_load_buddy_gfp+0xc0/0x3a0 [ext4]
    Code: ... 48 3d 00 f0 ff ff 0f 87 ... <48> 8b 00 a8 04 ...
    RAX: 0000000000000000

Same RIP and the same instruction sequence as in both production crashes.

Verification
------------

I rebuilt the ext4 module against the same kernel and config with only this fix applied (verified via srcversion that the patched module was the one loaded) and re-ran the reproducer above for three full write/drop_caches/write cycles: no oops, no ext4 warnings in dmesg. The unpatched module panics on the first cycle.

Additional data
---------------

vmcores of both production crashes (captured by kdump) and of the reproduced crash on the test VM are available on request. No ext4 errors or warnings preceded the production oopses; the log simply ends a few minutes before the kdump timestamp.

Aleksandr Panfilenko (1):
ext4: fix NULL pointer dereference in ext4_mb_load_buddy_gfp()

 fs/ext4/mballoc.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

2.34.1

__filemap_get_folio() called without FGP_CREAT returns NULL, not an error pointer, when the folio is not present in the page cache. The fast path in ext4_mb_load_buddy_gfp() only checks IS_ERR(folio) before evaluating folio_test_uptodate(folio), so a NULL return leads to a NULL pointer dereference in the folio flags test.

The buddy cache folios can in fact disappear from the page cache: they are clean pages of the s_buddy_cache inode and can be reclaimed under memory pressure, while nothing sets EXT4_MB_GRP_NEED_INIT again, so ext4_mb_init_group() is not called to reload them. We have hit this twice in one day on 6.1.192 during background writeback on a server under heavy page cache churn (fresh OS install + database restore):

    BUG: kernel NULL pointer dereference, address: 0000000000000000
    Oops: 0000 [#1] PREEMPT SMP NOPTI
    CPU: 51 PID: 137989 Comm: kworker/u161:1 Kdump: loaded
    Workqueue: writeback wb_workfn (flush-253:1)
    RIP: 0010:ext4_mb_load_buddy_gfp+0xc0/0x3a0 [ext4]
    Code: ... 48 3d 00 f0 ff ff 0f 87 ... <48> 8b 00 a8 04 ...
    RAX: 0000000000000000
    Call Trace:
     ext4_mb_regular_allocator+0x1ba/0x600 [ext4]
     ext4_mb_new_blocks+0x3a9/0x530 [ext4]
     ext4_ext_map_blocks+0x670/0x8a0 [ext4]
     ext4_map_blocks+0x22b/0x5d0 [ext4]
     mpage_map_one_extent+0x60/0x170 [ext4]
     mpage_map_and_submit_extent+0x76/0x300 [ext4]
     ext4_writepages+0x6f8/0x860 [ext4]
     ...

The faulting instruction is the folio flags load (mov rax,[rax] with RAX == 0) between the IS_ERR() check and the PG_uptodate test.

Before the folio conversion this was handled correctly: the old code used find_get_page_flags() and explicitly tested "page == NULL" first, in both the bitmap lookup (converted by 99b150d84e49) and the buddy lookup (converted by 5eea586b47f0). Treat a NULL folio the same way as the "not uptodate / locked" cases and fall back to the FGP_CREAT path, which will re-create the folio. Also switch the two folio_put() guards to !IS_ERR_OR_NULL() so that we do not call folio_put(NULL) after taking the slow path with a NULL folio.

The error-path fix in c6a6c9694aad ("ext4: fix error pointer dereference in ext4_mb_load_buddy_gfp()") does not cover this case.

The bug reproduces deterministically on 6.1.192: loop-mount a fresh ext4 filesystem, write ~200 MB of files, sync, then run "echo 3 > /proc/sys/vm/drop_caches" and write more files. The drop evicts the buddy cache folios, and the next allocation in the writeback worker hits the NULL fast path within seconds:

    RIP: 0010:ext4_mb_load_buddy_gfp+0xc0/0x3a0 [ext4]
    Workqueue: writeback wb_workfn (flush-7:0)
    RAX: 0000000000000000
    Call Trace:
     ext4_mb_regular_allocator+0x1ba/0x600 [ext4]
     ext4_mb_new_blocks+0x3a9/0x530 [ext4]
     ...

With this patch applied (the ext4 module rebuilt against the same kernel and config), the same sequence completes cleanly over multiple cycles, with no oops and no ext4 warnings in dmesg.

---
 fs/ext4/mballoc.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

index a15285224..11ca5a148 100644
@@ -1715,7 +1715,8 @@ ext4_mb_load_buddy_gfp(struct super_block *sb, ext4_group_t group,
 
 	/* Avoid locking the folio in the fast path ... */
 	folio = __filemap_get_folio(inode->i_mapping, pnum, FGP_ACCESSED, 0);
-	if (IS_ERR(folio) || !folio_test_uptodate(folio) || folio_test_locked(folio)) {
+	if (IS_ERR_OR_NULL(folio) || !folio_test_uptodate(folio) ||
+	    folio_test_locked(folio)) {
 		/*
 		 * folio_test_locked is employed to detect ongoing folio
 		 * migrations, since concurrent migrations can lead to

@@ -1725,7 +1726,7 @@ ext4_mb_load_buddy_gfp(struct super_block *sb, ext4_group_t group,
 		 * try to get the folio with lock in both cases to avoid
 		 * concurrency.
 		 */
-		if (!IS_ERR(folio))
+		if (!IS_ERR_OR_NULL(folio))
 			folio_put(folio);
 		folio = __filemap_get_folio(inode->i_mapping, pnum,
 				FGP_LOCK | FGP_ACCESSED | FGP_CREAT, gfp);

@@ -1774,8 +1775,9 @@ ext4_mb_load_buddy_gfp(struct super_block *sb, ext4_group_t group,
 
 	/* we need another folio for the buddy */
 	folio = __filemap_get_folio(inode->i_mapping, pnum, FGP_ACCESSED, 0);
-	if (IS_ERR(folio) || !folio_test_uptodate(folio) || folio_test_locked(folio)) {
-		if (!IS_ERR(folio))
+	if (IS_ERR_OR_NULL(folio) || !folio_test_uptodate(folio) ||
+	    folio_test_locked(folio)) {
+		if (!IS_ERR_OR_NULL(folio))
 			folio_put(folio);
 		folio = __filemap_get_folio(inode->i_mapping, pnum,
 				FGP_LOCK | FGP_ACCESSED | FGP_CREAT, gfp);

2.34.1

https://lore.kernel.org/linux-ext4/[email protected]/T/#t

Thanks for the pointer - we missed Elliot's series. Please disregard our submission; it is superseded by his.

We reproduced the same bug on 6.1.192 (kernel-lt, AlmaLinux 9) and re-tested Elliot's ext4 patch there (three write/drop_caches/write cycles plus a small-files stress test, no oops).