WARNING in push_jmp_history
Subsystems (4)
syzbot
syzbotHello,
syzbot found the following issue on:
HEAD commit: c02d24a5af66 Add linux-next specific files for 20241003 git tree: linux-next console+strace: https://syzkaller.appspot.com/x/log.txt?x=17382707980000 kernel config: https://syzkaller.appspot.com/x/.config?x=94f9caf16c0af42d dashboard link: https://syzkaller.appspot.com/bug?extid=7e46cdef14bf496a3ab4 compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10b82707980000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16f4c327980000
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/641e642c9432/disk-c02d24a5.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/98aaf20c29e0/vmlinux-c02d24a5.xz
kernel image: https://storage.googleapis.com/syzbot-assets/c23099f2d86b/bzImage-c02d24a5.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
------------[ cut here ]------------ virt_to_cache: Object is not a Slab page! WARNING: CPU: 0 PID: 5232 at mm/slub.c:4655 virt_to_cache mm/slub.c:4655 [inline] WARNING: CPU: 0 PID: 5232 at mm/slub.c:4655 __do_krealloc mm/slub.c:4753 [inline] WARNING: CPU: 0 PID: 5232 at mm/slub.c:4655 krealloc_noprof+0x1b3/0x2e0 mm/slub.c:4838 Modules linked in: CPU: 0 UID: 0 PID: 5232 Comm: syz-executor250 Not tainted 6.12.0-rc1-next-20241003-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024 RIP: 0010:virt_to_cache mm/slub.c:4655 [inline] RIP: 0010:__do_krealloc mm/slub.c:4753 [inline] RIP: 0010:krealloc_noprof+0x1b3/0x2e0 mm/slub.c:4838 Code: 45 31 ff 45 31 f6 45 31 ed e9 21 ff ff ff c6 05 4e 2a 14 0e 01 90 48 c7 c7 24 f2 0b 8e 48 c7 c6 44 f2 0b 8e e8 3e 19 63 ff 90 <0f> 0b 90 90 e9 d9 fe ff ff f3 0f 1e fa 41 8b 45 08 f7 d0 a8 88 0f RSP: 0018:ffffc90003c36ba8 EFLAGS: 00010246 RAX: 3f2bb101b90db800 RBX: 0000000000000000 RCX: ffff88802bb01e00 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 RBP: ffff88807849c000 R08: ffffffff8155d412 R09: 1ffff110170c519a R10: dffffc0000000000 R11: ffffed10170c519b R12: 0000000000004000 R13: 0000000000000201 R14: 0000000000100cc0 R15: dffffc0000000000 FS: 0000555587952380(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00005594dac5c5d8 CR3: 00000000786d6000 CR4: 00000000003526f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <TASK> push_jmp_history+0x13c/0x5c0 kernel/bpf/verifier.c:3497 do_check+0x6716/0xfe40 kernel/bpf/verifier.c:18352 do_check_common+0x14bd/0x1dd0 kernel/bpf/verifier.c:21618 do_check_main kernel/bpf/verifier.c:21709 [inline] bpf_check+0x18a25/0x1e320 kernel/bpf/verifier.c:22421 bpf_prog_load+0x1667/0x20f0 kernel/bpf/syscall.c:2846 __sys_bpf+0x4ee/0x810 kernel/bpf/syscall.c:5634 __do_sys_bpf kernel/bpf/syscall.c:5741 [inline] __se_sys_bpf kernel/bpf/syscall.c:5739 [inline] __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5739 do_syscall_x64 arch/x86/entry/common.c:52 [inline] do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7fc05a9603e9 Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007ffd106d44d8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141 RAX: ffffffffffffffda RBX: 00007ffd106d46b8 RCX: 00007fc05a9603e9 RDX: 0000000000000048 RSI: 00000000200054c0 RDI: 0000000000000005 RBP: 00007fc05a9d4610 R08: 0000000000000000 R09: 0000000000000000 R10: 00000000ffffffff R11: 0000000000000246 R12: 0000000000000001 R13: 00007ffd106d46a8 R14: 0000000000000001 R15: 0000000000000001 </TASK>
---
This report is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at [email protected].
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
Eduard Zingerman
maintainerQuoted 4 lines · syzbot
Hello,
syzbot found the following issue on:
Quoted 8 lines
HEAD commit: c02d24a5af66 Add linux-next specific files for 20241003 git tree: linux-next console+strace: https://syzkaller.appspot.com/x/log.txt?x=17382707980000 kernel config: https://syzkaller.appspot.com/x/.config?x=94f9caf16c0af42d dashboard link: https://syzkaller.appspot.com/bug?extid=7e46cdef14bf496a3ab4 compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10b82707980000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16f4c327980000
When I try this reproducer the bpf syscall never exits (waited for 5 minutes). Here is the reproducer as a selftest:
SEC("kprobe")__success__naked void syzbot_bug(void)
{
asm volatile (
" r2 = *(u32 *)(r1 +140)\n" // 0
" r3 = *(u32 *)(r1 +76)\n" // 1
" r0 = r2\n" // 2
" if w0 > 0xffffff07 goto 1f\n" // 3
" if r3 <= r0 goto +16\n" // 4
" exit\n" // 5
"1: r6 = *(u16 *)(r1 +0)\n" // 6
" r7 = r6\n" // 7
"2: w7 += 447767737\n" // 8
" if w7 & 0x702000 goto 2b\n" // 9
" w7 &= 2024974\n" // 10
" r5 = r1\n" // 11
" r7 += r5\n" // 12
" if r7 s> 0x37d2 goto +0\n" // 13
" w3 *= w0\n" // 14
" r5 -= r7\n" // 15
" r4 = r5\n" // 16
" r0 += -458748\n" // 17
" if r3 < r4 goto 3f\n" // 18
" w0 >>= w0\n" // 19
"3: goto +0\n" // 20
" exit\n" // 21
::: __clobber_all);
}(e.g. can be put to tools/testing/selftests/bpf/progs/verifier_and.c
or any other verifier_*.c).
Inserting a few printks shows that the following instructions are verified in a loop:
... verification starts ...
[ 2.094272] do_check: env->insn_idx 0 [ 2.094345] do_check: env->insn_idx 1 [ 2.094417] do_check: env->insn_idx 2 [ 2.094486] do_check: env->insn_idx 3 [ 2.094585] do_check: env->insn_idx 4 [ 2.094675] do_check: env->insn_idx 5 [ 2.094748] do_check: env->insn_idx 21 [ 2.094879] do_check: env->insn_idx 6 [ 2.095005] do_check: env->insn_idx 7 [ 2.095074] do_check: env->insn_idx 8 <------ let's call this point LBL [ 2.095156] do_check: env->insn_idx 9 [ 2.095264] do_check: env->insn_idx 8 [ 2.095372] do_check: env->insn_idx 9 [ 2.095497] do_check: env->insn_idx 8 [ 2.095579] do_check: env->insn_idx 9 [ 2.095716] do_check: env->insn_idx 8 [ 2.095892] do_check: env->insn_idx 9 [ 2.096070] do_check: env->insn_idx 8 [ 2.096151] do_check: env->insn_idx 9 [ 2.096314] do_check: env->insn_idx 8 [ 2.096402] do_check: env->insn_idx 9 [ 2.096570] do_check: env->insn_idx 8 [ 2.096646] do_check: env->insn_idx 9 [ 2.096840] do_check: env->insn_idx 8 [ 2.096921] do_check: env->insn_idx 9 [ 2.097040] do_check: env->insn_idx 10 [ 2.097113] do_check: env->insn_idx 11 [ 2.097195] do_check: env->insn_idx 12 [ 2.097417] do_check: env->insn_idx 13 [ 2.097521] do_check: env->insn_idx 14 [ 2.097597] do_check: env->insn_idx 15 [ 2.097688] do_check: env->insn_idx 16 [ 2.097774] do_check: env->insn_idx 17 [ 2.097866] do_check: env->insn_idx 18 [ 2.097990] do_check: env->insn_idx 19 [ 2.098050] do_check: env->insn_idx 20 [ 2.098119] do_check: env->insn_idx 21 [ 2.098195] do_check: env->insn_idx 20 [ 2.098347] do_check: env->insn_idx 21 [ 2.098414] do_check: env->insn_idx 14 [ 2.098556] do_check: env->insn_idx 15 [ 2.098629] do_check: env->insn_idx 16 [ 2.098700] do_check: env->insn_idx 17 [ 2.098767] do_check: env->insn_idx 18 [ 2.098842] do_check: env->insn_idx 8 [ 2.098984] do_check: env->insn_idx 9 [ 2.099108] do_check: env->insn_idx 8 [ 2.099171] do_check: env->insn_idx 9 [ 2.099304] do_check: env->insn_idx 8 [ 2.099368] do_check: env->insn_idx 9 [ 2.099505] do_check: env->insn_idx 8 [ 2.099568] do_check: env->insn_idx 9 [ 2.099703] do_check: env->insn_idx 8 [ 2.099774] do_check: env->insn_idx 9 [ 2.099921] do_check: env->insn_idx 8 [ 2.099984] do_check: env->insn_idx 9 [ 2.100133] do_check: env->insn_idx 8 [ 2.100200] do_check: env->insn_idx 9 [ 2.100349] do_check: env->insn_idx 8 [ 2.100413] do_check: env->insn_idx 9 [ 2.100503] do_check: env->insn_idx 10 [ 2.100566] do_check: env->insn_idx 11 [ 2.100636] do_check: env->insn_idx 12 [ 2.100813] do_check: env->insn_idx 13 [ 2.100909] do_check: env->insn_idx 14 [ 2.100972] do_check: env->insn_idx 15 [ 2.101047] do_check: env->insn_idx 16 [ 2.101117] do_check: env->insn_idx 17 [ 2.101185] do_check: env->insn_idx 18 [ 2.101250] do_check: env->insn_idx 14 [ 2.101389] do_check: env->insn_idx 15 [ 2.101462] do_check: env->insn_idx 16 [ 2.101531] do_check: env->insn_idx 17 [ 2.101598] do_check: env->insn_idx 18
... verification repeats from LBL ...
[...]
syzbot
syzbotsyzbot has bisected this issue to:
syzbot has bisected this issue to:
commit d0a38fad51cc70ab3dd3c59b54d8079ac19220b9
Author: Feng Tang <[email protected]> Date: Wed Sep 11 06:45:34 2024 +0000
mm/slub: Improve redzone check and zeroing for krealloc()
bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=11ddbb80580000 start commit: c02d24a5af66 Add linux-next specific files for 20241003 git tree: linux-next final oops: https://syzkaller.appspot.com/x/report.txt?x=13ddbb80580000 console output: https://syzkaller.appspot.com/x/log.txt?x=15ddbb80580000 kernel config: https://syzkaller.appspot.com/x/.config?x=94f9caf16c0af42d dashboard link: https://syzkaller.appspot.com/bug?extid=7e46cdef14bf496a3ab4 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10b82707980000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16f4c327980000
For information about bisection process see: https://goo.gl/tpsmEJ#bisection
Eduard Zingerman
maintainerQuoted 18 lines · syzbot
syzbot has bisected this issue to:
commit
d0a38fad51cc70ab3dd3c59b54d8079ac19220b9Author: Feng Tang <[email protected]> Date: Wed Sep 11 06:45:34 2024 +0000mm/slub: Improve redzone check and zeroing for
krealloc()bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=11ddbb80580000 start commit: c02d24a5af66 Add linux-next specific files for 20241003 git tree: linux-next final oops: https://syzkaller.appspot.com/x/report.txt?x=13ddbb80580000 console output: https://syzkaller.appspot.com/x/log.txt?x=15ddbb80580000 kernel config: https://syzkaller.appspot.com/x/.config?x=94f9caf16c0af42d dashboard link: https://syzkaller.appspot.com/bug?extid=7e46cdef14bf496a3ab4 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10b82707980000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16f4c327980000
Quoted 4 lines
Reported-by [email protected]Fixes d0a38fad51cc ("mm/slub: Improve redzone check and zeroing for krealloc()")For information about bisection process see: https://goo.gl/tpsmEJ#bisection
There are two issues demonstrated by this repro:
- one is mm/slub related;
- another one is verification taking forever.
About the mm/slub related. Applying the following patch with additional logging on top of commit d0a38fad51cc identified by syzbot:
------- 8< ------------------------------------------------------------
kernel/bpf/verifier.c +2 -0
@@ -3494,7 +3494,9 @@ static int push_jmp_history(struct bpf_verifier_env *env, struct bpf_verifier_st cnt++; alloc_size =kmalloc_size_roundup (size_mul (cnt,sizeof (*p))); +printk ("push_jmp_history: #1 cur->jmp_history=%p\n" , cur->jmp_history); p =krealloc (cur->jmp_history, alloc_size,GFP_USER ); +printk ("push_jmp_history: #2 cur->jmp_history=%p\n" , p);if (!p)return -ENOMEM ; cur->jmp_history = p;
mm/slub.c +1 -1
@@ -4627,7 +4627,7 @@ static inline struct kmem_cache *virt_to_cache(const void *obj)struct slab *slab; slab =virt_to_slab (obj); -if (WARN_ONCE (!slab,"%s: Object is not a Slab page!\n" , __func__)) +if (WARN_ONCE (!slab,"%s: Object %p is not a Slab page!\n" , __func__, obj))return NULL ;return slab->slab_cache; }
------------------------------------------------------------ >8 -------
Produces the following log:
l1: [ 2.942120] push_jmp_history: #2 cur->jmp_history=00000000a0f6f503
l2: [ 2.944445] push_jmp_history: #1 cur->jmp_history=00000000a0f6f503
l3: [ 2.944560] ------------[ cut here ]------------
l4: [ 2.944647] virt_to_cache: Object 00000000a0f6f503 is not a Slab page!
l5: [ 2.944765] WARNING: CPU: 0 PID: 145 at mm/slub.c:4630 krealloc_noprof (mm/slub.c:4630 mm/slub.c:4728 mm/slub.c:4813)
l6: [ 2.944906] Modules linked in:
l7: [ 2.945134] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-2.fc40 04/01/2014
l8: [ 2.945285] RIP: 0010:krealloc_noprof (mm/slub.c:4630 mm/slub.c:4728 mm/slub.c:4813)
...
l9: [ 2.952088] BUG: kernel NULL pointer dereference, address: 000000000000001c
l10: [ 2.952171] #PF: supervisor read access in kernel mode
l11: [ 2.952240] #PF: error_code(0x0000) - not-present page
l12: [ 2.952309] PGD 105d51067 P4D 105d51067 PUD 1013d0067 PMD 0
l13: [ 2.952402] Oops: Oops: 0000 [#1] PREEMPT SMP KASAN NOPTI
l14: [ 2.952611] Tainted: [W]=WARN
l15: [ 2.952664] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-2.fc40 04/01/2014
l16: [ 2.952794] RIP: 0010:krealloc_noprof (mm/slub.c:0 mm/slub.c:4729 mm/slub.c:4813) Lines l{1,2,4} show that address 0xa0f6f503 was first allocated by krealloc and then krealloc failed to recognize it as such.
The warning at l3 is reported by virt_to_cache() called from __do_krealloc():
4715 static __always_inline __realloc_size(2) void *
4716 __do_krealloc(const void *p, size_t new_size, gfp_t flags)
4717 {
4718 void *ret;
4719 size_t ks;
4720 int orig_size = 0;
4721 struct kmem_cache *s;
4722
4723 /* Check for double-free. */
4724 if (likely(!ZERO_OR_NULL_PTR(p))) {
4725 if (!kasan_check_byte(p))
4726 return NULL;
4727
4728 s = virt_to_cache(p);
4729 orig_size = get_orig_size(s, (void *)p);
4730 ks = s->object_size;When virt_to_cache() reports the warning it returns NULL. Hence variable 's' at line 4728 is NULL and this causes null pointer dereference at line 4730, reported at l9.
Lines 4725-4730 were changed by commit d0a38fad51cc identified by syzbot, previously 'ks' was identified using other means.
Feng, this issue seem unrelated to BPF verifier, could you please take a look?
Best regards,
Eduard
Quoted 22 lines · Eduard Zingerman
Quoted 21 lines · syzbot
syzbot has bisected this issue to:
commit
d0a38fad51cc70ab3dd3c59b54d8079ac19220b9Author: Feng Tang <[email protected]> Date: Wed Sep 11 06:45:34 2024 +0000mm/slub: Improve redzone check and zeroing for
krealloc()bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=11ddbb80580000 start commit: c02d24a5af66 Add linux-next specific files for 20241003 git tree: linux-next final oops: https://syzkaller.appspot.com/x/report.txt?x=13ddbb80580000 console output: https://syzkaller.appspot.com/x/log.txt?x=15ddbb80580000 kernel config: https://syzkaller.appspot.com/x/.config?x=94f9caf16c0af42d dashboard link: https://syzkaller.appspot.com/bug?extid=7e46cdef14bf496a3ab4 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10b82707980000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16f4c327980000Reported-by [email protected]Fixes d0a38fad51cc ("mm/slub: Improve redzone check and zeroing for krealloc()")
Quoted 8 lines
Quoted 1 line
For information about bisection process see: https://goo.gl/tpsmEJ#bisection
There are two issues demonstrated by this repro:
- one is mm/slub related;
- another one is verification taking forever.About the mm/slub related. Applying the following patch with additional logging on top of commit
d0a38fad51ccidentified by syzbot:
The slab one is known from other reports and the problematic commit was removed from -next since then.
Quoted 87 lines
------- 8< ------------------------------------------------------------
kernel/bpf/verifier.c +2
-0@@ -3494,7 +3494,9 @@ static int push_jmp_history(struct bpf_verifier_env *env, struct bpf_verifier_st cnt++; alloc_size =kmalloc_size_roundup (size_mul (cnt,sizeof (*p))); +printk ("push_jmp_history: #1 cur->jmp_history=%p\n" , cur->jmp_history); p =krealloc (cur->jmp_history, alloc_size,GFP_USER ); +printk ("push_jmp_history: #2 cur->jmp_history=%p\n" , p);if (!p)return -ENOMEM ; cur->jmp_history = p;mm/slub.c +1
-2@@ -4627,7 +4627,7 @@ static inline struct kmem_cache *virt_to_cache(const void *obj)struct slab *slab; slab =virt_to_slab (obj); -if (WARN_ONCE (!slab,"%s: Object is not a Slab page!\n" , __func__)) +if (WARN_ONCE (!slab,"%s: Object %p is not a Slab page!\n" , __func__, obj))return NULL ;return slab->slab_cache; } ------------------------------------------------------------ >8 -------Produces the following log:
l1: [ 2.942120] push_jmp_history: #2 cur->jmp_history=00000000a0f6f503 l2: [ 2.944445] push_jmp_history: #1 cur->jmp_history=00000000a0f6f503 l3: [ 2.944560] ------------[ cut here ]------------ l4: [ 2.944647] virt_to_cache: Object 00000000a0f6f503 is not a Slab page! l5: [ 2.944765] WARNING: CPU: 0 PID: 145 at mm/slub.c:4630 krealloc_noprof (mm/slub.c:4630 mm/slub.c:4728 mm/slub.c:4813) l6: [ 2.944906] Modules linked in: l7: [ 2.945134] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-2.fc40 04/01/2014 l8: [ 2.945285] RIP: 0010:krealloc_noprof (mm/slub.c:4630 mm/slub.c:4728 mm/slub.c:4813) ... l9: [ 2.952088] BUG: kernel NULL pointer dereference, address: 000000000000001c l10: [ 2.952171] #PF: supervisor read access in kernel mode l11: [ 2.952240] #PF: error_code(0x0000) - not-present page l12: [ 2.952309] PGD 105d51067 P4D 105d51067 PUD 1013d0067 PMD 0 l13: [ 2.952402] Oops: Oops: 0000 [#1] PREEMPT SMP KASAN NOPTI l14: [ 2.952611] Tainted: [W]=WARN l15: [ 2.952664] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-2.fc40 04/01/2014 l16: [ 2.952794] RIP: 0010:krealloc_noprof (mm/slub.c:0 mm/slub.c:4729 mm/slub.c:4813)Lines l{1,2,4} show that address
0xa0f6f503was first allocated by krealloc and then krealloc failed to recognize it as such.The warning at l3 is reported by
virt_to_cache()called from__do_krealloc():4715 static __always_inline __realloc_size(2) void * 4716 __do_krealloc(const void *p, size_t new_size, gfp_t flags) 4717 { 4718 void *ret; 4719 size_t ks; 4720 int orig_size = 0; 4721 struct kmem_cache *s; 4722 4723 /* Check for double-free. */ 4724 if (likely(!ZERO_OR_NULL_PTR(p))) { 4725 if (!kasan_check_byte(p)) 4726 return NULL; 4727 4728 s = virt_to_cache(p); 4729 orig_size = get_orig_size(s, (void *)p); 4730 ks = s->object_size;When
virt_to_cache()reports the warning it returns NULL. Hence variable 's' at line 4728 is NULL and this causes null pointer dereference at line 4730, reported at l9.Lines 4725-4730 were changed by commit
d0a38fad51ccidentified by syzbot, previously 'ks' was identified using other means.Feng, this issue seem unrelated to BPF verifier, could you please take a look?
Best regards,
Eduard