Skip to contentSkip to conversation

4.4rc3 nfsd/btrfs kasan warning.

DISCUSSION Dave Jones 4 participants 7 messages Dec 2, 2015

lore
Share & tools
lore

Dave Jones

author
#0 lore raw

Got a few of these in the logs this morning after an overnight rsync over nfs to an exported btrfs volume.

Dave

==================================================================

BUG: KASAN: stack-out-of-bounds in setup_cluster_bitmap+0xc4/0x5a0 at addr ffff88039bef6828
Read of size 8 by task nfsd/1009
page:ffffea000e6fbd80 count:0 mapcount:0 mapping:          (null) index:0x0
flags: 0x8000000000000000()
page dumped because: kasan: bad access detected
CPU: 1 PID: 1009 Comm: nfsd Tainted: G        W       4.4.0-rc3-backup-debug+ #1
 ffff880065647b50 000000006bb712c2 ffff88039bef6640 ffffffffa680a43e
 0000004559c00000 ffff88039bef66c8 ffffffffa62638d1 ffffffffa61121c0
 ffff8803a5769de8 0000000000000296 ffff8803a5769df0 0000000000046280
Call Trace:
 [<ffffffffa680a43e>] dump_stack+0x4b/0x6d
 [<ffffffffa62638d1>] kasan_report_error+0x501/0x520
 [<ffffffffa61121c0>] ? debug_show_all_locks+0x1e0/0x1e0
 [<ffffffffa6263948>] kasan_report+0x58/0x60
 [<ffffffffa6814b00>] ? rb_last+0x10/0x40
 [<ffffffffa66f8af4>] ? setup_cluster_bitmap+0xc4/0x5a0
 [<ffffffffa6262ead>] __asan_load8+0x5d/0x70
 [<ffffffffa66f8af4>] setup_cluster_bitmap+0xc4/0x5a0
 [<ffffffffa66f675a>] ? setup_cluster_no_bitmap+0x6a/0x400
 [<ffffffffa66fcd16>] btrfs_find_space_cluster+0x4b6/0x640
 [<ffffffffa66fc860>] ? btrfs_alloc_from_cluster+0x4e0/0x4e0
 [<ffffffffa66fc36e>] ? btrfs_return_cluster_to_free_space+0x9e/0xb0
 [<ffffffffa702dc37>] ? _raw_spin_unlock+0x27/0x40
 [<ffffffffa666a1a1>] find_free_extent+0xba1/0x1520
 [<ffffffffa6669600>] ? btrfs_delalloc_reserve_space+0x70/0x70
 [<ffffffffa6119276>] ? do_raw_spin_lock+0x116/0x1a0
 [<ffffffffa6119407>] ? do_raw_spin_unlock+0x97/0x130
 [<ffffffffa702dc37>] ? _raw_spin_unlock+0x27/0x40
 [<ffffffffa6651555>] ? get_alloc_profile+0x1c5/0x320
 [<ffffffffa666ab90>] ? btrfs_reserve_extent+0x70/0x1d0
 [<ffffffffa666abe0>] btrfs_reserve_extent+0xc0/0x1d0
 [<ffffffffa666b0af>] btrfs_alloc_tree_block+0x3bf/0x680
 [<ffffffffa61121c0>] ? debug_show_all_locks+0x1e0/0x1e0
 [<ffffffffa666acf0>] ? btrfs_reserve_extent+0x1d0/0x1d0
 [<ffffffffa62633b6>] ? memcpy+0x36/0x40
 [<ffffffffa66c3337>] ? read_extent_buffer+0xe7/0x160
 [<ffffffffa6642c0f>] __btrfs_cow_block+0x28f/0x9b0
 [<ffffffffa6208a28>] ? mark_page_accessed+0x18/0xd0
 [<ffffffffa6642980>] ? update_ref_for_cow+0x540/0x540
 [<ffffffffa6133335>] ? debug_lockdep_rcu_enabled+0x35/0x40
 [<ffffffffa66e96af>] ? btrfs_try_tree_write_lock+0x5f/0xe0
 [<ffffffffa66e90f0>] ? btrfs_set_lock_blocking_rw+0x110/0x160
 [<ffffffffa66435cf>] btrfs_cow_block+0x1cf/0x380
 [<ffffffffa6649773>] btrfs_search_slot+0x413/0x11e0
 [<ffffffffa6649360>] ? split_leaf+0xc50/0xc50
 [<ffffffffa6641686>] ? btrfs_alloc_path+0x26/0x30
 [<ffffffffa625bba3>] ? set_track+0x83/0x140
 [<ffffffffa610f66d>] ? mark_lock+0x6d/0x8a0
 [<ffffffffa6671cea>] btrfs_lookup_csum+0xba/0x260
 [<ffffffffa610d244>] ? __lock_is_held+0x84/0xc0
 [<ffffffffa6671c30>] ? truncate_one_csum+0x1c0/0x1c0
 [<ffffffffa613325a>] ? rcu_read_lock_sched_held+0x8a/0xa0
 [<ffffffffa625fbc3>] ? kmem_cache_alloc+0x1c3/0x280
 [<ffffffffa6673f8d>] btrfs_csum_file_blocks+0x2bd/0xac0
 [<ffffffffa6673cd0>] ? btrfs_del_csums+0x490/0x490
 [<ffffffffa6260b87>] ? kfree+0xb7/0x230
 [<ffffffffa676aa5a>] ? copy_items+0x6ab/0xd2d
 [<ffffffffa676aa5a>] ? copy_items+0x6ab/0xd2d
 [<ffffffffa676aa89>] copy_items+0x6da/0xd2d
 [<ffffffffa66e9001>] ? btrfs_set_lock_blocking_rw+0x21/0x160
 [<ffffffffa676a3af>] ? assfail.constprop.22+0x1e/0x1e
 [<ffffffffa664ec61>] ? btrfs_search_forward+0x541/0x600
 [<ffffffffa66c3337>] ? read_extent_buffer+0xe7/0x160
 [<ffffffffa66ec627>] ? btrfs_item_key_to_cpu+0xb7/0xf0
 [<ffffffffa66ec570>] ? check_parent_dirs_for_sync+0x200/0x200
 [<ffffffffa676c6e0>] btrfs_log_inode+0x7a9/0x11fa
 [<ffffffffa676bf37>] ? btrfs_log_changed_extents+0x883/0x883
 [<ffffffffa610f66d>] ? mark_lock+0x6d/0x8a0
 [<ffffffffa610ff2e>] ? mark_held_locks+0x8e/0xc0
 [<ffffffffa7027f95>] ? mutex_lock_nested+0x3a5/0x510
 [<ffffffffa61100f2>] ? trace_hardirqs_on_caller+0x192/0x290
 [<ffffffffa610f66d>] ? mark_lock+0x6d/0x8a0
 [<ffffffffa6133335>] ? debug_lockdep_rcu_enabled+0x35/0x40
 [<ffffffffa610ff2e>] ? mark_held_locks+0x8e/0xc0
 [<ffffffffa7027a00>] ? __mutex_unlock_slowpath+0xe0/0x1c0
 [<ffffffffa61100f2>] ? trace_hardirqs_on_caller+0x192/0x290
 [<ffffffffa61101fd>] ? trace_hardirqs_on+0xd/0x10
 [<ffffffffa66f1cf4>] btrfs_log_inode_parent+0x404/0x1440
 [<ffffffffa61121c0>] ? debug_show_all_locks+0x1e0/0x1e0
 [<ffffffffa61121c0>] ? debug_show_all_locks+0x1e0/0x1e0
 [<ffffffffa66f18f0>] ? btrfs_end_log_trans+0x50/0x50
 [<ffffffffa6133335>] ? debug_lockdep_rcu_enabled+0x35/0x40
 [<ffffffffa612e93a>] ? debug_lockdep_rcu_enabled.part.36+0x1a/0x30
 [<ffffffffa6133335>] ? debug_lockdep_rcu_enabled+0x35/0x40
 [<ffffffffa6290aae>] ? dget_parent+0x8e/0x2f0
 [<ffffffffa6290ade>] ? dget_parent+0xbe/0x2f0
 [<ffffffffa66f46aa>] btrfs_log_dentry_safe+0x6a/0x90
 [<ffffffffa66aca5f>] btrfs_sync_file+0x4df/0x690
 [<ffffffffa66ac580>] ? start_ordered_ops+0x30/0x30
 [<ffffffffa62d4830>] ? __fsnotify_update_child_dentry_flags+0x30/0x30
 [<ffffffffa62bdc3d>] vfs_fsync_range+0x5d/0x120
 [<ffffffffa66ac580>] ? start_ordered_ops+0x30/0x30
 [<ffffffffa64ae7c6>] nfsd_vfs_write+0x356/0x650
 [<ffffffffa64ae470>] ? nfsd_readv+0xa0/0xa0
 [<ffffffffa6133335>] ? debug_lockdep_rcu_enabled+0x35/0x40
 [<ffffffffa64b230f>] nfsd_write+0xff/0x120
 [<ffffffffa6839e34>] ? __list_add+0x74/0xf0
 [<ffffffffa64bb4f7>] nfsd3_proc_write+0x1c7/0x2d0
 [<ffffffffa64b7bdf>] ? nfsd_cache_lookup+0x6ef/0xa90
 [<ffffffffa64bb330>] ? nfsd3_proc_symlink+0x1f0/0x1f0
 [<ffffffffa64a4b15>] nfsd_dispatch+0x185/0x370
 [<ffffffffa64bb330>] ? nfsd3_proc_symlink+0x1f0/0x1f0
 [<ffffffffa6fecd96>] svc_process_common+0x8c6/0xda0
 [<ffffffffa64a4990>] ? nfsd_svc+0x770/0x770
 [<ffffffffa6fec4d0>] ? svc_printk+0x180/0x180
 [<ffffffffa610d1e5>] ? __lock_is_held+0x25/0xc0
 [<ffffffffa6feefbb>] svc_process+0x22b/0x450
 [<ffffffffa64a3cfc>] nfsd+0x23c/0x370
 [<ffffffffa64a3ac5>] ? nfsd+0x5/0x370
 [<ffffffffa64a3ac0>] ? nfsd_destroy+0x1f0/0x1f0
 [<ffffffffa60ce496>] kthread+0x196/0x1c0
 [<ffffffffa60ce300>] ? __kthread_parkme+0xe0/0xe0
 [<ffffffffa610fec3>] ? mark_held_locks+0x23/0xc0
 [<ffffffffa60ce300>] ? __kthread_parkme+0xe0/0xe0
 [<ffffffffa702e82f>] ret_from_fork+0x3f/0x70
 [<ffffffffa60ce300>] ? __kthread_parkme+0xe0/0xe0
Memory state around the buggy address:
 ffff88039bef6700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 ffff88039bef6780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>ffff88039bef6800: 00 00 00 00 00 f1 f1 f1 f1 00 00 f4 f4 f3 f3 f3
                                  ^
 ffff88039bef6880: f3 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 ffff88039bef6900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
#1 lore raw
Quoted 2 lines · Dave Jones

Got a few of these in the logs this morning after an overnight rsync over nfs to an exported btrfs volume.

That's probably us and not NFS, what line is that in setup_cluster_bitmap? Thanks,

Josef

author
#2 lore raw
Quoted 6 lines · Josef Bacik
Quoted 2 lines · Dave Jones

Got a few of these in the logs this morning after an overnight rsync over nfs to an exported btrfs volume.

That's probably us and not NFS, what line is that in setup_cluster_bitmap? Thanks,

If my math is correct, it's this..

        if (entry->offset != bitmap_offset)

I don't seem to be able to trigger it on demand unfortunatly.

Dave

#3 lore raw
Quoted 5 lines · Dave Jones
Quoted 4 lines · Josef Bacik
Quoted 2 lines · Dave Jones

Got a few of these in the logs this morning after an overnight rsync over nfs to an exported btrfs volume.

Quoted 8 lines
Quoted 2 lines

That's probably us and not NFS, what line is that in setup_cluster_bitmap? Thanks,

If my math is correct, it's this..

        if (entry->offset != bitmap_offset)

I don't seem to be able to trigger it on demand unfortunatly.

Is it possible we're blowing the stack? It seems pretty tricky to get a stack out of bounds out of this code without flat out blowing through it.

-chris

author
#4 lore raw
Quoted 11 lines · Chris Mason
Quoted 10 lines · Dave Jones
Quoted 6 lines · Josef Bacik
Quoted 2 lines · Dave Jones

Got a few of these in the logs this morning after an overnight rsync over nfs to an exported btrfs volume.

That's probably us and not NFS, what line is that in setup_cluster_bitmap? Thanks,

If my math is correct, it's this..

Quoted 7 lines
Quoted 3 lines
        if (entry->offset != bitmap_offset)

I don't seem to be able to trigger it on demand unfortunatly.

Is it possible we're blowing the stack? It seems pretty tricky to get a stack out of bounds out of this code without flat out blowing through it.

Hm, there is a lot of debug crap on the stack from lockdep etc, though I didn't get any warnings from the other stack overflow checks.

Dave

#5 lore raw
In reply to Chris Mason

2015-12-02 20:14 GMT+03:00 Chris Mason <[email protected]>:

Quoted 11 lines · Dave Jones
Quoted 10 lines · Josef Bacik
Quoted 6 lines · Dave Jones
Quoted 2 lines

Got a few of these in the logs this morning after an overnight rsync over nfs to an exported btrfs volume.

That's probably us and not NFS, what line is that in setup_cluster_bitmap? Thanks,

If my math is correct, it's this..

Quoted 8 lines
Quoted 3 lines
        if (entry->offset != bitmap_offset)

I don't seem to be able to trigger it on demand unfortunatly.

Is it possible we're blowing the stack? It seems pretty tricky to get a stack out of bounds out of this code without flat out blowing through it.

I think it just empty bitmaps list.
list_first_entry() can't be used on empty list.

BTW, there is similar report
http://lkml.kernel.org/r/<trinity-c7a088d8-bb35-484e-bf27-dbd9a94a804c-1448959367092@3capp-webde-bs56>

#6 lore raw
Quoted 16 lines · Andrey Ryabinin

2015-12-02 20:14 GMT+03:00 Chris Mason <[email protected]>:

Quoted 15 lines · Dave Jones
Quoted 13 lines · Josef Bacik
Quoted 6 lines · Dave Jones
Quoted 2 lines

Got a few of these in the logs this morning after an overnight rsync over nfs to an exported btrfs volume.

That's probably us and not NFS, what line is that in setup_cluster_bitmap? Thanks,

If my math is correct, it's this..

        if (entry->offset != bitmap_offset)

I don't seem to be able to trigger it on demand unfortunatly.

Quoted 7 lines
Quoted 4 lines

Is it possible we're blowing the stack? It seems pretty tricky to get a stack out of bounds out of this code without flat out blowing through it.

I think it just empty bitmaps list.
list_first_entry() can't be used on empty list.

Ohh, I was so busy looking for free'd entries I missed that. Good point.

-chris