kernel BUG at linux-3.16.7-ckt25/fs/dcache.c:2373!
Hello,
probably during heavy IO our file server crashed on a BUG_ON in dache.c, probably triggered by NFS:
Quoted 41 lines
------------[ cut here ]------------ kernel BUG at /var/build/temp/tmp.BPql4ErveJ/pbuilder/linux-3.16.7-ckt25/fs/dcache.c:2373! invalid opcode: 0000 [#1] SMP Modules linked in: nfsv4 dns_resolver nfsv3 btrfs raid6_pq xor crc32c_generic ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi s_acl auth_rpcgss nfs fscache lockd sunrpc quota_v2 quota_tree lpc_ich mfd_core psmouse i2c_i801 tpm_tis processor pcspkr serio_raw i2c_core tpm rng_core thermal_sys rc16 dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm_mod sd_mod crc_t10dif crct10dif_common sg sr_mod cdrom ata_generic ehci_pci uhci_hcd ata_piix e1000 ehci_hcd mptspi mptscsih mptbase floppy scsi_transport_spi usbcore libata button usb_common CPU: 0 PID: 1455 Comm: nfsd Not tainted 3.16.0-ucs193-amd64 #1 Debian 3.16.7-ckt25-2~bpo70+1~ucs3.3.193.201604181018 Hardware name: FUJITSU SIEMENS PRIMERGY RX200S2/D1790/M73IL, BIOS 6.0 Rev. R04A5F5.1790 01/16/2006 task: ffff880121cb03d0 ti: ffff8800d0cac000 task.ti: ffff8800d0cac000 RIP: 0010:[<ffffffff811d2613>] [<ffffffff811d2613>] __d_rehash+0x53/0x60 RSP: 0018:ffff8800d0cafaa0 EFLAGS: 00010282 RAX: 0000000000043e72 RBX: ffff8800939d4578 RCX: 000000000000000d RDX: ffffc90000005000 RSI: ffffc90000224390 RDI: ffff8800b8d53018 RBP: ffff8800b8d53018 R08: 0000000000000000 R09: 0000000000000000 R10: 0100000000000b15 R11: 0000000000000000 R12: ffff8800d1388198 R13: ffff8800b8d53018 R14: ffff8800b8d53b58 R15: ffff8800d0cafbe0 FS: 0000000000000000(0000) GS:ffff880127c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffff600400 CR3: 00000001007d7000 CR4: 00000000000007f0 Stack: ffffffff811d2a3d ffff8800d0cafb27 ffff8800d1388198 ffff8800b8d53b58 ffff8800d1388198 ffff8800b8d53b58 ffff8800b8d53b58 ffff880122829860 ffff8800b8d53b58 ffff8800d0cafbe0 ffffffff811c44f9 ffff8800b8d53b58 Call Trace: [<ffffffff811d2a3d>] ? d_materialise_unique+0x8d/0x3d0 [<ffffffff811c44f9>] ? lookup_real+0x19/0x50 [<ffffffff811c4b27>] ? __lookup_hash+0x37/0x50 [<ffffffff811c92fd>] ? lookup_one_len+0xbd/0x110 [<ffffffff81241159>] ? reconnect_path+0x109/0x2d0 [<ffffffffa0367e20>] ? nfsd_setuser_and_check_port+0xa0/0xa0 [nfsd] [<ffffffff8124144f>] ? exportfs_decode_fh+0x12f/0x300 [<ffffffffa036cee5>] ? expkey_match+0x45/0x50 [nfsd] [<ffffffffa02df314>] ? cache_check+0x114/0x3b0 [sunrpc] [<ffffffffa036da5d>] ? exp_find+0xfd/0x1b0 [nfsd] [<ffffffffa036898c>] ? fh_verify+0x2fc/0x5a0 [nfsd] [<ffffffffa0378df3>] ? nfsd4_proc_compound+0x4c3/0x790 [nfsd] [<ffffffffa0364df4>] ? nfsd_dispatch+0xe4/0x230 [nfsd] [<ffffffffa02d3dae>] ? svc_process_common+0x33e/0x6a0 [sunrpc] [<ffffffffa0364680>] ? nfsd_destroy+0x70/0x70 [nfsd] [<ffffffffa02d44d0>] ? svc_process+0x110/0x160 [sunrpc] [<ffffffffa0364747>] ? nfsd+0xc7/0x140 [nfsd]
Quoted 8 lines
[<ffffffff8108e141>] ? kthread+0xc1/0xe0 [<ffffffff8108e080>] ? flush_kthread_worker+0x80/0x80 [<ffffffff81572598>] ? ret_from_fork+0x58/0x90 [<ffffffff8108e080>] ? flush_kthread_worker+0x80/0x80 Code: 89 47 08 74 04 48 89 50 08 48 89 77 10 48 83 ca 01 48 89 16 0f ba 36 00 c3 0f 1f 80 00 00 00 00 f3 90 48 8b 06 a8 01 75 f7 eb b9 <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 8b 47 RIP [<ffffffff811d2613>] __d_rehash+0x53/0x60 RSP <ffff8800d0cafaa0> ---[ end trace 18587f42c106645d ]---
There was a report 4 years ago where the same BUG_ON() was hit, but during some renames:
<http://linux-fsdevel.vger.kernel.narkive.com/GcMw2p94/kernel-bug-at-fs-dcache-c-2105-d-rehash-bug-on-d-unhashed-entry>
Our kernel is 3.16.7-ckt25 from Debian + ckt26 and ckt27 on top. I don't know exactly what our server was doing at that time, but we were setting up a Debian mirror at that time on "btrfs" on an iSCSI storage connected to that host.
The host also imports 2 NFS shares from other servers and exports 3 NFS volumes itself.
Last week-end the same server got stuck while being backed up:
Quoted 28 lines
BUG: soft lockup - CPU#1 stuck for 22s! [bacula-fd:16612] Modules linked in: nfsv4 dns_resolver nfsv3 btrfs raid6_pq xor crc32c_generic ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi rpcsec_gss_krb5 nfsd nfs_acl auth_rpcgss nfs fscache lockd sunrpc quota_v2 quota_tree lpc_ich mfd_core psmouse tpm_tis tpm i2c_i801 rng_core serio_raw processor i2c_core e752x_edac evdev shpchp edac_core pcspkr thermal_sys ext4 jbd2 crc16 dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm_mod sd_mod crc_t10dif crct10dif_common usb_storage sg sr_mod cdrom ata_generic floppy ehci_pci uhci_hcd ata_piix ehci_hcd mptspi mptscsih libata usbcore e1000 mptbase scsi_transport_spi button usb_common CPU: 1 PID: 16612 Comm: bacula-fd Not tainted 3.16.0-ucs193-amd64 #1 Debian 3.16.7-ckt25-2~bpo70+1~ucs3.3.193.201604181018 Hardware name: FUJITSU SIEMENS PRIMERGY RX200S2/D1790/M73IL, BIOS 6.0 Rev. R04A5F5.1790 01/16/2006 task: ffff8800c6bf6ce0 ti: ffff8800732cc000 task.ti: ffff8800732cc000 RIP: 0010:[<ffffffff811d36c0>] [<ffffffff811d36c0>] __d_lookup_rcu+0x80/0x150 RSP: 0018:ffff8800732cfc58 EFLAGS: 00000282 RAX: 0000000000000000 RBX: 000102d000000000 RCX: 000000000000000d RDX: ffffc90000005000 RSI: ffff8800732cfe10 RDI: ffff880012c2b918 RBP: ffff880012c2b918 R08: ffff880104804050 R09: 0000000000000023 R10: ffffffffffffffff R11: 7bfe9e189e400000 R12: ffff8800beb5f080 R13: ffff88011b2102e8 R14: ffffffff811a035e R15: 00000000000102d0 FS: 00007fa2e701f700(0000) GS:ffff880127c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffffffffff600400 CR3: 00000000bd005000 CR4: 00000000000007e0 Stack: 0000000000000006 ffff880104804050 0000000000000000 ffffffff811c6449 ffff8800732cfcc0 ffff8800732cfd48 ffff8800732cfd38 0000000000000000 ffff880012c2b918 ffff880121f0e060 ffff8800732cfe00 ffffffff811c569e Call Trace: [<ffffffff811c6449>] ? link_path_walk+0x69/0x850 [<ffffffff811c569e>] ? lookup_fast+0x3e/0x2c0 [<ffffffff811c6d68>] ? path_lookupat+0x138/0x740 [<ffffffff814bda33>] ? tcp_sendmsg+0xd3/0xd20 [<ffffffff812138f5>] ? posix_acl_xattr_get+0x45/0xb0 [<ffffffff811c73a4>] ? filename_lookup+0x34/0xd0 [<ffffffff811c5cf1>] ? getname_flags.part.21+0x91/0x140 [<ffffffff811cb289>] ? user_path_at_empty+0x99/0x120
Quoted 8 lines
[<ffffffff811f7db4>] ? fsnotify+0x234/0x300 [<ffffffff811ba4cf>] ? do_sync_write+0x5f/0x90 [<ffffffff811bfc00>] ? vfs_fstatat+0x40/0x90 [<ffffffff811bfcc2>] ? SYSC_newlstat+0x12/0x30 [<ffffffff811bb33c>] ? vfs_write+0x16c/0x200 [<ffffffff811bb80c>] ? SyS_write+0x7c/0xb0 [<ffffffff8157264d>] ? system_call_fast_compare_end+0x10/0x15 Code: 00 00 00 4d 89 e9 49 c7 c2 ff ff ff ff 49 c1 e9 20 eb 14 0f 1f 84 00 00 00 00 00 48 8b 1b 48 85 db 0f 84 84 00 00 00 4c 8d 63 f8 <8b> 43 fc 48 39 6b 10 75 e7 48 83 7b 08 00 74 e0 89 c2 83 e2 fe
The trace-back triggered by arch_trigger_all_cpu_backtrace() shows the following two functions in addition on top of link_path_walk():
Quoted 3 lines
[<
ffffffff811d36c7>] ?__d_lookup_rcu+0x87/0x150[<ffffffff811c5f3b>] ?__inode_permission+0x7b/0xd0[<ffffffff811c6449>] ?link_path_walk+0x69/0x850
...
Has someone seen similar problems with 3.16?
If you need more info, just ask.
Please cc: on replies as I'm not subscribed to linux-fsdevel (but LKML)
Philipp
Signature · 15 lines
Philipp Hahn Open Source Software Engineer Univention GmbH be open. Mary-Somerville-Str. 1 D-28359 Bremen Tel.: +49 421 22232-0 Fax : +49 421 22232-99 [email protected] http://www.univention.de/ Geschäftsführer: Peter H. Ganten HRB 20755 Amtsgericht Bremen Steuer-Nr.: 71-597-02876
Quoted 55 lines · Philipp Hahn
Hello,
probably during heavy IO our file server crashed on a
BUG_ONindache.c, probably triggered by NFS:Quoted 41 lines
------------[ cut here ]------------ kernel BUG at /var/build/temp/tmp.BPql4ErveJ/pbuilder/linux-3.16.7-ckt25/fs/dcache.c:2373! invalid opcode: 0000 [#1] SMP Modules linked in: nfsv4 dns_resolver nfsv3 btrfs raid6_pq xor crc32c_generic ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi s_acl auth_rpcgss nfs fscache lockd sunrpc quota_v2 quota_tree lpc_ich mfd_core psmouse i2c_i801 tpm_tis processor pcspkr serio_raw i2c_core tpm rng_core thermal_sys rc16 dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm_mod sd_mod crc_t10dif crct10dif_common sg sr_mod cdrom ata_generic ehci_pci uhci_hcd ata_piix e1000 ehci_hcd mptspi mptscsih mptbase floppy scsi_transport_spi usbcore libata button usb_common CPU: 0 PID: 1455 Comm: nfsd Not tainted 3.16.0-ucs193-amd64 #1 Debian 3.16.7-ckt25-2~bpo70+1~ucs3.3.193.201604181018 Hardware name: FUJITSU SIEMENS PRIMERGY RX200S2/D1790/M73IL, BIOS 6.0 Rev. R04A5F5.1790 01/16/2006 task: ffff880121cb03d0 ti: ffff8800d0cac000 task.ti: ffff8800d0cac000 RIP: 0010:[<ffffffff811d2613>] [<ffffffff811d2613>] __d_rehash+0x53/0x60 RSP: 0018:ffff8800d0cafaa0 EFLAGS: 00010282 RAX: 0000000000043e72 RBX: ffff8800939d4578 RCX: 000000000000000d RDX: ffffc90000005000 RSI: ffffc90000224390 RDI: ffff8800b8d53018 RBP: ffff8800b8d53018 R08: 0000000000000000 R09: 0000000000000000 R10: 0100000000000b15 R11: 0000000000000000 R12: ffff8800d1388198 R13: ffff8800b8d53018 R14: ffff8800b8d53b58 R15: ffff8800d0cafbe0 FS: 0000000000000000(0000) GS:ffff880127c00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffff600400 CR3: 00000001007d7000 CR4: 00000000000007f0 Stack: ffffffff811d2a3d ffff8800d0cafb27 ffff8800d1388198 ffff8800b8d53b58 ffff8800d1388198 ffff8800b8d53b58 ffff8800b8d53b58 ffff880122829860 ffff8800b8d53b58 ffff8800d0cafbe0 ffffffff811c44f9 ffff8800b8d53b58 Call Trace: [<ffffffff811d2a3d>] ? d_materialise_unique+0x8d/0x3d0 [<ffffffff811c44f9>] ? lookup_real+0x19/0x50 [<ffffffff811c4b27>] ? __lookup_hash+0x37/0x50 [<ffffffff811c92fd>] ? lookup_one_len+0xbd/0x110 [<ffffffff81241159>] ? reconnect_path+0x109/0x2d0 [<ffffffffa0367e20>] ? nfsd_setuser_and_check_port+0xa0/0xa0 [nfsd] [<ffffffff8124144f>] ? exportfs_decode_fh+0x12f/0x300 [<ffffffffa036cee5>] ? expkey_match+0x45/0x50 [nfsd] [<ffffffffa02df314>] ? cache_check+0x114/0x3b0 [sunrpc] [<ffffffffa036da5d>] ? exp_find+0xfd/0x1b0 [nfsd] [<ffffffffa036898c>] ? fh_verify+0x2fc/0x5a0 [nfsd] [<ffffffffa0378df3>] ? nfsd4_proc_compound+0x4c3/0x790 [nfsd] [<ffffffffa0364df4>] ? nfsd_dispatch+0xe4/0x230 [nfsd] [<ffffffffa02d3dae>] ? svc_process_common+0x33e/0x6a0 [sunrpc] [<ffffffffa0364680>] ? nfsd_destroy+0x70/0x70 [nfsd] [<ffffffffa02d44d0>] ? svc_process+0x110/0x160 [sunrpc] [<ffffffffa0364747>] ? nfsd+0xc7/0x140 [nfsd]Quoted 8 lines
[<ffffffff8108e141>] ? kthread+0xc1/0xe0 [<ffffffff8108e080>] ? flush_kthread_worker+0x80/0x80 [<ffffffff81572598>] ? ret_from_fork+0x58/0x90 [<ffffffff8108e080>] ? flush_kthread_worker+0x80/0x80 Code: 89 47 08 74 04 48 89 50 08 48 89 77 10 48 83 ca 01 48 89 16 0f ba 36 00 c3 0f 1f 80 00 00 00 00 f3 90 48 8b 06 a8 01 75 f7 eb b9 <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 8b 47 RIP [<ffffffff811d2613>] __d_rehash+0x53/0x60 RSP <ffff8800d0cafaa0> ---[ end trace 18587f42c106645d ]---
Quoted 6 lines
There was a report 4 years ago where the same
BUG_ON()was hit, but during some renames:
<http://linux-fsdevel.vger.kernel.narkive.com/GcMw2p94/kernel-bug-at-fs-dcache-c-2105-d-rehash-bug-on-d-unhashed-entry>Our kernel is 3.16.7-ckt25 from Debian + ckt26 and ckt27 on top.
So this is the BUG_ON(!d_unhashed(..)) at the top of __d_rehash? I don't know the -ckt kernels, and I don't have a BUG at line 2373 of dcache.c, or something else?
There have been a lot of changes in the area since 3.16.7.
--b.
Quoted 81 lines
I don't know exactly what our server was doing at that time, but we were setting up a Debian mirror at that time on "btrfs" on an iSCSI storage connected to that host.
The host also imports 2 NFS shares from other servers and exports 3 NFS volumes itself.Last week-end the same server got stuck while being backed up:
Quoted 28 lines
BUG: soft lockup - CPU#1 stuck for 22s! [bacula-fd:16612] Modules linked in: nfsv4 dns_resolver nfsv3 btrfs raid6_pq xor crc32c_generic ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi rpcsec_gss_krb5 nfsd nfs_acl auth_rpcgss nfs fscache lockd sunrpc quota_v2 quota_tree lpc_ich mfd_core psmouse tpm_tis tpm i2c_i801 rng_core serio_raw processor i2c_core e752x_edac evdev shpchp edac_core pcspkr thermal_sys ext4 jbd2 crc16 dm_snapshot dm_bufio dm_mirror dm_region_hash dm_log dm_mod sd_mod crc_t10dif crct10dif_common usb_storage sg sr_mod cdrom ata_generic floppy ehci_pci uhci_hcd ata_piix ehci_hcd mptspi mptscsih libata usbcore e1000 mptbase scsi_transport_spi button usb_common CPU: 1 PID: 16612 Comm: bacula-fd Not tainted 3.16.0-ucs193-amd64 #1 Debian 3.16.7-ckt25-2~bpo70+1~ucs3.3.193.201604181018 Hardware name: FUJITSU SIEMENS PRIMERGY RX200S2/D1790/M73IL, BIOS 6.0 Rev. R04A5F5.1790 01/16/2006 task: ffff8800c6bf6ce0 ti: ffff8800732cc000 task.ti: ffff8800732cc000 RIP: 0010:[<ffffffff811d36c0>] [<ffffffff811d36c0>] __d_lookup_rcu+0x80/0x150 RSP: 0018:ffff8800732cfc58 EFLAGS: 00000282 RAX: 0000000000000000 RBX: 000102d000000000 RCX: 000000000000000d RDX: ffffc90000005000 RSI: ffff8800732cfe10 RDI: ffff880012c2b918 RBP: ffff880012c2b918 R08: ffff880104804050 R09: 0000000000000023 R10: ffffffffffffffff R11: 7bfe9e189e400000 R12: ffff8800beb5f080 R13: ffff88011b2102e8 R14: ffffffff811a035e R15: 00000000000102d0 FS: 00007fa2e701f700(0000) GS:ffff880127c80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffffffffff600400 CR3: 00000000bd005000 CR4: 00000000000007e0 Stack: 0000000000000006 ffff880104804050 0000000000000000 ffffffff811c6449 ffff8800732cfcc0 ffff8800732cfd48 ffff8800732cfd38 0000000000000000 ffff880012c2b918 ffff880121f0e060 ffff8800732cfe00 ffffffff811c569e Call Trace: [<ffffffff811c6449>] ? link_path_walk+0x69/0x850 [<ffffffff811c569e>] ? lookup_fast+0x3e/0x2c0 [<ffffffff811c6d68>] ? path_lookupat+0x138/0x740 [<ffffffff814bda33>] ? tcp_sendmsg+0xd3/0xd20 [<ffffffff812138f5>] ? posix_acl_xattr_get+0x45/0xb0 [<ffffffff811c73a4>] ? filename_lookup+0x34/0xd0 [<ffffffff811c5cf1>] ? getname_flags.part.21+0x91/0x140 [<ffffffff811cb289>] ? user_path_at_empty+0x99/0x120Quoted 8 lines
[<ffffffff811f7db4>] ? fsnotify+0x234/0x300 [<ffffffff811ba4cf>] ? do_sync_write+0x5f/0x90 [<ffffffff811bfc00>] ? vfs_fstatat+0x40/0x90 [<ffffffff811bfcc2>] ? SYSC_newlstat+0x12/0x30 [<ffffffff811bb33c>] ? vfs_write+0x16c/0x200 [<ffffffff811bb80c>] ? SyS_write+0x7c/0xb0 [<ffffffff8157264d>] ? system_call_fast_compare_end+0x10/0x15 Code: 00 00 00 4d 89 e9 49 c7 c2 ff ff ff ff 49 c1 e9 20 eb 14 0f 1f 84 00 00 00 00 00 48 8b 1b 48 85 db 0f 84 84 00 00 00 4c 8d 63 f8 <8b> 43 fc 48 39 6b 10 75 e7 48 83 7b 08 00 74 e0 89 c2 83 e2 feThe trace-back triggered by
arch_trigger_all_cpu_backtrace()shows the following two functions in addition on top oflink_path_walk():Quoted 3 lines
[<
ffffffff811d36c7>] ?__d_lookup_rcu+0x87/0x150[<ffffffff811c5f3b>] ?__inode_permission+0x7b/0xd0[<ffffffff811c6449>] ?link_path_walk+0x69/0x850...
Has someone seen similar problems with 3.16?
If you need more info, just ask.
Please cc: on replies as I'm not subscribed to linux-fsdevel (but LKML)
Philipp
Signature · 19 lines
Philipp Hahn Open Source Software Engineer Univention GmbH be open. Mary-Somerville-Str. 1 D-28359 Bremen Tel.: +49 421 22232-0 Fax : +49 421 22232-99 [email protected] http://www.univention.de/ Geschäftsführer: Peter H. Ganten HRB 20755 Amtsgericht Bremen Steuer-Nr.: 71-597-02876 -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Hello,
Am 02.06.2016 um 22:02 schrieb J. Bruce Fields:
Quoted 6 lines · Philipp Hahn
Quoted 5 lines
probably during heavy IO our file server crashed on a
BUG_ONindache.c, probably triggered by NFS:Quoted 2 lines
------------[ cut here ]------------ kernel BUG at /var/build/temp/tmp.BPql4ErveJ/pbuilder/linux-3.16.7-ckt25/fs/dcache.c:2373!
...
Quoted 5 lines
Quoted 1 line
Our kernel is 3.16.7-ckt25 from Debian + ckt26 and ckt27 on top.
So this is the BUG_ON(!d_unhashed(..)) at the top of
__d_rehash? I don't know the -ckt kernels, and I don't have a BUG at line 2373 ofdcache.c, or something else?
Yes, exactly that first line.
Quoted 1 line
There have been a lot of changes in the area since 3.16.7.
Too bad.
Any concrete hint or should I better spent my time convincing management to install a newer Linux kernel?
Anyway, thank you for the response.
Philipp
Quoted 9 lines · Philipp Hahn
Hello,
Am 02.06.2016 um 22:02 schrieb J. Bruce Fields:
Quoted 6 lines · Philipp Hahn
Quoted 5 lines
probably during heavy IO our file server crashed on a
BUG_ONindache.c, probably triggered by NFS:Quoted 2 lines
------------[ cut here ]------------ kernel BUG at /var/build/temp/tmp.BPql4ErveJ/pbuilder/linux-3.16.7-ckt25/fs/dcache.c:2373!
Quoted 8 lines
...
Quoted 5 lines
Quoted 1 line
Our kernel is 3.16.7-ckt25 from Debian + ckt26 and ckt27 on top.
So this is the BUG_ON(!d_unhashed(..)) at the top of
__d_rehash? I don't know the -ckt kernels, and I don't have a BUG at line 2373 ofdcache.c, or something else?Yes, exactly that first line.
Oh, from googling... it looks like that's what they're calling the Ubuntu kernels these days? It'd be best to have them triage this first.
Quoted 5 lines
Quoted 1 line
There have been a lot of changes in the area since 3.16.7.
Too bad.
Any concrete hint or should I better spent my time convincing management to install a newer Linux kernel?
I don't know if a newer kernel's better, but if you were able to reproduce this frequently then experimenting with different kernel versions might be one way to work this out.
I suppose 75a2352d0110960aeee1a28ddc09a55f97c99100 might in theory be a fix? Not a tall sure of that, though.
--b.
Quoted 4 lines
Anyway, thank you for the response.
Philipp
Quoted 6 lines · Philipp Hahn
Quoted 5 lines
Hello,
Am 02.06.2016 um 22:02 schrieb J. Bruce Fields:
Quoted 2 lines · Philipp Hahn
Quoted 1 line
probably during heavy IO our file server crashed on a
BUG_ONin
dache.c,
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
probably triggered by NFS:
Quoted 2 lines
------------[ cut here ]------------ kernel BUG at
/var/build/temp/tmp.BPql4ErveJ/pbuilder/linux-3.16.7-ckt25/fs/dcache.c:2373!
Quoted 18 lines
Quoted 8 lines
...
Quoted 5 lines
Quoted 1 line
Our kernel is 3.16.7-ckt25 from Debian + ckt26 and ckt27 on top.
So this is the BUG_ON(!d_unhashed(..)) at the top of
__d_rehash? I don't know the -ckt kernels, and I don't have a BUG at line 2373 ofdcache.c, or something else?Yes, exactly that first line.
Oh, from googling... it looks like that's what they're calling the Ubuntu kernels these days? It'd be best to have them triage this first.
Quoted 5 lines
Quoted 1 line
There have been a lot of changes in the area since 3.16.7.
Too bad.
Any concrete hint or should I better spent my time convincing management to install a newer Linux kernel?
Quoted 6 lines
I don't know if a newer kernel's better, but if you were able to reproduce this frequently then experimenting with different kernel versions might be one way to work this out.
I suppose
75a2352d0110960aeee1a28ddc09a55f97c99100might in theory be a fix? Not a tall sure of that, though.
I've been seeing this bug regularly as well and raised a Debian bug report[1]
- is it likely that changing the underlying filesystem may avoid the bug? The system where I see the bug has both ext4 on LVM and some btrfs
- are there any other possible workarounds, such as disabling SMP or changing filesystem mount options on the server or the NFS clients?
- Debian has kernel 4.7.8-1~bpo8+1 in the backports for stable/jessie users[1], is it confirmed that kernel fixes the problem?
- if somebody updates their NFS server's kernel from 3.16 to 4.7.8-1~bpo8+1, will the NFS userland stuff be compatible? It looks like the version[2] in stable is 1.2.8-9 while testing and unstable have 1.2.8-9.2. The difference between the two is summarized in the changelog[3], are any of these recent fixes serious enough that there should be a stable-update or backport of 1.2.8-9.2?
Regards,
Daniel
1. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=847549
2. https://packages.qa.debian.org/n/nfs-utils.html
3.
http://metadata.ftp-master.debian.org/changelogs/main/n/nfs-utils/nfs-utils_1.2.8-9.2_changelog