XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length), file: fs/xfs/xfs_trans_buf.c, line: 568
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
crc=1
fsstress -z -fsymlink=1 -n99999999 -p4 -d /mnt/test
XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length),
file: fs/xfs/xfs_trans_buf.c, line: 568
------------[ cut here ]------------ kernel BUG at fs/xfs/xfs_message.c:108! invalid opcode: 0000 [#1] SMP Modules linked in: xfs libcrc32c fuse ebtable_nat
nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle bnep nf_conntrack_ipv4 nf_defrag_ipv4 bluetooth xt_conntrack nf_conntrack rfkill ebtable_filter ebtables ip6table_filter ip6_tables snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm snd_page_alloc snd_timer snd joydev soundcore i2c_piix4 pcspkr mperf virtio_balloon floppy uinput qxl drm_kms_helper ttm drm virtio_blk virtio_net i2c_core
CPU: 0 PID: 1419 Comm: fsstress Not tainted 3.11.0-rc1+ #10 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800d65b5dc0 ti: ffff8800d10ba000 task.ti: ffff8800d10ba000 RIP: 0010:[<ffffffffa02b8812>] [<ffffffffa02b8812>] assfail+0x22/0x30 [xfs] RSP: 0018:ffff8800d10bb998 EFLAGS: 00010292 RAX: 000000000000006b RBX: ffff8800d67be3a0 RCX: 0000000000000000 RDX: ffff88011fc0ee48 RSI: ffff88011fc0d038 RDI: ffff88011fc0d038 RBP: ffff8800d10bb998 R08: 0000000000000000 R09: 000000000000020a R10: ffffffff81858260 R11: 0000000000000209 R12: ffff8800d165d500 R13: ffff8800d1158980 R14: 0000000000001007 R15: ffff8800d1cb8300 FS: 00007f1efd2ce740(0000) GS:ffff88011fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f1ef80fb018 CR3: 0000000036edb000 CR4: 00000000000006f0 Stack: ffff8800d10bb9e8 ffffffffa031d549 000000fc24a6f000 00000e20000000d3 ffff8800d10bb9f8 ffff8800d67c3040 ffff8800d1cb8208 ffff8800d1cb81e8 ffff8800d67c3000 ffff8800d1cb8300 ffff8800d10bba48 ffffffffa02e7c1c Call Trace: [<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs] [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs] [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs] [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs] [<ffffffffa02ec965>] xfs_dir_createname+0x1d5/0x1e0 [xfs] [<ffffffffa02c1607>] ? kmem_alloc+0x67/0xf0 [xfs] [<ffffffffa02becb9>] xfs_symlink+0x619/0xa20 [xfs] [<ffffffff811abad6>] ? _d_rehash+0x36/0x40 [<ffffffff8119f498>] ? __lookup_hash+0x38/0x50 [<ffffffff8119f4c9>] ? lookup_hash+0x19/0x20 [<ffffffff811a21ee>] ? kern_path_create+0x8e/0x170 [<ffffffffa02b5e5c>] xfs_vn_symlink+0x5c/0xe0 [xfs] [<ffffffff811a3939>] vfs_symlink+0x99/0x100 [<ffffffff811a59d6>] SyS_symlinkat+0x66/0xd0 [<ffffffff811a5a56>] SyS_symlink+0x16/0x20 [<ffffffff81645442>] system_call_fastpath+0x16/0x1b Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 f1 41 89 d0 48
c7 c6 70 50 33 a0 48 89 fa 31 c0 48 89 e5 31 ff e8 de fb ff ff <0f> 0b 66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 RIP [<ffffffffa02b8812>] assfail+0x22/0x30 [xfs]
RSP <ffff8800d10bb998>
---[ end trace 9578edaae955ff56 ]---
I repeated the test on a crc=0 fs (with -isize=512) and could not reproduce during fsstress. I let it populate to about 10GB and ultimately hit the same assert on unlink during a post-test cleanup:
crc=0
rm -rf /mnt/test
XFS: Assertion failed: first <= last && last < BBTOB(bp->b_length),
file: fs/xfs/xfs_trans_buf.c, line: 568
------------[ cut here ]------------ kernel BUG at fs/xfs/xfs_message.c:108! invalid opcode: 0000 [#1] SMP Modules linked in: xfs libcrc32c fuse ebtable_nat
nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_nat nf_nat_ipv6 ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables bnep bluetooth rfkill ip6table_filter ip6_tables snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm snd_page_alloc snd_timer snd soundcore joydev pcspkr virtio_balloon i2c_piix4 floppy mperf uinput qxl drm_kms_helper ttm drm virtio_net virtio_blk i2c_core
CPU: 1 PID: 2198 Comm: rm Not tainted 3.11.0-rc1+ #10 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8801161ec650 ti: ffff8800c803e000 task.ti: ffff8800c803e000 RIP: 0010:[<ffffffffa02c6812>] [<ffffffffa02c6812>] assfail+0x22/0x30 [xfs] RSP: 0018:ffff8800c803fa98 EFLAGS: 00010292 RAX: 000000000000006b RBX: ffff8801029a6e80 RCX: 0000000000000000 RDX: ffff88011fc8ee48 RSI: ffff88011fc8d038 RDI: ffff88011fc8d038 RBP: ffff8800c803fa98 R08: 0000000000000000 R09: 0000000000000209 R10: ffffffff81858260 R11: 0000000000000208 R12: ffff8800302bd200 R13: ffff8800d25e0850 R14: 000000000000122f R15: ffff8800d271f010 FS: 00007f28ef9bf740(0000) GS:ffff88011fc80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 000000000153a000 CR3: 00000000b1fd3000 CR4: 00000000000006e0 Stack: ffff8800c803fae8 ffffffffa032b549 00800201008006cc 000000100185febe ffffffffa033fcb0 ffff8800ade0c010 ffff8800ade0c000 ffff8800d3c2b9e0 ffff8800d25e0850 ffff8800d271f010 ffff8800c803fb58 ffffffffa02f61ff Call Trace: [<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs] [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs] [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs] [<ffffffffa02e16ce>] ? xfs_bmap_last_extent+0x6e/0xb0 [xfs] [<ffffffffa02fa5b5>] xfs_dir_removename+0x195/0x1a0 [xfs] [<ffffffffa0310b69>] xfs_remove+0x2a9/0x410 [xfs] [<ffffffffa02c3ca2>] xfs_vn_unlink+0x52/0xa0 [xfs] [<ffffffff811a260e>] vfs_unlink+0x9e/0x110 [<ffffffff811a2821>] do_unlinkat+0x1a1/0x230 [<ffffffff811a592b>] SyS_unlinkat+0x1b/0x40 [<ffffffff81645442>] system_call_fastpath+0x16/0x1b Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 f1 41 89 d0 48
c7 c6 70 30 34 a0 48 89 fa 31 c0 48 89 e5 31 ff e8 de fb ff ff <0f> 0b 66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 RIP [<ffffffffa02c6812>] assfail+0x22/0x30 [xfs]
RSP <ffff8800c803fa98>
---[ end trace 3ef54f36db3ba0c5 ]---
Info on the crc=0 fs is as follows:
meta-data=/dev/vdb isize=512 agcount=4, agsize=6553600 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0
data = bsize=4096 blocks=26214400, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal bsize=4096 blocks=12800, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0Brian
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
Eeek.
So both crashes are directory related. What is the top XFS kernel commit for these tests?
Have you seen this on earlier versions of the kernel?
Thanks,
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 8 lines · Mark Tinguely
Quoted 7 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
Quoted 8 lines
Quoted 2 lines
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
Eeek.
So both crashes are directory related. What is the top XFS kernel commit for these tests?
Hi Mark,
It was the latest from at some point yesterday. I don't think I've pulled since, so I'm at:
3e3c51ce xfs: add xfs sb v4 support for dirent filetype field
Quoted 2 lines
Have you seen this on earlier versions of the kernel?
Well I hit an issue first on my dev. branch for finobt hacking, which is currently based on a slightly older commit:
2c2bcc07 xfs: call roundup_64() to calculate the min_logblks
Unfortunately I don't have the precise error/stack from that failure on hand to double check whether it's the exact same failure. I can try to regenerate it a bit later today.
Brian
Quoted 3 lines
Thanks,
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 31 lines · Brian Foster
Quoted 8 lines · Mark Tinguely
Quoted 7 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
Quoted 8 lines
Quoted 2 lines
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
Eeek.
So both crashes are directory related. What is the top XFS kernel commit for these tests?
Hi Mark,
It was the latest from at some point yesterday. I don't think I've pulled since, so I'm at:
3e3c51cexfs: add xfs sb v4 support for dirent filetype fieldQuoted 2 lines
Have you seen this on earlier versions of the kernel?
Well I hit an issue first on my dev. branch for finobt hacking, which is currently based on a slightly older commit:
Quoted 12 lines
2c2bcc07xfs: callroundup_64()to calculate themin_logblksUnfortunately I don't have the precise error/stack from that failure on hand to double check whether it's the exact same failure. I can try to regenerate it a bit later today.
Brian
Quoted 3 lines
Thanks,
--Mark.
Good, I just want to make sure the common directory mods for the file type patch was not the cause. I will try to recreate it too.
Thanks.
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 39 lines · Mark Tinguely
Quoted 38 lines · Brian Foster
Quoted 9 lines · Mark Tinguely
Quoted 8 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on
a CRC enabled filesystem (with otherwise default mkfs options):Quoted 8 lines
Quoted 2 lines
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
Eeek.
So both crashes are directory related. What is the top XFS kernel commit for these tests?
Hi Mark,
It was the latest from at some point yesterday. I don't think I've pulled since, so I'm at:
3e3c51cexfs: add xfs sb v4 support for dirent filetype fieldQuoted 2 lines
Have you seen this on earlier versions of the kernel?
Well I hit an issue first on my dev. branch for finobt hacking, which is currently based on a slightly older commit:
2c2bcc07xfs: callroundup_64()to calculate themin_logblksUnfortunately I don't have the precise error/stack from that failure on hand to double check whether it's the exact same failure. I can try to regenerate it a bit later today.
Quoted 9 lines
Quoted 6 lines
Brian
Quoted 3 lines
Thanks,
--Mark.
Good, I just want to make sure the common directory mods for the file type patch was not the cause. I will try to recreate it too.
And just to confirm, I can hit the assert from both tests on a 2c2bcc07 kernel (pre-dirent ftype change).
Brian
Quoted 9 lines
Thanks.
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 46 lines · Brian Foster
Quoted 45 lines · Mark Tinguely
Quoted 44 lines · Brian Foster
Quoted 9 lines · Mark Tinguely
Quoted 8 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on
a CRC enabled filesystem (with otherwise default mkfs options):Quoted 8 lines
Quoted 2 lines
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
Eeek.
So both crashes are directory related. What is the top XFS kernel commit for these tests?
Hi Mark,
It was the latest from at some point yesterday. I don't think I've pulled since, so I'm at:
3e3c51cexfs: add xfs sb v4 support for dirent filetype fieldQuoted 2 lines
Have you seen this on earlier versions of the kernel?
Well I hit an issue first on my dev. branch for finobt hacking, which is currently based on a slightly older commit:
2c2bcc07xfs: callroundup_64()to calculate themin_logblksUnfortunately I don't have the precise error/stack from that failure on hand to double check whether it's the exact same failure. I can try to regenerate it a bit later today.
Brian
Quoted 3 lines
Thanks,
--Mark.
Quoted 8 lines
Quoted 3 lines
Good, I just want to make sure the common directory mods for the file type patch was not the cause. I will try to recreate it too.
And just to confirm, I can hit the assert from both tests on a
2c2bcc07kernel (pre-dirent ftype change).Brian
Suspected as much, but thank-you for the confirmation.
I guess the next bi-section would be Linux 3.11 to make sure the user/kernel sync did not introduce anything. I thought those moves were clean.
I will shut up and try to help bisect.
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 7 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
Quoted 7 lines
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
crc=1
fsstress -z -fsymlink=1 -n99999999 -p4 -d/mnt/testXFS: Assertion failed: first <= last && last <
BBTOB(bp->b_length),
Directory buffer overrun.
Quoted 5 lines
[<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs] [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs] [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs] [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]
During a split.
Easily reproduced with "seq 200000 | xargs touch" as Michael Semon reported last week.
The fix demonstrates my concerns about modifying directory code - the CRC changes missed a *fundamental* directory format definition, and we've only just tripped over it....
Quoted 3 lines
rm -rf
/mnt/testXFS: Assertion failed: first <= last && last <
BBTOB(bp->b_length),
Directory buffer overrun.
Quoted 4 lines
[<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs] [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs] [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs]
During a merge. Not sure why that is happening on a v4 filesystem. V5 filesystem, yes, due to the above bug but v4 should not be affected.
Cheers,
Dave.
Signature · 16 lines
Dave Chinner [email protected] xfs: fix calculation of the number of node entries in a dir3 node From: Dave Chinner <[email protected]> The calculation doesn't take into account the size of the dir v3 header, so overestimates the hash entries in a node. This causes directory buffer overruns when splitting and merging nodes. Signed-off-by: Dave Chinner <[email protected]> --- fs/xfs/xfs_da_btree.h | 11 +++++++++-- fs/xfs/xfs_dir2.c | 16 ++++++++++------ 2 files changed, 19 insertions(+), 8 deletions(-)
fs/xfs/xfs_da_btree.h +9 -2
@@ -133,12 +133,19 @@ extern void xfs_da3_node_hdr_to_disk(struct xfs_da_intnode *to, struct xfs_da3_icnode_hdr *from);static inline int -xfs_da3_node_hdr_size (struct xfs_da_intnode *dap) +__xfs_da3_node_hdr_size (bool v3) { -if (dap->hdr.info.magic ==cpu_to_be16 (XFS_DA3_NODE_MAGIC )) +if (v3)return sizeof (struct xfs_da3_node_hdr );return sizeof (struct xfs_da_node_hdr ); } +static inline int +xfs_da3_node_hdr_size (struct xfs_da_intnode *dap) +{ +bool v3 = dap->hdr.info.magic ==cpu_to_be16 (XFS_DA3_NODE_MAGIC ); + +return __xfs_da3_node_hdr_size (v3); +}static inline struct xfs_da_node_entry *xfs_da3_node_tree_p (struct xfs_da_intnode *dap)
fs/xfs/xfs_dir2.c +10 -6
@@ -90,6 +90,9 @@ voidxfs_dir_mount ( xfs_mount_t *mp) { +int nodehdr_size; + +ASSERT (xfs_sb_version_hasdirv2 (&mp->m_sb));ASSERT ((1 << (mp->m_sb.sb_blocklog + mp->m_sb.sb_dirblklog)) <=XFS_MAX_BLOCKSIZE );
@@ -98,12 +101,13 @@ xfs_dir_mount( mp->m_dirdatablk =xfs_dir2_db_to_da (mp,XFS_DIR2_DATA_FIRSTDB (mp)); mp->m_dirleafblk =xfs_dir2_db_to_da (mp,XFS_DIR2_LEAF_FIRSTDB (mp)); mp->m_dirfreeblk =xfs_dir2_db_to_da (mp,XFS_DIR2_FREE_FIRSTDB (mp)); - mp->m_attr_node_ents = - (mp->m_sb.sb_blocksize - (uint )sizeof (xfs_da_node_hdr_t)) / - (uint )sizeof (xfs_da_node_entry_t); - mp->m_dir_node_ents = - (mp->m_dirblksize - (uint )sizeof (xfs_da_node_hdr_t)) / - (uint )sizeof (xfs_da_node_entry_t); + + nodehdr_size =__xfs_da3_node_hdr_size (xfs_sb_version_hascrc (&mp->m_sb)); + mp->m_attr_node_ents = (mp->m_sb.sb_blocksize - nodehdr_size) / + (uint )sizeof (xfs_da_node_entry_t); + mp->m_dir_node_ents = (mp->m_dirblksize - nodehdr_size) / + (uint )sizeof (xfs_da_node_entry_t); + mp->m_dir_magicpct = (mp->m_dirblksize *37 ) /100 ;if (xfs_sb_version_hasasciici (&mp->m_sb)) mp->m_dirnameops = &xfs_ascii_ci_nameops;
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Brian Foster
Tested-byQuoted 39 lines · Dave Chinner
Quoted 7 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
Quoted 7 lines
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
crc=1
fsstress -z -fsymlink=1 -n99999999 -p4 -d/mnt/testXFS: Assertion failed: first <= last && last <
BBTOB(bp->b_length),Directory buffer overrun.
Quoted 5 lines
[<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs] [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs] [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs] [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]During a split.
Easily reproduced with "seq 200000 | xargs touch" as Michael Semon reported last week.
The fix demonstrates my concerns about modifying directory code - the CRC changes missed a *fundamental* directory format definition, and we've only just tripped over it....
Quoted 3 lines
rm -rf
/mnt/testXFS: Assertion failed: first <= last && last <
BBTOB(bp->b_length),Directory buffer overrun.
Quoted 9 lines
Quoted 4 lines
[<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs] [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs] [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs]During a merge. Not sure why that is happening on a v4 filesystem. V5 filesystem, yes, due to the above bug but v4 should not be affected.
Interesting, thanks Dave. FWIW, I no longer reproduce the assert in either scenario with this patch applied. I also don't see how it would make a difference for a v4 superblock filesystem. Perhaps that particular test was bogus. I haven't heard if Mark happened to reproduce that one. Regardless, consider it:
(xfs: fix calculation of the number of node entries in a dir3 node)
Brian
Quoted 4 lines
Cheers,
Dave.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 25 lines · Brian Foster
Quoted 24 lines · Dave Chinner
Quoted 7 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
Quoted 7 lines
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
crc=1
fsstress -z -fsymlink=1 -n99999999 -p4 -d/mnt/testXFS: Assertion failed: first<= last&& last<
BBTOB(bp->b_length),Directory buffer overrun.
Quoted 5 lines
[<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs] [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs] [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs] [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]
Quoted 8 lines
Quoted 8 lines
During a split.
Easily reproduced with "seq 200000 | xargs touch" as Michael Semon reported last week.
The fix demonstrates my concerns about modifying directory code - the CRC changes missed a *fundamental* directory format definition, and we've only just tripped over it....
I agree. As we see here, bugs in common directory code effect all filesystems. It may not matter if the feature the code was written for is enabled or not.
Quoted 22 lines
Quoted 6 lines
Quoted 3 lines
rm -rf
/mnt/testXFS: Assertion failed: first<= last&& last<
BBTOB(bp->b_length),Directory buffer overrun.
Quoted 9 lines
Quoted 4 lines
[<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs] [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs] [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs]During a merge. Not sure why that is happening on a v4 filesystem. V5 filesystem, yes, due to the above bug but v4 should not be affected.
Interesting, thanks Dave. FWIW, I no longer reproduce the assert in either scenario with this patch applied. I also don't see how it would make a difference for a v4 superblock filesystem. Perhaps that particular test was bogus. I haven't heard if Mark happened to reproduce that one. Regardless, consider it:
Quoted 3 lines
Tested-by Brian Foster<[email protected]>(xfs: fix calculation of the number of node entries in a dir3 node)
I got the XFS v4 to assert on the remove in Linux 3.10 and 3.11.
With the patch, a shorter test on Linux 3.10 did not assert. I will do the full test on Linux 3.10/3.11, review and report back.
Quoted 6 lines
Brian
Quoted 3 lines
Cheers,
Dave.
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 31 lines · Mark Tinguely
Quoted 30 lines · Brian Foster
Quoted 29 lines · Dave Chinner
Quoted 7 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
Quoted 7 lines
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
crc=1
fsstress -z -fsymlink=1 -n99999999 -p4 -d/mnt/testXFS: Assertion failed: first<= last&& last<
BBTOB(bp->b_length),Directory buffer overrun.
Quoted 5 lines
[<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs] [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs] [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs] [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]During a split.
Easily reproduced with "seq 200000 | xargs touch" as Michael Semon reported last week.
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
The fix demonstrates my concerns about modifying directory code - the CRC changes missed a *fundamental* directory format definition, and we've only just tripped over it....
I agree. As we see here, bugs in common directory code effect all filesystems. It may not matter if the feature the code was written for is enabled or not.
Well, this is *only* a v5 bug. The fact is, the only difference the change I made makes to v4 filesystems is that it removed the typedef from the sizeof calculation. On my test systems, the value mp->m_dir_node_ents is identical for v4 filesystems with or without the patch applied.....
Quoted 11 lines
Quoted 11 lines
Quoted 4 lines
During a merge. Not sure why that is happening on a v4 filesystem. V5 filesystem, yes, due to the above bug but v4 should not be affected.
Interesting, thanks Dave. FWIW, I no longer reproduce the assert in either scenario with this patch applied. I also don't see how it would make a difference for a v4 superblock filesystem. Perhaps that particular test was bogus. I haven't heard if Mark happened to reproduce that one. Regardless, consider it:
Quoted 5 lines
Quoted 3 lines
Tested-by Brian Foster<[email protected]>(xfs: fix calculation of the number of node entries in a dir3 node)
I got the XFS v4 to assert on the remove in Linux 3.10 and 3.11.
Did you test 3.9 - before the crc changes were made to the filesystem? i.e. if an invalid mp->m_dir_node_ents value is the real cause of the v4 filesystem problem, then it should reproduce on just about every kernel we chose to test.
Quoted 2 lines
With the patch, a shorter test on Linux 3.10 did not assert. I will do the full test on Linux 3.10/3.11, review and report back.
Because nobody can explain why this patch would fix a problem on a v4 filesystem, we need more triage of the v4 problem needs to be done. I haven't been able to reproduce the unlink issue (and don't have time to do everything), so could you triage the problem further, Mark? We really need to understand the root cause of the problem on v4 filesystems so we can determine what the impact of it is...
Cheers,
Dave.
Quoted 10 lines
Quoted 7 lines
Brian
Quoted 3 lines
Cheers,
Dave.
--Mark.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 78 lines · Dave Chinner
Quoted 31 lines · Mark Tinguely
Quoted 30 lines · Brian Foster
Quoted 29 lines · Dave Chinner
Quoted 7 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
Quoted 7 lines
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
crc=1
fsstress -z -fsymlink=1 -n99999999 -p4 -d/mnt/testXFS: Assertion failed: first<= last&& last<
BBTOB(bp->b_length),Directory buffer overrun.
Quoted 5 lines
[<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs] [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs] [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs] [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]During a split.
Easily reproduced with "seq 200000 | xargs touch" as Michael Semon reported last week.
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
The fix demonstrates my concerns about modifying directory code - the CRC changes missed a *fundamental* directory format definition, and we've only just tripped over it....
I agree. As we see here, bugs in common directory code effect all filesystems. It may not matter if the feature the code was written for is enabled or not.
Well, this is *only* a v5 bug. The fact is, the only difference the change I made makes to v4 filesystems is that it removed the typedef from the sizeof calculation. On my test systems, the value
mp->m_dir_node_entsis identical for v4 filesystems with or without the patch applied.....Quoted 11 lines
Quoted 11 lines
Quoted 4 lines
During a merge. Not sure why that is happening on a v4 filesystem. V5 filesystem, yes, due to the above bug but v4 should not be affected.
Interesting, thanks Dave. FWIW, I no longer reproduce the assert in either scenario with this patch applied. I also don't see how it would make a difference for a v4 superblock filesystem. Perhaps that particular test was bogus. I haven't heard if Mark happened to reproduce that one. Regardless, consider it:
Quoted 5 lines
Quoted 3 lines
Tested-by Brian Foster<[email protected]>(xfs: fix calculation of the number of node entries in a dir3 node)
I got the XFS v4 to assert on the remove in Linux 3.10 and 3.11.
Did you test 3.9 - before the crc changes were made to the filesystem? i.e. if an invalid
mp->m_dir_node_entsvalue is the real cause of the v4 filesystem problem, then it should reproduce on just about every kernel we chose to test.Quoted 2 lines
With the patch, a shorter test on Linux 3.10 did not assert. I will do the full test on Linux 3.10/3.11, review and report back.
Because nobody can explain why this patch would fix a problem on a v4 filesystem, we need more triage of the v4 problem needs to be done. I haven't been able to reproduce the unlink issue (and don't have time to do everything), so could you triage the problem further, Mark? We really need to understand the root cause of the problem on v4 filesystems so we can determine what the impact of it is...
Quoted 3 lines
Cheers,
Dave.
A full test still asserts on the remove with the patched Linux 3.10 - I am about 50% into the retest of Linux 3.10 and then I was planning to move back to Linux 3.9.
kdump did not work, so I have no vmcore and therefore no productive information.
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 23 lines · Mark Tinguely
Quoted 22 lines · Dave Chinner
Quoted 21 lines · Mark Tinguely
Quoted 20 lines · Brian Foster
Quoted 19 lines · Dave Chinner
Quoted 17 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and
eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I
hit it through a couple different paths, first while running fsstress on
a CRC enabled filesystem (with otherwise default mkfs options):(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
crc=1
fsstress -z -fsymlink=1 -n99999999 -p4 -d/mnt/testXFS: Assertion failed: first<= last&& last<
BBTOB(bp->b_length),
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Directory buffer overrun.
A full test still asserts on the remove with the patched Linux 3.10 - I am about 50% into the retest of Linux 3.10 and then I was planning to move back to Linux 3.9.
kdump did not work, so I have no vmcore and therefore no productive information.
Confirmed the Linux 3.10 asserts. Linux 3.9 does not assert.
I will fix the kdump and try to catch a Linux 3.10 assert.
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 24 lines · Dave Chinner
Quoted 7 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
Quoted 7 lines
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
crc=1
fsstress -z -fsymlink=1 -n99999999 -p4 -d/mnt/testXFS: Assertion failed: first <= last && last <
BBTOB(bp->b_length),Directory buffer overrun.
Quoted 5 lines
[<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs] [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs] [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs] [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs]
Quoted 8 lines
During a split.
Easily reproduced with "seq 200000 | xargs touch" as Michael Semon reported last week.
The fix demonstrates my concerns about modifying directory code - the CRC changes missed a *fundamental* directory format definition, and we've only just tripped over it....
Don't fret too much over it. This test was part of coreutils, which is something that I rebuild after a glibc upgrade. Had glibc-2.18 been released six weeks ago, then I would have stumbled onto this XFS issue six weeks ago.
Quoted 11 lines
Quoted 3 lines
rm -rf
/mnt/testXFS: Assertion failed: first <= last && last <
BBTOB(bp->b_length),Directory buffer overrun.
Quoted 4 lines
[<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs] [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs] [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs]
Quoted 7 lines
During a merge. Not sure why that is happening on a v4 filesystem. V5 filesystem, yes, due to the above bug but v4 should not be affected.
Cheers,
Dave.
Your patch looks good, and I even applied it to vanilla 3.10.9, along with Jeff Liu's MAX_LFS_FILESIZE patch. [Murphy's Law states that if I didn't use Jeff's patch, then I would run xfstests generic/308 on accident, leading to a hung umount. Happens every single time.] Both patches applied cleanly to kernels on a 2.8 GHz i686 Pentium 4 PC that was running Slackware 14.0 Linux.
Naturally, seq 200000 | xargs touch was run for v5 and v4 XFS file systems. All was okay. The removal of the populated directory went fine as well.
The v5 file systems were tested using a 3.11-rc7+ git kernel. xfstests was run from the start of generic/ through generic/127; and that went fine. Some of the xfs/* series was run but merely scanned because the v5-output-cleanup patches were not readily available.
The v4 file systems were tested with a patched vanilla 3.10.9 kernel, and some of generic was run, with patched and unpatched kernels showing the same good results, very little difference in timing overall.
Thanks!
Michael
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Mark Tinguely
Reviewed-byQuoted 10 lines · Dave Chinner
xfs: fix calculation of the number of node entries in a dir3 node
From: Dave Chinner<[email protected]>
The calculation doesn't take into account the size of the dir v3 header, so overestimates the hash entries in a node. This causes directory buffer overruns when splitting and merging nodes.
Signed-off-by Dave Chinner<[email protected]>---
Makes sense to me for the inode v3 assert.
The remove assert with the same test must be in common code; it can be triggered in both versions of the inode (Linux 3.10 - TOT). The assert on the remove may require the test to be run until the filesystem is full before doing the remove.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Ben Myers
AppliedQuoted 12 lines · Mark Tinguely
Quoted 10 lines · Dave Chinner
xfs: fix calculation of the number of node entries in a dir3 node
From: Dave Chinner<[email protected]>
The calculation doesn't take into account the size of the dir v3 header, so overestimates the hash entries in a node. This causes directory buffer overruns when splitting and merging nodes.
Signed-off-by Dave Chinner<[email protected]>---
Quoted 8 lines
Makes sense to me for the inode v3 assert.
Reviewed-by Mark Tinguely <[email protected]>The remove assert with the same test must be in common code; it can be triggered in both versions of the inode (Linux 3.10 - TOT). The assert on the remove may require the test to be run until the filesystem is full before doing the remove.
Applied.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 135 lines · Brian Foster
Hi all,
I hit an assert on a debug kernel while beating on some finobt work and eventually reproduced it on unmodified/TOT xfs/xfsprogs as of today. I hit it through a couple different paths, first while running fsstress on a CRC enabled filesystem (with otherwise default mkfs options):
(These tests are running on a 4p, 4GB VM against a 100GB virtio disk, hosted on a single spindle desktop box).
crc=1
fsstress -z -fsymlink=1 -n99999999 -p4 -d/mnt/testXFS: Assertion failed: first<= last&& last<
BBTOB(bp->b_length),
file:fs/xfs/xfs_trans_buf.c, line: 568------------[ cut here ]------------ kernel BUG at fs/xfs/xfs_message.c:108! invalid opcode: 0000 [#1] SMP Modules linked in: xfs libcrc32c fuse ebtable_nat
nf_conntrack_netbios_nsnf_conntrack_broadcastipt_MASQUERADEip6table_natnf_nat_ipv6ip6table_mangleip6t_REJECTnf_conntrack_ipv6nf_defrag_ipv6iptable_natnf_nat_ipv4nf_natiptable_manglebnepnf_conntrack_ipv4nf_defrag_ipv4bluetoothxt_conntracknf_conntrackrfkillebtable_filterebtablesip6table_filterip6_tablessnd_hda_intelsnd_hda_codecsnd_hwdepsnd_seqsnd_seq_devicesnd_pcmsnd_page_allocsnd_timersnd joydev soundcorei2c_piix4pcspkr mperfvirtio_balloonfloppy uinput qxldrm_kms_helperttm drmvirtio_blkvirtio_neti2c_coreCPU: 0 PID: 1419 Comm: fsstress Not tainted 3.11.0-rc1+ #10 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800d65b5dc0 ti: ffff8800d10ba000 task.ti: ffff8800d10ba000 RIP: 0010:[<ffffffffa02b8812>] [<ffffffffa02b8812>] assfail+0x22/0x30 [xfs] RSP: 0018:ffff8800d10bb998 EFLAGS: 00010292 RAX: 000000000000006b RBX: ffff8800d67be3a0 RCX: 0000000000000000 RDX: ffff88011fc0ee48 RSI: ffff88011fc0d038 RDI: ffff88011fc0d038 RBP: ffff8800d10bb998 R08: 0000000000000000 R09: 000000000000020a R10: ffffffff81858260 R11: 0000000000000209 R12: ffff8800d165d500 R13: ffff8800d1158980 R14: 0000000000001007 R15: ffff8800d1cb8300 FS: 00007f1efd2ce740(0000) GS:ffff88011fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f1ef80fb018 CR3: 0000000036edb000 CR4: 00000000000006f0 Stack: ffff8800d10bb9e8 ffffffffa031d549 000000fc24a6f000 00000e20000000d3 ffff8800d10bb9f8 ffff8800d67c3040 ffff8800d1cb8208 ffff8800d1cb81e8 ffff8800d67c3000 ffff8800d1cb8300 ffff8800d10bba48 ffffffffa02e7c1c Call Trace: [<ffffffffa031d549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02e7c1c>] xfs_da3_node_add+0x11c/0x210 [xfs] [<ffffffffa02ea703>] xfs_da3_node_split+0xc3/0x230 [xfs] [<ffffffffa02eaa18>] xfs_da3_split+0x1a8/0x410 [xfs] [<ffffffffa02f743f>] xfs_dir2_node_addname+0x47f/0xde0 [xfs] [<ffffffffa02ec965>] xfs_dir_createname+0x1d5/0x1e0 [xfs] [<ffffffffa02c1607>] ? kmem_alloc+0x67/0xf0 [xfs] [<ffffffffa02becb9>] xfs_symlink+0x619/0xa20 [xfs] [<ffffffff811abad6>] ? _d_rehash+0x36/0x40 [<ffffffff8119f498>] ? __lookup_hash+0x38/0x50 [<ffffffff8119f4c9>] ? lookup_hash+0x19/0x20 [<ffffffff811a21ee>] ? kern_path_create+0x8e/0x170 [<ffffffffa02b5e5c>] xfs_vn_symlink+0x5c/0xe0 [xfs] [<ffffffff811a3939>] vfs_symlink+0x99/0x100 [<ffffffff811a59d6>] SyS_symlinkat+0x66/0xd0 [<ffffffff811a5a56>] SyS_symlink+0x16/0x20 [<ffffffff81645442>] system_call_fastpath+0x16/0x1b Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 f1 41 89 d0 48c7 c6 70 50 33 a0 48 89 fa 31 c0 48 89 e5 31 ff e8 de fb ff ff<0f> 0b 66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 RIP [<
ffffffffa02b8812>]assfail+0x22/0x30[xfs]
RSP<ffff8800d10bb998>---[ end trace 9578edaae955ff56 ]---I repeated the test on a crc=0 fs (with -isize=512) and could not reproduce during fsstress. I let it populate to about 10GB and ultimately hit the same assert on unlink during a post-test cleanup:
crc=0
rm -rf/mnt/testXFS: Assertion failed: first<= last&& last<
BBTOB(bp->b_length),
file:fs/xfs/xfs_trans_buf.c, line: 568------------[ cut here ]------------ kernel BUG at fs/xfs/xfs_message.c:108! invalid opcode: 0000 [#1] SMP Modules linked in: xfs libcrc32c fuse ebtable_nat
nf_conntrack_netbios_nsnf_conntrack_broadcastipt_MASQUERADEip6table_natnf_nat_ipv6ip6table_mangleip6t_REJECTnf_conntrack_ipv6nf_defrag_ipv6iptable_natnf_nat_ipv4nf_natiptable_manglenf_conntrack_ipv4nf_defrag_ipv4xt_conntracknf_conntrackebtable_filterebtables bnep bluetooth rfkillip6table_filterip6_tablessnd_hda_intelsnd_hda_codecsnd_hwdepsnd_seqsnd_seq_devicesnd_pcmsnd_page_allocsnd_timersnd soundcore joydev pcspkrvirtio_ballooni2c_piix4floppy mperf uinput qxldrm_kms_helperttm drmvirtio_netvirtio_blki2c_coreCPU: 1 PID: 2198 Comm: rm Not tainted 3.11.0-rc1+ #10 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8801161ec650 ti: ffff8800c803e000 task.ti: ffff8800c803e000 RIP: 0010:[<ffffffffa02c6812>] [<ffffffffa02c6812>] assfail+0x22/0x30 [xfs] RSP: 0018:ffff8800c803fa98 EFLAGS: 00010292 RAX: 000000000000006b RBX: ffff8801029a6e80 RCX: 0000000000000000 RDX: ffff88011fc8ee48 RSI: ffff88011fc8d038 RDI: ffff88011fc8d038 RBP: ffff8800c803fa98 R08: 0000000000000000 R09: 0000000000000209 R10: ffffffff81858260 R11: 0000000000000208 R12: ffff8800302bd200 R13: ffff8800d25e0850 R14: 000000000000122f R15: ffff8800d271f010 FS: 00007f28ef9bf740(0000) GS:ffff88011fc80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 000000000153a000 CR3: 00000000b1fd3000 CR4: 00000000000006e0 Stack: ffff8800c803fae8 ffffffffa032b549 00800201008006cc 000000100185febe ffffffffa033fcb0 ffff8800ade0c010 ffff8800ade0c000 ffff8800d3c2b9e0 ffff8800d25e0850 ffff8800d271f010 ffff8800c803fb58 ffffffffa02f61ff Call Trace: [<ffffffffa032b549>] xfs_trans_log_buf+0x89/0x1b0 [xfs] [<ffffffffa02f61ff>] xfs_da3_node_unbalance+0xef/0x1d0 [xfs] [<ffffffffa02f98b0>] xfs_da3_join+0x240/0x290 [xfs] [<ffffffffa030659b>] xfs_dir2_node_removename+0x69b/0x8b0 [xfs] [<ffffffffa02e16ce>] ? xfs_bmap_last_extent+0x6e/0xb0 [xfs] [<ffffffffa02fa5b5>] xfs_dir_removename+0x195/0x1a0 [xfs] [<ffffffffa0310b69>] xfs_remove+0x2a9/0x410 [xfs] [<ffffffffa02c3ca2>] xfs_vn_unlink+0x52/0xa0 [xfs] [<ffffffff811a260e>] vfs_unlink+0x9e/0x110 [<ffffffff811a2821>] do_unlinkat+0x1a1/0x230 [<ffffffff811a592b>] SyS_unlinkat+0x1b/0x40 [<ffffffff81645442>] system_call_fastpath+0x16/0x1b Code: 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 f1 41 89 d0 48c7 c6 70 30 34 a0 48 89 fa 31 c0 48 89 e5 31 ff e8 de fb ff ff<0f> 0b 66 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 RIP [<
ffffffffa02c6812>]assfail+0x22/0x30[xfs]
RSP<ffff8800c803fa98>---[ end trace 3ef54f36db3ba0c5 ]---Info on the crc=0 fs is as follows:
meta-data=/dev/vdb isize=512 agcount=4, agsize=6553600 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0
data = bsize=4096 blocks=26214400, imaxpct=25
= sunit=0 swidth=0 blks
Quoted 7 lines
naming =version 2 bsize=4096 ascii-ci=0 log =internal bsize=4096 blocks=12800, version=2= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0Brian
FYI:
The second (rm version) of the test bisects to the patch:
commit f5ea110044fa858925a880b4fa9f551bfa2dfc38
xfs: add CRCs to dir2/da node blocks
---
The secret to tripping over the bug is run the test until fsstress fills the filesystem before removing the files. So an error handling?
I use the test:
#!/bin/sh
ltp/fsstress -z -s 1378390208 -fsymlink=1 -n9999999 -p4 -d /test2 cd /test2
sync
rm -rf *
If your filesystem is smaller, decrease the -n to make the test faster.
I have still not gotten a core, though Michael Semon sent one.
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 8 lines · Mark Tinguely
The secret to tripping over the bug is run the test until fsstress fills the filesystem before removing the files. So an error handling?
I use the test:
#!/bin/sh
Quoted 8 lines
ltp/fsstress -z -s 1378390208 -fsymlink=1 -n9999999 -p4 -d /test2 cd /test2
sync
rm -rf *If your filesystem is smaller, decrease the -n to make the test faster.
I have still not gotten a core, though Michael Semon sent one.
It would be useful if we could wire this up for xfstests
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 14 lines · Christoph Hellwig
Quoted 13 lines · Mark Tinguely
The secret to tripping over the bug is run the test until fsstress fills the filesystem before removing the files. So an error handling?
I use the test:
#!/bin/sh
ltp/fsstress -z -s 1378390208 -fsymlink=1 -n9999999 -p4 -d /test2 cd /test2
sync
rm -rf *
Quoted 6 lines
Quoted 3 lines
If your filesystem is smaller, decrease the -n to make the test faster.
I have still not gotten a core, though Michael Semon sent one.
It would be useful if we could wire this up for xfstests
Nod
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 14 lines · Christoph Hellwig
Quoted 13 lines · Mark Tinguely
The secret to tripping over the bug is run the test until fsstress fills the filesystem before removing the files. So an error handling?
I use the test:
#!/bin/sh
ltp/fsstress -z -s 1378390208 -fsymlink=1 -n9999999 -p4 -d /test2 cd /test2
sync
rm -rf *
Quoted 5 lines
Quoted 3 lines
If your filesystem is smaller, decrease the -n to make the test faster.
I have still not gotten a core, though Michael Semon sent one.
It would be useful if we could wire this up for xfstests
Just set it up accurately because it takes a long time. It takes a
while to create the links. Additionally, fsstress will keep iterating after the FS is full, and all those extra iterations take time.
In testing on x86, Brian's test will succeed in 10 GB but fail in 11 GB. I don't know if that is the case on x86_64 as well. After the failure case is met and the rm operation hits the assert, continued mount-and-rm operations will also cause the assert to fire, without needing to fill the FS with links again. I don't know when that "bonus condition" stops happening. xfs_repair makes matters worse, IIRC.
That's about all I can add. Thanks!
Michael
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs