2.6.24-rc2 XFS nfsd hang

With 2.6.24-rc2 (amd64) I sometimes (usually but perhaps not always) see a hang when accessing some NFS exported XFS filesystems. Local access to these filesystems ahead of time works without problems.

This does not occur with 2.6.23.1. The filesystem does not appear to be corrupt.

The call chain for the wedged process is:

    [ 1462.911256] nfsd          D ffffffff80547840  4760  2966      2
    [ 1462.911283]  ffff81010414d4d0 0000000000000046 0000000000000000 ffff81010414d610
    [ 1462.911322]  ffff810104cbc6e0 ffff81010414d480 ffffffff80746dc0 ffffffff80746dc0
    [ 1462.911360]  ffffffff80744020 ffffffff80746dc0 ffff81010129c140 ffff8101000ad100
    [ 1462.911391] Call Trace:
    [ 1462.911417]  [<ffffffff8052e638>] __down+0xe9/0x101
    [ 1462.911437]  [<ffffffff8022cc80>] default_wake_function+0x0/0xe
    [ 1462.911458]  [<ffffffff8052e275>] __down_failed+0x35/0x3a
    [ 1462.911480]  [<ffffffff8035ac25>] _xfs_buf_find+0x84/0x24d
    [ 1462.911501]  [<ffffffff8035ad34>] _xfs_buf_find+0x193/0x24d
    [ 1462.911522]  [<ffffffff803599b1>] xfs_buf_lock+0x43/0x45
    [ 1462.911543]  [<ffffffff8035ad5b>] _xfs_buf_find+0x1ba/0x24d
    [ 1462.911564]  [<ffffffff8035ae48>] xfs_buf_get_flags+0x5a/0x14b
    [ 1462.911586]  [<ffffffff8035b490>] xfs_buf_read_flags+0x12/0x86
    [ 1462.911607]  [<ffffffff8034ecf6>] xfs_trans_read_buf+0x4c/0x2cf
    [ 1462.911629]  [<ffffffff803292be>] xfs_da_do_buf+0x41b/0x65b
    [ 1462.911652]  [<ffffffff80329568>] xfs_da_read_buf+0x24/0x29
    [ 1462.911673]  [<ffffffff8032be40>] xfs_dir2_block_lookup_int+0x4d/0x1ab
    [ 1462.911694]  [<ffffffff8032be40>] xfs_dir2_block_lookup_int+0x4d/0x1ab
    [ 1462.911717]  [<ffffffff8032c718>] xfs_dir2_block_lookup+0x15/0x8e
    [ 1462.911738]  [<ffffffff8032b8e1>] xfs_dir_lookup+0xd2/0x12c
    [ 1462.911761]  [<ffffffff8036d658>] submit_bio+0x10d/0x114
    [ 1462.911781]  [<ffffffff8034fb56>] xfs_dir_lookup_int+0x2c/0xc5
    [ 1462.911802]  [<ffffffff802507a2>] lockdep_init_map+0x90/0x495
    [ 1462.911823]  [<ffffffff80353436>] xfs_lookup+0x44/0x6f
    [ 1462.911843]  [<ffffffff8035e364>] xfs_vn_lookup+0x29/0x60
    [ 1462.915246]  [<ffffffff8028856c>] __lookup_hash+0xe5/0x109
    [ 1462.915267]  [<ffffffff802893dd>] lookup_one_len+0x41/0x4e
    [ 1462.915289]  [<ffffffff80303d05>] compose_entry_fh+0xc1/0x117
    [ 1462.915311]  [<ffffffff80303f4c>] encode_entry+0x17c/0x38b
    [ 1462.915333]  [<ffffffff80261e4e>] find_or_create_page+0x3f/0xc9
    [ 1462.915355]  [<ffffffff8035a2c0>] _xfs_buf_lookup_pages+0x2c1/0x2f6
    [ 1462.915377]  [<ffffffff8052ec6b>] _spin_unlock+0x1f/0x49
    [ 1462.915399]  [<ffffffff8027e632>] cache_alloc_refill+0x1ba/0x4b9
    [ 1462.915424]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915448]  [<ffffffff8030416b>] nfs3svc_encode_entry_plus+0x10/0x13
    [ 1462.915469]  [<ffffffff8032c67c>] xfs_dir2_block_getdents+0x15b/0x1e2
    [ 1462.915491]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915514]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915534]  [<ffffffff8032b6da>] xfs_readdir+0x91/0xb6
    [ 1462.915557]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915579]  [<ffffffff8035be9d>] xfs_file_readdir+0x31/0x40
    [ 1462.915599]  [<ffffffff8028c9f8>] vfs_readdir+0x61/0x93
    [ 1462.915619]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915642]  [<ffffffff802fc78e>] nfsd_readdir+0x6d/0xc5
    [ 1462.915663]  [<ffffffff80303158>] nfsd3_proc_readdirplus+0x114/0x204
    [ 1462.915686]  [<ffffffff802f8b82>] nfsd_dispatch+0xde/0x1b6
    [ 1462.915706]  [<ffffffff805215cd>] svc_process+0x3f8/0x717
    [ 1462.915729]  [<ffffffff802f9148>] nfsd+0x1a9/0x2c1
    [ 1462.915749]  [<ffffffff8020c648>] child_rip+0xa/0x12
    [ 1462.915769]  [<ffffffff80520af8>] __svc_create_thread+0xea/0x1eb
    [ 1462.915792]  [<ffffffff802f8f9f>] nfsd+0x0/0x2c1
    [ 1462.915812]  [<ffffffff8020c63e>] child_rip+0x0/0x12

Over time other processes pile up beind this.

    [ 1462.910728] nfsd          D ffffffffffffffff  5440  2965      2
    [ 1462.910769]  ffff8101040cdd40 0000000000000046 0000000000000001 ffff810103471900
    [ 1462.910812]  ffff8101029a72c0 ffff8101040cdcf0 ffffffff80746dc0 ffffffff80746dc0
    [ 1462.910852]  ffffffff80744020 ffffffff80746dc0 ffff81010008e0c0 ffff8101012a1040
    [ 1462.910882] Call Trace:
    [ 1462.910909]  [<ffffffff802fbadf>] nfsd_permission+0x95/0xeb
    [ 1462.910931]  [<ffffffff8028c9dd>] vfs_readdir+0x46/0x93
    [ 1462.910950]  [<ffffffff8052d729>] mutex_lock_nested+0x165/0x27c
    [ 1462.910971]  [<ffffffff8052ec6b>] _spin_unlock+0x1f/0x49
    [ 1462.910994]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.911015]  [<ffffffff8028c9dd>] vfs_readdir+0x46/0x93
    [ 1462.911037]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.911057]  [<ffffffff802fc78e>] nfsd_readdir+0x6d/0xc5
    [ 1462.911079]  [<ffffffff80303158>] nfsd3_proc_readdirplus+0x114/0x204
    [ 1462.911102]  [<ffffffff802f8b82>] nfsd_dispatch+0xde/0x1b6
    [ 1462.911122]  [<ffffffff805215cd>] svc_process+0x3f8/0x717
    [ 1462.911143]  [<ffffffff802f9148>] nfsd+0x1a9/0x2c1
    [ 1462.911165]  [<ffffffff8020c648>] child_rip+0xa/0x12
    [ 1462.911184]  [<ffffffff80520af8>] __svc_create_thread+0xea/0x1eb
    [ 1462.911206]  [<ffffffff802f8f9f>] nfsd+0x0/0x2c1
    [ 1462.911225]  [<ffffffff8020c63e>] child_rip+0x0/0x12

Any suggestions other than to bisect this? (Bisection might be painful as it crosses the x86-merge.)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

I wonder if this is a similar hang to what Christian was seeing here:
http://lkml.org/lkml/2007/11/13/319

Benny

Quoted 97 lines · Chris Wedgwood

With 2.6.24-rc2 (amd64) I sometimes (usually but perhaps not always) see a hang when accessing some NFS exported XFS filesystems. Local access to these filesystems ahead of time works without problems.

This does not occur with 2.6.23.1. The filesystem does not appear to be corrupt.

The call chain for the wedged process is:

    [ 1462.911256] nfsd          D ffffffff80547840  4760  2966      2
    [ 1462.911283]  ffff81010414d4d0 0000000000000046 0000000000000000 ffff81010414d610
    [ 1462.911322]  ffff810104cbc6e0 ffff81010414d480 ffffffff80746dc0 ffffffff80746dc0
    [ 1462.911360]  ffffffff80744020 ffffffff80746dc0 ffff81010129c140 ffff8101000ad100
    [ 1462.911391] Call Trace:
    [ 1462.911417]  [<ffffffff8052e638>] __down+0xe9/0x101
    [ 1462.911437]  [<ffffffff8022cc80>] default_wake_function+0x0/0xe
    [ 1462.911458]  [<ffffffff8052e275>] __down_failed+0x35/0x3a
    [ 1462.911480]  [<ffffffff8035ac25>] _xfs_buf_find+0x84/0x24d
    [ 1462.911501]  [<ffffffff8035ad34>] _xfs_buf_find+0x193/0x24d
    [ 1462.911522]  [<ffffffff803599b1>] xfs_buf_lock+0x43/0x45
    [ 1462.911543]  [<ffffffff8035ad5b>] _xfs_buf_find+0x1ba/0x24d
    [ 1462.911564]  [<ffffffff8035ae48>] xfs_buf_get_flags+0x5a/0x14b
    [ 1462.911586]  [<ffffffff8035b490>] xfs_buf_read_flags+0x12/0x86
    [ 1462.911607]  [<ffffffff8034ecf6>] xfs_trans_read_buf+0x4c/0x2cf
    [ 1462.911629]  [<ffffffff803292be>] xfs_da_do_buf+0x41b/0x65b
    [ 1462.911652]  [<ffffffff80329568>] xfs_da_read_buf+0x24/0x29
    [ 1462.911673]  [<ffffffff8032be40>] xfs_dir2_block_lookup_int+0x4d/0x1ab
    [ 1462.911694]  [<ffffffff8032be40>] xfs_dir2_block_lookup_int+0x4d/0x1ab
    [ 1462.911717]  [<ffffffff8032c718>] xfs_dir2_block_lookup+0x15/0x8e
    [ 1462.911738]  [<ffffffff8032b8e1>] xfs_dir_lookup+0xd2/0x12c
    [ 1462.911761]  [<ffffffff8036d658>] submit_bio+0x10d/0x114
    [ 1462.911781]  [<ffffffff8034fb56>] xfs_dir_lookup_int+0x2c/0xc5
    [ 1462.911802]  [<ffffffff802507a2>] lockdep_init_map+0x90/0x495
    [ 1462.911823]  [<ffffffff80353436>] xfs_lookup+0x44/0x6f
    [ 1462.911843]  [<ffffffff8035e364>] xfs_vn_lookup+0x29/0x60
    [ 1462.915246]  [<ffffffff8028856c>] __lookup_hash+0xe5/0x109
    [ 1462.915267]  [<ffffffff802893dd>] lookup_one_len+0x41/0x4e
    [ 1462.915289]  [<ffffffff80303d05>] compose_entry_fh+0xc1/0x117
    [ 1462.915311]  [<ffffffff80303f4c>] encode_entry+0x17c/0x38b
    [ 1462.915333]  [<ffffffff80261e4e>] find_or_create_page+0x3f/0xc9
    [ 1462.915355]  [<ffffffff8035a2c0>] _xfs_buf_lookup_pages+0x2c1/0x2f6
    [ 1462.915377]  [<ffffffff8052ec6b>] _spin_unlock+0x1f/0x49
    [ 1462.915399]  [<ffffffff8027e632>] cache_alloc_refill+0x1ba/0x4b9
    [ 1462.915424]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915448]  [<ffffffff8030416b>] nfs3svc_encode_entry_plus+0x10/0x13
    [ 1462.915469]  [<ffffffff8032c67c>] xfs_dir2_block_getdents+0x15b/0x1e2
    [ 1462.915491]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915514]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915534]  [<ffffffff8032b6da>] xfs_readdir+0x91/0xb6
    [ 1462.915557]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915579]  [<ffffffff8035be9d>] xfs_file_readdir+0x31/0x40
    [ 1462.915599]  [<ffffffff8028c9f8>] vfs_readdir+0x61/0x93
    [ 1462.915619]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915642]  [<ffffffff802fc78e>] nfsd_readdir+0x6d/0xc5
    [ 1462.915663]  [<ffffffff80303158>] nfsd3_proc_readdirplus+0x114/0x204
    [ 1462.915686]  [<ffffffff802f8b82>] nfsd_dispatch+0xde/0x1b6
    [ 1462.915706]  [<ffffffff805215cd>] svc_process+0x3f8/0x717
    [ 1462.915729]  [<ffffffff802f9148>] nfsd+0x1a9/0x2c1
    [ 1462.915749]  [<ffffffff8020c648>] child_rip+0xa/0x12
    [ 1462.915769]  [<ffffffff80520af8>] __svc_create_thread+0xea/0x1eb
    [ 1462.915792]  [<ffffffff802f8f9f>] nfsd+0x0/0x2c1
    [ 1462.915812]  [<ffffffff8020c63e>] child_rip+0x0/0x12

Over time other processes pile up beind this.

    [ 1462.910728] nfsd          D ffffffffffffffff  5440  2965      2
    [ 1462.910769]  ffff8101040cdd40 0000000000000046 0000000000000001 ffff810103471900
    [ 1462.910812]  ffff8101029a72c0 ffff8101040cdcf0 ffffffff80746dc0 ffffffff80746dc0
    [ 1462.910852]  ffffffff80744020 ffffffff80746dc0 ffff81010008e0c0 ffff8101012a1040
    [ 1462.910882] Call Trace:
    [ 1462.910909]  [<ffffffff802fbadf>] nfsd_permission+0x95/0xeb
    [ 1462.910931]  [<ffffffff8028c9dd>] vfs_readdir+0x46/0x93
    [ 1462.910950]  [<ffffffff8052d729>] mutex_lock_nested+0x165/0x27c
    [ 1462.910971]  [<ffffffff8052ec6b>] _spin_unlock+0x1f/0x49
    [ 1462.910994]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.911015]  [<ffffffff8028c9dd>] vfs_readdir+0x46/0x93
    [ 1462.911037]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.911057]  [<ffffffff802fc78e>] nfsd_readdir+0x6d/0xc5
    [ 1462.911079]  [<ffffffff80303158>] nfsd3_proc_readdirplus+0x114/0x204
    [ 1462.911102]  [<ffffffff802f8b82>] nfsd_dispatch+0xde/0x1b6
    [ 1462.911122]  [<ffffffff805215cd>] svc_process+0x3f8/0x717
    [ 1462.911143]  [<ffffffff802f9148>] nfsd+0x1a9/0x2c1
    [ 1462.911165]  [<ffffffff8020c648>] child_rip+0xa/0x12
    [ 1462.911184]  [<ffffffff80520af8>] __svc_create_thread+0xea/0x1eb
    [ 1462.911206]  [<ffffffff802f8f9f>] nfsd+0x0/0x2c1
    [ 1462.911225]  [<ffffffff8020c63e>] child_rip+0x0/0x12

Any suggestions other than to bisect this? (Bisection might be painful as it crosses the x86-merge.)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Quoted 2 lines · Benny Halevy

I wonder if this is a similar hang to what Christian was seeing here:
http://lkml.org/lkml/2007/11/13/319

Ah, thanks for noticing that. Christian Kujau, is /data an xfs partition?

Quoted 2 lines

Any suggestions other than to bisect this? (Bisection might be painful as it crosses the x86-merge.)

Yes, the nfsd process only got stuck when I did ls(1) (with or without -l) on a NFS share which contained a XFS partition.

Since NFS was not working (the nfsd processes were already in D state), to mount a CIFS share from the very same server (and the same client).

Upon accessing the /data/sub part of the CIFS share, the client hung, waiting for the server to respond (the [cifs] kernel thread on the client was spinning, waiting for i/o). On the server, similar things as with the nfsd processes happened

[<c040914c>] mutex_lock_nested+0xcc/0x2c0
[<c016dc64>] do_lookup+0xa4/0x190
[<c016f6f9>] __link_path_walk+0x749/0xd10
[<c016fd04>] link_path_walk+0x44/0xc0
[<c016fd98>] path_walk+0x18/0x20
[<c016ff98>] do_path_lookup+0x78/0x1c0
[<c0170998>] __user_walk_fd+0x38/0x60
[<c0169bd1>] vfs_stat_fd+0x21/0x50
[<c0169ca1>] vfs_stat+0x11/0x20
[<c0169cc4>] sys_stat64+0x14/0x30
[<c01028d6>] sysenter_past_esp+0x5f/0xa5

I'm not sure what you're doing here, but a viable work-around for now might be to use nfsv2 mounts, something like

mount -o vers=2 ...
or to keep v3 and disable readdirplus doing something like:
mount -o vers=3,nordirplus ...

OK, I'll try this. I hope this can be fixed somehow before 2.6.24...

Quoted 1 line · Christian Kujau

OK, I'll try this. I hope this can be fixed somehow before 2.6.24...

Well, one simple nasty idea would be something like:

fs/Kconfig +1 -1

index 429a002..da231fd 100644
@@ -1604,7 +1604,7 @@ config NFS_FS
 
 config NFS_V3
 	bool "Provide NFSv3 client support"
-	depends on NFS_FS
+	depends on NFS_FS && !XFS
 	help
 	  Say Y here if you want your NFS client to be able to speak version
 	  3 of the NFS protocol.

So people who are likely to be affect just side-step the issue until it's resolved.

Very funny, but disabling XFS on the client won't help.

Oops, I meant it for NFSD... and I'm somewhat serious. I'm not saying it's a good long term solution, but a potentially safer short-term workaround.

Quoted 3 lines · Chris Wedgwood

Oops, I meant it for NFSD... and I'm somewhat serious. I'm not saying it's a good long term solution, but a potentially safer short-term workaround.

I've opened http://bugzilla.kernel.org/show_bug.cgi?id=9400 to track this one (and to not forget about it :)).

I wonder why so few people are seeing this, I'd have assumed that NFSv3 && XFS is not sooo exotic...

Christian.

Signature · 3 lines

BOFH excuse #273:

The cord jumped over and hit the power switch.

Quoted 2 lines

I wonder why so few people are seeing this, I'd have assumed that NFSv3 && XFS is not sooo exotic...

Still on 2.6.23.x here (also use nfsv3 + xfs).

With 2.6.24-rc2 (amd64) I sometimes (usually but perhaps not always) see a hang when accessing some NFS exported XFS filesystems. Local access to these filesystems ahead of time works without problems.

This does not occur with 2.6.23.1. The filesystem does not appear to be corrupt.

After some bisection pain (sg broken in the middle and XFS not compiling in other places) the regression seems to be:

   commit 051e7cd44ab8f0f7c2958371485b4a1ff64a8d1b
   Author: Christoph Hellwig <[email protected]>
   Date:   Tue Aug 28 13:58:24 2007 +1000

With 2.6.24-rc2 (amd64) I sometimes (usually but perhaps not always) see a hang when accessing some NFS exported XFS filesystems. Local access to these filesystems ahead of time works without problems.

This does not occur with 2.6.23.1. The filesystem does not appear to be corrupt.

    [ 1462.911360]  ffffffff80744020 ffffffff80746dc0 ffff81010129c140 ffff8101000ad100
    [ 1462.911391] Call Trace:
    [ 1462.911417]  [<ffffffff8052e638>] __down+0xe9/0x101
    [ 1462.911437]  [<ffffffff8022cc80>] default_wake_function+0x0/0xe
    [ 1462.911458]  [<ffffffff8052e275>] __down_failed+0x35/0x3a
    [ 1462.911480]  [<ffffffff8035ac25>] _xfs_buf_find+0x84/0x24d
    [ 1462.911501]  [<ffffffff8035ad34>] _xfs_buf_find+0x193/0x24d
    [ 1462.911522]  [<ffffffff803599b1>] xfs_buf_lock+0x43/0x45

    [ 1462.915534]  [<ffffffff8032b6da>] xfs_readdir+0x91/0xb6
    [ 1462.915557]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915579]  [<ffffffff8035be9d>] xfs_file_readdir+0x31/0x40
    [ 1462.915599]  [<ffffffff8028c9f8>] vfs_readdir+0x61/0x93
    [ 1462.915619]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915642]  [<ffffffff802fc78e>] nfsd_readdir+0x6d/0xc5

Quoted 8 lines · Chris Wedgwood

With 2.6.24-rc2 (amd64) I sometimes (usually but perhaps not always) see a hang when accessing some NFS exported XFS filesystems. Local access to these filesystems ahead of time works without problems.

This does not occur with 2.6.23.1. The filesystem does not appear to be corrupt.

Quoted 8 lines

    [ 1462.911360]  ffffffff80744020 ffffffff80746dc0 ffff81010129c140 ffff8101000ad100
    [ 1462.911391] Call Trace:
    [ 1462.911417]  [<ffffffff8052e638>] __down+0xe9/0x101
    [ 1462.911437]  [<ffffffff8022cc80>] default_wake_function+0x0/0xe
    [ 1462.911458]  [<ffffffff8052e275>] __down_failed+0x35/0x3a
    [ 1462.911480]  [<ffffffff8035ac25>] _xfs_buf_find+0x84/0x24d
    [ 1462.911501]  [<ffffffff8035ad34>] _xfs_buf_find+0x193/0x24d
    [ 1462.911522]  [<ffffffff803599b1>] xfs_buf_lock+0x43/0x45

this is bp->b_sema which lookup wants.

Quoted 6 lines

    [ 1462.915534]  [<ffffffff8032b6da>] xfs_readdir+0x91/0xb6
    [ 1462.915557]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915579]  [<ffffffff8035be9d>] xfs_file_readdir+0x31/0x40
    [ 1462.915599]  [<ffffffff8028c9f8>] vfs_readdir+0x61/0x93
    [ 1462.915619]  [<ffffffff8030415b>] nfs3svc_encode_entry_plus+0x0/0x13
    [ 1462.915642]  [<ffffffff802fc78e>] nfsd_readdir+0x6d/0xc5

and this is the nasty nfsd case where a filldir callback calls back into lookup. I suspect we're somehow holding b_sema already. Previously this was okay because we weren't inside the actualy readdir code when calling filldir but operate on a copy of the data.

This gem has bitten other filesystem before, I'll see if I can find a way around it.

This must have come up before; feel free to remind me: is there any way to make the interface easier to use? (E.g. would it help if the filldir callback could be passed a dentry?)

Quoted 3 lines · J. Bruce Fields

This must have come up before; feel free to remind me: is there any way to make the interface easier to use? (E.g. would it help if the filldir callback could be passed a dentry?)

The best thing for the filesystem would be to have a readdirplus (or have it folded into readdir) instead of calling into lookup from ->filldir.

Quoted 5 lines · Christoph Hellwig

Quoted 3 lines · J. Bruce Fields

This must have come up before; feel free to remind me: is there any way to make the interface easier to use? (E.g. would it help if the filldir callback could be passed a dentry?)

Quoted 3 lines

The best thing for the filesystem would be to have a readdirplus (or have it folded into readdir) instead of calling into lookup from ->filldir.

And the readdirplus would pass a dentry to its equivalent of ->filldir? Or something else?

Quoted 10 lines · J. Bruce Fields

Quoted 8 lines · Christoph Hellwig

Quoted 3 lines · J. Bruce Fields

This must have come up before; feel free to remind me: is there any way to make the interface easier to use? (E.g. would it help if the filldir callback could be passed a dentry?)

The best thing for the filesystem would be to have a readdirplus (or have it folded into readdir) instead of calling into lookup from ->filldir.

Quoted 2 lines

And the readdirplus would pass a dentry to its equivalent of ->filldir? Or something else?

Personally I'd prefer it to only grow a struct stat or rather it's members But the nfsd code currently expects a dentry so this might require some major refactoring.

Quoted 3 lines

Personally I'd prefer it to only grow a struct stat or rather it's members But the nfsd code currently expects a dentry so this might require some major refactoring.

Well, we need to check for mountpoints, for example, so I don't see any way out of needing a dentry. What's the drawback?

Quoted 6 lines · J. Bruce Fields

Quoted 3 lines

Personally I'd prefer it to only grow a struct stat or rather it's members But the nfsd code currently expects a dentry so this might require some major refactoring.

Well, we need to check for mountpoints, for example, so I don't see any way out of needing a dentry. What's the drawback?

You're right - we'd probably need the dentry. The drawback is that we need to always get it in the dcache. Which might be a good thing depending on the workload.

The current readdir implementation deadlocks on a btree buffers locks because nfsd calls back into ->lookup from the filldir callback. The only short-term fix for this is to revert to the old inefficient double-buffering scheme.

This patch does exactly that and reverts xfs_file_readdir to what's basically the 2.6.23 version minus the uio and vnops junk.

This patch does exactly that and reverts xfs_file_readdir to what's basically the 2.6.23 version minus the uio and vnops junk.

Quoted 2 lines · Christoph Hellwig

This patch does exactly that and reverts xfs_file_readdir to what's basically the 2.6.23 version minus the uio and vnops junk.

Thanks, works here too (without nordirplus as a mountoption). Am I supposed to close the bug[0] or do you guys want to leave this open to track the Real Fix (TM) for 2.6.25?

The current readdir implementation deadlocks on a btree buffers locks because nfsd calls back into ->lookup from the filldir callback. The only short-term fix for this is to revert to the old inefficient double-buffering scheme.

This patch does exactly that and reverts xfs_file_readdir to what's basically the 2.6.23 version minus the uio and vnops junk.

I'll try to find something more optimal for 2.6.25 or at least find a way to use the proper version for local access.

Signed-off-by Christoph Hellwig <[email protected]>

Index: linux-2.6/fs/xfs/linux-2.6/xfs_file.c
===================================================================

fs/xfs/linux-2.6/xfs_file.c +124 -1

@@ -218,6 +218,15 @@
 }
 #endif /* CONFIG_XFS_DMAPI */
 
+/*
+ * Unfortunately we can't just use the clean and simple readdir implementation
+ * below, because nfs might call back into ->lookup from the filldir callback
+ * and that will deadlock the low-level btree code.
+ *
+ * Hopefully we'll find a better workaround that allows to use the optimal
+ * version at least for local readdirs for 2.6.25.
+ */
+#if 0
 STATIC int
 xfs_file_readdir(
 	struct file	*filp,

@@ -249,6 +258,121 @@
 		return -error;
 	return 0;
 }
+#else
+
+struct hack_dirent {
+	int		namlen;
+	loff_t		offset;
+	u64		ino;
+	unsigned int	d_type;
+	char		name[];
+};
+
+struct hack_callback {
+	char		*dirent;
+	size_t		len;
+	size_t		used;
+};
+
+STATIC int
+xfs_hack_filldir(
+	void		*__buf,
+	const char	*name,
+	int		namlen,
+	loff_t		offset,
+	u64		ino,
+	unsigned int	d_type)
+{
+	struct hack_callback *buf = __buf;
+	struct hack_dirent *de = (struct hack_dirent *)(buf->dirent + buf->used);
+
+	if (buf->used + sizeof(struct hack_dirent) + namlen > buf->len)
+		return -EINVAL;
+
+	de->namlen = namlen;
+	de->offset = offset;
+	de->ino = ino;
+	de->d_type = d_type;
+	memcpy(de->name, name, namlen);
+	buf->used += sizeof(struct hack_dirent) + namlen;
+	return 0;
+}
+
+STATIC int
+xfs_file_readdir(
+	struct file	*filp,
+	void		*dirent,
+	filldir_t	filldir)
+{
+	struct inode	*inode = filp->f_path.dentry->d_inode;
+	xfs_inode_t	*ip = XFS_I(inode);
+	struct hack_callback buf;
+	struct hack_dirent *de;
+	int		error;
+	loff_t		size;
+	int		eof = 0;
+	xfs_off_t       start_offset, curr_offset, offset;
+
+	/*
+	 * Try fairly hard to get memory
+	 */
+	buf.len = PAGE_CACHE_SIZE;
+	do {
+		buf.dirent = kmalloc(buf.len, GFP_KERNEL);
+		if (buf.dirent)
+			break;
+		buf.len >>= 1;
+	} while (buf.len >= 1024);
+
+	if (!buf.dirent)
+		return -ENOMEM;
+
+	curr_offset = filp->f_pos;
+	if (curr_offset == 0x7fffffff)
+		offset = 0xffffffff;
+	else
+		offset = filp->f_pos;
+
+	while (!eof) {
+		int reclen;
+		start_offset = offset;
+
+		buf.used = 0;
+		error = -xfs_readdir(ip, &buf, buf.len, &offset,
+				     xfs_hack_filldir);
+		if (error || offset == start_offset) {
+			size = 0;
+			break;
+		}
+
+		size = buf.used;
+		de = (struct hack_dirent *)buf.dirent;
+		while (size > 0) {
+			if (filldir(dirent, de->name, de->namlen,
+					curr_offset & 0x7fffffff,
+					de->ino, de->d_type)) {
+				goto done;
+			}
+
+			reclen = sizeof(struct hack_dirent) + de->namlen;
+			size -= reclen;
+			curr_offset = de->offset /* & 0x7fffffff */;
+			de = (struct hack_dirent *)((char *)de + reclen);
+		}
+	}
+
+ done:
+ 	if (!error) {
+		if (size == 0)
+			filp->f_pos = offset & 0x7fffffff;
+		else if (de)
+			filp->f_pos = curr_offset;
+	}
+
+	kfree(buf.dirent);
+	return error;
+}
+#endif
 
 STATIC int
 xfs_file_mmap(
-

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Quoted 4 lines · Christoph Hellwig

The current readdir implementation deadlocks on a btree buffers locks because nfsd calls back into ->lookup from the filldir callback. The only short-term fix for this is to revert to the old inefficient double-buffering scheme.

Probably why Steve did this: :)

xfs_file.c
----------------------------
revision 1.40
date: 2001/03/15 23:33:20; author: lord; state: Exp; lines: +54 -17
modid: 2.4.x-xfs:slinx:90125a
Change linvfs_readdir to allocate a buffer, call xfs to fill it, and then call the filldir function on each entry. This is instead of doing the filldir deep in the bowels of xfs which causes locking problems.
----------------------------

Looks like the readdir is in the bowels of the btree code when filldir gets called here, there are probably locks on several buffers in the btree at this point. This will only show up for large directories I bet.

Just rambling, not a single line of code was consulted in writing this message.

Quoted 4 lines · Stephen Lord

Looks like the readdir is in the bowels of the btree code when filldir gets called here, there are probably locks on several buffers in the btree at this point. This will only show up for large directories I bet.

I see it for fairly small directories. Larger than what you can stuff into an inode but less than a block (I'm not checking but fairly sure that's the case).

Quoted 3 lines

Just rambling, not a single line of code was consulted in writing this message.

Can you explain why the offset is capped and treated in an 'odd way' at all?

fs/xfs/linux-2.6/xfs_file.c +5 -0

+       curr_offset = filp->f_pos;
+       if (curr_offset == 0x7fffffff)
+               offset = 0xffffffff;
+       else
+               offset = filp->f_pos;

and later the offset to filldir is masked. Is that some restriction in filldir?

Wow, was it really that long ago!

Looks like the readdir is in the bowels of the btree code when filldir gets called
here, there are probably locks on several buffers in the btree at this point. This
will only show up for large directories I bet.

The xfs readdir code has the complete xfs inode number in its hands at this point
(filldir is not necessarily getting all the bits of it). All we are doing the lookup
for really is to get the inode number back again so we can get the inode and get the attributes. Rather dumb really. There has got to be a way of doing a callout structure here so that the inode number can be pushed through filldir and back into an fs specific call. The fs then can do a lookup
by id - which is what it does most of the time for resolving nfs handles anyway. Should be more efficient than the current scheme.

Hmmm, don't see the point of "eof" local var now. Previously bhv_vop_readdir() returned eof.
I presume if we don't move the offset (offset == startoffset) then we're done and break out?
So we lost eof when going to the filldir in the getdents code etc...