Skip to content

rcu-walk and dcache scaling

Nick Piggin ceph-devel linux-fsdevel lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest45 of 46 patches 45 repliesIn review

45 of 46 patches are in this archive. The cover letter announces 46.

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
2/46fs: d_validate fixesfs/dcache.c+7 -184 replies
3/46kernel: kmem_ptr_validate considered harmfulinclude/linux/slab.h mm/slab.c mm/slob.c +2 more+1 -990 replies
4/46fs: dcache documentation cleanupinclude/linux/dcache.h+6 -120 replies
5/46fs: change d_delete semanticsDocumentation/filesystems/porting Documentation/filesystems/vfs.txt arch/ia64/kernel/perfmon.c +17 more+45 -390 replies
6/46cifs: dont overwrite dentry name in d_revalidatefs/cifs/dir.c+24 -190 replies
7/46jfs: dont overwrite dentry name in d_revalidatefs/jfs/namei.c+35 -80 replies
8/46fs: change d_compare for rcu-walkDocumentation/filesystems/Locking Documentation/filesystems/porting Documentation/filesystems/vfs.txt +19 more+228 -1520 replies
9/46fs: change d_hash for rcu-walkDocumentation/filesystems/Locking Documentation/filesystems/porting Documentation/filesystems/vfs.txt +20 more+88 -490 replies
10/46hostfs: simplify lockingfs/dcache.c fs/hostfs/hostfs_kern.c include/linux/dcache.h+24 -170 replies
11/46fs: dcache scale hashfs/dcache.c include/linux/dcache.h+30 -119 replies
12/46fs: dcache scale lrufs/dcache.c+84 -282 replies
13/46fs: dcache scale dentry refcountarch/powerpc/platforms/cell/spufs/inode.c drivers/infiniband/hw/ipath/ipath_fs.c drivers/infiniband/hw/qib/qib_fs.c +18 more+126 -690 replies
14/46fs: dcache scale d_unhashedarch/powerpc/platforms/cell/spufs/inode.c drivers/usb/core/inode.c fs/autofs4/autofs_i.h +7 more+109 -560 replies
15/46fs: dcache scale subdirsdrivers/staging/smbfs/cache.c drivers/usb/core/inode.c fs/autofs4/autofs_i.h +13 more+314 -1340 replies
16/46fs: scale inode alias listfs/9p/vfs_inode.c fs/affs/amigaffs.c fs/cifs/inode.c +6 more+78 -90 replies
17/46fs: Use rename lock and RCU for multi-step operationsdrivers/staging/pohmelfs/path_entry.c fs/autofs4/waitq.c fs/dcache.c +2 more+153 -3112 replies
18/46fs: increase d_name lock coveragefs/dcache.c+16 -70 replies
19/46fs: dcache remove dcache_lockDocumentation/filesystems/Locking Documentation/filesystems/dentry-locking.txt Documentation/filesystems/porting +37 more+120 -3150 replies
20/46fs: dcache avoid starvation in dcache multi-step operationsfs/dcache.c+42 -140 replies
21/46fs: dcache reduce dput lockingfs/dcache.c+25 -310 replies
22/46fs: dcache reduce locking in d_allocfs/dcache.c+4 -20 replies
23/46fs: dcache reduce dcache_inode_lockfs/dcache.c+12 -120 replies
24/46fs: dcache rationalise dget variantsarch/powerpc/platforms/cell/spufs/inode.c drivers/infiniband/hw/ipath/ipath_fs.c drivers/infiniband/hw/qib/qib_fs.c +9 more+23 -460 replies
25/46fs: dcache reduce d_parent lockingfs/dcache.c+14 -110 replies
26/46fs: dcache reduce prune_one_dentry lockingfs/dcache.c+15 -120 replies
27/46fs: reduce dcache_inode_lock width in lru scanningfs/dcache.c+8 -80 replies
28/46fs: use RCU in shrink_dentry_list to reduce lock nestingfs/dcache.c+21 -170 replies
29/46fs: consolidate dentry kill sequencefs/dcache.c+62 -750 replies
30/46fs: icache RCU free inodesDocumentation/filesystems/porting arch/powerpc/platforms/cell/spufs/inode.c drivers/staging/pohmelfs/inode.c +56 more+483 -680 replies
31/46fs: avoid inode RCU freeing for pseudo fsfs/inode.c fs/pipe.c include/linux/fs.h +2 more+22 -90 replies
32/46kernel: optimise seqlockinclude/linux/seqlock.h+60 -70 replies
33/46fs: rcu-walk for path lookupDocumentation/filesystems/dentry-locking.txt Documentation/filesystems/path-lookup.txt fs/dcache.c +4 more+962 -3400 replies
34/46fs: fs_struct use seqlockfs/fs_struct.c fs/namei.c include/linux/fs_struct.h+35 -140 replies
35/46fs: dcache remove d_mountedfs/autofs4/expire.c fs/dcache.c fs/namespace.c +1 more+58 -270 replies
36/46fs: dcache reduce branches in lookup patharch/ia64/kernel/perfmon.c drivers/staging/autofs/root.c fs/9p/vfs_inode.c +59 more+165 -1330 replies
37/46fs: cache optimise dentry and inode for rcu-walkfs/dcache.c include/linux/dcache.h include/linux/fs.h+37 -370 replies
38/46fs: prefetch inode data in dcache lookupfs/dcache.c+3 -00 replies
39/46fs: d_revalidate_rcu for rcu-walkfs/dcache.c fs/ecryptfs/dentry.c fs/fat/namei_vfat.c +4 more+85 -340 replies
40/46fs: provide rcu-walk aware permission i_opsfs/namei.c include/linux/fs.h+107 -300 replies
41/46fs: provide simple rcu-walk ACL implementationfs/btrfs/acl.c fs/btrfs/ctree.h fs/btrfs/inode.c +19 more+123 -510 replies
42/46kernel: add bl_listinclude/linux/list_bl.h include/linux/rculist_bl.h+269 -00 replies
43/46bit_spinlock: add required includesinclude/linux/bit_spinlock.h+4 -00 replies
44/46fs: dcache per-bucket dcache hash lockingfs/dcache.c fs/super.c include/linux/dcache.h +1 more+97 -630 replies
45/46fs: dcache per-inode inode alias lockingfs/9p/vfs_inode.c fs/affs/amigaffs.c fs/dcache.c +5 more+62 -550 replies
46/46fs: improve scalability of pseudo filesystemsfs/anon_inodes.c fs/dcache.c fs/pipe.c +2 more+16 -30 replies

Reviews trailers per patch

1 of 45 patches has a Reviewed-by or Acked-by.

PatchPaul E. McKenney
2/46
3/46
4/46
5/46
6/46
7/46
8/46
9/46
10/46
11/46
12/46
13/46
14/46
15/46
16/46
17/46
18/46
19/46
20/46
21/46
22/46
23/46
24/46
25/46
26/46
27/46
28/46
29/46
30/46
31/46
32/46
33/46
34/46
35/46
36/46
37/46
38/46
39/46
40/46
41/46
42/46R-b
43/46
44/46
45/46
46/46

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH 00/46] rcu-walk and dcache scaling

git://git.kernel.org/pub/scm/linux/kernel/git/npiggin/linux-npiggin.git vfs-scale-working

Here is an new set of vfs patches for review, not that there was much interest last time they were posted. It is structured like:

* preparation patches
* introduce new locks to take over dcache_lock, then remove it
* cleaning up and reworking things for new locks
* rcu-walk path walking
* start on some fine grained locking steps

Thanks,
Nick

Nick Piggin (46):
Revert "fs: use RCU read side protection in d_validate"
fs: d_validate fixes
kernel: kmem_ptr_validate considered harmful
fs: dcache documentation cleanup
fs: change d_delete semantics
cifs: dont overwrite dentry name in d_revalidate
jfs: dont overwrite dentry name in d_revalidate
fs: change d_compare for rcu-walk
fs: change d_hash for rcu-walk
hostfs: simplify locking
fs: dcache scale hash
fs: dcache scale lru
fs: dcache scale dentry refcount
fs: dcache scale d_unhashed
fs: dcache scale subdirs
fs: scale inode alias list
fs: Use rename lock and RCU for multi-step operations
fs: increase d_name lock coverage
fs: dcache remove dcache_lock
fs: dcache avoid starvation in dcache multi-step operations
fs: dcache reduce dput locking
fs: dcache reduce locking in d_alloc
fs: dcache reduce dcache_inode_lock
fs: dcache rationalise dget variants
fs: dcache reduce d_parent locking
fs: dcache reduce prune_one_dentry locking
fs: reduce dcache_inode_lock width in lru scanning
fs: use RCU in shrink_dentry_list to reduce lock nesting
fs: consolidate dentry kill sequence
fs: icache RCU free inodes
fs: avoid inode RCU freeing for pseudo fs
kernel: optimise seqlock
fs: rcu-walk for path lookup
fs: fs_struct use seqlock
fs: dcache remove d_mounted
fs: dcache reduce branches in lookup path
fs: cache optimise dentry and inode for rcu-walk
fs: prefetch inode data in dcache lookup
fs: d_revalidate_rcu for rcu-walk
fs: provide rcu-walk aware permission i_ops
fs: provide simple rcu-walk ACL implementation
kernel: add bl_list
bit_spinlock: add required includes
fs: dcache per-bucket dcache hash locking
fs: dcache per-inode inode alias locking
fs: improve scalability of pseudo filesystems

 Documentation/filesystems/Locking            |   23 +-
 Documentation/filesystems/dentry-locking.txt |  174 ----
 Documentation/filesystems/path-lookup.txt    |  247 ++++++
 Documentation/filesystems/porting            |   45 +-
 Documentation/filesystems/vfs.txt            |   54 +-
 arch/ia64/kernel/perfmon.c                   |    4 +-
 arch/powerpc/platforms/cell/spufs/inode.c    |   18 +-
 drivers/infiniband/hw/ipath/ipath_fs.c       |    8 +-
 drivers/infiniband/hw/qib/qib_fs.c           |    5 +-
 drivers/staging/autofs/root.c                |    2 +-
 drivers/staging/pohmelfs/inode.c             |   11 +-
 drivers/staging/pohmelfs/path_entry.c        |   17 +-
 drivers/staging/smbfs/cache.c                |   10 +-
 drivers/usb/core/inode.c                     |   12 +-
 fs/9p/vfs_dentry.c                           |    4 +-
 fs/9p/vfs_inode.c                            |   39 +-
 fs/adfs/dir.c                                |   13 +-
 fs/adfs/super.c                              |   11 +-
 fs/affs/amigaffs.c                           |    4 +-
 fs/affs/namei.c                              |   66 +-
 fs/affs/super.c                              |   11 +-
 fs/afs/dir.c                                 |    6 +-
 fs/afs/super.c                               |   10 +-
 fs/anon_inodes.c                             |    4 +-
 fs/autofs4/autofs_i.h                        |   21 +-
 fs/autofs4/expire.c                          |  143 ++--
 fs/autofs4/inode.c                           |    2 +-
 fs/autofs4/root.c                            |   78 +-
 fs/autofs4/waitq.c                           |   23 +-
 fs/befs/linuxvfs.c                           |   10 +-
 fs/bfs/inode.c                               |    9 +-
 fs/block_dev.c                               |    9 +-
 fs/btrfs/acl.c                               |   19 +-
 fs/btrfs/ctree.h                             |    4 +-
 fs/btrfs/export.c                            |    4 +-
 fs/btrfs/inode.c                             |   27 +-
 fs/ceph/dir.c                                |   21 +-
 fs/ceph/inode.c                              |   27 +-
 fs/ceph/mds_client.c                         |    2 +-
 fs/cifs/cifsfs.c                             |    9 +-
 fs/cifs/dir.c                                |   75 +-
 fs/cifs/inode.c                              |   14 +-
 fs/cifs/link.c                               |    4 +-
 fs/cifs/readdir.c                            |    6 +-
 fs/coda/cache.c                              |    4 +-
 fs/coda/dir.c                                |    8 +-
 fs/coda/inode.c                              |    9 +-
 fs/configfs/configfs_internal.h              |    4 +-
 fs/configfs/dir.c                            |   13 +-
 fs/configfs/inode.c                          |    8 +-
 fs/dcache.c                                  | 1224 ++++++++++++++++++--------
 fs/ecryptfs/dentry.c                         |   15 +-
 fs/ecryptfs/inode.c                          |    8 +-
 fs/ecryptfs/main.c                           |    4 +-
 fs/ecryptfs/super.c                          |   12 +-
 fs/efs/super.c                               |    9 +-
 fs/exofs/super.c                             |    9 +-
 fs/exportfs/expfs.c                          |   14 +-
 fs/ext2/acl.c                                |   11 +-
 fs/ext2/acl.h                                |    8 +-
 fs/ext2/file.c                               |    2 +-
 fs/ext2/namei.c                              |    4 +-
 fs/ext2/super.c                              |    9 +-
 fs/ext3/acl.c                                |   11 +-
 fs/ext3/acl.h                                |    8 +-
 fs/ext3/file.c                               |    2 +-
 fs/ext3/namei.c                              |    4 +-
 fs/ext3/super.c                              |    9 +-
 fs/ext4/acl.c                                |   11 +-
 fs/ext4/acl.h                                |    4 +-
 fs/ext4/file.c                               |    2 +-
 fs/ext4/namei.c                              |    4 +-
 fs/ext4/super.c                              |    9 +-
 fs/fat/inode.c                               |   13 +-
 fs/fat/namei_msdos.c                         |   24 +-
 fs/fat/namei_vfat.c                          |   69 +-
 fs/filesystems.c                             |    3 +
 fs/freevxfs/vxfs_inode.c                     |    9 +-
 fs/fs_struct.c                               |   10 +
 fs/fuse/dir.c                                |    9 +-
 fs/fuse/inode.c                              |   13 +-
 fs/generic_acl.c                             |   20 +
 fs/gfs2/dentry.c                             |    5 +-
 fs/gfs2/export.c                             |    4 +-
 fs/gfs2/ops_fstype.c                         |    2 +-
 fs/gfs2/ops_inode.c                          |    2 +-
 fs/gfs2/super.c                              |    9 +-
 fs/hfs/dir.c                                 |    2 +-
 fs/hfs/hfs_fs.h                              |    7 +-
 fs/hfs/string.c                              |   17 +-
 fs/hfs/super.c                               |   11 +-
 fs/hfsplus/dir.c                             |    2 +-
 fs/hfsplus/hfsplus_fs.h                      |    7 +-
 fs/hfsplus/super.c                           |   12 +-
 fs/hfsplus/unicode.c                         |   17 +-
 fs/hostfs/hostfs_kern.c                      |   37 +-
 fs/hpfs/dentry.c                             |   26 +-
 fs/hpfs/super.c                              |    9 +-
 fs/hppfs/hppfs.c                             |    9 +-
 fs/hugetlbfs/inode.c                         |    9 +-
 fs/inode.c                                   |   16 +-
 fs/isofs/inode.c                             |  127 ++--
 fs/isofs/namei.c                             |    5 +-
 fs/jffs2/super.c                             |    9 +-
 fs/jfs/namei.c                               |   60 +-
 fs/jfs/super.c                               |   12 +-
 fs/libfs.c                                   |   63 +-
 fs/locks.c                                   |    2 +-
 fs/logfs/inode.c                             |    9 +-
 fs/minix/inode.c                             |    9 +-
 fs/minix/namei.c                             |    2 +-
 fs/namei.c                                   |  855 +++++++++++++++----
 fs/namespace.c                               |   29 +-
 fs/ncpfs/dir.c                               |   66 +-
 fs/ncpfs/inode.c                             |   11 +-
 fs/ncpfs/ncplib_kernel.h                     |   16 +-
 fs/nfs/dir.c                                 |   17 +-
 fs/nfs/getroot.c                             |   10 +-
 fs/nfs/inode.c                               |    9 +-
 fs/nfs/namespace.c                           |   17 +-
 fs/nfs/unlink.c                              |    2 +-
 fs/nfsd/vfs.c                                |    5 +-
 fs/nilfs2/super.c                            |   12 +-
 fs/notify/fsnotify.c                         |    8 +-
 fs/ntfs/inode.c                              |    9 +-
 fs/ocfs2/dcache.c                            |   10 +-
 fs/ocfs2/dlmfs/dlmfs.c                       |    9 +-
 fs/ocfs2/export.c                            |    4 +-
 fs/ocfs2/namei.c                             |   10 +-
 fs/ocfs2/super.c                             |    9 +-
 fs/openpromfs/inode.c                        |    9 +-
 fs/pipe.c                                    |   10 +-
 fs/proc/base.c                               |   14 +-
 fs/proc/generic.c                            |    4 +-
 fs/proc/inode.c                              |    9 +-
 fs/proc/proc_sysctl.c                        |   18 +-
 fs/qnx4/inode.c                              |    9 +-
 fs/reiserfs/super.c                          |    9 +-
 fs/reiserfs/xattr.c                          |    2 +-
 fs/romfs/super.c                             |    9 +-
 fs/squashfs/super.c                          |    9 +-
 fs/super.c                                   |    3 +-
 fs/sysfs/dir.c                               |   22 +-
 fs/sysv/inode.c                              |    9 +-
 fs/sysv/namei.c                              |    5 +-
 fs/sysv/super.c                              |    2 +-
 fs/ubifs/super.c                             |   10 +-
 fs/udf/super.c                               |    9 +-
 fs/ufs/super.c                               |    9 +-
 fs/xfs/linux-2.6/xfs_acl.c                   |    8 +-
 fs/xfs/linux-2.6/xfs_iops.c                  |    8 +-
 fs/xfs/xfs_acl.h                             |    4 +-
 fs/xfs/xfs_iget.c                            |   13 +-
 include/linux/bit_spinlock.h                 |    4 +
 include/linux/dcache.h                       |  183 ++--
 include/linux/fs.h                           |   61 +-
 include/linux/fs_struct.h                    |    3 +
 include/linux/fsnotify.h                     |    2 -
 include/linux/fsnotify_backend.h             |   11 +-
 include/linux/generic_acl.h                  |    1 +
 include/linux/list_bl.h                      |  141 +++
 include/linux/namei.h                        |   16 +-
 include/linux/ncp_fs.h                       |    4 +-
 include/linux/posix_acl.h                    |   19 +
 include/linux/rculist_bl.h                   |  128 +++
 include/linux/seqlock.h                      |   67 ++-
 include/linux/slab.h                         |    2 -
 ipc/mqueue.c                                 |    9 +-
 kernel/cgroup.c                              |   29 +-
 mm/filemap.c                                 |    3 -
 mm/shmem.c                                   |   15 +-
 mm/slab.c                                    |   32 +-
 mm/slob.c                                    |    5 -
 mm/slub.c                                    |   40 -
 mm/util.c                                    |   21 -
 net/socket.c                                 |    5 +-
 net/sunrpc/rpc_pipe.c                        |   14 +-
 security/selinux/selinuxfs.c                 |   16 +-
 security/tomoyo/realpath.c                   |    1 +
 179 files changed, 3848 insertions(+), 1711 deletions(-)
 delete mode 100644 Documentation/filesystems/dentry-locking.txt
 create mode 100644 Documentation/filesystems/path-lookup.txt
 create mode 100644 include/linux/list_bl.h
 create mode 100644 include/linux/rculist_bl.h