Skip to content

xfs: lockless buffer lookups

Dave Chinner linux-xfs 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest6 patches 9 repliesIn review
  2. v26 patches 17 repliesSuperseded
  3. v11 patch 1 replySuperseded

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/6xfs: rework xfs_buf_incore() APIfs/xfs/libxfs/xfs_attr_remote.c fs/xfs/scrub/repair.c fs/xfs/xfs_buf.c +2 more+41 -370 replies
2/6xfs: break up xfs_buf_find() into individual piecesfs/xfs/xfs_buf.c+105 -541 reply
3/6xfs: merge xfs_buf_find() and xfs_buf_get_map()fs/xfs/xfs_buf.c+95 -1123 replies
4/6xfs: reduce the number of atomic when locking a buffer after lookupfs/xfs/xfs_buf.c+3 -20 replies
5/6xfs: remove a superflous hash lookup when inserting new buffersfs/xfs/xfs_buf.c+9 -40 replies
6/6xfs: lockless buffer lookupfs/xfs/xfs_buf.c fs/xfs/xfs_buf.h+16 -71 reply

Reviews trailers per patch

All 6 patches have a Reviewed-by or Acked-by.

PatchDarrick J. WongChristoph Hellwig
1/6R-b
2/6R-b
3/6R-b
4/6R-b
5/6R-b R-b
6/6R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [PATCH 0/6 v3] xfs: lockless buffer lookups

Hi folks,

Current work to merge the XFS inode life cycle with the VFS indoe life cycle is finding some interesting issues. If we have a path that hits buffer trylocks fairly hard (e.g. a non-blocking background inode freeing function), we end up hitting massive contention on the buffer cache hash locks:

- 92.71% 0.05% [kernel] [k] xfs_inodegc_worker
- 92.67% xfs_inodegc_worker
- 92.13% xfs_inode_unlink
- 91.52% xfs_inactive_ifree
- 85.63% xfs_read_agi
- 85.61% xfs_trans_read_buf_map
- 85.59% xfs_buf_read_map
- xfs_buf_get_map
- 85.55% xfs_buf_find
- 72.87% _raw_spin_lock
- do_raw_spin_lock
71.86% __pv_queued_spin_lock_slowpath
- 8.74% xfs_buf_rele
- 7.88% _raw_spin_lock
- 7.88% do_raw_spin_lock 7.63% __pv_queued_spin_lock_slowpath
- 1.70% xfs_buf_trylock
- 1.68% down_trylock
- 1.41% _raw_spin_lock_irqsave
- 1.39% do_raw_spin_lock __pv_queued_spin_lock_slowpath
- 0.76% _raw_spin_unlock
0.75% do_raw_spin_unlock

This is basically hammering the pag->pag_buf_lock from lots of CPUs doing trylocks at the same time. Most of the buffer trylock operations ultimately fail after we've done the lookup, so we're really hammering the buf hash lock whilst making no progress.

We can also see significant spinlock traffic on the same lock just under normal operation when lots of tasks are accessing metadata from the same AG, so let's avoid all this by creating a lookup fast path which leverages the rhashtable's ability to do rcu protected lookups.

This is a rework of the initial lockless buffer lookup patch I sent here:

https://lore.kernel.org/linux-xfs/[email protected]/

And the alternative cleanup sent by Christoph here:

https://lore.kernel.org/linux-xfs/[email protected]/

This version isn't quite a short as Christophs, but it does roughly the same thing in killing the two-phase _xfs_buf_find() call mechanism. It separates the fast and slow paths a little more cleanly and doesn't have context dependent buffer return state from the slow path that the caller needs to handle. It also picks up the rhashtable insert optimisation that Christoph added.

This series passes fstests under several different configs and does not cause any obvious regressions in scalability testing that has been performed. Hence I'm proposing this as potential 5.20 cycle material.

Thoughts, comments?

Version 3:
- rebased onto linux-xfs/for-next
- rearranged some of the changes to avoid repeated shuffling of code to different locations
- fixed typos in commits
- s/xfs_buf_find_verify/xfs_buf_map_verify/
- s/xfs_buf_find_fast/xfs_buf_lookup/

Version 2:
- https://lore.kernel.org/linux-xfs/[email protected]/
- based on 5.19-rc2
- high speed collision of original proposals.

Initial versions:
- https://lore.kernel.org/linux-xfs/[email protected]/
- https://lore.kernel.org/linux-xfs/[email protected]/