gfs2: iopen glock locking scheme rework
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest11 patches 0 repliesNew
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/11 | gfs2: Allow ASPACE glocks to also have an lvbfs/ | +3 | 0 replies | |
| 2/11 | gfs2: Don't add glocks to the LRU while still in usefs/ fs/ fs/ | +1 | 0 replies | |
| 3/11 | gfs2: Keep track of deletes in inode LVBsfs/ fs/ fs/ +2 more | +32 | 0 replies | |
| 4/11 | gfs2: Turn gl_delete into a delayed workfs/ fs/ fs/ +2 more | +11 | 0 replies | |
| 5/11 | gfs2: Give up the iopen glock on contentionfs/ fs/ fs/ | +58 | 0 replies | |
| 6/11 | gfs2: Try harder to delete inodes locallyfs/ | +47 | 0 replies | |
| 7/11 | gfs2: Minor gfs2_lookup_by_inum cleanupfs/ fs/ fs/ +1 more | +14 | 0 replies | |
| 8/11 | gfs2: Move inode generation number check into gfs2_inode_lookupfs/ | +18 | 0 replies | |
| 8/11 | gfs2: Move inode generation number check into gfs2_inode_lookupfs/ | +6 | 0 replies | |
| 9/11 | gfs2: Check inode generation number in delete_work_funcfs/ fs/ | +7 | 0 replies | |
| 10/11 | gfs2: Wake up when setting GLF_DEMOTEfs/ | +14 | 0 replies | |
| 11/11 | gfs2: Smarter iopen glock waitingfs/ fs/ | +40 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [Cluster-devel] [PATCH 00/11] gfs2: iopen glock locking scheme rework
This patch queue reworks the iopen glock locking scheme to avoid resource group glock bouncing and cache thrashing when inodes are deleted that have been accessed on other cluster nodes.
When the link count of an inode drops to zero, the inode is marked GFS2_BLKST_UNLINKED in the bitmap. Before the inode can be deleted, we need to make sure that it's no longer in use locally or on any other cluster node. We use the inode reference count to keep track of local users, and the iopen glock to keep track of other users across the
cluster: all nodes that have an inode cached or in active use keep a shared lock on the iopen glock. When deleting an inode, the iopen glock is promoted to an exclusive lock first. Other nodes may keep the inode open, so there's no guarantee that we'll get that exclusive lock in definite time.
To work around that, when a node fails to get that exclusive lock, it will rely on the other nodes to perform the delete. The delete will happen eventually, on the first node that obtains the iopen glock exclusively, leading to resource group glock bouncing and the associated loss of caching. Note that the inode doesn't have to be in active use on multiple nodes at the same time to trigger that behavior; all it takes is to delete the inode on another node that the one on which it was created.
This patch queue reworks this locking scheme in the following ways: when trying to obtain the iopen glock exclusively, nodes will give the other nodes holding the inode cached a chance to drop the inode and release their shared lock. When a node cannot drop the inode immeidately in response to such a request (because the inode is held open), it will signal that to the cluster by causing activity on the associated inode glock.
Because we cannot rely on existing nodes to cooperate in this new scheme, after giving up a shared iopen glock, we need to verify that the inode was deleted eventually. This would normally require accessing the resource group, negating any benefits we've had so far, but we can remember that the inode was deleted in the inode glock lock value block (LVB) to get around that.
Thanks,
Andreas
Andreas Gruenbacher (10):
gfs2: Don't add glocks to the LRU while still in use
gfs2: Keep track of deletes in inode LVBs
gfs2: Turn gl_delete into a delayed work
gfs2: Give up the iopen glock on contention
gfs2: Try harder to delete inodes locally
gfs2: Minor gfs2_lookup_by_inum cleanup
gfs2: Move inode generation number check into gfs2_inode_lookup
gfs2: Check inode generation number in delete_work_func
gfs2: Wake up when setting GLF_DEMOTE
gfs2: Smarter iopen glock waiting
Bob Peterson (1):
gfs2: Allow ASPACE glocks to also have an lvb
fs/gfs2/export.c | 4 +- fs/gfs2/glock.c | 137 ++++++++++++++++++++++++++++--- fs/gfs2/glock.h | 4 +- fs/gfs2/glops.c | 5 +- fs/gfs2/incore.h | 8 +- fs/gfs2/inode.c | 34 ++++++-- fs/gfs2/inode.h | 2 +- fs/gfs2/rgrp.c | 3 +- fs/gfs2/super.c | 72 +++++++++++++--- include/uapi/linux/gfs2_ondisk.h | 6 ++ 10 files changed, 237 insertions(+), 38 deletions(-)
Signature · 1 line
2.20.1