NFSD duplicate reply cache optimizations
Subsystems (2)
Chuck Lever
maintainercover NFSD duplicate reply cache optimizations
From: Chuck Lever <[email protected]>
Two minor optimizations for the NFSD duplicate reply cache. These showed slight improvement in WRITE throughput on exported fast file systems.
Chuck Lever (2):
NFSD: Delay adding new entries to LRU
NFSD: Reduce DRC bucket size
fs/nfsd/nfscache.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)
Signature · 1 line
2.50.0
Chuck Lever
maintainerPATCH 1/2 NFSD: Delay adding new entries to LRU
From: Chuck Lever <[email protected]>
Neil Brown observes:
Quoted 3 lines
I would not include
RC_INPROGentries in the lru at all - they are always ignored, and will be added when they are switched toRCU_DONE.
I also removed a stale comment.
--- fs/nfsd/nfscache.c | 6 ------ 1 file changed, 6 deletions(-)
fs/nfsd/nfscache.c +0 -6
@@ -237,10 +237,6 @@ void nfsd_reply_cache_shutdown(struct nfsd_net *nn) } -/* -* Move cache entry to end of LRU list, and queue the cleaner to run if it's -* not already scheduled. -*/ static void lru_put_end (struct nfsd_drc_bucket *b,struct nfsd_cacherep *rp) {
@@ -453,8 +449,6 @@ nfsd_cache_insert(struct nfsd_drc_bucket *b, struct nfsd_cacherep *key, nn->longest_chain_cachesize, atomic_read(&nn->num_drc_entries)); } - -lru_put_end (b, ret);return ret; }
Signature · 1 line
2.50.0
Re: [PATCH v1 1/2] NFSD: Delay adding new entries to LRU
Quoted 31 lines · Chuck Lever
From: Chuck Lever <[email protected]>
Neil Brown observes:
Quoted 3 lines
I would not include
RC_INPROGentries in the lru at all - they are always ignored, and will be added when they are switched toRCU_DONE.I also removed a stale comment.
Suggested-by NeilBrown <[email protected]>Signed-off-by Chuck Lever <[email protected]>--- fs/nfsd/nfscache.c | 6 ------ 1 file changed, 6 deletions(-)fs/nfsd/nfscache.c +0
-4@@ -237,10 +237,6 @@ void nfsd_reply_cache_shutdown(struct nfsd_net *nn) } -/* -* Move cache entry to end of LRU list, and queue the cleaner to run if it's -* not already scheduled. -*/ static void lru_put_end (struct nfsd_drc_bucket *b,struct nfsd_cacherep *rp) {
fs/nfsd/nfscache.c nfsd_cache_insert(struct nfsd_drc_bucket *b, struct nfsd_cacherep *key, · line 452
Quoted 6 lines
fs/nfsd/nfscache.c +0
-2@@ -453,8 +449,6 @@ nfsd_cache_insert(struct nfsd_drc_bucket *b, struct nfsd_cacherep *key, nn->longest_chain_cachesize, atomic_read(&nn->num_drc_entries)); } - -lru_put_end (b, ret);return ret;
A result of this change is that entries in the lru never have
->c_state == RC_INPROG
They are always RC_DONE.
So this can be added to the patch:
fs/nfsd/nfscache.c +0 -6
@@ -272,12 +272,6 @@ nfsd_prune_bucket_locked(struct nfsd_net *nn, struct nfsd_drc_bucket *b,/* The bucket LRU is ordered oldest-first. */ list_for_each_entry_safe (rp, tmp, &b->lru_head, c_lru) { -/* -* Don't free entries attached to calls that are still -* in-progress, but do keep scanning the list. -*/ -if (rp->c_state ==RC_INPROG ) -continue ;if (atomic_read (&nn->num_drc_entries) <= nn->max_drc_entries &&time_before (expiry, rp->c_timestamp))
With that added:
Thanks,
NeilBrown
Quoted 6 lines
fs/nfsd/nfscache.c +0
-0}Signature · 1 line
2.50.0
Chuck Lever
maintainerRe: [PATCH v1 1/2] NFSD: Delay adding new entries to LRU
Quoted 50 lines · NeilBrown
Quoted 31 lines · Chuck Lever
From: Chuck Lever <[email protected]>
Neil Brown observes:
Quoted 3 lines
I would not include
RC_INPROGentries in the lru at all - they are always ignored, and will be added when they are switched toRCU_DONE.I also removed a stale comment.
Suggested-by NeilBrown <[email protected]>Signed-off-by Chuck Lever <[email protected]>--- fs/nfsd/nfscache.c | 6 ------ 1 file changed, 6 deletions(-)fs/nfsd/nfscache.c +0
-4@@ -237,10 +237,6 @@ void nfsd_reply_cache_shutdown(struct nfsd_net *nn) } -/* -* Move cache entry to end of LRU list, and queue the cleaner to run if it's -* not already scheduled. -*/ static void lru_put_end (struct nfsd_drc_bucket *b,struct nfsd_cacherep *rp) {fs/nfsd/nfscache.c nfsd_cache_insert(struct nfsd_drc_bucket *b, struct nfsd_cacherep *key, · line 452
Quoted 6 lines
fs/nfsd/nfscache.c +0
-2@@ -453,8 +449,6 @@ nfsd_cache_insert(struct nfsd_drc_bucket *b, struct nfsd_cacherep *key, nn->longest_chain_cachesize, atomic_read(&nn->num_drc_entries)); } - -lru_put_end (b, ret);return ret;A result of this change is that entries in the lru never have
->c_state == RC_INPROGThey are always
RC_DONE.So this can be added to the patch:
fs/nfsd/nfscache.c +0
-0@@ -272,12 +272,6 @@ nfsd_prune_bucket_locked(struct nfsd_net *nn, struct nfsd_drc_bucket *b,/* The bucket LRU is ordered oldest-first. */
fs/nfsd/nfscache.c nfsd_prune_bucket_locked(struct nfsd_net *nn, struct nfsd_drc_bucket *b, · line 277
Quoted 10 lines
fs/nfsd/nfscache.c +0
-6list_for_each_entry_safe (rp, tmp, &b->lru_head, c_lru) { -/* -* Don't free entries attached to calls that are still -* in-progress, but do keep scanning the list. -*/ -if (rp->c_state ==RC_INPROG ) -continue ;if (atomic_read (&nn->num_drc_entries) <= nn->max_drc_entries &&time_before (expiry, rp->c_timestamp))
I didn't add this bit because I couldn't convince myself that it was impossible for an entry in RC_INPROG state to get into the LRU somehow. But if you think it is not possible, I'll buy that and add this snippet.
Quoted 15 lines
With that added:
Reviewed-by NeilBrown <[email protected]>Thanks,
NeilBrownQuoted 6 lines
fs/nfsd/nfscache.c +0
-0}Signature · 1 line
2.50.0
Signature · 1 line
Chuck Lever
Chuck Lever
maintainerPATCH 2/2 NFSD: Reduce DRC bucket size
In reply to Chuck LeverFrom: Chuck Lever <[email protected]>
The common case is that a DRC lookup will not find the XID in the bucket. Reduce the amount of pointer chasing during the lookup by keeping fewer entries in each hash bucket.
Changing the bucket size constant forces the size of the DRC hash table to increase, and the height of each bucket r-b tree to be reduced.
--- fs/nfsd/nfscache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/nfsd/nfscache.c +1 -1
@@ -27,7 +27,7 @@* cache size, the idea being that when the cache is at its maximum number * of entries, then this should be the average number of entries per bucket. */ -#define TARGET_BUCKET_SIZE 64 +#define TARGET_BUCKET_SIZE 8struct nfsd_drc_bucket {struct rb_root rb_head;
Signature · 1 line
2.50.0
Quoted 6 lines · Chuck Lever
From: Chuck Lever <[email protected]>
Two minor optimizations for the NFSD duplicate reply cache. These showed slight improvement in WRITE throughput on exported fast file systems.
Quoted 6 lines
Chuck Lever (2):
NFSD: Delay adding new entries to LRU
NFSD: Reduce DRC bucket sizefs/nfsd/nfscache.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)