mm/zswap: Improve with alloc_workqueue() call
Subsystems (2)
PATCH 1/1 mm/zswap: Improve with alloc_workqueue() call
Use alloc_workqueue() to create and set finer work item attributes instead of create_workqueue() which is to be deprecated.
--- mm/zswap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
mm/zswap.c +2 -1
@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; } - shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,0 );if (!shrink_wq) goto fallback_fail;
Signature · 1 line
2.34.1
Nhat Pham
maintainerQuoted 15 lines · Ronald Monthero
Use
alloc_workqueue()to create and set finer work item attributes instead ofcreate_workqueue()which is to be deprecated.Signed-off-by Ronald Monthero <[email protected]>--- mm/zswap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/zswap.c +0
-0
mm/zswap.c static int zswap_setup(void) · line 1624
Quoted 6 lines
mm/zswap.c +2
-1@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; } - shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,0 );
Hmmm this changes the current behavior a bit right? create_workqueue() is currently defined as:
alloc_workqueue("%s", __WQ_LEGACY | WQ_MEM_RECLAIM, 1, (name))
I think this should be noted in the changelog, at the very least, even if it is fine. We should be as explicit as possible about behavior changes.
Quoted 7 lines
mm/zswap.c +0
-1if (!shrink_wq) goto fallback_fail; --2.34.1
Hi Nhat,
Thanks for checking.
Quoted 21 lines · Nhat Pham
Quoted 18 lines · Ronald Monthero
Use
alloc_workqueue()to create and set finer work item attributes instead ofcreate_workqueue()which is to be deprecated.Signed-off-by Ronald Monthero <[email protected]>--- mm/zswap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/zswap.c +0
-0@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; }
Quoted 8 lines
mm/zswap.c static int zswap_setup(void) · line 1624
Quoted 3 lines
mm/zswap.c +2
-1- shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,0 );Hmmm this changes the current behavior a bit right?
create_workqueue()is currently defined as:
alloc_workqueue("%s", __WQ_LEGACY | WQ_MEM_RECLAIM, 1, (name))
create_workqueue is deprecated and it's observed that most of the subsystems have changed to using alloc_workqueue. So it's a small minority of few remnant instances in the kernel and some drivers still using create_workqueue. With the create_workqueue defined as is , it hardcodes the workqueue flags to be per cpu and unbound in nature and not giving the flexibility of using any other wait queue flags/attributes. ( WQ_CPU_INTENSIVE, WQ_HIGHPRI, WQ_RESCUER, WQ_FREEZEABLE, WQ_UNBOUND) . Hence most of the subsystems and drivers use the alloc_workqueue( ) api.
#define create_workqueue(name) \alloc_workqueue("%s", __WQ_LEGACY | WQ_MEM_RECLAIM, 1, (name))
Quoted 4 lines
I think this should be noted in the changelog, at the very least, even if it is fine. We should be as explicit as possible about behavior changes.
imo, it's sort of known and consistently changed for quite some time already.
https://lists.openwall.net/linux-kernel/2016/06/07/1086
https://lists.openwall.net/linux-kernel/2011/01/03/124
https://lwn.net/Articles/403891/ => quoted: "The long-term plan, it
seems, is to convert all create_workqueue() users over to an appropriate alloc_workqueue() call; eventually create_workqueue() will be removed"
glad to take some suggestions , thoughts ?
BR,
ronald
Nhat Pham
maintainerQuoted 58 lines · Ronald Monthero
Hi Nhat,
Thanks for checking.Quoted 21 lines · Nhat Pham
Quoted 18 lines · Ronald Monthero
Use
alloc_workqueue()to create and set finer work item attributes instead ofcreate_workqueue()which is to be deprecated.Signed-off-by Ronald Monthero <[email protected]>--- mm/zswap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/zswap.c +0
-0@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; }Quoted 8 lines
mm/zswap.c static int zswap_setup(void) · line 1624
Quoted 3 lines
mm/zswap.c +2
-1- shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,0 );Hmmm this changes the current behavior a bit right?
create_workqueue()is currently defined as:
alloc_workqueue("%s", __WQ_LEGACY | WQ_MEM_RECLAIM, 1, (name))
create_workqueueis deprecated and it's observed that most of the subsystems have changed to usingalloc_workqueue. So it's a small minority of few remnant instances in the kernel and some drivers still usingcreate_workqueue. With thecreate_workqueuedefined as is , it hardcodes the workqueue flags to be per cpu and unbound in nature and not giving the flexibility of using any other wait queue flags/attributes. (WQ_CPU_INTENSIVE,WQ_HIGHPRI,WQ_RESCUER,WQ_FREEZEABLE,WQ_UNBOUND) . Hence most of the subsystems and drivers use thealloc_workqueue( )api.
#definecreate_workqueue(name)\alloc_workqueue("%s", __WQ_LEGACY | WQ_MEM_RECLAIM, 1, (name))Quoted 4 lines
I think this should be noted in the changelog, at the very least, even if it is fine. We should be as explicit as possible about behavior changes.
imo, it's sort of known and consistently changed for quite some time already.
https://lists.openwall.net/linux-kernel/2016/06/07/1086
https://lists.openwall.net/linux-kernel/2011/01/03/124
https://lwn.net/Articles/403891/ => quoted: "The long-term plan, it
seems, is to convert allcreate_workqueue()users over to an appropriatealloc_workqueue()call; eventuallycreate_workqueue()will be removed"
Quoted 4 lines
glad to take some suggestions , thoughts ?
BR,
ronald
I should have been clearer. I'm not against the change per-se - I agree that we should replace create_workqueue() with alloc_workqueue(). What I meant was, IIUC, there are two behavioral changes with this new workqueue creation:
a) We're replacing a bounded workqueue (which as you noted, is fixed by create_workqueue()) with an unbounded one (WQ_UNBOUND). This seems fine to me - I doubt locality buys us much here.
b) create_workqueue() limits the number of concurrent per-cpu execution contexts at 1 (i.e only one single global reclaimer), whereas after this patch this is set to the default value. This seems fine to me too - I don't remember us taking advantage of the previous concurrency limitation. Also, in practice, the task_struct is one-to-one with the zswap_pool's anyway, and most of the time, there is just a single pool being used. (But it begs the question - what's the point of using 0 instead of 1 here?)
Both seem fine (to me anyway - other reviewers feel free to take a look and fact-check everything). I just feel like this should be explicitly noted in the changelog, IMHO, in case we are mistaken and need to revisit this :) Either way, not a NACK from me.
Nhat Pham
maintainerQuoted 2 lines · Nhat Pham
concurrency limitation. Also, in practice, the
task_structis
/s/task/work.
I was looking at some articles about tasks recently, so my brain just auto-completed. I was referring to pool->shrink_work here.
Quoted 1 line · Nhat Pham
..
< snipped >
Quoted 8 lines
I should have been clearer. I'm not against the change per-se - I agree that we should replace
create_workqueue()withalloc_workqueue(). What I meant was, IIUC, there are two behavioral changes with this new workqueue creation:a) We're replacing a bounded workqueue (which as you noted, is fixed by
create_workqueue()) with an unbounded one (WQ_UNBOUND). This seems fine to me - I doubt locality buys us much here.
Yes the workqueue attribute change per se but the existing functionality remains seamless and the change is more a forward looking change. imo under a memory pressure scenario an unbound workqueue might workaround the scenario better as the number of backing pools is dynamic. And with the WQ_UNBOUND attribute the scheduler is more open to exercise some improvisations in any demanding scenarios for offloading cpu time slicing for workers, ie if any other worker of the same primary cpu had to be served due to workers with WQ_HIGHPRI and WQ_CPU_INTENSIVE.Although normal and highpri worker-pools don't interact with each other, the target cpu atleast need not be the same if our worker for zswap is WQ_UNBOUND.
Also noting that the existing wq of zwap worker has the WQ_MEM_RECLAIM attribute, so is there a rescue worker for zswap during a memory pressure scenario ?
Quoting: "All work items which might be used on code paths that handle memory reclaim are required to be queued on wq's that have a rescue-worker reserved for execution under memory pressure. Else it is possible that the worker-pool deadlocks waiting for execution contexts to free up"
Also additional thought if adding WQ_FREEZABLE attribute while creating the zswap worker make sense in scenarios to handle freeze and unfreeze of specific cgroups or file system wide freeze and unfreeze scenarios ? Does zswap worker participate in freeze/unfreeze code path scenarios ?
Quoted 8 lines
b)
create_workqueue()limits the number of concurrent per-cpu execution contexts at 1 (i.e only one single global reclaimer), whereas after this patch this is set to the default value. This seems fine to me too - I don't remember us taking advantage of the previous concurrency limitation. Also, in practice, thetask_structis one-to-one with the zswap_pool's anyway, and most of the time, there is just a single pool being used. (But it begs the question - what's the point of using 0 instead of 1 here?)
Nothing in particular but I left it at default 0, which can go upto 256 ( @maxactive per cpu).
But if zswap worker is always intended to only have 1 active worker per cpu, then that's fine with 1, otherwise a default setting might be flexible for scaling.
just a thought, does having a higher value help for larger memory systems ?
Quoted 4 lines
Both seem fine (to me anyway - other reviewers feel free to take a look and fact-check everything). I just feel like this should be explicitly noted in the changelog, IMHO, in case we are mistaken and need to revisit this :) Either way, not a NACK from me.
Thanks Nhat, for checking. Above are my thoughts, I could be missing some info or incorrect
on certain fronts so I would seek clarifications.
Also thanks in advance to other experts/maintainers, please share feedback and suggestions.
BR,
ronald
Nhat Pham
maintainerQuoted 15 lines · Ronald Monthero
Quoted 1 line · Nhat Pham
..
< snipped >Quoted 8 lines
I should have been clearer. I'm not against the change per-se - I agree that we should replace
create_workqueue()withalloc_workqueue(). What I meant was, IIUC, there are two behavioral changes with this new workqueue creation:a) We're replacing a bounded workqueue (which as you noted, is fixed by
create_workqueue()) with an unbounded one (WQ_UNBOUND). This seems fine to me - I doubt locality buys us much here.
Quoted 11 lines
Yes the workqueue attribute change per se but the existing functionality remains seamless and the change is more a forward looking change. imo under a memory pressure scenario an unbound workqueue might workaround the scenario better as the number of backing pools is dynamic. And with the
WQ_UNBOUNDattribute the scheduler is more open to exercise some improvisations in any demanding scenarios for offloading cpu time slicing for workers, ie if any other worker of the same primary cpu had to be served due to workers withWQ_HIGHPRIand WQ_CPU_INTENSIVE.Although normal and highpri worker-pools don't interact with each other, the target cpu atleast need not be the same if our worker for zswap isWQ_UNBOUND.
I don't object to the change per-se. I just meant that these changes/discussion should be spelled out in the patch's changelog :) IMHO, we should document behavior changes if there are any. For instance, when we switched to kmap_local_page() for zswap, there is a discussion in the changelog regarding how it differs from the old (and deprecated) kmap_atomic():
https://lore.kernel.org/linux-mm/[email protected]/
and how that difference doesn't matter in the case of zswap.
Quoted 9 lines
Also noting that the existing wq of zwap worker has the
WQ_MEM_RECLAIMattribute, so is there a rescue worker for zswap during a memory pressure scenario ?
Quoting: "All work items which might be used on code paths that handle memory reclaim are required to be queued on wq's that have a rescue-worker reserved for execution under memory pressure. Else it is possible that the worker-pool deadlocks waiting for execution contexts to free up"
Seems like it, but this behavior is not changed by your patch :) So i'm not too concerned by it one way or another.
Quoted 6 lines
Also additional thought if adding
WQ_FREEZABLEattribute while creating the zswap worker make sense in scenarios to handle freeze and unfreeze of specific cgroups or file system wide freeze and unfreeze scenarios ? Does zswap worker participate in freeze/unfreeze code path scenarios ?
I don't think so, no? This zswap worker is scheduled upon the zswap pool limit hit (which happens on the zswap store/swapping/memory reclaim) path.
Quoted 10 lines
Quoted 9 lines
b)
create_workqueue()limits the number of concurrent per-cpu execution contexts at 1 (i.e only one single global reclaimer), whereas after this patch this is set to the default value. This seems fine to me too - I don't remember us taking advantage of the previous concurrency limitation. Also, in practice, thetask_structis one-to-one with the zswap_pool's anyway, and most of the time, there is just a single pool being used. (But it begs the question - what's the point of using 0 instead of 1 here?)
Quoted 6 lines
Nothing in particular but I left it at default 0, which can go upto 256 ( @maxactive per cpu).
But if zswap worker is always intended to only have 1 active worker per cpu, then that's fine with 1, otherwise a default setting might be flexible for scaling.
just a thought, does having a higher value help for larger memory systems ?
I don't think having higher value helps here tbh. We only have one work_struct per pool, so it shouldn't make a difference either way :)
Quoted 6 lines
Quoted 5 lines
Both seem fine (to me anyway - other reviewers feel free to take a look and fact-check everything). I just feel like this should be explicitly noted in the changelog, IMHO, in case we are mistaken and need to revisit this :) Either way, not a NACK from me.
Quoted 8 lines
Thanks Nhat, for checking. Above are my thoughts, I could be missing some info or incorrect
on certain fronts so I would seek clarifications.
Also thanks in advance to other experts/maintainers, please share feedback and suggestions.BR,
ronald
Also +Chris Li, who is also working on improving zswap :)
PATCH 1/1 mm/zswap: Improve with alloc_workqueue() call
The core-api create_workqueue is deprecated, this patch replaces the create_workqueue with alloc_workqueue. The previous implementation workqueue of zswap was a bounded workqueue, this patch uses alloc_workqueue() to create an unbounded workqueue. The WQ_UNBOUND attribute is desirable making the workqueue not localized to a specific cpu so that the scheduler is free to exercise improvisations in any demanding scenarios for offloading cpu time slices for workqueues.
For example if any other workqueues of the same primary cpu had to be served which are WQ_HIGHPRI and WQ_CPU_INTENSIVE. Also Unbound workqueue happens to be more efficient
in a system during memory pressure scenarios in comparison
to a bounded workqueue.
shrink_wq = alloc_workqueue("zswap-shrink",
WQ_UNBOUND|WQ_MEM_RECLAIM, 1);
Overall the change suggested in this patch should be seamless and does not alter the existing behavior, other than the improvisation to be an unbounded workqueue.
--- mm/zswap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
mm/zswap.c +2 -1
@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; } - shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,1 );if (!shrink_wq) goto fallback_fail;
Signature · 1 line
2.34.1
Nhat Pham
maintainerOn Tue, Jan 16, 2024 at 5:32 AM Ronald Monthero
<[email protected]> wrote:
+ Johannes and Yosry
Quoted 9 lines
The core-api
create_workqueueis deprecated, this patch replaces thecreate_workqueuewithalloc_workqueue. The previous implementation workqueue of zswap was a bounded workqueue, this patch usesalloc_workqueue()to create an unbounded workqueue. TheWQ_UNBOUNDattribute is desirable making the workqueue not localized to a specific cpu so that the scheduler is free to exercise improvisations in any demanding scenarios for offloading cpu time slices for workqueues.
nit: extra space between paragraph would be nice.
Quoted 23 lines
For example if any other workqueues of the same primary cpu had to be served which are
WQ_HIGHPRIandWQ_CPU_INTENSIVE. Also Unbound workqueue happens to be more efficient in a system during memory pressure scenarios in comparison
to a bounded workqueue.
shrink_wq= alloc_workqueue("zswap-shrink",WQ_UNBOUND|WQ_MEM_RECLAIM, 1);Overall the change suggested in this patch should be seamless and does not alter the existing behavior, other than the improvisation to be an unbounded workqueue.
Signed-off-by Ronald Monthero <[email protected]>--- mm/zswap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/zswap.c +0
-0
mm/zswap.c static int zswap_setup(void) · line 1624
Quoted 6 lines
mm/zswap.c +2
-1@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; } - shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,1 );
Have you benchmarked this to check if there is any regression, just to be safe? With an unbounded workqueue, you're gaining scheduling flexibility at the cost of cache locality. My intuition is that it doesn't matter too much here, but you should probably double check by stress testing - run some workload with a relatively small zswap pool limit (i.e heavy global writeback), and see if there is any difference in performance.
Quoted 6 lines
mm/zswap.c +0
-1if (!shrink_wq) goto fallback_fail; --2.34.1
On a different note, I wonder if it would help to perform synchronous reclaim here instead. With our current design, the zswap store failure (due to global limit hit) would leave the incoming page going to swap instead, creating an LRU inversion. Not sure if that's ideal.
Quoted 44 lines · Nhat Pham
On Tue, Jan 16, 2024 at 5:32 AM Ronald Monthero
<[email protected]> wrote:+ Johannes and Yosry
Quoted 9 lines
The core-api
create_workqueueis deprecated, this patch replaces thecreate_workqueuewithalloc_workqueue. The previous implementation workqueue of zswap was a bounded workqueue, this patch usesalloc_workqueue()to create an unbounded workqueue. TheWQ_UNBOUNDattribute is desirable making the workqueue not localized to a specific cpu so that the scheduler is free to exercise improvisations in any demanding scenarios for offloading cpu time slices for workqueues.nit: extra space between paragraph would be nice.
Quoted 26 lines
For example if any other workqueues of the same primary cpu had to be served which are
WQ_HIGHPRIandWQ_CPU_INTENSIVE. Also Unbound workqueue happens to be more efficient in a system during memory pressure scenarios in comparison
to a bounded workqueue.
shrink_wq= alloc_workqueue("zswap-shrink",WQ_UNBOUND|WQ_MEM_RECLAIM, 1);Overall the change suggested in this patch should be seamless and does not alter the existing behavior, other than the improvisation to be an unbounded workqueue.
Signed-off-by Ronald Monthero <[email protected]>--- mm/zswap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/zswap.c +0
-0@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; }
Quoted 11 lines
mm/zswap.c static int zswap_setup(void) · line 1624
Quoted 3 lines
mm/zswap.c +2
-1- shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,1 );Have you benchmarked this to check if there is any regression, just to be safe? With an unbounded workqueue, you're gaining scheduling flexibility at the cost of cache locality. My intuition is that it doesn't matter too much here, but you should probably double check by stress testing - run some workload with a relatively small zswap pool limit (i.e heavy global writeback), and see if there is any difference in performance.
I also think this shouldn't make a large difference. The global shrinking work is already expensive, and I imagine that it exhausts the caches anyway by iterating memcgs. A performance smoketest would be reassuring for sure, but I believe it won't make a difference.
Keep in mind that even with WQ_UNBOUND, we prefer the local CPU (see wq_select_unbound_cpu()), so it will take more than global writeback to observe a difference. The local CPU must not be in wq_unbound_cpumask, or CONFIG_DEBUG_WQ_FORCE_RR_CPU should be on.
Quoted 4 lines
Quoted 4 lines
mm/zswap.c +0
-0if (!shrink_wq) goto fallback_fail;
Quoted 8 lines
Quoted 3 lines
--
2.34.1On a different note, I wonder if it would help to perform synchronous reclaim here instead. With our current design, the zswap store failure (due to global limit hit) would leave the incoming page going to swap instead, creating an LRU inversion. Not sure if that's ideal.
The global shrink path keeps reclaiming until zswap can accept again (by default, that means reclaiming 10% of the total limit). I think this is too expensive to be done synchronously.
Johannes Weiner
maintainerQuoted 57 lines · Yosry Ahmed
Quoted 44 lines · Nhat Pham
On Tue, Jan 16, 2024 at 5:32 AM Ronald Monthero
<[email protected]> wrote:+ Johannes and Yosry
Quoted 9 lines
The core-api
create_workqueueis deprecated, this patch replaces thecreate_workqueuewithalloc_workqueue. The previous implementation workqueue of zswap was a bounded workqueue, this patch usesalloc_workqueue()to create an unbounded workqueue. TheWQ_UNBOUNDattribute is desirable making the workqueue not localized to a specific cpu so that the scheduler is free to exercise improvisations in any demanding scenarios for offloading cpu time slices for workqueues.nit: extra space between paragraph would be nice.
Quoted 26 lines
For example if any other workqueues of the same primary cpu had to be served which are
WQ_HIGHPRIandWQ_CPU_INTENSIVE. Also Unbound workqueue happens to be more efficient in a system during memory pressure scenarios in comparison
to a bounded workqueue.
shrink_wq= alloc_workqueue("zswap-shrink",WQ_UNBOUND|WQ_MEM_RECLAIM, 1);Overall the change suggested in this patch should be seamless and does not alter the existing behavior, other than the improvisation to be an unbounded workqueue.
Signed-off-by Ronald Monthero <[email protected]>--- mm/zswap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/zswap.c +0
-0@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; }Quoted 11 lines
mm/zswap.c static int zswap_setup(void) · line 1624
Quoted 3 lines
mm/zswap.c +2
-1- shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,1 );Have you benchmarked this to check if there is any regression, just to be safe? With an unbounded workqueue, you're gaining scheduling flexibility at the cost of cache locality. My intuition is that it doesn't matter too much here, but you should probably double check by stress testing - run some workload with a relatively small zswap pool limit (i.e heavy global writeback), and see if there is any difference in performance.
Quoted 4 lines
I also think this shouldn't make a large difference. The global shrinking work is already expensive, and I imagine that it exhausts the caches anyway by iterating memcgs. A performance smoketest would be reassuring for sure, but I believe it won't make a difference.
The LRU inherently makes the shrinker work on the oldest and coldest entries, so I doubt we benefit a lot from cache locality there.
What could make a difference though is the increased concurrency by switching max_active from 1 to 0. This could cause a higher rate of shrinker runs, which might increase lock contention and reclaim volume. That part would be good to double check with the shrinker benchmarks.
Quoted 8 lines
Quoted 4 lines
On a different note, I wonder if it would help to perform synchronous reclaim here instead. With our current design, the zswap store failure (due to global limit hit) would leave the incoming page going to swap instead, creating an LRU inversion. Not sure if that's ideal.
The global shrink path keeps reclaiming until zswap can accept again (by default, that means reclaiming 10% of the total limit). I think this is too expensive to be done synchronously.
That thresholding code is a bit weird right now.
It wakes the shrinker and rejects at the same time. We're guaranteed to see rejections, even if the shrinker has no trouble flushing some entries a split second later.
It would make more sense to wake the shrinker at e.g. 95% full and have it run until 90%.
But with that in place we also *should* do synchronous reclaim once we hit 100%. Just enough to make room for the store. This is important to catch the case where reclaim rate exceeds swapout rate. Rejecting and going to swap means the reclaimer will be throttled down to IO rate anyway, and the app latency isn't any worse. But this way we keep the pipeline alive, and keep swapping out the oldest zswap entries, instead of rejecting and swapping what would be the hottest ones.
Quoted 5 lines · Johannes Weiner
Quoted 5 lines
Quoted 5 lines
On Tue, Jan 16, 2024 at 5:32 AM Ronald Monthero
Quoted 4 lines
mm/zswap.c +0
-0@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; }
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
mm/zswap.c static int zswap_setup(void) · line 1624
Quoted 3 lines
mm/zswap.c +2
-1- shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,1 );What could make a difference though is the increased concurrency by switching
max_activefrom 1 to 0. This could cause a higher rate of shrinker runs, which might increase lock contention and reclaim volume. That part would be good to double check with the shrinker benchmarks.
Nevermind, I clearly can't read.
Could still be worthwhile testing with the default 0, but it's not a concern in the patch as-is.
Quoted 17 lines · Johannes Weiner
Quoted 5 lines · Johannes Weiner
Quoted 5 lines
Quoted 5 lines
On Tue, Jan 16, 2024 at 5:32 AM Ronald Monthero
Quoted 4 lines
mm/zswap.c +0
-0@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; }Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
mm/zswap.c static int zswap_setup(void) · line 1624
Quoted 3 lines
mm/zswap.c +2
-1- shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,1 );What could make a difference though is the increased concurrency by switching
max_activefrom 1 to 0. This could cause a higher rate of shrinker runs, which might increase lock contention and reclaim volume. That part would be good to double check with the shrinker benchmarks.
Quoted 1 line
Nevermind, I clearly can't read.
Regardless of max_active, we only have one shrink_work per zswap pool, and we can only have one instance of the work running at any time, right?
Quoted 6 lines
Could still be worthwhile testing with the default 0, but it's not a concern in the patch as-is.
Acked-by Johannes Weiner <[email protected]>
Nhat Pham
maintainerQuoted 19 lines · Yosry Ahmed
Quoted 17 lines · Johannes Weiner
Quoted 14 lines · Johannes Weiner
Quoted 8 lines
Quoted 8 lines
On Tue, Jan 16, 2024 at 5:32 AM Ronald Monthero
mm/zswap.c static int zswap_setup(void) · line 1624
Quoted 7 lines
mm/zswap.c +2
-1@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; } - shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,1 );What could make a difference though is the increased concurrency by switching
max_activefrom 1 to 0. This could cause a higher rate of shrinker runs, which might increase lock contention and reclaim volume. That part would be good to double check with the shrinker benchmarks.
Quoted 5 lines
Quoted 1 line
Nevermind, I clearly can't read.
Regardless of
max_active, we only have oneshrink_workper zswap pool, and we can only have one instance of the work running at any time, right?
I believe so, yeah. Well I guess you can have a weird setup where somehow multiple pools are full and submit shrink_work concurrently? But who does that :) But let's just keep it as is to reduce our mental workload (i.e not having to keep track of what changes) would be ideal.
Quoted 7 lines
Quoted 7 lines
Could still be worthwhile testing with the default 0, but it's not a concern in the patch as-is.
Acked-by Johannes Weiner <[email protected]>
Quoted 18 lines
Quoted 8 lines
Quoted 4 lines
On a different note, I wonder if it would help to perform synchronous reclaim here instead. With our current design, the zswap store failure (due to global limit hit) would leave the incoming page going to swap instead, creating an LRU inversion. Not sure if that's ideal.
The global shrink path keeps reclaiming until zswap can accept again (by default, that means reclaiming 10% of the total limit). I think this is too expensive to be done synchronously.
That thresholding code is a bit weird right now.
It wakes the shrinker and rejects at the same time. We're guaranteed to see rejections, even if the shrinker has no trouble flushing some entries a split second later.
It would make more sense to wake the shrinker at e.g. 95% full and have it run until 90%.
Quoted 7 lines
But with that in place we also *should* do synchronous reclaim once we hit 100%. Just enough to make room for the store. This is important to catch the case where reclaim rate exceeds swapout rate. Rejecting and going to swap means the reclaimer will be throttled down to IO rate anyway, and the app latency isn't any worse. But this way we keep the pipeline alive, and keep swapping out the oldest zswap entries, instead of rejecting and swapping what would be the hottest ones.
I fully agree with the thresholding code being weird, and with waking up the shrinker before the pool is full. What I don't understand is how we can do synchronous reclaim when we hit 100% and still respect the acceptance threshold :/
Are you proposing we change the semantics of the acceptance threshold to begin with?
Johannes Weiner
maintainerQuoted 26 lines · Yosry Ahmed
Quoted 18 lines
Quoted 8 lines
Quoted 4 lines
On a different note, I wonder if it would help to perform synchronous reclaim here instead. With our current design, the zswap store failure (due to global limit hit) would leave the incoming page going to swap instead, creating an LRU inversion. Not sure if that's ideal.
The global shrink path keeps reclaiming until zswap can accept again (by default, that means reclaiming 10% of the total limit). I think this is too expensive to be done synchronously.
That thresholding code is a bit weird right now.
It wakes the shrinker and rejects at the same time. We're guaranteed to see rejections, even if the shrinker has no trouble flushing some entries a split second later.
It would make more sense to wake the shrinker at e.g. 95% full and have it run until 90%.
Quoted 7 lines
But with that in place we also *should* do synchronous reclaim once we hit 100%. Just enough to make room for the store. This is important to catch the case where reclaim rate exceeds swapout rate. Rejecting and going to swap means the reclaimer will be throttled down to IO rate anyway, and the app latency isn't any worse. But this way we keep the pipeline alive, and keep swapping out the oldest zswap entries, instead of rejecting and swapping what would be the hottest ones.
Quoted 7 lines
I fully agree with the thresholding code being weird, and with waking up the shrinker before the pool is full. What I don't understand is how we can do synchronous reclaim when we hit 100% and still respect the acceptance threshold :/
Are you proposing we change the semantics of the acceptance threshold to begin with?
I kind of am. It's worth looking at the history of this knob.
It was added in 2020 by 45190f01dd402112d3d22c0ddc4152994f9e1e55, and from the changelogs and the code in this patch I do not understand how this was supposed to work.
It also *didn't* work for very basic real world applications. See Domenico's follow-up (e0228d590beb0d0af345c58a282f01afac5c57f3), which effectively reverted it to get halfway reasonable behavior.
If there are no good usecases for this knob, then I think it makes sense to phase it out again.
Quoted 42 lines · Johannes Weiner
Quoted 26 lines · Yosry Ahmed
Quoted 18 lines
Quoted 8 lines
Quoted 4 lines
On a different note, I wonder if it would help to perform synchronous reclaim here instead. With our current design, the zswap store failure (due to global limit hit) would leave the incoming page going to swap instead, creating an LRU inversion. Not sure if that's ideal.
The global shrink path keeps reclaiming until zswap can accept again (by default, that means reclaiming 10% of the total limit). I think this is too expensive to be done synchronously.
That thresholding code is a bit weird right now.
It wakes the shrinker and rejects at the same time. We're guaranteed to see rejections, even if the shrinker has no trouble flushing some entries a split second later.
It would make more sense to wake the shrinker at e.g. 95% full and have it run until 90%.
Quoted 7 lines
But with that in place we also *should* do synchronous reclaim once we hit 100%. Just enough to make room for the store. This is important to catch the case where reclaim rate exceeds swapout rate. Rejecting and going to swap means the reclaimer will be throttled down to IO rate anyway, and the app latency isn't any worse. But this way we keep the pipeline alive, and keep swapping out the oldest zswap entries, instead of rejecting and swapping what would be the hottest ones.
Quoted 7 lines
I fully agree with the thresholding code being weird, and with waking up the shrinker before the pool is full. What I don't understand is how we can do synchronous reclaim when we hit 100% and still respect the acceptance threshold :/
Are you proposing we change the semantics of the acceptance threshold to begin with?
I kind of am. It's worth looking at the history of this knob.
It was added in 2020 by
45190f01dd402112d3d22c0ddc4152994f9e1e55, and from the changelogs and the code in this patch I do not understand how this was supposed to work.
Quoted 6 lines
It also *didn't* work for very basic real world applications. See Domenico's follow-up (
e0228d590beb0d0af345c58a282f01afac5c57f3), which effectively reverted it to get halfway reasonable behavior.If there are no good usecases for this knob, then I think it makes sense to phase it out again.
I am always nervous about removing/altering user visible knobs, but if you think it's fine then I am all for it. I think it makes more sense to start writeback early to avoid the whole situation if possible, and synchronously reclaim a little bit if we hit 100%. I think the proactive writeback should reduce the amount of synchronous IO we need to do in reclaim as well, so we may see some latency improvements.
Nhat Pham
maintainerQuoted 11 lines · Johannes Weiner
Quoted 9 lines · Yosry Ahmed
Quoted 9 lines
Quoted 8 lines
Quoted 4 lines
On a different note, I wonder if it would help to perform synchronous reclaim here instead. With our current design, the zswap store failure (due to global limit hit) would leave the incoming page going to swap instead, creating an LRU inversion. Not sure if that's ideal.
The global shrink path keeps reclaiming until zswap can accept again (by default, that means reclaiming 10% of the total limit). I think this is too expensive to be done synchronously.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
That thresholding code is a bit weird right now.
It wakes the shrinker and rejects at the same time. We're guaranteed to see rejections, even if the shrinker has no trouble flushing some entries a split second later.
It would make more sense to wake the shrinker at e.g. 95% full and have it run until 90%.
Yep, we should be reclaiming zswap objects way ahead of the pool limit. Hence the new shrinker, which is memory pressure-driven (i.e independent of zswap internal limits), and will typically be triggered even if the pool is not full. During experiments, I never observe the pool becoming full, with the default settings. I'd be happy to extend it (or build in extra shrinking logic) to cover these pool limits too, if it turns out to be necessary.
Quoted 23 lines
Quoted 9 lines
Quoted 8 lines
But with that in place we also *should* do synchronous reclaim once we hit 100%. Just enough to make room for the store. This is important to catch the case where reclaim rate exceeds swapout rate. Rejecting and going to swap means the reclaimer will be throttled down to IO rate anyway, and the app latency isn't any worse. But this way we keep the pipeline alive, and keep swapping out the oldest zswap entries, instead of rejecting and swapping what would be the hottest ones.
Quoted 7 lines
I fully agree with the thresholding code being weird, and with waking up the shrinker before the pool is full. What I don't understand is how we can do synchronous reclaim when we hit 100% and still respect the acceptance threshold :/
Are you proposing we change the semantics of the acceptance threshold to begin with?
I kind of am. It's worth looking at the history of this knob.
It was added in 2020 by
45190f01dd402112d3d22c0ddc4152994f9e1e55, and from the changelogs and the code in this patch I do not understand how this was supposed to work.
Quoted 6 lines
It also *didn't* work for very basic real world applications. See Domenico's follow-up (
e0228d590beb0d0af345c58a282f01afac5c57f3), which effectively reverted it to get halfway reasonable behavior.If there are no good usecases for this knob, then I think it makes sense to phase it out again.
Yeah this was my original proposal - remove this knob altogether :) Based on a cursory read, it just seems like zswap was originally trying to shrink (synchronously) one "object", then try to check if the pool size is now under the limit. This is indeed insufficient. However, I'm not quite convinced by the solution (hysteresis) either.
Maybe we can synchronously shrink a la Domenico, i.e until the pool can accept new pages, but this time capacity-based (maybe under the limit + some headroom, 1 page for example)? This is just so that the immediate incoming zswap store succeeds - we can still have the shrinker freeing up space later on (or maybe keep an asynchronous pool-limit based shrinker around).
Thanks for the reviews.
This patch is available at
https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-zswap-improve-with-alloc_workqueue-call.patch
This patch will later appear in the mm-unstable branch at
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
The -mm tree is included into linux-next via the mm-everything branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
------------------------------------------------------
From: Ronald Monthero <[email protected]>
Subject: mm/zswap: improve with alloc_workqueue() call
Date: Tue, 16 Jan 2024 23:31:45 +1000
The core-api create_workqueue is deprecated, this patch replaces the create_workqueue with alloc_workqueue. The previous implementation workqueue of zswap was a bounded workqueue, this patch uses alloc_workqueue() to create an unbounded workqueue. The WQ_UNBOUND attribute is desirable making the workqueue not localized to a specific cpu so that the scheduler is free to exercise improvisations in any demanding scenarios for offloading cpu time slices for workqueues. For example if any other workqueues of the same primary cpu had to be served which are WQ_HIGHPRI and WQ_CPU_INTENSIVE. Also Unbound workqueue happens to be more efficient in a system during memory pressure scenarios in comparison to a bounded workqueue.
shrink_wq = alloc_workqueue("zswap-shrink",
WQ_UNBOUND|WQ_MEM_RECLAIM, 1);
Overall the change suggested in this patch should be seamless and does not alter the existing behavior, other than the improvisation to be an unbounded workqueue.
---
mm/zswap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
mm/zswap.c renamed from mm/zswap.c~mm-zswap-improve-with-alloc_workqueue-call +2 -1
@@ -1884,7 +1884,8 @@ static int zswap_setup(void) zswap_enabled = false; } - shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,1 );if (!shrink_wq) goto fallback_fail;
_
Quoted 72 lines · Nhat Pham
Quoted 11 lines · Johannes Weiner
Quoted 9 lines · Yosry Ahmed
Quoted 9 lines
Quoted 8 lines
Quoted 4 lines
On a different note, I wonder if it would help to perform synchronous reclaim here instead. With our current design, the zswap store failure (due to global limit hit) would leave the incoming page going to swap instead, creating an LRU inversion. Not sure if that's ideal.
The global shrink path keeps reclaiming until zswap can accept again (by default, that means reclaiming 10% of the total limit). I think this is too expensive to be done synchronously.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
That thresholding code is a bit weird right now.
It wakes the shrinker and rejects at the same time. We're guaranteed to see rejections, even if the shrinker has no trouble flushing some entries a split second later.
It would make more sense to wake the shrinker at e.g. 95% full and have it run until 90%.
Yep, we should be reclaiming zswap objects way ahead of the pool limit. Hence the new shrinker, which is memory pressure-driven (i.e independent of zswap internal limits), and will typically be triggered even if the pool is not full. During experiments, I never observe the pool becoming full, with the default settings. I'd be happy to extend it (or build in extra shrinking logic) to cover these pool limits too, if it turns out to be necessary.
Quoted 23 lines
Quoted 9 lines
Quoted 8 lines
But with that in place we also *should* do synchronous reclaim once we hit 100%. Just enough to make room for the store. This is important to catch the case where reclaim rate exceeds swapout rate. Rejecting and going to swap means the reclaimer will be throttled down to IO rate anyway, and the app latency isn't any worse. But this way we keep the pipeline alive, and keep swapping out the oldest zswap entries, instead of rejecting and swapping what would be the hottest ones.
Quoted 7 lines
I fully agree with the thresholding code being weird, and with waking up the shrinker before the pool is full. What I don't understand is how we can do synchronous reclaim when we hit 100% and still respect the acceptance threshold :/
Are you proposing we change the semantics of the acceptance threshold to begin with?
I kind of am. It's worth looking at the history of this knob.
It was added in 2020 by
45190f01dd402112d3d22c0ddc4152994f9e1e55, and from the changelogs and the code in this patch I do not understand how this was supposed to work.Quoted 6 lines
It also *didn't* work for very basic real world applications. See Domenico's follow-up (
e0228d590beb0d0af345c58a282f01afac5c57f3), which effectively reverted it to get halfway reasonable behavior.If there are no good usecases for this knob, then I think it makes sense to phase it out again.
Yeah this was my original proposal - remove this knob altogether :) Based on a cursory read, it just seems like zswap was originally trying to shrink (synchronously) one "object", then try to check if the pool size is now under the limit. This is indeed insufficient. However, I'm not quite convinced by the solution (hysteresis) either.
Maybe we can synchronously shrink a la Domenico, i.e until the pool can accept new pages, but this time capacity-based (maybe under the limit + some headroom, 1 page for example)? This is just so that the immediate incoming zswap store succeeds - we can still have the shrinker freeing up space later on (or maybe keep an asynchronous pool-limit based shrinker around).
Quoted 48 lines · Nhat Pham
On Tue, Jan 16, 2024 at 5:32 AM Ronald Monthero
<[email protected]> wrote:+ Johannes and Yosry
Quoted 9 lines
The core-api
create_workqueueis deprecated, this patch replaces thecreate_workqueuewithalloc_workqueue. The previous implementation workqueue of zswap was a bounded workqueue, this patch usesalloc_workqueue()to create an unbounded workqueue. TheWQ_UNBOUNDattribute is desirable making the workqueue not localized to a specific cpu so that the scheduler is free to exercise improvisations in any demanding scenarios for offloading cpu time slices for workqueues.nit: extra space between paragraph would be nice.
Quoted 29 lines
For example if any other workqueues of the same primary cpu had to be served which are
WQ_HIGHPRIandWQ_CPU_INTENSIVE. Also Unbound workqueue happens to be more efficient in a system during memory pressure scenarios in comparison
to a bounded workqueue.
shrink_wq= alloc_workqueue("zswap-shrink",WQ_UNBOUND|WQ_MEM_RECLAIM, 1);Overall the change suggested in this patch should be seamless and does not alter the existing behavior, other than the improvisation to be an unbounded workqueue.
Signed-off-by Ronald Monthero <[email protected]>--- mm/zswap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/zswap.c +2
-1@@ -1620,7 +1620,8 @@ static int zswap_setup(void) zswap_enabled = false; } - shrink_wq =create_workqueue ("zswap-shrink" ); + shrink_wq =alloc_workqueue ("zswap-shrink" , +WQ_UNBOUND |WQ_MEM_RECLAIM ,1 );
[...]
Quoted 6 lines
Quoted 6 lines
if (!shrink_wq) goto fallback_fail;--
2.34.1
FWIW: