mm/damon: simplify stop mechanism
Subsystems (2)
PATCH 1/1 mm/damon: simplify stop mechanism
An kernel thread can exit gracefully with kthread_stop(). So we don't need a new flag 'kdamond_stop'. And to make sure the task struct is not freed when accessing it, get task struct on start and put it on stop.
And since the return value of 'before_terminate' callback is never used, we make it have no return value.
--- include/linux/damon.h | 3 +-- mm/damon/core.c | 59 +++++++++++++------------------------------ mm/damon/dbgfs.c | 5 ++-- 3 files changed, 20 insertions(+), 47 deletions(-)
include/linux/damon.h +1 -2
@@ -322,7 +322,7 @@ struct damon_callback {int (*before_start)(struct damon_ctx *context);int (*after_sampling)(struct damon_ctx *context);int (*after_aggregation)(struct damon_ctx *context); -int (*before_terminate)(struct damon_ctx *context); +void (*before_terminate)(struct damon_ctx *context); };/**
@@ -381,7 +381,6 @@ struct damon_ctx {/* public: */ struct task_struct *kdamond; -bool kdamond_stop;struct mutex kdamond_lock;struct damon_primitive primitive;
mm/damon/core.c +17 -42
137 lines not on this page · raw message
mm/damon/dbgfs.c +2 -3
@@ -645,18 +645,17 @@ static void dbgfs_fill_ctx_dir(struct dentry *dir, struct damon_ctx *ctx)debugfs_create_file (file_names[i],0600 , dir, ctx, fops[i]); } -static int dbgfs_before_terminate (struct damon_ctx *ctx) +static void dbgfs_before_terminate (struct damon_ctx *ctx) {struct damon_target *t, *next;if (!targetid_is_pid (ctx)) -return 0 ; +return ;damon_for_each_target_safe (t, next, ctx) {put_pid ((struct pid *)t->id);damon_destroy_target (t); } -return 0 ; }static struct damon_ctx *dbgfs_new_ctx (void )
Signature · 1 line
2.32.0
SeongJae Park
maintainerHello Changbin,
Quoted 3 lines · On Tue, 26 Oct 2021 23:30:33 +0800 Changbin Du <[email protected]> wrote:
An kernel thread can exit gracefully with
kthread_stop(). So we don't need a new flag 'kdamond_stop'. And to make sure the task struct is not freed when accessing it, get task struct on start and put it on stop.
We previously considered using kthread_stop() here. However, we resulted in current code because kdamond can be self-terminated when all target processes are invalid[1].
Seems this patch is also not fully prepared for the self-termination case. I left some comments below.
[1] https://lore.kernel.org/linux-mm/[email protected]/
Quoted 3 lines
And since the return value of '
before_terminate' callback is never used, we make it have no return value.
This looks nice to me. Could you please send this again as a separate patch?
Quoted 12 lines
Signed-off-by Changbin Du <[email protected]>--- include/linux/damon.h | 3 +-- mm/damon/core.c | 59 +++++++++++++------------------------------ mm/damon/dbgfs.c | 5 ++-- 3 files changed, 20 insertions(+), 47 deletions(-)include/linux/damon.h +0
-0
[...]
Quoted 2 lines
mm/damon/core.c +0
-0
[...]
Quoted 15 lines
mm/damon/core.c +1
-3@@ -1069,7 +1048,7 @@ static int kdamond_fn(void *data) sz_limit);if (ctx->callback.after_aggregation && ctx->callback.after_aggregation (ctx)) -set_kdamond_stop (ctx); + done = true;kdamond_apply_schemes (ctx);kdamond_reset_aggregated (ctx);kdamond_split_regions (ctx);@@ -1088,16 +1067,12 @@ static int kdamond_fn(void *data)damon_destroy_region (r, t); } -if (ctx->callback.before_terminate && - ctx->callback.before_terminate (ctx))
mm/damon/core.c static int kdamond_fn(void *data) · line 1100
Quoted 10 lines
mm/damon/core.c +2
-4-set_kdamond_stop (ctx); +if (ctx->callback.before_terminate) + ctx->callback.before_terminate (ctx);if (ctx->primitive.cleanup) ctx->primitive.cleanup (ctx);pr_debug ("kdamond (%d) finishes\n" , current->pid); -mutex_lock (&ctx->kdamond_lock); - ctx->kdamond =NULL ; -mutex_unlock (&ctx->kdamond_lock);
When kdamond is self-terminating, ctx->kdamond will not be nullfified. As a result, this patch can introduce some errors like below:
# cd /sys/kernel/debug/damon
# sleep 60 &
[1] 1926
# echo $(pidof sleep) > target_ids
# echo on > monitor_on
# cat monitor_on
on
# # after 60 seconds, sleep finishes and kdamond is self-terminated
# cat monitor_on
off
# echo 42 > target_ids
bash: echo: write error: Device or resource busyIf we simply restore the nullification here with the mutex locking, we would result in a deadlock because __damon_stop() calls kthread_stop() while holding ctx->kdamond_lock.
Also, the reference count of ctx->kdamond, which increased by __damon_start(), would not be decreased in the case.
If I'm missing something, please let me know.
Thanks,
SJ
[...]
Quoted 15 lines · SeongJae Park
Hello Changbin,
Quoted 3 lines · On Tue, 26 Oct 2021 23:30:33 +0800 Changbin Du <[email protected]> wrote:
An kernel thread can exit gracefully with
kthread_stop(). So we don't need a new flag 'kdamond_stop'. And to make sure the task struct is not freed when accessing it, get task struct on start and put it on stop.We previously considered using
kthread_stop()here. However, we resulted in current code because kdamond can be self-terminated when all target processes are invalid[1].Seems this patch is also not fully prepared for the self-termination case. I left some comments below.
Quoted 8 lines
[1] https://lore.kernel.org/linux-mm/[email protected]/
Quoted 3 lines
And since the return value of '
before_terminate' callback is never used, we make it have no return value.This looks nice to me. Could you please send this again as a separate patch?
Sure, I'll do it later.
Quoted 58 lines
Quoted 12 lines
Signed-off-by Changbin Du <[email protected]>--- include/linux/damon.h | 3 +-- mm/damon/core.c | 59 +++++++++++++------------------------------ mm/damon/dbgfs.c | 5 ++-- 3 files changed, 20 insertions(+), 47 deletions(-)include/linux/damon.h +0
-0[...]
Quoted 2 lines
mm/damon/core.c +0
-0[...]
Quoted 15 lines
mm/damon/core.c +1
-3@@ -1069,7 +1048,7 @@ static int kdamond_fn(void *data) sz_limit);if (ctx->callback.after_aggregation && ctx->callback.after_aggregation (ctx)) -set_kdamond_stop (ctx); + done = true;kdamond_apply_schemes (ctx);kdamond_reset_aggregated (ctx);kdamond_split_regions (ctx);@@ -1088,16 +1067,12 @@ static int kdamond_fn(void *data)damon_destroy_region (r, t); } -if (ctx->callback.before_terminate && - ctx->callback.before_terminate (ctx))mm/damon/core.c static int kdamond_fn(void *data) · line 1100
Quoted 10 lines
mm/damon/core.c +2
-4-set_kdamond_stop (ctx); +if (ctx->callback.before_terminate) + ctx->callback.before_terminate (ctx);if (ctx->primitive.cleanup) ctx->primitive.cleanup (ctx);pr_debug ("kdamond (%d) finishes\n" , current->pid); -mutex_lock (&ctx->kdamond_lock); - ctx->kdamond =NULL ; -mutex_unlock (&ctx->kdamond_lock);When kdamond is self-terminating,
ctx->kdamondwill not be nullfified. As a result, this patch can introduce some errors like below:# cd /sys/kernel/debug/damon # sleep 60 & [1] 1926 # echo $(pidof sleep) > target_ids # echo on > monitor_on # cat monitor_on on # # after 60 seconds, sleep finishes and kdamond is self-terminated # cat monitor_on off # echo 42 > target_ids bash: echo: write error: Device or resource busy
Quoted 7 lines
If we simply restore the nullification here with the mutex locking, we would result in a deadlock because
__damon_stop()callskthread_stop()while holdingctx->kdamond_lock.Also, the reference count of
ctx->kdamond, which increased by__damon_start(), would not be decreased in the case.
If so, I suppose below change should work correctly (still set ctx->kdamond to NULL at the end of kdamond).
static int __damon_stop(struct damon_ctx *ctx)
{
+ struct task_struct *tsk;
+
mutex_lock(&ctx->kdamond_lock);
- if (ctx->kdamond) {
- ctx->kdamond_stop = true;
+ tsk = ctx->kdamond;
+ if (tsk) {
+ get_task_struct(tsk);
mutex_unlock(&ctx->kdamond_lock);- while (damon_kdamond_running(ctx))
- usleep_range(ctx->sample_interval,
- ctx->sample_interval * 2);
+ kthread_stop(tsk);
+ put_task_struct(tsk);
return 0;
}
mutex_unlock(&ctx->kdamond_lock);Quoted 7 lines
If I'm missing something, please let me know.
Thanks,
SJ[...]
Signature · 2 lines
Cheers, Changbin Du
SeongJae Park
maintainerQuoted 104 lines · On Wed, 27 Oct 2021 14:13:38 +0800 Changbin Du <[email protected]> wrote:
Quoted 15 lines · SeongJae Park
Hello Changbin,
Quoted 3 lines · On Tue, 26 Oct 2021 23:30:33 +0800 Changbin Du <[email protected]> wrote:
An kernel thread can exit gracefully with
kthread_stop(). So we don't need a new flag 'kdamond_stop'. And to make sure the task struct is not freed when accessing it, get task struct on start and put it on stop.We previously considered using
kthread_stop()here. However, we resulted in current code because kdamond can be self-terminated when all target processes are invalid[1].Seems this patch is also not fully prepared for the self-termination case. I left some comments below.
Quoted 8 lines
[1] https://lore.kernel.org/linux-mm/[email protected]/
Quoted 3 lines
And since the return value of '
before_terminate' callback is never used, we make it have no return value.This looks nice to me. Could you please send this again as a separate patch?
Sure, I'll do it later.
Quoted 58 lines
Quoted 12 lines
Signed-off-by Changbin Du <[email protected]>--- include/linux/damon.h | 3 +-- mm/damon/core.c | 59 +++++++++++++------------------------------ mm/damon/dbgfs.c | 5 ++-- 3 files changed, 20 insertions(+), 47 deletions(-)include/linux/damon.h +0
-0[...]
Quoted 2 lines
mm/damon/core.c +0
-0[...]
Quoted 15 lines
mm/damon/core.c +1
-3@@ -1069,7 +1048,7 @@ static int kdamond_fn(void *data) sz_limit);if (ctx->callback.after_aggregation && ctx->callback.after_aggregation (ctx)) -set_kdamond_stop (ctx); + done = true;kdamond_apply_schemes (ctx);kdamond_reset_aggregated (ctx);kdamond_split_regions (ctx);@@ -1088,16 +1067,12 @@ static int kdamond_fn(void *data)damon_destroy_region (r, t); } -if (ctx->callback.before_terminate && - ctx->callback.before_terminate (ctx))mm/damon/core.c static int kdamond_fn(void *data) · line 1100
Quoted 10 lines
mm/damon/core.c +2
-4-set_kdamond_stop (ctx); +if (ctx->callback.before_terminate) + ctx->callback.before_terminate (ctx);if (ctx->primitive.cleanup) ctx->primitive.cleanup (ctx);pr_debug ("kdamond (%d) finishes\n" , current->pid); -mutex_lock (&ctx->kdamond_lock); - ctx->kdamond =NULL ; -mutex_unlock (&ctx->kdamond_lock);When kdamond is self-terminating,
ctx->kdamondwill not be nullfified. As a result, this patch can introduce some errors like below:# cd /sys/kernel/debug/damon # sleep 60 & [1] 1926 # echo $(pidof sleep) > target_ids # echo on > monitor_on # cat monitor_on on # # after 60 seconds, sleep finishes and kdamond is self-terminated # cat monitor_on off # echo 42 > target_ids bash: echo: write error: Device or resource busyQuoted 7 lines
If we simply restore the nullification here with the mutex locking, we would result in a deadlock because
__damon_stop()callskthread_stop()while holdingctx->kdamond_lock.Also, the reference count of
ctx->kdamond, which increased by__damon_start(), would not be decreased in the case.If so, I suppose below change should work correctly (still set
ctx->kdamondto NULL at the end of kdamond).static int
__damon_stop(struct damon_ctx *ctx)
{(quoted diff) +4
-2+ struct task_struct *tsk; + mutex_lock(&ctx->kdamond_lock); - if (ctx->kdamond) { - ctx->kdamond_stop = true; + tsk = ctx->kdamond; + if (tsk) {
Review
Quoted 10 lines
(quoted diff) +3
-3+ get_task_struct(tsk); mutex_unlock(&ctx->kdamond_lock); - while (damon_kdamond_running(ctx)) - usleep_range(ctx->sample_interval, - ctx->sample_interval * 2); + kthread_stop(tsk); + put_task_struct(tsk); return 0; } mutex_unlock(&ctx->kdamond_lock);
Agreed, this seems to make sense. Could you please send a complete patch?
Thanks,
SJ
Quoted 13 lines
Quoted 9 lines
If I'm missing something, please let me know.
Thanks,
SJ[...]
Signature · 2 lines
Cheers, Changbin Du