Skip to contentSkip to conversation

mm/damon: simplify stop mechanism

PATCH Superseded Changbin Du 2 participants 4 messages Oct 26, 2021 – Oct 27, 2021

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/damon: simplify stop mechanism

An kernel thread can exit gracefully with kthread_stop(). So we don't need a new flag 'kdamond_stop'. And to make sure the task struct is not freed when accessing it, get task struct on start and put it on stop.

And since the return value of 'before_terminate' callback is never used, we make it have no return value.

---
 include/linux/damon.h |  3 +--
 mm/damon/core.c       | 59 +++++++++++++------------------------------
 mm/damon/dbgfs.c      |  5 ++--
 3 files changed, 20 insertions(+), 47 deletions(-)
include/linux/damon.h +1 -2
index a14b3cc54cab..041966786270 100644
@@ -322,7 +322,7 @@ struct damon_callback {
 	int (*before_start)(struct damon_ctx *context);
 	int (*after_sampling)(struct damon_ctx *context);
 	int (*after_aggregation)(struct damon_ctx *context);
-	int (*before_terminate)(struct damon_ctx *context);
+	void (*before_terminate)(struct damon_ctx *context);
 };
 
 /**
@@ -381,7 +381,6 @@ struct damon_ctx {
 
 /* public: */
 	struct task_struct *kdamond;
-	bool kdamond_stop;
 	struct mutex kdamond_lock;
 
 	struct damon_primitive primitive;
mm/damon/core.c +17 -42

137 lines not on this page · raw message

mm/damon/dbgfs.c +2 -3
index befb27a29aab..eccc14b34901 100644
@@ -645,18 +645,17 @@ static void dbgfs_fill_ctx_dir(struct dentry *dir, struct damon_ctx *ctx)
 		debugfs_create_file(file_names[i], 0600, dir, ctx, fops[i]);
 }
 
-static int dbgfs_before_terminate(struct damon_ctx *ctx)
+static void dbgfs_before_terminate(struct damon_ctx *ctx)
 {
 	struct damon_target *t, *next;
 
 	if (!targetid_is_pid(ctx))
-		return 0;
+		return;
 
 	damon_for_each_target_safe(t, next, ctx) {
 		put_pid((struct pid *)t->id);
 		damon_destroy_target(t);
 	}
-	return 0;
 }
 
 static struct damon_ctx *dbgfs_new_ctx(void)
Signature · 1 line
2.32.0
maintainer
#1 lore raw

Hello Changbin,

Quoted 3 lines · On Tue, 26 Oct 2021 23:30:33 +0800 Changbin Du <[email protected]> wrote:

An kernel thread can exit gracefully with kthread_stop(). So we don't need a new flag 'kdamond_stop'. And to make sure the task struct is not freed when accessing it, get task struct on start and put it on stop.

We previously considered using kthread_stop() here. However, we resulted in current code because kdamond can be self-terminated when all target processes are invalid[1].

Seems this patch is also not fully prepared for the self-termination case. I left some comments below.

[1] https://lore.kernel.org/linux-mm/[email protected]/

Quoted 3 lines

And since the return value of 'before_terminate' callback is never used, we make it have no return value.

This looks nice to me. Could you please send this again as a separate patch?

Quoted 12 lines
---
 include/linux/damon.h |  3 +--
 mm/damon/core.c       | 59 +++++++++++++------------------------------
 mm/damon/dbgfs.c      |  5 ++--
 3 files changed, 20 insertions(+), 47 deletions(-)
include/linux/damon.h +0 -0
index a14b3cc54cab..041966786270 100644

[...]

Quoted 2 lines
mm/damon/core.c +0 -0

[...]

Quoted 15 lines
mm/damon/core.c +1 -3
@@ -1069,7 +1048,7 @@ static int kdamond_fn(void *data)
 					sz_limit);
 			if (ctx->callback.after_aggregation &&
 					ctx->callback.after_aggregation(ctx))
-				set_kdamond_stop(ctx);
+				done = true;
 			kdamond_apply_schemes(ctx);
 			kdamond_reset_aggregated(ctx);
 			kdamond_split_regions(ctx);
@@ -1088,16 +1067,12 @@ static int kdamond_fn(void *data)
 			damon_destroy_region(r, t);
 	}
 
-	if (ctx->callback.before_terminate &&
-			ctx->callback.before_terminate(ctx))

mm/damon/core.c static int kdamond_fn(void *data) · line 1100

Quoted 10 lines
mm/damon/core.c +2 -4
-		set_kdamond_stop(ctx);
+	if (ctx->callback.before_terminate)
+		ctx->callback.before_terminate(ctx);
 	if (ctx->primitive.cleanup)
 		ctx->primitive.cleanup(ctx);
 
 	pr_debug("kdamond (%d) finishes\n", current->pid);
-	mutex_lock(&ctx->kdamond_lock);
-	ctx->kdamond = NULL;
-	mutex_unlock(&ctx->kdamond_lock);

When kdamond is self-terminating, ctx->kdamond will not be nullfified. As a result, this patch can introduce some errors like below:

    # cd /sys/kernel/debug/damon
    # sleep 60 &
    [1] 1926
    # echo $(pidof sleep) > target_ids
    # echo on > monitor_on
    # cat monitor_on
    on
    # # after 60 seconds, sleep finishes and kdamond is self-terminated
    # cat monitor_on
    off
    # echo 42 > target_ids
    bash: echo: write error: Device or resource busy

If we simply restore the nullification here with the mutex locking, we would result in a deadlock because __damon_stop() calls kthread_stop() while holding ctx->kdamond_lock.

Also, the reference count of ctx->kdamond, which increased by __damon_start(), would not be decreased in the case.

If I'm missing something, please let me know.

Thanks,
SJ

[...]

#2 lore raw
Quoted 15 lines · SeongJae Park

Hello Changbin,

Quoted 3 lines · On Tue, 26 Oct 2021 23:30:33 +0800 Changbin Du <[email protected]> wrote:

An kernel thread can exit gracefully with kthread_stop(). So we don't need a new flag 'kdamond_stop'. And to make sure the task struct is not freed when accessing it, get task struct on start and put it on stop.

We previously considered using kthread_stop() here. However, we resulted in current code because kdamond can be self-terminated when all target processes are invalid[1].

Seems this patch is also not fully prepared for the self-termination case. I left some comments below.

Quoted 8 lines

[1] https://lore.kernel.org/linux-mm/[email protected]/

Quoted 3 lines

And since the return value of 'before_terminate' callback is never used, we make it have no return value.

This looks nice to me. Could you please send this again as a separate patch?

Sure, I'll do it later.

Quoted 58 lines
Quoted 12 lines
---
 include/linux/damon.h |  3 +--
 mm/damon/core.c       | 59 +++++++++++++------------------------------
 mm/damon/dbgfs.c      |  5 ++--
 3 files changed, 20 insertions(+), 47 deletions(-)
include/linux/damon.h +0 -0
index a14b3cc54cab..041966786270 100644

[...]

Quoted 2 lines
mm/damon/core.c +0 -0

[...]

Quoted 15 lines
mm/damon/core.c +1 -3
@@ -1069,7 +1048,7 @@ static int kdamond_fn(void *data)
 					sz_limit);
 			if (ctx->callback.after_aggregation &&
 					ctx->callback.after_aggregation(ctx))
-				set_kdamond_stop(ctx);
+				done = true;
 			kdamond_apply_schemes(ctx);
 			kdamond_reset_aggregated(ctx);
 			kdamond_split_regions(ctx);
@@ -1088,16 +1067,12 @@ static int kdamond_fn(void *data)
 			damon_destroy_region(r, t);
 	}
 
-	if (ctx->callback.before_terminate &&
-			ctx->callback.before_terminate(ctx))

mm/damon/core.c static int kdamond_fn(void *data) · line 1100

Quoted 10 lines
mm/damon/core.c +2 -4
-		set_kdamond_stop(ctx);
+	if (ctx->callback.before_terminate)
+		ctx->callback.before_terminate(ctx);
 	if (ctx->primitive.cleanup)
 		ctx->primitive.cleanup(ctx);
 
 	pr_debug("kdamond (%d) finishes\n", current->pid);
-	mutex_lock(&ctx->kdamond_lock);
-	ctx->kdamond = NULL;
-	mutex_unlock(&ctx->kdamond_lock);

When kdamond is self-terminating, ctx->kdamond will not be nullfified. As a result, this patch can introduce some errors like below:

    # cd /sys/kernel/debug/damon
    # sleep 60 &
    [1] 1926
    # echo $(pidof sleep) > target_ids
    # echo on > monitor_on
    # cat monitor_on
    on
    # # after 60 seconds, sleep finishes and kdamond is self-terminated
    # cat monitor_on
    off
    # echo 42 > target_ids
    bash: echo: write error: Device or resource busy
Quoted 7 lines

If we simply restore the nullification here with the mutex locking, we would result in a deadlock because __damon_stop() calls kthread_stop() while holding ctx->kdamond_lock.

Also, the reference count of ctx->kdamond, which increased by __damon_start(), would not be decreased in the case.

If so, I suppose below change should work correctly (still set ctx->kdamond to NULL at the end of kdamond).

static int __damon_stop(struct damon_ctx *ctx)
{
+ struct task_struct *tsk;
+

        mutex_lock(&ctx->kdamond_lock);

- if (ctx->kdamond) {
- ctx->kdamond_stop = true;
+ tsk = ctx->kdamond;
+ if (tsk) {

+               get_task_struct(tsk);
                mutex_unlock(&ctx->kdamond_lock);

- while (damon_kdamond_running(ctx))
- usleep_range(ctx->sample_interval,
- ctx->sample_interval * 2);
+ kthread_stop(tsk);

+               put_task_struct(tsk);
                return 0;
        }
        mutex_unlock(&ctx->kdamond_lock);
Quoted 7 lines

If I'm missing something, please let me know.

Thanks,
SJ

[...]

Signature · 2 lines
Cheers,
Changbin Du
maintainer
#3 lore raw
Quoted 104 lines · On Wed, 27 Oct 2021 14:13:38 +0800 Changbin Du <[email protected]> wrote:
Quoted 15 lines · SeongJae Park

Hello Changbin,

Quoted 3 lines · On Tue, 26 Oct 2021 23:30:33 +0800 Changbin Du <[email protected]> wrote:

An kernel thread can exit gracefully with kthread_stop(). So we don't need a new flag 'kdamond_stop'. And to make sure the task struct is not freed when accessing it, get task struct on start and put it on stop.

We previously considered using kthread_stop() here. However, we resulted in current code because kdamond can be self-terminated when all target processes are invalid[1].

Seems this patch is also not fully prepared for the self-termination case. I left some comments below.

Quoted 8 lines

[1] https://lore.kernel.org/linux-mm/[email protected]/

Quoted 3 lines

And since the return value of 'before_terminate' callback is never used, we make it have no return value.

This looks nice to me. Could you please send this again as a separate patch?

Sure, I'll do it later.

Quoted 58 lines
Quoted 12 lines
---
 include/linux/damon.h |  3 +--
 mm/damon/core.c       | 59 +++++++++++++------------------------------
 mm/damon/dbgfs.c      |  5 ++--
 3 files changed, 20 insertions(+), 47 deletions(-)
include/linux/damon.h +0 -0
index a14b3cc54cab..041966786270 100644

[...]

Quoted 2 lines
mm/damon/core.c +0 -0

[...]

Quoted 15 lines
mm/damon/core.c +1 -3
@@ -1069,7 +1048,7 @@ static int kdamond_fn(void *data)
 					sz_limit);
 			if (ctx->callback.after_aggregation &&
 					ctx->callback.after_aggregation(ctx))
-				set_kdamond_stop(ctx);
+				done = true;
 			kdamond_apply_schemes(ctx);
 			kdamond_reset_aggregated(ctx);
 			kdamond_split_regions(ctx);
@@ -1088,16 +1067,12 @@ static int kdamond_fn(void *data)
 			damon_destroy_region(r, t);
 	}
 
-	if (ctx->callback.before_terminate &&
-			ctx->callback.before_terminate(ctx))

mm/damon/core.c static int kdamond_fn(void *data) · line 1100

Quoted 10 lines
mm/damon/core.c +2 -4
-		set_kdamond_stop(ctx);
+	if (ctx->callback.before_terminate)
+		ctx->callback.before_terminate(ctx);
 	if (ctx->primitive.cleanup)
 		ctx->primitive.cleanup(ctx);
 
 	pr_debug("kdamond (%d) finishes\n", current->pid);
-	mutex_lock(&ctx->kdamond_lock);
-	ctx->kdamond = NULL;
-	mutex_unlock(&ctx->kdamond_lock);

When kdamond is self-terminating, ctx->kdamond will not be nullfified. As a result, this patch can introduce some errors like below:

    # cd /sys/kernel/debug/damon
    # sleep 60 &
    [1] 1926
    # echo $(pidof sleep) > target_ids
    # echo on > monitor_on
    # cat monitor_on
    on
    # # after 60 seconds, sleep finishes and kdamond is self-terminated
    # cat monitor_on
    off
    # echo 42 > target_ids
    bash: echo: write error: Device or resource busy
Quoted 7 lines

If we simply restore the nullification here with the mutex locking, we would result in a deadlock because __damon_stop() calls kthread_stop() while holding ctx->kdamond_lock.

Also, the reference count of ctx->kdamond, which increased by __damon_start(), would not be decreased in the case.

If so, I suppose below change should work correctly (still set ctx->kdamond to NULL at the end of kdamond).

static int __damon_stop(struct damon_ctx *ctx)
{

(quoted diff) +4 -2
+       struct task_struct *tsk;
+
        mutex_lock(&ctx->kdamond_lock);
-       if (ctx->kdamond) {
-               ctx->kdamond_stop = true;
+       tsk = ctx->kdamond;
+       if (tsk) {

Review

Quoted 10 lines
(quoted diff) +3 -3
+               get_task_struct(tsk);
                mutex_unlock(&ctx->kdamond_lock);
-               while (damon_kdamond_running(ctx))
-                       usleep_range(ctx->sample_interval,
-                                       ctx->sample_interval * 2);
+               kthread_stop(tsk);
+               put_task_struct(tsk);
                return 0;
        }
        mutex_unlock(&ctx->kdamond_lock);

Agreed, this seems to make sense. Could you please send a complete patch?

Thanks,
SJ

Quoted 13 lines
Quoted 9 lines

If I'm missing something, please let me know.

Thanks,
SJ

[...]

Signature · 2 lines
Cheers,
Changbin Du