Skip to contentSkip to conversation

Damos filter cleanup code and implement workingset

RFC In review Huan Yang 2 participants 8 messages Sep 19, 2023 – Sep 25, 2023

Subsystems (2)
lore
Share & tools
lore

Huan Yang

author
#0 lore raw

cover Damos filter cleanup code and implement workingset

Now damos support filter contains two type.
The first is scheme filter which will invoke each scheme apply, second is scheme action filter, which will filter out unwanted action.

But this implement is scattered in different implementations and hard to reuse or extend.

This patchset clean up those filter code, move into filter.c and add header to expose filter func.(patch 2) And add a new filter "workingset" to protect workingset page.

Do we need this and cleanup it?

Huan Yang (2):
mm/damos/filter: Add workingset page filter
mm/damos/filter: Damos filter cleanup

 include/linux/damon.h    |  62 +-----------------
 mm/damon/Makefile        |   2 +-
 mm/damon/core-test.h     |   7 ++
 mm/damon/core.c          |  93 ++++-----------------------
 mm/damon/filter.c        | 135 +++++++++++++++++++++++++++++++++++++++
 mm/damon/filter.h        | 119 ++++++++++++++++++++++++++++++++++
 mm/damon/paddr.c         |  29 +++------
 mm/damon/reclaim.c       |  48 +++++++++++---
 mm/damon/sysfs-schemes.c |   1 +
 9 files changed, 325 insertions(+), 171 deletions(-)
 create mode 100644 mm/damon/filter.c
 create mode 100644 mm/damon/filter.h
Signature · 1 line
2.34.1
author
#1 lore raw

RFC 1/2 mm/damos/filter: Add workingset page filter

Some page if marked as workingset, we'd better not touch it. So, for damon pa scheme, this patch just add an new filter DAMOS_FILTER_TYPE_WORKINGSET, which will filter page if it marked as workingset.

Like skip anon, add a control skip_workingset.

To make code clean, fold reclaim's filter create logic into __damon_reclaim_create_filters.

---
 include/linux/damon.h | 17 +++++++++-------
 mm/damon/core-test.h  |  7 +++++++
 mm/damon/paddr.c      |  4 ++++
 mm/damon/reclaim.c    | 47 ++++++++++++++++++++++++++++++++++---------
 4 files changed, 58 insertions(+), 17 deletions(-)
include/linux/damon.h +10 -7
index ae2664d1d5f1..8e8f35df6a5e 100644
@@ -228,22 +228,25 @@ struct damos_stat {
  * @DAMOS_FILTER_TYPE_MEMCG:	Specific memcg's pages.
  * @DAMOS_FILTER_TYPE_ADDR:	Address range.
  * @DAMOS_FILTER_TYPE_TARGET:	Data Access Monitoring target.
+ * @DAMOS_FILTER_TYPE_WORKINGSET: Workingset pages, need protect.
  * @NR_DAMOS_FILTER_TYPES:	Number of filter types.
  *
- * The anon pages type and memcg type filters are handled by underlying
- * &struct damon_operations as a part of scheme action trying, and therefore
- * accounted as 'tried'.  In contrast, other types are handled by core layer
- * before trying of the action and therefore not accounted as 'tried'.
+ * The anon pages type, memcg type, workingset page type filters are handled
+ * by underlying &struct damon_operations as a part of scheme action trying,
+ * and therefore accounted as 'tried'.  In contrast, other types are handled
+ * by core layer before trying of the action and therefore not accounted as
+ * 'tried'.
  *
  * The support of the filters that handled by &struct damon_operations depend
  * on the running &struct damon_operations.
- * &enum DAMON_OPS_PADDR supports both anon pages type and memcg type filters,
- * while &enum DAMON_OPS_VADDR and &enum DAMON_OPS_FVADDR don't support any of
- * the two types.
+ * &enum DAMON_OPS_PADDR supports both anon pages type, memcg type and
+ * workingset page type filters, while &enum DAMON_OPS_VADDR and &enum
+ * DAMON_OPS_FVADDR don't support any of the two types.
  */
 enum damos_filter_type {
 	DAMOS_FILTER_TYPE_ANON,
 	DAMOS_FILTER_TYPE_MEMCG,
+	DAMOS_FILTER_TYPE_WORKINGSET,
 	DAMOS_FILTER_TYPE_ADDR,
 	DAMOS_FILTER_TYPE_TARGET,
 	NR_DAMOS_FILTER_TYPES,
mm/damon/core-test.h +7 -0
index 6cc8b245586d..c752e6e3cf3e 100644
@@ -351,6 +351,13 @@ static void damos_test_new_filter(struct kunit *test)
 	KUNIT_EXPECT_PTR_EQ(test, filter->list.prev, &filter->list);
 	KUNIT_EXPECT_PTR_EQ(test, filter->list.next, &filter->list);
 	damos_destroy_filter(filter);
+
+	filter = damos_new_filter(DAMOS_FILTER_TYPE_WORKINGSET, true);
+	KUNIT_EXPECT_EQ(test, filter->type, DAMOS_FILTER_TYPE_WORKINGSET);
+	KUNIT_EXPECT_EQ(test, filter->matching, true);
+	KUNIT_EXPECT_PTR_EQ(test, filter->list.prev, &filter->list);
+	KUNIT_EXPECT_PTR_EQ(test, filter->list.next, &filter->list);
+	damos_destroy_filter(filter);
 }
 
 static void damos_test_filter_out(struct kunit *test)
mm/damon/paddr.c +4 -0
index 909db25efb35..8a690505e033 100644
@@ -12,6 +12,7 @@
 #include <linux/pagemap.h>
 #include <linux/rmap.h>
 #include <linux/swap.h>
+#include <linux/mm_inline.h>
 
 #include "../internal.h"
 #include "ops-common.h"
@@ -204,6 +205,9 @@ static bool __damos_pa_filter_out(struct damos_filter *filter,
 			matched = filter->memcg_id == mem_cgroup_id(memcg);
 		rcu_read_unlock();
 		break;
+	case DAMOS_FILTER_TYPE_WORKINGSET:
+		matched = folio_test_workingset(folio);
+		break;
 	default:
 		break;
 	}
mm/damon/reclaim.c +37 -10

68 lines not on this page · raw message

Signature · 1 line
2.34.1
author
#2 lore raw

RFC 2/2 mm/damos/filter: Damos filter cleanup

In reply to Huan Yang

Now damos support filter contains two type.
The first is scheme filter which will invoke each scheme apply, second is scheme action filter, which will filter out unwanted action.

But this implement is scattered in different implementations and hard to reuse or extend.

This patch clean up those filter code, move into filter.c and add header to expose filter func.

Each filter implement register itself into filter_fn array, and user can invoke this by there inited type in filter.

---
 include/linux/damon.h    |  65 +------------------
 mm/damon/Makefile        |   2 +-
 mm/damon/core.c          |  93 ++++-----------------------
 mm/damon/filter.c        | 135 +++++++++++++++++++++++++++++++++++++++
 mm/damon/filter.h        | 119 ++++++++++++++++++++++++++++++++++
 mm/damon/paddr.c         |  31 +++------
 mm/damon/reclaim.c       |   1 +
 mm/damon/sysfs-schemes.c |   1 +
 8 files changed, 280 insertions(+), 167 deletions(-)
 create mode 100644 mm/damon/filter.c
 create mode 100644 mm/damon/filter.h
include/linux/damon.h +3 -62
index 8e8f35df6a5e..03c718b30bfe 100644
@@ -222,63 +222,6 @@ struct damos_stat {
 	unsigned long qt_exceeds;
 };
 
-/**
- * enum damos_filter_type - Type of memory for &struct damos_filter
- * @DAMOS_FILTER_TYPE_ANON:	Anonymous pages.
- * @DAMOS_FILTER_TYPE_MEMCG:	Specific memcg's pages.
- * @DAMOS_FILTER_TYPE_ADDR:	Address range.
- * @DAMOS_FILTER_TYPE_TARGET:	Data Access Monitoring target.
- * @DAMOS_FILTER_TYPE_WORKINGSET: Workingset pages, need protect.
- * @NR_DAMOS_FILTER_TYPES:	Number of filter types.
- *
- * The anon pages type, memcg type, workingset page type filters are handled
- * by underlying &struct damon_operations as a part of scheme action trying,
- * and therefore accounted as 'tried'.  In contrast, other types are handled
- * by core layer before trying of the action and therefore not accounted as
- * 'tried'.
- *
- * The support of the filters that handled by &struct damon_operations depend
- * on the running &struct damon_operations.
- * &enum DAMON_OPS_PADDR supports both anon pages type, memcg type and
- * workingset page type filters, while &enum DAMON_OPS_VADDR and &enum
- * DAMON_OPS_FVADDR don't support any of the two types.
- */
-enum damos_filter_type {
-	DAMOS_FILTER_TYPE_ANON,
-	DAMOS_FILTER_TYPE_MEMCG,
-	DAMOS_FILTER_TYPE_WORKINGSET,
-	DAMOS_FILTER_TYPE_ADDR,
-	DAMOS_FILTER_TYPE_TARGET,
-	NR_DAMOS_FILTER_TYPES,
-};
-
-/**
- * struct damos_filter - DAMOS action target memory filter.
- * @type:	Type of the page.
- * @matching:	If the matching page should filtered out or in.
- * @memcg_id:	Memcg id of the question if @type is DAMOS_FILTER_MEMCG.
- * @addr_range:	Address range if @type is DAMOS_FILTER_TYPE_ADDR.
- * @target_idx:	Index of the &struct damon_target of
- *		&damon_ctx->adaptive_targets if @type is
- *		DAMOS_FILTER_TYPE_TARGET.
- * @list:	List head for siblings.
- *
- * Before applying the &damos->action to a memory region, DAMOS checks if each
- * page of the region matches to this and avoid applying the action if so.
- * Support of each filter type depends on the running &struct damon_operations
- * and the type.  Refer to &enum damos_filter_type for more detai.
- */
-struct damos_filter {
-	enum damos_filter_type type;
-	bool matching;
-	union {
-		unsigned short memcg_id;
-		struct damon_addr_range addr_range;
-		int target_idx;
-	};
-	struct list_head list;
-};
-
 /**
  * struct damos_access_pattern - Target access pattern of the given scheme.
  * @min_sz_region:	Minimum size of target regions.
@@ -607,16 +550,14 @@ static inline void damon_insert_region(struct damon_region *r,
 	t->nr_regions++;
 }
 
+void damon_split_region_at(struct damon_target *t,
+				  struct damon_region *r, unsigned long sz_r);
+
 void damon_add_region(struct damon_region *r, struct damon_target *t);
 void damon_destroy_region(struct damon_region *r, struct damon_target *t);
 int damon_set_regions(struct damon_target *t, struct damon_addr_range *ranges,
 		unsigned int nr_ranges);
 
-struct damos_filter *damos_new_filter(enum damos_filter_type type,
-		bool matching);
-void damos_add_filter(struct damos *s, struct damos_filter *f);
-void damos_destroy_filter(struct damos_filter *f);
-
 struct damos *damon_new_scheme(struct damos_access_pattern *pattern,
 			enum damos_action action, struct damos_quota *quota,
 			struct damos_watermarks *wmarks);
mm/damon/Makefile +1 -1
index f7add3f4aa79..789ab0a9d9c9 100644
@@ -1,6 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 
-obj-y				:= core.o
+obj-y				:= core.o filter.o
 obj-$(CONFIG_DAMON_VADDR)	+= ops-common.o vaddr.o
 obj-$(CONFIG_DAMON_PADDR)	+= ops-common.o paddr.o
 obj-$(CONFIG_DAMON_SYSFS)	+= sysfs-common.o sysfs-schemes.o sysfs.o
mm/damon/core.c +12 -81

124 lines not on this page · raw message

mm/damon/filter.c new file +135 -0

135 lines not on this page · raw message

mm/damon/filter.h new file +119 -0

119 lines not on this page · raw message

mm/damon/paddr.c +8 -23
index 8a690505e033..748300655ba4 100644
@@ -16,6 +16,7 @@
 
 #include "../internal.h"
 #include "ops-common.h"
+#include "filter.h"
 
 static bool __damon_pa_mkold(struct folio *folio, struct vm_area_struct *vma,
 		unsigned long addr, void *arg)
@@ -189,30 +190,14 @@ static unsigned int damon_pa_check_accesses(struct damon_ctx *ctx)
 static bool __damos_pa_filter_out(struct damos_filter *filter,
 		struct folio *folio)
 {
-	bool matched = false;
-	struct mem_cgroup *memcg;
-
-	switch (filter->type) {
-	case DAMOS_FILTER_TYPE_ANON:
-		matched = folio_test_anon(folio);
-		break;
-	case DAMOS_FILTER_TYPE_MEMCG:
-		rcu_read_lock();
-		memcg = folio_memcg_check(folio);
-		if (!memcg)
-			matched = false;
-		else
-			matched = filter->memcg_id == mem_cgroup_id(memcg);
-		rcu_read_unlock();
-		break;
-	case DAMOS_FILTER_TYPE_WORKINGSET:
-		matched = folio_test_workingset(folio);
-		break;
-	default:
-		break;
-	}
+	struct damos_filter_ctx dfctx = {
+		.action = {
+			.folio = folio,
+		},
+		.filter = filter,
+	};
 
-	return matched == filter->matching;
+	return damon_filter_invoke(&dfctx);
 }
 
 /*
mm/damon/reclaim.c +1 -0
index 26ae8fa5d088..1fdd55c03f1f 100644
@@ -12,6 +12,7 @@
 #include <linux/module.h>
 
 #include "modules-common.h"
+#include "filter.h"
 
 #ifdef MODULE_PARAM_PREFIX
 #undef MODULE_PARAM_PREFIX
mm/damon/sysfs-schemes.c +1 -0
index 527e7d17eb3b..98cd93b28b21 100644
@@ -8,6 +8,7 @@
 #include <linux/slab.h>
 
 #include "sysfs-common.h"
+#include "filter.h"
 
 /*
  * scheme region directory
Signature · 1 line
2.34.1
maintainer
#3 lore raw
In reply to Huan Yang

Hi Huan,

First of all, thank you for this patch. I have some high level comments and questions as below.

Quoted 3 lines · On Tue, 19 Sep 2023 17:52:33 +0800 Huan Yang <[email protected]> wrote:

Now damos support filter contains two type.
The first is scheme filter which will invoke each scheme apply, second is scheme action filter, which will filter out unwanted action.

IMHO, that's not clear definition of the types. Conceptually, all the filters are same. Nonetheless, they are having different behaviors because they are handled in different layer depending on which layer is more efficient to handle those[1].

I agree this is complicated and a bit verbose explanation, but I don't feel your scheme vs action definition is making it easier to understand.

Quoted 2 lines

But this implement is scattered in different implementations

Indeed the implementation is scattered in core layer and the ops layer depending on the filter types. But I think this is needed for efficient handling of those.

Quoted 1 line

and hard to reuse or extend.

From your first patch, which extending the filter, the essential change for the extension is adding another enum to 'enum damos_filter_type' (1 line) and addition of switch case in '__damos_pa_filter_out()' (3 lines). I don't think it looks too complicated. If you're saying it was hard to understand which part really need to be modifed, I think that makes sense. But in the case, we might need more comments rather than structural change.

Quoted 3 lines

This patchset clean up those filter code, move into filter.c and add header to expose filter func.(patch 2)

Again, I agree more code cleanup is needed. But I'm not sure if this is the right way. Especially, I'm unsure if it really need to have a dedicated file. To my humble eyes, it doesn't look like making code clearly easier to read compared to the current version. This is probably because I don't feel your scheme vs action definition is clear to understand. Neither it is deduplicating code. The patch adds lines more that deleted ones, according to its diffstat. For the reason, I don't see clear benefit of this change. I might be biased, having NIH (Not Invented Here) sindrome, or missing something. Please let me know if so.

Quoted 1 line

And add a new filter "workingset" to protect workingset page.

Can you explain expected use cases of this new filter type in more detail? Specifically, for what scheme action and under what situation this new filter type will be needed? If you have some test data for the use case and could share it, it would be very helpful for me to understand why it is needed.

Quoted 2 lines

Do we need this and cleanup it?

I think I cannot answer for now, and your further clarification and patient explanation could be helpful.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/tree/Documentation/mm/damon/design.rst?h=mm-everything-2023-09-20-19-38#n400

Thanks,
SJ

Quoted 22 lines

Huan Yang (2):
mm/damos/filter: Add workingset page filter
mm/damos/filter: Damos filter cleanup

 include/linux/damon.h    |  62 +-----------------
 mm/damon/Makefile        |   2 +-
 mm/damon/core-test.h     |   7 ++
 mm/damon/core.c          |  93 ++++-----------------------
 mm/damon/filter.c        | 135 +++++++++++++++++++++++++++++++++++++++
 mm/damon/filter.h        | 119 ++++++++++++++++++++++++++++++++++
 mm/damon/paddr.c         |  29 +++------
 mm/damon/reclaim.c       |  48 +++++++++++---
 mm/damon/sysfs-schemes.c |   1 +
 9 files changed, 325 insertions(+), 171 deletions(-)
 create mode 100644 mm/damon/filter.c
 create mode 100644 mm/damon/filter.h
Signature · 1 line
2.34.1
author
#4 lore raw

HI SJ,

Thanks for your patient response.

Quoted 8 lines

[Some people who received this message don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Hi Huan,

First of all, thank you for this patch. I have some high level comments and questions as below.

Quoted 8 lines
Quoted 3 lines · On Tue, 19 Sep 2023 17:52:33 +0800 Huan Yang <[email protected]> wrote:

Now damos support filter contains two type.
The first is scheme filter which will invoke each scheme apply, second is scheme action filter, which will filter out unwanted action.

IMHO, that's not clear definition of the types. Conceptually, all the filters are same. Nonetheless, they are having different behaviors because they are handled in different layer depending on which layer is more efficient to handle

Thanks for these instructions to help me understand the design idea of damos filter.

Quoted 9 lines

those[1].

I agree this is complicated and a bit verbose explanation, but I don't feel your scheme vs action definition is making it easier to understand.

Quoted 1 line

But this implement is scattered in different implementations

Indeed the implementation is scattered in core layer and the ops layer depending on the filter types. But I think this is needed for efficient handling of those.

IMO, some simple filter can have a common implement, like anon filter? And,

for some special type, each layer offer their own?

Quoted 6 lines
Quoted 2 lines

and hard to reuse or extend.

From your first patch, which extending the filter, the essential change for the
extension is adding another enum to 'enum damos_filter_type' (1 line) and addition of switch case in '__damos_pa_filter_out()' (3 lines). I don't think it looks too complicated. If you're saying it was hard to understand which

Yes, indeed.

Quoted 7 lines

part really need to be modifed, I think that makes sense. But in the case, we might need more comments rather than structural change.

Quoted 2 lines

This patchset clean up those filter code, move into filter.c and add header to expose filter func.(patch 2)

Again, I agree more code cleanup is needed. But I'm not sure if this is the right way. Especially, I'm unsure if it really need to have a dedicated file.

I think the filter code scatter into each layer may make code hard to reuse, other ops

may need anon filter or something. So, make code into a dedicated file may good?

(just my opinion.)

Quoted 3 lines

To my humble eyes, it doesn't look like making code clearly easier to read compared to the current version. This is probably because I don't feel your scheme vs action definition is clear to understand. Neither it is

Yes, indeed, current code not clean, the idea didn't take shape.

Quoted 2 lines

deduplicating code. The patch adds lines more that deleted ones, according to its diffstat. For the reason, I don't see clear benefit of this change.

In this code, maybe just a little benefit when other ops need to filter anon page? :)

Quoted 7 lines

I
might be biased, having NIH (Not Invented Here) sindrome, or missing something. Please let me know if so.

Quoted 1 line

And add a new filter "workingset" to protect workingset page.

Can you explain expected use cases of this new filter type in more detail? Specifically, for what scheme action and under what situation this new filter

IMO, page if marked as workingset, mean page in task's core workload(maybe have

seen the refault, and trigger refault protect). So, for lru sort or reclaim,

we'd better not touch it?(If any wrong, please let me know.)

Quoted 2 lines

type will be needed? If you have some test data for the use case and could share it, it would be very helpful for me to understand why it is needed.

Sorry, this type just from my knowledge of MM, have no test data.

For futher learn of DAMON, I'll try it.

Quoted 10 lines
Quoted 2 lines

Do we need this and cleanup it?

I think I cannot answer for now, and your further clarification and patient explanation could be helpful.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/tree/Documentation/mm/damon/design.rst?h=mm-everything-2023-09-20-19-38#n400

Thanks,
SJ

Thanks,

Huan

Quoted 22 lines
Quoted 22 lines

Huan Yang (2):
mm/damos/filter: Add workingset page filter
mm/damos/filter: Damos filter cleanup

  include/linux/damon.h    |  62 +-----------------
  mm/damon/Makefile        |   2 +-
  mm/damon/core-test.h     |   7 ++
  mm/damon/core.c          |  93 ++++-----------------------
  mm/damon/filter.c        | 135 +++++++++++++++++++++++++++++++++++++++
  mm/damon/filter.h        | 119 ++++++++++++++++++++++++++++++++++
  mm/damon/paddr.c         |  29 +++------
  mm/damon/reclaim.c       |  48 +++++++++++---
  mm/damon/sysfs-schemes.c |   1 +
  9 files changed, 325 insertions(+), 171 deletions(-)

create mode 100644 mm/damon/filter.c
create mode 100644 mm/damon/filter.h

--
2.34.1

maintainer
#5 lore raw

Hi Huan,

Quoted 3 lines · On Fri, 22 Sep 2023 02:54:58 +0000 杨欢 <[email protected]> wrote:

HI SJ,

Thanks for your patient response.

Happy to hear this kind acknowledgement :)

Quoted 23 lines
Quoted 8 lines

[Some people who received this message don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Hi Huan,

First of all, thank you for this patch. I have some high level comments and questions as below.

Quoted 8 lines
Quoted 3 lines · On Tue, 19 Sep 2023 17:52:33 +0800 Huan Yang <[email protected]> wrote:

Now damos support filter contains two type.
The first is scheme filter which will invoke each scheme apply, second is scheme action filter, which will filter out unwanted action.

IMHO, that's not clear definition of the types. Conceptually, all the filters are same. Nonetheless, they are having different behaviors because they are handled in different layer depending on which layer is more efficient to handle

Thanks for these instructions to help me understand the design idea of damos filter.

Quoted 5 lines

those[1].

I agree this is complicated and a bit verbose explanation, but I don't feel your scheme vs action definition is making it easier to understand.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

But this implement is scattered in different implementations

Indeed the implementation is scattered in core layer and the ops layer depending on the filter types. But I think this is needed for efficient handling of those.

IMO, some simple filter can have a common implement, like anon filter? And, for some special type, each layer offer their own?

Do you mean there could be two filter types that better to be handled in different layer for efficiency, but share common implementation? Could you please give me a more specific example?

Quoted 16 lines
Quoted 7 lines
Quoted 3 lines

and hard to reuse or extend.

From your first patch, which extending the filter, the essential change for the
extension is adding another enum to 'enum damos_filter_type' (1 line) and addition of switch case in '__damos_pa_filter_out()' (3 lines). I don't think it looks too complicated. If you're saying it was hard to understand which

Yes, indeed.

Quoted 7 lines

part really need to be modifed, I think that makes sense. But in the case, we might need more comments rather than structural change.

Quoted 2 lines

This patchset clean up those filter code, move into filter.c and add header to expose filter func.(patch 2)

Again, I agree more code cleanup is needed. But I'm not sure if this is the right way. Especially, I'm unsure if it really need to have a dedicated file.

Quoted 7 lines

I think the filter code scatter into each layer may make code hard to reuse, other ops

may need anon filter or something. So, make code into a dedicated file may good?

(just my opinion.)

Again, I'm not super confident about my understanding. But sounds like you're partly worrying about duplication of some code in each ops implementation (modules in same layer). Please correct me if I'm wrong, with specific, detailed and realistic examples.

If my guess is not that wrong, I can agree to the concern. Nevertheless, we already have a dedicated source file for such common code between ops implementaions, namely ops-common.c.

That said, we don't have duplicated DAMOS filters implementation code in multipe ops implementation at the moment. It could have such duplication in the future, but I think it's not too late to make such cleanup after or just before such duplication heppen. IOW, I'd suggest to not make changes for something that _might_ happen in future.

Quoted 9 lines
Quoted 4 lines

To my humble eyes, it doesn't look like making code clearly easier to read compared to the current version. This is probably because I don't feel your scheme vs action definition is clear to understand. Neither it is

Yes, indeed, current code not clean, the idea didn't take shape.

Quoted 2 lines

deduplicating code. The patch adds lines more that deleted ones, according to its diffstat. For the reason, I don't see clear benefit of this change.

In this code, maybe just a little benefit when other ops need to filter anon page? :)

As mentioned above, it's unclear when, how, and for what benefit we will support anon pages filter from vaddr. I'd again suggest to not make change only for future change that not clear if we really want to make for now.

Quoted 8 lines
Quoted 7 lines

I
might be biased, having NIH (Not Invented Here) sindrome, or missing something. Please let me know if so.

Quoted 1 line

And add a new filter "workingset" to protect workingset page.

Can you explain expected use cases of this new filter type in more detail? Specifically, for what scheme action and under what situation this new filter

Quoted 6 lines

IMO, page if marked as workingset, mean page in task's core workload(maybe have

seen the refault, and trigger refault protect). So, for lru sort or reclaim,

we'd better not touch it?(If any wrong, please let me know.)

Still unclear to me. Could I ask you more specific and detailed case?

Quoted 7 lines
Quoted 3 lines

type will be needed? If you have some test data for the use case and could share it, it would be very helpful for me to understand why it is needed.

Sorry, this type just from my knowledge of MM, have no test data.

For futher learn of DAMON, I'll try it.

Yes, that will be very helpful.

And from this point, I'm getting an impression that the purpose of this RFC is not for making a real change for specific use case that assumed to make real benefits, but just for getting opinions about some imaginable changes that _might_ be helpful, and _might_ be made in future. If so, making the point more clear would be helpful for me to give you opinion earlier. If that's the case, my opinion is this. I agree DAMON code has many rooms of improvement in terms of readability, so cleanup patches are welcome. Nevertheless, I'd prefer to make changes only when it is reasonable to expect it's providing _real_ improvement just after be applied, or at least very near and specific future.

Thanks,
SJ

Quoted 39 lines
Quoted 11 lines
Quoted 3 lines

Do we need this and cleanup it?

I think I cannot answer for now, and your further clarification and patient explanation could be helpful.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/tree/Documentation/mm/damon/design.rst?h=mm-everything-2023-09-20-19-38#n400

Thanks,
SJ

Thanks,

Huan

Quoted 22 lines
Quoted 22 lines

Huan Yang (2):
mm/damos/filter: Add workingset page filter
mm/damos/filter: Damos filter cleanup

  include/linux/damon.h    |  62 +-----------------
  mm/damon/Makefile        |   2 +-
  mm/damon/core-test.h     |   7 ++
  mm/damon/core.c          |  93 ++++-----------------------
  mm/damon/filter.c        | 135 +++++++++++++++++++++++++++++++++++++++
  mm/damon/filter.h        | 119 ++++++++++++++++++++++++++++++++++
  mm/damon/paddr.c         |  29 +++------
  mm/damon/reclaim.c       |  48 +++++++++++---
  mm/damon/sysfs-schemes.c |   1 +
  9 files changed, 325 insertions(+), 171 deletions(-)

create mode 100644 mm/damon/filter.c
create mode 100644 mm/damon/filter.h

--
2.34.1

author
#6 lore raw

HI SJ,

Quoted 33 lines

Hi Huan,

Quoted 3 lines · On Fri, 22 Sep 2023 02:54:58 +0000 杨欢 <[email protected]> wrote:

HI SJ,

Thanks for your patient response.

Happy to hear this kind acknowledgement :)

Quoted 24 lines
Quoted 8 lines

[Some people who received this message don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Hi Huan,

First of all, thank you for this patch. I have some high level comments and questions as below.

Quoted 8 lines
Quoted 3 lines · On Tue, 19 Sep 2023 17:52:33 +0800 Huan Yang <[email protected]> wrote:

Now damos support filter contains two type.
The first is scheme filter which will invoke each scheme apply, second is scheme action filter, which will filter out unwanted action.

IMHO, that's not clear definition of the types. Conceptually, all the filters are same. Nonetheless, they are having different behaviors because they are handled in different layer depending on which layer is more efficient to handle

Thanks for these instructions to help me understand the design idea of damos filter.

Quoted 6 lines

those[1].

I agree this is complicated and a bit verbose explanation, but I don't feel your scheme vs action definition is making it easier to understand.

Quoted 1 line

But this implement is scattered in different implementations

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Indeed the implementation is scattered in core layer and the ops layer depending on the filter types. But I think this is needed for efficient handling of those.

IMO, some simple filter can have a common implement, like anon filter? And, for some special type, each layer offer their own?

Do you mean there could be two filter types that better to be handled in different layer for efficiency, but share common implementation? Could you please give me a more specific example?

It's hard to offer a specific example due to current ops implement is so great to cover

much situation. Maybe Heterogeneous memory may add a new ops(just examples of

imprudence, like intel's or other network memory?) . And offer a common ops filter code

can may some simple type be reused.

Quoted 21 lines
Quoted 14 lines
Quoted 6 lines
Quoted 2 lines

and hard to reuse or extend.

From your first patch, which extending the filter, the essential change for the
extension is adding another enum to 'enum damos_filter_type' (1 line) and addition of switch case in '__damos_pa_filter_out()' (3 lines). I don't think it looks too complicated. If you're saying it was hard to understand which

Yes, indeed.

Quoted 7 lines

part really need to be modifed, I think that makes sense. But in the case, we might need more comments rather than structural change.

Quoted 2 lines

This patchset clean up those filter code, move into filter.c and add header to expose filter func.(patch 2)

Again, I agree more code cleanup is needed. But I'm not sure if this is the right way. Especially, I'm unsure if it really need to have a dedicated file.

Quoted 7 lines

I think the filter code scatter into each layer may make code hard to reuse, other ops

may need anon filter or something. So, make code into a dedicated file may good?

(just my opinion.)

Quoted 8 lines

Again, I'm not super confident about my understanding. But sounds like you're partly worrying about duplication of some code in each ops implementation (modules in same layer). Please correct me if I'm wrong, with specific, detailed and realistic examples.

If my guess is not that wrong, I can agree to the concern. Nevertheless, we already have a dedicated source file for such common code between ops implementaions, namely ops-common.c.

Yes, no need to split a single file to drive filter.

Quoted 6 lines

That said, we don't have duplicated DAMOS filters implementation code in multipe ops implementation at the moment. It could have such duplication in the future, but I think it's not too late to make such cleanup after or just before such duplication heppen. IOW, I'd suggest to not make changes for something that _might_ happen in future.

Yes, indeed. We needn't to make this right now.(That's the why RFC, meanwhile, my code is

not good.)

Quoted 20 lines
Quoted 9 lines
Quoted 4 lines

To my humble eyes, it doesn't look like making code clearly easier to read compared to the current version. This is probably because I don't feel your scheme vs action definition is clear to understand. Neither it is

Yes, indeed, current code not clean, the idea didn't take shape.

Quoted 2 lines

deduplicating code. The patch adds lines more that deleted ones, according to its diffstat. For the reason, I don't see clear benefit of this change.

In this code, maybe just a little benefit when other ops need to filter anon page? :)

As mentioned above, it's unclear when, how, and for what benefit we will support anon pages filter from vaddr. I'd again suggest to not make change only for future change that not clear if we really want to make for now.

Quoted 7 lines
Quoted 7 lines

I
might be biased, having NIH (Not Invented Here) sindrome, or missing something. Please let me know if so.

Quoted 1 line

And add a new filter "workingset" to protect workingset page.

Can you explain expected use cases of this new filter type in more detail? Specifically, for what scheme action and under what situation this new filter

Quoted 7 lines
Quoted 6 lines

IMO, page if marked as workingset, mean page in task's core workload(maybe have

seen the refault, and trigger refault protect). So, for lru sort or reclaim,

we'd better not touch it?(If any wrong, please let me know.)

Still unclear to me. Could I ask you more specific and detailed case?

I may have misunderstood, I suggest you just look:

Page workingset flag will mark to page when page need to deactive or refault and

find it already marked. In some memory path(migrate, reclaim or else.), touch

workingset page require special attention.(Enter psi mm stall or else)

So, I think help filter out this is helpful.(Of course, just thought experiment, no helpful data).

As, above and below. This RFC patch. :). I will share when get valuable data.

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

type will be needed? If you have some test data for the use case and could share it, it would be very helpful for me to understand why it is needed.

Sorry, this type just from my knowledge of MM, have no test data.

Quoted 7 lines
Quoted 1 line

For futher learn of DAMON, I'll try it.

Yes, that will be very helpful.

And from this point, I'm getting an impression that the purpose of this RFC is not for making a real change for specific use case that assumed to make real benefits, but just for getting opinions about some imaginable changes that _might_ be helpful, and _might_ be made in future. If so, making the point

Yes, I just learn DAMON a little time, and offer some thinking for this.

Quoted 5 lines

more clear would be helpful for me to give you opinion earlier. If that's the case, my opinion is this. I agree DAMON code has many rooms of improvement in terms of readability, so cleanup patches are welcome. Nevertheless, I'd prefer to make changes only when it is reasonable to expect it's providing _real_ improvement just after be applied, or at least very near and specific future.

Yes, keep this and change when we need indeed. :)

Quoted 4 lines

Thanks,
SJ

Thans,

Huan

Quoted 34 lines
Quoted 34 lines
Quoted 9 lines
Quoted 1 line

Do we need this and cleanup it?

I think I cannot answer for now, and your further clarification and patient explanation could be helpful.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/tree/Documentation/mm/damon/design.rst?h=mm-everything-2023-09-20-19-38#n400

Thanks,
SJ

Thanks,

Huan

Quoted 21 lines
Quoted 21 lines

Huan Yang (2):
mm/damos/filter: Add workingset page filter
mm/damos/filter: Damos filter cleanup

   include/linux/damon.h    |  62 +-----------------
   mm/damon/Makefile        |   2 +-
   mm/damon/core-test.h     |   7 ++
   mm/damon/core.c          |  93 ++++-----------------------
   mm/damon/filter.c        | 135 +++++++++++++++++++++++++++++++++++++++
   mm/damon/filter.h        | 119 ++++++++++++++++++++++++++++++++++
   mm/damon/paddr.c         |  29 +++------
   mm/damon/reclaim.c       |  48 +++++++++++---
   mm/damon/sysfs-schemes.c |   1 +
   9 files changed, 325 insertions(+), 171 deletions(-)

create mode 100644 mm/damon/filter.c
create mode 100644 mm/damon/filter.h

--
2.34.1

maintainer
#7 lore raw

Hi Huan,

Quoted 152 lines · On Mon, 25 Sep 2023 02:10:58 +0000 杨欢 <[email protected]> wrote:

HI SJ,

Quoted 33 lines

Hi Huan,

Quoted 3 lines · On Fri, 22 Sep 2023 02:54:58 +0000 杨欢 <[email protected]> wrote:

HI SJ,

Thanks for your patient response.

Happy to hear this kind acknowledgement :)

Quoted 24 lines
Quoted 8 lines

[Some people who received this message don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Hi Huan,

First of all, thank you for this patch. I have some high level comments and questions as below.

Quoted 8 lines
Quoted 3 lines · On Tue, 19 Sep 2023 17:52:33 +0800 Huan Yang <[email protected]> wrote:

Now damos support filter contains two type.
The first is scheme filter which will invoke each scheme apply, second is scheme action filter, which will filter out unwanted action.

IMHO, that's not clear definition of the types. Conceptually, all the filters are same. Nonetheless, they are having different behaviors because they are handled in different layer depending on which layer is more efficient to handle

Thanks for these instructions to help me understand the design idea of damos filter.

Quoted 6 lines

those[1].

I agree this is complicated and a bit verbose explanation, but I don't feel your scheme vs action definition is making it easier to understand.

Quoted 1 line

But this implement is scattered in different implementations

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Indeed the implementation is scattered in core layer and the ops layer depending on the filter types. But I think this is needed for efficient handling of those.

IMO, some simple filter can have a common implement, like anon filter? And, for some special type, each layer offer their own?

Do you mean there could be two filter types that better to be handled in different layer for efficiency, but share common implementation? Could you please give me a more specific example?

It's hard to offer a specific example due to current ops implement is so great to cover

much situation. Maybe Heterogeneous memory may add a new ops(just examples of

imprudence, like intel's or other network memory?) . And offer a common ops filter code

can may some simple type be reused.

Quoted 21 lines
Quoted 14 lines
Quoted 6 lines
Quoted 2 lines

and hard to reuse or extend.

From your first patch, which extending the filter, the essential change for the
extension is adding another enum to 'enum damos_filter_type' (1 line) and addition of switch case in '__damos_pa_filter_out()' (3 lines). I don't think it looks too complicated. If you're saying it was hard to understand which

Yes, indeed.

Quoted 7 lines

part really need to be modifed, I think that makes sense. But in the case, we might need more comments rather than structural change.

Quoted 2 lines

This patchset clean up those filter code, move into filter.c and add header to expose filter func.(patch 2)

Again, I agree more code cleanup is needed. But I'm not sure if this is the right way. Especially, I'm unsure if it really need to have a dedicated file.

Quoted 7 lines

I think the filter code scatter into each layer may make code hard to reuse, other ops

may need anon filter or something. So, make code into a dedicated file may good?

(just my opinion.)

Quoted 8 lines

Again, I'm not super confident about my understanding. But sounds like you're partly worrying about duplication of some code in each ops implementation (modules in same layer). Please correct me if I'm wrong, with specific, detailed and realistic examples.

If my guess is not that wrong, I can agree to the concern. Nevertheless, we already have a dedicated source file for such common code between ops implementaions, namely ops-common.c.

Yes, no need to split a single file to drive filter.

Quoted 6 lines

That said, we don't have duplicated DAMOS filters implementation code in multipe ops implementation at the moment. It could have such duplication in the future, but I think it's not too late to make such cleanup after or just before such duplication heppen. IOW, I'd suggest to not make changes for something that _might_ happen in future.

Yes, indeed. We needn't to make this right now.(That's the why RFC, meanwhile, my code is

not good.)

Quoted 20 lines
Quoted 9 lines
Quoted 4 lines

To my humble eyes, it doesn't look like making code clearly easier to read compared to the current version. This is probably because I don't feel your scheme vs action definition is clear to understand. Neither it is

Yes, indeed, current code not clean, the idea didn't take shape.

Quoted 2 lines

deduplicating code. The patch adds lines more that deleted ones, according to its diffstat. For the reason, I don't see clear benefit of this change.

In this code, maybe just a little benefit when other ops need to filter anon page? :)

As mentioned above, it's unclear when, how, and for what benefit we will support anon pages filter from vaddr. I'd again suggest to not make change only for future change that not clear if we really want to make for now.

Quoted 7 lines
Quoted 7 lines

I
might be biased, having NIH (Not Invented Here) sindrome, or missing something. Please let me know if so.

Quoted 1 line

And add a new filter "workingset" to protect workingset page.

Can you explain expected use cases of this new filter type in more detail? Specifically, for what scheme action and under what situation this new filter

Quoted 7 lines
Quoted 6 lines

IMO, page if marked as workingset, mean page in task's core workload(maybe have

seen the refault, and trigger refault protect). So, for lru sort or reclaim,

we'd better not touch it?(If any wrong, please let me know.)

Still unclear to me. Could I ask you more specific and detailed case?

I may have misunderstood, I suggest you just look:

Page workingset flag will mark to page when page need to deactive or refault and

find it already marked. In some memory path(migrate, reclaim or else.), touch

workingset page require special attention.(Enter psi mm stall or else)

So, I think help filter out this is helpful.(Of course, just thought experiment, no helpful data).

As, above and below. This RFC patch. :). I will share when get valuable data.

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

type will be needed? If you have some test data for the use case and could share it, it would be very helpful for me to understand why it is needed.

Sorry, this type just from my knowledge of MM, have no test data.

Quoted 7 lines
Quoted 1 line

For futher learn of DAMON, I'll try it.

Yes, that will be very helpful.

And from this point, I'm getting an impression that the purpose of this RFC is not for making a real change for specific use case that assumed to make real benefits, but just for getting opinions about some imaginable changes that _might_ be helpful, and _might_ be made in future. If so, making the point

Quoted 7 lines

Yes, I just learn DAMON a little time, and offer some thinking for this.

Quoted 5 lines

more clear would be helpful for me to give you opinion earlier. If that's the case, my opinion is this. I agree DAMON code has many rooms of improvement in terms of readability, so cleanup patches are welcome. Nevertheless, I'd prefer to make changes only when it is reasonable to expect it's providing _real_ improvement just after be applied, or at least very near and specific future.

Yes, keep this and change when we need indeed. :)

Ok, makes sense. Let's do further discussion after some more data and realistic detailed example be available. :) Looking forward to!

Thanks,
SJ

Quoted 43 lines
Quoted 4 lines

Thanks,
SJ

Thans,

Huan

Quoted 34 lines
Quoted 34 lines
Quoted 9 lines
Quoted 1 line

Do we need this and cleanup it?

I think I cannot answer for now, and your further clarification and patient explanation could be helpful.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/tree/Documentation/mm/damon/design.rst?h=mm-everything-2023-09-20-19-38#n400

Thanks,
SJ

Thanks,

Huan

Quoted 21 lines
Quoted 21 lines

Huan Yang (2):
mm/damos/filter: Add workingset page filter
mm/damos/filter: Damos filter cleanup

   include/linux/damon.h    |  62 +-----------------
   mm/damon/Makefile        |   2 +-
   mm/damon/core-test.h     |   7 ++
   mm/damon/core.c          |  93 ++++-----------------------
   mm/damon/filter.c        | 135 +++++++++++++++++++++++++++++++++++++++
   mm/damon/filter.h        | 119 ++++++++++++++++++++++++++++++++++
   mm/damon/paddr.c         |  29 +++------
   mm/damon/reclaim.c       |  48 +++++++++++---
   mm/damon/sysfs-schemes.c |   1 +
   9 files changed, 325 insertions(+), 171 deletions(-)

create mode 100644 mm/damon/filter.c
create mode 100644 mm/damon/filter.h

--
2.34.1