Skip to contentSkip to conversation

mm/damon: add a DAMOS filter type for page granularity access recheck

RFC PATCH In review SeongJae Park 2 participants 6 messages Mar 11, 2024 – Apr 9, 2024

Subsystems (2)
lore
Share & tools
lore

SeongJae Park

author maintainer
#0 lore raw

v2 cover mm/damon: add a DAMOS filter type for page granularity access recheck

Changes from RFC v1
(https://lore.kernel.org/r/[email protected])
- Rename __damon_pa_young() to damon_folio_young_one()
- Mark the folio as old from the filter for the next check

DAMON allows users to specify desired ranges of overhead and accuracy of the monitoring, and do its best effort to make most lightweight and accurate results. A recent discussion for tiered memory management support from DAMON[1] revealed that the best effort accuracy may not suffice in some use cases, while increasing the minimum accuracy can incur too high overhead. The discussion further concluded finding memory regions of specific access pattern via DAMON first, and then double checking the access of the region again in finer granularity could help increasing the accuracy while keeping the low overhead.

Add a new type of DAMOS filter, namely YOUNG for such a case. Like anon and memcg, the type of filter is applied to each page of the memory region of DAMOS target memory region, and check if the page is accessed since the last check. Because this is a filter type that applied in page granularity, the support depends on DAMON operations set. Because there are expected usages of this filter for physical address space based DAMOS usage[1], implement the support for only DAMON operations set for the physical address space, paddr.

[1] https://lore.kernel.org/r/[email protected]

SeongJae Park (4):
mm/damon/paddr: implement damon_folio_young()
mm/damon/paddr: implement damon_folio_mkold()
mm/damon: add DAMOS filter type YOUNG
mm/damon/paddr: support DAMOS filter type YOUNG

 include/linux/damon.h    |  2 ++
 mm/damon/paddr.c         | 64 +++++++++++++++++++++++++---------------
 mm/damon/sysfs-schemes.c |  1 +
 3 files changed, 43 insertions(+), 24 deletions(-)

base-commit: 13043cde46a19e72d37965b67b74e564623f65e7

Signature · 1 line
2.39.2
author maintainer
#1 lore raw

RFC PATCH v2 1/4 mm/damon/paddr: implement damon_folio_young()

damon_pa_young() receives physical address, get the folio covering the address, and show if the folio is accessed since the last check. Split the internal logic for checking access to the given folio, for future reuse of the logic from code that already got the folio of the address of the question. Also, change the rmap walker function's name from __damon_pa_young() to damon_folio_young_one(), for consistent naming.

---
 mm/damon/paddr.c | 32 +++++++++++++++++++-------------
 1 file changed, 19 insertions(+), 13 deletions(-)
mm/damon/paddr.c +19 -13
index 5e6dc312072c..25c3ba2a9eaf 100644
@@ -79,8 +79,8 @@ static void damon_pa_prepare_access_checks(struct damon_ctx *ctx)
 	}
 }
 
-static bool __damon_pa_young(struct folio *folio, struct vm_area_struct *vma,
-		unsigned long addr, void *arg)
+static bool damon_folio_young_one(struct folio *folio,
+		struct vm_area_struct *vma, unsigned long addr, void *arg)
 {
 	bool *accessed = arg;
 	DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, addr, 0);
@@ -111,38 +111,44 @@ static bool __damon_pa_young(struct folio *folio, struct vm_area_struct *vma,
 	return *accessed == false;
 }
 
-static bool damon_pa_young(unsigned long paddr, unsigned long *folio_sz)
+static bool damon_folio_young(struct folio *folio)
 {
-	struct folio *folio = damon_get_folio(PHYS_PFN(paddr));
 	bool accessed = false;
 	struct rmap_walk_control rwc = {
 		.arg = &accessed,
-		.rmap_one = __damon_pa_young,
+		.rmap_one = damon_folio_young_one,
 		.anon_lock = folio_lock_anon_vma_read,
 	};
 	bool need_lock;
 
-	if (!folio)
-		return false;
-
 	if (!folio_mapped(folio) || !folio_raw_mapping(folio)) {
 		if (folio_test_idle(folio))
-			accessed = false;
+			return false;
 		else
-			accessed = true;
-		goto out;
+			return true;
 	}
 
 	need_lock = !folio_test_anon(folio) || folio_test_ksm(folio);
 	if (need_lock && !folio_trylock(folio))
-		goto out;
+		return false;
 
 	rmap_walk(folio, &rwc);
 
 	if (need_lock)
 		folio_unlock(folio);
 
-out:
+	return accessed;
+}
+
+static bool damon_pa_young(unsigned long paddr, unsigned long *folio_sz)
+{
+	struct folio *folio = damon_get_folio(PHYS_PFN(paddr));
+	bool accessed;
+
+	if (!folio)
+		return false;
+
+	accessed = damon_folio_young(folio);
 	*folio_sz = folio_size(folio);
 	folio_put(folio);
 	return accessed;
Signature · 1 line
2.39.2
author maintainer
#2 lore raw

RFC PATCH v2 2/4 mm/damon/paddr: implement damon_folio_mkold()

In reply to SeongJae Park

damon_pa_mkold() receives a physical address, finds the folio covering the address, and makes the folio as old. Split the internal logic for checking access to the given folio, for future reuse of the logic. Also, change the name of the rmap walker from __damon_pa_mkold() to damon_folio_mkold_one() for more consistent naming.

---
 mm/damon/paddr.c | 27 ++++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)
mm/damon/paddr.c +16 -11
index 25c3ba2a9eaf..310b803c6277 100644
@@ -16,8 +16,8 @@
 #include "../internal.h"
 #include "ops-common.h"
 
-static bool __damon_pa_mkold(struct folio *folio, struct vm_area_struct *vma,
-		unsigned long addr, void *arg)
+static bool damon_folio_mkold_one(struct folio *folio,
+		struct vm_area_struct *vma, unsigned long addr, void *arg)
 {
 	DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, addr, 0);
 
@@ -31,33 +31,38 @@ static bool __damon_pa_mkold(struct folio *folio, struct vm_area_struct *vma,
 	return true;
 }
 
-static void damon_pa_mkold(unsigned long paddr)
+static void damon_folio_mkold(struct folio *folio)
 {
-	struct folio *folio = damon_get_folio(PHYS_PFN(paddr));
 	struct rmap_walk_control rwc = {
-		.rmap_one = __damon_pa_mkold,
+		.rmap_one = damon_folio_mkold_one,
 		.anon_lock = folio_lock_anon_vma_read,
 	};
 	bool need_lock;
 
-	if (!folio)
-		return;
-
 	if (!folio_mapped(folio) || !folio_raw_mapping(folio)) {
 		folio_set_idle(folio);
-		goto out;
+		return;
 	}
 
 	need_lock = !folio_test_anon(folio) || folio_test_ksm(folio);
 	if (need_lock && !folio_trylock(folio))
-		goto out;
+		return;
 
 	rmap_walk(folio, &rwc);
 
 	if (need_lock)
 		folio_unlock(folio);
 
-out:
+}
+
+static void damon_pa_mkold(unsigned long paddr)
+{
+	struct folio *folio = damon_get_folio(PHYS_PFN(paddr));
+
+	if (!folio)
+		return;
+
+	damon_folio_mkold(folio);
 	folio_put(folio);
 }
 
Signature · 1 line
2.39.2
author maintainer
#3 lore raw

RFC PATCH v2 3/4 mm/damon: add DAMOS filter type YOUNG

In reply to SeongJae Park

Define yet another DAMOS filter type, YOUNG. Like anon and memcg, the type of filter will be applied to each page in the memory region, and check if the page is accessed since the last check.

Note that this commit is only defining the type. Implementation of it should be made on DAMON operations sets. A couple of commits for the implementation on 'paddr' DAMON operations set will follow.

---
 include/linux/damon.h    | 2 ++
 mm/damon/sysfs-schemes.c | 1 +
 2 files changed, 3 insertions(+)
include/linux/damon.h +2 -0
index 886d07294f4e..f7da65e1ac04 100644
@@ -297,6 +297,7 @@ struct damos_stat {
  * enum damos_filter_type - Type of memory for &struct damos_filter
  * @DAMOS_FILTER_TYPE_ANON:	Anonymous pages.
  * @DAMOS_FILTER_TYPE_MEMCG:	Specific memcg's pages.
+ * @DAMOS_FILTER_TYPE_YOUNG:	Recently accessed pages.
  * @DAMOS_FILTER_TYPE_ADDR:	Address range.
  * @DAMOS_FILTER_TYPE_TARGET:	Data Access Monitoring target.
  * @NR_DAMOS_FILTER_TYPES:	Number of filter types.
@@ -315,6 +316,7 @@ struct damos_stat {
 enum damos_filter_type {
 	DAMOS_FILTER_TYPE_ANON,
 	DAMOS_FILTER_TYPE_MEMCG,
+	DAMOS_FILTER_TYPE_YOUNG,
 	DAMOS_FILTER_TYPE_ADDR,
 	DAMOS_FILTER_TYPE_TARGET,
 	NR_DAMOS_FILTER_TYPES,
mm/damon/sysfs-schemes.c +1 -0
index 53a90ac678fb..bea5bc52846a 100644
@@ -343,6 +343,7 @@ static struct damon_sysfs_scheme_filter *damon_sysfs_scheme_filter_alloc(void)
 static const char * const damon_sysfs_scheme_filter_type_strs[] = {
 	"anon",
 	"memcg",
+	"young",
 	"addr",
 	"target",
 };
Signature · 1 line
2.39.2
author maintainer
#4 lore raw

RFC PATCH v2 4/4 mm/damon/paddr: support DAMOS filter type YOUNG

In reply to SeongJae Park

DAMOS filter of type YOUNG is defined, but not yet implemented by any DAMON operations set. Add the implementation to the DAMON operations set for the physical address space, paddr.

---
 mm/damon/paddr.c | 5 +++++
 1 file changed, 5 insertions(+)
mm/damon/paddr.c +5 -0
index 310b803c6277..5685ba485097 100644
@@ -214,6 +214,11 @@ static bool __damos_pa_filter_out(struct damos_filter *filter,
 			matched = filter->memcg_id == mem_cgroup_id(memcg);
 		rcu_read_unlock();
 		break;
+	case DAMOS_FILTER_TYPE_YOUNG:
+		matched = damon_folio_young(folio);
+		if (matched)
+			damon_folio_mkold(folio);
+		break;
 	default:
 		break;
 	}
Signature · 1 line
2.39.2
Tested-by on the v2 series
#5 lore raw
In reply to SeongJae Park

Hi SeongJae,

Quoted 24 lines · On Mon, 11 Mar 2024 13:45:41 -0700 SeongJae Park <[email protected]> wrote:

Changes from RFC v1
(https://lore.kernel.org/r/[email protected])
- Rename __damon_pa_young() to damon_folio_young_one()
- Mark the folio as old from the filter for the next check

DAMON allows users to specify desired ranges of overhead and accuracy of the monitoring, and do its best effort to make most lightweight and accurate results. A recent discussion for tiered memory management support from DAMON[1] revealed that the best effort accuracy may not suffice in some use cases, while increasing the minimum accuracy can incur too high overhead. The discussion further concluded finding memory regions of specific access pattern via DAMON first, and then double checking the access of the region again in finer granularity could help increasing the accuracy while keeping the low overhead.

Add a new type of DAMOS filter, namely YOUNG for such a case. Like anon and memcg, the type of filter is applied to each page of the memory region of DAMOS target memory region, and check if the page is accessed since the last check. Because this is a filter type that applied in page granularity, the support depends on DAMON operations set. Because there are expected usages of this filter for physical address space based DAMOS usage[1], implement the support for only DAMON operations set for the physical address space, paddr.

Quoted 8 lines

[1] https://lore.kernel.org/r/[email protected]

SeongJae Park (4):
mm/damon/paddr: implement damon_folio_young()
mm/damon/paddr: implement damon_folio_mkold()
mm/damon: add DAMOS filter type YOUNG
mm/damon/paddr: support DAMOS filter type YOUNG

I have tested your patches for evaluation of [1] and it all works fine. Thanks very much for doing this work!

[1] https://lore.kernel.org/damon/[email protected]

Quoted 10 lines
 include/linux/damon.h    |  2 ++
 mm/damon/paddr.c         | 64 +++++++++++++++++++++++++---------------
 mm/damon/sysfs-schemes.c |  1 +
 3 files changed, 43 insertions(+), 24 deletions(-)

base-commit: 13043cde46a19e72d37965b67b74e564623f65e7

Signature · 1 line
2.39.2