KSM: support smart-scan feature
v2 cover KSM: support smart-scan feature
This patch series makes two changes:
- Disable the smart-scan feature for ksm01 - ksm04
This is necessary to make sure that the volatile metrics have the expected counts.
- Add a new test for the smart-scan feature
The new test verifies that the smart-scan feature skips pages when it has been enabled for a VMA.
Versions:
- V2:
- Disable smart_scan in the test setup structure
- Remove the changes in create_same_memory()
- Add the new testcase ksm07 for the smart scan test
Stefan Roesch (2):
mem: disable KSM smart scan for ksm tests
add ksm test for smart-scan feature
testcases/kernel/mem/.gitignore | 1 + testcases/kernel/mem/include/mem.h | 1 + testcases/kernel/mem/ksm/ksm01.c | 2 + testcases/kernel/mem/ksm/ksm02.c | 2 + testcases/kernel/mem/ksm/ksm03.c | 2 + testcases/kernel/mem/ksm/ksm04.c | 2 + testcases/kernel/mem/ksm/ksm07.c | 69 +++++++++++++++++++++++++++++ testcases/kernel/mem/lib/mem.c | 70 ++++++++++++++++++++++++++++++ 8 files changed, 149 insertions(+) create mode 100644 testcases/kernel/mem/ksm/ksm07.c
base-commit: 8c89ef3d451087ed6e18750bd5eedd10e5ab3d2e
Signature · 1 line
2.39.3
PATCH v2 1/2 mem: disable KSM smart scan for ksm tests
This disables the "smart scan" KSM feature to make sure that the volatile count remains at 0.
--- testcases/kernel/mem/ksm/ksm01.c | 2 ++ testcases/kernel/mem/ksm/ksm02.c | 2 ++ testcases/kernel/mem/ksm/ksm03.c | 2 ++ testcases/kernel/mem/ksm/ksm04.c | 2 ++ 4 files changed, 8 insertions(+)
testcases/kernel/mem/ksm/ksm01.c +2 -0
@@ -86,6 +86,8 @@ static struct tst_test test = { TST_SR_SKIP_MISSING | TST_SR_TCONF_RO}, {"/sys/kernel/mm/ksm/merge_across_nodes" ,"1" , TST_SR_SKIP_MISSING | TST_SR_TCONF_RO}, + {"/sys/kernel/mm/ksm/smart_scan", "0", + TST_SR_SKIP_MISSING | TST_SR_TBROK_RO}, {} }, .needs_kconfigs = (const char *const[]){
testcases/kernel/mem/ksm/ksm02.c +2 -0
@@ -107,6 +107,8 @@ static struct tst_test test = { TST_SR_SKIP_MISSING | TST_SR_TCONF_RO}, {"/sys/kernel/mm/ksm/merge_across_nodes" ,"1" , TST_SR_SKIP_MISSING | TST_SR_TCONF_RO}, + {"/sys/kernel/mm/ksm/smart_scan", "0", + TST_SR_SKIP_MISSING | TST_SR_TBROK_RO}, {} }, .needs_kconfigs = (const char *const[]){
testcases/kernel/mem/ksm/ksm03.c +2 -0
@@ -89,6 +89,8 @@ static struct tst_test test = { TST_SR_SKIP_MISSING | TST_SR_TCONF_RO}, {"/sys/kernel/mm/ksm/merge_across_nodes" ,"1" , TST_SR_SKIP_MISSING | TST_SR_TCONF_RO}, + {"/sys/kernel/mm/ksm/smart_scan", "0", + TST_SR_SKIP_MISSING | TST_SR_TBROK_RO}, {} }, .needs_kconfigs = (const char *const[]){
testcases/kernel/mem/ksm/ksm04.c +2 -0
@@ -109,6 +109,8 @@ static struct tst_test test = { TST_SR_SKIP_MISSING | TST_SR_TCONF_RO}, {"/sys/kernel/mm/ksm/merge_across_nodes" ,"1" , TST_SR_SKIP_MISSING | TST_SR_TCONF_RO}, + {"/sys/kernel/mm/ksm/smart_scan", "0", + TST_SR_SKIP_MISSING | TST_SR_TBROK_RO}, {} }, .needs_kconfigs = (const char *const[]){
Signature · 1 line
2.39.3
Re: [PATCH v2 1/2] mem: disable KSM smart scan for ksm tests
Hi Stefan,
Quoted 5 lines
This disables the "smart scan" KSM feature to make sure that the volatile count remains at 0.
Signed-off-by Stefan Roesch <[email protected]>
Quoted 7 lines
Reported-by kernel test robot <[email protected]>--- testcases/kernel/mem/ksm/ksm01.c | 2 ++ testcases/kernel/mem/ksm/ksm02.c | 2 ++ testcases/kernel/mem/ksm/ksm03.c | 2 ++ testcases/kernel/mem/ksm/ksm04.c | 2 ++
Li suggested in v1, that also ksm0[56].c should disable smart_scan (ksm05.c is questionable, but since you prepared ksm07 it can IMHO be disable in both ksm0[56].c).
https://lore.kernel.org/ltp/CAEemH2fqamX720diM1N+iN9a8HM30_5sHg8V0EMHgHdrh3iZPw@mail.gmail.com/
With that, you can add:
Kind regards,
Petr
Re: [PATCH v2 1/2] mem: disable KSM smart scan for ksm tests
Quoted 17 lines · Petr Vorel
Hi Stefan,
Quoted 5 lines
This disables the "smart scan" KSM feature to make sure that the volatile count remains at 0.
Signed-off-by Stefan Roesch <[email protected]>Quoted 7 lines
Reported-by kernel test robot <[email protected]>--- testcases/kernel/mem/ksm/ksm01.c | 2 ++ testcases/kernel/mem/ksm/ksm02.c | 2 ++ testcases/kernel/mem/ksm/ksm03.c | 2 ++ testcases/kernel/mem/ksm/ksm04.c | 2 ++Li suggested in v1, that also ksm0[56].c should disable
smart_scan(ksm05.cis questionable, but since you prepared ksm07 it can IMHO be disable in both ksm0[56].c).
Quoted 7 lines
https://lore.kernel.org/ltp/CAEemH2fqamX720diM1N+iN9a8HM30_5sHg8V0EMHgHdrh3iZPw@mail.gmail.com/
With that, you can add:
Reviewed-by Petr Vorel <[email protected]>Kind regards,
Petr
The next version will also disable smart scan for ksm05 and ksm06
Re: [PATCH v2 1/2] mem: disable KSM smart scan for ksm tests
In reply to Petr VorelQuoted 17 lines · Petr Vorel
Hi Stefan,
Quoted 5 lines
This disables the "smart scan" KSM feature to make sure that the volatile count remains at 0.
Signed-off-by Stefan Roesch <[email protected]>Quoted 7 lines
Reported-by kernel test robot <[email protected]>--- testcases/kernel/mem/ksm/ksm01.c | 2 ++ testcases/kernel/mem/ksm/ksm02.c | 2 ++ testcases/kernel/mem/ksm/ksm03.c | 2 ++ testcases/kernel/mem/ksm/ksm04.c | 2 ++Li suggested in v1, that also ksm0[56].c should disable
smart_scan(ksm05.cis questionable, but since you prepared ksm07 it can IMHO be disable in both ksm0[56].c).
Quoted 7 lines
https://lore.kernel.org/ltp/CAEemH2fqamX720diM1N+iN9a8HM30_5sHg8V0EMHgHdrh3iZPw@mail.gmail.com/
With that, you can add:
Reviewed-by Petr Vorel <[email protected]>Kind regards,
Petr
The next version will also disable smart scan for ksm05 and ksm06
PATCH v2 2/2 add ksm test for smart-scan feature
In reply to Stefan RoeschThis adds a new ksm (kernel samepage merging) test to evaluate the new smart scan feature. It allocates a page and fills it with 'a' characters. It captures the pages_skipped counter, waits for a few iterations and captures the pages_skipped counter again. The expectation is that over 50% of the page scans are skipped (There is only one page that has KSM enabled and it gets scanned during each iteration and it cannot be de-duplicated).
--- testcases/kernel/mem/.gitignore | 1 + testcases/kernel/mem/include/mem.h | 1 + testcases/kernel/mem/ksm/ksm07.c | 69 +++++++++++++++++++++++++++++ testcases/kernel/mem/lib/mem.c | 70 ++++++++++++++++++++++++++++++ 4 files changed, 141 insertions(+) create mode 100644 testcases/kernel/mem/ksm/ksm07.c
testcases/kernel/mem/.gitignore +1 -0
@@ -53,6 +53,7 @@ /ksm/ksm04 /ksm/ksm05 /ksm/ksm06 +/ksm/ksm07 /mem/mem02 /mmapstress/mmap-corruption01 /mmapstress/mmapstress01
testcases/kernel/mem/include/mem.h +1 -0
@@ -48,6 +48,7 @@ void testoom(int mempolicy, int lite, int retcode, int allow_sigkill);/* KSM */ void create_same_memory (int size,int num,int unit); +void create_memory_for_smartscan (void );void test_ksm_merge_across_nodes (unsigned long nr_pages);void ksm_group_check (int run,int pg_shared,int pg_sharing,int pg_volatile,int pg_unshared,int sleep_msecs,int pages_to_scan);
testcases/kernel/mem/ksm/ksm07.c new file +69 -0
69 lines not on this page · raw message
testcases/kernel/mem/lib/mem.c +70 -0
76 lines not on this page · raw message
Signature · 1 line
2.39.3
Re: [PATCH v2 2/2] add ksm test for smart-scan feature
Hi Stefan,
Quoted 4 lines
testcases/kernel/mem/.gitignore | 1 + testcases/kernel/mem/include/mem.h | 1 + testcases/kernel/mem/ksm/ksm07.c | 69 +++++++++++++++++++++++++++++ testcases/kernel/mem/lib/mem.c | 70 ++++++++++++++++++++++++++++++
You need to add 'ksm07 ksm07' line to runtest/mm. If it's the only error, we can fix this before merge.
Kind regards,
Petr
Re: [PATCH v2 2/2] add ksm test for smart-scan feature
Quoted 6 lines · Petr Vorel
Hi Stefan,
Quoted 4 lines
testcases/kernel/mem/.gitignore | 1 + testcases/kernel/mem/include/mem.h | 1 + testcases/kernel/mem/ksm/ksm07.c | 69 +++++++++++++++++++++++++++++ testcases/kernel/mem/lib/mem.c | 70 ++++++++++++++++++++++++++++++
Quoted 6 lines
You need to add 'ksm07 ksm07' line to runtest/mm. If it's the only error, we can fix this before merge.
Kind regards,
Petr
The next version will also update runtest/mm.
Re: [PATCH v2 2/2] add ksm test for smart-scan feature
In reply to Stefan RoeschHi Stefan,
...
Quoted 5 lines
+++
b/testcases/kernel/mem/ksm/ksm07.ctestcases/kernel/mem/ksm/ksm07.c new file +3
-0@@ -0,0 +1,69 @@ +/* +* Copyright (C) 2010-2017 Red Hat, Inc. +*
testcases/kernel/mem/ksm/ksm07.c line 12
Quoted 10 lines
testcases/kernel/mem/ksm/ksm07.c new file +9
-0+* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See +* the GNU General Public License for more details.
NOTE: we use SPDX instead of verbose GPL (see ksm06.c).
Quoted 1 line
+ *
NOTE: we have special doc format which starts like this (see ksm06.c):
/*\
* [Description]
*
* ...
Quoted 7 lines
+ * Kernel Samepage Merging (KSM)
+ *
+ * This adds a new ksm (kernel samepage merging) test to evaluate the new
+ * smart scan feature. It allocates a page and fills it with 'a'
+ * characters. It captures thepages_skippedcounter, waits for a few
+ * iterations and captures thepages_skippedcounter again. The expectation
+ * is that over 50% of the page scans are skipped (There is only one page
Quoted 8 lines
+ * that has KSM enabled and it gets scanned during each iteration and it
+ * cannot be de-duplicated).
+ *
+ * Prerequisites:
+ *
+ * 1) ksm and ksmtuned daemons need to be disabled. Otherwise, it could
+ * distrub the testing as they also change some ksm tunables depends
+ * on current workloads.
Hm, we don't have to any helper in LTP API to detect this, so at least we document this.
Quoted 2 lines
+ *
+ */
The result is then uploaded:
https://github.com/linux-test-project/ltp/releases/download/20230929/metadata.20230929.html
Therefore please use:
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (C) 2010-2017 Red Hat, Inc.
*/
/*\
* [Description]
*
* Kernel Samepage Merging (KSM)
*
* This adds a new ksm (kernel samepage merging) test to evaluate the new
* smart scan feature. It allocates a page and fills it with 'a'
* characters. It captures the pages_skipped counter, waits for a few
* iterations and captures the pages_skipped counter again. The expectation
* is that over 50% of the page scans are skipped (There is only one page
* that has KSM enabled and it gets scanned during each iteration and it
* cannot be de-duplicated).
*
* Prerequisites:
*
* 1) ksm and ksmtuned daemons need to be disabled. Otherwise, it could
* distrub the testing as they also change some ksm tunables depends
* on current workloads.
*/
Quoted 8 lines
testcases/kernel/mem/ksm/ksm07.c new file +8
-0+ +#include <sys/types.h> +#include <sys/mman.h> +#include <sys/stat.h> +#include <sys/wait.h> +#include <errno.h> +#include <fcntl.h> +#include <signal.h>
testcases/kernel/mem/ksm/ksm07.c line 48
Quoted 10 lines
testcases/kernel/mem/ksm/ksm07.c new file +10
-0+#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> +#include "../include/mem.h" +#include "ksm_common.h" + +static void verify_ksm (void ) +{ +create_memory_for_smartscan ();
I wonder, if we reusable create_memory_for_smartscan() later. Maybe it should be put into ksm07 for the start.
Also, the test needs to run on older kernel - exit with TCONF (which is not considered as a failure) instead of TBROK which does now:
mem.c:488: TBROK: Failed to open FILE '/sys/kernel/mm/ksm/pages_skipped' for reading: ENOENT (2)
If the testing code stays in testcases/kernel/mem/lib/mem.c, you will have to stat() it. But if it's really just this test specific and you move it to ksm07.c, then you can simply add the handling via .save_restore.
Quoted 11 lines
testcases/kernel/mem/ksm/ksm07.c new file +11
-0+} + +static struct tst_test test = { + .needs_root =1 , + .forks_child =1 , + .options = (struct tst_option []) { + {} + }, + .save_restore = (const struct tst_path_val []) { + {"/sys/kernel/mm/ksm/run" ,NULL ,TST_SR_TBROK }, + {"/sys/kernel/mm/ksm/sleep_millisecs" ,NULL ,TST_SR_TBROK },
testcases/kernel/mem/ksm/ksm07.c line 69
Quoted 10 lines
testcases/kernel/mem/ksm/ksm07.c new file +10
-0+ {"/sys/kernel/mm/ksm/smart_scan" ,"1" , + TST_SR_SKIP_MISSING | TST_SR_TBROK_RO}, + {} + }, + .needs_kconfigs = (const char *const[]){ +"CONFIG_KSM=y" , +NULL + }, + .test_all = verify_ksm, +};
Also, there are missing static:
$ cd testcases/kernel/mem/ksm/; make check-ksm07
CHECK testcases/kernel/mem/ksm/ksm07.c
ksm07.c:1: WARNING: Missing or malformed SPDX-License-Identifier tag in line 1
make: [../../../../include/mk/rules.mk:56: check-ksm07] Error 1 (ignored)
ksm07.c: note: in included file:
ksm_common.h:14:5: warning: Symbol 'size' has no prototype or library ('tst_') prefix. Should it be static?
ksm_common.h:14:29: warning: Symbol 'num' has no prototype or library ('tst_') prefix. Should it be static?
ksm_common.h:14:38: warning: Symbol 'unit' has no prototype or library ('tst_') prefix. Should it be static?
ksm_common.h:15:6: warning: Symbol 'opt_sizestr' has no prototype or library ('tst_') prefix. Should it be static?
ksm_common.h:15:20: warning: Symbol 'opt_numstr' has no prototype or library ('tst_') prefix. Should it be static?
ksm_common.h:15:33: warning: Symbol 'opt_unitstr' has no prototype or library ('tst_') prefix. Should it be static?Kind regards,
Petr
Re: [PATCH v2 2/2] add ksm test for smart-scan feature
Quoted 66 lines · Petr Vorel
Hi Stefan,
...
Quoted 5 lines
+++
b/testcases/kernel/mem/ksm/ksm07.ctestcases/kernel/mem/ksm/ksm07.c new file +3
-0@@ -0,0 +1,69 @@ +/* +* Copyright (C) 2010-2017 Red Hat, Inc. +* testcases/kernel/mem/ksm/ksm07.c line 12
Quoted 10 lines
testcases/kernel/mem/ksm/ksm07.c new file +9
-0+* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See +* the GNU General Public License for more details. NOTE: we use SPDX instead of verbose GPL (see
ksm06.c).Quoted 1 line
+ *
NOTE: we have special doc format which starts like this (see
ksm06.c):
/*\
* [Description]
*
* ...Quoted 7 lines
+ * Kernel Samepage Merging (KSM)
+ *
+ * This adds a new ksm (kernel samepage merging) test to evaluate the new
+ * smart scan feature. It allocates a page and fills it with 'a'
+ * characters. It captures thepages_skippedcounter, waits for a few
+ * iterations and captures thepages_skippedcounter again. The expectation
+ * is that over 50% of the page scans are skipped (There is only one pageQuoted 8 lines
+ * that has KSM enabled and it gets scanned during each iteration and it
+ * cannot be de-duplicated).
+ *
+ * Prerequisites:
+ *
+ * 1) ksm and ksmtuned daemons need to be disabled. Otherwise, it could
+ * distrub the testing as they also change some ksm tunables depends
+ * on current workloads.Hm, we don't have to any helper in LTP API to detect this, so at least we document this.
Quoted 2 lines
+ *
+ */The result is then uploaded:
https://github.com/linux-test-project/ltp/releases/download/20230929/metadata.20230929.htmlTherefore please use:
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright (C) 2010-2017 Red Hat, Inc.
*/
/*\
* [Description]
*
* Kernel Samepage Merging (KSM)
*
* This adds a new ksm (kernel samepage merging) test to evaluate the new
* smart scan feature. It allocates a page and fills it with 'a'
* characters. It captures thepages_skippedcounter, waits for a few
* iterations and captures thepages_skippedcounter again. The expectation
* is that over 50% of the page scans are skipped (There is only one page
* that has KSM enabled and it gets scanned during each iteration and it
Quoted 9 lines
* cannot be de-duplicated).
*
* Prerequisites:
*
* 1) ksm and ksmtuned daemons need to be disabled. Otherwise, it could
* distrub the testing as they also change some ksm tunables depends
* on current workloads.
*/
The next version will use the above comment as documentation.
Quoted 12 lines
Quoted 12 lines
testcases/kernel/mem/ksm/ksm07.c new file +12
-0+ +#include <sys/types.h> +#include <sys/mman.h> +#include <sys/stat.h> +#include <sys/wait.h> +#include <errno.h> +#include <fcntl.h> +#include <signal.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h>
Quoted 9 lines
testcases/kernel/mem/ksm/ksm07.c line 48
Quoted 6 lines
testcases/kernel/mem/ksm/ksm07.c new file +6
-0+#include "../include/mem.h" +#include "ksm_common.h" + +static void verify_ksm (void ) +{ +create_memory_for_smartscan ();I wonder, if we reusable
create_memory_for_smartscan()later. Maybe it should be put into ksm07 for the start.
I moved it over to ksm07.c and renamed it to create_memory.
Quoted 4 lines
Also, the test needs to run on older kernel - exit with TCONF (which is not considered as a failure) instead of TBROK which does now:
mem.c:488: TBROK: Failed to open FILE '/sys/kernel/mm/ksm/pages_skipped' for reading: ENOENT (2)
Changed it to TCONF.
Quoted 19 lines
If the testing code stays in
testcases/kernel/mem/lib/mem.c, you will have tostat()it. But if it's really just this test specific and you move it toksm07.c, then you can simply add the handling via .save_restore.Quoted 15 lines
testcases/kernel/mem/ksm/ksm07.c new file +15
-0+} + +static struct tst_test test = { + .needs_root =1 , + .forks_child =1 , + .options = (struct tst_option []) { + {} + }, + .save_restore = (const struct tst_path_val []) { + {"/sys/kernel/mm/ksm/run" ,NULL ,TST_SR_TBROK }, + {"/sys/kernel/mm/ksm/sleep_millisecs" ,NULL ,TST_SR_TBROK }, + {"/sys/kernel/mm/ksm/smart_scan" ,"1" , +TST_SR_SKIP_MISSING |TST_SR_TBROK_RO }, + {} + },
Quoted 9 lines
testcases/kernel/mem/ksm/ksm07.c line 69
Quoted 6 lines
testcases/kernel/mem/ksm/ksm07.c new file +6
-0+ .needs_kconfigs = (const char *const[]){ +"CONFIG_KSM=y" , +NULL + }, + .test_all = verify_ksm, +};Also, there are missing static:
These are declared as non-static in ksm_common.h.
Quoted 17 lines
$ cd testcases/kernel/mem/ksm/; make check-ksm07 CHECK testcases/kernel/mem/ksm/ksm07.c ksm07.c:1: WARNING: Missing or malformed SPDX-License-Identifier tag in line 1 make: [../../../../include/mk/rules.mk:56: check-ksm07] Error 1 (ignored) ksm07.c: note: in included file: ksm_common.h:14:5: warning: Symbol 'size' has no prototype or library ('tst_') prefix. Should it be static? ksm_common.h:14:29: warning: Symbol 'num' has no prototype or library ('tst_') prefix. Should it be static? ksm_common.h:14:38: warning: Symbol 'unit' has no prototype or library ('tst_') prefix. Should it be static? ksm_common.h:15:6: warning: Symbol 'opt_sizestr' has no prototype or library ('tst_') prefix. Should it be static? ksm_common.h:15:20: warning: Symbol 'opt_numstr' has no prototype or library ('tst_') prefix. Should it be static? ksm_common.h:15:33: warning: Symbol 'opt_unitstr' has no prototype or library ('tst_') prefix. Should it be static?Kind regards,
Petr