mm/thp: add two new case

PATCH Superseded Zhouping Liu 2 participants 7 messages Apr 15, 2013

lore

Share & tools

lore

Zhouping Liu

author

2013-04-15 06:01 UTC #0 lore raw

v3 cover mm/thp: add two new case

The patchset is desinged to test THP functionality.

when one process allocate hugepage aligned anonymouse pages, kernel thread 'khugepaged' controlled by sysfs knobs /sys/kernel/mm/transparent_hugepage/* will scan them, and make them as transparent hugepage if they are suited, you can find out how many transparent hugepages are there in one process from /proc/<pid>/smaps, among the file contents, 'AnonHugePages' entry stand for transparent hugepage.

V1 to V2:
accepted Wanlong's suggestion, added some comment for '10s' added 'static' key word for single-file-only variant.

V2 to V3:
added some comments for low memory system.

Wanlong, I included you into 'Tested-by' thread, thanks for your reviewing and testing the patchset.

Zhouping Liu (3):
mm/thp: new case thp04.c
lib/mem: introduce a new function set_global_mempolicy() mm/thp: add new case thp05

 runtest/mm                         |   6 ++
 testcases/kernel/mem/include/mem.h |  12 +++
 testcases/kernel/mem/lib/mem.c     | 203 ++++++++++++++++++++++++++++++++++++-
 testcases/kernel/mem/thp/thp04.c   | 136 +++++++++++++++++++++++++
 testcases/kernel/mem/thp/thp05.c   | 152 +++++++++++++++++++++++++++
 5 files changed, 508 insertions(+), 1 deletion(-)
 create mode 100644 testcases/kernel/mem/thp/thp04.c
 create mode 100644 testcases/kernel/mem/thp/thp05.c

Signature · 13 lines

1.7.11.7


------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Zhouping Liu

author

2013-04-15 06:01 UTC #1 lore raw

PATCH v3 1/3 mm/thp: new case thp04.c

The case is desinged to test THP functionality.

Tested-by Wanlong Gao <[email protected]>

Signed-off-by Zhouping Liu <[email protected]>

---
 runtest/mm                         |   3 +
 testcases/kernel/mem/include/mem.h |  11 +++
 testcases/kernel/mem/lib/mem.c     | 192 +++++++++++++++++++++++++++++++++++++
 testcases/kernel/mem/thp/thp04.c   | 136 ++++++++++++++++++++++++++
 4 files changed, 342 insertions(+)
 create mode 100644 testcases/kernel/mem/thp/thp04.c

runtest/mm +3 -0

index 56b83f8..3fbb20f 100644
@@ -84,6 +84,9 @@ swapping01 swapping01 -i 5
 thp01 thp01 -I 120
 thp02 thp02
 thp03 thp03
+thp04_1 thp04
+thp04_2 thp04 -n 10 -N 20
+thp04_3 thp04 -n 1 -N 300
 
 vma01 vma01
 vma02 vma02

testcases/kernel/mem/include/mem.h +11 -0

index fdf558e..ccae47d 100644
@@ -32,6 +32,17 @@ void testoom(int mempolicy, int lite);
 
 #define PATH_KSM		"/sys/kernel/mm/ksm/"
 
+/* THP */
+
+#define PATH_THP		"/sys/kernel/mm/transparent_hugepage/"
+#define PATH_KHPD		PATH_THP "khugepaged/"
+
+int opt_nr_child, opt_nr_thps;
+char *opt_nr_child_str, *opt_nr_thps_str;
+void test_transparent_hugepage(int nr_child, int nr_thps, int hg_aligned);
+void check_thp_options(int *nr_child, int *nr_thps);
+void thp_usage(void);
+
 /* HUGETLB */
 
 #define PATH_SHMMAX		"/proc/sys/kernel/shmmax"

testcases/kernel/mem/lib/mem.c +192 -0

198 lines not on this page · raw message

testcases/kernel/mem/thp/thp04.c new file +136 -0

136 lines not on this page · raw message

Signature · 13 lines

1.7.11.7


------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Zhouping Liu

author

2013-04-15 06:01 UTC #2 lore raw

PATCH v3 2/3 lib/mem: introduce a new function set_global_mempolicy()

Fork out a new function set_global_mempolicy() from the previous testoom() func, which will be useful for other func.

Tested-by Wanlong Gao <[email protected]>

Signed-off-by Zhouping Liu <[email protected]>

---
 testcases/kernel/mem/lib/mem.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

testcases/kernel/mem/lib/mem.c +6 -1

index 589f3a2..4a06581 100644
@@ -91,7 +91,7 @@ void oom(int testcase, int lite)
 	}
 }
 
-void testoom(int mempolicy, int lite)
+static void set_global_mempolicy(int mempolicy)
 {
 #if HAVE_NUMA_H && HAVE_LINUX_MEMPOLICY_H && HAVE_NUMAIF_H \
 	&& HAVE_MPOL_CONSTANTS

@@ -133,6 +133,11 @@ void testoom(int mempolicy, int lite)
 			tst_brkm(TBROK|TERRNO, cleanup, "set_mempolicy");
 	}
 #endif
+}
+
+void testoom(int mempolicy, int lite)
+{
+	set_global_mempolicy(mempolicy);
 
 	tst_resm(TINFO, "start normal OOM testing.");
 	oom(NORMAL, lite);

Signature · 13 lines

1.7.11.7


------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Zhouping Liu

author

2013-04-15 06:01 UTC #3 lore raw

PATCH v3 3/3 mm/thp: add new case thp05

added new case thp05, which is used to test transparent hugepage under mempolicy.

Tested-by Wanlong Gao <[email protected]>

Signed-off-by Zhouping Liu <[email protected]>

---
 runtest/mm                         |   3 +
 testcases/kernel/mem/include/mem.h |   3 +-
 testcases/kernel/mem/lib/mem.c     |   6 +-
 testcases/kernel/mem/thp/thp04.c   |   2 +-
 testcases/kernel/mem/thp/thp05.c   | 152 +++++++++++++++++++++++++++++++++++++
 5 files changed, 163 insertions(+), 3 deletions(-)
 create mode 100644 testcases/kernel/mem/thp/thp05.c

runtest/mm +3 -0

index 3fbb20f..7c7abf1 100644
@@ -87,6 +87,9 @@ thp03 thp03
 thp04_1 thp04
 thp04_2 thp04 -n 10 -N 20
 thp04_3 thp04 -n 1 -N 300
+thp05_1 thp05
+thp05_2 thp05 -n 10 -N 20
+thp05_3 thp05 -n 1 -N 300
 
 vma01 vma01
 vma02 vma02

testcases/kernel/mem/include/mem.h +2 -1

index ccae47d..6b020af 100644
@@ -39,7 +39,8 @@ void testoom(int mempolicy, int lite);
 
 int opt_nr_child, opt_nr_thps;
 char *opt_nr_child_str, *opt_nr_thps_str;
-void test_transparent_hugepage(int nr_child, int nr_thps, int hg_aligned);
+void test_transparent_hugepage(int nr_child, int nr_thps,
+			       int hg_aligned, int mempolicy);
 void check_thp_options(int *nr_child, int *nr_thps);
 void thp_usage(void);

testcases/kernel/mem/lib/mem.c +5 -1

index 4a06581..a2559a2 100644
@@ -622,12 +622,16 @@ static void verify_thp_size(int *child, int nr_child, int nr_thps)
 	}
 }
 
-void test_transparent_hugepage(int nr_child, int nr_thps, int hg_aligned)
+void test_transparent_hugepage(int nr_child, int nr_thps,
+			       int hg_aligned, int mempolicy)
 {
 	unsigned long hugepagesize, memfree;
 	int i, *pid, ret, status;
 	char path[BUFSIZ];
 
+	if (mempolicy)
+		set_global_mempolicy(mempolicy);
+
 	memfree = read_meminfo("MemFree:");
 	tst_resm(TINFO, "The current MemFree is %luMB", memfree / KB);
 	if (memfree < MB)

testcases/kernel/mem/thp/thp04.c +1 -1

index 0f6c553..ff41f75 100644
@@ -83,7 +83,7 @@ int main(int argc, char *argv[])
 	for (lc = 0; TEST_LOOPING(lc); lc++) {
 		tst_count = 0;
 
-		test_transparent_hugepage(nr_child, nr_thps, 1);
+		test_transparent_hugepage(nr_child, nr_thps, 1, 0);
 	}
 
 	cleanup();

testcases/kernel/mem/thp/thp05.c new file +152 -0

152 lines not on this page · raw message

Signature · 13 lines

1.7.11.7


------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Wanlong Gao

2013-04-15 07:46 UTC #4 lore raw

In reply to Zhouping Liu

Hi Zhouping:

Quoted 10 lines

The patchset is desinged to test THP functionality.
when one process allocate hugepage aligned anonymouse pages, kernel thread 'khugepaged' controlled by sysfs knobs /sys/kernel/mm/transparent_hugepage/* will scan them, and make them as transparent hugepage if they are suited, you can find out how many transparent hugepages are there in one process from /proc/<pid>/smaps, among the file contents, 'AnonHugePages' entry stand for transparent hugepage.

Quoted 6 lines

V1 to V2:
accepted Wanlong's suggestion, added some comment for '10s' added 'static' key word for single-file-only variant.
V2 to V3:
added some comments for low memory system.

I just see that on the kernel before v2.6.39, the "defrag" interface is "[yes] no" or "yes [no]"(eg. RHEL6). While it's changed to boolean after commit e27e6151b154 in v2.6.39-rc4.

So, you should deal with the multi-style interface of this "defrag",

+ SAFE_FILE_SCANF(cleanup, PATH_KHPD "defrag", "%ld", &defrag);

Thanks,
Wanlong Gao

Quoted 9 lines

Wanlong, I included you into 'Tested-by' thread, thanks for your reviewing and testing the patchset.
Zhouping Liu (3):
mm/thp: new case thp04.c
lib/mem: introduce a new function set_global_mempolicy() mm/thp: add new case thp05

Quoted 9 lines

 runtest/mm                         |   6 ++
 testcases/kernel/mem/include/mem.h |  12 +++
 testcases/kernel/mem/lib/mem.c     | 203 ++++++++++++++++++++++++++++++++++++-
 testcases/kernel/mem/thp/thp04.c   | 136 +++++++++++++++++++++++++
 testcases/kernel/mem/thp/thp05.c   | 152 +++++++++++++++++++++++++++
 5 files changed, 508 insertions(+), 1 deletion(-)
 create mode 100644 testcases/kernel/mem/thp/thp04.c
 create mode 100644 testcases/kernel/mem/thp/thp05.c

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Zhouping Liu

author

2013-04-15 09:28 UTC #5 lore raw

Quoted 15 lines · Wanlong Gao

Hi Zhouping:
Quoted 14 lines
The patchset is desinged to test THP functionality.
when one process allocate hugepage aligned anonymouse pages, kernel thread 'khugepaged' controlled by sysfs knobs /sys/kernel/mm/transparent_hugepage/* will scan them, and make them as transparent hugepage if they are suited, you can find out how many transparent hugepages are there in one process from /proc/<pid>/smaps, among the file contents, 'AnonHugePages' entry stand for transparent hugepage.
V1 to V2:
accepted Wanlong's suggestion, added some comment for '10s' added 'static' key word for single-file-only variant.

Quoted 5 lines

Quoted 2 lines
V2 to V3:
added some comments for low memory system.
I just see that on the kernel before v2.6.39, the "defrag" interface is "[yes] no" or "yes [no]"(eg. RHEL6). While it's changed to boolean after commit e27e6151b154 in v2.6.39-rc4.

Good catch, I didn't find that.

I checked the code again, found that khugepaged/defrag is related to defragment,
and any value of it can't effect the current thp0* cases, so I plan to remove it from
khugepaged_scan_done() in v4.

For future cases, which will be related to khugepaged/defrag, we can set the value
inside setup().

Thanks,
Zhouping

Quoted 5 lines

So, you should deal with the multi-style interface of this "defrag",
(quoted diff) +1 -0
+		SAFE_FILE_SCANF(cleanup, PATH_KHPD "defrag", "%ld", &defrag);

Quoted 20 lines

Thanks,
Wanlong Gao
Quoted 17 lines
Wanlong, I included you into 'Tested-by' thread, thanks for your reviewing and testing the patchset.
Zhouping Liu (3):
mm/thp: new case thp04.c
lib/mem: introduce a new function set_global_mempolicy() mm/thp: add new case thp05
  runtest/mm                         |   6 ++
  testcases/kernel/mem/include/mem.h |  12 +++
  testcases/kernel/mem/lib/mem.c     | 203 ++++++++++++++++++++++++++++++++++++-
  testcases/kernel/mem/thp/thp04.c   | 136 +++++++++++++++++++++++++
  testcases/kernel/mem/thp/thp05.c   | 152 +++++++++++++++++++++++++++
  5 files changed, 508 insertions(+), 1 deletion(-)
create mode 100644 testcases/kernel/mem/thp/thp04.c create mode 100644 testcases/kernel/mem/thp/thp05.c

Wanlong Gao

2013-04-15 09:41 UTC #6 lore raw

Quoted 22 lines · Zhouping Liu

Quoted 15 lines · Wanlong Gao
Hi Zhouping:
Quoted 14 lines
The patchset is desinged to test THP functionality.
when one process allocate hugepage aligned anonymouse pages, kernel thread 'khugepaged' controlled by sysfs knobs /sys/kernel/mm/transparent_hugepage/* will scan them, and make them as transparent hugepage if they are suited, you can find out how many transparent hugepages are there in one process from /proc/<pid>/smaps, among the file contents, 'AnonHugePages' entry stand for transparent hugepage.
V1 to V2:
accepted Wanlong's suggestion, added some comment for '10s' added 'static' key word for single-file-only variant.
Quoted 5 lines
Quoted 2 lines
V2 to V3:
added some comments for low memory system.
I just see that on the kernel before v2.6.39, the "defrag" interface is "[yes] no" or "yes [no]"(eg. RHEL6). While it's changed to boolean after commit e27e6151b154 in v2.6.39-rc4.

Quoted 5 lines

Good catch, I didn't find that.
I checked the code again, found that khugepaged/defrag is related to defragment, and any value of it can't effect the current thp0* cases, so I plan to remove it from khugepaged_scan_done() in v4.

Yeah, agree.

Thanks,
Wanlong Gao

Quoted 34 lines

For future cases, which will be related to khugepaged/defrag, we can set the value inside setup().
Thanks,
Zhouping
Quoted 25 lines
So, you should deal with the multi-style interface of this "defrag",
+ SAFE_FILE_SCANF(cleanup, PATH_KHPD "defrag", "%ld", &defrag);
Thanks,
Wanlong Gao
Quoted 17 lines
Wanlong, I included you into 'Tested-by' thread, thanks for your reviewing and testing the patchset.
Zhouping Liu (3):
mm/thp: new case thp04.c
lib/mem: introduce a new function set_global_mempolicy() mm/thp: add new case thp05
  runtest/mm                         |   6 ++
  testcases/kernel/mem/include/mem.h |  12 +++
  testcases/kernel/mem/lib/mem.c     | 203 ++++++++++++++++++++++++++++++++++++-
  testcases/kernel/mem/thp/thp04.c   | 136 +++++++++++++++++++++++++
  testcases/kernel/mem/thp/thp05.c   | 152 +++++++++++++++++++++++++++
  5 files changed, 508 insertions(+), 1 deletion(-)
create mode 100644 testcases/kernel/mem/thp/thp04.c create mode 100644 testcases/kernel/mem/thp/thp05.c