Migrating the libhugetlbfs/testcases/slbpacaflush.c test
PATCH v3 1/1 Migrating the libhugetlbfs/testcases/slbpacaflush.c test
We are verifying ppc64 kernels prior to 2.6.15-rc5 exhibit a bug in the hugepage SLB flushing path. When opening new hugetlb areas, updating masks in the thread_struct and copying to the PACA only occurs on the CPU where segments are opened, leading to potential stale copies in other CPUs. This bug can be triggered by multiple threads sharing the mm or a single thread migrating between CPUs, particularly evident in a close-to-idle system, as other processes may flush the SLB and prevent the bug from manifesting.
Original test originates from https://github.com/libhugetlbfs/libhugetlbfs/blob/master/tests/slbpacaflush.c
---
V3:
-Addressed below requseted changes
1. static globals
2. Replaced sysfs scanning with affinity mask helper
3. Corrected sched_setaffinity() calls
4. Added SAFE_MUNMAP() in cleanup
5. Better PASS message for inconclusive case
---
V2:
-Addressed below requested changes
1. Removed the blank lines
2. Ran make-check and fixed the issues
3. As suggested used TERRNO
4. Defined CPU_SETSIZE/8 at the top
5. use get_nprocs_conf() directly as suggested
--- runtest/hugetlb | 1 + testcases/kernel/mem/.gitignore | 1 + .../kernel/mem/hugetlb/hugemmap/hugemmap41.c | 124 ++++++++++++++++++ 3 files changed, 126 insertions(+) create mode 100644 testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c
runtest/hugetlb +1 -0
@@ -35,6 +35,7 @@ hugemmap29 hugemmap29 hugemmap30 hugemmap30 hugemmap31 hugemmap31 hugemmap32 hugemmap32 +hugemmap41 hugemmap41 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -m
testcases/kernel/mem/.gitignore +1 -0
@@ -34,6 +34,7 @@ /hugetlb/hugemmap/hugemmap30 /hugetlb/hugemmap/hugemmap31 /hugetlb/hugemmap/hugemmap32 +/hugetlb/hugemmap/hugemmap41 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03
testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c new file +124 -0
124 lines not on this page · raw message
Signature · 5 lines
2.43.5 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hi!
Quoted 105 lines
runtest/hugetlb +1
-0@@ -35,6 +35,7 @@ hugemmap29 hugemmap29 hugemmap30 hugemmap30 hugemmap31 hugemmap31 hugemmap32 hugemmap32 +hugemmap41 hugemmap41 hugemmap05_1 hugemmap05 -m hugemmap05_2 hugemmap05 -s hugemmap05_3 hugemmap05 -s -mtestcases/kernel/mem/.gitignore +1
-0@@ -34,6 +34,7 @@ /hugetlb/hugemmap/hugemmap30 /hugetlb/hugemmap/hugemmap31 /hugetlb/hugemmap/hugemmap32 +/hugetlb/hugemmap/hugemmap41 /hugetlb/hugeshmat/hugeshmat01 /hugetlb/hugeshmat/hugeshmat02 /hugetlb/hugeshmat/hugeshmat03testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c new file +75
-0@@ -0,0 +1,124 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2005-2006 IBM Corporation. +* Author: David Gibson & Adam Litke +*/ +/*\ +* [Description] +* +* ppc64 kernels (prior to 2.6.15-rc5) have a bug in the hugepage SLB +* flushing path. After opening new hugetlb areas, we update the +* masks in the thread_struct, copy to the PACA, then do slbies on +* each CPU. The trouble is we only copy to the PACA on the CPU where +* we're opening the segments, which can leave a stale copy in the +* PACAs on other CPUs. +* +* This can be triggered either with multiple threads sharing the mm, +* or with a single thread which is migrated from one CPU, to another +* (where the mapping occurs), then back again (where we touch the +* stale SLB). We use the second method in this test, since it's +* easier to force (using sched_setaffinity). However it relies on a +* close-to-idle system, if any process other than a kernel thread +* runs on the first CPU between runs of the test process, the SLB +* will be flushed and we won't trigger the bug, hence the +* PASS_INCONCLUSIVE(). Obviously, this test won't work on a 1-cpu +* system (should get CONFIG() on the sched_setaffinity) +* +*/ +#define _GNU_SOURCE + +#include "hugetlb.h" + +#include <stdio.h> +#include <sched.h> +#include <unistd.h> + +#define MNTPOINT "hugetlbfs/" + +static long hpage_size; +static int fd; +static void *p; +static volatile unsigned long *q; +static int online_cpus[2 ]; +static int err; +static cpu_set_t cpu0, cpu1; + +/* +* Helper to get online CPUs (based on thread affinity) +* Returns number of CPUs found, fills in `online_cpus[]`. +*/ +static unsigned int tst_get_online_cpus (int online_cpus[],unsigned int online_cpus_cnt) +{ +cpu_set_t mask; +unsigned int count =0 ; + +if (sched_getaffinity (0 ,sizeof (mask), &mask) <0 ) +tst_brk (TBROK |TERRNO ,"sched_getaffinity() failed" ); + +for (int i =0 ; i <CPU_SETSIZE && count < online_cpus_cnt; i++) { +if (CPU_ISSET (i, &mask)) + online_cpus[count++] = i; + } + +return count; +} + +static void run_test (void ) +{ +if (tst_get_online_cpus (online_cpus,2 ) !=2 ) +tst_brk (TCONF ,"Require at least 2 online CPUs." ); + +CPU_ZERO (&cpu0); +CPU_SET (online_cpus[0 ], &cpu0); + +CPU_ZERO (&cpu1); +CPU_SET (online_cpus[1 ], &cpu1);
testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c line 85
Quoted 10 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c new file +10
-0+ + err =sched_setaffinity (getpid (),sizeof (cpu_set_t), &cpu0); +if (err !=0 ) +tst_res (TFAIL |TERRNO ,"sched_setaffinity(cpu%d)" , online_cpus[0 ]); + + err =sched_setaffinity (getpid (),sizeof (cpu_set_t), &cpu1); +if (err !=0 ) +tst_res (TFAIL |TERRNO ,"sched_setaffinity(cpu%d)" , online_cpus[1 ]); + + p =SAFE_MMAP (NULL , hpage_size,PROT_READ |PROT_WRITE ,MAP_SHARED , fd,0 );
I wonder if it wouldn't be easier to pass MAP_HUGETLB instead of creating a file on a hugetlb. Or is there something that prevents us doing that?
testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c line 89
Quoted 3 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c new file +3
-0+ err =sched_setaffinity (getpid (),sizeof (cpu_set_t), &cpu0); +if (err !=0 ) +tst_res (TFAIL ,"sched_setaffinity(cpu%d)" , online_cpus[0 ]);
Missing tab before tst_res()
Also given the amount of setaffinity calls here it may make sense to add SAFE_SCHED_SETAFFINITY() to the library.
testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c line 103
Quoted 13 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c new file +13
-0+ q = (volatile unsigned long *)(p +getpagesize ()); + *q =0xdeadbeef ; + +tst_res (TPASS ,"Nothing bad happened, probably." ); + +SAFE_MUNMAP (p, hpage_size); +} + +static void setup (void ) +{ + hpage_size =tst_get_hugepage_size (); + +/* Fix: pass 3 arguments */
No comments like this please.
testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c line 106
Quoted 3 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c new file +3
-0+ fd =tst_creat_unlinked (MNTPOINT ,O_RDWR |O_CREAT ,0644 ); +if (fd <0 ) +tst_brk (TBROK |TERRNO ,"tst_creat_unlinked() failed" );
tst_creat_unlinked() cannot fail.
Quoted 24 lines
testcases/kernel/mem/hugetlb/hugemmap/hugemmap41.c new file +18
-0+} + +static void cleanup (void ) +{ +if (fd >0 ) +SAFE_CLOSE (fd); +} + +static struct tst_test test = { + .needs_root =1 , + .mntpoint =MNTPOINT , + .needs_hugetlbfs =1 , + .needs_tmpdir =1 , + .setup = setup, + .cleanup = cleanup, + .test_all = run_test, + .hugepages = {1 ,TST_NEEDS }, +};Signature · 5 lines
2.43.5 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp