Fixes for compaction_test
Subsystems (4)
v2 cover Fixes for compaction_test
The compaction_test memory selftest introduces fragmentation in memory and then tries to allocate as many hugepages as possible. This series addresses some problems.
On Aarch64, if nr_hugepages == 0, then the test trivially succeeds since compaction_index becomes 0, which is less than 3, due to no division by zero exception being raised. We fix that by checking for division by zero.
Secondly, correctly set the number of hugepages to zero before trying to set a large number of them.
Now, consider a situation in which, at the start of the test, a non-zero number of hugepages have been already set (while running the entire selftests/mm suite, or manually by the admin). The test operates on 80% of memory to avoid OOM-killer invocation, and because some memory is already blocked by hugepages, it would increase the chance of OOM-killing. Also, since mem_free used in check_compaction() is the value before we set nr_hugepages to zero, the chance that the compaction_index will be small is very high if the preset nr_hugepages was high, leading to a bogus test success.
This series applies on top of the stable 6.9 kernel.
Changes in v2:
- Handle an unsigned long number of hugepages
- Combine the first patch (previously standalone) with this series
Link to v1:
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/
Dev Jain (3):
selftests/mm: compaction_test: Fix bogus test success on Aarch64 selftests/mm: compaction_test: Fix incorrect write of zero to
nr_hugepages
selftests/mm: compaction_test: Fix bogus test success and reduce
probability of OOM-killer invocation
tools/testing/selftests/mm/compaction_test.c | 85 ++++++++++++++------ 1 file changed, 60 insertions(+), 25 deletions(-)
Signature · 1 line
2.34.1
PATCH v2 1/3 selftests/mm: compaction_test: Fix bogus test success on Aarch64
Currently, if at runtime we are not able to allocate a huge page, the test will trivially pass on Aarch64 due to no exception being raised on division by zero while computing compaction_index. Fix that by checking for nr_hugepages == 0. Anyways, in general, avoid a division by zero by exiting the program beforehand. While at it, fix a typo, and handle the case where the number of hugepages may overflow an integer.
Changes in v2:
- Combine with this series, handle unsigned long number of hugepages
v1:
- https://lore.kernel.org/all/[email protected]/
--- tools/testing/selftests/mm/compaction_test.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)
tools/testing/selftests/mm/compaction_test.c +13 -7
@@ -82,12 +82,13 @@ int prereq(void)return -1 ; } -int check_compaction (unsigned long mem_free,unsigned int hugepage_size) +int check_compaction (unsigned long mem_free,unsigned long hugepage_size) { +unsigned long nr_hugepages_ul;int fd, ret =-1 ;int compaction_index =0 ; -char initial_nr_hugepages[10 ] = {0 }; -char nr_hugepages[10 ] = {0 }; +char initial_nr_hugepages[20 ] = {0 }; +char nr_hugepages[20 ] = {0 };/* We want to test with 80% of available memory. Else, OOM killer comes in to play */
@@ -134,7 +135,12 @@ int check_compaction(unsigned long mem_free, unsigned int hugepage_size)/* We should have been able to request at least 1/3 rd of the memory in huge pages */ - compaction_index = mem_free/(atoi (nr_hugepages) * hugepage_size); + nr_hugepages_ul =strtoul (nr_hugepages,NULL ,10 ); +if (!nr_hugepages_ul) { +ksft_print_msg ("ERROR: No memory is available as huge pages\n" ); + goto close_fd; + } + compaction_index = mem_free/(nr_hugepages_ul * hugepage_size);lseek (fd,0 ,SEEK_SET );
@@ -145,11 +151,11 @@ int check_compaction(unsigned long mem_free, unsigned int hugepage_size) goto close_fd; } -ksft_print_msg ("Number of huge pages allocated = %d\n" , -atoi (nr_hugepages)); +ksft_print_msg ("Number of huge pages allocated = %lu\n" , + nr_hugepages_ul);if (compaction_index >3 ) { -ksft_print_msg ("ERROR: Less that 1/%d of memory is available\n" +ksft_print_msg ("ERROR: Less than 1/%d of memory is available\n" "as huge pages\n" , compaction_index); goto close_fd; }
Signature · 1 line
2.34.1
PATCH v2 2/3 selftests/mm: compaction_test: Fix incorrect write of zero to nr_hugepages
In reply to Dev JainCurrently, the test tries to set nr_hugepages to zero, but that is not actually done because the file offset is not reset after read(). Fix that using lseek().
--- tools/testing/selftests/mm/compaction_test.c | 2 ++ 1 file changed, 2 insertions(+)
tools/testing/selftests/mm/compaction_test.c +2 -0
@@ -108,6 +108,8 @@ int check_compaction(unsigned long mem_free, unsigned long hugepage_size) goto close_fd; } +lseek (fd,0 ,SEEK_SET ); +/* Start with the initial condition of 0 huge pages*/ if (write (fd,"0" ,sizeof (char )) !=sizeof (char )) { ksft_print_msg("Failed to write 0 to /proc/sys/vm/nr_hugepages: %s\n",
Signature · 1 line
2.34.1
PATCH v2 3/3 selftests/mm: compaction_test: Fix bogus test success and reduce probability of OOM-killer invocation
In reply to Dev JainReset nr_hugepages to zero before the start of the test.
If a non-zero number of hugepages is already set before the start of the test, the following problems arise:
- The probability of the test getting OOM-killed increases.
Proof: The test wants to run on 80% of available memory to prevent OOM-killing (see original code comments). Let the value of mem_free at the start of the test, when nr_hugepages = 0, be x. In the other case, when nr_hugepages > 0, let the memory consumed by hugepages be y. In the former case, the test operates on 0.8 * x of memory. In the latter, the test operates on 0.8 * (x - y) of memory, with y already filled, hence, memory consumed is y + 0.8 * (x - y) = 0.8 * x + 0.2 * y > 0.8 * x. Q.E.D
- The probability of a bogus test success increases.
Proof: Let the memory consumed by hugepages be greater than 25% of x, with x and y defined as above. The definition of compaction_index is c_index = (x - y)/z where z is the memory consumed by hugepages after trying to increase them again. In check_compaction(), we set the number of hugepages to zero, and then increase them back; the probability that they will be set back to consume at least y amount of memory again is very high (since there is not much delay between the two attempts of changing nr_hugepages). Hence, z >= y > (x/4) (by the 25% assumption). Therefore,c_index = (x - y)/z <= (x - y)/y = x/y - 1 < 4 - 1 = 3 hence, c_index can always be forced to be less than 3, thereby the test succeeding always. Q.E.D
Changes in v2:
- Handle unsigned long number of hugepages
v1:
- https://lore.kernel.org/all/[email protected]/
NOTE: This patch depends on the previous one.
--- tools/testing/selftests/mm/compaction_test.c | 71 ++++++++++++++------ 1 file changed, 49 insertions(+), 22 deletions(-)
tools/testing/selftests/mm/compaction_test.c +49 -22
@@ -82,13 +82,16 @@ int prereq(void)return -1 ; } -int check_compaction (unsigned long mem_free,unsigned long hugepage_size) +int check_compaction (unsigned long mem_free,unsigned long hugepage_size, +unsigned long initial_nr_hugepages) {unsigned long nr_hugepages_ul;int fd, ret =-1 ;int compaction_index =0 ; -char initial_nr_hugepages[20 ] = {0 };char nr_hugepages[20 ] = {0 }; +char init_nr_hugepages[20 ] = {0 }; + +sprintf (init_nr_hugepages,"%lu" , initial_nr_hugepages);/* We want to test with 80% of available memory. Else, OOM killer comes in to play */
@@ -102,23 +105,6 @@ int check_compaction(unsigned long mem_free, unsigned long hugepage_size) goto out; } -if (read (fd, initial_nr_hugepages,sizeof (initial_nr_hugepages)) <=0 ) { -ksft_print_msg ("Failed to read from /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - goto close_fd; - } - -lseek (fd,0 ,SEEK_SET ); - -/* Start with the initial condition of 0 huge pages*/ -if (write (fd,"0" ,sizeof (char )) !=sizeof (char )) { -ksft_print_msg ("Failed to write 0 to /proc/sys/vm/nr_hugepages: %s\n" , -strerror (errno)); - goto close_fd; - } - -lseek (fd,0 ,SEEK_SET ); -/* Request a large number of huge pages. The Kernel will allocate as much as it can */ if (write (fd,"100000" , (6 *sizeof (char ))) != (6 *sizeof (char ))) {
@@ -146,8 +132,8 @@ int check_compaction(unsigned long mem_free, unsigned long hugepage_size)lseek (fd,0 ,SEEK_SET ); -if (write (fd, initial_nr_hugepages,strlen (initial_nr_hugepages)) - !=strlen (initial_nr_hugepages)) { +if (write (fd, init_nr_hugepages,strlen (init_nr_hugepages)) + !=strlen (init_nr_hugepages)) {ksft_print_msg ("Failed to write value to /proc/sys/vm/nr_hugepages: %s\n" ,strerror (errno)); goto close_fd;
@@ -171,6 +157,41 @@ int check_compaction(unsigned long mem_free, unsigned long hugepage_size)return ret; } +int set_zero_hugepages (unsigned long *initial_nr_hugepages) +{ +int fd, ret =-1 ; +char nr_hugepages[20 ] = {0 }; + + fd =open ("/proc/sys/vm/nr_hugepages" ,O_RDWR |O_NONBLOCK ); +if (fd <0 ) { +ksft_print_msg ("Failed to open /proc/sys/vm/nr_hugepages: %s\n" , +strerror (errno)); + goto out; + } +if (read (fd, nr_hugepages,sizeof (nr_hugepages)) <=0 ) { +ksft_print_msg ("Failed to read from /proc/sys/vm/nr_hugepages: %s\n" , +strerror (errno)); + goto close_fd; + } + +lseek (fd,0 ,SEEK_SET ); + +/* Start with the initial condition of 0 huge pages */ +if (write (fd,"0" ,sizeof (char )) !=sizeof (char )) { +ksft_print_msg ("Failed to write 0 to /proc/sys/vm/nr_hugepages: %s\n" , +strerror (errno)); + goto close_fd; + } + + *initial_nr_hugepages =strtoul (nr_hugepages,NULL ,10 ); + ret =0 ; + + close_fd: +close (fd); + + out: +return ret; +}int main (int argc,char **argv) {
@@ -181,6 +202,7 @@ int main(int argc, char **argv)unsigned long mem_free =0 ;unsigned long hugepage_size =0 ;long mem_fragmentable_MB =0 ; +unsigned long initial_nr_hugepages;ksft_print_header ();
@@ -189,6 +211,10 @@ int main(int argc, char **argv)ksft_set_plan (1 ); +/* Start the test without hugepages reducing mem_free */ +if (set_zero_hugepages (&initial_nr_hugepages)) +ksft_exit_fail (); + lim.rlim_cur =RLIM_INFINITY ; lim.rlim_max =RLIM_INFINITY ;if (setrlimit (RLIMIT_MEMLOCK , &lim))
@@ -232,7 +258,8 @@ int main(int argc, char **argv) entry = entry->next; } -if (check_compaction (mem_free, hugepage_size) ==0 ) +if (check_compaction (mem_free, hugepage_size, + initial_nr_hugepages) ==0 )ksft_exit_pass ();ksft_exit_fail ();
Signature · 1 line
2.34.1