selftests/vm: 8x compaction_test speedup
Subsystems (2)
cover selftests/vm: 8x compaction_test speedup
Hi,
Maybe this should really be an RFC, given that I don't fully understand why the compaction_test.c program was mmap'ing 1 MB at a time. So apologies in advance if I've mucked up something important, but if so, maybe we can still find a way to get this fixed up to something better.
Anyway: there are 20+ tests in tools/testing/selftests/vm/. The entire running time for these (via run_vmtest.sh) is about 56 seconds, of which over half is due to just one test: compaction_test, which takes 27 sec! (A runner-up is HMM, at 11 sec, so it's up for a look next.) The other tests mostly take a few ms, and a few take 1.0 sec.
This drops the compaction_test run time from 27, to 3.3 sec. Enjoy. :)
thanks,
John Hubbard
NVIDIA
John Hubbard (1):
selftests/vm: 8x compaction_test speedup
tools/testing/selftests/vm/compaction_test.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
Signature · 1 line
2.28.0
PATCH 1/1 selftests/vm: 8x compaction_test speedup
This patch reduces the running time for compaction_test from about 27 sec, to 3.3 sec, which is about an 8x speedup.
These numbers are for an Intel x86_64 system with 32 GB of DRAM.
The compaction_test.c program was spending most of its time doing mmap(), 1 MB at a time, on about 25 GB of memory.
Instead, do the mmaps 100 MB at a time. (Going past 100 MB doesn't make things go much faster, because other parts of the program are using the remaining time.)
--- tools/testing/selftests/vm/compaction_test.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
tools/testing/selftests/vm/compaction_test.c +6 -5
@@ -18,7 +18,8 @@#include "../kselftest.h" -#define MAP_SIZE 1048576 +#define MAP_SIZE_MB 100 +#define MAP_SIZE (MAP_SIZE_MB * 1024 * 1024)struct map_list {void *map;
@@ -165,7 +166,7 @@ int main(int argc, char **argv)void *map =NULL ;unsigned long mem_free =0 ;unsigned long hugepage_size =0 ; -unsigned long mem_fragmentable =0 ; +long mem_fragmentable_MB =0 ;if (prereq () !=0 ) { printf("Either the sysctl compact_unevictable_allowed is not\n"
@@ -190,9 +191,9 @@ int main(int argc, char **argv)return -1 ; } - mem_fragmentable = mem_free *0.8 /1024 ; + mem_fragmentable_MB = mem_free *0.8 /1024 ; -while (mem_fragmentable >0 ) { +while (mem_fragmentable_MB >0 ) { map =mmap (NULL ,MAP_SIZE ,PROT_READ |PROT_WRITE ,MAP_ANONYMOUS |MAP_PRIVATE |MAP_LOCKED ,-1 ,0 );if (map ==MAP_FAILED )
@@ -213,7 +214,7 @@ int main(int argc, char **argv)for (i =0 ; i <MAP_SIZE ; i += page_size) *(unsigned long *)(map + i) = (unsigned long )map + i; - mem_fragmentable--; + mem_fragmentable_MB -=MAP_SIZE_MB ; }for (entry = list; entry !=NULL ; entry = entry->next) {
Signature · 1 line
2.28.0
Andrew Morton
maintainerQuoted 11 lines · On Fri, 2 Oct 2020 01:06:21 -0700 John Hubbard <[email protected]> wrote:
This patch reduces the running time for
compaction_testfrom about 27 sec, to 3.3 sec, which is about an 8x speedup.These numbers are for an Intel
x86_64system with 32 GB of DRAM.The
compaction_test.cprogram was spending most of its time doingmmap(), 1 MB at a time, on about 25 GB of memory.Instead, do the mmaps 100 MB at a time. (Going past 100 MB doesn't make things go much faster, because other parts of the program are using the remaining time.)
Seems nice. It's been 5 years, but hopefully Sri is still at Akamai?
Quoted 45 lines
tools/testing/selftests/vm/compaction_test.c +6
-5@@ -18,7 +18,8 @@#include "../kselftest.h" -#define MAP_SIZE 1048576 +#define MAP_SIZE_MB 100 +#define MAP_SIZE (MAP_SIZE_MB * 1024 * 1024)struct map_list {void *map;@@ -165,7 +166,7 @@ int main(int argc, char **argv)void *map =NULL ;unsigned long mem_free =0 ;unsigned long hugepage_size =0 ; -unsigned long mem_fragmentable =0 ; +long mem_fragmentable_MB =0 ;if (prereq () !=0 ) { printf("Either the sysctl compact_unevictable_allowed is not\n"@@ -190,9 +191,9 @@ int main(int argc, char **argv)return -1 ; } - mem_fragmentable = mem_free *0.8 /1024 ; + mem_fragmentable_MB = mem_free *0.8 /1024 ; -while (mem_fragmentable >0 ) { +while (mem_fragmentable_MB >0 ) { map =mmap (NULL ,MAP_SIZE ,PROT_READ |PROT_WRITE ,MAP_ANONYMOUS |MAP_PRIVATE |MAP_LOCKED ,-1 ,0 );if (map ==MAP_FAILED )@@ -213,7 +214,7 @@ int main(int argc, char **argv)for (i =0 ; i <MAP_SIZE ; i += page_size) *(unsigned long *)(map + i) = (unsigned long )map + i; - mem_fragmentable--; + mem_fragmentable_MB -=MAP_SIZE_MB ; }for (entry = list; entry !=NULL ; entry = entry->next) {Signature · 1 line
2.28.0
On 10/2/20, 6:05 PM, "Andrew Morton" <[email protected]> wrote:
On Fri, 2 Oct 2020 01:06:21 -0700 John Hubbard <[email protected]> wrote:
> This patch reduces the running time for compaction_test from about 27 > sec, to 3.3 sec, which is about an 8x speedup.
>
> These numbers are for an Intel x86_64 system with 32 GB of DRAM. >
> The compaction_test.c program was spending most of its time doing > mmap(), 1 MB at a time, on about 25 GB of memory.
>
> Instead, do the mmaps 100 MB at a time. (Going past 100 MB doesn't make > things go much faster, because other parts of the program are using the > remaining time.)
Seems nice. It's been 5 years, but hopefully Sri is still at Akamai?
Looks good to me.
Thanks
-Sri