Skip to contentSkip to conversation

fio: Add advise THP option to mmap engine

PATCH In review Keith Busch 2 participants 2 messages Apr 18, 2019 Block layer

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 fio: Add advise THP option to mmap engine

The Linux specific transparent hugepage memory advisory has potentially significant implications for how the memory management behaves. If the platform supports it, add a new mmap ioengine specific option that advises HUGEPAGE on an mmap'ed range. The option availability is detected during configure. If the option is set, fio can test THP when used with private anonymous memory (i.e. mmap /dev/zero).

---
v1 -> v2:

Added a 'configure' check for MADV_HUGEPAGE support rather than just consider only OS Linux

Fixed cases when MADV_HUGEPAGE is not supported

Run madvise when after the original mmap() rather than during prep

Changelog updates

 configure      | 27 +++++++++++++++++++++++++++
 engines/mmap.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
 optgroup.h     |  2 ++
 3 files changed, 81 insertions(+), 2 deletions(-)
configure +27 -0
index 6e549cdc..3c882f0f 100755
@@ -2326,6 +2326,30 @@ if compile_prog "-Wimplicit-fallthrough" "" "-Wimplicit-fallthrough"; then
 fi
 print_config "-Wimplicit-fallthrough" "$fallthrough"
 
+##########################################
+# check for MADV_HUGEPAGE support
+if test "$thp" != "yes" ; then
+  thp="no"
+fi
+if test "$esx" != "yes" ; then
+  cat > $TMPC <<EOF
+#include <sys/mman.h>
+int main(void)
+{
+  return madvise(0, 0x1000, MADV_HUGEPAGE);
+}
+EOF
+  if compile_prog "" "" "thp" ; then
+    thp=yes
+  else
+    if test "$thp" = "yes" ; then
+      feature_not_found "Transparent Huge Page" ""
+    fi
+    thp=no
+  fi
+fi
+print_config "MADV_HUGEPAGE" "$thp"
+
 #############################################################################
 
 if test "$wordsize" = "64" ; then
@@ -2600,6 +2624,9 @@ fi
 if test "$fallthrough" = "yes"; then
   CFLAGS="$CFLAGS -Wimplicit-fallthrough"
 fi
+if test "$thp" = "yes" ; then
+  output_sym "CONFIG_HAVE_THP"
+fi
 
 echo "LIBS+=$LIBS" >> $config_host_mak
 echo "GFIO_LIBS+=$GFIO_LIBS" >> $config_host_mak
engines/mmap.c +52 -2

93 lines not on this page · raw message

optgroup.h +2 -0
index adf4d09b..bf1bb036 100644
@@ -61,6 +61,7 @@ enum opt_category_group {
 	__FIO_OPT_G_MTD,
 	__FIO_OPT_G_HDFS,
 	__FIO_OPT_G_SG,
+	__FIO_OPT_G_MMAP,
 	__FIO_OPT_G_NR,
 
 	FIO_OPT_G_RATE		= (1ULL << __FIO_OPT_G_RATE),
@@ -97,6 +98,7 @@ enum opt_category_group {
 	FIO_OPT_G_MTD		= (1ULL << __FIO_OPT_G_MTD),
 	FIO_OPT_G_HDFS		= (1ULL << __FIO_OPT_G_HDFS),
 	FIO_OPT_G_SG		= (1ULL << __FIO_OPT_G_SG),
+	FIO_OPT_G_MMAP		= (1ULL << __FIO_OPT_G_MMAP),
 	FIO_OPT_G_INVALID	= (1ULL << __FIO_OPT_G_NR),
 };
 
Signature · 1 line
2.14.4
maintainer
#1 lore raw
Quoted 6 lines · Keith Busch

The Linux specific transparent hugepage memory advisory has potentially significant implications for how the memory management behaves. If the platform supports it, add a new mmap ioengine specific option that advises HUGEPAGE on an mmap'ed range. The option availability is detected during configure. If the option is set, fio can test THP when used with private anonymous memory (i.e. mmap /dev/zero).

Thanks Keith, applied.

Signature · 1 line
Jens Axboe