Skip to contentSkip to conversation

Introduce nfs-readahead-udev

RFC PATCH In review Thiago Becker 2 participants 12 messages Mar 11, 2022 – Mar 18, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover Introduce nfs-readahead-udev

Recent changes in the linux kernel caused NFS readahead to default to 128 from the previous default of 15 * rsize. This causes performance penalties to some read-heavy workloads, which can be fixed by tuning the readahead for that given mount.

Specifically, the read troughput on a sec=krb5p mount drops by 50-75% when comparing the default readahead with a readahead of 15360.

Previous discussions:
https://lore.kernel.org/linux-nfs/[email protected]/
I attempted to add a non-kernel option to mount.nfs, and it was rejected.

https://lore.kernel.org/linux-nfs/[email protected]/
Attempted to add a mount option to the kernel, rejected as well.

I had started a separate tool to set the readahead of BDIs, but the scope is specifically for NFS, so I would like to get the community feeling for having this in nfs-utils.

This patch series introduces nfs-readahead-udev, a utility to automatically set NFS readahead when NFS is mounted. The utility is triggered by udev when a new BDI is added, returns to udev the value of the readahead that should be used.

The tool currently supports setting read ahead per mountpoint, nfs major version, or by a global default value.

Thiago Becker (7):
Create nfs-readahead-udev
readahead: configure udev
readahead: create logging facility
readahead: only set readahead for nfs devices.
readahead: create the configuration file
readahead: add mountpoint and fstype options
readahead: documentation

 .gitignore                                    |   6 +
 configure.ac                                  |   4 +
 tools/Makefile.am                             |   2 +-
 tools/nfs-readahead-udev/99-nfs_bdi.rules.in  |   1 +
 tools/nfs-readahead-udev/Makefile.am          |  26 +++
 tools/nfs-readahead-udev/config_parser.c      |  25 +++
 tools/nfs-readahead-udev/config_parser.h      |  14 ++
 tools/nfs-readahead-udev/list.h               |  48 ++++
 tools/nfs-readahead-udev/log.h                |  16 ++
 tools/nfs-readahead-udev/main.c               | 211 ++++++++++++++++++
 .../nfs-readahead-udev/nfs-readahead-udev.man |  47 ++++
 tools/nfs-readahead-udev/parser.y             |  85 +++++++
 tools/nfs-readahead-udev/readahead.conf       |  15 ++
 tools/nfs-readahead-udev/scanner.l            |  19 ++
 tools/nfs-readahead-udev/syslog.c             |  47 ++++
 15 files changed, 565 insertions(+), 1 deletion(-)
 create mode 100644 tools/nfs-readahead-udev/99-nfs_bdi.rules.in
 create mode 100644 tools/nfs-readahead-udev/Makefile.am
 create mode 100644 tools/nfs-readahead-udev/config_parser.c
 create mode 100644 tools/nfs-readahead-udev/config_parser.h
 create mode 100644 tools/nfs-readahead-udev/list.h
 create mode 100644 tools/nfs-readahead-udev/log.h
 create mode 100644 tools/nfs-readahead-udev/main.c
 create mode 100644 tools/nfs-readahead-udev/nfs-readahead-udev.man
 create mode 100644 tools/nfs-readahead-udev/parser.y
 create mode 100644 tools/nfs-readahead-udev/readahead.conf
 create mode 100644 tools/nfs-readahead-udev/scanner.l
 create mode 100644 tools/nfs-readahead-udev/syslog.c
Signature · 1 line
2.35.1
#1 lore raw

RFC PATCH v2 1/7 Create nfs-readahead-udev

This tool is invoked by udev to find and set the readahead value to NFS mounts.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1946283

---
 .gitignore                           | 1 +
 configure.ac                         | 1 +
 tools/Makefile.am                    | 2 +-
 tools/nfs-readahead-udev/Makefile.am | 3 +++
 tools/nfs-readahead-udev/main.c      | 7 +++++++
 5 files changed, 13 insertions(+), 1 deletion(-)
 create mode 100644 tools/nfs-readahead-udev/Makefile.am
 create mode 100644 tools/nfs-readahead-udev/main.c
.gitignore +1 -0
index c89d1cd2..c99269a4 100644
@@ -61,6 +61,7 @@ utils/statd/statd
 tools/locktest/testlk
 tools/getiversion/getiversion
 tools/nfsconf/nfsconf
+tools/nfs-readahead-udev/nfs-readahead-udev
 support/export/mount.h
 support/export/mount_clnt.c
 support/export/mount_xdr.c
configure.ac +1 -0
index e0f5a930..7e5ba5d9 100644
@@ -737,6 +737,7 @@ AC_CONFIG_FILES([
 	tools/rpcgen/Makefile
 	tools/mountstats/Makefile
 	tools/nfs-iostat/Makefile
+	tools/nfs-readahead-udev/Makefile
 	tools/rpcctl/Makefile
 	tools/nfsdclnts/Makefile
 	tools/nfsconf/Makefile
tools/Makefile.am +1 -1
index c3feabbe..621cde03 100644
@@ -12,6 +12,6 @@ if CONFIG_NFSDCLD
 OPTDIRS += nfsdclddb
 endif
 
-SUBDIRS = locktest rpcdebug nlmtest mountstats nfs-iostat rpcctl nfsdclnts $(OPTDIRS)
+SUBDIRS = locktest rpcdebug nlmtest mountstats nfs-iostat rpcctl nfsdclnts nfs-readahead-udev $(OPTDIRS)
 
 MAINTAINERCLEANFILES = Makefile.in
tools/nfs-readahead-udev/Makefile.am new file +3 -0
new file mode 100644
index 00000000..5455e954
@@ -0,0 +1,3 @@
+libexec_PROGRAMS = nfs-readahead-udev
+nfs_readahead_udev_SOURCES = main.c
+
tools/nfs-readahead-udev/main.c new file +7 -0
new file mode 100644
index 00000000..e454108e
@@ -0,0 +1,7 @@
+#include <stdio.h>
+
+int main(int argc, char **argv, char **envp)
+{
+	unsigned int readahead = 128;
+	printf("%d\n", readahead);
+}
Signature · 1 line
2.35.1
#2 lore raw

RFC PATCH v2 2/7 readahead: configure udev

In reply to Thiago Becker

Set the udev rule to call the readahead utility.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1946283

---
 tools/nfs-readahead-udev/99-nfs_bdi.rules.in | 1 +
 tools/nfs-readahead-udev/Makefile.am         | 8 ++++++++
 2 files changed, 9 insertions(+)
 create mode 100644 tools/nfs-readahead-udev/99-nfs_bdi.rules.in
tools/nfs-readahead-udev/99-nfs_bdi.rules.in new file +1 -0
new file mode 100644
index 00000000..15f8a995
@@ -0,0 +1 @@
+SUBSYSTEM=="bdi", ACTION=="add", PROGRAM="_libexecdir_/nfs-readahead-udev", ATTR{read_ahead_kb}="%c"
tools/nfs-readahead-udev/Makefile.am +8 -0
index 5455e954..873cc175 100644
@@ -1,3 +1,11 @@
 libexec_PROGRAMS = nfs-readahead-udev
 nfs_readahead_udev_SOURCES = main.c
 
+udev_rulesdir = /etc/udev/rules.d
+udev_rules_DATA = 99-nfs_bdi.rules
+
+99-nfs_bdi.rules: 99-nfs_bdi.rules.in $(builddefs)
+	$(SED) "s|_libexecdir_|@libexecdir@|g" 99-nfs_bdi.rules.in > $@
+
+clean-local:
+	$(RM) 99-nfs_bdi.rules
Signature · 1 line
2.35.1
#3 lore raw

RFC PATCH v2 3/7 readahead: create logging facility

In reply to Thiago Becker

Create logs for nfs-readahead-udev, logging to syslog.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1946283

---
 tools/nfs-readahead-udev/Makefile.am |  2 +-
 tools/nfs-readahead-udev/log.h       | 16 ++++++++++
 tools/nfs-readahead-udev/main.c      |  8 +++++
 tools/nfs-readahead-udev/syslog.c    | 47 ++++++++++++++++++++++++++++
 4 files changed, 72 insertions(+), 1 deletion(-)
 create mode 100644 tools/nfs-readahead-udev/log.h
 create mode 100644 tools/nfs-readahead-udev/syslog.c
tools/nfs-readahead-udev/Makefile.am +1 -1
index 873cc175..5078db9a 100644
@@ -1,5 +1,5 @@
 libexec_PROGRAMS = nfs-readahead-udev
-nfs_readahead_udev_SOURCES = main.c
+nfs_readahead_udev_SOURCES = main.c syslog.c
 
 udev_rulesdir = /etc/udev/rules.d
 udev_rules_DATA = 99-nfs_bdi.rules
tools/nfs-readahead-udev/log.h new file +16 -0
new file mode 100644
index 00000000..2a14e552
@@ -0,0 +1,16 @@
+#ifndef __ra_log_h__
+#define __ra_log_h__
+#include <stdarg.h>
+
+extern void log_open(void);
+extern void log_close(void);
+
+extern void debug(const char *, ...);
+extern void info(const char *, ...);
+extern void notice(const char *, ...);
+extern void warn(const char *, ...);
+extern void err(const char *, ...);
+extern void crit(const char *, ...);
+extern void alert(const char *, ...);
+extern void emerg(const char *, ...);
+#endif
tools/nfs-readahead-udev/main.c +8 -0
index e454108e..dd2c9f8c 100644
@@ -1,7 +1,15 @@
 #include <stdio.h>
 
+#include "log.h"
+
 int main(int argc, char **argv, char **envp)
 {
 	unsigned int readahead = 128;
+
+	log_open();
+
+	info("Setting the readahead to 128\n");
+
+	log_close();
 	printf("%d\n", readahead);
 }
tools/nfs-readahead-udev/syslog.c new file +47 -0

47 lines not on this page · raw message

Signature · 1 line
2.35.1
#4 lore raw

RFC PATCH v2 4/7 readahead: only set readahead for nfs devices.

In reply to Thiago Becker

Limit setting the readahead for nfs devices.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1946283

---
 tools/nfs-readahead-udev/99-nfs_bdi.rules.in |   2 +-
 tools/nfs-readahead-udev/Makefile.am         |   1 +
 tools/nfs-readahead-udev/main.c              | 132 ++++++++++++++++++-
 3 files changed, 133 insertions(+), 2 deletions(-)
tools/nfs-readahead-udev/99-nfs_bdi.rules.in +1 -1
index 15f8a995..744c59be 100644
@@ -1 +1 @@
-SUBSYSTEM=="bdi", ACTION=="add", PROGRAM="_libexecdir_/nfs-readahead-udev", ATTR{read_ahead_kb}="%c"
+SUBSYSTEM=="bdi", ACTION=="add", PROGRAM="_libexecdir_/nfs-readahead-udev %k", ATTR{read_ahead_kb}="%c"
tools/nfs-readahead-udev/Makefile.am +1 -0
index 5078db9a..551d22e9 100644
@@ -1,5 +1,6 @@
 libexec_PROGRAMS = nfs-readahead-udev
 nfs_readahead_udev_SOURCES = main.c syslog.c
+nfs_readahead_udev_LDFLAGS= -lmount
 
 udev_rulesdir = /etc/udev/rules.d
 udev_rules_DATA = 99-nfs_bdi.rules
tools/nfs-readahead-udev/main.c +131 -1

146 lines not on this page · raw message

Signature · 1 line
2.35.1
#5 lore raw

RFC PATCH v2 5/7 readahead: create the configuration file

In reply to Thiago Becker

At this stage, the configuration file only accepts the default value.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1946283

---
 .gitignore                               |  5 ++
 configure.ac                             |  3 +
 tools/nfs-readahead-udev/Makefile.am     | 16 ++++-
 tools/nfs-readahead-udev/config_parser.c | 25 ++++++++
 tools/nfs-readahead-udev/config_parser.h | 14 +++++
 tools/nfs-readahead-udev/list.h          | 48 +++++++++++++++
 tools/nfs-readahead-udev/main.c          | 68 ++++++++++++++++++++-
 tools/nfs-readahead-udev/parser.y        | 78 ++++++++++++++++++++++++
 tools/nfs-readahead-udev/readahead.conf  |  1 +
 tools/nfs-readahead-udev/scanner.l       | 14 +++++
 10 files changed, 269 insertions(+), 3 deletions(-)
 create mode 100644 tools/nfs-readahead-udev/config_parser.c
 create mode 100644 tools/nfs-readahead-udev/config_parser.h
 create mode 100644 tools/nfs-readahead-udev/list.h
 create mode 100644 tools/nfs-readahead-udev/parser.y
 create mode 100644 tools/nfs-readahead-udev/readahead.conf
 create mode 100644 tools/nfs-readahead-udev/scanner.l
.gitignore +5 -0
index c99269a4..340ee8fb 100644
@@ -17,6 +17,7 @@ aclocal/ltoptions.m4
 aclocal/ltsugar.m4
 aclocal/ltversion.m4
 aclocal/lt~obsolete.m4
+ylwrap
 # files generated by configure
 confdefs.h
 config.cache
@@ -61,7 +62,11 @@ utils/statd/statd
 tools/locktest/testlk
 tools/getiversion/getiversion
 tools/nfsconf/nfsconf
+tools/nfs-readahead-udev/99-nfs_bdi.rules
 tools/nfs-readahead-udev/nfs-readahead-udev
+tools/nfs-readahead-udev/parser.c
+tools/nfs-readahead-udev/parser.h
+tools/nfs-readahead-udev/scanner.c
 support/export/mount.h
 support/export/mount_clnt.c
 support/export/mount_xdr.c
configure.ac +3 -0
index 7e5ba5d9..03cdf8f6 100644
@@ -300,7 +300,10 @@ AC_PROG_INSTALL
 AC_PROG_LN_S
 AC_PROG_MAKE_SET
 AC_PROG_LIBTOOL
+AC_PROG_YACC
 AM_PROG_CC_C_O
+AM_PROG_LEX
+
 
 if test "x$cross_compiling" = "xno"; then
 	CC_FOR_BUILD=${CC_FOR_BUILD-${CC-gcc}}
tools/nfs-readahead-udev/Makefile.am +14 -2
index 551d22e9..010350aa 100644
@@ -1,12 +1,24 @@
 libexec_PROGRAMS = nfs-readahead-udev
-nfs_readahead_udev_SOURCES = main.c syslog.c
+BUILT_SOURCES = parser.c parser.h scanner.c
+nfs_readahead_udev_SOURCES = main.c syslog.c config_parser.c parser.y scanner.l
 nfs_readahead_udev_LDFLAGS= -lmount
+AM_YFLAGS = -d
 
-udev_rulesdir = /etc/udev/rules.d
+udev_rulesdir = $(sysconfdir)/udev/rules.d
 udev_rules_DATA = 99-nfs_bdi.rules
 
+ra_confdir = $(sysconfdir)
+ra_conf_DATA = readahead.conf
+
 99-nfs_bdi.rules: 99-nfs_bdi.rules.in $(builddefs)
 	$(SED) "s|_libexecdir_|@libexecdir@|g" 99-nfs_bdi.rules.in > $@
 
 clean-local:
 	$(RM) 99-nfs_bdi.rules
+	$(RM) parser.c parser.h scanner.c
+
+parser.$(OBJEXT): CFLAGS += -Wno-strict-prototypes
+
+scanner.$(OBJEXT): CFLAGS += -Wno-strict-prototypes
+
+main.$(OBJEXT): CFLAGS += -DSYSCONFDIR=\"@sysconfdir@\"
tools/nfs-readahead-udev/config_parser.c new file +25 -0
new file mode 100644
index 00000000..24d58a6b
@@ -0,0 +1,25 @@
+#include <stdlib.h>
+#include <string.h>
+#include "config_parser.h"
+
+struct config_entry *config_entry_new(void);
+
+struct config_entry *config_entry_new(void) {
+	struct config_entry *new = malloc(sizeof(struct config_entry));
+	if (!new) {
+		return NULL; // Make this an err_ptr
+	}
+
+	memset(new, 0, sizeof(struct config_entry));
+	list_init(&new->list);
+	return new;
+}
+
+
+void config_entry_free(struct config_entry *ce)
+{
+#define sfree(ce) if (ce) free(ce)
+	sfree(ce->mountpoint);
+	sfree(ce->fstype);
+#undef sfree
+}
tools/nfs-readahead-udev/config_parser.h new file +14 -0
new file mode 100644
index 00000000..f9f138bb
@@ -0,0 +1,14 @@
+#ifndef __ra_libparser_h__
+#define __ra_libparser_h__
+#include "list.h"
+
+struct config_entry {
+	struct list_head list;
+	char *mountpoint;
+	char *fstype;
+	int readahead;
+};
+
+extern int parse_config(const char *, struct list_head *config_list);
+extern void config_entry_free(struct config_entry *);
+#endif
tools/nfs-readahead-udev/list.h new file +48 -0

48 lines not on this page · raw message

tools/nfs-readahead-udev/main.c +67 -1

96 lines not on this page · raw message

tools/nfs-readahead-udev/parser.y new file +78 -0

78 lines not on this page · raw message

tools/nfs-readahead-udev/readahead.conf new file +1 -0
new file mode 100644
index 00000000..988b30c7
@@ -0,0 +1 @@
+default				readahead=128
tools/nfs-readahead-udev/scanner.l new file +14 -0
new file mode 100644
index 00000000..d1ceb90b
@@ -0,0 +1,14 @@
+%{
+#include "parser.h"
+#define yyterminate() return END_CONFIG
+%}
+%option noyywrap
+%%
+default		{ return DEFAULT; }
+readahead	{ return READAHEAD; }
+[ \t]		;
+#[^\n]*\n	{ return ENDL; }
+\n		{ return ENDL; }
+[0-9]+		{ yylval.ival = atoi(yytext); return INT; }
+=		{ return EQ; }
+%%
Signature · 1 line
2.35.1
#6 lore raw

RFC PATCH v2 6/7 readahead: add mountpoint and fstype options

In reply to Thiago Becker

Add ways to configure the system by mountpoint or fstype.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1946283

---
 tools/nfs-readahead-udev/parser.y  | 15 +++++++++++----
 tools/nfs-readahead-udev/scanner.l |  5 +++++
 2 files changed, 16 insertions(+), 4 deletions(-)
tools/nfs-readahead-udev/parser.y +11 -4
index f6db05c4..5951c99d 100644
@@ -10,20 +10,25 @@ extern FILE *yyin;
 void yyerror(const char *s);
 
 // This should be visible only to this file
-extern struct config_entry *config_entry_new(void);
+extern struct config_entry *config_entry_new();
 
 struct config_entry *current;
 %}
 
 %union {
+	char *sval;
 	int ival;
 }
 
+%token <sval> STRING
 %token <ival> INT
 %token EQ
 %token ENDL
 %token DEFAULT
+%token MOUNTPOINT
+%token FSTYPE
 %token READAHEAD
+%token <sval> FS
 %token END_CONFIG 0
 
 %%
@@ -35,7 +40,7 @@ line:
 	tokens endls {
 		struct config_entry *new = config_entry_new();
 		list_add(&current->list, &new->list);
-		current = new;
+		current = new;		
 	}
 
 
@@ -49,9 +54,11 @@ default:
 	DEFAULT
 
 pair:
-	READAHEAD EQ INT	{ current->readahead = $3; }
+	MOUNTPOINT EQ STRING	{ current->mountpoint = $3; }
+	| FSTYPE EQ FS		{ current->fstype = $3; }
+	| READAHEAD EQ INT	{ current->readahead = $3; }
 
-endls:
+endls: 
 	endls ENDL | ENDL
 
 %%
tools/nfs-readahead-udev/scanner.l +5 -0
index d1ceb90b..c6fb3f0c 100644
@@ -5,10 +5,15 @@
 %option noyywrap
 %%
 default		{ return DEFAULT; }
+mountpoint	{ return MOUNTPOINT; }
+fstype		{ return FSTYPE; }
 readahead	{ return READAHEAD; }
+nfs4		{ yylval.sval = strdup(yytext); return FS; }
+nfs		{ yylval.sval = strdup(yytext); return FS; }
 [ \t]		;
 #[^\n]*\n	{ return ENDL; }
 \n		{ return ENDL; }
 [0-9]+		{ yylval.ival = atoi(yytext); return INT; }
+[a-zA-Z0-9/]+	{ yylval.sval = strdup(yytext); return STRING; }
 =		{ return EQ; }
 %%
Signature · 1 line
2.35.1
#7 lore raw

RFC PATCH v2 7/7 readahead: documentation

In reply to Thiago Becker

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1946283

---
 tools/nfs-readahead-udev/Makefile.am          |  2 +
 .../nfs-readahead-udev/nfs-readahead-udev.man | 47 +++++++++++++++++++
 tools/nfs-readahead-udev/readahead.conf       | 14 ++++++
 3 files changed, 63 insertions(+)
 create mode 100644 tools/nfs-readahead-udev/nfs-readahead-udev.man
tools/nfs-readahead-udev/Makefile.am +2 -0
index 010350aa..eaa9b90e 100644
@@ -10,6 +10,8 @@ udev_rules_DATA = 99-nfs_bdi.rules
 ra_confdir = $(sysconfdir)
 ra_conf_DATA = readahead.conf
 
+man5_MANS = nfs-readahead-udev.man
+
 99-nfs_bdi.rules: 99-nfs_bdi.rules.in $(builddefs)
 	$(SED) "s|_libexecdir_|@libexecdir@|g" 99-nfs_bdi.rules.in > $@
 
tools/nfs-readahead-udev/nfs-readahead-udev.man new file +47 -0

47 lines not on this page · raw message

tools/nfs-readahead-udev/readahead.conf +14 -0
index 988b30c7..bce830f1 100644
@@ -1 +1,15 @@
+# nfs-readahead-udev configuration file.
+#
+# This file configures the readahead for nfs mounts when those are anounced by the kernel.
+# The file is composed on lines that can contain either the default configuration (applied to
+# any nfs mount that does not match any of the other lines) or a combination of
+#   mountpoint=<mountpoint> where mountpoint is the mount point for the file system
+#   fstype=<nfs|nfs4> specifies that this configuration should only apply to a specific nfs
+#     version.
+# Every line must contain a readahead option, with the expected readahead value.
 default				readahead=128
+
+# mountpoint=/mnt		readahead=4194304
+# fstype=nfs			readahead=4194304
+# fstype=nfs4			readahead=4194304
+# mountpoint=/mnt	fstype=nfs4	readahead=4194304
Signature · 1 line
2.35.1
#8 lore raw

Re: [RFC v2 PATCH 7/7] readahead: documentation

Quoted 63 lines · Thiago Becker

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1946283

---
  tools/nfs-readahead-udev/Makefile.am          |  2 +
  .../nfs-readahead-udev/nfs-readahead-udev.man | 47 +++++++++++++++++++
  tools/nfs-readahead-udev/readahead.conf       | 14 ++++++
  3 files changed, 63 insertions(+)

create mode 100644 tools/nfs-readahead-udev/nfs-readahead-udev.man

tools/nfs-readahead-udev/Makefile.am +2 -0
index 010350aa..eaa9b90e 100644
@@ -10,6 +10,8 @@ udev_rules_DATA = 99-nfs_bdi.rules
  ra_confdir = $(sysconfdir)
  ra_conf_DATA = readahead.conf
 
+man5_MANS = nfs-readahead-udev.man
+
  99-nfs_bdi.rules: 99-nfs_bdi.rules.in $(builddefs)
  	$(SED) "s|_libexecdir_|@libexecdir@|g" 99-nfs_bdi.rules.in > $@
 
tools/nfs-readahead-udev/nfs-readahead-udev.man new file +35 -0
new file mode 100644
index 00000000..2477d5b3
@@ -0,0 +1,47 @@
+.\" Manpage for nfs-readahead-udev.
+.nh
+.ad l
+.TH man 5 "08 Mar 2022" "1.0" "nfs-readahead-udev man page"
+.SH NAME
+
+nfs-readahead-udev \- Find the readahead for a given NFS mount
+
+.SH SYNOPSIS
+
+nfs-readahead-udev <device>
+
+.SH DESCRIPTION
+
+\fInfs-readahead-udev\fR is a tool intended to be used with udev to set the \fIread_ahead_kb\fR parameter of NFS mounts, according to the configuration file (see \fICONFIGURATION\fR). \fIdevice\fR is the device number for the NFS backing device as provided by the kernel.
+
+.SH CONFIGURATION
+
+The configuration file (\fI/etc/readahead.conf\fR) contains the readahead configuration, and is formatted as follows.
+
+<LINES> ::= <LINES> <LINE> | <LINE>
+
+<LINE> ::= <TOKENS> <ENDL>
+
+<TOKENS> ::= <TOKENS> <TOKEN> | <TOKEN>
+
+<TOKEN> ::= default | <PAIR>
+
+<PAIR> ::= mountpoint = <mountpoint> | fstype = <nfs|nfs4> | readahead = <readahead>
+
+\fImountpoint\fR is the path in the system where the file system is mounted.
+
+\fIreadahead\fR is an integer to readahead.
+
+\fIfstype\fR is either \fInfs\fR or \fInfs4\fR.

tools/nfs-readahead-udev/nfs-readahead-udev.man line 45

Quoted 10 lines
tools/nfs-readahead-udev/nfs-readahead-udev.man new file +10 -0
+
+.SH SEE ALSO
+
+mount.nfs(8), nfs(5), udev(7), bcc-readahead(8)
+
+.SH BUGS
+
+No known bugs.
+
+.SH AUTHOR

I think it might make sense to added some examples on how the command will be used.

Quoted 12 lines
tools/nfs-readahead-udev/nfs-readahead-udev.man new file +2 -0
+
+Thiago Rafael Becker <[email protected]>
tools/nfs-readahead-udev/readahead.conf +5 -0
index 988b30c7..bce830f1 100644
@@ -1 +1,15 @@
+# nfs-readahead-udev configuration file.
+#
+# This file configures the readahead for nfs mounts when those are anounced by the kernel.
+# The file is composed on lines that can contain either the default configuration (applied to
+# any nfs mount that does not match any of the other lines) or a combination of

tools/nfs-readahead-udev/readahead.conf line 15

Quoted 10 lines
tools/nfs-readahead-udev/readahead.conf +9 -0
+#   mountpoint=<mountpoint> where mountpoint is the mount point for the file system
+#   fstype=<nfs|nfs4> specifies that this configuration should only apply to a specific nfs
+#     version.
+# Every line must contain a readahead option, with the expected readahead value.
  default				readahead=128
+
+# mountpoint=/mnt		readahead=4194304
+# fstype=nfs			readahead=4194304
+# fstype=nfs4			readahead=4194304
+# mountpoint=/mnt	fstype=nfs4	readahead=4194304

Would it make sense to try added these to nfs.conf?

I must admin I'm a bit impressed with your lex and yacc routines in patch 5, I have not seen those in a while.. but that does add more dependencies to nfs-utils and as well as yet another config file to manage.

steved.

#9 lore raw

Re: [RFC v2 PATCH 7/7] readahead: documentation

Thanks for the comments.

Quoted 2 lines · Steve Dickson

I think it might make sense to added some examples on how the command will be used.

Will do.

Quoted 7 lines

Would it make sense to try added these to nfs.conf?

I must admin I'm a bit impressed with your lex and yacc routines in patch 5, I have not seen those in a while.. but that does add more dependencies to nfs-utils and as well as yet another config file to manage.

I debated with a funko pop between writing the parser and using a parser builder. Originally, the idea was to provide more possibilities, like configuring the read ahead of other devices, but those BDIs don't announce themselves when they are added like NFS does. I also don't know how useful it would be. So this idea is scrapped for now.

Here are the debate conclusions:
- writing the parser would add some complexity that lex/yacc handles
- lex/yacc add build time dependencies and some bloat
- error handling is painful with lex/yacc
- lex/yacc are well known
- using a parser combinator would be my preference, but I couldn't find any written in c that didn't add a new library to the systems

It seemed to me that using lex/yacc was the best choice taking into account the needs of both limited storage systems and large servers.

I can still see some configurations that may be added in the future, like setting the read ahead per server, network segment, but I believe that the configurations we have cover most of the use cases.

I've added a new configuration because I planned to have a more fine grained configuration for multiple file systems/designs, and that would add a lot in nfs.conf (and tie the tool to nfs).

And, to be completely honest, I took what I've done in
https://github.com/trbecker/readahead-utils
and ported it to nfs-utils.

That's the history so far. I don't mind taking stuff out if it would make the software more maintainable.

Quoted 2 lines

steved.

thiago

#10 lore raw
In reply to Thiago Becker
Quoted 53 lines · Thiago Becker

Recent changes in the linux kernel caused NFS readahead to default to 128 from the previous default of 15 * rsize. This causes performance penalties to some read-heavy workloads, which can be fixed by tuning the readahead for that given mount.

Specifically, the read troughput on a sec=krb5p mount drops by 50-75% when comparing the default readahead with a readahead of 15360.

Previous discussions:
https://lore.kernel.org/linux-nfs/[email protected]/
I attempted to add a non-kernel option to mount.nfs, and it was rejected.

https://lore.kernel.org/linux-nfs/[email protected]/
Attempted to add a mount option to the kernel, rejected as well.

I had started a separate tool to set the readahead of BDIs, but the scope is specifically for NFS, so I would like to get the community feeling for having this in nfs-utils.

This patch series introduces nfs-readahead-udev, a utility to automatically set NFS readahead when NFS is mounted. The utility is triggered by udev when a new BDI is added, returns to udev the value of the readahead that should be used.

The tool currently supports setting read ahead per mountpoint, nfs major version, or by a global default value.

Thiago Becker (7):
Create nfs-readahead-udev
readahead: configure udev
readahead: create logging facility
readahead: only set readahead for nfs devices.
readahead: create the configuration file
readahead: add mountpoint and fstype options
readahead: documentation

  .gitignore                                    |   6 +
  configure.ac                                  |   4 +
  tools/Makefile.am                             |   2 +-
  tools/nfs-readahead-udev/99-nfs_bdi.rules.in  |   1 +
  tools/nfs-readahead-udev/Makefile.am          |  26 +++
  tools/nfs-readahead-udev/config_parser.c      |  25 +++
  tools/nfs-readahead-udev/config_parser.h      |  14 ++
  tools/nfs-readahead-udev/list.h               |  48 ++++
  tools/nfs-readahead-udev/log.h                |  16 ++
  tools/nfs-readahead-udev/main.c               | 211 ++++++++++++++++++
  .../nfs-readahead-udev/nfs-readahead-udev.man |  47 ++++
  tools/nfs-readahead-udev/parser.y             |  85 +++++++
  tools/nfs-readahead-udev/readahead.conf       |  15 ++
  tools/nfs-readahead-udev/scanner.l            |  19 ++
  tools/nfs-readahead-udev/syslog.c             |  47 ++++
  15 files changed, 565 insertions(+), 1 deletion(-)
Quoted 13 lines

create mode 100644 tools/nfs-readahead-udev/99-nfs_bdi.rules.in create mode 100644 tools/nfs-readahead-udev/Makefile.am create mode 100644 tools/nfs-readahead-udev/config_parser.c create mode 100644 tools/nfs-readahead-udev/config_parser.h create mode 100644 tools/nfs-readahead-udev/list.h create mode 100644 tools/nfs-readahead-udev/log.h create mode 100644 tools/nfs-readahead-udev/main.c create mode 100644 tools/nfs-readahead-udev/nfs-readahead-udev.man create mode 100644 tools/nfs-readahead-udev/parser.y create mode 100644 tools/nfs-readahead-udev/readahead.conf create mode 100644 tools/nfs-readahead-udev/scanner.l create mode 100644 tools/nfs-readahead-udev/syslog.c

My apologies for waiting late on this... I was on PTO.

I really don't like the name of the is command. It does not follow any of the naming conventions we used in the past... Can you please rename the command to nfsrahead.

tia,

steved.

#11 lore raw
Quoted 5 lines · Steve Dickson

My apologies for waiting late on this... I was on PTO.

I really don't like the name of the is command. It does not follow any of the naming conventions we used in the past... Can you please rename the command to nfsrahead.

I have this on my git, I'm just waiting for more comments to release a new version.

Quoted 3 lines

tia,

steved.

thiago