Perf Bench: Locking Microbenchmark

Subject: [PATCH] Perf Bench: Locking Microbenchmark

In response to this thread https://lkml.org/lkml/2014/2/11/93, this is a micro benchmark that stresses locking contention in the kernel with creat(2) system call by spawning multiple processes to spam this system call. This workload generate similar results and contentions in AIM7 fserver workload but can generate outputs within seconds.

With the creat system call the contention vary on what locks are used in the particular file system. I have ran this benchmark only on ext4 and xfs file system.

Running the creat workload on ext4 show contention in the mutex lock that is used by ext4_orphan_add() and ext4_orphan_del() to add or delete an inode from the list of inodes. At the same time running the creat workload on xfs show contention in the spinlock that is used by xsf_log_commit_cil() to commit a transaction to the Committed Item List.

Here is a comparison of this benchmark with AIM7 running fserver workload at 500-1000 users along with a perf trace running on ext4 file system.

Test machine is a 8-sockets 80 cores Westmere system HT-off on v3.17-rc6.

	AIM7		AIM7		perf-bench	perf-bench

Users Jobs/min Jobs/min/child Ops/sec Ops/sec/child
500 119668.25 239.34 104249 208
600 126074.90 210.12 106136 176
700 128662.42 183.80 106175 151
800 119822.05 149.78 106290 132
900 106150.25 117.94 105230 116
1000 104681.29 104.68 106489 106

Perf trace for AIM7 fserver:
14.51% reaim [kernel.kallsyms] [k] osq_lock

4.98%	reaim  		reaim			[.] add_long
4.98%	reaim  		reaim			[.] add_int
4.31%	reaim  		[kernel.kallsyms]	[k] mutex_spin_on_owner

...

Perf trace of perf bench creat
22.37% locking-creat [kernel.kallsyms] [k] osq_lock

5.77%	locking-creat  [kernel.kallsyms]	[k] mutex_spin_on_owner
5.31%	locking-creat  [kernel.kallsyms]	[k] _raw_spin_lock
5.15%	locking-creat  [jbd2]			[k] jbd2_journal_put_journal_head

...

Changes since v1:
- Added -j options to specified jobs per processes.
- Change name of microbenchmark from creat to vfs.
- Change all instances of threads to proccess.

---
 tools/perf/Documentation/perf-bench.txt |   8 +
 tools/perf/Makefile.perf                |   1 +
 tools/perf/bench/bench.h                |   1 +
 tools/perf/bench/locking.c              | 261 ++++++++++++++++++++++++++++++++
 tools/perf/builtin-bench.c              |   8 +
 5 files changed, 279 insertions(+)
 create mode 100644 tools/perf/bench/locking.c

index f6480cb..31144af 100644
@@ -58,6 +58,9 @@ SUBSYSTEM
 'futex'::
 	Futex stressing benchmarks.
 
+'locking'::
+        Locking stressing benchmarks that produce similiar result as AIM7 fserver.
+
 'all'::
 	All benchmark subsystems.
 

@@ -213,6 +216,11 @@ Suite for evaluating wake calls.
 *requeue*::
 Suite for evaluating requeue calls.
 
+SUITES FOR 'locking'
+~~~~~~~~~~~~~~~~~~
+*vfs*::
+Suite for evaluating vfs locking contention through creat(2).
+
 SEE ALSO
 --------
 linkperf:perf[1]

index 262916f..c8bee04 100644
@@ -443,6 +443,7 @@ BUILTIN_OBJS += $(OUTPUT)bench/mem-memset.o
 BUILTIN_OBJS += $(OUTPUT)bench/futex-hash.o
 BUILTIN_OBJS += $(OUTPUT)bench/futex-wake.o
 BUILTIN_OBJS += $(OUTPUT)bench/futex-requeue.o
+BUILTIN_OBJS += $(OUTPUT)bench/locking.o
 
 BUILTIN_OBJS += $(OUTPUT)builtin-diff.o
 BUILTIN_OBJS += $(OUTPUT)builtin-evlist.o

index 3c4dd44..19468c5 100644
@@ -34,6 +34,7 @@ extern int bench_mem_memset(int argc, const char **argv, const char *prefix);
 extern int bench_futex_hash(int argc, const char **argv, const char *prefix);
 extern int bench_futex_wake(int argc, const char **argv, const char *prefix);
 extern int bench_futex_requeue(int argc, const char **argv, const char *prefix);
+extern int bench_locking_vfs(int argc, const char **argv, const char *prefix);
 
 #define BENCH_FORMAT_DEFAULT_STR	"default"
 #define BENCH_FORMAT_DEFAULT		0

index b9a56fa..fdfb089 100644
@@ -63,6 +63,13 @@ static struct bench futex_benchmarks[] = {
 	{ NULL,		NULL,						NULL			}
 };
 
+static struct bench locking_benchmarks[] = {
+	{ "vfs",	"Benchmark vfs using creat(2)",			bench_locking_vfs	},
+	{ "all",        "Run all benchmarks in this suite",		NULL			},
+	{ NULL,		NULL,						NULL			}
+};
+
+
 struct collection {
 	const char	*name;
 	const char	*summary;

@@ -76,6 +83,7 @@ static struct collection collections[] = {
 	{ "numa",	"NUMA scheduling and MM benchmarks",		numa_benchmarks		},
 #endif
 	{"futex",       "Futex stressing benchmarks",                   futex_benchmarks        },
+	{"locking",     "Kernel locking benchmarks",                    locking_benchmarks      },
 	{ "all",	"All benchmarks",				NULL			},
 	{ NULL,		NULL,						NULL			}
 };

1.9.1