Skip to contentSkip to conversation

LTP wrapper for Syzkaller reproducers

PATCH Superseded Richard Palethorpe 6 participants 16 messages Nov 7, 2019 – Nov 26, 2019 Library code

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover LTP wrapper for Syzkaller reproducers

Hello,

This allows the user to run ./configure --with-syzkaller-repros and it will automatically fetch the reproducers and generate the necessary LTP runtest files which use 'syzwrap' to wrap the Syzkaller reproducers.

Optimistically speaking, this should allow you to run the reproducers in your normal LTP test setup if you reduce the timeouts and your test runner is reasonably advanced (see README.md).

I have been testing this with a modified version of Metan's runltp-ng which almost runs the reproducers smoothly ;-). We haven't got around to documenting it yet, but you can find it here:

https://github.com/richiejp/runltp-ng/tree/syzkaller

I began adding some sandboxing to syzwrap from the Syzkaller executor, but I found this simply broke many of the reproducers which need access to TAP/TUN. I guess the reproducers can be regenerated with the sanboxing built in if the user cares enough. However I have left in the ability for syzwrap to optionally add some sandboxing. This does appear to help prevent the root file system from being trashed at the cost of breaking some reproducers.

Of course if the test runner can reset the SUT to a good state, then the sandboxing is not necessary. This is quite easy to do with QEMU using image overlays.

Richard Palethorpe (2):
Wrapper for Syzkaller reproducers
lib: Allow timeout coefficient < 1

 .gitmodules                                  |   5 +
 configure.ac                                 |  11 ++
 include/mk/features.mk.default               |   2 +
 include/mk/features.mk.in                    |   2 +
 lib/tst_test.c                               |   2 +-
 runtest/.gitignore                           |   1 +
 testcases/kernel/Makefile                    |   1 +
 testcases/kernel/syzkaller-repros/.gitignore |   1 +
 testcases/kernel/syzkaller-repros/Makefile   | 100 ++++++++++++++
 testcases/kernel/syzkaller-repros/README.md  |  45 +++++++
 testcases/kernel/syzkaller-repros/syzwrap.c  | 133 +++++++++++++++++++
 11 files changed, 302 insertions(+), 1 deletion(-)
 create mode 100644 runtest/.gitignore
 create mode 100644 testcases/kernel/syzkaller-repros/.gitignore
 create mode 100644 testcases/kernel/syzkaller-repros/Makefile
 create mode 100644 testcases/kernel/syzkaller-repros/README.md
 create mode 100644 testcases/kernel/syzkaller-repros/syzwrap.c
Signature · 1 line
2.23.0
#1 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

Allows one to run the Syzkaller reproducers as part of the LTP.

---
 .gitmodules                                  |   5 +
 configure.ac                                 |  11 ++
 include/mk/features.mk.default               |   2 +
 include/mk/features.mk.in                    |   2 +
 runtest/.gitignore                           |   1 +
 testcases/kernel/Makefile                    |   1 +
 testcases/kernel/syzkaller-repros/.gitignore |   1 +
 testcases/kernel/syzkaller-repros/Makefile   | 100 ++++++++++++++
 testcases/kernel/syzkaller-repros/README.md  |  45 +++++++
 testcases/kernel/syzkaller-repros/syzwrap.c  | 133 +++++++++++++++++++
 10 files changed, 301 insertions(+)
 create mode 100644 runtest/.gitignore
 create mode 100644 testcases/kernel/syzkaller-repros/.gitignore
 create mode 100644 testcases/kernel/syzkaller-repros/Makefile
 create mode 100644 testcases/kernel/syzkaller-repros/README.md
 create mode 100644 testcases/kernel/syzkaller-repros/syzwrap.c
.gitmodules +5 -0
index 1c9e9c38a..6a2d31f51 100644
@@ -1,3 +1,8 @@
 [submodule "testcases/kernel/mce-test"]
 	path = testcases/kernel/mce-test
 	url = git://git.kernel.org/pub/scm/linux/kernel/git/gong.chen/mce-test.git
+[submodule "testcases/linux-arts"]
+	path = testcases/linux-arts
+	url = https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-arts.git
+	shallow = true
+	ignore = all
configure.ac +11 -0
index 3785dff63..ec4cae483 100644
@@ -184,6 +184,17 @@ else
     AC_SUBST([WITH_OPEN_POSIX_TESTSUITE],["no"])
 fi
 
+AC_ARG_WITH([syzkaller-repros],
+  [AC_HELP_STRING([--with-syzkaller-repros],
+    [compile and install Syzkaller reproducers (default=no)])],
+  [with_syzkaller_repros=$withval]
+)
+if test "x$with_syzkaller_repros" = xyes; then
+    AC_SUBST([WITH_SYZKALLER_REPROS],["yes"])
+else
+    AC_SUBST([WITH_SYZKALLER_REPROS],["no"])
+fi
+
 # testcases/realtime requires bash and python.
 if test "x$with_bash" = xyes && test "x$with_python" = xyes; then
     AC_ARG_WITH([realtime-testsuite],
include/mk/features.mk.default +2 -0
index 3a6cc5176..71fb48c60 100644
@@ -47,3 +47,5 @@ WITH_REALTIME_TESTSUITE		:= no
 else
 WITH_REALTIME_TESTSUITE		:= no
 endif
+
+WITH_SYZKALLER_REPROS		:= no
include/mk/features.mk.in +2 -0
index 8e561b738..3ab7f4721 100644
@@ -47,3 +47,5 @@ WITH_REALTIME_TESTSUITE		:= no
 else
 WITH_REALTIME_TESTSUITE		:= @WITH_REALTIME_TESTSUITE@
 endif
+
+WITH_SYZKALLER_REPROS		:= @WITH_SYZKALLER_REPROS@
runtest/.gitignore new file +1 -0
new file mode 100644
index 000000000..2ae05bfac
@@ -0,0 +1 @@
+syzkaller*
testcases/kernel/Makefile +1 -0
index 3319b3163..0150cfb4f 100644
@@ -53,6 +53,7 @@ SUBDIRS			+= connectors \
 			   sched \
 			   security \
 			   sound \
+			   syzkaller-repros \
 			   tracing \
 			   uevents \
 
testcases/kernel/syzkaller-repros/.gitignore new file +1 -0
new file mode 100644
index 000000000..dbda1c71f
@@ -0,0 +1 @@
+syzwrap
testcases/kernel/syzkaller-repros/Makefile new file +100 -0

100 lines not on this page · raw message

testcases/kernel/syzkaller-repros/README.md new file +45 -0

45 lines not on this page · raw message

testcases/kernel/syzkaller-repros/syzwrap.c new file +133 -0

133 lines not on this page · raw message

Signature · 1 line
2.23.0
#2 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

Quoted 400 lines · Richard Palethorpe

Allows one to run the Syzkaller reproducers as part of the LTP.

---
 .gitmodules                                  |   5 +
 configure.ac                                 |  11 ++
 include/mk/features.mk.default               |   2 +
 include/mk/features.mk.in                    |   2 +
 runtest/.gitignore                           |   1 +
 testcases/kernel/Makefile                    |   1 +
 testcases/kernel/syzkaller-repros/.gitignore |   1 +
 testcases/kernel/syzkaller-repros/Makefile   | 100 ++++++++++++++
 testcases/kernel/syzkaller-repros/README.md  |  45 +++++++
 testcases/kernel/syzkaller-repros/syzwrap.c  | 133 +++++++++++++++++++
 10 files changed, 301 insertions(+)
 create mode 100644 runtest/.gitignore
 create mode 100644 testcases/kernel/syzkaller-repros/.gitignore
 create mode 100644 testcases/kernel/syzkaller-repros/Makefile
 create mode 100644 testcases/kernel/syzkaller-repros/README.md
 create mode 100644 testcases/kernel/syzkaller-repros/syzwrap.c
.gitmodules +5 -0
index 1c9e9c38a..6a2d31f51 100644
@@ -1,3 +1,8 @@
 [submodule "testcases/kernel/mce-test"]
        path = testcases/kernel/mce-test
        url = git://git.kernel.org/pub/scm/linux/kernel/git/gong.chen/mce-test.git
+[submodule "testcases/linux-arts"]
+       path = testcases/linux-arts
+       url = https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-arts.git
+       shallow = true
+       ignore = all
configure.ac +11 -0
index 3785dff63..ec4cae483 100644
@@ -184,6 +184,17 @@ else
     AC_SUBST([WITH_OPEN_POSIX_TESTSUITE],["no"])
 fi
 
+AC_ARG_WITH([syzkaller-repros],
+  [AC_HELP_STRING([--with-syzkaller-repros],
+    [compile and install Syzkaller reproducers (default=no)])],
+  [with_syzkaller_repros=$withval]
+)
+if test "x$with_syzkaller_repros" = xyes; then
+    AC_SUBST([WITH_SYZKALLER_REPROS],["yes"])
+else
+    AC_SUBST([WITH_SYZKALLER_REPROS],["no"])
+fi
+
 # testcases/realtime requires bash and python.
 if test "x$with_bash" = xyes && test "x$with_python" = xyes; then
     AC_ARG_WITH([realtime-testsuite],
include/mk/features.mk.default +2 -0
index 3a6cc5176..71fb48c60 100644
@@ -47,3 +47,5 @@ WITH_REALTIME_TESTSUITE               := no
 else
 WITH_REALTIME_TESTSUITE                := no
 endif
+
+WITH_SYZKALLER_REPROS          := no
include/mk/features.mk.in +2 -0
index 8e561b738..3ab7f4721 100644
@@ -47,3 +47,5 @@ WITH_REALTIME_TESTSUITE               := no
 else
 WITH_REALTIME_TESTSUITE                := @WITH_REALTIME_TESTSUITE@
 endif
+
+WITH_SYZKALLER_REPROS          := @WITH_SYZKALLER_REPROS@
runtest/.gitignore new file +1 -0
new file mode 100644
index 000000000..2ae05bfac
@@ -0,0 +1 @@
+syzkaller*
testcases/kernel/Makefile +1 -0
index 3319b3163..0150cfb4f 100644
@@ -53,6 +53,7 @@ SUBDIRS                       += connectors \
                           sched \
                           security \
                           sound \
+                          syzkaller-repros \
                           tracing \
                           uevents \
 
testcases/kernel/syzkaller-repros/.gitignore new file +1 -0
new file mode 100644
index 000000000..dbda1c71f
@@ -0,0 +1 @@
+syzwrap
testcases/kernel/syzkaller-repros/Makefile new file +100 -0
new file mode 100644
index 000000000..d40d61ac1
@@ -0,0 +1,100 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2019 Linux Test Project
+
+top_srcdir             ?= ../../..
+
+include $(top_srcdir)/include/mk/testcases.mk
+
+CFLAGS                 += -D_GNU_SOURCE
+
+ifeq ($(WITH_SYZKALLER_REPROS),yes)
+
+# The number of reproducers in each runtest file
+SYZKALLER_RUNFILES_SIZE ?= 100
+
+# Extra arguments to pass to syzwrap. Uncomment the below to add some
+# sandboxing.
+# SYZWRAP_ARGS ?= -s
+
+# Location where reproducers are installed
+SYZKALLER_INSTALL_DIR ?= $(abspath $(DESTDIR)/$(prefix)/testcases/bin)
+
+# If the reproducers directory is missing then we automatically clone the repo.
+# We then have to call make recursively to revaluate the targets
+SYZKALLER_REPROS_DIR ?= $(abs_top_srcdir)/testcases/linux-arts/syzkaller-repros/linux
+$(SYZKALLER_REPROS_DIR):
+       git submodule update --init $(abs_top_srcdir)/testcases/linux-arts
+       $(MAKE) syzkaller_runfiles
+
+SYZKALLER_REPROS_SRCS = $(wildcard $(SYZKALLER_REPROS_DIR)/*.c)
+
+# Some useful compiler flags for the LTP will cause problems with the
+# syzkaller repros so the repros have seperate flags
+SYZKALLER_CFLAGS ?= -pthread
+SYZKALLER_REPROS = $(subst $(abs_top_srcdir),$(abs_top_builddir),$(SYZKALLER_REPROS_SRCS:.c=))
+$(SYZKALLER_REPROS): %: %.c
+       -@if grep -q "__NR_mmap2" $^; then \
+               M32="-m32"; \
+       fi; \
+       $(CC) $(SYZKALLER_CFLAGS) $$M32 $(SYZKALLER_LDFLAGS) $^ -o $@; \
+       echo $(CC) $(SYZKALLER_CFLAGS) $$M32 $(SYZKALLER_LDFLAGS) $^ -o $@;
+
+# Generate the names of the runtest files. This uses Shell arithmetic to
+# calculate how many runtest files there will be.
+define SYZKALLER_RUNFILES !=
+       n=$(words $(SYZKALLER_REPROS));
+       m=$(SYZKALLER_RUNFILES_SIZE);
+       i=$$(( $$n / $$m + ($$n % $$m > 0) ));
+       while test $$i -gt 0;
+       do
+               echo $(top_srcdir)/runtest/syzkaller$$i;
+               i=$$(($$i - 1));
+       done
+endef
+
+# Get the index part of a runtest files name
+syz_n = $(subst $(top_srcdir)/runtest/syzkaller,,$(1))
+syz_m = $(SYZKALLER_RUNFILES_SIZE)
+# Gives the index of the first reproducer in a runtest file
+syz_i = $(shell echo $$((($(call syz_n,$(1)) - 1) * $(2) + 1)))
+# Gives the index of the last reproducer in a runtest file
+syz_j = $(shell echo $$(( $(call syz_i,$(1),$(2)) + $(2) - 1 )))
+# Gvien a runtest file name, get the reproducers it should contain
+syz_wordlist = $(wordlist $(call syz_i,$(1),$(syz_m)),$(call syz_j,$(1),$(syz_m)),$(SYZKALLER_REPROS))
+
+define syz_runfile_line
+$(notdir $(exe)) syzwrap $(SYZWRAP_ARGS) -d $(SYZKALLER_INSTALL_DIR) -n $(notdir $(exe))
+
+endef
+
+# Generate the runtest files based on the reproducer names and batch size.
+$(SYZKALLER_RUNFILES): $(SYZKALLER_REPROS)
+       @echo "Writing $@"
+       $(file >$@)
+       $(foreach exe,$(call syz_wordlist,$@),$(file >>$@,$(syz_runfile_line)))
+
+.PHONY: syzkaller_runfiles
+syzkaller_runfiles: $(SYZKALLER_RUNFILES) | $(SYZKALLER_REPROS_DIR)
+
+all: $(SYZKALLER_RUNFILES) | $(SYZKALLER_REPROS_DIR)
+
+# There are too many reproducers to pass all at once to rm, so we just pass
+# one at a time
+syzkaller_clean:
+       $(foreach f, $(SYZKALLER_REPROS), $(RM) $(f))
+CLEAN_DEPS += syzkaller_clean
+CLEAN_TARGETS += $(SYZKALLER_RUNFILES)
+
+INSTALL_MODE ?= 0775
+
+# For some reason part of the path is missing if we just try to install these
+# by adding them to INSTALL_FILES
+SYZKALLER_REPROS_INSTALLED := $(subst $(SYZKALLER_REPROS_DIR),$(SYZKALLER_INSTALL_DIR),$(SYZKALLER_REPROS))
+$(SYZKALLER_REPROS_INSTALLED): $(SYZKALLER_INSTALL_DIR)/%: $(SYZKALLER_REPROS_DIR)/%
+       install -m $(INSTALL_MODE) -T $< $@
+
+install: $(SYZKALLER_REPROS_INSTALLED)
+
+endif
+
+include $(top_srcdir)/include/mk/generic_leaf_target.mk
testcases/kernel/syzkaller-repros/README.md new file +45 -0
new file mode 100644
index 000000000..2c88efd01
@@ -0,0 +1,45 @@
+# LTP wrapper for Syzkaller reproducers
+
+This allows you to run the autogenerated C bug reproducers from the Syzkaller
+fuzzer within the LTP framework. Meaning that you may use an existing test
+runner compatible with the LTP (with some constraints, see below).
+
+## Instructions
+
+1. Run `ltp/configure` with `--with-syzkaller-repros`.
+2. Build and install the LTP as normal.
+3. Run one or more of syzkallerN runtest files where N is a number.
+
+Make will automatically download the reproducers into `testcases/linux-arts`
+using git-submodule if necessary.
+
+By default each runtest file contains 100 reproducers. You may change this by
+overriding `SYZKALLER_RUNFILES_SIZE`.
+
+Extra parameters can be sent to syzwrap using `SYZWRAP_ARGS`. See `syzwrap
+-h`.
+
+## Kernel Requirements
+
+It is strongly recommended that you use KASAN and other debugging kernel
+features. See the Syzkaller documentation for the configuration you should
+use.
+
+## Test Runner Requirements
+
+Unlike most LTP tests these reproducers can leave your system in a broken
+state even if no bug is triggered.
+
+You will need to:
+
+A) Reboot the SUT
+B) Reset at least the root filesystem to a known good state
+
+Every time syzwrap fails.
+
+If syzwrap fails with TBROK or fails to run at all, then you probably need to
+reset the system and rerun that test. If a test fails with TFAIL, you may also
+want to run it once again with a clean state.
+
+It might be the case that some reproducers write to random devices or do other
+things which can effect the outside world.
testcases/kernel/syzkaller-repros/syzwrap.c new file +133 -1
new file mode 100644
index 000000000..9f5d16078
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2019 Richard Palethorpe <[email protected]>
+ *
+ * Run a single reproducer generated by the Syzkaller fuzzer.
+ */
+
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <sys/prctl.h>
+#include <sched.h>
+#include <signal.h>
+#include <stdio.h>
+#include <pwd.h>
+
+#include "tst_test.h"
+#include "tst_taint.h"
+#include "tst_safe_stdio.h"
+
+#define SANDBOX_HELP "\n"\
+       "-s\t Add some sandboxing around the reproducer. This will prevent some\n"\
+       "\t reproducers from creating network devices and thus prevent them from\n"\
+       "\t working. However it will also prevent some reproducers from trashing\n"\
+       "\t the system using root privileges. Note that you may generate the\n"\
+       "\t reproducers with various types of sandboxing built in using\n"\
+       "\t syz-reprolist"
+
+static char *dir;
+static char *name;
+static char *path;
+
+static char *sandbox;
+
+static struct tst_option options[] = {
+       {"d:", &dir, "\n-d PATH\t Mandatory directory containing reproducers"},
+       {"n:", &name, "-n NAME\t Mandatory executable name of reproducer"},
+       {"s", &sandbox, SANDBOX_HELP},
+       {NULL, NULL, NULL}
+};
+
+static void become_nobody(void)
+{
+       struct passwd *pw;
+       int gid, uid;
+
+       setgroups(0, NULL);
+
+       pw = getpwnam("nobody");
+       if (pw) {
+               gid = pw->pw_gid;
+               uid = pw->pw_uid;
+       } else {
+               gid = 65534;
+               uid = 65534;
+       }
+
+       SAFE_SETREGID(gid, gid);
+       SAFE_SETREUID(uid, uid);
+}
+
+static void setup(void)
+{
+       tst_taint_init(TST_TAINT_W | TST_TAINT_D | TST_TAINT_L);
+
+       if (!dir)
+               tst_brk(TBROK, "No reproducer directory specified");
+
+       if (!name)
+               tst_brk(TBROK, "No reproducer name specified");
+
+       tst_res(TINFO, "https://syzkaller.appspot.com/bug?id=%s", name);
+
+       SAFE_ASPRINTF(&path, "%s/%s", dir, name);
+       tst_res(TINFO, "%s", path);
+}
+
+static void run(void)
+{
+       unsigned int backoff = 100;
+       int rem, status, sent_kill = 0;
+       float exec_time_start = (float)tst_timeout_remaining();
+       int pid;
+
+       if (sandbox)
+               SAFE_UNSHARE(CLONE_NEWPID);
+
+       pid = SAFE_FORK();
+       if (!pid) {
+               if (sandbox) {
+                       SAFE_UNSHARE(CLONE_NEWNET);
+                       become_nobody();
+               }
+
+               if (prctl(PR_SET_DUMPABLE, 1, 0, 0, 0)) {
+                       tst_res(TWARN | TERRNO,
+                               "Failed to set dumpable; won't be able to open /proc/self/*");
+               }
+
+               execl(path, name, NULL);
+               tst_brk(TBROK | TERRNO, "Failed to exec reproducer");
+       }
+
+       while (!waitpid(pid, &status, WNOHANG)) {
+               rem = tst_timeout_remaining();
+
+               if (!sent_kill && rem / exec_time_start < 0.5) {
+                       tst_res(TINFO, "Timeout; killing reproducer");
+
+                       TEST(kill(pid, SIGKILL));
+                       if (TST_RET == -1)
+                               tst_res(TWARN | TTERRNO, "kill() failed");
+                       else
+                               sent_kill = 1;
+               }
+
+               usleep(backoff);
+               backoff = MIN(2 * backoff, 1000000);
+       }
+
+       if (tst_taint_check()) {
+               tst_res(TFAIL, "Kernel is tainted");
+       } else {
+               tst_res(TPASS, "Kernel is not tainted");
+       }
+}
+
+static struct tst_test test = {
+       .setup = setup,
+       .test_all = run,
+       .options = options,
+       .needs_tmpdir = 1,
+       .forks_child = 1
+};
--
Quoted 2 lines

2.23.0

+syzkaller mailing list FTR

#3 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

Quoted 139 lines · Dmitry Vyukov via ltp
Quoted 27 lines · Richard Palethorpe

Allows one to run the Syzkaller reproducers as part of the LTP.

---
 .gitmodules                                  |   5 +
 configure.ac                                 |  11 ++
 include/mk/features.mk.default               |   2 +
 include/mk/features.mk.in                    |   2 +
 runtest/.gitignore                           |   1 +
 testcases/kernel/Makefile                    |   1 +
 testcases/kernel/syzkaller-repros/.gitignore |   1 +
 testcases/kernel/syzkaller-repros/Makefile   | 100 ++++++++++++++
 testcases/kernel/syzkaller-repros/README.md  |  45 +++++++
 testcases/kernel/syzkaller-repros/syzwrap.c  | 133 +++++++++++++++++++
 10 files changed, 301 insertions(+)
 create mode 100644 runtest/.gitignore
 create mode 100644 testcases/kernel/syzkaller-repros/.gitignore
 create mode 100644 testcases/kernel/syzkaller-repros/Makefile
 create mode 100644 testcases/kernel/syzkaller-repros/README.md
 create mode 100644 testcases/kernel/syzkaller-repros/syzwrap.c
.gitmodules +0 -0
index 1c9e9c38a..6a2d31f51 100644

.gitmodules line 3

Quoted 3 lines
.gitmodules +0 -0
@@ -1,3 +1,8 @@
 [submodule "testcases/kernel/mce-test"]
        path = testcases/kernel/mce-test
        url = git://

git.kernel.org/pub/scm/linux/kernel/git/gong.chen/mce-test.git

.gitmodules

Quoted 3 lines
.gitmodules +3 -0
+[submodule "testcases/linux-arts"]
+       path = testcases/linux-arts
+       url =

https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-arts.git

include/mk/features.mk

Quoted 25 lines
.gitmodules +2 -0
+       shallow = true
+       ignore = all
configure.ac +11 -0
index 3785dff63..ec4cae483 100644
@@ -184,6 +184,17 @@ else
     AC_SUBST([WITH_OPEN_POSIX_TESTSUITE],["no"])
 fi
 
+AC_ARG_WITH([syzkaller-repros],
+  [AC_HELP_STRING([--with-syzkaller-repros],
+    [compile and install Syzkaller reproducers (default=no)])],
+  [with_syzkaller_repros=$withval]
+)
+if test "x$with_syzkaller_repros" = xyes; then
+    AC_SUBST([WITH_SYZKALLER_REPROS],["yes"])
+else
+    AC_SUBST([WITH_SYZKALLER_REPROS],["no"])
+fi
+
 # testcases/realtime requires bash and python.
 if test "x$with_bash" = xyes && test "x$with_python" = xyes; then
     AC_ARG_WITH([realtime-testsuite],
include/mk/features.mk renamed from include/mk/features.mk.default +0 -0

.default

Review

Quoted 39 lines

index 3a6cc5176..71fb48c60 100644

include/mk/features.mk.default +2 -0
@@ -47,3 +47,5 @@ WITH_REALTIME_TESTSUITE               := no
 else
 WITH_REALTIME_TESTSUITE                := no
 endif
+
+WITH_SYZKALLER_REPROS          := no
include/mk/features.mk.in +2 -0
index 8e561b738..3ab7f4721 100644
@@ -47,3 +47,5 @@ WITH_REALTIME_TESTSUITE               := no
 else
 WITH_REALTIME_TESTSUITE                := @WITH_REALTIME_TESTSUITE@
 endif
+
+WITH_SYZKALLER_REPROS          := @WITH_SYZKALLER_REPROS@
runtest/.gitignore new file +1 -0
new file mode 100644
index 000000000..2ae05bfac
@@ -0,0 +1 @@
+syzkaller*
testcases/kernel/Makefile +1 -0
index 3319b3163..0150cfb4f 100644
@@ -53,6 +53,7 @@ SUBDIRS                       += connectors \
                           sched \
                           security \
                           sound \
+                          syzkaller-repros \
                           tracing \
                           uevents \
 
(quoted diff) +0 -0

b/testcases/kernel/syzkaller-repros/.gitignore

Review

Quoted 7 lines

new file mode 100644
index 000000000..dbda1c71f

testcases/kernel/syzkaller-repros/.gitignore new file +1 -0
@@ -0,0 +1 @@
+syzwrap
(quoted diff) +0 -0

b/testcases/kernel/syzkaller-repros/Makefile

Quoted 17 lines

new file mode 100644
index 000000000..d40d61ac1

testcases/kernel/syzkaller-repros/Makefile new file +12 -0
@@ -0,0 +1,100 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2019 Linux Test Project
+
+top_srcdir             ?= ../../..
+
+include $(top_srcdir)/include/mk/testcases.mk
+
+CFLAGS                 += -D_GNU_SOURCE
+
+ifeq ($(WITH_SYZKALLER_REPROS),yes)
+
+# The number of reproducers in each runtest file

testcases/kernel/syzkaller-repros/Makefile line 22

Quoted 10 lines
testcases/kernel/syzkaller-repros/Makefile new file +10 -0
+SYZKALLER_RUNFILES_SIZE ?= 100
+
+# Extra arguments to pass to syzwrap. Uncomment the below to add some
+# sandboxing.
+# SYZWRAP_ARGS ?= -s
+
+# Location where reproducers are installed
+SYZKALLER_INSTALL_DIR ?= $(abspath $(DESTDIR)/$(prefix)/testcases/bin)
+
+# If the reproducers directory is missing then we automatically clone

the repo.

Quoted 7 lines

testcases/kernel/syzkaller-repros/Makefile line 24

Quoted 2 lines
testcases/kernel/syzkaller-repros/Makefile new file +2 -0
+# We then have to call make recursively to revaluate the targets
+SYZKALLER_REPROS_DIR ?=

$(abs_top_srcdir)/testcases/linux-arts/syzkaller-repros/linux

testcases/kernel/syzkaller-repros/Makefile line 26

Quoted 2 lines
testcases/kernel/syzkaller-repros/Makefile new file +2 -0
+$(SYZKALLER_REPROS_DIR):
+       git submodule update --init

$(abs_top_srcdir)/testcases/linux-arts

Just to try build it in LTP and hit errors:

# cd ltp-new/
# make autotools
# ./configure --with-syzkaller-repros
# make -j32
...
error: pathspec '/root/ltp-new/testcases/linux-arts' did not match any
file(s) known to git
make[3]: *** [/root/ltp-new/testcases/kernel/syzkaller-repros/Makefile:26:
/root/ltp-new/testcases/linux-arts/syzkaller-repros/linux] Error 1
make[3]: Leaving directory '/root/ltp-new/testcases/kernel/syzkaller-repros'
make[2]: *** [../../include/mk/generic_trunk_target.inc:93: all] Error 2
make[2]: Leaving directory '/root/ltp-new/testcases/kernel'
make[1]: *** [../include/mk/generic_trunk_target.inc:93: all] Error 2
make[1]: Leaving directory '/root/ltp-new/testcases'
make: *** [Makefile:108: testcases-all] Error 2
Quoted 304 lines

testcases/kernel/syzkaller-repros/Makefile

Quoted 8 lines
testcases/kernel/syzkaller-repros/Makefile new file +8 -0
+       $(MAKE) syzkaller_runfiles
+
+SYZKALLER_REPROS_SRCS = $(wildcard $(SYZKALLER_REPROS_DIR)/*.c)
+
+# Some useful compiler flags for the LTP will cause problems with the
+# syzkaller repros so the repros have seperate flags
+SYZKALLER_CFLAGS ?= -pthread
+SYZKALLER_REPROS = $(subst

$(abs_top_srcdir),$(abs_top_builddir),$(SYZKALLER_REPROS_SRCS:.c=))

testcases/kernel/syzkaller-repros/Makefile

Quoted 6 lines
testcases/kernel/syzkaller-repros/Makefile new file +6 -0
+$(SYZKALLER_REPROS): %: %.c
+       -@if grep -q "__NR_mmap2" $^; then \
+               M32="-m32"; \
+       fi; \
+       $(CC) $(SYZKALLER_CFLAGS) $$M32 $(SYZKALLER_LDFLAGS) $^ -o $@; \
+       echo $(CC) $(SYZKALLER_CFLAGS) $$M32 $(SYZKALLER_LDFLAGS) $^ -o

$@;

Quoted 13 lines
testcases/kernel/syzkaller-repros/Makefile new file +13 -0
+
+# Generate the names of the runtest files. This uses Shell arithmetic to
+# calculate how many runtest files there will be.
+define SYZKALLER_RUNFILES !=
+       n=$(words $(SYZKALLER_REPROS));
+       m=$(SYZKALLER_RUNFILES_SIZE);
+       i=$$(( $$n / $$m + ($$n % $$m > 0) ));
+       while test $$i -gt 0;
+       do
+               echo $(top_srcdir)/runtest/syzkaller$$i;
+               i=$$(($$i - 1));
+       done
+endef

testcases/kernel/syzkaller-repros/Makefile

Quoted 10 lines
testcases/kernel/syzkaller-repros/Makefile new file +10 -0
+
+# Get the index part of a runtest files name
+syz_n = $(subst $(top_srcdir)/runtest/syzkaller,,$(1))
+syz_m = $(SYZKALLER_RUNFILES_SIZE)
+# Gives the index of the first reproducer in a runtest file
+syz_i = $(shell echo $$((($(call syz_n,$(1)) - 1) * $(2) + 1)))
+# Gives the index of the last reproducer in a runtest file
+syz_j = $(shell echo $$(( $(call syz_i,$(1),$(2)) + $(2) - 1 )))
+# Gvien a runtest file name, get the reproducers it should contain
+syz_wordlist = $(wordlist $(call syz_i,$(1),$(syz_m)),$(call

syz_j,$(1),$(syz_m)),$(SYZKALLER_REPROS))

testcases/kernel/syzkaller-repros/Makefile

Quoted 3 lines
testcases/kernel/syzkaller-repros/Makefile new file +3 -0
+
+define syz_runfile_line
+$(notdir $(exe)) syzwrap $(SYZWRAP_ARGS) -d $(SYZKALLER_INSTALL_DIR) -n

$(notdir $(exe))

testcases/kernel/syzkaller-repros/Makefile

Quoted 4 lines
testcases/kernel/syzkaller-repros/Makefile new file +4 -0
+
+endef
+
+# Generate the runtest files based on the reproducer names and batch

size.

testcases/kernel/syzkaller-repros/Makefile

Quoted 12 lines
testcases/kernel/syzkaller-repros/Makefile new file +4 -0
+$(SYZKALLER_RUNFILES): $(SYZKALLER_REPROS)
+       @echo "Writing $@"
+       $(file >$@)
+       $(foreach exe,$(call syz_wordlist,$@),$(file
Quoted 1 line

$@,$(syz_runfile_line)))

testcases/kernel/syzkaller-repros/Makefile new file +7 -0
+
+.PHONY: syzkaller_runfiles
+syzkaller_runfiles: $(SYZKALLER_RUNFILES) | $(SYZKALLER_REPROS_DIR)
+
+all: $(SYZKALLER_RUNFILES) | $(SYZKALLER_REPROS_DIR)
+
+# There are too many reproducers to pass all at once to rm, so we just

pass

testcases/kernel/syzkaller-repros/Makefile

Quoted 9 lines
testcases/kernel/syzkaller-repros/Makefile new file +9 -0
+# one at a time
+syzkaller_clean:
+       $(foreach f, $(SYZKALLER_REPROS), $(RM) $(f))
+CLEAN_DEPS += syzkaller_clean
+CLEAN_TARGETS += $(SYZKALLER_RUNFILES)
+
+INSTALL_MODE ?= 0775
+
+# For some reason part of the path is missing if we just try to install

these

testcases/kernel/syzkaller-repros/Makefile

Quoted 2 lines
testcases/kernel/syzkaller-repros/Makefile new file +2 -0
+# by adding them to INSTALL_FILES
+SYZKALLER_REPROS_INSTALLED := $(subst

$(SYZKALLER_REPROS_DIR),$(SYZKALLER_INSTALL_DIR),$(SYZKALLER_REPROS))

Quoted 1 line

+$(SYZKALLER_REPROS_INSTALLED): $(SYZKALLER_INSTALL_DIR)/%:

$(SYZKALLER_REPROS_DIR)/%

Review

Quoted 8 lines
testcases/kernel/syzkaller-repros/Makefile new file +7 -0
+       install -m $(INSTALL_MODE) -T $< $@
+
+install: $(SYZKALLER_REPROS_INSTALLED)
+
+endif
+
+include $(top_srcdir)/include/mk/generic_leaf_target.mk
(quoted diff) +0 -0

b/testcases/kernel/syzkaller-repros/README.md

testcases/kernel/syzkaller-repros/README.md line 3

Quoted 8 lines

new file mode 100644
index 000000000..2c88efd01

testcases/kernel/syzkaller-repros/README.md new file +3 -0
@@ -0,0 +1,45 @@
+# LTP wrapper for Syzkaller reproducers
+
+This allows you to run the autogenerated C bug reproducers from the

Syzkaller

testcases/kernel/syzkaller-repros/README.md line 4

Quoted 1 line
testcases/kernel/syzkaller-repros/README.md new file +1 -0
+fuzzer within the LTP framework. Meaning that you may use an existing

test

testcases/kernel/syzkaller-repros/README.md

Quoted 9 lines
testcases/kernel/syzkaller-repros/README.md new file +9 -0
+runner compatible with the LTP (with some constraints, see below).
+
+## Instructions
+
+1. Run `ltp/configure` with `--with-syzkaller-repros`.
+2. Build and install the LTP as normal.
+3. Run one or more of syzkallerN runtest files where N is a number.
+
+Make will automatically download the reproducers into

testcases/linux-arts

testcases/kernel/syzkaller-repros/README.md

Quoted 3 lines
testcases/kernel/syzkaller-repros/README.md new file +3 -0
+using git-submodule if necessary.
+
+By default each runtest file contains 100 reproducers. You may change

this by

testcases/kernel/syzkaller-repros/README.md

Quoted 3 lines
testcases/kernel/syzkaller-repros/README.md new file +3 -0
+overriding `SYZKALLER_RUNFILES_SIZE`.
+
+Extra parameters can be sent to syzwrap using `SYZWRAP_ARGS`. See

`syzwrap

testcases/kernel/syzkaller-repros/README.md

Quoted 6 lines
testcases/kernel/syzkaller-repros/README.md new file +6 -0
+-h`.
+
+## Kernel Requirements
+
+It is strongly recommended that you use KASAN and other debugging kernel
+features. See the Syzkaller documentation for the configuration you

should

testcases/kernel/syzkaller-repros/README.md

Quoted 5 lines
testcases/kernel/syzkaller-repros/README.md new file +5 -0
+use.
+
+## Test Runner Requirements
+
+Unlike most LTP tests these reproducers can leave your system in a

broken

testcases/kernel/syzkaller-repros/README.md

Quoted 10 lines
testcases/kernel/syzkaller-repros/README.md new file +10 -0
+state even if no bug is triggered.
+
+You will need to:
+
+A) Reboot the SUT
+B) Reset at least the root filesystem to a known good state
+
+Every time syzwrap fails.
+
+If syzwrap fails with TBROK or fails to run at all, then you probably

need to

testcases/kernel/syzkaller-repros/README.md line 41

Quoted 1 line
testcases/kernel/syzkaller-repros/README.md new file +1 -0
+reset the system and rerun that test. If a test fails with TFAIL, you

may also

testcases/kernel/syzkaller-repros/README.md

Quoted 3 lines
testcases/kernel/syzkaller-repros/README.md new file +3 -0
+want to run it once again with a clean state.
+
+It might be the case that some reproducers write to random devices or

do other

Review

Quoted 2 lines
testcases/kernel/syzkaller-repros/README.md new file +1 -0
+things which can effect the outside world.
(quoted diff) +0 -0

b/testcases/kernel/syzkaller-repros/syzwrap.c

Quoted 16 lines

new file mode 100644
index 000000000..9f5d16078

testcases/kernel/syzkaller-repros/syzwrap.c new file +11 -0
@@ -0,0 +1,133 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2019 Richard Palethorpe <[email protected]>
+ *
+ * Run a single reproducer generated by the Syzkaller fuzzer.
+ */
+
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <sys/prctl.h>
+#include <sched.h>

testcases/kernel/syzkaller-repros/syzwrap.c line 21

Quoted 10 lines
testcases/kernel/syzkaller-repros/syzwrap.c new file +10 -0
+#include <signal.h>
+#include <stdio.h>
+#include <pwd.h>
+
+#include "tst_test.h"
+#include "tst_taint.h"
+#include "tst_safe_stdio.h"
+
+#define SANDBOX_HELP "\n"\
+       "-s\t Add some sandboxing around the reproducer. This will

prevent some\n"\

testcases/kernel/syzkaller-repros/syzwrap.c line 22

Quoted 1 line
testcases/kernel/syzkaller-repros/syzwrap.c new file +1 -0
+       "\t reproducers from creating network devices and thus prevent

them from\n"\

testcases/kernel/syzkaller-repros/syzwrap.c line 23

Quoted 1 line
testcases/kernel/syzkaller-repros/syzwrap.c new file +1 -0
+       "\t working. However it will also prevent some reproducers from

trashing\n"\

testcases/kernel/syzkaller-repros/syzwrap.c line 24

Quoted 1 line
testcases/kernel/syzkaller-repros/syzwrap.c new file +1 -0
+       "\t the system using root privileges. Note that you may generate

the\n"\

testcases/kernel/syzkaller-repros/syzwrap.c line 25

Quoted 1 line
testcases/kernel/syzkaller-repros/syzwrap.c new file +1 -0
+       "\t reproducers with various types of sandboxing built in

using\n"\

testcases/kernel/syzkaller-repros/syzwrap.c

Quoted 10 lines
testcases/kernel/syzkaller-repros/syzwrap.c new file +10 -0
+       "\t syz-reprolist"
+
+static char *dir;
+static char *name;
+static char *path;
+
+static char *sandbox;
+
+static struct tst_option options[] = {
+       {"d:", &dir, "\n-d PATH\t Mandatory directory containing

reproducers"},

testcases/kernel/syzkaller-repros/syzwrap.c line 36

Quoted 1 line
testcases/kernel/syzkaller-repros/syzwrap.c new file +1 -0
+       {"n:", &name, "-n NAME\t Mandatory executable name of

reproducer"},

Quoted 50 lines
testcases/kernel/syzkaller-repros/syzwrap.c new file +50 -0
+       {"s", &sandbox, SANDBOX_HELP},
+       {NULL, NULL, NULL}
+};
+
+static void become_nobody(void)
+{
+       struct passwd *pw;
+       int gid, uid;
+
+       setgroups(0, NULL);
+
+       pw = getpwnam("nobody");
+       if (pw) {
+               gid = pw->pw_gid;
+               uid = pw->pw_uid;
+       } else {
+               gid = 65534;
+               uid = 65534;
+       }
+
+       SAFE_SETREGID(gid, gid);
+       SAFE_SETREUID(uid, uid);
+}
+
+static void setup(void)
+{
+       tst_taint_init(TST_TAINT_W | TST_TAINT_D | TST_TAINT_L);
+
+       if (!dir)
+               tst_brk(TBROK, "No reproducer directory specified");
+
+       if (!name)
+               tst_brk(TBROK, "No reproducer name specified");
+
+       tst_res(TINFO, "https://syzkaller.appspot.com/bug?id=%s", name);
+
+       SAFE_ASPRINTF(&path, "%s/%s", dir, name);
+       tst_res(TINFO, "%s", path);
+}
+
+static void run(void)
+{
+       unsigned int backoff = 100;
+       int rem, status, sent_kill = 0;
+       float exec_time_start = (float)tst_timeout_remaining();
+       int pid;
+
+       if (sandbox)
+               SAFE_UNSHARE(CLONE_NEWPID);
+

testcases/kernel/syzkaller-repros/syzwrap.c

Quoted 10 lines
testcases/kernel/syzkaller-repros/syzwrap.c new file +10 -0
+       pid = SAFE_FORK();
+       if (!pid) {
+               if (sandbox) {
+                       SAFE_UNSHARE(CLONE_NEWNET);
+                       become_nobody();
+               }
+
+               if (prctl(PR_SET_DUMPABLE, 1, 0, 0, 0)) {
+                       tst_res(TWARN | TERRNO,
+                               "Failed to set dumpable; won't be able

to open /proc/self/*");

Quoted 5 lines
testcases/kernel/syzkaller-repros/syzwrap.c new file +5 -0
+               }
+
+               execl(path, name, NULL);
+               tst_brk(TBROK | TERRNO, "Failed to exec reproducer");
+       }

testcases/kernel/syzkaller-repros/syzwrap.c

Quoted 10 lines
testcases/kernel/syzkaller-repros/syzwrap.c new file +10 -0
+
+       while (!waitpid(pid, &status, WNOHANG)) {
+               rem = tst_timeout_remaining();
+
+               if (!sent_kill && rem / exec_time_start < 0.5) {
+                       tst_res(TINFO, "Timeout; killing reproducer");
+
+                       TEST(kill(pid, SIGKILL));
+                       if (TST_RET == -1)
+                               tst_res(TWARN | TTERRNO, "kill()

failed");

Quoted 23 lines
testcases/kernel/syzkaller-repros/syzwrap.c new file +22 -1
+                       else
+                               sent_kill = 1;
+               }
+
+               usleep(backoff);
+               backoff = MIN(2 * backoff, 1000000);
+       }
+
+       if (tst_taint_check()) {
+               tst_res(TFAIL, "Kernel is tainted");
+       } else {
+               tst_res(TPASS, "Kernel is not tainted");
+       }
+}
+
+static struct tst_test test = {
+       .setup = setup,
+       .test_all = run,
+       .options = options,
+       .needs_tmpdir = 1,
+       .forks_child = 1
+};
--
Quoted 2 lines

2.23.0

+syzkaller mailing list FTR

--
Mailing list info: https://lists.linux.it/listinfo/ltp

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191112/c246ba42/attachment-0001.htm>
#4 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

Hello,

Quoted 4 lines · Li Wang

Just to try build it in LTP and hit errors:

Quoted 17 lines
# cd ltp-new/
# make autotools
# ./configure --with-syzkaller-repros
# make -j32
...
error: pathspec '/root/ltp-new/testcases/linux-arts' did not match any
file(s) known to git
make[3]: *** [/root/ltp-new/testcases/kernel/syzkaller-repros/Makefile:26:
/root/ltp-new/testcases/linux-arts/syzkaller-repros/linux] Error 1
make[3]: Leaving directory '/root/ltp-new/testcases/kernel/syzkaller-repros'
make[2]: *** [../../include/mk/generic_trunk_target.inc:93: all] Error 2
make[2]: Leaving directory '/root/ltp-new/testcases/kernel'
make[1]: *** [../include/mk/generic_trunk_target.inc:93: all] Error 2
make[1]: Leaving directory '/root/ltp-new/testcases'
make: *** [Makefile:108: testcases-all] Error 2

What happens if you try to pull the git submodule manually?

i.e. do git submodule update --init testcases/linux-arts

It looks like it failed on the line where it gets the submodule, so I am wondering if you have an old git version?

Signature · 2 lines
Thank you,
Richard.
#5 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

Quoted 28 lines · Richard Palethorpe

Hello,

Quoted 4 lines · Li Wang

Just to try build it in LTP and hit errors:

Quoted 8 lines
# cd ltp-new/
# make autotools
# ./configure --with-syzkaller-repros
# make -j32
...
error: pathspec '/root/ltp-new/testcases/linux-arts' did not match any
file(s) known to git
make[3]: ***

[/root/ltp-new/testcases/kernel/syzkaller-repros/Makefile:26:

Quoted 2 lines

/root/ltp-new/testcases/linux-arts/syzkaller-repros/linux] Error 1 make[3]: Leaving directory

'/root/ltp-new/testcases/kernel/syzkaller-repros'

Quoted 7 lines

make[2]: *** [../../include/mk/generic_trunk_target.inc:93: all] Error 2 make[2]: Leaving directory '/root/ltp-new/testcases/kernel' make[1]: *** [../include/mk/generic_trunk_target.inc:93: all] Error 2 make[1]: Leaving directory '/root/ltp-new/testcases'
make: *** [Makefile:108: testcases-all] Error 2

Quoted 2 lines

What happens if you try to pull the git submodule manually?

# pwd
/root/ltp-new
# git submodule update --init testcases/linux-arts
error: pathspec 'testcases/linux-arts' did not match any file(s) known to
git
Quoted 5 lines

i.e. do git submodule update --init testcases/linux-arts

It looks like it failed on the line where it gets the submodule, so I am wondering if you have an old git version?

Not sure if that related to git-version, I'm trying to look into it.

# git --version
git version 2.23.0
# uname -r
5.3.8-300.fc31.x86_64
Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191112/bc3ad8ad/attachment-0001.htm>
#6 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

Hi Richard,

Quoted 44 lines · Li Wang
Quoted 28 lines · Richard Palethorpe

Hello,

Quoted 4 lines · Li Wang

Just to try build it in LTP and hit errors:

Quoted 8 lines
# cd ltp-new/
# make autotools
# ./configure --with-syzkaller-repros
# make -j32
...
error: pathspec '/root/ltp-new/testcases/linux-arts' did not match any
file(s) known to git
make[3]: ***

[/root/ltp-new/testcases/kernel/syzkaller-repros/Makefile:26:

Quoted 2 lines

/root/ltp-new/testcases/linux-arts/syzkaller-repros/linux] Error 1 make[3]: Leaving directory

'/root/ltp-new/testcases/kernel/syzkaller-repros'

Quoted 7 lines

make[2]: *** [../../include/mk/generic_trunk_target.inc:93: all] Error 2 make[2]: Leaving directory '/root/ltp-new/testcases/kernel' make[1]: *** [../include/mk/generic_trunk_target.inc:93: all] Error 2 make[1]: Leaving directory '/root/ltp-new/testcases'
make: *** [Makefile:108: testcases-all] Error 2

Quoted 2 lines

What happens if you try to pull the git submodule manually?

# pwd
/root/ltp-new
# git submodule update --init testcases/linux-arts
error: pathspec 'testcases/linux-arts' did not match any file(s) known to
git
Quoted 1 line
Quoted 8 lines
Quoted 4 lines

i.e. do git submodule update --init testcases/linux-arts

It looks like it failed on the line where it gets the submodule, so I am wondering if you have an old git version?

Not sure if that related to git-version, I'm trying to look into it.

I have tried many times with different systems but still not working.

My question is did you add the submodule via 'git-submodule add' commands? or just modify the .gitmodules file by hand?

i.e: `git submodule add
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-arts.git
testcases/linux-arts`

I don't understand why I can't see the subproject commit-id in your patch. I have to perform git-submodule-add locally again then it could compile for me.

i.e:

testcases/linux-arts new file +1 -0
new file mode 160000
index 0000000..07759b8
@@ -0,0 +1 @@
+Subproject commit 07759b820a9cbf01333d861d8eb2613b20d1ede4

Or did I missing anything?

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191113/9ab76791/attachment.htm>
#7 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

Hello,

Quoted 79 lines · Li Wang

Hi Richard,

Quoted 44 lines · Li Wang
Quoted 28 lines · Richard Palethorpe

Hello,

Quoted 4 lines · Li Wang

Just to try build it in LTP and hit errors:

Quoted 8 lines
# cd ltp-new/
# make autotools
# ./configure --with-syzkaller-repros
# make -j32
...
error: pathspec '/root/ltp-new/testcases/linux-arts' did not match any
file(s) known to git
make[3]: ***

[/root/ltp-new/testcases/kernel/syzkaller-repros/Makefile:26:

Quoted 2 lines

/root/ltp-new/testcases/linux-arts/syzkaller-repros/linux] Error 1 make[3]: Leaving directory

'/root/ltp-new/testcases/kernel/syzkaller-repros'

Quoted 7 lines

make[2]: *** [../../include/mk/generic_trunk_target.inc:93: all] Error 2 make[2]: Leaving directory '/root/ltp-new/testcases/kernel' make[1]: *** [../include/mk/generic_trunk_target.inc:93: all] Error 2 make[1]: Leaving directory '/root/ltp-new/testcases'
make: *** [Makefile:108: testcases-all] Error 2

Quoted 2 lines

What happens if you try to pull the git submodule manually?

# pwd
/root/ltp-new
# git submodule update --init testcases/linux-arts
error: pathspec 'testcases/linux-arts' did not match any file(s) known to
git
Quoted 1 line
Quoted 8 lines
Quoted 4 lines

i.e. do git submodule update --init testcases/linux-arts

It looks like it failed on the line where it gets the submodule, so I am wondering if you have an old git version?

Not sure if that related to git-version, I'm trying to look into it.

I have tried many times with different systems but still not working.

My question is did you add the submodule via 'git-submodule add' commands? or just modify the .gitmodules file by hand?

i.e: `git submodule add
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-arts.git
testcases/linux-arts`

I don't understand why I can't see the subproject commit-id in your patch. I have to perform git-submodule-add locally again then it could compile for me.

i.e:

testcases/linux-arts new file +1 -0
new file mode 160000
index 0000000..07759b8
@@ -0,0 +1 @@
+Subproject commit 07759b820a9cbf01333d861d8eb2613b20d1ede4
Quoted 1 line

Or did I missing anything?

Sorry, I think it is probably my fault. I added the submodule properly, but I set 'ignore = all' to .gitmodules which prevents the commit-id from being added to the patchset.

I set ignore=all because I thought it would probably be best to always get the HEAD of linux-arts, at least to begin with. Otherwise we have to remember to update it.

--
Thank you,
Richard.

#8 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

In reply to Richard Palethorpe

Hi Richard,

Some more queries below.

configure.ac else · line 190

Quoted 9 lines · Richard Palethorpe

Allows one to run the Syzkaller reproducers as part of the LTP.

...

configure.ac +4 -0
+AC_ARG_WITH([syzkaller-repros],
+  [AC_HELP_STRING([--with-syzkaller-repros],
+    [compile and install Syzkaller reproducers (default=no)])],
+  [with_syzkaller_repros=$withval]

To strictly, the [action-if-not-given] should be added too?

Quoted 14 lines
configure.ac +6 -0
+)
+if test "x$with_syzkaller_repros" = xyes; then
+    AC_SUBST([WITH_SYZKALLER_REPROS],["yes"])
+else
+    AC_SUBST([WITH_SYZKALLER_REPROS],["no"])
+fi
...
runtest/.gitignore new file +0 -0
new file mode 100644
index 000000000..2ae05bfac

runtest/.gitignore line 1

Quoted 1 line
runtest/.gitignore new file +1 -0
@@ -0,0 +1 @@
+syzkaller*

Why adding syzkaller* in here?

Quoted 31 lines
testcases/kernel/Makefile +1 -0
index 3319b3163..0150cfb4f 100644
@@ -53,6 +53,7 @@ SUBDIRS                       += connectors \
                           sched \
                           security \
                           sound \
+                          syzkaller-repros \
                           tracing \
                           uevents \
 
testcases/kernel/syzkaller-repros/.gitignore new file +1 -0
new file mode 100644
index 000000000..dbda1c71f
@@ -0,0 +1 @@
+syzwrap
testcases/kernel/syzkaller-repros/Makefile new file +4 -0
new file mode 100644
...
+# Some useful compiler flags for the LTP will cause problems with the
+# syzkaller repros so the repros have seperate flags
+SYZKALLER_CFLAGS ?= -pthread
+SYZKALLER_REPROS = $(subst

$(abs_top_srcdir),$(abs_top_builddir),$(SYZKALLER_REPROS_SRCS:.c=)) +$(SYZKALLER_REPROS): %: %.c

testcases/kernel/syzkaller-repros/Makefile line 37

Quoted 2 lines
testcases/kernel/syzkaller-repros/Makefile new file +2 -0
+       -@if grep -q "__NR_mmap2" $^; then \
+               M32="-m32"; \

I got compiling errors on s390x:
gcc: error: unrecognized command line option ?-m32?; did you mean ?-m31??

My other concern is syzkaller (I guess maybe) have some package dependencies, and that will break the compiler phase on the embedded system.

Quoted 8 lines

....

testcases/kernel/syzkaller-repros/Makefile new file +6 -0
+       while (!waitpid(pid, &status, WNOHANG)) {
+               rem = tst_timeout_remaining();
+
+               if (!sent_kill && rem / exec_time_start < 0.5) {
+                       tst_res(TINFO, "Timeout; killing reproducer");
+

testcases/kernel/syzkaller-repros/Makefile

Quoted 10 lines
testcases/kernel/syzkaller-repros/Makefile new file +10 -0
+                       TEST(kill(pid, SIGKILL));
+                       if (TST_RET == -1)
+                               tst_res(TWARN | TTERRNO, "kill() failed");
+                       else
+                               sent_kill = 1;
+               }
+
+               usleep(backoff);
+               backoff = MIN(2 * backoff, 1000000);
+       }

Force to kill a timeout test process is fine, but one thing makes me worried is that we don't do any cleanup work(e.g. release hugepage, devices or other resources) for the children, that very probably causes many additional issues in the next testing and not easy to reproduce it in a new run.

testcases/kernel/syzkaller-repros/Makefile

Quoted 6 lines
testcases/kernel/syzkaller-repros/Makefile new file +6 -0
+
+       if (tst_taint_check()) {
+               tst_res(TFAIL, "Kernel is tainted");
+       } else {
+               tst_res(TPASS, "Kernel is not tainted");
+       }

If this is the only condition to judge if all tests pass or not, we may miss some test failure logs after running, unless we don't care about that.

Btw, I can't even finish one round for the test then system panic there.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20191126/afa8ad90/attachment.htm>
#9 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

Hello,

Quoted 11 lines · Li Wang

Hi Richard,

Some more queries below.

Quoted 4 lines · Richard Palethorpe

Allows one to run the Syzkaller reproducers as part of the LTP.

...

Quoted 7 lines

configure.ac else · line 190

Quoted 5 lines
configure.ac +4 -0
+AC_ARG_WITH([syzkaller-repros],
+  [AC_HELP_STRING([--with-syzkaller-repros],
+    [compile and install Syzkaller reproducers (default=no)])],
+  [with_syzkaller_repros=$withval]

To strictly, the [action-if-not-given] should be added too?

Were the other 'with' options updated recently to have that? I just copied this from the other options.

Quoted 11 lines
Quoted 11 lines
configure.ac +6 -0
+)
+if test "x$with_syzkaller_repros" = xyes; then
+    AC_SUBST([WITH_SYZKALLER_REPROS],["yes"])
+else
+    AC_SUBST([WITH_SYZKALLER_REPROS],["no"])
+fi
...
runtest/.gitignore new file +0 -0
new file mode 100644
Quoted 8 lines

runtest/.gitignore line 1

Quoted 6 lines

index 000000000..2ae05bfac

runtest/.gitignore new file +1 -0
@@ -0,0 +1 @@
+syzkaller*

Why adding syzkaller* in here?

The runtest files are automatically generated.

Quoted 31 lines
Quoted 31 lines
testcases/kernel/Makefile +1 -0
index 3319b3163..0150cfb4f 100644
@@ -53,6 +53,7 @@ SUBDIRS                       += connectors \
                           sched \
                           security \
                           sound \
+                          syzkaller-repros \
                           tracing \
                           uevents \
 
testcases/kernel/syzkaller-repros/.gitignore new file +1 -0
new file mode 100644
index 000000000..dbda1c71f
@@ -0,0 +1 @@
+syzwrap
testcases/kernel/syzkaller-repros/Makefile new file +4 -0
new file mode 100644
...
+# Some useful compiler flags for the LTP will cause problems with the
+# syzkaller repros so the repros have seperate flags
+SYZKALLER_CFLAGS ?= -pthread
+SYZKALLER_REPROS = $(subst

$(abs_top_srcdir),$(abs_top_builddir),$(SYZKALLER_REPROS_SRCS:.c=))

Quoted 8 lines

testcases/kernel/syzkaller-repros/Makefile line 37

Quoted 4 lines

+$(SYZKALLER_REPROS): %: %.c

testcases/kernel/syzkaller-repros/Makefile new file +2 -0
+       -@if grep -q "__NR_mmap2" $^; then \
+               M32="-m32"; \

I got compiling errors on s390x:
gcc: error: unrecognized command line option ?-m32?; did you mean
?-m31??

I have only tried these on x86_64 so far and I think that is all we can support to begin with.

Quoted 4 lines

My other concern is syzkaller (I guess maybe) have some package dependencies, and that will break the compiler phase on the embedded system.

This is true, the reproducers do have dependencies and it seems to vary (randomly view some of the C files). However this is one of the reasons why they are only installed if --with-syzkaller-repros is set.

Quoted 18 lines
Quoted 18 lines

....

testcases/kernel/syzkaller-repros/Makefile new file +15 -0
+       while (!waitpid(pid, &status, WNOHANG)) {
+               rem = tst_timeout_remaining();
+
+               if (!sent_kill && rem / exec_time_start < 0.5) {
+                       tst_res(TINFO, "Timeout; killing reproducer");
+
+                       TEST(kill(pid, SIGKILL));
+                       if (TST_RET == -1)
+                               tst_res(TWARN | TTERRNO, "kill() failed");
+                       else
+                               sent_kill = 1;
+               }
+
+               usleep(backoff);
+               backoff = MIN(2 * backoff, 1000000);
Quoted 8 lines
Quoted 2 lines

+ }

Force to kill a timeout test process is fine, but one thing makes me worried is that we don't do any cleanup work(e.g. release hugepage, devices or other resources) for the children, that very probably causes many additional issues in the next testing and not easy to reproduce it in a new run.

The reproducer itself should clear up some of this. It is also possible to regenerate the reproducers with more sandboxing AFAICT. However the sandboxing itself has some requirements.

I don't know how we could possibly cleanup after every reproducer, except by taking a complete snapshot of the system and reverting to it. Which is what I recommend doing in the README.

This is another reason for the --with-syzkaller-repros flag. The test runner must do extra work compared to normal LTP tests. I'm also not sure it would be safe to run these on baremetal unless you regenerate them with the extra sandboxing.

Quoted 10 lines

testcases/kernel/syzkaller-repros/Makefile

Quoted 8 lines
testcases/kernel/syzkaller-repros/Makefile new file +6 -0
+
+       if (tst_taint_check()) {
+               tst_res(TFAIL, "Kernel is tainted");
+       } else {
+               tst_res(TPASS, "Kernel is not tainted");
+       }
Quoted 3 lines

If this is the only condition to judge if all tests pass or not, we may miss some test failure logs after running, unless we don't care about that.

They produce a lot of output, some of it might be useful for returning TCONF (like when sandboxing is enabled in syzwrap and this stops network devices from being created). However I think it is more important to provide some basic functionality than work on stuff like this.

Quoted 2 lines

Btw, I can't even finish one round for the test then system panic there.

Same. There are a lot of reproducers for unsolved bugs, plus some of them probably shouldn't be run as root without the extra sandboxing. So they will crash the system just by randomly deleting or overwriting stuff.

--
Thank you,
Richard.

#10 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

Hi,

Quoted 10 lines
Quoted 7 lines

configure.ac else · line 190

Quoted 4 lines
configure.ac +4 -0
+AC_ARG_WITH([syzkaller-repros],
+  [AC_HELP_STRING([--with-syzkaller-repros],
+    [compile and install Syzkaller reproducers (default=no)])],
+  [with_syzkaller_repros=$withval]

To strictly, the [action-if-not-given] should be added too?

Were the other 'with' options updated recently to have that? I just copied this from the other options.

Yep, you'll just add
[with_syzkaller_repros=no]
(looking at the default off, probably safer for the start)

Quoted 26 lines
Quoted 12 lines
Quoted 11 lines
testcases/kernel/Makefile +1 -0
index 3319b3163..0150cfb4f 100644
@@ -53,6 +53,7 @@ SUBDIRS                       += connectors \
                           sched \
                           security \
                           sound \
+                          syzkaller-repros \
                           tracing \
                           uevents \
Quoted 13 lines

testcases/kernel/Makefile

Quoted 8 lines
testcases/kernel/Makefile +4 -0
+# Some useful compiler flags for the LTP will cause problems with the
+# syzkaller repros so the repros have seperate flags
+SYZKALLER_CFLAGS ?= -pthread
+SYZKALLER_REPROS = $(subst

$(abs_top_srcdir),$(abs_top_builddir),$(SYZKALLER_REPROS_SRCS:.c=)) +$(SYZKALLER_REPROS): %: %.c

testcases/kernel/Makefile +2 -0
+       -@if grep -q "__NR_mmap2" $^; then \
+               M32="-m32"; \

I got compiling errors on s390x:
gcc: error: unrecognized command line option ?-m32?; did you mean
?-m31??

Quoted 2 lines

I have only tried these on x86_64 so far and I think that is all we can support to begin with.

Maybe filter it out in testcases/kernel/Makefile with something like this:
ifneq (,$(filter $(HOST_CPU),x86 x86_64))
FILTER_OUT_DIRS += syzkaller-repros
endif

Quoted 7 lines
Quoted 3 lines

My other concern is syzkaller (I guess maybe) have some package dependencies, and that will break the compiler phase on the embedded system.

This is true, the reproducers do have dependencies and it seems to vary (randomly view some of the C files). However this is one of the reasons why they are only installed if --with-syzkaller-repros is set.

Maybe later we manage to generate autotools config based on pkg-config.

Kind regards,
Petr

#11 lore raw

PATCH 1/2 Wrapper for Syzkaller reproducers

In reply to Richard Palethorpe

Hi,

...

Quoted 1 line

+++ b/testcases/kernel/syzkaller-repros/Makefile

...

testcases/kernel/syzkaller-repros/Makefile line 26

Quoted 5 lines
testcases/kernel/syzkaller-repros/Makefile new file +5 -0
+# If the reproducers directory is missing then we automatically clone the repo.
+# We then have to call make recursively to revaluate the targets
+SYZKALLER_REPROS_DIR ?= $(abs_top_srcdir)/testcases/linux-arts/syzkaller-repros/linux
+$(SYZKALLER_REPROS_DIR):
+	git submodule update --init $(abs_top_srcdir)/testcases/linux-arts

Hm, that does not get submodule code.
Reproduced locally and also on travis:
https://travis-ci.org/pevik/ltp/builds/617186693

https://api.travis-ci.org/v3/job/617186701/log.txt
fatal: Not a git repository (or any of the parent directories): .git

or

https://api.travis-ci.org/v3/job/617186700/log.txt
error: pathspec '/usr/src/ltp/testcases/linux-arts' did not match any file(s) known to git

Maybe it's due the fact I already run locally (and the same did travis):
git submodule update --init --recursive

Kind regards,
Petr

#12 lore raw

PATCH 2/2 lib: Allow timeout coefficient < 1

In reply to Richard Palethorpe

For the Syzkaller reproducers we want to allow the test runner to set the timeout to be a fraction of the default amount.

---
 lib/tst_test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/tst_test.c +1 -1
index 6239acf89..2e00fd86b 100644
@@ -1107,7 +1107,7 @@ void tst_set_timeout(int timeout)
 	if (mul) {
 		float m = atof(mul);
 
-		if (m < 1)
+		if (m <= 0)
 			tst_brk(TBROK, "Invalid timeout multiplier '%s'", mul);
 
 		results->timeout = results->timeout * m + 0.5;
Signature · 1 line
2.23.0
#13 lore raw

PATCH 2/2 lib: Allow timeout coefficient < 1

Hi,

Quoted 13 lines

For the Syzkaller reproducers we want to allow the test runner to set the timeout to be a fraction of the default amount.

---
 lib/tst_test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/tst_test.c +0 -0
index 6239acf89..2e00fd86b 100644

lib/tst_test.c void tst_set_timeout(int timeout) · line 1113

Quoted 8 lines
lib/tst_test.c +1 -1
@@ -1107,7 +1107,7 @@ void tst_set_timeout(int timeout)
 	if (mul) {
 		float m = atof(mul);
 
-		if (m < 1)
+		if (m <= 0)
 			tst_brk(TBROK, "Invalid timeout multiplier '%s'", mul);
 
 		results->timeout = results->timeout * m + 0.5;

Update for current master:

lib/tst_test.c +2 -2
index 8d7dee2cc..2513ce7ba 100644
@@ -1190,8 +1190,8 @@ unsigned int tst_multiply_timeout(unsigned int timeout)
 			timeout_mul = 1;
 		}
 	}
-	if (timeout_mul < 1)
-		tst_brk(TBROK, "LTP_TIMEOUT_MUL must to be int >= 1! (%.2f)",
+	if (timeout_mul <= 0)
+		tst_brk(TBROK, "LTP_TIMEOUT_MUL must to be int >= 0! (%.2f)",
 				timeout_mul);
 
 	if (timeout < 1)

Kind regards,
Petr

#14 lore raw

PATCH 2/2 lib: Allow timeout coefficient < 1

In reply to Richard Palethorpe

Hi!

Quoted 2 lines

For the Syzkaller reproducers we want to allow the test runner to set the timeout to be a fraction of the default amount.

I guess that this is fine for prototyping but for the real world usage the syswrap.c should have a .timeout set in the tst_test structure instead.

Quoted 24 lines
---
 lib/tst_test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/tst_test.c +1 -1
index 6239acf89..2e00fd86b 100644
@@ -1107,7 +1107,7 @@ void tst_set_timeout(int timeout)
 	if (mul) {
 		float m = atof(mul);
 
-		if (m < 1)
+		if (m <= 0)
 			tst_brk(TBROK, "Invalid timeout multiplier '%s'", mul);
 
 		results->timeout = results->timeout * m + 0.5;
Signature · 5 lines
2.23.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 2 lines
Cyril Hrubis
[email protected]
#15 lore raw

cover LTP wrapper for Syzkaller reproducers

In reply to Richard Palethorpe
Quoted 50 lines · Richard Palethorpe

Hello,

This allows the user to run ./configure --with-syzkaller-repros and it will automatically fetch the reproducers and generate the necessary LTP runtest files which use 'syzwrap' to wrap the Syzkaller reproducers.

Optimistically speaking, this should allow you to run the reproducers in your normal LTP test setup if you reduce the timeouts and your test runner is reasonably advanced (see README.md).

I have been testing this with a modified version of Metan's runltp-ng which almost runs the reproducers smoothly ;-). We haven't got around to documenting it yet, but you can find it here:

https://github.com/richiejp/runltp-ng/tree/syzkaller

I began adding some sandboxing to syzwrap from the Syzkaller executor, but I found this simply broke many of the reproducers which need access to TAP/TUN. I guess the reproducers can be regenerated with the sanboxing built in if the user cares enough. However I have left in the ability for syzwrap to optionally add some sandboxing. This does appear to help prevent the root file system from being trashed at the cost of breaking some reproducers.

Of course if the test runner can reset the SUT to a good state, then the sandboxing is not necessary. This is quite easy to do with QEMU using image overlays.

Richard Palethorpe (2):
Wrapper for Syzkaller reproducers
lib: Allow timeout coefficient < 1

 .gitmodules                                  |   5 +
 configure.ac                                 |  11 ++
 include/mk/features.mk.default               |   2 +
 include/mk/features.mk.in                    |   2 +
 lib/tst_test.c                               |   2 +-
 runtest/.gitignore                           |   1 +
 testcases/kernel/Makefile                    |   1 +
 testcases/kernel/syzkaller-repros/.gitignore |   1 +
 testcases/kernel/syzkaller-repros/Makefile   | 100 ++++++++++++++
 testcases/kernel/syzkaller-repros/README.md  |  45 +++++++
 testcases/kernel/syzkaller-repros/syzwrap.c  | 133 +++++++++++++++++++
 11 files changed, 302 insertions(+), 1 deletion(-)
 create mode 100644 runtest/.gitignore
 create mode 100644 testcases/kernel/syzkaller-repros/.gitignore
 create mode 100644 testcases/kernel/syzkaller-repros/Makefile
 create mode 100644 testcases/kernel/syzkaller-repros/README.md
 create mode 100644 testcases/kernel/syzkaller-repros/syzwrap.c
Quoted 3 lines

--
2.23.0

+syzkaller mailing list FTR