Skip to contentSkip to conversation

tests/fuzz: Review notes

PATCH Applied Philippe Mathieu-Daudé 3 participants 10 messages May 14, 2020 – May 15, 2020 Kbuild

lore
Share & tools
lore
#0 lore raw

cover tests/fuzz: Review notes

I took some code notes while testing Alex's toy.
Sending them as patches.

Alex, I had theses issues while building following docs/devel/fuzzing.txt:

CC nbd/server.o
nbd/server.c:1937:1: error: unused function 'glib_listautoptr_cleanup_NBDExtentArray' [-Werror,-Wunused-function] G_DEFINE_AUTOPTR_CLEANUP_FUNC(NBDExtentArray, nbd_extent_array_free);
^
/usr/include/glib-2.0/glib/gmacros.h:462:22: note: expanded from macro 'G_DEFINE_AUTOPTR_CLEANUP_FUNC'

    static inline void _GLIB_AUTOPTR_LIST_FUNC_NAME(TypeName) (GList **_l) { g_list_free_full (*_l, (GDestroyNotify) func); } \
                       ^
  /usr/include/glib-2.0/glib/gmacros.h:443:48: note: expanded from macro '_GLIB_AUTOPTR_LIST_FUNC_NAME'
  #define _GLIB_AUTOPTR_LIST_FUNC_NAME(TypeName) glib_listautoptr_cleanup_##TypeName
                                                 ^

Solved by using './configure ... --extra-cflags=-Wno-unused-function'

    LINK    i386-softmmu/qemu-fuzz-i386

/usr/lib64/clang/9.0.1/lib/linux/libclang_rt.asan-aarch64.a(asan_interceptors_vfork.S.o): warning: common of `__interception::real_vfork' overridden by definition /usr/lib64/clang/9.0.1/lib/linux/libclang_rt.asan-aarch64.a(asan_interceptors.cc.o): warning: defined here

I simply ignore this warning.

Alex, could you provide few more examples (in particular about instantiating devices, or useful libfuzzer arguments)?

Also, I suppose you are using a script to generate the bug reports:

I can reproduce it in qemu 5.0 using:

cat << EOF | qemu-system ...
outl 0xcf8 ...
outl 0xcfc ...
...
EOF

Is this script available in the QEMU repository? I couldn't find it.

Regards,

Phil.

Philippe Mathieu-Daudé (6):
tests/fuzz/Makefile: Do not link code using unavailable devices
Makefile: List fuzz targets in 'make help' tests/fuzz: Add missing space in test description tests/fuzz: Remove unuseful/unused typedefs tests/fuzz: Extract pciconfig_fuzz_qos() method tests/fuzz: Extract ioport_fuzz_qtest() method

 Makefile                          |  6 +++-
 tests/qtest/fuzz/i440fx_fuzz.c    | 47 ++++++++++++++++++++-----------
 tests/qtest/fuzz/Makefile.include |  6 ++--
 3 files changed, 38 insertions(+), 21 deletions(-)
Signature · 1 line
2.21.3
#1 lore raw

PATCH 1/6 tests/fuzz/Makefile: Do not link code using unavailable devices

Some devices availability depends on CONFIG options. Use these options to only link tests when requested device is available.

---
 tests/qtest/fuzz/Makefile.include | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qtest/fuzz/Makefile.include +3 -3
index cde3e9636c..f259d866c9 100644
@@ -7,9 +7,9 @@ fuzz-obj-y += tests/qtest/fuzz/fork_fuzz.o
 fuzz-obj-y += tests/qtest/fuzz/qos_fuzz.o
 
 # Targets
-fuzz-obj-y += tests/qtest/fuzz/i440fx_fuzz.o
-fuzz-obj-y += tests/qtest/fuzz/virtio_net_fuzz.o
-fuzz-obj-y += tests/qtest/fuzz/virtio_scsi_fuzz.o
+fuzz-obj-$(CONFIG_PCI_I440FX) += tests/qtest/fuzz/i440fx_fuzz.o
+fuzz-obj-$(CONFIG_VIRTIO_NET) += tests/qtest/fuzz/virtio_net_fuzz.o
+fuzz-obj-$(CONFIG_SCSI) += tests/qtest/fuzz/virtio_scsi_fuzz.o
 
 FUZZ_CFLAGS += -I$(SRC_PATH)/tests -I$(SRC_PATH)/tests/qtest
 
Signature · 1 line
2.21.3
#2 lore raw

PATCH 2/6 Makefile: List fuzz targets in 'make help'

In reply to Philippe Mathieu-Daudé

List softmmu fuzz targets in 'make help' output:

  $ make help
  ...
  Architecture specific targets:
  aarch64-softmmu/all            - Build for aarch64-softmmu
  aarch64-softmmu/fuzz           - Build fuzzer for aarch64-softmmu
  alpha-softmmu/all              - Build for alpha-softmmu
  alpha-softmmu/fuzz             - Build fuzzer for alpha-softmmu
  arm-softmmu/all                - Build for arm-softmmu
  arm-softmmu/fuzz               - Build fuzzer for arm-softmmu
  ...
---
 Makefile | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Makefile +5 -1
index 34275f57c9..40e4f7677b 100644
@@ -1252,7 +1252,11 @@ endif
 	@$(if $(TARGET_DIRS), \
 		echo 'Architecture specific targets:'; \
 		$(foreach t, $(TARGET_DIRS), \
-		$(call print-help-run,$(t)/all,Build for $(t));) \
+		$(call print-help-run,$(t)/all,Build for $(t)); \
+		$(if $(CONFIG_FUZZ), \
+			$(if $(findstring softmmu,$(t)), \
+				$(call print-help-run,$(t)/fuzz,Build fuzzer for $(t)); \
+		))) \
 		echo '')
 	@$(if $(TOOLS), \
 		echo 'Tools targets:'; \
Signature · 1 line
2.21.3
#3 lore raw

PATCH 3/6 tests/fuzz: Add missing space in test description

In reply to Philippe Mathieu-Daudé

---
Note: The descriptions are not very accurate (copy/paste?)

---
 tests/qtest/fuzz/i440fx_fuzz.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qtest/fuzz/i440fx_fuzz.c +3 -3
index ab5f112584..96fed9ff12 100644
@@ -159,7 +159,7 @@ static void register_pci_fuzz_targets(void)
     /* Uses simple qtest commands and reboots to reset state */
     fuzz_add_target(&(FuzzTarget){
                 .name = "i440fx-qtest-reboot-fuzz",
-                .description = "Fuzz the i440fx using raw qtest commands and"
+                .description = "Fuzz the i440fx using raw qtest commands and "
                                "rebooting after each run",
                 .get_init_cmdline = i440fx_argv,
                 .fuzz = i440fx_fuzz_qtest});
@@ -167,7 +167,7 @@ static void register_pci_fuzz_targets(void)
     /* Uses libqos and forks to prevent state leakage */
     fuzz_add_qos_target(&(FuzzTarget){
                 .name = "i440fx-qos-fork-fuzz",
-                .description = "Fuzz the i440fx using raw qtest commands and"
+                .description = "Fuzz the i440fx using raw qtest commands and "
                                "rebooting after each run",
                 .pre_vm_init = &fork_init,
                 .fuzz = i440fx_fuzz_qos_fork,},
@@ -182,7 +182,7 @@ static void register_pci_fuzz_targets(void)
      */
     fuzz_add_qos_target(&(FuzzTarget){
                 .name = "i440fx-qos-noreset-fuzz",
-                .description = "Fuzz the i440fx using raw qtest commands and"
+                .description = "Fuzz the i440fx using raw qtest commands and "
                                "rebooting after each run",
                 .fuzz = i440fx_fuzz_qos,},
                 "i440FX-pcihost",
Signature · 1 line
2.21.3
#4 lore raw

PATCH 4/6 tests/fuzz: Remove unuseful/unused typedefs

In reply to Philippe Mathieu-Daudé

These typedefs are not used. Use a simple structure, remote the typedefs.

---
 tests/qtest/fuzz/i440fx_fuzz.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
tests/qtest/fuzz/i440fx_fuzz.c +4 -6
index 96fed9ff12..c197b026db 100644
@@ -45,12 +45,11 @@ static void i440fx_fuzz_qtest(QTestState *s,
      * loop over the Data, breaking it up into actions. each action has an
      * opcode, address offset and value
      */
-    typedef struct QTestFuzzAction {
+    struct {
         uint8_t opcode;
         uint8_t addr;
         uint32_t value;
-    } QTestFuzzAction;
-    QTestFuzzAction a;
+    } a;
 
     while (Size >= sizeof(a)) {
         /* make a copy of the action so we can normalize the values in-place */
@@ -91,19 +90,18 @@ static void i440fx_fuzz_qos(QTestState *s,
      * Same as i440fx_fuzz_qtest, but using QOS. devfn is incorporated into the
      * value written over Port IO
      */
-    typedef struct QOSFuzzAction {
+    struct {
         uint8_t opcode;
         uint8_t offset;
         int devfn;
         uint32_t value;
-    } QOSFuzzAction;
+    } a;
 
     static QPCIBus *bus;
     if (!bus) {
         bus = qpci_new_pc(s, fuzz_qos_alloc);
     }
 
-    QOSFuzzAction a;
     while (Size >= sizeof(a)) {
         memcpy(&a, Data, sizeof(a));
         switch (a.opcode % ACTION_MAX) {
Signature · 1 line
2.21.3
#5 lore raw

PATCH 5/6 tests/fuzz: Extract pciconfig_fuzz_qos() method

In reply to Philippe Mathieu-Daudé

Extract the generic pciconfig_fuzz_qos() method from i440fx_fuzz_qos(). This will help to write tests not specific to the i440FX controller.

---
 tests/qtest/fuzz/i440fx_fuzz.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
tests/qtest/fuzz/i440fx_fuzz.c +14 -6
index c197b026db..558fa17c93 100644
@@ -84,7 +84,7 @@ static void i440fx_fuzz_qtest(QTestState *s,
     flush_events(s);
 }
 
-static void i440fx_fuzz_qos(QTestState *s,
+static void pciconfig_fuzz_qos(QTestState *s, QPCIBus *bus,
         const unsigned char *Data, size_t Size) {
     /*
      * Same as i440fx_fuzz_qtest, but using QOS. devfn is incorporated into the
@@ -97,11 +97,6 @@ static void i440fx_fuzz_qos(QTestState *s,
         uint32_t value;
     } a;
 
-    static QPCIBus *bus;
-    if (!bus) {
-        bus = qpci_new_pc(s, fuzz_qos_alloc);
-    }
-
     while (Size >= sizeof(a)) {
         memcpy(&a, Data, sizeof(a));
         switch (a.opcode % ACTION_MAX) {
@@ -130,6 +125,19 @@ static void i440fx_fuzz_qos(QTestState *s,
     flush_events(s);
 }
 
+static void i440fx_fuzz_qos(QTestState *s,
+                            const unsigned char *Data,
+                            size_t Size)
+{
+    static QPCIBus *bus;
+
+    if (!bus) {
+        bus = qpci_new_pc(s, fuzz_qos_alloc);
+    }
+
+    pciconfig_fuzz_qos(s, bus, Data, Size);
+}
+
 static void i440fx_fuzz_qos_fork(QTestState *s,
         const unsigned char *Data, size_t Size) {
     if (fork() == 0) {
Signature · 1 line
2.21.3
#6 lore raw

PATCH 6/6 tests/fuzz: Extract ioport_fuzz_qtest() method

In reply to Philippe Mathieu-Daudé

Extract generic ioport_fuzz_qtest() method from i440fx_fuzz_qtest(). This will help to write tests not specific to the i440FX controller.

---
 tests/qtest/fuzz/i440fx_fuzz.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
tests/qtest/fuzz/i440fx_fuzz.c +9 -2
index 558fa17c93..bcd6769b4c 100644
@@ -39,7 +39,7 @@ enum action_id {
     ACTION_MAX
 };
 
-static void i440fx_fuzz_qtest(QTestState *s,
+static void ioport_fuzz_qtest(QTestState *s,
         const unsigned char *Data, size_t Size) {
     /*
      * loop over the Data, breaking it up into actions. each action has an
@@ -84,10 +84,17 @@ static void i440fx_fuzz_qtest(QTestState *s,
     flush_events(s);
 }
 
+static void i440fx_fuzz_qtest(QTestState *s,
+                              const unsigned char *Data,
+                              size_t Size)
+{
+    ioport_fuzz_qtest(s, Data, Size);
+}
+
 static void pciconfig_fuzz_qos(QTestState *s, QPCIBus *bus,
         const unsigned char *Data, size_t Size) {
     /*
-     * Same as i440fx_fuzz_qtest, but using QOS. devfn is incorporated into the
+     * Same as ioport_fuzz_qtest, but using QOS. devfn is incorporated into the
      * value written over Port IO
      */
     struct {
Signature · 1 line
2.21.3
#7 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 51 lines · Philippe Mathieu-Daudé

I took some code notes while testing Alex's toy.
Sending them as patches.

Alex, I had theses issues while building following docs/devel/fuzzing.txt:

CC nbd/server.o
nbd/server.c:1937:1: error: unused function 'glib_listautoptr_cleanup_NBDExtentArray' [-Werror,-Wunused-function] G_DEFINE_AUTOPTR_CLEANUP_FUNC(NBDExtentArray, nbd_extent_array_free);
^
/usr/include/glib-2.0/glib/gmacros.h:462:22: note: expanded from macro 'G_DEFINE_AUTOPTR_CLEANUP_FUNC'

    static inline void _GLIB_AUTOPTR_LIST_FUNC_NAME(TypeName) (GList **_l) { g_list_free_full (*_l, (GDestroyNotify) func); } \
                       ^
  /usr/include/glib-2.0/glib/gmacros.h:443:48: note: expanded from macro '_GLIB_AUTOPTR_LIST_FUNC_NAME'
  #define _GLIB_AUTOPTR_LIST_FUNC_NAME(TypeName) glib_listautoptr_cleanup_##TypeName
                                                 ^

Solved by using './configure ... --extra-cflags=-Wno-unused-function'

    LINK    i386-softmmu/qemu-fuzz-i386

/usr/lib64/clang/9.0.1/lib/linux/libclang_rt.asan-aarch64.a(asan_interceptors_vfork.S.o): warning: common of `__interception::real_vfork' overridden by definition /usr/lib64/clang/9.0.1/lib/linux/libclang_rt.asan-aarch64.a(asan_interceptors.cc.o): warning: defined here

I simply ignore this warning.

Alex, could you provide few more examples (in particular about instantiating devices, or useful libfuzzer arguments)?

Also, I suppose you are using a script to generate the bug reports:

I can reproduce it in qemu 5.0 using:

cat << EOF | qemu-system ...
outl 0xcf8 ...
outl 0xcfc ...
...
EOF

Is this script available in the QEMU repository? I couldn't find it.

Regards,

Phil.

Philippe Mathieu-Daudé (6):
tests/fuzz/Makefile: Do not link code using unavailable devices
Makefile: List fuzz targets in 'make help' tests/fuzz: Add missing space in test description tests/fuzz: Remove unuseful/unused typedefs tests/fuzz: Extract pciconfig_fuzz_qos() method tests/fuzz: Extract ioport_fuzz_qtest() method

Quoted 8 lines
 Makefile                          |  6 +++-
 tests/qtest/fuzz/i440fx_fuzz.c    | 47 ++++++++++++++++++++-----------
 tests/qtest/fuzz/Makefile.include |  6 ++--
 3 files changed, 38 insertions(+), 21 deletions(-)
Signature · 1 line
2.21.3

Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan

#8 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 2 lines · Philippe Mathieu-Daudé

I took some code notes while testing Alex's toy.
Sending them as patches.

Thank you for these! I'd send my review tags, if they weren't already pulled.

Quoted 8 lines

Alex, I had theses issues while building following docs/devel/fuzzing.txt:

CC nbd/server.o
nbd/server.c:1937:1: error: unused function 'glib_listautoptr_cleanup_NBDExtentArray' [-Werror,-Wunused-function] G_DEFINE_AUTOPTR_CLEANUP_FUNC(NBDExtentArray, nbd_extent_array_free);
^
/usr/include/glib-2.0/glib/gmacros.h:462:22: note: expanded from macro 'G_DEFINE_AUTOPTR_CLEANUP_FUNC'

Quoted 6 lines
    static inline void _GLIB_AUTOPTR_LIST_FUNC_NAME(TypeName) (GList **_l) { g_list_free_full (*_l, (GDestroyNotify) func); } \
                       ^
  /usr/include/glib-2.0/glib/gmacros.h:443:48: note: expanded from macro '_GLIB_AUTOPTR_LIST_FUNC_NAME'
  #define _GLIB_AUTOPTR_LIST_FUNC_NAME(TypeName) glib_listautoptr_cleanup_##TypeName
                                                 ^

Solved by using './configure ... --extra-cflags=-Wno-unused-function'

Interesting that this doesn't show up for standard builds. I'll have to look into this more.

Quoted 5 lines
    LINK    i386-softmmu/qemu-fuzz-i386

/usr/lib64/clang/9.0.1/lib/linux/libclang_rt.asan-aarch64.a(asan_interceptors_vfork.S.o): warning: common of `__interception::real_vfork' overridden by definition /usr/lib64/clang/9.0.1/lib/linux/libclang_rt.asan-aarch64.a(asan_interceptors.cc.o): warning: defined here

I simply ignore this warning.

Yes, I get the same warning with this:

cat << EOF | clang-9 -Wl,--warn-common -fsanitize=fuzzer,address -x c - #include <sys/types.h>
#include <stdint.h>
int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { return 0; } EOF

Might report this to the LLVM/compiler-rt folks.

Quoted 2 lines

Alex, could you provide few more examples (in particular about instantiating devices, or useful libfuzzer arguments)?

Yes - I'll send this out with my next fuzzing patches.

Quoted 11 lines

Also, I suppose you are using a script to generate the bug reports:

I can reproduce it in qemu 5.0 using:

cat << EOF | qemu-system ...
outl 0xcf8 ...
outl 0xcfc ...
...
EOF

Is this script available in the QEMU repository? I couldn't find it.

Right now I just added some unbuffered writes to qtest.c. I'll try to find a clean way of doing this and I'll add a way to automatically convert a crashing input to qemu arguments + qtest command sequence.

Thanks again!
-Alex

Quoted 20 lines

Regards,

Phil.

Philippe Mathieu-Daudé (6):
tests/fuzz/Makefile: Do not link code using unavailable devices
Makefile: List fuzz targets in 'make help' tests/fuzz: Add missing space in test description tests/fuzz: Remove unuseful/unused typedefs tests/fuzz: Extract pciconfig_fuzz_qos() method tests/fuzz: Extract ioport_fuzz_qtest() method

 Makefile                          |  6 +++-
 tests/qtest/fuzz/i440fx_fuzz.c    | 47 ++++++++++++++++++++-----------
 tests/qtest/fuzz/Makefile.include |  6 ++--
 3 files changed, 38 insertions(+), 21 deletions(-)
Signature · 1 line
2.21.3
#9 lore raw
Quoted 6 lines · Alexander Bulekov
Quoted 2 lines · Philippe Mathieu-Daudé

I took some code notes while testing Alex's toy.
Sending them as patches.

Thank you for these! I'd send my review tags, if they weren't already pulled.

The series is queued but not pulled. Review tags are always welcome, and maintainers might add them after-queue before-pull when they have time.

Quoted 16 lines
Quoted 14 lines

Alex, I had theses issues while building following docs/devel/fuzzing.txt:

CC nbd/server.o
nbd/server.c:1937:1: error: unused function 'glib_listautoptr_cleanup_NBDExtentArray' [-Werror,-Wunused-function]

   G_DEFINE_AUTOPTR_CLEANUP_FUNC(NBDExtentArray, nbd_extent_array_free);
   ^
   /usr/include/glib-2.0/glib/gmacros.h:462:22: note: expanded from macro 'G_DEFINE_AUTOPTR_CLEANUP_FUNC'
     static inline void _GLIB_AUTOPTR_LIST_FUNC_NAME(TypeName) (GList **_l) { g_list_free_full (*_l, (GDestroyNotify) func); } \
                        ^
   /usr/include/glib-2.0/glib/gmacros.h:443:48: note: expanded from macro '_GLIB_AUTOPTR_LIST_FUNC_NAME'
   #define _GLIB_AUTOPTR_LIST_FUNC_NAME(TypeName) glib_listautoptr_cleanup_##TypeName
                                                  ^
Quoted 1 line

Solved by using './configure ... --extra-cflags=-Wno-unused-function'

Quoted 2 lines

Interesting that this doesn't show up for standard builds. I'll have to look into this more.

I don't think it is related to your work.

Quoted 32 lines
Quoted 6 lines
     LINK    i386-softmmu/qemu-fuzz-i386

/usr/lib64/clang/9.0.1/lib/linux/libclang_rt.asan-aarch64.a(asan_interceptors_vfork.S.o): warning: common of `__interception::real_vfork' overridden by definition /usr/lib64/clang/9.0.1/lib/linux/libclang_rt.asan-aarch64.a(asan_interceptors.cc.o): warning: defined here

I simply ignore this warning.

Yes, I get the same warning with this:

cat << EOF | clang-9 -Wl,--warn-common -fsanitize=fuzzer,address -x c - #include <sys/types.h>
#include <stdint.h>
int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { return 0; } EOF

Might report this to the LLVM/compiler-rt folks.

Quoted 2 lines

Alex, could you provide few more examples (in particular about instantiating devices, or useful libfuzzer arguments)?

Yes - I'll send this out with my next fuzzing patches.

Quoted 10 lines

Also, I suppose you are using a script to generate the bug reports:

   I can reproduce it in qemu 5.0 using:

   cat << EOF | qemu-system ...
   outl 0xcf8 ...
   outl 0xcfc ...
   ...
   EOF
Quoted 7 lines
Quoted 1 line

Is this script available in the QEMU repository? I couldn't find it.

Right now I just added some unbuffered writes to qtest.c. I'll try to find a clean way of doing this and I'll add a way to automatically convert a crashing input to qemu arguments + qtest command sequence.

Thanks again!

You are welcomed, waiting for your following series :)

Quoted 23 lines

-Alex

Quoted 20 lines

Regards,

Phil.

Philippe Mathieu-Daudé (6):
tests/fuzz/Makefile: Do not link code using unavailable devices
Makefile: List fuzz targets in 'make help' tests/fuzz: Add missing space in test description tests/fuzz: Remove unuseful/unused typedefs tests/fuzz: Extract pciconfig_fuzz_qos() method tests/fuzz: Extract ioport_fuzz_qtest() method

  Makefile                          |  6 +++-
  tests/qtest/fuzz/i440fx_fuzz.c    | 47 ++++++++++++++++++++-----------
  tests/qtest/fuzz/Makefile.include |  6 ++--
  3 files changed, 38 insertions(+), 21 deletions(-)
Signature · 1 line
2.21.3