Skip to contentSkip to conversation

modules: add meta-data database

PATCH In review Gerd Hoffmann 6 participants 46 messages Jun 24, 2021 – Aug 28, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v4 cover modules: add meta-data database

This patch series adds support for module meta-data. Today this is either hard-coded in qemu (see qemu_load_module_for_opts) or handled with manually maintained lists in util/module (see module_deps[] and qom_modules[]). This series replaced that scheme with annotation macros, so the meta-data can go into the module source code and -- for example -- the module_obj() annotations can go next to the TypeInfo struct for the object class.

Patches 1-3 put the infrastructure in place: Add the annotation macros, add a script to collect the meta-data, add a script to compile the meta-data into C source code which we can then add to qemu.

Patch 4 - check module dependencies (Jose, new in v4).

Patches 5-13 add annotations macros to the modules we have.

Patches 14-16 put the modinfo database into use and remove the module_deps[] and qom_modules[] lists.

Patch 16 adds two tracepoints for easier trouble-shooting.

Patches 18-20 add support for target-specific modules.

Patches 21-24 add documentation for all of the above (new in v4, was separate series).

Patches 25-29 start building accelerators modular. So far it is only qtest (all archs) and a small fraction of tcg (x86 only).

Patches 30-34 add support for registering hmp commands so they can be implemented as module (new in v4, was separate series).

take care,
Gerd

Gerd Hoffmann (33):
modules: add modinfo macros
modules: collect module meta-data
modules: generate modinfo.c
modules: add qxl module annotations
modules: add virtio-gpu module annotations
modules: add chardev module annotations
modules: add audio module annotations
modules: add usb-redir module annotations
modules: add ccid module annotations
modules: add ui module annotations
modules: add s390x module annotations
modules: add block module annotations
modules: use modinfo for dependencies
modules: use modinfo for qom load
modules: use modinfo for qemu opts load
modules: add tracepoints
modules: check arch and block load on mismatch
modules: check arch on qom lookup
modules: target-specific module build infrastructure
modules: add documentation for module sourcesets
modules: add module_obj() note to QOM docs
modules: module.h kerneldoc annotations
modules: hook up modules.h to docs build
accel: autoload modules
accel: add qtest module annotations
accel: build qtest modular
accel: add tcg module annotations
accel: build tcg modular
monitor: allow register hmp commands
usb: drop usb_host_dev_is_scsi_storage hook monitor/usb: register 'info usbhost' dynamically
usb: build usb-host as module
monitor/tcg: move tcg hmp commands to accel/tcg, register them
dynamically

Jose R. Ziviani (1):
modules: check if all dependencies can be satisfied

 scripts/modinfo-collect.py      |  67 +++++++++++
 scripts/modinfo-generate.py     |  97 ++++++++++++++++
 include/hw/usb.h                |   7 +-
 include/monitor/monitor.h       |   3 +
 include/qemu/module.h           |  74 ++++++++++++
 accel/accel-common.c            |   2 +-
 accel/accel-softmmu.c           |   2 +-
 accel/qtest/qtest.c             |   2 +
 accel/tcg/hmp.c                 |  29 +++++
 accel/tcg/tcg-accel-ops.c       |   1 +
 accel/tcg/tcg-all.c             |   1 +
 audio/spiceaudio.c              |   2 +
 block/iscsi-opts.c              |   1 +
 chardev/baum.c                  |   1 +
 chardev/spice.c                 |   4 +
 hw/display/qxl.c                |   4 +
 hw/display/vhost-user-gpu-pci.c |   1 +
 hw/display/vhost-user-gpu.c     |   1 +
 hw/display/vhost-user-vga.c     |   1 +
 hw/display/virtio-gpu-base.c    |   1 +
 hw/display/virtio-gpu-gl.c      |   3 +
 hw/display/virtio-gpu-pci-gl.c  |   3 +
 hw/display/virtio-gpu-pci.c     |   2 +
 hw/display/virtio-gpu.c         |   1 +
 hw/display/virtio-vga-gl.c      |   3 +
 hw/display/virtio-vga.c         |   2 +
 hw/ppc/spapr.c                  |   2 +-
 hw/s390x/virtio-ccw-gpu.c       |   3 +
 hw/usb/ccid-card-emulated.c     |   1 +
 hw/usb/ccid-card-passthru.c     |   1 +
 hw/usb/dev-storage-bot.c        |   1 +
 hw/usb/dev-storage-classic.c    |   1 +
 hw/usb/dev-uas.c                |   1 +
 hw/usb/host-libusb.c            |  38 ++----
 hw/usb/host-stub.c              |  45 -------
 hw/usb/redirect.c               |   1 +
 monitor/hmp.c                   |   7 ++
 monitor/misc.c                  |  34 +++---
 softmmu/vl.c                    |  24 ++--
 stubs/module-opts.c             |   4 -
 ui/egl-headless.c               |   4 +
 ui/gtk.c                        |   4 +
 ui/sdl2.c                       |   4 +
 ui/spice-app.c                  |   3 +
 ui/spice-core.c                 |   5 +
 util/module.c                   | 200 ++++++++++++++++++--------------
 accel/qtest/meson.build         |   8 +-
 accel/tcg/meson.build           |   6 +-
 docs/devel/build-system.rst     |  17 +++
 docs/devel/index.rst            |   1 +
 docs/devel/modules.rst          |   5 +
 docs/devel/qom.rst              |   8 ++
 hmp-commands-info.hx            |   3 -
 hw/usb/meson.build              |  10 +-
 meson.build                     |  82 +++++++++++++
 util/trace-events               |   4 +
 56 files changed, 624 insertions(+), 218 deletions(-)
 create mode 100755 scripts/modinfo-collect.py
 create mode 100755 scripts/modinfo-generate.py
 create mode 100644 accel/tcg/hmp.c
 delete mode 100644 hw/usb/host-stub.c
 create mode 100644 docs/devel/modules.rst
Signature · 1 line
2.31.1
#1 lore raw

PATCH v4 1/34 modules: add modinfo macros

Add macros for module info annotations.

Instead of having that module meta-data stored in lists in util/module.c place directly in the module source code.

---
 include/qemu/module.h | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
include/qemu/module.h +25 -0
index 944d403cbd15..81ef086da023 100644
@@ -73,4 +73,29 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail);
 void module_load_qom_one(const char *type);
 void module_load_qom_all(void);
 
+/*
+ * module info annotation macros
+ *
+ * scripts/modinfo-collect.py will collect module info,
+ * using the preprocessor and -DQEMU_MODINFO
+ */
+#ifdef QEMU_MODINFO
+# define modinfo(kind, value) \
+    MODINFO_START kind value MODINFO_END
+#else
+# define modinfo(kind, value)
+#endif
+
+/* module implements QOM type <name> */
+#define module_obj(name) modinfo(obj, name)
+
+/* module has a dependency on <name> */
+#define module_dep(name) modinfo(dep, name)
+
+/* module is for target architecture <name> */
+#define module_arch(name) modinfo(arch, name)
+
+/* module registers QemuOpts <name> */
+#define module_opts(name) modinfo(opts, name)
+
 #endif
Signature · 1 line
2.31.1
#2 lore raw

Re: [PATCH v4 01/34] modules: add modinfo macros

Quoted 5 lines · Gerd Hoffmann

Add macros for module info annotations.

Instead of having that module meta-data stored in lists in util/module.c place directly in the module source code.

[...]

include/qemu/module.h bool module_load_one(const char *prefix, const char *lib_name, bool mayfail); · line 90

Quoted 2 lines
include/qemu/module.h +2 -0
+/* module implements QOM type <name> */
+#define module_obj(name) modinfo(obj, name)

Can we make OBJECT_DEFINE_TYPE*() use this macro automatically?

Signature · 1 line
Eduardo
#3 lore raw

Re: [PATCH v4 01/34] modules: add modinfo macros

Quoted 12 lines · Eduardo Habkost
Quoted 5 lines · Gerd Hoffmann

Add macros for module info annotations.

Instead of having that module meta-data stored in lists in util/module.c place directly in the module source code.

[...]

include/qemu/module.h bool module_load_one(const char *prefix, const char *lib_name, bool mayfail); · line 90

Quoted 2 lines
include/qemu/module.h +2 -0
+/* module implements QOM type <name> */
+#define module_obj(name) modinfo(obj, name)

Can we make OBJECT_DEFINE_TYPE*() use this macro automatically?

Yeah, that's possible. I would do it as a separate patch though, because Gerd is on vacation and he asked me to include it in a pull request before soft freeze.

Thanks,

Paolo

#4 lore raw

PATCH v4 2/34 modules: collect module meta-data

In reply to Gerd Hoffmann

Add script to collect the module meta-data from the source code, store the results in *.modinfo files.

---
 scripts/modinfo-collect.py | 67 ++++++++++++++++++++++++++++++++++++++
 meson.build                | 16 +++++++++
 2 files changed, 83 insertions(+)
 create mode 100755 scripts/modinfo-collect.py
scripts/modinfo-collect.py new file +67 -0
new file mode 100755
index 000000000000..4acb188c3e89
@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+import os
+import sys
+import json
+import shlex
+import subprocess
+
+def find_command(src, target, compile_commands):
+    for command in compile_commands:
+        if command['file'] != src:
+            continue
+        if target != '' and command['command'].find(target) == -1:
+            continue
+        return command['command']
+    return 'false'
+
+def process_command(src, command):
+    skip = False
+    arg = False
+    out = []
+    for item in shlex.split(command):
+        if arg:
+            out.append(x)
+            arg = False
+            continue
+        if skip:
+            skip = False
+            continue
+        if item == '-MF' or item == '-MQ' or item == '-o':
+            skip = True
+            continue
+        if item == '-c':
+            skip = True
+            continue
+        out.append(item)
+    out.append('-DQEMU_MODINFO')
+    out.append('-E')
+    out.append(src)
+    return out
+
+def main(args):
+    target = ''
+    if args[0] == '--target':
+        args.pop(0)
+        target = args.pop(0)
+        print("MODINFO_DEBUG target %s" % target)
+        arch = target[:-8] # cut '-softmmu'
+        print("MODINFO_START arch \"%s\" MODINFO_END" % arch)
+    with open('compile_commands.json') as f:
+        compile_commands = json.load(f)
+    for src in args:
+        print("MODINFO_DEBUG src %s" % src)
+        command = find_command(src, target, compile_commands)
+        cmdline = process_command(src, command)
+        print("MODINFO_DEBUG cmd", cmdline)
+        result = subprocess.run(cmdline, stdout = subprocess.PIPE,
+                                universal_newlines = True)
+        if result.returncode != 0:
+            sys.exit(result.returncode)
+        for line in result.stdout.split('\n'):
+            if line.find('MODINFO') != -1:
+                print(line)
+
+if __name__ == "__main__":
+    main(sys.argv[1:])
meson.build +16 -0
index d8a92666fbcd..30ca06c6c2d4 100644
@@ -2021,6 +2021,9 @@ subdir('tests/qtest/fuzz')
 # Library dependencies #
 ########################
 
+modinfo_collect = find_program('scripts/modinfo-collect.py')
+modinfo_files = []
+
 block_mods = []
 softmmu_mods = []
 foreach d, list : modules
@@ -2034,6 +2037,19 @@ foreach d, list : modules
       else
         softmmu_mods += sl
       endif
+      if module_ss.sources() != []
+        # FIXME: Should use sl.extract_all_objects(recursive: true) as
+        # input. Sources can be used multiple times but objects are
+        # unique when it comes to lookup in compile_commands.json.
+        # Depnds on a mesion version with
+        # https://github.com/mesonbuild/meson/pull/8900
+        modinfo_files += custom_target(d + '-' + m + '.modinfo',
+                                       output: d + '-' + m + '.modinfo',
+                                       input: module_ss.sources(),
+                                       capture: true,
+                                       build_by_default: true, # to be removed when added to a target
+                                       command: [modinfo_collect, '@INPUT@'])
+      endif
     else
       if d == 'block'
         block_ss.add_all(module_ss)
Signature · 1 line
2.31.1
#5 lore raw

PATCH v4 3/34 modules: generate modinfo.c

In reply to Gerd Hoffmann

Add script to generate C source with a small
database containing the module meta-data.

---
 scripts/modinfo-generate.py | 84 +++++++++++++++++++++++++++++++++++++
 include/qemu/module.h       | 17 ++++++++
 softmmu/vl.c                |  4 ++
 util/module.c               | 11 +++++
 meson.build                 | 13 +++++-
 5 files changed, 128 insertions(+), 1 deletion(-)
 create mode 100755 scripts/modinfo-generate.py
scripts/modinfo-generate.py new file +84 -0
new file mode 100755
index 000000000000..a6d98a6bc4cc
@@ -0,0 +1,84 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+import os
+import sys
+
+def print_array(name, values):
+    if len(values) == 0:
+        return
+    list = ", ".join(values)
+    print("    .%s = ((const char*[]){ %s, NULL })," % (name, list))
+
+def parse_line(line):
+    kind = ""
+    data = ""
+    get_kind = False
+    get_data = False
+    for item in line.split():
+        if item == "MODINFO_START":
+            get_kind = True
+            continue
+        if item.startswith("MODINFO_END"):
+            get_data = False
+            continue
+        if get_kind:
+            kind = item
+            get_kind = False
+            get_data = True
+            continue
+        if get_data:
+            data += " " + item
+            continue
+    return (kind, data)
+
+def generate(name, lines):
+    arch = ""
+    objs = []
+    deps = []
+    opts = []
+    for line in lines:
+        if line.find("MODINFO_START") != -1:
+            (kind, data) = parse_line(line)
+            if kind == 'obj':
+                objs.append(data)
+            elif kind == 'dep':
+                deps.append(data)
+            elif kind == 'opts':
+                opts.append(data)
+            elif kind == 'arch':
+                arch = data;
+            else:
+                print("unknown:", kind)
+                exit(1)
+
+    print("    .name = \"%s\"," % name)
+    if arch != "":
+        print("    .arch = %s," % arch)
+    print_array("objs", objs)
+    print_array("deps", deps)
+    print_array("opts", opts)
+    print("},{");
+
+def print_pre():
+    print("/* generated by scripts/modinfo-generate.py */")
+    print("#include \"qemu/osdep.h\"")
+    print("#include \"qemu/module.h\"")
+    print("const QemuModinfo qemu_modinfo[] = {{")
+
+def print_post():
+    print("    /* end of list */")
+    print("}};")
+
+def main(args):
+    print_pre()
+    for modinfo in args:
+        with open(modinfo) as f:
+            lines = f.readlines()
+        print("    /* %s */" % modinfo)
+        (basename, ext) = os.path.splitext(modinfo)
+        generate(basename, lines)
+    print_post()
+
+if __name__ == "__main__":
+    main(sys.argv[1:])
include/qemu/module.h +17 -0
index 81ef086da023..a98748d501d3 100644
@@ -98,4 +98,21 @@ void module_load_qom_all(void);
 /* module registers QemuOpts <name> */
 #define module_opts(name) modinfo(opts, name)
 
+/*
+ * module info database
+ *
+ * scripts/modinfo-generate.c will build this using the data collected
+ * by scripts/modinfo-collect.py
+ */
+typedef struct QemuModinfo QemuModinfo;
+struct QemuModinfo {
+    const char *name;
+    const char *arch;
+    const char **objs;
+    const char **deps;
+    const char **opts;
+};
+extern const QemuModinfo qemu_modinfo[];
+void module_init_info(const QemuModinfo *info);
+
 #endif
softmmu/vl.c +4 -0
index feb4d201f30f..e001505d79f7 100644
@@ -2740,6 +2740,10 @@ void qemu_init(int argc, char **argv, char **envp)
     error_init(argv[0]);
     qemu_init_exec_dir(argv[0]);
 
+#ifdef CONFIG_MODULES
+    module_init_info(qemu_modinfo);
+#endif
+
     qemu_init_subsystems();
 
     /* first pass of option parsing */
util/module.c +11 -0
index eee8ff2de136..8d3e8275b9f7 100644
@@ -110,6 +110,17 @@ void module_call_init(module_init_type type)
 }
 
 #ifdef CONFIG_MODULES
+
+static const QemuModinfo module_info_stub[] = { {
+    /* end of list */
+} };
+static const QemuModinfo *module_info = module_info_stub;
+
+void module_init_info(const QemuModinfo *info)
+{
+    module_info = info;
+}
+
 static int module_load_file(const char *fname, bool mayfail, bool export_symbols)
 {
     GModule *g_module;
meson.build +12 -1
index 30ca06c6c2d4..6ae2b9ab9443 100644
@@ -2022,6 +2022,7 @@ subdir('tests/qtest/fuzz')
 ########################
 
 modinfo_collect = find_program('scripts/modinfo-collect.py')
+modinfo_generate = find_program('scripts/modinfo-generate.py')
 modinfo_files = []
 
 block_mods = []
@@ -2047,7 +2048,6 @@ foreach d, list : modules
                                        output: d + '-' + m + '.modinfo',
                                        input: module_ss.sources(),
                                        capture: true,
-                                       build_by_default: true, # to be removed when added to a target
                                        command: [modinfo_collect, '@INPUT@'])
       endif
     else
@@ -2060,6 +2060,17 @@ foreach d, list : modules
   endforeach
 endforeach
 
+if enable_modules
+  modinfo_src = custom_target('modinfo.c',
+                              output: 'modinfo.c',
+                              input: modinfo_files,
+                              command: [modinfo_generate, '@INPUT@'],
+                              capture: true)
+  modinfo_lib = static_library('modinfo', modinfo_src)
+  modinfo_dep = declare_dependency(link_whole: modinfo_lib)
+  softmmu_ss.add(modinfo_dep)
+endif
+
 nm = find_program('nm')
 undefsym = find_program('scripts/undefsym.py')
 block_syms = custom_target('block.syms', output: 'block.syms',
Signature · 1 line
2.31.1
#6 lore raw

PATCH v4 4/34 modules: check if all dependencies can be satisfied

In reply to Gerd Hoffmann

From: "Jose R. Ziviani" <[email protected]>

Verifies if all dependencies are correctly listed in the modinfo.c too and stop the builds if they're not.

---
 scripts/modinfo-generate.py | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
scripts/modinfo-generate.py +14 -1
index a6d98a6bc4cc..f559eed0077a 100755
@@ -59,6 +59,7 @@ def generate(name, lines):
     print_array("deps", deps)
     print_array("opts", opts)
     print("},{");
+    return deps
 
 def print_pre():
     print("/* generated by scripts/modinfo-generate.py */")
@@ -71,14 +72,26 @@ def print_post():
     print("}};")
 
 def main(args):
+    deps = {}
     print_pre()
     for modinfo in args:
         with open(modinfo) as f:
             lines = f.readlines()
         print("    /* %s */" % modinfo)
         (basename, ext) = os.path.splitext(modinfo)
-        generate(basename, lines)
+        deps[basename] = generate(basename, lines)
     print_post()
 
+    flattened_deps = {flat.strip('" ') for dep in deps.values() for flat in dep}
+    error = False
+    for dep in flattened_deps:
+        if dep not in deps.keys():
+            print("Dependency {} cannot be satisfied".format(dep),
+                  file=sys.stderr)
+            error = True
+
+    if error:
+        exit(1)
+
 if __name__ == "__main__":
     main(sys.argv[1:])
Signature · 1 line
2.31.1
#7 lore raw

PATCH v4 5/34 modules: add qxl module annotations

In reply to Gerd Hoffmann
---
 hw/display/qxl.c | 4 ++++
 1 file changed, 4 insertions(+)
hw/display/qxl.c +4 -0
index 6e1f8ff1b2a7..84f99088e0a0 100644
@@ -2522,6 +2522,7 @@ static const TypeInfo qxl_primary_info = {
     .parent        = TYPE_PCI_QXL,
     .class_init    = qxl_primary_class_init,
 };
+module_obj("qxl-vga");
 
 static void qxl_secondary_class_init(ObjectClass *klass, void *data)
 {
@@ -2538,6 +2539,7 @@ static const TypeInfo qxl_secondary_info = {
     .parent        = TYPE_PCI_QXL,
     .class_init    = qxl_secondary_class_init,
 };
+module_obj("qxl");
 
 static void qxl_register_types(void)
 {
@@ -2547,3 +2549,5 @@ static void qxl_register_types(void)
 }
 
 type_init(qxl_register_types)
+
+module_dep("ui-spice-core");
Signature · 1 line
2.31.1
#8 lore raw

PATCH v4 6/34 modules: add virtio-gpu module annotations

In reply to Gerd Hoffmann
---
 hw/display/vhost-user-gpu-pci.c | 1 +
 hw/display/vhost-user-gpu.c     | 1 +
 hw/display/vhost-user-vga.c     | 1 +
 hw/display/virtio-gpu-base.c    | 1 +
 hw/display/virtio-gpu-gl.c      | 3 +++
 hw/display/virtio-gpu-pci-gl.c  | 3 +++
 hw/display/virtio-gpu-pci.c     | 2 ++
 hw/display/virtio-gpu.c         | 1 +
 hw/display/virtio-vga-gl.c      | 3 +++
 hw/display/virtio-vga.c         | 2 ++
 10 files changed, 18 insertions(+)
hw/display/vhost-user-gpu-pci.c +1 -0
index a02b23ecaf11..daefcf710159 100644
@@ -43,6 +43,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_gpu_pci_info = {
     .instance_size = sizeof(VhostUserGPUPCI),
     .instance_init = vhost_user_gpu_pci_initfn,
 };
+module_obj(TYPE_VHOST_USER_GPU_PCI);
 
 static void vhost_user_gpu_pci_register_types(void)
 {
hw/display/vhost-user-gpu.c +1 -0
index 6cdaa1c73b9b..32ef0061f924 100644
@@ -596,6 +596,7 @@ static const TypeInfo vhost_user_gpu_info = {
     .instance_finalize = vhost_user_gpu_instance_finalize,
     .class_init = vhost_user_gpu_class_init,
 };
+module_obj(TYPE_VHOST_USER_GPU);
 
 static void vhost_user_gpu_register_types(void)
 {
hw/display/vhost-user-vga.c +1 -0
index a34a99856d73..072c9c65bc75 100644
@@ -44,6 +44,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_vga_info = {
     .instance_size = sizeof(VhostUserVGA),
     .instance_init = vhost_user_vga_inst_initfn,
 };
+module_obj(TYPE_VHOST_USER_VGA);
 
 static void vhost_user_vga_register_types(void)
 {
hw/display/virtio-gpu-base.c +1 -0
index dd294276cb38..c8da4806e0bb 100644
@@ -256,6 +256,7 @@ static const TypeInfo virtio_gpu_base_info = {
     .class_init = virtio_gpu_base_class_init,
     .abstract = true
 };
+module_obj(TYPE_VIRTIO_GPU_BASE);
 
 static void
 virtio_register_types(void)
hw/display/virtio-gpu-gl.c +3 -0
index d971b480806a..7ab93bf8c829 100644
@@ -154,6 +154,7 @@ static const TypeInfo virtio_gpu_gl_info = {
     .instance_size = sizeof(VirtIOGPUGL),
     .class_init = virtio_gpu_gl_class_init,
 };
+module_obj(TYPE_VIRTIO_GPU_GL);
 
 static void virtio_register_types(void)
 {
@@ -161,3 +162,5 @@ static void virtio_register_types(void)
 }
 
 type_init(virtio_register_types)
+
+module_dep("hw-display-virtio-gpu");
hw/display/virtio-gpu-pci-gl.c +3 -0
index 902dda345275..99b14a07185e 100644
@@ -46,6 +46,7 @@ static const VirtioPCIDeviceTypeInfo virtio_gpu_gl_pci_info = {
     .instance_size = sizeof(VirtIOGPUGLPCI),
     .instance_init = virtio_gpu_gl_initfn,
 };
+module_obj(TYPE_VIRTIO_GPU_GL_PCI);
 
 static void virtio_gpu_gl_pci_register_types(void)
 {
@@ -53,3 +54,5 @@ static void virtio_gpu_gl_pci_register_types(void)
 }
 
 type_init(virtio_gpu_gl_pci_register_types)
+
+module_dep("hw-display-virtio-gpu-pci");
hw/display/virtio-gpu-pci.c +2 -0
index d742a30aecf7..e36eee0c409b 100644
@@ -64,6 +64,7 @@ static const TypeInfo virtio_gpu_pci_base_info = {
     .class_init = virtio_gpu_pci_base_class_init,
     .abstract = true
 };
+module_obj(TYPE_VIRTIO_GPU_PCI_BASE);
 
 #define TYPE_VIRTIO_GPU_PCI "virtio-gpu-pci"
 typedef struct VirtIOGPUPCI VirtIOGPUPCI;
@@ -90,6 +91,7 @@ static const VirtioPCIDeviceTypeInfo virtio_gpu_pci_info = {
     .instance_size = sizeof(VirtIOGPUPCI),
     .instance_init = virtio_gpu_initfn,
 };
+module_obj(TYPE_VIRTIO_GPU_PCI);
 
 static void virtio_gpu_pci_register_types(void)
 {
hw/display/virtio-gpu.c +1 -0
index e183f4ecdaa5..6b7f643951fe 100644
@@ -1427,6 +1427,7 @@ static const TypeInfo virtio_gpu_info = {
     .class_size = sizeof(VirtIOGPUClass),
     .class_init = virtio_gpu_class_init,
 };
+module_obj(TYPE_VIRTIO_GPU);
 
 static void virtio_register_types(void)
 {
hw/display/virtio-vga-gl.c +3 -0
index c971340ebb1a..f22549097c5e 100644
@@ -36,6 +36,7 @@ static VirtioPCIDeviceTypeInfo virtio_vga_gl_info = {
     .instance_size = sizeof(VirtIOVGAGL),
     .instance_init = virtio_vga_gl_inst_initfn,
 };
+module_obj(TYPE_VIRTIO_VGA_GL);
 
 static void virtio_vga_register_types(void)
 {
@@ -45,3 +46,5 @@ static void virtio_vga_register_types(void)
 }
 
 type_init(virtio_vga_register_types)
+
+module_dep("hw-display-virtio-vga");
hw/display/virtio-vga.c +2 -0
index d3c640406152..9e57f61e9edb 100644
@@ -239,6 +239,7 @@ static TypeInfo virtio_vga_base_info = {
     .class_init    = virtio_vga_base_class_init,
     .abstract      = true,
 };
+module_obj(TYPE_VIRTIO_VGA_BASE);
 
 #define TYPE_VIRTIO_VGA "virtio-vga"
 
@@ -268,6 +269,7 @@ static VirtioPCIDeviceTypeInfo virtio_vga_info = {
     .instance_size = sizeof(VirtIOVGA),
     .instance_init = virtio_vga_inst_initfn,
 };
+module_obj(TYPE_VIRTIO_VGA);
 
 static void virtio_vga_register_types(void)
 {
Signature · 1 line
2.31.1
#9 lore raw

PATCH v4 7/34 modules: add chardev module annotations

In reply to Gerd Hoffmann
---
 chardev/baum.c  | 1 +
 chardev/spice.c | 4 ++++
 2 files changed, 5 insertions(+)
chardev/baum.c +1 -0
index 5deca778bc44..79d618e35045 100644
@@ -680,6 +680,7 @@ static const TypeInfo char_braille_type_info = {
     .instance_finalize = char_braille_finalize,
     .class_init = char_braille_class_init,
 };
+module_obj(TYPE_CHARDEV_BRAILLE);
 
 static void register_types(void)
 {
chardev/spice.c +4 -0
index 1104426e3a11..3ffb3fdc0dac 100644
@@ -366,6 +366,7 @@ static const TypeInfo char_spice_type_info = {
     .class_init = char_spice_class_init,
     .abstract = true,
 };
+module_obj(TYPE_CHARDEV_SPICE);
 
 static void char_spicevmc_class_init(ObjectClass *oc, void *data)
 {
@@ -396,6 +397,7 @@ static const TypeInfo char_spiceport_type_info = {
     .parent = TYPE_CHARDEV_SPICE,
     .class_init = char_spiceport_class_init,
 };
+module_obj(TYPE_CHARDEV_SPICEPORT);
 
 static void register_types(void)
 {
@@ -405,3 +407,5 @@ static void register_types(void)
 }
 
 type_init(register_types);
+
+module_dep("ui-spice-core");
Signature · 1 line
2.31.1
#10 lore raw

PATCH v4 8/34 modules: add audio module annotations

In reply to Gerd Hoffmann
---
 audio/spiceaudio.c | 2 ++
 1 file changed, 2 insertions(+)
audio/spiceaudio.c +2 -0
index 999bfbde47c5..a8d370fe6f31 100644
@@ -317,3 +317,5 @@ static void register_audio_spice(void)
     audio_driver_register(&spice_audio_driver);
 }
 type_init(register_audio_spice);
+
+module_dep("ui-spice-core");
Signature · 1 line
2.31.1
#11 lore raw

PATCH v4 9/34 modules: add usb-redir module annotations

In reply to Gerd Hoffmann
---
 hw/usb/redirect.c | 1 +
 1 file changed, 1 insertion(+)
hw/usb/redirect.c +1 -0
index 6a75b0dc4ab2..4ec9326e0582 100644
@@ -2608,6 +2608,7 @@ static const TypeInfo usbredir_dev_info = {
     .class_init    = usbredir_class_initfn,
     .instance_init = usbredir_instance_init,
 };
+module_obj(TYPE_USB_REDIR);
 
 static void usbredir_register_types(void)
 {
Signature · 1 line
2.31.1
#12 lore raw

PATCH v4 10/34 modules: add ccid module annotations

In reply to Gerd Hoffmann
---
 hw/usb/ccid-card-emulated.c | 1 +
 hw/usb/ccid-card-passthru.c | 1 +
 2 files changed, 2 insertions(+)
hw/usb/ccid-card-emulated.c +1 -0
index 5c76bed77aa0..6c8c0355e099 100644
@@ -612,6 +612,7 @@ static const TypeInfo emulated_card_info = {
     .instance_size = sizeof(EmulatedState),
     .class_init    = emulated_class_initfn,
 };
+module_obj(TYPE_EMULATED_CCID);
 
 static void ccid_card_emulated_register_types(void)
 {
hw/usb/ccid-card-passthru.c +1 -0
index 7212d0d7fb5e..fa3040fb7154 100644
@@ -414,6 +414,7 @@ static const TypeInfo passthru_card_info = {
     .instance_size = sizeof(PassthruState),
     .class_init    = passthru_class_initfn,
 };
+module_obj(TYPE_CCID_PASSTHRU);
 
 static void ccid_card_passthru_register_types(void)
 {
Signature · 1 line
2.31.1
#13 lore raw

PATCH v4 11/34 modules: add ui module annotations

In reply to Gerd Hoffmann
---
 ui/egl-headless.c | 4 ++++
 ui/gtk.c          | 4 ++++
 ui/sdl2.c         | 4 ++++
 ui/spice-app.c    | 3 +++
 ui/spice-core.c   | 5 +++++
 5 files changed, 20 insertions(+)
ui/egl-headless.c +4 -0
index da377a74af69..75404e0e8700 100644
@@ -213,3 +213,7 @@ static void register_egl(void)
 }
 
 type_init(register_egl);
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
ui/gtk.c +4 -0
index 98046f577b9d..376b4d528daa 100644
@@ -2333,3 +2333,7 @@ static void register_gtk(void)
 }
 
 type_init(register_gtk);
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
ui/sdl2.c +4 -0
index a203cb0239e1..36d9010cb6c1 100644
@@ -918,3 +918,7 @@ static void register_sdl1(void)
 }
 
 type_init(register_sdl1);
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
ui/spice-app.c +3 -0
index 4325ac2d9c54..641f4a9d53e3 100644
@@ -221,3 +221,6 @@ static void register_spice_app(void)
 }
 
 type_init(register_spice_app);
+
+module_dep("ui-spice-core");
+module_dep("chardev-spice");
ui/spice-core.c +5 -0
index 272d19b0c152..86d43783acac 100644
@@ -1037,3 +1037,8 @@ static void spice_register_config(void)
     qemu_add_opts(&qemu_spice_opts);
 }
 opts_init(spice_register_config);
+module_opts("spice");
+
+#ifdef CONFIG_OPENGL
+module_dep("ui-opengl");
+#endif
Signature · 1 line
2.31.1
#14 lore raw

PATCH v4 12/34 modules: add s390x module annotations

In reply to Gerd Hoffmann
---
 hw/s390x/virtio-ccw-gpu.c | 3 +++
 1 file changed, 3 insertions(+)
hw/s390x/virtio-ccw-gpu.c +3 -0
index 75a9e4bb3908..5868a2a07093 100644
@@ -59,6 +59,7 @@ static const TypeInfo virtio_ccw_gpu = {
     .instance_init = virtio_ccw_gpu_instance_init,
     .class_init    = virtio_ccw_gpu_class_init,
 };
+module_obj(TYPE_VIRTIO_GPU_CCW);
 
 static void virtio_ccw_gpu_register(void)
 {
@@ -68,3 +69,5 @@ static void virtio_ccw_gpu_register(void)
 }
 
 type_init(virtio_ccw_gpu_register)
+
+module_arch("s390x");
Signature · 1 line
2.31.1
#15 lore raw

PATCH v4 13/34 modules: add block module annotations

In reply to Gerd Hoffmann
---
 block/iscsi-opts.c | 1 +
 1 file changed, 1 insertion(+)
block/iscsi-opts.c +1 -0
index afaf8837d6c1..4f2da405e645 100644
@@ -68,3 +68,4 @@ static void iscsi_block_opts_init(void)
 }
 
 block_init(iscsi_block_opts_init);
+module_opts("iscsi");
Signature · 1 line
2.31.1
#16 lore raw

PATCH v4 14/34 modules: use modinfo for dependencies

In reply to Gerd Hoffmann

Use module database for module dependencies.
Drop hard-coded dependency list.

---
 util/module.c | 55 ++++++++++++++++++++-------------------------------
 1 file changed, 21 insertions(+), 34 deletions(-)
util/module.c +21 -34
index 8d3e8275b9f7..7d7b69cbdaca 100644
@@ -182,28 +182,6 @@ static int module_load_file(const char *fname, bool mayfail, bool export_symbols
 out:
     return ret;
 }
-
-static const struct {
-    const char *name;
-    const char *dep;
-} module_deps[] = {
-    { "audio-spice",    "ui-spice-core" },
-    { "chardev-spice",  "ui-spice-core" },
-    { "hw-display-qxl", "ui-spice-core" },
-    { "ui-spice-app",   "ui-spice-core" },
-    { "ui-spice-app",   "chardev-spice" },
-
-    { "hw-display-virtio-gpu-gl", "hw-display-virtio-gpu" },
-    { "hw-display-virtio-gpu-pci-gl", "hw-display-virtio-gpu-pci" },
-    { "hw-display-virtio-vga-gl", "hw-display-virtio-vga" },
-
-#ifdef CONFIG_OPENGL
-    { "ui-egl-headless", "ui-opengl"    },
-    { "ui-gtk",          "ui-opengl"    },
-    { "ui-sdl",          "ui-opengl"    },
-    { "ui-spice-core",   "ui-opengl"    },
-#endif
-};
 #endif
 
 bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
@@ -219,9 +197,11 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
     char *dirs[5];
     char *module_name;
     int i = 0, n_dirs = 0;
-    int ret, dep;
+    int ret;
     bool export_symbols = false;
     static GHashTable *loaded_modules;
+    const QemuModinfo *modinfo;
+    const char **sl;
 
     if (!g_module_supported()) {
         fprintf(stderr, "Module is not supported by system.\n");
@@ -234,23 +214,30 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
 
     module_name = g_strdup_printf("%s%s", prefix, lib_name);
 
-    for (dep = 0; dep < ARRAY_SIZE(module_deps); dep++) {
-        if (strcmp(module_name, module_deps[dep].name) == 0) {
-            /* we depend on another module */
-            module_load_one("", module_deps[dep].dep, false);
-        }
-        if (strcmp(module_name, module_deps[dep].dep) == 0) {
-            /* another module depends on us */
-            export_symbols = true;
-        }
-    }
-
     if (g_hash_table_contains(loaded_modules, module_name)) {
         g_free(module_name);
         return true;
     }
     g_hash_table_add(loaded_modules, module_name);
 
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (modinfo->deps) {
+            if (strcmp(modinfo->name, module_name) == 0) {
+                /* we depend on other module(s) */
+                for (sl = modinfo->deps; *sl != NULL; sl++) {
+                    module_load_one("", *sl, false);
+                }
+            } else {
+                for (sl = modinfo->deps; *sl != NULL; sl++) {
+                    if (strcmp(module_name, *sl) == 0) {
+                        /* another module depends on us */
+                        export_symbols = true;
+                    }
+                }
+            }
+        }
+    }
+
     search_dir = getenv("QEMU_MODULE_DIR");
     if (search_dir != NULL) {
         dirs[n_dirs++] = g_strdup_printf("%s", search_dir);
Signature · 1 line
2.31.1
#17 lore raw

PATCH v4 15/34 modules: use modinfo for qom load

In reply to Gerd Hoffmann

Use module database to figure which module implements a given QOM type. Drop hard-coded object list.

---
 util/module.c | 77 ++++++++++++++++-----------------------------------
 1 file changed, 24 insertions(+), 53 deletions(-)
util/module.c +24 -53
index 7d7b69cbdaca..745ae0fb20ed 100644
@@ -280,80 +280,51 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
     return success;
 }
 
-/*
- * Building devices and other qom objects modular is mostly useful in
- * case they have dependencies to external shared libraries, so we can
- * cut down the core qemu library dependencies.  Which is the case for
- * only a very few devices & objects.
- *
- * So with the expectation that this will be rather the exception than
- * the rule and the list will not gain that many entries, go with a
- * simple manually maintained list for now.
- *
- * The list must be sorted by module (module_load_qom_all() needs this).
- */
-static struct {
-    const char *type;
-    const char *prefix;
-    const char *module;
-} const qom_modules[] = {
-    { "ccid-card-passthru",    "hw-", "usb-smartcard"         },
-    { "ccid-card-emulated",    "hw-", "usb-smartcard"         },
-    { "usb-redir",             "hw-", "usb-redirect"          },
-    { "qxl-vga",               "hw-", "display-qxl"           },
-    { "qxl",                   "hw-", "display-qxl"           },
-    { "virtio-gpu-device",     "hw-", "display-virtio-gpu"    },
-    { "virtio-gpu-gl-device",  "hw-", "display-virtio-gpu-gl" },
-    { "vhost-user-gpu",        "hw-", "display-virtio-gpu"    },
-    { "virtio-gpu-pci-base",   "hw-", "display-virtio-gpu-pci" },
-    { "virtio-gpu-pci",        "hw-", "display-virtio-gpu-pci" },
-    { "virtio-gpu-gl-pci",     "hw-", "display-virtio-gpu-pci-gl" },
-    { "vhost-user-gpu-pci",    "hw-", "display-virtio-gpu-pci" },
-    { "virtio-gpu-ccw",        "hw-", "s390x-virtio-gpu-ccw"   },
-    { "virtio-vga-base",       "hw-", "display-virtio-vga"    },
-    { "virtio-vga",            "hw-", "display-virtio-vga"    },
-    { "virtio-vga-gl",         "hw-", "display-virtio-vga-gl" },
-    { "vhost-user-vga",        "hw-", "display-virtio-vga"    },
-    { "chardev-braille",       "chardev-", "baum"             },
-    { "chardev-spicevmc",      "chardev-", "spice"            },
-    { "chardev-spiceport",     "chardev-", "spice"            },
-};
+#ifdef CONFIG_MODULES
 
 static bool module_loaded_qom_all;
 
 void module_load_qom_one(const char *type)
 {
-    int i;
+    const QemuModinfo *modinfo;
+    const char **sl;
 
     if (!type) {
         return;
     }
-    for (i = 0; i < ARRAY_SIZE(qom_modules); i++) {
-        if (strcmp(qom_modules[i].type, type) == 0) {
-            module_load_one(qom_modules[i].prefix,
-                            qom_modules[i].module,
-                            false);
-            return;
+
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (!modinfo->objs) {
+            continue;
+        }
+        for (sl = modinfo->objs; *sl != NULL; sl++) {
+            if (strcmp(type, *sl) == 0) {
+                module_load_one("", modinfo->name, false);
+            }
         }
     }
 }
 
 void module_load_qom_all(void)
 {
-    int i;
+    const QemuModinfo *modinfo;
 
     if (module_loaded_qom_all) {
         return;
     }
-    for (i = 0; i < ARRAY_SIZE(qom_modules); i++) {
-        if (i > 0 && (strcmp(qom_modules[i - 1].module,
-                             qom_modules[i].module) == 0 &&
-                      strcmp(qom_modules[i - 1].prefix,
-                             qom_modules[i].prefix) == 0)) {
-            /* one module implementing multiple types -> load only once */
+
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (!modinfo->objs) {
             continue;
         }
-        module_load_one(qom_modules[i].prefix, qom_modules[i].module, true);
+        module_load_one("", modinfo->name, false);
     }
     module_loaded_qom_all = true;
 }
+
+#else
+
+void module_load_qom_one(const char *type) {}
+void module_load_qom_all(void) {}
+
+#endif
Signature · 1 line
2.31.1
#18 lore raw

PATCH v4 16/34 modules: use modinfo for qemu opts load

In reply to Gerd Hoffmann

Use module database to figure which module adds given QemuOpts group.

---
 softmmu/vl.c        | 17 -----------------
 stubs/module-opts.c |  4 ----
 util/module.c       | 19 +++++++++++++++++++
 3 files changed, 19 insertions(+), 21 deletions(-)
softmmu/vl.c +0 -17
index e001505d79f7..5c26e80126db 100644
@@ -2675,23 +2675,6 @@ void qmp_x_exit_preconfig(Error **errp)
     }
 }
 
-#ifdef CONFIG_MODULES
-void qemu_load_module_for_opts(const char *group)
-{
-    static bool spice_tried;
-    if (g_str_equal(group, "spice") && !spice_tried) {
-        ui_module_load_one("spice-core");
-        spice_tried = true;
-    }
-
-    static bool iscsi_tried;
-    if (g_str_equal(group, "iscsi") && !iscsi_tried) {
-        block_module_load_one("iscsi");
-        iscsi_tried = true;
-    }
-}
-#endif
-
 void qemu_init(int argc, char **argv, char **envp)
 {
     QemuOpts *opts;
stubs/module-opts.c +0 -4
index a7d0e4ad6ead..5412429ea869 100644
@@ -1,6 +1,2 @@
 #include "qemu/osdep.h"
 #include "qemu/config-file.h"
-
-void qemu_load_module_for_opts(const char *group)
-{
-}
util/module.c +19 -0
index 745ae0fb20ed..a9ec2da9972e 100644
@@ -20,6 +20,7 @@
 #include "qemu/queue.h"
 #include "qemu/module.h"
 #include "qemu/cutils.h"
+#include "qemu/config-file.h"
 #ifdef CONFIG_MODULE_UPGRADES
 #include "qemu-version.h"
 #endif
@@ -322,8 +323,26 @@ void module_load_qom_all(void)
     module_loaded_qom_all = true;
 }
 
+void qemu_load_module_for_opts(const char *group)
+{
+    const QemuModinfo *modinfo;
+    const char **sl;
+
+    for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (!modinfo->opts) {
+            continue;
+        }
+        for (sl = modinfo->opts; *sl != NULL; sl++) {
+            if (strcmp(group, *sl) == 0) {
+                module_load_one("", modinfo->name, false);
+            }
+        }
+    }
+}
+
 #else
 
+void qemu_load_module_for_opts(const char *group) {}
 void module_load_qom_one(const char *type) {}
 void module_load_qom_all(void) {}
 
Signature · 1 line
2.31.1
#19 lore raw

PATCH v4 17/34 modules: add tracepoints

In reply to Gerd Hoffmann

One for module load and one for qom type lookup.

---
 util/module.c     | 3 +++
 util/trace-events | 4 ++++
 2 files changed, 7 insertions(+)
util/module.c +3 -0
index a9ec2da9972e..acaaecad56c9 100644
@@ -24,6 +24,7 @@
 #ifdef CONFIG_MODULE_UPGRADES
 #include "qemu-version.h"
 #endif
+#include "trace.h"
 
 typedef struct ModuleEntry
 {
@@ -176,6 +177,7 @@ static int module_load_file(const char *fname, bool mayfail, bool export_symbols
         ret = 0;
     }
 
+    trace_module_load_module(fname);
     QTAILQ_FOREACH_SAFE(e, &dso_init_list, node, next) {
         QTAILQ_REMOVE(&dso_init_list, e, node);
         g_free(e);
@@ -294,6 +296,7 @@ void module_load_qom_one(const char *type)
         return;
     }
 
+    trace_module_lookup_object_type(type);
     for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
         if (!modinfo->objs) {
             continue;
util/trace-events +4 -0
index 806cac14a762..c8f53d7d9fc3 100644
@@ -100,3 +100,7 @@ uffd_create_fd_api_failed(int err) "errno: %i"
 uffd_create_fd_api_noioctl(uint64_t ioctl_req, uint64_t ioctl_supp) "ioctl_req: 0x%" PRIx64 "ioctl_supp: 0x%" PRIx64
 uffd_register_memory_failed(void *addr, uint64_t length, uint64_t mode, int err) "addr: %p length: %" PRIu64 " mode: 0x%" PRIx64 " errno: %i"
 uffd_unregister_memory_failed(void *addr, uint64_t length, int err) "addr: %p length: %" PRIu64 " errno: %i"
+
+# module.c
+module_load_module(const char *name) "file %s"
+module_lookup_object_type(const char *name) "name %s"
Signature · 1 line
2.31.1
#20 lore raw

PATCH v4 18/34 modules: check arch and block load on mismatch

In reply to Gerd Hoffmann

Add module_allow_arch() to set the target architecture. In case a module is limited to some arch verify arches match and ignore the module if not.

---
 include/qemu/module.h |  1 +
 softmmu/vl.c          |  3 +++
 util/module.c         | 29 +++++++++++++++++++++++++++++
 3 files changed, 33 insertions(+)
include/qemu/module.h +1 -0
index a98748d501d3..7f4b1af8198c 100644
@@ -72,6 +72,7 @@ void module_call_init(module_init_type type);
 bool module_load_one(const char *prefix, const char *lib_name, bool mayfail);
 void module_load_qom_one(const char *type);
 void module_load_qom_all(void);
+void module_allow_arch(const char *arch);
 
 /*
  * module info annotation macros
softmmu/vl.c +3 -0
index 5c26e80126db..a70e7b4658e8 100644
@@ -126,6 +126,8 @@
 #include "sysemu/iothread.h"
 #include "qemu/guest-random.h"
 
+#include "config-host.h"
+
 #define MAX_VIRTIO_CONSOLES 1
 
 typedef struct BlockdevOptionsQueueEntry {
@@ -2725,6 +2727,7 @@ void qemu_init(int argc, char **argv, char **envp)
 
 #ifdef CONFIG_MODULES
     module_init_info(qemu_modinfo);
+    module_allow_arch(TARGET_NAME);
 #endif
 
     qemu_init_subsystems();
util/module.c +29 -0
index acaaecad56c9..065aed09ffef 100644
@@ -117,12 +117,33 @@ static const QemuModinfo module_info_stub[] = { {
     /* end of list */
 } };
 static const QemuModinfo *module_info = module_info_stub;
+static const char *module_arch;
 
 void module_init_info(const QemuModinfo *info)
 {
     module_info = info;
 }
 
+void module_allow_arch(const char *arch)
+{
+    module_arch = arch;
+}
+
+static bool module_check_arch(const QemuModinfo *modinfo)
+{
+    if (modinfo->arch) {
+        if (!module_arch) {
+            /* no arch set -> ignore all */
+            return false;
+        }
+        if (strcmp(module_arch, modinfo->arch) != 0) {
+            /* mismatch */
+            return false;
+        }
+    }
+    return true;
+}
+
 static int module_load_file(const char *fname, bool mayfail, bool export_symbols)
 {
     GModule *g_module;
@@ -224,6 +245,13 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)
     g_hash_table_add(loaded_modules, module_name);
 
     for (modinfo = module_info; modinfo->name != NULL; modinfo++) {
+        if (modinfo->arch) {
+            if (strcmp(modinfo->name, module_name) == 0) {
+                if (!module_check_arch(modinfo)) {
+                    return false;
+                }
+            }
+        }
         if (modinfo->deps) {
             if (strcmp(modinfo->name, module_name) == 0) {
                 /* we depend on other module(s) */
@@ -345,6 +373,7 @@ void qemu_load_module_for_opts(const char *group)
 
 #else
 
+void module_allow_arch(const char *arch) {}
 void qemu_load_module_for_opts(const char *group) {}
 void module_load_qom_one(const char *type) {}
 void module_load_qom_all(void) {}
Signature · 1 line
2.31.1
#21 lore raw

PATCH v4 19/34 modules: check arch on qom lookup

In reply to Gerd Hoffmann

With target-specific modules we can have multiple modules implementing the same object. Therefore we have to check the target arch on lookup to find the correct module.

---
 util/module.c | 6 ++++++
 1 file changed, 6 insertions(+)
util/module.c +6 -0
index 065aed09ffef..6bb4ad915a1c 100644
@@ -329,6 +329,9 @@ void module_load_qom_one(const char *type)
         if (!modinfo->objs) {
             continue;
         }
+        if (!module_check_arch(modinfo)) {
+            continue;
+        }
         for (sl = modinfo->objs; *sl != NULL; sl++) {
             if (strcmp(type, *sl) == 0) {
                 module_load_one("", modinfo->name, false);
@@ -349,6 +352,9 @@ void module_load_qom_all(void)
         if (!modinfo->objs) {
             continue;
         }
+        if (!module_check_arch(modinfo)) {
+            continue;
+        }
         module_load_one("", modinfo->name, false);
     }
     module_loaded_qom_all = true;
Signature · 1 line
2.31.1
#22 lore raw

Re: [PATCH v4 19/34] modules: check arch on qom lookup

Hi,

(old patch)

Quoted 3 lines · Gerd Hoffmann

With target-specific modules we can have multiple modules implementing the same object. Therefore we have to check the target arch on lookup to find the correct module.

"multiple modules implementing the same object." seems a design mistake to me.

Assuming we clean the tree of target-specific modules "implementing the same object" -- due to heterogeneous emulation --, is there another use case for this check?

Quoted 29 lines
---
  util/module.c | 6 ++++++
  1 file changed, 6 insertions(+)
util/module.c +6 -0
index 065aed09ffef..6bb4ad915a1c 100644
@@ -329,6 +329,9 @@ void module_load_qom_one(const char *type)
          if (!modinfo->objs) {
              continue;
          }
+        if (!module_check_arch(modinfo)) {
+            continue;
+        }
          for (sl = modinfo->objs; *sl != NULL; sl++) {
              if (strcmp(type, *sl) == 0) {
                  module_load_one("", modinfo->name, false);
@@ -349,6 +352,9 @@ void module_load_qom_all(void)
          if (!modinfo->objs) {
              continue;
          }
+        if (!module_check_arch(modinfo)) {
+            continue;
+        }
          module_load_one("", modinfo->name, false);
      }
      module_loaded_qom_all = true;
#23 lore raw

Re: [PATCH v4 19/34] modules: check arch on qom lookup

Quoted 11 lines · Philippe Mathieu-Daudé

Hi,

(old patch)

Quoted 3 lines · Gerd Hoffmann

With target-specific modules we can have multiple modules implementing the same object. Therefore we have to check the target arch on lookup to find the correct module.

"multiple modules implementing the same object." seems a design mistake to me.

IIRC that is (or was?) a problem with tcg or kvm modules, not fully sure that ever happened in mainline qemu as the tcg modularization effort stalled at some point. But some object had the same name on all architectures. Which is not a problem when linked into qemu-system-${arch} but is a problem when built as module.

Quoted 2 lines

Assuming we clean the tree of target-specific modules "implementing the same object" -- due to heterogeneous emulation --,

Oh, yes, when linking multiple archs into one qemu binary the name duplication is a problem even in the non-modular case.

Quoted 2 lines

is there
another use case for this check?

I don't think so.

take care,
Gerd

#24 lore raw

PATCH v4 20/34 modules: target-specific module build infrastructure

In reply to Gerd Hoffmann
---
 meson.build | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
meson.build +37 -0
index 6ae2b9ab9443..76020b43fb32 100644
@@ -1781,6 +1781,7 @@ user_ss = ss.source_set()
 util_ss = ss.source_set()
 
 modules = {}
+target_modules = {}
 hw_arch = {}
 target_arch = {}
 target_softmmu_arch = {}
@@ -2060,6 +2061,42 @@ foreach d, list : modules
   endforeach
 endforeach
 
+foreach d, list : target_modules
+  foreach m, module_ss : list
+    if enable_modules and targetos != 'windows'
+      foreach target : target_dirs
+        if target.endswith('-softmmu')
+          config_target = config_target_mak[target]
+          config_target += config_host
+          target_inc = [include_directories('target' / config_target['TARGET_BASE_ARCH'])]
+          c_args = ['-DNEED_CPU_H',
+                    '-DCONFIG_TARGET="@[email protected]"'.format(target),
+                    '-DCONFIG_DEVICES="@[email protected]"'.format(target)]
+          target_module_ss = module_ss.apply(config_target, strict: false)
+          if target_module_ss.sources() != []
+            module_name = d + '-' + m + '-' + config_target['TARGET_NAME']
+            sl = static_library(module_name,
+                                [genh, target_module_ss.sources()],
+                                dependencies: [modulecommon, target_module_ss.dependencies()],
+                                include_directories: target_inc,
+                                c_args: c_args,
+                                pic: true)
+            softmmu_mods += sl
+            # FIXME: Should use sl.extract_all_objects(recursive: true) too.
+            modinfo_files += custom_target(module_name + '.modinfo',
+                                           output: module_name + '.modinfo',
+                                           input: target_module_ss.sources(),
+                                           capture: true,
+                                           command: [modinfo_collect, '--target', target, '@INPUT@'])
+          endif
+        endif
+      endforeach
+    else
+      specific_ss.add_all(module_ss)
+    endif
+  endforeach
+endforeach
+
 if enable_modules
   modinfo_src = custom_target('modinfo.c',
                               output: 'modinfo.c',
Signature · 1 line
2.31.1
#25 lore raw

PATCH v4 21/34 modules: add documentation for module sourcesets

In reply to Gerd Hoffmann
---
 docs/devel/build-system.rst | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
docs/devel/build-system.rst +17 -0
index 7ef36f42d0f5..fd1650442ecc 100644
@@ -272,6 +272,23 @@ Target-dependent emulator sourcesets:
     target_arch += {'arm': arm_ss}
     target_softmmu_arch += {'arm': arm_softmmu_ss}
 
+Module sourcesets:
+  There are two dictionaries for modules: `modules` is used for
+  target-independent modules and `target_modules` is used for
+  target-dependent modules.  When modules are disabled the `module`
+  source sets are added to `softmmu_ss` and the `target_modules`
+  source sets are added to `specific_ss`.
+
+  Both dictionaries are nested.  One dictionary is created per
+  subdirectory, and these per-subdirectory dictionaries are added to
+  the toplevel dictionaries.  For example::
+
+    hw_display_modules = {}
+    qxl_ss = ss.source_set()
+    ...
+    hw_display_modules += { 'qxl': qxl_ss }
+    modules += { 'hw-display': hw_display_modules }
+
 Utility sourcesets:
   All binaries link with a static library `libqemuutil.a`.  This library
   is built from several sourcesets; most of them however host generated
Signature · 1 line
2.31.1
#26 lore raw

PATCH v4 22/34 modules: add module_obj() note to QOM docs

In reply to Gerd Hoffmann
---
 docs/devel/qom.rst | 8 ++++++++
 1 file changed, 8 insertions(+)
docs/devel/qom.rst +8 -0
index 42d0dc4f4da8..e5fe3597cd82 100644
@@ -87,6 +87,14 @@ specific type:
    #define MY_DEVICE(obj) \
       OBJECT_CHECK(MyDevice, obj, TYPE_MY_DEVICE)
 
+In case the ObjectClass implementation can be built as module a
+module_obj() line must be added to make sure qemu loads the module
+when the object is needed.
+
+.. code-block:: c
+
+   module_obj(TYPE_MY_DEVICE);
+
 Class Initialization
 ====================
 
Signature · 1 line
2.31.1
#27 lore raw

PATCH v4 23/34 modules: module.h kerneldoc annotations

In reply to Gerd Hoffmann
---
 include/qemu/module.h | 59 +++++++++++++++++++++++++++++++++----------
 1 file changed, 45 insertions(+), 14 deletions(-)
include/qemu/module.h +45 -14
index 7f4b1af8198c..8bc80535a4d4 100644
@@ -74,11 +74,18 @@ void module_load_qom_one(const char *type);
 void module_load_qom_all(void);
 void module_allow_arch(const char *arch);
 
-/*
- * module info annotation macros
+/**
+ * DOC: module info annotation macros
  *
- * scripts/modinfo-collect.py will collect module info,
- * using the preprocessor and -DQEMU_MODINFO
+ * `scripts/modinfo-collect.py` will collect module info,
+ * using the preprocessor and -DQEMU_MODINFO.
+ *
+ * `scripts/modinfo-generate.py` will create a module meta-data database
+ * from the collected information so qemu knows about module
+ * dependencies and QOM objects implemented by modules.
+ *
+ * See `*.modinfo` and `modinfo.c` in the build directory to check the
+ * script results.
  */
 #ifdef QEMU_MODINFO
 # define modinfo(kind, value) \
@@ -87,24 +94,48 @@ void module_allow_arch(const char *arch);
 # define modinfo(kind, value)
 #endif
 
-/* module implements QOM type <name> */
+/**
+ * module_obj
+ *
+ * @name: QOM type.
+ *
+ * This module implements QOM type @name.
+ */
 #define module_obj(name) modinfo(obj, name)
 
-/* module has a dependency on <name> */
+/**
+ * module_dep
+ *
+ * @name: module name
+ *
+ * This module depends on module @name.
+ */
 #define module_dep(name) modinfo(dep, name)
 
-/* module is for target architecture <name> */
+/**
+ * module_arch
+ *
+ * @arch: target architecture
+ *
+ * This module is for target architecture @arch.
+ *
+ * Note that target-dependent modules are tagged automatically, so
+ * this is only needed in case target-independent modules should be
+ * restricted.  Use case example: the ccw bus is implemented by s390x
+ * only.
+ */
 #define module_arch(name) modinfo(arch, name)
 
-/* module registers QemuOpts <name> */
+/**
+ * module_opts
+ *
+ * @name: QemuOpts name
+ *
+ * This module registers QemuOpts @name.
+ */
 #define module_opts(name) modinfo(opts, name)
 
-/*
- * module info database
- *
- * scripts/modinfo-generate.c will build this using the data collected
- * by scripts/modinfo-collect.py
- */
+/* module info database (created by scripts/modinfo-generate.py) */
 typedef struct QemuModinfo QemuModinfo;
 struct QemuModinfo {
     const char *name;
Signature · 1 line
2.31.1
#28 lore raw

PATCH v4 24/34 modules: hook up modules.h to docs build

In reply to Gerd Hoffmann
---
 docs/devel/index.rst   | 1 +
 docs/devel/modules.rst | 5 +++++
 2 files changed, 6 insertions(+)
 create mode 100644 docs/devel/modules.rst
docs/devel/index.rst +1 -0
index 977c3893bdaf..ba90badbbd6d 100644
@@ -41,6 +41,7 @@ Contents:
    s390-dasd-ipl
    clocks
    qom
+   modules
    block-coroutine-wrapper
    multi-process
    ebpf_rss
docs/devel/modules.rst new file +5 -0
new file mode 100644
index 000000000000..066f347b89ba
@@ -0,0 +1,5 @@
+============
+Qemu modules
+============
+
+.. kernel-doc:: include/qemu/module.h
Signature · 1 line
2.31.1
#29 lore raw

PATCH v4 25/34 accel: autoload modules

In reply to Gerd Hoffmann

Call module_object_class_by_name() instead of object_class_by_name() for objects possibly implemented as module

---
 accel/accel-common.c  | 2 +-
 accel/accel-softmmu.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
accel/accel-common.c +1 -1
index cf07f78421d6..7b8ec7e0f72a 100644
@@ -44,7 +44,7 @@ static const TypeInfo accel_type = {
 AccelClass *accel_find(const char *opt_name)
 {
     char *class_name = g_strdup_printf(ACCEL_CLASS_NAME("%s"), opt_name);
-    AccelClass *ac = ACCEL_CLASS(object_class_by_name(class_name));
+    AccelClass *ac = ACCEL_CLASS(module_object_class_by_name(class_name));
     g_free(class_name);
     return ac;
 }
accel/accel-softmmu.c +1 -1
index 50fa5acaa401..67276e4f5222 100644
@@ -72,7 +72,7 @@ void accel_init_ops_interfaces(AccelClass *ac)
     g_assert(ac_name != NULL);
 
     ops_name = g_strdup_printf("%s" ACCEL_OPS_SUFFIX, ac_name);
-    ops = ACCEL_OPS_CLASS(object_class_by_name(ops_name));
+    ops = ACCEL_OPS_CLASS(module_object_class_by_name(ops_name));
     g_free(ops_name);
 
     /*
Signature · 1 line
2.31.1
#30 lore raw

PATCH v4 26/34 accel: add qtest module annotations

In reply to Gerd Hoffmann

Add module annotations for qtest so autoloading works.

---
 accel/qtest/qtest.c | 2 ++
 1 file changed, 2 insertions(+)
accel/qtest/qtest.c +2 -0
index edb29f6fa4c0..7e6b8110d52b 100644
@@ -45,6 +45,7 @@ static const TypeInfo qtest_accel_type = {
     .parent = TYPE_ACCEL,
     .class_init = qtest_accel_class_init,
 };
+module_obj(TYPE_QTEST_ACCEL);
 
 static void qtest_accel_ops_class_init(ObjectClass *oc, void *data)
 {
@@ -61,6 +62,7 @@ static const TypeInfo qtest_accel_ops_type = {
     .class_init = qtest_accel_ops_class_init,
     .abstract = true,
 };
+module_obj(ACCEL_OPS_NAME("qtest"));
 
 static void qtest_type_init(void)
 {
Signature · 1 line
2.31.1
#31 lore raw

PATCH v4 27/34 accel: build qtest modular

In reply to Gerd Hoffmann

Allow building accelerators as module.
Start with qtest as first user.

---
 accel/qtest/meson.build | 8 ++------
 meson.build             | 6 ++++++
 2 files changed, 8 insertions(+), 6 deletions(-)
accel/qtest/meson.build +2 -6
index a2f327645980..4c6560029336 100644
@@ -1,6 +1,2 @@
-qtest_ss = ss.source_set()
-qtest_ss.add(files(
-  'qtest.c',
-))
-
-specific_ss.add_all(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'], if_true: qtest_ss)
+qtest_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'],
+                    if_true: files('qtest.c'))
meson.build +6 -0
index 76020b43fb32..d2cacc145a91 100644
@@ -1780,6 +1780,9 @@ trace_ss = ss.source_set()
 user_ss = ss.source_set()
 util_ss = ss.source_set()
 
+# accel modules
+qtest_module_ss = ss.source_set()
+
 modules = {}
 target_modules = {}
 hw_arch = {}
@@ -2018,6 +2021,9 @@ specific_ss.add_all(when: 'CONFIG_LINUX_USER', if_true: linux_user_ss)
 subdir('tests/qtest/libqos')
 subdir('tests/qtest/fuzz')
 
+# accel modules
+target_modules += { 'accel' : { 'qtest': qtest_module_ss }}
+
 ########################
 # Library dependencies #
 ########################
Signature · 1 line
2.31.1
#32 lore raw

PATCH v4 28/34 accel: add tcg module annotations

In reply to Gerd Hoffmann

Add module annotations for tcg so autoloading works.

---
 accel/tcg/tcg-accel-ops.c | 1 +
 accel/tcg/tcg-all.c       | 1 +
 2 files changed, 2 insertions(+)
accel/tcg/tcg-accel-ops.c +1 -0
index 7191315aeed4..1a8e8390bd60 100644
@@ -124,6 +124,7 @@ static const TypeInfo tcg_accel_ops_type = {
     .class_init = tcg_accel_ops_class_init,
     .abstract = true,
 };
+module_obj(ACCEL_OPS_NAME("tcg"));
 
 static void tcg_accel_ops_register_types(void)
 {
accel/tcg/tcg-all.c +1 -0
index 00803f76d870..d6336a9c966d 100644
@@ -238,6 +238,7 @@ static const TypeInfo tcg_accel_type = {
     .class_init = tcg_accel_class_init,
     .instance_size = sizeof(TCGState),
 };
+module_obj(TYPE_TCG_ACCEL);
 
 static void register_accel_types(void)
 {
Signature · 1 line
2.31.1
#33 lore raw

PATCH v4 29/34 accel: build tcg modular

In reply to Gerd Hoffmann

Build tcg accel ops as module.
Which is only a small fraction of tcg.
Also only x86 for now.

---
 accel/tcg/meson.build |  5 ++++-
 meson.build           | 14 +++++++++++++-
 2 files changed, 17 insertions(+), 2 deletions(-)
accel/tcg/meson.build +4 -1
index 1236ac7b910b..0ae9180282e3 100644
@@ -15,8 +15,11 @@ specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
 
 specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
   'cputlb.c',
+))
+
+tcg_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
   'tcg-accel-ops.c',
   'tcg-accel-ops-mttcg.c',
   'tcg-accel-ops-icount.c',
-  'tcg-accel-ops-rr.c'
+  'tcg-accel-ops-rr.c',
 ))
meson.build +13 -1
index d2cacc145a91..9f1ca4177073 100644
@@ -92,6 +92,8 @@ if cpu in ['x86', 'x86_64']
   }
 endif
 
+modular_tcg = ['i386-softmmu', 'x86_64-softmmu']
+
 edk2_targets = [ 'arm-softmmu', 'aarch64-softmmu', 'i386-softmmu', 'x86_64-softmmu' ]
 install_edk2_blobs = false
 if get_option('install_blobs')
@@ -1311,6 +1313,11 @@ foreach target : target_dirs
       elif sym == 'CONFIG_XEN' and have_xen_pci_passthrough
         config_target += { 'CONFIG_XEN_PCI_PASSTHROUGH': 'y' }
       endif
+      if target in modular_tcg
+        config_target += { 'CONFIG_TCG_MODULAR': 'y' }
+      else
+        config_target += { 'CONFIG_TCG_BUILTIN': 'y' }
+      endif
       accel_kconfig += [ sym + '=y' ]
     endif
   endforeach
@@ -1782,6 +1789,7 @@ util_ss = ss.source_set()
 
 # accel modules
 qtest_module_ss = ss.source_set()
+tcg_module_ss = ss.source_set()
 
 modules = {}
 target_modules = {}
@@ -2022,7 +2030,11 @@ subdir('tests/qtest/libqos')
 subdir('tests/qtest/fuzz')
 
 # accel modules
-target_modules += { 'accel' : { 'qtest': qtest_module_ss }}
+tcg_real_module_ss = ss.source_set()
+tcg_real_module_ss.add_all(when: 'CONFIG_TCG_MODULAR', if_true: tcg_module_ss)
+specific_ss.add_all(when: 'CONFIG_TCG_BUILTIN', if_true: tcg_module_ss)
+target_modules += { 'accel' : { 'qtest': qtest_module_ss,
+                                'tcg': tcg_real_module_ss }}
 
 ########################
 # Library dependencies #
Signature · 1 line
2.31.1
#34 lore raw

PATCH v4 30/34 monitor: allow register hmp commands

In reply to Gerd Hoffmann

Allow commands having a NULL cmd pointer, add a function to set the pointer later. Use case: allow modules implement hmp commands.

---
 include/monitor/monitor.h |  3 +++
 monitor/hmp.c             |  7 +++++++
 monitor/misc.c            | 16 ++++++++++++++++
 3 files changed, 26 insertions(+)
include/monitor/monitor.h +3 -0
index 1211d6e6d69f..1a8a369b50b2 100644
@@ -51,4 +51,7 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags);
 void monitor_fdset_dup_fd_remove(int dup_fd);
 int64_t monitor_fdset_dup_fd_find(int dup_fd);
 
+void monitor_register_hmp(const char *name, bool info,
+                          void (*cmd)(Monitor *mon, const QDict *qdict));
+
 #endif /* MONITOR_H */
monitor/hmp.c +7 -0
index 6c0b33a0b19d..d50c3124e1e1 100644
@@ -1089,6 +1089,13 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
         return;
     }
 
+    if (!cmd->cmd) {
+        /* FIXME: is it useful to try autoload modules here ??? */
+        monitor_printf(&mon->common, "Command \"%.*s\" is not available.\n",
+                       (int)(cmdline - cmd_start), cmd_start);
+        return;
+    }
+
     qdict = monitor_parse_arguments(&mon->common, &cmdline, cmd);
     if (!qdict) {
         while (cmdline > cmd_start && qemu_isspace(cmdline[-1])) {
monitor/misc.c +16 -0
index 1539e18557f0..ad476c6e51ea 100644
@@ -1974,6 +1974,22 @@ static void sortcmdlist(void)
           compare_mon_cmd);
 }
 
+void monitor_register_hmp(const char *name, bool info,
+                          void (*cmd)(Monitor *mon, const QDict *qdict))
+{
+    HMPCommand *table = info ? hmp_info_cmds : hmp_cmds;
+
+    while (table->name != NULL) {
+        if (strcmp(table->name, name) == 0) {
+            g_assert(table->cmd == NULL);
+            table->cmd = cmd;
+            return;
+        }
+        table++;
+    }
+    g_assert_not_reached();
+}
+
 void monitor_init_globals(void)
 {
     monitor_init_globals_core();
Signature · 1 line
2.31.1
Acked-by on v4 30/34
#35 lore raw

Re: [PATCH v4 30/34] monitor: allow register hmp commands

Quoted 4 lines · * Gerd Hoffmann ([email protected]) wrote:

Allow commands having a NULL cmd pointer, add a function to set the pointer later. Use case: allow modules implement hmp commands.

So this is OK, so

however, I can imagine:
a) Auto load as you suggest
b) Making the error better 'Command ... needs module ....'
c) Don't actually define the command in the tables at all; make the module actually add the command to the table.

Dave

Quoted 66 lines
---
 include/monitor/monitor.h |  3 +++
 monitor/hmp.c             |  7 +++++++
 monitor/misc.c            | 16 ++++++++++++++++
 3 files changed, 26 insertions(+)
include/monitor/monitor.h +3 -0
index 1211d6e6d69f..1a8a369b50b2 100644
@@ -51,4 +51,7 @@ int monitor_fdset_dup_fd_add(int64_t fdset_id, int flags);
 void monitor_fdset_dup_fd_remove(int dup_fd);
 int64_t monitor_fdset_dup_fd_find(int dup_fd);
 
+void monitor_register_hmp(const char *name, bool info,
+                          void (*cmd)(Monitor *mon, const QDict *qdict));
+
 #endif /* MONITOR_H */
monitor/hmp.c +7 -0
index 6c0b33a0b19d..d50c3124e1e1 100644
@@ -1089,6 +1089,13 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
         return;
     }
 
+    if (!cmd->cmd) {
+        /* FIXME: is it useful to try autoload modules here ??? */
+        monitor_printf(&mon->common, "Command \"%.*s\" is not available.\n",
+                       (int)(cmdline - cmd_start), cmd_start);
+        return;
+    }
+
     qdict = monitor_parse_arguments(&mon->common, &cmdline, cmd);
     if (!qdict) {
         while (cmdline > cmd_start && qemu_isspace(cmdline[-1])) {
monitor/misc.c +16 -0
index 1539e18557f0..ad476c6e51ea 100644
@@ -1974,6 +1974,22 @@ static void sortcmdlist(void)
           compare_mon_cmd);
 }
 
+void monitor_register_hmp(const char *name, bool info,
+                          void (*cmd)(Monitor *mon, const QDict *qdict))
+{
+    HMPCommand *table = info ? hmp_info_cmds : hmp_cmds;
+
+    while (table->name != NULL) {
+        if (strcmp(table->name, name) == 0) {
+            g_assert(table->cmd == NULL);
+            table->cmd = cmd;
+            return;
+        }
+        table++;
+    }
+    g_assert_not_reached();
+}
+
 void monitor_init_globals(void)
 {
     monitor_init_globals_core();
Signature · 1 line
2.31.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#36 lore raw

Re: [PATCH v4 30/34] monitor: allow register hmp commands

Quoted 4 lines · Dr. David Alan Gilbert
Quoted 3 lines · * Gerd Hoffmann ([email protected]) wrote:

Allow commands having a NULL cmd pointer, add a function to set the pointer later. Use case: allow modules implement hmp commands.

Quoted 8 lines
Quoted 1 line

So this is OK, so

however, I can imagine:
a) Auto load as you suggest

Not sure about that. The tcg monitor commands are pointless when you picked another accelerator, and "info usbhost" would probably also be most useful when trouble-shooting usb-host issues. That's why I left it as FIXME question for now. But can certainly be done, we can add something along the lines of 'module_hmp("info usbhost");' to the meta-data database and autoload based on that (or use it for more verbose error messages).

Quoted 2 lines

c) Don't actually define the command in the tables at all; make the module actually add the command to the table.

Another possible approach. I don't see a need for modules to expand the list of commands though, so I only set the function pointer for existing table entries ...

take care,
Gerd

#37 lore raw

PATCH v4 31/34 usb: drop usb_host_dev_is_scsi_storage hook

In reply to Gerd Hoffmann

Introduce an usb device flag instead, set it when usb-host looks at the device descriptors anyway. Also set it for emulated storage devices, for consistency. Add an inline helper function to check the flag.

---
 include/hw/usb.h             |  7 ++++++-
 hw/ppc/spapr.c               |  2 +-
 hw/usb/dev-storage-bot.c     |  1 +
 hw/usb/dev-storage-classic.c |  1 +
 hw/usb/dev-uas.c             |  1 +
 hw/usb/host-libusb.c         | 36 +++++++-----------------------------
 hw/usb/host-stub.c           |  5 -----
 7 files changed, 17 insertions(+), 36 deletions(-)
include/hw/usb.h +6 -1
index 436e07b30404..33668dd0a99a 100644
@@ -219,6 +219,7 @@ enum USBDeviceFlags {
     USB_DEV_FLAG_IS_HOST,
     USB_DEV_FLAG_MSOS_DESC_ENABLE,
     USB_DEV_FLAG_MSOS_DESC_IN_USE,
+    USB_DEV_FLAG_IS_SCSI_STORAGE,
 };
 
 /* definition of a USB device */
@@ -465,7 +466,6 @@ void usb_generic_async_ctrl_complete(USBDevice *s, USBPacket *p);
 
 /* usb-linux.c */
 void hmp_info_usbhost(Monitor *mon, const QDict *qdict);
-bool usb_host_dev_is_scsi_storage(USBDevice *usbdev);
 
 /* usb ports of the VM */
 
@@ -561,6 +561,11 @@ const char *usb_device_get_product_desc(USBDevice *dev);
 
 const USBDesc *usb_device_get_usb_desc(USBDevice *dev);
 
+static inline bool usb_device_is_scsi_storage(USBDevice *dev)
+{
+    return dev->flags & (1 << USB_DEV_FLAG_IS_SCSI_STORAGE);
+}
+
 /* quirks.c */
 
 /* In bulk endpoints are streaming data sources (iow behave like isoc eps) */
hw/ppc/spapr.c +1 -1
index 4dd90b75cc52..f83a081af0f1 100644
@@ -3106,7 +3106,7 @@ static char *spapr_get_fw_dev_path(FWPathProvider *p, BusState *bus,
      */
     if (strcmp("usb-host", qdev_fw_name(dev)) == 0) {
         USBDevice *usbdev = CAST(USBDevice, dev, TYPE_USB_DEVICE);
-        if (usb_host_dev_is_scsi_storage(usbdev)) {
+        if (usb_device_is_scsi_storage(usbdev)) {
             return g_strdup_printf("storage@%s/disk", usbdev->port->path);
         }
     }
hw/usb/dev-storage-bot.c +1 -0
index 6aad026d1133..68ebaca10c66 100644
@@ -32,6 +32,7 @@ static void usb_msd_bot_realize(USBDevice *dev, Error **errp)
 
     usb_desc_create_serial(dev);
     usb_desc_init(dev);
+    dev->flags |= (1 << USB_DEV_FLAG_IS_SCSI_STORAGE);
     if (d->hotplugged) {
         s->dev.auto_attach = 0;
     }
hw/usb/dev-storage-classic.c +1 -0
index 00cb34b22f02..3d017a4e6791 100644
@@ -64,6 +64,7 @@ static void usb_msd_storage_realize(USBDevice *dev, Error **errp)
 
     usb_desc_create_serial(dev);
     usb_desc_init(dev);
+    dev->flags |= (1 << USB_DEV_FLAG_IS_SCSI_STORAGE);
     scsi_bus_new(&s->bus, sizeof(s->bus), DEVICE(dev),
                  &usb_msd_scsi_info_storage, NULL);
     scsi_dev = scsi_bus_legacy_add_drive(&s->bus, blk, 0, !!s->removable,
hw/usb/dev-uas.c +1 -0
index d2bd85d3f6bb..263056231c79 100644
@@ -926,6 +926,7 @@ static void usb_uas_realize(USBDevice *dev, Error **errp)
     QTAILQ_INIT(&uas->requests);
     uas->status_bh = qemu_bh_new(usb_uas_send_status_bh, uas);
 
+    dev->flags |= (1 << USB_DEV_FLAG_IS_SCSI_STORAGE);
     scsi_bus_new(&uas->bus, sizeof(uas->bus), DEVICE(dev),
                  &usb_uas_scsi_info, NULL);
 }
hw/usb/host-libusb.c +7 -29
index 2518306f527f..e6d21aa8e1d3 100644
@@ -770,6 +770,13 @@ static void usb_host_speed_compat(USBHostDevice *s)
         for (i = 0; i < conf->bNumInterfaces; i++) {
             for (a = 0; a < conf->interface[i].num_altsetting; a++) {
                 intf = &conf->interface[i].altsetting[a];
+
+                if (intf->bInterfaceClass == LIBUSB_CLASS_MASS_STORAGE &&
+                    intf->bInterfaceSubClass == 6) { /* SCSI */
+                    udev->flags |= (1 << USB_DEV_FLAG_IS_SCSI_STORAGE);
+                    break;
+                }
+
                 for (e = 0; e < intf->bNumEndpoints; e++) {
                     endp = &intf->endpoint[e];
                     type = endp->bmAttributes & 0x3;
@@ -1893,35 +1900,6 @@ static void usb_host_auto_check(void *unused)
     timer_mod(usb_auto_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + 2000);
 }
 
-/**
- * Check whether USB host device has a USB mass storage SCSI interface
- */
-bool usb_host_dev_is_scsi_storage(USBDevice *ud)
-{
-    USBHostDevice *uhd = USB_HOST_DEVICE(ud);
-    struct libusb_config_descriptor *conf;
-    const struct libusb_interface_descriptor *intf;
-    bool is_scsi_storage = false;
-    int i;
-
-    if (!uhd || libusb_get_active_config_descriptor(uhd->dev, &conf) != 0) {
-        return false;
-    }
-
-    for (i = 0; i < conf->bNumInterfaces; i++) {
-        intf = &conf->interface[i].altsetting[ud->altsetting[i]];
-        if (intf->bInterfaceClass == LIBUSB_CLASS_MASS_STORAGE &&
-            intf->bInterfaceSubClass == 6) {                 /* 6 means SCSI */
-            is_scsi_storage = true;
-            break;
-        }
-    }
-
-    libusb_free_config_descriptor(conf);
-
-    return is_scsi_storage;
-}
-
 void hmp_info_usbhost(Monitor *mon, const QDict *qdict)
 {
     libusb_device **devs = NULL;
hw/usb/host-stub.c +0 -5
index 80809ceba542..bbe69baa390f 100644
@@ -38,8 +38,3 @@ void hmp_info_usbhost(Monitor *mon, const QDict *qdict)
 {
     monitor_printf(mon, "USB host devices not supported\n");
 }
-
-bool usb_host_dev_is_scsi_storage(USBDevice *ud)
-{
-    return false;
-}
Signature · 1 line
2.31.1
#38 lore raw

PATCH v4 32/34 monitor/usb: register 'info usbhost' dynamically

In reply to Gerd Hoffmann
---
 hw/usb/host-libusb.c |  1 +
 hw/usb/host-stub.c   | 40 ----------------------------------------
 hmp-commands-info.hx |  1 -
 hw/usb/meson.build   |  4 +---
 4 files changed, 2 insertions(+), 44 deletions(-)
 delete mode 100644 hw/usb/host-stub.c
hw/usb/host-libusb.c +1 -0
index e6d21aa8e1d3..2b7f87872ce3 100644
@@ -1781,6 +1781,7 @@ static TypeInfo usb_host_dev_info = {
 static void usb_host_register_types(void)
 {
     type_register_static(&usb_host_dev_info);
+    monitor_register_hmp("usbhost", true, hmp_info_usbhost);
 }
 
 type_init(usb_host_register_types)
hw/usb/host-stub.c deleted +0 -40
deleted file mode 100644
index bbe69baa390f..000000000000
@@ -1,40 +0,0 @@
-/*
- * Stub host USB redirector
- *
- * Copyright (c) 2005 Fabrice Bellard
- *
- * Copyright (c) 2008 Max Krasnyansky
- *      Support for host device auto connect & disconnect
- *      Major rewrite to support fully async operation
- *
- * Copyright 2008 TJ <[email protected]>
- *      Added flexible support for /dev/bus/usb /sys/bus/usb/devices in addition
- *      to the legacy /proc/bus/usb USB device discovery and handling
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-
-#include "qemu/osdep.h"
-#include "hw/usb.h"
-#include "monitor/monitor.h"
-
-void hmp_info_usbhost(Monitor *mon, const QDict *qdict)
-{
-    monitor_printf(mon, "USB host devices not supported\n");
-}
hmp-commands-info.hx +0 -1
index fb59c27200cb..ce42aef47acb 100644
@@ -368,7 +368,6 @@ ERST
         .args_type  = "",
         .params     = "",
         .help       = "show host USB devices",
-        .cmd        = hmp_info_usbhost,
     },
 
 SRST
hw/usb/meson.build +1 -3
index f357270d0b6b..3d8f2ae99302 100644
@@ -73,9 +73,7 @@ endif
 
 # usb pass-through
 softmmu_ss.add(when: ['CONFIG_USB', 'CONFIG_USB_LIBUSB', libusb],
-               if_true: files('host-libusb.c'),
-               if_false: files('host-stub.c'))
-softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('host-stub.c'))
+               if_true: files('host-libusb.c'))
 
 softmmu_ss.add(when: ['CONFIG_USB', 'CONFIG_XEN', libusb], if_true: files('xen-usb.c'))
 
Signature · 1 line
2.31.1
#39 lore raw

PATCH v4 33/34 usb: build usb-host as module

In reply to Gerd Hoffmann

Drop one more shared library dependency (libusb) from core qemu.

---
 hw/usb/host-libusb.c | 1 +
 hw/usb/meson.build   | 8 ++++++--
 2 files changed, 7 insertions(+), 2 deletions(-)
hw/usb/host-libusb.c +1 -0
index 2b7f87872ce3..c0f314462aaf 100644
@@ -1777,6 +1777,7 @@ static TypeInfo usb_host_dev_info = {
     .class_init    = usb_host_class_initfn,
     .instance_init = usb_host_instance_init,
 };
+module_obj(TYPE_USB_HOST_DEVICE);
 
 static void usb_host_register_types(void)
 {
hw/usb/meson.build +6 -2
index 3d8f2ae99302..0a6029ec9797 100644
@@ -72,8 +72,12 @@ if config_host.has_key('CONFIG_USB_REDIR')
 endif
 
 # usb pass-through
-softmmu_ss.add(when: ['CONFIG_USB', 'CONFIG_USB_LIBUSB', libusb],
-               if_true: files('host-libusb.c'))
+if config_host.has_key('CONFIG_USB_LIBUSB')
+  usbhost_ss = ss.source_set()
+  usbhost_ss.add(when: ['CONFIG_USB', libusb],
+                 if_true: files('host-libusb.c'))
+  hw_usb_modules += {'host': usbhost_ss}
+endif
 
 softmmu_ss.add(when: ['CONFIG_USB', 'CONFIG_XEN', libusb], if_true: files('xen-usb.c'))
 
Signature · 1 line
2.31.1
#40 lore raw

PATCH v4 34/34 monitor/tcg: move tcg hmp commands to accel/tcg, register them dynamically

In reply to Gerd Hoffmann

One more little step towards modular tcg ...

---
 accel/tcg/hmp.c       | 29 +++++++++++++++++++++++++++++
 monitor/misc.c        | 18 ------------------
 accel/tcg/meson.build |  1 +
 hmp-commands-info.hx  |  2 --
 4 files changed, 30 insertions(+), 20 deletions(-)
 create mode 100644 accel/tcg/hmp.c
accel/tcg/hmp.c new file +29 -0
new file mode 100644
index 000000000000..a6e72fdb3ed6
@@ -0,0 +1,29 @@
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "exec/exec-all.h"
+#include "monitor/monitor.h"
+#include "sysemu/tcg.h"
+
+static void hmp_info_jit(Monitor *mon, const QDict *qdict)
+{
+    if (!tcg_enabled()) {
+        error_report("JIT information is only available with accel=tcg");
+        return;
+    }
+
+    dump_exec_info();
+    dump_drift_info();
+}
+
+static void hmp_info_opcount(Monitor *mon, const QDict *qdict)
+{
+    dump_opcount_info();
+}
+
+static void hmp_tcg_register(void)
+{
+    monitor_register_hmp("jit", true, hmp_info_jit);
+    monitor_register_hmp("opcount", true, hmp_info_opcount);
+}
+
+type_init(hmp_tcg_register);
monitor/misc.c +0 -18
index ad476c6e51ea..b28874d6dc59 100644
@@ -320,24 +320,6 @@ static void hmp_info_registers(Monitor *mon, const QDict *qdict)
     }
 }
 
-#ifdef CONFIG_TCG
-static void hmp_info_jit(Monitor *mon, const QDict *qdict)
-{
-    if (!tcg_enabled()) {
-        error_report("JIT information is only available with accel=tcg");
-        return;
-    }
-
-    dump_exec_info();
-    dump_drift_info();
-}
-
-static void hmp_info_opcount(Monitor *mon, const QDict *qdict)
-{
-    dump_opcount_info();
-}
-#endif
-
 static void hmp_info_sync_profile(Monitor *mon, const QDict *qdict)
 {
     int64_t max = qdict_get_try_int(qdict, "max", 10);
accel/tcg/meson.build +1 -0
index 0ae9180282e3..137a1a44cc0a 100644
@@ -15,6 +15,7 @@ specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
 
 specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
   'cputlb.c',
+  'hmp.c',
 ))
 
 tcg_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
hmp-commands-info.hx +0 -2
index ce42aef47acb..27206ac049df 100644
@@ -274,7 +274,6 @@ ERST
         .args_type  = "",
         .params     = "",
         .help       = "show dynamic compiler info",
-        .cmd        = hmp_info_jit,
     },
 #endif
 
@@ -289,7 +288,6 @@ ERST
         .args_type  = "",
         .params     = "",
         .help       = "show dynamic compiler opcode counters",
-        .cmd        = hmp_info_opcount,
     },
 #endif
 
Signature · 1 line
2.31.1
Acked-by on v4 34/34
#41 lore raw

Re: [PATCH v4 34/34] monitor/tcg: move tcg hmp commands to accel/tcg, register them dynamically

Quoted 3 lines · * Gerd Hoffmann ([email protected]) wrote:

One more little step towards modular tcg ...

Quoted 107 lines
---
 accel/tcg/hmp.c       | 29 +++++++++++++++++++++++++++++
 monitor/misc.c        | 18 ------------------
 accel/tcg/meson.build |  1 +
 hmp-commands-info.hx  |  2 --
 4 files changed, 30 insertions(+), 20 deletions(-)
 create mode 100644 accel/tcg/hmp.c
accel/tcg/hmp.c new file +29 -0
new file mode 100644
index 000000000000..a6e72fdb3ed6
@@ -0,0 +1,29 @@
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "exec/exec-all.h"
+#include "monitor/monitor.h"
+#include "sysemu/tcg.h"
+
+static void hmp_info_jit(Monitor *mon, const QDict *qdict)
+{
+    if (!tcg_enabled()) {
+        error_report("JIT information is only available with accel=tcg");
+        return;
+    }
+
+    dump_exec_info();
+    dump_drift_info();
+}
+
+static void hmp_info_opcount(Monitor *mon, const QDict *qdict)
+{
+    dump_opcount_info();
+}
+
+static void hmp_tcg_register(void)
+{
+    monitor_register_hmp("jit", true, hmp_info_jit);
+    monitor_register_hmp("opcount", true, hmp_info_opcount);
+}
+
+type_init(hmp_tcg_register);
monitor/misc.c +0 -18
index ad476c6e51ea..b28874d6dc59 100644
@@ -320,24 +320,6 @@ static void hmp_info_registers(Monitor *mon, const QDict *qdict)
     }
 }
 
-#ifdef CONFIG_TCG
-static void hmp_info_jit(Monitor *mon, const QDict *qdict)
-{
-    if (!tcg_enabled()) {
-        error_report("JIT information is only available with accel=tcg");
-        return;
-    }
-
-    dump_exec_info();
-    dump_drift_info();
-}
-
-static void hmp_info_opcount(Monitor *mon, const QDict *qdict)
-{
-    dump_opcount_info();
-}
-#endif
-
 static void hmp_info_sync_profile(Monitor *mon, const QDict *qdict)
 {
     int64_t max = qdict_get_try_int(qdict, "max", 10);
accel/tcg/meson.build +1 -0
index 0ae9180282e3..137a1a44cc0a 100644
@@ -15,6 +15,7 @@ specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
 
 specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
   'cputlb.c',
+  'hmp.c',
 ))
 
 tcg_module_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
hmp-commands-info.hx +0 -2
index ce42aef47acb..27206ac049df 100644
@@ -274,7 +274,6 @@ ERST
         .args_type  = "",
         .params     = "",
         .help       = "show dynamic compiler info",
-        .cmd        = hmp_info_jit,
     },
 #endif
 
@@ -289,7 +288,6 @@ ERST
         .args_type  = "",
         .params     = "",
         .help       = "show dynamic compiler opcode counters",
-        .cmd        = hmp_info_opcount,
     },
 #endif
 
Signature · 1 line
2.31.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Reviewed-by on the v4 series
#42 lore raw
In reply to Gerd Hoffmann

Hello Gerd,

Reviewed and tested successfully here. Thank you!

Quoted 142 lines · Gerd Hoffmann

This patch series adds support for module meta-data. Today this is either hard-coded in qemu (see qemu_load_module_for_opts) or handled with manually maintained lists in util/module (see module_deps[] and qom_modules[]). This series replaced that scheme with annotation macros, so the meta-data can go into the module source code and -- for example -- the module_obj() annotations can go next to the TypeInfo struct for the object class.

Patches 1-3 put the infrastructure in place: Add the annotation macros, add a script to collect the meta-data, add a script to compile the meta-data into C source code which we can then add to qemu.

Patch 4 - check module dependencies (Jose, new in v4).

Patches 5-13 add annotations macros to the modules we have.

Patches 14-16 put the modinfo database into use and remove the module_deps[] and qom_modules[] lists.

Patch 16 adds two tracepoints for easier trouble-shooting.

Patches 18-20 add support for target-specific modules.

Patches 21-24 add documentation for all of the above (new in v4, was separate series).

Patches 25-29 start building accelerators modular. So far it is only qtest (all archs) and a small fraction of tcg (x86 only).

Patches 30-34 add support for registering hmp commands so they can be implemented as module (new in v4, was separate series).

take care,
Gerd

Gerd Hoffmann (33):
modules: add modinfo macros
modules: collect module meta-data
modules: generate modinfo.c
modules: add qxl module annotations
modules: add virtio-gpu module annotations
modules: add chardev module annotations
modules: add audio module annotations
modules: add usb-redir module annotations
modules: add ccid module annotations
modules: add ui module annotations
modules: add s390x module annotations
modules: add block module annotations
modules: use modinfo for dependencies
modules: use modinfo for qom load
modules: use modinfo for qemu opts load
modules: add tracepoints
modules: check arch and block load on mismatch
modules: check arch on qom lookup
modules: target-specific module build infrastructure
modules: add documentation for module sourcesets
modules: add module_obj() note to QOM docs
modules: module.h kerneldoc annotations
modules: hook up modules.h to docs build
accel: autoload modules
accel: add qtest module annotations
accel: build qtest modular
accel: add tcg module annotations
accel: build tcg modular
monitor: allow register hmp commands
usb: drop usb_host_dev_is_scsi_storage hook monitor/usb: register 'info usbhost' dynamically
usb: build usb-host as module
monitor/tcg: move tcg hmp commands to accel/tcg, register them
dynamically

Jose R. Ziviani (1):
modules: check if all dependencies can be satisfied

 scripts/modinfo-collect.py      |  67 +++++++++++
 scripts/modinfo-generate.py     |  97 ++++++++++++++++
 include/hw/usb.h                |   7 +-
 include/monitor/monitor.h       |   3 +
 include/qemu/module.h           |  74 ++++++++++++
 accel/accel-common.c            |   2 +-
 accel/accel-softmmu.c           |   2 +-
 accel/qtest/qtest.c             |   2 +
 accel/tcg/hmp.c                 |  29 +++++
 accel/tcg/tcg-accel-ops.c       |   1 +
 accel/tcg/tcg-all.c             |   1 +
 audio/spiceaudio.c              |   2 +
 block/iscsi-opts.c              |   1 +
 chardev/baum.c                  |   1 +
 chardev/spice.c                 |   4 +
 hw/display/qxl.c                |   4 +
 hw/display/vhost-user-gpu-pci.c |   1 +
 hw/display/vhost-user-gpu.c     |   1 +
 hw/display/vhost-user-vga.c     |   1 +
 hw/display/virtio-gpu-base.c    |   1 +
 hw/display/virtio-gpu-gl.c      |   3 +
 hw/display/virtio-gpu-pci-gl.c  |   3 +
 hw/display/virtio-gpu-pci.c     |   2 +
 hw/display/virtio-gpu.c         |   1 +
 hw/display/virtio-vga-gl.c      |   3 +
 hw/display/virtio-vga.c         |   2 +
 hw/ppc/spapr.c                  |   2 +-
 hw/s390x/virtio-ccw-gpu.c       |   3 +
 hw/usb/ccid-card-emulated.c     |   1 +
 hw/usb/ccid-card-passthru.c     |   1 +
 hw/usb/dev-storage-bot.c        |   1 +
 hw/usb/dev-storage-classic.c    |   1 +
 hw/usb/dev-uas.c                |   1 +
 hw/usb/host-libusb.c            |  38 ++----
 hw/usb/host-stub.c              |  45 -------
 hw/usb/redirect.c               |   1 +
 monitor/hmp.c                   |   7 ++
 monitor/misc.c                  |  34 +++---
 softmmu/vl.c                    |  24 ++--
 stubs/module-opts.c             |   4 -
 ui/egl-headless.c               |   4 +
 ui/gtk.c                        |   4 +
 ui/sdl2.c                       |   4 +
 ui/spice-app.c                  |   3 +
 ui/spice-core.c                 |   5 +
 util/module.c                   | 200 ++++++++++++++++++--------------
 accel/qtest/meson.build         |   8 +-
 accel/tcg/meson.build           |   6 +-
 docs/devel/build-system.rst     |  17 +++
 docs/devel/index.rst            |   1 +
 docs/devel/modules.rst          |   5 +
 docs/devel/qom.rst              |   8 ++
 hmp-commands-info.hx            |   3 -
 hw/usb/meson.build              |  10 +-
 meson.build                     |  82 +++++++++++++
 util/trace-events               |   4 +
 56 files changed, 624 insertions(+), 218 deletions(-)
 create mode 100755 scripts/modinfo-collect.py
 create mode 100755 scripts/modinfo-generate.py
 create mode 100644 accel/tcg/hmp.c
 delete mode 100644 hw/usb/host-stub.c
 create mode 100644 docs/devel/modules.rst
Signature · 1 line
2.31.1
#43 lore raw
In reply to Gerd Hoffmann
Quoted 7 lines · * Gerd Hoffmann ([email protected]) wrote:

This patch series adds support for module meta-data. Today this is either hard-coded in qemu (see qemu_load_module_for_opts) or handled with manually maintained lists in util/module (see module_deps[] and qom_modules[]). This series replaced that scheme with annotation macros, so the meta-data can go into the module source code and -- for example -- the module_obj() annotations can go next to the TypeInfo struct for the object class.

So this is slightly off-topic for the series; but kind of relevant, but...
Is there a way to inhibit module loading after a given point?

I ask, because there's a fairly well known security escalation that takes advantage of NSS loading of PAM modules; typically you have your nice sandboxed application, you write out your nasty .so into the sandbox and then somehow get your application to trigger the PAM module load.
Now, what stops the same attack here?

Dave

Quoted 134 lines

Patches 1-3 put the infrastructure in place: Add the annotation macros, add a script to collect the meta-data, add a script to compile the meta-data into C source code which we can then add to qemu.

Patch 4 - check module dependencies (Jose, new in v4).

Patches 5-13 add annotations macros to the modules we have.

Patches 14-16 put the modinfo database into use and remove the module_deps[] and qom_modules[] lists.

Patch 16 adds two tracepoints for easier trouble-shooting.

Patches 18-20 add support for target-specific modules.

Patches 21-24 add documentation for all of the above (new in v4, was separate series).

Patches 25-29 start building accelerators modular. So far it is only qtest (all archs) and a small fraction of tcg (x86 only).

Patches 30-34 add support for registering hmp commands so they can be implemented as module (new in v4, was separate series).

take care,
Gerd

Gerd Hoffmann (33):
modules: add modinfo macros
modules: collect module meta-data
modules: generate modinfo.c
modules: add qxl module annotations
modules: add virtio-gpu module annotations
modules: add chardev module annotations
modules: add audio module annotations
modules: add usb-redir module annotations
modules: add ccid module annotations
modules: add ui module annotations
modules: add s390x module annotations
modules: add block module annotations
modules: use modinfo for dependencies
modules: use modinfo for qom load
modules: use modinfo for qemu opts load
modules: add tracepoints
modules: check arch and block load on mismatch
modules: check arch on qom lookup
modules: target-specific module build infrastructure
modules: add documentation for module sourcesets
modules: add module_obj() note to QOM docs
modules: module.h kerneldoc annotations
modules: hook up modules.h to docs build
accel: autoload modules
accel: add qtest module annotations
accel: build qtest modular
accel: add tcg module annotations
accel: build tcg modular
monitor: allow register hmp commands
usb: drop usb_host_dev_is_scsi_storage hook monitor/usb: register 'info usbhost' dynamically
usb: build usb-host as module
monitor/tcg: move tcg hmp commands to accel/tcg, register them
dynamically

Jose R. Ziviani (1):
modules: check if all dependencies can be satisfied

 scripts/modinfo-collect.py      |  67 +++++++++++
 scripts/modinfo-generate.py     |  97 ++++++++++++++++
 include/hw/usb.h                |   7 +-
 include/monitor/monitor.h       |   3 +
 include/qemu/module.h           |  74 ++++++++++++
 accel/accel-common.c            |   2 +-
 accel/accel-softmmu.c           |   2 +-
 accel/qtest/qtest.c             |   2 +
 accel/tcg/hmp.c                 |  29 +++++
 accel/tcg/tcg-accel-ops.c       |   1 +
 accel/tcg/tcg-all.c             |   1 +
 audio/spiceaudio.c              |   2 +
 block/iscsi-opts.c              |   1 +
 chardev/baum.c                  |   1 +
 chardev/spice.c                 |   4 +
 hw/display/qxl.c                |   4 +
 hw/display/vhost-user-gpu-pci.c |   1 +
 hw/display/vhost-user-gpu.c     |   1 +
 hw/display/vhost-user-vga.c     |   1 +
 hw/display/virtio-gpu-base.c    |   1 +
 hw/display/virtio-gpu-gl.c      |   3 +
 hw/display/virtio-gpu-pci-gl.c  |   3 +
 hw/display/virtio-gpu-pci.c     |   2 +
 hw/display/virtio-gpu.c         |   1 +
 hw/display/virtio-vga-gl.c      |   3 +
 hw/display/virtio-vga.c         |   2 +
 hw/ppc/spapr.c                  |   2 +-
 hw/s390x/virtio-ccw-gpu.c       |   3 +
 hw/usb/ccid-card-emulated.c     |   1 +
 hw/usb/ccid-card-passthru.c     |   1 +
 hw/usb/dev-storage-bot.c        |   1 +
 hw/usb/dev-storage-classic.c    |   1 +
 hw/usb/dev-uas.c                |   1 +
 hw/usb/host-libusb.c            |  38 ++----
 hw/usb/host-stub.c              |  45 -------
 hw/usb/redirect.c               |   1 +
 monitor/hmp.c                   |   7 ++
 monitor/misc.c                  |  34 +++---
 softmmu/vl.c                    |  24 ++--
 stubs/module-opts.c             |   4 -
 ui/egl-headless.c               |   4 +
 ui/gtk.c                        |   4 +
 ui/sdl2.c                       |   4 +
 ui/spice-app.c                  |   3 +
 ui/spice-core.c                 |   5 +
 util/module.c                   | 200 ++++++++++++++++++--------------
 accel/qtest/meson.build         |   8 +-
 accel/tcg/meson.build           |   6 +-
 docs/devel/build-system.rst     |  17 +++
 docs/devel/index.rst            |   1 +
 docs/devel/modules.rst          |   5 +
 docs/devel/qom.rst              |   8 ++
 hmp-commands-info.hx            |   3 -
 hw/usb/meson.build              |  10 +-
 meson.build                     |  82 +++++++++++++
 util/trace-events               |   4 +
 56 files changed, 624 insertions(+), 218 deletions(-)
 create mode 100755 scripts/modinfo-collect.py
 create mode 100755 scripts/modinfo-generate.py
 create mode 100644 accel/tcg/hmp.c
 delete mode 100644 hw/usb/host-stub.c
 create mode 100644 docs/devel/modules.rst
Signature · 1 line
2.31.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#44 lore raw
Quoted 9 lines · Dr. David Alan Gilbert
Quoted 7 lines · * Gerd Hoffmann ([email protected]) wrote:

This patch series adds support for module meta-data. Today this is either hard-coded in qemu (see qemu_load_module_for_opts) or handled with manually maintained lists in util/module (see module_deps[] and qom_modules[]). This series replaced that scheme with annotation macros, so the meta-data can go into the module source code and -- for example -- the module_obj() annotations can go next to the TypeInfo struct for the object class.

Quoted 3 lines

So this is slightly off-topic for the series; but kind of relevant, but...
Is there a way to inhibit module loading after a given point?

We could block loading after machine initialization. Has implications for hotplug though.

Quoted 6 lines

I ask, because there's a fairly well known security escalation that takes advantage of NSS loading of PAM modules; typically you have your nice sandboxed application, you write out your nasty .so into the sandbox and then somehow get your application to trigger the PAM module load.
Now, what stops the same attack here?

Placing a new .so at some random directory wouldn't work, qemu only loads modules from the search path (but I guess the same is true for pam).

With this patch series applied all modules are listed the in modinfo.c database (even if we don't have any metadata about them), so we could easily limit loading to modules known at compile time. Not sure how much that alone would improve security though, when the attacker is able to write to the qemu module directory it isn't much of a problem to just overwrite one of the existing modules.

We could try work with hashes or signatures stored in modinfo ...

take care,
Gerd

#45 lore raw
Quoted 10 lines · * Gerd Hoffmann ([email protected]) wrote:
Quoted 9 lines · Dr. David Alan Gilbert
Quoted 7 lines · * Gerd Hoffmann ([email protected]) wrote:

This patch series adds support for module meta-data. Today this is either hard-coded in qemu (see qemu_load_module_for_opts) or handled with manually maintained lists in util/module (see module_deps[] and qom_modules[]). This series replaced that scheme with annotation macros, so the meta-data can go into the module source code and -- for example -- the module_obj() annotations can go next to the TypeInfo struct for the object class.

Quoted 6 lines
Quoted 3 lines

So this is slightly off-topic for the series; but kind of relevant, but...
Is there a way to inhibit module loading after a given point?

We could block loading after machine initialization. Has implications for hotplug though.

Yes; I was thinking perhaps a command to explicitly disable autoloading if people worried about it.

Quoted 10 lines
Quoted 6 lines

I ask, because there's a fairly well known security escalation that takes advantage of NSS loading of PAM modules; typically you have your nice sandboxed application, you write out your nasty .so into the sandbox and then somehow get your application to trigger the PAM module load.
Now, what stops the same attack here?

Placing a new .so at some random directory wouldn't work, qemu only loads modules from the search path (but I guess the same is true for pam).

Yes, I'm failing to find the CVE I vaguely remember about the details of how it was messed up.

Dave

Quoted 12 lines

With this patch series applied all modules are listed the in modinfo.c database (even if we don't have any metadata about them), so we could easily limit loading to modules known at compile time. Not sure how much that alone would improve security though, when the attacker is able to write to the qemu module directory it isn't much of a problem to just overwrite one of the existing modules.

We could try work with hashes or signatures stored in modinfo ...

take care,
Gerd

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK