modules: add metadata database
Subsystems (2)
v2 cover modules: add metadata database
This patch series adds support for module metadata. Here are the pieces of the puzzle:
(1) Macros are added to store metadata in a .modinfo elf section (idea stolen from the linux kernel).
(2) A utility to scan modules, collect metadata from the .modinfo sections, store it in a file (modinfo.json) for later consumption by qemu. Can also be easily inspected using 'jq'.
(3) Adding annotations to the modules we have.
(4) Drop hard-coded lists from utils/module.c
take care,
Gerd
Gerd Hoffmann (18):
modules: add metadata macros, add qxl module annotations
qapi: add ModuleInfo schema
modules: add qemu-modinfo utility
modules: add virtio-gpu module annotations
modules: add chardev module annotations
modules: add audio module annotations
modules: add usb-redir module annotations
modules: add ccid module annotations
modules: add ui module annotations
modules: add s390x module annotations
modules: add block module annotations
modules: add module_load_path_init helper
modules: load modinfo.json
modules: use modinfo for dependencies
modules: use modinfo for qom load
modules: use modinfo for qemu opts load
modules: check arch and block load on mismatch [fixup] module_load_modinfo
include/qemu/module.h | 23 +++ audio/spiceaudio.c | 2 + block/iscsi-opts.c | 1 + chardev/baum.c | 1 + chardev/spice.c | 4 + hw/display/qxl.c | 4 + hw/display/vhost-user-gpu-pci.c | 1 + hw/display/vhost-user-gpu.c | 1 + hw/display/vhost-user-vga.c | 1 + hw/display/virtio-gpu-base.c | 1 + hw/display/virtio-gpu-gl.c | 3 + hw/display/virtio-gpu-pci-gl.c | 3 + hw/display/virtio-gpu-pci.c | 2 + hw/display/virtio-gpu.c | 1 + hw/display/virtio-vga-gl.c | 3 + hw/display/virtio-vga.c | 2 + hw/s390x/virtio-ccw-gpu.c | 3 + hw/usb/ccid-card-emulated.c | 1 + hw/usb/ccid-card-passthru.c | 1 + hw/usb/redirect.c | 1 + qemu-modinfo.c | 270 ++++++++++++++++++++++++++++++ softmmu/vl.c | 20 +-- stubs/module-opts.c | 4 - ui/egl-headless.c | 4 + ui/gtk.c | 4 + ui/sdl2.c | 4 + ui/spice-app.c | 3 + ui/spice-core.c | 5 + util/module.c | 282 +++++++++++++++++++------------- meson.build | 11 ++ qapi/meson.build | 1 + qapi/modules.json | 36 ++++ qapi/qapi-schema.json | 1 + util/trace-events | 3 + 34 files changed, 576 insertions(+), 131 deletions(-) create mode 100644 qemu-modinfo.c create mode 100644 qapi/modules.json
Signature · 1 line
2.31.1
PATCH v2 1/18 modules: add metadata macros, add qxl module annotations
Stealing an idea from the linux kernel: Place module metadata in an .modinfo elf section. This patch adds macros and qxl module annotations as example.
--- include/qemu/module.h | 22 ++++++++++++++++++++++ hw/display/qxl.c | 4 ++++ 2 files changed, 26 insertions(+)
include/qemu/module.h +22 -0
@@ -73,4 +73,26 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail);void module_load_qom_one (const char *type);void module_load_qom_all (void ); +/* +* macros to store module metadata in a .modinfo section. +* qemu-modinfo utility will collect the metadata. +* +* Use "objdump -t -s -j .modinfo ${module}.so" to inspect. +*/ + +#define ___PASTE (a, b) a##b +#define __PASTE (a, b) ___PASTE(a, b) + +#define modinfo (kind, value) \ + static const char __PASTE(kind, __LINE__)[] \ + __attribute__((__used__)) \ + __attribute__((section(".modinfo"))) \ + __attribute__((aligned(1))) \ + = stringify(kind) "=" value + +#define module_obj (name) modinfo(obj, name) +#define module_dep (name) modinfo(dep, name) +#define module_arch (name) modinfo(arch, name) +#define module_opts (name) modinfo(opts, name) +#endif
hw/display/qxl.c +4 -0
@@ -2522,6 +2522,7 @@ static const TypeInfo qxl_primary_info = { .parent = TYPE_PCI_QXL, .class_init = qxl_primary_class_init, }; +module_obj ("qxl-vga" );static void qxl_secondary_class_init (ObjectClass *klass,void *data) {
@@ -2538,6 +2539,7 @@ static const TypeInfo qxl_secondary_info = { .parent = TYPE_PCI_QXL, .class_init = qxl_secondary_class_init, }; +module_obj ("qxl" );static void qxl_register_types (void ) {
@@ -2547,3 +2549,5 @@ static void qxl_register_types(void) } type_init(qxl_register_types) + +module_dep("ui-spice-core");
Signature · 1 line
2.31.1
PATCH v2 2/18 qapi: add ModuleInfo schema
In reply to Gerd HoffmannAdd QAPI schema for the module info database.
--- qapi/meson.build | 1 + qapi/modules.json | 36 ++++++++++++++++++++++++++++++++++++ qapi/qapi-schema.json | 1 + 3 files changed, 38 insertions(+) create mode 100644 qapi/modules.json
qapi/meson.build +1 -0
@@ -36,6 +36,7 @@ qapi_all_modules = [ 'migration', 'misc', 'misc-target', + 'modules', 'net', 'pragma', 'qom',
qapi/modules.json new file +36 -0
@@ -0,0 +1,36 @@ +# -*- Mode: Python -*- +# vim: filetype=python + +## +# @ModuleInfo: +# +# qemu module metadata +# +# @name: module name +# +# @objs: list of qom objects implemented by the module. +# +# @deps: list of other modules this module depends on. +# +# @arch: module architecture. +# +# @opts: qemu opts implemented by module. +# +# Since: 6.1 +## +{ 'struct': 'ModuleInfo', + 'data': { 'name' : 'str', + '*objs' : ['str'], + '*deps' : ['str'], + '*arch' : 'str', + '*opts' : 'str'}} + +## +# @Modules: +# +# qemu module list +# +# Since: 6.1 +## +{ 'struct': 'Modules', + 'data': { 'list' : ['ModuleInfo']}}
qapi/qapi-schema.json +1 -0
@@ -93,3 +93,4 @@ { 'include': 'audio.json' } { 'include': 'acpi.json' } { 'include': 'pci.json' } +{ 'include': 'modules.json' }
Signature · 1 line
2.31.1
Re: [PATCH v2 02/18] qapi: add ModuleInfo schema
Quoted 28 lines · Gerd Hoffmann
Add QAPI schema for the module info database.
Signed-off-by Gerd Hoffmann <[email protected]>--- qapi/meson.build | 1 + qapi/modules.json | 36 ++++++++++++++++++++++++++++++++++++ qapi/qapi-schema.json | 1 + 3 files changed, 38 insertions(+) create mode 100644 qapi/modules.jsonqapi/meson.build +1
-0@@ -36,6 +36,7 @@ qapi_all_modules = [ 'migration', 'misc', 'misc-target', + 'modules', 'net', 'pragma', 'qom',qapi/modules.json new file +0
-0
qapi/modules.json line 3
Quoted 3 lines
qapi/modules.json new file +3
-0@@ -0,0 +1,36 @@ +# -*- Mode: Python -*- +# vim: filetype=python +
Missing a section heading like
##
# = Loadable modules
##
Without it, the contents gets appended to whatever section precedes it in qapi-schema.json, which is almost certainly not what you want.
qapi/modules.json line 7
Quoted 4 lines
qapi/modules.json new file +4
-0+## +# @ModuleInfo: +# +# qemu module metadata
It's spelled QEMU :)
Suggest "Loadable module meta-data".
qapi/modules.json line 11
Quoted 4 lines
qapi/modules.json new file +4
-0+# +# @name: module name +# +# @objs: list of qom objects implemented by the module.
s/qom objects/QOM types/
qapi/modules.json line 13
Quoted 2 lines
qapi/modules.json new file +2
-0+# +# @deps: list of other modules this module depends on.
Suggest to spell out that these are @name of other loadable modules.
qapi/modules.json line 15
Quoted 2 lines
qapi/modules.json new file +2
-0+# +# @arch: module architecture.
Semantics?
Should this be enum SysEmuTarget?
qapi/modules.json line 17
Quoted 2 lines
qapi/modules.json new file +2
-0+# +# @opts: qemu opts implemented by module.
Is this the name of a QemuOptsList?
Since this isn't a list, a module can only ever provide one QemuOptsList. Sure that's okay?
Quoted 9 lines
qapi/modules.json new file +9
-0+# +# Since: 6.1 +## +{ 'struct': 'ModuleInfo', + 'data': { 'name' : 'str', + '*objs' : ['str'], + '*deps' : ['str'], + '*arch' : 'str', + '*opts' : 'str'}}
qapi/modules.json line 36
Quoted 10 lines
qapi/modules.json new file +10
-0+ +## +# @Modules: +# +# qemu module list +# +# Since: 6.1 +## +{ 'struct': 'Modules', + 'data': { 'list' : ['ModuleInfo']}}
This defines only types, no QMP commands or events. Why do you need the types to be QAPI types?
qapi/qapi-schema.json line 96
Quoted 9 lines
qapi/qapi-schema.json +1
-0@@ -93,3 +93,4 @@ { 'include': 'audio.json' } { 'include': 'acpi.json' } { 'include': 'pci.json' } +{ 'include': 'modules.json' }
Is this the place you want the section to be? Remember, generated documentation follows source order.
Re: [PATCH v2 02/18] qapi: add ModuleInfo schema
Hi,
Quoted 5 lines
Quoted 1 line
+# @arch: module architecture.
Semantics?
Should this be enum SysEmuTarget?
Probably, will check ...
Quoted 6 lines
Quoted 1 line
+# @opts: qemu opts implemented by module.
Is this the name of a QemuOptsList?
Since this isn't a list, a module can only ever provide one QemuOptsList. Sure that's okay?
For the current two in-tree cases yes, and I don't expect this to change in the future. We could turn this into a list though to make it future-proof.
Quoted 5 lines
qapi/modules.json line 36
Quoted 2 lines
qapi/modules.json new file +2
-0+{ 'struct': 'Modules', + 'data': { 'list' : ['ModuleInfo']}}This defines only types, no QMP commands or events. Why do you need the types to be QAPI types?
Want re-use the code to serialize/parse json from/to structs. (see patches #3 + #13).
Quoted 10 lines
qapi/qapi-schema.json line 96
Quoted 7 lines
qapi/qapi-schema.json +1
-0@@ -93,3 +93,4 @@ { 'include': 'audio.json' } { 'include': 'acpi.json' } { 'include': 'pci.json' } +{ 'include': 'modules.json' }Is this the place you want the section to be? Remember, generated documentation follows source order.
Ah, *this* the ordering is important for. I'll check, was just appending to the end of the list ...
thanks,
Gerd
Re: [PATCH v2 02/18] qapi: add ModuleInfo schema
Quoted 12 lines · Gerd Hoffmann
Hi,
Quoted 5 lines
Quoted 1 line
+# @arch: module architecture.
Semantics?
Should this be enum SysEmuTarget?
Probably, will check ...
Quoted 2 lines
Quoted 1 line
+# @opts: qemu opts implemented by module.
Quoted 8 lines
Quoted 4 lines
Is this the name of a QemuOptsList?
Since this isn't a list, a module can only ever provide one QemuOptsList. Sure that's okay?
For the current two in-tree cases yes, and I don't expect this to change in the future. We could turn this into a list though to make it future-proof.
If it's not much of a bother, then why not?
Quoted 8 lines
Quoted 5 lines
qapi/modules.json line 36
Quoted 2 lines
qapi/modules.json new file +2
-0+{ 'struct': 'Modules', + 'data': { 'list' : ['ModuleInfo']}}This defines only types, no QMP commands or events. Why do you need the types to be QAPI types?
Want re-use the code to serialize/parse json from/to structs. (see patches #3 + #13).
Okay, that's fair.
[...]
PATCH v2 3/18 modules: add qemu-modinfo utility
In reply to Gerd HoffmannScan .modinfo sections of qemu modules,
write module metadata to modinfo.json.
--- qemu-modinfo.c | 270 +++++++++++++++++++++++++++++++++++++++++++++++++ meson.build | 11 ++ 2 files changed, 281 insertions(+) create mode 100644 qemu-modinfo.c
qemu-modinfo.c new file +270 -0
@@ -0,0 +1,270 @@ +/* +* QEMU module parser +* +* read modules, find modinfo section, parse & store metadata. +* +* Copyright Red Hat, Inc. 2021 +* +* Authors: +* Gerd Hoffmann <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +*/ +#include "qemu/osdep.h" +#include "elf.h" +#include <stdint.h> +#include <dirent.h> + +#include "qapi/qapi-types-modules.h" +#include "qapi/qapi-visit-modules.h" +#include "qapi/qobject-output-visitor.h" +#include "qapi/qmp/qjson.h" +#include "qapi/qmp/qstring.h" + +#if INTPTR_MAX ==INT32_MAX +# define Elf_Ehdr Elf32_Ehdr +# define Elf_Shdr Elf32_Shdr +# define ELFCLASS ELFCLASS32 +#elif INTPTR_MAX ==INT64_MAX +# define Elf_Ehdr Elf64_Ehdr +# define Elf_Shdr Elf64_Shdr +# define ELFCLASS ELFCLASS64 +#else +# error Huh? Neither 32-bit nor 64-bit host. +#endif + +static const char *moddir =CONFIG_QEMU_MODDIR ; +static const char *dsosuf =CONFIG_HOST_DSOSUF ; + +static ModuleInfo *modinfo (const char *module,char *info,size_t size) +{ +ModuleInfo *modinfo; +strList *sl; +size_t pos =0 , len; + + modinfo =g_new0 (ModuleInfo,1 ); + modinfo->name =g_strdup (module); + +if (info) { +do { +if (strncmp (info + pos,"obj=" ,4 ) ==0 ) { + sl =g_new0 (strList,1 ); + sl->value =g_strdup (info + pos +4 ); + sl->next = modinfo->objs; + modinfo->objs = sl; + modinfo->has_objs = true; + }else if (strncmp (info + pos,"dep=" ,4 ) ==0 ) { + sl =g_new0 (strList,1 ); + sl->value =g_strdup (info + pos +4 ); + sl->next = modinfo->deps; + modinfo->deps = sl; + modinfo->has_deps = true; + }else if (strncmp (info + pos,"arch=" ,5 ) ==0 ) { + modinfo->arch =g_strdup (info + pos +5 ); + modinfo->has_arch = true; + }else if (strncmp (info + pos,"opts=" ,5 ) ==0 ) { + modinfo->opts =g_strdup (info + pos +5 ); + modinfo->has_opts = true; + }else { +fprintf (stderr,"unknown tag: %s\n" , info + pos); +exit (1 ); + } + len =strlen (info + pos) +1 ; + pos += len; + }while (pos < size); + } + +return modinfo; +} + +static void elf_read_section_hdr (FILE *fp,Elf_Ehdr *ehdr, +int section,Elf_Shdr *shdr) +{ +size_t pos, len; +int ret; + + pos = ehdr->e_shoff + section * ehdr->e_shentsize; + len =MIN (ehdr->e_shentsize,sizeof (*shdr)); + + ret =fseek (fp, pos,SEEK_SET ); +if (ret !=0 ) { +fprintf (stderr,"seek error\n" ); +exit (1 ); + } + +memset (shdr,0 ,sizeof (*shdr)); + ret =fread (shdr, len,1 , fp); +if (ret !=1 ) { +fprintf (stderr,"read error\n" ); +exit (1 ); + } +} + +static void *elf_read_section (FILE *fp,Elf_Ehdr *ehdr, +int section,size_t *size) +{ +Elf_Shdr shdr; +void *data; +int ret; + +elf_read_section_hdr (fp, ehdr, section, &shdr); +if (shdr.sh_offset && shdr.sh_size) { + ret =fseek (fp, shdr.sh_offset,SEEK_SET ); +if (ret !=0 ) { +fprintf (stderr,"seek error\n" ); +exit (1 ); + } + + data =g_malloc (shdr.sh_size); + ret =fread (data, shdr.sh_size,1 , fp); +if (ret !=1 ) { +fprintf (stderr,"read error\n" ); +exit (1 ); + } + *size = shdr.sh_size; + }else { + data =NULL ; + *size =0 ; + } +return data; +} + +static ModuleInfo *elf_parse_module (const char *module, +const char *filename) +{ +Elf_Ehdr ehdr; +Elf_Shdr shdr; +FILE *fp; +int ret, i; +char *str; +size_t str_size; +char *info; +size_t info_size; + + fp =fopen (filename,"r" ); +if (NULL == fp) { +fprintf (stderr,"open %s: %s\n" , filename,strerror (errno)); +exit (1 ); + } + + ret =fread (&ehdr,sizeof (ehdr),1 , fp); +if (ret !=1 ) { +fprintf (stderr,"read error (%s)\n" , filename); +exit (1 ); + } + +if (ehdr.e_ident[EI_MAG0 ] !=ELFMAG0 || + ehdr.e_ident[EI_MAG1 ] !=ELFMAG1 || + ehdr.e_ident[EI_MAG2 ] !=ELFMAG2 || + ehdr.e_ident[EI_MAG3 ] !=ELFMAG3 ) { +fprintf (stderr,"not an elf file (%s)\n" , filename); +exit (1 ); + } +if (ehdr.e_ident[EI_CLASS ] !=ELFCLASS64 ) { +fprintf (stderr,"elf class mismatch (%s)\n" , filename); +exit (1 ); + } +if (ehdr.e_shoff ==0 ) { +fprintf (stderr,"no section header (%s)\n" , filename); +exit (1 ); + } + +/* read string table */ +if (ehdr.e_shstrndx ==0 ) { +fprintf (stderr,"no section strings (%s)\n" , filename); +exit (1 ); + } + str =elf_read_section (fp, &ehdr, ehdr.e_shstrndx, &str_size); +if (NULL == str) { +fprintf (stderr,"no section strings (%s)\n" , filename); +exit (1 ); + } + +/* find and read modinfo section */ + info =NULL ; +for (i =0 ; i < ehdr.e_shnum; i++) { +elf_read_section_hdr (fp, &ehdr, i, &shdr); +if (!shdr.sh_name) { +continue ; + } +if (strcmp (str + shdr.sh_name,".modinfo" ) ==0 ) { + info =elf_read_section (fp, &ehdr, i, &info_size); + } + } +fclose (fp); + +return modinfo (module, info, info_size); +} + +int main (int argc,char **argv) +{ +DIR *dir; +FILE *fp; +ModuleInfo *modinfo; +ModuleInfoList *modlist; +Modules *modules; +Visitor *v; +QObject *obj; +Error *errp =NULL ; +struct dirent *ent; +char *ext, *file, *name; +GString *gjson; +QString *qjson; +const char *json; + +if (argc >1 ) { + moddir = argv[1 ]; + } + + dir =opendir (moddir); +if (dir ==NULL ) { +fprintf (stderr,"opendir(%s): %s\n" , moddir,strerror (errno)); +exit (1 ); + } + + modules =g_new0 (Modules,1 ); +while (NULL != (ent =readdir (dir))) { + ext =strrchr (ent->d_name,'.' ); +if (!ext) { +continue ; + } +if (strcmp (ext, dsosuf) !=0 ) { +continue ; + } + + name =g_strndup (ent->d_name, ext - ent->d_name); + file =g_strdup_printf ("%s/%s" , moddir, ent->d_name); + modinfo =elf_parse_module (name, file); +g_free (file); +g_free (name); + + modlist =g_new0 (ModuleInfoList,1 ); + modlist->value = modinfo; + modlist->next = modules->list; + modules->list = modlist; + } +closedir (dir); + + v =qobject_output_visitor_new (&obj); +visit_type_Modules (v,NULL , &modules, &errp); +visit_complete (v, &obj); +visit_free (v); + + gjson =qobject_to_json (obj); + qjson =qstring_from_gstring (gjson); + json =qstring_get_str (qjson); + + file =g_strdup_printf ("%s/modinfo.json" , moddir); + fp =fopen (file,"w" ); +if (fp ==NULL ) { +fprintf (stderr,"open(%s): %s\n" , file,strerror (errno)); +exit (1 ); + } +fprintf (fp,"%s" , json); +fclose (fp); + +printf ("%s written\n" , file); +g_free (file); +return 0 ; +}
meson.build +11 -0
@@ -2380,6 +2380,17 @@ if xkbcommon.found() dependencies: [qemuutil, xkbcommon], install: have_tools) endif +if config_host.has_key('CONFIG_MODULES') + qemu_modinfo = executable('qemu-modinfo', files('qemu-modinfo.c') + genh, + dependencies: [glib, qemuutil], install: have_tools) + custom_target('modinfo.json', + input: [ softmmu_mods, block_mods ], + output: 'modinfo.json', + install: true, + install_dir: qemu_moddir, + command: [ qemu_modinfo, '.' ]) +endif + if have_tools qemu_img = executable('qemu-img', [files('qemu-img.c'), hxdep], dependencies: [authz, block, crypto, io, qom, qemuutil], install: true)
Signature · 1 line
2.31.1
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Hi Paolo,
meson.build if xkbcommon.found() · line 2392
Quoted 10 lines
meson.build +10
-0+if config_host.has_key('CONFIG_MODULES') + qemu_modinfo = executable('qemu-modinfo', files('qemu-modinfo.c') + genh, + dependencies: [glib, qemuutil], install: have_tools) + custom_target('modinfo.json', + input: [ softmmu_mods, block_mods ], + output: 'modinfo.json', + install: true, + install_dir: qemu_moddir, + command: [ qemu_modinfo, '.' ]) +endif
I have trouble with this one. Tried to declaring the modules as "input" to make sure meson will only run qemu-modinfo when it is done building the module. But now and then I get build errors because qemu-modinfo runs in parallel to a module build and qemu-modinfo throws an read error because of that.
Any clue what is going on here?
thanks,
Gerd
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 18 lines · Gerd Hoffmann
Hi Paolo,
meson.build if xkbcommon.found() · line 2392
Quoted 10 lines
meson.build +10
-0+if config_host.has_key('CONFIG_MODULES') + qemu_modinfo = executable('qemu-modinfo', files('qemu-modinfo.c') + genh, + dependencies: [glib, qemuutil], install: have_tools) + custom_target('modinfo.json', + input: [ softmmu_mods, block_mods ], + output: 'modinfo.json', + install: true, + install_dir: qemu_moddir, + command: [ qemu_modinfo, '.' ]) +endifI have trouble with this one. Tried to declaring the modules as "input" to make sure meson will only run qemu-modinfo when it is done building the module. But now and then I get build errors because qemu-modinfo runs in parallel to a module build and qemu-modinfo throws an read error because of that.
softmmu_mods and block_mods are both lists already, so this sets a nested list and I wonder if it confuses meson ? eg do you need
input: softmmu_mods + block_mods
Alternatively there is option to do:
'depends: softmmu_mods + block_mods
though the meson docs claim that's not required if they're already listed against 'input:'
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 20 lines · Daniel P. Berrangé
Quoted 18 lines · Gerd Hoffmann
Hi Paolo,
meson.build if xkbcommon.found() · line 2392
Quoted 10 lines
meson.build +10
-0+if config_host.has_key('CONFIG_MODULES') + qemu_modinfo = executable('qemu-modinfo', files('qemu-modinfo.c') + genh, + dependencies: [glib, qemuutil], install: have_tools) + custom_target('modinfo.json', + input: [ softmmu_mods, block_mods ], + output: 'modinfo.json', + install: true, + install_dir: qemu_moddir, + command: [ qemu_modinfo, '.' ]) +endifI have trouble with this one. Tried to declaring the modules as "input" to make sure meson will only run qemu-modinfo when it is done building the module. But now and then I get build errors because qemu-modinfo runs in parallel to a module build and qemu-modinfo throws an read error because of that.
Quoted 4 lines
softmmu_modsandblock_modsare both lists already, so this sets a nested list and I wonder if it confuses meson ? eg do you needinput:
softmmu_mods+block_mods
No, that should be fine. Perhaps it's because the inputs are not part of the command? You can check what the build.ninja rule for modinfo.json looks like.
Also:
- it would be better to support both directories and file names, so that stale modules are not included in modinfo.json
- modinfo.json needs to be disabled on non-ELF platforms (x86, Windows). One alternative is to use libbfd instead of including an ELF parser.
- If modinfo.json has to be installed, you need to build modinfo for the build machine in order to support cross compiling. That however would require a cross libbfd, which is a pain. Do you really need to install it? Can the functionality instead be included in the main QEMU binary with a query-modules command or something like that.
Paolo
Quoted 6 lines
Alternatively there is option to do:
'depends:
softmmu_mods+block_modsthough the meson docs claim that's not required if they're already listed against 'input:'
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 5 lines · Paolo Bonzini
- modinfo.json needs to be disabled on non-ELF platforms (x86, Windows). One alternative is to use libbfd instead of including an ELF parser.
- If modinfo.json has to be installed, you need to build modinfo for the
build machine in order to support cross compiling. That however would require a cross libbfd, which is a pain. Do you really need to install it? Can the functionality instead be included in the main QEMU binary with a query-modules command or something like that.
Another possibility to eschew .o parsing is to add something like this to the sources
#ifdef QEMU_MODINFO
#define MODULE_METADATA(key, value) \
=<>= MODINFO key value
#else
#define MODULE_METADATA(key, value)
#endif
MODULE_METADATA("opts", "spice")
A Python script could parse compile_commands.json, add -E -DQEMU_MODINFO to the command-line option, and look in the output for the metadata.
Paolo
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Hi,
Quoted 10 lines
Another possibility to eschew .o parsing is to add something like this to the sources
#ifdef
QEMU_MODINFO
#defineMODULE_METADATA(key, value)\
=<>= MODINFO key value
#else
#defineMODULE_METADATA(key, value)
#endif
Quoted 4 lines
MODULE_METADATA("opts", "spice")A Python script could parse compile_commands.json, add -E -DQEMU_MODINFO to the command-line option, and look in the output for the metadata.
Hmm, worth trying, although I guess it would be easier to code this up straight in meson.build and pull the information you need out of the source set, especially as you'll know then which source files are compiled into which module.
take care,
Gerd
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 14 lines · Gerd Hoffmann
Hi,
Quoted 12 lines
Another possibility to eschew .o parsing is to add something like this to the sources
#ifdef
QEMU_MODINFO
#defineMODULE_METADATA(key, value)\
=<>= MODINFO key value
#else
#defineMODULE_METADATA(key, value)
#endif
MODULE_METADATA("opts", "spice")
Quoted 7 lines
Quoted 2 lines
A Python script could parse compile_commands.json, add -E -DQEMU_MODINFO to the command-line option, and look in the output for the metadata.
Hmm, worth trying, although I guess it would be easier to code this up straight in meson.build and pull the information you need out of the source set, especially as you'll know then which source files are compiled into which module.
Hmm, looks like I actually need both. Seems there is no easy way to get the cflags out of a source_set to construct a cpp command line. Pulling this out of compile_commands.json with jq works though.
With the patch below I get nice ${module}.modinfo files with the metadata, now I only need a (probably python) script to collect them and create a modinfo.c which we can link into qemu.
take care,
Gerd
From 3edc033935d2dd4ec607ac6395548a327151ad06 Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann <[email protected]>
Date: Tue, 15 Jun 2021 09:23:52 +0200
Subject: [PATCH] try -DQEMU_MODINFO
--- include/qemu/module.h | 22 ++++++---------------- meson.build | 7 +++++++ scripts/modinfo.sh | 8 ++++++++ 3 files changed, 21 insertions(+), 16 deletions(-) create mode 100644 scripts/modinfo.sh
include/qemu/module.h +6 -16
@@ -74,22 +74,12 @@ void module_load_qom_one(const char *type);void module_load_qom_all (void );void module_allow_arch (const char *arch); -/* -* macros to store module metadata in a .modinfo section. -* qemu-modinfo utility will collect the metadata. -* -* Use "objdump -t -s -j .modinfo ${module}.so" to inspect. -*/ - -#define ___PASTE (a, b) a##b -#define __PASTE (a, b) ___PASTE(a, b) - -#define modinfo (kind, value) \ - static const char __PASTE(kind, __LINE__)[] \ - __attribute__((__used__)) \ - __attribute__((section(".modinfo"))) \ - __attribute__((aligned(1))) \ - = stringify(kind) "=" value +#ifdef QEMU_MODINFO +# define modinfo (kind, value) \ + MODINFO_START kind value MODINFO_END +#else +# define modinfo (kind, value) +#endif #define module_obj (name) modinfo(obj, name)#define module_dep (name) modinfo(dep, name)
meson.build +7 -0
@@ -2050,12 +2050,19 @@ target_modules += { 'accel' : { 'qtest': qtest_module_ss, block_mods = [] softmmu_mods = [] +modinfo = find_program('scripts/modinfo.sh') foreach d, list : modules foreach m, module_ss : list if enable_modules and targetos != 'windows' module_ss = module_ss.apply(config_all, strict: false) sl = static_library(d + '-' + m, [genh, module_ss.sources()], dependencies: [modulecommon, module_ss.dependencies()], pic: true) + custom_target(d + '-' + m + '.modinfo', + output: d + '-' + m + '.modinfo', + input: module_ss.sources(), + build_by_default: true, # to be removed when added to a target + capture: true, + command: [modinfo, '@INPUT@']) if d == 'block' block_mods += sl else
scripts/modinfo.sh new file +8 -0
@@ -0,0 +1,8 @@ +#!/bin/sh +for input in "$@"; do + command=$(jq -r ".[] | select(.file == \"$input\") | .command " compile_commands.json) + command="${command%% -M*}" + command="$command -DQEMU_MODINFO -E $input" + $command +done | grep MODINFO +exit 0
Signature · 1 line
2.31.1
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 11 lines
Quoted 7 lines
Quoted 2 lines
A Python script could parse compile_commands.json, add -E -DQEMU_MODINFO to the command-line option, and look in the output for the metadata.
Hmm, worth trying, although I guess it would be easier to code this up straight in meson.build and pull the information you need out of the source set, especially as you'll know then which source files are compiled into which module.
Hmm, looks like I actually need both. Seems there is no easy way to get the cflags out of a
source_setto construct a cpp command line. Pulling this out of compile_commands.json with jq works though.
Well, easy until I look at target-specific modules where the "source file" -> "command" mapping isn't unique any more. Which makes this route less attractive ...
Any idea on getting the cflags in meson.build Or maybe I can somehow ask meson to run the cpp pass only for a given source set?
thanks,
Gerd
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 6 lines · Gerd Hoffmann
Quoted 3 lines
Hmm, looks like I actually need both. Seems there is no easy way to get the cflags out of a
source_setto construct a cpp command line. Pulling this out of compile_commands.json with jq works though.Well, easy until I look at target-specific modules where the "source file" -> "command" mapping isn't unique any more. Which makes this route less attractive ...
I was almost giving up... but it looks like the result of extract_all_objects(recursive: true) can be passed to custom_target(). Then you can match it after compile_commands.json's "output" key.
Paolo
Quoted 2 lines
Any idea on getting the cflags in meson.build Or maybe I can somehow ask meson to run the cpp pass only for a given source set?
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · Gerd Hoffmann
Quoted 3 lines
Hmm, looks like I actually need both. Seems there is no easy way to get the cflags out of a
source_setto construct a cpp command line. Pulling this out of compile_commands.json with jq works though.Well, easy until I look at target-specific modules where the "source file" -> "command" mapping isn't unique any more. Which makes this route less attractive ...
I was almost giving up... but it looks like the result of
extract_all_objects(recursive: true)can be passed tocustom_target(). Then you can match it after compile_commands.json's "output" key.
Seems the custom_target commands do not land in compile_commands.json.
But I have figured meanwhile that looking for the target name in the command line works reliable. That will will match -DCONFIG_TARGET="${target}-config-target.h".
Current WIP patch below, seems to work nicely. Whole patch series needs an overhaul now ...
From 70c96336e38f1a7f114aee2c7ef023546cc560e5 Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann <[email protected]>
Date: Tue, 15 Jun 2021 09:23:52 +0200
Subject: [PATCH] try -DQEMU_MODINFO
--- scripts/modinfo-collect.py | 66 +++++++++++++++++++++++++++++++++++++ scripts/modinfo-generate.py | 61 ++++++++++++++++++++++++++++++++++ include/qemu/module.h | 33 ++++++++++--------- meson.build | 28 +++++++++++++++- 4 files changed, 171 insertions(+), 17 deletions(-) create mode 100755 scripts/modinfo-collect.py create mode 100755 scripts/modinfo-generate.py
scripts/modinfo-collect.py new file +66 -0
@@ -0,0 +1,66 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +import os +import sys +import json +import shlex +import subprocess + +def find_command(src, target, compile_commands): + for command in compile_commands: + if command['file'] != src: + continue + if target != '' and command['command'].find(target) == -1: + continue + return command['command'] + return 'false' + +def process_command(src, command): + skip = False + arg = False + out = [] + for item in shlex.split(command): + if arg: + out.append(x) + arg = False + continue + if skip: + skip = False + continue + if item == '-MF' or item == '-MQ' or item == '-o': + skip = True + continue + if item == '-c': + skip = True + continue + out.append(item) + out.append('-DQEMU_MODINFO') + out.append('-E') + out.append(src) + return out + +def main(args): + target = '' + if args[0] == '--target': + args.pop(0) + target = args.pop(0) + print("MODINFO_DEBUG target %s" % target) + arch = target[:-8] # cut '-softmmu' + print("MODINFO_START arch \"%s\" MODINFO_END" % arch) + with open('compile_commands.json') as f: + compile_commands = json.load(f) + for src in args: + print("MODINFO_DEBUG src %s" % src) + command = find_command(src, target, compile_commands) + cmdline = process_command(src, command) + print("MODINFO_DEBUG cmd", cmdline) + result = subprocess.run(cmdline, capture_output = True, text = True) + if result.returncode != 0: + sys.exit(result.returncode) + for line in result.stdout.split('\n'): + if line.find('MODINFO') != -1: + print(line) + +if __name__ == "__main__": + main(sys.argv[1:])
scripts/modinfo-generate.py new file +61 -0
61 lines not on this page · raw message
include/qemu/module.h +17 -16
@@ -74,26 +74,27 @@ void module_load_qom_one(const char *type);void module_load_qom_all (void );void module_allow_arch (const char *arch); -/* -* macros to store module metadata in a .modinfo section. -* qemu-modinfo utility will collect the metadata. -* -* Use "objdump -t -s -j .modinfo ${module}.so" to inspect. -*/ - -#define ___PASTE (a, b) a##b -#define __PASTE (a, b) ___PASTE(a, b) - -#define modinfo (kind, value) \ - static const char __PASTE(kind, __LINE__)[] \ - __attribute__((__used__)) \ - __attribute__((section(".modinfo"))) \ - __attribute__((aligned(1))) \ - = stringify(kind) "=" value +/* scripts/modinfo.sh collects module info (using -DQEMU_MODINFO) */ +#ifdef QEMU_MODINFO +# define modinfo (kind, value) \ + MODINFO_START kind value MODINFO_END +#else +# define modinfo (kind, value) +#endif #define module_obj (name) modinfo(obj, name)#define module_dep (name) modinfo(dep, name)#define module_arch (name) modinfo(arch, name)#define module_opts (name) modinfo(opts, name) +typedef struct QemuModinfo QemuModinfo ; +struct QemuModinfo { +const char *name; +const char *arch; +const char **objs; +const char **deps; +const char **opts; +}; +extern const QemuModinfo qemu_modinfo[]; +#endif
meson.build +27 -1
@@ -2048,6 +2048,10 @@ target_modules += { 'accel' : { 'qtest': qtest_module_ss, # Library dependencies # ######################## +modinfo_collect = find_program('scripts/modinfo-collect.py') +modinfo_generate = find_program('scripts/modinfo-generate.py') +modinfo_files = [] + block_mods = [] softmmu_mods = [] foreach d, list : modules
@@ -2056,6 +2060,11 @@ foreach d, list : modules module_ss = module_ss.apply(config_all, strict: false) sl = static_library(d + '-' + m, [genh, module_ss.sources()], dependencies: [modulecommon, module_ss.dependencies()], pic: true) + modinfo_files += custom_target(d + '-' + m + '.modinfo', + output: d + '-' + m + '.modinfo', + input: module_ss.sources(), + capture: true, + command: [modinfo_collect, '@INPUT@']) if d == 'block' block_mods += sl else
@@ -2084,12 +2093,18 @@ foreach d, list : target_modules '-DCONFIG_DEVICES="@[email protected]"'.format(target)] target_module_ss = module_ss.apply(config_target, strict: false) if target_module_ss.sources() != [] - sl = static_library(d + '-' + m + '-' + config_target['TARGET_NAME'], + module_name = d + '-' + m + '-' + config_target['TARGET_NAME'] + sl = static_library(module_name, [genh, target_module_ss.sources()], dependencies: [modulecommon, target_module_ss.dependencies()], include_directories: target_inc, c_args: c_args, pic: true) + modinfo_files += custom_target(module_name + '.modinfo', + output: module_name + '.modinfo', + input: target_module_ss.sources(), + capture: true, + command: [modinfo_collect, '--target', target, '@INPUT@']) softmmu_mods += sl endif endif
@@ -2100,6 +2115,17 @@ foreach d, list : target_modules endforeach endforeach +if enable_modules + modinfo_src = custom_target('modinfo.c', + output: 'modinfo.c', + input: modinfo_files, + command: [modinfo_generate, '@INPUT@'], + capture: true) + modinfo_lib = static_library('modinfo', modinfo_src) + modinfo_dep = declare_dependency(link_whole: modinfo_lib) + softmmu_ss.add(modinfo_dep) +endif + nm = find_program('nm') undefsym = find_program('scripts/undefsym.py') block_syms = custom_target('block.syms', output: 'block.syms',
Signature · 1 line
2.31.1
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 5 lines · Gerd Hoffmann
Quoted 3 lines
I was almost giving up... but it looks like the result of
extract_all_objects(recursive: true)can be passed tocustom_target(). Then you can match it after compile_commands.json's "output" key.Seems the
custom_targetcommands do not land in compile_commands.json.
No, they don't.
The idea was expressed a bit too concisely. :) I was thinking of using extract_all_objects on the module static library, passing the result to modinfo-collect, and looking up the names in compile_commands.json.
Paolo
Quoted 6 lines
But I have figured meanwhile that looking for the target name in the command line works reliable. That will will match -DCONFIG_TARGET="${target}-config-target.h".
Current WIP patch below, seems to work nicely. Whole patch series needs an overhaul now ...
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
In reply to Paolo BonziniHi,
Quoted 5 lines
Quoted 4 lines
softmmu_modsandblock_modsare both lists already, so this sets a nested list and I wonder if it confuses meson ? eg do you needinput:
softmmu_mods+block_mods
Quoted 8 lines
No, that should be fine. Perhaps it's because the inputs are not part of the command? You can check what the build.ninja rule for modinfo.json looks like.
Also:
- it would be better to support both directories and file names, so that stale modules are not included in modinfo.json
When turning qemu-modinfo into a pure build-utility (see below) there is no reason to not explicitly list all module files on the command line.
Quoted 1 line
- modinfo.json needs to be disabled on non-ELF platforms (x86, Windows).
On windows modules are not supported.
Do we have any other non-ELF platforms?
Quoted 5 lines
One alternative is to use libbfd instead of including an ELF parser.
- If modinfo.json has to be installed, you need to build modinfo for the build machine in order to support cross compiling. That however would require a cross libbfd, which is a pain. Do you really need to install it?
Do you mean installing modinfo.json or installing qemu-modinfo? For the latter I can see that not installing it removes some cross-compiling headaches. And, yes, we can turn this into a pure build utility generating a static database (be it json or -- as suggested by Daniel -- a C source file with the data structures).
Quoted 2 lines
Can the functionality instead be included in the main QEMU binary with a query-modules command or something like that.
Well, the meta-data database is meant for qemu itself, not external users. I was just using json because the infrastructure to serialize + parse json exists. Not sure a "query-modinfo" command makes sense. Would be trivial to implement though if libvirt finds this useful (assuming we stick to json).
I was toying with a completely different idea: Have a "qemu -generate-modinfo". That would basically try to load each module, while doing so record the type_register() (+ other register) calls the module is doing, when done write out the database with the registrations done by each module.
Problem with that approach is that it doesn't work for module dependencies ...
Comments on the idea? Suggestions for the module dependency problem? Could maybe libbfd be used to find module (symbol) dependencies automatically without writing a full dynamic linker?
take care,
Gerd
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 16 lines · Gerd Hoffmann
Hi,
Quoted 5 lines
Quoted 4 lines
softmmu_modsandblock_modsare both lists already, so this sets a nested list and I wonder if it confuses meson ? eg do you needinput:
softmmu_mods+block_modsQuoted 8 lines
No, that should be fine. Perhaps it's because the inputs are not part of the command? You can check what the build.ninja rule for modinfo.json looks like.
Also:
- it would be better to support both directories and file names, so that stale modules are not included in modinfo.json
Quoted 6 lines
When turning qemu-modinfo into a pure build-utility (see below) there is no reason to not explicitly list all module files on the command line.
Quoted 1 line
- modinfo.json needs to be disabled on non-ELF platforms (x86, Windows).
On windows modules are not supported.
Does anyone recall why modules aren't supported on Windows
Quoted 1 line
Do we have any other non-ELF platforms?
macOS uses dynlib IIUC ?
Quoted 6 lines
Problem with that approach is that it doesn't work for module dependencies ...
Comments on the idea? Suggestions for the module dependency problem? Could maybe libbfd be used to find module (symbol) dependencies automatically without writing a full dynamic linker?
Is there any value in exploring use of libclang ? It gives us a real C parser that we can use to extract information from the C source. In libvirt we have experimental patches (not yet merged) using libclang to auto-generate XML parser helpers from struct annotations. It is quite nice compared to any other hacks for extracting information from C source files without using a proper parser. libclang can be accessed from Python3 via its bindings and IIUC should be usable on all our build platforms
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 7 lines
Quoted 6 lines
Problem with that approach is that it doesn't work for module dependencies ...
Comments on the idea? Suggestions for the module dependency problem? Could maybe libbfd be used to find module (symbol) dependencies automatically without writing a full dynamic linker?
Quoted 8 lines
Is there any value in exploring use of libclang ? It gives us a real C parser that we can use to extract information from the C source. In libvirt we have experimental patches (not yet merged) using libclang to auto-generate XML parser helpers from struct annotations. It is quite nice compared to any other hacks for extracting information from C source files without using a proper parser. libclang can be accessed from Python3 via its bindings and IIUC should be usable on all our build platforms
Could you do something along the lines of ...
(1) find constructors
(2) find type_register() calls in the constructor and the TypeInfo structs passed to those calls.
(3) inspect the TypeInfo structs to figure the QOM type names.
... with libclang?
take care,
Gerd
Re: [PATCH v2 03/18] modules: add qemu-modinfo utility
Quoted 16 lines · Gerd Hoffmann
Quoted 7 lines
Quoted 6 lines
Problem with that approach is that it doesn't work for module dependencies ...
Comments on the idea? Suggestions for the module dependency problem? Could maybe libbfd be used to find module (symbol) dependencies automatically without writing a full dynamic linker?
Quoted 8 lines
Is there any value in exploring use of libclang ? It gives us a real C parser that we can use to extract information from the C source. In libvirt we have experimental patches (not yet merged) using libclang to auto-generate XML parser helpers from struct annotations. It is quite nice compared to any other hacks for extracting information from C source files without using a proper parser. libclang can be accessed from Python3 via its bindings and IIUC should be usable on all our build platforms
Quoted 8 lines
Could you do something along the lines of ...
(1) find constructors
(2) findtype_register()calls in the constructor and the TypeInfo structs passed to those calls.
(3) inspect the TypeInfo structs to figure the QOM type names.... with libclang?
In theory that should all be doable. I'm not very familiar myself with libclang, but IIUC you basically get given the abstract syntax tree and have to traverse it to find the info you want. This is kind of low level but the info should all be there if you know how to find it.
As an answer to (1) and part of (2), the following code I hacked up quickly, finds all constructors that contain "type_register" calls. Would need to find the arg to the calls and match that up to the static structs too.
from clang.cindex import Index, CursorKind
def is_constructor(cursor):
for bit in cursor.get_children():
if bit.kind == CursorKind.UNEXPOSED_ATTR:
for tok in bit.get_tokens():
if tok.spelling == "constructor":
return True
return False
def find_constructors(cursor):
for cursor in cursor.get_children():
if cursor.kind == CursorKind.FUNCTION_DECL:
if is_constructor(cursor):
yield cursor
def has_type_register(cursor):
for cursor in constructor.get_children():
if cursor.kind == CursorKind.COMPOUND_STMT:
for c in cursor.get_children():
if c.kind == CursorKind.CALL_EXPR:
if c.displayname == "type_register":
return True
return Falseindex = Index.create()
tu = index.parse("demo.c")
for constructor in find_constructors(tu.cursor):
has_reg = has_type_register(constructor)
if has_reg:
print("Constructor with type_register: " + constructor.displayname)
I tested with a short example
#include <stdio.h>
struct Foo {
int bar;
};
static void type_register(struct Foo *foo) {
printf("%d\n", foo->bar);
}
__attribute__((constructor)) static void startit(void) {
static struct Foo foo = { 42 }; type_register(&foo);
}
int main(int argc, char **argv) {
printf("Running main\n");
}
$ python demo.py Constructor with type register: startit()
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v2 4/18 modules: add virtio-gpu module annotations
In reply to Gerd Hoffmann--- hw/display/vhost-user-gpu-pci.c | 1 + hw/display/vhost-user-gpu.c | 1 + hw/display/vhost-user-vga.c | 1 + hw/display/virtio-gpu-base.c | 1 + hw/display/virtio-gpu-gl.c | 3 +++ hw/display/virtio-gpu-pci-gl.c | 3 +++ hw/display/virtio-gpu-pci.c | 2 ++ hw/display/virtio-gpu.c | 1 + hw/display/virtio-vga-gl.c | 3 +++ hw/display/virtio-vga.c | 2 ++ 10 files changed, 18 insertions(+)
hw/display/vhost-user-gpu-pci.c +1 -0
@@ -43,6 +43,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_gpu_pci_info = { .instance_size = sizeof(VhostUserGPUPCI), .instance_init = vhost_user_gpu_pci_initfn, }; +module_obj (TYPE_VHOST_USER_GPU_PCI );static void vhost_user_gpu_pci_register_types (void ) {
hw/display/vhost-user-gpu.c +1 -0
@@ -596,6 +596,7 @@ static const TypeInfo vhost_user_gpu_info = { .instance_finalize = vhost_user_gpu_instance_finalize, .class_init = vhost_user_gpu_class_init, }; +module_obj (TYPE_VHOST_USER_GPU );static void vhost_user_gpu_register_types (void ) {
hw/display/vhost-user-vga.c +1 -0
@@ -44,6 +44,7 @@ static const VirtioPCIDeviceTypeInfo vhost_user_vga_info = { .instance_size = sizeof(VhostUserVGA), .instance_init = vhost_user_vga_inst_initfn, }; +module_obj (TYPE_VHOST_USER_VGA );static void vhost_user_vga_register_types (void ) {
hw/display/virtio-gpu-base.c +1 -0
@@ -256,6 +256,7 @@ static const TypeInfo virtio_gpu_base_info = { .class_init = virtio_gpu_base_class_init, .abstract = true }; +module_obj (TYPE_VIRTIO_GPU_BASE );static void virtio_register_types (void )
hw/display/virtio-gpu-gl.c +3 -0
@@ -154,6 +154,7 @@ static const TypeInfo virtio_gpu_gl_info = { .instance_size = sizeof(VirtIOGPUGL), .class_init = virtio_gpu_gl_class_init, }; +module_obj (TYPE_VIRTIO_GPU_GL );static void virtio_register_types (void ) {
@@ -161,3 +162,5 @@ static void virtio_register_types(void) } type_init(virtio_register_types) + +module_dep("hw-display-virtio-gpu");
hw/display/virtio-gpu-pci-gl.c +3 -0
@@ -46,6 +46,7 @@ static const VirtioPCIDeviceTypeInfo virtio_gpu_gl_pci_info = { .instance_size = sizeof(VirtIOGPUGLPCI), .instance_init = virtio_gpu_gl_initfn, }; +module_obj (TYPE_VIRTIO_GPU_GL_PCI );static void virtio_gpu_gl_pci_register_types (void ) {
@@ -53,3 +54,5 @@ static void virtio_gpu_gl_pci_register_types(void) } type_init(virtio_gpu_gl_pci_register_types) + +module_dep("hw-display-virtio-gpu-pci");
hw/display/virtio-gpu-pci.c +2 -0
@@ -64,6 +64,7 @@ static const TypeInfo virtio_gpu_pci_base_info = { .class_init = virtio_gpu_pci_base_class_init, .abstract = true }; +module_obj (TYPE_VIRTIO_GPU_PCI_BASE );#define TYPE_VIRTIO_GPU_PCI "virtio-gpu-pci"typedef struct VirtIOGPUPCI VirtIOGPUPCI ;
@@ -90,6 +91,7 @@ static const VirtioPCIDeviceTypeInfo virtio_gpu_pci_info = { .instance_size = sizeof(VirtIOGPUPCI), .instance_init = virtio_gpu_initfn, }; +module_obj (TYPE_VIRTIO_GPU_PCI );static void virtio_gpu_pci_register_types (void ) {
hw/display/virtio-gpu.c +1 -0
@@ -1419,6 +1419,7 @@ static const TypeInfo virtio_gpu_info = { .class_size = sizeof(VirtIOGPUClass), .class_init = virtio_gpu_class_init, }; +module_obj (TYPE_VIRTIO_GPU );static void virtio_register_types (void ) {
hw/display/virtio-vga-gl.c +3 -0
@@ -36,6 +36,7 @@ static VirtioPCIDeviceTypeInfo virtio_vga_gl_info = { .instance_size = sizeof(VirtIOVGAGL), .instance_init = virtio_vga_gl_inst_initfn, }; +module_obj (TYPE_VIRTIO_VGA_GL );static void virtio_vga_register_types (void ) {
@@ -45,3 +46,5 @@ static void virtio_vga_register_types(void) } type_init(virtio_vga_register_types) + +module_dep("hw-display-virtio-vga");
hw/display/virtio-vga.c +2 -0
@@ -239,6 +239,7 @@ static TypeInfo virtio_vga_base_info = { .class_init = virtio_vga_base_class_init, .abstract = true, }; +module_obj (TYPE_VIRTIO_VGA_BASE );#define TYPE_VIRTIO_VGA "virtio-vga"
@@ -268,6 +269,7 @@ static VirtioPCIDeviceTypeInfo virtio_vga_info = { .instance_size = sizeof(VirtIOVGA), .instance_init = virtio_vga_inst_initfn, }; +module_obj (TYPE_VIRTIO_VGA );static void virtio_vga_register_types (void ) {
Signature · 1 line
2.31.1
PATCH v2 5/18 modules: add chardev module annotations
In reply to Gerd Hoffmann--- chardev/baum.c | 1 + chardev/spice.c | 4 ++++ 2 files changed, 5 insertions(+)
chardev/baum.c +1 -0
@@ -680,6 +680,7 @@ static const TypeInfo char_braille_type_info = { .instance_finalize = char_braille_finalize, .class_init = char_braille_class_init, }; +module_obj (TYPE_CHARDEV_BRAILLE );static void register_types (void ) {
chardev/spice.c +4 -0
@@ -366,6 +366,7 @@ static const TypeInfo char_spice_type_info = { .class_init = char_spice_class_init, .abstract = true, }; +module_obj (TYPE_CHARDEV_SPICE );static void char_spicevmc_class_init (ObjectClass *oc,void *data) {
@@ -396,6 +397,7 @@ static const TypeInfo char_spiceport_type_info = { .parent = TYPE_CHARDEV_SPICE, .class_init = char_spiceport_class_init, }; +module_obj (TYPE_CHARDEV_SPICEPORT );static void register_types (void ) {
@@ -405,3 +407,5 @@ static void register_types(void) }type_init (register_types); + +module_dep ("ui-spice-core" );
Signature · 1 line
2.31.1
PATCH v2 6/18 modules: add audio module annotations
In reply to Gerd Hoffmann--- audio/spiceaudio.c | 2 ++ 1 file changed, 2 insertions(+)
audio/spiceaudio.c +2 -0
@@ -317,3 +317,5 @@ static void register_audio_spice(void)audio_driver_register (&spice_audio_driver); }type_init (register_audio_spice); + +module_dep ("ui-spice-core" );
Signature · 1 line
2.31.1
PATCH v2 7/18 modules: add usb-redir module annotations
In reply to Gerd Hoffmann--- hw/usb/redirect.c | 1 + 1 file changed, 1 insertion(+)
hw/usb/redirect.c +1 -0
@@ -2608,6 +2608,7 @@ static const TypeInfo usbredir_dev_info = { .class_init = usbredir_class_initfn, .instance_init = usbredir_instance_init, }; +module_obj (TYPE_USB_REDIR );static void usbredir_register_types (void ) {
Signature · 1 line
2.31.1
PATCH v2 8/18 modules: add ccid module annotations
In reply to Gerd Hoffmann--- hw/usb/ccid-card-emulated.c | 1 + hw/usb/ccid-card-passthru.c | 1 + 2 files changed, 2 insertions(+)
hw/usb/ccid-card-emulated.c +1 -0
@@ -612,6 +612,7 @@ static const TypeInfo emulated_card_info = { .instance_size = sizeof(EmulatedState), .class_init = emulated_class_initfn, }; +module_obj (TYPE_EMULATED_CCID );static void ccid_card_emulated_register_types (void ) {
hw/usb/ccid-card-passthru.c +1 -0
@@ -414,6 +414,7 @@ static const TypeInfo passthru_card_info = { .instance_size = sizeof(PassthruState), .class_init = passthru_class_initfn, }; +module_obj (TYPE_CCID_PASSTHRU );static void ccid_card_passthru_register_types (void ) {
Signature · 1 line
2.31.1
PATCH v2 9/18 modules: add ui module annotations
In reply to Gerd Hoffmann--- ui/egl-headless.c | 4 ++++ ui/gtk.c | 4 ++++ ui/sdl2.c | 4 ++++ ui/spice-app.c | 3 +++ ui/spice-core.c | 5 +++++ 5 files changed, 20 insertions(+)
ui/egl-headless.c +4 -0
@@ -213,3 +213,7 @@ static void register_egl(void) }type_init (register_egl); + +#ifdef CONFIG_OPENGL +module_dep ("ui-opengl" ); +#endif
ui/gtk.c +4 -0
@@ -2333,3 +2333,7 @@ static void register_gtk(void) }type_init (register_gtk); + +#ifdef CONFIG_OPENGL +module_dep ("ui-opengl" ); +#endif
ui/sdl2.c +4 -0
@@ -918,3 +918,7 @@ static void register_sdl1(void) }type_init (register_sdl1); + +#ifdef CONFIG_OPENGL +module_dep ("ui-opengl" ); +#endif
ui/spice-app.c +3 -0
@@ -221,3 +221,6 @@ static void register_spice_app(void) }type_init (register_spice_app); + +module_dep ("ui-spice-core" ); +module_dep ("chardev-spice" );
ui/spice-core.c +5 -0
@@ -1037,3 +1037,8 @@ static void spice_register_config(void)qemu_add_opts (&qemu_spice_opts); }opts_init (spice_register_config); +module_opts ("spice" ); + +#ifdef CONFIG_OPENGL +module_dep ("ui-opengl" ); +#endif
Signature · 1 line
2.31.1
PATCH v2 10/18 modules: add s390x module annotations
In reply to Gerd Hoffmann--- hw/s390x/virtio-ccw-gpu.c | 3 +++ 1 file changed, 3 insertions(+)
hw/s390x/virtio-ccw-gpu.c +3 -0
@@ -59,6 +59,7 @@ static const TypeInfo virtio_ccw_gpu = { .instance_init = virtio_ccw_gpu_instance_init, .class_init = virtio_ccw_gpu_class_init, }; +module_obj (TYPE_VIRTIO_GPU_CCW );static void virtio_ccw_gpu_register (void ) {
@@ -68,3 +69,5 @@ static void virtio_ccw_gpu_register(void) } type_init(virtio_ccw_gpu_register) + +module_arch("s390x");
Signature · 1 line
2.31.1
PATCH v2 11/18 modules: add block module annotations
In reply to Gerd Hoffmann--- block/iscsi-opts.c | 1 + 1 file changed, 1 insertion(+)
block/iscsi-opts.c +1 -0
@@ -68,3 +68,4 @@ static void iscsi_block_opts_init(void) }block_init (iscsi_block_opts_init); +module_opts ("iscsi" );
Signature · 1 line
2.31.1
PATCH v2 12/18 modules: add module_load_path_init helper
In reply to Gerd HoffmannFactor out module search path initialization to the new module_load_path_init() helper. Also store the search path in global variables and keep it so we have to do it only once.
--- util/module.c | 58 ++++++++++++++++++++++++++++++--------------------- 1 file changed, 34 insertions(+), 24 deletions(-)
util/module.c +34 -24
@@ -110,6 +110,36 @@ void module_call_init(module_init_type type) }#ifdef CONFIG_MODULES + +static char *module_dirs[5 ]; +static int module_ndirs; + +static void module_load_path_init (void ) +{ +const char *search_dir; + +if (module_ndirs) { +return ; + } + + search_dir =getenv ("QEMU_MODULE_DIR" ); +if (search_dir !=NULL ) { + module_dirs[module_ndirs++] =g_strdup_printf ("%s" , search_dir); + } + module_dirs[module_ndirs++] =get_relocated_path (CONFIG_QEMU_MODDIR ); + module_dirs[module_ndirs++] =g_strdup (qemu_get_exec_dir ()); + +#ifdef CONFIG_MODULE_UPGRADES + version_dir =g_strcanon (g_strdup (QEMU_PKGVERSION ), + G_CSET_A_2_Z G_CSET_a_2_z G_CSET_DIGITS "+-.~", +'_' ); + module_dirs[module_ndirs++] =g_strdup_printf ("/var/run/qemu/%s" , version_dir); +#endif + +assert (module_ndirs <=ARRAY_SIZE (module_dirs)); + +} +static int module_load_file (const char *fname,bool mayfail,bool export_symbols) {GModule *g_module;
@@ -204,10 +234,8 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)#ifdef CONFIG_MODULE_UPGRADES char *version_dir;#endif -const char *search_dir; -char *dirs[5 ];char *module_name; -int i =0 , n_dirs =0 ; +int i =0 ;int ret, dep;bool export_symbols = false;static GHashTable *loaded_modules;
@@ -240,25 +268,11 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail) }g_hash_table_add (loaded_modules, module_name); - search_dir =getenv ("QEMU_MODULE_DIR" ); -if (search_dir !=NULL ) { - dirs[n_dirs++] =g_strdup_printf ("%s" , search_dir); - } - dirs[n_dirs++] =get_relocated_path (CONFIG_QEMU_MODDIR ); - dirs[n_dirs++] =g_strdup (qemu_get_exec_dir ()); +module_load_path_init (); -#ifdef CONFIG_MODULE_UPGRADES - version_dir =g_strcanon (g_strdup (QEMU_PKGVERSION ), - G_CSET_A_2_Z G_CSET_a_2_z G_CSET_DIGITS "+-.~", -'_' ); - dirs[n_dirs++] =g_strdup_printf ("/var/run/qemu/%s" , version_dir); -#endif - -assert (n_dirs <=ARRAY_SIZE (dirs)); - -for (i =0 ; i < n_dirs; i++) { +for (i =0 ; i < module_ndirs; i++) { fname =g_strdup_printf ("%s/%s%s" , - dirs[i], module_name,CONFIG_HOST_DSOSUF ); + module_dirs[i], module_name,CONFIG_HOST_DSOSUF ); ret =module_load_file (fname, mayfail, export_symbols);g_free (fname); fname =NULL ;
@@ -274,10 +288,6 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail) g_free(module_name); } -for (i =0 ; i < n_dirs; i++) { -g_free (dirs[i]); - } -#endif return success; }
Signature · 1 line
2.31.1
PATCH v2 13/18 modules: load modinfo.json
In reply to Gerd HoffmannLoad and parse the module info database.
--- util/module.c | 54 +++++++++++++++++++++++++++++++++++++++++++++++ util/trace-events | 3 +++ 2 files changed, 57 insertions(+)
util/module.c +54 -0
@@ -20,9 +20,16 @@#include "qemu/queue.h" #include "qemu/module.h" #include "qemu/cutils.h" +#include "qemu/error-report.h" #ifdef CONFIG_MODULE_UPGRADES #include "qemu-version.h" #endif +#include "trace.h" + +#include "qapi/error.h" +#include "qapi/qapi-types-modules.h" +#include "qapi/qapi-visit-modules.h" +#include "qapi/qobject-input-visitor.h" typedef struct ModuleEntry {
@@ -111,6 +118,7 @@ void module_call_init(module_init_type type)#ifdef CONFIG_MODULES +static Modules *modinfo;static char *module_dirs[5 ];static int module_ndirs;
@@ -137,7 +145,52 @@ static void module_load_path_init(void)#endif assert (module_ndirs <=ARRAY_SIZE (module_dirs)); +} +static void module_load_modinfo (void ) +{ +char *file, *json; +FILE *fp; +int i, size; +Visitor *v; +Error *errp =NULL ; + +if (modinfo) { +return ; + } + +for (i =0 ; i < module_ndirs; i++) { + file =g_strdup_printf ("%s/modinfo.json" , module_dirs[i]); + fp =fopen (file,"r" ); +if (fp !=NULL ) { +break ; + } +g_free (file); + } +if (NULL == fp) { +warn_report ("No modinfo.json file found." ); +return ; + }else { +trace_module_load_modinfo (file); + } + +fseek (fp,0 ,SEEK_END ); + size =ftell (fp); +fseek (fp,0 ,SEEK_SET ); + json =g_malloc0 (size +1 ); +fread (json, size,1 , fp); + json[size] =0 ; +fclose (fp); + + v =qobject_input_visitor_new_str (json,NULL , &errp); +if (errp) { +error_reportf_err (errp,"parse error (%s)" , file); +g_free (file); +return ; + } +visit_type_Modules (v,NULL , &modinfo, &errp); +visit_free (v); +g_free (file); }static int module_load_file (const char *fname,bool mayfail,bool export_symbols)
@@ -269,6 +322,7 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)g_hash_table_add (loaded_modules, module_name);module_load_path_init (); +module_load_modinfo ();for (i =0 ; i < module_ndirs; i++) { fname = g_strdup_printf("%s/%s%s",
util/trace-events +3 -0
@@ -100,3 +100,6 @@ uffd_create_fd_api_failed(int err) "errno: %i" uffd_create_fd_api_noioctl(uint64_t ioctl_req, uint64_t ioctl_supp) "ioctl_req: 0x%" PRIx64 "ioctl_supp: 0x%" PRIx64 uffd_register_memory_failed(void *addr, uint64_t length, uint64_t mode, int err) "addr: %p length: %" PRIu64 " mode: 0x%" PRIx64 " errno: %i" uffd_unregister_memory_failed(void *addr, uint64_t length, int err) "addr: %p length: %" PRIu64 " errno: %i" + +# module.c +module_load_modinfo(const char *filename) "modinfo %s"
Signature · 1 line
2.31.1
PATCH v2 14/18 modules: use modinfo for dependencies
In reply to Gerd HoffmannUse module database for module dependencies.
Drop hard-coded dependency list.
--- util/module.c | 55 ++++++++++++++++++++------------------------------- 1 file changed, 21 insertions(+), 34 deletions(-)
util/module.c +21 -34
@@ -254,28 +254,6 @@ static int module_load_file(const char *fname, bool mayfail, bool export_symbols out: return ret; } - -static const struct { -const char *name; -const char *dep; -} module_deps[] = { - {"audio-spice" ,"ui-spice-core" }, - {"chardev-spice" ,"ui-spice-core" }, - {"hw-display-qxl" ,"ui-spice-core" }, - {"ui-spice-app" ,"ui-spice-core" }, - {"ui-spice-app" ,"chardev-spice" }, - - {"hw-display-virtio-gpu-gl" ,"hw-display-virtio-gpu" }, - {"hw-display-virtio-gpu-pci-gl" ,"hw-display-virtio-gpu-pci" }, - {"hw-display-virtio-vga-gl" ,"hw-display-virtio-vga" }, - -#ifdef CONFIG_OPENGL - {"ui-egl-headless" ,"ui-opengl" }, - {"ui-gtk" ,"ui-opengl" }, - {"ui-sdl" ,"ui-opengl" }, - {"ui-spice-core" ,"ui-opengl" }, -#endif -};#endif bool module_load_one (const char *prefix,const char *lib_name,bool mayfail)
@@ -289,9 +267,11 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)#endif char *module_name;int i =0 ; -int ret, dep; +int ret;bool export_symbols = false;static GHashTable *loaded_modules; +ModuleInfoList *modlist; +strList *sl;if (!g_module_supported ()) {fprintf (stderr,"Module is not supported by system.\n" );
@@ -304,17 +284,6 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail) module_name =g_strdup_printf ("%s%s" , prefix, lib_name); -for (dep =0 ; dep <ARRAY_SIZE (module_deps); dep++) { -if (strcmp (module_name, module_deps[dep].name) ==0 ) { -/* we depend on another module */ -module_load_one ("" , module_deps[dep].dep, false); - } -if (strcmp (module_name, module_deps[dep].dep) ==0 ) { -/* another module depends on us */ - export_symbols = true; - } - } -if (g_hash_table_contains (loaded_modules, module_name)) {g_free (module_name);return true;
@@ -324,6 +293,24 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)module_load_path_init ();module_load_modinfo (); +for (modlist = modinfo->list; modlist !=NULL ; modlist = modlist->next) { +if (modlist->value->has_deps) { +if (strcmp (modlist->value->name, module_name) ==0 ) { +/* we depend on other module(s) */ +for (sl = modlist->value->deps; sl !=NULL ; sl = sl->next) { +module_load_one ("" , sl->value, false); + } + }else { +for (sl = modlist->value->deps; sl !=NULL ; sl = sl->next) { +if (strcmp (module_name, sl->value) ==0 ) { +/* another module depends on us */ + export_symbols = true; + } + } + } + } + } +for (i =0 ; i < module_ndirs; i++) { fname =g_strdup_printf ("%s/%s%s" , module_dirs[i], module_name,CONFIG_HOST_DSOSUF );
Signature · 1 line
2.31.1
PATCH v2 15/18 modules: use modinfo for qom load
In reply to Gerd HoffmannUse module database to figure which module implements a given QOM type. Drop hard-coded object list.
--- util/module.c | 83 +++++++++++++++++++-------------------------------- 1 file changed, 30 insertions(+), 53 deletions(-)
util/module.c +30 -53
@@ -333,80 +333,57 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)return success; } -/* -* Building devices and other qom objects modular is mostly useful in -* case they have dependencies to external shared libraries, so we can -* cut down the core qemu library dependencies. Which is the case for -* only a very few devices & objects. -* -* So with the expectation that this will be rather the exception than -* the rule and the list will not gain that many entries, go with a -* simple manually maintained list for now. -* -* The list must be sorted by module (module_load_qom_all() needs this). -*/ -static struct { -const char *type; -const char *prefix; -const char *module; -}const qom_modules[] = { - {"ccid-card-passthru" ,"hw-" ,"usb-smartcard" }, - {"ccid-card-emulated" ,"hw-" ,"usb-smartcard" }, - {"usb-redir" ,"hw-" ,"usb-redirect" }, - {"qxl-vga" ,"hw-" ,"display-qxl" }, - {"qxl" ,"hw-" ,"display-qxl" }, - {"virtio-gpu-device" ,"hw-" ,"display-virtio-gpu" }, - {"virtio-gpu-gl-device" ,"hw-" ,"display-virtio-gpu-gl" }, - {"vhost-user-gpu" ,"hw-" ,"display-virtio-gpu" }, - {"virtio-gpu-pci-base" ,"hw-" ,"display-virtio-gpu-pci" }, - {"virtio-gpu-pci" ,"hw-" ,"display-virtio-gpu-pci" }, - {"virtio-gpu-gl-pci" ,"hw-" ,"display-virtio-gpu-pci-gl" }, - {"vhost-user-gpu-pci" ,"hw-" ,"display-virtio-gpu-pci" }, - {"virtio-gpu-ccw" ,"hw-" ,"s390x-virtio-gpu-ccw" }, - {"virtio-vga-base" ,"hw-" ,"display-virtio-vga" }, - {"virtio-vga" ,"hw-" ,"display-virtio-vga" }, - {"virtio-vga-gl" ,"hw-" ,"display-virtio-vga-gl" }, - {"vhost-user-vga" ,"hw-" ,"display-virtio-vga" }, - {"chardev-braille" ,"chardev-" ,"baum" }, - {"chardev-spicevmc" ,"chardev-" ,"spice" }, - {"chardev-spiceport" ,"chardev-" ,"spice" }, -}; +#ifdef CONFIG_MODULES static bool module_loaded_qom_all;void module_load_qom_one (const char *type) { -int i; +ModuleInfoList *modlist; +strList *sl;if (!type) {return ; } -for (i =0 ; i <ARRAY_SIZE (qom_modules); i++) { -if (strcmp (qom_modules[i].type, type) ==0 ) { -module_load_one (qom_modules[i].prefix, - qom_modules[i].module, - false); -return ; + +module_load_path_init (); +module_load_modinfo (); + +for (modlist = modinfo->list; modlist !=NULL ; modlist = modlist->next) { +if (!modlist->value->has_objs) { +continue ; + } +for (sl = modlist->value->objs; sl !=NULL ; sl = sl->next) { +if (strcmp (type, sl->value) ==0 ) { +module_load_one ("" , modlist->value->name, false); + } } } }void module_load_qom_all (void ) { -int i; +ModuleInfoList *modlist;if (module_loaded_qom_all) {return ; } -for (i =0 ; i <ARRAY_SIZE (qom_modules); i++) { -if (i >0 && (strcmp (qom_modules[i -1 ].module, - qom_modules[i].module) ==0 && -strcmp (qom_modules[i -1 ].prefix, - qom_modules[i].prefix) ==0 )) { -/* one module implementing multiple types -> load only once */ + +module_load_path_init (); +module_load_modinfo (); + +for (modlist = modinfo->list; modlist !=NULL ; modlist = modlist->next) { +if (!modlist->value->has_objs) {continue ; } -module_load_one (qom_modules[i].prefix, qom_modules[i].module, true); +module_load_one ("" , modlist->value->name, false); } module_loaded_qom_all = true; } + +#else + +void module_load_qom_one (const char *type) {} +void module_load_qom_all (void ) {} + +#endif
Signature · 1 line
2.31.1
PATCH v2 16/18 modules: use modinfo for qemu opts load
In reply to Gerd HoffmannUse module database to figure which module adds given QemuOpts group.
--- softmmu/vl.c | 17 ----------------- stubs/module-opts.c | 4 ---- util/module.c | 19 +++++++++++++++++++ 3 files changed, 19 insertions(+), 21 deletions(-)
softmmu/vl.c +0 -17
@@ -2675,23 +2675,6 @@ void qmp_x_exit_preconfig(Error **errp) } } -#ifdef CONFIG_MODULES -void qemu_load_module_for_opts (const char *group) -{ -static bool spice_tried; -if (g_str_equal (group,"spice" ) && !spice_tried) { -ui_module_load_one ("spice-core" ); - spice_tried = true; - } - -static bool iscsi_tried; -if (g_str_equal (group,"iscsi" ) && !iscsi_tried) { -block_module_load_one ("iscsi" ); - iscsi_tried = true; - } -} -#endif -void qemu_init (int argc,char **argv,char **envp) {QemuOpts *opts;
stubs/module-opts.c +0 -4
@@ -1,6 +1,2 @@#include "qemu/osdep.h" #include "qemu/config-file.h" - -void qemu_load_module_for_opts (const char *group) -{ -}
util/module.c +19 -0
@@ -21,6 +21,7 @@#include "qemu/module.h" #include "qemu/cutils.h" #include "qemu/error-report.h" +#include "qemu/config-file.h" #ifdef CONFIG_MODULE_UPGRADES #include "qemu-version.h" #endif
@@ -381,8 +382,26 @@ void module_load_qom_all(void) module_loaded_qom_all = true; } +void qemu_load_module_for_opts (const char *group) +{ +ModuleInfoList *modlist; + +module_load_path_init (); +module_load_modinfo (); + +for (modlist = modinfo->list; modlist !=NULL ; modlist = modlist->next) { +if (!modlist->value->has_opts) { +continue ; + } +if (strcmp (modlist->value->opts, group) ==0 ) { +module_load_one ("" , modlist->value->name, false); + } + } +} +#else +void qemu_load_module_for_opts (const char *group) {}void module_load_qom_one (const char *type) {}void module_load_qom_all (void ) {}
Signature · 1 line
2.31.1
PATCH v2 17/18 modules: check arch and block load on mismatch
In reply to Gerd HoffmannAdd module_allow_arch() to set the target architecture. In case a module is limited to some arch verify arches match and ignore the module if not.
--- include/qemu/module.h | 1 + softmmu/vl.c | 3 +++ util/module.c | 15 +++++++++++++++ 3 files changed, 19 insertions(+)
include/qemu/module.h +1 -0
@@ -72,6 +72,7 @@ void module_call_init(module_init_type type);bool module_load_one (const char *prefix,const char *lib_name,bool mayfail);void module_load_qom_one (const char *type);void module_load_qom_all (void ); +void module_allow_arch (const char *arch);/* * macros to store module metadata in a .modinfo section.
softmmu/vl.c +3 -0
@@ -126,6 +126,8 @@#include "sysemu/iothread.h" #include "qemu/guest-random.h" +#include "config-host.h" +#define MAX_VIRTIO_CONSOLES 1 typedef struct BlockdevOptionsQueueEntry {
@@ -2723,6 +2725,7 @@ void qemu_init(int argc, char **argv, char **envp)error_init (argv[0 ]);qemu_init_exec_dir (argv[0 ]); +module_allow_arch (TARGET_NAME );qemu_init_subsystems ();/* first pass of option parsing */
util/module.c +15 -0
@@ -122,6 +122,12 @@ void module_call_init(module_init_type type)static Modules *modinfo;static char *module_dirs[5 ];static int module_ndirs; +static const char *module_arch; + +void module_allow_arch (const char *arch) +{ + module_arch = arch; +}static void module_load_path_init (void ) {
@@ -295,6 +301,14 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)module_load_modinfo ();for (modlist = modinfo->list; modlist !=NULL ; modlist = modlist->next) { +if (modlist->value->has_arch) { +if (strcmp (modlist->value->name, module_name) ==0 ) { +if (!module_arch || +strcmp (modlist->value->arch, module_arch) !=0 ) { +return false; + } + } + }if (modlist->value->has_deps) {if (strcmp (modlist->value->name, module_name) ==0 ) {/* we depend on other module(s) */
@@ -401,6 +415,7 @@ void qemu_load_module_for_opts(const char *group)#else +void module_allow_arch (const char *arch) {}void qemu_load_module_for_opts (const char *group) {}void module_load_qom_one (const char *type) {}void module_load_qom_all (void ) {}
Signature · 1 line
2.31.1
Re: [PATCH v2 17/18] modules: check arch and block load on mismatch
Quoted 63 lines · Gerd Hoffmann
Add
module_allow_arch()to set the target architecture. In case a module is limited to some arch verify arches match and ignore the module if not.Signed-off-by Gerd Hoffmann <[email protected]>--- include/qemu/module.h | 1 + softmmu/vl.c | 3 +++ util/module.c | 15 +++++++++++++++ 3 files changed, 19 insertions(+)include/qemu/module.h +1
-0@@ -72,6 +72,7 @@ void module_call_init(module_init_type type);bool module_load_one (const char *prefix,const char *lib_name,bool mayfail);void module_load_qom_one (const char *type);void module_load_qom_all (void ); +void module_allow_arch (const char *arch);/* * macros to store module metadata in a .modinfo section. softmmu/vl.c +3
-0@@ -126,6 +126,8 @@#include "sysemu/iothread.h" #include "qemu/guest-random.h" +#include "config-host.h" +#define MAX_VIRTIO_CONSOLES 1 typedef struct BlockdevOptionsQueueEntry {@@ -2723,6 +2725,7 @@ void qemu_init(int argc, char **argv, char **envp)error_init (argv[0 ]);qemu_init_exec_dir (argv[0 ]); +module_allow_arch (TARGET_NAME );qemu_init_subsystems ();/* first pass of option parsing */ util/module.c +6
-0@@ -122,6 +122,12 @@ void module_call_init(module_init_type type)static Modules *modinfo;static char *module_dirs[5 ];static int module_ndirs; +static const char *module_arch; + +void module_allow_arch (const char *arch) +{ + module_arch = arch; +}static void module_load_path_init (void ) {@@ -295,6 +301,14 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)module_load_modinfo ();
util/module.c bool module_load_one(const char *prefix, const char *lib_name, bool mayfail) · line 311
Quoted 10 lines
util/module.c +8
-0for (modlist = modinfo->list; modlist !=NULL ; modlist = modlist->next) { +if (modlist->value->has_arch) { +if (strcmp (modlist->value->name, module_name) ==0 ) { +if (!module_arch || +strcmp (modlist->value->arch, module_arch) !=0 ) { +return false; + } + } + }
I have a little hard time following the code paths, but IIUC, with this change, instead of "module.so" we would have multiple copies of something like "module-$arch.so" ? Then we load them all, read their modinfo section and discard the ones with non-matching arch ?
If that is a correct understanding, then I wonder about the scalability of it. We have 30 system emulator targets right now, so if we get even a few arch specific modues, that's going to be alot of modules we're loading, checking and discarding.
Wouldn't it be simpler if we just made use of the directory layout /usr/lib64/qemu/$mod.so for common modules and /usr/lib64/qemu/$arch/$mod.so for arch specific modules. That would let us load only the modules we know are applicable for the system target arch and not need this post-load filtering from metadata.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 17/18] modules: check arch and block load on mismatch
Hi,
Quoted 10 lines
util/module.c bool module_load_one(const char *prefix, const char *lib_name, bool mayfail) · line 311
Quoted 9 lines
for (modlist = modinfo->list; modlist != NULL; modlist = modlist->next) {util/module.c +8
-0+if (modlist->value->has_arch) { +if (strcmp (modlist->value->name, module_name) ==0 ) { +if (!module_arch || +strcmp (modlist->value->arch, module_arch) !=0 ) { +return false; + } + } + }
Quoted 3 lines
I have a little hard time following the code paths, but IIUC, with this change, instead of "module.so" we would have multiple copies of something like "module-$arch.so" ?
Not yet with this series, but easily doable on top of this (see other patch series sent today).
Quoted 2 lines
Then we load them all, read their modinfo section
and discard the ones with non-matching arch ?
No. There is a utility reading the modinfo section (patch #2), write out the info as json (patch #2 has the schema), then qemu will read that json file (patch #13) ...
Quoted 3 lines
for arch specific modules. That would let us load only the modules we know are applicable for the system target arch and not need this post-load filtering from metadata.
... so it's pre-load filtering, not post-load.
take care,
Gerd
Re: [PATCH v2 17/18] modules: check arch and block load on mismatch
Quoted 19 lines · Gerd Hoffmann
Hi,
Quoted 10 lines
util/module.c bool module_load_one(const char *prefix, const char *lib_name, bool mayfail) · line 311
Quoted 9 lines
for (modlist = modinfo->list; modlist != NULL; modlist = modlist->next) {util/module.c +8
-0+if (modlist->value->has_arch) { +if (strcmp (modlist->value->name, module_name) ==0 ) { +if (!module_arch || +strcmp (modlist->value->arch, module_arch) !=0 ) { +return false; + } + } + }Quoted 3 lines
I have a little hard time following the code paths, but IIUC, with this change, instead of "module.so" we would have multiple copies of something like "module-$arch.so" ?
Not yet with this series, but easily doable on top of this (see other patch series sent today).
Quoted 6 lines
Quoted 2 lines
Then we load them all, read their modinfo section
and discard the ones with non-matching arch ?No. There is a utility reading the modinfo section (patch #2), write out the info as json (patch #2 has the schema), then qemu will read that json file (patch #13) ...
Ah ok, missed that.
Is the JSON file completely static, listing all modules that were built regardless of whether they are currently installed, or would it need to be refreshed when installing/uninstalling RPMs with modules ? I would think we can do the former and simply handle missing modules on disk fairly easily
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 17/18] modules: check arch and block load on mismatch
Hi,
Quoted 5 lines
Is the JSON file completely static, listing all modules that were built regardless of whether they are currently installed, or would it need to be refreshed when installing/uninstalling RPMs with modules ? I would think we can do the former and simply handle missing modules on disk fairly easily
We can do both. The file is generated and installed as part of the build/install process, and it can be simply used as-is even if some of the modules are missing.
It's also possible to update the modinfo.json file in postinstall / postuninstall by simply running qemu-modinfo, so only the modules actually installed are listed there.
take care,
Gerd
Re: [PATCH v2 17/18] modules: check arch and block load on mismatch
Quoted 12 lines · Gerd Hoffmann
Hi,
Quoted 5 lines
Is the JSON file completely static, listing all modules that were built regardless of whether they are currently installed, or would it need to be refreshed when installing/uninstalling RPMs with modules ? I would think we can do the former and simply handle missing modules on disk fairly easily
We can do both. The file is generated and installed as part of the build/install process, and it can be simply used as-is even if some of the modules are missing.
Quoted 8 lines
It's also possible to update the modinfo.json file in postinstall / postuninstall by simply running qemu-modinfo, so only the modules actually installed are listed there.
take care,
Gerd
I fail to see why that extra complication is needed at all.
Why don't we just build the modules for the targets we intend to build, and install them as .so files in a target arch directory?
What problem is the json solving? Sorry if obvious.
Thanks,
Claudio
Re: [PATCH v2 17/18] modules: check arch and block load on mismatch
Hi,
Quoted 4 lines
Quoted 4 lines
We can do both. The file is generated and installed as part of the build/install process, and it can be simply used as-is even if some of the modules are missing.
Quoted 8 lines
Quoted 3 lines
It's also possible to update the modinfo.json file in postinstall / postuninstall by simply running qemu-modinfo, so only the modules actually installed are listed there.
I fail to see why that extra complication is needed at all.
Why don't we just build the modules for the targets we intend to build, and install them as .so files in a target arch directory?
There is more meta-data we need for modules: Which QOM types are implemented by which module (for on-demand loading), dependencies between modules and also which command line options (aka QemuOpts) are handled by which modules.
Possibly more in the future, maybe we'll support modules registering monitor commands dynamically some day (like usb-host implementing "info usbhost" or tcg implementing "info jit" + "info opcount") and we'd like store that information in the module database too.
If we have such a module database anyway it IMHO makes alot of sense to simply store the target arch there too instead of using something else.
Quoted 1 line
What problem is the json solving?
Well, the goal is to store meta-data about modules, in a way which:
(a) Doesn't require manually maintained lists. This is what we have right now (arrays in utils/module.c). Works ok-ish for a small number of modules, but becomes increasingly problematic with the growing number of modules.
(b) Doesn't require opening each individual module on each qemu run to get the information.
I'm not particularly attached to using json for that, it is just that we already have infrastructure to parse/serialize structs from/to json because we need that for qapi anyway.
take care,
Gerd
Re: [PATCH v2 17/18] modules: check arch and block load on mismatch
Quoted 40 lines · Gerd Hoffmann
Hi,
Quoted 4 lines
Quoted 4 lines
We can do both. The file is generated and installed as part of the build/install process, and it can be simply used as-is even if some of the modules are missing.
Quoted 8 lines
Quoted 3 lines
It's also possible to update the modinfo.json file in postinstall / postuninstall by simply running qemu-modinfo, so only the modules actually installed are listed there.
I fail to see why that extra complication is needed at all.
Why don't we just build the modules for the targets we intend to build, and install them as .so files in a target arch directory?
There is more meta-data we need for modules: Which QOM types are implemented by which module (for on-demand loading), dependencies between modules and also which command line options (aka QemuOpts) are handled by which modules.
Possibly more in the future, maybe we'll support modules registering monitor commands dynamically some day (like usb-host implementing "info usbhost" or tcg implementing "info jit" + "info opcount") and we'd like store that information in the module database too.
If we have such a module database anyway it IMHO makes alot of sense to simply store the target arch there too instead of using something else.
Quoted 1 line
What problem is the json solving?
Well, the goal is to store meta-data about modules, in a way which:
(a) Doesn't require manually maintained lists. This is what we have right now (arrays in
utils/module.c). Works ok-ish for a small number of modules, but becomes increasingly problematic with the growing number of modules.
(b) Doesn't require opening each individual module on each qemu run to get the information.
Quoted 3 lines
I'm not particularly attached to using json for that, it is just that we already have infrastructure to parse/serialize structs from/to json because we need that for qapi anyway.
If we can generate json, we could generate .c code which has all the data statically declared and just link it in to QEMU. If we don't need ability to update the metadata post-build then it would be equivalent functionally. If we need to be able to update the metadata to precisely matcch the set of installed modules though, then a separate json file looks best.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 17/18] modules: check arch and block load on mismatch
Hi,
Quoted 6 lines
Quoted 3 lines
I'm not particularly attached to using json for that, it is just that we already have infrastructure to parse/serialize structs from/to json because we need that for qapi anyway.
If we can generate json, we could generate .c code which has all the data statically declared and just link it in to QEMU.
Yes, I think that would work too.
take care,
Gerd
Re: [PATCH v2 17/18] modules: check arch and block load on mismatch
In reply to Daniel P. BerrangéQuoted 85 lines · Daniel P. Berrangé
Quoted 63 lines · Gerd Hoffmann
Add
module_allow_arch()to set the target architecture. In case a module is limited to some arch verify arches match and ignore the module if not.Signed-off-by Gerd Hoffmann <[email protected]>--- include/qemu/module.h | 1 + softmmu/vl.c | 3 +++ util/module.c | 15 +++++++++++++++ 3 files changed, 19 insertions(+)include/qemu/module.h +1
-0@@ -72,6 +72,7 @@ void module_call_init(module_init_type type);bool module_load_one (const char *prefix,const char *lib_name,bool mayfail);void module_load_qom_one (const char *type);void module_load_qom_all (void ); +void module_allow_arch (const char *arch);/* * macros to store module metadata in a .modinfo section. softmmu/vl.c +3
-0@@ -126,6 +126,8 @@#include "sysemu/iothread.h" #include "qemu/guest-random.h" +#include "config-host.h" +#define MAX_VIRTIO_CONSOLES 1 typedef struct BlockdevOptionsQueueEntry {@@ -2723,6 +2725,7 @@ void qemu_init(int argc, char **argv, char **envp)error_init (argv[0 ]);qemu_init_exec_dir (argv[0 ]); +module_allow_arch (TARGET_NAME );qemu_init_subsystems ();/* first pass of option parsing */ util/module.c +6
-0@@ -122,6 +122,12 @@ void module_call_init(module_init_type type)static Modules *modinfo;static char *module_dirs[5 ];static int module_ndirs; +static const char *module_arch; + +void module_allow_arch (const char *arch) +{ + module_arch = arch; +}static void module_load_path_init (void ) {@@ -295,6 +301,14 @@ bool module_load_one(const char *prefix, const char *lib_name, bool mayfail)module_load_modinfo ();util/module.c bool module_load_one(const char *prefix, const char *lib_name, bool mayfail) · line 311
Quoted 10 lines
util/module.c +8
-0for (modlist = modinfo->list; modlist !=NULL ; modlist = modlist->next) { +if (modlist->value->has_arch) { +if (strcmp (modlist->value->name, module_name) ==0 ) { +if (!module_arch || +strcmp (modlist->value->arch, module_arch) !=0 ) { +return false; + } + } + }I have a little hard time following the code paths, but IIUC, with this change, instead of "module.so" we would have multiple copies of something like "module-$arch.so" ? Then we load them all, read their modinfo section and discard the ones with non-matching arch ?
If that is a correct understanding, then I wonder about the scalability of it. We have 30 system emulator targets right now, so if we get even a few arch specific modues, that's going to be alot of modules we're loading, checking and discarding.
Quoted 9 lines
Wouldn't it be simpler if we just made use of the directory layout
/usr/lib64/qemu/$mod.sofor common modules and/usr/lib64/qemu/$arch/$mod.sofor arch specific modules. That would let us load only the modules we know are applicable for the system target arch and not need this post-load filtering from metadata.Regards,
Daniel
agreed.
Claudio
PATCH v2 18/18 module_load_modinfo
In reply to Gerd Hoffmann--- util/module.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
util/module.c +3 -3
@@ -158,7 +158,7 @@ static void module_load_modinfo(void) {char *file, *json;FILE *fp; -int i, size; +int i, size, ret;Visitor *v;Error *errp =NULL ;
@@ -185,8 +185,8 @@ static void module_load_modinfo(void) size =ftell (fp);fseek (fp,0 ,SEEK_SET ); json =g_malloc0 (size +1 ); -fread (json, size,1 , fp); - json[size] =0 ; + ret =fread (json,1 , size, fp); + json[ret] =0 ;fclose (fp); v =qobject_input_visitor_new_str (json,NULL , &errp);
Signature · 1 line
2.31.1
Re: [PATCH v2 18/18] [fixup] module_load_modinfo
Quoted 1 line · Gerd Hoffmann
Signed-off-by Gerd Hoffmann <[email protected]>
Oops. That should have been squashed into patch #13.
take care,
Gerd
Quoted 11 lines · Gerd Hoffmann
This patch series adds support for module metadata. Here are the pieces of the puzzle:
(1) Macros are added to store metadata in a .modinfo elf section (idea stolen from the linux kernel).
(2) A utility to scan modules, collect metadata from the .modinfo sections, store it in a file (modinfo.json) for later consumption by qemu. Can also be easily inspected using 'jq'.
(3) Adding annotations to the modules we have.
(4) Drop hard-coded lists fromutils/module.c
Quoted 2 lines
take care,
Gerd
The background has disappeared compared with V1.
V1 says:
"Background is that the hard-coded lists in util/module.c are somewhat ugly and also wouldn't work very well with a large number of modules, so I'm looking for something else."
Can you write more about what the actual high level goals of this series are?
We are in general making QEMU more and more difficult to get into, requiring more and more investment for new contributors to get productive.
Is the additional complexity justified? What is the benefit, and is simplification a goal of the series as well?
Quoted 59 lines
Gerd Hoffmann (18):
modules: add metadata macros, add qxl module annotations
qapi: add ModuleInfo schema
modules: add qemu-modinfo utility
modules: add virtio-gpu module annotations
modules: add chardev module annotations
modules: add audio module annotations
modules: add usb-redir module annotations
modules: add ccid module annotations
modules: add ui module annotations
modules: add s390x module annotations
modules: add block module annotations
modules: addmodule_load_path_inithelper
modules: load modinfo.json
modules: use modinfo for dependencies
modules: use modinfo for qom load
modules: use modinfo for qemu opts load
modules: check arch and block load on mismatch [fixup]module_load_modinfoinclude/qemu/module.h | 23 +++ audio/spiceaudio.c | 2 + block/iscsi-opts.c | 1 + chardev/baum.c | 1 + chardev/spice.c | 4 + hw/display/qxl.c | 4 + hw/display/vhost-user-gpu-pci.c | 1 + hw/display/vhost-user-gpu.c | 1 + hw/display/vhost-user-vga.c | 1 + hw/display/virtio-gpu-base.c | 1 + hw/display/virtio-gpu-gl.c | 3 + hw/display/virtio-gpu-pci-gl.c | 3 + hw/display/virtio-gpu-pci.c | 2 + hw/display/virtio-gpu.c | 1 + hw/display/virtio-vga-gl.c | 3 + hw/display/virtio-vga.c | 2 + hw/s390x/virtio-ccw-gpu.c | 3 + hw/usb/ccid-card-emulated.c | 1 + hw/usb/ccid-card-passthru.c | 1 + hw/usb/redirect.c | 1 + qemu-modinfo.c | 270 ++++++++++++++++++++++++++++++ softmmu/vl.c | 20 +-- stubs/module-opts.c | 4 - ui/egl-headless.c | 4 + ui/gtk.c | 4 + ui/sdl2.c | 4 + ui/spice-app.c | 3 + ui/spice-core.c | 5 + util/module.c | 282 +++++++++++++++++++------------- meson.build | 11 ++ qapi/meson.build | 1 + qapi/modules.json | 36 ++++ qapi/qapi-schema.json | 1 + util/trace-events | 3 + 34 files changed, 576 insertions(+), 131 deletions(-) create mode 100644 qemu-modinfo.c create mode 100644 qapi/modules.json
Quoted 15 lines · Claudio Fontana
Quoted 11 lines · Gerd Hoffmann
This patch series adds support for module metadata. Here are the pieces of the puzzle:
(1) Macros are added to store metadata in a .modinfo elf section (idea stolen from the linux kernel).
(2) A utility to scan modules, collect metadata from the .modinfo sections, store it in a file (modinfo.json) for later consumption by qemu. Can also be easily inspected using 'jq'.
(3) Adding annotations to the modules we have.
(4) Drop hard-coded lists fromutils/module.cQuoted 2 lines
take care,
Gerd
Quoted 7 lines
The background has disappeared compared with V1.
V1 says:
"Background is that the hard-coded lists in
util/module.care somewhat ugly and also wouldn't work very well with a large number of modules, so I'm looking for something else."
Well, it's point (4) now (a bit short indeed ...).
Quoted 1 line
Can you write more about what the actual high level goals of this series are?
Right now we have information about modules hard-coded in various places in qemu. Most obvious ones are module_deps[] and qom_modules[] (both in util/module.c), but also qemu_load_module_for_opts() (in softmmu/vl.c) and maybe more I missed.
So, when you go build some qom object modular today you'll have to go add that to the qom_modules[] list. With this patch series applied you'll go add a 'module_obj("typename");' to your source file instead.
Same goes for other metadata, see the "add $foo module annotations" patches for examples.
Quoted 6 lines
We are in general making QEMU more and more difficult to get into, requiring more and more investment for new contributors to get productive.
Is the additional complexity justified? What is the benefit, and is simplification a goal of the series as well?
IMHO it is a simplification for developers. Modules are more self-contained with this in place. You just add the annotation macros and you are done. No need to edit manually maintained lists at some non-obvious place elsewhere in the tree. Also no patch conflicts in those lists. We have type_init() + friends for simliar reasons.
The price for that simplification is the new utility needed which collects and stores the metadata. But that is something you only need to worry about when actually working on module support. The build system keeps the database automatically up-to-date and most developers shouldn't even notice that it is there.
take care,
Gerd
Quoted 24 lines · Gerd Hoffmann
Quoted 15 lines · Claudio Fontana
Quoted 11 lines · Gerd Hoffmann
This patch series adds support for module metadata. Here are the pieces of the puzzle:
(1) Macros are added to store metadata in a .modinfo elf section (idea stolen from the linux kernel).
(2) A utility to scan modules, collect metadata from the .modinfo sections, store it in a file (modinfo.json) for later consumption by qemu. Can also be easily inspected using 'jq'.
(3) Adding annotations to the modules we have.
(4) Drop hard-coded lists fromutils/module.cQuoted 2 lines
take care,
GerdQuoted 7 lines
The background has disappeared compared with V1.
V1 says:
"Background is that the hard-coded lists in
util/module.care somewhat ugly and also wouldn't work very well with a large number of modules, so I'm looking for something else."
Quoted 8 lines
Well, it's point (4) now (a bit short indeed ...).
Quoted 1 line
Can you write more about what the actual high level goals of this series are?
Right now we have information about modules hard-coded in various places in qemu. Most obvious ones are module_deps[] and qom_modules[] (both in
util/module.c), but alsoqemu_load_module_for_opts()(insoftmmu/vl.c) and maybe more I missed.
Maybe a good idea to find out.
Quoted 8 lines
So, when you go build some qom object modular today you'll have to go add that to the qom_modules[] list. With this patch series applied you'll go add a '
module_obj("typename");' to your source file instead.Same goes for other metadata, see the "add $foo module annotations" patches for examples.
Quoted 8 lines
Quoted 6 lines
We are in general making QEMU more and more difficult to get into, requiring more and more investment for new contributors to get productive.
Is the additional complexity justified? What is the benefit, and is simplification a goal of the series as well?
IMHO it is a simplification for developers. Modules are more
So the whole endevour here is to remove the need to update modules in two/three places?
It might simplify life for developers adding a module,
but it is at a cost for the developers trying to keep the plumbing to work in my opinion.
Based on the information you provided, the reason this whole series exists seems to be to remove the need to update modules in multiple places.
Should the design focus be on that and that only?
Is there a real need to copy over the mechanism from the kernel, or are the requirements actually different/simpler here?
Quoted 15 lines
self-contained with this in place. You just add the annotation macros and you are done. No need to edit manually maintained lists at some non-obvious place elsewhere in the tree. Also no patch conflicts in those lists. We have
type_init()+ friends for simliar reasons.The price for that simplification is the new utility needed which collects and stores the metadata. But that is something you only need to worry about when actually working on module support. The build system keeps the database automatically up-to-date and most developers shouldn't even notice that it is there.
take care,
Gerd
Hi,
Quoted 3 lines
Based on the information you provided, the reason this whole series exists seems to be to remove the need to update modules in multiple places.
Well, I'm trying to improve the way we handle module meta-data (see other mail just send for details).
Quoted 2 lines
Is there a real need to copy over the mechanism from the kernel, or are the requirements actually different/simpler here?
Even though the exact kind of meta-data is different (qemu wants know qom types implemented by module, kernel wants know what hardware a module can drive) the fundamental problem is the same: We need some meta-data about modules for lookup. The workflow is simliar too:
The meta-data is stored in a .modinfo elf section. Then a utility collects that data and stores them in a database.
For the kernel 'depmod' stores the data in /lib/modules/$version/modules.$name (with modules.alias being the largest database).
I'm trying to have qemu-modinfo store this in modinfo.json.
take care,
Gerd