contrib/plugins: add dlcall to call host functions from a guest
v5 cover contrib/plugins: add dlcall to call host functions from a guest
Hi,
This patch adds a single plugin, contrib/plugins/dlcall.c (~240 lines, no changes to QEMU core), that lets a linux-user guest call functions in the host's native shared libraries instead of emulating them.
It is the natural next step on top of the vCPU syscall-filter callback that I contributed and that was merged earlier:
https://lore.kernel.org/qemu-devel/[email protected]/
Why bother? Because it turns slow, instruction-by-instruction emulation of a library into a native host call. Some results, all on completely unmodified guest binaries:
* minizip (the stock zlib utility) compresses several times faster, because the actual deflate runs natively on the host instead of being translated.
* Real OpenGL/Vulkan games run under qemu-user: SuperTuxKart and Hollow Knight are playable, with their graphics calls going straight to the host GPU.
How it works
============
The guest makes a system call with a reserved number (4096 by default) that no real Linux ABI uses. Its first argument selects a pass-through operation, and the rest carry operands:
syscall(4096, op, arg1, arg2, ...)
| | \............ operands (pointers / values)
| \................. which pass-through operation
\....................... the reserved "magic" numberThe plugin registers a vCPU syscall filter: before QEMU forwards a syscall to the host kernel, the filter runs, sees 4096, performs the operation on the host, writes the result back, and tells QEMU the syscall is consumed, so the real kernel never sees it.
The whole interface is just a handful of primitives:
* query a host attribute
* dlopen / dlclose a host shared library
* dlsym a symbol, and read the last dlerror
* invoke a resolved host function with a void(void *, void *) signature
That is all the plugin does. It knows nothing about zlib, X11 or OpenGL, or about any library's calling convention.
The same machinery also runs in reverse: when a host function needs to call back into the guest (a qsort comparator, an allocator, a GUI or game callback), control re-enters the guest to run the callback and then resumes the suspended host call. This reentry is what lets stateful, callback-driven APIs work, not just leaf functions.
Why the plugin belongs in QEMU, and the rest does not
=====================================================
Only the plugin lives in the tree. Everything else is ordinary userspace:
--- userspace (out of tree, not tied to any DBT) -------------
guest: unmodified program -> guest runtime + thunk libs
--------------------------------------------------------------
| syscall(4096, op, args) (only crossing point)
v
=== inside QEMU: THIS PATCH, ~240 lines ======================
dlcall plugin: dlopen / dlsym / invoke a host fn
==============================================================
|
v
--- userspace (out of tree) ----------------------------------
host: host runtime + thunk libs -> real libz / libGL ...--------------------------------------------------------------
The split is deliberate, and it is why only this one file is proposed for the tree:
* This plugin defines the most general interaction interface for native
pass-through: the magic-syscall ABI between an emulated guest and its emulator. That contract is what every pass-through implementation builds on, so it belongs in a stable, shared place.
* It is also the only piece that is inherently QEMU-specific: it plugs into QEMU's syscall-filter hook and runs inside the QEMU process. The argument marshalling, calling conventions, callbacks/reentry and per-library coverage are not tied to any particular DBT and behave as ordinary userspace, so they should stay out of tree rather than couple QEMU to them.
Background: we presented this approach at KVM Forum 2025, "Lorelei: Enable QEMU to Leverage Native Shared Libraries":
https://www.youtube.com/watch?v=_jioQFm7wyU
The userspace side
==================
A fair point on v3 was that, on its own, the plugin is only half of an
interface: useful only if the other half (the guest/host runtimes and the per-library thunks) is public and specified, rather than a private demo. That half is now available as a standalone, documented, CI-tested project, Lorelei:
https://github.com/rover2024/lorelei
Lorelei provides the guest and host runtimes and a Thunk Library Compiler (TLC, built on Clang LibTooling) that reads a library's headers and generates the guest and host thunks automatically, including the awkward cases of function-pointer callbacks and variadic functions. It has CI for x86_64, arm64 and riscv64 hosts. Lorelei and its thunk libraries are MIT-licensed.
Because the plugin is not upstream yet, Lorelei currently builds and tests against the QEMU fork that carries it. Its releases ship prebuilt guest and host thunk trees for x86_64, aarch64 and riscv64 hosts, so trying it out does not require building from source:
https://github.com/rover2024/lorelei/releases/tag/v1.0.1.0
The plugin stays deliberately minimal and prescribes nothing about how thunking is done. Lorelei is one reference implementation of the userspace side. Any toolchain, or another instrumentation framework, can implement the same dlcall interface.
A from-scratch walkthrough of the bare mechanism, with the minizip and OpenGL/X11 examples above, is also available here:
https://github.com/rover2024/qemu-passthrough-test
It is fully opt-in (loaded with -plugin) and targets linux-user, where the guest and host already share a trust domain. The test cases use x86_64 guests and run on x86_64, arm64 and riscv64 Linux hosts.
Following the v4 review (Pierrick Bouvier), this version adds a documentation patch (patch 2/2) for docs/about/emulation.rst with a minimal example, and now references Lorelei as one complete userspace implementation of the interface, from both that documentation and the plugin's header comment.
Feedback on the plugin and on the pass-through approach is welcome.
Changes since v4:
* Added a documentation patch for docs/about/emulation.rst (Pierrick Bouvier): what the plugin does, its trusted-guests and guest_base == 0 constraints, how to load it, and a minimal example, with a pointer to Lorelei.
* The plugin's header comment now points to Lorelei as one complete userspace implementation. The plugin code is otherwise unchanged.
Changes since v3:
* Lorelei, the userspace toolchain that implements this interface, is now a public, documented, CI-tested project, with a Thunk Library Compiler that generates the guest/host thunks automatically. This addresses the v3 feedback that the interface needs a public implementation behind it. The plugin code itself is unchanged.
Changes since v2:
* Dropped the RFC tag. The approach was positively received on v2.
* Rebased on master and adjusted the syscall-filter callback to its updated signature (int64_t sysret, added userdata, dropped the plugin id argument).
Changes since v1:
* Renamed the plugin from "passthrough" to "dlcall" (Pierrick Bouvier). The old name was too generic. The name "dlcall" reflects what the plugin actually does (dlopen/dlsym a host symbol and call it) and avoids confusion with QEMU's existing plugin hostcall concept (QEMU_PLUGIN_*_HOSTCALL).
* Made the magic syscall number configurable at load time via the "syscall_num=N" argument, defaulting to 4096 and rejecting values low enough to clash with a real syscall (Pierrick Bouvier).
v1: https://lore.kernel.org/qemu-devel/[email protected]/
v2: https://lore.kernel.org/qemu-devel/[email protected]/
v3: https://lore.kernel.org/qemu-devel/[email protected]/
v4: https://lore.kernel.org/qemu-devel/[email protected]/
Thanks,
Ziyang Zhang
Ziyang Zhang (2):
contrib/plugins: add a minimal dlcall plugin docs/about/emulation: document the dlcall plugin
contrib/plugins/dlcall.c | 246 ++++++++++++++++++++++++++++++++++++ contrib/plugins/meson.build | 5 + docs/about/emulation.rst | 55 ++++++++ 3 files changed, 306 insertions(+) create mode 100644 contrib/plugins/dlcall.c
Signature · 1 line
2.34.1
PATCH v5 1/2 contrib/plugins: add a minimal dlcall plugin
Add a minimal dlcall plugin that lets the guest invoke host functions through magic system calls. The plugin registers a vCPU syscall filter callback that intercepts a reserved syscall number and dispatches a set of pass-through operations: querying host attributes, loading and freeing shared libraries, resolving symbols, retrieving the last library error, and invoking a host function through a common interface.
The magic syscall number defaults to 4096 and can be overridden at load time with the "syscall_num=N" argument; values low enough to clash with a real syscall are rejected.
--- contrib/plugins/dlcall.c | 246 ++++++++++++++++++++++++++++++++++++ contrib/plugins/meson.build | 5 + 2 files changed, 251 insertions(+) create mode 100644 contrib/plugins/dlcall.c
contrib/plugins/dlcall.c new file +246 -0
@@ -0,0 +1,246 @@ +/* +* Copyright (C) 2026, Ziyang Zhang <[email protected]> +* +* dlcall plugin: lets a linux-user guest invoke host functions by issuing a +* magic system call. The guest can ask QEMU to dlopen() a host shared +* library, dlsym() a symbol, and call it with guest-supplied arguments. +* +* The plugin is minimal and prescribes nothing about how a library is +* thunked. Any toolchain can implement the userspace side. Lorelei is a +* complete, MIT-licensed implementation (guest/host runtimes plus a thunk +* compiler that generates thunks from a library's headers): +* https://github.com/rover2024/lorelei +* +* See docs/about/emulation.rst|Dlcall for details and examples. +* +* WARNING: trusted guests only. The guest can load arbitrary host libraries +* and execute arbitrary host code with arbitrary arguments, i.e. full code +* execution in the QEMU host process. It is NOT a sandbox and provides no +* isolation; only load it for guests you fully trust. +* +* WARNING: requires guest_base == 0, which is qemu-user's default. Pointer +* operands are dereferenced as host addresses directly, and the invoked host +* functions dereference guest pointers with no address translation, so guest +* and host must share a single address space. A non-zero guest_base (e.g. set +* via -B/-R) would make every pointer off by guest_base and hit unrelated +* host memory. +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include <assert.h> +#include <errno.h> +#include <string.h> +#include <stdio.h> +#include <glib.h> +#include <dlfcn.h> + +#include <qemu-plugin.h> + +QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION; + +/* +* The magic system call number for dlcall. +* +* It defaults to DLCALL_SYSCALL_DEFAULT and can be overridden at load time +* with the "syscall_num=N" argument. To avoid hijacking a real syscall the +* guest might issue, N must be at least DLCALL_SYSCALL_MIN: every Linux ABI +* keeps its syscall numbers well below this; numbers from here up are free. +*/ +enum { +DLCALL_SYSCALL_DEFAULT =4096 , +DLCALL_SYSCALL_MIN =4096 , +}; + +static int64_t dlcall_syscall_num =DLCALL_SYSCALL_DEFAULT ; + +/* +* dlcall calling convention. +* +* The guest issues the magic system call (dlcall_syscall_num). The first +* argument (a1) is one of the call IDs below; the remaining arguments (a2, a3, +* a4, ...) are that ID's operands. All pointer operands are guest virtual +* addresses that the plugin dereferences as host addresses directly (see the +* guest_base requirement above). Results are written back through +* caller-provided "out" pointers rather than returned in the syscall value. +* +* The syscall return value (*sysret) only reports dispatch status: 0 on a +* recognised ID, -EINVAL for an unknown one. The actual success/failure of an +* operation (e.g. a NULL handle from dlopen) is delivered through its out +* pointer, exactly like the underlying libdl call. +* +* Operands per ID: +* +* DLCALL_ID_GET_HOST_ATTRIBUTE +* a2 const char *key in: attribute name to query +* a3 const char **attr_ptr out: matching value, or NULL if unknown +* +* DLCALL_ID_LOAD_LIBRARY (wraps dlopen) +* a2 const char *path in: library path +* a3 int flags in: dlopen() flags (e.g. RTLD_NOW) +* a4 void **handle_ptr out: library handle, or NULL on failure +* +* DLCALL_ID_GET_PROC_ADDRESS (wraps dlsym) +* a2 void *handle in: library handle +* a3 const char *name in: symbol name +* a4 void **entry_ptr out: symbol address, or NULL if not found +* +* DLCALL_ID_FREE_LIBRARY (wraps dlclose) +* a2 void *handle in: library handle +* a3 int *ret_ptr out: dlclose() return value (0 on success) +* +* DLCALL_ID_GET_LIBRARY_ERROR (wraps dlerror) +* a2 const char **error_ptr out: last libdl error string, or NULL +* +* DLCALL_ID_INVOKE_PROC (calls the symbol) +* a2 void *proc in: function pointer, signature +* void (*)(void *arg1, void *arg2) +* a3 void *arg1 in: first argument forwarded to proc +* a4 void *arg2 in: second argument forwarded to proc +*/ +enum DlcallID { +DLCALL_ID_GET_HOST_ATTRIBUTE , +DLCALL_ID_LOAD_LIBRARY , +DLCALL_ID_GET_PROC_ADDRESS , +DLCALL_ID_FREE_LIBRARY , +DLCALL_ID_GET_LIBRARY_ERROR , +DLCALL_ID_INVOKE_PROC , +}; + +static inline const char *query_host_attribute (const char *key) +{ +if (strcmp (key,"emu" ) ==0 ) { +return "qemu" ; + } +return NULL ; +} + +static inline void invoke_proc (void *proc,void *arg1,void *arg2) +{ +typedef void (*Func )(void */*arg1*/ ,void */*arg2*/ ); +Func func = (Func ) proc; +func (arg1, arg2); +} + +static bool vcpu_syscall_filter (unsigned int vcpu_index, +int64_t num,uint64_t a1,uint64_t a2, +uint64_t a3,uint64_t a4,uint64_t a5, +uint64_t a6,uint64_t a7,uint64_t a8, +int64_t *sysret,void *userdata) +{ +if (num == dlcall_syscall_num) { +switch (a1) { +/* Query host attribute by a reserved key. */ +case DLCALL_ID_GET_HOST_ATTRIBUTE : { +const char *key = (const char *) a2; +const char **attr_ptr = (const char **) a3; +assert (attr_ptr); + *attr_ptr =query_host_attribute (key); + *sysret =0 ; +break ; + } + +/* Load a shared library. */ +case DLCALL_ID_LOAD_LIBRARY : { +const char *path = (const char *) a2; +int flags = (int ) a3; +void **handle_ptr = (void **) a4; +assert (handle_ptr); + *handle_ptr =dlopen (path, flags); + *sysret =0 ; +break ; + } + +/* Get the address of a function in a shared library. */ +case DLCALL_ID_GET_PROC_ADDRESS : { +void *handle = (void *) a2; +const char *name = (const char *) a3; +void **entry_ptr = (void **) a4; +assert (entry_ptr); + *entry_ptr =dlsym (handle, name); + *sysret =0 ; +break ; + } + +/* Free a shared library. */ +case DLCALL_ID_FREE_LIBRARY : { +void *handle = (void *) a2; +int *ret_ptr = (int *) a3; + *ret_ptr =dlclose (handle); + *sysret =0 ; +break ; + } + +/* Get the last error message for a library event. */ +case DLCALL_ID_GET_LIBRARY_ERROR : { +const char **error_ptr = (const char **) a2; + *error_ptr =dlerror (); + *sysret =0 ; +break ; + } + +/* Invoke a function of a common interface. */ +case DLCALL_ID_INVOKE_PROC : { +void *proc = (void *) a2; +void *arg1 = (void *) a3; +void *arg2 = (void *) a4; +assert (proc); +invoke_proc (proc, arg1, arg2); + *sysret =0 ; +break ; + } + +default : + *sysret = -EINVAL ; +break ; + } +return true; + } +return false; +} + +QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id, +const qemu_info_t *info, +int argc,char **argv) +{ +if (info->system_emulation) { +fprintf (stderr,"plugin dlcall: only useful for user emulation\n" ); +return -1 ; + } + +for (int i =0 ; i < argc; i++) { +char *opt = argv[i]; +g_auto (GStrv) tokens =g_strsplit (opt,"=" ,2 ); +if (g_strcmp0 (tokens[0 ],"syscall_num" ) ==0 ) { +const char *val = tokens[1 ]; +char *endptr =NULL ; +guint64 num; +if (!val || *val =='\0' ) { +fprintf (stderr, +"plugin dlcall: missing value for syscall_num\n" ); +return -1 ; + } + num =g_ascii_strtoull (val, &endptr,0 ); +if (*endptr !='\0' ||g_strrstr (val,"-" ) !=NULL ) { +fprintf (stderr, +"plugin dlcall: invalid syscall_num '%s'\n" , val); +return -1 ; + } +if (num <DLCALL_SYSCALL_MIN || num >G_MAXINT64 ) { +fprintf (stderr, +"plugin dlcall: syscall_num %s is out of range; " +"it must be >= %d to avoid clashing with a real " +"syscall\n" , val,DLCALL_SYSCALL_MIN ); +return -1 ; + } + dlcall_syscall_num = (int64_t ) num; + }else { +fprintf (stderr,"plugin dlcall: unknown option '%s'\n" , opt); +return -1 ; + } + } + +qemu_plugin_register_vcpu_syscall_filter_cb (id, vcpu_syscall_filter,NULL ); + +return 0 ; +}
contrib/plugins/meson.build +5 -0
@@ -19,6 +19,11 @@ if host_os != 'windows' contrib_plugins += 'lockstep.c' endif +if host_os == 'linux' + # dlcall passes guest calls through to host libraries; linux-user only + contrib_plugins += 'dlcall.c' +endif + if 'cpp' in all_languages contrib_plugins += 'cpp.cpp' endif
Signature · 1 line
2.34.1
PATCH v5 2/2 docs/about/emulation: document the dlcall plugin
In reply to Ziyang ZhangDocument the dlcall plugin under Example Plugins: what it does, the trusted- guests and guest_base == 0 constraints, how to load it, and a pointer to Lorelei, a complete userspace implementation, for prebuilt thunks and a runnable example.
--- docs/about/emulation.rst | 55 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+)
docs/about/emulation.rst +55 -0
@@ -1046,6 +1046,61 @@ Count traps This plugin counts the number of interrupts (asynchronous events), exceptions (synchronous events) and host calls (e.g. semihosting) per cpu. +Dlcall +...... + +``contrib/plugins/dlcall.c`` + +This plugin lets a linux-user guest call functions in the host's native shared +libraries instead of emulating them, turning slow instruction-by-instruction +emulation of a library into a native host call. For example, the stock zlib +``minizip`` utility compresses several times faster, because the actual +``deflate`` runs on the host's native libz while ``minizip`` itself stays +emulated. + +The guest issues a reserved "magic" system call (4096 by default, configurable +with ``syscall_num=N``) whose first argument selects a pass-through operation: +dlopen/dlclose a host library, dlsym a symbol, and invoke a resolved host +function. The plugin performs the operation on the host and consumes the +syscall, so the real kernel never sees it. + +.. warning:: + + Trusted guests only. The guest can load arbitrary host libraries and run + arbitrary code in the QEMU host process. The plugin is not a sandbox and + provides no isolation. It also requires ``guest_base == 0`` (qemu-user's + default), as guest pointers are dereferenced as host addresses with no + translation. + +The plugin is deliberately minimal and knows nothing about any particular +library or its calling convention. Turning a real library into working thunks, +including argument marshalling, callbacks and variadic functions, is done +entirely in userspace, and any toolchain can implement the interface. +`Lorelei <https://github.com/rover2024/lorelei>`_ is a complete, MIT-licensed +implementation: it provides the guest and host runtimes and a Thunk Library +Compiler that generates the thunks from a library's headers, with prebuilt +thunks (zlib, lzma, ...) for x86_64, aarch64 and riscv64 hosts. + +Loading the plugin is all that is required from QEMU's side: + +.. code-block:: shell + + qemu-x86_64 -plugin contrib/plugins/libdlcall.so <guest-program> ... + +See `Lorelei <https://github.com/rover2024/lorelei>`_ for prebuilt thunk trees, +the runtime environment they expect, and a full runnable example (for instance, +running an unmodified ``minizip`` over the host's native libz). + +.. list-table:: Dlcall plugin arguments + :widths: 20 80 + :header-rows: 1 + + * - Option + - Description + * - syscall_num=N + - The magic syscall number the guest issues (default 4096). Must be high + enough not to clash with a real syscall. + Other emulation features ------------------------
Signature · 1 line
2.34.1
Re: [PATCH v5 2/2] docs/about/emulation: document the dlcall plugin
Quoted 20 lines · Ziyang Zhang
Document the dlcall plugin under Example Plugins: what it does, the trusted- guests and
guest_base== 0 constraints, how to load it, and a pointer to Lorelei, a complete userspace implementation, for prebuilt thunks and a runnable example.Signed-off-by Ziyang Zhang <[email protected]>--- docs/about/emulation.rst | 55 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+)docs/about/emulation.rst +2
-0@@ -1046,6 +1046,61 @@ Count traps This plugin counts the number of interrupts (asynchronous events), exceptions (synchronous events) and host calls (e.g. semihosting) per cpu. +Dlcall +......
docs/about/emulation.rst Count traps · line 1060
Quoted 10 lines
docs/about/emulation.rst +10
-0+ +``contrib/plugins/dlcall.c`` + +This plugin lets a linux-user guest call functions in the host's native shared +libraries instead of emulating them, turning slow instruction-by-instruction +emulation of a library into a native host call. For example, the stock zlib +``minizip`` utility compresses several times faster, because the actual +``deflate`` runs on the host's native libz while ``minizip`` itself stays +emulated. +
From what I understand, dlcall is just a function call interception mechanism. It happens that it's used for accelerating execution by leveraging native libraries, but we could also leverage the same interface to implement something like ftrace.
Thus, the introduction should focus on that.
Quoted 13 lines
docs/about/emulation.rst +13
-0+The guest issues a reserved "magic" system call (4096 by default, configurable +with ``syscall_num=N``) whose first argument selects a pass-through operation: +dlopen/dlclose a host library, dlsym a symbol, and invoke a resolved host +function. The plugin performs the operation on the host and consumes the +syscall, so the real kernel never sees it. + +.. warning:: + + Trusted guests only. The guest can load arbitrary host libraries and run + arbitrary code in the QEMU host process. The plugin is not a sandbox and + provides no isolation. It also requires ``guest_base == 0`` (qemu-user's + default), as guest pointers are dereferenced as host addresses with no + translation.
docs/about/emulation.rst Count traps · line 1083
Quoted 10 lines
docs/about/emulation.rst +10
-0+ +The plugin is deliberately minimal and knows nothing about any particular +library or its calling convention. Turning a real library into working thunks, +including argument marshalling, callbacks and variadic functions, is done +entirely in userspace, and any toolchain can implement the interface. +`Lorelei <https://github.com/rover2024/lorelei>`_ is a complete, MIT-licensed +implementation: it provides the guest and host runtimes and a Thunk Library +Compiler that generates the thunks from a library's headers, with prebuilt +thunks (zlib, lzma, ...) for x86_64, aarch64 and riscv64 hosts. +
An example of usage for this plugin is Lorelei, which can accelerate execution by leveraging native libraries for function calls, similar to box64. (+ example you mention in introduction).
Quoted 13 lines
docs/about/emulation.rst +13
-0+Loading the plugin is all that is required from QEMU's side: + +.. code-block:: shell + + qemu-x86_64 -plugin contrib/plugins/libdlcall.so <guest-program> ... + +See `Lorelei <https://github.com/rover2024/lorelei>`_ for prebuilt thunk trees, +the runtime environment they expect, and a full runnable example (for instance, +running an unmodified ``minizip`` over the host's native libz). + +.. list-table:: Dlcall plugin arguments + :widths: 20 80 + :header-rows: 1
docs/about/emulation.rst Count traps · line 1106
Quoted 10 lines
docs/about/emulation.rst +7
-0+ + * - Option + - Description + * - syscall_num=N + - The magic syscall number the guest issues (default 4096). Must be high + enough not to clash with a real syscall. + Other emulation features ------------------------
Regards,
Pierrick
Quoted 6 lines · Ziyang Zhang
Hi,
This patch adds a single plugin,
contrib/plugins/dlcall.c(~240 lines, no changes to QEMU core), that lets a linux-user guest call functions in the host's native shared libraries instead of emulating them.
This PATCH is deprecated. I will add the ready-to-use toolchain and submit another PATCH very soon.
Quoted 8 lines
Changes since v4:
* Added a documentation patch for
docs/about/emulation.rst(Pierrick Bouvier): what the plugin does, its trusted-guests andguest_base== 0 constraints, how to load it, and a minimal example, with a pointer to Lorelei.
* The plugin's header comment now points to Lorelei as one complete userspace implementation. The plugin code is otherwise unchanged.