Skip to contentSkip to conversation

contrib/plugins: add a minimal passthrough plugin

RFC PATCH Superseded Ziyang Zhang 2 participants 9 messages Jun 17 – Jun 22 Arm/Amlogic Meson SoC support

v1 of 12 Superseded Latest is v12 →

lore
Share & tools
lore
#0 lore raw

cover contrib/plugins: add a minimal passthrough plugin

Hi,

This RFC adds a single plugin, contrib/plugins/passthrough.c (under 200 lines, no changes to QEMU core), that lets a linux-user guest call functions in the host's native shared libraries instead of emulating them.

It is the natural next step on top of the vCPU syscall-filter callback that I contributed and that was merged earlier:

https://lore.kernel.org/qemu-devel/[email protected]/

Why bother? Because it turns slow, instruction-by-instruction emulation of a library into a native host call. Some results, all on completely unmodified guest binaries:

* minizip (the stock zlib utility) compresses several times faster, because the actual deflate runs natively on the host instead of being translated.
* Real OpenGL/Vulkan games run under qemu-user: SuperTuxKart and Hollow Knight are playable, with their graphics calls going straight to the host GPU.

You can watch the demos build, run, and report timings in CI, without checking anything out:

https://github.com/rover2024/qemu-passthrough-test/actions/runs/27671747420

How it works
============

The guest makes a system call with a reserved number (4096) that no real Linux ABI uses. Its first argument selects a pass-through operation; the rest carry operands:

syscall(4096, op, arg1, arg2, ...)

          |     |    \............ operands (pointers / values)
          |     \................. which pass-through operation
          \....................... the reserved "magic" number

The plugin registers a vCPU syscall filter: before QEMU forwards a syscall to the host kernel, the filter runs, sees 4096, performs the operation on the host, writes the result back, and tells QEMU the syscall is consumed, so the real kernel never sees it.

The whole interface is just a handful of primitives:

* query a host attribute
* dlopen / dlclose a host shared library
* dlsym a symbol, and read the last dlerror
* invoke a resolved host function with a void(void *, void *) signature

That is all the plugin does. It knows nothing about zlib, X11 or OpenGL, or about any library's calling convention.

The same machinery also runs in reverse: when a host function needs to call back into the guest (a qsort comparator, an allocator, a GUI or game callback), control re-enters the guest to run the callback and then resumes the suspended host call. This reentry is what lets stateful, callback-driven APIs work, not just leaf functions.

Why the plugin belongs in QEMU, and the rest does not
=====================================================

Only the plugin lives in the tree. Everything else is ordinary userspace:

--- userspace (out of tree, not tied to any DBT) -------------

      guest: unmodified program  ->  guest runtime + thunk libs
  --------------------------------------------------------------
                 |  syscall(4096, op, args)   (only crossing point)
                 v
  === inside QEMU: THIS PATCH, < 200 lines =====================
      passthrough plugin:  dlopen / dlsym / invoke a host fn
  ==============================================================
                 |
                 v
  --- userspace (out of tree) ----------------------------------
      host: host runtime + thunk libs  ->  real libz / libGL ...

--------------------------------------------------------------

A complete reference implementation, with the minizip and OpenGL/X11 examples above, is here:

https://github.com/rover2024/qemu-passthrough-test

The split is deliberate, and it is why only this one file is proposed for the tree:

* This plugin defines the most general interaction interface for native
pass-through: the magic-syscall ABI between an emulated guest and its emulator. That contract is what every pass-through implementation builds on, so it belongs in a stable, shared place.
* It is also the only piece that is inherently QEMU-specific: it plugs into QEMU's syscall-filter hook and runs inside the QEMU process. The argument marshalling, calling conventions, callbacks/reentry and per-library coverage are not tied to any particular DBT and behave as ordinary userspace, so they should stay out of tree rather than couple QEMU to them.

Background: we presented this approach at KVM Forum 2025, "Lorelei: Enable QEMU to Leverage Native Shared Libraries":

https://www.youtube.com/watch?v=_jioQFm7wyU

It is fully opt-in (loaded with -plugin) and targets linux-user, where the guest and host already share a trust domain. The test cases use x86_64 guests and run on x86_64, arm64 and riscv64 Linux hosts.

This is an RFC: I would welcome feedback on the plugin itself and on the pass-through approach in general.

Thanks,
Ziyang Zhang

Ziyang Zhang (1):
contrib/plugins: add a minimal passthrough plugin

 contrib/plugins/meson.build   |   1 +
 contrib/plugins/passthrough.c | 187 ++++++++++++++++++++++++++++++++++
 2 files changed, 188 insertions(+)
 create mode 100644 contrib/plugins/passthrough.c
Signature · 1 line
2.34.1
#1 lore raw

RFC PATCH 1/1 contrib/plugins: add a minimal passthrough plugin

Add a minimal passthrough plugin that lets the guest invoke host functions through magic system calls. The plugin registers a vCPU syscall filter callback that intercepts a reserved syscall number and dispatches a set of pass-through operations: querying host attributes, loading and freeing shared libraries, resolving symbols, retrieving the last library error, and invoking a host function through a common interface.

---
 contrib/plugins/meson.build   |   1 +
 contrib/plugins/passthrough.c | 187 ++++++++++++++++++++++++++++++++++
 2 files changed, 188 insertions(+)
 create mode 100644 contrib/plugins/passthrough.c
contrib/plugins/meson.build +1 -0
index 099319e7a1..5bfeb1cd60 100644
@@ -9,6 +9,7 @@ contrib_plugins = [
 'howvec.c',
 'hwprofile.c',
 'ips.c',
+'passthrough.c',
 'stoptrigger.c',
 'traps.c',
 'uftrace.c',
contrib/plugins/passthrough.c new file +187 -0

187 lines not on this page · raw message

Signature · 1 line
2.34.1
#2 lore raw
Quoted 210 lines · Ziyang Zhang

Add a minimal passthrough plugin that lets the guest invoke host functions through magic system calls. The plugin registers a vCPU syscall filter callback that intercepts a reserved syscall number and dispatches a set of pass-through operations: querying host attributes, loading and freeing shared libraries, resolving symbols, retrieving the last library error, and invoking a host function through a common interface.

---
 contrib/plugins/meson.build   |   1 +
 contrib/plugins/passthrough.c | 187 ++++++++++++++++++++++++++++++++++
 2 files changed, 188 insertions(+)
 create mode 100644 contrib/plugins/passthrough.c
contrib/plugins/meson.build +1 -0
index 099319e7a1..5bfeb1cd60 100644
@@ -9,6 +9,7 @@ contrib_plugins = [
 'howvec.c',
 'hwprofile.c',
 'ips.c',
+'passthrough.c',
 'stoptrigger.c',
 'traps.c',
 'uftrace.c',
contrib/plugins/passthrough.c new file +177 -0
new file mode 100644
index 0000000000..f2db9acdfe
@@ -0,0 +1,187 @@
+/*
+ * Copyright (C) 2026, Ziyang Zhang <[email protected]>
+ *
+ * Passthrough Plugin: lets a guest invoke host functions via a magic
+ * system call. This grants the guest full host access (dlopen/dlsym and
+ * arbitrary function calls), so use it only with trusted guests.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include <assert.h>
+#include <errno.h>
+#include <string.h>
+#include <stdio.h>
+#include <glib.h>
+#include <dlfcn.h>
+
+#include <qemu-plugin.h>
+
+QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
+
+/* The magic system call number for pass-through. */
+enum {
+    SYSCALL_PASSTHROUGH_NUMBER = 4096,
+};
+
+/*
+ * Pass-through calling convention.
+ *
+ * The guest issues system call SYSCALL_PASSTHROUGH_NUMBER. The first syscall
+ * argument (a1) is one of the call IDs below; the remaining arguments (a2, a3,
+ * a4, ...) are that ID's operands. All pointer operands are guest virtual
+ * addresses that the plugin dereferences as host addresses directly. This
+ * assumes guest_base is 0, so the guest and host address spaces coincide;
+ * with a non-zero guest_base every pointer operand would be off by guest_base
+ * and the dereferences would hit unrelated host memory. Results are written
+ * back through caller-provided "out" pointers rather than returned in the
+ * syscall value.
+ *
+ * The syscall return value (*sysret) only reports dispatch status: 0 on a
+ * recognised ID, -EINVAL for an unknown one. The actual success/failure of an
+ * operation (e.g. a NULL handle from dlopen) is delivered through its out
+ * pointer, exactly like the underlying libdl call.
+ *
+ * Operands per ID:
+ *
+ *   PASSTHROUGH_ID_GET_HOST_ATTRIBUTE
+ *     a2  const char *key        in:  attribute name to query
+ *     a3  const char **attr_ptr  out: matching value, or NULL if unknown
+ *
+ *   PASSTHROUGH_ID_LOAD_LIBRARY                            (wraps dlopen)
+ *     a2  const char *path       in:  library path
+ *     a3  int flags              in:  dlopen() flags (e.g. RTLD_NOW)
+ *     a4  void **handle_ptr      out: library handle, or NULL on failure
+ *
+ *   PASSTHROUGH_ID_GET_PROC_ADDRESS                        (wraps dlsym)
+ *     a2  void *handle           in:  library handle
+ *     a3  const char *name       in:  symbol name
+ *     a4  void **entry_ptr       out: symbol address, or NULL if not found
+ *
+ *   PASSTHROUGH_ID_FREE_LIBRARY                            (wraps dlclose)
+ *     a2  void *handle           in:  library handle
+ *     a3  int *ret_ptr           out: dlclose() return value (0 on success)
+ *
+ *   PASSTHROUGH_ID_GET_LIBRARY_ERROR                       (wraps dlerror)
+ *     a2  const char **error_ptr out: last libdl error string, or NULL
+ *
+ *   PASSTHROUGH_ID_INVOKE_PROC                             (calls the symbol)
+ *     a2  void *proc             in:  function pointer, signature
+ *                                     void (*)(void *arg1, void *arg2)
+ *     a3  void *arg1             in:  first argument forwarded to proc
+ *     a4  void *arg2             in:  second argument forwarded to proc
+ */
+enum PassThroughID {
+    PASSTHROUGH_ID_GET_HOST_ATTRIBUTE,
+    PASSTHROUGH_ID_LOAD_LIBRARY,
+    PASSTHROUGH_ID_GET_PROC_ADDRESS,
+    PASSTHROUGH_ID_FREE_LIBRARY,
+    PASSTHROUGH_ID_GET_LIBRARY_ERROR,
+    PASSTHROUGH_ID_INVOKE_PROC,
+};
+
+static inline const char *query_host_attribute(const char *key)
+{
+    if (strcmp(key, "emu") == 0) {
+        return "qemu";
+    }
+    return NULL;
+}
+
+static inline void invoke_proc(void *proc, void *arg1, void *arg2)
+{
+    typedef void (*Func)(void * /*arg1*/, void * /*arg2*/);
+    Func func = (Func) proc;
+    func(arg1, arg2);
+}
+
+static bool vcpu_syscall_filter(qemu_plugin_id_t id, unsigned int vcpu_index,
+                                int64_t num, uint64_t a1, uint64_t a2,
+                                uint64_t a3, uint64_t a4, uint64_t a5,
+                                uint64_t a6, uint64_t a7, uint64_t a8,
+                                uint64_t *sysret)
+{
+    if (num == SYSCALL_PASSTHROUGH_NUMBER) {
+        switch (a1) {
+        /* Query host attribute by a reserved key. */
+        case PASSTHROUGH_ID_GET_HOST_ATTRIBUTE: {
+            const char *key = (const char *) a2;
+            const char **attr_ptr = (const char **) a3;
+            assert(attr_ptr);
+            *attr_ptr = query_host_attribute(key);
+            *sysret = 0;
+            break;
+        }
+
+        /* Load a shared library. */
+        case PASSTHROUGH_ID_LOAD_LIBRARY: {
+            const char *path = (const char *) a2;
+            int flags = (int) a3;
+            void **handle_ptr = (void **) a4;
+            assert(handle_ptr);
+            *handle_ptr = dlopen(path, flags);
+            *sysret = 0;
+            break;
+        }
+
+        /* Get the address of a function in a shared library. */
+        case PASSTHROUGH_ID_GET_PROC_ADDRESS: {
+            void *handle = (void *) a2;
+            const char *name = (const char *) a3;
+            void **entry_ptr = (void **) a4;
+            assert(entry_ptr);
+            *entry_ptr = dlsym(handle, name);
+            *sysret = 0;
+            break;
+        }
+
+        /* Free a shared library. */
+        case PASSTHROUGH_ID_FREE_LIBRARY: {
+            void *handle = (void *) a2;
+            int *ret_ptr = (int *) a3;
+            *ret_ptr = dlclose(handle);
+            *sysret = 0;
+            break;
+        }
+
+        /* Get the last error message for a library event. */
+        case PASSTHROUGH_ID_GET_LIBRARY_ERROR: {
+            const char **error_ptr = (const char **) a2;
+            *error_ptr = dlerror();
+            *sysret = 0;
+            break;
+        }
+
+        /* Invoke a function of a common interface. */
+        case PASSTHROUGH_ID_INVOKE_PROC: {
+            void *proc = (void *) a2;
+            void *arg1 = (void *) a3;
+            void *arg2 = (void *) a4;
+            assert(proc);
+            invoke_proc(proc, arg1, arg2);
+            *sysret = 0;
+            break;
+        }
+
+        default:
+            *sysret = -EINVAL;
+            break;
+        }
+        return true;
+    }
+    return false;
+}
+
+QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id,
+                                           const qemu_info_t *info,
+                                           int argc, char **argv)

contrib/plugins/passthrough.c line 187

Quoted 10 lines
contrib/plugins/passthrough.c new file +10 -0
+{
+    if (info->system_emulation) {
+        fprintf(stderr, "plugin passthrough: only useful for user emulation\n");
+        return -1;
+    }
+
+    qemu_plugin_register_vcpu_syscall_filter_cb(id, vcpu_syscall_filter);
+
+    return 0;
+}

I don't have any specific comment on the plugin itself, it's pretty clear and straightforward.

Maybe you might want to be able to tune syscall number with a parameter, but it's definitely just a suggestion.

Regards,
Pierrick

#3 lore raw
In reply to Ziyang Zhang

Hi Ziyang,

Quoted 21 lines · Ziyang Zhang

Hi,

This RFC adds a single plugin, contrib/plugins/passthrough.c (under 200 lines, no changes to QEMU core), that lets a linux-user guest call functions in the host's native shared libraries instead of emulating them.

It is the natural next step on top of the vCPU syscall-filter callback that I contributed and that was merged earlier:

https://lore.kernel.org/qemu-devel/[email protected]/

Why bother? Because it turns slow, instruction-by-instruction emulation of a library into a native host call. Some results, all on completely unmodified guest binaries:

* minizip (the stock zlib utility) compresses several times faster, because the actual deflate runs natively on the host instead of being translated.
* Real OpenGL/Vulkan games run under qemu-user: SuperTuxKart and Hollow Knight are playable, with their graphics calls going straight to the host GPU.

Quoted 5 lines

You can watch the demos build, run, and report timings in CI, without checking anything out:

https://github.com/rover2024/qemu-passthrough-test/actions/runs/27671747420

Thanks for doing this reproduction effort, that's nice to be able to see the result without having to compile that ourselves.

Quoted 24 lines

How it works
============

The guest makes a system call with a reserved number (4096) that no real Linux ABI uses. Its first argument selects a pass-through operation; the rest carry operands:

syscall(4096, op, arg1, arg2, ...)

          |     |    \............ operands (pointers / values)
          |     \................. which pass-through operation
          \....................... the reserved "magic" number

The plugin registers a vCPU syscall filter: before QEMU forwards a syscall to the host kernel, the filter runs, sees 4096, performs the operation on the host, writes the result back, and tells QEMU the syscall is consumed, so the real kernel never sees it.

The whole interface is just a handful of primitives:

* query a host attribute
* dlopen / dlclose a host shared library
* dlsym a symbol, and read the last dlerror
* invoke a resolved host function with a void(void *, void *) signature

Quoted 9 lines

That is all the plugin does. It knows nothing about zlib, X11 or OpenGL, or about any library's calling convention.

The same machinery also runs in reverse: when a host function needs to call back into the guest (a qsort comparator, an allocator, a GUI or game callback), control re-enters the guest to run the callback and then resumes the suspended host call. This reentry is what lets stateful, callback-driven APIs work, not just leaf functions.

That is a nice and clear interface, great!
I think it's something you can easily port to any other instrumentation framework, once you access to any kind of "special" calls.

Quoted 40 lines

Why the plugin belongs in QEMU, and the rest does not
=====================================================

Only the plugin lives in the tree. Everything else is ordinary userspace:

--- userspace (out of tree, not tied to any DBT) -------------

      guest: unmodified program  ->  guest runtime + thunk libs
  --------------------------------------------------------------
                 |  syscall(4096, op, args)   (only crossing point)
                 v
  === inside QEMU: THIS PATCH, < 200 lines =====================
      passthrough plugin:  dlopen / dlsym / invoke a host fn
  ==============================================================
                 |
                 v
  --- userspace (out of tree) ----------------------------------
      host: host runtime + thunk libs  ->  real libz / libGL ...

--------------------------------------------------------------

A complete reference implementation, with the minizip and OpenGL/X11 examples above, is here:

https://github.com/rover2024/qemu-passthrough-test

The split is deliberate, and it is why only this one file is proposed for the tree:

* This plugin defines the most general interaction interface for native
pass-through: the magic-syscall ABI between an emulated guest and its emulator. That contract is what every pass-through implementation builds on, so it belongs in a stable, shared place.
* It is also the only piece that is inherently QEMU-specific: it plugs into QEMU's syscall-filter hook and runs inside the QEMU process. The argument marshalling, calling conventions, callbacks/reentry and per-library coverage are not tied to any particular DBT and behave as ordinary userspace, so they should stay out of tree rather than couple QEMU to them.

Background: we presented this approach at KVM Forum 2025, "Lorelei: Enable QEMU to Leverage Native Shared Libraries":

Quoted 9 lines

https://www.youtube.com/watch?v=_jioQFm7wyU

It is fully opt-in (loaded with -plugin) and targets linux-user, where the guest and host already share a trust domain. The test cases use x86_64 guests and run on x86_64, arm64 and riscv64 Linux hosts.

This is an RFC: I would welcome feedback on the plugin itself and on the pass-through approach in general.

What will be your next steps from this?
From what I understand, the current passthrough example has entirely been written by hand, and does not use any specific toolchain like the approach you mention before. Do you plan to use that again?

Once this API is stable is fixed and document, you'll be able to add that to different instrumentation frameworks. You'll need a nice name also, or at least something less generic than "passthrough" :).

Quoted 11 lines

Thanks,
Ziyang Zhang

Ziyang Zhang (1):
contrib/plugins: add a minimal passthrough plugin

 contrib/plugins/meson.build   |   1 +
 contrib/plugins/passthrough.c | 187 ++++++++++++++++++++++++++++++++++
 2 files changed, 188 insertions(+)
 create mode 100644 contrib/plugins/passthrough.c

Regards,
Pierrick

#4 lore raw

v2 cover contrib/plugins: add a minimal dlcall plugin

In reply to Ziyang Zhang

Hi all,

This RFC adds a single plugin, contrib/plugins/dlcall.c (~230 lines, no changes to QEMU core), that lets a linux-user guest call functions in the host's native shared libraries instead of emulating them.

It is the natural next step on top of the vCPU syscall-filter callback that I contributed and that was merged earlier:

https://lore.kernel.org/qemu-devel/[email protected]/

Why bother? Because it turns slow, instruction-by-instruction emulation of a library into a native host call. Some results, all on completely unmodified guest binaries:

* minizip (the stock zlib utility) compresses several times faster, because the actual deflate runs natively on the host instead of being translated.
* Real OpenGL/Vulkan games run under qemu-user: SuperTuxKart and Hollow Knight are playable, with their graphics calls going straight to the host GPU.

You can watch the demos build, run, and report timings in CI, without checking anything out:

https://github.com/rover2024/qemu-passthrough-test/actions/runs/27671747420

How it works
============

The guest makes a system call with a reserved number (4096 by default) that no real Linux ABI uses. Its first argument selects a pass-through operation; the rest carry operands:

syscall(4096, op, arg1, arg2, ...)

          |     |    \............ operands (pointers / values)
          |     \................. which pass-through operation
          \....................... the reserved "magic" number

The plugin registers a vCPU syscall filter: before QEMU forwards a syscall to the host kernel, the filter runs, sees 4096, performs the operation on the host, writes the result back, and tells QEMU the syscall is consumed, so the real kernel never sees it.

The whole interface is just a handful of primitives:

* query a host attribute
* dlopen / dlclose a host shared library
* dlsym a symbol, and read the last dlerror
* invoke a resolved host function with a void(void *, void *) signature

That is all the plugin does. It knows nothing about zlib, X11 or OpenGL, or about any library's calling convention.

The same machinery also runs in reverse: when a host function needs to call back into the guest (a qsort comparator, an allocator, a GUI or game callback), control re-enters the guest to run the callback and then resumes the suspended host call. This reentry is what lets stateful, callback-driven APIs work, not just leaf functions.

Why the plugin belongs in QEMU, and the rest does not
=====================================================

Only the plugin lives in the tree. Everything else is ordinary userspace:

--- userspace (out of tree, not tied to any DBT) -------------

      guest: unmodified program  ->  guest runtime + thunk libs
  --------------------------------------------------------------
                 |  syscall(4096, op, args)   (only crossing point)
                 v
  === inside QEMU: THIS PATCH, ~230 lines ======================
      dlcall plugin:  dlopen / dlsym / invoke a host fn
  ==============================================================
                 |
                 v
  --- userspace (out of tree) ----------------------------------
      host: host runtime + thunk libs  ->  real libz / libGL ...

--------------------------------------------------------------

A complete reference implementation, with the minizip and OpenGL/X11 examples above, is here:

https://github.com/rover2024/qemu-passthrough-test

The split is deliberate, and it is why only this one file is proposed for the tree:

* This plugin defines the most general interaction interface for native
pass-through: the magic-syscall ABI between an emulated guest and its emulator. That contract is what every pass-through implementation builds on, so it belongs in a stable, shared place.
* It is also the only piece that is inherently QEMU-specific: it plugs into QEMU's syscall-filter hook and runs inside the QEMU process. The argument marshalling, calling conventions, callbacks/reentry and per-library coverage are not tied to any particular DBT and behave as ordinary userspace, so they should stay out of tree rather than couple QEMU to them.

Background: we presented this approach at KVM Forum 2025, "Lorelei: Enable QEMU to Leverage Native Shared Libraries":

https://www.youtube.com/watch?v=_jioQFm7wyU

A note on automation
====================

The userspace thunks in that reference implementation are currently hand-written rather than generated by the LLVM-based toolchain from the talk. That is a deliberate choice for a demo: the automated toolchain pulls in a full LLVM installation, which adds substantial setup time, and the example projects are slow to build and cannot be reduced to a single Makefile. Hand-writing the thunks was the cheaper path to a self-contained, reproducible demo -- and it was already enough to get minizip working end to end.

For real, large-scale use I will rely on the automated toolchain. The key point is that hand-written vs. generated thunks is entirely decoupled from this plugin and from the magic-syscall interface it defines: the toolchain only emits out-of-tree userspace code and never touches the in-tree plugin. Automation becomes mandatory for complex, callback-heavy targets such as the OpenGL/Vulkan games, which is the direction this work is heading next.

It is fully opt-in (loaded with -plugin) and targets linux-user, where the guest and host already share a trust domain. The test cases use x86_64 guests and run on x86_64, arm64 and riscv64 Linux hosts.

This is an RFC: I would welcome feedback on the plugin itself and on the pass-through approach in general.

Changes since v1:

* Renamed the plugin from "passthrough" to "dlcall" (Pierrick Bouvier). The old name was too generic; "dlcall" reflects what the plugin actually does (dlopen/dlsym a host symbol and call it) and avoids confusion with QEMU's existing plugin hostcall concept (QEMU_PLUGIN_*_HOSTCALL).
* Made the magic syscall number configurable at load time via the "syscall_num=N" argument, defaulting to 4096 and rejecting values low enough to clash with a real syscall (Pierrick Bouvier).

v1: https://lore.kernel.org/qemu-devel/[email protected]/

Thanks,
Ziyang

Ziyang Zhang (1):
contrib/plugins: add a minimal dlcall plugin

 contrib/plugins/dlcall.c    | 229 ++++++++++++++++++++++++++++++++++++
 contrib/plugins/meson.build |   1 +
 2 files changed, 230 insertions(+)
 create mode 100644 contrib/plugins/dlcall.c
Signature · 1 line
2.34.1
#5 lore raw

RFC PATCH v2 1/1 contrib/plugins: add a minimal dlcall plugin

Add a minimal dlcall plugin that lets the guest invoke host functions through magic system calls. The plugin registers a vCPU syscall filter callback that intercepts a reserved syscall number and dispatches a set of pass-through operations: querying host attributes, loading and freeing shared libraries, resolving symbols, retrieving the last library error, and invoking a host function through a common interface.

The magic syscall number defaults to 4096 and can be overridden at load time with the "syscall_num=N" argument; values low enough to clash with a real syscall are rejected.

---
 contrib/plugins/dlcall.c    | 229 ++++++++++++++++++++++++++++++++++++
 contrib/plugins/meson.build |   1 +
 2 files changed, 230 insertions(+)
 create mode 100644 contrib/plugins/dlcall.c
contrib/plugins/dlcall.c new file +229 -0
new file mode 100644
index 0000000000..c48907fcd5
@@ -0,0 +1,229 @@
+/*
+ * Copyright (C) 2026, Ziyang Zhang <[email protected]>
+ *
+ * dlcall plugin: lets a guest invoke host functions via a magic
+ * system call. This grants the guest full host access (dlopen/dlsym and
+ * arbitrary function calls), so use it only with trusted guests.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include <assert.h>
+#include <errno.h>
+#include <string.h>
+#include <stdio.h>
+#include <glib.h>
+#include <dlfcn.h>
+
+#include <qemu-plugin.h>
+
+QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
+
+/*
+ * The magic system call number for dlcall.
+ *
+ * It defaults to DLCALL_SYSCALL_DEFAULT and can be overridden at load time
+ * with the "syscall_num=N" argument. To avoid hijacking a real syscall the
+ * guest might issue, N must be at least DLCALL_SYSCALL_MIN: every Linux ABI
+ * keeps its syscall numbers well below this, so any number from here up is free.
+ */
+enum {
+    DLCALL_SYSCALL_DEFAULT = 4096,
+    DLCALL_SYSCALL_MIN = 4096,
+};
+
+static int64_t dlcall_syscall_num = DLCALL_SYSCALL_DEFAULT;
+
+/*
+ * dlcall calling convention.
+ *
+ * The guest issues the magic system call (dlcall_syscall_num). The first
+ * argument (a1) is one of the call IDs below; the remaining arguments (a2, a3,
+ * a4, ...) are that ID's operands. All pointer operands are guest virtual
+ * addresses that the plugin dereferences as host addresses directly. This
+ * assumes guest_base is 0, so the guest and host address spaces coincide;
+ * with a non-zero guest_base every pointer operand would be off by guest_base
+ * and the dereferences would hit unrelated host memory. Results are written
+ * back through caller-provided "out" pointers rather than returned in the
+ * syscall value.
+ *
+ * The syscall return value (*sysret) only reports dispatch status: 0 on a
+ * recognised ID, -EINVAL for an unknown one. The actual success/failure of an
+ * operation (e.g. a NULL handle from dlopen) is delivered through its out
+ * pointer, exactly like the underlying libdl call.
+ *
+ * Operands per ID:
+ *
+ *   DLCALL_ID_GET_HOST_ATTRIBUTE
+ *     a2  const char *key        in:  attribute name to query
+ *     a3  const char **attr_ptr  out: matching value, or NULL if unknown
+ *
+ *   DLCALL_ID_LOAD_LIBRARY                            (wraps dlopen)
+ *     a2  const char *path       in:  library path
+ *     a3  int flags              in:  dlopen() flags (e.g. RTLD_NOW)
+ *     a4  void **handle_ptr      out: library handle, or NULL on failure
+ *
+ *   DLCALL_ID_GET_PROC_ADDRESS                        (wraps dlsym)
+ *     a2  void *handle           in:  library handle
+ *     a3  const char *name       in:  symbol name
+ *     a4  void **entry_ptr       out: symbol address, or NULL if not found
+ *
+ *   DLCALL_ID_FREE_LIBRARY                            (wraps dlclose)
+ *     a2  void *handle           in:  library handle
+ *     a3  int *ret_ptr           out: dlclose() return value (0 on success)
+ *
+ *   DLCALL_ID_GET_LIBRARY_ERROR                       (wraps dlerror)
+ *     a2  const char **error_ptr out: last libdl error string, or NULL
+ *
+ *   DLCALL_ID_INVOKE_PROC                             (calls the symbol)
+ *     a2  void *proc             in:  function pointer, signature
+ *                                     void (*)(void *arg1, void *arg2)
+ *     a3  void *arg1             in:  first argument forwarded to proc
+ *     a4  void *arg2             in:  second argument forwarded to proc
+ */
+enum DlcallID {
+    DLCALL_ID_GET_HOST_ATTRIBUTE,
+    DLCALL_ID_LOAD_LIBRARY,
+    DLCALL_ID_GET_PROC_ADDRESS,
+    DLCALL_ID_FREE_LIBRARY,
+    DLCALL_ID_GET_LIBRARY_ERROR,
+    DLCALL_ID_INVOKE_PROC,
+};
+
+static inline const char *query_host_attribute(const char *key)
+{
+    if (strcmp(key, "emu") == 0) {
+        return "qemu";
+    }
+    return NULL;
+}
+
+static inline void invoke_proc(void *proc, void *arg1, void *arg2)
+{
+    typedef void (*Func)(void * /*arg1*/, void * /*arg2*/);
+    Func func = (Func) proc;
+    func(arg1, arg2);
+}
+
+static bool vcpu_syscall_filter(qemu_plugin_id_t id, unsigned int vcpu_index,
+                                int64_t num, uint64_t a1, uint64_t a2,
+                                uint64_t a3, uint64_t a4, uint64_t a5,
+                                uint64_t a6, uint64_t a7, uint64_t a8,
+                                uint64_t *sysret)
+{
+    if (num == dlcall_syscall_num) {
+        switch (a1) {
+        /* Query host attribute by a reserved key. */
+        case DLCALL_ID_GET_HOST_ATTRIBUTE: {
+            const char *key = (const char *) a2;
+            const char **attr_ptr = (const char **) a3;
+            assert(attr_ptr);
+            *attr_ptr = query_host_attribute(key);
+            *sysret = 0;
+            break;
+        }
+
+        /* Load a shared library. */
+        case DLCALL_ID_LOAD_LIBRARY: {
+            const char *path = (const char *) a2;
+            int flags = (int) a3;
+            void **handle_ptr = (void **) a4;
+            assert(handle_ptr);
+            *handle_ptr = dlopen(path, flags);
+            *sysret = 0;
+            break;
+        }
+
+        /* Get the address of a function in a shared library. */
+        case DLCALL_ID_GET_PROC_ADDRESS: {
+            void *handle = (void *) a2;
+            const char *name = (const char *) a3;
+            void **entry_ptr = (void **) a4;
+            assert(entry_ptr);
+            *entry_ptr = dlsym(handle, name);
+            *sysret = 0;
+            break;
+        }
+
+        /* Free a shared library. */
+        case DLCALL_ID_FREE_LIBRARY: {
+            void *handle = (void *) a2;
+            int *ret_ptr = (int *) a3;
+            *ret_ptr = dlclose(handle);
+            *sysret = 0;
+            break;
+        }
+
+        /* Get the last error message for a library event. */
+        case DLCALL_ID_GET_LIBRARY_ERROR: {
+            const char **error_ptr = (const char **) a2;
+            *error_ptr = dlerror();
+            *sysret = 0;
+            break;
+        }
+
+        /* Invoke a function of a common interface. */
+        case DLCALL_ID_INVOKE_PROC: {
+            void *proc = (void *) a2;
+            void *arg1 = (void *) a3;
+            void *arg2 = (void *) a4;
+            assert(proc);
+            invoke_proc(proc, arg1, arg2);
+            *sysret = 0;
+            break;
+        }
+
+        default:
+            *sysret = -EINVAL;
+            break;
+        }
+        return true;
+    }
+    return false;
+}
+
+QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id,
+                                           const qemu_info_t *info,
+                                           int argc, char **argv)
+{
+    if (info->system_emulation) {
+        fprintf(stderr, "plugin dlcall: only useful for user emulation\n");
+        return -1;
+    }
+
+    for (int i = 0; i < argc; i++) {
+        char *opt = argv[i];
+        g_auto(GStrv) tokens = g_strsplit(opt, "=", 2);
+        if (g_strcmp0(tokens[0], "syscall_num") == 0) {
+            const char *val = tokens[1];
+            char *endptr = NULL;
+            guint64 num;
+            if (!val || *val == '\0') {
+                fprintf(stderr,
+                        "plugin dlcall: missing value for syscall_num\n");
+                return -1;
+            }
+            num = g_ascii_strtoull(val, &endptr, 0);
+            if (*endptr != '\0' || g_strrstr(val, "-") != NULL) {
+                fprintf(stderr,
+                        "plugin dlcall: invalid syscall_num '%s'\n", val);
+                return -1;
+            }
+            if (num < DLCALL_SYSCALL_MIN || num > G_MAXINT64) {
+                fprintf(stderr,
+                        "plugin dlcall: syscall_num %s is out of range; "
+                        "it must be >= %d to avoid clashing with a real "
+                        "syscall\n", val, DLCALL_SYSCALL_MIN);
+                return -1;
+            }
+            dlcall_syscall_num = (int64_t) num;
+        } else {
+            fprintf(stderr, "plugin dlcall: unknown option '%s'\n", opt);
+            return -1;
+        }
+    }
+
+    qemu_plugin_register_vcpu_syscall_filter_cb(id, vcpu_syscall_filter);
+
+    return 0;
+}
contrib/plugins/meson.build +1 -0
index 099319e7a1..e17f3e5387 100644
@@ -2,6 +2,7 @@ contrib_plugins = [
 'bbv.c',
 'cache.c',
 'cflow.c',
+'dlcall.c',
 'drcov.c',
 'execlog.c',
 'hotblocks.c',
Signature · 1 line
2.34.1
#6 lore raw

Re: [RFC PATCH v2 0/1] contrib/plugins: add a minimal dlcall plugin

In reply to Ziyang Zhang
Quoted 112 lines · Ziyang Zhang

Hi all,

This RFC adds a single plugin, contrib/plugins/dlcall.c (~230 lines, no changes to QEMU core), that lets a linux-user guest call functions in the host's native shared libraries instead of emulating them.

It is the natural next step on top of the vCPU syscall-filter callback that I contributed and that was merged earlier:

https://lore.kernel.org/qemu-devel/[email protected]/

Why bother? Because it turns slow, instruction-by-instruction emulation of a library into a native host call. Some results, all on completely unmodified guest binaries:

* minizip (the stock zlib utility) compresses several times faster, because the actual deflate runs natively on the host instead of being translated.
* Real OpenGL/Vulkan games run under qemu-user: SuperTuxKart and Hollow Knight are playable, with their graphics calls going straight to the host GPU.

You can watch the demos build, run, and report timings in CI, without checking anything out:

https://github.com/rover2024/qemu-passthrough-test/actions/runs/27671747420

How it works
============

The guest makes a system call with a reserved number (4096 by default) that no real Linux ABI uses. Its first argument selects a pass-through operation; the rest carry operands:

syscall(4096, op, arg1, arg2, ...)

          |     |    \............ operands (pointers / values)
          |     \................. which pass-through operation
          \....................... the reserved "magic" number

The plugin registers a vCPU syscall filter: before QEMU forwards a syscall to the host kernel, the filter runs, sees 4096, performs the operation on the host, writes the result back, and tells QEMU the syscall is consumed, so the real kernel never sees it.

The whole interface is just a handful of primitives:

* query a host attribute
* dlopen / dlclose a host shared library
* dlsym a symbol, and read the last dlerror
* invoke a resolved host function with a void(void *, void *) signature

That is all the plugin does. It knows nothing about zlib, X11 or OpenGL, or about any library's calling convention.

The same machinery also runs in reverse: when a host function needs to call back into the guest (a qsort comparator, an allocator, a GUI or game callback), control re-enters the guest to run the callback and then resumes the suspended host call. This reentry is what lets stateful, callback-driven APIs work, not just leaf functions.

Why the plugin belongs in QEMU, and the rest does not
=====================================================

Only the plugin lives in the tree. Everything else is ordinary userspace:

--- userspace (out of tree, not tied to any DBT) -------------

      guest: unmodified program  ->  guest runtime + thunk libs
  --------------------------------------------------------------
                 |  syscall(4096, op, args)   (only crossing point)
                 v
  === inside QEMU: THIS PATCH, ~230 lines ======================
      dlcall plugin:  dlopen / dlsym / invoke a host fn
  ==============================================================
                 |
                 v
  --- userspace (out of tree) ----------------------------------
      host: host runtime + thunk libs  ->  real libz / libGL ...

--------------------------------------------------------------

A complete reference implementation, with the minizip and OpenGL/X11 examples above, is here:

https://github.com/rover2024/qemu-passthrough-test

The split is deliberate, and it is why only this one file is proposed for the tree:

* This plugin defines the most general interaction interface for native
pass-through: the magic-syscall ABI between an emulated guest and its emulator. That contract is what every pass-through implementation builds on, so it belongs in a stable, shared place.
* It is also the only piece that is inherently QEMU-specific: it plugs into QEMU's syscall-filter hook and runs inside the QEMU process. The argument marshalling, calling conventions, callbacks/reentry and per-library coverage are not tied to any particular DBT and behave as ordinary userspace, so they should stay out of tree rather than couple QEMU to them.

Background: we presented this approach at KVM Forum 2025, "Lorelei: Enable QEMU to Leverage Native Shared Libraries":

https://www.youtube.com/watch?v=_jioQFm7wyU

A note on automation
====================

The userspace thunks in that reference implementation are currently hand-written rather than generated by the LLVM-based toolchain from the talk. That is a deliberate choice for a demo: the automated toolchain pulls in a full LLVM installation, which adds substantial setup time, and the example projects are slow to build and cannot be reduced to a single Makefile. Hand-writing the thunks was the cheaper path to a self-contained, reproducible demo -- and it was already enough to get minizip working end to end.

Quoted 7 lines

For real, large-scale use I will rely on the automated toolchain. The key point is that hand-written vs. generated thunks is entirely decoupled from this plugin and from the magic-syscall interface it defines: the toolchain only emits out-of-tree userspace code and never touches the in-tree plugin. Automation becomes mandatory for complex, callback-heavy targets such as the OpenGL/Vulkan games, which is the direction this work is heading next.

Makes sense, thanks for your answer.

Quoted 30 lines

It is fully opt-in (loaded with -plugin) and targets linux-user, where the guest and host already share a trust domain. The test cases use x86_64 guests and run on x86_64, arm64 and riscv64 Linux hosts.

This is an RFC: I would welcome feedback on the plugin itself and on the pass-through approach in general.

Changes since v1:

* Renamed the plugin from "passthrough" to "dlcall" (Pierrick Bouvier). The old name was too generic; "dlcall" reflects what the plugin actually does (dlopen/dlsym a host symbol and call it) and avoids confusion with QEMU's existing plugin hostcall concept (QEMU_PLUGIN_*_HOSTCALL).
* Made the magic syscall number configurable at load time via the "syscall_num=N" argument, defaulting to 4096 and rejecting values low enough to clash with a real syscall (Pierrick Bouvier).

v1: https://lore.kernel.org/qemu-devel/[email protected]/

Thanks,
Ziyang

Ziyang Zhang (1):
contrib/plugins: add a minimal dlcall plugin

 contrib/plugins/dlcall.c    | 229 ++++++++++++++++++++++++++++++++++++
 contrib/plugins/meson.build |   1 +
 2 files changed, 230 insertions(+)
 create mode 100644 contrib/plugins/dlcall.c
#7 lore raw

Re: [RFC PATCH v2 0/1] contrib/plugins: add a minimal dlcall plugin

Hi Pierrick,

Quoted 3 lines · Pierrick Bouvier

Makes sense, thanks for your answer.

Thanks, glad it makes sense.

While I'm here, a question about a convention I noticed: most of the QEMU code that loads external modules goes through GModule (g_module_open() / g_module_symbol()) rather than the libdl functions directly. Is there a specific reason for preferring GModule?

https://github.com/qemu/qemu/blob/3b50303f9563a42538a1fd5c0ea7f952e23016e1/plugins/loader.c#L190

https://github.com/qemu/qemu/blob/3b50303f9563a42538a1fd5c0ea7f952e23016e1/util/module.c#L171

In the plugin I used dlopen()/dlsym() directly, because their interface is more standard and more flexible for this use case. For example, g_module_open() does not expose RTLD_DEFAULT, which I rely on. And since recent glibc (2.34+) folds libdl into libc, no explicit -ldl is needed, so simply adding the file to meson.build builds cleanly.

Does using libdl directly in a plugin violate any convention I should be aware of? If GModule is preferred for portability or some other reason, I'm happy to switch where feasible.

Thanks,
Ziyang Zhang

#8 lore raw

Re: [RFC PATCH v2 0/1] contrib/plugins: add a minimal dlcall plugin

Quoted 7 lines · Ziyang Zhang

Hi Pierrick,

Quoted 3 lines · Pierrick Bouvier

Makes sense, thanks for your answer.

Quoted 7 lines

Thanks, glad it makes sense.

While I'm here, a question about a convention I noticed: most of the QEMU code that loads external modules goes through GModule (g_module_open() / g_module_symbol()) rather than the libdl functions directly. Is there a specific reason for preferring GModule?

Mostly for portabilty reasons I would say, since Windows does not expose this. Not sure if MacOS/BDSs have their own quirks compared to Linux also.

Quoted 12 lines

https://github.com/qemu/qemu/
blob/3b50303f9563a42538a1fd5c0ea7f952e23016e1/plugins/loader.c#L190

https://github.com/qemu/qemu/
blob/3b50303f9563a42538a1fd5c0ea7f952e23016e1/util/module.c#L171

In the plugin I used dlopen()/dlsym() directly, because their interface is more standard and more flexible for this use case. For example, g_module_open() does not expose RTLD_DEFAULT, which I rely on. And since recent glibc (2.34+) folds libdl into libc, no explicit -ldl is needed, so simply adding the file to meson.build builds cleanly.

Windows does not have a semantic like RTLD_DEFAULT, since all symbols are solved from a specific library (import are symbol name + dll name), and not globally, like on Linux.

Quoted 4 lines

Does using libdl directly in a plugin violate any convention I should be aware of? If GModule is preferred for portability or some other reason, I'm happy to switch where feasible.

As long as it compiles on all platforms, I don't mind too much if it's written this way, since ultimately, it's a plugin written only for linux-user. We could deactivate it conditionally in meson.build if no linux-user target is built.

Quoted 3 lines

Thanks,
Ziyang Zhang