Skip to contentSkip to conversation

Introduce QEMU userspace ebpf support

RFC PATCH In review Zhang Chen 5 participants 31 messages Jun 17, 2022 – Jul 1, 2022

Subsystems (2)
lore
Share & tools
lore

Zhang Chen

author
#0 lore raw

cover Introduce QEMU userspace ebpf support

Hi All,

The goal of this series is to bring the power of ebpf to QEMU.
It makes QEMU have the ability to extend the capabilities without requiring changing source code. Just need to load the eBPF binary file even at VM runtime. And already have some userspace ebpf implementation like: Intel DPDK eBPF, windows eBPF, etc.. The original idea suggested by Jason Wang.

eBPF is a revolutionary technology with origins in the Linux kernel
that can run sandboxed programs in an operating system kernel. It is used to safely and efficiently extend the capabilities of the kernel without requiring to change kernel source code or load kernel modules.(from https://ebpf.io/)

KVM already got benefits from it, but QEMU did not. Hence we want
to bring the power of eBPF to QEMU. It can load binary eBPF program even when VM running. At the same time, add some hooks in QEMU as the user space eBPF load point. Do the things on different layers.

That’s the advantages of kernel eBPF. Most of the functions can be
implemented in QEMU. This series just a start of the Power of Programmability.

1). Safety:

Building on the foundation of seeing and understanding all system calls and combining that with a packet and socket-level view of all networking operations allows for revolutionary new approaches to securing systems.

2). Tracing & Profiling:

The ability to attach eBPF programs to trace points as well as kernel and user application probe points allows unprecedented visibility into the runtime behavior of applications and the system itself.

3). Networking:

The combination of programmability and efficiency makes eBPF a natural fit for all packet processing requirements of networking solutions.

4). Observability & Monitoring:

Instead of relying on static counters and gauges exposed by the perating system, eBPF enables the collection & in-kernel aggregation of custom metrics and generation of visibility events based on a wide range of possible sources.

QEMU userspace ebpf design based on ubpf project (https://github.com/iovisor/ubpf).
The most mature userspace ebpf implementation. This project officially support by iovisor(Like BCC and bpftrace). This project includes an eBPF assembler, disassembler, interpreter (for all platforms), and JIT compiler (for x86-64 and Arm64 targets). Qemu userspace ebpf make the ubpf project as the git submodule.

Current implementation support load ebpf program and run it in
net/filter-ubpf module, this filter can support any user defined rules to hanle network packet. At the same time, it's easy for other developers to use the ubpf infrastructue in QEMU's other modules from the function in /ebpf/ubpf.c, and it support JIT.

For the uBPF License is Apache License 2.0, It's OK to compatible
with QEMU’s GPLv2 LICENSE same as mason.

TODO: Need to add more comments and test-case for ubpf, current
implementation not include ebpf verifier. But I think maybe it's not a big problem, current ebpf load/unload API exposed by QMP command. Qemu is a userspace program, if someone want to hack QEMU, no need to load a malicious ubpf program, it can hack QEMU code or crash QEMU on host directly(different from kernel ebpf needs strict inspection, but yes, it still need basic check).

Any comments are welcome.

Thanks
Chen

Zhang Chen (12):
configure: Add iovisor/ubpf project as a submodule for QEMU
meson: Add ubpf build config and misc
ebpf/uBPF: Introduce userspace ebpf data structure ebpf/uBPF: Introduce ubpf initialize functions ebpf/uBPF: Add qemu_prepare_ubpf to load ebpf binary ebpf/uBPF: Add qemu_ubpf_run_once excute real ebpf program net/filter: Introduce filter-ubpf module
qapi: Add FilterUbpfProperties and qemu-options softmmu/vl.c: Add filter-ubpf for netdev as other netfilters net/filter-ubpf.c: run the ubpf program to handle network packet docs/devel: Add userspace-ebpf.rst
test/qtest: Add ubpf basic test case

 .gitmodules                         |   3 +
 configure                           |  20 +++
 docs/devel/userspace-ebpf.rst       | 106 ++++++++++++++
 ebpf/meson.build                    |   1 +
 ebpf/ubpf-stub.c                    |  35 +++++
 ebpf/ubpf.c                         | 217 ++++++++++++++++++++++++++++
 ebpf/ubpf.h                         |  44 ++++++
 meson.build                         |  47 ++++++
 meson_options.txt                   |   3 +
 net/filter-ubpf.c                   | 185 ++++++++++++++++++++++++
 net/meson.build                     |   1 +
 qapi/qom.json                       |  18 +++
 qemu-options.hx                     |   6 +
 scripts/coverity-scan/COMPONENTS.md |   3 +
 scripts/meson-buildoptions.sh       |   5 +
 softmmu/vl.c                        |   3 +-
 tests/qtest/demo_ubpf.o             | Bin 0 -> 544 bytes
 tests/qtest/integer_5.mem           | Bin 0 -> 4 bytes
 tests/qtest/meson.build             |   3 +-
 tests/qtest/ubpf-test.c             |  64 ++++++++
 ubpf                                |   1 +
 21 files changed, 763 insertions(+), 2 deletions(-)
 create mode 100644 docs/devel/userspace-ebpf.rst
 create mode 100644 ebpf/ubpf-stub.c
 create mode 100644 ebpf/ubpf.c
 create mode 100644 ebpf/ubpf.h
 create mode 100644 net/filter-ubpf.c
 create mode 100644 tests/qtest/demo_ubpf.o
 create mode 100644 tests/qtest/integer_5.mem
 create mode 100644 tests/qtest/ubpf-test.c
 create mode 160000 ubpf
Signature · 1 line
2.25.1
author
#1 lore raw

RFC PATCH 1/12 configure: Add iovisor/ubpf project as a submodule for QEMU

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

---
 .gitmodules |  3 +++
 configure   | 20 ++++++++++++++++++++
 ubpf        |  1 +
 3 files changed, 24 insertions(+)
 create mode 160000 ubpf
.gitmodules +3 -0
index b8bff47df8..30fb082f39 100644
@@ -64,3 +64,6 @@
 [submodule "tests/lcitool/libvirt-ci"]
 	path = tests/lcitool/libvirt-ci
 	url = https://gitlab.com/libvirt/libvirt-ci.git
+[submodule "ubpf"]
+	path = ubpf
+	url = https://github.com/iovisor/ubpf.git
configure +20 -0
index e69537c756..7dde1429dc 100755
@@ -326,6 +326,7 @@ else
   slirp="auto"
 fi
 fdt="auto"
+ubpf="auto"
 
 # 2. Automatically enable/disable other options
 tcg="enabled"
@@ -820,6 +821,14 @@ for opt do
   ;;
   --enable-slirp=*) slirp="$optarg"
   ;;
+  --disable-ubpf) ubpf="disabled"
+  ;;
+  --enable-ubpf) ubpf="enabled"
+  ;;
+  --enable-ubpf=git) ubpf="internal"
+  ;;
+  --enable-ubpf=*) ubpf="$optarg"
+  ;;
   --disable-tcg) tcg="disabled"
                  plugins="no"
   ;;
@@ -2176,6 +2185,16 @@ if test "$have_ubsan" = "yes"; then
   QEMU_LDFLAGS="-fsanitize=undefined $QEMU_LDFLAGS"
 fi
 
+##########################################
+# check for ubpf
+
+case "$ubpf" in
+  auto | enabled | internal)
+    # Simpler to always update submodule, even if not needed.
+    git_submodules="${git_submodules} ubpf"
+    ;;
+esac
+
 ##########################################
 
 # Exclude --warn-common with TSan to suppress warnings from the TSan libraries.
@@ -2664,6 +2683,7 @@ if test "$skip_meson" = no; then
   # QEMU options
   test "$cfi" != false && meson_option_add "-Dcfi=$cfi"
   test "$fdt" != auto && meson_option_add "-Dfdt=$fdt"
+  test "$ubpf" != auto && meson_option_add "-Dubpf=$ubpf"
   test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE"
   test "$qemu_suffix" != qemu && meson_option_add "-Dqemu_suffix=$qemu_suffix"
   test "$slirp" != auto && meson_option_add "-Dslirp=$slirp"
ubpf new file +1 -0
new file mode 160000
index 0000000000..0dd334daf4
@@ -0,0 +1 @@
+Subproject commit 0dd334daf4849137fa40d2b7676d2bf920d5c81d
Signature · 1 line
2.25.1
#2 lore raw

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 2 lines · Zhang Chen

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to take advantage of it.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#3 lore raw

RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 17 lines

-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Friday, June 17, 2022 4:05 PM
To: Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Thomas Huth <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 8 lines
Quoted 2 lines · Zhang Chen

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to take advantage of it.

Yes, agree. It's the best way to use the uBPF project. But current status is distros(ubuntu, RHEL...) does not ship the iovisor/ubpf like the iovisor/bcc. So I have to do it. Or do you have any better suggestions?

Thanks
Chen

Quoted 9 lines

With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|

|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-
https://www.instagram.com/dberrange :|
#4 lore raw

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 26 lines · Chen
Quoted 17 lines

-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Friday, June 17, 2022 4:05 PM
To: Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Thomas Huth <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 8 lines
Quoted 2 lines · Zhang Chen

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to take advantage of it.

Quoted 4 lines

Yes, agree. It's the best way to use the uBPF project. But current status is distros(ubuntu, RHEL...) does not ship the iovisor/ubpf like the iovisor/bcc. So I have to do it. Or do you have any better suggestions?

If distros want to support the functionality, they can add packages for it IMHO.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#5 lore raw

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 27 lines · Daniel P. Berrangé
Quoted 26 lines · Chen
Quoted 25 lines

-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Friday, June 17, 2022 4:05 PM
To: Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Thomas Huth <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 2 lines · Zhang Chen

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to take advantage of it.

Quoted 7 lines
Quoted 4 lines

Yes, agree. It's the best way to use the uBPF project. But current status is distros(ubuntu, RHEL...) does not ship the iovisor/ubpf like the iovisor/bcc. So I have to do it. Or do you have any better suggestions?

If distros want to support the functionality, they can add packages for it IMHO.

Yes, let's please avoid new submodules. Submodules can sometimes be a real PITA (e.g. if you forget to update before rsync'ing your code to a machine that has limited internet access), and if users install QEMU from sources, they can also install ubpf from sources, too. And if distros want to support this feature, they can package ubpf on their own, as Daniel said.

Thomas

#6 lore raw

RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 54 lines

-----Original Message-----
From: Thomas Huth <[email protected]>
Sent: Monday, June 20, 2022 4:47 PM
To: Daniel P. Berrangé <[email protected]>; Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 12 lines · Daniel P. Berrangé
Quoted 11 lines · Chen
Quoted 11 lines

-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Friday, June 17, 2022 4:05 PM
To: Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Thomas Huth <[email protected]>;

Laurent

Quoted 15 lines
Quoted 15 lines
Quoted 14 lines

Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 2 lines · Zhang Chen

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to take advantage of it.

Quoted 7 lines
Quoted 4 lines

Yes, agree. It's the best way to use the uBPF project. But current status is distros(ubuntu, RHEL...) does not ship the iovisor/ubpf like the iovisor/bcc. So I have to do it. Or do you have any better suggestions?

If distros want to support the functionality, they can add packages for it IMHO.

Quoted 6 lines

Yes, let's please avoid new submodules. Submodules can sometimes be a real PITA (e.g. if you forget to update before rsync'ing your code to a machine that has limited internet access), and if users install QEMU from sources, they can also install ubpf from sources, too.
And if distros want to support this feature, they can package ubpf on their own, as Daniel said.

Hi Daniel and Thomas,

I don't know much the background history of QEMU submodules, but meson build
is a submodule for QEMU too. It means user can't install QEMU from sources
with limited internet access.
And back to Daniel's comments, Yes, the best way is distros add the ubpf packages,
But maybe it's too late to implement new features for us. We can introduce the submodule now and auto change to the distros's lib when distros add it. For example QEMU's submodule SLIRP do it in the same way. It's already added by most distros and still as a QEMU submodule. It make user experience the latest technology with no other dependencies. uBPF infrastructure have the ability to extend the capabilities without requiring changing source code. If we not allow it, we have to re-implement all the eBPF assembler, disassembler, interpreter, and JIT compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by license issue).

Thanks
Chen

Quoted 2 lines

Thomas

#7 lore raw

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 61 lines · Chen
Quoted 54 lines

-----Original Message-----
From: Thomas Huth <[email protected]>
Sent: Monday, June 20, 2022 4:47 PM
To: Daniel P. Berrangé <[email protected]>; Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 12 lines · Daniel P. Berrangé
Quoted 11 lines · Chen
Quoted 11 lines

-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Friday, June 17, 2022 4:05 PM
To: Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Thomas Huth <[email protected]>;

Laurent

Quoted 15 lines
Quoted 15 lines
Quoted 14 lines

Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 2 lines · Zhang Chen

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to take advantage of it.

Quoted 7 lines
Quoted 4 lines

Yes, agree. It's the best way to use the uBPF project. But current status is distros(ubuntu, RHEL...) does not ship the iovisor/ubpf like the iovisor/bcc. So I have to do it. Or do you have any better suggestions?

If distros want to support the functionality, they can add packages for it IMHO.

Quoted 6 lines

Yes, let's please avoid new submodules. Submodules can sometimes be a real PITA (e.g. if you forget to update before rsync'ing your code to a machine that has limited internet access), and if users install QEMU from sources, they can also install ubpf from sources, too.
And if distros want to support this feature, they can package ubpf on their own, as Daniel said.

Quoted 5 lines

Hi Daniel and Thomas,

I don't know much the background history of QEMU submodules, but meson build is a submodule for QEMU too. It means user can't install QEMU from sources with limited internet access.

There is no written policy, but I think the general consensus is that we only ship code in submodules if:

1) It's not available in a required version in distros yet

and

2) it is essentially required to build QEMU (like meson) or if the feature has been part of the QEMU sources before and then moved to a separate repository (like slirp).

We ship meson as a submodule since we require some meson features that are not available with the meson versions in the distros yet. Once the distros catch up, we'll likely remove the meson submodule from QEMU.

Quoted 3 lines

And back to Daniel's comments, Yes, the best way is distros add the ubpf packages,
But maybe it's too late to implement new features for us. We can introduce the submodule now and auto change to the distros's lib when distros add it. For example QEMU's submodule SLIRP do it in the same way.

slirp used to be part of the QEMU repository, but then has been moved to a separate project a while ago. However, at that point in time there weren't any packages ins distros yet, so we had to include it as a submodule.

Now that the distros ship it, too, we're planning to remove the slirp submodule from QEMU soon, see:

https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html

Quoted 2 lines

It make user experience the latest technology
with no other dependencies.

Well, that's only true if we update the submodule in QEMU regularly. If we forget to update, we could easily miss some important (maybe even security related) fixes from the upstream projects. This can be a nightmare for distros, when they then have to go around and look into each and every projects whether they embed a certain code module that needs a CVE fix. It's better if it can be fixed in one central spot instead.

Quoted 3 lines

uBPF infrastructure have the ability to extend the capabilities without requiring
changing source code. If we not allow it, we have to re-implement all the eBPF assembler, disassembler, interpreter, and JIT compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by license issue).

Not sure whether I understood that statement right ... nobody said that QEMU should not allow it - we just suggested to rely on a system installation of ubpf instead of embedding the code. Or is that not possible?? (I don't know that project yet - isn't it possible to compile it as a shared library?)

Thomas

#8 lore raw

RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 131 lines

-----Original Message-----
From: Thomas Huth <[email protected]>
Sent: Monday, June 20, 2022 5:44 PM
To: Zhang, Chen <[email protected]>; Daniel P. Berrangé <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 10 lines · Chen
Quoted 10 lines

-----Original Message-----
From: Thomas Huth <[email protected]>
Sent: Monday, June 20, 2022 4:47 PM
To: Daniel P. Berrangé <[email protected]>; Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>;

Markus

Quoted 27 lines
Quoted 27 lines

Armbruster <[email protected]>; Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 12 lines · Daniel P. Berrangé
Quoted 11 lines · Chen
Quoted 11 lines

-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Friday, June 17, 2022 4:05 PM
To: Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Thomas Huth <[email protected]>;

Laurent

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines · Zhang Chen
Quoted 2 lines

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to take advantage

of it.

Quoted 17 lines
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Quoted 1 line

Yes, agree. It's the best way to use the uBPF project. But current status is distros(ubuntu, RHEL...) does not ship the iovisor/ubpf like the iovisor/bcc. So I have to do it. Or do you have any better suggestions?

If distros want to support the functionality, they can add packages for it IMHO.

Quoted 6 lines

Yes, let's please avoid new submodules. Submodules can sometimes be a real PITA (e.g. if you forget to update before rsync'ing your code to a machine that has limited internet access), and if users install QEMU from sources, they can also install ubpf from sources, too. And if distros want to support this feature, they can package ubpf on their own, as Daniel said.

Quoted 3 lines

Hi Daniel and Thomas,

I don't know much the background history of QEMU submodules, but

meson

Quoted 2 lines

build is a submodule for QEMU too. It means user can't install QEMU from sources with limited internet access.

There is no written policy, but I think the general consensus is that we only ship code in submodules if:

1) It's not available in a required version in distros yet

and

2) it is essentially required to build QEMU (like meson) or if the feature has been part of the QEMU sources before and then moved to a separate repository (like slirp).

We ship meson as a submodule since we require some meson features that are not available with the meson versions in the distros yet. Once the distros catch up, we'll likely remove the meson submodule from QEMU.

Quoted 3 lines

And back to Daniel's comments, Yes, the best way is distros add the ubpf packages, But maybe it's too late to implement new features for us. We can introduce the submodule now and auto change to the distros's

lib when distros add it. For example QEMU's submodule SLIRP do it in the same way.

slirp used to be part of the QEMU repository, but then has been moved to a separate project a while ago. However, at that point in time there weren't any packages ins distros yet, so we had to include it as a submodule.

Now that the distros ship it, too, we're planning to remove the slirp submodule from QEMU soon, see:

https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html

Quoted 2 lines

It make user experience the latest technology with no other dependencies.

Well, that's only true if we update the submodule in QEMU regularly. If we forget to update, we could easily miss some important (maybe even security related) fixes from the upstream projects. This can be a nightmare for distros, when they then have to go around and look into each and every projects whether they embed a certain code module that needs a CVE fix. It's better if it can be fixed in one central spot instead.

Quoted 3 lines

uBPF infrastructure have the ability to extend the capabilities without requiring changing source code. If we not allow it, we have to re-implement all the eBPF assembler, disassembler, interpreter, and JIT

Quoted 7 lines

compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by license issue).

Not sure whether I understood that statement right ... nobody said that QEMU should not allow it - we just suggested to rely on a system installation of ubpf instead of embedding the code. Or is that not possible?? (I don't know that project yet - isn't it possible to compile it as a shared library?)

Thanks for your details explanation.
It looks better to introduce the uBPF shared library for QEMU. For example:
./configure --ubpf-lib=path

If yes, I think it's fine for me.

Thanks
Chen

Quoted 2 lines

Thomas

#9 lore raw

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 139 lines · Chen
Quoted 131 lines

-----Original Message-----
From: Thomas Huth <[email protected]>
Sent: Monday, June 20, 2022 5:44 PM
To: Zhang, Chen <[email protected]>; Daniel P. Berrangé <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 10 lines · Chen
Quoted 10 lines

-----Original Message-----
From: Thomas Huth <[email protected]>
Sent: Monday, June 20, 2022 4:47 PM
To: Daniel P. Berrangé <[email protected]>; Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>;

Markus

Quoted 27 lines
Quoted 27 lines

Armbruster <[email protected]>; Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 12 lines · Daniel P. Berrangé
Quoted 11 lines · Chen
Quoted 11 lines

-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Friday, June 17, 2022 4:05 PM
To: Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Thomas Huth <[email protected]>;

Laurent

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines · Zhang Chen
Quoted 2 lines

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to take advantage

of it.

Quoted 17 lines
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Quoted 1 line

Yes, agree. It's the best way to use the uBPF project. But current status is distros(ubuntu, RHEL...) does not ship the iovisor/ubpf like the iovisor/bcc. So I have to do it. Or do you have any better suggestions?

If distros want to support the functionality, they can add packages for it IMHO.

Quoted 6 lines

Yes, let's please avoid new submodules. Submodules can sometimes be a real PITA (e.g. if you forget to update before rsync'ing your code to a machine that has limited internet access), and if users install QEMU from sources, they can also install ubpf from sources, too. And if distros want to support this feature, they can package ubpf on their own, as Daniel said.

Quoted 3 lines

Hi Daniel and Thomas,

I don't know much the background history of QEMU submodules, but

meson

Quoted 2 lines

build is a submodule for QEMU too. It means user can't install QEMU from sources with limited internet access.

There is no written policy, but I think the general consensus is that we only ship code in submodules if:

1) It's not available in a required version in distros yet

and

2) it is essentially required to build QEMU (like meson) or if the feature has been part of the QEMU sources before and then moved to a separate repository (like slirp).

We ship meson as a submodule since we require some meson features that are not available with the meson versions in the distros yet. Once the distros catch up, we'll likely remove the meson submodule from QEMU.

Quoted 3 lines

And back to Daniel's comments, Yes, the best way is distros add the ubpf packages, But maybe it's too late to implement new features for us. We can introduce the submodule now and auto change to the distros's

lib when distros add it. For example QEMU's submodule SLIRP do it in the same way.

slirp used to be part of the QEMU repository, but then has been moved to a separate project a while ago. However, at that point in time there weren't any packages ins distros yet, so we had to include it as a submodule.

Now that the distros ship it, too, we're planning to remove the slirp submodule from QEMU soon, see:

https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html

Quoted 2 lines

It make user experience the latest technology with no other dependencies.

Well, that's only true if we update the submodule in QEMU regularly. If we forget to update, we could easily miss some important (maybe even security related) fixes from the upstream projects. This can be a nightmare for distros, when they then have to go around and look into each and every projects whether they embed a certain code module that needs a CVE fix. It's better if it can be fixed in one central spot instead.

Quoted 3 lines

uBPF infrastructure have the ability to extend the capabilities without requiring changing source code. If we not allow it, we have to re-implement all the eBPF assembler, disassembler, interpreter, and JIT

Quoted 7 lines

compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by license issue).

Not sure whether I understood that statement right ... nobody said that QEMU should not allow it - we just suggested to rely on a system installation of ubpf instead of embedding the code. Or is that not possible?? (I don't know that project yet - isn't it possible to compile it as a shared library?)

Quoted 4 lines

Thanks for your details explanation.
It looks better to introduce the uBPF shared library for QEMU. For example:
./configure --ubpf-lib=path

I've not looked, so maybe it already does this, but ideally 'uBPF' would ship a 'pkg-config' file, so that apps can automatically find it and set the right cflags/libs etc for the compiler. For configure integration, normally we'd expect it to be --enable-ubpf/--disable-ubpf, with it automatically enabling itself if the pkg-config file is found. Take a look at handling of some existing libraries we depend on for examples.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#10 lore raw

RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 173 lines

-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Monday, June 20, 2022 6:37 PM
To: Zhang, Chen <[email protected]>
Cc: Thomas Huth <[email protected]>; Jason Wang <[email protected]>; qemu-dev <[email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 40 lines · Chen
Quoted 40 lines

-----Original Message-----
From: Thomas Huth <[email protected]>
Sent: Monday, June 20, 2022 5:44 PM
To: Zhang, Chen <[email protected]>; Daniel P. Berrangé <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 10 lines · Chen
Quoted 10 lines

-----Original Message-----
From: Thomas Huth <[email protected]>
Sent: Monday, June 20, 2022 4:47 PM
To: Daniel P. Berrangé <[email protected]>; Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>;

Markus

Quoted 11 lines
Quoted 11 lines

Armbruster <[email protected]>; Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 3 lines · Daniel P. Berrangé
Quoted 2 lines · Chen
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Friday, June 17, 2022 4:05 PM
To: Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster

<[email protected]>;

Quoted 25 lines
Quoted 25 lines
Quoted 9 lines
Quoted 9 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Peter Maydell <[email protected]>; Thomas Huth <[email protected]>;

Laurent

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines · Zhang Chen
Quoted 2 lines

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to take advantage

of it.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 1 line

Yes, agree. It's the best way to use the uBPF project. But current status is distros(ubuntu, RHEL...) does not ship the iovisor/ubpf like the iovisor/bcc. So I have to do it. Or do you have any better suggestions?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

If distros want to support the functionality, they can add packages for it IMHO.

Yes, let's please avoid new submodules. Submodules can sometimes be a real PITA (e.g. if you forget to update before rsync'ing your code to a machine that has limited internet access), and if users install QEMU from sources, they can also install ubpf from

sources, too.

Quoted 17 lines
Quoted 17 lines
Quoted 6 lines
Quoted 2 lines

And if distros want to support this feature, they can package ubpf on their own, as Daniel said.

Hi Daniel and Thomas,

I don't know much the background history of QEMU submodules, but

meson

Quoted 2 lines

build is a submodule for QEMU too. It means user can't install QEMU from sources with limited internet access.

There is no written policy, but I think the general consensus is that we only ship code in submodules if:

1) It's not available in a required version in distros yet

and

Quoted 7 lines
Quoted 7 lines

2) it is essentially required to build QEMU (like meson) or if the feature has been part of the QEMU sources before and then moved to a separate repository (like slirp).

We ship meson as a submodule since we require some meson features that are not available with the meson versions in the distros yet. Once the distros catch up, we'll likely remove the meson submodule from

QEMU.

Quoted 11 lines
Quoted 11 lines
Quoted 5 lines

And back to Daniel's comments, Yes, the best way is distros add the ubpf packages, But maybe it's too late to implement new features for us. We can introduce the submodule now and auto change to the distros's

lib when distros add it. For example QEMU's submodule SLIRP do it in the same way.

slirp used to be part of the QEMU repository, but then has been moved to a separate project a while ago. However, at that point in time there weren't any packages ins distros yet, so we had to include it as

a submodule.

Quoted 31 lines
Quoted 22 lines

Now that the distros ship it, too, we're planning to remove the slirp submodule from QEMU soon, see:

https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00974.html

Quoted 2 lines

It make user experience the latest technology with no other dependencies.

Well, that's only true if we update the submodule in QEMU regularly. If we forget to update, we could easily miss some important (maybe even security
related) fixes from the upstream projects. This can be a nightmare for distros, when they then have to go around and look into each and every projects whether they embed a certain code module that needs a CVE fix. It's better if it can be fixed in one central spot instead.

Quoted 4 lines

uBPF infrastructure have the ability to extend the capabilities without requiring changing source code. If we not allow it, we have to re-implement all the eBPF assembler, disassembler, interpreter, and JIT

Quoted 8 lines

compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by license issue).

Not sure whether I understood that statement right ... nobody said that QEMU should not allow it - we just suggested to rely on a system installation of ubpf instead of embedding the code. Or is that not possible?? (I don't know that project yet - isn't it possible to compile it as a shared library?)

Quoted 4 lines

Thanks for your details explanation.
It looks better to introduce the uBPF shared library for QEMU. For example:
./configure --ubpf-lib=path

Quoted 7 lines

I've not looked, so maybe it already does this, but ideally 'uBPF' would ship a 'pkg-config' file, so that apps can automatically find it and set the right cflags/libs etc for the compiler. For configure integration, normally we'd expect it to be --enable-ubpf/--disable-ubpf, with it automatically enabling itself if the pkg-config file is found.
Take a look at handling of some existing libraries we depend on for examples.

OK, thanks your comments. I reviewed uBPF code, but no pkg-config file.
The make install just copy the libubpf.so/.a to /usr/lib.
Fortunately, writing this file looks easy. I will try to do it and make QEMU use the libubpf.so like "pixman".

Thanks
Chen

Quoted 8 lines

With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|

|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-
https://www.instagram.com/dberrange :|
#11 lore raw

Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

In reply to Zhang, Chen
Quoted 32 lines · Chen
Quoted 23 lines · Daniel P. Berrangé
Quoted 12 lines · Chen
Quoted 11 lines
Quoted 10 lines · Zhang Chen
Quoted 2 lines

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to take advantage of it.

Quoted 7 lines
Quoted 4 lines

Yes, agree. It's the best way to use the uBPF project. But current status is distros(ubuntu, RHEL...) does not ship the iovisor/ubpf like the iovisor/bcc. So I have to do it. Or do you have any better suggestions?

If distros want to support the functionality, they can add packages for it IMHO.

Quoted 6 lines

Yes, let's please avoid new submodules. Submodules can sometimes be a real PITA (e.g. if you forget to update before rsync'ing your code to a machine that has limited internet access), and if users install QEMU from sources, they can also install ubpf from sources, too.
And if distros want to support this feature, they can package ubpf on their own, as Daniel said.

Hi Daniel and Thomas,

Quoted 10 lines

I don't know much the background history of QEMU submodules, but meson build
is a submodule for QEMU too. It means user can't install QEMU from sources
with limited internet access.
And back to Daniel's comments, Yes, the best way is distros add the ubpf packages,
But maybe it's too late to implement new features for us. We can introduce the submodule now and auto change to the distros's lib when distros add it. For example QEMU's submodule SLIRP do it in the same way. It's already added by most distros and still as a QEMU submodule. It make user experience the latest technology with no other dependencies. uBPF infrastructure have the ability to extend the capabilities without requiring changing source code. If we not allow it, we have to re-implement all the eBPF assembler, disassembler, interpreter, and JIT compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by license issue).

Slirp is a different scenario. That was functionality that was historically integrated into QEMU and was then spun out into a standalone project. Since we had existing users on existing distro releases dependant on Slirp, we wanted to give a smooth upgrade experiance by bundling Slirp. Essentially the goal was to avoid the regression if someone deployed new QEMU on existing distros.

Meson is a fairly similar scenario. We wanted to swap the build system in QEMU over to Meson, and that change would affect all existing users of QEMU. Many distros didn't have a new enough meson, and so bundling it in QEMU enables us to give a smooth upgrade path without any regression for existing users on existing distros.

This patch, however, is proposing an entirely new piece of functionality that has no existing users and even once present will be used by relatively few users compartively speaking. As such there is no upgrade compatibility / regression scenario that we need to worry about. Anyone interested in the new functionality can be reasonably asked to either wait for the distro to package it, or build it themselves.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#12 lore raw

RE: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 82 lines

-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Monday, June 20, 2022 6:01 PM
To: Zhang, Chen <[email protected]>
Cc: Thomas Huth <[email protected]>; Jason Wang <[email protected]>; qemu-dev <[email protected]>; Paolo Bonzini <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 01/12] configure: Add iovisor/ubpf project as a submodule for QEMU

Quoted 6 lines · Chen
Quoted 6 lines · Daniel P. Berrangé
Quoted 3 lines · Chen
Quoted 2 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines · Zhang Chen
Quoted 2 lines

Make iovisor/ubpf project be a git submodule for QEMU. It will auto clone ubpf project when configure QEMU.

I don't think we need todo this. As it is brand new functionality we don't have any back compat issues. We should just expect the distros to ship ubpf if they want their QEMU builds to

take advantage of it.

Quoted 17 lines
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Quoted 1 line

Yes, agree. It's the best way to use the uBPF project. But current status is distros(ubuntu, RHEL...) does not ship the iovisor/ubpf like the iovisor/bcc. So I have to do it. Or do you have any better suggestions?

If distros want to support the functionality, they can add packages for it IMHO.

Quoted 6 lines

Yes, let's please avoid new submodules. Submodules can sometimes be a real PITA (e.g. if you forget to update before rsync'ing your code to a machine that has limited internet access), and if users install QEMU from sources, they can also install ubpf from sources, too. And if distros want to support this feature, they can package ubpf on their own, as Daniel said.

Quoted 3 lines

Hi Daniel and Thomas,

I don't know much the background history of QEMU submodules, but

meson

Quoted 5 lines

build is a submodule for QEMU too. It means user can't install QEMU from sources with limited internet access.
And back to Daniel's comments, Yes, the best way is distros add the ubpf packages, But maybe it's too late to implement new features for us. We can introduce the submodule now and auto change to the distros's

lib when distros add it. For example QEMU's submodule SLIRP do it in the same way.

Quoted 4 lines

It's already added by most distros and still as a QEMU submodule. It make user experience the latest technology with no other dependencies. uBPF infrastructure have the ability to extend the capabilities without requiring changing source code. If we not allow it, we have to re-

implement all the eBPF assembler, disassembler, interpreter, and JIT compiler like DPDK userspace eBPF support (DPDK can't use ubpf project by license issue).

Slirp is a different scenario. That was functionality that was historically integrated into QEMU and was then spun out into a standalone project. Since we had existing users on existing distro releases dependant on Slirp, we wanted to give a smooth upgrade experiance by bundling Slirp. Essentially the goal was to avoid the regression if someone deployed new QEMU on existing distros.

Meson is a fairly similar scenario. We wanted to swap the build system in QEMU over to Meson, and that change would affect all existing users of QEMU.
Many distros didn't have a new enough meson, and so bundling it in QEMU enables us to give a smooth upgrade path without any regression for existing users on existing distros.

Quoted 7 lines

This patch, however, is proposing an entirely new piece of functionality that has no existing users and even once present will be used by relatively few users compartively speaking. As such there is no upgrade compatibility / regression scenario that we need to worry about. Anyone interested in the new functionality can be reasonably asked to either wait for the distro to package it, or build it themselves.

OK, got your point.
For this series, should we introduce an external library "libubpf" in QEMU configure? If this library is found, the relevant files will be compiled and the feature can be enabled in QEMU.

Thanks
Chen

Quoted 8 lines

With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|

|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-
https://www.instagram.com/dberrange :|
author
#13 lore raw

RFC PATCH 2/12 meson: Add ubpf build config and misc

In reply to Zhang Chen

Make meson to build iovisor/ubpf code in Qemu.

---
 meson.build                         | 47 +++++++++++++++++++++++++++++
 meson_options.txt                   |  3 ++
 scripts/coverity-scan/COMPONENTS.md |  3 ++
 scripts/meson-buildoptions.sh       |  5 +++
 4 files changed, 58 insertions(+)
meson.build +47 -0
index 21cd949082..f370c1aba7 100644
@@ -2717,9 +2717,53 @@ if not fdt.found() and fdt_required.length() > 0
   error('fdt not available but required by targets ' + ', '.join(fdt_required))
 endif
 
+ubpf = not_found
+ubpf_opt = 'disabled'
+if have_system
+  ubpf_opt = get_option('ubpf')
+  if ubpf_opt in ['enabled', 'auto', 'system']
+    have_internal = fs.exists(meson.current_source_dir() / 'ubpf/vm/Makefile')
+    ubpf = dependency('ubpf', kwargs: static_kwargs,
+                      method: 'pkg-config',
+                      required: ubpf_opt == 'system' or
+                      ubpf_opt == 'enabled' and not have_internal)
+    if ubpf.found()
+      ubpf_opt = 'system'
+    elif have_internal
+      ubpf_opt = 'internal'
+    else
+      ubpf_opt = 'disabled'
+    endif
+  endif
+  if ubpf_opt == 'internal'
+    ubpf_data = configuration_data()
+
+    ubpf_files = files(
+      'ubpf/vm/ubpf_jit_x86_64.c',
+      'ubpf/vm/ubpf_vm.c',
+      'ubpf/vm/ubpf_loader.c',
+    )
+
+    ubpf_cargs = [
+      '-Wno-error', '-w',
+      '-include', 'ubpf-defs.h'
+    ]
+
+    configure_file(output: 'ubpf-defs.h', configuration: ubpf_data)
+    ubpf_inc = include_directories('ubpf/vm', 'ubpf/vm/inc')
+    libubpf = static_library('ubpf',
+                             sources: ubpf_files,
+                             c_args: ubpf_cargs,
+                             include_directories: ubpf_inc)
+    ubpf = declare_dependency(link_with: libubpf,
+                              include_directories: ubpf_inc)
+  endif
+endif
+
 config_host_data.set('CONFIG_CAPSTONE', capstone.found())
 config_host_data.set('CONFIG_FDT', fdt.found())
 config_host_data.set('CONFIG_SLIRP', slirp.found())
+config_host_data.set('CONFIG_UBPF', ubpf.found())
 
 #####################
 # Generated sources #
@@ -3046,6 +3090,8 @@ subdir('softmmu')
 common_ss.add(capstone)
 specific_ss.add(files('cpu.c', 'disas.c', 'gdbstub.c'), capstone)
 
+common_ss.add(ubpf)
+
 # Work around a gcc bug/misfeature wherein constant propagation looks
 # through an alias:
 #   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99696
@@ -3911,6 +3957,7 @@ summary_info += {'libudev':           libudev}
 # Dummy dependency, keep .found()
 summary_info += {'FUSE lseek':        fuse_lseek.found()}
 summary_info += {'selinux':           selinux}
+summary_info += {'ubpf support':      ubpf_opt == 'internal' ? ubpf_opt : ubpf}
 summary(summary_info, bool_yn: true, section: 'Dependencies')
 
 if not supported_cpus.contains(cpu)
meson_options.txt +3 -0
index 2de94af037..1eb9164857 100644
@@ -262,6 +262,9 @@ option('slirp', type: 'combo', value: 'auto',
 option('fdt', type: 'combo', value: 'auto',
        choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
        description: 'Whether and how to find the libfdt library')
+option('ubpf', type: 'combo', value: 'auto',
+       choices: ['disabled', 'enabled', 'auto', 'system', 'internal'],
+       description: 'Whether and how to find the ubpf library')
 
 option('selinux', type: 'feature', value: 'auto',
        description: 'SELinux support in qemu-nbd')
scripts/coverity-scan/COMPONENTS.md +3 -0
index 183f26a32c..dd28116674 100644
@@ -72,6 +72,9 @@ char
 capstone
   ~ (/qemu)?(/capstone/.*)
 
+ubpf
+  ~ (/qemu)?(/ubpf/vm/.*)
+
 crypto
   ~ (/qemu)?((/include)?/crypto/.*|/hw/.*/crypto.*)
 
scripts/meson-buildoptions.sh +5 -0
index 00ea4d8cd1..044dde1cff 100644
@@ -37,6 +37,8 @@ meson_options_help() {
   printf "%s\n" '                           getrandom()'
   printf "%s\n" '  --enable-slirp[=CHOICE]  Whether and how to find the slirp library'
   printf "%s\n" '                           (choices: auto/disabled/enabled/internal/system)'
+  printf "%s\n" '  --enable-ubpf[=CHOICE]   Whether and how to find the ubpf library'
+  printf "%s\n" '                           (choices: auto/disabled/enabled/internal/system)'
   printf "%s\n" '  --enable-strip           Strip targets on install'
   printf "%s\n" '  --enable-tcg-interpreter TCG with bytecode interpreter (slow)'
   printf "%s\n" '  --enable-trace-backends=CHOICES'
@@ -379,6 +381,9 @@ _meson_option_parse() {
     --enable-slirp=*) quote_sh "-Dslirp=$2" ;;
     --enable-slirp-smbd) printf "%s" -Dslirp_smbd=enabled ;;
     --disable-slirp-smbd) printf "%s" -Dslirp_smbd=disabled ;;
+    --enable-ubpf) printf "%s" -Dubpf=enabled ;;
+    --disable-ubpf) printf "%s" -Dubpf=disabled ;;
+    --enable-ubpf=*) quote_sh "-Dubpf=$2" ;;
     --enable-smartcard) printf "%s" -Dsmartcard=enabled ;;
     --disable-smartcard) printf "%s" -Dsmartcard=disabled ;;
     --enable-snappy) printf "%s" -Dsnappy=enabled ;;
Signature · 1 line
2.25.1
author
#14 lore raw

RFC PATCH 3/12 ebpf/uBPF: Introduce userspace ebpf data structure

In reply to Zhang Chen

Add ebpf/ubpf.h for the UbpfState.

---
 ebpf/ubpf.h | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
 create mode 100644 ebpf/ubpf.h
ebpf/ubpf.h new file +37 -0
new file mode 100644
index 0000000000..2562fff503
@@ -0,0 +1,37 @@
+/*
+ * QEMU Userspace eBPF Header
+ *
+ * Copyright(C) 2022 Intel Corporation.
+ *
+ * Author:
+ *  Zhang Chen <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_UBPF_H
+#define QEMU_UBPF_H
+
+#include <ubpf.h>
+#include <math.h>
+#include <elf.h>
+
+#define MAX_LEN (1024 * 1024)
+
+typedef struct UbpfState {
+    bool jit;
+    char *code_path;
+    void *code;
+    size_t code_len;
+    char *target_path;
+    void *target;
+    size_t target_len;
+    struct ubpf_vm *vm;
+    ubpf_jit_fn fn;
+    int type;
+    char *func;
+} UbpfState;
+
+#endif /* QEMU_UBPF_H */
Signature · 1 line
2.25.1
author
#15 lore raw

RFC PATCH 4/12 ebpf/uBPF: Introduce ubpf initialize functions

In reply to Zhang Chen

Introduce ubpf.c/ubpf-stub.c with basic read and init_jit functions. Add ubpf related .c files to meson.build.

---
 ebpf/meson.build |   1 +
 ebpf/ubpf-stub.c |  24 +++++++++++
 ebpf/ubpf.c      | 101 +++++++++++++++++++++++++++++++++++++++++++++++
 ebpf/ubpf.h      |   4 ++
 4 files changed, 130 insertions(+)
 create mode 100644 ebpf/ubpf-stub.c
 create mode 100644 ebpf/ubpf.c
ebpf/meson.build +1 -0
index 2dd0fd8948..f4457fbd28 100644
@@ -1 +1,2 @@
 softmmu_ss.add(when: libbpf, if_true: files('ebpf_rss.c'), if_false: files('ebpf_rss-stub.c'))
+softmmu_ss.add(when: ubpf, if_true: files('ubpf.c'), if_false: files('ubpf-stub.c'))
ebpf/ubpf-stub.c new file +24 -0
new file mode 100644
index 0000000000..2e8bf15b91
@@ -0,0 +1,24 @@
+/*
+ * QEMU Userspace eBPF Stub File
+ *
+ * Copyright(C) 2022 Intel Corporation.
+ *
+ * Author:
+ *  Zhang Chen <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+bool qemu_ubpf_read_code(UbpfState *u_ebpf, char *path)
+{
+    return 0;
+}
+
+bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path)
+{
+    return 0;
+}
+
+void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit) {}
ebpf/ubpf.c new file +101 -0

101 lines not on this page · raw message

ebpf/ubpf.h +4 -0
index 2562fff503..808c02565c 100644
@@ -34,4 +34,8 @@ typedef struct UbpfState {
     char *func;
 } UbpfState;
 
+bool qemu_ubpf_read_code(UbpfState *u_ebpf, char *path);
+bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path);
+void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit);
+
 #endif /* QEMU_UBPF_H */
Signature · 1 line
2.25.1
author
#16 lore raw

RFC PATCH 5/12 ebpf/uBPF: Add qemu_prepare_ubpf to load ebpf binary

In reply to Zhang Chen

The qemu_prepare_ubpf() can load user defined userspace ebpf binary file to Qemu userspace ebpf VM but not run it. The ebpf program will triggered in the hook point.

---
 ebpf/ubpf-stub.c |   5 +++
 ebpf/ubpf.c      | 100 +++++++++++++++++++++++++++++++++++++++++++++++
 ebpf/ubpf.h      |   1 +
 3 files changed, 106 insertions(+)
ebpf/ubpf-stub.c +5 -0
index 2e8bf15b91..885bd954b7 100644
@@ -22,3 +22,8 @@ bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path)
 }
 
 void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit) {}
+
+int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path)
+{
+    return 0;
+}
ebpf/ubpf.c +100 -0

103 lines not on this page · raw message

ebpf/ubpf.h +1 -0
index 808c02565c..9a35efbeb6 100644
@@ -37,5 +37,6 @@ typedef struct UbpfState {
 bool qemu_ubpf_read_code(UbpfState *u_ebpf, char *path);
 bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path);
 void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit);
+int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path);
 
 #endif /* QEMU_UBPF_H */
Signature · 1 line
2.25.1
author
#17 lore raw

RFC PATCH 6/12 ebpf/uBPF: Add qemu_ubpf_run_once excute real ebpf program

In reply to Zhang Chen

Before running this function, we need to ensure that the userspace ebpf program has been loaded correctly.

---
 ebpf/ubpf-stub.c |  6 ++++++
 ebpf/ubpf.c      | 16 ++++++++++++++++
 ebpf/ubpf.h      |  2 ++
 3 files changed, 24 insertions(+)
ebpf/ubpf-stub.c +6 -0
index 885bd954b7..70da421629 100644
@@ -27,3 +27,9 @@ int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path)
 {
     return 0;
 }
+
+uint64_t qemu_ubpf_run_once(UbpfState *u_ebpf, void *target,
+                            size_t target_len)
+{
+    return 0;
+}
ebpf/ubpf.c +16 -0
index d65fffeda3..8ac513c7ed 100644
@@ -199,3 +199,19 @@ int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path)
 
     return 0;
 }
+
+uint64_t qemu_ubpf_run_once(UbpfState *u_ebpf, void *target,
+                            size_t target_len)
+{
+    uint64_t result;
+
+    if (u_ebpf->jit) {
+        result = u_ebpf->fn(target, target_len);
+    } else {
+        if (ubpf_exec(u_ebpf->vm, target, target_len, &result) < 0) {
+            result = UINT64_MAX;
+        }
+    }
+
+    return result;
+}
ebpf/ubpf.h +2 -0
index 9a35efbeb6..fc40e84e51 100644
@@ -38,5 +38,7 @@ bool qemu_ubpf_read_code(UbpfState *u_ebpf, char *path);
 bool qemu_ubpf_read_target(UbpfState *u_ebpf, char *path);
 void qemu_ubpf_init_jit(UbpfState *u_ebpf, bool jit);
 int qemu_ubpf_prepare(UbpfState *u_ebpf, char *code_path);
+uint64_t qemu_ubpf_run_once(UbpfState *u_ebpf, void *target,
+                            size_t target_len);
 
 #endif /* QEMU_UBPF_H */
Signature · 1 line
2.25.1
author
#18 lore raw

RFC PATCH 7/12 net/filter: Introduce filter-ubpf module

In reply to Zhang Chen

The filter-ubpf module able to load user defined ebpf program to handle network packet based on filter framework.

---
 net/filter-ubpf.c | 149 ++++++++++++++++++++++++++++++++++++++++++++++
 net/meson.build   |   1 +
 2 files changed, 150 insertions(+)
 create mode 100644 net/filter-ubpf.c
net/filter-ubpf.c new file +149 -0
new file mode 100644
index 0000000000..c63a021759
@@ -0,0 +1,149 @@
+/*
+ * QEMU Userspace eBPF Support
+ *
+ * Copyright(C) 2022 Intel Corporation.
+ *
+ * Author:
+ *  Zhang Chen <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "net/filter.h"
+#include "net/net.h"
+#include "qapi/error.h"
+#include "qom/object.h"
+#include "qemu/main-loop.h"
+#include "qemu/error-report.h"
+#include "trace.h"
+#include "ebpf/ubpf.h"
+
+#define TYPE_FILTER_UBPF "filter-ubpf"
+OBJECT_DECLARE_SIMPLE_TYPE(FiliterUbpfState, FILTER_UBPF)
+
+struct FiliterUbpfState {
+    NetFilterState parent_obj;
+    bool ip_mode;
+    char *handler;
+    UbpfState ubpf;
+};
+
+static ssize_t filter_ubpf_receive_iov(NetFilterState *nf,
+                                       NetClientState *sender,
+                                       unsigned flags,
+                                       const struct iovec *iov,
+                                       int iovcnt,
+                                       NetPacketSent *sent_cb)
+{
+    /* TODO: handle packet by loaded userspace ebpf program */
+
+    return 0;
+}
+
+static void filter_ubpf_cleanup(NetFilterState *nf)
+{
+    /* cleanup */
+}
+
+static void filter_ubpf_setup(NetFilterState *nf, Error **errp)
+{
+    FiliterUbpfState *s = FILTER_UBPF(nf);
+
+    if (s->handler == NULL) {
+        error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND, "filter-ubpf parameter"\
+                  " 'ubpf-handler' cannot be empty");
+        return;
+    }
+
+    qemu_ubpf_init_jit(&s->ubpf, true);
+
+    if (qemu_ubpf_prepare(&s->ubpf, s->handler)) {
+        error_set(errp, ERROR_CLASS_DEVICE_NOT_ACTIVE, "filter-ubpf parameter"\
+                  " 'ubpf-handler' cannot be load");
+        return;
+    }
+}
+
+static char *filter_ubpf_get_handler(Object *obj, Error **errp)
+{
+    FiliterUbpfState *s = FILTER_UBPF(obj);
+
+    return g_strdup(s->handler);
+}
+
+static void filter_ubpf_set_handler(Object *obj,
+                                    const char *value,
+                                    Error **errp)
+{
+    FiliterUbpfState *s = FILTER_UBPF(obj);
+
+    g_free(s->handler);
+    s->handler = g_strdup(value);
+    if (!s->handler) {
+        error_setg(errp, "filter ubpf needs 'ubpf-handler' "
+                   "property set");
+        return;
+    }
+}
+
+static bool filter_ubpf_get_mode(Object *obj, Error **errp)
+{
+    FiliterUbpfState *s = FILTER_UBPF(obj);
+
+    return s->ip_mode;
+}
+
+static void filter_ubpf_set_mode(Object *obj, bool value, Error **errp)
+{
+    FiliterUbpfState *s = FILTER_UBPF(obj);
+
+    s->ip_mode = value;
+}
+
+static void filter_ubpf_class_init(ObjectClass *oc, void *data)
+{
+    NetFilterClass *nfc = NETFILTER_CLASS(oc);
+
+    object_class_property_add_str(oc, "ubpf-handler",
+                                  filter_ubpf_get_handler,
+                                  filter_ubpf_set_handler);
+    object_class_property_add_bool(oc, "ip-mode",
+                                   filter_ubpf_get_mode,
+                                   filter_ubpf_set_mode);
+
+    nfc->setup = filter_ubpf_setup;
+    nfc->cleanup = filter_ubpf_cleanup;
+    nfc->receive_iov = filter_ubpf_receive_iov;
+}
+
+static void filter_ubpf_init(Object *obj)
+{
+    FiliterUbpfState *s = FILTER_UBPF(obj);
+
+    /* Filter-ubpf default is ip_mode */
+    s->ip_mode = true;
+}
+
+static void filter_ubpf_fini(Object *obj)
+{
+    /* do some thing */
+}
+
+static const TypeInfo filter_ubpf_info = {
+    .name = TYPE_FILTER_UBPF,
+    .parent = TYPE_NETFILTER,
+    .class_init = filter_ubpf_class_init,
+    .instance_init = filter_ubpf_init,
+    .instance_finalize = filter_ubpf_fini,
+    .instance_size = sizeof(FiliterUbpfState),
+};
+
+static void register_types(void)
+{
+    type_register_static(&filter_ubpf_info);
+}
+
+type_init(register_types);
net/meson.build +1 -0
index 754e2d1d40..177078fa7a 100644
@@ -14,6 +14,7 @@ softmmu_ss.add(files(
   'queue.c',
   'socket.c',
   'util.c',
+  'filter-ubpf.c',
 ))
 
 softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('filter-replay.c'))
Signature · 1 line
2.25.1
author
#19 lore raw

RFC PATCH 8/12 qapi: Add FilterUbpfProperties and qemu-options

In reply to Zhang Chen

Add filter-ubpf related QOM and qemu-options.

---
 qapi/qom.json   | 18 ++++++++++++++++++
 qemu-options.hx |  6 ++++++
 2 files changed, 24 insertions(+)
qapi/qom.json +18 -0
index 6a653c6636..820a5218e8 100644
@@ -444,6 +444,22 @@
   'base': 'NetfilterProperties',
   'data': { '*vnet_hdr_support': 'bool' } }
 
+##
+# @FilterUbpfProperties:
+#
+# Properties for filter-ubpf objects.
+#
+# @ip-mode: if true, IP packet handle mode is enabled(default: true).
+#
+# @ubpf-handler: The filename where the userspace ebpf packets handler.
+#
+# Since: 7.1
+##
+{ 'struct': 'FilterUbpfProperties',
+  'base': 'NetfilterProperties',
+  'data': { '*ip-mode': 'bool',
+            '*ubpf-handler': 'str' } }
+
 ##
 # @InputBarrierProperties:
 #
@@ -845,6 +861,7 @@
     'filter-redirector',
     'filter-replay',
     'filter-rewriter',
+    'filter-ubpf',
     'input-barrier',
     { 'name': 'input-linux',
       'if': 'CONFIG_LINUX' },
@@ -911,6 +928,7 @@
       'filter-redirector':          'FilterRedirectorProperties',
       'filter-replay':              'NetfilterProperties',
       'filter-rewriter':            'FilterRewriterProperties',
+      'filter-ubpf':                'FilterUbpfProperties',
       'input-barrier':              'InputBarrierProperties',
       'input-linux':                { 'type': 'InputLinuxProperties',
                                       'if': 'CONFIG_LINUX' },
qemu-options.hx +6 -0
index 60cf188da4..3dfb858867 100644
@@ -5080,6 +5080,12 @@ SRST
         stored. The file format is libpcap, so it can be analyzed with
         tools such as tcpdump or Wireshark.
 
+    ``-object filter-ubpf,id=id,netdev=dev,ubpf-handler=filename[,ip-mode][,position=head|tail|id=<id>][,insert=behind|before]``
+        filter-ubpf is the userspace ebpf network traffic handler on netdev dev
+        from the userspace ebpf handler file specified by filename.
+        If disable ip_mode, the loaded ebpf program will handle raw
+        network packet.
+
     ``-object colo-compare,id=id,primary_in=chardevid,secondary_in=chardevid,outdev=chardevid,iothread=id[,vnet_hdr_support][,notify_dev=id][,compare_timeout=@var{ms}][,expired_scan_cycle=@var{ms}][,max_queue_size=@var{size}]``
         Colo-compare gets packet from primary\_in chardevid and
         secondary\_in, then compare whether the payload of primary packet
Signature · 1 line
2.25.1
#20 lore raw

Re: [RFC PATCH 08/12] qapi: Add FilterUbpfProperties and qemu-options

Quoted 13 lines · Zhang Chen

Add filter-ubpf related QOM and qemu-options.

---
 qapi/qom.json   | 18 ++++++++++++++++++
 qemu-options.hx |  6 ++++++
 2 files changed, 24 insertions(+)
qapi/qom.json +0 -0
index 6a653c6636..820a5218e8 100644

qapi/qom.json line 452

Quoted 9 lines
qapi/qom.json +6 -0
@@ -444,6 +444,22 @@
   'base': 'NetfilterProperties',
   'data': { '*vnet_hdr_support': 'bool' } }
 
+##
+# @FilterUbpfProperties:
+#
+# Properties for filter-ubpf objects.
+#
+# @ip-mode: if true, IP packet handle mode is enabled(default: true).

Space between "enabled" and "(default: true)", please.

I'm not sure about the name @ip-mode. A mode tends to be an enum. A boolean tends to be a flag, like @disable-packed-handle-mode. Note that I reverted the sense there, to make the default false.

Quoted 34 lines
qapi/qom.json +12 -0
+#
+# @ubpf-handler: The filename where the userspace ebpf packets handler.
+#
+# Since: 7.1
+##
+{ 'struct': 'FilterUbpfProperties',
+  'base': 'NetfilterProperties',
+  'data': { '*ip-mode': 'bool',
+            '*ubpf-handler': 'str' } }
+
 ##
 # @InputBarrierProperties:
 #
@@ -845,6 +861,7 @@
     'filter-redirector',
     'filter-replay',
     'filter-rewriter',
+    'filter-ubpf',
     'input-barrier',
     { 'name': 'input-linux',
       'if': 'CONFIG_LINUX' },
@@ -911,6 +928,7 @@
       'filter-redirector':          'FilterRedirectorProperties',
       'filter-replay':              'NetfilterProperties',
       'filter-rewriter':            'FilterRewriterProperties',
+      'filter-ubpf':                'FilterUbpfProperties',
       'input-barrier':              'InputBarrierProperties',
       'input-linux':                { 'type': 'InputLinuxProperties',
                                       'if': 'CONFIG_LINUX' },
qemu-options.hx +0 -0
index 60cf188da4..3dfb858867 100644

qemu-options.hx SRST · line 5085

Quoted 6 lines
qemu-options.hx +3 -0
@@ -5080,6 +5080,12 @@ SRST
         stored. The file format is libpcap, so it can be analyzed with
         tools such as tcpdump or Wireshark.
 
+    ``-object filter-ubpf,id=id,netdev=dev,ubpf-handler=filename[,ip-mode][,position=head|tail|id=<id>][,insert=behind|before]``
+        filter-ubpf is the userspace ebpf network traffic handler on netdev dev
+        from the userspace ebpf handler file specified by filename.

I believe the conventional capitalization is eBPF.

qemu-options.hx SRST · line 5086

Quoted 1 line
qemu-options.hx +1 -0
+        If disable ip_mode, the loaded ebpf program will handle raw

Markup: ``ip_mode``.

qemu-options.hx SRST · line 5087

Quoted 1 line
qemu-options.hx +1 -0
+        network packet.

Suggest something like "If ``ip_mode`` is off, the eBPF program is fed raw network packets" (hope I'm not misinterpreting things).

Quoted 4 lines
qemu-options.hx +1 -0
+
     ``-object colo-compare,id=id,primary_in=chardevid,secondary_in=chardevid,outdev=chardevid,iothread=id[,vnet_hdr_support][,notify_dev=id][,compare_timeout=@var{ms}][,expired_scan_cycle=@var{ms}][,max_queue_size=@var{size}]``
         Colo-compare gets packet from primary\_in chardevid and
         secondary\_in, then compare whether the payload of primary packet
#21 lore raw

RE: [RFC PATCH 08/12] qapi: Add FilterUbpfProperties and qemu-options

Quoted 44 lines

-----Original Message-----
From: Markus Armbruster <[email protected]>
Sent: Monday, June 20, 2022 3:45 PM
To: Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Daniel P.Berrangé <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Peter Maydell <[email protected]>; Thomas Huth <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 08/12] qapi: Add FilterUbpfProperties and qemu- options

Quoted 13 lines · Zhang Chen

Add filter-ubpf related QOM and qemu-options.

---
 qapi/qom.json   | 18 ++++++++++++++++++
 qemu-options.hx |  6 ++++++
 2 files changed, 24 insertions(+)
qapi/qom.json index renamed from qapi/qom.json +0 -0

6a653c6636..820a5218e8 100644

qapi/qom.json line 452

Quoted 9 lines
qapi/qom.json +6 -0
@@ -444,6 +444,22 @@
   'base': 'NetfilterProperties',
   'data': { '*vnet_hdr_support': 'bool' } }
 
+##
+# @FilterUbpfProperties:
+#
+# Properties for filter-ubpf objects.
+#
+# @ip-mode: if true, IP packet handle mode is enabled(default: true).

Space between "enabled" and "(default: true)", please.

Quoted 3 lines

I'm not sure about the name @ip-mode. A mode tends to be an enum. A boolean tends to be a flag, like @disable-packed-handle-mode. Note that I reverted the sense there, to make the default false.

Thanks for your review Markus~

Makes sense. Current mode just include ip-mode and raw-mode, it looks Need change to a enum mode for it, maybe will add other mode in the future.

Quoted 39 lines

qapi/qom.json line 454

Quoted 3 lines
qapi/qom.json +2 -0
+#
+# @ubpf-handler: The filename where the userspace ebpf packets

handler.

Quoted 35 lines
qapi/qom.json +10 -1
+#
+# Since: 7.1
+##
+{ 'struct': 'FilterUbpfProperties',
+  'base': 'NetfilterProperties',
+  'data': { '*ip-mode': 'bool',
+            '*ubpf-handler': 'str' } }
+
 ##
 # @InputBarrierProperties:
 #
@@ -845,6 +861,7 @@
     'filter-redirector',
     'filter-replay',
     'filter-rewriter',
+    'filter-ubpf',
     'input-barrier',
     { 'name': 'input-linux',
       'if': 'CONFIG_LINUX' },
@@ -911,6 +928,7 @@
       'filter-redirector':          'FilterRedirectorProperties',
       'filter-replay':              'NetfilterProperties',
       'filter-rewriter':            'FilterRewriterProperties',
+      'filter-ubpf':                'FilterUbpfProperties',
       'input-barrier':              'InputBarrierProperties',
       'input-linux':                { 'type': 'InputLinuxProperties',
                                       'if': 'CONFIG_LINUX' }, diff
--git a/qemu-options.hx b/qemu-options.hx index 60cf188da4..3dfb858867

100644

qemu-options.hx +0 -0
@@ -5080,6 +5080,12 @@ SRST
         stored. The file format is libpcap, so it can be analyzed with
         tools such as tcpdump or Wireshark.
 
Quoted 7 lines

qemu-options.hx SRST · line 5083

Quoted 1 line
qemu-options.hx +1 -0
+    ``-object filter-ubpf,id=id,netdev=dev,ubpf-handler=filename[,ip-

mode][,position=head|tail|id=<id>][,insert=behind|before]``

qemu-options.hx SRST · line 5084

Quoted 1 line
qemu-options.hx +1 -0
+        filter-ubpf is the userspace ebpf network traffic handler on netdev

dev

qemu-options.hx SRST · line 5085

Quoted 1 line
qemu-options.hx +1 -0
+        from the userspace ebpf handler file specified by filename.

I believe the conventional capitalization is eBPF.

Yes, will fix it.

Quoted 4 lines

qemu-options.hx SRST · line 5086

Quoted 2 lines
qemu-options.hx +1 -0
+        If disable ip_mode, the loaded ebpf program will handle raw

Markup: ``ip_mode``.

OK.

Quoted 5 lines

qemu-options.hx SRST · line 5087

Quoted 2 lines
qemu-options.hx +1 -0
+        network packet.

Suggest something like "If ``ip_mode`` is off, the eBPF program is fed raw network packets" (hope I'm not misinterpreting things).

OK. I will address your comments in next version.

Thanks
Chen

Quoted 9 lines

qemu-options.hx SRST · line 5089

Quoted 3 lines
qemu-options.hx +1 -0
+
     ``-object colo-

compare,id=id,primary_in=chardevid,secondary_in=chardevid,outdev=chard evid,iothread=id[,vnet_hdr_support][,notify_dev=id][,compare_timeout=@ var{ms}][,expired_scan_cycle=@var{ms}][,max_queue_size=@var{size}]``

Quoted 3 lines
qemu-options.hx +0 -0
         Colo-compare gets packet from primary\_in chardevid and
         secondary\_in, then compare whether the payload of primary

packet

author
#22 lore raw

RFC PATCH 9/12 softmmu/vl.c: Add filter-ubpf for netdev as other netfilters

In reply to Zhang Chen
---
 softmmu/vl.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
softmmu/vl.c +2 -1
index 4c1e94b00e..d924fb1c71 100644
@@ -1822,7 +1822,8 @@ static bool object_create_early(const char *type)
         g_str_equal(type, "filter-redirector") ||
         g_str_equal(type, "colo-compare") ||
         g_str_equal(type, "filter-rewriter") ||
-        g_str_equal(type, "filter-replay")) {
+        g_str_equal(type, "filter-replay") ||
+        g_str_equal(type, "filter-ubpf")) {
         return false;
     }
 
Signature · 1 line
2.25.1
author
#23 lore raw

RFC PATCH 10/12 net/filter-ubpf.c: run the ubpf program to handle network packet

In reply to Zhang Chen

Run the loaded userspace ebpf program with the packet.

---
 net/filter-ubpf.c | 40 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 38 insertions(+), 2 deletions(-)
net/filter-ubpf.c +38 -2
index c63a021759..554cc24d8f 100644
@@ -20,6 +20,8 @@
 #include "qemu/error-report.h"
 #include "trace.h"
 #include "ebpf/ubpf.h"
+#include "colo.h"
+#include "util.h"
 
 #define TYPE_FILTER_UBPF "filter-ubpf"
 OBJECT_DECLARE_SIMPLE_TYPE(FiliterUbpfState, FILTER_UBPF)
@@ -38,9 +40,43 @@ static ssize_t filter_ubpf_receive_iov(NetFilterState *nf,
                                        int iovcnt,
                                        NetPacketSent *sent_cb)
 {
-    /* TODO: handle packet by loaded userspace ebpf program */
+    FiliterUbpfState *s = FILTER_UBPF(nf);
+    size_t size;
+    char *buf;
+    Packet *pkt = NULL;
+    uint64_t result;
+
+    size = iov_size(iov, iovcnt);
+    if (!size) {
+        return 0;
+    }
+
+    buf = g_malloc(size);
+    if (unlikely(iov_to_buf(iov, iovcnt, 0, buf, size) != size)) {
+        g_free(buf);
+        return 0;
+    }
+
+    pkt = packet_new_nocopy(buf, size, 0);
 
-    return 0;
+    if (parse_packet_early(pkt)) {
+        packet_destroy(pkt, NULL);
+        pkt = NULL;
+        return 0;
+    }
+
+    if (s->ip_mode) {
+        result = qemu_ubpf_run_once(&s->ubpf, pkt->ip, sizeof(struct ip));
+    } else {
+        result = qemu_ubpf_run_once(&s->ubpf, pkt->data, pkt->size);
+    }
+
+    /* If result == 1, means trigger the ebpf program rules */
+    if (result) {
+        return -1;
+    } else {
+        return 0;
+    }
 }
 
 static void filter_ubpf_cleanup(NetFilterState *nf)
Signature · 1 line
2.25.1
author
#24 lore raw

RFC PATCH 11/12 docs/devel: Add userspace-ebpf.rst

In reply to Zhang Chen

Introduce userspace ebpf basic knowledge.

---
 docs/devel/userspace-ebpf.rst | 106 ++++++++++++++++++++++++++++++++++
 1 file changed, 106 insertions(+)
 create mode 100644 docs/devel/userspace-ebpf.rst
docs/devel/userspace-ebpf.rst new file +106 -0
new file mode 100644
index 0000000000..41eb9b04d6
@@ -0,0 +1,106 @@
+===========================
+Userspace eBPF support
+===========================
+
+eBPF is a revolutionary technology with origins in the Linux kernel that
+can run sandboxed programs in an operating system kernel. It is used to
+safely and efficiently extend the capabilities of the kernel without
+requiring to change kernel source code or load kernel
+modules.(from https://ebpf.io/)
+
+Recently, I worked on QEMU net filter related jobs, like netfilter/iptables
+in kernel. We noticed kernel extend the netfilter original cBPF to eBPF,
+
+It make Linux kernel have the ability to load code dynamically. Why not
+enable user space eBPF in QEMU? It can load binary eBPF program even
+when VM running. Add some hooks in QEMU as the user space eBPF load point.
+Do the things on different layers. The original idea from Jason Wang.
+
+
+That’s the advantages of kernel eBPF. Most of the functions can be
+implemented in QEMU. The Power of Programmability.
+
+    1). Safety:
+
+    Building on the foundation of seeing and understanding all system
+    calls and combining that with a packet and socket-level view of all
+    networking operations allows for revolutionary new approaches to
+    securing systems.
+
+    2). Tracing & Profiling:
+
+    The ability to attach eBPF programs to trace points as well as kernel
+    and user application probe points allows unprecedented visibility into
+    the runtime behavior of applications and the system itself.
+
+    3). Networking:
+
+    The combination of programmability and efficiency makes eBPF a natural
+    fit for all packet processing requirements of networking solutions.
+
+    4). Observability & Monitoring:
+
+    Instead of relying on static counters and gauges exposed by the
+    perating system, eBPF enables the collection & in-kernel aggregation
+    of custom metrics and generation of visibility events based on a wide
+    range of possible sources.
+
+    Qemu userspace ebpf design based on ubpf project (https://github.com/iovisor/ubpf).
+    The most mature userspace ebpf implementation. This project officially
+    support by iovisor(Like BCC and bpftrace). Qemu userspace ebpf make
+    the ubpf project as the git submodule.
+
+    Current implementation support load ebpf program and run it in
+    filter-ubpf module, developer can easy reuse the ubpf function in
+    Qemu's other modules from the function in /ebpf/ubpf.c, And it support JIT.
+    For the uBPF License is Apache License 2.0, It's OK to compatible
+    with QEMU’s GPLv2 LICENSE same as mason.
+
+    How to use it:
+    1. Write your ebpf C program. For example filter dst IP:
+
+    bpf_filter.c
+
+#include <arpa/inet.h>
+#include <stdint.h>
+
+#define ONE_ONE_ONE_ONE 0x01010101
+
+struct ipv4_header {
+    uint8_t ver_ihl;
+    uint8_t tos;
+    uint16_t total_length;
+    uint16_t id;
+    uint16_t frag;
+    uint8_t ttl;
+    uint8_t proto;
+    uint16_t csum;
+    uint32_t src;
+    uint32_t dst;
+};
+
+int is_dst_one_one_one_one(void *opaque) {
+    struct ipv4_header *ipv4_header = (struct ipv4_header*)opaque;
+
+    if (ntohl(ipv4_header->dst) == ONE_ONE_ONE_ONE) {
+        return 1;
+    }
+
+    return 0;
+}
+
+    2. Build it with clang:
+      clang -O2 -target bpf -c bpf_filter.c -o ip_dst.o
+
+    3. Load it with Qemu filter-ubpf:
+      -object filter-ubpf,netdev=hn0,id=ubpf1,queue=tx,ip-mode=on,
+       ubpf-handler=ip_dst.o
+
+    4. Boot the VM and it will filt IP dst 1.1.1.1 packet.
+
+
+TODO: Need to add more comments and test-case for ubpf, current
+      implementation not include ebpf verifier. Qemu is a userspace
+      program, not like kernel ebpf run code in kernel space, I think
+      if the someone want to hack Qemu code no need to load a malicious
+      ubpf program, he can hack Qemu code directly.
Signature · 1 line
2.25.1
author
#25 lore raw

RFC PATCH 12/12 test/qtest: Add ubpf basic test case

In reply to Zhang Chen

TODO: This test case does not work. Need add ubpf.h header in qtest compile "-I ../ubpf/vm -I ../ubpf/vm/inc".
I'm not sure if we need it in qtest. Because normal tests/qtest not including external module test case like fdt. Or we just need a qtest case for filter-ubpf module.
This test will load pre-compiled ebpf binary and run it in QEMU.

---
 tests/qtest/demo_ubpf.o   | Bin 0 -> 544 bytes
 tests/qtest/integer_5.mem | Bin 0 -> 4 bytes
 tests/qtest/meson.build   |   3 +-
 tests/qtest/ubpf-test.c   |  64 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 66 insertions(+), 1 deletion(-)
 create mode 100644 tests/qtest/demo_ubpf.o
 create mode 100644 tests/qtest/integer_5.mem
 create mode 100644 tests/qtest/ubpf-test.c
tests/qtest/demo_ubpf.o new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..960a411c224348548db42d9ae2716ae3ef4ea249
GIT binary patch
tests/qtest/integer_5.mem new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..a786e127004dd9e94e88fda7742d248237ad8885
GIT binary patch
tests/qtest/meson.build +2 -1
index 31287a9173..9cc629e22b 100644
@@ -94,7 +94,8 @@ qtests_i386 = \
    'migration-test',
    'test-x86-cpuid-compat',
    'numa-test',
-   'test-filter-redirector'
+   'test-filter-redirector',
+   'ubpf-test'
   ]
 
 if dbus_display
tests/qtest/ubpf-test.c new file +64 -0

64 lines not on this page · raw message

Signature · 1 line
2.25.1
#26 lore raw

Re: [RFC PATCH 12/12] test/qtest: Add ubpf basic test case

Quoted 9 lines · Zhang Chen

TODO: This test case does not work. Need add ubpf.h header in qtest compile "-I ../ubpf/vm -I ../ubpf/vm/inc".
I'm not sure if we need it in qtest. Because normal tests/qtest not including external module test case like fdt. Or we just need a qtest case for filter-ubpf module.
This test will load pre-compiled ebpf binary and run it in QEMU.

---

[...]

Quoted 60 lines
tests/qtest/ubpf-test.c new file +54 -0
new file mode 100644
index 0000000000..6e70a99320
@@ -0,0 +1,64 @@
+/*
+ * QEMU Userspace eBPF test case
+ *
+ * Copyright(C) 2022 Intel Corporation.
+ *
+ * Author:
+ *  Zhang Chen <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "libqtest.h"
+#include "ebpf/ubpf.h"
+
+/*
+ * Demo userspace ebpf program
+ * The test binary use clang to build this source code:
+ * demo_ubpf.c
+ *
+ * #include <stdint.h>
+ *
+ * static uint32_t double_it(uint32_t a)
+ * {
+ *      return (a * 2);
+ * }
+ *
+ * uint32_t bpf_prog(int32_t *arg) {
+ *       uint32_t result = 0;
+ *       result = double_it(*arg);
+ *
+ *       return result;
+ * }
+ *
+ * Build the userspace ebpf program binary file:
+ * clang -O2 -target bpf -c demo_ubpf.c -o demo_ubpf.o
+ *
+ * The external terget source:
+ * printf "%b" '\x05\x00\x00\x00' > integer_5.mem
+ *
+ */
+
+int main(int argc, char **argc)
+{
+    UbpfState u_ebpf;
+    char program_path[] = "demo_ubpf.o";
+    /* uBPF can read target from internal source or external source*/
+    char target_path[] = "integer_5.mem";
+
+    qemu_ubpf_init_jit(&u_ebpf, true);
+
+    g_assert_cmpuint(qemu_ubpf_prepare(&u_ebpf, program_path), ==, 0);

tests/qtest/ubpf-test.c line 64

Quoted 10 lines
tests/qtest/ubpf-test.c new file +10 -0
+
+    g_assert_true(qemu_ubpf_read_target(&u_ebpf, target_path));
+
+    g_assert_cmpuint(qemu_run_ubpf_once(&u_ebpf, u_ebpf.target,
+		     u_ebpf.target_len), ==, 10);
+
+    ubpf_destroy(u_ebpf.vm);
+
+    return 0;
+}

Apart from the #include "libqtest.h" there is nothing related to qtest in here ... should this maybe rather go into test/unit/ instead?

Thomas

#27 lore raw

RE: [RFC PATCH 12/12] test/qtest: Add ubpf basic test case

Quoted 91 lines

-----Original Message-----
From: Thomas Huth <[email protected]>
Sent: Friday, June 17, 2022 5:34 PM
To: Zhang, Chen <[email protected]>; Jason Wang <[email protected]>; qemu-dev <[email protected]>; Paolo Bonzini <[email protected]>; Daniel P. Berrangé <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>
Cc: Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 12/12] test/qtest: Add ubpf basic test case

Quoted 9 lines · Zhang Chen

TODO: This test case does not work. Need add ubpf.h header in qtest compile "-I ../ubpf/vm -I ../ubpf/vm/inc".
I'm not sure if we need it in qtest. Because normal tests/qtest not including external module test case like fdt. Or we just need a qtest case for filter-ubpf module.
This test will load pre-compiled ebpf binary and run it in QEMU.

---

[...]

Quoted 65 lines
tests/qtest/ubpf-test.c new renamed from tests/qtest/ubpf-test.c +0 -0

file mode 100644 index 0000000000..6e70a99320

tests/qtest/ubpf-test.c new file +60 -0
@@ -0,0 +1,64 @@
+/*
+ * QEMU Userspace eBPF test case
+ *
+ * Copyright(C) 2022 Intel Corporation.
+ *
+ * Author:
+ *  Zhang Chen <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "libqtest.h"
+#include "ebpf/ubpf.h"
+
+/*
+ * Demo userspace ebpf program
+ * The test binary use clang to build this source code:
+ * demo_ubpf.c
+ *
+ * #include <stdint.h>
+ *
+ * static uint32_t double_it(uint32_t a)
+ * {
+ *      return (a * 2);
+ * }
+ *
+ * uint32_t bpf_prog(int32_t *arg) {
+ *       uint32_t result = 0;
+ *       result = double_it(*arg);
+ *
+ *       return result;
+ * }
+ *
+ * Build the userspace ebpf program binary file:
+ * clang -O2 -target bpf -c demo_ubpf.c -o demo_ubpf.o
+ *
+ * The external terget source:
+ * printf "%b" '\x05\x00\x00\x00' > integer_5.mem
+ *
+ */
+
+int main(int argc, char **argc)
+{
+    UbpfState u_ebpf;
+    char program_path[] = "demo_ubpf.o";
+    /* uBPF can read target from internal source or external source*/
+    char target_path[] = "integer_5.mem";
+
+    qemu_ubpf_init_jit(&u_ebpf, true);
+
+    g_assert_cmpuint(qemu_ubpf_prepare(&u_ebpf, program_path), ==,
+ 0);
+
+    g_assert_true(qemu_ubpf_read_target(&u_ebpf, target_path));
+
+    g_assert_cmpuint(qemu_run_ubpf_once(&u_ebpf, u_ebpf.target,
+		     u_ebpf.target_len), ==, 10);
Quoted 8 lines

tests/qtest/ubpf-test.c line 64

Quoted 5 lines
tests/qtest/ubpf-test.c new file +5 -0
+
+    ubpf_destroy(u_ebpf.vm);
+
+    return 0;
+}

Apart from the #include "libqtest.h" there is nothing related to qtest in here ... should this maybe rather go into test/unit/ instead?

Rethink about it, I think you are right.
The only issue is can we involve submodule's header file in tests/unit? I can't find meson/fdt/SLIRP test cases in the tests.

Thanks
Chen

Quoted 2 lines

Thomas

#28 lore raw

Re: [RFC PATCH 12/12] test/qtest: Add ubpf basic test case

Quoted 24 lines · Chen
Quoted 24 lines

-----Original Message-----
From: Thomas Huth <[email protected]>
Sent: Friday, June 17, 2022 5:34 PM
To: Zhang, Chen <[email protected]>; Jason Wang <[email protected]>; qemu-dev <[email protected]>; Paolo Bonzini <[email protected]>; Daniel P. Berrangé <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>
Cc: Peter Maydell <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>; Andrew Melnychenko <[email protected]>
Subject: Re: [RFC PATCH 12/12] test/qtest: Add ubpf basic test case

Quoted 9 lines · Zhang Chen

TODO: This test case does not work. Need add ubpf.h header in qtest compile "-I ../ubpf/vm -I ../ubpf/vm/inc".
I'm not sure if we need it in qtest. Because normal tests/qtest not including external module test case like fdt. Or we just need a qtest case for filter-ubpf module.
This test will load pre-compiled ebpf binary and run it in QEMU.

---

[...]

Quoted 6 lines
Quoted 2 lines

Apart from the #include "libqtest.h" there is nothing related to qtest in here ... should this maybe rather go into test/unit/ instead?

Rethink about it, I think you are right.
The only issue is can we involve submodule's header file in tests/unit? I can't find meson/fdt/SLIRP test cases in the tests.

Well, the test should only be enabled in meson.build if the CONFIG_UBPF switch is also enabled, then you can be sure that the header files of ubpf are also available when it gets compiled.

Thomas

#29 lore raw
In reply to Zhang Chen

Hi all,
Nice idea.
It would be great if future patches would add the BPF map support(if uBPF allows it).

Quoted 128 lines · Zhang Chen

Hi All,

The goal of this series is to bring the power of ebpf to QEMU.
It makes QEMU have the ability to extend the capabilities without requiring changing source code. Just need to load the eBPF binary file even at VM runtime. And already have some userspace ebpf implementation like: Intel DPDK eBPF, windows eBPF, etc.. The original idea suggested by Jason Wang.

eBPF is a revolutionary technology with origins in the Linux kernel
that can run sandboxed programs in an operating system kernel. It is used to safely and efficiently extend the capabilities of the kernel without requiring to change kernel source code or load kernel modules.(from https://ebpf.io/)

KVM already got benefits from it, but QEMU did not. Hence we want
to bring the power of eBPF to QEMU. It can load binary eBPF program even when VM running. At the same time, add some hooks in QEMU as the user space eBPF load point. Do the things on different layers.

That’s the advantages of kernel eBPF. Most of the functions can be
implemented in QEMU. This series just a start of the Power of Programmability.

1). Safety:

Building on the foundation of seeing and understanding all system calls and combining that with a packet and socket-level view of all networking operations allows for revolutionary new approaches to securing systems.

2). Tracing & Profiling:

The ability to attach eBPF programs to trace points as well as kernel and user application probe points allows unprecedented visibility into the runtime behavior of applications and the system itself.

3). Networking:

The combination of programmability and efficiency makes eBPF a natural fit for all packet processing requirements of networking solutions.

4). Observability & Monitoring:

Instead of relying on static counters and gauges exposed by the perating system, eBPF enables the collection & in-kernel aggregation of custom metrics and generation of visibility events based on a wide range of possible sources.

QEMU userspace ebpf design based on ubpf project (https://github.com/iovisor/ubpf).
The most mature userspace ebpf implementation. This project officially support by iovisor(Like BCC and bpftrace). This project includes an eBPF assembler, disassembler, interpreter (for all platforms), and JIT compiler (for x86-64 and Arm64 targets). Qemu userspace ebpf make the ubpf project as the git submodule.

Current implementation support load ebpf program and run it in
net/filter-ubpf module, this filter can support any user defined rules to hanle network packet. At the same time, it's easy for other developers to use the ubpf infrastructue in QEMU's other modules from the function in /ebpf/ubpf.c, and it support JIT.

For the uBPF License is Apache License 2.0, It's OK to compatible
with QEMU’s GPLv2 LICENSE same as mason.

TODO: Need to add more comments and test-case for ubpf, current
implementation not include ebpf verifier. But I think maybe it's not a big problem, current ebpf load/unload API exposed by QMP command. Qemu is a userspace program, if someone want to hack QEMU, no need to load a malicious ubpf program, it can hack QEMU code or crash QEMU on host directly(different from kernel ebpf needs strict inspection, but yes, it still need basic check).

Any comments are welcome.

Thanks
Chen

Zhang Chen (12):
configure: Add iovisor/ubpf project as a submodule for QEMU
meson: Add ubpf build config and misc
ebpf/uBPF: Introduce userspace ebpf data structure ebpf/uBPF: Introduce ubpf initialize functions ebpf/uBPF: Add qemu_prepare_ubpf to load ebpf binary ebpf/uBPF: Add qemu_ubpf_run_once excute real ebpf program net/filter: Introduce filter-ubpf module
qapi: Add FilterUbpfProperties and qemu-options softmmu/vl.c: Add filter-ubpf for netdev as other netfilters net/filter-ubpf.c: run the ubpf program to handle network packet docs/devel: Add userspace-ebpf.rst
test/qtest: Add ubpf basic test case

 .gitmodules                         |   3 +
 configure                           |  20 +++
 docs/devel/userspace-ebpf.rst       | 106 ++++++++++++++
 ebpf/meson.build                    |   1 +
 ebpf/ubpf-stub.c                    |  35 +++++
 ebpf/ubpf.c                         | 217 ++++++++++++++++++++++++++++
 ebpf/ubpf.h                         |  44 ++++++
 meson.build                         |  47 ++++++
 meson_options.txt                   |   3 +
 net/filter-ubpf.c                   | 185 ++++++++++++++++++++++++
 net/meson.build                     |   1 +
 qapi/qom.json                       |  18 +++
 qemu-options.hx                     |   6 +
 scripts/coverity-scan/COMPONENTS.md |   3 +
 scripts/meson-buildoptions.sh       |   5 +
 softmmu/vl.c                        |   3 +-
 tests/qtest/demo_ubpf.o             | Bin 0 -> 544 bytes
 tests/qtest/integer_5.mem           | Bin 0 -> 4 bytes
 tests/qtest/meson.build             |   3 +-
 tests/qtest/ubpf-test.c             |  64 ++++++++
 ubpf                                |   1 +
 21 files changed, 763 insertions(+), 2 deletions(-)
 create mode 100644 docs/devel/userspace-ebpf.rst
 create mode 100644 ebpf/ubpf-stub.c
 create mode 100644 ebpf/ubpf.c
 create mode 100644 ebpf/ubpf.h
 create mode 100644 net/filter-ubpf.c
 create mode 100644 tests/qtest/demo_ubpf.o
 create mode 100644 tests/qtest/integer_5.mem
 create mode 100644 tests/qtest/ubpf-test.c
 create mode 160000 ubpf

--
2.25.1

#30 lore raw
Quoted 16 lines

-----Original Message-----
From: Andrew Melnichenko <[email protected]>
Sent: Wednesday, June 29, 2022 6:43 PM
To: Zhang, Chen <[email protected]>
Cc: Jason Wang <[email protected]>; qemu-dev <qemu- [email protected]>; Paolo Bonzini <[email protected]>; Daniel P. Berrangé <[email protected]>; Eduardo Habkost <[email protected]>; Eric Blake <[email protected]>; Markus Armbruster <[email protected]>; Peter Maydell <[email protected]>; Thomas Huth <[email protected]>; Laurent Vivier <[email protected]>; Yuri Benditovich <[email protected]>
Subject: Re: [RFC PATCH 00/12] Introduce QEMU userspace ebpf support

Quoted 4 lines

Hi all,
Nice idea.
It would be great if future patches would add the BPF map support(if uBPF allows it).

The BPF map support is very useful. But current uBPF project don't support this yet. According to the previous discussion with Thomas and Daniel, we should avoid Introduce new git submodule for QEMU. And related general discussion:
Why we should avoid new submodules if possible:
https://www.mail-archive.com/[email protected]/msg897339.html

I think for the future development, we have to submit patch to the uBPF project.

Thanks
Chen

Quoted 130 lines · Zhang Chen
Quoted 16 lines

Hi All,

The goal of this series is to bring the power of ebpf to QEMU.
It makes QEMU have the ability to extend the capabilities without requiring changing source code. Just need to load the eBPF binary file even at VM runtime. And already have some userspace ebpf implementation like: Intel DPDK eBPF, windows eBPF, etc.. The original idea suggested by Jason Wang.

eBPF is a revolutionary technology with origins in the Linux
kernel that can run sandboxed programs in an operating system kernel. It is used to safely and efficiently extend the capabilities of the kernel without requiring to change kernel source code or load kernel modules.(from https://ebpf.io/)

Quoted 3 lines

KVM already got benefits from it, but QEMU did not. Hence we want
to bring the power of eBPF to QEMU. It can load binary eBPF program even when VM running. At the same time, add some hooks in QEMU as

the

Quoted 4 lines

user space eBPF load point. Do the things on different layers.

That’s the advantages of kernel eBPF. Most of the functions can be
implemented in QEMU. This series just a start of the Power of

Programmability.

Quoted 19 lines

1). Safety:

Building on the foundation of seeing and understanding all system calls and combining that with a packet and socket-level view of all networking operations allows for revolutionary new approaches to securing systems.

2). Tracing & Profiling:

The ability to attach eBPF programs to trace points as well as kernel and user application probe points allows unprecedented visibility into the runtime behavior of applications and the system itself.

3). Networking:

The combination of programmability and efficiency makes eBPF a natural fit for all packet processing requirements of networking solutions.

Quoted 8 lines

4). Observability & Monitoring:

Instead of relying on static counters and gauges exposed by the perating system, eBPF enables the collection & in-kernel aggregation of custom metrics and generation of visibility events based on a wide range of possible sources.

QEMU userspace ebpf design based on ubpf project

(https://github.com/iovisor/ubpf).

Quoted 65 lines

The most mature userspace ebpf implementation. This project officially support by iovisor(Like BCC and bpftrace). This project includes an eBPF assembler, disassembler, interpreter (for all platforms), and JIT compiler (for x86-64 and Arm64 targets). Qemu userspace ebpf make the ubpf project as the git submodule.

Current implementation support load ebpf program and run it in
net/filter-ubpf module, this filter can support any user defined rules to hanle network packet. At the same time, it's easy for other developers to use the ubpf infrastructue in QEMU's other modules from the function in /ebpf/ubpf.c, and it support JIT.

For the uBPF License is Apache License 2.0, It's OK to compatible
with QEMU’s GPLv2 LICENSE same as mason.

TODO: Need to add more comments and test-case for ubpf, current
implementation not include ebpf verifier. But I think maybe it's not a big problem, current ebpf load/unload API exposed by QMP command. Qemu is a userspace program, if someone want to hack QEMU, no need to load a malicious ubpf program, it can hack QEMU code or crash QEMU on host directly(different from kernel ebpf needs strict inspection, but yes, it still need basic check).

Any comments are welcome.

Thanks
Chen

Zhang Chen (12):
configure: Add iovisor/ubpf project as a submodule for QEMU
meson: Add ubpf build config and misc
ebpf/uBPF: Introduce userspace ebpf data structure ebpf/uBPF: Introduce ubpf initialize functions ebpf/uBPF: Add qemu_prepare_ubpf to load ebpf binary ebpf/uBPF: Add qemu_ubpf_run_once excute real ebpf program net/filter: Introduce filter-ubpf module
qapi: Add FilterUbpfProperties and qemu-options softmmu/vl.c: Add filter-ubpf for netdev as other netfilters net/filter-ubpf.c: run the ubpf program to handle network packet docs/devel: Add userspace-ebpf.rst
test/qtest: Add ubpf basic test case

 .gitmodules                         |   3 +
 configure                           |  20 +++
 docs/devel/userspace-ebpf.rst       | 106 ++++++++++++++
 ebpf/meson.build                    |   1 +
 ebpf/ubpf-stub.c                    |  35 +++++
 ebpf/ubpf.c                         | 217 ++++++++++++++++++++++++++++
 ebpf/ubpf.h                         |  44 ++++++
 meson.build                         |  47 ++++++
 meson_options.txt                   |   3 +
 net/filter-ubpf.c                   | 185 ++++++++++++++++++++++++
 net/meson.build                     |   1 +
 qapi/qom.json                       |  18 +++
 qemu-options.hx                     |   6 +
 scripts/coverity-scan/COMPONENTS.md |   3 +
 scripts/meson-buildoptions.sh       |   5 +
 softmmu/vl.c                        |   3 +-
 tests/qtest/demo_ubpf.o             | Bin 0 -> 544 bytes
 tests/qtest/integer_5.mem           | Bin 0 -> 4 bytes
 tests/qtest/meson.build             |   3 +-
 tests/qtest/ubpf-test.c             |  64 ++++++++
 ubpf                                |   1 +
 21 files changed, 763 insertions(+), 2 deletions(-)  create mode
Quoted 4 lines

100644 docs/devel/userspace-ebpf.rst create mode 100644 ebpf/ubpf-stub.c create mode 100644 ebpf/ubpf.c create mode 100644 ebpf/ubpf.h create mode 100644 net/filter-ubpf.c create mode 100644 tests/qtest/demo_ubpf.o create mode 100644

tests/qtest/integer_5.mem

Quoted 5 lines

create mode 100644 tests/qtest/ubpf-test.c create mode 160000 ubpf

--
2.25.1