Add vmnet.framework based network backend
Subsystems (2)
v5 cover Add vmnet.framework based network backend
macOS provides networking API for VMs called 'vmnet.framework':
https://developer.apple.com/documentation/vmnet
We can provide its support as the new QEMU network backends which represent three different vmnet.framework interface usage modes:
* vmnet-shared:
allows the guest to communicate with other guests in shared mode and also with external network (Internet) via NAT. Has (macOS-provided) DHCP server; subnet mask and IP range can be configured;
* vmnet-host:
allows the guest to communicate with other guests in host mode. By default has enabled DHCP as vmnet-shared, but providing network unique id (uuid) can make vmnet-host interfaces isolated from each other and also disables DHCP.
* vmnet-bridged:
bridges the guest with a physical network interface.
This backends cannot work on macOS Catalina 10.15 cause we use vmnet.framework API provided only with macOS 11 and newer. Seems that it is not a problem, because QEMU guarantees to work on two most recent versions of macOS which now are Big Sur (11) and Monterey (12).
Also, we have one inconvenient restriction: vmnet.framework interfaces can create only privileged user:$ sudo qemu-system-x86_64 -nic vmnet-shared
Attempt of vmnet-* netdev creation being unprivileged user fails with vmnet's 'general failure'.
This happens because vmnet.framework requires com.apple.vm.networking entitlement which is: "restricted to developers of virtualization software. To request this entitlement, contact your Apple representative." as Apple documentation says:
https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networking
One more note: we still have quite useful but not supported 'vmnet.framework' features as creating port forwarding rules, IPv6 NAT prefix specifying and so on.
Nevertheless, new backends work fine and tested within qemu-system-x86-64 on macOS Bir Sur 11.5.2 host with such nic models:
* e1000-82545em
* virtio-net-pci
* vmxnet3
The guests were:
* macOS 10.15.7
* Ubuntu Bionic (server cloudimg)
This series partially reuses patches by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/
So I included his signed-off line into one of the commit messages and also here.
v1 -> v2:
Since v1 minor typos were fixed, patches rebased onto latest master, redundant changes removed (small commits squashed)
v2 -> v3:
- QAPI style fixes
- Typos fixes in comments
- #include's updated to be in sync with recent master
v3 -> v4:
- Support vmnet interfaces isolation feature
- Support vmnet-host network uuid setting feature
- Refactored sources a bit
v4 -> v5:
- Missed 6.2 boat, now 7.0 candidate
- Fix qapi netdev descriptions and styles (@subnetmask -> @subnet-mask)
- Support vmnet-shared IPv6 prefix setting feature
Vladislav Yaroshchuk (6):
net/vmnet: add vmnet dependency and customizable option net/vmnet: add vmnet backends to qapi/net net/vmnet: implement shared mode (vmnet-shared) net/vmnet: implement host mode (vmnet-host) net/vmnet: implement bridged mode (vmnet-bridged) net/vmnet: update qemu-options.hx
meson.build | 4 + meson_options.txt | 2 + net/clients.h | 11 ++ net/meson.build | 7 + net/net.c | 10 ++ net/vmnet-bridged.m | 111 ++++++++++++ net/vmnet-common.m | 325 ++++++++++++++++++++++++++++++++++ net/vmnet-host.c | 111 ++++++++++++ net/vmnet-shared.c | 92 ++++++++++ net/vmnet_int.h | 48 +++++ qapi/net.json | 127 ++++++++++++- qemu-options.hx | 25 +++ scripts/meson-buildoptions.sh | 3 + 13 files changed, 874 insertions(+), 2 deletions(-) create mode 100644 net/vmnet-bridged.m create mode 100644 net/vmnet-common.m create mode 100644 net/vmnet-host.c create mode 100644 net/vmnet-shared.c create mode 100644 net/vmnet_int.h
Signature · 1 line
2.23.0
PATCH v5 1/6 net/vmnet: add vmnet dependency and customizable option
--- meson.build | 4 ++++ meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 9 insertions(+)
meson.build +4 -0
@@ -477,6 +477,8 @@ if cocoa.found() and get_option('gtk').enabled() error('Cocoa and GTK+ cannot be enabled at the same time') endif +vmnet = dependency('appleframeworks', modules: 'vmnet', required: get_option('vmnet')) + seccomp = not_found if not get_option('seccomp').auto() or have_system or have_tools seccomp = dependency('libseccomp', version: '>=2.3.0',
@@ -1455,6 +1457,7 @@ config_host_data.set('CONFIG_SECCOMP', seccomp.found()) config_host_data.set('CONFIG_SNAPPY', snappy.found()) config_host_data.set('CONFIG_USB_LIBUSB', libusb.found()) config_host_data.set('CONFIG_VDE', vde.found()) +config_host_data.set('CONFIG_VMNET', vmnet.found()) config_host_data.set('CONFIG_VHOST_USER_BLK_SERVER', have_vhost_user_blk_server) config_host_data.set('CONFIG_VNC', vnc.found()) config_host_data.set('CONFIG_VNC_JPEG', jpeg.found())
@@ -3383,6 +3386,7 @@ endif summary_info += {'JACK support': jack} summary_info += {'brlapi support': brlapi} summary_info += {'vde support': vde} +summary_info += {'vmnet.framework support': vmnet} summary_info += {'netmap support': have_netmap} summary_info += {'l2tpv3 support': have_l2tpv3} summary_info += {'Linux AIO support': libaio}
meson_options.txt +2 -0
@@ -147,6 +147,8 @@ option('netmap', type : 'feature', value : 'auto', description: 'netmap network backend support') option('vde', type : 'feature', value : 'auto', description: 'vde network backend support') +option('vmnet', type : 'feature', value : 'auto', + description: 'vmnet.framework network backend support') option('virglrenderer', type : 'feature', value : 'auto', description: 'virgl rendering support') option('vnc', type : 'feature', value : 'auto',
scripts/meson-buildoptions.sh +3 -0
@@ -81,6 +81,7 @@ meson_options_help() { printf "%s\n" ' u2f U2F emulation support' printf "%s\n" ' usb-redir libusbredir support' printf "%s\n" ' vde vde network backend support' + printf "%s\n" ' vmnet vmnet.framework network backend support' printf "%s\n" ' vhost-user-blk-server' printf "%s\n" ' build vhost-user-blk server' printf "%s\n" ' virglrenderer virgl rendering support'
@@ -239,6 +240,8 @@ _meson_option_parse() { --disable-usb-redir) printf "%s" -Dusb_redir=disabled ;; --enable-vde) printf "%s" -Dvde=enabled ;; --disable-vde) printf "%s" -Dvde=disabled ;; + --enable-vmnet) printf "%s" -Dvmnet=enabled ;; + --disable-vmnet) printf "%s" -Dvmnet=disabled ;; --enable-vhost-user-blk-server) printf "%s" -Dvhost_user_blk_server=enabled ;; --disable-vhost-user-blk-server) printf "%s" -Dvhost_user_blk_server=disabled ;; --enable-virglrenderer) printf "%s" -Dvirglrenderer=enabled ;;
Signature · 1 line
2.23.0
PATCH v5 2/6 net/vmnet: add vmnet backends to qapi/net
In reply to Vladislav YaroshchukCreate separate netdevs for each vmnet operating mode:
- vmnet-host
- vmnet-shared
- vmnet-bridged
--- net/clients.h | 11 ++++ net/meson.build | 7 +++ net/net.c | 10 ++++ net/vmnet-bridged.m | 25 +++++++++ net/vmnet-common.m | 20 +++++++ net/vmnet-host.c | 24 +++++++++ net/vmnet-shared.c | 25 +++++++++ net/vmnet_int.h | 25 +++++++++ qapi/net.json | 127 +++++++++++++++++++++++++++++++++++++++++++- 9 files changed, 272 insertions(+), 2 deletions(-) create mode 100644 net/vmnet-bridged.m create mode 100644 net/vmnet-common.m create mode 100644 net/vmnet-host.c create mode 100644 net/vmnet-shared.c create mode 100644 net/vmnet_int.h
net/clients.h +11 -0
@@ -63,4 +63,15 @@ int net_init_vhost_user(const Netdev *netdev, const char *name,int net_init_vhost_vdpa (const Netdev *netdev,const char *name,NetClientState *peer,Error **errp); +#ifdef CONFIG_VMNET +int net_init_vmnet_host (const Netdev *netdev,const char *name, +NetClientState *peer,Error **errp); + +int net_init_vmnet_shared (const Netdev *netdev,const char *name, +NetClientState *peer,Error **errp); + +int net_init_vmnet_bridged (const Netdev *netdev,const char *name, +NetClientState *peer,Error **errp); +#endif /* CONFIG_VMNET */ +#endif /* QEMU_NET_CLIENTS_H */
net/meson.build +7 -0
@@ -42,4 +42,11 @@ softmmu_ss.add(when: 'CONFIG_POSIX', if_true: files(tap_posix)) softmmu_ss.add(when: 'CONFIG_WIN32', if_true: files('tap-win32.c')) softmmu_ss.add(when: 'CONFIG_VHOST_NET_VDPA', if_true: files('vhost-vdpa.c')) +vmnet_files = files( + 'vmnet-common.m', + 'vmnet-bridged.m', + 'vmnet-host.c', + 'vmnet-shared.c' +) +softmmu_ss.add(when: vmnet, if_true: vmnet_files) subdir('can')
net/net.c +10 -0
@@ -1021,6 +1021,11 @@ static int (* const net_client_init_fun[NET_CLIENT_DRIVER__MAX])( #ifdef CONFIG_L2TPV3 [NET_CLIENT_DRIVER_L2TPV3 ] = net_init_l2tpv3, #endif +#ifdef CONFIG_VMNET + [NET_CLIENT_DRIVER_VMNET_HOST] = net_init_vmnet_host, + [NET_CLIENT_DRIVER_VMNET_SHARED] = net_init_vmnet_shared, + [NET_CLIENT_DRIVER_VMNET_BRIDGED] = net_init_vmnet_bridged, +#endif /* CONFIG_VMNET */ };
@@ -1106,6 +1111,11 @@ void show_netdevs(void)#endif #ifdef CONFIG_VHOST_VDPA "vhost-vdpa", +#endif +#ifdef CONFIG_VMNET +"vmnet-host" , +"vmnet-shared" , + "vmnet-bridged", #endif };
net/vmnet-bridged.m new file +25 -0
@@ -0,0 +1,25 @@ +/* + * vmnet-bridged.m + * + * Copyright(c) 2021 Vladislav Yaroshchuk <[email protected]> + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qapi/qapi-types-net.h" +#include "vmnet_int.h" +#include "clients.h" +#include "qemu/error-report.h" +#include "qapi/error.h" + +#include <vmnet/vmnet.h> + +int net_init_vmnet_bridged(const Netdev *netdev, const char *name, + NetClientState *peer, Error **errp) +{ + error_setg(errp, "vmnet-bridged is not implemented yet"); + return -1; +}
net/vmnet-common.m new file +20 -0
@@ -0,0 +1,20 @@ +/* + * vmnet-common.m - network client wrapper for Apple vmnet.framework + * + * Copyright(c) 2021 Vladislav Yaroshchuk <[email protected]> + * Copyright(c) 2021 Phillip Tennen <[email protected]> + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qapi/qapi-types-net.h" +#include "vmnet_int.h" +#include "clients.h" +#include "qemu/error-report.h" +#include "qapi/error.h" + +#include <vmnet/vmnet.h> +
net/vmnet-host.c new file +24 -0
@@ -0,0 +1,24 @@ +/* +* vmnet-host.c +* +* Copyright(c) 2021 Vladislav Yaroshchuk <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#include "qemu/osdep.h" +#include "qapi/qapi-types-net.h" +#include "vmnet_int.h" +#include "clients.h" +#include "qemu/error-report.h" +#include "qapi/error.h" + +#include <vmnet/vmnet.h> + +int net_init_vmnet_host (const Netdev *netdev,const char *name, +NetClientState *peer,Error **errp) { +error_setg (errp,"vmnet-host is not implemented yet" ); +return -1 ; +}
net/vmnet-shared.c new file +25 -0
@@ -0,0 +1,25 @@ +/* +* vmnet-shared.c +* +* Copyright(c) 2021 Vladislav Yaroshchuk <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#include "qemu/osdep.h" +#include "qapi/qapi-types-net.h" +#include "vmnet_int.h" +#include "clients.h" +#include "qemu/error-report.h" +#include "qapi/error.h" + +#include <vmnet/vmnet.h> + +int net_init_vmnet_shared (const Netdev *netdev,const char *name, +NetClientState *peer,Error **errp) +{ +error_setg (errp,"vmnet-shared is not implemented yet" ); +return -1 ; +}
net/vmnet_int.h new file +25 -0
@@ -0,0 +1,25 @@ +/* +* vmnet_int.h +* +* Copyright(c) 2021 Vladislav Yaroshchuk <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ +#ifndef VMNET_INT_H +#define VMNET_INT_H + +#include "qemu/osdep.h" +#include "vmnet_int.h" +#include "clients.h" + +#include <vmnet/vmnet.h> + +typedef struct VmnetCommonState { +NetClientState nc; + +}VmnetCommonState ; + + +#endif /* VMNET_INT_H */
qapi/net.json +125 -2
154 lines not on this page · raw message
Signature · 1 line
2.23.0
PATCH v5 3/6 net/vmnet: implement shared mode (vmnet-shared)
In reply to Vladislav Yaroshchuk--- net/vmnet-common.m | 305 +++++++++++++++++++++++++++++++++++++++++++++ net/vmnet-shared.c | 75 ++++++++++- net/vmnet_int.h | 23 ++++ 3 files changed, 399 insertions(+), 4 deletions(-)
net/vmnet-common.m +305 -0
@@ -10,6 +10,8 @@ */ #include "qemu/osdep.h" +#include "qemu/main-loop.h" +#include "qemu/log.h" #include "qapi/qapi-types-net.h" #include "vmnet_int.h" #include "clients.h"
@@ -17,4 +19,307 @@ #include "qapi/error.h" #include <vmnet/vmnet.h> +#include <dispatch/dispatch.h> +#ifdef DEBUG +#define D(x) x +#define D_LOG(...) qemu_log(__VA_ARGS__) +#else +#define D(x) do { } while (0) +#define D_LOG(...) do { } while (0) +#endif + +typedef struct vmpktdesc vmpktdesc_t; +typedef struct iovec iovec_t; + +static void vmnet_set_send_enabled(VmnetCommonState *s, bool enable) +{ + s->send_enabled = enable; +} + + +static void vmnet_send_completed(NetClientState *nc, ssize_t len) +{ + VmnetCommonState *s = DO_UPCAST(VmnetCommonState, nc, nc); + vmnet_set_send_enabled(s, true); +} + + +static void vmnet_send(NetClientState *nc, + interface_event_t event_id, + xpc_object_t event) +{ + assert(event_id == VMNET_INTERFACE_PACKETS_AVAILABLE); + + VmnetCommonState *s; + uint64_t packets_available; + + struct iovec *iov; + struct vmpktdesc *packets; + int pkt_cnt; + int i; + + vmnet_return_t if_status; + ssize_t size; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + packets_available = xpc_dictionary_get_uint64( + event, + vmnet_estimated_packets_available_key + ); + + pkt_cnt = (packets_available < VMNET_PACKETS_LIMIT) ? + packets_available : + VMNET_PACKETS_LIMIT; + + + iov = s->iov_buf; + packets = s->packets_buf; + + for (i = 0; i < pkt_cnt; ++i) { + packets[i].vm_pkt_size = s->max_packet_size; + packets[i].vm_pkt_iovcnt = 1; + packets[i].vm_flags = 0; + } + + if_status = vmnet_read(s->vmnet_if, packets, &pkt_cnt); + if (if_status != VMNET_SUCCESS) { + error_printf("vmnet: read failed: %s\n", + vmnet_status_map_str(if_status)); + } + qemu_mutex_lock_iothread(); + for (i = 0; i < pkt_cnt; ++i) { + size = qemu_send_packet_async(nc, + iov[i].iov_base, + packets[i].vm_pkt_size, + vmnet_send_completed); + if (size == 0) { + vmnet_set_send_enabled(s, false); + } else if (size < 0) { + break; + } + } + qemu_mutex_unlock_iothread(); + +} + + +static void vmnet_register_event_callback(VmnetCommonState *s) +{ + dispatch_queue_t avail_pkt_q = dispatch_queue_create( + "org.qemu.vmnet.if_queue", + DISPATCH_QUEUE_SERIAL + ); + + vmnet_interface_set_event_callback( + s->vmnet_if, + VMNET_INTERFACE_PACKETS_AVAILABLE, + avail_pkt_q, + ^(interface_event_t event_id, xpc_object_t event) { + if (s->send_enabled) { + vmnet_send(&s->nc, event_id, event); + } + }); +} + + +static void vmnet_bufs_init(VmnetCommonState *s) +{ + int i; + struct vmpktdesc *packets; + struct iovec *iov; + + packets = s->packets_buf; + iov = s->iov_buf; + + for (i = 0; i < VMNET_PACKETS_LIMIT; ++i) { + iov[i].iov_len = s->max_packet_size; + iov[i].iov_base = g_malloc0(iov[i].iov_len); + packets[i].vm_pkt_iov = iov + i; + } +} + + +const char *vmnet_status_map_str(vmnet_return_t status) +{ + switch (status) { + case VMNET_SUCCESS: + return "success"; + case VMNET_FAILURE: + return "general failure"; + case VMNET_MEM_FAILURE: + return "memory allocation failure"; + case VMNET_INVALID_ARGUMENT: + return "invalid argument specified"; + case VMNET_SETUP_INCOMPLETE: + return "interface setup is not complete"; + case VMNET_INVALID_ACCESS: + return "invalid access, permission denied"; + case VMNET_PACKET_TOO_BIG: + return "packet size is larger than MTU"; + case VMNET_BUFFER_EXHAUSTED: + return "buffers exhausted in kernel"; + case VMNET_TOO_MANY_PACKETS: + return "packet count exceeds limit"; + case VMNET_SHARING_SERVICE_BUSY: + return "conflict, sharing service is in use"; + default: + return "unknown vmnet error"; + } +} + + +int vmnet_if_create(NetClientState *nc, + xpc_object_t if_desc, + Error **errp, + void (*completion_callback)(xpc_object_t interface_param)) +{ + VmnetCommonState *s; + + dispatch_queue_t if_create_q; + dispatch_semaphore_t if_created_sem; + + __block vmnet_return_t if_status; + + if_create_q = dispatch_queue_create("org.qemu.vmnet.create", + DISPATCH_QUEUE_SERIAL); + if_created_sem = dispatch_semaphore_create(0); + + xpc_dictionary_set_bool( + if_desc, + vmnet_allocate_mac_address_key, + false + ); + + D(D_LOG("vmnet.start.interface_desc:\n"); + xpc_dictionary_apply(if_desc, + ^bool(const char *k, xpc_object_t v) { + char *desc = xpc_copy_description(v); + D_LOG(" %s=%s\n", k, desc); + free(desc); + return true; + })); + + s = DO_UPCAST(VmnetCommonState, nc, nc); + s->vmnet_if = vmnet_start_interface( + if_desc, + if_create_q, + ^(vmnet_return_t status, xpc_object_t interface_param) { + if_status = status; + if (status != VMNET_SUCCESS || !interface_param) { + dispatch_semaphore_signal(if_created_sem); + return; + } + + s->mtu = xpc_dictionary_get_uint64( + interface_param, + vmnet_mtu_key); + s->max_packet_size = xpc_dictionary_get_uint64( + interface_param, + vmnet_max_packet_size_key); + D(D_LOG("vmnet.start.interface_param:\n"); + xpc_dictionary_apply(interface_param, + ^bool(const char *k, xpc_object_t v) { + char *desc = xpc_copy_description(v); + D_LOG(" %s=%s\n", k, desc); + free(desc); + return true; + })); + dispatch_semaphore_signal(if_created_sem); + }); + + if (s->vmnet_if == NULL) { + error_setg(errp, "unable to create interface with requested params"); + return -1; + } + + dispatch_semaphore_wait(if_created_sem, DISPATCH_TIME_FOREVER); + dispatch_release(if_create_q); + + if (if_status != VMNET_SUCCESS) { + error_setg(errp, + "cannot create vmnet interface: %s", + vmnet_status_map_str(if_status)); + return -1; + } + + vmnet_register_event_callback(s); + vmnet_bufs_init(s); + vmnet_set_send_enabled(s, true); + + return 0; +} + + +ssize_t vmnet_receive_common(NetClientState *nc, + const uint8_t *buf, + size_t size) +{ + VmnetCommonState *s; + vmpktdesc_t packet; + iovec_t iov; + int pkt_cnt; + vmnet_return_t if_status; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + if (size > s->max_packet_size) { + warn_report("vmnet: packet is too big, %zu > %llu\n", + packet.vm_pkt_size, + s->max_packet_size); + return -1; + } + + iov.iov_base = (char *) buf; + iov.iov_len = size; + + packet.vm_pkt_iovcnt = 1; + packet.vm_flags = 0; + packet.vm_pkt_size = size; + packet.vm_pkt_iov = &iov; + + pkt_cnt = 1; + if_status = vmnet_write(s->vmnet_if, &packet, &pkt_cnt); + + if (if_status != VMNET_SUCCESS) { + error_report("vmnet: write error: %s\n", + vmnet_status_map_str(if_status)); + } + + if (if_status == VMNET_SUCCESS && pkt_cnt) { + return size; + } + return 0; +} + + +void vmnet_cleanup_common(NetClientState *nc) +{ + VmnetCommonState *s; + dispatch_queue_t if_destroy_q; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + qemu_purge_queued_packets(nc); + vmnet_set_send_enabled(s, false); + + if (s->vmnet_if == NULL) { + return; + } + + if_destroy_q = dispatch_queue_create( + "org.qemu.vmnet.destroy", + DISPATCH_QUEUE_SERIAL + ); + + vmnet_stop_interface( + s->vmnet_if, + if_destroy_q, + ^(vmnet_return_t status) { + }); + + for (int i = 0; i < VMNET_PACKETS_LIMIT; ++i) { + g_free(s->iov_buf[i].iov_base); + } +}
net/vmnet-shared.c +71 -4
87 lines not on this page · raw message
net/vmnet_int.h +23 -0
@@ -16,10 +16,33 @@#include <vmnet/vmnet.h> +#define VMNET_PACKETS_LIMIT 50 +typedef struct VmnetCommonState {NetClientState nc; +interface_ref vmnet_if; + +bool send_enabled; + +uint64_t mtu; +uint64_t max_packet_size; + +struct vmpktdesc packets_buf[VMNET_PACKETS_LIMIT ]; +struct iovec iov_buf[VMNET_PACKETS_LIMIT ]; }VmnetCommonState ; +const char *vmnet_status_map_str (vmnet_return_t status); + +int vmnet_if_create (NetClientState *nc, +xpc_object_t if_desc, +Error **errp, +void (*completion_callback)(xpc_object_t interface_param)); + +ssize_t vmnet_receive_common (NetClientState *nc, +const uint8_t *buf, +size_t size); + +void vmnet_cleanup_common (NetClientState *nc);#endif /* VMNET_INT_H */
Signature · 1 line
2.23.0
A message that is not in the archive (sent off-list or not yet received): CACGkMEuFGbH0xkp1K344kkgOzKcjn+iKQtgidgFWqmYkrG0KjQ@mail.gmail.com
Re: [PATCH v5 3/6] net/vmnet: implement shared mode (vmnet-shared)
пн, 15 нояб. 2021 г. в 07:47, Jason Wang <[email protected]>:
Quoted 11 lines
Quoted 4 lines · Vladislav Yaroshchuk
Signed-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>---
Commit log please.
Sorry, I don't understand what you mean here.
What is the 'commit log'?
Quoted 24 lines
Quoted 24 lines
net/vmnet-common.m | 305 +++++++++++++++++++++++++++++++++++++++++++++ net/vmnet-shared.c | 75 ++++++++++- net/vmnet_int.h | 23 ++++ 3 files changed, 399 insertions(+), 4 deletions(-)net/vmnet-common.m +3
-0@@ -10,6 +10,8 @@ */ #include "qemu/osdep.h" +#include "qemu/main-loop.h" +#include "qemu/log.h" #include "qapi/qapi-types-net.h" #include "vmnet_int.h" #include "clients.h"@@ -17,4 +19,307 @@ #include "qapi/error.h" #include <vmnet/vmnet.h> +#include <dispatch/dispatch.h>
Quoted 18 lines
Quoted 4 lines
net/vmnet-common.m +4
-0+#ifdef DEBUG +#define D(x) x +#define D_LOG(...) qemu_log(__VA_ARGS__) +#elsenet/vmnet-common.m line 37
Quoted 10 lines
net/vmnet-common.m +10
-0+#define D(x) do { } while (0) +#define D_LOG(...) do { } while (0) +#endif + +typedef struct vmpktdesc vmpktdesc_t; +typedef struct iovec iovec_t; + +static void vmnet_set_send_enabled(VmnetCommonState *s, bool enable) +{ + s->send_enabled = enable;Is there a way to disable the event when enable is false?
It seems there's no way except setting/unsetting the callback via vmnet_interface_set_event_callback. I decided to drop packages using s->send_enabled without dealing with the callback.
Quoted 240 lines
+}
Quoted 128 lines
net/vmnet-common.m +128
-0+ + +static void vmnet_send_completed(NetClientState *nc, ssize_t len) +{ + VmnetCommonState *s = DO_UPCAST(VmnetCommonState, nc, nc); + vmnet_set_send_enabled(s, true); +} + + +static void vmnet_send(NetClientState *nc, + interface_event_t event_id, + xpc_object_t event) +{ + assert(event_id == VMNET_INTERFACE_PACKETS_AVAILABLE); + + VmnetCommonState *s; + uint64_t packets_available; + + struct iovec *iov; + struct vmpktdesc *packets; + int pkt_cnt; + int i; + + vmnet_return_t if_status; + ssize_t size; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + packets_available = xpc_dictionary_get_uint64( + event, + vmnet_estimated_packets_available_key + ); + + pkt_cnt = (packets_available < VMNET_PACKETS_LIMIT) ? + packets_available : + VMNET_PACKETS_LIMIT; + + + iov = s->iov_buf; + packets = s->packets_buf; + + for (i = 0; i < pkt_cnt; ++i) { + packets[i].vm_pkt_size = s->max_packet_size; + packets[i].vm_pkt_iovcnt = 1; + packets[i].vm_flags = 0; + } + + if_status = vmnet_read(s->vmnet_if, packets, &pkt_cnt); + if (if_status != VMNET_SUCCESS) { + error_printf("vmnet: read failed: %s\n", + vmnet_status_map_str(if_status)); + } + qemu_mutex_lock_iothread(); + for (i = 0; i < pkt_cnt; ++i) { + size = qemu_send_packet_async(nc, + iov[i].iov_base, + packets[i].vm_pkt_size, + vmnet_send_completed); + if (size == 0) { + vmnet_set_send_enabled(s, false); + } else if (size < 0) { + break; + } + } + qemu_mutex_unlock_iothread(); + +} + + +static void vmnet_register_event_callback(VmnetCommonState *s) +{ + dispatch_queue_t avail_pkt_q = dispatch_queue_create( + "org.qemu.vmnet.if_queue", + DISPATCH_QUEUE_SERIAL + ); + + vmnet_interface_set_event_callback( + s->vmnet_if, + VMNET_INTERFACE_PACKETS_AVAILABLE, + avail_pkt_q, + ^(interface_event_t event_id, xpc_object_t event) { + if (s->send_enabled) { + vmnet_send(&s->nc, event_id, event); + } + }); +} + + +static void vmnet_bufs_init(VmnetCommonState *s) +{ + int i; + struct vmpktdesc *packets; + struct iovec *iov; + + packets = s->packets_buf; + iov = s->iov_buf; + + for (i = 0; i < VMNET_PACKETS_LIMIT; ++i) { + iov[i].iov_len = s->max_packet_size; + iov[i].iov_base = g_malloc0(iov[i].iov_len); + packets[i].vm_pkt_iov = iov + i; + } +} + + +const char *vmnet_status_map_str(vmnet_return_t status) +{ + switch (status) { + case VMNET_SUCCESS: + return "success"; + case VMNET_FAILURE: + return "general failure"; + case VMNET_MEM_FAILURE: + return "memory allocation failure"; + case VMNET_INVALID_ARGUMENT: + return "invalid argument specified"; + case VMNET_SETUP_INCOMPLETE: + return "interface setup is not complete"; + case VMNET_INVALID_ACCESS: + return "invalid access, permission denied"; + case VMNET_PACKET_TOO_BIG: + return "packet size is larger than MTU"; + case VMNET_BUFFER_EXHAUSTED: + return "buffers exhausted in kernel"; + case VMNET_TOO_MANY_PACKETS: + return "packet count exceeds limit"; + case VMNET_SHARING_SERVICE_BUSY: + return "conflict, sharing service is in use";net/vmnet-common.m
Quoted 10 lines
net/vmnet-common.m +10
-0+ default: + return "unknown vmnet error"; + } +} + + +int vmnet_if_create(NetClientState *nc, + xpc_object_t if_desc, + Error **errp, + void (*completion_callback)(xpc_object_t
interface_param))Quoted 47 lines
net/vmnet-common.m +47
-0+{ + VmnetCommonState *s; + + dispatch_queue_t if_create_q; + dispatch_semaphore_t if_created_sem; + + __block vmnet_return_t if_status; + + if_create_q = dispatch_queue_create("org.qemu.vmnet.create", + DISPATCH_QUEUE_SERIAL); + if_created_sem = dispatch_semaphore_create(0); + + xpc_dictionary_set_bool( + if_desc, + vmnet_allocate_mac_address_key, + false + ); + + D(D_LOG("vmnet.start.interface_desc:\n"); + xpc_dictionary_apply(if_desc, + ^bool(const char *k, xpc_object_t v) { + char *desc = xpc_copy_description(v); + D_LOG(" %s=%s\n", k, desc); + free(desc); + return true; + })); + + s = DO_UPCAST(VmnetCommonState, nc, nc); + s->vmnet_if = vmnet_start_interface( + if_desc, + if_create_q, + ^(vmnet_return_t status, xpc_object_t interface_param) { + if_status = status; + if (status != VMNET_SUCCESS || !interface_param) { + dispatch_semaphore_signal(if_created_sem); + return; + } + + s->mtu = xpc_dictionary_get_uint64( + interface_param, + vmnet_mtu_key); + s->max_packet_size = xpc_dictionary_get_uint64( + interface_param, + vmnet_max_packet_size_key); + D(D_LOG("vmnet.start.interface_param:\n"); + xpc_dictionary_apply(interface_param, + ^bool(const char *k, xpc_object_t v) {net/vmnet-common.m
Quoted 10 lines
net/vmnet-common.m +10
-0+ char *desc = xpc_copy_description(v); + D_LOG(" %s=%s\n", k, desc); + free(desc); + return true; + })); + dispatch_semaphore_signal(if_created_sem); + }); + + if (s->vmnet_if == NULL) { + error_setg(errp, "unable to create interface with requestedparams");
Quoted 42 lines
net/vmnet-common.m +42
-0+ return -1; + } + + dispatch_semaphore_wait(if_created_sem, DISPATCH_TIME_FOREVER); + dispatch_release(if_create_q); + + if (if_status != VMNET_SUCCESS) { + error_setg(errp, + "cannot create vmnet interface: %s", + vmnet_status_map_str(if_status)); + return -1; + } + + vmnet_register_event_callback(s); + vmnet_bufs_init(s); + vmnet_set_send_enabled(s, true); + + return 0; +} + + +ssize_t vmnet_receive_common(NetClientState *nc, + const uint8_t *buf, + size_t size) +{ + VmnetCommonState *s; + vmpktdesc_t packet; + iovec_t iov; + int pkt_cnt; + vmnet_return_t if_status; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + if (size > s->max_packet_size) { + warn_report("vmnet: packet is too big, %zu > %llu\n", + packet.vm_pkt_size, + s->max_packet_size); + return -1; + } + + iov.iov_base = (char *) buf; + iov.iov_len = size;
Quoted 10 lines
net/vmnet-common.m line 280
Quoted 5 lines
net/vmnet-common.m +5
-0+ + packet.vm_pkt_iovcnt = 1; + packet.vm_flags = 0; + packet.vm_pkt_size = size; + packet.vm_pkt_iov = &iov;It looks to me vmnet framework supports iov so I wonder if a .receive_iov() is better because of its performance.
I've just tried to implement this call, and because of some reason vmnet_write fails with VMNET_INVALID_ARGUMENT when iovcnt > 1. Tested with vmxnet3. Collecting all the iovs into a big one and passing it to vmnet works fine (the default behaviour when only the .receive() but not the .receive_iov() is implemented).
This should be investigated, but currently I don't understand what exactly causes this error. The fact that vmnet.framework is a 'blackbox' makes the situation much worse.
Phillip's version is also broken:
https://patchew.org/QEMU/[email protected]/
Looks like this wasn't noticed before.
If it's applicable, we can use the .receive() only, and put .receive_iov() implementation to the backlog.
Quoted 181 lines
+
Quoted 121 lines
net/vmnet-common.m +44
-0+ pkt_cnt = 1; + if_status = vmnet_write(s->vmnet_if, &packet, &pkt_cnt); + + if (if_status != VMNET_SUCCESS) { + error_report("vmnet: write error: %s\n", + vmnet_status_map_str(if_status)); + } + + if (if_status == VMNET_SUCCESS && pkt_cnt) { + return size; + } + return 0; +} + + +void vmnet_cleanup_common(NetClientState *nc) +{ + VmnetCommonState *s; + dispatch_queue_t if_destroy_q; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + qemu_purge_queued_packets(nc); + vmnet_set_send_enabled(s, false); + + if (s->vmnet_if == NULL) { + return; + } + + if_destroy_q = dispatch_queue_create( + "org.qemu.vmnet.destroy", + DISPATCH_QUEUE_SERIAL + ); + + vmnet_stop_interface( + s->vmnet_if, + if_destroy_q, + ^(vmnet_return_t status) { + }); + + for (int i = 0; i < VMNET_PACKETS_LIMIT; ++i) { + g_free(s->iov_buf[i].iov_base); + } +}net/vmnet-shared.c +62
-2@@ -10,16 +10,83 @@#include "qemu/osdep.h" #include "qapi/qapi-types-net.h" +#include "qapi/error.h" #include "vmnet_int.h" #include "clients.h" -#include "qemu/error-report.h" -#include "qapi/error.h" #include <vmnet/vmnet.h> +typedef struct VmnetSharedState { +VmnetCommonState cs; +}VmnetSharedState ; + + +static xpc_object_t create_if_desc (const Netdev *netdev,Error **errp) +{ +const NetdevVmnetSharedOptions *options = &(netdev->u.vmnet_shared); +xpc_object_t if_desc =xpc_dictionary_create (NULL ,NULL ,0 ); + +xpc_dictionary_set_uint64 ( + if_desc, + vmnet_operation_mode_key, +VMNET_SHARED_MODE + ); + +xpc_dictionary_set_bool ( + if_desc, + vmnet_enable_isolation_key, + options->isolated + ); + +if (options->has_nat66_prefix) { +xpc_dictionary_set_string (if_desc, + vmnet_nat66_prefix_key, + options->nat66_prefix); + } + +if (options->has_dhcpstart || + options->has_dhcpend || + options->has_subnet_mask) { + +if (options->has_dhcpstart && + options->has_dhcpend && + options->has_subnet_mask) { + +xpc_dictionary_set_string (if_desc, + vmnet_start_address_key, + options->dhcpstart); +xpc_dictionary_set_string (if_desc, + vmnet_end_address_key, + options->dhcpend); +xpc_dictionary_set_string (if_desc, + vmnet_subnet_mask_key, + options->subnet_mask); + }else { +error_setg ( + errp, +"'dhcpstart', 'dhcpend', 'subnet_mask' " +"should be provided together" + ); + } + } + +return if_desc; +} + +static NetClientInfo net_vmnet_shared_info = { + .type =NET_CLIENT_DRIVER_VMNET_SHARED , + .size =sizeof (VmnetSharedState), + .receive = vmnet_receive_common,net/vmnet-shared.c line 87
Quoted 10 lines
net/vmnet-shared.c +5
-2+ .cleanup = vmnet_cleanup_common, +}; +int net_init_vmnet_shared (const Netdev *netdev,const char *name,NetClientState *peer,Error **errp) { -error_setg (errp,"vmnet-shared is not implemented yet" ); -return -1 ; + NetClientState *nc = qemu_new_net_client(&net_vmnet_shared_info, + peer, "vmnet-shared",name);
Quoted 25 lines
net/vmnet-shared.c +4
-0+xpc_object_t if_desc =create_if_desc (netdev, errp); + +return vmnet_if_create (nc, if_desc, errp,NULL ); } +net/vmnet_int.h +10
-0@@ -16,10 +16,33 @@#include <vmnet/vmnet.h> +#define VMNET_PACKETS_LIMIT 50 + typedef struct VmnetCommonState { NetClientState nc; + interface_ref vmnet_if; + + bool send_enabled; + + uint64_t mtu; + uint64_t max_packet_size; + + struct vmpktdesc packets_buf[VMNET_PACKETS_LIMIT];net/vmnet_int.h line 40
Quoted 10 lines
net/vmnet_int.h +7
-0+struct iovec iov_buf[VMNET_PACKETS_LIMIT ]; } VmnetCommonState; +const char *vmnet_status_map_str (vmnet_return_t status); + +int vmnet_if_create(NetClientState *nc, + xpc_object_t if_desc, + Error **errp, + void (*completion_callback)(xpc_object_t
interface_param));Quoted 11 lines
net/vmnet_int.h +6
-1+ +ssize_t vmnet_receive_common (NetClientState *nc, +const uint8_t *buf, +size_t size); + +void vmnet_cleanup_common (NetClientState *nc);#endif /* VMNET_INT_H */ --2.23.0
Signature · 3 lines
Best Regards, Vladislav Yaroshchuk
Re: [PATCH v5 3/6] net/vmnet: implement shared mode (vmnet-shared)
Quoted 12 lines · Vladislav Yaroshchuk
пн, 15 нояб. 2021 г. в 07:47, Jason Wang <[email protected]>:
Quoted 8 lines
Quoted 4 lines · Vladislav Yaroshchuk
Signed-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>---
Quoted 6 lines
Quoted 3 lines
Commit log please.
Sorry, I don't understand what you mean here.
What is the 'commit log'?
I meant the change log to describe the changes.
Quoted 44 lines
Quoted 26 lines
Quoted 26 lines
net/vmnet-common.m | 305 +++++++++++++++++++++++++++++++++++++++++++++ net/vmnet-shared.c | 75 ++++++++++- net/vmnet_int.h | 23 ++++ 3 files changed, 399 insertions(+), 4 deletions(-)net/vmnet-common.m +3
-0@@ -10,6 +10,8 @@ */ #include "qemu/osdep.h" +#include "qemu/main-loop.h" +#include "qemu/log.h" #include "qapi/qapi-types-net.h" #include "vmnet_int.h" #include "clients.h"@@ -17,4 +19,307 @@ #include "qapi/error.h" #include <vmnet/vmnet.h> +#include <dispatch/dispatch.h>Quoted 17 lines
Quoted 4 lines
net/vmnet-common.m +4
-0+#ifdef DEBUG +#define D(x) x +#define D_LOG(...) qemu_log(__VA_ARGS__) +#elsenet/vmnet-common.m line 37
Quoted 10 lines
net/vmnet-common.m +10
-0+#define D(x) do { } while (0) +#define D_LOG(...) do { } while (0) +#endif + +typedef struct vmpktdesc vmpktdesc_t; +typedef struct iovec iovec_t; + +static void vmnet_set_send_enabled(VmnetCommonState *s, bool enable) +{ + s->send_enabled = enable;Is there a way to disable the event when enable is false?
Quoted 4 lines
It seems there's no way except setting/unsetting the callback via
vmnet_interface_set_event_callback. I decided to drop packages usings->send_enabledwithout dealing with the callback.
ok.
Quoted 260 lines
Quoted 239 lines
Quoted 239 lines
net/vmnet-common.m +238
-0+} + + +static void vmnet_send_completed(NetClientState *nc, ssize_t len) +{ + VmnetCommonState *s = DO_UPCAST(VmnetCommonState, nc, nc); + vmnet_set_send_enabled(s, true); +} + + +static void vmnet_send(NetClientState *nc, + interface_event_t event_id, + xpc_object_t event) +{ + assert(event_id == VMNET_INTERFACE_PACKETS_AVAILABLE); + + VmnetCommonState *s; + uint64_t packets_available; + + struct iovec *iov; + struct vmpktdesc *packets; + int pkt_cnt; + int i; + + vmnet_return_t if_status; + ssize_t size; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + packets_available = xpc_dictionary_get_uint64( + event, + vmnet_estimated_packets_available_key + ); + + pkt_cnt = (packets_available < VMNET_PACKETS_LIMIT) ? + packets_available : + VMNET_PACKETS_LIMIT; + + + iov = s->iov_buf; + packets = s->packets_buf; + + for (i = 0; i < pkt_cnt; ++i) { + packets[i].vm_pkt_size = s->max_packet_size; + packets[i].vm_pkt_iovcnt = 1; + packets[i].vm_flags = 0; + } + + if_status = vmnet_read(s->vmnet_if, packets, &pkt_cnt); + if (if_status != VMNET_SUCCESS) { + error_printf("vmnet: read failed: %s\n", + vmnet_status_map_str(if_status)); + } + qemu_mutex_lock_iothread(); + for (i = 0; i < pkt_cnt; ++i) { + size = qemu_send_packet_async(nc, + iov[i].iov_base, + packets[i].vm_pkt_size, + vmnet_send_completed); + if (size == 0) { + vmnet_set_send_enabled(s, false); + } else if (size < 0) { + break; + } + } + qemu_mutex_unlock_iothread(); + +} + + +static void vmnet_register_event_callback(VmnetCommonState *s) +{ + dispatch_queue_t avail_pkt_q = dispatch_queue_create( + "org.qemu.vmnet.if_queue", + DISPATCH_QUEUE_SERIAL + ); + + vmnet_interface_set_event_callback( + s->vmnet_if, + VMNET_INTERFACE_PACKETS_AVAILABLE, + avail_pkt_q, + ^(interface_event_t event_id, xpc_object_t event) { + if (s->send_enabled) { + vmnet_send(&s->nc, event_id, event); + } + }); +} + + +static void vmnet_bufs_init(VmnetCommonState *s) +{ + int i; + struct vmpktdesc *packets; + struct iovec *iov; + + packets = s->packets_buf; + iov = s->iov_buf; + + for (i = 0; i < VMNET_PACKETS_LIMIT; ++i) { + iov[i].iov_len = s->max_packet_size; + iov[i].iov_base = g_malloc0(iov[i].iov_len); + packets[i].vm_pkt_iov = iov + i; + } +} + + +const char *vmnet_status_map_str(vmnet_return_t status) +{ + switch (status) { + case VMNET_SUCCESS: + return "success"; + case VMNET_FAILURE: + return "general failure"; + case VMNET_MEM_FAILURE: + return "memory allocation failure"; + case VMNET_INVALID_ARGUMENT: + return "invalid argument specified"; + case VMNET_SETUP_INCOMPLETE: + return "interface setup is not complete"; + case VMNET_INVALID_ACCESS: + return "invalid access, permission denied"; + case VMNET_PACKET_TOO_BIG: + return "packet size is larger than MTU"; + case VMNET_BUFFER_EXHAUSTED: + return "buffers exhausted in kernel"; + case VMNET_TOO_MANY_PACKETS: + return "packet count exceeds limit"; + case VMNET_SHARING_SERVICE_BUSY: + return "conflict, sharing service is in use"; + default: + return "unknown vmnet error"; + } +} + + +int vmnet_if_create(NetClientState *nc, + xpc_object_t if_desc, + Error **errp, + void (*completion_callback)(xpc_object_t interface_param)) +{ + VmnetCommonState *s; + + dispatch_queue_t if_create_q; + dispatch_semaphore_t if_created_sem; + + __block vmnet_return_t if_status; + + if_create_q = dispatch_queue_create("org.qemu.vmnet.create", + DISPATCH_QUEUE_SERIAL); + if_created_sem = dispatch_semaphore_create(0); + + xpc_dictionary_set_bool( + if_desc, + vmnet_allocate_mac_address_key, + false + ); + + D(D_LOG("vmnet.start.interface_desc:\n"); + xpc_dictionary_apply(if_desc, + ^bool(const char *k, xpc_object_t v) { + char *desc = xpc_copy_description(v); + D_LOG(" %s=%s\n", k, desc); + free(desc); + return true; + })); + + s = DO_UPCAST(VmnetCommonState, nc, nc); + s->vmnet_if = vmnet_start_interface( + if_desc, + if_create_q, + ^(vmnet_return_t status, xpc_object_t interface_param) { + if_status = status; + if (status != VMNET_SUCCESS || !interface_param) { + dispatch_semaphore_signal(if_created_sem); + return; + } + + s->mtu = xpc_dictionary_get_uint64( + interface_param, + vmnet_mtu_key); + s->max_packet_size = xpc_dictionary_get_uint64( + interface_param, + vmnet_max_packet_size_key); + D(D_LOG("vmnet.start.interface_param:\n"); + xpc_dictionary_apply(interface_param, + ^bool(const char *k, xpc_object_t v) { + char *desc = xpc_copy_description(v); + D_LOG(" %s=%s\n", k, desc); + free(desc); + return true; + })); + dispatch_semaphore_signal(if_created_sem); + }); + + if (s->vmnet_if == NULL) { + error_setg(errp, "unable to create interface with requested params"); + return -1; + } + + dispatch_semaphore_wait(if_created_sem, DISPATCH_TIME_FOREVER); + dispatch_release(if_create_q); + + if (if_status != VMNET_SUCCESS) { + error_setg(errp, + "cannot create vmnet interface: %s", + vmnet_status_map_str(if_status)); + return -1; + } + + vmnet_register_event_callback(s); + vmnet_bufs_init(s); + vmnet_set_send_enabled(s, true); + + return 0; +} + + +ssize_t vmnet_receive_common(NetClientState *nc, + const uint8_t *buf, + size_t size) +{ + VmnetCommonState *s; + vmpktdesc_t packet; + iovec_t iov; + int pkt_cnt; + vmnet_return_t if_status; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + if (size > s->max_packet_size) { + warn_report("vmnet: packet is too big, %zu > %llu\n", + packet.vm_pkt_size, + s->max_packet_size); + return -1; + } + + iov.iov_base = (char *) buf; + iov.iov_len = size;Quoted 9 lines
net/vmnet-common.m line 280
Quoted 5 lines
net/vmnet-common.m +5
-0+ + packet.vm_pkt_iovcnt = 1; + packet.vm_flags = 0; + packet.vm_pkt_size = size; + packet.vm_pkt_iov = &iov;It looks to me vmnet framework supports iov so I wonder if a .receive_iov() is better because of its performance.
I've just tried to implement this call, and because of some reason
vmnet_writefails withVMNET_INVALID_ARGUMENTwhen iovcnt > 1. Tested withvmxnet3. Collecting all the iovs into a big one and passing it to vmnet works fine (the default behaviour when only the .receive() but not the .receive_iov() is implemented).This should be investigated, but currently I don't understand what exactly causes this error. The fact that vmnet.framework is a 'blackbox' makes the situation much worse.
Quoted 6 lines
Phillip's version is also broken:
https://patchew.org/QEMU/[email protected]/
Looks like this wasn't noticed before.If it's applicable, we can use the .receive() only, and put .receive_iov() implementation to the backlog.
Ok, we can go with receive() first.
Thanks
Quoted 185 lines
Quoted 175 lines
Quoted 175 lines
net/vmnet-common.m +45
-0+ + pkt_cnt = 1; + if_status = vmnet_write(s->vmnet_if, &packet, &pkt_cnt); + + if (if_status != VMNET_SUCCESS) { + error_report("vmnet: write error: %s\n", + vmnet_status_map_str(if_status)); + } + + if (if_status == VMNET_SUCCESS && pkt_cnt) { + return size; + } + return 0; +} + + +void vmnet_cleanup_common(NetClientState *nc) +{ + VmnetCommonState *s; + dispatch_queue_t if_destroy_q; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + qemu_purge_queued_packets(nc); + vmnet_set_send_enabled(s, false); + + if (s->vmnet_if == NULL) { + return; + } + + if_destroy_q = dispatch_queue_create( + "org.qemu.vmnet.destroy", + DISPATCH_QUEUE_SERIAL + ); + + vmnet_stop_interface( + s->vmnet_if, + if_destroy_q, + ^(vmnet_return_t status) { + }); + + for (int i = 0; i < VMNET_PACKETS_LIMIT; ++i) { + g_free(s->iov_buf[i].iov_base); + } +}net/vmnet-shared.c +71
-4@@ -10,16 +10,83 @@#include "qemu/osdep.h" #include "qapi/qapi-types-net.h" +#include "qapi/error.h" #include "vmnet_int.h" #include "clients.h" -#include "qemu/error-report.h" -#include "qapi/error.h" #include <vmnet/vmnet.h> +typedef struct VmnetSharedState { +VmnetCommonState cs; +}VmnetSharedState ; + + +static xpc_object_t create_if_desc (const Netdev *netdev,Error **errp) +{ +const NetdevVmnetSharedOptions *options = &(netdev->u.vmnet_shared); +xpc_object_t if_desc =xpc_dictionary_create (NULL ,NULL ,0 ); + +xpc_dictionary_set_uint64 ( + if_desc, + vmnet_operation_mode_key, +VMNET_SHARED_MODE + ); + +xpc_dictionary_set_bool ( + if_desc, + vmnet_enable_isolation_key, + options->isolated + ); + +if (options->has_nat66_prefix) { +xpc_dictionary_set_string (if_desc, + vmnet_nat66_prefix_key, + options->nat66_prefix); + } + +if (options->has_dhcpstart || + options->has_dhcpend || + options->has_subnet_mask) { + +if (options->has_dhcpstart && + options->has_dhcpend && + options->has_subnet_mask) { + +xpc_dictionary_set_string (if_desc, + vmnet_start_address_key, + options->dhcpstart); +xpc_dictionary_set_string (if_desc, + vmnet_end_address_key, + options->dhcpend); +xpc_dictionary_set_string (if_desc, + vmnet_subnet_mask_key, + options->subnet_mask); + }else { +error_setg ( + errp, +"'dhcpstart', 'dhcpend', 'subnet_mask' " +"should be provided together" + ); + } + } + +return if_desc; +} + +static NetClientInfo net_vmnet_shared_info = { + .type =NET_CLIENT_DRIVER_VMNET_SHARED , + .size =sizeof (VmnetSharedState), + .receive = vmnet_receive_common, + .cleanup = vmnet_cleanup_common, +}; +int net_init_vmnet_shared (const Netdev *netdev,const char *name,NetClientState *peer,Error **errp) { -error_setg (errp,"vmnet-shared is not implemented yet" ); -return -1 ; +NetClientState *nc =qemu_new_net_client (&net_vmnet_shared_info, + peer,"vmnet-shared" , name); +xpc_object_t if_desc =create_if_desc (netdev, errp); + +return vmnet_if_create (nc, if_desc, errp,NULL ); } +net/vmnet_int.h +23
-0@@ -16,10 +16,33 @@#include <vmnet/vmnet.h> +#define VMNET_PACKETS_LIMIT 50 +typedef struct VmnetCommonState {NetClientState nc; +interface_ref vmnet_if; + +bool send_enabled; + +uint64_t mtu; +uint64_t max_packet_size; + +struct vmpktdesc packets_buf[VMNET_PACKETS_LIMIT ]; +struct iovec iov_buf[VMNET_PACKETS_LIMIT ]; }VmnetCommonState ; +const char *vmnet_status_map_str (vmnet_return_t status); + +int vmnet_if_create (NetClientState *nc, +xpc_object_t if_desc, +Error **errp, +void (*completion_callback)(xpc_object_t interface_param)); + +ssize_t vmnet_receive_common (NetClientState *nc, +const uint8_t *buf, +size_t size); + +void vmnet_cleanup_common (NetClientState *nc);Quoted 4 lines
Quoted 4 lines
net/vmnet_int.h +0
-1#endif /* VMNET_INT_H */ --2.23.0
--
Best Regards,Vladislav Yaroshchuk
Re: [PATCH v5 3/6] net/vmnet: implement shared mode (vmnet-shared)
пт, 19 нояб. 2021 г. в 05:57, Jason Wang <[email protected]>:
Quoted 14 lines
Quoted 12 lines · Vladislav Yaroshchuk
пн, 15 нояб. 2021 г. в 07:47, Jason Wang <[email protected]>:
Quoted 8 lines
Quoted 4 lines · Vladislav Yaroshchuk
Signed-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>---
Quoted 10 lines
Quoted 6 lines
Quoted 3 lines
Commit log please.
Sorry, I don't understand what you mean here.
What is the 'commit log'?I meant the change log to describe the changes.
You mean more detailed commit message?
If not, can you please provide a short example
Quoted 517 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
net/vmnet-common.m | 305+++++++++++++++++++++++++++++++++++++++++++++
Quoted 41 lines
Quoted 23 lines
Quoted 23 lines
net/vmnet-shared.c | 75 ++++++++++- net/vmnet_int.h | 23 ++++ 3 files changed, 399 insertions(+), 4 deletions(-)net/vmnet-common.m +3
-0@@ -10,6 +10,8 @@ */ #include "qemu/osdep.h" +#include "qemu/main-loop.h" +#include "qemu/log.h" #include "qapi/qapi-types-net.h" #include "vmnet_int.h" #include "clients.h"@@ -17,4 +19,307 @@ #include "qapi/error.h" #include <vmnet/vmnet.h> +#include <dispatch/dispatch.h>Quoted 17 lines
Quoted 4 lines
net/vmnet-common.m +4
-0+#ifdef DEBUG +#define D(x) x +#define D_LOG(...) qemu_log(__VA_ARGS__) +#elsenet/vmnet-common.m line 37
Quoted 10 lines
net/vmnet-common.m +10
-0+#define D(x) do { } while (0) +#define D_LOG(...) do { } while (0) +#endif + +typedef struct vmpktdesc vmpktdesc_t; +typedef struct iovec iovec_t; + +static void vmnet_set_send_enabled(VmnetCommonState *s, bool enable) +{ + s->send_enabled = enable;Is there a way to disable the event when enable is false?
Quoted 4 lines
It seems there's no way except setting/unsetting the callback via
vmnet_interface_set_event_callback. I decided to drop packages usings->send_enabledwithout dealing with the callback.ok.
Quoted 132 lines
Quoted 132 lines
Quoted 132 lines
net/vmnet-common.m +131
-0+} + + +static void vmnet_send_completed(NetClientState *nc, ssize_t len) +{ + VmnetCommonState *s = DO_UPCAST(VmnetCommonState, nc, nc); + vmnet_set_send_enabled(s, true); +} + + +static void vmnet_send(NetClientState *nc, + interface_event_t event_id, + xpc_object_t event) +{ + assert(event_id == VMNET_INTERFACE_PACKETS_AVAILABLE); + + VmnetCommonState *s; + uint64_t packets_available; + + struct iovec *iov; + struct vmpktdesc *packets; + int pkt_cnt; + int i; + + vmnet_return_t if_status; + ssize_t size; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + packets_available = xpc_dictionary_get_uint64( + event, + vmnet_estimated_packets_available_key + ); + + pkt_cnt = (packets_available < VMNET_PACKETS_LIMIT) ? + packets_available : + VMNET_PACKETS_LIMIT; + + + iov = s->iov_buf; + packets = s->packets_buf; + + for (i = 0; i < pkt_cnt; ++i) { + packets[i].vm_pkt_size = s->max_packet_size; + packets[i].vm_pkt_iovcnt = 1; + packets[i].vm_flags = 0; + } + + if_status = vmnet_read(s->vmnet_if, packets, &pkt_cnt); + if (if_status != VMNET_SUCCESS) { + error_printf("vmnet: read failed: %s\n", + vmnet_status_map_str(if_status)); + } + qemu_mutex_lock_iothread(); + for (i = 0; i < pkt_cnt; ++i) { + size = qemu_send_packet_async(nc, + iov[i].iov_base, + packets[i].vm_pkt_size, + vmnet_send_completed); + if (size == 0) { + vmnet_set_send_enabled(s, false); + } else if (size < 0) { + break; + } + } + qemu_mutex_unlock_iothread(); + +} + + +static void vmnet_register_event_callback(VmnetCommonState *s) +{ + dispatch_queue_t avail_pkt_q = dispatch_queue_create( + "org.qemu.vmnet.if_queue", + DISPATCH_QUEUE_SERIAL + ); + + vmnet_interface_set_event_callback( + s->vmnet_if, + VMNET_INTERFACE_PACKETS_AVAILABLE, + avail_pkt_q, + ^(interface_event_t event_id, xpc_object_t event) { + if (s->send_enabled) { + vmnet_send(&s->nc, event_id, event); + } + }); +} + + +static void vmnet_bufs_init(VmnetCommonState *s) +{ + int i; + struct vmpktdesc *packets; + struct iovec *iov; + + packets = s->packets_buf; + iov = s->iov_buf; + + for (i = 0; i < VMNET_PACKETS_LIMIT; ++i) { + iov[i].iov_len = s->max_packet_size; + iov[i].iov_base = g_malloc0(iov[i].iov_len); + packets[i].vm_pkt_iov = iov + i; + } +} + + +const char *vmnet_status_map_str(vmnet_return_t status) +{ + switch (status) { + case VMNET_SUCCESS: + return "success"; + case VMNET_FAILURE: + return "general failure"; + case VMNET_MEM_FAILURE: + return "memory allocation failure"; + case VMNET_INVALID_ARGUMENT: + return "invalid argument specified"; + case VMNET_SETUP_INCOMPLETE: + return "interface setup is not complete"; + case VMNET_INVALID_ACCESS: + return "invalid access, permission denied"; + case VMNET_PACKET_TOO_BIG: + return "packet size is larger than MTU"; + case VMNET_BUFFER_EXHAUSTED: + return "buffers exhausted in kernel"; + case VMNET_TOO_MANY_PACKETS: + return "packet count exceeds limit"; + case VMNET_SHARING_SERVICE_BUSY: + return "conflict, sharing service is in use"; + default: + return "unknown vmnet error";Quoted 8 lines
Quoted 8 lines
net/vmnet-common.m
Quoted 8 lines
net/vmnet-common.m +8
-0+ } +} + + +int vmnet_if_create(NetClientState *nc, + xpc_object_t if_desc, + Error **errp, + void (*completion_callback)(xpc_object_t
interface_param))Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
net/vmnet-common.m +40
-0+{ + VmnetCommonState *s; + + dispatch_queue_t if_create_q; + dispatch_semaphore_t if_created_sem; + + __block vmnet_return_t if_status; + + if_create_q = dispatch_queue_create("org.qemu.vmnet.create", + DISPATCH_QUEUE_SERIAL); + if_created_sem = dispatch_semaphore_create(0); + + xpc_dictionary_set_bool( + if_desc, + vmnet_allocate_mac_address_key, + false + ); + + D(D_LOG("vmnet.start.interface_desc:\n"); + xpc_dictionary_apply(if_desc, + ^bool(const char *k, xpc_object_t v) { + char *desc = xpc_copy_description(v); + D_LOG(" %s=%s\n", k, desc); + free(desc); + return true; + })); + + s = DO_UPCAST(VmnetCommonState, nc, nc); + s->vmnet_if = vmnet_start_interface( + if_desc, + if_create_q, + ^(vmnet_return_t status, xpc_object_t interface_param) { + if_status = status; + if (status != VMNET_SUCCESS || !interface_param) { + dispatch_semaphore_signal(if_created_sem); + return; + } + + s->mtu = xpc_dictionary_get_uint64( + interface_param,Quoted 8 lines
Quoted 8 lines
net/vmnet-common.m
Quoted 8 lines
net/vmnet-common.m +8
-0+ vmnet_mtu_key); + s->max_packet_size = xpc_dictionary_get_uint64( + interface_param, + vmnet_max_packet_size_key); + D(D_LOG("vmnet.start.interface_param:\n"); + xpc_dictionary_apply(interface_param, + ^bool(const char *k, xpc_object_t v) { + char *desc =
xpc_copy_description(v);Quoted 9 lines
Quoted 9 lines
net/vmnet-common.m
Quoted 9 lines
net/vmnet-common.m +9
-0+ D_LOG(" %s=%s\n", k, desc); + free(desc); + return true; + })); + dispatch_semaphore_signal(if_created_sem); + }); + + if (s->vmnet_if == NULL) { + error_setg(errp, "unable to create interface with requestedparams");
Quoted 63 lines
Quoted 42 lines
Quoted 42 lines
net/vmnet-common.m +42
-0+ return -1; + } + + dispatch_semaphore_wait(if_created_sem, DISPATCH_TIME_FOREVER); + dispatch_release(if_create_q); + + if (if_status != VMNET_SUCCESS) { + error_setg(errp, + "cannot create vmnet interface: %s", + vmnet_status_map_str(if_status)); + return -1; + } + + vmnet_register_event_callback(s); + vmnet_bufs_init(s); + vmnet_set_send_enabled(s, true); + + return 0; +} + + +ssize_t vmnet_receive_common(NetClientState *nc, + const uint8_t *buf, + size_t size) +{ + VmnetCommonState *s; + vmpktdesc_t packet; + iovec_t iov; + int pkt_cnt; + vmnet_return_t if_status; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + if (size > s->max_packet_size) { + warn_report("vmnet: packet is too big, %zu > %llu\n", + packet.vm_pkt_size, + s->max_packet_size); + return -1; + } + + iov.iov_base = (char *) buf; + iov.iov_len = size;Quoted 9 lines
net/vmnet-common.m line 280
Quoted 5 lines
net/vmnet-common.m +5
-0+ + packet.vm_pkt_iovcnt = 1; + packet.vm_flags = 0; + packet.vm_pkt_size = size; + packet.vm_pkt_iov = &iov;It looks to me vmnet framework supports iov so I wonder if a .receive_iov() is better because of its performance.
I've just tried to implement this call, and because of some reason
vmnet_writefails withVMNET_INVALID_ARGUMENTwhen iovcnt > 1. Tested withvmxnet3. Collecting all the iovs into a big one and passing it to vmnet works fine (the default behaviour when only the .receive() but not the .receive_iov() is implemented).This should be investigated, but currently I don't understand what exactly causes this error. The fact that vmnet.framework is a 'blackbox' makes the situation much worse.
Quoted 6 lines
Phillip's version is also broken:
https://patchew.org/QEMU/[email protected]/
Looks like this wasn't noticed before.If it's applicable, we can use the .receive() only, and put .receive_iov() implementation to the backlog.
Ok, we can go with
receive()first.Thanks
Quoted 60 lines
Quoted 60 lines
Quoted 60 lines
net/vmnet-common.m +45
-0+ + pkt_cnt = 1; + if_status = vmnet_write(s->vmnet_if, &packet, &pkt_cnt); + + if (if_status != VMNET_SUCCESS) { + error_report("vmnet: write error: %s\n", + vmnet_status_map_str(if_status)); + } + + if (if_status == VMNET_SUCCESS && pkt_cnt) { + return size; + } + return 0; +} + + +void vmnet_cleanup_common(NetClientState *nc) +{ + VmnetCommonState *s; + dispatch_queue_t if_destroy_q; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + qemu_purge_queued_packets(nc); + vmnet_set_send_enabled(s, false); + + if (s->vmnet_if == NULL) { + return; + } + + if_destroy_q = dispatch_queue_create( + "org.qemu.vmnet.destroy", + DISPATCH_QUEUE_SERIAL + ); + + vmnet_stop_interface( + s->vmnet_if, + if_destroy_q, + ^(vmnet_return_t status) { + }); + + for (int i = 0; i < VMNET_PACKETS_LIMIT; ++i) { + g_free(s->iov_buf[i].iov_base); + } +}net/vmnet-shared.c +1
-2@@ -10,16 +10,83 @@#include "qemu/osdep.h" #include "qapi/qapi-types-net.h" +#include "qapi/error.h" #include "vmnet_int.h" #include "clients.h" -#include "qemu/error-report.h" -#include "qapi/error.h" Quoted 8 lines
Quoted 8 lines
net/vmnet-shared.c
Quoted 8 lines
net/vmnet-shared.c +6
-0#include <vmnet/vmnet.h> +typedef struct VmnetSharedState { +VmnetCommonState cs; +}VmnetSharedState ; + + +static xpc_object_t create_if_desc(const Netdev *netdev, Error**errp)
Quoted 2 lines
Quoted 2 lines
net/vmnet-shared.c line 26
Quoted 2 lines
net/vmnet-shared.c +2
-0+{ + const NetdevVmnetSharedOptions *options =&(netdev->u.vmnet_shared);
Quoted 55 lines
Quoted 55 lines
Quoted 55 lines
net/vmnet-shared.c +55
-0+xpc_object_t if_desc =xpc_dictionary_create (NULL ,NULL ,0 ); + +xpc_dictionary_set_uint64 ( + if_desc, + vmnet_operation_mode_key, +VMNET_SHARED_MODE + ); + +xpc_dictionary_set_bool ( + if_desc, + vmnet_enable_isolation_key, + options->isolated + ); + +if (options->has_nat66_prefix) { +xpc_dictionary_set_string (if_desc, + vmnet_nat66_prefix_key, + options->nat66_prefix); + } + +if (options->has_dhcpstart || + options->has_dhcpend || + options->has_subnet_mask) { + +if (options->has_dhcpstart && + options->has_dhcpend && + options->has_subnet_mask) { + +xpc_dictionary_set_string (if_desc, + vmnet_start_address_key, + options->dhcpstart); +xpc_dictionary_set_string (if_desc, + vmnet_end_address_key, + options->dhcpend); +xpc_dictionary_set_string (if_desc, + vmnet_subnet_mask_key, + options->subnet_mask); + }else { +error_setg ( + errp, +"'dhcpstart', 'dhcpend', 'subnet_mask' " +"should be provided together" + ); + } + } + +return if_desc; +} + +static NetClientInfo net_vmnet_shared_info = { + .type =NET_CLIENT_DRIVER_VMNET_SHARED , + .size =sizeof (VmnetSharedState), + .receive = vmnet_receive_common, + .cleanup = vmnet_cleanup_common, +};Quoted 8 lines
Quoted 8 lines
net/vmnet-shared.c
Quoted 8 lines
net/vmnet-shared.c +3
-2+int net_init_vmnet_shared (const Netdev *netdev,const char *name,NetClientState *peer,Error **errp) { -error_setg (errp,"vmnet-shared is not implemented yet" ); -return -1 ; + NetClientState *nc = qemu_new_net_client(&net_vmnet_shared_info, + peer, "vmnet-shared",name);
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
net/vmnet-shared.c +4
-0+xpc_object_t if_desc =create_if_desc (netdev, errp); + +return vmnet_if_create (nc, if_desc, errp,NULL ); } +net/vmnet_int.h +11
-0@@ -16,10 +16,33 @@#include <vmnet/vmnet.h> +#define VMNET_PACKETS_LIMIT 50 + typedef struct VmnetCommonState { NetClientState nc; + interface_ref vmnet_if; + + bool send_enabled; + + uint64_t mtu; + uint64_t max_packet_size; + + struct vmpktdesc packets_buf[VMNET_PACKETS_LIMIT]; + struct iovec iov_buf[VMNET_PACKETS_LIMIT];Quoted 8 lines
Quoted 8 lines
net/vmnet_int.h
Quoted 8 lines
net/vmnet_int.h +6
-0} VmnetCommonState; +const char *vmnet_status_map_str (vmnet_return_t status); + +int vmnet_if_create(NetClientState *nc, + xpc_object_t if_desc, + Error **errp, + void (*completion_callback)(xpc_object_t
interface_param));Quoted 17 lines
Quoted 11 lines
Quoted 11 lines
net/vmnet_int.h +6
-1+ +ssize_t vmnet_receive_common (NetClientState *nc, +const uint8_t *buf, +size_t size); + +void vmnet_cleanup_common (NetClientState *nc);#endif /* VMNET_INT_H */ --2.23.0
--
Best Regards,Vladislav Yaroshchuk
Signature · 3 lines
Best Regards, Vladislav Yaroshchuk
Re: [PATCH v5 3/6] net/vmnet: implement shared mode (vmnet-shared)
Quoted 23 lines · Vladislav Yaroshchuk
пт, 19 нояб. 2021 г. в 05:57, Jason Wang <[email protected]>:
Quoted 19 lines
Quoted 16 lines · Vladislav Yaroshchuk
пн, 15 нояб. 2021 г. в 07:47, Jason Wang <[email protected]>:
Quoted 11 lines
Quoted 4 lines · Vladislav Yaroshchuk
Signed-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>---
Commit log please.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Sorry, I don't understand what you mean here.
What is the 'commit log'?I meant the change log to describe the changes.
You mean more detailed commit message?
(replied twice, missing the list for the first time)
Yes, with a patch more than 300+ lines of changes, we need a good change log.
Thanks
Quoted 516 lines
If not, can you please provide a short example
Quoted 315 lines
Quoted 44 lines
Quoted 26 lines
Quoted 26 lines
net/vmnet-common.m | 305 +++++++++++++++++++++++++++++++++++++++++++++ net/vmnet-shared.c | 75 ++++++++++- net/vmnet_int.h | 23 ++++ 3 files changed, 399 insertions(+), 4 deletions(-)net/vmnet-common.m +3
-0@@ -10,6 +10,8 @@ */ #include "qemu/osdep.h" +#include "qemu/main-loop.h" +#include "qemu/log.h" #include "qapi/qapi-types-net.h" #include "vmnet_int.h" #include "clients.h"@@ -17,4 +19,307 @@ #include "qapi/error.h" #include <vmnet/vmnet.h> +#include <dispatch/dispatch.h>Quoted 17 lines
Quoted 4 lines
net/vmnet-common.m +4
-0+#ifdef DEBUG +#define D(x) x +#define D_LOG(...) qemu_log(__VA_ARGS__) +#elsenet/vmnet-common.m line 37
Quoted 10 lines
net/vmnet-common.m +10
-0+#define D(x) do { } while (0) +#define D_LOG(...) do { } while (0) +#endif + +typedef struct vmpktdesc vmpktdesc_t; +typedef struct iovec iovec_t; + +static void vmnet_set_send_enabled(VmnetCommonState *s, bool enable) +{ + s->send_enabled = enable;Is there a way to disable the event when enable is false?
Quoted 4 lines
It seems there's no way except setting/unsetting the callback via
vmnet_interface_set_event_callback. I decided to drop packages usings->send_enabledwithout dealing with the callback.ok.
Quoted 264 lines
Quoted 239 lines
Quoted 239 lines
net/vmnet-common.m +238
-0+} + + +static void vmnet_send_completed(NetClientState *nc, ssize_t len) +{ + VmnetCommonState *s = DO_UPCAST(VmnetCommonState, nc, nc); + vmnet_set_send_enabled(s, true); +} + + +static void vmnet_send(NetClientState *nc, + interface_event_t event_id, + xpc_object_t event) +{ + assert(event_id == VMNET_INTERFACE_PACKETS_AVAILABLE); + + VmnetCommonState *s; + uint64_t packets_available; + + struct iovec *iov; + struct vmpktdesc *packets; + int pkt_cnt; + int i; + + vmnet_return_t if_status; + ssize_t size; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + packets_available = xpc_dictionary_get_uint64( + event, + vmnet_estimated_packets_available_key + ); + + pkt_cnt = (packets_available < VMNET_PACKETS_LIMIT) ? + packets_available : + VMNET_PACKETS_LIMIT; + + + iov = s->iov_buf; + packets = s->packets_buf; + + for (i = 0; i < pkt_cnt; ++i) { + packets[i].vm_pkt_size = s->max_packet_size; + packets[i].vm_pkt_iovcnt = 1; + packets[i].vm_flags = 0; + } + + if_status = vmnet_read(s->vmnet_if, packets, &pkt_cnt); + if (if_status != VMNET_SUCCESS) { + error_printf("vmnet: read failed: %s\n", + vmnet_status_map_str(if_status)); + } + qemu_mutex_lock_iothread(); + for (i = 0; i < pkt_cnt; ++i) { + size = qemu_send_packet_async(nc, + iov[i].iov_base, + packets[i].vm_pkt_size, + vmnet_send_completed); + if (size == 0) { + vmnet_set_send_enabled(s, false); + } else if (size < 0) { + break; + } + } + qemu_mutex_unlock_iothread(); + +} + + +static void vmnet_register_event_callback(VmnetCommonState *s) +{ + dispatch_queue_t avail_pkt_q = dispatch_queue_create( + "org.qemu.vmnet.if_queue", + DISPATCH_QUEUE_SERIAL + ); + + vmnet_interface_set_event_callback( + s->vmnet_if, + VMNET_INTERFACE_PACKETS_AVAILABLE, + avail_pkt_q, + ^(interface_event_t event_id, xpc_object_t event) { + if (s->send_enabled) { + vmnet_send(&s->nc, event_id, event); + } + }); +} + + +static void vmnet_bufs_init(VmnetCommonState *s) +{ + int i; + struct vmpktdesc *packets; + struct iovec *iov; + + packets = s->packets_buf; + iov = s->iov_buf; + + for (i = 0; i < VMNET_PACKETS_LIMIT; ++i) { + iov[i].iov_len = s->max_packet_size; + iov[i].iov_base = g_malloc0(iov[i].iov_len); + packets[i].vm_pkt_iov = iov + i; + } +} + + +const char *vmnet_status_map_str(vmnet_return_t status) +{ + switch (status) { + case VMNET_SUCCESS: + return "success"; + case VMNET_FAILURE: + return "general failure"; + case VMNET_MEM_FAILURE: + return "memory allocation failure"; + case VMNET_INVALID_ARGUMENT: + return "invalid argument specified"; + case VMNET_SETUP_INCOMPLETE: + return "interface setup is not complete"; + case VMNET_INVALID_ACCESS: + return "invalid access, permission denied"; + case VMNET_PACKET_TOO_BIG: + return "packet size is larger than MTU"; + case VMNET_BUFFER_EXHAUSTED: + return "buffers exhausted in kernel"; + case VMNET_TOO_MANY_PACKETS: + return "packet count exceeds limit"; + case VMNET_SHARING_SERVICE_BUSY: + return "conflict, sharing service is in use"; + default: + return "unknown vmnet error"; + } +} + + +int vmnet_if_create(NetClientState *nc, + xpc_object_t if_desc, + Error **errp, + void (*completion_callback)(xpc_object_t interface_param)) +{ + VmnetCommonState *s; + + dispatch_queue_t if_create_q; + dispatch_semaphore_t if_created_sem; + + __block vmnet_return_t if_status; + + if_create_q = dispatch_queue_create("org.qemu.vmnet.create", + DISPATCH_QUEUE_SERIAL); + if_created_sem = dispatch_semaphore_create(0); + + xpc_dictionary_set_bool( + if_desc, + vmnet_allocate_mac_address_key, + false + ); + + D(D_LOG("vmnet.start.interface_desc:\n"); + xpc_dictionary_apply(if_desc, + ^bool(const char *k, xpc_object_t v) { + char *desc = xpc_copy_description(v); + D_LOG(" %s=%s\n", k, desc); + free(desc); + return true; + })); + + s = DO_UPCAST(VmnetCommonState, nc, nc); + s->vmnet_if = vmnet_start_interface( + if_desc, + if_create_q, + ^(vmnet_return_t status, xpc_object_t interface_param) { + if_status = status; + if (status != VMNET_SUCCESS || !interface_param) { + dispatch_semaphore_signal(if_created_sem); + return; + } + + s->mtu = xpc_dictionary_get_uint64( + interface_param, + vmnet_mtu_key); + s->max_packet_size = xpc_dictionary_get_uint64( + interface_param, + vmnet_max_packet_size_key); + D(D_LOG("vmnet.start.interface_param:\n"); + xpc_dictionary_apply(interface_param, + ^bool(const char *k, xpc_object_t v) { + char *desc = xpc_copy_description(v); + D_LOG(" %s=%s\n", k, desc); + free(desc); + return true; + })); + dispatch_semaphore_signal(if_created_sem); + }); + + if (s->vmnet_if == NULL) { + error_setg(errp, "unable to create interface with requested params"); + return -1; + } + + dispatch_semaphore_wait(if_created_sem, DISPATCH_TIME_FOREVER); + dispatch_release(if_create_q); + + if (if_status != VMNET_SUCCESS) { + error_setg(errp, + "cannot create vmnet interface: %s", + vmnet_status_map_str(if_status)); + return -1; + } + + vmnet_register_event_callback(s); + vmnet_bufs_init(s); + vmnet_set_send_enabled(s, true); + + return 0; +} + + +ssize_t vmnet_receive_common(NetClientState *nc, + const uint8_t *buf, + size_t size) +{ + VmnetCommonState *s; + vmpktdesc_t packet; + iovec_t iov; + int pkt_cnt; + vmnet_return_t if_status; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + if (size > s->max_packet_size) { + warn_report("vmnet: packet is too big, %zu > %llu\n", + packet.vm_pkt_size, + s->max_packet_size); + return -1; + } + + iov.iov_base = (char *) buf; + iov.iov_len = size;Quoted 9 lines
net/vmnet-common.m line 280
Quoted 5 lines
net/vmnet-common.m +5
-0+ + packet.vm_pkt_iovcnt = 1; + packet.vm_flags = 0; + packet.vm_pkt_size = size; + packet.vm_pkt_iov = &iov;It looks to me vmnet framework supports iov so I wonder if a .receive_iov() is better because of its performance.
I've just tried to implement this call, and because of some reason
vmnet_writefails withVMNET_INVALID_ARGUMENTwhen iovcnt > 1. Tested withvmxnet3. Collecting all the iovs into a big one and passing it to vmnet works fine (the default behaviour when only the .receive() but not the .receive_iov() is implemented).This should be investigated, but currently I don't understand what exactly causes this error. The fact that vmnet.framework is a 'blackbox' makes the situation much worse.
Phillip's version is also broken:
https://patchew.org/QEMU/[email protected]/
Looks like this wasn't noticed before.Quoted 193 lines
Quoted 2 lines
If it's applicable, we can use the .receive() only, and put .receive_iov() implementation to the backlog.
Ok, we can go with
receive()first.Thanks
Quoted 185 lines
Quoted 175 lines
Quoted 175 lines
net/vmnet-common.m +45
-0+ + pkt_cnt = 1; + if_status = vmnet_write(s->vmnet_if, &packet, &pkt_cnt); + + if (if_status != VMNET_SUCCESS) { + error_report("vmnet: write error: %s\n", + vmnet_status_map_str(if_status)); + } + + if (if_status == VMNET_SUCCESS && pkt_cnt) { + return size; + } + return 0; +} + + +void vmnet_cleanup_common(NetClientState *nc) +{ + VmnetCommonState *s; + dispatch_queue_t if_destroy_q; + + s = DO_UPCAST(VmnetCommonState, nc, nc); + + qemu_purge_queued_packets(nc); + vmnet_set_send_enabled(s, false); + + if (s->vmnet_if == NULL) { + return; + } + + if_destroy_q = dispatch_queue_create( + "org.qemu.vmnet.destroy", + DISPATCH_QUEUE_SERIAL + ); + + vmnet_stop_interface( + s->vmnet_if, + if_destroy_q, + ^(vmnet_return_t status) { + }); + + for (int i = 0; i < VMNET_PACKETS_LIMIT; ++i) { + g_free(s->iov_buf[i].iov_base); + } +}net/vmnet-shared.c +71
-4@@ -10,16 +10,83 @@#include "qemu/osdep.h" #include "qapi/qapi-types-net.h" +#include "qapi/error.h" #include "vmnet_int.h" #include "clients.h" -#include "qemu/error-report.h" -#include "qapi/error.h" #include <vmnet/vmnet.h> +typedef struct VmnetSharedState { +VmnetCommonState cs; +}VmnetSharedState ; + + +static xpc_object_t create_if_desc (const Netdev *netdev,Error **errp) +{ +const NetdevVmnetSharedOptions *options = &(netdev->u.vmnet_shared); +xpc_object_t if_desc =xpc_dictionary_create (NULL ,NULL ,0 ); + +xpc_dictionary_set_uint64 ( + if_desc, + vmnet_operation_mode_key, +VMNET_SHARED_MODE + ); + +xpc_dictionary_set_bool ( + if_desc, + vmnet_enable_isolation_key, + options->isolated + ); + +if (options->has_nat66_prefix) { +xpc_dictionary_set_string (if_desc, + vmnet_nat66_prefix_key, + options->nat66_prefix); + } + +if (options->has_dhcpstart || + options->has_dhcpend || + options->has_subnet_mask) { + +if (options->has_dhcpstart && + options->has_dhcpend && + options->has_subnet_mask) { + +xpc_dictionary_set_string (if_desc, + vmnet_start_address_key, + options->dhcpstart); +xpc_dictionary_set_string (if_desc, + vmnet_end_address_key, + options->dhcpend); +xpc_dictionary_set_string (if_desc, + vmnet_subnet_mask_key, + options->subnet_mask); + }else { +error_setg ( + errp, +"'dhcpstart', 'dhcpend', 'subnet_mask' " +"should be provided together" + ); + } + } + +return if_desc; +} + +static NetClientInfo net_vmnet_shared_info = { + .type =NET_CLIENT_DRIVER_VMNET_SHARED , + .size =sizeof (VmnetSharedState), + .receive = vmnet_receive_common, + .cleanup = vmnet_cleanup_common, +}; +int net_init_vmnet_shared (const Netdev *netdev,const char *name,NetClientState *peer,Error **errp) { -error_setg (errp,"vmnet-shared is not implemented yet" ); -return -1 ; +NetClientState *nc =qemu_new_net_client (&net_vmnet_shared_info, + peer,"vmnet-shared" , name); +xpc_object_t if_desc =create_if_desc (netdev, errp); + +return vmnet_if_create (nc, if_desc, errp,NULL ); } +net/vmnet_int.h +23
-0@@ -16,10 +16,33 @@#include <vmnet/vmnet.h> +#define VMNET_PACKETS_LIMIT 50 +typedef struct VmnetCommonState {NetClientState nc; +interface_ref vmnet_if; + +bool send_enabled; + +uint64_t mtu; +uint64_t max_packet_size; + +struct vmpktdesc packets_buf[VMNET_PACKETS_LIMIT ]; +struct iovec iov_buf[VMNET_PACKETS_LIMIT ]; }VmnetCommonState ; +const char *vmnet_status_map_str (vmnet_return_t status); + +int vmnet_if_create (NetClientState *nc, +xpc_object_t if_desc, +Error **errp, +void (*completion_callback)(xpc_object_t interface_param)); + +ssize_t vmnet_receive_common (NetClientState *nc, +const uint8_t *buf, +size_t size); + +void vmnet_cleanup_common (NetClientState *nc);Quoted 4 lines
Quoted 4 lines
net/vmnet_int.h +0
-1#endif /* VMNET_INT_H */ --2.23.0
--
Best Regards,Vladislav Yaroshchuk
--
Best Regards,Vladislav Yaroshchuk
PATCH v5 4/6 net/vmnet: implement host mode (vmnet-host)
In reply to Vladislav Yaroshchuk--- net/vmnet-host.c | 99 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 93 insertions(+), 6 deletions(-)
net/vmnet-host.c +93 -6
@@ -9,16 +9,103 @@*/ #include "qemu/osdep.h" +#include "qemu/uuid.h" #include "qapi/qapi-types-net.h" -#include "vmnet_int.h" -#include "clients.h" -#include "qemu/error-report.h" #include "qapi/error.h" +#include "clients.h" +#include "vmnet_int.h" #include <vmnet/vmnet.h> +typedef struct VmnetHostState { +VmnetCommonState cs; +QemuUUID network_uuid; +}VmnetHostState ; + +static xpc_object_t create_if_desc (const Netdev *netdev, +NetClientState *nc, +Error **errp) +{ +const NetdevVmnetHostOptions *options = &(netdev->u.vmnet_host); +VmnetCommonState *cs =DO_UPCAST (VmnetCommonState, nc, nc); +VmnetHostState *hs =DO_UPCAST (VmnetHostState, cs, cs); + +xpc_object_t if_desc =xpc_dictionary_create (NULL ,NULL ,0 ); + +xpc_dictionary_set_uint64 ( + if_desc, + vmnet_operation_mode_key, +VMNET_HOST_MODE + ); + +xpc_dictionary_set_bool ( + if_desc, + vmnet_enable_isolation_key, + options->isolated + ); + +if (options->has_net_uuid) { +if (qemu_uuid_parse (options->net_uuid, &hs->network_uuid) <0 ) { +error_setg (errp,"Invalid UUID provided in 'net-uuid'" ); + } + +xpc_dictionary_set_uuid ( + if_desc, + vmnet_network_identifier_key, + hs->network_uuid.data + ); + } + +if (options->has_dhcpstart || + options->has_dhcpend || + options->has_subnet_mask) { + +if (options->has_dhcpstart && + options->has_dhcpend && + options->has_subnet_mask) { + +if (options->has_net_uuid) { +error_setg (errp, +"DHCP disabled for this interface " +"because 'net-uuid' is provided" ); + } + +xpc_dictionary_set_string (if_desc, + vmnet_start_address_key, + options->dhcpstart); +xpc_dictionary_set_string (if_desc, + vmnet_end_address_key, + options->dhcpend); +xpc_dictionary_set_string (if_desc, + vmnet_subnet_mask_key, + options->subnet_mask); + }else { +error_setg ( + errp, +"'dhcpstart', 'dhcpend', 'subnet_mask' " +"should be provided together" + ); + } + } + +return if_desc; +} + +static NetClientInfo net_vmnet_host_info = { + .type =NET_CLIENT_DRIVER_VMNET_HOST , + .size =sizeof (VmnetHostState), + .receive = vmnet_receive_common, + .cleanup = vmnet_cleanup_common, +}; +int net_init_vmnet_host (const Netdev *netdev,const char *name, -NetClientState *peer,Error **errp) { -error_setg (errp,"vmnet-host is not implemented yet" ); -return -1 ; +NetClientState *peer,Error **errp) +{ +NetClientState *nc; +xpc_object_t if_desc; + + nc =qemu_new_net_client (&net_vmnet_host_info, + peer,"vmnet-host" , name); + if_desc =create_if_desc (netdev, nc, errp); +return vmnet_if_create (nc, if_desc, errp,NULL ); }
Signature · 1 line
2.23.0
PATCH v5 5/6 net/vmnet: implement bridged mode (vmnet-bridged)
In reply to Vladislav Yaroshchuk--- net/vmnet-bridged.m | 98 ++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 92 insertions(+), 6 deletions(-)
net/vmnet-bridged.m +92 -6
@@ -10,16 +10,102 @@ #include "qemu/osdep.h" #include "qapi/qapi-types-net.h" -#include "vmnet_int.h" -#include "clients.h" -#include "qemu/error-report.h" #include "qapi/error.h" +#include "clients.h" +#include "vmnet_int.h" #include <vmnet/vmnet.h> +typedef struct VmnetBridgedState { + VmnetCommonState cs; +} VmnetBridgedState; + +static bool validate_ifname(const char *ifname) +{ + xpc_object_t shared_if_list = vmnet_copy_shared_interface_list(); + __block bool match = false; + + xpc_array_apply( + shared_if_list, + ^bool(size_t index, xpc_object_t value) { + if (strcmp(xpc_string_get_string_ptr(value), ifname) == 0) { + match = true; + return false; + } + return true; + }); + + return match; +} + +static const char *get_valid_ifnames(void) +{ + xpc_object_t shared_if_list = vmnet_copy_shared_interface_list(); + __block char *if_list = NULL; + + xpc_array_apply( + shared_if_list, + ^bool(size_t index, xpc_object_t value) { + if_list = g_strconcat(xpc_string_get_string_ptr(value), + " ", + if_list, + NULL); + return true; + }); + + if (if_list) { + return if_list; + } + return "[no interfaces]"; +} + +static xpc_object_t create_if_desc(const Netdev *netdev, Error **errp) +{ + const NetdevVmnetBridgedOptions *options = &(netdev->u.vmnet_bridged); + xpc_object_t if_desc = xpc_dictionary_create(NULL, NULL, 0); + + xpc_dictionary_set_uint64( + if_desc, + vmnet_operation_mode_key, + VMNET_BRIDGED_MODE + ); + + xpc_dictionary_set_bool( + if_desc, + vmnet_enable_isolation_key, + options->isolated + ); + + if (validate_ifname(options->ifname)) { + xpc_dictionary_set_string(if_desc, + vmnet_shared_interface_name_key, + options->ifname); + } else { + return NULL; + } + return if_desc; +} + +static NetClientInfo net_vmnet_bridged_info = { + .type = NET_CLIENT_DRIVER_VMNET_BRIDGED, + .size = sizeof(VmnetBridgedState), + .receive = vmnet_receive_common, + .cleanup = vmnet_cleanup_common, +}; + int net_init_vmnet_bridged(const Netdev *netdev, const char *name, NetClientState *peer, Error **errp) { - error_setg(errp, "vmnet-bridged is not implemented yet"); - return -1; -} + NetClientState *nc = qemu_new_net_client(&net_vmnet_bridged_info, + peer, "vmnet-bridged", name); + xpc_object_t if_desc = create_if_desc(netdev, errp);; + + if (!if_desc) { + error_setg(errp, + "unsupported ifname, should be one of: %s", + get_valid_ifnames()); + return -1; + } + + return vmnet_if_create(nc, if_desc, errp, NULL); +}
Signature · 1 line
2.23.0
PATCH v5 6/6 net/vmnet: update qemu-options.hx
In reply to Vladislav Yaroshchuk--- qemu-options.hx | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
qemu-options.hx +25 -0
@@ -2677,6 +2677,25 @@ DEF("netdev", HAS_ARG, QEMU_OPTION_netdev, #ifdef __linux__ "-netdev vhost-vdpa,id=str,vhostdev=/path/to/dev\n" " configure a vhost-vdpa network,Establish a vhost-vdpa netdev\n" +#endif +#ifdef CONFIG_VMNET + "-netdev vmnet-host,id=str[,isolated=on|off][,net-uuid=uuid]\n" + " [,dhcpstart=addr,dhcpend=addr,subnet-mask=mask]\n" + " configure a vmnet network backend in host mode with ID 'str',\n" + " isolate this interface from others with 'isolated',\n" + " configure its DHCP server and choose a subnet mask\n" + " or specify network UUID 'uuid' to disable DHCP and interact with\n" + " vmnet-host interfaces within this isolated network\n" + "-netdev vmnet-shared,id=str[,isolated=on|off][,nat66-prefix=addr]\n" + " [,dhcpstart=addr,dhcpend=addr,subnet-mask=mask]\n" + " configure a vmnet network backend in shared mode with ID 'str',\n" + " configure its DHCP server and choose a subnet mask,\n" + " set IPv6 ULA prefix (of length 64) to use for internal network,\n" + " isolate this interface from others with 'isolated'\n" + "-netdev vmnet-bridged,id=str,ifname=name[,isolated=on|off]\n" + " configure a vmnet network backend in bridged mode with ID 'str',\n" + " use 'ifname=name' to select a physical network interface to be bridged,\n" + " isolate this interface from others with 'isolated'\n" #endif "-netdev hubport,id=str,hubid=n[,netdev=nd]\n" " configure a hub port on the hub with ID 'n'\n", QEMU_ARCH_ALL)
@@ -2696,6 +2715,9 @@ DEF("nic", HAS_ARG, QEMU_OPTION_nic, #endif #ifdef CONFIG_POSIX "vhost-user|" +#endif +#ifdef CONFIG_VMNET + "vmnet-host|vmnet-shared|vmnet-bridged|" #endif "socket][,option][,...][mac=macaddr]\n" " initialize an on-board / default host NIC (using MAC address\n"
@@ -2718,6 +2740,9 @@ DEF("net", HAS_ARG, QEMU_OPTION_net, #endif #ifdef CONFIG_NETMAP "netmap|" +#endif +#ifdef CONFIG_VMNET + "vmnet-host|vmnet-shared|vmnet-bridged|" #endif "socket][,option][,option][,...]\n" " old way to initialize a host network interface\n"
Signature · 1 line
2.23.0
Quoted 30 lines · Vladislav Yaroshchuk
macOS provides networking API for VMs called 'vmnet.framework':
https://developer.apple.com/documentation/vmnetWe can provide its support as the new QEMU network backends which represent three different vmnet.framework interface usage modes:
*
vmnet-shared:
allows the guest to communicate with other guests in shared mode and also with external network (Internet) via NAT. Has (macOS-provided) DHCP server; subnet mask and IP range can be configured;*
vmnet-host:
allows the guest to communicate with other guests in host mode. By default has enabled DHCP asvmnet-shared, but providing network unique id (uuid) can makevmnet-hostinterfaces isolated from each other and also disables DHCP.*
vmnet-bridged:
bridges the guest with a physical network interface.This backends cannot work on macOS Catalina 10.15 cause we use vmnet.framework API provided only with macOS 11 and newer. Seems that it is not a problem, because QEMU guarantees to work on two most recent versions of macOS which now are Big Sur (11) and Monterey (12).
Also, we have one inconvenient restriction: vmnet.framework interfaces can create only privileged user:
$ sudo qemu-system-x86_64 -nic vmnet-shared
Quoted 8 lines
Attempt of
vmnet-*netdev creation being unprivileged user fails with vmnet's 'general failure'.This happens because vmnet.framework requires
com.apple.vm.networkingentitlement which is: "restricted to developers of virtualization software. To request this entitlement, contact your Apple representative." as Apple documentation says:
https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networking
Do you know how multipass work? Looks like it uses vmnet without privileges.
Thanks
Quoted 73 lines
One more note: we still have quite useful but not supported 'vmnet.framework' features as creating port forwarding rules, IPv6 NAT prefix specifying and so on.
Nevertheless, new backends work fine and tested within
qemu-system-x86-64on macOS Bir Sur 11.5.2 host with such nic models:
* e1000-82545em
* virtio-net-pci
* vmxnet3The guests were:
* macOS 10.15.7
* Ubuntu Bionic (server cloudimg)This series partially reuses patches by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/
So I included his signed-off line into one of the commit messages and also here.v1 -> v2:
Since v1 minor typos were fixed, patches rebased onto latest master, redundant changes removed (small commits squashed)v2 -> v3:
- QAPI style fixes
- Typos fixes in comments
-#include's updated to be in sync with recent master
v3 -> v4:
- Support vmnet interfaces isolation feature
- Support vmnet-host network uuid setting feature
- Refactored sources a bit
v4 -> v5:
- Missed 6.2 boat, now 7.0 candidate
- Fix qapi netdev descriptions and styles (@subnetmask -> @subnet-mask)
- Support vmnet-shared IPv6 prefix setting featureSigned-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>Vladislav Yaroshchuk (6):
net/vmnet: add vmnet dependency and customizable option net/vmnet: add vmnet backends to qapi/net net/vmnet: implement shared mode (vmnet-shared) net/vmnet: implement host mode (vmnet-host) net/vmnet: implement bridged mode (vmnet-bridged) net/vmnet: update qemu-options.hxmeson.build | 4 + meson_options.txt | 2 + net/clients.h | 11 ++ net/meson.build | 7 + net/net.c | 10 ++ net/vmnet-bridged.m | 111 ++++++++++++ net/vmnet-common.m | 325 ++++++++++++++++++++++++++++++++++ net/vmnet-host.c | 111 ++++++++++++ net/vmnet-shared.c | 92 ++++++++++ net/vmnet_int.h | 48 +++++ qapi/net.json | 127 ++++++++++++- qemu-options.hx | 25 +++ scripts/meson-buildoptions.sh | 3 + 13 files changed, 874 insertions(+), 2 deletions(-) create mode 100644 net/vmnet-bridged.m create mode 100644 net/vmnet-common.m create mode 100644 net/vmnet-host.c create mode 100644 net/vmnet-shared.c create mode 100644 net/vmnet_int.h--
2.23.0
пн, 15 нояб. 2021 г. в 07:53, Jason Wang <[email protected]>:
Quoted 37 lines
Quoted 30 lines · Vladislav Yaroshchuk
macOS provides networking API for VMs called 'vmnet.framework':
https://developer.apple.com/documentation/vmnetWe can provide its support as the new QEMU network backends which represent three different vmnet.framework interface usage modes:
*
vmnet-shared:
allows the guest to communicate with other guests in shared mode and also with external network (Internet) via NAT. Has (macOS-provided) DHCP server; subnet mask and IP range can be configured;*
vmnet-host:
allows the guest to communicate with other guests in host mode. By default has enabled DHCP asvmnet-shared, but providing network unique id (uuid) can makevmnet-hostinterfaces isolated from each other and also disables DHCP.*
vmnet-bridged:
bridges the guest with a physical network interface.This backends cannot work on macOS Catalina 10.15 cause we use vmnet.framework API provided only with macOS 11 and newer. Seems that it is not a problem, because QEMU guarantees to work on two most recent versions of macOS which now are Big Sur (11) and Monterey (12).
Also, we have one inconvenient restriction: vmnet.framework interfaces can create only privileged user:
$ sudo qemu-system-x86_64 -nic vmnet-sharedQuoted 5 lines
Attempt of
vmnet-*netdev creation being unprivileged user fails with vmnet's 'general failure'.This happens because vmnet.framework requires
com.apple.vm.networkingentitlement which is: "restricted to developers of virtualization
Quoted 10 lines
software.
Quoted 3 lines
To request this entitlement, contact your Apple representative." as Apple documentation says:
https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networking
Do you know how multipass work? Looks like it uses vmnet without privileges.
I've just checked this, and they still need root privileges. They have a multipassd daemon which is launched as root by launchd by default.
``` bash-5.1$ ps axo ruser,ruid,comm | grep multipass root 0 /Library/Application Support/com.canonical.multipass/bin/multipassd root 0 /Library/Application Support/com.canonical.multipass/bin/hyperkit ```
That's the reason why it's required to 'enter a password' while multipass installation:
it creates launchd plist (kinda launch rule) and places it to /Library/LaunchDaemons/,
which can be done only with root privileges.
``` bash-5.1$ ll /Library/LaunchDaemons | grep multipass -rw-r--r-- 1 root wheel 1.1K 15 Nov 12:47 com.canonical.multipassd.plist ```
And after this launchd launches this multipassd daemon as root every boot.
So an unprivileged user can launch a multipass VM instance, but actually thehyperkit process which interacts with vmnet is gonna be launched by multipassd
running as root.
tl;dr sadly, we can't interact with vmnet.framework without having our binary correctly
signed and being an unprivileged user. Root privileges or special signature with
entitlement is required.
Thanks
Quoted 2 lines
Thank you for your review, I will check it this week and reply as soon as possible.
Quoted 76 lines
Quoted 6 lines
One more note: we still have quite useful but not supported 'vmnet.framework' features as creating port forwarding rules, IPv6 NAT prefix specifying and so on.
Nevertheless, new backends work fine and tested within
qemu-system-x86-64Quoted 67 lines
on macOS Bir Sur 11.5.2 host with such nic models:
* e1000-82545em
* virtio-net-pci
* vmxnet3The guests were:
* macOS 10.15.7
* Ubuntu Bionic (server cloudimg)This series partially reuses patches by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/
So I included his signed-off line into one of the commit messages and also here.v1 -> v2:
Since v1 minor typos were fixed, patches rebased onto latest master, redundant changes removed (small commits squashed)v2 -> v3:
- QAPI style fixes
- Typos fixes in comments
-#include's updated to be in sync with recent master
v3 -> v4:
- Support vmnet interfaces isolation feature
- Support vmnet-host network uuid setting feature
- Refactored sources a bit
v4 -> v5:
- Missed 6.2 boat, now 7.0 candidate
- Fix qapi netdev descriptions and styles (@subnetmask -> @subnet-mask)
- Support vmnet-shared IPv6 prefix setting featureSigned-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>Vladislav Yaroshchuk (6):
net/vmnet: add vmnet dependency and customizable option net/vmnet: add vmnet backends to qapi/net net/vmnet: implement shared mode (vmnet-shared) net/vmnet: implement host mode (vmnet-host) net/vmnet: implement bridged mode (vmnet-bridged) net/vmnet: update qemu-options.hxmeson.build | 4 + meson_options.txt | 2 + net/clients.h | 11 ++ net/meson.build | 7 + net/net.c | 10 ++ net/vmnet-bridged.m | 111 ++++++++++++ net/vmnet-common.m | 325 ++++++++++++++++++++++++++++++++++ net/vmnet-host.c | 111 ++++++++++++ net/vmnet-shared.c | 92 ++++++++++ net/vmnet_int.h | 48 +++++ qapi/net.json | 127 ++++++++++++- qemu-options.hx | 25 +++ scripts/meson-buildoptions.sh | 3 + 13 files changed, 874 insertions(+), 2 deletions(-) create mode 100644 net/vmnet-bridged.m create mode 100644 net/vmnet-common.m create mode 100644 net/vmnet-host.c create mode 100644 net/vmnet-shared.c create mode 100644 net/vmnet_int.h--
2.23.0
Signature · 3 lines
Best Regards, Vladislav Yaroshchuk
Quoted 67 lines · Vladislav Yaroshchuk
пн, 15 нояб. 2021 г. в 07:53, Jason Wang <[email protected]>:
Quoted 36 lines
Quoted 33 lines · Vladislav Yaroshchuk
macOS provides networking API for VMs called 'vmnet.framework':
https://developer.apple.com/documentation/vmnetWe can provide its support as the new QEMU network backends which represent three different vmnet.framework interface usage modes:
*
vmnet-shared:
allows the guest to communicate with other guests in shared mode and also with external network (Internet) via NAT. Has (macOS-provided) DHCP server; subnet mask and IP range can be configured;*
vmnet-host:
allows the guest to communicate with other guests in host mode. By default has enabled DHCP asvmnet-shared, but providing network unique id (uuid) can makevmnet-hostinterfaces isolated from each other and also disables DHCP.*
vmnet-bridged:
bridges the guest with a physical network interface.This backends cannot work on macOS Catalina 10.15 cause we use vmnet.framework API provided only with macOS 11 and newer. Seems that it is not a problem, because QEMU guarantees to work on two most recent versions of macOS which now are Big Sur (11) and Monterey (12).
Also, we have one inconvenient restriction: vmnet.framework interfaces can create only privileged user:
$ sudo qemu-system-x86_64 -nic vmnet-sharedAttempt of
vmnet-*netdev creation being unprivileged user fails with vmnet's 'general failure'.Quoted 8 lines
Quoted 5 lines
This happens because vmnet.framework requires
com.apple.vm.networkingentitlement which is: "restricted to developers of virtualization software. To request this entitlement, contact your Apple representative." as Apple documentation says:
https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networkingDo you know how multipass work? Looks like it uses vmnet without privileges.
I've just checked this, and they still need root privileges. They have a
multipassddaemon which is launched as root by launchd by default.``` bash-5.1$ ps axo ruser,ruid,comm | grep multipass root 0 /Library/Application Support/com.canonical.multipass/bin/multipassd root 0 /Library/Application Support/com.canonical.multipass/bin/hyperkit ```That's the reason why it's required to 'enter a password' while multipass installation:
it creates launchd plist (kinda launch rule) and places it to /Library/LaunchDaemons/, which can be done only with root privileges.``` bash-5.1$ ll /Library/LaunchDaemons | grep multipass -rw-r--r-- 1 root wheel 1.1K 15 Nov 12:47 com.canonical.multipassd.plist ```
Quoted 5 lines
And after this launchd launches this multipassd daemon as root every boot.
So an unprivileged user can launch a multipass VM instance, but actually the
hyperkitprocess which interacts with vmnet is gonna be launched by multipassd running as root.
I wonder how it passes the vmnet object to qemu? Nothing obvious from the qemu command line that multipass launched:
-nic vmnet-macos,mode=shared,model=virtio-net-pci,mac=52:54:00:52:e8:e4
(But I haven't had time to check their qemu codes).
Quoted 4 lines
tl;dr sadly, we can't interact with vmnet.framework without having our binary correctly signed and being an unprivileged user. Root privileges or special signature with entitlement is required.
This is something similar to what happens in other OS. E.g for the tap backend, it can't be created without privileges. So qemu allows to:
1) the TAP to be created by privileged program like libvirt, and its fd could be passed to qemu via SCM_RIGHTS
2) run a set-uid helper to create and config TAP
This is something we need to consider now or in the future probably.
Thanks
Quoted 88 lines
Quoted 4 lines
Thanks
Thank you for your review, I will check it this week and reply as soon as possible.
Quoted 71 lines
Quoted 71 lines
One more note: we still have quite useful but not supported 'vmnet.framework' features as creating port forwarding rules, IPv6 NAT prefix specifying and so on.
Nevertheless, new backends work fine and tested within
qemu-system-x86-64on macOS Bir Sur 11.5.2 host with such nic models:
* e1000-82545em
* virtio-net-pci
* vmxnet3The guests were:
* macOS 10.15.7
* Ubuntu Bionic (server cloudimg)This series partially reuses patches by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/
So I included his signed-off line into one of the commit messages and also here.v1 -> v2:
Since v1 minor typos were fixed, patches rebased onto latest master, redundant changes removed (small commits squashed)v2 -> v3:
- QAPI style fixes
- Typos fixes in comments
-#include's updated to be in sync with recent master
v3 -> v4:
- Support vmnet interfaces isolation feature
- Support vmnet-host network uuid setting feature
- Refactored sources a bit
v4 -> v5:
- Missed 6.2 boat, now 7.0 candidate
- Fix qapi netdev descriptions and styles (@subnetmask -> @subnet-mask)
- Support vmnet-shared IPv6 prefix setting featureSigned-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>Vladislav Yaroshchuk (6):
net/vmnet: add vmnet dependency and customizable option net/vmnet: add vmnet backends to qapi/net net/vmnet: implement shared mode (vmnet-shared) net/vmnet: implement host mode (vmnet-host) net/vmnet: implement bridged mode (vmnet-bridged) net/vmnet: update qemu-options.hxmeson.build | 4 + meson_options.txt | 2 + net/clients.h | 11 ++ net/meson.build | 7 + net/net.c | 10 ++ net/vmnet-bridged.m | 111 ++++++++++++ net/vmnet-common.m | 325 ++++++++++++++++++++++++++++++++++ net/vmnet-host.c | 111 ++++++++++++ net/vmnet-shared.c | 92 ++++++++++ net/vmnet_int.h | 48 +++++ qapi/net.json | 127 ++++++++++++- qemu-options.hx | 25 +++ scripts/meson-buildoptions.sh | 3 + 13 files changed, 874 insertions(+), 2 deletions(-) create mode 100644 net/vmnet-bridged.m create mode 100644 net/vmnet-common.m create mode 100644 net/vmnet-host.c create mode 100644 net/vmnet-shared.c create mode 100644 net/vmnet_int.hQuoted 4 lines
Quoted 3 lines
--
2.23.0--
Best Regards,Vladislav Yaroshchuk
вт, 16 нояб. 2021 г. в 10:23, Jason Wang <[email protected]>:
Quoted 89 lines
Quoted 16 lines · Vladislav Yaroshchuk
пн, 15 нояб. 2021 г. в 07:53, Jason Wang <[email protected]>:
Quoted 12 lines
Quoted 9 lines · Vladislav Yaroshchuk
macOS provides networking API for VMs called 'vmnet.framework':
https://developer.apple.com/documentation/vmnetWe can provide its support as the new QEMU network backends which represent three different vmnet.framework interface usage modes:
*
vmnet-shared:
allows the guest to communicate with other guests in shared modeand
Quoted 1 line
Quoted 1 line
Quoted 1 line
also with external network (Internet) via NAT. Has
(macOS-provided)
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
DHCP server; subnet mask and IP range can be configured;*
vmnet-host:
allows the guest to communicate with other guests in host mode. By default has enabled DHCP asvmnet-shared, but providing network unique id (uuid) can makevmnet-hostinterfaces isolated from each other and also disables DHCP.*
vmnet-bridged:
bridges the guest with a physical network interface.This backends cannot work on macOS Catalina 10.15 cause we use vmnet.framework API provided only with macOS 11 and newer. Seems that it is not a problem, because QEMU guarantees to work on two most recent versions of macOS which now are Big Sur (11) and Monterey (12).
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Also, we have one inconvenient restriction: vmnet.framework interfaces can create only privileged user:
$ sudo qemu-system-x86_64 -nic vmnet-sharedAttempt of
vmnet-*netdev creation being unprivileged user failswith
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
vmnet's 'general failure'.
This happens because vmnet.framework requires
com.apple.vm.networkingQuoted 1 line
Quoted 1 line
Quoted 1 line
entitlement which is: "restricted to developers of virtualization
software.
Quoted 1 line
Quoted 1 line
Quoted 1 line
To request this entitlement, contact your Apple representative." as
Apple
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
documentation says:
https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networking
Quoted 2 lines
Quoted 2 lines
Do you know how multipass work? Looks like it uses vmnet without
privileges.
Quoted 8 lines
Quoted 1 line
I've just checked this, and they still need root privileges. They have a
multipassddaemon which is launched as root by launchd by default.```
bash-5.1$ ps axo ruser,ruid,comm | grep multipass
root 0 /Library/ApplicationSupport/com.canonical.multipass/bin/multipassd
Quoted 1 line
root 0 /Library/Application
Support/com.canonical.multipass/bin/hyperkit
Quoted 3 lines
```
That's the reason why it's required to 'enter a password' while
multipass installation:
Quoted 1 line
it creates launchd plist (kinda launch rule) and places it to
/Library/LaunchDaemons/,
Quoted 5 lines
which can be done only with root privileges.
```
bash-5.1$ ll /Library/LaunchDaemons | grep multipass
-rw-r--r-- 1 root wheel 1.1K 15 Nov 12:47com.canonical.multipassd.plist
Quoted 3 lines
```
And after this launchd launches this multipassd daemon as root every
boot.
Quoted 2 lines
So an unprivileged user can launch a multipass VM instance, but actually
the
Quoted 1 line
hyperkitprocess which interacts with vmnet is gonna be launched bymultipassd
Quoted 10 lines
Quoted 1 line
running as root.
I wonder how it passes the vmnet object to qemu? Nothing obvious from the qemu command line that multipass launched:
-nic vmnet-macos,mode=shared,model=virtio-net-pci,mac=52:54:00:52:e8:e4
(But I haven't had time to check their qemu codes).
It seems they just use QEMU with patch by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/
In that patch he does quite the same as we in this series, the difference remains in foreground: he creates one new 'vmnet-macos' netdev, and uses 'mode=shared' property to choose vmnet operating mode. I decided to create three different netdevs instead (vmnet-shared, vmnet-host, vmnet-bridged). Also I've added some features related to isolation and ipv6.
Quoted 1 line
I wonder how it passes the vmnet object to qemu
I hope I clearly described this.
Quoted 7 lines
Quoted 2 lines
tl;dr sadly, we can't interact with vmnet.framework without having our
binary correctly
Quoted 1 line
signed and being an unprivileged user. Root privileges or special
signature with
Quoted 1 line
entitlement is required.
Quoted 10 lines
This is something similar to what happens in other OS. E.g for the tap backend, it can't be created without privileges. So qemu allows to:
1) the TAP to be created by privileged program like libvirt, and its fd could be passed to qemu via
SCM_RIGHTS
2) run a set-uid helper to create and config TAPThis is something we need to consider now or in the future probably.
Seems we can do nothing with this if we have qemu-bundled & direct vmnet.framework interaction, it always requires privileges or entitlement.
The workaround can be moving vmnet-related things to another helper binary running with privileges, and usage of this helper somewhere between qemu and vmnet.
I think for now it's applicable to leave it as is, having qemu that requires privileges for vmnet.framework usage.
Quoted 96 lines
Thanks
Quoted 5 lines
Quoted 3 lines
Thanks
Thank you for your review, I will check it this week and reply as soon
as possible.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
One more note: we still have quite useful but not supported 'vmnet.framework' features as creating port forwarding rules, IPv6 NAT prefix specifying and so on.
Nevertheless, new backends work fine and tested within
qemu-system-x86-64Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
on macOS Bir Sur 11.5.2 host with such nic models:
* e1000-82545em
* virtio-net-pci
* vmxnet3Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
The guests were:
* macOS 10.15.7
* Ubuntu Bionic (server cloudimg)This series partially reuses patches by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/
Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
So I included his signed-off line into one of the commit messages and also here.
v1 -> v2:
Since v1 minor typos were fixed, patches rebased onto latest master, redundant changes removed (small commits squashed)v2 -> v3:
- QAPI style fixes
- Typos fixes in comments
-#include's updated to be in sync with recent master
v3 -> v4:
- Support vmnet interfaces isolation feature
- Support vmnet-host network uuid setting feature
- Refactored sources a bit
v4 -> v5:
- Missed 6.2 boat, now 7.0 candidate
- Fix qapi netdev descriptions and styles (@subnetmask -> @subnet-mask)
- Support vmnet-shared IPv6 prefix setting featureSigned-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>Vladislav Yaroshchuk (6):
net/vmnet: add vmnet dependency and customizable option net/vmnet: add vmnet backends to qapi/net net/vmnet: implement shared mode (vmnet-shared) net/vmnet: implement host mode (vmnet-host) net/vmnet: implement bridged mode (vmnet-bridged) net/vmnet: update qemu-options.hxQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
meson.build | 4 + meson_options.txt | 2 + net/clients.h | 11 ++ net/meson.build | 7 + net/net.c | 10 ++ net/vmnet-bridged.m | 111 ++++++++++++ net/vmnet-common.m | 325++++++++++++++++++++++++++++++++++
Quoted 23 lines
Quoted 13 lines
Quoted 13 lines
net/vmnet-host.c | 111 ++++++++++++ net/vmnet-shared.c | 92 ++++++++++ net/vmnet_int.h | 48 +++++ qapi/net.json | 127 ++++++++++++- qemu-options.hx | 25 +++ scripts/meson-buildoptions.sh | 3 + 13 files changed, 874 insertions(+), 2 deletions(-) create mode 100644 net/vmnet-bridged.m create mode 100644 net/vmnet-common.m create mode 100644 net/vmnet-host.c create mode 100644 net/vmnet-shared.c create mode 100644 net/vmnet_int.hQuoted 4 lines
Quoted 3 lines
--
2.23.0--
Best Regards,Vladislav Yaroshchuk
Signature · 3 lines
Best Regards, Vladislav Yaroshchuk
Quoted 91 lines · Vladislav Yaroshchuk
вт, 16 нояб. 2021 г. в 10:23, Jason Wang <[email protected]>:
Quoted 76 lines
Quoted 67 lines · Vladislav Yaroshchuk
пн, 15 нояб. 2021 г. в 07:53, Jason Wang <[email protected]>:
Quoted 36 lines
Quoted 33 lines · Vladislav Yaroshchuk
macOS provides networking API for VMs called 'vmnet.framework':
https://developer.apple.com/documentation/vmnetWe can provide its support as the new QEMU network backends which represent three different vmnet.framework interface usage modes:
*
vmnet-shared:
allows the guest to communicate with other guests in shared mode and also with external network (Internet) via NAT. Has (macOS-provided) DHCP server; subnet mask and IP range can be configured;*
vmnet-host:
allows the guest to communicate with other guests in host mode. By default has enabled DHCP asvmnet-shared, but providing network unique id (uuid) can makevmnet-hostinterfaces isolated from each other and also disables DHCP.*
vmnet-bridged:
bridges the guest with a physical network interface.This backends cannot work on macOS Catalina 10.15 cause we use vmnet.framework API provided only with macOS 11 and newer. Seems that it is not a problem, because QEMU guarantees to work on two most recent versions of macOS which now are Big Sur (11) and Monterey (12).
Also, we have one inconvenient restriction: vmnet.framework interfaces can create only privileged user:
$ sudo qemu-system-x86_64 -nic vmnet-sharedAttempt of
vmnet-*netdev creation being unprivileged user fails with vmnet's 'general failure'.Quoted 8 lines
Quoted 5 lines
This happens because vmnet.framework requires
com.apple.vm.networkingentitlement which is: "restricted to developers of virtualization software. To request this entitlement, contact your Apple representative." as Apple documentation says:
https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networkingDo you know how multipass work? Looks like it uses vmnet without privileges.
I've just checked this, and they still need root privileges. They have a
multipassddaemon which is launched as root by launchd by default.``` bash-5.1$ ps axo ruser,ruid,comm | grep multipass root 0 /Library/Application Support/com.canonical.multipass/bin/multipassd root 0 /Library/Application Support/com.canonical.multipass/bin/hyperkit ```That's the reason why it's required to 'enter a password' while multipass installation:
it creates launchd plist (kinda launch rule) and places it to /Library/LaunchDaemons/, which can be done only with root privileges.``` bash-5.1$ ll /Library/LaunchDaemons | grep multipass -rw-r--r-- 1 root wheel 1.1K 15 Nov 12:47 com.canonical.multipassd.plist ```Quoted 5 lines
And after this launchd launches this multipassd daemon as root every boot.
So an unprivileged user can launch a multipass VM instance, but actually the
hyperkitprocess which interacts with vmnet is gonna be launched by multipassd running as root.Quoted 7 lines
I wonder how it passes the vmnet object to qemu? Nothing obvious from the qemu command line that multipass launched:
-nic vmnet-macos,mode=shared,model=virtio-net-pci,mac=52:54:00:52:e8:e4
(But I haven't had time to check their qemu codes).
It seems they just use QEMU with patch by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/
Quoted 6 lines
In that patch he does quite the same as we in this series, the difference remains in foreground: he creates one new 'vmnet-macos' netdev, and uses 'mode=shared' property to choose vmnet operating mode. I decided to create three different netdevs instead (vmnet-shared, vmnet-host, vmnet-bridged). Also I've added some features related to isolation and ipv6.
Ok.
Quoted 3 lines
Quoted 2 lines
I wonder how it passes the vmnet object to qemu
I hope I clearly described this.
A silly question, how did the 'hyperkit' process pass the vmnet object to qemu?
Quoted 16 lines
Quoted 6 lines
Quoted 5 lines
tl;dr sadly, we can't interact with vmnet.framework without having our binary correctly signed and being an unprivileged user. Root privileges or special signature with entitlement is required.
Quoted 9 lines
This is something similar to what happens in other OS. E.g for the tap backend, it can't be created without privileges. So qemu allows to:
1) the TAP to be created by privileged program like libvirt, and its fd could be passed to qemu via
SCM_RIGHTS
2) run a set-uid helper to create and config TAPThis is something we need to consider now or in the future probably.
Quoted 6 lines
Seems we can do nothing with this if we have qemu-bundled & direct vmnet.framework interaction, it always requires privileges or entitlement.
The workaround can be moving vmnet-related things to another helper binary running with privileges, and usage of this helper somewhere between qemu and vmnet.
Yes, that's my point.
Quoted 3 lines
I think for now it's applicable to leave it as is, having qemu that requires privileges for vmnet.framework usage.
Right, but we need to consider it for the future.
Btw, if you wish, you can list yourself as the maintainer for this backend.
Thanks
Quoted 99 lines
Quoted 93 lines
Thanks
Quoted 88 lines
Quoted 4 lines
Thanks
Thank you for your review, I will check it this week and reply as soon as possible.
Quoted 71 lines
Quoted 71 lines
One more note: we still have quite useful but not supported 'vmnet.framework' features as creating port forwarding rules, IPv6 NAT prefix specifying and so on.
Nevertheless, new backends work fine and tested within
qemu-system-x86-64on macOS Bir Sur 11.5.2 host with such nic models:
* e1000-82545em
* virtio-net-pci
* vmxnet3The guests were:
* macOS 10.15.7
* Ubuntu Bionic (server cloudimg)This series partially reuses patches by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/
So I included his signed-off line into one of the commit messages and also here.v1 -> v2:
Since v1 minor typos were fixed, patches rebased onto latest master, redundant changes removed (small commits squashed)v2 -> v3:
- QAPI style fixes
- Typos fixes in comments
-#include's updated to be in sync with recent master
v3 -> v4:
- Support vmnet interfaces isolation feature
- Support vmnet-host network uuid setting feature
- Refactored sources a bit
v4 -> v5:
- Missed 6.2 boat, now 7.0 candidate
- Fix qapi netdev descriptions and styles (@subnetmask -> @subnet-mask)
- Support vmnet-shared IPv6 prefix setting featureSigned-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>Vladislav Yaroshchuk (6):
net/vmnet: add vmnet dependency and customizable option net/vmnet: add vmnet backends to qapi/net net/vmnet: implement shared mode (vmnet-shared) net/vmnet: implement host mode (vmnet-host) net/vmnet: implement bridged mode (vmnet-bridged) net/vmnet: update qemu-options.hxmeson.build | 4 + meson_options.txt | 2 + net/clients.h | 11 ++ net/meson.build | 7 + net/net.c | 10 ++ net/vmnet-bridged.m | 111 ++++++++++++ net/vmnet-common.m | 325 ++++++++++++++++++++++++++++++++++ net/vmnet-host.c | 111 ++++++++++++ net/vmnet-shared.c | 92 ++++++++++ net/vmnet_int.h | 48 +++++ qapi/net.json | 127 ++++++++++++- qemu-options.hx | 25 +++ scripts/meson-buildoptions.sh | 3 + 13 files changed, 874 insertions(+), 2 deletions(-) create mode 100644 net/vmnet-bridged.m create mode 100644 net/vmnet-common.m create mode 100644 net/vmnet-host.c create mode 100644 net/vmnet-shared.c create mode 100644 net/vmnet_int.hQuoted 4 lines
Quoted 3 lines
--
2.23.0--
Best Regards,Vladislav Yaroshchuk
--
Best Regards,Vladislav Yaroshchuk
ср, 17 нояб. 2021 г. в 09:10, Jason Wang <[email protected]>:
Quoted 124 lines
Quoted 23 lines · Vladislav Yaroshchuk
вт, 16 нояб. 2021 г. в 10:23, Jason Wang <[email protected]>:
Quoted 19 lines
Quoted 16 lines · Vladislav Yaroshchuk
пн, 15 нояб. 2021 г. в 07:53, Jason Wang <[email protected]>:
Quoted 12 lines
Quoted 9 lines · Vladislav Yaroshchuk
macOS provides networking API for VMs called 'vmnet.framework':
https://developer.apple.com/documentation/vmnetWe can provide its support as the new QEMU network backends which represent three different vmnet.framework interface usage modes:
*
vmnet-shared:
allows the guest to communicate with other guests in sharedmode and
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
also with external network (Internet) via NAT. Has
(macOS-provided)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
DHCP server; subnet mask and IP range can be configured;*
vmnet-host:
allows the guest to communicate with other guests in host mode. By default has enabled DHCP asvmnet-shared, but providing network unique id (uuid) can makevmnet-hostinterfacesisolated
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
from each other and also disables DHCP.
*
vmnet-bridged:
bridges the guest with a physical network interface.This backends cannot work on macOS Catalina 10.15 cause we use vmnet.framework API provided only with macOS 11 and newer. Seems that it is not a problem, because QEMU guarantees to work on two
most
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
recent versions of macOS which now are Big Sur (11) and Monterey
(12).
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Also, we have one inconvenient restriction: vmnet.framework
interfaces
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
can create only privileged user:
$ sudo qemu-system-x86_64 -nic vmnet-sharedAttempt of
vmnet-*netdev creation being unprivileged user failswith
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
vmnet's 'general failure'.
This happens because vmnet.framework requires
com.apple.vm.networkingQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
entitlement which is: "restricted to developers of virtualization
software.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
To request this entitlement, contact your Apple representative."
as Apple
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
documentation says:
https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networking
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Do you know how multipass work? Looks like it uses vmnet without
privileges.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
I've just checked this, and they still need root privileges. They
have a
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
multipassddaemon which is launched as root by launchd by default.```
bash-5.1$ ps axo ruser,ruid,comm | grep multipass
root 0 /Library/ApplicationSupport/com.canonical.multipass/bin/multipassd
Quoted 1 line
Quoted 1 line
Quoted 1 line
root 0 /Library/Application
Support/com.canonical.multipass/bin/hyperkit
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
```
That's the reason why it's required to 'enter a password' while
multipass installation:
Quoted 1 line
Quoted 1 line
Quoted 1 line
it creates launchd plist (kinda launch rule) and places it to
/Library/LaunchDaemons/,
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
which can be done only with root privileges.
```
bash-5.1$ ll /Library/LaunchDaemons | grep multipass
-rw-r--r-- 1 root wheel 1.1K 15 Nov 12:47com.canonical.multipassd.plist
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
```
And after this launchd launches this multipassd daemon as root every
boot.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
So an unprivileged user can launch a multipass VM instance, but
actually the
Quoted 1 line
Quoted 1 line
Quoted 1 line
hyperkitprocess which interacts with vmnet is gonna be launched bymultipassd
Quoted 13 lines
Quoted 9 lines
Quoted 1 line
running as root.
I wonder how it passes the vmnet object to qemu? Nothing obvious from the qemu command line that multipass launched:
-nic vmnet-macos,mode=shared,model=virtio-net-pci,mac=52:54:00:52:e8:e4
(But I haven't had time to check their qemu codes).
It seems they just use QEMU with patch by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/Quoted 6 lines
In that patch he does quite the same as we in this series, the difference remains in foreground: he creates one new 'vmnet-macos' netdev, and uses 'mode=shared' property to choose vmnet operating mode. I decided to create three different netdevs instead (vmnet-shared, vmnet-host, vmnet-bridged). Also I've added some features related to isolation and ipv6.
Ok.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
Quoted 1 line
I wonder how it passes the vmnet object to qemu
I hope I clearly described this.
A silly question, how did the 'hyperkit' process pass the vmnet object to qemu?
I think I didn't describe it very well in the previous mail.
The 'hyperkit' and QEMU are two multipass 'drivers'.
hyperkit is an independent hypervisor based on xhyve (bhyve):
https://github.com/moby/hyperkit
So there are many ways to launch multipass VMs:
* VM <-> hyperkit <-> host
* VM <-> QEMU <-> host
* VM <-> virtualbox <-> host
* etc.
https://discourse.ubuntu.com/t/announcing-the-first-release-candidate-for-apple-m1-support/24445
So hyperkit doesn't pass anything to QEMU, it's a separate hypervisor and multipass driver. But it uses vmnet backend too, the same way as we do:
https://github.com/moby/hyperkit/blob/2f061e447e1435cdf1b9eda364cea6414f2c606b/src/lib/pci_virtio_net_vmnet.c#L250
Quoted 28 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
tl;dr sadly, we can't interact with vmnet.framework without having
our binary correctly
Quoted 1 line
Quoted 1 line
Quoted 1 line
signed and being an unprivileged user. Root privileges or special
signature with
Quoted 12 lines
Quoted 11 lines
Quoted 1 line
entitlement is required.
This is something similar to what happens in other OS. E.g for the tap backend, it can't be created without privileges. So qemu allows to:
1) the TAP to be created by privileged program like libvirt, and its fd could be passed to qemu via
SCM_RIGHTS
2) run a set-uid helper to create and config TAPThis is something we need to consider now or in the future probably.
Quoted 6 lines
Seems we can do nothing with this if we have qemu-bundled & direct vmnet.framework interaction, it always requires privileges or entitlement.
The workaround can be moving vmnet-related things to another helper binary running with privileges, and usage of this helper somewhere between qemu and vmnet.Yes, that's my point.
Quoted 1 line
Quoted 8 lines
Quoted 2 lines
I think for now it's applicable to leave it as is, having qemu that requires privileges for vmnet.framework usage.
Right, but we need to consider it for the future.
Btw, if you wish, you can list yourself as the maintainer for this backend.
Ok, thank you! Let me do this in the next patch version, after we finish our discussion and I fix all the issues you pointed in your review.
Quoted 109 lines
Thanks
Quoted 10 lines
Quoted 10 lines
Thanks
Quoted 6 lines
Quoted 4 lines
Thanks
Thank you for your review, I will check it this week and reply as
soon as possible.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
One more note: we still have quite useful but not supported 'vmnet.framework' features as creating port forwarding rules, IPv6 NAT prefix specifying and so on.
Nevertheless, new backends work fine and tested within
qemu-system-x86-64Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
on macOS Bir Sur 11.5.2 host with such nic models:
* e1000-82545em
* virtio-net-pci
* vmxnet3Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
The guests were:
* macOS 10.15.7
* Ubuntu Bionic (server cloudimg)This series partially reuses patches by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
So I included his signed-off line into one of the commit messages
and
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
also here.
v1 -> v2:
Since v1 minor typos were fixed, patches rebased onto latestmaster,
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
redundant changes removed (small commits squashed)
v2 -> v3:
- QAPI style fixes
- Typos fixes in comments
-#include's updated to be in sync with recent master
v3 -> v4:
- Support vmnet interfaces isolation feature
- Support vmnet-host network uuid setting feature
- Refactored sources a bit
v4 -> v5:
- Missed 6.2 boat, now 7.0 candidate
- Fix qapi netdev descriptions and styles (@subnetmask -> @subnet-mask)
- Support vmnet-shared IPv6 prefix setting featureSigned-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>Vladislav Yaroshchuk (6):
net/vmnet: add vmnet dependency and customizable option net/vmnet: add vmnet backends to qapi/net net/vmnet: implement shared mode (vmnet-shared) net/vmnet: implement host mode (vmnet-host) net/vmnet: implement bridged mode (vmnet-bridged) net/vmnet: update qemu-options.hxQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
meson.build | 4 + meson_options.txt | 2 + net/clients.h | 11 ++ net/meson.build | 7 + net/net.c | 10 ++ net/vmnet-bridged.m | 111 ++++++++++++ net/vmnet-common.m | 325++++++++++++++++++++++++++++++++++
Quoted 30 lines
Quoted 19 lines
Quoted 19 lines
Quoted 17 lines
Quoted 16 lines
net/vmnet-host.c | 111 ++++++++++++ net/vmnet-shared.c | 92 ++++++++++ net/vmnet_int.h | 48 +++++ qapi/net.json | 127 ++++++++++++- qemu-options.hx | 25 +++ scripts/meson-buildoptions.sh | 3 + 13 files changed, 874 insertions(+), 2 deletions(-) create mode 100644 net/vmnet-bridged.m create mode 100644 net/vmnet-common.m create mode 100644 net/vmnet-host.c create mode 100644 net/vmnet-shared.c create mode 100644 net/vmnet_int.h--
2.23.0Quoted 5 lines
Quoted 4 lines
--
Best Regards,Vladislav Yaroshchuk
--
Best Regards,Vladislav Yaroshchuk
Quoted 125 lines · Vladislav Yaroshchuk
ср, 17 нояб. 2021 г. в 09:10, Jason Wang <[email protected]>:
Quoted 101 lines
Quoted 91 lines · Vladislav Yaroshchuk
вт, 16 нояб. 2021 г. в 10:23, Jason Wang <[email protected]>:
Quoted 76 lines
Quoted 67 lines · Vladislav Yaroshchuk
пн, 15 нояб. 2021 г. в 07:53, Jason Wang <[email protected]>:
Quoted 36 lines
Quoted 33 lines · Vladislav Yaroshchuk
macOS provides networking API for VMs called 'vmnet.framework':
https://developer.apple.com/documentation/vmnetWe can provide its support as the new QEMU network backends which represent three different vmnet.framework interface usage modes:
*
vmnet-shared:
allows the guest to communicate with other guests in shared mode and also with external network (Internet) via NAT. Has (macOS-provided) DHCP server; subnet mask and IP range can be configured;*
vmnet-host:
allows the guest to communicate with other guests in host mode. By default has enabled DHCP asvmnet-shared, but providing network unique id (uuid) can makevmnet-hostinterfaces isolated from each other and also disables DHCP.*
vmnet-bridged:
bridges the guest with a physical network interface.This backends cannot work on macOS Catalina 10.15 cause we use vmnet.framework API provided only with macOS 11 and newer. Seems that it is not a problem, because QEMU guarantees to work on two most recent versions of macOS which now are Big Sur (11) and Monterey (12).
Also, we have one inconvenient restriction: vmnet.framework interfaces can create only privileged user:
$ sudo qemu-system-x86_64 -nic vmnet-sharedAttempt of
vmnet-*netdev creation being unprivileged user fails with vmnet's 'general failure'.Quoted 8 lines
Quoted 5 lines
This happens because vmnet.framework requires
com.apple.vm.networkingentitlement which is: "restricted to developers of virtualization software. To request this entitlement, contact your Apple representative." as Apple documentation says:
https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networkingDo you know how multipass work? Looks like it uses vmnet without privileges.
I've just checked this, and they still need root privileges. They have a
multipassddaemon which is launched as root by launchd by default.``` bash-5.1$ ps axo ruser,ruid,comm | grep multipass root 0 /Library/Application Support/com.canonical.multipass/bin/multipassd root 0 /Library/Application Support/com.canonical.multipass/bin/hyperkit ```That's the reason why it's required to 'enter a password' while multipass installation:
it creates launchd plist (kinda launch rule) and places it to /Library/LaunchDaemons/, which can be done only with root privileges.``` bash-5.1$ ll /Library/LaunchDaemons | grep multipass -rw-r--r-- 1 root wheel 1.1K 15 Nov 12:47 com.canonical.multipassd.plist ```Quoted 5 lines
And after this launchd launches this multipassd daemon as root every boot.
So an unprivileged user can launch a multipass VM instance, but actually the
hyperkitprocess which interacts with vmnet is gonna be launched by multipassd running as root.Quoted 7 lines
I wonder how it passes the vmnet object to qemu? Nothing obvious from the qemu command line that multipass launched:
-nic vmnet-macos,mode=shared,model=virtio-net-pci,mac=52:54:00:52:e8:e4
(But I haven't had time to check their qemu codes).
It seems they just use QEMU with patch by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/Quoted 6 lines
In that patch he does quite the same as we in this series, the difference remains in foreground: he creates one new 'vmnet-macos' netdev, and uses 'mode=shared' property to choose vmnet operating mode. I decided to create three different netdevs instead (vmnet-shared, vmnet-host, vmnet-bridged). Also I've added some features related to isolation and ipv6.
Quoted 8 lines
Ok.
Quoted 3 lines
Quoted 2 lines
I wonder how it passes the vmnet object to qemu
I hope I clearly described this.
A silly question, how did the 'hyperkit' process pass the vmnet object to qemu?
I think I didn't describe it very well in the previous mail. The 'hyperkit' and QEMU are two multipass 'drivers'.
hyperkit is an independent hypervisor based on xhyve (bhyve):
https://github.com/moby/hyperkitSo there are many ways to launch multipass VMs:
* VM <-> hyperkit <-> host
* VM <-> QEMU <-> host
* VM <-> virtualbox <-> host
* etc.
Quoted 6 lines
https://discourse.ubuntu.com/t/announcing-the-first-release-candidate-for-apple-m1-support/24445
So hyperkit doesn't pass anything to QEMU, it's a separate hypervisor and multipass driver. But it uses vmnet backend too, the same way as we do:
https://github.com/moby/hyperkit/blob/2f061e447e1435cdf1b9eda364cea6414f2c606b/src/lib/pci_virtio_net_vmnet.c#L250
Ok, I think I get it.
Quoted 30 lines
Quoted 30 lines
Quoted 17 lines
Quoted 7 lines
Quoted 6 lines
tl;dr sadly, we can't interact with vmnet.framework without having our binary correctly signed and being an unprivileged user. Root privileges or special signature with entitlement is required.
Quoted 9 lines
This is something similar to what happens in other OS. E.g for the tap backend, it can't be created without privileges. So qemu allows to:
1) the TAP to be created by privileged program like libvirt, and its fd could be passed to qemu via
SCM_RIGHTS
2) run a set-uid helper to create and config TAPThis is something we need to consider now or in the future probably.
Quoted 6 lines
Seems we can do nothing with this if we have qemu-bundled & direct vmnet.framework interaction, it always requires privileges or entitlement.
The workaround can be moving vmnet-related things to another helper binary running with privileges, and usage of this helper somewhere between qemu and vmnet.Yes, that's my point.
Quoted 3 lines
I think for now it's applicable to leave it as is, having qemu that requires privileges for vmnet.framework usage.
Quoted 8 lines
Quoted 4 lines
Right, but we need to consider it for the future.
Btw, if you wish, you can list yourself as the maintainer for this backend.
Ok, thank you! Let me do this in the next patch version, after we finish our discussion and I fix all the issues you pointed in your review.
Sure.
Thanks
Quoted 104 lines
Quoted 104 lines
Thanks
Quoted 99 lines
Quoted 93 lines
Thanks
Quoted 88 lines
Quoted 4 lines
Thanks
Thank you for your review, I will check it this week and reply as soon as possible.
Quoted 71 lines
Quoted 71 lines
One more note: we still have quite useful but not supported 'vmnet.framework' features as creating port forwarding rules, IPv6 NAT prefix specifying and so on.
Nevertheless, new backends work fine and tested within
qemu-system-x86-64on macOS Bir Sur 11.5.2 host with such nic models:
* e1000-82545em
* virtio-net-pci
* vmxnet3The guests were:
* macOS 10.15.7
* Ubuntu Bionic (server cloudimg)This series partially reuses patches by Phillip Tennen:
https://patchew.org/QEMU/[email protected]/
So I included his signed-off line into one of the commit messages and also here.v1 -> v2:
Since v1 minor typos were fixed, patches rebased onto latest master, redundant changes removed (small commits squashed)v2 -> v3:
- QAPI style fixes
- Typos fixes in comments
-#include's updated to be in sync with recent master
v3 -> v4:
- Support vmnet interfaces isolation feature
- Support vmnet-host network uuid setting feature
- Refactored sources a bit
v4 -> v5:
- Missed 6.2 boat, now 7.0 candidate
- Fix qapi netdev descriptions and styles (@subnetmask -> @subnet-mask)
- Support vmnet-shared IPv6 prefix setting featureSigned-off-by Phillip Tennen <[email protected]>Signed-off-by Vladislav Yaroshchuk <[email protected]>Vladislav Yaroshchuk (6):
net/vmnet: add vmnet dependency and customizable option net/vmnet: add vmnet backends to qapi/net net/vmnet: implement shared mode (vmnet-shared) net/vmnet: implement host mode (vmnet-host) net/vmnet: implement bridged mode (vmnet-bridged) net/vmnet: update qemu-options.hxmeson.build | 4 + meson_options.txt | 2 + net/clients.h | 11 ++ net/meson.build | 7 + net/net.c | 10 ++ net/vmnet-bridged.m | 111 ++++++++++++ net/vmnet-common.m | 325 ++++++++++++++++++++++++++++++++++ net/vmnet-host.c | 111 ++++++++++++ net/vmnet-shared.c | 92 ++++++++++ net/vmnet_int.h | 48 +++++ qapi/net.json | 127 ++++++++++++- qemu-options.hx | 25 +++ scripts/meson-buildoptions.sh | 3 + 13 files changed, 874 insertions(+), 2 deletions(-) create mode 100644 net/vmnet-bridged.m create mode 100644 net/vmnet-common.m create mode 100644 net/vmnet-host.c create mode 100644 net/vmnet-shared.c create mode 100644 net/vmnet_int.hQuoted 4 lines
Quoted 3 lines
--
2.23.0--
Best Regards,Vladislav Yaroshchuk
--
Best Regards,Vladislav Yaroshchuk