introduce vdpa sample
v3 cover introduce vdpa sample
Hi,
This patchset introduces vdpa sample to demonstrate the vDPA use case.
v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments
v2 changes:
* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list,
* quit
* add application documentation
Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA
MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 10 files changed, 635 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
PATCH v3 1/2 vhost: introduce API to get vDPA device number
--- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimentalrte_vdpa_get_device (int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num (void);#endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
@@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num (void ) +{ +return vdpa_device_num; +}
Signature · 1 line
2.17.1
PATCH v3 2/2 examples/vdpa: introduce a new sample for vDPA
In reply to Xiaolong YeThe vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
--- MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 ++++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 7 files changed, 625 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>
doc/guides/sample_app_ug/index.rst +1 -0
@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +115 -0
115 lines not on this page · raw message
examples/Makefile +1 -1
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif
examples/vdpa/Makefile new file +32 -0
@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif
examples/vdpa/main.c new file +458 -0
458 lines not on this page · raw message
examples/vdpa/meson.build new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +)
Signature · 1 line
2.17.1
Re: [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA
Hi Xiaolong,
Quoted 80 lines
-----Original Message-----
From: Ye, Xiaolong
Sent: Friday, September 21, 2018 6:28 AM
To: [email protected]; Maxime Coquelin <[email protected]>; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>
Cc: Wang, Xiao W <[email protected]>; Rami Rosen <[email protected]>; Wang, Haiyue <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPAThe vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiao Wang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 ++++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 7 files changed, 625 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.buildMAINTAINERS +2
-0@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>doc/guides/sample_app_ug/index.rst +1
-0@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipelinedoc/guides/sample_app_ug/vdpa.rst new file +14
-0@@ -0,0 +1,115 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps
[...]
doc/guides/sample_app_ug/vdpa.rst line 68
Quoted 9 lines
doc/guides/sample_app_ug/vdpa.rst new file +9
-0+ +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address + 0 0000:06:00.2 + 1 0000:06:00.3
The features and queue numbers of each vDPA device could also be shown now, the doc should reflect this.
Quoted 100 lines
doc/guides/sample_app_ug/vdpa.rst new file +26
-0+ vdpa> create /tmp/vdpa-socket0 0000:06:00.2 + vdpa> create /tmp/vdpa-socket1 0000:06:00.3 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhostbackend
+VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A isdoc/guides/sample_app_ug/vdpa.rst new file +3
-0+the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VMon A,
doc/guides/sample_app_ug/vdpa.rst new file +17
-0+ in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrateexamples/Makefile +1
-1@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endifexamples/vdpa/Makefile new file +31
-0@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk +
[...]
Quoted 55 lines
examples/vdpa/Makefile new file +54
-0+ +static int +new_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { + printf("\nnew port %s, did: %d\n", + ifname, vports[i].did); + vports[i].vid = vid; + break; + } + } + + if (i >= MAX_VDPA_SAMPLE_PORTS) + return -1; + + return 0; +} + +static void +destroy_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strcmp(ifname, vports[i].ifname) == 0) { + printf("\ndestroy port %s, did: %d\n", + ifname, vports[i].did); + break; + } + } +} + +static const struct vhost_device_ops vdpa_sample_devops = { + .new_device = new_device, + .destroy_device = destroy_device, +}; + +static int +start_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + int did = vport->did; + + if (access(socket_path, F_OK) != -1) { + RTE_LOG(ERR, VDPA, + "%s exists, please remove it or specify another file andtry again.\n",
examples/vdpa/Makefile
Quoted 4 lines
examples/vdpa/Makefile new file +4
-0+ socket_path); + return -1; + } + ret = rte_vhost_driver_register(socket_path, vport->flags);
It would be great if we can provide "client mode" option for this sample to create a client mode vdpa.
Quoted 129 lines
examples/vdpa/Makefile new file +83
-0+ if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver failed: %s\n", + socket_path); + + ret = rte_vhost_driver_callback_register(socket_path, + &vdpa_sample_devops); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver ops failed: %s\n", + socket_path); + + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "attach vdpa device failed: %s\n", + socket_path); + + if (rte_vhost_driver_start(socket_path) < 0) + rte_exit(EXIT_FAILURE, + "start vhost driver failed: %s\n", + socket_path); + return 0; +} + +static void +close_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + + ret = rte_vhost_driver_detach_vdpa_device(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "attach vdpa device failed: %s\n", + socket_path); + + ret = rte_vhost_driver_unregister(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "Fail to unregister vhost driver for %s.\n", + socket_path); +} + +static void +vdpa_sample_quit(void) +{ + int i; + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { + if (vports[i].ifname[0] != '\0') + close_vdpa(&vports[i]); + } +} + +static void +signal_handler(int signum) +{ + if (signum == SIGINT || signum == SIGTERM) { + printf("\nSignal %d received, preparing to exit...\n", signum); + vdpa_sample_quit(); + exit(0); + } +} + +/* interactive cmds */ + +/* *** Help command with introduction. *** */ +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf( + cl, + "\n" + "The following commands are currently available:\n\n" + "Control:\n" + " help : Show interactive instructions.\n" + " list : list all available vdpa devices.\n" + " create <socket file> <vdev addr> : create a new vdpaport.\n"
examples/vdpa/Makefile new file +34
-0+ " quit : exit vdpa sample app.\n" + ); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, + .data = NULL, + .help_str = "show help", + .tokens = { + (void *)&cmd_help_help, + NULL, + }, +}; + +/* *** List all available vdpa devices *** */ +struct cmd_list_result { + cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed( + __attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + uint32_t queue_num; + uint64_t features; + struct rte_vdpa_device *vdev; + struct rte_pci_addr addr; + + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupportedfeatures\n");
examples/vdpa/Makefile new file +10
-0+ for (did = 0; did < dev_total; did++) { + vdev = rte_vdpa_get_device(did); + if (!vdev) + continue; + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa queue number " + "for device id %d.\n", did); + continue; + }
examples/vdpa/Makefile
Quoted 10 lines
examples/vdpa/Makefile new file +10
-0+ if (vdev->ops->get_features(did, &features) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa features " + "for device id %d.\n", did); + continue; + } + addr = vdev->addr.pci_addr; + cmdline_printf(cl, "%d\t\t"PCI_PRI_FMT"\t%d\t\t0x%lx\n", did, + addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features);
Use "%PRIu32"for queue_num, and use "PRIx64" instead of "%lx", otherwise you may have compile error on 32-bit machine.
examples/vdpa/Makefile
Quoted 9 lines
examples/vdpa/Makefile new file +9
-0+ } +} + +cmdline_parse_token_string_t cmd_action_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); + +cmdline_parse_inst_t cmd_list_vdpa_devices = { + .f = cmd_list_vdpa_devices_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */
This comment doesn't provide much info.
Remove the comment for all the cmdline_parse_inst_t instances to keep alignment.
Quoted 133 lines
examples/vdpa/Makefile new file +42
-0+ .help_str = "list all available vdpa devices", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_action_list, + NULL, + }, +}; + +/* *** Create new vdpa port *** */ +struct cmd_create_result { + cmdline_fixed_string_t action; + cmdline_fixed_string_t socket_path; + cmdline_fixed_string_t bdf; +}; + +static void cmd_create_vdpa_port_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + struct cmd_create_result *res = parsed_result; + struct rte_vdpa_dev_addr addr; + + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { + cmdline_printf(cl, "Unable to parse the given bdf.\n"); + return; + } + addr.type = PCI_ADDR; + did = rte_vdpa_find_device_id(&addr); + if (did < 0) { + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); + return; + } + + vports[devcnt].did = did; + + if (start_vdpa(&vports[devcnt]) == 0) + devcnt++; +} + +cmdline_parse_token_string_t cmd_action_create = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action,"create");
examples/vdpa/Makefile new file +2
-0+cmdline_parse_token_string_t cmd_socket_path = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path,NULL);
examples/vdpa/Makefile new file +87
-0+cmdline_parse_token_string_t cmd_bdf = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); + +cmdline_parse_inst_t cmd_create_vdpa_port = { + .f = cmd_create_vdpa_port_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "create a new vdpa port", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_action_create, + (void *)&cmd_socket_path, + (void *)&cmd_bdf, + NULL, + }, +}; + +/* *** QUIT *** */ +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + vdpa_sample_quit(); + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, + .data = NULL, + .help_str = "quit: exit application", + .tokens = { + (void *)&cmd_quit_quit, + NULL, + }, +}; +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, + (cmdline_parse_inst_t *)&cmd_quit, + NULL, +}; + +int +main(int argc, char *argv[]) +{ + char ch; + int i; + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "eal init failed\n"); + argc -= ret; + argv += ret; + + data_init(); + + signal(SIGINT, signal_handler); + signal(SIGTERM, signal_handler); + + ret = parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "invalid argument\n"); + + if (interactive == 1) { + cl = cmdline_stdin_new(main_ctx, "vdpa> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + } else { + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); + i++) { + vports[i].did = i; + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", + iface, i); + + start_vdpa(&vports[i]); + } +
examples/vdpa/Makefile
Quoted 10 lines
examples/vdpa/Makefile new file +10
-0+ printf("enter \'q\' to quit\n"); + while (scanf("%c", &ch)) { + if (ch == 'q') + break; + while (ch != '\n') { + if (scanf("%c", &ch)) + printf("%c", ch); + } + printf("enter \'q\' to quit\n"); + }
Do vdpa_sample_quit here to stop the vhost session and delete the vhost sockets.
Quoted 28 lines
examples/vdpa/Makefile new file +4
-0+ } + + return 0; +}examples/vdpa/meson.build new file +16
-1@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +) --
Quoted 1 line
2.17.1
Do not forget to add this into 18.11 release_note.
BRs,
Xiao
Re: [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 85 lines · Xiao W
Hi Xiaolong,
Quoted 80 lines
-----Original Message-----
From: Ye, Xiaolong
Sent: Friday, September 21, 2018 6:28 AM
To: [email protected]; Maxime Coquelin <[email protected]>; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>
Cc: Wang, Xiao W <[email protected]>; Rami Rosen <[email protected]>; Wang, Haiyue <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPAThe vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiao Wang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 ++++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 7 files changed, 625 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.buildMAINTAINERS +2
-0@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>doc/guides/sample_app_ug/index.rst +1
-0@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipelinedoc/guides/sample_app_ug/vdpa.rst new file +14
-0@@ -0,0 +1,115 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps[...]
Quoted 11 lines
doc/guides/sample_app_ug/vdpa.rst line 68
Quoted 9 lines
doc/guides/sample_app_ug/vdpa.rst new file +9
-0+ +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address + 0 0000:06:00.2 + 1 0000:06:00.3The features and queue numbers of each vDPA device could also be shown now, the doc should reflect this.
Got it.
Quoted 157 lines
Quoted 101 lines
doc/guides/sample_app_ug/vdpa.rst new file +26
-0+ vdpa> create /tmp/vdpa-socket0 0000:06:00.2 + vdpa> create /tmp/vdpa-socket1 0000:06:00.3 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhostbackend
+VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A isdoc/guides/sample_app_ug/vdpa.rst new file +3
-0+the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VMon A,
doc/guides/sample_app_ug/vdpa.rst new file +17
-0+ in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrateexamples/Makefile +1
-1@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endifexamples/vdpa/Makefile new file +31
-0@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk +[...]
Quoted 53 lines
examples/vdpa/Makefile new file +53
-0+ +static int +new_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) { + printf("\nnew port %s, did: %d\n", + ifname, vports[i].did); + vports[i].vid = vid; + break; + } + } + + if (i >= MAX_VDPA_SAMPLE_PORTS) + return -1; + + return 0; +} + +static void +destroy_device(int vid) +{ + char ifname[MAX_PATH_LEN]; + int i; + + rte_vhost_get_ifname(vid, ifname, sizeof(ifname)); + for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) { + if (strcmp(ifname, vports[i].ifname) == 0) { + printf("\ndestroy port %s, did: %d\n", + ifname, vports[i].did); + break; + } + } +} + +static const struct vhost_device_ops vdpa_sample_devops = { + .new_device = new_device, + .destroy_device = destroy_device, +}; + +static int +start_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + int did = vport->did; + + if (access(socket_path, F_OK) != -1) { + RTE_LOG(ERR, VDPA,
Quoted 8 lines
examples/vdpa/Makefile
Quoted 6 lines
examples/vdpa/Makefile new file +1
-0+ "%s exists, please remove it or specify another file andtry again.\n",
examples/vdpa/Makefile new file +4
-0+ socket_path); + return -1; + } + ret = rte_vhost_driver_register(socket_path, vport->flags);It would be great if we can provide "client mode" option for this sample to create a client mode vdpa.
Sure, I'll add client mode support.
Quoted 134 lines
Quoted 134 lines
examples/vdpa/Makefile new file +83
-0+ if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver failed: %s\n", + socket_path); + + ret = rte_vhost_driver_callback_register(socket_path, + &vdpa_sample_devops); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "register driver ops failed: %s\n", + socket_path); + + ret = rte_vhost_driver_attach_vdpa_device(socket_path, did); + if (ret != 0) + rte_exit(EXIT_FAILURE, + "attach vdpa device failed: %s\n", + socket_path); + + if (rte_vhost_driver_start(socket_path) < 0) + rte_exit(EXIT_FAILURE, + "start vhost driver failed: %s\n", + socket_path); + return 0; +} + +static void +close_vdpa(struct vdpa_port *vport) +{ + int ret; + char *socket_path = vport->ifname; + + ret = rte_vhost_driver_detach_vdpa_device(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "attach vdpa device failed: %s\n", + socket_path); + + ret = rte_vhost_driver_unregister(socket_path); + if (ret != 0) + RTE_LOG(ERR, VDPA, + "Fail to unregister vhost driver for %s.\n", + socket_path); +} + +static void +vdpa_sample_quit(void) +{ + int i; + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) { + if (vports[i].ifname[0] != '\0') + close_vdpa(&vports[i]); + } +} + +static void +signal_handler(int signum) +{ + if (signum == SIGINT || signum == SIGTERM) { + printf("\nSignal %d received, preparing to exit...\n", signum); + vdpa_sample_quit(); + exit(0); + } +} + +/* interactive cmds */ + +/* *** Help command with introduction. *** */ +struct cmd_help_result { + cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + cmdline_printf( + cl, + "\n" + "The following commands are currently available:\n\n" + "Control:\n" + " help : Show interactive instructions.\n" + " list : list all available vdpa devices.\n" + " create <socket file> <vdev addr> : create a new vdpaport.\n"
examples/vdpa/Makefile new file +34
-0+ " quit : exit vdpa sample app.\n" + ); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, + .data = NULL, + .help_str = "show help", + .tokens = { + (void *)&cmd_help_help, + NULL, + }, +}; + +/* *** List all available vdpa devices *** */ +struct cmd_list_result { + cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed( + __attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + uint32_t queue_num; + uint64_t features; + struct rte_vdpa_device *vdev; + struct rte_pci_addr addr; + + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupportedfeatures\n");
examples/vdpa/Makefile new file +14
-0+ for (did = 0; did < dev_total; did++) { + vdev = rte_vdpa_get_device(did); + if (!vdev) + continue; + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa queue number " + "for device id %d.\n", did); + continue; + } + if (vdev->ops->get_features(did, &features) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa features " + "for device id %d.\n", did);
Quoted 8 lines
examples/vdpa/Makefile
Quoted 6 lines
examples/vdpa/Makefile new file +6
-0+ continue; + } + addr = vdev->addr.pci_addr; + cmdline_printf(cl, "%d\t\t"PCI_PRI_FMT"\t%d\t\t0x%lx\n", did, + addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features);Use "%PRIu32"for
queue_num, and use "PRIx64" instead of "%lx", otherwise you may have compile error on 32-bit machine.
Got it.
Quoted 13 lines
examples/vdpa/Makefile
Quoted 10 lines
examples/vdpa/Makefile new file +9
-0+ } +} + +cmdline_parse_token_string_t cmd_action_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); + +cmdline_parse_inst_t cmd_list_vdpa_devices = { + .f = cmd_list_vdpa_devices_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */This comment doesn't provide much info.
Remove the comment for all thecmdline_parse_inst_tinstances to keep alignment.
Ok, I'll remove all useless comments.
Quoted 138 lines
Quoted 138 lines
examples/vdpa/Makefile new file +42
-0+ .help_str = "list all available vdpa devices", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_action_list, + NULL, + }, +}; + +/* *** Create new vdpa port *** */ +struct cmd_create_result { + cmdline_fixed_string_t action; + cmdline_fixed_string_t socket_path; + cmdline_fixed_string_t bdf; +}; + +static void cmd_create_vdpa_port_parsed(void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + struct cmd_create_result *res = parsed_result; + struct rte_vdpa_dev_addr addr; + + strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN); + if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) { + cmdline_printf(cl, "Unable to parse the given bdf.\n"); + return; + } + addr.type = PCI_ADDR; + did = rte_vdpa_find_device_id(&addr); + if (did < 0) { + cmdline_printf(cl, "Unable to find vdpa devide id.\n"); + return; + } + + vports[devcnt].did = did; + + if (start_vdpa(&vports[devcnt]) == 0) + devcnt++; +} + +cmdline_parse_token_string_t cmd_action_create = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action,"create");
examples/vdpa/Makefile new file +2
-0+cmdline_parse_token_string_t cmd_socket_path = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path,NULL);
examples/vdpa/Makefile new file +91
-0+cmdline_parse_token_string_t cmd_bdf = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); + +cmdline_parse_inst_t cmd_create_vdpa_port = { + .f = cmd_create_vdpa_port_parsed, /* function to call */ + .data = NULL, /* 2nd arg of func */ + .help_str = "create a new vdpa port", + .tokens = { /* token list, NULL terminated */ + (void *)&cmd_action_create, + (void *)&cmd_socket_path, + (void *)&cmd_bdf, + NULL, + }, +}; + +/* *** QUIT *** */ +struct cmd_quit_result { + cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + vdpa_sample_quit(); + cmdline_quit(cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, + .data = NULL, + .help_str = "quit: exit application", + .tokens = { + (void *)&cmd_quit_quit, + NULL, + }, +}; +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, + (cmdline_parse_inst_t *)&cmd_quit, + NULL, +}; + +int +main(int argc, char *argv[]) +{ + char ch; + int i; + int ret; + struct cmdline *cl; + + ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "eal init failed\n"); + argc -= ret; + argv += ret; + + data_init(); + + signal(SIGINT, signal_handler); + signal(SIGTERM, signal_handler); + + ret = parse_args(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "invalid argument\n"); + + if (interactive == 1) { + cl = cmdline_stdin_new(main_ctx, "vdpa> "); + if (cl == NULL) + rte_panic("Cannot create cmdline instance\n"); + cmdline_interact(cl); + cmdline_stdin_exit(cl); + } else { + for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); + i++) { + vports[i].did = i; + snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d", + iface, i); + + start_vdpa(&vports[i]); + } + + printf("enter \'q\' to quit\n"); + while (scanf("%c", &ch)) { + if (ch == 'q') + break;
Quoted 8 lines
examples/vdpa/Makefile
Quoted 6 lines
examples/vdpa/Makefile new file +6
-0+ while (ch != '\n') { + if (scanf("%c", &ch)) + printf("%c", ch); + } + printf("enter \'q\' to quit\n"); + }Do
vdpa_sample_quithere to stop the vhost session and delete the vhost sockets.
It needs to call vdpa_sample_quit here.
Quoted 24 lines
Quoted 24 lines
examples/vdpa/Makefile new file +4
-0+ } + + return 0; +}examples/vdpa/meson.build new file +13
-0@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true
Quoted 8 lines
Quoted 6 lines
examples/vdpa/meson.build new file +3
-1+sources = files( + 'main.c' +) --2.17.1
Do not forget to add this into 18.11
release_note.
Ok, does this need to make a new separate commit?
Thanks,
Xiaolong
Quoted 4 lines
BRs,
Xiao
v4 cover introduce vdpa sample
In reply to Xiaolong YeHi,
This patchset introduces vdpa sample to demonstrate the vDPA use case.
v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.
v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments
v2 changes:
* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list,
* quit
* add application documentation
Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA
MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 10 files changed, 635 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
PATCH v4 1/2 vhost: introduce API to get vDPA device number
It's used to get number of available registered vDPA devices.
--- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimentalrte_vdpa_get_device (int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num (void);#endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
@@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num (void ) +{ +return vdpa_device_num; +}
Signature · 1 line
2.17.1
PATCH v4 2/2 examples/vdpa: introduce a new sample for vDPA
In reply to Xiaolong YeThe vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
--- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 118 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 466 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
@@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes -----------
doc/guides/sample_app_ug/index.rst +1 -0
@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +118 -0
118 lines not on this page · raw message
examples/Makefile +1 -1
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif
examples/vdpa/Makefile new file +32 -0
@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif
examples/vdpa/main.c new file +466 -0
466 lines not on this page · raw message
examples/vdpa/meson.build new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +)
Signature · 1 line
2.17.1
Re: [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA
Hi Xiaolong,
Thanks for the update, 2 small comments below.
Quoted 133 lines
-----Original Message-----
From: Ye, Xiaolong
Sent: Monday, September 24, 2018 4:43 PM
To: [email protected]; Maxime Coquelin <[email protected]>; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>
Cc: Wang, Xiao W <[email protected]>; Rami Rosen <[email protected]>; Wang, Haiyue <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPAThe vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiao Wang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 118 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 466 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.buildMAINTAINERS +2
-0@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>doc/guides/rel_notes/release_18_11.rst +8
-0@@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes -----------doc/guides/sample_app_ug/index.rst +1
-0@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipelinedoc/guides/sample_app_ug/vdpa.rst new file +36
-0@@ -0,0 +1,118 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps +------------- + +This section shows the steps of how to start VMs with vDPA vhost-user +backend and verify network connection & live migration. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vdpa`` sub-directory. + +Start the vdpa example +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] + +where + +* --client means running vdpa app in client mode, in the client mode, QEMUneeds
doc/guides/sample_app_ug/vdpa.rst new file +9
-0+ to run as the server mode and take charge of socket file creation. +* --iface specifies the path prefix of the UNIX domain socket file, e.g. + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> + (n starts from 0). +* --interactive means run the vdpa sample in interactive mode, currently 4 + internal cmds are supported: + + 1. help: show help message + 2. list: list all available vdpa devices
doc/guides/sample_app_ug/vdpa.rst line 55
Quoted 10 lines
doc/guides/sample_app_ug/vdpa.rst new file +10
-0+ 3. create: create a new vdpa port with socket file and vdpa device address + 4. quit: unregister vhost driver and exit the application + +Take IFCVF driver for example: + +.. code-block:: console + + ./vdpa --log-level=9 -c 0x6 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ + -- --interactive
To demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations, We can use "-c 0x2" to indicate that no need to allocate dedicated worker threads.
Quoted 38 lines
doc/guides/sample_app_ug/vdpa.rst new file +38
-0+ +.. note:: + We need to bind vfio-pci to VFs before running vdpa sample. + + * modprobe vfio-pci + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 + +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address queue num supported features + 0 0000:06:00.3 1 0x5572362272 + 1 0000:06:00.4 1 0x5572362272 + + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA.
[...]
Quoted 39 lines
doc/guides/sample_app_ug/vdpa.rst new file +18
-0+ +/* *** List all available vdpa devices *** */ +struct cmd_list_result { + cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed( + __attribute__((unused)) void *parsed_result, + struct cmdline *cl, + __attribute__((unused)) void *data) +{ + int did; + uint32_t queue_num; + uint64_t features; + struct rte_vdpa_device *vdev; + struct rte_pci_addr addr; + + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupportedfeatures\n");
doc/guides/sample_app_ug/vdpa.rst new file +19
-0+ for (did = 0; did < dev_total; did++) { + vdev = rte_vdpa_get_device(did); + if (!vdev) + continue; + if (vdev->ops->get_queue_num(did, &queue_num) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa queue number " + "for device id %d.\n", did); + continue; + } + if (vdev->ops->get_features(did, &features) < 0) { + RTE_LOG(ERR, VDPA, + "failed to get vdpa features " + "for device id %d.\n", did); + continue; + } + addr = vdev->addr.pci_addr; + cmdline_printf(cl, +"%d\t\t"PCI_PRI_FMT"\t%"PRIu32"\t\t0x%"PRIu64"\n", did,
doc/guides/sample_app_ug/vdpa.rst
Quoted 2 lines
doc/guides/sample_app_ug/vdpa.rst new file +2
-0+ addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features);
Use PRIx64 instead of PRIu64 for features.
You can add a blank space between "PRIx64" and the other section to make it more readable. Refer to:
lib/librte_vhost/vhost_user.c: "guest memory region %u, size: 0x%" PRIx64 "\n"
BRs,
Xiao
Re: [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 4 lines · Xiao W
Hi Xiaolong,
Thanks for the update, 2 small comments below.
[snip]
Quoted 7 lines
doc/guides/sample_app_ug/vdpa.rst line 55
Quoted 3 lines
doc/guides/sample_app_ug/vdpa.rst new file +3
-0+ ./vdpa --log-level=9 -c 0x6 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ + -- --interactiveTo demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations, We can use "-c
0x2" to indicate that no need to allocate dedicated worker threads.
Got it, will do.
Quoted 2 lines
Quoted 2 lines
+
+.. note::
[snip]
Quoted 8 lines
examples/vdpa/main.c line 311
Quoted 3 lines
"%d\t\t"PCI_PRI_FMT"\t%"PRIu32"\t\t0x%"PRIu64"\n", did,examples/vdpa/main.c new file +2
-0+ addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features);Use PRIx64 instead of PRIu64 for features.
You can add a blank space between "PRIx64" and the other section to make it more readable. Refer to:lib/librte_vhost/vhost_user.c: "guest memory region %u, size: 0x%" PRIx64 "\n"
Got it, will do.
Thanks,
Xiaolong
Quoted 3 lines
BRs,
Xiao
v5 cover introduce vdpa sample
In reply to Xiaolong YeHi,
This patchset introduces vdpa sample to demonstrate the vDPA use case.
v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst
v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.
v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments
v2 changes:
* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list,
* quit
* add application documentation
Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA
MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 10 files changed, 635 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
PATCH v5 1/2 vhost: introduce API to get vDPA device number
It's used to get number of available registered vDPA devices.
--- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimentalrte_vdpa_get_device (int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num (void);#endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
@@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num (void ) +{ +return vdpa_device_num; +}
Signature · 1 line
2.17.1
PATCH v5 2/2 examples/vdpa: introduce a new sample for vDPA
In reply to Xiaolong YeThe vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
--- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 118 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 466 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
@@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes -----------
doc/guides/sample_app_ug/index.rst +1 -0
@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +118 -0
118 lines not on this page · raw message
examples/Makefile +1 -1
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif
examples/vdpa/Makefile new file +32 -0
@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif
examples/vdpa/main.c new file +466 -0
466 lines not on this page · raw message
examples/vdpa/meson.build new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +)
Signature · 1 line
2.17.1
Re: [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPA
Hi,
Quoted 134 lines
-----Original Message-----
From: Ye, Xiaolong
Sent: Tuesday, September 25, 2018 8:07 PM
To: [email protected]; Maxime Coquelin <[email protected]>; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>
Cc: Wang, Xiao W <[email protected]>; Rami Rosen <[email protected]>; Wang, Haiyue <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPAThe vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiao Wang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 118 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 466 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.buildMAINTAINERS +2
-0@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>doc/guides/rel_notes/release_18_11.rst +8
-0@@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes -----------doc/guides/sample_app_ug/index.rst +1
-0@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipelinedoc/guides/sample_app_ug/vdpa.rst new file +32
-0@@ -0,0 +1,118 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps +------------- + +This section shows the steps of how to start VMs with vDPA vhost-user +backend and verify network connection & live migration. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vdpa`` sub-directory. + +Start the vdpa example +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface
SOCKET_PATH]doc/guides/sample_app_ug/vdpa.rst new file +4
-0+ +where + +* --client means running vdpa app in client mode, in the client mode, QEMUneeds
doc/guides/sample_app_ug/vdpa.rst new file +9
-0+ to run as the server mode and take charge of socket file creation. +* --iface specifies the path prefix of the UNIX domain socket file, e.g. + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> + (n starts from 0). +* --interactive means run the vdpa sample in interactive mode, currently 4 + internal cmds are supported: + + 1. help: show help message + 2. list: list all available vdpa devices
doc/guides/sample_app_ug/vdpa.rst line 55
Quoted 10 lines
doc/guides/sample_app_ug/vdpa.rst new file +10
-0+ 3. create: create a new vdpa port with socket file and vdpa device address + 4. quit: unregister vhost driver and exit the application + +Take IFCVF driver for example: + +.. code-block:: console + + ./vdpa --log-level=9 -c 0x2 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ + -- --interactive
DPDK has log level 1-8, setting the log-level=9 looks strange. Here you can add description about the whitelisted device, since they are virtio ring compatible devices.
Quoted 22 lines
doc/guides/sample_app_ug/vdpa.rst new file +22
-0+ +.. note:: + We need to bind vfio-pci to VFs before running vdpa sample. + + * modprobe vfio-pci + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 + +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address queue num supported features + 0 0000:06:00.3 1 0x5572362272 + 1 0000:06:00.4 1 0x5572362272 + + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 + +.. _vdpa_app_run_vm: + +Start the VMs
doc/guides/sample_app_ug/vdpa.rst line 87
Quoted 10 lines
doc/guides/sample_app_ug/vdpa.rst new file +10
-0+~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \
In order to leverage EPT, add parameter "page-per-vq=on" for the virtio-net-pci deivce.
Quoted 116 lines
doc/guides/sample_app_ug/vdpa.rst new file +10
-0+ +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhostbackend
+VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A isdoc/guides/sample_app_ug/vdpa.rst new file +3
-0+the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VMon A,
doc/guides/sample_app_ug/vdpa.rst new file +17
-0+ in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrateexamples/Makefile +1
-1@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endifexamples/vdpa/Makefile new file +32
-0@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endifexamples/vdpa/main.c new file +25
-0@@ -0,0 +1,466 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2018 Intel Corporation +*/ + +#include <getopt.h> +#include <signal.h> +#include <stdint.h> +#include <string.h> +#include <unistd.h> + +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_vhost.h> +#include <rte_vdpa.h> +#include <rte_pci.h> + +#include <cmdline_parse.h> +#include <cmdline_socket.h> +#include <cmdline_parse_string.h> +#include <cmdline.h> + +#define MAX_PATH_LEN 128 +#define MAX_VDPA_SAMPLE_PORTS 1024 +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 +
[...]
Quoted 39 lines
examples/vdpa/main.c new file +19
-0+}; + +/* *** List all available vdpa devices *** */ +struct cmd_list_result { +cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed ( + __attribute__((unused))void *parsed_result, +struct cmdline *cl, + __attribute__((unused))void *data) +{ +int did; +uint32_t queue_num; +uint64_t features; +struct rte_vdpa_device *vdev; +struct rte_pci_addr addr; + + cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupportedfeatures\n");
examples/vdpa/main.c new file +19
-0+for (did =0 ; did < dev_total; did++) { + vdev =rte_vdpa_get_device (did); +if (!vdev) +continue ; +if (vdev->ops->get_queue_num (did, &queue_num) <0 ) { +RTE_LOG (ERR ,VDPA , +"failed to get vdpa queue number " +"for device id %d.\n" , did); +continue ; + } +if (vdev->ops->get_features (did, &features) <0 ) { +RTE_LOG (ERR ,VDPA , +"failed to get vdpa features " +"for device id %d.\n" , did); +continue ; + } + addr = vdev->addr.pci_addr; +cmdline_printf (cl, + "%d\t\t" PCI_PRI_FMT "\t%" PRIu32 "\t\t0x%" PRIu64
Quoted 1 line
"\n", did,
s/PRIu64/PRIx64
examples/vdpa/main.c line 313
Quoted 4 lines
examples/vdpa/main.c new file +4
-0+ addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features); + } +}
BRs,
Xiao
v6 cover introduce vdpa sample
In reply to Xiaolong YeHi,
This patchset introduces vdpa sample to demonstrate the vDPA use case.
v6 changes:
* improve the document according to Xiao's comments
* fix a typo, PRIu64 -> PRIx64
v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst
v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.
v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments
v2 changes:
* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list, quit
* add application documentation
Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA
MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 10 files changed, 635 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
PATCH v6 1/2 vhost: introduce API to get vDPA device number
It's used to get number of available registered vDPA devices.
--- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimentalrte_vdpa_get_device (int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num (void);#endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
@@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num (void ) +{ +return vdpa_device_num; +}
Signature · 1 line
2.17.1
PATCH v6 2/2 examples/vdpa: introduce a new sample for vDPA
In reply to Xiaolong YeThe vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
--- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 466 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 646 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
@@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes -----------
doc/guides/sample_app_ug/index.rst +1 -0
@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +120 -0
120 lines not on this page · raw message
examples/Makefile +1 -1
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif
examples/vdpa/Makefile new file +32 -0
@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif
examples/vdpa/main.c new file +466 -0
466 lines not on this page · raw message
examples/vdpa/meson.build new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +)
Signature · 1 line
2.17.1
Re: [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 274 lines · Xiaolong Ye
The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiao Wang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 466 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 646 insertions(+), 1 deletion(-)create mode 100644
doc/guides/sample_app_ug/vdpa.rstcreate mode 100644 examples/vdpa/Makefile create mode 100644examples/vdpa/main.ccreate mode 100644 examples/vdpa/meson.buildMAINTAINERS +2
-0@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>doc/guides/rel_notes/release_18_11.rst +8
-0@@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes -----------doc/guides/sample_app_ug/index.rst +1
-0@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipelinedoc/guides/sample_app_ug/vdpa.rst new file +120
-0@@ -0,0 +1,120 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps +------------- + +This section shows the steps of how to start VMs with vDPA vhost-user +backend and verify network connection & live migration. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vdpa`` sub-directory. + +Start the vdpa example +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] + +where + +* --client means running vdpa app in client mode, in the client mode, QEMU needs + to run as the server mode and take charge of socket file creation. +* --iface specifies the path prefix of the UNIX domain socket file, e.g. + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> + (n starts from 0). +* --interactive means run the vdpa sample in interactive mode, currently 4 + internal cmds are supported: + + 1. help: show help message + 2. list: list all available vdpa devices + 3. create: create a new vdpa port with socket file and vdpa device address + 4. quit: unregister vhost driver and exit the application + +Take IFCVF driver for example: + +.. code-block:: console + + ./vdpa -c 0x2 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ + -- --interactive + +.. note:: + Here 0000:06:00.3 and 0000:06:00.4 refer to virtio ring compatible devices, + and we need to bind vfio-pci to them before running vdpa sample. + + * modprobe vfio-pci + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 + +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address queue num supported features + 0 0000:06:00.3 1 0x14c238020 + 1 0000:06:00.4 1 0x14c238020 + 2 0000:06:00.5 1 0x14c238020 + + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhost backend +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is +the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VM on A, + in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrateexamples/Makefile +1
-1@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endifexamples/vdpa/Makefile new file +32
-0@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endifexamples/vdpa/main.c new file +23
-0@@ -0,0 +1,466 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2018 Intel Corporation +*/ + +#include <getopt.h> +#include <signal.h> +#include <stdint.h> +#include <string.h> +#include <unistd.h> + +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_vhost.h> +#include <rte_vdpa.h> +#include <rte_pci.h> + +#include <cmdline_parse.h> +#include <cmdline_socket.h> +#include <cmdline_parse_string.h> +#include <cmdline.h> + +#define MAX_PATH_LEN 128 +#define MAX_VDPA_SAMPLE_PORTS 1024
examples/vdpa/main.c line 33
Quoted 10 lines
examples/vdpa/main.c new file +10
-0+#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 + +struct vdpa_port { +char ifname[MAX_PATH_LEN ]; +int did; +int vid; +uint64_t flags; +}; + +struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS ];
I guess it can be static?
Quoted 64 lines
examples/vdpa/main.c new file +64
-0+ +static char iface[MAX_PATH_LEN ]; +static int dev_total; +static int devcnt; +static int interactive; +static int client_mode; + +/* display usage */ +static void +vdpa_usage (const char *prgname) +{ +printf ("Usage: %s [EAL options] -- " +" --interactive|-i: run in interactive mode.\n" +" --iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n" +" --client: register a vhost-user socket as client mode.\n" , + prgname); +} + +static int +parse_args (int argc,char **argv) +{ +static const char *short_option ="i" ; +static struct option long_option[] = { + {"iface" , required_argument,NULL ,0 }, + {"interactive" , no_argument, &interactive,1 }, + {"client" , no_argument, &client_mode,1 }, + {NULL ,0 ,0 ,0 }, + }; +int opt, idx; +char *prgname = argv[0 ]; + +while ((opt =getopt_long (argc, argv, short_option, long_option, &idx)) + !=EOF ) { +switch (opt) { +case 'i' : +printf ("Interactive-mode selected\n" ); + interactive =1 ; +break ; +/* long options */ +case 0 : +if (strncmp (long_option[idx].name,"iface" , +MAX_PATH_LEN ) ==0 ) { +strncpy (iface, optarg,MAX_PATH_LEN ); +printf ("iface %s\n" , iface); + } +if (!strcmp (long_option[idx].name,"interactive" )) { +printf ("Interactive-mode selected\n" ); + interactive =1 ; + } +break ; + +default : +vdpa_usage (prgname); +return -1 ; + } + } + +if (iface[0 ] =='\0' && interactive ==0 ) { +vdpa_usage (prgname); +return -1 ; + } + +return 0 ; +}
examples/vdpa/main.c line 107
Quoted 10 lines
examples/vdpa/main.c new file +10
-0+ +static int +data_init (void ) +{ + dev_total =rte_vdpa_get_device_num (); +if (dev_total <=0 ) { +printf ("No available vdpa device found\n" ); +return -1 ; + } + devcnt =0 ;
This isn't needed as it is static.
examples/vdpa/main.c line 108
Quoted 1 line
examples/vdpa/main.c new file +1
-0+ interactive =0 ;
Ditto
examples/vdpa/main.c line 109
Quoted 1 line
examples/vdpa/main.c new file +1
-0+memset (iface,0 ,sizeof (iface));
Ditto
examples/vdpa/main.c line 110
Quoted 1 line
examples/vdpa/main.c new file +1
-0+memset (vports,0 ,sizeof (vports));
And this one too when it will be made static.
Appart these small comments, it looks good to me.
When fixed, please apply my:
Thanks,
Maxime
Re: [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA
On 09/27, Maxime Coquelin wrote:
[snip]
Quoted 9 lines
examples/vdpa/main.c line 107
Quoted 1 line
examples/vdpa/main.c new file +1
-0+ devcnt =0 ;This isn't needed as it is static.
examples/vdpa/main.c line 108
Quoted 1 line
examples/vdpa/main.c new file +1
-0+ interactive =0 ;Ditto
examples/vdpa/main.c line 109
Quoted 1 line
examples/vdpa/main.c new file +1
-0+memset (iface,0 ,sizeof (iface));Ditto
Quoted 9 lines
examples/vdpa/main.c line 110
Quoted 1 line
examples/vdpa/main.c new file +1
-0+memset (vports,0 ,sizeof (vports));And this one too when it will be made static.
Appart these small comments, it looks good to me.
When fixed, please apply my:Reviewed-by Maxime coquelin <[email protected]>
Thanks for your comments, I'll adpot all the static related suggetions.
Thanks,
Xiaolong
Quoted 2 lines
Thanks,
Maxime
Hi,
Quoted 62 lines
-----Original Message-----
From: Ye, Xiaolong
Sent: Wednesday, September 26, 2018 5:07 PM
To: [email protected]; Maxime Coquelin <[email protected]>; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>
Cc: Wang, Xiao W <[email protected]>; Rami Rosen <[email protected]>; Wang, Haiyue <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [PATCH v6 0/2] introduce vdpa sampleHi,
This patchset introduces vdpa sample to demonstrate the vDPA use case.
v6 changes:
* improve the document according to Xiao's comments
* fix a typo, PRIu64 -> PRIx64v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations invdpa.rstv4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve thevdpa.rstaccordingly
* remove some useless comments
* add introduction in 18.11 release note.v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review commentsv2 changes:
* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list, quit
* add application documentationXiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPAMAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 10 files changed, 635 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
Quoted 2 lines
--
2.17.1
Series Acked-by: Xiao Wang <[email protected]>
BRs,
Xiao
v7 cover introduce vdpa sample
In reply to Xiaolong YeHi,
This patchset introduces vdpa sample to demonstrate the vDPA use case.
v7 changes:
* make vports static
* avoid unnecessary static variable initialization
v6 changes:
* improve the document according to Xiao's comments
* fix a typo, PRIu64 -> PRIx64
v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst
v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.
v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments
v2 changes:
* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list, quit
* add application documentation
Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA
MAINTAINERS | 2 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 115 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 458 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 10 files changed, 635 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
PATCH v7 1/2 vhost: introduce API to get vDPA device number
It's used to get number of available registered vDPA devices.
--- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimentalrte_vdpa_get_device (int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num (void);#endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
@@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num (void ) +{ +return vdpa_device_num; +}
Signature · 1 line
2.17.1
PATCH v7 2/2 examples/vdpa: introduce a new sample for vDPA
In reply to Xiaolong YeThe vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
--- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 462 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 642 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
@@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes -----------
doc/guides/sample_app_ug/index.rst +1 -0
@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +120 -0
120 lines not on this page · raw message
examples/Makefile +1 -1
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif
examples/vdpa/Makefile new file +32 -0
@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif
examples/vdpa/main.c new file +462 -0
462 lines not on this page · raw message
examples/vdpa/meson.build new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +)
Signature · 1 line
2.17.1
Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 665 lines · Xiaolong Ye
The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Reviewed-by Maxime coquelin <[email protected]>Acked-by Xiao Wang <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 462 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 642 insertions(+), 1 deletion(-)create mode 100644
doc/guides/sample_app_ug/vdpa.rstcreate mode 100644 examples/vdpa/Makefile create mode 100644examples/vdpa/main.ccreate mode 100644 examples/vdpa/meson.buildMAINTAINERS +2
-0@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>doc/guides/rel_notes/release_18_11.rst +8
-0@@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes -----------doc/guides/sample_app_ug/index.rst +1
-0@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipelinedoc/guides/sample_app_ug/vdpa.rst new file +120
-0@@ -0,0 +1,120 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2018 Intel Corporation. + +Vdpa Sample Application +======================= + +The vdpa sample application creates vhost-user sockets by using the +vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes +virtio ring compatible devices to serve virtio driver directly to enable +datapath acceleration. As vDPA driver can help to set up vhost datapath, +this application doesn't need to launch dedicated worker threads for vhost +enqueue/dequeue operations. + +Testing steps +------------- + +This section shows the steps of how to start VMs with vDPA vhost-user +backend and verify network connection & live migration. + +Build +~~~~~ + +To compile the sample application see :doc:`compiling`. + +The application is located in the ``vdpa`` sub-directory. + +Start the vdpa example +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + ./vdpa [EAL options] -- [--client] [--interactive|-i] or [--iface SOCKET_PATH] + +where + +* --client means running vdpa app in client mode, in the client mode, QEMU needs + to run as the server mode and take charge of socket file creation. +* --iface specifies the path prefix of the UNIX domain socket file, e.g. + /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n> + (n starts from 0). +* --interactive means run the vdpa sample in interactive mode, currently 4 + internal cmds are supported: + + 1. help: show help message + 2. list: list all available vdpa devices + 3. create: create a new vdpa port with socket file and vdpa device address + 4. quit: unregister vhost driver and exit the application + +Take IFCVF driver for example: + +.. code-block:: console + + ./vdpa -c 0x2 -n 4 --socket-mem 1024,1024 \ + -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \ + -- --interactive + +.. note:: + Here 0000:06:00.3 and 0000:06:00.4 refer to virtio ring compatible devices, + and we need to bind vfio-pci to them before running vdpa sample. + + * modprobe vfio-pci + * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4 + +Then we can create 2 vdpa ports in interactive cmdline. + +.. code-block:: console + + vdpa> list + device id device address queue num supported features + 0 0000:06:00.3 1 0x14c238020 + 1 0000:06:00.4 1 0x14c238020 + 2 0000:06:00.5 1 0x14c238020 + + vdpa> create /tmp/vdpa-socket0 0000:06:00.3 + vdpa> create /tmp/vdpa-socket1 0000:06:00.4 + +.. _vdpa_app_run_vm: + +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \ + +After the VMs launches, we can login the VMs and configure the ip, verify the +network connection via ping or netperf. + +.. note:: + Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA. + +Live Migration +~~~~~~~~~~~~~~ +vDPA supports cross-backend live migration, user can migrate SW vhost backend +VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is +the source host with SW vhost VM and B is the destination host with vDPA. + +1. Start vdpa sample and launch a VM with exact same parameters as the VM on A, + in migration-listen mode: + +.. code-block:: console + + B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT)) + +2. Start the migration (on source host): + +.. code-block:: console + + A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT) + +3. Check the status (on source host): + +.. code-block:: console + + A: (qemu) info migrateexamples/Makefile +1
-1@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endifexamples/vdpa/Makefile new file +32
-0@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endifexamples/vdpa/main.c new file +413
-0@@ -0,0 +1,462 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2018 Intel Corporation +*/ + +#include <getopt.h> +#include <signal.h> +#include <stdint.h> +#include <string.h> +#include <unistd.h> + +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_vhost.h> +#include <rte_vdpa.h> +#include <rte_pci.h> + +#include <cmdline_parse.h> +#include <cmdline_socket.h> +#include <cmdline_parse_string.h> +#include <cmdline.h> + +#define MAX_PATH_LEN 128 +#define MAX_VDPA_SAMPLE_PORTS 1024 +#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1 + +struct vdpa_port { +char ifname[MAX_PATH_LEN ]; +int did; +int vid; +uint64_t flags; +}; + +static struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS ]; + +static char iface[MAX_PATH_LEN ]; +static int dev_total; +static int devcnt; +static int interactive; +static int client_mode; + +/* display usage */ +static void +vdpa_usage (const char *prgname) +{ +printf ("Usage: %s [EAL options] -- " +" --interactive|-i: run in interactive mode.\n" +" --iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n" +" --client: register a vhost-user socket as client mode.\n" , + prgname); +} + +static int +parse_args (int argc,char **argv) +{ +static const char *short_option ="i" ; +static struct option long_option[] = { + {"iface" , required_argument,NULL ,0 }, + {"interactive" , no_argument, &interactive,1 }, + {"client" , no_argument, &client_mode,1 }, + {NULL ,0 ,0 ,0 }, + }; +int opt, idx; +char *prgname = argv[0 ]; + +while ((opt =getopt_long (argc, argv, short_option, long_option, &idx)) + !=EOF ) { +switch (opt) { +case 'i' : +printf ("Interactive-mode selected\n" ); + interactive =1 ; +break ; +/* long options */ +case 0 : +if (strncmp (long_option[idx].name,"iface" , +MAX_PATH_LEN ) ==0 ) { +strncpy (iface, optarg,MAX_PATH_LEN ); +printf ("iface %s\n" , iface); + } +if (!strcmp (long_option[idx].name,"interactive" )) { +printf ("Interactive-mode selected\n" ); + interactive =1 ; + } +break ; + +default : +vdpa_usage (prgname); +return -1 ; + } + } + +if (iface[0 ] =='\0' && interactive ==0 ) { +vdpa_usage (prgname); +return -1 ; + } + +return 0 ; +} + +static int +data_init (void ) +{ + dev_total =rte_vdpa_get_device_num (); +if (dev_total <=0 ) { +printf ("No available vdpa device found\n" ); +return -1 ; + } +return 0 ; +} + +static int +new_device (int vid) +{ +char ifname[MAX_PATH_LEN ]; +int i; + +rte_vhost_get_ifname (vid, ifname,sizeof (ifname)); +for (i =0 ; i <MAX_VDPA_SAMPLE_PORTS ; i++) { +if (strncmp (ifname, vports[i].ifname,MAX_PATH_LEN ) ==0 ) { +printf ("\nnew port %s, did: %d\n" , + ifname, vports[i].did); + vports[i].vid = vid; +break ; + } + } + +if (i >=MAX_VDPA_SAMPLE_PORTS ) +return -1 ; + +return 0 ; +} + +static void +destroy_device (int vid) +{ +char ifname[MAX_PATH_LEN ]; +int i; + +rte_vhost_get_ifname (vid, ifname,sizeof (ifname)); +for (i =0 ; i <MAX_VDPA_SAMPLE_PORTS ; i++) { +if (strcmp (ifname, vports[i].ifname) ==0 ) { +printf ("\ndestroy port %s, did: %d\n" , + ifname, vports[i].did); +break ; + } + } +} + +static const struct vhost_device_ops vdpa_sample_devops = { + .new_device = new_device, + .destroy_device = destroy_device, +}; + +static int +start_vdpa (struct vdpa_port *vport) +{ +int ret; +char *socket_path = vport->ifname; +int did = vport->did; + +if (client_mode) + vport->flags |=RTE_VHOST_USER_CLIENT ; + +if (access (socket_path,F_OK ) !=-1 && !client_mode) { +RTE_LOG (ERR ,VDPA , +"%s exists, please remove it or specify another file and try again.\n" , + socket_path); +return -1 ; + } + ret =rte_vhost_driver_register (socket_path, vport->flags); +if (ret !=0 ) +rte_exit (EXIT_FAILURE , +"register driver failed: %s\n" , + socket_path); + + ret =rte_vhost_driver_callback_register (socket_path, + &vdpa_sample_devops); +if (ret !=0 ) +rte_exit (EXIT_FAILURE , +"register driver ops failed: %s\n" , + socket_path); + + ret =rte_vhost_driver_attach_vdpa_device (socket_path, did); +if (ret !=0 ) +rte_exit (EXIT_FAILURE , +"attach vdpa device failed: %s\n" , + socket_path); + +if (rte_vhost_driver_start (socket_path) <0 ) +rte_exit (EXIT_FAILURE , +"start vhost driver failed: %s\n" , + socket_path); +return 0 ; +} + +static void +close_vdpa (struct vdpa_port *vport) +{ +int ret; +char *socket_path = vport->ifname; + + ret =rte_vhost_driver_detach_vdpa_device (socket_path); +if (ret !=0 ) +RTE_LOG (ERR ,VDPA , +"detach vdpa device failed: %s\n" , + socket_path); + + ret =rte_vhost_driver_unregister (socket_path); +if (ret !=0 ) +RTE_LOG (ERR ,VDPA , +"Fail to unregister vhost driver for %s.\n" , + socket_path); +} + +static void +vdpa_sample_quit (void ) +{ +int i; +for (i =0 ; i <RTE_MIN (MAX_VDPA_SAMPLE_PORTS , dev_total); i++) { +if (vports[i].ifname[0 ] !='\0' ) +close_vdpa (&vports[i]); + } +} + +static void +signal_handler (int signum) +{ +if (signum ==SIGINT || signum ==SIGTERM ) { +printf ("\nSignal %d received, preparing to exit...\n" , signum); +vdpa_sample_quit (); +exit (0 ); + } +} + +/* interactive cmds */ + +/* *** Help command with introduction. *** */ +struct cmd_help_result { +cmdline_fixed_string_t help; +}; + +static void cmd_help_parsed (__attribute__((unused))void *parsed_result, +struct cmdline *cl, + __attribute__((unused))void *data) +{ +cmdline_printf ( + cl, +"\n" +"The following commands are currently available:\n\n" +"Control:\n" +" help : Show interactive instructions.\n" +" list : list all available vdpa devices.\n" +" create <socket file> <vdev addr> : create a new vdpa port.\n" +" quit : exit vdpa sample app.\n" + ); +} + +cmdline_parse_token_string_t cmd_help_help = + TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help"); + +cmdline_parse_inst_t cmd_help = { + .f = cmd_help_parsed, + .data =NULL , + .help_str ="show help" , + .tokens = { + (void *)&cmd_help_help, +NULL , + }, +}; + +/* *** List all available vdpa devices *** */ +struct cmd_list_result { +cmdline_fixed_string_t action; +}; + +static void cmd_list_vdpa_devices_parsed ( + __attribute__((unused))void *parsed_result, +struct cmdline *cl, + __attribute__((unused))void *data) +{ +int did; +uint32_t queue_num; +uint64_t features; +struct rte_vdpa_device *vdev; +struct rte_pci_addr addr; + +cmdline_printf (cl,"device id\tdevice address\tqueue num\tsupported features\n" ); +for (did =0 ; did < dev_total; did++) { + vdev =rte_vdpa_get_device (did); +if (!vdev) +continue ; +if (vdev->ops->get_queue_num (did, &queue_num) <0 ) { +RTE_LOG (ERR ,VDPA , +"failed to get vdpa queue number " +"for device id %d.\n" , did); +continue ; + } +if (vdev->ops->get_features (did, &features) <0 ) { +RTE_LOG (ERR ,VDPA , +"failed to get vdpa features " +"for device id %d.\n" , did); +continue ; + } + addr = vdev->addr.pci_addr; +cmdline_printf (cl, +"%d\t\t" PCI_PRI_FMT "\t%" PRIu32"\t\t0x%" PRIx64"\n" , + did, addr.domain, addr.bus, addr.devid, + addr.function, queue_num, features); + } +} + +cmdline_parse_token_string_t cmd_action_list = + TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list"); + +cmdline_parse_inst_t cmd_list_vdpa_devices = { + .f = cmd_list_vdpa_devices_parsed, + .data =NULL , + .help_str ="list all available vdpa devices" , + .tokens = { + (void *)&cmd_action_list, +NULL , + }, +}; + +/* *** Create new vdpa port *** */ +struct cmd_create_result { +cmdline_fixed_string_t action; +cmdline_fixed_string_t socket_path; +cmdline_fixed_string_t bdf; +}; + +static void cmd_create_vdpa_port_parsed (void *parsed_result, +struct cmdline *cl, + __attribute__((unused))void *data) +{ +int did; +struct cmd_create_result *res = parsed_result; +struct rte_vdpa_dev_addr addr; + +strncpy (vports[devcnt].ifname, res->socket_path,MAX_PATH_LEN ); +if (rte_pci_addr_parse (res->bdf, &addr.pci_addr) !=0 ) { +cmdline_printf (cl,"Unable to parse the given bdf.\n" ); +return ; + } + addr.type =PCI_ADDR ; + did =rte_vdpa_find_device_id (&addr); +if (did <0 ) { +cmdline_printf (cl,"Unable to find vdpa devide id.\n" ); +return ; + } + + vports[devcnt].did = did; + +if (start_vdpa (&vports[devcnt]) ==0 ) + devcnt++; +} + +cmdline_parse_token_string_t cmd_action_create = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, "create"); +cmdline_parse_token_string_t cmd_socket_path = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, NULL); +cmdline_parse_token_string_t cmd_bdf = + TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL); + +cmdline_parse_inst_t cmd_create_vdpa_port = { + .f = cmd_create_vdpa_port_parsed, + .data =NULL , + .help_str ="create a new vdpa port" , + .tokens = { + (void *)&cmd_action_create, + (void *)&cmd_socket_path, + (void *)&cmd_bdf, +NULL , + }, +}; + +/* *** QUIT *** */ +struct cmd_quit_result { +cmdline_fixed_string_t quit; +}; + +static void cmd_quit_parsed (__attribute__((unused))void *parsed_result, +struct cmdline *cl, + __attribute__((unused))void *data) +{ +vdpa_sample_quit (); +cmdline_quit (cl); +} + +cmdline_parse_token_string_t cmd_quit_quit = + TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit"); + +cmdline_parse_inst_t cmd_quit = { + .f = cmd_quit_parsed, + .data =NULL , + .help_str ="quit: exit application" , + .tokens = { + (void *)&cmd_quit_quit, +NULL , + }, +}; +cmdline_parse_ctx_t main_ctx[] = { + (cmdline_parse_inst_t *)&cmd_help, + (cmdline_parse_inst_t *)&cmd_list_vdpa_devices, + (cmdline_parse_inst_t *)&cmd_create_vdpa_port, + (cmdline_parse_inst_t *)&cmd_quit, +NULL , +}; + +int +main (int argc,char *argv[]) +{ +char ch; +int i;
examples/vdpa/main.c line 423
Quoted 10 lines
examples/vdpa/main.c new file +10
-0+int ret; +struct cmdline *cl; + + ret =rte_eal_init (argc, argv); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"eal init failed\n" ); + argc -= ret; + argv += ret; + +data_init ();
You need to check return from data_init().
If this is fine for you, I can change to below code when applying:
ret = data_init();
if (ret < 0)
rte_exit(EXIT_FAILURE, "data init failed\n");
Quoted 63 lines
examples/vdpa/main.c new file +39
-0+ +signal (SIGINT , signal_handler); +signal (SIGTERM , signal_handler); + + ret =parse_args (argc, argv); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"invalid argument\n" ); + +if (interactive ==1 ) { + cl =cmdline_stdin_new (main_ctx,"vdpa> " ); +if (cl ==NULL ) +rte_panic ("Cannot create cmdline instance\n" ); +cmdline_interact (cl); +cmdline_stdin_exit (cl); + }else { +for (i =0 ; i <RTE_MIN (MAX_VDPA_SAMPLE_PORTS , dev_total); + i++) { + vports[i].did = i; +snprintf (vports[i].ifname,MAX_PATH_LEN ,"%s%d" , + iface, i); + +start_vdpa (&vports[i]); + } + +printf ("enter \'q\' to quit\n" ); +while (scanf ("%c" , &ch)) { +if (ch =='q' ) +break ; +while (ch !='\n' ) { +if (scanf ("%c" , &ch)) +printf ("%c" , ch); + } +printf ("enter \'q\' to quit\n" ); + } +vdpa_sample_quit (); + } + +return 0 ; +}examples/vdpa/meson.build new file +16
-0@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +)
Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 11 lines · Maxime Coquelin
examples/vdpa/main.c line 423
Quoted 7 lines
examples/vdpa/main.c new file +7
-0+ ret =rte_eal_init (argc, argv); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"eal init failed\n" ); + argc -= ret; + argv += ret; + +data_init ();You need to check return from
data_init().
If this is fine for you, I can change to below code when applying:
Quoted 5 lines
ret =
data_init();
if (ret < 0)
rte_exit(EXIT_FAILURE, "data init failed\n");
I'm fine with this change.
Thanks,
Xiaolong
Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 12 lines · Ye Xiaolong
Quoted 11 lines · Maxime Coquelin
examples/vdpa/main.c line 423
Quoted 7 lines
examples/vdpa/main.c new file +7
-0+ ret =rte_eal_init (argc, argv); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"eal init failed\n" ); + argc -= ret; + argv += ret; + +data_init ();You need to check return from
data_init().
If this is fine for you, I can change to below code when applying:
Quoted 7 lines
Quoted 5 lines
ret =
data_init();
if (ret < 0)rte_exit(EXIT_FAILURE, "data init failed\n");I'm fine with this change.
Or even better, get rid of data_init and
call rte_vdpa_get_device_num(); in main directly.
Quoted 3 lines
Thanks,
Xiaolong
Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 21 lines · Maxime Coquelin
Quoted 18 lines · Ye Xiaolong
Quoted 16 lines · Maxime Coquelin
examples/vdpa/main.c line 423
Quoted 7 lines
examples/vdpa/main.c new file +7
-0+ ret =rte_eal_init (argc, argv); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"eal init failed\n" ); + argc -= ret; + argv += ret; + +data_init ();You need to check return from
data_init().
If this is fine for you, I can change to below code when applying:ret =
data_init();
if (ret < 0)rte_exit(EXIT_FAILURE, "data init failed\n");
Quoted 4 lines
Quoted 1 line
I'm fine with this change.
Or even better, get rid of
data_initand
callrte_vdpa_get_device_num(); in main directly.
Sounds better, as we've removed unnecessary static variable initialization in data_init, it does nothing but call rte_vdpa_get_device_num, we can remove this wrap. Would you do this change when applying or you perfer I send a new revision?
Thanks,
Xiaolong
Quoted 4 lines
Quoted 4 lines
Thanks,
Xiaolong
Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 24 lines · Ye Xiaolong
Quoted 23 lines · Maxime Coquelin
Quoted 19 lines · Ye Xiaolong
Quoted 11 lines · Maxime Coquelin
examples/vdpa/main.c line 423
Quoted 7 lines
examples/vdpa/main.c new file +7
-0+ ret =rte_eal_init (argc, argv); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"eal init failed\n" ); + argc -= ret; + argv += ret; + +data_init ();You need to check return from
data_init().
If this is fine for you, I can change to below code when applying:Quoted 5 lines
ret =
data_init();
if (ret < 0)rte_exit(EXIT_FAILURE, "data init failed\n");I'm fine with this change.
Quoted 7 lines
Quoted 2 lines
Or even better, get rid of
data_initand
callrte_vdpa_get_device_num(); in main directly.Sounds better, as we've removed unnecessary static variable initialization in
data_init, it does nothing but callrte_vdpa_get_device_num, we can remove this wrap. Would you do this change when applying or you perfer I send a new revision?
Please send a new revision, and in the commit message, fix first letter of my family name with an upper case, and move my R-b below your sign-off.
Thanks!
Maxime
Quoted 8 lines
Thanks,
XiaolongQuoted 4 lines
Quoted 4 lines
Thanks,
Xiaolong
Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 30 lines · Maxime Coquelin
Quoted 27 lines · Ye Xiaolong
Quoted 25 lines · Maxime Coquelin
Quoted 12 lines · Ye Xiaolong
Quoted 11 lines · Maxime Coquelin
examples/vdpa/main.c line 423
Quoted 7 lines
examples/vdpa/main.c new file +7
-0+ ret =rte_eal_init (argc, argv); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"eal init failed\n" ); + argc -= ret; + argv += ret; + +data_init ();You need to check return from
data_init().
If this is fine for you, I can change to below code when applying:Quoted 7 lines
Quoted 5 lines
ret =
data_init();
if (ret < 0)rte_exit(EXIT_FAILURE, "data init failed\n");I'm fine with this change.
Or even better, get rid of
data_initand
callrte_vdpa_get_device_num(); in main directly.
Quoted 8 lines
Quoted 4 lines
Sounds better, as we've removed unnecessary static variable initialization in
data_init, it does nothing but callrte_vdpa_get_device_num, we can remove this wrap. Would you do this change when applying or you perfer I send a new revision?Please send a new revision, and in the commit message, fix first letter of my family name with an upper case, and move my R-b below your sign-off.
Got it.
Thanks,
Xiaolong
Quoted 12 lines
Thanks!
MaximeQuoted 8 lines
Thanks,
XiaolongQuoted 4 lines
Quoted 4 lines
Thanks,
Xiaolong
v8 cover introduce vdpa sample
In reply to Xiaolong YeHi,
This patchset introduces vdpa sample to demonstrate the vDPA use case.
v8 changes:
* move body of data_init to the main function
* adjust Reviewed-by/Acked-by tag postion
v7 changes:
* make vports static
* avoid unnecessary static variable initialization
v6 changes:
* improve the document according to Xiao's comments
* fix a typo, PRIu64 -> PRIx64
v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst
v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.
v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments
v2 changes:
* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list, quit
* add application documentation
Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA
MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 453 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 11 files changed, 643 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
PATCH v8 1/2 vhost: introduce API to get vDPA device number
It's used to get number of available registered vDPA devices.
--- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimentalrte_vdpa_get_device (int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num (void);#endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
@@ -67,6 +67,7 @@ EXPERIMENTAL { rte_vdpa_unregister_device; rte_vdpa_find_device_id; rte_vdpa_get_device; + rte_vdpa_get_device_num; rte_vhost_driver_attach_vdpa_device; rte_vhost_driver_detach_vdpa_device; rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)return vdpa_devices[did]; } + +int +rte_vdpa_get_device_num (void ) +{ +return vdpa_device_num; +}
Signature · 1 line
2.17.1
Re: [PATCH v8 1/2] vhost: introduce API to get vDPA device number
Quoted 16 lines · Xiaolong Ye
It's used to get number of available registered vDPA devices.
Signed-off-by Xiaolong Ye <[email protected]>Acked-by Xiao Wang <[email protected]>Reviewed-by Maxime Coquelin <[email protected]>--- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+)lib/librte_vhost/rte_vdpa.h +0
-0
lib/librte_vhost/rte_vdpa.h rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); · line 89
Quoted 6 lines
lib/librte_vhost/rte_vdpa.h +3
-0@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimentalrte_vdpa_get_device (int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num (void);
Hi Xiaolong,
Is rte_vdpa_get_device_num() an public API? (It should be since it will be used in an application in next patch.)
If so, header needs to be part of API documentation:
1- Update doc/api/doxy-api-index.md to include this header. 2- Update all APIs comments to be an doxygen comment. 3- Add doxygen comment to all data structures.
I can see rte_vdpa.h not documented at all, it is not just this API, so it can be better to send a new patch to address above items.
Thanks,
ferruh
Re: [PATCH v8 1/2] vhost: introduce API to get vDPA device number
Hi, Ferruh,
Quoted 29 lines · Ferruh Yigit
Quoted 16 lines · Xiaolong Ye
It's used to get number of available registered vDPA devices.
Signed-off-by Xiaolong Ye <[email protected]>Acked-by Xiao Wang <[email protected]>Reviewed-by Maxime Coquelin <[email protected]>--- lib/librte_vhost/rte_vdpa.h | 3 +++ lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 ++++++ 3 files changed, 10 insertions(+)lib/librte_vhost/rte_vdpa.h +0
-0lib/librte_vhost/rte_vdpa.h rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); · line 89
Quoted 6 lines
lib/librte_vhost/rte_vdpa.h +3
-0@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); struct rte_vdpa_device * __rte_experimentalrte_vdpa_get_device (int did); +/* Get current available vdpa device number */ +int __rte_experimental +rte_vdpa_get_device_num (void);Hi Xiaolong,
Is
rte_vdpa_get_device_num()an public API? (It should be since it will be used in an application in next patch.)
Quoted 7 lines
If so, header needs to be part of API documentation:
1- Updatedoc/api/doxy-api-index.mdto include this header. 2- Update all APIs comments to be an doxygen comment. 3- Add doxygen comment to all data structures.I can see
rte_vdpa.hnot documented at all, it is not just this API, so it can be better to send a new patch to address above items.
Thanks for the reminder, I'll send a new patch for it.
Thanks,
Xiaolong
Quoted 3 lines
Thanks,
ferruh
PATCH v8 2/2 examples/vdpa: introduce a new sample for vDPA
In reply to Xiaolong YeThe vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
--- MAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 453 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + 8 files changed, 633 insertions(+), 1 deletion(-) create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst F: examples/vhost_scsi/ F: doc/guides/sample_app_ug/vhost_scsi.rst F: examples/vhost_crypto/ +F: examples/vdpa/ +F: doc/guides/sample_app_ug/vdpa.rst Vhost PMD M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
@@ -67,6 +67,14 @@ New Features SR-IOV option in Hyper-V and Azure. This is an alternative to the previous vdev_netvsc, tap, and failsafe drivers combination. +* **Add a new sample for vDPA** + + The vdpa sample application creates vhost-user sockets by using the + vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes + virtio ring compatible devices to serve virtio driver directly to enable + datapath acceleration. As vDPA driver can help to set up vhost datapath, + this application doesn't need to launch dedicated worker threads for vhost + enqueue/dequeue operations. API Changes -----------
doc/guides/sample_app_ug/index.rst +1 -0
@@ -45,6 +45,7 @@ Sample Applications User Guides vhost vhost_scsi vhost_crypto + vdpa netmap_compatibility ip_pipeline test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +120 -0
120 lines not on this page · raw message
examples/Makefile +1 -1
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination endif DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer -DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi +DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y) DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto endif
examples/vdpa/Makefile new file +32 -0
@@ -0,0 +1,32 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp") +$(info This application can only operate in a linuxapp environment, \ +please change the definition of the RTE_TARGET environment variable) +all: +else + +# binary name +APP = vdpa + +# all source are stored in SRCS-y +SRCS-y := main.c + +CFLAGS += -O2 -D_FILE_OFFSET_BITS=64 +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -D_GNU_SOURCE +CFLAGS += -DALLOW_EXPERIMENTAL_API + +include $(RTE_SDK)/mk/rte.extapp.mk + +endif
examples/vdpa/main.c new file +453 -0
453 lines not on this page · raw message
examples/vdpa/meson.build new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Intel Corporation + +# meson file, for building this example as part of a main DPDK build. +# +# To build this example as a standalone application with an already-installed +# DPDK instance, use 'make' + +if host_machine.system() != 'linux' + build = false +endif +deps += 'vhost' +allow_experimental_apis = true +sources = files( + 'main.c' +)
Signature · 1 line
2.17.1
Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 10 lines · Xiaolong Ye
The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiaolong Ye <[email protected]>Acked-by Xiao Wang <[email protected]>Reviewed-by Maxime Coquelin <[email protected]>
<...>
Quoted 15 lines
examples/vdpa/main.c new file +15
-0+static int +parse_args (int argc,char **argv) +{ +static const char *short_option ="i" ; +static struct option long_option[] = { + {"iface" , required_argument,NULL ,0 }, + {"interactive" , no_argument, &interactive,1 }, + {"client" , no_argument, &client_mode,1 }, + {NULL ,0 ,0 ,0 }, + }; +int opt, idx; +char *prgname = argv[0 ]; + +while ((opt =getopt_long (argc, argv, short_option, long_option, &idx)) + !=EOF ) {
examples/vdpa/main.c line 76
Quoted 10 lines
examples/vdpa/main.c new file +10
-0+switch (opt) { +case 'i' : +printf ("Interactive-mode selected\n" ); + interactive =1 ; +break ; +/* long options */ +case 0 : +if (strncmp (long_option[idx].name,"iface" , +MAX_PATH_LEN ) ==0 ) { +strncpy (iface, optarg,MAX_PATH_LEN );
Giving compiler warning [1], rte_strscpy() is safer to use.
[1]
In function ‘parse_args’,
inlined from ‘main’ at .../examples/vdpa/main.c:419:8:
.../examples/vdpa/main.c:76:5: error: ‘strncpy’ specified bound 128 equals destination size [-Werror=stringop-truncation]
strncpy(iface, optarg, MAX_PATH_LEN);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA
Hi, Ferruh,
Quoted 55 lines · Ferruh Yigit
Quoted 10 lines · Xiaolong Ye
The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiaolong Ye <[email protected]>Acked-by Xiao Wang <[email protected]>Reviewed-by Maxime Coquelin <[email protected]><...>
Quoted 15 lines
examples/vdpa/main.c new file +15
-0+static int +parse_args (int argc,char **argv) +{ +static const char *short_option ="i" ; +static struct option long_option[] = { + {"iface" , required_argument,NULL ,0 }, + {"interactive" , no_argument, &interactive,1 }, + {"client" , no_argument, &client_mode,1 }, + {NULL ,0 ,0 ,0 }, + }; +int opt, idx; +char *prgname = argv[0 ]; + +while ((opt =getopt_long (argc, argv, short_option, long_option, &idx)) + !=EOF ) {examples/vdpa/main.c line 76
Quoted 10 lines
examples/vdpa/main.c new file +10
-0+switch (opt) { +case 'i' : +printf ("Interactive-mode selected\n" ); + interactive =1 ; +break ; +/* long options */ +case 0 : +if (strncmp (long_option[idx].name,"iface" , +MAX_PATH_LEN ) ==0 ) { +strncpy (iface, optarg,MAX_PATH_LEN );Giving compiler warning [1],
rte_strscpy()is safer to use.[1]
In function ‘parse_args’,inlined from ‘main’ at
.../examples/vdpa/main.c:419:8:
.../examples/vdpa/main.c:76:5: error: ‘strncpy’ specified bound 128 equals destination size [-Werror=stringop-truncation]
Quoted 5 lines
strncpy(iface, optarg, MAX_PATH_LEN);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Thanks for the finding, by the way, how do you get this warning? I can't see it in my local build, what compiler you used?
Thanks,
Xiaolong
Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA
Quoted 58 lines · Ye Xiaolong
Hi, Ferruh,
Quoted 55 lines · Ferruh Yigit
Quoted 10 lines · Xiaolong Ye
The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiaolong Ye <[email protected]>Acked-by Xiao Wang <[email protected]>Reviewed-by Maxime Coquelin <[email protected]><...>
Quoted 15 lines
examples/vdpa/main.c new file +15
-0+static int +parse_args (int argc,char **argv) +{ +static const char *short_option ="i" ; +static struct option long_option[] = { + {"iface" , required_argument,NULL ,0 }, + {"interactive" , no_argument, &interactive,1 }, + {"client" , no_argument, &client_mode,1 }, + {NULL ,0 ,0 ,0 }, + }; +int opt, idx; +char *prgname = argv[0 ]; + +while ((opt =getopt_long (argc, argv, short_option, long_option, &idx)) + !=EOF ) {examples/vdpa/main.c line 76
Quoted 10 lines
examples/vdpa/main.c new file +10
-0+switch (opt) { +case 'i' : +printf ("Interactive-mode selected\n" ); + interactive =1 ; +break ; +/* long options */ +case 0 : +if (strncmp (long_option[idx].name,"iface" , +MAX_PATH_LEN ) ==0 ) { +strncpy (iface, optarg,MAX_PATH_LEN );Giving compiler warning [1],
rte_strscpy()is safer to use.[1]
In function ‘parse_args’,inlined from ‘main’ at
.../examples/vdpa/main.c:419:8:
.../examples/vdpa/main.c:76:5: error: ‘strncpy’ specified bound 128 equals destination size [-Werror=stringop-truncation]
Quoted 8 lines
Quoted 5 lines
strncpy(iface, optarg, MAX_PATH_LEN); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Thanks for the finding, by the way, how do you get this warning? I can't see it in my local build, what compiler you used?
gcc,
$ gcc --version gcc (GCC) 8.1.1 20180712 (Red Hat 8.1.1-5)
Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA
In reply to Xiaolong YeHi,
Going back to this merged commit,
Saturday, September 29, 2018 12:48 AM, Xiaolong Ye:
Quoted 14 lines
Subject: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA
The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiaolong Ye <[email protected]>Acked-by Xiao Wang <[email protected]>Reviewed-by Maxime Coquelin <[email protected]>---
[...]
doc/guides/sample_app_ug/vdpa.rst
Quoted 13 lines
doc/guides/sample_app_ug/vdpa.rst new file +13
-0+ +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device + virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \
It doesn't seems like qemu has the id=vdpa and netdev=vdpa options. Grepping vdpa on qemu returns empty.
Am using QEMU v3.1.0-513-gfb177a0. Commit head is fb177a036f2e ("MAINTAINERS: Mark RISC-V as Supported").
Did I missed anything?
Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA
Hi,
Quoted 40 lines · Shahaf Shuler
Hi,
Going back to this merged commit,
Saturday, September 29, 2018 12:48 AM, Xiaolong Ye:
Quoted 14 lines
Subject: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA
The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiaolong Ye <[email protected]>Acked-by Xiao Wang <[email protected]>Reviewed-by Maxime Coquelin <[email protected]>---
[...]
doc/guides/sample_app_ug/vdpa.rst
Quoted 13 lines
doc/guides/sample_app_ug/vdpa.rst new file +13
-0+ +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps> \ + -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device + virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \It doesn't seems like qemu has the id=vdpa and netdev=vdpa options. Grepping vdpa on qemu returns empty.
Quoted 4 lines
Am using QEMU v3.1.0-513-gfb177a0. Commit head is
fb177a036f2e("MAINTAINERS: Mark RISC-V as Supported").Did I missed anything?
Not quite sure about your question here, the vdpa used in above qemu cmdline is just a normal string, you can replace it with other string like
-netdev type=vhost-user,id=mynet,chardev=char0 \
-device virtio-net-pci,netdev=mynet,...
The major support for vdpa is done in dpdk side as the vhost user backend, qemu will use normal vhost user protocol to communicate with it.
Thanks,
Xiaolong
Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA
Thursday, January 17, 2019 5:13 PM, Ye Xiaolong:
Quoted 53 lines
Subject: Re: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA
Hi,
Quoted 22 lines · Shahaf Shuler
Hi,
Going back to this merged commit,
Saturday, September 29, 2018 12:48 AM, Xiaolong Ye:
Quoted 14 lines
Subject: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA
The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.
Signed-off-by Xiaolong Ye <[email protected]>Acked-by Xiao Wang <[email protected]>Reviewed-by Maxime Coquelin <[email protected]>---
[...]
Quoted 10 lines
doc/guides/sample_app_ug/vdpa.rst
Quoted 10 lines
doc/guides/sample_app_ug/vdpa.rst new file +10
-0+ +Start the VMs +~~~~~~~~~~~~~ + +.. code-block:: console + + qemu-system-x86_64 -cpu host -enable-kvm \ + <snip> + -mem-prealloc \ + -chardev socket,id=char0,path=<socket_file created in above steps>\
Quoted 9 lines
Review
Quoted 3 lines
(quoted diff) +3
-0+ -netdev type=vhost-user,id=vdpa,chardev=char0 \ + -device + virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \It doesn't seems like qemu has the id=vdpa and netdev=vdpa options. Grepping vdpa on qemu returns empty.
Am using QEMU v3.1.0-513-gfb177a0. Commit head is
fb177a036f2e("
MAINTAINERS: Mark RISC-V as Supported").Quoted 3 lines
Did I missed anything?
Quoted 8 lines
Not quite sure about your question here, the vdpa used in above qemu cmdline is just a normal string, you can replace it with other string like
-netdev type=vhost-user,id=mynet,chardev=char0 \ -device virtio-net- pci,netdev=mynet,...
The major support for vdpa is done in dpdk side as the vhost user backend, qemu will use normal vhost user protocol to communicate with it.
Thanks. I mis-interpreted the option fields.
Quoted 3 lines
Thanks,
Xiaolong
Maxime Coquelin
AppliedQuoted 56 lines · Xiaolong Ye
Hi,
This patchset introduces vdpa sample to demonstrate the vDPA use case.
v8 changes:
* move body ofdata_initto the main function
* adjust Reviewed-by/Acked-by tag postionv7 changes:
* make vports static
* avoid unnecessary static variable initializationv6 changes:
* improve the document according to Xiao's comments
* fix a typo, PRIu64 -> PRIx64v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations invdpa.rstv4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve thevdpa.rstaccordingly
* remove some useless comments
* add introduction in 18.11 release note.v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review commentsv2 changes:
* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list, quit
* add application documentationXiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPAMAINTAINERS | 2 + doc/guides/rel_notes/release_18_11.rst | 8 + doc/guides/sample_app_ug/index.rst | 1 + doc/guides/sample_app_ug/vdpa.rst | 120 +++++++ examples/Makefile | 2 +- examples/vdpa/Makefile | 32 ++ examples/vdpa/main.c | 453 +++++++++++++++++++++++++ examples/vdpa/meson.build | 16 + lib/librte_vhost/rte_vdpa.h | 3 + lib/librte_vhost/rte_vhost_version.map | 1 + lib/librte_vhost/vdpa.c | 6 + 11 files changed, 643 insertions(+), 1 deletion(-)
Quoted 5 lines
create mode 100644
doc/guides/sample_app_ug/vdpa.rstcreate mode 100644 examples/vdpa/Makefile create mode 100644examples/vdpa/main.ccreate mode 100644 examples/vdpa/meson.build
Applied to dpdk-next-virtio/master.
Thanks,
Maxime