Skip to contentSkip to conversation

introduce vdpa sample

PATCH Superseded Xiaolong Ye 5 participants 41 messages Sep 20, 2018 – Jan 17, 2019 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

v3 cover introduce vdpa sample

Hi,

This patchset introduces vdpa sample to demonstrate the vDPA use case.

v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments

v2 changes:

* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list,
* quit
* add application documentation

Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA

 MAINTAINERS                            |   2 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 115 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 458 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 lib/librte_vhost/rte_vdpa.h            |   3 +
 lib/librte_vhost/rte_vhost_version.map |   1 +
 lib/librte_vhost/vdpa.c                |   6 +
 10 files changed, 635 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
#1 lore raw

PATCH v3 1/2 vhost: introduce API to get vDPA device number

---
 lib/librte_vhost/rte_vdpa.h            | 3 +++
 lib/librte_vhost/rte_vhost_version.map | 1 +
 lib/librte_vhost/vdpa.c                | 6 ++++++
 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
index 90465ca26..b8223e337 100644
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr);
 struct rte_vdpa_device * __rte_experimental
 rte_vdpa_get_device(int did);
 
+/* Get current available vdpa device number */
+int __rte_experimental
+rte_vdpa_get_device_num(void);
 #endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
index da220dd02..ae39b6e21 100644
@@ -67,6 +67,7 @@ EXPERIMENTAL {
 	rte_vdpa_unregister_device;
 	rte_vdpa_find_device_id;
 	rte_vdpa_get_device;
+	rte_vdpa_get_device_num;
 	rte_vhost_driver_attach_vdpa_device;
 	rte_vhost_driver_detach_vdpa_device;
 	rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
index c82fd4370..c2c5dff1d 100644
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)
 
 	return vdpa_devices[did];
 }
+
+int
+rte_vdpa_get_device_num(void)
+{
+	return vdpa_device_num;
+}
Signature · 1 line
2.17.1
#2 lore raw

PATCH v3 2/2 examples/vdpa: introduce a new sample for vDPA

In reply to Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
 MAINTAINERS                        |   2 +
 doc/guides/sample_app_ug/index.rst |   1 +
 doc/guides/sample_app_ug/vdpa.rst  | 115 ++++++++
 examples/Makefile                  |   2 +-
 examples/vdpa/Makefile             |  32 ++
 examples/vdpa/main.c               | 458 +++++++++++++++++++++++++++++
 examples/vdpa/meson.build          |  16 +
 7 files changed, 625 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
 F: examples/vhost_scsi/
 F: doc/guides/sample_app_ug/vhost_scsi.rst
 F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
 Vhost PMD
 M: Maxime Coquelin <[email protected]>
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
     vhost
     vhost_scsi
     vhost_crypto
+    vdpa
     netmap_compatibility
     ip_pipeline
     test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +115 -0

115 lines not on this page · raw message

examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
 endif
examples/vdpa/Makefile new file +32 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
examples/vdpa/main.c new file +458 -0

458 lines not on this page · raw message

examples/vdpa/meson.build new file +16 -0
new file mode 100644
index 000000000..2e38a069f
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if host_machine.system() != 'linux'
+	build = false
+endif
+deps += 'vhost'
+allow_experimental_apis = true
+sources = files(
+	'main.c'
+)
\ No newline at end of file
Signature · 1 line
2.17.1
#3 lore raw

Re: [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA

Hi Xiaolong,

Quoted 80 lines

-----Original Message-----
From: Ye, Xiaolong
Sent: Friday, September 21, 2018 6:28 AM
To: [email protected]; Maxime Coquelin <[email protected]>; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>
Cc: Wang, Xiao W <[email protected]>; Rami Rosen <[email protected]>; Wang, Haiyue <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
 MAINTAINERS                        |   2 +
 doc/guides/sample_app_ug/index.rst |   1 +
 doc/guides/sample_app_ug/vdpa.rst  | 115 ++++++++
 examples/Makefile                  |   2 +-
 examples/vdpa/Makefile             |  32 ++
 examples/vdpa/main.c               | 458 +++++++++++++++++++++++++++++
 examples/vdpa/meson.build          |  16 +
 7 files changed, 625 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
 F: examples/vhost_scsi/
 F: doc/guides/sample_app_ug/vhost_scsi.rst
 F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
 Vhost PMD
 M: Maxime Coquelin <[email protected]>
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
     vhost
     vhost_scsi
     vhost_crypto
+    vdpa
     netmap_compatibility
     ip_pipeline
     test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +14 -0
new file mode 100644
index 000000000..44fe6736d
@@ -0,0 +1,115 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Vdpa Sample Application
+=======================
+
+The vdpa sample application creates vhost-user sockets by using the
+vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+virtio ring compatible devices to serve virtio driver directly to enable
+datapath acceleration. As vDPA driver can help to set up vhost datapath,
+this application doesn't need to launch dedicated worker threads for vhost
+enqueue/dequeue operations.
+
+Testing steps

[...]

doc/guides/sample_app_ug/vdpa.rst line 68

Quoted 9 lines
doc/guides/sample_app_ug/vdpa.rst new file +9 -0
+
+Then we can create 2 vdpa ports in interactive cmdline.
+
+.. code-block:: console
+
+        vdpa> list
+        device id       device address
+        0               0000:06:00.2
+        1               0000:06:00.3

The features and queue numbers of each vDPA device could also be shown now, the doc should reflect this.

Quoted 100 lines
doc/guides/sample_app_ug/vdpa.rst new file +26 -0
+        vdpa> create /tmp/vdpa-socket0 0000:06:00.2
+        vdpa> create /tmp/vdpa-socket1 0000:06:00.3
+
+.. _vdpa_app_run_vm:
+
+Start the VMs
+~~~~~~~~~~~~~
+
+.. code-block:: console
+
+       qemu-system-x86_64 -cpu host -enable-kvm \
+       <snip>
+       -mem-prealloc \
+       -chardev socket,id=char0,path=<socket_file created in above steps> \
+       -netdev type=vhost-user,id=vdpa,chardev=char0 \
+       -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \
+
+After the VMs launches, we can login the VMs and configure the ip, verify the
+network connection via ping or netperf.
+
+.. note::
+    Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA.
+
+Live Migration
+~~~~~~~~~~~~~~
+vDPA supports cross-backend live migration, user can migrate SW vhost

backend
+VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is

doc/guides/sample_app_ug/vdpa.rst new file +3 -0
+the source host with SW vhost VM and B is the destination host with vDPA.
+
+1. Start vdpa sample and launch a VM with exact same parameters as the VM

on A,

doc/guides/sample_app_ug/vdpa.rst new file +17 -0
+   in migration-listen mode:
+
+.. code-block:: console
+
+        B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT))
+
+2. Start the migration (on source host):
+
+.. code-block:: console
+
+        A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT)
+
+3. Check the status (on source host):
+
+.. code-block:: console
+
+        A: (qemu) info migrate
examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
 endif
examples/vdpa/Makefile new file +31 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+

[...]

Quoted 55 lines
examples/vdpa/Makefile new file +54 -0
+
+static int
+new_device(int vid)
+{
+	char ifname[MAX_PATH_LEN];
+	int i;
+
+	rte_vhost_get_ifname(vid, ifname, sizeof(ifname));
+	for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) {
+		if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) {
+			printf("\nnew port %s, did: %d\n",
+					ifname, vports[i].did);
+			vports[i].vid = vid;
+			break;
+		}
+	}
+
+	if (i >= MAX_VDPA_SAMPLE_PORTS)
+		return -1;
+
+	return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+	char ifname[MAX_PATH_LEN];
+	int i;
+
+	rte_vhost_get_ifname(vid, ifname, sizeof(ifname));
+	for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) {
+		if (strcmp(ifname, vports[i].ifname) == 0) {
+			printf("\ndestroy port %s, did: %d\n",
+					ifname, vports[i].did);
+			break;
+		}
+	}
+}
+
+static const struct vhost_device_ops vdpa_sample_devops = {
+	.new_device = new_device,
+	.destroy_device = destroy_device,
+};
+
+static int
+start_vdpa(struct vdpa_port *vport)
+{
+	int ret;
+	char *socket_path = vport->ifname;
+	int did = vport->did;
+
+	if (access(socket_path, F_OK) != -1) {
+		RTE_LOG(ERR, VDPA,
+			"%s exists, please remove it or specify another file and

try again.\n",

examples/vdpa/Makefile

Quoted 4 lines
examples/vdpa/Makefile new file +4 -0
+			socket_path);
+		return -1;
+	}
+	ret = rte_vhost_driver_register(socket_path, vport->flags);

It would be great if we can provide "client mode" option for this sample to create a client mode vdpa.

Quoted 129 lines
examples/vdpa/Makefile new file +83 -0
+	if (ret != 0)
+		rte_exit(EXIT_FAILURE,
+			"register driver failed: %s\n",
+			socket_path);
+
+	ret = rte_vhost_driver_callback_register(socket_path,
+			&vdpa_sample_devops);
+	if (ret != 0)
+		rte_exit(EXIT_FAILURE,
+			"register driver ops failed: %s\n",
+			socket_path);
+
+	ret = rte_vhost_driver_attach_vdpa_device(socket_path, did);
+	if (ret != 0)
+		rte_exit(EXIT_FAILURE,
+			"attach vdpa device failed: %s\n",
+			socket_path);
+
+	if (rte_vhost_driver_start(socket_path) < 0)
+		rte_exit(EXIT_FAILURE,
+			"start vhost driver failed: %s\n",
+			socket_path);
+	return 0;
+}
+
+static void
+close_vdpa(struct vdpa_port *vport)
+{
+	int ret;
+	char *socket_path = vport->ifname;
+
+	ret = rte_vhost_driver_detach_vdpa_device(socket_path);
+	if (ret != 0)
+		RTE_LOG(ERR, VDPA,
+				"attach vdpa device failed: %s\n",
+				socket_path);
+
+	ret = rte_vhost_driver_unregister(socket_path);
+	if (ret != 0)
+		RTE_LOG(ERR, VDPA,
+				"Fail to unregister vhost driver for %s.\n",
+				socket_path);
+}
+
+static void
+vdpa_sample_quit(void)
+{
+	int i;
+	for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) {
+		if (vports[i].ifname[0] != '\0')
+			close_vdpa(&vports[i]);
+	}
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\nSignal %d received, preparing to exit...\n", signum);
+		vdpa_sample_quit();
+		exit(0);
+	}
+}
+
+/* interactive cmds */
+
+/* *** Help command with introduction. *** */
+struct cmd_help_result {
+	cmdline_fixed_string_t help;
+};
+
+static void cmd_help_parsed(__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	cmdline_printf(
+		cl,
+		"\n"
+		"The following commands are currently available:\n\n"
+		"Control:\n"
+		"    help                                      : Show interactive instructions.\n"
+		"    list                                      : list all available vdpa devices.\n"
+		"    create <socket file> <vdev addr>          : create a new vdpa

port.\n"

examples/vdpa/Makefile new file +34 -0
+		"    quit                                      : exit vdpa sample app.\n"
+	);
+}
+
+cmdline_parse_token_string_t cmd_help_help =
+	TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help");
+
+cmdline_parse_inst_t cmd_help = {
+	.f = cmd_help_parsed,
+	.data = NULL,
+	.help_str = "show help",
+	.tokens = {
+		(void *)&cmd_help_help,
+		NULL,
+	},
+};
+
+/* *** List all available vdpa devices *** */
+struct cmd_list_result {
+	cmdline_fixed_string_t action;
+};
+
+static void cmd_list_vdpa_devices_parsed(
+		__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	int did;
+	uint32_t queue_num;
+	uint64_t features;
+	struct rte_vdpa_device *vdev;
+	struct rte_pci_addr addr;
+
+	cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported

features\n");

examples/vdpa/Makefile new file +10 -0
+	for (did = 0; did < dev_total; did++) {
+		vdev = rte_vdpa_get_device(did);
+		if (!vdev)
+			continue;
+		if (vdev->ops->get_queue_num(did, &queue_num) < 0) {
+			RTE_LOG(ERR, VDPA,
+				"failed to get vdpa queue number "
+				"for device id %d.\n", did);
+			continue;
+		}

examples/vdpa/Makefile

Quoted 10 lines
examples/vdpa/Makefile new file +10 -0
+		if (vdev->ops->get_features(did, &features) < 0) {
+			RTE_LOG(ERR, VDPA,
+				"failed to get vdpa features "
+				"for device id %d.\n", did);
+			continue;
+		}
+		addr = vdev->addr.pci_addr;
+		cmdline_printf(cl, "%d\t\t"PCI_PRI_FMT"\t%d\t\t0x%lx\n", did,
+				addr.domain, addr.bus, addr.devid,
+				addr.function, queue_num, features);

Use "%PRIu32"for queue_num, and use "PRIx64" instead of "%lx", otherwise you may have compile error on 32-bit machine.

examples/vdpa/Makefile

Quoted 9 lines
examples/vdpa/Makefile new file +9 -0
+	}
+}
+
+cmdline_parse_token_string_t cmd_action_list =
+	TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list");
+
+cmdline_parse_inst_t cmd_list_vdpa_devices = {
+	.f = cmd_list_vdpa_devices_parsed,  /* function to call */
+	.data = NULL,      /* 2nd arg of func */

This comment doesn't provide much info.
Remove the comment for all the cmdline_parse_inst_t instances to keep alignment.

Quoted 133 lines
examples/vdpa/Makefile new file +42 -0
+	.help_str = "list all available vdpa devices",
+	.tokens = {        /* token list, NULL terminated */
+		(void *)&cmd_action_list,
+		NULL,
+	},
+};
+
+/* *** Create new vdpa port *** */
+struct cmd_create_result {
+	cmdline_fixed_string_t action;
+	cmdline_fixed_string_t socket_path;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_create_vdpa_port_parsed(void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	int did;
+	struct cmd_create_result *res = parsed_result;
+	struct rte_vdpa_dev_addr addr;
+
+	strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN);
+	if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) {
+		cmdline_printf(cl, "Unable to parse the given bdf.\n");
+		return;
+	}
+	addr.type = PCI_ADDR;
+	did = rte_vdpa_find_device_id(&addr);
+	if (did < 0) {
+		cmdline_printf(cl, "Unable to find vdpa devide id.\n");
+		return;
+	}
+
+	vports[devcnt].did = did;
+
+	if (start_vdpa(&vports[devcnt]) == 0)
+		devcnt++;
+}
+
+cmdline_parse_token_string_t cmd_action_create =
+	TOKEN_STRING_INITIALIZER(struct cmd_create_result, action,

"create");

examples/vdpa/Makefile new file +2 -0
+cmdline_parse_token_string_t cmd_socket_path =
+	TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path,

NULL);

examples/vdpa/Makefile new file +87 -0
+cmdline_parse_token_string_t cmd_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_create_vdpa_port = {
+	.f = cmd_create_vdpa_port_parsed,  /* function to call */
+	.data = NULL,      /* 2nd arg of func */
+	.help_str = "create a new vdpa port",
+	.tokens = {        /* token list, NULL terminated */
+		(void *)&cmd_action_create,
+		(void *)&cmd_socket_path,
+		(void *)&cmd_bdf,
+		NULL,
+	},
+};
+
+/* *** QUIT *** */
+struct cmd_quit_result {
+	cmdline_fixed_string_t quit;
+};
+
+static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	vdpa_sample_quit();
+	cmdline_quit(cl);
+}
+
+cmdline_parse_token_string_t cmd_quit_quit =
+	TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit");
+
+cmdline_parse_inst_t cmd_quit = {
+	.f = cmd_quit_parsed,
+	.data = NULL,
+	.help_str = "quit: exit application",
+	.tokens = {
+		(void *)&cmd_quit_quit,
+		NULL,
+	},
+};
+cmdline_parse_ctx_t main_ctx[] = {
+	(cmdline_parse_inst_t *)&cmd_help,
+	(cmdline_parse_inst_t *)&cmd_list_vdpa_devices,
+	(cmdline_parse_inst_t *)&cmd_create_vdpa_port,
+	(cmdline_parse_inst_t *)&cmd_quit,
+	NULL,
+};
+
+int
+main(int argc, char *argv[])
+{
+	char ch;
+	int i;
+	int ret;
+	struct cmdline *cl;
+
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "eal init failed\n");
+	argc -= ret;
+	argv += ret;
+
+	data_init();
+
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	ret = parse_args(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "invalid argument\n");
+
+	if (interactive == 1) {
+		cl = cmdline_stdin_new(main_ctx, "vdpa> ");
+		if (cl == NULL)
+			rte_panic("Cannot create cmdline instance\n");
+		cmdline_interact(cl);
+		cmdline_stdin_exit(cl);
+	} else {
+		for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total);
+				i++) {
+			vports[i].did = i;
+			snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d",
+					iface, i);
+
+			start_vdpa(&vports[i]);
+		}
+

examples/vdpa/Makefile

Quoted 10 lines
examples/vdpa/Makefile new file +10 -0
+		printf("enter \'q\' to quit\n");
+		while (scanf("%c", &ch)) {
+			if (ch == 'q')
+				break;
+			while (ch != '\n') {
+				if (scanf("%c", &ch))
+					printf("%c", ch);
+			}
+			printf("enter \'q\' to quit\n");
+		}

Do vdpa_sample_quit here to stop the vhost session and delete the vhost sockets.

Quoted 28 lines
examples/vdpa/Makefile new file +4 -0
+	}
+
+	return 0;
+}
examples/vdpa/meson.build new file +16 -1
new file mode 100644
index 000000000..2e38a069f
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if host_machine.system() != 'linux'
+	build = false
+endif
+deps += 'vhost'
+allow_experimental_apis = true
+sources = files(
+	'main.c'
+)
\ No newline at end of file
--
Quoted 1 line

2.17.1

Do not forget to add this into 18.11 release_note.

BRs,
Xiao

#4 lore raw

Re: [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 85 lines · Xiao W

Hi Xiaolong,

Quoted 80 lines

-----Original Message-----
From: Ye, Xiaolong
Sent: Friday, September 21, 2018 6:28 AM
To: [email protected]; Maxime Coquelin <[email protected]>; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>
Cc: Wang, Xiao W <[email protected]>; Rami Rosen <[email protected]>; Wang, Haiyue <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [PATCH v3 2/2] examples/vdpa: introduce a new sample for vDPA

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
 MAINTAINERS                        |   2 +
 doc/guides/sample_app_ug/index.rst |   1 +
 doc/guides/sample_app_ug/vdpa.rst  | 115 ++++++++
 examples/Makefile                  |   2 +-
 examples/vdpa/Makefile             |  32 ++
 examples/vdpa/main.c               | 458 +++++++++++++++++++++++++++++
 examples/vdpa/meson.build          |  16 +
 7 files changed, 625 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
 F: examples/vhost_scsi/
 F: doc/guides/sample_app_ug/vhost_scsi.rst
 F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
 Vhost PMD
 M: Maxime Coquelin <[email protected]>
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
     vhost
     vhost_scsi
     vhost_crypto
+    vdpa
     netmap_compatibility
     ip_pipeline
     test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +14 -0
new file mode 100644
index 000000000..44fe6736d
@@ -0,0 +1,115 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Vdpa Sample Application
+=======================
+
+The vdpa sample application creates vhost-user sockets by using the
+vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+virtio ring compatible devices to serve virtio driver directly to enable
+datapath acceleration. As vDPA driver can help to set up vhost datapath,
+this application doesn't need to launch dedicated worker threads for vhost
+enqueue/dequeue operations.
+
+Testing steps

[...]

Quoted 11 lines

doc/guides/sample_app_ug/vdpa.rst line 68

Quoted 9 lines
doc/guides/sample_app_ug/vdpa.rst new file +9 -0
+
+Then we can create 2 vdpa ports in interactive cmdline.
+
+.. code-block:: console
+
+        vdpa> list
+        device id       device address
+        0               0000:06:00.2
+        1               0000:06:00.3

The features and queue numbers of each vDPA device could also be shown now, the doc should reflect this.

Got it.

Quoted 157 lines
Quoted 101 lines
doc/guides/sample_app_ug/vdpa.rst new file +26 -0
+        vdpa> create /tmp/vdpa-socket0 0000:06:00.2
+        vdpa> create /tmp/vdpa-socket1 0000:06:00.3
+
+.. _vdpa_app_run_vm:
+
+Start the VMs
+~~~~~~~~~~~~~
+
+.. code-block:: console
+
+       qemu-system-x86_64 -cpu host -enable-kvm \
+       <snip>
+       -mem-prealloc \
+       -chardev socket,id=char0,path=<socket_file created in above steps> \
+       -netdev type=vhost-user,id=vdpa,chardev=char0 \
+       -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \
+
+After the VMs launches, we can login the VMs and configure the ip, verify the
+network connection via ping or netperf.
+
+.. note::
+    Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA.
+
+Live Migration
+~~~~~~~~~~~~~~
+vDPA supports cross-backend live migration, user can migrate SW vhost

backend
+VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is

doc/guides/sample_app_ug/vdpa.rst new file +3 -0
+the source host with SW vhost VM and B is the destination host with vDPA.
+
+1. Start vdpa sample and launch a VM with exact same parameters as the VM

on A,

doc/guides/sample_app_ug/vdpa.rst new file +17 -0
+   in migration-listen mode:
+
+.. code-block:: console
+
+        B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT))
+
+2. Start the migration (on source host):
+
+.. code-block:: console
+
+        A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT)
+
+3. Check the status (on source host):
+
+.. code-block:: console
+
+        A: (qemu) info migrate
examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
 endif
examples/vdpa/Makefile new file +31 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+

[...]

Quoted 53 lines
examples/vdpa/Makefile new file +53 -0
+
+static int
+new_device(int vid)
+{
+	char ifname[MAX_PATH_LEN];
+	int i;
+
+	rte_vhost_get_ifname(vid, ifname, sizeof(ifname));
+	for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) {
+		if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) {
+			printf("\nnew port %s, did: %d\n",
+					ifname, vports[i].did);
+			vports[i].vid = vid;
+			break;
+		}
+	}
+
+	if (i >= MAX_VDPA_SAMPLE_PORTS)
+		return -1;
+
+	return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+	char ifname[MAX_PATH_LEN];
+	int i;
+
+	rte_vhost_get_ifname(vid, ifname, sizeof(ifname));
+	for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) {
+		if (strcmp(ifname, vports[i].ifname) == 0) {
+			printf("\ndestroy port %s, did: %d\n",
+					ifname, vports[i].did);
+			break;
+		}
+	}
+}
+
+static const struct vhost_device_ops vdpa_sample_devops = {
+	.new_device = new_device,
+	.destroy_device = destroy_device,
+};
+
+static int
+start_vdpa(struct vdpa_port *vport)
+{
+	int ret;
+	char *socket_path = vport->ifname;
+	int did = vport->did;
+
+	if (access(socket_path, F_OK) != -1) {
+		RTE_LOG(ERR, VDPA,
Quoted 8 lines

examples/vdpa/Makefile

Quoted 6 lines
examples/vdpa/Makefile new file +1 -0
+			"%s exists, please remove it or specify another file and

try again.\n",

examples/vdpa/Makefile new file +4 -0
+			socket_path);
+		return -1;
+	}
+	ret = rte_vhost_driver_register(socket_path, vport->flags);

It would be great if we can provide "client mode" option for this sample to create a client mode vdpa.

Sure, I'll add client mode support.

Quoted 134 lines
Quoted 134 lines
examples/vdpa/Makefile new file +83 -0
+	if (ret != 0)
+		rte_exit(EXIT_FAILURE,
+			"register driver failed: %s\n",
+			socket_path);
+
+	ret = rte_vhost_driver_callback_register(socket_path,
+			&vdpa_sample_devops);
+	if (ret != 0)
+		rte_exit(EXIT_FAILURE,
+			"register driver ops failed: %s\n",
+			socket_path);
+
+	ret = rte_vhost_driver_attach_vdpa_device(socket_path, did);
+	if (ret != 0)
+		rte_exit(EXIT_FAILURE,
+			"attach vdpa device failed: %s\n",
+			socket_path);
+
+	if (rte_vhost_driver_start(socket_path) < 0)
+		rte_exit(EXIT_FAILURE,
+			"start vhost driver failed: %s\n",
+			socket_path);
+	return 0;
+}
+
+static void
+close_vdpa(struct vdpa_port *vport)
+{
+	int ret;
+	char *socket_path = vport->ifname;
+
+	ret = rte_vhost_driver_detach_vdpa_device(socket_path);
+	if (ret != 0)
+		RTE_LOG(ERR, VDPA,
+				"attach vdpa device failed: %s\n",
+				socket_path);
+
+	ret = rte_vhost_driver_unregister(socket_path);
+	if (ret != 0)
+		RTE_LOG(ERR, VDPA,
+				"Fail to unregister vhost driver for %s.\n",
+				socket_path);
+}
+
+static void
+vdpa_sample_quit(void)
+{
+	int i;
+	for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) {
+		if (vports[i].ifname[0] != '\0')
+			close_vdpa(&vports[i]);
+	}
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\nSignal %d received, preparing to exit...\n", signum);
+		vdpa_sample_quit();
+		exit(0);
+	}
+}
+
+/* interactive cmds */
+
+/* *** Help command with introduction. *** */
+struct cmd_help_result {
+	cmdline_fixed_string_t help;
+};
+
+static void cmd_help_parsed(__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	cmdline_printf(
+		cl,
+		"\n"
+		"The following commands are currently available:\n\n"
+		"Control:\n"
+		"    help                                      : Show interactive instructions.\n"
+		"    list                                      : list all available vdpa devices.\n"
+		"    create <socket file> <vdev addr>          : create a new vdpa

port.\n"

examples/vdpa/Makefile new file +34 -0
+		"    quit                                      : exit vdpa sample app.\n"
+	);
+}
+
+cmdline_parse_token_string_t cmd_help_help =
+	TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help");
+
+cmdline_parse_inst_t cmd_help = {
+	.f = cmd_help_parsed,
+	.data = NULL,
+	.help_str = "show help",
+	.tokens = {
+		(void *)&cmd_help_help,
+		NULL,
+	},
+};
+
+/* *** List all available vdpa devices *** */
+struct cmd_list_result {
+	cmdline_fixed_string_t action;
+};
+
+static void cmd_list_vdpa_devices_parsed(
+		__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	int did;
+	uint32_t queue_num;
+	uint64_t features;
+	struct rte_vdpa_device *vdev;
+	struct rte_pci_addr addr;
+
+	cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported

features\n");

examples/vdpa/Makefile new file +14 -0
+	for (did = 0; did < dev_total; did++) {
+		vdev = rte_vdpa_get_device(did);
+		if (!vdev)
+			continue;
+		if (vdev->ops->get_queue_num(did, &queue_num) < 0) {
+			RTE_LOG(ERR, VDPA,
+				"failed to get vdpa queue number "
+				"for device id %d.\n", did);
+			continue;
+		}
+		if (vdev->ops->get_features(did, &features) < 0) {
+			RTE_LOG(ERR, VDPA,
+				"failed to get vdpa features "
+				"for device id %d.\n", did);
Quoted 8 lines

examples/vdpa/Makefile

Quoted 6 lines
examples/vdpa/Makefile new file +6 -0
+			continue;
+		}
+		addr = vdev->addr.pci_addr;
+		cmdline_printf(cl, "%d\t\t"PCI_PRI_FMT"\t%d\t\t0x%lx\n", did,
+				addr.domain, addr.bus, addr.devid,
+				addr.function, queue_num, features);

Use "%PRIu32"for queue_num, and use "PRIx64" instead of "%lx", otherwise you may have compile error on 32-bit machine.

Got it.

Quoted 13 lines

examples/vdpa/Makefile

Quoted 10 lines
examples/vdpa/Makefile new file +9 -0
+	}
+}
+
+cmdline_parse_token_string_t cmd_action_list =
+	TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list");
+
+cmdline_parse_inst_t cmd_list_vdpa_devices = {
+	.f = cmd_list_vdpa_devices_parsed,  /* function to call */
+	.data = NULL,      /* 2nd arg of func */

This comment doesn't provide much info.
Remove the comment for all the cmdline_parse_inst_t instances to keep alignment.

Ok, I'll remove all useless comments.

Quoted 138 lines
Quoted 138 lines
examples/vdpa/Makefile new file +42 -0
+	.help_str = "list all available vdpa devices",
+	.tokens = {        /* token list, NULL terminated */
+		(void *)&cmd_action_list,
+		NULL,
+	},
+};
+
+/* *** Create new vdpa port *** */
+struct cmd_create_result {
+	cmdline_fixed_string_t action;
+	cmdline_fixed_string_t socket_path;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_create_vdpa_port_parsed(void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	int did;
+	struct cmd_create_result *res = parsed_result;
+	struct rte_vdpa_dev_addr addr;
+
+	strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN);
+	if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) {
+		cmdline_printf(cl, "Unable to parse the given bdf.\n");
+		return;
+	}
+	addr.type = PCI_ADDR;
+	did = rte_vdpa_find_device_id(&addr);
+	if (did < 0) {
+		cmdline_printf(cl, "Unable to find vdpa devide id.\n");
+		return;
+	}
+
+	vports[devcnt].did = did;
+
+	if (start_vdpa(&vports[devcnt]) == 0)
+		devcnt++;
+}
+
+cmdline_parse_token_string_t cmd_action_create =
+	TOKEN_STRING_INITIALIZER(struct cmd_create_result, action,

"create");

examples/vdpa/Makefile new file +2 -0
+cmdline_parse_token_string_t cmd_socket_path =
+	TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path,

NULL);

examples/vdpa/Makefile new file +91 -0
+cmdline_parse_token_string_t cmd_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_create_vdpa_port = {
+	.f = cmd_create_vdpa_port_parsed,  /* function to call */
+	.data = NULL,      /* 2nd arg of func */
+	.help_str = "create a new vdpa port",
+	.tokens = {        /* token list, NULL terminated */
+		(void *)&cmd_action_create,
+		(void *)&cmd_socket_path,
+		(void *)&cmd_bdf,
+		NULL,
+	},
+};
+
+/* *** QUIT *** */
+struct cmd_quit_result {
+	cmdline_fixed_string_t quit;
+};
+
+static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	vdpa_sample_quit();
+	cmdline_quit(cl);
+}
+
+cmdline_parse_token_string_t cmd_quit_quit =
+	TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit");
+
+cmdline_parse_inst_t cmd_quit = {
+	.f = cmd_quit_parsed,
+	.data = NULL,
+	.help_str = "quit: exit application",
+	.tokens = {
+		(void *)&cmd_quit_quit,
+		NULL,
+	},
+};
+cmdline_parse_ctx_t main_ctx[] = {
+	(cmdline_parse_inst_t *)&cmd_help,
+	(cmdline_parse_inst_t *)&cmd_list_vdpa_devices,
+	(cmdline_parse_inst_t *)&cmd_create_vdpa_port,
+	(cmdline_parse_inst_t *)&cmd_quit,
+	NULL,
+};
+
+int
+main(int argc, char *argv[])
+{
+	char ch;
+	int i;
+	int ret;
+	struct cmdline *cl;
+
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "eal init failed\n");
+	argc -= ret;
+	argv += ret;
+
+	data_init();
+
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	ret = parse_args(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "invalid argument\n");
+
+	if (interactive == 1) {
+		cl = cmdline_stdin_new(main_ctx, "vdpa> ");
+		if (cl == NULL)
+			rte_panic("Cannot create cmdline instance\n");
+		cmdline_interact(cl);
+		cmdline_stdin_exit(cl);
+	} else {
+		for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total);
+				i++) {
+			vports[i].did = i;
+			snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d",
+					iface, i);
+
+			start_vdpa(&vports[i]);
+		}
+
+		printf("enter \'q\' to quit\n");
+		while (scanf("%c", &ch)) {
+			if (ch == 'q')
+				break;
Quoted 8 lines

examples/vdpa/Makefile

Quoted 6 lines
examples/vdpa/Makefile new file +6 -0
+			while (ch != '\n') {
+				if (scanf("%c", &ch))
+					printf("%c", ch);
+			}
+			printf("enter \'q\' to quit\n");
+		}

Do vdpa_sample_quit here to stop the vhost session and delete the vhost sockets.

It needs to call vdpa_sample_quit here.

Quoted 24 lines
Quoted 24 lines
examples/vdpa/Makefile new file +4 -0
+	}
+
+	return 0;
+}
examples/vdpa/meson.build new file +13 -0
new file mode 100644
index 000000000..2e38a069f
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if host_machine.system() != 'linux'
+	build = false
+endif
+deps += 'vhost'
+allow_experimental_apis = true
Quoted 8 lines
Quoted 6 lines
examples/vdpa/meson.build new file +3 -1
+sources = files(
+	'main.c'
+)
\ No newline at end of file
--

2.17.1

Do not forget to add this into 18.11 release_note.

Ok, does this need to make a new separate commit?

Thanks,
Xiaolong

Quoted 4 lines

BRs,
Xiao

#5 lore raw

v4 cover introduce vdpa sample

In reply to Xiaolong Ye

Hi,

This patchset introduces vdpa sample to demonstrate the vDPA use case.

v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.

v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments

v2 changes:

* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list,
* quit
* add application documentation

Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA

 MAINTAINERS                            |   2 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 115 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 458 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 lib/librte_vhost/rte_vdpa.h            |   3 +
 lib/librte_vhost/rte_vhost_version.map |   1 +
 lib/librte_vhost/vdpa.c                |   6 +
 10 files changed, 635 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
#6 lore raw

PATCH v4 1/2 vhost: introduce API to get vDPA device number

It's used to get number of available registered vDPA devices.

---
 lib/librte_vhost/rte_vdpa.h            | 3 +++
 lib/librte_vhost/rte_vhost_version.map | 1 +
 lib/librte_vhost/vdpa.c                | 6 ++++++
 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
index 90465ca26..b8223e337 100644
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr);
 struct rte_vdpa_device * __rte_experimental
 rte_vdpa_get_device(int did);
 
+/* Get current available vdpa device number */
+int __rte_experimental
+rte_vdpa_get_device_num(void);
 #endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
index da220dd02..ae39b6e21 100644
@@ -67,6 +67,7 @@ EXPERIMENTAL {
 	rte_vdpa_unregister_device;
 	rte_vdpa_find_device_id;
 	rte_vdpa_get_device;
+	rte_vdpa_get_device_num;
 	rte_vhost_driver_attach_vdpa_device;
 	rte_vhost_driver_detach_vdpa_device;
 	rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
index c82fd4370..c2c5dff1d 100644
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)
 
 	return vdpa_devices[did];
 }
+
+int
+rte_vdpa_get_device_num(void)
+{
+	return vdpa_device_num;
+}
Signature · 1 line
2.17.1
#7 lore raw

PATCH v4 2/2 examples/vdpa: introduce a new sample for vDPA

In reply to Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
 MAINTAINERS                            |   2 +
 doc/guides/rel_notes/release_18_11.rst |   8 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 118 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 466 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 8 files changed, 644 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
 F: examples/vhost_scsi/
 F: doc/guides/sample_app_ug/vhost_scsi.rst
 F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
 Vhost PMD
 M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
index bc9b74ec4..dd53a9ecf 100644
@@ -67,6 +67,14 @@ New Features
   SR-IOV option in Hyper-V and Azure. This is an alternative to the previous
   vdev_netvsc, tap, and failsafe drivers combination.
 
+* **Add a new sample for vDPA**
+
+  The vdpa sample application creates vhost-user sockets by using the
+  vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+  virtio ring compatible devices to serve virtio driver directly to enable
+  datapath acceleration. As vDPA driver can help to set up vhost datapath,
+  this application doesn't need to launch dedicated worker threads for vhost
+  enqueue/dequeue operations.
 
 API Changes
 -----------
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
     vhost
     vhost_scsi
     vhost_crypto
+    vdpa
     netmap_compatibility
     ip_pipeline
     test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +118 -0

118 lines not on this page · raw message

examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
 endif
examples/vdpa/Makefile new file +32 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
examples/vdpa/main.c new file +466 -0

466 lines not on this page · raw message

examples/vdpa/meson.build new file +16 -0
new file mode 100644
index 000000000..2e38a069f
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if host_machine.system() != 'linux'
+	build = false
+endif
+deps += 'vhost'
+allow_experimental_apis = true
+sources = files(
+	'main.c'
+)
\ No newline at end of file
Signature · 1 line
2.17.1
#8 lore raw

Re: [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA

Hi Xiaolong,

Thanks for the update, 2 small comments below.

Quoted 133 lines

-----Original Message-----
From: Ye, Xiaolong
Sent: Monday, September 24, 2018 4:43 PM
To: [email protected]; Maxime Coquelin <[email protected]>; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>
Cc: Wang, Xiao W <[email protected]>; Rami Rosen <[email protected]>; Wang, Haiyue <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
 MAINTAINERS                            |   2 +
 doc/guides/rel_notes/release_18_11.rst |   8 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 118 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 466 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 8 files changed, 644 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
 F: examples/vhost_scsi/
 F: doc/guides/sample_app_ug/vhost_scsi.rst
 F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
 Vhost PMD
 M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
index bc9b74ec4..dd53a9ecf 100644
@@ -67,6 +67,14 @@ New Features
   SR-IOV option in Hyper-V and Azure. This is an alternative to the previous
   vdev_netvsc, tap, and failsafe drivers combination.
 
+* **Add a new sample for vDPA**
+
+  The vdpa sample application creates vhost-user sockets by using the
+  vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+  virtio ring compatible devices to serve virtio driver directly to enable
+  datapath acceleration. As vDPA driver can help to set up vhost datapath,
+  this application doesn't need to launch dedicated worker threads for vhost
+  enqueue/dequeue operations.
 
 API Changes
 -----------
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
     vhost
     vhost_scsi
     vhost_crypto
+    vdpa
     netmap_compatibility
     ip_pipeline
     test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +36 -0
new file mode 100644
index 000000000..d05728a37
@@ -0,0 +1,118 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Vdpa Sample Application
+=======================
+
+The vdpa sample application creates vhost-user sockets by using the
+vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+virtio ring compatible devices to serve virtio driver directly to enable
+datapath acceleration. As vDPA driver can help to set up vhost datapath,
+this application doesn't need to launch dedicated worker threads for vhost
+enqueue/dequeue operations.
+
+Testing steps
+-------------
+
+This section shows the steps of how to start VMs with vDPA vhost-user
+backend and verify network connection & live migration.
+
+Build
+~~~~~
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``vdpa`` sub-directory.
+
+Start the vdpa example
+~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+        ./vdpa [EAL options]  -- [--client] [--interactive|-i] or [--iface SOCKET_PATH]
+
+where
+
+* --client means running vdpa app in client mode, in the client mode, QEMU

needs

doc/guides/sample_app_ug/vdpa.rst new file +9 -0
+  to run as the server mode and take charge of socket file creation.
+* --iface specifies the path prefix of the UNIX domain socket file, e.g.
+  /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n>
+  (n starts from 0).
+* --interactive means run the vdpa sample in interactive mode, currently 4
+  internal cmds are supported:
+
+  1. help: show help message
+  2. list: list all available vdpa devices

doc/guides/sample_app_ug/vdpa.rst line 55

Quoted 10 lines
doc/guides/sample_app_ug/vdpa.rst new file +10 -0
+  3. create: create a new vdpa port with socket file and vdpa device address
+  4. quit: unregister vhost driver and exit the application
+
+Take IFCVF driver for example:
+
+.. code-block:: console
+
+        ./vdpa --log-level=9 -c 0x6 -n 4 --socket-mem 1024,1024 \
+                -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \
+                -- --interactive

To demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations, We can use "-c 0x2" to indicate that no need to allocate dedicated worker threads.

Quoted 38 lines
doc/guides/sample_app_ug/vdpa.rst new file +38 -0
+
+.. note::
+    We need to bind vfio-pci to VFs before running vdpa sample.
+
+    * modprobe vfio-pci
+    * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4
+
+Then we can create 2 vdpa ports in interactive cmdline.
+
+.. code-block:: console
+
+        vdpa> list
+        device id       device address  queue num       supported features
+        0               0000:06:00.3    1               0x5572362272
+        1               0000:06:00.4    1               0x5572362272
+
+        vdpa> create /tmp/vdpa-socket0 0000:06:00.3
+        vdpa> create /tmp/vdpa-socket1 0000:06:00.4
+
+.. _vdpa_app_run_vm:
+
+Start the VMs
+~~~~~~~~~~~~~
+
+.. code-block:: console
+
+       qemu-system-x86_64 -cpu host -enable-kvm \
+       <snip>
+       -mem-prealloc \
+       -chardev socket,id=char0,path=<socket_file created in above steps> \
+       -netdev type=vhost-user,id=vdpa,chardev=char0 \
+       -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \
+
+After the VMs launches, we can login the VMs and configure the ip, verify the
+network connection via ping or netperf.
+
+.. note::
+    Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA.

[...]

Quoted 39 lines
doc/guides/sample_app_ug/vdpa.rst new file +18 -0
+
+/* *** List all available vdpa devices *** */
+struct cmd_list_result {
+	cmdline_fixed_string_t action;
+};
+
+static void cmd_list_vdpa_devices_parsed(
+		__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	int did;
+	uint32_t queue_num;
+	uint64_t features;
+	struct rte_vdpa_device *vdev;
+	struct rte_pci_addr addr;
+
+	cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported

features\n");

doc/guides/sample_app_ug/vdpa.rst new file +19 -0
+	for (did = 0; did < dev_total; did++) {
+		vdev = rte_vdpa_get_device(did);
+		if (!vdev)
+			continue;
+		if (vdev->ops->get_queue_num(did, &queue_num) < 0) {
+			RTE_LOG(ERR, VDPA,
+				"failed to get vdpa queue number "
+				"for device id %d.\n", did);
+			continue;
+		}
+		if (vdev->ops->get_features(did, &features) < 0) {
+			RTE_LOG(ERR, VDPA,
+				"failed to get vdpa features "
+				"for device id %d.\n", did);
+			continue;
+		}
+		addr = vdev->addr.pci_addr;
+		cmdline_printf(cl,
+
	"%d\t\t"PCI_PRI_FMT"\t%"PRIu32"\t\t0x%"PRIu64"\n", did,

doc/guides/sample_app_ug/vdpa.rst

Quoted 2 lines
doc/guides/sample_app_ug/vdpa.rst new file +2 -0
+			addr.domain, addr.bus, addr.devid,
+			addr.function, queue_num, features);

Use PRIx64 instead of PRIu64 for features.
You can add a blank space between "PRIx64" and the other section to make it more readable. Refer to:

	lib/librte_vhost/vhost_user.c:                  "guest memory region %u, size: 0x%" PRIx64 "\n" 

BRs,
Xiao

#9 lore raw

Re: [PATCH v4 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 4 lines · Xiao W

Hi Xiaolong,

Thanks for the update, 2 small comments below.

[snip]

Quoted 7 lines

doc/guides/sample_app_ug/vdpa.rst line 55

Quoted 3 lines
doc/guides/sample_app_ug/vdpa.rst new file +3 -0
+        ./vdpa --log-level=9 -c 0x6 -n 4 --socket-mem 1024,1024 \
+                -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \
+                -- --interactive

To demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations, We can use "-c 0x2" to indicate that no need to allocate dedicated worker threads.

Got it, will do.

Quoted 2 lines
Quoted 2 lines

+
+.. note::

[snip]

Quoted 8 lines

examples/vdpa/main.c line 311

Quoted 3 lines
	"%d\t\t"PCI_PRI_FMT"\t%"PRIu32"\t\t0x%"PRIu64"\n", did,
examples/vdpa/main.c new file +2 -0
+			addr.domain, addr.bus, addr.devid,
+			addr.function, queue_num, features);

Use PRIx64 instead of PRIu64 for features.
You can add a blank space between "PRIx64" and the other section to make it more readable. Refer to:

	lib/librte_vhost/vhost_user.c:                  "guest memory region %u, size: 0x%" PRIx64 "\n" 

Got it, will do.

Thanks,
Xiaolong

Quoted 3 lines

BRs,
Xiao

#10 lore raw

v5 cover introduce vdpa sample

In reply to Xiaolong Ye

Hi,

This patchset introduces vdpa sample to demonstrate the vDPA use case.

v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst

v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.

v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments

v2 changes:

* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list,
* quit
* add application documentation

Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA

 MAINTAINERS                            |   2 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 115 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 458 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 lib/librte_vhost/rte_vdpa.h            |   3 +
 lib/librte_vhost/rte_vhost_version.map |   1 +
 lib/librte_vhost/vdpa.c                |   6 +
 10 files changed, 635 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
#11 lore raw

PATCH v5 1/2 vhost: introduce API to get vDPA device number

It's used to get number of available registered vDPA devices.

---
 lib/librte_vhost/rte_vdpa.h            | 3 +++
 lib/librte_vhost/rte_vhost_version.map | 1 +
 lib/librte_vhost/vdpa.c                | 6 ++++++
 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
index 90465ca26..b8223e337 100644
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr);
 struct rte_vdpa_device * __rte_experimental
 rte_vdpa_get_device(int did);
 
+/* Get current available vdpa device number */
+int __rte_experimental
+rte_vdpa_get_device_num(void);
 #endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
index da220dd02..ae39b6e21 100644
@@ -67,6 +67,7 @@ EXPERIMENTAL {
 	rte_vdpa_unregister_device;
 	rte_vdpa_find_device_id;
 	rte_vdpa_get_device;
+	rte_vdpa_get_device_num;
 	rte_vhost_driver_attach_vdpa_device;
 	rte_vhost_driver_detach_vdpa_device;
 	rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
index c82fd4370..c2c5dff1d 100644
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)
 
 	return vdpa_devices[did];
 }
+
+int
+rte_vdpa_get_device_num(void)
+{
+	return vdpa_device_num;
+}
Signature · 1 line
2.17.1
#12 lore raw

PATCH v5 2/2 examples/vdpa: introduce a new sample for vDPA

In reply to Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
 MAINTAINERS                            |   2 +
 doc/guides/rel_notes/release_18_11.rst |   8 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 118 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 466 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 8 files changed, 644 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
 F: examples/vhost_scsi/
 F: doc/guides/sample_app_ug/vhost_scsi.rst
 F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
 Vhost PMD
 M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
index bc9b74ec4..dd53a9ecf 100644
@@ -67,6 +67,14 @@ New Features
   SR-IOV option in Hyper-V and Azure. This is an alternative to the previous
   vdev_netvsc, tap, and failsafe drivers combination.
 
+* **Add a new sample for vDPA**
+
+  The vdpa sample application creates vhost-user sockets by using the
+  vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+  virtio ring compatible devices to serve virtio driver directly to enable
+  datapath acceleration. As vDPA driver can help to set up vhost datapath,
+  this application doesn't need to launch dedicated worker threads for vhost
+  enqueue/dequeue operations.
 
 API Changes
 -----------
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
     vhost
     vhost_scsi
     vhost_crypto
+    vdpa
     netmap_compatibility
     ip_pipeline
     test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +118 -0

118 lines not on this page · raw message

examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
 endif
examples/vdpa/Makefile new file +32 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
examples/vdpa/main.c new file +466 -0

466 lines not on this page · raw message

examples/vdpa/meson.build new file +16 -0
new file mode 100644
index 000000000..2e38a069f
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if host_machine.system() != 'linux'
+	build = false
+endif
+deps += 'vhost'
+allow_experimental_apis = true
+sources = files(
+	'main.c'
+)
\ No newline at end of file
Signature · 1 line
2.17.1
#13 lore raw

Re: [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPA

Hi,

Quoted 134 lines

-----Original Message-----
From: Ye, Xiaolong
Sent: Tuesday, September 25, 2018 8:07 PM
To: [email protected]; Maxime Coquelin <[email protected]>; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>
Cc: Wang, Xiao W <[email protected]>; Rami Rosen <[email protected]>; Wang, Haiyue <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [PATCH v5 2/2] examples/vdpa: introduce a new sample for vDPA

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
 MAINTAINERS                            |   2 +
 doc/guides/rel_notes/release_18_11.rst |   8 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 118 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 466 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 8 files changed, 644 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
 F: examples/vhost_scsi/
 F: doc/guides/sample_app_ug/vhost_scsi.rst
 F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
 Vhost PMD
 M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
index bc9b74ec4..dd53a9ecf 100644
@@ -67,6 +67,14 @@ New Features
   SR-IOV option in Hyper-V and Azure. This is an alternative to the previous
   vdev_netvsc, tap, and failsafe drivers combination.
 
+* **Add a new sample for vDPA**
+
+  The vdpa sample application creates vhost-user sockets by using the
+  vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+  virtio ring compatible devices to serve virtio driver directly to enable
+  datapath acceleration. As vDPA driver can help to set up vhost datapath,
+  this application doesn't need to launch dedicated worker threads for vhost
+  enqueue/dequeue operations.
 
 API Changes
 -----------
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
     vhost
     vhost_scsi
     vhost_crypto
+    vdpa
     netmap_compatibility
     ip_pipeline
     test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +32 -0
new file mode 100644
index 000000000..a089393c0
@@ -0,0 +1,118 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Vdpa Sample Application
+=======================
+
+The vdpa sample application creates vhost-user sockets by using the
+vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+virtio ring compatible devices to serve virtio driver directly to enable
+datapath acceleration. As vDPA driver can help to set up vhost datapath,
+this application doesn't need to launch dedicated worker threads for vhost
+enqueue/dequeue operations.
+
+Testing steps
+-------------
+
+This section shows the steps of how to start VMs with vDPA vhost-user
+backend and verify network connection & live migration.
+
+Build
+~~~~~
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``vdpa`` sub-directory.
+
+Start the vdpa example
+~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+        ./vdpa [EAL options]  -- [--client] [--interactive|-i] or [--iface

SOCKET_PATH]

doc/guides/sample_app_ug/vdpa.rst new file +4 -0
+
+where
+
+* --client means running vdpa app in client mode, in the client mode, QEMU

needs

doc/guides/sample_app_ug/vdpa.rst new file +9 -0
+  to run as the server mode and take charge of socket file creation.
+* --iface specifies the path prefix of the UNIX domain socket file, e.g.
+  /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n>
+  (n starts from 0).
+* --interactive means run the vdpa sample in interactive mode, currently 4
+  internal cmds are supported:
+
+  1. help: show help message
+  2. list: list all available vdpa devices

doc/guides/sample_app_ug/vdpa.rst line 55

Quoted 10 lines
doc/guides/sample_app_ug/vdpa.rst new file +10 -0
+  3. create: create a new vdpa port with socket file and vdpa device address
+  4. quit: unregister vhost driver and exit the application
+
+Take IFCVF driver for example:
+
+.. code-block:: console
+
+        ./vdpa --log-level=9 -c 0x2 -n 4 --socket-mem 1024,1024 \
+                -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \
+                -- --interactive

DPDK has log level 1-8, setting the log-level=9 looks strange. Here you can add description about the whitelisted device, since they are virtio ring compatible devices.

Quoted 22 lines
doc/guides/sample_app_ug/vdpa.rst new file +22 -0
+
+.. note::
+    We need to bind vfio-pci to VFs before running vdpa sample.
+
+    * modprobe vfio-pci
+    * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4
+
+Then we can create 2 vdpa ports in interactive cmdline.
+
+.. code-block:: console
+
+        vdpa> list
+        device id       device address  queue num       supported features
+        0               0000:06:00.3    1               0x5572362272
+        1               0000:06:00.4    1               0x5572362272
+
+        vdpa> create /tmp/vdpa-socket0 0000:06:00.3
+        vdpa> create /tmp/vdpa-socket1 0000:06:00.4
+
+.. _vdpa_app_run_vm:
+
+Start the VMs

doc/guides/sample_app_ug/vdpa.rst line 87

Quoted 10 lines
doc/guides/sample_app_ug/vdpa.rst new file +10 -0
+~~~~~~~~~~~~~
+
+.. code-block:: console
+
+       qemu-system-x86_64 -cpu host -enable-kvm \
+       <snip>
+       -mem-prealloc \
+       -chardev socket,id=char0,path=<socket_file created in above steps> \
+       -netdev type=vhost-user,id=vdpa,chardev=char0 \
+       -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee \

In order to leverage EPT, add parameter "page-per-vq=on" for the virtio-net-pci deivce.

Quoted 116 lines
doc/guides/sample_app_ug/vdpa.rst new file +10 -0
+
+After the VMs launches, we can login the VMs and configure the ip, verify the
+network connection via ping or netperf.
+
+.. note::
+    Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA.
+
+Live Migration
+~~~~~~~~~~~~~~
+vDPA supports cross-backend live migration, user can migrate SW vhost

backend
+VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is

doc/guides/sample_app_ug/vdpa.rst new file +3 -0
+the source host with SW vhost VM and B is the destination host with vDPA.
+
+1. Start vdpa sample and launch a VM with exact same parameters as the VM

on A,

doc/guides/sample_app_ug/vdpa.rst new file +17 -0
+   in migration-listen mode:
+
+.. code-block:: console
+
+        B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT))
+
+2. Start the migration (on source host):
+
+.. code-block:: console
+
+        A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT)
+
+3. Check the status (on source host):
+
+.. code-block:: console
+
+        A: (qemu) info migrate
examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
 endif
examples/vdpa/Makefile new file +32 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
examples/vdpa/main.c new file +25 -0
new file mode 100644
index 000000000..7acda67c4
@@ -0,0 +1,466 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <getopt.h>
+#include <signal.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <rte_ethdev.h>
+#include <rte_malloc.h>
+#include <rte_vhost.h>
+#include <rte_vdpa.h>
+#include <rte_pci.h>
+
+#include <cmdline_parse.h>
+#include <cmdline_socket.h>
+#include <cmdline_parse_string.h>
+#include <cmdline.h>
+
+#define MAX_PATH_LEN 128
+#define MAX_VDPA_SAMPLE_PORTS 1024
+#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1
+

[...]

Quoted 39 lines
examples/vdpa/main.c new file +19 -0
+};
+
+/* *** List all available vdpa devices *** */
+struct cmd_list_result {
+	cmdline_fixed_string_t action;
+};
+
+static void cmd_list_vdpa_devices_parsed(
+		__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	int did;
+	uint32_t queue_num;
+	uint64_t features;
+	struct rte_vdpa_device *vdev;
+	struct rte_pci_addr addr;
+
+	cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported

features\n");

examples/vdpa/main.c new file +19 -0
+	for (did = 0; did < dev_total; did++) {
+		vdev = rte_vdpa_get_device(did);
+		if (!vdev)
+			continue;
+		if (vdev->ops->get_queue_num(did, &queue_num) < 0) {
+			RTE_LOG(ERR, VDPA,
+				"failed to get vdpa queue number "
+				"for device id %d.\n", did);
+			continue;
+		}
+		if (vdev->ops->get_features(did, &features) < 0) {
+			RTE_LOG(ERR, VDPA,
+				"failed to get vdpa features "
+				"for device id %d.\n", did);
+			continue;
+		}
+		addr = vdev->addr.pci_addr;
+		cmdline_printf(cl,
+			"%d\t\t" PCI_PRI_FMT "\t%" PRIu32 "\t\t0x%" PRIu64
Quoted 1 line

"\n", did,

s/PRIu64/PRIx64

examples/vdpa/main.c line 313

Quoted 4 lines
examples/vdpa/main.c new file +4 -0
+			addr.domain, addr.bus, addr.devid,
+			addr.function, queue_num, features);
+	}
+}

BRs,
Xiao

#14 lore raw

v6 cover introduce vdpa sample

In reply to Xiaolong Ye

Hi,

This patchset introduces vdpa sample to demonstrate the vDPA use case.

v6 changes:
* improve the document according to Xiao's comments
* fix a typo, PRIu64 -> PRIx64

v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst

v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.

v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments

v2 changes:

* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list, quit
* add application documentation

Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA

 MAINTAINERS                            |   2 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 115 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 458 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 lib/librte_vhost/rte_vdpa.h            |   3 +
 lib/librte_vhost/rte_vhost_version.map |   1 +
 lib/librte_vhost/vdpa.c                |   6 +
 10 files changed, 635 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
#15 lore raw

PATCH v6 1/2 vhost: introduce API to get vDPA device number

It's used to get number of available registered vDPA devices.

---
 lib/librte_vhost/rte_vdpa.h            | 3 +++
 lib/librte_vhost/rte_vhost_version.map | 1 +
 lib/librte_vhost/vdpa.c                | 6 ++++++
 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
index 90465ca26..b8223e337 100644
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr);
 struct rte_vdpa_device * __rte_experimental
 rte_vdpa_get_device(int did);
 
+/* Get current available vdpa device number */
+int __rte_experimental
+rte_vdpa_get_device_num(void);
 #endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
index da220dd02..ae39b6e21 100644
@@ -67,6 +67,7 @@ EXPERIMENTAL {
 	rte_vdpa_unregister_device;
 	rte_vdpa_find_device_id;
 	rte_vdpa_get_device;
+	rte_vdpa_get_device_num;
 	rte_vhost_driver_attach_vdpa_device;
 	rte_vhost_driver_detach_vdpa_device;
 	rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
index c82fd4370..c2c5dff1d 100644
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)
 
 	return vdpa_devices[did];
 }
+
+int
+rte_vdpa_get_device_num(void)
+{
+	return vdpa_device_num;
+}
Signature · 1 line
2.17.1
#16 lore raw

PATCH v6 2/2 examples/vdpa: introduce a new sample for vDPA

In reply to Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
 MAINTAINERS                            |   2 +
 doc/guides/rel_notes/release_18_11.rst |   8 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 120 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 466 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 8 files changed, 646 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
 F: examples/vhost_scsi/
 F: doc/guides/sample_app_ug/vhost_scsi.rst
 F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
 Vhost PMD
 M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
index bc9b74ec4..dd53a9ecf 100644
@@ -67,6 +67,14 @@ New Features
   SR-IOV option in Hyper-V and Azure. This is an alternative to the previous
   vdev_netvsc, tap, and failsafe drivers combination.
 
+* **Add a new sample for vDPA**
+
+  The vdpa sample application creates vhost-user sockets by using the
+  vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+  virtio ring compatible devices to serve virtio driver directly to enable
+  datapath acceleration. As vDPA driver can help to set up vhost datapath,
+  this application doesn't need to launch dedicated worker threads for vhost
+  enqueue/dequeue operations.
 
 API Changes
 -----------
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
     vhost
     vhost_scsi
     vhost_crypto
+    vdpa
     netmap_compatibility
     ip_pipeline
     test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +120 -0

120 lines not on this page · raw message

examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
 endif
examples/vdpa/Makefile new file +32 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
examples/vdpa/main.c new file +466 -0

466 lines not on this page · raw message

examples/vdpa/meson.build new file +16 -0
new file mode 100644
index 000000000..2e38a069f
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if host_machine.system() != 'linux'
+	build = false
+endif
+deps += 'vhost'
+allow_experimental_apis = true
+sources = files(
+	'main.c'
+)
\ No newline at end of file
Signature · 1 line
2.17.1
Reviewed-by on v6 2/2
#17 lore raw

Re: [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 274 lines · Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
  MAINTAINERS                            |   2 +
  doc/guides/rel_notes/release_18_11.rst |   8 +
  doc/guides/sample_app_ug/index.rst     |   1 +
  doc/guides/sample_app_ug/vdpa.rst      | 120 +++++++
  examples/Makefile                      |   2 +-
  examples/vdpa/Makefile                 |  32 ++
  examples/vdpa/main.c                   | 466 +++++++++++++++++++++++++
  examples/vdpa/meson.build              |  16 +
  8 files changed, 646 insertions(+), 1 deletion(-)

create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build

MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
  F: examples/vhost_scsi/
  F: doc/guides/sample_app_ug/vhost_scsi.rst
  F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
  Vhost PMD
  M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
index bc9b74ec4..dd53a9ecf 100644
@@ -67,6 +67,14 @@ New Features
    SR-IOV option in Hyper-V and Azure. This is an alternative to the previous
    vdev_netvsc, tap, and failsafe drivers combination.
 
+* **Add a new sample for vDPA**
+
+  The vdpa sample application creates vhost-user sockets by using the
+  vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+  virtio ring compatible devices to serve virtio driver directly to enable
+  datapath acceleration. As vDPA driver can help to set up vhost datapath,
+  this application doesn't need to launch dedicated worker threads for vhost
+  enqueue/dequeue operations.
 
  API Changes
  -----------
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
      vhost
      vhost_scsi
      vhost_crypto
+    vdpa
      netmap_compatibility
      ip_pipeline
      test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +120 -0
new file mode 100644
index 000000000..745f196ca
@@ -0,0 +1,120 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Vdpa Sample Application
+=======================
+
+The vdpa sample application creates vhost-user sockets by using the
+vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+virtio ring compatible devices to serve virtio driver directly to enable
+datapath acceleration. As vDPA driver can help to set up vhost datapath,
+this application doesn't need to launch dedicated worker threads for vhost
+enqueue/dequeue operations.
+
+Testing steps
+-------------
+
+This section shows the steps of how to start VMs with vDPA vhost-user
+backend and verify network connection & live migration.
+
+Build
+~~~~~
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``vdpa`` sub-directory.
+
+Start the vdpa example
+~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+        ./vdpa [EAL options]  -- [--client] [--interactive|-i] or [--iface SOCKET_PATH]
+
+where
+
+* --client means running vdpa app in client mode, in the client mode, QEMU needs
+  to run as the server mode and take charge of socket file creation.
+* --iface specifies the path prefix of the UNIX domain socket file, e.g.
+  /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n>
+  (n starts from 0).
+* --interactive means run the vdpa sample in interactive mode, currently 4
+  internal cmds are supported:
+
+  1. help: show help message
+  2. list: list all available vdpa devices
+  3. create: create a new vdpa port with socket file and vdpa device address
+  4. quit: unregister vhost driver and exit the application
+
+Take IFCVF driver for example:
+
+.. code-block:: console
+
+        ./vdpa -c 0x2 -n 4 --socket-mem 1024,1024 \
+                -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \
+                -- --interactive
+
+.. note::
+    Here 0000:06:00.3 and 0000:06:00.4 refer to virtio ring compatible devices,
+    and we need to bind vfio-pci to them before running vdpa sample.
+
+    * modprobe vfio-pci
+    * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4
+
+Then we can create 2 vdpa ports in interactive cmdline.
+
+.. code-block:: console
+
+        vdpa> list
+        device id       device address  queue num       supported features
+        0               0000:06:00.3    1               0x14c238020
+        1               0000:06:00.4    1               0x14c238020
+        2               0000:06:00.5    1               0x14c238020
+
+        vdpa> create /tmp/vdpa-socket0 0000:06:00.3
+        vdpa> create /tmp/vdpa-socket1 0000:06:00.4
+
+.. _vdpa_app_run_vm:
+
+Start the VMs
+~~~~~~~~~~~~~
+
+.. code-block:: console
+
+       qemu-system-x86_64 -cpu host -enable-kvm \
+       <snip>
+       -mem-prealloc \
+       -chardev socket,id=char0,path=<socket_file created in above steps> \
+       -netdev type=vhost-user,id=vdpa,chardev=char0 \
+       -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \
+
+After the VMs launches, we can login the VMs and configure the ip, verify the
+network connection via ping or netperf.
+
+.. note::
+    Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA.
+
+Live Migration
+~~~~~~~~~~~~~~
+vDPA supports cross-backend live migration, user can migrate SW vhost backend
+VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is
+the source host with SW vhost VM and B is the destination host with vDPA.
+
+1. Start vdpa sample and launch a VM with exact same parameters as the VM on A,
+   in migration-listen mode:
+
+.. code-block:: console
+
+        B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT))
+
+2. Start the migration (on source host):
+
+.. code-block:: console
+
+        A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT)
+
+3. Check the status (on source host):
+
+.. code-block:: console
+
+        A: (qemu) info migrate
examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
  DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
  endif
  DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
  ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
  DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
  endif
examples/vdpa/Makefile new file +32 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
examples/vdpa/main.c new file +23 -0
new file mode 100644
index 000000000..898286e30
@@ -0,0 +1,466 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <getopt.h>
+#include <signal.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <rte_ethdev.h>
+#include <rte_malloc.h>
+#include <rte_vhost.h>
+#include <rte_vdpa.h>
+#include <rte_pci.h>
+
+#include <cmdline_parse.h>
+#include <cmdline_socket.h>
+#include <cmdline_parse_string.h>
+#include <cmdline.h>
+
+#define MAX_PATH_LEN 128
+#define MAX_VDPA_SAMPLE_PORTS 1024

examples/vdpa/main.c line 33

Quoted 10 lines
examples/vdpa/main.c new file +10 -0
+#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1
+
+struct vdpa_port {
+	char ifname[MAX_PATH_LEN];
+	int did;
+	int vid;
+	uint64_t flags;
+};
+
+struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS];

I guess it can be static?

Quoted 64 lines
examples/vdpa/main.c new file +64 -0
+
+static char iface[MAX_PATH_LEN];
+static int dev_total;
+static int devcnt;
+static int interactive;
+static int client_mode;
+
+/* display usage */
+static void
+vdpa_usage(const char *prgname)
+{
+	printf("Usage: %s [EAL options] -- "
+				 "	--interactive|-i: run in interactive mode.\n"
+				 "	--iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n"
+				 "	--client: register a vhost-user socket as client mode.\n",
+				 prgname);
+}
+
+static int
+parse_args(int argc, char **argv)
+{
+	static const char *short_option = "i";
+	static struct option long_option[] = {
+		{"iface", required_argument, NULL, 0},
+		{"interactive", no_argument, &interactive, 1},
+		{"client", no_argument, &client_mode, 1},
+		{NULL, 0, 0, 0},
+	};
+	int opt, idx;
+	char *prgname = argv[0];
+
+	while ((opt = getopt_long(argc, argv, short_option, long_option, &idx))
+			!= EOF) {
+		switch (opt) {
+		case 'i':
+			printf("Interactive-mode selected\n");
+			interactive = 1;
+			break;
+		/* long options */
+		case 0:
+			if (strncmp(long_option[idx].name, "iface",
+						MAX_PATH_LEN) == 0) {
+				strncpy(iface, optarg, MAX_PATH_LEN);
+				printf("iface %s\n", iface);
+			}
+			if (!strcmp(long_option[idx].name, "interactive")) {
+				printf("Interactive-mode selected\n");
+				interactive = 1;
+			}
+			break;
+
+		default:
+			vdpa_usage(prgname);
+			return -1;
+		}
+	}
+
+	if (iface[0] == '\0' && interactive == 0) {
+		vdpa_usage(prgname);
+		return -1;
+	}
+
+	return 0;
+}

examples/vdpa/main.c line 107

Quoted 10 lines
examples/vdpa/main.c new file +10 -0
+
+static int
+data_init(void)
+{
+	dev_total = rte_vdpa_get_device_num();
+	if (dev_total <= 0) {
+		printf("No available vdpa device found\n");
+		return -1;
+	}
+	devcnt = 0;

This isn't needed as it is static.

examples/vdpa/main.c line 108

Quoted 1 line
examples/vdpa/main.c new file +1 -0
+	interactive = 0;

Ditto

examples/vdpa/main.c line 109

Quoted 1 line
examples/vdpa/main.c new file +1 -0
+	memset(iface, 0, sizeof(iface));

Ditto

examples/vdpa/main.c line 110

Quoted 1 line
examples/vdpa/main.c new file +1 -0
+	memset(vports, 0, sizeof(vports));

And this one too when it will be made static.

Appart these small comments, it looks good to me.
When fixed, please apply my:

Thanks,
Maxime

#18 lore raw

Re: [PATCH v6 2/2] examples/vdpa: introduce a new sample for vDPA

On 09/27, Maxime Coquelin wrote:
[snip]

Quoted 9 lines

examples/vdpa/main.c line 107

Quoted 1 line
examples/vdpa/main.c new file +1 -0
+	devcnt = 0;

This isn't needed as it is static.

examples/vdpa/main.c line 108

Quoted 1 line
examples/vdpa/main.c new file +1 -0
+	interactive = 0;

Ditto

examples/vdpa/main.c line 109

Quoted 1 line
examples/vdpa/main.c new file +1 -0
+	memset(iface, 0, sizeof(iface));

Ditto

Quoted 9 lines

examples/vdpa/main.c line 110

Quoted 1 line
examples/vdpa/main.c new file +1 -0
+	memset(vports, 0, sizeof(vports));

And this one too when it will be made static.

Appart these small comments, it looks good to me.
When fixed, please apply my:

Thanks for your comments, I'll adpot all the static related suggetions.

Thanks,
Xiaolong

Quoted 2 lines

Thanks,
Maxime

#19 lore raw
In reply to Xiaolong Ye

Hi,

Quoted 62 lines

-----Original Message-----
From: Ye, Xiaolong
Sent: Wednesday, September 26, 2018 5:07 PM
To: [email protected]; Maxime Coquelin <[email protected]>; Bie, Tiwei <[email protected]>; Wang, Zhihong <[email protected]>
Cc: Wang, Xiao W <[email protected]>; Rami Rosen <[email protected]>; Wang, Haiyue <[email protected]>; Ye, Xiaolong <[email protected]>
Subject: [PATCH v6 0/2] introduce vdpa sample

Hi,

This patchset introduces vdpa sample to demonstrate the vDPA use case.

v6 changes:
* improve the document according to Xiao's comments
* fix a typo, PRIu64 -> PRIx64

v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst

v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.

v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments

v2 changes:

* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list, quit
* add application documentation

Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA

 MAINTAINERS                            |   2 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 115 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 458 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 lib/librte_vhost/rte_vdpa.h            |   3 +
 lib/librte_vhost/rte_vhost_version.map |   1 +
 lib/librte_vhost/vdpa.c                |   6 +
 10 files changed, 635 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
Quoted 2 lines

--
2.17.1

Series Acked-by: Xiao Wang <[email protected]>

BRs,
Xiao

#20 lore raw

v7 cover introduce vdpa sample

In reply to Xiaolong Ye

Hi,

This patchset introduces vdpa sample to demonstrate the vDPA use case.

v7 changes:
* make vports static
* avoid unnecessary static variable initialization

v6 changes:
* improve the document according to Xiao's comments
* fix a typo, PRIu64 -> PRIx64

v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst

v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.

v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments

v2 changes:

* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list, quit
* add application documentation

Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA

 MAINTAINERS                            |   2 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 115 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 458 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 lib/librte_vhost/rte_vdpa.h            |   3 +
 lib/librte_vhost/rte_vhost_version.map |   1 +
 lib/librte_vhost/vdpa.c                |   6 +
 10 files changed, 635 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
#21 lore raw

PATCH v7 1/2 vhost: introduce API to get vDPA device number

It's used to get number of available registered vDPA devices.

---
 lib/librte_vhost/rte_vdpa.h            | 3 +++
 lib/librte_vhost/rte_vhost_version.map | 1 +
 lib/librte_vhost/vdpa.c                | 6 ++++++
 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
index 90465ca26..b8223e337 100644
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr);
 struct rte_vdpa_device * __rte_experimental
 rte_vdpa_get_device(int did);
 
+/* Get current available vdpa device number */
+int __rte_experimental
+rte_vdpa_get_device_num(void);
 #endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
index da220dd02..ae39b6e21 100644
@@ -67,6 +67,7 @@ EXPERIMENTAL {
 	rte_vdpa_unregister_device;
 	rte_vdpa_find_device_id;
 	rte_vdpa_get_device;
+	rte_vdpa_get_device_num;
 	rte_vhost_driver_attach_vdpa_device;
 	rte_vhost_driver_detach_vdpa_device;
 	rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
index c82fd4370..c2c5dff1d 100644
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)
 
 	return vdpa_devices[did];
 }
+
+int
+rte_vdpa_get_device_num(void)
+{
+	return vdpa_device_num;
+}
Signature · 1 line
2.17.1
#22 lore raw

PATCH v7 2/2 examples/vdpa: introduce a new sample for vDPA

In reply to Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
 MAINTAINERS                            |   2 +
 doc/guides/rel_notes/release_18_11.rst |   8 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 120 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 462 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 8 files changed, 642 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
 F: examples/vhost_scsi/
 F: doc/guides/sample_app_ug/vhost_scsi.rst
 F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
 Vhost PMD
 M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
index bc9b74ec4..dd53a9ecf 100644
@@ -67,6 +67,14 @@ New Features
   SR-IOV option in Hyper-V and Azure. This is an alternative to the previous
   vdev_netvsc, tap, and failsafe drivers combination.
 
+* **Add a new sample for vDPA**
+
+  The vdpa sample application creates vhost-user sockets by using the
+  vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+  virtio ring compatible devices to serve virtio driver directly to enable
+  datapath acceleration. As vDPA driver can help to set up vhost datapath,
+  this application doesn't need to launch dedicated worker threads for vhost
+  enqueue/dequeue operations.
 
 API Changes
 -----------
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
     vhost
     vhost_scsi
     vhost_crypto
+    vdpa
     netmap_compatibility
     ip_pipeline
     test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +120 -0

120 lines not on this page · raw message

examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
 endif
examples/vdpa/Makefile new file +32 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
examples/vdpa/main.c new file +462 -0

462 lines not on this page · raw message

examples/vdpa/meson.build new file +16 -0
new file mode 100644
index 000000000..2e38a069f
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if host_machine.system() != 'linux'
+	build = false
+endif
+deps += 'vhost'
+allow_experimental_apis = true
+sources = files(
+	'main.c'
+)
\ No newline at end of file
Signature · 1 line
2.17.1
#23 lore raw

Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 665 lines · Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
  MAINTAINERS                            |   2 +
  doc/guides/rel_notes/release_18_11.rst |   8 +
  doc/guides/sample_app_ug/index.rst     |   1 +
  doc/guides/sample_app_ug/vdpa.rst      | 120 +++++++
  examples/Makefile                      |   2 +-
  examples/vdpa/Makefile                 |  32 ++
  examples/vdpa/main.c                   | 462 +++++++++++++++++++++++++
  examples/vdpa/meson.build              |  16 +
  8 files changed, 642 insertions(+), 1 deletion(-)

create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build

MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
  F: examples/vhost_scsi/
  F: doc/guides/sample_app_ug/vhost_scsi.rst
  F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
  Vhost PMD
  M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
index bc9b74ec4..dd53a9ecf 100644
@@ -67,6 +67,14 @@ New Features
    SR-IOV option in Hyper-V and Azure. This is an alternative to the previous
    vdev_netvsc, tap, and failsafe drivers combination.
 
+* **Add a new sample for vDPA**
+
+  The vdpa sample application creates vhost-user sockets by using the
+  vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+  virtio ring compatible devices to serve virtio driver directly to enable
+  datapath acceleration. As vDPA driver can help to set up vhost datapath,
+  this application doesn't need to launch dedicated worker threads for vhost
+  enqueue/dequeue operations.
 
  API Changes
  -----------
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
      vhost
      vhost_scsi
      vhost_crypto
+    vdpa
      netmap_compatibility
      ip_pipeline
      test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +120 -0
new file mode 100644
index 000000000..745f196ca
@@ -0,0 +1,120 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2018 Intel Corporation.
+
+Vdpa Sample Application
+=======================
+
+The vdpa sample application creates vhost-user sockets by using the
+vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+virtio ring compatible devices to serve virtio driver directly to enable
+datapath acceleration. As vDPA driver can help to set up vhost datapath,
+this application doesn't need to launch dedicated worker threads for vhost
+enqueue/dequeue operations.
+
+Testing steps
+-------------
+
+This section shows the steps of how to start VMs with vDPA vhost-user
+backend and verify network connection & live migration.
+
+Build
+~~~~~
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``vdpa`` sub-directory.
+
+Start the vdpa example
+~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: console
+
+        ./vdpa [EAL options]  -- [--client] [--interactive|-i] or [--iface SOCKET_PATH]
+
+where
+
+* --client means running vdpa app in client mode, in the client mode, QEMU needs
+  to run as the server mode and take charge of socket file creation.
+* --iface specifies the path prefix of the UNIX domain socket file, e.g.
+  /tmp/vhost-user-, then the socket files will be named as /tmp/vhost-user-<n>
+  (n starts from 0).
+* --interactive means run the vdpa sample in interactive mode, currently 4
+  internal cmds are supported:
+
+  1. help: show help message
+  2. list: list all available vdpa devices
+  3. create: create a new vdpa port with socket file and vdpa device address
+  4. quit: unregister vhost driver and exit the application
+
+Take IFCVF driver for example:
+
+.. code-block:: console
+
+        ./vdpa -c 0x2 -n 4 --socket-mem 1024,1024 \
+                -w 0000:06:00.3,vdpa=1 -w 0000:06:00.4,vdpa=1 \
+                -- --interactive
+
+.. note::
+    Here 0000:06:00.3 and 0000:06:00.4 refer to virtio ring compatible devices,
+    and we need to bind vfio-pci to them before running vdpa sample.
+
+    * modprobe vfio-pci
+    * ./usertools/dpdk-devbind.py -b vfio-pci 06:00.3 06:00.4
+
+Then we can create 2 vdpa ports in interactive cmdline.
+
+.. code-block:: console
+
+        vdpa> list
+        device id       device address  queue num       supported features
+        0               0000:06:00.3    1               0x14c238020
+        1               0000:06:00.4    1               0x14c238020
+        2               0000:06:00.5    1               0x14c238020
+
+        vdpa> create /tmp/vdpa-socket0 0000:06:00.3
+        vdpa> create /tmp/vdpa-socket1 0000:06:00.4
+
+.. _vdpa_app_run_vm:
+
+Start the VMs
+~~~~~~~~~~~~~
+
+.. code-block:: console
+
+       qemu-system-x86_64 -cpu host -enable-kvm \
+       <snip>
+       -mem-prealloc \
+       -chardev socket,id=char0,path=<socket_file created in above steps> \
+       -netdev type=vhost-user,id=vdpa,chardev=char0 \
+       -device virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \
+
+After the VMs launches, we can login the VMs and configure the ip, verify the
+network connection via ping or netperf.
+
+.. note::
+    Suggest to use QEMU 3.0.0 which extends vhost-user for vDPA.
+
+Live Migration
+~~~~~~~~~~~~~~
+vDPA supports cross-backend live migration, user can migrate SW vhost backend
+VM to vDPA backend VM and vice versa. Here are the detailed steps. Assume A is
+the source host with SW vhost VM and B is the destination host with vDPA.
+
+1. Start vdpa sample and launch a VM with exact same parameters as the VM on A,
+   in migration-listen mode:
+
+.. code-block:: console
+
+        B: <qemu-command-line> -incoming tcp:0:4444 (or other PORT))
+
+2. Start the migration (on source host):
+
+.. code-block:: console
+
+        A: (qemu) migrate -d tcp:<B ip>:4444 (or other PORT)
+
+3. Check the status (on source host):
+
+.. code-block:: console
+
+        A: (qemu) info migrate
examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
  DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
  endif
  DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
  ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
  DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
  endif
examples/vdpa/Makefile new file +32 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
examples/vdpa/main.c new file +413 -0
new file mode 100644
index 000000000..b8055287f
@@ -0,0 +1,462 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+
+#include <getopt.h>
+#include <signal.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <rte_ethdev.h>
+#include <rte_malloc.h>
+#include <rte_vhost.h>
+#include <rte_vdpa.h>
+#include <rte_pci.h>
+
+#include <cmdline_parse.h>
+#include <cmdline_socket.h>
+#include <cmdline_parse_string.h>
+#include <cmdline.h>
+
+#define MAX_PATH_LEN 128
+#define MAX_VDPA_SAMPLE_PORTS 1024
+#define RTE_LOGTYPE_VDPA RTE_LOGTYPE_USER1
+
+struct vdpa_port {
+	char ifname[MAX_PATH_LEN];
+	int did;
+	int vid;
+	uint64_t flags;
+};
+
+static struct vdpa_port vports[MAX_VDPA_SAMPLE_PORTS];
+
+static char iface[MAX_PATH_LEN];
+static int dev_total;
+static int devcnt;
+static int interactive;
+static int client_mode;
+
+/* display usage */
+static void
+vdpa_usage(const char *prgname)
+{
+	printf("Usage: %s [EAL options] -- "
+				 "	--interactive|-i: run in interactive mode.\n"
+				 "	--iface <path>: specify the path prefix of the socket files, e.g. /tmp/vhost-user-.\n"
+				 "	--client: register a vhost-user socket as client mode.\n",
+				 prgname);
+}
+
+static int
+parse_args(int argc, char **argv)
+{
+	static const char *short_option = "i";
+	static struct option long_option[] = {
+		{"iface", required_argument, NULL, 0},
+		{"interactive", no_argument, &interactive, 1},
+		{"client", no_argument, &client_mode, 1},
+		{NULL, 0, 0, 0},
+	};
+	int opt, idx;
+	char *prgname = argv[0];
+
+	while ((opt = getopt_long(argc, argv, short_option, long_option, &idx))
+			!= EOF) {
+		switch (opt) {
+		case 'i':
+			printf("Interactive-mode selected\n");
+			interactive = 1;
+			break;
+		/* long options */
+		case 0:
+			if (strncmp(long_option[idx].name, "iface",
+						MAX_PATH_LEN) == 0) {
+				strncpy(iface, optarg, MAX_PATH_LEN);
+				printf("iface %s\n", iface);
+			}
+			if (!strcmp(long_option[idx].name, "interactive")) {
+				printf("Interactive-mode selected\n");
+				interactive = 1;
+			}
+			break;
+
+		default:
+			vdpa_usage(prgname);
+			return -1;
+		}
+	}
+
+	if (iface[0] == '\0' && interactive == 0) {
+		vdpa_usage(prgname);
+		return -1;
+	}
+
+	return 0;
+}
+
+static int
+data_init(void)
+{
+	dev_total = rte_vdpa_get_device_num();
+	if (dev_total <= 0) {
+		printf("No available vdpa device found\n");
+		return -1;
+	}
+	return 0;
+}
+
+static int
+new_device(int vid)
+{
+	char ifname[MAX_PATH_LEN];
+	int i;
+
+	rte_vhost_get_ifname(vid, ifname, sizeof(ifname));
+	for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) {
+		if (strncmp(ifname, vports[i].ifname, MAX_PATH_LEN) == 0) {
+			printf("\nnew port %s, did: %d\n",
+					ifname, vports[i].did);
+			vports[i].vid = vid;
+			break;
+		}
+	}
+
+	if (i >= MAX_VDPA_SAMPLE_PORTS)
+		return -1;
+
+	return 0;
+}
+
+static void
+destroy_device(int vid)
+{
+	char ifname[MAX_PATH_LEN];
+	int i;
+
+	rte_vhost_get_ifname(vid, ifname, sizeof(ifname));
+	for (i = 0; i < MAX_VDPA_SAMPLE_PORTS; i++) {
+		if (strcmp(ifname, vports[i].ifname) == 0) {
+			printf("\ndestroy port %s, did: %d\n",
+					ifname, vports[i].did);
+			break;
+		}
+	}
+}
+
+static const struct vhost_device_ops vdpa_sample_devops = {
+	.new_device = new_device,
+	.destroy_device = destroy_device,
+};
+
+static int
+start_vdpa(struct vdpa_port *vport)
+{
+	int ret;
+	char *socket_path = vport->ifname;
+	int did = vport->did;
+
+	if (client_mode)
+		vport->flags |= RTE_VHOST_USER_CLIENT;
+
+	if (access(socket_path, F_OK) != -1 && !client_mode) {
+		RTE_LOG(ERR, VDPA,
+			"%s exists, please remove it or specify another file and try again.\n",
+			socket_path);
+		return -1;
+	}
+	ret = rte_vhost_driver_register(socket_path, vport->flags);
+	if (ret != 0)
+		rte_exit(EXIT_FAILURE,
+			"register driver failed: %s\n",
+			socket_path);
+
+	ret = rte_vhost_driver_callback_register(socket_path,
+			&vdpa_sample_devops);
+	if (ret != 0)
+		rte_exit(EXIT_FAILURE,
+			"register driver ops failed: %s\n",
+			socket_path);
+
+	ret = rte_vhost_driver_attach_vdpa_device(socket_path, did);
+	if (ret != 0)
+		rte_exit(EXIT_FAILURE,
+			"attach vdpa device failed: %s\n",
+			socket_path);
+
+	if (rte_vhost_driver_start(socket_path) < 0)
+		rte_exit(EXIT_FAILURE,
+			"start vhost driver failed: %s\n",
+			socket_path);
+	return 0;
+}
+
+static void
+close_vdpa(struct vdpa_port *vport)
+{
+	int ret;
+	char *socket_path = vport->ifname;
+
+	ret = rte_vhost_driver_detach_vdpa_device(socket_path);
+	if (ret != 0)
+		RTE_LOG(ERR, VDPA,
+				"detach vdpa device failed: %s\n",
+				socket_path);
+
+	ret = rte_vhost_driver_unregister(socket_path);
+	if (ret != 0)
+		RTE_LOG(ERR, VDPA,
+				"Fail to unregister vhost driver for %s.\n",
+				socket_path);
+}
+
+static void
+vdpa_sample_quit(void)
+{
+	int i;
+	for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total); i++) {
+		if (vports[i].ifname[0] != '\0')
+			close_vdpa(&vports[i]);
+	}
+}
+
+static void
+signal_handler(int signum)
+{
+	if (signum == SIGINT || signum == SIGTERM) {
+		printf("\nSignal %d received, preparing to exit...\n", signum);
+		vdpa_sample_quit();
+		exit(0);
+	}
+}
+
+/* interactive cmds */
+
+/* *** Help command with introduction. *** */
+struct cmd_help_result {
+	cmdline_fixed_string_t help;
+};
+
+static void cmd_help_parsed(__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	cmdline_printf(
+		cl,
+		"\n"
+		"The following commands are currently available:\n\n"
+		"Control:\n"
+		"    help                                      : Show interactive instructions.\n"
+		"    list                                      : list all available vdpa devices.\n"
+		"    create <socket file> <vdev addr>          : create a new vdpa port.\n"
+		"    quit                                      : exit vdpa sample app.\n"
+	);
+}
+
+cmdline_parse_token_string_t cmd_help_help =
+	TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help");
+
+cmdline_parse_inst_t cmd_help = {
+	.f = cmd_help_parsed,
+	.data = NULL,
+	.help_str = "show help",
+	.tokens = {
+		(void *)&cmd_help_help,
+		NULL,
+	},
+};
+
+/* *** List all available vdpa devices *** */
+struct cmd_list_result {
+	cmdline_fixed_string_t action;
+};
+
+static void cmd_list_vdpa_devices_parsed(
+		__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	int did;
+	uint32_t queue_num;
+	uint64_t features;
+	struct rte_vdpa_device *vdev;
+	struct rte_pci_addr addr;
+
+	cmdline_printf(cl, "device id\tdevice address\tqueue num\tsupported features\n");
+	for (did = 0; did < dev_total; did++) {
+		vdev = rte_vdpa_get_device(did);
+		if (!vdev)
+			continue;
+		if (vdev->ops->get_queue_num(did, &queue_num) < 0) {
+			RTE_LOG(ERR, VDPA,
+				"failed to get vdpa queue number "
+				"for device id %d.\n", did);
+			continue;
+		}
+		if (vdev->ops->get_features(did, &features) < 0) {
+			RTE_LOG(ERR, VDPA,
+				"failed to get vdpa features "
+				"for device id %d.\n", did);
+			continue;
+		}
+		addr = vdev->addr.pci_addr;
+		cmdline_printf(cl,
+			"%d\t\t" PCI_PRI_FMT "\t%" PRIu32 "\t\t0x%" PRIx64 "\n",
+			did, addr.domain, addr.bus, addr.devid,
+			addr.function, queue_num, features);
+	}
+}
+
+cmdline_parse_token_string_t cmd_action_list =
+	TOKEN_STRING_INITIALIZER(struct cmd_list_result, action, "list");
+
+cmdline_parse_inst_t cmd_list_vdpa_devices = {
+	.f = cmd_list_vdpa_devices_parsed,
+	.data = NULL,
+	.help_str = "list all available vdpa devices",
+	.tokens = {
+		(void *)&cmd_action_list,
+		NULL,
+	},
+};
+
+/* *** Create new vdpa port *** */
+struct cmd_create_result {
+	cmdline_fixed_string_t action;
+	cmdline_fixed_string_t socket_path;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_create_vdpa_port_parsed(void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	int did;
+	struct cmd_create_result *res = parsed_result;
+	struct rte_vdpa_dev_addr addr;
+
+	strncpy(vports[devcnt].ifname, res->socket_path, MAX_PATH_LEN);
+	if (rte_pci_addr_parse(res->bdf, &addr.pci_addr) != 0) {
+		cmdline_printf(cl, "Unable to parse the given bdf.\n");
+		return;
+	}
+	addr.type = PCI_ADDR;
+	did = rte_vdpa_find_device_id(&addr);
+	if (did < 0) {
+		cmdline_printf(cl, "Unable to find vdpa devide id.\n");
+		return;
+	}
+
+	vports[devcnt].did = did;
+
+	if (start_vdpa(&vports[devcnt]) == 0)
+		devcnt++;
+}
+
+cmdline_parse_token_string_t cmd_action_create =
+	TOKEN_STRING_INITIALIZER(struct cmd_create_result, action, "create");
+cmdline_parse_token_string_t cmd_socket_path =
+	TOKEN_STRING_INITIALIZER(struct cmd_create_result, socket_path, NULL);
+cmdline_parse_token_string_t cmd_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_create_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_create_vdpa_port = {
+	.f = cmd_create_vdpa_port_parsed,
+	.data = NULL,
+	.help_str = "create a new vdpa port",
+	.tokens = {
+		(void *)&cmd_action_create,
+		(void *)&cmd_socket_path,
+		(void *)&cmd_bdf,
+		NULL,
+	},
+};
+
+/* *** QUIT *** */
+struct cmd_quit_result {
+	cmdline_fixed_string_t quit;
+};
+
+static void cmd_quit_parsed(__attribute__((unused)) void *parsed_result,
+		struct cmdline *cl,
+		__attribute__((unused)) void *data)
+{
+	vdpa_sample_quit();
+	cmdline_quit(cl);
+}
+
+cmdline_parse_token_string_t cmd_quit_quit =
+	TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit");
+
+cmdline_parse_inst_t cmd_quit = {
+	.f = cmd_quit_parsed,
+	.data = NULL,
+	.help_str = "quit: exit application",
+	.tokens = {
+		(void *)&cmd_quit_quit,
+		NULL,
+	},
+};
+cmdline_parse_ctx_t main_ctx[] = {
+	(cmdline_parse_inst_t *)&cmd_help,
+	(cmdline_parse_inst_t *)&cmd_list_vdpa_devices,
+	(cmdline_parse_inst_t *)&cmd_create_vdpa_port,
+	(cmdline_parse_inst_t *)&cmd_quit,
+	NULL,
+};
+
+int
+main(int argc, char *argv[])
+{
+	char ch;
+	int i;

examples/vdpa/main.c line 423

Quoted 10 lines
examples/vdpa/main.c new file +10 -0
+	int ret;
+	struct cmdline *cl;
+
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "eal init failed\n");
+	argc -= ret;
+	argv += ret;
+
+	data_init();

You need to check return from data_init().
If this is fine for you, I can change to below code when applying:

ret = data_init();
if (ret < 0)

    rte_exit(EXIT_FAILURE, "data init failed\n");
Quoted 63 lines
examples/vdpa/main.c new file +39 -0
+
+	signal(SIGINT, signal_handler);
+	signal(SIGTERM, signal_handler);
+
+	ret = parse_args(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "invalid argument\n");
+
+	if (interactive == 1) {
+		cl = cmdline_stdin_new(main_ctx, "vdpa> ");
+		if (cl == NULL)
+			rte_panic("Cannot create cmdline instance\n");
+		cmdline_interact(cl);
+		cmdline_stdin_exit(cl);
+	} else {
+		for (i = 0; i < RTE_MIN(MAX_VDPA_SAMPLE_PORTS, dev_total);
+				i++) {
+			vports[i].did = i;
+			snprintf(vports[i].ifname, MAX_PATH_LEN, "%s%d",
+					iface, i);
+
+			start_vdpa(&vports[i]);
+		}
+
+		printf("enter \'q\' to quit\n");
+		while (scanf("%c", &ch)) {
+			if (ch == 'q')
+				break;
+			while (ch != '\n') {
+				if (scanf("%c", &ch))
+					printf("%c", ch);
+			}
+			printf("enter \'q\' to quit\n");
+		}
+		vdpa_sample_quit();
+	}
+
+	return 0;
+}
examples/vdpa/meson.build new file +16 -0
new file mode 100644
index 000000000..2e38a069f
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if host_machine.system() != 'linux'
+	build = false
+endif
+deps += 'vhost'
+allow_experimental_apis = true
+sources = files(
+	'main.c'
+)
\ No newline at end of file
#24 lore raw

Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 11 lines · Maxime Coquelin

examples/vdpa/main.c line 423

Quoted 7 lines
examples/vdpa/main.c new file +7 -0
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "eal init failed\n");
+	argc -= ret;
+	argv += ret;
+
+	data_init();

You need to check return from data_init().
If this is fine for you, I can change to below code when applying:

Quoted 5 lines

ret = data_init();
if (ret < 0)
rte_exit(EXIT_FAILURE, "data init failed\n");

I'm fine with this change.

Thanks,
Xiaolong

#25 lore raw

Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 12 lines · Ye Xiaolong
Quoted 11 lines · Maxime Coquelin

examples/vdpa/main.c line 423

Quoted 7 lines
examples/vdpa/main.c new file +7 -0
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "eal init failed\n");
+	argc -= ret;
+	argv += ret;
+
+	data_init();

You need to check return from data_init().
If this is fine for you, I can change to below code when applying:

Quoted 7 lines
Quoted 5 lines

ret = data_init();
if (ret < 0)

    rte_exit(EXIT_FAILURE, "data init failed\n");

I'm fine with this change.

Or even better, get rid of data_init and
call rte_vdpa_get_device_num(); in main directly.

Quoted 3 lines

Thanks,
Xiaolong

#26 lore raw

Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 21 lines · Maxime Coquelin
Quoted 18 lines · Ye Xiaolong
Quoted 16 lines · Maxime Coquelin

examples/vdpa/main.c line 423

Quoted 7 lines
examples/vdpa/main.c new file +7 -0
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "eal init failed\n");
+	argc -= ret;
+	argv += ret;
+
+	data_init();

You need to check return from data_init().
If this is fine for you, I can change to below code when applying:

ret = data_init();
if (ret < 0)

    rte_exit(EXIT_FAILURE, "data init failed\n");
Quoted 4 lines
Quoted 1 line

I'm fine with this change.

Or even better, get rid of data_init and
call rte_vdpa_get_device_num(); in main directly.

Sounds better, as we've removed unnecessary static variable initialization in data_init, it does nothing but call rte_vdpa_get_device_num, we can remove this wrap. Would you do this change when applying or you perfer I send a new revision?

Thanks,
Xiaolong

Quoted 4 lines
Quoted 4 lines

Thanks,
Xiaolong

#27 lore raw

Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 24 lines · Ye Xiaolong
Quoted 23 lines · Maxime Coquelin
Quoted 19 lines · Ye Xiaolong
Quoted 11 lines · Maxime Coquelin

examples/vdpa/main.c line 423

Quoted 7 lines
examples/vdpa/main.c new file +7 -0
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "eal init failed\n");
+	argc -= ret;
+	argv += ret;
+
+	data_init();

You need to check return from data_init().
If this is fine for you, I can change to below code when applying:

Quoted 5 lines

ret = data_init();
if (ret < 0)

     rte_exit(EXIT_FAILURE, "data init failed\n");

I'm fine with this change.

Quoted 7 lines
Quoted 2 lines

Or even better, get rid of data_init and
call rte_vdpa_get_device_num(); in main directly.

Sounds better, as we've removed unnecessary static variable initialization in data_init, it does nothing but call rte_vdpa_get_device_num, we can remove this wrap. Would you do this change when applying or you perfer I send a new revision?

Please send a new revision, and in the commit message, fix first letter of my family name with an upper case, and move my R-b below your sign-off.

Thanks!
Maxime

Quoted 8 lines

Thanks,
Xiaolong

Quoted 4 lines
Quoted 4 lines

Thanks,
Xiaolong

#28 lore raw

Re: [PATCH v7 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 30 lines · Maxime Coquelin
Quoted 27 lines · Ye Xiaolong
Quoted 25 lines · Maxime Coquelin
Quoted 12 lines · Ye Xiaolong
Quoted 11 lines · Maxime Coquelin

examples/vdpa/main.c line 423

Quoted 7 lines
examples/vdpa/main.c new file +7 -0
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE, "eal init failed\n");
+	argc -= ret;
+	argv += ret;
+
+	data_init();

You need to check return from data_init().
If this is fine for you, I can change to below code when applying:

Quoted 7 lines
Quoted 5 lines

ret = data_init();
if (ret < 0)

     rte_exit(EXIT_FAILURE, "data init failed\n");

I'm fine with this change.

Or even better, get rid of data_init and
call rte_vdpa_get_device_num(); in main directly.

Quoted 8 lines
Quoted 4 lines

Sounds better, as we've removed unnecessary static variable initialization in data_init, it does nothing but call rte_vdpa_get_device_num, we can remove this wrap. Would you do this change when applying or you perfer I send a new revision?

Please send a new revision, and in the commit message, fix first letter of my family name with an upper case, and move my R-b below your sign-off.

Got it.

Thanks,
Xiaolong

Quoted 12 lines

Thanks!
Maxime

Quoted 8 lines

Thanks,
Xiaolong

Quoted 4 lines
Quoted 4 lines

Thanks,
Xiaolong

#29 lore raw

v8 cover introduce vdpa sample

In reply to Xiaolong Ye

Hi,

This patchset introduces vdpa sample to demonstrate the vDPA use case.

v8 changes:
* move body of data_init to the main function
* adjust Reviewed-by/Acked-by tag postion

v7 changes:
* make vports static
* avoid unnecessary static variable initialization

v6 changes:
* improve the document according to Xiao's comments
* fix a typo, PRIu64 -> PRIx64

v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst

v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.

v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments

v2 changes:

* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list, quit
* add application documentation

Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA

 MAINTAINERS                            |   2 +
 doc/guides/rel_notes/release_18_11.rst |   8 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 120 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 453 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 lib/librte_vhost/rte_vdpa.h            |   3 +
 lib/librte_vhost/rte_vhost_version.map |   1 +
 lib/librte_vhost/vdpa.c                |   6 +
 11 files changed, 643 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
Signature · 1 line
2.17.1
#30 lore raw

PATCH v8 1/2 vhost: introduce API to get vDPA device number

It's used to get number of available registered vDPA devices.

---
 lib/librte_vhost/rte_vdpa.h            | 3 +++
 lib/librte_vhost/rte_vhost_version.map | 1 +
 lib/librte_vhost/vdpa.c                | 6 ++++++
 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +3 -0
index 90465ca26..b8223e337 100644
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr);
 struct rte_vdpa_device * __rte_experimental
 rte_vdpa_get_device(int did);
 
+/* Get current available vdpa device number */
+int __rte_experimental
+rte_vdpa_get_device_num(void);
 #endif /* _RTE_VDPA_H_ */
lib/librte_vhost/rte_vhost_version.map +1 -0
index da220dd02..ae39b6e21 100644
@@ -67,6 +67,7 @@ EXPERIMENTAL {
 	rte_vdpa_unregister_device;
 	rte_vdpa_find_device_id;
 	rte_vdpa_get_device;
+	rte_vdpa_get_device_num;
 	rte_vhost_driver_attach_vdpa_device;
 	rte_vhost_driver_detach_vdpa_device;
 	rte_vhost_driver_get_vdpa_device_id;
lib/librte_vhost/vdpa.c +6 -0
index c82fd4370..c2c5dff1d 100644
@@ -113,3 +113,9 @@ rte_vdpa_get_device(int did)
 
 	return vdpa_devices[did];
 }
+
+int
+rte_vdpa_get_device_num(void)
+{
+	return vdpa_device_num;
+}
Signature · 1 line
2.17.1
#31 lore raw

Re: [PATCH v8 1/2] vhost: introduce API to get vDPA device number

Quoted 16 lines · Xiaolong Ye

It's used to get number of available registered vDPA devices.

---
 lib/librte_vhost/rte_vdpa.h            | 3 +++
 lib/librte_vhost/rte_vhost_version.map | 1 +
 lib/librte_vhost/vdpa.c                | 6 ++++++
 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +0 -0
index 90465ca26..b8223e337 100644

lib/librte_vhost/rte_vdpa.h rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); · line 89

Quoted 6 lines
lib/librte_vhost/rte_vdpa.h +3 -0
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr);
 struct rte_vdpa_device * __rte_experimental
 rte_vdpa_get_device(int did);
 
+/* Get current available vdpa device number */
+int __rte_experimental
+rte_vdpa_get_device_num(void);

Hi Xiaolong,

Is rte_vdpa_get_device_num() an public API? (It should be since it will be used in an application in next patch.)

If so, header needs to be part of API documentation:
1- Update doc/api/doxy-api-index.md to include this header. 2- Update all APIs comments to be an doxygen comment. 3- Add doxygen comment to all data structures.

I can see rte_vdpa.h not documented at all, it is not just this API, so it can be better to send a new patch to address above items.

Thanks,
ferruh

#32 lore raw

Re: [PATCH v8 1/2] vhost: introduce API to get vDPA device number

Hi, Ferruh,

Quoted 29 lines · Ferruh Yigit
Quoted 16 lines · Xiaolong Ye

It's used to get number of available registered vDPA devices.

---
 lib/librte_vhost/rte_vdpa.h            | 3 +++
 lib/librte_vhost/rte_vhost_version.map | 1 +
 lib/librte_vhost/vdpa.c                | 6 ++++++
 3 files changed, 10 insertions(+)
lib/librte_vhost/rte_vdpa.h +0 -0
index 90465ca26..b8223e337 100644

lib/librte_vhost/rte_vdpa.h rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr); · line 89

Quoted 6 lines
lib/librte_vhost/rte_vdpa.h +3 -0
@@ -84,4 +84,7 @@ rte_vdpa_find_device_id(struct rte_vdpa_dev_addr *addr);
 struct rte_vdpa_device * __rte_experimental
 rte_vdpa_get_device(int did);
 
+/* Get current available vdpa device number */
+int __rte_experimental
+rte_vdpa_get_device_num(void);

Hi Xiaolong,

Is rte_vdpa_get_device_num() an public API? (It should be since it will be used in an application in next patch.)

Quoted 7 lines

If so, header needs to be part of API documentation:
1- Update doc/api/doxy-api-index.md to include this header. 2- Update all APIs comments to be an doxygen comment. 3- Add doxygen comment to all data structures.

I can see rte_vdpa.h not documented at all, it is not just this API, so it can be better to send a new patch to address above items.

Thanks for the reminder, I'll send a new patch for it.

Thanks,
Xiaolong

Quoted 3 lines

Thanks,
ferruh

#33 lore raw

PATCH v8 2/2 examples/vdpa: introduce a new sample for vDPA

In reply to Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---
 MAINTAINERS                            |   2 +
 doc/guides/rel_notes/release_18_11.rst |   8 +
 doc/guides/sample_app_ug/index.rst     |   1 +
 doc/guides/sample_app_ug/vdpa.rst      | 120 +++++++
 examples/Makefile                      |   2 +-
 examples/vdpa/Makefile                 |  32 ++
 examples/vdpa/main.c                   | 453 +++++++++++++++++++++++++
 examples/vdpa/meson.build              |  16 +
 8 files changed, 633 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/vdpa.rst
 create mode 100644 examples/vdpa/Makefile
 create mode 100644 examples/vdpa/main.c
 create mode 100644 examples/vdpa/meson.build
MAINTAINERS +2 -0
index 5967c1dd3..5656f18e8 100644
@@ -683,6 +683,8 @@ F: doc/guides/sample_app_ug/vhost.rst
 F: examples/vhost_scsi/
 F: doc/guides/sample_app_ug/vhost_scsi.rst
 F: examples/vhost_crypto/
+F: examples/vdpa/
+F: doc/guides/sample_app_ug/vdpa.rst
 
 Vhost PMD
 M: Maxime Coquelin <[email protected]>
doc/guides/rel_notes/release_18_11.rst +8 -0
index bc9b74ec4..dd53a9ecf 100644
@@ -67,6 +67,14 @@ New Features
   SR-IOV option in Hyper-V and Azure. This is an alternative to the previous
   vdev_netvsc, tap, and failsafe drivers combination.
 
+* **Add a new sample for vDPA**
+
+  The vdpa sample application creates vhost-user sockets by using the
+  vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes
+  virtio ring compatible devices to serve virtio driver directly to enable
+  datapath acceleration. As vDPA driver can help to set up vhost datapath,
+  this application doesn't need to launch dedicated worker threads for vhost
+  enqueue/dequeue operations.
 
 API Changes
 -----------
doc/guides/sample_app_ug/index.rst +1 -0
index 5bedf4f6f..74b12af85 100644
@@ -45,6 +45,7 @@ Sample Applications User Guides
     vhost
     vhost_scsi
     vhost_crypto
+    vdpa
     netmap_compatibility
     ip_pipeline
     test_pipeline
doc/guides/sample_app_ug/vdpa.rst new file +120 -0

120 lines not on this page · raw message

examples/Makefile +1 -1
index 481720cb6..356fcb1cd 100644
@@ -65,7 +65,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_HASH),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += tep_termination
 endif
 DIRS-$(CONFIG_RTE_LIBRTE_TIMER) += timer
-DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi
+DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost vhost_scsi vdpa
 ifeq ($(CONFIG_RTE_LIBRTE_CRYPTODEV),y)
 DIRS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_crypto
 endif
examples/vdpa/Makefile new file +32 -0
new file mode 100644
index 000000000..42672a2bc
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
+$(info This application can only operate in a linuxapp environment, \
+please change the definition of the RTE_TARGET environment variable)
+all:
+else
+
+# binary name
+APP = vdpa
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+CFLAGS += -O2 -D_FILE_OFFSET_BITS=64
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -D_GNU_SOURCE
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+include $(RTE_SDK)/mk/rte.extapp.mk
+
+endif
examples/vdpa/main.c new file +453 -0

453 lines not on this page · raw message

examples/vdpa/meson.build new file +16 -0
new file mode 100644
index 000000000..2e38a069f
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+if host_machine.system() != 'linux'
+	build = false
+endif
+deps += 'vhost'
+allow_experimental_apis = true
+sources = files(
+	'main.c'
+)
\ No newline at end of file
Signature · 1 line
2.17.1
#34 lore raw

Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 10 lines · Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

<...>

Quoted 15 lines
examples/vdpa/main.c new file +15 -0
+static int
+parse_args(int argc, char **argv)
+{
+	static const char *short_option = "i";
+	static struct option long_option[] = {
+		{"iface", required_argument, NULL, 0},
+		{"interactive", no_argument, &interactive, 1},
+		{"client", no_argument, &client_mode, 1},
+		{NULL, 0, 0, 0},
+	};
+	int opt, idx;
+	char *prgname = argv[0];
+
+	while ((opt = getopt_long(argc, argv, short_option, long_option, &idx))
+			!= EOF) {

examples/vdpa/main.c line 76

Quoted 10 lines
examples/vdpa/main.c new file +10 -0
+		switch (opt) {
+		case 'i':
+			printf("Interactive-mode selected\n");
+			interactive = 1;
+			break;
+		/* long options */
+		case 0:
+			if (strncmp(long_option[idx].name, "iface",
+						MAX_PATH_LEN) == 0) {
+				strncpy(iface, optarg, MAX_PATH_LEN);

Giving compiler warning [1], rte_strscpy() is safer to use.

[1]
In function ‘parse_args’,

inlined from ‘main’ at .../examples/vdpa/main.c:419:8:

.../examples/vdpa/main.c:76:5: error: ‘strncpy’ specified bound 128 equals destination size [-Werror=stringop-truncation]

     strncpy(iface, optarg, MAX_PATH_LEN);



     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#35 lore raw

Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA

Hi, Ferruh,

Quoted 55 lines · Ferruh Yigit
Quoted 10 lines · Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

<...>

Quoted 15 lines
examples/vdpa/main.c new file +15 -0
+static int
+parse_args(int argc, char **argv)
+{
+	static const char *short_option = "i";
+	static struct option long_option[] = {
+		{"iface", required_argument, NULL, 0},
+		{"interactive", no_argument, &interactive, 1},
+		{"client", no_argument, &client_mode, 1},
+		{NULL, 0, 0, 0},
+	};
+	int opt, idx;
+	char *prgname = argv[0];
+
+	while ((opt = getopt_long(argc, argv, short_option, long_option, &idx))
+			!= EOF) {

examples/vdpa/main.c line 76

Quoted 10 lines
examples/vdpa/main.c new file +10 -0
+		switch (opt) {
+		case 'i':
+			printf("Interactive-mode selected\n");
+			interactive = 1;
+			break;
+		/* long options */
+		case 0:
+			if (strncmp(long_option[idx].name, "iface",
+						MAX_PATH_LEN) == 0) {
+				strncpy(iface, optarg, MAX_PATH_LEN);

Giving compiler warning [1], rte_strscpy() is safer to use.

[1]
In function ‘parse_args’,

inlined from ‘main’ at .../examples/vdpa/main.c:419:8:

.../examples/vdpa/main.c:76:5: error: ‘strncpy’ specified bound 128 equals destination size [-Werror=stringop-truncation]

Quoted 5 lines
    strncpy(iface, optarg, MAX_PATH_LEN);



    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Thanks for the finding, by the way, how do you get this warning? I can't see it in my local build, what compiler you used?

Thanks,
Xiaolong

#36 lore raw

Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA

Quoted 58 lines · Ye Xiaolong

Hi, Ferruh,

Quoted 55 lines · Ferruh Yigit
Quoted 10 lines · Xiaolong Ye

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

<...>

Quoted 15 lines
examples/vdpa/main.c new file +15 -0
+static int
+parse_args(int argc, char **argv)
+{
+	static const char *short_option = "i";
+	static struct option long_option[] = {
+		{"iface", required_argument, NULL, 0},
+		{"interactive", no_argument, &interactive, 1},
+		{"client", no_argument, &client_mode, 1},
+		{NULL, 0, 0, 0},
+	};
+	int opt, idx;
+	char *prgname = argv[0];
+
+	while ((opt = getopt_long(argc, argv, short_option, long_option, &idx))
+			!= EOF) {

examples/vdpa/main.c line 76

Quoted 10 lines
examples/vdpa/main.c new file +10 -0
+		switch (opt) {
+		case 'i':
+			printf("Interactive-mode selected\n");
+			interactive = 1;
+			break;
+		/* long options */
+		case 0:
+			if (strncmp(long_option[idx].name, "iface",
+						MAX_PATH_LEN) == 0) {
+				strncpy(iface, optarg, MAX_PATH_LEN);

Giving compiler warning [1], rte_strscpy() is safer to use.

[1]
In function ‘parse_args’,

inlined from ‘main’ at .../examples/vdpa/main.c:419:8:

.../examples/vdpa/main.c:76:5: error: ‘strncpy’ specified bound 128 equals destination size [-Werror=stringop-truncation]

Quoted 8 lines
Quoted 5 lines
    strncpy(iface, optarg, MAX_PATH_LEN);



    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Thanks for the finding, by the way, how do you get this warning? I can't see it in my local build, what compiler you used?

gcc,

$ gcc --version
gcc (GCC) 8.1.1 20180712 (Red Hat 8.1.1-5)
#37 lore raw

Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA

In reply to Xiaolong Ye

Hi,

Going back to this merged commit,

Saturday, September 29, 2018 12:48 AM, Xiaolong Ye:

Quoted 14 lines

Subject: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---

[...]

doc/guides/sample_app_ug/vdpa.rst

Quoted 13 lines
doc/guides/sample_app_ug/vdpa.rst new file +13 -0
+
+Start the VMs
+~~~~~~~~~~~~~
+
+.. code-block:: console
+
+       qemu-system-x86_64 -cpu host -enable-kvm \
+       <snip>
+       -mem-prealloc \
+       -chardev socket,id=char0,path=<socket_file created in above steps> \
+       -netdev type=vhost-user,id=vdpa,chardev=char0 \
+       -device
+ virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \

It doesn't seems like qemu has the id=vdpa and netdev=vdpa options. Grepping vdpa on qemu returns empty.

Am using QEMU v3.1.0-513-gfb177a0. Commit head is fb177a036f2e ("MAINTAINERS: Mark RISC-V as Supported").

Did I missed anything?

#38 lore raw

Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA

Hi,

Quoted 40 lines · Shahaf Shuler

Hi,

Going back to this merged commit,

Saturday, September 29, 2018 12:48 AM, Xiaolong Ye:

Quoted 14 lines

Subject: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---

[...]

doc/guides/sample_app_ug/vdpa.rst

Quoted 13 lines
doc/guides/sample_app_ug/vdpa.rst new file +13 -0
+
+Start the VMs
+~~~~~~~~~~~~~
+
+.. code-block:: console
+
+       qemu-system-x86_64 -cpu host -enable-kvm \
+       <snip>
+       -mem-prealloc \
+       -chardev socket,id=char0,path=<socket_file created in above steps> \
+       -netdev type=vhost-user,id=vdpa,chardev=char0 \
+       -device
+ virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \

It doesn't seems like qemu has the id=vdpa and netdev=vdpa options. Grepping vdpa on qemu returns empty.

Quoted 4 lines

Am using QEMU v3.1.0-513-gfb177a0. Commit head is fb177a036f2e ("MAINTAINERS: Mark RISC-V as Supported").

Did I missed anything?

Not quite sure about your question here, the vdpa used in above qemu cmdline is just a normal string, you can replace it with other string like

-netdev type=vhost-user,id=mynet,chardev=char0 \
-device virtio-net-pci,netdev=mynet,...

The major support for vdpa is done in dpdk side as the vhost user backend, qemu will use normal vhost user protocol to communicate with it.

Thanks,
Xiaolong

#39 lore raw

Re: [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA

Thursday, January 17, 2019 5:13 PM, Ye Xiaolong:

Quoted 53 lines

Subject: Re: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA

Hi,

Quoted 22 lines · Shahaf Shuler

Hi,

Going back to this merged commit,

Saturday, September 29, 2018 12:48 AM, Xiaolong Ye:

Quoted 14 lines

Subject: [dpdk-dev] [PATCH v8 2/2] examples/vdpa: introduce a new sample for vDPA

The vdpa sample application creates vhost-user sockets by using the vDPA backend. vDPA stands for vhost Data Path Acceleration which utilizes virtio ring compatible devices to serve virtio driver directly to enable datapath acceleration. As vDPA driver can help to set up vhost datapath, this application doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations.

---

[...]

Quoted 10 lines

doc/guides/sample_app_ug/vdpa.rst

Quoted 10 lines
doc/guides/sample_app_ug/vdpa.rst new file +10 -0
+
+Start the VMs
+~~~~~~~~~~~~~
+
+.. code-block:: console
+
+       qemu-system-x86_64 -cpu host -enable-kvm \
+       <snip>
+       -mem-prealloc \
+       -chardev socket,id=char0,path=<socket_file created in above steps>

\

Quoted 9 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+       -netdev type=vhost-user,id=vdpa,chardev=char0 \
+       -device
+ virtio-net-pci,netdev=vdpa,mac=00:aa:bb:cc:dd:ee,page-per-vq=on \

It doesn't seems like qemu has the id=vdpa and netdev=vdpa options. Grepping vdpa on qemu returns empty.

Am using QEMU v3.1.0-513-gfb177a0. Commit head is fb177a036f2e

("MAINTAINERS: Mark RISC-V as Supported").

Quoted 3 lines

Did I missed anything?

Quoted 8 lines

Not quite sure about your question here, the vdpa used in above qemu cmdline is just a normal string, you can replace it with other string like

-netdev type=vhost-user,id=mynet,chardev=char0 \ -device virtio-net- pci,netdev=mynet,...

The major support for vdpa is done in dpdk side as the vhost user backend, qemu will use normal vhost user protocol to communicate with it.

Thanks. I mis-interpreted the option fields.

Quoted 3 lines

Thanks,
Xiaolong

#40 lore raw
In reply to Xiaolong Ye
Quoted 56 lines · Xiaolong Ye

Hi,

This patchset introduces vdpa sample to demonstrate the vDPA use case.

v8 changes:
* move body of data_init to the main function
* adjust Reviewed-by/Acked-by tag postion

v7 changes:
* make vports static
* avoid unnecessary static variable initialization

v6 changes:
* improve the document according to Xiao's comments
* fix a typo, PRIu64 -> PRIx64

v5 changes:
* improve print format and correct from "PRIu64" to "PRIx64"
* use "-c 0x2" to better demonstrate app doesn't need to launch dedicated worker threads for vhost enqueue/dequeue operations in vdpa.rst

v4 changes:
* add client mode support
* improve the format to list the vDPA device info and improve the vdpa.rst accordingly
* remove some useless comments
* add introduction in 18.11 release note.

v3 changes:
* list cmd would show queue number and supported features of vdpa devices.
* address Xiao's review comments

v2 changes:

* fix a compilation error reported by Rosen
* improve create cmd in interactive mode and add two new cmds: list, quit
* add application documentation

Xiaolong Ye (2):
vhost: introduce API to get vDPA device number
examples/vdpa: introduce a new sample for vDPA

  MAINTAINERS                            |   2 +
  doc/guides/rel_notes/release_18_11.rst |   8 +
  doc/guides/sample_app_ug/index.rst     |   1 +
  doc/guides/sample_app_ug/vdpa.rst      | 120 +++++++
  examples/Makefile                      |   2 +-
  examples/vdpa/Makefile                 |  32 ++
  examples/vdpa/main.c                   | 453 +++++++++++++++++++++++++
  examples/vdpa/meson.build              |  16 +
  lib/librte_vhost/rte_vdpa.h            |   3 +
  lib/librte_vhost/rte_vhost_version.map |   1 +
  lib/librte_vhost/vdpa.c                |   6 +
  11 files changed, 643 insertions(+), 1 deletion(-)
Quoted 5 lines

create mode 100644 doc/guides/sample_app_ug/vdpa.rst create mode 100644 examples/vdpa/Makefile create mode 100644 examples/vdpa/main.c create mode 100644 examples/vdpa/meson.build

Applied to dpdk-next-virtio/master.

Thanks,
Maxime