Skip to contentSkip to conversation

toolstack support for generic virtio devices on Arm

PATCH In review Viresh Kumar 4 participants 9 messages Dec 14, 2022

lore
Share & tools
lore
#0 lore raw

v10 cover toolstack support for generic virtio devices on Arm

Hello,

This patchset adds toolstack support for I2C, GPIO and generic virtio devices. This is inspired from the work done by Oleksandr for the Disk device.

This is developed as part of Linaro's Project Stratos, where we are working towards Hypervisor agnostic Rust based backends [1].

This is based of Xen's master branch.

V9->V10:
- Swap I2C/GPIO compatible strings, they were wrongly added earlier.
- Arrange tags in timely order and add new ones.

V8->V9:
- Drop changes to tools/ocaml/libs/xl/genwrap.py file.
- Replace GPIO with I2C in a comment.
- Add Reviewed-by tags.

V7->V8:
- Use macros for compatible string names.
- Use strcmp() instead of strncmp() at several places.
- Rename "virtio,devices" to "virtio,device" in commit log.
- Remove extra call to fdt_end_node().
- Disallow "unknown" in xenstore transport.
- Use libxl__strdup().
- Update documentation.
- Remove DEVICE_ADDREMOVE(virtio).

V6->V7:
- Support generic virtio devices too. They are passed with type=virtio,device, and we only create the MMIO DT node for them.
- Add links to DT bindings of I2C and GPIO, in code and documentation.
- Call libxl__device_add() for all hypervisor types.
- Add (0, "UNKNOWN") for libxl_virtio_transport.
- Removed libxl_virtio_backend and libxl_virtioinfo, as they were unused.
- Remove unnecessary stuff from libxl__virtio_from_xenstore() and add support for type and transport.
- Add backend=domid in documentation and replace compatible with type.
- Improved commit logs.

V5->V6:
- The cleanup patches are sent separately [2].
- We don't add I2C or GPIO specific device changes anymore, rather we create generic "virtio" devices. The "type" of a virtio devices helps us identify the right device, and create an entry in the DT node. The same can be used for all Virtio devices now.
- Update man page xl.cfg.

V4->V5:
- Fixed indentation at few places.
- Removed/added blank lines.
- Added few comments.
- Added review tags from Oleksandr.
- Rebased over latest staging branch.

V3->V4:
- Update virtio_enabled independently of all devices, so we don't miss setting it to true.

- Add iommu handling for i2c/gpio and move it as part of make_virtio_mmio_node_common(), which gets backend_domid parameter as a result.

V2->V3:
- Rebased over latest tree and made changes according to changes in Oleksandr's patches from sometime back.
- Minor cleanups.

V1->V2:
- Patches 3/6 and 4/6 are new.
- Patches 5/6 and 6/6 updated based on the above two patches.
- Added link to the bindings for I2C and GPIO.
- Rebased over latest master branch.

Thanks.

--
Viresh

[1] https://lore.kernel.org/xen-devel/20220414092358.kepxbmnrtycz7mhe@vireshk-i7/

Viresh Kumar (3):
libxl: Add support for generic virtio device
xl: Add support to parse generic virtio device
docs: Add documentation for generic virtio devices

 docs/man/xl.cfg.5.pod.in                  |  33 +++++
 tools/libs/light/Makefile                 |   1 +
 tools/libs/light/libxl_arm.c              | 100 +++++++++++++++
 tools/libs/light/libxl_create.c           |   4 +
 tools/libs/light/libxl_internal.h         |   6 +
 tools/libs/light/libxl_types.idl          |  18 +++
 tools/libs/light/libxl_types_internal.idl |   1 +
 tools/libs/light/libxl_virtio.c           | 144 ++++++++++++++++++++++
 tools/xl/xl_parse.c                       |  81 ++++++++++++
 9 files changed, 388 insertions(+)
 create mode 100644 tools/libs/light/libxl_virtio.c
Signature · 1 line
2.31.1.272.g89b43f80a514
#1 lore raw

PATCH v10 1/3 libxl: Add support for generic virtio device

This patch adds basic support for configuring and assisting generic Virtio backends, which could run in any domain.

An example of domain configuration for mmio based Virtio I2C device is:
virtio = ["type=virtio,device22,transport=mmio"]

To make this work on Arm, allocate Virtio MMIO params (IRQ and memory region) and pass them to the backend and update guest device-tree to create a DT node for the Virtio devices.

Add special support for I2C and GPIO devices, which require the "compatible" DT property to be set, among other device specific properties. Support for generic virtio devices is also added, which just need a MMIO node but not any special DT properties, for such devices the user needs to pass "virtio,device" in the "type" string.

The parsing of generic virtio device configurations will be done in a separate commit.

---
 tools/libs/light/Makefile                 |   1 +
 tools/libs/light/libxl_arm.c              | 100 +++++++++++++++
 tools/libs/light/libxl_create.c           |   4 +
 tools/libs/light/libxl_internal.h         |   6 +
 tools/libs/light/libxl_types.idl          |  18 +++
 tools/libs/light/libxl_types_internal.idl |   1 +
 tools/libs/light/libxl_virtio.c           | 144 ++++++++++++++++++++++
 7 files changed, 274 insertions(+)
 create mode 100644 tools/libs/light/libxl_virtio.c
tools/libs/light/Makefile +1 -0
index 374be1cfab25..4fddcc6f51d7 100644
@@ -106,6 +106,7 @@ OBJS-y += libxl_vdispl.o
 OBJS-y += libxl_pvcalls.o
 OBJS-y += libxl_vsnd.o
 OBJS-y += libxl_vkb.o
+OBJS-y += libxl_virtio.o
 OBJS-y += libxl_genid.o
 OBJS-y += _libxl_types.o
 OBJS-y += libxl_flask.o
tools/libs/light/libxl_arm.c +100 -0

118 lines not on this page · raw message

tools/libs/light/libxl_create.c +4 -0
index 612eacfc7fac..beec3f6b6fec 100644
@@ -1752,6 +1752,10 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
         libxl__device_add(gc, domid, &libxl__pvcallsif_devtype,
                           &d_config->pvcallsifs[i]);
 
+    for (i = 0; i < d_config->num_virtios; i++)
+        libxl__device_add(gc, domid, &libxl__virtio_devtype,
+                          &d_config->virtios[i]);
+
     switch (d_config->c_info.type) {
     case LIBXL_DOMAIN_TYPE_HVM:
     {
tools/libs/light/libxl_internal.h +6 -0
index a7c447c10e5f..0dc8b8f210cb 100644
@@ -166,6 +166,11 @@
 /* Convert pfn to physical address space. */
 #define pfn_to_paddr(x) ((uint64_t)(x) << XC_PAGE_SHIFT)
 
+/* Virtio device types */
+#define VIRTIO_DEVICE_TYPE_GENERIC   "virtio,device"
+#define VIRTIO_DEVICE_TYPE_I2C       "virtio,device22"
+#define VIRTIO_DEVICE_TYPE_GPIO      "virtio,device29"
+
 /* logging */
 _hidden void libxl__logv(libxl_ctx *ctx, xentoollog_level msglevel, int errnoval,
              const char *file /* may be 0 */, int line /* ignored if !file */,
@@ -4003,6 +4008,7 @@ static inline int *libxl__device_type_get_num(
 
 extern const libxl__device_type libxl__vfb_devtype;
 extern const libxl__device_type libxl__vkb_devtype;
+extern const libxl__device_type libxl__virtio_devtype;
 extern const libxl__device_type libxl__disk_devtype;
 extern const libxl__device_type libxl__nic_devtype;
 extern const libxl__device_type libxl__vtpm_devtype;
tools/libs/light/libxl_types.idl +18 -0
index 9e3d33cb5a59..0cfad8508dbd 100644
@@ -278,6 +278,11 @@ libxl_vkb_backend = Enumeration("vkb_backend", [
     (2, "LINUX")
     ])
 
+libxl_virtio_transport = Enumeration("virtio_transport", [
+    (0, "UNKNOWN"),
+    (1, "MMIO"),
+    ])
+
 libxl_passthrough = Enumeration("passthrough", [
     (0, "default"),
     (1, "disabled"),
@@ -703,6 +708,18 @@ libxl_device_vkb = Struct("device_vkb", [
     ("multi_touch_num_contacts", uint32)
     ])
 
+libxl_device_virtio = Struct("device_virtio", [
+    ("backend_domid", libxl_domid),
+    ("backend_domname", string),
+    ("type", string),
+    ("transport", libxl_virtio_transport),
+    ("devid", libxl_devid),
+    # Note that virtio-mmio parameters (irq and base) are for internal
+    # use by libxl and can't be modified.
+    ("irq", uint32),
+    ("base", uint64)
+    ])
+
 libxl_device_disk = Struct("device_disk", [
     ("backend_domid", libxl_domid),
     ("backend_domname", string),
@@ -980,6 +997,7 @@ libxl_domain_config = Struct("domain_config", [
     ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")),
     ("vfbs", Array(libxl_device_vfb, "num_vfbs")),
     ("vkbs", Array(libxl_device_vkb, "num_vkbs")),
+    ("virtios", Array(libxl_device_virtio, "num_virtios")),
     ("vtpms", Array(libxl_device_vtpm, "num_vtpms")),
     ("p9s", Array(libxl_device_p9, "num_p9s")),
     ("pvcallsifs", Array(libxl_device_pvcallsif, "num_pvcallsifs")),
tools/libs/light/libxl_types_internal.idl +1 -0
index fb0f4f23d7c2..e24288f1a59e 100644
@@ -33,6 +33,7 @@ libxl__device_kind = Enumeration("device_kind", [
     (15, "VSND"),
     (16, "VINPUT"),
     (17, "VIRTIO_DISK"),
+    (18, "VIRTIO"),
     ])
 
 libxl__console_backend = Enumeration("console_backend", [
tools/libs/light/libxl_virtio.c new file +144 -0

144 lines not on this page · raw message

Signature · 1 line
2.31.1.272.g89b43f80a514
#2 lore raw

PATCH v10 2/3 xl: Add support to parse generic virtio device

In reply to Viresh Kumar

This patch adds basic support for parsing generic Virtio backend.

An example of domain configuration for mmio based Virtio I2C device is:
virtio = ["type=virtio,device22,transport=mmio"]

---
 tools/xl/xl_parse.c | 81 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
tools/xl/xl_parse.c +81 -0
index 644ab8f8fd36..853e9f357a1a 100644
@@ -1208,6 +1208,83 @@ static void parse_vkb_list(const XLU_Config *config,
     if (rc) exit(EXIT_FAILURE);
 }
 
+static int parse_virtio_config(libxl_device_virtio *virtio, char *token)
+{
+    char *oparg;
+    int rc;
+
+    if (MATCH_OPTION("backend", token, oparg)) {
+        virtio->backend_domname = strdup(oparg);
+    } else if (MATCH_OPTION("type", token, oparg)) {
+        virtio->type = strdup(oparg);
+    } else if (MATCH_OPTION("transport", token, oparg)) {
+        rc = libxl_virtio_transport_from_string(oparg, &virtio->transport);
+        if (rc) return rc;
+    } else {
+        fprintf(stderr, "Unknown string \"%s\" in virtio spec\n", token);
+        return -1;
+    }
+
+    return 0;
+}
+
+static void parse_virtio_list(const XLU_Config *config,
+                              libxl_domain_config *d_config)
+{
+    XLU_ConfigList *virtios;
+    const char *item;
+    char *buf = NULL, *oparg, *str = NULL;
+    int rc;
+
+    if (!xlu_cfg_get_list (config, "virtio", &virtios, 0, 0)) {
+        int entry = 0;
+        while ((item = xlu_cfg_get_listitem(virtios, entry)) != NULL) {
+            libxl_device_virtio *virtio;
+            char *p;
+
+            virtio = ARRAY_EXTEND_INIT(d_config->virtios, d_config->num_virtios,
+                                       libxl_device_virtio_init);
+
+            buf = strdup(item);
+
+            p = strtok(buf, ",");
+            while (p != NULL)
+            {
+                while (*p == ' ') p++;
+
+                // Type may contain a comma, do special handling.
+                if (MATCH_OPTION("type", p, oparg)) {
+                    if (!strncmp(oparg, "virtio", strlen("virtio"))) {
+                        char *p2 = strtok(NULL, ",");
+                        str = malloc(strlen(p) + strlen(p2) + 2);
+
+                        strcpy(str, p);
+                        strcat(str, ",");
+                        strcat(str, p2);
+                        p = str;
+                    }
+                }
+
+                rc = parse_virtio_config(virtio, p);
+                if (rc) goto out;
+
+                free(str);
+                str = NULL;
+                p = strtok(NULL, ",");
+            }
+
+            entry++;
+            free(buf);
+        }
+    }
+
+    return;
+
+out:
+    free(buf);
+    if (rc) exit(EXIT_FAILURE);
+}
+
 void parse_config_data(const char *config_source,
                        const char *config_data,
                        int config_len,
@@ -2753,6 +2830,10 @@ void parse_config_data(const char *config_source,
 
     parse_vkb_list(config, d_config);
 
+    d_config->virtios = NULL;
+    d_config->num_virtios = 0;
+    parse_virtio_list(config, d_config);
+
     xlu_cfg_get_defbool(config, "xend_suspend_evtchn_compat",
                         &c_info->xend_suspend_evtchn_compat, 0);
 
Signature · 1 line
2.31.1.272.g89b43f80a514
#3 lore raw

PATCH v10 3/3 docs: Add documentation for generic virtio devices

In reply to Viresh Kumar

This patch updates xl.cfg man page with details of generic Virtio device related information.

---
 docs/man/xl.cfg.5.pod.in | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
docs/man/xl.cfg.5.pod.in +33 -0
index ec444fb2ba79..024bceeb61b2 100644
@@ -1585,6 +1585,39 @@ Set maximum height for pointer device.
 
 =back
 
+=item B<virtio=[ "VIRTIO_DEVICE_STRING", "VIRTIO_DEVICE_STRING", ...]>
+
+Specifies the Virtio devices to be provided to the guest.
+
+Each B<VIRTIO_DEVICE_STRING> is a comma-separated list of C<KEY=VALUE> settings
+from the following list. As a special case, a single comma is allowed in the
+VALUE of the "type" KEY, where the VALUE is set with "virtio,device<N>".
+
+=over 4
+
+=item B<backend=domain-id>
+
+Specifies the backend domain name or id, defaults to dom0.
+
+=item B<type=STRING>
+
+Specifies the compatible string for the specific Virtio device. The same will be
+written in the Device Tree compatible property of the Virtio device. For
+example, "type=virtio,device22" for the I2C device, whose device-tree binding is
+present here:
+
+L<https://www.kernel.org/doc/Documentation/devicetree/bindings/i2c/i2c-virtio.yaml>
+
+For generic virtio devices, where we don't need to set special or compatible
+properties in the Device Tree, the type field must be set to "virtio,device".
+
+=item B<transport=STRING>
+
+Specifies the transport mechanism for the Virtio device, only "mmio" is
+supported for now.
+
+=back
+
 =item B<tee="STRING">
 
 B<Arm only.> Set TEE type for the guest. TEE is a Trusted Execution
Signature · 1 line
2.31.1.272.g89b43f80a514
#4 lore raw
In reply to Viresh Kumar
Quoted 11 lines · Viresh Kumar

This patchset adds toolstack support for I2C, GPIO and generic virtio devices. This is inspired from the work done by Oleksandr for the Disk device.

This is developed as part of Linaro's Project Stratos, where we are working towards Hypervisor agnostic Rust based backends [1].

This is based of Xen's master branch.

V9->V10:
- Swap I2C/GPIO compatible strings, they were wrongly added earlier.
- Arrange tags in timely order and add new ones.

The affected patch surely needed all tags to be dropped which weren't explicitly indicated to remain valid across the bug fix.

Jan

#5 lore raw
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Viresh Kumar

This patchset adds toolstack support for I2C, GPIO and generic virtio devices. This is inspired from the work done by Oleksandr for the Disk device.

This is developed as part of Linaro's Project Stratos, where we are working towards Hypervisor agnostic Rust based backends [1].

Quoted 8 lines
Quoted 5 lines

This is based of Xen's master branch.

V9->V10:
- Swap I2C/GPIO compatible strings, they were wrongly added earlier.
- Arrange tags in timely order and add new ones.

The affected patch surely needed all tags to be dropped which weren't explicitly indicated to remain valid across the bug fix.

Hmm, since it was a very small change (only fixed the compatible strings) I thought it would be okay to carry the tags and rather harsh to the reviewers to drop them, since they invested a lot of time reviewing these. Though I do get what you are suggesting here.

Should I ask Oleksandr / Anthony to provide their tags for patch 1/3 again ?

Signature · 1 line
viresh
#6 lore raw
Quoted 17 lines · Viresh Kumar
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Viresh Kumar

This patchset adds toolstack support for I2C, GPIO and generic virtio devices. This is inspired from the work done by Oleksandr for the Disk device.

This is developed as part of Linaro's Project Stratos, where we are working towards Hypervisor agnostic Rust based backends [1].

Quoted 8 lines
Quoted 5 lines

This is based of Xen's master branch.

V9->V10:
- Swap I2C/GPIO compatible strings, they were wrongly added earlier.
- Arrange tags in timely order and add new ones.

The affected patch surely needed all tags to be dropped which weren't explicitly indicated to remain valid across the bug fix.

Quoted 7 lines

Hmm, since it was a very small change (only fixed the compatible strings) I thought it would be okay to carry the tags and rather harsh to the reviewers to drop them, since they invested a lot of time reviewing these. Though I do get what you are suggesting here.

Should I ask Oleksandr / Anthony to provide their tags for patch 1/3 again ?

Let's simply wait for them to confirm their tags.

Jan

#7 lore raw

On 14.12.22 11:09, Jan Beulich wrote:

Hello all

Quoted 16 lines · Viresh Kumar
Quoted 15 lines · Jan Beulich
Quoted 14 lines · Viresh Kumar
Quoted 11 lines

This patchset adds toolstack support for I2C, GPIO and generic virtio devices. This is inspired from the work done by Oleksandr for the Disk device.

This is developed as part of Linaro's Project Stratos, where we are working towards Hypervisor agnostic Rust based backends [1].

This is based of Xen's master branch.

V9->V10:
- Swap I2C/GPIO compatible strings, they were wrongly added earlier.
- Arrange tags in timely order and add new ones.

The affected patch surely needed all tags to be dropped which weren't explicitly indicated to remain valid across the bug fix.

Quoted 8 lines
Quoted 7 lines

Hmm, since it was a very small change (only fixed the compatible strings) I thought it would be okay to carry the tags and rather harsh to the reviewers to drop them, since they invested a lot of time reviewing these. Though I do get what you are suggesting here.

Should I ask Oleksandr / Anthony to provide their tags for patch 1/3 again ?

Let's simply wait for them to confirm their tags.

Viresh, I confirm that my R-b for #1/3 is still valid (thanks for making a change). I probably should have mentioned that explicitly when making a comment.

Quoted 2 lines

Jan

Signature · 3 lines
Regards,

Oleksandr Tyshchenko
#8 lore raw
In reply to Viresh Kumar
Quoted 14 lines · Viresh Kumar
Quoted 13 lines · Jan Beulich
Quoted 11 lines · Viresh Kumar

This patchset adds toolstack support for I2C, GPIO and generic virtio devices. This is inspired from the work done by Oleksandr for the Disk device.

This is developed as part of Linaro's Project Stratos, where we are working towards Hypervisor agnostic Rust based backends [1].

This is based of Xen's master branch.

V9->V10:
- Swap I2C/GPIO compatible strings, they were wrongly added earlier.
- Arrange tags in timely order and add new ones.

Quoted 7 lines
Quoted 2 lines

The affected patch surely needed all tags to be dropped which weren't explicitly indicated to remain valid across the bug fix.

Hmm, since it was a very small change (only fixed the compatible strings) I thought it would be okay to carry the tags and rather harsh to the reviewers to drop them, since they invested a lot of time reviewing these. Though I do get what you are suggesting here.

I would have carry over the tags as well in this case.

If the tag were dropped, it wouldn't really have been an issue for me, it doesn't take much time to review a patch series that I've already reviewed. I usually look at a diff with the previous version (with git range-diff, patchew can help as well[1]), and my previous comment if needed.

But dropping rvb tag of non-maintainer maybe more "harsh" as you says as they may not have time to re-apply their tags before patch gets committed.

Quoted 2 lines

Should I ask Oleksandr / Anthony to provide their tags for patch 1/3 again ?

My tag is still valid. Thanks.

Cheers,

[1] https://patchew.org/Xen/[email protected]/
Then look for "diff against: v9"

Signature · 1 line
Anthony PERARD