mdev based hardware virtio offloading support
Subsystems (16)
v12 cover mdev based hardware virtio offloading support
Hi all:
There are hardwares that can do virtio datapath offloading while having its own control path. This path tries to implement a mdev based unified API to support using kernel virtio driver to drive those devices. This is done by introducing a new mdev transport for virtio (virtio_mdev) and register itself as a new kind of mdev driver. Then it provides a unified way for kernel virtio driver to talk with mdev device implementation.
Though the series only contains kernel driver support, the goal is to make the transport generic enough to support userspace drivers. This means vhost-mdev[1] could be built on top as well by resuing the transport.
A sample driver is also implemented which simulate a virito-net loopback ethernet device on top of vringh + workqueue. This could be used as a reference implementation for real hardware driver.
Also a real IFC VF driver was also posted here[2] which is a good reference for vendors who is interested in their own virtio datapath offloading product.
Consider mdev framework only support VFIO device and driver right now, this series also extend it to support other types. This is done through decoupling VFIO specific bits out of mdev core and make mdev an independent module that allows to be used by multiple types of buses.
Pktgen test was done with virito-net + mvnet loop back device.
Please review.
[1] https://lkml.org/lkml/2019/11/5/424
[2] https://lkml.org/lkml/2019/11/5/227
Changes from V11:
- decouple VFIO specific bits out of mdev core
- make mdev an indepdent module to allow buses other than VFIO mdev
- allow structure composition of mdev through specifiy the size of mdev structure
- introduce mdev_vfio structure and store the VFIO specific callbacks there
- don't use "mdev" bus for virtio, use a new "mdev_virtio" bus, and store the virtio specific callbacks in mdev_virtio structure.
- do the class_id, matching on top of "mdev_virtio" bus
Changes from V10:
- rename mvnet to mvnet_loopback
- fix typo in the help text for sample Kconfig
Changes from V9:
- Tweak the help text for virito-mdev kconfig
Changes from V8:
- try silent checkpatch, some are still there becuase they were inherited from virtio_config_ops which needs to be resolved in an independent series
- tweak on the comment and doc
- remove VIRTIO_MDEV_F_VERSION_1 completely
- rename CONFIG_VIRTIO_MDEV_DEVICE to CONFIG_VIRTIO_MDEV
Changes from V7:
- drop {set|get}_mdev_features for virtio
- typo and comment style fixes
Changes from V6:
- rename ops files and compile guard
Changes from V5:
- use dev_warn() instead of WARN(1) when class id is not set
- validate id_table before trying to do matching between device and driver
- add wildcard for modpost script
- use unique name for id_table
- move get_mdev_features() to be the first member of virtio_device_ops and more comments for it
- typo fixes for the comments above virtio_mdev_ops
Changes from V4:
- keep mdev_set_class() for the device that doesn't use device ops
- use union for device ops pointer in mdev_device
- introduce class specific helper for getting is device ops
- use WARN_ON instead of BUG_ON in mdev_set_virtio_ops
- explain details of get_mdev_features() and get_vendor_id()
- distinguish the optional virito device ops from mandatory ones and make get_generation() optional
- rename vfio_mdev.h to vfio_mdev_ops.h, rename virito_mdev.h to virtio_mdev_ops.h
- don't abuse version fileds in virtio_mdev structure, use features instead
- fix warning during device remove
- style & docs tweaks and typo fixes
Changes from V3:
- document that class id (device ops) must be specified in create()
- add WARN() when trying to set class_id when it has already set
- add WARN() when class_id is not specified in create() and correctly return an error in this case
- correct the prototype of mdev_set_class() in the doc
- add documention of mdev_set_class()
- remove the unnecessary "class_id_fail" label when class id is not specified in create()
- convert id_table in vfio_mdev to const
- move mdev_set_class and its friends after mdev_uuid()
- suqash the patch of bus uevent into patch of introducing class id
- tweak the words in the docs per Cornelia suggestion
- tie class_id and device ops through class specific initialization routine like mdev_set_vfio_ops()
- typos fixes in the docs of virtio-mdev callbacks
- document the usage of virtqueues in struct virtio_mdev_device
- remove the useless vqs array in struct virtio_mdev_device
- rename MDEV_ID_XXX to MDEV_CLASS_ID_XXX
Changes from V2:
- fail when class_id is not specified
- drop the vringh patch
- match the doc to the code
- tweak the commit log
- move device_ops from parent to mdev device
- remove the unused MDEV_ID_VHOST
Changes from V1:
- move virtio_mdev.c to drivers/virtio
- store class_id in mdev_device instead of mdev_parent
- store device_ops in mdev_device instead of mdev_parent
- reorder the patch, vringh fix comes first
- really silent compiling warnings
- really switch to use u16 for class_id
- uevent and modpost support for mdev class_id
- vraious tweaks per comments from Parav
Changes from RFC-V2:
- silent compile warnings on some specific configuration
- use u16 instead u8 for class id
- reseve MDEV_ID_VHOST for future vhost-mdev work
- introduce "virtio" type for mvnet and make "vhost" type for future work
- add entries in MAINTAINER
- tweak and typos fixes in commit log
Changes from RFC-V1:
- rename device id to class id
- add docs for class id and device specific ops (device_ops)
- split device_ops into seperate headers
- drop the mdev_set_dma_ops()
- use device_ops to implement the transport API, then it's not a part of UAPI any more
- use GFP_ATOMIC in mvnet sample device and other tweaks
- set_vring_base/get_vring_base support for mvnet device
Jason Wang (6):
mdev: make mdev bus agnostic
mdev: split out VFIO bus specific parent ops
mdev: move to drivers/
mdev: introduce mediated virtio bus
virtio: introduce a mdev based transport
docs: sample driver to demonstrate how to implement virtio-mdev
framework
.../driver-api/vfio-mediated-device.rst | 94 ++-
MAINTAINERS | 11 +-
drivers/Kconfig | 2 +
drivers/Makefile | 1 +
drivers/gpu/drm/i915/gvt/kvmgt.c | 26 +-
drivers/mdev/Kconfig | 29 +
drivers/mdev/Makefile | 7 +
drivers/{vfio => }/mdev/mdev_core.c | 107 ++-
drivers/{vfio => }/mdev/mdev_driver.c | 29 +-
drivers/{vfio => }/mdev/mdev_private.h | 18 +-
drivers/{vfio => }/mdev/mdev_sysfs.c | 0
drivers/mdev/vfio.c | 76 ++
drivers/mdev/virtio.c | 126 ++++
drivers/s390/cio/vfio_ccw_ops.c | 23 +-
drivers/s390/crypto/vfio_ap_ops.c | 34 +-
drivers/s390/crypto/vfio_ap_private.h | 2 +-
drivers/vfio/mdev/Kconfig | 11 -
drivers/vfio/mdev/Makefile | 4 -
drivers/vfio/mdev/vfio_mdev.c | 45 +-
drivers/vfio/vfio_iommu_type1.c | 6 +-
drivers/virtio/Kconfig | 13 +
drivers/virtio/Makefile | 1 +
drivers/virtio/virtio_mdev.c | 409 +++++++++++
include/linux/mdev.h | 49 +-
include/linux/mdev_vfio.h | 68 ++
include/linux/mdev_virtio.h | 168 +++++
include/linux/mod_devicetable.h | 8 +
samples/Kconfig | 10 +
samples/vfio-mdev/Makefile | 1 +
samples/vfio-mdev/mbochs.c | 26 +-
samples/vfio-mdev/mdpy.c | 27 +-
samples/vfio-mdev/mtty.c | 24 +-
samples/vfio-mdev/mvnet_loopback.c | 690 ++++++++++++++++++
scripts/mod/devicetable-offsets.c | 3 +
scripts/mod/file2alias.c | 12 +
35 files changed, 1924 insertions(+), 236 deletions(-)
create mode 100644 drivers/mdev/Kconfig
create mode 100644 drivers/mdev/Makefile
rename drivers/{vfio => }/mdev/mdev_core.c (76%)
rename drivers/{vfio => }/mdev/mdev_driver.c (83%)
rename drivers/{vfio => }/mdev/mdev_private.h (81%)
rename drivers/{vfio => }/mdev/mdev_sysfs.c (100%)
create mode 100644 drivers/mdev/vfio.c
create mode 100644 drivers/mdev/virtio.c
create mode 100644 drivers/virtio/virtio_mdev.c
create mode 100644 include/linux/mdev_vfio.h
create mode 100644 include/linux/mdev_virtio.h
create mode 100644 samples/vfio-mdev/mvnet_loopback.cSignature · 6 lines
2.19.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v12 1/6 mdev: make mdev bus agnostic
Current mdev is tied to a VFIO specific "mdev" bus. This prevent mdev from being used by other types of API/buses. So this patch tries to make mdev bus agnostic through making a mdev core a thin module:
- decouple VFIO bus specific bits from mdev_core.c to mdev_vfio.c and introduce mdev_vfio module
- require to specify the type of bus when registering mdev device and mdev driver
With those modifications mdev become a generic module that could be used by multiple types of virtual buses and devices.
--- .../driver-api/vfio-mediated-device.rst | 68 ++++++------ MAINTAINERS | 1 + drivers/gpu/drm/i915/gvt/kvmgt.c | 10 +- drivers/s390/cio/vfio_ccw_ops.c | 6 +- drivers/s390/crypto/vfio_ap_ops.c | 21 ++-- drivers/s390/crypto/vfio_ap_private.h | 2 +- drivers/vfio/mdev/Kconfig | 17 ++- drivers/vfio/mdev/Makefile | 4 +- drivers/vfio/mdev/mdev_core.c | 104 +++++++++++++----- drivers/vfio/mdev/mdev_driver.c | 29 ++--- drivers/vfio/mdev/mdev_private.h | 13 ++- drivers/vfio/mdev/mdev_vfio.c | 48 ++++++++ drivers/vfio/mdev/vfio_mdev.c | 5 +- drivers/vfio/vfio_iommu_type1.c | 6 +- include/linux/mdev.h | 16 ++- include/linux/mdev_vfio.h | 25 +++++ samples/vfio-mdev/mbochs.c | 8 +- samples/vfio-mdev/mdpy.c | 8 +- samples/vfio-mdev/mtty.c | 8 +- 19 files changed, 270 insertions(+), 129 deletions(-) create mode 100644 drivers/vfio/mdev/mdev_vfio.c create mode 100644 include/linux/mdev_vfio.h
Documentation/driver-api/vfio-mediated-device.rst +36 -32
@@ -49,35 +49,37 @@ devices as examples, as these devices are the first devices to use this module:: +---------------+ | | - | +-----------+ | mdev_register_driver() +--------------+ - | | | +<------------------------+ | - | | mdev | | | | - | | bus | +------------------------>+ vfio_mdev.ko |<-> VFIO user - | | driver | | probe()/remove() | | APIs - | | | | +--------------+ - | +-----------+ | + | MDEV CORE | mdev_register_driver() +--------------+ + | MODULE +<------------------------+ | + | mdev.ko | | | + | +------------------------>+ vfio_mdev.ko |<-> VFIO user + | | probe()/remove() | | APIs + | | +--------------+ + +---+-------+---+ + | /|\ + | | +callbacks| | mdev_register_device() + | | mdev_register_bus() + \|/ | + +---+-------+---+ + | | mdev_vfio_register_device() +--------------+ + | +<-----------------------------+ | + | | | nvidia.ko |<-> physical + | +----------------------------->+ | device + | MDEV VFIO | callbacks +--------------+ + | Physical | + | device | mdev_vfio_register_device() +--------------+ + | interface |<-----------------------------+ | + | | | i915.ko |<-> physical + | mdev_vfio.ko +----------------------------->+ | device + | | callbacks +--------------+ + | | + | | mdev_vfio_register_device() +--------------+ + | +<-----------------------------+ | + | | | ccw_device.ko|<-> physical + | +----------------------------->+ | device + | | callbacks +--------------+ | | - | MDEV CORE | - | MODULE | - | mdev.ko | - | +-----------+ | mdev_register_device() +--------------+ - | | | +<------------------------+ | - | | | | | nvidia.ko |<-> physical - | | | +------------------------>+ | device - | | | | callbacks +--------------+ - | | Physical | | - | | device | | mdev_register_device() +--------------+ - | | interface | |<------------------------+ | - | | | | | i915.ko |<-> physical - | | | +------------------------>+ | device - | | | | callbacks +--------------+ - | | | | - | | | | mdev_register_device() +--------------+ - | | | +<------------------------+ | - | | | | | ccw_device.ko|<-> physical - | | | +------------------------>+ | device - | | | | callbacks +--------------+ - | +-----------+ | +---------------+
@@ -116,7 +118,8 @@ to register and unregister itself with the core driver: * Register:: extern int mdev_register_driver(struct mdev_driver *drv, - struct module *owner); + struct module *owner, + struct bus_type *bus); * Unregister::
@@ -159,11 +162,12 @@ The callbacks in the mdev_parent_ops structure are as follows: * write: write emulation callback * mmap: mmap emulation callback -A driver should use the mdev_parent_ops structure in the function call to -register itself with the mdev core driver:: +A driver should use the mdev_parent_ops structure and bus type in the +function call to register itself with the mdev core driver:: extern int mdev_register_device(struct device *dev, - const struct mdev_parent_ops *ops); + const struct mdev_parent_ops *ops, + struct bus_type *bus); However, the mdev_parent_ops structure is not required in the function call that a driver should use to unregister itself with the mdev core driver::
MAINTAINERS +1 -0
@@ -17121,6 +17121,7 @@ S: Maintained F: Documentation/driver-api/vfio-mediated-device.rst F: drivers/vfio/mdev/ F: include/linux/mdev.h +F: include/linux/mdev_vfio.h F: samples/vfio-mdev/ VFIO PLATFORM DRIVER
drivers/gpu/drm/i915/gvt/kvmgt.c +5 -5
@@ -41,7 +41,7 @@#include <linux/uuid.h> #include <linux/kvm_host.h> #include <linux/vfio.h> -#include <linux/mdev.h> +#include <linux/mdev_vfio.h> #include <linux/debugfs.h> #include <linux/nospec.h>
@@ -1554,7 +1554,7 @@ static ssize_t vgpu_id_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct mdev_device *mdev = mdev_from_dev(dev); + struct mdev_device *mdev = vfio_mdev_from_dev(dev);if (mdev) { struct intel_vgpu *vgpu = (struct intel_vgpu *)
@@ -1568,7 +1568,7 @@ static ssize_t hw_id_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct mdev_device *mdev = mdev_from_dev(dev); + struct mdev_device *mdev = vfio_mdev_from_dev(dev);if (mdev) { struct intel_vgpu *vgpu = (struct intel_vgpu *)
@@ -1623,12 +1623,12 @@ static int kvmgt_host_init(struct device *dev, void *gvt, const void *ops)return -EFAULT ; intel_vgpu_ops.supported_type_groups = kvm_vgpu_type_groups; -return mdev_register_device (dev, &intel_vgpu_ops); +return mdev_vfio_register_device (dev, &intel_vgpu_ops); }static void kvmgt_host_exit (struct device *dev) { -mdev_unregister_device (dev); +mdev_vfio_unregister_device (dev); }static int kvmgt_page_track_add (unsigned long handle,u64 gfn)
drivers/s390/cio/vfio_ccw_ops.c +3 -3
@@ -11,7 +11,7 @@*/ #include <linux/vfio.h> -#include <linux/mdev.h> +#include <linux/mdev_vfio.h> #include <linux/nospec.h> #include <linux/slab.h>
@@ -588,10 +588,10 @@ static const struct mdev_parent_ops vfio_ccw_mdev_ops = {int vfio_ccw_mdev_reg (struct subchannel *sch) { -return mdev_register_device (&sch->dev, &vfio_ccw_mdev_ops); +return mdev_vfio_register_device (&sch->dev, &vfio_ccw_mdev_ops); }void vfio_ccw_mdev_unreg (struct subchannel *sch) { -mdev_unregister_device (&sch->dev); +mdev_vfio_unregister_device (&sch->dev); }
drivers/s390/crypto/vfio_ap_ops.c +11 -10
@@ -602,7 +602,7 @@ static ssize_t assign_adapter_store(struct device *dev, {int ret;unsigned long apid; -struct mdev_device *mdev =mdev_from_dev (dev); +struct mdev_device *mdev =vfio_mdev_from_dev (dev);struct ap_matrix_mdev *matrix_mdev =mdev_get_drvdata (mdev);/* If the guest is running, disallow assignment of adapter */
@@ -668,7 +668,7 @@ static ssize_t unassign_adapter_store(struct device *dev, {int ret;unsigned long apid; -struct mdev_device *mdev =mdev_from_dev (dev); +struct mdev_device *mdev =vfio_mdev_from_dev (dev);struct ap_matrix_mdev *matrix_mdev =mdev_get_drvdata (mdev);/* If the guest is running, disallow un-assignment of adapter */
@@ -748,7 +748,7 @@ static ssize_t assign_domain_store(struct device *dev, {int ret;unsigned long apqi; -struct mdev_device *mdev =mdev_from_dev (dev); +struct mdev_device *mdev =vfio_mdev_from_dev (dev);struct ap_matrix_mdev *matrix_mdev =mdev_get_drvdata (mdev);unsigned long max_apqi = matrix_mdev->matrix.aqm_max;
@@ -810,7 +810,7 @@ static ssize_t unassign_domain_store(struct device *dev, {int ret;unsigned long apqi; -struct mdev_device *mdev =mdev_from_dev (dev); +struct mdev_device *mdev =vfio_mdev_from_dev (dev);struct ap_matrix_mdev *matrix_mdev =mdev_get_drvdata (mdev);/* If the guest is running, disallow un-assignment of domain */
@@ -854,7 +854,7 @@ static ssize_t assign_control_domain_store(struct device *dev, {int ret;unsigned long id; -struct mdev_device *mdev =mdev_from_dev (dev); +struct mdev_device *mdev =vfio_mdev_from_dev (dev);struct ap_matrix_mdev *matrix_mdev =mdev_get_drvdata (mdev);/* If the guest is running, disallow assignment of control domain */
@@ -903,7 +903,7 @@ static ssize_t unassign_control_domain_store(struct device *dev, {int ret;unsigned long domid; -struct mdev_device *mdev =mdev_from_dev (dev); +struct mdev_device *mdev =vfio_mdev_from_dev (dev);struct ap_matrix_mdev *matrix_mdev =mdev_get_drvdata (mdev);unsigned long max_domid = matrix_mdev->matrix.adm_max;
@@ -933,7 +933,7 @@ static ssize_t control_domains_show(struct device *dev,int nchars =0 ;int n;char *bufpos = buf; -struct mdev_device *mdev =mdev_from_dev (dev); +struct mdev_device *mdev =vfio_mdev_from_dev (dev);struct ap_matrix_mdev *matrix_mdev =mdev_get_drvdata (mdev);unsigned long max_domid = matrix_mdev->matrix.adm_max;
@@ -952,7 +952,7 @@ static DEVICE_ATTR_RO(control_domains);static ssize_t matrix_show (struct device *dev,struct device_attribute *attr,char *buf) { -struct mdev_device *mdev =mdev_from_dev (dev); +struct mdev_device *mdev =vfio_mdev_from_dev (dev);struct ap_matrix_mdev *matrix_mdev =mdev_get_drvdata (mdev);char *bufpos = buf;unsigned long apid;
@@ -1295,10 +1295,11 @@ int vfio_ap_mdev_register(void) {atomic_set (&matrix_dev->available_instances,MAX_ZDEV_ENTRIES_EXT ); -return mdev_register_device (&matrix_dev->device, &vfio_ap_matrix_ops); +return mdev_vfio_register_device (&matrix_dev->device, + &vfio_ap_matrix_ops); }void vfio_ap_mdev_unregister (void ) { -mdev_unregister_device (&matrix_dev->device); +mdev_vfio_unregister_device (&matrix_dev->device); }
drivers/s390/crypto/vfio_ap_private.h +1 -1
@@ -14,7 +14,7 @@#include <linux/types.h> #include <linux/device.h> -#include <linux/mdev.h> +#include <linux/mdev_vfio.h> #include <linux/delay.h> #include <linux/mutex.h> #include <linux/kvm_host.h>
drivers/vfio/mdev/Kconfig +13 -4
@@ -1,15 +1,24 @@ -# SPDX-License-Identifier: GPL-2.0-only -config VFIO_MDEV +config MDEV tristate "Mediated device driver framework" - depends on VFIO default n help Provides a framework to virtualize devices. - See Documentation/driver-api/vfio-mediated-device.rst for more details. If you don't know what do here, say N. +config VFIO_MDEV + tristate "VFIO Mediated device driver" + depends on VFIO && MDEV + default n + help + Proivdes a mediated BUS for userspace driver through VFIO + framework. See Documentation/vfio-mediated-device.txt for + more details. + + If you don't know what do here, say N. + + config VFIO_MDEV_DEVICE tristate "VFIO driver for Mediated devices" depends on VFIO && VFIO_MDEV
drivers/vfio/mdev/Makefile +2 -2
@@ -1,6 +1,6 @@ -# SPDX-License-Identifier: GPL-2.0-only mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o -obj-$(CONFIG_VFIO_MDEV) += mdev.o +obj-$(CONFIG_MDEV) += mdev.o +obj-$(CONFIG_VFIO_MDEV) += mdev_vfio.o obj-$(CONFIG_VFIO_MDEV_DEVICE) += vfio_mdev.o
drivers/vfio/mdev/mdev_core.c +78 -26
189 lines not on this page · raw message
drivers/vfio/mdev/mdev_driver.c +9 -20
@@ -10,6 +10,7 @@#include <linux/device.h> #include <linux/iommu.h> #include <linux/mdev.h> +#include <linux/mdev_vfio.h> #include "mdev_private.h"
@@ -37,7 +38,7 @@ static void mdev_detach_iommu(struct mdev_device *mdev)dev_info (&mdev->dev,"MDEV: detaching iommu\n" ); } -static int mdev_probe (struct device *dev) +int mdev_probe (struct device *dev) {struct mdev_driver *drv =to_mdev_driver (dev->driver);struct mdev_device *mdev =to_mdev_device (dev);
@@ -55,8 +56,9 @@ static int mdev_probe(struct device *dev)return ret; } +EXPORT_SYMBOL (mdev_probe); -static int mdev_remove (struct device *dev) +int mdev_remove (struct device *dev) {struct mdev_driver *drv =to_mdev_driver (dev->driver);struct mdev_device *mdev =to_mdev_device (dev);
@@ -68,26 +70,22 @@ static int mdev_remove(struct device *dev)return 0 ; } - -struct bus_type mdev_bus_type = { - .name ="mdev" , - .probe = mdev_probe, - .remove = mdev_remove, -}; -EXPORT_SYMBOL_GPL (mdev_bus_type); +EXPORT_SYMBOL (mdev_remove);/** * mdev_register_driver - register a new MDEV driver * @drv: the driver to register * @owner: module owner of driver to be registered +* @bus: but that the driver wants to attach * * Returns a negative value on error, otherwise 0. **/ -int mdev_register_driver (struct mdev_driver *drv,struct module *owner) +int mdev_register_driver (struct mdev_driver *drv,struct module *owner, +struct bus_type *bus) {/* initialize common driver fields */ drv->driver.name = drv->name; - drv->driver.bus = &mdev_bus_type; + drv->driver.bus = bus; drv->driver.owner = owner;/* register with core */
@@ -105,12 +103,3 @@ void mdev_unregister_driver(struct mdev_driver *drv) }EXPORT_SYMBOL (mdev_unregister_driver); -int mdev_bus_register (void ) -{ -return bus_register (&mdev_bus_type); -} - -void mdev_bus_unregister (void ) -{ -bus_unregister (&mdev_bus_type); -}
drivers/vfio/mdev/mdev_private.h +9 -4
@@ -10,12 +10,10 @@ #ifndef MDEV_PRIVATE_H #define MDEV_PRIVATE_H -int mdev_bus_register(void); -void mdev_bus_unregister(void); - struct mdev_parent { struct device *dev; const struct mdev_parent_ops *ops; + struct bus_type *bus; struct kref ref; struct list_head next; struct kset *mdev_types_kset;
@@ -35,8 +33,15 @@ struct mdev_device {bool active; }; +struct mdev_class_compat { +struct class_compat *class_compat; +struct bus_type *bus; +struct list_head next; +}; + +#define to_mdev_device (dev) container_of(dev, struct mdev_device, dev) -#define dev_is_mdev (d) ((d)->bus == &mdev_bus_type) +#define dev_is_mdev (d, bus) ((d)->bus == bus)struct mdev_type {struct kobject kobj;
drivers/vfio/mdev/mdev_vfio.c new file +48 -0
48 lines not on this page · raw message
drivers/vfio/mdev/vfio_mdev.c +3 -2
@@ -13,7 +13,7 @@#include <linux/kernel.h> #include <linux/slab.h> #include <linux/vfio.h> -#include <linux/mdev.h> +#include <linux/mdev_vfio.h> #include "mdev_private.h"
@@ -128,7 +128,8 @@ static struct mdev_driver vfio_mdev_driver = {static int __init vfio_mdev_init (void ) { -return mdev_register_driver (&vfio_mdev_driver,THIS_MODULE ); +return mdev_register_driver (&vfio_mdev_driver,THIS_MODULE , + &mdev_vfio_bus_type); } static void __exit vfio_mdev_exit(void)
drivers/vfio/vfio_iommu_type1.c +3 -3
@@ -34,7 +34,7 @@#include <linux/uaccess.h> #include <linux/vfio.h> #include <linux/workqueue.h> -#include <linux/mdev.h> +#include <linux/mdev_vfio.h> #include <linux/notifier.h> #include <linux/dma-iommu.h> #include <linux/irqdomain.h>
@@ -1405,10 +1405,10 @@ static bool vfio_bus_is_mdev(struct bus_type *bus)struct bus_type *mdev_bus;bool ret = false; - mdev_bus =symbol_get (mdev_bus_type); + mdev_bus =symbol_get (mdev_vfio_bus_type);if (mdev_bus) { ret = (bus == mdev_bus); -symbol_put (mdev_bus_type); +symbol_put (mdev_vfio_bus_type); }return ret;
include/linux/mdev.h +11 -5
@@ -133,16 +133,22 @@ void *mdev_get_drvdata(struct mdev_device *mdev);void mdev_set_drvdata (struct mdev_device *mdev,void *data);const guid_t *mdev_uuid (struct mdev_device *mdev); -extern struct bus_type mdev_bus_type; - -int mdev_register_device (struct device *dev,const struct mdev_parent_ops *ops); +int mdev_register_device (struct device *dev,const struct mdev_parent_ops *ops, +struct bus_type *bus);void mdev_unregister_device (struct device *dev); -int mdev_register_driver (struct mdev_driver *drv,struct module *owner); +int mdev_register_driver (struct mdev_driver *drv,struct module *owner, +struct bus_type *bus);void mdev_unregister_driver (struct mdev_driver *drv);struct device *mdev_parent_dev (struct mdev_device *mdev);struct device *mdev_dev (struct mdev_device *mdev); -struct mdev_device *mdev_from_dev (struct device *dev); +struct mdev_device *mdev_from_dev (struct device *dev,struct bus_type *bus); + +int mdev_probe (struct device *dev); +int mdev_remove (struct device *dev); + +int mdev_register_bus (struct bus_type *bus); +void mdev_unregister_bus (struct bus_type *bus);#endif /* MDEV_H */
include/linux/mdev_vfio.h new file +25 -0
@@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* VFIO Mediated device definition +* +* Copyright (c) 2019, Red Hat. All rights reserved. +* Author: Jason Wang <[email protected]> +*/ + +#ifndef MDEV_VFIO_H +#define MDEV_VFIO_H + +#include <linux/mdev.h> + +extern struct bus_type mdev_vfio_bus_type; + +int mdev_vfio_register_device (struct device *dev, +const struct mdev_parent_ops *ops); +void mdev_vfio_unregister_device (struct device *dev); + +static inline struct mdev_device *vfio_mdev_from_dev (struct device *dev) +{ +return mdev_from_dev (dev, &mdev_vfio_bus_type); +} + +#endif
samples/vfio-mdev/mbochs.c +4 -4
@@ -29,7 +29,7 @@#include <linux/vfio.h> #include <linux/iommu.h> #include <linux/sysfs.h> -#include <linux/mdev.h> +#include <linux/mdev_vfio.h> #include <linux/pci.h> #include <linux/dma-buf.h> #include <linux/highmem.h>
@@ -1332,7 +1332,7 @@ static ssize_t memory_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct mdev_device *mdev = mdev_from_dev(dev); + struct mdev_device *mdev = vfio_mdev_from_dev(dev);struct mdev_state *mdev_state =mdev_get_drvdata (mdev);return sprintf (buf,"%d MB\n" , mdev_state->type->mbytes);
@@ -1468,7 +1468,7 @@ static int __init mbochs_dev_init(void)if (ret) goto failed2; - ret =mdev_register_device (&mbochs_dev, &mdev_fops); + ret =mdev_vfio_register_device (&mbochs_dev, &mdev_fops);if (ret) goto failed3;
@@ -1487,7 +1487,7 @@ static int __init mbochs_dev_init(void) static void __exit mbochs_dev_exit(void) { mbochs_dev.bus =NULL ; -mdev_unregister_device (&mbochs_dev); +mdev_vfio_unregister_device (&mbochs_dev);device_unregister (&mbochs_dev);cdev_del (&mbochs_cdev);
samples/vfio-mdev/mdpy.c +4 -4
@@ -25,7 +25,7 @@#include <linux/vfio.h> #include <linux/iommu.h> #include <linux/sysfs.h> -#include <linux/mdev.h> +#include <linux/mdev_vfio.h> #include <linux/pci.h> #include <drm/drm_fourcc.h> #include "mdpy-defs.h"
@@ -639,7 +639,7 @@ static ssize_t resolution_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct mdev_device *mdev = mdev_from_dev(dev); + struct mdev_device *mdev = vfio_mdev_from_dev(dev);struct mdev_state *mdev_state =mdev_get_drvdata (mdev); return sprintf(buf, "%dx%d\n",
@@ -775,7 +775,7 @@ static int __init mdpy_dev_init(void)if (ret) goto failed2; - ret =mdev_register_device (&mdpy_dev, &mdev_fops); + ret =mdev_vfio_register_device (&mdpy_dev, &mdev_fops);if (ret) goto failed3;
@@ -794,7 +794,7 @@ static int __init mdpy_dev_init(void) static void __exit mdpy_dev_exit(void) { mdpy_dev.bus =NULL ; -mdev_unregister_device (&mdpy_dev); +mdev_vfio_unregister_device (&mdpy_dev);device_unregister (&mdpy_dev);cdev_del (&mdpy_cdev);
samples/vfio-mdev/mtty.c +4 -4
@@ -26,7 +26,7 @@#include <linux/sysfs.h> #include <linux/ctype.h> #include <linux/file.h> -#include <linux/mdev.h> +#include <linux/mdev_vfio.h> #include <linux/pci.h> #include <linux/serial.h> #include <uapi/linux/serial_reg.h>
@@ -1285,7 +1285,7 @@ static ssize_t sample_mdev_dev_show(struct device *dev, struct device_attribute *attr, char *buf) { - if (mdev_from_dev(dev)) + if (vfio_mdev_from_dev(dev)) return sprintf(buf, "This is MDEV %s\n", dev_name(dev)); return sprintf(buf, "\n");
@@ -1445,7 +1445,7 @@ static int __init mtty_dev_init(void)if (ret) goto failed2; - ret =mdev_register_device (&mtty_dev.dev, &mdev_fops); + ret =mdev_vfio_register_device (&mtty_dev.dev, &mdev_fops);if (ret) goto failed3;
@@ -1471,7 +1471,7 @@ static int __init mtty_dev_init(void) static void __exit mtty_dev_exit(void) { mtty_dev.dev.bus =NULL ; -mdev_unregister_device (&mtty_dev.dev); +mdev_vfio_unregister_device (&mtty_dev.dev);device_unregister (&mtty_dev.dev);idr_destroy (&mtty_dev.vd_idr);
Signature · 6 lines
2.19.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v12 2/6 mdev: split out VFIO bus specific parent ops
In reply to Jason WangThe only thing left for generalizing mdev is the VFIO specific parent ops. This is basically the open/release/read/write/ioctl/mmap.
To support this, mdev core is extend to support a specific size of structure during create, this will allow to compose mdev structure into mdev vfio structure and place the VFIO specific callbacks there like:
struct mdev_vfio {
struct mdev_device mdev;
const struct mdev_vfio_ops *ops;
};Helpers for setting and getting the ops were introduced to support mdev vfio device to set ops and vfio mdev driver to use the ops.
--- .../driver-api/vfio-mediated-device.rst | 34 +++++++++------ drivers/gpu/drm/i915/gvt/kvmgt.c | 16 ++++--- drivers/s390/cio/vfio_ccw_ops.c | 17 +++++--- drivers/s390/crypto/vfio_ap_ops.c | 13 ++++-- drivers/vfio/mdev/mdev_core.c | 5 ++- drivers/vfio/mdev/mdev_private.h | 5 +++ drivers/vfio/mdev/mdev_vfio.c | 30 ++++++++++++- drivers/vfio/mdev/vfio_mdev.c | 38 ++++++++-------- include/linux/mdev.h | 37 ---------------- include/linux/mdev_vfio.h | 43 +++++++++++++++++++ samples/vfio-mdev/mbochs.c | 18 +++++--- samples/vfio-mdev/mdpy.c | 19 +++++--- samples/vfio-mdev/mtty.c | 16 ++++--- 13 files changed, 189 insertions(+), 102 deletions(-)
Documentation/driver-api/vfio-mediated-device.rst +22 -12
@@ -153,26 +153,36 @@ callbacks per mdev parent device, per mdev type, or any other categorization. Vendor drivers are expected to be fully asynchronous in this respect or provide their own internal resource protection.) -The callbacks in the mdev_parent_ops structure are as follows: +A driver should use the mdev_parent_ops structure in the function call +to register itself with the mdev core driver:: -* open: open callback of mediated device -* close: close callback of mediated device -* ioctl: ioctl callback of mediated device + extern int mdev_vfio_register_device(struct device *dev, + const struct mdev_parent_ops *ops); + +However, the mdev_parent_ops structure is not required in the function call +that a driver should use to unregister itself with the mdev core driver:: + + extern void mdev_vfio_unregister_device(struct device *dev); + +The VFIO specific callbacks is abstracted in mdev_vfio_ops structure +are as follows: + +* open: open callback of VFIO mediated device +* close: close callback of VFIO mediated device +* ioctl: ioctl callback of VFIO mediated device * read : read emulation callback * write: write emulation callback * mmap: mmap emulation callback -A driver should use the mdev_parent_ops structure and bus type in the -function call to register itself with the mdev core driver:: +During the creation of VFIO mediated device, mdev_vfio_ops need to be +specified:: - extern int mdev_register_device(struct device *dev, - const struct mdev_parent_ops *ops, - struct bus_type *bus); + void mdev_vfio_set_ops(struct mdev_device *mdev, + const struct mdev_vfio_ops *ops); -However, the mdev_parent_ops structure is not required in the function call -that a driver should use to unregister itself with the mdev core driver:: +Those callbacks could be fetched by drivers through:: - extern void mdev_unregister_device(struct device *dev); + const struct mdev_vfio_ops *mdev_vfio_get_ops(struct mdev_device *mdev); Mediated Device Management Interface Through sysfs
drivers/gpu/drm/i915/gvt/kvmgt.c +10 -6
@@ -643,6 +643,8 @@ static void kvmgt_put_vfio_device(void *vgpu)vfio_device_put (((struct intel_vgpu *)vgpu)->vdev.vfio_device); } +static const struct mdev_vfio_ops intel_mdev_vfio_ops; +static int intel_vgpu_create (struct kobject *kobj,struct mdev_device *mdev) {struct intel_vgpu *vgpu =NULL ;
@@ -678,6 +680,7 @@ static int intel_vgpu_create(struct kobject *kobj, struct mdev_device *mdev) dev_name(mdev_dev(mdev))); ret =0 ; +mdev_vfio_set_ops (mdev, &intel_mdev_vfio_ops); out:return ret; }
@@ -1598,20 +1601,21 @@ static const struct attribute_group *intel_vgpu_groups[] = { NULL, }; -static struct mdev_parent_ops intel_vgpu_ops = { - .mdev_attr_groups = intel_vgpu_groups, - .create = intel_vgpu_create, - .remove = intel_vgpu_remove, - +static const struct mdev_vfio_ops intel_mdev_vfio_ops = { .open = intel_vgpu_open, .release = intel_vgpu_release, - .read = intel_vgpu_read, .write = intel_vgpu_write, .mmap = intel_vgpu_mmap, .ioctl = intel_vgpu_ioctl, }; +static struct mdev_parent_ops intel_vgpu_ops = { + .mdev_attr_groups = intel_vgpu_groups, + .create = intel_vgpu_create, + .remove = intel_vgpu_remove, +}; +static int kvmgt_host_init (struct device *dev,void *gvt,const void *ops) {struct attribute **kvm_type_attrs;
drivers/s390/cio/vfio_ccw_ops.c +12 -5
@@ -110,6 +110,8 @@ static struct attribute_group *mdev_type_groups[] = { NULL, }; +static const struct mdev_vfio_ops vfio_ccw_mdev_vfio_ops; +static int vfio_ccw_mdev_create (struct kobject *kobj,struct mdev_device *mdev) { struct vfio_ccw_private *private =
@@ -129,6 +131,8 @@ static int vfio_ccw_mdev_create(struct kobject *kobj, struct mdev_device *mdev) private->sch->schid.ssid, private->sch->schid.sch_no); +mdev_vfio_set_ops (mdev, &vfio_ccw_mdev_vfio_ops); +return 0 ; }
@@ -574,16 +578,19 @@ static ssize_t vfio_ccw_mdev_ioctl(struct mdev_device *mdev, } } -static const struct mdev_parent_ops vfio_ccw_mdev_ops = { - .owner =THIS_MODULE , - .supported_type_groups = mdev_type_groups, - .create = vfio_ccw_mdev_create, - .remove = vfio_ccw_mdev_remove, +static const struct mdev_vfio_ops vfio_ccw_mdev_vfio_ops = { .open = vfio_ccw_mdev_open, .release = vfio_ccw_mdev_release, .read = vfio_ccw_mdev_read, .write = vfio_ccw_mdev_write, .ioctl = vfio_ccw_mdev_ioctl, +} + +static const struct mdev_parent_ops vfio_ccw_mdev_ops = { + .owner =THIS_MODULE , + .supported_type_groups = mdev_type_groups, + .create = vfio_ccw_mdev_create, + .remove = vfio_ccw_mdev_remove, };int vfio_ccw_mdev_reg (struct subchannel *sch)
drivers/s390/crypto/vfio_ap_ops.c +10 -3
@@ -321,6 +321,8 @@ static void vfio_ap_matrix_init(struct ap_config_info *info, matrix->adm_max = info->apxa ? info->Nd :15 ; } +static const struct mdev_vfio_ops vfio_ap_matrix_mdev_ops; +static int vfio_ap_mdev_create (struct kobject *kobj,struct mdev_device *mdev) {struct ap_matrix_mdev *matrix_mdev;
@@ -343,6 +345,8 @@ static int vfio_ap_mdev_create(struct kobject *kobj, struct mdev_device *mdev)list_add (&matrix_mdev->node, &matrix_dev->mdev_list);mutex_unlock (&matrix_dev->lock); +mdev_vfio_set_ops (mdev, &vfio_ap_matrix_mdev_ops); +return 0 ; }
@@ -1280,15 +1284,18 @@ static ssize_t vfio_ap_mdev_ioctl(struct mdev_device *mdev,return ret; } +static const struct mdev_vfio_ops vfio_ap_matrix_mdev_ops = { + .open = vfio_ap_mdev_open, + .release = vfio_ap_mdev_release, + .ioctl = vfio_ap_mdev_ioctl, +} +static const struct mdev_parent_ops vfio_ap_matrix_ops = { .owner =THIS_MODULE , .supported_type_groups = vfio_ap_mdev_type_groups, .mdev_attr_groups = vfio_ap_mdev_attr_groups, .create = vfio_ap_mdev_create, .remove = vfio_ap_mdev_remove, - .open = vfio_ap_mdev_open, - .release = vfio_ap_mdev_release, - .ioctl = vfio_ap_mdev_ioctl, };int vfio_ap_mdev_register (void )
drivers/vfio/mdev/mdev_core.c +3 -2
@@ -179,7 +179,7 @@ static struct class_compat *mdev_alloc_class_compat(struct bus_type *bus)* Returns a negative value on error, otherwise 0. */ int mdev_register_device (struct device *dev,const struct mdev_parent_ops *ops, -struct bus_type *bus) +struct bus_type *bus,size_t dev_size) {int ret;struct mdev_parent *parent;
@@ -217,6 +217,7 @@ int mdev_register_device(struct device *dev, const struct mdev_parent_ops *ops, parent->dev = dev; parent->ops = ops; parent->bus = bus; + parent->dev_size = dev_size;mutex_lock (&compat_list_lock); class_compat =mdev_alloc_class_compat (bus);
@@ -339,7 +340,7 @@ int mdev_device_create(struct kobject *kobj, } } - mdev =kzalloc (sizeof (*mdev),GFP_KERNEL ); + mdev =kzalloc (parent->dev_size,GFP_KERNEL );if (!mdev) {mutex_unlock (&mdev_list_lock); ret = -ENOMEM ;
drivers/vfio/mdev/mdev_private.h +5 -0
@@ -20,6 +20,7 @@ struct mdev_parent {struct list_head type_list;/* Synchronize device creation/removal with parent unregistration */ struct rw_semaphore unreg_sem; +size_t dev_size; };struct mdev_device {
@@ -66,4 +67,8 @@ int mdev_device_create(struct kobject *kobj, struct device *dev, const guid_t *uuid);int mdev_device_remove (struct device *dev); +int mdev_register_device (struct device *dev,const struct mdev_parent_ops *ops, +struct bus_type *bus,size_t dev_size); +void mdev_unregister_device (struct device *dev); +#endif /* MDEV_PRIVATE_H */
drivers/vfio/mdev/mdev_vfio.c +29 -1
@@ -4,6 +4,8 @@#include <linux/device.h> #include <linux/mdev_vfio.h> +#include "mdev_private.h" +#define DRIVER_VERSION "0.1"#define DRIVER_AUTHOR "Jason Wang"#define DRIVER_DESC "Mediated VFIO bus"
@@ -15,6 +17,31 @@ struct bus_type mdev_vfio_bus_type = { };EXPORT_SYMBOL (mdev_vfio_bus_type); +#define to_vfio_mdev_device (mdev) container_of(mdev, \ + struct mdev_vfio_device, mdev) + +struct mdev_vfio_device { +struct mdev_device mdev; +const struct mdev_vfio_ops *ops; +}; + +void mdev_vfio_set_ops (struct mdev_device *mdev, +const struct mdev_vfio_ops *ops) +{ +struct mdev_vfio_device *mdev_vfio =to_vfio_mdev_device (mdev); + + mdev_vfio->ops = ops; +} +EXPORT_SYMBOL (mdev_vfio_set_ops); + +const struct mdev_vfio_ops *mdev_vfio_get_ops (struct mdev_device *mdev) +{ +struct mdev_vfio_device *mdev_vfio =to_vfio_mdev_device (mdev); + +return mdev_vfio->ops; +} +EXPORT_SYMBOL (mdev_vfio_get_ops); +static int __init mdev_init (void ) {return mdev_register_bus (&mdev_vfio_bus_type);
@@ -28,7 +55,8 @@ static void __exit mdev_exit(void)int mdev_vfio_register_device (struct device *dev,const struct mdev_parent_ops *ops) { -return mdev_register_device (dev, ops, &mdev_vfio_bus_type); +return mdev_register_device (dev, ops, &mdev_vfio_bus_type, +sizeof (struct mdev_vfio_device )); }EXPORT_SYMBOL (mdev_vfio_register_device);
drivers/vfio/mdev/vfio_mdev.c +19 -19
@@ -24,16 +24,16 @@static int vfio_mdev_open (void *device_data) {struct mdev_device *mdev = device_data; -struct mdev_parent *parent = mdev->parent; +const struct mdev_vfio_ops *ops =mdev_vfio_get_ops (mdev);int ret; -if (unlikely (!parent->ops->open)) +if (unlikely (!ops->open))return -EINVAL ;if (!try_module_get (THIS_MODULE ))return -ENODEV ; - ret = parent->ops->open (mdev); + ret = ops->open (mdev);if (ret)module_put (THIS_MODULE );
@@ -43,10 +43,10 @@ static int vfio_mdev_open(void *device_data)static void vfio_mdev_release (void *device_data) {struct mdev_device *mdev = device_data; -struct mdev_parent *parent = mdev->parent; +const struct mdev_vfio_ops *ops =mdev_vfio_get_ops (mdev); -if (likely (parent->ops->release)) - parent->ops->release (mdev); +if (likely (ops->release)) + ops->release (mdev);module_put (THIS_MODULE ); }
@@ -55,47 +55,47 @@ static long vfio_mdev_unlocked_ioctl(void *device_data, unsigned int cmd, unsigned long arg) {struct mdev_device *mdev = device_data; -struct mdev_parent *parent = mdev->parent; +const struct mdev_vfio_ops *ops =mdev_vfio_get_ops (mdev); -if (unlikely (!parent->ops->ioctl)) +if (unlikely (!ops->ioctl))return -EINVAL ; -return parent->ops->ioctl (mdev, cmd, arg); +return ops->ioctl (mdev, cmd, arg); }static ssize_t vfio_mdev_read (void *device_data,char __user *buf,size_t count,loff_t *ppos) {struct mdev_device *mdev = device_data; -struct mdev_parent *parent = mdev->parent; +const struct mdev_vfio_ops *ops =mdev_vfio_get_ops (mdev); -if (unlikely (!parent->ops->read)) +if (unlikely (!ops->read))return -EINVAL ; -return parent->ops->read (mdev, buf, count, ppos); +return ops->read (mdev, buf, count, ppos); }static ssize_t vfio_mdev_write (void *device_data,const char __user *buf,size_t count,loff_t *ppos) {struct mdev_device *mdev = device_data; -struct mdev_parent *parent = mdev->parent; +const struct mdev_vfio_ops *ops =mdev_vfio_get_ops (mdev); -if (unlikely (!parent->ops->write)) +if (unlikely (!ops->write))return -EINVAL ; -return parent->ops->write (mdev, buf, count, ppos); +return ops->write (mdev, buf, count, ppos); }static int vfio_mdev_mmap (void *device_data,struct vm_area_struct *vma) {struct mdev_device *mdev = device_data; -struct mdev_parent *parent = mdev->parent; +const struct mdev_vfio_ops *ops =mdev_vfio_get_ops (mdev); -if (unlikely (!parent->ops->mmap)) +if (unlikely (!ops->mmap))return -EINVAL ; -return parent->ops->mmap (mdev, vma); +return ops->mmap (mdev, vma); }static const struct vfio_device_ops vfio_mdev_dev_ops = {
@@ -110,7 +110,7 @@ static const struct vfio_device_ops vfio_mdev_dev_ops = {static int vfio_mdev_probe (struct device *dev) { -struct mdev_device *mdev =to_mdev_device (dev); +struct mdev_device *mdev =mdev_from_dev (dev, &mdev_vfio_bus_type);return vfio_add_group_dev (dev, &vfio_mdev_dev_ops, mdev); }
include/linux/mdev.h +0 -37
@@ -48,30 +48,6 @@ struct device *mdev_get_iommu_device(struct device *dev);* @mdev: mdev_device device structure which is being * destroyed * Returns integer: success (0) or error (< 0) -* @open: Open mediated device. -* @mdev: mediated device. -* Returns integer: success (0) or error (< 0) -* @release: release mediated device -* @mdev: mediated device. -* @read: Read emulation callback -* @mdev: mediated device structure -* @buf: read buffer -* @count: number of bytes to read -* @ppos: address. -* Retuns number on bytes read on success or error. -* @write: Write emulation callback -* @mdev: mediated device structure -* @buf: write buffer -* @count: number of bytes to be written -* @ppos: address. -* Retuns number on bytes written on success or error. -* @ioctl: IOCTL callback -* @mdev: mediated device structure -* @cmd: ioctl command -* @arg: arguments to ioctl -* @mmap: mmap callback -* @mdev: mediated device structure -* @vma: vma structure * Parent device that support mediated device should be registered with mdev * module with mdev_parent_ops structure. **/
@@ -83,15 +59,6 @@ struct mdev_parent_ops {int (*create)(struct kobject *kobj,struct mdev_device *mdev);int (*remove)(struct mdev_device *mdev); -int (*open)(struct mdev_device *mdev); -void (*release)(struct mdev_device *mdev); -ssize_t (*read)(struct mdev_device *mdev,char __user *buf, -size_t count,loff_t *ppos); -ssize_t (*write)(struct mdev_device *mdev,const char __user *buf, -size_t count,loff_t *ppos); -long (*ioctl)(struct mdev_device *mdev,unsigned int cmd, -unsigned long arg); -int (*mmap)(struct mdev_device *mdev,struct vm_area_struct *vma); };/* interface for exporting mdev supported type attributes */
@@ -133,10 +100,6 @@ void *mdev_get_drvdata(struct mdev_device *mdev);void mdev_set_drvdata (struct mdev_device *mdev,void *data);const guid_t *mdev_uuid (struct mdev_device *mdev); -int mdev_register_device (struct device *dev,const struct mdev_parent_ops *ops, -struct bus_type *bus); -void mdev_unregister_device (struct device *dev); -int mdev_register_driver (struct mdev_driver *drv,struct module *owner,struct bus_type *bus);void mdev_unregister_driver (struct mdev_driver *drv);
include/linux/mdev_vfio.h +43 -0
53 lines not on this page · raw message
samples/vfio-mdev/mbochs.c +12 -6
@@ -516,6 +516,8 @@ static int mbochs_reset(struct mdev_device *mdev)return 0 ; } +static const struct mdev_vfio_ops mdev_ops; +static int mbochs_create (struct kobject *kobj,struct mdev_device *mdev) {const struct mbochs_type *type =mbochs_find_type (kobj);
@@ -561,6 +563,7 @@ static int mbochs_create(struct kobject *kobj, struct mdev_device *mdev)mbochs_reset (mdev); mbochs_used_mbytes += type->mbytes; +mdev_vfio_set_ops (mdev, &mdev_ops);return 0 ; err_mem:
@@ -1418,12 +1421,7 @@ static struct attribute_group *mdev_type_groups[] = { NULL, }; -static const struct mdev_parent_ops mdev_fops = { - .owner =THIS_MODULE , - .mdev_attr_groups = mdev_dev_groups, - .supported_type_groups = mdev_type_groups, - .create = mbochs_create, - .remove = mbochs_remove, +static const struct mdev_vfio_ops mdev_ops = { .open = mbochs_open, .release = mbochs_close, .read = mbochs_read,
@@ -1432,6 +1430,14 @@ static const struct mdev_parent_ops mdev_fops = { .mmap = mbochs_mmap, }; +static const struct mdev_parent_ops mdev_fops = { + .owner =THIS_MODULE , + .mdev_attr_groups = mdev_dev_groups, + .supported_type_groups = mdev_type_groups, + .create = mbochs_create, + .remove = mbochs_remove, +}; +static const struct file_operations vd_fops = { .owner =THIS_MODULE , };
samples/vfio-mdev/mdpy.c +13 -6
@@ -226,6 +226,8 @@ static int mdpy_reset(struct mdev_device *mdev)return 0 ; } +static const struct mdev_vfio_ops mdev_ops; +static int mdpy_create (struct kobject *kobj,struct mdev_device *mdev) {const struct mdpy_type *type =mdpy_find_type (kobj);
@@ -269,6 +271,8 @@ static int mdpy_create(struct kobject *kobj, struct mdev_device *mdev)mdpy_reset (mdev); mdpy_count++; + +mdev_vfio_set_ops (mdev, &mdev_ops);return 0 ; }
@@ -725,12 +729,7 @@ static struct attribute_group *mdev_type_groups[] = { NULL, }; -static const struct mdev_parent_ops mdev_fops = { - .owner =THIS_MODULE , - .mdev_attr_groups = mdev_dev_groups, - .supported_type_groups = mdev_type_groups, - .create = mdpy_create, - .remove = mdpy_remove, +static const struct mdev_vfio_ops mdev_ops = { .open = mdpy_open, .release = mdpy_close, .read = mdpy_read,
@@ -739,6 +738,14 @@ static const struct mdev_parent_ops mdev_fops = { .mmap = mdpy_mmap, }; +static const struct mdev_parent_ops mdev_fops = { + .owner =THIS_MODULE , + .mdev_attr_groups = mdev_dev_groups, + .supported_type_groups = mdev_type_groups, + .create = mdpy_create, + .remove = mdpy_remove, +}; +static const struct file_operations vd_fops = { .owner =THIS_MODULE , };
samples/vfio-mdev/mtty.c +11 -5
@@ -708,6 +708,8 @@ static ssize_t mdev_access(struct mdev_device *mdev, u8 *buf, size_t count,return ret; } +static const struct mdev_vfio_ops mdev_ops; +static int mtty_create (struct kobject *kobj,struct mdev_device *mdev) {struct mdev_state *mdev_state;
@@ -755,6 +757,7 @@ static int mtty_create(struct kobject *kobj, struct mdev_device *mdev)list_add (&mdev_state->next, &mdev_devices_list);mutex_unlock (&mdev_list_lock); +mdev_vfio_set_ops (mdev, &mdev_ops);return 0 ; }
@@ -1387,6 +1390,14 @@ static struct attribute_group *mdev_type_groups[] = { NULL, }; +static const struct mdev_vfio_ops mdev_ops = { + .open = mtty_open, + .release = mtty_close, + .read = mtty_read, + .write = mtty_write, + .ioctl = mtty_ioctl, +}; +static const struct mdev_parent_ops mdev_fops = { .owner =THIS_MODULE , .dev_attr_groups = mtty_dev_groups,
@@ -1394,11 +1405,6 @@ static const struct mdev_parent_ops mdev_fops = { .supported_type_groups = mdev_type_groups, .create = mtty_create, .remove = mtty_remove, - .open = mtty_open, - .release = mtty_close, - .read = mtty_read, - .write = mtty_write, - .ioctl = mtty_ioctl, };static void mtty_device_release (struct device *dev)
Signature · 1 line
2.19.1
PATCH v12 3/6 mdev: move to drivers/
In reply to Jason WangMdev now is nothing VFIO specific, let's move it to upper directory.
---
MAINTAINERS | 7 +++++--
drivers/Kconfig | 2 ++
drivers/Makefile | 1 +
drivers/mdev/Kconfig | 19 ++++++++++++++++++
drivers/mdev/Makefile | 5 +++++
drivers/{vfio => }/mdev/mdev_core.c | 0
drivers/{vfio => }/mdev/mdev_driver.c | 0
drivers/{vfio => }/mdev/mdev_private.h | 0
drivers/{vfio => }/mdev/mdev_sysfs.c | 0
.../{vfio/mdev/mdev_vfio.c => mdev/vfio.c} | 0
drivers/vfio/mdev/Kconfig | 20 -------------------
drivers/vfio/mdev/Makefile | 4 ----
drivers/vfio/mdev/vfio_mdev.c | 2 --
13 files changed, 32 insertions(+), 28 deletions(-)
create mode 100644 drivers/mdev/Kconfig
create mode 100644 drivers/mdev/Makefile
rename drivers/{vfio => }/mdev/mdev_core.c (100%)
rename drivers/{vfio => }/mdev/mdev_driver.c (100%)
rename drivers/{vfio => }/mdev/mdev_private.h (100%)
rename drivers/{vfio => }/mdev/mdev_sysfs.c (100%)
rename drivers/{vfio/mdev/mdev_vfio.c => mdev/vfio.c} (100%)MAINTAINERS +5 -2
@@ -17111,15 +17111,18 @@ T: git git://github.com/awilliam/linux-vfio.git S: Maintained F: Documentation/driver-api/vfio.rst F: drivers/vfio/ +F: drivers/mdev/vfio.c F: include/linux/vfio.h F: include/uapi/linux/vfio.h -VFIO MEDIATED DEVICE DRIVERS +MEDIATED DEVICE DRIVERS +M: Alex Williamson <[email protected]> M: Kirti Wankhede <[email protected]> +R: Cornelia Huck <[email protected]> L: [email protected] S: Maintained F: Documentation/driver-api/vfio-mediated-device.rst -F: drivers/vfio/mdev/ +F: drivers/mdev F: include/linux/mdev.h F: include/linux/mdev_vfio.h F: samples/vfio-mdev/
drivers/Kconfig +2 -0
@@ -228,4 +228,6 @@ source "drivers/interconnect/Kconfig" source "drivers/counter/Kconfig" +source "drivers/mdev/Kconfig" + endmenu
drivers/Makefile +1 -0
@@ -186,3 +186,4 @@ obj-$(CONFIG_SIOX) += siox/ obj-$(CONFIG_GNSS) += gnss/ obj-$(CONFIG_INTERCONNECT) += interconnect/ obj-$(CONFIG_COUNTER) += counter/ +obj-$(CONFIG_MDEV) += mdev/
drivers/mdev/Kconfig new file +19 -0
@@ -0,0 +1,19 @@ + +config MDEV + tristate "Mediated device driver framework" + default n + help + Provides a framework to virtualize devices. + + If you don't know what do here, say N. + +config VFIO_MDEV + tristate "VFIO Mediated device driver" + depends on VFIO && MDEV + default n + help + Proivdes a mediated BUS for userspace driver through VFIO + framework. See Documentation/vfio-mediated-device.txt for + more details. + + If you don't know what do here, say N.
drivers/mdev/Makefile new file +5 -0
@@ -0,0 +1,5 @@ + +mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o +mdev_vfio-y := vfio.o +obj-$(CONFIG_MDEV) += mdev.o +obj-$(CONFIG_VFIO_MDEV) += mdev_vfio.o
drivers/mdev/mdev_core.c renamed from drivers/vfio/mdev/mdev_core.c +0 -0
drivers/mdev/mdev_driver.c renamed from drivers/vfio/mdev/mdev_driver.c +0 -0
drivers/mdev/mdev_private.h renamed from drivers/vfio/mdev/mdev_private.h +0 -0
drivers/mdev/mdev_sysfs.c renamed from drivers/vfio/mdev/mdev_sysfs.c +0 -0
drivers/mdev/vfio.c renamed from drivers/vfio/mdev/mdev_vfio.c +0 -0
drivers/vfio/mdev/Kconfig +0 -20
@@ -1,24 +1,4 @@ -config MDEV - tristate "Mediated device driver framework" - default n - help - Provides a framework to virtualize devices. - - If you don't know what do here, say N. - -config VFIO_MDEV - tristate "VFIO Mediated device driver" - depends on VFIO && MDEV - default n - help - Proivdes a mediated BUS for userspace driver through VFIO - framework. See Documentation/vfio-mediated-device.txt for - more details. - - If you don't know what do here, say N. - - config VFIO_MDEV_DEVICE tristate "VFIO driver for Mediated devices" depends on VFIO && VFIO_MDEV
drivers/vfio/mdev/Makefile +0 -4
@@ -1,6 +1,2 @@ -mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o - -obj-$(CONFIG_MDEV) += mdev.o -obj-$(CONFIG_VFIO_MDEV) += mdev_vfio.o obj-$(CONFIG_VFIO_MDEV_DEVICE) += vfio_mdev.o
drivers/vfio/mdev/vfio_mdev.c +0 -2
@@ -15,8 +15,6 @@#include <linux/vfio.h> #include <linux/mdev_vfio.h> -#include "mdev_private.h" -#define DRIVER_VERSION "0.1"#define DRIVER_AUTHOR "NVIDIA Corporation"#define DRIVER_DESC "VFIO based driver for Mediated device"
Signature · 1 line
2.19.1
PATCH v12 4/6 mdev: introduce mediated virtio bus
In reply to Jason WangThis patch implements a mediated virtio bus over mdev framework. This will be used by the future virtio-mdev and vhost-mdev on top to allow driver from either userspace or kernel to control the device which is capable of offloading virtio datapath.
--- MAINTAINERS | 2 + drivers/mdev/Kconfig | 10 ++ drivers/mdev/Makefile | 2 + drivers/mdev/virtio.c | 126 +++++++++++++++++++++++ include/linux/mdev_virtio.h | 163 ++++++++++++++++++++++++++++++ include/linux/mod_devicetable.h | 8 ++ scripts/mod/devicetable-offsets.c | 3 + scripts/mod/file2alias.c | 12 +++ 8 files changed, 326 insertions(+) create mode 100644 drivers/mdev/virtio.c create mode 100644 include/linux/mdev_virtio.h
MAINTAINERS +2 -0
@@ -17251,6 +17251,8 @@ F: include/linux/virtio*.h F: include/uapi/linux/virtio_*.h F: drivers/crypto/virtio/ F: mm/balloon_compaction.c +F: include/linux/mdev_virtio.h +F: drivers/mdev/virtio.c VIRTIO BLOCK AND SCSI DRIVERS M: "Michael S. Tsirkin" <[email protected]>
drivers/mdev/Kconfig +10 -0
@@ -17,3 +17,13 @@ config VFIO_MDEV more details. If you don't know what do here, say N. + +config MDEV_VIRTIO + tristate "Mediated VIRTIO bus" + depends on VIRTIO && MDEV + default n + help + Proivdes a mediated BUS for virtio. It could be used by + either kenrel driver or userspace driver. + + If you don't know what do here, say N.
drivers/mdev/Makefile +2 -0
@@ -1,5 +1,7 @@ mdev-y := mdev_core.o mdev_sysfs.o mdev_driver.o mdev_vfio-y := vfio.o +mdev_virtio-y := virtio.o obj-$(CONFIG_MDEV) += mdev.o obj-$(CONFIG_VFIO_MDEV) += mdev_vfio.o +obj-$(CONFIG_MDEV_VIRTIO) += mdev_virtio.o
drivers/mdev/virtio.c new file +126 -0
126 lines not on this page · raw message
include/linux/mdev_virtio.h new file +163 -0
163 lines not on this page · raw message
include/linux/mod_devicetable.h +8 -0
@@ -821,4 +821,12 @@ struct wmi_device_id { const void *context; }; +/** +* struct mdev_class_id - MDEV VIRTIO device class identifier +* @id: Used to identify a specific class of device, e.g vfio-mdev device. +*/ +struct mdev_virtio_class_id { +__u16 id; +}; +#endif /* LINUX_MOD_DEVICETABLE_H */
scripts/mod/devicetable-offsets.c +3 -0
@@ -231,5 +231,8 @@ int main(void)DEVID (wmi_device_id);DEVID_FIELD (wmi_device_id, guid_string); +DEVID (mdev_virtio_class_id); +DEVID_FIELD (mdev_virtio_class_id, id); +return 0 ; }
scripts/mod/file2alias.c +12 -0
@@ -1335,6 +1335,17 @@ static int do_wmi_entry(const char *filename, void *symval, char *alias)return 1 ; } +/* looks like: "mdev_virtio:cN" */ +static int do_mdev_virtio_entry (const char *filename,void *symval,char *alias) +{ +DEF_FIELD (symval, mdev_virtio_class_id, id); + +sprintf (alias,"mdev_virtio:c%02X" , id); +add_wildcard (alias); +return 1 ; +} + +/* Does namelen bytes of name exactly match the symbol? */ static bool sym_is (const char *name,unsigned namelen,const char *symbol) {
@@ -1407,6 +1418,7 @@ static const struct devtable devtable[] = { {"typec", SIZE_typec_device_id, do_typec_entry}, {"tee", SIZE_tee_client_device_id, do_tee_entry}, {"wmi", SIZE_wmi_device_id, do_wmi_entry}, + {"mdev_virtio", SIZE_mdev_virtio_class_id, do_mdev_virtio_entry}, };/* Create MODULE_ALIAS() statements.
Signature · 6 lines
2.19.1 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
PATCH v12 5/6 virtio: introduce a mdev based transport
In reply to Jason WangThis patch introduces a new mdev transport for virtio. This is used to use kernel virtio driver to drive the mediated device that is capable of populating virtqueue directly.
A new virtio-mdev driver will be registered to the mdev bus, when a new virtio-mdev device is probed, it will register the device with mdev based config ops. This means it is a software transport between mdev driver and mdev device. The transport was implemented through bus_ops of mdev parent.
--- drivers/virtio/Kconfig | 13 ++ drivers/virtio/Makefile | 1 + drivers/virtio/virtio_mdev.c | 409 +++++++++++++++++++++++++++++++++++ include/linux/mdev_virtio.h | 5 + 4 files changed, 428 insertions(+) create mode 100644 drivers/virtio/virtio_mdev.c
drivers/virtio/Kconfig +13 -0
@@ -43,6 +43,19 @@ config VIRTIO_PCI_LEGACY If unsure, say Y. +config VIRTIO_MDEV + tristate "MDEV driver for virtio devices" + depends on MDEV_VIRTIO + default n + help + This driver provides support for virtio based paravirtual + device driver over MDEV bus. For this to be useful, you need + an appropriate virtio mdev device implementation that + operates on a physical device to allow the datapath of virtio + to be offloaded to hardware. + + If unsure, say M. + config VIRTIO_PMEM tristate "Support for virtio pmem driver" depends on VIRTIO
drivers/virtio/Makefile +1 -0
@@ -6,3 +6,4 @@ virtio_pci-y := virtio_pci_modern.o virtio_pci_common.o virtio_pci-$(CONFIG_VIRTIO_PCI_LEGACY) += virtio_pci_legacy.o obj-$(CONFIG_VIRTIO_BALLOON) += virtio_balloon.o obj-$(CONFIG_VIRTIO_INPUT) += virtio_input.o +obj-$(CONFIG_VIRTIO_MDEV) += virtio_mdev.o
drivers/virtio/virtio_mdev.c new file +409 -0
409 lines not on this page · raw message
include/linux/mdev_virtio.h +5 -0
@@ -25,6 +25,11 @@ struct virtio_mdev_callback {void *private; }; +enum { +MDEV_VIRTIO_CLASS_ID_VIRTIO =1 , +/* New entries must be added here */ +}; +/** * struct mdev_virtio_device_ops - Structure to be registered for each * mdev device to register the device for virtio/vhost drivers.
Signature · 1 line
2.19.1
Michael S. Tsirkin
maintainerRe: [PATCH V12 5/6] virtio: introduce a mdev based transport
Quoted 428 lines · Jason Wang
This patch introduces a new mdev transport for virtio. This is used to use kernel virtio driver to drive the mediated device that is capable of populating virtqueue directly.
A new virtio-mdev driver will be registered to the mdev bus, when a new virtio-mdev device is probed, it will register the device with mdev based config ops. This means it is a software transport between mdev driver and mdev device. The transport was implemented through
bus_opsof mdev parent.Signed-off-by Jason Wang <[email protected]>--- drivers/virtio/Kconfig | 13 ++ drivers/virtio/Makefile | 1 + drivers/virtio/virtio_mdev.c | 409 +++++++++++++++++++++++++++++++++++ include/linux/mdev_virtio.h | 5 + 4 files changed, 428 insertions(+) create mode 100644 drivers/virtio/virtio_mdev.cdrivers/virtio/Kconfig +13
-0@@ -43,6 +43,19 @@ config VIRTIO_PCI_LEGACY If unsure, say Y. +config VIRTIO_MDEV + tristate "MDEV driver for virtio devices" + depends on MDEV_VIRTIO + default n + help + This driver provides support for virtio based paravirtual + device driver over MDEV bus. For this to be useful, you need + an appropriate virtio mdev device implementation that + operates on a physical device to allow the datapath of virtio + to be offloaded to hardware. + + If unsure, say M. + config VIRTIO_PMEM tristate "Support for virtio pmem driver" depends on VIRTIOdrivers/virtio/Makefile +1
-0@@ -6,3 +6,4 @@ virtio_pci-y := virtio_pci_modern.o virtio_pci_common.o virtio_pci-$(CONFIG_VIRTIO_PCI_LEGACY) += virtio_pci_legacy.o obj-$(CONFIG_VIRTIO_BALLOON) += virtio_balloon.o obj-$(CONFIG_VIRTIO_INPUT) += virtio_input.o +obj-$(CONFIG_VIRTIO_MDEV) += virtio_mdev.odrivers/virtio/virtio_mdev.c new file +370
-0@@ -0,0 +1,409 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* VIRTIO based driver for Mediated device +* +* Copyright (c) 2019, Red Hat. All rights reserved. +* Author: Jason Wang <[email protected]> +* +*/ + +#include <linux/init.h> +#include <linux/module.h> +#include <linux/device.h> +#include <linux/kernel.h> +#include <linux/slab.h> +#include <linux/uuid.h> +#include <linux/virtio.h> +#include <linux/mdev_virtio.h> +#include <linux/virtio_config.h> +#include <linux/virtio_ring.h> + +#define DRIVER_VERSION "0.1" +#define DRIVER_AUTHOR "Red Hat Corporation" +#define DRIVER_DESC "VIRTIO based driver for Mediated device" + +#define to_virtio_mdev_device (dev) \ + container_of(dev, struct virtio_mdev_device, vdev) + +struct virtio_mdev_device { +struct virtio_device vdev; +struct mdev_device *mdev; +u64 features; + +/* The lock to protect virtqueue list */ +spinlock_t lock; +/* List of virtio_mdev_vq_info */ +struct list_head virtqueues; +}; + +struct virtio_mdev_vq_info { +/* the actual virtqueue */ +struct virtqueue *vq; + +/* the list node for the virtqueues list */ +struct list_head node; +}; + +static struct mdev_device *vm_get_mdev (struct virtio_device *vdev) +{ +struct virtio_mdev_device *vm_dev =to_virtio_mdev_device (vdev); +struct mdev_device *mdev = vm_dev->mdev; + +return mdev; +} + +static void virtio_mdev_get (struct virtio_device *vdev,unsigned offset, +void *buf,unsigned len) +{ +struct mdev_device *mdev =vm_get_mdev (vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); + + ops->get_config (mdev, offset, buf, len); +} + +static void virtio_mdev_set (struct virtio_device *vdev,unsigned offset, +const void *buf,unsigned len) +{ +struct mdev_device *mdev =vm_get_mdev (vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); + + ops->set_config (mdev, offset, buf, len); +} + +static u32 virtio_mdev_generation (struct virtio_device *vdev) +{ +struct mdev_device *mdev =vm_get_mdev (vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); + + +if (ops->get_generation) +return ops->get_generation (mdev); + +return 0 ; +} + +static u8 virtio_mdev_get_status (struct virtio_device *vdev) +{ +struct mdev_device *mdev =vm_get_mdev (vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); + +return ops->get_status (mdev); +} + +static void virtio_mdev_set_status (struct virtio_device *vdev,u8 status) +{ +struct mdev_device *mdev =vm_get_mdev (vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); + +return ops->set_status (mdev, status); +} + +static void virtio_mdev_reset (struct virtio_device *vdev) +{ +struct mdev_device *mdev =vm_get_mdev (vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); + +return ops->set_status (mdev,0 ); +} + +static bool virtio_mdev_notify (struct virtqueue *vq) +{ +struct mdev_device *mdev =vm_get_mdev (vq->vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); + + ops->kick_vq (mdev, vq->index); + +return true; +} + +static irqreturn_t virtio_mdev_config_cb (void *private) +{ +struct virtio_mdev_device *vm_dev = private; + +virtio_config_changed (&vm_dev->vdev); + +return IRQ_HANDLED ; +} + +static irqreturn_t virtio_mdev_virtqueue_cb (void *private) +{ +struct virtio_mdev_vq_info *info = private; + +return vring_interrupt (0 , info->vq); +} + +static struct virtqueue * +virtio_mdev_setup_vq (struct virtio_device *vdev,unsigned int index, +void (*callback)(struct virtqueue *vq), +const char *name,bool ctx) +{ +struct virtio_mdev_device *vm_dev =to_virtio_mdev_device (vdev); +struct mdev_device *mdev =vm_get_mdev (vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); +struct virtio_mdev_vq_info *info; +struct virtio_mdev_callback cb; +struct virtqueue *vq; +u64 desc_addr, driver_addr, device_addr; +unsigned long flags; +u32 align, num; +int err; + +if (!name) +return NULL ; + +/* Queue shouldn't already be set up. */ +if (ops->get_vq_ready (mdev, index)) +return ERR_PTR (-ENOENT ); + +/* Allocate and fill out our active queue description */ + info =kmalloc (sizeof (*info),GFP_KERNEL ); +if (!info) +return ERR_PTR (-ENOMEM ); + + num = ops->get_vq_num_max (mdev); +if (num ==0 ) { + err = -ENOENT ; + goto error_new_virtqueue; + } + +/* Create the vring */ + align = ops->get_vq_align (mdev); + vq =vring_create_virtqueue (index, num, align, vdev, + true, true, ctx, + virtio_mdev_notify, callback, name); +if (!vq) { + err = -ENOMEM ; + goto error_new_virtqueue; + } + +/* Setup virtqueue callback */ + cb.callback = virtio_mdev_virtqueue_cb; + cb.private = info; + ops->set_vq_cb (mdev, index, &cb); + ops->set_vq_num (mdev, index,virtqueue_get_vring_size (vq)); + + desc_addr =virtqueue_get_desc_addr (vq); + driver_addr =virtqueue_get_avail_addr (vq); + device_addr =virtqueue_get_used_addr (vq); + +if (ops->set_vq_address (mdev, index, + desc_addr, driver_addr, + device_addr)) { + err = -EINVAL ; + goto err_vq; + } + + ops->set_vq_ready (mdev, index,1 ); + + vq->priv = info; + info->vq = vq; + +spin_lock_irqsave (&vm_dev->lock, flags); +list_add (&info->node, &vm_dev->virtqueues); +spin_unlock_irqrestore (&vm_dev->lock, flags); + +return vq; + +err_vq: +vring_del_virtqueue (vq); +error_new_virtqueue: + ops->set_vq_ready (mdev, index,0 ); +WARN_ON (ops->get_vq_ready (mdev, index)); +kfree (info); +return ERR_PTR (err); +} + +static void virtio_mdev_del_vq (struct virtqueue *vq) +{ +struct virtio_mdev_device *vm_dev =to_virtio_mdev_device (vq->vdev); +struct mdev_device *mdev = vm_dev->mdev; +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); +struct virtio_mdev_vq_info *info = vq->priv; +unsigned int index = vq->index; +unsigned long flags; + +spin_lock_irqsave (&vm_dev->lock, flags); +list_del (&info->node); +spin_unlock_irqrestore (&vm_dev->lock, flags); + +/* Select and deactivate the queue */ + ops->set_vq_ready (mdev, index,0 ); +WARN_ON (ops->get_vq_ready (mdev, index)); + +vring_del_virtqueue (vq); + +kfree (info); +} + +static void virtio_mdev_del_vqs (struct virtio_device *vdev) +{ +struct virtqueue *vq, *n; + +list_for_each_entry_safe (vq, n, &vdev->vqs, list) +virtio_mdev_del_vq (vq); +} + +static int virtio_mdev_find_vqs (struct virtio_device *vdev,unsigned nvqs, +struct virtqueue *vqs[], +vq_callback_t *callbacks[], +const char *const names[], +const bool *ctx, +struct irq_affinity *desc) +{ +struct virtio_mdev_device *vm_dev =to_virtio_mdev_device (vdev); +struct mdev_device *mdev =vm_get_mdev (vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); +struct virtio_mdev_callback cb; +int i, err, queue_idx =0 ; + +for (i =0 ; i < nvqs; ++i) { +if (!names[i]) { + vqs[i] =NULL ; +continue ; + } + + vqs[i] =virtio_mdev_setup_vq (vdev, queue_idx++, + callbacks[i], names[i], ctx ? + ctx[i] : false); +if (IS_ERR (vqs[i])) { + err =PTR_ERR (vqs[i]); + goto err_setup_vq; + } + } + + cb.callback = virtio_mdev_config_cb; + cb.private = vm_dev; + ops->set_config_cb (mdev, &cb); + +return 0 ; + +err_setup_vq: +virtio_mdev_del_vqs (vdev); +return err; +} + +static u64 virtio_mdev_get_features (struct virtio_device *vdev) +{ +struct mdev_device *mdev =vm_get_mdev (vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); + +return ops->get_features (mdev); +} + +static int virtio_mdev_finalize_features (struct virtio_device *vdev) +{ +struct mdev_device *mdev =vm_get_mdev (vdev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); + +/* Give virtio_ring a chance to accept features. */ +vring_transport_features (vdev); + +return ops->set_features (mdev, vdev->features); +} + +static const char *virtio_mdev_bus_name (struct virtio_device *vdev) +{ +struct virtio_mdev_device *vm_dev =to_virtio_mdev_device (vdev); +struct mdev_device *mdev = vm_dev->mdev; + +return dev_name (mdev_dev (mdev)); +} + +static const struct virtio_config_ops virtio_mdev_config_ops = { + .get = virtio_mdev_get, + .set = virtio_mdev_set, + .generation = virtio_mdev_generation, + .get_status = virtio_mdev_get_status, + .set_status = virtio_mdev_set_status, + .reset = virtio_mdev_reset, + .find_vqs = virtio_mdev_find_vqs, + .del_vqs = virtio_mdev_del_vqs, + .get_features = virtio_mdev_get_features, + .finalize_features = virtio_mdev_finalize_features, + .bus_name = virtio_mdev_bus_name, +}; + +static void virtio_mdev_release_dev (struct device *_d) +{ +struct virtio_device *vdev = + container_of(_d, struct virtio_device, dev); +struct virtio_mdev_device *vm_dev = + container_of(vdev, struct virtio_mdev_device, vdev); +struct mdev_device *mdev = vm_dev->mdev; + +devm_kfree (mdev_dev (mdev), vm_dev); +} + +static int virtio_mdev_probe (struct device *dev) +{ +struct mdev_device *mdev =mdev_virtio_from_dev (dev); +const struct mdev_virtio_ops *ops =mdev_virtio_get_ops (mdev); +struct virtio_mdev_device *vm_dev; +int rc; + + vm_dev =devm_kzalloc (dev,sizeof (*vm_dev),GFP_KERNEL ); +if (!vm_dev) +return -ENOMEM ; + + vm_dev->vdev.dev.parent = dev; + vm_dev->vdev.dev.release = virtio_mdev_release_dev; + vm_dev->vdev.config = &virtio_mdev_config_ops; + vm_dev->mdev = mdev; +INIT_LIST_HEAD (&vm_dev->virtqueues); +spin_lock_init (&vm_dev->lock); + + vm_dev->vdev.id.device = ops->get_device_id (mdev); +if (vm_dev->vdev.id.device ==0 ) +return -ENODEV ; + + vm_dev->vdev.id.vendor = ops->get_vendor_id (mdev); + rc =register_virtio_device (&vm_dev->vdev); +if (rc) +put_device (dev); +else +dev_set_drvdata (dev, vm_dev); + +return rc; +} + +static void virtio_mdev_remove (struct device *dev) +{
drivers/virtio/virtio_mdev.c line 380
Quoted 10 lines
drivers/virtio/virtio_mdev.c new file +10
-0+struct virtio_mdev_device *vm_dev =dev_get_drvdata (dev); + +unregister_virtio_device (&vm_dev->vdev); +} + +static const struct mdev_virtio_class_id virtio_id_table[] = { + {MDEV_VIRTIO_CLASS_ID_VIRTIO }, + {0 }, +}; +
Do we still need the class ID? It's a virtio mdev bus, do we need a virtio class as well?
Quoted 47 lines
drivers/virtio/virtio_mdev.c new file +29
-0+MODULE_DEVICE_TABLE (mdev_virtio, virtio_id_table); + +static struct mdev_virtio_driver virtio_mdev_driver = { + .drv = { + .name ="virtio_mdev" , + .probe = virtio_mdev_probe, + .remove = virtio_mdev_remove, + }, + .id_table = virtio_id_table, +}; + +static int __init virtio_mdev_init (void ) +{ +return mdev_register_driver (&virtio_mdev_driver.drv,THIS_MODULE , + &mdev_virtio_bus_type); +} + +static void __exit virtio_mdev_exit(void) +{ +mdev_unregister_driver (&virtio_mdev_driver.drv); +} + +module_init(virtio_mdev_init) +module_exit(virtio_mdev_exit) + +MODULE_VERSION(DRIVER_VERSION); +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR(DRIVER_AUTHOR); +MODULE_DESCRIPTION(DRIVER_DESC);include/linux/mdev_virtio.h +5
-0@@ -25,6 +25,11 @@ struct virtio_mdev_callback {void *private; }; +enum { +MDEV_VIRTIO_CLASS_ID_VIRTIO =1 , +/* New entries must be added here */ +}; +/** * struct mdev_virtio_device_ops - Structure to be registered for each * mdev device to register the device for virtio/vhost drivers. Signature · 1 line
2.19.1
Re: [PATCH V12 5/6] virtio: introduce a mdev based transport
Quoted 8 lines · Michael S. Tsirkin
drivers/virtio/virtio_mdev.c line 380
Quoted 5 lines
drivers/virtio/virtio_mdev.c new file +5
-0+static const struct mdev_virtio_class_id virtio_id_table[] = { + {MDEV_VIRTIO_CLASS_ID_VIRTIO }, + {0 }, +}; +Do we still need the class ID? It's a virtio mdev bus, do we need a virtio class as well?
If we want to have auto match between vhost-mdev driver and vhost-mdev device, we need this.
Otherwise, user need to manually probe or bind driver to the device.
Thanks
PATCH v12 6/6 docs: sample driver to demonstrate how to implement virtio-mdev framework
In reply to Jason WangThis sample driver creates mdev device that simulate virtio net device over virtio mdev transport. The device is implemented through vringh and workqueue. A device specific dma ops is to make sure HVA is used directly as the IOVA. This should be sufficient for kernel virtio driver to work.
Only 'virtio' type is supported right now. I plan to add 'vhost' type on top which requires some virtual IOMMU implemented in this sample driver.
--- MAINTAINERS | 1 + samples/Kconfig | 10 + samples/vfio-mdev/Makefile | 1 + samples/vfio-mdev/mvnet_loopback.c | 690 +++++++++++++++++++++++++++++ 4 files changed, 702 insertions(+) create mode 100644 samples/vfio-mdev/mvnet_loopback.c
MAINTAINERS +1 -0
@@ -17228,6 +17228,7 @@ F: net/vmw_vsock/virtio_transport.c F: drivers/net/vsockmon.c F: drivers/vhost/vsock.c F: tools/testing/vsock/ +F: samples/vfio-mdev/mvnet_loopback.c VIRTIO CONSOLE DRIVER M: Amit Shah <[email protected]>
samples/Kconfig +10 -0
@@ -131,6 +131,16 @@ config SAMPLE_VFIO_MDEV_MDPY mediated device. It is a simple framebuffer and supports the region display interface (VFIO_GFX_PLANE_TYPE_REGION). +config SAMPLE_VIRTIO_MDEV_NET_LOOPBACK + tristate "Build loopback VIRTIO net example mediated device sample code -- loadable modules only" + depends on MDEV_VIRTIO && VHOST_RING && m + help + Build a networking sample device for use as a virtio + mediated device. The device cooperates with virtio-mdev bus + driver to present an virtio ethernet driver for + kernel. It simply loopbacks all packets from its TX + virtqueue to its RX virtqueue. + config SAMPLE_VFIO_MDEV_MDPY_FB tristate "Build VFIO mdpy example guest fbdev driver -- loadable module only" depends on FB && m
samples/vfio-mdev/Makefile +1 -0
@@ -3,3 +3,4 @@ obj-$(CONFIG_SAMPLE_VFIO_MDEV_MTTY) += mtty.o obj-$(CONFIG_SAMPLE_VFIO_MDEV_MDPY) += mdpy.o obj-$(CONFIG_SAMPLE_VFIO_MDEV_MDPY_FB) += mdpy-fb.o obj-$(CONFIG_SAMPLE_VFIO_MDEV_MBOCHS) += mbochs.o +obj-$(CONFIG_SAMPLE_VIRTIO_MDEV_NET_LOOPBACK) += mvnet_loopback.o
samples/vfio-mdev/mvnet_loopback.c new file +690 -0
690 lines not on this page · raw message
Signature · 1 line
2.19.1
Patchwork
CI✗ Fi.CI.BUILD: failure for mdev based hardware virtio offloading support
In reply to Jason Wang== Series Details ==
== Series Details ==
Series: mdev based hardware virtio offloading support
URL : https://patchwork.freedesktop.org/series/69596/
State : failure
== Summary ==
Applying: mdev: make mdev bus agnostic
Using index info to reconstruct a base tree...
M MAINTAINERS
M drivers/gpu/drm/i915/gvt/kvmgt.c
Falling back to patching base and 3-way merge... Auto-merging drivers/gpu/drm/i915/gvt/kvmgt.c
CONFLICT (content): Merge conflict in drivers/gpu/drm/i915/gvt/kvmgt.c Auto-merging MAINTAINERS
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch' to see the failed patch Patch failed at 0001 mdev: make mdev bus agnostic
When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 36 lines · Jason Wang
Hi all:
There are hardwares that can do virtio datapath offloading while having its own control path. This path tries to implement a mdev based unified API to support using kernel virtio driver to drive those devices. This is done by introducing a new mdev transport for virtio (
virtio_mdev) and register itself as a new kind of mdev driver. Then it provides a unified way for kernel virtio driver to talk with mdev device implementation.Though the series only contains kernel driver support, the goal is to make the transport generic enough to support userspace drivers. This means vhost-mdev[1] could be built on top as well by resuing the transport.
A sample driver is also implemented which simulate a virito-net loopback ethernet device on top of vringh + workqueue. This could be used as a reference implementation for real hardware driver.
Also a real IFC VF driver was also posted here[2] which is a good reference for vendors who is interested in their own virtio datapath offloading product.
Consider mdev framework only support VFIO device and driver right now, this series also extend it to support other types. This is done through decoupling VFIO specific bits out of mdev core and make mdev an independent module that allows to be used by multiple types of buses.
Pktgen test was done with virito-net + mvnet loop back device.
Please review.
[1] https://lkml.org/lkml/2019/11/5/424
[2] https://lkml.org/lkml/2019/11/5/227
Quoted 10 lines
Changes from V11:
- decouple VFIO specific bits out of mdev core
- make mdev an indepdent module to allow buses other than VFIO mdev
- allow structure composition of mdev through specifiy the size of mdev structure
- introducemdev_vfiostructure and store the VFIO specific callbacks there
- don't use "mdev" bus for virtio, use a new "mdev_virtio" bus, and store the virtio specific callbacks inmdev_virtiostructure.
- do theclass_id, matching on top of "mdev_virtio" bus
It looks to me this series get some conflicts with gvt-linux.git. Will fix them and post v13.
Thanks
Quoted 150 lines
Changes from V10:
- rename mvnet tomvnet_loopback
- fix typo in the help text for sampleKconfigChanges from V9:
- Tweak the help text for virito-mdev kconfigChanges from V8:
- try silent checkpatch, some are still there becuase they were inherited fromvirtio_config_opswhich needs to be resolved in an independent series
- tweak on the comment and doc
- removeVIRTIO_MDEV_F_VERSION_1completely
- renameCONFIG_VIRTIO_MDEV_DEVICEtoCONFIG_VIRTIO_MDEVChanges from V7:
- drop {set|get}_mdev_features for virtio
- typo and comment style fixesChanges from V6:
- rename ops files and compile guardChanges from V5:
- usedev_warn()instead of WARN(1) when class id is not set
- validateid_tablebefore trying to do matching between device and driver
- add wildcard for modpost script
- use unique name forid_table
- moveget_mdev_features()to be the first member ofvirtio_device_opsand more comments for it
- typo fixes for the comments abovevirtio_mdev_opsChanges from V4:
- keepmdev_set_class()for the device that doesn't use device ops
- use union for device ops pointer inmdev_device
- introduce class specific helper for getting is device ops
- useWARN_ONinstead ofBUG_ONinmdev_set_virtio_ops
- explain details ofget_mdev_features()andget_vendor_id()
- distinguish the optional virito device ops from mandatory ones and makeget_generation()optional
- renamevfio_mdev.htovfio_mdev_ops.h, renamevirito_mdev.htovirtio_mdev_ops.h
- don't abuse version fileds invirtio_mdevstructure, use features instead
- fix warning during device remove
- style & docs tweaks and typo fixesChanges from V3:
- document that class id (device ops) must be specified increate()
- addWARN()when trying to setclass_idwhen it has already set
- addWARN()whenclass_idis not specified increate()and correctly return an error in this case
- correct the prototype ofmdev_set_class()in the doc
- add documention ofmdev_set_class()
- remove the unnecessary "class_id_fail" label when class id is not specified increate()
- convertid_tableinvfio_mdevto const
- movemdev_set_classand its friends aftermdev_uuid()
- suqash the patch of bus uevent into patch of introducing class id
- tweak the words in the docs per Cornelia suggestion
- tieclass_idand device ops through class specific initialization routine likemdev_set_vfio_ops()
- typos fixes in the docs of virtio-mdev callbacks
- document the usage of virtqueues in structvirtio_mdev_device
- remove the useless vqs array in structvirtio_mdev_device
- renameMDEV_ID_XXXtoMDEV_CLASS_ID_XXXChanges from V2:
- fail whenclass_idis not specified
- drop the vringh patch
- match the doc to the code
- tweak the commit log
- movedevice_opsfrom parent to mdev device
- remove the unusedMDEV_ID_VHOSTChanges from V1:
- movevirtio_mdev.ctodrivers/virtio
- storeclass_idinmdev_deviceinstead ofmdev_parent
- storedevice_opsinmdev_deviceinstead ofmdev_parent
- reorder the patch, vringh fix comes first
- really silent compiling warnings
- really switch to use u16 forclass_id
- uevent and modpost support for mdevclass_id
- vraious tweaks per comments from ParavChanges from RFC-V2:
- silent compile warnings on some specific configuration
- use u16 instead u8 for class id
- reseveMDEV_ID_VHOSTfor future vhost-mdev work
- introduce "virtio" type for mvnet and make "vhost" type for future work
- add entries in MAINTAINER
- tweak and typos fixes in commit logChanges from RFC-V1:
- rename device id to class id
- add docs for class id and device specific ops (device_ops)
- splitdevice_opsinto seperate headers
- drop themdev_set_dma_ops()
- usedevice_opsto implement the transport API, then it's not a part of UAPI any more
- useGFP_ATOMICin mvnet sample device and other tweaks
- set_vring_base/get_vring_base support for mvnet deviceJason Wang (6):
mdev: make mdev bus agnostic
mdev: split out VFIO bus specific parent ops
mdev: move todrivers/
mdev: introduce mediated virtio bus
virtio: introduce a mdev based transport
docs: sample driver to demonstrate how to implement virtio-mdev
framework.../driver-api/vfio-mediated-device.rst | 94 ++- MAINTAINERS | 11 +- drivers/Kconfig | 2 + drivers/Makefile | 1 + drivers/gpu/drm/i915/gvt/kvmgt.c | 26 +- drivers/mdev/Kconfig | 29 + drivers/mdev/Makefile | 7 + drivers/{vfio => }/mdev/mdev_core.c | 107 ++- drivers/{vfio => }/mdev/mdev_driver.c | 29 +- drivers/{vfio => }/mdev/mdev_private.h | 18 +- drivers/{vfio => }/mdev/mdev_sysfs.c | 0 drivers/mdev/vfio.c | 76 ++ drivers/mdev/virtio.c | 126 ++++ drivers/s390/cio/vfio_ccw_ops.c | 23 +- drivers/s390/crypto/vfio_ap_ops.c | 34 +- drivers/s390/crypto/vfio_ap_private.h | 2 +- drivers/vfio/mdev/Kconfig | 11 - drivers/vfio/mdev/Makefile | 4 - drivers/vfio/mdev/vfio_mdev.c | 45 +- drivers/vfio/vfio_iommu_type1.c | 6 +- drivers/virtio/Kconfig | 13 + drivers/virtio/Makefile | 1 + drivers/virtio/virtio_mdev.c | 409 +++++++++++ include/linux/mdev.h | 49 +- include/linux/mdev_vfio.h | 68 ++ include/linux/mdev_virtio.h | 168 +++++ include/linux/mod_devicetable.h | 8 + samples/Kconfig | 10 + samples/vfio-mdev/Makefile | 1 + samples/vfio-mdev/mbochs.c | 26 +- samples/vfio-mdev/mdpy.c | 27 +- samples/vfio-mdev/mtty.c | 24 +- samples/vfio-mdev/mvnet_loopback.c | 690 ++++++++++++++++++ scripts/mod/devicetable-offsets.c | 3 + scripts/mod/file2alias.c | 12 + 35 files changed, 1924 insertions(+), 236 deletions(-)
Quoted 13 lines
create mode 100644
drivers/mdev/Kconfigcreate mode 100644drivers/mdev/Makefilerenamedrivers/{vfio=>}/mdev/mdev_core.c(76%) renamedrivers/{vfio=>}/mdev/mdev_driver.c(83%) renamedrivers/{vfio=>}/mdev/mdev_private.h(81%) renamedrivers/{vfio=>}/mdev/mdev_sysfs.c(100%) create mode 100644drivers/mdev/vfio.ccreate mode 100644drivers/mdev/virtio.ccreate mode 100644drivers/virtio/virtio_mdev.ccreate mode 100644include/linux/mdev_vfio.hcreate mode 100644include/linux/mdev_virtio.hcreate mode 100644samples/vfio-mdev/mvnet_loopback.c
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel