vdpa: generalize vdpa simulator and add block device
Subsystems (2)
cover vdpa: generalize vdpa simulator and add block device
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.
This series moves the network device simulator in a new module (vdpa_sim_net) and leaves the generic functions in the vdpa_sim core module, allowing the possibility to add new vDPA device simulators. Then we added a new vdpa_sim_blk module to simulate a block device.
I'm not sure about patch 11 ("vringh: allow vringh_iov_xfer() to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modify vringh_iov_xfer().
As Max reported, I'm also seeing errors with vdpa_sim_blk related to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:
vringh: Failed to access avail idx at 00000000e8deb2cc
vringh: Failed to read head: idx 6289 address 00000000e1ad1d50
vringh: Failed to get flags at 000000006635d7a3
virtio_vdpa vdpa0: vringh_iov_push_iotlb() error: -14 offset: 0x2840000 len: 0x20000 virtio_vdpa vdpa0: vringh_iov_pull_iotlb() error: -14 offset: 0x58ee000 len: 0x3000
These errors should all be related to the fact that iotlb_translate() fails with -EINVAL, so it seems that we miss some mapping.
I'll debug more carefully, in the meantime can you give a first review?
Thanks,
Stefano
Max Gurtovoy (4):
vhost-vdpa: add support for vDPA blk devices
vdpa: split vdpasim to core and net modules
vdpa_sim: remove hard-coded virtq count
vdpa: add vdpa simulator for block device
Stefano Garzarella (8):
vdpa_sim: remove the limit of IOTLB entries
vdpa_sim: add struct vdpasim_device to store device properties
vdpa_sim: move config management outside of the core
vdpa_sim: use kvmalloc to allocate vdpasim->buffer
vdpa_sim: make vdpasim->buffer size configurable
vdpa_sim: split vdpasim_virtqueue's iov field in riov and wiov
vringh: allow vringh_iov_xfer() to skip bytes when ptr is NULL
vdpa_sim_blk: implement ramdisk behaviour
drivers/vdpa/vdpa_sim/vdpa_sim.h | 117 +++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.c | 283 +++++---------------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 251 ++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 172 ++++++++++++++++ drivers/vhost/vdpa.c | 11 +- drivers/vhost/vringh.c | 16 +- drivers/vdpa/Kconfig | 16 +- drivers/vdpa/vdpa_sim/Makefile | 2 + 8 files changed, 628 insertions(+), 240 deletions(-) create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim.h create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_blk.c create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_net.c
Signature · 1 line
2.26.2
RFC PATCH 1/12 vhost-vdpa: add support for vDPA blk devices
From: Max Gurtovoy <[email protected]>
Currently only net devices can act as vDPA backends. Add an infrastructure for block devices will basic feature list that will be increased in the future.
--- drivers/vhost/vdpa.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
drivers/vhost/vdpa.c +8 -3
@@ -22,6 +22,7 @@#include <linux/nospec.h> #include <linux/vhost.h> #include <linux/virtio_net.h> +#include <linux/virtio_blk.h> #include "vhost.h"
@@ -194,6 +195,9 @@ static int vhost_vdpa_config_validate(struct vhost_vdpa *v,case VIRTIO_ID_NET : size =sizeof (struct virtio_net_config );break ; +case VIRTIO_ID_BLOCK : + size =sizeof (struct virtio_blk_config ); +break ; }if (c->len ==0 )
@@ -975,12 +979,13 @@ static void vhost_vdpa_release_dev(struct device *device)static int vhost_vdpa_probe (struct vdpa_device *vdpa) {const struct vdpa_config_ops *ops = vdpa->config; +u32 device_id = ops->get_device_id (vdpa);struct vhost_vdpa *v;int minor;int r; -/* Currently, we only accept the network devices. */ -if (ops->get_device_id (vdpa) !=VIRTIO_ID_NET ) +/* Currently, we only accept the network and block devices. */ +if (device_id !=VIRTIO_ID_NET && device_id !=VIRTIO_ID_BLOCK )return -ENOTSUPP ; v =kzalloc (sizeof (*v),GFP_KERNEL | __GFP_RETRY_MAYFAIL);
@@ -998,7 +1003,7 @@ static int vhost_vdpa_probe(struct vdpa_device *vdpa) v->minor = minor; v->vdpa = vdpa; v->nvqs = vdpa->nvqs; - v->virtio_id = ops->get_device_id (vdpa); + v->virtio_id = device_id;device_initialize (&v->dev); v->dev.release = vhost_vdpa_release_dev;
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 01/12] vhost-vdpa: add support for vDPA blk devices
Quoted 13 lines · Stefano Garzarella
drivers/vhost/vdpa.c +1
-0@@ -22,6 +22,7 @@#include <linux/nospec.h> #include <linux/vhost.h> #include <linux/virtio_net.h> +#include <linux/virtio_blk.h> #include "vhost.h"
drivers/vhost/vdpa.c static int vhost_vdpa_config_validate(struct vhost_vdpa *v, · line 203
Quoted 9 lines
drivers/vhost/vdpa.c +3
-0@@ -194,6 +195,9 @@ static int vhost_vdpa_config_validate(struct vhost_vdpa *v,case VIRTIO_ID_NET : size =sizeof (struct virtio_net_config );break ; +case VIRTIO_ID_BLOCK : + size =sizeof (struct virtio_blk_config ); +break ; }if (c->len ==0 )
Can vdpa_config_ops->get/set_config() handle the size check instead of hardcoding device-specific knowledge into drivers/vhost/vdpa.c?
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 01/12] vhost-vdpa: add support for vDPA blk devices
Quoted 22 lines · Stefan Hajnoczi
Quoted 21 lines · Stefano Garzarella
drivers/vhost/vdpa.c +4
-0@@ -22,6 +22,7 @@#include <linux/nospec.h> #include <linux/vhost.h> #include <linux/virtio_net.h> +#include <linux/virtio_blk.h> #include "vhost.h" @@ -194,6 +195,9 @@ static int vhost_vdpa_config_validate(struct vhost_vdpa *v,case VIRTIO_ID_NET : size =sizeof (struct virtio_net_config );break ; +case VIRTIO_ID_BLOCK : + size =sizeof (struct virtio_blk_config ); +break ; }
Quoted 4 lines
drivers/vhost/vdpa.c static int vhost_vdpa_config_validate(struct vhost_vdpa *v, · line 203
Quoted 1 line
drivers/vhost/vdpa.c +0
-0if (c->len ==0 )Can vdpa_config_ops->get/set_config() handle the size check instead of hardcoding device-specific knowledge into
drivers/vhost/vdpa.c?
I agree that this should be better. For example we already check if the buffer is large enough in the simulator callbacks, we only need to return an error in case it is not true.
@Jason, do you think it's okay to add a return value to vdpa_config_ops->get/set_config() to handle the size check?
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RFC PATCH 2/12 vdpa: split vdpasim to core and net modules
In reply to Stefano GarzarellaFrom: Max Gurtovoy <[email protected]>
Introduce new vdpa_sim_net and vdpa_sim (core) drivers. This is a preparation for adding a vdpa simulator module for block devices.
---
v1:
- Removed unused headers
- Removed empty module_init() module_exit()
- Moved vdpasim_is_little_endian() in vdpa_sim.h
- Moved vdpasim16_to_cpu/cpu_to_vdpasim16() in vdpa_sim.h
- Added vdpasim*_to_cpu/cpu_to_vdpasim*() also for 32 and 64
- Replaced 'select VDPA_SIM' with 'depends on VDPA_SIM' since selected option can not depend on other [Jason]
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 110 +++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.c | 285 ++++++--------------------- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 153 ++++++++++++++ drivers/vdpa/Kconfig | 7 +- drivers/vdpa/vdpa_sim/Makefile | 1 + 5 files changed, 329 insertions(+), 227 deletions(-) create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim.h create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_net.c
drivers/vdpa/vdpa_sim/vdpa_sim.h new file +110 -0
@@ -0,0 +1,110 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (c) 2020, Red Hat Inc. All rights reserved. +*/ + +#ifndef _VDPA_SIM_H +#define _VDPA_SIM_H + +#include <linux/vringh.h> +#include <linux/vdpa.h> +#include <linux/vhost_iotlb.h> +#include <uapi/linux/virtio_config.h> +#include <uapi/linux/virtio_net.h> +#include <uapi/linux/virtio_blk.h> + +#define DRV_VERSION "0.1" +#define DRV_AUTHOR "Jason Wang <[email protected]>" +#define DRV_LICENSE "GPL v2" + +#define VDPASIM_QUEUE_ALIGN PAGE_SIZE +#define VDPASIM_QUEUE_MAX 256 +#define VDPASIM_VENDOR_ID 0 +#define VDPASIM_VQ_NUM 0x2 + +#define VDPASIM_FEATURES ((1ULL << VIRTIO_F_ANY_LAYOUT) | \ + (1ULL << VIRTIO_F_VERSION_1) | \ + (1ULL << VIRTIO_F_ACCESS_PLATFORM)) + +struct vdpasim ; + +struct vdpasim_virtqueue { +struct vringh vring; +struct vringh_kiov iov; +unsigned short head; +bool ready; +u64 desc_addr; +u64 device_addr; +u64 driver_addr; +u32 num; +void *private; +irqreturn_t (*cb)(void *data); +}; + +struct vdpasim_init_attr { +u32 device_id; +u64 features; +work_func_t work_fn; +int batch_mapping; +}; + +/* State of each vdpasim device */ +struct vdpasim { +struct vdpa_device vdpa; +struct vdpasim_virtqueue vqs[VDPASIM_VQ_NUM ]; +struct work_struct work; +/* spinlock to synchronize virtqueue state */ +spinlock_t lock; +/* virtio config according to device type */ +void *config; +struct vhost_iotlb *iommu; +void *buffer; +u32 device_id; +u32 status; +u32 generation; +u64 features; +u64 supported_features; +/* spinlock to synchronize iommu table */ +spinlock_t iommu_lock; +}; + +struct vdpasim *vdpasim_create (struct vdpasim_init_attr *attr); + +/* TODO: cross-endian support */ +static inline bool vdpasim_is_little_endian (struct vdpasim *vdpasim) +{ +return virtio_legacy_is_little_endian () || + (vdpasim->features & (1ULL <<VIRTIO_F_VERSION_1 )); +} + +static inline u16 vdpasim16_to_cpu (struct vdpasim *vdpasim,__virtio16 val) +{ +return __virtio16_to_cpu (vdpasim_is_little_endian (vdpasim), val); +} + +static inline __virtio16 cpu_to_vdpasim16 (struct vdpasim *vdpasim,u16 val) +{ +return __cpu_to_virtio16 (vdpasim_is_little_endian (vdpasim), val); +} + +static inline u32 vdpasim32_to_cpu (struct vdpasim *vdpasim,__virtio32 val) +{ +return __virtio32_to_cpu (vdpasim_is_little_endian (vdpasim), val); +} + +static inline __virtio32 cpu_to_vdpasim32 (struct vdpasim *vdpasim,u32 val) +{ +return __cpu_to_virtio32 (vdpasim_is_little_endian (vdpasim), val); +} + +static inline u64 vdpasim64_to_cpu (struct vdpasim *vdpasim,__virtio64 val) +{ +return __virtio64_to_cpu (vdpasim_is_little_endian (vdpasim), val); +} + +static inline __virtio64 cpu_to_vdpasim64 (struct vdpasim *vdpasim,u64 val) +{ +return __cpu_to_virtio64 (vdpasim_is_little_endian (vdpasim), val); +} + +#endif
drivers/vdpa/vdpa_sim/vdpa_sim.c +59 -226
416 lines not on this page · raw message
drivers/vdpa/vdpa_sim/vdpa_sim_net.c new file +153 -0
153 lines not on this page · raw message
drivers/vdpa/Kconfig +6 -1
@@ -9,11 +9,16 @@ menuconfig VDPA if VDPA config VDPA_SIM - tristate "vDPA device simulator" + tristate "vDPA simulator core" depends on RUNTIME_TESTING_MENU && HAS_DMA select DMA_OPS select VHOST_RING default n + +config VDPA_SIM_NET + tristate "vDPA simulator for networking device" + depends on VDPA_SIM + default n help vDPA networking device simulator which loop TX traffic back to RX. This device is used for testing, prototyping and
drivers/vdpa/vdpa_sim/Makefile +1 -0
@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_VDPA_SIM) += vdpa_sim.o +obj-$(CONFIG_VDPA_SIM_NET) += vdpa_sim_net.o
Signature · 1 line
2.26.2
kernel test robot
build botRe: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
Hi Stefano,
[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on linus/master]
[also build test WARNING on v5.10-rc3 next-20201113]
[cannot apply to vhost/linux-next linux/master]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Stefano-Garzarella/vdpa-generalize-vdpa-simulator-and-add-block-device/20201113-215247 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 585e5b17b92dead8a3aca4e3c9876fbca5f7e0ba config: x86_64-randconfig-a005-20201113 (attached as .config) compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project 9e0c35655b6e8186baef8840b26ba4090503b554)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/12adecc2babdd7bf8e2bd80ab956da09ddd29355
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Stefano-Garzarella/vdpa-generalize-vdpa-simulator-and-add-block-device/20201113-215247 git checkout 12adecc2babdd7bf8e2bd80ab956da09ddd29355
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
>> drivers/vdpa/vdpa_sim/vdpa_sim.c:211:6: warning: variable 'dev' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (!vdpasim->config)
^~~~~~~~~~~~~~~~
drivers/vdpa/vdpa_sim/vdpa_sim.c:242:13: note: uninitialized use occurs here
put_device(dev);
^~~
drivers/vdpa/vdpa_sim/vdpa_sim.c:211:2: note: remove the 'if' if its condition is always false
if (!vdpasim->config)
^~~~~~~~~~~~~~~~~~~~~
drivers/vdpa/vdpa_sim/vdpa_sim.c:189:20: note: initialize the variable 'dev' to silence this warning
struct device *dev;
^
= NULL1 warning generated.
vim +211 drivers/vdpa/vdpa_sim/vdpa_sim.c
183
184 struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)
185 {
186 const struct vdpa_config_ops *ops;
187 struct vdpasim *vdpasim;
188 u32 device_id;
189 struct device *dev;
190 int i, size, ret = -ENOMEM;
191
192 device_id = attr->device_id;
193 /* Currently, we only accept the network and block devices. */
194 if (device_id != VIRTIO_ID_NET && device_id != VIRTIO_ID_BLOCK)
195 return ERR_PTR(-EOPNOTSUPP);
196
197 if (attr->batch_mapping)
198 ops = &vdpasim_batch_config_ops;
199 else
200 ops = &vdpasim_config_ops;
201
202 vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, VDPASIM_VQ_NUM);
203 if (!vdpasim)
204 goto err_alloc;
205
206 if (device_id == VIRTIO_ID_NET)
207 size = sizeof(struct virtio_net_config);
208 else
209 size = sizeof(struct virtio_blk_config);
210 vdpasim->config = kzalloc(size, GFP_KERNEL);
> 211 if (!vdpasim->config)
212 goto err_iommu;
213
214 vdpasim->device_id = device_id;
215 vdpasim->supported_features = attr->features;
216 INIT_WORK(&vdpasim->work, attr->work_fn);
217 spin_lock_init(&vdpasim->lock);
218 spin_lock_init(&vdpasim->iommu_lock);
219
220 dev = &vdpasim->vdpa.dev;
221 dev->dma_mask = &dev->coherent_dma_mask;
222 if (dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)))
223 goto err_iommu;
224 set_dma_ops(dev, &vdpasim_dma_ops);
225
226 vdpasim->iommu = vhost_iotlb_alloc(2048, 0);
227 if (!vdpasim->iommu)
228 goto err_iommu;
229
230 vdpasim->buffer = kmalloc(PAGE_SIZE, GFP_KERNEL);
231 if (!vdpasim->buffer)
232 goto err_iommu;
233
234 for (i = 0; i < VDPASIM_VQ_NUM; i++)
235 vringh_set_iotlb(&vdpasim->vqs[i].vring, vdpasim->iommu);
236
237 vdpasim->vdpa.dma_dev = dev;
238
239 return vdpasim;
240
241 err_iommu:
242 put_device(dev);
243 err_alloc:
244 return ERR_PTR(ret);
245 }
246 EXPORT_SYMBOL_GPL(vdpasim_create);
247
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
In reply to Stefano GarzarellaQuoted 8 lines · Stefano Garzarella
From: Max Gurtovoy <[email protected]>
Introduce new
vdpa_sim_netandvdpa_sim(core) drivers. This is a preparation for adding a vdpa simulator module for block devices.Signed-off-by Max Gurtovoy <[email protected]>[sgarzare: various cleanups/fixes]Signed-off-by Stefano Garzarella <[email protected]>
Quoted 9 lines
---
v1:
- Removed unused headers
- Removed emptymodule_init()module_exit()
- Movedvdpasim_is_little_endian()invdpa_sim.h
- Moved vdpasim16_to_cpu/cpu_to_vdpasim16() invdpa_sim.h
- Added vdpasim*_to_cpu/cpu_to_vdpasim*() also for 32 and 64
- Replaced 'selectVDPA_SIM' with 'depends onVDPA_SIM' since selected option can not depend on other [Jason]
If possible, I would suggest to split this patch further:
1) convert to use void *config, and an attribute for setting config size during allocation
2) introduce supported_features
3) other attributes (#vqs)
4) rename config ops (more generic one)
5) introduce ops for set|get_config, set_get_features
6) real split
Quoted 391 lines
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 110 +++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.c | 285 ++++++--------------------- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 153 ++++++++++++++ drivers/vdpa/Kconfig | 7 +- drivers/vdpa/vdpa_sim/Makefile | 1 + 5 files changed, 329 insertions(+), 227 deletions(-)create mode 100644
drivers/vdpa/vdpa_sim/vdpa_sim.hcreate mode 100644drivers/vdpa/vdpa_sim/vdpa_sim_net.cdrivers/vdpa/vdpa_sim/vdpa_sim.h new file +110
-0@@ -0,0 +1,110 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (c) 2020, Red Hat Inc. All rights reserved. +*/ + +#ifndef _VDPA_SIM_H +#define _VDPA_SIM_H + +#include <linux/vringh.h> +#include <linux/vdpa.h> +#include <linux/vhost_iotlb.h> +#include <uapi/linux/virtio_config.h> +#include <uapi/linux/virtio_net.h> +#include <uapi/linux/virtio_blk.h> + +#define DRV_VERSION "0.1" +#define DRV_AUTHOR "Jason Wang <[email protected]>" +#define DRV_LICENSE "GPL v2" + +#define VDPASIM_QUEUE_ALIGN PAGE_SIZE +#define VDPASIM_QUEUE_MAX 256 +#define VDPASIM_VENDOR_ID 0 +#define VDPASIM_VQ_NUM 0x2 + +#define VDPASIM_FEATURES ((1ULL << VIRTIO_F_ANY_LAYOUT) | \ + (1ULL << VIRTIO_F_VERSION_1) | \ + (1ULL << VIRTIO_F_ACCESS_PLATFORM)) + +struct vdpasim ; + +struct vdpasim_virtqueue { +struct vringh vring; +struct vringh_kiov iov; +unsigned short head; +bool ready; +u64 desc_addr; +u64 device_addr; +u64 driver_addr; +u32 num; +void *private; +irqreturn_t (*cb)(void *data); +}; + +struct vdpasim_init_attr { +u32 device_id; +u64 features; +work_func_t work_fn; +int batch_mapping; +}; + +/* State of each vdpasim device */ +struct vdpasim { +struct vdpa_device vdpa; +struct vdpasim_virtqueue vqs[VDPASIM_VQ_NUM ]; +struct work_struct work; +/* spinlock to synchronize virtqueue state */ +spinlock_t lock; +/* virtio config according to device type */ +void *config; +struct vhost_iotlb *iommu; +void *buffer; +u32 device_id; +u32 status; +u32 generation; +u64 features; +u64 supported_features; +/* spinlock to synchronize iommu table */ +spinlock_t iommu_lock; +}; + +struct vdpasim *vdpasim_create (struct vdpasim_init_attr *attr); + +/* TODO: cross-endian support */ +static inline bool vdpasim_is_little_endian (struct vdpasim *vdpasim) +{ +return virtio_legacy_is_little_endian () || + (vdpasim->features & (1ULL <<VIRTIO_F_VERSION_1 )); +} + +static inline u16 vdpasim16_to_cpu (struct vdpasim *vdpasim,__virtio16 val) +{ +return __virtio16_to_cpu (vdpasim_is_little_endian (vdpasim), val); +} + +static inline __virtio16 cpu_to_vdpasim16 (struct vdpasim *vdpasim,u16 val) +{ +return __cpu_to_virtio16 (vdpasim_is_little_endian (vdpasim), val); +} + +static inline u32 vdpasim32_to_cpu (struct vdpasim *vdpasim,__virtio32 val) +{ +return __virtio32_to_cpu (vdpasim_is_little_endian (vdpasim), val); +} + +static inline __virtio32 cpu_to_vdpasim32 (struct vdpasim *vdpasim,u32 val) +{ +return __cpu_to_virtio32 (vdpasim_is_little_endian (vdpasim), val); +} + +static inline u64 vdpasim64_to_cpu (struct vdpasim *vdpasim,__virtio64 val) +{ +return __virtio64_to_cpu (vdpasim_is_little_endian (vdpasim), val); +} + +static inline __virtio64 cpu_to_vdpasim64 (struct vdpasim *vdpasim,u64 val) +{ +return __cpu_to_virtio64 (vdpasim_is_little_endian (vdpasim), val); +} + +#endif drivers/vdpa/vdpa_sim/vdpa_sim.c +21
-179@@ -1,107 +1,16 @@// SPDX-License-Identifier: GPL-2.0-only /* -* VDPA networking device simulator. +* VDPA simulator core. * * Copyright (c) 2020, Red Hat Inc. All rights reserved. * Author: Jason Wang <[email protected]> * */ -#include <linux/init.h> #include <linux/module.h> -#include <linux/device.h> -#include <linux/kernel.h> -#include <linux/fs.h> -#include <linux/poll.h> -#include <linux/slab.h> -#include <linux/sched.h> -#include <linux/wait.h> -#include <linux/uuid.h> -#include <linux/iommu.h> #include <linux/dma-map-ops.h> -#include <linux/sysfs.h> -#include <linux/file.h> -#include <linux/etherdevice.h> -#include <linux/vringh.h> -#include <linux/vdpa.h> -#include <linux/virtio_byteorder.h> -#include <linux/vhost_iotlb.h> -#include <uapi/linux/virtio_config.h> -#include <uapi/linux/virtio_net.h> - -#define DRV_VERSION "0.1" -#define DRV_AUTHOR "Jason Wang <[email protected]>" -#define DRV_DESC "vDPA Device Simulator" -#define DRV_LICENSE "GPL v2" - -static int batch_mapping =1 ; -module_param (batch_mapping, int,0444 ); -MODULE_PARM_DESC (batch_mapping,"Batched mapping 1 -Enable; 0 - Disable" ); - -static char *macaddr; -module_param (macaddr, charp,0 ); -MODULE_PARM_DESC (macaddr,"Ethernet MAC address" ); - -struct vdpasim_virtqueue { -struct vringh vring; -struct vringh_kiov iov; -unsigned short head; -bool ready; -u64 desc_addr; -u64 device_addr; -u64 driver_addr; -u32 num; -void *private; -irqreturn_t (*cb)(void *data); -}; - -#define VDPASIM_QUEUE_ALIGN PAGE_SIZE -#define VDPASIM_QUEUE_MAX 256 -#define VDPASIM_DEVICE_ID 0x1 -#define VDPASIM_VENDOR_ID 0 -#define VDPASIM_VQ_NUM 0x2 -#define VDPASIM_NAME "vdpasim-netdev" - -static u64 vdpasim_features = (1ULL <<VIRTIO_F_ANY_LAYOUT ) | - (1ULL <<VIRTIO_F_VERSION_1 ) | - (1ULL <<VIRTIO_F_ACCESS_PLATFORM ) | - (1ULL <<VIRTIO_NET_F_MAC ); - -/* State of each vdpasim device */ -struct vdpasim { -struct vdpa_device vdpa; -struct vdpasim_virtqueue vqs[VDPASIM_VQ_NUM ]; -struct work_struct work; -/* spinlock to synchronize virtqueue state */ -spinlock_t lock; -struct virtio_net_config config; -struct vhost_iotlb *iommu; -void *buffer; -u32 status; -u32 generation; -u64 features; -/* spinlock to synchronize iommu table */ -spinlock_t iommu_lock; -}; - -/* TODO: cross-endian support */ -static inline bool vdpasim_is_little_endian (struct vdpasim *vdpasim) -{ -return virtio_legacy_is_little_endian () || - (vdpasim->features & (1ULL <<VIRTIO_F_VERSION_1 )); -} - -static inline u16 vdpasim16_to_cpu (struct vdpasim *vdpasim,__virtio16 val) -{ -return __virtio16_to_cpu (vdpasim_is_little_endian (vdpasim), val); -} - -static inline __virtio16 cpu_to_vdpasim16 (struct vdpasim *vdpasim,u16 val) -{ -return __cpu_to_virtio16 (vdpasim_is_little_endian (vdpasim), val); -} -static struct vdpasim *vdpasim_dev; +#include "vdpa_sim.h" static struct vdpasim *vdpa_to_sim (struct vdpa_device *vdpa) {@@ -119,7 +28,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) {struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; - vringh_init_iotlb(&vq->vring, vdpasim_features, + vringh_init_iotlb(&vq->vring, vdpasim->supported_features, VDPASIM_QUEUE_MAX, false, (struct vring_desc *)(uintptr_t)vq->desc_addr, (struct vring_avail *)@@ -128,7 +37,8 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) (uintptr_t)vq->device_addr); } -static void vdpasim_vq_reset (struct vdpasim_virtqueue *vq) +static void vdpasim_vq_reset (struct vdpasim *vdpasim, +struct vdpasim_virtqueue *vq) { vq->ready = false; vq->desc_addr =0 ;@@ -136,8 +46,8 @@ static void vdpasim_vq_reset(struct vdpasim_virtqueue *vq) vq->device_addr =0 ; vq->cb =NULL ; vq->private =NULL ; -vringh_init_iotlb (&vq->vring, vdpasim_features,VDPASIM_QUEUE_MAX , - false,NULL ,NULL ,NULL ); +vringh_init_iotlb (&vq->vring, vdpasim->supported_features, +VDPASIM_QUEUE_MAX , false,NULL ,NULL ,NULL ); }static void vdpasim_reset (struct vdpasim *vdpasim)@@ -145,7 +55,7 @@ static void vdpasim_reset(struct vdpasim *vdpasim)int i;for (i =0 ; i <VDPASIM_VQ_NUM ; i++) -vdpasim_vq_reset (&vdpasim->vqs[i]); +vdpasim_vq_reset (vdpasim, &vdpasim->vqs[i]);spin_lock (&vdpasim->iommu_lock);vhost_iotlb_reset (vdpasim->iommu);@@ -156,80 +66,6 @@ static void vdpasim_reset(struct vdpasim *vdpasim) ++vdpasim->generation; } -static void vdpasim_work (struct work_struct *work) -{ - struct vdpasim *vdpasim = container_of(work, struct - vdpasim, work); -struct vdpasim_virtqueue *txq = &vdpasim->vqs[1 ]; -struct vdpasim_virtqueue *rxq = &vdpasim->vqs[0 ]; -ssize_t read, write; -size_t total_write; -int pkts =0 ; -int err; - -spin_lock (&vdpasim->lock); - -if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) - goto out; - -if (!txq->ready || !rxq->ready) - goto out; - -while (true) { - total_write =0 ; - err =vringh_getdesc_iotlb (&txq->vring, &txq->iov,NULL , - &txq->head,GFP_ATOMIC ); -if (err <=0 ) -break ; - - err =vringh_getdesc_iotlb (&rxq->vring,NULL , &rxq->iov, - &rxq->head,GFP_ATOMIC ); -if (err <=0 ) { -vringh_complete_iotlb (&txq->vring, txq->head,0 ); -break ; - } - -while (true) { - read =vringh_iov_pull_iotlb (&txq->vring, &txq->iov, - vdpasim->buffer, -PAGE_SIZE ); -if (read <=0 ) -break ; - - write =vringh_iov_push_iotlb (&rxq->vring, &rxq->iov, - vdpasim->buffer, read); -if (write <=0 ) -break ; - - total_write += write; - } - -/* Make sure data is wrote before advancing index */ -smp_wmb (); - -vringh_complete_iotlb (&txq->vring, txq->head,0 ); -vringh_complete_iotlb (&rxq->vring, rxq->head, total_write); - -/* Make sure used is visible before rasing the interrupt. */ -smp_wmb (); - -local_bh_disable (); -if (txq->cb) - txq->cb (txq->private); -if (rxq->cb) - rxq->cb (rxq->private); -local_bh_enable (); - -if (++pkts >4 ) { -schedule_work (&vdpasim->work); - goto out; - } - } - -out: -spin_unlock (&vdpasim->lock); -} -static int dir_to_perm (enum dma_data_direction dir) {int perm = -EFAULT ;@@ -342,26 +178,42 @@ static const struct dma_map_ops vdpasim_dma_ops = { .free = vdpasim_free_coherent, }; -static const struct vdpa_config_ops vdpasim_net_config_ops; -static const struct vdpa_config_ops vdpasim_net_batch_config_ops; +static const struct vdpa_config_ops vdpasim_config_ops; +static const struct vdpa_config_ops vdpasim_batch_config_ops; -static struct vdpasim *vdpasim_create (void ) +struct vdpasim *vdpasim_create (struct vdpasim_init_attr *attr) {const struct vdpa_config_ops *ops;struct vdpasim *vdpasim; +u32 device_id;struct device *dev; -int ret = -ENOMEM ; +int i, size, ret = -ENOMEM ; -if (batch_mapping) - ops = &vdpasim_net_batch_config_ops; + device_id = attr->device_id; +/* Currently, we only accept the network and block devices. */ +if (device_id !=VIRTIO_ID_NET && device_id !=VIRTIO_ID_BLOCK ) +return ERR_PTR (-EOPNOTSUPP ); + +if (attr->batch_mapping) + ops = &vdpasim_batch_config_ops;else - ops = &vdpasim_net_config_ops; + ops = &vdpasim_config_ops;
drivers/vdpa/vdpa_sim/vdpa_sim.c static const struct dma_map_ops vdpasim_dma_ops = { · line 209
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +4
-1vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, VDPASIM_VQ_NUM);if (!vdpasim) goto err_alloc; -INIT_WORK (&vdpasim->work, vdpasim_work); +if (device_id ==VIRTIO_ID_NET ) + size =sizeof (struct virtio_net_config ); +else + size =sizeof (struct virtio_blk_config );
It's better to avoid such if/else consider we may introduce more type of devices.
Can we have an attribute of config size instead?
Quoted 18 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +7
-4+ vdpasim->config =kzalloc (size,GFP_KERNEL ); +if (!vdpasim->config) + goto err_iommu; + + vdpasim->device_id = device_id; + vdpasim->supported_features = attr->features; +INIT_WORK (&vdpasim->work, attr->work_fn);spin_lock_init (&vdpasim->lock);spin_lock_init (&vdpasim->iommu_lock);@@ -379,23 +231,10 @@ static struct vdpasim *vdpasim_create(void)if (!vdpasim->buffer) goto err_iommu; -if (macaddr) { -mac_pton (macaddr, vdpasim->config.mac); -if (!is_valid_ether_addr (vdpasim->config.mac)) { - ret = -EADDRNOTAVAIL ;
drivers/vdpa/vdpa_sim/vdpa_sim.c static struct vdpasim *vdpasim_create(void) · line 235
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +2
-8- goto err_iommu; - } - }else { -eth_random_addr (vdpasim->config.mac); - } - -vringh_set_iotlb (&vdpasim->vqs[0 ].vring, vdpasim->iommu); -vringh_set_iotlb (&vdpasim->vqs[1 ].vring, vdpasim->iommu); +for (i =0 ; i <VDPASIM_VQ_NUM ; i++) +vringh_set_iotlb (&vdpasim->vqs[i].vring, vdpasim->iommu);
And an attribute of #vqs here.
Quoted 40 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +5
-6vdpasim->vdpa.dma_dev = dev; - ret =vdpa_register_device (&vdpasim->vdpa); -if (ret) - goto err_iommu;return vdpasim;@@ -404,6 +243,7 @@ static struct vdpasim *vdpasim_create(void) err_alloc:return ERR_PTR (ret); } +EXPORT_SYMBOL_GPL (vdpasim_create); static int vdpasim_set_vq_address(struct vdpa_device *vdpa, u16 idx, u64 desc_area, u64 driver_area,@@ -498,28 +338,34 @@ static u32 vdpasim_get_vq_align(struct vdpa_device *vdpa)static u64 vdpasim_get_features (struct vdpa_device *vdpa) { -return vdpasim_features; +struct vdpasim *vdpasim =vdpa_to_sim (vdpa); + +return vdpasim->supported_features; }static int vdpasim_set_features (struct vdpa_device *vdpa,u64 features) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -struct virtio_net_config *config = &vdpasim->config;/* DMA mapping must be done by driver */ if (!(features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM )))return -EINVAL ; - vdpasim->features = features & vdpasim_features; + vdpasim->features = features & vdpasim->supported_features;/* We generally only know whether guest is using the legacy interface * here, so generally that's the earliest we can set config fields.
drivers/vdpa/vdpa_sim/vdpa_sim.c static u32 vdpasim_get_vq_align(struct vdpa_device *vdpa) · line 367
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +7
-0* Note: We actually require VIRTIO_F_ACCESS_PLATFORM above which * implies VIRTIO_F_VERSION_1, but let's not try to be clever here. */ +if (vdpasim->device_id ==VIRTIO_ID_NET ) { +struct virtio_net_config *config = + (struct virtio_net_config *)vdpasim->config; + + config->mtu =cpu_to_vdpasim16 (vdpasim,1500 ); + config->status =cpu_to_vdpasim16 (vdpasim,VIRTIO_NET_S_LINK_UP ); + }
Can we introduce callbacks of set_features/get_features here to avoid dealing of device type specific codes in generic simulator code?
Quoted 19 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +3
-3- config->mtu =cpu_to_vdpasim16 (vdpasim,1500 ); - config->status =cpu_to_vdpasim16 (vdpasim,VIRTIO_NET_S_LINK_UP );return 0 ; }@@ -536,7 +382,9 @@ static u16 vdpasim_get_vq_num_max(struct vdpa_device *vdpa)static u32 vdpasim_get_device_id (struct vdpa_device *vdpa) { -return VDPASIM_DEVICE_ID ; +struct vdpasim *vdpasim =vdpa_to_sim (vdpa); + +return vdpasim->device_id; }static u32 vdpasim_get_vendor_id (struct vdpa_device *vdpa)@@ -572,8 +420,12 @@ static void vdpasim_get_config(struct vdpa_device *vdpa, unsigned int offset, {
drivers/vdpa/vdpa_sim/vdpa_sim.c static void vdpasim_get_config(struct vdpa_device *vdpa, unsigned int offset, · line 428
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +6
-2struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -if (offset + len <sizeof (struct virtio_net_config )) -memcpy (buf, (u8 *)&vdpasim->config + offset, len); +if (vdpasim->device_id ==VIRTIO_ID_BLOCK && + (offset + len <sizeof (struct virtio_blk_config ))) +memcpy (buf, vdpasim->config + offset, len); +else if (vdpasim->device_id ==VIRTIO_ID_NET && + (offset + len <sizeof (struct virtio_net_config ))) +memcpy (buf, vdpasim->config + offset, len);
Similarly, can we introduce set/get_config ops?
Quoted 68 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +4
-23} static void vdpasim_set_config(struct vdpa_device *vdpa, unsigned int offset,@@ -659,9 +511,10 @@ static void vdpasim_free(struct vdpa_device *vdpa)kfree (vdpasim->buffer);if (vdpasim->iommu)vhost_iotlb_free (vdpasim->iommu); +kfree (vdpasim->config); } -static const struct vdpa_config_ops vdpasim_net_config_ops = { +static const struct vdpa_config_ops vdpasim_config_ops = { .set_vq_address = vdpasim_set_vq_address, .set_vq_num = vdpasim_set_vq_num, .kick_vq = vdpasim_kick_vq,@@ -688,7 +541,7 @@ static const struct vdpa_config_ops vdpasim_net_config_ops = { .free = vdpasim_free, }; -static const struct vdpa_config_ops vdpasim_net_batch_config_ops = { +static const struct vdpa_config_ops vdpasim_batch_config_ops = { .set_vq_address = vdpasim_set_vq_address, .set_vq_num = vdpasim_set_vq_num, .kick_vq = vdpasim_kick_vq,@@ -714,27 +567,7 @@ static const struct vdpa_config_ops vdpasim_net_batch_config_ops = { .free = vdpasim_free, }; -static int __init vdpasim_dev_init (void ) -{ - vdpasim_dev =vdpasim_create (); - -if (!IS_ERR (vdpasim_dev)) -return 0 ; - -return PTR_ERR (vdpasim_dev); -} - -static void __exit vdpasim_dev_exit(void) -{ -struct vdpa_device *vdpa = &vdpasim_dev->vdpa; - -vdpa_unregister_device (vdpa); -} - -module_init(vdpasim_dev_init) -module_exit(vdpasim_dev_exit) -MODULE_VERSION (DRV_VERSION );MODULE_LICENSE (DRV_LICENSE );MODULE_AUTHOR (DRV_AUTHOR ); -MODULE_DESCRIPTION(DRV_DESC); +MODULE_DESCRIPTION ("vDPA Simulator core" );drivers/vdpa/vdpa_sim/vdpa_sim_net.c new file +9
-0@@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* VDPA simulator for networking device. +* +* Copyright (c) 2020, Red Hat Inc. All rights reserved. +* Author: Jason Wang <[email protected]> +* +*/ +
drivers/vdpa/vdpa_sim/vdpa_sim_net.c line 19
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_net.c new file +10
-0+#include <linux/module.h> +#include <linux/etherdevice.h> + +#include "vdpa_sim.h" + +#define VDPASIM_NET_FEATURES (1ULL << VIRTIO_NET_F_MAC) + +static int batch_mapping =1 ; +module_param (batch_mapping, int,0444 ); +MODULE_PARM_DESC (batch_mapping,"Batched mapping 1 - Enable; 0 - Disable" );
I think batch_mapping should belong to vpda_sim core module.
Quoted 143 lines
drivers/vdpa/vdpa_sim/vdpa_sim_net.c new file +134
-0+ +static char *macaddr; +module_param (macaddr, charp,0 ); +MODULE_PARM_DESC (macaddr,"Ethernet MAC address" ); + +static struct vdpasim *vdpasim_net_dev; + +static void vdpasim_net_work (struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); +struct vdpasim_virtqueue *txq = &vdpasim->vqs[1 ]; +struct vdpasim_virtqueue *rxq = &vdpasim->vqs[0 ]; +ssize_t read, write; +size_t total_write; +int pkts =0 ; +int err; + +spin_lock (&vdpasim->lock); + +if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) + goto out; + +if (!txq->ready || !rxq->ready) + goto out; + +while (true) { + total_write =0 ; + err =vringh_getdesc_iotlb (&txq->vring, &txq->iov,NULL , + &txq->head,GFP_ATOMIC ); +if (err <=0 ) +break ; + + err =vringh_getdesc_iotlb (&rxq->vring,NULL , &rxq->iov, + &rxq->head,GFP_ATOMIC ); +if (err <=0 ) { +vringh_complete_iotlb (&txq->vring, txq->head,0 ); +break ; + } + +while (true) { + read =vringh_iov_pull_iotlb (&txq->vring, &txq->iov, + vdpasim->buffer, +PAGE_SIZE ); +if (read <=0 ) +break ; + + write =vringh_iov_push_iotlb (&rxq->vring, &rxq->iov, + vdpasim->buffer, read); +if (write <=0 ) +break ; + + total_write += write; + } + +/* Make sure data is wrote before advancing index */ +smp_wmb (); + +vringh_complete_iotlb (&txq->vring, txq->head,0 ); +vringh_complete_iotlb (&rxq->vring, rxq->head, total_write); + +/* Make sure used is visible before rasing the interrupt. */ +smp_wmb (); + +local_bh_disable (); +if (txq->cb) + txq->cb (txq->private); +if (rxq->cb) + rxq->cb (rxq->private); +local_bh_enable (); + +if (++pkts >4 ) { +schedule_work (&vdpasim->work); + goto out; + } + } + +out: +spin_unlock (&vdpasim->lock); +} + +static int __init vdpasim_net_init (void ) +{ +struct vdpasim_init_attr attr = {}; +struct virtio_net_config *config; +int ret; + + attr.device_id =VIRTIO_ID_NET ; + attr.features =VDPASIM_FEATURES |VDPASIM_NET_FEATURES ; + attr.work_fn = vdpasim_net_work; + attr.batch_mapping = batch_mapping; + vdpasim_net_dev =vdpasim_create (&attr); +if (IS_ERR (vdpasim_net_dev)) { + ret =PTR_ERR (vdpasim_net_dev); + goto out; + } + + config = (struct virtio_net_config *)vdpasim_net_dev->config; + +if (macaddr) { +mac_pton (macaddr, config->mac); +if (!is_valid_ether_addr (config->mac)) { + ret = -EADDRNOTAVAIL ; + goto put_dev; + } + }else { +eth_random_addr (config->mac); + } + + ret =vdpa_register_device (&vdpasim_net_dev->vdpa); +if (ret) + goto put_dev; + +return 0 ; + +put_dev: +put_device (&vdpasim_net_dev->vdpa.dev); +out: +return ret; +} + +static void __exit vdpasim_net_exit(void) +{ +struct vdpa_device *vdpa = &vdpasim_net_dev->vdpa; + +vdpa_unregister_device (vdpa); +} + +module_init (vdpasim_net_init); +module_exit (vdpasim_net_exit); + +MODULE_VERSION (DRV_VERSION ); +MODULE_LICENSE (DRV_LICENSE ); +MODULE_AUTHOR (DRV_AUTHOR ); +MODULE_DESCRIPTION ("vDPA Device Simulator for networking device" );drivers/vdpa/Kconfig +0
-1@@ -9,11 +9,16 @@ menuconfig VDPA if VDPA config VDPA_SIM - tristate "vDPA device simulator"
drivers/vdpa/Kconfig menuconfig VDPA · line 21
Quoted 10 lines
drivers/vdpa/Kconfig +6
-0+ tristate "vDPA simulator core" depends on RUNTIME_TESTING_MENU && HAS_DMA select DMA_OPS select VHOST_RING default n + +config VDPA_SIM_NET + tristate "vDPA simulator for networking device" + depends on VDPA_SIM + default n
I remember somebody told me that if we don't enable a module it was disabled by default.
Thanks
drivers/vdpa/vdpa_sim/Makefile line 3
Quoted 11 lines
drivers/vdpa/Kconfig +0
-0help vDPA networking device simulator which loop TX traffic back to RX. This device is used for testing, prototyping anddrivers/vdpa/vdpa_sim/Makefile +1
-0@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_VDPA_SIM) += vdpa_sim.o +obj-$(CONFIG_VDPA_SIM_NET) += vdpa_sim_net.o
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
Quoted 21 lines · Jason Wang
Quoted 8 lines · Stefano Garzarella
From: Max Gurtovoy <[email protected]>
Introduce new
vdpa_sim_netandvdpa_sim(core) drivers. This is a preparation for adding a vdpa simulator module for block devices.Signed-off-by Max Gurtovoy <[email protected]>[sgarzare: various cleanups/fixes]Signed-off-by Stefano Garzarella <[email protected]>Quoted 9 lines
---
v1:
- Removed unused headers
- Removed emptymodule_init()module_exit()
- Movedvdpasim_is_little_endian()invdpa_sim.h
- Moved vdpasim16_to_cpu/cpu_to_vdpasim16() invdpa_sim.h
- Added vdpasim*_to_cpu/cpu_to_vdpasim*() also for 32 and 64
- Replaced 'selectVDPA_SIM' with 'depends onVDPA_SIM' since selected option can not depend on other [Jason]
Quoted 9 lines
If possible, I would suggest to split this patch further:
1) convert to use void *config, and an attribute for setting config size during allocation
2) introducesupported_features
3) other attributes (#vqs)
4) rename config ops (more generic one)
5) introduce ops for set|get_config,set_get_features
6) real split
Okay, I'll try to split Max's patch following your suggestion. It should be cleaner.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
In reply to Jason WangHi Jason,
I just discovered that I missed the other questions in this email, sorry for that!
Quoted 31 lines · Jason Wang
Quoted 8 lines · Stefano Garzarella
From: Max Gurtovoy <[email protected]>
Introduce new
vdpa_sim_netandvdpa_sim(core) drivers. This is a preparation for adding a vdpa simulator module for block devices.Signed-off-by Max Gurtovoy <[email protected]>[sgarzare: various cleanups/fixes]Signed-off-by Stefano Garzarella <[email protected]>Quoted 9 lines
---
v1:
- Removed unused headers
- Removed emptymodule_init()module_exit()
- Movedvdpasim_is_little_endian()invdpa_sim.h
- Moved vdpasim16_to_cpu/cpu_to_vdpasim16() invdpa_sim.h
- Added vdpasim*_to_cpu/cpu_to_vdpasim*() also for 32 and 64
- Replaced 'selectVDPA_SIM' with 'depends onVDPA_SIM' since selected option can not depend on other [Jason]If possible, I would suggest to split this patch further:
1) convert to use void *config, and an attribute for setting config size during allocation
2) introducesupported_features
3) other attributes (#vqs)
4) rename config ops (more generic one)
5) introduce ops for set|get_config,set_get_features
6) real split
[...]
Quoted 37 lines
Quoted 23 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +12
-6-static const struct vdpa_config_ops vdpasim_net_config_ops; -static const struct vdpa_config_ops vdpasim_net_batch_config_ops; +static const struct vdpa_config_ops vdpasim_config_ops; +static const struct vdpa_config_ops vdpasim_batch_config_ops; -static struct vdpasim *vdpasim_create (void ) +struct vdpasim *vdpasim_create (struct vdpasim_init_attr *attr) {const struct vdpa_config_ops *ops;struct vdpasim *vdpasim; +u32 device_id;struct device *dev; -int ret = -ENOMEM ; +int i, size, ret = -ENOMEM ; -if (batch_mapping) - ops = &vdpasim_net_batch_config_ops; + device_id = attr->device_id; +/* Currently, we only accept the network and block devices. */ +if (device_id !=VIRTIO_ID_NET && device_id !=VIRTIO_ID_BLOCK ) +return ERR_PTR (-EOPNOTSUPP ); + +if (attr->batch_mapping) + ops = &vdpasim_batch_config_ops;else drivers/vdpa/vdpa_sim/vdpa_sim.c static const struct dma_map_ops vdpasim_dma_ops = { · line 209
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +5
-2- ops = &vdpasim_net_config_ops; + ops = &vdpasim_config_ops; vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, VDPASIM_VQ_NUM);if (!vdpasim) goto err_alloc; -INIT_WORK (&vdpasim->work, vdpasim_work); +if (device_id ==VIRTIO_ID_NET ) + size =sizeof (struct virtio_net_config ); +else + size =sizeof (struct virtio_blk_config );It's better to avoid such if/else consider we may introduce more type of devices.
Quoted 1 line
Can we have an attribute of config size instead?
Yes, I'll move the patch 7 before this.
About config size and set/get_config ops, I'm not sure if it is better to hidden everything under the new set/get_config ops, allocating the config structure in each device, or leave the allocation in the core and update it like now.
Quoted 24 lines
Quoted 24 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +7
-10+ vdpasim->config =kzalloc (size,GFP_KERNEL ); +if (!vdpasim->config) + goto err_iommu; + + vdpasim->device_id = device_id; + vdpasim->supported_features = attr->features; +INIT_WORK (&vdpasim->work, attr->work_fn);spin_lock_init (&vdpasim->lock);spin_lock_init (&vdpasim->iommu_lock);@@ -379,23 +231,10 @@ static struct vdpasim *vdpasim_create(void)if (!vdpasim->buffer) goto err_iommu; -if (macaddr) { -mac_pton (macaddr, vdpasim->config.mac); -if (!is_valid_ether_addr (vdpasim->config.mac)) { - ret = -EADDRNOTAVAIL ; - goto err_iommu; - } - }else { -eth_random_addr (vdpasim->config.mac); - } -
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c static struct vdpasim *vdpasim_create(void)
Quoted 5 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +2
-2-vringh_set_iotlb (&vdpasim->vqs[0 ].vring, vdpasim->iommu); -vringh_set_iotlb (&vdpasim->vqs[1 ].vring, vdpasim->iommu); +for (i =0 ; i <VDPASIM_VQ_NUM ; i++) +vringh_set_iotlb (&vdpasim->vqs[i].vring, vdpasim->iommu);And an attribute of #vqs here.
Yes.
Quoted 39 lines
Quoted 39 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +7
-6vdpasim->vdpa.dma_dev = dev; - ret =vdpa_register_device (&vdpasim->vdpa); -if (ret) - goto err_iommu;return vdpasim;@@ -404,6 +243,7 @@ static struct vdpasim *vdpasim_create(void) err_alloc:return ERR_PTR (ret); } +EXPORT_SYMBOL_GPL (vdpasim_create); static int vdpasim_set_vq_address(struct vdpa_device *vdpa, u16 idx, u64 desc_area, u64 driver_area,@@ -498,28 +338,34 @@ static u32 vdpasim_get_vq_align(struct vdpa_device *vdpa)static u64 vdpasim_get_features (struct vdpa_device *vdpa) { -return vdpasim_features; +struct vdpasim *vdpasim =vdpa_to_sim (vdpa); + +return vdpasim->supported_features; }static int vdpasim_set_features (struct vdpa_device *vdpa,u64 features) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -struct virtio_net_config *config = &vdpasim->config;/* DMA mapping must be done by driver */ if (!(features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM )))return -EINVAL ; - vdpasim->features = features & vdpasim_features; + vdpasim->features = features & vdpasim->supported_features;/* We generally only know whether guest is using the legacy interface * here, so generally that's the earliest we can set config fields. * Note: We actually require VIRTIO_F_ACCESS_PLATFORM above which * implies VIRTIO_F_VERSION_1, but let's not try to be clever here. */ +if (vdpasim->device_id ==VIRTIO_ID_NET ) { + struct virtio_net_config *config =
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c static u32 vdpasim_get_vq_align(struct vdpa_device *vdpa) · line 367
Quoted 5 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +5
-0+ (struct virtio_net_config *)vdpasim->config; + + config->mtu =cpu_to_vdpasim16 (vdpasim,1500 ); + config->status =cpu_to_vdpasim16 (vdpasim,VIRTIO_NET_S_LINK_UP ); + }Can we introduce callbacks of set_features/get_features here to avoid dealing of device type specific codes in generic simulator code?
Yes, I'll do.
Quoted 24 lines
Quoted 24 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +4
-5- config->mtu =cpu_to_vdpasim16 (vdpasim,1500 ); - config->status =cpu_to_vdpasim16 (vdpasim, VIRTIO_NET_S_LINK_UP);return 0 ; }@@ -536,7 +382,9 @@ static u16 vdpasim_get_vq_num_max(struct vdpa_device *vdpa)static u32 vdpasim_get_device_id (struct vdpa_device *vdpa) { -return VDPASIM_DEVICE_ID ; +struct vdpasim *vdpasim =vdpa_to_sim (vdpa); + +return vdpasim->device_id; }static u32 vdpasim_get_vendor_id (struct vdpa_device *vdpa)@@ -572,8 +420,12 @@ static void vdpasim_get_config(struct vdpa_device *vdpa, unsigned int offset, {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -if (offset + len <sizeof (struct virtio_net_config )) -memcpy (buf, (u8 *)&vdpasim->config + offset, len); + if (vdpasim->device_id == VIRTIO_ID_BLOCK &&
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c static void vdpasim_get_config(struct vdpa_device *vdpa, unsigned int offset, · line 428
Quoted 5 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +5
-0+ (offset + len < sizeof(struct virtio_blk_config))) +memcpy (buf, vdpasim->config + offset, len); + else if (vdpasim->device_id == VIRTIO_ID_NET && + (offset + len < sizeof(struct virtio_net_config))) + memcpy(buf, vdpasim->config + offset, len);Similarly, can we introduce set/get_config ops?
Ditto.
Quoted 2 lines
[...]
Quoted 19 lines
Quoted 19 lines
drivers/vdpa/vdpa_sim/vdpa_sim_net.c new file +13
-0@@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* VDPA simulator for networking device. +* +* Copyright (c) 2020, Red Hat Inc. All rights reserved. +* Author: Jason Wang <[email protected]> +* +*/ + +#include <linux/module.h> +#include <linux/etherdevice.h> + +#include "vdpa_sim.h"
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_net.c line 19
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim_net.c new file +6
-0+ +#define VDPASIM_NET_FEATURES (1ULL << VIRTIO_NET_F_MAC) + +static int batch_mapping =1 ; +module_param (batch_mapping, int,0444 ); +MODULE_PARM_DESC (batch_mapping,"Batched mapping 1 - Enable; 0 - Disable" );I think
batch_mappingshould belong tovpda_simcore module.
Yes, I agree, I'll leave it in the core.
Quoted 2 lines
[...]
Quoted 13 lines
Quoted 13 lines
drivers/vdpa/Kconfig +1
-1@@ -9,11 +9,16 @@ menuconfig VDPA if VDPA config VDPA_SIM - tristate "vDPA device simulator" + tristate "vDPA simulator core" depends on RUNTIME_TESTING_MENU && HAS_DMA select DMA_OPS select VHOST_RING default n
Quoted 8 lines
drivers/vdpa/Kconfig menuconfig VDPA · line 21
Quoted 5 lines
drivers/vdpa/Kconfig +5
-0+ +config VDPA_SIM_NET + tristate "vDPA simulator for networking device" + depends on VDPA_SIM + default nI remember somebody told me that if we don't enable a module it was disabled by default.
So, should I remove "default n" from vdpa_sim* entries?
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
Quoted 3 lines · Stefano Garzarella
Hi Jason,
I just discovered that I missed the other questions in this email, sorry for that!
No problem :)
Quoted 73 lines · Jason Wang
Quoted 31 lines · Stefano Garzarella
Quoted 8 lines
From: Max Gurtovoy <[email protected]>
Introduce new
vdpa_sim_netandvdpa_sim(core) drivers. This is a preparation for adding a vdpa simulator module for block devices.Signed-off-by Max Gurtovoy <[email protected]>[sgarzare: various cleanups/fixes]Signed-off-by Stefano Garzarella <[email protected]>Quoted 9 lines
---
v1:
- Removed unused headers
- Removed emptymodule_init()module_exit()
- Movedvdpasim_is_little_endian()invdpa_sim.h
- Moved vdpasim16_to_cpu/cpu_to_vdpasim16() invdpa_sim.h
- Added vdpasim*_to_cpu/cpu_to_vdpasim*() also for 32 and 64
- Replaced 'selectVDPA_SIM' with 'depends onVDPA_SIM' since selected option can not depend on other [Jason]If possible, I would suggest to split this patch further:
1) convert to use void *config, and an attribute for setting config size during allocation
2) introducesupported_features
3) other attributes (#vqs)
4) rename config ops (more generic one)
5) introduce ops for set|get_config,set_get_features
6) real split[...]
Quoted 37 lines
Quoted 25 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +3
-3-static const struct vdpa_config_ops vdpasim_net_config_ops; -static const struct vdpa_config_ops vdpasim_net_batch_config_ops; +static const struct vdpa_config_ops vdpasim_config_ops; +static const struct vdpa_config_ops vdpasim_batch_config_ops; -static struct vdpasim *vdpasim_create (void ) +struct vdpasim *vdpasim_create (struct vdpasim_init_attr *attr){
const structvdpa_config_ops*ops;
struct vdpasim *vdpasim;
+ u32device_id;
struct device *dev;
- int ret =-ENOMEM;
+ int i, size, ret =-ENOMEM;
- if (batch_mapping)
- ops =&vdpasim_net_batch_config_ops;
+device_id=attr->device_id;
+ /* Currently, we only accept the network and block devices. */ + if (device_id!=VIRTIO_ID_NET&&device_id!=VIRTIO_ID_BLOCK) + returnERR_PTR(-EOPNOTSUPP);
+
+ if (attr->batch_mapping)
+ ops =&vdpasim_batch_config_ops;
else
- ops =&vdpasim_net_config_ops;
+ ops =&vdpasim_config_ops;Quoted 8 lines
vdpasim =
vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, VDPASIM_VQ_NUM); if (!vdpasim)
gotoerr_alloc;
-INIT_WORK(&vdpasim->work, vdpasim_work);
+ if (device_id==VIRTIO_ID_NET)
+ size =sizeof(struct virtio_net_config);
+ else
+ size =sizeof(struct virtio_blk_config);It's better to avoid such if/else consider we may introduce more type of devices.
Quoted 5 lines
Quoted 1 line
Can we have an attribute of config size instead?
Yes, I'll move the patch 7 before this.
About config size and set/get_config ops, I'm not sure if it is better to hidden everything under the new set/get_config ops, allocating the config structure in each device, or leave the allocation in the core and update it like now.
I think we'd better to avoid having any type specific codes in generic sim codes.
[...]
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
+config
VDPA_SIM_NET
+ tristate "vDPA simulator for networking device" + depends onVDPA_SIM
+ default nI remember somebody told me that if we don't enable a module it was disabled by default.
So, should I remove "default n" from vdpa_sim* entries?
Yes, but please do that in another patch.
Thanks
Quoted 4 lines
Thanks,
Stefano
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
In reply to Stefano GarzarellaHi Stefano,
url: https://github.com/0day-ci/linux/commits/Stefano-Garzarella/vdpa-generalize-vdpa-simulator-and-add-block-device/20201113-215247 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 585e5b17b92dead8a3aca4e3c9876fbca5f7e0ba config: x86_64-randconfig-m001-20201114 (attached as .config) compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
If you fix the issue, kindly add following tag as appropriate
smatch warnings:drivers/vdpa/vdpa_sim/vdpa_sim.c:242 vdpasim_create() error: uninitialized symbol 'dev'.
vim +/dev +242 drivers/vdpa/vdpa_sim/vdpa_sim.c
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 184 struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)
2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 185 {
de91a4d0e725db3 drivers/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-08-04 186 const struct vdpa_config_ops *ops;
2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 187 struct vdpasim *vdpasim;
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 188 u32 device_id;
2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 189 struct device *dev;
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 190 int i, size, ret = -ENOMEM;
2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 191
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 192 device_id = attr->device_id;
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 193 /* Currently, we only accept the network and block devices. */
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 194 if (device_id != VIRTIO_ID_NET && device_id != VIRTIO_ID_BLOCK)
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 195 return ERR_PTR(-EOPNOTSUPP);
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 196
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 197 if (attr->batch_mapping)
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 198 ops = &vdpasim_batch_config_ops;
de91a4d0e725db3 drivers/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-08-04 199 else
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 200 ops = &vdpasim_config_ops;
de91a4d0e725db3 drivers/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-08-04 201
a9974489b61c09c drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-08-04 202 vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, VDPASIM_VQ_NUM);
2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 203 if (!vdpasim)
2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 204 goto err_alloc;
2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 205
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 206 if (device_id == VIRTIO_ID_NET)
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 207 size = sizeof(struct virtio_net_config);
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 208 else
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 209 size = sizeof(struct virtio_blk_config);
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 210 vdpasim->config = kzalloc(size, GFP_KERNEL);
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 211 if (!vdpasim->config)
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 212 goto err_iommu;
^^^^^^^^^^^^^^"dev" not initialized on this path.
12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 213 12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 214 vdpasim->device_id = device_id; 12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 215 vdpasim->supported_features = attr->features; 12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 216 INIT_WORK(&vdpasim->work, attr->work_fn); 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 217 spin_lock_init(&vdpasim->lock); 1e3e792650d2c0d drivers/vdpa/vdpa_sim/vdpa_sim.c Michael S. Tsirkin 2020-08-10 218 spin_lock_init(&vdpasim->iommu_lock); 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 219 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 220 dev = &vdpasim->vdpa.dev; 1eca16b231570c8 drivers/vdpa/vdpa_sim/vdpa_sim.c Laurent Vivier 2020-10-27 221 dev->dma_mask = &dev->coherent_dma_mask; 1eca16b231570c8 drivers/vdpa/vdpa_sim/vdpa_sim.c Laurent Vivier 2020-10-27 222 if (dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64))) 1eca16b231570c8 drivers/vdpa/vdpa_sim/vdpa_sim.c Laurent Vivier 2020-10-27 223 goto err_iommu; 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 224 set_dma_ops(dev, &vdpasim_dma_ops); 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 225 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 226 vdpasim->iommu = vhost_iotlb_alloc(2048, 0); 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 227 if (!vdpasim->iommu) 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 228 goto err_iommu; 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 229 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 230 vdpasim->buffer = kmalloc(PAGE_SIZE, GFP_KERNEL); 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 231 if (!vdpasim->buffer) 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 232 goto err_iommu; 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 233 12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 234 for (i = 0; i < VDPASIM_VQ_NUM; i++) 12adecc2babdd7b drivers/vdpa/vdpa_sim/vdpa_sim.c Max Gurtovoy 2020-11-13 235 vringh_set_iotlb(&vdpasim->vqs[i].vring, vdpasim->iommu); 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 236 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 237 vdpasim->vdpa.dma_dev = dev; 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 238 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 239 return vdpasim; 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 240 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 241 err_iommu: 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 @242 put_device(dev); 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 243 err_alloc: 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 244 return ERR_PTR(ret); 2c53d0f64c06f45 drivers/virtio/vdpa/vdpa_sim/vdpa_sim.c Jason Wang 2020-03-26 245 }
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
Hi Dan,
Quoted 7 lines · Dan Carpenter
Hi Stefano,
url: https://github.com/0day-ci/linux/commits/Stefano-Garzarella/vdpa-generalize-vdpa-simulator-and-add-block-device/20201113-215247 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 585e5b17b92dead8a3aca4e3c9876fbca5f7e0ba config: x86_64-randconfig-m001-20201114 (attached as .config) compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
Quoted 6 lines
If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>Reported-by Dan Carpenter <[email protected]>smatch warnings:
drivers/vdpa/vdpa_sim/vdpa_sim.c:242vdpasim_create()error: uninitialized symbol 'dev'.
thanks for the report!
'kernel test robot' already sent me the same warning, and I'm surprised that my gcc 10.2.1 didn't report anything to me.
I'll fix in the next version.
Thanks,
Stefano
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
Quoted 16 lines · Stefano Garzarella
Hi Dan,
Quoted 13 lines · Dan Carpenter
Hi Stefano,
url: https://github.com/0day-ci/linux/commits/Stefano-Garzarella/vdpa-generalize-vdpa-simulator-and-add-block-device/20201113-215247
base:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git585e5b17b92dead8a3aca4e3c9876fbca5f7e0ba
config: x86_64-randconfig-m001-20201114 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>Reported-by Dan Carpenter <[email protected]>
Quoted 6 lines
Quoted 2 lines
smatch warnings:
drivers/vdpa/vdpa_sim/vdpa_sim.c:242vdpasim_create()error: uninitialized symbol 'dev'.thanks for the report!
'kernel test robot' already sent me the same warning, and I'm surprised that my gcc 10.2.1 didn't report anything to me.
These were disabled in commit 78a5255ffb6a ("Stop the ad-hoc games with -Wno-maybe-initialized"). You'd have to build with W=1
Quoted 2 lines
I'll fix in the next version.
Which static checker was the kbuild test bot using? I have spent some ten minutes searching for the report but I haven't been able to find it.
regards,
dan carpenter
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
Quoted 20 lines · Dan Carpenter
Quoted 19 lines · Stefano Garzarella
Hi Dan,
Quoted 15 lines · Dan Carpenter
Hi Stefano,
url: https://github.com/0day-ci/linux/commits/Stefano-Garzarella/vdpa-generalize-vdpa-simulator-and-add-block-device/20201113-215247
base:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git585e5b17b92dead8a3aca4e3c9876fbca5f7e0ba
config: x86_64-randconfig-m001-20201114 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>Reported-by Dan Carpenter <[email protected]>smatch warnings:
drivers/vdpa/vdpa_sim/vdpa_sim.c:242vdpasim_create()error: uninitialized symbol 'dev'.
Quoted 7 lines
Quoted 3 lines
thanks for the report!
'kernel test robot' already sent me the same warning, and I'm surprised that my gcc 10.2.1 didn't report anything to me.These were disabled in commit
78a5255ffb6a("Stop the ad-hoc games with -Wno-maybe-initialized"). You'd have to build with W=1
Thanks for pointed that out!
Quoted 5 lines
Quoted 2 lines
I'll fix in the next version.
Which static checker was the kbuild test bot using? I have spent some ten minutes searching for the report but I haven't been able to find it.
This is the report that I received:
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org/thread/W3XHO6YPAPK7P2HXRXGA455BBGYTPHD5/
Note: the search box in lists.01.org is not working well...
Thanks,
Stefano
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
Quoted 34 lines · Stefano Garzarella
Quoted 20 lines · Dan Carpenter
Quoted 19 lines · Stefano Garzarella
Hi Dan,
Quoted 15 lines · Dan Carpenter
Hi Stefano,
url: https://github.com/0day-ci/linux/commits/Stefano-Garzarella/vdpa-generalize-vdpa-simulator-and-add-block-device/20201113-215247
base:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git585e5b17b92dead8a3aca4e3c9876fbca5f7e0ba
config: x86_64-randconfig-m001-20201114 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>Reported-by Dan Carpenter <[email protected]>smatch warnings:
drivers/vdpa/vdpa_sim/vdpa_sim.c:242vdpasim_create()error: uninitialized symbol 'dev'.Quoted 7 lines
Quoted 3 lines
thanks for the report!
'kernel test robot' already sent me the same warning, and I'm surprised that my gcc 10.2.1 didn't report anything to me.These were disabled in commit
78a5255ffb6a("Stop the ad-hoc games with -Wno-maybe-initialized"). You'd have to build with W=1Thanks for pointed that out!
Quoted 3 lines
Quoted 2 lines
I'll fix in the next version.
Quoted 7 lines
Quoted 2 lines
Which static checker was the kbuild test bot using? I have spent some ten minutes searching for the report but I haven't been able to find it.
This is the report that I received:
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org/thread/W3XHO6YPAPK7P2HXRXGA455BBGYTPHD5/Note: the search box in lists.01.org is not working well...
This is going to be frustrating because the W=1 warnings will come first then the Smatch warnings. The reason that the Smatch warnings exist is because GCC doesn't catch everthing... So they both are required, but most warnings are going to be duplicative.
Also I think people assume I'm going to look at the warning and then check whether it exists in their latest devel tree. But actually I just look at the warning email and forward it on.
Anyway, if you don't want duplicate static checker warnings, don't introduce bugs. I'm really good at ignoring complaints about duplicates.
regards,
dan carpenter
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
Quoted 48 lines · Dan Carpenter
Quoted 34 lines · Stefano Garzarella
Quoted 20 lines · Dan Carpenter
Quoted 19 lines · Stefano Garzarella
Hi Dan,
Quoted 15 lines · Dan Carpenter
Hi Stefano,
url: https://github.com/0day-ci/linux/commits/Stefano-Garzarella/vdpa-generalize-vdpa-simulator-and-add-block-device/20201113-215247
base:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git585e5b17b92dead8a3aca4e3c9876fbca5f7e0ba
config: x86_64-randconfig-m001-20201114 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>Reported-by Dan Carpenter <[email protected]>smatch warnings:
drivers/vdpa/vdpa_sim/vdpa_sim.c:242vdpasim_create()error: uninitialized symbol 'dev'.Quoted 7 lines
Quoted 3 lines
thanks for the report!
'kernel test robot' already sent me the same warning, and I'm surprised that my gcc 10.2.1 didn't report anything to me.These were disabled in commit
78a5255ffb6a("Stop the ad-hoc games with -Wno-maybe-initialized"). You'd have to build with W=1Thanks for pointed that out!
Quoted 3 lines
Quoted 2 lines
I'll fix in the next version.
Quoted 7 lines
Quoted 2 lines
Which static checker was the kbuild test bot using? I have spent some ten minutes searching for the report but I haven't been able to find it.
This is the report that I received:
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org/thread/W3XHO6YPAPK7P2HXRXGA455BBGYTPHD5/Note: the search box in lists.01.org is not working well...
This is going to be frustrating because the W=1 warnings will come first then the Smatch warnings. The reason that the Smatch warnings exist is because GCC doesn't catch everthing... So they both are required, but most warnings are going to be duplicative.
Quoted 7 lines
Also I think people assume I'm going to look at the warning and then check whether it exists in their latest devel tree. But actually I just look at the warning email and forward it on.
Anyway, if you don't want duplicate static checker warnings, don't introduce bugs. I'm really good at ignoring complaints about duplicates.
It's fine for me to have duplicate static checker warnings.
I thanked you and only said that I had received the same warning and I was surprised that my compiler hadn't reported anything to me, as it was a real bug.
Stefano
Re: [PATCH RFC 02/12] vdpa: split vdpasim to core and net modules
Quoted 58 lines · Stefano Garzarella
Quoted 49 lines · Dan Carpenter
Quoted 35 lines · Stefano Garzarella
Quoted 9 lines · Dan Carpenter
Quoted 8 lines · Stefano Garzarella
Hi Dan,
Quoted 5 lines · Dan Carpenter
Hi Stefano,
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git >
Quoted 11 lines
Quoted 11 lines
585e5b17b92dead8a3aca4e3c9876fbca5f7e0baQuoted 9 lines
config: x86_64-randconfig-m001-20201114 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>Reported-by Dan Carpenter <[email protected]>smatch warnings:
drivers/vdpa/vdpa_sim/vdpa_sim.c:242vdpasim_create()error: uninitialized symbol 'dev'.Quoted 7 lines
Quoted 3 lines
thanks for the report!
'kernel test robot' already sent me the same warning, and I'm surprised that my gcc 10.2.1 didn't report anything to me.These were disabled in commit
78a5255ffb6a("Stop the ad-hoc games with -Wno-maybe-initialized"). You'd have to build with W=1Thanks for pointed that out!
Quoted 3 lines
Quoted 2 lines
I'll fix in the next version.
Quoted 7 lines
Quoted 2 lines
Which static checker was the kbuild test bot using? I have spent some ten minutes searching for the report but I haven't been able to find it.
This is the report that I received:
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org/thread/W3XHO6YPAPK7P2HXRXGA455BBGYTPHD5/Note: the search box in lists.01.org is not working well...
This is going to be frustrating because the W=1 warnings will come first then the Smatch warnings. The reason that the Smatch warnings exist is because GCC doesn't catch everthing... So they both are required, but most warnings are going to be duplicative.
Quoted 7 lines
Also I think people assume I'm going to look at the warning and then check whether it exists in their latest devel tree. But actually I just look at the warning email and forward it on.
Anyway, if you don't want duplicate static checker warnings, don't introduce bugs. I'm really good at ignoring complaints about duplicates.
Quoted 5 lines
It's fine for me to have duplicate static checker warnings.
I thanked you and only said that I had received the same warning and I was surprised that my compiler hadn't reported anything to me, as it was a real bug.
Yeah. Not you. This is the third or fourth time it has happened this morning so I was just curiuos what was going on.
regards,
dan carpenter
RFC PATCH 3/12 vdpa_sim: remove hard-coded virtq count
In reply to Stefano GarzarellaFrom: Max Gurtovoy <[email protected]>
Add a new attribute that will define the number of virt queues to be created for the vdpasim device.
---
v1:
- use kcalloc() instead of kmalloc_array() since some function expects variables initialized to zero
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 5 +++-- drivers/vdpa/vdpa_sim/vdpa_sim.c | 14 +++++++++++--- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 3 +++ 3 files changed, 17 insertions(+), 5 deletions(-)
drivers/vdpa/vdpa_sim/vdpa_sim.h +3 -2
@@ -20,7 +20,6 @@#define VDPASIM_QUEUE_ALIGN PAGE_SIZE#define VDPASIM_QUEUE_MAX 256#define VDPASIM_VENDOR_ID 0 -#define VDPASIM_VQ_NUM 0x2#define VDPASIM_FEATURES ((1ULL << VIRTIO_F_ANY_LAYOUT) | \ (1ULL << VIRTIO_F_VERSION_1) | \
@@ -46,12 +45,13 @@ struct vdpasim_init_attr {u64 features;work_func_t work_fn;int batch_mapping; +int nvqs; };/* State of each vdpasim device */ struct vdpasim {struct vdpa_device vdpa; -struct vdpasim_virtqueue vqs[VDPASIM_VQ_NUM ]; +struct vdpasim_virtqueue *vqs;struct work_struct work;/* spinlock to synchronize virtqueue state */ spinlock_t lock;
@@ -64,6 +64,7 @@ struct vdpasim {u32 generation;u64 features;u64 supported_features; +int nvqs;/* spinlock to synchronize iommu table */ spinlock_t iommu_lock; };
drivers/vdpa/vdpa_sim/vdpa_sim.c +11 -3
@@ -54,7 +54,7 @@ static void vdpasim_reset(struct vdpasim *vdpasim) {int i; -for (i =0 ; i <VDPASIM_VQ_NUM ; i++) +for (i =0 ; i < vdpasim->nvqs; i++)vdpasim_vq_reset (vdpasim, &vdpasim->vqs[i]);spin_lock (&vdpasim->iommu_lock);
@@ -199,7 +199,8 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)else ops = &vdpasim_config_ops; - vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, VDPASIM_VQ_NUM); + vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, + attr->nvqs);if (!vdpasim) goto err_alloc;
@@ -211,8 +212,14 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->config) goto err_iommu; + vdpasim->vqs =kcalloc (attr->nvqs,sizeof (struct vdpasim_virtqueue ), +GFP_KERNEL ); +if (!vdpasim->vqs) + goto err_iommu; + vdpasim->device_id = device_id; vdpasim->supported_features = attr->features; + vdpasim->nvqs = attr->nvqs;INIT_WORK (&vdpasim->work, attr->work_fn);spin_lock_init (&vdpasim->lock);spin_lock_init (&vdpasim->iommu_lock);
@@ -231,7 +238,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->buffer) goto err_iommu; -for (i =0 ; i <VDPASIM_VQ_NUM ; i++) +for (i =0 ; i < vdpasim->nvqs; i++)vringh_set_iotlb (&vdpasim->vqs[i].vring, vdpasim->iommu); vdpasim->vdpa.dma_dev = dev;
@@ -511,6 +518,7 @@ static void vdpasim_free(struct vdpa_device *vdpa)kfree (vdpasim->buffer);if (vdpasim->iommu)vhost_iotlb_free (vdpasim->iommu); +kfree (vdpasim->vqs);kfree (vdpasim->config); }
drivers/vdpa/vdpa_sim/vdpa_sim_net.c +3 -0
@@ -14,6 +14,8 @@#define VDPASIM_NET_FEATURES (1ULL << VIRTIO_NET_F_MAC) +#define VDPASIM_NET_VQ_NUM 2 +static int batch_mapping =1 ;module_param (batch_mapping, int,0444 );MODULE_PARM_DESC (batch_mapping,"Batched mapping 1 - Enable; 0 - Disable" );
@@ -105,6 +107,7 @@ static int __init vdpasim_net_init(void) attr.device_id =VIRTIO_ID_NET ; attr.features =VDPASIM_FEATURES |VDPASIM_NET_FEATURES ; + attr.nvqs =VDPASIM_NET_VQ_NUM ; attr.work_fn = vdpasim_net_work; attr.batch_mapping = batch_mapping; vdpasim_net_dev =vdpasim_create (&attr);
Signature · 1 line
2.26.2
Re: [PATCH RFC 03/12] vdpa_sim: remove hard-coded virtq count
Quoted 5 lines · Stefano Garzarella
From: Max Gurtovoy <[email protected]>
Add a new attribute that will define the number of virt queues to be created for the vdpasim device.
Quoted 7 lines
Signed-off-by Max Gurtovoy <[email protected]>[sgarzare: replace kmalloc_array() with kcalloc()]Signed-off-by Stefano Garzarella <[email protected]>---
v1:
- usekcalloc()instead ofkmalloc_array()since some function expects variables initialized to zero
Looks good, one nit, I prefer to do this before patch 2.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RFC PATCH 4/12 vdpa: add vdpa simulator for block device
In reply to Stefano GarzarellaFrom: Max Gurtovoy <[email protected]>
This will allow running vDPA for virtio block protocol.
---
v1:
- Removed unused headers
- Used cpu_to_vdpasim*() to store config fields
- Replaced 'select VDPA_SIM' with 'depends on VDPA_SIM' since selected option can not depend on other [Jason]
- Start with a single queue for now [Jason]
- Add comments to memory barriers
--- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 124 +++++++++++++++++++++++++++ drivers/vdpa/Kconfig | 9 ++ drivers/vdpa/vdpa_sim/Makefile | 1 + 3 files changed, 134 insertions(+) create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +124 -0
@@ -0,0 +1,124 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* VDPA simulator for block device. +* +* Copyright (c) 2020, Mellanox Technologies. All rights reserved. +* +*/ + +#include <linux/module.h> + +#include "vdpa_sim.h" + +#define VDPASIM_BLK_FEATURES ((1ULL << VIRTIO_BLK_F_SIZE_MAX) | \ + (1ULL << VIRTIO_BLK_F_SEG_MAX) | \ + (1ULL << VIRTIO_BLK_F_BLK_SIZE) | \ + (1ULL << VIRTIO_BLK_F_TOPOLOGY) | \ + (1ULL << VIRTIO_BLK_F_MQ)) + +#define VDPASIM_BLK_CAPACITY 0x40000 +#define VDPASIM_BLK_SIZE_MAX 0x1000 +#define VDPASIM_BLK_SEG_MAX 32 +#define VDPASIM_BLK_VQ_NUM 1 + +static struct vdpasim *vdpasim_blk_dev; + +static void vdpasim_blk_work (struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); +u8 status =VIRTIO_BLK_S_OK ; +int i; + +spin_lock (&vdpasim->lock); + +if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) + goto out; + +for (i =0 ; i <VDPASIM_BLK_VQ_NUM ; i++) { +struct vdpasim_virtqueue *vq = &vdpasim->vqs[i]; + +if (!vq->ready) +continue ; + +while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, + &vq->head,GFP_ATOMIC ) >0 ) { + +int write; + + vq->iov.i = vq->iov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); +if (write <=0 ) +break ; + +/* Make sure data is wrote before advancing index */ +smp_wmb (); + +vringh_complete_iotlb (&vq->vring, vq->head, write); + +/* Make sure used is visible before rasing the interrupt. */ +smp_wmb (); + +if (vringh_need_notify_iotlb (&vq->vring) >0 ) +vringh_notify (&vq->vring); + +local_bh_disable (); +if (vq->cb) + vq->cb (vq->private); +local_bh_enable (); + } + } +out: +spin_unlock (&vdpasim->lock); + +} + +static int __init vdpasim_blk_init (void ) +{ +struct vdpasim_init_attr attr = {}; +struct virtio_blk_config *config; +int ret; + + attr.device_id =VIRTIO_ID_BLOCK ; + attr.features =VDPASIM_FEATURES |VDPASIM_BLK_FEATURES ; + attr.work_fn = vdpasim_blk_work; + vdpasim_blk_dev =vdpasim_create (&attr); +if (IS_ERR (vdpasim_blk_dev)) { + ret =PTR_ERR (vdpasim_blk_dev); + goto out; + } + + config = (struct virtio_blk_config *)vdpasim_blk_dev->config; + config->capacity =cpu_to_vdpasim64 (vdpasim_blk_dev,VDPASIM_BLK_CAPACITY ); + config->size_max =cpu_to_vdpasim32 (vdpasim_blk_dev,VDPASIM_BLK_SIZE_MAX ); + config->seg_max =cpu_to_vdpasim32 (vdpasim_blk_dev,VDPASIM_BLK_SEG_MAX ); + config->num_queues =cpu_to_vdpasim16 (vdpasim_blk_dev,VDPASIM_BLK_VQ_NUM ); + config->min_io_size =cpu_to_vdpasim16 (vdpasim_blk_dev,1 ); + config->opt_io_size =cpu_to_vdpasim32 (vdpasim_blk_dev,1 ); + config->blk_size =cpu_to_vdpasim32 (vdpasim_blk_dev,512 ); + + ret =vdpa_register_device (&vdpasim_blk_dev->vdpa); +if (ret) + goto put_dev; + +return 0 ; + +put_dev: +put_device (&vdpasim_blk_dev->vdpa.dev); +out: +return ret; +} + +static void __exit vdpasim_blk_exit(void) +{ +struct vdpa_device *vdpa = &vdpasim_blk_dev->vdpa; + +vdpa_unregister_device (vdpa); +} + +module_init(vdpasim_blk_init) +module_exit(vdpasim_blk_exit) + +MODULE_VERSION(DRV_VERSION); +MODULE_LICENSE(DRV_LICENSE); +MODULE_AUTHOR("Max Gurtovoy <[email protected]>"); +MODULE_DESCRIPTION ("vDPA Device Simulator for block device" );
drivers/vdpa/Kconfig +9 -0
@@ -24,6 +24,15 @@ config VDPA_SIM_NET to RX. This device is used for testing, prototyping and development of vDPA. +config VDPA_SIM_BLOCK + tristate "vDPA simulator for block device" + depends on VDPA_SIM + default n + help + vDPA block device simulator which terminates IO request in a + memory buffer. This device is used for testing, prototyping and + development of vDPA. + config IFCVF tristate "Intel IFC VF vDPA driver" depends on PCI_MSI
drivers/vdpa/vdpa_sim/Makefile +1 -0
@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_VDPA_SIM) += vdpa_sim.o obj-$(CONFIG_VDPA_SIM_NET) += vdpa_sim_net.o +obj-$(CONFIG_VDPA_SIM_BLOCK) += vdpa_sim_blk.o
Signature · 1 line
2.26.2
Re: [PATCH RFC 04/12] vdpa: add vdpa simulator for block device
Quoted 80 lines · Stefano Garzarella
From: Max Gurtovoy <[email protected]>
This will allow running vDPA for virtio block protocol.
Signed-off-by Max Gurtovoy <[email protected]>[sgarzare: various cleanups/fixes]Signed-off-by Stefano Garzarella <[email protected]>---
v1:
- Removed unused headers
- Used cpu_to_vdpasim*() to store config fields
- Replaced 'selectVDPA_SIM' with 'depends onVDPA_SIM' since selected option can not depend on other [Jason]
- Start with a single queue for now [Jason]
- Add comments to memory barriers--- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 124 +++++++++++++++++++++++++++ drivers/vdpa/Kconfig | 9 ++ drivers/vdpa/vdpa_sim/Makefile | 1 + 3 files changed, 134 insertions(+)create mode 100644
drivers/vdpa/vdpa_sim/vdpa_sim_blk.cdrivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +52
-0@@ -0,0 +1,124 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* VDPA simulator for block device. +* +* Copyright (c) 2020, Mellanox Technologies. All rights reserved. +* +*/ + +#include <linux/module.h> + +#include "vdpa_sim.h" + +#define VDPASIM_BLK_FEATURES ((1ULL << VIRTIO_BLK_F_SIZE_MAX) | \ + (1ULL << VIRTIO_BLK_F_SEG_MAX) | \ + (1ULL << VIRTIO_BLK_F_BLK_SIZE) | \ + (1ULL << VIRTIO_BLK_F_TOPOLOGY) | \ + (1ULL << VIRTIO_BLK_F_MQ)) + +#define VDPASIM_BLK_CAPACITY 0x40000 +#define VDPASIM_BLK_SIZE_MAX 0x1000 +#define VDPASIM_BLK_SEG_MAX 32 +#define VDPASIM_BLK_VQ_NUM 1 + +static struct vdpasim *vdpasim_blk_dev; + +static void vdpasim_blk_work (struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); +u8 status =VIRTIO_BLK_S_OK ; +int i; + +spin_lock (&vdpasim->lock); + +if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) + goto out; + +for (i =0 ; i <VDPASIM_BLK_VQ_NUM ; i++) { +struct vdpasim_virtqueue *vq = &vdpasim->vqs[i]; + +if (!vq->ready) +continue ; + +while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, + &vq->head,GFP_ATOMIC ) >0 ) { + +int write; + + vq->iov.i = vq->iov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); +if (write <=0 ) +break ; +
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 62
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +10
-0+/* Make sure data is wrote before advancing index */ +smp_wmb (); + +vringh_complete_iotlb (&vq->vring, vq->head, write); + +/* Make sure used is visible before rasing the interrupt. */ +smp_wmb (); + +if (vringh_need_notify_iotlb (&vq->vring) >0 ) +vringh_notify (&vq->vring);
Do we initialize vrh->notify anywhere? And This seems duplicated with the following vq->cb.
I think the correct way is to initialize vrh->notify and use vringh_need_notify_iotlb()/vringh_notify() instead of the vq->cb here.
And while at it, it's better to convert net simulator to do the same.
Thanks
Quoted 91 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +62
-0+ +local_bh_disable (); +if (vq->cb) + vq->cb (vq->private); +local_bh_enable (); + } + } +out: +spin_unlock (&vdpasim->lock); + +} + +static int __init vdpasim_blk_init (void ) +{ +struct vdpasim_init_attr attr = {}; +struct virtio_blk_config *config; +int ret; + + attr.device_id =VIRTIO_ID_BLOCK ; + attr.features =VDPASIM_FEATURES |VDPASIM_BLK_FEATURES ; + attr.work_fn = vdpasim_blk_work; + vdpasim_blk_dev =vdpasim_create (&attr); +if (IS_ERR (vdpasim_blk_dev)) { + ret =PTR_ERR (vdpasim_blk_dev); + goto out; + } + + config = (struct virtio_blk_config *)vdpasim_blk_dev->config; + config->capacity =cpu_to_vdpasim64 (vdpasim_blk_dev,VDPASIM_BLK_CAPACITY ); + config->size_max =cpu_to_vdpasim32 (vdpasim_blk_dev,VDPASIM_BLK_SIZE_MAX ); + config->seg_max =cpu_to_vdpasim32 (vdpasim_blk_dev,VDPASIM_BLK_SEG_MAX ); + config->num_queues =cpu_to_vdpasim16 (vdpasim_blk_dev,VDPASIM_BLK_VQ_NUM ); + config->min_io_size =cpu_to_vdpasim16 (vdpasim_blk_dev,1 ); + config->opt_io_size =cpu_to_vdpasim32 (vdpasim_blk_dev,1 ); + config->blk_size =cpu_to_vdpasim32 (vdpasim_blk_dev,512 ); + + ret =vdpa_register_device (&vdpasim_blk_dev->vdpa); +if (ret) + goto put_dev; + +return 0 ; + +put_dev: +put_device (&vdpasim_blk_dev->vdpa.dev); +out: +return ret; +} + +static void __exit vdpasim_blk_exit(void) +{ +struct vdpa_device *vdpa = &vdpasim_blk_dev->vdpa; + +vdpa_unregister_device (vdpa); +} + +module_init(vdpasim_blk_init) +module_exit(vdpasim_blk_exit) + +MODULE_VERSION(DRV_VERSION); +MODULE_LICENSE(DRV_LICENSE); +MODULE_AUTHOR("Max Gurtovoy <[email protected]>"); +MODULE_DESCRIPTION ("vDPA Device Simulator for block device" );drivers/vdpa/Kconfig +9
-0@@ -24,6 +24,15 @@ config VDPA_SIM_NET to RX. This device is used for testing, prototyping and development of vDPA. +config VDPA_SIM_BLOCK + tristate "vDPA simulator for block device" + depends on VDPA_SIM + default n + help + vDPA block device simulator which terminates IO request in a + memory buffer. This device is used for testing, prototyping and + development of vDPA. + config IFCVF tristate "Intel IFC VF vDPA driver" depends on PCI_MSIdrivers/vdpa/vdpa_sim/Makefile +1
-0@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_VDPA_SIM) += vdpa_sim.o obj-$(CONFIG_VDPA_SIM_NET) += vdpa_sim_net.o +obj-$(CONFIG_VDPA_SIM_BLOCK) += vdpa_sim_blk.o
Re: [PATCH RFC 04/12] vdpa: add vdpa simulator for block device
Quoted 96 lines · Jason Wang
Quoted 80 lines · Stefano Garzarella
From: Max Gurtovoy <[email protected]>
This will allow running vDPA for virtio block protocol.
Signed-off-by Max Gurtovoy <[email protected]>[sgarzare: various cleanups/fixes]Signed-off-by Stefano Garzarella <[email protected]>---
v1:
- Removed unused headers
- Used cpu_to_vdpasim*() to store config fields
- Replaced 'selectVDPA_SIM' with 'depends onVDPA_SIM' since selected option can not depend on other [Jason]
- Start with a single queue for now [Jason]
- Add comments to memory barriers--- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 124 +++++++++++++++++++++++++++ drivers/vdpa/Kconfig | 9 ++ drivers/vdpa/vdpa_sim/Makefile | 1 + 3 files changed, 134 insertions(+) create mode 100644 drivers/vdpa/vdpa_sim/vdpa_sim_blk.cdrivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +52
-0@@ -0,0 +1,124 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* VDPA simulator for block device. +* +* Copyright (c) 2020, Mellanox Technologies. All rights reserved. +* +*/ + +#include <linux/module.h> + +#include "vdpa_sim.h" + +#define VDPASIM_BLK_FEATURES ((1ULL << VIRTIO_BLK_F_SIZE_MAX) | \ + (1ULL << VIRTIO_BLK_F_SEG_MAX) | \ + (1ULL << VIRTIO_BLK_F_BLK_SIZE) | \ + (1ULL << VIRTIO_BLK_F_TOPOLOGY) | \ + (1ULL << VIRTIO_BLK_F_MQ)) + +#define VDPASIM_BLK_CAPACITY 0x40000 +#define VDPASIM_BLK_SIZE_MAX 0x1000 +#define VDPASIM_BLK_SEG_MAX 32 +#define VDPASIM_BLK_VQ_NUM 1 + +static struct vdpasim *vdpasim_blk_dev; + +static void vdpasim_blk_work (struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); +u8 status =VIRTIO_BLK_S_OK ; +int i; + +spin_lock (&vdpasim->lock); + +if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) + goto out; + +for (i =0 ; i <VDPASIM_BLK_VQ_NUM ; i++) { +struct vdpasim_virtqueue *vq = &vdpasim->vqs[i]; + +if (!vq->ready) +continue ; + +while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, + &vq->head,GFP_ATOMIC ) >0 ) { + +int write; + + vq->iov.i = vq->iov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); +if (write <=0 ) +break ; +drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 62
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +10
-0+/* Make sure data is wrote before advancing index */ +smp_wmb (); + +vringh_complete_iotlb (&vq->vring, vq->head, write); + +/* Make sure used is visible before rasing the interrupt. */ +smp_wmb (); + +if (vringh_need_notify_iotlb (&vq->vring) >0 ) +vringh_notify (&vq->vring);Do we initialize
vrh->notifyanywhere? And This seems duplicated with the followingvq->cb.
Quoted 1 line
I think the correct way is to initialize
vrh->notifyand use vringh_need_notify_iotlb()/vringh_notify() instead of thevq->cbhere.
Okay, so I'll set vrh->notify in the vdpasim core with a function that calls vq->cb() (the callback set through .set_vq_cb).
Quoted 2 lines
And while at it, it's better to convert net simulator to do the same.
Sure.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 04/12] vdpa: add vdpa simulator for block device
In reply to Stefano GarzarellaQuoted 16 lines · Stefano Garzarella
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +16
-0+static void vdpasim_blk_work (struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); +u8 status =VIRTIO_BLK_S_OK ; +int i; + +spin_lock (&vdpasim->lock); + +if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) + goto out; + +for (i =0 ; i <VDPASIM_BLK_VQ_NUM ; i++) { +struct vdpasim_virtqueue *vq = &vdpasim->vqs[i]; + +if (!vq->ready) +continue ;
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 51
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +10
-0+ +while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, + &vq->head,GFP_ATOMIC ) >0 ) { + +int write; + + vq->iov.i = vq->iov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); +if (write <=0 ) +break ;
We're lucky the guest driver doesn't crash after VIRTIO_BLK_T_GET_ID? :)
Re: [PATCH RFC 04/12] vdpa: add vdpa simulator for block device
Quoted 21 lines · Stefan Hajnoczi
Quoted 20 lines · Stefano Garzarella
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +20
-0+static void vdpasim_blk_work (struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); +u8 status =VIRTIO_BLK_S_OK ; +int i; + +spin_lock (&vdpasim->lock); + +if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) + goto out; + +for (i =0 ; i <VDPASIM_BLK_VQ_NUM ; i++) { +struct vdpasim_virtqueue *vq = &vdpasim->vqs[i]; + +if (!vq->ready) +continue ; + +while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, + &vq->head,GFP_ATOMIC ) >0 ) { +
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 51
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +6
-0+int write; + + vq->iov.i = vq->iov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); +if (write <=0 ) +break ;We're lucky the guest driver doesn't crash after
VIRTIO_BLK_T_GET_ID? :)
The crash could happen if the simulator doesn't put the string terminator, but in virtio_blk.c, the serial_show() initialize the buffer putting the string terminator in the VIRTIO_BLK_ID_BYTES element:
buf[VIRTIO_BLK_ID_BYTES] = '\0';
err = virtblk_get_id(disk, buf);This should prevent the issue, right?
However in the last patch of this series I implemented VIRTIO_BLK_T_GET_ID support :-)
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 04/12] vdpa: add vdpa simulator for block device
Quoted 35 lines · Stefano Garzarella
Quoted 21 lines · Stefan Hajnoczi
Quoted 20 lines · Stefano Garzarella
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +20
-0+static void vdpasim_blk_work (struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); +u8 status =VIRTIO_BLK_S_OK ; +int i; + +spin_lock (&vdpasim->lock); + +if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) + goto out; + +for (i =0 ; i <VDPASIM_BLK_VQ_NUM ; i++) { +struct vdpasim_virtqueue *vq = &vdpasim->vqs[i]; + +if (!vq->ready) +continue ; + +while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, + &vq->head,GFP_ATOMIC ) >0 ) { +Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 51
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +6
-0+int write; + + vq->iov.i = vq->iov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); +if (write <=0 ) +break ;We're lucky the guest driver doesn't crash after
VIRTIO_BLK_T_GET_ID? :)The crash could happen if the simulator doesn't put the string terminator, but in
virtio_blk.c, theserial_show()initialize the buffer putting the string terminator in theVIRTIO_BLK_ID_BYTESelement:
Quoted 7 lines
buf[VIRTIO_BLK_ID_BYTES] = '\0'; err = virtblk_get_id(disk, buf);This should prevent the issue, right?
However in the last patch of this series I implemented
VIRTIO_BLK_T_GET_IDsupport :-)
Windows, BSD, macOS, etc guest drivers aren't necessarily going to terminate or initialize the serial string buffer.
Anyway, the later patch that implements VIRTIO_BLK_T_GET_ID solves this issue! Thanks.
Stefan
Re: [PATCH RFC 04/12] vdpa: add vdpa simulator for block device
Quoted 39 lines · Stefan Hajnoczi
Quoted 38 lines · Stefano Garzarella
Quoted 21 lines · Stefan Hajnoczi
Quoted 20 lines · Stefano Garzarella
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +20
-0+static void vdpasim_blk_work (struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); +u8 status =VIRTIO_BLK_S_OK ; +int i; + +spin_lock (&vdpasim->lock); + +if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) + goto out; + +for (i =0 ; i <VDPASIM_BLK_VQ_NUM ; i++) { +struct vdpasim_virtqueue *vq = &vdpasim->vqs[i]; + +if (!vq->ready) +continue ; + +while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, + &vq->head,GFP_ATOMIC ) >0 ) { +Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 51
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +6
-0+int write; + + vq->iov.i = vq->iov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); +if (write <=0 ) +break ;We're lucky the guest driver doesn't crash after
VIRTIO_BLK_T_GET_ID? :)The crash could happen if the simulator doesn't put the string terminator, but in
virtio_blk.c, theserial_show()initialize the buffer putting the string terminator in theVIRTIO_BLK_ID_BYTESelement:buf[VIRTIO_BLK_ID_BYTES] = '\0'; err = virtblk_get_id(disk, buf);
Quoted 7 lines
Quoted 4 lines
This should prevent the issue, right?
However in the last patch of this series I implemented
VIRTIO_BLK_T_GET_IDsupport :-)Windows, BSD, macOS, etc guest drivers aren't necessarily going to terminate or initialize the serial string buffer.
Unfortunately I discovered that VIRTIO_BLK_T_GET_ID is not in the VIRTIO specs, so, just for curiosity, I checked the QEMU code and I found this:
case VIRTIO_BLK_T_GET_ID:
{
/*
* NB: per existing s/n string convention the string is
* terminated by '\0' only when shorter than buffer.
*/
const char *serial = s->conf.serial ? s->conf.serial : "";
size_t size = MIN(strlen(serial) + 1,
MIN(iov_size(in_iov, in_num),
VIRTIO_BLK_ID_BYTES));
iov_from_buf(in_iov, in_num, 0, serial, size);
virtio_blk_req_complete(req, VIRTIO_BLK_S_OK);
virtio_blk_free_request(req);
break;
}It seems that the device emulation in QEMU expects that the driver terminates the serial string buffer.
Do you know why VIRTIO_BLK_T_GET_ID is not in the specs? Should we add it?
Thanks,
Stefano
Quoted 6 lines
Anyway, the later patch that implements
VIRTIO_BLK_T_GET_IDsolves this issue! Thanks.Stefan
Re: [PATCH RFC 04/12] vdpa: add vdpa simulator for block device
Quoted 67 lines · Stefano Garzarella
Quoted 39 lines · Stefan Hajnoczi
Quoted 38 lines · Stefano Garzarella
Quoted 21 lines · Stefan Hajnoczi
Quoted 20 lines · Stefano Garzarella
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +20
-0+static void vdpasim_blk_work (struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); +u8 status =VIRTIO_BLK_S_OK ; +int i; + +spin_lock (&vdpasim->lock); + +if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) + goto out; + +for (i =0 ; i <VDPASIM_BLK_VQ_NUM ; i++) { +struct vdpasim_virtqueue *vq = &vdpasim->vqs[i]; + +if (!vq->ready) +continue ; + +while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, + &vq->head,GFP_ATOMIC ) >0 ) { +Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 51
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +6
-0+int write; + + vq->iov.i = vq->iov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); +if (write <=0 ) +break ;We're lucky the guest driver doesn't crash after
VIRTIO_BLK_T_GET_ID? :)The crash could happen if the simulator doesn't put the string terminator, but in
virtio_blk.c, theserial_show()initialize the buffer putting the string terminator in theVIRTIO_BLK_ID_BYTESelement:buf[VIRTIO_BLK_ID_BYTES] = '\0'; err = virtblk_get_id(disk, buf);Quoted 7 lines
Quoted 4 lines
This should prevent the issue, right?
However in the last patch of this series I implemented
VIRTIO_BLK_T_GET_IDsupport :-)Windows, BSD, macOS, etc guest drivers aren't necessarily going to terminate or initialize the serial string buffer.
Unfortunately I discovered that
VIRTIO_BLK_T_GET_IDis not in the VIRTIO specs, so, just for curiosity, I checked the QEMU code and I found this:case VIRTIO_BLK_T_GET_ID: { /* * NB: per existing s/n string convention the string is * terminated by '\0' only when shorter than buffer. */ const char *serial = s->conf.serial ? s->conf.serial : ""; size_t size = MIN(strlen(serial) + 1, MIN(iov_size(in_iov, in_num), VIRTIO_BLK_ID_BYTES)); iov_from_buf(in_iov, in_num, 0, serial, size); virtio_blk_req_complete(req, VIRTIO_BLK_S_OK); virtio_blk_free_request(req); break; }
Quoted 5 lines
It seems that the device emulation in QEMU expects that the driver terminates the serial string buffer.
Do you know why
VIRTIO_BLK_T_GET_IDis not in the specs? Should we add it?
It's about to be merged into the VIRTIO spec:
https://github.com/oasis-tcs/virtio-spec/issues/63
Stefan
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 04/12] vdpa: add vdpa simulator for block device
Quoted 68 lines · Stefan Hajnoczi
Quoted 67 lines · Stefano Garzarella
Quoted 39 lines · Stefan Hajnoczi
Quoted 38 lines · Stefano Garzarella
Quoted 21 lines · Stefan Hajnoczi
Quoted 20 lines · Stefano Garzarella
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +20
-0+static void vdpasim_blk_work (struct work_struct *work) +{ + struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); +u8 status =VIRTIO_BLK_S_OK ; +int i; + +spin_lock (&vdpasim->lock); + +if (!(vdpasim->status &VIRTIO_CONFIG_S_DRIVER_OK )) + goto out; + +for (i =0 ; i <VDPASIM_BLK_VQ_NUM ; i++) { +struct vdpasim_virtqueue *vq = &vdpasim->vqs[i]; + +if (!vq->ready) +continue ; + +while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, + &vq->head,GFP_ATOMIC ) >0 ) { +Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 51
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c new file +6
-0+int write; + + vq->iov.i = vq->iov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); +if (write <=0 ) +break ;We're lucky the guest driver doesn't crash after
VIRTIO_BLK_T_GET_ID? :)The crash could happen if the simulator doesn't put the string terminator, but in
virtio_blk.c, theserial_show()initialize the buffer putting the string terminator in theVIRTIO_BLK_ID_BYTESelement:buf[VIRTIO_BLK_ID_BYTES] = '\0'; err = virtblk_get_id(disk, buf);Quoted 7 lines
Quoted 4 lines
This should prevent the issue, right?
However in the last patch of this series I implemented
VIRTIO_BLK_T_GET_IDsupport :-)Windows, BSD, macOS, etc guest drivers aren't necessarily going to terminate or initialize the serial string buffer.
Unfortunately I discovered that
VIRTIO_BLK_T_GET_IDis not in the VIRTIO specs, so, just for curiosity, I checked the QEMU code and I found this:case VIRTIO_BLK_T_GET_ID: { /* * NB: per existing s/n string convention the string is * terminated by '\0' only when shorter than buffer. */ const char *serial = s->conf.serial ? s->conf.serial : ""; size_t size = MIN(strlen(serial) + 1, MIN(iov_size(in_iov, in_num), VIRTIO_BLK_ID_BYTES)); iov_from_buf(in_iov, in_num, 0, serial, size); virtio_blk_req_complete(req, VIRTIO_BLK_S_OK); virtio_blk_free_request(req); break; }
Quoted 9 lines
Quoted 5 lines
It seems that the device emulation in QEMU expects that the driver terminates the serial string buffer.
Do you know why
VIRTIO_BLK_T_GET_IDis not in the specs? Should we add it?It's about to be merged into the VIRTIO spec:
https://github.com/oasis-tcs/virtio-spec/issues/63
Great! Thanks for the link!
Stefano
RFC PATCH 5/12 vdpa_sim: remove the limit of IOTLB entries
In reply to Stefano GarzarellaThe simulated devices can support multiple queues, so this limit should be defined according to the number of queues supported by the device.
Since we are in a simulator, let's simply remove that limit.
--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/vdpa/vdpa_sim/vdpa_sim.c +1 -1
@@ -230,7 +230,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr) goto err_iommu;set_dma_ops (dev, &vdpasim_dma_ops); - vdpasim->iommu =vhost_iotlb_alloc (2048 ,0 ); + vdpasim->iommu =vhost_iotlb_alloc (0 ,0 );if (!vdpasim->iommu) goto err_iommu;
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 05/12] vdpa_sim: remove the limit of IOTLB entries
Quoted 8 lines · Stefano Garzarella
The simulated devices can support multiple queues, so this limit should be defined according to the number of queues supported by the device.
Since we are in a simulator, let's simply remove that limit.
Suggested-by Jason Wang <[email protected]>Signed-off-by Stefano Garzarella <[email protected]>
It would be good to introduce a macro instead of using the magic 0 here.
Thanks
Quoted 9 lines
--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/vdpa/vdpa_sim/vdpa_sim.c +0
-0
drivers/vdpa/vdpa_sim/vdpa_sim.c struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr) · line 236
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-1@@ -230,7 +230,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr) goto err_iommu;set_dma_ops (dev, &vdpasim_dma_ops); - vdpasim->iommu =vhost_iotlb_alloc (2048 ,0 ); + vdpasim->iommu =vhost_iotlb_alloc (0 ,0 );if (!vdpasim->iommu) goto err_iommu;
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 05/12] vdpa_sim: remove the limit of IOTLB entries
Quoted 8 lines · Jason Wang
Quoted 6 lines · Stefano Garzarella
The simulated devices can support multiple queues, so this limit should be defined according to the number of queues supported by the device.
Since we are in a simulator, let's simply remove that limit.
Quoted 7 lines
Quoted 2 lines
Suggested-by Jason Wang <[email protected]>Signed-off-by Stefano Garzarella <[email protected]>Acked-by Jason Wang <[email protected]>It would be good to introduce a macro instead of using the magic 0 here.
Done.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RFC PATCH 6/12 vdpa_sim: add struct vdpasim_device to store device properties
In reply to Stefano GarzarellaMove device properties used during the entire life cycle in a new structure to simplify the copy of these fields during the vdpasim initialization.
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 17 ++++++++------ drivers/vdpa/vdpa_sim/vdpa_sim.c | 33 ++++++++++++++-------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 8 +++++-- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 9 +++++--- 4 files changed, 38 insertions(+), 29 deletions(-)
drivers/vdpa/vdpa_sim/vdpa_sim.h +10 -7
@@ -40,12 +40,17 @@ struct vdpasim_virtqueue {irqreturn_t (*cb)(void *data); }; +struct vdpasim_device { +u64 supported_features; +u32 id; +int nvqs; +}; +struct vdpasim_init_attr { -u32 device_id; -u64 features; +struct vdpasim_device device; +int batch_mapping; +work_func_t work_fn; -int batch_mapping; -int nvqs; };/* State of each vdpasim device */
@@ -53,18 +58,16 @@ struct vdpasim {struct vdpa_device vdpa;struct vdpasim_virtqueue *vqs;struct work_struct work; +struct vdpasim_device device;/* spinlock to synchronize virtqueue state */ spinlock_t lock;/* virtio config according to device type */ void *config;struct vhost_iotlb *iommu;void *buffer; -u32 device_id;u32 status;u32 generation;u64 features; -u64 supported_features; -int nvqs;/* spinlock to synchronize iommu table */ spinlock_t iommu_lock; };
drivers/vdpa/vdpa_sim/vdpa_sim.c +16 -17
@@ -28,7 +28,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) {struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; - vringh_init_iotlb(&vq->vring, vdpasim->supported_features, + vringh_init_iotlb(&vq->vring, vdpasim->device.supported_features, VDPASIM_QUEUE_MAX, false, (struct vring_desc *)(uintptr_t)vq->desc_addr, (struct vring_avail *)
@@ -46,7 +46,7 @@ static void vdpasim_vq_reset(struct vdpasim *vdpasim, vq->device_addr =0 ; vq->cb =NULL ; vq->private =NULL ; -vringh_init_iotlb (&vq->vring, vdpasim->supported_features, +vringh_init_iotlb (&vq->vring, vdpasim->device.supported_features,VDPASIM_QUEUE_MAX , false,NULL ,NULL ,NULL ); }
@@ -54,7 +54,7 @@ static void vdpasim_reset(struct vdpasim *vdpasim) {int i; -for (i =0 ; i < vdpasim->nvqs; i++) +for (i =0 ; i < vdpasim->device.nvqs; i++)vdpasim_vq_reset (vdpasim, &vdpasim->vqs[i]);spin_lock (&vdpasim->iommu_lock);
@@ -189,7 +189,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)struct device *dev;int i, size, ret = -ENOMEM ; - device_id = attr->device_id; + device_id = attr->device.id;/* Currently, we only accept the network and block devices. */ if (device_id !=VIRTIO_ID_NET && device_id !=VIRTIO_ID_BLOCK )return ERR_PTR (-EOPNOTSUPP );
@@ -200,10 +200,12 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr) ops = &vdpasim_config_ops; vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, - attr->nvqs); + attr->device.nvqs);if (!vdpasim) goto err_alloc; + vdpasim->device = attr->device; +if (device_id ==VIRTIO_ID_NET ) size =sizeof (struct virtio_net_config );else
@@ -212,14 +214,11 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->config) goto err_iommu; - vdpasim->vqs =kcalloc (attr->nvqs,sizeof (struct vdpasim_virtqueue ), -GFP_KERNEL ); + vdpasim->vqs =kcalloc (vdpasim->device.nvqs, +sizeof (struct vdpasim_virtqueue ),GFP_KERNEL );if (!vdpasim->vqs) goto err_iommu; - vdpasim->device_id = device_id; - vdpasim->supported_features = attr->features; - vdpasim->nvqs = attr->nvqs;INIT_WORK (&vdpasim->work, attr->work_fn);spin_lock_init (&vdpasim->lock);spin_lock_init (&vdpasim->iommu_lock);
@@ -238,7 +237,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->buffer) goto err_iommu; -for (i =0 ; i < vdpasim->nvqs; i++) +for (i =0 ; i < vdpasim->device.nvqs; i++)vringh_set_iotlb (&vdpasim->vqs[i].vring, vdpasim->iommu); vdpasim->vdpa.dma_dev = dev;
@@ -347,7 +346,7 @@ static u64 vdpasim_get_features(struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -return vdpasim->supported_features; +return vdpasim->device.supported_features; }static int vdpasim_set_features (struct vdpa_device *vdpa,u64 features)
@@ -358,14 +357,14 @@ static int vdpasim_set_features(struct vdpa_device *vdpa, u64 features)if (!(features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM )))return -EINVAL ; - vdpasim->features = features & vdpasim->supported_features; + vdpasim->features = features & vdpasim->device.supported_features;/* We generally only know whether guest is using the legacy interface * here, so generally that's the earliest we can set config fields. * Note: We actually require VIRTIO_F_ACCESS_PLATFORM above which * implies VIRTIO_F_VERSION_1, but let's not try to be clever here. */ -if (vdpasim->device_id ==VIRTIO_ID_NET ) { +if (vdpasim->device.id ==VIRTIO_ID_NET ) {struct virtio_net_config *config = (struct virtio_net_config *)vdpasim->config;
@@ -391,7 +390,7 @@ static u32 vdpasim_get_device_id(struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -return vdpasim->device_id; +return vdpasim->device.id; }static u32 vdpasim_get_vendor_id (struct vdpa_device *vdpa)
@@ -427,10 +426,10 @@ static void vdpasim_get_config(struct vdpa_device *vdpa, unsigned int offset, {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -if (vdpasim->device_id ==VIRTIO_ID_BLOCK && +if (vdpasim->device.id ==VIRTIO_ID_BLOCK && (offset + len <sizeof (struct virtio_blk_config )))memcpy (buf, vdpasim->config + offset, len); -else if (vdpasim->device_id ==VIRTIO_ID_NET && +else if (vdpasim->device.id ==VIRTIO_ID_NET && (offset + len <sizeof (struct virtio_net_config )))memcpy (buf, vdpasim->config + offset, len); }
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +6 -2
@@ -78,9 +78,13 @@ static int __init vdpasim_blk_init(void)struct virtio_blk_config *config;int ret; - attr.device_id =VIRTIO_ID_BLOCK ; - attr.features =VDPASIM_FEATURES |VDPASIM_BLK_FEATURES ; + attr.device.id =VIRTIO_ID_BLOCK ; + attr.device.supported_features =VDPASIM_FEATURES | +VDPASIM_BLK_FEATURES ; + attr.device.nvqs =VDPASIM_BLK_VQ_NUM ; + attr.work_fn = vdpasim_blk_work; + vdpasim_blk_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_blk_dev)) { ret =PTR_ERR (vdpasim_blk_dev);
drivers/vdpa/vdpa_sim/vdpa_sim_net.c +6 -3
@@ -105,11 +105,14 @@ static int __init vdpasim_net_init(void)struct virtio_net_config *config;int ret; - attr.device_id =VIRTIO_ID_NET ; - attr.features =VDPASIM_FEATURES |VDPASIM_NET_FEATURES ; - attr.nvqs =VDPASIM_NET_VQ_NUM ; + attr.device.id =VIRTIO_ID_NET ; + attr.device.supported_features =VDPASIM_FEATURES | +VDPASIM_NET_FEATURES ; + attr.device.nvqs =VDPASIM_NET_VQ_NUM ; + attr.work_fn = vdpasim_net_work; attr.batch_mapping = batch_mapping; + vdpasim_net_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_net_dev)) { ret =PTR_ERR (vdpasim_net_dev);
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 06/12] vdpa_sim: add struct vdpasim_device to store device properties
Quoted 5 lines · Stefano Garzarella
Move device properties used during the entire life cycle in a new structure to simplify the copy of these fields during the vdpasim initialization.
Signed-off-by Stefano Garzarella <[email protected]>
It would be better to do it before patch 2.
Quoted 210 lines
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 17 ++++++++------ drivers/vdpa/vdpa_sim/vdpa_sim.c | 33 ++++++++++++++-------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 8 +++++-- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 9 +++++--- 4 files changed, 38 insertions(+), 29 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim.h +10
-7@@ -40,12 +40,17 @@ struct vdpasim_virtqueue {irqreturn_t (*cb)(void *data); }; +struct vdpasim_device { +u64 supported_features; +u32 id; +int nvqs; +}; +struct vdpasim_init_attr { -u32 device_id; -u64 features; +struct vdpasim_device device; +int batch_mapping; +work_func_t work_fn; -int batch_mapping; -int nvqs; };/* State of each vdpasim device */ @@ -53,18 +58,16 @@ struct vdpasim {struct vdpa_device vdpa;struct vdpasim_virtqueue *vqs;struct work_struct work; +struct vdpasim_device device;/* spinlock to synchronize virtqueue state */ spinlock_t lock;/* virtio config according to device type */ void *config;struct vhost_iotlb *iommu;void *buffer; -u32 device_id;u32 status;u32 generation;u64 features; -u64 supported_features; -int nvqs;/* spinlock to synchronize iommu table */ spinlock_t iommu_lock; };drivers/vdpa/vdpa_sim/vdpa_sim.c +16
-17@@ -28,7 +28,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) {struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; - vringh_init_iotlb(&vq->vring, vdpasim->supported_features, + vringh_init_iotlb(&vq->vring, vdpasim->device.supported_features, VDPASIM_QUEUE_MAX, false, (struct vring_desc *)(uintptr_t)vq->desc_addr, (struct vring_avail *)@@ -46,7 +46,7 @@ static void vdpasim_vq_reset(struct vdpasim *vdpasim, vq->device_addr =0 ; vq->cb =NULL ; vq->private =NULL ; -vringh_init_iotlb (&vq->vring, vdpasim->supported_features, +vringh_init_iotlb (&vq->vring, vdpasim->device.supported_features,VDPASIM_QUEUE_MAX , false,NULL ,NULL ,NULL ); }@@ -54,7 +54,7 @@ static void vdpasim_reset(struct vdpasim *vdpasim) {int i; -for (i =0 ; i < vdpasim->nvqs; i++) +for (i =0 ; i < vdpasim->device.nvqs; i++)vdpasim_vq_reset (vdpasim, &vdpasim->vqs[i]);spin_lock (&vdpasim->iommu_lock);@@ -189,7 +189,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)struct device *dev;int i, size, ret = -ENOMEM ; - device_id = attr->device_id; + device_id = attr->device.id;/* Currently, we only accept the network and block devices. */ if (device_id !=VIRTIO_ID_NET && device_id !=VIRTIO_ID_BLOCK )return ERR_PTR (-EOPNOTSUPP );@@ -200,10 +200,12 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr) ops = &vdpasim_config_ops; vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, - attr->nvqs); + attr->device.nvqs);if (!vdpasim) goto err_alloc; + vdpasim->device = attr->device; +if (device_id ==VIRTIO_ID_NET ) size =sizeof (struct virtio_net_config );else @@ -212,14 +214,11 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->config) goto err_iommu; - vdpasim->vqs =kcalloc (attr->nvqs,sizeof (struct vdpasim_virtqueue ), -GFP_KERNEL ); + vdpasim->vqs =kcalloc (vdpasim->device.nvqs, +sizeof (struct vdpasim_virtqueue ),GFP_KERNEL );if (!vdpasim->vqs) goto err_iommu; - vdpasim->device_id = device_id; - vdpasim->supported_features = attr->features; - vdpasim->nvqs = attr->nvqs;INIT_WORK (&vdpasim->work, attr->work_fn);spin_lock_init (&vdpasim->lock);spin_lock_init (&vdpasim->iommu_lock);@@ -238,7 +237,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->buffer) goto err_iommu; -for (i =0 ; i < vdpasim->nvqs; i++) +for (i =0 ; i < vdpasim->device.nvqs; i++)vringh_set_iotlb (&vdpasim->vqs[i].vring, vdpasim->iommu); vdpasim->vdpa.dma_dev = dev;@@ -347,7 +346,7 @@ static u64 vdpasim_get_features(struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -return vdpasim->supported_features; +return vdpasim->device.supported_features; }static int vdpasim_set_features (struct vdpa_device *vdpa,u64 features)@@ -358,14 +357,14 @@ static int vdpasim_set_features(struct vdpa_device *vdpa, u64 features)if (!(features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM )))return -EINVAL ; - vdpasim->features = features & vdpasim->supported_features; + vdpasim->features = features & vdpasim->device.supported_features;/* We generally only know whether guest is using the legacy interface * here, so generally that's the earliest we can set config fields. * Note: We actually require VIRTIO_F_ACCESS_PLATFORM above which * implies VIRTIO_F_VERSION_1, but let's not try to be clever here. */ -if (vdpasim->device_id ==VIRTIO_ID_NET ) { +if (vdpasim->device.id ==VIRTIO_ID_NET ) {struct virtio_net_config *config = (struct virtio_net_config *)vdpasim->config;@@ -391,7 +390,7 @@ static u32 vdpasim_get_device_id(struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -return vdpasim->device_id; +return vdpasim->device.id; }static u32 vdpasim_get_vendor_id (struct vdpa_device *vdpa)@@ -427,10 +426,10 @@ static void vdpasim_get_config(struct vdpa_device *vdpa, unsigned int offset, {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -if (vdpasim->device_id ==VIRTIO_ID_BLOCK && +if (vdpasim->device.id ==VIRTIO_ID_BLOCK && (offset + len <sizeof (struct virtio_blk_config )))memcpy (buf, vdpasim->config + offset, len); -else if (vdpasim->device_id ==VIRTIO_ID_NET && +else if (vdpasim->device.id ==VIRTIO_ID_NET && (offset + len <sizeof (struct virtio_net_config )))memcpy (buf, vdpasim->config + offset, len); }drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +6
-2@@ -78,9 +78,13 @@ static int __init vdpasim_blk_init(void)struct virtio_blk_config *config;int ret; - attr.device_id =VIRTIO_ID_BLOCK ; - attr.features =VDPASIM_FEATURES |VDPASIM_BLK_FEATURES ; + attr.device.id =VIRTIO_ID_BLOCK ; + attr.device.supported_features =VDPASIM_FEATURES | +VDPASIM_BLK_FEATURES ; + attr.device.nvqs =VDPASIM_BLK_VQ_NUM ; + attr.work_fn = vdpasim_blk_work; + vdpasim_blk_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_blk_dev)) { ret =PTR_ERR (vdpasim_blk_dev);drivers/vdpa/vdpa_sim/vdpa_sim_net.c +0
-1@@ -105,11 +105,14 @@ static int __init vdpasim_net_init(void)struct virtio_net_config *config;int ret; - attr.device_id =VIRTIO_ID_NET ;
drivers/vdpa/vdpa_sim/vdpa_sim_net.c static int __init vdpasim_net_init(void) · line 115
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_net.c +6
-2- attr.features =VDPASIM_FEATURES |VDPASIM_NET_FEATURES ; - attr.nvqs =VDPASIM_NET_VQ_NUM ; + attr.device.id =VIRTIO_ID_NET ; + attr.device.supported_features =VDPASIM_FEATURES | +VDPASIM_NET_FEATURES ; + attr.device.nvqs =VDPASIM_NET_VQ_NUM ; + attr.work_fn = vdpasim_net_work; attr.batch_mapping = batch_mapping; +
Unnecessary changes.
Thanks
drivers/vdpa/vdpa_sim/vdpa_sim_net.c static int __init vdpasim_net_init(void) · line 118
Quoted 3 lines
drivers/vdpa/vdpa_sim/vdpa_sim_net.c +0
-0vdpasim_net_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_net_dev)) { ret =PTR_ERR (vdpasim_net_dev);
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 06/12] vdpa_sim: add struct vdpasim_device to store device properties
Quoted 11 lines · Jason Wang
Quoted 5 lines · Stefano Garzarella
Move device properties used during the entire life cycle in a new structure to simplify the copy of these fields during the vdpasim initialization.
Signed-off-by Stefano Garzarella <[email protected]>It would be better to do it before patch 2.
Okay, I'll move this patch.
Quoted 193 lines
Quoted 193 lines
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 17 ++++++++------ drivers/vdpa/vdpa_sim/vdpa_sim.c | 33 ++++++++++++++-------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 8 +++++-- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 9 +++++--- 4 files changed, 38 insertions(+), 29 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim.h +10
-7@@ -40,12 +40,17 @@ struct vdpasim_virtqueue {irqreturn_t (*cb)(void *data); }; +struct vdpasim_device { +u64 supported_features; +u32 id; +int nvqs; +}; +struct vdpasim_init_attr { -u32 device_id; -u64 features; +struct vdpasim_device device; +int batch_mapping; +work_func_t work_fn; -int batch_mapping; -int nvqs; };/* State of each vdpasim device */ @@ -53,18 +58,16 @@ struct vdpasim {struct vdpa_device vdpa;struct vdpasim_virtqueue *vqs;struct work_struct work; +struct vdpasim_device device;/* spinlock to synchronize virtqueue state */ spinlock_t lock;/* virtio config according to device type */ void *config;struct vhost_iotlb *iommu;void *buffer; -u32 device_id;u32 status;u32 generation;u64 features; -u64 supported_features; -int nvqs;/* spinlock to synchronize iommu table */ spinlock_t iommu_lock; };drivers/vdpa/vdpa_sim/vdpa_sim.c +16
-17@@ -28,7 +28,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) {struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; - vringh_init_iotlb(&vq->vring, vdpasim->supported_features, + vringh_init_iotlb(&vq->vring, vdpasim->device.supported_features, VDPASIM_QUEUE_MAX, false, (struct vring_desc *)(uintptr_t)vq->desc_addr, (struct vring_avail *)@@ -46,7 +46,7 @@ static void vdpasim_vq_reset(struct vdpasim *vdpasim, vq->device_addr =0 ; vq->cb =NULL ; vq->private =NULL ; -vringh_init_iotlb (&vq->vring, vdpasim->supported_features, +vringh_init_iotlb (&vq->vring, vdpasim->device.supported_features,VDPASIM_QUEUE_MAX , false,NULL ,NULL ,NULL ); }@@ -54,7 +54,7 @@ static void vdpasim_reset(struct vdpasim *vdpasim) {int i; -for (i =0 ; i < vdpasim->nvqs; i++) +for (i =0 ; i < vdpasim->device.nvqs; i++)vdpasim_vq_reset (vdpasim, &vdpasim->vqs[i]);spin_lock (&vdpasim->iommu_lock);@@ -189,7 +189,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)struct device *dev;int i, size, ret = -ENOMEM ; - device_id = attr->device_id; + device_id = attr->device.id;/* Currently, we only accept the network and block devices. */ if (device_id !=VIRTIO_ID_NET && device_id !=VIRTIO_ID_BLOCK )return ERR_PTR (-EOPNOTSUPP );@@ -200,10 +200,12 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr) ops = &vdpasim_config_ops; vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, - attr->nvqs); + attr->device.nvqs);if (!vdpasim) goto err_alloc; + vdpasim->device = attr->device; +if (device_id ==VIRTIO_ID_NET ) size =sizeof (struct virtio_net_config );else @@ -212,14 +214,11 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->config) goto err_iommu; - vdpasim->vqs =kcalloc (attr->nvqs,sizeof (struct vdpasim_virtqueue ), -GFP_KERNEL ); + vdpasim->vqs =kcalloc (vdpasim->device.nvqs, +sizeof (struct vdpasim_virtqueue ),GFP_KERNEL );if (!vdpasim->vqs) goto err_iommu; - vdpasim->device_id = device_id; - vdpasim->supported_features = attr->features; - vdpasim->nvqs = attr->nvqs;INIT_WORK (&vdpasim->work, attr->work_fn);spin_lock_init (&vdpasim->lock);spin_lock_init (&vdpasim->iommu_lock);@@ -238,7 +237,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->buffer) goto err_iommu; -for (i =0 ; i < vdpasim->nvqs; i++) +for (i =0 ; i < vdpasim->device.nvqs; i++)vringh_set_iotlb (&vdpasim->vqs[i].vring, vdpasim->iommu); vdpasim->vdpa.dma_dev = dev;@@ -347,7 +346,7 @@ static u64 vdpasim_get_features(struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -return vdpasim->supported_features; +return vdpasim->device.supported_features; }static int vdpasim_set_features (struct vdpa_device *vdpa,u64 features)@@ -358,14 +357,14 @@ static int vdpasim_set_features(struct vdpa_device *vdpa, u64 features)if (!(features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM )))return -EINVAL ; - vdpasim->features = features & vdpasim->supported_features; + vdpasim->features = features & vdpasim->device.supported_features;/* We generally only know whether guest is using the legacy interface * here, so generally that's the earliest we can set config fields. * Note: We actually require VIRTIO_F_ACCESS_PLATFORM above which * implies VIRTIO_F_VERSION_1, but let's not try to be clever here. */ -if (vdpasim->device_id ==VIRTIO_ID_NET ) { +if (vdpasim->device.id ==VIRTIO_ID_NET ) {struct virtio_net_config *config = (struct virtio_net_config *)vdpasim->config;@@ -391,7 +390,7 @@ static u32 vdpasim_get_device_id(struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -return vdpasim->device_id; +return vdpasim->device.id; }static u32 vdpasim_get_vendor_id (struct vdpa_device *vdpa)@@ -427,10 +426,10 @@ static void vdpasim_get_config(struct vdpa_device *vdpa, unsigned int offset, {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -if (vdpasim->device_id ==VIRTIO_ID_BLOCK && +if (vdpasim->device.id ==VIRTIO_ID_BLOCK && (offset + len <sizeof (struct virtio_blk_config )))memcpy (buf, vdpasim->config + offset, len); -else if (vdpasim->device_id ==VIRTIO_ID_NET && +else if (vdpasim->device.id ==VIRTIO_ID_NET && (offset + len <sizeof (struct virtio_net_config )))memcpy (buf, vdpasim->config + offset, len); }drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +6
-2@@ -78,9 +78,13 @@ static int __init vdpasim_blk_init(void)struct virtio_blk_config *config;int ret; - attr.device_id =VIRTIO_ID_BLOCK ; - attr.features =VDPASIM_FEATURES |VDPASIM_BLK_FEATURES ; + attr.device.id =VIRTIO_ID_BLOCK ; + attr.device.supported_features =VDPASIM_FEATURES | +VDPASIM_BLK_FEATURES ; + attr.device.nvqs =VDPASIM_BLK_VQ_NUM ; + attr.work_fn = vdpasim_blk_work; + vdpasim_blk_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_blk_dev)) { ret =PTR_ERR (vdpasim_blk_dev);drivers/vdpa/vdpa_sim/vdpa_sim_net.c +3
-3@@ -105,11 +105,14 @@ static int __init vdpasim_net_init(void)struct virtio_net_config *config;int ret; - attr.device_id =VIRTIO_ID_NET ; - attr.features =VDPASIM_FEATURES |VDPASIM_NET_FEATURES ; - attr.nvqs =VDPASIM_NET_VQ_NUM ; + attr.device.id =VIRTIO_ID_NET ; + attr.device.supported_features =VDPASIM_FEATURES | +VDPASIM_NET_FEATURES ;
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_net.c static int __init vdpasim_net_init(void) · line 115
Quoted 5 lines
drivers/vdpa/vdpa_sim/vdpa_sim_net.c +3
-0+ attr.device.nvqs =VDPASIM_NET_VQ_NUM ; + attr.work_fn = vdpasim_net_work; attr.batch_mapping = batch_mapping; +Unnecessary changes.
I'll remove these new lines.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 06/12] vdpa_sim: add struct vdpasim_device to store device properties
In reply to Stefano GarzarellaQuoted 20 lines · Stefano Garzarella
Move device properties used during the entire life cycle in a new structure to simplify the copy of these fields during the vdpasim initialization.
Signed-off-by Stefano Garzarella <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 17 ++++++++------ drivers/vdpa/vdpa_sim/vdpa_sim.c | 33 ++++++++++++++-------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 8 +++++-- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 9 +++++--- 4 files changed, 38 insertions(+), 29 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim.h +0
-0@@ -40,12 +40,17 @@ struct vdpasim_virtqueue {irqreturn_t (*cb)(void *data); };
drivers/vdpa/vdpa_sim/vdpa_sim.h struct vdpasim_virtqueue { · line 50
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim.h +7
-2+struct vdpasim_device { +u64 supported_features; +u32 id; +int nvqs; +}; +struct vdpasim_init_attr { -u32 device_id; -u64 features; +struct vdpasim_device device;
It's unclear to me what the exact purpose of struct vdpasim_device is. At least the name reminds me of struct device, which this is not.
Should this be called just struct vdpasim_attr or struct vdpasim_dev_attr? In other words, the attributes that are needed even after intialization?
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 06/12] vdpa_sim: add struct vdpasim_device to store device properties
Quoted 35 lines · Stefan Hajnoczi
Quoted 20 lines · Stefano Garzarella
Move device properties used during the entire life cycle in a new structure to simplify the copy of these fields during the vdpasim initialization.
Signed-off-by Stefano Garzarella <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 17 ++++++++------ drivers/vdpa/vdpa_sim/vdpa_sim.c | 33 ++++++++++++++-------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 8 +++++-- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 9 +++++--- 4 files changed, 38 insertions(+), 29 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim.h +0
-0@@ -40,12 +40,17 @@ struct vdpasim_virtqueue {irqreturn_t (*cb)(void *data); };drivers/vdpa/vdpa_sim/vdpa_sim.h struct vdpasim_virtqueue { · line 50
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim.h +7
-2+struct vdpasim_device { +u64 supported_features; +u32 id; +int nvqs; +}; +struct vdpasim_init_attr { -u32 device_id; -u64 features; +struct vdpasim_device device;It's unclear to me what the exact purpose of struct
vdpasim_deviceis. At least the name reminds me of struct device, which this is not.
Quoted 3 lines
Should this be called just struct
vdpasim_attror structvdpasim_dev_attr? In other words, the attributes that are needed even after intialization?
Yes, they are attributes that are needed even after intialization, so I think vdpasim_dev_attr should be better.
I'll change it and I'll try to write a better commit message.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RFC PATCH 7/12 vdpa_sim: move config management outside of the core
In reply to Stefano GarzarellaIn order to simplify the code of the vdpa_sim core, we move the config management in each device simulator.
The device must provide the size of config structure and a callback to update this structure called during the vdpasim_set_features().
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 5 +++-- drivers/vdpa/vdpa_sim/vdpa_sim.c | 29 +++++----------------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 27 ++++++++++++++++---------- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 12 ++++++++++++ 4 files changed, 37 insertions(+), 36 deletions(-)
drivers/vdpa/vdpa_sim/vdpa_sim.h +3 -2
@@ -10,8 +10,6 @@#include <linux/vdpa.h> #include <linux/vhost_iotlb.h> #include <uapi/linux/virtio_config.h> -#include <uapi/linux/virtio_net.h> -#include <uapi/linux/virtio_blk.h> #define DRV_VERSION "0.1"#define DRV_AUTHOR "Jason Wang <[email protected]>"
@@ -42,8 +40,11 @@ struct vdpasim_virtqueue {struct vdpasim_device {u64 supported_features; +size_t config_size;u32 id;int nvqs; + +void (*update_config)(struct vdpasim *vdpasim); };struct vdpasim_init_attr {
drivers/vdpa/vdpa_sim/vdpa_sim.c +5 -24
@@ -185,14 +185,8 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr) {const struct vdpa_config_ops *ops;struct vdpasim *vdpasim; -u32 device_id;struct device *dev; -int i, size, ret = -ENOMEM ; - - device_id = attr->device.id; -/* Currently, we only accept the network and block devices. */ -if (device_id !=VIRTIO_ID_NET && device_id !=VIRTIO_ID_BLOCK ) -return ERR_PTR (-EOPNOTSUPP ); +int i, ret = -ENOMEM ;if (attr->batch_mapping) ops = &vdpasim_batch_config_ops;
@@ -206,11 +200,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr) vdpasim->device = attr->device; -if (device_id ==VIRTIO_ID_NET ) - size =sizeof (struct virtio_net_config ); -else - size =sizeof (struct virtio_blk_config ); - vdpasim->config =kzalloc (size,GFP_KERNEL ); + vdpasim->config =kzalloc (vdpasim->device.config_size,GFP_KERNEL );if (!vdpasim->config) goto err_iommu;
@@ -364,13 +354,8 @@ static int vdpasim_set_features(struct vdpa_device *vdpa, u64 features)* Note: We actually require VIRTIO_F_ACCESS_PLATFORM above which * implies VIRTIO_F_VERSION_1, but let's not try to be clever here. */ -if (vdpasim->device.id ==VIRTIO_ID_NET ) { -struct virtio_net_config *config = - (struct virtio_net_config *)vdpasim->config; - - config->mtu =cpu_to_vdpasim16 (vdpasim,1500 ); - config->status =cpu_to_vdpasim16 (vdpasim,VIRTIO_NET_S_LINK_UP ); - } +if (vdpasim->device.update_config) + vdpasim->device.update_config (vdpasim);return 0 ; }
@@ -426,11 +411,7 @@ static void vdpasim_get_config(struct vdpa_device *vdpa, unsigned int offset, {struct vdpasim *vdpasim =vdpa_to_sim (vdpa); -if (vdpasim->device.id ==VIRTIO_ID_BLOCK && - (offset + len <sizeof (struct virtio_blk_config ))) -memcpy (buf, vdpasim->config + offset, len); -else if (vdpasim->device.id ==VIRTIO_ID_NET && - (offset + len <sizeof (struct virtio_net_config ))) +if (offset + len < vdpasim->device.config_size)memcpy (buf, vdpasim->config + offset, len); }
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +17 -10
@@ -7,6 +7,7 @@*/ #include <linux/module.h> +#include <uapi/linux/virtio_blk.h> #include "vdpa_sim.h"
@@ -72,16 +73,31 @@ static void vdpasim_blk_work(struct work_struct *work) } +static void vdpasim_blk_update_config (struct vdpasim *vdpasim) +{ +struct virtio_blk_config *config = + (struct virtio_blk_config *)vdpasim->config; + + config->capacity =cpu_to_vdpasim64 (vdpasim,VDPASIM_BLK_CAPACITY ); + config->size_max =cpu_to_vdpasim32 (vdpasim,VDPASIM_BLK_SIZE_MAX ); + config->seg_max =cpu_to_vdpasim32 (vdpasim,VDPASIM_BLK_SEG_MAX ); + config->num_queues =cpu_to_vdpasim16 (vdpasim,VDPASIM_BLK_VQ_NUM ); + config->min_io_size =cpu_to_vdpasim16 (vdpasim,1 ); + config->opt_io_size =cpu_to_vdpasim32 (vdpasim,1 ); + config->blk_size =cpu_to_vdpasim32 (vdpasim,512 ); +} +static int __init vdpasim_blk_init (void ) {struct vdpasim_init_attr attr = {}; -struct virtio_blk_config *config;int ret; attr.device.id =VIRTIO_ID_BLOCK ; attr.device.supported_features =VDPASIM_FEATURES |VDPASIM_BLK_FEATURES ; attr.device.nvqs =VDPASIM_BLK_VQ_NUM ; + attr.device.config_size =sizeof (struct virtio_blk_config ); + attr.device.update_config = vdpasim_blk_update_config; attr.work_fn = vdpasim_blk_work;
@@ -91,15 +107,6 @@ static int __init vdpasim_blk_init(void) goto out; } - config = (struct virtio_blk_config *)vdpasim_blk_dev->config; - config->capacity =cpu_to_vdpasim64 (vdpasim_blk_dev,VDPASIM_BLK_CAPACITY ); - config->size_max =cpu_to_vdpasim32 (vdpasim_blk_dev,VDPASIM_BLK_SIZE_MAX ); - config->seg_max =cpu_to_vdpasim32 (vdpasim_blk_dev,VDPASIM_BLK_SEG_MAX ); - config->num_queues =cpu_to_vdpasim16 (vdpasim_blk_dev,VDPASIM_BLK_VQ_NUM ); - config->min_io_size =cpu_to_vdpasim16 (vdpasim_blk_dev,1 ); - config->opt_io_size =cpu_to_vdpasim32 (vdpasim_blk_dev,1 ); - config->blk_size =cpu_to_vdpasim32 (vdpasim_blk_dev,512 ); - ret =vdpa_register_device (&vdpasim_blk_dev->vdpa);if (ret) goto put_dev;
drivers/vdpa/vdpa_sim/vdpa_sim_net.c +12 -0
@@ -9,6 +9,7 @@#include <linux/module.h> #include <linux/etherdevice.h> +#include <uapi/linux/virtio_net.h> #include "vdpa_sim.h"
@@ -99,6 +100,15 @@ static void vdpasim_net_work(struct work_struct *work)spin_unlock (&vdpasim->lock); } +static void vdpasim_net_update_config (struct vdpasim *vdpasim) +{ +struct virtio_net_config *config = + (struct virtio_net_config *)vdpasim->config; + + config->mtu =cpu_to_vdpasim16 (vdpasim,1500 ); + config->status =cpu_to_vdpasim16 (vdpasim,VIRTIO_NET_S_LINK_UP ); +} +static int __init vdpasim_net_init (void ) {struct vdpasim_init_attr attr = {};
@@ -109,6 +119,8 @@ static int __init vdpasim_net_init(void) attr.device.supported_features =VDPASIM_FEATURES |VDPASIM_NET_FEATURES ; attr.device.nvqs =VDPASIM_NET_VQ_NUM ; + attr.device.config_size =sizeof (struct virtio_net_config ); + attr.device.update_config = vdpasim_net_update_config; attr.work_fn = vdpasim_net_work; attr.batch_mapping = batch_mapping;
Signature · 1 line
2.26.2
Re: [PATCH RFC 07/12] vdpa_sim: move config management outside of the core
Quoted 5 lines · Stefano Garzarella
In order to simplify the code of the
vdpa_simcore, we move the config management in each device simulator.The device must provide the size of config structure and a callback to update this structure called during the
vdpasim_set_features().
Similarly, I suggest to do this before patch 2, then there's no need for the conversion of blk device.
Quoted 23 lines
Signed-off-by Stefano Garzarella <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 5 +++-- drivers/vdpa/vdpa_sim/vdpa_sim.c | 29 +++++----------------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 27 ++++++++++++++++---------- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 12 ++++++++++++ 4 files changed, 37 insertions(+), 36 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim.h +0
-2@@ -10,8 +10,6 @@#include <linux/vdpa.h> #include <linux/vhost_iotlb.h> #include <uapi/linux/virtio_config.h> -#include <uapi/linux/virtio_net.h> -#include <uapi/linux/virtio_blk.h> #define DRV_VERSION "0.1"#define DRV_AUTHOR "Jason Wang <[email protected]>"
drivers/vdpa/vdpa_sim/vdpa_sim.h struct vdpasim_virtqueue { · line 47
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim.h +3
-0@@ -42,8 +40,11 @@ struct vdpasim_virtqueue {struct vdpasim_device {u64 supported_features; +size_t config_size;u32 id;int nvqs; + +void (*update_config)(struct vdpasim *vdpasim);
Let's use set_config/get_config to align with virtio/vhost.
Other looks good.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 07/12] vdpa_sim: move config management outside of the core
Quoted 11 lines · Jason Wang
Quoted 5 lines · Stefano Garzarella
In order to simplify the code of the
vdpa_simcore, we move the config management in each device simulator.The device must provide the size of config structure and a callback to update this structure called during the
vdpasim_set_features().Similarly, I suggest to do this before patch 2, then there's no need for the conversion of blk device.
I'll do.
Quoted 25 lines
Quoted 25 lines
Signed-off-by Stefano Garzarella <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 5 +++-- drivers/vdpa/vdpa_sim/vdpa_sim.c | 29 +++++----------------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 27 ++++++++++++++++---------- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 12 ++++++++++++ 4 files changed, 37 insertions(+), 36 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim.h +0
-2@@ -10,8 +10,6 @@#include <linux/vdpa.h> #include <linux/vhost_iotlb.h> #include <uapi/linux/virtio_config.h> -#include <uapi/linux/virtio_net.h> -#include <uapi/linux/virtio_blk.h> #define DRV_VERSION "0.1"#define DRV_AUTHOR "Jason Wang <[email protected]>"@@ -42,8 +40,11 @@ struct vdpasim_virtqueue {struct vdpasim_device {u64 supported_features;
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim.h struct vdpasim_virtqueue { · line 47
Quoted 5 lines
drivers/vdpa/vdpa_sim/vdpa_sim.h +3
-0+size_t config_size;u32 id;int nvqs; + +void (*update_config)(struct vdpasim *vdpasim);Let's use set_config/get_config to align with virtio/vhost.
Yes, it's better,
Quoted 2 lines
Other looks good.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RFC PATCH 8/12 vdpa_sim: use kvmalloc to allocate vdpasim->buffer
In reply to Stefano GarzarellaThe next patch will make the buffer size configurable from each device.
Since the buffer could be larger than a page, we use kvmalloc() instead of kmalloc().
--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/vdpa/vdpa_sim/vdpa_sim.c +2 -2
@@ -223,7 +223,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->iommu) goto err_iommu; - vdpasim->buffer =kmalloc (PAGE_SIZE ,GFP_KERNEL ); + vdpasim->buffer =kvmalloc (PAGE_SIZE ,GFP_KERNEL );if (!vdpasim->buffer) goto err_iommu;
@@ -495,7 +495,7 @@ static void vdpasim_free(struct vdpa_device *vdpa)struct vdpasim *vdpasim =vdpa_to_sim (vdpa);cancel_work_sync (&vdpasim->work); -kfree (vdpasim->buffer); +kvfree (vdpasim->buffer);if (vdpasim->iommu)vhost_iotlb_free (vdpasim->iommu);kfree (vdpasim->vqs);
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 08/12] vdpa_sim: use kvmalloc to allocate vdpasim->buffer
Quoted 6 lines · Stefano Garzarella
The next patch will make the buffer size configurable from each device.
Since the buffer could be larger than a page, we usekvmalloc()instead ofkmalloc().Signed-off-by Stefano Garzarella <[email protected]>
Thanks
Quoted 26 lines
--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim.c +2
-2@@ -223,7 +223,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->iommu) goto err_iommu; - vdpasim->buffer =kmalloc (PAGE_SIZE ,GFP_KERNEL ); + vdpasim->buffer =kvmalloc (PAGE_SIZE ,GFP_KERNEL );if (!vdpasim->buffer) goto err_iommu;@@ -495,7 +495,7 @@ static void vdpasim_free(struct vdpa_device *vdpa)struct vdpasim *vdpasim =vdpa_to_sim (vdpa);cancel_work_sync (&vdpasim->work); -kfree (vdpasim->buffer); +kvfree (vdpasim->buffer);if (vdpasim->iommu)vhost_iotlb_free (vdpasim->iommu);kfree (vdpasim->vqs);
RFC PATCH 9/12 vdpa_sim: make vdpasim->buffer size configurable
In reply to Stefano GarzarellaAllow each device to specify the size of the buffer allocated in vdpa_sim.
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 1 + drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 1 + 4 files changed, 4 insertions(+), 1 deletion(-)
drivers/vdpa/vdpa_sim/vdpa_sim.h +1 -0
@@ -49,6 +49,7 @@ struct vdpasim_device {struct vdpasim_init_attr {struct vdpasim_device device; +size_t buffer_size;int batch_mapping;work_func_t work_fn;
drivers/vdpa/vdpa_sim/vdpa_sim.c +1 -1
@@ -223,7 +223,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->iommu) goto err_iommu; - vdpasim->buffer =kvmalloc (PAGE_SIZE ,GFP_KERNEL ); + vdpasim->buffer =kvmalloc (attr->buffer_size,GFP_KERNEL );if (!vdpasim->buffer) goto err_iommu;
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +1 -0
@@ -100,6 +100,7 @@ static int __init vdpasim_blk_init(void) attr.device.update_config = vdpasim_blk_update_config; attr.work_fn = vdpasim_blk_work; + attr.buffer_size =PAGE_SIZE ; vdpasim_blk_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_blk_dev)) {
drivers/vdpa/vdpa_sim/vdpa_sim_net.c +1 -0
@@ -124,6 +124,7 @@ static int __init vdpasim_net_init(void) attr.work_fn = vdpasim_net_work; attr.batch_mapping = batch_mapping; + attr.buffer_size =PAGE_SIZE ; vdpasim_net_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_net_dev)) {
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 09/12] vdpa_sim: make vdpasim->buffer size configurable
Quoted 4 lines · Stefano Garzarella
Allow each device to specify the size of the buffer allocated in
vdpa_sim.Signed-off-by Stefano Garzarella <[email protected]>
Quoted 49 lines
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 +- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 1 + drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 1 + 4 files changed, 4 insertions(+), 1 deletion(-)drivers/vdpa/vdpa_sim/vdpa_sim.h +1
-0@@ -49,6 +49,7 @@ struct vdpasim_device {struct vdpasim_init_attr {struct vdpasim_device device; +size_t buffer_size;int batch_mapping;work_func_t work_fn;drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-1@@ -223,7 +223,7 @@ struct vdpasim *vdpasim_create(struct vdpasim_init_attr *attr)if (!vdpasim->iommu) goto err_iommu; - vdpasim->buffer =kvmalloc (PAGE_SIZE ,GFP_KERNEL ); + vdpasim->buffer =kvmalloc (attr->buffer_size,GFP_KERNEL );if (!vdpasim->buffer) goto err_iommu;drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +1
-0@@ -100,6 +100,7 @@ static int __init vdpasim_blk_init(void) attr.device.update_config = vdpasim_blk_update_config; attr.work_fn = vdpasim_blk_work; + attr.buffer_size =PAGE_SIZE ; vdpasim_blk_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_blk_dev)) {drivers/vdpa/vdpa_sim/vdpa_sim_net.c +0
-0
drivers/vdpa/vdpa_sim/vdpa_sim_net.c static int __init vdpasim_net_init(void) · line 130
Quoted 7 lines
drivers/vdpa/vdpa_sim/vdpa_sim_net.c +1
-0@@ -124,6 +124,7 @@ static int __init vdpasim_net_init(void) attr.work_fn = vdpasim_net_work; attr.batch_mapping = batch_mapping; + attr.buffer_size =PAGE_SIZE ; vdpasim_net_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_net_dev)) {
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RFC PATCH 10/12 vdpa_sim: split vdpasim_virtqueue's iov field in riov and wiov
In reply to Stefano Garzarellavringh_getdesc_iotlb() manages 2 iovs for writable and readable descriptors. This is very useful for the block device, where for each request we have both types of descriptor.
Let's split the vdpasim_virtqueue's iov field in riov and wiov to use them with vringh_getdesc_iotlb().
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 3 ++- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 6 +++--- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 8 ++++---- 3 files changed, 9 insertions(+), 8 deletions(-)
drivers/vdpa/vdpa_sim/vdpa_sim.h +2 -1
@@ -27,7 +27,8 @@ struct vdpasim;struct vdpasim_virtqueue {struct vringh vring; -struct vringh_kiov iov; +struct vringh_kiov riov; +struct vringh_kiov wiov;unsigned short head;bool ready;u64 desc_addr;
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +3 -3
@@ -41,13 +41,13 @@ static void vdpasim_blk_work(struct work_struct *work)if (!vq->ready)continue ; -while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, +while (vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, &vq->head,GFP_ATOMIC ) >0 ) {int write; - vq->iov.i = vq->iov.used -1 ; - write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); + vq->wiov.i = vq->wiov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, &status,1 );if (write <=0 )break ;
drivers/vdpa/vdpa_sim/vdpa_sim_net.c +4 -4
@@ -47,12 +47,12 @@ static void vdpasim_net_work(struct work_struct *work)while (true) { total_write =0 ; - err =vringh_getdesc_iotlb (&txq->vring, &txq->iov,NULL , + err =vringh_getdesc_iotlb (&txq->vring, &txq->riov,NULL , &txq->head,GFP_ATOMIC );if (err <=0 )break ; - err =vringh_getdesc_iotlb (&rxq->vring,NULL , &rxq->iov, + err =vringh_getdesc_iotlb (&rxq->vring,NULL , &rxq->wiov, &rxq->head,GFP_ATOMIC );if (err <=0 ) {vringh_complete_iotlb (&txq->vring, txq->head,0 );
@@ -60,13 +60,13 @@ static void vdpasim_net_work(struct work_struct *work) }while (true) { - read =vringh_iov_pull_iotlb (&txq->vring, &txq->iov, + read =vringh_iov_pull_iotlb (&txq->vring, &txq->riov, vdpasim->buffer,PAGE_SIZE );if (read <=0 )break ; - write =vringh_iov_push_iotlb (&rxq->vring, &rxq->iov, + write =vringh_iov_push_iotlb (&rxq->vring, &rxq->wiov, vdpasim->buffer, read);if (write <=0 )break ;
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 10/12] vdpa_sim: split vdpasim_virtqueue's iov field in riov and wiov
Quoted 8 lines · Stefano Garzarella
vringh_getdesc_iotlb()manages 2 iovs for writable and readable descriptors. This is very useful for the block device, where for each request we have both types of descriptor.Let's split the vdpasim_virtqueue's iov field in riov and wiov to use them with
vringh_getdesc_iotlb().Signed-off-by Stefano Garzarella <[email protected]>
Quoted 76 lines
--- drivers/vdpa/vdpa_sim/vdpa_sim.h | 3 ++- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 6 +++--- drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 8 ++++---- 3 files changed, 9 insertions(+), 8 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim.h +2
-1@@ -27,7 +27,8 @@ struct vdpasim;struct vdpasim_virtqueue {struct vringh vring; -struct vringh_kiov iov; +struct vringh_kiov riov; +struct vringh_kiov wiov;unsigned short head;bool ready;u64 desc_addr;drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +3
-3@@ -41,13 +41,13 @@ static void vdpasim_blk_work(struct work_struct *work)if (!vq->ready)continue ; -while (vringh_getdesc_iotlb (&vq->vring, &vq->iov, &vq->iov, +while (vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, &vq->head,GFP_ATOMIC ) >0 ) {int write; - vq->iov.i = vq->iov.used -1 ; - write =vringh_iov_push_iotlb (&vq->vring, &vq->iov, &status,1 ); + vq->wiov.i = vq->wiov.used -1 ; + write =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, &status,1 );if (write <=0 )break ;drivers/vdpa/vdpa_sim/vdpa_sim_net.c +4
-4@@ -47,12 +47,12 @@ static void vdpasim_net_work(struct work_struct *work)while (true) { total_write =0 ; - err =vringh_getdesc_iotlb (&txq->vring, &txq->iov,NULL , + err =vringh_getdesc_iotlb (&txq->vring, &txq->riov,NULL , &txq->head,GFP_ATOMIC );if (err <=0 )break ; - err =vringh_getdesc_iotlb (&rxq->vring,NULL , &rxq->iov, + err =vringh_getdesc_iotlb (&rxq->vring,NULL , &rxq->wiov, &rxq->head,GFP_ATOMIC );if (err <=0 ) {vringh_complete_iotlb (&txq->vring, txq->head,0 );@@ -60,13 +60,13 @@ static void vdpasim_net_work(struct work_struct *work) }while (true) { - read =vringh_iov_pull_iotlb (&txq->vring, &txq->iov, + read =vringh_iov_pull_iotlb (&txq->vring, &txq->riov, vdpasim->buffer,PAGE_SIZE );if (read <=0 )break ; - write =vringh_iov_push_iotlb (&rxq->vring, &rxq->iov, + write =vringh_iov_push_iotlb (&rxq->vring, &rxq->wiov, vdpasim->buffer, read);if (write <=0 )break ;
Re: [PATCH RFC 10/12] vdpa_sim: split vdpasim_virtqueue's iov field in riov and wiov
In reply to Stefano GarzarellaQuoted 6 lines · Stefano Garzarella
vringh_getdesc_iotlb()manages 2 iovs for writable and readable descriptors. This is very useful for the block device, where for each request we have both types of descriptor.Let's split the vdpasim_virtqueue's iov field in riov and wiov to use them with
vringh_getdesc_iotlb().
Is riov/wiov naming common? VIRTIO uses "in" (device-to-driver) and "out" (driver-to-device). Using VIRTIO terminology might be clearer.
Stefan
Re: [PATCH RFC 10/12] vdpa_sim: split vdpasim_virtqueue's iov field in riov and wiov
Quoted 10 lines · Stefan Hajnoczi
Quoted 6 lines · Stefano Garzarella
vringh_getdesc_iotlb()manages 2 iovs for writable and readable descriptors. This is very useful for the block device, where for each request we have both types of descriptor.Let's split the vdpasim_virtqueue's iov field in riov and wiov to use them with
vringh_getdesc_iotlb().Is riov/wiov naming common? VIRTIO uses "in" (device-to-driver) and "out" (driver-to-device). Using VIRTIO terminology might be clearer.
I followed the vringh_getdesc_iotlb() attribute names, but I agree that "in" and "out" would be better. I lost multiple times with read/write...
I'll fix!
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RFC PATCH 11/12 vringh: allow vringh_iov_xfer() to skip bytes when ptr is NULL
In reply to Stefano GarzarellaIn some cases, it may be useful to provide a way to skip a number of bytes in a vringh_iov.
In order to keep vringh_iov consistent, let's reuse vringh_iov_xfer() logic and skip bytes when the ptr is NULL.
---
I'm not sure if this is the best option, maybe we can add a new function vringh_iov_skip().
Suggestions?
--- drivers/vhost/vringh.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)
drivers/vhost/vringh.c +11 -5
@@ -75,7 +75,9 @@ static inline int __vringh_get_head(const struct vringh *vrh,return head; } -/* Copy some bytes to/from the iovec. Returns num copied. */ +/* Copy some bytes to/from the iovec. Returns num copied. +* If ptr is NULL, skips at most len bytes. +*/ static inline ssize_t vringh_iov_xfer(struct vringh *vrh, struct vringh_kiov *iov, void *ptr, size_t len,
@@ -89,12 +91,16 @@ static inline ssize_t vringh_iov_xfer(struct vringh *vrh,size_t partlen; partlen =min (iov->iov[iov->i].iov_len, len); - err =xfer (vrh, iov->iov[iov->i].iov_base, ptr, partlen); -if (err) -return err; + +if (ptr) { + err =xfer (vrh, iov->iov[iov->i].iov_base, ptr, partlen); +if (err) +return err; + ptr += partlen; + } + done += partlen; len -= partlen; - ptr += partlen; iov->consumed += partlen; iov->iov[iov->i].iov_len -= partlen; iov->iov[iov->i].iov_base += partlen;
Signature · 6 lines
2.26.2 _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 11/12] vringh: allow vringh_iov_xfer() to skip bytes when ptr is NULL
Quoted 6 lines · Stefano Garzarella
In some cases, it may be useful to provide a way to skip a number of bytes in a
vringh_iov.In order to keep
vringh_iovconsistent, let's reusevringh_iov_xfer()logic and skip bytes when the ptr is NULL.
Quoted 7 lines
Signed-off-by Stefano Garzarella <[email protected]>---
I'm not sure if this is the best option, maybe we can add a new function
vringh_iov_skip().Suggestions?
I might be worth to check whether we can convert vringh_iov to use iov iterator then we can use iov_iterator_advance() here.
Thanks
Quoted 30 lines
--- drivers/vhost/vringh.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)drivers/vhost/vringh.c +7
-4@@ -75,7 +75,9 @@ static inline int __vringh_get_head(const struct vringh *vrh,return head; } -/* Copy some bytes to/from the iovec. Returns num copied. */ +/* Copy some bytes to/from the iovec. Returns num copied. +* If ptr is NULL, skips at most len bytes. +*/ static inline ssize_t vringh_iov_xfer(struct vringh *vrh, struct vringh_kiov *iov, void *ptr, size_t len,@@ -89,12 +91,16 @@ static inline ssize_t vringh_iov_xfer(struct vringh *vrh,size_t partlen; partlen =min (iov->iov[iov->i].iov_len, len); - err =xfer (vrh, iov->iov[iov->i].iov_base, ptr, partlen); -if (err) -return err; + +if (ptr) { + err =xfer (vrh, iov->iov[iov->i].iov_base, ptr, partlen); +if (err)
drivers/vhost/vringh.c static inline ssize_t vringh_iov_xfer(struct vringh *vrh, · line 106
Quoted 10 lines
drivers/vhost/vringh.c +4
-1+return err; + ptr += partlen; + } + done += partlen; len -= partlen; - ptr += partlen; iov->consumed += partlen; iov->iov[iov->i].iov_len -= partlen; iov->iov[iov->i].iov_base += partlen;
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 11/12] vringh: allow vringh_iov_xfer() to skip bytes when ptr is NULL
Quoted 11 lines · Jason Wang
Quoted 9 lines · Stefano Garzarella
In some cases, it may be useful to provide a way to skip a number of bytes in a
vringh_iov.In order to keep
vringh_iovconsistent, let's reusevringh_iov_xfer()logic and skip bytes when the ptr is NULL.Signed-off-by Stefano Garzarella <[email protected]>---
Quoted 7 lines
Quoted 4 lines
I'm not sure if this is the best option, maybe we can add a new function
vringh_iov_skip().Suggestions?
I might be worth to check whether we can convert
vringh_iovto use iov iterator then we can useiov_iterator_advance()here.
Make sense, I'll take a look.
Thanks for the suggestion,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RFC PATCH 12/12 vdpa_sim_blk: implement ramdisk behaviour
In reply to Stefano GarzarellaThe previous implementation wrote only the status of each request. This patch implements a more accurate block device simulator, providing a ramdisk-like behavior.
Also handle VIRTIO_BLK_T_GET_ID request, always answering the "vdpa_blk_sim" string.
--- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 151 +++++++++++++++++++++++---- 1 file changed, 133 insertions(+), 18 deletions(-)
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +133 -18
@@ -7,6 +7,7 @@*/ #include <linux/module.h> +#include <linux/blkdev.h> #include <uapi/linux/virtio_blk.h> #include "vdpa_sim.h"
@@ -24,10 +25,137 @@static struct vdpasim *vdpasim_blk_dev; +static int vdpasim_blk_handle_req (struct vdpasim *vdpasim, +struct vdpasim_virtqueue *vq) +{ +size_t wrote =0 , to_read =0 , to_write =0 ; +struct virtio_blk_outhdr hdr; +uint8_t status; +uint32_t type; +ssize_t bytes; +loff_t offset; +int i, ret; + +vringh_kiov_cleanup (&vq->riov); +vringh_kiov_cleanup (&vq->wiov); + + ret =vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, + &vq->head,GFP_ATOMIC ); +if (ret !=1 ) +return ret; + +for (i =0 ; i < vq->wiov.used; i++) + to_write += vq->wiov.iov[i].iov_len; + to_write -=1 ;/* last byte is the status */ + +for (i =0 ; i < vq->riov.used; i++) + to_read += vq->riov.iov[i].iov_len; + + bytes =vringh_iov_pull_iotlb (&vq->vring, &vq->riov, &hdr,sizeof (hdr)); +if (bytes !=sizeof (hdr)) +return 0 ; + + to_read -= bytes; + + type =le32_to_cpu (hdr.type); + offset =le64_to_cpu (hdr.sector) <<SECTOR_SHIFT ; + status =VIRTIO_BLK_S_OK ; + +switch (type) { +case VIRTIO_BLK_T_IN : +if (offset + to_write >VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ) { +dev_err (&vdpasim->vdpa.dev, +"reading over the capacity - offset: 0x%llx len: 0x%lx\n" , + offset, to_write); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + bytes =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, + vdpasim->buffer + offset, + to_write); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_push_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n" , + bytes, offset, to_write); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + wrote += bytes; +break ; + +case VIRTIO_BLK_T_OUT : +if (offset + to_read >VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ) { +dev_err (&vdpasim->vdpa.dev, +"writing over the capacity - offset: 0x%llx len: 0x%lx\n" , + offset, to_read); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + bytes =vringh_iov_pull_iotlb (&vq->vring, &vq->riov, + vdpasim->buffer + offset, + to_read); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_pull_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n" , + bytes, offset, to_read); + status =VIRTIO_BLK_S_IOERR ; +break ; + } +break ; + +case VIRTIO_BLK_T_GET_ID : { +char id[VIRTIO_BLK_ID_BYTES ] ="vdpa_blk_sim" ; + + bytes =vringh_iov_push_iotlb (&vq->vring, + &vq->wiov, id, +VIRTIO_BLK_ID_BYTES ); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_push_iotlb() error: %ld\n" , bytes); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + wrote += bytes; +break ; + } + +default : +dev_warn (&vdpasim->vdpa.dev, +"Unsupported request type %d\n" , type); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + +/* if VIRTIO_BLK_T_IN or VIRTIO_BLK_T_GET_ID fail, we need to skip +* the remaining bytes to put the status in the last byte +*/ +if (to_write - wrote >0 ) { +vringh_iov_push_iotlb (&vq->vring, &vq->wiov,NULL , + to_write - wrote); + } + +/* last byte is the status */ + bytes =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, &status,1 ); +if (bytes !=1 ) +return 0 ; + + wrote += bytes; + +/* Make sure data is wrote before advancing index */ +smp_wmb (); + +vringh_complete_iotlb (&vq->vring, vq->head, wrote); + +return ret; +} +static void vdpasim_blk_work (struct work_struct *work) { struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); -u8 status =VIRTIO_BLK_S_OK ;int i;spin_lock (&vdpasim->lock);
@@ -41,21 +169,7 @@ static void vdpasim_blk_work(struct work_struct *work)if (!vq->ready)continue ; -while (vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, - &vq->head,GFP_ATOMIC ) >0 ) { - -int write; - - vq->wiov.i = vq->wiov.used -1 ; - write =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, &status,1 ); -if (write <=0 ) -break ; - -/* Make sure data is wrote before advancing index */ -smp_wmb (); - -vringh_complete_iotlb (&vq->vring, vq->head, write); - +while (vdpasim_blk_handle_req (vdpasim, vq) >0 ) {/* Make sure used is visible before rasing the interrupt. */ smp_wmb ();
@@ -67,6 +181,7 @@ static void vdpasim_blk_work(struct work_struct *work) vq->cb (vq->private);local_bh_enable (); } + } out:spin_unlock (&vdpasim->lock);
@@ -84,7 +199,7 @@ static void vdpasim_blk_update_config(struct vdpasim *vdpasim) config->num_queues =cpu_to_vdpasim16 (vdpasim,VDPASIM_BLK_VQ_NUM ); config->min_io_size =cpu_to_vdpasim16 (vdpasim,1 ); config->opt_io_size =cpu_to_vdpasim32 (vdpasim,1 ); - config->blk_size =cpu_to_vdpasim32 (vdpasim,512 ); + config->blk_size =cpu_to_vdpasim32 (vdpasim,SECTOR_SIZE ); }static int __init vdpasim_blk_init (void )
@@ -100,7 +215,7 @@ static int __init vdpasim_blk_init(void) attr.device.update_config = vdpasim_blk_update_config; attr.work_fn = vdpasim_blk_work; - attr.buffer_size =PAGE_SIZE ; + attr.buffer_size =VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ; vdpasim_blk_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_blk_dev)) {
Signature · 1 line
2.26.2
kernel test robot
build botRe: [PATCH RFC 12/12] vdpa_sim_blk: implement ramdisk behaviour
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
Hi Stefano,
[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on linus/master]
[also build test WARNING on v5.10-rc3 next-20201113]
[cannot apply to vhost/linux-next linux/master]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Stefano-Garzarella/vdpa-generalize-vdpa-simulator-and-add-block-device/20201113-215247 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 585e5b17b92dead8a3aca4e3c9876fbca5f7e0ba config: mips-allyesconfig (attached as .config) compiler: mips-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/65adaf3b3bcf9692c072ae2f93c1d784426a224d
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Stefano-Garzarella/vdpa-generalize-vdpa-simulator-and-add-block-device/20201113-215247 git checkout 65adaf3b3bcf9692c072ae2f93c1d784426a224d
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=mips
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
In file included from include/linux/device.h:15,
from include/linux/genhd.h:35,
from include/linux/blkdev.h:8,
from drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:10:
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c: In function 'vdpasim_blk_handle_req':
>> drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:68:5: warning: format '%lx' expects argument of type 'long unsigned int', but argument 4 has type 'size_t' {aka 'unsigned int'} [-Wformat=]
68 | "reading over the capacity - offset: 0x%llx len: 0x%lx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:19:22: note: in definition of macro 'dev_fmt'
19 | #define dev_fmt(fmt) fmt
| ^~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:67:4: note: in expansion of macro 'dev_err'
67 | dev_err(&vdpasim->vdpa.dev,
| ^~~~~~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:68:58: note: format string is defined here
68 | "reading over the capacity - offset: 0x%llx len: 0x%lx\n",
| ~~^
| |
| long unsigned int
| %x
In file included from include/linux/device.h:15,
from include/linux/genhd.h:35,
from include/linux/blkdev.h:8,
from drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:10:
>> drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:79:5: warning: format '%ld' expects argument of type 'long int', but argument 3 has type 'ssize_t' {aka 'int'} [-Wformat=]
79 | "vringh_iov_push_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:19:22: note: in definition of macro 'dev_fmt'
19 | #define dev_fmt(fmt) fmt
| ^~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:78:4: note: in expansion of macro 'dev_err'
78 | dev_err(&vdpasim->vdpa.dev,
| ^~~~~~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:79:39: note: format string is defined here
79 | "vringh_iov_push_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n",
| ~~^
| |
| long int
| %d
In file included from include/linux/device.h:15,
from include/linux/genhd.h:35,
from include/linux/blkdev.h:8,
from drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:10:
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:79:5: warning: format '%lx' expects argument of type 'long unsigned int', but argument 5 has type 'size_t' {aka 'unsigned int'} [-Wformat=]
79 | "vringh_iov_push_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:19:22: note: in definition of macro 'dev_fmt'
19 | #define dev_fmt(fmt) fmt
| ^~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:78:4: note: in expansion of macro 'dev_err'
78 | dev_err(&vdpasim->vdpa.dev,
| ^~~~~~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:79:65: note: format string is defined here
79 | "vringh_iov_push_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n",
| ~~^
| |
| long unsigned int
| %x
In file included from include/linux/device.h:15,
from include/linux/genhd.h:35,
from include/linux/blkdev.h:8,
from drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:10:
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:91:5: warning: format '%lx' expects argument of type 'long unsigned int', but argument 4 has type 'size_t' {aka 'unsigned int'} [-Wformat=]
91 | "writing over the capacity - offset: 0x%llx len: 0x%lx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:19:22: note: in definition of macro 'dev_fmt'
19 | #define dev_fmt(fmt) fmt
| ^~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:90:4: note: in expansion of macro 'dev_err'
90 | dev_err(&vdpasim->vdpa.dev,
| ^~~~~~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:91:58: note: format string is defined here
91 | "writing over the capacity - offset: 0x%llx len: 0x%lx\n",
| ~~^
| |
| long unsigned int
| %x
In file included from include/linux/device.h:15,
from include/linux/genhd.h:35,
from include/linux/blkdev.h:8,
from drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:10:
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:102:5: warning: format '%ld' expects argument of type 'long int', but argument 3 has type 'ssize_t' {aka 'int'} [-Wformat=]
102 | "vringh_iov_pull_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:19:22: note: in definition of macro 'dev_fmt'
19 | #define dev_fmt(fmt) fmt
| ^~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:101:4: note: in expansion of macro 'dev_err'
101 | dev_err(&vdpasim->vdpa.dev,
| ^~~~~~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:102:39: note: format string is defined here
102 | "vringh_iov_pull_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n",
| ~~^
| |
| long int
| %d
In file included from include/linux/device.h:15,
from include/linux/genhd.h:35,
from include/linux/blkdev.h:8,
from drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:10:
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:102:5: warning: format '%lx' expects argument of type 'long unsigned int', but argument 5 has type 'size_t' {aka 'unsigned int'} [-Wformat=]
102 | "vringh_iov_pull_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:19:22: note: in definition of macro 'dev_fmt'
19 | #define dev_fmt(fmt) fmt
| ^~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:101:4: note: in expansion of macro 'dev_err'
101 | dev_err(&vdpasim->vdpa.dev,
| ^~~~~~~
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:102:65: note: format string is defined here
102 | "vringh_iov_pull_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n",
| ~~^
| |
| long unsigned int
| %x
In file included from include/linux/device.h:15,
from include/linux/genhd.h:35,
from include/linux/blkdev.h:8,
from drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:10:
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c:117:5: warning: format '%ld' expects argument of type 'long int', but argument 3 has type 'ssize_t' {aka 'int'} [-Wformat=]
117 | "vringh_iov_push_iotlb() error: %ld\n", bytes);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/dev_printk.h:19:22: note: in definition of macro 'dev_fmt'
19 | #define dev_fmt(fmt) fmt
| ^~~
vim +68 drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
27
28 static int vdpasim_blk_handle_req(struct vdpasim *vdpasim,
29 struct vdpasim_virtqueue *vq)
30 {
31 size_t wrote = 0, to_read = 0, to_write = 0;
32 struct virtio_blk_outhdr hdr;
33 uint8_t status;
34 uint32_t type;
35 ssize_t bytes;
36 loff_t offset;
37 int i, ret;
38
39 vringh_kiov_cleanup(&vq->riov);
40 vringh_kiov_cleanup(&vq->wiov);
41
42 ret = vringh_getdesc_iotlb(&vq->vring, &vq->riov, &vq->wiov,
43 &vq->head, GFP_ATOMIC);
44 if (ret != 1)
45 return ret;
46
47 for (i = 0; i < vq->wiov.used; i++)
48 to_write += vq->wiov.iov[i].iov_len;
49 to_write -= 1; /* last byte is the status */
50
51 for (i = 0; i < vq->riov.used; i++)
52 to_read += vq->riov.iov[i].iov_len;
53
54 bytes = vringh_iov_pull_iotlb(&vq->vring, &vq->riov, &hdr, sizeof(hdr));
55 if (bytes != sizeof(hdr))
56 return 0;
57
58 to_read -= bytes;
59
60 type = le32_to_cpu(hdr.type);
61 offset = le64_to_cpu(hdr.sector) << SECTOR_SHIFT;
62 status = VIRTIO_BLK_S_OK;
63
64 switch (type) {
65 case VIRTIO_BLK_T_IN:
66 if (offset + to_write > VDPASIM_BLK_CAPACITY << SECTOR_SHIFT) {
67 dev_err(&vdpasim->vdpa.dev,
> 68 "reading over the capacity - offset: 0x%llx len: 0x%lx\n",
69 offset, to_write);
70 status = VIRTIO_BLK_S_IOERR;
71 break;
72 }
73
74 bytes = vringh_iov_push_iotlb(&vq->vring, &vq->wiov,
75 vdpasim->buffer + offset,
76 to_write);
77 if (bytes < 0) {
78 dev_err(&vdpasim->vdpa.dev,
> 79 "vringh_iov_push_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n",
80 bytes, offset, to_write);
81 status = VIRTIO_BLK_S_IOERR;
82 break;
83 }
84
85 wrote += bytes;
86 break;
87
88 case VIRTIO_BLK_T_OUT:
89 if (offset + to_read > VDPASIM_BLK_CAPACITY << SECTOR_SHIFT) {
90 dev_err(&vdpasim->vdpa.dev,
91 "writing over the capacity - offset: 0x%llx len: 0x%lx\n",
92 offset, to_read);
93 status = VIRTIO_BLK_S_IOERR;
94 break;
95 }
96
97 bytes = vringh_iov_pull_iotlb(&vq->vring, &vq->riov,
98 vdpasim->buffer + offset,
99 to_read); 100 if (bytes < 0) {
101 dev_err(&vdpasim->vdpa.dev,
102 "vringh_iov_pull_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n",
103 bytes, offset, to_read);
104 status = VIRTIO_BLK_S_IOERR;
105 break;
106 }
107 break;
108
109 case VIRTIO_BLK_T_GET_ID: {
110 char id[VIRTIO_BLK_ID_BYTES] = "vdpa_blk_sim";
111
112 bytes = vringh_iov_push_iotlb(&vq->vring,
113 &vq->wiov, id,
114 VIRTIO_BLK_ID_BYTES);
115 if (bytes < 0) {
116 dev_err(&vdpasim->vdpa.dev,
117 "vringh_iov_push_iotlb() error: %ld\n", bytes);
118 status = VIRTIO_BLK_S_IOERR;
119 break;
120 }
121
122 wrote += bytes;
123 break;
124 }
125
126 default:
127 dev_warn(&vdpasim->vdpa.dev,
128 "Unsupported request type %d\n", type);
129 status = VIRTIO_BLK_S_IOERR;
130 break;
131 }
132
133 /* if VIRTIO_BLK_T_IN or VIRTIO_BLK_T_GET_ID fail, we need to skip
134 * the remaining bytes to put the status in the last byte
135 */
136 if (to_write - wrote > 0) {
137 vringh_iov_push_iotlb(&vq->vring, &vq->wiov, NULL,
138 to_write - wrote);
139 }
140
141 /* last byte is the status */
142 bytes = vringh_iov_push_iotlb(&vq->vring, &vq->wiov, &status, 1);
143 if (bytes != 1)
144 return 0;
145
146 wrote += bytes;
147
148 /* Make sure data is wrote before advancing index */
149 smp_wmb();
150
151 vringh_complete_iotlb(&vq->vring, vq->head, wrote);
152
153 return ret;
154 }
155
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [PATCH RFC 12/12] vdpa_sim_blk: implement ramdisk behaviour
In reply to Stefano GarzarellaQuoted 6 lines · Stefano Garzarella
The previous implementation wrote only the status of each request. This patch implements a more accurate block device simulator, providing a ramdisk-like behavior.
Also handle
VIRTIO_BLK_T_GET_IDrequest, always answering the "vdpa_blk_sim" string.
Let's use a separate patch for this.
Quoted 25 lines
Signed-off-by Stefano Garzarella <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 151 +++++++++++++++++++++++---- 1 file changed, 133 insertions(+), 18 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +4
-0@@ -7,6 +7,7 @@*/ #include <linux/module.h> +#include <linux/blkdev.h> #include <uapi/linux/virtio_blk.h> #include "vdpa_sim.h" @@ -24,10 +25,137 @@static struct vdpasim *vdpasim_blk_dev; +static int vdpasim_blk_handle_req (struct vdpasim *vdpasim, +struct vdpasim_virtqueue *vq) +{
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 40
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +10
-0+size_t wrote =0 , to_read =0 , to_write =0 ; +struct virtio_blk_outhdr hdr; +uint8_t status; +uint32_t type; +ssize_t bytes; +loff_t offset; +int i, ret; + +vringh_kiov_cleanup (&vq->riov); +vringh_kiov_cleanup (&vq->wiov);
It looks to me we should do those after vringh_get_desc_iotlb()? See comment above vringh_getdesc_kern().
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 48
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +8
-0+ + ret =vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, + &vq->head,GFP_ATOMIC ); +if (ret !=1 ) +return ret; + +for (i =0 ; i < vq->wiov.used; i++) + to_write += vq->wiov.iov[i].iov_len;
It's better to introduce a helper for this (or consider to use iov iterator).
Quoted 52 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +52
-0+ to_write -=1 ;/* last byte is the status */ + +for (i =0 ; i < vq->riov.used; i++) + to_read += vq->riov.iov[i].iov_len; + + bytes =vringh_iov_pull_iotlb (&vq->vring, &vq->riov, &hdr,sizeof (hdr)); +if (bytes !=sizeof (hdr)) +return 0 ; + + to_read -= bytes; + + type =le32_to_cpu (hdr.type); + offset =le64_to_cpu (hdr.sector) <<SECTOR_SHIFT ; + status =VIRTIO_BLK_S_OK ; + +switch (type) { +case VIRTIO_BLK_T_IN : +if (offset + to_write >VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ) { +dev_err (&vdpasim->vdpa.dev, +"reading over the capacity - offset: 0x%llx len: 0x%lx\n" , + offset, to_write); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + bytes =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, + vdpasim->buffer + offset, + to_write); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_push_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n" , + bytes, offset, to_write); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + wrote += bytes; +break ; + +case VIRTIO_BLK_T_OUT : +if (offset + to_read >VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ) { +dev_err (&vdpasim->vdpa.dev, +"writing over the capacity - offset: 0x%llx len: 0x%lx\n" , + offset, to_read); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + bytes =vringh_iov_pull_iotlb (&vq->vring, &vq->riov, + vdpasim->buffer + offset, + to_read); +if (bytes <0 ) {
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 110
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +10
-0+dev_err (&vdpasim->vdpa.dev, +"vringh_iov_pull_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n" , + bytes, offset, to_read); + status =VIRTIO_BLK_S_IOERR ; +break ; + } +break ; + +case VIRTIO_BLK_T_GET_ID : { +char id[VIRTIO_BLK_ID_BYTES ] ="vdpa_blk_sim" ;
Let's use a global static one?
Quoted 76 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +46
-16+ + bytes =vringh_iov_push_iotlb (&vq->vring, + &vq->wiov, id, +VIRTIO_BLK_ID_BYTES ); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_push_iotlb() error: %ld\n" , bytes); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + wrote += bytes; +break ; + } + +default : +dev_warn (&vdpasim->vdpa.dev, +"Unsupported request type %d\n" , type); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + +/* if VIRTIO_BLK_T_IN or VIRTIO_BLK_T_GET_ID fail, we need to skip +* the remaining bytes to put the status in the last byte +*/ +if (to_write - wrote >0 ) { +vringh_iov_push_iotlb (&vq->vring, &vq->wiov,NULL , + to_write - wrote); + } + +/* last byte is the status */ + bytes =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, &status,1 ); +if (bytes !=1 ) +return 0 ; + + wrote += bytes; + +/* Make sure data is wrote before advancing index */ +smp_wmb (); + +vringh_complete_iotlb (&vq->vring, vq->head, wrote); + +return ret; +} +static void vdpasim_blk_work (struct work_struct *work) { struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); -u8 status =VIRTIO_BLK_S_OK ;int i;spin_lock (&vdpasim->lock);@@ -41,21 +169,7 @@ static void vdpasim_blk_work(struct work_struct *work)if (!vq->ready)continue ; -while (vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, - &vq->head,GFP_ATOMIC ) >0 ) { - -int write; - - vq->wiov.i = vq->wiov.used -1 ; - write =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, &status,1 ); -if (write <=0 ) -break ; - -/* Make sure data is wrote before advancing index */ -smp_wmb (); - -vringh_complete_iotlb (&vq->vring, vq->head, write); - +while (vdpasim_blk_handle_req (vdpasim, vq) >0 ) {/* Make sure used is visible before rasing the interrupt. */ smp_wmb ();
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c static void vdpasim_blk_work(struct work_struct *work) · line 184
Quoted 4 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +1
-0@@ -67,6 +181,7 @@ static void vdpasim_blk_work(struct work_struct *work) vq->cb (vq->private);local_bh_enable (); } +
Unnecessary change.
Thanks
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 12/12] vdpa_sim_blk: implement ramdisk behaviour
Quoted 6 lines · Jason Wang
Quoted 4 lines · Stefano Garzarella
The previous implementation wrote only the status of each request. This patch implements a more accurate block device simulator, providing a ramdisk-like behavior.
Quoted 6 lines
Quoted 2 lines
Also handle
VIRTIO_BLK_T_GET_IDrequest, always answering the "vdpa_blk_sim" string.Let's use a separate patch for this.
Okay, I'll do.
Quoted 27 lines
Quoted 27 lines
Signed-off-by Stefano Garzarella <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 151 +++++++++++++++++++++++---- 1 file changed, 133 insertions(+), 18 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +9
-0@@ -7,6 +7,7 @@*/ #include <linux/module.h> +#include <linux/blkdev.h> #include <uapi/linux/virtio_blk.h> #include "vdpa_sim.h" @@ -24,10 +25,137 @@static struct vdpasim *vdpasim_blk_dev; +static int vdpasim_blk_handle_req (struct vdpasim *vdpasim, +struct vdpasim_virtqueue *vq) +{ +size_t wrote =0 , to_read =0 , to_write =0 ; +struct virtio_blk_outhdr hdr; +uint8_t status; +uint32_t type; +ssize_t bytes;
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 40
Quoted 5 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +5
-0+loff_t offset; +int i, ret; + +vringh_kiov_cleanup (&vq->riov); +vringh_kiov_cleanup (&vq->wiov);It looks to me we should do those after
vringh_get_desc_iotlb()? See comment abovevringh_getdesc_kern().
Do you mean after the last vringh_iov_push_iotlb()?
Because vringh_kiov_cleanup() will free the allocated iov[].
Quoted 13 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 48
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +8
-0+ + ret =vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, + &vq->head,GFP_ATOMIC ); +if (ret !=1 ) +return ret; + +for (i =0 ; i < vq->wiov.used; i++) + to_write += vq->wiov.iov[i].iov_len;It's better to introduce a helper for this (or consider to use iov iterator).
Okay, I'll try to find the best solution.
Quoted 61 lines
Quoted 61 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +57
-0+ to_write -=1 ;/* last byte is the status */ + +for (i =0 ; i < vq->riov.used; i++) + to_read += vq->riov.iov[i].iov_len; + + bytes =vringh_iov_pull_iotlb (&vq->vring, &vq->riov, &hdr,sizeof (hdr)); +if (bytes !=sizeof (hdr)) +return 0 ; + + to_read -= bytes; + + type =le32_to_cpu (hdr.type); + offset =le64_to_cpu (hdr.sector) <<SECTOR_SHIFT ; + status =VIRTIO_BLK_S_OK ; + +switch (type) { +case VIRTIO_BLK_T_IN : +if (offset + to_write >VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ) { +dev_err (&vdpasim->vdpa.dev, + "reading over the capacity - offset: 0x%llx len: 0x%lx\n", + offset, to_write); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + bytes =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, + vdpasim->buffer + offset, + to_write); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_push_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n" , + bytes, offset, to_write); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + wrote += bytes; +break ; + +case VIRTIO_BLK_T_OUT : +if (offset + to_read >VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ) { +dev_err (&vdpasim->vdpa.dev, +"writing over the capacity - offset: 0x%llx len: 0x%lx\n" , + offset, to_read); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + bytes =vringh_iov_pull_iotlb (&vq->vring, &vq->riov, + vdpasim->buffer + offset, + to_read); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_pull_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n" , + bytes, offset, to_read); + status =VIRTIO_BLK_S_IOERR ; +break ;
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 110
Quoted 5 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +5
-0+ } +break ; + +case VIRTIO_BLK_T_GET_ID : { +char id[VIRTIO_BLK_ID_BYTES ] ="vdpa_blk_sim" ;Let's use a global static one?
I'll do.
Quoted 75 lines
Quoted 75 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +45
-12+ + bytes =vringh_iov_push_iotlb (&vq->vring, + &vq->wiov, id, +VIRTIO_BLK_ID_BYTES ); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_push_iotlb() error: %ld\n" , bytes); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + wrote += bytes; +break ; + } + +default : +dev_warn (&vdpasim->vdpa.dev, +"Unsupported request type %d\n" , type); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + +/* if VIRTIO_BLK_T_IN or VIRTIO_BLK_T_GET_ID fail, we need to skip +* the remaining bytes to put the status in the last byte +*/ +if (to_write - wrote >0 ) { +vringh_iov_push_iotlb (&vq->vring, &vq->wiov,NULL , + to_write - wrote); + } + +/* last byte is the status */ + bytes =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, &status,1 ); +if (bytes !=1 ) +return 0 ; + + wrote += bytes; + +/* Make sure data is wrote before advancing index */ +smp_wmb (); + +vringh_complete_iotlb (&vq->vring, vq->head, wrote); + +return ret; +} +static void vdpasim_blk_work (struct work_struct *work) { struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); -u8 status =VIRTIO_BLK_S_OK ;int i;spin_lock (&vdpasim->lock);@@ -41,21 +169,7 @@ static void vdpasim_blk_work(struct work_struct *work)if (!vq->ready)continue ; -while (vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, - &vq->head,GFP_ATOMIC ) >0 ) { - -int write; - - vq->wiov.i = vq->wiov.used -1 ; - write =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, &status,1 ); -if (write <=0 ) -break ; - -/* Make sure data is wrote before advancing index */
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +1
-4-smp_wmb (); - -vringh_complete_iotlb (&vq->vring, vq->head, write); - +while (vdpasim_blk_handle_req (vdpasim, vq) >0 ) {/* Make sure used is visible before rasing the interrupt. */ smp_wmb ();
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c static void vdpasim_blk_work(struct work_struct *work) · line 184
Quoted 5 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +1
-0@@ -67,6 +181,7 @@ static void vdpasim_blk_work(struct work_struct *work) vq->cb (vq->private);local_bh_enable (); } +Unnecessary change.
Removed.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerRe: [PATCH RFC 12/12] vdpa_sim_blk: implement ramdisk behaviour
In reply to Stefano GarzarellaQuoted 6 lines · Stefano Garzarella
The previous implementation wrote only the status of each request. This patch implements a more accurate block device simulator, providing a ramdisk-like behavior.
Also handle
VIRTIO_BLK_T_GET_IDrequest, always answering the "vdpa_blk_sim" string.
Maybe an ioctl to specify the id makes more sense.
Quoted 207 lines
Signed-off-by Stefano Garzarella <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 151 +++++++++++++++++++++++---- 1 file changed, 133 insertions(+), 18 deletions(-)drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +133
-18@@ -7,6 +7,7 @@*/ #include <linux/module.h> +#include <linux/blkdev.h> #include <uapi/linux/virtio_blk.h> #include "vdpa_sim.h" @@ -24,10 +25,137 @@static struct vdpasim *vdpasim_blk_dev; +static int vdpasim_blk_handle_req (struct vdpasim *vdpasim, +struct vdpasim_virtqueue *vq) +{ +size_t wrote =0 , to_read =0 , to_write =0 ; +struct virtio_blk_outhdr hdr; +uint8_t status; +uint32_t type; +ssize_t bytes; +loff_t offset; +int i, ret; + +vringh_kiov_cleanup (&vq->riov); +vringh_kiov_cleanup (&vq->wiov); + + ret =vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, + &vq->head,GFP_ATOMIC ); +if (ret !=1 ) +return ret; + +for (i =0 ; i < vq->wiov.used; i++) + to_write += vq->wiov.iov[i].iov_len; + to_write -=1 ;/* last byte is the status */ + +for (i =0 ; i < vq->riov.used; i++) + to_read += vq->riov.iov[i].iov_len; + + bytes =vringh_iov_pull_iotlb (&vq->vring, &vq->riov, &hdr,sizeof (hdr)); +if (bytes !=sizeof (hdr)) +return 0 ; + + to_read -= bytes; + + type =le32_to_cpu (hdr.type); + offset =le64_to_cpu (hdr.sector) <<SECTOR_SHIFT ; + status =VIRTIO_BLK_S_OK ; + +switch (type) { +case VIRTIO_BLK_T_IN : +if (offset + to_write >VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ) { +dev_err (&vdpasim->vdpa.dev, +"reading over the capacity - offset: 0x%llx len: 0x%lx\n" , + offset, to_write); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + bytes =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, + vdpasim->buffer + offset, + to_write); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_push_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n" , + bytes, offset, to_write); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + wrote += bytes; +break ; + +case VIRTIO_BLK_T_OUT : +if (offset + to_read >VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ) { +dev_err (&vdpasim->vdpa.dev, +"writing over the capacity - offset: 0x%llx len: 0x%lx\n" , + offset, to_read); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + bytes =vringh_iov_pull_iotlb (&vq->vring, &vq->riov, + vdpasim->buffer + offset, + to_read); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_pull_iotlb() error: %ld offset: 0x%llx len: 0x%lx\n" , + bytes, offset, to_read); + status =VIRTIO_BLK_S_IOERR ; +break ; + } +break ; + +case VIRTIO_BLK_T_GET_ID : { +char id[VIRTIO_BLK_ID_BYTES ] ="vdpa_blk_sim" ; + + bytes =vringh_iov_push_iotlb (&vq->vring, + &vq->wiov, id, +VIRTIO_BLK_ID_BYTES ); +if (bytes <0 ) { +dev_err (&vdpasim->vdpa.dev, +"vringh_iov_push_iotlb() error: %ld\n" , bytes); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + + wrote += bytes; +break ; + } + +default : +dev_warn (&vdpasim->vdpa.dev, +"Unsupported request type %d\n" , type); + status =VIRTIO_BLK_S_IOERR ; +break ; + } + +/* if VIRTIO_BLK_T_IN or VIRTIO_BLK_T_GET_ID fail, we need to skip +* the remaining bytes to put the status in the last byte +*/ +if (to_write - wrote >0 ) { +vringh_iov_push_iotlb (&vq->vring, &vq->wiov,NULL , + to_write - wrote); + } + +/* last byte is the status */ + bytes =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, &status,1 ); +if (bytes !=1 ) +return 0 ; + + wrote += bytes; + +/* Make sure data is wrote before advancing index */ +smp_wmb (); + +vringh_complete_iotlb (&vq->vring, vq->head, wrote); + +return ret; +} +static void vdpasim_blk_work (struct work_struct *work) { struct vdpasim *vdpasim = container_of(work, struct vdpasim, work); -u8 status =VIRTIO_BLK_S_OK ;int i;spin_lock (&vdpasim->lock);@@ -41,21 +169,7 @@ static void vdpasim_blk_work(struct work_struct *work)if (!vq->ready)continue ; -while (vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, - &vq->head,GFP_ATOMIC ) >0 ) { - -int write; - - vq->wiov.i = vq->wiov.used -1 ; - write =vringh_iov_push_iotlb (&vq->vring, &vq->wiov, &status,1 ); -if (write <=0 ) -break ; - -/* Make sure data is wrote before advancing index */ -smp_wmb (); - -vringh_complete_iotlb (&vq->vring, vq->head, write); - +while (vdpasim_blk_handle_req (vdpasim, vq) >0 ) {/* Make sure used is visible before rasing the interrupt. */ smp_wmb ();@@ -67,6 +181,7 @@ static void vdpasim_blk_work(struct work_struct *work) vq->cb (vq->private);local_bh_enable (); } + } out:spin_unlock (&vdpasim->lock);@@ -84,7 +199,7 @@ static void vdpasim_blk_update_config(struct vdpasim *vdpasim) config->num_queues =cpu_to_vdpasim16 (vdpasim,VDPASIM_BLK_VQ_NUM ); config->min_io_size =cpu_to_vdpasim16 (vdpasim,1 ); config->opt_io_size =cpu_to_vdpasim32 (vdpasim,1 ); - config->blk_size =cpu_to_vdpasim32 (vdpasim,512 ); + config->blk_size =cpu_to_vdpasim32 (vdpasim,SECTOR_SIZE ); }static int __init vdpasim_blk_init (void )@@ -100,7 +215,7 @@ static int __init vdpasim_blk_init(void) attr.device.update_config = vdpasim_blk_update_config; attr.work_fn = vdpasim_blk_work; - attr.buffer_size =PAGE_SIZE ; + attr.buffer_size =VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ; vdpasim_blk_dev =vdpasim_create (&attr);if (IS_ERR (vdpasim_blk_dev)) {Signature · 1 line
2.26.2
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 12/12] vdpa_sim_blk: implement ramdisk behaviour
Quoted 9 lines · Michael S. Tsirkin
Quoted 6 lines · Stefano Garzarella
The previous implementation wrote only the status of each request. This patch implements a more accurate block device simulator, providing a ramdisk-like behavior.
Also handle
VIRTIO_BLK_T_GET_IDrequest, always answering the "vdpa_blk_sim" string.Maybe an ioctl to specify the id makes more sense.
I agree that make sense to make it configurable from the user, but I'm not sure an ioctl() is the best interface with this device simulator.
Maybe we can use a module parameter as in the net simulator or even better using the new vdpa management tool recently proposed (I need to check better how we can extend it).
What do you think?
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 12/12] vdpa_sim_blk: implement ramdisk behaviour
In reply to Stefano GarzarellaQuoted 13 lines · Stefano Garzarella
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +1
-0@@ -7,6 +7,7 @@*/ #include <linux/module.h> +#include <linux/blkdev.h> #include <uapi/linux/virtio_blk.h> #include "vdpa_sim.h"
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 29
Quoted 5 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +2
-0@@ -24,10 +25,137 @@static struct vdpasim *vdpasim_blk_dev; +static int vdpasim_blk_handle_req (struct vdpasim *vdpasim, +struct vdpasim_virtqueue *vq)
This function has a non-standard int return value. Please document it.
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +10
-0+{ +size_t wrote =0 , to_read =0 , to_write =0 ; +struct virtio_blk_outhdr hdr; +uint8_t status; +uint32_t type; +ssize_t bytes; +loff_t offset; +int i, ret; + +vringh_kiov_cleanup (&vq->riov);
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 49
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +10
-0+vringh_kiov_cleanup (&vq->wiov); + + ret =vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, + &vq->head,GFP_ATOMIC ); +if (ret !=1 ) +return ret; + +for (i =0 ; i < vq->wiov.used; i++) + to_write += vq->wiov.iov[i].iov_len; + to_write -=1 ;/* last byte is the status */
What if vq->wiov.used == 0?
Quoted 7 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +7
-0+ +for (i =0 ; i < vq->riov.used; i++) + to_read += vq->riov.iov[i].iov_len; + + bytes =vringh_iov_pull_iotlb (&vq->vring, &vq->riov, &hdr,sizeof (hdr)); +if (bytes !=sizeof (hdr)) +return 0 ;
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 66
Quoted 10 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +10
-0+ + to_read -= bytes; + + type =le32_to_cpu (hdr.type); + offset =le64_to_cpu (hdr.sector) <<SECTOR_SHIFT ; + status =VIRTIO_BLK_S_OK ; + +switch (type) { +case VIRTIO_BLK_T_IN : +if (offset + to_write >VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ) {
Integer overflow is not handled.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH RFC 12/12] vdpa_sim_blk: implement ramdisk behaviour
Quoted 15 lines · Stefan Hajnoczi
Quoted 14 lines · Stefano Garzarella
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +1
-0@@ -7,6 +7,7 @@*/ #include <linux/module.h> +#include <linux/blkdev.h> #include <uapi/linux/virtio_blk.h> #include "vdpa_sim.h" @@ -24,10 +25,137 @@
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 29
Quoted 4 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +2
-0static struct vdpasim *vdpasim_blk_dev; +static int vdpasim_blk_handle_req (struct vdpasim *vdpasim, +struct vdpasim_virtqueue *vq)This function has a non-standard int return value. Please document it.
Yes, I'll do.
Quoted 15 lines
Quoted 15 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +14
-0+{ +size_t wrote =0 , to_read =0 , to_write =0 ; +struct virtio_blk_outhdr hdr; +uint8_t status; +uint32_t type; +ssize_t bytes; +loff_t offset; +int i, ret; + +vringh_kiov_cleanup (&vq->riov); +vringh_kiov_cleanup (&vq->wiov); + + ret =vringh_getdesc_iotlb (&vq->vring, &vq->riov, &vq->wiov, + &vq->head,GFP_ATOMIC );
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 49
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +6
-0+if (ret !=1 ) +return ret; + +for (i =0 ; i < vq->wiov.used; i++) + to_write += vq->wiov.iov[i].iov_len; + to_write -=1 ;/* last byte is the status */ What if
vq->wiov.used== 0?
Right, we should discard the descriptor.
Quoted 12 lines
Quoted 12 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +11
-0+ +for (i =0 ; i < vq->riov.used; i++) + to_read += vq->riov.iov[i].iov_len; + + bytes =vringh_iov_pull_iotlb (&vq->vring, &vq->riov, &hdr,sizeof (hdr)); +if (bytes !=sizeof (hdr)) +return 0 ; + + to_read -= bytes; + + type =le32_to_cpu (hdr.type);
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c line 66
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c +6
-0+ offset =le64_to_cpu (hdr.sector) <<SECTOR_SHIFT ; + status =VIRTIO_BLK_S_OK ; + +switch (type) { +case VIRTIO_BLK_T_IN : +if (offset + to_write >VDPASIM_BLK_CAPACITY <<SECTOR_SHIFT ) {Integer overflow is not handled.
I'll fix.
Thanks,
Stefano
Quoted 20 lines · Stefano Garzarella
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.This series moves the network device simulator in a new module (
vdpa_sim_net) and leaves the generic functions in thevdpa_simcore module, allowing the possibility to add new vDPA device simulators. Then we added a newvdpa_sim_blkmodule to simulate a block device.I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer()to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modifyvringh_iov_xfer().As Max reported, I'm also seeing errors with
vdpa_sim_blkrelated to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:vringh: Failed to access avail idx at
00000000e8deb2cc
vringh: Failed to read head: idx 6289 address00000000e1ad1d50
vringh: Failed to get flags at000000006635d7a3
Quoted 5 lines
virtio_vdpa vdpa0: vringh_iov_push_iotlb() error: -14 offset: 0x2840000 len: 0x20000 virtio_vdpa vdpa0: vringh_iov_pull_iotlb() error: -14 offset: 0x58ee000 len: 0x3000These errors should all be related to the fact that
iotlb_translate()fails with-EINVAL, so it seems that we miss some mapping.
Is this only reproducible when there's multiple co-current accessing of IOTLB? If yes, it's probably a hint that some kind of synchronization is still missed somewhere.
It might be useful to log the dma_map/unmp in both virtio_ring and vringh to see who is missing the map.
Thanks
Quoted 31 lines
I'll debug more carefully, in the meantime can you give a first review?
Thanks,
StefanoMax Gurtovoy (4):
vhost-vdpa: add support for vDPA blk devices
vdpa: split vdpasim to core and net modules
vdpa_sim: remove hard-coded virtq count
vdpa: add vdpa simulator for block deviceStefano Garzarella (8):
vdpa_sim: remove the limit of IOTLB entries
vdpa_sim: add structvdpasim_deviceto store device properties
vdpa_sim: move config management outside of the core
vdpa_sim: use kvmalloc to allocatevdpasim->buffer
vdpa_sim: makevdpasim->buffersize configurable
vdpa_sim: split vdpasim_virtqueue's iov field in riov and wiov
vringh: allowvringh_iov_xfer()to skip bytes when ptr is NULL
vdpa_sim_blk: implement ramdisk behaviourdrivers/vdpa/vdpa_sim/vdpa_sim.h | 117 +++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.c | 283 +++++---------------------- drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 251 ++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 172 ++++++++++++++++ drivers/vhost/vdpa.c | 11 +- drivers/vhost/vringh.c | 16 +- drivers/vdpa/Kconfig | 16 +- drivers/vdpa/vdpa_sim/Makefile | 2 + 8 files changed, 628 insertions(+), 240 deletions(-)
Quoted 4 lines
create mode 100644
drivers/vdpa/vdpa_sim/vdpa_sim.hcreate mode 100644drivers/vdpa/vdpa_sim/vdpa_sim_blk.ccreate mode 100644drivers/vdpa/vdpa_sim/vdpa_sim_net.c
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 22 lines · Jason Wang
Quoted 20 lines · Stefano Garzarella
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.This series moves the network device simulator in a new module (
vdpa_sim_net) and leaves the generic functions in thevdpa_simcore module, allowing the possibility to add new vDPA device simulators. Then we added a newvdpa_sim_blkmodule to simulate a block device.I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer()to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modifyvringh_iov_xfer().As Max reported, I'm also seeing errors with
vdpa_sim_blkrelated to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:vringh: Failed to access avail idx at
00000000e8deb2cc
vringh: Failed to read head: idx 6289 address00000000e1ad1d50
vringh: Failed to get flags at000000006635d7a3
Quoted 8 lines
Quoted 5 lines
virtio_vdpavdpa0:vringh_iov_push_iotlb()error: -14 offset:0x2840000len:0x20000virtio_vdpavdpa0:vringh_iov_pull_iotlb()error: -14 offset:0x58ee000len:0x3000These errors should all be related to the fact that
iotlb_translate()fails with-EINVAL, so it seems that we miss some mapping.Is this only reproducible when there's multiple co-current accessing of IOTLB? If yes, it's probably a hint that some kind of synchronization is still missed somewhere.
Yeah, maybe this is the case where virtio_ring and vringh use IOTLB concorrentetively.
Quoted 2 lines
It might be useful to log the dma_map/unmp in both
virtio_ringand vringh to see who is missing the map.
I'll try.
Thanks for the hints,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 27 lines · Jason Wang
Quoted 25 lines · Stefano Garzarella
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.This series moves the network device simulator in a new module (
vdpa_sim_net) and leaves the generic functions in thevdpa_simcore module, allowing the possibility to add new vDPA device simulators. Then we added a newvdpa_sim_blkmodule to simulate a block device.I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer()to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modifyvringh_iov_xfer().As Max reported, I'm also seeing errors with
vdpa_sim_blkrelated to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:vringh: Failed to access avail idx at
00000000e8deb2cc
vringh: Failed to read head: idx 6289 address00000000e1ad1d50
vringh: Failed to get flags at000000006635d7a3
virtio_vdpavdpa0:vringh_iov_push_iotlb()error: -14 offset:
0x2840000len:0x20000
virtio_vdpavdpa0:vringh_iov_pull_iotlb()error: -14 offset:
0x58ee000len:0x3000
Quoted 8 lines
Quoted 2 lines
These errors should all be related to the fact that
iotlb_translate()fails with-EINVAL, so it seems that we miss some mapping.Is this only reproducible when there's multiple co-current accessing of IOTLB? If yes, it's probably a hint that some kind of synchronization is still missed somewhere.
It might be useful to log the dma_map/unmp in both
virtio_ringand vringh to see who is missing the map.
Just an update about these issues with vdpa-sim-blk.
I've been focusing a little bit on these failures over the last few days and have found two issues related to the IOTLB/IOMMU:
1. Some requests coming from the block layer fills the SG list with multiple buffers that had the same physical address. This happens for example while using 'mkfs', at some points multiple sectors are zeroed so multiple SG elements point to the same physical page that is zeroed. Since we are using vhost_iotlb_del_range() in the vdpasim_unmap_page(), this removes all the overlapped ranges. I fixed removing a single map in vdpasim_unmap_page(), but has an alternative we can implement some kind of reference counts.
2. There was a race between dma_map/unmap and the worker thread, since both are accessing the IOMMU. Taking the iommu_lock while using vhost_iotlb_* API in the worker thread fixes the "vringh: Failed to *" issues.
Whit these issues fixed the vdpa-blk simulator seems to work well. I'll send the patches next week or after the break.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 32 lines · Stefano Garzarella
Quoted 31 lines · Jason Wang
Quoted 20 lines · Stefano Garzarella
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.This series moves the network device simulator in a new module (
vdpa_sim_net) and leaves the generic functions in thevdpa_simcore module, allowing the possibility to add new vDPA device simulators. Then we added a newvdpa_sim_blkmodule to simulate a block device.I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer()to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modifyvringh_iov_xfer().As Max reported, I'm also seeing errors with
vdpa_sim_blkrelated to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:vringh: Failed to access avail idx at
00000000e8deb2cc
vringh: Failed to read head: idx 6289 address00000000e1ad1d50
vringh: Failed to get flags at000000006635d7a3Quoted 5 lines
virtio_vdpavdpa0:vringh_iov_push_iotlb()error: -14 offset:0x2840000len:0x20000virtio_vdpavdpa0:vringh_iov_pull_iotlb()error: -14 offset:0x58ee000len:0x3000These errors should all be related to the fact that
iotlb_translate()fails with-EINVAL, so it seems that we miss some mapping.Is this only reproducible when there's multiple co-current accessing of IOTLB? If yes, it's probably a hint that some kind of synchronization is still missed somewhere.
Quoted 8 lines
Quoted 2 lines
It might be useful to log the dma_map/unmp in both
virtio_ringand vringh to see who is missing the map.Just an update about these issues with vdpa-sim-blk.
I've been focusing a little bit on these failures over the last few days and have found two issues related to the IOTLB/IOMMU:1. Some requests coming from the block layer fills the SG list with multiple buffers that had the same physical address. This happens for example while using 'mkfs', at some points multiple sectors are zeroed so multiple SG elements point to the same physical page that is zeroed. Since we are using
vhost_iotlb_del_range()in thevdpasim_unmap_page(), this removes all the overlapped ranges. I fixed removing a single map invdpasim_unmap_page(), but has an alternative we can implement some kind of reference counts.
I think we need to do what hardware do. So using refcount is probably not a good ida.
Quoted 5 lines
2. There was a race between dma_map/unmap and the worker thread, since both are accessing the IOMMU. Taking the
iommu_lockwhile using vhost_iotlb_* API in the worker thread fixes the "vringh: Failed to *" issues.Whit these issues fixed the vdpa-blk simulator seems to work well. I'll send the patches next week or after the break.
Good to know this.
Thanks
Quoted 4 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 37 lines · Jason Wang
Quoted 35 lines · Stefano Garzarella
Quoted 33 lines · Jason Wang
Quoted 20 lines · Stefano Garzarella
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.This series moves the network device simulator in a new module (
vdpa_sim_net) and leaves the generic functions in thevdpa_simcore module, allowing the possibility to add new vDPA device simulators. Then we added a newvdpa_sim_blkmodule to simulate a block device.I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer()to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modifyvringh_iov_xfer().As Max reported, I'm also seeing errors with
vdpa_sim_blkrelated to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:vringh: Failed to access avail idx at
00000000e8deb2cc
vringh: Failed to read head: idx 6289 address00000000e1ad1d50
vringh: Failed to get flags at000000006635d7a3Quoted 5 lines
virtio_vdpavdpa0:vringh_iov_push_iotlb()error: -14 offset:0x2840000len:0x20000virtio_vdpavdpa0:vringh_iov_pull_iotlb()error: -14 offset:0x58ee000len:0x3000These errors should all be related to the fact that
iotlb_translate()fails with-EINVAL, so it seems that we miss some mapping.Is this only reproducible when there's multiple co-current accessing of IOTLB? If yes, it's probably a hint that some kind of synchronization is still missed somewhere.
It might be useful to log the dma_map/unmp in both
virtio_ringand vringh to see who is missing the map.
Quoted 8 lines
Quoted 5 lines
Just an update about these issues with vdpa-sim-blk.
I've been focusing a little bit on these failures over the last few days and have found two issues related to the IOTLB/IOMMU:1. Some requests coming from the block layer fills the SG list with multiple buffers that had the same physical address. This happens for example while using 'mkfs', at some points multiple sectors are zeroed so multiple SG elements point to the same physical page that is zeroed. Since we are using
vhost_iotlb_del_range()in thevdpasim_unmap_page(), this removes all the overlapped ranges. I fixed removing a single map invdpasim_unmap_page(), but has an alternative we can implement some kind of reference counts.I think we need to do what hardware do. So using refcount is probably not a good ida.
Okay, so since we are using for simplicity an identical mapping, we are assigning the same dma_addr to multiple pages.
So, it should be okay to remove a single mapping checking the others parameters (i.e. dir, size).
I'll send a patch, so with the code it should be easier :-)
Thanks,
Stefano
Quoted 5 lines
Quoted 5 lines
2. There was a race between dma_map/unmap and the worker thread, since both are accessing the IOMMU. Taking the
iommu_lockwhile using vhost_iotlb_* API in the worker thread fixes the "vringh: Failed to *" issues.
Quoted 14 lines
Quoted 2 lines
Whit these issues fixed the vdpa-blk simulator seems to work well. I'll send the patches next week or after the break.
Good to know this.
Thanks
Quoted 4 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 41 lines · Stefano Garzarella
Quoted 40 lines · Jason Wang
Quoted 38 lines · Stefano Garzarella
Quoted 25 lines · Jason Wang
Quoted 23 lines · Stefano Garzarella
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.This series moves the network device simulator in a new module (
vdpa_sim_net) and leaves the generic functions in thevdpa_simcore module, allowing the possibility to add new vDPA device simulators. Then we added a newvdpa_sim_blkmodule to simulate a block device.I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer()to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modifyvringh_iov_xfer().As Max reported, I'm also seeing errors with
vdpa_sim_blkrelated to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:vringh: Failed to access avail idx at
00000000e8deb2cc
vringh: Failed to read head: idx 6289 address00000000e1ad1d50
vringh: Failed to get flags at000000006635d7a3
virtio_vdpavdpa0:vringh_iov_push_iotlb()error: -14 offset:0x2840000len:0x20000virtio_vdpavdpa0:vringh_iov_pull_iotlb()error: -14 offset:0x58ee000len:0x3000Quoted 8 lines
Quoted 2 lines
These errors should all be related to the fact that
iotlb_translate()fails with-EINVAL, so it seems that we miss some mapping.Is this only reproducible when there's multiple co-current accessing of IOTLB? If yes, it's probably a hint that some kind of synchronization is still missed somewhere.
It might be useful to log the dma_map/unmp in both
virtio_ringand vringh to see who is missing the map.Just an update about these issues with vdpa-sim-blk.
I've been focusing a little bit on these failures over the last few days and have found two issues related to the IOTLB/IOMMU:
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
1. Some requests coming from the block layer fills the SG list with multiple buffers that had the same physical address. This happens for example while using 'mkfs', at some points multiple sectors are zeroed so multiple SG elements point to the same physical page that is zeroed. Since we are using
vhost_iotlb_del_range()in thevdpasim_unmap_page(), this removes all the overlapped ranges. I fixed removing a single map invdpasim_unmap_page(), but has an alternative we can implement some kind of reference counts.I think we need to do what hardware do. So using refcount is probably not a good ida.
Okay, so since we are using for simplicity an identical mapping, we are assigning the same
dma_addrto multiple pages.
I think I get you now. That's the root cause for the failure.
Then I think we need an simple iova allocator for vdpa simulator, and it might be useful for VDUSE as well.
Thanks
Quoted 28 lines
So, it should be okay to remove a single mapping checking the others parameters (i.e. dir, size).
I'll send a patch, so with the code it should be easier :-)
Thanks,
StefanoQuoted 19 lines
Quoted 7 lines
2. There was a race between dma_map/unmap and the worker thread, since both are accessing the IOMMU. Taking the
iommu_lockwhile using vhost_iotlb_* API in the worker thread fixes the "vringh: Failed to *" issues.Whit these issues fixed the vdpa-blk simulator seems to work well. I'll send the patches next week or after the break.
Good to know this.
Thanks
Quoted 4 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 47 lines · Jason Wang
Quoted 45 lines · Stefano Garzarella
Quoted 44 lines · Jason Wang
Quoted 40 lines · Stefano Garzarella
Quoted 25 lines · Jason Wang
Quoted 23 lines · Stefano Garzarella
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.This series moves the network device simulator in a new module (
vdpa_sim_net) and leaves the generic functions in thevdpa_simcore module, allowing the possibility to add new vDPA device simulators. Then we added a newvdpa_sim_blkmodule to simulate a block device.I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer()to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modifyvringh_iov_xfer().As Max reported, I'm also seeing errors with
vdpa_sim_blkrelated to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:vringh: Failed to access avail idx at
00000000e8deb2cc
vringh: Failed to read head: idx 6289 address00000000e1ad1d50
vringh: Failed to get flags at000000006635d7a3
virtio_vdpavdpa0:vringh_iov_push_iotlb()error: -14 offset:0x2840000len:0x20000virtio_vdpavdpa0:vringh_iov_pull_iotlb()error: -14 offset:0x58ee000len:0x3000Quoted 8 lines
Quoted 2 lines
These errors should all be related to the fact that
iotlb_translate()fails with-EINVAL, so it seems that we miss some mapping.Is this only reproducible when there's multiple co-current accessing of IOTLB? If yes, it's probably a hint that some kind of synchronization is still missed somewhere.
It might be useful to log the dma_map/unmp in both
virtio_ringand vringh to see who is missing the map.Just an update about these issues with vdpa-sim-blk.
I've been focusing a little bit on these failures over the last few days and have found two issues related to the IOTLB/IOMMU:1. Some requests coming from the block layer fills the SG list with multiple buffers that had the same physical address. This happens for example while using 'mkfs', at some points multiple sectors are zeroed so multiple SG elements point to the same physical page that is zeroed. Since we are using
vhost_iotlb_del_range()in thevdpasim_unmap_page(), this removes all the overlapped ranges. I fixed removing a single map invdpasim_unmap_page(), but has an alternative we can implement some kind of reference counts.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
I think we need to do what hardware do. So using refcount is probably not a good ida.
Okay, so since we are using for simplicity an identical mapping, we are assigning the same
dma_addrto multiple pages.I think I get you now. That's the root cause for the failure.
Yes, sorry, I didn't explain well previously.
Quoted 2 lines
Then I think we need an simple iova allocator for vdpa simulator, and it might be useful for VDUSE as well.
Okay, I'll work on it.
If you have an example to follow or some pointers, they are welcome :-)
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 55 lines · Stefano Garzarella
Quoted 47 lines · Jason Wang
Quoted 45 lines · Stefano Garzarella
Quoted 44 lines · Jason Wang
Quoted 40 lines · Stefano Garzarella
Quoted 25 lines · Jason Wang
Quoted 23 lines · Stefano Garzarella
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.This series moves the network device simulator in a new module (
vdpa_sim_net) and leaves the generic functions in thevdpa_simcore module, allowing the possibility to add new vDPA device simulators. Then we added a newvdpa_sim_blkmodule to simulate a block device.I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer()to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modifyvringh_iov_xfer().As Max reported, I'm also seeing errors with
vdpa_sim_blkrelated to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:vringh: Failed to access avail idx at
00000000e8deb2cc
vringh: Failed to read head: idx 6289 address00000000e1ad1d50
vringh: Failed to get flags at000000006635d7a3
virtio_vdpavdpa0:vringh_iov_push_iotlb()error: -14 offset:0x2840000len:0x20000virtio_vdpavdpa0:vringh_iov_pull_iotlb()error: -14 offset:0x58ee000len:0x3000Quoted 8 lines
Quoted 2 lines
These errors should all be related to the fact that
iotlb_translate()fails with-EINVAL, so it seems that we miss some mapping.Is this only reproducible when there's multiple co-current accessing of IOTLB? If yes, it's probably a hint that some kind of synchronization is still missed somewhere.
It might be useful to log the dma_map/unmp in both
virtio_ringand vringh to see who is missing the map.Just an update about these issues with vdpa-sim-blk.
I've been focusing a little bit on these failures over the last few days and have found two issues related to the IOTLB/IOMMU:1. Some requests coming from the block layer fills the SG list with multiple buffers that had the same physical address. This happens for example while using 'mkfs', at some points multiple sectors are zeroed so multiple SG elements point to the same physical page that is zeroed. Since we are using
vhost_iotlb_del_range()in thevdpasim_unmap_page(), this removes all the overlapped ranges. I fixed removing a single map invdpasim_unmap_page(), but has an alternative we can implement some kind of reference counts.Quoted 6 lines
Quoted 3 lines
Quoted 1 line
I think we need to do what hardware do. So using refcount is probably not a good ida.
Okay, so since we are using for simplicity an identical mapping, we are assigning the same
dma_addrto multiple pages.I think I get you now. That's the root cause for the failure.
Quoted 7 lines
Yes, sorry, I didn't explain well previously.
Quoted 2 lines
Then I think we need an simple iova allocator for vdpa simulator, and it might be useful for VDUSE as well.
Okay, I'll work on it.
If you have an example to follow or some pointers, they are welcome :-)
Kernel had implemented one in iova.c but I'm not sure we need the complexity like that. Or we can just use rbtree or idr to implement a simpler one.
Thanks
Quoted 4 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 60 lines · Jason Wang
Quoted 58 lines · Stefano Garzarella
Quoted 47 lines · Jason Wang
Quoted 45 lines · Stefano Garzarella
Quoted 44 lines · Jason Wang
Quoted 40 lines · Stefano Garzarella
Quoted 25 lines · Jason Wang
Quoted 23 lines · Stefano Garzarella
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.This series moves the network device simulator in a new module (
vdpa_sim_net) and leaves the generic functions in thevdpa_simcore module, allowing the possibility to add new vDPA device simulators. Then we added a newvdpa_sim_blkmodule to simulate a block device.I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer()to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modifyvringh_iov_xfer().As Max reported, I'm also seeing errors with
vdpa_sim_blkrelated to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:vringh: Failed to access avail idx at
00000000e8deb2cc
vringh: Failed to read head: idx 6289 address00000000e1ad1d50
vringh: Failed to get flags at000000006635d7a3
virtio_vdpavdpa0:vringh_iov_push_iotlb()error: -14 offset:0x2840000len:0x20000virtio_vdpavdpa0:vringh_iov_pull_iotlb()error: -14 offset:0x58ee000len:0x3000Quoted 8 lines
Quoted 2 lines
These errors should all be related to the fact that
iotlb_translate()fails with-EINVAL, so it seems that we miss some mapping.Is this only reproducible when there's multiple co-current accessing of IOTLB? If yes, it's probably a hint that some kind of synchronization is still missed somewhere.
It might be useful to log the dma_map/unmp in both
virtio_ringand vringh to see who is missing the map.Just an update about these issues with vdpa-sim-blk.
I've been focusing a little bit on these failures over the last few days and have found two issues related to the IOTLB/IOMMU:1. Some requests coming from the block layer fills the SG list with multiple buffers that had the same physical address. This happens for example while using 'mkfs', at some points multiple sectors are zeroed so multiple SG elements point to the same physical page that is zeroed. Since we are using
vhost_iotlb_del_range()in thevdpasim_unmap_page(), this removes all the overlapped ranges. I fixed removing a single map invdpasim_unmap_page(), but has an alternative we can implement some kind of reference counts.Quoted 6 lines
Quoted 3 lines
Quoted 1 line
I think we need to do what hardware do. So using refcount is probably not a good ida.
Okay, so since we are using for simplicity an identical mapping, we are assigning the same
dma_addrto multiple pages.I think I get you now. That's the root cause for the failure.
Yes, sorry, I didn't explain well previously.
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Then I think we need an simple iova allocator for vdpa simulator, and it might be useful for VDUSE as well.
Okay, I'll work on it.
If you have an example to follow or some pointers, they are welcome :-)Kernel had implemented one in
iova.cbut I'm not sure we need the complexity like that. Or we can just use rbtree or idr to implement a simpler one.
VDUSE[1] implements another allocator, but it's still complicated since it needs to track bounce pages. I feel like we'd better start from a simple one.
Thanks
[1] https://www.spinics.net/lists/linux-mm/msg231576.html
Quoted 8 lines
Thanks
Quoted 4 lines
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 60 lines · Jason Wang
Quoted 58 lines · Stefano Garzarella
Quoted 47 lines · Jason Wang
Quoted 45 lines · Stefano Garzarella
Quoted 44 lines · Jason Wang
Quoted 40 lines · Stefano Garzarella
Quoted 25 lines · Jason Wang
Quoted 23 lines · Stefano Garzarella
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.This series moves the network device simulator in a new module (
vdpa_sim_net) and leaves the generic functions in thevdpa_simcore module, allowing the possibility to add new vDPA device simulators. Then we added a newvdpa_sim_blkmodule to simulate a block device.I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer()to skip bytes when ptr is NULL"), maybe we can add a new functions instead of modifyvringh_iov_xfer().As Max reported, I'm also seeing errors with
vdpa_sim_blkrelated to iotlb and vringh when there is high load, these are some of the error messages I can see randomly:vringh: Failed to access avail idx at
00000000e8deb2cc
vringh: Failed to read head: idx 6289 address00000000e1ad1d50
vringh: Failed to get flags at000000006635d7a3
virtio_vdpavdpa0:vringh_iov_push_iotlb()error: -14 offset:0x2840000len:0x20000virtio_vdpavdpa0:vringh_iov_pull_iotlb()error: -14 offset:0x58ee000len:0x3000Quoted 8 lines
Quoted 2 lines
These errors should all be related to the fact that
iotlb_translate()fails with-EINVAL, so it seems that we miss some mapping.Is this only reproducible when there's multiple co-current accessing of IOTLB? If yes, it's probably a hint that some kind of synchronization is still missed somewhere.
It might be useful to log the dma_map/unmp in both
virtio_ringand vringh to see who is missing the map.Just an update about these issues with vdpa-sim-blk.
I've been focusing a little bit on these failures over the last few days and have found two issues related to the IOTLB/IOMMU:1. Some requests coming from the block layer fills the SG list with multiple buffers that had the same physical address. This happens for example while using 'mkfs', at some points multiple sectors are zeroed so multiple SG elements point to the same physical page that is zeroed. Since we are using
vhost_iotlb_del_range()in thevdpasim_unmap_page(), this removes all the overlapped ranges. I fixed removing a single map invdpasim_unmap_page(), but has an alternative we can implement some kind of reference counts.Quoted 6 lines
Quoted 3 lines
Quoted 1 line
I think we need to do what hardware do. So using refcount is probably not a good ida.
Okay, so since we are using for simplicity an identical mapping, we are assigning the same
dma_addrto multiple pages.I think I get you now. That's the root cause for the failure.
Yes, sorry, I didn't explain well previously.
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Then I think we need an simple iova allocator for vdpa simulator, and it might be useful for VDUSE as well.
Okay, I'll work on it.
If you have an example to follow or some pointers, they are welcome :-)Kernel had implemented one in
iova.cbut I'm not sure we need the complexity like that. Or we can just use rbtree or idr to implement a simpler one.
Yeah, I found it and I started to integrate it in the simulator.
Also if it appears complicated, it seams to me that it should be simple to integrate.
I'll give it a try, and if it is too complicate, I'll switch to a simple rbtree.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization