Extending VIRTIO with a data transfer test
Subsystems (2)
cover Extending VIRTIO with a data transfer test
Hello All,
This series implement VIRTIO-CCW transport and a VIRTIO-CCW data transfer test.
The first patch is for allowing different architectures to use the existing VIRTIO framework used by ARM.
We then need a callback in the CSS enumeration to select a device, and bind it with the VIRTIO transport, patches 2 and 3.
We need to be able to disable IRQ on the processor to protect part of the code until we are ready to receive an IRQ, and we need to have the registration and handling of IRQ at the VIRTIO level, this is patch 4.
To be able to receive data we extend the comon VIRTIO code with an input routine, patch 5.
Patch 6 and 7 are the two tests patches to implement simple VIRTIO initialization, very few tests of integrity now which can be expend later, and a data transfer test to check various alignement and sizes of buffers and to check the data integrity with a simple checksum.
To test the last part you will need the associated QEMU device published in another series on the QEMU mailing list.
Regards,
Pierre
Pierre Morel (7):
arm: virtio: move VIRTIO transport initialization inside virtio-mmio
s390x: css: add callback for emnumeration
s390x: virtio: CCW transport implementation
s390x: css: registering IRQ
virtio: implement the virtio_add_inbuf routine
s390x: virtio tests setup
s390x: virtio data transfer
lib/s390x/css.h | 24 ++- lib/s390x/css_lib.c | 31 +++- lib/s390x/virtio-ccw.c | 374 +++++++++++++++++++++++++++++++++++++++++ lib/s390x/virtio-ccw.h | 111 ++++++++++++ lib/virtio-config.h | 30 ++++ lib/virtio-mmio.c | 2 +- lib/virtio-mmio.h | 2 - lib/virtio.c | 37 +++- lib/virtio.h | 2 + s390x/Makefile | 3 + s390x/css.c | 2 +- s390x/unittests.cfg | 4 + s390x/virtio_pong.c | 315 ++++++++++++++++++++++++++++++++++ 13 files changed, 924 insertions(+), 13 deletions(-) create mode 100644 lib/s390x/virtio-ccw.c create mode 100644 lib/s390x/virtio-ccw.h create mode 100644 lib/virtio-config.h create mode 100644 s390x/virtio_pong.c
Signature · 1 line
2.25.1
PATCH 1/7 arm: virtio: move VIRTIO transport initialization inside virtio-mmio
To be able to use different VIRTIO transport in the future we need the initialisation entry call of the transport to be inside the transport file and keep the VIRTIO level transport agnostic.
--- lib/virtio-mmio.c | 2 +- lib/virtio-mmio.h | 2 -- lib/virtio.c | 5 ----- 3 files changed, 1 insertion(+), 8 deletions(-)
lib/virtio-mmio.c +1 -1
@@ -173,7 +173,7 @@ static struct virtio_device *virtio_mmio_dt_bind(u32 devid)return &vm_dev->vdev; } -struct virtio_device *virtio_mmio_bind (u32 devid) +struct virtio_device *virtio_bind (u32 devid) {return virtio_mmio_dt_bind (devid); }
lib/virtio-mmio.h +0 -2
@@ -60,6 +60,4 @@ struct virtio_mmio_device { void *base; }; -extern struct virtio_device *virtio_mmio_bind (u32 devid); -#endif /* _VIRTIO_MMIO_H_ */
lib/virtio.c +0 -5
@@ -123,8 +123,3 @@ void *virtqueue_get_buf(struct virtqueue *_vq, unsigned int *len)return ret; } - -struct virtio_device *virtio_bind (u32 devid) -{ -return virtio_mmio_bind (devid); -}
Signature · 1 line
2.25.1
Re: [kvm-unit-tests PATCH 1/7] arm: virtio: move VIRTIO transport initialization inside virtio-mmio
Sorry for the late reply - still trying to get my Inbox under control again ...
Quoted 41 lines · Pierre Morel
To be able to use different VIRTIO transport in the future we need the initialisation entry call of the transport to be inside the transport file and keep the VIRTIO level transport agnostic.
Signed-off-by Pierre Morel <[email protected]>--- lib/virtio-mmio.c | 2 +- lib/virtio-mmio.h | 2 -- lib/virtio.c | 5 ----- 3 files changed, 1 insertion(+), 8 deletions(-)lib/virtio-mmio.c +1
-1@@ -173,7 +173,7 @@ static struct virtio_device *virtio_mmio_dt_bind(u32 devid)return &vm_dev->vdev; } -struct virtio_device *virtio_mmio_bind (u32 devid) +struct virtio_device *virtio_bind (u32 devid) {return virtio_mmio_dt_bind (devid); }lib/virtio-mmio.h +0
-2@@ -60,6 +60,4 @@ struct virtio_mmio_device { void *base; }; -extern struct virtio_device *virtio_mmio_bind (u32 devid); -#endif /* _VIRTIO_MMIO_H_ */ lib/virtio.c +0
-0
lib/virtio.c void *virtqueue_get_buf(struct virtqueue *_vq, unsigned int *len) · line 130
Quoted 8 lines
lib/virtio.c +0
-5@@ -123,8 +123,3 @@ void *virtqueue_get_buf(struct virtqueue *_vq, unsigned int *len)return ret; } - -struct virtio_device *virtio_bind (u32 devid) -{ -return virtio_mmio_bind (devid); -}
I agree that this needs to be improved somehow, but I'm not sure whether moving the function to virtio-mmio.c is the right solution. I guess the original idea was that virtio_bind() could cope with multiple transports, i.e. when there is support for virtio-pci, it could choose between mmio and pci on ARM, or between CCW and PCI on s390x.
So maybe this should rather get an "#if defined(__arm__) || defined(__aarch64__)" instead? Drew, what's your opinion here?
Thomas
Re: [kvm-unit-tests PATCH 1/7] arm: virtio: move VIRTIO transport initialization inside virtio-mmio
Quoted 47 lines · Thomas Huth
Sorry for the late reply - still trying to get my Inbox under control again ...
Quoted 44 lines · Pierre Morel
To be able to use different VIRTIO transport in the future we need the initialisation entry call of the transport to be inside the transport file and keep the VIRTIO level transport agnostic.
Signed-off-by Pierre Morel <[email protected]>--- lib/virtio-mmio.c | 2 +- lib/virtio-mmio.h | 2 -- lib/virtio.c | 5 ----- 3 files changed, 1 insertion(+), 8 deletions(-)lib/virtio-mmio.c +1
-1@@ -173,7 +173,7 @@ static struct virtio_device *virtio_mmio_dt_bind(u32 devid)return &vm_dev->vdev; } -struct virtio_device *virtio_mmio_bind (u32 devid) +struct virtio_device *virtio_bind (u32 devid) {return virtio_mmio_dt_bind (devid); }lib/virtio-mmio.h +0
-2@@ -60,6 +60,4 @@ struct virtio_mmio_device { void *base; }; -extern struct virtio_device *virtio_mmio_bind (u32 devid); -#endif /* _VIRTIO_MMIO_H_ */ lib/virtio.c +0
-4@@ -123,8 +123,3 @@ void *virtqueue_get_buf(struct virtqueue *_vq, unsigned int *len)return ret; } - -struct virtio_device *virtio_bind (u32 devid) -{ -return virtio_mmio_bind (devid);
Quoted 8 lines
lib/virtio.c void *virtqueue_get_buf(struct virtqueue *_vq, unsigned int *len) · line 125
Quoted 2 lines
lib/virtio.c +0
-1-}I agree that this needs to be improved somehow, but I'm not sure whether moving the function to
virtio-mmio.cis the right solution. I guess the original idea was thatvirtio_bind()could cope with multiple transports, i.e. when there is support for virtio-pci, it could choose between mmio and pci on ARM, or between CCW and PCI on s390x.
That's right. If we wanted to use virtio-pci on ARM, then, after implementing virtio_pci_bind(), we'd change this to
struct virtio_device *virtio_bind(u32 devid)
{
struct virtio_device *dev = virtio_mmio_bind(devid); if (!dev)
dev = virtio_pci_bind(devid);
return dev;}
Then, we'd use config selection logic in the test harness to decide how to construct the QEMU command line in order to choose between mmio and pci.
Quoted 3 lines
So maybe this should rather get an "#if
defined(__arm__)||defined(__aarch64__)" instead? Drew, what's your opinion here?
Yup, but I think I'd prefer we do it in the header, like below, and then also implement something like the above for virtio_bind().
lib/virtio-mmio.h +7 -0
@@ -60,6 +60,13 @@ struct virtio_mmio_device { void *base; }; +#if defined(__arm__) || defined(__aarch64__)extern struct virtio_device *virtio_mmio_bind (u32 devid); +#else +static inline struct virtio_device *virtio_mmio_bind (u32 devid) +{ +return NULL ; +} +#endif #endif /* _VIRTIO_MMIO_H_ */
Thanks,
drew
Re: [kvm-unit-tests PATCH 1/7] arm: virtio: move VIRTIO transport initialization inside virtio-mmio
Quoted 64 lines · Andrew Jones
Quoted 47 lines · Thomas Huth
Sorry for the late reply - still trying to get my Inbox under control again ...
Quoted 44 lines · Pierre Morel
To be able to use different VIRTIO transport in the future we need the initialisation entry call of the transport to be inside the transport file and keep the VIRTIO level transport agnostic.
Signed-off-by Pierre Morel <[email protected]>--- lib/virtio-mmio.c | 2 +- lib/virtio-mmio.h | 2 -- lib/virtio.c | 5 ----- 3 files changed, 1 insertion(+), 8 deletions(-)lib/virtio-mmio.c +1
-1@@ -173,7 +173,7 @@ static struct virtio_device *virtio_mmio_dt_bind(u32 devid)return &vm_dev->vdev; } -struct virtio_device *virtio_mmio_bind (u32 devid) +struct virtio_device *virtio_bind (u32 devid) {return virtio_mmio_dt_bind (devid); }lib/virtio-mmio.h +0
-2@@ -60,6 +60,4 @@ struct virtio_mmio_device { void *base; }; -extern struct virtio_device *virtio_mmio_bind (u32 devid); -#endif /* _VIRTIO_MMIO_H_ */ lib/virtio.c +0
-4@@ -123,8 +123,3 @@ void *virtqueue_get_buf(struct virtqueue *_vq, unsigned int *len)return ret; } - -struct virtio_device *virtio_bind (u32 devid) -{ -return virtio_mmio_bind (devid);Quoted 8 lines
lib/virtio.c void *virtqueue_get_buf(struct virtqueue *_vq, unsigned int *len) · line 125
Quoted 2 lines
lib/virtio.c +0
-1-}I agree that this needs to be improved somehow, but I'm not sure whether moving the function to
virtio-mmio.cis the right solution. I guess the original idea was thatvirtio_bind()could cope with multiple transports, i.e. when there is support for virtio-pci, it could choose between mmio and pci on ARM, or between CCW and PCI on s390x.That's right. If we wanted to use virtio-pci on ARM, then, after implementing
virtio_pci_bind(), we'd change this tostruct virtio_device *virtio_bind(u32 devid) { struct virtio_device *dev = virtio_mmio_bind(devid);
Quoted 8 lines
if (!dev) dev = virtio_pci_bind(devid); return dev;}
Then, we'd use config selection logic in the test harness to decide how to construct the QEMU command line in order to choose between mmio and pci.
OK, I understand.
Quoted 28 lines
Quoted 4 lines
So maybe this should rather get an "#if
defined(__arm__)||defined(__aarch64__)" instead? Drew, what's your opinion here?Yup, but I think I'd prefer we do it in the header, like below, and then also implement something like the above for
virtio_bind().lib/virtio-mmio.h +7
-0@@ -60,6 +60,13 @@ struct virtio_mmio_device { void *base; }; +#if defined(__arm__) || defined(__aarch64__)extern struct virtio_device *virtio_mmio_bind (u32 devid); +#else +static inline struct virtio_device *virtio_mmio_bind (u32 devid) +{ +return NULL ; +} +#endif #endif /* _VIRTIO_MMIO_H_ */
Quoted 3 lines
Thanks,
drew
Thanks, I will modify accordingly.
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH 2/7 s390x: css: add callback for emnumeration
In reply to Pierre MorelWe will need to look for a device inside the channel subsystem based upon device specificities.
Let's provide a callback for an upper layer to be called during the enumeration of the channel subsystem.
--- lib/s390x/css.h | 3 ++- lib/s390x/css_lib.c | 4 +++- s390x/css.c | 2 +- 3 files changed, 6 insertions(+), 3 deletions(-)
lib/s390x/css.h +2 -1
@@ -278,7 +278,8 @@ void dump_irb(struct irb *irbp);void dump_pmcw (struct pmcw *p);void dump_orb (struct orb *op); -int css_enumerate (void ); +typedef int (enumerate_cb_t )(int ); +int css_enumerate (enumerate_cb_t *f);#define MAX_ENABLE_RETRIES 5#define IO_SCH_ISC 3
lib/s390x/css_lib.c +3 -1
@@ -117,7 +117,7 @@ bool get_chsc_scsc(void)* On success return the first subchannel ID found. * On error return an invalid subchannel ID containing cc */ -int css_enumerate (void ) +int css_enumerate (enumerate_cb_t *f) {struct pmcw *pmcw = &schib.pmcw;int scn_found =0 ;
@@ -153,6 +153,8 @@ int css_enumerate(void) schid = scn |SCHID_ONE ;report_info ("Found subchannel %08x" , scn |SCHID_ONE ); dev_found++; +if (f) +f (scn |SCHID_ONE ); } out:
s390x/css.c +1 -1
@@ -29,7 +29,7 @@ struct ccw1 *ccw;static void test_enumerate (void ) { - test_device_sid =css_enumerate (); + test_device_sid =css_enumerate (NULL );if (test_device_sid &SCHID_ONE ) {report (1 ,"Schid of first I/O device: 0x%08x" , test_device_sid);return ;
Signature · 1 line
2.25.1
Re: [kvm-unit-tests PATCH 2/7] s390x: css: add callback for emnumeration
Quoted 32 lines · Pierre Morel
We will need to look for a device inside the channel subsystem based upon device specificities.
Let's provide a callback for an upper layer to be called during the enumeration of the channel subsystem.
Signed-off-by Pierre Morel <[email protected]>--- lib/s390x/css.h | 3 ++- lib/s390x/css_lib.c | 4 +++- s390x/css.c | 2 +- 3 files changed, 6 insertions(+), 3 deletions(-)lib/s390x/css.h +2
-1@@ -278,7 +278,8 @@ void dump_irb(struct irb *irbp);void dump_pmcw (struct pmcw *p);void dump_orb (struct orb *op); -int css_enumerate (void ); +typedef int (enumerate_cb_t )(int ); +int css_enumerate (enumerate_cb_t *f);#define MAX_ENABLE_RETRIES 5#define IO_SCH_ISC 3lib/s390x/css_lib.c +0
-0
lib/s390x/css_lib.c bool get_chsc_scsc(void) · line 120
Quoted 5 lines
lib/s390x/css_lib.c +1
-1@@ -117,7 +117,7 @@ bool get_chsc_scsc(void)* On success return the first subchannel ID found. * On error return an invalid subchannel ID containing cc */ -int css_enumerate (void ) +int css_enumerate (enumerate_cb_t *f)
I'd maybe call it "cb" instead of "f" ... but that's just my personal taste, I guess.
Quoted 18 lines
lib/s390x/css_lib.c +2
-0{struct pmcw *pmcw = &schib.pmcw;int scn_found =0 ;@@ -153,6 +153,8 @@ int css_enumerate(void) schid = scn |SCHID_ONE ;report_info ("Found subchannel %08x" , scn |SCHID_ONE ); dev_found++; +if (f) +f (scn |SCHID_ONE ); } out:s390x/css.c +0
-0
s390x/css.c struct ccw1 *ccw; · line 35
Quoted 8 lines
s390x/css.c +1
-1@@ -29,7 +29,7 @@ struct ccw1 *ccw;static void test_enumerate (void ) { - test_device_sid =css_enumerate (); + test_device_sid =css_enumerate (NULL );if (test_device_sid &SCHID_ONE ) {report (1 ,"Schid of first I/O device: 0x%08x" , test_device_sid);return ;
Re: [kvm-unit-tests PATCH 2/7] s390x: css: add callback for emnumeration
Quoted 30 lines · Thomas Huth
Quoted 29 lines · Pierre Morel
We will need to look for a device inside the channel subsystem based upon device specificities.
Let's provide a callback for an upper layer to be called during the enumeration of the channel subsystem.
Signed-off-by Pierre Morel <[email protected]>---
lib/s390x/css.h | 3 ++-
lib/s390x/css_lib.c| 4 +++-
s390x/css.c | 2 +-3 files changed, 6 insertions(+), 3 deletions(-)lib/s390x/css.h +2
-1@@ -278,7 +278,8 @@ void dump_irb(struct irb *irbp); void dump_pmcw(struct pmcw *p); void dump_orb(struct orb *op); -int css_enumerate (void ); +typedef int (enumerate_cb_t )(int ); +int css_enumerate (enumerate_cb_t *f); #define MAX_ENABLE_RETRIES 5 #define IO_SCH_ISC 3lib/s390x/css_lib.c +0
-0
Quoted 8 lines
lib/s390x/css_lib.c bool get_chsc_scsc(void) · line 120
Quoted 6 lines
lib/s390x/css_lib.c +1
-1@@ -117,7 +117,7 @@ bool get_chsc_scsc(void)* On success return the first subchannel ID found. * On error return an invalid subchannel ID containing cc */ -int css_enumerate (void ) +int css_enumerate (enumerate_cb_t *f)I'd maybe call it "cb" instead of "f" ... but that's just my personal taste, I guess.
I can use cb, looks better.
Quoted 19 lines
Quoted 19 lines
lib/s390x/css_lib.c +2
-0{ struct pmcw *pmcw = &schib.pmcw; int scn_found = 0;@@ -153,6 +153,8 @@ int css_enumerate(void) schid = scn | SCHID_ONE; report_info("Found subchannel %08x", scn | SCHID_ONE); dev_found++; + if (f) + f(scn | SCHID_ONE); }out:
s390x/css.c +0
-0@@ -29,7 +29,7 @@ struct ccw1 *ccw; static void test_enumerate(void) {
Quoted 9 lines
s390x/css.c struct ccw1 *ccw;
Quoted 6 lines
s390x/css.c +1
-1- test_device_sid = css_enumerate(); + test_device_sid = css_enumerate(NULL); if (test_device_sid & SCHID_ONE) { report(1, "Schid of first I/O device: 0x%08x", test_device_sid); return;Reviewed-by Thomas Huth <[email protected]>
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH 3/7 s390x: virtio: CCW transport implementation
In reply to Pierre MorelThis is the implementation of the virtio-ccw transport level.
We only support VIRTIO revision 0.
--- lib/s390x/virtio-ccw.c | 374 +++++++++++++++++++++++++++++++++++++++++ lib/s390x/virtio-ccw.h | 111 ++++++++++++ lib/virtio-config.h | 30 ++++ s390x/Makefile | 2 + 4 files changed, 517 insertions(+) create mode 100644 lib/s390x/virtio-ccw.c create mode 100644 lib/s390x/virtio-ccw.h create mode 100644 lib/virtio-config.h
lib/s390x/virtio-ccw.c new file +374 -0
@@ -0,0 +1,374 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* Virtio CCW Library +* +* Copyright (c) 2021 IBM Corp +* +* Authors: +* Pierre Morel <[email protected]> +* +*/ + +#include <libcflat.h> +#include <alloc_page.h> +#include <asm/page.h> +#include <string.h> +#include <interrupt.h> +#include <asm/arch_def.h> +#include <asm/facility.h> +#include <asm/uv.h> + +#include <css.h> +#include <virtio.h> +#include <virtio-config.h> +#include <virtio-ccw.h> +#include <malloc_io.h> + +static struct linked_list vcdev_list = { + .prev = &vcdev_list, + .next = &vcdev_list +}; + +static inline uint32_t swap16 (uint32_t b) +{ +return (((b &0xff00U ) <<8 ) | + ((b &0x00ff ) >>8 )); +} + +static inline uint32_t swap32 (uint32_t b) +{ +return (((b &0x000000ffU ) <<24 ) | + ((b &0x0000ff00U ) <<8 ) | + ((b &0x00ff0000U ) >>8 ) | + ((b &0xff000000U ) >>24 )); +} + +static inline uint64_t swap64 (uint64_t x) +{ +return (((x &0x00000000000000ffULL ) <<56 ) | + ((x &0x000000000000ff00ULL ) <<40 ) | + ((x &0x0000000000ff0000ULL ) <<24 ) | + ((x &0x00000000ff000000ULL ) <<8 ) | + ((x &0x000000ff00000000ULL ) >>8 ) | + ((x &0x0000ff0000000000ULL ) >>24 ) | + ((x &0x00ff000000000000ULL ) >>40 ) | + ((x &0xff00000000000000ULL ) >>56 )); +} + +/* +* flags: flags for CCW +* Returns !0 on failure +* Returns 0 on success +*/ +int ccw_send (struct virtio_ccw_device *vcdev,int code,void *data,int count, +unsigned char flags) +{ +struct ccw1 *ccw; +int ret =-1 ; + + ccw =alloc_io_mem (sizeof (*ccw),0 ); +if (!ccw) +return ret; + +/* Build the CCW chain with a single CCW */ + ccw->code = code; + ccw->flags = flags; + ccw->count = count; + ccw->data_address = (unsigned long )data; + + ret =start_ccw1_chain (vcdev->schid, ccw); +if (!ret) + ret =wait_and_check_io_completion (vcdev->schid); + +free_io_mem (ccw,sizeof (*ccw)); +return ret; +} + +int virtio_ccw_set_revision (struct virtio_ccw_device *vcdev) +{ +struct virtio_rev_info *rev_info; +int ret =-1 ; + + rev_info =alloc_io_mem (sizeof (*rev_info),0 ); +if (!rev_info) +return ret; + + rev_info->revision =VIRTIO_CCW_REV_MAX ; + rev_info->revision =0 ; +do { + ret =ccw_send (vcdev,CCW_CMD_SET_VIRTIO_REV , rev_info, +sizeof (*rev_info),0 ); + }while (ret && rev_info->revision--); + +free_io_mem (rev_info,sizeof (*rev_info)); + +return ret ?-1 : rev_info->revision; +} + +int virtio_ccw_reset (struct virtio_ccw_device *vcdev) +{ +return ccw_send (vcdev,CCW_CMD_VDEV_RESET ,0 ,0 ,0 ); +} + +int virtio_ccw_read_status (struct virtio_ccw_device *vcdev) +{ +return ccw_send (vcdev,CCW_CMD_READ_STATUS , &vcdev->status, +sizeof (vcdev->status),0 ); +} + +int virtio_ccw_write_status (struct virtio_ccw_device *vcdev) +{ +return ccw_send (vcdev,CCW_CMD_WRITE_STATUS , &vcdev->status, +sizeof (vcdev->status),0 ); +} + +int virtio_ccw_read_features (struct virtio_ccw_device *vcdev,uint64_t *features) +{ +struct virtio_feature_desc *f_desc = &vcdev->f_desc; + + f_desc->index =0 ; +if (ccw_send (vcdev,CCW_CMD_READ_FEAT , f_desc,sizeof (*f_desc),0 )) +return -1 ; + *features =swap32 (f_desc->features); + + f_desc->index =1 ; +if (ccw_send (vcdev,CCW_CMD_READ_FEAT , f_desc,sizeof (*f_desc),0 )) +return -1 ; + *features |= (uint64_t )swap32 (f_desc->features) <<32 ; + +return 0 ; +} + +int virtio_ccw_write_features (struct virtio_ccw_device *vcdev,uint64_t features) +{ +struct virtio_feature_desc *f_desc = &vcdev->f_desc; + + f_desc->index =0 ; + f_desc->features =swap32 ((uint32_t )features &0xffffffff ); +if (ccw_send (vcdev,CCW_CMD_WRITE_FEAT , &f_desc,sizeof (*f_desc),0 )) +return -1 ; + + f_desc->index =1 ; + f_desc->features =swap32 ((uint32_t )(features >>32 ) &0xffffffff ); +if (ccw_send (vcdev,CCW_CMD_WRITE_FEAT , &f_desc,sizeof (*f_desc),0 )) +return -1 ; + +return 0 ; +} + +int virtio_ccw_read_config (struct virtio_ccw_device *vcdev) +{ +return ccw_send (vcdev,CCW_CMD_READ_CONF , &vcdev->config, +sizeof (vcdev->config),0 ); +} + +int virtio_ccw_write_config (struct virtio_ccw_device *vcdev) +{ +return ccw_send (vcdev,CCW_CMD_WRITE_CONF , &vcdev->config, +sizeof (vcdev->config),0 ); +} + +int virtio_ccw_setup_indicators (struct virtio_ccw_device *vcdev) +{ + vcdev->ind =alloc_io_mem (sizeof (PAGE_SIZE ),0 ); +if (ccw_send (vcdev,CCW_CMD_SET_IND , &vcdev->ind, +sizeof (vcdev->ind),0 )) +return -1 ; + + vcdev->conf_ind =alloc_io_mem (PAGE_SIZE ,0 ); +if (ccw_send (vcdev,CCW_CMD_SET_CONF_IND , &vcdev->conf_ind, +sizeof (vcdev->conf_ind),0 )) +return -1 ; + +return 0 ; +} + +static uint64_t virtio_ccw_notify_host (int schid,int queue,uint64_t cookie) +{ + register unsigned long nr asm("1") = 0x03; + register unsigned long s asm("2") = schid; + register unsigned long q asm("3") = queue; + registerlong rc asm("2" ); + register long c asm("4") = cookie; + + asmvolatile ("diag 2,4,0x500\n" + :"=d" (rc) + :"d" (nr),"d" (s),"d" (q),"d" (c) + :"memory" ,"cc" ); +return rc; +} + +static bool virtio_ccw_notify (struct virtqueue *vq) +{ +struct virtio_ccw_device *vcdev =to_vc_device (vq->vdev); +struct virtio_ccw_vq_info *info = vq->priv; + + info->cookie =virtio_ccw_notify_host (vcdev->schid, vq->index, + info->cookie); +if (info->cookie <0 ) +return false; +return true; +} + +/* allocates a vring_virtqueue but returns a pointer to the +* virtqueue inside of it or NULL on error. +*/ +static struct virtqueue *setup_vq (struct virtio_device *vdev,int index, +void (*callback)(struct virtqueue *vq), +const char *name) +{ +struct virtio_ccw_device *vcdev =to_vc_device (vdev); +struct virtio_ccw_vq_info *info; +struct vring_virtqueue *vq; +struct vring *vr; +void *queue; + + vq =alloc_io_mem (sizeof (*vq),0 ); + info =alloc_io_mem (sizeof (*info),0 ); + queue =alloc_io_mem (4 *PAGE_SIZE ,0 ); +assert (vq && queue && info); + + info->info_block =alloc_io_mem (sizeof (*info->info_block),0 ); +assert (info->info_block); + + vcdev->vq_conf.index = index; +if (ccw_send (vcdev,CCW_CMD_READ_VQ_CONF , &vcdev->vq_conf, +sizeof (vcdev->vq_conf),0 )) +return NULL ; + +vring_init_virtqueue (vq, index, vcdev->vq_conf.max_num,PAGE_SIZE , vdev, + queue, virtio_ccw_notify, callback, name); + + vr = &vq->vring; + info->info_block->s.desc = vr->desc; + info->info_block->s.index = index; + info->info_block->s.num = vr->num; + info->info_block->s.avail = vr->avail; + info->info_block->s.used = vr->used; + + info->info_block->l.desc = vr->desc; + info->info_block->l.index = index; + info->info_block->l.num = vr->num; + info->info_block->l.align =PAGE_SIZE ; + +if (ccw_send (vcdev,CCW_CMD_SET_VQ , info->info_block, +sizeof (info->info_block->l),0 )) +return NULL ; + + info->vq = &vq->vq; + vq->vq.priv = info; + +return &vq->vq; +} + +static int virtio_ccw_find_vqs (struct virtio_device *vdev,unsigned int nvqs, +struct virtqueue *vqs[],vq_callback_t *callbacks[], +const char *names[]) +{ +int i; + +for (i =0 ; i < nvqs; ++i) { + vqs[i] =setup_vq (vdev, i, + callbacks ? callbacks[i] :NULL , + names ? names[i] :"" ); +if (!vqs[i]) +return -1 ; + } + +return 0 ; +} + +static void virtio_ccw_config_get (struct virtio_device *vdev, +unsigned int offset,void *buf, +unsigned int len) +{ +struct virtio_ccw_device *vcdev =to_vc_device (vdev); + +if (virtio_ccw_read_config (vcdev)) +return ; +memcpy (buf, vcdev->config, len); +} + +static void virtio_ccw_config_set (struct virtio_device *vdev, +unsigned int offset,const void *buf, +unsigned int len) +{ +struct virtio_ccw_device *vcdev =to_vc_device (vdev); + +memcpy (vcdev->config, buf, len); +virtio_ccw_write_config (vcdev); +} + +static const struct virtio_config_ops virtio_ccw_ops = { + .get = virtio_ccw_config_get, + .set = virtio_ccw_config_set, + .find_vqs = virtio_ccw_find_vqs, +}; + +const struct virtio_config_ops *virtio_ccw_register (void ) +{ +return &virtio_ccw_ops; +} + +static int sense (struct virtio_ccw_device *vcdev) +{ +struct senseid *senseid; + + senseid =alloc_io_mem (sizeof (*senseid),0 ); +assert (senseid); + +assert (!ccw_send (vcdev,CCW_CMD_SENSE_ID , senseid,sizeof (*senseid),0 )); + +assert (senseid->reserved ==0xff ); + + vcdev->cu_type = senseid->cu_type; + vcdev->cu_model = senseid->cu_model; + vcdev->dev_type = senseid->dev_type; + vcdev->dev_model = senseid->dev_model; + +return 0 ; +} + +static struct virtio_ccw_device *find_vcdev_by_devid (int devid) +{ +struct virtio_ccw_device *dev; +struct linked_list *l; + +for (l = vcdev_list.next; l != &vcdev_list; l = l->next) { + dev = container_of(l, struct virtio_ccw_device, list); +if (dev->cu_model == devid) +return dev; + } +return NULL ; +} + +struct virtio_device *virtio_bind (u32 devid) +{ +struct virtio_ccw_device *vcdev; + + vcdev =find_vcdev_by_devid (devid); + +return &vcdev->vdev; +} + +static int virtio_enumerate (int schid) +{ +struct virtio_ccw_device *vcdev; + + vcdev =alloc_io_mem (sizeof (*vcdev),0 ); +assert (vcdev); + vcdev->schid = schid; + +list_add (&vcdev_list, &vcdev->list); + +assert (css_enable (schid,IO_SCH_ISC ) ==0 ); +sense (vcdev); + +return 0 ; +} + +/* Must get a param */ +bool virtio_ccw_init (void ) +{ +return css_enumerate (virtio_enumerate) !=0 ; +}
lib/s390x/virtio-ccw.h new file +111 -0
111 lines not on this page · raw message
lib/virtio-config.h new file +30 -0
@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef _VIRTIO_CONFIG_H_ +#define _VIRTIO_CONFIG_H_ + +#define VIRTIO_CONFIG_S_ACKNOWLEDGE 0x01 +#define VIRTIO_CONFIG_S_DRIVER 0x02 +#define VIRTIO_CONFIG_S_DRIVER_OK 0x04 +#define VIRTIO_CONFIG_S_FEATURES_OK 0x08 +#define VIRTIO_CONFIG_S_NEEDS_RESET 0x40 +#define VIRTIO_CONFIG_S_FAILED 0x80 + +/* Generic VIRTIO Features */ +#define VIRTIO_F_NOTIFY_ON_EMPTY 24 +#define VIRTIO_F_ANY_LAYOUT 27 + +/* Transport specific features */ +#define VIRTIO_TRANSPORT_F_START 28 +#define VIRTIO_F_RING_INDIRECT_DESC 28 +#define VIRTIO_F_RING_EVENT_IDX 29 +#define VIRTIO_TRANSPORT_F_END 38 + +/* New features starting with version 1 */ +#define VIRTIO_F_VERSION_1 32 +#define VIRTIO_F_IOMMU_PLATFORM 33 +#define VIRTIO_F_RING_PACKED 34 +#define VIRTIO_F_ORDER_PLATFORM 36 +#define VIRTIO_F_SR_IOV 37 + +#endif /* _VIRTIO_CONFIG_H_ */
s390x/Makefile +2 -0
@@ -71,6 +71,8 @@ cflatobjs += lib/s390x/css_dump.o cflatobjs += lib/s390x/css_lib.o cflatobjs += lib/s390x/malloc_io.o cflatobjs += lib/s390x/uv.o +cflatobjs += lib/virtio.o +cflatobjs += lib/s390x/virtio-ccw.o OBJDIRS += lib/s390x
Signature · 1 line
2.25.1
Re: [kvm-unit-tests PATCH 3/7] s390x: virtio: CCW transport implementation
Quoted 9 lines · Pierre Morel
This is the implementation of the virtio-ccw transport level.
We only support VIRTIO revision 0.
Signed-off-by Pierre Morel <[email protected]>--- lib/s390x/virtio-ccw.c | 374 +++++++++++++++++++++++++++++++++++++++++ lib/s390x/virtio-ccw.h | 111 ++++++++++++ lib/virtio-config.h | 30 ++++
I'd probably just drop these config defines into lib/virtio.h.
Thanks,
drew
Re: [kvm-unit-tests PATCH 3/7] s390x: virtio: CCW transport implementation
Quoted 11 lines · Andrew Jones
Quoted 9 lines · Pierre Morel
This is the implementation of the virtio-ccw transport level.
We only support VIRTIO revision 0.
Signed-off-by Pierre Morel <[email protected]>--- lib/s390x/virtio-ccw.c | 374 +++++++++++++++++++++++++++++++++++++++++ lib/s390x/virtio-ccw.h | 111 ++++++++++++ lib/virtio-config.h | 30 ++++
Quoted 5 lines
I'd probably just drop these config defines into
lib/virtio.h.Thanks,
drew
I will do that.
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [kvm-unit-tests PATCH 3/7] s390x: virtio: CCW transport implementation
In reply to Pierre MorelQuoted 3 lines · Pierre Morel
This is the implementation of the virtio-ccw transport level.
We only support VIRTIO revision 0.
That means only legacy virtio? Wouldn't it be better to shoot for modern virtio instead?
Quoted 63 lines
Signed-off-by Pierre Morel <[email protected]>--- lib/s390x/virtio-ccw.c | 374 +++++++++++++++++++++++++++++++++++++++++ lib/s390x/virtio-ccw.h | 111 ++++++++++++ lib/virtio-config.h | 30 ++++ s390x/Makefile | 2 + 4 files changed, 517 insertions(+)create mode 100644
lib/s390x/virtio-ccw.c
create mode 100644lib/s390x/virtio-ccw.h
create mode 100644lib/virtio-config.hlib/s390x/virtio-ccw.c new file +46
-0@@ -0,0 +1,374 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* Virtio CCW Library +* +* Copyright (c) 2021 IBM Corp +* +* Authors: +* Pierre Morel <[email protected]> +* +*/ + +#include <libcflat.h> +#include <alloc_page.h> +#include <asm/page.h> +#include <string.h> +#include <interrupt.h> +#include <asm/arch_def.h> +#include <asm/facility.h> +#include <asm/uv.h> + +#include <css.h> +#include <virtio.h> +#include <virtio-config.h> +#include <virtio-ccw.h> +#include <malloc_io.h> + +static struct linked_list vcdev_list = { + .prev = &vcdev_list, + .next = &vcdev_list +}; + +static inline uint32_t swap16 (uint32_t b) +{ +return (((b &0xff00U ) <<8 ) | + ((b &0x00ff ) >>8 )); +} + +static inline uint32_t swap32 (uint32_t b) +{ +return (((b &0x000000ffU ) <<24 ) | + ((b &0x0000ff00U ) <<8 ) | + ((b &0x00ff0000U ) >>8 ) | + ((b &0xff000000U ) >>24 )); +} + +static inline uint64_t swap64 (uint64_t x)
lib/s390x/virtio-ccw.c line 56
Quoted 10 lines
lib/s390x/virtio-ccw.c new file +10
-0+{ +return (((x &0x00000000000000ffULL ) <<56 ) | + ((x &0x000000000000ff00ULL ) <<40 ) | + ((x &0x0000000000ff0000ULL ) <<24 ) | + ((x &0x00000000ff000000ULL ) <<8 ) | + ((x &0x000000ff00000000ULL ) >>8 ) | + ((x &0x0000ff0000000000ULL ) >>24 ) | + ((x &0x00ff000000000000ULL ) >>40 ) | + ((x &0xff00000000000000ULL ) >>56 )); +}
We already have macros for swapping in lib/asm-generic/io.h ... could you use those instead?
Quoted 30 lines
lib/s390x/virtio-ccw.c new file +30
-0+/* +* flags: flags for CCW +* Returns !0 on failure +* Returns 0 on success +*/ +int ccw_send (struct virtio_ccw_device *vcdev,int code,void *data,int count, +unsigned char flags) +{ +struct ccw1 *ccw; +int ret =-1 ; + + ccw =alloc_io_mem (sizeof (*ccw),0 ); +if (!ccw) +return ret; + +/* Build the CCW chain with a single CCW */ + ccw->code = code; + ccw->flags = flags; + ccw->count = count; + ccw->data_address = (unsigned long )data; + + ret =start_ccw1_chain (vcdev->schid, ccw); +if (!ret) + ret =wait_and_check_io_completion (vcdev->schid); + +free_io_mem (ccw,sizeof (*ccw)); +return ret; +} + +int virtio_ccw_set_revision (struct virtio_ccw_device *vcdev)
lib/s390x/virtio-ccw.c line 97
Quoted 10 lines
lib/s390x/virtio-ccw.c new file +10
-0+{ +struct virtio_rev_info *rev_info; +int ret =-1 ; + + rev_info =alloc_io_mem (sizeof (*rev_info),0 ); +if (!rev_info) +return ret; + + rev_info->revision =VIRTIO_CCW_REV_MAX ; + rev_info->revision =0 ;
Either VIRTIO_CCW_REV_MAX or 0, but not both?
Quoted 30 lines
lib/s390x/virtio-ccw.c new file +30
-0+do { + ret =ccw_send (vcdev,CCW_CMD_SET_VIRTIO_REV , rev_info, +sizeof (*rev_info),0 ); + }while (ret && rev_info->revision--); + +free_io_mem (rev_info,sizeof (*rev_info)); + +return ret ?-1 : rev_info->revision; +} + +int virtio_ccw_reset (struct virtio_ccw_device *vcdev) +{ +return ccw_send (vcdev,CCW_CMD_VDEV_RESET ,0 ,0 ,0 ); +} + +int virtio_ccw_read_status (struct virtio_ccw_device *vcdev) +{ +return ccw_send (vcdev,CCW_CMD_READ_STATUS , &vcdev->status, +sizeof (vcdev->status),0 ); +} + +int virtio_ccw_write_status (struct virtio_ccw_device *vcdev) +{ +return ccw_send (vcdev,CCW_CMD_WRITE_STATUS , &vcdev->status, +sizeof (vcdev->status),0 ); +} + +int virtio_ccw_read_features (struct virtio_ccw_device *vcdev,uint64_t *features) +{ +struct virtio_feature_desc *f_desc = &vcdev->f_desc;
lib/s390x/virtio-ccw.c line 137
Quoted 10 lines
lib/s390x/virtio-ccw.c new file +10
-0+ + f_desc->index =0 ; +if (ccw_send (vcdev,CCW_CMD_READ_FEAT , f_desc,sizeof (*f_desc),0 )) +return -1 ; + *features =swap32 (f_desc->features); + + f_desc->index =1 ; +if (ccw_send (vcdev,CCW_CMD_READ_FEAT , f_desc,sizeof (*f_desc),0 )) +return -1 ; + *features |= (uint64_t )swap32 (f_desc->features) <<32 ;
Weren't the upper feature bits only available for modern virtio anyway?
Quoted 43 lines
lib/s390x/virtio-ccw.c new file +43
-0+return 0 ; +} + +int virtio_ccw_write_features (struct virtio_ccw_device *vcdev,uint64_t features) +{ +struct virtio_feature_desc *f_desc = &vcdev->f_desc; + + f_desc->index =0 ; + f_desc->features =swap32 ((uint32_t )features &0xffffffff ); +if (ccw_send (vcdev,CCW_CMD_WRITE_FEAT , &f_desc,sizeof (*f_desc),0 )) +return -1 ; + + f_desc->index =1 ; + f_desc->features =swap32 ((uint32_t )(features >>32 ) &0xffffffff ); +if (ccw_send (vcdev,CCW_CMD_WRITE_FEAT , &f_desc,sizeof (*f_desc),0 )) +return -1 ; + +return 0 ; +} + +int virtio_ccw_read_config (struct virtio_ccw_device *vcdev) +{ +return ccw_send (vcdev,CCW_CMD_READ_CONF , &vcdev->config, +sizeof (vcdev->config),0 ); +} + +int virtio_ccw_write_config (struct virtio_ccw_device *vcdev) +{ +return ccw_send (vcdev,CCW_CMD_WRITE_CONF , &vcdev->config, +sizeof (vcdev->config),0 ); +} + +int virtio_ccw_setup_indicators (struct virtio_ccw_device *vcdev) +{ + vcdev->ind =alloc_io_mem (sizeof (PAGE_SIZE ),0 ); +if (ccw_send (vcdev,CCW_CMD_SET_IND , &vcdev->ind, +sizeof (vcdev->ind),0 )) +return -1 ; + + vcdev->conf_ind =alloc_io_mem (PAGE_SIZE ,0 ); +if (ccw_send (vcdev,CCW_CMD_SET_CONF_IND , &vcdev->conf_ind, +sizeof (vcdev->conf_ind),0 )) +return -1 ;
lib/s390x/virtio-ccw.c line 191
Quoted 10 lines
lib/s390x/virtio-ccw.c new file +10
-0+ +return 0 ; +} + +static uint64_t virtio_ccw_notify_host (int schid,int queue,uint64_t cookie) +{ + register unsigned long nr asm("1") = 0x03; + register unsigned long s asm("2") = schid; + register unsigned long q asm("3") = queue; + registerlong rc asm("2" );
Using asm("2") for two variables looks somewhat weird ... but ok, as long as it works... (otherwise you could also use only one variable and mark it as input + output parameter below).
Quoted 169 lines
lib/s390x/virtio-ccw.c new file +169
-0+ register long c asm("4") = cookie; + + asmvolatile ("diag 2,4,0x500\n" + :"=d" (rc) + :"d" (nr),"d" (s),"d" (q),"d" (c) + :"memory" ,"cc" ); +return rc; +} + +static bool virtio_ccw_notify (struct virtqueue *vq) +{ +struct virtio_ccw_device *vcdev =to_vc_device (vq->vdev); +struct virtio_ccw_vq_info *info = vq->priv; + + info->cookie =virtio_ccw_notify_host (vcdev->schid, vq->index, + info->cookie); +if (info->cookie <0 ) +return false; +return true; +} + +/* allocates a vring_virtqueue but returns a pointer to the +* virtqueue inside of it or NULL on error. +*/ +static struct virtqueue *setup_vq (struct virtio_device *vdev,int index, +void (*callback)(struct virtqueue *vq), +const char *name) +{ +struct virtio_ccw_device *vcdev =to_vc_device (vdev); +struct virtio_ccw_vq_info *info; +struct vring_virtqueue *vq; +struct vring *vr; +void *queue; + + vq =alloc_io_mem (sizeof (*vq),0 ); + info =alloc_io_mem (sizeof (*info),0 ); + queue =alloc_io_mem (4 *PAGE_SIZE ,0 ); +assert (vq && queue && info); + + info->info_block =alloc_io_mem (sizeof (*info->info_block),0 ); +assert (info->info_block); + + vcdev->vq_conf.index = index; +if (ccw_send (vcdev,CCW_CMD_READ_VQ_CONF , &vcdev->vq_conf, +sizeof (vcdev->vq_conf),0 )) +return NULL ; + +vring_init_virtqueue (vq, index, vcdev->vq_conf.max_num,PAGE_SIZE , vdev, + queue, virtio_ccw_notify, callback, name); + + vr = &vq->vring; + info->info_block->s.desc = vr->desc; + info->info_block->s.index = index; + info->info_block->s.num = vr->num; + info->info_block->s.avail = vr->avail; + info->info_block->s.used = vr->used; + + info->info_block->l.desc = vr->desc; + info->info_block->l.index = index; + info->info_block->l.num = vr->num; + info->info_block->l.align =PAGE_SIZE ; + +if (ccw_send (vcdev,CCW_CMD_SET_VQ , info->info_block, +sizeof (info->info_block->l),0 )) +return NULL ; + + info->vq = &vq->vq; + vq->vq.priv = info; + +return &vq->vq; +} + +static int virtio_ccw_find_vqs (struct virtio_device *vdev,unsigned int nvqs, +struct virtqueue *vqs[],vq_callback_t *callbacks[], +const char *names[]) +{ +int i; + +for (i =0 ; i < nvqs; ++i) { + vqs[i] =setup_vq (vdev, i, + callbacks ? callbacks[i] :NULL , + names ? names[i] :"" ); +if (!vqs[i]) +return -1 ; + } + +return 0 ; +} + +static void virtio_ccw_config_get (struct virtio_device *vdev, +unsigned int offset,void *buf, +unsigned int len) +{ +struct virtio_ccw_device *vcdev =to_vc_device (vdev); + +if (virtio_ccw_read_config (vcdev)) +return ; +memcpy (buf, vcdev->config, len); +} + +static void virtio_ccw_config_set (struct virtio_device *vdev, +unsigned int offset,const void *buf, +unsigned int len) +{ +struct virtio_ccw_device *vcdev =to_vc_device (vdev); + +memcpy (vcdev->config, buf, len); +virtio_ccw_write_config (vcdev); +} + +static const struct virtio_config_ops virtio_ccw_ops = { + .get = virtio_ccw_config_get, + .set = virtio_ccw_config_set, + .find_vqs = virtio_ccw_find_vqs, +}; + +const struct virtio_config_ops *virtio_ccw_register (void ) +{ +return &virtio_ccw_ops; +} + +static int sense (struct virtio_ccw_device *vcdev) +{ +struct senseid *senseid; + + senseid =alloc_io_mem (sizeof (*senseid),0 ); +assert (senseid); + +assert (!ccw_send (vcdev,CCW_CMD_SENSE_ID , senseid,sizeof (*senseid),0 )); + +assert (senseid->reserved ==0xff ); + + vcdev->cu_type = senseid->cu_type; + vcdev->cu_model = senseid->cu_model; + vcdev->dev_type = senseid->dev_type; + vcdev->dev_model = senseid->dev_model; + +return 0 ; +} + +static struct virtio_ccw_device *find_vcdev_by_devid (int devid) +{ +struct virtio_ccw_device *dev; +struct linked_list *l; + +for (l = vcdev_list.next; l != &vcdev_list; l = l->next) { + dev = container_of(l, struct virtio_ccw_device, list); +if (dev->cu_model == devid) +return dev; + } +return NULL ; +} + +struct virtio_device *virtio_bind (u32 devid) +{ +struct virtio_ccw_device *vcdev; + + vcdev =find_vcdev_by_devid (devid); + +return &vcdev->vdev; +} + +static int virtio_enumerate (int schid) +{ +struct virtio_ccw_device *vcdev; + + vcdev =alloc_io_mem (sizeof (*vcdev),0 ); +assert (vcdev); + vcdev->schid = schid;
lib/s390x/virtio-ccw.c line 370
Quoted 10 lines
lib/s390x/virtio-ccw.c new file +10
-0+ +list_add (&vcdev_list, &vcdev->list); + +assert (css_enable (schid,IO_SCH_ISC ) ==0 ); +sense (vcdev); + +return 0 ; +} + +/* Must get a param */
I don't understand that comment, could you elaborate?
lib/s390x/virtio-ccw.c line 374
Quoted 4 lines
lib/s390x/virtio-ccw.c new file +4
-0+bool virtio_ccw_init (void ) +{ +return css_enumerate (virtio_enumerate) !=0 ; +}
Thomas
Re: [kvm-unit-tests PATCH 3/7] s390x: virtio: CCW transport implementation
Quoted 6 lines · Thomas Huth
Quoted 3 lines · Pierre Morel
This is the implementation of the virtio-ccw transport level.
We only support VIRTIO revision 0.
That means only legacy virtio? Wouldn't it be better to shoot for modern virtio instead?
Yes but can we do it in a second series?
We will need more chqnges in the comon libraries.
Quoted 68 lines
Quoted 68 lines
Signed-off-by Pierre Morel <[email protected]>---
lib/s390x/virtio-ccw.c| 374 +++++++++++++++++++++++++++++++++++++++++lib/s390x/virtio-ccw.h| 111 ++++++++++++ lib/virtio-config.h | 30 ++++
s390x/Makefile | 2 +4 files changed, 517 insertions(+)create mode 100644
lib/s390x/virtio-ccw.c
create mode 100644lib/s390x/virtio-ccw.h
create mode 100644lib/virtio-config.hlib/s390x/virtio-ccw.c new file +50
-0@@ -0,0 +1,374 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* Virtio CCW Library +* +* Copyright (c) 2021 IBM Corp +* +* Authors: +* Pierre Morel <[email protected]> +* +*/ + +#include <libcflat.h> +#include <alloc_page.h> +#include <asm/page.h> +#include <string.h> +#include <interrupt.h> +#include <asm/arch_def.h> +#include <asm/facility.h> +#include <asm/uv.h> + +#include <css.h> +#include <virtio.h> +#include <virtio-config.h> +#include <virtio-ccw.h> +#include <malloc_io.h> + +static struct linked_list vcdev_list = { + .prev = &vcdev_list, + .next = &vcdev_list +}; + +static inline uint32_t swap16 (uint32_t b) +{ + return (((b & 0xff00U) << 8) | + ((b & 0x00ff) >> 8)); +} + +static inline uint32_t swap32 (uint32_t b) +{ + return (((b & 0x000000ffU) << 24) | + ((b & 0x0000ff00U) << 8) | + ((b & 0x00ff0000U) >> 8) | + ((b & 0xff000000U) >> 24)); +} + +static inline uint64_t swap64 (uint64_t x) +{ + return (((x & 0x00000000000000ffULL) << 56) | + ((x & 0x000000000000ff00ULL) << 40) | + ((x & 0x0000000000ff0000ULL) << 24) |
Quoted 8 lines
lib/s390x/virtio-ccw.c line 56
Quoted 6 lines
lib/s390x/virtio-ccw.c new file +6
-0+ ((x & 0x00000000ff000000ULL) << 8) | + ((x & 0x000000ff00000000ULL) >> 8) | + ((x & 0x0000ff0000000000ULL) >> 24) | + ((x & 0x00ff000000000000ULL) >> 40) | + ((x & 0xff00000000000000ULL) >> 56)); +}We already have macros for swapping in
lib/asm-generic/io.h... could you use those instead?
Yes, I will.
Quoted 36 lines
Quoted 36 lines
lib/s390x/virtio-ccw.c new file +6
-0+/* +* flags: flags for CCW +* Returns !0 on failure +* Returns 0 on success +*/ +int ccw_send(struct virtio_ccw_device *vcdev, int code, void *data,int count,
+ unsigned char flags)lib/s390x/virtio-ccw.c new file +27
-0+{ + struct ccw1 *ccw; + int ret = -1; + + ccw = alloc_io_mem(sizeof(*ccw), 0); + if (!ccw) + return ret; + + /* Build the CCW chain with a single CCW */ + ccw->code = code; + ccw->flags = flags; + ccw->count = count; + ccw->data_address = (unsigned long)data; + + ret = start_ccw1_chain(vcdev->schid, ccw); + if (!ret) + ret = wait_and_check_io_completion(vcdev->schid); + + free_io_mem(ccw, sizeof(*ccw)); + return ret; +} + +int virtio_ccw_set_revision (struct virtio_ccw_device *vcdev) +{ + struct virtio_rev_info *rev_info; + int ret = -1; +
Quoted 8 lines
Quoted 6 lines
+
rev_info=alloc_io_mem(sizeof(*rev_info), 0); + if (!rev_info)
+ return ret;
+
+rev_info->revision=VIRTIO_CCW_REV_MAX; +rev_info->revision= 0;Either
VIRTIO_CCW_REV_MAXor 0, but not both?
:D
Seems the second line has been forgotten.
Quoted 36 lines
Quoted 36 lines
+ do {
+ ret = ccw_send(vcdev,CCW_CMD_SET_VIRTIO_REV,rev_info, +sizeof(*rev_info), 0);
+ } while (ret && rev_info->revision--);lib/s390x/virtio-ccw.c new file +24
-0+ + free_io_mem(rev_info, sizeof(*rev_info)); + + return ret ? -1 : rev_info->revision; +} + +int virtio_ccw_reset (struct virtio_ccw_device *vcdev) +{ + return ccw_send(vcdev, CCW_CMD_VDEV_RESET, 0, 0, 0); +} + +int virtio_ccw_read_status (struct virtio_ccw_device *vcdev) +{ + return ccw_send(vcdev, CCW_CMD_READ_STATUS, &vcdev->status, + sizeof(vcdev->status), 0); +} + +int virtio_ccw_write_status (struct virtio_ccw_device *vcdev) +{ + return ccw_send(vcdev, CCW_CMD_WRITE_STATUS, &vcdev->status, + sizeof(vcdev->status), 0); +} + +int virtio_ccw_read_features(struct virtio_ccw_device *vcdev,
uint64_t*features)lib/s390x/virtio-ccw.c new file +6
-0+{ + struct virtio_feature_desc *f_desc = &vcdev->f_desc; + + f_desc->index = 0; + if (ccw_send(vcdev, CCW_CMD_READ_FEAT, f_desc, sizeof(*f_desc), 0)) + return -1;
Quoted 8 lines
Quoted 6 lines
+ *features =
swap32(f_desc->features);
+
+f_desc->index= 1;
+ if (ccw_send(vcdev, CCW_CMD_READ_FEAT, f_desc, sizeof(*f_desc), 0)) + return -1;
+ *features |= (uint64_t)swap32(f_desc->features) << 32;Weren't the upper feature bits only available for modern virtio anyway?
Yes.
I have the intention to upgrade to Rev. 1 when I get enough time for it. Should I remove this? It does not induce problem does it?
Quoted 52 lines
Quoted 52 lines
+ return 0;
lib/s390x/virtio-ccw.c new file +3
-0+} + +int virtio_ccw_write_features(struct virtio_ccw_device *vcdev,
uint64_tfeatures)lib/s390x/virtio-ccw.c new file +6
-0+{ + struct virtio_feature_desc *f_desc = &vcdev->f_desc; + + f_desc->index = 0; + f_desc->features = swap32((uint32_t)features & 0xffffffff); + if (ccw_send(vcdev, CCW_CMD_WRITE_FEAT, &f_desc, sizeof(*f_desc),0))
+ return -1;
+
+f_desc->index= 1;
+f_desc->features=swap32((uint32_t)(features >> 32) & 0xffffffff); + if (ccw_send(vcdev, CCW_CMD_WRITE_FEAT, &f_desc, sizeof(*f_desc), 0))
+ return -1;lib/s390x/virtio-ccw.c new file +32
-0+ + return 0; +} + +int virtio_ccw_read_config (struct virtio_ccw_device *vcdev) +{ + return ccw_send(vcdev, CCW_CMD_READ_CONF, &vcdev->config, + sizeof(vcdev->config), 0); +} + +int virtio_ccw_write_config (struct virtio_ccw_device *vcdev) +{ + return ccw_send(vcdev, CCW_CMD_WRITE_CONF, &vcdev->config, + sizeof(vcdev->config), 0); +} + +int virtio_ccw_setup_indicators (struct virtio_ccw_device *vcdev) +{ + vcdev->ind = alloc_io_mem(sizeof(PAGE_SIZE), 0); + if (ccw_send(vcdev, CCW_CMD_SET_IND, &vcdev->ind, + sizeof(vcdev->ind), 0)) + return -1; + + vcdev->conf_ind = alloc_io_mem(PAGE_SIZE, 0); + if (ccw_send(vcdev, CCW_CMD_SET_CONF_IND, &vcdev->conf_ind, + sizeof(vcdev->conf_ind), 0)) + return -1; + + return 0; +} + +static uint64_t virtio_ccw_notify_host(int schid, int queue, uint64_t
Quoted 8 lines
Quoted 6 lines
cookie)
+{
+ register unsigned long nrasm("1")=0x03;
+ register unsigned long sasm("2")= schid;
+ register unsigned long qasm("3")= queue;
+ register long rcasm("2");Using
asm("2")for two variables looks somewhat weird ... but ok, as long as it works... (otherwise you could also use only one variable and mark it as input + output parameter below).
OK
Quoted 177 lines
Quoted 177 lines
+ register long c
asm("4")= cookie;lib/s390x/virtio-ccw.c new file +47
-0+ + asm volatile ("diag 2,4,0x500\n" + : "=d" (rc) + : "d" (nr), "d" (s), "d" (q), "d"(c) + : "memory", "cc"); + return rc; +} + +static bool virtio_ccw_notify (struct virtqueue *vq) +{ + struct virtio_ccw_device *vcdev = to_vc_device(vq->vdev); + struct virtio_ccw_vq_info *info = vq->priv; + + info->cookie = virtio_ccw_notify_host(vcdev->schid, vq->index, + info->cookie); + if (info->cookie < 0) + return false; + return true; +} + +/* allocates a vring_virtqueue but returns a pointer to the +* virtqueue inside of it or NULL on error. +*/ +static struct virtqueue *setup_vq (struct virtio_device *vdev,int index, + void (*callback)(struct virtqueue *vq), + const char *name) +{ + struct virtio_ccw_device *vcdev = to_vc_device(vdev); + struct virtio_ccw_vq_info *info; + struct vring_virtqueue *vq; + struct vring *vr; + void *queue; + + vq = alloc_io_mem(sizeof(*vq), 0); + info = alloc_io_mem(sizeof(*info), 0); + queue = alloc_io_mem(4 * PAGE_SIZE, 0); + assert(vq && queue && info); + + info->info_block = alloc_io_mem(sizeof(*info->info_block), 0); + assert(info->info_block); + + vcdev->vq_conf.index = index; + if (ccw_send(vcdev, CCW_CMD_READ_VQ_CONF, &vcdev->vq_conf, + sizeof(vcdev->vq_conf), 0)) + return NULL; + + vring_init_virtqueue(vq, index, vcdev->vq_conf.max_num,
PAGE_SIZE, vdev,
+ queue,virtio_ccw_notify, callback, name);lib/s390x/virtio-ccw.c new file +24
-0+ + vr = &vq->vring; + info->info_block->s.desc = vr->desc; + info->info_block->s.index = index; + info->info_block->s.num = vr->num; + info->info_block->s.avail = vr->avail; + info->info_block->s.used = vr->used; + + info->info_block->l.desc = vr->desc; + info->info_block->l.index = index; + info->info_block->l.num = vr->num; + info->info_block->l.align = PAGE_SIZE; + + if (ccw_send(vcdev, CCW_CMD_SET_VQ, info->info_block, + sizeof(info->info_block->l), 0)) + return NULL; + + info->vq = &vq->vq; + vq->vq.priv = info; + + return &vq->vq; +} + +static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsignedint nvqs,
+ struct virtqueue *vqs[],vq_callback_t*callbacks[], + const char *names[])lib/s390x/virtio-ccw.c new file +54
-0+{ + int i; + + for (i = 0; i < nvqs; ++i) { + vqs[i] = setup_vq(vdev, i, + callbacks ? callbacks[i] : NULL, + names ? names[i] : ""); + if (!vqs[i]) + return -1; + } + + return 0; +} + +static void virtio_ccw_config_get (struct virtio_device *vdev, + unsigned int offset, void *buf, + unsigned int len) +{ + struct virtio_ccw_device *vcdev = to_vc_device(vdev); + + if (virtio_ccw_read_config(vcdev)) + return; + memcpy(buf, vcdev->config, len); +} + +static void virtio_ccw_config_set (struct virtio_device *vdev, + unsigned int offset, const void *buf, + unsigned int len) +{ + struct virtio_ccw_device *vcdev = to_vc_device(vdev); + + memcpy(vcdev->config, buf, len); + virtio_ccw_write_config(vcdev); +} + +static const struct virtio_config_ops virtio_ccw_ops = { + .get = virtio_ccw_config_get, + .set = virtio_ccw_config_set, + .find_vqs = virtio_ccw_find_vqs, +}; + +const struct virtio_config_ops *virtio_ccw_register (void ) +{ + return &virtio_ccw_ops; +} + +static int sense(struct virtio_ccw_device *vcdev) +{ + struct senseid *senseid; + + senseid = alloc_io_mem(sizeof(*senseid), 0); + assert(senseid); + + assert(!ccw_send(vcdev, CCW_CMD_SENSE_ID, senseid,
sizeof(*senseid), 0));lib/s390x/virtio-ccw.c new file +44
-0+ + assert(senseid->reserved == 0xff); + + vcdev->cu_type = senseid->cu_type; + vcdev->cu_model = senseid->cu_model; + vcdev->dev_type = senseid->dev_type; + vcdev->dev_model = senseid->dev_model; + + return 0; +} + +static struct virtio_ccw_device *find_vcdev_by_devid (int devid) +{ + struct virtio_ccw_device *dev; + struct linked_list *l; + + for (l = vcdev_list.next; l != &vcdev_list; l = l->next) { + dev = container_of(l, struct virtio_ccw_device, list); + if (dev->cu_model == devid) + return dev; + } + return NULL; +} + +struct virtio_device *virtio_bind (u32 devid) +{ + struct virtio_ccw_device *vcdev; + + vcdev = find_vcdev_by_devid(devid); + + return &vcdev->vdev; +} + +static int virtio_enumerate (int schid) +{ + struct virtio_ccw_device *vcdev; + + vcdev = alloc_io_mem(sizeof(*vcdev), 0); + assert(vcdev); + vcdev->schid = schid; + + list_add(&vcdev_list, &vcdev->list); + + assert(css_enable(schid, IO_SCH_ISC) == 0);
Quoted 8 lines
lib/s390x/virtio-ccw.c line 370
Quoted 6 lines
+ sense(vcdev);
lib/s390x/virtio-ccw.c new file +5
-0+ + return 0; +} + +/* Must get a param */ I don't understand that comment, could you elaborate?
No, sorry.
I forgot what I wanted to say here.
May be it comes back when I will work on the comments from you and Andrew.
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [kvm-unit-tests PATCH 3/7] s390x: virtio: CCW transport implementation
Quoted 11 lines · Pierre Morel
Quoted 6 lines · Thomas Huth
Quoted 3 lines · Pierre Morel
This is the implementation of the virtio-ccw transport level.
We only support VIRTIO revision 0.
That means only legacy virtio? Wouldn't it be better to shoot for modern virtio instead?
Yes but can we do it in a second series?
Sure.
Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
+int
virtio_ccw_read_features(struct virtio_ccw_device *vcdev, uint64_t *features)lib/s390x/virtio-ccw.c new file +6
-0+{ + struct virtio_feature_desc *f_desc = &vcdev->f_desc; + + f_desc->index = 0; + if (ccw_send(vcdev, CCW_CMD_READ_FEAT, f_desc, sizeof(*f_desc), 0)) + return -1;Quoted 8 lines
Quoted 6 lines
+ *features =
swap32(f_desc->features);
+
+f_desc->index= 1;
+ if (ccw_send(vcdev, CCW_CMD_READ_FEAT, f_desc, sizeof(*f_desc), 0)) + return -1;
+ *features |= (uint64_t)swap32(f_desc->features) << 32;Weren't the upper feature bits only available for modern virtio anyway?
Quoted 3 lines
Yes.
I have the intention to upgrade to Rev. 1 when I get enough time for it. Should I remove this? It does not induce problem does it?
No problem - maybe simply add a comment that the upper bits are for virtio 1.0 and later.
Thomas
Re: [kvm-unit-tests PATCH 3/7] s390x: virtio: CCW transport implementation
Quoted 34 lines · Thomas Huth
Quoted 11 lines · Pierre Morel
Quoted 6 lines · Thomas Huth
Quoted 3 lines · Pierre Morel
This is the implementation of the virtio-ccw transport level.
We only support VIRTIO revision 0.
That means only legacy virtio? Wouldn't it be better to shoot for modern virtio instead?
Yes but can we do it in a second series?
Sure.
Quoted 19 lines
Quoted 18 lines
Quoted 8 lines
+int
virtio_ccw_read_features(struct virtio_ccw_device *vcdev, uint64_t *features)lib/s390x/virtio-ccw.c new file +5
-0+{ + struct virtio_feature_desc *f_desc = &vcdev->f_desc; + + f_desc->index = 0; + if (ccw_send(vcdev, CCW_CMD_READ_FEAT, f_desc, sizeof(*f_desc),0))
Quoted 8 lines
+ return -1;
+ *features =swap32(f_desc->features);
+
+f_desc->index= 1;
+ if (ccw_send(vcdev, CCW_CMD_READ_FEAT, f_desc, sizeof(*f_desc), 0))
+ return -1;
+ *features |= (uint64_t)swap32(f_desc->features) << 32;Weren't the upper feature bits only available for modern virtio anyway?
Quoted 5 lines
Quoted 3 lines
Yes.
I have the intention to upgrade to Rev. 1 when I get enough time for it. Should I remove this? It does not induce problem does it?No problem - maybe simply add a comment that the upper bits are for virtio 1.0 and later.
OK,
Thanks,
Pierre
Quoted 3 lines
Thomas
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH 4/7 s390x: css: registering IRQ
In reply to Pierre MorelRegistering IRQ for the CSS level.
--- lib/s390x/css.h | 21 +++++++++++++++++++++ lib/s390x/css_lib.c | 27 +++++++++++++++++++++++++-- 2 files changed, 46 insertions(+), 2 deletions(-)
lib/s390x/css.h +21 -0
@@ -402,4 +402,25 @@ struct measurement_block_format1 { uint32_t irq_prio_delay_time; }; +#include <asm/arch_def.h> +static inline void disable_io_irq (void ) +{ +uint64_t mask; + + mask =extract_psw_mask (); + mask &= ~PSW_MASK_IO ; +load_psw_mask (mask); +} + +static inline void enable_io_irq (void ) +{ +uint64_t mask; + + mask =extract_psw_mask (); + mask |=PSW_MASK_IO ; +load_psw_mask (mask); +} + +int register_css_irq_func (void (*f)(void )); +int unregister_css_irq_func (void (*f)(void ));#endif
lib/s390x/css_lib.c +25 -2
@@ -350,8 +350,29 @@ bool css_disable_mb(int schid)return retry_count >0 ; } -static struct irb irb; +static void (*css_irq_func)(void ); + +int register_css_irq_func (void (*f)(void )) +{ +if (css_irq_func) +return -1 ; + css_irq_func = f; +assert (register_io_int_func (css_irq_io) ==0 ); +enable_io_isc (0x80 >>IO_SCH_ISC ); +return 0 ; +} +int unregister_css_irq_func (void (*f)(void )) +{ +if (css_irq_func != f) +return -1 ; +enable_io_isc (0 ); +unregister_io_int_func (css_irq_io); + css_irq_func =NULL ; +return 0 ; +} + +static struct irb irb;void css_irq_io (void ) {int ret =0 ;
@@ -386,7 +407,9 @@ void css_irq_io(void)report (0 ,"tsch reporting sch %08x as not operational" , sid);break ;case 0 : -/* Stay humble on success */ +/* Call upper level IRQ routine */ +if (css_irq_func) +css_irq_func ();break ; } pop:
Signature · 1 line
2.25.1
Re: [kvm-unit-tests PATCH 4/7] s390x: css: registering IRQ
Quoted 45 lines · Pierre Morel
Registering IRQ for the CSS level.
Signed-off-by Pierre Morel <[email protected]>--- lib/s390x/css.h | 21 +++++++++++++++++++++ lib/s390x/css_lib.c | 27 +++++++++++++++++++++++++-- 2 files changed, 46 insertions(+), 2 deletions(-)lib/s390x/css.h +21
-0@@ -402,4 +402,25 @@ struct measurement_block_format1 { uint32_t irq_prio_delay_time; }; +#include <asm/arch_def.h> +static inline void disable_io_irq (void ) +{ +uint64_t mask; + + mask =extract_psw_mask (); + mask &= ~PSW_MASK_IO ; +load_psw_mask (mask); +} + +static inline void enable_io_irq (void ) +{ +uint64_t mask; + + mask =extract_psw_mask (); + mask |=PSW_MASK_IO ; +load_psw_mask (mask); +} + +int register_css_irq_func (void (*f)(void )); +int unregister_css_irq_func (void (*f)(void ));#endif lib/s390x/css_lib.c +0
-0@@ -350,8 +350,29 @@ bool css_disable_mb(int schid)return retry_count >0 ; }
lib/s390x/css_lib.c bool css_disable_mb(int schid) · line 360
Quoted 10 lines
lib/s390x/css_lib.c +8
-1-static struct irb irb; +static void (*css_irq_func)(void ); + +int register_css_irq_func (void (*f)(void )) +{ +if (css_irq_func) +return -1 ; + css_irq_func = f; +assert (register_io_int_func (css_irq_io) ==0 );
It's unlikely that we ever disable assert() in the k-u-t, but anyway, I'd prefer not to put function calls within assert() statements, just in case. So could you please replace this with:
rc = register_io_int_func(css_irq_io); assert(rc == 0);
?
Quoted 20 lines
lib/s390x/css_lib.c +14
-0+enable_io_isc (0x80 >>IO_SCH_ISC ); +return 0 ; +} +int unregister_css_irq_func (void (*f)(void )) +{ +if (css_irq_func != f) +return -1 ; +enable_io_isc (0 ); +unregister_io_int_func (css_irq_io); + css_irq_func =NULL ; +return 0 ; +} + +static struct irb irb;void css_irq_io (void ) {int ret =0 ;
lib/s390x/css_lib.c void css_irq_io(void) · line 415
Quoted 10 lines
lib/s390x/css_lib.c +3
-1@@ -386,7 +407,9 @@ void css_irq_io(void)report (0 ,"tsch reporting sch %08x as not operational" , sid);break ;case 0 : -/* Stay humble on success */ +/* Call upper level IRQ routine */ +if (css_irq_func) +css_irq_func ();break ; } pop:
Re: [kvm-unit-tests PATCH 4/7] s390x: css: registering IRQ
Quoted 57 lines · Thomas Huth
Quoted 43 lines · Pierre Morel
Registering IRQ for the CSS level.
Signed-off-by Pierre Morel <[email protected]>---
lib/s390x/css.h | 21 +++++++++++++++++++++lib/s390x/css_lib.c| 27 +++++++++++++++++++++++++--2 files changed, 46 insertions(+), 2 deletions(-)lib/s390x/css.h +21
-0@@ -402,4 +402,25 @@ struct measurement_block_format1 { uint32_t irq_prio_delay_time; }; +#include <asm/arch_def.h> +static inline void disable_io_irq (void ) +{ + uint64_t mask; + + mask = extract_psw_mask(); + mask &= ~PSW_MASK_IO; + load_psw_mask(mask); +} + +static inline void enable_io_irq (void ) +{ + uint64_t mask; + + mask = extract_psw_mask(); + mask |= PSW_MASK_IO; + load_psw_mask(mask); +} + +int register_css_irq_func (void (*f)(void )); +int unregister_css_irq_func (void (*f)(void )); #endiflib/s390x/css_lib.c +0
-0@@ -350,8 +350,29 @@ bool css_disable_mb(int schid) return retry_count > 0;lib/s390x/css_lib.c bool css_disable_mb(int schid) · line 359
Quoted 10 lines
lib/s390x/css_lib.c +8
-1} -static struct irb irb; +static void (*css_irq_func)(void ); + +int register_css_irq_func (void (*f)(void )) +{ + if (css_irq_func) + return -1; + css_irq_func = f; + assert(register_io_int_func(css_irq_io) == 0);It's unlikely that we ever disable
assert()in the k-u-t, but anyway, I'd prefer not to put function calls withinassert()statements, just in case. So could you please replace this with:
Quoted 3 lines
rc =
register_io_int_func(css_irq_io);
assert(rc == 0);Quoted 1 line
?
OK
Quoted 24 lines
Quoted 24 lines
+
enable_io_isc(0x80 >> IO_SCH_ISC);
+ return 0;lib/s390x/css_lib.c +12
-0+} +int unregister_css_irq_func (void (*f)(void )) +{ + if (css_irq_func != f) + return -1; + enable_io_isc(0); + unregister_io_int_func(css_irq_io); + css_irq_func = NULL; + return 0; +} + +static struct irb irb;void
css_irq_io(void)
{
int ret = 0;lib/s390x/css_lib.c +0
-0@@ -386,7 +407,9 @@ void css_irq_io(void) report(0, "tsch reporting sch %08x as not operational", sid); break;case 0:
- /* Stay humble on success */
+ /* Call upper level IRQ routine */
Quoted 9 lines
Quoted 6 lines
+ if (
css_irq_func)
+css_irq_func();
break;
}
pop:Reviewed-by Thomas Huth <[email protected]>
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH 5/7 virtio: implement the virtio_add_inbuf routine
In reply to Pierre MorelTo communicate in both directions with a VIRTIO device we need to add the incoming communication to the VIRTIO level.
--- lib/virtio.c | 32 ++++++++++++++++++++++++++++++++ lib/virtio.h | 2 ++ 2 files changed, 34 insertions(+)
lib/virtio.c +32 -0
@@ -47,6 +47,38 @@ void vring_init_virtqueue(struct vring_virtqueue *vq, unsigned index, vq->data[i] =NULL ; } +int virtqueue_add_inbuf (struct virtqueue *_vq,char *buf,unsigned int len) +{ +struct vring_virtqueue *vq =to_vvq (_vq); +unsigned int avail; +int head; + +assert (buf); +assert (len); + +if (!vq->vq.num_free) +return -1 ; + + --vq->vq.num_free; + + head = vq->free_head; + + vq->vring.desc[head].flags =0 ; + vq->vring.desc[head].addr =virt_to_phys (buf); + vq->vring.desc[head].len = len; + + vq->free_head = vq->vring.desc[head].next; + + vq->data[head] = buf; + + avail = (vq->vring.avail->idx & (vq->vring.num -1 )); + vq->vring.avail->ring[avail] = head; +wmb ();/* be sure to update the ring before updating the idx */ + vq->vring.avail->idx++; + vq->num_added++; + +return 0 ; +}int virtqueue_add_outbuf (struct virtqueue *_vq,char *buf,unsigned int len) {struct vring_virtqueue *vq =to_vvq (_vq);
lib/virtio.h +2 -0
@@ -141,6 +141,8 @@ extern void vring_init_virtqueue(struct vring_virtqueue *vq, unsigned index, const char *name);extern int virtqueue_add_outbuf (struct virtqueue *vq,char *buf,unsigned int len); +extern int virtqueue_add_inbuf (struct virtqueue *vq,char *buf, +unsigned int len);extern bool virtqueue_kick (struct virtqueue *vq);extern void detach_buf (struct vring_virtqueue *vq,unsigned head);extern void *virtqueue_get_buf (struct virtqueue *_vq,unsigned int *len);
Signature · 1 line
2.25.1
Re: [kvm-unit-tests PATCH 5/7] virtio: implement the virtio_add_inbuf routine
Quoted 68 lines · Pierre Morel
To communicate in both directions with a VIRTIO device we need to add the incoming communication to the VIRTIO level.
Signed-off-by Pierre Morel <[email protected]>--- lib/virtio.c | 32 ++++++++++++++++++++++++++++++++ lib/virtio.h | 2 ++ 2 files changed, 34 insertions(+)lib/virtio.c +32
-0@@ -47,6 +47,38 @@ void vring_init_virtqueue(struct vring_virtqueue *vq, unsigned index, vq->data[i] =NULL ; } +int virtqueue_add_inbuf (struct virtqueue *_vq,char *buf,unsigned int len) +{ +struct vring_virtqueue *vq =to_vvq (_vq); +unsigned int avail; +int head; + +assert (buf); +assert (len); + +if (!vq->vq.num_free) +return -1 ; + + --vq->vq.num_free; + + head = vq->free_head; + + vq->vring.desc[head].flags =0 ; + vq->vring.desc[head].addr =virt_to_phys (buf); + vq->vring.desc[head].len = len; + + vq->free_head = vq->vring.desc[head].next; + + vq->data[head] = buf; + + avail = (vq->vring.avail->idx & (vq->vring.num -1 )); + vq->vring.avail->ring[avail] = head; +wmb ();/* be sure to update the ring before updating the idx */ + vq->vring.avail->idx++; + vq->num_added++; + +return 0 ; +}int virtqueue_add_outbuf (struct virtqueue *_vq,char *buf,unsigned int len) {struct vring_virtqueue *vq =to_vvq (_vq);lib/virtio.h +2
-0@@ -141,6 +141,8 @@ extern void vring_init_virtqueue(struct vring_virtqueue *vq, unsigned index, const char *name);extern int virtqueue_add_outbuf (struct virtqueue *vq,char *buf,unsigned int len); +extern int virtqueue_add_inbuf (struct virtqueue *vq,char *buf, +unsigned int len);extern bool virtqueue_kick (struct virtqueue *vq);extern void detach_buf (struct vring_virtqueue *vq,unsigned head);extern void *virtqueue_get_buf (struct virtqueue *_vq,unsigned int *len);Signature · 1 line
2.25.1
Re: [kvm-unit-tests PATCH 5/7] virtio: implement the virtio_add_inbuf routine
Quoted 63 lines · Andrew Jones
Quoted 62 lines · Pierre Morel
To communicate in both directions with a VIRTIO device we need to add the incoming communication to the VIRTIO level.
Signed-off-by Pierre Morel <[email protected]>--- lib/virtio.c | 32 ++++++++++++++++++++++++++++++++ lib/virtio.h | 2 ++ 2 files changed, 34 insertions(+)lib/virtio.c +32
-0@@ -47,6 +47,38 @@ void vring_init_virtqueue(struct vring_virtqueue *vq, unsigned index, vq->data[i] =NULL ; } +int virtqueue_add_inbuf (struct virtqueue *_vq,char *buf,unsigned int len) +{ +struct vring_virtqueue *vq =to_vvq (_vq); +unsigned int avail; +int head; + +assert (buf); +assert (len); + +if (!vq->vq.num_free) +return -1 ; + + --vq->vq.num_free; + + head = vq->free_head; + + vq->vring.desc[head].flags =0 ; + vq->vring.desc[head].addr =virt_to_phys (buf); + vq->vring.desc[head].len = len; + + vq->free_head = vq->vring.desc[head].next; + + vq->data[head] = buf; + + avail = (vq->vring.avail->idx & (vq->vring.num -1 )); + vq->vring.avail->ring[avail] = head; +wmb ();/* be sure to update the ring before updating the idx */ + vq->vring.avail->idx++; + vq->num_added++; + +return 0 ; +}int virtqueue_add_outbuf (struct virtqueue *_vq,char *buf,unsigned int len) {struct vring_virtqueue *vq =to_vvq (_vq);lib/virtio.h +2
-0@@ -141,6 +141,8 @@ extern void vring_init_virtqueue(struct vring_virtqueue *vq, unsigned index, const char *name);extern int virtqueue_add_outbuf (struct virtqueue *vq,char *buf,unsigned int len); +extern int virtqueue_add_inbuf (struct virtqueue *vq,char *buf, +unsigned int len);
Quoted 9 lines
Quoted 6 lines
lib/virtio.h +0
-0extern bool virtqueue_kick (struct virtqueue *vq);extern void detach_buf (struct vring_virtqueue *vq,unsigned head);extern void *virtqueue_get_buf (struct virtqueue *_vq,unsigned int *len);Signature · 1 line
2.25.1Reviewed-by Andrew Jones <[email protected]>
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH 6/7 s390x: virtio tests setup
In reply to Pierre MorelThis patch can be squatch with the next one, "s390x: virtio data transfer" and is separated to ease review.
In this patch we initialize the VIRTIO device.
There are currently no error insertion, the goal is to get an initialized device to check data transfer within the next patch.
Future development will include error response checks.
--- s390x/Makefile | 1 + s390x/unittests.cfg | 4 + s390x/virtio_pong.c | 208 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 213 insertions(+) create mode 100644 s390x/virtio_pong.c
s390x/Makefile +1 -0
@@ -24,6 +24,7 @@ tests += $(TEST_DIR)/mvpg.elf tests += $(TEST_DIR)/uv-host.elf tests += $(TEST_DIR)/edat.elf tests += $(TEST_DIR)/mvpg-sie.elf +tests += $(TEST_DIR)/virtio_pong.elf tests_binary = $(patsubst %.elf,%.bin,$(tests)) ifneq ($(HOST_KEY_DOCUMENT),)
s390x/unittests.cfg +4 -0
@@ -109,3 +109,7 @@ file = edat.elf [mvpg-sie] file = mvpg-sie.elf + +[virtio-pong] +file = uv-virtio.elf +extra_params = -device virtio-pong-cww
s390x/virtio_pong.c new file +208 -0
208 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [kvm-unit-tests PATCH 6/7] s390x: virtio tests setup
Review
Quoted 2 lines · Pierre Morel
(quoted diff) +2
-0+ +#define VIRTIO_ID_PONG 30 /* virtio pong */
I take it this is a virtio test device that ping-pong's I/O. It sounds useful for other VIRTIO transports too. Can it be ported? Hmm, I can't find it in QEMU at all?
Thanks,
drew
Re: [kvm-unit-tests PATCH 6/7] s390x: virtio tests setup
Quoted 7 lines · Andrew Jones
Review
Quoted 2 lines · Pierre Morel
(quoted diff) +2
-0+ +#define VIRTIO_ID_PONG 30 /* virtio pong */I take it this is a virtio test device that ping-pong's I/O. It sounds useful for other VIRTIO transports too. Can it be ported? Hmm, I can't find it in QEMU at all?
I also wonder whether we could do testing with an existing device instead? E.g. do a loopback with a virtio-serial device? Or use two virtio-net devices, connect them to a QEMU hub and send a packet from one device to the other? ... that would be a little bit more complicated here, but would not require a PONG device upstream first, so it could also be used for testing older versions of QEMU...
Thomas
Re: [kvm-unit-tests PATCH 6/7] s390x: virtio tests setup
Quoted 9 lines · Thomas Huth
Quoted 7 lines · Andrew Jones
Review
Quoted 2 lines · Pierre Morel
(quoted diff) +2
-0+ +#define VIRTIO_ID_PONG 30 /* virtio pong */I take it this is a virtio test device that ping-pong's I/O. It sounds useful for other VIRTIO transports too. Can it be ported? Hmm, I can't find it in QEMU at all?
Quoted 5 lines
I also wonder whether we could do testing with an existing device instead? E.g. do a loopback with a virtio-serial device? Or use two virtio-net devices, connect them to a QEMU hub and send a packet from one device to the other? ... that would be a little bit more complicated here, but would not require a PONG device upstream first, so it could also be used for testing older versions of QEMU...
Thomas
Yes having a dedicated device has the drawback that we need it in QEMU.
On the other hand using a specific device, serial or network, wouldn't we get trapped with a reduce set of test possibilities?
The idea was to have a dedicated test device, which could be flexible and extended to test all VIRTIO features, even the current implementation is yet far from it.
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [kvm-unit-tests PATCH 6/7] s390x: virtio tests setup
Quoted 14 lines · Pierre Morel
Quoted 11 lines · Thomas Huth
Quoted 7 lines · Andrew Jones
Review
Quoted 2 lines · Pierre Morel
(quoted diff) +2
-0+ +#define VIRTIO_ID_PONG 30 /* virtio pong */I take it this is a virtio test device that ping-pong's I/O. It sounds useful for other VIRTIO transports too. Can it be ported? Hmm, I can't find it in QEMU at all?
I also wonder whether we could do testing with an existing device instead? E.g. do a loopback with a virtio-serial device? Or use two virtio-net devices, connect them to a QEMU hub and send a packet from one device to the other? ... that would be a little bit more complicated here, but would not require a PONG device upstream first, so it could also be used for testing older versions of QEMU...
Quoted 8 lines
Quoted 3 lines
Thomas
Yes having a dedicated device has the drawback that we need it in QEMU.
On the other hand using a specific device, serial or network, wouldn't we get trapped with a reduce set of test possibilities?The idea was to have a dedicated test device, which could be flexible and extended to test all VIRTIO features, even the current implementation is yet far from it.
Do you have anything in the works that could only be tested with a dedicated test device? If not, I'd rather go with the loopback via virtio-net, I think (you can peek into the s390-ccw bios sources to see how to send packets via virtio-net, shouldn't be too hard to do, I think).
The pong device could later be added on top for additional tests that are not possible with virtio-net. And having some basic tests with virito-net has also the advantage that the k-u-t work with QEMU binaries where the pong device is not available, e.g. older versions and downstream versions that only enable the bare minimum of devices to keep the attack surface small.
Thomas
Re: [kvm-unit-tests PATCH 6/7] s390x: virtio tests setup
Quoted 38 lines · Thomas Huth
Quoted 24 lines · Pierre Morel
Quoted 17 lines · Thomas Huth
Quoted 7 lines · Andrew Jones
Review
Quoted 2 lines · Pierre Morel
(quoted diff) +2
-0+ +#define VIRTIO_ID_PONG 30 /* virtio pong */I take it this is a virtio test device that ping-pong's I/O. It sounds useful for other VIRTIO transports too. Can it be ported? Hmm, I can't find it in QEMU at all?
I also wonder whether we could do testing with an existing device instead? E.g. do a loopback with a virtio-serial device? Or use two virtio-net devices, connect them to a QEMU hub and send a packet from one device to the other? ... that would be a little bit more complicated here, but would not require a PONG device upstream first, so it could also be used for testing older versions of QEMU...
Quoted 3 lines
Thomas
Quoted 7 lines
Yes having a dedicated device has the drawback that we need it in QEMU. On the other hand using a specific device, serial or network, wouldn't we get trapped with a reduce set of test possibilities?
The idea was to have a dedicated test device, which could be flexible and extended to test all VIRTIO features, even the current implementation is yet far from it.
Do you have anything in the works that could only be tested with a dedicated test device? If not, I'd rather go with the loopback via virtio-net, I think (you can peek into the s390-ccw bios sources to see how to send packets via virtio-net, shouldn't be too hard to do, I think).
Quoted 6 lines
The pong device could later be added on top for additional tests that are not possible with virtio-net. And having some basic tests with virito-net has also the advantage that the k-u-t work with QEMU binaries where the pong device is not available, e.g. older versions and downstream versions that only enable the bare minimum of devices to keep the attack surface small.
I'd also like to see the testdev we already have, qemu:chardev/testdev.c, get more functions, but I'm not sure virtio-serial will allow you to exercise all the virtio functionality that you'd like to.
Thanks,
drew
Re: [kvm-unit-tests PATCH 6/7] s390x: virtio tests setup
Quoted 46 lines · Andrew Jones
Quoted 38 lines · Thomas Huth
Quoted 24 lines · Pierre Morel
Quoted 17 lines · Thomas Huth
Quoted 7 lines · Andrew Jones
Review
Quoted 2 lines · Pierre Morel
(quoted diff) +2
-0+ +#define VIRTIO_ID_PONG 30 /* virtio pong */I take it this is a virtio test device that ping-pong's I/O. It sounds useful for other VIRTIO transports too. Can it be ported? Hmm, I can't find it in QEMU at all?
I also wonder whether we could do testing with an existing device instead? E.g. do a loopback with a virtio-serial device? Or use two virtio-net devices, connect them to a QEMU hub and send a packet from one device to the other? ... that would be a little bit more complicated here, but would not require a PONG device upstream first, so it could also be used for testing older versions of QEMU...
Quoted 3 lines
Thomas
Quoted 7 lines
Yes having a dedicated device has the drawback that we need it in QEMU. On the other hand using a specific device, serial or network, wouldn't we get trapped with a reduce set of test possibilities?
The idea was to have a dedicated test device, which could be flexible and extended to test all VIRTIO features, even the current implementation is yet far from it.
Do you have anything in the works that could only be tested with a dedicated test device? If not, I'd rather go with the loopback via virtio-net, I think (you can peek into the s390-ccw bios sources to see how to send packets via virtio-net, shouldn't be too hard to do, I think).
Quoted 6 lines
The pong device could later be added on top for additional tests that are not possible with virtio-net. And having some basic tests with virito-net has also the advantage that the k-u-t work with QEMU binaries where the pong device is not available, e.g. older versions and downstream versions that only enable the bare minimum of devices to keep the attack surface small.
Quoted 7 lines
I'd also like to see the testdev we already have, qemu:chardev/testdev.c, get more functions, but I'm not sure virtio-serial will allow you to exercise all the virtio functionality that you'd like to.
Thanks,
drew
Yes, that is why I did not used it first.
But OK, I understand what you both want and will build something in that direction, virtio-net, virtio-serial and come back later to something independent of existing devices if we find it does have a purpose. Thanks for the comments.
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [kvm-unit-tests PATCH 6/7] s390x: virtio tests setup
In reply to Andrew JonesQuoted 11 lines · Andrew Jones
Review
Quoted 2 lines · Pierre Morel
(quoted diff) +2
-0+ +#define VIRTIO_ID_PONG 30 /* virtio pong */I take it this is a virtio test device that ping-pong's I/O. It sounds useful for other VIRTIO transports too. Can it be ported? Hmm, I can't find it in QEMU at all?
Thanks,
drew
It could certainly be ported, I will study the question.
I sent a first version of the QEMU part here:
https://marc.info/?l=kvm&m=163006146622427&w=3
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH 7/7 s390x: virtio data transfer
In reply to Pierre MorelWe use a test device, "PONG" to transfer chunks of data of different size and alignment and compare a checksum calculated before the sending with the VIRTIO device checksum response.
--- s390x/virtio_pong.c | 107 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+)
s390x/virtio_pong.c +107 -0
@@ -39,6 +39,112 @@ static struct virtio_ccw_device *vcdev;static struct virtqueue *out_vq;static struct virtqueue *in_vq; +static bool virtio_read (struct virtqueue *q,char **buf,unsigned int *len) +{ +int ret; +char *p; + + ret =virtqueue_add_inbuf (q, *buf, *len); +if (ret <0 ) +return false; + +disable_io_irq (); +virtqueue_kick (q); +wait_for_interrupt (PSW_MASK_IO ); + +do { + p =virtqueue_get_buf (q, len); + }while (!p); + + *buf = (void *)p; + +return true; +} + +static bool virtio_write (struct virtqueue *q,char *buf,unsigned int len) +{ +int ret; + + ret =virtqueue_add_outbuf (q, buf, len); +if (ret <0 ) +return false; + +virtqueue_kick (q); +while (!virtqueue_get_buf (q, &len)) + ; + +return true; +} + +static unsigned int simple_checksum (char *buf,unsigned int len) +{ +unsigned int sum =0 ; + +while (len--) { + sum += *buf * *buf +7 * *buf +3 ; + buf++; + } + +return sum; +} + +static void pong_write (char *buf,unsigned int len) +{ +unsigned int cksum; +unsigned int cksum_ret; +char *ret_ptr = (char *)&cksum_ret; + + cksum =simple_checksum (buf, len); +report (virtio_write (out_vq, buf, len),"Sending data: %08x" , cksum); + + len =sizeof (cksum_ret); +report (virtio_read (in_vq, &ret_ptr, &len), +"Receiving checksum: %08x" , cksum_ret); + +report (cksum == cksum_ret,"Verifying checksum" ); +} + +static struct { +const char *name; +int size; +int offset; +} chunks[] = { + {"Small buffer" ,3 ,0 }, + {"Page aligned" ,4096 ,0 }, + {"Large page aligned" ,0x00100000 ,0 }, + {"Page unaligned" ,4096 ,0x107 }, + {"Random data" ,5119 ,0x107 }, + {NULL ,0 ,0 } +}; + +static void test_pong_data (void ) +{ +char *buf; +char *p; +int len; +int i; + +if (vcdev->state !=VCDEV_INIT ) { +report_skip ("Device non initialized" ); +return ; + } + +for (i =0 ; chunks[i].name; i++) { +report_prefix_push (chunks[i].name); + + len = chunks[i].size + chunks[i].offset; + buf =alloc_io_mem (len,0 ); + + p = buf + chunks[i].offset; +memset (p,0xA5 , chunks[i].size); +pong_write (p, chunks[i].size); + +free_io_mem (buf, len); + +report_prefix_pop (); + } +} +static void test_find_vqs (void ) {struct virtio_device *vdev = &vcdev->vdev;
@@ -186,6 +292,7 @@ static struct { { "CCW Bus", test_virtio_ccw_bus }, { "CCW Device", test_virtio_device_init }, { "Queues setup", test_find_vqs }, + { "Data transfer", test_pong_data }, { NULL, NULL } };
Signature · 1 line
2.25.1