s390: virtio: support protected virtualization
Subsystems (6)
Halil Pasic
maintainerv4 cover s390: virtio: support protected virtualization
Enhanced virtualization protection technology may require the use of bounce buffers for I/O. While support for this was built into the virtio core, virtio-ccw wasn't changed accordingly.
Some background on technology (not part of this series) and the terminology used.
* Protected Virtualization (PV):
Protected Virtualization guarantees, that non-shared memory of a guest that operates in PV mode private to that guest. I.e. any attempts by the hypervisor or other guests to access it will result in an exception. If supported by the environment (machine, KVM, guest VM) a guest can decide to change into PV mode by doing the appropriate ultravisor calls.
* Ultravisor:
A hardware/firmware entity that manages PV guests, and polices access to their memory. A PV guest prospect needs to interact with the ultravisor, to enter PV mode, and potentially to share pages (for I/O which should be encrypted by the guest). A guest interacts with the ultravisor via so called ultravisor calls. A hypervisor needs to interact with the ultravisor to facilitate interpretation, emulation and swapping. A hypervisor interacts with the ultravisor via ultravisor calls and via the SIE state description. Generally the ultravisor sanitizes hypervisor inputs so that the guest can not be corrupted (except for denial of service.
What needs to be done
=====================
Thus what needs to be done to bring virtio-ccw up to speed with respect to protected virtualization is:
* use some 'new' common virtio stuff
* make sure that virtio-ccw specific stuff uses shared memory when talking to the hypervisor (except control/communication blocks like ORB, these are handled by the ultravisor)
* make sure the DMA API does what is necessary to talk through shared memory if we are a protected virtualization guest.
* make sure the common IO layer plays along as well (airqs, sense).
Important notes
================
* This patch set is based on Martins features branch
(git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux.git branch 'features').
* Documentation is still very sketchy. I'm committed to improving this, but I'm currently hampered by some dependencies currently.
* The existing naming in the common infrastructure (kernel internal interfaces) is pretty much based on the AMD SEV terminology. Thus the names aren't always perfect. There might be merit to changing these names to more abstract ones. I did not put much thought into that at the current stage.
* Testing: Please use iommu_platform=on for any virtio devices you are going to test this code with (so virtio actually uses the DMA API).
@Sebastian: I kept your r-b on patch 2 "s390/cio: introduce DMA pools to cio" despite the small changes pointed out below. Please do complain if it ain't OK for you!
Change log
==========
v3 --> v4
* fixed cleanup in css_bus_init() (Connie)
* made cio.h include genalloc.h instead of a forward declaration (Connie)
* added comments about dma_mask/coherent_dma_mask values (Connie)
* fixed error handling in virtio_ccw_init() (Connie)
* got rid of the *vc_dma* wrappers (Connie)
* added some Reviewed-bys
* rebased on top of current master, no changes were necessary
v2 --> v3:
* patch 2/8
potential cio_dma_pool_init() returning NULL issue fixed potential cio_gp_dma_create() returning NULL issue fixed warning issues with doc type comments fixed unused define statement removed
* patch 3/8
potential cio_gp_dma_create() returning NULL issue fixed whitespace issue fixed
warning issues with doc type comments fixed
* patch 8/8
potential cio_dma_zalloc() returning NULL issue fixed
v1 --> v2:
* patch "virtio/s390: use vring_create_virtqueue" went already upstream
* patch "virtio/s390: DMA support for virtio-ccw" went already upstream
* patch "virtio/s390: enable packed ring" went already upstream
* Made dev.dma_mask point to dev.coherent_dma_mask for css, subchannel and ccw devices.
* While rebasing 's390/airq: use DMA memory for adapter interrupts' the newly introduced kmem_cache was replaced with an equivalent dma_pool; the kalloc() allocations are now replaced with cio_dma_zalloc() allocations to avoid wasting almost a full page.
* Made virtio-ccw use the new AIRQ_IV_CACHELINE flag.
* fixed all remaining checkpatch issues
RFC --> v1:
* Fixed bugs found by Connie (may_reduce and handling reduced, warning, split move -- thanks Connie!).
* Fixed console bug found by Sebastian (thanks Sebastian!).
* Removed the completely useless duplicate of dma-mapping.h spotted by Christoph (thanks Christoph!).
* Don't use the global DMA pool for subchannel and ccw device owned memory as requested by Sebastian. Consequences:
* Both subchannel and ccw devices have their dma masks now (both specifying 31 bit addressable)
* We require at least 2 DMA pages per ccw device now, most of this memory is wasted though.
* DMA memory allocated by virtio is also 31 bit addressable now as virtio uses the parent (which is the ccw device).
* Enabled packed ring.
* Rebased onto Martins feature branch; using the actual uv (ultravisor) interface instead of TODO comments.
* Added some explanations to the cover letter (Connie, David).
* Squashed a couple of patches together and fixed some text stuff.
Halil Pasic (8):
s390/mm: force swiotlb for protected virtualization s390/cio: introduce DMA pools to cio
s390/cio: add basic protected virtualization support s390/airq: use DMA memory for adapter interrupts virtio/s390: use cacheline aligned airq bit vectors virtio/s390: add indirection to indicators access virtio/s390: use DMA memory for ccw I/O and classic notifiers virtio/s390: make airq summary indicators DMA
arch/s390/Kconfig | 5 + arch/s390/include/asm/airq.h | 2 + arch/s390/include/asm/ccwdev.h | 4 + arch/s390/include/asm/cio.h | 11 ++ arch/s390/include/asm/mem_encrypt.h | 18 ++ arch/s390/mm/init.c | 47 ++++++ drivers/s390/cio/airq.c | 32 ++-- drivers/s390/cio/ccwreq.c | 9 +- drivers/s390/cio/cio.h | 2 + drivers/s390/cio/css.c | 132 ++++++++++++++- drivers/s390/cio/device.c | 68 ++++++-- drivers/s390/cio/device_fsm.c | 49 +++--- drivers/s390/cio/device_id.c | 20 ++- drivers/s390/cio/device_ops.c | 21 ++- drivers/s390/cio/device_pgid.c | 22 +-- drivers/s390/cio/device_status.c | 24 +-- drivers/s390/cio/io_sch.h | 20 ++- drivers/s390/virtio/virtio_ccw.c | 248 +++++++++++++++------------- 18 files changed, 534 insertions(+), 200 deletions(-) create mode 100644 arch/s390/include/asm/mem_encrypt.h
Signature · 1 line
2.17.1
Halil Pasic
maintainerPATCH v4 1/8 s390/mm: force swiotlb for protected virtualization
On s390, protected virtualization guests have to use bounced I/O buffers. That requires some plumbing.
Let us make sure, any device that uses DMA API with direct ops correctly is spared from the problems, that a hypervisor attempting I/O to a non-shared page would bring.
--- arch/s390/Kconfig | 4 +++ arch/s390/include/asm/mem_encrypt.h | 18 +++++++++++ arch/s390/mm/init.c | 47 +++++++++++++++++++++++++++++ 3 files changed, 69 insertions(+) create mode 100644 arch/s390/include/asm/mem_encrypt.h
arch/s390/Kconfig +4 -0
@@ -1,4 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 +config ARCH_HAS_MEM_ENCRYPT + def_bool y + config MMU def_bool y
@@ -187,6 +190,7 @@ config S390 select VIRT_CPU_ACCOUNTING select ARCH_HAS_SCALED_CPUTIME select HAVE_NMI + select SWIOTLB config SCHED_OMIT_FRAME_POINTER
arch/s390/include/asm/mem_encrypt.h new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef S390_MEM_ENCRYPT_H__ +#define S390_MEM_ENCRYPT_H__ + +#ifndef __ASSEMBLY__ + +#define sme_me_mask 0ULL + +static inline bool sme_active (void ) {return false; } +extern bool sev_active (void ); + +int set_memory_encrypted (unsigned long addr,int numpages); +int set_memory_decrypted (unsigned long addr,int numpages); + +#endif /* __ASSEMBLY__ */ + +#endif /* S390_MEM_ENCRYPT_H__ */ +
arch/s390/mm/init.c +47 -0
77 lines not on this page · raw message
Signature · 1 line
2.17.1
Halil Pasic
maintainerPATCH v4 2/8 s390/cio: introduce DMA pools to cio
In reply to Halil PasicTo support protected virtualization cio will need to make sure the memory used for communication with the hypervisor is DMA memory.
Let us introduce one global pool for cio.
Our DMA pools are implemented as a gen_pool backed with DMA pages. The idea is to avoid each allocation effectively wasting a page, as we typically allocate much less than PAGE_SIZE.
--- arch/s390/Kconfig | 1 + arch/s390/include/asm/cio.h | 11 +++ drivers/s390/cio/css.c | 131 ++++++++++++++++++++++++++++++++++-- 3 files changed, 139 insertions(+), 4 deletions(-)
arch/s390/Kconfig +1 -0
@@ -191,6 +191,7 @@ config S390 select ARCH_HAS_SCALED_CPUTIME select HAVE_NMI select SWIOTLB + select GENERIC_ALLOCATOR config SCHED_OMIT_FRAME_POINTER
arch/s390/include/asm/cio.h +11 -0
@@ -7,6 +7,7 @@#include <linux/spinlock.h> #include <linux/bitops.h> +#include <linux/genalloc.h> #include <asm/types.h> #define LPM_ANYPATH 0xff
@@ -328,6 +329,16 @@ static inline u8 pathmask_to_pos(u8 mask)void channel_subsystem_reinit (void );extern void css_schedule_reprobe (void ); +extern void *cio_dma_zalloc (size_t size); +extern void cio_dma_free (void *cpu_addr,size_t size); +extern struct device *cio_get_dma_css_dev (void ); + +void *cio_gp_dma_zalloc (struct gen_pool *gp_dma,struct device *dma_dev, +size_t size); +void cio_gp_dma_free (struct gen_pool *gp_dma,void *cpu_addr,size_t size); +void cio_gp_dma_destroy (struct gen_pool *gp_dma,struct device *dma_dev); +struct gen_pool *cio_gp_dma_create (struct device *dma_dev,int nr_pages); +/* Function from drivers/s390/cio/chsc.c */ int chsc_sstpc (void *page,unsigned int op,u16 ctrl,u64 *clock_delta);int chsc_sstpi (void *page,void *result,size_t size);
drivers/s390/cio/css.c +127 -4
167 lines not on this page · raw message
Signature · 1 line
2.17.1
Halil Pasic
maintainerPATCH v4 3/8 s390/cio: add basic protected virtualization support
In reply to Halil PasicAs virtio-ccw devices are channel devices, we need to use the dma area within the common I/O layer for any communication with the hypervisor.
Note that we do not need to use that area for control blocks directly referenced by instructions, e.g. the orb.
It handles neither QDIO in the common code, nor any device type specific stuff (like channel programs constructed by the DASD driver).
An interesting side effect is that virtio structures are now going to get allocated in 31 bit addressable storage.
--- arch/s390/include/asm/ccwdev.h | 4 ++ drivers/s390/cio/ccwreq.c | 9 +++-- drivers/s390/cio/device.c | 68 ++++++++++++++++++++++++++------ drivers/s390/cio/device_fsm.c | 49 +++++++++++++---------- drivers/s390/cio/device_id.c | 20 +++++----- drivers/s390/cio/device_ops.c | 21 +++++++++- drivers/s390/cio/device_pgid.c | 22 ++++++----- drivers/s390/cio/device_status.c | 24 +++++------ drivers/s390/cio/io_sch.h | 20 +++++++--- drivers/s390/virtio/virtio_ccw.c | 10 ----- 10 files changed, 164 insertions(+), 83 deletions(-)
arch/s390/include/asm/ccwdev.h +4 -0
@@ -226,6 +226,10 @@ extern int ccw_device_enable_console(struct ccw_device *);extern void ccw_device_wait_idle (struct ccw_device *);extern int ccw_device_force_console (struct ccw_device *); +extern void *ccw_device_dma_zalloc (struct ccw_device *cdev,size_t size); +extern void ccw_device_dma_free (struct ccw_device *cdev, +void *cpu_addr,size_t size); +int ccw_device_siosl (struct ccw_device *);extern void ccw_device_get_schid (struct ccw_device *,struct subchannel_id *);
drivers/s390/cio/ccwreq.c +5 -4
@@ -63,7 +63,7 @@ static void ccwreq_stop(struct ccw_device *cdev, int rc)return ; req->done =1 ;ccw_device_set_timeout (cdev,0 ); -memset (&cdev->private->irb,0 ,sizeof (struct irb )); +memset (&cdev->private->dma_area->irb,0 ,sizeof (struct irb ));if (rc && rc != -ENODEV && req->drc) rc = req->drc; req->callback (cdev, req->data, rc);
@@ -86,7 +86,7 @@ static void ccwreq_do(struct ccw_device *cdev)continue ; }/* Perform start function. */ -memset (&cdev->private->irb,0 ,sizeof (struct irb )); +memset (&cdev->private->dma_area->irb,0 ,sizeof (struct irb )); rc =cio_start (sch, cp, (u8 ) req->mask);if (rc ==0 ) {/* I/O started successfully. */
@@ -169,7 +169,7 @@ int ccw_request_cancel(struct ccw_device *cdev)*/ static enum io_status ccwreq_status (struct ccw_device *cdev,struct irb *lcirb) { -struct irb *irb = &cdev->private->irb; +struct irb *irb = &cdev->private->dma_area->irb;struct cmd_scsw *scsw = &irb->scsw.cmd;enum uc_todo todo;
@@ -187,7 +187,8 @@ static enum io_status ccwreq_status(struct ccw_device *cdev, struct irb *lcirb)CIO_TRACE_EVENT (2 ,"sensedata" );CIO_HEX_EVENT (2 , &cdev->private->dev_id,sizeof (struct ccw_dev_id )); -CIO_HEX_EVENT (2 , &cdev->private->irb.ecw,SENSE_MAX_COUNT ); +CIO_HEX_EVENT (2 , &cdev->private->dma_area->irb.ecw, +SENSE_MAX_COUNT );/* Check for command reject. */ if (irb->ecw[0 ] &SNS0_CMD_REJECT )return IO_REJECTED ;
drivers/s390/cio/device.c +57 -11
129 lines not on this page · raw message
drivers/s390/cio/device_fsm.c +29 -20
112 lines not on this page · raw message
drivers/s390/cio/device_id.c +11 -9
@@ -99,7 +99,7 @@ static int diag210_to_senseid(struct senseid *senseid, struct diag210 *diag)static int diag210_get_dev_info (struct ccw_device *cdev) {struct ccw_dev_id *dev_id = &cdev->private->dev_id; -struct senseid *senseid = &cdev->private->senseid; +struct senseid *senseid = &cdev->private->dma_area->senseid;struct diag210 diag_data;int rc;
@@ -134,8 +134,10 @@ static int diag210_get_dev_info(struct ccw_device *cdev)static void snsid_init (struct ccw_device *cdev) { cdev->private->flags.esid =0 ; -memset (&cdev->private->senseid,0 ,sizeof (cdev->private->senseid)); - cdev->private->senseid.cu_type =0xffff ; + +memset (&cdev->private->dma_area->senseid,0 , +sizeof (cdev->private->dma_area->senseid)); + cdev->private->dma_area->senseid.cu_type =0xffff ; }/*
@@ -143,16 +145,16 @@ static void snsid_init(struct ccw_device *cdev)*/ static int snsid_check (struct ccw_device *cdev,void *data) { -struct cmd_scsw *scsw = &cdev->private->irb.scsw.cmd; +struct cmd_scsw *scsw = &cdev->private->dma_area->irb.scsw.cmd;int len =sizeof (struct senseid ) - scsw->count;/* Check for incomplete SENSE ID data. */ if (len <SENSE_ID_MIN_LEN ) goto out_restart; -if (cdev->private->senseid.cu_type ==0xffff ) +if (cdev->private->dma_area->senseid.cu_type ==0xffff ) goto out_restart;/* Check for incompatible SENSE ID data. */ -if (cdev->private->senseid.reserved !=0xff ) +if (cdev->private->dma_area->senseid.reserved !=0xff )return -EOPNOTSUPP ;/* Check for extended-identification information. */ if (len >SENSE_ID_BASIC_LEN )
@@ -170,7 +172,7 @@ static int snsid_check(struct ccw_device *cdev, void *data)static void snsid_callback (struct ccw_device *cdev,void *data,int rc) {struct ccw_dev_id *id = &cdev->private->dev_id; -struct senseid *senseid = &cdev->private->senseid; +struct senseid *senseid = &cdev->private->dma_area->senseid;int vm =0 ;if (rc &&MACHINE_IS_VM ) {
@@ -200,7 +202,7 @@ void ccw_device_sense_id_start(struct ccw_device *cdev) {struct subchannel *sch =to_subchannel (cdev->dev.parent);struct ccw_request *req = &cdev->private->req; -struct ccw1 *cp = cdev->private->iccws; +struct ccw1 *cp = cdev->private->dma_area->iccws;CIO_TRACE_EVENT (4 ,"snsid" );CIO_HEX_EVENT (4 , &cdev->private->dev_id,sizeof (cdev->private->dev_id));
@@ -208,7 +210,7 @@ void ccw_device_sense_id_start(struct ccw_device *cdev)snsid_init (cdev);/* Channel program setup. */ cp->cmd_code =CCW_CMD_SENSE_ID ; - cp->cda = (u32) (addr_t) &cdev->private->senseid; + cp->cda = (u32) (addr_t) &cdev->private->dma_area->senseid; cp->count =sizeof (struct senseid ); cp->flags =CCW_FLAG_SLI ;/* Request setup. */
drivers/s390/cio/device_ops.c +19 -2
@@ -429,8 +429,8 @@ struct ciw *ccw_device_get_ciw(struct ccw_device *cdev, __u32 ct)if (cdev->private->flags.esid ==0 )return NULL ;for (ciw_cnt =0 ; ciw_cnt <MAX_CIWS ; ciw_cnt++) -if (cdev->private->senseid.ciw[ciw_cnt].ct == ct) -return cdev->private->senseid.ciw + ciw_cnt; +if (cdev->private->dma_area->senseid.ciw[ciw_cnt].ct == ct) +return cdev->private->dma_area->senseid.ciw + ciw_cnt;return NULL ; }
@@ -699,6 +699,23 @@ void ccw_device_get_schid(struct ccw_device *cdev, struct subchannel_id *schid) }EXPORT_SYMBOL_GPL (ccw_device_get_schid); +/* +* Allocate zeroed dma coherent 31 bit addressable memory using +* the subchannels dma pool. Maximal size of allocation supported +* is PAGE_SIZE. +*/ +void *ccw_device_dma_zalloc (struct ccw_device *cdev,size_t size) +{ +return cio_gp_dma_zalloc (cdev->private->dma_pool, &cdev->dev, size); +} +EXPORT_SYMBOL (ccw_device_dma_zalloc); + +void ccw_device_dma_free (struct ccw_device *cdev,void *cpu_addr,size_t size) +{ +cio_gp_dma_free (cdev->private->dma_pool, cpu_addr, size); +} +EXPORT_SYMBOL (ccw_device_dma_free); +EXPORT_SYMBOL (ccw_device_set_options_mask);EXPORT_SYMBOL (ccw_device_set_options);EXPORT_SYMBOL (ccw_device_clear_options);
drivers/s390/cio/device_pgid.c +12 -10
@@ -57,7 +57,7 @@ static void verify_done(struct ccw_device *cdev, int rc)static void nop_build_cp (struct ccw_device *cdev) {struct ccw_request *req = &cdev->private->req; -struct ccw1 *cp = cdev->private->iccws; +struct ccw1 *cp = cdev->private->dma_area->iccws; cp->cmd_code =CCW_CMD_NOOP ; cp->cda =0 ;
@@ -134,9 +134,9 @@ static void nop_callback(struct ccw_device *cdev, void *data, int rc)static void spid_build_cp (struct ccw_device *cdev,u8 fn) {struct ccw_request *req = &cdev->private->req; -struct ccw1 *cp = cdev->private->iccws; +struct ccw1 *cp = cdev->private->dma_area->iccws;int i =pathmask_to_pos (req->lpm); -struct pgid *pgid = &cdev->private->pgid[i]; +struct pgid *pgid = &cdev->private->dma_area->pgid[i]; pgid->inf.fc = fn; cp->cmd_code =CCW_CMD_SET_PGID ;
@@ -300,7 +300,7 @@ static int pgid_cmp(struct pgid *p1, struct pgid *p2)static void pgid_analyze (struct ccw_device *cdev,struct pgid **p,int *mismatch,u8 *reserved,u8 *reset) { -struct pgid *pgid = &cdev->private->pgid[0 ]; +struct pgid *pgid = &cdev->private->dma_area->pgid[0 ];struct pgid *first =NULL ;int lpm;int i;
@@ -342,7 +342,7 @@ static u8 pgid_to_donepm(struct ccw_device *cdev) lpm =0x80 >> i;if ((cdev->private->pgid_valid_mask & lpm) ==0 )continue ; - pgid = &cdev->private->pgid[i]; + pgid = &cdev->private->dma_area->pgid[i];if (sch->opm & lpm) {if (pgid->inf.ps.state1 !=SNID_STATE1_GROUPED )continue ;
@@ -368,7 +368,8 @@ static void pgid_fill(struct ccw_device *cdev, struct pgid *pgid)int i;for (i =0 ; i <8 ; i++) -memcpy (&cdev->private->pgid[i], pgid,sizeof (struct pgid )); +memcpy (&cdev->private->dma_area->pgid[i], pgid, +sizeof (struct pgid )); }/*
@@ -435,12 +436,12 @@ static void snid_done(struct ccw_device *cdev, int rc)static void snid_build_cp (struct ccw_device *cdev) {struct ccw_request *req = &cdev->private->req; -struct ccw1 *cp = cdev->private->iccws; +struct ccw1 *cp = cdev->private->dma_area->iccws;int i =pathmask_to_pos (req->lpm);/* Channel program setup. */ cp->cmd_code =CCW_CMD_SENSE_PGID ; - cp->cda = (u32) (addr_t) &cdev->private->pgid[i]; + cp->cda = (u32) (addr_t) &cdev->private->dma_area->pgid[i]; cp->count =sizeof (struct pgid ); cp->flags =CCW_FLAG_SLI ; req->cp = cp;
@@ -516,7 +517,8 @@ static void verify_start(struct ccw_device *cdev) sch->lpm = sch->schib.pmcw.pam;/* Initialize PGID data. */ -memset (cdev->private->pgid,0 ,sizeof (cdev->private->pgid)); +memset (cdev->private->dma_area->pgid,0 , +sizeof (cdev->private->dma_area->pgid)); cdev->private->pgid_valid_mask =0 ; cdev->private->pgid_todo_mask = sch->schib.pmcw.pam; cdev->private->path_notoper_mask =0 ;
@@ -626,7 +628,7 @@ struct stlck_data {static void stlck_build_cp (struct ccw_device *cdev,void *buf1,void *buf2) {struct ccw_request *req = &cdev->private->req; -struct ccw1 *cp = cdev->private->iccws; +struct ccw1 *cp = cdev->private->dma_area->iccws; cp[0 ].cmd_code =CCW_CMD_STLCK ; cp[0 ].cda = (u32 ) (addr_t ) buf1;
drivers/s390/cio/device_status.c +12 -12
@@ -79,15 +79,15 @@ ccw_device_accumulate_ecw(struct ccw_device *cdev, struct irb *irb)* are condition that have to be met for the extended control * bit to have meaning. Sick. */ - cdev->private->irb.scsw.cmd.ectl =0 ; + cdev->private->dma_area->irb.scsw.cmd.ectl =0 ;if ((irb->scsw.cmd.stctl &SCSW_STCTL_ALERT_STATUS ) && !(irb->scsw.cmd.stctl &SCSW_STCTL_INTER_STATUS )) - cdev->private->irb.scsw.cmd.ectl = irb->scsw.cmd.ectl; + cdev->private->dma_area->irb.scsw.cmd.ectl = irb->scsw.cmd.ectl;/* Check if extended control word is valid. */ -if (!cdev->private->irb.scsw.cmd.ectl) +if (!cdev->private->dma_area->irb.scsw.cmd.ectl)return ;/* Copy concurrent sense / model dependent information. */ -memcpy (&cdev->private->irb.ecw, irb->ecw,sizeof (irb->ecw)); +memcpy (&cdev->private->dma_area->irb.ecw, irb->ecw,sizeof (irb->ecw)); }/*
@@ -118,7 +118,7 @@ ccw_device_accumulate_esw(struct ccw_device *cdev, struct irb *irb)if (!ccw_device_accumulate_esw_valid (irb))return ; - cdev_irb = &cdev->private->irb; + cdev_irb = &cdev->private->dma_area->irb;/* Copy last path used mask. */ cdev_irb->esw.esw1.lpum = irb->esw.esw1.lpum;
@@ -210,7 +210,7 @@ ccw_device_accumulate_irb(struct ccw_device *cdev, struct irb *irb)ccw_device_path_notoper (cdev);/* No irb accumulation for transport mode irbs. */ if (scsw_is_tm (&irb->scsw)) { -memcpy (&cdev->private->irb, irb,sizeof (struct irb )); +memcpy (&cdev->private->dma_area->irb, irb,sizeof (struct irb ));return ; }/*
@@ -219,7 +219,7 @@ ccw_device_accumulate_irb(struct ccw_device *cdev, struct irb *irb)if (!scsw_is_solicited (&irb->scsw))return ; - cdev_irb = &cdev->private->irb; + cdev_irb = &cdev->private->dma_area->irb;/* * If the clear function had been performed, all formerly pending
@@ -227,7 +227,7 @@ ccw_device_accumulate_irb(struct ccw_device *cdev, struct irb *irb)* intermediate accumulated status to the device driver. */ if (irb->scsw.cmd.fctl &SCSW_FCTL_CLEAR_FUNC ) -memset (&cdev->private->irb,0 ,sizeof (struct irb )); +memset (&cdev->private->dma_area->irb,0 ,sizeof (struct irb ));/* Copy bits which are valid only for the start function. */ if (irb->scsw.cmd.fctl &SCSW_FCTL_START_FUNC ) {
@@ -329,9 +329,9 @@ ccw_device_do_sense(struct ccw_device *cdev, struct irb *irb)/* * We have ending status but no sense information. Do a basic sense. */ - sense_ccw = &to_io_private (sch)->sense_ccw; + sense_ccw = &to_io_private (sch)->dma_area->sense_ccw; sense_ccw->cmd_code =CCW_CMD_BASIC_SENSE ; - sense_ccw->cda = (__u32 )__pa (cdev->private->irb.ecw); + sense_ccw->cda = (__u32 )__pa (cdev->private->dma_area->irb.ecw); sense_ccw->count =SENSE_MAX_COUNT ; sense_ccw->flags =CCW_FLAG_SLI ;
@@ -364,7 +364,7 @@ ccw_device_accumulate_basic_sense(struct ccw_device *cdev, struct irb *irb)if (!(irb->scsw.cmd.dstat &DEV_STAT_UNIT_CHECK ) && (irb->scsw.cmd.dstat &DEV_STAT_CHN_END )) { - cdev->private->irb.esw.esw0.erw.cons =1 ; + cdev->private->dma_area->irb.esw.esw0.erw.cons =1 ; cdev->private->flags.dosense =0 ; }/* Check if path verification is required. */
@@ -386,7 +386,7 @@ ccw_device_accumulate_and_sense(struct ccw_device *cdev, struct irb *irb)/* Check for basic sense. */ if (cdev->private->flags.dosense && !(irb->scsw.cmd.dstat &DEV_STAT_UNIT_CHECK )) { - cdev->private->irb.esw.esw0.erw.cons =1 ; + cdev->private->dma_area->irb.esw.esw0.erw.cons =1 ; cdev->private->flags.dosense =0 ;return 0 ; }
drivers/s390/cio/io_sch.h +15 -5
@@ -9,15 +9,20 @@#include "css.h" #include "orb.h" +struct io_subchannel_dma_area { +struct ccw1 sense_ccw;/* static ccw for sense command */ +}; +struct io_subchannel_private {union orb orb;/* operation request block */ -struct ccw1 sense_ccw;/* static ccw for sense command */ struct ccw_device *cdev;/* pointer to the child ccw device */ struct {unsigned int suspend:1 ;/* allow suspend */ unsigned int prefetch:1 ;/* deny prefetch */ unsigned int inter:1 ;/* suppress intermediate interrupts */ } __packed options; +struct io_subchannel_dma_area *dma_area; +dma_addr_t dma_area_dma; }__aligned(8) ;#define to_io_private (n) ((struct io_subchannel_private *) \
@@ -115,6 +120,13 @@ enum cdev_todo {#define FAKE_CMD_IRB 1#define FAKE_TM_IRB 2 +struct ccw_device_dma_area { +struct senseid senseid;/* SenseID info */ +struct ccw1 iccws[2 ];/* ccws for SNID/SID/SPGID commands */ +struct irb irb;/* device status */ +struct pgid pgid[8 ];/* path group IDs per chpid*/ +}; +struct ccw_device_private {struct ccw_device *cdev;struct subchannel *sch;
@@ -156,11 +168,7 @@ struct ccw_device_private { } __attribute__((packed)) flags;unsigned long intparm;/* user interruption parameter */ struct qdio_irq *qdio_data; -struct irb irb;/* device status */ int async_kill_io_rc; -struct senseid senseid;/* SenseID info */ -struct pgid pgid[8 ];/* path group IDs per chpid*/ -struct ccw1 iccws[2 ];/* ccws for SNID/SID/SPGID commands */ struct work_struct todo_work;enum cdev_todo todo;wait_queue_head_t wait_q;
@@ -169,6 +177,8 @@ struct ccw_device_private {struct list_head cmb_list;/* list of measured devices */ u64 cmb_start_time;/* clock value of cmb reset */ void *cmb_wait;/* deferred cmb enable/disable */ +struct gen_pool *dma_pool; +struct ccw_device_dma_area *dma_area;enum interruption_class int_class; };
drivers/s390/virtio/virtio_ccw.c +0 -10
@@ -66,7 +66,6 @@ struct virtio_ccw_device {bool device_lost;unsigned int config_ready;void *airq_info; -u64 dma_mask; };struct vq_info_block_legacy {
@@ -1255,16 +1254,7 @@ static int virtio_ccw_online(struct ccw_device *cdev) ret = -ENOMEM ; goto out_free; } - vcdev->vdev.dev.parent = &cdev->dev; - cdev->dev.dma_mask = &vcdev->dma_mask; -/* we are fine with common virtio infrastructure using 64 bit DMA */ - ret =dma_set_mask_and_coherent (&cdev->dev,DMA_BIT_MASK (64 )); -if (ret) { -dev_warn (&cdev->dev,"Failed to enable 64-bit DMA.\n" ); - goto out_free; - } - vcdev->config_block =kzalloc (sizeof (*vcdev->config_block),GFP_DMA |GFP_KERNEL );if (!vcdev->config_block) {
Signature · 1 line
2.17.1
Halil Pasic
maintainerPATCH v4 4/8 s390/airq: use DMA memory for adapter interrupts
In reply to Halil PasicProtected virtualization guests have to use shared pages for airq notifier bit vectors, because hypervisor needs to write these bits.
Let us make sure we allocate DMA memory for the notifier bit vectors by replacing the kmem_cache with a dma_cache and kalloc() with cio_dma_zalloc().
--- arch/s390/include/asm/airq.h | 2 ++ drivers/s390/cio/airq.c | 32 ++++++++++++++++++++------------ drivers/s390/cio/cio.h | 2 ++ drivers/s390/cio/css.c | 1 + 4 files changed, 25 insertions(+), 12 deletions(-)
arch/s390/include/asm/airq.h +2 -0
@@ -11,6 +11,7 @@#define _ASM_S390_AIRQ_H#include <linux/bit_spinlock.h> +#include <linux/dma-mapping.h> struct airq_struct {struct hlist_node list;/* Handler queueing. */
@@ -29,6 +30,7 @@ void unregister_adapter_interrupt(struct airq_struct *airq);/* Adapter interrupt bit vector */ struct airq_iv {unsigned long *vector;/* Adapter interrupt bit vector */ +dma_addr_t vector_dma;/* Adapter interrupt bit vector dma */ unsigned long *avail;/* Allocation bit mask for the bit vector */ unsigned long *bitlock;/* Lock bit mask for the bit vector */ unsigned long *ptr;/* Pointer associated with each bit */
drivers/s390/cio/airq.c +20 -12
@@ -16,9 +16,11 @@#include <linux/mutex.h> #include <linux/rculist.h> #include <linux/slab.h> +#include <linux/dmapool.h> #include <asm/airq.h> #include <asm/isc.h> +#include <asm/cio.h> #include "cio.h" #include "cio_debug.h"
@@ -27,7 +29,7 @@static DEFINE_SPINLOCK (airq_lists_lock);static struct hlist_head airq_lists[MAX_ISC +1 ]; -static struct kmem_cache *airq_iv_cache; +static struct dma_pool *airq_iv_cache;/** * register_adapter_interrupt() - register adapter interrupt handler
@@ -115,6 +117,11 @@ void __init init_airq_interrupts(void)setup_irq (THIN_INTERRUPT , &airq_interrupt); } +static inline unsigned long iv_size (unsigned long bits) +{ +return BITS_TO_LONGS (bits) *sizeof (unsigned long ); +} +/** * airq_iv_create - create an interrupt vector * @bits: number of bits in the interrupt vector
@@ -132,17 +139,18 @@ struct airq_iv *airq_iv_create(unsigned long bits, unsigned long flags) goto out; iv->bits = bits; iv->flags = flags; - size =BITS_TO_LONGS (bits) *sizeof (unsigned long ); + size =iv_size (bits);if (flags &AIRQ_IV_CACHELINE ) {if ((cache_line_size () *BITS_PER_BYTE ) < bits) goto out_free; - iv->vector =kmem_cache_zalloc (airq_iv_cache,GFP_KERNEL ); + iv->vector =dma_pool_zalloc (airq_iv_cache,GFP_KERNEL , + &iv->vector_dma);if (!iv->vector) goto out_free; }else { - iv->vector =kzalloc (size,GFP_KERNEL ); + iv->vector =cio_dma_zalloc (size);if (!iv->vector) goto out_free; }
@@ -179,9 +187,9 @@ struct airq_iv *airq_iv_create(unsigned long bits, unsigned long flags)kfree (iv->bitlock);kfree (iv->avail);if (iv->flags &AIRQ_IV_CACHELINE ) -kmem_cache_free (airq_iv_cache, iv->vector); +dma_pool_free (airq_iv_cache, iv->vector, iv->vector_dma);else -kfree (iv->vector); +cio_dma_free (iv->vector, size);kfree (iv); out:return NULL ;
@@ -198,9 +206,9 @@ void airq_iv_release(struct airq_iv *iv)kfree (iv->ptr);kfree (iv->bitlock);if (iv->flags &AIRQ_IV_CACHELINE ) -kmem_cache_free (airq_iv_cache, iv->vector); +dma_pool_free (airq_iv_cache, iv->vector, iv->vector_dma);else -kfree (iv->vector); +cio_dma_free (iv->vector,iv_size (iv->bits));kfree (iv->avail);kfree (iv); }
@@ -295,12 +303,12 @@ unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start, }EXPORT_SYMBOL (airq_iv_scan); -static int __init airq_init (void ) +int __init airq_init (void ) { - airq_iv_cache =kmem_cache_create ("airq_iv_cache" ,cache_line_size (), -cache_line_size (),0 ,NULL ); + airq_iv_cache =dma_pool_create ("airq_iv_cache" ,cio_get_dma_css_dev (), +cache_line_size (), +cache_line_size (),PAGE_SIZE );if (!airq_iv_cache)return -ENOMEM ;return 0 ; } -subsys_initcall (airq_init);
drivers/s390/cio/cio.h +2 -0
@@ -135,6 +135,8 @@ extern int cio_commit_config(struct subchannel *sch);int cio_tm_start_key (struct subchannel *sch,struct tcw *tcw,u8 lpm,u8 key);int cio_tm_intrg (struct subchannel *sch); +extern int __init airq_init (void ); +/* Use with care. */ #ifdef CONFIG_CCW_CONSOLE extern struct subchannel *cio_probe_console (void );
drivers/s390/cio/css.c +1 -0
@@ -1182,6 +1182,7 @@ static int __init css_bus_init(void) ret =cio_dma_pool_init ();if (ret) goto out_unregister_pmn; +airq_init (); css_init_done =1 ;/* Enable default isc for I/O subchannels. */
Signature · 1 line
2.17.1
Halil Pasic
maintainerPATCH v4 5/8 virtio/s390: use cacheline aligned airq bit vectors
In reply to Halil PasicThe flag AIRQ_IV_CACHELINE was recently added to airq_iv_create(). Let us use it! We actually wanted the vector to span a cacheline all along.
--- drivers/s390/virtio/virtio_ccw.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/s390/virtio/virtio_ccw.c +2 -1
@@ -216,7 +216,8 @@ static struct airq_info *new_airq_info(void)if (!info)return NULL ;rwlock_init (&info->lock); - info->aiv =airq_iv_create (VIRTIO_IV_BITS ,AIRQ_IV_ALLOC |AIRQ_IV_PTR ); + info->aiv =airq_iv_create (VIRTIO_IV_BITS ,AIRQ_IV_ALLOC |AIRQ_IV_PTR + |AIRQ_IV_CACHELINE );if (!info->aiv) {kfree (info);return NULL ;
Signature · 1 line
2.17.1
Halil Pasic
maintainerPATCH v4 6/8 virtio/s390: add indirection to indicators access
In reply to Halil PasicThis will come in handy soon when we pull out the indicators from virtio_ccw_device to a memory area that is shared with the hypervisor (in particular for protected virtualization guests).
--- drivers/s390/virtio/virtio_ccw.c | 40 ++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 15 deletions(-)
drivers/s390/virtio/virtio_ccw.c +25 -15
@@ -68,6 +68,16 @@ struct virtio_ccw_device {void *airq_info; }; +static inline unsigned long *indicators (struct virtio_ccw_device *vcdev) +{ +return &vcdev->indicators; +} + +static inline unsigned long *indicators2 (struct virtio_ccw_device *vcdev) +{ +return &vcdev->indicators2; +} +struct vq_info_block_legacy {__u64 queue;__u32 align;
@@ -338,17 +348,17 @@ static void virtio_ccw_drop_indicator(struct virtio_ccw_device *vcdev, ccw->cda = (__u32 )(unsigned long ) thinint_area; }else {/* payload is the address of the indicators */ - indicatorp =kmalloc (sizeof (&vcdev->indicators), + indicatorp =kmalloc (sizeof (indicators (vcdev)),GFP_DMA |GFP_KERNEL );if (!indicatorp)return ; *indicatorp =0 ; ccw->cmd_code =CCW_CMD_SET_IND ; - ccw->count =sizeof (&vcdev->indicators); + ccw->count =sizeof (indicators (vcdev)); ccw->cda = (__u32 )(unsigned long ) indicatorp; }/* Deregister indicators from host. */ - vcdev->indicators =0 ; + *indicators (vcdev) =0 ; ccw->flags =0 ; ret = ccw_io_helper(vcdev, ccw, vcdev->is_thinint ?
@@ -657,10 +667,10 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,* We need a data area under 2G to communicate. Our payload is * the address of the indicators. */ - indicatorp =kmalloc (sizeof (&vcdev->indicators),GFP_DMA |GFP_KERNEL ); + indicatorp =kmalloc (sizeof (indicators (vcdev)),GFP_DMA |GFP_KERNEL );if (!indicatorp) goto out; - *indicatorp = (unsigned long ) &vcdev->indicators; + *indicatorp = (unsigned long )indicators (vcdev);if (vcdev->is_thinint) { ret =virtio_ccw_register_adapter_ind (vcdev, vqs, nvqs, ccw);if (ret)
@@ -669,21 +679,21 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs, }if (!vcdev->is_thinint) {/* Register queue indicators with host. */ - vcdev->indicators =0 ; + *indicators (vcdev) =0 ; ccw->cmd_code =CCW_CMD_SET_IND ; ccw->flags =0 ; - ccw->count =sizeof (&vcdev->indicators); + ccw->count =sizeof (indicators (vcdev)); ccw->cda = (__u32 )(unsigned long ) indicatorp; ret =ccw_io_helper (vcdev, ccw,VIRTIO_CCW_DOING_SET_IND );if (ret) goto out; }/* Register indicators2 with host for config changes */ - *indicatorp = (unsigned long ) &vcdev->indicators2; - vcdev->indicators2 =0 ; + *indicatorp = (unsigned long )indicators2 (vcdev); + *indicators2 (vcdev) =0 ; ccw->cmd_code =CCW_CMD_SET_CONF_IND ; ccw->flags =0 ; - ccw->count =sizeof (&vcdev->indicators2); + ccw->count =sizeof (indicators2 (vcdev)); ccw->cda = (__u32 )(unsigned long ) indicatorp; ret =ccw_io_helper (vcdev, ccw,VIRTIO_CCW_DOING_SET_CONF_IND );if (ret)
@@ -1093,17 +1103,17 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev, vcdev->err = -EIO ; }virtio_ccw_check_activity (vcdev, activity); -for_each_set_bit (i, &vcdev->indicators, -sizeof (vcdev->indicators) *BITS_PER_BYTE ) { +for_each_set_bit (i,indicators (vcdev), +sizeof (*indicators (vcdev)) *BITS_PER_BYTE ) {/* The bit clear must happen before the vring kick. */ -clear_bit (i, &vcdev->indicators); +clear_bit (i,indicators (vcdev));barrier (); vq =virtio_ccw_vq_by_ind (vcdev, i);vring_interrupt (0 , vq); } -if (test_bit (0 , &vcdev->indicators2)) { +if (test_bit (0 ,indicators2 (vcdev))) {virtio_config_changed (&vcdev->vdev); -clear_bit (0 , &vcdev->indicators2); +clear_bit (0 ,indicators2 (vcdev)); } }
Signature · 1 line
2.17.1
Halil Pasic
maintainerPATCH v4 7/8 virtio/s390: use DMA memory for ccw I/O and classic notifiers
In reply to Halil PasicBefore virtio-ccw could get away with not using DMA API for the pieces of memory it does ccw I/O with. With protected virtualization this has to change, since the hypervisor needs to read and sometimes also write these pieces of memory.
The hypervisor is supposed to poke the classic notifiers, if these are used, out of band with regards to ccw I/O. So these need to be allocated as DMA memory (which is shared memory for protected virtualization guests).
Let us factor out everything from struct virtio_ccw_device that needs to be DMA memory in a satellite that is allocated as such.
Note: The control blocks of I/O instructions do not need to be shared. These are marshalled by the ultravisor.
--- drivers/s390/virtio/virtio_ccw.c | 171 ++++++++++++++++--------------- 1 file changed, 90 insertions(+), 81 deletions(-)
drivers/s390/virtio/virtio_ccw.c +90 -81
@@ -46,9 +46,15 @@ struct vq_config_block {#define VIRTIO_CCW_CONFIG_SIZE 0x100/* same as PCI config space size, should be enough for all drivers */ +struct vcdev_dma_area { +unsigned long indicators; +unsigned long indicators2; +struct vq_config_block config_block; +__u8 status; +}; +struct virtio_ccw_device {struct virtio_device vdev; -__u8 *status;__u8 config[VIRTIO_CCW_CONFIG_SIZE ];struct ccw_device *cdev;__u32 curr_io;
@@ -58,24 +64,22 @@ struct virtio_ccw_device {spinlock_t lock;struct mutex io_lock;/* Serializes I/O requests */ struct list_head virtqueues; -unsigned long indicators; -unsigned long indicators2; -struct vq_config_block *config_block;bool is_thinint;bool going_away;bool device_lost;unsigned int config_ready;void *airq_info; +struct vcdev_dma_area *dma_area; };static inline unsigned long *indicators (struct virtio_ccw_device *vcdev) { -return &vcdev->indicators; +return &vcdev->dma_area->indicators; }static inline unsigned long *indicators2 (struct virtio_ccw_device *vcdev) { -return &vcdev->indicators2; +return &vcdev->dma_area->indicators2; }struct vq_info_block_legacy {
@@ -336,8 +340,8 @@ static void virtio_ccw_drop_indicator(struct virtio_ccw_device *vcdev,struct airq_info *airq_info = vcdev->airq_info;if (vcdev->is_thinint) { - thinint_area =kzalloc (sizeof (*thinint_area), -GFP_DMA |GFP_KERNEL ); + thinint_area =ccw_device_dma_zalloc (vcdev->cdev, +sizeof (*thinint_area));if (!thinint_area)return ; thinint_area->summary_indicator =
@@ -348,8 +352,8 @@ static void virtio_ccw_drop_indicator(struct virtio_ccw_device *vcdev, ccw->cda = (__u32 )(unsigned long ) thinint_area; }else {/* payload is the address of the indicators */ - indicatorp =kmalloc (sizeof (indicators (vcdev)), -GFP_DMA |GFP_KERNEL ); + indicatorp =ccw_device_dma_zalloc (vcdev->cdev, +sizeof (indicators (vcdev)));if (!indicatorp)return ; *indicatorp =0 ;
@@ -369,8 +373,8 @@ static void virtio_ccw_drop_indicator(struct virtio_ccw_device *vcdev, "Failed to deregister indicators (%d)\n", ret); else if (vcdev->is_thinint) virtio_ccw_drop_indicators(vcdev); -kfree (indicatorp ); -kfree (thinint_area); +ccw_device_dma_free (vcdev->cdev, indicatorp,sizeof (indicators (vcdev))); +ccw_device_dma_free (vcdev->cdev, thinint_area,sizeof (*thinint_area)); } static inline long __do_kvm_notify(struct subchannel_id schid,
@@ -417,15 +421,15 @@ static int virtio_ccw_read_vq_conf(struct virtio_ccw_device *vcdev, {int ret; - vcdev->config_block->index = index; + vcdev->dma_area->config_block.index = index; ccw->cmd_code =CCW_CMD_READ_VQ_CONF ; ccw->flags =0 ; ccw->count =sizeof (struct vq_config_block ); - ccw->cda = (__u32 )(unsigned long )(vcdev->config_block); + ccw->cda = (__u32 )(unsigned long )(&vcdev->dma_area->config_block); ret =ccw_io_helper (vcdev, ccw,VIRTIO_CCW_DOING_READ_VQ_CONF );if (ret)return ret; -return vcdev->config_block->num ?: -ENOENT ; +return vcdev->dma_area->config_block.num ?: -ENOENT ; }static void virtio_ccw_del_vq (struct virtqueue *vq,struct ccw1 *ccw)
@@ -470,7 +474,8 @@ static void virtio_ccw_del_vq(struct virtqueue *vq, struct ccw1 *ccw) ret, index);vring_del_virtqueue (vq); -kfree (info->info_block); +ccw_device_dma_free (vcdev->cdev, info->info_block, +sizeof (*info->info_block));kfree (info); }
@@ -480,7 +485,7 @@ static void virtio_ccw_del_vqs(struct virtio_device *vdev)struct ccw1 *ccw;struct virtio_ccw_device *vcdev =to_vc_device (vdev); - ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return ;
@@ -489,7 +494,7 @@ static void virtio_ccw_del_vqs(struct virtio_device *vdev)list_for_each_entry_safe (vq, n, &vdev->vqs, list)virtio_ccw_del_vq (vq, ccw); -kfree (ccw); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw)); } static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,
@@ -512,8 +517,8 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev, err = -ENOMEM ; goto out_err; } - info->info_block =kzalloc (sizeof (*info->info_block), -GFP_DMA |GFP_KERNEL ); + info->info_block =ccw_device_dma_zalloc (vcdev->cdev, +sizeof (*info->info_block));if (!info->info_block) {dev_warn (&vcdev->cdev->dev,"no info block\n" ); err = -ENOMEM ;
@@ -577,7 +582,8 @@ static struct virtqueue *virtio_ccw_setup_vq(struct virtio_device *vdev,if (vq)vring_del_virtqueue (vq);if (info) { -kfree (info->info_block); +ccw_device_dma_free (vcdev->cdev, info->info_block, +sizeof (*info->info_block)); }kfree (info);return ERR_PTR (err);
@@ -591,7 +597,8 @@ static int virtio_ccw_register_adapter_ind(struct virtio_ccw_device *vcdev,struct virtio_thinint_area *thinint_area =NULL ;struct airq_info *info; - thinint_area =kzalloc (sizeof (*thinint_area),GFP_DMA |GFP_KERNEL ); + thinint_area =ccw_device_dma_zalloc (vcdev->cdev, +sizeof (*thinint_area));if (!thinint_area) { ret = -ENOMEM ; goto out;
@@ -627,7 +634,7 @@ static int virtio_ccw_register_adapter_ind(struct virtio_ccw_device *vcdev,virtio_ccw_drop_indicators (vcdev); } out: -kfree (thinint_area); +ccw_device_dma_free (vcdev->cdev, thinint_area,sizeof (*thinint_area));return ret; }
@@ -643,7 +650,7 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,int ret, i, queue_idx =0 ;struct ccw1 *ccw; - ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return -ENOMEM ;
@@ -667,7 +674,8 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,* We need a data area under 2G to communicate. Our payload is * the address of the indicators. */ - indicatorp =kmalloc (sizeof (indicators (vcdev)),GFP_DMA |GFP_KERNEL ); + indicatorp =ccw_device_dma_zalloc (vcdev->cdev, +sizeof (indicators (vcdev)));if (!indicatorp) goto out; *indicatorp = (unsigned long )indicators (vcdev);
@@ -699,12 +707,16 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,if (ret) goto out; -kfree (indicatorp); -kfree (ccw); +if (indicatorp) +ccw_device_dma_free (vcdev->cdev, indicatorp, +sizeof (indicators (vcdev))); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw));return 0 ; out: -kfree (indicatorp); -kfree (ccw); +if (indicatorp) +ccw_device_dma_free (vcdev->cdev, indicatorp, +sizeof (indicators (vcdev))); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw));virtio_ccw_del_vqs (vdev);return ret; }
@@ -714,12 +726,12 @@ static void virtio_ccw_reset(struct virtio_device *vdev)struct virtio_ccw_device *vcdev =to_vc_device (vdev);struct ccw1 *ccw; - ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return ;/* Zero status bits. */ - *vcdev->status =0 ; + vcdev->dma_area->status =0 ;/* Send a reset ccw on device. */ ccw->cmd_code =CCW_CMD_VDEV_RESET ;
@@ -727,22 +739,22 @@ static void virtio_ccw_reset(struct virtio_device *vdev) ccw->count =0 ; ccw->cda =0 ;ccw_io_helper (vcdev, ccw,VIRTIO_CCW_DOING_RESET ); -kfree (ccw); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw)); }static u64 virtio_ccw_get_features (struct virtio_device *vdev) {struct virtio_ccw_device *vcdev =to_vc_device (vdev);struct virtio_feature_desc *features; +struct ccw1 *ccw;int ret;u64 rc; -struct ccw1 *ccw; - ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return 0 ; - features =kzalloc (sizeof (*features),GFP_DMA |GFP_KERNEL ); + features =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*features));if (!features) { rc =0 ; goto out_free;
@@ -775,8 +787,8 @@ static u64 virtio_ccw_get_features(struct virtio_device *vdev) rc |= (u64 )le32_to_cpu (features->features) <<32 ; out_free: -kfree (features); -kfree (ccw); +ccw_device_dma_free (vcdev->cdev, features,sizeof (*features)); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw));return rc; }
@@ -801,11 +813,11 @@ static int virtio_ccw_finalize_features(struct virtio_device *vdev)return -EINVAL ; } - ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return -ENOMEM ; - features =kzalloc (sizeof (*features),GFP_DMA |GFP_KERNEL ); + features =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*features));if (!features) { ret = -ENOMEM ; goto out_free;
@@ -840,8 +852,8 @@ static int virtio_ccw_finalize_features(struct virtio_device *vdev) ret =ccw_io_helper (vcdev, ccw,VIRTIO_CCW_DOING_WRITE_FEAT ); out_free: -kfree (features); -kfree (ccw); +ccw_device_dma_free (vcdev->cdev, features,sizeof (*features)); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw));return ret; }
@@ -855,11 +867,12 @@ static void virtio_ccw_get_config(struct virtio_device *vdev,void *config_area;unsigned long flags; - ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return ; - config_area =kzalloc (VIRTIO_CCW_CONFIG_SIZE ,GFP_DMA |GFP_KERNEL ); + config_area =ccw_device_dma_zalloc (vcdev->cdev, +VIRTIO_CCW_CONFIG_SIZE );if (!config_area) goto out_free;
@@ -881,8 +894,8 @@ static void virtio_ccw_get_config(struct virtio_device *vdev,memcpy (buf, config_area + offset, len); out_free: -kfree (config_area); -kfree (ccw); +ccw_device_dma_free (vcdev->cdev, config_area,VIRTIO_CCW_CONFIG_SIZE ); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw)); } static void virtio_ccw_set_config(struct virtio_device *vdev,
@@ -894,11 +907,12 @@ static void virtio_ccw_set_config(struct virtio_device *vdev,void *config_area;unsigned long flags; - ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return ; - config_area =kzalloc (VIRTIO_CCW_CONFIG_SIZE ,GFP_DMA |GFP_KERNEL ); + config_area =ccw_device_dma_zalloc (vcdev->cdev, +VIRTIO_CCW_CONFIG_SIZE );if (!config_area) goto out_free;
@@ -917,61 +931,61 @@ static void virtio_ccw_set_config(struct virtio_device *vdev,ccw_io_helper (vcdev, ccw,VIRTIO_CCW_DOING_WRITE_CONFIG ); out_free: -kfree (config_area); -kfree (ccw); +ccw_device_dma_free (vcdev->cdev, config_area,VIRTIO_CCW_CONFIG_SIZE ); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw)); }static u8 virtio_ccw_get_status (struct virtio_device *vdev) {struct virtio_ccw_device *vcdev =to_vc_device (vdev); -u8 old_status = *vcdev->status; +u8 old_status = vcdev->dma_area->status;struct ccw1 *ccw;if (vcdev->revision <1 ) -return *vcdev->status; +return vcdev->dma_area->status; - ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return old_status; ccw->cmd_code =CCW_CMD_READ_STATUS ; ccw->flags =0 ; - ccw->count =sizeof (*vcdev->status); - ccw->cda = (__u32 )(unsigned long )vcdev->status; + ccw->count =sizeof (vcdev->dma_area->status); + ccw->cda = (__u32 )(unsigned long )&vcdev->dma_area->status;ccw_io_helper (vcdev, ccw,VIRTIO_CCW_DOING_READ_STATUS );/* * If the channel program failed (should only happen if the device * was hotunplugged, and then we clean up via the machine check -* handler anyway), vcdev->status was not overwritten and we just +* handler anyway), vcdev->dma_area->status was not overwritten and we just * return the old status, which is fine. */ -kfree (ccw); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw)); -return *vcdev->status; +return vcdev->dma_area->status; }static void virtio_ccw_set_status (struct virtio_device *vdev,u8 status) {struct virtio_ccw_device *vcdev =to_vc_device (vdev); -u8 old_status = *vcdev->status; +u8 old_status = vcdev->dma_area->status;struct ccw1 *ccw;int ret; - ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return ;/* Write the status to the host. */ - *vcdev->status = status; + vcdev->dma_area->status = status; ccw->cmd_code =CCW_CMD_WRITE_STATUS ; ccw->flags =0 ; ccw->count =sizeof (status); - ccw->cda = (__u32 )(unsigned long )vcdev->status; + ccw->cda = (__u32 )(unsigned long )&vcdev->dma_area->status; ret =ccw_io_helper (vcdev, ccw,VIRTIO_CCW_DOING_WRITE_STATUS );/* Write failed? We assume status is unchanged. */ if (ret) - *vcdev->status = old_status; -kfree (ccw); + vcdev->dma_area->status = old_status; +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw)); }static const char *virtio_ccw_bus_name (struct virtio_device *vdev)
@@ -1004,8 +1018,8 @@ static void virtio_ccw_release_dev(struct device *_d)struct virtio_device *dev =dev_to_virtio (_d);struct virtio_ccw_device *vcdev =to_vc_device (dev); -kfree (vcdev->status); -kfree (vcdev->config_block); +ccw_device_dma_free (vcdev->cdev, vcdev->dma_area, +sizeof (*vcdev->dma_area));kfree (vcdev); }
@@ -1213,12 +1227,12 @@ static int virtio_ccw_set_transport_rev(struct virtio_ccw_device *vcdev)struct ccw1 *ccw;int ret; - ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return -ENOMEM ; - rev =kzalloc (sizeof (*rev),GFP_DMA |GFP_KERNEL ); + rev =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*rev));if (!rev) { -kfree (ccw); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw));return -ENOMEM ; }
@@ -1248,8 +1262,8 @@ static int virtio_ccw_set_transport_rev(struct virtio_ccw_device *vcdev) } }while (ret == -EOPNOTSUPP ); -kfree (ccw); -kfree (rev); +ccw_device_dma_free (vcdev->cdev, ccw,sizeof (*ccw)); +ccw_device_dma_free (vcdev->cdev, rev,sizeof (*rev));return ret; }
@@ -1266,14 +1280,10 @@ static int virtio_ccw_online(struct ccw_device *cdev) goto out_free; } vcdev->vdev.dev.parent = &cdev->dev; - vcdev->config_block =kzalloc (sizeof (*vcdev->config_block), -GFP_DMA |GFP_KERNEL ); -if (!vcdev->config_block) { - ret = -ENOMEM ; - goto out_free; - } - vcdev->status =kzalloc (sizeof (*vcdev->status),GFP_DMA |GFP_KERNEL ); -if (!vcdev->status) { + vcdev->cdev = cdev; + vcdev->dma_area =ccw_device_dma_zalloc (vcdev->cdev, +sizeof (*vcdev->dma_area)); +if (!vcdev->dma_area) { ret = -ENOMEM ; goto out_free; }
@@ -1282,7 +1292,6 @@ static int virtio_ccw_online(struct ccw_device *cdev) vcdev->vdev.dev.release = virtio_ccw_release_dev; vcdev->vdev.config = &virtio_ccw_config_ops; - vcdev->cdev = cdev;init_waitqueue_head (&vcdev->wait_q);INIT_LIST_HEAD (&vcdev->virtqueues);spin_lock_init (&vcdev->lock);
@@ -1313,8 +1322,8 @@ static int virtio_ccw_online(struct ccw_device *cdev)return ret; out_free:if (vcdev) { -kfree (vcdev->status); -kfree (vcdev->config_block); +ccw_device_dma_free (vcdev->cdev, vcdev->dma_area, +sizeof (*vcdev->dma_area)); }kfree (vcdev);return ret;
Signature · 1 line
2.17.1
Halil Pasic
maintainerPATCH v4 8/8 virtio/s390: make airq summary indicators DMA
In reply to Halil PasicHypervisor needs to interact with the summary indicators, so these need to be DMA memory as well (at least for protected virtualization guests).
--- drivers/s390/virtio/virtio_ccw.c | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-)
drivers/s390/virtio/virtio_ccw.c +24 -8
@@ -140,11 +140,17 @@ static int virtio_ccw_use_airq = 1;struct airq_info {rwlock_t lock; -u8 summary_indicator; +u8 summary_indicator_idx;struct airq_struct airq;struct airq_iv *aiv; };static struct airq_info *airq_areas[MAX_AIRQ_AREAS ]; +static u8 *summary_indicators; + +static inline u8 *get_summary_indicator (struct airq_info *info) +{ +return summary_indicators + info->summary_indicator_idx; +}#define CCW_CMD_SET_VQ 0x13#define CCW_CMD_VDEV_RESET 0x33
@@ -209,7 +215,7 @@ static void virtio_airq_handler(struct airq_struct *airq, bool floating)break ;vring_interrupt (0 , (void *)airq_iv_get_ptr (info->aiv, ai)); } - info->summary_indicator =0 ; + *(get_summary_indicator (info)) =0 ;smp_wmb ();/* Walk through indicators field, summary indicator not active. */ for (ai =0 ;;) {
@@ -221,7 +227,7 @@ static void virtio_airq_handler(struct airq_struct *airq, bool floating)read_unlock (&info->lock); } -static struct airq_info *new_airq_info (void ) +static struct airq_info *new_airq_info (int index) {struct airq_info *info;int rc;
@@ -237,7 +243,8 @@ static struct airq_info *new_airq_info(void)return NULL ; } info->airq.handler = virtio_airq_handler; - info->airq.lsi_ptr = &info->summary_indicator; + info->summary_indicator_idx = index; + info->airq.lsi_ptr =get_summary_indicator (info); info->airq.lsi_mask =0xff ; info->airq.isc =VIRTIO_AIRQ_ISC ; rc =register_adapter_interrupt (&info->airq);
@@ -259,7 +266,7 @@ static unsigned long get_airq_indicator(struct virtqueue *vqs[], int nvqs,for (i =0 ; i <MAX_AIRQ_AREAS && !indicator_addr; i++) {if (!airq_areas[i]) - airq_areas[i] =new_airq_info (); + airq_areas[i] =new_airq_info (i); info = airq_areas[i];if (!info)return 0 ;
@@ -345,7 +352,7 @@ static void virtio_ccw_drop_indicator(struct virtio_ccw_device *vcdev,if (!thinint_area)return ; thinint_area->summary_indicator = - (unsigned long ) &airq_info->summary_indicator; + (unsigned long )get_summary_indicator (airq_info); thinint_area->isc =VIRTIO_AIRQ_ISC ; ccw->cmd_code =CCW_CMD_SET_IND_ADAPTER ; ccw->count =sizeof (*thinint_area);
@@ -613,7 +620,7 @@ static int virtio_ccw_register_adapter_ind(struct virtio_ccw_device *vcdev, } info = vcdev->airq_info; thinint_area->summary_indicator = - (unsigned long ) &info->summary_indicator; + (unsigned long )get_summary_indicator (info); thinint_area->isc =VIRTIO_AIRQ_ISC ; ccw->cmd_code =CCW_CMD_SET_IND_ADAPTER ; ccw->flags =CCW_FLAG_SLI ;
@@ -1493,8 +1500,17 @@ static void __init no_auto_parse(void)static int __init virtio_ccw_init (void ) { +int rc; +/* parse no_auto string before we do anything further */ no_auto_parse (); -return ccw_driver_register (&virtio_ccw_driver); + + summary_indicators =cio_dma_zalloc (MAX_AIRQ_AREAS ); +if (!summary_indicators) +return -ENOMEM ; + rc =ccw_driver_register (&virtio_ccw_driver); +if (rc) +cio_dma_free (summary_indicators,MAX_AIRQ_AREAS ); +return rc; }device_initcall (virtio_ccw_init);
Signature · 1 line
2.17.1
Re: [PATCH v4 2/8] s390/cio: introduce DMA pools to cio
In reply to Halil PasicQuoted 16 lines · On Thu, 6 Jun 2019 13:51:21 +0200 Halil Pasic <[email protected]> wrote:
To support protected virtualization cio will need to make sure the memory used for communication with the hypervisor is DMA memory.
Let us introduce one global pool for cio.
Our DMA pools are implemented as a
gen_poolbacked with DMA pages. The idea is to avoid each allocation effectively wasting a page, as we typically allocate much less thanPAGE_SIZE.Signed-off-by Halil Pasic <[email protected]>Reviewed-by Sebastian Ott <[email protected]>--- arch/s390/Kconfig | 1 + arch/s390/include/asm/cio.h | 11 +++ drivers/s390/cio/css.c | 131 ++++++++++++++++++++++++++++++++++-- 3 files changed, 139 insertions(+), 4 deletions(-)
(...)
drivers/s390/cio/css.c static struct notifier_block css_power_notifier = { · line 1080
Quoted 5 lines
drivers/s390/cio/css.c +5
-0+void cio_gp_dma_destroy (struct gen_pool *gp_dma,struct device *dma_dev) +{ +if (!gp_dma) +return ; +/* this is qite ugly but no better idea */
typo: s/qite/quite/
Quoted 3 lines
drivers/s390/cio/css.c +3
-0+gen_pool_for_each_chunk (gp_dma, __gp_dma_free_dma, dma_dev); +gen_pool_destroy (gp_dma); +}
(...)
Cornelia Huck
maintainerRe: [PATCH v4 4/8] s390/airq: use DMA memory for adapter interrupts
In reply to Halil PasicQuoted 16 lines · On Thu, 6 Jun 2019 13:51:23 +0200 Halil Pasic <[email protected]> wrote:
Protected virtualization guests have to use shared pages for airq notifier bit vectors, because hypervisor needs to write these bits.
Let us make sure we allocate DMA memory for the notifier bit vectors by replacing the
kmem_cachewith adma_cacheandkalloc()withcio_dma_zalloc().Signed-off-by Halil Pasic <[email protected]>Reviewed-by Sebastian Ott <[email protected]>--- arch/s390/include/asm/airq.h | 2 ++ drivers/s390/cio/airq.c | 32 ++++++++++++++++++++------------ drivers/s390/cio/cio.h | 2 ++ drivers/s390/cio/css.c | 1 + 4 files changed, 25 insertions(+), 12 deletions(-)
(...)
Quoted 4 lines
drivers/s390/cio/airq.c +0
-0@@ -295,12 +303,12 @@ unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start, }EXPORT_SYMBOL (airq_iv_scan);
drivers/s390/cio/airq.c unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start, · line 312
Quoted 10 lines
drivers/s390/cio/airq.c +4
-3-static int __init airq_init (void ) +int __init airq_init (void ) { - airq_iv_cache =kmem_cache_create ("airq_iv_cache" ,cache_line_size (), -cache_line_size (),0 ,NULL ); + airq_iv_cache =dma_pool_create ("airq_iv_cache" ,cio_get_dma_css_dev (), +cache_line_size (), +cache_line_size (),PAGE_SIZE );if (!airq_iv_cache)return -ENOMEM ;
Sorry about not noticing that in the last iteration; but you may return an error here if airq_iv_cache could not be allocated...
Quoted 3 lines
drivers/s390/cio/airq.c +0
-1return 0 ; } -subsys_initcall (airq_init);
(...)
drivers/s390/cio/css.c static int __init css_bus_init(void) · line 1185
Quoted 9 lines
drivers/s390/cio/css.c +1
-0@@ -1182,6 +1182,7 @@ static int __init css_bus_init(void) ret =cio_dma_pool_init ();if (ret) goto out_unregister_pmn; +airq_init ();
...but don't check the return code here. Probably a pathological case, but shouldn't you handle that error as well?
Quoted 3 lines
drivers/s390/cio/css.c +0
-0css_init_done =1 ;/* Enable default isc for I/O subchannels. */
Re: [PATCH v4 8/8] virtio/s390: make airq summary indicators DMA
In reply to Halil PasicQuoted 8 lines · On Thu, 6 Jun 2019 13:51:27 +0200 Halil Pasic <[email protected]> wrote:
Hypervisor needs to interact with the summary indicators, so these need to be DMA memory as well (at least for protected virtualization guests).
Signed-off-by Halil Pasic <[email protected]>--- drivers/s390/virtio/virtio_ccw.c | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-)
Re: [PATCH v4 7/8] virtio/s390: use DMA memory for ccw I/O and classic notifiers
In reply to Halil PasicQuoted 21 lines · On Thu, 6 Jun 2019 13:51:26 +0200 Halil Pasic <[email protected]> wrote:
Before virtio-ccw could get away with not using DMA API for the pieces of memory it does ccw I/O with. With protected virtualization this has to change, since the hypervisor needs to read and sometimes also write these pieces of memory.
The hypervisor is supposed to poke the classic notifiers, if these are used, out of band with regards to ccw I/O. So these need to be allocated as DMA memory (which is shared memory for protected virtualization guests).
Let us factor out everything from struct
virtio_ccw_devicethat needs to be DMA memory in a satellite that is allocated as such.Note: The control blocks of I/O instructions do not need to be shared. These are marshalled by the ultravisor.
Signed-off-by Halil Pasic <[email protected]>Reviewed-by Pierre Morel <[email protected]>--- drivers/s390/virtio/virtio_ccw.c | 171 ++++++++++++++++--------------- 1 file changed, 90 insertions(+), 81 deletions(-)
(...)
drivers/s390/virtio/virtio_ccw.c static void virtio_ccw_reset(struct virtio_device *vdev) · line 739
Quoted 8 lines
static u64
virtio_ccw_get_features(struct virtio_device *vdev){drivers/s390/virtio/virtio_ccw.c +1
-1struct virtio_ccw_device *vcdev =to_vc_device (vdev);struct virtio_feature_desc *features; +struct ccw1 *ccw;int ret;u64 rc; -struct ccw1 *ccw;
I'd probably not have included unneeded code movement here, but no need to respin for that.
Quoted 11 lines
drivers/s390/virtio/virtio_ccw.c +2
-2- ccw =kzalloc (sizeof (*ccw),GFP_DMA |GFP_KERNEL ); + ccw =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*ccw));if (!ccw)return 0 ; - features =kzalloc (sizeof (*features),GFP_DMA |GFP_KERNEL ); + features =ccw_device_dma_zalloc (vcdev->cdev,sizeof (*features));if (!features) { rc =0 ; goto out_free;
(...)
Cornelia Huck
maintainerQuoted 2 lines · On Thu, 6 Jun 2019 13:51:19 +0200 Halil Pasic <[email protected]> wrote:
* Documentation is still very sketchy. I'm committed to improving this, but I'm currently hampered by some dependencies currently.
Have the "dependencies" been resolved in the meantime? It probably would be a good idea to include some documentation for what needs to be dma and what doesn't somewhere in the kernel documentation (IIRC we have a s390 drivers 'book' partially generated from kerneldoc; there's some general document about the cio interfaces as well, but I'm not sure how up-to-date that is.)
I think the code in here looks sane from my point of view (except for the one easy-to-fix issue I found); I would be fine with the virtio-ccw patches making it into the kernel via the s390 tree (and not via the virtio tree).
Michael S. Tsirkin
maintainerQuoted 13 lines · Cornelia Huck
Quoted 2 lines · On Thu, 6 Jun 2019 13:51:19 +0200 Halil Pasic <[email protected]> wrote:
* Documentation is still very sketchy. I'm committed to improving this, but I'm currently hampered by some dependencies currently.
Have the "dependencies" been resolved in the meantime? It probably would be a good idea to include some documentation for what needs to be dma and what doesn't somewhere in the kernel documentation (IIRC we have a s390 drivers 'book' partially generated from kerneldoc; there's some general document about the cio interfaces as well, but I'm not sure how up-to-date that is.)
Quoted 4 lines
I think the code in here looks sane from my point of view (except for the one easy-to-fix issue I found); I would be fine with the virtio-ccw patches making it into the kernel via the s390 tree (and not via the virtio tree).
Yes, me too.
Halil Pasic
maintainerRe: [PATCH v4 4/8] s390/airq: use DMA memory for adapter interrupts
In reply to Cornelia HuckQuoted 46 lines · On Tue, 11 Jun 2019 12:17:21 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 16 lines · On Thu, 6 Jun 2019 13:51:23 +0200 Halil Pasic <[email protected]> wrote:
Protected virtualization guests have to use shared pages for airq notifier bit vectors, because hypervisor needs to write these bits.
Let us make sure we allocate DMA memory for the notifier bit vectors by replacing the
kmem_cachewith adma_cacheandkalloc()withcio_dma_zalloc().Signed-off-by Halil Pasic <[email protected]>Reviewed-by Sebastian Ott <[email protected]>--- arch/s390/include/asm/airq.h | 2 ++ drivers/s390/cio/airq.c | 32 ++++++++++++++++++++------------ drivers/s390/cio/cio.h | 2 ++ drivers/s390/cio/css.c | 1 + 4 files changed, 25 insertions(+), 12 deletions(-)(...)
Quoted 4 lines
drivers/s390/cio/airq.c +0
-0@@ -295,12 +303,12 @@ unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start, }EXPORT_SYMBOL (airq_iv_scan);drivers/s390/cio/airq.c unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start, · line 312
Quoted 10 lines
drivers/s390/cio/airq.c +4
-3-static int __init airq_init (void ) +int __init airq_init (void ) { - airq_iv_cache = ) "airq_iv_cache", cache_line_size(), - cache_line_size(), 0, NULL); + airq_iv_cache =dma_pool_create ("airq_iv_cache" ,cio_get_dma_css_dev (), +cache_line_size (), +cache_line_size (),PAGE_SIZE );if (!airq_iv_cache)return -ENOMEM ;Sorry about not noticing that in the last iteration; but you may return an error here if
airq_iv_cachecould not be allocated...Quoted 3 lines
drivers/s390/cio/airq.c +0
-1return 0 ; } -subsys_initcall (airq_init);(...)
Quoted 13 lines
drivers/s390/cio/css.c static int __init css_bus_init(void) · line 1185
Quoted 9 lines
drivers/s390/cio/css.c +1
-0@@ -1182,6 +1182,7 @@ static int __init css_bus_init(void) ret =cio_dma_pool_init ();if (ret) goto out_unregister_pmn; +airq_init ();...but don't check the return code here. Probably a pathological case, but shouldn't you handle that error as well?
Tricky business... The problem is that the airq stuff ain't 'private' to the CIO subsystem (e.g. zPCI). I'm afraid failing to init css won't really prevent all usages.
My first thought was, that this is more or less analogous to what we had before. Namely kmem_cache_create() and dma_pool_create() should fail under similar circumstances, and the return value of airq_init() was ignored in do_initcall_level(). So I was like ignoring it seems to be consistent with previous state.
But, ouch, there is a big difference! While kmem_cache_zalloc() seems to tolerate the first argument (pointer to kmem_cache) being NULL the dma_pool_zalloc() does not.
IMHO the cleanest thing to do at this stage is to check if the airq_iv_cache is NULL and fail the allocation if it is (to preserve previous behavior).
I would prefer having a separate discussion on eventually changing the behavior (e.g. fail css initialization).
Connie, would that work with you? Thanks for spotting this!
Regards,
Halil
Quoted 4 lines
Quoted 3 lines
drivers/s390/cio/css.c +0
-0css_init_done =1 ;/* Enable default isc for I/O subchannels. */
Cornelia Huck
maintainerRe: [PATCH v4 4/8] s390/airq: use DMA memory for adapter interrupts
Quoted 63 lines · On Tue, 11 Jun 2019 16:27:21 +0200 Halil Pasic <[email protected]> wrote:
Quoted 46 lines · On Tue, 11 Jun 2019 12:17:21 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 16 lines · On Thu, 6 Jun 2019 13:51:23 +0200 Halil Pasic <[email protected]> wrote:
Protected virtualization guests have to use shared pages for airq notifier bit vectors, because hypervisor needs to write these bits.
Let us make sure we allocate DMA memory for the notifier bit vectors by replacing the
kmem_cachewith adma_cacheandkalloc()withcio_dma_zalloc().Signed-off-by Halil Pasic <[email protected]>Reviewed-by Sebastian Ott <[email protected]>--- arch/s390/include/asm/airq.h | 2 ++ drivers/s390/cio/airq.c | 32 ++++++++++++++++++++------------ drivers/s390/cio/cio.h | 2 ++ drivers/s390/cio/css.c | 1 + 4 files changed, 25 insertions(+), 12 deletions(-)(...)
Quoted 4 lines
drivers/s390/cio/airq.c +0
-0@@ -295,12 +303,12 @@ unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start, }EXPORT_SYMBOL (airq_iv_scan);drivers/s390/cio/airq.c unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start, · line 312
Quoted 10 lines
drivers/s390/cio/airq.c +4
-3-static int __init airq_init (void ) +int __init airq_init (void ) { - airq_iv_cache = ) "airq_iv_cache", cache_line_size(), - cache_line_size(), 0, NULL); + airq_iv_cache =dma_pool_create ("airq_iv_cache" ,cio_get_dma_css_dev (), +cache_line_size (), +cache_line_size (),PAGE_SIZE );if (!airq_iv_cache)return -ENOMEM ;Sorry about not noticing that in the last iteration; but you may return an error here if
airq_iv_cachecould not be allocated...Quoted 3 lines
drivers/s390/cio/airq.c +0
-1return 0 ; } -subsys_initcall (airq_init);(...)
Quoted 13 lines
drivers/s390/cio/css.c static int __init css_bus_init(void) · line 1185
Quoted 9 lines
drivers/s390/cio/css.c +1
-0@@ -1182,6 +1182,7 @@ static int __init css_bus_init(void) ret =cio_dma_pool_init ();if (ret) goto out_unregister_pmn; +airq_init ();...but don't check the return code here. Probably a pathological case, but shouldn't you handle that error as well?
Quoted 3 lines
Tricky business... The problem is that the airq stuff ain't 'private' to the CIO subsystem (e.g. zPCI). I'm afraid failing to init css won't really prevent all usages.
Architecture-wise, there's an unfortunate tie-in of some things like zPCI with the channel subsystem (most of that seems to come in via chsc and machine checks; but as you say, airq as well). I'd basically consider css to be a base system for virtually any I/O on s390...
Quoted 10 lines
My first thought was, that this is more or less analogous to what we had before. Namely
kmem_cache_create()anddma_pool_create()should fail under similar circumstances, and the return value ofairq_init()was ignored indo_initcall_level(). So I was like ignoring it seems to be consistent with previous state.But, ouch, there is a big difference! While
kmem_cache_zalloc()seems to tolerate the first argument (pointer tokmem_cache) being NULL thedma_pool_zalloc()does not.
Yeah. While previously continuing with a failed allocation simply was not very workable, now we actually would end up with crashes :(
Quoted 4 lines
IMHO the cleanest thing to do at this stage is to check if the
airq_iv_cacheis NULL and fail the allocation if it is (to preserve previous behavior).
That's probably the least invasive fix for now. Did you check whether any of the other dma pools this series introduces have a similar problem due to init not failing?
Quoted 3 lines
I would prefer having a separate discussion on eventually changing the behavior (e.g. fail css initialization).
I did a quick check of the common I/O layer code and one place that looks dangerous is the chsc initialization (where we get two pages that are later accessed unconditionally by the code).
All of this is related to not being able to fulfill some basic memory availability requirements early during boot and then discovering that pulling the emergency break did not actually stop the train. I'd vote for calling panic() if the common I/O layer cannot perform its setup; but as this is really a pathological case I also think we should solve that independently of this patch series.
Quoted 2 lines
Connie, would that work with you? Thanks for spotting this!
Yeah, let's give your approach a try.
Halil Pasic
maintainerRe: [PATCH v4 4/8] s390/airq: use DMA memory for adapter interrupts
Quoted 66 lines · On Tue, 11 Jun 2019 18:19:44 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 63 lines · On Tue, 11 Jun 2019 16:27:21 +0200 Halil Pasic <[email protected]> wrote:
Quoted 59 lines · On Tue, 11 Jun 2019 12:17:21 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 16 lines · On Thu, 6 Jun 2019 13:51:23 +0200 Halil Pasic <[email protected]> wrote:
Protected virtualization guests have to use shared pages for airq notifier bit vectors, because hypervisor needs to write these bits.
Let us make sure we allocate DMA memory for the notifier bit vectors by replacing the
kmem_cachewith adma_cacheandkalloc()withcio_dma_zalloc().Signed-off-by Halil Pasic <[email protected]>Reviewed-by Sebastian Ott <[email protected]>--- arch/s390/include/asm/airq.h | 2 ++ drivers/s390/cio/airq.c | 32 ++++++++++++++++++++------------ drivers/s390/cio/cio.h | 2 ++ drivers/s390/cio/css.c | 1 + 4 files changed, 25 insertions(+), 12 deletions(-)(...)
Quoted 4 lines
(quoted diff) +0
-0@@ -295,12 +303,12 @@ unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start, } EXPORT_SYMBOL(airq_iv_scan);Review unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start, · line 312
Quoted 10 lines
(quoted diff) +4
-3-static int __init airq_init(void) +int __init airq_init(void) { - airq_iv_cache = ) "airq_iv_cache", cache_line_size(), - cache_line_size(), 0, NULL); + airq_iv_cache = dma_pool_create("airq_iv_cache", cio_get_dma_css_dev(), + cache_line_size(), + cache_line_size(), PAGE_SIZE); if (!airq_iv_cache) return -ENOMEM;Sorry about not noticing that in the last iteration; but you may return an error here if
airq_iv_cachecould not be allocated...Quoted 3 lines
(quoted diff) +0
-1return 0; } -subsys_initcall(airq_init);(...)
drivers/s390/cio/css.c static int __init css_bus_init(void) · line 1185
Quoted 9 lines
drivers/s390/cio/css.c +1
-0@@ -1182,6 +1182,7 @@ static int __init css_bus_init(void) ret =cio_dma_pool_init ();if (ret) goto out_unregister_pmn; +airq_init ();...but don't check the return code here. Probably a pathological case, but shouldn't you handle that error as well?
Quoted 9 lines
Quoted 3 lines
Tricky business... The problem is that the airq stuff ain't 'private' to the CIO subsystem (e.g. zPCI). I'm afraid failing to init css won't really prevent all usages.
Architecture-wise, there's an unfortunate tie-in of some things like zPCI with the channel subsystem (most of that seems to come in via chsc and machine checks; but as you say, airq as well). I'd basically consider css to be a base system for virtually any I/O on s390...
I tend to agree.
Quoted 7 lines
Quoted 7 lines
My first thought was, that this is more or less analogous to what we had before. Namely
kmem_cache_create()anddma_pool_create()should fail under similar circumstances, and the return value ofairq_init()was ignored indo_initcall_level(). So I was like ignoring it seems to be consistent with previous state.
Quoted 7 lines
Quoted 3 lines
But, ouch, there is a big difference! While
kmem_cache_zalloc()seems to tolerate the first argument (pointer tokmem_cache) being NULL thedma_pool_zalloc()does not.Yeah. While previously continuing with a failed allocation simply was not very workable, now we actually would end up with crashes :(
Nod.
Quoted 9 lines
Quoted 4 lines
IMHO the cleanest thing to do at this stage is to check if the
airq_iv_cacheis NULL and fail the allocation if it is (to preserve previous behavior).That's probably the least invasive fix for now. Did you check whether any of the other dma pools this series introduces have a similar problem due to init not failing?
Good question!
I did a quick check. virtio_ccw_init() should be OK, because we don't register the driver if allocation fails, so the thing is going to end up dysfunctional as expected.
If however cio_dma_pool_init() fails, then we end up with the same problem with airqs, just on the !AIRQ_IV_CACHELINE code path. It can be fixed analogously: make cio_dma_zalloc() fail all allocation if cio_dma_pool_init() failed before.
The rest should be OK.
Quoted 8 lines
Quoted 3 lines
I would prefer having a separate discussion on eventually changing the behavior (e.g. fail css initialization).
I did a quick check of the common I/O layer code and one place that looks dangerous is the chsc initialization (where we get two pages that are later accessed unconditionally by the code).
Quoted 7 lines
All of this is related to not being able to fulfill some basic memory availability requirements early during boot and then discovering that pulling the emergency break did not actually stop the train. I'd vote for calling
panic()if the common I/O layer cannot perform its setup; but as this is really a pathological case I also think we should solve that independently of this patch series.
panic() sounds very reasonable to me. As an user I would like to see a message that tells me, I'm trying to boot with insufficient RAM. Is there such a message somewhere?
Quoted 5 lines
Quoted 2 lines
Connie, would that work with you? Thanks for spotting this!
Yeah, let's give your approach a try.
OK. I intend to send out v5 with these changes tomorrow in the afternoon:
drivers/s390/cio/airq.c +3 -2
@@ -142,7 +142,8 @@ struct airq_iv *airq_iv_create(unsigned long bits, unsigned long flags) size =iv_size (bits);if (flags &AIRQ_IV_CACHELINE ) { -if ((cache_line_size () *BITS_PER_BYTE ) < bits) +if ((cache_line_size () *BITS_PER_BYTE ) < bits + || !airq_iv_cache) goto out_free; iv->vector = dma_pool_zalloc(airq_iv_cache, GFP_KERNEL,
@@ -186,7 +187,7 @@ struct airq_iv *airq_iv_create(unsigned long bits, unsigned long flags)kfree (iv->ptr);kfree (iv->bitlock);kfree (iv->avail); -if (iv->flags &AIRQ_IV_CACHELINE ) +if (iv->flags &AIRQ_IV_CACHELINE && iv->vector)dma_pool_free (airq_iv_cache, iv->vector, iv->vector_dma);else cio_dma_free (iv->vector, size);
drivers/s390/cio/css.c +2 -0
@@ -1128,6 +1128,8 @@ void cio_gp_dma_free(struct gen_pool *gp_dma, void *cpu_addr, size_t size)*/ void *cio_dma_zalloc (size_t size) { +if (!cio_dma_pool) +return NULL ;return cio_gp_dma_zalloc (cio_dma_pool,cio_get_dma_css_dev (), size); }
Cornelia Huck
maintainerRe: [PATCH v4 4/8] s390/airq: use DMA memory for adapter interrupts
Quoted 17 lines · On Wed, 12 Jun 2019 02:32:31 +0200 Halil Pasic <[email protected]> wrote:
Quoted 11 lines · On Tue, 11 Jun 2019 18:19:44 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 3 lines · On Tue, 11 Jun 2019 16:27:21 +0200 Halil Pasic <[email protected]> wrote:
IMHO the cleanest thing to do at this stage is to check if the
airq_iv_cacheis NULL and fail the allocation if it is (to preserve previous behavior).That's probably the least invasive fix for now. Did you check whether any of the other dma pools this series introduces have a similar problem due to init not failing?
Good question!
Quoted 8 lines
I did a quick check.
virtio_ccw_init()should be OK, because we don't register the driver if allocation fails, so the thing is going to end up dysfunctional as expected.If however
cio_dma_pool_init()fails, then we end up with the same problem with airqs, just on the !AIRQ_IV_CACHELINE code path. It can be fixed analogously: makecio_dma_zalloc()fail all allocation ifcio_dma_pool_init()failed before.
Ok, makes sense.
Quoted 19 lines
The rest should be OK.
Quoted 8 lines
Quoted 3 lines
I would prefer having a separate discussion on eventually changing the behavior (e.g. fail css initialization).
I did a quick check of the common I/O layer code and one place that looks dangerous is the chsc initialization (where we get two pages that are later accessed unconditionally by the code).
Quoted 7 lines
All of this is related to not being able to fulfill some basic memory availability requirements early during boot and then discovering that pulling the emergency break did not actually stop the train. I'd vote for calling
panic()if the common I/O layer cannot perform its setup; but as this is really a pathological case I also think we should solve that independently of this patch series.
Quoted 3 lines
panic()sounds very reasonable to me. As an user I would like to see a message that tells me, I'm trying to boot with insufficient RAM. Is there such a message somewhere?
You could add it in the panic() message :) I would not spend overly much time on this, though, as this really sounds like someone is trying to run on a system that is way too tiny memory-wise for doing anything useful.
Quoted 15 lines
Quoted 6 lines
Quoted 3 lines
Connie, would that work with you? Thanks for spotting this!
Yeah, let's give your approach a try.
OK. I intend to send out v5 with these changes tomorrow in the afternoon:
drivers/s390/cio/airq.c +0
-0
drivers/s390/cio/airq.c struct airq_iv *airq_iv_create(unsigned long bits, unsigned long flags) · line 146
Quoted 6 lines
drivers/s390/cio/airq.c +2
-1@@ -142,7 +142,8 @@ struct airq_iv *airq_iv_create(unsigned long bits, unsigned long flags) size =iv_size (bits);if (flags &AIRQ_IV_CACHELINE ) { -if ((cache_line_size () *BITS_PER_BYTE ) < bits) +if ((cache_line_size () *BITS_PER_BYTE ) < bits + || !airq_iv_cache)
It's perhaps a bit more readable if you keep checking for airq_iv_cache on a separate if statement, but that's a matter of taste, I guess.
Quoted 17 lines
drivers/s390/cio/airq.c +1
-1goto out_free; iv->vector = dma_pool_zalloc(airq_iv_cache, GFP_KERNEL,@@ -186,7 +187,7 @@ struct airq_iv *airq_iv_create(unsigned long bits, unsigned long flags)kfree (iv->ptr);kfree (iv->bitlock);kfree (iv->avail); -if (iv->flags &AIRQ_IV_CACHELINE ) +if (iv->flags &AIRQ_IV_CACHELINE && iv->vector)dma_pool_free (airq_iv_cache, iv->vector, iv->vector_dma);else cio_dma_free (iv->vector, size);drivers/s390/cio/css.c +0
-0
drivers/s390/cio/css.c void cio_gp_dma_free(struct gen_pool *gp_dma, void *cpu_addr, size_t size) · line 1135
Quoted 8 lines
drivers/s390/cio/css.c +2
-0@@ -1128,6 +1128,8 @@ void cio_gp_dma_free(struct gen_pool *gp_dma, void *cpu_addr, size_t size)*/ void *cio_dma_zalloc (size_t size) { +if (!cio_dma_pool) +return NULL ;return cio_gp_dma_zalloc (cio_dma_pool,cio_get_dma_css_dev (), size); }
Just looked at patch 2 again, will comment there.
Cornelia Huck
maintainerRe: [PATCH v4 2/8] s390/cio: introduce DMA pools to cio
In reply to Halil PasicOn Thu, 6 Jun 2019 13:51:21 +0200
Halil Pasic <[email protected]> wrote:
[just looked at it in the context of failed init]
drivers/s390/cio/css.c static struct notifier_block css_power_notifier = { · line 1067
Quoted 2 lines
drivers/s390/cio/css.c +2
-0+static void __gp_dma_free_dma (struct gen_pool *pool, +struct gen_pool_chunk *chunk,void *data)
Just to note: the 'pool' is mandated by the api for this callback.
drivers/s390/cio/css.c static struct notifier_block css_power_notifier = { · line 1079
Quoted 12 lines
drivers/s390/cio/css.c +12
-0+{ +size_t chunk_size = chunk->end_addr - chunk->start_addr +1 ; + +dma_free_coherent ((struct device *) data, chunk_size, + (void *) chunk->start_addr, + (dma_addr_t ) chunk->phys_addr); +} + +void cio_gp_dma_destroy (struct gen_pool *gp_dma,struct device *dma_dev) +{ +if (!gp_dma) +return ;
So this seems fine.
Quoted 11 lines
drivers/s390/cio/css.c +11
-0+/* this is qite ugly but no better idea */ +gen_pool_for_each_chunk (gp_dma, __gp_dma_free_dma, dma_dev); +gen_pool_destroy (gp_dma); +} + +static int cio_dma_pool_init (void ) +{ +/* No need to free up the resources: compiled in */ + cio_dma_pool =cio_gp_dma_create (cio_get_dma_css_dev (),1 ); +if (!cio_dma_pool) +return -ENOMEM ;
drivers/s390/cio/css.c static struct notifier_block css_power_notifier = { · line 1100
Quoted 10 lines
drivers/s390/cio/css.c +10
-0+return 0 ; +} + +void *cio_gp_dma_zalloc (struct gen_pool *gp_dma,struct device *dma_dev, +size_t size) +{ +dma_addr_t dma_addr; +unsigned long addr; +size_t chunk_size; +
I'd probably do a quick exit for !gp_dma here as well (just to be on the safe side).
Quoted 7 lines
drivers/s390/cio/css.c +7
-0+ addr =gen_pool_alloc (gp_dma, size); +while (!addr) { + chunk_size =round_up (size,PAGE_SIZE ); + addr = (unsigned long )dma_alloc_coherent (dma_dev, + chunk_size, &dma_addr,CIO_DMA_GFP ); +if (!addr) +return NULL ;
drivers/s390/cio/css.c static struct notifier_block css_power_notifier = { · line 1117
Quoted 10 lines
drivers/s390/cio/css.c +10
-0+gen_pool_add_virt (gp_dma, addr, dma_addr, chunk_size,-1 ); + addr =gen_pool_alloc (gp_dma, size); + } +return (void *) addr; +} + +void cio_gp_dma_free (struct gen_pool *gp_dma,void *cpu_addr,size_t size) +{ +if (!cpu_addr) +return ;
This already makes it safe enough.
drivers/s390/cio/css.c static struct notifier_block css_power_notifier = { · line 1130
Quoted 13 lines
drivers/s390/cio/css.c +13
-0+memset (cpu_addr,0 , size); +gen_pool_free (gp_dma, (unsigned long ) cpu_addr, size); +} + +/* +* Allocate dma memory from the css global pool. Intended for memory not +* specific to any single device within the css. The allocated memory +* is not guaranteed to be 31-bit addressable. +* +* Caution: Not suitable for early stuff like console. +*/ +void *cio_dma_zalloc (size_t size) +{
If you check in the called function, you do not need to check here. Probably a matter of taste.
drivers/s390/cio/css.c static struct notifier_block css_power_notifier = { · line 1135
Quoted 5 lines
drivers/s390/cio/css.c +5
-0+return cio_gp_dma_zalloc (cio_dma_pool,cio_get_dma_css_dev (), size); +} + +void cio_dma_free (void *cpu_addr,size_t size) +{
This one should be safe due to the check in the called function.
Quoted 6 lines
drivers/s390/cio/css.c +3
-0+cio_gp_dma_free (cio_dma_pool, cpu_addr, size); +} +/* * Now that the driver core is running, we can setup our channel subsystem. * The struct subchannel's are created during probing.
Halil Pasic
maintainerRe: [PATCH v4 4/8] s390/airq: use DMA memory for adapter interrupts
In reply to Cornelia HuckQuoted 24 lines · On Wed, 12 Jun 2019 08:21:27 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 21 lines · On Wed, 12 Jun 2019 02:32:31 +0200 Halil Pasic <[email protected]> wrote:
Quoted 11 lines · On Tue, 11 Jun 2019 18:19:44 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 3 lines · On Tue, 11 Jun 2019 16:27:21 +0200 Halil Pasic <[email protected]> wrote:
IMHO the cleanest thing to do at this stage is to check if the
airq_iv_cacheis NULL and fail the allocation if it is (to preserve previous behavior).That's probably the least invasive fix for now. Did you check whether any of the other dma pools this series introduces have a similar problem due to init not failing?
Good question!
I did a quick check.
virtio_ccw_init()should be OK, because we don't register the driver if allocation fails, so the thing is going to end up dysfunctional as expected.
Quoted 6 lines
Quoted 4 lines
If however
cio_dma_pool_init()fails, then we end up with the same problem with airqs, just on the !AIRQ_IV_CACHELINE code path. It can be fixed analogously: makecio_dma_zalloc()fail all allocation ifcio_dma_pool_init()failed before.Ok, makes sense.
v5 is out with the fixes. I have no ack/r-b from you for patch 4. Would you like to give some, or should I proceed without?
Regards,
Halil
Cornelia Huck
maintainerRe: [PATCH v4 4/8] s390/airq: use DMA memory for adapter interrupts
Quoted 32 lines · On Wed, 12 Jun 2019 15:33:24 +0200 Halil Pasic <[email protected]> wrote:
Quoted 29 lines · On Wed, 12 Jun 2019 08:21:27 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 25 lines · On Wed, 12 Jun 2019 02:32:31 +0200 Halil Pasic <[email protected]> wrote:
Quoted 11 lines · On Tue, 11 Jun 2019 18:19:44 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 3 lines · On Tue, 11 Jun 2019 16:27:21 +0200 Halil Pasic <[email protected]> wrote:
IMHO the cleanest thing to do at this stage is to check if the
airq_iv_cacheis NULL and fail the allocation if it is (to preserve previous behavior).That's probably the least invasive fix for now. Did you check whether any of the other dma pools this series introduces have a similar problem due to init not failing?
Good question!
I did a quick check.
virtio_ccw_init()should be OK, because we don't register the driver if allocation fails, so the thing is going to end up dysfunctional as expected.If however
cio_dma_pool_init()fails, then we end up with the same problem with airqs, just on the !AIRQ_IV_CACHELINE code path. It can be fixed analogously: makecio_dma_zalloc()fail all allocation ifcio_dma_pool_init()failed before.
Quoted 4 lines
Quoted 1 line
Ok, makes sense.
v5 is out with the fixes. I have no ack/r-b from you for patch 4. Would you like to give some, or should I proceed without?
Please give this some more time, I will look at this later.