pci pass-through support for qemu/KVM on s390

From: Frank Blaschka <[email protected]>

This patch adds GISA (Guest Interrupt State Area) support to s390 kvm. GISA can be used for exitless interrupts. The patch provides a set of functions for GISA related operations like accessing GISA fields or registering ISCs for alert. Exploiters of GISA will follow with additional patches.

Signed-off-by Frank Blaschka <[email protected]>

Quoted 9 lines · On 04.09.14 12:52, [email protected] wrote:

From: Frank Blaschka <[email protected]>

This patch adds GISA (Guest Interrupt State Area) support to s390 kvm. GISA can be used for exitless interrupts. The patch provides a set of functions for GISA related operations like accessing GISA fields or registering ISCs for alert. Exploiters of GISA will follow with additional patches.

Signed-off-by Frank Blaschka <[email protected]>

That's a nice feature. However, please make sure that you maintain the abstraction levels.

What should happen is that you request an irqfd from FLIC. Then you associate that irqfd with the PCI device.

Thanks to that association, both parties can now talk to each other and negotiate their GISA number space and make sure things are connected.

However, it should always be possible to do things without this direct IRQ injection.

So you should be able to receive an irqfd event when an IRQ happened, so that VFIO user space applications can also handle interrupts for example.

And the same applies for interrupt injection. We also need to be able to inject an adapter interrupt from QEMU for emulated devices ;).

Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Frank Blaschka <[email protected]>

This patch implemets PCI pass-through kernel support for s390. Design approach is very similar to the x86 device assignment. User space executes the KVM_ASSIGN_PCI_DEVICE ioctl to create a proxy instance in the kernel KVM and connect this instance to the host pci device. s390 pci instructions are intercepted in kernel and operations are passed directly to the assigned pci device. To take advantage of all system z specific virtualization features we need to access the SIE control block residing in KVM. Also we have to enable z pci devices with special configuration information coming form the SIE block as well.

Signed-off-by Frank Blaschka <[email protected]>

---
 arch/s390/include/asm/kvm_host.h |    1 
 arch/s390/kvm/Makefile           |    2 
 arch/s390/kvm/intercept.c        |    1 
 arch/s390/kvm/kvm-s390.c         |   33 
 arch/s390/kvm/kvm-s390.h         |   17 
 arch/s390/kvm/pci.c              | 2130 +++++++++++++++++++++++++++++++++++++++
 arch/s390/kvm/priv.c             |   21 
 7 files changed, 2202 insertions(+), 3 deletions(-)

This set of patches implements pci pass-through support for qemu/KVM on s390. PCI support on s390 is very different from other platforms.
Major differences are:

1) all PCI operations are driven by special s390 instructions

2) all s390 PCI instructions are privileged

3) PCI config and memory spaces can not be mmap'ed

4) no classic interrupts (INTX, MSI). The pci hw understands the concept of requesting MSIX irqs but irqs are delivered as s390 adapter irqs.

5) For DMA access there is always an IOMMU required.

s390 pci implementation
does not support a complete memory to iommu mapping, dma mappings are created on request.

6) The OS does not get any informations about the physical layout of the PCI bus.

7) To take advantage of system z specific virtualization features we need to access the SIE control block residing in the kernel KVM

8) To enable system z specific virtualization features we have to manipulate the zpci device in kernel.

For this reasons I decided to implement a kernel based approach similar to x86 device assignment. There is a new qemu device (s390-pci) representing a pass through device on the host. Here is a sample qemu device configuration:

-device s390-pci,host=0000:00:00.0

The device executes the KVM_ASSIGN_PCI_DEVICE ioctl to create a proxy instance in the kernel KVM and connect this instance to the host pci device.

kernel patches apply to linux-kvm

s390: cio: chsc function to register GIB
s390: pci: export pci functions for pass-through usage
KVM: s390: Add GISA support
KVM: s390: Add PCI pass-through support

qemu patches apply to qemu-master

s390: Add PCI bus support
s390: Add PCI pass-through device support

Feedback and discussion is highly welcome ...

Quoted 5 lines · On Thu, 2014-09-04 at 12:52 +0200, [email protected] wrote:

This set of patches implements pci pass-through support for qemu/KVM on s390. PCI support on s390 is very different from other platforms.
Major differences are:

1) all PCI operations are driven by special s390 instructions

Generating config cycles is always arch specific.

Quoted 1 line

2) all s390 PCI instructions are privileged

While the operations to generate config cycles on x86 are not privileged, they must be arbitrated between accesses, so in a sense they're privileged.

Quoted 1 line

3) PCI config and memory spaces can not be mmap'ed

VFIO has mapping flags that allow any region to specify mmap support.

Quoted 2 lines

4) no classic interrupts (INTX, MSI). The pci hw understands the concept of requesting MSIX irqs but irqs are delivered as s390 adapter irqs.

VFIO delivers interrupts as eventfds regardless of the underlying platform mechanism.

Quoted 1 line

5) For DMA access there is always an IOMMU required.

x86 requires the same.

Quoted 3 lines

s390 pci implementation
does not support a complete memory to iommu mapping, dma mappings are created on request.

Sounds like POWER.

Quoted 3 lines

6) The OS does not get any informations about the physical layout of the PCI bus.

If that means that every device is isolated (seems unlikely for multifunction devices) then that makes IOMMU group support really easy.

Quoted 2 lines

7) To take advantage of system z specific virtualization features we need to access the SIE control block residing in the kernel KVM

The KVM-VFIO device allows interaction between VFIO devices and KVM.

Quoted 2 lines

8) To enable system z specific virtualization features we have to manipulate the zpci device in kernel.

VFIO supports different device backends, currently pci_dev and working towards platform devices. zpci might just be an extension to standard pci.

Quoted 21 lines

For this reasons I decided to implement a kernel based approach similar to x86 device assignment. There is a new qemu device (s390-pci) representing a pass through device on the host. Here is a sample qemu device configuration:

-device s390-pci,host=0000:00:00.0

The device executes the KVM_ASSIGN_PCI_DEVICE ioctl to create a proxy instance in the kernel KVM and connect this instance to the host pci device.

kernel patches apply to linux-kvm

s390: cio: chsc function to register GIB
s390: pci: export pci functions for pass-through usage
KVM: s390: Add GISA support
KVM: s390: Add PCI pass-through support

qemu patches apply to qemu-master

s390: Add PCI bus support
s390: Add PCI pass-through device support

Quoted 1 line

Feedback and discussion is highly welcome ...

KVM-based device assignment needs to go away. It's a horrible model for devices, it offers very little protection to the kernel, assumes every device is fully isolated and visible to the IOMMU, relies on smattering of sysfs files to operate, etc. x86, POWER, and ARM are all moving to VFIO-based device assignment. Why is s390 special enough to repeat all the mistakes that x86 did? Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 11 lines · Alex Williamson

Quoted 5 lines · On Thu, 2014-09-04 at 12:52 +0200, [email protected] wrote:

This set of patches implements pci pass-through support for qemu/KVM on s390. PCI support on s390 is very different from other platforms.
Major differences are:

1) all PCI operations are driven by special s390 instructions

Generating config cycles is always arch specific.

Quoted 1 line

2) all s390 PCI instructions are privileged

Quoted 8 lines

While the operations to generate config cycles on x86 are not privileged, they must be arbitrated between accesses, so in a sense they're privileged.

Quoted 1 line

3) PCI config and memory spaces can not be mmap'ed

VFIO has mapping flags that allow any region to specify mmap support.

Hi Alex,

thx for your reply.

Let me elaborate a little bit ore on 1 - 3. Config and memory space can not be accessed via memory operations. You have to use special s390 instructions. This instructions can not be executed in user space. So there is no other way than executing this instructions in kernel. Yes vfio does support a slow path via ioctrl we could use, but this seems suboptimal from performance point of view.

Quoted 7 lines

Quoted 3 lines

4) no classic interrupts (INTX, MSI). The pci hw understands the concept of requesting MSIX irqs but irqs are delivered as s390 adapter irqs.

VFIO delivers interrupts as eventfds regardless of the underlying platform mechanism.

yes that's right, but then we have to do platform specific stuff to present the irq to the guest. I do not say this is impossible but we have add s390 specific code to vfio.

Quoted 9 lines

Quoted 2 lines

5) For DMA access there is always an IOMMU required.

x86 requires the same.

Quoted 3 lines

s390 pci implementation
does not support a complete memory to iommu mapping, dma mappings are created on request.

Quoted 1 line

Sounds like POWER.

Don't know the details from power, maybe it is similar but not the same. We might be able to extend vfio to have a new interface allowing us to do DMA mappings on request.

Quoted 8 lines

Quoted 4 lines

6) The OS does not get any informations about the physical layout of the PCI bus.

If that means that every device is isolated (seems unlikely for multifunction devices) then that makes IOMMU group support really easy.

OK

Quoted 8 lines

Quoted 2 lines

7) To take advantage of system z specific virtualization features we need to access the SIE control block residing in the kernel KVM

The KVM-VFIO device allows interaction between VFIO devices and KVM.

Quoted 2 lines

8) To enable system z specific virtualization features we have to manipulate the zpci device in kernel.

Quoted 4 lines

VFIO supports different device backends, currently pci_dev and working towards platform devices. zpci might just be an extension to standard pci.

7 - 8 At least this is not as straightforward as the pure kernel approach, but I have to dig into that in more detail if we could only agree on a vfio solution.

Quoted 22 lines

Quoted 22 lines

For this reasons I decided to implement a kernel based approach similar to x86 device assignment. There is a new qemu device (s390-pci) representing a pass through device on the host. Here is a sample qemu device configuration:

-device s390-pci,host=0000:00:00.0

The device executes the KVM_ASSIGN_PCI_DEVICE ioctl to create a proxy instance in the kernel KVM and connect this instance to the host pci device.

kernel patches apply to linux-kvm

s390: cio: chsc function to register GIB
s390: pci: export pci functions for pass-through usage
KVM: s390: Add GISA support
KVM: s390: Add PCI pass-through support

qemu patches apply to qemu-master

s390: Add PCI bus support
s390: Add PCI pass-through device support

Quoted 9 lines

Quoted 1 line

Feedback and discussion is highly welcome ...

KVM-based device assignment needs to go away. It's a horrible model for devices, it offers very little protection to the kernel, assumes every device is fully isolated and visible to the IOMMU, relies on smattering of sysfs files to operate, etc. x86, POWER, and ARM are all moving to VFIO-based device assignment. Why is s390 special enough to repeat all the mistakes that x86 did? Thanks,

Is this your personal opinion or was this a strategic decision of the QEMU/KVM community? Can anybody give us direction about this?

Actually I can understand your point. In the last weeks I did some development and testing regarding the use of vfio too. But the in kernel solutions seems to offer the best performance and most straighforward implementation for our platform.

Quoted 23 lines · Frank Blaschka

Quoted 11 lines · Alex Williamson

Quoted 5 lines · On Thu, 2014-09-04 at 12:52 +0200, [email protected] wrote:

This set of patches implements pci pass-through support for qemu/KVM on s390. PCI support on s390 is very different from other platforms.
Major differences are:

1) all PCI operations are driven by special s390 instructions

Generating config cycles is always arch specific.

Quoted 1 line

2) all s390 PCI instructions are privileged

Quoted 8 lines

While the operations to generate config cycles on x86 are not privileged, they must be arbitrated between accesses, so in a sense they're privileged.

Quoted 1 line

3) PCI config and memory spaces can not be mmap'ed

VFIO has mapping flags that allow any region to specify mmap support.

Hi Alex,

Quoted 8 lines

thx for your reply.

Let me elaborate a little bit ore on 1 - 3. Config and memory space can not be accessed via memory operations. You have to use special s390 instructions. This instructions can not be executed in user space. So there is no other way than executing this instructions in kernel. Yes vfio does support a slow path via ioctrl we could use, but this seems suboptimal from performance point of view.

Ah, I missed the "memory spaces" part ;). I agree that it's "suboptimal" to call into the kernel for every PCI access, but I still think that VFIO provides the correct abstraction layer for us to use. If nothing else, it would at least give us identical configuration to x86 and nice debugability en par with the other platforms.

Quoted 8 lines

Quoted 7 lines

Quoted 3 lines

4) no classic interrupts (INTX, MSI). The pci hw understands the concept of requesting MSIX irqs but irqs are delivered as s390 adapter irqs.

VFIO delivers interrupts as eventfds regardless of the underlying platform mechanism.

Quoted 3 lines

yes that's right, but then we have to do platform specific stuff to present the irq to the guest. I do not say this is impossible but we have add s390 specific code to vfio.

Not at all - interrupt delivery is completely transparent to VFIO.

Quoted 9 lines

Quoted 9 lines

Quoted 2 lines

5) For DMA access there is always an IOMMU required.

x86 requires the same.

Quoted 3 lines

s390 pci implementation
does not support a complete memory to iommu mapping, dma mappings are created on request.

Quoted 5 lines

Quoted 1 line

Sounds like POWER.

Don't know the details from power, maybe it is similar but not the same. We might be able to extend vfio to have a new interface allowing us to do DMA mappings on request.

We already have that.

Quoted 19 lines

Quoted 8 lines

Quoted 4 lines

6) The OS does not get any informations about the physical layout of the PCI bus.

If that means that every device is isolated (seems unlikely for multifunction devices) then that makes IOMMU group support really easy.

OK

Quoted 8 lines

Quoted 2 lines

7) To take advantage of system z specific virtualization features we need to access the SIE control block residing in the kernel KVM

The KVM-VFIO device allows interaction between VFIO devices and KVM.

Quoted 2 lines

8) To enable system z specific virtualization features we have to manipulate the zpci device in kernel.

Quoted 7 lines

Quoted 4 lines

VFIO supports different device backends, currently pci_dev and working towards platform devices. zpci might just be an extension to standard pci.

7 - 8 At least this is not as straightforward as the pure kernel approach, but I have to dig into that in more detail if we could only agree on a vfio solution.

Please do so, yes :).

Quoted 32 lines

Quoted 22 lines

Quoted 22 lines

For this reasons I decided to implement a kernel based approach similar to x86 device assignment. There is a new qemu device (s390-pci) representing a pass through device on the host. Here is a sample qemu device configuration:

-device s390-pci,host=0000:00:00.0

The device executes the KVM_ASSIGN_PCI_DEVICE ioctl to create a proxy instance in the kernel KVM and connect this instance to the host pci device.

kernel patches apply to linux-kvm

s390: cio: chsc function to register GIB
s390: pci: export pci functions for pass-through usage
KVM: s390: Add GISA support
KVM: s390: Add PCI pass-through support

qemu patches apply to qemu-master

s390: Add PCI bus support
s390: Add PCI pass-through device support

Quoted 9 lines

Quoted 1 line

Feedback and discussion is highly welcome ...

KVM-based device assignment needs to go away. It's a horrible model for devices, it offers very little protection to the kernel, assumes every device is fully isolated and visible to the IOMMU, relies on smattering of sysfs files to operate, etc. x86, POWER, and ARM are all moving to VFIO-based device assignment. Why is s390 special enough to repeat all the mistakes that x86 did? Thanks,

Quoted 7 lines

Is this your personal opinion or was this a strategic decision of the QEMU/KVM community? Can anybody give us direction about this?

Actually I can understand your point. In the last weeks I did some development and testing regarding the use of vfio too. But the in kernel solutions seems to offer the best performance and most straighforward implementation for our platform.

I don't see why there should be any difference in performance between the two approaches if done right. However, we'd get a lot of benefits. Most notably the fact that s390 is not different from everyone else.

I think you'll see that it's pretty straight forward to do things VFIO style once you get the hang of it :).

Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 49 lines · Alexander Graf

Quoted 23 lines · Frank Blaschka

Quoted 11 lines · Alex Williamson

Quoted 5 lines · On Thu, 2014-09-04 at 12:52 +0200, [email protected] wrote:

This set of patches implements pci pass-through support for qemu/KVM on s390. PCI support on s390 is very different from other platforms.
Major differences are:

1) all PCI operations are driven by special s390 instructions

Generating config cycles is always arch specific.

Quoted 1 line

2) all s390 PCI instructions are privileged

Quoted 8 lines

While the operations to generate config cycles on x86 are not privileged, they must be arbitrated between accesses, so in a sense they're privileged.

Quoted 1 line

3) PCI config and memory spaces can not be mmap'ed

VFIO has mapping flags that allow any region to specify mmap support.

Hi Alex,

Quoted 8 lines

thx for your reply.

Let me elaborate a little bit ore on 1 - 3. Config and memory space can not be accessed via memory operations. You have to use special s390 instructions. This instructions can not be executed in user space. So there is no other way than executing this instructions in kernel. Yes vfio does support a slow path via ioctrl we could use, but this seems suboptimal from performance point of view.

Ah, I missed the "memory spaces" part ;). I agree that it's "suboptimal" to call into the kernel for every PCI access, but I still think that VFIO provides the correct abstraction layer for us to use. If nothing else, it would at least give us identical configuration to x86 and nice debugability en par with the other platforms.

Quoted 8 lines

Quoted 7 lines

Quoted 3 lines

4) no classic interrupts (INTX, MSI). The pci hw understands the concept of requesting MSIX irqs but irqs are delivered as s390 adapter irqs.

VFIO delivers interrupts as eventfds regardless of the underlying platform mechanism.

Quoted 6 lines

Quoted 3 lines

yes that's right, but then we have to do platform specific stuff to present the irq to the guest. I do not say this is impossible but we have add s390 specific code to vfio.

Not at all - interrupt delivery is completely transparent to VFIO.

interrupt yes, but MSIX no

Quoted 11 lines

Quoted 11 lines

Quoted 10 lines

Quoted 2 lines

5) For DMA access there is always an IOMMU required.

x86 requires the same.

Quoted 3 lines

s390 pci implementation
does not support a complete memory to iommu mapping, dma mappings are created on request.

Sounds like POWER.

Quoted 6 lines

Quoted 3 lines

Don't know the details from power, maybe it is similar but not the same. We might be able to extend vfio to have a new interface allowing us to do DMA mappings on request.

We already have that.

Great, can you give me some pointers how to use? Thx!

This set of patches implements pci pass-through support for qemu/KVM on s390. PCI support on s390 is very different from other platforms.
Major differences are:

1) all PCI operations are driven by special s390 instructions
2) all s390 PCI instructions are privileged
3) PCI config and memory spaces can not be mmap'ed

4) no classic interrupts (INTX, MSI). The pci hw understands the concept of requesting MSIX irqs but irqs are delivered as s390 adapter irqs.

5) For DMA access there is always an IOMMU required. s390 pci implementation does not support a complete memory to iommu mapping, dma mappings are created on request.

6) The OS does not get any informations about the physical layout of the PCI bus.

7) To take advantage of system z specific virtualization features we need to access the SIE control block residing in the kernel KVM

8) To enable system z specific virtualization features we have to manipulate the zpci device in kernel.

For this reasons I decided to implement a kernel based approach similar to x86 device assignment. There is a new qemu device (s390-pci) representing a

Quoted 7 lines · On 04.09.14 12:52, [email protected] wrote:

This set of patches implements pci pass-through support for qemu/KVM on s390. PCI support on s390 is very different from other platforms.
Major differences are:

1) all PCI operations are driven by special s390 instructions
2) all s390 PCI instructions are privileged
3) PCI config and memory spaces can not be mmap'ed

That's ok, vfio abstracts config space anyway.

Quoted 2 lines

4) no classic interrupts (INTX, MSI). The pci hw understands the concept of requesting MSIX irqs but irqs are delivered as s390 adapter irqs.

This is in line with other implementations. Interrupts go from

device -> PHB -> PIC -> CPU

(some times you can have another converter device in between)

In your case, the PHB converts INTX and MSI interrupts to Adapter interrupts to go to the floating interrupt controller. Same thing as everyone else really.

Quoted 3 lines

5) For DMA access there is always an IOMMU required. s390 pci implementation does not support a complete memory to iommu mapping, dma mappings are created on request.

Sounds great :). So I suppose we should implement a guest facing IOMMU?

Quoted 2 lines

6) The OS does not get any informations about the physical layout of the PCI bus.

So how does it know whether different devices are behind the same IOMMU context? Or can we assume that every device has its own context?

Quoted 3 lines

7) To take advantage of system z specific virtualization features we need to access the SIE control block residing in the kernel KVM

Pleas elaborate.

Quoted 2 lines

8) To enable system z specific virtualization features we have to manipulate the zpci device in kernel.

Why?

Quoted 3 lines

For this reasons I decided to implement a kernel based approach similar to x86 device assignment. There is a new qemu device (s390-pci) representing a

I fail to see the rationale and I definitely don't want to see anything even remotely similar to the legacy x86 device assignment on s390 ;).

Can't we just enhance VFIO?

Also, I think we'll get the cleanest model if we start off with an implementation that allows us to add emulated PCI devices to an s390x machine and only then follow on with physical ones.

Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 18 lines · Alexander Graf

Quoted 7 lines · On 04.09.14 12:52, [email protected] wrote:

This set of patches implements pci pass-through support for qemu/KVM on s390. PCI support on s390 is very different from other platforms.
Major differences are:

1) all PCI operations are driven by special s390 instructions
2) all s390 PCI instructions are privileged
3) PCI config and memory spaces can not be mmap'ed

That's ok, vfio abstracts config space anyway.

Quoted 2 lines

4) no classic interrupts (INTX, MSI). The pci hw understands the concept of requesting MSIX irqs but irqs are delivered as s390 adapter irqs.

This is in line with other implementations. Interrupts go from

Quoted 8 lines

device -> PHB -> PIC -> CPU

(some times you can have another converter device in between)

In your case, the PHB converts INTX and MSI interrupts to Adapter interrupts to go to the floating interrupt controller. Same thing as everyone else really.

Yes, I think this can be done, but we need s390 specific changes in vfio.

Quoted 6 lines

Quoted 3 lines

5) For DMA access there is always an IOMMU required. s390 pci implementation does not support a complete memory to iommu mapping, dma mappings are created on request.

Sounds great :). So I suppose we should implement a guest facing IOMMU?

Quoted 5 lines

Quoted 2 lines

6) The OS does not get any informations about the physical layout of the PCI bus.

So how does it know whether different devices are behind the same IOMMU context? Or can we assume that every device has its own context?

Actually yes

Quoted 5 lines

Quoted 4 lines

7) To take advantage of system z specific virtualization features we need to access the SIE control block residing in the kernel KVM

Quoted 7 lines

Pleas elaborate.

Quoted 2 lines

8) To enable system z specific virtualization features we have to manipulate the zpci device in kernel.

Why?

We have following s390 specific virtualization features:

1) interpretive execution of pci load/store instruction. If we use this function pci access does not get intercepted (no SIE exit) but is handled via microcode. To enable this we have to disable zpci device and enable it again with information from the SIE control block.

Further in qemu problem is: vfio traps access to
MSIX table so we have to find another way programming msix if we do not get intercepts for memory space access.

2) Adapter event forwarding (with alerting). This is a mechanism the adpater event (irq) is directly forwarded to the guest. To set this up we also need to manipulate the zpci device (in kernel) with information form the SIE block. Exploiting GISA is only one part of this mechanism.

Both might be possible with some more or less nice looking vfio extensions. As I said before we have to dig more into. Also this can be further optimazation steps later if we have a running vfio implementation on the platform.

Quoted 5 lines

Quoted 4 lines

For this reasons I decided to implement a kernel based approach similar to x86 device assignment. There is a new qemu device (s390-pci) representing a

Quoted 5 lines

I fail to see the rationale and I definitely don't want to see anything even remotely similar to the legacy x86 device assignment on s390 ;).

Can't we just enhance VFIO?

Probably yes, but we need some vfio changes (kernel and qemu)

Quoted 5 lines

Also, I think we'll get the cleanest model if we start off with an implementation that allows us to add emulated PCI devices to an s390x machine and only then follow on with physical ones.

I can already do this. With some more s390 intercepts a device can be detected and guest is able to access config/memory space. Unfortunately s390 platform does not support I/O bars so non of the emulated devices will work on the platform ...

Quoted 4 lines

Quoted 4 lines

1) interpretive execution of pci load/store instruction. If we use this function pci access does not get intercepted (no SIE exit) but is handled via microcode. To enable this we have to disable zpci device and enable it again with information from the SIE control block.

Hrm. So how about you create a special vm ioctl for KVM that allows you to attach a VFIO device fd into the KVM VM context? Then the default would stay "accessible by mmap traps", but we could accelerate it with KVM.

Il 06/09/2014 01:19, Alexander Graf ha scritto:

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

1) interpretive execution of pci load/store instruction. If we use this function pci access does not get intercepted (no SIE exit) but is handled via microcode. To enable this we have to disable zpci device and enable it again with information from the SIE control block.

Quoted 3 lines

Hrm. So how about you create a special vm ioctl for KVM that allows you to attach a VFIO device fd into the KVM VM context? Then the default would stay "accessible by mmap traps", but we could accelerate it with KVM.

There is already KVM_DEV_VFIO_GROUP_ADD and KVM_DEV_VFIO_GROUP_DEL.

Right now, they result in a call to kvm_arch_register_noncoherent_dma or kvm_arch_unregister_noncoherent_dma, but you can add more hooks.