KVM: Improve MMIO Coalescing API
Subsystems (3)
cover KVM: Improve MMIO Coalescing API
The current MMIO coalescing design has a few drawbacks which limit its usefulness. Currently all coalesced MMIO zones use the same ring buffer. That means that upon a userspace exit we have to handle potentially unrelated MMIO writes synchronously. And a VM-wide lock needs to be taken in the kernel when an MMIO exit occurs.
Additionally, there is no direct way for userspace to be notified about coalesced MMIO writes. If the next MMIO exit to userspace is when the ring buffer has filled then a substantial (and unbounded) amount of time may have passed since the first coalesced MMIO.
This series adds new ioctls to KVM that allow for greater control by making it possible to associate different MMIO zones with different ring buffers. It also allows userspace to use poll() to check for coalesced writes in order to avoid userspace exits in vCPU threads (see patch 3 for why this can be useful).
The idea of improving the API in this way originally came from Paul Durrant ([email protected]) but the implementation is mine.
The first patch in the series is a bug in the existing code that I discovered while writing a selftest and can be merged independently.
Ilias Stamatis (6):
KVM: Fix coalesced_mmio_has_room()
KVM: Add KVM_CREATE_COALESCED_MMIO_BUFFER ioctl
KVM: Support poll() on coalesced mmio buffer fds
KVM: Add KVM_(UN)REGISTER_COALESCED_MMIO2 ioctls
KVM: Documentation: Document v2 of coalesced MMIO API
KVM: selftests: Add coalesced_mmio_test
Documentation/virt/kvm/api.rst | 91 +++++ include/linux/kvm_host.h | 1 + include/uapi/linux/kvm.h | 18 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/coalesced_mmio_test.c | 310 ++++++++++++++++++ virt/kvm/coalesced_mmio.c | 202 +++++++++++- virt/kvm/coalesced_mmio.h | 17 +- virt/kvm/kvm_main.c | 40 ++- 8 files changed, 659 insertions(+), 21 deletions(-) create mode 100644 tools/testing/selftests/kvm/coalesced_mmio_test.c
Signature · 1 line
2.34.1
PATCH 1/6 KVM: Fix coalesced_mmio_has_room()
The following calculation used in coalesced_mmio_has_room() to check whether the ring buffer is full is wrong and only allows half the buffer to be used.
avail = (ring->first - last - 1) % KVM_COALESCED_MMIO_MAX; if (avail == 0)
/* full */
The % operator in C is not the modulo operator but the remainder operator. Modulo and remainder operators differ with respect to negative values. But all values are unsigned in this case anyway.
The above might have worked as expected in python for example:
Quoted 1 line
Quoted 1 line
Quoted 1 line
(-86) % 170
84
However it doesn't work the same way in C.
printf("avail: %d\n", (-86) % 170);printf("avail: %u\n", (-86) % 170);printf("avail: %u\n", (-86u) % 170u);
Using gcc-11 these print:
avail: -86
avail: 4294967210
avail: 0
Fix the calculation and allow all but one entries in the buffer to be used as originally intended.
--- virt/kvm/coalesced_mmio.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
virt/kvm/coalesced_mmio.c +1 -3
@@ -43,7 +43,6 @@ static int coalesced_mmio_in_range(struct kvm_coalesced_mmio_dev *dev,static int coalesced_mmio_has_room (struct kvm_coalesced_mmio_dev *dev,u32 last) {struct kvm_coalesced_mmio_ring *ring; -unsigned avail;/* Are we able to batch it ? */
@@ -52,8 +51,7 @@ static int coalesced_mmio_has_room(struct kvm_coalesced_mmio_dev *dev, u32 last)* there is always one unused entry in the buffer */ ring = dev->kvm->coalesced_mmio_ring; - avail = (ring->first - last -1 ) %KVM_COALESCED_MMIO_MAX ; -if (avail ==0 ) { +if ((last +1 ) %KVM_COALESCED_MMIO_MAX ==READ_ONCE (ring->first)) {/* full */ return 0 ; }
Signature · 1 line
2.34.1
Re: [PATCH 1/6] KVM: Fix coalesced_mmio_has_room()
Quoted 28 lines · Ilias Stamatis
The following calculation used in
coalesced_mmio_has_room()to check whether the ring buffer is full is wrong and only allows half the buffer to be used.avail = (
ring->first- last - 1) %KVM_COALESCED_MMIO_MAX; if (avail == 0)/* full */The % operator in C is not the modulo operator but the remainder operator. Modulo and remainder operators differ with respect to negative values. But all values are unsigned in this case anyway.
The above might have worked as expected in python for example:
Quoted 1 line
Quoted 1 line
Quoted 1 line
(-86) % 170
84
However it doesn't work the same way in C.
printf("avail: %d\n", (-86) % 170);printf("avail: %u\n", (-86) % 170);printf("avail: %u\n", (-86u) % 170u);Using gcc-11 these print:
avail: -86
avail: 4294967210
avail: 0
Quoted 9 lines
Fix the calculation and allow all but one entries in the buffer to be used as originally intended.
Fixes 105f8d40a737 ("KVM: Calculate available entries in coalesced mmio ring")Signed-off-by Ilias Stamatis <[email protected]>--- virt/kvm/coalesced_mmio.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
Re: [PATCH 1/6] KVM: Fix coalesced_mmio_has_room()
In reply to Ilias StamatisQuoted 22 lines · Ilias Stamatis
The following calculation used in
coalesced_mmio_has_room()to check whether the ring buffer is full is wrong and only allows half the buffer to be used.avail = (
ring->first- last - 1) %KVM_COALESCED_MMIO_MAX; if (avail == 0)/* full */The % operator in C is not the modulo operator but the remainder operator. Modulo and remainder operators differ with respect to negative values. But all values are unsigned in this case anyway.
The above might have worked as expected in python for example:
Quoted 1 line
Quoted 1 line
Quoted 1 line
(-86) % 170
84
However it doesn't work the same way in C.
printf("avail: %d\n", (-86) % 170);printf("avail: %u\n", (-86) % 170);printf("avail: %u\n", (-86u) % 170u);
Quoted 5 lines
Using gcc-11 these print:
avail: -86
avail: 4294967210
avail: 0
Where exactly do you see a problem? As you correctly point out, all values are unsigned, so unsigned arithmetics with wraparound applies, and then % operator is applied to the resulting unsigned value. Out your three examples, only the last one is relevant, and it's perfectly the intended behavior.
Thanks,
Roman.
Amazon Web Services Development Center Germany GmbH Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597
Re: [PATCH 1/6] KVM: Fix coalesced_mmio_has_room()
Quoted 29 lines · Roman Kagan
Quoted 22 lines · Ilias Stamatis
The following calculation used in
coalesced_mmio_has_room()to check whether the ring buffer is full is wrong and only allows half the buffer to be used.avail = (
ring->first- last - 1) %KVM_COALESCED_MMIO_MAX; if (avail == 0)/* full */The % operator in C is not the modulo operator but the remainder operator. Modulo and remainder operators differ with respect to negative values. But all values are unsigned in this case anyway.
The above might have worked as expected in python for example:
Quoted 1 line
Quoted 1 line
Quoted 1 line
(-86) % 170
84
However it doesn't work the same way in C.
printf("avail: %d\n", (-86) % 170);printf("avail: %u\n", (-86) % 170);printf("avail: %u\n", (-86u) % 170u);Quoted 5 lines
Using gcc-11 these print:
avail: -86
avail: 4294967210
avail: 0
Quoted 8 lines
Where exactly do you see a problem? As you correctly point out, all values are unsigned, so unsigned arithmetics with wraparound applies, and then % operator is applied to the resulting unsigned value. Out your three examples, only the last one is relevant, and it's perfectly the intended behavior.
Thanks,
Roman.
KVM_COALESCED_MMIO_MAX on x86 is 170, which means the ring buffer has 170 entries (169 of which should be usable).
If first = 0 and last = 85 then the calculation gives 0 available entries in which case we consider the buffer to be full and we exit to userspace. But we shouldn't as there are still 84 unused entries.
So I don't see how this could have been the intended behaviour.
Ilias
Re: [PATCH 1/6] KVM: Fix coalesced_mmio_has_room()
Quoted 39 lines · Ilias
Quoted 29 lines · Roman Kagan
Quoted 22 lines · Ilias Stamatis
The following calculation used in
coalesced_mmio_has_room()to check whether the ring buffer is full is wrong and only allows half the buffer to be used.avail = (
ring->first- last - 1) %KVM_COALESCED_MMIO_MAX; if (avail == 0)/* full */The % operator in C is not the modulo operator but the remainder operator. Modulo and remainder operators differ with respect to negative values. But all values are unsigned in this case anyway.
The above might have worked as expected in python for example:
Quoted 1 line
Quoted 1 line
Quoted 1 line
(-86) % 170
84
However it doesn't work the same way in C.
printf("avail: %d\n", (-86) % 170);printf("avail: %u\n", (-86) % 170);printf("avail: %u\n", (-86u) % 170u);Quoted 5 lines
Using gcc-11 these print:
avail: -86
avail: 4294967210
avail: 0Quoted 8 lines
Where exactly do you see a problem? As you correctly point out, all values are unsigned, so unsigned arithmetics with wraparound applies, and then % operator is applied to the resulting unsigned value. Out your three examples, only the last one is relevant, and it's perfectly the intended behavior.
Thanks,
Roman.
Quoted 6 lines
KVM_COALESCED_MMIO_MAXon x86 is 170, which means the ring buffer has 170 entries (169 of which should be usable).If first = 0 and last = 85 then the calculation gives 0 available entries in which case we consider the buffer to be full and we exit to userspace. But we shouldn't as there are still 84 unused entries.
You want to add a stride instead
avail = (ring->first + KVM_COALESCED_MMIO_MAX - 1 - last) % KVM_COALESCED_MMIO_MAX;
As long as the stride is smaller than half the wraparound, you can think of the values being on an infinite non-negative axis.
Roman.
Amazon Web Services Development Center Germany GmbH Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597
PATCH 2/6 KVM: Add KVM_CREATE_COALESCED_MMIO_BUFFER ioctl
In reply to Ilias StamatisThe current MMIO coalescing design has a few drawbacks which limit its usefulness. Currently all coalesced MMIO zones use the same ring buffer. That means that upon a userspace exit we have to handle potentially unrelated MMIO writes synchronously. And a VM-wide lock needs to be taken in the kernel when an MMIO exit occurs.
Additionally, there is no direct way for userspace to be notified about coalesced MMIO writes. If the next MMIO exit to userspace is when the ring buffer has filled then a substantial (and unbounded) amount of time may have passed since the first coalesced MMIO.
Add a KVM_CREATE_COALESCED_MMIO_BUFFER ioctl to KVM. This ioctl simply returns a file descriptor to the caller but does not allocate a ring buffer. Userspace can then pass this fd to mmap() to actually allocate a buffer and map it to its address space.
Subsequent patches will allow userspace to:
- Associate the fd with a coalescing zone when registering it so that writes to that zone are accumulated in that specific ring buffer rather than the VM-wide one.
- Poll for MMIO writes using this fd.
--- include/linux/kvm_host.h | 1 + include/uapi/linux/kvm.h | 2 + virt/kvm/coalesced_mmio.c | 142 +++++++++++++++++++++++++++++++++++--- virt/kvm/coalesced_mmio.h | 9 +++ virt/kvm/kvm_main.c | 4 ++ 5 files changed, 150 insertions(+), 8 deletions(-)
include/linux/kvm_host.h +1 -0
@@ -799,6 +799,7 @@ struct kvm {struct kvm_coalesced_mmio_ring *coalesced_mmio_ring;spinlock_t ring_lock;struct list_head coalesced_zones; +struct list_head coalesced_buffers;#endif struct mutex irq_lock;
include/uapi/linux/kvm.h +2 -0
@@ -1548,4 +1548,6 @@ struct kvm_create_guest_memfd { __u64 reserved[6]; }; +#define KVM_CREATE_COALESCED_MMIO_BUFFER _IO(KVMIO, 0xd5) +#endif /* __LINUX_KVM_H */
virt/kvm/coalesced_mmio.c +134 -8
210 lines not on this page · raw message
virt/kvm/coalesced_mmio.h +9 -0
@@ -20,6 +20,14 @@ struct kvm_coalesced_mmio_dev {struct kvm_io_device dev;struct kvm *kvm;struct kvm_coalesced_mmio_zone zone; +struct kvm_coalesced_mmio_buffer_dev *buffer_dev; +}; + +struct kvm_coalesced_mmio_buffer_dev { +struct list_head list; +struct kvm *kvm; +spinlock_t ring_lock; +struct kvm_coalesced_mmio_ring *ring; };int kvm_coalesced_mmio_init (struct kvm *kvm);
@@ -28,6 +36,7 @@ int kvm_vm_ioctl_register_coalesced_mmio(struct kvm *kvm, struct kvm_coalesced_mmio_zone *zone);int kvm_vm_ioctl_unregister_coalesced_mmio (struct kvm *kvm,struct kvm_coalesced_mmio_zone *zone); +int kvm_vm_ioctl_create_coalesced_mmio_buffer (struct kvm *kvm);#else
virt/kvm/kvm_main.c +4 -0
@@ -5169,6 +5169,10 @@ static long kvm_vm_ioctl(struct file *filp, r =kvm_vm_ioctl_unregister_coalesced_mmio (kvm, &zone); break; } +case KVM_CREATE_COALESCED_MMIO_BUFFER : { + r =kvm_vm_ioctl_create_coalesced_mmio_buffer (kvm); +break ; + }#endif case KVM_IRQFD : {struct kvm_irqfd data;
Signature · 1 line
2.34.1
Re: [PATCH 2/6] KVM: Add KVM_CREATE_COALESCED_MMIO_BUFFER ioctl
Quoted 23 lines · Ilias Stamatis
The current MMIO coalescing design has a few drawbacks which limit its usefulness. Currently all coalesced MMIO zones use the same ring buffer. That means that upon a userspace exit we have to handle potentially unrelated MMIO writes synchronously. And a VM-wide lock needs to be taken in the kernel when an MMIO exit occurs.
Additionally, there is no direct way for userspace to be notified about coalesced MMIO writes. If the next MMIO exit to userspace is when the ring buffer has filled then a substantial (and unbounded) amount of time may have passed since the first coalesced MMIO.
Add a
KVM_CREATE_COALESCED_MMIO_BUFFERioctl to KVM. This ioctl simply returns a file descriptor to the caller but does not allocate a ring buffer. Userspace can then pass this fd tommap()to actually allocate a buffer and map it to its address space.Subsequent patches will allow userspace to:
- Associate the fd with a coalescing zone when registering it so that writes to that zone are accumulated in that specific ring buffer rather than the VM-wide one.
- Poll for MMIO writes using this fd.
Quoted 9 lines
Signed-off-by Ilias Stamatis <[email protected]>--- include/linux/kvm_host.h | 1 + include/uapi/linux/kvm.h | 2 + virt/kvm/coalesced_mmio.c | 142 +++++++++++++++++++++++++++++++++++--- virt/kvm/coalesced_mmio.h | 9 +++ virt/kvm/kvm_main.c | 4 ++ 5 files changed, 150 insertions(+), 8 deletions(-)
PATCH 3/6 KVM: Support poll() on coalesced mmio buffer fds
In reply to Ilias StamatisThere is no direct way for userspace to be notified about coalesced MMIO writes when using KVM_REGISTER_COALESCED_MMIO. If the next MMIO exit is when the ring buffer has filled then a substantial (and unbounded) amount of time may have passed since the first coalesced MMIO.
To improve this, make it possible for userspace to use poll() and select() on the fd returned by the KVM_CREATE_COALESCED_MMIO_BUFFER ioctl. This way a userspace VMM could have dedicated threads that deal with writes to specific MMIO zones.
For example, a common use of MMIO, particularly in the realm of network devices, is as a doorbell. A write to a doorbell register will trigger the device to initiate a DMA transfer.
When a network device is emulated by userspace a write to a doorbell register would typically result in an MMIO exit so that userspace can emulate the DMA transfer in a timely manner. No further processing can be done until userspace performs the necessary emulation and re-invokes KVM_RUN. Even if userspace makes use of another thread to emulate the DMA transfer such MMIO exits are disruptive to the vCPU and they may also be quite frequent if, for example, the vCPU is sending a sequence of short packets to the network device.
By supporting poll() on coalesced buffer fds, userspace can have dedicated threads wait for new doorbell writes and avoid the performance hit of userspace exits on the main vCPU threads.
--- virt/kvm/coalesced_mmio.c | 20 ++++++++++++++++++++ virt/kvm/coalesced_mmio.h | 1 + 2 files changed, 21 insertions(+)
virt/kvm/coalesced_mmio.c +20 -0
@@ -16,6 +16,7 @@#include <linux/slab.h> #include <linux/kvm.h> #include <linux/anon_inodes.h> +#include <linux/poll.h> #include "coalesced_mmio.h"
@@ -98,6 +99,10 @@ static int coalesced_mmio_write(struct kvm_vcpu *vcpu,smp_wmb (); ring->last = (insert +1 ) %KVM_COALESCED_MMIO_MAX ;spin_unlock (lock); + +if (dev->buffer_dev) +wake_up_interruptible (&dev->buffer_dev->wait_queue); +return 0 ; }
@@ -224,9 +229,23 @@ static int coalesced_mmio_buffer_release(struct inode *inode, struct file *file)return 0 ; } +static __poll_t coalesced_mmio_buffer_poll (struct file *file,struct poll_table_struct *wait) +{ +struct kvm_coalesced_mmio_buffer_dev *dev = file->private_data; +__poll_t mask =0 ; + +poll_wait (file, &dev->wait_queue, wait); + +if (READ_ONCE (dev->ring->first) !=READ_ONCE (dev->ring->last)) + mask =POLLIN |POLLRDNORM ; + +return mask; +} +static const struct file_operations coalesced_mmio_buffer_ops = { .mmap = coalesced_mmio_buffer_mmap, .release = coalesced_mmio_buffer_release, + .poll = coalesced_mmio_buffer_poll, };int kvm_vm_ioctl_create_coalesced_mmio_buffer (struct kvm *kvm)
@@ -240,6 +259,7 @@ int kvm_vm_ioctl_create_coalesced_mmio_buffer(struct kvm *kvm)return -ENOMEM ; dev->kvm = kvm; +init_waitqueue_head (&dev->wait_queue);spin_lock_init (&dev->ring_lock); ret = anon_inode_getfd("coalesced_mmio_buf", &coalesced_mmio_buffer_ops,
virt/kvm/coalesced_mmio.h +1 -0
@@ -26,6 +26,7 @@ struct kvm_coalesced_mmio_dev {struct kvm_coalesced_mmio_buffer_dev {struct list_head list;struct kvm *kvm; +wait_queue_head_t wait_queue;spinlock_t ring_lock;struct kvm_coalesced_mmio_ring *ring; };
Signature · 1 line
2.34.1
Re: [PATCH 3/6] KVM: Support poll() on coalesced mmio buffer fds
Quoted 27 lines · Ilias Stamatis
There is no direct way for userspace to be notified about coalesced MMIO writes when using
KVM_REGISTER_COALESCED_MMIO. If the next MMIO exit is when the ring buffer has filled then a substantial (and unbounded) amount of time may have passed since the first coalesced MMIO.To improve this, make it possible for userspace to use
poll()andselect()on the fd returned by theKVM_CREATE_COALESCED_MMIO_BUFFERioctl. This way a userspace VMM could have dedicated threads that deal with writes to specific MMIO zones.For example, a common use of MMIO, particularly in the realm of network devices, is as a doorbell. A write to a doorbell register will trigger the device to initiate a DMA transfer.
When a network device is emulated by userspace a write to a doorbell register would typically result in an MMIO exit so that userspace can emulate the DMA transfer in a timely manner. No further processing can be done until userspace performs the necessary emulation and re-invokes
KVM_RUN. Even if userspace makes use of another thread to emulate the DMA transfer such MMIO exits are disruptive to the vCPU and they may also be quite frequent if, for example, the vCPU is sending a sequence of short packets to the network device.By supporting
poll()on coalesced buffer fds, userspace can have dedicated threads wait for new doorbell writes and avoid the performance hit of userspace exits on the main vCPU threads.
Quoted 6 lines
Signed-off-by Ilias Stamatis <[email protected]>--- virt/kvm/coalesced_mmio.c | 20 ++++++++++++++++++++ virt/kvm/coalesced_mmio.h | 1 + 2 files changed, 21 insertions(+)
PATCH 4/6 KVM: Add KVM_(UN)REGISTER_COALESCED_MMIO2 ioctls
In reply to Ilias StamatisAdd 2 new ioctls, KVM_REGISTER_COALESCED_MMIO2 and KVM_UNREGISTER_COALESCED_MMIO2. These do the same thing as their v1 equivalents except an fd returned by KVM_CREATE_COALESCED_MMIO_BUFFER needs to be passed as an argument to them.
The fd representing a ring buffer is associated with an MMIO region registered for coalescing and all writes to that region are accumulated there. This is in contrast to the v1 API where all regions have to share the same buffer. Nevertheless, userspace code can still use the same ring buffer for multiple zones if it wishes to do so.
Userspace can check for the availability of the new API by checking if the KVM_CAP_COALESCED_MMIO2 capability is supported.
--- include/uapi/linux/kvm.h | 16 ++++++++++++++++ virt/kvm/coalesced_mmio.c | 36 ++++++++++++++++++++++++++++++------ virt/kvm/coalesced_mmio.h | 7 ++++--- virt/kvm/kvm_main.c | 36 +++++++++++++++++++++++++++++++++++- 4 files changed, 85 insertions(+), 10 deletions(-)
include/uapi/linux/kvm.h +16 -0
@@ -467,6 +467,16 @@ struct kvm_coalesced_mmio_zone { }; }; +struct kvm_coalesced_mmio_zone2 { +__u64 addr; +__u32 size; +union { +__u32 pad; +__u32 pio; + }; +int buffer_fd; +}; +struct kvm_coalesced_mmio {__u64 phys_addr;__u32 len;
@@ -917,6 +927,7 @@ struct kvm_enable_cap {#define KVM_CAP_MEMORY_ATTRIBUTES 233#define KVM_CAP_GUEST_MEMFD 234#define KVM_CAP_VM_TYPES 235 +#define KVM_CAP_COALESCED_MMIO2 236struct kvm_irq_routing_irqchip {__u32 irqchip;
@@ -1548,6 +1559,11 @@ struct kvm_create_guest_memfd { __u64 reserved[6]; }; +/* Available with KVM_CAP_COALESCED_MMIO2 */ #define KVM_CREATE_COALESCED_MMIO_BUFFER _IO(KVMIO, 0xd5) +#define KVM_REGISTER_COALESCED_MMIO2 \ + _IOW(KVMIO, 0xd6, struct kvm_coalesced_mmio_zone2) +#define KVM_UNREGISTER_COALESCED_MMIO2 \ + _IOW(KVMIO, 0xd7, struct kvm_coalesced_mmio_zone2)#endif /* __LINUX_KVM_H */
virt/kvm/coalesced_mmio.c +30 -6
@@ -277,19 +277,40 @@ int kvm_vm_ioctl_create_coalesced_mmio_buffer(struct kvm *kvm) }int kvm_vm_ioctl_register_coalesced_mmio (struct kvm *kvm, -struct kvm_coalesced_mmio_zone *zone) +struct kvm_coalesced_mmio_zone2 *zone, +bool use_buffer_fd) { -int ret; +int ret =0 ; +struct file *file;struct kvm_coalesced_mmio_dev *dev;struct kvm_coalesced_mmio_buffer_dev *buffer_dev =NULL ;if (zone->pio !=1 && zone->pio !=0 )return -EINVAL ; +if (use_buffer_fd) { + file =fget (zone->buffer_fd); +if (!file) +return -EBADF ; + +if (file->f_op != &coalesced_mmio_buffer_ops) { +fput (file); +return -EINVAL ; + } + + buffer_dev = file->private_data; +if (!buffer_dev->ring) { +fput (file); +return -ENOBUFS ; + } + } + dev =kzalloc (sizeof (struct kvm_coalesced_mmio_dev ),GFP_KERNEL_ACCOUNT ); -if (!dev) -return -ENOMEM ; +if (!dev) { + ret = -ENOMEM ; + goto out_free_file; + }kvm_iodevice_init (&dev->dev, &coalesced_mmio_ops); dev->kvm = kvm;
@@ -305,17 +326,20 @@ int kvm_vm_ioctl_register_coalesced_mmio(struct kvm *kvm,list_add_tail (&dev->list, &kvm->coalesced_zones);mutex_unlock (&kvm->slots_lock); -return 0 ; + goto out_free_file; out_free_dev:mutex_unlock (&kvm->slots_lock);kfree (dev); +out_free_file: +if (use_buffer_fd) +fput (file);return ret; }int kvm_vm_ioctl_unregister_coalesced_mmio (struct kvm *kvm, -struct kvm_coalesced_mmio_zone *zone) +struct kvm_coalesced_mmio_zone2 *zone) {struct kvm_coalesced_mmio_dev *dev, *tmp;int r;
virt/kvm/coalesced_mmio.h +4 -3
@@ -19,7 +19,7 @@ struct kvm_coalesced_mmio_dev {struct list_head list;struct kvm_io_device dev;struct kvm *kvm; -struct kvm_coalesced_mmio_zone zone; +struct kvm_coalesced_mmio_zone2 zone;struct kvm_coalesced_mmio_buffer_dev *buffer_dev; };
@@ -34,9 +34,10 @@ struct kvm_coalesced_mmio_buffer_dev {int kvm_coalesced_mmio_init (struct kvm *kvm);void kvm_coalesced_mmio_free (struct kvm *kvm);int kvm_vm_ioctl_register_coalesced_mmio (struct kvm *kvm, -struct kvm_coalesced_mmio_zone *zone); +struct kvm_coalesced_mmio_zone2 *zone, +bool use_buffer_fd);int kvm_vm_ioctl_unregister_coalesced_mmio (struct kvm *kvm, -struct kvm_coalesced_mmio_zone *zone); +struct kvm_coalesced_mmio_zone2 *zone);int kvm_vm_ioctl_create_coalesced_mmio_buffer (struct kvm *kvm);#else
virt/kvm/kvm_main.c +35 -1
@@ -4815,6 +4815,7 @@ static int kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)#ifdef CONFIG_KVM_MMIO case KVM_CAP_COALESCED_MMIO :return KVM_COALESCED_MMIO_PAGE_OFFSET ; +case KVM_CAP_COALESCED_MMIO2 :case KVM_CAP_COALESCED_PIO :return 1 ;#endif
@@ -5153,15 +5154,48 @@ static long kvm_vm_ioctl(struct file *filp,#ifdef CONFIG_KVM_MMIO case KVM_REGISTER_COALESCED_MMIO : {struct kvm_coalesced_mmio_zone zone; +struct kvm_coalesced_mmio_zone2 zone2; r = -EFAULT ;if (copy_from_user (&zone, argp,sizeof (zone))) goto out; - r =kvm_vm_ioctl_register_coalesced_mmio (kvm, &zone); + + zone2.addr = zone.addr; + zone2.size = zone.size; + zone2.pio = zone.pio; + zone2.buffer_fd =-1 ; + + r =kvm_vm_ioctl_register_coalesced_mmio (kvm, &zone2, false); +break ; + } +case KVM_REGISTER_COALESCED_MMIO2 : { +struct kvm_coalesced_mmio_zone2 zone; + + r = -EFAULT ; +if (copy_from_user (&zone, argp,sizeof (zone))) + goto out; + + r =kvm_vm_ioctl_register_coalesced_mmio (kvm, &zone, true);break ; }case KVM_UNREGISTER_COALESCED_MMIO : {struct kvm_coalesced_mmio_zone zone; +struct kvm_coalesced_mmio_zone2 zone2; + + r = -EFAULT ; +if (copy_from_user (&zone, argp,sizeof (zone))) + goto out; + + zone2.addr = zone.addr; + zone2.size = zone.size; + zone2.pio = zone.pio; + zone2.buffer_fd =-1 ; + + r =kvm_vm_ioctl_unregister_coalesced_mmio (kvm, &zone2); +break ; + } +case KVM_UNREGISTER_COALESCED_MMIO2 : { +struct kvm_coalesced_mmio_zone2 zone; r = -EFAULT ;if (copy_from_user (&zone, argp,sizeof (zone)))
Signature · 1 line
2.34.1
Re: [PATCH 4/6] KVM: Add KVM_(UN)REGISTER_COALESCED_MMIO2 ioctls
Quoted 14 lines · Ilias Stamatis
Add 2 new ioctls,
KVM_REGISTER_COALESCED_MMIO2andKVM_UNREGISTER_COALESCED_MMIO2. These do the same thing as their v1 equivalents except an fd returned byKVM_CREATE_COALESCED_MMIO_BUFFERneeds to be passed as an argument to them.The fd representing a ring buffer is associated with an MMIO region registered for coalescing and all writes to that region are accumulated there. This is in contrast to the v1 API where all regions have to share the same buffer. Nevertheless, userspace code can still use the same ring buffer for multiple zones if it wishes to do so.
Userspace can check for the availability of the new API by checking if the
KVM_CAP_COALESCED_MMIO2capability is supported.
Quoted 8 lines
Signed-off-by Ilias Stamatis <[email protected]>--- include/uapi/linux/kvm.h | 16 ++++++++++++++++ virt/kvm/coalesced_mmio.c | 36 ++++++++++++++++++++++++++++++------ virt/kvm/coalesced_mmio.h | 7 ++++--- virt/kvm/kvm_main.c | 36 +++++++++++++++++++++++++++++++++++- 4 files changed, 85 insertions(+), 10 deletions(-)
kernel test robot
build botRe: [PATCH 4/6] KVM: Add KVM_(UN)REGISTER_COALESCED_MMIO2 ioctls
In reply to Ilias Stamatiskernel test robot noticed the following build errors:
Hi Ilias,
kernel test robot noticed the following build errors:
[auto build test ERROR on kvm/queue]
[also build test ERROR on mst-vhost/linux-next linus/master v6.10-rc7] [cannot apply to kvm/linux-next next-20240712]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Ilias-Stamatis/KVM-Fix-coalesced_mmio_has_room/20240710-222059 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue patch link: https://lore.kernel.org/r/20240710085259.2125131-5-ilstam%40amazon.com patch subject: [PATCH 4/6] KVM: Add KVM_(UN)REGISTER_COALESCED_MMIO2 ioctls config: loongarch-randconfig-001-20240713 (https://download.01.org/0day-ci/archive/20240714/[email protected]/config) compiler: loongarch64-linux-gcc (GCC) 14.1.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240714/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
arch/loongarch/kvm/../../../virt/kvm/coalesced_mmio.c: In function 'kvm_vm_ioctl_register_coalesced_mmio':
>> arch/loongarch/kvm/../../../virt/kvm/coalesced_mmio.c:292:24: error: implicit declaration of function 'fget'; did you mean 'sget'? [-Wimplicit-function-declaration]
292 | file = fget(zone->buffer_fd);
| ^~~~
| sget
>> arch/loongarch/kvm/../../../virt/kvm/coalesced_mmio.c:292:22: error: assignment to 'struct file *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
292 | file = fget(zone->buffer_fd);
| ^
>> arch/loongarch/kvm/../../../virt/kvm/coalesced_mmio.c:297:25: error: implicit declaration of function 'fput'; did you mean 'iput'? [-Wimplicit-function-declaration]
297 | fput(file);
| ^~~~
| iputvim +292 arch/loongarch/kvm/../../../virt/kvm/coalesced_mmio.c
278
279 int kvm_vm_ioctl_register_coalesced_mmio(struct kvm *kvm,
280 struct kvm_coalesced_mmio_zone2 *zone,
281 bool use_buffer_fd)
282 {
283 int ret = 0;
284 struct file *file;
285 struct kvm_coalesced_mmio_dev *dev;
286 struct kvm_coalesced_mmio_buffer_dev *buffer_dev = NULL;
287
288 if (zone->pio != 1 && zone->pio != 0)
289 return -EINVAL;
290
291 if (use_buffer_fd) {
> 292 file = fget(zone->buffer_fd);
293 if (!file)
294 return -EBADF;
295
296 if (file->f_op != &coalesced_mmio_buffer_ops) {
> 297 fput(file); 298 return -EINVAL;
299 }
300
301 buffer_dev = file->private_data;
302 if (!buffer_dev->ring) {
303 fput(file);
304 return -ENOBUFS;
305 }
306 }
307
308 dev = kzalloc(sizeof(struct kvm_coalesced_mmio_dev),
309 GFP_KERNEL_ACCOUNT);
310 if (!dev) {
311 ret = -ENOMEM;
312 goto out_free_file;
313 }
314
315 kvm_iodevice_init(&dev->dev, &coalesced_mmio_ops);
316 dev->kvm = kvm;
317 dev->zone = *zone;
318 dev->buffer_dev = buffer_dev;
319
320 mutex_lock(&kvm->slots_lock);
321 ret = kvm_io_bus_register_dev(kvm,
322 zone->pio ? KVM_PIO_BUS : KVM_MMIO_BUS,
323 zone->addr, zone->size, &dev->dev);
324 if (ret < 0)
325 goto out_free_dev;
326 list_add_tail(&dev->list, &kvm->coalesced_zones);
327 mutex_unlock(&kvm->slots_lock);
328
329 goto out_free_file;
330
331 out_free_dev:
332 mutex_unlock(&kvm->slots_lock);
333 kfree(dev);
334 out_free_file:
335 if (use_buffer_fd)
336 fput(file);
337
338 return ret;
339 }
340
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH 4/6] KVM: Add KVM_(UN)REGISTER_COALESCED_MMIO2 ioctls
In reply to Ilias Stamatiskernel test robot noticed the following build errors:
Hi Ilias,
kernel test robot noticed the following build errors:
[auto build test ERROR on kvm/queue]
[also build test ERROR on mst-vhost/linux-next linus/master v6.10-rc7] [cannot apply to kvm/linux-next next-20240712]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Ilias-Stamatis/KVM-Fix-coalesced_mmio_has_room/20240710-222059 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue patch link: https://lore.kernel.org/r/20240710085259.2125131-5-ilstam%40amazon.com patch subject: [PATCH 4/6] KVM: Add KVM_(UN)REGISTER_COALESCED_MMIO2 ioctls config: powerpc-allmodconfig (https://download.01.org/0day-ci/archive/20240714/[email protected]/config) compiler: powerpc64-linux-gcc (GCC) 14.1.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240714/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
arch/powerpc/kvm/../../../virt/kvm/coalesced_mmio.c: In function 'kvm_vm_ioctl_register_coalesced_mmio':
>> arch/powerpc/kvm/../../../virt/kvm/coalesced_mmio.c:292:24: error: implicit declaration of function 'fget'; did you mean 'sget'? [-Wimplicit-function-declaration]
292 | file = fget(zone->buffer_fd);
| ^~~~
| sget
>> arch/powerpc/kvm/../../../virt/kvm/coalesced_mmio.c:292:22: error: assignment to 'struct file *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
292 | file = fget(zone->buffer_fd);
| ^
>> arch/powerpc/kvm/../../../virt/kvm/coalesced_mmio.c:297:25: error: implicit declaration of function 'fput'; did you mean 'iput'? [-Wimplicit-function-declaration]
297 | fput(file);
| ^~~~
| iputvim +292 arch/powerpc/kvm/../../../virt/kvm/coalesced_mmio.c
278
279 int kvm_vm_ioctl_register_coalesced_mmio(struct kvm *kvm,
280 struct kvm_coalesced_mmio_zone2 *zone,
281 bool use_buffer_fd)
282 {
283 int ret = 0;
284 struct file *file;
285 struct kvm_coalesced_mmio_dev *dev;
286 struct kvm_coalesced_mmio_buffer_dev *buffer_dev = NULL;
287
288 if (zone->pio != 1 && zone->pio != 0)
289 return -EINVAL;
290
291 if (use_buffer_fd) {
> 292 file = fget(zone->buffer_fd);
293 if (!file)
294 return -EBADF;
295
296 if (file->f_op != &coalesced_mmio_buffer_ops) {
> 297 fput(file); 298 return -EINVAL;
299 }
300
301 buffer_dev = file->private_data;
302 if (!buffer_dev->ring) {
303 fput(file);
304 return -ENOBUFS;
305 }
306 }
307
308 dev = kzalloc(sizeof(struct kvm_coalesced_mmio_dev),
309 GFP_KERNEL_ACCOUNT);
310 if (!dev) {
311 ret = -ENOMEM;
312 goto out_free_file;
313 }
314
315 kvm_iodevice_init(&dev->dev, &coalesced_mmio_ops);
316 dev->kvm = kvm;
317 dev->zone = *zone;
318 dev->buffer_dev = buffer_dev;
319
320 mutex_lock(&kvm->slots_lock);
321 ret = kvm_io_bus_register_dev(kvm,
322 zone->pio ? KVM_PIO_BUS : KVM_MMIO_BUS,
323 zone->addr, zone->size, &dev->dev);
324 if (ret < 0)
325 goto out_free_dev;
326 list_add_tail(&dev->list, &kvm->coalesced_zones);
327 mutex_unlock(&kvm->slots_lock);
328
329 goto out_free_file;
330
331 out_free_dev:
332 mutex_unlock(&kvm->slots_lock);
333 kfree(dev);
334 out_free_file:
335 if (use_buffer_fd)
336 fput(file);
337
338 return ret;
339 }
340
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
PATCH 5/6 KVM: Documentation: Document v2 of coalesced MMIO API
In reply to Ilias StamatisDocument the KVM_CREATE_COALESCED_MMIO_BUFFER andKVM_REGISTER_COALESCED_MMIO2 ioctls.
--- Documentation/virt/kvm/api.rst | 91 ++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+)
Documentation/virt/kvm/api.rst +91 -0
@@ -4913,6 +4913,8 @@ For the definition of struct kvm_nested_state, see KVM_GET_NESTED_STATE. :Parameters: struct kvm_coalesced_mmio_zone :Returns: 0 on success, < 0 on error +KVM_(UN)REGISTER_COALESCED_MMIO2 can be used instead if available. + Coalesced I/O is a performance optimization that defers hardware register write emulation so that userspace exits are avoided. It is typically used to reduce the overhead of emulating frequently accessed
@@ -6352,6 +6354,95 @@ a single guest_memfd file, but the bound ranges must not overlap). See KVM_SET_USER_MEMORY_REGION2 for additional details. +4.143 KVM_CREATE_COALESCED_MMIO_BUFFER +------------------------------------- + +:Capability: KVM_CAP_COALESCED_MMIO2 +:Architectures: all +:Type: vm ioctl +:Parameters: none +:Returns: An fd on success, < 0 on error + +Returns an fd, but does not allocate a buffer. Also see +KVM_REGISTER_COALESCED_MMIO2. + +The fd must be first passed to mmap() to allocate a page to be used as a ring +buffer that is shared between kernel and userspace. The page must be +interpreted as a struct kvm_coalesced_mmio_ring. + +:: + + struct kvm_coalesced_mmio_ring { + __u32 first, last; + struct kvm_coalesced_mmio coalesced_mmio[]; + }; + +The kernel will increment the last index and userspace is expected to do the +same with the first index after consuming entries. The upper bound of the +coalesced_mmio array is defined as KVM_COALESCED_MMIO_MAX. + +:: + + struct kvm_coalesced_mmio { + __u64 phys_addr; + __u32 len; + union { + __u32 pad; + __u32 pio; + }; + __u8 data[8]; + }; + +After allocating a buffer with mmap(), the fd must be passed as an argument to +KVM_REGISTER_COALESCED_MMIO2 to associate an I/O region to which writes are +coalesced with the ring buffer. Multiple I/O regions can be associated with the +same ring buffer. + +poll() is also supported on the fd so that userspace can be notified of I/O +writes without having to wait until the next exit to userspace. + +4.144 KVM_(UN)REGISTER_COALESCED_MMIO2 +------------------------------------- + +:Capability: KVM_CAP_COALESCED_MMIO2 +:Architectures: all +:Type: vm ioctl +:Parameters: struct kvm_coalesced_mmio_zone2 +:Returns: 0 on success, < 0 on error + +Coalesced I/O is a performance optimization that defers hardware register write +emulation so that userspace exits are avoided. It is typically used to reduce +the overhead of emulating frequently accessed hardware registers. + +When a hardware register is configured for coalesced I/O, write accesses do not +exit to userspace and their value is recorded in a ring buffer that is shared +between kernel and userspace. + +:: + + struct kvm_coalesced_mmio_zone2 { + __u64 addr; + __u32 size; + union { + __u32 pad; + __u32 pio; + }; + int buffer_fd; + }; + +KVM_CREATE_COALESCED_MMIO_BUFFER must be used to allocate a buffer fd which +must be first mmaped before passed to KVM_REGISTER_COALESCED_MMIO2, otherwise +the ioctl will fail. + +Coalesced I/O is used if one or more write accesses to a hardware register can +be deferred until a read or a write to another hardware register on the same +device. This last access will cause a vmexit and userspace will process +accesses from the ring buffer before emulating it. That will avoid exiting to +userspace on repeated writes. + +Alternatively, userspace can call poll() on the buffer fd if it wishes to be +notified of new I/O writes that way. + 5. The kvm_run structure ========================
Signature · 1 line
2.34.1
Re: [PATCH 5/6] KVM: Documentation: Document v2 of coalesced MMIO API
Quoted 8 lines · Ilias Stamatis
Document the
KVM_CREATE_COALESCED_MMIO_BUFFERandKVM_REGISTER_COALESCED_MMIO2ioctls.Signed-off-by Ilias Stamatis <[email protected]>--- Documentation/virt/kvm/api.rst | 91 ++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+)
PATCH 6/6 KVM: selftests: Add coalesced_mmio_test
In reply to Ilias StamatisTest the KVM_CREATE_COALESCED_MMIO_BUFFER, KVM_REGISTER_COALESCED_MMIO2 and KVM_UNREGISTER_COALESCED_MMIO2 ioctls.
--- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/coalesced_mmio_test.c | 310 ++++++++++++++++++ 2 files changed, 311 insertions(+) create mode 100644 tools/testing/selftests/kvm/coalesced_mmio_test.c
tools/testing/selftests/kvm/Makefile +1 -0
@@ -145,6 +145,7 @@ TEST_GEN_PROGS_x86_64 += set_memory_region_test TEST_GEN_PROGS_x86_64 += steal_time TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test TEST_GEN_PROGS_x86_64 += system_counter_offset_test +TEST_GEN_PROGS_x86_64 += coalesced_mmio_test # Compiled outputs used by test targets TEST_GEN_PROGS_EXTENDED_x86_64 += x86_64/nx_huge_pages_test
tools/testing/selftests/kvm/coalesced_mmio_test.c new file +310 -0
310 lines not on this page · raw message
Signature · 1 line
2.34.1