vfio-pci/zdev: require KVM to be built-in
Subsystems (6)
PATCH 1/1 vfio-pci/zdev: require KVM to be built-in
Fix build errors when CONFIG_KVM=m:
s390-linux-ld: drivers/vfio/pci/vfio_pci_zdev.o: in function `vfio_pci_zdev_open_device':
vfio_pci_zdev.c:(.text+0x242): undefined reference to `kvm_s390_pci_register_kvm'
s390-linux-ld: drivers/vfio/pci/vfio_pci_zdev.o: in function `vfio_pci_zdev_close_device':
vfio_pci_zdev.c:(.text+0x296): undefined reference to `kvm_s390_pci_unregister_kvm'
Having a bool Kconfig symbol depend on a tristate symbol can often lead to problems like this.
--- drivers/vfio/pci/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/vfio/pci/Kconfig +1 -1
@@ -46,7 +46,7 @@ endif config VFIO_PCI_ZDEV_KVM bool "VFIO PCI extensions for s390x KVM passthrough" - depends on S390 && KVM + depends on S390 && KVM=y default y help Support s390x-specific extensions to enable support for enhancements
Thank you Randy for this good catch.
However forcing KVM to be include statically in the kernel when using VFIO_PCI extensions is not a good solution for us I think.
I suggest we better do something like:
----
arch/s390/include/asm/kvm_host.h +1 -1
@@ -1038,7 +1038,7 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}#define __KVM_HAVE_ARCH_VM_FREEvoid kvm_arch_free_vm (struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM +#if defined(CONFIG_VFIO_PCI_ZDEV_KVM ) || defined(CONFIG_VFIO_PCI_ZDEV_KVM_MODULE )int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm);void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev);#else
drivers/vfio/pci/Kconfig +2 -2
@@ -45,9 +45,9 @@ config VFIO_PCI_IGD endif config VFIO_PCI_ZDEV_KVM - bool "VFIO PCI extensions for s390x KVM passthrough" + def_tristate y + prompt "VFIO PCI extensions for s390x KVM passthrough" depends on S390 && KVM - default y help Support s390x-specific extensions to enable support for
enhancements
to KVM passthrough capabilities, such as interpretive
execution of
----
What do you think? It seems to me it solves the problem, what do you think?
Regards,
Pierre
Quoted 35 lines · Randy Dunlap
Fix build errors when
CONFIG_KVM=m:s390-linux-ld:
drivers/vfio/pci/vfio_pci_zdev.o: in function `vfio_pci_zdev_open_device':
vfio_pci_zdev.c:(.text+0x242): undefined reference to `kvm_s390_pci_register_kvm'
s390-linux-ld:drivers/vfio/pci/vfio_pci_zdev.o: in function `vfio_pci_zdev_close_device':
vfio_pci_zdev.c:(.text+0x296): undefined reference to `kvm_s390_pci_unregister_kvm'Having a bool
Kconfigsymbol depend on a tristate symbol can often lead to problems like this.Fixes 8061d1c31f1a ("vfio-pci/zdev: add open/close device hooks")Signed-off-by Randy Dunlap <[email protected]>Reported-by kernel test robot <[email protected]>Cc Matthew Rosato <[email protected]>Cc Christian Borntraeger <[email protected]>Cc Pierre Morel <[email protected]>Cc Eric Farman <[email protected]>--- drivers/vfio/pci/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/vfio/pci/Kconfig +1
-1@@ -46,7 +46,7 @@ endif config VFIO_PCI_ZDEV_KVM bool "VFIO PCI extensions for s390x KVM passthrough" - depends on S390 && KVM + depends on S390 && KVM=y default y help Support s390x-specific extensions to enable support for enhancements
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Hi--
Quoted 12 lines · Pierre Morel
Thank you Randy for this good catch.
However forcing KVM to be include statically in the kernel when usingVFIO_PCIextensions is not a good solution for us I think.I suggest we better do something like:
----
arch/s390/include/asm/kvm_host.h +0
-0
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1041
Quoted 5 lines
arch/s390/include/asm/kvm_host.h +1
-1@@ -1038,7 +1038,7 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} #define __KVM_HAVE_ARCH_VM_FREE void kvm_arch_free_vm(struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM +#if defined(CONFIG_VFIO_PCI_ZDEV_KVM ) || defined(CONFIG_VFIO_PCI_ZDEV_KVM_MODULE )
This all looks good except for the line above.
It should be:
#if IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM)
Thanks.
Quoted 26 lines
arch/s390/include/asm/kvm_host.h +0
-0int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm); void kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev); #elsedrivers/vfio/pci/Kconfig +0
-0@@ -45,9 +45,9 @@ config VFIO_PCI_IGDendif
config
VFIO_PCI_ZDEV_KVM
- bool "VFIO PCI extensions for s390x KVM passthrough"
+def_tristatey
+ prompt "VFIO PCI extensions for s390x KVM passthrough" depends on S390 && KVM
- default y
help
Support s390x-specific extensions to enable support for enhancements to KVM passthrough capabilities, such as interpretive execution of----
What do you think? It seems to me it solves the problem, what do you think?
Regards,
Pierre
Signature · 1 line
~Randy
Quoted 24 lines · Randy Dunlap
Hi--
Quoted 12 lines · Pierre Morel
Thank you Randy for this good catch.
However forcing KVM to be include statically in the kernel when usingVFIO_PCIextensions is not a good solution for us I think.I suggest we better do something like:
----
arch/s390/include/asm/kvm_host.h +0
-0arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1041
Quoted 5 lines
arch/s390/include/asm/kvm_host.h +1
-1@@ -1038,7 +1038,7 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} #define __KVM_HAVE_ARCH_VM_FREE void kvm_arch_free_vm(struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM +#if defined(CONFIG_VFIO_PCI_ZDEV_KVM ) || defined(CONFIG_VFIO_PCI_ZDEV_KVM_MODULE )This all looks good except for the line above.
It should be:
Quoted 3 lines
#if
IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM)Thanks.
Yes, better, thanks.
How do we do? Should I repost it with reported-by you or do you want to post it?
Pierre
Quoted 30 lines
Quoted 28 lines
arch/s390/include/asm/kvm_host.h +0
-0int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm); void kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev); #elsedrivers/vfio/pci/Kconfig +2
-3@@ -45,9 +45,9 @@ config VFIO_PCI_IGD endif config VFIO_PCI_ZDEV_KVM - bool "VFIO PCI extensions for s390x KVM passthrough" + def_tristate y + prompt "VFIO PCI extensions for s390x KVM passthrough" depends on S390 && KVM - default y help Support s390x-specific extensions to enable support for enhancements to KVM passthrough capabilities, such as interpretive execution of ----What do you think? It seems to me it solves the problem, what do you think?
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Randy,
I need to provide the correction patch rapidly.
Without answer I will propose the patch.
Regards,
Pierre
Quoted 70 lines · Pierre Morel
Quoted 26 lines · Randy Dunlap
Hi--
Quoted 12 lines · Pierre Morel
Thank you Randy for this good catch.
However forcing KVM to be include statically in the kernel when usingVFIO_PCIextensions is not a good solution for us I think.I suggest we better do something like:
----
arch/s390/include/asm/kvm_host.h +0
-0arch/s390/include/asm/kvm_host.h static inline void · line 1043
Quoted 7 lines
arch/s390/include/asm/kvm_host.h +1
-1@@ -1038,7 +1038,7 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} #define __KVM_HAVE_ARCH_VM_FREE void kvm_arch_free_vm(struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM +#if defined(CONFIG_VFIO_PCI_ZDEV_KVM ) || defined(CONFIG_VFIO_PCI_ZDEV_KVM_MODULE )This all looks good except for the line above.
It should be:Quoted 3 lines
#if
IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM)Thanks.
Yes, better, thanks.
How do we do? Should I repost it with reported-by you or do you want to post it?Pierre
Quoted 30 lines
Quoted 28 lines
arch/s390/include/asm/kvm_host.h +0
-0int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm);void
kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev); #elsedrivers/vfio/pci/Kconfig +0
-0@@ -45,9 +45,9 @@ config VFIO_PCI_IGDendif
config
VFIO_PCI_ZDEV_KVM
- bool "VFIO PCI extensions for s390x KVM passthrough"
+def_tristatey
+ prompt "VFIO PCI extensions for s390x KVM passthrough" depends on S390 && KVM
- default y
help
Support s390x-specific extensions to enable support for enhancements to KVM passthrough capabilities, such as interpretive execution of----
What do you think? It seems to me it solves the problem, what do you think?
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Quoted 7 lines · Pierre Morel
Randy,
I need to provide the correction patch rapidly.
Without answer I will propose the patch.Regards,
Pierre
Please go ahead with it.
Thanks.
Quoted 71 lines · Pierre Morel
Quoted 68 lines · Randy Dunlap
Quoted 24 lines
Hi--
Quoted 12 lines · Pierre Morel
Thank you Randy for this good catch.
However forcing KVM to be include statically in the kernel when usingVFIO_PCIextensions is not a good solution for us I think.I suggest we better do something like:
----
arch/s390/include/asm/kvm_host.h +0
-0arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1041
Quoted 5 lines
arch/s390/include/asm/kvm_host.h +1
-1@@ -1038,7 +1038,7 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} #define __KVM_HAVE_ARCH_VM_FREE void kvm_arch_free_vm(struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM +#if defined(CONFIG_VFIO_PCI_ZDEV_KVM ) || defined(CONFIG_VFIO_PCI_ZDEV_KVM_MODULE )This all looks good except for the line above.
It should be:Quoted 3 lines
#if
IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM)Thanks.
Yes, better, thanks.
How do we do? Should I repost it with reported-by you or do you want to post it?Pierre
Quoted 30 lines
Quoted 28 lines
arch/s390/include/asm/kvm_host.h +0
-0int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm); void kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev); #elsedrivers/vfio/pci/Kconfig +0
-0@@ -45,9 +45,9 @@ config VFIO_PCI_IGDendif
config
VFIO_PCI_ZDEV_KVM
- bool "VFIO PCI extensions for s390x KVM passthrough"
+def_tristatey
+ prompt "VFIO PCI extensions for s390x KVM passthrough" depends on S390 && KVM
- default y
help
Support s390x-specific extensions to enable support for enhancements to KVM passthrough capabilities, such as interpretive execution of----
What do you think? It seems to me it solves the problem, what do you think?
Regards,
Pierre
Signature · 1 line
~Randy
Matthew Rosato
maintainerQuoted 27 lines · Pierre Morel
Quoted 24 lines · Randy Dunlap
Hi--
Quoted 12 lines · Pierre Morel
Thank you Randy for this good catch.
However forcing KVM to be include statically in the kernel when usingVFIO_PCIextensions is not a good solution for us I think.I suggest we better do something like:
----
arch/s390/include/asm/kvm_host.h +0
-0arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1041
Quoted 5 lines
arch/s390/include/asm/kvm_host.h +1
-1@@ -1038,7 +1038,7 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} #define __KVM_HAVE_ARCH_VM_FREE void kvm_arch_free_vm(struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM +#if defined(CONFIG_VFIO_PCI_ZDEV_KVM ) || defined(CONFIG_VFIO_PCI_ZDEV_KVM_MODULE )This all looks good except for the line above.
It should be:
Quoted 8 lines
Quoted 3 lines
#if
IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM)Thanks.
Yes, better, thanks.
How do we do? Should I repost it with reported-by you or do you want to post it?Pierre
Thanks for looking into this while I was away.
I think the issue is not just CONFIG_KVM=m && CONFIG_VFIO_PCI_ZDEV_KVM=y -- it also requires CONFIG_VFIO_PCI=y && CONFIG_VFIO_PCI_CORE=y. This combination results in building in vfio_pci (which tries to link the calls to kvm_s390_pci_register_kvm and kvm_s390_pci_unregister_kvm which is not built in).
However... this tristate + IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM) check in kvm_host.h will not solve the issue. Rather, due to the #ifdef CONFIG_VFIO_PCI_ZDEV_KVM in include/linux/vfio_pci_core.h, this change will just cause us to never call kvm_s390_pci_register_kvm or kvm_s390_pci_unregister_kvm when CONFIG_VFIO_PCI_ZDEV_KVM=m, effectively treating CONFIG_VFIO_PCI_ZDEV_KVM=m as a 'n' and we don't get the zdev kvm extensions, which I don't think was the intent.
I'm still thinking & am open to other ideas, but one solution to avoiding building in KVM would be to go back to using symbol_get for these 2 interfaces (kvm_s390_pci_register_kvm and kvm_s390_pci_unregister_kvm) as was done in a prior version of this series like virt/kvm/vfio.c does and otherwise leave CONFIG_VFIO_PCI_ZDEV_KVM as-is.
drivers/vfio/pci/vfio_pci_zdev.c +19 -2
@@ -144,6 +144,8 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev,int vfio_pci_zdev_open_device (struct vfio_pci_core_device *vdev) {struct zpci_dev *zdev =to_zpci (vdev->pdev); +int (*fn)(struct zpci_dev *zdev,struct kvm *kvm); +int rc;if (!zdev)return -ENODEV ;
@@ -151,15 +153,30 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev)if (!vdev->vdev.kvm)return 0 ; -return kvm_s390_pci_register_kvm (zdev, vdev->vdev.kvm); + fn =symbol_get (kvm_s390_pci_register_kvm); +if (!fn) +return -EPERM ; + + rc =fn (zdev, vdev->vdev.kvm); + +symbol_put (kvm_s390_pci_register_kvm); + +return rc; }void vfio_pci_zdev_close_device (struct vfio_pci_core_device *vdev) {struct zpci_dev *zdev =to_zpci (vdev->pdev); +void (*fn)(struct zpci_dev *zdev);if (!zdev || !vdev->vdev.kvm)return ; -kvm_s390_pci_unregister_kvm (zdev); + fn =symbol_get (kvm_s390_pci_unregister_kvm); +if (!fn) +return ; + +fn (zdev); + +symbol_put (kvm_s390_pci_unregister_kvm); }
Quoted 83 lines · Matthew Rosato
Quoted 27 lines · Pierre Morel
Quoted 24 lines · Randy Dunlap
Hi--
Quoted 12 lines · Pierre Morel
Thank you Randy for this good catch.
However forcing KVM to be include statically in the kernel when usingVFIO_PCIextensions is not a good solution for us I think.I suggest we better do something like:
----
arch/s390/include/asm/kvm_host.h +0
-0arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1041
Quoted 5 lines
arch/s390/include/asm/kvm_host.h +1
-1@@ -1038,7 +1038,7 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} #define __KVM_HAVE_ARCH_VM_FREE void kvm_arch_free_vm(struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM +#if defined(CONFIG_VFIO_PCI_ZDEV_KVM ) || defined(CONFIG_VFIO_PCI_ZDEV_KVM_MODULE )This all looks good except for the line above.
It should be:Quoted 8 lines
Quoted 3 lines
#if
IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM)Thanks.
Yes, better, thanks.
How do we do? Should I repost it with reported-by you or do you want to post it?Pierre
Thanks for looking into this while I was away.
I think the issue is not just
CONFIG_KVM=m&&CONFIG_VFIO_PCI_ZDEV_KVM=y-- it also requiresCONFIG_VFIO_PCI=y&&CONFIG_VFIO_PCI_CORE=y. This combination results in building invfio_pci(which tries to link the calls tokvm_s390_pci_register_kvmandkvm_s390_pci_unregister_kvmwhich is not built in).However... this tristate +
IS_ENABLED(CONFIG_VFIO_PCI_ZDEV_KVM)check inkvm_host.hwill not solve the issue. Rather, due to the #ifdefCONFIG_VFIO_PCI_ZDEV_KVMininclude/linux/vfio_pci_core.h, this change will just cause us to never callkvm_s390_pci_register_kvmorkvm_s390_pci_unregister_kvmwhenCONFIG_VFIO_PCI_ZDEV_KVM=m, effectively treatingCONFIG_VFIO_PCI_ZDEV_KVM=mas a 'n' and we don't get the zdev kvm extensions, which I don't think was the intent.I'm still thinking & am open to other ideas, but one solution to avoiding building in KVM would be to go back to using
symbol_getfor these 2 interfaces (kvm_s390_pci_register_kvmandkvm_s390_pci_unregister_kvm) as was done in a prior version of this series likevirt/kvm/vfio.cdoes and otherwise leaveCONFIG_VFIO_PCI_ZDEV_KVMas-is.drivers/vfio/pci/vfio_pci_zdev.c +12
-1@@ -144,6 +144,8 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev,int vfio_pci_zdev_open_device (struct vfio_pci_core_device *vdev) {struct zpci_dev *zdev =to_zpci (vdev->pdev); +int (*fn)(struct zpci_dev *zdev,struct kvm *kvm); +int rc;if (!zdev)return -ENODEV ;@@ -151,15 +153,30 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev)if (!vdev->vdev.kvm)return 0 ; -return kvm_s390_pci_register_kvm (zdev, vdev->vdev.kvm); + fn =symbol_get (kvm_s390_pci_register_kvm); +if (!fn) +return -EPERM ; + + rc =fn (zdev, vdev->vdev.kvm); + +symbol_put (kvm_s390_pci_register_kvm); + +return rc; }void vfio_pci_zdev_close_device (struct vfio_pci_core_device *vdev) {struct zpci_dev *zdev =to_zpci (vdev->pdev); +void (*fn)(struct zpci_dev *zdev);if (!zdev || !vdev->vdev.kvm)return ;
drivers/vfio/pci/vfio_pci_zdev.c int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev) · line 182
Quoted 10 lines
drivers/vfio/pci/vfio_pci_zdev.c +7
-1-kvm_s390_pci_unregister_kvm (zdev); + fn =symbol_get (kvm_s390_pci_unregister_kvm); +if (!fn) +return ; + +fn (zdev); + +symbol_put (kvm_s390_pci_unregister_kvm); }
Hello Matt,
In between I came to another solution that seems to satisfy the dependencies. Still need to check that the functionality is still intact :)
I send you the proposition as a reply.
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH 1/1 KVM: s390: pci: VFIO_PCI ZDEV configuration fix
Fixing configuration for VFIO PCI interpretation.
--- arch/s390/include/asm/kvm_host.h | 9 --------- arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/pci.c | 4 ++-- drivers/vfio/pci/Kconfig | 4 ++-- include/linux/vfio_pci_core.h | 2 +- 5 files changed, 6 insertions(+), 15 deletions(-)
arch/s390/include/asm/kvm_host.h +0 -9
@@ -1038,16 +1038,7 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}#define __KVM_HAVE_ARCH_VM_FREEvoid kvm_arch_free_vm (struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm);void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev); -#else -static inline int kvm_s390_pci_register_kvm (struct zpci_dev *dev, -struct kvm *kvm) -{ -return -EPERM ; -} -static inline void kvm_s390_pci_unregister_kvm (struct zpci_dev *dev) {} -#endif #endif
arch/s390/kvm/Makefile +1 -1
@@ -9,6 +9,6 @@ ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o +kvm-y += pci.o -kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o obj-$(CONFIG_KVM) += kvm.o
arch/s390/kvm/pci.c +2 -2
@@ -435,7 +435,7 @@ int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm) {int rc; -if (!zdev) +if (!IS_ENABLED (CONFIG_VFIO_PCI_ZDEV_KVM ) || !zdev)return -EINVAL ;mutex_lock (&zdev->kzdev_lock);
@@ -516,7 +516,7 @@ void kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev) {struct kvm *kvm; -if (!zdev) +if (!IS_ENABLED (CONFIG_VFIO_PCI_ZDEV_KVM ) || !zdev)return ;mutex_lock (&zdev->kzdev_lock);
drivers/vfio/pci/Kconfig +2 -2
@@ -45,9 +45,9 @@ config VFIO_PCI_IGD endif config VFIO_PCI_ZDEV_KVM - bool "VFIO PCI extensions for s390x KVM passthrough" + def_tristate y + prompt "VFIO PCI extensions for s390x KVM passthrough" depends on S390 && KVM - default y help Support s390x-specific extensions to enable support for enhancements to KVM passthrough capabilities, such as interpretive execution of
include/linux/vfio_pci_core.h +1 -1
@@ -205,7 +205,7 @@ static inline int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) }#endif -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM +#if IS_ENABLED (CONFIG_VFIO_PCI_ZDEV_KVM )int vfio_pci_info_zdev_add_caps (struct vfio_pci_core_device *vdev,struct vfio_info_cap *caps);int vfio_pci_zdev_open_device (struct vfio_pci_core_device *vdev);
Signature · 1 line
2.31.1
Matthew Rosato
maintainerRe: [PATCH] KVM: s390: pci: VFIO_PCI ZDEV configuration fix
Quoted 42 lines · Pierre Morel
Fixing configuration for VFIO PCI interpretation.
Reported-by Randy Dunlap <[email protected]>Reported-by kernel test robot <[email protected]>Signed-off-by Pierre Morel <[email protected]>Fixes 09340b2fca007 ("KVM: s390: pci: add routines to start/stop inter..")Fixes c435c54639aa5 ("vfio/pci: introduce CONFIG_VFIO_PCI_ZDEV_KVM..")Cc <[email protected]>--- arch/s390/include/asm/kvm_host.h | 9 --------- arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/pci.c | 4 ++-- drivers/vfio/pci/Kconfig | 4 ++-- include/linux/vfio_pci_core.h | 2 +- 5 files changed, 6 insertions(+), 15 deletions(-)arch/s390/include/asm/kvm_host.h +0
-9@@ -1038,16 +1038,7 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}#define __KVM_HAVE_ARCH_VM_FREEvoid kvm_arch_free_vm (struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm);void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev); -#else -static inline int kvm_s390_pci_register_kvm (struct zpci_dev *dev, -struct kvm *kvm) -{ -return -EPERM ; -} -static inline void kvm_s390_pci_unregister_kvm (struct zpci_dev *dev) {} -#endif #endif arch/s390/kvm/Makefile +0
-0
arch/s390/kvm/Makefile ccflags-y := -Ivirt/kvm -Iarch/s390/kvm · line 13
Quoted 6 lines
arch/s390/kvm/Makefile +1
-1@@ -9,6 +9,6 @@ ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o +kvm-y += pci.o -kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o
You would need to switch this to CONFIG_PCI at least, else we get build errors with CONFIG_PCI=n.
Quoted 44 lines
arch/s390/kvm/Makefile +0
-0obj-$(CONFIG_KVM) += kvm.oarch/s390/kvm/pci.c +2
-2@@ -435,7 +435,7 @@ int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm) {int rc; -if (!zdev) +if (!IS_ENABLED (CONFIG_VFIO_PCI_ZDEV_KVM ) || !zdev)return -EINVAL ;mutex_lock (&zdev->kzdev_lock);@@ -516,7 +516,7 @@ void kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev) {struct kvm *kvm; -if (!zdev) +if (!IS_ENABLED (CONFIG_VFIO_PCI_ZDEV_KVM ) || !zdev)return ;mutex_lock (&zdev->kzdev_lock);drivers/vfio/pci/Kconfig +2
-2@@ -45,9 +45,9 @@ config VFIO_PCI_IGD endif config VFIO_PCI_ZDEV_KVM - bool "VFIO PCI extensions for s390x KVM passthrough" + def_tristate y + prompt "VFIO PCI extensions for s390x KVM passthrough" depends on S390 && KVM - default y help Support s390x-specific extensions to enable support for enhancements to KVM passthrough capabilities, such as interpretive execution ofinclude/linux/vfio_pci_core.h +0
-0
include/linux/vfio_pci_core.h static inline int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) · line 211
Quoted 8 lines
include/linux/vfio_pci_core.h +1
-1@@ -205,7 +205,7 @@ static inline int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) }#endif -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM +#if IS_ENABLED (CONFIG_VFIO_PCI_ZDEV_KVM )int vfio_pci_info_zdev_add_caps (struct vfio_pci_core_device *vdev,struct vfio_info_cap *caps);int vfio_pci_zdev_open_device (struct vfio_pci_core_device *vdev);
This still doesn't seem quite right... I tried some variations:
1)CONFIG_KVM=mCONFIG_VFIO_PCI_CORE=mCONFIG_VFIO_PCI=mCONFIG_VFIO_PCI_ZDEV_KVM=m
compiles, works with a small change:
include/linux/sched/user.h +1 -1
@@ -25,7 +25,7 @@ struct user_struct {#if defined(CONFIG_PERF_EVENTS ) || defined(CONFIG_BPF_SYSCALL ) || \ defined(CONFIG_NET ) || defined(CONFIG_IO_URING ) || \ - defined(CONFIG_VFIO_PCI_ZDEV_KVM ) +IS_ENABLED (CONFIG_VFIO_PCI_ZDEV_KVM )
2)CONFIG_KVM=yCONFIG_VFIO_PCI_CORE=mCONFIG_VFIO_PCI=mCONFIG_VFIO_PCI_ZDEV_KVM=m
compiles, works with above user.h change
3)CONFIG_KVM=yCONFIG_VFIO_PCI_CORE=yCONFIG_VFIO_PCI=yCONFIG_VFIO_PCI_ZDEV_KVM=y
compiles, works with above user.h change
4)CONFIG_KVM=mCONFIG_VFIO_PCI_CORE=yCONFIG_VFIO_PCI=yCONFIG_VFIO_PCI_ZDEV_KVM=m
fails with:
ld: drivers/vfio/pci/vfio_pci_core.o: in function `vfio_pci_core_enable':/usr/src/linux/drivers/vfio/pci/vfio_pci_core.c:320: undefined reference to `vfio_pci_zdev_open_device'
ld: /usr/src/linux/drivers/vfio/pci/vfio_pci_core.c:349: undefined reference to `vfio_pci_zdev_close_device'
ld: drivers/vfio/pci/vfio_pci_core.o: in function `vfio_pci_core_disable':/usr/src/linux/drivers/vfio/pci/vfio_pci_core.c:428: undefined reference to `vfio_pci_zdev_close_device'
ld: drivers/vfio/pci/vfio_pci_core.o: in function `vfio_pci_core_ioctl':/usr/src/linux/drivers/vfio/pci/vfio_pci_core.c:712: undefined reference to `vfio_pci_info_zdev_add_caps'
5)CONFIG_KVM=mCONFIG_VFIO_PCI_CORE=yCONFIG_VFIO_PCI=yCONFIG_VFIO_PCI_ZDEV_KVM=y
This forces CONFIG_VFIO_PCI_ZDEV_KVM to 'm' and fails as above.
Re: [PATCH] KVM: s390: pci: VFIO_PCI ZDEV configuration fix
Quoted 21 lines · Matthew Rosato
Quoted 2 lines · Pierre Morel
Fixing configuration for VFIO PCI interpretation.
4)
CONFIG_KVM=mCONFIG_VFIO_PCI_CORE=yCONFIG_VFIO_PCI=yCONFIG_VFIO_PCI_ZDEV_KVM=mfails with:
ld:
drivers/vfio/pci/vfio_pci_core.o: in function `vfio_pci_core_enable':/usr/src/linux/drivers/vfio/pci/vfio_pci_core.c:320: undefined reference to `vfio_pci_zdev_open_device'
ld:/usr/src/linux/drivers/vfio/pci/vfio_pci_core.c:349: undefined reference to `vfio_pci_zdev_close_device'
ld:drivers/vfio/pci/vfio_pci_core.o: in function `vfio_pci_core_disable':/usr/src/linux/drivers/vfio/pci/vfio_pci_core.c:428: undefined reference to `vfio_pci_zdev_close_device'
ld:drivers/vfio/pci/vfio_pci_core.o: in function `vfio_pci_core_ioctl':/usr/src/linux/drivers/vfio/pci/vfio_pci_core.c:712: undefined reference to `vfio_pci_info_zdev_add_caps'
Quoted 10 lines
5)
CONFIG_KVM=mCONFIG_VFIO_PCI_CORE=yCONFIG_VFIO_PCI=yCONFIG_VFIO_PCI_ZDEV_KVM=yThis forces
CONFIG_VFIO_PCI_ZDEV_KVMto 'm' and fails as above.
yes right
I was really too stupid yesterday
I think you are right and you should go as you proposed or use some pointer hook inside S390 to register with KVM.
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH 1/1 KVM: s390: pci: Hook to access KVM lowlevel from VFIO
In reply to Pierre MorelWe have a cross dependency between KVM and VFIO.
To be able to keep both subsystem modular we add a registering hook inside the S390 core code.
This fixes a build problem when VFIO is built-in and KVM is built as a module or excluded.
--- arch/s390/include/asm/kvm_host.h | 17 ++++++----------- arch/s390/kvm/pci.c | 10 ++++++---- arch/s390/pci/Makefile | 2 ++ arch/s390/pci/pci_kvm_hook.c | 11 +++++++++++ drivers/vfio/pci/vfio_pci_zdev.c | 8 ++++++-- 5 files changed, 31 insertions(+), 17 deletions(-) create mode 100644 arch/s390/pci/pci_kvm_hook.c
arch/s390/include/asm/kvm_host.h +6 -11
@@ -1038,16 +1038,11 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}#define __KVM_HAVE_ARCH_VM_FREEvoid kvm_arch_free_vm (struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm); -void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev); -#else -static inline int kvm_s390_pci_register_kvm (struct zpci_dev *dev, -struct kvm *kvm) -{ -return -EPERM ; -} -static inline void kvm_s390_pci_unregister_kvm (struct zpci_dev *dev) {} -#endif +struct kvm_register_hook { +int (*kvm_register)(void *opaque,struct kvm *kvm); +void (*kvm_unregister)(void *opaque); +}; + +extern struct kvm_register_hook kvm_pci_hook;#endif
arch/s390/kvm/pci.c +6 -4
@@ -431,8 +431,9 @@ static void kvm_s390_pci_dev_release(struct zpci_dev *zdev)* available, enable them and let userspace indicate whether or not they will * be used (specify SHM bit to disable). */ -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm) +static int kvm_s390_pci_register_kvm (void *opaque,struct kvm *kvm) { +struct zpci_dev *zdev = opaque;int rc;if (!zdev)
@@ -510,10 +511,10 @@ int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm)kvm_put_kvm (kvm);return rc; } -EXPORT_SYMBOL_GPL (kvm_s390_pci_register_kvm); -void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev) +static void kvm_s390_pci_unregister_kvm (void *opaque) { +struct zpci_dev *zdev = opaque;struct kvm *kvm;if (!zdev)
@@ -566,7 +567,6 @@ void kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev)kvm_put_kvm (kvm); } -EXPORT_SYMBOL_GPL (kvm_s390_pci_unregister_kvm);void kvm_s390_pci_init_list (struct kvm *kvm) {
@@ -678,6 +678,8 @@ int kvm_s390_pci_init(void)spin_lock_init (&aift->gait_lock);mutex_init (&aift->aift_lock); + kvm_pci_hook.kvm_register = kvm_s390_pci_register_kvm; + kvm_pci_hook.kvm_unregister = kvm_s390_pci_unregister_kvm;return 0 ; }
arch/s390/pci/Makefile +2 -0
@@ -7,3 +7,5 @@ obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_dma.o pci_clp.o pci_sysfs.o \ pci_event.o pci_debug.o pci_insn.o pci_mmio.o \ pci_bus.o obj-$(CONFIG_PCI_IOV) += pci_iov.o + +obj-y += pci_kvm_hook.o
arch/s390/pci/pci_kvm_hook.c new file +11 -0
@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* VFIO ZPCI devices support +* +* Copyright (C) IBM Corp. 2022. All rights reserved. +* Author(s): Pierre Morel <[email protected]> +*/ +#include <linux/kvm_host.h> + +struct kvm_register_hook kvm_pci_hook; +EXPORT_SYMBOL_GPL (kvm_pci_hook);
drivers/vfio/pci/vfio_pci_zdev.c +6 -2
@@ -151,7 +151,10 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev)if (!vdev->vdev.kvm)return 0 ; -return kvm_s390_pci_register_kvm (zdev, vdev->vdev.kvm); +if (kvm_pci_hook.kvm_register) +return kvm_pci_hook.kvm_register (zdev, vdev->vdev.kvm); + +return -ENOENT ; }void vfio_pci_zdev_close_device (struct vfio_pci_core_device *vdev)
@@ -161,5 +164,6 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev)if (!zdev || !vdev->vdev.kvm)return ; -kvm_s390_pci_unregister_kvm (zdev); +if (kvm_pci_hook.kvm_unregister) +return kvm_pci_hook.kvm_unregister (zdev); }
Signature · 1 line
2.31.1
Matthew Rosato
maintainerRe: [PATCH] KVM: s390: pci: Hook to access KVM lowlevel from VFIO
Quoted 1 line · Pierre Morel
We have a cross dependency between KVM and VFIO.
maybe add something like 'when using s390 vfio_pci_zdev extensions for PCI passthrough'
Quoted 5 lines
To be able to keep both subsystem modular we add a registering hook inside the S390 core code.
This fixes a build problem when VFIO is built-in and KVM is built as a module or excluded.
s/or excluded//
There's no problem when KVM is excluded, that forces CONFIG_VFIO_PCI_ZDEV_KVM=n because of the 'depends on S390 && KVM'.
Quoted 25 lines
Reported-by Randy Dunlap <[email protected]>Reported-by kernel test robot <[email protected]>Signed-off-by Pierre Morel <[email protected]>Fixes 09340b2fca007 ("KVM: s390: pci: add routines to start/stop inter..")Cc <[email protected]>--- arch/s390/include/asm/kvm_host.h | 17 ++++++----------- arch/s390/kvm/pci.c | 10 ++++++---- arch/s390/pci/Makefile | 2 ++ arch/s390/pci/pci_kvm_hook.c | 11 +++++++++++ drivers/vfio/pci/vfio_pci_zdev.c | 8 ++++++-- 5 files changed, 31 insertions(+), 17 deletions(-) create mode 100644 arch/s390/pci/pci_kvm_hook.carch/s390/include/asm/kvm_host.h +0
-2@@ -1038,16 +1038,11 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}#define __KVM_HAVE_ARCH_VM_FREEvoid kvm_arch_free_vm (struct kvm *kvm); -#ifdef CONFIG_VFIO_PCI_ZDEV_KVM -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm);
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1041
Quoted 10 lines
arch/s390/include/asm/kvm_host.h +1
-9-void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev); -#else -static inline int kvm_s390_pci_register_kvm (struct zpci_dev *dev, -struct kvm *kvm) -{ -return -EPERM ; -} -static inline void kvm_s390_pci_unregister_kvm (struct zpci_dev *dev) {} -#endif +struct kvm_register_hook {
Nit: zpci_kvm_register_hook ? Just to make it clear it's for zpci.
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1046
Quoted 5 lines
arch/s390/include/asm/kvm_host.h +5
-0+int (*kvm_register)(void *opaque,struct kvm *kvm); +void (*kvm_unregister)(void *opaque); +}; + +extern struct kvm_register_hook kvm_pci_hook;
Nit: kvm_zpci_hook ?
Quoted 52 lines
arch/s390/include/asm/kvm_host.h +0
-0#endif arch/s390/kvm/pci.c +6
-4@@ -431,8 +431,9 @@ static void kvm_s390_pci_dev_release(struct zpci_dev *zdev)* available, enable them and let userspace indicate whether or not they will * be used (specify SHM bit to disable). */ -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm) +static int kvm_s390_pci_register_kvm (void *opaque,struct kvm *kvm) { +struct zpci_dev *zdev = opaque;int rc;if (!zdev)@@ -510,10 +511,10 @@ int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm)kvm_put_kvm (kvm);return rc; } -EXPORT_SYMBOL_GPL (kvm_s390_pci_register_kvm); -void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev) +static void kvm_s390_pci_unregister_kvm (void *opaque) { +struct zpci_dev *zdev = opaque;struct kvm *kvm;if (!zdev)@@ -566,7 +567,6 @@ void kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev)kvm_put_kvm (kvm); } -EXPORT_SYMBOL_GPL (kvm_s390_pci_unregister_kvm);void kvm_s390_pci_init_list (struct kvm *kvm) {@@ -678,6 +678,8 @@ int kvm_s390_pci_init(void)spin_lock_init (&aift->gait_lock);mutex_init (&aift->aift_lock); + kvm_pci_hook.kvm_register = kvm_s390_pci_register_kvm; + kvm_pci_hook.kvm_unregister = kvm_s390_pci_unregister_kvm;return 0 ; }arch/s390/pci/Makefile +0
-0
arch/s390/pci/Makefile obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_dma.o pci_clp.o pci_sysfs.o \ · line 11
Quoted 5 lines
arch/s390/pci/Makefile +2
-0@@ -7,3 +7,5 @@ obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_dma.o pci_clp.o pci_sysfs.o \ pci_event.o pci_debug.o pci_insn.o pci_mmio.o \ pci_bus.o obj-$(CONFIG_PCI_IOV) += pci_iov.o + +obj-y += pci_kvm_hook.o
I guess it doesn't harm anything to add this unconditionally, but I think it would also be OK to just include this in the CONFIG_PCI list - vfio_pci_zdev and arch/s390/kvm/pci all rely on CONFIG_PCI via CONFIG_VFIO_PCI_ZDEV_KVM which implies PCI via VFIO_PCI.
Quoted 7 lines
arch/s390/pci/pci_kvm_hook.c new file +1
-0@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only
arch/s390/pci/pci_kvm_hook.c line 11
Quoted 10 lines
arch/s390/pci/pci_kvm_hook.c new file +10
-0+/* +* VFIO ZPCI devices support +* +* Copyright (C) IBM Corp. 2022. All rights reserved. +* Author(s): Pierre Morel <[email protected]> +*/ +#include <linux/kvm_host.h> + +struct kvm_register_hook kvm_pci_hook; +EXPORT_SYMBOL_GPL (kvm_pci_hook);
Following the comments above, zpci_kvm_register_hook, kvm_zpci_hook ?
I'm not sure if this really needs to be in a separate file or if it could just go into arch/s390/pci.c with the zpci_aipb -- If going the route of a separate file, up to Niklas whether he wants this under the S390 PCI maintainership or added to the list for s390 vfio-pci like arch/kvm/pci* and vfio_pci_zdev.
Quoted 17 lines
drivers/vfio/pci/vfio_pci_zdev.c +4
-1@@ -151,7 +151,10 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev)if (!vdev->vdev.kvm)return 0 ; -return kvm_s390_pci_register_kvm (zdev, vdev->vdev.kvm); +if (kvm_pci_hook.kvm_register) +return kvm_pci_hook.kvm_register (zdev, vdev->vdev.kvm); + +return -ENOENT ; }void vfio_pci_zdev_close_device (struct vfio_pci_core_device *vdev)
drivers/vfio/pci/vfio_pci_zdev.c void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev) · line 168
Quoted 6 lines
drivers/vfio/pci/vfio_pci_zdev.c +2
-1@@ -161,5 +164,6 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev)if (!zdev || !vdev->vdev.kvm)return ; -kvm_s390_pci_unregister_kvm (zdev); +if (kvm_pci_hook.kvm_unregister) +return kvm_pci_hook.kvm_unregister (zdev);
No need for the return here, this is a void function calling a void function.
Overall, this looks good to me and survives a series of compile and device passthrough tests on my end, just a matter of a few of these minor comments above. Thanks for tackling this Pierre!
Re: [PATCH] KVM: s390: pci: Hook to access KVM lowlevel from VFIO
Quoted 8 lines · Matthew Rosato
Quoted 1 line · Pierre Morel
We have a cross dependency between KVM and VFIO.
maybe add something like 'when using s390
vfio_pci_zdevextensions for PCI passthrough'Quoted 3 lines
To be able to keep both subsystem modular we add a registering hook inside the S390 core code.
Quoted 6 lines
Quoted 2 lines
This fixes a build problem when VFIO is built-in and KVM is built as a module or excluded.
s/or excluded//
There's no problem when KVM is excluded, that forces
CONFIG_VFIO_PCI_ZDEV_KVM=nbecause of the 'depends on S390 && KVM'.
OK
Quoted 24 lines
Quoted 24 lines
Reported-by Randy Dunlap <[email protected]>Reported-by kernel test robot <[email protected]>Signed-off-by Pierre Morel <[email protected]>Fixes 09340b2fca007 ("KVM: s390: pci: add routines to start/stop inter..")Cc <[email protected]>--- arch/s390/include/asm/kvm_host.h | 17 ++++++----------- arch/s390/kvm/pci.c | 10 ++++++---- arch/s390/pci/Makefile | 2 ++ arch/s390/pci/pci_kvm_hook.c | 11 +++++++++++ drivers/vfio/pci/vfio_pci_zdev.c | 8 ++++++-- 5 files changed, 31 insertions(+), 17 deletions(-)create mode 100644
arch/s390/pci/pci_kvm_hook.carch/s390/include/asm/kvm_host.h +0
-0@@ -1038,16 +1038,11 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}#define __KVM_HAVE_ARCH_VM_FREEvoid kvm_arch_free_vm (struct kvm *kvm);
Quoted 14 lines
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1041
Quoted 12 lines
arch/s390/include/asm/kvm_host.h +1
-11-#ifdef CONFIG_VFIO_PCI_ZDEV_KVM -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm); -void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev); -#else -static inline int kvm_s390_pci_register_kvm (struct zpci_dev *dev, -struct kvm *kvm) -{ -return -EPERM ; -} -static inline void kvm_s390_pci_unregister_kvm (struct zpci_dev *dev) {} -#endif +struct kvm_register_hook {Nit:
zpci_kvm_register_hook? Just to make it clear it's for zpci.
OK
Quoted 8 lines
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1046
Quoted 6 lines
arch/s390/include/asm/kvm_host.h +5
-0+int (*kvm_register)(void *opaque,struct kvm *kvm); +void (*kvm_unregister)(void *opaque); +}; + +extern struct kvm_register_hook kvm_pci_hook;Nit:
kvm_zpci_hook?
OK too,
Quoted 46 lines
Quoted 46 lines
arch/s390/include/asm/kvm_host.h +0
-0#endif arch/s390/kvm/pci.c +6
-4@@ -431,8 +431,9 @@ static void kvm_s390_pci_dev_release(struct zpci_dev *zdev)* available, enable them and let userspace indicate whether or not they will * be used (specify SHM bit to disable). */ -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm) +static int kvm_s390_pci_register_kvm (void *opaque,struct kvm *kvm) { +struct zpci_dev *zdev = opaque;int rc;if (!zdev)@@ -510,10 +511,10 @@ int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm)kvm_put_kvm (kvm);return rc; } -EXPORT_SYMBOL_GPL (kvm_s390_pci_register_kvm); -void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev) +static void kvm_s390_pci_unregister_kvm (void *opaque) { +struct zpci_dev *zdev = opaque;struct kvm *kvm;if (!zdev)@@ -566,7 +567,6 @@ void kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev)kvm_put_kvm (kvm); } -EXPORT_SYMBOL_GPL (kvm_s390_pci_unregister_kvm);void kvm_s390_pci_init_list (struct kvm *kvm) {@@ -678,6 +678,8 @@ int kvm_s390_pci_init(void)spin_lock_init (&aift->gait_lock);mutex_init (&aift->aift_lock); + kvm_pci_hook.kvm_register = kvm_s390_pci_register_kvm; + kvm_pci_hook.kvm_unregister = kvm_s390_pci_unregister_kvm;
Quoted 14 lines
arch/s390/pci/Makefile obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_dma.o pci_clp.o pci_sysfs.o \ · line 11
Quoted 12 lines
arch/s390/kvm/pci.c +0
-0return 0 ; }arch/s390/pci/Makefile +2
-0@@ -7,3 +7,5 @@ obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_dma.o pci_clp.o pci_sysfs.o \ pci_event.o pci_debug.o pci_insn.o pci_mmio.o \ pci_bus.o obj-$(CONFIG_PCI_IOV) += pci_iov.o + +obj-y += pci_kvm_hook.oI guess it doesn't harm anything to add this unconditionally, but I think it would also be OK to just include this in the
CONFIG_PCIlist -vfio_pci_zdevandarch/s390/kvm/pciall rely onCONFIG_PCIviaCONFIG_VFIO_PCI_ZDEV_KVMwhich implies PCI viaVFIO_PCI.
Right,CONFIG_PCI is a bool so we can put the hook in arch/s390/pci/pci.c and use a defined(CONFIG_PCI) to protect the initialization inside KVM.
Quoted 12 lines
Quoted 12 lines
arch/s390/pci/pci_kvm_hook.c new file +5
-0@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* VFIO ZPCI devices support +* +* Copyright (C) IBM Corp. 2022. All rights reserved.
Quoted 8 lines
arch/s390/pci/pci_kvm_hook.c line 11
Quoted 6 lines
arch/s390/pci/pci_kvm_hook.c new file +6
-0+* Author(s): Pierre Morel <[email protected]> +*/ +#include <linux/kvm_host.h> + +struct kvm_register_hook kvm_pci_hook; +EXPORT_SYMBOL_GPL (kvm_pci_hook);Following the comments above,
zpci_kvm_register_hook,kvm_zpci_hook?
OK
Quoted 2 lines
I'm not sure if this really needs to be in a separate file or if it could just go into
arch/s390/pci.cwith thezpci_aipb-- If going the route of a separate file, up to Niklas whether he wants this under the S390 PCI maintainership or added to the list for s390 vfio-pci likearch/kvm/pci*andvfio_pci_zdev.
agreed no need for a separate file, it is much better.
Quoted 21 lines
Quoted 21 lines
drivers/vfio/pci/vfio_pci_zdev.c +4
-1@@ -151,7 +151,10 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev)if (!vdev->vdev.kvm)return 0 ; -return kvm_s390_pci_register_kvm (zdev, vdev->vdev.kvm); +if (kvm_pci_hook.kvm_register) +return kvm_pci_hook.kvm_register (zdev, vdev->vdev.kvm); + +return -ENOENT ; }void vfio_pci_zdev_close_device (struct vfio_pci_core_device *vdev)@@ -161,5 +164,6 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev)if (!zdev || !vdev->vdev.kvm)return ;
Quoted 5 lines
drivers/vfio/pci/vfio_pci_zdev.c void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev) · line 168
Quoted 3 lines
drivers/vfio/pci/vfio_pci_zdev.c +2
-1-kvm_s390_pci_unregister_kvm (zdev); +if (kvm_pci_hook.kvm_unregister) +return kvm_pci_hook.kvm_unregister (zdev);No need for the return here, this is a void function calling a void function.
right.
Quoted 4 lines
Overall, this looks good to me and survives a series of compile and device passthrough tests on my end, just a matter of a few of these minor comments above. Thanks for tackling this Pierre!
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Niklas Schnelle
maintainerRe: [PATCH] KVM: s390: pci: Hook to access KVM lowlevel from VFIO
In reply to Matthew RosatoQuoted 29 lines · Matthew Rosato
Quoted 1 line · Pierre Morel
We have a cross dependency between KVM and VFIO.
maybe add something like 'when using s390
vfio_pci_zdevextensions for PCI passthrough'Quoted 5 lines
To be able to keep both subsystem modular we add a registering hook inside the S390 core code.
This fixes a build problem when VFIO is built-in and KVM is built as a module or excluded.
s/or excluded//
There's no problem when KVM is excluded, that forces
CONFIG_VFIO_PCI_ZDEV_KVM=nbecause of the 'depends on S390 && KVM'.Quoted 14 lines
Reported-by Randy Dunlap <[email protected]>Reported-by kernel test robot <[email protected]>Signed-off-by Pierre Morel <[email protected]>Fixes 09340b2fca007 ("KVM: s390: pci: add routines to start/stop inter..")Cc <[email protected]>--- arch/s390/include/asm/kvm_host.h | 17 ++++++----------- arch/s390/kvm/pci.c | 10 ++++++---- arch/s390/pci/Makefile | 2 ++ arch/s390/pci/pci_kvm_hook.c | 11 +++++++++++ drivers/vfio/pci/vfio_pci_zdev.c | 8 ++++++-- 5 files changed, 31 insertions(+), 17 deletions(-) create mode 100644 arch/s390/pci/pci_kvm_hook.c
Quoted 4 lines
arch/s390/include/asm/kvm_host.h
Quoted 4 lines
arch/s390/include/asm/kvm_host.h +0
-0
I added Janosch as second S390 KVM maintainer in case he wants to chime in.
Quoted 4 lines
Quoted 4 lines
arch/s390/include/asm/kvm_host.h +0
-0@@ -1038,16 +1038,11 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}#define __KVM_HAVE_ARCH_VM_FREEvoid kvm_arch_free_vm (struct kvm *kvm);
Quoted 14 lines
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1041
Quoted 12 lines
arch/s390/include/asm/kvm_host.h +1
-11-#ifdef CONFIG_VFIO_PCI_ZDEV_KVM -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm); -void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev); -#else -static inline int kvm_s390_pci_register_kvm (struct zpci_dev *dev, -struct kvm *kvm) -{ -return -EPERM ; -} -static inline void kvm_s390_pci_unregister_kvm (struct zpci_dev *dev) {} -#endif +struct kvm_register_hook {Nit:
zpci_kvm_register_hook? Just to make it clear it's for zpci.
Hmm, I guess one could re-use the same struct for another such KVM dependency but I lean towards the same thinking as Matt, for now this is for zpci so stay specific we can always generalize later.
Nit: For me hook and register together sound a bit redudant, maybe "zpci_kvm_register"? Also question for Matt as a native speaker, should it rather be "registration" when used as a noun here?
Quoted 3 lines
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1043
Quoted 3 lines
arch/s390/include/asm/kvm_host.h +2
-0+int (*kvm_register)(void *opaque,struct kvm *kvm); +void (*kvm_unregister)(void *opaque);
I do wonder if this needs to be opague "struct zpci_dev" should be defined even if CONFIG_PCI is unset.
Quoted 5 lines
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1046
Quoted 3 lines
arch/s390/include/asm/kvm_host.h +3
-0+}; + +extern struct kvm_register_hook kvm_pci_hook;Nit:
kvm_zpci_hook?
Analogous to zpci_kvm_regist(er|ration) I would call the variable simply zpci_kvm i.e. the type is a registration and the variable is the instance of it that links zpci and kvm.
Quoted 81 lines
Quoted 53 lines
arch/s390/include/asm/kvm_host.h +0
-0#endif arch/s390/kvm/pci.c +6
-4@@ -431,8 +431,9 @@ static void kvm_s390_pci_dev_release(struct zpci_dev *zdev)* available, enable them and let userspace indicate whether or not they will * be used (specify SHM bit to disable). */ -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm) +static int kvm_s390_pci_register_kvm (void *opaque,struct kvm *kvm) { +struct zpci_dev *zdev = opaque;int rc;if (!zdev)@@ -510,10 +511,10 @@ int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm)kvm_put_kvm (kvm);return rc; } -EXPORT_SYMBOL_GPL (kvm_s390_pci_register_kvm); -void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev) +static void kvm_s390_pci_unregister_kvm (void *opaque) { +struct zpci_dev *zdev = opaque;struct kvm *kvm;if (!zdev)@@ -566,7 +567,6 @@ void kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev)kvm_put_kvm (kvm); } -EXPORT_SYMBOL_GPL (kvm_s390_pci_unregister_kvm);void kvm_s390_pci_init_list (struct kvm *kvm) {@@ -678,6 +678,8 @@ int kvm_s390_pci_init(void)spin_lock_init (&aift->gait_lock);mutex_init (&aift->aift_lock); + kvm_pci_hook.kvm_register = kvm_s390_pci_register_kvm; + kvm_pci_hook.kvm_unregister = kvm_s390_pci_unregister_kvm;return 0 ; }arch/s390/pci/Makefile +0
-0arch/s390/pci/Makefile obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_dma.o pci_clp.o pci_sysfs.o \ · line 11
Quoted 5 lines
arch/s390/pci/Makefile +2
-0@@ -7,3 +7,5 @@ obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_dma.o pci_clp.o pci_sysfs.o \ pci_event.o pci_debug.o pci_insn.o pci_mmio.o \ pci_bus.o obj-$(CONFIG_PCI_IOV) += pci_iov.o + +obj-y += pci_kvm_hook.oI guess it doesn't harm anything to add this unconditionally, but I think it would also be OK to just include this in the
CONFIG_PCIlist -vfio_pci_zdevandarch/s390/kvm/pciall rely onCONFIG_PCIviaCONFIG_VFIO_PCI_ZDEV_KVMwhich implies PCI viaVFIO_PCI.Quoted 7 lines
arch/s390/pci/pci_kvm_hook.c new file +1
-0@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only arch/s390/pci/pci_kvm_hook.c line 11
Quoted 10 lines
arch/s390/pci/pci_kvm_hook.c new file +10
-0+/* +* VFIO ZPCI devices support +* +* Copyright (C) IBM Corp. 2022. All rights reserved. +* Author(s): Pierre Morel <[email protected]> +*/ +#include <linux/kvm_host.h> + +struct kvm_register_hook kvm_pci_hook; +EXPORT_SYMBOL_GPL (kvm_pci_hook);Following the comments above,
zpci_kvm_register_hook,kvm_zpci_hook?
Quoted 1 line
I'm not sure if this really needs to be in a separate file or if it could just go into
arch/s390/pci.cwith thezpci_aipb-- If going the route of a separate file, up to Niklas whether he wants this under the S390 PCI maintainership or added to the list for s390 vfio-pci likearch/kvm/pci*andvfio_pci_zdev.
I'm fine with a separate file, pci.c is long enough as it is. I also don't have a problem with having it maintained as part of S390 PCI but logically I think it does fall more under arch/kvm/pci* so one could argue it should be added in the MAINTAINERS file in that section. If you change the struct name as I proposed above I would probably go with "pci_kvm_register.c"
Quoted 21 lines
Quoted 21 lines
drivers/vfio/pci/vfio_pci_zdev.c +4
-1@@ -151,7 +151,10 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev)if (!vdev->vdev.kvm)return 0 ; -return kvm_s390_pci_register_kvm (zdev, vdev->vdev.kvm); +if (kvm_pci_hook.kvm_register) +return kvm_pci_hook.kvm_register (zdev, vdev->vdev.kvm); + +return -ENOENT ; }void vfio_pci_zdev_close_device (struct vfio_pci_core_device *vdev)@@ -161,5 +164,6 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev)if (!zdev || !vdev->vdev.kvm)return ;
Quoted 8 lines
drivers/vfio/pci/vfio_pci_zdev.c void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev) · line 168
Quoted 3 lines
drivers/vfio/pci/vfio_pci_zdev.c +2
-1-kvm_s390_pci_unregister_kvm (zdev); +if (kvm_pci_hook.kvm_unregister) +return kvm_pci_hook.kvm_unregister (zdev);No need for the return here, this is a void function calling a void function.
Overall, this looks good to me and survives a series of compile and device passthrough tests on my end, just a matter of a few of these minor comments above. Thanks for tackling this Pierre!
Yes I agree, overall this looks good to me though I'm admittedly not very knowledgable about how to best handle module dependencies like this. It does look cleaner than the symbol_get() alternative we discussed.
Matthew Rosato
maintainerRe: [PATCH] KVM: s390: pci: Hook to access KVM lowlevel from VFIO
Quoted 57 lines · Niklas Schnelle
Quoted 29 lines · Matthew Rosato
Quoted 1 line · Pierre Morel
We have a cross dependency between KVM and VFIO.
maybe add something like 'when using s390
vfio_pci_zdevextensions for PCI passthrough'Quoted 5 lines
To be able to keep both subsystem modular we add a registering hook inside the S390 core code.
This fixes a build problem when VFIO is built-in and KVM is built as a module or excluded.
s/or excluded//
There's no problem when KVM is excluded, that forces
CONFIG_VFIO_PCI_ZDEV_KVM=nbecause of the 'depends on S390 && KVM'.Quoted 14 lines
Reported-by Randy Dunlap <[email protected]>Reported-by kernel test robot <[email protected]>Signed-off-by Pierre Morel <[email protected]>Fixes 09340b2fca007 ("KVM: s390: pci: add routines to start/stop inter..")Cc <[email protected]>--- arch/s390/include/asm/kvm_host.h | 17 ++++++----------- arch/s390/kvm/pci.c | 10 ++++++---- arch/s390/pci/Makefile | 2 ++ arch/s390/pci/pci_kvm_hook.c | 11 +++++++++++ drivers/vfio/pci/vfio_pci_zdev.c | 8 ++++++-- 5 files changed, 31 insertions(+), 17 deletions(-) create mode 100644 arch/s390/pci/pci_kvm_hook.cQuoted 4 lines
arch/s390/include/asm/kvm_host.h
Quoted 4 lines
arch/s390/include/asm/kvm_host.h +0
-0I added Janosch as second S390 KVM maintainer in case he wants to chime in.
Quoted 4 lines
Quoted 4 lines
arch/s390/include/asm/kvm_host.h +0
-0@@ -1038,16 +1038,11 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}#define __KVM_HAVE_ARCH_VM_FREEvoid kvm_arch_free_vm (struct kvm *kvm);Quoted 14 lines
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1041
Quoted 12 lines
arch/s390/include/asm/kvm_host.h +1
-11-#ifdef CONFIG_VFIO_PCI_ZDEV_KVM -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm); -void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev); -#else -static inline int kvm_s390_pci_register_kvm (struct zpci_dev *dev, -struct kvm *kvm) -{ -return -EPERM ; -} -static inline void kvm_s390_pci_unregister_kvm (struct zpci_dev *dev) {} -#endif +struct kvm_register_hook {Nit:
zpci_kvm_register_hook? Just to make it clear it's for zpci.
Quoted 3 lines
Hmm, I guess one could re-use the same struct for another such KVM dependency but I lean towards the same thinking as Matt, for now this is for zpci so stay specific we can always generalize later.
Yes, let's keep this zpci-specific.
Quoted 5 lines
Nit: For me hook and register together sound a bit redudant, maybe "
zpci_kvm_register"? Also question for Matt as a native speaker, should it rather be "registration" when used as a noun here?
Maybe just drop the 'register'. If there is a need for a 3rd function later, for example, it might not be related to registration.
e.g. struct kvm_zpci_hook {
...
};
extern struct kvm_zpci_hook zpci_kvm;
Quoted 15 lines
Quoted 4 lines
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1043
Quoted 4 lines
arch/s390/include/asm/kvm_host.h +2
-0+int (*kvm_register)(void *opaque,struct kvm *kvm); +void (*kvm_unregister)(void *opaque);I do wonder if this needs to be opague "struct
zpci_dev" should be defined even ifCONFIG_PCIis unset.Quoted 5 lines
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1046
Quoted 3 lines
arch/s390/include/asm/kvm_host.h +3
-0+}; + +extern struct kvm_register_hook kvm_pci_hook;Nit:
kvm_zpci_hook?
Quoted 4 lines
Analogous to
zpci_kvm_regist(er|ration)I would call the variable simplyzpci_kvmi.e. the type is a registration and the variable is the instance of it that links zpci and kvm.
Yeah, see above.
Quoted 81 lines
Quoted 81 lines
Quoted 53 lines
arch/s390/include/asm/kvm_host.h +0
-0#endif arch/s390/kvm/pci.c +6
-4@@ -431,8 +431,9 @@ static void kvm_s390_pci_dev_release(struct zpci_dev *zdev)* available, enable them and let userspace indicate whether or not they will * be used (specify SHM bit to disable). */ -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm) +static int kvm_s390_pci_register_kvm (void *opaque,struct kvm *kvm) { +struct zpci_dev *zdev = opaque;int rc;if (!zdev)@@ -510,10 +511,10 @@ int kvm_s390_pci_register_kvm(struct zpci_dev *zdev, struct kvm *kvm)kvm_put_kvm (kvm);return rc; } -EXPORT_SYMBOL_GPL (kvm_s390_pci_register_kvm); -void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev) +static void kvm_s390_pci_unregister_kvm (void *opaque) { +struct zpci_dev *zdev = opaque;struct kvm *kvm;if (!zdev)@@ -566,7 +567,6 @@ void kvm_s390_pci_unregister_kvm(struct zpci_dev *zdev)kvm_put_kvm (kvm); } -EXPORT_SYMBOL_GPL (kvm_s390_pci_unregister_kvm);void kvm_s390_pci_init_list (struct kvm *kvm) {@@ -678,6 +678,8 @@ int kvm_s390_pci_init(void)spin_lock_init (&aift->gait_lock);mutex_init (&aift->aift_lock); + kvm_pci_hook.kvm_register = kvm_s390_pci_register_kvm; + kvm_pci_hook.kvm_unregister = kvm_s390_pci_unregister_kvm;return 0 ; }arch/s390/pci/Makefile +0
-0arch/s390/pci/Makefile obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_dma.o pci_clp.o pci_sysfs.o \ · line 11
Quoted 5 lines
arch/s390/pci/Makefile +2
-0@@ -7,3 +7,5 @@ obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_dma.o pci_clp.o pci_sysfs.o \ pci_event.o pci_debug.o pci_insn.o pci_mmio.o \ pci_bus.o obj-$(CONFIG_PCI_IOV) += pci_iov.o + +obj-y += pci_kvm_hook.oI guess it doesn't harm anything to add this unconditionally, but I think it would also be OK to just include this in the
CONFIG_PCIlist -vfio_pci_zdevandarch/s390/kvm/pciall rely onCONFIG_PCIviaCONFIG_VFIO_PCI_ZDEV_KVMwhich implies PCI viaVFIO_PCI.Quoted 7 lines
arch/s390/pci/pci_kvm_hook.c new file +1
-0@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only arch/s390/pci/pci_kvm_hook.c line 11
Quoted 10 lines
arch/s390/pci/pci_kvm_hook.c new file +10
-0+/* +* VFIO ZPCI devices support +* +* Copyright (C) IBM Corp. 2022. All rights reserved. +* Author(s): Pierre Morel <[email protected]> +*/ +#include <linux/kvm_host.h> + +struct kvm_register_hook kvm_pci_hook; +EXPORT_SYMBOL_GPL (kvm_pci_hook);Following the comments above,
zpci_kvm_register_hook,kvm_zpci_hook?
Quoted 8 lines
Quoted 1 line
I'm not sure if this really needs to be in a separate file or if it could just go into
arch/s390/pci.cwith thezpci_aipb-- If going the route of a separate file, up to Niklas whether he wants this under the S390 PCI maintainership or added to the list for s390 vfio-pci likearch/kvm/pci*andvfio_pci_zdev.I'm fine with a separate file,
pci.cis long enough as it is. I also don't have a problem with having it maintained as part of S390 PCI but logically I think it does fall more underarch/kvm/pci*so one could argue it should be added in theMAINTAINERSfile in that section. If you change the struct name as I proposed above I would probably go with "pci_kvm_register.c"
OK, no problem with me for a separate file then, or maintaining said file. But I guess not pci_kvm_register.c per my comments above
Quoted 37 lines
Quoted 22 lines
Quoted 22 lines
drivers/vfio/pci/vfio_pci_zdev.c +4
-1@@ -151,7 +151,10 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev)if (!vdev->vdev.kvm)return 0 ; -return kvm_s390_pci_register_kvm (zdev, vdev->vdev.kvm); +if (kvm_pci_hook.kvm_register) +return kvm_pci_hook.kvm_register (zdev, vdev->vdev.kvm); + +return -ENOENT ; }void vfio_pci_zdev_close_device (struct vfio_pci_core_device *vdev)@@ -161,5 +164,6 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev)if (!zdev || !vdev->vdev.kvm)return ;Quoted 8 lines
drivers/vfio/pci/vfio_pci_zdev.c void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev) · line 168
Quoted 3 lines
drivers/vfio/pci/vfio_pci_zdev.c +2
-1-kvm_s390_pci_unregister_kvm (zdev); +if (kvm_pci_hook.kvm_unregister) +return kvm_pci_hook.kvm_unregister (zdev);No need for the return here, this is a void function calling a void function.
Overall, this looks good to me and survives a series of compile and device passthrough tests on my end, just a matter of a few of these minor comments above. Thanks for tackling this Pierre!
Yes I agree, overall this looks good to me though I'm admittedly not very knowledgable about how to best handle module dependencies like this. It does look cleaner than the
symbol_get()alternative we discussed.
Niklas Schnelle
maintainerRe: [PATCH] KVM: s390: pci: Hook to access KVM lowlevel from VFIO
Quoted 62 lines · Matthew Rosato
Quoted 57 lines · Niklas Schnelle
Quoted 29 lines · Matthew Rosato
Quoted 1 line · Pierre Morel
We have a cross dependency between KVM and VFIO.
maybe add something like 'when using s390
vfio_pci_zdevextensions for PCI passthrough'Quoted 5 lines
To be able to keep both subsystem modular we add a registering hook inside the S390 core code.
This fixes a build problem when VFIO is built-in and KVM is built as a module or excluded.
s/or excluded//
There's no problem when KVM is excluded, that forces
CONFIG_VFIO_PCI_ZDEV_KVM=nbecause of the 'depends on S390 && KVM'.Quoted 14 lines
Reported-by Randy Dunlap <[email protected]>Reported-by kernel test robot <[email protected]>Signed-off-by Pierre Morel <[email protected]>Fixes 09340b2fca007 ("KVM: s390: pci: add routines to start/stop inter..")Cc <[email protected]>--- arch/s390/include/asm/kvm_host.h | 17 ++++++----------- arch/s390/kvm/pci.c | 10 ++++++---- arch/s390/pci/Makefile | 2 ++ arch/s390/pci/pci_kvm_hook.c | 11 +++++++++++ drivers/vfio/pci/vfio_pci_zdev.c | 8 ++++++-- 5 files changed, 31 insertions(+), 17 deletions(-) create mode 100644 arch/s390/pci/pci_kvm_hook.cQuoted 4 lines
arch/s390/include/asm/kvm_host.h
Quoted 4 lines
arch/s390/include/asm/kvm_host.h +0
-0I added Janosch as second S390 KVM maintainer in case he wants to chime in.
Quoted 4 lines
Quoted 4 lines
arch/s390/include/asm/kvm_host.h +0
-0@@ -1038,16 +1038,11 @@ static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}#define __KVM_HAVE_ARCH_VM_FREEvoid kvm_arch_free_vm (struct kvm *kvm);Quoted 14 lines
arch/s390/include/asm/kvm_host.h static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {} · line 1041
Quoted 12 lines
arch/s390/include/asm/kvm_host.h +1
-11-#ifdef CONFIG_VFIO_PCI_ZDEV_KVM -int kvm_s390_pci_register_kvm (struct zpci_dev *zdev,struct kvm *kvm); -void kvm_s390_pci_unregister_kvm (struct zpci_dev *zdev); -#else -static inline int kvm_s390_pci_register_kvm (struct zpci_dev *dev, -struct kvm *kvm) -{ -return -EPERM ; -} -static inline void kvm_s390_pci_unregister_kvm (struct zpci_dev *dev) {} -#endif +struct kvm_register_hook {Nit:
zpci_kvm_register_hook? Just to make it clear it's for zpci.Quoted 3 lines
Hmm, I guess one could re-use the same struct for another such KVM dependency but I lean towards the same thinking as Matt, for now this is for zpci so stay specific we can always generalize later.
Quoted 8 lines
Yes, let's keep this zpci-specific.
Quoted 4 lines
Nit: For me hook and register together sound a bit redudant, maybe "
zpci_kvm_register"? Also question for Matt as a native speaker, should it rather be "registration" when used as a noun here?Maybe just drop the 'register'. If there is a need for a 3rd function later, for example, it might not be related to registration.
Yes, that sounds good and makes sense so "zpci_kvm_hook".
Quoted 7 lines
e.g. struct kvm_zpci_hook { ... };extern struct
kvm_zpci_hookzpci_kvm;
---8<---
Quoted 23 lines
Quoted 23 lines
Quoted 22 lines
Quoted 8 lines
arch/s390/pci/pci_kvm_hook.c new file +1
-0@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only arch/s390/pci/pci_kvm_hook.c line 11
Quoted 10 lines
arch/s390/pci/pci_kvm_hook.c new file +10
-0+/* +* VFIO ZPCI devices support +* +* Copyright (C) IBM Corp. 2022. All rights reserved. +* Author(s): Pierre Morel <[email protected]> +*/ +#include <linux/kvm_host.h> + +struct kvm_register_hook kvm_pci_hook; +EXPORT_SYMBOL_GPL (kvm_pci_hook);Following the comments above,
zpci_kvm_register_hook,kvm_zpci_hook?I'm not sure if this really needs to be in a separate file or if it could just go into
arch/s390/pci.cwith thezpci_aipb-- If going the route of a separate file, up to Niklas whether he wants this under the S390 PCI maintainership or added to the list for s390 vfio-pci likearch/kvm/pci*andvfio_pci_zdev.
Quoted 8 lines
Quoted 6 lines
I'm fine with a separate file,
pci.cis long enough as it is. I also don't have a problem with having it maintained as part of S390 PCI but logically I think it does fall more underarch/kvm/pci*so one could argue it should be added in theMAINTAINERSfile in that section. If you change the struct name as I proposed above I would probably go with "pci_kvm_register.c"OK, no problem with me for a separate file then, or maintaining said file. But I guess not
pci_kvm_register.cper my comments above
Yes, let's go with pci_kvm_hook.c then