s390 KVM PCI backports for 6.1
Subsystems (3)
v2 cover s390 KVM PCI backports for 6.1
Hi,
This patchset backports the s390 KVM PCI fixes to stable-6.1. The patchset includes one dependency patch (patch 1) from Jason Gunthorpe, to export free_uid(). This is needed to resolve a compilation error. I have resolved all other merge conflicts and tested them on s390.
Thanks
Farhan
Farhan Ali (6):
KVM: s390: pci: Reject adapter interrupt forwarding if already enabled
KVM: s390: pci: Fix memory accounting for pinned/unpinned pages
KVM: s390: pci: Fix missing error codes and memory unaccounting
KVM: s390: pci: Fix NULL dereference on AIBV allocation failure
KVM: s390: pci: Fix resource leak on IRQ registration failure
KVM: s390: pci: Validate AIBV and AISB before pinning guest pages
Jason Gunthorpe (1):
kernel/user: Allow user_struct::locked_vm to be usable for iommufd
Matthew Rosato (1):
KVM: s390: pci: Fix aisb calculation
arch/s390/kvm/pci.c | 114 +++++++++++++++++++++++++++++-------- arch/s390/kvm/pci.h | 2 + include/linux/sched/user.h | 2 +- kernel/user.c | 1 + 4 files changed, 93 insertions(+), 26 deletions(-)
Signature · 1 line
2.43.0
PATCH v2 1/8 kernel/user: Allow user_struct::locked_vm to be usable for iommufd
From: Jason Gunthorpe <[email protected]>
Following the pattern of io_uring, perf, skb, and bpf, iommfd will use user->locked_vm for accounting pinned pages. Ensure the value is included in the struct and export free_uid() as iommufd is modular.
user->locked_vm is the good accounting to use for ulimit because it is per-user, and the security sandboxing of locked pages is not supposed to be per-process. Other places (vfio, vdpa and infiniband) have used mm->pinned_vm and/or mm->locked_vm for accounting pinned pages, but this is only per-process and inconsistent with the new FOLL_LONGTERM users in the kernel.
Concurrent work is underway to try to put this in a cgroup, so everything can be consistent and the kernel can provide a FOLL_LONGTERM limit that actually provides security.
--- include/linux/sched/user.h | 2 +- kernel/user.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
include/linux/sched/user.h +1 -1
@@ -25,7 +25,7 @@ struct user_struct {#if defined(CONFIG_PERF_EVENTS ) || defined(CONFIG_BPF_SYSCALL ) || \ defined(CONFIG_NET ) || defined(CONFIG_IO_URING ) || \ - defined(CONFIG_VFIO_PCI_ZDEV_KVM ) + defined(CONFIG_VFIO_PCI_ZDEV_KVM ) ||IS_ENABLED (CONFIG_IOMMUFD )atomic_long_t locked_vm;#endif #ifdef CONFIG_WATCH_QUEUE
kernel/user.c +1 -0
@@ -185,6 +185,7 @@ void free_uid(struct user_struct *up)if (refcount_dec_and_lock_irqsave (&up->__count, &uidhash_lock, &flags))free_user (up, flags); } +EXPORT_SYMBOL_GPL (free_uid);struct user_struct *alloc_uid (kuid_t uid) {
Signature · 1 line
2.43.0
PATCH v2 2/8 KVM: s390: pci: Reject adapter interrupt forwarding if already enabled
In reply to Farhan AliThe MPCIFC instruction doesn't allow registering adapter interrupts without first unregistering. So reject any request to enable interrupt forwarding if its already enabled for the zPCI device. This also fixes overwriting and thus leaking resources when the ioctl is called multiple times for the same device.
--- arch/s390/kvm/pci.c | 4 ++++ 1 file changed, 4 insertions(+)
arch/s390/kvm/pci.c +4 -0
@@ -239,6 +239,10 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,if (zdev->gisa ==0 )return -EINVAL ; +/* AIF already enabled for the device */ +if (zdev->kzdev->fib.fmt0.aibv !=0 ) +return -EINVAL ; + kvm = zdev->kzdev->kvm; msi_vecs = min_t(unsigned int, fib->fmt0.noi, zdev->max_msi);
Signature · 1 line
2.43.0
PATCH v2 3/8 KVM: s390: pci: Fix memory accounting for pinned/unpinned pages
In reply to Farhan AliThe account_mem() and unaccount_mem() functions call get_uid() which increments the reference count of struct user_struct on every invocation. But we don't decrement the count by calling free_uid(). It also accounted/unaccounted the pages against the current->mm. But its possible the unaccount_mem() can be called from a different process context than the one that originally pinned the pages.
Let's fix this by storing the pinning process user_struct and mm_struct when accounting for pinned pages, and subsequently free these resources when the pages are unpinned.
--- arch/s390/kvm/pci.c | 48 ++++++++++++++++++++++++++++++++------------- arch/s390/kvm/pci.h | 2 ++ 2 files changed, 36 insertions(+), 14 deletions(-)
arch/s390/kvm/pci.c +34 -14
@@ -191,34 +191,54 @@ static int kvm_zpci_clear_airq(struct zpci_dev *zdev)return cc ? -EIO :0 ; } -static inline void unaccount_mem (unsigned long nr_pages) +static inline void unaccount_mem (struct kvm_zdev *kzdev,unsigned long nr_pages) { -struct user_struct *user =get_uid (current_user ()); +struct user_struct *user = kzdev->user_account; +struct mm_struct *mm_account = kzdev->mm_account; -if (user) +if (user) {atomic_long_sub (nr_pages, &user->locked_vm); -if (current->mm) -atomic64_sub (nr_pages, ¤t->mm->pinned_vm); +free_uid (user); + kzdev->user_account =NULL ; + } + +if (mm_account) { +atomic64_sub (nr_pages, &mm_account->pinned_vm); +mmdrop (mm_account); + kzdev->mm_account =NULL ; + } } -static inline int account_mem (unsigned long nr_pages) +static inline int account_mem (struct kvm_zdev *kzdev,unsigned long nr_pages) {struct user_struct *user =get_uid (current_user ());unsigned long page_limit, cur_pages, new_pages; +int rc =0 ; page_limit =rlimit (RLIMIT_MEMLOCK ) >>PAGE_SHIFT ; + cur_pages =atomic_long_read (&user->locked_vm);do { - cur_pages =atomic_long_read (&user->locked_vm); new_pages = cur_pages + nr_pages; -if (new_pages > page_limit) -return -ENOMEM ; - }while (atomic_long_cmpxchg (&user->locked_vm, cur_pages, - new_pages) != cur_pages); +if (new_pages > page_limit) { + rc = -ENOMEM ; + goto out; + } + }while (!atomic_long_try_cmpxchg (&user->locked_vm, &cur_pages, new_pages)); + +if (current->mm) { +mmgrab (current->mm); +atomic64_add (nr_pages, ¤t->mm->pinned_vm); + } -atomic64_add (nr_pages, ¤t->mm->pinned_vm); + kzdev->user_account = user; + kzdev->mm_account = current->mm;return 0 ; + +out: +free_uid (user); +return rc; } static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,
@@ -281,7 +301,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, }/* Account for pinned pages, roll back on failure */ -if (account_mem (pcount)) +if (account_mem (zdev->kzdev, pcount)) goto unpin2;/* AISB must be allocated before we can fill in GAITE */
@@ -402,7 +422,7 @@ static int kvm_s390_pci_aif_disable(struct zpci_dev *zdev, bool force) pcount++; }if (pcount >0 ) -unaccount_mem (pcount); +unaccount_mem (kzdev, pcount); out:mutex_unlock (&aift->aift_lock);
arch/s390/kvm/pci.h +2 -0
@@ -22,6 +22,8 @@ struct kvm_zdev {struct kvm *kvm;struct zpci_fib fib;struct list_head entry; +struct user_struct *user_account; +struct mm_struct *mm_account; };struct zpci_gaite {
Signature · 1 line
2.43.0
PATCH v2 4/8 KVM: s390: pci: Fix missing error codes and memory unaccounting
In reply to Farhan AliIn kvm_s390_pci_aif_enable() two error paths failed to set an error code, causing the function to return 0 on failure. It also failed to rollback memory accounting on failure. Fix both by propagating an error code on failure and calling unaccount_mem() in the cleanup path.
--- arch/s390/kvm/pci.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
arch/s390/kvm/pci.c +7 -2
@@ -301,14 +301,17 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, }/* Account for pinned pages, roll back on failure */ -if (account_mem (zdev->kzdev, pcount)) + rc =account_mem (zdev->kzdev, pcount); +if (rc) goto unpin2;/* AISB must be allocated before we can fill in GAITE */ mutex_lock (&aift->aift_lock); bit =airq_iv_alloc_bit (aift->sbv); -if (bit ==-1UL ) +if (bit ==-1UL ) { + rc = -ENOMEM ; goto unlock; + } zdev->aisb = bit;/* store the summary bit number */ zdev->aibv = airq_iv_create(msi_vecs, AIRQ_IV_DATA | AIRQ_IV_BITLOCK |
@@ -352,6 +355,8 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,return rc; unlock: +if (pcount >0 ) +unaccount_mem (zdev->kzdev, pcount);mutex_unlock (&aift->aift_lock); unpin2:if (fib->fmt0.sum ==1 )
Signature · 1 line
2.43.0
PATCH v2 5/8 KVM: s390: pci: Fix NULL dereference on AIBV allocation failure
In reply to Farhan AliThe airq_iv_create() can return NULL on failure, but the return value was never checked. If it fails, zdev->aibv will be NULL and fail when dereferenced in kvm_zpci_set_airq(). Add a NULL check and free the previously allocated AISB bit and zdev->aisb on failure.
--- arch/s390/kvm/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/s390/kvm/pci.c +8 -0
@@ -318,6 +318,11 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,AIRQ_IV_GUESTVEC , phys_to_virt(fib->fmt0.aibv)); +if (!zdev->aibv) { + rc = -ENOMEM ; + goto free_aisb; + } +spin_lock_irq (&aift->gait_lock); gaite = aift->gait + zdev->aisb;
@@ -354,6 +359,9 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, rc =kvm_zpci_set_airq (zdev);return rc; +free_aisb: +airq_iv_free_bit (aift->sbv, zdev->aisb); + zdev->aisb =0 ; unlock:if (pcount >0 )unaccount_mem (zdev->kzdev, pcount);
Signature · 1 line
2.43.0
PATCH v2 6/8 KVM: s390: pci: Fix resource leak on IRQ registration failure
In reply to Farhan AliCurrently if kvm_zpci_set_airq() fails, kvm_s390_pci_aif_enable() returns the error code but doesn't do any resource cleanup thus leaking resources. Fix this by cleaning up all the resources such as the GAITE, AIBV, AISB and unpinning any pinned pages. While at it, remove dead code that stored FIB values that were never referenced.
As part of the cleanup, we are also holding the aift_lock a bit longer, as we hold the lock while executing the MPCIFC instruction. Though this is not strictly necessary, it means we don't have to drop and re-acquire in the error case.
--- arch/s390/kvm/pci.c | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-)
arch/s390/kvm/pci.c +21 -8
@@ -345,19 +345,32 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, aift->kzdev[zdev->aisb] = zdev->kzdev;spin_unlock_irq (&aift->gait_lock); -/* Update guest FIB for re-issue */ - fib->fmt0.aisbo = zdev->aisb &63 ; - fib->fmt0.aisb =virt_to_phys (aift->sbv->vector + (zdev->aisb /64 ) *8 ); - fib->fmt0.isc = gisc; -/* Save some guest fib values in the host for later use */ - zdev->kzdev->fib.fmt0.isc = fib->fmt0.isc; + zdev->kzdev->fib.fmt0.isc = gisc; zdev->kzdev->fib.fmt0.aibv = fib->fmt0.aibv; -mutex_unlock (&aift->aift_lock);/* Issue the clp to setup the irq now */ rc =kvm_zpci_set_airq (zdev); -return rc; +if (!rc) { +mutex_unlock (&aift->aift_lock); +return rc; + } + +/* Start cleanup */ + zdev->kzdev->fib.fmt0.isc =0 ; + zdev->kzdev->fib.fmt0.aibv =0 ; + +spin_lock_irq (&aift->gait_lock); + gaite->count--; + gaite->aisb =0 ; + gaite->gisc =0 ; + gaite->aisbo =0 ; + gaite->gisa =0 ; + aift->kzdev[zdev->aisb] =NULL ; +spin_unlock_irq (&aift->gait_lock); + +airq_iv_release (zdev->aibv); + zdev->aibv =NULL ; free_aisb:airq_iv_free_bit (aift->sbv, zdev->aisb);
Signature · 1 line
2.43.0
PATCH v2 7/8 KVM: s390: pci: Validate AIBV and AISB before pinning guest pages
In reply to Farhan AliThe AIBV holds one bit per MSI-X vector for a given function. The size of the bit vector is derived from the NOI and the AIBVO. If the size of the AIBV exceeds a single page boundary, then reject the request as we cannot safely pin the guest AIBV.
Similarly reject the request if the AISB address is not 8-byte aligned as the architecture requires doubleword alignment for the summary bit address. Since the AISBO can address up to 64 bits, the size of the AISB can only be 8 bytes for the function. This also ensures the AISB doesn't exceed a single page boundary.
--- arch/s390/kvm/pci.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)
arch/s390/kvm/pci.c +15 -1
@@ -245,7 +245,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, bool assist) {struct page *pages[1 ], *aibv_page, *aisb_page =NULL ; -unsigned int msi_vecs, idx; +unsigned int msi_vecs, idx, size;struct zpci_gaite *gaite;unsigned long hva, bit;struct kvm *kvm;
@@ -272,6 +272,14 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,return gisc;/* Replace AIBV address */ + size =BITS_TO_LONGS (msi_vecs + fib->fmt0.aibvo) *sizeof (unsigned long ); + npages =DIV_ROUND_UP ((fib->fmt0.aibv & ~PAGE_MASK ) + size,PAGE_SIZE ); +/* AIBV cannot span more than 1 page */ +if (npages >1 ) { + rc = -EINVAL ; + goto out; + } + idx =srcu_read_lock (&kvm->srcu); hva =gfn_to_hva (kvm,gpa_to_gfn ((gpa_t )fib->fmt0.aibv)); npages =pin_user_pages_fast (hva,1 ,FOLL_WRITE |FOLL_LONGTERM , pages);
@@ -287,6 +295,12 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,/* Pin the guest AISB if one was specified */ if (fib->fmt0.sum ==1 ) { +/* AISB must be dword aligned */ +if (fib->fmt0.aisb &0x7 ) { + rc = -EINVAL ; + goto unpin1; + } + idx =srcu_read_lock (&kvm->srcu); hva =gfn_to_hva (kvm,gpa_to_gfn ((gpa_t )fib->fmt0.aisb)); npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM,
Signature · 1 line
2.43.0
PATCH v2 8/8 KVM: s390: pci: Fix aisb calculation
In reply to Farhan AliFrom: Matthew Rosato <[email protected]>
The current implementation of aisb calculation will erroneously index via an unsigned long * as well as multiply by 8B for every 64-bits in the offset; only one or the other is required. This throws off aisb calculations once the number of devices exceeds 64, and can result in out-of-bounds access as well as failure to indicate summary bits associated with those devices in guests.
Fix this by converting to a physical address before applying the offset, as is already done in arch/s390/pci/pci_irq.c.
--- arch/s390/kvm/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/s390/kvm/pci.c +1 -1
@@ -167,7 +167,7 @@ static int kvm_zpci_set_airq(struct zpci_dev *zdev) fib.fmt0.noi =airq_iv_end (zdev->aibv); fib.fmt0.aibv =virt_to_phys (zdev->aibv->vector); fib.fmt0.aibvo =0 ; - fib.fmt0.aisb =virt_to_phys (aift->sbv->vector + (zdev->aisb /64 ) *8 ); + fib.fmt0.aisb =virt_to_phys (aift->sbv->vector) + (zdev->aisb /64 ) *8 ; fib.fmt0.aisbo = zdev->aisb &63 ; fib.gd = zdev->gisa;
Signature · 1 line
2.43.0
Sasha Levin
maintainerQuoted 4 lines
This patchset backports the s390 KVM PCI fixes to stable-6.1. The patchset includes one dependency patch (patch 1) from Jason Gunthorpe, to export
free_uid(). This is needed to resolve a compilation error. I have resolved all other merge conflicts and tested them on s390.
This one is based on released 6.1.y, but the 6.1 queue already carries 2/8, 5/8 and 7/8. With 5/8 queued the free_aisb: label sits ahead of unlock:, so 4/8 no longer applies at all and a 3-way merge can't recover it either. And taking 3/8 without 4/8 would open a fresh unbounded mm_struct/user_struct leak, so a partial series isn't an option.
Could you rebase onto the 6.1 queue - drop 2/8, 5/8 and 7/8, and re-resolve 4/8 over the free_aisb: block that is already there?
Signature · 2 lines
Thanks, Sasha
Quoted 4 lines · Sasha Levin
Quoted 4 lines
This patchset backports the s390 KVM PCI fixes to stable-6.1. The patchset includes one dependency patch (patch 1) from Jason Gunthorpe, to export
free_uid(). This is needed to resolve a compilation error. I have resolved all other merge conflicts and tested them on s390.
Quoted 8 lines
This one is based on released 6.1.y, but the 6.1 queue already carries 2/8, 5/8 and 7/8. With 5/8 queued the
free_aisb: label sits ahead of unlock:, so 4/8 no longer applies at all and a 3-way merge can't recover it either. And taking 3/8 without 4/8 would open a fresh unbounded mm_struct/user_struct leak, so a partial series isn't an option.Could you rebase onto the 6.1 queue - drop 2/8, 5/8 and 7/8, and re-resolve 4/8 over the
free_aisb: block that is already there?
Yup, will send out a new revision dropping the patches already merged to 6.1-rc1. Didn't realize 2,5, and 7 were already applied.
Thanks
Farhan