vfio: Get/put KVM only for the first/last vfio_df_open/close in cdev path
Subsystems (2)
PATCH 1/1 vfio: Get/put KVM only for the first/last vfio_df_open/close in cdev path
In the device cdev path, adjust the handling of the KVM reference count to only increment with the first vfio_df_open() and decrement after the final vfio_df_close(). This change addresses a KVM reference leak that occurs when a device cdev file is opened multiple times and attempts to bind to iommufd repeatedly.
Currently, vfio_df_get_kvm_safe() is invoked prior to each vfio_df_open() in the cdev path during iommufd binding. The corresponding vfio_device_put_kvm() is executed either when iommufd is unbound or if an error occurs during the binding process.
However, issues arise when a device binds to iommufd more than once. The second vfio_df_open() will fail during iommufd binding, and vfio_device_put_kvm() will be triggered, setting device->kvm to NULL. Consequently, when iommufd is unbound from the first successfully bound device, vfio_device_put_kvm() becomes ineffective, leading to a leak in the KVM reference count.
Below is the calltrace that will be produced in this scenario when the KVM module is unloaded afterwards, reporting "BUG kvm_vcpu (Tainted: G S):
Objects remaining in kvm_vcpu on __kmem_cache_shutdown()".
Call Trace: <TASK> dump_stack_lvl+0x80/0xc0 slab_err+0xb0/0xf0 ? __kmem_cache_shutdown+0xc1/0x4e0 ? rcu_is_watching+0x11/0x50 ? lock_acquired+0x144/0x3c0 __kmem_cache_shutdown+0x1b7/0x4e0 kmem_cache_destroy+0xa6/0x260 kvm_exit+0x80/0xc0 [kvm] vmx_exit+0xe/0x20 [kvm_intel] __x64_sys_delete_module+0x143/0x250 ? ktime_get_coarse_real_ts64+0xd3/0xe0 ? syscall_trace_enter+0x143/0x210 do_syscall_64+0x6f/0x140 entry_SYSCALL_64_after_hwframe+0x76/0x7e
--- drivers/vfio/device_cdev.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
drivers/vfio/device_cdev.c +9 -4
@@ -65,6 +65,7 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, {struct vfio_device *device = df->device;struct vfio_device_bind_iommufd bind; +bool put_kvm = false;unsigned long minsz;int ret;
@@ -101,12 +102,15 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, }/* -* Before the device open, get the KVM pointer currently +* Before the device's first open, get the KVM pointer currently * associated with the device file (if there is) and obtain -* a reference. This reference is held until device closed. +* a reference. This reference is held until device's last closed. * Save the pointer in the device for use by drivers. */ -vfio_df_get_kvm_safe (df); +if (device->open_count ==0 ) { +vfio_df_get_kvm_safe (df); + put_kvm = true; + } ret =vfio_df_open (df);if (ret)
@@ -129,7 +133,8 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, out_close_device:vfio_df_close (df); out_put_kvm: -vfio_device_put_kvm (device); +if (put_kvm) +vfio_device_put_kvm (device);iommufd_ctx_put (df->iommufd); df->iommufd =NULL ; out_unlock:
base-commit: 6ba59ff4227927d3a8530fc2973b80e94b54d58f
Signature · 1 line
2.43.2
Quoted 11 lines · Yan Zhao
In the device cdev path, adjust the handling of the KVM reference count to only increment with the first
vfio_df_open()and decrement after the finalvfio_df_close(). This change addresses a KVM reference leak that occurs when a device cdev file is opened multiple times and attempts to bind to iommufd repeatedly.Currently,
vfio_df_get_kvm_safe()is invoked prior to eachvfio_df_open()in the cdev path during iommufd binding. The correspondingvfio_device_put_kvm()is executed either when iommufd is unbound or if an error occurs during the binding process.
Quoted 6 lines
However, issues arise when a device binds to iommufd more than once. The second
vfio_df_open()will fail during iommufd binding, andvfio_device_put_kvm()will be triggered, settingdevice->kvmto NULL. Consequently, when iommufd is unbound from the first successfully bound device,vfio_device_put_kvm()becomes ineffective, leading to a leak in the KVM reference count.
Good catch!!!
Quoted 59 lines
Below is the calltrace that will be produced in this scenario when the KVM module is unloaded afterwards, reporting "BUG
kvm_vcpu(Tainted: G S):
Objects remaining inkvm_vcpuon__kmem_cache_shutdown()".Call Trace: <TASK> dump_stack_lvl+0x80/0xc0 slab_err+0xb0/0xf0 ? __kmem_cache_shutdown+0xc1/0x4e0 ? rcu_is_watching+0x11/0x50 ? lock_acquired+0x144/0x3c0 __kmem_cache_shutdown+0x1b7/0x4e0 kmem_cache_destroy+0xa6/0x260 kvm_exit+0x80/0xc0 [kvm] vmx_exit+0xe/0x20 [kvm_intel] __x64_sys_delete_module+0x143/0x250 ? ktime_get_coarse_real_ts64+0xd3/0xe0 ? syscall_trace_enter+0x143/0x210 do_syscall_64+0x6f/0x140 entry_SYSCALL_64_after_hwframe+0x76/0x7eFixes 5fcc26969a16 ("vfio: Add VFIO_DEVICE_BIND_IOMMUFD")Signed-off-by Yan Zhao <[email protected]>--- drivers/vfio/device_cdev.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)drivers/vfio/device_cdev.c +7
-3@@ -65,6 +65,7 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, {struct vfio_device *device = df->device;struct vfio_device_bind_iommufd bind; +bool put_kvm = false;unsigned long minsz;int ret;@@ -101,12 +102,15 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, }/* -* Before the device open, get the KVM pointer currently +* Before the device's first open, get the KVM pointer currently * associated with the device file (if there is) and obtain -* a reference. This reference is held until device closed. +* a reference. This reference is held until device's last closed. * Save the pointer in the device for use by drivers. */ -vfio_df_get_kvm_safe (df); +if (device->open_count ==0 ) { +vfio_df_get_kvm_safe (df); + put_kvm = true; + } ret =vfio_df_open (df);if (ret)
drivers/vfio/device_cdev.c long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, · line 136
Quoted 5 lines
drivers/vfio/device_cdev.c +1
-1@@ -129,7 +133,8 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, out_close_device:vfio_df_close (df); out_put_kvm: -vfio_device_put_kvm (device); +if (put_kvm)
you may use if (device->open_count == 0) as well here to save a bool. Otherwise looks good to me.
Quoted 6 lines
drivers/vfio/device_cdev.c +1
-0+vfio_device_put_kvm (device);iommufd_ctx_put (df->iommufd); df->iommufd =NULL ; out_unlock:base-commit:
6ba59ff4227927d3a8530fc2973b80e94b54d58f
BTW. The vfio_device_get_kvm_safe() is not supposed to be invoked multiple times by design as the second call would override the device->put_kvm and device->kvm. This does not change the put_kvm nor the kvm though. But not a good thing anyghow. maybe worth a warn like below.
drivers/vfio/vfio_main.c +2 -0
@@ -408,6 +408,8 @@ void vfio_device_get_kvm_safe(struct vfio_device *device, struct kvm *kvm)if (!kvm)return ; +WARN_ON (device->put_kvm || device->kvm); + pfn =symbol_get (kvm_put_kvm);if (WARN_ON (!pfn))return ;
Signature · 2 lines
Regards, Yi Liu
Quoted 77 lines · Yi Liu
Quoted 11 lines · Yan Zhao
In the device cdev path, adjust the handling of the KVM reference count to only increment with the first
vfio_df_open()and decrement after the finalvfio_df_close(). This change addresses a KVM reference leak that occurs when a device cdev file is opened multiple times and attempts to bind to iommufd repeatedly.Currently,
vfio_df_get_kvm_safe()is invoked prior to eachvfio_df_open()in the cdev path during iommufd binding. The correspondingvfio_device_put_kvm()is executed either when iommufd is unbound or if an error occurs during the binding process.Quoted 6 lines
However, issues arise when a device binds to iommufd more than once. The second
vfio_df_open()will fail during iommufd binding, andvfio_device_put_kvm()will be triggered, settingdevice->kvmto NULL. Consequently, when iommufd is unbound from the first successfully bound device,vfio_device_put_kvm()becomes ineffective, leading to a leak in the KVM reference count.Good catch!!!
Quoted 56 lines
Below is the calltrace that will be produced in this scenario when the KVM module is unloaded afterwards, reporting "BUG
kvm_vcpu(Tainted: G S):
Objects remaining inkvm_vcpuon__kmem_cache_shutdown()".Call Trace: <TASK> dump_stack_lvl+0x80/0xc0 slab_err+0xb0/0xf0 ? __kmem_cache_shutdown+0xc1/0x4e0 ? rcu_is_watching+0x11/0x50 ? lock_acquired+0x144/0x3c0 __kmem_cache_shutdown+0x1b7/0x4e0 kmem_cache_destroy+0xa6/0x260 kvm_exit+0x80/0xc0 [kvm] vmx_exit+0xe/0x20 [kvm_intel] __x64_sys_delete_module+0x143/0x250 ? ktime_get_coarse_real_ts64+0xd3/0xe0 ? syscall_trace_enter+0x143/0x210 do_syscall_64+0x6f/0x140 entry_SYSCALL_64_after_hwframe+0x76/0x7eFixes 5fcc26969a16 ("vfio: Add VFIO_DEVICE_BIND_IOMMUFD")Signed-off-by Yan Zhao <[email protected]>--- drivers/vfio/device_cdev.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)drivers/vfio/device_cdev.c +7
-3@@ -65,6 +65,7 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, {struct vfio_device *device = df->device;struct vfio_device_bind_iommufd bind; +bool put_kvm = false;unsigned long minsz;int ret;@@ -101,12 +102,15 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, }/* -* Before the device open, get the KVM pointer currently +* Before the device's first open, get the KVM pointer currently * associated with the device file (if there is) and obtain -* a reference. This reference is held until device closed. +* a reference. This reference is held until device's last closed. * Save the pointer in the device for use by drivers. */ -vfio_df_get_kvm_safe (df); +if (device->open_count ==0 ) { +vfio_df_get_kvm_safe (df); + put_kvm = true; + } ret =vfio_df_open (df);if (ret)@@ -129,7 +133,8 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,
Quoted 8 lines
drivers/vfio/device_cdev.c long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, · line 136
Quoted 5 lines
drivers/vfio/device_cdev.c +1
-1out_close_device:vfio_df_close (df); out_put_kvm: -vfio_device_put_kvm (device); +if (put_kvm)you may use if (
device->open_count== 0) as well here to save a bool. Otherwise looks good to me.
Upon here, device->open_count is not necessarily 0 even for the first open. The failure can be after a successful increment of device->open_count.
Maybe renaming "bool put_kvm" to "bool is_first_open" to save an assignment in most common case?
Quoted 1 line
Reviewed-by Yi Liu <[email protected]>
Thanks:)
Quoted 11 lines
Quoted 7 lines
drivers/vfio/device_cdev.c +1
-0+vfio_device_put_kvm (device);iommufd_ctx_put (df->iommufd); df->iommufd =NULL ; out_unlock:base-commit:
6ba59ff4227927d3a8530fc2973b80e94b54d58fBTW. The
vfio_device_get_kvm_safe()is not supposed to be invoked multiple times by design as the second call would override thedevice->put_kvmanddevice->kvm. This does not change theput_kvmnor the kvm though. But not a
"kvm" may also be changed if the second bind is from a different VM.
drivers/vfio/vfio_main.c void vfio_device_get_kvm_safe(struct vfio_device · line 412
Quoted 12 lines
good thing anyghow. maybe worth a warn like below.
drivers/vfio/vfio_main.c +1
-0@@ -408,6 +408,8 @@ void vfio_device_get_kvm_safe(struct vfio_device *device, struct kvm *kvm)if (!kvm)return ; +WARN_ON (device->put_kvm || device->kvm);
Yes, better.
Quoted 7 lines
drivers/vfio/vfio_main.c +0
-0pfn =symbol_get (kvm_put_kvm);if (WARN_ON (!pfn))return ;Signature · 3 lines
-- Regards, Yi Liu
Quoted 101 lines · Yan Zhao
Quoted 77 lines · Yi Liu
Quoted 11 lines · Yan Zhao
In the device cdev path, adjust the handling of the KVM reference count to only increment with the first
vfio_df_open()and decrement after the finalvfio_df_close(). This change addresses a KVM reference leak that occurs when a device cdev file is opened multiple times and attempts to bind to iommufd repeatedly.Currently,
vfio_df_get_kvm_safe()is invoked prior to eachvfio_df_open()in the cdev path during iommufd binding. The correspondingvfio_device_put_kvm()is executed either when iommufd is unbound or if an error occurs during the binding process.Quoted 6 lines
However, issues arise when a device binds to iommufd more than once. The second
vfio_df_open()will fail during iommufd binding, andvfio_device_put_kvm()will be triggered, settingdevice->kvmto NULL. Consequently, when iommufd is unbound from the first successfully bound device,vfio_device_put_kvm()becomes ineffective, leading to a leak in the KVM reference count.Good catch!!!
Quoted 56 lines
Below is the calltrace that will be produced in this scenario when the KVM module is unloaded afterwards, reporting "BUG
kvm_vcpu(Tainted: G S):
Objects remaining inkvm_vcpuon__kmem_cache_shutdown()".Call Trace: <TASK> dump_stack_lvl+0x80/0xc0 slab_err+0xb0/0xf0 ? __kmem_cache_shutdown+0xc1/0x4e0 ? rcu_is_watching+0x11/0x50 ? lock_acquired+0x144/0x3c0 __kmem_cache_shutdown+0x1b7/0x4e0 kmem_cache_destroy+0xa6/0x260 kvm_exit+0x80/0xc0 [kvm] vmx_exit+0xe/0x20 [kvm_intel] __x64_sys_delete_module+0x143/0x250 ? ktime_get_coarse_real_ts64+0xd3/0xe0 ? syscall_trace_enter+0x143/0x210 do_syscall_64+0x6f/0x140 entry_SYSCALL_64_after_hwframe+0x76/0x7eFixes 5fcc26969a16 ("vfio: Add VFIO_DEVICE_BIND_IOMMUFD")Signed-off-by Yan Zhao <[email protected]>--- drivers/vfio/device_cdev.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)drivers/vfio/device_cdev.c +7
-3@@ -65,6 +65,7 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, {struct vfio_device *device = df->device;struct vfio_device_bind_iommufd bind; +bool put_kvm = false;unsigned long minsz;int ret;@@ -101,12 +102,15 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, }/* -* Before the device open, get the KVM pointer currently +* Before the device's first open, get the KVM pointer currently * associated with the device file (if there is) and obtain -* a reference. This reference is held until device closed. +* a reference. This reference is held until device's last closed. * Save the pointer in the device for use by drivers. */ -vfio_df_get_kvm_safe (df); +if (device->open_count ==0 ) { +vfio_df_get_kvm_safe (df); + put_kvm = true; + } ret =vfio_df_open (df);if (ret)@@ -129,7 +133,8 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,Quoted 8 lines
drivers/vfio/device_cdev.c long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, · line 136
Quoted 5 lines
drivers/vfio/device_cdev.c +1
-1out_close_device:vfio_df_close (df); out_put_kvm: -vfio_device_put_kvm (device); +if (put_kvm)you may use if (
device->open_count== 0) as well here to save a bool. Otherwise looks good to me.Upon here,
device->open_countis not necessarily 0 even for the first open. The failure can be after a successful increment ofdevice->open_count.Maybe renaming "bool
put_kvm" to "boolis_first_open" to save an assignment in most common case?Quoted 1 line
Reviewed-by Yi Liu <[email protected]>Thanks:)
Quoted 6 lines
Quoted 6 lines
drivers/vfio/device_cdev.c +1
-0+vfio_device_put_kvm (device);iommufd_ctx_put (df->iommufd); df->iommufd =NULL ; out_unlock:
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
base-commit:
6ba59ff4227927d3a8530fc2973b80e94b54d58fBTW. The
vfio_device_get_kvm_safe()is not supposed to be invoked multiple times by design as the second call would override thedevice->put_kvmanddevice->kvm. This does not change theput_kvmnor the kvm though. But not a"kvm" may also be changed if the second bind is from a different VM.
yep.
Quoted 21 lines
drivers/vfio/vfio_main.c void vfio_device_get_kvm_safe(struct vfio_device · line 412
Quoted 12 lines
good thing anyghow. maybe worth a warn like below.
drivers/vfio/vfio_main.c +1
-0@@ -408,6 +408,8 @@ void vfio_device_get_kvm_safe(struct vfio_device *device, struct kvm *kvm)if (!kvm)return ; +WARN_ON (device->put_kvm || device->kvm);Yes, better.
Quoted 7 lines
drivers/vfio/vfio_main.c +0
-0pfn =symbol_get (kvm_put_kvm);if (WARN_ON (!pfn))return ;Signature · 3 lines
-- Regards, Yi Liu
Signature · 2 lines
Regards, Yi Liu
Quoted 14 lines
From: Zhao, Yan Y <[email protected]>
Sent: Friday, June 28, 2024 11:19 PMIn the device cdev path, adjust the handling of the KVM reference count to only increment with the first
vfio_df_open()and decrement after the finalvfio_df_close(). This change addresses a KVM reference leak that occurs when a device cdev file is opened multiple times and attempts to bind to iommufd repeatedly.Currently,
vfio_df_get_kvm_safe()is invoked prior to eachvfio_df_open()in the cdev path during iommufd binding. The correspondingvfio_device_put_kvm()is executed either when iommufd is unbound or if an error occurs during the binding process.
Quoted 6 lines
However, issues arise when a device binds to iommufd more than once. The second
vfio_df_open()will fail during iommufd binding, andvfio_device_put_kvm()will be triggered, settingdevice->kvmto NULL. Consequently, when iommufd is unbound from the first successfully bound device,vfio_device_put_kvm()becomes ineffective, leading to a leak in the KVM reference count.
To be accurate this happens only when two binds are issued via different fds otherwise below check will happen earlier when two binds are in a same fd:
/* one device cannot be bound twice */
if (df->access_granted) {
ret = -EINVAL;
goto out_unlock;
}Quoted 71 lines
Below is the calltrace that will be produced in this scenario when the KVM module is unloaded afterwards, reporting "BUG
kvm_vcpu(Tainted: G S):
Objects remaining inkvm_vcpuon__kmem_cache_shutdown()".Call Trace: <TASK> dump_stack_lvl+0x80/0xc0 slab_err+0xb0/0xf0 ? __kmem_cache_shutdown+0xc1/0x4e0 ? rcu_is_watching+0x11/0x50 ? lock_acquired+0x144/0x3c0 __kmem_cache_shutdown+0x1b7/0x4e0 kmem_cache_destroy+0xa6/0x260 kvm_exit+0x80/0xc0 [kvm] vmx_exit+0xe/0x20 [kvm_intel] __x64_sys_delete_module+0x143/0x250 ? ktime_get_coarse_real_ts64+0xd3/0xe0 ? syscall_trace_enter+0x143/0x210 do_syscall_64+0x6f/0x140 entry_SYSCALL_64_after_hwframe+0x76/0x7eFixes 5fcc26969a16 ("vfio: Add VFIO_DEVICE_BIND_IOMMUFD")Signed-off-by Yan Zhao <[email protected]>--- drivers/vfio/device_cdev.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)drivers/vfio/device_cdev.c +7
-3@@ -65,6 +65,7 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, {struct vfio_device *device = df->device;struct vfio_device_bind_iommufd bind; +bool put_kvm = false;unsigned long minsz;int ret;@@ -101,12 +102,15 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, }/* -* Before the device open, get the KVM pointer currently +* Before the device's first open, get the KVM pointer currently * associated with the device file (if there is) and obtain -* a reference. This reference is held until device closed. +* a reference. This reference is held until device's last closed. * Save the pointer in the device for use by drivers. */ -vfio_df_get_kvm_safe (df); +if (device->open_count ==0 ) { +vfio_df_get_kvm_safe (df); + put_kvm = true; + } ret =vfio_df_open (df);drivers/vfio/device_cdev.c +2
-1if (ret)@@ -129,7 +133,8 @@ long vfio_df_ioctl_bind_iommufd(structvfio_device_file *df, out_close_device:vfio_df_close (df ); out_put_kvm: -vfio_device_put_kvm (device); +if (put_kvm) +vfio_device_put_kvm (device);iommufd_ctx_put (df->iommufd); df->iommufd =NULL ;
Quoted 2 lines
out_unlock:
what about extending vfio_df_open() to unify the get/put_kvm() and open_count trick in one place?
int vfio_df_open(struct vfio_device_file *df, struct kvm *kvm,
spinlock_t *kvm_ref_lock)
{
struct vfio_device *device = df->device;
int ret = 0;
lockdep_assert_held(&device->dev_set->lock);
if (device->open_count == 0) {
spin_lock(kvm_ref_lock);
vfio_device_get_kvm_safe(device, kvm);
spin_unlock(kvm_ref_lock);
}
/*
* Only the group path allows the device to be opened multiple
* times. The device cdev path doesn't have a secure way for it.
*/
if (device->open_count != 0 && !df->group)
return -EINVAL;
device->open_count++;
if (device->open_count == 1) {
ret = vfio_df_device_first_open(df);
if (ret)
device->open_count--;
}
if (ret)
vfio_device_put_kvm(device);
return ret;
}void vfio_df_close(struct vfio_device_file *df)
{
struct vfio_device *device = df->device;
lockdep_assert_held(&device->dev_set->lock);
vfio_assert_device_open(device);
if (device->open_count == 1) {
vfio_df_device_last_close(df);
vfio_device_put_kvm(device);
}
device->open_count--;
}Quoted 25 lines · Kevin
Quoted 14 lines
From: Zhao, Yan Y <[email protected]>
Sent: Friday, June 28, 2024 11:19 PMIn the device cdev path, adjust the handling of the KVM reference count to only increment with the first
vfio_df_open()and decrement after the finalvfio_df_close(). This change addresses a KVM reference leak that occurs when a device cdev file is opened multiple times and attempts to bind to iommufd repeatedly.Currently,
vfio_df_get_kvm_safe()is invoked prior to eachvfio_df_open()in the cdev path during iommufd binding. The correspondingvfio_device_put_kvm()is executed either when iommufd is unbound or if an error occurs during the binding process.Quoted 6 lines
However, issues arise when a device binds to iommufd more than once. The second
vfio_df_open()will fail during iommufd binding, andvfio_device_put_kvm()will be triggered, settingdevice->kvmto NULL. Consequently, when iommufd is unbound from the first successfully bound device,vfio_device_put_kvm()becomes ineffective, leading to a leak in the KVM reference count.To be accurate this happens only when two binds are issued via different fds otherwise below check will happen earlier when two binds are in a same fd:
Quoted 5 lines
/* one device cannot be bound twice */
if (df->access_granted) {
ret = -EINVAL;
goto out_unlock;
}yes
Quoted 77 lines
Quoted 71 lines
Below is the calltrace that will be produced in this scenario when the KVM module is unloaded afterwards, reporting "BUG
kvm_vcpu(Tainted: G S):
Objects remaining inkvm_vcpuon__kmem_cache_shutdown()".Call Trace: <TASK> dump_stack_lvl+0x80/0xc0 slab_err+0xb0/0xf0 ? __kmem_cache_shutdown+0xc1/0x4e0 ? rcu_is_watching+0x11/0x50 ? lock_acquired+0x144/0x3c0 __kmem_cache_shutdown+0x1b7/0x4e0 kmem_cache_destroy+0xa6/0x260 kvm_exit+0x80/0xc0 [kvm] vmx_exit+0xe/0x20 [kvm_intel] __x64_sys_delete_module+0x143/0x250 ? ktime_get_coarse_real_ts64+0xd3/0xe0 ? syscall_trace_enter+0x143/0x210 do_syscall_64+0x6f/0x140 entry_SYSCALL_64_after_hwframe+0x76/0x7eFixes 5fcc26969a16 ("vfio: Add VFIO_DEVICE_BIND_IOMMUFD")Signed-off-by Yan Zhao <[email protected]>--- drivers/vfio/device_cdev.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)drivers/vfio/device_cdev.c +7
-3@@ -65,6 +65,7 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, {struct vfio_device *device = df->device;struct vfio_device_bind_iommufd bind; +bool put_kvm = false;unsigned long minsz;int ret;@@ -101,12 +102,15 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, }/* -* Before the device open, get the KVM pointer currently +* Before the device's first open, get the KVM pointer currently * associated with the device file (if there is) and obtain -* a reference. This reference is held until device closed. +* a reference. This reference is held until device's last closed. * Save the pointer in the device for use by drivers. */ -vfio_df_get_kvm_safe (df); +if (device->open_count ==0 ) { +vfio_df_get_kvm_safe (df); + put_kvm = true; + } ret =vfio_df_open (df);if (ret)
drivers/vfio/device_cdev.c +2
-1@@ -129,7 +133,8 @@ long vfio_df_ioctl_bind_iommufd(structvfio_device_file *df, out_close_device:vfio_df_close (df ); out_put_kvm: -vfio_device_put_kvm (device); +if (put_kvm) +vfio_device_put_kvm (device);iommufd_ctx_put (df->iommufd); df->iommufd =NULL ;Quoted 2 lines
out_unlock:what about extending
vfio_df_open()to unify the get/put_kvm() andopen_counttrick in one place?
Quoted 3 lines
int vfio_df_open(struct
vfio_device_file*df, struct kvm *kvm,
spinlock_t*kvm_ref_lock)
{
this should work. But need a comment to note why need pass in both kvm and kvm_ref_lock given df has both of them. :)
Quoted 10 lines
struct vfio_device *device = df->device;
int ret = 0;
lockdep_assert_held(&device->dev_set->lock);
if (device->open_count == 0) {
spin_lock(kvm_ref_lock);
vfio_device_get_kvm_safe(device, kvm);
spin_unlock(kvm_ref_lock);
}perhaps it can be put in the "if (device->open_count == 1)" branch, just before invoking vfio_df_device_first_open().
Quoted 33 lines
/* * Only the group path allows the device to be opened multiple * times. The device cdev path doesn't have a secure way for it. */ if (device->open_count != 0 && !df->group) return -EINVAL; device->open_count++; if (device->open_count == 1) { ret = vfio_df_device_first_open(df); if (ret) device->open_count--; } if (ret) vfio_device_put_kvm(device); return ret; }void
vfio_df_close(struct vfio_device_file *df)
{
structvfio_device*device =df->device;lockdep_assert_held(&device->dev_set->lock); vfio_assert_device_open(device); if (device->open_count == 1) { vfio_df_device_last_close(df); vfio_device_put_kvm(device); } device->open_count--; }
Signature · 2 lines
Regards, Yi Liu
Quoted 111 lines · Yi Liu
Quoted 25 lines · Kevin
Quoted 14 lines
From: Zhao, Yan Y <[email protected]>
Sent: Friday, June 28, 2024 11:19 PMIn the device cdev path, adjust the handling of the KVM reference count to only increment with the first
vfio_df_open()and decrement after the finalvfio_df_close(). This change addresses a KVM reference leak that occurs when a device cdev file is opened multiple times and attempts to bind to iommufd repeatedly.Currently,
vfio_df_get_kvm_safe()is invoked prior to eachvfio_df_open()in the cdev path during iommufd binding. The correspondingvfio_device_put_kvm()is executed either when iommufd is unbound or if an error occurs during the binding process.Quoted 6 lines
However, issues arise when a device binds to iommufd more than once. The second
vfio_df_open()will fail during iommufd binding, andvfio_device_put_kvm()will be triggered, settingdevice->kvmto NULL. Consequently, when iommufd is unbound from the first successfully bound device,vfio_device_put_kvm()becomes ineffective, leading to a leak in the KVM reference count.To be accurate this happens only when two binds are issued via different fds otherwise below check will happen earlier when two binds are in a same fd:
Quoted 5 lines
/* one device cannot be bound twice */ if (df->access_granted) { ret = -EINVAL; goto out_unlock; }yes
Quoted 77 lines
Quoted 71 lines
Below is the calltrace that will be produced in this scenario when the KVM module is unloaded afterwards, reporting "BUG
kvm_vcpu(Tainted: G S):
Objects remaining inkvm_vcpuon__kmem_cache_shutdown()".Call Trace: <TASK> dump_stack_lvl+0x80/0xc0 slab_err+0xb0/0xf0 ? __kmem_cache_shutdown+0xc1/0x4e0 ? rcu_is_watching+0x11/0x50 ? lock_acquired+0x144/0x3c0 __kmem_cache_shutdown+0x1b7/0x4e0 kmem_cache_destroy+0xa6/0x260 kvm_exit+0x80/0xc0 [kvm] vmx_exit+0xe/0x20 [kvm_intel] __x64_sys_delete_module+0x143/0x250 ? ktime_get_coarse_real_ts64+0xd3/0xe0 ? syscall_trace_enter+0x143/0x210 do_syscall_64+0x6f/0x140 entry_SYSCALL_64_after_hwframe+0x76/0x7eFixes 5fcc26969a16 ("vfio: Add VFIO_DEVICE_BIND_IOMMUFD")Signed-off-by Yan Zhao <[email protected]>--- drivers/vfio/device_cdev.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)drivers/vfio/device_cdev.c +7
-3@@ -65,6 +65,7 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, {struct vfio_device *device = df->device;struct vfio_device_bind_iommufd bind; +bool put_kvm = false;unsigned long minsz;int ret;@@ -101,12 +102,15 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, }/* -* Before the device open, get the KVM pointer currently +* Before the device's first open, get the KVM pointer currently * associated with the device file (if there is) and obtain -* a reference. This reference is held until device closed. +* a reference. This reference is held until device's last closed. * Save the pointer in the device for use by drivers. */ -vfio_df_get_kvm_safe (df); +if (device->open_count ==0 ) { +vfio_df_get_kvm_safe (df); + put_kvm = true; + } ret =vfio_df_open (df);if (ret)
drivers/vfio/device_cdev.c +2
-1@@ -129,7 +133,8 @@ long vfio_df_ioctl_bind_iommufd(structvfio_device_file *df, out_close_device:vfio_df_close (df ); out_put_kvm: -vfio_device_put_kvm (device); +if (put_kvm) +vfio_device_put_kvm (device);iommufd_ctx_put (df->iommufd); df->iommufd =NULL ;Quoted 2 lines
out_unlock:what about extending
vfio_df_open()to unify the get/put_kvm() andopen_counttrick in one place?
Quoted 6 lines
Quoted 3 lines
int vfio_df_open(struct
vfio_device_file*df, struct kvm *kvm,
spinlock_t*kvm_ref_lock)
{this should work. But need a comment to note why need pass in both kvm and
kvm_ref_lockgiven df has both of them. :)
So why to pass them?
What about making vfio_device_group_get_kvm_safe() or vfio_df_get_kvm_safe() not static and calling one of them in vfio_df_open() according to the df->group ?
Quoted 6 lines
Quoted 6 lines
struct vfio_device *device = df->device; int ret = 0; lockdep_assert_held(&device->dev_set->lock);
Quoted 8 lines
Quoted 5 lines
if (device->open_count == 0) { spin_lock(kvm_ref_lock); vfio_device_get_kvm_safe(device, kvm); spin_unlock(kvm_ref_lock); }perhaps it can be put in the "if (
device->open_count== 1)" branch, just before invokingvfio_df_device_first_open().
What about just moving the get/put_kvm into vfio_df_device_first_open()?
Quoted 37 lines
Quoted 33 lines
/* * Only the group path allows the device to be opened multiple * times. The device cdev path doesn't have a secure way for it. */ if (device->open_count != 0 && !df->group) return -EINVAL; device->open_count++; if (device->open_count == 1) { ret = vfio_df_device_first_open(df); if (ret) device->open_count--; } if (ret) vfio_device_put_kvm(device); return ret; }void
vfio_df_close(struct vfio_device_file *df)
{
structvfio_device*device =df->device;lockdep_assert_held(&device->dev_set->lock); vfio_assert_device_open(device); if (device->open_count == 1) { vfio_df_device_last_close(df); vfio_device_put_kvm(device); } device->open_count--; }Signature · 2 lines
Regards, Yi Liu
Quoted 9 lines
From: Zhao, Yan Y <[email protected]>
Sent: Monday, July 1, 2024 8:02 PMQuoted 5 lines · Yi Liu
Quoted 4 lines · Kevin
what about extending
vfio_df_open()to unify the get/put_kvm() andopen_counttrick in one place?
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
int vfio_df_open(struct
vfio_device_file*df, struct kvm *kvm,
spinlock_t*kvm_ref_lock)
{this should work. But need a comment to note why need pass in both kvm and
kvm_ref_lockgiven df has both of them. :)So why to pass them?
hmm actually passing them is wrong especially for the group path. We have to get kvm upon the first reference to the pointer otherwise it is prone to use-after-free issue.
Quoted 7 lines
What about making
vfio_device_group_get_kvm_safe()orvfio_df_get_kvm_safe()
not static and calling one of them invfio_df_open()according to the df-Quoted 1 line
group
?
yeah, this sounds better.