A bug fix to the error handling path in vfio_device_open()
Subsystems (2)
cover A bug fix to the error handling path in vfio_device_open()
The open_count is unnecessarily placed in the group_rwsem in the error handling path.
Yi Liu (1):
vfio: Move "device->open_count--" out of group_rwsem in
vfio_device_open()
drivers/vfio/vfio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Signature · 1 line
2.27.0
PATCH 1/1 vfio: Move "device->open_count--" out of group_rwsem in vfio_device_open()
No need to protect open_count with group_rwsem
--- drivers/vfio/vfio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/vfio/vfio.c +1 -1
@@ -1146,10 +1146,10 @@ static struct file *vfio_device_open(struct vfio_device *device)if (device->open_count ==1 && device->ops->close_device) device->ops->close_device (device); err_undo_count: +up_read (&device->group->group_rwsem); device->open_count--;if (device->open_count ==0 && device->kvm) device->kvm =NULL ; -up_read (&device->group->group_rwsem);mutex_unlock (&device->dev_set->lock);module_put (device->dev->driver->owner); err_unassign_container:
Signature · 1 line
2.27.0
Re: [Patch 1/1] vfio: Move "device->open_count--" out of group_rwsem in vfio_device_open()
Quoted 7 lines · Yi Liu
No need to protect
open_countwithgroup_rwsemFixes 421cfe6596f6 ("vfio: remove VFIO_GROUP_NOTIFY_SET_KVM")cc Matthew Rosato <[email protected]>cc Jason Gunthorpe <[email protected]>Signed-off-by Yi Liu <[email protected]>
Seems pretty harmless as-is, but you are correct group_rwsem can be dropped earlier; we do not protect the count with group_rwsem elsewhere (see vfio_device_fops_release as a comparison, where we already drop group_rwsem before open_count--)
FWIW, this change now also drops group_rswem before setting device->kvm = NULL, but that's also OK (again, just like vfio_device_fops_release) -- While the setting of device->kvm before open_device is technically done while holding the group_rwsem, this is done to protect the group kvm value we are copying from, and we should not be relying on that to protect the contents of device->kvm; instead we assume this value will not change until after the device is closed and while under the dev_set->lock.
Quoted 20 lines
--- drivers/vfio/vfio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/vfio/vfio.c +1
-1@@ -1146,10 +1146,10 @@ static struct file *vfio_device_open(struct vfio_device *device)if (device->open_count ==1 && device->ops->close_device) device->ops->close_device (device); err_undo_count: +up_read (&device->group->group_rwsem); device->open_count--;if (device->open_count ==0 && device->kvm) device->kvm =NULL ; -up_read (&device->group->group_rwsem);mutex_unlock (&device->dev_set->lock);module_put (device->dev->driver->owner); err_unassign_container:
Re: [Patch 1/1] vfio: Move "device->open_count--" out of group_rwsem in vfio_device_open()
Quoted 10 lines · Matthew Rosato
Quoted 7 lines · Yi Liu
No need to protect
open_countwithgroup_rwsemFixes 421cfe6596f6 ("vfio: remove VFIO_GROUP_NOTIFY_SET_KVM")cc Matthew Rosato <[email protected]>cc Jason Gunthorpe <[email protected]>Signed-off-by Yi Liu <[email protected]>Seems pretty harmless as-is, but you are correct
group_rwsemcan be dropped earlier; we do not protect the count withgroup_rwsemelsewhere (seevfio_device_fops_releaseas a comparison, where we already dropgroup_rwsembefore open_count--)
yes. this is exactly how I found it. Normally, I compare the err handling path with the release function to see if they are aligned. :-)
Quoted 1 line
FWIW, this change now also drops
group_rswembefore settingdevice->kvm= NULL, but that's also OK (again, just likevfio_device_fops_release) -- While the setting ofdevice->kvmbeforeopen_deviceis technically done while holding thegroup_rwsem, this is done to protect the group kvm value we are copying from, and we should not be relying on that to protect the contents ofdevice->kvm; instead we assume this value will not change until after the device is closed and while under thedev_set->lock.
yes. set device->kvm to be NULL has no need to hold group_rwsem. BTW. I
also doubt whether the device->ops->open_device(device) and device->ops->close_device(device) should be protected by group_rwsem or not. seems not, right? group_rwsem protects the fields under vfio_group.
For the open_device/close_device() device->dev_set->lock is enough. Maybe
another nit fix.
Quoted 24 lines
Reviewed-by Matthew Rosato <[email protected]>Quoted 21 lines
---
drivers/vfio/vfio.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)drivers/vfio/vfio.c +0
-0@@ -1146,10 +1146,10 @@ static struct file *vfio_device_open(struct vfio_device *device) if (device->open_count == 1 && device->ops->close_device) device->ops->close_device(device);
err_undo_count:
+up_read(&device->group->group_rwsem);
device->open_count--;
if (device->open_count== 0 &&device->kvm)
device->kvm= NULL;
-up_read(&device->group->group_rwsem);
mutex_unlock(&device->dev_set->lock);
module_put(device->dev->driver->owner);
err_unassign_container:
Signature · 2 lines
Regards, Yi Liu
RE: [Patch 1/1] vfio: Move "device->open_count--" out of group_rwsem in vfio_device_open()
Quoted 14 lines
From: Liu, Yi L <[email protected]>
Sent: Tuesday, June 21, 2022 9:32 AMQuoted 10 lines · Matthew Rosato
Quoted 7 lines · Yi Liu
No need to protect
open_countwithgroup_rwsemFixes 421cfe6596f6 ("vfio: remove VFIO_GROUP_NOTIFY_SET_KVM")cc Matthew Rosato <[email protected]>cc Jason Gunthorpe <[email protected]>Signed-off-by Yi Liu <[email protected]>Seems pretty harmless as-is, but you are correct
group_rwsemcan be
Quoted 8 lines
dropped
Quoted 2 lines
earlier; we do not protect the count with
group_rwsemelsewhere (seevfio_device_fops_releaseas a comparison, where we already drop
group_rwsemQuoted 1 line
before open_count--)
yes. this is exactly how I found it. Normally, I compare the err handling path with the release function to see if they are aligned. :-)
In this case we don't need a FIX tag. It's kind of optimization.
Quoted 11 lines
Quoted 6 lines
FWIW, this change now also drops
group_rswembefore setting device- kvm =
NULL, but that's also OK (again, just likevfio_device_fops_release) -- While the setting ofdevice->kvmbeforeopen_deviceis technically done while holding thegroup_rwsem, this is done to protect the group kvmvalue
Quoted 3 lines
we are copying from, and we should not be relying on that to protect the contents of
device->kvm; instead we assume this value will not change until after the device is closed and while under thedev_set->lock.
Quoted 7 lines
yes. set
device->kvmto be NULL has no need to holdgroup_rwsem. BTW. I also doubt whether thedevice->ops->open_device(device)anddevice->ops->close_device(device)should be protected bygroup_rwsemor not. seems not, right?group_rwsemprotects the fields undervfio_group. For the open_device/close_device()device->dev_set->lockis enough. Maybe another nit fix.
group->rwsem is to protect device->group->kvm from being changed by vfio_file_set_kvm() before it is copied to device->kvm.
Re: [Patch 1/1] vfio: Move "device->open_count--" out of group_rwsem in vfio_device_open()
Quoted 17 lines · Kevin
Quoted 17 lines
From: Liu, Yi L <[email protected]>
Sent: Tuesday, June 21, 2022 9:32 AMQuoted 10 lines · Matthew Rosato
Quoted 7 lines · Yi Liu
No need to protect
open_countwithgroup_rwsemFixes 421cfe6596f6 ("vfio: remove VFIO_GROUP_NOTIFY_SET_KVM")cc Matthew Rosato <[email protected]>cc Jason Gunthorpe <[email protected]>Signed-off-by Yi Liu <[email protected]>Seems pretty harmless as-is, but you are correct
group_rwsemcan bedropped
Quoted 2 lines
earlier; we do not protect the count with
group_rwsemelsewhere (seevfio_device_fops_releaseas a comparison, where we already drop
Quoted 7 lines
Quoted 5 lines
group_rwsemQuoted 1 line
before open_count--)
yes. this is exactly how I found it. Normally, I compare the err handling path with the release function to see if they are aligned. :-)
In this case we don't need a FIX tag. It's kind of optimization.
ok.
Quoted 19 lines
Quoted 11 lines
Quoted 6 lines
FWIW, this change now also drops
group_rswembefore setting device- kvm =
NULL, but that's also OK (again, just likevfio_device_fops_release) -- While the setting ofdevice->kvmbeforeopen_deviceis technically done while holding thegroup_rwsem, this is done to protect the group kvmvalue
Quoted 3 lines
we are copying from, and we should not be relying on that to protect the contents of
device->kvm; instead we assume this value will not change until after the device is closed and while under thedev_set->lock.Quoted 7 lines
yes. set
device->kvmto be NULL has no need to holdgroup_rwsem. BTW. I also doubt whether thedevice->ops->open_device(device)anddevice->ops->close_device(device)should be protected bygroup_rwsemor not. seems not, right?group_rwsemprotects the fields undervfio_group. For the open_device/close_device()device->dev_set->lockis enough. Maybe another nit fix.
Quoted 2 lines
group->rwsemis to protectdevice->group->kvmfrom being changed byvfio_file_set_kvm()before it is copied todevice->kvm.
yes. this is why vfio_device_open() holds the read lock of group_rwsem around the device->group->kvm copy. However, for the open_device(), callback, I don't think it is necessary to be protected by the group_rwsem lock.
Signature · 2 lines
Regards, Yi Liu
RE: [Patch 1/1] vfio: Move "device->open_count--" out of group_rwsem in vfio_device_open()
Quoted 25 lines
From: Liu, Yi L <[email protected]>
Sent: Tuesday, June 21, 2022 10:59 AMQuoted 9 lines
Quoted 9 lines
Quoted 6 lines
FWIW, this change now also drops
group_rswembefore setting device- kvm =
NULL, but that's also OK (again, just likevfio_device_fops_release) -- While the setting ofdevice->kvmbeforeopen_deviceis technically done while holding thegroup_rwsem, this is done to protect the group kvmvalue
Quoted 2 lines
we are copying from, and we should not be relying on that to protect the contents of
device->kvm; instead we assume this value will not changeuntil
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
after the device is closed and while under the
dev_set->lock.yes. set
device->kvmto be NULL has no need to holdgroup_rwsem. BTW.I
Quoted 2 lines
Quoted 2 lines
also doubt whether the
device->ops->open_device(device)anddevice->ops->close_device(device)should be protected bygroup_rwsemor
Quoted 2 lines
Quoted 2 lines
not. seems not, right?
group_rwsemprotects the fields undervfio_group. For the open_device/close_device()device->dev_set->lockis enough.Maybe
Quoted 3 lines
Quoted 2 lines
another nit fix.
Quoted 8 lines
Quoted 2 lines
group->rwsemis to protectdevice->group->kvmfrom being changed byvfio_file_set_kvm()before it is copied todevice->kvm.yes. this is why
vfio_device_open()holds the read lock ofgroup_rwsemaround thedevice->group->kvmcopy. However, for theopen_device(), callback, I don't think it is necessary to be protected by thegroup_rwsemlock.
The kvm object could be freed after device->kvm is set, if group_rwsem is not held when calling open_device(). Then you'll hit another use-after-free bug when mdev driver tries to obtain a reference on kvm.
Re: [Patch 1/1] vfio: Move "device->open_count--" out of group_rwsem in vfio_device_open()
Quoted 34 lines · Kevin
Quoted 25 lines
From: Liu, Yi L <[email protected]>
Sent: Tuesday, June 21, 2022 10:59 AMQuoted 9 lines
Quoted 9 lines
Quoted 6 lines
FWIW, this change now also drops
group_rswembefore setting device- kvm =
NULL, but that's also OK (again, just likevfio_device_fops_release) -- While the setting ofdevice->kvmbeforeopen_deviceis technically done while holding thegroup_rwsem, this is done to protect the group kvmvalue
Quoted 2 lines
we are copying from, and we should not be relying on that to protect the contents of
device->kvm; instead we assume this value will not changeuntil
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
after the device is closed and while under the
dev_set->lock.yes. set
device->kvmto be NULL has no need to holdgroup_rwsem. BTW.I
Quoted 2 lines
Quoted 2 lines
also doubt whether the
device->ops->open_device(device)anddevice->ops->close_device(device)should be protected bygroup_rwsemor
Quoted 2 lines
Quoted 2 lines
not. seems not, right?
group_rwsemprotects the fields undervfio_group. For the open_device/close_device()device->dev_set->lockis enough.Maybe
Quoted 3 lines
Quoted 2 lines
another nit fix.
Quoted 8 lines
Quoted 2 lines
group->rwsemis to protectdevice->group->kvmfrom being changed byvfio_file_set_kvm()before it is copied todevice->kvm.yes. this is why
vfio_device_open()holds the read lock ofgroup_rwsemaround thedevice->group->kvmcopy. However, for theopen_device(), callback, I don't think it is necessary to be protected by thegroup_rwsemlock.
Quoted 4 lines
The kvm object could be freed after
device->kvmis set, ifgroup_rwsemis not held when callingopen_device(). Then you'll hit another use-after-free bug when mdev driver tries to obtain a reference on kvm.
aha. I see. so group_rwsem prevents the kvm object free as such a thread should be blocked in the vfio_file_set_kvm() if the vfio_device_open() has held the group_rwsem. Hence kvm object is safe to be used. thanks!
Signature · 2 lines
Regards, Yi Liu
Re: [Patch 1/1] vfio: Move "device->open_count--" out of group_rwsem in vfio_device_open()
In reply to Yi LiuQuoted 1 line · Yi Liu
No need to protect
open_countwithgroup_rwsem
You should explain why this is a good change as you did in the emails here, and no cover letter required for single patches
Quoted 2 lines
Fixes 421cfe6596f6 ("vfio: remove VFIO_GROUP_NOTIFY_SET_KVM")
Extra space after fixes line.
No Fixes line for things that are not bugs.
Quoted 2 lines
cc Matthew Rosato <[email protected]>cc Jason Gunthorpe <[email protected]>
Cc:
Quoted 4 lines
Signed-off-by Yi Liu <[email protected]>--- drivers/vfio/vfio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
It seems OK but a bit unnecessary.
Jason
Re: [Patch 1/1] vfio: Move "device->open_count--" out of group_rwsem in vfio_device_open()
Quoted 15 lines · Jason Gunthorpe
Quoted 1 line · Yi Liu
No need to protect
open_countwithgroup_rwsemYou should explain why this is a good change as you did in the emails here, and no cover letter required for single patches
Quoted 2 lines
Fixes 421cfe6596f6 ("vfio: remove VFIO_GROUP_NOTIFY_SET_KVM")Extra space after fixes line.
No Fixes line for things that are not bugs.Quoted 2 lines
cc Matthew Rosato <[email protected]>cc Jason Gunthorpe <[email protected]>
Quoted 9 lines
Cc:
Quoted 4 lines
Signed-off-by Yi Liu <[email protected]>--- drivers/vfio/vfio.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)It seems OK but a bit unnecessary.
thanks, Jason. all good suggestions.I'd just want to see the device->dev_set->lock and the group_rwsem hold/release is symmetric in the case of err_undo_count is hit in the vfio_device_open(). Just like the order in vfio_device_fops_release().
Signature · 2 lines
Regards, Yi Liu