Skip to contentSkip to conversation

Use of usb_find_interface in open is racy

PATCH In review Russ Dill 5 participants 12 messages Nov 17, 2009 – Nov 18, 2009 USB

lore
Share & tools
lore

Russ Dill

author
#0 lore raw

Many usb drivers that create character devices use "struct usb_class_driver", a set of fops, and a usb_find_interface in their open call. A prime example is drivers/usb/usb-skeleton.c. A race occurs when userspace receives a hotplug event for the addition for the interface and then opens the associated device file before the device is added to the driver's klist_devices.

The usb core senses a new usb device (usb_new_device) and calls device_add. This eventually gets down to really_probe and the usb-skeleton probe function, skel_probe. skel_probe calls usb_register_dev() which registers the associated character device for skel_class. The hotplug events for the class device get emitted.

User space receives the hotplug event for the class device, makes the device node and notifies another program that opens the device node. The program opens the device node which calls into usb_open and then skel_open. skel_open calls usb_find_interface. usb_find_interfaces searches the klist_devices of skel_driver, finds no device associated with the minor number and returns NULL. skel_open returns -ENODEV.

Control returns to really_probe and really_probe calls driver_bound which adds the device to the list of devices associated with skel_driver (klist_devices).

I'm not sure what the right way to solve this is. A call to wait_for_device_probe() in the skel_open call before calling usb_find_interface fixes the problem, but it is a rather large hammer.

#1 lore raw

[ linux-usb should be CCed here at least ]

Quoted 27 lines · Russ Dill

Many usb drivers that create character devices use "struct usb_class_driver", a set of fops, and a usb_find_interface in their open call. A prime example is drivers/usb/usb-skeleton.c. A race occurs when userspace receives a hotplug event for the addition for the interface and then opens the associated device file before the device is added to the driver's klist_devices.

The usb core senses a new usb device (usb_new_device) and calls device_add. This eventually gets down to really_probe and the usb-skeleton probe function, skel_probe. skel_probe calls usb_register_dev() which registers the associated character device for skel_class. The hotplug events for the class device get emitted.

User space receives the hotplug event for the class device, makes the device node and notifies another program that opens the device node. The program opens the device node which calls into usb_open and then skel_open. skel_open calls usb_find_interface. usb_find_interfaces searches the klist_devices of skel_driver, finds no device associated with the minor number and returns NULL. skel_open returns -ENODEV.

Control returns to really_probe and really_probe calls driver_bound which adds the device to the list of devices associated with skel_driver (klist_devices).

I'm not sure what the right way to solve this is. A call to wait_for_device_probe() in the skel_open call before calling usb_find_interface fixes the problem, but it is a rather large hammer.

Signature · 2 lines
Jiri Kosina
SUSE Labs, Novell Inc.
#2 lore raw

Am Mittwoch, 18. November 2009 11:41:25 schrieb Jiri Kosina:

Quoted 7 lines
Quoted 7 lines

User space receives the hotplug event for the class device, makes the device node and notifies another program that opens the device node. The program opens the device node which calls into usb_open and then skel_open. skel_open calls usb_find_interface. usb_find_interfaces searches the klist_devices of skel_driver, finds no device associated with the minor number and returns NULL. skel_open returns -ENODEV.

Quoted 8 lines
Quoted 7 lines

Control returns to really_probe and really_probe calls driver_bound which adds the device to the list of devices associated with skel_driver (klist_devices).

I'm not sure what the right way to solve this is. A call to wait_for_device_probe() in the skel_open call before calling usb_find_interface fixes the problem, but it is a rather large hammer.

Device core code is hard to follow, but I tried.
How about simply covering all of usb_register_dev() with minor_rwsem?

Regards
Oliver

#3 lore raw
Quoted 12 lines · Oliver Neukum

Am Mittwoch, 18. November 2009 11:41:25 schrieb Jiri Kosina:

Quoted 11 lines
Quoted 11 lines

User space receives the hotplug event for the class device, makes the device node and notifies another program that opens the device node. The program opens the device node which calls into usb_open and then skel_open. skel_open calls usb_find_interface. usb_find_interfaces searches the klist_devices of skel_driver, finds no device associated with the minor number and returns NULL. skel_open returns -ENODEV.

Control returns to really_probe and really_probe calls driver_bound which adds the device to the list of devices associated with skel_driver (klist_devices).

Quoted 7 lines
Quoted 4 lines
Quoted 3 lines

I'm not sure what the right way to solve this is. A call to wait_for_device_probe() in the skel_open call before calling usb_find_interface fixes the problem, but it is a rather large hammer.

Device core code is hard to follow, but I tried.
How about simply covering all of usb_register_dev() with minor_rwsem?

That won't help. The window is between the end of usb_register_dev() and the end of skel_probe(), during which time the mutex isn't held.

Alan Stern

author
#4 lore raw
Quoted 19 lines · Alan Stern
Quoted 17 lines · Oliver Neukum

Am Mittwoch, 18. November 2009 11:41:25 schrieb Jiri Kosina:

Quoted 15 lines
Quoted 14 lines

User space receives the hotplug event for the class device, makes the device node and notifies another program that opens the device node. The program opens the device node which calls into usb_open and then skel_open. skel_open calls usb_find_interface. usb_find_interfaces searches the klist_devices of skel_driver, finds no device associated with the minor number and returns NULL. skel_open returns -ENODEV.

Control returns to really_probe and really_probe calls driver_bound which adds the device to the list of devices associated with skel_driver (klist_devices).

I'm not sure what the right way to solve this is. A call to wait_for_device_probe() in the skel_open call before calling usb_find_interface fixes the problem, but it is a rather large hammer.

Quoted 6 lines
Quoted 2 lines

Device core code is hard to follow, but I tried.
How about simply covering all of usb_register_dev() with minor_rwsem?

That won't help.  The window is between the end of usb_register_dev() and the end of skel_probe(), during which time the mutex isn't held.

Like I said above, the window goes all the way till the device gets added to the klist_devices on the driver.

author
#5 lore raw
In reply to Oliver Neukum
Quoted 12 lines · Oliver Neukum

Am Mittwoch, 18. November 2009 11:41:25 schrieb Jiri Kosina:

Quoted 11 lines
Quoted 11 lines

User space receives the hotplug event for the class device, makes the device node and notifies another program that opens the device node. The program opens the device node which calls into usb_open and then skel_open. skel_open calls usb_find_interface. usb_find_interfaces searches the klist_devices of skel_driver, finds no device associated with the minor number and returns NULL. skel_open returns -ENODEV.

Control returns to really_probe and really_probe calls driver_bound which adds the device to the list of devices associated with skel_driver (klist_devices).

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines

I'm not sure what the right way to solve this is. A call to wait_for_device_probe() in the skel_open call before calling usb_find_interface fixes the problem, but it is a rather large hammer.

Device core code is hard to follow, but I tried.
How about simply covering all of usb_register_dev() with minor_rwsem?

The race is open from the point that usb_register_dev calls device_create (at the end of usb_register_dev) till the point that skel_probe returns up to the point of really_probe, really_probe calls driver_bound, and driver_bound adds the device to the driver's klist_devices.

call stack:

device_create
usb_register_dev
skel_probe
usb_probe_device (via drv->probe())
really_probe

really_probe:

dev->driver = drv;
if (driver_sysfs_add(dev)) {
[...]
}

if (dev->bus->probe) {
[...]
} else if (drv->probe) {

#6 lore raw
In reply to Jiri Kosina
Quoted 22 lines · Jiri Kosina
Quoted 20 lines · Russ Dill

Many usb drivers that create character devices use "struct usb_class_driver", a set of fops, and a usb_find_interface in their open call. A prime example is drivers/usb/usb-skeleton.c. A race occurs when userspace receives a hotplug event for the addition for the interface and then opens the associated device file before the device is added to the driver's klist_devices.

The usb core senses a new usb device (usb_new_device) and calls device_add. This eventually gets down to really_probe and the usb-skeleton probe function, skel_probe. skel_probe calls usb_register_dev() which registers the associated character device for skel_class. The hotplug events for the class device get emitted.

User space receives the hotplug event for the class device, makes the device node and notifies another program that opens the device node. The program opens the device node which calls into usb_open and then skel_open. skel_open calls usb_find_interface. usb_find_interfaces searches the klist_devices of skel_driver, finds no device associated with the minor number and returns NULL. skel_open returns -ENODEV.

Quoted 7 lines
Quoted 7 lines

Control returns to really_probe and really_probe calls driver_bound which adds the device to the list of devices associated with skel_driver (klist_devices).

I'm not sure what the right way to solve this is. A call to wait_for_device_probe() in the skel_open call before calling usb_find_interface fixes the problem, but it is a rather large hammer.

I think the proper answer is for usb_find_interface() to use bus_for_each_dev() instead of driver_for_each_device().

Alan Stern

#7 lore raw
Quoted 28 lines · Alan Stern
Quoted 26 lines · Jiri Kosina
Quoted 24 lines · Russ Dill

Many usb drivers that create character devices use "struct usb_class_driver", a set of fops, and a usb_find_interface in their open call. A prime example is drivers/usb/usb-skeleton.c. A race occurs when userspace receives a hotplug event for the addition for the interface and then opens the associated device file before the device is added to the driver's klist_devices.

The usb core senses a new usb device (usb_new_device) and calls device_add. This eventually gets down to really_probe and the usb-skeleton probe function, skel_probe. skel_probe calls usb_register_dev() which registers the associated character device for skel_class. The hotplug events for the class device get emitted.

User space receives the hotplug event for the class device, makes the device node and notifies another program that opens the device node. The program opens the device node which calls into usb_open and then skel_open. skel_open calls usb_find_interface. usb_find_interfaces searches the klist_devices of skel_driver, finds no device associated with the minor number and returns NULL. skel_open returns -ENODEV.

Control returns to really_probe and really_probe calls driver_bound which adds the device to the list of devices associated with skel_driver (klist_devices).

Quoted 6 lines
Quoted 3 lines
Quoted 3 lines

I'm not sure what the right way to solve this is. A call to wait_for_device_probe() in the skel_open call before calling usb_find_interface fixes the problem, but it is a rather large hammer.

I think the proper answer is for usb_find_interface() to use bus_for_each_dev() instead of driver_for_each_device().

Yeah, I think so, sorry about that I think this is my fault :(

I'm travelling all today, could someone make up a patch for this before Thursday if it's an issue?

Russ, have you ever hit this problem, or did you just find it by looking at the code?

thanks,

greg k-h

author
#8 lore raw
Quoted 36 lines · Greg KH
Quoted 28 lines · Alan Stern
Quoted 26 lines · Jiri Kosina
Quoted 24 lines · Russ Dill

Many usb drivers that create character devices use "struct usb_class_driver", a set of fops, and a usb_find_interface in their open call. A prime example is drivers/usb/usb-skeleton.c. A race occurs when userspace receives a hotplug event for the addition for the interface and then opens the associated device file before the device is added to the driver's klist_devices.

The usb core senses a new usb device (usb_new_device) and calls device_add. This eventually gets down to really_probe and the usb-skeleton probe function, skel_probe. skel_probe calls usb_register_dev() which registers the associated character device for skel_class. The hotplug events for the class device get emitted.

User space receives the hotplug event for the class device, makes the device node and notifies another program that opens the device node. The program opens the device node which calls into usb_open and then skel_open. skel_open calls usb_find_interface. usb_find_interfaces searches the klist_devices of skel_driver, finds no device associated with the minor number and returns NULL. skel_open returns -ENODEV.

Control returns to really_probe and really_probe calls driver_bound which adds the device to the list of devices associated with skel_driver (klist_devices).

Quoted 6 lines
Quoted 3 lines
Quoted 3 lines

I'm not sure what the right way to solve this is. A call to wait_for_device_probe() in the skel_open call before calling usb_find_interface fixes the problem, but it is a rather large hammer.

I think the proper answer is for usb_find_interface() to use bus_for_each_dev() instead of driver_for_each_device().

Quoted 8 lines

Yeah, I think so, sorry about that I think this is my fault :(

I'm travelling all today, could someone make up a patch for this before Thursday if it's an issue?

Russ, have you ever hit this problem, or did you just find it by looking at the code?

Yes, I'm utilizing devtmpfs to create device nodes and I have an app that handles hotplug events directly (no udev). Even with that small amount of overhead, the race does not occur every time, but it still occurs most of the time.

My initial fix was to add the device to the driver's device list sooner, but the change to usb_find_interface seems less intrusive and less likely to cause problems with other code that calls driver_for_each_device, driver_find_device, or uses BUS_NOTIFY_BOUND_DRIVER.

I'll work up a patch with the change to usb_find_interface, do some testing, and email it.

author
#9 lore raw

PATCH 1/1 Close usb_find_interface race

In reply to Greg KH

USB drivers that create character devices call usb_register_dev in their probe function. This associates the usb_interface device with that minor number and creates the character device and announces it to the world. However, the driver's probe function is called before the new usb_interface is added to the driver's klist_devices.

This is a problem because userspace will respond to the character device creation announcement by opening the character device. The driver's open function will the call usb_find_interface to find the usb_interface associated with that minor number. usb_find_interface will walk the driver's list of devices and find the usb_interface with the matching minor number.

Because the announcement happens before the usb_interface is added to the driver's klist_devices, a race condition exists. A straightforward fix is to walk the list of devices on usb_bus_type instead since the device is added to that list before the announcement occurs.

bus_find_device calls get_device to bump the reference count on the found device. It is arguable that the reference count should be dropped by the caller of usb_find_interface instead of usb_find_interface, however, the current users of usb_find_interface do not expect this.

---
 drivers/usb/core/usb.c |   30 +++++++++++-------------------
 1 files changed, 11 insertions(+), 19 deletions(-)
drivers/usb/core/usb.c +10 -18
index b1b85ab..d1e9440 100644
@@ -130,24 +130,17 @@ struct usb_host_interface *usb_altnum_to_altsetting(
 }
 EXPORT_SYMBOL_GPL(usb_altnum_to_altsetting);
 
-struct find_interface_arg {
-	int minor;
-	struct usb_interface *interface;
-};
-
 static int __find_interface(struct device *dev, void *data)
 {
-	struct find_interface_arg *arg = data;
+	int *minor = data;
 	struct usb_interface *intf;
 
 	if (!is_usb_interface(dev))
 		return 0;
 
 	intf = to_usb_interface(dev);
-	if (intf->minor != -1 && intf->minor == arg->minor) {
-		arg->interface = intf;
+	if (intf->minor != -1 && intf->minor == *minor)
 		return 1;
-	}
 	return 0;
 }
 
@@ -156,21 +149,20 @@ static int __find_interface(struct device *dev, void *data)
  * @drv: the driver whose current configuration is considered
  * @minor: the minor number of the desired device
  *
- * This walks the driver device list and returns a pointer to the interface
+ * This walks the bus device list and returns a pointer to the interface
  * with the matching minor.  Note, this only works for devices that share the
  * USB major number.
  */
 struct usb_interface *usb_find_interface(struct usb_driver *drv, int minor)
 {
-	struct find_interface_arg argb;
-	int retval;
+	struct device *dev;
+
+	dev = bus_find_device(&usb_bus_type, NULL, &minor, __find_interface);
+
+	/* Drop reference count from bus_find_device */
+	put_device(dev);
 
-	argb.minor = minor;
-	argb.interface = NULL;
-	/* eat the error, it will be in argb.interface */
-	retval = driver_for_each_device(&drv->drvwrap.driver, NULL, &argb,
-					__find_interface);
-	return argb.interface;
+	return dev ? to_usb_interface(dev) : NULL;
 }
 EXPORT_SYMBOL_GPL(usb_find_interface);
 
Signature · 1 line
1.6.5
#10 lore raw

Re: [PATCH] Close usb_find_interface race

Quoted 23 lines · Russ Dill

USB drivers that create character devices call usb_register_dev in their probe function. This associates the usb_interface device with that minor number and creates the character device and announces it to the world. However, the driver's probe function is called before the new usb_interface is added to the driver's klist_devices.

This is a problem because userspace will respond to the character device creation announcement by opening the character device. The driver's open function will the call usb_find_interface to find the usb_interface associated with that minor number. usb_find_interface will walk the driver's list of devices and find the usb_interface with the matching minor number.

Because the announcement happens before the usb_interface is added to the driver's klist_devices, a race condition exists. A straightforward fix is to walk the list of devices on usb_bus_type instead since the device is added to that list before the announcement occurs.

bus_find_device calls get_device to bump the reference count on the found device. It is arguable that the reference count should be dropped by the caller of usb_find_interface instead of usb_find_interface, however, the current users of usb_find_interface do not expect this.

Quoted 4 lines
---
 drivers/usb/core/usb.c |   30 +++++++++++-------------------
 1 files changed, 11 insertions(+), 19 deletions(-)

Looks good, thanks for finding and fixing this. I'll queue it up.

greg k-h

author
#11 lore raw
In reply to Alan Stern
Quoted 28 lines · Alan Stern
Quoted 26 lines · Jiri Kosina
Quoted 24 lines · Russ Dill

Many usb drivers that create character devices use "struct usb_class_driver", a set of fops, and a usb_find_interface in their open call. A prime example is drivers/usb/usb-skeleton.c. A race occurs when userspace receives a hotplug event for the addition for the interface and then opens the associated device file before the device is added to the driver's klist_devices.

The usb core senses a new usb device (usb_new_device) and calls device_add. This eventually gets down to really_probe and the usb-skeleton probe function, skel_probe. skel_probe calls usb_register_dev() which registers the associated character device for skel_class. The hotplug events for the class device get emitted.

User space receives the hotplug event for the class device, makes the device node and notifies another program that opens the device node. The program opens the device node which calls into usb_open and then skel_open. skel_open calls usb_find_interface. usb_find_interfaces searches the klist_devices of skel_driver, finds no device associated with the minor number and returns NULL. skel_open returns -ENODEV.

Control returns to really_probe and really_probe calls driver_bound which adds the device to the list of devices associated with skel_driver (klist_devices).

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

I'm not sure what the right way to solve this is. A call to wait_for_device_probe() in the skel_open call before calling usb_find_interface fixes the problem, but it is a rather large hammer.

I think the proper answer is for usb_find_interface() to use bus_for_each_dev() instead of driver_for_each_device().

This would work, bus_find_device would probably be a better match.