i2c: fortify the subsystem against user-space induced deadlocks
Subsystems (2)
v2 cover i2c: fortify the subsystem against user-space induced deadlocks
From: Bartosz Golaszewski <[email protected]>
Several subsystems in the kernel that export device files to user-space suffer from a bug where keeping an open file descriptor associated with this device file, unbinding the device from its driver and then calling any of the supported system calls on that file descriptor will result in either a crash or - as is the case with i2c - a deadlock.
This behavior has been blamed on extensive usage of device resource management interfaces but it seems that devres has nothing to do with it, the problem would be the same whether using devres or freeing resources in .remove() that should survive the driver detach.
Many subsystems already deal with this by implementing some kind of flags in the character device data together with locking preventing the user-space from dropping the subsystem data from under the open device.
In i2c the deadlock comes from the fact that the function unregistering the adapter waits for a completion which will not be passed until all references to the character device are dropped.
The first patch in this series is just a tweak of return values of the notifier callback. The second addresses the deadlock problem in a way similar to how we fixed this issue in the GPIO subystem. Details are in the commit message.
v1 -> v2:
- keep the device release callback and use it to free the IDR number
- rebase on top of v6.2-rc1
Bartosz Golaszewski (2):
i2c: dev: fix notifier return values
i2c: dev: don't allow user-space to deadlock the kernel
drivers/i2c/i2c-core-base.c | 26 ++------- drivers/i2c/i2c-dev.c | 112 +++++++++++++++++++++++++++++------- include/linux/i2c.h | 2 - 3 files changed, 96 insertions(+), 44 deletions(-)
Signature · 1 line
2.37.2
PATCH v2 1/2 i2c: dev: fix notifier return values
From: Bartosz Golaszewski <[email protected]>
We have a set of return values that notifier callbacks can return. They should not return 0, error codes or anything other than those predefined values. Make the i2c character device's callback return NOTIFY_DONE or NOTIFY_OK depending on the situation.
--- drivers/i2c/i2c-dev.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
drivers/i2c/i2c-dev.c +8 -8
@@ -653,12 +653,12 @@ static int i2cdev_attach_adapter(struct device *dev, void *dummy)int res;if (dev->type != &i2c_adapter_type) -return 0 ; +return NOTIFY_DONE ; adap =to_i2c_adapter (dev); i2c_dev =get_free_i2c_dev (adap);if (IS_ERR (i2c_dev)) -return PTR_ERR (i2c_dev); +return NOTIFY_DONE ;cdev_init (&i2c_dev->cdev, &i2cdev_fops); i2c_dev->cdev.owner =THIS_MODULE ;
@@ -678,11 +678,11 @@ static int i2cdev_attach_adapter(struct device *dev, void *dummy) goto err_put_i2c_dev;pr_debug ("adapter [%s] registered as minor %d\n" , adap->name, adap->nr); -return 0 ; +return NOTIFY_OK ; err_put_i2c_dev:put_i2c_dev (i2c_dev, false); -return res; +return NOTIFY_DONE ; }static int i2cdev_detach_adapter (struct device *dev,void *dummy)
@@ -691,17 +691,17 @@ static int i2cdev_detach_adapter(struct device *dev, void *dummy)struct i2c_dev *i2c_dev;if (dev->type != &i2c_adapter_type) -return 0 ; +return NOTIFY_DONE ; adap =to_i2c_adapter (dev); i2c_dev =i2c_dev_get_by_minor (adap->nr);if (!i2c_dev)/* attach_adapter must have failed */ -return 0 ; +return NOTIFY_DONE ;put_i2c_dev (i2c_dev, true);pr_debug ("adapter [%s] unregistered\n" , adap->name); -return 0 ; +return NOTIFY_OK ; } static int i2cdev_notifier_call(struct notifier_block *nb, unsigned long action,
@@ -716,7 +716,7 @@ static int i2cdev_notifier_call(struct notifier_block *nb, unsigned long action,return i2cdev_detach_adapter (dev,NULL ); } -return 0 ; +return NOTIFY_DONE ; }static struct notifier_block i2cdev_notifier = {
Signature · 1 line
2.37.2
Wolfram Sang
AppliedRe: [PATCH v2 1/2] i2c: dev: fix notifier return values
Quoted 8 lines · Bartosz Golaszewski
From: Bartosz Golaszewski <[email protected]>
We have a set of return values that notifier callbacks can return. They should not return 0, error codes or anything other than those predefined values. Make the i2c character device's callback return
NOTIFY_DONEorNOTIFY_OKdepending on the situation.Signed-off-by Bartosz Golaszewski <[email protected]>
Applied to for-next, thanks!
I start reviewing patch 2 now...
Re: [PATCH v2 1/2] i2c: dev: fix notifier return values
In reply to Bartosz GolaszewskiHi Bartosz,
Quoted 8 lines · Bartosz Golaszewski
From: Bartosz Golaszewski <[email protected]>
We have a set of return values that notifier callbacks can return. They should not return 0, error codes or anything other than those predefined values. Make the i2c character device's callback return
NOTIFY_DONEorNOTIFY_OKdepending on the situation.Signed-off-by Bartosz Golaszewski <[email protected]>
Thanks for your patch, which is now commit cddf70d0bce71c2a ("i2c:
dev: fix notifier return values") in v6.3-rc1.
On SH/R-Mobile platforms, this leads to missing /dev/i2c-* entries. On R-Car Gen4, they are still present, as all I2C adapters are initialized after i2cdev.
Quoted 18 lines
drivers/i2c/i2c-dev.c +2
-2@@ -653,12 +653,12 @@ static int i2cdev_attach_adapter(struct device *dev, void *dummy)int res;if (dev->type != &i2c_adapter_type) -return 0 ; +return NOTIFY_DONE ; adap =to_i2c_adapter (dev); i2c_dev =get_free_i2c_dev (adap);if (IS_ERR (i2c_dev)) -return PTR_ERR (i2c_dev); +return NOTIFY_DONE ;cdev_init (&i2c_dev->cdev, &i2cdev_fops); i2c_dev->cdev.owner =THIS_MODULE ;
drivers/i2c/i2c-dev.c static int i2cdev_attach_adapter(struct device *dev, void *dummy) · line 681
Quoted 5 lines
drivers/i2c/i2c-dev.c +1
-1@@ -678,11 +678,11 @@ static int i2cdev_attach_adapter(struct device *dev, void *dummy) goto err_put_i2c_dev;pr_debug ("adapter [%s] registered as minor %d\n" , adap->name, adap->nr); -return 0 ; +return NOTIFY_OK ;
Unfortunately i2cdev_{at,de}tach_adapter() are not only used as notifiers (called from i2cdev_notifier_call()), but also called from i2c_dev_init():
/* Bind to already existing adapters right away */
i2c_for_each_dev(NULL, i2cdev_attach_adapter);and i2c_dev_exit():
i2c_for_each_dev(NULL, i2cdev_detach_adapter);
As soon i2c_dev_{at,de}tach_adapter() returns a non-zero value (e.g. NOTIFY_OK), {i2c,bus}_for_each_dev() aborts processing.
In i2c_dev_init(), this leads to a failure in registering any already existing i2c adapters after the first one, causing missing /dev/i2c-* entries.
In i2c_dev_exit(), this leads to a failure unregistering any but the first i2c adapter.
As there is no one-to-one mapping from error codes to notify codes, I think this cannot just be handled inside i2cdev_notifier_call() :-(
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Re: [PATCH v2 1/2] i2c: dev: fix notifier return values
Quoted 63 lines · Geert Uytterhoeven
Hi Bartosz,
Quoted 8 lines · Bartosz Golaszewski
From: Bartosz Golaszewski <[email protected]>
We have a set of return values that notifier callbacks can return. They should not return 0, error codes or anything other than those predefined values. Make the i2c character device's callback return
NOTIFY_DONEorNOTIFY_OKdepending on the situation.Signed-off-by Bartosz Golaszewski <[email protected]>Thanks for your patch, which is now commit
cddf70d0bce71c2a("i2c:
dev: fix notifier return values") in v6.3-rc1.On SH/R-Mobile platforms, this leads to missing
/dev/i2c-*entries. On R-Car Gen4, they are still present, as all I2C adapters are initialized after i2cdev.Quoted 18 lines
drivers/i2c/i2c-dev.c +2
-2@@ -653,12 +653,12 @@ static int i2cdev_attach_adapter(struct device *dev, void *dummy)int res;if (dev->type != &i2c_adapter_type) -return 0 ; +return NOTIFY_DONE ; adap =to_i2c_adapter (dev); i2c_dev =get_free_i2c_dev (adap);if (IS_ERR (i2c_dev)) -return PTR_ERR (i2c_dev); +return NOTIFY_DONE ;cdev_init (&i2c_dev->cdev, &i2cdev_fops); i2c_dev->cdev.owner =THIS_MODULE ;drivers/i2c/i2c-dev.c static int i2cdev_attach_adapter(struct device *dev, void *dummy) · line 681
Quoted 5 lines
drivers/i2c/i2c-dev.c +1
-1@@ -678,11 +678,11 @@ static int i2cdev_attach_adapter(struct device *dev, void *dummy) goto err_put_i2c_dev;pr_debug ("adapter [%s] registered as minor %d\n" , adap->name, adap->nr); -return 0 ; +return NOTIFY_OK ;Unfortunately i2cdev_{at,de}tach_adapter() are not only used as notifiers (called from
i2cdev_notifier_call()), but also called fromi2c_dev_init():/* Bind to already existing adapters right away */ i2c_for_each_dev(NULL, i2cdev_attach_adapter);and
i2c_dev_exit():i2c_for_each_dev(NULL, i2cdev_detach_adapter);As soon i2c_dev_{at,de}tach_adapter() returns a non-zero value (e.g.
NOTIFY_OK), {i2c,bus}_for_each_dev() aborts processing.In
i2c_dev_init(), this leads to a failure in registering any already existing i2c adapters after the first one, causing missing/dev/i2c-*entries.
Quoted 6 lines
In
i2c_dev_exit(), this leads to a failure unregistering any but the first i2c adapter.As there is no one-to-one mapping from error codes to notify codes, I think this cannot just be handled inside
i2cdev_notifier_call():-(
Would wrapping i2c_a/detach_adapter() in a notifier callback work? So that SH can call it directly while notifiers would call it indirectly through the wrapper?
Bart
Re: [PATCH v2 1/2] i2c: dev: fix notifier return values
Hi Bartosz,
Quoted 64 lines · Bartosz Golaszewski
Quoted 63 lines · Geert Uytterhoeven
Quoted 8 lines · Bartosz Golaszewski
From: Bartosz Golaszewski <[email protected]>
We have a set of return values that notifier callbacks can return. They should not return 0, error codes or anything other than those predefined values. Make the i2c character device's callback return
NOTIFY_DONEorNOTIFY_OKdepending on the situation.Signed-off-by Bartosz Golaszewski <[email protected]>Thanks for your patch, which is now commit
cddf70d0bce71c2a("i2c:
dev: fix notifier return values") in v6.3-rc1.On SH/R-Mobile platforms, this leads to missing
/dev/i2c-*entries. On R-Car Gen4, they are still present, as all I2C adapters are initialized after i2cdev.Quoted 18 lines
drivers/i2c/i2c-dev.c +2
-2@@ -653,12 +653,12 @@ static int i2cdev_attach_adapter(struct device *dev, void *dummy)int res;if (dev->type != &i2c_adapter_type) -return 0 ; +return NOTIFY_DONE ; adap =to_i2c_adapter (dev); i2c_dev =get_free_i2c_dev (adap);if (IS_ERR (i2c_dev)) -return PTR_ERR (i2c_dev); +return NOTIFY_DONE ;cdev_init (&i2c_dev->cdev, &i2cdev_fops); i2c_dev->cdev.owner =THIS_MODULE ;drivers/i2c/i2c-dev.c static int i2cdev_attach_adapter(struct device *dev, void *dummy) · line 681
Quoted 5 lines
drivers/i2c/i2c-dev.c +1
-1@@ -678,11 +678,11 @@ static int i2cdev_attach_adapter(struct device *dev, void *dummy) goto err_put_i2c_dev;pr_debug ("adapter [%s] registered as minor %d\n" , adap->name, adap->nr); -return 0 ; +return NOTIFY_OK ;Unfortunately i2cdev_{at,de}tach_adapter() are not only used as notifiers (called from
i2cdev_notifier_call()), but also called fromi2c_dev_init():/* Bind to already existing adapters right away */ i2c_for_each_dev(NULL, i2cdev_attach_adapter);and
i2c_dev_exit():i2c_for_each_dev(NULL, i2cdev_detach_adapter);As soon i2c_dev_{at,de}tach_adapter() returns a non-zero value (e.g.
NOTIFY_OK), {i2c,bus}_for_each_dev() aborts processing.In
i2c_dev_init(), this leads to a failure in registering any already existing i2c adapters after the first one, causing missing/dev/i2c-*entries.In
i2c_dev_exit(), this leads to a failure unregistering any but the first i2c adapter.
Quoted 6 lines
Quoted 2 lines
As there is no one-to-one mapping from error codes to notify codes, I think this cannot just be handled inside
i2cdev_notifier_call():-(Would wrapping i2c_a/detach_adapter() in a notifier callback work? So that SH can call it directly while notifiers would call it indirectly through the wrapper?
That would be a wrapper that ignores the NOTIFY_* return value, and always returns zero? I.e. we can no longer return an error. I guess that's OK, as i2c_dev_init() doesn't take any action based on the returned error code anyway.
The only error conditions that can happen in i2c_attach_adapter() are:
- "Out of device minors" message in get_free_i2c_dev(),
- WARN_ON(dev == WHITEOUT_DEV) in cdev_add(),
- Generic -ENOMEM.
Looks like all of the above can be ignored, as they are all unlikely to happen, and there is nothing to be done to recover...
Note that this is not "called directly from SH".
The SH/R-Mobile SoCs where I noticed the issue are ARM32.
I guess it can happen on other platforms, too, depending on initialization order...
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Re: [PATCH v2 1/2] i2c: dev: fix notifier return values
Hi Bartosz,
Quoted 68 lines · Geert Uytterhoeven
Quoted 67 lines · Bartosz Golaszewski
Quoted 65 lines · Geert Uytterhoeven
Quoted 8 lines · Bartosz Golaszewski
From: Bartosz Golaszewski <[email protected]>
We have a set of return values that notifier callbacks can return. They should not return 0, error codes or anything other than those predefined values. Make the i2c character device's callback return
NOTIFY_DONEorNOTIFY_OKdepending on the situation.Signed-off-by Bartosz Golaszewski <[email protected]>Thanks for your patch, which is now commit
cddf70d0bce71c2a("i2c:
dev: fix notifier return values") in v6.3-rc1.On SH/R-Mobile platforms, this leads to missing
/dev/i2c-*entries. On R-Car Gen4, they are still present, as all I2C adapters are initialized after i2cdev.Quoted 18 lines
drivers/i2c/i2c-dev.c +2
-2@@ -653,12 +653,12 @@ static int i2cdev_attach_adapter(struct device *dev, void *dummy)int res;if (dev->type != &i2c_adapter_type) -return 0 ; +return NOTIFY_DONE ; adap =to_i2c_adapter (dev); i2c_dev =get_free_i2c_dev (adap);if (IS_ERR (i2c_dev)) -return PTR_ERR (i2c_dev); +return NOTIFY_DONE ;cdev_init (&i2c_dev->cdev, &i2cdev_fops); i2c_dev->cdev.owner =THIS_MODULE ;drivers/i2c/i2c-dev.c static int i2cdev_attach_adapter(struct device *dev, void *dummy) · line 681
Quoted 5 lines
drivers/i2c/i2c-dev.c +1
-1@@ -678,11 +678,11 @@ static int i2cdev_attach_adapter(struct device *dev, void *dummy) goto err_put_i2c_dev;pr_debug ("adapter [%s] registered as minor %d\n" , adap->name, adap->nr); -return 0 ; +return NOTIFY_OK ;Unfortunately i2cdev_{at,de}tach_adapter() are not only used as notifiers (called from
i2cdev_notifier_call()), but also called fromi2c_dev_init():/* Bind to already existing adapters right away */ i2c_for_each_dev(NULL, i2cdev_attach_adapter);and
i2c_dev_exit():i2c_for_each_dev(NULL, i2cdev_detach_adapter);As soon i2c_dev_{at,de}tach_adapter() returns a non-zero value (e.g.
NOTIFY_OK), {i2c,bus}_for_each_dev() aborts processing.In
i2c_dev_init(), this leads to a failure in registering any already existing i2c adapters after the first one, causing missing/dev/i2c-*entries.In
i2c_dev_exit(), this leads to a failure unregistering any but the first i2c adapter.As there is no one-to-one mapping from error codes to notify codes, I think this cannot just be handled inside
i2cdev_notifier_call():-(
Quoted 8 lines
Quoted 3 lines
Would wrapping i2c_a/detach_adapter() in a notifier callback work? So that SH can call it directly while notifiers would call it indirectly through the wrapper?
That would be a wrapper that ignores the NOTIFY_* return value, and always returns zero? I.e. we can no longer return an error. I guess that's OK, as
i2c_dev_init()doesn't take any action based on the returned error code anyway.
This works, so I've sent a fix
https://lore.kernel.org/r/03a8cd13af352c4d990bc70b72df4915b9fa2874.1678347776.git.geert+renesas@glider.be
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
PATCH v2 2/2 i2c: dev: don't allow user-space to deadlock the kernel
In reply to Bartosz GolaszewskiFrom: Bartosz Golaszewski <[email protected]>
If we open an i2c character device and then unbind the underlying i2c adapter (either by unbinding it manually via sysfs or - for a real-life example - when unplugging a USB device with an i2c adaper), the kernel thread calling i2c_del_adapter() will become blocked waiting for the completion that only completes once all references to the character device get dropped.
In order to fix that, we introduce a couple changes. They need to be part of a single commit in order to preserve bisectability. First, drop the dev_release completion. That removes the risk of a deadlock but we now need to protect the character device structures against NULL pointer dereferences. To that end introduce an rw semaphore. It will protect the dummy i2c_client structure against dropping the adapter from under it. It will be taken for reading by all file_operations callbacks and for writing by the notifier's unbind handler. This way we don't prohibit the syscalls that don't get in each other's way from running concurrently but the adapter will not be unbound before all syscalls return.
Finally: upon being notified about an unbind event for the i2c adapter, we take the lock for writing and set the adapter pointer in the character device's structure to NULL. This "numbs down" the device - it still exists but is no longer functional. Meanwhile every syscall callback checks that pointer after taking the lock but before executing any code that requires it. If it's NULL, we return an error to user-space.
This way we can safely open an i2c device from user-space, unbind the device without triggering a deadlock and any subsequent system-call for the file descriptor associated with the removed adapter will gracefully fail.
--- drivers/i2c/i2c-core-base.c | 26 ++-------- drivers/i2c/i2c-dev.c | 96 ++++++++++++++++++++++++++++++++----- include/linux/i2c.h | 2 - 3 files changed, 88 insertions(+), 36 deletions(-)
drivers/i2c/i2c-core-base.c +5 -21
@@ -1139,7 +1139,11 @@ EXPORT_SYMBOL_GPL(i2c_new_ancillary_device);static void i2c_adapter_dev_release (struct device *dev) {struct i2c_adapter *adap =to_i2c_adapter (dev); -complete (&adap->dev_released); + +/* free bus id */ +mutex_lock (&core_lock); +idr_remove (&i2c_adapter_idr, adap->nr); +mutex_unlock (&core_lock); }unsigned int i2c_adapter_depth (struct i2c_adapter *adapter)
@@ -1512,9 +1516,7 @@ static int i2c_register_adapter(struct i2c_adapter *adap)return 0 ; out_reg: -init_completion (&adap->dev_released);device_unregister (&adap->dev); -wait_for_completion (&adap->dev_released); out_list:mutex_lock (&core_lock);idr_remove (&i2c_adapter_idr, adap->nr);
@@ -1713,25 +1715,7 @@ void i2c_del_adapter(struct i2c_adapter *adap)i2c_host_notify_irq_teardown (adap); -/* wait until all references to the device are gone -* -* FIXME: This is old code and should ideally be replaced by an -* alternative which results in decoupling the lifetime of the struct -* device from the i2c_adapter, like spi or netdev do. Any solution -* should be thoroughly tested with DEBUG_KOBJECT_RELEASE enabled! -*/ -init_completion (&adap->dev_released);device_unregister (&adap->dev); -wait_for_completion (&adap->dev_released); - -/* free bus id */ -mutex_lock (&core_lock); -idr_remove (&i2c_adapter_idr, adap->nr); -mutex_unlock (&core_lock); - -/* Clear the device structure in case this adapter is ever going to be -added again */ -memset (&adap->dev,0 ,sizeof (adap->dev)); }EXPORT_SYMBOL (i2c_del_adapter);
drivers/i2c/i2c-dev.c +83 -13
212 lines not on this page · raw message
include/linux/i2c.h +0 -2
@@ -14,7 +14,6 @@#include <linux/bits.h> #include <linux/mod_devicetable.h> #include <linux/device.h> /* for struct device */ -#include <linux/sched.h> /* for completion */ #include <linux/mutex.h> #include <linux/regulator/consumer.h> #include <linux/rtmutex.h>
@@ -739,7 +738,6 @@ struct i2c_adapter {int nr;char name[48 ]; -struct completion dev_released;struct mutex userspace_clients_lock;struct list_head userspace_clients;
Signature · 1 line
2.37.2
Re: [PATCH v2 2/2] i2c: dev: don't allow user-space to deadlock the kernel
Quoted 3 lines
In order to fix that, we introduce a couple changes. They need to be part of a single commit in order to preserve bisectability. First, drop
Do you have maybe a branch at hand with more finegrained commits just for reviewing?
Re: [PATCH v2 2/2] i2c: dev: don't allow user-space to deadlock the kernel
Quoted 8 lines · Wolfram Sang
Quoted 4 lines
In order to fix that, we introduce a couple changes. They need to be part of a single commit in order to preserve bisectability. First, drop
Do you have maybe a branch at hand with more finegrained commits just for reviewing?
No, not really. It can be logically split into two elements though:
removing the existing completion and then adding the locking bits which unfortunately go into many places. I tried to make the commit description as elaborate as possible but let me know if you have any specific questions.
Bart
Re: [PATCH v2 2/2] i2c: dev: don't allow user-space to deadlock the kernel
Quoted 2 lines
No, not really. It can be logically split into two elements though:
Ok, no worries.
Quoted 4 lines
removing the existing completion and then adding the locking bits which unfortunately go into many places. I tried to make the commit description as elaborate as possible but let me know if you have any specific questions.
Will do. The commit message _is_ nicely elaborate, still I need to dive into the topic thoroughly. I'll let you know how it goes.
Re: [PATCH v2 2/2] i2c: dev: don't allow user-space to deadlock the kernel
In reply to Bartosz GolaszewskiHi Bartosz,
Quoted 26 lines
If we open an i2c character device and then unbind the underlying i2c adapter (either by unbinding it manually via sysfs or - for a real-life example - when unplugging a USB device with an i2c adaper), the kernel thread calling
i2c_del_adapter()will become blocked waiting for the completion that only completes once all references to the character device get dropped.In order to fix that, we introduce a couple changes. They need to be part of a single commit in order to preserve bisectability. First, drop the
dev_releasecompletion. That removes the risk of a deadlock but we now need to protect the character device structures against NULL pointer dereferences. To that end introduce an rw semaphore. It will protect the dummyi2c_clientstructure against dropping the adapter from under it. It will be taken for reading by allfile_operationscallbacks and for writing by the notifier's unbind handler. This way we don't prohibit the syscalls that don't get in each other's way from running concurrently but the adapter will not be unbound before all syscalls return.Finally: upon being notified about an unbind event for the i2c adapter, we take the lock for writing and set the adapter pointer in the character device's structure to NULL. This "numbs down" the device - it still exists but is no longer functional. Meanwhile every syscall callback checks that pointer after taking the lock but before executing any code that requires it. If it's NULL, we return an error to user-space.
Quoted 6 lines
This way we can safely open an i2c device from user-space, unbind the device without triggering a deadlock and any subsequent system-call for the file descriptor associated with the removed adapter will gracefully fail.
Signed-off-by Bartosz Golaszewski <[email protected]>
First of all, thank you for tackling this problem. It was long overdue and I really appreciate that you took the initiative to get it solved.
Here are some review comments already. I'd like to do some more testing. So far, everything works nicely. Also, I'd like to invite more people to have a look at this code. We really don't want to have a regression here, so more eyes are very welcome.
drivers/i2c/i2c-core-base.c void i2c_del_adapter(struct i2c_adapter *adap) · line 1721
Quoted 10 lines
drivers/i2c/i2c-core-base.c +0
-6@@ -1713,25 +1715,7 @@ void i2c_del_adapter(struct i2c_adapter *adap)i2c_host_notify_irq_teardown (adap); -/* wait until all references to the device are gone -* -* FIXME: This is old code and should ideally be replaced by an -* alternative which results in decoupling the lifetime of the struct -* device from the i2c_adapter, like spi or netdev do. Any solution -* should be thoroughly tested with DEBUG_KOBJECT_RELEASE enabled!
Have you done this? Debugging with DEBUG_KOBJECT_RELEASE enabled?
drivers/i2c/i2c-core-base.c void i2c_del_adapter(struct i2c_adapter *adap) · line 1725
Quoted 4 lines
drivers/i2c/i2c-core-base.c +0
-3-*/ -init_completion (&adap->dev_released);device_unregister (&adap->dev); -wait_for_completion (&adap->dev_released);
So, this is basically the key change. I think you handled the userspace part via i2c-dev well. I don't have proof yet, but my gut feeling wonders if this is complete. Aren't there maybe sysfs-references as well. I need to check.
drivers/i2c/i2c-core-base.c void i2c_del_adapter(struct i2c_adapter *adap) · line 1734
Quoted 9 lines
drivers/i2c/i2c-core-base.c +0
-9- -/* free bus id */ -mutex_lock (&core_lock); -idr_remove (&i2c_adapter_idr, adap->nr); -mutex_unlock (&core_lock); - -/* Clear the device structure in case this adapter is ever going to be -added again */ -memset (&adap->dev,0 ,sizeof (adap->dev));
Any reason you didn't add this to release function above? Reading the introducing commit, the old drivers needings this still exist IMO. (Yes, they should be converted to use the i2c-mux subsystem, but I don't think someone will do this)
drivers/i2c/i2c-dev.c struct i2c_dev { · line 48
Quoted 5 lines
drivers/i2c/i2c-dev.c +1
-0@@ -44,8 +45,14 @@ struct i2c_dev {struct i2c_adapter *adap;struct device dev;struct cdev cdev; +struct rw_semaphore sem;
I'd like to name it 'rwsem' so it is always super-clear what it is.
drivers/i2c/i2c-dev.c struct i2c_dev { · line 51
Quoted 3 lines
drivers/i2c/i2c-dev.c +1
-0}; +static inline struct i2c_dev *to_i2c_dev (struct inode *ino)
I'd also prefer a more specific 'inode_to_i2c_dev' function name. Personally, I'd also name the argument 'inode' but I'll leave that to you.
drivers/i2c/i2c-dev.c struct i2c_dev { · line 54
Quoted 3 lines
drivers/i2c/i2c-dev.c +3
-0+{ + return container_of(ino->i_cdev, struct i2c_dev, cdev); +}
...
Doesn't the function 'name_show()' also need protection? It dereferences i2c_dev->adap.
So much for now,
Wolfram
Re: [PATCH v2 2/2] i2c: dev: don't allow user-space to deadlock the kernel
Quoted 49 lines · Wolfram Sang
Hi Bartosz,
Quoted 26 lines
If we open an i2c character device and then unbind the underlying i2c adapter (either by unbinding it manually via sysfs or - for a real-life example - when unplugging a USB device with an i2c adaper), the kernel thread calling
i2c_del_adapter()will become blocked waiting for the completion that only completes once all references to the character device get dropped.In order to fix that, we introduce a couple changes. They need to be part of a single commit in order to preserve bisectability. First, drop the
dev_releasecompletion. That removes the risk of a deadlock but we now need to protect the character device structures against NULL pointer dereferences. To that end introduce an rw semaphore. It will protect the dummyi2c_clientstructure against dropping the adapter from under it. It will be taken for reading by allfile_operationscallbacks and for writing by the notifier's unbind handler. This way we don't prohibit the syscalls that don't get in each other's way from running concurrently but the adapter will not be unbound before all syscalls return.Finally: upon being notified about an unbind event for the i2c adapter, we take the lock for writing and set the adapter pointer in the character device's structure to NULL. This "numbs down" the device - it still exists but is no longer functional. Meanwhile every syscall callback checks that pointer after taking the lock but before executing any code that requires it. If it's NULL, we return an error to user-space.
Quoted 6 lines
This way we can safely open an i2c device from user-space, unbind the device without triggering a deadlock and any subsequent system-call for the file descriptor associated with the removed adapter will gracefully fail.
Signed-off-by Bartosz Golaszewski <[email protected]>First of all, thank you for tackling this problem. It was long overdue and I really appreciate that you took the initiative to get it solved.
Here are some review comments already. I'd like to do some more testing. So far, everything works nicely. Also, I'd like to invite more people to have a look at this code. We really don't want to have a regression here, so more eyes are very welcome.
Quoted 4 lines
drivers/i2c/i2c-core-base.c +0
-0@@ -1713,25 +1715,7 @@ void i2c_del_adapter(struct i2c_adapter *adap)i2c_host_notify_irq_teardown (adap);
Quoted 9 lines
drivers/i2c/i2c-core-base.c void i2c_del_adapter(struct i2c_adapter *adap) · line 1721
Quoted 6 lines
drivers/i2c/i2c-core-base.c +0
-6-/* wait until all references to the device are gone -* -* FIXME: This is old code and should ideally be replaced by an -* alternative which results in decoupling the lifetime of the struct -* device from the i2c_adapter, like spi or netdev do. Any solution -* should be thoroughly tested with DEBUG_KOBJECT_RELEASE enabled! Have you done this? Debugging with
DEBUG_KOBJECT_RELEASEenabled?
Yes, I ran a simple test script with this option enabled - nothing suspicious reported.
Quoted 10 lines
drivers/i2c/i2c-core-base.c void i2c_del_adapter(struct i2c_adapter *adap) · line 1725
Quoted 4 lines
drivers/i2c/i2c-core-base.c +0
-3-*/ -init_completion (&adap->dev_released);device_unregister (&adap->dev); -wait_for_completion (&adap->dev_released);So, this is basically the key change. I think you handled the userspace part via i2c-dev well. I don't have proof yet, but my gut feeling wonders if this is complete. Aren't there maybe sysfs-references as well. I need to check.
Quoted 15 lines
drivers/i2c/i2c-core-base.c void i2c_del_adapter(struct i2c_adapter *adap) · line 1734
Quoted 9 lines
drivers/i2c/i2c-core-base.c +0
-9- -/* free bus id */ -mutex_lock (&core_lock); -idr_remove (&i2c_adapter_idr, adap->nr); -mutex_unlock (&core_lock); - -/* Clear the device structure in case this adapter is ever going to be -added again */ -memset (&adap->dev,0 ,sizeof (adap->dev));Any reason you didn't add this to release function above? Reading the introducing commit, the old drivers needings this still exist IMO. (Yes, they should be converted to use the i2c-mux subsystem, but I don't think someone will do this)
Good catch, added it back.
Quoted 16 lines
drivers/i2c/i2c-dev.c struct i2c_dev { · line 48
Quoted 5 lines
drivers/i2c/i2c-dev.c +1
-0@@ -44,8 +45,14 @@ struct i2c_dev {struct i2c_adapter *adap;struct device dev;struct cdev cdev; +struct rw_semaphore sem;I'd like to name it 'rwsem' so it is always super-clear what it is.
drivers/i2c/i2c-dev.c struct i2c_dev { · line 51
Quoted 3 lines
drivers/i2c/i2c-dev.c +1
-0}; +static inline struct i2c_dev *to_i2c_dev (struct inode *ino)I'd also prefer a more specific '
inode_to_i2c_dev' function name. Personally, I'd also name the argument 'inode' but I'll leave that to you.
Quoted 9 lines
drivers/i2c/i2c-dev.c struct i2c_dev { · line 54
Quoted 3 lines
drivers/i2c/i2c-dev.c +3
-0+{ + return container_of(ino->i_cdev, struct i2c_dev, cdev); +}...
Doesn't the function '
name_show()' also need protection? It dereferencesi2c_dev->adap.
Another good catch.
Quoted 4 lines
So much for now,
Wolfram
Thanks, will send a v2 shortly.
Bart
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH v2 2/2] i2c: dev: don't allow user-space to deadlock the kernel
Quoted 10 lines · Hillf Danton
On Thu, 29 Dec 2022 17:00:45 +0100 Bartosz Golaszewski <[email protected]>
drivers/i2c/i2c-dev.c struct i2c_dev { · line 48
Quoted 6 lines
drivers/i2c/i2c-dev.c +1
-0@@ -44,8 +45,14 @@ struct i2c_dev {struct i2c_adapter *adap;struct device dev;struct cdev cdev; +struct rw_semaphore sem;int state;
Quoted 7 lines
#define
STATE_RELEASED1
#defineSTATE_DETACHED2Review struct i2c_dev { · line 49
Quoted 1 line
(quoted diff) +0
-0};Because syscalls like read, write and ioctl have completed upon release, serializing the release and detach pathes is a cure to the issue.
Please elaborate because I have no idea what you mean by that.
Bart
Quoted 35 lines · Bartosz Golaszewski
From: Bartosz Golaszewski <[email protected]>
Several subsystems in the kernel that export device files to user-space suffer from a bug where keeping an open file descriptor associated with this device file, unbinding the device from its driver and then calling any of the supported system calls on that file descriptor will result in either a crash or - as is the case with i2c - a deadlock.
This behavior has been blamed on extensive usage of device resource management interfaces but it seems that devres has nothing to do with it, the problem would be the same whether using devres or freeing resources in .remove() that should survive the driver detach.
Many subsystems already deal with this by implementing some kind of flags in the character device data together with locking preventing the user-space from dropping the subsystem data from under the open device.
In i2c the deadlock comes from the fact that the function unregistering the adapter waits for a completion which will not be passed until all references to the character device are dropped.
The first patch in this series is just a tweak of return values of the notifier callback. The second addresses the deadlock problem in a way similar to how we fixed this issue in the GPIO subystem. Details are in the commit message.
v1 -> v2:
- keep the device release callback and use it to free the IDR number
- rebase on top of v6.2-rc1Bartosz Golaszewski (2):
i2c: dev: fix notifier return values
i2c: dev: don't allow user-space to deadlock the kernel
Quoted 8 lines
drivers/i2c/i2c-core-base.c | 26 ++------- drivers/i2c/i2c-dev.c | 112 +++++++++++++++++++++++++++++------- include/linux/i2c.h | 2 - 3 files changed, 96 insertions(+), 44 deletions(-)--
2.37.2
Hi Wolfram,
It's been two weeks without any comments on this series and over a month since v1 so let me send a gentle ping on this.
Bart
Hi Bart,
Quoted 2 lines
It's been two weeks without any comments on this series and over a month since v1 so let me send a gentle ping on this.
This series has a priority for the next merge window. I love it! However, patch 2 is difficult to review because I need to dive in deeply into the topic. Hopes are that I can do this next week.
Thanks and happy hacking,
Wolfram