slow-work: add (module*)work->owner to fix races with module clients
RFC PATCH 1/1 slow-work: add (module*)work->owner to fix races with module clients
(Applies to Linus' git master:626f380d)
Hi All,
I found this while working on KVM. I actually posted this patch with
a KVM
series yesterday and standalone earlier today, but neither seems to have made it to the lists. I suspect there is an issue with git-mail/postfix on my system.
I digress. This is a repost with the patch by itself, and rebased to Linus' tree instead of kvm.git. Apologies if the system finally corrects itself and the others show up.
Thoughts?
Regards,
-Greg
-----------------------------
slow-work: add (module*)work->owner to fix races with module clients
The slow_work facility was designed to use reference counting instead of barriers for synchronization. The reference counting mechanism is implemented as a vtable op (->get_ref, ->put_ref) callback. This is problematic for module use of the slow_work facility because it is impossible
to synchronize against the .text installed in the callbacks: There is no way to ensure that the slow-work threads have completely exited the .text in question and rmmod may yank it out from under the slow_work thread.
This patch attempts to address this issue by transparently mapping "struct module* owner" to the slow_work item, and maintaining a module reference count coincident with the more externally visible reference count. Since the slow_work facility is resident in kernel, it should be a race-free location to issue a module_put() call. This will ensure that modules can properly cleanup before exiting.
A module_get()/module_put() pair on slow_work_enqueue() and the subsequent dequeue technically adds the overhead of the atomic operations for every work item scheduled. However, slow_work is designed for deferring relatively long-running and/or sleepy tasks to begin with, so this overhead will hopefully be negligible.
---
include/linux/slow-work.h | 4 ++++ kernel/slow-work.c | 6 ++++++ 2 files changed, 10 insertions(+), 0 deletions(-)
include/linux/slow-work.h +4 -0
@@ -17,6 +17,7 @@#ifdef CONFIG_SLOW_WORK #include <linux/sysctl.h> +#include <linux/module.h> struct slow_work ;
@@ -42,6 +43,7 @@ struct slow_work_ops {* queued */ struct slow_work { +struct module *owner;unsigned long flags;#define SLOW_WORK_PENDING 0/* item pending (further) execution */ #define SLOW_WORK_EXECUTING 1/* item currently executing */
@@ -61,6 +63,7 @@ struct slow_work {static inline void slow_work_init (struct slow_work *work,const struct slow_work_ops *ops) { + work->owner =THIS_MODULE ; work->flags =0 ; work->ops = ops;INIT_LIST_HEAD (&work->link);
@@ -78,6 +81,7 @@ static inline void slow_work_init(struct slow_work *work,static inline void vslow_work_init (struct slow_work *work,const struct slow_work_ops *ops) { + work->owner =THIS_MODULE ; work->flags =1 <<SLOW_WORK_VERY_SLOW ; work->ops = ops;INIT_LIST_HEAD (&work->link);
kernel/slow-work.c +6 -0
@@ -220,6 +220,8 @@ static bool slow_work_execute(void) } work->ops->put_ref (work); +barrier ();/* ensure that put_ref is not re-ordered with module_put */ +module_put (work->owner);return true; auto_requeue:
@@ -299,6 +301,8 @@ int slow_work_enqueue(struct slow_work *work)if (test_bit (SLOW_WORK_EXECUTING , &work->flags)) {set_bit (SLOW_WORK_ENQ_DEFERRED , &work->flags); }else { +if (!try_module_get (work->owner)) + goto cant_get_mod;if (work->ops->get_ref (work) <0 ) goto cant_get_ref;if (test_bit (SLOW_WORK_VERY_SLOW , &work->flags))
@@ -313,6 +317,8 @@ int slow_work_enqueue(struct slow_work *work)return 0 ; cant_get_ref: +module_put (work->owner); +cant_get_mod:spin_unlock_irqrestore (&slow_work_queue_lock, flags);return -EAGAIN ; }
Quoted 93 lines · Gregory Haskins
(Applies to Linus' git master:626f380d)
Hi All,
I found this while working on KVM. I actually posted this patch with
a KVM
series yesterday and standalone earlier today, but neither seems to have made it to the lists. I suspect there is an issue with git-mail/postfix on my system.I digress. This is a repost with the patch by itself, and rebased to Linus' tree instead of kvm.git. Apologies if the system finally corrects itself and the others show up.
Thoughts?
Regards,
-Greg-----------------------------
slow-work: add (module*)work->owner to fix races with module clients
The
slow_workfacility was designed to use reference counting instead of barriers for synchronization. The reference counting mechanism is implemented as a vtable op (->get_ref, ->put_ref) callback. This is problematic for module use of theslow_workfacility because it is impossible
to synchronize against the .text installed in the callbacks: There is no way to ensure that the slow-work threads have completely exited the .text in question and rmmod may yank it out from under theslow_workthread.This patch attempts to address this issue by transparently mapping "struct module* owner" to the
slow_workitem, and maintaining a module reference count coincident with the more externally visible reference count. Since theslow_workfacility is resident in kernel, it should be a race-free location to issue amodule_put()call. This will ensure that modules can properly cleanup before exiting.A module_get()/module_put() pair on
slow_work_enqueue()and the subsequent dequeue technically adds the overhead of the atomic operations for every work item scheduled. However,slow_workis designed for deferring relatively long-running and/or sleepy tasks to begin with, so this overhead will hopefully be negligible.Signed-off-by Gregory Haskins <[email protected]>CC David Howells <[email protected]>---
include/linux/slow-work.h | 4 ++++ kernel/slow-work.c | 6 ++++++ 2 files changed, 10 insertions(+), 0 deletions(-)include/linux/slow-work.h +4
-0@@ -17,6 +17,7 @@#ifdef CONFIG_SLOW_WORK #include <linux/sysctl.h> +#include <linux/module.h> struct slow_work ;@@ -42,6 +43,7 @@ struct slow_work_ops {* queued */ struct slow_work { +struct module *owner;unsigned long flags;#define SLOW_WORK_PENDING 0/* item pending (further) execution */ #define SLOW_WORK_EXECUTING 1/* item currently executing */ @@ -61,6 +63,7 @@ struct slow_work {static inline void slow_work_init (struct slow_work *work,const struct slow_work_ops *ops) { + work->owner =THIS_MODULE ; work->flags =0 ; work->ops = ops;INIT_LIST_HEAD (&work->link);@@ -78,6 +81,7 @@ static inline void slow_work_init(struct slow_work *work,static inline void vslow_work_init (struct slow_work *work,const struct slow_work_ops *ops) { + work->owner =THIS_MODULE ; work->flags =1 <<SLOW_WORK_VERY_SLOW ; work->ops = ops;INIT_LIST_HEAD (&work->link);kernel/slow-work.c +0
-0
kernel/slow-work.c static bool slow_work_execute(void) · line 223
Quoted 4 lines
kernel/slow-work.c +0
-0@@ -220,6 +220,8 @@ static bool slow_work_execute(void) } work->ops->put_ref (work);
On this front: I also wonder if this put_ref is racing since we cannot guarantee pointer stability if
the object is kfree'd as a result of dropping the last ref. I do not know enough about compilers to say whether work or work->ops invalidation would cause problems with the call-return, but it seems dangerous at best. An alternative might be to copy the put_ref pointer prior to the call. Something like
slowwork_putref_t put_ref = work->ops->put_ref;
....
put_ref(work);might be better. However, I am not sure if it really matters so I did not address this issue yet.
-Greg
Quoted 26 lines
kernel/slow-work.c +2
-0+barrier ();/* ensure that put_ref is not re-ordered with module_put */ +module_put (work->owner);return true;
auto_requeue:kernel/slow-work.c +4
-0@@ -299,6 +301,8 @@ int slow_work_enqueue(struct slow_work *work)if (test_bit (SLOW_WORK_EXECUTING , &work->flags)) {set_bit (SLOW_WORK_ENQ_DEFERRED , &work->flags); }else { +if (!try_module_get (work->owner)) + goto cant_get_mod;if (work->ops->get_ref (work) <0 ) goto cant_get_ref;if (test_bit (SLOW_WORK_VERY_SLOW , &work->flags))@@ -313,6 +317,8 @@ int slow_work_enqueue(struct slow_work *work)return 0 ; cant_get_ref: +module_put (work->owner); +cant_get_mod:spin_unlock_irqrestore (&slow_work_queue_lock, flags);return -EAGAIN ; }
Quoted 109 lines · Gregory Haskins
Quoted 93 lines · Gregory Haskins
(Applies to Linus' git master:626f380d)
Hi All,
I found this while working on KVM. I actually posted this patch with
a KVM
series yesterday and standalone earlier today, but neither seems to have made it to the lists. I suspect there is an issue with git-mail/postfix on my system.I digress. This is a repost with the patch by itself, and rebased to Linus' tree instead of kvm.git. Apologies if the system finally corrects itself and the others show up.
Thoughts?
Regards,
-Greg-----------------------------
slow-work: add (module*)work->owner to fix races with module clients
The
slow_workfacility was designed to use reference counting instead of barriers for synchronization. The reference counting mechanism is implemented as a vtable op (->get_ref, ->put_ref) callback. This is problematic for module use of theslow_workfacility because it is impossible
to synchronize against the .text installed in the callbacks: There is no way to ensure that the slow-work threads have completely exited the .text in question and rmmod may yank it out from under theslow_workthread.This patch attempts to address this issue by transparently mapping "struct module* owner" to the
slow_workitem, and maintaining a module reference count coincident with the more externally visible reference count. Since theslow_workfacility is resident in kernel, it should be a race-free location to issue amodule_put()call. This will ensure that modules can properly cleanup before exiting.A module_get()/module_put() pair on
slow_work_enqueue()and the subsequent dequeue technically adds the overhead of the atomic operations for every work item scheduled. However,slow_workis designed for deferring relatively long-running and/or sleepy tasks to begin with, so this overhead will hopefully be negligible.Signed-off-by Gregory Haskins <[email protected]>CC David Howells <[email protected]>---
include/linux/slow-work.h | 4 ++++ kernel/slow-work.c | 6 ++++++ 2 files changed, 10 insertions(+), 0 deletions(-)include/linux/slow-work.h +4
-0@@ -17,6 +17,7 @@#ifdef CONFIG_SLOW_WORK #include <linux/sysctl.h> +#include <linux/module.h> struct slow_work ;@@ -42,6 +43,7 @@ struct slow_work_ops {* queued */ struct slow_work { +struct module *owner;unsigned long flags;#define SLOW_WORK_PENDING 0/* item pending (further) execution */ #define SLOW_WORK_EXECUTING 1/* item currently executing */ @@ -61,6 +63,7 @@ struct slow_work {static inline void slow_work_init (struct slow_work *work,const struct slow_work_ops *ops) { + work->owner =THIS_MODULE ; work->flags =0 ; work->ops = ops;INIT_LIST_HEAD (&work->link);@@ -78,6 +81,7 @@ static inline void slow_work_init(struct slow_work *work,static inline void vslow_work_init (struct slow_work *work,const struct slow_work_ops *ops) { + work->owner =THIS_MODULE ; work->flags =1 <<SLOW_WORK_VERY_SLOW ; work->ops = ops;INIT_LIST_HEAD (&work->link);kernel/slow-work.c +0
-0kernel/slow-work.c static bool slow_work_execute(void) · line 224
Quoted 5 lines
kernel/slow-work.c +0
-0@@ -220,6 +220,8 @@ static bool slow_work_execute(void) } work->ops->put_ref (work);On this front: I also wonder if this
put_refis racing since we cannot guarantee pointer stability if
the object is kfree'd as a result of dropping the last ref. I do not know enough about compilers to say whether work orwork->opsinvalidation would cause problems with the call-return, but it seems dangerous at best. An alternative might be to copy theput_refpointer prior to the call. Something like
Quoted 14 lines
slowwork_putref_t put_ref = work->ops->put_ref; .... put_ref(work);might be better. However, I am not sure if it really matters so I did not address this issue yet.
-Greg
Quoted 3 lines
kernel/slow-work.c +2
-0+barrier ();/* ensure that put_ref is not re-ordered with module_put */ +module_put (work->owner);
Ugg..speaking of using invalidated pointers! I need to cache "owner" here as well.
-Greg
Quoted 28 lines
Quoted 25 lines
return true;
auto_requeue:kernel/slow-work.c +4
-0@@ -299,6 +301,8 @@ int slow_work_enqueue(struct slow_work *work)if (test_bit (SLOW_WORK_EXECUTING , &work->flags)) {set_bit (SLOW_WORK_ENQ_DEFERRED , &work->flags); }else { +if (!try_module_get (work->owner)) + goto cant_get_mod;if (work->ops->get_ref (work) <0 ) goto cant_get_ref;if (test_bit (SLOW_WORK_VERY_SLOW , &work->flags))@@ -313,6 +317,8 @@ int slow_work_enqueue(struct slow_work *work)return 0 ; cant_get_ref: +module_put (work->owner); +cant_get_mod:spin_unlock_irqrestore (&slow_work_queue_lock, flags);return -EAGAIN ; }
Quoted 5 lines · Gregory Haskins
I found this while working on KVM. I actually posted this patch with
a KVM
series yesterday and standalone earlier today, but neither seems to have made it to the lists. I suspect there is an issue with git-mail/postfix on my system.
Also, your mail client has damaged the whitespace in the patch.
Quoted 2 lines
struct
slow_work{
+ struct module *owner;
Can you add it to slow_work_ops instead?
Quoted 4 lines
work->ops->put_ref(work);+
barrier(); /* ensure thatput_refis not re-ordered withmodule_put=
*/
+module_put(work->owner);
Ummm... Can it be? module_put() and put_ref() are both out of line - surely the compiler isn't allowed to reorder them? If it's the CPU doing it then barrier() isn't going to save you.
Note, however, that work may not be dereferenced like this after put_ref() is called, unless you're sure that there's still a reference outstanding.
kernel/slow-work.c int slow_work_enqueue(struct slow_work *work) · line 305
Quoted 2 lines
kernel/slow-work.c +2
-0+if (!try_module_get (work->owner)) + goto cant_get_mod;
Note that this may result in a module getting stuck in unloading. It may need to do some work to complete the unload, and this will prevent that.
A better way might be to have put_ref() return, say, a pointer to a completion struct, and if not NULL, have the caller of put_ref() call complete() on it. That way you don't need to touch the module count, but can have something in put_ref() keep track of when the last object is released and have its caller invoke a completion to celebrate this fact.
David
Quoted 12 lines · David Howells
Quoted 6 lines · Gregory Haskins
I found this while working on KVM. I actually posted this patch with
a KVM
series yesterday and standalone earlier today, but neither seems to have made it to the lists. I suspect there is an issue with git-mail/postfix on my system.Also, your mail client has damaged the whitespace in the patch.
Yeah, sorry about that. When git-mail was failing I cut-n-pasted into thunderbird and it munged it a bit. v2 should be better as it came out of git directly after I fixed the postfix misconfig.
Quoted 7 lines
Quoted 4 lines
struct
slow_work{
+ struct module *owner;Can you add it to
slow_work_opsinstead?
Yeah, that makes sense.
Quoted 11 lines
Quoted 6 lines
work->ops->put_ref(work);+
barrier(); /* ensure thatput_refis not re-ordered withmodule_put=
*/
+module_put(work->owner);Ummm... Can it be?
module_put()andput_ref()are both out of line - surely the compiler isn't allowed to reorder them? If it's the CPU doing it thenbarrier()isn't going to save you.
Good point. I added that at the last minute without engaging my brain. :) Will remove.
Quoted 4 lines
Note, however, that work may not be dereferenced like this after
put_ref()is called, unless you're sure that there's still a reference outstanding.
Yeah, I noticed that too immediately after sending. It should be better in v2 (which should be in your inbox already)
Quoted 7 lines
kernel/slow-work.c int slow_work_enqueue(struct slow_work *work) · line 305
Quoted 3 lines
kernel/slow-work.c +2
-0+if (!try_module_get (work->owner)) + goto cant_get_mod;Note that this may result in a module getting stuck in unloading. It may need to do some work to complete the unload, and this will prevent that.
Can we set the stake in the ground that you can only call slow_work_enqueue() from a module if you know that there is at least one reference to the module being held? This seems like a core requirement anyway.
The follow up question would be: if so, should we use __module_get() instead ot try_module_get() to annotate that (in addition to a comment, of course).
Quoted 6 lines
A better way might be to have
put_ref()return, say, a pointer to a completion struct, and if not NULL, have the caller ofput_ref()callcomplete()on it. That way you don't need to touch the module count, but can have something input_ref()keep track of when the last object is released and have its caller invoke a completion to celebrate this fact.
That sounds interesting, but I am not sure if we would get into a similar conundrum or be awkward to manage. I am in a conf-call ATM so I can't think clear enough to tell for sure. ;) Let me give it some thought and get back to you, though.
Thanks David!
-Greg