Fix ordering of ftrace/livepatch calls on module load and unload
Subsystems (3)
v3 cover Fix ordering of ftrace/livepatch calls on module load and unload
As explained here [1], livepatch modules are failing to initialize properly because the ftrace coming module notifier (which calls ftrace_module_enable()) runs *after* the livepatch module notifier (which enables the patch(es)). Thus livepatch attempts to apply patches to modules before ftrace_module_enable() is even called for the corresponding module(s). As a result, patch modules break. Ftrace code must run before livepatch on module load, and the reverse is true on module unload.
For ftrace and livepatch, order of initialization (plus exit/cleanup code) is important for loading and unloading modules, and using module notifiers to perform this work is not ideal since it is not always clear what gets called when. In this patchset, dependence on the module notifier call chain is removed in favor of hard coding the corresponding function calls in the module loader. This promotes better code visibility and ensures that ftrace and livepatch code get called in the correct order on patch module load and unload.
Tested the changes with a test livepatch module that patches 9p and nilfs2, and verified that the issue described in [1] is fixed.
Patches are based on linux-next.
v1 can be found here -
https://lkml.kernel.org/g/[email protected]
v2 can be found here -
https://lkml.kernel.org/g/[email protected]
v3:
- Fix incorrect comments
- Rename klp_module_{enable,disable} to klp_module_{coming,going}
- Remove externs from livepatch.h
- Fix error handling in kernel/module.c
v2:
- Instead of splitting the ftrace and livepatch notifiers into coming + going notifiers and adjusting their priorities, remove ftrace and livepatch notifiers completely and hard-code the necessary function calls in the module loader.
[1] http://lkml.kernel.org/g/[email protected]
Jessica Yu (2):
ftrace/module: remove ftrace module notifier livepatch/module: remove livepatch module notifier
include/linux/ftrace.h | 6 +- include/linux/livepatch.h | 9 +++ kernel/livepatch/core.c | 153 +++++++++++++++++++++++----------------------- kernel/module.c | 24 +++++++- kernel/trace/ftrace.c | 36 +---------- 5 files changed, 112 insertions(+), 116 deletions(-)
Signature · 1 line
2.4.3
PATCH v3 1/2 ftrace/module: remove ftrace module notifier
Remove the ftrace module notifier in favor of directly calling ftrace_module_enable() and ftrace_release_mod() in the module loader. Hard-coding the function calls directly in the module loader removes dependence on the module notifier call chain and provides better visibility and control over what gets called when, which is important to kernel utilities such as livepatch.
--- include/linux/ftrace.h | 6 ++++-- kernel/module.c | 4 ++++ kernel/trace/ftrace.c | 36 +----------------------------------- 3 files changed, 9 insertions(+), 37 deletions(-)
include/linux/ftrace.h +4 -2
@@ -603,6 +603,7 @@ extern int ftrace_arch_read_dyn_info(char *buf, int size);extern int skip_trace (unsigned long ip);extern void ftrace_module_init (struct module *mod); +extern void ftrace_module_enable (struct module *mod);extern void ftrace_release_mod (struct module *mod);extern void ftrace_disable_daemon (void );
@@ -612,8 +613,9 @@ static inline int skip_trace(unsigned long ip) { return 0; }static inline int ftrace_force_update (void ) {return 0 ; }static inline void ftrace_disable_daemon (void ) { }static inline void ftrace_enable_daemon (void ) { } -static inline void ftrace_release_mod (struct module *mod) {} -static inline void ftrace_module_init (struct module *mod) {} +static inline void ftrace_module_init (struct module *mod) { } +static inline void ftrace_module_enable (struct module *mod) { } +static inline void ftrace_release_mod (struct module *mod) { }static inline __init int register_ftrace_command (struct ftrace_func_command *cmd) {return -EINVAL ;
kernel/module.c +4 -0
@@ -984,6 +984,8 @@ SYSCALL_DEFINE2(delete_module, const char __user *, name_user, mod->exit ();blocking_notifier_call_chain (&module_notify_list,MODULE_STATE_GOING , mod); +ftrace_release_mod (mod); +async_synchronize_full ();/* Store the name of the last unloaded module for diagnostic purposes */
@@ -3313,6 +3315,7 @@ fail:module_put (mod);blocking_notifier_call_chain (&module_notify_list,MODULE_STATE_GOING , mod); +ftrace_release_mod (mod);free_module (mod);wake_up_all (&module_wq);return ret;
@@ -3389,6 +3392,7 @@ static int complete_formation(struct module *mod, struct load_info *info) mod->state =MODULE_STATE_COMING ;mutex_unlock (&module_mutex); +ftrace_module_enable (mod);blocking_notifier_call_chain (&module_notify_list,MODULE_STATE_COMING , mod);return 0 ;
kernel/trace/ftrace.c +1 -35
@@ -4961,7 +4961,7 @@ void ftrace_release_mod(struct module *mod)mutex_unlock (&ftrace_lock); } -static void ftrace_module_enable (struct module *mod) +void ftrace_module_enable (struct module *mod) {struct dyn_ftrace *rec;struct ftrace_page *pg;
@@ -5038,38 +5038,8 @@ void ftrace_module_init(struct module *mod)ftrace_process_locs (mod, mod->ftrace_callsites, mod->ftrace_callsites + mod->num_ftrace_callsites); } - -static int ftrace_module_notify (struct notifier_block *self, -unsigned long val,void *data) -{ -struct module *mod = data; - -switch (val) { -case MODULE_STATE_COMING : -ftrace_module_enable (mod); -break ; -case MODULE_STATE_GOING : -ftrace_release_mod (mod); -break ; -default : -break ; - } - -return 0 ; -} -#else -static int ftrace_module_notify (struct notifier_block *self, -unsigned long val,void *data) -{ -return 0 ; -}#endif /* CONFIG_MODULES */ -struct notifier_block ftrace_module_nb = { - .notifier_call = ftrace_module_notify, - .priority =INT_MIN ,/* Run after anything that can remove kprobes */ -}; -void __init ftrace_init (void ) {extern unsigned long __start_mcount_loc[];
@@ -5098,10 +5068,6 @@ void __init ftrace_init(void) __start_mcount_loc, __stop_mcount_loc); - ret =register_module_notifier (&ftrace_module_nb); -if (ret) -pr_warning ("Failed to register trace ftrace module exit notifier\n" ); -set_ftrace_early_filters ();return ;
Signature · 1 line
2.4.3
Re: [PATCH v3 1/2] ftrace/module: remove ftrace module notifier
Quoted 8 lines · Jessica Yu
Remove the ftrace module notifier in favor of directly calling
ftrace_module_enable()andftrace_release_mod()in the module loader. Hard-coding the function calls directly in the module loader removes dependence on the module notifier call chain and provides better visibility and control over what gets called when, which is important to kernel utilities such as livepatch.Signed-off-by Jessica Yu <[email protected]>
Best Regards,
Petr
Re: [PATCH v3 1/2] ftrace/module: remove ftrace module notifier
In reply to Jessica YuQuoted 8 lines · Jessica Yu
Remove the ftrace module notifier in favor of directly calling
ftrace_module_enable()andftrace_release_mod()in the module loader. Hard-coding the function calls directly in the module loader removes dependence on the module notifier call chain and provides better visibility and control over what gets called when, which is important to kernel utilities such as livepatch.Signed-off-by Jessica Yu <[email protected]>
This is also fixing the bug described in the cover letter, so it would be good to add a description of that here and a corresponding "Fixes:" tag.
Signature · 1 line
Josh
PATCH v3 2/2 livepatch/module: remove livepatch module notifier
In reply to Jessica YuRemove the livepatch module notifier in favor of directly enabling and disabling patches to modules in the module loader. Hard-coding the function calls ensures that ftrace_module_enable() is run before klp_module_coming() during module load, and that klp_module_going() is run before ftrace_release_mod() during module unload. This way, ftrace and livepatch code is run in the correct order during the module load/unload sequence without dependence on the module notifier call chain.
This fixes a notifier ordering issue in which the ftrace module notifier (and hence ftrace_module_enable()) for coming modules was being called after klp_module_notify(), which caused livepatch modules to initialize incorrectly.
--- include/linux/livepatch.h | 9 +++ kernel/livepatch/core.c | 153 +++++++++++++++++++++++----------------------- kernel/module.c | 20 +++++- 3 files changed, 103 insertions(+), 79 deletions(-)
include/linux/livepatch.h +9 -0
@@ -134,6 +134,15 @@ int klp_unregister_patch(struct klp_patch *);int klp_enable_patch (struct klp_patch *);int klp_disable_patch (struct klp_patch *); +/* Called from the module loader during module coming/going states */ +int klp_module_coming (struct module *mod); +void klp_module_going (struct module *mod); + +#else /* !CONFIG_LIVEPATCH */ + +static inline int klp_module_coming (struct module *mod) {return 0 ; } +static inline void klp_module_going (struct module *mod) { } +#endif /* CONFIG_LIVEPATCH */ #endif /* _LINUX_LIVEPATCH_H_ */
kernel/livepatch/core.c +76 -77
212 lines not on this page · raw message
kernel/module.c +18 -2
@@ -53,6 +53,7 @@#include <asm/sections.h> #include <linux/tracepoint.h> #include <linux/ftrace.h> +#include <linux/livepatch.h> #include <linux/async.h> #include <linux/percpu.h> #include <linux/kmemleak.h>
@@ -984,6 +985,7 @@ SYSCALL_DEFINE2(delete_module, const char __user *, name_user, mod->exit ();blocking_notifier_call_chain (&module_notify_list,MODULE_STATE_GOING , mod); +klp_module_going (mod);ftrace_release_mod (mod);async_synchronize_full ();
@@ -3315,6 +3317,7 @@ fail:module_put (mod);blocking_notifier_call_chain (&module_notify_list,MODULE_STATE_GOING , mod); +klp_module_going (mod);ftrace_release_mod (mod);free_module (mod);wake_up_all (&module_wq);
@@ -3377,8 +3380,10 @@ static int complete_formation(struct module *mod, struct load_info *info)/* Find duplicate symbols (must be called under lock). */ err =verify_export_symbols (mod); -if (err <0 ) - goto out; +if (err <0 ) { +mutex_unlock (&module_mutex); +return err; + }/* This relies on module_mutex for list integrity. */ module_bug_finalize (info->hdr, info->sechdrs, mod);
@@ -3393,12 +3398,22 @@ static int complete_formation(struct module *mod, struct load_info *info)mutex_unlock (&module_mutex);ftrace_module_enable (mod); + err =klp_module_coming (mod); +if (err) + goto out; +blocking_notifier_call_chain (&module_notify_list,MODULE_STATE_COMING , mod);return 0 ; out: +mutex_lock (&module_mutex); +module_bug_cleanup (mod);mutex_unlock (&module_mutex); + +module_disable_ro (mod); +module_disable_nx (mod); +return err; }
@@ -3549,6 +3564,7 @@ static int load_module(struct load_info *info, const char __user *uargs,blocking_notifier_call_chain (&module_notify_list,MODULE_STATE_GOING , mod); +klp_module_going (mod);/* we can't deallocate the module until we clear memory protection */ module_disable_ro (mod);
Signature · 1 line
2.4.3
Re: [PATCH v3 2/2] livepatch/module: remove livepatch module notifier
Quoted 45 lines · Jessica Yu
Remove the livepatch module notifier in favor of directly enabling and disabling patches to modules in the module loader. Hard-coding the function calls ensures that
ftrace_module_enable()is run beforeklp_module_coming()during module load, and thatklp_module_going()is run beforeftrace_release_mod()during module unload. This way, ftrace and livepatch code is run in the correct order during the module load/unload sequence without dependence on the module notifier call chain.This fixes a notifier ordering issue in which the ftrace module notifier (and hence
ftrace_module_enable()) for coming modules was being called afterklp_module_notify(), which caused livepatch modules to initialize incorrectly.Signed-off-by Jessica Yu <[email protected]>--- include/linux/livepatch.h | 9 +++ kernel/livepatch/core.c | 153 +++++++++++++++++++++++----------------------- kernel/module.c | 20 +++++- 3 files changed, 103 insertions(+), 79 deletions(-)include/linux/livepatch.h +9
-0@@ -134,6 +134,15 @@ int klp_unregister_patch(struct klp_patch *);int klp_enable_patch (struct klp_patch *);int klp_disable_patch (struct klp_patch *); +/* Called from the module loader during module coming/going states */ +int klp_module_coming (struct module *mod); +void klp_module_going (struct module *mod); + +#else /* !CONFIG_LIVEPATCH */ + +static inline int klp_module_coming (struct module *mod) {return 0 ; } +static inline void klp_module_going (struct module *mod) { } +#endif /* CONFIG_LIVEPATCH */ #endif /* _LINUX_LIVEPATCH_H_ */ kernel/livepatch/core.c +1
-0+int klp_module_coming (struct module *mod)
[...]
Quoted 1 line
kernel/livepatch/core.c +1
-0+ obj->mod = mod;
[...]
Quoted 6 lines
kernel/livepatch/core.c +6
-0+ ret =klp_init_object_loaded (patch, obj); +if (ret) { +pr_warn ("failed to initialize patch '%s' for module '%s' (%d)\n" , + patch->mod->name, obj->mod->name, ret); + goto err; + }
[...]
Quoted 6 lines
kernel/livepatch/core.c +6
-0+ ret =klp_enable_object (obj); +if (ret) { +pr_warn ("failed to apply patch '%s' to module '%s' (%d)\n" , + patch->mod->name, obj->mod->name, ret); + goto err; + }
[...]
kernel/livepatch/core.c int klp_register_patch(struct klp_patch *patch) · line 928
Quoted 9 lines
kernel/livepatch/core.c +9
-0+ +err: +/* +* If a patch is unsuccessfully applied, return +* error to the module loader. +*/ +pr_warn ("patch '%s' failed for module '%s', refusing to load module '%s'\n" , + patch->mod->name, obj->mod->name, obj->mod->name); + obj->mod =NULL ;
To be on the safe side, we should replace this assignment with:
klp_free_object_loaded(mod);
It clears obj->mod and also all func->old_addr for this module. By other words, it leaves the structures in the same state as klp_module_going(). I am sorry that I have missed this before.
Otherwise, it looks fine. So, with the above suggested change:
Best Regards,
Petr
Quoted 16 lines · Jessica Yu
As explained here [1], livepatch modules are failing to initialize properly because the ftrace coming module notifier (which calls
ftrace_module_enable()) runs *after* the livepatch module notifier (which enables the patch(es)). Thus livepatch attempts to apply patches to modules beforeftrace_module_enable()is even called for the corresponding module(s). As a result, patch modules break. Ftrace code must run before livepatch on module load, and the reverse is true on module unload.For ftrace and livepatch, order of initialization (plus exit/cleanup code) is important for loading and unloading modules, and using module notifiers to perform this work is not ideal since it is not always clear what gets called when. In this patchset, dependence on the module notifier call chain is removed in favor of hard coding the corresponding function calls in the module loader. This promotes better code visibility and ensures that ftrace and livepatch code get called in the correct order on patch module load and unload.
Quoted 2 lines
Tested the changes with a test livepatch module that patches 9p and nilfs2, and verified that the issue described in [1] is fixed.
Since Rusty agreed to your suggested changes for splitting up complete_formation() and for setting mod->state = MODULE_STATE_GOING before calling the going notifiers in the error path, can you do a v4 with those changes? They should probably be split up like:
1. split up complete_formation()
2. set MODULE_STATE_GOING before calling going notifiers in error path
3. remove ftrace module notifier
4. remove livepatch module notifier
Signature · 1 line
Josh
+++ Josh Poimboeuf [08/02/16 11:48 -0600]:
Quoted 25 lines · Jessica Yu
Quoted 16 lines
As explained here [1], livepatch modules are failing to initialize properly because the ftrace coming module notifier (which calls
ftrace_module_enable()) runs *after* the livepatch module notifier (which enables the patch(es)). Thus livepatch attempts to apply patches to modules beforeftrace_module_enable()is even called for the corresponding module(s). As a result, patch modules break. Ftrace code must run before livepatch on module load, and the reverse is true on module unload.For ftrace and livepatch, order of initialization (plus exit/cleanup code) is important for loading and unloading modules, and using module notifiers to perform this work is not ideal since it is not always clear what gets called when. In this patchset, dependence on the module notifier call chain is removed in favor of hard coding the corresponding function calls in the module loader. This promotes better code visibility and ensures that ftrace and livepatch code get called in the correct order on patch module load and unload.
Quoted 2 lines
Tested the changes with a test livepatch module that patches 9p and nilfs2, and verified that the issue described in [1] is fixed.
Since Rusty agreed to your suggested changes for splitting up
complete_formation()and for settingmod->state=MODULE_STATE_GOINGbefore calling the going notifiers in the error path, can you do a v4 with those changes? They should probably be split up like:
Quoted 4 lines
1. split up
complete_formation()
2. setMODULE_STATE_GOINGbefore calling going notifiers in error path
3. remove ftrace module notifier
4. remove livepatch module notifier
Sure, I'll do that.
Jessica
Quoted 6 lines · Jessica Yu
Quoted 4 lines
1. split up
complete_formation()
2. setMODULE_STATE_GOINGbefore calling going notifiers in error path
3. remove ftrace module notifier
4. remove livepatch module notifierSure, I'll do that.
Also, please when sending out v4, don't forget to include all the Acked-by/Reviewed-by's received so far (for 1/2 you have the substantial ones already :) ).
Thanks a lot,
Signature · 2 lines
Jiri Kosina SUSE Labs