x86: Add a list for custom page fault handlers.
Subsystems (4)
PATCH 1/1 x86: Add a list for custom page fault handlers.
From: Pekka Paalanen <[email protected]>
Provides kernel modules a way to register custom page fault handlers. On every page fault, except those handled in vmalloc_fault(), this will call a list of registered functions. The functions may handle the fault and force do_page_fault() to return immediately.
This functionality is similar to the now removed page fault notifiers. Custom page fault handlers are used by debugging and reverse engineering tools. Mmio-trace is one such tool and a patch to add it into the tree will follow.
The custom page fault handlers are called from the exact same points in do_page_fault() as the page fault notifiers were.
---
This change is x86 specific purely because my intended in-tree user of it is mmio-trace, which AFAIK is really useful only on x86 for reverse engineering binary drivers. If someone later wishes to merge this with notify_page_fault() from kprobes, they can do that.
I am not subscribed to LKML, so please keep me in CC.
I will reply to this email with the next patch adding mmio-trace into kernel.
Thanks,
pq
arch/x86/Kconfig.debug | 9 ++++++ arch/x86/mm/fault_32.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/mm/fault_64.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ include/asm-x86/kdebug.h | 9 ++++++ 4 files changed, 146 insertions(+), 0 deletions(-)
arch/x86/Kconfig.debug +9 -0
@@ -112,4 +112,13 @@ config IOMMU_LEAK Add a simple leak tracer to the IOMMU code. This is useful when you are debugging a buggy device driver that leaks IOMMU mappings. +config PAGE_FAULT_HANDLERS + bool "Custom page fault handlers" + depends on DEBUG_KERNEL + help + Allow the use of custom page fault handlers. A kernel module may + register a function that is called on every page fault not handled + for vmalloc. Custom handlers are used by some debugging and reverse + engineering tools. + endmenu
arch/x86/mm/fault_32.c +64 -0
82 lines not on this page · raw message
arch/x86/mm/fault_64.c +64 -0
82 lines not on this page · raw message
include/asm-x86/kdebug.h +9 -0
@@ -30,4 +30,13 @@ extern void dump_pagetable(unsigned long);extern unsigned long oops_begin (void );extern void oops_end (unsigned long ); +struct pf_handler { +struct hlist_node hlist; +int (*handler)(struct pt_regs *regs,unsigned long error_code, +unsigned long address); +}; + +extern void register_page_fault_handler (struct pf_handler *new_pfh); +extern void unregister_page_fault_handler (struct pf_handler *old_pfh); +#endif
Signature · 1 line
1.5.3.7
RFC PATCH 1/1 x86: mmiotrace - trace memory mapped IO
Hello,
the patch itself is slightly bigger than recommended for LKML, so it is available at:
http://jumi.lut.fi/~paalanen/scratch/mmio24/0002-x86-mmiotrace-trace-memory-mapped-IO.patch.txt
This seems to work fine on UP machine, I have not tested yet with with an SMP machine. Originally mmio-trace was UP-only, so there probably still are SMP issues. Mmio-trace has existed with its current purpose for about a year now.
The following is a copy of the patch description:
Mmiotrace is a tool for trapping memory mapped IO (MMIO) accesses within the kernel. It is used for debugging and especially for reverse engineering evil binary drivers.
Mmiotrace works by wrapping the ioremap family of kernel functions and marking the returned pages as not present. Access to the IO memory triggers a page fault, which will be handled by mmiotrace's custom page fault handler. This will single-step the faulted instruction with the MMIO page marked as present. Access logs are directed to user space via relay and debug_fs.
This page fault approach is necessary, because binary drivers have readl/writel etc. calls inlined and therefore extremely difficult to trap with with e.g. kprobes.
This patch depends on the custom page fault handlers patch.
---
More information on mmio-trace can be found at
http://nouveau.freedesktop.org/wiki/MmioTrace
and the out-of-three module with user space parts is at
http://gitweb.freedesktop.org/?p=users/pq/mmio-trace.git;a=summary
The out-of-tree git does not include all changes I made for the in-tree version.
Mmiotrace does not appear to procude excruciating slowness when used. Starting X with traced nvidia binary drivers takes a couple of seconds longer than normally. Starting X, running glxgears for a moment and shutting down X results in an mmio-trace log of roughly 1.5 million events. Replace glxgears by running Quake3-demo's first demo play twice and end up with 3.5 million events. Quake3-demo framerate with mmio-traced driver goes down to 59 fps, compared to the normal 83 fps.
I know of one bad failure case with mmio-trace. It may trigger a complete system freeze when tracing a certain nvidia driver version on a machine with an 8000-series (nv50) graphics chip. Netconsole, serial console and NMI watchdog have so far been useless, and I am starting to believe that it is the nvidia driver that does something bad to make mmio-trace fall over. I have heard that more recent nvidia drivers do not make it crash anymore.
I'm hoping mmiotrace could be included in 2.6.25. It would help FOSS driver projects (Nouveau, maybe madwifi, too) to gather the information they need. Very often the information comes from ordinary users, for whom going through kernel patching would be a major struggle.
Thanks,
Pekka
Re: [RFC PATCH] x86: mmiotrace - trace memory mapped IO
Quoted 1 line · On Sun, 27 Jan 2008 19:55:36 +0200 Pekka Paalanen <[email protected]> wrote:
The following is a copy of the patch description:
...
Quoted 6 lines
Mmiotrace works by wrapping the ioremap family of kernel functions and marking the returned pages as not present. Access to the IO memory triggers a page fault, which will be handled by mmiotrace's custom page fault handler. This will single-step the faulted instruction with the MMIO page marked as present. Access logs are directed to user space via relay and
debug_fs.
Hi,
a test user found a problem with mmiotrace (ported to x86/mm git tree). The symbol 'init_mm' is undefined, but required when building the 32-bit mmiotrace kernel module.
commit 3abf024d2abb79614d8c4cb25a70d5596f77d0ad
Author: Thomas Gleixner <[email protected]> Date: Wed Jan 30 13:30:28 2008 +0100
x86: nuke a ton of unused exports
This commit removes EXPORT_SYMBOL(init_mm). Apparently mmiotrace requires this symbol, but I do not know why. I do not use init_mm myself, so apparently some function I call is inlined and requires it. Page table or attribute handling stuff maybe?
Is it possible to simply put back this export or should I dig deeper into why it seems to be required?
I can provide the simple patch to put it back in my next patch set, if this is the right solution.
I will post the patch set after this issue is solved. In the mean time, for the interested, the current patches are here:
http://jumi.lut.fi/~paalanen/scratch/mmio25/
Ps. I tested x86/mm version of mmiotrace on my 64-bit system and it seems to work ok. I had to write a simple test module because I did not have any drivers that a) called ioremap, b) were not broken on x86/mm, and c) I had the hardware to actually use it. So this testing is very light.
Signature · 2 lines
Pekka Paalanen http://www.iki.fi/pq/
Quoted 4 lines · Pekka Paalanen
arch/x86/Kconfig.debug | 9 ++++++ arch/x86/mm/fault_32.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/mm/fault_64.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ include/asm-x86/kdebug.h | 9 ++++++
hm, could you please base this against x86.git#mm, see:
http://redhat.com/~mingo/x86.git/README
thanks,
Ingo
arch/x86/mm/fault_32.c line 56
Quoted 9 lines · Pekka Paalanen
arch/x86/mm/fault_32.c +9
-0+void unregister_page_fault_handler (struct pf_handler *old_pfh) +{ +might_sleep (); +spin_lock (&pf_handlers_writer); +hlist_del_rcu (&old_pfh->hlist); +spin_unlock (&pf_handlers_writer); +synchronize_rcu (); +} +EXPORT_SYMBOL_GPL (unregister_page_fault_handler);
Could you remove that sync_rcu() call and put in the comments that the calling function is to take care of proper freeing.
sync_rcu() is a _very_ expensive call, and oft time it is not needed in favour of call_rcu().
From: Pekka Paalanen <[email protected]>
Provides kernel modules a way to register custom page fault handlers. On every page fault, except those handled in vmalloc_fault(), this will call a list of registered functions. The functions may handle the fault and force do_page_fault() to return immediately.
This functionality is similar to the now removed page fault notifiers. Custom page fault handlers are used by debugging and reverse engineering tools. Mmio-trace is one such tool and a patch to add it into the tree will follow.
The custom page fault handlers are called from the exact same points in do_page_fault() as the page fault notifiers were.
---
Ingo, ported Pekka's patch to current x86.git for review. I have not incorporated the RCU comments from Peter Zjistra, but I have folded some of the static functions into on handle_custom_pf function that will match the handle_kprobe_fault helper that will be ready for RC1.
Harvey
arch/x86/Kconfig.debug | 9 ++++++++ arch/x86/mm/fault.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+), 0 deletions(-)
arch/x86/Kconfig.debug +9 -0
@@ -225,4 +225,13 @@ config CPA_DEBUG help Do change_page_attr self tests at boot. +config PAGE_FAULT_HANDLERS + bool "Custom page fault handlers" + depends on DEBUG_KERNEL + help + Allow the use of custom page fault handlers. A kernel module may + register a function that is called on every page fault not handled + for vmalloc. Custom handlers are used by some debugging and reverse + engineering tools. + endmenu
arch/x86/mm/fault.c +25 -0
@@ -49,6 +49,54 @@#define PF_RSVD (1<<3)#define PF_INSTR (1<<4) +#ifdef CONFIG_PAGE_FAULT_HANDLERS +static HLIST_HEAD (pf_handlers);/* protected by RCU */ +static DEFINE_SPINLOCK (pf_handlers_writer); + +void register_page_fault_handler (struct pf_handler *new_pfh) +{ +spin_lock (&pf_handlers_writer); +hlist_add_head_rcu (&new_pfh->hlist, &pf_handlers); +spin_unlock (&pf_handlers_writer); +} +EXPORT_SYMBOL_GPL (register_page_fault_handler); + +void unregister_page_fault_handler (struct pf_handler *old_pfh) +{ +might_sleep (); +spin_lock (&pf_handlers_writer); +hlist_del_rcu (&old_pfh->hlist); +spin_unlock (&pf_handlers_writer); +synchronize_rcu (); +} +EXPORT_SYMBOL_GPL (unregister_page_fault_handler); +#endif + +/* returns non-zero if do_page_fault() should return */ +static int handle_custom_pf(struct pt_regs *regs, unsigned long
error_code,
+ unsigned long address)
+{
+#ifdef CONFIG_PAGE_FAULT_HANDLERS
+ int ret = 0;
+ struct pf_handler *cur;
+ struct hlist_node *ncur;
+
+ if (hlist_empty(&pf_handlers))
+ return 0;
+
+ rcu_read_lock();
+ hlist_for_each_entry_rcu(cur, ncur, &pf_handlers, hlist) {
+ ret = cur->handler(regs, error_code, address);
+ if (ret)
+ break;
+ }
+ rcu_read_unlock();
+ return ret;
+#else
+ return 0;
+#endif
+}
+
static inline int notify_page_fault(struct pt_regs *regs) {
#ifdef CONFIG_KPROBES
@@ -588,6 +636,9 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code)
if (notify_page_fault(regs))
return;
+ if (handle_custom_pf(regs, error_code, address))
+ return;
+
/*
* We fault-in kernel-space virtual memory on-demand. The
* 'reference' page table is init_mm.pgd.
Signature · 1 line
1.5.4.rc4.1142.gf5a97
From: Pekka Paalanen <[email protected]>
Provides kernel modules a way to register custom page fault handlers. On every page fault, except those handled in vmalloc_fault(), this will call a list of registered functions. The functions may handle the fault and force do_page_fault() to return immediately.
This functionality is similar to the now removed page fault notifiers. Custom page fault handlers are used by debugging and reverse engineering tools. Mmio-trace is one such tool and a patch to add it into the tree will follow.
The custom page fault handlers are called from the exact same points in do_page_fault() as the page fault notifiers were.
---
Sorry, attached the wrong version to my last message missing the kdebug.h hunk. This is still just a straight port to current x86.git.
arch/x86/Kconfig.debug | 9 ++++++++ arch/x86/mm/fault.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++ include/asm-x86/kdebug.h | 8 +++++++ 3 files changed, 68 insertions(+), 0 deletions(-)
arch/x86/Kconfig.debug +9 -0
@@ -225,4 +225,13 @@ config CPA_DEBUG help Do change_page_attr self tests at boot. +config PAGE_FAULT_HANDLERS + bool "Custom page fault handlers" + depends on DEBUG_KERNEL + help + Allow the use of custom page fault handlers. A kernel module may + register a function that is called on every page fault not handled + for vmalloc. Custom handlers are used by some debugging and reverse + engineering tools. + endmenu
arch/x86/mm/fault.c +51 -0
63 lines not on this page · raw message
include/asm-x86/kdebug.h +8 -0
@@ -35,4 +35,12 @@ extern void dump_pagetable(unsigned long);extern unsigned long oops_begin (void );extern void oops_end (unsigned long ,struct pt_regs *,int signr); +struct pf_handler { +struct hlist_node hlist; +int (*handler)(struct pt_regs *regs,unsigned long error_code, +unsigned long address); +}; + +extern void register_page_fault_handler (struct pf_handler *new_pfh); +extern void unregister_page_fault_handler (struct pf_handler *old_pfh);#endif
Signature · 1 line
1.5.4.rc4.1142.gf5a97
Quoted 2 lines · On Tue, 29 Jan 2008 18:34:13 -0800 Harvey Harrison <[email protected]>…
Sorry, attached the wrong version to my last message missing the
kdebug.hhunk. This is still just a straight port to current x86.git.
Wow, thanks.
I have to say I already did this for myself, but I had to
change too many things to simply resubmit without testing. And it turned out porting mmiotrace to x86/mm was a non-trivial task. I have not been able to test yet even though I have been working on it every night. Today I will see if things work for me, and hopefully also for someone else. I might send new patches today or tomorrow, if all goes well.
btw. I have a bad feeling about global_flush_tlb() disappearing, it was used in mmiotrace code, and I don't know what to use instead. The obvious replacement for it is a static function elsewhere. Currently I am simply hoping it was a redundant leftover from the time when we had trouble making mmiotrace stable.
To be honest, I haven't yet had the time to educate myself about what is TLB.
Quoted 7 lines
arch/x86/mm/fault.c +0
-0@@ -49,6 +49,54 @@#define PF_RSVD (1<<3)#define PF_INSTR (1<<4)
arch/x86/mm/fault.c line 60
Quoted 10 lines
arch/x86/mm/fault.c +9
-0+#ifdef CONFIG_PAGE_FAULT_HANDLERS +static HLIST_HEAD (pf_handlers);/* protected by RCU */ +static DEFINE_SPINLOCK (pf_handlers_writer); + +void register_page_fault_handler (struct pf_handler *new_pfh) +{ +spin_lock (&pf_handlers_writer); +hlist_add_head_rcu (&new_pfh->hlist, &pf_handlers); +spin_unlock (&pf_handlers_writer);
Here I had to change to irqsave/irqrestore versions, as I found a possible lock dependency issue. Lockdep debugging code pointed it out. I'm not sure I fixed it completely, but using spin_lock_irqsave() is a safe bet.
arch/x86/mm/fault.c line 71
Quoted 11 lines
arch/x86/mm/fault.c +11
-0+} +EXPORT_SYMBOL_GPL (register_page_fault_handler); + +void unregister_page_fault_handler (struct pf_handler *old_pfh) +{ +might_sleep (); +spin_lock (&pf_handlers_writer); +hlist_del_rcu (&old_pfh->hlist); +spin_unlock (&pf_handlers_writer); +synchronize_rcu (); +}
And here. And I also removed sync RCU and might_sleep().
Quoted 4 lines
arch/x86/mm/fault.c +4
-0+EXPORT_SYMBOL_GPL (unregister_page_fault_handler); +#endif + +/* returns non-zero if do_page_fault() should return */
arch/x86/mm/fault.c line 85
Quoted 10 lines
arch/x86/mm/fault.c +10
-0+static int handle_custom_pf (struct pt_regs *regs,unsigned long error_code, +unsigned long address) +{ +#ifdef CONFIG_PAGE_FAULT_HANDLERS +int ret =0 ; +struct pf_handler *cur; +struct hlist_node *ncur; + +if (hlist_empty (&pf_handlers)) +return 0 ;
This looks prettier than my code. I might change my code to this.
Quoted 17 lines
arch/x86/mm/fault.c +13
-0+rcu_read_lock (); +hlist_for_each_entry_rcu (cur, ncur, &pf_handlers, hlist) { + ret = cur->handler (regs, error_code, address); +if (ret) +break ; + } +rcu_read_unlock (); +return ret; +#else +return 0 ; +#endif +} +static inline int notify_page_fault (struct pt_regs *regs) {#ifdef CONFIG_KPROBES
arch/x86/mm/fault.c void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code) · line 641
Quoted 6 lines
arch/x86/mm/fault.c +3
-0@@ -588,6 +636,9 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code)if (notify_page_fault (regs))return ; +if (handle_custom_pf (regs, error_code, address)) +return ; +
This is the non-trivial change that I really want to test before submitting again. The call site of the handlers changes, and vmalloc related faults reach the custom handlers. I don't think this would be a problem for mmiotrace, but I don't know enough to be sure.
I guess using vmalloc'd memory from a page fault handler would be doomed, anyway, so no point worrying about it, right?
if (unlikely(handle_custom_pf(regs, error_code, address))) Maybe like this, even?
It is still supposedly making a function call every time, at least when CONFIG_PAGE_FAULT_HANDLERS is defined. Would it have a non-negligible impact on performance when no custom handlers are registered?
My original version had the separate inline function to check if the list is empty. Which one is preferrable?
I'm hoping distros would ship some kernels with PAGE_FAULT_HANDLERS enabled as it would make contributors' lives easier. IIRC some already have enabled RELAY and DEBUG_FS for mmiotrace. This is why I think keeping the performance impact to the minimum here is important, for the case of no custom handlers registered.
Thanks,
Signature · 2 lines
Pekka Paalanen http://www.iki.fi/pq/
Quoted 7 lines · Pekka Paalanen
Quoted 7 lines
static inline int
notify_page_fault(struct pt_regs *regs){
#ifdefCONFIG_KPROBESarch/x86/mm/fault.c +0
-0@@ -588,6 +636,9 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code)if (notify_page_fault (regs))return ;
Quoted 10 lines
arch/x86/mm/fault.c void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code) · line 641
Quoted 3 lines
arch/x86/mm/fault.c +3
-0+if (handle_custom_pf (regs, error_code, address)) +return ; +This is the non-trivial change that I really want to test before submitting again. The call site of the handlers changes, and vmalloc related faults reach the custom handlers. I don't think this would be a problem for mmiotrace, but I don't know enough to be sure. I guess using vmalloc'd memory from a page fault handler would be doomed, anyway, so no point worrying about it, right?
could you please send us a patch for the whole mmiotrace kernel-side feature, so that we can have a look at the general structure of this? (and the interaction with change_page_attr(), etc.) Even if it's not functional (and wont even build/boot) at the moment. Thanks,
Ingo
PATCH v2 1/1 x86: Add a list for custom page fault handlers.
Quoted 4 lines
could you please send us a patch for the whole mmiotrace kernel-side feature, so that we can have a look at the general structure of this? (and the interaction with
change_page_attr(), etc.) Even if it's not functional (and wont even build/boot) at the moment. Thanks,
Very well, first the revised custom page fault handler patch. Changes since the previous submit:
- use spin_lock_irqsave instead of spin_lock
- Harvey Harrison's clean-up with the #ifdefs
- handler call site moved earlier
- remove sync RCU call
I'm not aware of any functional problems with this one.
Arjan, you said you don't like this. May I ask why?
This is most convinient for mmiotrace as it is meant to be a module. I'm also using this as an excuse to let other people to get into the page fault handler with their out-of-tree-today modules.
Quoted 1 line
From
adeb0f4f95f176d97c1157031b5dd01f49d4cb66Mon Sep 17 00:00:00 2001
From: Pekka Paalanen <[email protected]>
Date: Wed, 30 Jan 2008 21:49:52 +0200
Subject: [PATCH] x86: Add a list for custom page fault handlers.
Provides kernel modules a way to register custom page fault handlers. On every page fault this will call a list of registered functions. The functions may handle the fault and force do_page_fault() to return immediately.
This functionality is similar to the now removed page fault notifiers. Custom page fault handlers are used by debugging and reverse engineering tools. Mmiotrace is one such tool and a patch to add it into the tree will follow.
The custom page fault handlers are called earlier in do_page_fault() than the page fault notifiers were.
--- arch/x86/Kconfig.debug | 8 ++++++ arch/x86/mm/fault.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++ include/asm-x86/kdebug.h | 9 +++++++ 3 files changed, 73 insertions(+), 0 deletions(-)
arch/x86/Kconfig.debug +8 -0
@@ -138,6 +138,14 @@ config IOMMU_LEAK Add a simple leak tracer to the IOMMU code. This is useful when you are debugging a buggy device driver that leaks IOMMU mappings. +config PAGE_FAULT_HANDLERS + bool "Custom page fault handlers" + depends on DEBUG_KERNEL + help + Allow the use of custom page fault handlers. A kernel module may + register a function that is called on every page fault. Custom + handlers are used by some debugging and reverse engineering tools. + # # IO delay types: #
arch/x86/mm/fault.c +56 -0
68 lines not on this page · raw message
include/asm-x86/kdebug.h +9 -0
@@ -35,4 +35,13 @@ extern void dump_pagetable(unsigned long);extern unsigned long oops_begin (void );extern void oops_end (unsigned long ,struct pt_regs *,int signr); +struct pf_handler { +struct hlist_node hlist; +int (*handler)(struct pt_regs *regs,unsigned long error_code, +unsigned long address); +}; + +extern void register_page_fault_handler (struct pf_handler *new_pfh); +extern void unregister_page_fault_handler (struct pf_handler *old_pfh); +#endif
Signature · 1 line
1.5.3.7
Quoted 13 lines · On Thu, 31 Jan 2008 18:02:53 +0200 Pekka Paalanen <[email protected]> wrote:
Quoted 5 lines
could you please send us a patch for the whole mmiotrace kernel-side feature, so that we can have a look at the general structure of this? (and the interaction with
change_page_attr(), etc.) Even if it's not functional (and wont even build/boot) at the moment. Thanks,Very well, first the revised custom page fault handler patch. Changes since the previous submit:
- usespin_lock_irqsaveinstead ofspin_lock
- Harvey Harrison's clean-up with the #ifdefs
- handler call site moved earlier
- remove sync RCU call
Quoted 3 lines
I'm not aware of any functional problems with this one.
Arjan, you said you don't like this. May I ask why?
it's overhead that's rarely used, and worse, it suffers from the LSM disease:
it's a hook without visibility into the callers. Esp if there's only 1 user, a direct call is not only faster, it also shows you what is going on when you're reading the code. THe page fault stuff is tricky enough as it is without any "call arbitrary thing" hooks into it
Quoted 1 line
This is most convinient for mmiotrace as it is meant to be a module.
that's not per se mutually exclusive, there's other cases like this already (but even then, I don't want to go through hoops to make mmiotracing a module; for me I can totally see it becoming a full mature member of the kernel, and if that means a few things have to be changed in the normal kernel, or infrastructure added, I'm totally ok with that)
Quoted 2 lines
I'm also using this as an excuse to let other people to get into the page fault handler with their out-of-tree-today modules.
this is exactly the point.. it becomes a totally opaque thing, a random hook.
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 10 lines · On Thu, 31 Jan 2008 08:15:07 -0800 Arjan van de Ven <[email protected]> wrote:
Quoted 7 lines · On Thu, 31 Jan 2008 18:02:53 +0200 Pekka Paalanen <[email protected]> wrote:
Very well, first the revised custom page fault handler patch. Changes since the previous submit:
- usespin_lock_irqsaveinstead ofspin_lock
- Harvey Harrison's clean-up with the #ifdefs
- handler call site moved earlier
- remove sync RCU call
Quoted 5 lines
it's overhead that's rarely used, and worse, it suffers from the LSM disease:
it's a hook without visibility into the callers. Esp if there's only 1 user, a direct call is not only faster, it also shows you what is going on when you're reading the code. THe page fault stuff is tricky enough as it is without any "call arbitrary thing" hooks into it
I understand your point.
Quoted 7 lines
Quoted 1 line
This is most convinient for mmiotrace as it is meant to be a module.
that's not per se mutually exclusive, there's other cases like this already (but even then, I don't want to go through hoops to make mmiotracing a module; for me I can totally see it becoming a full mature member of the kernel, and if that means a few things have to be changed in the normal kernel, or infrastructure added, I'm totally ok with that)
I have two reasons why I'd like to let it be a module:
- it's "broken", the relay buffers are cleared on module unload
- it's a lot easier to push updated version for testing to people
Ok, the first one is just a silly excuse, but the second one is to avoid forcing non-developer users to patch their kernels. This is a much needed tool for the Nouveau project even in it's current form.
Quoted 4 lines
Quoted 2 lines
I'm also using this as an excuse to let other people to get into the page fault handler with their out-of-tree-today modules.
this is exactly the point.. it becomes a totally opaque thing, a random hook.
And this was the third reason I wanted a module, but I understand this is an unacceptable justification.
What are these "othes cases" I could look at as an example of a hook, that is meant for a single specific module? I do not understand what could prevent out-of-tree modules from using the hook if my module can use it (and my module is not using it at the same time).
What is the verdict?
Will the custom page fault handler patch v2 get merged as it is, do I need to modify it (how?), do I need to rewrite it based on some example (which?), or do I have to drop the whole idea of mmiotrace being kernel module and make it work as built-in?
In the case of making it a built-in, I do not think I have any chance to get it done for 2.6.25.
There's one more option. I could make kmmio.c things as builtin, and keep the mmio-mod.c as the module. This is how things were in the very beginning of mmiotrace, when the page fault notifiers did not exist. This could be an acceptable compromise, no? But I don't think I will get that ready for 2.6.25, either.
What about merging the custom page fault handler patch now, and removing it for 2.6.26?
Looks like Beulich and Roskin will be disappointed anyway. They were hoping for a usable page fault hook.
Signature · 2 lines
Pekka Paalanen http://www.iki.fi/pq/
Quoted 7 lines · Pekka Paalanen
I have two reasons why I'd like to let it be a module:
- it's "broken", the relay buffers are cleared on module unload
- it's a lot easier to push updated version for testing to peopleOk, the first one is just a silly excuse, but the second one is to avoid forcing non-developer users to patch their kernels. This is a much needed tool for the Nouveau project even in it's current form.
i dont think this second problem is a practical one: i've test-merged mmiotrace two days ago into x86.git and it's holding up very well in my testing (it has not caused a single hickup so far). It's nice optional debug functionality and i dont see any reason why not to merge it.
a few cleanup requests:
- please make the pagefault callbacks explicit as Arjan has asked
- any reason why it lives in arch/x86/kernel/mmiotrace/? It should be a prime-time member of arch/x86/mm/ i think.
- please make it build-in-able, not modules-only.
- [ longer-term: think about integrating pf_in.c with the x86 emulator/disassembler of KVM. ]
Ingo
Quoted 4 lines · On Sun, 3 Feb 2008 08:03:21 +0100 Ingo Molnar <[email protected]> wrote:
i dont think this second problem is a practical one: i've test-merged mmiotrace two days ago into x86.git and it's holding up very well in my testing (it has not caused a single hickup so far). It's nice optional debug functionality and i dont see any reason why not to merge it.
Good to hear!
I've made some fixes since my last mmiotrace patch (v2). Rewritten things to use lookup_address(), added EXPORT_SYMBOL(lookup_address) into pageattr.c to make it work (this hopefully fixed the undefined symbol 'init_mm' on x86-32), and fixed relay-buffer-full flags on SMP.
Quoted 3 lines
a few cleanup requests:
- please make the pagefault callbacks explicit as Arjan has asked
What does this mean in practice? It should be able to call into a module, so the only thing I can come up with is a single settable function pointer. Just like I proposed in patch v2, but a function pointer instead of a list. Is this what you and Arjan suggest?
We are talking about the hook in do_page_fault(), right?
Quoted 2 lines
- any reason why it lives in
arch/x86/kernel/mmiotrace/? It should be a prime-time member ofarch/x86/mm/i think.
I was waiting for someone to comment on that, I didn't know what was the proper place. I'll move it into there.
Quoted 1 line
- please make it build-in-able, not modules-only.
Then I need a way to clear the relay buffers at some point. Currently they get cleared only on module unload. Maybe preferably also allocate and deallocate them dynamically. I've no idea how to do this yet, any pointers to examples?
Hmm, I guess relay buffers could be allocated on the first intercepted call to ioremap, but then the userspace could not be started before hand, and at that point it is too late.
Making it build-in-able will take so much time that I'd prefer to aim it for 2.6.26.
Quoted 2 lines
- [ longer-term: think about integrating
pf_in.cwith the x86 emulator/disassembler of KVM. ]
Yikes. Like I replied to Arjan, it looks daunting.
Thanks, I'll do what I can. When do you need the final (as for 2.6.25) mmiotrace patch, or is it going in as v2 and then we fix it in RCs?
Signature · 2 lines
Pekka Paalanen http://www.iki.fi/pq/
PATCH 1/4 x86 mmiotrace: use lookup_address()
In reply to Ingo MolnarUse lookup_address() from pageattr.c instead of doing the same manually. Also had to EXPORT_SYMBOL(lookup_address) to make this work for modules. This also fixes "undefined symbol 'init_mm'" compile error for x86_32.
--- arch/x86/kernel/mmiotrace/kmmio.c | 46 +++++++++++++++++++++++----------- arch/x86/kernel/mmiotrace/mmio-mod.c | 19 +++++++++----- arch/x86/mm/pageattr.c | 1 + 3 files changed, 44 insertions(+), 22 deletions(-)
arch/x86/kernel/mmiotrace/kmmio.c +31 -15
@@ -20,6 +20,7 @@#include <asm/cacheflush.h> #include <asm/errno.h> #include <asm/tlbflush.h> +#include <asm/pgtable.h> #include "kmmio.h"
@@ -117,40 +118,55 @@ static struct kmmio_fault_page *get_kmmio_fault_page(unsigned long page)return NULL ; } -static void arm_kmmio_fault_page (unsigned long page,int *large) +static void arm_kmmio_fault_page (unsigned long page,int *page_level) {unsigned long address = page &PAGE_MASK ; -pgd_t *pgd =pgd_offset_k (address); -pud_t *pud =pud_offset (pgd, address); -pmd_t *pmd =pmd_offset (pud, address); -pte_t *pte =pte_offset_kernel (pmd, address); +int level; +pte_t *pte =lookup_address (address, &level); -if (pmd_large (*pmd)) { +if (!pte) { +printk (KERN_ERR "Error in %s: no pte for page 0x%08lx\n" , + __FUNCTION__, page); +return ; + } + +if (level ==PG_LEVEL_2M ) { +pmd_t *pmd = (pmd_t *)pte;set_pmd (pmd,__pmd (pmd_val (*pmd) & ~_PAGE_PRESENT)); -if (large) - *large =1 ; }else { +/* PG_LEVEL_4K */ set_pte (pte,__pte (pte_val (*pte) & ~_PAGE_PRESENT)); } +if (page_level) + *page_level = level; +__flush_tlb_one (page); } -static void disarm_kmmio_fault_page (unsigned long page,int *large) +static void disarm_kmmio_fault_page (unsigned long page,int *page_level) {unsigned long address = page &PAGE_MASK ; -pgd_t *pgd =pgd_offset_k (address); -pud_t *pud =pud_offset (pgd, address); -pmd_t *pmd =pmd_offset (pud, address); -pte_t *pte =pte_offset_kernel (pmd, address); +int level; +pte_t *pte =lookup_address (address, &level); -if (large && *large) { +if (!pte) { +printk (KERN_ERR "Error in %s: no pte for page 0x%08lx\n" , + __FUNCTION__, page); +return ; + } + +if (level ==PG_LEVEL_2M ) { +pmd_t *pmd = (pmd_t *)pte;set_pmd (pmd,__pmd (pmd_val (*pmd) | _PAGE_PRESENT)); - *large =0 ; }else { +/* PG_LEVEL_4K */ set_pte (pte,__pte (pte_val (*pte) | _PAGE_PRESENT)); } +if (page_level) + *page_level = level; +__flush_tlb_one (page); }
arch/x86/kernel/mmiotrace/mmio-mod.c +12 -7
@@ -120,19 +120,24 @@ static int write_marker(struct file *file, const char __user *buffer,static void print_pte (unsigned long address) { -pgd_t *pgd =pgd_offset_k (address); -pud_t *pud =pud_offset (pgd, address); -pmd_t *pmd =pmd_offset (pud, address); -if (pmd_large (*pmd)) { +int level; +pte_t *pte =lookup_address (address, &level); + +if (!pte) { +printk (KERN_ERR "Error in %s: no pte for page 0x%08lx\n" , + __FUNCTION__, address); +return ; + } + +if (level ==PG_LEVEL_2M ) { printk(KERN_EMERG MODULE_NAME ": 4MB pages are not ""currently supported: %lx\n" , address);BUG (); } printk(KERN_DEBUG MODULE_NAME ": pte for 0x%lx: 0x%lx 0x%lx\n", - address, -pte_val (*pte_offset_kernel (pmd, address)), - pte_val(*pte_offset_kernel(pmd, address)) & _PAGE_PRESENT); + address,pte_val (*pte), + pte_val(*pte) & _PAGE_PRESENT); }/*
arch/x86/mm/pageattr.c +1 -0
@@ -225,6 +225,7 @@ pte_t *lookup_address(unsigned long address, int *level)return pte_offset_kernel (pmd, address); } +EXPORT_SYMBOL (lookup_address);/* * Set the new pmd in all the pgds we know about:
Signature · 1 line
1.5.3.7
PATCH 2/4 x86 mmiotrace: fix relay-buffer-full flag for SMP
Relay has per-cpu buffers, but mmiotrace was using only a single flag for detecting buffer full/not-full transitions. The new code makes this per-cpu and actually counts missed events.
--- arch/x86/kernel/mmiotrace/mmio-mod.c | 26 ++++++++++++++++---------- 1 files changed, 16 insertions(+), 10 deletions(-)
arch/x86/kernel/mmiotrace/mmio-mod.c +16 -10
@@ -29,6 +29,7 @@#include <asm/pgtable.h> #include <linux/mmiotrace.h> #include <asm/e820.h> /* for ISA_START_ADDRESS */ +#include <asm/atomic.h> #include "kmmio.h" #include "pf_in.h"
@@ -47,9 +48,13 @@ struct trap_reason {int active_traces; }; +/* Accessed per-cpu. */ static struct trap_reason pf_reason[NR_CPUS ];static struct mm_io_header_rw cpu_trace[NR_CPUS ]; +/* Access to this is not per-cpu. */ +static atomic_t dropped[NR_CPUS ]; +static struct file_operations mmio_fops = { .owner =THIS_MODULE , };
@@ -57,7 +62,6 @@ static struct file_operations mmio_fops = {static const size_t subbuf_size =256 *1024 ;static struct rchan *chan;static struct dentry *dir; -static int suspended;/* XXX should this be per cpu? */ static struct proc_dir_entry *proc_marker_file;/* module parameters */
@@ -269,19 +273,21 @@ static void post(struct kmmio_probe *p, unsigned long condition,static int subbuf_start_handler (struct rchan_buf *buf,void *subbuf,void *prev_subbuf,size_t prev_padding) { +unsigned int cpu = buf->cpu; +atomic_t *drop = &dropped[cpu]; +int count;if (relay_buf_full (buf)) { -if (!suspended) { - suspended =1 ; - printk(KERN_ERR MODULE_NAME -": cpu %d buffer full!!!\n" , - smp_processor_id()); +if (atomic_inc_return (drop) ==1 ) { + printk(KERN_ERR MODULE_NAME ": cpu %d buffer full!\n", + cpu); }return 0 ; - }else if (suspended) { - suspended =0 ; + }else if ((count =atomic_read (drop))) { printk(KERN_ERR MODULE_NAME -": cpu %d buffer no longer full.\n" , - smp_processor_id()); +": cpu %d buffer no longer full, " +"missed %d events.\n" , + cpu, count); +atomic_sub (count, drop); }return 1 ;
Signature · 1 line
1.5.3.7
Re: [PATCH 2/4] x86 mmiotrace: fix relay-buffer-full flag for SMP
Quoted 22 lines · Pekka Paalanen
Relay has per-cpu buffers, but mmiotrace was using only a single flag for detecting buffer full/not-full transitions. The new code makes this per-cpu and actually counts missed events.
Signed-off-by Pekka Paalanen <[email protected]>--- arch/x86/kernel/mmiotrace/mmio-mod.c | 26 ++++++++++++++++---------- 1 files changed, 16 insertions(+), 10 deletions(-)arch/x86/kernel/mmiotrace/mmio-mod.c +1
-0@@ -29,6 +29,7 @@#include <asm/pgtable.h> #include <linux/mmiotrace.h> #include <asm/e820.h> /* for ISA_START_ADDRESS */ +#include <asm/atomic.h> #include "kmmio.h" #include "pf_in.h"
arch/x86/kernel/mmiotrace/mmio-mod.c struct trap_reason { · line 57
Quoted 10 lines
arch/x86/kernel/mmiotrace/mmio-mod.c +4
-0@@ -47,9 +48,13 @@ struct trap_reason {int active_traces; }; +/* Accessed per-cpu. */ static struct trap_reason pf_reason[NR_CPUS ];static struct mm_io_header_rw cpu_trace[NR_CPUS ]; +/* Access to this is not per-cpu. */ +static atomic_t dropped[NR_CPUS ]; +
Please dont introduce NR_CPUS new arrays, since people are working hard to zap them from kernel.
You probably can use a per_cpu variable ?
Thank you
Re: [PATCH 2/4] x86 mmiotrace: fix relay-buffer-full flag for SMP
Quoted 13 lines · On Tue, 05 Feb 2008 21:44:07 +0100 Eric Dumazet <[email protected]> wrote:
Quoted 12 lines · Pekka Paalanen
arch/x86/kernel/mmiotrace/mmio-mod.c +1
-0@@ -47,9 +48,13 @@ struct trap_reason {int active_traces; }; +/* Accessed per-cpu. */ static struct trap_reason pf_reason[NR_CPUS ];static struct mm_io_header_rw cpu_trace[NR_CPUS ];
Quoted 8 lines
arch/x86/kernel/mmiotrace/mmio-mod.c struct trap_reason { · line 57
Quoted 3 lines
arch/x86/kernel/mmiotrace/mmio-mod.c +3
-0+/* Access to this is not per-cpu. */ +static atomic_t dropped[NR_CPUS ]; +Please dont introduce
NR_CPUSnew arrays, since people are working hard to zap them from kernel.You probably can use a
per_cpuvariable ?
Yes, it would probably be more appropriate to use DEFINE_PER_CPU() for 'pf_reason' and 'cpu_trace', but I wasn't sure since the examples of DEFINE_PER_CPU I saw always had integers or pointers, not whole structs. Is it okay for whole structs?
'dropped' on the other hand is not accessed in per-cpu style, any cpu may access any element. DEFINE_PER_CPU is not valid here, is it?
Thanks for the note, I knew I should fix that at some point.
Signature · 2 lines
Pekka Paalanen http://www.iki.fi/pq/
Re: [PATCH 2/4] x86 mmiotrace: fix relay-buffer-full flag for SMP
Quoted 21 lines · Pekka Paalanen
Quoted 18 lines · On Tue, 05 Feb 2008 21:44:07 +0100 Eric Dumazet <[email protected]> wrote:
Quoted 5 lines · Pekka Paalanen
arch/x86/kernel/mmiotrace/mmio-mod.c +0
-0arch/x86/kernel/mmiotrace/mmio-mod.c struct trap_reason { · line 57
Quoted 10 lines
arch/x86/kernel/mmiotrace/mmio-mod.c +4
-0@@ -47,9 +48,13 @@ struct trap_reason {int active_traces; }; +/* Accessed per-cpu. */ static struct trap_reason pf_reason[NR_CPUS ];static struct mm_io_header_rw cpu_trace[NR_CPUS ]; +/* Access to this is not per-cpu. */ +static atomic_t dropped[NR_CPUS ]; +Please dont introduce
NR_CPUSnew arrays, since people are working hard to zap them from kernel.
Quoted 6 lines
Quoted 1 line
You probably can use a
per_cpuvariable ?Yes, it would probably be more appropriate to use
DEFINE_PER_CPU()for 'pf_reason' and 'cpu_trace', but I wasn't sure since the examples ofDEFINE_PER_CPUI saw always had integers or pointers, not whole structs. Is it okay for whole structs?
yes you can use a structure, you can check for example :
net/ipv4/route.c:static DEFINE_PER_CPU(struct rt_cache_stat, rt_cache_stat);
Quoted 3 lines
'dropped' on the other hand is not accessed in per-cpu style, any cpu may access any element.
DEFINE_PER_CPUis not valid here, is it?
It is valid, you can use per_cpu() accessor to get a pointer to a particular cpu data.
check net/ipv4/route.c for an example :
return &per_cpu(rt_cache_stat, cpu);
PATCH 1/1 x86 mmiotrace: Use percpu instead of arrays.
--- arch/x86/mm/kmmio.c | 27 +++++++++------- arch/x86/mm/mmio-mod.c | 80 +++++++++++++++++++++++++---------------------- 2 files changed, 58 insertions(+), 49 deletions(-)
arch/x86/mm/kmmio.c +15 -12
@@ -16,6 +16,7 @@#include <linux/uaccess.h> #include <linux/ptrace.h> #include <linux/preempt.h> +#include <linux/percpu.h> #include <asm/io.h> #include <asm/cacheflush.h> #include <asm/errno.h>
@@ -49,7 +50,8 @@ static unsigned int handler_registered;static struct list_head kmmio_page_table[KMMIO_PAGE_TABLE_SIZE ];static LIST_HEAD (kmmio_probes); -static struct kmmio_context kmmio_ctx[NR_CPUS ]; +/* Accessed per-cpu */ +static DEFINE_PER_CPU(struct kmmio_context, kmmio_ctx);static struct notifier_block nb_die = { .notifier_call = kmmio_die_notifier
@@ -173,8 +175,7 @@ static void disarm_kmmio_fault_page(unsigned long page, int *page_level)*/ static int kmmio_handler (struct pt_regs *regs,unsigned long addr) { -struct kmmio_context *ctx; -int cpu; +struct kmmio_context *ctx = &get_cpu_var (kmmio_ctx);/* * Preemption is now disabled to prevent process switch during
@@ -187,8 +188,6 @@ static int kmmio_handler(struct pt_regs *regs, unsigned long addr)* And that interrupt triggers a kmmio trap? */ preempt_disable (); - cpu =smp_processor_id (); - ctx = &kmmio_ctx[cpu];/* interrupts disabled and CPU-local data => atomicity guaranteed. */ if (ctx->active) {
@@ -199,7 +198,7 @@ static int kmmio_handler(struct pt_regs *regs, unsigned long addr)*/ printk (KERN_EMERG "mmiotrace: recursive probe hit on CPU %d, " "for address %lu. Ignoring.\n" , - cpu, addr); +smp_processor_id (), addr); goto no_kmmio; } ctx->active++;
@@ -231,6 +230,7 @@ static int kmmio_handler(struct pt_regs *regs, unsigned long addr)/* We hold lock, now we set present bit in PTE and single step. */ disarm_kmmio_fault_page (ctx->fpage->page,NULL ); +put_cpu_var (kmmio_ctx);return 1 ; no_kmmio_locked:
@@ -238,6 +238,7 @@ no_kmmio_locked: ctx->active--; no_kmmio:preempt_enable_no_resched (); +put_cpu_var (kmmio_ctx);/* page fault not handled by kmmio */ return 0 ; }
@@ -249,11 +250,11 @@ no_kmmio:*/ static int post_kmmio_handler (unsigned long condition,struct pt_regs *regs) { -int cpu =smp_processor_id (); -struct kmmio_context *ctx = &kmmio_ctx[cpu]; +int ret =0 ; +struct kmmio_context *ctx = &get_cpu_var (kmmio_ctx);if (!ctx->active) -return 0 ; + goto out;if (ctx->probe && ctx->probe->post_handler) ctx->probe->post_handler (ctx->probe, condition, regs);
@@ -273,10 +274,12 @@ static int post_kmmio_handler(unsigned long condition, struct pt_regs *regs)* will have TF set, in which case, continue the remaining processing * of do_debug, as if this is not a probe hit. */ -if (regs->flags &TF_MASK ) -return 0 ; +if (!(regs->flags &TF_MASK )) + ret =1 ; -return 1 ; +out: +put_cpu_var (kmmio_ctx); +return ret; }static int add_kmmio_fault_page (unsigned long page)
arch/x86/mm/mmio-mod.c +43 -37
182 lines not on this page · raw message
Signature · 1 line
1.5.3.7
PATCH 3/4 x86 mmiotrace: comment about user space ABI
In reply to Pekka Paalanen--- include/linux/mmiotrace.h | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-)
include/linux/mmiotrace.h +5 -1
@@ -3,6 +3,10 @@#include <asm/types.h> +/* +* If you change anything here, you must bump MMIO_VERSION. +* This is the relay data format for user space. +*/ #define MMIO_VERSION 0x04/* mm_io_header.type */
@@ -23,7 +27,7 @@ enum mm_io_opcode { /* payload type: */ };struct mm_io_header { -__u32 type; +__u32 type;/* see MMIO_* macros above */ __u32 sec;/* timestamp */ __u32 nsec;__u32 pid;/* PID of the process, or 0 for kernel core */
Signature · 1 line
1.5.3.7
PATCH 4/4 x86 mmiotrace: move files into arch/x86/mm/
In reply to Pekka PaalanenAs this patch is too big for the list, it can be found at:
http://jumi.lut.fi/~paalanen/scratch/mmio25-b/0004-x86-mmiotrace-move-files-into-arch-x86-mm.patch
The patch is 85 kB and Documentation/SubmittingPatches says I should not post it to the list if it exceeds 40 kB.
Ingo, I can also send this directly to your email, if you wish.
Signature · 2 lines
Pekka Paalanen http://www.iki.fi/pq/
Re: [PATCH 4/4] x86 mmiotrace: move files into arch/x86/mm/
Quoted 5 lines · On Tue, 5 Feb 2008 22:39:58 +0200 Pekka Paalanen wrote:
As this patch is too big for the list, it can be found at:
http://jumi.lut.fi/~paalanen/scratch/mmio25-b/0004-x86-mmiotrace-move-files-into-arch-x86-mm.patchThe patch is 85 kB and
Documentation/SubmittingPatchessays I should not post it to the list if it exceeds 40 kB.
Hm, we need to change that comment. The mailing list limit (for lkml) is now 400 KB AFAIK. Certainly we see plenty of patches that are larger than 40 KB.
Quoted 1 line
Ingo, I can also send this directly to your email, if you wish.
Please email it to the mailing list.
---
~Randy
Re: [PATCH 4/4] x86 mmiotrace: move files into arch/x86/mm/
Quoted 1 line
From
788b7aed85c8ef0256dc8e71ae981c4973a6a304Mon Sep 17 00:00:00 2001
From: Pekka Paalanen <[email protected]>
Date: Mon, 4 Feb 2008 21:26:41 +0200
Subject: [PATCH] x86 mmiotrace: move files into arch/x86/mm/.
---
Hi,
Sorry for being silent, but I am having trouble with my Internet service, I have been practically cut off since Wednesday.
Christoph, this patch shows my last version of mmiotrace, since it is moving all mmiotrace specific files, except include/linux/mmiotrace.h which contains just the relay data representation for user space. This does not include changes Ingo or someone else may have done.
When I catch up with x86/mm git, I can repost everything in one patch series. Given that my ISP's hardware permits...
About the page fault hook, I've been trying to ask what to do with it exactly, wishing for a pointer to an existing example or somesuch. So in lack of better ideas, I'll redo my very first idea of a single function pointer, and see what you think of it. Peter Zijlstra said on IRC it would never get accepted and suggested the list approach, which was the first I posted to LKML.
arch/x86/kernel/Makefile | 2 - arch/x86/kernel/mmiotrace/Makefile | 4 - arch/x86/kernel/mmiotrace/kmmio.c | 407 ---------------------- arch/x86/kernel/mmiotrace/kmmio.h | 58 --- arch/x86/kernel/mmiotrace/mmio-mod.c | 538 ----------------------------- arch/x86/kernel/mmiotrace/pf_in.c | 489 -------------------------- arch/x86/kernel/mmiotrace/pf_in.h | 39 -- arch/x86/kernel/mmiotrace/testmmiotrace.c | 77 ---- arch/x86/mm/Makefile | 5 + arch/x86/mm/kmmio.c | 407 ++++++++++++++++++++++ arch/x86/mm/kmmio.h | 58 +++ arch/x86/mm/mmio-mod.c | 538 +++++++++++++++++++++++++++++ arch/x86/mm/pf_in.c | 489 ++++++++++++++++++++++++++ arch/x86/mm/pf_in.h | 39 ++ arch/x86/mm/testmmiotrace.c | 77 ++++ 15 files changed, 1613 insertions(+), 1614 deletions(-) delete mode 100644 arch/x86/kernel/mmiotrace/Makefile delete mode 100644 arch/x86/kernel/mmiotrace/kmmio.c delete mode 100644 arch/x86/kernel/mmiotrace/kmmio.h delete mode 100644 arch/x86/kernel/mmiotrace/mmio-mod.c delete mode 100644 arch/x86/kernel/mmiotrace/pf_in.c delete mode 100644 arch/x86/kernel/mmiotrace/pf_in.h delete mode 100644 arch/x86/kernel/mmiotrace/testmmiotrace.c create mode 100644 arch/x86/mm/kmmio.c create mode 100644 arch/x86/mm/kmmio.h create mode 100644 arch/x86/mm/mmio-mod.c create mode 100644 arch/x86/mm/pf_in.c create mode 100644 arch/x86/mm/pf_in.h create mode 100644 arch/x86/mm/testmmiotrace.c
arch/x86/kernel/Makefile +0 -2
@@ -62,8 +62,6 @@ obj-$(CONFIG_KGDB) += kgdb.o kgdb-jmp_$(BITS).o obj-$(CONFIG_VM86) += vm86_32.o obj-$(CONFIG_EARLY_PRINTK) += early_printk.o -obj-$(CONFIG_MMIOTRACE) += mmiotrace/ - obj-$(CONFIG_HPET_TIMER) += hpet.o obj-$(CONFIG_K8_NB) += k8.o
arch/x86/kernel/mmiotrace/Makefile deleted +0 -4
@@ -1,4 +0,0 @@ -obj-$(CONFIG_MMIOTRACE) += mmiotrace.o -mmiotrace-objs := pf_in.o kmmio.o mmio-mod.o - -obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o
arch/x86/kernel/mmiotrace/kmmio.c deleted +0 -407
407 lines not on this page · raw message
arch/x86/kernel/mmiotrace/kmmio.h deleted +0 -58
58 lines not on this page · raw message
arch/x86/kernel/mmiotrace/mmio-mod.c deleted +0 -538
538 lines not on this page · raw message
arch/x86/kernel/mmiotrace/pf_in.c deleted +0 -489
489 lines not on this page · raw message
arch/x86/kernel/mmiotrace/pf_in.h deleted +0 -39
@@ -1,39 +0,0 @@ -/* -* Fault Injection Test harness (FI) -* Copyright (C) Intel Crop. -* -* This program is free software; you can redistribute it and/or -* modify it under the terms of the GNU General Public License -* as published by the Free Software Foundation; either version 2 -* of the License, or (at your option) any later version. -* -* This program is distributed in the hope that it will be useful, -* but WITHOUT ANY WARRANTY; without even the implied warranty of -* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -* GNU General Public License for more details. -* -* You should have received a copy of the GNU General Public License -* along with this program; if not, write to the Free Software -* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, -* USA. -* -*/ - -#ifndef __PF_H_ -#define __PF_H_ - -enum reason_type { -NOT_ME ,/* page fault is not in regions */ -NOTHING ,/* access others point in regions */ -REG_READ ,/* read from addr to reg */ -REG_WRITE ,/* write from reg to addr */ -IMM_WRITE ,/* write from imm to addr */ -OTHERS /* Other instructions can not intercept */ -}; - -enum reason_type get_ins_type (unsigned long ins_addr); -unsigned int get_ins_mem_width (unsigned long ins_addr); -unsigned long get_ins_reg_val (unsigned long ins_addr,struct pt_regs *regs); -unsigned long get_ins_imm_val (unsigned long ins_addr); - -#endif /* __PF_H_ */
arch/x86/kernel/mmiotrace/testmmiotrace.c deleted +0 -77
77 lines not on this page · raw message
arch/x86/mm/Makefile +5 -0
@@ -3,3 +3,8 @@ include ${srctree}/arch/x86/mm/Makefile_32 else include ${srctree}/arch/x86/mm/Makefile_64 endif + +obj-$(CONFIG_MMIOTRACE) += mmiotrace.o +mmiotrace-objs := pf_in.o kmmio.o mmio-mod.o + +obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o
arch/x86/mm/kmmio.c new file +407 -0
407 lines not on this page · raw message
arch/x86/mm/kmmio.h new file +58 -0
58 lines not on this page · raw message
arch/x86/mm/mmio-mod.c new file +538 -0
538 lines not on this page · raw message
arch/x86/mm/pf_in.c new file +489 -0
369 lines not on this page · raw message
arch/x86/mm/testmmiotrace.c new file +77 -0
77 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
1.5.3.7
Part of the diff is not shown here. View the raw message.
Re: [PATCH 4/4] x86 mmiotrace: move files into arch/x86/mm/
In reply to Pekka PaalanenPekka,
i've picked up your latest cleanups, but a fundamental worry still
remains: all the ugly 'custom pagefault handlers' stuff. Lets just have one clean direct callback to mmiotrace, ok?
Ingo
Re: [PATCH 4/4] x86 mmiotrace: move files into arch/x86/mm/
Quoted 6 lines · Ingo Molnar
Pekka,
i've picked up your latest cleanups, but a fundamental worry still
remains: all the ugly 'custom pagefault handlers' stuff. Lets just have one clean direct callback to mmiotrace, ok?
Yes, please. And any chance we you could resend the current version of the patch to lkml? I've lost a little track of how it looks currently.
RFC PATCH 1/1 x86: explicit call to mmiotrace in do_page_fault()
The custom page fault handler list is replaced with a single function pointer. All related functions and variables are renamed for mmiotrace.
--- arch/x86/Kconfig.debug | 14 ++++----- arch/x86/mm/fault.c | 66 +++++++++++++++++++++++---------------------- arch/x86/mm/kmmio.c | 14 +++++----- include/asm-x86/kdebug.h | 12 +++----- 4 files changed, 52 insertions(+), 54 deletions(-)
arch/x86/Kconfig.debug +6 -8
@@ -169,20 +169,18 @@ config IOMMU_LEAK Add a simple leak tracer to the IOMMU code. This is useful when you are debugging a buggy device driver that leaks IOMMU mappings. -config PAGE_FAULT_HANDLERS - bool "Custom page fault handlers" - depends on DEBUG_KERNEL - help - Allow the use of custom page fault handlers. A kernel module may - register a function that is called on every page fault. Custom - handlers are used by some debugging and reverse engineering tools. +config MMIOTRACE_HOOKS + bool + default n config MMIOTRACE tristate "Memory mapped IO tracing" - depends on DEBUG_KERNEL && PAGE_FAULT_HANDLERS && RELAY && DEBUG_FS + depends on DEBUG_KERNEL && RELAY && DEBUG_FS + select MMIOTRACE_HOOKS default n help This will build a kernel module called mmiotrace. + Making this a built-in is heavily discouraged. Mmiotrace traces Memory Mapped I/O access and is meant for debugging and reverse engineering. The kernel module offers wrapped
arch/x86/mm/fault.c +34 -32
94 lines not on this page · raw message
arch/x86/mm/kmmio.c +7 -7
@@ -51,10 +51,6 @@ static LIST_HEAD(kmmio_probes);static struct kmmio_context kmmio_ctx[NR_CPUS ]; -static struct pf_handler kmmio_pf_hook = { - .handler = kmmio_page_fault -}; -static struct notifier_block nb_die = { .notifier_call = kmmio_die_notifier };
@@ -77,7 +73,8 @@ void cleanup_kmmio(void)* kmmio_page_table, kmmio_probes */ if (handler_registered) { -unregister_page_fault_handler (&kmmio_pf_hook); +if (mmiotrace_unregister_pf (&kmmio_page_fault)) +BUG ();synchronize_rcu (); }unregister_die_notifier (&nb_die);
@@ -343,8 +340,11 @@ int register_kmmio_probe(struct kmmio_probe *p) }if (!handler_registered) { -register_page_fault_handler (&kmmio_pf_hook); - handler_registered++; +if (mmiotrace_register_pf (&kmmio_page_fault)) +printk (KERN_ERR "mmiotrace: Cannot register page " +"fault handler.\n" ); +else + handler_registered++; } out:
include/asm-x86/kdebug.h +5 -7
@@ -35,13 +35,11 @@ extern void dump_pagetable(unsigned long);extern unsigned long oops_begin (void );extern void oops_end (unsigned long ,struct pt_regs *,int signr); -struct pf_handler { -struct hlist_node hlist; -int (*handler)(struct pt_regs *regs,unsigned long error_code, -unsigned long address); -}; +typedef int (*pf_handler_func )(struct pt_regs *regs, +unsigned long error_code, +unsigned long address); -extern void register_page_fault_handler (struct pf_handler *new_pfh); -extern void unregister_page_fault_handler (struct pf_handler *old_pfh); +extern int mmiotrace_register_pf (pf_handler_func new_pfh); +extern int mmiotrace_unregister_pf (pf_handler_func old_pfh);#endif
Signature · 1 line
1.5.3.7
Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()
arch/x86/Kconfig.debug config IOMMU_LEAK · line 183
Quoted 4 lines
arch/x86/Kconfig.debug +1
-0default n help This will build a kernel module called mmiotrace. + Making this a built-in is heavily discouraged.
why is this? Wouldn't it be nice if distros just shipped with this in their kernel by default for example?
Quoted 15 lines
arch/x86/mm/fault.c +5
-7-#ifdef CONFIG_PAGE_FAULT_HANDLERS -static HLIST_HEAD (pf_handlers);/* protected by RCU */ -static DEFINE_SPINLOCK (pf_handlers_writer); +#ifdef CONFIG_MMIOTRACE_HOOKS +static pf_handler_func mmiotrace_pf_handler;/* protected by RCU */ +static DEFINE_SPINLOCK (mmiotrace_handler_lock); -void register_page_fault_handler (struct pf_handler *new_pfh) +int mmiotrace_register_pf (pf_handler_func new_pfh) { +int ret =0 ;unsigned long flags; -spin_lock_irqsave (&pf_handlers_writer, flags); -hlist_add_head_rcu (&new_pfh->hlist, &pf_handlers); -spin_unlock_irqrestore (&pf_handlers_writer, flags);
arch/x86/mm/fault.c line 68
Quoted 10 lines
arch/x86/mm/fault.c +8
-1+spin_lock_irqsave (&mmiotrace_handler_lock, flags); +if (mmiotrace_pf_handler) + ret = -EBUSY ; +else + mmiotrace_pf_handler = new_pfh; +spin_unlock_irqrestore (&mmiotrace_handler_lock, flags); +return ret; } -EXPORT_SYMBOL_GPL (register_page_fault_handler); +EXPORT_SYMBOL_GPL (mmiotrace_register_pf);
why not use a weak function instead? should be a ton simpler.
Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()
Quoted 7 lines · On Sat, 9 Feb 2008 10:01:33 -0800 Arjan van de Ven <[email protected]> wrote:
arch/x86/Kconfig.debug config IOMMU_LEAK · line 183
Quoted 4 lines
arch/x86/Kconfig.debug +1
-0default n help This will build a kernel module called mmiotrace. + Making this a built-in is heavily discouraged.why is this? Wouldn't it be nice if distros just shipped with this in their kernel by default for example?
It's currently not too nice as a built-in. Originally I did have it depend on 'm' but maybe Ingo removed it? It's not depending on 'm' anymore in x86/mm git tree Kconfig.
By default it reserves 32 MB/cpu of relay buffers during module init. It's this big based on experimentation with the nvidia proprietary driver. There is no other way than unloading the module to free the buffers, or even to clear them. This means that every time you start the user space logger application, it gets old data first. Also poking the module parameters via /sys is not allowed.
So it's not actually broken as far as I know, it's just not nice IMHO. And I've never tried it as built-in. I don't think anyone ever has. I do have a plan to make it a nicer built-in, but it's a long way.
Quoted 4 lines
Quoted 4 lines
arch/x86/mm/fault.c +3
-0+#ifdef CONFIG_MMIOTRACE_HOOKS +static pf_handler_func mmiotrace_pf_handler;/* protected by RCU */ +static DEFINE_SPINLOCK (mmiotrace_handler_lock);
Quoted 15 lines
arch/x86/mm/fault.c line 68
Quoted 13 lines
arch/x86/mm/fault.c +10
-0+int mmiotrace_register_pf (pf_handler_func new_pfh) { +int ret =0 ;unsigned long flags; +spin_lock_irqsave (&mmiotrace_handler_lock, flags); +if (mmiotrace_pf_handler) + ret = -EBUSY ; +else + mmiotrace_pf_handler = new_pfh; +spin_unlock_irqrestore (&mmiotrace_handler_lock, flags); +return ret; } +EXPORT_SYMBOL_GPL (mmiotrace_register_pf);why not use a weak function instead? should be a ton simpler.
Ookay, this is the first time I hear about weak functions, and that I could accomplish something like this with them. I'll be googling around. These are completely new beasts to me.
Thanks for the keywords, finally I have a clue to follow.
Signature · 2 lines
Pekka Paalanen http://www.iki.fi/pq/
Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()
Hi Pekka,
Quoted 6 lines · Pekka Paalanen
By default it reserves 32 MB/cpu of relay buffers during module init. It's this big based on experimentation with the nvidia proprietary driver. There is no other way than unloading the module to free the buffers, or even to clear them. This means that every time you start the user space logger application, it gets old data first. Also poking the module parameters via /sys is not allowed.
So how is mmiotrace different from, say, blktrace that has an ioctl() to setup/teardown the relay channels (for mmiotrace it can be a "status" file that does the setup/teardown when written some appropriate values)?
Pekka
Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()
Quoted 10 lines · On Sat, 9 Feb 2008 20:56:07 +0200 "Pekka Enberg" <[email protected]> wrote:
Hi Pekka,
Quoted 6 lines · Pekka Paalanen
By default it reserves 32 MB/cpu of relay buffers during module init. It's this big based on experimentation with the nvidia proprietary driver. There is no other way than unloading the module to free the buffers, or even to clear them. This means that every time you start the user space logger application, it gets old data first. Also poking the module parameters via /sys is not allowed.
Quoted 4 lines
So how is mmiotrace different from, say, blktrace that has an
ioctl()to setup/teardown the relay channels (for mmiotrace it can be a "status" file that does the setup/teardown when written some appropriate values)?
No difference. I just have to do it at some point. I can't work on this every day and when it comes to kernel, I'm a complete newbie. So things will take time. If people want that I fix this before mmiotrace goes into mainline, then so be it.
blktrace, thanks for the hint, I should take a look at that.
Signature · 2 lines
Pekka Paalanen http://www.iki.fi/pq/
Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()
Hi,
Quoted 4 lines · Pekka Paalanen
No difference. I just have to do it at some point. I can't work on this every day and when it comes to kernel, I'm a complete newbie. So things will take time. If people want that I fix this before mmiotrace goes into mainline, then so be it.
Not my call but please keep in mind that once it's in the kernel we can't go breaking the ABI (well, we might as this is strictly a debugging mechanism, but still). Anyway, it shouldn't be that hard to implement and there are plenty of examples in the kernel and you can always as for help if you hit a brick wall ;-).
Pekka
Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()
In reply to Arjan van de VenQuoted 7 lines · Arjan van de Ven
arch/x86/Kconfig.debug config IOMMU_LEAK · line 183
Quoted 4 lines
arch/x86/Kconfig.debug +1
-0default n help This will build a kernel module called mmiotrace. + Making this a built-in is heavily discouraged.why is this? Wouldn't it be nice if distros just shipped with this in their kernel by default for example?
so default m?
Quoted 19 lines
Quoted 19 lines
arch/x86/mm/fault.c +9
-7-#ifdef CONFIG_PAGE_FAULT_HANDLERS -static HLIST_HEAD (pf_handlers);/* protected by RCU */ -static DEFINE_SPINLOCK (pf_handlers_writer); +#ifdef CONFIG_MMIOTRACE_HOOKS +static pf_handler_func mmiotrace_pf_handler;/* protected by RCU */ +static DEFINE_SPINLOCK (mmiotrace_handler_lock); -void register_page_fault_handler (struct pf_handler *new_pfh) +int mmiotrace_register_pf (pf_handler_func new_pfh) { +int ret =0 ;unsigned long flags; -spin_lock_irqsave (&pf_handlers_writer, flags); -hlist_add_head_rcu (&new_pfh->hlist, &pf_handlers); -spin_unlock_irqrestore (&pf_handlers_writer, flags); +spin_lock_irqsave (&mmiotrace_handler_lock, flags); +if (mmiotrace_pf_handler) + ret = -EBUSY ; +else
Quoted 8 lines
arch/x86/mm/fault.c line 68
Quoted 6 lines
arch/x86/mm/fault.c +4
-1+ mmiotrace_pf_handler = new_pfh; +spin_unlock_irqrestore (&mmiotrace_handler_lock, flags); +return ret; } -EXPORT_SYMBOL_GPL (register_page_fault_handler); +EXPORT_SYMBOL_GPL (mmiotrace_register_pf);why not use a weak function instead? should be a ton simpler.
AFAIC weak functions are a built time thing, not fixed up on module load. So I'm not seeing the applicability there.
Also, it would force a function call, even for the unlikely case the tracer is set.
Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()
In reply to Pekka PaalanenQuoted 24 lines · Pekka Paalanen
arch/x86/mm/fault.c +15
-0+int mmiotrace_register_pf (pf_handler_func new_pfh) { +int ret =0 ;unsigned long flags; +spin_lock_irqsave (&mmiotrace_handler_lock, flags); +if (mmiotrace_pf_handler) + ret = -EBUSY ; +else + mmiotrace_pf_handler = new_pfh; +spin_unlock_irqrestore (&mmiotrace_handler_lock, flags); +return ret; } +EXPORT_SYMBOL_GPL (mmiotrace_register_pf);/** +* mmiotrace_unregister_pf: * The caller must ensure @old_pfh is not in use anymore before freeing it. +* This function does not guarantee it. The handler function pointer is +* protected by RCU, so you can do this by e.g. calling synchronize_rcu(). */ +int mmiotrace_unregister_pf (pf_handler_func old_pfh) { +int ret =0 ;
arch/x86/mm/fault.c
Quoted 10 lines
arch/x86/mm/fault.c +7
-0unsigned long flags; +spin_lock_irqsave (&mmiotrace_handler_lock, flags); +if (mmiotrace_pf_handler != old_pfh) + ret = -EPERM ; +else + mmiotrace_pf_handler =NULL ; +spin_unlock_irqrestore (&mmiotrace_handler_lock, flags); +return ret; }
So here you set/unset mmiotrace_pf_handler under protection of a spinlock.
arch/x86/mm/fault.c line 103
Quoted 13 lines
/* returns non-zero if
do_page_fault()should return */arch/x86/mm/fault.c +6
-0+static inline int call_mmiotrace (struct pt_regs *regs, +unsigned long error_code, +unsigned long address) { +#ifdef CONFIG_MMIOTRACE_HOOKS int ret =0 ;rcu_read_lock (); +if (mmiotrace_pf_handler) + ret =mmiotrace_pf_handler (regs, error_code, address);rcu_read_unlock ();return ret;#else
But here you use it under rcu_read_lock(). This doesn't match.
RFC PATCH v2 1/1 x86: explicit call to mmiotrace in do_page_fault()
The custom page fault handler list is replaced with a single function pointer. All related functions and variables are renamed for mmiotrace.
---
Quoted 2 lines · On Sat, 09 Feb 2008 19:39:30 +0100 Peter Zijlstra <[email protected]>…
So here you set/unset
mmiotrace_pf_handlerunder protection of a spinlock.
..
Quoted 1 line
But here you use it under
rcu_read_lock(). This doesn't match.
Thanks, I fixed it.
Sorry Jan and Pavel, I forgot to CC you in the first go of this patch. If this makes it into mainline, I don't think it will be left there for many kernel versions. I plan to make kmmio.h as the API towards modules in the future, and the page fault callback will "disappear". I think madwifi could use just fine the kmmio.h functions, if it works basically just like mmiotrace.
arch/x86/Kconfig.debug | 14 ++++----- arch/x86/mm/fault.c | 68 ++++++++++++++++++++++++--------------------- arch/x86/mm/kmmio.c | 14 +++++----- include/asm-x86/kdebug.h | 12 +++----- 4 files changed, 54 insertions(+), 54 deletions(-)
arch/x86/Kconfig.debug +6 -8
@@ -169,20 +169,18 @@ config IOMMU_LEAK Add a simple leak tracer to the IOMMU code. This is useful when you are debugging a buggy device driver that leaks IOMMU mappings. -config PAGE_FAULT_HANDLERS - bool "Custom page fault handlers" - depends on DEBUG_KERNEL - help - Allow the use of custom page fault handlers. A kernel module may - register a function that is called on every page fault. Custom - handlers are used by some debugging and reverse engineering tools. +config MMIOTRACE_HOOKS + bool + default n config MMIOTRACE tristate "Memory mapped IO tracing" - depends on DEBUG_KERNEL && PAGE_FAULT_HANDLERS && RELAY && DEBUG_FS + depends on DEBUG_KERNEL && RELAY && DEBUG_FS + select MMIOTRACE_HOOKS default n help This will build a kernel module called mmiotrace. + Making this a built-in is heavily discouraged. Mmiotrace traces Memory Mapped I/O access and is meant for debugging and reverse engineering. The kernel module offers wrapped
arch/x86/mm/fault.c +36 -32
96 lines not on this page · raw message
arch/x86/mm/kmmio.c +7 -7
@@ -51,10 +51,6 @@ static LIST_HEAD(kmmio_probes);static struct kmmio_context kmmio_ctx[NR_CPUS ]; -static struct pf_handler kmmio_pf_hook = { - .handler = kmmio_page_fault -}; -static struct notifier_block nb_die = { .notifier_call = kmmio_die_notifier };
@@ -77,7 +73,8 @@ void cleanup_kmmio(void)* kmmio_page_table, kmmio_probes */ if (handler_registered) { -unregister_page_fault_handler (&kmmio_pf_hook); +if (mmiotrace_unregister_pf (&kmmio_page_fault)) +BUG ();synchronize_rcu (); }unregister_die_notifier (&nb_die);
@@ -343,8 +340,11 @@ int register_kmmio_probe(struct kmmio_probe *p) }if (!handler_registered) { -register_page_fault_handler (&kmmio_pf_hook); - handler_registered++; +if (mmiotrace_register_pf (&kmmio_page_fault)) +printk (KERN_ERR "mmiotrace: Cannot register page " +"fault handler.\n" ); +else + handler_registered++; } out:
include/asm-x86/kdebug.h +5 -7
@@ -35,13 +35,11 @@ extern void dump_pagetable(unsigned long);extern unsigned long oops_begin (void );extern void oops_end (unsigned long ,struct pt_regs *,int signr); -struct pf_handler { -struct hlist_node hlist; -int (*handler)(struct pt_regs *regs,unsigned long error_code, -unsigned long address); -}; +typedef int (*pf_handler_func )(struct pt_regs *regs, +unsigned long error_code, +unsigned long address); -extern void register_page_fault_handler (struct pf_handler *new_pfh); -extern void unregister_page_fault_handler (struct pf_handler *old_pfh); +extern int mmiotrace_register_pf (pf_handler_func new_pfh); +extern int mmiotrace_unregister_pf (pf_handler_func old_pfh);#endif
Signature · 1 line
1.5.3.7
Re: [RFC PATCH v2] x86: explicit call to mmiotrace in do_page_fault()
Quoted 6 lines · Pekka Paalanen
Sorry Jan and Pavel, I forgot to CC you in the first go of this patch. If this makes it into mainline, I don't think it will be left there for many kernel versions. I plan to make
kmmio.has the API towards modules in the future, and the page fault callback will "disappear". I think madwifi could use just fine thekmmio.hfunctions, if it works basically just like mmiotrace.
In fact, the tracing version of MadWifi has a directory called mmiotrace with the files lifted from nouveau. I guess if will just stay in sync with the upstream.
Many thanks for pushing it into the kernel.
Signature · 2 lines
Regards, Pavel Roskin
Re: [RFC PATCH v2] x86: explicit call to mmiotrace in do_page_fault()
Hello,
it seems 2.6.25-rc1 is out and mmiotrace did not make it. Thank you all for your feedback.
Ingo, you can throw away all my patches. I will start from scratch and come back when I have made mmiotrace built-in-able and got rid of the page fault hook. It should make a nicer patch set as a whole.
Signature · 2 lines
Pekka Paalanen http://www.iki.fi/pq/
Re: [PATCH 1/4] x86 mmiotrace: use lookup_address()
In reply to Pekka PaalanenQuoted 4 lines · Pekka Paalanen
Use
lookup_address()frompageattr.cinstead of doing the same manually. Also had toEXPORT_SYMBOL(lookup_address)to make this work for modules. This also fixes "undefined symbol 'init_mm'" compile error forx86_32.
This should be a _GPL export for sure.
Re: [PATCH 1/4] x86 mmiotrace: use lookup_address()
Quoted 7 lines · Christoph Hellwig
Quoted 4 lines · Pekka Paalanen
Use
lookup_address()frompageattr.cinstead of doing the same manually. Also had toEXPORT_SYMBOL(lookup_address)to make this work for modules. This also fixes "undefined symbol 'init_mm'" compile error forx86_32.This should be a
_GPLexport for sure.
i've changed it to that.
Ingo
RFC PATCH v2 1/1 x86: mmiotrace - trace memory mapped IO
In reply to Pekka PaalanenQuoted 4 lines
Quoted 4 lines
could you please send us a patch for the whole mmiotrace kernel-side feature, so that we can have a look at the general structure of this? (and the interaction with
change_page_attr(), etc.) Even if it's not functional (and wont even build/boot) at the moment. Thanks,
And here's mmiotrace. This works for me (amd64), but not for a test user who has 32-bit x86. We have not had the chance to debug it yet. The symptom is "everything locked up" after insmodding testmmiotrace.ko, targeting the mid-point of his VRAM. And there's the "undefined symbol init_mm" problem on 32-bit, but we just put the export back in (not included in these patches).
No-one has tried this on SMP. I have a Core 2 Duo laptop where I will test this when I get the chance.
Quoted 1 line
From
10000bf67af4900b75a56cb47726d7b63d11d7e8Mon Sep 17 00:00:00 2001
From: Pekka Paalanen <[email protected]>
Date: Wed, 30 Jan 2008 21:51:51 +0200
Subject: [PATCH] x86: mmiotrace - trace memory mapped IO
Mmiotrace is a tool for trapping memory mapped IO (MMIO) accesses within the kernel. It is used for debugging and especially for reverse engineering evil binary drivers.
Mmiotrace works by wrapping the ioremap family of kernel functions and marking the returned pages as not present. Access to the IO memory triggers a page fault, which will be handled by mmiotrace's custom page fault handler. This will single-step the faulted instruction with the MMIO page marked as present. Access logs are directed to user space via relay and debug_fs.
This page fault approach is necessary, because binary drivers have readl/writel etc. calls inlined and therefore extremely difficult to trap with with e.g. kprobes.
This patch depends on the custom page fault handlers patch.
--- arch/x86/Kconfig.debug | 28 ++ arch/x86/kernel/Makefile | 2 + arch/x86/kernel/mmiotrace/Makefile | 4 + arch/x86/kernel/mmiotrace/kmmio.c | 391 +++++++++++++++++++++ arch/x86/kernel/mmiotrace/kmmio.h | 58 ++++ arch/x86/kernel/mmiotrace/mmio-mod.c | 527 +++++++++++++++++++++++++++++ arch/x86/kernel/mmiotrace/pf_in.c | 489 ++++++++++++++++++++++++++ arch/x86/kernel/mmiotrace/pf_in.h | 39 +++ arch/x86/kernel/mmiotrace/testmmiotrace.c | 77 +++++ include/linux/mmiotrace.h | 62 ++++ 10 files changed, 1677 insertions(+), 0 deletions(-) create mode 100644 arch/x86/kernel/mmiotrace/Makefile create mode 100644 arch/x86/kernel/mmiotrace/kmmio.c create mode 100644 arch/x86/kernel/mmiotrace/kmmio.h create mode 100644 arch/x86/kernel/mmiotrace/mmio-mod.c create mode 100644 arch/x86/kernel/mmiotrace/pf_in.c create mode 100644 arch/x86/kernel/mmiotrace/pf_in.h create mode 100644 arch/x86/kernel/mmiotrace/testmmiotrace.c create mode 100644 include/linux/mmiotrace.h
arch/x86/Kconfig.debug +28 -0
@@ -146,6 +146,34 @@ config PAGE_FAULT_HANDLERS register a function that is called on every page fault. Custom handlers are used by some debugging and reverse engineering tools. +config MMIOTRACE + tristate "Memory mapped IO tracing" + depends on DEBUG_KERNEL && m + depends on PAGE_FAULT_HANDLERS && RELAY && DEBUG_FS + default n + help + This will build a kernel module called mmiotrace. + + Mmiotrace traces Memory Mapped I/O access and is meant for debugging + and reverse engineering. The kernel module offers wrapped + versions of the ioremap family of functions. The driver to be traced + must be modified to call these wrappers. A user space program is + required to collect the MMIO data. + + See http://nouveau.freedesktop.org/wiki/MmioTrace + If you are not helping to develop drivers, say N. + +config MMIOTRACE_TEST + tristate "Test module for mmiotrace" + depends on MMIOTRACE && m + default n + help + This is a dumb module for testing mmiotrace. It is very dangerous + as it will write garbage to IO memory starting at a given address. + However, it should be safe to use on e.g. unused portion of VRAM. + + Say N, unless you absolutely know what you are doing. + # # IO delay types: #
arch/x86/kernel/Makefile +2 -0
@@ -61,6 +61,8 @@ obj-$(CONFIG_KGDB) += kgdb.o kgdb-jmp_$(BITS).o obj-$(CONFIG_VM86) += vm86_32.o obj-$(CONFIG_EARLY_PRINTK) += early_printk.o +obj-$(CONFIG_MMIOTRACE) += mmiotrace/ + obj-$(CONFIG_HPET_TIMER) += hpet.o obj-$(CONFIG_K8_NB) += k8.o
arch/x86/kernel/mmiotrace/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +obj-$(CONFIG_MMIOTRACE) += mmiotrace.o +mmiotrace-objs := pf_in.o kmmio.o mmio-mod.o + +obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o
arch/x86/kernel/mmiotrace/kmmio.c new file +391 -0
391 lines not on this page · raw message
arch/x86/kernel/mmiotrace/kmmio.h new file +58 -0
58 lines not on this page · raw message
arch/x86/kernel/mmiotrace/mmio-mod.c new file +527 -0
527 lines not on this page · raw message
arch/x86/kernel/mmiotrace/pf_in.c new file +489 -0
489 lines not on this page · raw message
arch/x86/kernel/mmiotrace/pf_in.h new file +39 -0
@@ -0,0 +1,39 @@ +/* +* Fault Injection Test harness (FI) +* Copyright (C) Intel Crop. +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License +* as published by the Free Software Foundation; either version 2 +* of the License, or (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, +* USA. +* +*/ + +#ifndef __PF_H_ +#define __PF_H_ + +enum reason_type { +NOT_ME ,/* page fault is not in regions */ +NOTHING ,/* access others point in regions */ +REG_READ ,/* read from addr to reg */ +REG_WRITE ,/* write from reg to addr */ +IMM_WRITE ,/* write from imm to addr */ +OTHERS /* Other instructions can not intercept */ +}; + +enum reason_type get_ins_type (unsigned long ins_addr); +unsigned int get_ins_mem_width (unsigned long ins_addr); +unsigned long get_ins_reg_val (unsigned long ins_addr,struct pt_regs *regs); +unsigned long get_ins_imm_val (unsigned long ins_addr); + +#endif /* __PF_H_ */
arch/x86/kernel/mmiotrace/testmmiotrace.c new file +77 -0
77 lines not on this page · raw message
include/linux/mmiotrace.h new file +62 -0
62 lines not on this page · raw message
Signature · 1 line
1.5.3.7
Re: [RFC PATCH v2] x86: mmiotrace - trace memory mapped IO
Quoted 13 lines · On Thu, 31 Jan 2008 18:16:30 +0200 Pekka Paalanen <[email protected]> wrote:
Quoted 5 lines
Quoted 5 lines
could you please send us a patch for the whole mmiotrace kernel-side feature, so that we can have a look at the general structure of this? (and the interaction with
change_page_attr(), etc.) Even if it's not functional (and wont even build/boot) at the moment. Thanks,And here's mmiotrace. This works for me (amd64), but not for a test user who has 32-bit x86. We have not had the chance to debug it yet. The symptom is "everything locked up" after insmodding testmmiotrace.ko, targeting the mid-point of his VRAM. And there's the "undefined symbol
init_mm" problem on 32-bit, but we just put the export back in (not included in these patches).
Quoted 2 lines
No-one has tried this on SMP. I have a Core 2 Duo laptop where I will test this when I get the chance.
+static void print_pte(unsigned long address)
Quoted 16 lines
arch/x86/kernel/mmiotrace/mmio-mod.c new file +7
-0+{ +pgd_t *pgd =pgd_offset_k (address); +pud_t *pud =pud_offset (pgd, address); +pmd_t *pmd =pmd_offset (pud, address); +if (pmd_large (*pmd)) { + printk(KERN_EMERG MODULE_NAME ": 4MB pages are not " + "currentlysupported: %lx\n",
arch/x86/kernel/mmiotrace/mmio-mod.c new file +4
-0+ address); +BUG (); + } + printk(KERN_DEBUG MODULE_NAME ": pte for 0x%lx: 0x%lx0x%lx\n",
arch/x86/kernel/mmiotrace/mmio-mod.c new file +3
-0+ address, +pte_val (*pte_offset_kernel (pmd, address)), + pte_val(*pte_offset_kernel(pmd, address)) &
Quoted 1 line
_PAGE_PRESENT); +}
ok this wants to use some of the page_address() and other helpers from pageattr.c; that should be easy to solve during integration
(I can even see a more generic version of this function moving there)
Quoted 7 lines
arch/x86/kernel/mmiotrace/mmio-mod.c new file +3
-0+/* file_create() callback. Creates relay file in debugfs. */ +static struct dentry *create_buf_file_handler (const char *filename, + struct dentry*parent,
(quoted diff) +2
-0+ int mode, + struct rchan_buf*buf,
arch/x86/kernel/mmiotrace/mmio-mod.c line 299
Quoted 10 lines
arch/x86/kernel/mmiotrace/mmio-mod.c new file +10
-0+ int *is_global) +{ +struct dentry *buf_file; + + mmio_fops.read = relay_file_operations.read; + mmio_fops.open = relay_file_operations.open; + mmio_fops.poll = relay_file_operations.poll; + mmio_fops.mmap = relay_file_operations.mmap; + mmio_fops.release = relay_file_operations.release; + mmio_fops.splice_read = relay_file_operations.splice_read;
can't we do this at compile time?
Quoted 20 lines
(quoted diff) +2
-0+ +void __iomem *ioremap_cache_trace(unsigned long offset, unsignedlong size) +{
(quoted diff) +2
-0+ void __iomem *p = ioremap_cache(offset, size); + printk(KERN_DEBUG MODULE_NAME ": ioremap_cache(0x%lx, 0x%lx)= %p\n",
(quoted diff) +1
-0+ offset,size, p);
arch/x86/kernel/mmiotrace/mmio-mod.c new file +6
-0+ioremap_trace_core (offset, size, p); +return p; +} +EXPORT_SYMBOL (ioremap_cache_trace); + +void __iomem *ioremap_nocache_trace(unsigned long offset, unsignedlong size) +{
(quoted diff) +2
-0+ void __iomem *p = ioremap_nocache(offset, size); + printk(KERN_DEBUG MODULE_NAME ": ioremap_nocache(0x%lx,0x%lx) = %p\n",
(quoted diff) +1
-0+ offset,size, p);
arch/x86/kernel/mmiotrace/mmio-mod.c line 419
Quoted 4 lines
arch/x86/kernel/mmiotrace/mmio-mod.c new file +4
-0+ioremap_trace_core (offset, size, p); +return p; +} +EXPORT_SYMBOL (ioremap_nocache_trace);
I would rather keep the split I think and make the trace call explicit
or.. even nicer, we could go fancy and allow the mmio tracer to "subscribe" to, say, a PCI bar, and auto-enable tracing when the bar gets ioremap'd
about the isntruction decoding.. would be nice if we can share that with kvm so that we can debug that stuff once ;-)
other than that I think the code is quite nice already, seems getting this merge ready isn't going to be an enormous deal in terms of complexity
Re: [RFC PATCH v2] x86: mmiotrace - trace memory mapped IO
Quoted 9 lines · On Thu, 31 Jan 2008 08:29:06 -0800 Arjan van de Ven <[email protected]> wrote:
Quoted 6 lines · On Thu, 31 Jan 2008 18:16:30 +0200 Pekka Paalanen <[email protected]> wrote:
And here's mmiotrace. This works for me (amd64), but not for a test user who has 32-bit x86. We have not had the chance to debug it yet. The symptom is "everything locked up" after insmodding testmmiotrace.ko, targeting the mid-point of his VRAM. And there's the "undefined symbol
init_mm" problem on 32-bit, but we just put the export back in (not included in these patches).
This problem seems to have been due to kgdb being enabled. Quote from Jaap Stolk, the test user:
"(test)mmiotrace will trigger wait for serial debugger if kgdb and CONFIG_KGDB_ATTACH_WAIT is enabled."
After disabling it, mmiotrace ran fine.
Quoted 2 lines
Quoted 2 lines
No-one has tried this on SMP. I have a Core 2 Duo laptop where I will test this when I get the chance.
I tested, worked just fine. Well, the test module does not really test simultaneous iomem access. And I have an issue with following relay buffers' full/not-full status, I'm looking into that now.
Quoted 3 lines
ok this wants to use some of the
page_address()and other helpers frompageattr.c; that should be easy to solve during integration
(I can even see a more generic version of this function moving there)
page_address() seems to be dealing with struct page, someone said we usually don't have that (ioremap).
Is this what you had in mind? I had to add an export for lookup_address() to make this work.
arch/x86/kernel/mmiotrace/kmmio.c | 46 +++++++++++++++++++++++----------- arch/x86/kernel/mmiotrace/mmio-mod.c | 19 +++++++++----- arch/x86/mm/pageattr.c | 1 + 3 files changed, 44 insertions(+), 22 deletions(-)
arch/x86/kernel/mmiotrace/kmmio.c +31 -15
@@ -20,6 +20,7 @@#include <asm/cacheflush.h> #include <asm/errno.h> #include <asm/tlbflush.h> +#include <asm/pgtable.h> #include "kmmio.h"
@@ -117,40 +118,55 @@ static struct kmmio_fault_page *get_kmmio_fault_page(unsigned long page)return NULL ; } -static void arm_kmmio_fault_page (unsigned long page,int *large) +static void arm_kmmio_fault_page (unsigned long page,int *page_level) {unsigned long address = page &PAGE_MASK ; -pgd_t *pgd =pgd_offset_k (address); -pud_t *pud =pud_offset (pgd, address); -pmd_t *pmd =pmd_offset (pud, address); -pte_t *pte =pte_offset_kernel (pmd, address); +int level; +pte_t *pte =lookup_address (address, &level); -if (pmd_large (*pmd)) { +if (!pte) { +printk (KERN_ERR "Error in %s: no pte for page 0x%08lx\n" , + __FUNCTION__, page); +return ; + } + +if (level ==PG_LEVEL_2M ) { +pmd_t *pmd = (pmd_t *)pte;set_pmd (pmd,__pmd (pmd_val (*pmd) & ~_PAGE_PRESENT)); -if (large) - *large =1 ; }else { +/* PG_LEVEL_4K */ set_pte (pte,__pte (pte_val (*pte) & ~_PAGE_PRESENT)); } +if (page_level) + *page_level = level; +__flush_tlb_one (page); } -static void disarm_kmmio_fault_page (unsigned long page,int *large) +static void disarm_kmmio_fault_page (unsigned long page,int *page_level) {unsigned long address = page &PAGE_MASK ; -pgd_t *pgd =pgd_offset_k (address); -pud_t *pud =pud_offset (pgd, address); -pmd_t *pmd =pmd_offset (pud, address); -pte_t *pte =pte_offset_kernel (pmd, address); +int level; +pte_t *pte =lookup_address (address, &level); -if (large && *large) { +if (!pte) { +printk (KERN_ERR "Error in %s: no pte for page 0x%08lx\n" , + __FUNCTION__, page); +return ; + } + +if (level ==PG_LEVEL_2M ) { +pmd_t *pmd = (pmd_t *)pte;set_pmd (pmd,__pmd (pmd_val (*pmd) | _PAGE_PRESENT)); - *large =0 ; }else { +/* PG_LEVEL_4K */ set_pte (pte,__pte (pte_val (*pte) | _PAGE_PRESENT)); } +if (page_level) + *page_level = level; +__flush_tlb_one (page); }
arch/x86/kernel/mmiotrace/mmio-mod.c +12 -7
@@ -120,19 +120,24 @@ static int write_marker(struct file *file, const char __user *buffer,static void print_pte (unsigned long address) { -pgd_t *pgd =pgd_offset_k (address); -pud_t *pud =pud_offset (pgd, address); -pmd_t *pmd =pmd_offset (pud, address); -if (pmd_large (*pmd)) { +int level; +pte_t *pte =lookup_address (address, &level); + +if (!pte) { +printk (KERN_ERR "Error in %s: no pte for page 0x%08lx\n" , + __FUNCTION__, address); +return ; + } + +if (level ==PG_LEVEL_2M ) { printk(KERN_EMERG MODULE_NAME ": 4MB pages are not ""currently supported: %lx\n" , address);BUG (); } printk(KERN_DEBUG MODULE_NAME ": pte for 0x%lx: 0x%lx 0x%lx\n", - address, -pte_val (*pte_offset_kernel (pmd, address)), - pte_val(*pte_offset_kernel(pmd, address)) & _PAGE_PRESENT); + address,pte_val (*pte), + pte_val(*pte) & _PAGE_PRESENT); }/*
arch/x86/mm/pageattr.c +1 -0
@@ -194,6 +194,7 @@ pte_t *lookup_address(unsigned long address, int *level) *level =PG_LEVEL_4K ;return pte_offset_kernel (pmd, address); } +EXPORT_SYMBOL (lookup_address);static void __set_pmd_pte (pte_t *kpte,unsigned long address,pte_t pte) {
Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ mmio_fops.read = relay_file_operations.read; + mmio_fops.open = relay_file_operations.open; + mmio_fops.poll = relay_file_operations.poll; + mmio_fops.mmap = relay_file_operations.mmap; + mmio_fops.release = relay_file_operations.release; + mmio_fops.splice_read = relay_file_operations.splice_read;can't we do this at compile time?
Nope, the compiler says the initializer is not constant.
Quoted 18 lines
Quoted 18 lines
(quoted diff) +2
-0+ +void __iomem *ioremap_cache_trace(unsigned long offset, unsignedlong size) +{
(quoted diff) +2
-0+ void __iomem *p = ioremap_cache(offset, size); + printk(KERN_DEBUG MODULE_NAME ": ioremap_cache(0x%lx, 0x%lx)= %p\n",
(quoted diff) +1
-0+ offset,size, p);
(quoted diff) +6
-0+ ioremap_trace_core(offset, size, p); + return p; +} +EXPORT_SYMBOL(ioremap_cache_trace); + +void __iomem *ioremap_nocache_trace(unsigned long offset, unsignedlong size) +{
(quoted diff) +2
-0+ void __iomem *p = ioremap_nocache(offset, size); + printk(KERN_DEBUG MODULE_NAME ": ioremap_nocache(0x%lx,0x%lx) = %p\n",
Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +1
-0+ offset,size, p);
(quoted diff) +4
-0+ ioremap_trace_core(offset, size, p); + return p; +} +EXPORT_SYMBOL(ioremap_nocache_trace);I would rather keep the split I think and make the trace call explicit
I don't think I understand what you mean. Get rid of (do_)ioremap_trace_core?
Quoted 2 lines
or.. even nicer, we could go fancy and allow the mmio tracer to "subscribe" to, say, a PCI bar, and auto-enable tracing when the bar gets ioremap'd
Might be cool, but not until there's no hope of getting into 2.6.25.
Quoted 2 lines
about the isntruction decoding.. would be nice if we can share that with kvm so that we can debug that stuff once ;-)
Uhh yeah, if you mean the stuff in drivers/kvm/x86_emulate.c, it indeed looks like it's decoding instructions. Other than that, I understood almost nothing in there. There's one gigantic function that seems to decode and emulate an instruction. I would have to refactor all that code and I'm not up for it.
Quoted 2 lines
other than that I think the code is quite nice already, seems getting this merge ready isn't going to be an enormous deal in terms of complexity
Thank you for your kind and constructive comments.
Signature · 2 lines
Pekka Paalanen http://www.iki.fi/pq/
Quoted 7 lines · On Sun, 27 Jan 2008 18:52:38 +0200 Pekka Paalanen <[email protected]> wrote:
From: Pekka Paalanen <[email protected]>
Provides kernel modules a way to register custom page fault handlers. On every page fault, except those handled in
vmalloc_fault(), this will call a list of registered functions. The functions may handle the fault and forcedo_page_fault()to return immediately.
Quoted 7 lines
This functionality is similar to the now removed page fault notifiers. Custom page fault handlers are used by debugging and reverse engineering tools. Mmio-trace is one such tool and a patch to add it into the tree will follow.
The custom page fault handlers are called from the exact same points in
do_page_fault()as the page fault notifiers were.
to be honest I don't like this approach.
I fully would like to see mmio tracing in the kernel, but please make the hook explicit and direct (eg call the mmio tracing function, not some list) instead...
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org