Skip to contentSkip to conversation

x86: Add a list for custom page fault handlers.

PATCH Superseded Pekka Paalanen 10 participants 43 messages Jan 27, 2008 – Feb 11, 2008

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86: Add a list for custom page fault handlers.

From: Pekka Paalanen <[email protected]>

Provides kernel modules a way to register custom page fault handlers. On every page fault, except those handled in vmalloc_fault(), this will call a list of registered functions. The functions may handle the fault and force do_page_fault() to return immediately.

This functionality is similar to the now removed page fault notifiers. Custom page fault handlers are used by debugging and reverse engineering tools. Mmio-trace is one such tool and a patch to add it into the tree will follow.

The custom page fault handlers are called from the exact same points in do_page_fault() as the page fault notifiers were.

---

This change is x86 specific purely because my intended in-tree user of it is mmio-trace, which AFAIK is really useful only on x86 for reverse engineering binary drivers. If someone later wishes to merge this with notify_page_fault() from kprobes, they can do that.

I am not subscribed to LKML, so please keep me in CC.

I will reply to this email with the next patch adding mmio-trace into kernel.

Thanks,
pq

 arch/x86/Kconfig.debug   |    9 ++++++
 arch/x86/mm/fault_32.c   |   64 ++++++++++++++++++++++++++++++++++++++++++++++
 arch/x86/mm/fault_64.c   |   64 ++++++++++++++++++++++++++++++++++++++++++++++
 include/asm-x86/kdebug.h |    9 ++++++
 4 files changed, 146 insertions(+), 0 deletions(-)
arch/x86/Kconfig.debug +9 -0
index 761ca7b..920a63d 100644
@@ -112,4 +112,13 @@ config IOMMU_LEAK
 	  Add a simple leak tracer to the IOMMU code. This is useful when you
 	  are debugging a buggy device driver that leaks IOMMU mappings.
 
+config PAGE_FAULT_HANDLERS
+	bool "Custom page fault handlers"
+	depends on DEBUG_KERNEL
+	help
+	  Allow the use of custom page fault handlers. A kernel module may
+	  register a function that is called on every page fault not handled
+	  for vmalloc. Custom handlers are used by some debugging and reverse
+	  engineering tools.
+
 endmenu
arch/x86/mm/fault_32.c +64 -0

82 lines not on this page · raw message

arch/x86/mm/fault_64.c +64 -0

82 lines not on this page · raw message

include/asm-x86/kdebug.h +9 -0
index e2f9b62..7449ad2 100644
@@ -30,4 +30,13 @@ extern void dump_pagetable(unsigned long);
 extern unsigned long oops_begin(void);
 extern void oops_end(unsigned long);
 
+struct pf_handler {
+	struct hlist_node hlist;
+	int (*handler)(struct pt_regs *regs, unsigned long error_code,
+						unsigned long address);
+};
+
+extern void register_page_fault_handler(struct pf_handler *new_pfh);
+extern void unregister_page_fault_handler(struct pf_handler *old_pfh);
+
 #endif
Signature · 1 line
1.5.3.7
#1 lore raw

RFC PATCH 1/1 x86: mmiotrace - trace memory mapped IO

Hello,

the patch itself is slightly bigger than recommended for LKML, so it is available at:
http://jumi.lut.fi/~paalanen/scratch/mmio24/0002-x86-mmiotrace-trace-memory-mapped-IO.patch.txt

This seems to work fine on UP machine, I have not tested yet with with an SMP machine. Originally mmio-trace was UP-only, so there probably still are SMP issues. Mmio-trace has existed with its current purpose for about a year now.

The following is a copy of the patch description:

Mmiotrace is a tool for trapping memory mapped IO (MMIO) accesses within the kernel. It is used for debugging and especially for reverse engineering evil binary drivers.

Mmiotrace works by wrapping the ioremap family of kernel functions and marking the returned pages as not present. Access to the IO memory triggers a page fault, which will be handled by mmiotrace's custom page fault handler. This will single-step the faulted instruction with the MMIO page marked as present. Access logs are directed to user space via relay and debug_fs.

This page fault approach is necessary, because binary drivers have readl/writel etc. calls inlined and therefore extremely difficult to trap with with e.g. kprobes.

This patch depends on the custom page fault handlers patch.

---

More information on mmio-trace can be found at
http://nouveau.freedesktop.org/wiki/MmioTrace
and the out-of-three module with user space parts is at
http://gitweb.freedesktop.org/?p=users/pq/mmio-trace.git;a=summary

The out-of-tree git does not include all changes I made for the in-tree version.

Mmiotrace does not appear to procude excruciating slowness when used. Starting X with traced nvidia binary drivers takes a couple of seconds longer than normally. Starting X, running glxgears for a moment and shutting down X results in an mmio-trace log of roughly 1.5 million events. Replace glxgears by running Quake3-demo's first demo play twice and end up with 3.5 million events. Quake3-demo framerate with mmio-traced driver goes down to 59 fps, compared to the normal 83 fps.

I know of one bad failure case with mmio-trace. It may trigger a complete system freeze when tracing a certain nvidia driver version on a machine with an 8000-series (nv50) graphics chip. Netconsole, serial console and NMI watchdog have so far been useless, and I am starting to believe that it is the nvidia driver that does something bad to make mmio-trace fall over. I have heard that more recent nvidia drivers do not make it crash anymore.

I'm hoping mmiotrace could be included in 2.6.25. It would help FOSS driver projects (Nouveau, maybe madwifi, too) to gather the information they need. Very often the information comes from ordinary users, for whom going through kernel patching would be a major struggle.

Thanks,
Pekka

#2 lore raw

Re: [RFC PATCH] x86: mmiotrace - trace memory mapped IO

Quoted 1 line · On Sun, 27 Jan 2008 19:55:36 +0200 Pekka Paalanen <[email protected]> wrote:

The following is a copy of the patch description:

...

Quoted 6 lines

Mmiotrace works by wrapping the ioremap family of kernel functions and marking the returned pages as not present. Access to the IO memory triggers a page fault, which will be handled by mmiotrace's custom page fault handler. This will single-step the faulted instruction with the MMIO page marked as present. Access logs are directed to user space via relay and debug_fs.

Hi,

a test user found a problem with mmiotrace (ported to x86/mm git tree). The symbol 'init_mm' is undefined, but required when building the 32-bit mmiotrace kernel module.

commit 3abf024d2abb79614d8c4cb25a70d5596f77d0ad

Author: Thomas Gleixner <[email protected]>
Date:   Wed Jan 30 13:30:28 2008 +0100

x86: nuke a ton of unused exports

This commit removes EXPORT_SYMBOL(init_mm). Apparently mmiotrace requires this symbol, but I do not know why. I do not use init_mm myself, so apparently some function I call is inlined and requires it. Page table or attribute handling stuff maybe?

Is it possible to simply put back this export or should I dig deeper into why it seems to be required?
I can provide the simple patch to put it back in my next patch set, if this is the right solution.

I will post the patch set after this issue is solved. In the mean time, for the interested, the current patches are here:
http://jumi.lut.fi/~paalanen/scratch/mmio25/

Ps. I tested x86/mm version of mmiotrace on my 64-bit system and it seems to work ok. I had to write a simple test module because I did not have any drivers that a) called ioremap, b) were not broken on x86/mm, and c) I had the hardware to actually use it. So this testing is very light.

Signature · 2 lines
Pekka Paalanen
http://www.iki.fi/pq/
#3 lore raw
In reply to Pekka Paalanen
Quoted 4 lines · Pekka Paalanen
 arch/x86/Kconfig.debug   |    9 ++++++
 arch/x86/mm/fault_32.c   |   64 ++++++++++++++++++++++++++++++++++++++++++++++
 arch/x86/mm/fault_64.c   |   64 ++++++++++++++++++++++++++++++++++++++++++++++
 include/asm-x86/kdebug.h |    9 ++++++

hm, could you please base this against x86.git#mm, see:

    http://redhat.com/~mingo/x86.git/README

thanks,

Ingo

#4 lore raw
In reply to Pekka Paalanen

arch/x86/mm/fault_32.c line 56

Quoted 9 lines · Pekka Paalanen
arch/x86/mm/fault_32.c +9 -0
+void unregister_page_fault_handler(struct pf_handler *old_pfh)
+{
+	might_sleep();
+	spin_lock(&pf_handlers_writer);
+	hlist_del_rcu(&old_pfh->hlist);
+	spin_unlock(&pf_handlers_writer);
+	synchronize_rcu();
+}
+EXPORT_SYMBOL_GPL(unregister_page_fault_handler);

Could you remove that sync_rcu() call and put in the comments that the calling function is to take care of proper freeing.

sync_rcu() is a _very_ expensive call, and oft time it is not needed in favour of call_rcu().

#5 lore raw
In reply to Pekka Paalanen

From: Pekka Paalanen <[email protected]>

Provides kernel modules a way to register custom page fault handlers. On every page fault, except those handled in vmalloc_fault(), this will call a list of registered functions. The functions may handle the fault and force do_page_fault() to return immediately.

This functionality is similar to the now removed page fault notifiers. Custom page fault handlers are used by debugging and reverse engineering tools. Mmio-trace is one such tool and a patch to add it into the tree will follow.

The custom page fault handlers are called from the exact same points in do_page_fault() as the page fault notifiers were.

---
Ingo, ported Pekka's patch to current x86.git for review. I have not incorporated the RCU comments from Peter Zjistra, but I have folded some of the static functions into on handle_custom_pf function that will match the handle_kprobe_fault helper that will be ready for RC1.

Harvey

 arch/x86/Kconfig.debug |    9 ++++++++
 arch/x86/mm/fault.c    |   51
++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+), 0 deletions(-)
arch/x86/Kconfig.debug +9 -0
index 2e1e3af..9b44bc5 100644
@@ -225,4 +225,13 @@ config CPA_DEBUG
 	help
 	  Do change_page_attr self tests at boot.
 
+config PAGE_FAULT_HANDLERS
+	bool "Custom page fault handlers"
+	depends on DEBUG_KERNEL
+	help
+	  Allow the use of custom page fault handlers. A kernel module may
+	  register a function that is called on every page fault not handled
+	  for vmalloc. Custom handlers are used by some debugging and reverse
+	  engineering tools.
+
 endmenu
arch/x86/mm/fault.c +25 -0
index e28cc52..c6c8164 100644
@@ -49,6 +49,54 @@
 #define PF_RSVD		(1<<3)
 #define PF_INSTR	(1<<4)
 
+#ifdef CONFIG_PAGE_FAULT_HANDLERS
+static HLIST_HEAD(pf_handlers); /* protected by RCU */
+static DEFINE_SPINLOCK(pf_handlers_writer);
+
+void register_page_fault_handler(struct pf_handler *new_pfh)
+{
+	spin_lock(&pf_handlers_writer);
+	hlist_add_head_rcu(&new_pfh->hlist, &pf_handlers);
+	spin_unlock(&pf_handlers_writer);
+}
+EXPORT_SYMBOL_GPL(register_page_fault_handler);
+
+void unregister_page_fault_handler(struct pf_handler *old_pfh)
+{
+	might_sleep();
+	spin_lock(&pf_handlers_writer);
+	hlist_del_rcu(&old_pfh->hlist);
+	spin_unlock(&pf_handlers_writer);
+	synchronize_rcu();
+}
+EXPORT_SYMBOL_GPL(unregister_page_fault_handler);
+#endif
+
+/* returns non-zero if do_page_fault() should return */
+static int handle_custom_pf(struct pt_regs *regs, unsigned long

error_code,
+ unsigned long address)
+{
+#ifdef CONFIG_PAGE_FAULT_HANDLERS
+ int ret = 0;
+ struct pf_handler *cur;
+ struct hlist_node *ncur;
+
+ if (hlist_empty(&pf_handlers))
+ return 0;
+
+ rcu_read_lock();
+ hlist_for_each_entry_rcu(cur, ncur, &pf_handlers, hlist) {
+ ret = cur->handler(regs, error_code, address);
+ if (ret)
+ break;
+ }
+ rcu_read_unlock();
+ return ret;
+#else
+ return 0;
+#endif
+}
+
static inline int notify_page_fault(struct pt_regs *regs) {
#ifdef CONFIG_KPROBES
@@ -588,6 +636,9 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code)
if (notify_page_fault(regs))
return;

+ if (handle_custom_pf(regs, error_code, address))
+ return;
+
/*
* We fault-in kernel-space virtual memory on-demand. The
* 'reference' page table is init_mm.pgd.

Signature · 1 line
1.5.4.rc4.1142.gf5a97
#6 lore raw

From: Pekka Paalanen <[email protected]>

Provides kernel modules a way to register custom page fault handlers. On every page fault, except those handled in vmalloc_fault(), this will call a list of registered functions. The functions may handle the fault and force do_page_fault() to return immediately.

This functionality is similar to the now removed page fault notifiers. Custom page fault handlers are used by debugging and reverse engineering tools. Mmio-trace is one such tool and a patch to add it into the tree will follow.

The custom page fault handlers are called from the exact same points in do_page_fault() as the page fault notifiers were.

---
Sorry, attached the wrong version to my last message missing the kdebug.h hunk. This is still just a straight port to current x86.git.

 arch/x86/Kconfig.debug   |    9 ++++++++
 arch/x86/mm/fault.c      |   51 ++++++++++++++++++++++++++++++++++++++++++++++
 include/asm-x86/kdebug.h |    8 +++++++
 3 files changed, 68 insertions(+), 0 deletions(-)
arch/x86/Kconfig.debug +9 -0
index 2e1e3af..9b44bc5 100644
@@ -225,4 +225,13 @@ config CPA_DEBUG
 	help
 	  Do change_page_attr self tests at boot.
 
+config PAGE_FAULT_HANDLERS
+	bool "Custom page fault handlers"
+	depends on DEBUG_KERNEL
+	help
+	  Allow the use of custom page fault handlers. A kernel module may
+	  register a function that is called on every page fault not handled
+	  for vmalloc. Custom handlers are used by some debugging and reverse
+	  engineering tools.
+
 endmenu
arch/x86/mm/fault.c +51 -0

63 lines not on this page · raw message

include/asm-x86/kdebug.h +8 -0
index dd442a1..ba03368 100644
@@ -35,4 +35,12 @@ extern void dump_pagetable(unsigned long);
 extern unsigned long oops_begin(void);
 extern void oops_end(unsigned long, struct pt_regs *, int signr);
 
+struct pf_handler {
+	struct hlist_node hlist;
+	int (*handler)(struct pt_regs *regs, unsigned long error_code,
+		       unsigned long address);
+};
+
+extern void register_page_fault_handler(struct pf_handler *new_pfh);
+extern void unregister_page_fault_handler(struct pf_handler *old_pfh);
 #endif
Signature · 1 line
1.5.4.rc4.1142.gf5a97
#7 lore raw
Quoted 2 lines · On Tue, 29 Jan 2008 18:34:13 -0800 Harvey Harrison <[email protected]>…

Sorry, attached the wrong version to my last message missing the kdebug.h hunk. This is still just a straight port to current x86.git.

Wow, thanks.

I have to say I already did this for myself, but I had to
change too many things to simply resubmit without testing. And it turned out porting mmiotrace to x86/mm was a non-trivial task. I have not been able to test yet even though I have been working on it every night. Today I will see if things work for me, and hopefully also for someone else. I might send new patches today or tomorrow, if all goes well.

btw. I have a bad feeling about global_flush_tlb() disappearing, it was used in mmiotrace code, and I don't know what to use instead. The obvious replacement for it is a static function elsewhere. Currently I am simply hoping it was a redundant leftover from the time when we had trouble making mmiotrace stable.

To be honest, I haven't yet had the time to educate myself about what is TLB.

Quoted 7 lines
arch/x86/mm/fault.c +0 -0
index e28cc52..c6c8164 100644
@@ -49,6 +49,54 @@
 #define PF_RSVD		(1<<3)
 #define PF_INSTR	(1<<4)

arch/x86/mm/fault.c line 60

Quoted 10 lines
arch/x86/mm/fault.c +9 -0
 
+#ifdef CONFIG_PAGE_FAULT_HANDLERS
+static HLIST_HEAD(pf_handlers); /* protected by RCU */
+static DEFINE_SPINLOCK(pf_handlers_writer);
+
+void register_page_fault_handler(struct pf_handler *new_pfh)
+{
+	spin_lock(&pf_handlers_writer);
+	hlist_add_head_rcu(&new_pfh->hlist, &pf_handlers);
+	spin_unlock(&pf_handlers_writer);

Here I had to change to irqsave/irqrestore versions, as I found a possible lock dependency issue. Lockdep debugging code pointed it out. I'm not sure I fixed it completely, but using spin_lock_irqsave() is a safe bet.

arch/x86/mm/fault.c line 71

Quoted 11 lines
arch/x86/mm/fault.c +11 -0
+}
+EXPORT_SYMBOL_GPL(register_page_fault_handler);
+
+void unregister_page_fault_handler(struct pf_handler *old_pfh)
+{
+	might_sleep();
+	spin_lock(&pf_handlers_writer);
+	hlist_del_rcu(&old_pfh->hlist);
+	spin_unlock(&pf_handlers_writer);
+	synchronize_rcu();
+}

And here. And I also removed sync RCU and might_sleep().

Quoted 4 lines
arch/x86/mm/fault.c +4 -0
+EXPORT_SYMBOL_GPL(unregister_page_fault_handler);
+#endif
+
+/* returns non-zero if do_page_fault() should return */

arch/x86/mm/fault.c line 85

Quoted 10 lines
arch/x86/mm/fault.c +10 -0
+static int handle_custom_pf(struct pt_regs *regs, unsigned long error_code,
+			    unsigned long address)
+{
+#ifdef CONFIG_PAGE_FAULT_HANDLERS
+	int ret = 0;
+	struct pf_handler *cur;
+	struct hlist_node *ncur;
+
+	if (hlist_empty(&pf_handlers))
+		return 0;

This looks prettier than my code. I might change my code to this.

Quoted 17 lines
arch/x86/mm/fault.c +13 -0
+	rcu_read_lock();
+	hlist_for_each_entry_rcu(cur, ncur, &pf_handlers, hlist) {
+		ret = cur->handler(regs, error_code, address);
+		if (ret)
+			break;
+	}
+	rcu_read_unlock();
+	return ret;
+#else
+	return 0;
+#endif
+}
+
 static inline int notify_page_fault(struct pt_regs *regs)
 {
 #ifdef CONFIG_KPROBES

arch/x86/mm/fault.c void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code) · line 641

Quoted 6 lines
arch/x86/mm/fault.c +3 -0
@@ -588,6 +636,9 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code)
 	if (notify_page_fault(regs))
 		return;
 
+	if (handle_custom_pf(regs, error_code, address))
+		return;
+

This is the non-trivial change that I really want to test before submitting again. The call site of the handlers changes, and vmalloc related faults reach the custom handlers. I don't think this would be a problem for mmiotrace, but I don't know enough to be sure.
I guess using vmalloc'd memory from a page fault handler would be doomed, anyway, so no point worrying about it, right?

if (unlikely(handle_custom_pf(regs, error_code, address))) Maybe like this, even?

It is still supposedly making a function call every time, at least when CONFIG_PAGE_FAULT_HANDLERS is defined. Would it have a non-negligible impact on performance when no custom handlers are registered?

My original version had the separate inline function to check if the list is empty. Which one is preferrable?

I'm hoping distros would ship some kernels with PAGE_FAULT_HANDLERS enabled as it would make contributors' lives easier. IIRC some already have enabled RELAY and DEBUG_FS for mmiotrace. This is why I think keeping the performance impact to the minimum here is important, for the case of no custom handlers registered.

Thanks,

Signature · 2 lines
Pekka Paalanen
http://www.iki.fi/pq/
#8 lore raw
Quoted 7 lines · Pekka Paalanen
Quoted 7 lines

static inline int notify_page_fault(struct pt_regs *regs) {
#ifdef CONFIG_KPROBES

arch/x86/mm/fault.c +0 -0
@@ -588,6 +636,9 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code)
 	if (notify_page_fault(regs))
 		return;
 
Quoted 10 lines

arch/x86/mm/fault.c void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code) · line 641

Quoted 3 lines
arch/x86/mm/fault.c +3 -0
+	if (handle_custom_pf(regs, error_code, address))
+		return;
+

This is the non-trivial change that I really want to test before submitting again. The call site of the handlers changes, and vmalloc related faults reach the custom handlers. I don't think this would be a problem for mmiotrace, but I don't know enough to be sure. I guess using vmalloc'd memory from a page fault handler would be doomed, anyway, so no point worrying about it, right?

could you please send us a patch for the whole mmiotrace kernel-side feature, so that we can have a look at the general structure of this? (and the interaction with change_page_attr(), etc.) Even if it's not functional (and wont even build/boot) at the moment. Thanks,

Ingo

#9 lore raw

PATCH v2 1/1 x86: Add a list for custom page fault handlers.

Quoted 4 lines

could you please send us a patch for the whole mmiotrace kernel-side feature, so that we can have a look at the general structure of this? (and the interaction with change_page_attr(), etc.) Even if it's not functional (and wont even build/boot) at the moment. Thanks,

Very well, first the revised custom page fault handler patch. Changes since the previous submit:
- use spin_lock_irqsave instead of spin_lock
- Harvey Harrison's clean-up with the #ifdefs
- handler call site moved earlier
- remove sync RCU call

I'm not aware of any functional problems with this one.

Arjan, you said you don't like this. May I ask why?
This is most convinient for mmiotrace as it is meant to be a module. I'm also using this as an excuse to let other people to get into the page fault handler with their out-of-tree-today modules.

Quoted 1 line

From adeb0f4f95f176d97c1157031b5dd01f49d4cb66 Mon Sep 17 00:00:00 2001

From: Pekka Paalanen <[email protected]>
Date: Wed, 30 Jan 2008 21:49:52 +0200
Subject: [PATCH] x86: Add a list for custom page fault handlers.

Provides kernel modules a way to register custom page fault handlers. On every page fault this will call a list of registered functions. The functions may handle the fault and force do_page_fault() to return immediately.

This functionality is similar to the now removed page fault notifiers. Custom page fault handlers are used by debugging and reverse engineering tools. Mmiotrace is one such tool and a patch to add it into the tree will follow.

The custom page fault handlers are called earlier in do_page_fault() than the page fault notifiers were.

---
 arch/x86/Kconfig.debug   |    8 ++++++
 arch/x86/mm/fault.c      |   56 ++++++++++++++++++++++++++++++++++++++++++++++
 include/asm-x86/kdebug.h |    9 +++++++
 3 files changed, 73 insertions(+), 0 deletions(-)
arch/x86/Kconfig.debug +8 -0
index 2e1e3af..1185dcc 100644
@@ -138,6 +138,14 @@ config IOMMU_LEAK
 	  Add a simple leak tracer to the IOMMU code. This is useful when you
 	  are debugging a buggy device driver that leaks IOMMU mappings.
 
+config PAGE_FAULT_HANDLERS
+	bool "Custom page fault handlers"
+	depends on DEBUG_KERNEL
+	help
+	  Allow the use of custom page fault handlers. A kernel module may
+	  register a function that is called on every page fault. Custom
+	  handlers are used by some debugging and reverse engineering tools.
+
 #
 # IO delay types:
 #
arch/x86/mm/fault.c +56 -0

68 lines not on this page · raw message

include/asm-x86/kdebug.h +9 -0
index dd442a1..7ae2118 100644
@@ -35,4 +35,13 @@ extern void dump_pagetable(unsigned long);
 extern unsigned long oops_begin(void);
 extern void oops_end(unsigned long, struct pt_regs *, int signr);
 
+struct pf_handler {
+	struct hlist_node hlist;
+	int (*handler)(struct pt_regs *regs, unsigned long error_code,
+						unsigned long address);
+};
+
+extern void register_page_fault_handler(struct pf_handler *new_pfh);
+extern void unregister_page_fault_handler(struct pf_handler *old_pfh);
+
 #endif
Signature · 1 line
1.5.3.7
#10 lore raw
Quoted 13 lines · On Thu, 31 Jan 2008 18:02:53 +0200 Pekka Paalanen <[email protected]> wrote:
Quoted 5 lines

could you please send us a patch for the whole mmiotrace kernel-side feature, so that we can have a look at the general structure of this? (and the interaction with change_page_attr(), etc.) Even if it's not functional (and wont even build/boot) at the moment. Thanks,

Very well, first the revised custom page fault handler patch. Changes since the previous submit:
- use spin_lock_irqsave instead of spin_lock
- Harvey Harrison's clean-up with the #ifdefs
- handler call site moved earlier
- remove sync RCU call

Quoted 3 lines

I'm not aware of any functional problems with this one.

Arjan, you said you don't like this. May I ask why?

it's overhead that's rarely used, and worse, it suffers from the LSM disease:
it's a hook without visibility into the callers. Esp if there's only 1 user, a direct call is not only faster, it also shows you what is going on when you're reading the code. THe page fault stuff is tricky enough as it is without any "call arbitrary thing" hooks into it

Quoted 1 line

This is most convinient for mmiotrace as it is meant to be a module.

that's not per se mutually exclusive, there's other cases like this already (but even then, I don't want to go through hoops to make mmiotracing a module; for me I can totally see it becoming a full mature member of the kernel, and if that means a few things have to be changed in the normal kernel, or infrastructure added, I'm totally ok with that)

Quoted 2 lines

I'm also using this as an excuse to let other people to get into the page fault handler with their out-of-tree-today modules.

this is exactly the point.. it becomes a totally opaque thing, a random hook.

Signature · 3 lines
If you want to reach me at my work email, use [email protected]
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
#11 lore raw
Quoted 10 lines · On Thu, 31 Jan 2008 08:15:07 -0800 Arjan van de Ven <[email protected]> wrote:
Quoted 7 lines · On Thu, 31 Jan 2008 18:02:53 +0200 Pekka Paalanen <[email protected]> wrote:

Very well, first the revised custom page fault handler patch. Changes since the previous submit:
- use spin_lock_irqsave instead of spin_lock
- Harvey Harrison's clean-up with the #ifdefs
- handler call site moved earlier
- remove sync RCU call

Quoted 5 lines

it's overhead that's rarely used, and worse, it suffers from the LSM disease:
it's a hook without visibility into the callers. Esp if there's only 1 user, a direct call is not only faster, it also shows you what is going on when you're reading the code. THe page fault stuff is tricky enough as it is without any "call arbitrary thing" hooks into it

I understand your point.

Quoted 7 lines
Quoted 1 line

This is most convinient for mmiotrace as it is meant to be a module.

that's not per se mutually exclusive, there's other cases like this already (but even then, I don't want to go through hoops to make mmiotracing a module; for me I can totally see it becoming a full mature member of the kernel, and if that means a few things have to be changed in the normal kernel, or infrastructure added, I'm totally ok with that)

I have two reasons why I'd like to let it be a module:
- it's "broken", the relay buffers are cleared on module unload
- it's a lot easier to push updated version for testing to people

Ok, the first one is just a silly excuse, but the second one is to avoid forcing non-developer users to patch their kernels. This is a much needed tool for the Nouveau project even in it's current form.

Quoted 4 lines
Quoted 2 lines

I'm also using this as an excuse to let other people to get into the page fault handler with their out-of-tree-today modules.

this is exactly the point.. it becomes a totally opaque thing, a random hook.

And this was the third reason I wanted a module, but I understand this is an unacceptable justification.

What are these "othes cases" I could look at as an example of a hook, that is meant for a single specific module? I do not understand what could prevent out-of-tree modules from using the hook if my module can use it (and my module is not using it at the same time).

What is the verdict?
Will the custom page fault handler patch v2 get merged as it is, do I need to modify it (how?), do I need to rewrite it based on some example (which?), or do I have to drop the whole idea of mmiotrace being kernel module and make it work as built-in?

In the case of making it a built-in, I do not think I have any chance to get it done for 2.6.25.

There's one more option. I could make kmmio.c things as builtin, and keep the mmio-mod.c as the module. This is how things were in the very beginning of mmiotrace, when the page fault notifiers did not exist. This could be an acceptable compromise, no? But I don't think I will get that ready for 2.6.25, either.

What about merging the custom page fault handler patch now, and removing it for 2.6.26?

Looks like Beulich and Roskin will be disappointed anyway. They were hoping for a usable page fault hook.

Signature · 2 lines
Pekka Paalanen
http://www.iki.fi/pq/
#12 lore raw
Quoted 7 lines · Pekka Paalanen

I have two reasons why I'd like to let it be a module:
- it's "broken", the relay buffers are cleared on module unload
- it's a lot easier to push updated version for testing to people

Ok, the first one is just a silly excuse, but the second one is to avoid forcing non-developer users to patch their kernels. This is a much needed tool for the Nouveau project even in it's current form.

i dont think this second problem is a practical one: i've test-merged mmiotrace two days ago into x86.git and it's holding up very well in my testing (it has not caused a single hickup so far). It's nice optional debug functionality and i dont see any reason why not to merge it.

a few cleanup requests:

- please make the pagefault callbacks explicit as Arjan has asked

- any reason why it lives in arch/x86/kernel/mmiotrace/? It should be a prime-time member of arch/x86/mm/ i think.

- please make it build-in-able, not modules-only.

- [ longer-term: think about integrating pf_in.c with the x86 emulator/disassembler of KVM. ]

Ingo

#13 lore raw
Quoted 4 lines · On Sun, 3 Feb 2008 08:03:21 +0100 Ingo Molnar <[email protected]> wrote:

i dont think this second problem is a practical one: i've test-merged mmiotrace two days ago into x86.git and it's holding up very well in my testing (it has not caused a single hickup so far). It's nice optional debug functionality and i dont see any reason why not to merge it.

Good to hear!

I've made some fixes since my last mmiotrace patch (v2). Rewritten things to use lookup_address(), added EXPORT_SYMBOL(lookup_address) into pageattr.c to make it work (this hopefully fixed the undefined symbol 'init_mm' on x86-32), and fixed relay-buffer-full flags on SMP.

Quoted 3 lines

a few cleanup requests:

- please make the pagefault callbacks explicit as Arjan has asked

What does this mean in practice? It should be able to call into a module, so the only thing I can come up with is a single settable function pointer. Just like I proposed in patch v2, but a function pointer instead of a list. Is this what you and Arjan suggest?
We are talking about the hook in do_page_fault(), right?

Quoted 2 lines

- any reason why it lives in arch/x86/kernel/mmiotrace/? It should be a prime-time member of arch/x86/mm/ i think.

I was waiting for someone to comment on that, I didn't know what was the proper place. I'll move it into there.

Quoted 1 line

- please make it build-in-able, not modules-only.

Then I need a way to clear the relay buffers at some point. Currently they get cleared only on module unload. Maybe preferably also allocate and deallocate them dynamically. I've no idea how to do this yet, any pointers to examples?

Hmm, I guess relay buffers could be allocated on the first intercepted call to ioremap, but then the userspace could not be started before hand, and at that point it is too late.

Making it build-in-able will take so much time that I'd prefer to aim it for 2.6.26.

Quoted 2 lines

- [ longer-term: think about integrating pf_in.c with the x86 emulator/disassembler of KVM. ]

Yikes. Like I replied to Arjan, it looks daunting.

Thanks, I'll do what I can. When do you need the final (as for 2.6.25) mmiotrace patch, or is it going in as v2 and then we fix it in RCs?

Signature · 2 lines
Pekka Paalanen
http://www.iki.fi/pq/
#14 lore raw

PATCH 1/4 x86 mmiotrace: use lookup_address()

In reply to Ingo Molnar

Use lookup_address() from pageattr.c instead of doing the same manually. Also had to EXPORT_SYMBOL(lookup_address) to make this work for modules. This also fixes "undefined symbol 'init_mm'" compile error for x86_32.

---
 arch/x86/kernel/mmiotrace/kmmio.c    |   46 +++++++++++++++++++++++-----------
 arch/x86/kernel/mmiotrace/mmio-mod.c |   19 +++++++++-----
 arch/x86/mm/pageattr.c               |    1 +
 3 files changed, 44 insertions(+), 22 deletions(-)
arch/x86/kernel/mmiotrace/kmmio.c +31 -15
index 8ba48f9..28411da 100644
@@ -20,6 +20,7 @@
 #include <asm/cacheflush.h>
 #include <asm/errno.h>
 #include <asm/tlbflush.h>
+#include <asm/pgtable.h>
 
 #include "kmmio.h"
 
@@ -117,40 +118,55 @@ static struct kmmio_fault_page *get_kmmio_fault_page(unsigned long page)
 	return NULL;
 }
 
-static void arm_kmmio_fault_page(unsigned long page, int *large)
+static void arm_kmmio_fault_page(unsigned long page, int *page_level)
 {
 	unsigned long address = page & PAGE_MASK;
-	pgd_t *pgd = pgd_offset_k(address);
-	pud_t *pud = pud_offset(pgd, address);
-	pmd_t *pmd = pmd_offset(pud, address);
-	pte_t *pte = pte_offset_kernel(pmd, address);
+	int level;
+	pte_t *pte = lookup_address(address, &level);
 
-	if (pmd_large(*pmd)) {
+	if (!pte) {
+		printk(KERN_ERR "Error in %s: no pte for page 0x%08lx\n",
+						__FUNCTION__, page);
+		return;
+	}
+
+	if (level == PG_LEVEL_2M) {
+		pmd_t *pmd = (pmd_t *)pte;
 		set_pmd(pmd, __pmd(pmd_val(*pmd) & ~_PAGE_PRESENT));
-		if (large)
-			*large = 1;
 	} else {
+		/* PG_LEVEL_4K */
 		set_pte(pte, __pte(pte_val(*pte) & ~_PAGE_PRESENT));
 	}
 
+	if (page_level)
+		*page_level = level;
+
 	__flush_tlb_one(page);
 }
 
-static void disarm_kmmio_fault_page(unsigned long page, int *large)
+static void disarm_kmmio_fault_page(unsigned long page, int *page_level)
 {
 	unsigned long address = page & PAGE_MASK;
-	pgd_t *pgd = pgd_offset_k(address);
-	pud_t *pud = pud_offset(pgd, address);
-	pmd_t *pmd = pmd_offset(pud, address);
-	pte_t *pte = pte_offset_kernel(pmd, address);
+	int level;
+	pte_t *pte = lookup_address(address, &level);
 
-	if (large && *large) {
+	if (!pte) {
+		printk(KERN_ERR "Error in %s: no pte for page 0x%08lx\n",
+						__FUNCTION__, page);
+		return;
+	}
+
+	if (level == PG_LEVEL_2M) {
+		pmd_t *pmd = (pmd_t *)pte;
 		set_pmd(pmd, __pmd(pmd_val(*pmd) | _PAGE_PRESENT));
-		*large = 0;
 	} else {
+		/* PG_LEVEL_4K */
 		set_pte(pte, __pte(pte_val(*pte) | _PAGE_PRESENT));
 	}
 
+	if (page_level)
+		*page_level = level;
+
 	__flush_tlb_one(page);
 }
 
arch/x86/kernel/mmiotrace/mmio-mod.c +12 -7
index 4a8c859..82ae920 100644
@@ -120,19 +120,24 @@ static int write_marker(struct file *file, const char __user *buffer,
 
 static void print_pte(unsigned long address)
 {
-	pgd_t *pgd = pgd_offset_k(address);
-	pud_t *pud = pud_offset(pgd, address);
-	pmd_t *pmd = pmd_offset(pud, address);
-	if (pmd_large(*pmd)) {
+	int level;
+	pte_t *pte = lookup_address(address, &level);
+
+	if (!pte) {
+		printk(KERN_ERR "Error in %s: no pte for page 0x%08lx\n",
+						__FUNCTION__, address);
+		return;
+	}
+
+	if (level == PG_LEVEL_2M) {
 		printk(KERN_EMERG MODULE_NAME ": 4MB pages are not "
 						"currently supported: %lx\n",
 						address);
 		BUG();
 	}
 	printk(KERN_DEBUG MODULE_NAME ": pte for 0x%lx: 0x%lx 0x%lx\n",
-		address,
-		pte_val(*pte_offset_kernel(pmd, address)),
-		pte_val(*pte_offset_kernel(pmd, address)) & _PAGE_PRESENT);
+					address, pte_val(*pte),
+					pte_val(*pte) & _PAGE_PRESENT);
 }
 
 /*
arch/x86/mm/pageattr.c +1 -0
index bb55a78..fc7e9ea 100644
@@ -225,6 +225,7 @@ pte_t *lookup_address(unsigned long address, int *level)
 
 	return pte_offset_kernel(pmd, address);
 }
+EXPORT_SYMBOL(lookup_address);
 
 /*
  * Set the new pmd in all the pgds we know about:
Signature · 1 line
1.5.3.7
#15 lore raw

PATCH 2/4 x86 mmiotrace: fix relay-buffer-full flag for SMP

Relay has per-cpu buffers, but mmiotrace was using only a single flag for detecting buffer full/not-full transitions. The new code makes this per-cpu and actually counts missed events.

---
 arch/x86/kernel/mmiotrace/mmio-mod.c |   26 ++++++++++++++++----------
 1 files changed, 16 insertions(+), 10 deletions(-)
arch/x86/kernel/mmiotrace/mmio-mod.c +16 -10
index 82ae920..f492b65 100644
@@ -29,6 +29,7 @@
 #include <asm/pgtable.h>
 #include <linux/mmiotrace.h>
 #include <asm/e820.h> /* for ISA_START_ADDRESS */
+#include <asm/atomic.h>
 
 #include "kmmio.h"
 #include "pf_in.h"
@@ -47,9 +48,13 @@ struct trap_reason {
 	int active_traces;
 };
 
+/* Accessed per-cpu. */
 static struct trap_reason pf_reason[NR_CPUS];
 static struct mm_io_header_rw cpu_trace[NR_CPUS];
 
+/* Access to this is not per-cpu. */
+static atomic_t dropped[NR_CPUS];
+
 static struct file_operations mmio_fops = {
 	.owner = THIS_MODULE,
 };
@@ -57,7 +62,6 @@ static struct file_operations mmio_fops = {
 static const size_t subbuf_size = 256*1024;
 static struct rchan *chan;
 static struct dentry *dir;
-static int suspended;      /* XXX should this be per cpu? */
 static struct proc_dir_entry *proc_marker_file;
 
 /* module parameters */
@@ -269,19 +273,21 @@ static void post(struct kmmio_probe *p, unsigned long condition,
 static int subbuf_start_handler(struct rchan_buf *buf, void *subbuf,
 					void *prev_subbuf, size_t prev_padding)
 {
+	unsigned int cpu = buf->cpu;
+	atomic_t *drop = &dropped[cpu];
+	int count;
 	if (relay_buf_full(buf)) {
-		if (!suspended) {
-			suspended = 1;
-			printk(KERN_ERR MODULE_NAME
-						": cpu %d buffer full!!!\n",
-						smp_processor_id());
+		if (atomic_inc_return(drop) == 1) {
+			printk(KERN_ERR MODULE_NAME ": cpu %d buffer full!\n",
+									cpu);
 		}
 		return 0;
-	} else if (suspended) {
-		suspended = 0;
+	} else if ((count = atomic_read(drop))) {
 		printk(KERN_ERR MODULE_NAME
-					": cpu %d buffer no longer full.\n",
-					smp_processor_id());
+					": cpu %d buffer no longer full, "
+					"missed %d events.\n",
+					cpu, count);
+		atomic_sub(count, drop);
 	}
 
 	return 1;
Signature · 1 line
1.5.3.7
#16 lore raw

Re: [PATCH 2/4] x86 mmiotrace: fix relay-buffer-full flag for SMP

Quoted 22 lines · Pekka Paalanen

Relay has per-cpu buffers, but mmiotrace was using only a single flag for detecting buffer full/not-full transitions. The new code makes this per-cpu and actually counts missed events.

---
 arch/x86/kernel/mmiotrace/mmio-mod.c |   26 ++++++++++++++++----------
 1 files changed, 16 insertions(+), 10 deletions(-)
arch/x86/kernel/mmiotrace/mmio-mod.c +1 -0
index 82ae920..f492b65 100644
@@ -29,6 +29,7 @@
 #include <asm/pgtable.h>
 #include <linux/mmiotrace.h>
 #include <asm/e820.h> /* for ISA_START_ADDRESS */
+#include <asm/atomic.h>
 
 #include "kmmio.h"
 #include "pf_in.h"

arch/x86/kernel/mmiotrace/mmio-mod.c struct trap_reason { · line 57

Quoted 10 lines
arch/x86/kernel/mmiotrace/mmio-mod.c +4 -0
@@ -47,9 +48,13 @@ struct trap_reason {
 	int active_traces;
 };
 
+/* Accessed per-cpu. */
 static struct trap_reason pf_reason[NR_CPUS];
 static struct mm_io_header_rw cpu_trace[NR_CPUS];
 
+/* Access to this is not per-cpu. */
+static atomic_t dropped[NR_CPUS];
+

Please dont introduce NR_CPUS new arrays, since people are working hard to zap them from kernel.

You probably can use a per_cpu variable ?

Thank you

#17 lore raw

Re: [PATCH 2/4] x86 mmiotrace: fix relay-buffer-full flag for SMP

Quoted 13 lines · On Tue, 05 Feb 2008 21:44:07 +0100 Eric Dumazet <[email protected]> wrote:
Quoted 12 lines · Pekka Paalanen
arch/x86/kernel/mmiotrace/mmio-mod.c +1 -0
index 82ae920..f492b65 100644
@@ -47,9 +48,13 @@ struct trap_reason {
 	int active_traces;
 };
 
+/* Accessed per-cpu. */
 static struct trap_reason pf_reason[NR_CPUS];
 static struct mm_io_header_rw cpu_trace[NR_CPUS];
 
Quoted 8 lines

arch/x86/kernel/mmiotrace/mmio-mod.c struct trap_reason { · line 57

Quoted 3 lines
arch/x86/kernel/mmiotrace/mmio-mod.c +3 -0
+/* Access to this is not per-cpu. */
+static atomic_t dropped[NR_CPUS];
+

Please dont introduce NR_CPUS new arrays, since people are working hard to zap them from kernel.

You probably can use a per_cpu variable ?

Yes, it would probably be more appropriate to use DEFINE_PER_CPU() for 'pf_reason' and 'cpu_trace', but I wasn't sure since the examples of DEFINE_PER_CPU I saw always had integers or pointers, not whole structs. Is it okay for whole structs?

'dropped' on the other hand is not accessed in per-cpu style, any cpu may access any element. DEFINE_PER_CPU is not valid here, is it?

Thanks for the note, I knew I should fix that at some point.

Signature · 2 lines
Pekka Paalanen
http://www.iki.fi/pq/
#18 lore raw

Re: [PATCH 2/4] x86 mmiotrace: fix relay-buffer-full flag for SMP

Quoted 21 lines · Pekka Paalanen
Quoted 18 lines · On Tue, 05 Feb 2008 21:44:07 +0100 Eric Dumazet <[email protected]> wrote:
Quoted 5 lines · Pekka Paalanen
arch/x86/kernel/mmiotrace/mmio-mod.c +0 -0
index 82ae920..f492b65 100644

arch/x86/kernel/mmiotrace/mmio-mod.c struct trap_reason { · line 57

Quoted 10 lines
arch/x86/kernel/mmiotrace/mmio-mod.c +4 -0
@@ -47,9 +48,13 @@ struct trap_reason {
 	int active_traces;
 };
 
+/* Accessed per-cpu. */
 static struct trap_reason pf_reason[NR_CPUS];
 static struct mm_io_header_rw cpu_trace[NR_CPUS];
 
+/* Access to this is not per-cpu. */
+static atomic_t dropped[NR_CPUS];
+

Please dont introduce NR_CPUS new arrays, since people are working hard to zap them from kernel.

Quoted 6 lines
Quoted 1 line

You probably can use a per_cpu variable ?

Yes, it would probably be more appropriate to use DEFINE_PER_CPU() for 'pf_reason' and 'cpu_trace', but I wasn't sure since the examples of DEFINE_PER_CPU I saw always had integers or pointers, not whole structs. Is it okay for whole structs?

yes you can use a structure, you can check for example :

net/ipv4/route.c:static DEFINE_PER_CPU(struct rt_cache_stat, rt_cache_stat);

Quoted 3 lines

'dropped' on the other hand is not accessed in per-cpu style, any cpu may access any element. DEFINE_PER_CPU is not valid here, is it?

It is valid, you can use per_cpu() accessor to get a pointer to a particular cpu data.

check net/ipv4/route.c for an example :

return &per_cpu(rt_cache_stat, cpu);

#19 lore raw

PATCH 1/1 x86 mmiotrace: Use percpu instead of arrays.

---
 arch/x86/mm/kmmio.c    |   27 +++++++++-------
 arch/x86/mm/mmio-mod.c |   80 +++++++++++++++++++++++++----------------------
 2 files changed, 58 insertions(+), 49 deletions(-)
arch/x86/mm/kmmio.c +15 -12
index e759f7c..5e239d0 100644
@@ -16,6 +16,7 @@
 #include <linux/uaccess.h>
 #include <linux/ptrace.h>
 #include <linux/preempt.h>
+#include <linux/percpu.h>
 #include <asm/io.h>
 #include <asm/cacheflush.h>
 #include <asm/errno.h>
@@ -49,7 +50,8 @@ static unsigned int handler_registered;
 static struct list_head kmmio_page_table[KMMIO_PAGE_TABLE_SIZE];
 static LIST_HEAD(kmmio_probes);
 
-static struct kmmio_context kmmio_ctx[NR_CPUS];
+/* Accessed per-cpu */
+static DEFINE_PER_CPU(struct kmmio_context, kmmio_ctx);
 
 static struct notifier_block nb_die = {
 	.notifier_call = kmmio_die_notifier
@@ -173,8 +175,7 @@ static void disarm_kmmio_fault_page(unsigned long page, int *page_level)
  */
 static int kmmio_handler(struct pt_regs *regs, unsigned long addr)
 {
-	struct kmmio_context *ctx;
-	int cpu;
+	struct kmmio_context *ctx = &get_cpu_var(kmmio_ctx);
 
 	/*
 	 * Preemption is now disabled to prevent process switch during
@@ -187,8 +188,6 @@ static int kmmio_handler(struct pt_regs *regs, unsigned long addr)
 	 * And that interrupt triggers a kmmio trap?
 	 */
 	preempt_disable();
-	cpu = smp_processor_id();
-	ctx = &kmmio_ctx[cpu];
 
 	/* interrupts disabled and CPU-local data => atomicity guaranteed. */
 	if (ctx->active) {
@@ -199,7 +198,7 @@ static int kmmio_handler(struct pt_regs *regs, unsigned long addr)
 		 */
 		printk(KERN_EMERG "mmiotrace: recursive probe hit on CPU %d, "
 					"for address %lu. Ignoring.\n",
-					cpu, addr);
+					smp_processor_id(), addr);
 		goto no_kmmio;
 	}
 	ctx->active++;
@@ -231,6 +230,7 @@ static int kmmio_handler(struct pt_regs *regs, unsigned long addr)
 	/* We hold lock, now we set present bit in PTE and single step. */
 	disarm_kmmio_fault_page(ctx->fpage->page, NULL);
 
+	put_cpu_var(kmmio_ctx);
 	return 1;
 
 no_kmmio_locked:
@@ -238,6 +238,7 @@ no_kmmio_locked:
 	ctx->active--;
 no_kmmio:
 	preempt_enable_no_resched();
+	put_cpu_var(kmmio_ctx);
 	/* page fault not handled by kmmio */
 	return 0;
 }
@@ -249,11 +250,11 @@ no_kmmio:
  */
 static int post_kmmio_handler(unsigned long condition, struct pt_regs *regs)
 {
-	int cpu = smp_processor_id();
-	struct kmmio_context *ctx = &kmmio_ctx[cpu];
+	int ret = 0;
+	struct kmmio_context *ctx = &get_cpu_var(kmmio_ctx);
 
 	if (!ctx->active)
-		return 0;
+		goto out;
 
 	if (ctx->probe && ctx->probe->post_handler)
 		ctx->probe->post_handler(ctx->probe, condition, regs);
@@ -273,10 +274,12 @@ static int post_kmmio_handler(unsigned long condition, struct pt_regs *regs)
 	 * will have TF set, in which case, continue the remaining processing
 	 * of do_debug, as if this is not a probe hit.
 	 */
-	if (regs->flags & TF_MASK)
-		return 0;
+	if (!(regs->flags & TF_MASK))
+		ret = 1;
 
-	return 1;
+out:
+	put_cpu_var(kmmio_ctx);
+	return ret;
 }
 
 static int add_kmmio_fault_page(unsigned long page)
arch/x86/mm/mmio-mod.c +43 -37

182 lines not on this page · raw message

Signature · 1 line
1.5.3.7
#20 lore raw

PATCH 3/4 x86 mmiotrace: comment about user space ABI

In reply to Pekka Paalanen
---
 include/linux/mmiotrace.h |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)
include/linux/mmiotrace.h +5 -1
index cb24782..6ec288f 100644
@@ -3,6 +3,10 @@
 
 #include <asm/types.h>
 
+/*
+ * If you change anything here, you must bump MMIO_VERSION.
+ * This is the relay data format for user space.
+ */
 #define MMIO_VERSION 0x04
 
 /* mm_io_header.type */
@@ -23,7 +27,7 @@ enum mm_io_opcode {          /* payload type: */
 };
 
 struct mm_io_header {
-	__u32 type;
+	__u32 type;     /* see MMIO_* macros above */
 	__u32 sec;      /* timestamp */
 	__u32 nsec;
 	__u32 pid;      /* PID of the process, or 0 for kernel core */
Signature · 1 line
1.5.3.7
#21 lore raw

PATCH 4/4 x86 mmiotrace: move files into arch/x86/mm/

In reply to Pekka Paalanen

As this patch is too big for the list, it can be found at:
http://jumi.lut.fi/~paalanen/scratch/mmio25-b/0004-x86-mmiotrace-move-files-into-arch-x86-mm.patch

The patch is 85 kB and Documentation/SubmittingPatches says I should not post it to the list if it exceeds 40 kB.

Ingo, I can also send this directly to your email, if you wish.

Signature · 2 lines
Pekka Paalanen
http://www.iki.fi/pq/
#22 lore raw

Re: [PATCH 4/4] x86 mmiotrace: move files into arch/x86/mm/

Quoted 5 lines · On Tue, 5 Feb 2008 22:39:58 +0200 Pekka Paalanen wrote:

As this patch is too big for the list, it can be found at:
http://jumi.lut.fi/~paalanen/scratch/mmio25-b/0004-x86-mmiotrace-move-files-into-arch-x86-mm.patch

The patch is 85 kB and Documentation/SubmittingPatches says I should not post it to the list if it exceeds 40 kB.

Hm, we need to change that comment. The mailing list limit (for lkml) is now 400 KB AFAIK. Certainly we see plenty of patches that are larger than 40 KB.

Quoted 1 line

Ingo, I can also send this directly to your email, if you wish.

Please email it to the mailing list.

---
~Randy

#23 lore raw

Re: [PATCH 4/4] x86 mmiotrace: move files into arch/x86/mm/

Quoted 1 line

From 788b7aed85c8ef0256dc8e71ae981c4973a6a304 Mon Sep 17 00:00:00 2001

From: Pekka Paalanen <[email protected]>
Date: Mon, 4 Feb 2008 21:26:41 +0200
Subject: [PATCH] x86 mmiotrace: move files into arch/x86/mm/.

---

Hi,

Sorry for being silent, but I am having trouble with my Internet service, I have been practically cut off since Wednesday.

Christoph, this patch shows my last version of mmiotrace, since it is moving all mmiotrace specific files, except include/linux/mmiotrace.h which contains just the relay data representation for user space. This does not include changes Ingo or someone else may have done.

When I catch up with x86/mm git, I can repost everything in one patch series. Given that my ISP's hardware permits...

About the page fault hook, I've been trying to ask what to do with it exactly, wishing for a pointer to an existing example or somesuch. So in lack of better ideas, I'll redo my very first idea of a single function pointer, and see what you think of it. Peter Zijlstra said on IRC it would never get accepted and suggested the list approach, which was the first I posted to LKML.

 arch/x86/kernel/Makefile                  |    2 -
 arch/x86/kernel/mmiotrace/Makefile        |    4 -
 arch/x86/kernel/mmiotrace/kmmio.c         |  407 ----------------------
 arch/x86/kernel/mmiotrace/kmmio.h         |   58 ---
 arch/x86/kernel/mmiotrace/mmio-mod.c      |  538 -----------------------------
 arch/x86/kernel/mmiotrace/pf_in.c         |  489 --------------------------
 arch/x86/kernel/mmiotrace/pf_in.h         |   39 --
 arch/x86/kernel/mmiotrace/testmmiotrace.c |   77 ----
 arch/x86/mm/Makefile                      |    5 +
 arch/x86/mm/kmmio.c                       |  407 ++++++++++++++++++++++
 arch/x86/mm/kmmio.h                       |   58 +++
 arch/x86/mm/mmio-mod.c                    |  538 +++++++++++++++++++++++++++++
 arch/x86/mm/pf_in.c                       |  489 ++++++++++++++++++++++++++
 arch/x86/mm/pf_in.h                       |   39 ++
 arch/x86/mm/testmmiotrace.c               |   77 ++++
 15 files changed, 1613 insertions(+), 1614 deletions(-)
 delete mode 100644 arch/x86/kernel/mmiotrace/Makefile
 delete mode 100644 arch/x86/kernel/mmiotrace/kmmio.c
 delete mode 100644 arch/x86/kernel/mmiotrace/kmmio.h
 delete mode 100644 arch/x86/kernel/mmiotrace/mmio-mod.c
 delete mode 100644 arch/x86/kernel/mmiotrace/pf_in.c
 delete mode 100644 arch/x86/kernel/mmiotrace/pf_in.h
 delete mode 100644 arch/x86/kernel/mmiotrace/testmmiotrace.c
 create mode 100644 arch/x86/mm/kmmio.c
 create mode 100644 arch/x86/mm/kmmio.h
 create mode 100644 arch/x86/mm/mmio-mod.c
 create mode 100644 arch/x86/mm/pf_in.c
 create mode 100644 arch/x86/mm/pf_in.h
 create mode 100644 arch/x86/mm/testmmiotrace.c
arch/x86/kernel/Makefile +0 -2
index 61f5c73..b046f50 100644
@@ -62,8 +62,6 @@ obj-$(CONFIG_KGDB)		+= kgdb.o kgdb-jmp_$(BITS).o
 obj-$(CONFIG_VM86)		+= vm86_32.o
 obj-$(CONFIG_EARLY_PRINTK)	+= early_printk.o
 
-obj-$(CONFIG_MMIOTRACE)		+= mmiotrace/
-
 obj-$(CONFIG_HPET_TIMER) 	+= hpet.o
 
 obj-$(CONFIG_K8_NB)		+= k8.o
arch/x86/kernel/mmiotrace/Makefile deleted +0 -4
deleted file mode 100644
index d6905f7..0000000
@@ -1,4 +0,0 @@
-obj-$(CONFIG_MMIOTRACE) += mmiotrace.o
-mmiotrace-objs := pf_in.o kmmio.o mmio-mod.o
-
-obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o
arch/x86/kernel/mmiotrace/kmmio.c deleted +0 -407

407 lines not on this page · raw message

arch/x86/kernel/mmiotrace/kmmio.h deleted +0 -58

58 lines not on this page · raw message

arch/x86/kernel/mmiotrace/mmio-mod.c deleted +0 -538

538 lines not on this page · raw message

arch/x86/kernel/mmiotrace/pf_in.c deleted +0 -489

489 lines not on this page · raw message

arch/x86/kernel/mmiotrace/pf_in.h deleted +0 -39
deleted file mode 100644
index e05341a..0000000
@@ -1,39 +0,0 @@
-/*
- *  Fault Injection Test harness (FI)
- *  Copyright (C) Intel Crop.
- *
- *  This program is free software; you can redistribute it and/or
- *  modify it under the terms of the GNU General Public License
- *  as published by the Free Software Foundation; either version 2
- *  of the License, or (at your option) any later version.
- *
- *  This program is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *  GNU General Public License for more details.
- *
- *  You should have received a copy of the GNU General Public License
- *  along with this program; if not, write to the Free Software
- *  Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307,
- *  USA.
- *
- */
-
-#ifndef __PF_H_
-#define __PF_H_
-
-enum reason_type {
-	NOT_ME,	/* page fault is not in regions */
-	NOTHING,	/* access others point in regions */
-	REG_READ,	/* read from addr to reg */
-	REG_WRITE,	/* write from reg to addr */
-	IMM_WRITE,	/* write from imm to addr */
-	OTHERS	/* Other instructions can not intercept */
-};
-
-enum reason_type get_ins_type(unsigned long ins_addr);
-unsigned int get_ins_mem_width(unsigned long ins_addr);
-unsigned long get_ins_reg_val(unsigned long ins_addr, struct pt_regs *regs);
-unsigned long get_ins_imm_val(unsigned long ins_addr);
-
-#endif /* __PF_H_ */
arch/x86/kernel/mmiotrace/testmmiotrace.c deleted +0 -77

77 lines not on this page · raw message

arch/x86/mm/Makefile +5 -0
index 9832910..6fe5727 100644
@@ -3,3 +3,8 @@ include ${srctree}/arch/x86/mm/Makefile_32
 else
 include ${srctree}/arch/x86/mm/Makefile_64
 endif
+
+obj-$(CONFIG_MMIOTRACE) += mmiotrace.o
+mmiotrace-objs := pf_in.o kmmio.o mmio-mod.o
+
+obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o
arch/x86/mm/kmmio.c new file +407 -0

407 lines not on this page · raw message

arch/x86/mm/kmmio.h new file +58 -0

58 lines not on this page · raw message

arch/x86/mm/mmio-mod.c new file +538 -0

538 lines not on this page · raw message

arch/x86/mm/pf_in.c new file +489 -0

369 lines not on this page · raw message

arch/x86/mm/pf_in.h new file +39 -0
new file mode 100644
index 0000000..e05341a

Lines of this file are not shown. View the raw message.

arch/x86/mm/testmmiotrace.c new file +77 -0

77 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.5.3.7

Part of the diff is not shown here. View the raw message.

#24 lore raw

Re: [PATCH 4/4] x86 mmiotrace: move files into arch/x86/mm/

In reply to Pekka Paalanen

Pekka,

i've picked up your latest cleanups, but a fundamental worry still
remains: all the ugly 'custom pagefault handlers' stuff. Lets just have one clean direct callback to mmiotrace, ok?

Ingo

#25 lore raw

Re: [PATCH 4/4] x86 mmiotrace: move files into arch/x86/mm/

Quoted 6 lines · Ingo Molnar

Pekka,

i've picked up your latest cleanups, but a fundamental worry still
remains: all the ugly 'custom pagefault handlers' stuff. Lets just have one clean direct callback to mmiotrace, ok?

Yes, please. And any chance we you could resend the current version of the patch to lkml? I've lost a little track of how it looks currently.

#26 lore raw

RFC PATCH 1/1 x86: explicit call to mmiotrace in do_page_fault()

The custom page fault handler list is replaced with a single function pointer. All related functions and variables are renamed for mmiotrace.

---
 arch/x86/Kconfig.debug   |   14 ++++-----
 arch/x86/mm/fault.c      |   66 +++++++++++++++++++++++----------------------
 arch/x86/mm/kmmio.c      |   14 +++++-----
 include/asm-x86/kdebug.h |   12 +++-----
 4 files changed, 52 insertions(+), 54 deletions(-)
arch/x86/Kconfig.debug +6 -8
index 3daf6c1..0fe4120 100644
@@ -169,20 +169,18 @@ config IOMMU_LEAK
 	  Add a simple leak tracer to the IOMMU code. This is useful when you
 	  are debugging a buggy device driver that leaks IOMMU mappings.
 
-config PAGE_FAULT_HANDLERS
-	bool "Custom page fault handlers"
-	depends on DEBUG_KERNEL
-	help
-	  Allow the use of custom page fault handlers. A kernel module may
-	  register a function that is called on every page fault. Custom
-	  handlers are used by some debugging and reverse engineering tools.
+config MMIOTRACE_HOOKS
+	bool
+	default n
 
 config MMIOTRACE
 	tristate "Memory mapped IO tracing"
-	depends on DEBUG_KERNEL && PAGE_FAULT_HANDLERS && RELAY && DEBUG_FS
+	depends on DEBUG_KERNEL && RELAY && DEBUG_FS
+	select MMIOTRACE_HOOKS
 	default n
 	help
 	  This will build a kernel module called mmiotrace.
+	  Making this a built-in is heavily discouraged.
 
 	  Mmiotrace traces Memory Mapped I/O access and is meant for debugging
 	  and reverse engineering. The kernel module offers wrapped
arch/x86/mm/fault.c +34 -32

94 lines not on this page · raw message

arch/x86/mm/kmmio.c +7 -7
index 28411da..e759f7c 100644
@@ -51,10 +51,6 @@ static LIST_HEAD(kmmio_probes);
 
 static struct kmmio_context kmmio_ctx[NR_CPUS];
 
-static struct pf_handler kmmio_pf_hook = {
-	.handler = kmmio_page_fault
-};
-
 static struct notifier_block nb_die = {
 	.notifier_call = kmmio_die_notifier
 };
@@ -77,7 +73,8 @@ void cleanup_kmmio(void)
 	 * kmmio_page_table, kmmio_probes
 	 */
 	if (handler_registered) {
-		unregister_page_fault_handler(&kmmio_pf_hook);
+		if (mmiotrace_unregister_pf(&kmmio_page_fault))
+			BUG();
 		synchronize_rcu();
 	}
 	unregister_die_notifier(&nb_die);
@@ -343,8 +340,11 @@ int register_kmmio_probe(struct kmmio_probe *p)
 	}
 
 	if (!handler_registered) {
-		register_page_fault_handler(&kmmio_pf_hook);
-		handler_registered++;
+		if (mmiotrace_register_pf(&kmmio_page_fault))
+			printk(KERN_ERR "mmiotrace: Cannot register page "
+					"fault handler.\n");
+		else
+			handler_registered++;
 	}
 
 out:
include/asm-x86/kdebug.h +5 -7
index 7ae2118..4680817 100644
@@ -35,13 +35,11 @@ extern void dump_pagetable(unsigned long);
 extern unsigned long oops_begin(void);
 extern void oops_end(unsigned long, struct pt_regs *, int signr);
 
-struct pf_handler {
-	struct hlist_node hlist;
-	int (*handler)(struct pt_regs *regs, unsigned long error_code,
-						unsigned long address);
-};
+typedef int (*pf_handler_func)(struct pt_regs *regs,
+				unsigned long error_code,
+				unsigned long address);
 
-extern void register_page_fault_handler(struct pf_handler *new_pfh);
-extern void unregister_page_fault_handler(struct pf_handler *old_pfh);
+extern int mmiotrace_register_pf(pf_handler_func new_pfh);
+extern int mmiotrace_unregister_pf(pf_handler_func old_pfh);
 
 #endif
Signature · 1 line
1.5.3.7
#27 lore raw

Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()

arch/x86/Kconfig.debug config IOMMU_LEAK · line 183

Quoted 4 lines
arch/x86/Kconfig.debug +1 -0
 	default n
 	help
 	  This will build a kernel module called mmiotrace.
+	  Making this a built-in is heavily discouraged.

why is this? Wouldn't it be nice if distros just shipped with this in their kernel by default for example?

Quoted 15 lines
arch/x86/mm/fault.c +5 -7
-#ifdef CONFIG_PAGE_FAULT_HANDLERS
-static HLIST_HEAD(pf_handlers); /* protected by RCU */
-static DEFINE_SPINLOCK(pf_handlers_writer);
+#ifdef CONFIG_MMIOTRACE_HOOKS
+static pf_handler_func mmiotrace_pf_handler; /* protected by RCU */
+static DEFINE_SPINLOCK(mmiotrace_handler_lock);
 
-void register_page_fault_handler(struct pf_handler *new_pfh)
+int mmiotrace_register_pf(pf_handler_func new_pfh)
 {
+	int ret = 0;
 	unsigned long flags;
-	spin_lock_irqsave(&pf_handlers_writer, flags);
-	hlist_add_head_rcu(&new_pfh->hlist, &pf_handlers);
-	spin_unlock_irqrestore(&pf_handlers_writer, flags);

arch/x86/mm/fault.c line 68

Quoted 10 lines
arch/x86/mm/fault.c +8 -1
+	spin_lock_irqsave(&mmiotrace_handler_lock, flags);
+	if (mmiotrace_pf_handler)
+		ret = -EBUSY;
+	else
+		mmiotrace_pf_handler = new_pfh;
+	spin_unlock_irqrestore(&mmiotrace_handler_lock, flags);
+	return ret;
 }
-EXPORT_SYMBOL_GPL(register_page_fault_handler);
+EXPORT_SYMBOL_GPL(mmiotrace_register_pf);

why not use a weak function instead? should be a ton simpler.

#28 lore raw

Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()

Quoted 7 lines · On Sat, 9 Feb 2008 10:01:33 -0800 Arjan van de Ven <[email protected]> wrote:

arch/x86/Kconfig.debug config IOMMU_LEAK · line 183

Quoted 4 lines
arch/x86/Kconfig.debug +1 -0
 	default n
 	help
 	  This will build a kernel module called mmiotrace.
+	  Making this a built-in is heavily discouraged.

why is this? Wouldn't it be nice if distros just shipped with this in their kernel by default for example?

It's currently not too nice as a built-in. Originally I did have it depend on 'm' but maybe Ingo removed it? It's not depending on 'm' anymore in x86/mm git tree Kconfig.

By default it reserves 32 MB/cpu of relay buffers during module init. It's this big based on experimentation with the nvidia proprietary driver. There is no other way than unloading the module to free the buffers, or even to clear them. This means that every time you start the user space logger application, it gets old data first. Also poking the module parameters via /sys is not allowed.

So it's not actually broken as far as I know, it's just not nice IMHO. And I've never tried it as built-in. I don't think anyone ever has. I do have a plan to make it a nicer built-in, but it's a long way.

Quoted 4 lines
Quoted 4 lines
arch/x86/mm/fault.c +3 -0
+#ifdef CONFIG_MMIOTRACE_HOOKS
+static pf_handler_func mmiotrace_pf_handler; /* protected by RCU */
+static DEFINE_SPINLOCK(mmiotrace_handler_lock);
Quoted 15 lines

arch/x86/mm/fault.c line 68

Quoted 13 lines
arch/x86/mm/fault.c +10 -0
+int mmiotrace_register_pf(pf_handler_func new_pfh)
 {
+	int ret = 0;
 	unsigned long flags;
+	spin_lock_irqsave(&mmiotrace_handler_lock, flags);
+	if (mmiotrace_pf_handler)
+		ret = -EBUSY;
+	else
+		mmiotrace_pf_handler = new_pfh;
+	spin_unlock_irqrestore(&mmiotrace_handler_lock, flags);
+	return ret;
 }
+EXPORT_SYMBOL_GPL(mmiotrace_register_pf);

why not use a weak function instead? should be a ton simpler.

Ookay, this is the first time I hear about weak functions, and that I could accomplish something like this with them. I'll be googling around. These are completely new beasts to me.

Thanks for the keywords, finally I have a clue to follow.

Signature · 2 lines
Pekka Paalanen
http://www.iki.fi/pq/
#29 lore raw

Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()

Hi Pekka,

Quoted 6 lines · Pekka Paalanen

By default it reserves 32 MB/cpu of relay buffers during module init. It's this big based on experimentation with the nvidia proprietary driver. There is no other way than unloading the module to free the buffers, or even to clear them. This means that every time you start the user space logger application, it gets old data first. Also poking the module parameters via /sys is not allowed.

So how is mmiotrace different from, say, blktrace that has an ioctl() to setup/teardown the relay channels (for mmiotrace it can be a "status" file that does the setup/teardown when written some appropriate values)?

Pekka

#30 lore raw

Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()

Quoted 10 lines · On Sat, 9 Feb 2008 20:56:07 +0200 "Pekka Enberg" <[email protected]> wrote:

Hi Pekka,

Quoted 6 lines · Pekka Paalanen

By default it reserves 32 MB/cpu of relay buffers during module init. It's this big based on experimentation with the nvidia proprietary driver. There is no other way than unloading the module to free the buffers, or even to clear them. This means that every time you start the user space logger application, it gets old data first. Also poking the module parameters via /sys is not allowed.

Quoted 4 lines

So how is mmiotrace different from, say, blktrace that has an ioctl() to setup/teardown the relay channels (for mmiotrace it can be a "status" file that does the setup/teardown when written some appropriate values)?

No difference. I just have to do it at some point. I can't work on this every day and when it comes to kernel, I'm a complete newbie. So things will take time. If people want that I fix this before mmiotrace goes into mainline, then so be it.

blktrace, thanks for the hint, I should take a look at that.

Signature · 2 lines
Pekka Paalanen
http://www.iki.fi/pq/
#31 lore raw

Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()

Hi,

Quoted 4 lines · Pekka Paalanen

No difference. I just have to do it at some point. I can't work on this every day and when it comes to kernel, I'm a complete newbie. So things will take time. If people want that I fix this before mmiotrace goes into mainline, then so be it.

Not my call but please keep in mind that once it's in the kernel we can't go breaking the ABI (well, we might as this is strictly a debugging mechanism, but still). Anyway, it shouldn't be that hard to implement and there are plenty of examples in the kernel and you can always as for help if you hit a brick wall ;-).

Pekka

#32 lore raw

Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()

In reply to Arjan van de Ven
Quoted 7 lines · Arjan van de Ven

arch/x86/Kconfig.debug config IOMMU_LEAK · line 183

Quoted 4 lines
arch/x86/Kconfig.debug +1 -0
 	default n
 	help
 	  This will build a kernel module called mmiotrace.
+	  Making this a built-in is heavily discouraged.

why is this? Wouldn't it be nice if distros just shipped with this in their kernel by default for example?

so default m?

Quoted 19 lines
Quoted 19 lines
arch/x86/mm/fault.c +9 -7
-#ifdef CONFIG_PAGE_FAULT_HANDLERS
-static HLIST_HEAD(pf_handlers); /* protected by RCU */
-static DEFINE_SPINLOCK(pf_handlers_writer);
+#ifdef CONFIG_MMIOTRACE_HOOKS
+static pf_handler_func mmiotrace_pf_handler; /* protected by RCU */
+static DEFINE_SPINLOCK(mmiotrace_handler_lock);
 
-void register_page_fault_handler(struct pf_handler *new_pfh)
+int mmiotrace_register_pf(pf_handler_func new_pfh)
 {
+	int ret = 0;
 	unsigned long flags;
-	spin_lock_irqsave(&pf_handlers_writer, flags);
-	hlist_add_head_rcu(&new_pfh->hlist, &pf_handlers);
-	spin_unlock_irqrestore(&pf_handlers_writer, flags);
+	spin_lock_irqsave(&mmiotrace_handler_lock, flags);
+	if (mmiotrace_pf_handler)
+		ret = -EBUSY;
+	else
Quoted 8 lines

arch/x86/mm/fault.c line 68

Quoted 6 lines
arch/x86/mm/fault.c +4 -1
+		mmiotrace_pf_handler = new_pfh;
+	spin_unlock_irqrestore(&mmiotrace_handler_lock, flags);
+	return ret;
 }
-EXPORT_SYMBOL_GPL(register_page_fault_handler);
+EXPORT_SYMBOL_GPL(mmiotrace_register_pf);

why not use a weak function instead? should be a ton simpler.

AFAIC weak functions are a built time thing, not fixed up on module load. So I'm not seeing the applicability there.

Also, it would force a function call, even for the unlikely case the tracer is set.

#33 lore raw

Re: [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault()

In reply to Pekka Paalanen
Quoted 24 lines · Pekka Paalanen
arch/x86/mm/fault.c +15 -0
+int mmiotrace_register_pf(pf_handler_func new_pfh)
 {
+	int ret = 0;
 	unsigned long flags;
 
+	spin_lock_irqsave(&mmiotrace_handler_lock, flags);
+	if (mmiotrace_pf_handler)
+		ret = -EBUSY;
+	else
+		mmiotrace_pf_handler = new_pfh;
+	spin_unlock_irqrestore(&mmiotrace_handler_lock, flags);
+	return ret;
 }
+EXPORT_SYMBOL_GPL(mmiotrace_register_pf);
 
 /**
+ * mmiotrace_unregister_pf:
  * The caller must ensure @old_pfh is not in use anymore before freeing it.
+ * This function does not guarantee it. The handler function pointer is
+ * protected by RCU, so you can do this by e.g. calling synchronize_rcu().
  */
+int mmiotrace_unregister_pf(pf_handler_func old_pfh)
 {
+	int ret = 0;

arch/x86/mm/fault.c

Quoted 10 lines
arch/x86/mm/fault.c +7 -0
 	unsigned long flags;
 
+	spin_lock_irqsave(&mmiotrace_handler_lock, flags);
+	if (mmiotrace_pf_handler != old_pfh)
+		ret = -EPERM;
+	else
+		mmiotrace_pf_handler = NULL;
+	spin_unlock_irqrestore(&mmiotrace_handler_lock, flags);
+	return ret;
 }

So here you set/unset mmiotrace_pf_handler under protection of a spinlock.

arch/x86/mm/fault.c line 103

Quoted 13 lines

/* returns non-zero if do_page_fault() should return */

arch/x86/mm/fault.c +6 -0
+static inline int call_mmiotrace(struct pt_regs *regs,
+					unsigned long error_code,
+					unsigned long address)
 {
+#ifdef CONFIG_MMIOTRACE_HOOKS
 	int ret = 0;
 	rcu_read_lock();
+	if (mmiotrace_pf_handler)
+		ret = mmiotrace_pf_handler(regs, error_code, address);
 	rcu_read_unlock();
 	return ret;
 #else

But here you use it under rcu_read_lock(). This doesn't match.

#34 lore raw

RFC PATCH v2 1/1 x86: explicit call to mmiotrace in do_page_fault()

The custom page fault handler list is replaced with a single function pointer. All related functions and variables are renamed for mmiotrace.

---

Quoted 2 lines · On Sat, 09 Feb 2008 19:39:30 +0100 Peter Zijlstra <[email protected]>…

So here you set/unset mmiotrace_pf_handler under protection of a spinlock.

..

Quoted 1 line

But here you use it under rcu_read_lock(). This doesn't match.

Thanks, I fixed it.

Sorry Jan and Pavel, I forgot to CC you in the first go of this patch. If this makes it into mainline, I don't think it will be left there for many kernel versions. I plan to make kmmio.h as the API towards modules in the future, and the page fault callback will "disappear". I think madwifi could use just fine the kmmio.h functions, if it works basically just like mmiotrace.

 arch/x86/Kconfig.debug   |   14 ++++-----
 arch/x86/mm/fault.c      |   68 ++++++++++++++++++++++++---------------------
 arch/x86/mm/kmmio.c      |   14 +++++-----
 include/asm-x86/kdebug.h |   12 +++-----
 4 files changed, 54 insertions(+), 54 deletions(-)
arch/x86/Kconfig.debug +6 -8
index 3daf6c1..0fe4120 100644
@@ -169,20 +169,18 @@ config IOMMU_LEAK
 	  Add a simple leak tracer to the IOMMU code. This is useful when you
 	  are debugging a buggy device driver that leaks IOMMU mappings.
 
-config PAGE_FAULT_HANDLERS
-	bool "Custom page fault handlers"
-	depends on DEBUG_KERNEL
-	help
-	  Allow the use of custom page fault handlers. A kernel module may
-	  register a function that is called on every page fault. Custom
-	  handlers are used by some debugging and reverse engineering tools.
+config MMIOTRACE_HOOKS
+	bool
+	default n
 
 config MMIOTRACE
 	tristate "Memory mapped IO tracing"
-	depends on DEBUG_KERNEL && PAGE_FAULT_HANDLERS && RELAY && DEBUG_FS
+	depends on DEBUG_KERNEL && RELAY && DEBUG_FS
+	select MMIOTRACE_HOOKS
 	default n
 	help
 	  This will build a kernel module called mmiotrace.
+	  Making this a built-in is heavily discouraged.
 
 	  Mmiotrace traces Memory Mapped I/O access and is meant for debugging
 	  and reverse engineering. The kernel module offers wrapped
arch/x86/mm/fault.c +36 -32

96 lines not on this page · raw message

arch/x86/mm/kmmio.c +7 -7
index 28411da..e759f7c 100644
@@ -51,10 +51,6 @@ static LIST_HEAD(kmmio_probes);
 
 static struct kmmio_context kmmio_ctx[NR_CPUS];
 
-static struct pf_handler kmmio_pf_hook = {
-	.handler = kmmio_page_fault
-};
-
 static struct notifier_block nb_die = {
 	.notifier_call = kmmio_die_notifier
 };
@@ -77,7 +73,8 @@ void cleanup_kmmio(void)
 	 * kmmio_page_table, kmmio_probes
 	 */
 	if (handler_registered) {
-		unregister_page_fault_handler(&kmmio_pf_hook);
+		if (mmiotrace_unregister_pf(&kmmio_page_fault))
+			BUG();
 		synchronize_rcu();
 	}
 	unregister_die_notifier(&nb_die);
@@ -343,8 +340,11 @@ int register_kmmio_probe(struct kmmio_probe *p)
 	}
 
 	if (!handler_registered) {
-		register_page_fault_handler(&kmmio_pf_hook);
-		handler_registered++;
+		if (mmiotrace_register_pf(&kmmio_page_fault))
+			printk(KERN_ERR "mmiotrace: Cannot register page "
+					"fault handler.\n");
+		else
+			handler_registered++;
 	}
 
 out:
include/asm-x86/kdebug.h +5 -7
index 7ae2118..4680817 100644
@@ -35,13 +35,11 @@ extern void dump_pagetable(unsigned long);
 extern unsigned long oops_begin(void);
 extern void oops_end(unsigned long, struct pt_regs *, int signr);
 
-struct pf_handler {
-	struct hlist_node hlist;
-	int (*handler)(struct pt_regs *regs, unsigned long error_code,
-						unsigned long address);
-};
+typedef int (*pf_handler_func)(struct pt_regs *regs,
+				unsigned long error_code,
+				unsigned long address);
 
-extern void register_page_fault_handler(struct pf_handler *new_pfh);
-extern void unregister_page_fault_handler(struct pf_handler *old_pfh);
+extern int mmiotrace_register_pf(pf_handler_func new_pfh);
+extern int mmiotrace_unregister_pf(pf_handler_func old_pfh);
 
 #endif
Signature · 1 line
1.5.3.7
#35 lore raw

Re: [RFC PATCH v2] x86: explicit call to mmiotrace in do_page_fault()

Quoted 6 lines · Pekka Paalanen

Sorry Jan and Pavel, I forgot to CC you in the first go of this patch. If this makes it into mainline, I don't think it will be left there for many kernel versions. I plan to make kmmio.h as the API towards modules in the future, and the page fault callback will "disappear". I think madwifi could use just fine the kmmio.h functions, if it works basically just like mmiotrace.

In fact, the tracing version of MadWifi has a directory called mmiotrace with the files lifted from nouveau. I guess if will just stay in sync with the upstream.

Many thanks for pushing it into the kernel.

Signature · 2 lines
Regards,
Pavel Roskin
#36 lore raw

Re: [RFC PATCH v2] x86: explicit call to mmiotrace in do_page_fault()

Hello,

it seems 2.6.25-rc1 is out and mmiotrace did not make it. Thank you all for your feedback.

Ingo, you can throw away all my patches. I will start from scratch and come back when I have made mmiotrace built-in-able and got rid of the page fault hook. It should make a nicer patch set as a whole.

Signature · 2 lines
Pekka Paalanen
http://www.iki.fi/pq/
#37 lore raw

Re: [PATCH 1/4] x86 mmiotrace: use lookup_address()

In reply to Pekka Paalanen
Quoted 4 lines · Pekka Paalanen

Use lookup_address() from pageattr.c instead of doing the same manually. Also had to EXPORT_SYMBOL(lookup_address) to make this work for modules. This also fixes "undefined symbol 'init_mm'" compile error for x86_32.

This should be a _GPL export for sure.

#38 lore raw

Re: [PATCH 1/4] x86 mmiotrace: use lookup_address()

Quoted 7 lines · Christoph Hellwig
Quoted 4 lines · Pekka Paalanen

Use lookup_address() from pageattr.c instead of doing the same manually. Also had to EXPORT_SYMBOL(lookup_address) to make this work for modules. This also fixes "undefined symbol 'init_mm'" compile error for x86_32.

This should be a _GPL export for sure.

i've changed it to that.

Ingo

#39 lore raw

RFC PATCH v2 1/1 x86: mmiotrace - trace memory mapped IO

In reply to Pekka Paalanen
Quoted 4 lines
Quoted 4 lines

could you please send us a patch for the whole mmiotrace kernel-side feature, so that we can have a look at the general structure of this? (and the interaction with change_page_attr(), etc.) Even if it's not functional (and wont even build/boot) at the moment. Thanks,

And here's mmiotrace. This works for me (amd64), but not for a test user who has 32-bit x86. We have not had the chance to debug it yet. The symptom is "everything locked up" after insmodding testmmiotrace.ko, targeting the mid-point of his VRAM. And there's the "undefined symbol init_mm" problem on 32-bit, but we just put the export back in (not included in these patches).

No-one has tried this on SMP. I have a Core 2 Duo laptop where I will test this when I get the chance.

Quoted 1 line

From 10000bf67af4900b75a56cb47726d7b63d11d7e8 Mon Sep 17 00:00:00 2001

From: Pekka Paalanen <[email protected]>
Date: Wed, 30 Jan 2008 21:51:51 +0200
Subject: [PATCH] x86: mmiotrace - trace memory mapped IO

Mmiotrace is a tool for trapping memory mapped IO (MMIO) accesses within the kernel. It is used for debugging and especially for reverse engineering evil binary drivers.

Mmiotrace works by wrapping the ioremap family of kernel functions and marking the returned pages as not present. Access to the IO memory triggers a page fault, which will be handled by mmiotrace's custom page fault handler. This will single-step the faulted instruction with the MMIO page marked as present. Access logs are directed to user space via relay and debug_fs.

This page fault approach is necessary, because binary drivers have readl/writel etc. calls inlined and therefore extremely difficult to trap with with e.g. kprobes.

This patch depends on the custom page fault handlers patch.

---
 arch/x86/Kconfig.debug                    |   28 ++
 arch/x86/kernel/Makefile                  |    2 +
 arch/x86/kernel/mmiotrace/Makefile        |    4 +
 arch/x86/kernel/mmiotrace/kmmio.c         |  391 +++++++++++++++++++++
 arch/x86/kernel/mmiotrace/kmmio.h         |   58 ++++
 arch/x86/kernel/mmiotrace/mmio-mod.c      |  527 +++++++++++++++++++++++++++++
 arch/x86/kernel/mmiotrace/pf_in.c         |  489 ++++++++++++++++++++++++++
 arch/x86/kernel/mmiotrace/pf_in.h         |   39 +++
 arch/x86/kernel/mmiotrace/testmmiotrace.c |   77 +++++
 include/linux/mmiotrace.h                 |   62 ++++
 10 files changed, 1677 insertions(+), 0 deletions(-)
 create mode 100644 arch/x86/kernel/mmiotrace/Makefile
 create mode 100644 arch/x86/kernel/mmiotrace/kmmio.c
 create mode 100644 arch/x86/kernel/mmiotrace/kmmio.h
 create mode 100644 arch/x86/kernel/mmiotrace/mmio-mod.c
 create mode 100644 arch/x86/kernel/mmiotrace/pf_in.c
 create mode 100644 arch/x86/kernel/mmiotrace/pf_in.h
 create mode 100644 arch/x86/kernel/mmiotrace/testmmiotrace.c
 create mode 100644 include/linux/mmiotrace.h
arch/x86/Kconfig.debug +28 -0
index 1185dcc..486dca5 100644
@@ -146,6 +146,34 @@ config PAGE_FAULT_HANDLERS
 	  register a function that is called on every page fault. Custom
 	  handlers are used by some debugging and reverse engineering tools.
 
+config MMIOTRACE
+	tristate "Memory mapped IO tracing"
+	depends on DEBUG_KERNEL && m
+	depends on PAGE_FAULT_HANDLERS && RELAY && DEBUG_FS
+	default n
+	help
+	  This will build a kernel module called mmiotrace.
+
+	  Mmiotrace traces Memory Mapped I/O access and is meant for debugging
+	  and reverse engineering. The kernel module offers wrapped
+	  versions of the ioremap family of functions. The driver to be traced
+	  must be modified to call these wrappers. A user space program is
+	  required to collect the MMIO data.
+
+	  See http://nouveau.freedesktop.org/wiki/MmioTrace
+	  If you are not helping to develop drivers, say N.
+
+config MMIOTRACE_TEST
+	tristate "Test module for mmiotrace"
+	depends on MMIOTRACE && m
+	default n
+	help
+	  This is a dumb module for testing mmiotrace. It is very dangerous
+	  as it will write garbage to IO memory starting at a given address.
+	  However, it should be safe to use on e.g. unused portion of VRAM.
+
+	  Say N, unless you absolutely know what you are doing.
+
 #
 # IO delay types:
 #
arch/x86/kernel/Makefile +2 -0
index 7954e9d..469673f 100644
@@ -61,6 +61,8 @@ obj-$(CONFIG_KGDB)		+= kgdb.o kgdb-jmp_$(BITS).o
 obj-$(CONFIG_VM86)		+= vm86_32.o
 obj-$(CONFIG_EARLY_PRINTK)	+= early_printk.o
 
+obj-$(CONFIG_MMIOTRACE)		+= mmiotrace/
+
 obj-$(CONFIG_HPET_TIMER) 	+= hpet.o
 
 obj-$(CONFIG_K8_NB)		+= k8.o
arch/x86/kernel/mmiotrace/Makefile new file +4 -0
new file mode 100644
index 0000000..d6905f7
@@ -0,0 +1,4 @@
+obj-$(CONFIG_MMIOTRACE) += mmiotrace.o
+mmiotrace-objs := pf_in.o kmmio.o mmio-mod.o
+
+obj-$(CONFIG_MMIOTRACE_TEST) += testmmiotrace.o
arch/x86/kernel/mmiotrace/kmmio.c new file +391 -0

391 lines not on this page · raw message

arch/x86/kernel/mmiotrace/kmmio.h new file +58 -0

58 lines not on this page · raw message

arch/x86/kernel/mmiotrace/mmio-mod.c new file +527 -0

527 lines not on this page · raw message

arch/x86/kernel/mmiotrace/pf_in.c new file +489 -0

489 lines not on this page · raw message

arch/x86/kernel/mmiotrace/pf_in.h new file +39 -0
new file mode 100644
index 0000000..e05341a
@@ -0,0 +1,39 @@
+/*
+ *  Fault Injection Test harness (FI)
+ *  Copyright (C) Intel Crop.
+ *
+ *  This program is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU General Public License
+ *  as published by the Free Software Foundation; either version 2
+ *  of the License, or (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307,
+ *  USA.
+ *
+ */
+
+#ifndef __PF_H_
+#define __PF_H_
+
+enum reason_type {
+	NOT_ME,	/* page fault is not in regions */
+	NOTHING,	/* access others point in regions */
+	REG_READ,	/* read from addr to reg */
+	REG_WRITE,	/* write from reg to addr */
+	IMM_WRITE,	/* write from imm to addr */
+	OTHERS	/* Other instructions can not intercept */
+};
+
+enum reason_type get_ins_type(unsigned long ins_addr);
+unsigned int get_ins_mem_width(unsigned long ins_addr);
+unsigned long get_ins_reg_val(unsigned long ins_addr, struct pt_regs *regs);
+unsigned long get_ins_imm_val(unsigned long ins_addr);
+
+#endif /* __PF_H_ */
arch/x86/kernel/mmiotrace/testmmiotrace.c new file +77 -0

77 lines not on this page · raw message

include/linux/mmiotrace.h new file +62 -0

62 lines not on this page · raw message

Signature · 1 line
1.5.3.7
#40 lore raw

Re: [RFC PATCH v2] x86: mmiotrace - trace memory mapped IO

Quoted 13 lines · On Thu, 31 Jan 2008 18:16:30 +0200 Pekka Paalanen <[email protected]> wrote:
Quoted 5 lines
Quoted 5 lines

could you please send us a patch for the whole mmiotrace kernel-side feature, so that we can have a look at the general structure of this? (and the interaction with change_page_attr(), etc.) Even if it's not functional (and wont even build/boot) at the moment. Thanks,

And here's mmiotrace. This works for me (amd64), but not for a test user who has 32-bit x86. We have not had the chance to debug it yet. The symptom is "everything locked up" after insmodding testmmiotrace.ko, targeting the mid-point of his VRAM. And there's the "undefined symbol init_mm" problem on 32-bit, but we just put the export back in (not included in these patches).

Quoted 2 lines

No-one has tried this on SMP. I have a Core 2 Duo laptop where I will test this when I get the chance.

+static void print_pte(unsigned long address)

Quoted 16 lines
arch/x86/kernel/mmiotrace/mmio-mod.c new file +7 -0
+{
+	pgd_t *pgd = pgd_offset_k(address);
+	pud_t *pud = pud_offset(pgd, address);
+	pmd_t *pmd = pmd_offset(pud, address);
+	if (pmd_large(*pmd)) {
+		printk(KERN_EMERG MODULE_NAME ": 4MB pages are not "
+						"currently

supported: %lx\n",

arch/x86/kernel/mmiotrace/mmio-mod.c new file +4 -0
+						address);
+		BUG();
+	}
+	printk(KERN_DEBUG MODULE_NAME ": pte for 0x%lx: 0x%lx

0x%lx\n",

arch/x86/kernel/mmiotrace/mmio-mod.c new file +3 -0
+		address,
+		pte_val(*pte_offset_kernel(pmd, address)),
+		pte_val(*pte_offset_kernel(pmd, address)) &
Quoted 1 line

_PAGE_PRESENT); +}

ok this wants to use some of the page_address() and other helpers from pageattr.c; that should be easy to solve during integration
(I can even see a more generic version of this function moving there)

Quoted 7 lines
arch/x86/kernel/mmiotrace/mmio-mod.c new file +3 -0
+/* file_create() callback.  Creates relay file in debugfs. */
+static struct dentry *create_buf_file_handler(const char *filename,
+						struct dentry

*parent,

(quoted diff) +2 -0
+						int mode,
+						struct rchan_buf

*buf,

arch/x86/kernel/mmiotrace/mmio-mod.c line 299

Quoted 10 lines
arch/x86/kernel/mmiotrace/mmio-mod.c new file +10 -0
+						int *is_global)
+{
+	struct dentry *buf_file;
+
+	mmio_fops.read = relay_file_operations.read;
+	mmio_fops.open = relay_file_operations.open;
+	mmio_fops.poll = relay_file_operations.poll;
+	mmio_fops.mmap = relay_file_operations.mmap;
+	mmio_fops.release = relay_file_operations.release;
+	mmio_fops.splice_read = relay_file_operations.splice_read;

can't we do this at compile time?

Quoted 20 lines
(quoted diff) +2 -0
+
+void __iomem *ioremap_cache_trace(unsigned long offset, unsigned

long size) +{

(quoted diff) +2 -0
+	void __iomem *p = ioremap_cache(offset, size);
+	printk(KERN_DEBUG MODULE_NAME ": ioremap_cache(0x%lx, 0x%lx)

= %p\n",

(quoted diff) +1 -0
+							offset,

size, p);

arch/x86/kernel/mmiotrace/mmio-mod.c new file +6 -0
+	ioremap_trace_core(offset, size, p);
+	return p;
+}
+EXPORT_SYMBOL(ioremap_cache_trace);
+
+void __iomem *ioremap_nocache_trace(unsigned long offset, unsigned

long size) +{

(quoted diff) +2 -0
+	void __iomem *p = ioremap_nocache(offset, size);
+	printk(KERN_DEBUG MODULE_NAME ": ioremap_nocache(0x%lx,

0x%lx) = %p\n",

(quoted diff) +1 -0
+							offset,

size, p);

arch/x86/kernel/mmiotrace/mmio-mod.c line 419

Quoted 4 lines
arch/x86/kernel/mmiotrace/mmio-mod.c new file +4 -0
+	ioremap_trace_core(offset, size, p);
+	return p;
+}
+EXPORT_SYMBOL(ioremap_nocache_trace);

I would rather keep the split I think and make the trace call explicit

or.. even nicer, we could go fancy and allow the mmio tracer to "subscribe" to, say, a PCI bar, and auto-enable tracing when the bar gets ioremap'd

about the isntruction decoding.. would be nice if we can share that with kvm so that we can debug that stuff once ;-)

other than that I think the code is quite nice already, seems getting this merge ready isn't going to be an enormous deal in terms of complexity

#41 lore raw

Re: [RFC PATCH v2] x86: mmiotrace - trace memory mapped IO

Quoted 9 lines · On Thu, 31 Jan 2008 08:29:06 -0800 Arjan van de Ven <[email protected]> wrote:
Quoted 6 lines · On Thu, 31 Jan 2008 18:16:30 +0200 Pekka Paalanen <[email protected]> wrote:

And here's mmiotrace. This works for me (amd64), but not for a test user who has 32-bit x86. We have not had the chance to debug it yet. The symptom is "everything locked up" after insmodding testmmiotrace.ko, targeting the mid-point of his VRAM. And there's the "undefined symbol init_mm" problem on 32-bit, but we just put the export back in (not included in these patches).

This problem seems to have been due to kgdb being enabled. Quote from Jaap Stolk, the test user:
"(test)mmiotrace will trigger wait for serial debugger if kgdb and CONFIG_KGDB_ATTACH_WAIT is enabled."
After disabling it, mmiotrace ran fine.

Quoted 2 lines
Quoted 2 lines

No-one has tried this on SMP. I have a Core 2 Duo laptop where I will test this when I get the chance.

I tested, worked just fine. Well, the test module does not really test simultaneous iomem access. And I have an issue with following relay buffers' full/not-full status, I'm looking into that now.

Quoted 3 lines

ok this wants to use some of the page_address() and other helpers from pageattr.c; that should be easy to solve during integration
(I can even see a more generic version of this function moving there)

page_address() seems to be dealing with struct page, someone said we usually don't have that (ioremap).

Is this what you had in mind? I had to add an export for lookup_address() to make this work.

 arch/x86/kernel/mmiotrace/kmmio.c    |   46 +++++++++++++++++++++++-----------
 arch/x86/kernel/mmiotrace/mmio-mod.c |   19 +++++++++-----
 arch/x86/mm/pageattr.c               |    1 +
 3 files changed, 44 insertions(+), 22 deletions(-)
arch/x86/kernel/mmiotrace/kmmio.c +31 -15
index 8ba48f9..28411da 100644
@@ -20,6 +20,7 @@
 #include <asm/cacheflush.h>
 #include <asm/errno.h>
 #include <asm/tlbflush.h>
+#include <asm/pgtable.h>
 
 #include "kmmio.h"
 
@@ -117,40 +118,55 @@ static struct kmmio_fault_page *get_kmmio_fault_page(unsigned long page)
 	return NULL;
 }
 
-static void arm_kmmio_fault_page(unsigned long page, int *large)
+static void arm_kmmio_fault_page(unsigned long page, int *page_level)
 {
 	unsigned long address = page & PAGE_MASK;
-	pgd_t *pgd = pgd_offset_k(address);
-	pud_t *pud = pud_offset(pgd, address);
-	pmd_t *pmd = pmd_offset(pud, address);
-	pte_t *pte = pte_offset_kernel(pmd, address);
+	int level;
+	pte_t *pte = lookup_address(address, &level);
 
-	if (pmd_large(*pmd)) {
+	if (!pte) {
+		printk(KERN_ERR "Error in %s: no pte for page 0x%08lx\n",
+						__FUNCTION__, page);
+		return;
+	}
+
+	if (level == PG_LEVEL_2M) {
+		pmd_t *pmd = (pmd_t *)pte;
 		set_pmd(pmd, __pmd(pmd_val(*pmd) & ~_PAGE_PRESENT));
-		if (large)
-			*large = 1;
 	} else {
+		/* PG_LEVEL_4K */
 		set_pte(pte, __pte(pte_val(*pte) & ~_PAGE_PRESENT));
 	}
 
+	if (page_level)
+		*page_level = level;
+
 	__flush_tlb_one(page);
 }
 
-static void disarm_kmmio_fault_page(unsigned long page, int *large)
+static void disarm_kmmio_fault_page(unsigned long page, int *page_level)
 {
 	unsigned long address = page & PAGE_MASK;
-	pgd_t *pgd = pgd_offset_k(address);
-	pud_t *pud = pud_offset(pgd, address);
-	pmd_t *pmd = pmd_offset(pud, address);
-	pte_t *pte = pte_offset_kernel(pmd, address);
+	int level;
+	pte_t *pte = lookup_address(address, &level);
 
-	if (large && *large) {
+	if (!pte) {
+		printk(KERN_ERR "Error in %s: no pte for page 0x%08lx\n",
+						__FUNCTION__, page);
+		return;
+	}
+
+	if (level == PG_LEVEL_2M) {
+		pmd_t *pmd = (pmd_t *)pte;
 		set_pmd(pmd, __pmd(pmd_val(*pmd) | _PAGE_PRESENT));
-		*large = 0;
 	} else {
+		/* PG_LEVEL_4K */
 		set_pte(pte, __pte(pte_val(*pte) | _PAGE_PRESENT));
 	}
 
+	if (page_level)
+		*page_level = level;
+
 	__flush_tlb_one(page);
 }
 
arch/x86/kernel/mmiotrace/mmio-mod.c +12 -7
index 4a8c859..82ae920 100644
@@ -120,19 +120,24 @@ static int write_marker(struct file *file, const char __user *buffer,
 
 static void print_pte(unsigned long address)
 {
-	pgd_t *pgd = pgd_offset_k(address);
-	pud_t *pud = pud_offset(pgd, address);
-	pmd_t *pmd = pmd_offset(pud, address);
-	if (pmd_large(*pmd)) {
+	int level;
+	pte_t *pte = lookup_address(address, &level);
+
+	if (!pte) {
+		printk(KERN_ERR "Error in %s: no pte for page 0x%08lx\n",
+						__FUNCTION__, address);
+		return;
+	}
+
+	if (level == PG_LEVEL_2M) {
 		printk(KERN_EMERG MODULE_NAME ": 4MB pages are not "
 						"currently supported: %lx\n",
 						address);
 		BUG();
 	}
 	printk(KERN_DEBUG MODULE_NAME ": pte for 0x%lx: 0x%lx 0x%lx\n",
-		address,
-		pte_val(*pte_offset_kernel(pmd, address)),
-		pte_val(*pte_offset_kernel(pmd, address)) & _PAGE_PRESENT);
+					address, pte_val(*pte),
+					pte_val(*pte) & _PAGE_PRESENT);
 }
 
 /*
arch/x86/mm/pageattr.c +1 -0
index b8a8408..7f0a84e 100644
@@ -194,6 +194,7 @@ pte_t *lookup_address(unsigned long address, int *level)
 	*level = PG_LEVEL_4K;
 	return pte_offset_kernel(pmd, address);
 }
+EXPORT_SYMBOL(lookup_address);
 
 static void __set_pmd_pte(pte_t *kpte, unsigned long address, pte_t pte)
 {
Quoted 8 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+	mmio_fops.read = relay_file_operations.read;
+	mmio_fops.open = relay_file_operations.open;
+	mmio_fops.poll = relay_file_operations.poll;
+	mmio_fops.mmap = relay_file_operations.mmap;
+	mmio_fops.release = relay_file_operations.release;
+	mmio_fops.splice_read = relay_file_operations.splice_read;

can't we do this at compile time?

Nope, the compiler says the initializer is not constant.

Quoted 18 lines
Quoted 18 lines
(quoted diff) +2 -0
+
+void __iomem *ioremap_cache_trace(unsigned long offset, unsigned

long size) +{

(quoted diff) +2 -0
+	void __iomem *p = ioremap_cache(offset, size);
+	printk(KERN_DEBUG MODULE_NAME ": ioremap_cache(0x%lx, 0x%lx)

= %p\n",

(quoted diff) +1 -0
+							offset,

size, p);

(quoted diff) +6 -0
+	ioremap_trace_core(offset, size, p);
+	return p;
+}
+EXPORT_SYMBOL(ioremap_cache_trace);
+
+void __iomem *ioremap_nocache_trace(unsigned long offset, unsigned

long size) +{

(quoted diff) +2 -0
+	void __iomem *p = ioremap_nocache(offset, size);
+	printk(KERN_DEBUG MODULE_NAME ": ioremap_nocache(0x%lx,

0x%lx) = %p\n",

Quoted 8 lines

Review

Quoted 6 lines
(quoted diff) +1 -0
+							offset,

size, p);

(quoted diff) +4 -0
+	ioremap_trace_core(offset, size, p);
+	return p;
+}
+EXPORT_SYMBOL(ioremap_nocache_trace);

I would rather keep the split I think and make the trace call explicit

I don't think I understand what you mean. Get rid of (do_)ioremap_trace_core?

Quoted 2 lines

or.. even nicer, we could go fancy and allow the mmio tracer to "subscribe" to, say, a PCI bar, and auto-enable tracing when the bar gets ioremap'd

Might be cool, but not until there's no hope of getting into 2.6.25.

Quoted 2 lines

about the isntruction decoding.. would be nice if we can share that with kvm so that we can debug that stuff once ;-)

Uhh yeah, if you mean the stuff in drivers/kvm/x86_emulate.c, it indeed looks like it's decoding instructions. Other than that, I understood almost nothing in there. There's one gigantic function that seems to decode and emulate an instruction. I would have to refactor all that code and I'm not up for it.

Quoted 2 lines

other than that I think the code is quite nice already, seems getting this merge ready isn't going to be an enormous deal in terms of complexity

Thank you for your kind and constructive comments.

Signature · 2 lines
Pekka Paalanen
http://www.iki.fi/pq/
#42 lore raw
In reply to Pekka Paalanen
Quoted 7 lines · On Sun, 27 Jan 2008 18:52:38 +0200 Pekka Paalanen <[email protected]> wrote:

From: Pekka Paalanen <[email protected]>

Provides kernel modules a way to register custom page fault handlers. On every page fault, except those handled in vmalloc_fault(), this will call a list of registered functions. The functions may handle the fault and force do_page_fault() to return immediately.

Quoted 7 lines

This functionality is similar to the now removed page fault notifiers. Custom page fault handlers are used by debugging and reverse engineering tools. Mmio-trace is one such tool and a patch to add it into the tree will follow.

The custom page fault handlers are called from the exact same points in do_page_fault() as the page fault notifiers were.

to be honest I don't like this approach.
I fully would like to see mmio tracing in the kernel, but please make the hook explicit and direct (eg call the mmio tracing function, not some list) instead...

Signature · 3 lines
If you want to reach me at my work email, use [email protected]
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org