Skip to contentSkip to conversation

x86/mm/kmmio: Fix mmiotrace for page unaligned addresses

PATCH In review Karol Herbst 6 participants 10 messages Nov 27, 2017 – Dec 13, 2017

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/mm/kmmio: Fix mmiotrace for page unaligned addresses

If something calls ioremap with an address not aligned to PAGE_SIZE, the returned address might be not aligned as well. This led to a probe registered on exactly the returned address, but the entire page was armed for mmiotracing.

On calling iounmap the address passed to unregister_kmmio_probe was PAGE_SIZE aligned by the caller leading to a complete freeze of the machine.

We should always page align addresses while (un)registerung mappings, because the mmiotracer works on top of pages, not mappings. We still keep track of the probes based on their real addresses and lengths though, because the mmiotrace still needs to know what are mapped memory regions.

Also move the call to mmiotrace_iounmap prior page aligning the address, so that all probes are unregistered properly, otherwise the kernel ends up failing memory allocations randomly after disabling the mmiotracer.

---
 arch/x86/mm/ioremap.c |  4 ++--
 arch/x86/mm/kmmio.c   | 12 +++++++-----
 2 files changed, 9 insertions(+), 7 deletions(-)
arch/x86/mm/ioremap.c +2 -2
index 34f0e1847dd6..5d4c358778dd 100644
@@ -349,11 +349,11 @@ void iounmap(volatile void __iomem *addr)
 		return;
 	}
 
+	mmiotrace_iounmap(addr);
+
 	addr = (volatile void __iomem *)
 		(PAGE_MASK & (unsigned long __force)addr);
 
-	mmiotrace_iounmap(addr);
-
 	/* Use the vm area unlocked, assuming the caller
 	   ensures there isn't another iounmap for the same address
 	   in parallel. Reuse of the virtual address is prevented by
arch/x86/mm/kmmio.c +7 -5
index c21c2ed04612..58477ec3d66d 100644
@@ -435,17 +435,18 @@ int register_kmmio_probe(struct kmmio_probe *p)
 	unsigned long flags;
 	int ret = 0;
 	unsigned long size = 0;
+	unsigned long addr = p->addr & PAGE_MASK;
 	const unsigned long size_lim = p->len + (p->addr & ~PAGE_MASK);
 	unsigned int l;
 	pte_t *pte;
 
 	spin_lock_irqsave(&kmmio_lock, flags);
-	if (get_kmmio_probe(p->addr)) {
+	if (get_kmmio_probe(addr)) {
 		ret = -EEXIST;
 		goto out;
 	}
 
-	pte = lookup_address(p->addr, &l);
+	pte = lookup_address(addr, &l);
 	if (!pte) {
 		ret = -EINVAL;
 		goto out;
@@ -454,7 +455,7 @@ int register_kmmio_probe(struct kmmio_probe *p)
 	kmmio_count++;
 	list_add_rcu(&p->list, &kmmio_probes);
 	while (size < size_lim) {
-		if (add_kmmio_fault_page(p->addr + size))
+		if (add_kmmio_fault_page(addr + size))
 			pr_err("Unable to set page fault.\n");
 		size += page_level_size(l);
 	}
@@ -528,19 +529,20 @@ void unregister_kmmio_probe(struct kmmio_probe *p)
 {
 	unsigned long flags;
 	unsigned long size = 0;
+	unsigned long addr = p->addr & PAGE_MASK;
 	const unsigned long size_lim = p->len + (p->addr & ~PAGE_MASK);
 	struct kmmio_fault_page *release_list = NULL;
 	struct kmmio_delayed_release *drelease;
 	unsigned int l;
 	pte_t *pte;
 
-	pte = lookup_address(p->addr, &l);
+	pte = lookup_address(addr, &l);
 	if (!pte)
 		return;
 
 	spin_lock_irqsave(&kmmio_lock, flags);
 	while (size < size_lim) {
-		release_kmmio_fault_page(p->addr + size, &release_list);
+		release_kmmio_fault_page(addr + size, &release_list);
 		size += page_level_size(l);
 	}
 	list_del_rcu(&p->list);
Signature · 6 lines
2.14.3

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
#1 lore raw
x86/mm/kmmio: Fix mmiotrace for page unaligned addresses
Commit-ID:  6d60ce384d1d5ca32b595244db4077a419acc687
Gitweb:     https://git.kernel.org/tip/6d60ce384d1d5ca32b595244db4077a419acc687
Author:     Karol Herbst <[email protected]>
AuthorDate: Mon, 27 Nov 2017 08:51:39 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Mon, 11 Dec 2017 15:35:18 +0100

x86/mm/kmmio: Fix mmiotrace for page unaligned addresses

If something calls ioremap() with an address not aligned to PAGE_SIZE, the returned address might be not aligned as well. This led to a probe registered on exactly the returned address, but the entire page was armed for mmiotracing.

On calling iounmap() the address passed to unregister_kmmio_probe() was PAGE_SIZE aligned by the caller leading to a complete freeze of the machine.

We should always page align addresses while (un)registerung mappings, because the mmiotracer works on top of pages, not mappings. We still keep track of the probes based on their real addresses and lengths though, because the mmiotrace still needs to know what are mapped memory regions.

Also move the call to mmiotrace_iounmap() prior page aligning the address, so that all probes are unregistered properly, otherwise the kernel ends up failing memory allocations randomly after disabling the mmiotracer.

---
 arch/x86/mm/ioremap.c |  4 ++--
 arch/x86/mm/kmmio.c   | 12 +++++++-----
 2 files changed, 9 insertions(+), 7 deletions(-)
arch/x86/mm/ioremap.c +2 -2
index 6e4573b..c45b6ec 100644
@@ -404,11 +404,11 @@ void iounmap(volatile void __iomem *addr)
 		return;
 	}
 
+	mmiotrace_iounmap(addr);
+
 	addr = (volatile void __iomem *)
 		(PAGE_MASK & (unsigned long __force)addr);
 
-	mmiotrace_iounmap(addr);
-
 	/* Use the vm area unlocked, assuming the caller
 	   ensures there isn't another iounmap for the same address
 	   in parallel. Reuse of the virtual address is prevented by
arch/x86/mm/kmmio.c +7 -5
index c21c2ed..58477ec 100644
@@ -435,17 +435,18 @@ int register_kmmio_probe(struct kmmio_probe *p)
 	unsigned long flags;
 	int ret = 0;
 	unsigned long size = 0;
+	unsigned long addr = p->addr & PAGE_MASK;
 	const unsigned long size_lim = p->len + (p->addr & ~PAGE_MASK);
 	unsigned int l;
 	pte_t *pte;
 
 	spin_lock_irqsave(&kmmio_lock, flags);
-	if (get_kmmio_probe(p->addr)) {
+	if (get_kmmio_probe(addr)) {
 		ret = -EEXIST;
 		goto out;
 	}
 
-	pte = lookup_address(p->addr, &l);
+	pte = lookup_address(addr, &l);
 	if (!pte) {
 		ret = -EINVAL;
 		goto out;
@@ -454,7 +455,7 @@ int register_kmmio_probe(struct kmmio_probe *p)
 	kmmio_count++;
 	list_add_rcu(&p->list, &kmmio_probes);
 	while (size < size_lim) {
-		if (add_kmmio_fault_page(p->addr + size))
+		if (add_kmmio_fault_page(addr + size))
 			pr_err("Unable to set page fault.\n");
 		size += page_level_size(l);
 	}
@@ -528,19 +529,20 @@ void unregister_kmmio_probe(struct kmmio_probe *p)
 {
 	unsigned long flags;
 	unsigned long size = 0;
+	unsigned long addr = p->addr & PAGE_MASK;
 	const unsigned long size_lim = p->len + (p->addr & ~PAGE_MASK);
 	struct kmmio_fault_page *release_list = NULL;
 	struct kmmio_delayed_release *drelease;
 	unsigned int l;
 	pte_t *pte;
 
-	pte = lookup_address(p->addr, &l);
+	pte = lookup_address(addr, &l);
 	if (!pte)
 		return;
 
 	spin_lock_irqsave(&kmmio_lock, flags);
 	while (size < size_lim) {
-		release_kmmio_fault_page(p->addr + size, &release_list);
+		release_kmmio_fault_page(addr + size, &release_list);
 		size += page_level_size(l);
 	}
 	list_del_rcu(&p->list);
maintainer
#2 lore raw
Quoted 8 lines · tip-bot for Karol Herbst
Commit-ID:  6d60ce384d1d5ca32b595244db4077a419acc687
Gitweb:     https://git.kernel.org/tip/6d60ce384d1d5ca32b595244db4077a419acc687
Author:     Karol Herbst <[email protected]>
AuthorDate: Mon, 27 Nov 2017 08:51:39 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Mon, 11 Dec 2017 15:35:18 +0100

x86/mm/kmmio: Fix mmiotrace for page unaligned addresses

OK, let me hijack this thread since apparently people use and care about mmiotrace.

I was recently auditing the x86 tlb flushing and ran across this 'thing'. Can someone please explain to me how this is supposed to work and how its not completely broken?

#3 lore raw
Quoted 8 lines · Peter Zijlstra
Quoted 7 lines · tip-bot for Karol Herbst
Commit-ID:  6d60ce384d1d5ca32b595244db4077a419acc687
Gitweb:     https://git.kernel.org/tip/6d60ce384d1d5ca32b595244db4077a419acc687
Author:     Karol Herbst <[email protected]>
AuthorDate: Mon, 27 Nov 2017 08:51:39 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Mon, 11 Dec 2017 15:35:18 +0100
Quoted 8 lines
Quoted 1 line

x86/mm/kmmio: Fix mmiotrace for page unaligned addresses

OK, let me hijack this thread since apparently people use and care about mmiotrace.

I was recently auditing the x86 tlb flushing and ran across this 'thing'. Can someone please explain to me how this is supposed to work and how its not completely broken?

(I have Cc:-ed other gents as well.)

Thanks,

Ingo

#4 lore raw
Quoted 11 lines · Ingo Molnar
Quoted 8 lines · Peter Zijlstra
Quoted 7 lines · tip-bot for Karol Herbst
Commit-ID:  6d60ce384d1d5ca32b595244db4077a419acc687
Gitweb:     https://git.kernel.org/tip/6d60ce384d1d5ca32b595244db4077a419acc687
Author:     Karol Herbst <[email protected]>
AuthorDate: Mon, 27 Nov 2017 08:51:39 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Mon, 11 Dec 2017 15:35:18 +0100
Quoted 8 lines
Quoted 8 lines
Quoted 1 line

x86/mm/kmmio: Fix mmiotrace for page unaligned addresses

OK, let me hijack this thread since apparently people use and care about mmiotrace.

I was recently auditing the x86 tlb flushing and ran across this 'thing'. Can someone please explain to me how this is supposed to work and how its not completely broken?

The "thing" being mmiotrace, or the "thing" being page-unaligned addresses?

If the former, its primary use-case is for snooping on the NVIDIA proprietary GPU driver in order to figure out how to drive the underlying hardware. The driver does ioremap's to get at PCI space, which mmiotrace "steals" and provides pages without a present bit set, along with a fault handler. When the fault handler is hit, it reinstates the faulting page, and single-steps the faulting instruction reading the before/after regs to determine what happened (doesn't work universally, but enough for instructions used for PCI MMIO accesses). See mmio-mod.c::pre and post (the latter is called from the debug handler).

You may be interested in reading Documentation/trace/mmiotrace.txt::How Mmiotrace Works

Cheers,

-ilia

maintainer
#5 lore raw
Quoted 1 line · Ilia Mirkin

The "thing" being mmiotrace, or the "thing" being page-unaligned addresses?

mmiotrace

Quoted 7 lines

If the former, its primary use-case is for snooping on the NVIDIA proprietary GPU driver in order to figure out how to drive the underlying hardware. The driver does ioremap's to get at PCI space, which mmiotrace "steals" and provides pages without a present bit set, along with a fault handler. When the fault handler is hit, it reinstates the faulting page, and single-steps the faulting instruction

At which point you have valid page-tables and another CPU can access that page too.

Quoted 4 lines

reading the before/after regs to determine what happened (doesn't work universally, but enough for instructions used for PCI MMIO accesses). See mmio-mod.c::pre and post (the latter is called from the debug handler).

And after that you only invalidate the TLBs for the CPU that took the initial fault, leaving possibly stale TLBs on other CPUs.

So this 'thing' has huge gaping SMP holes in.

#6 lore raw
Quoted 21 lines · Peter Zijlstra
Quoted 1 line · Ilia Mirkin

The "thing" being mmiotrace, or the "thing" being page-unaligned addresses?

mmiotrace

Quoted 7 lines

If the former, its primary use-case is for snooping on the NVIDIA proprietary GPU driver in order to figure out how to drive the underlying hardware. The driver does ioremap's to get at PCI space, which mmiotrace "steals" and provides pages without a present bit set, along with a fault handler. When the fault handler is hit, it reinstates the faulting page, and single-steps the faulting instruction

At which point you have valid page-tables and another CPU can access that page too.

Quoted 4 lines

reading the before/after regs to determine what happened (doesn't work universally, but enough for instructions used for PCI MMIO accesses). See mmio-mod.c::pre and post (the latter is called from the debug handler).

Quoted 5 lines

And after that you only invalidate the TLBs for the CPU that took the initial fault, leaving possibly stale TLBs on other CPUs.

So this 'thing' has huge gaping SMP holes in.

Sure does! Probably why the following happens when mmiotrace is enabled:

void enable_mmiotrace(void)

{
        mutex_lock(&mmiotrace_mutex);
        if (is_enabled())
                goto out;

        if (nommiotrace)
                pr_info("MMIO tracing disabled.\n");
        kmmio_init();
        enter_uniprocessor();
        spin_lock_irq(&trace_lock);
        atomic_inc(&mmiotrace_enabled);
        spin_unlock_irq(&trace_lock);
        pr_info("enabled.\n");
out:
        mutex_unlock(&mmiotrace_mutex);
}
maintainer
#7 lore raw
Quoted 5 lines · Ilia Mirkin
Quoted 1 line

So this 'thing' has huge gaping SMP holes in.

Sure does! Probably why the following happens when mmiotrace is enabled:

void enable_mmiotrace(void)

Quoted 9 lines
{
        mutex_lock(&mmiotrace_mutex);
        if (is_enabled())
                goto out;

        if (nommiotrace)
                pr_info("MMIO tracing disabled.\n");
        kmmio_init();
        enter_uniprocessor();
	^^^^^

Ah! I completely missed that. OK, that makes it much less broken :-)

If I don't forget, I'll add some comments to this file to clarify that.

Thanks!

Quoted 7 lines
        spin_lock_irq(&trace_lock);
        atomic_inc(&mmiotrace_enabled);
        spin_unlock_irq(&trace_lock);
        pr_info("enabled.\n");
out:
        mutex_unlock(&mmiotrace_mutex);
}
maintainer
#8 lore raw
Quoted 11 lines · On Tue, 12 Dec 2017 15:51:05 +0100 Peter Zijlstra <[email protected]> wrote:
Quoted 10 lines · Ilia Mirkin
Quoted 1 line

So this 'thing' has huge gaping SMP holes in.

Sure does! Probably why the following happens when mmiotrace is enabled:

void enable_mmiotrace(void)

{
        mutex_lock(&mmiotrace_mutex);
        if (is_enabled())
                goto out;
Quoted 8 lines
Quoted 4 lines
        if (nommiotrace)
                pr_info("MMIO tracing disabled.\n");
        kmmio_init();
        enter_uniprocessor();  
	^^^^^

Ah! I completely missed that. OK, that makes it much less broken :-)

/me just saw this thread.

Once I saw your initial complaint, my first thought was "I think he doesn't know that it causes the system to turn into a uniprocessor first".

My ftrace tests have a test that switches to each tracer, and this mmiotrace catches the most bugs after an rc1 release. The bugs are triggered by switching to and from uniprocessor mode. Hopefully, with the new hotplug code, that will be a thing of the past.

Quoted 2 lines

If I don't forget, I'll add some comments to this file to clarify that.

Great, thanks!

-- Steve

Quoted 1 line
#9 lore raw
In reply to Ingo Molnar

Hi Peter,

the basic idea is to detect if a driver accesses a memory region mapped through ioremap. This is super usefull for reverse engineering closed source drivers like the Nvidia GPU driver.

So here is what it does:
1. on ioremap the entire memory region mapped is registered in the mmiotracer and marked as not presen, which basically leads to page faults on acces
2. mmiotrace is the registered page fault handler for those pages and while handling the page (which basically means marking them as presen, because they were never missing in the first place) it parses the current instruction to detect if it was a read or write and writes relevant information into a file. This includes address accessed, value read/written, type of instruction
3. after single stepping, the page is marked as not present again
4. on unmap time, mmiotrace unregisteres those regions and marks them as present

this is more or less the basic idea.

And to answer your question how it is not completely broken: I don't know. It works for us (more or less, we can't parse repeat instructions as one example what does not work) and if we come across issues we try to fix them on the way.

Anyway, this is a super useful tool to record and debug what a driver is doing with hardware and helps tracking down a lot of this, especially for Nouveau.

I hope that helps.

Quoted 25 lines · Ingo Molnar
Quoted 8 lines · Peter Zijlstra
Quoted 7 lines · tip-bot for Karol Herbst
Commit-ID:  6d60ce384d1d5ca32b595244db4077a419acc687
Gitweb:     https://git.kernel.org/tip/6d60ce384d1d5ca32b595244db4077a419acc687
Author:     Karol Herbst <[email protected]>
AuthorDate: Mon, 27 Nov 2017 08:51:39 +0100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Mon, 11 Dec 2017 15:35:18 +0100
Quoted 8 lines
Quoted 1 line

x86/mm/kmmio: Fix mmiotrace for page unaligned addresses

OK, let me hijack this thread since apparently people use and care about mmiotrace.

I was recently auditing the x86 tlb flushing and ran across this 'thing'. Can someone please explain to me how this is supposed to work and how its not completely broken?

(I have Cc:-ed other gents as well.)

Thanks,

Ingo