Skip to contentSkip to conversation

arch/i386/pci/mmconfig.c tlb flush fix

PATCH In review OGAWA Hirofumi 3 participants 5 messages Sep 26, 2006

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 arch/i386/pci/mmconfig.c tlb flush fix

Hi,

We use the fixmap for accessing pci config space in pci_mmcfg_read/write(). The problem is in pci_exp_set_dev_base(). It is caching a last accessed address to avoid calling set_fixmap_nocache() whenever pci_mmcfg_read/write() is used.

static inline void pci_exp_set_dev_base(int bus, int devfn)

{
	u32 dev_base = base | (bus << 20) | (devfn << 12);
	if (dev_base != mmcfg_last_accessed_device) {
		mmcfg_last_accessed_device = dev_base;
		set_fixmap_nocache(FIX_PCIE_MCFG, dev_base);
	}
}

cpu0 cpu1
---------------------------------------------------------------------------

    pci_mmcfg_read("device-A")
        pci_exp_set_dev_base()
            set_fixmap_nocache()
                                              pci_mmcfg_read("device-B")
                                                  pci_exp_set_dev_base()
                                                      set_fixmap_nocache()
    pci_mmcfg_read("device-B")
        pci_exp_set_dev_base()
            /* doesn't flush tlb */

But if cpus accessed the above order, the second pci_mmcfg_read() on cpu0 doesn't flush the TLB, because "mmcfg_last_accessed_device" is device-B. So, second pci_mmcfg_read() on cpu0 accesses a device-A via a previous TLB cache.

This patches fixes this situation by adds "mmcfg_last_accessed_cpu" check.

Please apply.

---

 arch/i386/pci/mmconfig.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/i386/pci/mmconfig.c renamed from arch/i386/pci/mmconfig.c~mmconfig-tlb-race-fix +5 -1
diff -puN arch/i386/pci/mmconfig.c~mmconfig-tlb-race-fix arch/i386/pci/mmconfig.c
@@ -26,6 +26,7 @@
 
 /* The base address of the last MMCONFIG device accessed */
 static u32 mmcfg_last_accessed_device;
+static int mmcfg_last_accessed_cpu;
 
 static DECLARE_BITMAP(fallback_slots, MAX_CHECK_BUS*32);
 
@@ -70,8 +71,11 @@ static u32 get_base_addr(unsigned int se
 static inline void pci_exp_set_dev_base(unsigned int base, int bus, int devfn)
 {
 	u32 dev_base = base | (bus << 20) | (devfn << 12);
-	if (dev_base != mmcfg_last_accessed_device) {
+	int cpu = smp_processor_id();
+	if (dev_base != mmcfg_last_accessed_device ||
+	    cpu != mmcfg_last_accessed_cpu) {
 		mmcfg_last_accessed_device = dev_base;
+		mmcfg_last_accessed_cpu = cpu;
 		set_fixmap_nocache(FIX_PCIE_MCFG, dev_base);
 	}
 }

_

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#1 lore raw
Quoted 6 lines · OGAWA Hirofumi

Hi,

We use the fixmap for accessing pci config space in pci_mmcfg_read/write(). The problem is in pci_exp_set_dev_base(). It is caching a last accessed address to avoid calling set_fixmap_nocache() whenever pci_mmcfg_read/write() is used.

Good catch. I already had another report of mmconfig corruption on i386, but didn't have time to look at it yet.

Will be definitely stable material once it hit mainline.

-Andi

#2 lore raw
Quoted 8 lines · Andi Kleen
Quoted 5 lines · OGAWA Hirofumi

We use the fixmap for accessing pci config space in pci_mmcfg_read/write(). The problem is in pci_exp_set_dev_base(). It is caching a last accessed address to avoid calling set_fixmap_nocache() whenever pci_mmcfg_read/write() is used.

Quoted 4 lines

Good catch. I already had another report of mmconfig corruption on i386, but didn't have time to look at it yet.

Will be definitely stable material once it hit mainline.

Indeed.

If I found this in mainline, should I send something like a reminder email with patch to you, Greg?

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#3 lore raw
Quoted 13 lines · OGAWA Hirofumi
Quoted 11 lines · Andi Kleen
Quoted 5 lines · OGAWA Hirofumi

We use the fixmap for accessing pci config space in pci_mmcfg_read/write(). The problem is in pci_exp_set_dev_base(). It is caching a last accessed address to avoid calling set_fixmap_nocache() whenever pci_mmcfg_read/write() is used.

Good catch. I already had another report of mmconfig corruption on i386, but didn't have time to look at it yet.

Quoted 6 lines
Quoted 1 line

Will be definitely stable material once it hit mainline.

Indeed.

If I found this in mainline, should I send something like a reminder email with patch to you, Greg?

I have it already queued for both.

-Andi

#4 lore raw
Quoted 14 lines · Andi Kleen
Quoted 13 lines · OGAWA Hirofumi
Quoted 11 lines · Andi Kleen
Quoted 5 lines · OGAWA Hirofumi

We use the fixmap for accessing pci config space in pci_mmcfg_read/write(). The problem is in pci_exp_set_dev_base(). It is caching a last accessed address to avoid calling set_fixmap_nocache() whenever pci_mmcfg_read/write() is used.

Good catch. I already had another report of mmconfig corruption on i386, but didn't have time to look at it yet.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Will be definitely stable material once it hit mainline.

Indeed.

If I found this in mainline, should I send something like a reminder email with patch to you, Greg?

I have it already queued for both.

Oh, thanks.

Signature · 1 line
OGAWA Hirofumi <[email protected]>