Skip to contentSkip to conversation

xen/SandyBridge: reserve pages when integrated graphics

PATCH Superseded Xudong Hao 3 participants 11 messages Mar 22, 2013 – Mar 26, 2013

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Xudong Hao

author
#0 lore raw

PATCH 1/1 xen/SandyBridge: reserve pages when integrated graphics

SNB graphics devices have a bug that prevent them from accessing certain memory ranges, namely anything below 1M and in the pages listed in the table.

Xen does not initialize below 1MB to heap, i.e. below 1MB pages don't be allocated, so it's unnecessary to reserve memory below the 1 MB mark that has not already been reserved.

So reserve those pages listed in the table at xen boot if set detect a SNB gfx device on the CPU to avoid GPU hangs.

---
 xen/arch/x86/mm.c                    | 14 ++++++++++++
 xen/common/page_alloc.c              | 42 +++++++++++++++++++++++++-----------
 xen/drivers/passthrough/vtd/quirks.c |  2 +-
 xen/include/asm-x86/mm.h             |  1 +
 xen/include/asm-x86/pci.h            |  5 +++++
 5 files changed, 50 insertions(+), 14 deletions(-)
xen/arch/x86/mm.c +14 -0
index add93ac..5e706af 100644
@@ -121,6 +121,7 @@
 #include <xen/trace.h>
 #include <asm/setup.h>
 #include <asm/fixmap.h>
+#include <asm/pci.h>
 
 /* Mapping of the fixmap space needed early. */
 l1_pgentry_t __attribute__ ((__section__ (".bss.page_aligned")))
@@ -5597,6 +5598,19 @@ void arch_dump_shared_mem_info(void)
             mem_sharing_get_nr_saved_mfns());
 }
 
+char *__init get_platform_badpages(void)
+{
+    u32 igd_id;
+    static char __initdata bad_pfns[] = 
+                {"0x20050,0x20110,0x20130,0x20138,0x40004"};
+
+    igd_id = pci_conf_read32(0, 0, 2, 0, 0);
+    if (!IS_SNB_GFX(igd_id))
+        return NULL;
+
+    return bad_pfns;
+}
+
 /*
  * Local variables:
  * mode: C
xen/common/page_alloc.c +29 -13

57 lines not on this page · raw message

xen/drivers/passthrough/vtd/quirks.c +1 -1
index d79a155..70d57de 100644
@@ -31,6 +31,7 @@
 #include <xen/keyhandler.h>
 #include <asm/msi.h>
 #include <asm/irq.h>
+#include <asm/pci.h>
 #include <mach_apic.h>
 #include "iommu.h"
 #include "dmar.h"
@@ -47,7 +48,6 @@
 #define IS_CTG(id)    (id == 0x2a408086)
 #define IS_ILK(id)    (id == 0x00408086 || id == 0x00448086 || id== 0x00628086 || id == 0x006A8086)
 #define IS_CPT(id)    (id == 0x01008086 || id == 0x01048086)
-#define IS_SNB_GFX(id) (id == 0x01068086 || id == 0x01168086 || id == 0x01268086 || id == 0x01028086 || id == 0x01128086 || id == 0x01228086 || id == 0x010A8086)
 
 static u32 __read_mostly ioh_id;
 static u32 __initdata igd_id;
xen/include/asm-x86/mm.h +1 -0
index 4f89dae..7af069f 100644
@@ -322,6 +322,7 @@ int is_iomem_page(unsigned long mfn);
 
 void clear_superpage_mark(struct page_info *page);
 
+char *get_platform_badpages(void);
 /* Per page locks:
  * page_lock() is used for two purposes: pte serialization, and memory sharing.
  *
xen/include/asm-x86/pci.h +5 -0
index 7bcb702..e0598fd 100644
@@ -1,6 +1,11 @@
 #ifndef __X86_PCI_H__
 #define __X86_PCI_H__
 
+#define IS_SNB_GFX(id) (id == 0x01068086 || id == 0x01168086 \
+                        || id == 0x01268086 || id == 0x01028086 \
+                        || id == 0x01128086 || id == 0x01228086 \
+                        || id == 0x010A8086 )
+
 struct arch_pci_dev {
     vmask_t used_vectors;
 };
Signature · 1 line
1.7.12.1
#1 lore raw

xen/arch/x86/mm.c void arch_dump_shared_mem_info(void) · line 5605

Quoted 6 lines
Quoted 1 line
Quoted 1 line · Xudong Hao
xen/arch/x86/mm.c +5 -0
+char *__init get_platform_badpages(void)
+{
+    u32 igd_id;
+    static char __initdata bad_pfns[] = 
+                {"0x20050,0x20110,0x20130,0x20138,0x40004"};

With quite a bit of trouble I managed to find all applicable spec updates, but none of them mentions 0x20110.

xen/common/page_alloc.c void __init init_boot_pages(paddr_t ps, paddr_t pe) · line 200

Quoted 11 lines
xen/common/page_alloc.c +11 -0
+#ifdef CONFIG_X86
+    /* 
+     * Here we put platform-specific memory range workarounds, i.e.
+     * memory known to be corrupt or otherwise in need to be reserved on
+     * specific platforms.
+     * We get these certain pages and put them in bad-page list.
+     */
+    p = get_platform_badpages();
+    if ( p )
+        remove_bad_pages(p);
+#endif

I also dislike the re-use of the command line parsing code here. There's no need to do this for a list of known MFNs, the hook could provide an array of unsigned long instead.

And finally it is my understanding that these pages aren't bad in any way, it must only be ensured they don't get passed to the graphics engine. Which in particular means using these pages for Xen's own purposes (i.e. not passing them to any domain) would be quite fine. Yes, I realize it's only a handful of pages, but it seems odd anyway.

Jan

#2 lore raw
Quoted 8 lines

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Friday, March 22, 2013 6:27 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] xen/SandyBridge: reserve pages when integrated graphics

Quoted 1 line
Quoted 1 line
Quoted 1 line · Xudong Hao
Quoted 9 lines

xen/arch/x86/mm.c void arch_dump_shared_mem_info(void) · line 5605

Quoted 5 lines
xen/arch/x86/mm.c +5 -0
+char *__init get_platform_badpages(void)
+{
+    u32 igd_id;
+    static char __initdata bad_pfns[] =
+                {"0x20050,0x20110,0x20130,0x20138,0x40004"};

With quite a bit of trouble I managed to find all applicable spec updates, but none of them mentions 0x20110.

I did not get detail spec either, but linux kernel has remove this page 0x20110000 already for the same reason, refer to commit: a9acc5365dbda29f7be2884efb63771dc24bd815

Quoted 7 lines
Quoted 7 lines
xen/common/page_alloc.c +7 -0
+#ifdef CONFIG_X86
+    /*
+     * Here we put platform-specific memory range workarounds, i.e.
+     * memory known to be corrupt or otherwise in need to be reserved on
+     * specific platforms.
+     * We get these certain pages and put them in bad-page list.
+     */
Quoted 9 lines

xen/common/page_alloc.c void __init init_boot_pages(paddr_t ps, paddr_t pe) · line 200

Quoted 4 lines
xen/common/page_alloc.c +4 -0
+    p = get_platform_badpages();
+    if ( p )
+        remove_bad_pages(p);
+#endif

I also dislike the re-use of the command line parsing code here. There's no need to do this for a list of known MFNs, the hook could provide an array of unsigned long instead.

Right. bootmem_region_zap() can do this directly.

Quoted 7 lines

And finally it is my understanding that these pages aren't bad in any way, it must only be ensured they don't get passed to the graphics engine. Which in particular means using these pages for Xen's own purposes (i.e. not passing them to any domain) would be quite fine. Yes, I realize it's only a handful of pages, but it seems odd anyway.

We all know these pages can be used for Xen, but not for domains. But as you said, it's only 5 pages, the simplest solution is removing them in xen starting, so that they don't be allocate to any domains.

-Thanks
Xudong

#3 lore raw
Quoted 38 lines
Quoted 9 lines
Quoted 1 line · Xudong

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Friday, March 22, 2013 6:27 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] xen/SandyBridge: reserve pages when integrated graphics

Quoted 1 line
Quoted 1 line
Quoted 1 line · Xudong Hao
Quoted 9 lines

xen/arch/x86/mm.c void arch_dump_shared_mem_info(void) · line 5605

Quoted 5 lines
xen/arch/x86/mm.c +5 -0
+char *__init get_platform_badpages(void)
+{
+    u32 igd_id;
+    static char __initdata bad_pfns[] =
+                {"0x20050,0x20110,0x20130,0x20138,0x40004"};

With quite a bit of trouble I managed to find all applicable spec updates, but none of them mentions 0x20110.

I did not get detail spec either, but linux kernel has remove this page 0x20110000 already for the same reason, refer to commit:
a9acc5365dbda29f7be2884efb63771dc24bd815

Quoted 7 lines
Quoted 7 lines
xen/common/page_alloc.c +7 -0
+#ifdef CONFIG_X86
+    /*
+     * Here we put platform-specific memory range workarounds, i.e.
+     * memory known to be corrupt or otherwise in need to be reserved on
+     * specific platforms.
+     * We get these certain pages and put them in bad-page list.
+     */
Quoted 9 lines

xen/common/page_alloc.c void __init init_boot_pages(paddr_t ps, paddr_t pe) · line 200

Quoted 4 lines
xen/common/page_alloc.c +4 -0
+    p = get_platform_badpages();
+    if ( p )
+        remove_bad_pages(p);
+#endif

I also dislike the re-use of the command line parsing code here. There's no need to do this for a list of known MFNs, the hook could provide an array of unsigned long instead.

Quoted 1 line

Right. bootmem_region_zap() can do this directly.

So I supposed we're going to see a v2 then, with both points addressed?

Jan

#4 lore raw
Quoted 43 lines

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, March 25, 2013 5:28 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: RE: [PATCH] xen/SandyBridge: reserve pages when integrated graphics

Quoted 7 lines
Quoted 7 lines
Quoted 1 line · Xudong

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Friday, March 22, 2013 6:27 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] xen/SandyBridge: reserve pages when integrated

graphics

Quoted 11 lines
Quoted 11 lines

xen/arch/x86/mm.c void arch_dump_shared_mem_info(void) · line 5605

Quoted 7 lines
Quoted 2 lines
Quoted 2 lines · Xudong Hao
xen/arch/x86/mm.c +5 -0
+char *__init get_platform_badpages(void)
+{
+    u32 igd_id;
+    static char __initdata bad_pfns[] =
+                {"0x20050,0x20110,0x20130,0x20138,0x40004"};

With quite a bit of trouble I managed to find all applicable spec updates, but none of them mentions 0x20110.

Quoted 8 lines

I did not get detail spec either, but linux kernel has remove this page 0x20110000 already for the same reason, refer to commit:
a9acc5365dbda29f7be2884efb63771dc24bd815

Quoted 4 lines
Quoted 4 lines
xen/common/page_alloc.c +4 -0
+#ifdef CONFIG_X86
+    /*
+     * Here we put platform-specific memory range workarounds, i.e.
+     * memory known to be corrupt or otherwise in need to be reserved

on

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
xen/common/page_alloc.c +7 -0
+     * specific platforms.
+     * We get these certain pages and put them in bad-page list.
+     */
+    p = get_platform_badpages();
+    if ( p )
+        remove_bad_pages(p);
+#endif
Quoted 9 lines
Quoted 5 lines
Quoted 4 lines

I also dislike the re-use of the command line parsing code here. There's no need to do this for a list of known MFNs, the hook could provide an array of unsigned long instead.

Right. bootmem_region_zap() can do this directly.

So I supposed we're going to see a v2 then, with both points addressed?

Surely, but v2 will modify only one point. I don't know what's the 2nd point you mean?

#5 lore raw
Quoted 48 lines
Quoted 48 lines
Quoted 1 line · Xudong

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, March 25, 2013 5:28 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: RE: [PATCH] xen/SandyBridge: reserve pages when integrated graphics

Quoted 7 lines
Quoted 7 lines
Quoted 1 line · Xudong

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Friday, March 22, 2013 6:27 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] xen/SandyBridge: reserve pages when integrated

graphics

Quoted 11 lines
Quoted 11 lines

xen/arch/x86/mm.c void arch_dump_shared_mem_info(void) · line 5605

Quoted 7 lines
Quoted 2 lines
Quoted 2 lines · Xudong Hao
xen/arch/x86/mm.c +5 -0
+char *__init get_platform_badpages(void)
+{
+    u32 igd_id;
+    static char __initdata bad_pfns[] =
+                {"0x20050,0x20110,0x20130,0x20138,0x40004"};

With quite a bit of trouble I managed to find all applicable spec updates, but none of them mentions 0x20110.

Quoted 8 lines

I did not get detail spec either, but linux kernel has remove this page 0x20110000 already for the same reason, refer to commit:
a9acc5365dbda29f7be2884efb63771dc24bd815

Quoted 4 lines
Quoted 4 lines
xen/common/page_alloc.c +4 -0
+#ifdef CONFIG_X86
+    /*
+     * Here we put platform-specific memory range workarounds, i.e.
+     * memory known to be corrupt or otherwise in need to be reserved

on

Quoted 12 lines
Quoted 12 lines

xen/common/page_alloc.c void __init init_boot_pages(paddr_t ps, paddr_t pe) · line 200

Quoted 7 lines
xen/common/page_alloc.c +7 -0
+     * specific platforms.
+     * We get these certain pages and put them in bad-page list.
+     */
+    p = get_platform_badpages();
+    if ( p )
+        remove_bad_pages(p);
+#endif

I also dislike the re-use of the command line parsing code here. There's no need to do this for a list of known MFNs, the hook could provide an array of unsigned long instead.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Right. bootmem_region_zap() can do this directly.

So I supposed we're going to see a v2 then, with both points addressed?

Surely, but v2 will modify only one point. I don't know what's the 2nd point you mean?

- removal of page 0x20110
- using MFNs directly rather than passing a string to parse

Jan

#6 lore raw
Quoted 60 lines

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, March 25, 2013 5:54 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: RE: [PATCH] xen/SandyBridge: reserve pages when integrated graphics

Quoted 7 lines
Quoted 7 lines
Quoted 1 line · Xudong

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, March 25, 2013 5:28 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: RE: [PATCH] xen/SandyBridge: reserve pages when integrated

graphics

Quoted 20 lines
Quoted 20 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines · Xudong

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Friday, March 22, 2013 6:27 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] xen/SandyBridge: reserve pages when integrated

graphics

Quoted 11 lines
Quoted 11 lines

xen/arch/x86/mm.c void arch_dump_shared_mem_info(void) · line 5605

Quoted 7 lines
Quoted 2 lines
Quoted 2 lines · Xudong Hao
xen/arch/x86/mm.c +5 -0
+char *__init get_platform_badpages(void)
+{
+    u32 igd_id;
+    static char __initdata bad_pfns[] =
+                {"0x20050,0x20110,0x20130,0x20138,0x40004"};

With quite a bit of trouble I managed to find all applicable spec updates, but none of them mentions 0x20110.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

I did not get detail spec either, but linux kernel has remove this page 0x20110000 already for the same reason, refer to commit:
a9acc5365dbda29f7be2884efb63771dc24bd815

Quoted 3 lines
Quoted 3 lines
xen/common/page_alloc.c +3 -0
+#ifdef CONFIG_X86
+    /*
+     * Here we put platform-specific memory range workarounds,

i.e.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * memory known to be corrupt or otherwise in need to be

reserved

Quoted 15 lines
Quoted 15 lines

on

Quoted 13 lines
Quoted 8 lines

xen/common/page_alloc.c void __init init_boot_pages(paddr_t ps, paddr_t pe) · line 200

Quoted 7 lines
xen/common/page_alloc.c +7 -0
+     * specific platforms.
+     * We get these certain pages and put them in bad-page list.
+     */
+    p = get_platform_badpages();
+    if ( p )
+        remove_bad_pages(p);
+#endif
Quoted 4 lines

I also dislike the re-use of the command line parsing code here. There's no need to do this for a list of known MFNs, the hook could provide an array of unsigned long instead.

Right. bootmem_region_zap() can do this directly.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

So I supposed we're going to see a v2 then, with both points addressed?

Surely, but v2 will modify only one point. I don't know what's the 2nd point you mean?

- removal of page 0x20110

Linux kernel has put page 0x20110000 as bad, so we do not need to remove it from bad page list. Linux commit: a9acc5365dbda29f7be2884efb63771dc24bd815

#7 lore raw
Quoted 7 lines
Quoted 3 lines
Quoted 1 line · Xudong

From: Jan Beulich [mailto:[email protected]]
- removal of page 0x20110

Linux kernel has put page 0x20110000 as bad, so we do not need to remove it from bad page list.
Linux commit: a9acc5365dbda29f7be2884efb63771dc24bd815

So then your earlier response was misleading, or at least confusing me:

"I did not get detail spec either, but linux kernel has remove this page
0x20110000 already for the same reason, refer to commit:
a9acc5365dbda29f7be2884efb63771dc24bd815".

"Removed" to me meant removed from the list of bad PFNs. Note that I didn't look at the commit, I just took your response as confirming that this particular PFN got added to the list in error. Now that you say that you actually meant the exact opposite, we're back at the question why that PFN is in the list in the first place, considering that none of the spec updates say so.

Jan

#8 lore raw
Quoted 18 lines

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, March 25, 2013 9:07 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: RE: [PATCH] xen/SandyBridge: reserve pages when integrated graphics

Quoted 7 lines
Quoted 3 lines
Quoted 1 line · Xudong

From: Jan Beulich [mailto:[email protected]]
- removal of page 0x20110

Linux kernel has put page 0x20110000 as bad, so we do not need to remove it from bad page list.
Linux commit: a9acc5365dbda29f7be2884efb63771dc24bd815

So then your earlier response was misleading, or at least confusing me:

Quoted 7 lines

"I did not get detail spec either, but linux kernel has remove this page
0x20110000 already for the same reason, refer to commit:
a9acc5365dbda29f7be2884efb63771dc24bd815".

"Removed" to me meant removed from the list of bad PFNs. Note that I didn't look at the commit, I just took your response as confirming that this particular PFN got added to the list in error.

Sorry to confuse you, the earlier response really a little misleading. Actually I want to say this page is inaccessible for SandyBridge graphic device.

Quoted 4 lines

Now that you say that you actually meant the exact opposite, we're back at the question why that PFN is in the list in the first place, considering that none of the spec updates say so.

Let me try to find out the description on spec.

#9 lore raw

Include page 0x20110000, the five pages are in spec and I have confirmed.

Quoted 43 lines

-----Original Message-----
From: Hao, Xudong
Sent: Monday, March 25, 2013 9:37 PM
To: Jan Beulich
Cc: [email protected]; [email protected]
Subject: RE: [PATCH] xen/SandyBridge: reserve pages when integrated graphics

Quoted 6 lines

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, March 25, 2013 9:07 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: RE: [PATCH] xen/SandyBridge: reserve pages when integrated

graphics

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines · Xudong

From: Jan Beulich [mailto:[email protected]]
- removal of page 0x20110

Linux kernel has put page 0x20110000 as bad, so we do not need to remove

it

Quoted 6 lines
Quoted 2 lines

from bad page list.
Linux commit: a9acc5365dbda29f7be2884efb63771dc24bd815

So then your earlier response was misleading, or at least confusing me:

Quoted 7 lines

"I did not get detail spec either, but linux kernel has remove this page
0x20110000 already for the same reason, refer to commit:
a9acc5365dbda29f7be2884efb63771dc24bd815".

"Removed" to me meant removed from the list of bad PFNs. Note that I didn't look at the commit, I just took your response as confirming that this particular PFN got added to the list in error.

Sorry to confuse you, the earlier response really a little misleading. Actually I want to say this page is inaccessible for SandyBridge graphic device.

Quoted 4 lines

Now that you say that you actually meant the exact opposite, we're back at the question why that PFN is in the list in the first place, considering that none of the spec updates say so.

Let me try to find out the description on spec.

#10 lore raw
Quoted 1 line · Xudong

Include page 0x20110000, the five pages are in spec and I have confirmed.

Could you also include the name of the spec and an URL (if there is one) for it. That way in the future if there are some lingering questions one can just reference the spec.

Quoted 51 lines
Quoted 36 lines

-----Original Message-----
From: Hao, Xudong
Sent: Monday, March 25, 2013 9:37 PM
To: Jan Beulich
Cc: [email protected]; [email protected]
Subject: RE: [PATCH] xen/SandyBridge: reserve pages when integrated graphics

Quoted 6 lines

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, March 25, 2013 9:07 PM
To: Hao, Xudong
Cc: [email protected]; [email protected]
Subject: RE: [PATCH] xen/SandyBridge: reserve pages when integrated

graphics

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines · Xudong

From: Jan Beulich [mailto:[email protected]]
- removal of page 0x20110

Linux kernel has put page 0x20110000 as bad, so we do not need to remove

it

Quoted 6 lines
Quoted 2 lines

from bad page list.
Linux commit: a9acc5365dbda29f7be2884efb63771dc24bd815

So then your earlier response was misleading, or at least confusing me:

Quoted 7 lines

"I did not get detail spec either, but linux kernel has remove this page
0x20110000 already for the same reason, refer to commit:
a9acc5365dbda29f7be2884efb63771dc24bd815".

"Removed" to me meant removed from the list of bad PFNs. Note that I didn't look at the commit, I just took your response as confirming that this particular PFN got added to the list in error.

Quoted 8 lines

Sorry to confuse you, the earlier response really a little misleading. Actually I want to say this page is inaccessible for SandyBridge graphic device.

Quoted 4 lines

Now that you say that you actually meant the exact opposite, we're back at the question why that PFN is in the list in the first place, considering that none of the spec updates say so.

Let me try to find out the description on spec.

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel