Skip to contentSkip to conversation

Make __FIXADDR_TOP variable to allow it to make space for a hypervisor

PATCH Chris Wright 3 participants 7 messages Aug 1, 2006 – Aug 2, 2006 2 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw

Review

Quoted 2 lines · * Jeremy Fitzhardinge ([email protected]) wrote:
(quoted diff) +1 -1
-#define MAXMEM			(-__PAGE_OFFSET-__VMALLOC_RESERVE)
+#define MAXMEM			(__FIXADDR_TOP-__PAGE_OFFSET-__VMALLOC_RESERVE)

In the native case we lose one page of lowmem now.

thanks,
-chris

#3 lore raw
Quoted 5 lines · On Tue, 1 Aug 2006 02:03:30 -0700 Chris Wright <[email protected]> wrote:

Review

Quoted 2 lines · * Jeremy Fitzhardinge ([email protected]) wrote:
(quoted diff) +1 -1
-#define MAXMEM			(-__PAGE_OFFSET-__VMALLOC_RESERVE)
+#define MAXMEM			(__FIXADDR_TOP-__PAGE_OFFSET-__VMALLOC_RESERVE)

In the native case we lose one page of lowmem now.

erm, isn't this the hunk which gave one of my machines a 4kb highmem zone? I have memories of reverting it.

#4 lore raw
Quoted 11 lines · * Andrew Morton ([email protected]) wrote:
Quoted 5 lines · On Tue, 1 Aug 2006 02:03:30 -0700 Chris Wright <[email protected]> wrote:

Review

Quoted 2 lines · * Jeremy Fitzhardinge ([email protected]) wrote:
(quoted diff) +1 -1
-#define MAXMEM			(-__PAGE_OFFSET-__VMALLOC_RESERVE)
+#define MAXMEM			(__FIXADDR_TOP-__PAGE_OFFSET-__VMALLOC_RESERVE)

In the native case we lose one page of lowmem now.

erm, isn't this the hunk which gave one of my machines a 4kb highmem zone? I have memories of reverting it.

Yes, that does sound quite familiar. I couldn't find the thread, do you recall any of the details? I expect it's the same issue as the off by one page I mentioned above.

thanks,
-chris

#5 lore raw
Quoted 10 lines · Chris Wright
Quoted 9 lines · * Andrew Morton ([email protected]) wrote:
Quoted 5 lines · On Tue, 1 Aug 2006 02:03:30 -0700 Chris Wright <[email protected]> wrote:

Review

Quoted 2 lines · * Jeremy Fitzhardinge ([email protected]) wrote:
(quoted diff) +1 -1
-#define MAXMEM			(-__PAGE_OFFSET-__VMALLOC_RESERVE)
+#define MAXMEM			(__FIXADDR_TOP-__PAGE_OFFSET-__VMALLOC_RESERVE)

In the native case we lose one page of lowmem now.

Quoted 6 lines
Quoted 2 lines

erm, isn't this the hunk which gave one of my machines a 4kb highmem zone? I have memories of reverting it.

Yes, that does sound quite familiar. I couldn't find the thread, do you recall any of the details? I expect it's the same issue as the off by one page I mentioned above.

Good catch Andrew. I did say we'd fix this. Will frob this patch further...

Here's my final reply from my archives:

Subject:
Re: sparsemem panic in 2.6.17-rc5-mm1 and -mm2
Date:
Wed, 07 Jun 2006 16:49:12 +1000

Quoted 6 lines · Andrew Morton
Quoted 1 line · On Wed, 07 Jun 2006 15:36:25 +1000 Rusty Russell <[email protected]> wrote:

You now have 1 page more memory available in your system.

The kernel has differing opinions about that:

Quoted 9 lines

BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 0000000038000000 (usable)
BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
0MB HIGHMEM available.
896MB LOWMEM available.

Sure, the pages are reserved, but we still map them into the kernel address space.

Quoted 8 lines

Review

Quoted 5 lines

I'm sure Gerd will slap me if I'm wrong on this. Here's the patch fragment:

(quoted diff) +1 -1
-#define MAXMEM                 (-__PAGE_OFFSET-__VMALLOC_RESERVE)
+#define MAXMEM                 (__FIXADDR_TOP-__PAGE_OFFSET-__VMALLOC_RESERVE)

Well. Applying this with `patch -R' would presumably restore the situation. But not having a clue why this change was made, I didn't bother trying it.

Actually, the comment above __VMALLOC_RESERVE already says "This much address space is reserved for vmalloc() and iomap() as well as fixmap mappings.", so it should have already been taken into account there.

So, please revert this. When we introduce an actual CONFIG_MEMORY hole, we'll patch in an explicit "-MEMHOLE_SIZE" or something here.

Quoted 3 lines

From what you're saying, it appears that this patch is an unrelated change, to fix the off-by-one? And that if this machine had anything other than exactly 7*128MB of physical memory, I wouldn't have noticed?

You wouldn't have noticed, yes. But I'm not convinced the "fix" was right anyway.

Thanks for chasing this!
Rusty.

Signature · 1 line
Help! Save Australia from the worst of the DMCA: http://linux.org.au/law
#6 lore raw
Quoted 14 lines · * Rusty Russell ([email protected]) wrote:
Quoted 13 lines · Chris Wright
Quoted 11 lines · * Andrew Morton ([email protected]) wrote:
Quoted 5 lines · On Tue, 1 Aug 2006 02:03:30 -0700 Chris Wright <[email protected]> wrote:

Review

Quoted 2 lines · * Jeremy Fitzhardinge ([email protected]) wrote:
(quoted diff) +1 -1
-#define MAXMEM			(-__PAGE_OFFSET-__VMALLOC_RESERVE)
+#define MAXMEM			(__FIXADDR_TOP-__PAGE_OFFSET-__VMALLOC_RESERVE)

In the native case we lose one page of lowmem now.

erm, isn't this the hunk which gave one of my machines a 4kb highmem zone? I have memories of reverting it.

Quoted 6 lines
Quoted 3 lines

Yes, that does sound quite familiar. I couldn't find the thread, do you recall any of the details? I expect it's the same issue as the off by one page I mentioned above.

Good catch Andrew. I did say we'd fix this. Will frob this patch further...

Here's an updated patch. Rather than use __FIXADDR_TOP to adjust for MAXMEM, directly update __VMALLOC_RESERVE which is used to reserve the space for vmalloc, iomap, and fixmap (as comments aptly point out). I tested this with a bunch of configurations, and booted a XenoLinux kernel with this patch as well.

thanks,
-chris
--

Make __FIXADDR_TOP variable to allow it to make space for a hypervisor.

Make __FIXADDR_TOP a variable, so that it can be set to not get in the way of address space a hypervisor may want to reserve.

Original patch by Gerd Hoffmann <[email protected]>

---
 arch/i386/Kconfig         |    1 +
 arch/i386/mm/init.c       |   42 ++++++++++++++++++++++++++++++++++++++++++
 arch/i386/mm/pgtable.c    |   19 +++++++++++++++++++
 include/asm-i386/fixmap.h |    7 ++++++-
 4 files changed, 68 insertions(+), 1 deletion(-)
arch/i386/Kconfig +1 -0
diff -r 5183f1f33cf4 arch/i386/Kconfig
@@ -792,6 +792,7 @@ config COMPAT_VDSO
 config COMPAT_VDSO
 	bool "Compat VDSO support"
 	default y
+	depends on !PARAVIRT
 	help
 	  Map the VDSO to the predictable old-style address too.
 	---help---
arch/i386/mm/init.c +42 -0

48 lines not on this page · raw message

arch/i386/mm/pgtable.c +19 -0
diff -r 5183f1f33cf4 arch/i386/mm/pgtable.c
@@ -12,6 +12,7 @@
 #include <linux/slab.h>
 #include <linux/pagemap.h>
 #include <linux/spinlock.h>
+#include <linux/module.h>
 
 #include <asm/system.h>
 #include <asm/pgtable.h>
@@ -137,6 +138,12 @@ void set_pmd_pfn(unsigned long vaddr, un
 	__flush_tlb_one(vaddr);
 }
 
+static int fixmaps;
+#ifndef CONFIG_COMPAT_VDSO
+unsigned long __FIXADDR_TOP = 0xfffff000;
+EXPORT_SYMBOL(__FIXADDR_TOP);
+#endif
+
 void __set_fixmap (enum fixed_addresses idx, unsigned long phys, pgprot_t flags)
 {
 	unsigned long address = __fix_to_virt(idx);
@@ -146,6 +153,18 @@ void __set_fixmap (enum fixed_addresses 
 		return;
 	}
 	set_pte_pfn(address, phys >> PAGE_SHIFT, flags);
+	fixmaps++;
+}
+
+void set_fixaddr_top(unsigned long top)
+{
+	BUG_ON(fixmaps > 0);
+#ifdef CONFIG_COMPAT_VDSO
+	BUG_ON(top - PAGE_SIZE != __FIXADDR_TOP);
+#else
+	__FIXADDR_TOP = top - PAGE_SIZE;
+	__VMALLOC_RESERVE -= top;
+#endif
 }
 
 pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
include/asm-i386/fixmap.h +6 -1
diff -r 5183f1f33cf4 include/asm-i386/fixmap.h
@@ -19,7 +19,11 @@
  * Leave one empty page between vmalloc'ed areas and
  * the start of the fixmap.
  */
-#define __FIXADDR_TOP	0xfffff000
+#ifndef CONFIG_COMPAT_VDSO
+extern unsigned long __FIXADDR_TOP;
+#else
+#define __FIXADDR_TOP  0xfffff000
+#endif
 
 #ifndef __ASSEMBLY__
 #include <linux/kernel.h>
@@ -93,6 +97,7 @@ enum fixed_addresses {
 
 extern void __set_fixmap (enum fixed_addresses idx,
 					unsigned long phys, pgprot_t flags);
+extern void set_fixaddr_top(unsigned long top);
 
 #define set_fixmap(idx, phys) \
 		__set_fixmap(idx, phys, PAGE_KERNEL)
#7 lore raw
Quoted 5 lines · Chris Wright

Here's an updated patch. Rather than use __FIXADDR_TOP to adjust for MAXMEM, directly update __VMALLOC_RESERVE which is used to reserve the space for vmalloc, iomap, and fixmap (as comments aptly point out). I tested this with a bunch of configurations, and booted a XenoLinux kernel with this patch as well.

Just one minor point:

arch/i386/mm/pgtable.c void __set_fixmap (enum fixed_addresses · line 168

Quoted 10 lines
arch/i386/mm/pgtable.c +9 -0
+void set_fixaddr_top(unsigned long top)
+{
+	BUG_ON(fixmaps > 0);
+#ifdef CONFIG_COMPAT_VDSO
+	BUG_ON(top - PAGE_SIZE != __FIXADDR_TOP);
+#else
+	__FIXADDR_TOP = top - PAGE_SIZE;
+	__VMALLOC_RESERVE -= top;
+#endif
 }

This no longer seems to be an appropriate name. How about set_address_top_reserve or something?

void set_address_top_reserve(unsigned long reserve)

{
	BUG_ON(fixmaps > 0);
#ifdef CONFIG_COMPAT_VDSO
	BUG_ON(reserve != 0);
#else
	__FIXADDR_TOP = -reserve - PAGE_SIZE;
	__VMALLOC_RESERVE += reserve;

#endif
}

(I *think* I got the logic here correct).

Rusty.

Signature · 1 line
Help! Save Australia from the worst of the DMCA: http://linux.org.au/law
#8 lore raw
Quoted 9 lines · * Rusty Russell ([email protected]) wrote:
Quoted 5 lines · Chris Wright

Here's an updated patch. Rather than use __FIXADDR_TOP to adjust for MAXMEM, directly update __VMALLOC_RESERVE which is used to reserve the space for vmalloc, iomap, and fixmap (as comments aptly point out). I tested this with a bunch of configurations, and booted a XenoLinux kernel with this patch as well.

Just one minor point:

Quoted 13 lines

Review

Quoted 10 lines
(quoted diff) +9 -0
+void set_fixaddr_top(unsigned long top)
+{
+	BUG_ON(fixmaps > 0);
+#ifdef CONFIG_COMPAT_VDSO
+	BUG_ON(top - PAGE_SIZE != __FIXADDR_TOP);
+#else
+	__FIXADDR_TOP = top - PAGE_SIZE;
+	__VMALLOC_RESERVE -= top;
+#endif
 }

This no longer seems to be an appropriate name. How about set_address_top_reserve or something?

Sure how about reserve_top_address()? Requires a respin of the next patch as well. I've done that, but it still needs to be updated to early_param, so I won't repost that one.

thanks,
-chris
--

Subject: Make __FIXADDR_TOP variable to allow it to make space for a hypervisor.

Make __FIXADDR_TOP a variable, so that it can be set to not get in the way of address space a hypervisor may want to reserve.

Original patch by Gerd Hoffmann <[email protected]>

---
 arch/i386/Kconfig         |    1 +
 arch/i386/mm/init.c       |   42 ++++++++++++++++++++++++++++++++++++++++++
 arch/i386/mm/pgtable.c    |   26 ++++++++++++++++++++++++++
 include/asm-i386/fixmap.h |    7 ++++++-
 4 files changed, 75 insertions(+), 1 deletion(-)

===================================================================

arch/i386/Kconfig +1 -0
@@ -792,6 +792,7 @@ config COMPAT_VDSO
 config COMPAT_VDSO
 	bool "Compat VDSO support"
 	default y
+	depends on !PARAVIRT
 	help
 	  Map the VDSO to the predictable old-style address too.
 	---help---

===================================================================

arch/i386/mm/init.c +42 -0

48 lines not on this page · raw message

===================================================================

arch/i386/mm/pgtable.c +26 -0
@@ -12,6 +12,7 @@
 #include <linux/slab.h>
 #include <linux/pagemap.h>
 #include <linux/spinlock.h>
+#include <linux/module.h>
 
 #include <asm/system.h>
 #include <asm/pgtable.h>
@@ -137,6 +138,12 @@ void set_pmd_pfn(unsigned long vaddr, un
 	__flush_tlb_one(vaddr);
 }
 
+static int fixmaps;
+#ifndef CONFIG_COMPAT_VDSO
+unsigned long __FIXADDR_TOP = 0xfffff000;
+EXPORT_SYMBOL(__FIXADDR_TOP);
+#endif
+
 void __set_fixmap (enum fixed_addresses idx, unsigned long phys, pgprot_t flags)
 {
 	unsigned long address = __fix_to_virt(idx);
@@ -146,6 +153,25 @@ void __set_fixmap (enum fixed_addresses 
 		return;
 	}
 	set_pte_pfn(address, phys >> PAGE_SHIFT, flags);
+	fixmaps++;
+}
+
+/**
+ * reserve_top_address - reserves a hole in the top of kernel address space
+ * @reserve - size of hole to reserve
+ *
+ * Can be used to relocate the fixmap area and poke a hole in the top
+ * of kernel address space to make room for a hypervisor.
+ */
+void reserve_top_address(unsigned long reserve)
+{
+	BUG_ON(fixmaps > 0);
+#ifdef CONFIG_COMPAT_VDSO
+	BUG_ON(reserve != 0);
+#else
+	__FIXADDR_TOP = -reserve - PAGE_SIZE;
+	__VMALLOC_RESERVE += reserve;
+#endif
 }
 
 pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)

===================================================================

include/asm-i386/fixmap.h +6 -1
@@ -19,7 +19,11 @@
  * Leave one empty page between vmalloc'ed areas and
  * the start of the fixmap.
  */
-#define __FIXADDR_TOP	0xfffff000
+#ifndef CONFIG_COMPAT_VDSO
+extern unsigned long __FIXADDR_TOP;
+#else
+#define __FIXADDR_TOP  0xfffff000
+#endif
 
 #ifndef __ASSEMBLY__
 #include <linux/kernel.h>
@@ -93,6 +97,7 @@ enum fixed_addresses {
 
 extern void __set_fixmap (enum fixed_addresses idx,
 					unsigned long phys, pgprot_t flags);
+extern void reserve_top_address(unsigned long reserve);
 
 #define set_fixmap(idx, phys) \
 		__set_fixmap(idx, phys, PAGE_KERNEL)