sparsemem support for mips with highmem

I just got sparsemem working on our MIPS 32 platform. I'm not sure if anyone
has done that before since there seems to be a couple of problems in the arch specific code.

Well I realize that it is blazingly simple to turn on sparsemem, but for the idiots (like myself)
out there I created a howto file to put in the Documentation directory just because I thought
it would be a good idea to have some official info on it written down somewhere.

it saved me a ton of space by the way. it seems to work great.

 arch/mips/kernel/setup.c     |   18 +++++++++++++++++-
 arch/mips/mm/init.c          |    3 +++
 include/asm-mips/sparsemem.h |    6 ++++++
 3 files changed, 26 insertions(+), 1 deletions(-)

arch/mips/kernel/setup.c +0 -1

index f8a535a..6ff0f72 100644
@@ -405,7 +405,6 @@ static void __init bootmem_init(void)
 
 		/* Register lowmem ranges */
 		free_bootmem(PFN_PHYS(start), size << PAGE_SHIFT);
-		memory_present(0, start, end);
 	}
 
 	/*

@@ -417,6 +416,23 @@ static void __init bootmem_init(void)
 	 * Reserve initrd memory if needed.

arch/mips/kernel/setup.c +5 -0

+When page_to_pfn() or pfn_to_page() are called there is a bit of overhead to
+look up the proper memory section to get to the page_table, but this
+is small compared to the memory you are likely to save. So, it's not the
+default, but should be used if you have big holes in physical memory.
+

+6) Gotchas

arch/mips/kernel/setup.c +17 -0

+
+One trick that I encountered when I was turning this on for MIPS was that there
+was some code in mem_init() that set the "reserved" flag for pages that were not
+valid RAM. This caused my kernel to crash when I enabled sparsemem since those
+pages (and page descriptors) didn't actually exist. I changed my code by adding
+lines like below:
+
+
+	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
+		struct page *page = pfn_to_page(tmp);
+
+   +		if (!pfn_valid(tmp))
+   +			continue;
+   +
+		if (!page_is_ram(tmp)) {
+			SetPageReserved(page);
+			continue;

Review

Quoted 12 lines

(quoted diff) +9 -0

+		if (boot_mem_map.map[i].type != BOOT_MEM_RAM)
+			continue;
+
+		start = PFN_UP(boot_mem_map.map[i].addr);
+		end   = PFN_DOWN(boot_mem_map.map[i].addr
+				    + boot_mem_map.map[i].size);
+
+		memory_present(0, start, end);
+	}
 }

Is that aligning really necessary? I'm just curious because if it is, it would probably be good to stick it inside memory_present().

include/asm-mips/sparsemem.h line 16

Quoted 8 lines · C Michael Sundius

include/asm-mips/sparsemem.h +6 -0

+
+#ifndef CONFIG_64BIT
+#define SECTION_SIZE_BITS       27	/* 128 MiB */
+#define MAX_PHYSMEM_BITS        31	/* 2 GiB   */
+#else
 #define SECTION_SIZE_BITS       28
 #define MAX_PHYSMEM_BITS        35
+#endif

why is this needed ?

Quoted 13 lines · Thomas Bogendoerfer

include/asm-mips/sparsemem.h line 16

Quoted 9 lines · C Michael Sundius

include/asm-mips/sparsemem.h +6 -0

+
+#ifndef CONFIG_64BIT
+#define SECTION_SIZE_BITS       27	/* 128 MiB */
+#define MAX_PHYSMEM_BITS        31	/* 2 GiB   */
+#else
 #define SECTION_SIZE_BITS       28
 #define MAX_PHYSMEM_BITS        35
+#endif

why is this needed ?

I'm sure Michael can speak to the specifics. But, in general, making SECTION_SIZE_BITS smaller is good if you have lots of small holes in memory. It does this at the cost if increasing the size of the mem_section[] array.

MAX_PHYSMEM_BITS should be as as small as possible, but not so small that it restricts the amount of RAM that your systems support. i>>?Increasing it has the effect of increasing the size of the mem_section[] array.

My guess would be that Michael knew that his 32-bit MIPS platform only ever has 2GB of memory. He also knew that its holes (or RAM) come in 128MB sections. This configuration lets him save the most amount of memory with SPARSEMEM.

Michael, I *guess* you could also include a wee bit on how you chose your numbers in the documentation. Not a big deal, though.

-- Dave

One thing that I had thought about and also came up when my peers here reviewed my changes was that we probably could put those bit numbers (at the very least the segment size) in the .config file.

we decided that the power that be might have had a reason for that and we left it not wanting to meddle with the other arch's.

Dave, do you have a comment about that?

yes, actually the top two bits are used in MIPS as segment bits.

Quoted 2 lines · C Michael Sundius

yes, actually the top two bits are used in MIPS as segment bits.

you are confusing virtual addresses with physcial addresses. There are even 32bit CPU, which could address more than 4GB physical addresses via TLB entries.

Thomas.

Ah, your right. thanks. "but it's not necessar*il*y a good idea". That
is to say, we don't put
memory above 2 GiB. No need to make the mem_section[] array bigger than
need be.

This gives further credence for it to be a configurable in Kconfig as well.

Quoted 8 lines · C Michael Sundius

Ah, your right. thanks. "but it's not necessar*il*y a good idea". That is to say, we don't put
memory above 2 GiB. No need to make the mem_section[] array bigger than need be.

This gives further credence for it to be a configurable in Kconfig as well.

I definitely don't want it to be something that users see. It is never enough overhead to really care. On a 16TB system with 16MB sections, the mem_section[] array is still only 16MB!!

So, I'd say to just make it as big as the arch needs in the worst case (smallest SECTION_SIZE_BITS and largest MAX_PHYSMEM_BITS) and leave it. We might even want to merge the 32 and 64-bit versions.

For your 32-bit version, we now use:
8 bytes (2 32-bit words) for each mem_section[]
2GB/128MB sections = 16
So, that's only 512 bytes.

i>>?For the 64-bit version, we now use:
16 bytes (2 64-bit words) for each mem_section[]
32GB/256MB sections = 128
So, that's only 2048 bytes.

If we were to merge the 32 and 64-bit versions to:
#define SECTION_SIZE_BITS 27
#define MAX_PHYSMEM_BITS 35

Your 32-bit version would go to 2048 bytes, and the 64-bit version would go to 4096 bytes. The 32-bit version would we able to address more memory, and the 64-bit version would be able to handle smaller memory holes more efficiently.

-- Dave

include/asm-mips/sparsemem.h +0 -0

index 795ac6c..64376db 100644
@@ -6,7 +6,7 @@
  * SECTION_SIZE_BITS           2^N: how big each section will be
  * MAX_PHYSMEM_BITS            2^N: how much memory we can have in

that space
*/

Quoted 11 lines · C Michael Sundius

include/asm-mips/sparsemem.h +0 -0

index 795ac6c..64376db 100644
@@ -6,7 +6,7 @@
  * SECTION_SIZE_BITS           2^N: how big each section will be
  * MAX_PHYSMEM_BITS            2^N: how much memory we can have in

that space
*/

include/asm-mips/sparsemem.h line 10

Quoted 3 lines

include/asm-mips/sparsemem.h +1 -1

-#define SECTION_SIZE_BITS       28
+#define SECTION_SIZE_BITS       27     /* 128 MiB */
 #define MAX_PHYSMEM_BITS        35

This looks great to me, as long as the existing MIPS users like it.

Ah, compromise :] that's why you get paid the big bux dave. thanks.

(quoted diff) +2 -0

+Sparsemem divides up physical memory in your system into N section of M
 
                                                            sections
 
+bytes. Page descriptors are created for only those sections that

This overhead can be avoided by configuring sparsemem to use a virtual vmemmap (CONFIG_SPARSEMEM_VMEMMAP). In that case it can be used for non NUMA since the overhead is less than even FLATMEM.

Quoted 3 lines · Christoph Lameter

This overhead can be avoided by configuring sparsemem to use a virtual vmemmap (CONFIG_SPARSEMEM_VMEMMAP). In that case it can be used for non NUMA since the overhead is less than even FLATMEM.

Is that all it takes these days, or do you need some other arch-specific code to help out?

Quoted 2 lines · Dave Hansen

Quoted 1 line · Christoph Lameter

This overhead can be avoided by configuring sparsemem to use a

virtual vmemmap

Quoted 1 line

Quoted 1 line

(CONFIG_SPARSEMEM_VMEMMAP). In that case it can be used for non

NUMA since the

Quoted 2 lines

Quoted 1 line

overhead is less than even FLATMEM.

Quoted 2 lines

Is that all it takes these days, or do you need some other arch-specific code to help out?

Some information is in mm/sparse-vmemmap.c. Simplest configuration is to use vmalloc for the populate function. Otherwise the arch can do what it wants to reduce the overhead of virtual mappings (in the x86 case we use a 2M TLB entry, and since 2M TLBs are also used for the 1-1 physical mapping the overhead is the same as for 1-1 mappings).

you said that the simplest configuration is to use vmalloc for the populate function.
could you expand on that? (i didn't see that the populate function used vmalloc or maybe
we are talking about a different populate function).

this work w/ mips which i understand uses only 2 levels can I just take out the part of
the function that sets up the middle level table?

Quoted 11 lines · Randy Dunlap

(quoted diff) +5 -0

+Sparsemem divides up physical memory in your system into N section of M
 
                                                            sections
 
+bytes. Page descriptors are created for only those sections that
+actually exist (as far as the sparsemem code is concerned). This allows
+for holes in the physical memory without having to waste space by
+creating page discriptors for those pages that do not exist.
 
               descriptors

Review

Quoted 4 lines

(quoted diff) +4 -0

+When page_to_pfn() or pfn_to_page() are called there is a bit of overhead to
+look up the proper memory section to get to the descriptors, but this
+is small compared to the memory you are likely to save. So, it's not the
+default, but should be used if you have big holes in physical memory.

This overhead can be avoided by configuring sparsemem to use a virtual vmemmap (CONFIG_SPARSEMEM_VMEMMAP). In that case it can be used for non NUMA since the overhead is less than even FLATMEM.

On MIPS processors, the kernel runs in unmapped memory, i.e. the TLB isn't even
used, so I don't think you can use that trick. So, this comment doesn't apply to
all processors.

Quoted 5 lines · David VomLehn

On MIPS processors, the kernel runs in unmapped memory, i.e. the TLB isn't even
used, so I don't think you can use that trick. So, this comment doesn't apply to
all processors.

In that case you have a choice between the overhead of sparsemem lookups in every pfn_to_page or using TLB entries to create a virtually mapped memmap which may create TLB pressure.

The virtually mapped memmap results in smaller code and is typically more effective since the processor caches the TLB entries.

Quoted 8 lines · Christoph Lameter

Quoted 5 lines · David VomLehn

On MIPS processors, the kernel runs in unmapped memory, i.e. the TLB isn't even
used, so I don't think you can use that trick. So, this comment doesn't apply to
all processors.

Quoted 6 lines

In that case you have a choice between the overhead of sparsemem lookups in every pfn_to_page or using TLB entries to create a virtually mapped memmap which may create TLB pressure.

The virtually mapped memmap results in smaller code and is typically more effective since the processor caches the TLB entries.

I'm pretty ignorant on this subject, but I think this is worth discussing. On a MIPS processor, access to low memory bypasses the TLB entirely. I think what you are suggesting is to use mapped addresses to make all of low memory virtually contiguous. On a MIPS processor, we could do this by allocating a "wired" TLB entry for each physically contiguous block of memory. Wired TLB entries are never replaced, so they are very efficient for long-lived mappings such as this. Using the TLB in this way does increase TLB pressure, but most platforms probably have a very small number of "holes" in their memory. So, this may be a small overhead.

If we took this approach, we could then have a single, simple memmap array where pfn_to_page looks just about the same as it looks with a flat memory model.

If I'm understand what you are suggesting correctly (a big if), the downside is that we'd pay the cost of a TLB match for each non-cached low memory data access. It seems to me that would be a higher cost than having the occasional, more expensive, sparsemem lookup in pfn_to_page.

Anyone with more in-depth MIPS processor architecture knowledge care to weigh in on this?
--
David VomLehn

Quoted 2 lines

The virtually mapped memmap results in smaller code and is typically more effective since the processor caches the TLB entries.

I'm pretty ignorant on this subject, but I think this is worth discussing. On a MIPS processor, access to low memory bypasses the TLB entirely. I think what you are suggesting is to use mapped addresses to make all of low memory virtually contiguous. On a MIPS processor, we

could do this by allocating a "wired" TLB entry for each physically contiguous block of memory. Wired TLB entries are never replaced, so they are very efficient for long-lived mappings such as this. Using the TLB in this way does increase TLB pressure, but most platforms probably have a very small number of "holes" in their memory. So, this may be a small overhead.

If I'm understand what you are suggesting correctly (a big if), the downside is that we'd pay the cost of a TLB match for each non-cached low memory data access. It seems to me that would be a higher cost than having the occasional, more expensive, sparsemem lookup in pfn_to_page.

Quoted 6 lines · David VomLehn

Quoted 2 lines

The virtually mapped memmap results in smaller code and is typically more effective since the processor caches the TLB entries.

I'm pretty ignorant on this subject, but I think this is worth discussing. On a MIPS processor, access to low memory bypasses the TLB entirely. I think what you are suggesting is to use mapped addresses to make all of low memory virtually contiguous. On a MIPS processor, we

No the virtual area is only used to map the memory map (the array of page structs). That is just a small fraction of memory.

Quoted 2 lines

could do this by allocating a "wired" TLB entry for each physically contiguous block of memory.

That would consume precious resources.

Just place the memmap into the vmalloc area gets you there. TLB entries should be loaded on demand.

Quoted 1 line

If I'm understand what you are suggesting correctly (a big if)

The cost going through a TLB mapping is only incurred for accesses to the memmap array. Not for general memory accesses.

For a flat memory model, the page descriptors array memmap is contiguously allocated in low memory. For sparse memory, you only allocate memory to hold page descriptors that actually exist. If you don't enable CONFIG_SPARSEMEM_VMEMMAP, you introduce a level of indirection where the top bits of an address gives you an index into an array that points to an array of page descriptors for that section of memory. This has some performance impact relative to flat memory due to the extra memory access to read the pointer to the array of page descriptors.

If you do enable CONFIG_SPARSEMEM_VMEMMAP, you still allocate memory to hold page descriptors, but you map that memory into virtual space so that a given page descriptor for a physical address is at the offset from the beginning of the virtual memmap corresponding to the page frame number of that address. This gives you a single memmap, just like you had in the flat memory case, though memmap now lives in virtual address space. Since memmap now lives in virtual address space, you don't need to use any memory to back the virtual addresses that correspond to the holes in your physical memory, which is how you save a lot of physical memory. The performance impact relative to flag memory is now that of having to go through the TLB to get to the page descriptor.

If you are using CONFIG_SPARSEMEM_VMEMMAP and the corresponding TLB entry is present, you expect this will be faster than the extra memory access you do when CONFIG_SPARSEMEM_VMEMMAP is not enabled, even if that memory is in cache. This seems like a pretty reasonable expectation to me. Since TLB entries cover much more memory than the cache, it also seems like there would be a much better chance that you already have the corresponding TLB entry than having the indirect memory pointer in cache. And, in the worst case, reading the TLB entry is just another memory access, so it's closely equivalent to not enabling CONFIG_SPARSEMEM_VMEMMAP.

So, if I understand this right, the overhead on a MIPS processor using flat memory versus using sparse memory with CONFIG_SPARSEMEM_VMEMMAP enabled would be mostly the difference between accessing unmapped memory, which doesn't go through the TLB, and mapped memory, which does. Even though there is some impact due to TLB misses, this should be pretty reasonable. What a way cool approach!

My guess would be that Michael knew that his 32-bit MIPS platform only ever has 2GB of memory.

fixed patch

Documentation/sparsemem.txt new file +21 -0

new file mode 100644
index 0000000..6aea0d1
@@ -0,0 +1,93 @@
+Sparsemem divides up physical memory in your system into N section of M
+bytes. Page descriptors are created for only those sections that
+actually exist (as far as the sparsemem code is concerned). This allows
+for holes in the physical memory without having to waste space by
+creating page discriptors for those pages that do not exist.
+When page_to_pfn() or pfn_to_page() are called there is a bit of overhead to
+look up the proper memory section to get to the descriptors, but this
+is small compared to the memory you are likely to save. So, it's not the
+default, but should be used if you have big holes in physical memory.
+
+Note that discontiguous memory is more closely related to NUMA machines
+and if you are a single CPU system use sparsemem and not discontig. 
+It's much simpler. 
+
+1) CALL MEMORY_PRESENT()
+Once the bootmem allocator is up and running, you should call the
+sparsemem function "memory_present(node, pfn_start, pfn_end)" for each
+block of memory that exists on your system.
+
+2) DETERMINE AND SET THE SIZE OF SECTIONS AND PHYSMEM
+The size of N and M above depend upon your architecture

Documentation/sparsemem.txt new file +14 -0

+
+if they don't already exist, where: 
+
+ * SECTION_SIZE_BITS            2^M: how big each section will be
+ * MAX_PHYSMEM_BITS             2^N: how much memory we can have in that
+                                     space
+
+3) INITIALIZE SPARSE MEMORY
+You should make sure that you initialize the sparse memory code by calling 
+
+	bootmem_init();
+  +	sparse_init();
+	paging_init();
+

Documentation/sparsemem.txt new file +25 -0

+
+5) CONFIG
+Run make menuconfig or make gconfig, as you like, and turn on the sparsemem
+memory model under the "Kernel Type" --> "Memory Model" and then build your
+kernel.
+
+
+6) Gotchas
+
+One trick that I encountered when I was turning this on for MIPS was that there
+was some code in mem_init() that set the "reserved" flag for pages that were not
+valid RAM. This caused my kernel to crash when I enabled sparsemem since those
+pages (and page descriptors) didn't actually exist. I changed my code by adding
+lines like below:
+
+
+	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
+		struct page *page = pfn_to_page(tmp);
+
+   +		if (!pfn_valid(tmp))
+   +			continue;
+   +
+		if (!page_is_ram(tmp)) {
+			SetPageReserved(page);
+			continue;

Documentation/sparsemem.txt new file +3 -0

+
+
+

arch/mips/kernel/setup.c +17 -1

index c6a063b..5b1af87 100644
@@ -408,7 +408,6 @@ static void __init bootmem_init(void)
 
 		/* Register lowmem ranges */
 		free_bootmem(PFN_PHYS(start), size << PAGE_SHIFT);
-		memory_present(0, start, end);
 	}
 
 	/*

@@ -420,6 +419,23 @@ static void __init bootmem_init(void)
 	 * Reserve initrd memory if needed.
 	 */
 	finalize_initrd();
+
+	/* call memory present for all the ram */
+	for (i = 0; i < boot_mem_map.nr_map; i++) {
+		unsigned long start, end;
+
+		/*
+		 * memory present only usable memory.
+		 */
+		if (boot_mem_map.map[i].type != BOOT_MEM_RAM)
+			continue;
+
+		start = PFN_UP(boot_mem_map.map[i].addr);
+		end   = PFN_DOWN(boot_mem_map.map[i].addr
+				    + boot_mem_map.map[i].size);
+
+		memory_present(0, start, end);
+	}
 }
 
 #endif	/* CONFIG_SGI_IP27 */

arch/mips/mm/init.c +3 -0

index 137c14b..31496a1 100644
@@ -414,6 +414,9 @@ void __init mem_init(void)
 	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
 		struct page *page = pfn_to_page(tmp);
 
+		if (!pfn_valid(tmp))
+			continue;
+
 		if (!page_is_ram(tmp)) {
 			SetPageReserved(page);
 			continue;

include/asm-mips/sparsemem.h +1 -0

index 795ac6c..9faaf59 100644
@@ -6,8 +6,14 @@
  * SECTION_SIZE_BITS		2^N: how big each section will be
  * MAX_PHYSMEM_BITS		2^N: how much memory we can have in that space
  */
+

Hi Michael,

After I read this link, noticed that you have the following patch, but when I check up the mips-linux, the patch is not there.

I wonder if you could explain to me a little bit?

Thank you!

John
P.S.: I also worked at SciAtl a few years ago in IPTV division.

Otherwise it looks good to me. I see from the rest of the thread that there is some discussion over the sizes of these, with that sorted.

Acked-by Andy Whitcroft <[email protected]>

-apw