Skip to contentSkip to conversation

memblock and bootmem problems if start + size = 4GB

PATCH Michal Simek 2 participants 9 messages Dec 19, 2011 – Dec 30, 2011

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi,

I have reached some problems with memblock and bootmem code for some configurations. We can completely setup the whole system and all addresses in it.
The problem happens if we place main memory to the end of address space when mem_start + size reach 4GB limit.

For example:

mem_start      0xF000 0000
mem_size       0x1000 0000 (or better lowmem size)
mem_end        0xFFFF FFFF

start + size 0x1 0000 0000 (u32 limit reached).

I have done some patches which completely remove start + size values from architecture specific code but I have found some problem in generic code too.

For example in bootmem code where are three places where physaddr + size is used. I would prefer to retype it to u64 because baseaddr and size don't need to be 2^n.

Is it correct solution? If yes, I will create proper patch.

mm/bootmem.c +3 -3
index 1a77012..45a691a 100644
@@ -371,7 +371,7 @@ void __init free_bootmem_node(pg_data_t *pgdat, unsigned long physaddr,
        kmemleak_free_part(__va(physaddr), size);
 
        start = PFN_UP(physaddr);
-       end = PFN_DOWN(physaddr + size);
+       end = PFN_DOWN((u64)physaddr + (u64)size);
 
        mark_bootmem_node(pgdat->bdata, start, end, 0, 0);
 }
@@ -414,7 +414,7 @@ int __init reserve_bootmem_node(pg_data_t *pgdat, unsigned long physaddr,
        unsigned long start, end;
 
        start = PFN_DOWN(physaddr);
-       end = PFN_UP(physaddr + size);
+       end = PFN_UP((u64)physaddr + (u64)size);
 
        return mark_bootmem_node(pgdat->bdata, start, end, 1, flags);
 }
@@ -435,7 +435,7 @@ int __init reserve_bootmem(unsigned long addr, unsigned long size,
        unsigned long start, end;
 
        start = PFN_DOWN(addr);
-       end = PFN_UP(addr + size);
+       end = PFN_UP((u64)addr + (u64)size);
 
        return mark_bootmem(start, end, 1, flags);
 }

The similar problem is with memblock code.

include/linux/memblock.h +2 -2
index e6b843e..55d5279 100644
@@ -127,7 +127,7 @@ static inline unsigned long memblock_region_memory_base_pfn(const struct membloc
  */
 static inline unsigned long memblock_region_memory_end_pfn(const struct memblock_region *reg)
 {
-       return PFN_DOWN(reg->base + reg->size);
+       return PFN_DOWN((u64)reg->base + (u64)reg->size);
 }
 
 /**
@@ -145,7 +145,7 @@ static inline unsigned long memblock_region_reserved_base_pfn(const struct membl
  */
 static inline unsigned long memblock_region_reserved_end_pfn(const struct memblock_region *reg)
 {
-       return PFN_UP(reg->base + reg->size);
+       return PFN_UP((u64)reg->base + (u64)reg->size);
 }
 
 #define for_each_memblock(memblock_type, region)                                       \

Plus fixing two conditions in memblock_find_base for the same reasons.

mm/memblock.c +2 -2
index 84bec49..6c443bd 100644
@@ -131,10 +131,10 @@ static phys_addr_t __init_memblock memblock_find_base(phys_addr_t size,
 
                if (memblocksize < size)
                        continue;
-               if ((memblockbase + memblocksize) <= start)
+               if ((memblockbase + memblocksize - 1) <= start)
                        break;
                bottom = max(memblockbase, start);
-               top = min(memblockbase + memblocksize, end);
+               top = min(memblockbase + memblocksize - 1, end);
                if (bottom >= top)
                        continue;
                found = memblock_find_region(bottom, top, size, align);

Thanks,
Michal

Signature · 11 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#1 lore raw

Hello, Michal.

Quoted 11 lines · Michal Simek

I have reached some problems with memblock and bootmem code for some configurations. We can completely setup the whole system and all addresses in it.
The problem happens if we place main memory to the end of address space when mem_start + size reach 4GB limit.

For example:

mem_start      0xF000 0000
mem_size       0x1000 0000 (or better lowmem size)
mem_end        0xFFFF FFFF

start + size 0x1 0000 0000 (u32 limit reached).

Quoted 7 lines

I have done some patches which completely remove start + size values from architecture specific code but I have found some problem in generic code too.

For example in bootmem code where are three places where physaddr + size is used. I would prefer to retype it to u64 because baseaddr and size don't need to be 2^n.

Is it correct solution? If yes, I will create proper patch.

Yeah, that's an inherent problem in using [) ranges but I think chopping off the last page probably is simpler and more robust solution. Currently, memblock_add_region() would simply ignore if address range overflows but making it just ignore the last page is several lines of addition. Wouldn't that be effective enough while staying very simple?

Thanks.

Signature · 8 lines
tejun

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#2 lore raw

Hi Tejun,

Quoted 18 lines · Michal Simek
Quoted 17 lines

I have reached some problems with memblock and bootmem code for some configurations. We can completely setup the whole system and all addresses in it.
The problem happens if we place main memory to the end of address space when mem_start + size reach 4GB limit.

For example:

mem_start      0xF000 0000
mem_size       0x1000 0000 (or better lowmem size)
mem_end        0xFFFF FFFF

start + size 0x1 0000 0000 (u32 limit reached).

I have done some patches which completely remove start + size values from architecture specific code but I have found some problem in generic code too.

For example in bootmem code where are three places where physaddr + size is used. I would prefer to retype it to u64 because baseaddr and size don't need to be 2^n.

Quoted 8 lines
Quoted 1 line

Is it correct solution? If yes, I will create proper patch.

Yeah, that's an inherent problem in using [) ranges but I think chopping off the last page probably is simpler and more robust solution. Currently, memblock_add_region() would simply ignore if address range overflows but making it just ignore the last page is several lines of addition. Wouldn't that be effective enough while staying very simple?

The main problem is with PFN_DOWN/UP macros and it is in __init section. The result will be definitely u32 type (for 32bit archs) anyway and seems to me better solution than ignoring the last page.

Is there any internal kernel test code to test all pages - try to allocate/use/test it? It will be especially good to do so on the last page to see if there is any problem or not.

That two conditions in memblock should be ok.

Thanks,
Michal

Signature · 11 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#3 lore raw
Quoted 29 lines · Michal Simek

Hi Tejun,

Quoted 18 lines · Michal Simek
Quoted 17 lines

I have reached some problems with memblock and bootmem code for some configurations. We can completely setup the whole system and all addresses in it.
The problem happens if we place main memory to the end of address space when mem_start + size reach 4GB limit.

For example:

mem_start      0xF000 0000
mem_size       0x1000 0000 (or better lowmem size)
mem_end        0xFFFF FFFF

start + size 0x1 0000 0000 (u32 limit reached).

I have done some patches which completely remove start + size values from architecture specific code but I have found some problem in generic code too.

For example in bootmem code where are three places where physaddr + size is used. I would prefer to retype it to u64 because baseaddr and size don't need to be 2^n.

Quoted 8 lines
Quoted 1 line

Is it correct solution? If yes, I will create proper patch.

Yeah, that's an inherent problem in using [) ranges but I think chopping off the last page probably is simpler and more robust solution. Currently, memblock_add_region() would simply ignore if address range overflows but making it just ignore the last page is several lines of addition. Wouldn't that be effective enough while staying very simple?

Quoted 8 lines

The main problem is with PFN_DOWN/UP macros and it is in __init section. The result will be definitely u32 type (for 32bit archs) anyway and seems to me better solution than ignoring the last page.

Is there any internal kernel test code to test all pages - try to allocate/use/test it? It will be especially good to do so on the last page to see if there is any problem or not.

That two conditions in memblock should be ok.

Tejun and Andrew: any other comment?

Thanks,
Michal

Signature · 11 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#4 lore raw
In reply to Michal Simek

Hello,

Quoted 10 lines · Michal Simek
Quoted 6 lines

Yeah, that's an inherent problem in using [) ranges but I think chopping off the last page probably is simpler and more robust solution. Currently, memblock_add_region() would simply ignore if address range overflows but making it just ignore the last page is several lines of addition. Wouldn't that be effective enough while staying very simple?

The main problem is with PFN_DOWN/UP macros and it is in __init section. The result will be definitely u32 type (for 32bit archs) anyway and seems to me better solution than ignoring the last page.

Other than being able to use one more 4k page, is there any other benefit? Maybe others had different experiences but in my exprience trying to extend range coverages - be it stack top/end pointers, address ranges or whatnot - using [] ranges or special flag usually ended up adding complexity while adding almost nothing tangible. On extreme cases, people even carry separate valid flag to use %NULL as valid address, which is pretty silly, IMHO. So, unless there's some benefit that I'm missing, I still think it's an overkill. It's more complex and difficult to test and verify. Why bother for a single page?

Thanks.

Signature · 8 lines
tejun

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#5 lore raw
Quoted 13 lines · Tejun Heo

Hello,

Quoted 9 lines · Michal Simek
Quoted 6 lines

Yeah, that's an inherent problem in using [) ranges but I think chopping off the last page probably is simpler and more robust solution. Currently, memblock_add_region() would simply ignore if address range overflows but making it just ignore the last page is several lines of addition. Wouldn't that be effective enough while staying very simple?

The main problem is with PFN_DOWN/UP macros and it is in __init section. The result will be definitely u32 type (for 32bit archs) anyway and seems to me better solution than ignoring the last page.

Quoted 5 lines

Other than being able to use one more 4k page, is there any other benefit? Maybe others had different experiences but in my exprience trying to extend range coverages - be it stack top/end pointers, address ranges or whatnot - using [] ranges or special flag usually ended up adding complexity while adding almost nothing tangible.

First of all I don't like to use your term "extend range coverages".
We don't want to extend any ranges - we just wanted to place memory to the end of address space and be able to work with. It is limitation which should be fixed somehow. And I would expect that PFN_XX(base + size) will be in u32 range.

Probably the best solution will be to use PFN macro in one place and do not covert addresses in common code.

+ change parameters in bootmem code because some arch do free_bootmem_node(..., PFN_PHYS(), ...)
and
reserve_bootmem_node(..., PFN_PHYS(), ...)

and then in that functions(free/reseve_bootmem_code) are used PFN_DOWN/PFN_UP macros.
If alignment is handled by architecture code (which I believe is) then should be possible to change parameters.

For example:
void __init free_bootmem_node(pg_data_t *pgdat, unsigned long start_pfn,
unsigned long end_pfn)

int __init reserve_bootmem_node(pg_data_t *pgdat, unsigned long start_pfn,
unsigned long end_pfn, int flags)

Is there any reason to use use physical addresses instead of pfns in bootmem code?

Quoted 6 lines

On
extreme cases, people even carry separate valid flag to use %NULL as valid address, which is pretty silly, IMHO. So, unless there's some benefit that I'm missing, I still think it's an overkill. It's more complex and difficult to test and verify. Why bother for a single page?

Where do you think this page should be placed? In common code or in architecture memory code where one page from the top of 4G should be subtract?

Thanks,
Michal

Signature · 11 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#6 lore raw

Hello,

Quoted 3 lines · Michal Simek

First of all I don't like to use your term "extend range coverages". We don't want to extend any ranges - we just wanted to place memory to the end of address space and be able to work with.

It is, as long as we use address ranges. Either we can express length of zero or include the last address.

Quoted 10 lines

It is limitation which should be fixed somehow.
And I would expect that PFN_XX(base + size) will be in u32 range.

Probably the best solution will be to use PFN macro in one place and do not covert addresses in common code.

+ change parameters in bootmem code because some arch do free_bootmem_node(..., PFN_PHYS(), ...)
and
reserve_bootmem_node(..., PFN_PHYS(), ...)

So now we're talking about a lot of code just for ONE page and regardless of the representation in the memblock or other memory management code, I think trying to use that page is fundamentally a bad idea. There are a lot of places in the kernel where phys_addr_t is used. Using that one last page risks obscure overflow bug if any of them is using [start,end) ranges and bugs triggered such way would be extremely difficult to track down. It doesn't make any sense to do that for that one last page. It's less severe but in the same vein as trying to use %NULL as a valid address. It's an absurdly silly tradeoff.

So, FWIW, I think that is a horrible idea.

Quoted 9 lines
Quoted 6 lines

On
extreme cases, people even carry separate valid flag to use %NULL as valid address, which is pretty silly, IMHO. So, unless there's some benefit that I'm missing, I still think it's an overkill. It's more complex and difficult to test and verify. Why bother for a single page?

Where do you think this page should be placed? In common code or in architecture memory code where one page from the top of 4G should be subtract?

With the pending updates to memblock code in tip scheduled for the coming merge window, I *think* it would be a single (or a few) line change in memblock_add_region() where it checks for overflow.

Thanks.

Signature · 8 lines
tejun

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#7 lore raw
Quoted 21 lines · Tejun Heo

Hello,

Quoted 3 lines · Michal Simek

First of all I don't like to use your term "extend range coverages". We don't want to extend any ranges - we just wanted to place memory to the end of address space and be able to work with.

It is, as long as we use address ranges. Either we can express length of zero or include the last address.

Quoted 10 lines

It is limitation which should be fixed somehow.
And I would expect that PFN_XX(base + size) will be in u32 range.

Probably the best solution will be to use PFN macro in one place and do not covert addresses in common code.

+ change parameters in bootmem code because some arch do free_bootmem_node(..., PFN_PHYS(), ...)
and
reserve_bootmem_node(..., PFN_PHYS(), ...)

Quoted 5 lines

So now we're talking about a lot of code just for ONE page and regardless of the representation in the memblock or other memory management code, I think trying to use that page is fundamentally a bad idea. There are a lot of places in the kernel where phys_addr_t is used.

I haven't said to replace phys_addr_t!
My point was something like this (just as example on parisc and free_bootmem_node). The problematic part is kmemleak code which could be good reason not to change it.

Thanks,
Michal

arch/parisc/mm/init.c +2 -2
index 82f364e..b83ee32 100644
@@ -291,8 +291,8 @@ static void __init setup_bootmem(void)
                                                start_pfn,
                                                (start_pfn + npages) );
                free_bootmem_node(NODE_DATA(i),
-                                 (start_pfn << PAGE_SHIFT),
-                                 (npages << PAGE_SHIFT) );
+                                 start_pfn,
+                                 npages);
                bootmap_pfn += (bootmap_size + PAGE_SIZE - 1) >> PAGE_SHIFT;
                if ((start_pfn + npages) > max_pfn)
                        max_pfn = start_pfn + npages;
mm/bootmem.c +4 -9
index 45a691a..dfbfc47 100644
@@ -363,17 +363,12 @@ static int __init mark_bootmem(unsigned long start, unsigned long end,
  *
  * The range must reside completely on the specified node.
  */
-void __init free_bootmem_node(pg_data_t *pgdat, unsigned long physaddr,
-                             unsigned long size)
+void __init free_bootmem_node(pg_data_t *pgdat, unsigned long startpfn,
+                             unsigned long endpfn)
 {
-       unsigned long start, end;
-
-       kmemleak_free_part(__va(physaddr), size);
+       kmemleak_free_part(__va(startpfn << PAGE_SHIFT), (endpfn - startpfn) << PAGE_SHIFT);
 
-       start = PFN_UP(physaddr);
-       end = PFN_DOWN((u64)physaddr + (u64)size);
-
-       mark_bootmem_node(pgdat->bdata, start, end, 0, 0);
+       mark_bootmem_node(pgdat->bdata, startpfn, endpfn, 0, 0);
 }
 
 
Signature · 11 lines
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#8 lore raw

Hello,

Quoted 5 lines · Michal Simek

I haven't said to replace phys_addr_t!
My point was something like this (just as example on parisc and free_bootmem_node).
The problematic part is kmemleak code which could be good reason not to change it.

I think it's still a bad idea and you haven't provided any justification for it. Think about it - any user which uses pa() may get that last page and if that user is using [start,end) range, it may overflow. It doesn't even matter how you implement it. I just can't understand why you obsess about that last page. It doesn't matter. Just add those few lines to exclude the last single page and be done with it. Unless you're gonna provide rationale for why adding such risk and more complexity makes sense for that single last page which most BIOSes wouldn't even map, I don't really think this thread is going anywhere.

Thanks.

Signature · 8 lines
tejun

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>