x86-64: improve e820_search_gap()
PATCH 1/1 x86-64: improve e820_search_gap()
Impact: bug fix
Blindly putting the gap close after max_pfn is in conflict with that same memory range potentially being used by hotplugged memory.
Also, make the function static to ensure there are no other users that could depend on the previous behavior regarding the way start_addr gets specified.
--- arch/x86/include/asm/e820.h | 2 -- arch/x86/kernel/e820.c | 11 +++++++++-- 2 files changed, 9 insertions(+), 4 deletions(-)
arch/x86/include/asm/e820.h +0 -2
@@ -79,8 +79,6 @@ extern u64 e820_remove_range(u64 start, int checktype);extern void update_e820 (void );extern void e820_setup_gap (void ); -extern int e820_search_gap (unsigned long *gapstart,unsigned long *gapsize, -unsigned long start_addr,unsigned long long end_addr);struct setup_data ;extern void parse_e820_ext (struct setup_data *data,unsigned long pa_data);
arch/x86/kernel/e820.c +9 -2
@@ -574,13 +574,19 @@ static void __init update_e820_saved(voi/* * Search for a gap in the e820 memory space from start_addr to end_addr. */ -__init int e820_search_gap (unsigned long *gapstart,unsigned long *gapsize, +static int __init +e820_search_gap (unsigned long *gapstart,unsigned long *gapsize,unsigned long start_addr,unsigned long long end_addr) {unsigned long long last;int i = e820.nr_map;int found =0 ; +#ifdef CONFIG_X86_64 +if (start_addr >=MAX_GAP_END ) + last = end_addr ?: (1UL << boot_cpu_data.x86_phys_bits); + else +#endif last = (end_addr && end_addr <MAX_GAP_END ) ? end_addr :MAX_GAP_END ;while (--i >=0 ) {
@@ -626,11 +632,12 @@ __init void e820_setup_gap(void)#ifdef CONFIG_X86_64 if (!found) { - gapstart = (max_pfn <<PAGE_SHIFT ) +1024 *1024 ;printk (KERN_ERR "PCI: Warning: Cannot find a gap in the 32bit " "address range\n" KERN_ERR "PCI: Unassigned devices with 32bit resource " "registers may break!\n" ); + found =e820_search_gap (&gapstart, &gapsize,MAX_GAP_END ,0 ); +WARN_ON (!found); }#endif
H. Peter Anvin
reviewerQuoted 11 lines · Jan Beulich
Impact: bug fix
Blindly putting the gap close after
max_pfnis in conflict with that same memory range potentially being used by hotplugged memory.Also, make the function static to ensure there are no other users that could depend on the previous behavior regarding the way
start_addrgets specified.Signed-off-by Jan Beulich <[email protected]>
So blindly locate it somewhere else? How is that inherently better? Wouldn't a machine with hotplug memory (which doesn't bother advertising that fact so we can reserve the address space) be just as likely to use a sparse memory space, since one can hardly expect the hardware to pack the space (packing in hardware is why PCs generally have a mostly-contiguous RAM space) when the memory is hotplugged?
I think I'm missing something fundamental...
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 14 lines
Quoted 1 line
Quoted 1 line
"H. Peter Anvin" <[email protected]> 08.05.09 06:59 >>>
Quoted 11 lines · Jan Beulich
Impact: bug fix
Blindly putting the gap close after
max_pfnis in conflict with that same memory range potentially being used by hotplugged memory.Also, make the function static to ensure there are no other users that could depend on the previous behavior regarding the way
start_addrgets specified.Signed-off-by Jan Beulich <[email protected]>
Quoted 6 lines
So blindly locate it somewhere else? How is that inherently better? Wouldn't a machine with hotplug memory (which doesn't bother advertising that fact so we can reserve the address space) be just as likely to use a sparse memory space, since one can hardly expect the hardware to pack the space (packing in hardware is why PCs generally have a mostly-contiguous RAM space) when the memory is hotplugged?
Why blindly? Aren't hotpluggable memory ranges supposed to be reserved in the E820 map?
Jan
H. Peter Anvin
reviewerQuoted 11 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
So blindly locate it somewhere else? How is that inherently better? Wouldn't a machine with hotplug memory (which doesn't bother advertising that fact so we can reserve the address space) be just as likely to use a sparse memory space, since one can hardly expect the hardware to pack the space (packing in hardware is why PCs generally have a mostly-contiguous RAM space) when the memory is hotplugged?
Why blindly? Aren't hotpluggable memory ranges supposed to be reserved in the E820 map?
That's was my reaction too... so I guess I'm misreading the patch. Could you perhaps add a bit more to the description, specifically of what policy your patch implements.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 3 lines · Jan Beulich
Why blindly? Aren't hotpluggable memory ranges supposed to be reserved in the E820 map?
They are supposed to be reserved in SRAT, but not in e820.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
H. Peter Anvin
reviewerQuoted 5 lines · Andi Kleen
Quoted 2 lines · Jan Beulich
Why blindly? Aren't hotpluggable memory ranges supposed to be reserved in the E820 map?
They are supposed to be reserved in SRAT, but not in e820.
Ah, okay. Perhaps we should fold this information into our internal "e820" map? If so I guess the question is how soon we can do that.
-hpa
Quoted 9 lines · H. Peter Anvin
Quoted 5 lines · Andi Kleen
Quoted 2 lines · Jan Beulich
Why blindly? Aren't hotpluggable memory ranges supposed to be reserved in the E820 map?
They are supposed to be reserved in SRAT, but not in e820.
Ah, okay. Perhaps we should fold this information into our internal "e820" map? If so I guess the question is how soon we can do that.
You could do that or just query SRAT too, but ...
The problem is really that there are still systems which have hidden holes which are not reserved anywhere. When I last hacked on the gap algorithm it triggered subtle bugs. So this will never be fully reliable.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
H. Peter Anvin
reviewerQuoted 5 lines · Andi Kleen
Quoted 4 lines · H. Peter Anvin
Quoted 3 lines · Andi Kleen
Quoted 2 lines · Jan Beulich
Why blindly? Aren't hotpluggable memory ranges supposed to be reserved in the E820 map?
Quoted 9 lines
Quoted 2 lines
Quoted 1 line
They are supposed to be reserved in SRAT, but not in e820.
Ah, okay. Perhaps we should fold this information into our internal "e820" map? If so I guess the question is how soon we can do that.
You could do that or just query SRAT too, but ...
The problem is really that there are still systems which have hidden holes which are not reserved anywhere. When I last hacked on the gap algorithm it triggered subtle bugs. So this will never be fully reliable.
That's without a question.
However, as far as querying SRAT, I don't like the idea of spreading the knowledge of the system memory map out between a bunch of different places, each of which have a little piece of the puzzle. It puts a huge onus on the user to know what mechanisms are actually available, and really makes a shitty interface.
-hpa
Quoted 5 lines
However, as far as querying SRAT, I don't like the idea of spreading the knowledge of the system memory map out between a bunch of different places, each of which have a little piece of the puzzle. It puts a huge onus on the user to know what mechanisms are actually available, and really makes a shitty interface.
AFAIK another popular OS always combines mappings from all sources (e820, SRAT, PCI, PNP, ACPI etc.) in the query before allocating anything. Something like that might be a reasonable long term direction for Linux too, but it's probably also a can of worms to handle the conflicts between the various sources (e.g. e820 reserves a lot of things in other sources too). It would be a rather large change. Maybe that would handle the systems I thought of above.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
H. Peter Anvin
reviewerQuoted 6 lines · Andi Kleen
Quoted 5 lines
However, as far as querying SRAT, I don't like the idea of spreading the knowledge of the system memory map out between a bunch of different places, each of which have a little piece of the puzzle. It puts a huge onus on the user to know what mechanisms are actually available, and really makes a shitty interface.
Quoted 7 lines
AFAIK another popular OS always combines mappings from all sources (e820, SRAT, PCI, PNP, ACPI etc.) in the query before allocating anything. Something like that might be a reasonable long term direction for Linux too, but it's probably also a can of worms to handle the conflicts between the various sources (e.g. e820 reserves a lot of things in other sources too). It would be a rather large change. Maybe that would handle the systems I thought of above.
You *always* have a conflict resolution policy... whether or not it is explicit or accidental, and whether or not it is the result of merging the data or just accessing multiple data sources is another matter. It might be hard to replicate an accidental policy in an explicit way, or the accidental policy really might make no sense, which may mean behavior changes.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 15 lines · H. Peter Anvin
Quoted 6 lines · Andi Kleen
Quoted 5 lines
However, as far as querying SRAT, I don't like the idea of spreading the knowledge of the system memory map out between a bunch of different places, each of which have a little piece of the puzzle. It puts a huge onus on the user to know what mechanisms are actually available, and really makes a shitty interface.
Quoted 7 lines
AFAIK another popular OS always combines mappings from all sources (e820, SRAT, PCI, PNP, ACPI etc.) in the query before allocating anything. Something like that might be a reasonable long term direction for Linux too, but it's probably also a can of worms to handle the conflicts between the various sources (e.g. e820 reserves a lot of things in other sources too). It would be a rather large change. Maybe that would handle the systems I thought of above.
Quoted 1 line
You *always* have a conflict resolution policy... whether or not it is
Well right now we ignore ACPI/PNP data, sometimes ignore PCI data and only look in e820 and sometimes only use PCI/e820 and sometimes only use SRAT (or at least it was like this at some point)
BTW SRAT hot range can be also wrong, some BIOS always had a full 512GB range even though they don't support hotplug.
However I haven't seen a system which supports hotplug (not that there are very many of those) where the entry was not there, so at least that one should be safe.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
H. Peter Anvin
reviewerQuoted 5 lines · Andi Kleen
Well right now we ignore ACPI/PNP data, sometimes ignore PCI data and only look in e820 and sometimes only use PCI/e820 and sometimes only use SRAT (or at least it was like this at some point)
That *is* a conflict resolution policy. Quite probably an accidental one.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.