Config NO_BOOTMEM breaks my amd64 box
Subsystems (3)
Please make NO_BOOTMEM default to n, at least for amd64, where I've found that it leads to all kinds of strange, undebuggable boot hangs and errors (with relatively current Fedora development userland).
Also, the help text for the item makes little sense to a non-expert in this area:
" ---help---
Use early_res directly instead of bootmem before slab is ready.
- allocator (buddy) [generic]
- early allocator (bootmem) [generic]
- very early allocator (reserve_early*()) [x86]
- very very early allocator (early brk model) [x86]
So reduce one layer between early allocator to final allocator."
I had no idea what all this meant, so trusted the default=y and then spent several hours wondering why everything was breaking, and would likley not have figured it out in linear time without a suggestion from Dave Airlie.
- James
Signature · 2 lines
James Morris <[email protected]>
H. Peter Anvin
reviewerQuoted 3 lines · James Morris
Please make
NO_BOOTMEMdefault to n, at least for amd64, where I've found that it leads to all kinds of strange, undebuggable boot hangs and errors (with relatively current Fedora development userland).
Have you tested it with the latest fixes that are now in Linus' tree (-rc3)?
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 6 lines · H. Peter Anvin
Quoted 3 lines · James Morris
Please make
NO_BOOTMEMdefault to n, at least for amd64, where I've found that it leads to all kinds of strange, undebuggable boot hangs and errors (with relatively current Fedora development userland).Have you tested it with the latest fixes that are now in Linus' tree (-rc3)?
Yes, it was happening with -rc3.
Signature · 2 lines
James Morris <[email protected]>
Quoted 10 lines · James Morris
Quoted 6 lines · H. Peter Anvin
Quoted 3 lines · James Morris
Please make
NO_BOOTMEMdefault to n, at least for amd64, where I've found that it leads to all kinds of strange, undebuggable boot hangs and errors (with relatively current Fedora development userland).Have you tested it with the latest fixes that are now in Linus' tree (-rc3)?
Yes, it was happening with -rc3.
please send out bootlog if possible.
BTW please trygit://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git
Thanks
Yinghai
Quoted 11 lines · James Morris
Quoted 7 lines · H. Peter Anvin
Quoted 4 lines · James Morris
Please make
NO_BOOTMEMdefault to n, at least for amd64, where I've found that it leads to all kinds of strange, undebuggable boot hangs and errors (with relatively current Fedora development userland).Have you tested it with the latest fixes that are now in Linus' tree (-rc3)?
Yes, it was happening with -rc3.
Could you please send the bootlog that Yinghai asked for, plus also one that you get with NO_BOOTMEM turned off (for comparison)?
Also, when did you first hit this bug? This code has been upstream for almost a month, and it was in linux-next before that - so you should have hit this much sooner. A rough timeframe would suffice. I suppose you were booting upstream kernels during the merge window as well?
We can flip the default around if there's no fix available based on the bootlogs. (Plus the help text should definitely be improved.)
Thanks,
Ingo
Quoted 15 lines · Ingo Molnar
Quoted 11 lines · James Morris
Quoted 7 lines · H. Peter Anvin
Quoted 4 lines · James Morris
Please make
NO_BOOTMEMdefault to n, at least for amd64, where I've found that it leads to all kinds of strange, undebuggable boot hangs and errors (with relatively current Fedora development userland).Have you tested it with the latest fixes that are now in Linus' tree (-rc3)?
Yes, it was happening with -rc3.
Quoted 7 lines
Could you please send the bootlog that Yinghai asked for, plus also one that you get with
NO_BOOTMEMturned off (for comparison)?Also, when did you first hit this bug? This code has been upstream for almost a month, and it was in linux-next before that - so you should have hit this much sooner. A rough timeframe would suffice. I suppose you were booting upstream kernels during the merge window as well?
A default y config option causing regressions still at rc3? and you guys keep going? This is the sort of shit Linus would flame me for a day or two for,
Can we get some f'ing consistency here?
Dave.
Quoted 13 lines
We can flip the default around if there's no fix available based on the bootlogs. (Plus the help text should definitely be improved.)
Thanks,
Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 5 lines · Dave Airlie
A default y config option causing regressions still at rc3? and you guys keep going? This is the sort of shit Linus would flame me for a day or two for,
Can we get some f'ing consistency here?
Yeah. I think we need to remove the crap.
I thought the problems were known, and fixed in -rc3. Clearly they weren't. And by now it's not about changing the default any more - by now it's about removing the known-crap code.
Linus
Quoted 4 lines · Linus Torvalds
Quoted 3 lines · Dave Airlie
A default y config option causing regressions still at rc3? [...]
Quoted 8 lines
Quoted 4 lines
[...] and you guys keep going? This is the sort of shit Linus would flame me for a day or two for,
Can we get some f'ing consistency here?
Yeah. I think we need to remove the crap.
I thought the problems were known, and fixed in -rc3. Clearly they weren't.
Yeah.
It would still be nice to get the before/after bootlogs, because we'd like to map out any remaining bugs.
Quoted 2 lines
And by now it's not about changing the default any more - by now it's about removing the known-crap code.
Ok, we can certainly do that too.
Should we scrap the whole x86 bootmem conversion to begin with? I'm not sure there's any fundamentally less risky way to it so if we try this again in .35 we might run into similar regressions and i'd like to avoid that. I wouldnt mind not having to do that at all, it's been a lot of pain to pull it off and the lmb conversion looks even more intrusive.
Thanks,
Ingo
Quoted 24 lines · Dave Airlie
Quoted 15 lines · Ingo Molnar
Quoted 11 lines · James Morris
Quoted 7 lines · H. Peter Anvin
Quoted 4 lines · James Morris
Please make
NO_BOOTMEMdefault to n, at least for amd64, where I've found that it leads to all kinds of strange, undebuggable boot hangs and errors (with relatively current Fedora development userland).Have you tested it with the latest fixes that are now in Linus' tree (-rc3)?
Yes, it was happening with -rc3.
Quoted 7 lines
Could you please send the bootlog that Yinghai asked for, plus also one that you get with
NO_BOOTMEMturned off (for comparison)?Also, when did you first hit this bug? This code has been upstream for almost a month, and it was in linux-next before that - so you should have hit this much sooner. A rough timeframe would suffice. I suppose you were booting upstream kernels during the merge window as well?
Quoted 5 lines
A default y config option causing regressions still at rc3? and you guys keep going? This is the sort of shit Linus would flame me for a day or two for,
Can we get some f'ing consistency here?
Note, without trying to defend the bootmem conversion itself, which didnt work out well, this is not some optional new driver feature that was default-y randomly but it was an infrastructure change that was to be made unconditional in .35.
The flag was basically a testing/debug flag to allow the old code to be used too, in case the new code was buggy. This is what helped James to report this today, instead of forcing James through a very difficult ~14-reboot bisection.
Thanks,
Ingo
Quoted 18 lines · Ingo Molnar
Quoted 11 lines · James Morris
Quoted 7 lines · H. Peter Anvin
Quoted 4 lines · James Morris
Please make
NO_BOOTMEMdefault to n, at least for amd64, where I've found that it leads to all kinds of strange, undebuggable boot hangs and errors (with relatively current Fedora development userland).Have you tested it with the latest fixes that are now in Linus' tree (-rc3)?
Yes, it was happening with -rc3.
Could you please send the bootlog that Yinghai asked for, plus also one that you get with
NO_BOOTMEMturned off (for comparison)?
Quoted 8 lines
Also, when did you first hit this bug? This code has been upstream for almost a month, and it was in linux-next before that - so you should have hit this much sooner. A rough timeframe would suffice. I suppose you were booting upstream kernels during the merge window as well?
We can flip the default around if there's no fix available based on the bootlogs. (Plus the help text should definitely be improved.)
Are you testing this btw with initramfs/initrds? I suspect lots of testing is being done by people on monolithic kernels, this is just a misc guess, considering I couldn't boot from when this landed until rc3 with this option on a basic 32-bit install on a dual-core 64-bit CPU, it suggested a hole of some sort
in the test coverage.
Dave
Quoted 28 lines · Dave Airlie
Quoted 18 lines · Ingo Molnar
Quoted 11 lines · James Morris
Quoted 7 lines · H. Peter Anvin
Quoted 4 lines · James Morris
Please make
NO_BOOTMEMdefault to n, at least for amd64, where I've found that it leads to all kinds of strange, undebuggable boot hangs and errors (with relatively current Fedora development userland).Have you tested it with the latest fixes that are now in Linus' tree (-rc3)?
Yes, it was happening with -rc3.
Could you please send the bootlog that Yinghai asked for, plus also one that you get with
NO_BOOTMEMturned off (for comparison)?Quoted 8 lines
Also, when did you first hit this bug? This code has been upstream for almost a month, and it was in linux-next before that - so you should have hit this much sooner. A rough timeframe would suffice. I suppose you were booting upstream kernels during the merge window as well?
We can flip the default around if there's no fix available based on the bootlogs. (Plus the help text should definitely be improved.)
Quoted 6 lines
Are you testing this btw with initramfs/initrds? I suspect lots of testing is being done by people on monolithic kernels, this is just a misc guess, considering I couldn't boot from when this landed until rc3 with this option on a basic 32-bit install on a dual-core 64-bit CPU, it suggested a hole of some sort
in the test coverage.
so -rc3 is working your setup?
Yinghai
H. Peter Anvin
reviewerQuoted 6 lines · Dave Airlie
Are you testing this btw with initramfs/initrds? I suspect lots of testing is being done by people on monolithic kernels, this is just a misc guess, considering I couldn't boot from when this landed until rc3 with this option on a basic 32-bit install on a dual-core 64-bit CPU, it suggested a hole of some sort
in the test coverage.
Hi Dave,
The only bug report I remember getting from you had no details and was in reply to another bug report which was, indeed, addressed, so we had every reason to believe it was being dealt with with the patchset which did indeed go into -rc3 (and does address a problem with initramfs in particular cases.)
Clearly James Morris' problem is something unrelated, and regardless of course of action we need to track it down.
If you also are having problems with -rc3 we would really appreciate as much detail as possible -- boot logs at the very minimum -- so we have a chance to at all track down the problems that do exist.
-hpa
Quoted 5 lines · Ingo Molnar
Quoted 2 lines
Yes, it was happening with -rc3.
Could you please send the bootlog that Yinghai asked for, plus also one that you get with
NO_BOOTMEMturned off (for comparison)?
I don't have the old boot logs, and have since upgraded the system further.
IIRC, the boot was failing after not being able to find the root fs (ext3/lvm/raid0). I thought it was a dracut issue, but it seemed to be fixed by enabling bootmem.
Quoted 4 lines
Also, when did you first hit this bug? This code has been upstream for almost a month, and it was in linux-next before that - so you should have hit this much sooner. A rough timeframe would suffice. I suppose you were booting upstream kernels during the merge window as well?
In this case, in the last few days (also when I first saw or noticed the bootmem option). I was booting relatively recent linus kernels during the merge window, although my main work was being done on an older upstream kernel.
Signature · 2 lines
James Morris <[email protected]>
Quoted 10 lines · James Morris
Quoted 5 lines · Ingo Molnar
Quoted 2 lines
Yes, it was happening with -rc3.
Could you please send the bootlog that Yinghai asked for, plus also one that you get with
NO_BOOTMEMturned off (for comparison)?I don't have the old boot logs, and have since upgraded the system further.
Please, could you send any bootlog then that we could work from? That way we could check the memory layout and guess the rough shape of the early allocations, etc.
Quoted 3 lines
IIRC, the boot was failing after not being able to find the root fs (ext3/lvm/raid0). I thought it was a dracut issue, but it seemed to be fixed by enabling bootmem.
Ok - initrd unpack failing or initial mount failing is consistent with the initrd getting corrupted by overlapping early reservations due to allocator bug.
Quoted 9 lines
Quoted 4 lines
Also, when did you first hit this bug? This code has been upstream for almost a month, and it was in linux-next before that - so you should have hit this much sooner. A rough timeframe would suffice. I suppose you were booting upstream kernels during the merge window as well?
In this case, in the last few days (also when I first saw or noticed the bootmem option). I was booting relatively recent linus kernels during the merge window, although my main work was being done on an older upstream kernel.
Ok, so it's not an old regression but possibly a bug in one of the fixes. Not good.
Ingo
H. Peter Anvin
reviewerQuoted 11 lines · James Morris
Quoted 5 lines · Ingo Molnar
Quoted 2 lines
Yes, it was happening with -rc3.
Could you please send the bootlog that Yinghai asked for, plus also one that you get with
NO_BOOTMEMturned off (for comparison)?I don't have the old boot logs, and have since upgraded the system further.
Upgraded how? The problem no longer happens?
Quoted 3 lines
IIRC, the boot was failing after not being able to find the root fs (ext3/lvm/raid0). I thought it was a dracut issue, but it seemed to be fixed by enabling bootmem.
This would rather match the problem that was addressed by the patch in -rc3. Any help in reproducing it would be great.
-hpa
Quoted 6 lines · H. Peter Anvin
Quoted 5 lines · James Morris
Quoted 3 lines · Ingo Molnar
Quoted 2 lines
Yes, it was happening with -rc3.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Could you please send the bootlog that Yinghai asked for, plus also one that you get with
NO_BOOTMEMturned off (for comparison)?I don't have the old boot logs, and have since upgraded the system further.
Upgraded how? The problem no longer happens?
Upgraded to the latest rawhide userland -- I have not since tested with bootmem off. I'll try and do so again when I get a chance.
Quoted 10 lines
Quoted 4 lines
IIRC, the boot was failing after not being able to find the root fs (ext3/lvm/raid0). I thought it was a dracut issue, but it seemed to be fixed by enabling bootmem.
This would rather match the problem that was addressed by the patch in -rc3. Any help in reproducing it would be great.
-hpa
Signature · 2 lines
James Morris <[email protected]>
H. Peter Anvin
reviewerQuoted 6 lines · James Morris
Quoted 2 lines
Upgraded how? The problem no longer happens?
Upgraded to the latest rawhide userland -- I have not since tested with bootmem off. I'll try and do so again when I get a chance.
That would be great. The sooner the better, obviously.
-hpa
Quoted 9 lines · H. Peter Anvin
Quoted 6 lines · James Morris
Quoted 2 lines
Upgraded how? The problem no longer happens?
Upgraded to the latest rawhide userland -- I have not since tested with bootmem off. I'll try and do so again when I get a chance.
That would be great. The sooner the better, obviously.
I'm not seeing any problems now, with current Linus and rawhide. I'll leave bootmem off and see if anything comes up again.
Signature · 2 lines
James Morris <[email protected]>
Quoted 6 lines · James Morris
Quoted 4 lines · H. Peter Anvin
Quoted 3 lines · James Morris
Quoted 2 lines
Upgraded how? The problem no longer happens?
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Upgraded to the latest rawhide userland -- I have not since tested with bootmem off. I'll try and do so again when I get a chance.
That would be great. The sooner the better, obviously.
I'm not seeing any problems now, with current Linus and rawhide. I'll leave bootmem off and see if anything comes up again.
(a current bootlog would still be nice)
Dave, can you reproduce any of these problems with Linus's latest?
Ingo
Quoted 12 lines · Ingo Molnar
Quoted 10 lines · James Morris
Quoted 8 lines · H. Peter Anvin
Quoted 6 lines · James Morris
Quoted 2 lines
Upgraded how? The problem no longer happens?
Upgraded to the latest rawhide userland -- I have not since tested with bootmem off. I'll try and do so again when I get a chance.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
That would be great. The sooner the better, obviously.
I'm not seeing any problems now, with current Linus and rawhide. I'll leave bootmem off and see if anything comes up again.
(a current bootlog would still be nice)
Dave, can you reproduce any of these problems with Linus's latest?
ping? Can you or Dave reproduce the bug with -rc3 or later kernels? (If not then it probably means that the bug you triggered was already fixed at the time you reported it, as hpa suspected.)
Thanks,
Ingo
Quoted 20 lines · Ingo Molnar
Quoted 17 lines · Ingo Molnar
Quoted 14 lines · James Morris
Quoted 9 lines · H. Peter Anvin
Quoted 6 lines · James Morris
Quoted 2 lines
Upgraded how? The problem no longer happens?
Upgraded to the latest rawhide userland -- I have not since tested with bootmem off. I'll try and do so again when I get a chance.
That would be great. The sooner the better, obviously.
I'm not seeing any problems now, with current Linus and rawhide. I'll leave bootmem off and see if anything comes up again.
Quoted 7 lines
Quoted 3 lines
(a current bootlog would still be nice)
Dave, can you reproduce any of these problems with Linus's latest?
ping? Can you or Dave reproduce the bug with -rc3 or later kernels? (If not then it probably means that the bug you triggered was already fixed at the time you reported it, as hpa suspected.)
James already reported -rc3 fix the problem for him.
Dave implied -rc3 fixed problem for him
Thanks
Yinghai
Quoted 25 lines · Yinghai
Quoted 24 lines · Ingo Molnar
Quoted 20 lines · Ingo Molnar
Quoted 6 lines · James Morris
Quoted 4 lines · H. Peter Anvin
Quoted 3 lines · James Morris
Quoted 2 lines
Upgraded how? The problem no longer happens?
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Upgraded to the latest rawhide userland -- I have not since tested with bootmem off. I'll try and do so again when I get a chance.
That would be great. The sooner the better, obviously.
I'm not seeing any problems now, with current Linus and rawhide. I'll leave bootmem off and see if anything comes up again.
(a current bootlog would still be nice)
Dave, can you reproduce any of these problems with Linus's latest?
Quoted 7 lines
Quoted 3 lines
ping? Can you or Dave reproduce the bug with -rc3 or later kernels? (If not then it probably means that the bug you triggered was already fixed at the time you reported it, as hpa suspected.)
James already reported -rc3 fix the problem for him.
Dave implied -rc3 fixed problem for him
Hm, i'm confused, does this mean that it was all fixed upstream already when Dave and James sent their complaints?
Would be nice to have a confirmation from Dave for that (beyond 'implying' it), to not keep this thread open-ended.
Thanks,
Ingo
Quoted 32 lines · Ingo Molnar
Quoted 29 lines · Yinghai
Quoted 27 lines · Ingo Molnar
Quoted 12 lines · Ingo Molnar
Quoted 10 lines · James Morris
Quoted 8 lines · H. Peter Anvin
Quoted 6 lines · James Morris
Quoted 2 lines
Upgraded how? The problem no longer happens?
Upgraded to the latest rawhide userland -- I have not since tested with bootmem off. I'll try and do so again when I get a chance.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
That would be great. The sooner the better, obviously.
I'm not seeing any problems now, with current Linus and rawhide. I'll leave bootmem off and see if anything comes up again.
(a current bootlog would still be nice)
Dave, can you reproduce any of these problems with Linus's latest?
ping? Can you or Dave reproduce the bug with -rc3 or later kernels? (If not then it probably means that the bug you triggered was already fixed at the time you reported it, as hpa suspected.)
Quoted 6 lines
Quoted 3 lines
James already reported -rc3 fix the problem for him.
Dave implied -rc3 fixed problem for him
Hm, i'm confused, does this mean that it was all fixed upstream already when Dave and James sent their complaints?
When I reported it, it was only at rc2 stage so not fixed upstream at all.
Quoted 3 lines
Would be nice to have a confirmation from Dave for that (beyond 'implying' it), to not keep this thread open-ended.
Okay I built a linus head and it booted on the previously broken machine. with CONFIG_NO_BOOTMEM=y
Dave.
Quoted 3 lines · Ingo Molnar
ping? Can you or Dave reproduce the bug with -rc3 or later kernels? (If not then it probably means that the bug you triggered was already fixed at the time you reported it, as hpa suspected.)
I haven't seen it since.
- James
Signature · 2 lines
James Morris <[email protected]>
Quoted 7 lines · James Morris
Quoted 3 lines · Ingo Molnar
ping? Can you or Dave reproduce the bug with -rc3 or later kernels? (If not then it probably means that the bug you triggered was already fixed at the time you reported it, as hpa suspected.)
I haven't seen it since.
Great, thanks!
Ingo
Quoted 10 lines · James Morris
Quoted 6 lines · H. Peter Anvin
Quoted 3 lines · James Morris
Please make
NO_BOOTMEMdefault to n, at least for amd64, where I've found that it leads to all kinds of strange, undebuggable boot hangs and errors (with relatively current Fedora development userland).Have you tested it with the latest fixes that are now in Linus' tree (-rc3)?
Yes, it was happening with -rc3.
in case, you have one 32bit system without RAM installed on node0. please check
Thanks
Yinghai
Subject: [PATCH] x86: Fix 32bit system without RAM on Node0
when 32bit numa is used, free_all_bootmem() will still only go over with node id 0.
If node 0 doesn't have RAM installed, We need to go with node1 because early_node_map still use 1 for all ranges, and ram from node1 becom low ram.
Try to use MAX_NUMNODES like 64 numa does.
--- arch/x86/mm/init_32.c | 5 +++++ 1 file changed, 5 insertions(+)
arch/x86/mm/init_32.c +5 -0
@@ -875,7 +875,12 @@ void __init mem_init(void)BUG_ON (!mem_map);#endif /* this will put all low memory onto the freelists */ +#if defined(CONFIG_NO_BOOTMEM ) && defined(MAX_NUMNODES ) +/* In case some 32bit systems don't have RAM installed on node0 */ + totalram_pages +=free_all_memory_core_early (MAX_NUMNODES ); +#else totalram_pages +=free_all_bootmem (); +#endif reservedpages =0 ;for (tmp =0 ; tmp < max_low_pfn; tmp++)
arch/x86/mm/init_32.c void __init mem_init(void) · line 880
Quoted 9 lines · Yinghai Lu
arch/x86/mm/init_32.c +3
-0@@ -875,7 +875,12 @@ void __init mem_init(void)BUG_ON (!mem_map);#endif /* this will put all low memory onto the freelists */ +#if defined(CONFIG_NO_BOOTMEM ) && defined(MAX_NUMNODES ) +/* In case some 32bit systems don't have RAM installed on node0 */ + totalram_pages +=free_all_memory_core_early (MAX_NUMNODES );
(Note: tab whitespace damage)
arch/x86/mm/init_32.c void __init mem_init(void) · line 882
Quoted 2 lines
arch/x86/mm/init_32.c +1
-0+#else totalram_pages +=free_all_bootmem ();
So we get into this branch if CONFIG_NO_BOOTMEM is enabled but MAX_NUMNODES is not defined? Doesnt look right.
arch/x86/mm/init_32.c void __init mem_init(void) · line 883
Quoted 1 line
arch/x86/mm/init_32.c +1
-0+#endif
Btw., and i said this before, i absolutely hate the CONFIG_NO_BOOTMEM naming as well (a negative in the option), but it is was what expresses the 'this is where we want to go' state better and thus CONFIG_NO_BOOTMEM removal will be a straight removal instead of a removal of the inverse.
Thanks,
Ingo
Quoted 15 lines · Ingo Molnar
arch/x86/mm/init_32.c void __init mem_init(void) · line 880
Quoted 9 lines · Yinghai Lu
arch/x86/mm/init_32.c +3
-0@@ -875,7 +875,12 @@ void __init mem_init(void)BUG_ON (!mem_map);#endif /* this will put all low memory onto the freelists */ +#if defined(CONFIG_NO_BOOTMEM ) && defined(MAX_NUMNODES ) +/* In case some 32bit systems don't have RAM installed on node0 */ + totalram_pages +=free_all_memory_core_early (MAX_NUMNODES );(Note: tab whitespace damage)
Quoted 5 lines
arch/x86/mm/init_32.c void __init mem_init(void) · line 882
Quoted 2 lines
arch/x86/mm/init_32.c +1
-0+#else totalram_pages +=free_all_bootmem ();So we get into this branch if
CONFIG_NO_BOOTMEMis enabled butMAX_NUMNODESis not defined? Doesnt look right.
yes.
free_all_bootmem() will call free_all_memory_core_early(NODE_DATA(0)->node_id);
Thanks
Yinghai Lu
Quoted 22 lines · Yinghai Lu
Quoted 15 lines · Ingo Molnar
arch/x86/mm/init_32.c void __init mem_init(void) · line 880
Quoted 9 lines · Yinghai Lu
arch/x86/mm/init_32.c +3
-0@@ -875,7 +875,12 @@ void __init mem_init(void)BUG_ON (!mem_map);#endif /* this will put all low memory onto the freelists */ +#if defined(CONFIG_NO_BOOTMEM ) && defined(MAX_NUMNODES ) +/* In case some 32bit systems don't have RAM installed on node0 */ + totalram_pages +=free_all_memory_core_early (MAX_NUMNODES );(Note: tab whitespace damage)
Quoted 5 lines
arch/x86/mm/init_32.c void __init mem_init(void) · line 882
Quoted 2 lines
arch/x86/mm/init_32.c +1
-0+#else totalram_pages +=free_all_bootmem ();So we get into this branch if
CONFIG_NO_BOOTMEMis enabled butMAX_NUMNODESis not defined? Doesnt look right.
Quoted 6 lines
yes.
free_all_bootmem()will call free_all_memory_core_early(NODE_DATA(0)->node_id);Thanks
Well and that whole #ifdeffery is disgusting as well - even if the goal was to remove CONFIG_NO_BOOTMEM ASAP.
Please learn to use proper intermediate helper functions and at minimum put the conversion ugliness somewhere that doesnt intrude our daily flow in .c files. The best rule is to _never ever_ put an #ifdef construct into a .c file. It doesnt matter what the goal if the #ifdef is - such ugliness in code is never justified.
Thanks,
Ingo
Quoted 32 lines · Ingo Molnar
Quoted 22 lines · Yinghai Lu
Quoted 15 lines · Ingo Molnar
arch/x86/mm/init_32.c void __init mem_init(void) · line 880
Quoted 9 lines · Yinghai Lu
arch/x86/mm/init_32.c +3
-0@@ -875,7 +875,12 @@ void __init mem_init(void)BUG_ON (!mem_map);#endif /* this will put all low memory onto the freelists */ +#if defined(CONFIG_NO_BOOTMEM ) && defined(MAX_NUMNODES ) +/* In case some 32bit systems don't have RAM installed on node0 */ + totalram_pages +=free_all_memory_core_early (MAX_NUMNODES );(Note: tab whitespace damage)
Quoted 5 lines
arch/x86/mm/init_32.c void __init mem_init(void) · line 882
Quoted 2 lines
arch/x86/mm/init_32.c +1
-0+#else totalram_pages +=free_all_bootmem ();So we get into this branch if
CONFIG_NO_BOOTMEMis enabled butMAX_NUMNODESis not defined? Doesnt look right.Quoted 6 lines
yes.
free_all_bootmem()will call free_all_memory_core_early(NODE_DATA(0)->node_id);Thanks
Quoted 9 lines
Well and that whole #ifdeffery is disgusting as well - even if the goal was to remove
CONFIG_NO_BOOTMEMASAP.Please learn to use proper intermediate helper functions and at minimum put the conversion ugliness somewhere that doesnt intrude our daily flow in .c files. The best rule is to
_neverever_put an #ifdef construct into a .c file. It doesnt matter what the goal if the #ifdef is - such ugliness in code is never justified.
if you agree that i can have one nobootmem.c in mm/
Thanks
Yinghai
Quoted 36 lines · Yinghai Lu
Quoted 35 lines · Ingo Molnar
Quoted 22 lines · Yinghai Lu
Quoted 15 lines · Ingo Molnar
arch/x86/mm/init_32.c void __init mem_init(void) · line 880
Quoted 9 lines · Yinghai Lu
arch/x86/mm/init_32.c +3
-0@@ -875,7 +875,12 @@ void __init mem_init(void)BUG_ON (!mem_map);#endif /* this will put all low memory onto the freelists */ +#if defined(CONFIG_NO_BOOTMEM ) && defined(MAX_NUMNODES ) +/* In case some 32bit systems don't have RAM installed on node0 */ + totalram_pages +=free_all_memory_core_early (MAX_NUMNODES );(Note: tab whitespace damage)
Quoted 5 lines
arch/x86/mm/init_32.c void __init mem_init(void) · line 882
Quoted 2 lines
arch/x86/mm/init_32.c +1
-0+#else totalram_pages +=free_all_bootmem ();So we get into this branch if
CONFIG_NO_BOOTMEMis enabled butMAX_NUMNODESis not defined? Doesnt look right.Quoted 6 lines
yes.
free_all_bootmem()will call free_all_memory_core_early(NODE_DATA(0)->node_id);Thanks
Well and that whole #ifdeffery is disgusting as well - even if the goal was to remove
CONFIG_NO_BOOTMEMASAP.
Quoted 8 lines
Quoted 6 lines
Please learn to use proper intermediate helper functions and at minimum put the conversion ugliness somewhere that doesnt intrude our daily flow in .c files. The best rule is to
_neverever_put an #ifdef construct into a .c file. It doesnt matter what the goal if the #ifdef is - such ugliness in code is never justified.if you agree that i can have one
nobootmem.cinmm/
I think what we want is your lmb series, with CONFIG_NO_BOOTMEM eliminated altogether and x86 converted to pure (extended) lmb facilities, and without any traces of bootmem left in x86.
I.e. a really clean series with no CONFIG_NO_BOOTMEM kind of #ifdef crap left around. This means 'nobootmem.c' (albeit saner than an #ifdef jungle) would be moot as well.
We tried the dual model as it seemed prudent from a testing/conversion POV (and it certainly allowed people to turn the new code off), but it's rather ugly and we still have bugs left.
This means that if Linus likes that approach the conversion will be very binary and very painful. The other option would be to go back to bootmem and forget about the whole nobootmem and lmb thing.
Ingo
Quoted 3 lines · Ingo Molnar
I think what we want is your lmb series, with
CONFIG_NO_BOOTMEMeliminated altogether and x86 converted to pure (extended) lmb facilities, and without any traces of bootmem left in x86.
That does not make much sense as bootmem is not only used on the architecture side but also in generic code. So you either have to emulate the API on x86 or get lmb in a state to replace bootmem on _all_ architectures.
Quoted 7 lines
I.e. a really clean series with no
CONFIG_NO_BOOTMEMkind of #ifdef crap left around. This means 'nobootmem.c' (albeit saner than an #ifdef jungle) would be moot as well.We tried the dual model as it seemed prudent from a testing/conversion POV (and it certainly allowed people to turn the new code off), but it's rather ugly and we still have bugs left.
I think this was an implementation thing rather than a problem with the model per se.
As written above, you can hardly get away without emulating the bootmem API during transition.
Quoted 3 lines
This means that if Linus likes that approach the conversion will be very binary and very painful. The other option would be to go back to bootmem and forget about the whole nobootmem and lmb thing.
I suppose it would be safest to replace early_res with lmb first to get in sync with the other archs using it.
Step two would be to extend LMB and implement a bootmem emulation API on top of it so that architectures can switch over to non-bootmem mode one by one. Then you can drop the real bootmem code and switch generic code to use LMB natively, also site by site. And finally, drop the emulation API.
If other architectures object to removing bootmem, there really is no point for x86 to even try it.
For step one to work out, it's probably easiest to fully revert to the .33 state than having to replace early_res while in its current state?
H. Peter Anvin
reviewerQuoted 4 lines · Yinghai Lu
Quoted 4 lines
Well and that whole #ifdeffery is disgusting as well - even if the goal was to remove
CONFIG_NO_BOOTMEMASAP.
Quoted 8 lines
Quoted 5 lines
Please learn to use proper intermediate helper functions and at minimum put the conversion ugliness somewhere that doesnt intrude our daily flow in .c files. The best rule is to
_neverever_put an #ifdef construct into a .c file. It doesnt matter what the goal if the #ifdef is - such ugliness in code is never justified.if you agree that i can have one
nobootmem.cinmm/
That would be better, or more commonly, use inlines.
I'm still totally puzzled about this patch as well as the comment:
+#if defined(CONFIG_NO_BOOTMEM) && defined(MAX_NUMNODES)
+ /* In case some 32bit systems don't have RAM installed on node0 */
+ totalram_pages += free_all_memory_core_early(MAX_NUMNODES);
+#else
totalram_pages += free_all_bootmem();
+#endif
Why is that "32 bits" specific? Second, MAX_NUMNODES is defined whenever <linux/numa.h> is included, so what on Earth is this supposed to signify? Are you trying to say MAX_NUMNODES > 1? Or are you trying to say CONFIG_NUMA?
Furthermore, I really don't see the connection between this and James Morris' reported problem, which he reports as "amd64", which presumably is an x86-64 kernel and not 32 bits... James, is that correct? Any more details you can give about the system? I *really* don't want to go into cargo cult programming mode, that would suck eggs no matter what.
-hpa
Quoted 26 lines · H. Peter Anvin
Quoted 4 lines · Yinghai Lu
Quoted 4 lines
Well and that whole #ifdeffery is disgusting as well - even if the goal was to remove
CONFIG_NO_BOOTMEMASAP.Quoted 8 lines
Quoted 5 lines
Please learn to use proper intermediate helper functions and at minimum put the conversion ugliness somewhere that doesnt intrude our daily flow in .c files. The best rule is to
_neverever_put an #ifdef construct into a .c file. It doesnt matter what the goal if the #ifdef is - such ugliness in code is never justified.if you agree that i can have one
nobootmem.cinmm/That would be better, or more commonly, use inlines.
I'm still totally puzzled about this patch as well as the comment:
(quoted diff) +5
-0+#if defined(CONFIG_NO_BOOTMEM) && defined(MAX_NUMNODES) + /* In case some 32bit systems don't have RAM installed on node0 */ + totalram_pages += free_all_memory_core_early(MAX_NUMNODES); +#else totalram_pages += free_all_bootmem(); +#endif
Quoted 4 lines
Why is that "32 bits" specific? Second,
MAX_NUMNODESis defined whenever <linux/numa.h> is included, so what on Earth is this supposed to signify? Are you trying to sayMAX_NUMNODES> 1? Or are you trying to sayCONFIG_NUMA?
you are right, this one should be more clear.
Subject: [PATCH -v2] nobootmem, x86: Fix 32bit system without RAM on Node0
when 32bit numa is used, free_all_bootmem() will still only go over with node id 0.
If node 0 doesn't have RAM installed, We need to go with node1 because early_node_map still use 1 for all ranges, and ram from node1 becom low ram.
Try to use MAX_NUMNODES like 64 numa does.
--- mm/bootmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/bootmem.c +1 -1
@@ -303,7 +303,7 @@ unsigned long __init free_all_bootmem_nounsigned long __init free_all_bootmem (void ) {#ifdef CONFIG_NO_BOOTMEM -return free_all_memory_core_early (NODE_DATA (0 )->node_id); +return free_all_memory_core_early (MAX_NUMNODES );#else return free_all_bootmem_core (NODE_DATA (0 )->bdata);#endif
Quoted 6 lines
Furthermore, I really don't see the connection between this and James Morris' reported problem, which he reports as "amd64", which presumably is an x86-64 kernel and not 32 bits... James, is that correct? Any more details you can give about the system? I *really* don't want to go into cargo cult programming mode, that would suck eggs no matter what.
it happened one of my test setup, node0 ram disappear somehow. and i found the 32bit numa doesn't work on that.
Thanks
Yinghai
H. Peter Anvin
reviewerQuoted 20 lines · Yinghai Lu
--- mm/bootmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Index:
linux-2.6/mm/bootmem.c
===================================================================mm/bootmem.c +1
-1@@ -303,7 +303,7 @@ unsigned long __init free_all_bootmem_nounsigned long __init free_all_bootmem (void ) {#ifdef CONFIG_NO_BOOTMEM -return free_all_memory_core_early (NODE_DATA (0 )->node_id); +return free_all_memory_core_early (MAX_NUMNODES );#else return free_all_bootmem_core (NODE_DATA (0 )->bdata);#endif Quoted 2 lines
Quoted 9 lines
Quoted 5 lines
Furthermore, I really don't see the connection between this and James Morris' reported problem, which he reports as "amd64", which presumably is an x86-64 kernel and not 32 bits... James, is that correct? Any more details you can give about the system? I *really* don't want to go into cargo cult programming mode, that would suck eggs no matter what.
it happened one of my test setup, node0 ram disappear somehow. and i found the 32bit numa doesn't work on that.
... which is useful and valid, but I still think this isn't related to James' problem, if James' problem wasn't actually fixed in -rc3. That's the part that I'm afraid I have to be confused about... all the known problems except the above are fixed in -rc3, and I'd at least like to have a validated bug report of any sort before saying it should all be tossed.
This patch looks a lot better. The whole use of MAX_NUMNODES as a sentinel (which appears inherited from mm/page_alloc.c, and as such is a pre-existing convention which is also invoked here) really could use a comment, though.
-hpa
Quoted 6 lines · H. Peter Anvin
On 03/31/2010 04:54 PM, Yinghai Lu wrote:
This patch looks a lot better. The whole use of
MAX_NUMNODESas a sentinel (which appears inherited frommm/page_alloc.c, and as such is a pre-existing convention which is also invoked here) really could use a comment, though.
sure. will have updated one with coments there
Thanks
Yinghai
PATCH 1/1 nobootmem/bootmem, x86: Fix 32bit numa system without RAM on Node0
In reply to H. Peter Anvinon one system without RAM on nod0, got following dump with 32bit numa kernel
early_node_map[4] active PFN ranges
1: 0x00000010 -> 0x00000099
1: 0x00000100 -> 0x0007da00
1: 0x0007e800 -> 0x0007ffa0
1: 0x0007ffae -> 0x0007ffb0Subtract (29 early reservations)
#000 [0000001000 - 0000002000]
#001 [0000089000 - 000008f000]
#002 [0000091000 - 0000093500]
#003 [0000094000 - 0000099000]
#004 [0000099400 - 0000100000]
#005 [0000200000 - 0000eb7644]
#006 [0000eb8000 - 0000ec327c]
#007 [007c400000 - 007c40e000]
#008 [007c440000 - 007c44e000]
#009 [007c480000 - 007c48e000]
#010 [007c4c0000 - 007c4ce000]
#011 [007c500000 - 007c50e000]
#012 [007c540000 - 007c54e000]
#013 [007c580000 - 007c58e000]
#014 [007c5c0000 - 007c5ce000]
#015 [007c674000 - 007cbfe000]
#016 [007cbfe500 - 007cbfe530]
#017 [007cbfe540 - 007cbfe5d0]
#018 [007cbfe600 - 007cbfe620]
#019 [007cbfe640 - 007cbfe660]
#020 [007cbfe680 - 007cbfe684]
#021 [007cbfe6c0 - 007cbfe6c4]
#022 [007cbfe700 - 007cbfe77e]
#023 [007cbfe780 - 007cbfe7fe]
#024 [007cbfe800 - 007cbfec54]
#025 [007cbfec80 - 007cbfeede]
#026 [007cbfef00 - 007cbfef2d]
#027 [007cbfef40 - 007e800000]
#028 [007e9ca000 - 007ff95000]
(0 free memory ranges)
Initializing HighMem for node 0 (00000000:00000000)
Initializing HighMem for node 1 (00000000:00000000)
Memory: 0k/2096832k available (6662k kernel code, 2096300k reserved, 4829k data, 484k init, 0k highmem) virtual kernel memory layout:
fixmap : 0xff637000 - 0xfffff000 (10016 kB)
pkmap : 0xff200000 - 0xff400000 (2048 kB)
vmalloc : 0xc07b0000 - 0xff1fe000 (1002 MB)
lowmem : 0x40000000 - 0xbffb0000 (2047 MB)
.init : 0x40d39000 - 0x40db2000 ( 484 kB)
.data : 0x40881924 - 0x40d38e1c (4829 kB)
.text : 0x40200000 - 0x40881924 (6662 kB)
Checking if this processor honours the WP bit even in supervisor mode...Ok.
swapper: page allocation failure. order:0, mode:0x0
Pid: 0, comm: swapper Not tainted 2.6.34-rc3-tip-03818-g4b1ea6c-dirty #35
Call Trace:
[<4087a5dc>] ? printk+0xf/0x11
[<40286728>] __alloc_pages_nodemask+0x417/0x487
[<402a9ce1>] new_slab+0xe2/0x1fe
[<402aa5b2>] kmem_cache_open+0x185/0x358
[<402abbc0>] T.954+0x1c/0x60
[<40d52a29>] kmem_cache_init+0x24/0x113
[<40d39738>] start_kernel+0x166/0x2e4
[<40d3940e>] ? unknown_bootoption+0x0/0x18e
[<40d390ce>] i386_start_kernel+0xce/0xd5Mem-Info:
Node 1 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
Node 1 Normal per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:0 dirty:0 writeback:0 unstable:0
free:0 slab_reclaimable:0 slab_unreclaimable:0
mapped:0 shmem:0 pagetables:0 bounce:0
When 32bit numa is used, free_all_bootmem() will still only go over with node id 0.
If node 0 doesn't have RAM installed, We need to go with node1 because early_node_map still use 1 for all ranges, and ram from node1 become low ram.
Try to use MAX_NUMNODES like 64 numa does.
Also fixes BOOTMEM path by loop bdata_list.
Note: this bug exist before We have NO_BOOTMEM support.
-v3: add more comments, and fix bootmem path too.
--- mm/bootmem.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)
mm/bootmem.c +15 -2
@@ -303,9 +303,22 @@ unsigned long __init free_all_bootmem_nounsigned long __init free_all_bootmem (void ) {#ifdef CONFIG_NO_BOOTMEM -return free_all_memory_core_early (NODE_DATA (0 )->node_id); +/* +* We need to use MAX_NUMNODES instead of NODE_DATA(0)->node_id +* because in some case like Node0 doesnt have RAM installed +* low ram will be on Node1 +* Use MAX_NUMNODES will make sure all ranges in early_node_map[] +* will be used instead of only Node0 related +*/ +return free_all_memory_core_early (MAX_NUMNODES );#else -return free_all_bootmem_core (NODE_DATA (0 )->bdata); +unsigned long total_pages =0 ; +bootmem_data_t *bdata; + +list_for_each_entry (bdata, &bdata_list, list) + total_pages =free_all_bootmem_core (bdata); + +return total_pages;#endif }
H. Peter Anvin
reviewerRe: [PATCH -v3] nobootmem/bootmem, x86: Fix 32bit numa system without RAM on Node0
Please address the separate bug fix in a separate patch.
Quoted 114 lines · Yinghai Lu
on one system without RAM on nod0, got following dump with 32bit numa kernel
early_node_map[4] active PFN ranges
1:0x00000010->0x00000099
1:0x00000100->0x0007da00
1:0x0007e800->0x0007ffa0
1:0x0007ffae->0x0007ffb0Subtract (29 early reservations)
#000 [0000001000 - 0000002000]
#001 [0000089000 -000008f000]
#002 [0000091000 - 0000093500]
#003 [0000094000 - 0000099000]
#004 [0000099400 - 0000100000]
#005 [0000200000 -0000eb7644]
#006 [0000eb8000-0000ec327c]
#007 [007c400000-007c40e000]
#008 [007c440000-007c44e000]
#009 [007c480000-007c48e000]
#010 [007c4c0000-007c4ce000]
#011 [007c500000-007c50e000]
#012 [007c540000-007c54e000]
#013 [007c580000-007c58e000]
#014 [007c5c0000-007c5ce000]
#015 [007c674000-007cbfe000]
#016 [007cbfe500-007cbfe530]
#017 [007cbfe540-007cbfe5d0]
#018 [007cbfe600-007cbfe620]
#019 [007cbfe640-007cbfe660]
#020 [007cbfe680-007cbfe684]
#021 [007cbfe6c0-007cbfe6c4]
#022 [007cbfe700-007cbfe77e]
#023 [007cbfe780-007cbfe7fe]
#024 [007cbfe800-007cbfec54]
#025 [007cbfec80-007cbfeede]
#026 [007cbfef00-007cbfef2d]
#027 [007cbfef40-007e800000]
#028 [007e9ca000-007ff95000]
(0 free memory ranges)
Initializing HighMem for node 0 (00000000:00000000)
Initializing HighMem for node 1 (00000000:00000000)
Memory: 0k/2096832k available (6662k kernel code, 2096300k reserved, 4829k data, 484k init, 0k highmem) virtual kernel memory layout:fixmap : 0xff637000 - 0xfffff000 (10016 kB) pkmap : 0xff200000 - 0xff400000 (2048 kB) vmalloc : 0xc07b0000 - 0xff1fe000 (1002 MB) lowmem : 0x40000000 - 0xbffb0000 (2047 MB) .init : 0x40d39000 - 0x40db2000 ( 484 kB) .data : 0x40881924 - 0x40d38e1c (4829 kB) .text : 0x40200000 - 0x40881924 (6662 kB) Checking if this processor honours the WP bit even in supervisor mode...Ok. swapper: page allocation failure. order:0, mode:0x0 Pid: 0, comm: swapper Not tainted 2.6.34-rc3-tip-03818-g4b1ea6c-dirty #35 Call Trace: [<4087a5dc>] ? printk+0xf/0x11 [<40286728>] __alloc_pages_nodemask+0x417/0x487 [<402a9ce1>] new_slab+0xe2/0x1fe [<402aa5b2>] kmem_cache_open+0x185/0x358 [<402abbc0>] T.954+0x1c/0x60 [<40d52a29>] kmem_cache_init+0x24/0x113 [<40d39738>] start_kernel+0x166/0x2e4 [<40d3940e>] ? unknown_bootoption+0x0/0x18e [<40d390ce>] i386_start_kernel+0xce/0xd5Mem-Info:
Node 1 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
Node 1 Normal per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:0 dirty:0 writeback:0 unstable:0
free:0 slab_reclaimable:0 slab_unreclaimable:0
mapped:0 shmem:0 pagetables:0 bounce:0When 32bit numa is used,
free_all_bootmem()will still only go over with node id 0.If node 0 doesn't have RAM installed, We need to go with node1 because
early_node_mapstill use 1 for all ranges, and ram from node1 become low ram.Try to use
MAX_NUMNODESlike 64 numa does.Also fixes BOOTMEM path by loop
bdata_list.
Note: this bug exist before We haveNO_BOOTMEMsupport.-v3: add more comments, and fix bootmem path too.
Signed-off-by Yinghai Lu <[email protected]>---
mm/bootmem.c| 17 +++++++++++++++--1 file changed, 15 insertions(+), 2 deletions(-)Index:
linux-2.6/mm/bootmem.c
===================================================================mm/bootmem.c +8
-2@@ -303,9 +303,22 @@ unsigned long __init free_all_bootmem_nounsigned long __init free_all_bootmem (void ) {#ifdef CONFIG_NO_BOOTMEM -return free_all_memory_core_early (NODE_DATA (0 )->node_id); +/* +* We need to use MAX_NUMNODES instead of NODE_DATA(0)->node_id +* because in some case like Node0 doesnt have RAM installed +* low ram will be on Node1 +* Use MAX_NUMNODES will make sure all ranges in early_node_map[] +* will be used instead of only Node0 related +*/ +return free_all_memory_core_early (MAX_NUMNODES );#else -return free_all_bootmem_core (NODE_DATA (0 )->bdata);
mm/bootmem.c unsigned long __init free_all_bootmem_no · line 324
Quoted 10 lines
mm/bootmem.c +7
-0+unsigned long total_pages =0 ; +bootmem_data_t *bdata; + +list_for_each_entry (bdata, &bdata_list, list) + total_pages =free_all_bootmem_core (bdata); + +return total_pages;#endif }
--
Sent from my mobile phone, pardon any lack of formatting.
Re: [PATCH -v3] nobootmem/bootmem, x86: Fix 32bit numa system without RAM on Node0
Quoted 1 line · H. Peter Anvin
Please address the separate bug fix in a separate patch.
ok.
PATCH 1/2 nobootmem, x86: Fix 32bit numa system without RAM on Node0
In reply to H. Peter Anvinon one system without RAM on nod0, got following dump with 32bit numa kernel
early_node_map[4] active PFN ranges
1: 0x00000010 -> 0x00000099
1: 0x00000100 -> 0x0007da00
1: 0x0007e800 -> 0x0007ffa0
1: 0x0007ffae -> 0x0007ffb0...
Subtract (29 early reservations)
#000 [0000001000 - 0000002000]
#001 [0000089000 - 000008f000]
#002 [0000091000 - 0000093500]
...
#027 [007cbfef40 - 007e800000]
#028 [007e9ca000 - 007ff95000]
(0 free memory ranges)
Initializing HighMem for node 0 (00000000:00000000)
Initializing HighMem for node 1 (00000000:00000000)
Memory: 0k/2096832k available (6662k kernel code, 2096300k reserved, 4829k data, 484k init, 0k highmem)
...
Checking if this processor honours the WP bit even in supervisor mode...Ok.
swapper: page allocation failure. order:0, mode:0x0
Pid: 0, comm: swapper Not tainted 2.6.34-rc3-tip-03818-g4b1ea6c-dirty #35
Call Trace: [<4087a5dc>] ? printk+0xf/0x11 [<40286728>] __alloc_pages_nodemask+0x417/0x487 [<402a9ce1>] new_slab+0xe2/0x1fe [<402aa5b2>] kmem_cache_open+0x185/0x358 [<402abbc0>] T.954+0x1c/0x60 [<40d52a29>] kmem_cache_init+0x24/0x113 [<40d39738>] start_kernel+0x166/0x2e4 [<40d3940e>] ? unknown_bootoption+0x0/0x18e [<40d390ce>] i386_start_kernel+0xce/0xd5
Mem-Info:
Node 1 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
Node 1 Normal per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:0 dirty:0 writeback:0 unstable:0
free:0 slab_reclaimable:0 slab_unreclaimable:0
mapped:0 shmem:0 pagetables:0 bounce:0
When 32bit numa is used, free_all_bootmem() will still only go over with node id 0.
If node 0 doesn't have RAM installed, We need to go with node1 because early_node_map still use 1 for all ranges, and ram from node1 become low ram.
Try to use MAX_NUMNODES like 64 numa does.
Note: BOOTMEM path has the same problem.
this bug exist before We have NO_BOOTMEM support.
-v3: add more comments, and fix bootmem path too.
-v4: seperate bootmem path fix
--- mm/bootmem.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
mm/bootmem.c +8 -1
@@ -303,7 +303,14 @@ unsigned long __init free_all_bootmem_nounsigned long __init free_all_bootmem (void ) {#ifdef CONFIG_NO_BOOTMEM -return free_all_memory_core_early (NODE_DATA (0 )->node_id); +/* +* We need to use MAX_NUMNODES instead of NODE_DATA(0)->node_id +* because in some case like Node0 doesnt have RAM installed +* low ram will be on Node1 +* Use MAX_NUMNODES will make sure all ranges in early_node_map[] +* will be used instead of only Node0 related +*/ +return free_all_memory_core_early (MAX_NUMNODES );#else return free_all_bootmem_core (NODE_DATA (0 )->bdata);#endif
PATCH 2/2 bootmem, x86: Fix 32bit numa system without RAM on Node0
When 32bit numa is used, free_all_bootmem() will still only go over with node id 0.
If node 0 doesn't have RAM installed, We need to go with node1 because early_node_map still use 1 for all ranges, and ram from node1 become low ram.
this one fixes BOOTMEM path by loop bdata_list.
-v3: add more comments, and fix bootmem path too.
-v4: seperate from one big patch
--- mm/bootmem.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
mm/bootmem.c +7 -1
@@ -312,7 +312,13 @@ unsigned long __init free_all_bootmem(vo*/ return free_all_memory_core_early (MAX_NUMNODES );#else -return free_all_bootmem_core (NODE_DATA (0 )->bdata); +unsigned long total_pages =0 ; +bootmem_data_t *bdata; + +list_for_each_entry (bdata, &bdata_list, list) + total_pages +=free_all_bootmem_core (bdata); + +return total_pages;#endif }
[tip:x86/urgent] bootmem, x86: Fix 32bit numa system without RAM on node 0
Commit-ID: aa235fc712f379d4194cff9217f07026c452c141 Gitweb: http://git.kernel.org/tip/aa235fc712f379d4194cff9217f07026c452c141 Author: Yinghai Lu <[email protected]> AuthorDate: Wed, 31 Mar 2010 20:45:27 -0700 Committer: H. Peter Anvin <[email protected]> CommitDate: Thu, 1 Apr 2010 14:41:19 -0700
bootmem, x86: Fix 32bit numa system without RAM on node 0
When 32bit numa is used, free_all_bootmem() will still only go over with node id 0.
If node 0 doesn't have RAM installed, the lowest populated node becomes low RAM.
This one fixes BOOTMEM path by iterating over the bdata_list.
-v3: add more comments, and fix bootmem path too.
-v4: seperate from one big patch
LKML-Reference: <[email protected]>
--- mm/bootmem.c | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-)
mm/bootmem.c +7 -1
@@ -312,7 +312,13 @@ unsigned long __init free_all_bootmem(void)*/ return free_all_memory_core_early (MAX_NUMNODES );#else -return free_all_bootmem_core (NODE_DATA (0 )->bdata); +unsigned long total_pages =0 ; +bootmem_data_t *bdata; + +list_for_each_entry (bdata, &bdata_list, list) + total_pages +=free_all_bootmem_core (bdata); + +return total_pages;#endif }
[tip:x86/urgent] nobootmem, x86: Fix 32bit numa system without RAM on node 0
In reply to Yinghai LuCommit-ID: 337998587f802535896e9ed16d19f97915ccd368 Gitweb: http://git.kernel.org/tip/337998587f802535896e9ed16d19f97915ccd368 Author: Yinghai Lu <[email protected]> AuthorDate: Wed, 31 Mar 2010 20:44:09 -0700 Committer: H. Peter Anvin <[email protected]> CommitDate: Thu, 1 Apr 2010 14:39:29 -0700
nobootmem, x86: Fix 32bit numa system without RAM on node 0
On one system without RAM on node0, got following boot dump with a 32 bit NUMA kernel:
early_node_map[4] active PFN ranges
1: 0x00000010 -> 0x00000099
1: 0x00000100 -> 0x0007da00
1: 0x0007e800 -> 0x0007ffa0
1: 0x0007ffae -> 0x0007ffb0...
Subtract (29 early reservations)
#000 [0000001000 - 0000002000]
#001 [0000089000 - 000008f000]
#002 [0000091000 - 0000093500]
...
#027 [007cbfef40 - 007e800000]
#028 [007e9ca000 - 007ff95000]
(0 free memory ranges)
Initializing HighMem for node 0 (00000000:00000000)
Initializing HighMem for node 1 (00000000:00000000)
Memory: 0k/2096832k available (6662k kernel code, 2096300k reserved, 4829k data, 484k init, 0k highmem)
...
Checking if this processor honours the WP bit even in supervisor mode...Ok.
swapper: page allocation failure. order:0, mode:0x0
Pid: 0, comm: swapper Not tainted 2.6.34-rc3-tip-03818-g4b1ea6c-dirty #35
Call Trace: [<4087a5dc>] ? printk+0xf/0x11 [<40286728>] __alloc_pages_nodemask+0x417/0x487 [<402a9ce1>] new_slab+0xe2/0x1fe [<402aa5b2>] kmem_cache_open+0x185/0x358 [<402abbc0>] T.954+0x1c/0x60 [<40d52a29>] kmem_cache_init+0x24/0x113 [<40d39738>] start_kernel+0x166/0x2e4 [<40d3940e>] ? unknown_bootoption+0x0/0x18e [<40d390ce>] i386_start_kernel+0xce/0xd5
Mem-Info:
Node 1 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
Node 1 Normal per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:0 dirty:0 writeback:0 unstable:0
free:0 slab_reclaimable:0 slab_unreclaimable:0
mapped:0 shmem:0 pagetables:0 bounce:0
When 32bit NUMA is used, free_all_bootmem() will still only go over with node id 0.
If node 0 doesn't have RAM installed, We need to go with node1 because early_node_map still use 1 for all ranges, and ram from node1 become low ram.
Use MAX_NUMNODES like 64-bit NUMA does.
Note: BOOTMEM path has the same problem.
this bug exist before We have NO_BOOTMEM support.
-v3: add more comments, and fix bootmem path too.
-v4: seperate bootmem path fix
LKML-Reference: <[email protected]>
--- mm/bootmem.c | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-)
mm/bootmem.c +8 -1
@@ -303,7 +303,14 @@ unsigned long __init free_all_bootmem_node(pg_data_t *pgdat)unsigned long __init free_all_bootmem (void ) {#ifdef CONFIG_NO_BOOTMEM -return free_all_memory_core_early (NODE_DATA (0 )->node_id); +/* +* We need to use MAX_NUMNODES instead of NODE_DATA(0)->node_id +* because in some case like Node0 doesnt have RAM installed +* low ram will be on Node1 +* Use MAX_NUMNODES will make sure all ranges in early_node_map[] +* will be used instead of only Node0 related +*/ +return free_all_memory_core_early (MAX_NUMNODES );#else return free_all_bootmem_core (NODE_DATA (0 )->bdata);#endif
Quoted 2 lines · James Morris
Also, the help text for the item makes little sense to a non-expert in this area:
I too noticed this absolutely catastrophic "help" text but forgot to send a bug report.
Either this option can be explained and the text fixed, or it cannot be explained and shouldn't be an option in the first place.
Signature · 3 lines
Stefan Richter -=====-==-=- --== ===== http://arcgraph.de/sr/