x86: fix nodes_cover_memory
PATCH 1/1 x86: fix nodes_cover_memory
found one system that missed one entry for one node in SRAT, and that SRAT is not rejected by nodes_cover_memory()
it turns out that we can not use absent_page_in_range to calaulate e820ram, bacause that will use early_node_map and that is AND result of e820 and SRAT.
revert back to use e820_hole_size instead.
also change that difference checking to 1M instead of 4G, because e820ram, and pxmram are in pages.
[ Impact: reject wrong SRAT tables ]
--- arch/x86/mm/srat_64.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/mm/srat_64.c +2 -2
@@ -345,9 +345,9 @@ static int __init nodes_cover_memory(con pxmram =0 ; } - e820ram = max_pfn -absent_pages_in_range (0 , max_pfn); + e820ram = max_pfn - (e820_hole_size (0 , max_pfn<<PAGE_SHIFT )>>PAGE_SHIFT );/* We seem to lose 3 pages somewhere. Allow a bit of slack. */ -if ((long )(e820ram - pxmram) >=1 *1024 *1024 ) { +if ((long )(e820ram - pxmram) >= (1 <<(20 -PAGE_SHIFT ))) { printk(KERN_ERR "SRAT: PXMs only cover %luMB of your %luMB e820 RAM. Not used.\n", (pxmram << PAGE_SHIFT) >> 20,
Quoted 8 lines · Yinghai Lu
found one system that missed one entry for one node in SRAT, and that SRAT is not rejected by
nodes_cover_memory()it turns out that we can not use
absent_page_in_rangeto calaulate e820ram, bacause that will useearly_node_mapand that is AND result of e820 and SRAT.
Correct, good spot.
Quoted 2 lines
revert back to use
e820_hole_sizeinstead.
I think the patch fixing this part of the problem is good, but the changelog could be better. It took me a while to figure out what the problem was and why this patch addressed it.
How about something like the following?
====
Sanity check the e820 against the SRAT table using only information from the e820 map
node_cover_memory() sanity checks the SRAT table by ensuring that all PXMs cover the memory reported in the e820. However, when calculating the size of the holes in the e820, it uses the early_node_map[] which contains information taken from both SRAT and e820. If the SRAT is missing an entry, then it is not detected that the SRAT table is incorrect and missing entries.
This patch uses the e820 map to calculate the holes instead of early_node_map[].
====
As an aside, it strikes me as odd that we discard an entire SRAT because it is missing an entry in the e820. The impact may only be that the affinity for a range of memory is incorrect, but it does not necessarily mean that the entire table is incorrect. The intention of the code appears to be "if there is any error in the SRAT, it's best ignored" though so maybe it's best left alone.
Quoted 3 lines
also change that difference checking to 1M instead of 4G, because e820ram, and pxmram are in pages.
While I agree with you, this should be a separate patch with its own changelog. Something like
===
Allow 1MB of slack between the e820 map and SRAT, not 4GB
It is expected that there be slight differences between the e820 map and the SRAT table and the intention was that 1MB of slack be allowed. The calculation comparing e820ram and pxmram assumes the units are bytes, when they are in fact pages. This means 4GB of slack is being allowed, not 1MB. This patch makes the correct comparison
===
(1<<(20 - PAGE_SHIFT)) is a bit unreadable. At the very least, change the comment above from "Allow a bit of slack" to "Allow 1MB of slack" so the next reader knows what the intention of (1<<(20 - PAGE_SHIFT)) is.
Thanks
Quoted 26 lines
[ Impact: reject wrong SRAT tables ]
Signed-off-by Yinghai Lu <[email protected]>Cc Mel Gorman <[email protected]>--- arch/x86/mm/srat_64.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Index:
linux-2.6/arch/x86/mm/srat_64.c
===================================================================arch/x86/mm/srat_64.c +2
-2@@ -345,9 +345,9 @@ static int __init nodes_cover_memory(con pxmram =0 ; } - e820ram = max_pfn -absent_pages_in_range (0 , max_pfn); + e820ram = max_pfn - (e820_hole_size (0 , max_pfn<<PAGE_SHIFT )>>PAGE_SHIFT );/* We seem to lose 3 pages somewhere. Allow a bit of slack. */ -if ((long )(e820ram - pxmram) >=1 *1024 *1024 ) { +if ((long )(e820ram - pxmram) >= (1 <<(20 -PAGE_SHIFT ))) { printk(KERN_ERR "SRAT: PXMs only cover %luMB of your %luMB e820 RAM. Not used.\n", (pxmram << PAGE_SHIFT) >> 20,
Signature · 3 lines
Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab
Quoted 58 lines · Mel Gorman
Quoted 8 lines · Yinghai Lu
found one system that missed one entry for one node in SRAT, and that SRAT is not rejected by
nodes_cover_memory()it turns out that we can not use
absent_page_in_rangeto calaulate e820ram, bacause that will useearly_node_mapand that is AND result of e820 and SRAT.Correct, good spot.
Quoted 2 lines
revert back to use
e820_hole_sizeinstead.I think the patch fixing this part of the problem is good, but the changelog could be better. It took me a while to figure out what the problem was and why this patch addressed it.
How about something like the following?
====
Sanity check the e820 against the SRAT table using only information from the e820 map
node_cover_memory()sanity checks the SRAT table by ensuring that all PXMs cover the memory reported in the e820. However, when calculating the size of the holes in the e820, it uses the early_node_map[] which contains information taken from both SRAT and e820. If the SRAT is missing an entry, then it is not detected that the SRAT table is incorrect and missing entries.This patch uses the e820 map to calculate the holes instead of early_node_map[].
====As an aside, it strikes me as odd that we discard an entire SRAT because it is missing an entry in the e820. The impact may only be that the affinity for a range of memory is incorrect, but it does not necessarily mean that the entire table is incorrect. The intention of the code appears to be "if there is any error in the SRAT, it's best ignored" though so maybe it's best left alone.
Quoted 3 lines
also change that difference checking to 1M instead of 4G, because e820ram, and pxmram are in pages.
While I agree with you, this should be a separate patch with its own changelog. Something like
===
Allow 1MB of slack between the e820 map and SRAT, not 4GBIt is expected that there be slight differences between the e820 map and the SRAT table and the intention was that 1MB of slack be allowed. The calculation comparing e820ram and pxmram assumes the units are bytes, when they are in fact pages. This means 4GB of slack is being allowed, not 1MB. This patch makes the correct comparison
===
Quoted 5 lines
(1<<(20 -
PAGE_SHIFT)) is a bit unreadable. At the very least, change the comment above from "Allow a bit of slack" to "Allow 1MB of slack" so the next reader knows what the intention of (1<<(20 -PAGE_SHIFT)) is.Thanks
thanks Mel!
Yinghai, mind resending the patch as two patches, with Mel's changelogs in place and with Mel's Acked-by as well?
Thanks,
Ingo
Mel Gorman
Acked-byQuoted 65 lines · Ingo Molnar
Quoted 62 lines · Mel Gorman
Quoted 8 lines · Yinghai Lu
found one system that missed one entry for one node in SRAT, and that SRAT is not rejected by
nodes_cover_memory()it turns out that we can not use
absent_page_in_rangeto calaulate e820ram, bacause that will useearly_node_mapand that is AND result of e820 and SRAT.Correct, good spot.
Quoted 2 lines
revert back to use
e820_hole_sizeinstead.I think the patch fixing this part of the problem is good, but the changelog could be better. It took me a while to figure out what the problem was and why this patch addressed it.
How about something like the following?
====
Sanity check the e820 against the SRAT table using only information from the e820 map
node_cover_memory()sanity checks the SRAT table by ensuring that all PXMs cover the memory reported in the e820. However, when calculating the size of the holes in the e820, it uses the early_node_map[] which contains information taken from both SRAT and e820. If the SRAT is missing an entry, then it is not detected that the SRAT table is incorrect and missing entries.This patch uses the e820 map to calculate the holes instead of early_node_map[].
====As an aside, it strikes me as odd that we discard an entire SRAT because it is missing an entry in the e820. The impact may only be that the affinity for a range of memory is incorrect, but it does not necessarily mean that the entire table is incorrect. The intention of the code appears to be "if there is any error in the SRAT, it's best ignored" though so maybe it's best left alone.
Quoted 3 lines
also change that difference checking to 1M instead of 4G, because e820ram, and pxmram are in pages.
While I agree with you, this should be a separate patch with its own changelog. Something like
===
Allow 1MB of slack between the e820 map and SRAT, not 4GBIt is expected that there be slight differences between the e820 map and the SRAT table and the intention was that 1MB of slack be allowed. The calculation comparing e820ram and pxmram assumes the units are bytes, when they are in fact pages. This means 4GB of slack is being allowed, not 1MB. This patch makes the correct comparison
===(1<<(20 -
PAGE_SHIFT)) is a bit unreadable. At the very least, change the comment above from "Allow a bit of slack" to "Allow 1MB of slack" so the next reader knows what the intention of (1<<(20 -PAGE_SHIFT)) is.
Quoted 7 lines
Quoted 1 line
Thanks
thanks Mel!
Yinghai, mind resending the patch as two patches, with Mel's changelogs in place and with Mel's Acked-by as well?
My bad for leaving it out, but yes. Assuming there are two patches with the updated changelog, I'm fine with adding the
Signature · 3 lines
Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab
PATCH 1/2 x86: Sanity check the e820 against the SRAT table using e820 map only
In reply to Ingo Molnarnode_cover_memory() sanity checks the SRAT table by ensuring that all PXMs cover the memory reported in the e820. However, when calculating the size of the holes in the e820, it uses the early_node_map[] which contains information taken from both SRAT and e820. If the SRAT is missing an entry, then it is not detected that the SRAT table is incorrect and missing entries.
This patch uses the e820 map to calculate the holes instead of early_node_map[].
comment is from Mel.
[ Impact: reject wrong SRAT tables ]
--- arch/x86/mm/srat_64.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/mm/srat_64.c +1 -1
@@ -345,7 +345,7 @@ static int __init nodes_cover_memory(con pxmram =0 ; } - e820ram = max_pfn -absent_pages_in_range (0 , max_pfn); + e820ram = max_pfn - (e820_hole_size (0 , max_pfn<<PAGE_SHIFT )>>PAGE_SHIFT );/* We seem to lose 3 pages somewhere. Allow a bit of slack. */ if ((long )(e820ram - pxmram) >=1 *1024 *1024 ) { printk(KERN_ERR
PATCH 2/2 x86: Allow 1MB of slack between the e820 map and SRAT, not 4GB
It is expected that there be slight differences between the e820 map and the SRAT table and the intention was that 1MB of slack be allowed. The calculation comparing e820ram and pxmram assumes the units are bytes, when they are in fact pages. This means 4GB of slack is being allowed, not 1MB. This patch makes the correct comparison
comment is from Mel.
[ Impact: don't accept the srat that could dump 2g etc ram ]
--- arch/x86/mm/srat_64.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/mm/srat_64.c +2 -2
@@ -346,8 +346,8 @@ static int __init nodes_cover_memory(con } e820ram = max_pfn - (e820_hole_size (0 , max_pfn<<PAGE_SHIFT )>>PAGE_SHIFT ); -/* We seem to lose 3 pages somewhere. Allow a bit of slack. */ -if ((long )(e820ram - pxmram) >=1 *1024 *1024 ) { +/* We seem to lose 3 pages somewhere. Allow 1M of slack. */ +if ((long )(e820ram - pxmram) >= (1 <<(20 -PAGE_SHIFT ))) { printk(KERN_ERR "SRAT: PXMs only cover %luMB of your %luMB e820 RAM. Not used.\n", (pxmram << PAGE_SHIFT) >> 20,
[tip:x86/mm] x86: Allow 1MB of slack between the e820 map and SRAT, not 4GB
Commit-ID: 0964b0562bb9c93194e852b47bab2397b9e11c18 Gitweb: http://git.kernel.org/tip/0964b0562bb9c93194e852b47bab2397b9e11c18 Author: Yinghai Lu <[email protected]> AuthorDate: Fri, 8 May 2009 00:37:34 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 11 May 2009 11:38:21 +0200
x86: Allow 1MB of slack between the e820 map and SRAT, not 4GB
It is expected that there might be slight differences between the e820 map and the SRAT table and the intention was that 1MB of slack be allowed.
The calculation comparing e820ram and pxmram assumes the units are bytes, when they are in fact pages. This means 4GB of slack is being allowed, not 1MB. This patch makes the correct comparison.
comment is from Mel.
[ Impact: don't accept buggy SRATs that could dump up to 4G of RAM ]
LKML-Reference: <[email protected]>
--- arch/x86/mm/srat_64.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/mm/srat_64.c +2 -2
@@ -346,8 +346,8 @@ static int __init nodes_cover_memory(const struct bootnode *nodes) } e820ram = max_pfn - (e820_hole_size (0 , max_pfn<<PAGE_SHIFT )>>PAGE_SHIFT ); -/* We seem to lose 3 pages somewhere. Allow a bit of slack. */ -if ((long )(e820ram - pxmram) >=1 *1024 *1024 ) { +/* We seem to lose 3 pages somewhere. Allow 1M of slack. */ +if ((long )(e820ram - pxmram) >= (1 <<(20 -PAGE_SHIFT ))) { printk(KERN_ERR "SRAT: PXMs only cover %luMB of your %luMB e820 RAM. Not used.\n", (pxmram << PAGE_SHIFT) >> 20,
Re: [PATCH 1/2] x86: Sanity check the e820 against the SRAT table using e820 map only
In reply to Yinghai Luarch/x86/mm/srat_64.c static int __init nodes_cover_memory(con · line 348
Quoted 2 lines · Yinghai Lu
arch/x86/mm/srat_64.c +1
-1- e820ram = max_pfn -absent_pages_in_range (0 , max_pfn); + e820ram = max_pfn - (e820_hole_size (0 , max_pfn<<PAGE_SHIFT )>>PAGE_SHIFT );
btw., it would be nice to have a debug check that prints a warning if the SRAT does not cover all RAM, or if it covers RAM that is not present in the e820 map. Such a warning might alert us to double check all the PXM settings in the SRAT and could uncover more quirks like the above ...
Ingo
Re: [PATCH 1/2] x86: Sanity check the e820 against the SRAT table using e820 map only
Quoted 10 lines · Ingo Molnar
arch/x86/mm/srat_64.c static int __init nodes_cover_memory(con · line 348
Quoted 2 lines · Yinghai Lu
arch/x86/mm/srat_64.c +1
-1- e820ram = max_pfn -absent_pages_in_range (0 , max_pfn); + e820ram = max_pfn - (e820_hole_size (0 , max_pfn<<PAGE_SHIFT )>>PAGE_SHIFT );btw., it would be nice to have a debug check that prints a warning if the SRAT does not cover all RAM, or if it covers RAM that is not present in the e820 map. Such a warning might alert us to double check all the PXM settings in the SRAT and could uncover more quirks like the above ...
we trust e820 than SRAT mem entries.
so early_node_map already are result of in e820 range AND srat. (the range in SRAT that is not in e820 is dumped)
just check srat table (without AND) with e820 will have some confusing. because SRAT would cover the hole below 4g that is not in E820.
YH
[tip:x86/mm] x86: Sanity check the e820 against the SRAT table using e820 map only
In reply to Yinghai LuCommit-ID: b37ab91907e9002925f4217e3bbd496aa12c2fa3 Gitweb: http://git.kernel.org/tip/b37ab91907e9002925f4217e3bbd496aa12c2fa3 Author: Yinghai Lu <[email protected]> AuthorDate: Fri, 8 May 2009 00:36:44 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 11 May 2009 11:35:07 +0200
x86: Sanity check the e820 against the SRAT table using e820 map only
node_cover_memory() sanity checks the SRAT table by ensuring that all PXMs cover the memory reported in the e820.
However, when calculating the size of the holes in the e820, it uses the early_node_map[] which contains information taken from both SRAT and e820. If the SRAT is missing an entry, then it is not detected that the SRAT table is incorrect and missing entries.
This patch uses the e820 map to calculate the holes instead of early_node_map[].
comment is from Mel.
[ Impact: reject incorrect SRAT tables ]
LKML-Reference: <[email protected]>
--- arch/x86/mm/srat_64.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/mm/srat_64.c +1 -1
@@ -345,7 +345,7 @@ static int __init nodes_cover_memory(const struct bootnode *nodes) pxmram =0 ; } - e820ram = max_pfn -absent_pages_in_range (0 , max_pfn); + e820ram = max_pfn - (e820_hole_size (0 , max_pfn<<PAGE_SHIFT )>>PAGE_SHIFT );/* We seem to lose 3 pages somewhere. Allow a bit of slack. */ if ((long )(e820ram - pxmram) >=1 *1024 *1024 ) { printk(KERN_ERR