ACPI: NUMA: Fix overlap when extending memblks to fill CFMWS
PATCH 1/1 ACPI: NUMA: Fix overlap when extending memblks to fill CFMWS
From: Alison Schofield <[email protected]>
When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses numa_fill_memblks() to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.
The calculation of the memblks to fill has an off-by-one error, that causes numa_init to fail when it sees the overlap:
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]
Fix by making the 'end' parameter to numa_fill_memblks() exclusive.
--- drivers/acpi/numa/srat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/acpi/numa/srat.c +1 -1
@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, cfmws = (struct acpi_cedt_cfmws *)header; start = cfmws->base_hpa; - end = cfmws->base_hpa + cfmws->window_size; + end = cfmws->base_hpa + cfmws->window_size -1 ;/* * The SRAT may have already described NUMA details for all,
base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86
Signature · 1 line
2.37.3
[ add Rafael and linux-acpi for their awareness ]
Rafael, since the issue came in through cxl development I will take the fix through the cxl tree unless you have an objection.
Quoted 43 lines · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses
numa_fill_memblks()to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.The calculation of the memblks to fill has an off-by-one error, that causes
numa_initto fail when it sees the overlap:[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]Fix by making the 'end' parameter to
numa_fill_memblks()exclusive.Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")Reported-by "Huang, Ying" <[email protected]>Suggested-by Dan Williams <[email protected]>Signed-off-by Alison Schofield <[email protected]>--- drivers/acpi/numa/srat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/acpi/numa/srat.c +1
-1@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, cfmws = (struct acpi_cedt_cfmws *)header; start = cfmws->base_hpa; - end = cfmws->base_hpa + cfmws->window_size; + end = cfmws->base_hpa + cfmws->window_size -1 ;/* * The SRAT may have already described NUMA details for all, base-commit:
b85ea95d086471afb4ad062012a4d73cd328fa86Signature · 1 line
2.37.3
Quoted 15 lines · On 12/22/23 17:00, [email protected] wrote:
From: Alison Schofield <[email protected]>
When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses
numa_fill_memblks()to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.The calculation of the memblks to fill has an off-by-one error, that causes
numa_initto fail when it sees the overlap:[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]
Quoted 6 lines
Fix by making the 'end' parameter to
numa_fill_memblks()exclusive.Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")Reported-by "Huang, Ying" <[email protected]>Suggested-by Dan Williams <[email protected]>Signed-off-by Alison Schofield <[email protected]>
Quoted 19 lines
--- drivers/acpi/numa/srat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/acpi/numa/srat.c +1
-1@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, cfmws = (struct acpi_cedt_cfmws *)header; start = cfmws->base_hpa; - end = cfmws->base_hpa + cfmws->window_size; + end = cfmws->base_hpa + cfmws->window_size -1 ;/* * The SRAT may have already described NUMA details for all, base-commit:
b85ea95d086471afb4ad062012a4d73cd328fa86
Quoted 2 lines · On Fri, Dec 22, 2023 at 04:00:25PM -0800, [email protected] wrote:
From: Alison Schofield <[email protected]>
Quoted 41 lines
When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses
numa_fill_memblks()to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.The calculation of the memblks to fill has an off-by-one error, that causes
numa_initto fail when it sees the overlap:[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]Fix by making the 'end' parameter to
numa_fill_memblks()exclusive.Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")Reported-by "Huang, Ying" <[email protected]>Suggested-by Dan Williams <[email protected]>Signed-off-by Alison Schofield <[email protected]>--- drivers/acpi/numa/srat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/acpi/numa/srat.c +1
-1@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, cfmws = (struct acpi_cedt_cfmws *)header; start = cfmws->base_hpa; - end = cfmws->base_hpa + cfmws->window_size; + end = cfmws->base_hpa + cfmws->window_size -1 ;/* * The SRAT may have already described NUMA details for all, base-commit:
b85ea95d086471afb4ad062012a4d73cd328fa86Signature · 1 line
2.37.3
Quoted 30 lines · <[email protected]> writes:
From: Alison Schofield <[email protected]>
When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses
numa_fill_memblks()to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.The calculation of the memblks to fill has an off-by-one error, that causes
numa_initto fail when it sees the overlap:[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]Fix by making the 'end' parameter to
numa_fill_memblks()exclusive.Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")Reported-by "Huang, Ying" <[email protected]>Suggested-by Dan Williams <[email protected]>Signed-off-by Alison Schofield <[email protected]>--- drivers/acpi/numa/srat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/acpi/numa/srat.c +0
-0
drivers/acpi/numa/srat.c static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, · line 311
Quoted 5 lines
drivers/acpi/numa/srat.c +1
-1@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, cfmws = (struct acpi_cedt_cfmws *)header; start = cfmws->base_hpa; - end = cfmws->base_hpa + cfmws->window_size; + end = cfmws->base_hpa + cfmws->window_size -1 ;
IIUC, this makes end inclusive instead of exclusive. That is, from "[start, end)" to "[start, end]". Am I right?
If so, later in the function, "numa_add_memblk(node, start, end)" is called, where "end" is expected to be exclusive. And I think that it's better to make the parameter "end" of numa_fill_memblks() consistent with other numa_memblk family functions, that is, exclusive.
Quoted 4 lines
drivers/acpi/numa/srat.c +0
-0/* * The SRAT may have already described NUMA details for all, base-commit:
b85ea95d086471afb4ad062012a4d73cd328fa86
--
Best Regards,
Huang, Ying
Quoted 33 lines · Ying
Quoted 31 lines · <[email protected]> writes:
From: Alison Schofield <[email protected]>
When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses
numa_fill_memblks()to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.The calculation of the memblks to fill has an off-by-one error, that causes
numa_initto fail when it sees the overlap:[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]Fix by making the 'end' parameter to
numa_fill_memblks()exclusive.Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")Reported-by "Huang, Ying" <[email protected]>Suggested-by Dan Williams <[email protected]>Signed-off-by Alison Schofield <[email protected]>--- drivers/acpi/numa/srat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/acpi/numa/srat.c +0
-0@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header,
Quoted 7 lines
drivers/acpi/numa/srat.c static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, · line 311
Quoted 4 lines
drivers/acpi/numa/srat.c +1
-1cfmws = (struct acpi_cedt_cfmws *)header; start = cfmws->base_hpa; - end = cfmws->base_hpa + cfmws->window_size; + end = cfmws->base_hpa + cfmws->window_size -1 ;IIUC, this makes end inclusive instead of exclusive. That is, from "[start, end)" to "[start, end]". Am I right?
You are right.
numa_fill_memblks() is confusing. It's kernel doc comments says end is inclusive, then in-code comments that end is exclusive, and then it fails to handle the exclusive case correctly.
I think this change in numa_fill_memblks() that you suggested (off-list) is the right way to go:
Quoted 8 lines
Quoted 8 lines
arch/x86/mm/numa.c +0
-0@@ -1007,7 +1007,7 @@ int __init numa_fill_memblks(u64 start, u64 end)for (int i =0 ; i < mi->nr_blks; i++) {struct numa_memblk *bi = &mi->blk[i];
Quoted 6 lines
arch/x86/mm/numa.c int __init numa_fill_memblks(u64 start, u64 end) · line 1013
Quoted 6 lines
arch/x86/mm/numa.c +1
-1-if (start < bi->end && end >= bi->start) { +if (start < bi->end && end > bi->start) { blk[count] = &mi->blk[i]; count++; }
I'll also clean up the in code & kernel doc comments.
Thanks for the catch!
Alison
Quoted 6 lines
If so, later in the function, "
numa_add_memblk(node, start, end)" is called, where "end" is expected to be exclusive. And I think that it's better to make the parameter "end" ofnuma_fill_memblks()consistent with othernuma_memblkfamily functions, that is, exclusive.
Agree.
Quoted 8 lines
Quoted 4 lines
arch/x86/mm/numa.c +0
-0/* * The SRAT may have already described NUMA details for all, base-commit:
b85ea95d086471afb4ad062012a4d73cd328fa86--
Best Regards,
Huang, Ying
Quoted 33 lines · Ying
Quoted 31 lines · <[email protected]> writes:
From: Alison Schofield <[email protected]>
When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses
numa_fill_memblks()to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.The calculation of the memblks to fill has an off-by-one error, that causes
numa_initto fail when it sees the overlap:[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]Fix by making the 'end' parameter to
numa_fill_memblks()exclusive.Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")Reported-by "Huang, Ying" <[email protected]>Suggested-by Dan Williams <[email protected]>Signed-off-by Alison Schofield <[email protected]>--- drivers/acpi/numa/srat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/acpi/numa/srat.c +0
-0@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header,
Quoted 7 lines
drivers/acpi/numa/srat.c static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, · line 311
Quoted 4 lines
drivers/acpi/numa/srat.c +1
-1cfmws = (struct acpi_cedt_cfmws *)header; start = cfmws->base_hpa; - end = cfmws->base_hpa + cfmws->window_size; + end = cfmws->base_hpa + cfmws->window_size -1 ;IIUC, this makes end inclusive instead of exclusive. That is, from "[start, end)" to "[start, end]". Am I right?
"Exlcusive" => Exclude the last value ("cfmws->base_hpa + cfmws->window_size") from the range:
[cfmws->base_hpa, cfmws->window_size)
Quoted 2 lines
If so, later in the function, "
numa_add_memblk(node, start, end)" is called, where "end" is expected to be exclusive.
Good eye, yes.
Quoted 3 lines
And I think that it's better to make the parameter "end" of
numa_fill_memblks()consistent with othernuma_memblkfamily functions, that is, exclusive.
s/exclusive/inclusive/, but yes for the immediate fix. It is unfortunate that "end" here is different than the typical meaning of end as in 'struct resource' and 'struct range'. I would love to see a conversion along the lines of:
arch/x86/mm/numa_internal.h +1 -2
@@ -6,8 +6,7 @@#include <asm/numa.h> struct numa_memblk { -u64 start; -u64 end; +struct range r;int nid; };
...and then intoduce helpers in include/linux/range.h to remove the error-prone open-coded versions of all the range comparison math.