Skip to contentSkip to conversation

ACPI: NUMA: Fix overlap when extending memblks to fill CFMWS

PATCH Superseded [email protected] 5 participants 7 messages Dec 23, 2023 – Jan 2, 2024

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ACPI: NUMA: Fix overlap when extending memblks to fill CFMWS

From: Alison Schofield <[email protected]>

When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses numa_fill_memblks() to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.

The calculation of the memblks to fill has an off-by-one error, that causes numa_init to fail when it sees the overlap:

[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]

Fix by making the 'end' parameter to numa_fill_memblks() exclusive.

Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")
---
 drivers/acpi/numa/srat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/acpi/numa/srat.c +1 -1
index 12f330b0eac0..b99062f7c412 100644
@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header,
 
 	cfmws = (struct acpi_cedt_cfmws *)header;
 	start = cfmws->base_hpa;
-	end = cfmws->base_hpa + cfmws->window_size;
+	end = cfmws->base_hpa + cfmws->window_size - 1;
 
 	/*
 	 * The SRAT may have already described NUMA details for all,

base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86

Signature · 1 line
2.37.3
#1 lore raw

[ add Rafael and linux-acpi for their awareness ]

Rafael, since the issue came in through cxl development I will take the fix through the cxl tree unless you have an objection.

Quoted 43 lines · alison.schofield@ wrote:

From: Alison Schofield <[email protected]>

When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses numa_fill_memblks() to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.

The calculation of the memblks to fill has an off-by-one error, that causes numa_init to fail when it sees the overlap:

[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]

Fix by making the 'end' parameter to numa_fill_memblks() exclusive.

Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")
---
 drivers/acpi/numa/srat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/acpi/numa/srat.c +1 -1
index 12f330b0eac0..b99062f7c412 100644
@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header,
 
 	cfmws = (struct acpi_cedt_cfmws *)header;
 	start = cfmws->base_hpa;
-	end = cfmws->base_hpa + cfmws->window_size;
+	end = cfmws->base_hpa + cfmws->window_size - 1;
 
 	/*
 	 * The SRAT may have already described NUMA details for all,

base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86

Signature · 1 line
2.37.3
Reviewed-by on v1 1/1
#2 lore raw
In reply to [email protected]
Quoted 15 lines · On 12/22/23 17:00, [email protected] wrote:

From: Alison Schofield <[email protected]>

When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses numa_fill_memblks() to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.

The calculation of the memblks to fill has an off-by-one error, that causes numa_init to fail when it sees the overlap:

[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]

Quoted 6 lines

Fix by making the 'end' parameter to numa_fill_memblks() exclusive.

Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")
Quoted 19 lines
---
 drivers/acpi/numa/srat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/acpi/numa/srat.c +1 -1
index 12f330b0eac0..b99062f7c412 100644
@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header,
 
 	cfmws = (struct acpi_cedt_cfmws *)header;
 	start = cfmws->base_hpa;
-	end = cfmws->base_hpa + cfmws->window_size;
+	end = cfmws->base_hpa + cfmws->window_size - 1;
 
 	/*
 	 * The SRAT may have already described NUMA details for all,

base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86

Reviewed-by Fan Ni on v1 1/1
#3 lore raw
In reply to [email protected]
Quoted 2 lines · On Fri, Dec 22, 2023 at 04:00:25PM -0800, [email protected] wrote:

From: Alison Schofield <[email protected]>

Quoted 41 lines

When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses numa_fill_memblks() to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.

The calculation of the memblks to fill has an off-by-one error, that causes numa_init to fail when it sees the overlap:

[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]

Fix by making the 'end' parameter to numa_fill_memblks() exclusive.

Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")
---
 drivers/acpi/numa/srat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/acpi/numa/srat.c +1 -1
index 12f330b0eac0..b99062f7c412 100644
@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header,
 
 	cfmws = (struct acpi_cedt_cfmws *)header;
 	start = cfmws->base_hpa;
-	end = cfmws->base_hpa + cfmws->window_size;
+	end = cfmws->base_hpa + cfmws->window_size - 1;
 
 	/*
 	 * The SRAT may have already described NUMA details for all,

base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86

Signature · 1 line
2.37.3
#4 lore raw
In reply to [email protected]
Quoted 30 lines · <[email protected]> writes:

From: Alison Schofield <[email protected]>

When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses numa_fill_memblks() to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.

The calculation of the memblks to fill has an off-by-one error, that causes numa_init to fail when it sees the overlap:

[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]

Fix by making the 'end' parameter to numa_fill_memblks() exclusive.

Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")
---
 drivers/acpi/numa/srat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/acpi/numa/srat.c +0 -0
index 12f330b0eac0..b99062f7c412 100644

drivers/acpi/numa/srat.c static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, · line 311

Quoted 5 lines
drivers/acpi/numa/srat.c +1 -1
@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header,
 
 	cfmws = (struct acpi_cedt_cfmws *)header;
 	start = cfmws->base_hpa;
-	end = cfmws->base_hpa + cfmws->window_size;
+	end = cfmws->base_hpa + cfmws->window_size - 1;

IIUC, this makes end inclusive instead of exclusive. That is, from "[start, end)" to "[start, end]". Am I right?

If so, later in the function, "numa_add_memblk(node, start, end)" is called, where "end" is expected to be exclusive. And I think that it's better to make the parameter "end" of numa_fill_memblks() consistent with other numa_memblk family functions, that is, exclusive.

Quoted 4 lines
drivers/acpi/numa/srat.c +0 -0
 	/*
 	 * The SRAT may have already described NUMA details for all,
 

base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86

--
Best Regards,
Huang, Ying

#5 lore raw
Quoted 33 lines · Ying
Quoted 31 lines · <[email protected]> writes:

From: Alison Schofield <[email protected]>

When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses numa_fill_memblks() to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.

The calculation of the memblks to fill has an off-by-one error, that causes numa_init to fail when it sees the overlap:

[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]

Fix by making the 'end' parameter to numa_fill_memblks() exclusive.

Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")
---
 drivers/acpi/numa/srat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/acpi/numa/srat.c +0 -0
index 12f330b0eac0..b99062f7c412 100644
@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header,
 
Quoted 7 lines

drivers/acpi/numa/srat.c static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, · line 311

Quoted 4 lines
drivers/acpi/numa/srat.c +1 -1
 	cfmws = (struct acpi_cedt_cfmws *)header;
 	start = cfmws->base_hpa;
-	end = cfmws->base_hpa + cfmws->window_size;
+	end = cfmws->base_hpa + cfmws->window_size - 1;

IIUC, this makes end inclusive instead of exclusive. That is, from "[start, end)" to "[start, end]". Am I right?

You are right.

numa_fill_memblks() is confusing. It's kernel doc comments says end is inclusive, then in-code comments that end is exclusive, and then it fails to handle the exclusive case correctly.

I think this change in numa_fill_memblks() that you suggested (off-list) is the right way to go:

Quoted 8 lines
Quoted 8 lines
arch/x86/mm/numa.c +0 -0
index c01c5506fd4a..3850ba23fa90 100644
@@ -1007,7 +1007,7 @@ int __init numa_fill_memblks(u64 start, u64 end)
        for (int i = 0; i < mi->nr_blks; i++) {
                struct numa_memblk *bi = &mi->blk[i];
 
Quoted 6 lines

arch/x86/mm/numa.c int __init numa_fill_memblks(u64 start, u64 end) · line 1013

Quoted 6 lines
arch/x86/mm/numa.c +1 -1
-               if (start < bi->end && end >= bi->start) {
+               if (start < bi->end && end > bi->start) {
                        blk[count] = &mi->blk[i];
                        count++;
                }

I'll also clean up the in code & kernel doc comments.

Thanks for the catch!
Alison

Quoted 6 lines

If so, later in the function, "numa_add_memblk(node, start, end)" is called, where "end" is expected to be exclusive. And I think that it's better to make the parameter "end" of numa_fill_memblks() consistent with other numa_memblk family functions, that is, exclusive.

Agree.

Quoted 8 lines
Quoted 4 lines
arch/x86/mm/numa.c +0 -0
 	/*
 	 * The SRAT may have already described NUMA details for all,

base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86

--
Best Regards,
Huang, Ying

#6 lore raw
In reply to Huang, Ying
Quoted 33 lines · Ying
Quoted 31 lines · <[email protected]> writes:

From: Alison Schofield <[email protected]>

When the BIOS only partially describes a CFMWS Window in the SRAT the acpi driver uses numa_fill_memblks() to extend existing memblk(s) to fill the entire CFMWS Window, thereby applying the proximity domain to the entire CFMWS.

The calculation of the memblks to fill has an off-by-one error, that causes numa_init to fail when it sees the overlap:

[] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff]
[] ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0xffffffffff]
[] ACPI: SRAT: Node 1 PXM 1 [mem 0x10000000000-0x1ffffffffff]
[] node 0 [mem 0x100000000-0xffffffffff] overlaps with node 1 [mem 0x100000000-0x1ffffffffff]

Fix by making the 'end' parameter to numa_fill_memblks() exclusive.

Fixes 8f1004679987 ("ACPI/NUMA: Apply SRAT proximity domain to entire CFMWS window")
---
 drivers/acpi/numa/srat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/acpi/numa/srat.c +0 -0
index 12f330b0eac0..b99062f7c412 100644
@@ -308,7 +308,7 @@ static int __init acpi_parse_cfmws(union acpi_subtable_headers *header,
 
Quoted 7 lines

drivers/acpi/numa/srat.c static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, · line 311

Quoted 4 lines
drivers/acpi/numa/srat.c +1 -1
 	cfmws = (struct acpi_cedt_cfmws *)header;
 	start = cfmws->base_hpa;
-	end = cfmws->base_hpa + cfmws->window_size;
+	end = cfmws->base_hpa + cfmws->window_size - 1;

IIUC, this makes end inclusive instead of exclusive. That is, from "[start, end)" to "[start, end]". Am I right?

"Exlcusive" => Exclude the last value ("cfmws->base_hpa + cfmws->window_size") from the range:
[cfmws->base_hpa, cfmws->window_size)

Quoted 2 lines

If so, later in the function, "numa_add_memblk(node, start, end)" is called, where "end" is expected to be exclusive.

Good eye, yes.

Quoted 3 lines

And I think that it's better to make the parameter "end" of numa_fill_memblks() consistent with other numa_memblk family functions, that is, exclusive.

s/exclusive/inclusive/, but yes for the immediate fix. It is unfortunate that "end" here is different than the typical meaning of end as in 'struct resource' and 'struct range'. I would love to see a conversion along the lines of:

arch/x86/mm/numa_internal.h +1 -2
index 86860f279662..8a0eeac9d64b 100644
@@ -6,8 +6,7 @@
 #include <asm/numa.h>
 
 struct numa_memblk {
-       u64                     start;
-       u64                     end;
+       struct range            r;
        int                     nid;
 };
 

...and then intoduce helpers in include/linux/range.h to remove the error-prone open-coded versions of all the range comparison math.