cxl/acpi: Limit XOR map application based on host bridge ways
[email protected]
maintainerPATCH 1/1 cxl/acpi: Limit XOR map application based on host bridge ways
From: Alison Schofield <[email protected]>
The CXL specification defines one set of XOR maps per Host Bridge Interleave Granularity (HBIG), but the number of maps to apply depends on the Host Bridge Interleave Ways (HBIW) for each region.
Currently, cxl_xor_hpa_to_spa() incorrectly applies all available XOR maps regardless of the region's host bridge interleave ways. This causes incorrect address translations when multiple CXL regions with the same HBIG but different HBIW's coexist.
Example scenario with 3 XOR maps defined for 256-byte granularity:
- 4-way interleave region: should apply 2 maps (was applying 3)
- 8-way interleave region: should apply 3 maps (correct)
- 12-way interleave region: should apply 2 maps (was applying 3)
Per CXL 3.2 Section 9.18.1.4 and Table 9-22, fix this by using a lookup table to determine the correct number of XOR maps based on host bridge interleave ways.
--- drivers/cxl/acpi.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
drivers/cxl/acpi.c +13 -3
@@ -16,6 +16,16 @@ struct cxl_cxims_data { u64 xormaps[] __counted_by(nr_maps); }; +/* +* There is one CXIMS, therefore one set of XOR maps, that all CXL Windows with +* the same host bridge granularity share. The number of maps to apply at address +* translation is based on the Host Bridge interleave ways of the CXL Window. +* CXL Specification 3.2 Section 9.18.1.4 and Table 9-22. +*/ +static const int hbiw_to_nr_maps[] = { + [1 ] =0 , [2 ] =1 , [3 ] =0 , [4 ] =2 , [6 ] =1 , [8 ] =3 , [12 ] =2 , [16 ] =4 +}; +static const guid_t acpi_cxl_qtg_id_guid =GUID_INIT (0xF365F9A6 ,0xA7DE ,0x4071 ,0xA6 ,0x6A ,0xB4 ,0x0C ,0x0B ,0x4F ,0x8E ,0x52 );
@@ -23,13 +33,13 @@ static const guid_t acpi_cxl_qtg_id_guid =static u64 cxl_xor_hpa_to_spa (struct cxl_root_decoder *cxlrd,u64 hpa) { +int nr_maps_to_apply = hbiw_to_nr_maps[cxlrd->cxlsd.nr_targets];struct cxl_cxims_data *cximsd = cxlrd->platform_data; -int hbiw = cxlrd->cxlsd.nr_targets;u64 val;int pos;/* No xormaps for host bridge interleave ways of 1 or 3 */ -if (hbiw ==1 || hbiw ==3 ) +if (!nr_maps_to_apply)return hpa;/*
@@ -47,7 +57,7 @@ static u64 cxl_xor_hpa_to_spa(struct cxl_root_decoder *cxlrd, u64 hpa)* bits results in val==0, if odd the XOR result is val==1. */ -for (int i =0 ; i < cximsd->nr_maps; i++) { +for (int i =0 ; i < nr_maps_to_apply; i++) {if (!cximsd->xormaps[i])continue ; pos =__ffs (cximsd->xormaps[i]);
base-commit: f11a5f89910a7ae970fbce4fdc02d86a8ba8570f
Signature · 1 line
2.37.3
Quoted 11 lines · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
The CXL specification defines one set of XOR maps per Host Bridge Interleave Granularity (HBIG), but the number of maps to apply depends on the Host Bridge Interleave Ways (HBIW) for each region.
Currently,
cxl_xor_hpa_to_spa()incorrectly applies all available XOR maps regardless of the region's host bridge interleave ways. This causes incorrect address translations when multiple CXL regions with the same HBIG but different HBIW's coexist.
Quoted 4 lines
Example scenario with 3 XOR maps defined for 256-byte granularity:
- 4-way interleave region: should apply 2 maps (was applying 3)
- 8-way interleave region: should apply 3 maps (correct)
- 12-way interleave region: should apply 2 maps (was applying 3)
Ah, makes sense, thanks for including this detail.
Do you have cxl_test bits to backstop this fix, even if only in local not-yet-upstream-ready form?
Quoted 16 lines
Per CXL 3.2 Section 9.18.1.4 and Table 9-22, fix this by using a lookup table to determine the correct number of XOR maps based on host bridge interleave ways.
Fixes 3b2fedcd75e3 ("cxl: Restore XOR'd position bits during address translation")Signed-off-by Alison Schofield <[email protected]>--- drivers/cxl/acpi.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)drivers/cxl/acpi.c +0
-0
drivers/cxl/acpi.c struct cxl_cxims_data { · line 25
Quoted 10 lines
drivers/cxl/acpi.c +7
-0@@ -16,6 +16,16 @@ struct cxl_cxims_data { u64 xormaps[] __counted_by(nr_maps); }; +/* +* There is one CXIMS, therefore one set of XOR maps, that all CXL Windows with +* the same host bridge granularity share. The number of maps to apply at address +* translation is based on the Host Bridge interleave ways of the CXL Window. +* CXL Specification 3.2 Section 9.18.1.4 and Table 9-22. +*/ +static const int hbiw_to_nr_maps[] = {
I would feel better if this was defined:
static const int hbiw_to_nr_maps[CXL_DECODER_MAX_INTERLEAVE] = {
...just to have this pop up in a grep if someone wondered about the implications of bumping up CXL_DECODER_MAX_INTERLEAVE.
Quoted 9 lines
drivers/cxl/acpi.c +3
-0+ [1] = 0, [2] = 1, [3] = 0, [4] = 2, [6] = 1, [8] = 3, [12] = 2, [16] = 4 +}; +static const guid_t acpi_cxl_qtg_id_guid =GUID_INIT (0xF365F9A6 ,0xA7DE ,0x4071 ,0xA6 ,0x6A ,0xB4 ,0x0C ,0x0B ,0x4F ,0x8E ,0x52 );@@ -23,13 +33,13 @@ static const guid_t acpi_cxl_qtg_id_guid =static u64 cxl_xor_hpa_to_spa (struct cxl_root_decoder *cxlrd,u64 hpa)
drivers/cxl/acpi.c static const guid_t acpi_cxl_qtg_id_guid = · line 42
Quoted 10 lines
drivers/cxl/acpi.c +2
-2{ +int nr_maps_to_apply = hbiw_to_nr_maps[cxlrd->cxlsd.nr_targets];struct cxl_cxims_data *cximsd = cxlrd->platform_data; -int hbiw = cxlrd->cxlsd.nr_targets;u64 val;int pos;/* No xormaps for host bridge interleave ways of 1 or 3 */ -if (hbiw ==1 || hbiw ==3 ) +if (!nr_maps_to_apply)
I wonder if this also should have a...
static unsigned long const nr_targets_valid_mask = BIT(16) | BIT(12) |
BIT(8) | BIT(6) | GENMASK(4, 1);
...
WARN_ON_ONCE(!test_bit(cxlrd->cxlsd.nr_targets, &nr_targets_valid_mask)));...because I could not immediately convince myself that all callers will have been careful to not pass invalid nr_targets?
Quoted 13 lines
drivers/cxl/acpi.c +1
-1return hpa;/* @@ -47,7 +57,7 @@ static u64 cxl_xor_hpa_to_spa(struct cxl_root_decoder *cxlrd, u64 hpa)* bits results in val==0, if odd the XOR result is val==1. */ -for (int i =0 ; i < cximsd->nr_maps; i++) { +for (int i =0 ; i < nr_maps_to_apply; i++) {if (!cximsd->xormaps[i])continue ; pos =__ffs (cximsd->xormaps[i]);
Quoted 1 line
base-commit:
f11a5f89910a7ae970fbce4fdc02d86a8ba8570f
Feel free to add:
...with or without the @nr_targets_valid_mask if you determine it is not needed.
Alison Schofield
maintainerQuoted 17 lines · Dan Williams
Quoted 11 lines · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
The CXL specification defines one set of XOR maps per Host Bridge Interleave Granularity (HBIG), but the number of maps to apply depends on the Host Bridge Interleave Ways (HBIW) for each region.
Currently,
cxl_xor_hpa_to_spa()incorrectly applies all available XOR maps regardless of the region's host bridge interleave ways. This causes incorrect address translations when multiple CXL regions with the same HBIG but different HBIW's coexist.Quoted 4 lines
Example scenario with 3 XOR maps defined for 256-byte granularity:
- 4-way interleave region: should apply 2 maps (was applying 3)
- 8-way interleave region: should apply 3 maps (correct)
- 12-way interleave region: should apply 2 maps (was applying 3)
Quoted 4 lines
Ah, makes sense, thanks for including this detail.
Do you have
cxl_testbits to backstop this fix, even if only in local not-yet-upstream-ready form?
Yes. It is testable with the cxl/test patchset and CXL unit test script posted on the list. [1][2]
It is that test that uncovered this bug. My previous, out of tree test, applied the xormaps based on hbiw, the kernel did not.
[1] [PATCH 0/3] CXL: Add a loadable module for address translation
https://lore.kernel.org/linux-cxl/[email protected]/
[2] [ndctl PATCH] cxl: Add cxl-translate.sh unit test
https://lore.kernel.org/linux-cxl/[email protected]/
Quoted 30 lines
Quoted 17 lines
Per CXL 3.2 Section 9.18.1.4 and Table 9-22, fix this by using a lookup table to determine the correct number of XOR maps based on host bridge interleave ways.
Fixes 3b2fedcd75e3 ("cxl: Restore XOR'd position bits during address translation")Signed-off-by Alison Schofield <[email protected]>--- drivers/cxl/acpi.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)drivers/cxl/acpi.c +0
-0drivers/cxl/acpi.c struct cxl_cxims_data { · line 25
Quoted 10 lines
drivers/cxl/acpi.c +7
-0@@ -16,6 +16,16 @@ struct cxl_cxims_data { u64 xormaps[] __counted_by(nr_maps); }; +/* +* There is one CXIMS, therefore one set of XOR maps, that all CXL Windows with +* the same host bridge granularity share. The number of maps to apply at address +* translation is based on the Host Bridge interleave ways of the CXL Window. +* CXL Specification 3.2 Section 9.18.1.4 and Table 9-22. +*/ +static const int hbiw_to_nr_maps[] = {I would feel better if this was defined:
Quoted 8 lines
static const int hbiw_to_nr_maps[CXL_DECODER_MAX_INTERLEAVE] = {
...just to have this pop up in a grep if someone wondered about the implications of bumping up
CXL_DECODER_MAX_INTERLEAVE.drivers/cxl/acpi.c struct cxl_cxims_data { · line 28
Quoted 3 lines
drivers/cxl/acpi.c +3
-0+ [1] = 0, [2] = 1, [3] = 0, [4] = 2, [6] = 1, [8] = 3, [12] = 2, [16] = 4 +}; +
Got it.
Quoted 19 lines
Quoted 6 lines
drivers/cxl/acpi.c +0
-0static const guid_t acpi_cxl_qtg_id_guid =GUID_INIT (0xF365F9A6 ,0xA7DE ,0x4071 ,0xA6 ,0x6A ,0xB4 ,0x0C ,0x0B ,0x4F ,0x8E ,0x52 );@@ -23,13 +33,13 @@ static const guid_t acpi_cxl_qtg_id_guid =static u64 cxl_xor_hpa_to_spa (struct cxl_root_decoder *cxlrd,u64 hpa)drivers/cxl/acpi.c static const guid_t acpi_cxl_qtg_id_guid = · line 42
Quoted 10 lines
drivers/cxl/acpi.c +2
-2{ +int nr_maps_to_apply = hbiw_to_nr_maps[cxlrd->cxlsd.nr_targets];struct cxl_cxims_data *cximsd = cxlrd->platform_data; -int hbiw = cxlrd->cxlsd.nr_targets;u64 val;int pos;/* No xormaps for host bridge interleave ways of 1 or 3 */ -if (hbiw ==1 || hbiw ==3 ) +if (!nr_maps_to_apply)I wonder if this also should have a...
Quoted 8 lines
static unsigned long const nr_targets_valid_mask = BIT(16) | BIT(12) | BIT(8) | BIT(6) | GENMASK(4, 1); ... WARN_ON_ONCE(!test_bit(cxlrd->cxlsd.nr_targets, &nr_targets_valid_mask)));...because I could not immediately convince myself that all callers will have been careful to not pass invalid
nr_targets?
Here I assumed the only caller is the validated caller in the CXL Driver. However, in the patches that add testability of this function [3] I did add guards against bad hbiw's. Let me compare what you suggest here to what I did in the later patch. I did think it would made sense to add the checks in tha patch that exposes it further.
[3] cxl/acpi: Make the XOR calculations available for testing
https://lore.kernel.org/linux-cxl/e2ffaa6a29117fd9832542e4ac314b46ab2d1fd8.1754291501.git.alison.schofield@intel.com/
Quoted 13 lines
Quoted 13 lines
drivers/cxl/acpi.c +1
-1return hpa;/* @@ -47,7 +57,7 @@ static u64 cxl_xor_hpa_to_spa(struct cxl_root_decoder *cxlrd, u64 hpa)* bits results in val==0, if odd the XOR result is val==1. */ -for (int i =0 ; i < cximsd->nr_maps; i++) { +for (int i =0 ; i < nr_maps_to_apply; i++) {if (!cximsd->xormaps[i])continue ; pos =__ffs (cximsd->xormaps[i]);
Quoted 5 lines
Quoted 1 line
base-commit:
f11a5f89910a7ae970fbce4fdc02d86a8ba8570fFeel free to add:
Reviewed-by Dan Williams <[email protected]>
Thanks for the review!
Quoted 3 lines
...with or without the @nr_targets_valid_mask if you determine it is not needed.
Alison Schofield
maintainerQuoted 3 lines · Alison Schofield
Quoted 2 lines · Dan Williams
Quoted 1 line · alison.schofield@ wrote:
From: Alison Schofield <[email protected]>
snip
Quoted 24 lines
Quoted 15 lines
Review
Quoted 12 lines
static u64
cxl_xor_hpa_to_spa(struct cxl_root_decoder *cxlrd, u64 hpa){(quoted diff) +2
-2+ int nr_maps_to_apply = hbiw_to_nr_maps[cxlrd->cxlsd.nr_targets]; struct cxl_cxims_data *cximsd = cxlrd->platform_data; - int hbiw = cxlrd->cxlsd.nr_targets; u64 val; int pos; /* No xormaps for host bridge interleave ways of 1 or 3 */ - if (hbiw == 1 || hbiw == 3) + if (!nr_maps_to_apply)I wonder if this also should have a...
Quoted 8 lines
static unsigned long const nr_targets_valid_mask = BIT(16) | BIT(12) | BIT(8) | BIT(6) | GENMASK(4, 1); ... WARN_ON_ONCE(!test_bit(cxlrd->cxlsd.nr_targets, &nr_targets_valid_mask)));...because I could not immediately convince myself that all callers will have been careful to not pass invalid
nr_targets?
Quoted 8 lines
Here I assumed the only caller is the validated caller in the CXL Driver. However, in the patches that add testability of this function [3] I did add guards against bad hbiw's. Let me compare what you suggest here to what I did in the later patch. I did think it would made sense to add the checks in tha patch that exposes it further.
[3] cxl/acpi: Make the XOR calculations available for testing
https://lore.kernel.org/linux-cxl/e2ffaa6a29117fd9832542e4ac314b46ab2d1fd8.1754291501.git.alison.schofield@intel.com/
I took another look at this and see that the pending test module is not the only caller to check. I'm thinking the check should be at the time nr_targets gets stored...like this:
drivers/cxl/core/port.c +4 -1
@@ -1777,7 +1777,10 @@ static int cxl_switch_decoder_init(struct cxl_port *port, struct cxl_switch_decoder *cxlsd, int nr_targets) { - if (nr_targets > CXL_DECODER_MAX_INTERLEAVE) +static unsigned long const nr_targets_valid_mask =BIT (16 ) |BIT (12 ) | +BIT (8 ) |BIT (6 ) |GENMASK (4 ,1 ); + +if (!test_bit (nr_targets, &nr_targets_valid_mask))return -EINVAL ; cxlsd->nr_targets = nr_targets;
So - you'll see further checking when these are refactored to support the test module. The above fix to cxl_switch_decoder_init() seems needed regardless of this patch or it's future refactoring.
Quoted 27 lines
Quoted 15 lines
Quoted 15 lines
(quoted diff) +1
-1return hpa; /*@@ -47,7 +57,7 @@ static u64 cxl_xor_hpa_to_spa(struct cxl_root_decoder *cxlrd, u64 hpa) * bits results in val==0, if odd the XOR result is val==1. */ - for (int i = 0; i < cximsd->nr_maps; i++) { + for (int i = 0; i < nr_maps_to_apply; i++) { if (!cximsd->xormaps[i]) continue; pos = __ffs(cximsd->xormaps[i]);Quoted 5 lines
Quoted 1 line
base-commit:
f11a5f89910a7ae970fbce4fdc02d86a8ba8570fFeel free to add:
Reviewed-by Dan Williams <[email protected]>Thanks for the review!
Quoted 3 lines
...with or without the @nr_targets_valid_mask if you determine it is not needed.
Alison Schofield wrote:
[..]
Quoted 17 lines
drivers/cxl/core/port.c +4
-1@@ -1777,7 +1777,10 @@ static int cxl_switch_decoder_init(struct cxl_port *port, struct cxl_switch_decoder *cxlsd, int nr_targets) { - if (nr_targets > CXL_DECODER_MAX_INTERLEAVE) +static unsigned long const nr_targets_valid_mask =BIT (16 ) |BIT (12 ) | +BIT (8 ) |BIT (6 ) |GENMASK (4 ,1 ); + +if (!test_bit (nr_targets, &nr_targets_valid_mask))return -EINVAL ; cxlsd->nr_targets = nr_targets;
Quoted 3 lines
So - you'll see further checking when these are refactored to support the test module. The above fix to
cxl_switch_decoder_init()seems needed regardless of this patch or it's future refactoring.
Sounds good, and I assume this validation will be a helper shared by cxl_acpi and the cxl_core. Seems small enough to be a static inline in cxl.h.
Alison Schofield
maintainerQuoted 19 lines · Dan Williams
Alison Schofield wrote:
[..]Quoted 17 lines
drivers/cxl/core/port.c +4
-1@@ -1777,7 +1777,10 @@ static int cxl_switch_decoder_init(struct cxl_port *port, struct cxl_switch_decoder *cxlsd, int nr_targets) { - if (nr_targets > CXL_DECODER_MAX_INTERLEAVE) +static unsigned long const nr_targets_valid_mask =BIT (16 ) |BIT (12 ) | +BIT (8 ) |BIT (6 ) |GENMASK (4 ,1 ); + +if (!test_bit (nr_targets, &nr_targets_valid_mask))return -EINVAL ; cxlsd->nr_targets = nr_targets;
Quoted 7 lines
Quoted 3 lines
So - you'll see further checking when these are refactored to support the test module. The above fix to
cxl_switch_decoder_init()seems needed regardless of this patch or it's future refactoring.Sounds good, and I assume this validation will be a helper shared by
cxl_acpiand thecxl_core. Seems small enough to be a static inline incxl.h.
Wasn't heading towards a common helper, but am now. Thanks!
Quoted 16 lines · On 8/4/25 1:23 AM, [email protected] wrote:
From: Alison Schofield <[email protected]>
The CXL specification defines one set of XOR maps per Host Bridge Interleave Granularity (HBIG), but the number of maps to apply depends on the Host Bridge Interleave Ways (HBIW) for each region.
Currently,
cxl_xor_hpa_to_spa()incorrectly applies all available XOR maps regardless of the region's host bridge interleave ways. This causes incorrect address translations when multiple CXL regions with the same HBIG but different HBIW's coexist.Example scenario with 3 XOR maps defined for 256-byte granularity:
- 4-way interleave region: should apply 2 maps (was applying 3)
- 8-way interleave region: should apply 3 maps (correct)
- 12-way interleave region: should apply 2 maps (was applying 3)
Quoted 6 lines
Per CXL 3.2 Section 9.18.1.4 and Table 9-22, fix this by using a lookup table to determine the correct number of XOR maps based on host bridge interleave ways.
Fixes 3b2fedcd75e3 ("cxl: Restore XOR'd position bits during address translation")Signed-off-by Alison Schofield <[email protected]>
Quoted 52 lines
--- drivers/cxl/acpi.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)drivers/cxl/acpi.c +13
-3@@ -16,6 +16,16 @@ struct cxl_cxims_data { u64 xormaps[] __counted_by(nr_maps); }; +/* +* There is one CXIMS, therefore one set of XOR maps, that all CXL Windows with +* the same host bridge granularity share. The number of maps to apply at address +* translation is based on the Host Bridge interleave ways of the CXL Window. +* CXL Specification 3.2 Section 9.18.1.4 and Table 9-22. +*/ +static const int hbiw_to_nr_maps[] = { + [1 ] =0 , [2 ] =1 , [3 ] =0 , [4 ] =2 , [6 ] =1 , [8 ] =3 , [12 ] =2 , [16 ] =4 +}; +static const guid_t acpi_cxl_qtg_id_guid =GUID_INIT (0xF365F9A6 ,0xA7DE ,0x4071 ,0xA6 ,0x6A ,0xB4 ,0x0C ,0x0B ,0x4F ,0x8E ,0x52 );@@ -23,13 +33,13 @@ static const guid_t acpi_cxl_qtg_id_guid =static u64 cxl_xor_hpa_to_spa (struct cxl_root_decoder *cxlrd,u64 hpa) { +int nr_maps_to_apply = hbiw_to_nr_maps[cxlrd->cxlsd.nr_targets];struct cxl_cxims_data *cximsd = cxlrd->platform_data; -int hbiw = cxlrd->cxlsd.nr_targets;u64 val;int pos;/* No xormaps for host bridge interleave ways of 1 or 3 */ -if (hbiw ==1 || hbiw ==3 ) +if (!nr_maps_to_apply)return hpa;/* @@ -47,7 +57,7 @@ static u64 cxl_xor_hpa_to_spa(struct cxl_root_decoder *cxlrd, u64 hpa)* bits results in val==0, if odd the XOR result is val==1. */ -for (int i =0 ; i < cximsd->nr_maps; i++) { +for (int i =0 ; i < nr_maps_to_apply; i++) {if (!cximsd->xormaps[i])continue ; pos =__ffs (cximsd->xormaps[i]);base-commit:
f11a5f89910a7ae970fbce4fdc02d86a8ba8570f
Jonathan Cameron
Reviewed-byQuoted 17 lines · On Wed, 6 Aug 2025 14:46:49 -0700 Dave Jiang <[email protected]> wrote:
Quoted 16 lines · On 8/4/25 1:23 AM, [email protected] wrote:
From: Alison Schofield <[email protected]>
The CXL specification defines one set of XOR maps per Host Bridge Interleave Granularity (HBIG), but the number of maps to apply depends on the Host Bridge Interleave Ways (HBIW) for each region.
Currently,
cxl_xor_hpa_to_spa()incorrectly applies all available XOR maps regardless of the region's host bridge interleave ways. This causes incorrect address translations when multiple CXL regions with the same HBIG but different HBIW's coexist.Example scenario with 3 XOR maps defined for 256-byte granularity:
- 4-way interleave region: should apply 2 maps (was applying 3)
- 8-way interleave region: should apply 3 maps (correct)
- 12-way interleave region: should apply 2 maps (was applying 3)
Quoted 8 lines
Quoted 6 lines
Per CXL 3.2 Section 9.18.1.4 and Table 9-22, fix this by using a lookup table to determine the correct number of XOR maps based on host bridge interleave ways.
Fixes 3b2fedcd75e3 ("cxl: Restore XOR'd position bits during address translation")Signed-off-by Alison Schofield <[email protected]>Reviewed-by Dave Jiang <[email protected]>