Skip to content

cxl: Address translation support, part 2: Generic support and AMD Zen5 platform enablement

Robert Richter linux-cxl lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v2 Latest15 patches 36 repliesIn review

Changes in v2 from the cover letter · 20 lines

* rebased onto cxl/next,
* split of v1 in two parts:
* removed cleanups and updates from this series to post them as a separate series (Dave),
* this part 2 applies on top of part 1, v3,
* added tags to SOB chain,
* reworked architecture, vendor and platform setup (Jonathan):
* added patch "cxl/x86: Prepare for architectural platform setup",
* added function arch_cxl_port_platform_setup() plus a __weak versions for archs other than x86,
* moved code to core/x86,
* added comment to cxl_to_hpa_fn (Ben),
* updated year in copyright statement (Ben),
* cxl_port_calc_hpa(): Removed HPA check for zero (Jonathan), return 1 if modified,
* cxl_port_calc_pos(): Updated description and wording (Ben),
* added sereral patches around interleaving and SPA calculation in cxl_endpoint_decoder_initialize(),
* reworked iterator in cxl_endpoint_decoder_initialize() (Gregory),
* fixed region interleaving parameters() (Alison),
* fixed check in cxl_region_attach() (Alison),
* Clarified in coverletter that not all ports in a system must implement the to_hpa() callback (Terry).

[1] https://lore.kernel.org/linux-cxl/[email protected]/
[2] https://lore.kernel.org/linux-cxl/[email protected]/

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/15cxl: Modify address translation callback for generic usedrivers/cxl/acpi.c drivers/cxl/core/region.c drivers/cxl/cxl.h+9 -72 replies
2/15cxl: Introduce callback to translate an HPA range from a port to its parentdrivers/cxl/cxl.h+11 -01 reply
3/15cxl/region: Factor out code for interleaving calculationsdrivers/cxl/core/region.c+43 -203 replies
4/15cxl/region: Calculate endpoint's region position during initdrivers/cxl/core/region.c+21 -48 replies
5/15cxl/region: Calculate and store the SPA range of an endpointdrivers/cxl/core/region.c drivers/cxl/cxl.h+68 -144 replies
6/15cxl/region: Use endpoint's HPA range to find the port's decoderdrivers/cxl/core/region.c+3 -43 replies
7/15cxl/region: Use translated HPA ranges to find the port's decoderdrivers/cxl/core/region.c+58 -21 reply
8/15cxl/region: Use the endpoint's SPA range to find a regiondrivers/cxl/core/region.c+1 -23 replies
9/15cxl/region: Use the endpoint's SPA range to create a regiondrivers/cxl/core/region.c+3 -33 replies
10/15cxl/region: Use root decoders interleaving parameters to create a regiondrivers/cxl/core/region.c drivers/cxl/cxl.h+36 -55 replies
11/15cxl/region: Use the endpoint's SPA range to check a regiondrivers/cxl/core/region.c+8 -42 replies
12/15cxl/region: Lock decoders that need address translationdrivers/cxl/core/region.c+11 -01 reply
13/15cxl/x86: Prepare for architectural platform setupdrivers/cxl/core/Makefile drivers/cxl/core/core.h drivers/cxl/core/port.c +1 more+19 -01 reply
14/15cxl/amd: Enable Zen5 address translation using ACPI PRMTdrivers/cxl/Kconfig drivers/cxl/core/Makefile drivers/cxl/core/core.h +3 more+293 -17 replies
15/15MAINTAINERS: CXL: Add entry for AMD platform support (CXL_AMD)MAINTAINERS+7 -00 replies

Reviews trailers per patch

11 of 15 patches have a Reviewed-by or Acked-by.

PatchGregory PriceJonathan CameronJoshua Hahn
SeriesT-b
1/15R-b
2/15R-b
3/15R-b
4/15
5/15R-b
6/15R-b
7/15R-b
8/15R-b R-b
9/15R-b R-b
10/15
11/15R-b
12/15
13/15R-b
14/15T-b R-b
15/15

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v2 thread

Cover letter [PATCH v2 00/15] cxl: Address translation support, part 2: Generic support and AMD Zen5 platform enablement

This patch set adds support of address translation and enables this for AMD Zen5 platforms. This is a new appoach in response to an earlier attempt to implement CXL address translation [1] and the comments on it, esp. Dan's [2]. Dan suggested to solve this by walking the port hierarchy from the host port to the host bridge. When crossing memory domains from one port to the other, HPA translations are applied using a callback function to handle platform specifics.

The CXL driver currently does not implement address translation which assumes the host physical addresses (HPA) and system physical addresses (SPA) are equal.

Systems with different HPA and SPA addresses need address translation. If this is the case, the hardware addresses esp. used in the HDM decoder configurations are different to the system's or parent port address ranges. E.g. AMD Zen5 systems may be configured to use 'Normalized addresses'. Then, CXL endpoints have their own physical address base which is not the same as the SPA used by the CXL host bridge. Thus, addresses need to be translated from the endpoint's to its CXL host bridge's address range.

To enable address translation, the endpoint's HPA range must be translated to each of the parent port's address ranges up to the root decoder. This is implemented by traversing the decoder and port hierarchy from the endpoint up to the root port and applying platform specific translation functions to determine the next HPA range of the parent port where needed:

if (cxl_port->to_hpa)

    hpa = cxl_port->to_hpa(cxl_decoder, hpa)

A callback is introduced to translate an HPA range from a port to its parent. Not all ports in the hierarchy must implement this function, e.g. on Zen5 only the first root or switch port that connects the endpoints require address translation.

The root port's HPA range is equivalent to the system's SPA range and can then be used to find an endpoint's root port and region.

Also, translated HPA ranges must be used to calculate the endpoint position in the region.

Once the region was found, the decoders of all ports between the endpoint and the root port need to be found based on the translated HPA. Configuration checks and interleaving setup must be modified as necessary to support address translation.

Note that only auto-discovery of decoders is supported. Thus, decoders are locked and cannot be configured manually.

Finally, Zen5 address translation is enabled using ACPI PRMT.

This series bases on:

[PATCH v3 00/18] cxl: Address translation support, part 1: Cleanups and refactoring

Purpose of patches:
* Patches #1-#2: Introduction of address translation callback,
* Patches #3-#12: Functional changes for address translation (common code).
* #13: Architectural platform setup
* Patch #15, #15: AMD Zen5 address translation.

V2:
* rebased onto cxl/next,
* split of v1 in two parts:
* removed cleanups and updates from this series to post them as a separate series (Dave),
* this part 2 applies on top of part 1, v3,
* added tags to SOB chain,
* reworked architecture, vendor and platform setup (Jonathan):
* added patch "cxl/x86: Prepare for architectural platform setup",
* added function arch_cxl_port_platform_setup() plus a __weak versions for archs other than x86,
* moved code to core/x86,
* added comment to cxl_to_hpa_fn (Ben),
* updated year in copyright statement (Ben),
* cxl_port_calc_hpa(): Removed HPA check for zero (Jonathan), return 1 if modified,
* cxl_port_calc_pos(): Updated description and wording (Ben),
* added sereral patches around interleaving and SPA calculation in cxl_endpoint_decoder_initialize(),
* reworked iterator in cxl_endpoint_decoder_initialize() (Gregory),
* fixed region interleaving parameters() (Alison),
* fixed check in cxl_region_attach() (Alison),
* Clarified in coverletter that not all ports in a system must implement the to_hpa() callback (Terry).

[1] https://lore.kernel.org/linux-cxl/[email protected]/
[2] https://lore.kernel.org/linux-cxl/[email protected]/

Robert Richter (15):
cxl: Modify address translation callback for generic use
cxl: Introduce callback to translate an HPA range from a port to its
parent
cxl/region: Factor out code for interleaving calculations cxl/region: Calculate endpoint's region position during init cxl/region: Calculate and store the SPA range of an endpoint cxl/region: Use endpoint's HPA range to find the port's decoder cxl/region: Use translated HPA ranges to find the port's decoder cxl/region: Use the endpoint's SPA range to find a region cxl/region: Use the endpoint's SPA range to create a region cxl/region: Use root decoders interleaving parameters to create a
region
cxl/region: Use the endpoint's SPA range to check a region cxl/region: Lock decoders that need address translation cxl/x86: Prepare for architectural platform setup cxl/amd: Enable Zen5 address translation using ACPI PRMT
MAINTAINERS: CXL: Add entry for AMD platform support (CXL_AMD)

 MAINTAINERS                   |   7 +
 drivers/cxl/Kconfig           |   4 +
 drivers/cxl/acpi.c            |   4 +-
 drivers/cxl/core/Makefile     |   3 +
 drivers/cxl/core/core.h       |   4 +
 drivers/cxl/core/port.c       |   4 +
 drivers/cxl/core/region.c     | 323 ++++++++++++++++++++++++++++------
 drivers/cxl/core/x86/amd.c    | 259 +++++++++++++++++++++++++++
 drivers/cxl/core/x86/common.c |  14 ++
 drivers/cxl/cxl.h             |  19 +-
 10 files changed, 583 insertions(+), 58 deletions(-)
 create mode 100644 drivers/cxl/core/x86/amd.c
 create mode 100644 drivers/cxl/core/x86/common.c

base-commit: ed50d9abb9177ba106f47e2c48e1bf1804a6956c

Signature · 1 line
2.39.5