RFC about how to obtain PCIE TPH steer-tag on ARM64 platform
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The pcie_tph_get_cpu_st() function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], the cpu_uid should be "ACPI processor uid".
2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
Because the ARM64 platform generates steer-tag based on the MPIDR information (at least for the Kunpeng platform). Therefore, we have two option:
Option-1: convert logic core ID to ACPI process ID: use get_acpi_id_for_cpu()
to get ACPI process ID in pcie_tph_get_cpu_st() before invoke dsm [2], and BIOS/ACPI use process ID to get corresponding MPIDR, and then generate steer-tag from MPIDR.
Option-2: convert logic core ID to MPIDR: use cpu_logical_map() to get MPIDR
in pcie_tph_get_cpu_st() before invoke dsm, and BIOS/ACPI use it to generate steer-tag directly.
Option-1 complies with _DSM ECN, but requires BIOS/ACPI to maintain a mapping table from acpi_process_id to MPIDR.
Option 2 does not comply with _DSM ECN (if extension is required). But it is easy to implement and can be extended to the DT system (ACPI is not supported) I think.
Hope for your feedback.
With best regards,
Chengwen
[1] According to _DSM ECN, the input is defined as: "If the target
is a processor, then this field represents the ACPI Processor UID of the processor as specified in the MADT. If the target is a processor container, then this field represents the ACPI Processor UID of the processor container as specified in the PPTT"
[2] git diff about /drivers/pci/tph.c
@@ -289,6 +289,9 @@ int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type,
rp_acpi_handle = ACPI_HANDLE(rp->bus->bridge);
+#ifdef CONFIG_ARM64
+ cpu_uid = get_acpi_id_for_cpu(cpu_uid);
+#endif
if (tph_invoke_dsm(rp_acpi_handle, cpu_uid, &info) != AE_OK) {
*tag = 0;
return -EINVAL;Quoted 9 lines · fengchengwen
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The
pcie_tph_get_cpu_st()function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], thecpu_uidshould be "ACPI processor uid".
Quoted 5 lines
2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
PG?
Quoted 14 lines
Because the ARM64 platform generates steer-tag based on the MPIDR information (at least for the Kunpeng platform). Therefore, we have two option:
Option-1: convert logic core ID to ACPI process ID: use
get_acpi_id_for_cpu()to get ACPI process ID inpcie_tph_get_cpu_st()before invoke dsm [2], and BIOS/ACPI use process ID to get corresponding MPIDR, and then generate steer-tag from MPIDR.Option-2: convert logic core ID to MPIDR: use
cpu_logical_map()to
get MPIDR inpcie_tph_get_cpu_st()before invoke dsm, and BIOS/ACPI use it to generate steer-tag directly.
Quoted 6 lines
Option-1 complies with
_DSMECN, but requires BIOS/ACPI to maintain a mapping table fromacpi_process_idto MPIDR.Option 2 does not comply with
_DSMECN (if extension is required). But it is easy to implement and can be extended to the DT system (ACPI is not supported) I think.
Sounds like this would be of interest to any OS, not just Linux.
Possibly a topic for the PCI-SIG firmware working group
(https://members.pcisig.com/wg/Firmware/dashboard) or the ACPI spec working group (https://uefi.org/workinggroups)?
Quoted 20 lines
[1] According to
_DSMECN, the input is defined as: "If the target is a processor, then this field represents the ACPI Processor UID of the processor as specified in the MADT. If the target is a processor container, then this field represents the ACPI Processor UID of the processor container as specified in the PPTT"[2] git diff about /drivers/pci/tph.c
(quoted diff) +3
-0@@ -289,6 +289,9 @@ int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type, rp_acpi_handle = ACPI_HANDLE(rp->bus->bridge); +#ifdef CONFIG_ARM64 + cpu_uid = get_acpi_id_for_cpu(cpu_uid); +#endif if (tph_invoke_dsm(rp_acpi_handle, cpu_uid, &info) != AE_OK) { *tag = 0; return -EINVAL;
Quoted 10 lines · Bjorn Helgaas
Quoted 8 lines · fengchengwen
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The
pcie_tph_get_cpu_st()function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], thecpu_uidshould be "ACPI processor uid".
Quoted 17 lines
Quoted 5 lines
2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
PG?
Quoted 9 lines
Because the ARM64 platform generates steer-tag based on the MPIDR information (at least for the Kunpeng platform). Therefore, we have two option:
Option-1: convert logic core ID to ACPI process ID: use
get_acpi_id_for_cpu()to get ACPI process ID inpcie_tph_get_cpu_st()before invoke dsm [2], and BIOS/ACPI use process ID to get corresponding MPIDR, and then generate steer-tag from MPIDR.
In my opinion, if this is achievable in your BIOS/ACPI, it is clean vs. Option-2 and preferred.
Quoted 4 lines
Quoted 4 lines
Option-2: convert logic core ID to MPIDR: use
cpu_logical_map()to
get MPIDR inpcie_tph_get_cpu_st()before invoke dsm, and BIOS/ACPI use it to generate steer-tag directly.
This solution requires you to change the ECN and ratify it (as suggested by Bjorn below). The implementation can also be complicated.
Quoted 7 lines
Quoted 7 lines
Option-1 complies with
_DSMECN, but requires BIOS/ACPI to maintain a mapping table fromacpi_process_idto MPIDR.Option 2 does not comply with
_DSMECN (if extension is required). But it is easy to implement and can be extended to the DT system (ACPI is not supported) I think.
If you plan to revamp it, one (wild) idea is that ST retrieval can be extended to support:
1. ACPI _DSM
2. DT
3. Vendor specific
After that, your MPIDR solution can be plugged-in under (3).
Quoted 27 lines
Sounds like this would be of interest to any OS, not just Linux.
Possibly a topic for the PCI-SIG firmware working group
(https://members.pcisig.com/wg/Firmware/dashboard) or the ACPI spec working group (https://uefi.org/workinggroups)?Quoted 20 lines
[1] According to
_DSMECN, the input is defined as: "If the target is a processor, then this field represents the ACPI Processor UID of the processor as specified in the MADT. If the target is a processor container, then this field represents the ACPI Processor UID of the processor container as specified in the PPTT"[2] git diff about /drivers/pci/tph.c
(quoted diff) +3
-0@@ -289,6 +289,9 @@ int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type, rp_acpi_handle = ACPI_HANDLE(rp->bus->bridge); +#ifdef CONFIG_ARM64 + cpu_uid = get_acpi_id_for_cpu(cpu_uid); +#endif if (tph_invoke_dsm(rp_acpi_handle, cpu_uid, &info) != AE_OK) { *tag = 0; return -EINVAL;
Quoted 23 lines · Wei Huang
Quoted 22 lines · Bjorn Helgaas
Quoted 9 lines · fengchengwen
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The
pcie_tph_get_cpu_st()function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], thecpu_uidshould be "ACPI processor uid".Quoted 5 lines
2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
PG?
Quoted 4 lines
Because the ARM64 platform generates steer-tag based on the MPIDR information (at least for the Kunpeng platform). Therefore, we have two option:
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Option-1: convert logic core ID to ACPI process ID: use
get_acpi_id_for_cpu()to get ACPI process ID inpcie_tph_get_cpu_st()before invoke dsm [2], and BIOS/ACPI use process ID to get corresponding MPIDR, and then generate steer-tag from MPIDR.In my opinion, if this is achievable in your BIOS/ACPI, it is clean vs. Option-2 and preferred.
Yes, it's achievable for out BIOS/ACPI, and we will adopt this option.
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Option-2: convert logic core ID to MPIDR: use
cpu_logical_map()to
get MPIDR inpcie_tph_get_cpu_st()before invoke dsm, and BIOS/ACPI use it to generate steer-tag directly.This solution requires you to change the ECN and ratify it (as suggested by Bjorn below). The implementation can also be complicated.
Indeed, and it may take a long time.
@Wei, I have another question:
The cache is hierarchical, for example, L2 or L3. Does the DSM mode support the specified cache level? I checked the ECN doc and there are "Cache Reference" fields, but the kernel code don't enable them.
Quoted 44 lines
Quoted 8 lines
Quoted 8 lines
Option-1 complies with
_DSMECN, but requires BIOS/ACPI to maintain a mapping table fromacpi_process_idto MPIDR.Option 2 does not comply with
_DSMECN (if extension is required). But it is easy to implement and can be extended to the DT system (ACPI is not supported) I think.If you plan to revamp it, one (wild) idea is that ST retrieval can be extended to support:
1. ACPI_DSM
2. DT
3. Vendor specificAfter that, your MPIDR solution can be plugged-in under (3).
Quoted 27 lines
Sounds like this would be of interest to any OS, not just Linux.
Possibly a topic for the PCI-SIG firmware working group
(https://members.pcisig.com/wg/Firmware/dashboard) or the ACPI spec working group (https://uefi.org/workinggroups)?Quoted 20 lines
[1] According to
_DSMECN, the input is defined as: "If the target is a processor, then this field represents the ACPI Processor UID of the processor as specified in the MADT. If the target is a processor container, then this field represents the ACPI Processor UID of the processor container as specified in the PPTT"[2] git diff about /drivers/pci/tph.c
(quoted diff) +3
-0@@ -289,6 +289,9 @@ int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type, rp_acpi_handle = ACPI_HANDLE(rp->bus->bridge); +#ifdef CONFIG_ARM64 + cpu_uid = get_acpi_id_for_cpu(cpu_uid); +#endif if (tph_invoke_dsm(rp_acpi_handle, cpu_uid, &info) != AE_OK) { *tag = 0; return -EINVAL;
Quoted 40 lines · fengchengwen
Quoted 23 lines · Wei Huang
Quoted 22 lines · Bjorn Helgaas
Quoted 9 lines · fengchengwen
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The
pcie_tph_get_cpu_st()function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], thecpu_uidshould be "ACPI processor uid".Quoted 5 lines
2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
PG?
Quoted 4 lines
Because the ARM64 platform generates steer-tag based on the MPIDR information (at least for the Kunpeng platform). Therefore, we have two option:
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Option-1: convert logic core ID to ACPI process ID: use
get_acpi_id_for_cpu()to get ACPI process ID inpcie_tph_get_cpu_st()before invoke dsm [2], and BIOS/ACPI use process ID to get corresponding MPIDR, and then generate steer-tag from MPIDR.In my opinion, if this is achievable in your BIOS/ACPI, it is clean vs. Option-2 and preferred.
Yes, it's achievable for out BIOS/ACPI, and we will adopt this option.
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
Option-2: convert logic core ID to MPIDR: use
cpu_logical_map()to
get MPIDR inpcie_tph_get_cpu_st()before invoke dsm, and BIOS/ACPI use it to generate steer-tag directly.
Quoted 7 lines
Quoted 1 line
This solution requires you to change the ECN and ratify it (as suggested by Bjorn below). The implementation can also be complicated.
Indeed, and it may take a long time.
@Wei, I have another question:
The cache is hierarchical, for example, L2 or L3. Does the DSM mode support the specified cache level? I checked the ECN doc and there are "Cache Reference" fields, but the kernel code don't enable them.
ECN does provide the "Cache Reference" field as part of argument2. Our platforms don't use it and don't plan to use it soon. You can explore the use cases if your platforms need it.
I want to point out that PCIe TLP header doesn't have a field for cache level. So no matter what, TLP header will stuck with 8- or 16-bit STs.
Quoted 1 line
Quoted 45 lines · Wei Huang
Quoted 44 lines · fengchengwen
Quoted 23 lines · Wei Huang
Quoted 22 lines · Bjorn Helgaas
Quoted 9 lines · fengchengwen
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The
pcie_tph_get_cpu_st()function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], thecpu_uidshould be "ACPI processor uid".Quoted 5 lines
2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
PG?
Quoted 4 lines
Because the ARM64 platform generates steer-tag based on the MPIDR information (at least for the Kunpeng platform). Therefore, we have two option:
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Option-1: convert logic core ID to ACPI process ID: use
get_acpi_id_for_cpu()to get ACPI process ID inpcie_tph_get_cpu_st()before invoke dsm [2], and BIOS/ACPI use process ID to get corresponding MPIDR, and then generate steer-tag from MPIDR.In my opinion, if this is achievable in your BIOS/ACPI, it is clean vs. Option-2 and preferred.
Yes, it's achievable for out BIOS/ACPI, and we will adopt this option.
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Option-2: convert logic core ID to MPIDR: use
cpu_logical_map()to
get MPIDR inpcie_tph_get_cpu_st()before invoke dsm, and BIOS/ACPI use it to generate steer-tag directly.This solution requires you to change the ECN and ratify it (as suggested by Bjorn below). The implementation can also be complicated.
Indeed, and it may take a long time.
Quoted 7 lines
Quoted 3 lines
@Wei, I have another question:
The cache is hierarchical, for example, L2 or L3. Does the DSM mode support the specified cache level? I checked the ECN doc and there are "Cache Reference" fields, but the kernel code don't enable them.ECN does provide the "Cache Reference" field as part of argument2. Our platforms don't use it and don't plan to use it soon. You can explore the use cases if your platforms need it.
I want to point out that PCIe TLP header doesn't have a field for cache level. So no matter what, TLP header will stuck with 8- or 16-bit STs.
Okay, I will try to understand how to use "Cache Reference" as cache level parameter. I think this would give the user a more refined ability.
Quoted 4 lines
Quoted 2 lines
Quoted 10 lines · Bjorn Helgaas
Quoted 8 lines · fengchengwen
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The
pcie_tph_get_cpu_st()function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], thecpu_uidshould be "ACPI processor uid".
Quoted 7 lines
Quoted 5 lines
2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
PG?
partial good
Quoted 22 lines
Quoted 15 lines
Because the ARM64 platform generates steer-tag based on the MPIDR information (at least for the Kunpeng platform). Therefore, we have two option:
Option-1: convert logic core ID to ACPI process ID: use
get_acpi_id_for_cpu()to get ACPI process ID inpcie_tph_get_cpu_st()before invoke dsm [2], and BIOS/ACPI use process ID to get corresponding MPIDR, and then generate steer-tag from MPIDR.Option-2: convert logic core ID to MPIDR: use
cpu_logical_map()to
get MPIDR inpcie_tph_get_cpu_st()before invoke dsm, and BIOS/ACPI use it to generate steer-tag directly.Quoted 6 lines
Option-1 complies with
_DSMECN, but requires BIOS/ACPI to maintain a mapping table fromacpi_process_idto MPIDR.Option 2 does not comply with
_DSMECN (if extension is required). But it is easy to implement and can be extended to the DT system (ACPI is not supported) I think.
Quoted 5 lines
Sounds like this would be of interest to any OS, not just Linux.
Possibly a topic for the PCI-SIG firmware working group
(https://members.pcisig.com/wg/Firmware/dashboard) or the ACPI spec working group (https://uefi.org/workinggroups)?
Thanks
We finally decided to choose option-1.
Quoted 22 lines
Quoted 21 lines
[1] According to
_DSMECN, the input is defined as: "If the target is a processor, then this field represents the ACPI Processor UID of the processor as specified in the MADT. If the target is a processor container, then this field represents the ACPI Processor UID of the processor container as specified in the PPTT"[2] git diff about /drivers/pci/tph.c
(quoted diff) +3
-0@@ -289,6 +289,9 @@ int pcie_tph_get_cpu_st(struct pci_dev *pdev, enum tph_mem_type mem_type, rp_acpi_handle = ACPI_HANDLE(rp->bus->bridge); +#ifdef CONFIG_ARM64 + cpu_uid = get_acpi_id_for_cpu(cpu_uid); +#endif if (tph_invoke_dsm(rp_acpi_handle, cpu_uid, &info) != AE_OK) { *tag = 0; return -EINVAL;
Quoted 17 lines · fengchengwen
Quoted 16 lines · Bjorn Helgaas
Quoted 14 lines · fengchengwen
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The
pcie_tph_get_cpu_st()function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], thecpu_uidshould be "ACPI processor uid".2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
Quoted 3 lines
Quoted 1 line
PG?
partial good
I still don't know what "partial good" means :) Is that something from a spec?
Quoted 18 lines · Bjorn Helgaas
Quoted 17 lines · fengchengwen
Quoted 16 lines · Bjorn Helgaas
Quoted 14 lines · fengchengwen
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The
pcie_tph_get_cpu_st()function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], thecpu_uidshould be "ACPI processor uid".2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
PG?
partial good
I still don't know what "partial good" means :) Is that something from a spec?
Because of some issues (like manufacturing variances), certain circuits (e.g., some cores) might not work normally. For these cases, we can use the part at a reduced specification, which is the PG mentioned here.
Quoted 1 line
Quoted 21 lines · fengchengwen
Quoted 20 lines · Bjorn Helgaas
Quoted 19 lines · fengchengwen
Quoted 17 lines · Bjorn Helgaas
Quoted 9 lines · fengchengwen
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The
pcie_tph_get_cpu_st()function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], thecpu_uidshould be "ACPI processor uid".Quoted 5 lines
2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
PG?
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
partial good
I still don't know what "partial good" means :) Is that something from a spec?
Because of some issues (like manufacturing variances), certain circuits (e.g., some cores) might not work normally. For these cases, we can use the part at a reduced specification, which is the PG mentioned here.
Ah, got it :) If you go this direction, maybe you can expand that part of the commit log a bit, since I don't think "PG" or "partial good" is a widely-known term.
Quoted 27 lines · Bjorn Helgaas
Quoted 26 lines · fengchengwen
Quoted 24 lines · Bjorn Helgaas
Quoted 17 lines · fengchengwen
Quoted 16 lines · Bjorn Helgaas
Quoted 14 lines · fengchengwen
Hi all,
We want to enable PCIE TPH feature on ARM64 platform, but we encounter the following problem:
1. The
pcie_tph_get_cpu_st()function invokes the ACPI DSM method to obtain the steer-tag of the CPU. According to the definition of the DSM method [1], thecpu_uidshould be "ACPI processor uid".2. In the current implementation, the ACPI DSM method is invoked directly using the logical core number, which works on the x86 platform but does not work on the ARM64 platform because the logical core ID is not the same as the ACPI processor ID when the PG exists.
Quoted 3 lines
Quoted 1 line
PG?
partial good
I still don't know what "partial good" means :) Is that something from a spec?
Quoted 7 lines
Quoted 3 lines
Because of some issues (like manufacturing variances), certain circuits (e.g., some cores) might not work normally. For these cases, we can use the part at a reduced specification, which is the PG mentioned here.
Ah, got it :) If you go this direction, maybe you can expand that part of the commit log a bit, since I don't think "PG" or "partial good" is a widely-known term.
Alright, thank you for the reminder.