Skip to contentSkip to conversation

PCI: Consolidate TLP Log reading and printing

PATCH Superseded Ilpo Järvinen 2 participants 23 messages Sep 13, 2024 – Dec 13, 2024

Subsystems (2)

v6 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v6 cover PCI: Consolidate TLP Log reading and printing

This series has the remaining patches of the AER & DPC TLP Log handling consolidation and now includes a few minor improvements to the earlier accepted TLP Logging code.

v6:
- Preserve "AER:"/"DPC:" prefix on the printed TLP line
- New patch to add "AER:" also on other lines of the AER error dump

v5:
- Fix build with AER=y and DPC=n
- Match kerneldoc and function parameter name

v4:
- Added patches:
- Remove EXPORT of pcie_read_tlp_log()
- Moved code to pcie/tlp.c and build only with AER enabled
- Match variables in prototype and function
- int -> unsigned int conversion
- eetlp_prefix_max into own patch
- struct pcie_tlp_log param consistently called "log" within tlp.c
- Moved function prototypes into drivers/pci/pci.h
- Describe AER/DPC differences more clearly in one commit message

v3:
- Small rewording in a commit message

v2:
- Don't add EXPORT()s
- Don't include igxbe changes
- Don't use pr_cont() as it's incompatible with pci_err() and according to Andy Shevchenko should not be used in the first place

Ilpo Järvinen (8):
PCI: Don't expose pcie_read_tlp_log() outside of PCI subsystem
PCI: Move TLP Log handling to own file
PCI: Make pcie_read_tlp_log() signature same
PCI: Use unsigned int i in pcie_read_tlp_log()
PCI: Store # of supported End-End TLP Prefixes
PCI: Add TLP Prefix reading into pcie_read_tlp_log()
PCI: Create helper to print TLP Header and Prefix Log PCI/AER: Add prefixes to printouts

 drivers/pci/ats.c             |   2 +-
 drivers/pci/pci.c             |  28 ---------
 drivers/pci/pci.h             |   9 +++
 drivers/pci/pcie/Makefile     |   2 +-
 drivers/pci/pcie/aer.c        |  26 ++++----
 drivers/pci/pcie/dpc.c        |  14 ++---
 drivers/pci/pcie/tlp.c        | 109 ++++++++++++++++++++++++++++++++++
 drivers/pci/probe.c           |  14 +++--
 include/linux/aer.h           |   3 +-
 include/linux/pci.h           |   2 +-
 include/uapi/linux/pci_regs.h |   2 +
 11 files changed, 149 insertions(+), 62 deletions(-)
 create mode 100644 drivers/pci/pcie/tlp.c
Signature · 1 line
2.39.2
#1 lore raw

PATCH v6 1/8 PCI: Don't expose pcie_read_tlp_log() outside of PCI subsystem

pcie_read_tlp_log() was exposed by the commit 0a5a46a6a61b ("PCI/AER:
Generalize TLP Header Log reading") but this is now considered a mistake. No drivers outside of PCI subsystem should build their own diagnostic logging but should rely on PCI core doing it for them.

There's currently one driver (ixgbe) doing it independently which was the initial reason why the export was added but it was decided by the PCI maintainer that it's something that should be eliminated.

Remove the unwanted EXPORT of pcie_read_tlp_log() and remove it from include/linux/aer.h.

---
 drivers/pci/pci.c   | 1 -
 drivers/pci/pci.h   | 4 ++++
 include/linux/aer.h | 2 --
 3 files changed, 4 insertions(+), 3 deletions(-)
drivers/pci/pci.c +0 -1
index e3a49f66982d..378fc645424f 100644
@@ -1121,7 +1121,6 @@ int pcie_read_tlp_log(struct pci_dev *dev, int where,
 
 	return 0;
 }
-EXPORT_SYMBOL_GPL(pcie_read_tlp_log);
 
 /**
  * pci_restore_bars - restore a device's BAR values (e.g. after wake-up)
drivers/pci/pci.h +4 -0
index 79c8398f3938..b4fc6726eab3 100644
@@ -4,6 +4,8 @@
 
 #include <linux/pci.h>
 
+struct pcie_tlp_log;
+
 /* Number of possible devfns: 0.0 to 1f.7 inclusive */
 #define MAX_NR_DEVFNS 256
 
@@ -482,6 +484,8 @@ int aer_get_device_error_info(struct pci_dev *dev, struct aer_err_info *info);
 void aer_print_error(struct pci_dev *dev, struct aer_err_info *info);
 #endif	/* CONFIG_PCIEAER */
 
+int pcie_read_tlp_log(struct pci_dev *dev, int where, struct pcie_tlp_log *log);
+
 #ifdef CONFIG_PCIEPORTBUS
 /* Cached RCEC Endpoint Association */
 struct rcec_ea {
include/linux/aer.h +0 -2
index 4b97f38f3fcf..190a0a2061cd 100644
@@ -37,8 +37,6 @@ struct aer_capability_regs {
 	u16 uncor_err_source;
 };
 
-int pcie_read_tlp_log(struct pci_dev *dev, int where, struct pcie_tlp_log *log);
-
 #if defined(CONFIG_PCIEAER)
 int pci_aer_clear_nonfatal_status(struct pci_dev *dev);
 int pcie_aer_is_native(struct pci_dev *dev);
Signature · 1 line
2.39.2
Reviewed-by on v6 1/8
#2 lore raw

Re: [PATCH v6 1/8] PCI: Don't expose pcie_read_tlp_log() outside of PCI subsystem

Quoted 9 lines · On Fri, 13 Sep 2024 17:36:25 +0300 Ilpo Järvinen…

pcie_read_tlp_log() was exposed by the commit 0a5a46a6a61b ("PCI/AER:
Generalize TLP Header Log reading") but this is now considered a mistake. No drivers outside of PCI subsystem should build their own diagnostic logging but should rely on PCI core doing it for them.

There's currently one driver (ixgbe) doing it independently which was the initial reason why the export was added but it was decided by the PCI maintainer that it's something that should be eliminated.

Quoted 5 lines

Remove the unwanted EXPORT of pcie_read_tlp_log() and remove it from include/linux/aer.h.

FWIW LGTM

#3 lore raw

PATCH v6 2/8 PCI: Move TLP Log handling to own file

In reply to Ilpo Järvinen

TLP Log is PCIe feature and is processed only by AER and DPC. Configwise, DPC depends AER being enabled. In lack of better place, the TLP Log handling code was initially placed into pci.c but it can be easily placed in a separate file.

Move TLP Log handling code to own file under pcie/ subdirectory and include it only when AER is enabled.

---
 drivers/pci/pci.c         | 27 ---------------------------
 drivers/pci/pci.h         |  2 +-
 drivers/pci/pcie/Makefile |  2 +-
 drivers/pci/pcie/tlp.c    | 39 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 41 insertions(+), 29 deletions(-)
 create mode 100644 drivers/pci/pcie/tlp.c
drivers/pci/pci.c +0 -27
index 378fc645424f..47969712afe1 100644
@@ -1095,33 +1095,6 @@ static void pci_enable_acs(struct pci_dev *dev)
 	pci_write_config_word(dev, pos + PCI_ACS_CTRL, caps.ctrl);
 }
 
-/**
- * pcie_read_tlp_log - read TLP Header Log
- * @dev: PCIe device
- * @where: PCI Config offset of TLP Header Log
- * @tlp_log: TLP Log structure to fill
- *
- * Fill @tlp_log from TLP Header Log registers, e.g., AER or DPC.
- *
- * Return: 0 on success and filled TLP Log structure, <0 on error.
- */
-int pcie_read_tlp_log(struct pci_dev *dev, int where,
-		      struct pcie_tlp_log *tlp_log)
-{
-	int i, ret;
-
-	memset(tlp_log, 0, sizeof(*tlp_log));
-
-	for (i = 0; i < 4; i++) {
-		ret = pci_read_config_dword(dev, where + i * 4,
-					    &tlp_log->dw[i]);
-		if (ret)
-			return pcibios_err_to_errno(ret);
-	}
-
-	return 0;
-}
-
 /**
  * pci_restore_bars - restore a device's BAR values (e.g. after wake-up)
  * @dev: PCI device to have its BARs restored
drivers/pci/pci.h +1 -1
index b4fc6726eab3..532bb05a0411 100644
@@ -482,9 +482,9 @@ struct aer_err_info {
 
 int aer_get_device_error_info(struct pci_dev *dev, struct aer_err_info *info);
 void aer_print_error(struct pci_dev *dev, struct aer_err_info *info);
-#endif	/* CONFIG_PCIEAER */
 
 int pcie_read_tlp_log(struct pci_dev *dev, int where, struct pcie_tlp_log *log);
+#endif	/* CONFIG_PCIEAER */
 
 #ifdef CONFIG_PCIEPORTBUS
 /* Cached RCEC Endpoint Association */
drivers/pci/pcie/Makefile +1 -1
index 6461aa93fe76..591ef3177777 100644
@@ -7,7 +7,7 @@ pcieportdrv-y			:= portdrv.o rcec.o
 obj-$(CONFIG_PCIEPORTBUS)	+= pcieportdrv.o
 
 obj-y				+= aspm.o
-obj-$(CONFIG_PCIEAER)		+= aer.o err.o
+obj-$(CONFIG_PCIEAER)		+= aer.o err.o tlp.o
 obj-$(CONFIG_PCIEAER_INJECT)	+= aer_inject.o
 obj-$(CONFIG_PCIE_PME)		+= pme.o
 obj-$(CONFIG_PCIE_DPC)		+= dpc.o
drivers/pci/pcie/tlp.c new file +39 -0
new file mode 100644
index 000000000000..3f053cc62290
@@ -0,0 +1,39 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCIe TLP Log handling
+ *
+ * Copyright (C) 2024 Intel Corporation
+ */
+
+#include <linux/aer.h>
+#include <linux/pci.h>
+#include <linux/string.h>
+
+#include "../pci.h"
+
+/**
+ * pcie_read_tlp_log - read TLP Header Log
+ * @dev: PCIe device
+ * @where: PCI Config offset of TLP Header Log
+ * @tlp_log: TLP Log structure to fill
+ *
+ * Fill @tlp_log from TLP Header Log registers, e.g., AER or DPC.
+ *
+ * Return: 0 on success and filled TLP Log structure, <0 on error.
+ */
+int pcie_read_tlp_log(struct pci_dev *dev, int where,
+		      struct pcie_tlp_log *tlp_log)
+{
+	int i, ret;
+
+	memset(tlp_log, 0, sizeof(*tlp_log));
+
+	for (i = 0; i < 4; i++) {
+		ret = pci_read_config_dword(dev, where + i * 4,
+					    &tlp_log->dw[i]);
+		if (ret)
+			return pcibios_err_to_errno(ret);
+	}
+
+	return 0;
+}
Signature · 1 line
2.39.2
Reviewed-by on v6 2/8
#4 lore raw

Re: [PATCH v6 2/8] PCI: Move TLP Log handling to own file

Quoted 9 lines · On Fri, 13 Sep 2024 17:36:26 +0300 Ilpo Järvinen…

TLP Log is PCIe feature and is processed only by AER and DPC. Configwise, DPC depends AER being enabled. In lack of better place, the TLP Log handling code was initially placed into pci.c but it can be easily placed in a separate file.

Move TLP Log handling code to own file under pcie/ subdirectory and include it only when AER is enabled.

Can sort of see this might get used outside AER sometime in the future but for now this seems sensible.

Jonathan

#5 lore raw

PATCH v6 3/8 PCI: Make pcie_read_tlp_log() signature same

In reply to Ilpo Järvinen

pcie_read_tlp_log()'s prototype and function signature diverged due to changes made while applying.

Make the parameters of pcie_read_tlp_log() named identically.

---
 drivers/pci/pcie/tlp.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
drivers/pci/pcie/tlp.c +5 -6
index 3f053cc62290..2bf15749cd31 100644
@@ -15,22 +15,21 @@
  * pcie_read_tlp_log - read TLP Header Log
  * @dev: PCIe device
  * @where: PCI Config offset of TLP Header Log
- * @tlp_log: TLP Log structure to fill
+ * @log: TLP Log structure to fill
  *
- * Fill @tlp_log from TLP Header Log registers, e.g., AER or DPC.
+ * Fill @log from TLP Header Log registers, e.g., AER or DPC.
  *
  * Return: 0 on success and filled TLP Log structure, <0 on error.
  */
 int pcie_read_tlp_log(struct pci_dev *dev, int where,
-		      struct pcie_tlp_log *tlp_log)
+		      struct pcie_tlp_log *log)
 {
 	int i, ret;
 
-	memset(tlp_log, 0, sizeof(*tlp_log));
+	memset(log, 0, sizeof(*log));
 
 	for (i = 0; i < 4; i++) {
-		ret = pci_read_config_dword(dev, where + i * 4,
-					    &tlp_log->dw[i]);
+		ret = pci_read_config_dword(dev, where + i * 4, &log->dw[i]);
 		if (ret)
 			return pcibios_err_to_errno(ret);
 	}
Signature · 1 line
2.39.2
Reviewed-by on v6 3/8
#6 lore raw

Re: [PATCH v6 3/8] PCI: Make pcie_read_tlp_log() signature same

Quoted 6 lines · On Fri, 13 Sep 2024 17:36:27 +0300 Ilpo Järvinen…

pcie_read_tlp_log()'s prototype and function signature diverged due to changes made while applying.

Make the parameters of pcie_read_tlp_log() named identically.

Good to clean this up.

#7 lore raw

PATCH v6 4/8 PCI: Use unsigned int i in pcie_read_tlp_log()

In reply to Ilpo Järvinen

Loop variable i counting from 0 upwards does not need to be signed so make it unsigned int.

---
 drivers/pci/pcie/tlp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/pci/pcie/tlp.c +2 -1
index 2bf15749cd31..65ac7b5d8a87 100644
@@ -24,7 +24,8 @@
 int pcie_read_tlp_log(struct pci_dev *dev, int where,
 		      struct pcie_tlp_log *log)
 {
-	int i, ret;
+	unsigned int i;
+	int ret;
 
 	memset(log, 0, sizeof(*log));
 
Signature · 1 line
2.39.2
#8 lore raw

Re: [PATCH v6 4/8] PCI: Use unsigned int i in pcie_read_tlp_log()

Quoted 4 lines · On Fri, 13 Sep 2024 17:36:28 +0300 Ilpo Järvinen…

Loop variable i counting from 0 upwards does not need to be signed so make it unsigned int.

To me this is unnecessary noise, but it's harmless so no problem if you really like it.

Jonathan

Quoted 18 lines
---
 drivers/pci/pcie/tlp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/pci/pcie/tlp.c +2 -1
index 2bf15749cd31..65ac7b5d8a87 100644
@@ -24,7 +24,8 @@
 int pcie_read_tlp_log(struct pci_dev *dev, int where,
 		      struct pcie_tlp_log *log)
 {
-	int i, ret;
+	unsigned int i;
+	int ret;
 
 	memset(log, 0, sizeof(*log));
 
#9 lore raw

PATCH v6 5/8 PCI: Store # of supported End-End TLP Prefixes

In reply to Ilpo Järvinen

eetlp_prefix_path in the struct pci_dev tells if End-End TLP Prefixes are supported by the path or not, the value is only calculated if CONFIG_PCI_PASID is set.

The Max End-End TLP Prefixes field in the Device Capabilities Register 2 also tells how many (1-4) End-End TLP Prefixes are supported (PCIe r6 sec 7.5.3.15). The number of supported End-End Prefixes is useful for reading correct number of DWORDs from TLP Prefix Log register in AER capability (PCIe r6 sec 7.8.4.12).

Replace eetlp_prefix_path with eetlp_prefix_max and determine the number of supported End-End Prefixes regardless of CONFIG_PCI_PASID so that an upcoming commit generalizing TLP Prefix Log register reading does not have to read extra DWORDs for End-End Prefixes that never will be there.

---
 drivers/pci/ats.c             |  2 +-
 drivers/pci/probe.c           | 14 +++++++++-----
 include/linux/pci.h           |  2 +-
 include/uapi/linux/pci_regs.h |  1 +
 4 files changed, 12 insertions(+), 7 deletions(-)
drivers/pci/ats.c +1 -1
index c570892b2090..e13433dcfc82 100644
@@ -377,7 +377,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features)
 	if (WARN_ON(pdev->pasid_enabled))
 		return -EBUSY;
 
-	if (!pdev->eetlp_prefix_path && !pdev->pasid_no_tlp)
+	if (!pdev->eetlp_prefix_max && !pdev->pasid_no_tlp)
 		return -EINVAL;
 
 	if (!pasid)
drivers/pci/probe.c +9 -5
index b14b9876c030..0ab70ea6840c 100644
@@ -2228,8 +2228,8 @@ static void pci_configure_relaxed_ordering(struct pci_dev *dev)
 
 static void pci_configure_eetlp_prefix(struct pci_dev *dev)
 {
-#ifdef CONFIG_PCI_PASID
 	struct pci_dev *bridge;
+	unsigned int eetlp_max;
 	int pcie_type;
 	u32 cap;
 
@@ -2241,15 +2241,19 @@ static void pci_configure_eetlp_prefix(struct pci_dev *dev)
 		return;
 
 	pcie_type = pci_pcie_type(dev);
+
+	eetlp_max = FIELD_GET(PCI_EXP_DEVCAP2_EE_PREFIX_MAX, cap);
+	/* 00b means 4 */
+	eetlp_max = eetlp_max ?: 4;
+
 	if (pcie_type == PCI_EXP_TYPE_ROOT_PORT ||
 	    pcie_type == PCI_EXP_TYPE_RC_END)
-		dev->eetlp_prefix_path = 1;
+		dev->eetlp_prefix_max = eetlp_max;
 	else {
 		bridge = pci_upstream_bridge(dev);
-		if (bridge && bridge->eetlp_prefix_path)
-			dev->eetlp_prefix_path = 1;
+		if (bridge && bridge->eetlp_prefix_max)
+			dev->eetlp_prefix_max = eetlp_max;
 	}
-#endif
 }
 
 static void pci_configure_serr(struct pci_dev *dev)
include/linux/pci.h +1 -1
index 4cf89a4b4cbc..29b754d32f79 100644
@@ -398,7 +398,7 @@ struct pci_dev {
 					   supported from root to here */
 #endif
 	unsigned int	pasid_no_tlp:1;		/* PASID works without TLP Prefix */
-	unsigned int	eetlp_prefix_path:1;	/* End-to-End TLP Prefix */
+	unsigned int	eetlp_prefix_max:3;	/* Max # of End-End TLP Prefixes, 0=not supported */
 
 	pci_channel_state_t error_state;	/* Current connectivity state */
 	struct device	dev;			/* Generic device interface */
include/uapi/linux/pci_regs.h +1 -0
index 94c00996e633..b297bc30ddcb 100644
@@ -661,6 +661,7 @@
 #define  PCI_EXP_DEVCAP2_OBFF_MSG	0x00040000 /* New message signaling */
 #define  PCI_EXP_DEVCAP2_OBFF_WAKE	0x00080000 /* Re-use WAKE# for OBFF */
 #define  PCI_EXP_DEVCAP2_EE_PREFIX	0x00200000 /* End-End TLP Prefix */
+#define  PCI_EXP_DEVCAP2_EE_PREFIX_MAX	0x00c00000 /* Max End-End TLP Prefixes */
 #define PCI_EXP_DEVCTL2		0x28	/* Device Control 2 */
 #define  PCI_EXP_DEVCTL2_COMP_TIMEOUT	0x000f	/* Completion Timeout Value */
 #define  PCI_EXP_DEVCTL2_COMP_TMOUT_DIS	0x0010	/* Completion Timeout Disable */
Signature · 1 line
2.39.2
#10 lore raw

Re: [PATCH v6 5/8] PCI: Store # of supported End-End TLP Prefixes

Quoted 59 lines · On Fri, 13 Sep 2024 17:36:29 +0300 Ilpo Järvinen…

eetlp_prefix_path in the struct pci_dev tells if End-End TLP Prefixes are supported by the path or not, the value is only calculated if CONFIG_PCI_PASID is set.

The Max End-End TLP Prefixes field in the Device Capabilities Register 2 also tells how many (1-4) End-End TLP Prefixes are supported (PCIe r6 sec 7.5.3.15). The number of supported End-End Prefixes is useful for reading correct number of DWORDs from TLP Prefix Log register in AER capability (PCIe r6 sec 7.8.4.12).

Replace eetlp_prefix_path with eetlp_prefix_max and determine the number of supported End-End Prefixes regardless of CONFIG_PCI_PASID so that an upcoming commit generalizing TLP Prefix Log register reading does not have to read extra DWORDs for End-End Prefixes that never will be there.

---
 drivers/pci/ats.c             |  2 +-
 drivers/pci/probe.c           | 14 +++++++++-----
 include/linux/pci.h           |  2 +-
 include/uapi/linux/pci_regs.h |  1 +
 4 files changed, 12 insertions(+), 7 deletions(-)
drivers/pci/ats.c +1 -1
index c570892b2090..e13433dcfc82 100644
@@ -377,7 +377,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features)
 	if (WARN_ON(pdev->pasid_enabled))
 		return -EBUSY;
 
-	if (!pdev->eetlp_prefix_path && !pdev->pasid_no_tlp)
+	if (!pdev->eetlp_prefix_max && !pdev->pasid_no_tlp)
 		return -EINVAL;
 
 	if (!pasid)
drivers/pci/probe.c +5 -1
index b14b9876c030..0ab70ea6840c 100644
@@ -2228,8 +2228,8 @@ static void pci_configure_relaxed_ordering(struct pci_dev *dev)
 
 static void pci_configure_eetlp_prefix(struct pci_dev *dev)
 {
-#ifdef CONFIG_PCI_PASID
 	struct pci_dev *bridge;
+	unsigned int eetlp_max;
 	int pcie_type;
 	u32 cap;
 
@@ -2241,15 +2241,19 @@ static void pci_configure_eetlp_prefix(struct pci_dev *dev)
 		return;
 
 	pcie_type = pci_pcie_type(dev);
+
+	eetlp_max = FIELD_GET(PCI_EXP_DEVCAP2_EE_PREFIX_MAX, cap);
+	/* 00b means 4 */
+	eetlp_max = eetlp_max ?: 4;

drivers/pci/probe.c static void pci_configure_eetlp_prefix(struct pci_dev *dev) · line 2254

Quoted 10 lines
drivers/pci/probe.c +3 -3
+
 	if (pcie_type == PCI_EXP_TYPE_ROOT_PORT ||
 	    pcie_type == PCI_EXP_TYPE_RC_END)
-		dev->eetlp_prefix_path = 1;
+		dev->eetlp_prefix_max = eetlp_max;
 	else {
 		bridge = pci_upstream_bridge(dev);
-		if (bridge && bridge->eetlp_prefix_path)
-			dev->eetlp_prefix_path = 1;
+		if (bridge && bridge->eetlp_prefix_max)

What happens if they disagree? That is the bridge supports 2 and the device 3?

Quoted 8 lines
drivers/pci/probe.c +1 -1
+			dev->eetlp_prefix_max = eetlp_max;
 	}
-#endif
 }
 
 
#11 lore raw

Re: [PATCH v6 5/8] PCI: Store # of supported End-End TLP Prefixes

Quoted 67 lines · Jonathan Cameron
Quoted 64 lines · On Fri, 13 Sep 2024 17:36:29 +0300 Ilpo Järvinen…

eetlp_prefix_path in the struct pci_dev tells if End-End TLP Prefixes are supported by the path or not, the value is only calculated if CONFIG_PCI_PASID is set.

The Max End-End TLP Prefixes field in the Device Capabilities Register 2 also tells how many (1-4) End-End TLP Prefixes are supported (PCIe r6 sec 7.5.3.15). The number of supported End-End Prefixes is useful for reading correct number of DWORDs from TLP Prefix Log register in AER capability (PCIe r6 sec 7.8.4.12).

Replace eetlp_prefix_path with eetlp_prefix_max and determine the number of supported End-End Prefixes regardless of CONFIG_PCI_PASID so that an upcoming commit generalizing TLP Prefix Log register reading does not have to read extra DWORDs for End-End Prefixes that never will be there.

---
 drivers/pci/ats.c             |  2 +-
 drivers/pci/probe.c           | 14 +++++++++-----
 include/linux/pci.h           |  2 +-
 include/uapi/linux/pci_regs.h |  1 +
 4 files changed, 12 insertions(+), 7 deletions(-)
drivers/pci/ats.c +1 -1
index c570892b2090..e13433dcfc82 100644
@@ -377,7 +377,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features)
 	if (WARN_ON(pdev->pasid_enabled))
 		return -EBUSY;
 
-	if (!pdev->eetlp_prefix_path && !pdev->pasid_no_tlp)
+	if (!pdev->eetlp_prefix_max && !pdev->pasid_no_tlp)
 		return -EINVAL;
 
 	if (!pasid)
drivers/pci/probe.c +7 -2
index b14b9876c030..0ab70ea6840c 100644
@@ -2228,8 +2228,8 @@ static void pci_configure_relaxed_ordering(struct pci_dev *dev)
 
 static void pci_configure_eetlp_prefix(struct pci_dev *dev)
 {
-#ifdef CONFIG_PCI_PASID
 	struct pci_dev *bridge;
+	unsigned int eetlp_max;
 	int pcie_type;
 	u32 cap;
 
@@ -2241,15 +2241,19 @@ static void pci_configure_eetlp_prefix(struct pci_dev *dev)
 		return;
 
 	pcie_type = pci_pcie_type(dev);
+
+	eetlp_max = FIELD_GET(PCI_EXP_DEVCAP2_EE_PREFIX_MAX, cap);
+	/* 00b means 4 */
+	eetlp_max = eetlp_max ?: 4;
+
 	if (pcie_type == PCI_EXP_TYPE_ROOT_PORT ||
 	    pcie_type == PCI_EXP_TYPE_RC_END)
-		dev->eetlp_prefix_path = 1;
+		dev->eetlp_prefix_max = eetlp_max;
Quoted 8 lines

drivers/pci/probe.c static void pci_configure_eetlp_prefix(struct pci_dev *dev) · line 2254

Quoted 5 lines
drivers/pci/probe.c +1 -2
 	else {
 		bridge = pci_upstream_bridge(dev);
-		if (bridge && bridge->eetlp_prefix_path)
-			dev->eetlp_prefix_path = 1;
+		if (bridge && bridge->eetlp_prefix_max)

What happens if they disagree? That is the bridge supports 2 and the device 3?

That's a good question.

The current code obviously only checks if Prefixes are supported or not so the max value doesn't matter for the existing code.

I went to read spec and my reading from TLP logging point of view is that the device's own maximum matters even if it might never get >2 Prefixes (r6.1 2.2.10.4 & 6.2.4).

AFAIK, things happen on low level and there's no way to program this value. So it's not like the kernel is telling that hw must only use x Prefixes at most if that's what you were worried about.

But there are more things to consider, e.g., I noticed End-End TLP Prefix Blocking in DevCtl2 which might impact the existing usage too and is not checked by the kernel currently.

My interest here lies on cleaning this up so I'm not sure if functional changes such as considering End-End TLP Prefix Blocking really matter for some case or not. I can obviously easily add the code for that too if it's required for this series to be acceptable but I don't have a test case for it. My main goal with this TLP logging consolidation series is to get to a point that extending it to support Flit mode is easier.

There's also TLP Prefix Log Present which I think the reader should consider, which matters to another patch in this series and I'm going to alter the length based on that flag.

Signature · 1 line
 i.
#12 lore raw

PATCH v6 6/8 PCI: Add TLP Prefix reading into pcie_read_tlp_log()

In reply to Ilpo Järvinen

pcie_read_tlp_log() handles only 4 Header Log DWORDs but TLP Prefix Log (PCIe r6.1 secs 7.8.4.12 & 7.9.14.13) may also be present.

Generalize pcie_read_tlp_log() and struct pcie_tlp_log to handle also TLP Prefix Log. The relevant registers are formatted identically in AER and DPC Capability, but has these variations:

a) The offsets of TLP Prefix Log registers vary.
b) DPC RP PIO TLP Prefix Log register can be < 4 DWORDs.

Therefore callers must pass the offset of the TLP Prefix Log register and the entire length to pcie_read_tlp_log() to be able to read the correct number of TLP Prefix DWORDs from the correct offset.

---
 drivers/pci/pci.h             |  5 +++-
 drivers/pci/pcie/aer.c        |  4 ++-
 drivers/pci/pcie/dpc.c        | 13 +++++-----
 drivers/pci/pcie/tlp.c        | 49 +++++++++++++++++++++++++++++++----
 include/linux/aer.h           |  1 +
 include/uapi/linux/pci_regs.h |  1 +
 6 files changed, 59 insertions(+), 14 deletions(-)
drivers/pci/pci.h +4 -1
index 532bb05a0411..b47844b97428 100644
@@ -483,7 +483,9 @@ struct aer_err_info {
 int aer_get_device_error_info(struct pci_dev *dev, struct aer_err_info *info);
 void aer_print_error(struct pci_dev *dev, struct aer_err_info *info);
 
-int pcie_read_tlp_log(struct pci_dev *dev, int where, struct pcie_tlp_log *log);
+int pcie_read_tlp_log(struct pci_dev *dev, int where, int where2,
+		      unsigned int tlp_len, struct pcie_tlp_log *log);
+unsigned int aer_tlp_log_len(struct pci_dev *dev);
 #endif	/* CONFIG_PCIEAER */
 
 #ifdef CONFIG_PCIEPORTBUS
@@ -502,6 +504,7 @@ void pci_dpc_init(struct pci_dev *pdev);
 void dpc_process_error(struct pci_dev *pdev);
 pci_ers_result_t dpc_reset_link(struct pci_dev *pdev);
 bool pci_dpc_recovered(struct pci_dev *pdev);
+unsigned int dpc_tlp_log_len(struct pci_dev *dev);
 #else
 static inline void pci_save_dpc_state(struct pci_dev *dev) { }
 static inline void pci_restore_dpc_state(struct pci_dev *dev) { }
drivers/pci/pcie/aer.c +3 -1
index 13b8586924ea..651d0c72802a 100644
@@ -1245,7 +1245,9 @@ int aer_get_device_error_info(struct pci_dev *dev, struct aer_err_info *info)
 
 		if (info->status & AER_LOG_TLP_MASKS) {
 			info->tlp_header_valid = 1;
-			pcie_read_tlp_log(dev, aer + PCI_ERR_HEADER_LOG, &info->tlp);
+			pcie_read_tlp_log(dev, aer + PCI_ERR_HEADER_LOG,
+					  aer + PCI_ERR_PREFIX_LOG,
+					  aer_tlp_log_len(dev), &info->tlp);
 		}
 	}
 
drivers/pci/pcie/dpc.c +6 -7
index 2b6ef7efa3c1..7933b3cedb59 100644
@@ -190,7 +190,7 @@ pci_ers_result_t dpc_reset_link(struct pci_dev *pdev)
 static void dpc_process_rp_pio_error(struct pci_dev *pdev)
 {
 	u16 cap = pdev->dpc_cap, dpc_status, first_error;
-	u32 status, mask, sev, syserr, exc, log, prefix;
+	u32 status, mask, sev, syserr, exc, log;
 	struct pcie_tlp_log tlp_log;
 	int i;
 
@@ -217,20 +217,19 @@ static void dpc_process_rp_pio_error(struct pci_dev *pdev)
 
 	if (pdev->dpc_rp_log_size < 4)
 		goto clear_status;
-	pcie_read_tlp_log(pdev, cap + PCI_EXP_DPC_RP_PIO_HEADER_LOG, &tlp_log);
+	pcie_read_tlp_log(pdev, cap + PCI_EXP_DPC_RP_PIO_HEADER_LOG,
+			  cap + PCI_EXP_DPC_RP_PIO_TLPPREFIX_LOG,
+			  dpc_tlp_log_len(pdev), &tlp_log);
 	pci_err(pdev, "TLP Header: %#010x %#010x %#010x %#010x\n",
 		tlp_log.dw[0], tlp_log.dw[1], tlp_log.dw[2], tlp_log.dw[3]);
+	for (i = 0; i < pdev->dpc_rp_log_size - 5; i++)
+		pci_err(pdev, "TLP Prefix Header: dw%d, %#010x\n", i, tlp_log.prefix[i]);
 
 	if (pdev->dpc_rp_log_size < 5)
 		goto clear_status;
 	pci_read_config_dword(pdev, cap + PCI_EXP_DPC_RP_PIO_IMPSPEC_LOG, &log);
 	pci_err(pdev, "RP PIO ImpSpec Log %#010x\n", log);
 
-	for (i = 0; i < pdev->dpc_rp_log_size - 5; i++) {
-		pci_read_config_dword(pdev,
-			cap + PCI_EXP_DPC_RP_PIO_TLPPREFIX_LOG + i * 4, &prefix);
-		pci_err(pdev, "TLP Prefix Header: dw%d, %#010x\n", i, prefix);
-	}
  clear_status:
 	pci_write_config_dword(pdev, cap + PCI_EXP_DPC_RP_PIO_STATUS, status);
 }
drivers/pci/pcie/tlp.c +44 -5

70 lines not on this page · raw message

include/linux/aer.h +1 -0
index 190a0a2061cd..dc498adaa1c8 100644
@@ -20,6 +20,7 @@ struct pci_dev;
 
 struct pcie_tlp_log {
 	u32 dw[4];
+	u32 prefix[4];
 };
 
 struct aer_capability_regs {
include/uapi/linux/pci_regs.h +1 -0
index b297bc30ddcb..b6f9012a3fc4 100644
@@ -803,6 +803,7 @@
 #define  PCI_ERR_ROOT_FATAL_RCV		0x00000040 /* Fatal Received */
 #define  PCI_ERR_ROOT_AER_IRQ		0xf8000000 /* Advanced Error Interrupt Message Number */
 #define PCI_ERR_ROOT_ERR_SRC	0x34	/* Error Source Identification */
+#define PCI_ERR_PREFIX_LOG	0x38	/* TLP Prefix LOG Register (up to 16 bytes) */
 
 /* Virtual Channel */
 #define PCI_VC_PORT_CAP1	0x04
Signature · 1 line
2.39.2
Reviewed-by on v6 6/8
#13 lore raw

Re: [PATCH v6 6/8] PCI: Add TLP Prefix reading into pcie_read_tlp_log()

Quoted 7 lines · On Fri, 13 Sep 2024 17:36:30 +0300 Ilpo Järvinen…

pcie_read_tlp_log() handles only 4 Header Log DWORDs but TLP Prefix Log (PCIe r6.1 secs 7.8.4.12 & 7.9.14.13) may also be present.

Generalize pcie_read_tlp_log() and struct pcie_tlp_log to handle also TLP Prefix Log. The relevant registers are formatted identically in AER and DPC Capability, but has these variations:

Quoted 8 lines

a) The offsets of TLP Prefix Log registers vary.
b) DPC RP PIO TLP Prefix Log register can be < 4 DWORDs.

Therefore callers must pass the offset of the TLP Prefix Log register and the entire length to pcie_read_tlp_log() to be able to read the correct number of TLP Prefix DWORDs from the correct offset.

Trivial comments below

Would have been nice if they'd just made the formats have the same sized holes etc!

drivers/pci/pcie/tlp.c line 13

Quoted 8 lines
drivers/pci/pcie/tlp.c +0 -0
index 65ac7b5d8a87..def9dd7b73e8 100644
@@ -11,26 +11,65 @@
 
 /**
  * pcie_read_tlp_log - read TLP Header Log

Maybe update this to read TLP Header and Prefix Logs

drivers/pci/pcie/tlp.c line 46

Quoted 3 lines
drivers/pci/pcie/tlp.c +1 -0
  * @dev: PCIe device
  * @where: PCI Config offset of TLP Header Log
+ * @where2: PCI Config offset of TLP Prefix Log

Is it worth giving it a more specific name than where2? Possibly renaming where as well!

Quoted 12 lines
drivers/pci/pcie/tlp.c +3 -2
+ * @tlp_len: TLP Log length (Header Log + TLP Prefix Log in DWORDs)
  * @log: TLP Log structure to fill
  *
  * Fill @log from TLP Header Log registers, e.g., AER or DPC.
  *
  * Return: 0 on success and filled TLP Log structure, <0 on error.
  */
-int pcie_read_tlp_log(struct pci_dev *dev, int where,
-		      struct pcie_tlp_log *log)
+int pcie_read_tlp_log(struct pci_dev *dev, int where, int where2,
+		      unsigned int tlp_len, struct pcie_tlp_log *log)
 {
#14 lore raw

Re: [PATCH v6 6/8] PCI: Add TLP Prefix reading into pcie_read_tlp_log()

Quoted 17 lines · Jonathan Cameron
Quoted 14 lines · On Fri, 13 Sep 2024 17:36:30 +0300 Ilpo Järvinen…

pcie_read_tlp_log() handles only 4 Header Log DWORDs but TLP Prefix Log (PCIe r6.1 secs 7.8.4.12 & 7.9.14.13) may also be present.

Generalize pcie_read_tlp_log() and struct pcie_tlp_log to handle also TLP Prefix Log. The relevant registers are formatted identically in AER and DPC Capability, but has these variations:

a) The offsets of TLP Prefix Log registers vary.
b) DPC RP PIO TLP Prefix Log register can be < 4 DWORDs.

Therefore callers must pass the offset of the TLP Prefix Log register and the entire length to pcie_read_tlp_log() to be able to read the correct number of TLP Prefix DWORDs from the correct offset.

Quoted 7 lines
Quoted 1 line

Trivial comments below

Would have been nice if they'd just made the formats have the same sized holes etc!

That's not even the worst problem.

They managed to copy-paste most of the stuff into DPC (copy-paste is really obvious because the text still refers to AER in a DPC section :-)) but forgot to add a few capability fields into the DPC capability, most importantly, the bit that indicates whether TLP was logged in Flit mode or not And now we get to keep the pieces how to interpret the Log Registers (relates to the follow up series). :-(

Quoted 8 lines

drivers/pci/pcie/tlp.c line 13

Quoted 8 lines
drivers/pci/pcie/tlp.c +0 -0
index 65ac7b5d8a87..def9dd7b73e8 100644
@@ -11,26 +11,65 @@
 
 /**
  * pcie_read_tlp_log - read TLP Header Log
Quoted 7 lines

Maybe update this to read TLP Header and Prefix Logs

drivers/pci/pcie/tlp.c line 46

Quoted 3 lines
drivers/pci/pcie/tlp.c +1 -0
  * @dev: PCIe device
  * @where: PCI Config offset of TLP Header Log
+ * @where2: PCI Config offset of TLP Prefix Log

Is it worth giving it a more specific name than where2? Possibly renaming where as well!

Sure, why not.

Signature · 1 line
 i.
Quoted 12 lines
Quoted 12 lines
drivers/pci/pcie/tlp.c +3 -2
+ * @tlp_len: TLP Log length (Header Log + TLP Prefix Log in DWORDs)
  * @log: TLP Log structure to fill
  *
  * Fill @log from TLP Header Log registers, e.g., AER or DPC.
  *
  * Return: 0 on success and filled TLP Log structure, <0 on error.
  */
-int pcie_read_tlp_log(struct pci_dev *dev, int where,
-		      struct pcie_tlp_log *log)
+int pcie_read_tlp_log(struct pci_dev *dev, int where, int where2,
+		      unsigned int tlp_len, struct pcie_tlp_log *log)
 {
#15 lore raw

Re: [PATCH v6 6/8] PCI: Add TLP Prefix reading into pcie_read_tlp_log()

Quoted 45 lines · Ilpo Järvinen
Quoted 17 lines · Jonathan Cameron
Quoted 14 lines · On Fri, 13 Sep 2024 17:36:30 +0300 Ilpo Järvinen…

pcie_read_tlp_log() handles only 4 Header Log DWORDs but TLP Prefix Log (PCIe r6.1 secs 7.8.4.12 & 7.9.14.13) may also be present.

Generalize pcie_read_tlp_log() and struct pcie_tlp_log to handle also TLP Prefix Log. The relevant registers are formatted identically in AER and DPC Capability, but has these variations:

a) The offsets of TLP Prefix Log registers vary.
b) DPC RP PIO TLP Prefix Log register can be < 4 DWORDs.

Therefore callers must pass the offset of the TLP Prefix Log register and the entire length to pcie_read_tlp_log() to be able to read the correct number of TLP Prefix DWORDs from the correct offset.

Quoted 7 lines
Quoted 1 line

Trivial comments below

Would have been nice if they'd just made the formats have the same sized holes etc!

That's not even the worst problem.

They managed to copy-paste most of the stuff into DPC (copy-paste is really obvious because the text still refers to AER in a DPC section :-)) but forgot to add a few capability fields into the DPC capability, most importantly, the bit that indicates whether TLP was logged in Flit mode or not And now we get to keep the pieces how to interpret the Log Registers (relates to the follow up series). :-(

Quoted 9 lines

drivers/pci/pcie/tlp.c line 13

Quoted 8 lines
drivers/pci/pcie/tlp.c +0 -0
index 65ac7b5d8a87..def9dd7b73e8 100644
@@ -11,26 +11,65 @@
 
 /**
  * pcie_read_tlp_log - read TLP Header Log

Maybe update this to read TLP Header and Prefix Logs

Quoted 8 lines
Quoted 6 lines

drivers/pci/pcie/tlp.c line 46

Quoted 3 lines
drivers/pci/pcie/tlp.c +1 -0
  * @dev: PCIe device
  * @where: PCI Config offset of TLP Header Log
+ * @where2: PCI Config offset of TLP Prefix Log

Is it worth giving it a more specific name than where2? Possibly renaming where as well!

Sure, why not.

Hi again,

After doing this rename, I rebased the Flit mode series on top of it and realized there's one small problem with naming where2. It will be overloaded between TLP Prefix Log (Non-Flit mode) and extended TLP Header Log (DW5+) (Flit mode) so I'm not sure if there's really going to be a good name for it.

Signature · 1 line
 i.
#16 lore raw

Re: [PATCH v6 6/8] PCI: Add TLP Prefix reading into pcie_read_tlp_log()

Quoted 56 lines · On Thu, 12 Dec 2024 20:48:38 +0200 (EET) Ilpo Järvinen…
Quoted 45 lines · Ilpo Järvinen
Quoted 17 lines · Jonathan Cameron
Quoted 14 lines · On Fri, 13 Sep 2024 17:36:30 +0300 Ilpo Järvinen…

pcie_read_tlp_log() handles only 4 Header Log DWORDs but TLP Prefix Log (PCIe r6.1 secs 7.8.4.12 & 7.9.14.13) may also be present.

Generalize pcie_read_tlp_log() and struct pcie_tlp_log to handle also TLP Prefix Log. The relevant registers are formatted identically in AER and DPC Capability, but has these variations:

a) The offsets of TLP Prefix Log registers vary.
b) DPC RP PIO TLP Prefix Log register can be < 4 DWORDs.

Therefore callers must pass the offset of the TLP Prefix Log register and the entire length to pcie_read_tlp_log() to be able to read the correct number of TLP Prefix DWORDs from the correct offset.

Quoted 7 lines
Quoted 1 line

Trivial comments below

Would have been nice if they'd just made the formats have the same sized holes etc!

That's not even the worst problem.

They managed to copy-paste most of the stuff into DPC (copy-paste is really obvious because the text still refers to AER in a DPC section :-)) but forgot to add a few capability fields into the DPC capability, most importantly, the bit that indicates whether TLP was logged in Flit mode or not And now we get to keep the pieces how to interpret the Log Registers (relates to the follow up series). :-(

Quoted 9 lines

drivers/pci/pcie/tlp.c line 13

Quoted 8 lines
drivers/pci/pcie/tlp.c +0 -0
index 65ac7b5d8a87..def9dd7b73e8 100644
@@ -11,26 +11,65 @@  
 
 /**
  * pcie_read_tlp_log - read TLP Header Log  

Maybe update this to read TLP Header and Prefix Logs

Quoted 8 lines
Quoted 6 lines

drivers/pci/pcie/tlp.c line 46

Quoted 3 lines
drivers/pci/pcie/tlp.c +1 -0
  * @dev: PCIe device
  * @where: PCI Config offset of TLP Header Log
+ * @where2: PCI Config offset of TLP Prefix Log  

Is it worth giving it a more specific name than where2? Possibly renaming where as well!

Sure, why not.

Quoted 8 lines

Hi again,

After doing this rename, I rebased the Flit mode series on top of it and realized there's one small problem with naming where2. It will be overloaded between TLP Prefix Log (Non-Flit mode) and extended TLP Header Log (DW5+) (Flit mode) so I'm not sure if there's really going to be a good name for it.

Ah. I should read that series ;)

Absolutely fine - sometime generic naming is the best we can do!

Jonathan

#17 lore raw

PATCH v6 7/8 PCI: Create helper to print TLP Header and Prefix Log

In reply to Ilpo Järvinen

Add pcie_print_tlp_log() helper to print TLP Header and Prefix Log. Print End-End Prefixes only if they are non-zero.

Consolidate the few places which currently print TLP using custom formatting.

The first attempt used pr_cont() instead of building a string first but it turns out pr_cont() is not compatible with pci_err() and prints on a separate line. When I asked about this, Andy Shevchenko suggested pr_cont() should not be used in the first place (to eventually get rid of it) so pr_cont() is now replaced with building the string first.

---
 drivers/pci/pci.h      |  2 ++
 drivers/pci/pcie/aer.c | 10 ++--------
 drivers/pci/pcie/dpc.c |  5 +----
 drivers/pci/pcie/tlp.c | 31 +++++++++++++++++++++++++++++++
 4 files changed, 36 insertions(+), 12 deletions(-)
drivers/pci/pci.h +2 -0
index b47844b97428..cd6ada5d5980 100644
@@ -486,6 +486,8 @@ void aer_print_error(struct pci_dev *dev, struct aer_err_info *info);
 int pcie_read_tlp_log(struct pci_dev *dev, int where, int where2,
 		      unsigned int tlp_len, struct pcie_tlp_log *log);
 unsigned int aer_tlp_log_len(struct pci_dev *dev);
+void pcie_print_tlp_log(const struct pci_dev *dev,
+			const struct pcie_tlp_log *log, const char *pfx);
 #endif	/* CONFIG_PCIEAER */
 
 #ifdef CONFIG_PCIEPORTBUS
drivers/pci/pcie/aer.c +2 -8
index 651d0c72802a..6484e3a66a41 100644
@@ -664,12 +664,6 @@ static void pci_rootport_aer_stats_incr(struct pci_dev *pdev,
 	}
 }
 
-static void __print_tlp_header(struct pci_dev *dev, struct pcie_tlp_log *t)
-{
-	pci_err(dev, "  TLP Header: %08x %08x %08x %08x\n",
-		t->dw[0], t->dw[1], t->dw[2], t->dw[3]);
-}
-
 static void __aer_print_error(struct pci_dev *dev,
 			      struct aer_err_info *info)
 {
@@ -724,7 +718,7 @@ void aer_print_error(struct pci_dev *dev, struct aer_err_info *info)
 	__aer_print_error(dev, info);
 
 	if (info->tlp_header_valid)
-		__print_tlp_header(dev, &info->tlp);
+		pcie_print_tlp_log(dev, &info->tlp, dev_fmt("  "));
 
 out:
 	if (info->id && info->error_dev_num > 1 && info->id == id)
@@ -796,7 +790,7 @@ void pci_print_aer(struct pci_dev *dev, int aer_severity,
 			aer->uncor_severity);
 
 	if (tlp_header_valid)
-		__print_tlp_header(dev, &aer->header_log);
+		pcie_print_tlp_log(dev, &aer->header_log, dev_fmt("  "));
 
 	trace_aer_event(dev_name(&dev->dev), (status & ~mask),
 			aer_severity, tlp_header_valid, &aer->header_log);
drivers/pci/pcie/dpc.c +1 -4
index 7933b3cedb59..86ae1cf88787 100644
@@ -220,10 +220,7 @@ static void dpc_process_rp_pio_error(struct pci_dev *pdev)
 	pcie_read_tlp_log(pdev, cap + PCI_EXP_DPC_RP_PIO_HEADER_LOG,
 			  cap + PCI_EXP_DPC_RP_PIO_TLPPREFIX_LOG,
 			  dpc_tlp_log_len(pdev), &tlp_log);
-	pci_err(pdev, "TLP Header: %#010x %#010x %#010x %#010x\n",
-		tlp_log.dw[0], tlp_log.dw[1], tlp_log.dw[2], tlp_log.dw[3]);
-	for (i = 0; i < pdev->dpc_rp_log_size - 5; i++)
-		pci_err(pdev, "TLP Prefix Header: dw%d, %#010x\n", i, tlp_log.prefix[i]);
+	pcie_print_tlp_log(pdev, &tlp_log, dev_fmt(""));
 
 	if (pdev->dpc_rp_log_size < 5)
 		goto clear_status;
drivers/pci/pcie/tlp.c +31 -0
index def9dd7b73e8..097ac8514e96 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/aer.h>
+#include <linux/array_size.h>
 #include <linux/pci.h>
 #include <linux/string.h>
 
@@ -76,3 +77,33 @@ int pcie_read_tlp_log(struct pci_dev *dev, int where, int where2,
 
 	return 0;
 }
+
+/**
+ * pcie_print_tlp_log - Print TLP Header / Prefix Log contents
+ * @dev: PCIe device
+ * @log: TLP Log structure
+ * @pfx: String prefix (for print out indentation)
+ *
+ * Prints TLP Header and Prefix Log information held by @log.
+ */
+void pcie_print_tlp_log(const struct pci_dev *dev,
+			const struct pcie_tlp_log *log, const char *pfx)
+{
+	char buf[(10 + 1) * (4 + ARRAY_SIZE(log->prefix)) + 14 + 1];
+	unsigned int i;
+	int len;
+
+	len = scnprintf(buf, sizeof(buf), "%#010x %#010x %#010x %#010x",
+			log->dw[0], log->dw[1], log->dw[2], log->dw[3]);
+
+	if (log->prefix[0])
+		len += scnprintf(buf + len, sizeof(buf) - len, " E-E Prefixes:");
+	for (i = 0; i < ARRAY_SIZE(log->prefix); i++) {
+		if (!log->prefix[i])
+			break;
+		len += scnprintf(buf + len, sizeof(buf) - len,
+				 " %#010x", log->prefix[i]);
+	}
+
+	pci_err(dev, "%sTLP Header: %s\n", pfx, buf);
+}
Signature · 1 line
2.39.2
Reviewed-by on v6 7/8
#18 lore raw

Re: [PATCH v6 7/8] PCI: Create helper to print TLP Header and Prefix Log

Quoted 6 lines · On Fri, 13 Sep 2024 17:36:31 +0300 Ilpo Järvinen…

Add pcie_print_tlp_log() helper to print TLP Header and Prefix Log. Print End-End Prefixes only if they are non-zero.

Consolidate the few places which currently print TLP using custom formatting.

Quoted 7 lines

The first attempt used pr_cont() instead of building a string first but it turns out pr_cont() is not compatible with pci_err() and prints on a separate line. When I asked about this, Andy Shevchenko suggested pr_cont() should not be used in the first place (to eventually get rid of it) so pr_cont() is now replaced with building the string first.

A couple of trivial things inline but looks good to me either way.

Quoted 13 lines
drivers/pci/pcie/tlp.c +1 -0
index def9dd7b73e8..097ac8514e96 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/aer.h>
+#include <linux/array_size.h>
 #include <linux/pci.h>
 #include <linux/string.h>
 

drivers/pci/pcie/tlp.c int pcie_read_tlp_log(struct pci_dev *dev, int where, int where2, · line 85

Quoted 9 lines
drivers/pci/pcie/tlp.c +6 -0
@@ -76,3 +77,33 @@ int pcie_read_tlp_log(struct pci_dev *dev, int where, int where2,
 
 	return 0;
 }
+
+/**
+ * pcie_print_tlp_log - Print TLP Header / Prefix Log contents
+ * @dev: PCIe device
+ * @log: TLP Log structure
+ * @pfx: String prefix (for print out indentation)

Code doesn't care if it is indentation or ponies. So does it make sense to say anything beyond String prefix?

drivers/pci/pcie/tlp.c int pcie_read_tlp_log(struct pci_dev *dev, int where, int where2, · line 92

Quoted 7 lines
drivers/pci/pcie/tlp.c +7 -0
+ *
+ * Prints TLP Header and Prefix Log information held by @log.
+ */
+void pcie_print_tlp_log(const struct pci_dev *dev,
+			const struct pcie_tlp_log *log, const char *pfx)
+{
+	char buf[(10 + 1) * (4 + ARRAY_SIZE(log->prefix)) + 14 + 1];

Can we associate the 14 with the prefixes string by having that as a const char * and using strlen() It was a tiny bit irritating to count the characters whilst reviewing this ;)

Quoted 17 lines
drivers/pci/pcie/tlp.c +17 -0
+	unsigned int i;
+	int len;
+
+	len = scnprintf(buf, sizeof(buf), "%#010x %#010x %#010x %#010x",
+			log->dw[0], log->dw[1], log->dw[2], log->dw[3]);
+
+	if (log->prefix[0])
+		len += scnprintf(buf + len, sizeof(buf) - len, " E-E Prefixes:");
+	for (i = 0; i < ARRAY_SIZE(log->prefix); i++) {
+		if (!log->prefix[i])
+			break;
+		len += scnprintf(buf + len, sizeof(buf) - len,
+				 " %#010x", log->prefix[i]);
+	}
+
+	pci_err(dev, "%sTLP Header: %s\n", pfx, buf);
+}
#19 lore raw

PATCH v6 8/8 PCI/AER: Add prefixes to printouts

In reply to Ilpo Järvinen

Only part of the AER diagnostic printouts use "AER:" prefix because they use low-level pci_printk() directly to allow selecting level.

Add "AER:" prefix to lines that are printed with pci_printk().

---
 drivers/pci/pcie/aer.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/pci/pcie/aer.c +6 -6
index 6484e3a66a41..c4ba4396a8dc 100644
@@ -685,8 +685,8 @@ static void __aer_print_error(struct pci_dev *dev,
 		if (!errmsg)
 			errmsg = "Unknown Error Bit";
 
-		pci_printk(level, dev, "   [%2d] %-22s%s\n", i, errmsg,
-				info->first_error == i ? " (First)" : "");
+		pci_printk(level, dev, "%s   [%2d] %-22s%s\n", dev_fmt(""), i,
+			   errmsg, info->first_error == i ? " (First)" : "");
 	}
 	pci_dev_aer_stats_incr(dev, info);
 }
@@ -708,12 +708,12 @@ void aer_print_error(struct pci_dev *dev, struct aer_err_info *info)
 
 	level = (info->severity == AER_CORRECTABLE) ? KERN_WARNING : KERN_ERR;
 
-	pci_printk(level, dev, "PCIe Bus Error: severity=%s, type=%s, (%s)\n",
-		   aer_error_severity_string[info->severity],
+	pci_printk(level, dev, "%sPCIe Bus Error: severity=%s, type=%s, (%s)\n",
+		   dev_fmt(""), aer_error_severity_string[info->severity],
 		   aer_error_layer[layer], aer_agent_string[agent]);
 
-	pci_printk(level, dev, "  device [%04x:%04x] error status/mask=%08x/%08x\n",
-		   dev->vendor, dev->device, info->status, info->mask);
+	pci_printk(level, dev, "%s  device [%04x:%04x] error status/mask=%08x/%08x\n",
+		   dev_fmt(""), dev->vendor, dev->device, info->status, info->mask);
 
 	__aer_print_error(dev, info);
 
Signature · 1 line
2.39.2
Reviewed-by on v6 8/8
#20 lore raw

Re: [PATCH v6 8/8] PCI/AER: Add prefixes to printouts

Quoted 6 lines · On Fri, 13 Sep 2024 17:36:32 +0300 Ilpo Järvinen…

Only part of the AER diagnostic printouts use "AER:" prefix because they use low-level pci_printk() directly to allow selecting level.

Add "AER:" prefix to lines that are printed with pci_printk().

Makes sense.

#21 lore raw
In reply to Ilpo Järvinen
Quoted 7 lines · Ilpo Järvinen

This series has the remaining patches of the AER & DPC TLP Log handling consolidation and now includes a few minor improvements to the earlier accepted TLP Logging code.

v6:
- Preserve "AER:"/"DPC:" prefix on the printed TLP line
- New patch to add "AER:" also on other lines of the AER error dump

Hi Bjorn,

A small reminder this series exists.

Signature · 1 line
 i.
Quoted 49 lines

v5:
- Fix build with AER=y and DPC=n
- Match kerneldoc and function parameter name

v4:
- Added patches:
- Remove EXPORT of pcie_read_tlp_log()
- Moved code to pcie/tlp.c and build only with AER enabled
- Match variables in prototype and function
- int -> unsigned int conversion
- eetlp_prefix_max into own patch
- struct pcie_tlp_log param consistently called "log" within tlp.c
- Moved function prototypes into drivers/pci/pci.h
- Describe AER/DPC differences more clearly in one commit message

v3:
- Small rewording in a commit message

v2:
- Don't add EXPORT()s
- Don't include igxbe changes
- Don't use pr_cont() as it's incompatible with pci_err() and according to Andy Shevchenko should not be used in the first place

Ilpo Järvinen (8):
PCI: Don't expose pcie_read_tlp_log() outside of PCI subsystem
PCI: Move TLP Log handling to own file
PCI: Make pcie_read_tlp_log() signature same
PCI: Use unsigned int i in pcie_read_tlp_log()
PCI: Store # of supported End-End TLP Prefixes
PCI: Add TLP Prefix reading into pcie_read_tlp_log()
PCI: Create helper to print TLP Header and Prefix Log PCI/AER: Add prefixes to printouts

 drivers/pci/ats.c             |   2 +-
 drivers/pci/pci.c             |  28 ---------
 drivers/pci/pci.h             |   9 +++
 drivers/pci/pcie/Makefile     |   2 +-
 drivers/pci/pcie/aer.c        |  26 ++++----
 drivers/pci/pcie/dpc.c        |  14 ++---
 drivers/pci/pcie/tlp.c        | 109 ++++++++++++++++++++++++++++++++++
 drivers/pci/probe.c           |  14 +++--
 include/linux/aer.h           |   3 +-
 include/linux/pci.h           |   2 +-
 include/uapi/linux/pci_regs.h |   2 +
 11 files changed, 149 insertions(+), 62 deletions(-)
 create mode 100644 drivers/pci/pcie/tlp.c
#22 lore raw
Quoted 6 lines · Ilpo Järvinen
Quoted 4 lines · Ilpo Järvinen

This series has the remaining patches of the AER & DPC TLP Log handling consolidation and now includes a few minor improvements to the earlier accepted TLP Logging code.

Quoted 7 lines
Quoted 3 lines

v6:
- Preserve "AER:"/"DPC:" prefix on the printed TLP line
- New patch to add "AER:" also on other lines of the AER error dump

Hi Bjorn,

A small reminder this series exists.

Another reminder about this series. :-)

Signature · 1 line
 i.