Skip to contentSkip to conversation

cxl/mem: Fix register block offset calculation

PATCH Superseded Ben Widawsky 3 participants 11 messages Apr 15, 2021 – May 19, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/3 cxl/mem: Fix register block offset calculation

The offset for the register block should be a 64K aligned value, and therefore FIELD_GET (which will shift) is not correct for the calculation.

From 8.1.9.1 of the CXL 2.0 spec:
A[31:16] of offset from the address contained by one of the Function's Base Address Registers to point to the base of the Register Block. Register Block Offset is 64K aligned. Hence A[15:0] is zero

Fix this by simply using a mask.

This wasn't found earlier because the primary development done in the QEMU environment only uses 0 offsets

Fixes 8adaf747c9f0b ("cxl/mem: Find device capabilities")
---
 drivers/cxl/mem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/cxl/mem.c +1 -1
index e3003f49b329..1b5078311f7d 100644
@@ -998,7 +998,7 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo,
 		return NULL;
 	}
 
-	offset = ((u64)reg_hi << 32) | FIELD_GET(CXL_REGLOC_ADDR_MASK, reg_lo);
+	offset = ((u64)reg_hi << 32) | (reg_lo & CXL_REGLOC_ADDR_MASK);
 	bar = FIELD_GET(CXL_REGLOC_BIR_MASK, reg_lo);
 
 	/* Basic sanity check that BAR is big enough */
Signature · 1 line
2.31.1
#1 lore raw

PATCH 2/3 cxl/mem: Print unknown capability IDs as hex

Trivial. The spec lists these as hex, so do the same here to make debugging easier.

Fixes 8adaf747c9f0b ("cxl/mem: Find device capabilities")
---
 drivers/cxl/mem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/cxl/mem.c +1 -1
index 1b5078311f7d..c05617b0ba4b 100644
@@ -939,7 +939,7 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm)
 			cxlm->memdev_regs = register_block;
 			break;
 		default:
-			dev_dbg(dev, "Unknown cap ID: %d (0x%x)\n", cap_id, offset);
+			dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);
 			break;
 		}
 	}
Signature · 1 line
2.31.1
maintainer
#2 lore raw

Re: [PATCH 2/3] cxl/mem: Print unknown capability IDs as hex

Quoted 17 lines · Ben Widawsky

Trivial. The spec lists these as hex, so do the same here to make debugging easier.

Fixes 8adaf747c9f0b ("cxl/mem: Find device capabilities")

---
 drivers/cxl/mem.c | 2 +-

 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/cxl/mem.c +0 -0
index 1b5078311f7d..c05617b0ba4b 100644
@@ -939,7 +939,7 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm)
  			cxlm->memdev_regs = register_block;
  			break;

  default:

drivers/cxl/mem.c static int cxl_mem_setup_regs(struct cxl_mem *cxlm) · line 942

Quoted 2 lines
drivers/cxl/mem.c +1 -1
-			dev_dbg(dev, "Unknown cap ID: %d (0x%x)\n", cap_id, offset);
+			dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);

Would %#x be better just for making it unambiguous? Maybe also change the adjacent 0x%x to %#x while at it.

Quoted 3 lines

  break;
  }
  }

#3 lore raw

PATCH 3/3 cxl/mem: Demarcate vendor specific capability IDs

In reply to Ben Widawsky

Vendor capabilities occupy 0x8000 to 0xFFFF according to CXL 2.0 spec 8.2.8.2.1 CXL Device Capabilities. While they are not defined by the spec, they are allowed and not "unknown". Call this detail out in the logs to let users easily distinguish the difference.

Fixes 8adaf747c9f0b ("cxl/mem: Find device capabilities")
---
 drivers/cxl/mem.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/cxl/mem.c +4 -1
index c05617b0ba4b..0909f73db994 100644
@@ -939,7 +939,10 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm)
 			cxlm->memdev_regs = register_block;
 			break;
 		default:
-			dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);
+			if (cap_id > 0x8000)
+				dev_dbg(dev, "Vendor cap ID: %x (0x%x)\n", cap_id, offset);
+			else
+				dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);
 			break;
 		}
 	}
Signature · 1 line
2.31.1
#4 lore raw

PATCH 3/3 cxl/mem: Demarcate vendor specific capability IDs

Vendor capabilities occupy 0x8000 to 0xFFFF according to CXL 2.0 spec 8.2.8.2.1 CXL Device Capabilities. While they are not defined by the spec, they are allowed and not "unknown". Call this detail out in the logs to let users easily distinguish the difference.

v2: Should be greater than or equal to (Ben)

Fixes 8adaf747c9f0b ("cxl/mem: Find device capabilities")
---
 drivers/cxl/mem.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/cxl/mem.c +4 -1
index c05617b0ba4b..28c7c29567b3 100644
@@ -939,7 +939,10 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm)
 			cxlm->memdev_regs = register_block;
 			break;
 		default:
-			dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);
+			if (cap_id >= 0x8000)
+				dev_dbg(dev, "Vendor cap ID: %x (0x%x)\n", cap_id, offset);
+			else
+				dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);
 			break;
 		}
 	}
Signature · 1 line
2.31.1
maintainer
#5 lore raw

Re: [PATCH 3/3] cxl/mem: Demarcate vendor specific capability IDs

Quoted 6 lines · Ben Widawsky

Vendor capabilities occupy 0x8000 to 0xFFFF according to CXL 2.0 spec 8.2.8.2.1 CXL Device Capabilities. While they are not defined by the spec, they are allowed and not "unknown". Call this detail out in the logs to let users easily distinguish the difference.

v2: Should be greater than or equal to (Ben)

If there's a v3, drop this to below the '---'. Otherwise note for Dan to drop when applying I guess :)

Quoted 23 lines
Fixes 8adaf747c9f0b ("cxl/mem: Find device capabilities")

---
 drivers/cxl/mem.c | 5 ++++-

 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/cxl/mem.c +0 -0
index c05617b0ba4b..28c7c29567b3 100644
@@ -939,7 +939,10 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm)
  			cxlm->memdev_regs = register_block;
  			break;

  default:

drivers/cxl/mem.c +4 -1
-			dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);
+			if (cap_id >= 0x8000)
+				dev_dbg(dev, "Vendor cap ID: %x (0x%x)\n", cap_id, offset);
+			else
+				dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);

  break;
  }
  }

#6 lore raw

Re: [PATCH 3/3] cxl/mem: Demarcate vendor specific capability IDs

Quoted 11 lines · Vishal L
Quoted 6 lines · Ben Widawsky

Vendor capabilities occupy 0x8000 to 0xFFFF according to CXL 2.0 spec 8.2.8.2.1 CXL Device Capabilities. While they are not defined by the spec, they are allowed and not "unknown". Call this detail out in the logs to let users easily distinguish the difference.

v2: Should be greater than or equal to (Ben)

If there's a v3, drop this to below the '---'. Otherwise note for Dan to drop when applying I guess :)

Thanks... This one is an old habit. I'll point out too I messed up the subject here.

Quoted 24 lines
Quoted 23 lines
Fixes 8adaf747c9f0b ("cxl/mem: Find device capabilities")
---
 drivers/cxl/mem.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/cxl/mem.c +4 -1
index c05617b0ba4b..28c7c29567b3 100644
@@ -939,7 +939,10 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm)
 cxlm->memdev_regs = register_block;
 break;
 default:
-dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);
+if (cap_id >= 0x8000)
+dev_dbg(dev, "Vendor cap ID: %x (0x%x)\n", cap_id, offset);
+else
+dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);
 break;
 }
 }
#7 lore raw

Re: [PATCH 3/3] cxl/mem: Demarcate vendor specific capability IDs

In reply to Ben Widawsky
Quoted 19 lines · Ben Widawsky

Vendor capabilities occupy 0x8000 to 0xFFFF according to CXL 2.0 spec 8.2.8.2.1 CXL Device Capabilities. While they are not defined by the spec, they are allowed and not "unknown". Call this detail out in the logs to let users easily distinguish the difference.

v2: Should be greater than or equal to (Ben)

Fixes 8adaf747c9f0b ("cxl/mem: Find device capabilities")
---
 drivers/cxl/mem.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/cxl/mem.c +0 -0
index c05617b0ba4b..28c7c29567b3 100644

drivers/cxl/mem.c static int cxl_mem_setup_regs(struct cxl_mem *cxlm) · line 945

Quoted 8 lines
drivers/cxl/mem.c +4 -1
@@ -939,7 +939,10 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm)
                        cxlm->memdev_regs = register_block;
                        break;
                default:
-                       dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);
+                       if (cap_id >= 0x8000)
+                               dev_dbg(dev, "Vendor cap ID: %x (0x%x)\n", cap_id, offset);
+                       else
+                               dev_dbg(dev, "Unknown cap ID: %x (0x%x)\n", cap_id, offset);

This wants the same %#x fixup that Vishal noted on patch2 [1], and I think it would be useful to clarify that the second number is indeed an offset: "Unknown cap_id: %#x offset: %#x\n"

[1]: http://lore.kernel.org/r/[email protected]

#8 lore raw

Re: [PATCH 3/3] cxl/mem: Demarcate vendor specific capability IDs

In reply to Ben Widawsky
Quoted 7 lines · Ben Widawsky

Vendor capabilities occupy 0x8000 to 0xFFFF according to CXL 2.0 spec 8.2.8.2.1 CXL Device Capabilities. While they are not defined by the spec, they are allowed and not "unknown". Call this detail out in the logs to let users easily distinguish the difference.

Fixes 8adaf747c9f0b ("cxl/mem: Find device capabilities")

Would need the abbrev length fixup if this was a fix, but I don't think I can justify this to Linus as a fix v5.12. It's new development for the next merge window.

Same comment for patch2.

#9 lore raw
In reply to Ben Widawsky
Quoted 11 lines · Ben Widawsky

The offset for the register block should be a 64K aligned value, and therefore FIELD_GET (which will shift) is not correct for the calculation.

From 8.1.9.1 of the CXL 2.0 spec:
A[31:16] of offset from the address contained by one of the Function's Base Address Registers to point to the base of the Register Block. Register Block Offset is 64K aligned. Hence A[15:0] is zero

Fix this by simply using a mask.

The above reads slightly funny to me, is this any clearer?

The "Register Offset Low" register of a "DVSEC Register Locator" contains the 64K aligned offset for the registers along with the BAR indicator and an id. The implementation was treating the "Register Block Offset Low" field a value rather than as a pre-aligned component of the 64-bit offset. So, just mask, don't mask and shift (FIELD_GET).

Quoted 5 lines

This wasn't found earlier because the primary development done in the QEMU environment only uses 0 offsets

Fixes 8adaf747c9f0b ("cxl/mem: Find device capabilities")

As I've learned, linux-next will flag this as the wrong format.

Fixes 8adaf747c9f0 ("cxl/mem: Find device capabilities")

...i.e. looks like your core.abbrev setting is 13 rather than 12 per Documentation/process/submitting-patches.rst

Other than that, fix looks good to me.

Quoted 21 lines
---
 drivers/cxl/mem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/cxl/mem.c +1 -2
index e3003f49b329..1b5078311f7d 100644
@@ -998,7 +998,7 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo,
                return NULL;
        }
 
-       offset = ((u64)reg_hi << 32) | FIELD_GET(CXL_REGLOC_ADDR_MASK, reg_lo);
+       offset = ((u64)reg_hi << 32) | (reg_lo & CXL_REGLOC_ADDR_MASK);
        bar = FIELD_GET(CXL_REGLOC_BIR_MASK, reg_lo);
 
        /* Basic sanity check that BAR is big enough */
--

2.31.1

#10 lore raw

PATCH v2 1/1 cxl/mem: Fix register block offset calculation

In reply to Ben Widawsky

The offset for the register block should be a 64K aligned value, and therefore FIELD_GET (which will shift) is not correct for the calculation.

From 8.1.9.1 of the CXL 2.0 spec:
A[31:16] of offset from the address contained by one of the Function's Base Address Registers to point to the base of the Register Block. Register Block Offset is 64K aligned. Hence A[15:0] is zero

The "Register Offset Low" register of a "DVSEC Register Locator" contains the 64K aligned offset for the registers along with the BAR indicator and an ID. The implementation was treating the "Register Block Offset Low" field a value rather than as a pre-aligned component of the 64-bit offset. So, just mask, don't mask and shift (FIELD_GET).

This wasn't found earlier because the primary development done in the QEMU environment only uses 0 offsets

Fixes 8adaf747c9f0 ("cxl/mem: Find device capabilities")
---
 drivers/cxl/mem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/cxl/mem.c +1 -1
index e3003f49b329..1b5078311f7d 100644
@@ -998,7 +998,7 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo,
 		return NULL;
 	}
 
-	offset = ((u64)reg_hi << 32) | FIELD_GET(CXL_REGLOC_ADDR_MASK, reg_lo);
+	offset = ((u64)reg_hi << 32) | (reg_lo & CXL_REGLOC_ADDR_MASK);
 	bar = FIELD_GET(CXL_REGLOC_BIR_MASK, reg_lo);
 
 	/* Basic sanity check that BAR is big enough */
Signature · 1 line
2.31.1