Skip to contentSkip to conversation

PCI: hv: Fix ring buffer size calculation

PATCH Superseded [email protected] 3 participants 4 messages Feb 15, 2024 – Feb 19, 2024

Subsystems (4)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 PCI: hv: Fix ring buffer size calculation

From: Michael Kelley <[email protected]>

For a physical PCI device that is passed through to a Hyper-V guest VM, current code specifies the VMBus ring buffer size as 4 pages. But this is an inappropriate dependency, since the amount of ring buffer space needed is unrelated to PAGE_SIZE. For example, on x86 the ring buffer size ends up as 16 Kbytes, while on ARM64 with 64 Kbyte pages, the ring size bloats to 256 Kbytes. The ring buffer for PCI pass-thru devices is used for only a few messages during device setup and removal, so any space above a few Kbytes is wasted.

Fix this by declaring the ring buffer size to be a fixed 16 Kbytes. Furthermore, use the VMBUS_RING_SIZE() macro so that the ring buffer header is properly accounted for, and so the size is rounded up to a page boundary, using the page size for which the kernel is built. While w/64 Kbyte pages this results in a 64 Kbyte ring buffer header plus a 64 Kbyte ring buffer, that's the smallest possible with that page size. It's still 128 Kbytes better than the current code.

---
Changes in v2:
* Use SZ_16K instead of 16 * 1024

---
 drivers/pci/controller/pci-hyperv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/controller/pci-hyperv.c +1 -1
index 1eaffff40b8d..baadc1e5090e 100644
@@ -465,7 +465,7 @@ struct pci_eject_response {
 	u32 status;
 } __packed;
 
-static int pci_ring_size = (4 * PAGE_SIZE);
+static int pci_ring_size = VMBUS_RING_SIZE(SZ_16K);
 
 /*
  * Driver specific state.
Signature · 1 line
2.25.1
Reviewed-by on v2 1/1
#1 lore raw
Quoted 35 lines · On Wed, 14 Feb 2024, [email protected] wrote:

From: Michael Kelley <[email protected]>

For a physical PCI device that is passed through to a Hyper-V guest VM, current code specifies the VMBus ring buffer size as 4 pages. But this is an inappropriate dependency, since the amount of ring buffer space needed is unrelated to PAGE_SIZE. For example, on x86 the ring buffer size ends up as 16 Kbytes, while on ARM64 with 64 Kbyte pages, the ring size bloats to 256 Kbytes. The ring buffer for PCI pass-thru devices is used for only a few messages during device setup and removal, so any space above a few Kbytes is wasted.

Fix this by declaring the ring buffer size to be a fixed 16 Kbytes. Furthermore, use the VMBUS_RING_SIZE() macro so that the ring buffer header is properly accounted for, and so the size is rounded up to a page boundary, using the page size for which the kernel is built. While w/64 Kbyte pages this results in a 64 Kbyte ring buffer header plus a 64 Kbyte ring buffer, that's the smallest possible with that page size. It's still 128 Kbytes better than the current code.

---
Changes in v2:
* Use SZ_16K instead of 16 * 1024

---
 drivers/pci/controller/pci-hyperv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/controller/pci-hyperv.c +0 -0
index 1eaffff40b8d..baadc1e5090e 100644

drivers/pci/controller/pci-hyperv.c struct pci_eject_response { · line 471

Quoted 8 lines
drivers/pci/controller/pci-hyperv.c +1 -1
@@ -465,7 +465,7 @@ struct pci_eject_response {
 	u32 status;
 } __packed;
 
-static int pci_ring_size = (4 * PAGE_SIZE);
+static int pci_ring_size = VMBUS_RING_SIZE(SZ_16K);
 
 /*
  * Driver specific state.

Hi,

You forgot to add #include <linux/sizes.h> for it.

With that fixed:

Signature · 1 line
 i.
#2 lore raw

From: Ilpo Järvinen <[email protected]> Sent: Friday, February 16, 2024 6:46 AM

Quoted 49 lines · On Wed, 14 Feb 2024, [email protected] wrote:
Quoted 18 lines

For a physical PCI device that is passed through to a Hyper-V guest VM, current code specifies the VMBus ring buffer size as 4 pages. But this is an inappropriate dependency, since the amount of ring buffer space needed is unrelated to PAGE_SIZE. For example, on x86 the ring buffer size ends up as 16 Kbytes, while on ARM64 with 64 Kbyte pages, the ring size bloats to 256 Kbytes. The ring buffer for PCI pass-thru devices is used for only a few messages during device setup and removal, so any space above a few Kbytes is wasted.

Fix this by declaring the ring buffer size to be a fixed 16 Kbytes. Furthermore, use the VMBUS_RING_SIZE() macro so that the ring buffer header is properly accounted for, and so the size is rounded up to a page boundary, using the page size for which the kernel is built. While w/64 Kbyte pages this results in a 64 Kbyte ring buffer header plus a 64 Kbyte ring buffer, that's the smallest possible with that page size. It's still 128 Kbytes better than the current code.

Quoted 3 lines
Cc <[email protected]> # 5.15.x
Reviewed-by Kuppuswamy Sathyanarayanan

<[email protected]>

drivers/pci/controller/pci-

Quoted 8 lines

---
Changes in v2:
* Use SZ_16K instead of 16 * 1024

---
 drivers/pci/controller/pci-hyperv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/controller/pci- renamed from drivers/pci/controller/pci-hyperv.c +0 -0

hyperv.c

drivers/pci/controller/pci-hyperv.c struct pci_eject_response { · line 471

Quoted 13 lines

index 1eaffff40b8d..baadc1e5090e 100644

drivers/pci/controller/pci-hyperv.c +1 -1
@@ -465,7 +465,7 @@ struct pci_eject_response {
 	u32 status;
 } __packed;
 
-static int pci_ring_size = (4 * PAGE_SIZE);
+static int pci_ring_size = VMBUS_RING_SIZE(SZ_16K);
 
 /*
  * Driver specific state.

Hi,

Quoted 6 lines

You forgot to add #include <linux/sizes.h> for it.

With that fixed:

Fixed in v3. I mis-interpreted your previous comment about adding the #include "if needed". It's not needed to compile correctly, as sizes.h is indirectly included through some other #include. But it's better to directly #include what's needed lest some unrelated change cause a failure.

Michael

#3 lore raw
Quoted 54 lines · Michael Kelley

From: Ilpo Järvinen <[email protected]> Sent: Friday, February 16, 2024 6:46 AM

Quoted 53 lines · On Wed, 14 Feb 2024, [email protected] wrote:
Quoted 18 lines

For a physical PCI device that is passed through to a Hyper-V guest VM, current code specifies the VMBus ring buffer size as 4 pages. But this is an inappropriate dependency, since the amount of ring buffer space needed is unrelated to PAGE_SIZE. For example, on x86 the ring buffer size ends up as 16 Kbytes, while on ARM64 with 64 Kbyte pages, the ring size bloats to 256 Kbytes. The ring buffer for PCI pass-thru devices is used for only a few messages during device setup and removal, so any space above a few Kbytes is wasted.

Fix this by declaring the ring buffer size to be a fixed 16 Kbytes. Furthermore, use the VMBUS_RING_SIZE() macro so that the ring buffer header is properly accounted for, and so the size is rounded up to a page boundary, using the page size for which the kernel is built. While w/64 Kbyte pages this results in a 64 Kbyte ring buffer header plus a 64 Kbyte ring buffer, that's the smallest possible with that page size. It's still 128 Kbytes better than the current code.

Quoted 3 lines
Cc <[email protected]> # 5.15.x
Reviewed-by Kuppuswamy Sathyanarayanan

<[email protected]>

drivers/pci/controller/pci-

Quoted 8 lines

---
Changes in v2:
* Use SZ_16K instead of 16 * 1024

---
 drivers/pci/controller/pci-hyperv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/controller/pci- renamed from drivers/pci/controller/pci-hyperv.c +0 -0

hyperv.c

drivers/pci/controller/pci-hyperv.c struct pci_eject_response { · line 471

Quoted 13 lines

index 1eaffff40b8d..baadc1e5090e 100644

drivers/pci/controller/pci-hyperv.c +1 -1
@@ -465,7 +465,7 @@ struct pci_eject_response {
 	u32 status;
 } __packed;
 
-static int pci_ring_size = (4 * PAGE_SIZE);
+static int pci_ring_size = VMBUS_RING_SIZE(SZ_16K);
 
 /*
  * Driver specific state.

Hi,

You forgot to add #include <linux/sizes.h> for it.

With that fixed:

Quoted 8 lines
Quoted 2 lines

Fixed in v3. I mis-interpreted your previous comment about adding the #include "if needed". It's not needed to compile correctly, as sizes.h is indirectly included through some other #include. But it's better to directly #include what's needed lest some unrelated change cause a failure.

Yes, we try include the headers we use in the .c file. I used "if needed" because I didn't check if it was already among the #includes in the file.

Our tools are lacking to enforce/check a file has correct set of #includes so it's currently based mostly on reviewers noticing something is wrong with #includes, hopefully some time in the future, the tools also catch up.

Signature · 1 line
 i.