Skip to contentSkip to conversation

vpci: allow unaligned accesses by the hardware domain

PATCH In review Roger Pau Monne 3 participants 10 messages Aug 6 – Aug 12

lore
Share & tools
lore
#0 lore raw

v2 cover vpci: allow unaligned accesses by the hardware domain

Hello,

Mostly the same as v1, plus an additional patch that refuses accesses that cross a device boundary in pci_mmcfg_{read,write}() itself.

Thanks, Roger.

Roger Pau Monne (2):
x86/pci: prevent cross-device accesses in pci_mmcfg_{read,write}() xen/vpci: allow unaligned accesses by the hardware domain

 tools/include/xen-tools/common-macros.h | 2 ++
 xen/arch/x86/x86_64/mmconfig_64.c       | 6 ++++--
 xen/arch/x86/x86_64/pci.c               | 8 ++++----
 xen/drivers/vpci/vpci.c                 | 6 ++++--
 4 files changed, 14 insertions(+), 8 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH v2 1/2 x86/pci: prevent cross-device accesses in pci_mmcfg_{read,write}()

Introduce a specific check that prevents an accesses from spilling across two devices.

---
Changes since v1:
- New in this version.

---
 xen/arch/x86/x86_64/mmconfig_64.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
xen/arch/x86/x86_64/mmconfig_64.c +4 -2
index 940cf6d7471b..91b1a398e646 100644
@@ -61,7 +61,8 @@ int pci_mmcfg_read(unsigned int seg, unsigned int bus,
     char __iomem *addr;
 
     /* Why do we have this when nobody checks it. How about a BUG()!? -AK */
-    if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095))) {
+    if (unlikely((bus > 255) || (devfn > 255) ||
+                 (reg + len > PCI_CFG_SPACE_EXP_SIZE))) {
 err:        *value = -1;
         return -EINVAL;
     }
@@ -91,7 +92,8 @@ int pci_mmcfg_write(unsigned int seg, unsigned int bus,
     char __iomem *addr;
 
     /* Why do we have this when nobody checks it. How about a BUG()!? -AK */
-    if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095)))
+    if (unlikely((bus > 255) || (devfn > 255) ||
+                 (reg + len > PCI_CFG_SPACE_EXP_SIZE)))
         return -EINVAL;
 
     addr = pci_dev_base(seg, bus, devfn);
Signature · 1 line
2.53.0
Reviewed-by on v2 1/2
#2 lore raw

Re: [PATCH v2 1/2] x86/pci: prevent cross-device accesses in pci_mmcfg_{read,write}()

Quoted 4 lines · Roger Pau Monne

Introduce a specific check that prevents an accesses from spilling across two devices.

albeit with a remark:

Quoted 13 lines
xen/arch/x86/x86_64/mmconfig_64.c +2 -1
@@ -61,7 +61,8 @@ int pci_mmcfg_read(unsigned int seg, unsigned int bus,
     char __iomem *addr;
 
     /* Why do we have this when nobody checks it. How about a BUG()!? -AK */
-    if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095))) {
+    if (unlikely((bus > 255) || (devfn > 255) ||
+                 (reg + len > PCI_CFG_SPACE_EXP_SIZE))) {
 err:        *value = -1;
         return -EINVAL;
     }

xen/arch/x86/x86_64/mmconfig_64.c int pci_mmcfg_write(unsigned int seg, unsigned int bus, · line 99

Quoted 9 lines
xen/arch/x86/x86_64/mmconfig_64.c +2 -1
@@ -91,7 +92,8 @@ int pci_mmcfg_write(unsigned int seg, unsigned int bus,
     char __iomem *addr;
 
     /* Why do we have this when nobody checks it. How about a BUG()!? -AK */
-    if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095)))
+    if (unlikely((bus > 255) || (devfn > 255) ||
+                 (reg + len > PCI_CFG_SPACE_EXP_SIZE)))
         return -EINVAL;
 
     addr = pci_dev_base(seg, bus, devfn);

In both cases the unlikely() uses won't have the intended effect, from all I know. They would help as used here only if the compiler managed to fold all three parts of the ||-expression into a single conditional branch, which I don't think it would end up doing.

Jan

#3 lore raw

Re: [PATCH v2 1/2] x86/pci: prevent cross-device accesses in pci_mmcfg_{read,write}()

Quoted 30 lines · Jan Beulich
Quoted 4 lines · Roger Pau Monne

Introduce a specific check that prevents an accesses from spilling across two devices.

albeit with a remark:

Quoted 21 lines
xen/arch/x86/x86_64/mmconfig_64.c +4 -2
@@ -61,7 +61,8 @@ int pci_mmcfg_read(unsigned int seg, unsigned int bus,
     char __iomem *addr;
 
     /* Why do we have this when nobody checks it. How about a BUG()!? -AK */
-    if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095))) {
+    if (unlikely((bus > 255) || (devfn > 255) ||
+                 (reg + len > PCI_CFG_SPACE_EXP_SIZE))) {
 err:        *value = -1;
         return -EINVAL;
     }
@@ -91,7 +92,8 @@ int pci_mmcfg_write(unsigned int seg, unsigned int bus,
     char __iomem *addr;
 
     /* Why do we have this when nobody checks it. How about a BUG()!? -AK */
-    if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095)))
+    if (unlikely((bus > 255) || (devfn > 255) ||
+                 (reg + len > PCI_CFG_SPACE_EXP_SIZE)))
         return -EINVAL;
 
Quoted 6 lines

xen/arch/x86/x86_64/mmconfig_64.c int pci_mmcfg_write(unsigned int seg, unsigned int bus, · line 99

Quoted 1 line
xen/arch/x86/x86_64/mmconfig_64.c +0 -0
     addr = pci_dev_base(seg, bus, devfn);

In both cases the unlikely() uses won't have the intended effect, from all I know. They would help as used here only if the compiler managed to fold all three parts of the ||-expression into a single conditional branch, which I don't think it would end up doing.

I don't mind dropping the unlikely() while changing the line. I tend to leave those alone if present, even when I'm not sure they are actually helpful. The comment ahead of the check is also not very useful IMO, but I've decided to leave it alone.

Thanks, Roger.

#4 lore raw

Re: [PATCH v2 1/2] x86/pci: prevent cross-device accesses in pci_mmcfg_{read,write}()

Quoted 38 lines · Roger Pau Monné
Quoted 30 lines · Jan Beulich
Quoted 4 lines · Roger Pau Monne

Introduce a specific check that prevents an accesses from spilling across two devices.

albeit with a remark:

Quoted 21 lines
xen/arch/x86/x86_64/mmconfig_64.c +4 -2
@@ -61,7 +61,8 @@ int pci_mmcfg_read(unsigned int seg, unsigned int bus,
     char __iomem *addr;
 
     /* Why do we have this when nobody checks it. How about a BUG()!? -AK */
-    if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095))) {
+    if (unlikely((bus > 255) || (devfn > 255) ||
+                 (reg + len > PCI_CFG_SPACE_EXP_SIZE))) {
 err:        *value = -1;
         return -EINVAL;
     }
@@ -91,7 +92,8 @@ int pci_mmcfg_write(unsigned int seg, unsigned int bus,
     char __iomem *addr;
 
     /* Why do we have this when nobody checks it. How about a BUG()!? -AK */
-    if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095)))
+    if (unlikely((bus > 255) || (devfn > 255) ||
+                 (reg + len > PCI_CFG_SPACE_EXP_SIZE)))
         return -EINVAL;
 
Quoted 6 lines

xen/arch/x86/x86_64/mmconfig_64.c int pci_mmcfg_write(unsigned int seg, unsigned int bus, · line 99

Quoted 1 line
xen/arch/x86/x86_64/mmconfig_64.c +0 -0
     addr = pci_dev_base(seg, bus, devfn);

In both cases the unlikely() uses won't have the intended effect, from all I know. They would help as used here only if the compiler managed to fold all three parts of the ||-expression into a single conditional branch, which I don't think it would end up doing.

Quoted 3 lines

I don't mind dropping the unlikely() while changing the line. I tend to leave those alone if present, even when I'm not sure they are actually helpful.

I think unlikely() is actually reasonable to have here, yet it would need to be four of them, not just one. Unless reachable from a DomU-accessible path, converting to BUG_ON() (or ASSERT_UNREACHABLE()) instead would also be an option.

Quoted 2 lines

The comment ahead of the check is also not very
useful IMO, but I've decided to leave it alone.

Indeed, that's an odd thing to survive over 20 years (I think).

Jan

#5 lore raw

Re: [PATCH v2 1/2] x86/pci: prevent cross-device accesses in pci_mmcfg_{read,write}()

Quoted 39 lines · Jan Beulich
Quoted 38 lines · Roger Pau Monné
Quoted 36 lines · Jan Beulich
Quoted 4 lines · Roger Pau Monne

Introduce a specific check that prevents an accesses from spilling across two devices.

albeit with a remark:

Quoted 13 lines
xen/arch/x86/x86_64/mmconfig_64.c +2 -1
@@ -61,7 +61,8 @@ int pci_mmcfg_read(unsigned int seg, unsigned int bus,
     char __iomem *addr;
 
     /* Why do we have this when nobody checks it. How about a BUG()!? -AK */
-    if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095))) {
+    if (unlikely((bus > 255) || (devfn > 255) ||
+                 (reg + len > PCI_CFG_SPACE_EXP_SIZE))) {
 err:        *value = -1;
         return -EINVAL;
     }

xen/arch/x86/x86_64/mmconfig_64.c int pci_mmcfg_write(unsigned int seg, unsigned int bus, · line 99

Quoted 9 lines
xen/arch/x86/x86_64/mmconfig_64.c +2 -1
@@ -91,7 +92,8 @@ int pci_mmcfg_write(unsigned int seg, unsigned int bus,
     char __iomem *addr;
 
     /* Why do we have this when nobody checks it. How about a BUG()!? -AK */
-    if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095)))
+    if (unlikely((bus > 255) || (devfn > 255) ||
+                 (reg + len > PCI_CFG_SPACE_EXP_SIZE)))
         return -EINVAL;
 
     addr = pci_dev_base(seg, bus, devfn);

In both cases the unlikely() uses won't have the intended effect, from all I know. They would help as used here only if the compiler managed to fold all three parts of the ||-expression into a single conditional branch, which I don't think it would end up doing.

Quoted 8 lines
Quoted 3 lines

I don't mind dropping the unlikely() while changing the line. I tend to leave those alone if present, even when I'm not sure they are actually helpful.

I think unlikely() is actually reasonable to have here, yet it would need to be four of them, not just one. Unless reachable from a DomU-accessible path, converting to BUG_ON() (or ASSERT_UNREACHABLE()) instead would also be an option.

Hm, let's do that in a separate patch, I don't want to merge it with the work here.

Quoted 4 lines
Quoted 2 lines

The comment ahead of the check is also not very
useful IMO, but I've decided to leave it alone.

Indeed, that's an odd thing to survive over 20 years (I think).

I can also take care of the comment there.

Thanks, Roger.

#7 lore raw

PATCH v2 2/2 xen/vpci: allow unaligned accesses by the hardware domain

In reply to Roger Pau Monne

It's possible for domains to generate unaligned PCI config space accesses when using ECAM, and hence vPCI should support those at least for the hardware domain. Such unaligned accesses to the PCI config space have been reported to come from ACPI logic.

Relax the checking in vpci_access_allowed() to allow such accesses for the hardware domain, and fix the handling in pci_conf_{read,write}{16,32}() to fulfill them using MMCFG.

MMCFG regions are identity exposed to the hardware domain, and hence such unaligned accesses can only come as a result of the host having MMCFG in the first place, as otherwise MMCFG won't be exposed to the hardware domain either.

Note that vpci_ecam_{read,write}() already refuse accesses that cross a device boundary unconditionally.

---
 tools/include/xen-tools/common-macros.h | 2 ++
 xen/arch/x86/x86_64/pci.c               | 8 ++++----
 xen/drivers/vpci/vpci.c                 | 6 ++++--
 3 files changed, 10 insertions(+), 6 deletions(-)
tools/include/xen-tools/common-macros.h +2 -0
index 88b4a0e5a693..1f9146b23b0e 100644
@@ -68,6 +68,8 @@
     })
 #endif
 
+#define IS_ALIGNED(val, align) (!((val) & ((align) - 1)))
+
 #define ROUNDUP(x, a) (((x) + (a) - 1) & ~((a) - 1))
 #define ROUNDDOWN(x, a) ((x) & ~((a) - 1))
 
xen/arch/x86/x86_64/pci.c +4 -4
index 8d33429103b9..6298141c3ca7 100644
@@ -26,7 +26,7 @@ uint8_t pci_conf_read8(pci_sbdf_t sbdf, unsigned int reg)
 
 uint16_t pci_conf_read16(pci_sbdf_t sbdf, unsigned int reg)
 {
-    if ( sbdf.seg || reg > 255 )
+    if ( sbdf.seg || reg > 255 || !IS_ALIGNED(reg, 2) )
     {
         uint32_t value;
 
@@ -39,7 +39,7 @@ uint16_t pci_conf_read16(pci_sbdf_t sbdf, unsigned int reg)
 
 uint32_t pci_conf_read32(pci_sbdf_t sbdf, unsigned int reg)
 {
-    if ( sbdf.seg || reg > 255 )
+    if ( sbdf.seg || reg > 255 || !IS_ALIGNED(reg, 4) )
     {
         uint32_t value;
 
@@ -60,7 +60,7 @@ void pci_conf_write8(pci_sbdf_t sbdf, unsigned int reg, uint8_t data)
 
 void pci_conf_write16(pci_sbdf_t sbdf, unsigned int reg, uint16_t data)
 {
-    if ( sbdf.seg || reg > 255 )
+    if ( sbdf.seg || reg > 255 || !IS_ALIGNED(reg, 2) )
         pci_mmcfg_write(sbdf.seg, sbdf.bus, sbdf.devfn, reg, 2, data);
     else
         pci_conf_write(PCI_CONF_ADDRESS(sbdf, reg), reg & 2, 2, data);
@@ -68,7 +68,7 @@ void pci_conf_write16(pci_sbdf_t sbdf, unsigned int reg, uint16_t data)
 
 void pci_conf_write32(pci_sbdf_t sbdf, unsigned int reg, uint32_t data)
 {
-    if ( sbdf.seg || reg > 255 )
+    if ( sbdf.seg || reg > 255 || !IS_ALIGNED(reg, 4) )
         pci_mmcfg_write(sbdf.seg, sbdf.bus, sbdf.devfn, reg, 4, data);
     else
         pci_conf_write(PCI_CONF_ADDRESS(sbdf, reg), 0, 4, data);
xen/drivers/vpci/vpci.c +4 -2
index 0ac9ec8b0475..9e2c27e3a300 100644
@@ -685,6 +685,8 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size,
 /* Helper function to check an access size and alignment on vpci space. */
 bool vpci_access_allowed(unsigned int reg, unsigned int len)
 {
+    const struct domain *currd = current->domain;
+
     /* Check access size. */
     if ( len != 1 && len != 2 && len != 4 && len != 8 )
         return false;
@@ -695,8 +697,8 @@ bool vpci_access_allowed(unsigned int reg, unsigned int len)
         return false;
 #endif
 
-    /* Check that access is size aligned. */
-    if ( (reg & (len - 1)) )
+    /* Refuse unaligned accesses for non-hardware domains. */
+    if ( !is_hardware_domain(currd) && !IS_ALIGNED(reg, len) )
         return false;
 
     return true;
Signature · 1 line
2.53.0
Reviewed-by on v2 2/2
#8 lore raw

Re: [PATCH v2 2/2] xen/vpci: allow unaligned accesses by the hardware domain

Quoted 14 lines · Roger Pau Monne

It's possible for domains to generate unaligned PCI config space accesses when using ECAM, and hence vPCI should support those at least for the hardware domain. Such unaligned accesses to the PCI config space have been reported to come from ACPI logic.

Relax the checking in vpci_access_allowed() to allow such accesses for the hardware domain, and fix the handling in pci_conf_{read,write}{16,32}() to fulfill them using MMCFG.

MMCFG regions are identity exposed to the hardware domain, and hence such unaligned accesses can only come as a result of the host having MMCFG in the first place, as otherwise MMCFG won't be exposed to the hardware domain either.

Quoted 5 lines

Note that vpci_ecam_{read,write}() already refuse accesses that cross a device boundary unconditionally.

Albeit without the intent to override possible remaining objections by Andrew.

Jan

Reviewed-by on v2 2/2
#9 lore raw

Re: [PATCH v2 2/2] xen/vpci: allow unaligned accesses by the hardware domain

In reply to Roger Pau Monne
Quoted 14 lines · Roger Pau Monne

It's possible for domains to generate unaligned PCI config space accesses when using ECAM, and hence vPCI should support those at least for the hardware domain. Such unaligned accesses to the PCI config space have been reported to come from ACPI logic.

Relax the checking in vpci_access_allowed() to allow such accesses for the hardware domain, and fix the handling in pci_conf_{read,write}{16,32}() to fulfill them using MMCFG.

MMCFG regions are identity exposed to the hardware domain, and hence such unaligned accesses can only come as a result of the host having MMCFG in the first place, as otherwise MMCFG won't be exposed to the hardware domain either.

Quoted 5 lines

Note that vpci_ecam_{read,write}() already refuse accesses that cross a device boundary unconditionally.