x86: Fix racy accesses to MSI-X Control register
cover x86: Fix racy accesses to MSI-X Control register
The main patch in this series is 3/3 with some preparatory patches to simplify the implementation. To summarize:
Concurrent access the the MSI-X control register are not serialized with a suitable lock. For example, in msix_capability_init() access use the pcidevs_lock() but some calls to msi_set_mask_bit() use the interrupt descriptor lock.
This can lead to MSI-X being incorrectly disabled and subsequent failures due to msix_memory_decoded() calls that check for MSI-X being enabled.
David
PATCH 1/3 x86/msi: consistently handle BAR mapping failures in MSI-X setup
When setting up an MSI-X vector in msix_capability_init() the error handling after a BAR mapping failure is different depending on whether the first page fails or a subsequent page. There's no reason to break working vectors so consistently use the later error handling behaviour.
The zap_on_error flag was added as part of XSA-337, beb54596cfda (x86/MSI-X: restrict reading of table/PBA bases from BARs), but appears to be unrelated to XSA-337 and is not useful because:
1. table.first and pba.first are not used unless msix->used_vectors > 0.
2. Force disabling MSI-X in this error path is not necessary as the per-vector mask is still still set.
CR: https://code.amazon.com/reviews/CR-79020908
--- xen/arch/x86/msi.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-)
xen/arch/x86/msi.c +1 -11
@@ -776,7 +776,7 @@ static int msix_capability_init(struct pci_dev *dev,u8 bus = dev->bus;u8 slot =PCI_SLOT (dev->devfn);u8 func =PCI_FUNC (dev->devfn); -bool maskall = msix->host_maskall, zap_on_error = false; +bool maskall = msix->host_maskall;unsigned int pos =pci_find_cap_offset (seg, bus, slot, func,PCI_CAP_ID_MSIX );
@@ -875,8 +875,6 @@ static int msix_capability_init(struct pci_dev *dev, BITS_TO_LONGS(msix->nr_entries) - 1);WARN_ON (rangeset_overlaps_range (mmio_ro_ranges, msix->pba.first, msix->pba.last)); - - zap_on_error = true; }else if ( !msix->table.first ) {
@@ -897,14 +895,6 @@ static int msix_capability_init(struct pci_dev *dev,if ( idx <0 ) { -if ( zap_on_error ) - { - msix->table.first =0 ; - msix->pba.first =0 ; - - control &= ~PCI_MSIX_FLAGS_ENABLE ; - } -pci_conf_write16 (dev->sbdf,msix_control_reg (pos), control);xfree (entry);return idx;
Signature · 1 line
2.30.2
Re: [PATCH 1/3] x86/msi: consistently handle BAR mapping failures in MSI-X setup
Quoted 5 lines · David Vrabel
When setting up an MSI-X vector in
msix_capability_init()the error handling after a BAR mapping failure is different depending on whether the first page fails or a subsequent page. There's no reason to break working vectors so consistently use the later error handling behaviour.
"zap_on_error" can only be set when there were no working vectors yet (msix->used_entries being zero), so I don't see what case this last sentence describes. In fact it was the intention with "zap_on_error" to leave previously set up vectors functional.
Quoted 5 lines
The
zap_on_errorflag was added as part of XSA-337,beb54596cfda(x86/MSI-X: restrict reading of table/PBA bases from BARs), but appears to be unrelated to XSA-337 and is not useful because:1. table.first and pba.first are not used unless
msix->used_vectors> 0.
This isn't true afaics. The condition around their setting up is involving more than just ->used_vectors:
if ( !msix->used_entries &&
(!msi ||
(is_hardware_domain(current->domain) &&
(dev->domain == current->domain || dev->domain == dom_io))) )Hence the associated "else if( !msix->table.first )" can also be taken if msix->used_entries is zero. And in case of a failure we need to force the error return there for DomU-s, which is achieved by clearing msix->table.first on the error handling path you alter.
Furthermore I'd consider it bad practice to leave stale values on record.
Quoted 2 lines
2. Force disabling MSI-X in this error path is not necessary as the per-vector mask is still still set.
I agree that we might be overly strict there, but to remove that disabling you'd need to further prove that no other inconsistencies can (later) result (this being on the safe side is where the connection to the rest of the XSA-337 changes comes from, along with the desire to not leave stale values around, as per above). Plus you'd want to justify why this error path is different from others in the function where we also disable MSI-X altogether (beyond the path you modify there's exactly one error path where we don't, and I now wonder why I had done it like that).
But then I may also be misunderstanding some of your intentions here. The "consistently" in the title and the associated first sentence of the description escape me for the moment: You're talking about things in terms of pages, when the handling really is in terms of entries.
Jan
PATCH 2/3 x86/msi: remove return value from msi_set_mask_bit()
In reply to David VrabelThe return value was only used for WARN()s or BUG()s so it has no functional purpose. Simplify the code by removing it.
The meaning of the return value and the purpose of the various WARNs() and BUGs() is rather unclear. The only failure path (where an MSI-X vector needs to be masked but the MSI-X table is not accessible) has a useful warning message.
CR: https://code.amazon.com/reviews/CR-79020927
--- xen/arch/x86/msi.c | 34 +++++++++------------------------- 1 file changed, 9 insertions(+), 25 deletions(-)
xen/arch/x86/msi.c +9 -25
@@ -314,7 +314,7 @@ int msi_maskable_irq(const struct msi_desc *entry) || entry->msi_attrib.maskbit; } -static bool msi_set_mask_bit (struct irq_desc *desc,bool host,bool guest) +static void msi_set_mask_bit (struct irq_desc *desc,bool host,bool guest) {struct msi_desc *entry = desc->msi_desc;struct pci_dev *pdev;
@@ -361,11 +361,6 @@ static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest)if (likely (control &PCI_MSIX_FLAGS_ENABLE ) )break ; - - entry->msi_attrib.host_masked = host; - entry->msi_attrib.guest_masked = guest; - - flag = true; }else if ( flag && !(control &PCI_MSIX_FLAGS_MASKALL ) ) {
@@ -385,14 +380,10 @@ static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest) control |=PCI_MSIX_FLAGS_MASKALL ;pci_conf_write16 (pdev->sbdf,msix_control_reg (entry->msi_attrib.pos), control); -return flag; -default : -return 0 ; +break ; } entry->msi_attrib.host_masked = host; entry->msi_attrib.guest_masked = guest; - -return 1 ; }static int msi_get_mask_bit (const struct msi_desc *entry)
@@ -418,16 +409,12 @@ static int msi_get_mask_bit(const struct msi_desc *entry) void cf_check mask_msi_irq(struct irq_desc *desc) { -if (unlikely (!msi_set_mask_bit (desc,1 , - desc->msi_desc->msi_attrib.guest_masked)) ) -BUG_ON (!(desc->status &IRQ_DISABLED )); +msi_set_mask_bit (desc,1 , desc->msi_desc->msi_attrib.guest_masked); } void cf_check unmask_msi_irq(struct irq_desc *desc) { -if (unlikely (!msi_set_mask_bit (desc,0 , - desc->msi_desc->msi_attrib.guest_masked)) ) -WARN (); +msi_set_mask_bit (desc,0 , desc->msi_desc->msi_attrib.guest_masked); }void guest_mask_msi_irq (struct irq_desc *desc,bool mask)
@@ -437,15 +424,13 @@ void guest_mask_msi_irq(struct irq_desc *desc, bool mask) static unsigned int cf_check startup_msi_irq(struct irq_desc *desc) { -if (unlikely (!msi_set_mask_bit (desc,0 , !!(desc->status &IRQ_GUEST ))) ) -WARN (); +msi_set_mask_bit (desc,0 , !!(desc->status &IRQ_GUEST ));return 0 ; } static void cf_check shutdown_msi_irq(struct irq_desc *desc) { -if (unlikely (!msi_set_mask_bit (desc,1 ,1 )) ) -BUG_ON (!(desc->status &IRQ_DISABLED )); +msi_set_mask_bit (desc,1 ,1 ); } void cf_check ack_nonmaskable_msi_irq(struct irq_desc *desc)
@@ -1358,10 +1343,9 @@ int pci_restore_msi_state(struct pci_dev *pdev)for ( i =0 ; ; ) { -if (unlikely (!msi_set_mask_bit (desc, - entry[i].msi_attrib.host_masked, - entry[i].msi_attrib.guest_masked)) ) -BUG (); +msi_set_mask_bit (desc, + entry[i].msi_attrib.host_masked, + entry[i].msi_attrib.guest_masked);if ( !--nr )break ;
Signature · 1 line
2.30.2
Re: [PATCH 2/3] x86/msi: remove return value from msi_set_mask_bit()
Quoted 7 lines · David Vrabel
The return value was only used for WARN()s or BUG()s so it has no functional purpose. Simplify the code by removing it.
The meaning of the return value and the purpose of the various
WARNs()andBUGs()is rather unclear. The only failure path (where an MSI-X vector needs to be masked but the MSI-X table is not accessible) has a useful warning message.
No, you're removing an important 2nd such path - the default case in the switch() statement. Getting there would mean another bug elsewhere, which we don't want to leave undetected for yet longer (and hence yet harder to debug once finally some misbehavior surfaces). That default case may warrant the addition of ASSERT_UNREACHABLE() according to the respective description in ./CODING_STYLE, but I don't see the removal of the "return" as acceptable (also for another reason as explained below).
The idea of the WARN() / BUG_ON() is to
- not leave failed unmasking unrecorded,
- not continue after failure to mask an entry.
This combines with the functions where the constructs are not having ways to properly propagate the errors to their callers.
Quoted 13 lines
xen/arch/x86/msi.c +0
-5@@ -361,11 +361,6 @@ static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest)if (likely (control &PCI_MSIX_FLAGS_ENABLE ) )break ; - - entry->msi_attrib.host_masked = host; - entry->msi_attrib.guest_masked = guest; - - flag = true; }else if ( flag && !(control &PCI_MSIX_FLAGS_MASKALL ) ) {
xen/arch/x86/msi.c static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest) · line 388
Quoted 4 lines
xen/arch/x86/msi.c +0
-1@@ -385,14 +380,10 @@ static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest) control |=PCI_MSIX_FLAGS_MASKALL ;pci_conf_write16 (pdev->sbdf,msix_control_reg (entry->msi_attrib.pos), control); -return flag;
Both this and ...
xen/arch/x86/msi.c static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest) · line 390
Quoted 2 lines
xen/arch/x86/msi.c +0
-2-default : -return 0 ;
... this have previously prevented to make it ...
xen/arch/x86/msi.c static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest) · line 386
Quoted 4 lines
xen/arch/x86/msi.c +1
-0+break ; } entry->msi_attrib.host_masked = host; entry->msi_attrib.guest_masked = guest;
... here. This is a change in behavior which is neither obviously benign nor properly justified in the description, yet clearly going beyond what the title says the patch is about.
Jan
Re: [PATCH 2/3] x86/msi: remove return value from msi_set_mask_bit()
Quoted 4 lines · Jan Beulich
The idea of the
WARN()/BUG_ON()is to
- not leave failed unmasking unrecorded,
- not continue after failure to mask an entry.
Then lets make msi_set_mask_bit() unable to fail with something like this (untested) patch. Would this be acceptable?
David
From 837649a70d44455f4fd98e2eaa46dcf35a56d00a Mon Sep 17 00:00:00 2001
From: David Vrabel <[email protected]>
Date: Fri, 11 Nov 2022 14:30:16 +0000
Subject: [PATCH] x86: Always enable memory space decodes when using MSI-X
Instead of the numerous (racy) checks for memory space accesses being enabled before writing the the MSI-X table, force Memory Space Enable to be set in the Command register if MSI-X is used.
This allows the memory_decoded() function and the associated error paths to be removed (since it will always return true). In particular, msi_set_mask_bit() can no longer fail and its return value is removed.
Note that if the PCI device is a virtual function, the relevant command register is in the corresponding physical function.
--- xen/arch/x86/include/asm/pci.h | 3 + xen/arch/x86/msi.c | 116 +++++++++------------------------ xen/arch/x86/pci.c | 39 ++++++++++- 3 files changed, 71 insertions(+), 87 deletions(-)
xen/arch/x86/include/asm/pci.h +3 -0
@@ -32,8 +32,11 @@ struct arch_pci_dev {domid_t pseudo_domid;mfn_t leaf_mfn;struct page_list_head pgtables_list; +uint16_t host_command; +uint16_t guest_command; }; +void pci_command_override (struct pci_dev *pdev,uint16_t val);int pci_conf_write_intercept (unsigned int seg,unsigned int bdf,unsigned int reg,unsigned int size,uint32_t *data);
xen/arch/x86/msi.c +0 -13
@@ -124,27 +124,11 @@ static void msix_put_fixmap(struct arch_msix *msix, int idx)spin_unlock (&msix->table_lock); } -static bool memory_decoded (const struct pci_dev *dev) -{ -pci_sbdf_t sbdf = dev->sbdf; - -if ( dev->info.is_virtfn ) - { - sbdf.bus = dev->info.physfn.bus; - sbdf.devfn = dev->info.physfn.devfn; - } - -return pci_conf_read16 (sbdf,PCI_COMMAND ) &PCI_COMMAND_MEMORY ; -} - static bool msix_memory_decoded(const struct pci_dev *dev, unsigned
int pos)
{
uint16_t control = pci_conf_read16(dev->sbdf, msix_control_reg(pos));- if ( !(control & PCI_MSIX_FLAGS_ENABLE) )
- return false;
-
- return memory_decoded(dev);
+ return control & PCI_MSIX_FLAGS_ENABLE;
}
/*
@@ -314,7 +298,7 @@ int msi_maskable_irq(const struct msi_desc *entry)
|| entry->msi_attrib.maskbit;
}
-static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest) +static void msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest)
{
struct msi_desc *entry = desc->msi_desc;
struct pci_dev *pdev;
@@ -354,45 +338,26 @@ static bool msi_set_mask_bit(struct irq_desc *desc, bool host, bool guest)
control | (PCI_MSIX_FLAGS_ENABLE |
PCI_MSIX_FLAGS_MASKALL));
}- if ( likely(memory_decoded(pdev)) )
- {
- writel(flag, entry->mask_base + PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET);
- readl(entry->mask_base + PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET);
- if ( likely(control & PCI_MSIX_FLAGS_ENABLE) )
- break;
-
- entry->msi_attrib.host_masked = host;
- entry->msi_attrib.guest_masked = guest;
+ writel(flag, entry->mask_base + PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET);
+ readl(entry->mask_base + PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET);
- flag = true;
- }
- else if ( flag && !(control & PCI_MSIX_FLAGS_MASKALL) )
+ if ( unlikely(!(control & PCI_MSIX_FLAGS_ENABLE)) )
{
- domid_t domid = pdev->domain->domain_id;
-
- maskall = true;
- if ( pdev->msix->warned != domid )
- {- pdev->msix->warned = domid;
- printk(XENLOG_G_WARNING
- "cannot mask IRQ %d: masking MSI-X on Dom%d's %pp\n",
- desc->irq, domid, &pdev->sbdf);
- }
+ pdev->msix->host_maskall = maskall;
+ if ( maskall || pdev->msix->guest_maskall )
+ control |= PCI_MSIX_FLAGS_MASKALL;
+ pci_conf_write16(pdev->sbdf,
+ msix_control_reg(entry->msi_attrib.pos), control);
}- pdev->msix->host_maskall = maskall;
- if ( maskall || pdev->msix->guest_maskall )
- control |= PCI_MSIX_FLAGS_MASKALL;
- pci_conf_write16(pdev->sbdf,
- msix_control_reg(entry->msi_attrib.pos), control);
- return flag;
+ break;
default:
- return 0;
+ ASSERT_UNREACHABLE();
+ break;
}
+
entry->msi_attrib.host_masked = host;
entry->msi_attrib.guest_masked = guest;-
- return 1;
}
static int msi_get_mask_bit(const struct msi_desc *entry)
@@ -418,16 +383,12 @@ static int msi_get_mask_bit(const struct msi_desc *entry)
void cf_check mask_msi_irq(struct irq_desc *desc) {
- if ( unlikely(!msi_set_mask_bit(desc, 1,
- desc->msi_desc->msi_attrib.guest_masked)) )
- BUG_ON(!(desc->status & IRQ_DISABLED));
+ msi_set_mask_bit(desc, 1, desc->msi_desc->msi_attrib.guest_masked);
}
void cf_check unmask_msi_irq(struct irq_desc *desc) {
- if ( unlikely(!msi_set_mask_bit(desc, 0,
- desc->msi_desc->msi_attrib.guest_masked)) )
- WARN();
+ msi_set_mask_bit(desc, 0, desc->msi_desc->msi_attrib.guest_masked);
}
void guest_mask_msi_irq(struct irq_desc *desc, bool mask)
@@ -437,15 +398,13 @@ void guest_mask_msi_irq(struct irq_desc *desc, bool mask)
static unsigned int cf_check startup_msi_irq(struct irq_desc *desc) {
- if ( unlikely(!msi_set_mask_bit(desc, 0, !!(desc->status & IRQ_GUEST))) )
- WARN();
+ msi_set_mask_bit(desc, 0, !!(desc->status & IRQ_GUEST));
return 0;}
static void cf_check shutdown_msi_irq(struct irq_desc *desc) {
- if ( unlikely(!msi_set_mask_bit(desc, 1, 1)) )
- BUG_ON(!(desc->status & IRQ_DISABLED));
+ msi_set_mask_bit(desc, 1, 1);
}
void cf_check ack_nonmaskable_msi_irq(struct irq_desc *desc)
@@ -785,6 +744,12 @@ static int msix_capability_init(struct pci_dev *dev,
ASSERT(pcidevs_locked());
+ /*
+ * Force enable access to the MSI-X tables, so access to the
+ * per-vector mask bits always works.
+ */
+ pci_command_override(dev, PCI_COMMAND_MEMORY);
+
control = pci_conf_read16(dev->sbdf, msix_control_reg(pos));
/*
* Ensure MSI-X interrupts are masked during setup. Some devices require
@@ -797,13 +762,6 @@ static int msix_capability_init(struct pci_dev *dev,
control | (PCI_MSIX_FLAGS_ENABLE |
PCI_MSIX_FLAGS_MASKALL));
- if ( unlikely(!memory_decoded(dev)) )
- {
- pci_conf_write16(dev->sbdf, msix_control_reg(pos),
- control & ~PCI_MSIX_FLAGS_ENABLE);
- return -ENXIO;
- }
-
if ( desc )
{
entry = alloc_msi_entry(1);@@ -1122,19 +1080,15 @@ static void __pci_disable_msix(struct msi_desc *entry)
BUG_ON(list_empty(&dev->msi_list));
- if ( likely(memory_decoded(dev)) )
- writel(1, entry->mask_base + PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET);
- else if ( !(control & PCI_MSIX_FLAGS_MASKALL) )
- {
- printk(XENLOG_WARNING "cannot disable IRQ %d: masking MSI-X on %pp\n",
- entry->irq, &dev->sbdf);
- maskall = true;
- }
+ writel(1, entry->mask_base + PCI_MSIX_ENTRY_VECTOR_CTRL_OFFSET);
+
dev->msix->host_maskall = maskall;
if ( maskall || dev->msix->guest_maskall )
control |= PCI_MSIX_FLAGS_MASKALL;
pci_conf_write16(dev->sbdf, msix_control_reg(pos), control);+ pci_command_override(dev, 0);
+
_pci_cleanup_msix(dev->msix);
}
@@ -1353,13 +1307,6 @@ int pci_restore_msi_state(struct pci_dev *pdev)
pci_conf_write16(pdev->sbdf, msix_control_reg(pos),
control | (PCI_MSIX_FLAGS_ENABLE |
PCI_MSIX_FLAGS_MASKALL));- if ( unlikely(!memory_decoded(pdev)) )
- {
- spin_unlock_irqrestore(&desc->lock, flags);
- pci_conf_write16(pdev->sbdf, msix_control_reg(pos),
- control & ~PCI_MSIX_FLAGS_ENABLE);
- return -ENXIO;
- }
}
type = entry->msi_attrib.type;@@ -1368,10 +1315,9 @@ int pci_restore_msi_state(struct pci_dev *pdev)
for ( i = 0; ; )
{- if ( unlikely(!msi_set_mask_bit(desc,
- entry[i].msi_attrib.host_masked,
- entry[i].msi_attrib.guest_masked)) )
- BUG();
+ msi_set_mask_bit(desc,
+ entry[i].msi_attrib.host_masked,
+ entry[i].msi_attrib.guest_masked);
if ( !--nr )
break;xen/arch/x86/pci.c +37 -2
@@ -69,6 +69,24 @@ void pci_conf_write(uint32_t cf8, uint8_t offset, uint8_t bytes, uint32_t data)spin_unlock_irqrestore (&pci_config_lock, flags); } +void pci_command_override (struct pci_dev *pdev,uint16_t val) +{ +pci_sbdf_t sbdf = pdev->sbdf; + +ASSERT (pcidevs_locked ()); + +if ( pdev->info.is_virtfn ) + { + sbdf.bus = pdev->info.physfn.bus; + sbdf.devfn = pdev->info.physfn.devfn; + + pdev =pci_get_pdev (NULL , sbdf); + } + + pdev->arch.host_command = val; +pci_conf_write16 (sbdf,PCI_COMMAND , pdev->arch.host_command | pdev->arch.guest_command); +} +int pci_conf_write_intercept (unsigned int seg,unsigned int bdf,unsigned int reg,unsigned int size,uint32_t *data)
@@ -85,14 +103,31 @@ int pci_conf_write_intercept(unsigned int seg, unsigned int bdf,* Avoid expensive operations when no hook is going to do anything * for the access anyway. */ -if ( reg <64 || reg >=256 ) +if ( reg !=PCI_COMMAND && (reg <64 || reg >=256 ) )return 0 ;pcidevs_lock (); pdev =pci_get_pdev (NULL ,PCI_SBDF (seg, bdf));if ( pdev ) - rc =pci_msi_conf_write_intercept (pdev, reg, size, data); + { +switch ( reg ) + { +case PCI_COMMAND : +if ( size ==2 ) + { + pdev->arch.guest_command = *data; + *data |= pdev->arch.host_command; + } +else + rc = -EACCESS ; +break ; + +default : + rc =pci_msi_conf_write_intercept (pdev, reg, size, data); +break ; + } + }pcidevs_unlock ();
Signature · 1 line
2.37.1
Re: [PATCH 2/3] x86/msi: remove return value from msi_set_mask_bit()
Quoted 10 lines · David Vrabel
Quoted 4 lines · Jan Beulich
The idea of the
WARN()/BUG_ON()is to
- not leave failed unmasking unrecorded,
- not continue after failure to mask an entry.Then lets make
msi_set_mask_bit()unable to fail with something like this (untested) patch. Would this be acceptable?David
Hmm, that's quite a bit of code churn for something which, for now at least, I'm not convinced needs changing. Yes, ...
Quoted 8 lines
From
837649a70d44455f4fd98e2eaa46dcf35a56d00aMon Sep 17 00:00:00 2001
From: David Vrabel <[email protected]>
Date: Fri, 11 Nov 2022 14:30:16 +0000
Subject: [PATCH] x86: Always enable memory space decodes when using MSI-XInstead of the numerous (racy) checks for memory space accesses being enabled before writing the the MSI-X table, force Memory Space Enable to be set in the Command register if MSI-X is used.
... there is some raciness there, but we assume a well-behaved Dom0 (and DomU don't have direct access to the decode enable bits), so the checks are more to be on the safe side (the original change attempted to merely deal with the specific pciback behavior, without impacting other [odd/unexpected] things Dom0 may be doing, as long as what it's doing isn't plain wrong/buggy).
Quoted 8 lines
This allows the
memory_decoded()function and the associated error paths to be removed (since it will always return true). In particular,msi_set_mask_bit()can no longer fail and its return value is removed.Note that if the PCI device is a virtual function, the relevant command register is in the corresponding physical function.
Signed-off-by David Vrabel <[email protected]>
What I'm missing in the description is a discussion of the safety of the change, in particular the taking away of the control of the memory decode bit from Dom0 (over certain periods of time). Without that I'm afraid I can't answer your question (and wouldn't want to review the change in detail).
Jan
PATCH 3/3 x86/pci: Fix racy accesses to MSI-X Control register
In reply to David VrabelConcurrent access the the MSI-X control register are not serialized with a suitable lock. For example, in msix_capability_init() access use the pcidevs_lock() but some calls to msi_set_mask_bit() use the interrupt descriptor lock.
This can lead to MSI-X being incorrectly disabled and subsequent failures due to msix_memory_decoded() calls that check for MSI-X being enabled.
This was seen with some non-compliant hardware that gated MSI-X messages on the per-vector mask bit only (i.e., the MSI-X Enable bit and Function Mask bits in the MSI-X Control register were ignored). An interrupt (with a pending move) for vector 0 would occur while vector 1 was being initialized in msix_capability_init(). Updates the the Control register would race and the vector 1 initialization would intermittently fail with -ENXIO.
Typically a race between initializing a vector and another vector being moved doesn't occur because:
1. Racy Control accesses only occur when MSI-X is (guest) disabled
2 Hardware should only raise interrupts when MSI-X is enabled and unmasked.
3. Xen always sets Function Mask when temporarily enabling MSI-X.
But there may be other race conditions depending on hardware and guest driver behaviour (e.g., disabling MSI-X when a IRQ has a pending move on another PCPU).
Fix this by:
1. Tracking the host and guest enable state in a similar way to the host and guest maskall state. Note that since multiple CPUs can be updating different vectors concurrently, a counter is needed for the host enable state.
2. Add a new lock for serialize the Control read/modify/write sequence.
3. Wrap the above in two helper functions (msix_update_lock(), and msix_update_unlock()), which bracket any MSI-X register updates that require MSI-X to be (temporarily) enabled.
SIM: https://t.corp.amazon.com/P63914633
CR: https://code.amazon.com/reviews/CR-79020945
--- xen/arch/x86/include/asm/msi.h | 3 + xen/arch/x86/msi.c | 215 +++++++++++++++++---------------- xen/drivers/passthrough/msi.c | 1 + 3 files changed, 114 insertions(+), 105 deletions(-)
xen/arch/x86/include/asm/msi.h +3 -0
@@ -237,7 +237,10 @@ struct arch_msix {int table_refcnt[MAX_MSIX_TABLE_PAGES ];int table_idx[MAX_MSIX_TABLE_PAGES ];spinlock_t table_lock; +spinlock_t control_lock;bool host_maskall, guest_maskall; +uint16_t host_enable; +bool guest_enable;domid_t warned; };
xen/arch/x86/msi.c +110 -105
365 lines not on this page · raw message
xen/drivers/passthrough/msi.c +1 -0
@@ -44,6 +44,7 @@ int pdev_msi_init(struct pci_dev *pdev)return -ENOMEM ;spin_lock_init (&msix->table_lock); +spin_lock_init (&msix->control_lock); ctrl =pci_conf_read16 (pdev->sbdf,msix_control_reg (pos)); msix->nr_entries =msix_table_size (ctrl);
Signature · 1 line
2.30.2
Re: [PATCH 3/3] x86/pci: Fix racy accesses to MSI-X Control register
Quoted 37 lines · David Vrabel
Concurrent access the the MSI-X control register are not serialized with a suitable lock. For example, in
msix_capability_init()access use thepcidevs_lock()but some calls tomsi_set_mask_bit()use the interrupt descriptor lock.This can lead to MSI-X being incorrectly disabled and subsequent failures due to
msix_memory_decoded()calls that check for MSI-X being enabled.This was seen with some non-compliant hardware that gated MSI-X messages on the per-vector mask bit only (i.e., the MSI-X Enable bit and Function Mask bits in the MSI-X Control register were ignored). An interrupt (with a pending move) for vector 0 would occur while vector 1 was being initialized in
msix_capability_init(). Updates the the Control register would race and the vector 1 initialization would intermittently fail with-ENXIO.Typically a race between initializing a vector and another vector being moved doesn't occur because:
1. Racy Control accesses only occur when MSI-X is (guest) disabled
2 Hardware should only raise interrupts when MSI-X is enabled and unmasked.
3. Xen always sets Function Mask when temporarily enabling MSI-X.
But there may be other race conditions depending on hardware and guest driver behaviour (e.g., disabling MSI-X when a IRQ has a pending move on another PCPU).
Fix this by:
1. Tracking the host and guest enable state in a similar way to the host and guest maskall state. Note that since multiple CPUs can be updating different vectors concurrently, a counter is needed for the host enable state.
Quoted 8 lines
2. Add a new lock for serialize the Control read/modify/write sequence.
3. Wrap the above in two helper functions (
msix_update_lock(), andmsix_update_unlock()), which bracket any MSI-X register updates that require MSI-X to be (temporarily) enabled.Signed-off-by David Vrabel <[email protected]>
Just a couple of mechanical / style comments, at least for now:
Quoted 3 lines
These tags shouldn't be here, unless they have a meaning in the "upstream" context.
xen/arch/x86/include/asm/msi.h struct arch_msix { · line 242
Quoted 9 lines
xen/arch/x86/include/asm/msi.h +2
-0@@ -237,7 +237,10 @@ struct arch_msix {int table_refcnt[MAX_MSIX_TABLE_PAGES ];int table_idx[MAX_MSIX_TABLE_PAGES ];spinlock_t table_lock; +spinlock_t control_lock;bool host_maskall, guest_maskall; +uint16_t host_enable;
If you want to keep this more narrow than "unsigned int", then please add a BUILD_BUG_ON() against NR_CPUS, so the need to update the field can be easily noticed (in some perhaps distant future).
Quoted 5 lines
xen/arch/x86/include/asm/msi.h +5
-0+static void msix_update_unlock (struct pci_dev *dev,unsigned int pos,uint16_t control) +{ +uint16_t new_control; +unsigned long flags; +
xen/arch/x86/include/asm/msi.h struct arch_msix {
Quoted 10 lines
xen/arch/x86/include/asm/msi.h +10
-0+spin_lock_irqsave (&dev->msix->control_lock, flags); + + dev->msix->host_enable--; + + new_control = control & ~(PCI_MSIX_FLAGS_ENABLE |PCI_MSIX_FLAGS_MASKALL ); + +if ( dev->msix->host_enable || dev->msix->guest_enable ) + new_control |=PCI_MSIX_FLAGS_ENABLE ; +if ( dev->msix->host_maskall || dev->msix->guest_maskall || dev->msix->host_enable ) + new_control |=PCI_MSIX_FLAGS_MASKALL ;
In particular this use of "host_enable" suggests the field wants to be named differently: It makes no sense to derive MASKALL from ENABLE without it being clear (from the name) that the field is an init-time override only.
xen/arch/x86/include/asm/msi.h static void msix_set_enable(struct pci_dev *dev, int enable) · line 355
Quoted 7 lines
xen/arch/x86/include/asm/msi.h +3
-0@@ -299,11 +350,16 @@ static void msix_set_enable(struct pci_dev *dev, int enable) pos =pci_find_cap_offset (seg, bus, slot, func,PCI_CAP_ID_MSIX );if ( pos ) { +spin_lock_irq (&dev->msix->control_lock); + + dev->msix->host_enable = false;
You explicitly say this isn't a boolean, so the initializer would better be 0.
Jan
Re: [PATCH 3/3] x86/pci: Fix racy accesses to MSI-X Control register
Quoted 15 lines · Jan Beulich
xen/arch/x86/include/asm/msi.h struct arch_msix { · line 242
Quoted 10 lines · David Vrabel
xen/arch/x86/include/asm/msi.h +2
-0@@ -237,7 +237,10 @@ struct arch_msix {int table_refcnt[MAX_MSIX_TABLE_PAGES ];int table_idx[MAX_MSIX_TABLE_PAGES ];spinlock_t table_lock; +spinlock_t control_lock;bool host_maskall, guest_maskall; +uint16_t host_enable;If you want to keep this more narrow than "unsigned int", then please add a
BUILD_BUG_ON()againstNR_CPUS, so the need to update the field can be easily noticed (in some perhaps distant future).
This is only incremented:
- while holding the pci_devs lock, or
- while holding a lock for one of the associated IRQs.
Since there are at most 4096 MSI-X vectors (and thus at most 4096 IRQs), the highest value this can be (even with >> 4096 PCPUs) is 4097, thus a uint16_t is fine.
Quoted 12 lines
Quoted 12 lines
xen/arch/x86/include/asm/msi.h +12
-0+static void msix_update_unlock (struct pci_dev *dev,unsigned int pos,uint16_t control) +{ +uint16_t new_control; +unsigned long flags; + +spin_lock_irqsave (&dev->msix->control_lock, flags); + + dev->msix->host_enable--; + + new_control = control & ~(PCI_MSIX_FLAGS_ENABLE |PCI_MSIX_FLAGS_MASKALL ); + +if ( dev->msix->host_enable || dev->msix->guest_enable )
Quoted 8 lines
xen/arch/x86/include/asm/msi.h
Quoted 3 lines
xen/arch/x86/include/asm/msi.h +3
-0+ new_control |=PCI_MSIX_FLAGS_ENABLE ; +if ( dev->msix->host_maskall || dev->msix->guest_maskall || dev->msix->host_enable ) + new_control |=PCI_MSIX_FLAGS_MASKALL ;In particular this use of "
host_enable" suggests the field wants to be named differently: It makes no sense to derive MASKALL from ENABLE without it being clear (from the name) that the field is an init-time override only.
I think the name as-is makes sense. It is analogous to the host_maskall and complements guest_enable. I can't think of a better name, so what name do you suggest.
David
Re: [PATCH 3/3] x86/pci: Fix racy accesses to MSI-X Control register
Quoted 17 lines · David Vrabel
Quoted 15 lines · Jan Beulich
xen/arch/x86/include/asm/msi.h struct arch_msix { · line 242
Quoted 10 lines · David Vrabel
xen/arch/x86/include/asm/msi.h +2
-0@@ -237,7 +237,10 @@ struct arch_msix {int table_refcnt[MAX_MSIX_TABLE_PAGES ];int table_idx[MAX_MSIX_TABLE_PAGES ];spinlock_t table_lock; +spinlock_t control_lock;bool host_maskall, guest_maskall; +uint16_t host_enable;If you want to keep this more narrow than "unsigned int", then please add a
BUILD_BUG_ON()againstNR_CPUS, so the need to update the field can be easily noticed (in some perhaps distant future).
Quoted 4 lines
This is only incremented:
- while holding the
pci_devslock, or
- while holding a lock for one of the associated IRQs.
How do the locks held matter here, especially given that - as iirc you say in the description - neither lock is held uniformly?
Quoted 3 lines
Since there are at most 4096 MSI-X vectors (and thus at most 4096 IRQs), the highest value this can be (even with >> 4096 PCPUs) is 4097, thus a
uint16_tis fine.
Where's the 4096 coming from as a limit for MSI-X vectors? DYM 2048, which is the per-device limit (because the qsize field is 11 bits wide)? If so, yes, I think that's indeed restricting how large the number can get.
Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
xen/arch/x86/include/asm/msi.h +12
-0+static void msix_update_unlock (struct pci_dev *dev,unsigned int pos,uint16_t control) +{ +uint16_t new_control; +unsigned long flags; + +spin_lock_irqsave (&dev->msix->control_lock, flags); + + dev->msix->host_enable--; + + new_control = control & ~(PCI_MSIX_FLAGS_ENABLE |PCI_MSIX_FLAGS_MASKALL ); + +if ( dev->msix->host_enable || dev->msix->guest_enable )Quoted 8 lines
xen/arch/x86/include/asm/msi.h
Quoted 3 lines
xen/arch/x86/include/asm/msi.h +3
-0+ new_control |=PCI_MSIX_FLAGS_ENABLE ; +if ( dev->msix->host_maskall || dev->msix->guest_maskall || dev->msix->host_enable ) + new_control |=PCI_MSIX_FLAGS_MASKALL ;In particular this use of "
host_enable" suggests the field wants to be named differently: It makes no sense to derive MASKALL from ENABLE without it being clear (from the name) that the field is an init-time override only.
Quoted 3 lines
I think the name as-is makes sense. It is analogous to the
host_maskalland complementsguest_enable. I can't think of a better name, so what name do you suggest.
I could only think of less neat ones like host_enable_override or forced_enable or some such. If I had any good name in mind, I would certainly have said so.
Jan
Re: [PATCH 3/3] x86/pci: Fix racy accesses to MSI-X Control register
Quoted 18 lines · Jan Beulich
Quoted 17 lines · David Vrabel
Quoted 15 lines · Jan Beulich
xen/arch/x86/include/asm/msi.h struct arch_msix { · line 242
Quoted 10 lines · David Vrabel
xen/arch/x86/include/asm/msi.h +2
-0@@ -237,7 +237,10 @@ struct arch_msix {int table_refcnt[MAX_MSIX_TABLE_PAGES ];int table_idx[MAX_MSIX_TABLE_PAGES ];spinlock_t table_lock; +spinlock_t control_lock;bool host_maskall, guest_maskall; +uint16_t host_enable;If you want to keep this more narrow than "unsigned int", then please add a
BUILD_BUG_ON()againstNR_CPUS, so the need to update the field can be easily noticed (in some perhaps distant future).
Quoted 7 lines
Quoted 4 lines
This is only incremented:
- while holding the
pci_devslock, or
- while holding a lock for one of the associated IRQs.How do the locks held matter here, especially given that - as iirc you say in the description - neither lock is held uniformly?
Because the usage here is:
lock()
host_enable++
...
host_enable--
unlock()
Quoted 7 lines
Quoted 3 lines
Since there are at most 4096 MSI-X vectors (and thus at most 4096 IRQs), the highest value this can be (even with >> 4096 PCPUs) is 4097, thus a
uint16_tis fine.Where's the 4096 coming from as a limit for MSI-X vectors? DYM 2048, which is the per-device limit (because the qsize field is 11 bits wide)? If so, yes, I think that's indeed restricting how large the number can get.
Yes, I did mean 2048 here.
Quoted 2 lines
I could only think of less neat ones like
host_enable_overrideorforced_enableor some such. If I had any good name in mind, I would > certainly have said so.
host_enable_override works for me.
David