Refactor msi/msix restore code Part2
PATCH v2 2/3 Refactor msi/msix restore code Part2
xen_initdom_restore_msi_irqs trigger a hypercall to restore addr/data/mask in dom0. It's better to do the same in default_restore_msi_irqs for baremetal.
Move restore of mask in default_restore_msi_irqs, this could avoid mask restored twice in dom0, and the logic for baremetal keep same.
First mask restore is in xen_initdom_restore_msi_irqs->PHYSDEVOP_restore_msi, Second restore is __pci_restore_msix_state->msix_mask_irq.
Mask bits are under full control of xen, and the entry->masked in dom0 kernel is invalid. restore an invalid value to mask register could mask the msix interrupt.
Without fix, qlcnic driver calling pci_reset_function will lost interrupt in dom0.
--- drivers/pci/msi.c | 17 ++++++++++++++--- 1 files changed, 14 insertions(+), 3 deletions(-)
drivers/pci/msi.c +14 -3
@@ -216,6 +216,8 @@ void unmask_msi_irq(struct irq_data *data)#ifdef HAVE_DEFAULT_MSI_RESTORE_IRQS void default_restore_msi_irqs (struct pci_dev *dev,int irq) { +int pos; +u16 control;struct msi_desc *entry; entry =NULL ;
@@ -228,8 +230,19 @@ void default_restore_msi_irqs(struct pci_dev *dev, int irq) entry = irq_get_msi_desc(irq); } -if (entry) +if (entry) {write_msi_msg (irq, &entry->msg); +if (dev->msix_enabled) { +msix_mask_irq (entry, entry->masked); +readl (entry->mask_base); + }else { + pos = entry->msi_attrib.pos; +pci_read_config_word (dev, pos +PCI_MSI_FLAGS , + &control); +msi_mask_irq (entry,msi_capable_mask (control), + entry->masked); + } + } }#endif
@@ -406,7 +419,6 @@ static void __pci_restore_msi_state(struct pci_dev *dev)arch_restore_msi_irqs (dev, dev->irq);pci_read_config_word (dev, dev->msi_cap +PCI_MSI_FLAGS , &control); -msi_mask_irq (entry,msi_capable_mask (control), entry->masked); control &= ~PCI_MSI_FLAGS_QSIZE ; control |= (entry->msi_attrib.multiple <<4 ) |PCI_MSI_FLAGS_ENABLE ;pci_write_config_word (dev, dev->msi_cap +PCI_MSI_FLAGS , control);
@@ -430,7 +442,6 @@ static void __pci_restore_msix_state(struct pci_dev *dev)list_for_each_entry (entry, &dev->msi_list, list) {arch_restore_msi_irqs (dev, entry->irq); -msix_mask_irq (entry, entry->masked); } control &= ~PCI_MSIX_FLAGS_MASKALL ;
Signature · 1 line
1.7.3
Bjorn Helgaas
maintainerQuoted 9 lines · Zhenzhong Duan
xen_initdom_restore_msi_irqstrigger a hypercall to restore addr/data/mask in dom0. It's better to do the same indefault_restore_msi_irqsfor baremetal.Move restore of mask in
default_restore_msi_irqs, this could avoid mask restored twice in dom0, and the logic for baremetal keep same.First mask restore is in
xen_initdom_restore_msi_irqs->PHYSDEVOP_restore_msi, Second restore is__pci_restore_msix_state->msix_mask_irq.
Quoted 6 lines
Mask bits are under full control of xen, and the
entry->maskedin dom0 kernel is invalid. restore an invalid value to mask register could mask the msix interrupt.Without fix, qlcnic driver calling
pci_reset_functionwill lost interrupt in dom0.
Konrad, this changelog still doesn't make any sense to me, but if you ack this, I guess I can apply it.
I guess there are also:
Jul 24 [PATCH 1/3] Refactor msi/msix restore code Part1 Jul 30 [PATCH 3/3 v2] Update x86_msi.restore_msi_irqs API param
and all three should be applied as a series?
Quoted 59 lines
Tested-by Sucheta Chakraborty <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>--- drivers/pci/msi.c | 17 ++++++++++++++--- 1 files changed, 14 insertions(+), 3 deletions(-)drivers/pci/msi.c +14
-4@@ -216,6 +216,8 @@ void unmask_msi_irq(struct irq_data *data)#ifdef HAVE_DEFAULT_MSI_RESTORE_IRQS void default_restore_msi_irqs (struct pci_dev *dev,int irq) { +int pos; +u16 control;struct msi_desc *entry; entry =NULL ;@@ -228,8 +230,19 @@ void default_restore_msi_irqs(struct pci_dev *dev, int irq) entry = irq_get_msi_desc(irq); } -if (entry) +if (entry) {write_msi_msg (irq, &entry->msg); +if (dev->msix_enabled) { +msix_mask_irq (entry, entry->masked); +readl (entry->mask_base); + }else { + pos = entry->msi_attrib.pos; +pci_read_config_word (dev, pos +PCI_MSI_FLAGS , + &control); +msi_mask_irq (entry,msi_capable_mask (control), + entry->masked); + } + } }#endif @@ -406,7 +419,6 @@ static void __pci_restore_msi_state(struct pci_dev *dev)arch_restore_msi_irqs (dev, dev->irq);pci_read_config_word (dev, dev->msi_cap +PCI_MSI_FLAGS , &control); -msi_mask_irq (entry,msi_capable_mask (control), entry->masked); control &= ~PCI_MSI_FLAGS_QSIZE ; control |= (entry->msi_attrib.multiple <<4 ) |PCI_MSI_FLAGS_ENABLE ;pci_write_config_word (dev, dev->msi_cap +PCI_MSI_FLAGS , control);@@ -430,7 +442,6 @@ static void __pci_restore_msix_state(struct pci_dev *dev)list_for_each_entry (entry, &dev->msi_list, list) {arch_restore_msi_irqs (dev, entry->irq); -msix_mask_irq (entry, entry->masked); } control &= ~PCI_MSIX_FLAGS_MASKALL ; --1.7.3
Bjorn Helgaas
maintainerQuoted 9 lines · Zhenzhong Duan
xen_initdom_restore_msi_irqstrigger a hypercall to restore addr/data/mask in dom0. It's better to do the same indefault_restore_msi_irqsfor baremetal.Move restore of mask in
default_restore_msi_irqs, this could avoid mask restored twice in dom0, and the logic for baremetal keep same.First mask restore is in
xen_initdom_restore_msi_irqs->PHYSDEVOP_restore_msi, Second restore is__pci_restore_msix_state->msix_mask_irq.
Quoted 6 lines
Mask bits are under full control of xen, and the
entry->maskedin dom0 kernel is invalid. restore an invalid value to mask register could mask the msix interrupt.Without fix, qlcnic driver calling
pci_reset_functionwill lost interrupt in dom0.
Konrad, this changelog still doesn't make any sense to me, but if you ack this, I guess I can apply it.
I guess there are also:
Jul 24 [PATCH 1/3] Refactor msi/msix restore code Part1 Jul 30 [PATCH 3/3 v2] Update x86_msi.restore_msi_irqs API param
and all three should be applied as a series?
Quoted 59 lines
Tested-by Sucheta Chakraborty <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>--- drivers/pci/msi.c | 17 ++++++++++++++--- 1 files changed, 14 insertions(+), 3 deletions(-)drivers/pci/msi.c +14
-4@@ -216,6 +216,8 @@ void unmask_msi_irq(struct irq_data *data)#ifdef HAVE_DEFAULT_MSI_RESTORE_IRQS void default_restore_msi_irqs (struct pci_dev *dev,int irq) { +int pos; +u16 control;struct msi_desc *entry; entry =NULL ;@@ -228,8 +230,19 @@ void default_restore_msi_irqs(struct pci_dev *dev, int irq) entry = irq_get_msi_desc(irq); } -if (entry) +if (entry) {write_msi_msg (irq, &entry->msg); +if (dev->msix_enabled) { +msix_mask_irq (entry, entry->masked); +readl (entry->mask_base); + }else { + pos = entry->msi_attrib.pos; +pci_read_config_word (dev, pos +PCI_MSI_FLAGS , + &control); +msi_mask_irq (entry,msi_capable_mask (control), + entry->masked); + } + } }#endif @@ -406,7 +419,6 @@ static void __pci_restore_msi_state(struct pci_dev *dev)arch_restore_msi_irqs (dev, dev->irq);pci_read_config_word (dev, dev->msi_cap +PCI_MSI_FLAGS , &control); -msi_mask_irq (entry,msi_capable_mask (control), entry->masked); control &= ~PCI_MSI_FLAGS_QSIZE ; control |= (entry->msi_attrib.multiple <<4 ) |PCI_MSI_FLAGS_ENABLE ;pci_write_config_word (dev, dev->msi_cap +PCI_MSI_FLAGS , control);@@ -430,7 +442,6 @@ static void __pci_restore_msix_state(struct pci_dev *dev)list_for_each_entry (entry, &dev->msi_list, list) {arch_restore_msi_irqs (dev, entry->irq); -msix_mask_irq (entry, entry->masked); } control &= ~PCI_MSIX_FLAGS_MASKALL ; --1.7.3
Quoted 15 lines · Bjorn Helgaas
Quoted 13 lines · Zhenzhong Duan
xen_initdom_restore_msi_irqstrigger a hypercall to restore addr/data/mask in dom0. It's better to do the same indefault_restore_msi_irqsfor baremetal.Move restore of mask in
default_restore_msi_irqs, this could avoid mask restored twice in dom0, and the logic for baremetal keep same.First mask restore is in
xen_initdom_restore_msi_irqs->PHYSDEVOP_restore_msi, Second restore is__pci_restore_msix_state->msix_mask_irq.Mask bits are under full control of xen, and the
entry->maskedin dom0 kernel is invalid. restore an invalid value to mask register could mask the msix interrupt.
Quoted 5 lines
Quoted 2 lines
Without fix, qlcnic driver calling
pci_reset_functionwill lost interrupt in dom0.Konrad, this changelog still doesn't make any sense to me, but if you ack this, I guess I can apply it.
Hey Bjorn,
Zhenzhong is patiently working to rewrite up the commit message based on my naive questions and emails back and forth. Once it is good shape he will post it. The code will look the same but the commit message will be a bit more verbose and clear.
Is there an ETA when you would like these? I recall the merge window is just around the corner - so when is your comfortable cut-off-day so that you can make a go/no-go decision?
Quoted 7 lines
I guess there are also:
Jul 24 [PATCH 1/3] Refactor msi/msix restore code Part1 Jul 30 [PATCH 3/3 v2] Update x86_msi.restore_msi_irqs API param
and all three should be applied as a series?
<scratches his head>.
I think the
Jul 30 [PATCH 3/3 v2] Update x86_msi.restore_msi_irqs API param
can go in anytime. That is mostly a cosmetic fixup in the API. Zhenzhong - right?
For the
Jul 24 [PATCH 1/3] Refactor msi/msix restore code Part1
I would have to look back at the code - as I can't remember what it was refactoring :-(
Quoted 60 lines
Quoted 60 lines
Tested-by Sucheta Chakraborty <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>--- drivers/pci/msi.c | 17 ++++++++++++++--- 1 files changed, 14 insertions(+), 3 deletions(-)drivers/pci/msi.c +14
-4@@ -216,6 +216,8 @@ void unmask_msi_irq(struct irq_data *data)#ifdef HAVE_DEFAULT_MSI_RESTORE_IRQS void default_restore_msi_irqs (struct pci_dev *dev,int irq) { +int pos; +u16 control;struct msi_desc *entry; entry =NULL ;@@ -228,8 +230,19 @@ void default_restore_msi_irqs(struct pci_dev *dev, int irq) entry = irq_get_msi_desc(irq); } -if (entry) +if (entry) {write_msi_msg (irq, &entry->msg); +if (dev->msix_enabled) { +msix_mask_irq (entry, entry->masked); +readl (entry->mask_base); + }else { + pos = entry->msi_attrib.pos; +pci_read_config_word (dev, pos +PCI_MSI_FLAGS , + &control); +msi_mask_irq (entry,msi_capable_mask (control), + entry->masked); + } + } }#endif @@ -406,7 +419,6 @@ static void __pci_restore_msi_state(struct pci_dev *dev)arch_restore_msi_irqs (dev, dev->irq);pci_read_config_word (dev, dev->msi_cap +PCI_MSI_FLAGS , &control); -msi_mask_irq (entry,msi_capable_mask (control), entry->masked); control &= ~PCI_MSI_FLAGS_QSIZE ; control |= (entry->msi_attrib.multiple <<4 ) |PCI_MSI_FLAGS_ENABLE ;pci_write_config_word (dev, dev->msi_cap +PCI_MSI_FLAGS , control);@@ -430,7 +442,6 @@ static void __pci_restore_msix_state(struct pci_dev *dev)list_for_each_entry (entry, &dev->msi_list, list) {arch_restore_msi_irqs (dev, entry->irq); -msix_mask_irq (entry, entry->masked); } control &= ~PCI_MSIX_FLAGS_MASKALL ; --1.7.3
Quoted 15 lines · Bjorn Helgaas
Quoted 13 lines · Zhenzhong Duan
xen_initdom_restore_msi_irqstrigger a hypercall to restore addr/data/mask in dom0. It's better to do the same indefault_restore_msi_irqsfor baremetal.Move restore of mask in
default_restore_msi_irqs, this could avoid mask restored twice in dom0, and the logic for baremetal keep same.First mask restore is in
xen_initdom_restore_msi_irqs->PHYSDEVOP_restore_msi, Second restore is__pci_restore_msix_state->msix_mask_irq.Mask bits are under full control of xen, and the
entry->maskedin dom0 kernel is invalid. restore an invalid value to mask register could mask the msix interrupt.
Quoted 5 lines
Quoted 2 lines
Without fix, qlcnic driver calling
pci_reset_functionwill lost interrupt in dom0.Konrad, this changelog still doesn't make any sense to me, but if you ack this, I guess I can apply it.
Hey Bjorn,
Zhenzhong is patiently working to rewrite up the commit message based on my naive questions and emails back and forth. Once it is good shape he will post it. The code will look the same but the commit message will be a bit more verbose and clear.
Is there an ETA when you would like these? I recall the merge window is just around the corner - so when is your comfortable cut-off-day so that you can make a go/no-go decision?
Quoted 7 lines
I guess there are also:
Jul 24 [PATCH 1/3] Refactor msi/msix restore code Part1 Jul 30 [PATCH 3/3 v2] Update x86_msi.restore_msi_irqs API param
and all three should be applied as a series?
<scratches his head>.
I think the
Jul 30 [PATCH 3/3 v2] Update x86_msi.restore_msi_irqs API param
can go in anytime. That is mostly a cosmetic fixup in the API. Zhenzhong - right?
For the
Jul 24 [PATCH 1/3] Refactor msi/msix restore code Part1
I would have to look back at the code - as I can't remember what it was refactoring :-(
Quoted 60 lines
Quoted 60 lines
Tested-by Sucheta Chakraborty <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>--- drivers/pci/msi.c | 17 ++++++++++++++--- 1 files changed, 14 insertions(+), 3 deletions(-)drivers/pci/msi.c +14
-4@@ -216,6 +216,8 @@ void unmask_msi_irq(struct irq_data *data)#ifdef HAVE_DEFAULT_MSI_RESTORE_IRQS void default_restore_msi_irqs (struct pci_dev *dev,int irq) { +int pos; +u16 control;struct msi_desc *entry; entry =NULL ;@@ -228,8 +230,19 @@ void default_restore_msi_irqs(struct pci_dev *dev, int irq) entry = irq_get_msi_desc(irq); } -if (entry) +if (entry) {write_msi_msg (irq, &entry->msg); +if (dev->msix_enabled) { +msix_mask_irq (entry, entry->masked); +readl (entry->mask_base); + }else { + pos = entry->msi_attrib.pos; +pci_read_config_word (dev, pos +PCI_MSI_FLAGS , + &control); +msi_mask_irq (entry,msi_capable_mask (control), + entry->masked); + } + } }#endif @@ -406,7 +419,6 @@ static void __pci_restore_msi_state(struct pci_dev *dev)arch_restore_msi_irqs (dev, dev->irq);pci_read_config_word (dev, dev->msi_cap +PCI_MSI_FLAGS , &control); -msi_mask_irq (entry,msi_capable_mask (control), entry->masked); control &= ~PCI_MSI_FLAGS_QSIZE ; control |= (entry->msi_attrib.multiple <<4 ) |PCI_MSI_FLAGS_ENABLE ;pci_write_config_word (dev, dev->msi_cap +PCI_MSI_FLAGS , control);@@ -430,7 +442,6 @@ static void __pci_restore_msix_state(struct pci_dev *dev)list_for_each_entry (entry, &dev->msi_list, list) {arch_restore_msi_irqs (dev, entry->irq); -msix_mask_irq (entry, entry->masked); } control &= ~PCI_MSIX_FLAGS_MASKALL ; --1.7.3
Quoted 36 lines · Konrad Rzeszutek Wilk
Quoted 11 lines · Bjorn Helgaas
Quoted 9 lines · Zhenzhong Duan
xen_initdom_restore_msi_irqstrigger a hypercall to restore addr/data/mask in dom0. It's better to do the same indefault_restore_msi_irqsfor baremetal.Move restore of mask in
default_restore_msi_irqs, this could avoid mask restored twice in dom0, and the logic for baremetal keep same.First mask restore is in
xen_initdom_restore_msi_irqs->PHYSDEVOP_restore_msi, Second restore is__pci_restore_msix_state->msix_mask_irq.Quoted 8 lines
Quoted 6 lines
Mask bits are under full control of xen, and the
entry->maskedin dom0 kernel is invalid. restore an invalid value to mask register could mask the msix interrupt.Without fix, qlcnic driver calling
pci_reset_functionwill lost interrupt in dom0.Konrad, this changelog still doesn't make any sense to me, but if you ack this, I guess I can apply it.
Hey Bjorn,
Zhenzhong is patiently working to rewrite up the commit message based on my naive questions and emails back and forth. Once it is good shape he will post it. The code will look the same but the commit message will be a bit more verbose and clear.
Is there an ETA when you would like these? I recall the merge window is just around the corner - so when is your comfortable cut-off-day so that you can make a go/no-go decision?
Quoted 5 lines
I guess there are also:
Jul 24 [PATCH 1/3] Refactor msi/msix restore code Part1 Jul 30 [PATCH 3/3 v2] Update x86_msi.restore_msi_irqs API param
Quoted 8 lines
Quoted 1 line
and all three should be applied as a series?
<scratches his head>.
I think the
Jul 30 [PATCH 3/3 v2] Update x86_msi.restore_msi_irqs API paramcan go in anytime. That is mostly a cosmetic fixup in the API. Zhenzhong - right?
Yes, 3rd patch doesn't depend on the first two.
One effect is making code looks consistent. But my main purpose is optimizing msix restore code in initial domain.
Before patch, dom0 calls PHYSDEVOP_restore_msi hypercall for every entry in dev->msi_list, now it calls PHYSDEVOP_restore_msi hypercall once.
Quoted 36 lines · Konrad Rzeszutek Wilk
Quoted 11 lines · Bjorn Helgaas
Quoted 9 lines · Zhenzhong Duan
xen_initdom_restore_msi_irqstrigger a hypercall to restore addr/data/mask in dom0. It's better to do the same indefault_restore_msi_irqsfor baremetal.Move restore of mask in
default_restore_msi_irqs, this could avoid mask restored twice in dom0, and the logic for baremetal keep same.First mask restore is in
xen_initdom_restore_msi_irqs->PHYSDEVOP_restore_msi, Second restore is__pci_restore_msix_state->msix_mask_irq.Quoted 8 lines
Quoted 6 lines
Mask bits are under full control of xen, and the
entry->maskedin dom0 kernel is invalid. restore an invalid value to mask register could mask the msix interrupt.Without fix, qlcnic driver calling
pci_reset_functionwill lost interrupt in dom0.Konrad, this changelog still doesn't make any sense to me, but if you ack this, I guess I can apply it.
Hey Bjorn,
Zhenzhong is patiently working to rewrite up the commit message based on my naive questions and emails back and forth. Once it is good shape he will post it. The code will look the same but the commit message will be a bit more verbose and clear.
Is there an ETA when you would like these? I recall the merge window is just around the corner - so when is your comfortable cut-off-day so that you can make a go/no-go decision?
Quoted 5 lines
I guess there are also:
Jul 24 [PATCH 1/3] Refactor msi/msix restore code Part1 Jul 30 [PATCH 3/3 v2] Update x86_msi.restore_msi_irqs API param
Quoted 8 lines
Quoted 1 line
and all three should be applied as a series?
<scratches his head>.
I think the
Jul 30 [PATCH 3/3 v2] Update x86_msi.restore_msi_irqs API paramcan go in anytime. That is mostly a cosmetic fixup in the API. Zhenzhong - right?
Yes, 3rd patch doesn't depend on the first two.
One effect is making code looks consistent. But my main purpose is optimizing msix restore code in initial domain.
Before patch, dom0 calls PHYSDEVOP_restore_msi hypercall for every entry in dev->msi_list, now it calls PHYSDEVOP_restore_msi hypercall once.