Skip to contentSkip to conversation

pci_get_device call from interrupt in reboot fixups

PATCH In review Denis V. Lunev 4 participants 10 messages Aug 3, 2007 – Aug 7, 2007

lore
Share & tools
lore

Denis V. Lunev

author
#0 lore raw

PATCH 1/1 pci_get_device call from interrupt in reboot fixups

The following calltrace is possible now:
handle_sysrq
machine_emergency_restart
mach_reboot_fixups
pci_get_device
pci_get_subsys
down_read
The patch obtains PCI device during initialization to avoid bothering PCI search engine in interrupt. Devices used in this code are not supposed to be pluggable, so it looks safe to keep them.

Signed-off-by Denis V. Lunev <[email protected]>

---

arch/i386/kernel/reboot_fixups.c +20 -4
index 03e1cce..873ad55 100644
@@ -37,6 +37,7 @@ struct device_fixup {
 	unsigned int vendor;
 	unsigned int device;
 	void (*reboot_fixup)(struct pci_dev *);
+	struct pci_dev *dev;
 };
 
 static struct device_fixup fixups_table[] = {

@@ -49,20 +50,35 @@ static struct device_fixup fixups_table[] = {
  * is a fixup, we call it and we expect to never return from it. if we
  * do return, we keep looking and then eventually fall back to the
  * standard mach_reboot on return.
+ *
+ * Unfortunately, this code can be called from an interrupt and it is
+ * impossible to get PCI device directly. So, lets prepare the list
+ * beforehand.
  */
 void mach_reboot_fixups(void)
 {
 	struct device_fixup *cur;
-	struct pci_dev *dev;
 	int i;
 
 	for (i=0; i < ARRAY_SIZE(fixups_table); i++) {
 		cur = &(fixups_table[i]);
-		dev = pci_get_device(cur->vendor, cur->device, NULL);
-		if (!dev)
+		if (cur->dev == NULL)
 			continue;
 
-		cur->reboot_fixup(dev);
+		cur->reboot_fixup(cur->dev);
+	}
+}
+
+int mach_fixup_init(void)
+{
+	struct device_fixup *cur;
+	int i;
+
+	for (i=0; i < ARRAY_SIZE(fixups_table); i++) {
+		cur = &(fixups_table[i]);
+		cur->dev = pci_get_device(cur->vendor, cur->device, NULL);
 	}
+	return 0;
 }
 
+module_init(mach_fixup_init);
#1 lore raw
Quoted 10 lines · Denis V. Lunev

The following calltrace is possible now:
handle_sysrq
machine_emergency_restart
mach_reboot_fixups
pci_get_device
pci_get_subsys
down_read
The patch obtains PCI device during initialization to avoid bothering PCI search engine in interrupt. Devices used in this code are not supposed to be pluggable, so it looks safe to keep them.

What devices are supposed to be affected here? Are you sure that they can't be removed later? Grabbing references here might mess with them in the future.

thanks,

greg k-h

#2 lore raw
Quoted 12 lines · Greg KH
Quoted 10 lines · Denis V. Lunev

The following calltrace is possible now:
handle_sysrq
machine_emergency_restart
mach_reboot_fixups
pci_get_device
pci_get_subsys
down_read
The patch obtains PCI device during initialization to avoid bothering PCI search engine in interrupt. Devices used in this code are not supposed to be pluggable, so it looks safe to keep them.

Quoted 3 lines

What devices are supposed to be affected here? Are you sure that they can't be removed later? Grabbing references here might mess with them in the future.

Right now the list is the following:
static struct device_fixup fixups_table[] = {
{ PCI_VENDOR_ID_CYRIX, PCI_DEVICE_ID_CYRIX_5530_LEGACY, cs5530a_warm_reset },
{ PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_CS5536_ISA, cs5536_warm_reset }, };

Though, if the approach is not suitable, we can skip fixups if we came from sysrq.

Regards,
Den

#3 lore raw
Quoted 23 lines · Denis V. Lunev
Quoted 12 lines · Greg KH
Quoted 10 lines · Denis V. Lunev

The following calltrace is possible now:
handle_sysrq
machine_emergency_restart
mach_reboot_fixups
pci_get_device
pci_get_subsys
down_read
The patch obtains PCI device during initialization to avoid bothering PCI search engine in interrupt. Devices used in this code are not supposed to be pluggable, so it looks safe to keep them.

Quoted 3 lines

What devices are supposed to be affected here? Are you sure that they can't be removed later? Grabbing references here might mess with them in the future.

Right now the list is the following:
static struct device_fixup fixups_table[] = {
{ PCI_VENDOR_ID_CYRIX, PCI_DEVICE_ID_CYRIX_5530_LEGACY, cs5530a_warm_reset },
{ PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_CS5536_ISA, cs5536_warm_reset }, };

Quoted 2 lines

Though, if the approach is not suitable, we can skip fixups if we came from sysrq.

I don't think we really need to do fixups when we are "crashing" like this. The user really isn't shutting down the kernel as it should normally do.

Andrew, I really don't want to change the PCI core to handle this, as we finally fixed a lot of issues with drivers trying to walk these lists from interrupt context. So if you want to just hide the warning message as we are shutting down, that's fine with me. Or just don't do the fixups. But grabbing a reference to the pci device is unsafe in my opinion and I do not want to do that.

thanks,

greg k-h

#4 lore raw
Quoted 31 lines · On Mon, 6 Aug 2007 19:49:10 -0700 Greg KH <[email protected]> wrote:
Quoted 23 lines · Denis V. Lunev
Quoted 12 lines · Greg KH
Quoted 10 lines · Denis V. Lunev

The following calltrace is possible now:
handle_sysrq
machine_emergency_restart
mach_reboot_fixups
pci_get_device
pci_get_subsys
down_read
The patch obtains PCI device during initialization to avoid bothering PCI search engine in interrupt. Devices used in this code are not supposed to be pluggable, so it looks safe to keep them.

Quoted 3 lines

What devices are supposed to be affected here? Are you sure that they can't be removed later? Grabbing references here might mess with them in the future.

Right now the list is the following:
static struct device_fixup fixups_table[] = {
{ PCI_VENDOR_ID_CYRIX, PCI_DEVICE_ID_CYRIX_5530_LEGACY, cs5530a_warm_reset },
{ PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_CS5536_ISA, cs5536_warm_reset }, };

Quoted 2 lines

Though, if the approach is not suitable, we can skip fixups if we came from sysrq.

I don't think we really need to do fixups when we are "crashing" like this. The user really isn't shutting down the kernel as it should normally do.

Quoted 7 lines

Andrew, I really don't want to change the PCI core to handle this, as we finally fixed a lot of issues with drivers trying to walk these lists from interrupt context. So if you want to just hide the warning message as we are shutting down, that's fine with me. Or just don't do the fixups. But grabbing a reference to the pci device is unsafe in my opinion and I do not want to do that.

OK, good decision ;)

One approach would be for some brave soul to pick his way through the reboot code and ensure that we are correctly and reliably setting system_state to SYSTEM_RESTART, then test that in __might_sleep().

But this does suppress somewhat-useful debugging just because of sysrq-B and I really wouldn't want to utilise the horrid system_state any more that we are presently doing. I think on balance that it would be better if we could do something more targetted, like modify emergency_restart() to test in_interrupt() and to then apologetically set some well-named global flag which will shut up __might_sleep(). Pretty foul, but I can't think of anything better.

#5 lore raw
Quoted 14 lines · On Tue, 7 Aug 2007 00:24:37 -0700 Andrew Morton <[email protected]>…
Quoted 7 lines

Andrew, I really don't want to change the PCI core to handle this, as we finally fixed a lot of issues with drivers trying to walk these lists from interrupt context. So if you want to just hide the warning message as we are shutting down, that's fine with me. Or just don't do the fixups. But grabbing a reference to the pci device is unsafe in my opinion and I do not want to do that.

OK, good decision ;)

One approach would be for some brave soul to pick his way through the reboot code and ensure that we are correctly and reliably setting system_state to SYSTEM_RESTART, then test that in __might_sleep().

Quoted 7 lines

But this does suppress somewhat-useful debugging just because of sysrq-B and I really wouldn't want to utilise the horrid system_state any more that we are presently doing. I think on balance that it would be better if we could do something more targetted, like modify emergency_restart() to test in_interrupt() and to then apologetically set some well-named global flag which will shut up __might_sleep(). Pretty foul, but I can't think of anything better.

ok, this might be better. How about we just stop calling mach_reboot_fixups() at sysrq-B time?

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

handle_sysrq
machine_emergency_restart
mach_reboot_fixups
pci_get_device
pci_get_subsys
down_read

#6 lore raw
Quoted 24 lines · Andrew Morton
Quoted 14 lines · On Tue, 7 Aug 2007 00:24:37 -0700 Andrew Morton <[email protected]>…
Quoted 7 lines

Andrew, I really don't want to change the PCI core to handle this, as we finally fixed a lot of issues with drivers trying to walk these lists from interrupt context. So if you want to just hide the warning message as we are shutting down, that's fine with me. Or just don't do the fixups. But grabbing a reference to the pci device is unsafe in my opinion and I do not want to do that.

OK, good decision ;)

One approach would be for some brave soul to pick his way through the reboot code and ensure that we are correctly and reliably setting system_state to SYSTEM_RESTART, then test that in __might_sleep().

Quoted 7 lines

But this does suppress somewhat-useful debugging just because of sysrq-B and I really wouldn't want to utilise the horrid system_state any more that we are presently doing. I think on balance that it would be better if we could do something more targetted, like modify emergency_restart() to test in_interrupt() and to then apologetically set some well-named global flag which will shut up __might_sleep(). Pretty foul, but I can't think of anything better.

Quoted 2 lines

ok, this might be better. How about we just stop calling mach_reboot_fixups() at sysrq-B time?

Fine with me, but what hardware will be messed up because of this?

thanks,

greg k-h

#7 lore raw
Quoted 23 lines · Greg KH
Quoted 22 lines · Andrew Morton
Quoted 20 lines · On Tue, 7 Aug 2007 00:24:37 -0700 Andrew Morton <[email protected]>…
Quoted 7 lines

Andrew, I really don't want to change the PCI core to handle this, as we finally fixed a lot of issues with drivers trying to walk these lists from interrupt context. So if you want to just hide the warning message as we are shutting down, that's fine with me. Or just don't do the fixups. But grabbing a reference to the pci device is unsafe in my opinion and I do not want to do that.

OK, good decision ;)

One approach would be for some brave soul to pick his way through the reboot code and ensure that we are correctly and reliably setting system_state to SYSTEM_RESTART, then test that in __might_sleep().

But this does suppress somewhat-useful debugging just because of sysrq-B and I really wouldn't want to utilise the horrid system_state any more that we are presently doing. I think on balance that it would be better if we could do something more targetted, like modify emergency_restart() to test in_interrupt() and to then apologetically set some well-named global flag which will shut up __might_sleep(). Pretty foul, but I can't think of anything better.

Quoted 4 lines
Quoted 2 lines

ok, this might be better. How about we just stop calling mach_reboot_fixups() at sysrq-B time?

Fine with me, but what hardware will be messed up because of this?

static struct device_fixup fixups_table[] = {

Quoted 4 lines
Quoted 4 lines

{ PCI_VENDOR_ID_CYRIX, PCI_DEVICE_ID_CYRIX_5530_LEGACY, cs5530a_warm_reset },
{ PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_CS5536_ISA, cs5536_warm_reset }, };

#8 lore raw
In reply to Andrew Morton
Quoted 45 lines · Andrew Morton
Quoted 29 lines · On Mon, 6 Aug 2007 19:49:10 -0700 Greg KH <[email protected]> wrote:
Quoted 22 lines · Denis V. Lunev
Quoted 11 lines · Greg KH
Quoted 10 lines · Denis V. Lunev

The following calltrace is possible now:
handle_sysrq
machine_emergency_restart
mach_reboot_fixups
pci_get_device
pci_get_subsys
down_read
The patch obtains PCI device during initialization to avoid bothering PCI search engine in interrupt. Devices used in this code are not supposed to be pluggable, so it looks safe to keep them.

Quoted 3 lines

What devices are supposed to be affected here? Are you sure that they can't be removed later? Grabbing references here might mess with them in the future.

Right now the list is the following:
static struct device_fixup fixups_table[] = {
{ PCI_VENDOR_ID_CYRIX, PCI_DEVICE_ID_CYRIX_5530_LEGACY, cs5530a_warm_reset },
{ PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_CS5536_ISA, cs5536_warm_reset }, };

Quoted 2 lines

Though, if the approach is not suitable, we can skip fixups if we came from sysrq.

I don't think we really need to do fixups when we are "crashing" like this. The user really isn't shutting down the kernel as it should normally do.

Quoted 7 lines

Andrew, I really don't want to change the PCI core to handle this, as we finally fixed a lot of issues with drivers trying to walk these lists from interrupt context. So if you want to just hide the warning message as we are shutting down, that's fine with me. Or just don't do the fixups. But grabbing a reference to the pci device is unsafe in my opinion and I do not want to do that.

OK, good decision ;)

One approach would be for some brave soul to pick his way through the reboot code and ensure that we are correctly and reliably setting system_state to SYSTEM_RESTART, then test that in __might_sleep().

Quoted 7 lines

But this does suppress somewhat-useful debugging just because of sysrq-B and I really wouldn't want to utilise the horrid system_state any more that we are presently doing. I think on balance that it would be better if we could do something more targetted, like modify emergency_restart() to test in_interrupt() and to then apologetically set some well-named global flag which will shut up __might_sleep(). Pretty foul, but I can't think of anything better.

__might_sleep prevention will solve the problem only partially :( There is a direct WARN_ON(in_interrupt()) in pci_get_subsys.

IMHO, calling down_read(&pci_bus_sem); from sysrq-B is not an option. I'll send a fixup disabling patch in a moment.

#9 lore raw
In reply to Denis V. Lunev
Quoted 11 lines · On Fri, 3 Aug 2007 14:39:24 +0400 "Denis V. Lunev" <[email protected]> wrote:

The following calltrace is possible now:
handle_sysrq
machine_emergency_restart
mach_reboot_fixups
pci_get_device
pci_get_subsys
down_read
The patch obtains PCI device during initialization to avoid bothering PCI search engine in interrupt. Devices used in this code are not supposed to be pluggable, so it looks safe to keep them.

hm.

Quoted 14 lines
arch/i386/kernel/reboot_fixups.c +1 -0
index 03e1cce..873ad55 100644
@@ -37,6 +37,7 @@ struct device_fixup {
 	unsigned int vendor;
 	unsigned int device;
 	void (*reboot_fixup)(struct pci_dev *);
+	struct pci_dev *dev;
 };
 
 static struct device_fixup fixups_table[] = {

arch/i386/kernel/reboot_fixups.c static struct device_fixup fixups_table[] = { · line 56

Quoted 7 lines
arch/i386/kernel/reboot_fixups.c +4 -0
@@ -49,20 +50,35 @@ static struct device_fixup fixups_table[] = {
  * is a fixup, we call it and we expect to never return from it. if we
  * do return, we keep looking and then eventually fall back to the
  * standard mach_reboot on return.
+ *
+ * Unfortunately, this code can be called from an interrupt and it is
+ * impossible to get PCI device directly. So, lets prepare the list
+ * beforehand.

This comment should tell the reader which interrupt path that is (ie: sysrq-B).

Quoted 22 lines
arch/i386/kernel/reboot_fixups.c +8 -4
  */
 void mach_reboot_fixups(void)
 {
 	struct device_fixup *cur;
-	struct pci_dev *dev;
 	int i;
 
 	for (i=0; i < ARRAY_SIZE(fixups_table); i++) {
 		cur = &(fixups_table[i]);
-		dev = pci_get_device(cur->vendor, cur->device, NULL);
-		if (!dev)
+		if (cur->dev == NULL)
 			continue;
 
-		cur->reboot_fixup(dev);
+		cur->reboot_fixup(cur->dev);
+	}
+}
+
+int mach_fixup_init(void)
+{
+	struct device_fixup *cur;

arch/i386/kernel/reboot_fixups.c static struct device_fixup fixups_table[] = { · line 84

Quoted 10 lines
arch/i386/kernel/reboot_fixups.c +7 -0
+	int i;
+
+	for (i=0; i < ARRAY_SIZE(fixups_table); i++) {
+		cur = &(fixups_table[i]);
+		cur->dev = pci_get_device(cur->vendor, cur->device, NULL);
 	}
+	return 0;
 }
 
+module_init(mach_fixup_init);

I'm not sure that we want to make core PCI code capable of being called from interrupt context just for the sake of sysrq-B. It adds complexity and maintenance hassles for something which is largely a debugging feature.

otoh, the patch is faily simple-looking and people _do_ use sysrq-B fairly often so I guess we'll find out if we break it again.

otoh2, perhaps we can find some quicky hack on the sysrq patch to shut up the might_sleep() warnings (which I presume is the only problem which is presently being exhibited?). Something like the unpleasant oops_in_progress, perhaps.

Greg, any preferences?