PM domain change on unbound devices warning on ipmi_si unload
Starting in 4.5-rc1, I noticed this warning on ipmi_si driver removal:
% modprobe ipmi_si
% rmmod ipmi_si
bus: 'platform': driver_probe_device: matched device IPI0001:00 with driver ipmi_si
bus: 'platform': really_probe: probing driver ipmi_si with device IPI0001:00 ipmi_si IPI0001:00: ipmi_si: probing via ACPIipmi_si IPI0001:00: [io 0x0ca2-0x0ca3] regsize 1 spacing 1 irq 0ipmi_si: Adding ACPI-specified kcs state machine
driver: 'ipmi_si': driver_bound: bound to device 'IPI0001:00'
bus: 'platform': really_probe: bound device IPI0001:00 to driver ipmi_si
IPMI System Interface driver.ipmi_si: probing via SMBIOSipmi_si: SMBIOS: io 0xda2 regsize 1 spacing 1 irq 0ipmi_si: Adding SMBIOS-specified kcs state machineipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0 Registering platform device 'ipmi_bmc.0674.66'. Parent at platform
driver: 'ipmi': driver_bound: bound to device 'ipmi_bmc.0674.66'ipmi_si IPI0001:00: Found new BMC (man_id: 0x000077, prod_id: 0x0674, dev_id: 0x42) ipmi_si IPI0001:00: IPMI kcs interface initialized
------------[ cut here ]------------ WARNING: CPU: 39 PID: 3678 at drivers/base/power/common.c:150 dev_pm_domain_set+0x52/0x60() PM domains can only be changed for unbound devices [ ... snip ... ] CPU: 39 PID: 3678 Comm: rmmod Tainted: G OE 4.5.0-rc1+ #57 Hardware name: Stratus ftServer 6800/G7LYY, BIOS BIOS Version 8.1:61 09/10/2015 0000000000000000 000000003883b68d ffff8820334d7d30 ffffffff8132caf0 ffff8820334d7d78 ffff8820334d7d68 ffffffff8107f1b6 ffff8810351eca00 0000000000000000 0000000000000001 0000000001c2a330 0000000001c29010 Call Trace: [<ffffffff8132caf0>] dump_stack+0x44/0x64 [<ffffffff8107f1b6>] warn_slowpath_common+0x86/0xc0 [<ffffffff8107f24c>] warn_slowpath_fmt+0x5c/0x80 [<ffffffff8145e152>] dev_pm_domain_set+0x52/0x60 [<ffffffff813a38ae>] acpi_dev_pm_detach+0x3f/0x84 [<ffffffff8145e0d7>] dev_pm_domain_detach+0x27/0x30 [<ffffffff814575f8>] platform_drv_remove+0x38/0x40 [<ffffffff814557ba>] __device_release_driver+0x9a/0x140 [<ffffffff81455968>] driver_detach+0xb8/0xc0 [<ffffffff814547b5>] bus_remove_driver+0x55/0xd0 [<ffffffff814560cc>] driver_unregister+0x2c/0x50 [<ffffffff814576b2>] platform_driver_unregister+0x12/0x20 [<ffffffffa02586c9>] cleanup_ipmi_si+0x29/0xa0 [ipmi_si] [<ffffffff81102100>] SyS_delete_module+0x190/0x220 [<ffffffff8167ffee>] entry_SYSCALL_64_fastpath+0x12/0x71 ---[ end trace 671ca97b9ac15462 ]---
My platform has two BMCs (perhaps this is messing with a refcount somewhere), but I wonder about the ordering of this code:
__device_release_driver(struct device *dev)
drv->remove(dev);
[ platform_drv_remove ]
...
dev_pm_domain_detach
device_is_bound
return dev->p && klist_node_attached(&dev->p->knode_driver)
...
klist_remove(&dev->p->knode_driver);Is the klist_remove at the bottom of __device_release_driver necessary to satisfy the earlier check in dev_pm_domain_detach's device_is_bound assertion? If so, could these be out of order?
This is core driver code, so I'm assuming it's not something as simple as the following (which avoided the warning on unload at least). Any suggestions or extra debugging ideas welcome! This occurs on every unload, so I'd be glad to test real solutions :)
Thanks,
-- Joe
-->8--
drivers/base/dd.c +1 -1
@@ -756,6 +756,7 @@ static void __device_release_driver(struct device *dev)pm_runtime_put_sync (dev); +klist_remove (&dev->p->knode_driver);if (dev->bus && dev->bus->remove) dev->bus->remove (dev);else if (drv->remove)
@@ -767,7 +768,6 @@ static void __device_release_driver(struct device *dev) dev->pm_domain->dismiss (dev);pm_runtime_reinit (dev); -klist_remove (&dev->p->knode_driver);device_pm_check_callbacks (dev);if (dev->bus) blocking_notifier_call_chain(&dev->bus->p->bus_notifier,
Quoted 4 lines · Joe Lawrence
Starting in 4.5-rc1, I noticed this warning on
ipmi_sidriver removal:% modprobe
ipmi_si
% rmmodipmi_si
Yes, I had just noticed this yesterday and I was going to try to trace it down today. I had assumed it was an IPMI driver problem, but after looking at it a bit, I don't think it is. I'll try to bisect to see what caused this, nothing obvious jumps out
-corey
Quoted 93 lines
bus: 'platform':
driver_probe_device: matched device IPI0001:00 with driveripmi_si
bus: 'platform':really_probe: probing driveripmi_siwith device IPI0001:00ipmi_siIPI0001:00:ipmi_si: probing via ACPIipmi_siIPI0001:00: [io 0x0ca2-0x0ca3] regsize 1 spacing 1 irq 0ipmi_si: Adding ACPI-specified kcs state machine
driver: 'ipmi_si':driver_bound: bound to device 'IPI0001:00'
bus: 'platform':really_probe: bound device IPI0001:00 to driveripmi_si
IPMI System Interface driver.ipmi_si: probing via SMBIOSipmi_si: SMBIOS: io0xda2regsize 1 spacing 1 irq 0ipmi_si: Adding SMBIOS-specified kcs state machineipmi_si: Trying ACPI-specified kcs state machine at i/o address0xca2, slave address0x0, irq 0 Registering platform device 'ipmi_bmc.0674.66'. Parent at platform
driver: 'ipmi':driver_bound: bound to device 'ipmi_bmc.0674.66'ipmi_siIPI0001:00: Found new BMC (man_id:0x000077,prod_id:0x0674,dev_id:0x42)ipmi_siIPI0001:00: IPMI kcs interface initialized------------[ cut here ]------------ WARNING: CPU: 39 PID: 3678 at drivers/base/power/common.c:150 dev_pm_domain_set+0x52/0x60() PM domains can only be changed for unbound devices [ ... snip ... ] CPU: 39 PID: 3678 Comm: rmmod Tainted: G OE 4.5.0-rc1+ #57 Hardware name: Stratus ftServer 6800/G7LYY, BIOS BIOS Version 8.1:61 09/10/2015 0000000000000000 000000003883b68d ffff8820334d7d30 ffffffff8132caf0 ffff8820334d7d78 ffff8820334d7d68 ffffffff8107f1b6 ffff8810351eca00 0000000000000000 0000000000000001 0000000001c2a330 0000000001c29010 Call Trace: [<ffffffff8132caf0>] dump_stack+0x44/0x64 [<ffffffff8107f1b6>] warn_slowpath_common+0x86/0xc0 [<ffffffff8107f24c>] warn_slowpath_fmt+0x5c/0x80 [<ffffffff8145e152>] dev_pm_domain_set+0x52/0x60 [<ffffffff813a38ae>] acpi_dev_pm_detach+0x3f/0x84 [<ffffffff8145e0d7>] dev_pm_domain_detach+0x27/0x30 [<ffffffff814575f8>] platform_drv_remove+0x38/0x40 [<ffffffff814557ba>] __device_release_driver+0x9a/0x140 [<ffffffff81455968>] driver_detach+0xb8/0xc0 [<ffffffff814547b5>] bus_remove_driver+0x55/0xd0 [<ffffffff814560cc>] driver_unregister+0x2c/0x50 [<ffffffff814576b2>] platform_driver_unregister+0x12/0x20 [<ffffffffa02586c9>] cleanup_ipmi_si+0x29/0xa0 [ipmi_si] [<ffffffff81102100>] SyS_delete_module+0x190/0x220 [<ffffffff8167ffee>] entry_SYSCALL_64_fastpath+0x12/0x71 ---[ end trace 671ca97b9ac15462 ]---My platform has two BMCs (perhaps this is messing with a refcount somewhere), but I wonder about the ordering of this code:
__device_release_driver(struct device *dev)drv->remove(dev); [ platform_drv_remove ] ... dev_pm_domain_detach device_is_bound return dev->p && klist_node_attached(&dev->p->knode_driver) ... klist_remove(&dev->p->knode_driver);Is the
klist_removeat the bottom of__device_release_drivernecessary to satisfy the earlier check in dev_pm_domain_detach'sdevice_is_boundassertion? If so, could these be out of order?This is core driver code, so I'm assuming it's not something as simple as the following (which avoided the warning on unload at least). Any suggestions or extra debugging ideas welcome! This occurs on every unload, so I'd be glad to test real solutions :)
Thanks,
-- Joe
-->8--
drivers/base/dd.c +1
-1@@ -756,6 +756,7 @@ static void __device_release_driver(struct device *dev)pm_runtime_put_sync (dev); +klist_remove (&dev->p->knode_driver);if (dev->bus && dev->bus->remove) dev->bus->remove (dev);else if (drv->remove)@@ -767,7 +768,6 @@ static void __device_release_driver(struct device *dev) dev->pm_domain->dismiss (dev);pm_runtime_reinit (dev); -klist_remove (&dev->p->knode_driver);device_pm_check_callbacks (dev);if (dev->bus) blocking_notifier_call_chain(&dev->bus->p->bus_notifier,
Tomeu, you added that check in
[989561de9b5112999475b406557d9c7e9e59c041] PM / Domains: add setter for dev.pm_domain
and either something is wrong in the platform device handling or elsewhere in the device code, if that check is valid.
-corey
Quoted 108 lines · Corey Minyard
Quoted 4 lines · Joe Lawrence
Starting in 4.5-rc1, I noticed this warning on
ipmi_sidriver removal:% modprobe
ipmi_si
% rmmodipmi_siYes, I had just noticed this yesterday and I was going to try to trace it down today. I had assumed it was an IPMI driver problem, but after looking at it a bit, I don't think it is. I'll try to bisect to see what caused this, nothing obvious jumps out
-corey
Quoted 95 lines
bus: 'platform':
driver_probe_device: matched device IPI0001:00 with driveripmi_si
bus: 'platform':really_probe: probing driveripmi_siwith device IPI0001:00ipmi_siIPI0001:00:ipmi_si: probing via ACPIipmi_siIPI0001:00: [io 0x0ca2-0x0ca3] regsize 1 spacing 1 irq 0ipmi_si: Adding ACPI-specified kcs state machine
driver: 'ipmi_si':driver_bound: bound to device 'IPI0001:00'
bus: 'platform':really_probe: bound device IPI0001:00 to driveripmi_si
IPMI System Interface driver.ipmi_si: probing via SMBIOSipmi_si: SMBIOS: io0xda2regsize 1 spacing 1 irq 0ipmi_si: Adding SMBIOS-specified kcs state machineipmi_si: Trying ACPI-specified kcs state machine at i/o address0xca2, slave address0x0, irq 0 Registering platform device 'ipmi_bmc.0674.66'. Parent at platform
driver: 'ipmi':driver_bound: bound to device 'ipmi_bmc.0674.66'ipmi_siIPI0001:00: Found new BMC (man_id:0x000077,prod_id:0x0674,dev_id:0x42)ipmi_siIPI0001:00: IPMI kcs interface initialized------------[ cut here ]------------ WARNING: CPU: 39 PID: 3678 at drivers/base/power/common.c:150 dev_pm_domain_set+0x52/0x60() PM domains can only be changed for unbound devices [ ... snip ... ] CPU: 39 PID: 3678 Comm: rmmod Tainted: G OE 4.5.0-rc1+ #57 Hardware name: Stratus ftServer 6800/G7LYY, BIOS BIOS Version 8.1:61 09/10/2015 0000000000000000 000000003883b68d ffff8820334d7d30 ffffffff8132caf0 ffff8820334d7d78 ffff8820334d7d68 ffffffff8107f1b6 ffff8810351eca00 0000000000000000 0000000000000001 0000000001c2a330 0000000001c29010 Call Trace: [<ffffffff8132caf0>] dump_stack+0x44/0x64 [<ffffffff8107f1b6>] warn_slowpath_common+0x86/0xc0 [<ffffffff8107f24c>] warn_slowpath_fmt+0x5c/0x80 [<ffffffff8145e152>] dev_pm_domain_set+0x52/0x60 [<ffffffff813a38ae>] acpi_dev_pm_detach+0x3f/0x84 [<ffffffff8145e0d7>] dev_pm_domain_detach+0x27/0x30 [<ffffffff814575f8>] platform_drv_remove+0x38/0x40 [<ffffffff814557ba>] __device_release_driver+0x9a/0x140 [<ffffffff81455968>] driver_detach+0xb8/0xc0 [<ffffffff814547b5>] bus_remove_driver+0x55/0xd0 [<ffffffff814560cc>] driver_unregister+0x2c/0x50 [<ffffffff814576b2>] platform_driver_unregister+0x12/0x20 [<ffffffffa02586c9>] cleanup_ipmi_si+0x29/0xa0 [ipmi_si] [<ffffffff81102100>] SyS_delete_module+0x190/0x220 [<ffffffff8167ffee>] entry_SYSCALL_64_fastpath+0x12/0x71 ---[ end trace 671ca97b9ac15462 ]---My platform has two BMCs (perhaps this is messing with a refcount somewhere), but I wonder about the ordering of this code:
__device_release_driver(struct device *dev)drv->remove(dev); [ platform_drv_remove ] ... dev_pm_domain_detach device_is_bound return dev->p && klist_node_attached(&dev->p->knode_driver) ... klist_remove(&dev->p->knode_driver);Is the
klist_removeat the bottom of__device_release_drivernecessary to satisfy the earlier check in dev_pm_domain_detach'sdevice_is_boundassertion? If so, could these be out of order?This is core driver code, so I'm assuming it's not something as simple as the following (which avoided the warning on unload at least). Any suggestions or extra debugging ideas welcome! This occurs on every unload, so I'd be glad to test real solutions :)
Thanks,
-- Joe
-->8--
drivers/base/dd.c +1
-0@@ -756,6 +756,7 @@ static void __device_release_driver(struct device *dev)pm_runtime_put_sync (dev); +klist_remove (&dev->p->knode_driver);if (dev->bus && dev->bus->remove) dev->bus->remove (dev);else if (drv->remove)drivers/base/dd.c +0
-1@@ -767,7 +768,6 @@ static void __device_release_driver(struct device *dev) dev->pm_domain->dismiss (dev);pm_runtime_reinit (dev); -klist_remove (&dev->p->knode_driver);device_pm_check_callbacks (dev);if (dev->bus)blocking_notifier_call_chain(&dev->bus->p->bus_notifier,
Quoted 9 lines · Corey Minyard
Tomeu, you added that check in
[
989561de9b5112999475b406557d9c7e9e59c041] PM / Domains: add setter for dev.pm_domainand either something is wrong in the platform device handling or elsewhere in the device code, if
that check is valid.
FYI, I'm hitting the exact same error on shutdown on one of my boxes.
Please Cc me on updates.
-- Steve
[53591.087861] kvm: exiting hardware virtualization [53591.104798] ------------[ cut here ]------------ [53591.110058] WARNING: CPU: 0 PID: 1 at /home/rostedt/work/git/linux-trace.git/drivers/base/power/common.c:150 dev_pm_domain_set+0x82/0x90() [53591.123716] PM domains can only be changed for unbound devices [53591.130158] Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables xt_CHECKSUM iptable_mangle bridge stp llc bluetooth lockd grace snd_hda_codec_hdmi snd_hda_codec_r ealtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hwdep snd_hda_core vhost_net tun vhost x86_pkg_temp_thermal iTCO_wdt snd_seq snd_seq_device snd_pcm macvtap coretemp me i_me iTCO_vendor_support hp_wmi rfkill sparse_keymap macvlan kvm_intel snd_timer mei lpc_ich snd i2c_i801 soundcore mfd_core kvm irqbypass acpi_cpufreq serio_raw wmi uinput crc32_p clmul i915 crc32c_intel i2c_algo_bit e1000e drm_kms_helper ptp drm pps_core i2c_core video sunrpc [53591.190440] CPU: 0 PID: 1 Comm: systemd-shutdow Not tainted 4.5.0-rc1-test+ #155 [53591.198453] Hardware name: Hewlett-Packard HP Compaq Pro 6300 SFF/339A, BIOS K01 v02.05 05/07/2012 [53591.208031] ffff8801195abc90 ffff8801195abc90 ffffffff8143ef63 ffff8801195abcd8 [53591.216168] ffff8801195abcc8 ffffffff810acad6 ffff8801192d2328 0000000000000000 [53591.224299] 0000000000000001 ffff8801192d2388 00000000fee1dead ffff8801195abd28 [53591.232429] Call Trace: [53591.235483] [<ffffffff8143ef63>] dump_stack+0x44/0x61 [53591.241232] [<ffffffff810acad6>] warn_slowpath_common+0x86/0xc0 [53591.247842] [<ffffffff810acb5c>] warn_slowpath_fmt+0x4c/0x50 [53591.254193] [<ffffffff810acb15>] ? warn_slowpath_fmt+0x5/0x50 [53591.260625] [<ffffffff8157cf42>] dev_pm_domain_set+0x82/0x90 [53591.266970] [<ffffffffa03f255e>] mei_me_remove+0xee/0x120 [mei_me] [53591.273842] [<ffffffff8147fe96>] pci_device_shutdown+0x36/0x70 [53591.280373] [<ffffffff81570680>] device_shutdown+0xe0/0x1e0 [53591.286642] [<ffffffff810d6df6>] kernel_restart_prepare+0x36/0x40 [53591.293439] [<ffffffff810d6f62>] kernel_restart+0x12/0x60 [53591.299543] [<ffffffff810d72ae>] SYSC_reboot+0x1ce/0x1f0 [53591.305562] [<ffffffffa0008077>] ? 0xffffffffa0008077 [53591.311323] [<ffffffff811b2e60>] ? stack_trace_call+0x40/0x60 [53591.317780] [<ffffffffa0008077>] ? 0xffffffffa0008077 [53591.323552] [<ffffffff8143eeb4>] ? _atomic_dec_and_lock+0x44/0xaf [53591.330374] [<ffffffff812a302b>] ? iput+0xbb/0x2c0 [53591.335880] [<ffffffff810d7325>] ? SyS_reboot+0x5/0x10 [53591.341748] [<ffffffff81003044>] ? lockdep_sys_exit_thunk+0x12/0x14 [53591.348743] [<ffffffff810d732e>] SyS_reboot+0xe/0x10 [53591.354440] [<ffffffff8185ee32>] entry_SYSCALL_64_fastpath+0x12/0x76 [53591.361534] ---[ end trace 63b298fc6d5920e4 ]--- [53591.377714] e1000e: EEE TX LPI TIMER: 00000011 [53591.468245] sd 0:0:0:0: [sda] Synchronizing SCSI cache [53591.487321] reboot: Restarting system [53591.491679] reboot: machine restart
Quoted 49 lines · Steven Rostedt
Quoted 9 lines · Corey Minyard
Tomeu, you added that check in
[
989561de9b5112999475b406557d9c7e9e59c041] PM / Domains: add setter for dev.pm_domainand either something is wrong in the platform device handling or elsewhere in the device code, if
that check is valid.FYI, I'm hitting the exact same error on shutdown on one of my boxes.
Please Cc me on updates.
-- Steve
[53591.087861] kvm: exiting hardware virtualization [53591.104798] ------------[ cut here ]------------ [53591.110058] WARNING: CPU: 0 PID: 1 at /home/rostedt/work/git/linux-trace.git/drivers/base/power/common.c:150 dev_pm_domain_set+0x82/0x90() [53591.123716] PM domains can only be changed for unbound devices [53591.130158] Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables xt_CHECKSUM iptable_mangle bridge stp llc bluetooth lockd grace snd_hda_codec_hdmi snd_hda_codec_r ealtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hwdep snd_hda_core vhost_net tun vhost x86_pkg_temp_thermal iTCO_wdt snd_seq snd_seq_device snd_pcm macvtap coretemp me i_me iTCO_vendor_support hp_wmi rfkill sparse_keymap macvlan kvm_intel snd_timer mei lpc_ich snd i2c_i801 soundcore mfd_core kvm irqbypass acpi_cpufreq serio_raw wmi uinput crc32_p clmul i915 crc32c_intel i2c_algo_bit e1000e drm_kms_helper ptp drm pps_core i2c_core video sunrpc [53591.190440] CPU: 0 PID: 1 Comm: systemd-shutdow Not tainted 4.5.0-rc1-test+ #155 [53591.198453] Hardware name: Hewlett-Packard HP Compaq Pro 6300 SFF/339A, BIOS K01 v02.05 05/07/2012 [53591.208031] ffff8801195abc90 ffff8801195abc90 ffffffff8143ef63 ffff8801195abcd8 [53591.216168] ffff8801195abcc8 ffffffff810acad6 ffff8801192d2328 0000000000000000 [53591.224299] 0000000000000001 ffff8801192d2388 00000000fee1dead ffff8801195abd28 [53591.232429] Call Trace: [53591.235483] [<ffffffff8143ef63>] dump_stack+0x44/0x61 [53591.241232] [<ffffffff810acad6>] warn_slowpath_common+0x86/0xc0 [53591.247842] [<ffffffff810acb5c>] warn_slowpath_fmt+0x4c/0x50 [53591.254193] [<ffffffff810acb15>] ? warn_slowpath_fmt+0x5/0x50 [53591.260625] [<ffffffff8157cf42>] dev_pm_domain_set+0x82/0x90 [53591.266970] [<ffffffffa03f255e>] mei_me_remove+0xee/0x120 [mei_me] [53591.273842] [<ffffffff8147fe96>] pci_device_shutdown+0x36/0x70 [53591.280373] [<ffffffff81570680>] device_shutdown+0xe0/0x1e0 [53591.286642] [<ffffffff810d6df6>] kernel_restart_prepare+0x36/0x40 [53591.293439] [<ffffffff810d6f62>] kernel_restart+0x12/0x60 [53591.299543] [<ffffffff810d72ae>] SYSC_reboot+0x1ce/0x1f0 [53591.305562] [<ffffffffa0008077>] ? 0xffffffffa0008077 [53591.311323] [<ffffffff811b2e60>] ? stack_trace_call+0x40/0x60 [53591.317780] [<ffffffffa0008077>] ? 0xffffffffa0008077 [53591.323552] [<ffffffff8143eeb4>] ? _atomic_dec_and_lock+0x44/0xaf [53591.330374] [<ffffffff812a302b>] ? iput+0xbb/0x2c0 [53591.335880] [<ffffffff810d7325>] ? SyS_reboot+0x5/0x10
Quoted 9 lines
[53591.341748] [<ffffffff81003044>] ? lockdep_sys_exit_thunk+0x12/0x14 [53591.348743] [<ffffffff810d732e>] SyS_reboot+0xe/0x10 [53591.354440] [<ffffffff8185ee32>] entry_SYSCALL_64_fastpath+0x12/0x76 [53591.361534] ---[ end trace 63b298fc6d5920e4 ]--- [53591.377714] e1000e: EEE TX LPI TIMER: 00000011 [53591.468245] sd 0:0:0:0: [sda] Synchronizing SCSI cache [53591.487321] reboot: Restarting system [53591.491679] reboot: machine restart
Looks like Rafael adjusted for the platform shutdown case here:
https://lkml.org/lkml/2016/1/11/515
Perhaps that would applicable to the pci shutdown as well?
-- Joe
Quoted 60 lines · Joe Lawrence
Quoted 49 lines · Steven Rostedt
Quoted 9 lines · Corey Minyard
Tomeu, you added that check in
[
989561de9b5112999475b406557d9c7e9e59c041] PM / Domains: add setter for dev.pm_domainand either something is wrong in the platform device handling or elsewhere in the device code, if
that check is valid.FYI, I'm hitting the exact same error on shutdown on one of my boxes.
Please Cc me on updates.
-- Steve
[53591.087861] kvm: exiting hardware virtualization [53591.104798] ------------[ cut here ]------------ [53591.110058] WARNING: CPU: 0 PID: 1 at /home/rostedt/work/git/linux-trace.git/drivers/base/power/common.c:150 dev_pm_domain_set+0x82/0x90() [53591.123716] PM domains can only be changed for unbound devices [53591.130158] Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables xt_CHECKSUM iptable_mangle bridge stp llc bluetooth lockd grace snd_hda_codec_hdmi snd_hda_codec_r ealtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hwdep snd_hda_core vhost_net tun vhost x86_pkg_temp_thermal iTCO_wdt snd_seq snd_seq_device snd_pcm macvtap coretemp me i_me iTCO_vendor_support hp_wmi rfkill sparse_keymap macvlan kvm_intel snd_timer mei lpc_ich snd i2c_i801 soundcore mfd_core kvm irqbypass acpi_cpufreq serio_raw wmi uinput crc32_p clmul i915 crc32c_intel i2c_algo_bit e1000e drm_kms_helper ptp drm pps_core i2c_core video sunrpc [53591.190440] CPU: 0 PID: 1 Comm: systemd-shutdow Not tainted 4.5.0-rc1-test+ #155 [53591.198453] Hardware name: Hewlett-Packard HP Compaq Pro 6300 SFF/339A, BIOS K01 v02.05 05/07/2012 [53591.208031] ffff8801195abc90 ffff8801195abc90 ffffffff8143ef63 ffff8801195abcd8 [53591.216168] ffff8801195abcc8 ffffffff810acad6 ffff8801192d2328 0000000000000000 [53591.224299] 0000000000000001 ffff8801192d2388 00000000fee1dead ffff8801195abd28 [53591.232429] Call Trace: [53591.235483] [<ffffffff8143ef63>] dump_stack+0x44/0x61 [53591.241232] [<ffffffff810acad6>] warn_slowpath_common+0x86/0xc0 [53591.247842] [<ffffffff810acb5c>] warn_slowpath_fmt+0x4c/0x50 [53591.254193] [<ffffffff810acb15>] ? warn_slowpath_fmt+0x5/0x50 [53591.260625] [<ffffffff8157cf42>] dev_pm_domain_set+0x82/0x90 [53591.266970] [<ffffffffa03f255e>] mei_me_remove+0xee/0x120 [mei_me] [53591.273842] [<ffffffff8147fe96>] pci_device_shutdown+0x36/0x70 [53591.280373] [<ffffffff81570680>] device_shutdown+0xe0/0x1e0 [53591.286642] [<ffffffff810d6df6>] kernel_restart_prepare+0x36/0x40 [53591.293439] [<ffffffff810d6f62>] kernel_restart+0x12/0x60 [53591.299543] [<ffffffff810d72ae>] SYSC_reboot+0x1ce/0x1f0 [53591.305562] [<ffffffffa0008077>] ? 0xffffffffa0008077 [53591.311323] [<ffffffff811b2e60>] ? stack_trace_call+0x40/0x60 [53591.317780] [<ffffffffa0008077>] ? 0xffffffffa0008077 [53591.323552] [<ffffffff8143eeb4>] ? _atomic_dec_and_lock+0x44/0xaf [53591.330374] [<ffffffff812a302b>] ? iput+0xbb/0x2c0 [53591.335880] [<ffffffff810d7325>] ? SyS_reboot+0x5/0x10Quoted 9 lines
[53591.341748] [<ffffffff81003044>] ? lockdep_sys_exit_thunk+0x12/0x14 [53591.348743] [<ffffffff810d732e>] SyS_reboot+0xe/0x10 [53591.354440] [<ffffffff8185ee32>] entry_SYSCALL_64_fastpath+0x12/0x76 [53591.361534] ---[ end trace 63b298fc6d5920e4 ]--- [53591.377714] e1000e: EEE TX LPI TIMER: 00000011 [53591.468245] sd 0:0:0:0: [sda] Synchronizing SCSI cache [53591.487321] reboot: Restarting system [53591.491679] reboot: machine restart
Quoted 5 lines
Looks like Rafael adjusted for the platform shutdown case here:
https://lkml.org/lkml/2016/1/11/515
Perhaps that would applicable to the pci shutdown as well?
To the MEI driver's shutdown rather. mei_me_remove() clears the pm_domain pointer which is sort of questionable, but then the warning may be overkill for this case.
Thanks,
Rafael
Quoted 63 lines · Rafael J. Wysocki
Quoted 62 lines · Joe Lawrence
Quoted 49 lines · Steven Rostedt
Quoted 9 lines · Corey Minyard
Tomeu, you added that check in
[
989561de9b5112999475b406557d9c7e9e59c041] PM / Domains: add setter for dev.pm_domainand either something is wrong in the platform device handling or elsewhere in the device code, if
that check is valid.FYI, I'm hitting the exact same error on shutdown on one of my boxes.
Please Cc me on updates.
-- Steve
[53591.087861] kvm: exiting hardware virtualization [53591.104798] ------------[ cut here ]------------ [53591.110058] WARNING: CPU: 0 PID: 1 at /home/rostedt/work/git/linux-trace.git/drivers/base/power/common.c:150 dev_pm_domain_set+0x82/0x90() [53591.123716] PM domains can only be changed for unbound devices [53591.130158] Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables xt_CHECKSUM iptable_mangle bridge stp llc bluetooth lockd grace snd_hda_codec_hdmi snd_hda_codec_r ealtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hwdep snd_hda_core vhost_net tun vhost x86_pkg_temp_thermal iTCO_wdt snd_seq snd_seq_device snd_pcm macvtap coretemp me i_me iTCO_vendor_support hp_wmi rfkill sparse_keymap macvlan kvm_intel snd_timer mei lpc_ich snd i2c_i801 soundcore mfd_core kvm irqbypass acpi_cpufreq serio_raw wmi uinput crc32_p clmul i915 crc32c_intel i2c_algo_bit e1000e drm_kms_helper ptp drm pps_core i2c_core video sunrpc [53591.190440] CPU: 0 PID: 1 Comm: systemd-shutdow Not tainted 4.5.0-rc1-test+ #155 [53591.198453] Hardware name: Hewlett-Packard HP Compaq Pro 6300 SFF/339A, BIOS K01 v02.05 05/07/2012 [53591.208031] ffff8801195abc90 ffff8801195abc90 ffffffff8143ef63 ffff8801195abcd8 [53591.216168] ffff8801195abcc8 ffffffff810acad6 ffff8801192d2328 0000000000000000 [53591.224299] 0000000000000001 ffff8801192d2388 00000000fee1dead ffff8801195abd28 [53591.232429] Call Trace: [53591.235483] [<ffffffff8143ef63>] dump_stack+0x44/0x61 [53591.241232] [<ffffffff810acad6>] warn_slowpath_common+0x86/0xc0 [53591.247842] [<ffffffff810acb5c>] warn_slowpath_fmt+0x4c/0x50 [53591.254193] [<ffffffff810acb15>] ? warn_slowpath_fmt+0x5/0x50 [53591.260625] [<ffffffff8157cf42>] dev_pm_domain_set+0x82/0x90 [53591.266970] [<ffffffffa03f255e>] mei_me_remove+0xee/0x120 [mei_me] [53591.273842] [<ffffffff8147fe96>] pci_device_shutdown+0x36/0x70 [53591.280373] [<ffffffff81570680>] device_shutdown+0xe0/0x1e0 [53591.286642] [<ffffffff810d6df6>] kernel_restart_prepare+0x36/0x40 [53591.293439] [<ffffffff810d6f62>] kernel_restart+0x12/0x60 [53591.299543] [<ffffffff810d72ae>] SYSC_reboot+0x1ce/0x1f0 [53591.305562] [<ffffffffa0008077>] ? 0xffffffffa0008077 [53591.311323] [<ffffffff811b2e60>] ? stack_trace_call+0x40/0x60 [53591.317780] [<ffffffffa0008077>] ? 0xffffffffa0008077 [53591.323552] [<ffffffff8143eeb4>] ? _atomic_dec_and_lock+0x44/0xaf [53591.330374] [<ffffffff812a302b>] ? iput+0xbb/0x2c0 [53591.335880] [<ffffffff810d7325>] ? SyS_reboot+0x5/0x10Quoted 9 lines
[53591.341748] [<ffffffff81003044>] ? lockdep_sys_exit_thunk+0x12/0x14 [53591.348743] [<ffffffff810d732e>] SyS_reboot+0xe/0x10 [53591.354440] [<ffffffff8185ee32>] entry_SYSCALL_64_fastpath+0x12/0x76 [53591.361534] ---[ end trace 63b298fc6d5920e4 ]--- [53591.377714] e1000e: EEE TX LPI TIMER: 00000011 [53591.468245] sd 0:0:0:0: [sda] Synchronizing SCSI cache [53591.487321] reboot: Restarting system [53591.491679] reboot: machine restartLooks like Rafael adjusted for the platform shutdown case here:
Quoted 7 lines
Quoted 3 lines
https://lkml.org/lkml/2016/1/11/515
Perhaps that would applicable to the pci shutdown as well?
To the MEI driver's shutdown rather.
mei_me_remove()clears thepm_domainpointer which is sort of questionable, but then the warning may be overkill for this case.
Since MEI device is running its own power management, in some cases we are using PM domains just to avoid going through the PCI runtime pm handlers to avoid going to D3. IIRC we couldn't use PCI_DEV_FLAGS_NO_D3 as this not unique for runtime handlers I think this requirement is unique to the MEI device, so I'm not sure it's worth to push it into the PCI layer, maybe we just quite the warning somehow.
Tomas
Quoted 72 lines · Tomas Winkler
Quoted 63 lines · Rafael J. Wysocki
Quoted 62 lines · Joe Lawrence
Quoted 49 lines · Steven Rostedt
Quoted 9 lines · Corey Minyard
Tomeu, you added that check in
[
989561de9b5112999475b406557d9c7e9e59c041] PM / Domains: add setter for dev.pm_domainand either something is wrong in the platform device handling or elsewhere in the device code, if
that check is valid.FYI, I'm hitting the exact same error on shutdown on one of my boxes.
Please Cc me on updates.
-- Steve
[53591.087861] kvm: exiting hardware virtualization [53591.104798] ------------[ cut here ]------------ [53591.110058] WARNING: CPU: 0 PID: 1 at /home/rostedt/work/git/linux-trace.git/drivers/base/power/common.c:150 dev_pm_domain_set+0x82/0x90() [53591.123716] PM domains can only be changed for unbound devices [53591.130158] Modules linked in: ip6table_filter ip6_tables ebtable_nat ebtables xt_CHECKSUM iptable_mangle bridge stp llc bluetooth lockd grace snd_hda_codec_hdmi snd_hda_codec_r ealtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hwdep snd_hda_core vhost_net tun vhost x86_pkg_temp_thermal iTCO_wdt snd_seq snd_seq_device snd_pcm macvtap coretemp me i_me iTCO_vendor_support hp_wmi rfkill sparse_keymap macvlan kvm_intel snd_timer mei lpc_ich snd i2c_i801 soundcore mfd_core kvm irqbypass acpi_cpufreq serio_raw wmi uinput crc32_p clmul i915 crc32c_intel i2c_algo_bit e1000e drm_kms_helper ptp drm pps_core i2c_core video sunrpc [53591.190440] CPU: 0 PID: 1 Comm: systemd-shutdow Not tainted 4.5.0-rc1-test+ #155 [53591.198453] Hardware name: Hewlett-Packard HP Compaq Pro 6300 SFF/339A, BIOS K01 v02.05 05/07/2012 [53591.208031] ffff8801195abc90 ffff8801195abc90 ffffffff8143ef63 ffff8801195abcd8 [53591.216168] ffff8801195abcc8 ffffffff810acad6 ffff8801192d2328 0000000000000000 [53591.224299] 0000000000000001 ffff8801192d2388 00000000fee1dead ffff8801195abd28 [53591.232429] Call Trace: [53591.235483] [<ffffffff8143ef63>] dump_stack+0x44/0x61 [53591.241232] [<ffffffff810acad6>] warn_slowpath_common+0x86/0xc0 [53591.247842] [<ffffffff810acb5c>] warn_slowpath_fmt+0x4c/0x50 [53591.254193] [<ffffffff810acb15>] ? warn_slowpath_fmt+0x5/0x50 [53591.260625] [<ffffffff8157cf42>] dev_pm_domain_set+0x82/0x90 [53591.266970] [<ffffffffa03f255e>] mei_me_remove+0xee/0x120 [mei_me] [53591.273842] [<ffffffff8147fe96>] pci_device_shutdown+0x36/0x70 [53591.280373] [<ffffffff81570680>] device_shutdown+0xe0/0x1e0 [53591.286642] [<ffffffff810d6df6>] kernel_restart_prepare+0x36/0x40 [53591.293439] [<ffffffff810d6f62>] kernel_restart+0x12/0x60 [53591.299543] [<ffffffff810d72ae>] SYSC_reboot+0x1ce/0x1f0 [53591.305562] [<ffffffffa0008077>] ? 0xffffffffa0008077 [53591.311323] [<ffffffff811b2e60>] ? stack_trace_call+0x40/0x60 [53591.317780] [<ffffffffa0008077>] ? 0xffffffffa0008077 [53591.323552] [<ffffffff8143eeb4>] ? _atomic_dec_and_lock+0x44/0xaf [53591.330374] [<ffffffff812a302b>] ? iput+0xbb/0x2c0 [53591.335880] [<ffffffff810d7325>] ? SyS_reboot+0x5/0x10Quoted 9 lines
[53591.341748] [<ffffffff81003044>] ? lockdep_sys_exit_thunk+0x12/0x14 [53591.348743] [<ffffffff810d732e>] SyS_reboot+0xe/0x10 [53591.354440] [<ffffffff8185ee32>] entry_SYSCALL_64_fastpath+0x12/0x76 [53591.361534] ---[ end trace 63b298fc6d5920e4 ]--- [53591.377714] e1000e: EEE TX LPI TIMER: 00000011 [53591.468245] sd 0:0:0:0: [sda] Synchronizing SCSI cache [53591.487321] reboot: Restarting system [53591.491679] reboot: machine restartLooks like Rafael adjusted for the platform shutdown case here:
Quoted 7 lines
Quoted 3 lines
https://lkml.org/lkml/2016/1/11/515
Perhaps that would applicable to the pci shutdown as well?
To the MEI driver's shutdown rather.
mei_me_remove()clears thepm_domainpointer which is sort of questionable, but then the warning may be overkill for this case.
Quoted 7 lines
Since MEI device is running its own power management, in some cases we are using PM domains just to avoid going through the PCI runtime pm handlers to avoid going to D3. IIRC we couldn't use
PCI_DEV_FLAGS_NO_D3as this not unique for runtime handlers I think this requirement is unique to the MEI device, so I'm not sure it's worth to push it into the PCI layer, maybe we just quite the warning somehow.
Does this help: https://patchwork.kernel.org/patch/8170821/ ?
Thanks,
Rafael
Quoted 2 lines · Rafael J. Wysocki
Does this help: https://patchwork.kernel.org/patch/8170821/ ?
No more warnings on ipmi_si removal over here with that patch.
Thanks,
-- Joe
Steven Rostedt
Tested-byQuoted 1 line · On Wed, 03 Feb 2016 01:56:45 +0100 "Rafael J. Wysocki" <[email protected]>…
Does this help: https://patchwork.kernel.org/patch/8170821/ ?
Yep!
-- Steve