pci logical vs. physical hotplug
* Kenji Kaneshige <[email protected]>:
Quoted 4 lines
I confirmed this patch fix the kernel oops problem I reported.
Reviewed-by Kenji Kaneshige <[email protected]>Tested-by Kenji Kaneshige <[email protected]>
Thank you for reviewing and testing.
Quoted 7 lines
By the way, /sys/bus/pci/slots/<slot> directory by acpiphp are remaining even after the parent bridge/bus of the slots are removed. At this point, acpiphp is working with struct
pci_busfor the already disabled pci bus. I guess some operation against the files under /sys/bus/pci/slots/<slot> directory would cause something problems. So I think we also need something mechanism to unregister acpiphp slots when the parent bus is removed.
Yes, I've been thinking about this (and thank you for your other mail confirming the issue).
The logical hotplug and physical hotplug don't play very nicely with each other.
I think one of the core issues is that logical hotplug allows function level granularity while physical hotplug is naturally restricted to physical slot granularity, which includes an entire hierarchy, from host bus down to function.
If a user uses logical hotplug to take out a piece of the tree, what does that mean if it's part of the physical slot/device?
What should happen?
Take something like this:
[0000:2e-4f]----00.0-[0000:2f-4f]--+-02.0-[0000:30-3f]--+-00.0 Intel GigE
| \-00.1 Intel GigE
\-04.0-[0000:40-4f]--+-00.0 Intel GigE
\-00.1 Intel GigEAssume that this is a quad-port NIC with a bridge in it, and that the physical slot is 0000:2e:00.0.
What should we do if the user does a logical hotplug on 0000:2f:04.0 and has acpiphp loaded?
If acpiphp tries to do anything to 0000:40: we'll probably get an oops.
But just because the user took out one piece of that tree doesn't mean that we should disable the entire slot. If that was the case, then he could just use the existing hotplug drivers.
I don't have a good answer for right now, other than, "don't try to mix logical and physical hotplug". I'm open to any ideas that you may have.
Thanks.
/ac
Quoted 16 lines · Alex Chiang
* Kenji Kaneshige <[email protected]>:
Quoted 4 lines
I confirmed this patch fix the kernel oops problem I reported.
Reviewed-by Kenji Kaneshige <[email protected]>Tested-by Kenji Kaneshige <[email protected]>Thank you for reviewing and testing.
Quoted 7 lines
By the way, /sys/bus/pci/slots/<slot> directory by acpiphp are remaining even after the parent bridge/bus of the slots are removed. At this point, acpiphp is working with struct
pci_busfor the already disabled pci bus. I guess some operation against the files under /sys/bus/pci/slots/<slot> directory would cause something problems. So I think we also need something mechanism to unregister acpiphp slots when the parent bus is removed.
Quoted 3 lines
Yes, I've been thinking about this (and thank you for your other mail confirming the issue).
The pciehp (and probably also shpchp) doesn't have this problem because /sys/bus/pci/slots/<slot> directories are unregistered when the parent bridge of those slots is removed. This is done with PCI driver model. The acpiphp and pci_slot driver doesn't match the PCI driver model, so I think those drivers need something other notification mechanism of PCI device removal. I guess this kind of notification mechanism might already exist in the kernel to bind ACPI device and PCI device, though it's still just my guess.
Please note that I'm not saying that pciehp doesn't have any problem about the interaction with logical hotplug mechanism. I think there are still remaining issues, and we need to solve them one by one.
Quoted 29 lines
The logical hotplug and physical hotplug don't play very nicely with each other.
I think one of the core issues is that logical hotplug allows function level granularity while physical hotplug is naturally restricted to physical slot granularity, which includes an entire hierarchy, from host bus down to function.
If a user uses logical hotplug to take out a piece of the tree, what does that mean if it's part of the physical slot/device?
What should happen?
Take something like this:
[0000:2e-4f]----00.0-[0000:2f-4f]--+-02.0-[0000:30-3f]--+-00.0 Intel GigE | \-00.1 Intel GigE \-04.0-[0000:40-4f]--+-00.0 Intel GigE \-00.1 Intel GigEAssume that this is a quad-port NIC with a bridge in it, and that the physical slot is 0000:2e:00.0.
What should we do if the user does a logical hotplug on 0000:2f:04.0 and has acpiphp loaded?
If acpiphp tries to do anything to 0000:40: we'll probably get an oops.
Quoted 8 lines
But just because the user took out one piece of that tree doesn't mean that we should disable the entire slot. If that was the case, then he could just use the existing hotplug drivers.
I don't have a good answer for right now, other than, "don't try to mix logical and physical hotplug". I'm open to any ideas that you may have.
Physical hotplug is:
physical hot-add = physical power on + logical hot-add; physical hot-removal = logical hot-removal + physical power off;
In the above case, some pieces of logical hot-removal has already been done. So I think what acpiphp need to do is remaining pieces of logical hot-removal of tree and power off.
Thanks,
Kenji Kaneshige