Hot Plug I/O Node spec 0.2.1
Quoted 7 lines · Hiroshi Aono
Hello hotplug developers,
I'm working on Hot Plug I/O Node work that is on Atlas project. I've been considering the specification of Hot Plug IO node and I've started the development about this.
Now I post Hot Plug I/O Node specification I want to do.
This isn't fixed yet. If you're interested, please feel free to add your ideas.
This looks good, a few minor points below.
Quoted 4 lines
3.1. Functions overview
This is the IO node hot plug function block diagram.
Quoted 9 lines
+---------------+ +-------------+ |Hotplug | +---------/Configuration/ |Application | | / File / +---------------+ | +-------------+ ---------|--------------|------------------------------------ +---------------+ +-------------------+ [kernel] | Hotplug | |Object Registration| | Filesystem | |Interface | User API +---------------+ +-------------------+
What is the "Object Redistration Interface" going to look like? What is needed to be sent from a configuration file to the IO Node resource manager? Doesn't the IO Node resource manager get all of the information it needs from the BIOS/ACPI interface?
Quoted 8 lines
3.3. Hotplug Filesystem
We'd like to use the hotplug filesystem. This is for PCI hotplug filesystem and we feel this is most suitable for the generic hotplug interface. The hotplug filesystem will be used for user interfaces. We will improve this to treat a hierarchical structure for describing IO node objects.
Quoted 7 lines
IO node hotplug hierarchical structure will be as follows:
--+ ionodeXX
+ bridgeXX
+ slotXX
+ slotXX
+ pic
"pic"? What is this?
Quoted 3 lines
3.4.2. Interface between IO node manager and PCI hotplug function
IO node hotplug uses the
/sbin/hotplugscript.
How? Doesn't a IO node just show up as a PCI device being added to the system?
Quoted 2 lines
When an IO node is added, a script for IO node will be executed and the PCI hotplug driver will be loaded.
Doesn't the PCI Hotplug driver have to be loaded to recognize that a IO node was inserted into the system?
Quoted 3 lines
Conversely, when an IO node is removed, a script for IO node will be executed and the PCI hotplug driver will stop all PCI devices under the IO node.
Will the removal event fire for every PCI slot, or just once for the node? If just once, I now understand why you need this IO node driver :)
Quoted 6 lines
3.4.3. Solutions for assignment of bus number, I/O address and interruption
Our 1st solution is to use the resources that the BIOS has allocated when booting. In this case, we expect that the BIOS supports the reservation of IO/Memory space gap and the bus numbers for hotplugging.
I am not aware of any other method of getting this information. I thought the BIOS _had_ to do this. Is there another way of doing this?
Quoted 33 lines
3.4.4. IRQ management
(We should discuss, later.)
3.4.5. Advanced solution
For more advanced solutions, we should consider active configuration of the IO node memory region and bus number region.
For example, when we want to expand region A because of insufficient IO memory area, we should do the following things:
o Narrow region B.
o Expand region A.MEMORY MAP MEMORY MAP
+-------------+ +-------------+ | | | | +-------------+-- +-------------+-- | |A (IO node 1) | |A (IO node 1) | | -> | | +-------------+-- | | | |B (IO node 2) +-------------+ | | | |B' (IO node 2) +-------------+-- +-------------+-- | | | |Figure 4 Advanced solution
--------------------------In addition, when some PCI cards work on IO node B, we should consider the following things:
Quoted 6 lines
o Suspend all of the devices on IO node 1
o Suspend the cards working on IO node 2
o Move card resources to region B'
o Re-program the configuration space for cards on IO node 2
o Resume the cards on IO node 2
o Resume all the devices on IO node 1
I can't see users tolerating their already working and configured devices shutting down, and then requiring to be reconfigured when you insert a new node. Is this an acceptable thing to you?
Quoted 48 lines
3.8. State and transition for IO node
This is the outline for the hot plug procedure:
1 Not Present
| (Insertion)
(1.1) FW: rises an interruption (SCI) (Note: FW: Firmware)
|
v
2 Present but not communicating
(2.1) OS: ACPI driver handles SCI interruption and
sends message "Inserted IO nodeXX" to the syslog. (for
debugging)
(2.2) FW: rises an interruption (SCI)
|
v
3 Communicating
(3.1) OS: ACPI driver handles SCI interruption and
sends message "Communicating" to the syslog. (for debugging)
|
v
4 Ready to Join
(4.1) FW: scans buses.
(4.2) FW: maps IO spaces.
(4.3) OS: scans buses and devices on IO node.
(4.4) OS: add IO node resources.
(4.5) OS: sets interrupt vectors.
(4.6) OS: call/sbin/hotplugwith notification of the new devices seen. |
v
5 Running
| (Removal)
(5.1) FW: calls an interruption (SCI) or user request
|
v
6 Prepare to disable
(6.1) OS: ACPI driver handles SCI interruption.
(6.2) OS: ACPI driver or node manager calls IO node event handler functions for pre-removal.
(6.3) OS: call remove method of port drivers on IO node and stops the IO
requests.
(6.4) OS: changes IO node state to be unavailable.
(6.5) OS: executes_PS3,_EJ0methods. (Power off and Eject) (6.6) OS: executes_STAto verify the node ejected successfully. |
v
7 Ready to Remove (Present but not communicating)
Quoted 8 lines
|(Physical removal)
(7.1) User: removes the IO node.
(7.1) FW: calls an interruption (SCI)
(7.2) OS: ACPI driver handles SCI interruption.
(7.3) OS: deletes the IO node resource.
|
v
Go to 1
IBM's BIOS does not work this way. Does this matter? On insertion, it only notifies the OS when the card has been powered up and configured (skipping steps 2 and 3 above). If an error occured when configuring (bus speed mismatch, etc.) this error is reported. Then the OS can assign physical resources, call /sbin/hotplug with the notification that a new device has been seen, and allows the driver to be loaded.
Most of the removal steps are not present too. The BIOS does most of it, and again, only notifies the OS when everything is done.
Because of this, a number of places where you will be notifing the OS user of something, the IBM driver can not. Do you see this being a problem?
thanks,
greg k-h
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
At Sun, 3 Feb 2002 00:19:15 -0800,
Greg KH wrote:
Hello Greg,
Quoted 10 lines · Hiroshi Aono
Quoted 7 lines
Hello hotplug developers,
I'm working on Hot Plug I/O Node work that is on Atlas project. I've been considering the specification of Hot Plug IO node and I've started the development about this.
Now I post Hot Plug I/O Node specification I want to do.
This isn't fixed yet. If you're interested, please feel free to add your ideas.This looks good, a few minor points below.
Thank you for your comments.
Quoted 15 lines
Quoted 5 lines
3.1. Functions overview
This is the IO node hot plug function block diagram.
Quoted 9 lines
+---------------+ +-------------+ |Hotplug | +---------/Configuration/ |Application | | / File / +---------------+ | +-------------+ ---------|--------------|------------------------------------ +---------------+ +-------------------+ [kernel] | Hotplug | |Object Registration| | Filesystem | |Interface | User API +---------------+ +-------------------+
Quoted 4 lines
What is the "Object Redistration Interface" going to look like? What is needed to be sent from a configuration file to the IO Node resource manager? Doesn't the IO Node resource manager get all of the information it needs from the BIOS/ACPI interface?
I expect that this mainly works for debugging.
I think the hardware like this may not work sufficiently at first. or it may be used for non-ACPI machine.
This configuration file will be stored hardware information. Of course IO Node resource manager should get all information from BIOS/ACPI.
I think the IO node driver and the Object Registration interface between IO node resource manager should have same interface.
Quoted 11 lines
Quoted 11 lines
3.3. Hotplug Filesystem
We'd like to use the hotplug filesystem. This is for PCI hotplug filesystem and we feel this is most suitable for the generic hotplug interface. The hotplug filesystem will be used for user interfaces. We will improve this to treat a hierarchical structure for describing IO node objects.
IO node hotplug hierarchical structure will be as follows:
Quoted 7 lines
Quoted 5 lines
--+ ionodeXX
+ bridgeXX
+ slotXX
+ slotXX
+ pic"pic"? What is this?
"pic" means programmable interrupt controller like an IOSAPIC.
Quoted 7 lines
Quoted 4 lines
3.4.2. Interface between IO node manager and PCI hotplug function
IO node hotplug uses the
/sbin/hotplugscript.How? Doesn't a IO node just show up as a PCI device being added to the system?
Our IO node doesn't show up as a PCI device.
Does IBM work so?
I can know the hardware structure through ACPI name space in advance.
Quoted 6 lines
Quoted 3 lines
When an IO node is added, a script for IO node will be executed and the PCI hotplug driver will be loaded.
Doesn't the PCI Hotplug driver have to be loaded to recognize that a IO node was inserted into the system?
If so, I think PCI hotplug driver should be loaded in advance.
Quoted 7 lines
Quoted 4 lines
Conversely, when an IO node is removed, a script for IO node will be executed and the PCI hotplug driver will stop all PCI devices under the IO node.
Will the removal event fire for every PCI slot, or just once for the node? If just once, I now understand why you need this IO node driver :)
Yes, I think it's just once.
Quoted 10 lines
Quoted 7 lines
3.4.3. Solutions for assignment of bus number, I/O address and interruption
Our 1st solution is to use the resources that the BIOS has allocated when booting. In this case, we expect that the BIOS supports the reservation of IO/Memory space gap and the bus numbers for hotplugging.
I am not aware of any other method of getting this information. I thought the BIOS _had_ to do this. Is there another way of doing this?
This means we can specify the IO/Memory space gap and bus numbers on a BIOS menu.
I think this is same rule for PCI hotplug. Is this right?
Quoted 41 lines
Quoted 34 lines
3.4.4. IRQ management
(We should discuss, later.)
3.4.5. Advanced solution
For more advanced solutions, we should consider active configuration of the IO node memory region and bus number region.
For example, when we want to expand region A because of insufficient IO memory area, we should do the following things:
o Narrow region B.
o Expand region A.MEMORY MAP MEMORY MAP
+-------------+ +-------------+ | | | | +-------------+-- +-------------+-- | |A (IO node 1) | |A (IO node 1) | | -> | | +-------------+-- | | | |B (IO node 2) +-------------+ | | | |B' (IO node 2) +-------------+-- +-------------+-- | | | |Figure 4 Advanced solution
--------------------------In addition, when some PCI cards work on IO node B, we should consider the following things:
Quoted 6 lines
o Suspend all of the devices on IO node 1
o Suspend the cards working on IO node 2
o Move card resources to region B'
o Re-program the configuration space for cards on IO node 2
o Resume the cards on IO node 2
o Resume all the devices on IO node 1
Quoted 3 lines
I can't see users tolerating their already working and configured devices shutting down, and then requiring to be reconfigured when you insert a new node. Is this an acceptable thing to you?
As a matter of fact, I think this is not realistic now. So I say this is "Advanced". At first, this is difficult for hardware implementation.
Quoted 58 lines
Quoted 49 lines
3.8. State and transition for IO node
This is the outline for the hot plug procedure:
1 Not Present
| (Insertion)
(1.1) FW: rises an interruption (SCI) (Note: FW: Firmware)
|
v
2 Present but not communicating
(2.1) OS: ACPI driver handles SCI interruption and
sends message "Inserted IO nodeXX" to the syslog. (for
debugging)
(2.2) FW: rises an interruption (SCI)
|
v
3 Communicating
(3.1) OS: ACPI driver handles SCI interruption and
sends message "Communicating" to the syslog. (for debugging)
|
v
4 Ready to Join
(4.1) FW: scans buses.
(4.2) FW: maps IO spaces.
(4.3) OS: scans buses and devices on IO node.
(4.4) OS: add IO node resources.
(4.5) OS: sets interrupt vectors.
(4.6) OS: call/sbin/hotplugwith notification of the new devices seen. |
v
5 Running
| (Removal)
(5.1) FW: calls an interruption (SCI) or user request
|
v
6 Prepare to disable
(6.1) OS: ACPI driver handles SCI interruption.
(6.2) OS: ACPI driver or node manager calls IO node event handler functions for pre-removal.
(6.3) OS: call remove method of port drivers on IO node and stops the IO
requests.
(6.4) OS: changes IO node state to be unavailable.
(6.5) OS: executes_PS3,_EJ0methods. (Power off and Eject) (6.6) OS: executes_STAto verify the node ejected successfully. |
v
7 Ready to Remove (Present but not communicating)Quoted 8 lines
|(Physical removal)
(7.1) User: removes the IO node.
(7.1) FW: calls an interruption (SCI)
(7.2) OS: ACPI driver handles SCI interruption.
(7.3) OS: deletes the IO node resource.
|
v
Go to 1
Quoted 6 lines
IBM's BIOS does not work this way. Does this matter? On insertion, it only notifies the OS when the card has been powered up and configured (skipping steps 2 and 3 above). If an error occured when configuring (bus speed mismatch, etc.) this error is reported. Then the OS can assign physical resources, call
/sbin/hotplugwith the notification that a new device has been seen, and allows the driver to be loaded.
Actually, hardware spec doesn't fix yet, so we can suggest this for our hardware people.
Personally, I feel that we don't need steps 2 and 3.
Quoted 7 lines
Most of the removal steps are not present too. The BIOS does most of it, and again, only notifies the OS when everything is done.
Because of this, a number of places where you will be notifing the OS user of something, the IBM driver can not. Do you see this being a problem?
I feel those hardware dependency part should be implement as a local IO node driver.
I think it will be ok if we decide the interface definitely between IO node driver and IO node resource manager.
Regards,
---
Hiroshi Aono, NEC Solutions
([email protected])
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel