One chart on pci bridge and its bus and their children

DISCUSSION Richard Yang 6 participants 19 messages Jun 5, 2012 – Jun 6, 2012 PCI

lore

Share & tools

lore

Richard Yang

author

2012-06-05 03:26 UTC #0 lore raw

All,

I draw a chart on the relationship of bridge/bus/devics.

I am not sure it is exact, so please correct me if it is not correct.

BTW, the pci_bus->self, pci_bus->bridge seem to point the same thing. So this is really necessary to do this?

                            Bus 0(pci_bus)                      
                            +--------------------+<----+
                            |                    |     |
                            +--------------------+     |
                                                       |
                                                       |
                             Bridge(pci_dev)           |                     
                            +--------------------+<----|---+--------------+
                            |subordinate(pci_bus)|-+   |   |              |
                            +--------------------+ |   |   |              |
                                                   |   |   |              |
                                                   |   |   |              |
                                                   |   |   |              |
                            Bus 1(pci_bus)         |   |   |              |
                       +--->+-------------------+<-+   |   |              |
                       |    |parent(pci_bus)    |------+   |              |
                       |    +-------------------+          |              |
                       |    |self(pci_dev)      |----------+              |
                       |    |bridge(device)     |                         |
                       |    +-------------------+                         |
                       |    |devices(list_head) |                         |
                       |    +--------+----------+                         |
                       |             |                                    |
                       |      ----+--+---+----                            |
   pci_dev             |          |      |          pci_dev               |
   +---------------+<--|----------+      +--------->+-----------------+   |
   |bus(pci_bus)   |---+                            |bus(pci_bus)     |   |
   +---------------+                                +-----------------+   |
   |dev.parent     |                                |dev.parent       |---+
   +---------------+                                +-----------------+

Signature · 2 lines

Richard Yang
Help you, Help me

Ram Pai

2012-06-05 04:13 UTC #1 lore raw

Quoted 6 lines · Richard Yang

All,
I draw a chart on the relationship of bridge/bus/devics.
I am not sure it is exact, so please correct me if it is not correct.

A clearer representation can be found at http://tldp.org/LDP/tlk/dd/pci.html
in section 6.6.1

Quoted 3 lines

BTW, the pci_bus->self, pci_bus->bridge seem to point the same thing. So this is really necessary to do this?

pci_bus->self points to the pci_dev structure, and pci_bus->bridge points to the device structure. Both correspond to the same bridge. BTW: device structure is the basic structure to represent any device in the system, not necessarily pci device.

Quoted 31 lines

                            Bus 0(pci_bus)                      
                            +--------------------+<----+
                            |                    |     |
                            +--------------------+     |
                                                       |
                                                       |
                             Bridge(pci_dev)           |                     
                            +--------------------+<----|---+--------------+
                            |subordinate(pci_bus)|-+   |   |              |
                            +--------------------+ |   |   |              |
                                                   |   |   |              |
                                                   |   |   |              |
                                                   |   |   |              |
                            Bus 1(pci_bus)         |   |   |              |
                       +--->+-------------------+<-+   |   |              |
                       |    |parent(pci_bus)    |------+   |              |
                       |    +-------------------+          |              |
                       |    |self(pci_dev)      |----------+              |
                       |    |bridge(device)     |                         |
                       |    +-------------------+                         |
                       |    |devices(list_head) |                         |
                       |    +--------+----------+                         |
                       |             |                                    |
                       |      ----+--+---+----                            |
   pci_dev             |          |      |          pci_dev               |
   +---------------+<--|----------+      +--------->+-----------------+   |
   |bus(pci_bus)   |---+                            |bus(pci_bus)     |   |
   +---------------+                                +-----------------+   |
   |dev.parent     |                                |dev.parent       |---+
   +---------------+                                +-----------------+

Bjorn Helgaas

maintainer

2012-06-05 04:38 UTC #2 lore raw

Quoted 9 lines · Ram Pai

Quoted 2 lines · Richard Yang
BTW, the pci_bus->self, pci_bus->bridge seem to point the same thing. So this is really necessary to do this?
pci_bus->self points to the pci_dev structure, and pci_bus->bridge points to the device structure. Both correspond to the same bridge. BTW: device structure is the basic structure to represent any device in the system, not necessarily pci device.

That's true, but I don't think it answers the question. If we have:

    pci_bus->bridge == pci_bus->self->dev

why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".

Yinghai Lu

2012-06-05 05:18 UTC #3 lore raw

Quoted 6 lines · Bjorn Helgaas

That's true, but I don't think it answers the question. If we have:
pci_bus->bridge == pci_bus->self->dev
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".

then how about root bus?

root bus ->self should be NULL.

root bus ->bridge is to the hostbridge->dev.

Yinghai

Yinghai Lu

2012-06-05 05:20 UTC #4 lore raw

Quoted 5 lines · Yinghai Lu

Quoted 4 lines · Bjorn Helgaas
That's true, but I don't think it answers the question. If we have:
pci_bus->bridge == pci_bus->self->dev

Quoted 8 lines

Quoted 2 lines
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".
then how about root bus?
root bus ->self should be NULL.
root bus ->bridge is to the hostbridge->dev.

also

b->bridge = get_device(&bridge->dev);

so there is one reference held by pci bus.

Richard Yang

author

2012-06-05 05:41 UTC #5 lore raw

Quoted 13 lines · Yinghai Lu

Quoted 12 lines · Yinghai Lu
Quoted 6 lines · Bjorn Helgaas
That's true, but I don't think it answers the question. If we have:
pci_bus->bridge == pci_bus->self->dev
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".
then how about root bus?
root bus ->self should be NULL.

Quoted 7 lines

Quoted 1 line
root bus ->bridge is to the hostbridge->dev.
also
b->bridge = get_device(&bridge->dev);
so there is one reference held by pci bus.

Hmm... so can we just have b->bridge, and then convert bridge to self if necessary?

Signature · 2 lines

Richard Yang
Help you, Help me

Ram Pai

2012-06-05 06:35 UTC #6 lore raw

In reply to Yinghai Lu

Quoted 6 lines · Yinghai Lu

Quoted 5 lines · Yinghai Lu
Quoted 4 lines · Bjorn Helgaas
That's true, but I don't think it answers the question. If we have:
pci_bus->bridge == pci_bus->self->dev

Quoted 8 lines

Quoted 8 lines
Quoted 2 lines
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".
then how about root bus?
root bus ->self should be NULL.
root bus ->bridge is to the hostbridge->dev.

Yinghai,

is there a reason why this can't be fixed in
pci_create_root_bus() by having
bus->self = bridge; /* 'bridge' is the pci_device of the corresponding host bridge */ get_device(&bridge->dev);

Jiang Liu

2012-06-05 14:41 UTC #7 lore raw

Quoted 14 lines · Ram Pai

Quoted 13 lines · Yinghai Lu
Quoted 12 lines · Yinghai Lu
Quoted 6 lines · Bjorn Helgaas
That's true, but I don't think it answers the question. If we have:
   pci_bus->bridge == pci_bus->self->dev
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".
then how about root bus?
root bus ->self should be NULL.

Quoted 7 lines

Quoted 1 line
Quoted 1 line
root bus ->bridge is to the hostbridge->dev.
Yinghai,
is there a reason why this can't be fixed in
pci_create_root_bus() by having
bus->self = bridge; /* 'bridge' is the pci_device of the corresponding host bridge */

Hi Ram,
A host bridge has no associated pci_device because it's out of the
PCI domain. PCI domain only covers the hierarchy starting from the host bridge, but doesn't include the host bridge itself.
Thanks!

Quoted 9 lines

   get_device(&bridge->dev);
RP
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Ram Pai

2012-06-05 16:43 UTC #8 lore raw

Quoted 19 lines · Jiang Liu

Quoted 18 lines · Ram Pai
Quoted 6 lines · Yinghai Lu
Quoted 5 lines · Yinghai Lu
Quoted 4 lines · Bjorn Helgaas
That's true, but I don't think it answers the question. If we have:
   pci_bus->bridge == pci_bus->self->dev
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".
then how about root bus?
root bus ->self should be NULL.
root bus ->bridge is to the hostbridge->dev.
Yinghai,

Quoted 7 lines

Quoted 3 lines
is there a reason why this can't be fixed in
pci_create_root_bus() by having
bus->self = bridge; /* 'bridge' is the pci_device of the corresponding host bridge */
Hi Ram,
A host bridge has no associated pci_device because it's out of the
PCI domain. PCI domain only covers the hierarchy starting from the host bridge, but doesn't include the host bridge itself.

That makes it even easier to kill pci_bus->bridge.

for non-root buses, we can access the bridge's device using bus->self->dev and for root bridges we can silently ignore the bridge's dev since we know that it does not exist; because pci_bus->self is null.

No?
RP

Ram Pai

2012-06-06 01:58 UTC #9 lore raw

Quoted 28 lines · Ram Pai

Quoted 19 lines · Jiang Liu
Quoted 18 lines · Ram Pai
Quoted 6 lines · Yinghai Lu
Quoted 5 lines · Yinghai Lu
Quoted 4 lines · Bjorn Helgaas
That's true, but I don't think it answers the question. If we have:
   pci_bus->bridge == pci_bus->self->dev
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".
then how about root bus?
root bus ->self should be NULL.
root bus ->bridge is to the hostbridge->dev.
Yinghai,
Quoted 7 lines
Quoted 3 lines
is there a reason why this can't be fixed in
pci_create_root_bus() by having
bus->self = bridge; /* 'bridge' is the pci_device of the corresponding host bridge */
Hi Ram,
A host bridge has no associated pci_device because it's out of the
PCI domain. PCI domain only covers the hierarchy starting from the host bridge, but doesn't include the host bridge itself.

Quoted 6 lines

That makes it even easier to kill pci_bus->bridge.
for non-root buses, we can access the bridge's device using bus->self->dev and for root bridges we can silently ignore the bridge's dev since we know that it does not exist; because pci_bus->self is null.

hmm..ok. I realize the struct device of the host bridge exists and needs to be accessed, even though it is not a pci_device. So killing bus->bridge is not trivial.

Yinghai Lu

2012-06-05 19:38 UTC #10 lore raw

In reply to Ram Pai

Quoted 14 lines · Ram Pai

Quoted 13 lines · Yinghai Lu
Quoted 12 lines · Yinghai Lu
Quoted 6 lines · Bjorn Helgaas
That's true, but I don't think it answers the question. If we have:
pci_bus->bridge == pci_bus->self->dev
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".
then how about root bus?
root bus ->self should be NULL.

Quoted 8 lines

Quoted 1 line
Quoted 1 line
root bus ->bridge is to the hostbridge->dev.
Yinghai,
is there a reason why this can't be fixed in
pci_create_root_bus() by having
bus->self = bridge; /* 'bridge' is the pci_device of the corresponding host bridge */ get_device(&bridge->dev);

self is pointer to pci_dev bridge.

and for root bus, it is NULL. that mean no pci bridge device related.

bridge is device, and have that assigned from host bridge dev, or pci bridge dev.

also for virtfn of iov, new bus from there will not have self assigned too. for it even bridge is not assigned.

so bridge is just generic way to save bridge (pci, host) device ...

Richard Yang

author

2012-06-06 06:18 UTC #11 lore raw

Quoted 12 lines · Yinghai Lu

Quoted 7 lines
Yinghai,
is there a reason why this can't be fixed in
pci_create_root_bus() by having
bus->self = bridge; /* 'bridge' is the pci_device of the corresponding host bridge */ get_device(&bridge->dev);
self is pointer to pci_dev bridge.
and for root bus, it is NULL. that mean no pci bridge device related.

Quoted 7 lines

bridge is device, and have that assigned from host bridge dev, or pci bridge dev.
also for virtfn of iov, new bus from there will not have self assigned too. for it even bridge is not assigned.
so bridge is just generic way to save bridge (pci, host) device ...

I did a summarize, we have three types of pci_bus.

Can we identify the type by following method?

+--------------------+--------------------------------------------------+
| bus type           |   identification                                 |
+--------------------+--------------------------------------------------+
| root_bus           |parent is NULL, bridge point to host_bridge dev   |
+--------------------+--------------------------------------------------+
| general_bus        |parent to parent bus, bridge point to the bridge  |
+--------------------+--------------------------------------------------+
| iov_bus            |parent to parent bus, bridge point to NULL        |
+--------------------+--------------------------------------------------+

Signature · 2 lines

Richard Yang
Help you, Help me

Richard Yang

author

2012-06-06 07:58 UTC #12 lore raw

Quoted 13 lines · Richard Yang

Quoted 12 lines · Yinghai Lu
Quoted 7 lines
Yinghai,
is there a reason why this can't be fixed in
pci_create_root_bus() by having
bus->self = bridge; /* 'bridge' is the pci_device of the corresponding host bridge */ get_device(&bridge->dev);
self is pointer to pci_dev bridge.
and for root bus, it is NULL. that mean no pci bridge device related.

Quoted 8 lines

Quoted 7 lines
bridge is device, and have that assigned from host bridge dev, or pci bridge dev.
also for virtfn of iov, new bus from there will not have self assigned too. for it even bridge is not assigned.
so bridge is just generic way to save bridge (pci, host) device ...

I did a summarize, we have three types of pci_bus.

Can we identify the type by following method?

+--------------------+--------------------------------------------------+
| bus type           |   identification                                 |
+--------------------+--------------------------------------------------+
| root_bus           |parent is NULL, bridge point to host_bridge dev   |
+--------------------+--------------------------------------------------+
| general_bus        |parent to parent bus, bridge point to the bridge  |
+--------------------+--------------------------------------------------+
| iov_bus            |parent to parent bus, bridge point to NULL        |
+--------------------+--------------------------------------------------+

Hmm... if we can determine which type a pci_bus is from this criteria. Sounds we don't need the self in pci_bus?

Signature · 2 lines

Richard Yang
Help you, Help me

Gavin Shan

2012-06-05 21:52 UTC #13 lore raw

In reply to Yinghai Lu

Hi YingHai,

Quoted 4 lines

Quoted 4 lines
Quoted 4 lines
That's true, but I don't think it answers the question. If we have:
pci_bus->bridge == pci_bus->self->dev

Quoted 8 lines

Quoted 8 lines
Quoted 2 lines
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".
then how about root bus?
root bus ->self should be NULL.
root bus ->bridge is to the hostbridge->dev.

As I understood, the current implementation is to trace the PCI host bridge through the PCI root bus. The problem would be avoided if you change it for a little bit to trace the PCI host bridge through (PCI domain number), which will simplify the current implementation in 3.5.RC1 as well even though not much. For example: When looking for the PCI host bridge for the child bus, we only need get the PCI domain number, then get the corresponding PCI host bridge. We needn't go through the whole PCI bus tree from the bottom to the top until the PCI root bus.

I'm not sure that will introduce performance improvement or not.

Quoted 7 lines

also
b->bridge = get_device(&bridge->dev);
so there is one reference held by pci bus.

If we ignore the return value of get_device() here, that would be possible to merge "bridge"/"self" for child PCI bus. However, we still need differentiate the PCI root bus and child bus because the "bridge"/"self" of PCI root bus would be "NULL", but "bridge"/"self" (might be merged to "bridge") would point pci_dev of the P2P bridge.

Thanks,
Gavin

Richard Yang

author

2012-06-05 06:41 UTC #14 lore raw

In reply to Yinghai Lu

Quoted 8 lines · Yinghai Lu

Quoted 6 lines · Bjorn Helgaas
That's true, but I don't think it answers the question. If we have:
pci_bus->bridge == pci_bus->self->dev
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".

Quoted 7 lines

then how about root bus?
root bus ->self should be NULL.
root bus ->bridge is to the hostbridge->dev.
Yinghai

Agree, root_pci_bus->self is NULL, and root_pci_bus->bridge point to a solo device structure, not in hostbridge.

BTW, if the dev is the hostbrige->dev, why not pci_host_bridge contain a field "dev"?

And, why root bus differs with other bus a lot?
Both root bus and other bus use self/bridge point to the "PCI BRIDGE" device.
I think we can determine whether this is a root bus by bus->parent?

Signature · 2 lines

Richard Yang
Help you, Help me

Jiang Liu

2012-06-05 14:32 UTC #15 lore raw

Quoted 20 lines · Richard Yang

Quoted 8 lines · Yinghai Lu
Quoted 6 lines · Bjorn Helgaas
That's true, but I don't think it answers the question. If we have:
   pci_bus->bridge == pci_bus->self->dev
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".
Quoted 7 lines
then how about root bus?
root bus ->self should be NULL.
root bus ->bridge is to the hostbridge->dev.
Yinghai
Agree, root_pci_bus->self is NULL, and root_pci_bus->bridge point to a solo device structure, not in hostbridge.

Quoted 7 lines

BTW, if the dev is the hostbrige->dev, why not pci_host_bridge contain a field "dev"?
And, why root bus differs with other bus a lot?
Both root bus and other bus use self/bridge point to the "PCI BRIDGE" device.
I think we can determine whether this is a root bus by bus->parent?

Hi Richard,

There's fundamental difference between root bridge and pci bridge.

A PCI bridge is a PCI to PCI bridge. It's presented as a PCI device on the primary bus and hosts the secondary bus. So the secondary bus is associated with the PCI device on the primary bus through pci_bus->self and pci_bus->device.

A root bridge is a system bus to PCI bus bridge, which connects PCI domain to system bus domain. The primary side is the system bus and the secondary side is a PCI bus, so there's a PCI bus associated with a root bridge, but there's no PCI device for the root bridge because it's primary side is system bus, out of the PCI domain.

pci_bus->bridge is a common abstraction of the device associated with a PCI bus, no matter it's a host bridge or P2P bridge. So we don't need to distinguish between host bridge and pci bridge when we want to hold a reference to the device associated with a PCI bus.

Thanks!
Gerry

Jiang Liu

2012-06-05 14:37 UTC #16 lore raw

In reply to Richard Yang

Quoted 15 lines · Richard Yang

Quoted 14 lines · Yinghai Lu
Quoted 6 lines · Bjorn Helgaas
That's true, but I don't think it answers the question. If we have:
   pci_bus->bridge == pci_bus->self->dev
why would we need both "self" and "bridge"? It would be interesting to try to remove "bridge" and replace uses of it with "self->dev".
then how about root bus?
root bus ->self should be NULL.
root bus ->bridge is to the hostbridge->dev.

Quoted 7 lines

Quoted 1 line
Yinghai
Agree, root_pci_bus->self is NULL, and root_pci_bus->bridge point to a solo device structure, not in hostbridge.
BTW, if the dev is the hostbrige->dev, why not pci_host_bridge contain a field "dev"?

With latest code from Yinghai, the host bridge is defined as:

struct pci_host_bridge {        
        struct device dev;
        struct pci_bus *bus;            /* root bus */
        struct list_head windows;       /* pci_host_bridge_windows */
        void (*release_fn)(struct pci_host_bridge *);
        void *release_data;
};

So I think it's already working in the way suggested by you.

Quoted 6 lines

And, why root bus differs with other bus a lot?
Both root bus and other bus use self/bridge point to the "PCI BRIDGE" device.
I think we can determine whether this is a root bus by bus->parent?

Yinghai Lu

2012-06-05 19:41 UTC #17 lore raw

In reply to Richard Yang

Quoted 4 lines · Richard Yang

On Mon, Jun 04, 2012 at 10:18:46PM -0700, Yinghai Lu wrote:
BTW, if the dev is the hostbrige->dev, why not pci_host_bridge contain a field "dev"?

we have that from 3.5-rc1

Quoted 5 lines

And, why root bus differs with other bus a lot?
Both root bus and other bus use self/bridge point to the "PCI BRIDGE" device.
I think we can determine whether this is a root bus by bus->parent?

self is for real pci dev bridge.
bridge is the for pci_dev bridge generic device pointer. or host bridge device pointer.

Yinghai

Richard Yang

author

2012-06-05 06:00 UTC #18 lore raw

In reply to Ram Pai

Quoted 10 lines · Ram Pai

Quoted 6 lines · Richard Yang
All,
I draw a chart on the relationship of bridge/bus/devics.
I am not sure it is exact, so please correct me if it is not correct.
A clearer representation can be found at http://tldp.org/LDP/tlk/dd/pci.html
in section 6.6.1

Thanks, Ram.

I found I missed one important field in pci_bus.

Quoted 42 lines

Quoted 4 lines

BTW, the pci_bus->self, pci_bus->bridge seem to point the same thing. So this is really necessary to do this?

Quoted 31 lines

                            Bus 0(pci_bus)                      
                            +--------------------+<----+
                            |                    |     |
                            +--------------------+     |
                                                       |
                                                       |
                             Bridge(pci_dev)           |                     
                            +--------------------+<----|---+--------------+
                            |subordinate(pci_bus)|-+   |   |              |
                            +--------------------+ |   |   |              |
                                                   |   |   |              |
                                                   |   |   |              |
                                                   |   |   |              |
                            Bus 1(pci_bus)         |   |   |              |
                       +--->+-------------------+<-+   |   |              |
                       |    |parent(pci_bus)    |------+   |              |
                       |    +-------------------+          |              |
                       |    |self(pci_dev)      |----------+              |
                       |    |bridge(device)     |                         |
                       |    +-------------------+                         |
                       |    |devices(list_head) |                         |
                       |    +--------+----------+                         |
                       |             |                                    |
                       |      ----+--+---+----                            |
   pci_dev             |          |      |          pci_dev               |
   +---------------+<--|----------+      +--------->+-----------------+   |
   |bus(pci_bus)   |---+                            |bus(pci_bus)     |   |
   +---------------+                                +-----------------+   |
   |dev.parent     |                                |dev.parent       |---+
   +---------------+                                +-----------------+

Signature · 2 lines

Richard Yang
Help you, Help me