KVM call minutes 2013-01-29

* Replacing select(2) so that we will not hit the 1024 fd_set limit in the future. (stefan)

Add checks for fd's bigger than 1024? multifunction devices uses lot of fd's for device.

Portability?
Use glib? and let it use poll underneath.
slirp is a problem.
in the end loop: moving to a glib event loop, how we arrive there is the discussion.

Il 29/01/2013 16:41, Juan Quintela ha scritto:

Quoted 10 lines

* Replacing select(2) so that we will not hit the 1024 fd_set limit in the future. (stefan)

Add checks for fd's bigger than 1024? multifunction devices uses lot of fd's for device.

Portability?
Use glib? and let it use poll underneath.
slirp is a problem.
in the end loop: moving to a glib event loop, how we arrive there is the discussion.

We can use g_poll while keeping the main-loop.c wrappers around the glib event loop. Both slirp and iohandler.c access the fd_sets randomly, so we need to remember some state between the fill and poll functions. We can use two main-loop.c functions:

int qemu_add_poll_fd(int fd, int events);

select: writes the events into three fd_sets, returns the file descriptor itself

poll: writes a GPollFD into a dynamically-sized array (of GPollFDs) and returns the index in the array.

int qemu_get_poll_fd_revents(int index);

select: takes the file descriptor (returned by qemu_add_poll_fd), makes up revents based on the three fd_sets

poll: takes the index into the array and returns the corresponding revents

iohandler.c can simply store the index into struct IOHandlerRecord, and use it later. slirp can do the same for struct socket.

The select code can be kept for Windows after POSIX switches to poll.

Paolo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 33 lines · Paolo Bonzini

Il 29/01/2013 16:41, Juan Quintela ha scritto:

Quoted 10 lines

* Replacing select(2) so that we will not hit the 1024 fd_set limit in the future. (stefan)

Add checks for fd's bigger than 1024? multifunction devices uses lot of fd's for device.

Portability?
Use glib? and let it use poll underneath.
slirp is a problem.
in the end loop: moving to a glib event loop, how we arrive there is the discussion.

We can use g_poll while keeping the main-loop.c wrappers around the glib event loop. Both slirp and iohandler.c access the fd_sets randomly, so we need to remember some state between the fill and poll functions. We can use two main-loop.c functions:

int qemu_add_poll_fd(int fd, int events);

select: writes the events into three fd_sets, returns the file descriptor itself

poll: writes a GPollFD into a dynamically-sized array (of GPollFDs) and returns the index in the array.

int qemu_get_poll_fd_revents(int index);

select: takes the file descriptor (returned by qemu_add_poll_fd), makes up revents based on the three fd_sets

poll: takes the index into the array and returns the corresponding revents

Quoted 4 lines

iohandler.c can simply store the index into struct IOHandlerRecord, and use it later. slirp can do the same for struct socket.

The select code can be kept for Windows after POSIX switches to poll.

Doesn't g_poll already do this under the covers for Windows?

* Buildbot: discussed on the list (Andreas retired it)

* Replacing select(2) so that we will not hit the 1024 fd_set limit in the future. (stefan)

Add checks for fd's bigger than 1024? multifunction devices uses lot of fd's for device.

Portability?
Use glib? and let it use poll underneath.
slirp is a problem.
in the end loop: moving to a glib event loop, how we arrive there is the discussion.

* Outstanding virtio work for 1.4
- Multiqueue virtio-net (Amos/Michael)
version appeared today, problably it is on mergeable state
- Refactorings (Fred/Peter)
unlike before the hard freeze
- virtio-ccw (Cornelia/Alex)
conflict with multiqueue problably (alex)
shouldn't (famous last words)
- Do virtio-ccw used old style virtio API, and make integrating the refactorings more difficult?
- Pushing refactorings to 1.5

* What's the plan for -device and IRQ assignment? (Alex)

Alex will fill this

Quoted 1 line · Juan Quintela

Alex will fill this

When using -device, we can not specify an IRQ line to attach to the device. This works for some special buses like PCI, but not in the generic case. We need it generically for virtio-mmio and for potential platform assigned vfio devices though.

The conclusion we came up with was that in order to model IRQ lines between arbitrary devices, we should use QOM and the QOM name space. Details are up for Anthony to fill in :).

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

The problems I ran into were (1) this is a lot of work (2) it basically requires that all bus children have been qdev/QOM-ified. Even with something like the ISA bus which is where I started, quite a few devices were not qdevified still.

Anthony Liguori <[email protected]> writes:

[...]

Quoted 4 lines

The problems I ran into were (1) this is a lot of work (2) it basically requires that all bus children have been qdev/QOM-ified. Even with something like the ISA bus which is where I started, quite a few devices were not qdevified still.

So what's the plan to complete the qdevification job? Lay really low and quietly hope the problem goes away? We've tried that for about three years, doesn't seem to work.

Anthony Liguori <[email protected]> writes:

[...]

Quoted 4 lines

The problems I ran into were (1) this is a lot of work (2) it basically requires that all bus children have been qdev/QOM-ified. Even with something like the ISA bus which is where I started, quite a few devices were not qdevified still.

So what's the plan to complete the qdevification job? Lay really low and quietly hope the problem goes away? We've tried that for about three years, doesn't seem to work.

Quoted 8 lines · Markus Armbruster

Anthony Liguori <[email protected]> writes:

[...]

Quoted 4 lines

The problems I ran into were (1) this is a lot of work (2) it basically requires that all bus children have been qdev/QOM-ified. Even with something like the ISA bus which is where I started, quite a few devices were not qdevified still.

Quoted 3 lines

So what's the plan to complete the qdevification job? Lay really low and quietly hope the problem goes away? We've tried that for about three years, doesn't seem to work.

Do we have a list of not-yet-qdevified devices? Maybe we need to start saying "fix X Y and Z or platform P is dropped from the next release". (This would of course be easier if we had a way to let users know that platform P was in danger...)

Quoted 13 lines · Peter Maydell

Quoted 11 lines · Markus Armbruster

Anthony Liguori <[email protected]> writes:

[...]

Quoted 4 lines

The problems I ran into were (1) this is a lot of work (2) it basically requires that all bus children have been qdev/QOM-ified. Even with something like the ISA bus which is where I started, quite a few devices were not qdevified still.

So what's the plan to complete the qdevification job? Lay really low and quietly hope the problem goes away? We've tried that for about three years, doesn't seem to work.

Quoted 4 lines

Do we have a list of not-yet-qdevified devices? Maybe we need to start saying "fix X Y and Z or platform P is dropped from the next release". (This would of course be easier if we had a way to let users know that platform P was in danger...)

I think that's a good idea. Only problem is identifying pre-qdev devices in the code requires code inspection (grep won't do, I'm afraid).

If we agree on a "qdevify or else" plan, I'd be prepared to help with the digging up of devices.

I disagree on the "or else" part. I have been qdev'ifying and QOM'ifying devices in my maintenance area, and progress is slow. It gets even slower if one leaves clearly maintained areas. I see no good reason to force a pistol on someone's breast, like you have done for IDE, unless there is a good reason to do so. Currently I don't see any.

Il 30/01/2013 14:44, Andreas Färber ha scritto:

Quoted 5 lines

I disagree on the "or else" part. I have been qdev'ifying and QOM'ifying devices in my maintenance area, and progress is slow. It gets even slower if one leaves clearly maintained areas. I see no good reason to force a pistol on someone's breast, like you have done for IDE, unless there is a good reason to do so. Currently I don't see any.

The reason for IDE is that it involved devices that are not SysBusDevices (the IDE disk devices). Having the same code work in two ways, one qdevified and one not, is bad.

For simple SysBusDevice you're changing a crappy default to a less bad one, but there's really little incentive to qdev/QOM-ification.

Am 30.01.2013 13:35, schrieb Markus Armbruster:

Quoted 15 lines

Quoted 13 lines · Peter Maydell

Quoted 11 lines · Markus Armbruster

Anthony Liguori <[email protected]> writes:

[...]

Quoted 4 lines

The problems I ran into were (1) this is a lot of work (2) it basically requires that all bus children have been qdev/QOM-ified. Even with something like the ISA bus which is where I started, quite a few devices were not qdevified still.

So what's the plan to complete the qdevification job? Lay really low and quietly hope the problem goes away? We've tried that for about three years, doesn't seem to work.

Quoted 8 lines

Quoted 4 lines

Do we have a list of not-yet-qdevified devices? Maybe we need to start saying "fix X Y and Z or platform P is dropped from the next release". (This would of course be easier if we had a way to let users know that platform P was in danger...)

I think that's a good idea. Only problem is identifying pre-qdev devices in the code requires code inspection (grep won't do, I'm afraid).

+1 That would address my request as well.

Having a list of low-hanging fruit on the Wiki might also give new contributors some ideas of where and how to start poking at the code.

Quoted 2 lines

If we agree on a "qdevify or else" plan, I'd be prepared to help with the digging up of devices.

I disagree on the "or else" part. I have been qdev'ifying and QOM'ifying devices in my maintenance area, and progress is slow. It gets even

slower if one leaves clearly maintained areas. I see no good reason to force a pistol on someone's breast, like you have done for IDE, unless there is a good reason to do so. Currently I don't see any.

Quoted 9 lines · Peter Maydell

Quoted 8 lines · Markus Armbruster

Anthony Liguori <[email protected]> writes:

[...]

Quoted 4 lines

The problems I ran into were (1) this is a lot of work (2) it basically requires that all bus children have been qdev/QOM-ified. Even with something like the ISA bus which is where I started, quite a few devices were not qdevified still.

Quoted 8 lines

Quoted 3 lines

So what's the plan to complete the qdevification job? Lay really low and quietly hope the problem goes away? We've tried that for about three years, doesn't seem to work.

Do we have a list of not-yet-qdevified devices? Maybe we need to start saying "fix X Y and Z or platform P is dropped from the next release". (This would of course be easier if we had a way to let users know that platform P was in danger...)

I think that's a good idea. Only problem is identifying pre-qdev devices in the code requires code inspection (grep won't do, I'm afraid).

If we agree on a "qdevify or else" plan, I'd be prepared to help with the digging up of devices.

* Portio port to new memory regions?
Andreas, could you fill?

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

Quoted 12 lines · Andreas Färber

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

Quoted 3 lines

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

How does this stuff work on real hardware? I would have expected that a PCI device registering the fact it has IO ports would have to do so via the PCI controller it is plugged into...

My naive don't-know-much-about-portio suggestion is that this should work the same way as memory regions: each device provides portio regions, and the controller for the bus (ISA or PCI) exposes those to the next layer up, and something at board level maps it all into the right places.

-- PMM

Quoted 13 lines · Peter Maydell

Quoted 12 lines · Andreas Färber

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

Quoted 8 lines

Quoted 3 lines

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

How does this stuff work on real hardware? I would have expected that a PCI device registering the fact it has IO ports would have to do so via the PCI controller it is plugged into...

All programming is done by the OS, devices do not register with controller.

Each bridge has two ways to claim an IO transaction:
- transaction is within the window programmed in the bridge
- subtractive decoding enabled and no one else claims the transaction

At the bus level, transaction happens on a bus and an appropriate device will claim it.

Quoted 7 lines

My naive don't-know-much-about-portio suggestion is that this should work the same way as memory regions: each device provides portio regions, and the controller for the bus (ISA or PCI) exposes those to the next layer up, and something at board level maps it all into the right places.

-- PMM

Quoted 23 lines · Michael S. Tsirkin

Quoted 13 lines · Peter Maydell

Quoted 12 lines · Andreas Färber

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

Quoted 8 lines

Quoted 3 lines

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

How does this stuff work on real hardware? I would have expected that a PCI device registering the fact it has IO ports would have to do so via the PCI controller it is plugged into...

Quoted 6 lines

All programming is done by the OS, devices do not register with controller.

Each bridge has two ways to claim an IO transaction:
- transaction is within the window programmed in the bridge
- subtractive decoding enabled and no one else claims the transaction

And there can only be one endpoint that accepts subtractive decoding and this is usually the ISA bridge.

Also note that there are some really special cases with PCI. The legacy VGA ports are always routed to the first device with a DISPLAY class type.

Likewise, with legacy IDE ports are routed to the first device with an IDE class. That's the only reason you can have these legacy devices not behind the ISA bridge.

Regards,

Anthony Liguori

Quoted 11 lines

At the bus level, transaction happens on a bus and an appropriate device will claim it.

Quoted 7 lines

My naive don't-know-much-about-portio suggestion is that this should work the same way as memory regions: each device provides portio regions, and the controller for the bus (ISA or PCI) exposes those to the next layer up, and something at board level maps it all into the right places.

-- PMM

Quoted 12 lines · Andreas Färber

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

    -> "<benh> no mac ever had one"

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

Quoted 3 lines

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

How does this stuff work on real hardware? I would have expected that a PCI device registering the fact it has IO ports would have to do so via the PCI controller it is plugged into...

My naive don't-know-much-about-portio suggestion is that this should work the same way as memory regions: each device provides portio regions, and the controller for the bus (ISA or PCI) exposes those to the next layer up, and something at board level maps it all into the right places.

Quoted 7 lines · Andreas Färber

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

Quoted 8 lines

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

How does this stuff work on real hardware? I would have expected that a PCI device registering the fact it has IO ports would have to do so via the PCI controller it is plugged into...

My naive don't-know-much-about-portio suggestion is that this should work the same way as memory regions: each device provides portio regions, and the controller for the bus (ISA or PCI) exposes those to the next layer up, and something at board level maps it all into the right places.

Quoted 22 lines · Peter Maydell

Quoted 7 lines · Andreas Färber

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

Quoted 8 lines

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

How does this stuff work on real hardware? I would have expected that a PCI device registering the fact it has IO ports would have to do so via the PCI controller it is plugged into...

Quoted 5 lines

My naive don't-know-much-about-portio suggestion is that this should work the same way as memory regions: each device provides portio regions, and the controller for the bus (ISA or PCI) exposes those to the next layer up, and something at board level maps it all into the right places.

Makes sense me, but I'm naive, too :)

For me, "I/O ports" are just an alternate address space some devices have. For instance, x86 CPUs have an extra pin for selecting I/O vs. memory address space. The ISA bus has separate read/write pins for memory and I/O.

This isn't terribly special. Mapping address spaces around is what devices bridging buses do.

I'd expect a system bus for an x86 CPU to have both a memory and an I/O address space.

I'd expect an ISA PC's sysbus - ISA bridge to map both directly.

I'd expect an ISA bridge for a sysbus without a separate I/O address space to map the ISA I/O address space into the sysbus's normal address space somehow.

PCI ISA bridges have their own rules, but I've gotten away with ignoring the details so far :)

Quoted 7 lines

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

Quoted 8 lines

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

How does this stuff work on real hardware? I would have expected that a PCI device registering the fact it has IO ports would have to do so via the PCI controller it is plugged into...

My naive don't-know-much-about-portio suggestion is that this should work the same way as memory regions: each device provides portio regions,

and the controller for the bus
(ISA or PCI) exposes those to the next layer up, and something at board level maps it all into the right places.

-- PMM

Am 30.01.2013 12:48, schrieb Peter Maydell:

Quoted 17 lines · Andreas Färber

Quoted 7 lines

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

Quoted 8 lines

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

Quoted 8 lines

How does this stuff work on real hardware? I would have expected that a PCI device registering the fact it has IO ports would have to do so via the PCI controller it is plugged into...

My naive don't-know-much-about-portio suggestion is that this should work the same way as memory regions: each device provides portio regions,

One remark on "same way as memory regions", me not knowing all the gory hardware details myself.

PIO often contradicts the normal MemoryRegion usage. I.e., for an MMIO device you would have a continuous region from say 0xa0000000 to 0xa007ffff inclusive and within that region you have some kind of sparse registers. With ISA ports you often have dense overlapping ranges, say, 0x3-0x6 byte-reads foo, while 0x4 word-write does bar.

This is handled by having lists of (offset, length, size, handler) quadruplets and consolidating those into MemoryRegions and aliases (cf. patches) that then have a validation function to check whether a particular access is valid and by whom it should be handled - that's what MemoryRegionPortio[] and similar APIs are good for.

So yes, it might be possible to have a device declare its ports at PCIDevice or DeviceState level, but it can't be directly passed through to MemoryRegion API in most cases, or conflicts would arise. At least that was my experience with PReP.

Andreas

Quoted 5 lines

and the controller for the bus
(ISA or PCI) exposes those to the next layer up, and something at board level maps it all into the right places.

-- PMM

Signature · 2 lines

SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Quoted 27 lines

Am 30.01.2013 12:48, schrieb Peter Maydell:

Quoted 17 lines · Andreas Färber

Quoted 7 lines

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

Quoted 8 lines

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

Quoted 8 lines

How does this stuff work on real hardware? I would have expected that a PCI device registering the fact it has IO ports would have to do so via the PCI controller it is plugged into...

My naive don't-know-much-about-portio suggestion is that this should work the same way as memory regions: each device provides portio regions,

Quoted 8 lines

One remark on "same way as memory regions", me not knowing all the gory hardware details myself.

PIO often contradicts the normal MemoryRegion usage. I.e., for an MMIO device you would have a continuous region from say 0xa0000000 to 0xa007ffff inclusive and within that region you have some kind of sparse registers. With ISA ports you often have dense overlapping ranges, say, 0x3-0x6 byte-reads foo, while 0x4 word-write does bar.

Hmm on x86 this is what happens with cf8..cfb range registers for example. We plan handle this ATM using memory region priorities.
Same would work for prep won't it?

Quoted 10 lines

This is handled by having lists of (offset, length, size, handler) quadruplets and consolidating those into MemoryRegions and aliases (cf. patches) that then have a validation function to check whether a particular access is valid and by whom it should be handled - that's what MemoryRegionPortio[] and similar APIs are good for.

So yes, it might be possible to have a device declare its ports at PCIDevice or DeviceState level, but it can't be directly passed through to MemoryRegion API in most cases, or conflicts would arise. At least that was my experience with PReP.

Am 30.01.2013 21:20, schrieb Michael S. Tsirkin:

Quoted 37 lines · Andreas Färber

Quoted 27 lines

Am 30.01.2013 12:48, schrieb Peter Maydell:

Quoted 17 lines · Andreas Färber

Quoted 7 lines

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

Quoted 8 lines

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

Quoted 8 lines

How does this stuff work on real hardware? I would have expected that a PCI device registering the fact it has IO ports would have to do so via the PCI controller it is plugged into...

My naive don't-know-much-about-portio suggestion is that this should work the same way as memory regions: each device provides portio regions,

Quoted 8 lines

One remark on "same way as memory regions", me not knowing all the gory hardware details myself.

PIO often contradicts the normal MemoryRegion usage. I.e., for an MMIO device you would have a continuous region from say 0xa0000000 to 0xa007ffff inclusive and within that region you have some kind of sparse registers. With ISA ports you often have dense overlapping ranges, say, 0x3-0x6 byte-reads foo, while 0x4 word-write does bar.

Quoted 3 lines

Hmm on x86 this is what happens with cf8..cfb range registers for example. We plan handle this ATM using memory region priorities.
Same would work for prep won't it?

Hm, my point was that iiuc a MemoryRegion is per-address-range whereas for I/O ports we seem to have per-data-width mappings. Priorities would allow us to say:

0x1 - 0xff is one region

    0x8-0xab      is a region with higher priority

but fallback for, e.g., word-access at 0xa0 to the lower-priority region being unsupported today, no? I.e., the region being opaque.

Having said that, for the purposes of this discussion PReP is pretty much a PC with a PowerPC CPU in it, unlike the modern CHRP machines.

Andreas

Quoted 10 lines

Quoted 10 lines

This is handled by having lists of (offset, length, size, handler) quadruplets and consolidating those into MemoryRegions and aliases (cf. patches) that then have a validation function to check whether a particular access is valid and by whom it should be handled - that's what MemoryRegionPortio[] and similar APIs are good for.

So yes, it might be possible to have a device declare its ports at PCIDevice or DeviceState level, but it can't be directly passed through to MemoryRegion API in most cases, or conflicts would arise. At least that was my experience with PReP.

Signature · 2 lines

SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Am 29.01.2013 16:41, schrieb Juan Quintela:

Quoted 2 lines

* Portio port to new memory regions?
Andreas, could you fill?

MemoryRegion's .old_portio mechanism requires workarounds for VGA on ppc, affecting among others the sPAPR PCI host bridge:
http://git.qemu.org/?p=qemu.git;a=commit;h=a3cfa18eb075c7ef78358ca1956fe7b01caa1724

Patches were posted and merged removing all .old_portio users but one:
hw/ioport.c:portio_list_add_1(), used by portio_list_add()

hw/isa-bus.c:    portio_list_add(piolist, isabus->address_space_io, start);
hw/qxl.c:    portio_list_add(qxl_vga_port_list,
pci_address_space_io(dev), 0x3b0);
hw/vga.c:        portio_list_add(vga_port_list, address_space_io, 0x3b0);
hw/vga.c:        portio_list_add(vbe_port_list, address_space_io, 0x1ce);

Proposal by hpoussin was to move _list_add() code to ISADevice:
http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg00508.html

Concerns:
* PCI devices (VGA, QXL) register I/O ports as well => above patches add dependency on ISABus to machines

     -> "<benh> no mac ever had one"

=> PCIDevice shouldn't use ISA API with NULL ISADevice
* Lack of avi: Who decides about memory API these days?

armbru and agraf concluded that moving this into ISA is wrong.

=> I will drop the remaining ioport patches from above series.

Suggestions on how to proceed with tackling the issue are welcome.

Regards,
Andreas

Signature · 2 lines

SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

An x86 CPU has a MMIO capability that's essentially 65 bits. Whether the top bit is set determines whether it's a "PIO" transaction or an "MMIO" transaction. A large chunk of that address space is invalid of course.

PCI has a 65 bit address space too. The 65th bit determines whether it's an IO transaction or an MMIO transaction.

For architectures that only have a 64-bit address space, what the PCI controller typically does is pick a 16-bit window within that address space to map to a PCI address with the 65th bit set.

bit set) to a single end point (usually the ISA-bridge). Which one it chooses is up to the chipset. This is called subtractive decoding because the PCI bus will wait multiple cycles for that device to claim the transaction before bouncing it.

2) There are special hacks in most PCI chipsets to route very specific addresses ranges to certain devices. Namely, legacy VGA IO transactions go to the first VGA device. Legacy IDE IO transactions go to the first IDE device. This doesn't need to be programmed in the BARs. It will just happen.

3) As it turns out, all legacy PIIX3 devices are positively decoded and sent to the ISA-bridge (because it's faster this way).

Notice the lack of the word "ISA" in all of this other than describing the PCI class of an end point.

So how should this be modeled?

On x86, the CPU has a pio address space. That can propagate down through the PCI bus which is what we do today.

On !x86, the PCI controller ought to setup a MemoryRegion for

PIO that devices can use to register on.

We probably need to do something like change the PCI VGA devices to export a MemoryRegion and allow the PCI controller to device how to register that as a subregion.

Quoted 7 lines · Anthony Liguori

An x86 CPU has a MMIO capability that's essentially 65 bits. Whether the top bit is set determines whether it's a "PIO" transaction or an "MMIO" transaction. A large chunk of that address space is invalid of course.

PCI has a 65 bit address space too. The 65th bit determines whether it's an IO transaction or an MMIO transaction.

This is somewhat an over simplification since IO and MMIO differs in other ways, such as ordering rules :-) But for the sake of memory regions decoding I suppose it will do.

Quoted 3 lines

For architectures that only have a 64-bit address space, what the PCI controller typically does is pick a 16-bit window within that address space to map to a PCI address with the 65th bit set.

Sort-of yes. The window doesn't have to be 16-bit (we commonly have larger IO space windows on powerpc) and there's a window per host bridge, so there's effectively more than one IO space (as there is more than one PCI MMIO space, with only a window off the CPU space routed to each brigde).

Making a hard wired assumption that the PCI (MMIO and IO) space relates directly to the CPU bus space is wrong on pretty much all !x86 architectures.

.../...

You make it sound like substractive decode is a chipset hack. It's not, it's specified in the PCI spec.

Quoted 5 lines

1) A chipset will route any non-positively decoded IO transaction (65th bit set) to a single end point (usually the ISA-bridge). Which one it chooses is up to the chipset. This is called subtractive decoding because the PCI bus will wait multiple cycles for that device to claim the transaction before bouncing it.

This is not a chipset matter. It's the ISA bridge itself that does substractive decoding.

There also exists P2P bridges doing such substractive decoding, this used to be fairly common with transparent bridges used for laptop docking.

Quoted 5 lines

2) There are special hacks in most PCI chipsets to route very specific addresses ranges to certain devices. Namely, legacy VGA IO transactions go to the first VGA device. Legacy IDE IO transactions go to the first IDE device. This doesn't need to be programmed in the BARs. It will just happen.

This is also mostly not a hack in the chipset. It's a well defined behaviour for legacy devices, sometimes call hard decoding. Of course often those devices are built into the chipset but they don't have to. Plug-in VGA devices will hard decode legacy VGA regions for both IO and MMIO by default (this can be disabled on most of them nowadays) for example. This has nothing to do with the chipset.

There's a specific bit in P2P bridge to control the forwarding of legacy transaction downstream (and VGA palette snoops), this is also fully specified in the PCI spec.

Quoted 3 lines

3) As it turns out, all legacy PIIX3 devices are positively decoded and sent to the ISA-bridge (because it's faster this way).

Chipsets don't "send to a bridge". It's the bridge itself that decodes.

Quoted 2 lines

Notice the lack of the word "ISA" in all of this other than describing the PCI class of an end point.

ISA is only relevant to the extent that the "legacy" regions of IO space originate from the original ISA addresses of devices (VGA, IDE, etc...) and to the extent that an ISA bus might still be present which will get the transactions that nothing else have decoded in that space.

Quoted 7 lines

So how should this be modeled?

On x86, the CPU has a pio address space. That can propagate down through the PCI bus which is what we do today.

On !x86, the PCI controller ought to setup a MemoryRegion for

downstream

Quoted 2 lines

PIO that devices can use to register on.

Quoted 3 lines

We probably need to do something like change the PCI VGA devices to export a MemoryRegion and allow the PCI controller to device how to register that as a subregion.

The VGA device should just register fixed address port IOs the same way it would register an IO BAR. Essentially, hard coded IO addresses (or memory, VGA does memory too, don't forget that) are equivalent to having an invisible BAR with a fixed value in it.

There should be no "global port IO" because that concept is broken on real multi-domain setups. Those "legacy" address ranges are just hard-wired sub regions of the normal PCI space on which the device sits on (unless you start doing real non-PCI ISA x86).

Cheers,
Ben.

Quoted 3 lines · Benjamin Herrenschmidt

There also exists P2P bridges doing such substractive decoding, this used to be fairly common with transparent bridges used for laptop docking.

I'm not sure I understand how this would work. How can two devices on the same PCI domain both do subtractive decoding? Indeed, the PCI spec even says:

"Subtractive decoding can be implemented by only one device on the bus
since it accepts all accesses not positively decoded by some other agent."

Quoted 6 lines

Quoted 5 lines

2) There are special hacks in most PCI chipsets to route very specific addresses ranges to certain devices. Namely, legacy VGA IO transactions go to the first VGA device. Legacy IDE IO transactions go to the first IDE device. This doesn't need to be programmed in the BARs. It will just happen.

Quoted 6 lines

This is also mostly not a hack in the chipset. It's a well defined behaviour for legacy devices, sometimes call hard decoding. Of course often those devices are built into the chipset but they don't have to. Plug-in VGA devices will hard decode legacy VGA regions for both IO and MMIO by default (this can be disabled on most of them nowadays) for example. This has nothing to do with the chipset.

So I understand what you're saying re: PCI because the devices actually assert DEVSEL to indicate that they handle the transaction.

But for PCI-E, doesn't the controller have to expressly identify what the target is? Is this done with the device class?

Quoted 3 lines

There's a specific bit in P2P bridge to control the forwarding of legacy transaction downstream (and VGA palette snoops), this is also fully specified in the PCI spec.

Ack.

Quoted 1 line

Quoted 5 lines

Quoted 2 lines

3) As it turns out, all legacy PIIX3 devices are positively decoded and sent to the ISA-bridge (because it's faster this way).

Chipsets don't "send to a bridge". It's the bridge itself that decodes.

With PCI...

Quoted 7 lines

Quoted 2 lines

Notice the lack of the word "ISA" in all of this other than describing the PCI class of an end point.

ISA is only relevant to the extent that the "legacy" regions of IO space originate from the original ISA addresses of devices (VGA, IDE, etc...) and to the extent that an ISA bus might still be present which will get the transactions that nothing else have decoded in that space.

Ack.

Quoted 10 lines

Quoted 7 lines

So how should this be modeled?

On x86, the CPU has a pio address space. That can propagate down through the PCI bus which is what we do today.

On !x86, the PCI controller ought to setup a MemoryRegion for

downstream

Quoted 2 lines

PIO that devices can use to register on.

Quoted 8 lines

Quoted 3 lines

We probably need to do something like change the PCI VGA devices to export a MemoryRegion and allow the PCI controller to device how to register that as a subregion.

The VGA device should just register fixed address port IOs the same way it would register an IO BAR. Essentially, hard coded IO addresses (or memory, VGA does memory too, don't forget that) are equivalent to having an invisible BAR with a fixed value in it.

Ack.

Quoted 5 lines

There should be no "global port IO" because that concept is broken on real multi-domain setups. Those "legacy" address ranges are just hard-wired sub regions of the normal PCI space on which the device sits on (unless you start doing real non-PCI ISA x86).

So, I think what you're suggesting (and I agree with), is that each PCI device should export one or more MemoryRegions and indicate what the MemoryRegions are for.

Potential options are:

- MMIO BAR
- PIO BAR
- IDE hard decode
- VGA hard decode
- subtractive decode

I'm very much in agreement if that's what you're suggesting.

Regards,

Anthony Liguori

Quoted 3 lines

Cheers,
Ben.

Quoted 17 lines · Benjamin Herrenschmidt

Quoted 7 lines · Anthony Liguori

An x86 CPU has a MMIO capability that's essentially 65 bits. Whether the top bit is set determines whether it's a "PIO" transaction or an "MMIO" transaction. A large chunk of that address space is invalid of course.

PCI has a 65 bit address space too. The 65th bit determines whether it's an IO transaction or an MMIO transaction.

This is somewhat an over simplification since IO and MMIO differs in other ways, such as ordering rules :-) But for the sake of memory regions decoding I suppose it will do.

Quoted 3 lines

For architectures that only have a 64-bit address space, what the PCI controller typically does is pick a 16-bit window within that address space to map to a PCI address with the 65th bit set.

Quoted 5 lines

Sort-of yes. The window doesn't have to be 16-bit (we commonly have larger IO space windows on powerpc) and there's a window per host bridge, so there's effectively more than one IO space (as there is more than one PCI MMIO space, with only a window off the CPU space routed to each brigde).

Ack.

Quoted 3 lines

Making a hard wired assumption that the PCI (MMIO and IO) space relates directly to the CPU bus space is wrong on pretty much all !x86 architectures.

Ack.

Quoted 5 lines

.../...

You make it sound like substractive decode is a chipset hack. It's not, it's specified in the PCI spec.

It's a hack :-) It's a well specified hack, but it's still a hack.

Quoted 8 lines

Quoted 5 lines

1) A chipset will route any non-positively decoded IO transaction (65th bit set) to a single end point (usually the ISA-bridge). Which one it chooses is up to the chipset. This is called subtractive decoding because the PCI bus will wait multiple cycles for that device to claim the transaction before bouncing it.

This is not a chipset matter. It's the ISA bridge itself that does substractive decoding.

The PCI bus can have one end point that that can be the target for subtractive decoding (not hard decoding, subtractive decoding). IOW, you can only have a single ISA Bridge within a single PCI domain.

You are right--chipset is the wrong word. I'm used to thinking in terms of only a single domain :-)

Quoted 3 lines

There also exists P2P bridges doing such substractive decoding, this used to be fairly common with transparent bridges used for laptop docking.

I'm not sure I understand how this would work. How can two devices on the same PCI domain both do subtractive decoding? Indeed, the PCI spec even says:

"Subtractive decoding can be implemented by only one device on the bus
since it accepts all accesses not positively decoded by some other agent."

Quoted 6 lines

Quoted 5 lines

2) There are special hacks in most PCI chipsets to route very specific addresses ranges to certain devices. Namely, legacy VGA IO transactions go to the first VGA device. Legacy IDE IO transactions go to the first IDE device. This doesn't need to be programmed in the BARs. It will just happen.

Quoted 6 lines

This is also mostly not a hack in the chipset. It's a well defined behaviour for legacy devices, sometimes call hard decoding. Of course often those devices are built into the chipset but they don't have to. Plug-in VGA devices will hard decode legacy VGA regions for both IO and MMIO by default (this can be disabled on most of them nowadays) for example. This has nothing to do with the chipset.

So I understand what you're saying re: PCI because the devices actually assert DEVSEL to indicate that they handle the transaction.

But for PCI-E, doesn't the controller have to expressly identify what the target is? Is this done with the device class?

Quoted 3 lines

There's a specific bit in P2P bridge to control the forwarding of legacy transaction downstream (and VGA palette snoops), this is also fully specified in the PCI spec.

Ack.

Quoted 6 lines

Quoted 3 lines

3) As it turns out, all legacy PIIX3 devices are positively decoded and sent to the ISA-bridge (because it's faster this way).

Chipsets don't "send to a bridge". It's the bridge itself that decodes.

With PCI...

Quoted 7 lines

Quoted 2 lines

Notice the lack of the word "ISA" in all of this other than describing the PCI class of an end point.

ISA is only relevant to the extent that the "legacy" regions of IO space originate from the original ISA addresses of devices (VGA, IDE, etc...) and to the extent that an ISA bus might still be present which will get the transactions that nothing else have decoded in that space.

Ack.

Quoted 10 lines

Quoted 7 lines

So how should this be modeled?

On x86, the CPU has a pio address space. That can propagate down through the PCI bus which is what we do today.

On !x86, the PCI controller ought to setup a MemoryRegion for

downstream

Quoted 2 lines

PIO that devices can use to register on.

Quoted 8 lines

Quoted 3 lines

We probably need to do something like change the PCI VGA devices to export a MemoryRegion and allow the PCI controller to device how to register that as a subregion.

The VGA device should just register fixed address port IOs the same way it would register an IO BAR. Essentially, hard coded IO addresses (or memory, VGA does memory too, don't forget that) are equivalent to having an invisible BAR with a fixed value in it.

Ack.

Quoted 5 lines

There should be no "global port IO" because that concept is broken on real multi-domain setups. Those "legacy" address ranges are just hard-wired sub regions of the normal PCI space on which the device sits on (unless you start doing real non-PCI ISA x86).

So, I think what you're suggesting (and I agree with), is that each PCI device should export one or more MemoryRegions and indicate what the MemoryRegions are for.

Potential options are:

- MMIO BAR
- PIO BAR
- IDE hard decode
- VGA hard decode
- subtractive decode

I'm very much in agreement if that's what you're suggesting.

Regards,

Anthony Liguori

Quoted 3 lines

Cheers,
Ben.

Well you can have a PCI bridge and a legacy device behind that. I think real PCI express devices can not be mapped onto legacy address ranges.

From a SW modelling standpoint, I don't think it's worth differentiating

Quoted 4 lines · Michael S. Tsirkin

Well you can have a PCI bridge and a legacy device behind that. I think real PCI express devices can not be mapped onto legacy address ranges.

In practice they do (VGA at least)

Quoted 1 line

From a SW modelling standpoint, I don't think it's worth differentiating

PCI and PCIE.

Cheers,
Ben.

Quoted 3 lines

In practice they do (VGA at least)

Quoted 1 line

From a SW modelling standpoint, I don't think it's worth

differentiating

Quoted 4 lines

PCI and PCIE.

Cheers,
Ben.

Interesting.
Do you have such hardware? Could you please dump
the output of lspci -vv?

Quoted 9 lines · Michael S. Tsirkin

Quoted 3 lines

In practice they do (VGA at least)

Quoted 1 line

From a SW modelling standpoint, I don't think it's worth

differentiating

Quoted 4 lines

PCI and PCIE.

Cheers,
Ben.

Quoted 3 lines

Interesting.
Do you have such hardware? Could you please dump
the output of lspci -vv?

Any ATI or nVidia card still supports hard decoding of VGA regions for the sake of legacy operating systems and BIOSes :-) I don't know about Intel but I suppose it's the same.

Quoted 10 lines · Benjamin Herrenschmidt

Quoted 9 lines · Michael S. Tsirkin

Quoted 3 lines

In practice they do (VGA at least)

Quoted 1 line

From a SW modelling standpoint, I don't think it's worth

differentiating

Quoted 4 lines

PCI and PCIE.

Cheers,
Ben.

Quoted 7 lines

Quoted 3 lines

Interesting.
Do you have such hardware? Could you please dump
the output of lspci -vv?

Any ATI or nVidia card still supports hard decoding of VGA regions for the sake of legacy operating systems and BIOSes :-) I don't know about Intel but I suppose it's the same.

For example:

-[0000:00]-+-00.0 Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx0 p

           +-04.0-[02]--+-00.0  Advanced Micro Devices [AMD] nee ATI Cedar PRO [Radeon HD 5450/6350]

00:04.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port D) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
I/O behind bridge: 0000c000-0000cfff
Memory behind bridge: fd100000-fd1fffff
Prefetchable memory behind bridge: 00000000d0000000-00000000dfffffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA+ MAbort- >Reset- FastB2B-
^^^^
VGA+ (VGA Enable) indicates positive decode of 0x3b0 - 0x3bb, 0x3c0 -
0x3df, and 0xa0000 - 0xbfff. Device 2:00.0 of course doesn't report
these "ISA" ranges as they're implicit in the VGA class code.

BTW, I've been working on vfio-pci support of VGA assignment which makes use of the VGA arbiter in the host to manipulate the VGA Enable control register, allowing us to select which device to access. The qemu side is simply registering memory regions for the VGA areas and expecting to be used with -vga none, but I'll adopt whatever strategy we choose for hard coded address range support. Current base patches at the links below. Thanks,

Alex

https://github.com/awilliam/qemu-vfio/commit/ea2befa59010a429dcf13c10dbccdf8b64e82fbd
https://github.com/awilliam/linux-vfio/commit/bae182d929229cbf1eaeb01e5fad4f77f81a4c61

Quoted 24 lines · Alex Williamson

Quoted 10 lines · Benjamin Herrenschmidt

Quoted 9 lines · Michael S. Tsirkin

Quoted 3 lines

In practice they do (VGA at least)

Quoted 1 line

From a SW modelling standpoint, I don't think it's worth

differentiating

Quoted 4 lines

PCI and PCIE.

Cheers,
Ben.

Quoted 7 lines

Quoted 3 lines

Interesting.
Do you have such hardware? Could you please dump
the output of lspci -vv?

Any ATI or nVidia card still supports hard decoding of VGA regions for the sake of legacy operating systems and BIOSes :-) I don't know about Intel but I suppose it's the same.

For example:

-[0000:00]-+-00.0 Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx0 p

           +-04.0-[02]--+-00.0  Advanced Micro Devices [AMD] nee ATI Cedar PRO [Radeon HD 5450/6350]

Quoted 14 lines

00:04.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port D) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
I/O behind bridge: 0000c000-0000cfff
Memory behind bridge: fd100000-fd1fffff
Prefetchable memory behind bridge: 00000000d0000000-00000000dfffffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA+ MAbort- >Reset- FastB2B-
^^^^
VGA+ (VGA Enable) indicates positive decode of 0x3b0 - 0x3bb, 0x3c0 -
0x3df, and 0xa0000 - 0xbfff. Device 2:00.0 of course doesn't report
these "ISA" ranges as they're implicit in the VGA class code.

OK but this appears behind a bridge. So the bridge configuration tells the root complex where to send accesses to the VGA.

But qemu currently puts devices directly on root bus.

And as far as I can tell when we present devices directly on bus 0, we pretend these are integrated in the root complex. The spec seems to say explicitly that root complex integrated devices should not use legacy addresses or support hotplug. So I would be surprised if such one appears in real world.

Luckily guests do not seem to be worried as long as we use ACPI.

Quoted 51 lines · Michael S. Tsirkin

Quoted 24 lines · Alex Williamson

Quoted 10 lines · Benjamin Herrenschmidt

Quoted 9 lines · Michael S. Tsirkin

Quoted 3 lines

In practice they do (VGA at least)

Quoted 1 line

From a SW modelling standpoint, I don't think it's worth

differentiating

Quoted 4 lines

PCI and PCIE.

Cheers,
Ben.

Quoted 7 lines

Quoted 3 lines

Interesting.
Do you have such hardware? Could you please dump
the output of lspci -vv?

Any ATI or nVidia card still supports hard decoding of VGA regions for the sake of legacy operating systems and BIOSes :-) I don't know about Intel but I suppose it's the same.

For example:

-[0000:00]-+-00.0 Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx0 p

           +-04.0-[02]--+-00.0  Advanced Micro Devices [AMD] nee ATI Cedar PRO [Radeon HD 5450/6350]

Quoted 14 lines

00:04.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port D) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
I/O behind bridge: 0000c000-0000cfff
Memory behind bridge: fd100000-fd1fffff
Prefetchable memory behind bridge: 00000000d0000000-00000000dfffffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA+ MAbort- >Reset- FastB2B-
^^^^
VGA+ (VGA Enable) indicates positive decode of 0x3b0 - 0x3bb, 0x3c0 -
0x3df, and 0xa0000 - 0xbfff. Device 2:00.0 of course doesn't report
these "ISA" ranges as they're implicit in the VGA class code.

OK but this appears behind a bridge. So the bridge configuration tells the root complex where to send accesses to the VGA.

But qemu currently puts devices directly on root bus.

And as far as I can tell when we present devices directly on bus 0, we pretend these are integrated in the root complex. The spec seems to say explicitly that root complex integrated devices should not use legacy addresses or support hotplug. So I would be surprised if such one appears in real world.

Quoted 1 line

Luckily guests do not seem to be worried as long as we use ACPI.

Yes, in fact I just figured out last night that Windows is unhappy with assigned PCI devices on bus 0 that claim to be an endpoint in their PCIe capability rather than an integrated endpoint. We'll need to do extra mangling of the PCIe capability to massage it into the guest visible topology.

Section 1.3.2.3 of the 3.0 spec says integrated endpoints must not require I/O resources claimed through BAR(s). VGA skirts around this by not having the legacy resources claimed by BARs, but instead being implicit.

Are there other sections restricting legacy I/O?

It's common that a plugin VGA card sits behind a root port where the bridge registers tell us about VGA routing,
but integrated VGA devices
are often on bus 0 though, here's an example:

-[0000:00]-+-00.0 Intel Corporation 2nd Generation Core Processor Family DRAM Controller
+-02.0 Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller

Often these systems will disable the integrated graphics when a plugin graphics is installed below a root port. I'm not sure how the system knows to route VGA to the integrated device vs the root port otherwise.

Here's a more interesting example:

-+-[0000:01]-+-00.0  NVIDIA Corporation GT218 [GeForce G210M]
 |           \-00.1  NVIDIA Corporation High Definition Audio Controller
 \-[0000:00]-+-00.0  Intel Corporation Mobile 4 Series Chipset Memory Controller Hub
             +-01.0  Intel Corporation Mobile 4 Series Chipset PCI Express Graphics Port

This system seems to have two host bridges with VGA behind each of them. There's no bridge to control VGA routing, so I don't know how the selection is done.

It's possible the g210m never sees legacy VGA
accesses in this mode. This bios has another mode which makes the g210m the primary graphics and hides the integrated graphics, essentially the same as I mention above with hiding integrated endpoint graphics when plugin graphics are used. Thanks,

Alex

Quoted 55 lines · Alex Williamson

Quoted 52 lines · Michael S. Tsirkin

Quoted 24 lines · Alex Williamson

Quoted 10 lines · Benjamin Herrenschmidt

Quoted 9 lines · Michael S. Tsirkin

Quoted 3 lines

In practice they do (VGA at least)

Quoted 1 line

From a SW modelling standpoint, I don't think it's worth

differentiating

Quoted 4 lines

PCI and PCIE.

Cheers,
Ben.

Quoted 7 lines

Quoted 3 lines

Interesting.
Do you have such hardware? Could you please dump
the output of lspci -vv?

Any ATI or nVidia card still supports hard decoding of VGA regions for the sake of legacy operating systems and BIOSes :-) I don't know about Intel but I suppose it's the same.

For example:

-[0000:00]-+-00.0 Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx0 p

           +-04.0-[02]--+-00.0  Advanced Micro Devices [AMD] nee ATI Cedar PRO [Radeon HD 5450/6350]

Quoted 14 lines

00:04.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port D) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
I/O behind bridge: 0000c000-0000cfff
Memory behind bridge: fd100000-fd1fffff
Prefetchable memory behind bridge: 00000000d0000000-00000000dfffffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA+ MAbort- >Reset- FastB2B-
^^^^
VGA+ (VGA Enable) indicates positive decode of 0x3b0 - 0x3bb, 0x3c0 -
0x3df, and 0xa0000 - 0xbfff. Device 2:00.0 of course doesn't report
these "ISA" ranges as they're implicit in the VGA class code.

OK but this appears behind a bridge. So the bridge configuration tells the root complex where to send accesses to the VGA.

But qemu currently puts devices directly on root bus.

And as far as I can tell when we present devices directly on bus 0, we pretend these are integrated in the root complex. The spec seems to say explicitly that root complex integrated devices should not use legacy addresses or support hotplug. So I would be surprised if such one appears in real world.

Luckily guests do not seem to be worried as long as we use ACPI.

Quoted 5 lines

Yes, in fact I just figured out last night that Windows is unhappy with assigned PCI devices on bus 0 that claim to be an endpoint in their PCIe capability rather than an integrated endpoint. We'll need to do extra mangling of the PCIe capability to massage it into the guest visible topology.

For now, just put you device behind an express bridge. This breaks acpi hotplug for now, but I'm looking into hotplug with bridges anyway.

If you really need it I can give you a hack for hotplug too.

Of course express does not allow hotplug of root complex parts but happens to work because we use ACPI.

Quoted 4 lines

Section 1.3.2.3 of the 3.0 spec says integrated endpoints must not require I/O resources claimed through BAR(s). VGA skirts around this by not having the legacy resources claimed by BARs, but instead being implicit.

Aha. I missed this point.

Quoted 1 line

Are there other sections restricting legacy I/O?

One other interesting things is that VGA enable bit (for bridge control register) does not appear in express spec at all.

Quoted 11 lines

It's common that a plugin VGA card sits behind a root port where the bridge registers tell us about VGA routing,
but integrated VGA devices
are often on bus 0 though, here's an example:

-[0000:00]-+-00.0 Intel Corporation 2nd Generation Core Processor Family DRAM Controller
+-02.0 Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller

Often these systems will disable the integrated graphics when a plugin graphics is installed below a root port. I'm not sure how the system knows to route VGA to the integrated device vs the root port otherwise.

I am guessing it disables the integrated graphics?

Quoted 7 lines

Here's a more interesting example:

-+-[0000:01]-+-00.0  NVIDIA Corporation GT218 [GeForce G210M]
 |           \-00.1  NVIDIA Corporation High Definition Audio Controller
 \-[0000:00]-+-00.0  Intel Corporation Mobile 4 Series Chipset Memory Controller Hub
             +-01.0  Intel Corporation Mobile 4 Series Chipset PCI Express Graphics Port

Quoted 3 lines

This system seems to have two host bridges with VGA behind each of them. There's no bridge to control VGA routing, so I don't know how the selection is done.

Is IO space disabled for the inactive card? Maybe that is how.

Quoted 56 lines · Michael S. Tsirkin

Quoted 55 lines · Alex Williamson

Quoted 52 lines · Michael S. Tsirkin

Quoted 24 lines · Alex Williamson

Quoted 10 lines · Benjamin Herrenschmidt

Quoted 9 lines · Michael S. Tsirkin

Quoted 3 lines

In practice they do (VGA at least)

Quoted 1 line

From a SW modelling standpoint, I don't think it's worth

differentiating

Quoted 4 lines

PCI and PCIE.

Cheers,
Ben.

Quoted 7 lines

Quoted 3 lines

Interesting.
Do you have such hardware? Could you please dump
the output of lspci -vv?

Any ATI or nVidia card still supports hard decoding of VGA regions for the sake of legacy operating systems and BIOSes :-) I don't know about Intel but I suppose it's the same.

For example:

-[0000:00]-+-00.0 Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx0 p

           +-04.0-[02]--+-00.0  Advanced Micro Devices [AMD] nee ATI Cedar PRO [Radeon HD 5450/6350]

Quoted 14 lines

00:04.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port D) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
I/O behind bridge: 0000c000-0000cfff
Memory behind bridge: fd100000-fd1fffff
Prefetchable memory behind bridge: 00000000d0000000-00000000dfffffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA+ MAbort- >Reset- FastB2B-
^^^^
VGA+ (VGA Enable) indicates positive decode of 0x3b0 - 0x3bb, 0x3c0 -
0x3df, and 0xa0000 - 0xbfff. Device 2:00.0 of course doesn't report
these "ISA" ranges as they're implicit in the VGA class code.

OK but this appears behind a bridge. So the bridge configuration tells the root complex where to send accesses to the VGA.

But qemu currently puts devices directly on root bus.

And as far as I can tell when we present devices directly on bus 0, we pretend these are integrated in the root complex. The spec seems to say explicitly that root complex integrated devices should not use legacy addresses or support hotplug. So I would be surprised if such one appears in real world.

Luckily guests do not seem to be worried as long as we use ACPI.

Quoted 8 lines

Quoted 5 lines

Yes, in fact I just figured out last night that Windows is unhappy with assigned PCI devices on bus 0 that claim to be an endpoint in their PCIe capability rather than an integrated endpoint. We'll need to do extra mangling of the PCIe capability to massage it into the guest visible topology.

For now, just put you device behind an express bridge. This breaks acpi hotplug for now, but I'm looking into hotplug with bridges anyway.

We have the problem in both directions though, Endpoints that should be Integrated Endpoints and Integrated Endpoints that should be Endpoints. So I think we need to mangle the type.

Quoted 4 lines

If you really need it I can give you a hack for hotplug too.

Of course express does not allow hotplug of root complex parts but happens to work because we use ACPI.

That's a little odd.

Quoted 11 lines

Quoted 4 lines

Section 1.3.2.3 of the 3.0 spec says integrated endpoints must not require I/O resources claimed through BAR(s). VGA skirts around this by not having the legacy resources claimed by BARs, but instead being implicit.

Aha. I missed this point.

Quoted 1 line

Are there other sections restricting legacy I/O?

One other interesting things is that VGA enable bit (for bridge control register) does not appear in express spec at all.

Yep, but it appears on hardware.

Quoted 21 lines

Quoted 11 lines

It's common that a plugin VGA card sits behind a root port where the bridge registers tell us about VGA routing,
but integrated VGA devices
are often on bus 0 though, here's an example:

-[0000:00]-+-00.0 Intel Corporation 2nd Generation Core Processor Family DRAM Controller
+-02.0 Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller

Often these systems will disable the integrated graphics when a plugin graphics is installed below a root port. I'm not sure how the system knows to route VGA to the integrated device vs the root port otherwise.

I am guessing it disables the integrated graphics?

Quoted 7 lines

Here's a more interesting example:

-+-[0000:01]-+-00.0  NVIDIA Corporation GT218 [GeForce G210M]
 |           \-00.1  NVIDIA Corporation High Definition Audio Controller
 \-[0000:00]-+-00.0  Intel Corporation Mobile 4 Series Chipset Memory Controller Hub
             +-01.0  Intel Corporation Mobile 4 Series Chipset PCI Express Graphics Port

Quoted 5 lines

Quoted 3 lines

This system seems to have two host bridges with VGA behind each of them. There's no bridge to control VGA routing, so I don't know how the selection is done.

Is IO space disabled for the inactive card? Maybe that is how.

The card has BAR defined I/O space resources. My guess is that VGA is just statically routed to the integrated device and the secondary works only in non-legacy mode until the BIOS switch is flipped, the integrated device is hidden and VGA is switched to static routing for the nvidia device. I suppose that means I'll never be able to assign the nvidia to a guest, at least not with any kind of legacy VGA support. Thanks,

Alex

Quoted 1 line

Luckily guests do not seem to be worried as long as we use ACPI.

Yes, in fact I just figured out last night that Windows is unhappy with assigned PCI devices on bus 0 that claim to be an endpoint in their PCIe capability rather than an integrated endpoint. We'll need to do extra mangling of the PCIe capability to massage it into the guest visible topology.

Section 1.3.2.3 of the 3.0 spec says integrated endpoints must not require I/O resources claimed through BAR(s). VGA skirts around this by not having the legacy resources claimed by BARs, but instead being implicit. Are there other sections restricting legacy I/O?

It's common that a plugin VGA card sits behind a root port where the bridge registers tell us about VGA routing, but integrated VGA devices are often on bus 0 though, here's an example:

-[0000:00]-+-00.0 Intel Corporation 2nd Generation Core Processor Family DRAM Controller
+-02.0 Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller

Often these systems will disable the integrated graphics when a plugin graphics is installed below a root port. I'm not sure how the system knows to route VGA to the integrated device vs the root port otherwise.

Here's a more interesting example:

-+-[0000:01]-+-00.0  NVIDIA Corporation GT218 [GeForce G210M]
 |           \-00.1  NVIDIA Corporation High Definition Audio Controller
 \-[0000:00]-+-00.0  Intel Corporation Mobile 4 Series Chipset Memory Controller Hub
             +-01.0  Intel Corporation Mobile 4 Series Chipset PCI Express Graphics Port

This system seems to have two host bridges with VGA behind each of them. There's no bridge to control VGA routing, so I don't know how the selection is done. It's possible the g210m never sees legacy VGA accesses in this mode. This bios has another mode which makes the g210m the primary graphics and hides the integrated graphics, essentially the same as I mention above with hiding integrated endpoint graphics when plugin graphics are used. Thanks,

Quoted 7 lines

Here's a more interesting example:

-+-[0000:01]-+-00.0  NVIDIA Corporation GT218 [GeForce G210M]
 |           \-00.1  NVIDIA Corporation High Definition Audio Controller
 \-[0000:00]-+-00.0  Intel Corporation Mobile 4 Series Chipset Memory Controller Hub
             +-01.0  Intel Corporation Mobile 4 Series Chipset PCI Express Graphics Port

Quoted 7 lines

This system seems to have two host bridges with VGA behind each of them. There's no bridge to control VGA routing, so I don't know how the selection is done. It's possible the g210m never sees legacy VGA accesses in this mode. This bios has another mode which makes the g210m the primary graphics and hides the integrated graphics, essentially the same as I mention above with hiding integrated endpoint graphics when plugin graphics are used. Thanks,

Wait, those are two different busses ... and there's no bridge ? Is that the funky x86 multi domain crackpot where you have multiple roots with non overlapping bus numbers in the same domain ?

Ben.

Quoted 7 lines · Alex Williamson

Quoted 1 line

Luckily guests do not seem to be worried as long as we use ACPI.

Yes, in fact I just figured out last night that Windows is unhappy with assigned PCI devices on bus 0 that claim to be an endpoint in their PCIe capability rather than an integrated endpoint. We'll need to do extra mangling of the PCIe capability to massage it into the guest visible topology.

If you are on bus 0, you need to either not have the capability, or if you do, have it be root complex or RC intergrated endpoint. It's fair game for any OS to assume that an endpoint will have a parent bridge (either a RC or a downstream port) and to muck around with link control etc...

Typically on my laptop with intel chipset, bus 0 has devices that just don't have any PCIe capabilities.

Quoted 4 lines

Section 1.3.2.3 of the 3.0 spec says integrated endpoints must not require I/O resources claimed through BAR(s). VGA skirts around this by not having the legacy resources claimed by BARs, but instead being implicit. Are there other sections restricting legacy I/O?

Right this is odd, I don't know why they put that in. Legacy endpoints don't have that limitation and I doubt system software actually cares.

On the other hand, I suspect that doesn't apply if you simply doesn't have the PCIe capability at all :-) IE, that's basically what my laptop looks like here. The Intel graphics appears on bus 0 and has IO ports mapped with a BAR and no PCIe cap.

Same with the on-chip SATA.

In fact they have a "PCI Advanced features" capability, but not PCIe.

Then they have a bunch of root complexes as siblings.

Quoted 10 lines

It's common that a plugin VGA card sits behind a root port where the bridge registers tell us about VGA routing, but integrated VGA devices are often on bus 0 though, here's an example:

-[0000:00]-+-00.0 Intel Corporation 2nd Generation Core Processor Family DRAM Controller
+-02.0 Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller

Often these systems will disable the integrated graphics when a plugin graphics is installed below a root port. I'm not sure how the system knows to route VGA to the integrated device vs the root port otherwise.

It's a good question... I would say the "cleanest" way is to use the VGA Enable bit of the root complex. If the RC is set to forward downstream, then the plug-in card gets the VGA cycles, else, they go to the integrated one (substractive decoding -style).

However, the PCI-E spec has removed that bit from the bridge control register definition :-)

So whatever mechanism those chipsets use has to be somewhat proprietary.

On the other hand, I don't see it hurting to make our own "proprietary" mechanism consist of using ... the bridge control VGA enable bit. IE. The bit is not used in the PCIe spec and probably never will be so we can use it for its original purpose.

Quoted 7 lines

Here's a more interesting example:

-+-[0000:01]-+-00.0  NVIDIA Corporation GT218 [GeForce G210M]
 |           \-00.1  NVIDIA Corporation High Definition Audio Controller
 \-[0000:00]-+-00.0  Intel Corporation Mobile 4 Series Chipset Memory Controller Hub
             +-01.0  Intel Corporation Mobile 4 Series Chipset PCI Express Graphics Port

Quoted 7 lines

This system seems to have two host bridges with VGA behind each of them. There's no bridge to control VGA routing, so I don't know how the selection is done. It's possible the g210m never sees legacy VGA accesses in this mode. This bios has another mode which makes the g210m the primary graphics and hides the integrated graphics, essentially the same as I mention above with hiding integrated endpoint graphics when plugin graphics are used. Thanks,

Wait, those are two different busses ... and there's no bridge ? Is that the funky x86 multi domain crackpot where you have multiple roots with non overlapping bus numbers in the same domain ?

OK but this appears behind a bridge. So the bridge configuration tells the root complex where to send accesses to the VGA.

But qemu currently puts devices directly on root bus.

And as far as I can tell when we present devices directly on bus 0, we pretend these are integrated in the root complex.

The spec seems to
say explicitly that root complex integrated devices should not use legacy addresses or support hotplug. So I would be surprised if such one appears in real world.

Luckily guests do not seem to be worried as long as we use ACPI.

Quoted 12 lines

BTW, I've been working on vfio-pci support of VGA assignment which makes use of the VGA arbiter in the host to manipulate the VGA Enable control register, allowing us to select which device to access. The qemu side is simply registering memory regions for the VGA areas and expecting to be used with -vga none, but I'll adopt whatever strategy we choose for hard coded address range support. Current base patches at the links below. Thanks,

Alex

https://github.com/awilliam/qemu-vfio/commit/ea2befa59010a429dcf13c10dbccdf8b64e82fbd
https://github.com/awilliam/linux-vfio/commit/bae182d929229cbf1eaeb01e5fad4f77f81a4c61

Quoted 2 lines · Michael S. Tsirkin

OK but this appears behind a bridge. So the bridge configuration tells the root complex where to send accesses to the VGA.

Sort-of, again the root complex isn't "sending" anything targeted here. PCIe is point to point and any device is behind a bridge, real or virtual.

Quoted 1 line

But qemu currently puts devices directly on root bus.

Sure, because qemu doesn't specifically model PCIe but something "else"

Quoted 2 lines

And as far as I can tell when we present devices directly on bus 0, we pretend these are integrated in the root complex.

Right, it's a bit gross.

Quoted 4 lines

The spec seems to
say explicitly that root complex integrated devices should not use legacy addresses or support hotplug. So I would be surprised if such one appears in real world.

Sure but that doesn't change the fact that there's no point in treating things differently between PCI and PCIe for the sake of address range decoding. The high level model remains the same.

Quoted 1 line

Luckily guests do not seem to be worried as long as we use ACPI.

Right, it all just looks like PCI to the guest anyway and is mostly treated as such for the sake of routing and decoding (until you turn on ARI but that's a different can of worms).

Quoted 12 lines

Quoted 12 lines

BTW, I've been working on vfio-pci support of VGA assignment which makes use of the VGA arbiter in the host to manipulate the VGA Enable control register, allowing us to select which device to access. The qemu side is simply registering memory regions for the VGA areas and expecting to be used with -vga none, but I'll adopt whatever strategy we choose for hard coded address range support. Current base patches at the links below. Thanks,

Alex

https://github.com/awilliam/qemu-vfio/commit/ea2befa59010a429dcf13c10dbccdf8b64e82fbd
https://github.com/awilliam/linux-vfio/commit/bae182d929229cbf1eaeb01e5fad4f77f81a4c61

hw/qxl.c: portio_list_add(qxl_vga_port_list,
pci_address_space_io(dev), 0x3b0);
hw/vga.c: portio_list_add(vga_port_list, address_space_io, 0x3b0);

Hi,

Quoted 3 lines

hw/qxl.c: portio_list_add(qxl_vga_port_list,
pci_address_space_io(dev), 0x3b0);
hw/vga.c: portio_list_add(vga_port_list, address_space_io, 0x3b0);

That reminds me I should solve this in a more elegant way.

qxl takes over the vga io ports. The reason it does this is because qxl switches into vga mode in case the vga ports are accessed while not in vga mode. After doing the check (and possibly switching mode) the vga handler is called to actually handle it.

That twist makes it a bit hard to convert vga ...

Anyone knows how one would do that with the memory api instead? I think taking over the ports is easy as the memory regions have priorities so I can simply register a region with higher priority. I have no clue how to forward the access to the vga code though.

Anyone has clues / suggestions?

thanks,
Gerd

Quoted 10 lines · Gerd Hoffmann

Quoted 3 lines

hw/qxl.c: portio_list_add(qxl_vga_port_list,
pci_address_space_io(dev), 0x3b0);
hw/vga.c: portio_list_add(vga_port_list, address_space_io, 0x3b0);

That reminds me I should solve this in a more elegant way.

qxl takes over the vga io ports. The reason it does this is because qxl switches into vga mode in case the vga ports are accessed while not in vga mode. After doing the check (and possibly switching mode) the vga handler is called to actually handle it.

The best way to handle this would be to remodel how we do VGA.

Make VGACommonState a proper QOM object and use it as the base class for QXL, CirrusVGA, QEMUVGA (std-vga), and VMwareVGA.

The VGA accessors should be exposed as a memory region but the sub class ought to be responsible for actually adding it to a subregion.

Quoted 8 lines

That twist makes it a bit hard to convert vga ...

Anyone knows how one would do that with the memory api instead? I think taking over the ports is easy as the memory regions have priorities so I can simply register a region with higher priority. I have no clue how to forward the access to the vga code though.

That should be possible with priorities, but I think it's wrong. There aren't two VGA devices. QXL is-a VGA device and the best way to override behavior of base VGA device is through polymorphism.

This isn't really a memory API issue, it's a modeling issue.

Regards,

Anthony Liguori

Quoted 4 lines

Anyone has clues / suggestions?

thanks,
Gerd

Am 30.01.2013 17:33, schrieb Anthony Liguori:

Quoted 13 lines

Quoted 10 lines · Gerd Hoffmann

Quoted 3 lines

hw/qxl.c: portio_list_add(qxl_vga_port_list,
pci_address_space_io(dev), 0x3b0);
hw/vga.c: portio_list_add(vga_port_list, address_space_io, 0x3b0);

That reminds me I should solve this in a more elegant way.

qxl takes over the vga io ports. The reason it does this is because qxl switches into vga mode in case the vga ports are accessed while not in vga mode. After doing the check (and possibly switching mode) the vga handler is called to actually handle it.

Quoted 4 lines

The best way to handle this would be to remodel how we do VGA.

Make VGACommonState a proper QOM object and use it as the base class for QXL, CirrusVGA, QEMUVGA (std-vga), and VMwareVGA.

That would require polymorphism since we already need to derive from PCIDevice or ISADevice respectively for interfacing with the bus...

Modern object-oriented languages have tried to avoid multi-inheritence due to arising complications, I thought. Wouldn't object if someone wanted to do the dirty implementation work though. ;)

Another such example is EHCI, with PCIDevice and SysBusDevice frontends, sharing an EHCIState struct and having helper functions operating on that core state only. Quite a few device share such a pattern today actually (serial, m48t59, ...).

Quoted 12 lines

The VGA accessors should be exposed as a memory region but the sub class ought to be responsible for actually adding it to a subregion.

Quoted 8 lines

That twist makes it a bit hard to convert vga ...

Anyone knows how one would do that with the memory api instead? I think taking over the ports is easy as the memory regions have priorities so I can simply register a region with higher priority. I have no clue how to forward the access to the vga code though.

Quoted 3 lines

That should be possible with priorities, but I think it's wrong. There aren't two VGA devices. QXL is-a VGA device and the best way to override behavior of base VGA device is through polymorphism.

In this particular case QXL is-a PCI VGA device though, so we can decouple it from core VGA modeling. Placing the MemoryRegionOps inside the Class (rather than static const) might be a short-term solution for overriding read/write handlers of a particular VGA MemoryRegion. :)

Cheers,
Andreas

Quoted 10 lines

This isn't really a memory API issue, it's a modeling issue.

Regards,

Anthony Liguori

Quoted 4 lines

Anyone has clues / suggestions?

thanks,
Gerd

Signature · 2 lines

SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Quoted 14 lines · Andreas Färber

Am 30.01.2013 17:33, schrieb Anthony Liguori:

Quoted 13 lines

Quoted 10 lines · Gerd Hoffmann

Quoted 3 lines

hw/qxl.c: portio_list_add(qxl_vga_port_list,
pci_address_space_io(dev), 0x3b0);
hw/vga.c: portio_list_add(vga_port_list, address_space_io, 0x3b0);

That reminds me I should solve this in a more elegant way.

qxl takes over the vga io ports. The reason it does this is because qxl switches into vga mode in case the vga ports are accessed while not in vga mode. After doing the check (and possibly switching mode) the vga handler is called to actually handle it.

Quoted 7 lines

Quoted 4 lines

The best way to handle this would be to remodel how we do VGA.

Make VGACommonState a proper QOM object and use it as the base class for QXL, CirrusVGA, QEMUVGA (std-vga), and VMwareVGA.

That would require polymorphism since we already need to derive from PCIDevice or ISADevice respectively for interfacing with the bus...

Nope. You can use composition:

QXLDevice is-a VGACommonState

QXLPCI is-a PCIDevice
has-a QXLDevice

Quoted 3 lines

Modern object-oriented languages have tried to avoid multi-inheritence due to arising complications, I thought. Wouldn't object if someone wanted to do the dirty implementation work though. ;)

There is no need for MI.

Quoted 4 lines

Another such example is EHCI, with PCIDevice and SysBusDevice frontends, sharing an EHCIState struct and having helper functions operating on that core state only. Quite a few device share such a pattern today actually (serial, m48t59, ...).

Yes, this is all about chipset modelling. Chipsets should derive from device and then be embedded in the appropriate bus device.

For instance.

SerialState is-a DeviceState

ISASerialState is-a ISADevice, has-a SerialState MMIOSerialState is-a SysbusDevice, has-a SerialState

This is what we're doing in practice, we just aren't modeling the chipsets and we're open coding the relationships (often in subtley different ways).

Regards,

Anthony Liguori

Quoted 38 lines

Quoted 12 lines

The VGA accessors should be exposed as a memory region but the sub class ought to be responsible for actually adding it to a subregion.

Quoted 8 lines

That twist makes it a bit hard to convert vga ...

Anyone knows how one would do that with the memory api instead? I think taking over the ports is easy as the memory regions have priorities so I can simply register a region with higher priority. I have no clue how to forward the access to the vga code though.

Quoted 3 lines

That should be possible with priorities, but I think it's wrong. There aren't two VGA devices. QXL is-a VGA device and the best way to override behavior of base VGA device is through polymorphism.

In this particular case QXL is-a PCI VGA device though, so we can decouple it from core VGA modeling. Placing the MemoryRegionOps inside the Class (rather than static const) might be a short-term solution for overriding read/write handlers of a particular VGA MemoryRegion. :)

Cheers,
Andreas

Quoted 10 lines

This isn't really a memory API issue, it's a modeling issue.

Regards,

Anthony Liguori

Quoted 4 lines

Anyone has clues / suggestions?

thanks,
Gerd

Signature · 2 lines

SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Quoted 6 lines · Anthony

Nope. You can use composition:

QXLDevice is-a VGACommonState

QXLPCI is-a PCIDevice
has-a QXLDevice

But why like this?
The distinction is artificial, isn't it?

Quoted 14 lines · Andreas Färber

Am 30.01.2013 17:33, schrieb Anthony Liguori:

Quoted 13 lines

Quoted 10 lines · Gerd Hoffmann

Quoted 3 lines

hw/qxl.c: portio_list_add(qxl_vga_port_list,
pci_address_space_io(dev), 0x3b0);
hw/vga.c: portio_list_add(vga_port_list, address_space_io, 0x3b0);

That reminds me I should solve this in a more elegant way.

qxl takes over the vga io ports. The reason it does this is because qxl switches into vga mode in case the vga ports are accessed while not in vga mode. After doing the check (and possibly switching mode) the vga handler is called to actually handle it.

Quoted 7 lines

Quoted 4 lines

The best way to handle this would be to remodel how we do VGA.

Make VGACommonState a proper QOM object and use it as the base class for QXL, CirrusVGA, QEMUVGA (std-vga), and VMwareVGA.

That would require polymorphism since we already need to derive from PCIDevice or ISADevice respectively for interfacing with the bus...

Nope. You can use composition:

QXLDevice is-a VGACommonState

QXLPCI is-a PCIDevice
has-a QXLDevice

Quoted 3 lines

Modern object-oriented languages have tried to avoid multi-inheritence due to arising complications, I thought. Wouldn't object if someone wanted to do the dirty implementation work though. ;)

There is no need for MI.

Quoted 4 lines

Another such example is EHCI, with PCIDevice and SysBusDevice frontends, sharing an EHCIState struct and having helper functions operating on that core state only. Quite a few device share such a pattern today actually (serial, m48t59, ...).

Yes, this is all about chipset modelling. Chipsets should derive from device and then be embedded in the appropriate bus device.

For instance.

SerialState is-a DeviceState

ISASerialState is-a ISADevice, has-a SerialState MMIOSerialState is-a SysbusDevice, has-a SerialState

This is what we're doing in practice, we just aren't modeling the chipsets and we're open coding the relationships (often in subtley different ways).

Regards,

Anthony Liguori

That would require polymorphism since we already need to derive from PCIDevice or ISADevice respectively for interfacing with the bus... Modern object-oriented languages have tried to avoid multi-inheritence due to arising complications, I thought. Wouldn't object if someone wanted to do the dirty implementation work though. ;)

Another such example is EHCI, with PCIDevice and SysBusDevice frontends,
sharing an EHCIState struct and having helper functions operating on that core state only. Quite a few device share such a pattern today actually (serial, m48t59, ...).

Quoted 7 lines · Andreas Färber

That would require polymorphism since we already need to derive from PCIDevice or ISADevice respectively for interfacing with the bus... Modern object-oriented languages have tried to avoid multi-inheritence due to arising complications, I thought. Wouldn't object if someone wanted to do the dirty implementation work though. ;)

Quoted 5 lines

Another such example is EHCI, with PCIDevice and SysBusDevice frontends,
sharing an EHCIState struct and having helper functions operating on that core state only. Quite a few device share such a pattern today actually (serial, m48t59, ...).

This is a design bug of your model :-) You shouldn't derive from your bus interface IMHO but from your functional interface, and have an ownership relation to the PCIDevice (a bit like IOKit does if my memory serves me well).

Cheers,
Ben.

Quoted 6 lines · Gerd Hoffmann

Hi,

Quoted 3 lines

hw/qxl.c: portio_list_add(qxl_vga_port_list,
pci_address_space_io(dev), 0x3b0);
hw/vga.c: portio_list_add(vga_port_list, address_space_io, 0x3b0);

Quoted 6 lines

That reminds me I should solve this in a more elegant way.

qxl takes over the vga io ports. The reason it does this is because qxl switches into vga mode in case the vga ports are accessed while not in vga mode. After doing the check (and possibly switching mode) the vga handler is called to actually handle it.

The best way to handle this would be to remodel how we do VGA.

Make VGACommonState a proper QOM object and use it as the base class for QXL, CirrusVGA, QEMUVGA (std-vga), and VMwareVGA.

The VGA accessors should be exposed as a memory region but the sub class ought to be responsible for actually adding it to a subregion.

Quoted 7 lines

That twist makes it a bit hard to convert vga ...

Anyone knows how one would do that with the memory api instead? I think taking over the ports is easy as the memory regions have priorities so I can simply register a region with higher priority. I have no clue how to forward the access to the vga code though.

That should be possible with priorities, but I think it's wrong. There aren't two VGA devices. QXL is-a VGA device and the best way to override behavior of base VGA device is through polymorphism.

This isn't really a memory API issue, it's a modeling issue.

Regards,

Anthony Liguori

Quoted 4 lines

Anyone has clues / suggestions?

thanks,
Gerd

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 1 line

Make VGACommonState a proper QOM object and use it as the base class

for

Quoted 1 line

QXL, CirrusVGA, QEMUVGA (std-vga), and VMwareVGA.

I think QXL should have-a VGA rather than being one. It completely bypasses the VGA infrastructure if not in VGA mode.