qemu: make default_write_config use mask table
Subsystems (3)
Michael S. Tsirkin
maintainerRe: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Anthony LiguoriQuoted 36 lines · Anthony Liguori
Quoted 10 lines · Mark McLoughlin
Quoted 8 lines · Anthony Liguori
Quoted 5 lines
Eventually the default configuration becomes increasingly unusable and you need a new baseline. You must still be able to fall back to the old baseline for older guests. I don't think games with configuration files can hide that.
(quoted diff) +0
-2-M pc1 -M pc2Quoted 9 lines
Quoted 4 lines
etc.
This is pretty easy to maintain with config files.
I think this would be reasonable, but it is essentially just a version number which you objected to on the basis that it would make cherry-picking harder for distros.
It doesn't have to be pc1, pc2. It could be pc-with-usb or pc-with-balloon. If a distro cherry picks in such a way that their pc is not a standard QEMU pc, they would add a new PC type that's specific to their distro.
Quoted 5 lines
One thing that would be nice with this '-M pc1' thing would be to retain '-M pc' as a symlink to the latest version. We'd also need a way to read the symlink too, so that you can query what the current latest version is and use that in future.
Another option is an explicit -M default which always uses the default machine for the architecture. Likewise, we would need a way to query what the default machine was for an architecture.
Quoted 8 lines
Quoted 4 lines
How would this machine type version relate to e.g. changing the default PCI class of virtio-blk? Would we bump the version number of all machine types can use virtio-blk?
You would introduce a new machine type. For instance, pc-virtio-class-other. The names don't have to look like that, I'm just doing that to make a point. This may mean that you end up with dozens of machine types but you preserve compatibility, which is a good thing.
And then pc-virtio-class-other-with-balloon-without-usb? Wouldn't it be more straightforward to have capability bits which can be switched on and off independently rather than trying to fit unrelated features into a machine type? IMO it only seems more work at first, and QA gets a bit nervious that they can't exhaustively test all options. But in the long run it simplifies things as you don't have to set policy and invent silly names.
Quoted 8 lines
Of course, the flip side is that you make preserving the machine config the duty of the user and we don't maintain compatible machine types. This won't work without a proper config file though so for now, we're stuck maintaining machine types.
Regards,
Anthony Liguori
Re: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
Quoted 8 lines · Michael S. Tsirkin
And then pc-virtio-class-other-with-balloon-without-usb? Wouldn't it be more straightforward to have capability bits which can be switched on and off independently rather than trying to fit unrelated features into a machine type? IMO it only seems more work at first, and QA gets a bit nervious that they can't exhaustively test all options. But in the long run it simplifies things as you don't have to set policy and invent silly names.
We're strictly talking about default machine configs. That has nothing to do with capabilities. You still need to know what the default set of enabled capabilities were and keep track of that. All that I'm suggesting is that we use the machine name to collapse the default set of capabilities into something that libvirt can track.
The advantage of using something more opaque like that is that it simplifies things for management tools as they don't have to keep track of "capabilities" that we're adding. Heck, you could even do:
pc-00000034
Where "pc-%08x" % (capabilities) :-)
Regards,
Anthony Liguori
Re: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Michael S. TsirkinQuoted 8 lines · Michael S. Tsirkin
And then pc-virtio-class-other-with-balloon-without-usb? Wouldn't it be more straightforward to have capability bits which can be switched on and off independently rather than trying to fit unrelated features into a machine type? IMO it only seems more work at first, and QA gets a bit nervious that they can't exhaustively test all options. But in the long run it simplifies things as you don't have to set policy and invent silly names.
We're strictly talking about default machine configs. That has nothing to do with capabilities. You still need to know what the default set of enabled capabilities were and keep track of that. All that I'm suggesting is that we use the machine name to collapse the default set of capabilities into something that libvirt can track.
The advantage of using something more opaque like that is that it simplifies things for management tools as they don't have to keep track of "capabilities" that we're adding. Heck, you could even do:
pc-00000034
Where "pc-%08x" % (capabilities) :-)
Regards,
Anthony Liguori
Re: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Mark McLoughlinQuoted 11 lines · Mark McLoughlin
Quoted 9 lines · Anthony Liguori
Quoted 6 lines
Eventually the
default configuration becomes increasingly unusable and you need a new baseline. You must still be able to fall back to the old baseline for older guests. I don't think games with configuration files can hide that.(quoted diff) +0
-2-M pc1 -M pc2
Quoted 9 lines
Quoted 4 lines
etc.
This is pretty easy to maintain with config files.
I think this would be reasonable, but it is essentially just a version number which you objected to on the basis that it would make cherry-picking harder for distros.
It doesn't have to be pc1, pc2. It could be pc-with-usb or pc-with-balloon. If a distro cherry picks in such a way that their pc is not a standard QEMU pc, they would add a new PC type that's specific to their distro.
Quoted 5 lines
One thing that would be nice with this '-M pc1' thing would be to retain '-M pc' as a symlink to the latest version. We'd also need a way to read the symlink too, so that you can query what the current latest version is and use that in future.
Another option is an explicit -M default which always uses the default machine for the architecture. Likewise, we would need a way to query what the default machine was for an architecture.
Quoted 4 lines
How would this machine type version relate to e.g. changing the default PCI class of virtio-blk? Would we bump the version number of all machine types can use virtio-blk?
You would introduce a new machine type. For instance, pc-virtio-class-other. The names don't have to look like that, I'm just doing that to make a point. This may mean that you end up with dozens of machine types but you preserve compatibility, which is a good thing.
Of course, the flip side is that you make preserving the machine config the duty of the user and we don't maintain compatible machine types. This won't work without a proper config file though so for now, we're stuck maintaining machine types.
Regards,
Anthony Liguori
Michael S. Tsirkin
maintainerRe: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Mark McLoughlinQuoted 17 lines · Mark McLoughlin
Quoted 7 lines · Anthony Liguori
Quoted 6 lines · Mark McLoughlin
On Wed, 2009-06-10 at 20:27 +0100, Jamie Lokier wrote:
= Solution - Separate configuration from compat hints =
As I suggested before:
Quoted 8 lines
Quoted 3 lines
- Allow the VM manager to dump compat hints; this would be an opaque file format, more like the savevm format than a config file
How is compat hints different from a device tree?
In my mind, that's what compat hints is. I don't see another sane way to implement it.
Quoted 8 lines
A device tree with a different purpose than a config file.
In its simplest form it could be a device tree with a version number for each device[1].
The other obvious piece to add to it would be PCI addresses, so that even if you remove a device, the addresses assigned to existing devices don't change.
Could you clarify this requirement please?
If we want to remove a device from under a running guest, you need hotplug. So we can't just remove several lines from the config and hope that it'll work simply because the PCI address is stable.
OTOH, if you reboot the guest, it's ok for addresses to change.
Quoted 10 lines
Cheers,
Mark.[1] - Adding such a per-device version number to the config file would solve problem (2)
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Michael S. Tsirkin
maintainerRe: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Mark McLoughlinQuoted 17 lines · Mark McLoughlin
Quoted 7 lines · Anthony Liguori
Quoted 6 lines · Mark McLoughlin
On Wed, 2009-06-10 at 20:27 +0100, Jamie Lokier wrote:
= Solution - Separate configuration from compat hints =
As I suggested before:
Quoted 8 lines
Quoted 3 lines
- Allow the VM manager to dump compat hints; this would be an opaque file format, more like the savevm format than a config file
How is compat hints different from a device tree?
In my mind, that's what compat hints is. I don't see another sane way to implement it.
Quoted 8 lines
A device tree with a different purpose than a config file.
In its simplest form it could be a device tree with a version number for each device[1].
The other obvious piece to add to it would be PCI addresses, so that even if you remove a device, the addresses assigned to existing devices don't change.
Could you clarify this requirement please?
If we want to remove a device from under a running guest, you need hotplug. So we can't just remove several lines from the config and hope that it'll work simply because the PCI address is stable.
OTOH, if you reboot the guest, it's ok for addresses to change.
Quoted 10 lines
Cheers,
Mark.[1] - Adding such a per-device version number to the config file would solve problem (2)
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
Quoted 5 lines · Michael S. Tsirkin
If we want to remove a device from under a running guest, you need hotplug. So we can't just remove several lines from the config and hope that it'll work simply because the PCI address is stable.
Why not?
Quoted 2 lines
OTOH, if you reboot the guest, it's ok for addresses to change.
No, it's not. Some guests depend on addressing for their configuration (for example older Linux guests will swap eth0/eth1 if you swap their slots).
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Michael S. TsirkinQuoted 5 lines · Michael S. Tsirkin
If we want to remove a device from under a running guest, you need hotplug. So we can't just remove several lines from the config and hope that it'll work simply because the PCI address is stable.
Why not?
Quoted 2 lines
OTOH, if you reboot the guest, it's ok for addresses to change.
No, it's not. Some guests depend on addressing for their configuration (for example older Linux guests will swap eth0/eth1 if you swap their slots).
Signature · 1 line
error compiling committee.c: too many arguments to function
Michael S. Tsirkin
maintainerRe: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
Quoted 8 lines · Avi Kivity
Quoted 5 lines · Michael S. Tsirkin
If we want to remove a device from under a running guest, you need hotplug. So we can't just remove several lines from the config and hope that it'll work simply because the PCI address is stable.
Why not?
E.g. configuration cycles address a specific bus/slot. You need cooperation from guest if you want to move a device.
Quoted 6 lines
Quoted 2 lines
OTOH, if you reboot the guest, it's ok for addresses to change.
No, it's not. Some guests depend on addressing for their configuration (for example older Linux guests will swap eth0/eth1 if you swap their slots).
Ah, I misunderstood what's meant by the address. I agree that it's useful to be able to control device's placement on the bus.
Signature · 1 line
MST
Michael S. Tsirkin
maintainerRe: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Avi KivityQuoted 8 lines · Avi Kivity
Quoted 5 lines · Michael S. Tsirkin
If we want to remove a device from under a running guest, you need hotplug. So we can't just remove several lines from the config and hope that it'll work simply because the PCI address is stable.
Why not?
E.g. configuration cycles address a specific bus/slot. You need cooperation from guest if you want to move a device.
Quoted 6 lines
Quoted 2 lines
OTOH, if you reboot the guest, it's ok for addresses to change.
No, it's not. Some guests depend on addressing for their configuration (for example older Linux guests will swap eth0/eth1 if you swap their slots).
Ah, I misunderstood what's meant by the address. I agree that it's useful to be able to control device's placement on the bus.
Signature · 1 line
MST
Re: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Michael S. Tsirkin
Quoted 5 lines
If we want to remove a device from under a running guest, you need hotplug. So we can't just remove several lines from the config and hope that it'll work simply because the PCI address is stable.
Quoted 7 lines
Quoted 2 lines
Why not?
E.g. configuration cycles address a specific bus/slot. You need cooperation from guest if you want to move a device.
By "remove several lines from the config" I understood the guest needs to be restarted. Of course if you don't restart the guest you need true hotplug.
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Michael S. TsirkinQuoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Michael S. Tsirkin
Quoted 5 lines
If we want to remove a device from under a running guest, you need hotplug. So we can't just remove several lines from the config and hope that it'll work simply because the PCI address is stable.
Quoted 7 lines
Quoted 2 lines
Why not?
E.g. configuration cycles address a specific bus/slot. You need cooperation from guest if you want to move a device.
By "remove several lines from the config" I understood the guest needs to be restarted. Of course if you don't restart the guest you need true hotplug.
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Michael S. TsirkinQuoted 6 lines · Michael S. Tsirkin
Quoted 3 lines · Mark McLoughlin
The other obvious piece to add to it would be PCI addresses, so that even if you remove a device, the addresses assigned to existing devices don't change.
Could you clarify this requirement please?
Avi clarified, but I've written it up here too:
https://fedoraproject.org/wiki/Features/KVM_Stable_PCI_Addresses
Cheers,
Mark.
Re: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Michael S. TsirkinQuoted 6 lines · Michael S. Tsirkin
Quoted 3 lines · Mark McLoughlin
The other obvious piece to add to it would be PCI addresses, so that even if you remove a device, the addresses assigned to existing devices don't change.
Could you clarify this requirement please?
Avi clarified, but I've written it up here too:
https://fedoraproject.org/wiki/Features/KVM_Stable_PCI_Addresses
Cheers,
Mark.
Re: Configuration vs. compat hints [was Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities]
In reply to Mark McLoughlinQuoted 9 lines · Mark McLoughlin
On Wed, 2009-06-10 at 20:27 +0100, Jamie Lokier wrote:
= Solution - Separate configuration from compat hints =
As I suggested before:
- Allow the VM manager to dump compat hints; this would be an opaque file format, more like the savevm format than a config file
How is compat hints different from a device tree?
In my mind, that's what compat hints is. I don't see another sane way to implement it.
Regards,
Anthony Liguori
Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities
In reply to Michael S. TsirkinQuoted 70 lines · Michael S. Tsirkin
Quoted 52 lines · Glauber Costa
Quoted 17 lines · Michael S. Tsirkin
Add routines to manage PCI capability list. First user will be MSI-X.
Signed-off-by Michael S. Tsirkin <[email protected]>--- hw/pci.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------ hw/pci.h | 18 +++++++++++- 2 files changed, 106 insertions(+), 10 deletions(-)hw/pci.c +0
-1@@ -130,12 +130,13 @@ void pci_device_save(PCIDevice *s, QEMUFile *f)int version = s->cap_present ?3 :2 ;int i; -qemu_put_be32 (f, version);/* PCI device version */ hw/pci.c void pci_device_save(PCIDevice *s, QEMUFile *f) · line 140
Quoted 10 lines
hw/pci.c +4
-2+/* PCI device version and capabilities */ +qemu_put_be32 (f, version); +if (version >=3 ) +qemu_put_be32 (f, s->cap_present);qemu_put_buffer (f, s->config,256 );for (i =0 ; i <4 ; i++)qemu_put_be32 (f, s->irq_state[i]); -if (version >=3 ) -qemu_put_be32 (f, s->cap_present); }What is it doing here?
You should just do it right in the first patch, instead of doing in one way there, and fixing here.Quoted 5 lines
hw/pci.c +0
-0int pci_device_load (PCIDevice *s,QEMUFile *f)@@ -146,12 +147,6 @@ int pci_device_load(PCIDevice *s, QEMUFile *f) version_id =qemu_get_be32 (f);if (version_id >3 )hw/pci.c int pci_device_load(PCIDevice *s, QEMUFile *f) · line 152
Quoted 10 lines
hw/pci.c +0
-6return -EINVAL ; -qemu_get_buffer (f, s->config,256 ); -pci_update_mappings (s); - -if (version_id >=2 ) -for (i =0 ; i <4 ; i ++) - s->irq_state[i] =qemu_get_be32 (f);if (version_id >=3 ) s->cap_present =qemu_get_be32 (f);else ditto.
Quoted 4 lines
hw/pci.c +0
-0@@ -160,6 +155,18 @@ int pci_device_load(PCIDevice *s, QEMUFile *f)if (s->cap_present & ~s->cap_supported)return -EINVAL ;Quoted 16 lines
Quoted 4 lines
hw/pci.c +4
-0+qemu_get_buffer (f, s->config,256 ); +pci_update_mappings (s); + +if (version_id >=2 )hw/pci.c int pci_device_load(PCIDevice *s, QEMUFile *f) · line 171
Quoted 10 lines
hw/pci.c +8
-0+for (i =0 ; i <4 ; i ++) + s->irq_state[i] =qemu_get_be32 (f); +/* Clear wmask and used bits for capabilities. +Must be restored separately, since capabilities can +be placed anywhere in config space. */ +memset (s->used,0 ,PCI_CONFIG_SPACE_SIZE ); +for (i =PCI_CONFIG_HEADER_SIZE ; i <PCI_CONFIG_SPACE_SIZE ; ++i) + s->wmask[i] =0xff ;return 0 ; }Sorry, I don't exactly understand it. Although it can be anywhere, what do we actually lose by keeping it at the same place in config space?
Quoted 6 lines
We lose the ability to let user control the capabilities exposed by the device.
And generally, I dislike arbitrary limitations. The PCI spec says the capability can be anywhere, implementing a linked list of caps is simple enough to not invent abritrary restrictions.
yes, but this is migration time, right?
caps can be anywhere, but we don't expect it to change during machine execution lifetime.
Or I am just confused by the name "pci_device_load" ?
Re: [Qemu-devel] [PATCHv3 03/13] qemu: add routines to manage PCI capabilities
In reply to Michael S. TsirkinQuoted 17 lines · Michael S. Tsirkin
Add routines to manage PCI capability list. First user will be MSI-X.
Signed-off-by Michael S. Tsirkin <[email protected]>--- hw/pci.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------ hw/pci.h | 18 +++++++++++- 2 files changed, 106 insertions(+), 10 deletions(-)hw/pci.c +0
-1@@ -130,12 +130,13 @@ void pci_device_save(PCIDevice *s, QEMUFile *f)int version = s->cap_present ?3 :2 ;int i; -qemu_put_be32 (f, version);/* PCI device version */
hw/pci.c void pci_device_save(PCIDevice *s, QEMUFile *f) · line 140
Quoted 10 lines
hw/pci.c +4
-2+/* PCI device version and capabilities */ +qemu_put_be32 (f, version); +if (version >=3 ) +qemu_put_be32 (f, s->cap_present);qemu_put_buffer (f, s->config,256 );for (i =0 ; i <4 ; i++)qemu_put_be32 (f, s->irq_state[i]); -if (version >=3 ) -qemu_put_be32 (f, s->cap_present); }
What is it doing here?
You should just do it right in the first patch, instead of doing in one way there, and fixing here.
Quoted 5 lines
hw/pci.c +0
-0int pci_device_load (PCIDevice *s,QEMUFile *f)@@ -146,12 +147,6 @@ int pci_device_load(PCIDevice *s, QEMUFile *f) version_id =qemu_get_be32 (f);if (version_id >3 )
hw/pci.c int pci_device_load(PCIDevice *s, QEMUFile *f) · line 152
Quoted 10 lines
hw/pci.c +0
-6return -EINVAL ; -qemu_get_buffer (f, s->config,256 ); -pci_update_mappings (s); - -if (version_id >=2 ) -for (i =0 ; i <4 ; i ++) - s->irq_state[i] =qemu_get_be32 (f);if (version_id >=3 ) s->cap_present =qemu_get_be32 (f);else
ditto.
Quoted 8 lines
hw/pci.c +4
-0@@ -160,6 +155,18 @@ int pci_device_load(PCIDevice *s, QEMUFile *f)if (s->cap_present & ~s->cap_supported)return -EINVAL ; +qemu_get_buffer (f, s->config,256 ); +pci_update_mappings (s); + +if (version_id >=2 )
hw/pci.c int pci_device_load(PCIDevice *s, QEMUFile *f) · line 171
Quoted 10 lines
hw/pci.c +8
-0+for (i =0 ; i <4 ; i ++) + s->irq_state[i] =qemu_get_be32 (f); +/* Clear wmask and used bits for capabilities. +Must be restored separately, since capabilities can +be placed anywhere in config space. */ +memset (s->used,0 ,PCI_CONFIG_SPACE_SIZE ); +for (i =PCI_CONFIG_HEADER_SIZE ; i <PCI_CONFIG_SPACE_SIZE ; ++i) + s->wmask[i] =0xff ;return 0 ; }
Sorry, I don't exactly understand it. Although it can be anywhere, what do we actually lose by keeping it at the same place in config space?
Quoted 24 lines
hw/pci.c +19
-0@@ -870,3 +877,76 @@ PCIDevice *pci_create_simple(PCIBus *bus, int devfn, const char *name)return (PCIDevice *)dev; } + +static int pci_find_space (PCIDevice *pdev,uint8_t size) +{ +int offset =PCI_CONFIG_HEADER_SIZE ; +int i; +for (i =PCI_CONFIG_HEADER_SIZE ; i <PCI_CONFIG_SPACE_SIZE ; ++i) +if (pdev->used[i]) + offset = i +1 ; +else if (i - offset +1 == size) +return offset; +return 0 ; +} + +static uint8_t pci_find_capability_list (PCIDevice *pdev,uint8_t cap_id, +uint8_t *prev_p) +{ +uint8_t next, prev; + +if (!(pdev->config[PCI_STATUS ] &PCI_STATUS_CAP_LIST ))
hw/pci.c PCIDevice *pci_create_simple(PCIBus *bus, int devfn, const char *name) · line 908
Quoted 10 lines
hw/pci.c +10
-0+return 0 ; + +for (prev =PCI_CAPABILITY_LIST ; (next = pdev->config[prev]); + prev = next +PCI_CAP_LIST_NEXT ) +if (pdev->config[next +PCI_CAP_LIST_ID ] == cap_id) +break ; + + *prev_p = prev; +return next; +}
I'd prefer to do:
if (prev_p) *prev_p = prev;
so we don't have to always pass a prev_p pointer. You have yourself a user where you don't need it in this very patch.
Quoted 99 lines
hw/pci.c +44
-0+ +/* Reserve space and add capability to the linked list in pci config space */ +int pci_add_capability (PCIDevice *pdev,uint8_t cap_id,uint8_t size) +{ +uint8_t offset =pci_find_space (pdev, size); +uint8_t *config = pdev->config + offset; +if (!offset) +return -ENOSPC ; + config[PCI_CAP_LIST_ID ] = cap_id; + config[PCI_CAP_LIST_NEXT ] = pdev->config[PCI_CAPABILITY_LIST ]; + pdev->config[PCI_CAPABILITY_LIST ] = offset; + pdev->config[PCI_STATUS ] |=PCI_STATUS_CAP_LIST ; +memset (pdev->used + offset,0xFF , size); +/* Make capability read-only by default */ +memset (pdev->wmask + offset,0 , size); +return offset; +} + +/* Unlink capability from the pci config space. */ +void pci_del_capability (PCIDevice *pdev,uint8_t cap_id,uint8_t size) +{ +uint8_t prev, offset =pci_find_capability_list (pdev, cap_id, &prev); +if (!offset) +return ; + pdev->config[prev] = pdev->config[offset +PCI_CAP_LIST_NEXT ]; +/* Make capability writeable again */ +memset (pdev->wmask + offset,0xff , size); +memset (pdev->used + offset,0 , size); + +if (!pdev->config[PCI_CAPABILITY_LIST ]) + pdev->config[PCI_STATUS ] &= ~PCI_STATUS_CAP_LIST ; +} + +/* Reserve space for capability at a known offset (to call after load). */ +void pci_reserve_capability (PCIDevice *pdev,uint8_t offset,uint8_t size) +{ +memset (pdev->used + offset,0xff , size); +} + +uint8_t pci_find_capability (PCIDevice *pdev,uint8_t cap_id) +{ +uint8_t prev; +return pci_find_capability_list (pdev, cap_id, &prev); +}hw/pci.h +17
-1@@ -123,6 +123,10 @@ typedef struct PCIIORegion {#define PCI_MIN_GNT 0x3e/* 8 bits */ #define PCI_MAX_LAT 0x3f/* 8 bits */ +/* Capability lists */ +#define PCI_CAP_LIST_ID 0/* Capability ID */ +#define PCI_CAP_LIST_NEXT 1/* Next capability in the list */ +#define PCI_REVISION 0x08/* obsolete, use PCI_REVISION_ID */ #define PCI_SUBVENDOR_ID 0x2c/* obsolete, use PCI_SUBSYSTEM_VENDOR_ID */ #define PCI_SUBDEVICE_ID 0x2e/* obsolete, use PCI_SUBSYSTEM_ID */ @@ -130,7 +134,7 @@ typedef struct PCIIORegion {/* Bits in the PCI Status Register (PCI 2.3 spec) */ #define PCI_STATUS_RESERVED1 0x007#define PCI_STATUS_INT_STATUS 0x008 -#define PCI_STATUS_CAPABILITIES 0x010 +#define PCI_STATUS_CAP_LIST 0x010#define PCI_STATUS_66MHZ 0x020#define PCI_STATUS_RESERVED2 0x040#define PCI_STATUS_FAST_BACK 0x080@@ -160,6 +164,9 @@ struct PCIDevice {/* Used to implement R/W bytes */ uint8_t wmask[PCI_CONFIG_SPACE_SIZE ]; +/* Used to allocate config space for capabilities. */ +uint8_t used[PCI_CONFIG_SPACE_SIZE ]; +/* the following fields are read only */ PCIBus *bus;int devfn;@@ -194,6 +201,15 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, uint32_t size, int type, PCIMapIORegionFunc *map_func); +int pci_add_capability (PCIDevice *pci_dev,uint8_t cap_id,uint8_t cap_size); + +void pci_del_capability (PCIDevice *pci_dev,uint8_t cap_id,uint8_t cap_size); + +void pci_reserve_capability (PCIDevice *pci_dev,uint8_t offset,uint8_t size); + +uint8_t pci_find_capability (PCIDevice *pci_dev,uint8_t cap_id); + +uint32_t pci_default_read_config (PCIDevice *d,uint32_t address,int len); void pci_default_write_config(PCIDevice *d,Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 4/13 qemu: helper routines for pci access.
In reply to a message not in the archiveAdd inline routines for convenient access to pci devices with correct (little) endianness. Will be used by MSI-X support.
--- hw/pci.h | 30 +++++++++++++++++++++++++++--- 1 files changed, 27 insertions(+), 3 deletions(-)
hw/pci.h +27 -3
@@ -240,21 +240,45 @@ PCIBus *pci_bridge_init(PCIBus *bus, int devfn, uint16_t vid, uint16_t did, pci_map_irq_fn map_irq, const char *name);static inline void +pci_set_word (uint8_t *config,uint16_t val) +{ +cpu_to_le16wu ((uint16_t *)config, val); +} + +static inline uint16_t +pci_get_word (uint8_t *config) +{ +return le16_to_cpupu ((uint16_t *)config); +} + +static inline void +pci_set_long (uint8_t *config,uint32_t val) +{ +cpu_to_le32wu ((uint32_t *)config, val); +} + +static inline uint32_t +pci_get_long (uint8_t *config) +{ +return le32_to_cpupu ((uint32_t *)config); +} + +static inline void pci_config_set_vendor_id (uint8_t *pci_config,uint16_t val) { -cpu_to_le16wu ((uint16_t *)&pci_config[PCI_VENDOR_ID ], val); +pci_set_word (&pci_config[PCI_VENDOR_ID ], val); }static inline void pci_config_set_device_id (uint8_t *pci_config,uint16_t val) { -cpu_to_le16wu ((uint16_t *)&pci_config[PCI_DEVICE_ID ], val); +pci_set_word (&pci_config[PCI_DEVICE_ID ], val); }static inline void pci_config_set_class (uint8_t *pci_config,uint16_t val) { -cpu_to_le16wu ((uint16_t *)&pci_config[PCI_CLASS_DEVICE ], val); +pci_set_word (&pci_config[PCI_CLASS_DEVICE ], val); }typedef void (*pci_qdev_initfn )(PCIDevice *dev);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 4/13 qemu: helper routines for pci access.
In reply to a message not in the archiveAdd inline routines for convenient access to pci devices with correct (little) endianness. Will be used by MSI-X support.
--- hw/pci.h | 30 +++++++++++++++++++++++++++--- 1 files changed, 27 insertions(+), 3 deletions(-)
hw/pci.h +27 -3
@@ -240,21 +240,45 @@ PCIBus *pci_bridge_init(PCIBus *bus, int devfn, uint16_t vid, uint16_t did, pci_map_irq_fn map_irq, const char *name);static inline void +pci_set_word (uint8_t *config,uint16_t val) +{ +cpu_to_le16wu ((uint16_t *)config, val); +} + +static inline uint16_t +pci_get_word (uint8_t *config) +{ +return le16_to_cpupu ((uint16_t *)config); +} + +static inline void +pci_set_long (uint8_t *config,uint32_t val) +{ +cpu_to_le32wu ((uint32_t *)config, val); +} + +static inline uint32_t +pci_get_long (uint8_t *config) +{ +return le32_to_cpupu ((uint32_t *)config); +} + +static inline void pci_config_set_vendor_id (uint8_t *pci_config,uint16_t val) { -cpu_to_le16wu ((uint16_t *)&pci_config[PCI_VENDOR_ID ], val); +pci_set_word (&pci_config[PCI_VENDOR_ID ], val); }static inline void pci_config_set_device_id (uint8_t *pci_config,uint16_t val) { -cpu_to_le16wu ((uint16_t *)&pci_config[PCI_DEVICE_ID ], val); +pci_set_word (&pci_config[PCI_DEVICE_ID ], val); }static inline void pci_config_set_class (uint8_t *pci_config,uint16_t val) { -cpu_to_le16wu ((uint16_t *)&pci_config[PCI_CLASS_DEVICE ], val); +pci_set_word (&pci_config[PCI_CLASS_DEVICE ], val); }typedef void (*pci_qdev_initfn )(PCIDevice *dev);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 5/13 qemu: MSI-X support functions
In reply to a message not in the archiveAdd functions implementing MSI-X support. First user will be virtio-pci. Note that platform must set a flag to declare MSI supported. For PC this will be set by APIC.
--- Makefile.target | 2 +- hw/msix.c | 423 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/msix.h | 35 +++++ hw/pci.h | 20 +++ 4 files changed, 479 insertions(+), 1 deletions(-) create mode 100644 hw/msix.c create mode 100644 hw/msix.h
Makefile.target +1 -1
@@ -486,7 +486,7 @@ endif #CONFIG_BSD_USER ifndef CONFIG_USER_ONLY OBJS=vl.o osdep.o monitor.o pci.o loader.o isa_mmio.o machine.o \ - gdbstub.o gdbstub-xml.o + gdbstub.o gdbstub-xml.o msix.o # virtio has to be here due to weird dependency between PCI and virtio-net. # need to fix this properly OBJS+=virtio-blk.o virtio-balloon.o virtio-net.o virtio-console.o
hw/msix.c new file +423 -0
423 lines not on this page · raw message
hw/msix.h new file +35 -0
@@ -0,0 +1,35 @@ +#ifndef QEMU_MSIX_H +#define QEMU_MSIX_H + +#include "qemu-common.h" + +int msix_init (PCIDevice *pdev,unsigned short nentries, +unsigned bar_nr,unsigned bar_size); + +void msix_write_config (PCIDevice *pci_dev,uint32_t address, +uint32_t val,int len); + +void msix_mmio_map (PCIDevice *pci_dev,int region_num, +uint32_t addr,uint32_t size,int type); + +int msix_uninit (PCIDevice *d); + +void msix_save (PCIDevice *dev,QEMUFile *f); +int msix_load (PCIDevice *dev,QEMUFile *f); + +int msix_enabled (PCIDevice *dev); +int msix_present (PCIDevice *dev); + +uint32_t msix_bar_size (PCIDevice *dev); + +int msix_vector_use (PCIDevice *dev,unsigned vector); +void msix_vector_unuse (PCIDevice *dev,unsigned vector); + +void msix_notify (PCIDevice *dev,unsigned vector); + +void msix_reset (PCIDevice *dev); + +extern int msix_disable; +extern int msix_supported; + +#endif
hw/pci.h +20 -0
@@ -156,6 +156,11 @@ typedef struct PCIIORegion {/* Size of the standard PCI config space */ #define PCI_CONFIG_SPACE_SIZE 0x100 +/* Bits in cap_supported/cap_present fields. */ +enum { +QEMU_PCI_CAP_MSIX =0x1 , +}; +struct PCIDevice {DeviceState qdev;/* PCI config space */
@@ -189,6 +194,21 @@ struct PCIDevice {/* Capability bits for save/load */ uint32_t cap_supported;uint32_t cap_present; + +/* Offset of MSI-X capability in config space */ +uint8_t msix_cap; + +/* MSI-X entries */ +int msix_entries_nr; + +/* Space to store MSIX table */ +uint8_t *msix_table_page; +/* MMIO index used to map MSIX table and pending bit entries. */ +int msix_mmio_index; +/* Reference-count for entries actually in use by driver. */ +unsigned *msix_entry_used; +/* Region including the MSI-X table */ +uint32_t msix_bar_size; }; PCIDevice *pci_register_device(PCIBus *bus, const char *name,
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 5/13 qemu: MSI-X support functions
In reply to a message not in the archiveAdd functions implementing MSI-X support. First user will be virtio-pci. Note that platform must set a flag to declare MSI supported. For PC this will be set by APIC.
--- Makefile.target | 2 +- hw/msix.c | 423 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/msix.h | 35 +++++ hw/pci.h | 20 +++ 4 files changed, 479 insertions(+), 1 deletions(-) create mode 100644 hw/msix.c create mode 100644 hw/msix.h
Makefile.target +1 -1
@@ -486,7 +486,7 @@ endif #CONFIG_BSD_USER ifndef CONFIG_USER_ONLY OBJS=vl.o osdep.o monitor.o pci.o loader.o isa_mmio.o machine.o \ - gdbstub.o gdbstub-xml.o + gdbstub.o gdbstub-xml.o msix.o # virtio has to be here due to weird dependency between PCI and virtio-net. # need to fix this properly OBJS+=virtio-blk.o virtio-balloon.o virtio-net.o virtio-console.o
hw/msix.c new file +423 -0
423 lines not on this page · raw message
hw/msix.h new file +35 -0
@@ -0,0 +1,35 @@ +#ifndef QEMU_MSIX_H +#define QEMU_MSIX_H + +#include "qemu-common.h" + +int msix_init (PCIDevice *pdev,unsigned short nentries, +unsigned bar_nr,unsigned bar_size); + +void msix_write_config (PCIDevice *pci_dev,uint32_t address, +uint32_t val,int len); + +void msix_mmio_map (PCIDevice *pci_dev,int region_num, +uint32_t addr,uint32_t size,int type); + +int msix_uninit (PCIDevice *d); + +void msix_save (PCIDevice *dev,QEMUFile *f); +int msix_load (PCIDevice *dev,QEMUFile *f); + +int msix_enabled (PCIDevice *dev); +int msix_present (PCIDevice *dev); + +uint32_t msix_bar_size (PCIDevice *dev); + +int msix_vector_use (PCIDevice *dev,unsigned vector); +void msix_vector_unuse (PCIDevice *dev,unsigned vector); + +void msix_notify (PCIDevice *dev,unsigned vector); + +void msix_reset (PCIDevice *dev); + +extern int msix_disable; +extern int msix_supported; + +#endif
hw/pci.h +20 -0
@@ -156,6 +156,11 @@ typedef struct PCIIORegion {/* Size of the standard PCI config space */ #define PCI_CONFIG_SPACE_SIZE 0x100 +/* Bits in cap_supported/cap_present fields. */ +enum { +QEMU_PCI_CAP_MSIX =0x1 , +}; +struct PCIDevice {DeviceState qdev;/* PCI config space */
@@ -189,6 +194,21 @@ struct PCIDevice {/* Capability bits for save/load */ uint32_t cap_supported;uint32_t cap_present; + +/* Offset of MSI-X capability in config space */ +uint8_t msix_cap; + +/* MSI-X entries */ +int msix_entries_nr; + +/* Space to store MSIX table */ +uint8_t *msix_table_page; +/* MMIO index used to map MSIX table and pending bit entries. */ +int msix_mmio_index; +/* Reference-count for entries actually in use by driver. */ +unsigned *msix_entry_used; +/* Region including the MSI-X table */ +uint32_t msix_bar_size; }; PCIDevice *pci_register_device(PCIBus *bus, const char *name,
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Re: [Qemu-devel] [PATCHv3 05/13] qemu: MSI-X support functions
Quoted 89 lines · Michael S. Tsirkin
Add functions implementing MSI-X support. First user will be virtio-pci. Note that platform must set a flag to declare MSI supported. For PC this will be set by APIC.
Signed-off-by Michael S. Tsirkin <[email protected]>--- Makefile.target | 2 +- hw/msix.c | 423 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/msix.h | 35 +++++ hw/pci.h | 20 +++ 4 files changed, 479 insertions(+), 1 deletions(-) create mode 100644 hw/msix.c create mode 100644 hw/msix.hMakefile.target +1
-1@@ -486,7 +486,7 @@ endif #CONFIG_BSD_USER ifndef CONFIG_USER_ONLY OBJS=vl.o osdep.o monitor.o pci.o loader.o isa_mmio.o machine.o \ - gdbstub.o gdbstub-xml.o + gdbstub.o gdbstub-xml.o msix.o # virtio has to be here due to weird dependency between PCI and virtio-net. # need to fix this properly OBJS+=virtio-blk.o virtio-balloon.o virtio-net.o virtio-console.ohw/msix.c new file +56
-0@@ -0,0 +1,423 @@ +/* +* MSI-X device support +* +* This module includes support for MSI-X in pci devices. +* +* Author: Michael S. Tsirkin <[email protected]> +* +* Copyright (c) 2009, Red Hat Inc, Michael S. Tsirkin ([email protected]) +* +* This work is licensed under the terms of the GNU GPL, version 2. See +* the COPYING file in the top-level directory. +*/ + +#include "hw.h" +#include "msix.h" +#include "pci.h" + +/* Declaration from linux/pci_regs.h */ +#define PCI_CAP_ID_MSIX 0x11/* MSI-X */ +#define PCI_MSIX_FLAGS 2/* Table at lower 11 bits */ +#define PCI_MSIX_FLAGS_QSIZE 0x7FF +#define PCI_MSIX_FLAGS_ENABLE (1 << 15) +#define PCI_MSIX_FLAGS_BIRMASK (7 << 0) + +/* MSI-X capability structure */ +#define MSIX_TABLE_OFFSET 4 +#define MSIX_PBA_OFFSET 8 +#define MSIX_CAP_LENGTH 12 + +/* MSI enable bit is in byte 1 in FLAGS register */ +#define MSIX_ENABLE_OFFSET (PCI_MSIX_FLAGS + 1) +#define MSIX_ENABLE_MASK (PCI_MSIX_FLAGS_ENABLE >> 8) + +/* MSI-X table format */ +#define MSIX_MSG_ADDR 0 +#define MSIX_MSG_UPPER_ADDR 4 +#define MSIX_MSG_DATA 8 +#define MSIX_VECTOR_CTRL 12 +#define MSIX_ENTRY_SIZE 16 +#define MSIX_VECTOR_MASK 0x1 + +/* How much space does an MSIX table need. */ +/* The spec requires giving the table structure +* a 4K aligned region all by itself. Align it to +* target pages so that drivers can do passthrough +* on the rest of the region. */ +#define MSIX_PAGE_SIZE TARGET_PAGE_ALIGN(0x1000) +/* Reserve second half of the page for pending bits */ +#define MSIX_PAGE_PENDING (MSIX_PAGE_SIZE / 2) +#define MSIX_MAX_ENTRIES 32 + + +#ifdef MSIX_DEBUG +#define DEBUG (fmt, ...) \ + do { \ + fprintf(stderr, "%s: " fmt, __func__ , __VA_ARGS__); \
hw/msix.c line 66
Quoted 10 lines
hw/msix.c new file +10
-0+ } while (0) +#else +#define DEBUG (fmt, ...) do { } while(0) +#endif + +/* Flag to globally disable MSI-X support */ +int msix_disable; + +/* Flag for interrupt controller to declare MSI-X support */ +int msix_supported;
maybe better to make it static, and provide msi_state() returning -1 for disabled, 0 for supported, etc...
Quoted 54 lines
hw/msix.c new file +53
-0+ +/* Add MSI-X capability to the config space for the device. */ +/* Given a bar and its size, add MSI-X table on top of it +* and fill MSI-X capability in the config space. +* Original bar size must be a power of 2 or 0. +* New bar size is returned. */ +static int msix_add_config (struct PCIDevice *pdev,unsigned short nentries, +unsigned bar_nr,unsigned bar_size) +{ +int config_offset; +uint8_t *config; +uint32_t new_size; + +if (nentries <1 || nentries >PCI_MSIX_FLAGS_QSIZE +1 ) +return -EINVAL ; +if (bar_size >0x80000000 ) +return -ENOSPC ; + +/* Add space for MSI-X structures */ +if (!bar_size) + new_size =MSIX_PAGE_SIZE ; +else if (bar_size <MSIX_PAGE_SIZE ) { + bar_size =MSIX_PAGE_SIZE ; + new_size =MSIX_PAGE_SIZE *2 ; + }else + new_size = bar_size *2 ; + + pdev->msix_bar_size = new_size; + config_offset =pci_add_capability (pdev,PCI_CAP_ID_MSIX ,MSIX_CAP_LENGTH ); +if (config_offset <0 ) +return config_offset; + config = pdev->config + config_offset; + +pci_set_word (config +PCI_MSIX_FLAGS , nentries -1 ); +/* Table on top of BAR */ +pci_set_long (config +MSIX_TABLE_OFFSET , bar_size | bar_nr); +/* Pending bits on top of that */ +pci_set_long (config +MSIX_PBA_OFFSET , (bar_size +MSIX_PAGE_PENDING ) | + bar_nr); + pdev->msix_cap = config_offset; +/* Make flags bit writeable. */ + pdev->wmask[config_offset +MSIX_ENABLE_OFFSET ] |=MSIX_ENABLE_MASK ; +return 0 ; +} + + +/* Initialize the MSI-X structures. Note: if MSI-X is supported, BAR size is +* modified, it should be retrieved with msix_bar_size. */ +int msix_init (struct PCIDevice *dev,unsigned short nentries, +unsigned bar_nr,unsigned bar_size) +{ +int ret = -ENOMEM ; +/* Nothing to do if MSI is not supported by interrupt controller */
hw/msix.c
Quoted 10 lines
hw/msix.c new file +10
-0+if (!msix_supported) +return -ENOTTY ; + +if (nentries >MSIX_MAX_ENTRIES ) +return -EINVAL ; + + dev->msix_entry_used =qemu_mallocz (MSIX_MAX_ENTRIES * +sizeof *dev->msix_entry_used); +if (!dev->msix_entry_used) + goto err_used;
no need to check. oom_checker will kill qemu if it fails.
hw/msix.c line 247
Quoted 4 lines
hw/msix.c new file +4
-0+ + dev->msix_table_page =qemu_mallocz (MSIX_PAGE_SIZE ); +if (!dev->msix_table_page) + goto err_page;
ditto.
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 05/13] qemu: MSI-X support functions
Quoted 93 lines · Glauber Costa
Quoted 92 lines · Michael S. Tsirkin
Add functions implementing MSI-X support. First user will be virtio-pci. Note that platform must set a flag to declare MSI supported. For PC this will be set by APIC.
Signed-off-by Michael S. Tsirkin <[email protected]>--- Makefile.target | 2 +- hw/msix.c | 423 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/msix.h | 35 +++++ hw/pci.h | 20 +++ 4 files changed, 479 insertions(+), 1 deletions(-) create mode 100644 hw/msix.c create mode 100644 hw/msix.hMakefile.target +1
-1@@ -486,7 +486,7 @@ endif #CONFIG_BSD_USER ifndef CONFIG_USER_ONLY OBJS=vl.o osdep.o monitor.o pci.o loader.o isa_mmio.o machine.o \ - gdbstub.o gdbstub-xml.o + gdbstub.o gdbstub-xml.o msix.o # virtio has to be here due to weird dependency between PCI and virtio-net. # need to fix this properly OBJS+=virtio-blk.o virtio-balloon.o virtio-net.o virtio-console.ohw/msix.c new file +59
-0@@ -0,0 +1,423 @@ +/* +* MSI-X device support +* +* This module includes support for MSI-X in pci devices. +* +* Author: Michael S. Tsirkin <[email protected]> +* +* Copyright (c) 2009, Red Hat Inc, Michael S. Tsirkin ([email protected]) +* +* This work is licensed under the terms of the GNU GPL, version 2. See +* the COPYING file in the top-level directory. +*/ + +#include "hw.h" +#include "msix.h" +#include "pci.h" + +/* Declaration from linux/pci_regs.h */ +#define PCI_CAP_ID_MSIX 0x11/* MSI-X */ +#define PCI_MSIX_FLAGS 2/* Table at lower 11 bits */ +#define PCI_MSIX_FLAGS_QSIZE 0x7FF +#define PCI_MSIX_FLAGS_ENABLE (1 << 15) +#define PCI_MSIX_FLAGS_BIRMASK (7 << 0) + +/* MSI-X capability structure */ +#define MSIX_TABLE_OFFSET 4 +#define MSIX_PBA_OFFSET 8 +#define MSIX_CAP_LENGTH 12 + +/* MSI enable bit is in byte 1 in FLAGS register */ +#define MSIX_ENABLE_OFFSET (PCI_MSIX_FLAGS + 1) +#define MSIX_ENABLE_MASK (PCI_MSIX_FLAGS_ENABLE >> 8) + +/* MSI-X table format */ +#define MSIX_MSG_ADDR 0 +#define MSIX_MSG_UPPER_ADDR 4 +#define MSIX_MSG_DATA 8 +#define MSIX_VECTOR_CTRL 12 +#define MSIX_ENTRY_SIZE 16 +#define MSIX_VECTOR_MASK 0x1 + +/* How much space does an MSIX table need. */ +/* The spec requires giving the table structure +* a 4K aligned region all by itself. Align it to +* target pages so that drivers can do passthrough +* on the rest of the region. */ +#define MSIX_PAGE_SIZE TARGET_PAGE_ALIGN(0x1000) +/* Reserve second half of the page for pending bits */ +#define MSIX_PAGE_PENDING (MSIX_PAGE_SIZE / 2) +#define MSIX_MAX_ENTRIES 32 + + +#ifdef MSIX_DEBUG +#define DEBUG (fmt, ...) \ + do { \ + fprintf(stderr, "%s: " fmt, __func__ , __VA_ARGS__); \ + } while (0) +#else +#define DEBUG (fmt, ...) do { } while(0)
Quoted 8 lines
hw/msix.c line 66
Quoted 7 lines
hw/msix.c new file +7
-0+#endif + +/* Flag to globally disable MSI-X support */ +int msix_disable; + +/* Flag for interrupt controller to declare MSI-X support */ +int msix_supported;maybe better to make it static,
It's not read-only either.
Quoted 2 lines
and provide
msi_state()returning -1 for disabled, 0 for supported, etc...
Matter of taste, I prefer a set of binary flags rather than yet another enum:msix_disable is controlled by user, msix_supported is a safety valve for non-PC platforms. It's easier to keep them separate IMO.
Quoted 66 lines
Quoted 54 lines
hw/msix.c new file +53
-0+ +/* Add MSI-X capability to the config space for the device. */ +/* Given a bar and its size, add MSI-X table on top of it +* and fill MSI-X capability in the config space. +* Original bar size must be a power of 2 or 0. +* New bar size is returned. */ +static int msix_add_config (struct PCIDevice *pdev,unsigned short nentries, +unsigned bar_nr,unsigned bar_size) +{ +int config_offset; +uint8_t *config; +uint32_t new_size; + +if (nentries <1 || nentries >PCI_MSIX_FLAGS_QSIZE +1 ) +return -EINVAL ; +if (bar_size >0x80000000 ) +return -ENOSPC ; + +/* Add space for MSI-X structures */ +if (!bar_size) + new_size =MSIX_PAGE_SIZE ; +else if (bar_size <MSIX_PAGE_SIZE ) { + bar_size =MSIX_PAGE_SIZE ; + new_size =MSIX_PAGE_SIZE *2 ; + }else + new_size = bar_size *2 ; + + pdev->msix_bar_size = new_size; + config_offset =pci_add_capability (pdev,PCI_CAP_ID_MSIX ,MSIX_CAP_LENGTH ); +if (config_offset <0 ) +return config_offset; + config = pdev->config + config_offset; + +pci_set_word (config +PCI_MSIX_FLAGS , nentries -1 ); +/* Table on top of BAR */ +pci_set_long (config +MSIX_TABLE_OFFSET , bar_size | bar_nr); +/* Pending bits on top of that */ +pci_set_long (config +MSIX_PBA_OFFSET , (bar_size +MSIX_PAGE_PENDING ) | + bar_nr); + pdev->msix_cap = config_offset; +/* Make flags bit writeable. */ + pdev->wmask[config_offset +MSIX_ENABLE_OFFSET ] |=MSIX_ENABLE_MASK ; +return 0 ; +} + + +/* Initialize the MSI-X structures. Note: if MSI-X is supported, BAR size is +* modified, it should be retrieved with msix_bar_size. */ +int msix_init (struct PCIDevice *dev,unsigned short nentries, +unsigned bar_nr,unsigned bar_size) +{ +int ret = -ENOMEM ; +/* Nothing to do if MSI is not supported by interrupt controller */ hw/msix.c
Quoted 10 lines
hw/msix.c new file +10
-0+if (!msix_supported) +return -ENOTTY ; + +if (nentries >MSIX_MAX_ENTRIES ) +return -EINVAL ; + + dev->msix_entry_used =qemu_mallocz (MSIX_MAX_ENTRIES * +sizeof *dev->msix_entry_used); +if (!dev->msix_entry_used) + goto err_used;no need to check.
oom_checkerwill kill qemu if it fails.
Quoted 6 lines
hw/msix.c line 247
Quoted 4 lines
hw/msix.c new file +4
-0+ + dev->msix_table_page =qemu_mallocz (MSIX_PAGE_SIZE ); +if (!dev->msix_table_page) + goto err_page;ditto.
Good point.
Signature · 1 line
MST
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 05/13] qemu: MSI-X support functions
In reply to Glauber CostaQuoted 93 lines · Glauber Costa
Quoted 92 lines · Michael S. Tsirkin
Add functions implementing MSI-X support. First user will be virtio-pci. Note that platform must set a flag to declare MSI supported. For PC this will be set by APIC.
Signed-off-by Michael S. Tsirkin <[email protected]>--- Makefile.target | 2 +- hw/msix.c | 423 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/msix.h | 35 +++++ hw/pci.h | 20 +++ 4 files changed, 479 insertions(+), 1 deletions(-) create mode 100644 hw/msix.c create mode 100644 hw/msix.hMakefile.target +1
-1@@ -486,7 +486,7 @@ endif #CONFIG_BSD_USER ifndef CONFIG_USER_ONLY OBJS=vl.o osdep.o monitor.o pci.o loader.o isa_mmio.o machine.o \ - gdbstub.o gdbstub-xml.o + gdbstub.o gdbstub-xml.o msix.o # virtio has to be here due to weird dependency between PCI and virtio-net. # need to fix this properly OBJS+=virtio-blk.o virtio-balloon.o virtio-net.o virtio-console.ohw/msix.c new file +59
-0@@ -0,0 +1,423 @@ +/* +* MSI-X device support +* +* This module includes support for MSI-X in pci devices. +* +* Author: Michael S. Tsirkin <[email protected]> +* +* Copyright (c) 2009, Red Hat Inc, Michael S. Tsirkin ([email protected]) +* +* This work is licensed under the terms of the GNU GPL, version 2. See +* the COPYING file in the top-level directory. +*/ + +#include "hw.h" +#include "msix.h" +#include "pci.h" + +/* Declaration from linux/pci_regs.h */ +#define PCI_CAP_ID_MSIX 0x11/* MSI-X */ +#define PCI_MSIX_FLAGS 2/* Table at lower 11 bits */ +#define PCI_MSIX_FLAGS_QSIZE 0x7FF +#define PCI_MSIX_FLAGS_ENABLE (1 << 15) +#define PCI_MSIX_FLAGS_BIRMASK (7 << 0) + +/* MSI-X capability structure */ +#define MSIX_TABLE_OFFSET 4 +#define MSIX_PBA_OFFSET 8 +#define MSIX_CAP_LENGTH 12 + +/* MSI enable bit is in byte 1 in FLAGS register */ +#define MSIX_ENABLE_OFFSET (PCI_MSIX_FLAGS + 1) +#define MSIX_ENABLE_MASK (PCI_MSIX_FLAGS_ENABLE >> 8) + +/* MSI-X table format */ +#define MSIX_MSG_ADDR 0 +#define MSIX_MSG_UPPER_ADDR 4 +#define MSIX_MSG_DATA 8 +#define MSIX_VECTOR_CTRL 12 +#define MSIX_ENTRY_SIZE 16 +#define MSIX_VECTOR_MASK 0x1 + +/* How much space does an MSIX table need. */ +/* The spec requires giving the table structure +* a 4K aligned region all by itself. Align it to +* target pages so that drivers can do passthrough +* on the rest of the region. */ +#define MSIX_PAGE_SIZE TARGET_PAGE_ALIGN(0x1000) +/* Reserve second half of the page for pending bits */ +#define MSIX_PAGE_PENDING (MSIX_PAGE_SIZE / 2) +#define MSIX_MAX_ENTRIES 32 + + +#ifdef MSIX_DEBUG +#define DEBUG (fmt, ...) \ + do { \ + fprintf(stderr, "%s: " fmt, __func__ , __VA_ARGS__); \ + } while (0) +#else +#define DEBUG (fmt, ...) do { } while(0)
Quoted 8 lines
hw/msix.c line 66
Quoted 7 lines
hw/msix.c new file +7
-0+#endif + +/* Flag to globally disable MSI-X support */ +int msix_disable; + +/* Flag for interrupt controller to declare MSI-X support */ +int msix_supported;maybe better to make it static,
It's not read-only either.
Quoted 2 lines
and provide
msi_state()returning -1 for disabled, 0 for supported, etc...
Matter of taste, I prefer a set of binary flags rather than yet another enum:msix_disable is controlled by user, msix_supported is a safety valve for non-PC platforms. It's easier to keep them separate IMO.
Quoted 66 lines
Quoted 54 lines
hw/msix.c new file +53
-0+ +/* Add MSI-X capability to the config space for the device. */ +/* Given a bar and its size, add MSI-X table on top of it +* and fill MSI-X capability in the config space. +* Original bar size must be a power of 2 or 0. +* New bar size is returned. */ +static int msix_add_config (struct PCIDevice *pdev,unsigned short nentries, +unsigned bar_nr,unsigned bar_size) +{ +int config_offset; +uint8_t *config; +uint32_t new_size; + +if (nentries <1 || nentries >PCI_MSIX_FLAGS_QSIZE +1 ) +return -EINVAL ; +if (bar_size >0x80000000 ) +return -ENOSPC ; + +/* Add space for MSI-X structures */ +if (!bar_size) + new_size =MSIX_PAGE_SIZE ; +else if (bar_size <MSIX_PAGE_SIZE ) { + bar_size =MSIX_PAGE_SIZE ; + new_size =MSIX_PAGE_SIZE *2 ; + }else + new_size = bar_size *2 ; + + pdev->msix_bar_size = new_size; + config_offset =pci_add_capability (pdev,PCI_CAP_ID_MSIX ,MSIX_CAP_LENGTH ); +if (config_offset <0 ) +return config_offset; + config = pdev->config + config_offset; + +pci_set_word (config +PCI_MSIX_FLAGS , nentries -1 ); +/* Table on top of BAR */ +pci_set_long (config +MSIX_TABLE_OFFSET , bar_size | bar_nr); +/* Pending bits on top of that */ +pci_set_long (config +MSIX_PBA_OFFSET , (bar_size +MSIX_PAGE_PENDING ) | + bar_nr); + pdev->msix_cap = config_offset; +/* Make flags bit writeable. */ + pdev->wmask[config_offset +MSIX_ENABLE_OFFSET ] |=MSIX_ENABLE_MASK ; +return 0 ; +} + + +/* Initialize the MSI-X structures. Note: if MSI-X is supported, BAR size is +* modified, it should be retrieved with msix_bar_size. */ +int msix_init (struct PCIDevice *dev,unsigned short nentries, +unsigned bar_nr,unsigned bar_size) +{ +int ret = -ENOMEM ; +/* Nothing to do if MSI is not supported by interrupt controller */ hw/msix.c
Quoted 10 lines
hw/msix.c new file +10
-0+if (!msix_supported) +return -ENOTTY ; + +if (nentries >MSIX_MAX_ENTRIES ) +return -EINVAL ; + + dev->msix_entry_used =qemu_mallocz (MSIX_MAX_ENTRIES * +sizeof *dev->msix_entry_used); +if (!dev->msix_entry_used) + goto err_used;no need to check.
oom_checkerwill kill qemu if it fails.
Quoted 6 lines
hw/msix.c line 247
Quoted 4 lines
hw/msix.c new file +4
-0+ + dev->msix_table_page =qemu_mallocz (MSIX_PAGE_SIZE ); +if (!dev->msix_table_page) + goto err_page;ditto.
Good point.
Signature · 1 line
MST
Re: [Qemu-devel] [PATCHv3 05/13] qemu: MSI-X support functions
In reply to Michael S. TsirkinQuoted 89 lines · Michael S. Tsirkin
Add functions implementing MSI-X support. First user will be virtio-pci. Note that platform must set a flag to declare MSI supported. For PC this will be set by APIC.
Signed-off-by Michael S. Tsirkin <[email protected]>--- Makefile.target | 2 +- hw/msix.c | 423 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/msix.h | 35 +++++ hw/pci.h | 20 +++ 4 files changed, 479 insertions(+), 1 deletions(-) create mode 100644 hw/msix.c create mode 100644 hw/msix.hMakefile.target +1
-1@@ -486,7 +486,7 @@ endif #CONFIG_BSD_USER ifndef CONFIG_USER_ONLY OBJS=vl.o osdep.o monitor.o pci.o loader.o isa_mmio.o machine.o \ - gdbstub.o gdbstub-xml.o + gdbstub.o gdbstub-xml.o msix.o # virtio has to be here due to weird dependency between PCI and virtio-net. # need to fix this properly OBJS+=virtio-blk.o virtio-balloon.o virtio-net.o virtio-console.ohw/msix.c new file +56
-0@@ -0,0 +1,423 @@ +/* +* MSI-X device support +* +* This module includes support for MSI-X in pci devices. +* +* Author: Michael S. Tsirkin <[email protected]> +* +* Copyright (c) 2009, Red Hat Inc, Michael S. Tsirkin ([email protected]) +* +* This work is licensed under the terms of the GNU GPL, version 2. See +* the COPYING file in the top-level directory. +*/ + +#include "hw.h" +#include "msix.h" +#include "pci.h" + +/* Declaration from linux/pci_regs.h */ +#define PCI_CAP_ID_MSIX 0x11/* MSI-X */ +#define PCI_MSIX_FLAGS 2/* Table at lower 11 bits */ +#define PCI_MSIX_FLAGS_QSIZE 0x7FF +#define PCI_MSIX_FLAGS_ENABLE (1 << 15) +#define PCI_MSIX_FLAGS_BIRMASK (7 << 0) + +/* MSI-X capability structure */ +#define MSIX_TABLE_OFFSET 4 +#define MSIX_PBA_OFFSET 8 +#define MSIX_CAP_LENGTH 12 + +/* MSI enable bit is in byte 1 in FLAGS register */ +#define MSIX_ENABLE_OFFSET (PCI_MSIX_FLAGS + 1) +#define MSIX_ENABLE_MASK (PCI_MSIX_FLAGS_ENABLE >> 8) + +/* MSI-X table format */ +#define MSIX_MSG_ADDR 0 +#define MSIX_MSG_UPPER_ADDR 4 +#define MSIX_MSG_DATA 8 +#define MSIX_VECTOR_CTRL 12 +#define MSIX_ENTRY_SIZE 16 +#define MSIX_VECTOR_MASK 0x1 + +/* How much space does an MSIX table need. */ +/* The spec requires giving the table structure +* a 4K aligned region all by itself. Align it to +* target pages so that drivers can do passthrough +* on the rest of the region. */ +#define MSIX_PAGE_SIZE TARGET_PAGE_ALIGN(0x1000) +/* Reserve second half of the page for pending bits */ +#define MSIX_PAGE_PENDING (MSIX_PAGE_SIZE / 2) +#define MSIX_MAX_ENTRIES 32 + + +#ifdef MSIX_DEBUG +#define DEBUG (fmt, ...) \ + do { \ + fprintf(stderr, "%s: " fmt, __func__ , __VA_ARGS__); \
hw/msix.c line 66
Quoted 10 lines
hw/msix.c new file +10
-0+ } while (0) +#else +#define DEBUG (fmt, ...) do { } while(0) +#endif + +/* Flag to globally disable MSI-X support */ +int msix_disable; + +/* Flag for interrupt controller to declare MSI-X support */ +int msix_supported;
maybe better to make it static, and provide msi_state() returning -1 for disabled, 0 for supported, etc...
Quoted 54 lines
hw/msix.c new file +53
-0+ +/* Add MSI-X capability to the config space for the device. */ +/* Given a bar and its size, add MSI-X table on top of it +* and fill MSI-X capability in the config space. +* Original bar size must be a power of 2 or 0. +* New bar size is returned. */ +static int msix_add_config (struct PCIDevice *pdev,unsigned short nentries, +unsigned bar_nr,unsigned bar_size) +{ +int config_offset; +uint8_t *config; +uint32_t new_size; + +if (nentries <1 || nentries >PCI_MSIX_FLAGS_QSIZE +1 ) +return -EINVAL ; +if (bar_size >0x80000000 ) +return -ENOSPC ; + +/* Add space for MSI-X structures */ +if (!bar_size) + new_size =MSIX_PAGE_SIZE ; +else if (bar_size <MSIX_PAGE_SIZE ) { + bar_size =MSIX_PAGE_SIZE ; + new_size =MSIX_PAGE_SIZE *2 ; + }else + new_size = bar_size *2 ; + + pdev->msix_bar_size = new_size; + config_offset =pci_add_capability (pdev,PCI_CAP_ID_MSIX ,MSIX_CAP_LENGTH ); +if (config_offset <0 ) +return config_offset; + config = pdev->config + config_offset; + +pci_set_word (config +PCI_MSIX_FLAGS , nentries -1 ); +/* Table on top of BAR */ +pci_set_long (config +MSIX_TABLE_OFFSET , bar_size | bar_nr); +/* Pending bits on top of that */ +pci_set_long (config +MSIX_PBA_OFFSET , (bar_size +MSIX_PAGE_PENDING ) | + bar_nr); + pdev->msix_cap = config_offset; +/* Make flags bit writeable. */ + pdev->wmask[config_offset +MSIX_ENABLE_OFFSET ] |=MSIX_ENABLE_MASK ; +return 0 ; +} + + +/* Initialize the MSI-X structures. Note: if MSI-X is supported, BAR size is +* modified, it should be retrieved with msix_bar_size. */ +int msix_init (struct PCIDevice *dev,unsigned short nentries, +unsigned bar_nr,unsigned bar_size) +{ +int ret = -ENOMEM ; +/* Nothing to do if MSI is not supported by interrupt controller */
hw/msix.c
Quoted 10 lines
hw/msix.c new file +10
-0+if (!msix_supported) +return -ENOTTY ; + +if (nentries >MSIX_MAX_ENTRIES ) +return -EINVAL ; + + dev->msix_entry_used =qemu_mallocz (MSIX_MAX_ENTRIES * +sizeof *dev->msix_entry_used); +if (!dev->msix_entry_used) + goto err_used;
no need to check. oom_checker will kill qemu if it fails.
hw/msix.c line 247
Quoted 4 lines
hw/msix.c new file +4
-0+ + dev->msix_table_page =qemu_mallocz (MSIX_PAGE_SIZE ); +if (!dev->msix_table_page) + goto err_page;
ditto.
Michael S. Tsirkin
maintainerPATCH v3 6/13 qemu: add flag to disable MSI-X by default
In reply to a message not in the archiveAdd global flag to disable MSI-X by default. This is useful primarily to make images loadable by older qemu (without msix). Even when MSI-X is disabled by flag, you can still load images that have MSI-X enabled.
--- hw/msix.c | 3 +++ qemu-options.hx | 2 ++ vl.c | 3 +++ 3 files changed, 8 insertions(+), 0 deletions(-)
hw/msix.c +3 -0
@@ -122,6 +122,9 @@ void msix_write_config(PCIDevice *dev, uint32_t addr, uint32_t val, int len) { unsigned enable_pos = dev->msix_cap + MSIX_ENABLE_OFFSET; +if (!(dev->cap_present &QEMU_PCI_CAP_MSIX )) +return ; +if (addr + len <= enable_pos || addr > enable_pos)return ;
qemu-options.hx +2 -0
@@ -1575,3 +1575,5 @@ DEF("semihosting", 0, QEMU_OPTION_semihosting, DEF("old-param", 0, QEMU_OPTION_old_param, "-old-param old param mode\n") #endif +DEF("disable-msix", 0, QEMU_OPTION_disable_msix, + "-disable-msix disable msix support for PCI devices (enabled by default)\n")
vl.c +3 -0
@@ -134,6 +134,7 @@ int main(int argc, char **argv)#include "hw/usb.h" #include "hw/pcmcia.h" #include "hw/pc.h" +#include "hw/msix.h" #include "hw/audiodev.h" #include "hw/isa.h" #include "hw/baum.h"
@@ -5557,6 +5558,8 @@ int main(int argc, char **argv, char **envp) xen_mode =XEN_ATTACH ;break ;#endif +case QEMU_OPTION_disable_msix: + msix_disable =1 ; } } }
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 6/13 qemu: add flag to disable MSI-X by default
In reply to a message not in the archiveAdd global flag to disable MSI-X by default. This is useful primarily to make images loadable by older qemu (without msix). Even when MSI-X is disabled by flag, you can still load images that have MSI-X enabled.
--- hw/msix.c | 3 +++ qemu-options.hx | 2 ++ vl.c | 3 +++ 3 files changed, 8 insertions(+), 0 deletions(-)
hw/msix.c +3 -0
@@ -122,6 +122,9 @@ void msix_write_config(PCIDevice *dev, uint32_t addr, uint32_t val, int len) { unsigned enable_pos = dev->msix_cap + MSIX_ENABLE_OFFSET; +if (!(dev->cap_present &QEMU_PCI_CAP_MSIX )) +return ; +if (addr + len <= enable_pos || addr > enable_pos)return ;
qemu-options.hx +2 -0
@@ -1575,3 +1575,5 @@ DEF("semihosting", 0, QEMU_OPTION_semihosting, DEF("old-param", 0, QEMU_OPTION_old_param, "-old-param old param mode\n") #endif +DEF("disable-msix", 0, QEMU_OPTION_disable_msix, + "-disable-msix disable msix support for PCI devices (enabled by default)\n")
vl.c +3 -0
@@ -134,6 +134,7 @@ int main(int argc, char **argv)#include "hw/usb.h" #include "hw/pcmcia.h" #include "hw/pc.h" +#include "hw/msix.h" #include "hw/audiodev.h" #include "hw/isa.h" #include "hw/baum.h"
@@ -5557,6 +5558,8 @@ int main(int argc, char **argv, char **envp) xen_mode =XEN_ATTACH ;break ;#endif +case QEMU_OPTION_disable_msix: + msix_disable =1 ; } } }
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 7/13 qemu: minimal MSI/MSI-X implementation for PC
In reply to a message not in the archiveImplement MSI support in APIC. Note that MSI and MMIO APIC registers are at the same memory location, but actually not on the global bus: MSI is on PCI bus, APIC is connected directly to the CPU. We map them on the global bus at the same address which happens to work because MSI registers are reserved in APIC MMIO and vice versa.
--- hw/apic.c | 43 +++++++++++++++++++++++++++++++++++++++---- 1 files changed, 39 insertions(+), 4 deletions(-)
hw/apic.c +39 -4
@@ -19,6 +19,8 @@*/ #include "hw.h" #include "pc.h" +#include "pci.h" +#include "msix.h" #include "qemu-timer.h" #include "host-utils.h"
@@ -63,6 +65,19 @@#define MAX_APICS 255#define MAX_APIC_WORDS 8 +/* Intel APIC constants: from include/asm/msidef.h */ +#define MSI_DATA_VECTOR_SHIFT 0 +#define MSI_DATA_VECTOR_MASK 0x000000ff +#define MSI_DATA_DELIVERY_MODE_SHIFT 8 +#define MSI_DATA_TRIGGER_SHIFT 15 +#define MSI_DATA_LEVEL_SHIFT 14 +#define MSI_ADDR_DEST_MODE_SHIFT 2 +#define MSI_ADDR_DEST_ID_SHIFT 12 +#define MSI_ADDR_DEST_ID_MASK 0x00ffff0 + +#define MSI_ADDR_BASE 0xfee00000 +#define MSI_ADDR_SIZE 0x100000 + typedef struct APICState { CPUState *cpu_env; uint32_t apicbase;
@@ -712,11 +727,31 @@ static uint32_t apic_mem_readl(void *opaque, target_phys_addr_t addr)return val; } +static void apic_send_msi (target_phys_addr_t addr,uint32 data) +{ +uint8_t dest = (addr &MSI_ADDR_DEST_ID_MASK ) >>MSI_ADDR_DEST_ID_SHIFT ; +uint8_t vector = (data &MSI_DATA_VECTOR_MASK ) >>MSI_DATA_VECTOR_SHIFT ; +uint8_t dest_mode = (addr >>MSI_ADDR_DEST_MODE_SHIFT ) &0x1 ; +uint8_t trigger_mode = (data >>MSI_DATA_TRIGGER_SHIFT ) &0x1 ; +uint8_t delivery = (data >>MSI_DATA_DELIVERY_MODE_SHIFT ) &0x7 ; +/* XXX: Ignore redirection hint. */ +apic_deliver_irq (dest, dest_mode, delivery, vector,0 , trigger_mode); +} +static void apic_mem_writel (void *opaque,target_phys_addr_t addr,uint32_t val) {CPUState *env;APICState *s; -int index; +int index = (addr >>4 ) &0xff ; +if (addr >0xfff || !index) { +/* MSI and MMIO APIC are at the same memory location, +* but actually not on the global bus: MSI is on PCI bus +* APIC is connected directly to the CPU. +* Mapping them on the global bus happens to work because +* MSI registers are reserved in APIC MMIO and vice versa. */ +apic_send_msi (addr, val); +return ; + } env = cpu_single_env;if (!env)
@@ -727,7 +762,6 @@ static void apic_mem_writel(void *opaque, target_phys_addr_t addr, uint32_t val)printf ("APIC write: %08x = %08x\n" , (uint32_t )addr, val);#endif - index = (addr >>4 ) &0xff ;switch (index) {case 0x02 : s->id = (val >>24 );
@@ -911,6 +945,7 @@ int apic_init(CPUState *env) s->cpu_env = env;apic_reset (s); + msix_supported =1 ;/* XXX: mapping more APICs at the same memory location */ if (apic_io_memory ==0 ) {
@@ -918,7 +953,8 @@ int apic_init(CPUState *env)on the global memory bus. */ apic_io_memory =cpu_register_io_memory (0 , apic_mem_read, apic_mem_write,NULL ); -cpu_register_physical_memory (s->apicbase & ~0xfff ,0x1000 , +/* XXX: what if the base changes? */ +cpu_register_physical_memory (MSI_ADDR_BASE ,MSI_ADDR_SIZE , apic_io_memory); } s->timer =qemu_new_timer (vm_clock, apic_timer, s);
@@ -929,4 +965,3 @@ int apic_init(CPUState *env) local_apics[s->id] = s;return 0 ; } -
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 7/13 qemu: minimal MSI/MSI-X implementation for PC
In reply to a message not in the archiveImplement MSI support in APIC. Note that MSI and MMIO APIC registers are at the same memory location, but actually not on the global bus: MSI is on PCI bus, APIC is connected directly to the CPU. We map them on the global bus at the same address which happens to work because MSI registers are reserved in APIC MMIO and vice versa.
--- hw/apic.c | 43 +++++++++++++++++++++++++++++++++++++++---- 1 files changed, 39 insertions(+), 4 deletions(-)
hw/apic.c +39 -4
@@ -19,6 +19,8 @@*/ #include "hw.h" #include "pc.h" +#include "pci.h" +#include "msix.h" #include "qemu-timer.h" #include "host-utils.h"
@@ -63,6 +65,19 @@#define MAX_APICS 255#define MAX_APIC_WORDS 8 +/* Intel APIC constants: from include/asm/msidef.h */ +#define MSI_DATA_VECTOR_SHIFT 0 +#define MSI_DATA_VECTOR_MASK 0x000000ff +#define MSI_DATA_DELIVERY_MODE_SHIFT 8 +#define MSI_DATA_TRIGGER_SHIFT 15 +#define MSI_DATA_LEVEL_SHIFT 14 +#define MSI_ADDR_DEST_MODE_SHIFT 2 +#define MSI_ADDR_DEST_ID_SHIFT 12 +#define MSI_ADDR_DEST_ID_MASK 0x00ffff0 + +#define MSI_ADDR_BASE 0xfee00000 +#define MSI_ADDR_SIZE 0x100000 + typedef struct APICState { CPUState *cpu_env; uint32_t apicbase;
@@ -712,11 +727,31 @@ static uint32_t apic_mem_readl(void *opaque, target_phys_addr_t addr)return val; } +static void apic_send_msi (target_phys_addr_t addr,uint32 data) +{ +uint8_t dest = (addr &MSI_ADDR_DEST_ID_MASK ) >>MSI_ADDR_DEST_ID_SHIFT ; +uint8_t vector = (data &MSI_DATA_VECTOR_MASK ) >>MSI_DATA_VECTOR_SHIFT ; +uint8_t dest_mode = (addr >>MSI_ADDR_DEST_MODE_SHIFT ) &0x1 ; +uint8_t trigger_mode = (data >>MSI_DATA_TRIGGER_SHIFT ) &0x1 ; +uint8_t delivery = (data >>MSI_DATA_DELIVERY_MODE_SHIFT ) &0x7 ; +/* XXX: Ignore redirection hint. */ +apic_deliver_irq (dest, dest_mode, delivery, vector,0 , trigger_mode); +} +static void apic_mem_writel (void *opaque,target_phys_addr_t addr,uint32_t val) {CPUState *env;APICState *s; -int index; +int index = (addr >>4 ) &0xff ; +if (addr >0xfff || !index) { +/* MSI and MMIO APIC are at the same memory location, +* but actually not on the global bus: MSI is on PCI bus +* APIC is connected directly to the CPU. +* Mapping them on the global bus happens to work because +* MSI registers are reserved in APIC MMIO and vice versa. */ +apic_send_msi (addr, val); +return ; + } env = cpu_single_env;if (!env)
@@ -727,7 +762,6 @@ static void apic_mem_writel(void *opaque, target_phys_addr_t addr, uint32_t val)printf ("APIC write: %08x = %08x\n" , (uint32_t )addr, val);#endif - index = (addr >>4 ) &0xff ;switch (index) {case 0x02 : s->id = (val >>24 );
@@ -911,6 +945,7 @@ int apic_init(CPUState *env) s->cpu_env = env;apic_reset (s); + msix_supported =1 ;/* XXX: mapping more APICs at the same memory location */ if (apic_io_memory ==0 ) {
@@ -918,7 +953,8 @@ int apic_init(CPUState *env)on the global memory bus. */ apic_io_memory =cpu_register_io_memory (0 , apic_mem_read, apic_mem_write,NULL ); -cpu_register_physical_memory (s->apicbase & ~0xfff ,0x1000 , +/* XXX: what if the base changes? */ +cpu_register_physical_memory (MSI_ADDR_BASE ,MSI_ADDR_SIZE , apic_io_memory); } s->timer =qemu_new_timer (vm_clock, apic_timer, s);
@@ -929,4 +965,3 @@ int apic_init(CPUState *env) local_apics[s->id] = s;return 0 ; } -
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Re: [Qemu-devel] [PATCHv3 07/13] qemu: minimal MSI/MSI-X implementation for PC
Quoted 19 lines
env = cpu_single_env; if (!env)hw/apic.c +1
-1@@ -727,7 +762,6 @@ static void apic_mem_writel(void *opaque, target_phys_addr_t addr, uint32_t val)printf ("APIC write: %08x = %08x\n" , (uint32_t )addr, val);#endif - index = (addr >>4 ) &0xff ;switch (index) {case 0x02 : s->id = (val >>24 );@@ -911,6 +945,7 @@ int apic_init(CPUState *env) s->cpu_env = env;apic_reset (s); + msix_supported =1 ;/* XXX: mapping more APICs at the same memory location */ if (apic_io_memory ==0 ) {
hw/apic.c int apic_init(CPUState *env) · line 958
Quoted 7 lines
hw/apic.c +2
-1@@ -918,7 +953,8 @@ int apic_init(CPUState *env)on the global memory bus. */ apic_io_memory =cpu_register_io_memory (0 , apic_mem_read, apic_mem_write,NULL ); -cpu_register_physical_memory (s->apicbase & ~0xfff ,0x1000 , +/* XXX: what if the base changes? */ +cpu_register_physical_memory (MSI_ADDR_BASE ,MSI_ADDR_SIZE , apic_io_memory);
+1
I think you have a point here. Your patch is in no way worse than what we had, but we're currently not handling correctly the case of base address changing. Guess it is not common in normal apic usage for OSes...
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 07/13] qemu: minimal MSI/MSI-X implementation for PC
Quoted 28 lines · Glauber Costa
Quoted 19 lines
env = cpu_single_env; if (!env)hw/apic.c +1
-1@@ -727,7 +762,6 @@ static void apic_mem_writel(void *opaque, target_phys_addr_t addr, uint32_t val)printf ("APIC write: %08x = %08x\n" , (uint32_t )addr, val);#endif - index = (addr >>4 ) &0xff ;switch (index) {case 0x02 : s->id = (val >>24 );@@ -911,6 +945,7 @@ int apic_init(CPUState *env) s->cpu_env = env;apic_reset (s); + msix_supported =1 ;/* XXX: mapping more APICs at the same memory location */ if (apic_io_memory ==0 ) {hw/apic.c int apic_init(CPUState *env) · line 958
Quoted 7 lines
hw/apic.c +2
-1@@ -918,7 +953,8 @@ int apic_init(CPUState *env)on the global memory bus. */ apic_io_memory =cpu_register_io_memory (0 , apic_mem_read, apic_mem_write,NULL ); -cpu_register_physical_memory (s->apicbase & ~0xfff ,0x1000 , +/* XXX: what if the base changes? */ +cpu_register_physical_memory (MSI_ADDR_BASE ,MSI_ADDR_SIZE , apic_io_memory);+1
Quoted 2 lines
I think you have a point here. Your patch is in no way worse than what we had, but we're currently not handling correctly the case of base address changing.
Yep.
Quoted 1 line
Guess it is not common in normal apic usage for OSes...
Signature · 1 line
MST
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 07/13] qemu: minimal MSI/MSI-X implementation for PC
In reply to Glauber CostaQuoted 28 lines · Glauber Costa
Quoted 19 lines
env = cpu_single_env; if (!env)hw/apic.c +1
-1@@ -727,7 +762,6 @@ static void apic_mem_writel(void *opaque, target_phys_addr_t addr, uint32_t val)printf ("APIC write: %08x = %08x\n" , (uint32_t )addr, val);#endif - index = (addr >>4 ) &0xff ;switch (index) {case 0x02 : s->id = (val >>24 );@@ -911,6 +945,7 @@ int apic_init(CPUState *env) s->cpu_env = env;apic_reset (s); + msix_supported =1 ;/* XXX: mapping more APICs at the same memory location */ if (apic_io_memory ==0 ) {hw/apic.c int apic_init(CPUState *env) · line 958
Quoted 7 lines
hw/apic.c +2
-1@@ -918,7 +953,8 @@ int apic_init(CPUState *env)on the global memory bus. */ apic_io_memory =cpu_register_io_memory (0 , apic_mem_read, apic_mem_write,NULL ); -cpu_register_physical_memory (s->apicbase & ~0xfff ,0x1000 , +/* XXX: what if the base changes? */ +cpu_register_physical_memory (MSI_ADDR_BASE ,MSI_ADDR_SIZE , apic_io_memory);+1
Quoted 2 lines
I think you have a point here. Your patch is in no way worse than what we had, but we're currently not handling correctly the case of base address changing.
Yep.
Quoted 1 line
Guess it is not common in normal apic usage for OSes...
Signature · 1 line
MST
Re: [Qemu-devel] [PATCHv3 07/13] qemu: minimal MSI/MSI-X implementation for PC
In reply to Michael S. TsirkinQuoted 19 lines
env = cpu_single_env; if (!env)hw/apic.c +1
-1@@ -727,7 +762,6 @@ static void apic_mem_writel(void *opaque, target_phys_addr_t addr, uint32_t val)printf ("APIC write: %08x = %08x\n" , (uint32_t )addr, val);#endif - index = (addr >>4 ) &0xff ;switch (index) {case 0x02 : s->id = (val >>24 );@@ -911,6 +945,7 @@ int apic_init(CPUState *env) s->cpu_env = env;apic_reset (s); + msix_supported =1 ;/* XXX: mapping more APICs at the same memory location */ if (apic_io_memory ==0 ) {
hw/apic.c int apic_init(CPUState *env) · line 958
Quoted 7 lines
hw/apic.c +2
-1@@ -918,7 +953,8 @@ int apic_init(CPUState *env)on the global memory bus. */ apic_io_memory =cpu_register_io_memory (0 , apic_mem_read, apic_mem_write,NULL ); -cpu_register_physical_memory (s->apicbase & ~0xfff ,0x1000 , +/* XXX: what if the base changes? */ +cpu_register_physical_memory (MSI_ADDR_BASE ,MSI_ADDR_SIZE , apic_io_memory);
+1
I think you have a point here. Your patch is in no way worse than what we had, but we're currently not handling correctly the case of base address changing. Guess it is not common in normal apic usage for OSes...
Michael S. Tsirkin
maintainerPATCH v3 8/13 qemu: add support for resizing regions
In reply to a message not in the archiveMake it possible to resize PCI regions. This will be used by virtio with MSI-X, where the region size depends on whether MSI-X is enabled, and can change across load/save.
--- hw/pci.c | 54 ++++++++++++++++++++++++++++++++++++------------------ hw/pci.h | 3 +++ 2 files changed, 39 insertions(+), 18 deletions(-)
hw/pci.c +36 -18
@@ -392,6 +392,41 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, *(uint32_t *)(pci_dev->wmask + addr) =cpu_to_le32 (wmask); } +static void pci_unmap_region (PCIDevice *d,PCIIORegion *r) +{ +if (r->addr ==-1 ) +return ; +if (r->type &PCI_ADDRESS_SPACE_IO ) { +int class; +/* NOTE: specific hack for IDE in PC case: +only one byte must be mapped. */ + class =pci_get_word (d->config +PCI_CLASS_DEVICE ); +if (class ==0x0101 && r->size ==4 ) { +isa_unassign_ioport (r->addr +2 ,1 ); + }else { +isa_unassign_ioport (r->addr, r->size); + } + }else { +cpu_register_physical_memory (pci_to_cpu_addr (r->addr), + r->size, +IO_MEM_UNASSIGNED ); +qemu_unregister_coalesced_mmio (r->addr, r->size); + } +} + +void pci_resize_io_region (PCIDevice *pci_dev,int region_num, +uint32_t size) +{ + +PCIIORegion *r = &pci_dev->io_regions[region_num]; +if (r->size == size) +return ; + r->size = size; +pci_unmap_region (pci_dev, r); + r->addr =-1 ; +pci_update_mappings (pci_dev); +} +static void pci_update_mappings (PCIDevice *d) {PCIIORegion *r;
@@ -445,24 +480,7 @@ static void pci_update_mappings(PCIDevice *d) }/* now do the real mapping */ if (new_addr != r->addr) { -if (r->addr !=-1 ) { -if (r->type &PCI_ADDRESS_SPACE_IO ) { -int class; -/* NOTE: specific hack for IDE in PC case: -only one byte must be mapped. */ - class = d->config[0x0a ] | (d->config[0x0b ] <<8 ); -if (class ==0x0101 && r->size ==4 ) { -isa_unassign_ioport (r->addr +2 ,1 ); - }else { -isa_unassign_ioport (r->addr, r->size); - } - }else { -cpu_register_physical_memory (pci_to_cpu_addr (r->addr), - r->size, -IO_MEM_UNASSIGNED ); -qemu_unregister_coalesced_mmio (r->addr, r->size); - } - } +pci_unmap_region (d, r); r->addr = new_addr;if (r->addr !=-1 ) { r->map_func (d, i, r->addr, r->size, r->type);
hw/pci.h +3 -0
@@ -221,6 +221,9 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, uint32_t size, int type, PCIMapIORegionFunc *map_func); +void pci_resize_io_region (PCIDevice *pci_dev,int region_num, +uint32_t size); +int pci_add_capability (PCIDevice *pci_dev,uint8_t cap_id,uint8_t cap_size);void pci_del_capability (PCIDevice *pci_dev,uint8_t cap_id,uint8_t cap_size);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 8/13 qemu: add support for resizing regions
In reply to a message not in the archiveMake it possible to resize PCI regions. This will be used by virtio with MSI-X, where the region size depends on whether MSI-X is enabled, and can change across load/save.
--- hw/pci.c | 54 ++++++++++++++++++++++++++++++++++++------------------ hw/pci.h | 3 +++ 2 files changed, 39 insertions(+), 18 deletions(-)
hw/pci.c +36 -18
@@ -392,6 +392,41 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, *(uint32_t *)(pci_dev->wmask + addr) =cpu_to_le32 (wmask); } +static void pci_unmap_region (PCIDevice *d,PCIIORegion *r) +{ +if (r->addr ==-1 ) +return ; +if (r->type &PCI_ADDRESS_SPACE_IO ) { +int class; +/* NOTE: specific hack for IDE in PC case: +only one byte must be mapped. */ + class =pci_get_word (d->config +PCI_CLASS_DEVICE ); +if (class ==0x0101 && r->size ==4 ) { +isa_unassign_ioport (r->addr +2 ,1 ); + }else { +isa_unassign_ioport (r->addr, r->size); + } + }else { +cpu_register_physical_memory (pci_to_cpu_addr (r->addr), + r->size, +IO_MEM_UNASSIGNED ); +qemu_unregister_coalesced_mmio (r->addr, r->size); + } +} + +void pci_resize_io_region (PCIDevice *pci_dev,int region_num, +uint32_t size) +{ + +PCIIORegion *r = &pci_dev->io_regions[region_num]; +if (r->size == size) +return ; + r->size = size; +pci_unmap_region (pci_dev, r); + r->addr =-1 ; +pci_update_mappings (pci_dev); +} +static void pci_update_mappings (PCIDevice *d) {PCIIORegion *r;
@@ -445,24 +480,7 @@ static void pci_update_mappings(PCIDevice *d) }/* now do the real mapping */ if (new_addr != r->addr) { -if (r->addr !=-1 ) { -if (r->type &PCI_ADDRESS_SPACE_IO ) { -int class; -/* NOTE: specific hack for IDE in PC case: -only one byte must be mapped. */ - class = d->config[0x0a ] | (d->config[0x0b ] <<8 ); -if (class ==0x0101 && r->size ==4 ) { -isa_unassign_ioport (r->addr +2 ,1 ); - }else { -isa_unassign_ioport (r->addr, r->size); - } - }else { -cpu_register_physical_memory (pci_to_cpu_addr (r->addr), - r->size, -IO_MEM_UNASSIGNED ); -qemu_unregister_coalesced_mmio (r->addr, r->size); - } - } +pci_unmap_region (d, r); r->addr = new_addr;if (r->addr !=-1 ) { r->map_func (d, i, r->addr, r->size, r->type);
hw/pci.h +3 -0
@@ -221,6 +221,9 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, uint32_t size, int type, PCIMapIORegionFunc *map_func); +void pci_resize_io_region (PCIDevice *pci_dev,int region_num, +uint32_t size); +int pci_add_capability (PCIDevice *pci_dev,uint8_t cap_id,uint8_t cap_size);void pci_del_capability (PCIDevice *pci_dev,uint8_t cap_id,uint8_t cap_size);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Re: [Qemu-devel] [PATCHv3 08/13] qemu: add support for resizing regions
Quoted 30 lines · Michael S. Tsirkin
Make it possible to resize PCI regions. This will be used by virtio with MSI-X, where the region size depends on whether MSI-X is enabled, and can change across load/save.
Signed-off-by Michael S. Tsirkin <[email protected]>--- hw/pci.c | 54 ++++++++++++++++++++++++++++++++++++------------------ hw/pci.h | 3 +++ 2 files changed, 39 insertions(+), 18 deletions(-)hw/pci.c +12
-0@@ -392,6 +392,41 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, *(uint32_t *)(pci_dev->wmask + addr) =cpu_to_le32 (wmask); } +static void pci_unmap_region (PCIDevice *d,PCIIORegion *r) +{ +if (r->addr ==-1 ) +return ; +if (r->type &PCI_ADDRESS_SPACE_IO ) { +int class; +/* NOTE: specific hack for IDE in PC case: +only one byte must be mapped. */ + class =pci_get_word (d->config +PCI_CLASS_DEVICE ); +if (class ==0x0101 && r->size ==4 ) { +isa_unassign_ioport (r->addr +2 ,1 ); + }else {
hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 416
Quoted 10 lines
hw/pci.c +10
-0+isa_unassign_ioport (r->addr, r->size); + } + }else { +cpu_register_physical_memory (pci_to_cpu_addr (r->addr), + r->size, +IO_MEM_UNASSIGNED ); +qemu_unregister_coalesced_mmio (r->addr, r->size); + } +} +
this is a good cleanup...
hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 429
Quoted 13 lines
hw/pci.c +13
-0+void pci_resize_io_region (PCIDevice *pci_dev,int region_num, +uint32_t size) +{ + +PCIIORegion *r = &pci_dev->io_regions[region_num]; +if (r->size == size) +return ; + r->size = size; +pci_unmap_region (pci_dev, r); + r->addr =-1 ; +pci_update_mappings (pci_dev); +} +
but the only user of this one seem to be commented out, and later removed. Why is this needed?
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 08/13] qemu: add support for resizing regions
Quoted 43 lines · Glauber Costa
Quoted 30 lines · Michael S. Tsirkin
Make it possible to resize PCI regions. This will be used by virtio with MSI-X, where the region size depends on whether MSI-X is enabled, and can change across load/save.
Signed-off-by Michael S. Tsirkin <[email protected]>--- hw/pci.c | 54 ++++++++++++++++++++++++++++++++++++------------------ hw/pci.h | 3 +++ 2 files changed, 39 insertions(+), 18 deletions(-)hw/pci.c +12
-0@@ -392,6 +392,41 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, *(uint32_t *)(pci_dev->wmask + addr) =cpu_to_le32 (wmask); } +static void pci_unmap_region (PCIDevice *d,PCIIORegion *r) +{ +if (r->addr ==-1 ) +return ; +if (r->type &PCI_ADDRESS_SPACE_IO ) { +int class; +/* NOTE: specific hack for IDE in PC case: +only one byte must be mapped. */ + class =pci_get_word (d->config +PCI_CLASS_DEVICE ); +if (class ==0x0101 && r->size ==4 ) { +isa_unassign_ioport (r->addr +2 ,1 ); + }else {hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 416
Quoted 10 lines
hw/pci.c +10
-0+isa_unassign_ioport (r->addr, r->size); + } + }else { +cpu_register_physical_memory (pci_to_cpu_addr (r->addr), + r->size, +IO_MEM_UNASSIGNED ); +qemu_unregister_coalesced_mmio (r->addr, r->size); + } +} +this is a good cleanup...
Quoted 16 lines
hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 429
Quoted 13 lines
hw/pci.c +13
-0+void pci_resize_io_region (PCIDevice *pci_dev,int region_num, +uint32_t size) +{ + +PCIIORegion *r = &pci_dev->io_regions[region_num]; +if (r->size == size) +return ; + r->size = size; +pci_unmap_region (pci_dev, r); + r->addr =-1 ; +pci_update_mappings (pci_dev); +} +but the only user of this one seem to be commented out, and later removed. Why is this needed?
Um, I think this needs to be called on load: virtio has a memmory region if and only if it has MSI-X.
Signature · 1 line
MST
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 08/13] qemu: add support for resizing regions
In reply to Glauber CostaQuoted 43 lines · Glauber Costa
Quoted 30 lines · Michael S. Tsirkin
Make it possible to resize PCI regions. This will be used by virtio with MSI-X, where the region size depends on whether MSI-X is enabled, and can change across load/save.
Signed-off-by Michael S. Tsirkin <[email protected]>--- hw/pci.c | 54 ++++++++++++++++++++++++++++++++++++------------------ hw/pci.h | 3 +++ 2 files changed, 39 insertions(+), 18 deletions(-)hw/pci.c +12
-0@@ -392,6 +392,41 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, *(uint32_t *)(pci_dev->wmask + addr) =cpu_to_le32 (wmask); } +static void pci_unmap_region (PCIDevice *d,PCIIORegion *r) +{ +if (r->addr ==-1 ) +return ; +if (r->type &PCI_ADDRESS_SPACE_IO ) { +int class; +/* NOTE: specific hack for IDE in PC case: +only one byte must be mapped. */ + class =pci_get_word (d->config +PCI_CLASS_DEVICE ); +if (class ==0x0101 && r->size ==4 ) { +isa_unassign_ioport (r->addr +2 ,1 ); + }else {hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 416
Quoted 10 lines
hw/pci.c +10
-0+isa_unassign_ioport (r->addr, r->size); + } + }else { +cpu_register_physical_memory (pci_to_cpu_addr (r->addr), + r->size, +IO_MEM_UNASSIGNED ); +qemu_unregister_coalesced_mmio (r->addr, r->size); + } +} +this is a good cleanup...
Quoted 16 lines
hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 429
Quoted 13 lines
hw/pci.c +13
-0+void pci_resize_io_region (PCIDevice *pci_dev,int region_num, +uint32_t size) +{ + +PCIIORegion *r = &pci_dev->io_regions[region_num]; +if (r->size == size) +return ; + r->size = size; +pci_unmap_region (pci_dev, r); + r->addr =-1 ; +pci_update_mappings (pci_dev); +} +but the only user of this one seem to be commented out, and later removed. Why is this needed?
Um, I think this needs to be called on load: virtio has a memmory region if and only if it has MSI-X.
Signature · 1 line
MST
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 08/13] qemu: add support for resizing regions
In reply to Glauber CostaQuoted 43 lines · Glauber Costa
Quoted 30 lines · Michael S. Tsirkin
Make it possible to resize PCI regions. This will be used by virtio with MSI-X, where the region size depends on whether MSI-X is enabled, and can change across load/save.
Signed-off-by Michael S. Tsirkin <[email protected]>--- hw/pci.c | 54 ++++++++++++++++++++++++++++++++++++------------------ hw/pci.h | 3 +++ 2 files changed, 39 insertions(+), 18 deletions(-)hw/pci.c +12
-0@@ -392,6 +392,41 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, *(uint32_t *)(pci_dev->wmask + addr) =cpu_to_le32 (wmask); } +static void pci_unmap_region (PCIDevice *d,PCIIORegion *r) +{ +if (r->addr ==-1 ) +return ; +if (r->type &PCI_ADDRESS_SPACE_IO ) { +int class; +/* NOTE: specific hack for IDE in PC case: +only one byte must be mapped. */ + class =pci_get_word (d->config +PCI_CLASS_DEVICE ); +if (class ==0x0101 && r->size ==4 ) { +isa_unassign_ioport (r->addr +2 ,1 ); + }else {hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 416
Quoted 10 lines
hw/pci.c +10
-0+isa_unassign_ioport (r->addr, r->size); + } + }else { +cpu_register_physical_memory (pci_to_cpu_addr (r->addr), + r->size, +IO_MEM_UNASSIGNED ); +qemu_unregister_coalesced_mmio (r->addr, r->size); + } +} +this is a good cleanup...
Quoted 16 lines
hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 429
Quoted 13 lines
hw/pci.c +13
-0+void pci_resize_io_region (PCIDevice *pci_dev,int region_num, +uint32_t size) +{ + +PCIIORegion *r = &pci_dev->io_regions[region_num]; +if (r->size == size) +return ; + r->size = size; +pci_unmap_region (pci_dev, r); + r->addr =-1 ; +pci_update_mappings (pci_dev); +} +but the only user of this one seem to be commented out, and later removed. Why is this needed?
This was the missing bit:
Set correct size for msi-x memory region when loading the device.
hw/virtio-pci.c +2 -0
@@ -133,6 +133,8 @@ static int virtio_pci_load_config(void * opaque, QEMUFile *f)return ret;if (msix_present (&proxy->pci_dev))qemu_get_be16s (f, &proxy->vdev->config_vector); + +pci_resize_io_region (&proxy->pci_dev,1 ,msix_bar_size (&proxy->pci_dev));return 0 ; }
Signature · 1 line
MST
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 08/13] qemu: add support for resizing regions
In reply to Glauber CostaQuoted 43 lines · Glauber Costa
Quoted 30 lines · Michael S. Tsirkin
Make it possible to resize PCI regions. This will be used by virtio with MSI-X, where the region size depends on whether MSI-X is enabled, and can change across load/save.
Signed-off-by Michael S. Tsirkin <[email protected]>--- hw/pci.c | 54 ++++++++++++++++++++++++++++++++++++------------------ hw/pci.h | 3 +++ 2 files changed, 39 insertions(+), 18 deletions(-)hw/pci.c +12
-0@@ -392,6 +392,41 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, *(uint32_t *)(pci_dev->wmask + addr) =cpu_to_le32 (wmask); } +static void pci_unmap_region (PCIDevice *d,PCIIORegion *r) +{ +if (r->addr ==-1 ) +return ; +if (r->type &PCI_ADDRESS_SPACE_IO ) { +int class; +/* NOTE: specific hack for IDE in PC case: +only one byte must be mapped. */ + class =pci_get_word (d->config +PCI_CLASS_DEVICE ); +if (class ==0x0101 && r->size ==4 ) { +isa_unassign_ioport (r->addr +2 ,1 ); + }else {hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 416
Quoted 10 lines
hw/pci.c +10
-0+isa_unassign_ioport (r->addr, r->size); + } + }else { +cpu_register_physical_memory (pci_to_cpu_addr (r->addr), + r->size, +IO_MEM_UNASSIGNED ); +qemu_unregister_coalesced_mmio (r->addr, r->size); + } +} +this is a good cleanup...
Quoted 16 lines
hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 429
Quoted 13 lines
hw/pci.c +13
-0+void pci_resize_io_region (PCIDevice *pci_dev,int region_num, +uint32_t size) +{ + +PCIIORegion *r = &pci_dev->io_regions[region_num]; +if (r->size == size) +return ; + r->size = size; +pci_unmap_region (pci_dev, r); + r->addr =-1 ; +pci_update_mappings (pci_dev); +} +but the only user of this one seem to be commented out, and later removed. Why is this needed?
This was the missing bit:
Set correct size for msi-x memory region when loading the device.
hw/virtio-pci.c +2 -0
@@ -133,6 +133,8 @@ static int virtio_pci_load_config(void * opaque, QEMUFile *f)return ret;if (msix_present (&proxy->pci_dev))qemu_get_be16s (f, &proxy->vdev->config_vector); + +pci_resize_io_region (&proxy->pci_dev,1 ,msix_bar_size (&proxy->pci_dev));return 0 ; }
Signature · 1 line
MST
Re: [Qemu-devel] [PATCHv3 08/13] qemu: add support for resizing regions
In reply to Michael S. TsirkinQuoted 30 lines · Michael S. Tsirkin
Make it possible to resize PCI regions. This will be used by virtio with MSI-X, where the region size depends on whether MSI-X is enabled, and can change across load/save.
Signed-off-by Michael S. Tsirkin <[email protected]>--- hw/pci.c | 54 ++++++++++++++++++++++++++++++++++++------------------ hw/pci.h | 3 +++ 2 files changed, 39 insertions(+), 18 deletions(-)hw/pci.c +12
-0@@ -392,6 +392,41 @@ void pci_register_io_region(PCIDevice *pci_dev, int region_num, *(uint32_t *)(pci_dev->wmask + addr) =cpu_to_le32 (wmask); } +static void pci_unmap_region (PCIDevice *d,PCIIORegion *r) +{ +if (r->addr ==-1 ) +return ; +if (r->type &PCI_ADDRESS_SPACE_IO ) { +int class; +/* NOTE: specific hack for IDE in PC case: +only one byte must be mapped. */ + class =pci_get_word (d->config +PCI_CLASS_DEVICE ); +if (class ==0x0101 && r->size ==4 ) { +isa_unassign_ioport (r->addr +2 ,1 ); + }else {
hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 416
Quoted 10 lines
hw/pci.c +10
-0+isa_unassign_ioport (r->addr, r->size); + } + }else { +cpu_register_physical_memory (pci_to_cpu_addr (r->addr), + r->size, +IO_MEM_UNASSIGNED ); +qemu_unregister_coalesced_mmio (r->addr, r->size); + } +} +
this is a good cleanup...
hw/pci.c void pci_register_io_region(PCIDevice *pci_dev, int region_num, · line 429
Quoted 13 lines
hw/pci.c +13
-0+void pci_resize_io_region (PCIDevice *pci_dev,int region_num, +uint32_t size) +{ + +PCIIORegion *r = &pci_dev->io_regions[region_num]; +if (r->size == size) +return ; + r->size = size; +pci_unmap_region (pci_dev, r); + r->addr =-1 ; +pci_update_mappings (pci_dev); +} +
but the only user of this one seem to be commented out, and later removed. Why is this needed?
Michael S. Tsirkin
maintainerPATCH v3 9/13 qemu: virtio support for many interrupt vectors
In reply to a message not in the archiveExtend virtio to support many interrupt vectors, and rearrange code in preparation for multi-vector support (mostly move reset out to bindings, because we will have to reset the vectors in transport-specific code). Actual bindings in pci, and use in net, to follow.
Load and save are not connected to bindings yet, so they are left stubbed out for now.
--- hw/syborg_virtio.c | 13 ++++++++-- hw/virtio-pci.c | 24 +++++++++++++++---- hw/virtio.c | 63 ++++++++++++++++++++++++++++++++++++++------------- hw/virtio.h | 10 ++++++- 4 files changed, 84 insertions(+), 26 deletions(-)
hw/syborg_virtio.c +10 -3
@@ -134,7 +134,10 @@ static void syborg_virtio_writel(void *opaque, target_phys_addr_t offset, vdev->features = value;break ;case SYBORG_VIRTIO_QUEUE_BASE : -virtio_queue_set_addr (vdev, vdev->queue_sel, value); +if (value ==0 ) +virtio_reset (vdev); +else +virtio_queue_set_addr (vdev, vdev->queue_sel, value);break ;case SYBORG_VIRTIO_QUEUE_SEL :if (value <VIRTIO_PCI_QUEUE_MAX )
@@ -228,7 +231,7 @@ static CPUWriteMemoryFunc *syborg_virtio_writefn[] = { syborg_virtio_writel }; -static void syborg_virtio_update_irq (void *opaque) +static void syborg_virtio_update_irq (void *opaque,uint16_t vector) {SyborgVirtIOProxy *proxy = opaque;int level;
@@ -239,7 +242,7 @@ static void syborg_virtio_update_irq(void *opaque) }static VirtIOBindings syborg_virtio_bindings = { - .update_irq = syborg_virtio_update_irq + .notify = syborg_virtio_update_irq };static void syborg_virtio_init (SyborgVirtIOProxy *proxy,VirtIODevice *vdev)
@@ -248,6 +251,8 @@ static void syborg_virtio_init(SyborgVirtIOProxy *proxy, VirtIODevice *vdev) proxy->vdev = vdev; +/* Don't support multiple vectors */ + proxy->vdev->nvectors =0 ;sysbus_init_irq (&proxy->busdev, &proxy->irq); iomemtype =cpu_register_io_memory (0 , syborg_virtio_readfn, syborg_virtio_writefn, proxy);
@@ -255,6 +260,8 @@ static void syborg_virtio_init(SyborgVirtIOProxy *proxy, VirtIODevice *vdev) proxy->id = ((uint32_t )0x1af4 <<16 ) | vdev->device_id; +qemu_register_reset (virtio_reset,0 , vdev); +virtio_bind_device (vdev, &syborg_virtio_bindings, proxy); }
hw/virtio-pci.c +19 -5
@@ -78,13 +78,19 @@ typedef struct {/* virtio device */ -static void virtio_pci_update_irq (void *opaque) +static void virtio_pci_notify (void *opaque,uint16_t vector) {VirtIOPCIProxy *proxy = opaque;qemu_set_irq (proxy->pci_dev.irq[0 ], proxy->vdev->isr &1 ); } +static void virtio_pci_reset (void *opaque) +{ +VirtIOPCIProxy *proxy = opaque; +virtio_reset (proxy->vdev); +} +static void virtio_ioport_write (void *opaque,uint32_t addr,uint32_t val) {VirtIOPCIProxy *proxy = opaque;
@@ -108,7 +114,10 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)break ;case VIRTIO_PCI_QUEUE_PFN : pa = (target_phys_addr_t )val <<VIRTIO_PCI_QUEUE_ADDR_SHIFT ; -virtio_queue_set_addr (vdev, vdev->queue_sel, pa); +if (pa ==0 ) +virtio_pci_reset (proxy); +else +virtio_queue_set_addr (vdev, vdev->queue_sel, pa);break ;case VIRTIO_PCI_QUEUE_SEL :if (val <VIRTIO_PCI_QUEUE_MAX )
@@ -120,7 +129,7 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)case VIRTIO_PCI_STATUS : vdev->status = val &0xFF ;if (vdev->status ==0 ) -virtio_reset (vdev); +virtio_pci_reset (proxy);break ; } }
@@ -158,7 +167,7 @@ static uint32_t virtio_ioport_read(void *opaque, uint32_t addr)/* reading from the ISR also clears it. */ ret = vdev->isr; vdev->isr =0 ; -virtio_update_irq (vdev); +qemu_set_irq (proxy->pci_dev.irq[0 ],0 );break ;default :break ;
@@ -243,7 +252,7 @@ static void virtio_map(PCIDevice *pci_dev, int region_num, }static const VirtIOBindings virtio_pci_bindings = { - .update_irq = virtio_pci_update_irq + .notify = virtio_pci_notify }; static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev,
@@ -255,6 +264,9 @@ static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev, proxy->vdev = vdev; +/* No support for multiple vectors yet. */ + proxy->vdev->nvectors =0 ; + config = proxy->pci_dev.config;pci_config_set_vendor_id (config, vendor);pci_config_set_device_id (config, device);
@@ -279,6 +291,8 @@ static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev,pci_register_io_region (&proxy->pci_dev,0 , size,PCI_ADDRESS_SPACE_IO , virtio_map); +qemu_register_reset (virtio_pci_reset,0 , proxy); +virtio_bind_device (vdev, &virtio_pci_bindings, proxy); }
hw/virtio.c +47 -16
166 lines not on this page · raw message
hw/virtio.h +8 -2
@@ -71,11 +71,13 @@ typedef struct VirtQueueElement } VirtQueueElement;typedef struct { -void (*update_irq)(void * opaque); +void (*notify)(void * opaque,uint16_t vector); }VirtIOBindings ;#define VIRTIO_PCI_QUEUE_MAX 16 +#define VIRTIO_NO_VECTOR 0xffff +struct VirtIODevice {const char *name;
@@ -85,6 +87,8 @@ struct VirtIODeviceuint32_t features;size_t config_len;void *config; +uint16_t config_vector; +int nvectors;uint32_t (*get_features)(VirtIODevice *vdev);uint32_t (*bad_features)(VirtIODevice *vdev);void (*set_features)(VirtIODevice *vdev,uint32_t val);
@@ -114,7 +118,7 @@ void virtio_notify(VirtIODevice *vdev, VirtQueue *vq);void virtio_save (VirtIODevice *vdev,QEMUFile *f); -void virtio_load (VirtIODevice *vdev,QEMUFile *f); +int virtio_load (VirtIODevice *vdev,QEMUFile *f);void virtio_cleanup (VirtIODevice *vdev);
@@ -140,6 +144,8 @@ void virtio_queue_set_addr(VirtIODevice *vdev, int n, target_phys_addr_t addr);target_phys_addr_t virtio_queue_get_addr (VirtIODevice *vdev,int n);int virtio_queue_get_num (VirtIODevice *vdev,int n);void virtio_queue_notify (VirtIODevice *vdev,int n); +uint16_t virtio_queue_vector (VirtIODevice *vdev,int n); +void virtio_queue_set_vector (VirtIODevice *vdev,int n,uint16_t vector);void virtio_reset (void *opaque);void virtio_update_irq (VirtIODevice *vdev);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 9/13 qemu: virtio support for many interrupt vectors
In reply to a message not in the archiveExtend virtio to support many interrupt vectors, and rearrange code in preparation for multi-vector support (mostly move reset out to bindings, because we will have to reset the vectors in transport-specific code). Actual bindings in pci, and use in net, to follow.
Load and save are not connected to bindings yet, so they are left stubbed out for now.
--- hw/syborg_virtio.c | 13 ++++++++-- hw/virtio-pci.c | 24 +++++++++++++++---- hw/virtio.c | 63 ++++++++++++++++++++++++++++++++++++++------------- hw/virtio.h | 10 ++++++- 4 files changed, 84 insertions(+), 26 deletions(-)
hw/syborg_virtio.c +10 -3
@@ -134,7 +134,10 @@ static void syborg_virtio_writel(void *opaque, target_phys_addr_t offset, vdev->features = value;break ;case SYBORG_VIRTIO_QUEUE_BASE : -virtio_queue_set_addr (vdev, vdev->queue_sel, value); +if (value ==0 ) +virtio_reset (vdev); +else +virtio_queue_set_addr (vdev, vdev->queue_sel, value);break ;case SYBORG_VIRTIO_QUEUE_SEL :if (value <VIRTIO_PCI_QUEUE_MAX )
@@ -228,7 +231,7 @@ static CPUWriteMemoryFunc *syborg_virtio_writefn[] = { syborg_virtio_writel }; -static void syborg_virtio_update_irq (void *opaque) +static void syborg_virtio_update_irq (void *opaque,uint16_t vector) {SyborgVirtIOProxy *proxy = opaque;int level;
@@ -239,7 +242,7 @@ static void syborg_virtio_update_irq(void *opaque) }static VirtIOBindings syborg_virtio_bindings = { - .update_irq = syborg_virtio_update_irq + .notify = syborg_virtio_update_irq };static void syborg_virtio_init (SyborgVirtIOProxy *proxy,VirtIODevice *vdev)
@@ -248,6 +251,8 @@ static void syborg_virtio_init(SyborgVirtIOProxy *proxy, VirtIODevice *vdev) proxy->vdev = vdev; +/* Don't support multiple vectors */ + proxy->vdev->nvectors =0 ;sysbus_init_irq (&proxy->busdev, &proxy->irq); iomemtype =cpu_register_io_memory (0 , syborg_virtio_readfn, syborg_virtio_writefn, proxy);
@@ -255,6 +260,8 @@ static void syborg_virtio_init(SyborgVirtIOProxy *proxy, VirtIODevice *vdev) proxy->id = ((uint32_t )0x1af4 <<16 ) | vdev->device_id; +qemu_register_reset (virtio_reset,0 , vdev); +virtio_bind_device (vdev, &syborg_virtio_bindings, proxy); }
hw/virtio-pci.c +19 -5
@@ -78,13 +78,19 @@ typedef struct {/* virtio device */ -static void virtio_pci_update_irq (void *opaque) +static void virtio_pci_notify (void *opaque,uint16_t vector) {VirtIOPCIProxy *proxy = opaque;qemu_set_irq (proxy->pci_dev.irq[0 ], proxy->vdev->isr &1 ); } +static void virtio_pci_reset (void *opaque) +{ +VirtIOPCIProxy *proxy = opaque; +virtio_reset (proxy->vdev); +} +static void virtio_ioport_write (void *opaque,uint32_t addr,uint32_t val) {VirtIOPCIProxy *proxy = opaque;
@@ -108,7 +114,10 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)break ;case VIRTIO_PCI_QUEUE_PFN : pa = (target_phys_addr_t )val <<VIRTIO_PCI_QUEUE_ADDR_SHIFT ; -virtio_queue_set_addr (vdev, vdev->queue_sel, pa); +if (pa ==0 ) +virtio_pci_reset (proxy); +else +virtio_queue_set_addr (vdev, vdev->queue_sel, pa);break ;case VIRTIO_PCI_QUEUE_SEL :if (val <VIRTIO_PCI_QUEUE_MAX )
@@ -120,7 +129,7 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)case VIRTIO_PCI_STATUS : vdev->status = val &0xFF ;if (vdev->status ==0 ) -virtio_reset (vdev); +virtio_pci_reset (proxy);break ; } }
@@ -158,7 +167,7 @@ static uint32_t virtio_ioport_read(void *opaque, uint32_t addr)/* reading from the ISR also clears it. */ ret = vdev->isr; vdev->isr =0 ; -virtio_update_irq (vdev); +qemu_set_irq (proxy->pci_dev.irq[0 ],0 );break ;default :break ;
@@ -243,7 +252,7 @@ static void virtio_map(PCIDevice *pci_dev, int region_num, }static const VirtIOBindings virtio_pci_bindings = { - .update_irq = virtio_pci_update_irq + .notify = virtio_pci_notify }; static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev,
@@ -255,6 +264,9 @@ static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev, proxy->vdev = vdev; +/* No support for multiple vectors yet. */ + proxy->vdev->nvectors =0 ; + config = proxy->pci_dev.config;pci_config_set_vendor_id (config, vendor);pci_config_set_device_id (config, device);
@@ -279,6 +291,8 @@ static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev,pci_register_io_region (&proxy->pci_dev,0 , size,PCI_ADDRESS_SPACE_IO , virtio_map); +qemu_register_reset (virtio_pci_reset,0 , proxy); +virtio_bind_device (vdev, &virtio_pci_bindings, proxy); }
hw/virtio.c +47 -16
166 lines not on this page · raw message
hw/virtio.h +8 -2
@@ -71,11 +71,13 @@ typedef struct VirtQueueElement } VirtQueueElement;typedef struct { -void (*update_irq)(void * opaque); +void (*notify)(void * opaque,uint16_t vector); }VirtIOBindings ;#define VIRTIO_PCI_QUEUE_MAX 16 +#define VIRTIO_NO_VECTOR 0xffff +struct VirtIODevice {const char *name;
@@ -85,6 +87,8 @@ struct VirtIODeviceuint32_t features;size_t config_len;void *config; +uint16_t config_vector; +int nvectors;uint32_t (*get_features)(VirtIODevice *vdev);uint32_t (*bad_features)(VirtIODevice *vdev);void (*set_features)(VirtIODevice *vdev,uint32_t val);
@@ -114,7 +118,7 @@ void virtio_notify(VirtIODevice *vdev, VirtQueue *vq);void virtio_save (VirtIODevice *vdev,QEMUFile *f); -void virtio_load (VirtIODevice *vdev,QEMUFile *f); +int virtio_load (VirtIODevice *vdev,QEMUFile *f);void virtio_cleanup (VirtIODevice *vdev);
@@ -140,6 +144,8 @@ void virtio_queue_set_addr(VirtIODevice *vdev, int n, target_phys_addr_t addr);target_phys_addr_t virtio_queue_get_addr (VirtIODevice *vdev,int n);int virtio_queue_get_num (VirtIODevice *vdev,int n);void virtio_queue_notify (VirtIODevice *vdev,int n); +uint16_t virtio_queue_vector (VirtIODevice *vdev,int n); +void virtio_queue_set_vector (VirtIODevice *vdev,int n,uint16_t vector);void virtio_reset (void *opaque);void virtio_update_irq (VirtIODevice *vdev);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 10/13 qemu: MSI-X support in virtio PCI
In reply to a message not in the archiveThis enables actual support for MSI-X in virtio PCI. First user will be virtio-net.
--- hw/virtio-pci.c | 152 ++++++++++++++++++++++++++++++++++++++++-------------- 1 files changed, 112 insertions(+), 40 deletions(-)
hw/virtio-pci.c +112 -40
@@ -18,6 +18,7 @@#include "virtio.h" #include "pci.h" //#include "sysemu.h" +#include "msix.h" /* from Linux's linux/virtio_pci.h */
@@ -47,7 +48,24 @@* a read-and-acknowledge. */ #define VIRTIO_PCI_ISR 19 -#define VIRTIO_PCI_CONFIG 20 +/* MSI-X registers: only enabled if MSI-X is enabled. */ +/* A 16-bit vector for configuration changes. */ +#define VIRTIO_MSI_CONFIG_VECTOR 20 +/* A 16-bit vector for selected queue notifications. */ +#define VIRTIO_MSI_QUEUE_VECTOR 22 + +/* Config space size */ +#define VIRTIO_PCI_CONFIG_NOMSI 20 +#define VIRTIO_PCI_CONFIG_MSI 24 +#define VIRTIO_PCI_REGION_SIZE (dev) (msix_present(dev) ? \ + VIRTIO_PCI_CONFIG_MSI : \ + VIRTIO_PCI_CONFIG_NOMSI) + +/* The remaining space is defined by each driver as the per-driver +* configuration space */ +#define VIRTIO_PCI_CONFIG (dev) (msix_enabled(dev) ? \ + VIRTIO_PCI_CONFIG_MSI : \ + VIRTIO_PCI_CONFIG_NOMSI)/* Virtio ABI version, if we increment this, we break the guest driver. */ #define VIRTIO_PCI_ABI_VERSION 0
@@ -81,14 +99,17 @@ typedef struct {static void virtio_pci_notify (void *opaque,uint16_t vector) {VirtIOPCIProxy *proxy = opaque; - -qemu_set_irq (proxy->pci_dev.irq[0 ], proxy->vdev->isr &1 ); +if (msix_enabled (&proxy->pci_dev)) +msix_notify (&proxy->pci_dev, vector); +else +qemu_set_irq (proxy->pci_dev.irq[0 ], proxy->vdev->isr &1 ); }static void virtio_pci_reset (void *opaque) {VirtIOPCIProxy *proxy = opaque;virtio_reset (proxy->vdev); +msix_reset (&proxy->pci_dev); }static void virtio_ioport_write (void *opaque,uint32_t addr,uint32_t val)
@@ -97,8 +118,6 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)VirtIODevice *vdev = proxy->vdev;target_phys_addr_t pa; - addr -= proxy->addr; -switch (addr) {case VIRTIO_PCI_GUEST_FEATURES :/* Guest does not negotiate properly? We have to assume nothing. */
@@ -131,17 +150,33 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)if (vdev->status ==0 )virtio_pci_reset (proxy);break ; +case VIRTIO_MSI_CONFIG_VECTOR : +msix_vector_unuse (&proxy->pci_dev, vdev->config_vector); +/* Make it possible for guest to discover an error took place. */ +if (msix_vector_use (&proxy->pci_dev, val) <0 ) + val =VIRTIO_NO_VECTOR ; + vdev->config_vector = val; +break ; +case VIRTIO_MSI_QUEUE_VECTOR : +msix_vector_unuse (&proxy->pci_dev, +virtio_queue_vector (vdev, vdev->queue_sel)); +/* Make it possible for guest to discover an error took place. */ +if (msix_vector_use (&proxy->pci_dev, val) <0 ) + val =VIRTIO_NO_VECTOR ; +virtio_queue_set_vector (vdev, vdev->queue_sel, val); +break ; +default : +fprintf (stderr,"%s: unexpected address 0x%x value 0x%x\n" , + __func__, addr, val); +break ; } } -static uint32_t virtio_ioport_read (void *opaque,uint32_t addr) +static uint32_t virtio_ioport_read (VirtIOPCIProxy *proxy,uint32_t addr) { -VirtIOPCIProxy *proxy = opaque;VirtIODevice *vdev = proxy->vdev;uint32_t ret =0xFFFFFFFF ; - addr -= proxy->addr; -switch (addr) {case VIRTIO_PCI_HOST_FEATURES : ret = vdev->get_features (vdev);
@@ -169,6 +204,12 @@ static uint32_t virtio_ioport_read(void *opaque, uint32_t addr) vdev->isr =0 ;qemu_set_irq (proxy->pci_dev.irq[0 ],0 );break ; +case VIRTIO_MSI_CONFIG_VECTOR : + ret = vdev->config_vector; +break ; +case VIRTIO_MSI_QUEUE_VECTOR : + ret =virtio_queue_vector (vdev, vdev->queue_sel); +break ;default :break ; }
@@ -179,42 +220,72 @@ static uint32_t virtio_ioport_read(void *opaque, uint32_t addr)static uint32_t virtio_pci_config_readb (void *opaque,uint32_t addr) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) +return virtio_ioport_read (proxy, addr); + addr -= config;return virtio_config_readb (proxy->vdev, addr); }static uint32_t virtio_pci_config_readw (void *opaque,uint32_t addr) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) +return virtio_ioport_read (proxy, addr); + addr -= config;return virtio_config_readw (proxy->vdev, addr); }static uint32_t virtio_pci_config_readl (void *opaque,uint32_t addr) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) +return virtio_ioport_read (proxy, addr); + addr -= config;return virtio_config_readl (proxy->vdev, addr); }static void virtio_pci_config_writeb (void *opaque,uint32_t addr,uint32_t val) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) { +virtio_ioport_write (proxy, addr, val); +return ; + } + addr -= config;virtio_config_writeb (proxy->vdev, addr, val); }static void virtio_pci_config_writew (void *opaque,uint32_t addr,uint32_t val) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) { +virtio_ioport_write (proxy, addr, val); +return ; + } + addr -= config;virtio_config_writew (proxy->vdev, addr, val); }static void virtio_pci_config_writel (void *opaque,uint32_t addr,uint32_t val) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) { +virtio_ioport_write (proxy, addr, val); +return ; + } + addr -= config;virtio_config_writel (proxy->vdev, addr, val); }
@@ -223,32 +294,26 @@ static void virtio_map(PCIDevice *pci_dev, int region_num, {VirtIOPCIProxy *proxy =container_of (pci_dev, VirtIOPCIProxy, pci_dev);VirtIODevice *vdev = proxy->vdev; -int i; +unsigned config_len =VIRTIO_PCI_REGION_SIZE (pci_dev) + vdev->config_len; proxy->addr = addr; -for (i =0 ; i <3 ; i++) { -register_ioport_write (addr,VIRTIO_PCI_CONFIG ,1 << i, - virtio_ioport_write, proxy); -register_ioport_read (addr,VIRTIO_PCI_CONFIG ,1 << i, - virtio_ioport_read, proxy); - } -if (vdev->config_len) { -register_ioport_write (addr +VIRTIO_PCI_CONFIG , vdev->config_len,1 , - virtio_pci_config_writeb, proxy); -register_ioport_write (addr +VIRTIO_PCI_CONFIG , vdev->config_len,2 , - virtio_pci_config_writew, proxy); -register_ioport_write (addr +VIRTIO_PCI_CONFIG , vdev->config_len,4 , - virtio_pci_config_writel, proxy); -register_ioport_read (addr +VIRTIO_PCI_CONFIG , vdev->config_len,1 , - virtio_pci_config_readb, proxy); -register_ioport_read (addr +VIRTIO_PCI_CONFIG , vdev->config_len,2 , - virtio_pci_config_readw, proxy); -register_ioport_read (addr +VIRTIO_PCI_CONFIG , vdev->config_len,4 , - virtio_pci_config_readl, proxy); +register_ioport_write (addr, config_len,1 , virtio_pci_config_writeb, proxy); +register_ioport_write (addr, config_len,2 , virtio_pci_config_writew, proxy); +register_ioport_write (addr, config_len,4 , virtio_pci_config_writel, proxy); +register_ioport_read (addr, config_len,1 , virtio_pci_config_readb, proxy); +register_ioport_read (addr, config_len,2 , virtio_pci_config_readw, proxy); +register_ioport_read (addr, config_len,4 , virtio_pci_config_readl, proxy); +if (vdev->config_len) vdev->get_config (vdev, vdev->config); - } +} + +static void virtio_write_config (PCIDevice *pci_dev,uint32_t address, +uint32_t val,int len) +{ +pci_default_write_config (pci_dev, address, val, len); +msix_write_config (pci_dev, address, val, len); }static const VirtIOBindings virtio_pci_bindings = {
@@ -264,9 +329,6 @@ static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev, proxy->vdev = vdev; -/* No support for multiple vectors yet. */ - proxy->vdev->nvectors =0 ; - config = proxy->pci_dev.config;pci_config_set_vendor_id (config, vendor);pci_config_set_device_id (config, device);
@@ -284,7 +346,17 @@ static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev, config[0x3d ] =1 ; - size =20 + vdev->config_len; +if (vdev->nvectors && !msix_init (&proxy->pci_dev, vdev->nvectors,1 ,0 )) { +pci_register_io_region (&proxy->pci_dev,1 , +msix_bar_size (&proxy->pci_dev), +PCI_ADDRESS_SPACE_MEM , + msix_mmio_map); + proxy->pci_dev.config_write = virtio_write_config; + proxy->pci_dev.unregister = msix_uninit; + }else + vdev->nvectors =0 ; + + size =VIRTIO_PCI_REGION_SIZE (&proxy->pci_dev) + vdev->config_len;if (size & (size-1 )) size =1 <<qemu_fls (size);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 10/13 qemu: MSI-X support in virtio PCI
In reply to a message not in the archiveThis enables actual support for MSI-X in virtio PCI. First user will be virtio-net.
--- hw/virtio-pci.c | 152 ++++++++++++++++++++++++++++++++++++++++-------------- 1 files changed, 112 insertions(+), 40 deletions(-)
hw/virtio-pci.c +112 -40
@@ -18,6 +18,7 @@#include "virtio.h" #include "pci.h" //#include "sysemu.h" +#include "msix.h" /* from Linux's linux/virtio_pci.h */
@@ -47,7 +48,24 @@* a read-and-acknowledge. */ #define VIRTIO_PCI_ISR 19 -#define VIRTIO_PCI_CONFIG 20 +/* MSI-X registers: only enabled if MSI-X is enabled. */ +/* A 16-bit vector for configuration changes. */ +#define VIRTIO_MSI_CONFIG_VECTOR 20 +/* A 16-bit vector for selected queue notifications. */ +#define VIRTIO_MSI_QUEUE_VECTOR 22 + +/* Config space size */ +#define VIRTIO_PCI_CONFIG_NOMSI 20 +#define VIRTIO_PCI_CONFIG_MSI 24 +#define VIRTIO_PCI_REGION_SIZE (dev) (msix_present(dev) ? \ + VIRTIO_PCI_CONFIG_MSI : \ + VIRTIO_PCI_CONFIG_NOMSI) + +/* The remaining space is defined by each driver as the per-driver +* configuration space */ +#define VIRTIO_PCI_CONFIG (dev) (msix_enabled(dev) ? \ + VIRTIO_PCI_CONFIG_MSI : \ + VIRTIO_PCI_CONFIG_NOMSI)/* Virtio ABI version, if we increment this, we break the guest driver. */ #define VIRTIO_PCI_ABI_VERSION 0
@@ -81,14 +99,17 @@ typedef struct {static void virtio_pci_notify (void *opaque,uint16_t vector) {VirtIOPCIProxy *proxy = opaque; - -qemu_set_irq (proxy->pci_dev.irq[0 ], proxy->vdev->isr &1 ); +if (msix_enabled (&proxy->pci_dev)) +msix_notify (&proxy->pci_dev, vector); +else +qemu_set_irq (proxy->pci_dev.irq[0 ], proxy->vdev->isr &1 ); }static void virtio_pci_reset (void *opaque) {VirtIOPCIProxy *proxy = opaque;virtio_reset (proxy->vdev); +msix_reset (&proxy->pci_dev); }static void virtio_ioport_write (void *opaque,uint32_t addr,uint32_t val)
@@ -97,8 +118,6 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)VirtIODevice *vdev = proxy->vdev;target_phys_addr_t pa; - addr -= proxy->addr; -switch (addr) {case VIRTIO_PCI_GUEST_FEATURES :/* Guest does not negotiate properly? We have to assume nothing. */
@@ -131,17 +150,33 @@ static void virtio_ioport_write(void *opaque, uint32_t addr, uint32_t val)if (vdev->status ==0 )virtio_pci_reset (proxy);break ; +case VIRTIO_MSI_CONFIG_VECTOR : +msix_vector_unuse (&proxy->pci_dev, vdev->config_vector); +/* Make it possible for guest to discover an error took place. */ +if (msix_vector_use (&proxy->pci_dev, val) <0 ) + val =VIRTIO_NO_VECTOR ; + vdev->config_vector = val; +break ; +case VIRTIO_MSI_QUEUE_VECTOR : +msix_vector_unuse (&proxy->pci_dev, +virtio_queue_vector (vdev, vdev->queue_sel)); +/* Make it possible for guest to discover an error took place. */ +if (msix_vector_use (&proxy->pci_dev, val) <0 ) + val =VIRTIO_NO_VECTOR ; +virtio_queue_set_vector (vdev, vdev->queue_sel, val); +break ; +default : +fprintf (stderr,"%s: unexpected address 0x%x value 0x%x\n" , + __func__, addr, val); +break ; } } -static uint32_t virtio_ioport_read (void *opaque,uint32_t addr) +static uint32_t virtio_ioport_read (VirtIOPCIProxy *proxy,uint32_t addr) { -VirtIOPCIProxy *proxy = opaque;VirtIODevice *vdev = proxy->vdev;uint32_t ret =0xFFFFFFFF ; - addr -= proxy->addr; -switch (addr) {case VIRTIO_PCI_HOST_FEATURES : ret = vdev->get_features (vdev);
@@ -169,6 +204,12 @@ static uint32_t virtio_ioport_read(void *opaque, uint32_t addr) vdev->isr =0 ;qemu_set_irq (proxy->pci_dev.irq[0 ],0 );break ; +case VIRTIO_MSI_CONFIG_VECTOR : + ret = vdev->config_vector; +break ; +case VIRTIO_MSI_QUEUE_VECTOR : + ret =virtio_queue_vector (vdev, vdev->queue_sel); +break ;default :break ; }
@@ -179,42 +220,72 @@ static uint32_t virtio_ioport_read(void *opaque, uint32_t addr)static uint32_t virtio_pci_config_readb (void *opaque,uint32_t addr) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) +return virtio_ioport_read (proxy, addr); + addr -= config;return virtio_config_readb (proxy->vdev, addr); }static uint32_t virtio_pci_config_readw (void *opaque,uint32_t addr) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) +return virtio_ioport_read (proxy, addr); + addr -= config;return virtio_config_readw (proxy->vdev, addr); }static uint32_t virtio_pci_config_readl (void *opaque,uint32_t addr) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) +return virtio_ioport_read (proxy, addr); + addr -= config;return virtio_config_readl (proxy->vdev, addr); }static void virtio_pci_config_writeb (void *opaque,uint32_t addr,uint32_t val) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) { +virtio_ioport_write (proxy, addr, val); +return ; + } + addr -= config;virtio_config_writeb (proxy->vdev, addr, val); }static void virtio_pci_config_writew (void *opaque,uint32_t addr,uint32_t val) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) { +virtio_ioport_write (proxy, addr, val); +return ; + } + addr -= config;virtio_config_writew (proxy->vdev, addr, val); }static void virtio_pci_config_writel (void *opaque,uint32_t addr,uint32_t val) {VirtIOPCIProxy *proxy = opaque; - addr -= proxy->addr +VIRTIO_PCI_CONFIG ; +uint32_t config =VIRTIO_PCI_CONFIG (&proxy->pci_dev); + addr -= proxy->addr; +if (addr < config) { +virtio_ioport_write (proxy, addr, val); +return ; + } + addr -= config;virtio_config_writel (proxy->vdev, addr, val); }
@@ -223,32 +294,26 @@ static void virtio_map(PCIDevice *pci_dev, int region_num, {VirtIOPCIProxy *proxy =container_of (pci_dev, VirtIOPCIProxy, pci_dev);VirtIODevice *vdev = proxy->vdev; -int i; +unsigned config_len =VIRTIO_PCI_REGION_SIZE (pci_dev) + vdev->config_len; proxy->addr = addr; -for (i =0 ; i <3 ; i++) { -register_ioport_write (addr,VIRTIO_PCI_CONFIG ,1 << i, - virtio_ioport_write, proxy); -register_ioport_read (addr,VIRTIO_PCI_CONFIG ,1 << i, - virtio_ioport_read, proxy); - } -if (vdev->config_len) { -register_ioport_write (addr +VIRTIO_PCI_CONFIG , vdev->config_len,1 , - virtio_pci_config_writeb, proxy); -register_ioport_write (addr +VIRTIO_PCI_CONFIG , vdev->config_len,2 , - virtio_pci_config_writew, proxy); -register_ioport_write (addr +VIRTIO_PCI_CONFIG , vdev->config_len,4 , - virtio_pci_config_writel, proxy); -register_ioport_read (addr +VIRTIO_PCI_CONFIG , vdev->config_len,1 , - virtio_pci_config_readb, proxy); -register_ioport_read (addr +VIRTIO_PCI_CONFIG , vdev->config_len,2 , - virtio_pci_config_readw, proxy); -register_ioport_read (addr +VIRTIO_PCI_CONFIG , vdev->config_len,4 , - virtio_pci_config_readl, proxy); +register_ioport_write (addr, config_len,1 , virtio_pci_config_writeb, proxy); +register_ioport_write (addr, config_len,2 , virtio_pci_config_writew, proxy); +register_ioport_write (addr, config_len,4 , virtio_pci_config_writel, proxy); +register_ioport_read (addr, config_len,1 , virtio_pci_config_readb, proxy); +register_ioport_read (addr, config_len,2 , virtio_pci_config_readw, proxy); +register_ioport_read (addr, config_len,4 , virtio_pci_config_readl, proxy); +if (vdev->config_len) vdev->get_config (vdev, vdev->config); - } +} + +static void virtio_write_config (PCIDevice *pci_dev,uint32_t address, +uint32_t val,int len) +{ +pci_default_write_config (pci_dev, address, val, len); +msix_write_config (pci_dev, address, val, len); }static const VirtIOBindings virtio_pci_bindings = {
@@ -264,9 +329,6 @@ static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev, proxy->vdev = vdev; -/* No support for multiple vectors yet. */ - proxy->vdev->nvectors =0 ; - config = proxy->pci_dev.config;pci_config_set_vendor_id (config, vendor);pci_config_set_device_id (config, device);
@@ -284,7 +346,17 @@ static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev, config[0x3d ] =1 ; - size =20 + vdev->config_len; +if (vdev->nvectors && !msix_init (&proxy->pci_dev, vdev->nvectors,1 ,0 )) { +pci_register_io_region (&proxy->pci_dev,1 , +msix_bar_size (&proxy->pci_dev), +PCI_ADDRESS_SPACE_MEM , + msix_mmio_map); + proxy->pci_dev.config_write = virtio_write_config; + proxy->pci_dev.unregister = msix_uninit; + }else + vdev->nvectors =0 ; + + size =VIRTIO_PCI_REGION_SIZE (&proxy->pci_dev) + vdev->config_len;if (size & (size-1 )) size =1 <<qemu_fls (size);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 11/13 qemu: request 3 vectors in virtio-net
In reply to a message not in the archiveRequest up to 3 vectors in virtio-net. Actual bindings might supply less.
--- hw/virtio-net.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
hw/virtio-net.c +1 -0
@@ -621,6 +621,7 @@ VirtIODevice *virtio_net_init(DeviceState *dev) n->mac_table.macs =qemu_mallocz (MAC_TABLE_ENTRIES *ETH_ALEN ); n->vlans =qemu_mallocz (MAX_VLAN >>3 ); + n->vdev.nvectors =3 ;register_savevm ("virtio-net" , virtio_net_id++,VIRTIO_NET_VM_VERSION , virtio_net_save, virtio_net_load, n);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 11/13 qemu: request 3 vectors in virtio-net
In reply to a message not in the archiveRequest up to 3 vectors in virtio-net. Actual bindings might supply less.
--- hw/virtio-net.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
hw/virtio-net.c +1 -0
@@ -621,6 +621,7 @@ VirtIODevice *virtio_net_init(DeviceState *dev) n->mac_table.macs =qemu_mallocz (MAC_TABLE_ENTRIES *ETH_ALEN ); n->vlans =qemu_mallocz (MAX_VLAN >>3 ); + n->vdev.nvectors =3 ;register_savevm ("virtio-net" , virtio_net_id++,VIRTIO_NET_VM_VERSION , virtio_net_save, virtio_net_load, n);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 12/13 qemu: virtio save/load bindings
In reply to a message not in the archiveImplement bindings for virtio save/load. Use them in virtio pci.
--- hw/virtio-pci.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++- hw/virtio.c | 31 ++++++++++++++----------------- hw/virtio.h | 4 ++++ 3 files changed, 66 insertions(+), 18 deletions(-)
hw/virtio-pci.c +48 -1
@@ -105,6 +105,48 @@ static void virtio_pci_notify(void *opaque, uint16_t vector)qemu_set_irq (proxy->pci_dev.irq[0 ], proxy->vdev->isr &1 ); } +static void virtio_pci_save_config (void * opaque,QEMUFile *f) +{ +VirtIOPCIProxy *proxy = opaque; +pci_device_save (&proxy->pci_dev, f); +msix_save (&proxy->pci_dev, f); +if (msix_present (&proxy->pci_dev)) +qemu_put_be16 (f, proxy->vdev->config_vector); +} + +static void virtio_pci_save_queue (void * opaque,int n,QEMUFile *f) +{ +VirtIOPCIProxy *proxy = opaque; +if (msix_present (&proxy->pci_dev)) +qemu_put_be16 (f,virtio_queue_vector (proxy->vdev, n)); +} + +static int virtio_pci_load_config (void * opaque,QEMUFile *f) +{ +VirtIOPCIProxy *proxy = opaque; +int ret; + ret =pci_device_load (&proxy->pci_dev, f); +if (ret) +return ret; + ret =msix_load (&proxy->pci_dev, f); +if (ret) +return ret; +if (msix_present (&proxy->pci_dev)) +qemu_get_be16s (f, &proxy->vdev->config_vector); +return 0 ; +} + +static int virtio_pci_load_queue (void * opaque,int n,QEMUFile *f) +{ +VirtIOPCIProxy *proxy = opaque; +uint16_t vector; +if (!msix_present (&proxy->pci_dev)) +return 0 ; +qemu_get_be16s (f, &vector); +virtio_queue_set_vector (proxy->vdev, n, vector); +return 0 ; +} +static void virtio_pci_reset (void *opaque) {VirtIOPCIProxy *proxy = opaque;
@@ -317,7 +359,12 @@ static void virtio_write_config(PCIDevice *pci_dev, uint32_t address, }static const VirtIOBindings virtio_pci_bindings = { - .notify = virtio_pci_notify + .notify = virtio_pci_notify, + .save_config = virtio_pci_save_config, + .load_config = virtio_pci_load_config, + .save_config = virtio_pci_save_config, + .save_queue = virtio_pci_save_queue, + .load_queue = virtio_pci_load_queue, }; static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev,
hw/virtio.c +14 -17
@@ -568,9 +568,8 @@ void virtio_save(VirtIODevice *vdev, QEMUFile *f) {int i; -/* FIXME: load/save binding. */ -//pci_device_save(&vdev->pci_dev, f); -//msix_save(&vdev->pci_dev, f); +if (vdev->binding->save_config) + vdev->binding->save_config (vdev->binding_opaque, f);qemu_put_8s (f, &vdev->status);qemu_put_8s (f, &vdev->isr);
@@ -596,19 +595,20 @@ void virtio_save(VirtIODevice *vdev, QEMUFile *f)qemu_put_be32 (f, vdev->vq[i].vring.num);qemu_put_be64 (f, vdev->vq[i].pa);qemu_put_be16s (f, &vdev->vq[i].last_avail_idx); -if (vdev->nvectors) -qemu_put_be16s (f, &vdev->vq[i].vector); +if (vdev->binding->save_queue) + vdev->binding->save_queue (vdev->binding_opaque, i, f); } }int virtio_load (VirtIODevice *vdev,QEMUFile *f) { -int num, i; +int num, i, ret; -/* FIXME: load/save binding. */ -//pci_device_load(&vdev->pci_dev, f); -//r = msix_load(&vdev->pci_dev, f); -//pci_resize_io_region(&vdev->pci_dev, 1, msix_bar_size(&vdev->pci_dev)); +if (vdev->binding->load_config) { + ret = vdev->binding->load_config (vdev->binding_opaque, f); +if (ret) +return ret; + }qemu_get_8s (f, &vdev->status);qemu_get_8s (f, &vdev->isr);
@@ -617,10 +617,6 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f) vdev->config_len =qemu_get_be32 (f);qemu_get_buffer (f, vdev->config, vdev->config_len); -if (vdev->nvectors) { -qemu_get_be16s (f, &vdev->config_vector); -//msix_vector_use(&vdev->pci_dev, vdev->config_vector); - } num =qemu_get_be32 (f);for (i =0 ; i < num; i++) {
@@ -631,9 +627,10 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f)if (vdev->vq[i].pa) {virtqueue_init (&vdev->vq[i]); } -if (vdev->nvectors) { -qemu_get_be16s (f, &vdev->vq[i].vector); -//msix_vector_use(&vdev->pci_dev, vdev->config_vector); +if (vdev->binding->load_queue) { + ret = vdev->binding->load_queue (vdev->binding_opaque, i, f); +if (ret) +return ret; } }
hw/virtio.h +4 -0
@@ -72,6 +72,10 @@ typedef struct VirtQueueElementtypedef struct {void (*notify)(void * opaque,uint16_t vector); +void (*save_config)(void * opaque,QEMUFile *f); +void (*save_queue)(void * opaque,int n,QEMUFile *f); +int (*load_config)(void * opaque,QEMUFile *f); +int (*load_queue)(void * opaque,int n,QEMUFile *f); }VirtIOBindings ;#define VIRTIO_PCI_QUEUE_MAX 16
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 12/13 qemu: virtio save/load bindings
In reply to a message not in the archiveImplement bindings for virtio save/load. Use them in virtio pci.
--- hw/virtio-pci.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++- hw/virtio.c | 31 ++++++++++++++----------------- hw/virtio.h | 4 ++++ 3 files changed, 66 insertions(+), 18 deletions(-)
hw/virtio-pci.c +48 -1
@@ -105,6 +105,48 @@ static void virtio_pci_notify(void *opaque, uint16_t vector)qemu_set_irq (proxy->pci_dev.irq[0 ], proxy->vdev->isr &1 ); } +static void virtio_pci_save_config (void * opaque,QEMUFile *f) +{ +VirtIOPCIProxy *proxy = opaque; +pci_device_save (&proxy->pci_dev, f); +msix_save (&proxy->pci_dev, f); +if (msix_present (&proxy->pci_dev)) +qemu_put_be16 (f, proxy->vdev->config_vector); +} + +static void virtio_pci_save_queue (void * opaque,int n,QEMUFile *f) +{ +VirtIOPCIProxy *proxy = opaque; +if (msix_present (&proxy->pci_dev)) +qemu_put_be16 (f,virtio_queue_vector (proxy->vdev, n)); +} + +static int virtio_pci_load_config (void * opaque,QEMUFile *f) +{ +VirtIOPCIProxy *proxy = opaque; +int ret; + ret =pci_device_load (&proxy->pci_dev, f); +if (ret) +return ret; + ret =msix_load (&proxy->pci_dev, f); +if (ret) +return ret; +if (msix_present (&proxy->pci_dev)) +qemu_get_be16s (f, &proxy->vdev->config_vector); +return 0 ; +} + +static int virtio_pci_load_queue (void * opaque,int n,QEMUFile *f) +{ +VirtIOPCIProxy *proxy = opaque; +uint16_t vector; +if (!msix_present (&proxy->pci_dev)) +return 0 ; +qemu_get_be16s (f, &vector); +virtio_queue_set_vector (proxy->vdev, n, vector); +return 0 ; +} +static void virtio_pci_reset (void *opaque) {VirtIOPCIProxy *proxy = opaque;
@@ -317,7 +359,12 @@ static void virtio_write_config(PCIDevice *pci_dev, uint32_t address, }static const VirtIOBindings virtio_pci_bindings = { - .notify = virtio_pci_notify + .notify = virtio_pci_notify, + .save_config = virtio_pci_save_config, + .load_config = virtio_pci_load_config, + .save_config = virtio_pci_save_config, + .save_queue = virtio_pci_save_queue, + .load_queue = virtio_pci_load_queue, }; static void virtio_init_pci(VirtIOPCIProxy *proxy, VirtIODevice *vdev,
hw/virtio.c +14 -17
@@ -568,9 +568,8 @@ void virtio_save(VirtIODevice *vdev, QEMUFile *f) {int i; -/* FIXME: load/save binding. */ -//pci_device_save(&vdev->pci_dev, f); -//msix_save(&vdev->pci_dev, f); +if (vdev->binding->save_config) + vdev->binding->save_config (vdev->binding_opaque, f);qemu_put_8s (f, &vdev->status);qemu_put_8s (f, &vdev->isr);
@@ -596,19 +595,20 @@ void virtio_save(VirtIODevice *vdev, QEMUFile *f)qemu_put_be32 (f, vdev->vq[i].vring.num);qemu_put_be64 (f, vdev->vq[i].pa);qemu_put_be16s (f, &vdev->vq[i].last_avail_idx); -if (vdev->nvectors) -qemu_put_be16s (f, &vdev->vq[i].vector); +if (vdev->binding->save_queue) + vdev->binding->save_queue (vdev->binding_opaque, i, f); } }int virtio_load (VirtIODevice *vdev,QEMUFile *f) { -int num, i; +int num, i, ret; -/* FIXME: load/save binding. */ -//pci_device_load(&vdev->pci_dev, f); -//r = msix_load(&vdev->pci_dev, f); -//pci_resize_io_region(&vdev->pci_dev, 1, msix_bar_size(&vdev->pci_dev)); +if (vdev->binding->load_config) { + ret = vdev->binding->load_config (vdev->binding_opaque, f); +if (ret) +return ret; + }qemu_get_8s (f, &vdev->status);qemu_get_8s (f, &vdev->isr);
@@ -617,10 +617,6 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f) vdev->config_len =qemu_get_be32 (f);qemu_get_buffer (f, vdev->config, vdev->config_len); -if (vdev->nvectors) { -qemu_get_be16s (f, &vdev->config_vector); -//msix_vector_use(&vdev->pci_dev, vdev->config_vector); - } num =qemu_get_be32 (f);for (i =0 ; i < num; i++) {
@@ -631,9 +627,10 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f)if (vdev->vq[i].pa) {virtqueue_init (&vdev->vq[i]); } -if (vdev->nvectors) { -qemu_get_be16s (f, &vdev->vq[i].vector); -//msix_vector_use(&vdev->pci_dev, vdev->config_vector); +if (vdev->binding->load_queue) { + ret = vdev->binding->load_queue (vdev->binding_opaque, i, f); +if (ret) +return ret; } }
hw/virtio.h +4 -0
@@ -72,6 +72,10 @@ typedef struct VirtQueueElementtypedef struct {void (*notify)(void * opaque,uint16_t vector); +void (*save_config)(void * opaque,QEMUFile *f); +void (*save_queue)(void * opaque,int n,QEMUFile *f); +int (*load_config)(void * opaque,QEMUFile *f); +int (*load_queue)(void * opaque,int n,QEMUFile *f); }VirtIOBindings ;#define VIRTIO_PCI_QUEUE_MAX 16
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Re: [Qemu-devel] [PATCHv3 12/13] qemu: virtio save/load bindings
duplicated save config.
Quoted 5 lines
hw/virtio.h +0
-0
hw/virtio.h typedef struct VirtQueueElement · line 80
Quoted 9 lines
hw/virtio.h +4
-0@@ -72,6 +72,10 @@ typedef struct VirtQueueElementtypedef struct {void (*notify)(void * opaque,uint16_t vector); +void (*save_config)(void * opaque,QEMUFile *f); +void (*save_queue)(void * opaque,int n,QEMUFile *f); +int (*load_config)(void * opaque,QEMUFile *f); +int (*load_queue)(void * opaque,int n,QEMUFile *f); }VirtIOBindings ;
So, what's the overall effect on a virtual machine that gets migrated, of a certain device not implementing one of those functions? Will it work? Will it break?
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 12/13] qemu: virtio save/load bindings
Quoted 8 lines · Glauber Costa
duplicated save config.
Quoted 6 lines
hw/virtio.h +0
-0@@ -72,6 +72,10 @@ typedef struct VirtQueueElement
Quoted 10 lines
hw/virtio.h typedef struct VirtQueueElement · line 80
Quoted 8 lines
hw/virtio.h +4
-0typedef struct {void (*notify)(void * opaque,uint16_t vector); +void (*save_config)(void * opaque,QEMUFile *f); +void (*save_queue)(void * opaque,int n,QEMUFile *f); +int (*load_config)(void * opaque,QEMUFile *f); +int (*load_queue)(void * opaque,int n,QEMUFile *f); }VirtIOBindings ;So, what's the overall effect on a virtual machine that gets migrated, of a certain device not implementing one of those functions?
Those are implemented by a transport (e.g. virtio_pci) not the device.
Quoted 1 line
Will it work? Will it break?
It will work - assuming there's nothing transport-specific you need to save and load. If there is - this patch is not breaking anything that isn't already broken ...
Signature · 1 line
MST
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 12/13] qemu: virtio save/load bindings
In reply to Glauber CostaQuoted 8 lines · Glauber Costa
duplicated save config.
Quoted 6 lines
hw/virtio.h +0
-0@@ -72,6 +72,10 @@ typedef struct VirtQueueElement
Quoted 10 lines
hw/virtio.h typedef struct VirtQueueElement · line 80
Quoted 8 lines
hw/virtio.h +4
-0typedef struct {void (*notify)(void * opaque,uint16_t vector); +void (*save_config)(void * opaque,QEMUFile *f); +void (*save_queue)(void * opaque,int n,QEMUFile *f); +int (*load_config)(void * opaque,QEMUFile *f); +int (*load_queue)(void * opaque,int n,QEMUFile *f); }VirtIOBindings ;So, what's the overall effect on a virtual machine that gets migrated, of a certain device not implementing one of those functions?
Those are implemented by a transport (e.g. virtio_pci) not the device.
Quoted 1 line
Will it work? Will it break?
It will work - assuming there's nothing transport-specific you need to save and load. If there is - this patch is not breaking anything that isn't already broken ...
Signature · 1 line
MST
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 12/13] qemu: virtio save/load bindings
In reply to Glauber CostaQuoted 1 line · Glauber Costa
duplicated save config.
Good catch. Fixed.
Signature · 1 line
MST
Michael S. Tsirkin
maintainerRe: [Qemu-devel] [PATCHv3 12/13] qemu: virtio save/load bindings
In reply to Glauber CostaQuoted 1 line · Glauber Costa
duplicated save config.
Good catch. Fixed.
Signature · 1 line
MST
Re: [Qemu-devel] [PATCHv3 12/13] qemu: virtio save/load bindings
In reply to Michael S. Tsirkinduplicated save config.
Quoted 5 lines
hw/virtio.h +0
-0
hw/virtio.h typedef struct VirtQueueElement · line 80
Quoted 9 lines
hw/virtio.h +4
-0@@ -72,6 +72,10 @@ typedef struct VirtQueueElementtypedef struct {void (*notify)(void * opaque,uint16_t vector); +void (*save_config)(void * opaque,QEMUFile *f); +void (*save_queue)(void * opaque,int n,QEMUFile *f); +int (*load_config)(void * opaque,QEMUFile *f); +int (*load_queue)(void * opaque,int n,QEMUFile *f); }VirtIOBindings ;
So, what's the overall effect on a virtual machine that gets migrated, of a certain device not implementing one of those functions? Will it work? Will it break?
Michael S. Tsirkin
maintainerPATCH v3 13/13 qemu: add pci_get/set_byte
In reply to a message not in the archiveAdd pci_get/set_byte to keep *_word and *_long access functions company. They are unused for now.
--- hw/pci.h | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-)
hw/pci.h +12 -0
@@ -263,6 +263,18 @@ PCIBus *pci_bridge_init(PCIBus *bus, int devfn, uint16_t vid, uint16_t did, pci_map_irq_fn map_irq, const char *name);static inline void +pci_set_byte (uint8_t *config,uint8_t val) +{ + *config = val; +} + +static inline uint8_t +pci_get_byte (uint8_t *config) +{ +return *config; +} + +static inline void pci_set_word (uint8_t *config,uint16_t val) {cpu_to_le16wu ((uint16_t *)config, val);
Signature · 1 line
1.6.3.1.56.g79e1.dirty
Michael S. Tsirkin
maintainerPATCH v3 13/13 qemu: add pci_get/set_byte
In reply to a message not in the archiveAdd pci_get/set_byte to keep *_word and *_long access functions company. They are unused for now.
--- hw/pci.h | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-)
hw/pci.h +12 -0
@@ -263,6 +263,18 @@ PCIBus *pci_bridge_init(PCIBus *bus, int devfn, uint16_t vid, uint16_t did, pci_map_irq_fn map_irq, const char *name);static inline void +pci_set_byte (uint8_t *config,uint8_t val) +{ + *config = val; +} + +static inline uint8_t +pci_get_byte (uint8_t *config) +{ +return *config; +} + +static inline void pci_set_word (uint8_t *config,uint16_t val) {cpu_to_le16wu ((uint16_t *)config, val);
Signature · 1 line
1.6.3.1.56.g79e1.dirty