8 NIC limit

Hello list:

I'm working on a project that calls for the creation of a firewall in KVM.
While adding a 20-interface trunk of virtio adapters to bring in a dual 10GB bond, I've discovered an 8 NIC limit in QEMU.

I found the following thread in the list archives detailing a similar problem:
http://kerneltrap.org/mailarchive/linux-kvm/2009/1/29/4848304

It includes a patch for the file qemu/net.h to allow 24 NICs:
https://bugs.launchpad.net/ubuntu/+source/qemu-kvm">qemu-kvm/+bug/595873/+attachment/1429544/+files/max_nics.patch

In my case I want to attach 29, and have simply changed line 8 to 30 from 24.

This will be the first patch I've ever had to do, and so far my internet search yields results that don't seem to apply.

Would someone like to recommend a pertinent tutorial?

Attached is a patch that allows qemu to have up to 32 NICs, without using the qdev -device method.

Quoted 3 lines · On 10/06/2010 12:46 AM, [email protected] wrote:

Attached is a patch that allows qemu to have up to 32 NICs, without using the qdev -device method.

I'd rather there be no fixed limit and we validate that when add fails because there isn't a TCP slot available, we do the right thing.

BTW, using -device, it should be possible to add a very high number of nics because you can specify the PCI address including a function. If this doesn't Just Work today, we should make it work.

Regards,

Anthony Liguori

BTW, using -device, it should be possible to add a very high number of nics because you can specify the PCI address including a function. If this doesn't Just Work today, we should make it work.

Attached is a patch that allows qemu to have up to 32 NICs, without using the qdev -device method.

Hi,

Using the legacy way of starting up NICs, I am hitting a limitation after 29 NICs ie no more than 29 are detected (that's because of the 32 PCI slot limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way? What is the maximum that can be supported by the qdev mothod?

Thanks
Anjali

Quoted 10 lines · Anjali Kulkarni

Hi,

Using the legacy way of starting up NICs, I am hitting a limitation after 29 NICs ie no more than 29 are detected (that's because of the 32 PCI slot limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way? What is the maximum that can be supported by the qdev mothod?

I got up to 104 without trying very hard using the following script:

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do

    args="$args -netdev user,id=eth${slot}_${fn}"
    args="$args -device 

virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile=" done
done

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

Quoted 25 lines · Anthony Liguori

Quoted 12 lines · Anjali Kulkarni

Hi,

Using the legacy way of starting up NICs, I am hitting a limitation after 29
NICs ie no more than 29 are detected (that's because of the 32 PCI slot limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way? What is the maximum
that can be supported by the qdev mothod?

I got up to 104 without trying very hard using the following script:

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do
args="$args -netdev user,id=eth${slot}_${fn}"
args="$args -device
virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile=" done
done

Quoted 5 lines

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

This is unlikely to work right wrt pci hotplug. If we want to support a large number of interfaces, we need true multiport cards.

Quoted 30 lines · Avi Kivity

Quoted 28 lines · Anthony Liguori

Quoted 12 lines · Anjali Kulkarni

Hi,

Using the legacy way of starting up NICs, I am hitting a limitation after 29
NICs ie no more than 29 are detected (that's because of the 32 PCI slot limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way? What is the maximum
that can be supported by the qdev mothod?

I got up to 104 without trying very hard using the following script:

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do

    args="$args -netdev user,id=eth${slot}_${fn}"
    args="$args -device

virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile=" done
done

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm

Quoted 6 lines

Quoted 3 lines

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

This is unlikely to work right wrt pci hotplug. If we want to support a large number of interfaces, we need true multiport cards.

Or a PCI bridge to wire up more PCI buses, so we raise the max limit for any type of device we emulate.

Daniel

I've always been sceptical of this. When physical systems have a large number of NICs, it's via multiple functions, not a bunch of PCI bridges.

Or a PCI bridge to wire up more PCI buses, so we raise the max limit for any type of device we emulate.

Quoted 3 lines · Daniel P. Berrange

Or a PCI bridge to wire up more PCI buses, so we raise the max limit for any type of device we emulate.

Break the 29/30/31 virtio-blk limit ... please!

Rich.

Quoted 7 lines · Richard W.M. Jones

Quoted 3 lines · Daniel P. Berrange

Or a PCI bridge to wire up more PCI buses, so we raise the max limit for any type of device we emulate.

Break the 29/30/31 virtio-blk limit ... please!

It was broken ages ago:

anthony@howler:~$ wc -l /proc/partitions; tail /proc/partitions
422 /proc/partitions
  251     1618          1 vdcx2
  251     1621     489951 vdcx5
  251     1632   10485760 vdcy
  251     1633    9992398 vdcy1
  251     1634          1 vdcy2
  251     1637     489951 vdcy5
  251     1648   10485760 vdcz
  251     1649    9992398 vdcz1
  251     1650          1 vdcz2
  251     1653     489951 vdcz5

This is what makes qdev so useful.

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do
args="$args -drive
file=/home/anthony/images/linux.img,if=none,snapshot=on,id=disk${slot}_${fn}"
args="$args -device
virtio-blk-pci,addr=${slot}.${fn},drive=disk${slot}_${fn},multifunction=on" done
done

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm -serial stdio

Regards,

Anthony Liguori

Quoted 3 lines

Rich.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Can you send me pointers to the qdev documentation? How can I use it? Will it allow us to scale above the 32 PCI limit?

Anjali

Quoted 56 lines · Anthony Liguori

Quoted 10 lines · Richard W.M. Jones

Quoted 4 lines · Daniel P. Berrange

Or a PCI bridge to wire up more PCI buses, so we raise the max limit for any type of device we emulate.

Break the 29/30/31 virtio-blk limit ... please!

It was broken ages ago:

anthony@howler:~$ wc -l /proc/partitions; tail /proc/partitions
422 /proc/partitions
   251     1618          1 vdcx2
   251     1621     489951 vdcx5
   251     1632   10485760 vdcy
   251     1633    9992398 vdcy1
   251     1634          1 vdcy2
   251     1637     489951 vdcy5
   251     1648   10485760 vdcz
   251     1649    9992398 vdcz1
   251     1650          1 vdcz2
   251     1653     489951 vdcz5

This is what makes qdev so useful.

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do
args="$args -drive
file=/home/anthony/images/linux.img,if=none,snapshot=on,id=disk${slot}_${fn}"
args="$args -device
virtio-blk-pci,addr=${slot}.${fn},drive=disk${slot}_${fn},multifunction=on" done
done

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm -serial stdio

Regards,

Anthony Liguori

Quoted 4 lines

Rich.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 3 lines · Anjali Kulkarni

Can you send me pointers to the qdev documentation? How can I use it? Will it allow us to scale above the 32 PCI limit?

It's all below. You just have to create a PCI device and mark the multifunction flag to on and then assign it a PCI address that includes a function number. Then you can pack 8 virtio PCI devices into a single slot.

Regards,

Anthony Liguori

Quoted 40 lines

Anjali

Quoted 34 lines · Anthony Liguori

Quoted 10 lines · Richard W.M. Jones

Quoted 4 lines · Daniel P. Berrange

Or a PCI bridge to wire up more PCI buses, so we raise the max limit for any type of device we emulate.

Break the 29/30/31 virtio-blk limit ... please!

It was broken ages ago:

anthony@howler:~$ wc -l /proc/partitions; tail /proc/partitions
422 /proc/partitions
   251     1618          1 vdcx2
   251     1621     489951 vdcx5
   251     1632   10485760 vdcy
   251     1633    9992398 vdcy1
   251     1634          1 vdcy2
   251     1637     489951 vdcy5
   251     1648   10485760 vdcz
   251     1649    9992398 vdcz1
   251     1650          1 vdcz2
   251     1653     489951 vdcz5

This is what makes qdev so useful.

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do
args="$args -drive

Quoted 23 lines

Quoted 22 lines

args="$args -device
virtio-blk-pci,addr=${slot}.${fn},drive=disk${slot}_${fn},multifunction=on" done
done

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm -serial stdio

Regards,

Anthony Liguori

Quoted 4 lines

Rich.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Thanks. Does this work for e1000 as well?

Also, does it support pci hotplug?

Anjali

Quoted 6 lines · Anthony Liguori

Quoted 4 lines · Anjali Kulkarni

Can you send me pointers to the qdev documentation? How can I use it? Will it allow us to scale above the 32 PCI limit?

Quoted 56 lines

It's all below. You just have to create a PCI device and mark the multifunction flag to on and then assign it a PCI address that includes a function number. Then you can pack 8 virtio PCI devices into a single slot.

Regards,

Anthony Liguori

Quoted 46 lines

Anjali

Quoted 39 lines · Anthony Liguori

Quoted 4 lines · Richard W.M. Jones

On Thu, Oct 14, 2010 at 01:10:47PM +0100, Daniel P. Berrange wrote:

Quoted 9 lines

Quoted 5 lines

Or a PCI bridge to wire up more PCI buses, so we raise the max limit for any type of device we emulate.

Break the 29/30/31 virtio-blk limit ... please!

It was broken ages ago:

anthony@howler:~$ wc -l /proc/partitions; tail /proc/partitions
422 /proc/partitions
    251     1618          1 vdcx2
    251     1621     489951 vdcx5
    251     1632   10485760 vdcy
    251     1633    9992398 vdcy1
    251     1634          1 vdcy2
    251     1637     489951 vdcy5
    251     1648   10485760 vdcz
    251     1649    9992398 vdcz1
    251     1650          1 vdcz2
    251     1653     489951 vdcz5

This is what makes qdev so useful.

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do
args="$args -drive

file=/home/anthony/images/linux.img,if=none,snapshot=on,id=disk${slot}_${fn}>>> "

Quoted 28 lines

Quoted 27 lines

Quoted 25 lines

args="$args -device
virtio-blk-pci,addr=${slot}.${fn},drive=disk${slot}_${fn},multifunction=on" done
done

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm -serial stdio

Regards,

Anthony Liguori

Quoted 5 lines

Rich.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 1 line · Anjali Kulkarni

Thanks. Does this work for e1000 as well?

Haven't tried. I don't know how various e1000 drivers would react.

Quoted 1 line

Also, does it support pci hotplug?

No, but that's fixable down the road.

Regards,

Anthony Liguori

Quoted 9 lines · Anthony Liguori

Quoted 2 lines · Anjali Kulkarni

Thanks. Does this work for e1000 as well?

Haven't tried. I don't know how various e1000 drivers would react.

Quoted 2 lines

Also, does it support pci hotplug?

Quoted 6 lines

No, but that's fixable down the road.

Regards,

Anthony Liguori

Probably not.

Quoted 4 lines · Richard W.M. Jones

Quoted 2 lines · Daniel P. Berrange

Or a PCI bridge to wire up more PCI buses, so we raise the max limit for any type of device we emulate.

Break the 29/30/31 virtio-blk limit ... please!

It was broken ages ago:

Quoted 25 lines · Anthony Liguori

Quoted 12 lines · Anjali Kulkarni

Hi,

Using the legacy way of starting up NICs, I am hitting a limitation after 29
NICs ie no more than 29 are detected (that's because of the 32 PCI slot limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way? What is the maximum
that can be supported by the qdev mothod?

I got up to 104 without trying very hard using the following script:

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do

    args="$args -netdev user,id=eth${slot}_${fn}"
    args="$args -device

virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile=" done
done

Quoted 5 lines

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

This is unlikely to work right wrt pci hotplug. If we want to support a large number of interfaces, we need true multiport cards.

What's the motivation for such a huge number of interfaces?

Quoted 27 lines · Avi Kivity

Quoted 25 lines · Anthony Liguori

Quoted 12 lines · Anjali Kulkarni

Hi,

Using the legacy way of starting up NICs, I am hitting a limitation after 29
NICs ie no more than 29 are detected (that's because of the 32 PCI slot limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way? What is the maximum
that can be supported by the qdev mothod?

I got up to 104 without trying very hard using the following script:

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do

     args="$args -netdev user,id=eth${slot}_${fn}"
     args="$args -device

virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile=" done
done

Quoted 6 lines

Quoted 6 lines

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

I'm amazed that works. Can't see how creating another qdev in the same slot makes a proper multifunction device.

Quoted 3 lines

This is unlikely to work right wrt pci hotplug. If we want to support a large number of interfaces, we need true multiport cards.

Indeed. As far as I know, we can't hot plug multifunction PCI devices.

Quoted 2 lines

What's the motivation for such a huge number of interfaces?

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 36 lines · Markus Armbruster

Quoted 33 lines · Avi Kivity

Quoted 31 lines · Anthony Liguori

Quoted 12 lines · Anjali Kulkarni

Hi,

Using the legacy way of starting up NICs, I am hitting a limitation after 29
NICs ie no more than 29 are detected (that's because of the 32 PCI slot limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way? What is the maximum
that can be supported by the qdev mothod?

I got up to 104 without trying very hard using the following script:

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do

     args="$args -netdev user,id=eth${slot}_${fn}"
     args="$args -device

virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile=" done
done

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

Quoted 3 lines

I'm amazed that works. Can't see how creating another qdev in the same slot makes a proper multifunction device.

multifunction=on sets the multifunction bit for the PCI device. Then it's a matter of setting the address to be a specific function.

Our default platform devices are actually multifunction.

Hi,

- If I want to pass in additional arguments to the guest OS while booting(in particular which slot I want to map a nic to) - is there any way to do it? Some kind of configuration file that I can pass in would also be ok for me.
- Is there a KVM/Qemu/e1000 test suite that is already available and I can us?

Thanks
Anjali

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 24 lines · Anthony Liguori

Quoted 10 lines · Anjali Kulkarni

Hi,

Using the legacy way of starting up NICs, I am hitting a limitation after 29 NICs ie no more than 29 are detected (that's because of the 32 PCI slot limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way? What is the maximum that can be supported by the qdev mothod?

I got up to 104 without trying very hard using the following script:

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do

    args="$args -netdev user,id=eth${slot}_${fn}"
    args="$args -device 

virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile="

done
done

Quoted 4 lines

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

This is unlikely to work right wrt pci hotplug.

If we want to support a large number of interfaces, we need true multiport cards.

What's the motivation for such a huge number of interfaces?

Quoted 3 lines

Quoted 1 line

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

This is unlikely to work right wrt pci hotplug.

Yes. Our hotplug design is based on devices.. This is wrong, it should be based on bus-level concepts (like PCI slots).

Quoted 1 line

If we want to support a large number of interfaces, we need true multiport cards.

This magic here creates a multiport virtio-net card so I'm not really sure what you're suggesting. It would certainly be nice to make this all more user friendly (and make hotplug work).

Quoted 5 lines · Anthony Liguori

Quoted 3 lines

Quoted 1 line

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

This is unlikely to work right wrt pci hotplug.

Quoted 6 lines

Yes. Our hotplug design is based on devices.. This is wrong, it should be based on bus-level concepts (like PCI slots).

Quoted 1 line

If we want to support a large number of interfaces, we need true multiport cards.

This magic here creates a multiport virtio-net card so I'm not really sure what you're suggesting. It would certainly be nice to make this all more user friendly (and make hotplug work).

The big issue is to fix hotplug.

I don't see how we can make it user friendly, without making the ordinary case even more unfriendly. Looks like we need yet another level of indirection here.

Quoted 10 lines · Avi Kivity

Quoted 9 lines · Anthony Liguori

Quoted 3 lines

Quoted 1 line

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

This is unlikely to work right wrt pci hotplug.

Yes. Our hotplug design is based on devices.. This is wrong, it should be based on bus-level concepts (like PCI slots).

Quoted 1 line

If we want to support a large number of interfaces, we need true multiport cards.

Quoted 4 lines

Quoted 2 lines

This magic here creates a multiport virtio-net card so I'm not really sure what you're suggesting. It would certainly be nice to make this all more user friendly (and make hotplug work).

The big issue is to fix hotplug.

Yes, but this is entirely independent of multifunction devices.

Today we shoe-horn hot remove into device_del. Instead, we should have explicit bus-level interfaces for hot remove.

Quoted 1 line · Anthony Liguori

On 10/13/2010 05:32 PM, Anjali Kulkarni wrote:

What's the motivation for such a huge number of interfaces?

BTW, I don't think it's possible to hot-add physical functions. I believe I know of a card that supports dynamic add of physical functions (pre-dating SR-IOV)

Quoted 24 lines · Anthony Liguori

Quoted 12 lines · Anjali Kulkarni

Hi,

Using the legacy way of starting up NICs, I am hitting a limitation after 29
NICs ie no more than 29 are detected (that's because of the 32 PCI slot limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way? What is the maximum
that can be supported by the qdev mothod?

I got up to 104 without trying very hard using the following script:

args=""
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do
args="$args -netdev user,id=eth${slot}_${fn}"
args="$args -device virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile="
done
done

Quoted 5 lines

x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args} -enable-kvm

The key is to make the virtio-net devices multifunction and to fill out all 8 functions for each slot.

This is unlikely to work right wrt pci hotplug. If we want to support a large number of interfaces, we need true multiport cards.

Quoted 6 lines · Anjali Kulkarni

Hi,

Can you elaborate what you mean by bridge support? I would really like to go to 256 interfaces.

Anjali

Isaku Yamahata wrote most of it, I'll let him comment.

Signature · 1 line

MST

Right now, qemu is internally ready to handle pci-to-pci bridge emulators and it's quite easy to add a new simple p2p bridge emulator. Just a single function to initialize vid/did and a single PCIDevinceInfo.

But it's difficult to use multi pci buses at the moment because other part isn't ready. It's only for developers who are interested in this area.

BTW, how many pci buses/slots/functions do you want at most?

What we have in the upstream repository is
- various fixes for p2p bridge support
- the library for p2p bridge support
- pcie root/upstream/downstream port emulators which is p2p bridges ioh3420/xio3130_upstream/xio3130_downstream

What is missing for multiple pci buses is
- a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this.
- a simple P2P bridge emulator for pc architecture
- seabios support
- DSDT for a new machine which has multi pci buses(IRQ routing)
- pv pci bus numbering
this isn't a must, I suppose
I have those patches locally.

thanks,

Quoted 13 lines · Michael S. Tsirkin

Quoted 6 lines · Anjali Kulkarni

Hi,

Can you elaborate what you mean by bridge support? I would really like to go to 256 interfaces.

Anjali

Isaku Yamahata wrote most of it, I'll let him comment.

Signature · 1 line

MST

Could you explain more on irq routing in dsdt and related code in qemu? I encountered a problem: emulated user networking (-net nic -net user) cannot work in my qemu-kvm+q35+pcie code. Your original q35 (pure qemu) works fine.

Right now, qemu is internally ready to handle pci-to-pci bridge emulators and it's quite easy to add a new simple p2p bridge emulator. Just a single function to initialize vid/did and a single PCIDevinceInfo.

But it's difficult to use multi pci buses at the moment because other part isn't ready. It's only for developers who are interested in this area.

BTW, how many pci buses/slots/functions do you want at most?

What we have in the upstream repository is
- various fixes for p2p bridge support
- the library for p2p bridge support
- pcie root/upstream/downstream port emulators which is p2p bridges ioh3420/xio3130_upstream/xio3130_downstream

What is missing for multiple pci buses is
- a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this.

- a simple P2P bridge emulator for pc architecture
- seabios support
- DSDT for a new machine which has multi pci buses(IRQ routing)
- pv pci bus numbering

this isn't a must, I suppose
I have those patches locally.

thanks,

Quoted 13 lines · Michael S. Tsirkin

Quoted 6 lines · Anjali Kulkarni

Hi,

Can you elaborate what you mean by bridge support? I would really like to go to 256 interfaces.

Anjali

Isaku Yamahata wrote most of it, I'll let him comment.

Signature · 1 line

MST

Signature · 1 line

yamahata

Quoted 3 lines

What is missing for multiple pci buses is
- a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this.

IMHO, what we really need is a way to specify bridges through qdev. Not a machine with a hard-coded buses.

The firmware paths used by Gleb's code might supply a good syntax to specify topology.

Quoted 4 lines

- a simple P2P bridge emulator for pc architecture
- seabios support
- DSDT for a new machine which has multi pci buses(IRQ routing)
- pv pci bus numbering

Interesting. What's the motivation for this?

Quoted 6 lines · Michael S. Tsirkin

Quoted 3 lines

What is missing for multiple pci buses is
- a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this.

IMHO, what we really need is a way to specify bridges through qdev. Not a machine with a hard-coded buses.

Yes, it's long-term goal.

Quoted 3 lines

The firmware paths used by Gleb's code might supply a good syntax to specify topology.

Quoted 6 lines

Quoted 4 lines

- a simple P2P bridge emulator for pc architecture
- seabios support
- DSDT for a new machine which has multi pci buses(IRQ routing)
- pv pci bus numbering

Interesting. What's the motivation for this?

For hot-plugging pci-to-pci bridge. It requires sparse pci bus numbering.

In fact, it's not so strong requirement.
Possibly I would have to dig into dsdt.

Signature · 1 line

yamahata

Quoted 19 lines · Isaku Yamahata

Quoted 6 lines · Michael S. Tsirkin

Quoted 3 lines

What is missing for multiple pci buses is
- a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this.

IMHO, what we really need is a way to specify bridges through qdev. Not a machine with a hard-coded buses.

Yes, it's long-term goal.

Quoted 8 lines

The firmware paths used by Gleb's code might supply a good syntax to specify topology.

Quoted 4 lines

- a simple P2P bridge emulator for pc architecture
- seabios support
- DSDT for a new machine which has multi pci buses(IRQ routing)
- pv pci bus numbering

Quoted 3 lines

Quoted 1 line

Interesting. What's the motivation for this?

For hot-plugging pci-to-pci bridge. It requires sparse pci bus numbering.

Not sure I understand. pci bus numbering should be completely up to the guest.

Quoted 2 lines

In fact, it's not so strong requirement.
Possibly I would have to dig into dsdt.

Yes, some guest-side technology like acpi might be needed.

What is missing for multiple pci buses is

- a simple P2P bridge emulator for pc architecture
- seabios support
- DSDT for a new machine which has multi pci buses(IRQ routing)

Hi,

Quoted 5 lines

What is missing for multiple pci buses is

- a simple P2P bridge emulator for pc architecture
- seabios support
- DSDT for a new machine which has multi pci buses(IRQ routing)

Is this really needed?

Hello list:

I'm working on a project that calls for the creation of a firewall in KVM.
While adding a 20-interface trunk of virtio adapters to bring in a dual 10GB bond, I've discovered an 8 NIC limit in QEMU.

Have you tried creating NICs with -device?

Quoted 6 lines · [email protected] writes:

Hello list:

I'm working on a project that calls for the creation of a firewall in KVM.
While adding a 20-interface trunk of virtio adapters to bring in a dual 10GB bond, I've discovered an 8 NIC limit in QEMU.

Have you tried creating NICs with -device? The limit shouldn't apply there.

Hello list:

I'm working on a project that calls for the creation of a firewall in KVM.
While adding a 20-interface trunk of virtio adapters to bring in a dual 10GB bond, I've discovered an 8 NIC limit in QEMU.

I found the following thread in the list archives detailing a similar problem:
http://kerneltrap.org/mailarchive/linux-kvm/2009/1/29/4848304

It includes a patch for the file qemu/net.h to allow 24 NICs:
https://bugs.launchpad.net/ubuntu/+source/qemu-kvm">qemu-kvm/+bug/595873/+attachment/1429544/+files/max_nics.patch

In my case I want to attach 29, and have simply changed line 8 to 30 from 24.

This will be the first patch I've ever had to do, and so far my internet search yields results that don't seem to apply.

Would someone like to recommend a pertinent tutorial?

Many thanks
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html