Skip to contentSkip to conversation

KVM/NET, paravirtual network device

announce Ingo Molnar 5 participants 7 messages Jan 11, 2007 4 messages not in the archive KVM

lore
Share & tools
lore
#0 lore raw

here's a quick update wrt. the KVM paravirtualization work: i'm currently working on KVM/NET, which is a Linux<->Linux virtual network device driver between the KVM host and the KVM guest. It uses the new KVM-paravirtualization infrastructure that i've sent to this list yesterday.

while the code is still a prototype, it's working already and here are some early TCP bandwidth (bw_tcp from guest to host) performance results which i'm sure you'll enjoy:

-----------------------------------------------------------

  | Native via loopback (mtu=1500)         :  235.57 MB/sec |
  | Native via loopback (mtu=4000)         :  487.12 MB/sec |
  -----------------------------------------------------------
  | Qemu via RTL-8029 emulation            :    2.84 MB/sec |
  | KVM via RTL-8029 emulation             :    7.41 MB/sec |
  -----------------------------------------------------------
  | KVM-para via KVM/NET device (mtu=1500) :  303.35 MB/sec |
  | KVM-para via KVM/NET device (mtu=4000) :  389.47 MB/sec |

-----------------------------------------------------------

Ingo

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

#2 lore raw
In reply to a message not in the archive
Quoted 9 lines · Ingo Molnar

here's a quick update wrt. the KVM paravirtualization work: i'm currently working on KVM/NET, which is a Linux<->Linux virtual network device driver between the KVM host and the KVM guest. It uses the new KVM-paravirtualization infrastructure that i've sent to this list yesterday.

while the code is still a prototype, it's working already and here are some early TCP bandwidth (bw_tcp from guest to host) performance results which i'm sure you'll enjoy:

I assume you use kernel support in the host as well? This is exactly the code we should share, IMHO; Xen already has such a thing, and I'd prefer not to have three implementations in the kernel....

Patches?
Rusty.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

#4 lore raw
In reply to a message not in the archive
Quoted 11 lines · Rusty Russell
Quoted 9 lines · Ingo Molnar

here's a quick update wrt. the KVM paravirtualization work: i'm currently working on KVM/NET, which is a Linux<->Linux virtual network device driver between the KVM host and the KVM guest. It uses the new KVM-paravirtualization infrastructure that i've sent to this list yesterday.

while the code is still a prototype, it's working already and here are some early TCP bandwidth (bw_tcp from guest to host) performance results which i'm sure you'll enjoy:

Quoted 5 lines

I assume you use kernel support in the host as well? This is exactly the code we should share, IMHO; Xen already has such a thing, and I'd prefer not to have three implementations in the kernel....

Patches?

no patches yet - it's still prototype. I hope to have something to look at tomorrow-ish. With lhype sharing ought to be possible, while Xen's model is quite different. I agree that we dont want 3 implementations in the kernel ...

Ingo

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

#6 lore raw
In reply to a message not in the archive
Quoted 19 lines · Ingo Molnar

here's a quick update wrt. the KVM paravirtualization work: i'm currently working on KVM/NET, which is a Linux<->Linux virtual network device driver between the KVM host and the KVM guest. It uses the new KVM-paravirtualization infrastructure that i've sent to this list yesterday.

while the code is still a prototype, it's working already and here are some early TCP bandwidth (bw_tcp from guest to host) performance results which i'm sure you'll enjoy:

-----------------------------------------------------------

  | Native via loopback (mtu=1500)         :  235.57 MB/sec |
  | Native via loopback (mtu=4000)         :  487.12 MB/sec |
  -----------------------------------------------------------
  | Qemu via RTL-8029 emulation            :    2.84 MB/sec |
  | KVM via RTL-8029 emulation             :    7.41 MB/sec |
  -----------------------------------------------------------
  | KVM-para via KVM/NET device (mtu=1500) :  303.35 MB/sec |
  | KVM-para via KVM/NET device (mtu=4000) :  389.47 MB/sec |
Quoted 3 lines

-----------------------------------------------------------

That's about a 5000% improvement :)

Amazing, both the bandwidth results and the speed in which it was reached. I'm really at a loss for words.

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
#5 lore raw
In reply to a message not in the archive
Quoted 3 lines · Ingo Molnar
Quoted 2 lines

here's a quick update wrt. the KVM paravirtualization work: i'm currently working on KVM/NET, which is a Linux<->Linux virtual

network

Quoted 5 lines
Quoted 5 lines

device driver between the KVM host and the KVM guest. It uses the new KVM-paravirtualization infrastructure that i've sent to this list yesterday.

while the code is still a prototype, it's working already and here

are

Quoted 1 line
Quoted 1 line

some early TCP bandwidth (bw_tcp from guest to host) performance

results

Quoted 5 lines
Quoted 1 line

which i'm sure you'll enjoy:

I assume you use kernel support in the host as well? This is exactly the code we should share, IMHO; Xen already has such a thing, and I'd prefer not to have three implementations in the kernel....

Well, Xen implementation has the grant tables and all their ring code (a good one thought). On the beginning you can start without convergence and aim towards it. Here is a link that describes Xen HVM pv driver performance:
http://www.xensource.com/files/summit_3/xen-pv-drivers.pdf
for 1500 mtu their doing 400Mbs rx, 500+Mbs tx. I guess Ingo will soon reach that...

Quoted 3 lines

Patches?
Rusty.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

#8 lore raw
In reply to a message not in the archive
Quoted 9 lines · Avi Kivity
Quoted 3 lines · Ingo Molnar
  | Qemu via RTL-8029 emulation            :    2.84 MB/sec |
  | KVM via RTL-8029 emulation             :    7.41 MB/sec |
  | KVM-para via KVM/NET device (mtu=1500) :  303.35 MB/sec |

That's about a 5000% improvement :)

Amazing, both the bandwidth results and the speed in which it was reached. I'm really at a loss for words.

Is maybe a better question to ask what it is that makes the Qemu hardware emulation so slow?

Andy

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

#10 lore raw
In reply to a message not in the archive
Quoted 6 lines · Andy Ross
Quoted 5 lines · Avi Kivity
Quoted 3 lines · Ingo Molnar
  | Qemu via RTL-8029 emulation            :    2.84 MB/sec |
  | KVM via RTL-8029 emulation             :    7.41 MB/sec |
  | KVM-para via KVM/NET device (mtu=1500) :  303.35 MB/sec |
Quoted 8 lines
Quoted 4 lines

That's about a 5000% improvement :)

Amazing, both the bandwidth results and the speed in which it was reached. I'm really at a loss for words.

Is maybe a better question to ask what it is that makes the Qemu hardware emulation so slow?

To send a packet through ne2k, you need about 44 pio writes. at ~10us per pio write, that's about 2k packets/sec, or 3MB/sec. RTL8029 is about twice as fast.

A pv interface can send multiple packets per hypercall. Working backward from Ingo's numbers, 300 MB/sec = 200k packets/sec = 2 packets/hypercall. Or perhaps his hardware is faster and he does 1 packet/hypercall. Or maybe I need to take the memcpy time into account at this bandwidth :)

Signature · 8 lines
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV