Skip to contentSkip to conversation

AlacrityVM benchmark numbers updated

DISCUSSION Gregory Haskins 2 participants 5 messages Aug 26, 2009

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

We are pleased to announce the availability of the latest networking benchmark numbers for AlacrityVM. We've made several tweaks to the original v0.1 release to improve performance. The most notable is a switch from get_user_pages to switch_mm+copy_[to/from]_user thanks to a review suggestion from Michael Tsirkin (as well as his patch to implement it).

This change alone accounted for freeing up an additional 1.2Gbps, which is over 25% improvement from v0.1. The previous numbers were 4560Gbps before the change, and 5708Gbps after (for 1500mtu over 10GE). This moves us ever closer to the goal of native performance under virtualization.

These changes will be incorporated into the upcoming v0.2 release. For more details, please visit our project wiki:

http://developer.novell.com/wiki/index.php/AlacrityVM

Kind Regards,
-Greg

#1 lore raw
Quoted 7 lines · Gregory Haskins

We are pleased to announce the availability of the latest networking benchmark numbers for AlacrityVM. We've made several tweaks to the original v0.1 release to improve performance. The most notable is a switch from get_user_pages to switch_mm+copy_[to/from]_user thanks to a review suggestion from Michael Tsirkin (as well as his patch to implement it).

Quoted 5 lines

This change alone accounted for freeing up an additional 1.2Gbps, which is over 25% improvement from v0.1. The previous numbers were 4560Gbps before the change, and 5708Gbps after (for 1500mtu over 10GE). This moves us ever closer to the goal of native performance under virtualization.

Interesting, it's good to see that copy_*_user() works so well. Note that there's a possible optimization that goes in the opposite direction - keep using get_user_pages(), but use the dma engine API to perform the actual copy. I expect that it will only be a win when using tso to transfer full pages. Large pages may also help.

Copyless tx also wants get_user_pages(). It makes sense to check if switch_mm() + get_user_pages_fast() gives better performance than get_user_pages().

Signature · 1 line
error compiling committee.c: too many arguments to function
#2 lore raw
Quoted 21 lines · Avi Kivity
Quoted 7 lines · Gregory Haskins

We are pleased to announce the availability of the latest networking benchmark numbers for AlacrityVM. We've made several tweaks to the original v0.1 release to improve performance. The most notable is a switch from get_user_pages to switch_mm+copy_[to/from]_user thanks to a review suggestion from Michael Tsirkin (as well as his patch to implement it).

Quoted 6 lines

This change alone accounted for freeing up an additional 1.2Gbps, which is over 25% improvement from v0.1. The previous numbers were 4560Gbps before the change, and 5708Gbps after (for 1500mtu over 10GE). This moves us ever closer to the goal of native performance under virtualization.

Interesting, it's good to see that copy_*_user() works so well. Note that there's a possible optimization that goes in the opposite direction
- keep using get_user_pages(), but use the dma engine API to perform the actual copy. I expect that it will only be a win when using tso to transfer full pages. Large pages may also help.

Quoted 3 lines

Copyless tx also wants get_user_pages(). It makes sense to check if switch_mm() + get_user_pages_fast() gives better performance than get_user_pages().

Actually, I have already look at this and it does indeed seem better to use switch_mm+gupf() over gup() by quite a large margin. You could then couple that with your DMA-engine idea to potentially gain even more benefits (though probably not for networking since most NICs have their own DMA engine anyway).

Kind Regards,
-Greg

#3 lore raw
Quoted 7 lines · Gregory Haskins

Actually, I have already look at this and it does indeed seem better to use switch_mm+gupf() over gup() by quite a large margin. You could then couple that with your DMA-engine idea to potentially gain even more benefits (though probably not for networking since most NICs have their own DMA engine anyway).

For tx, we'll just go copyless once we plumb the destructors properly. But for rx on a shared interface it is impossible to avoid the copy. You can only choose if you want it done by the cpu or a local dma engine.

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
#4 lore raw
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Gregory Haskins

Actually, I have already look at this and it does indeed seem better to use switch_mm+gupf() over gup() by quite a large margin. You could then couple that with your DMA-engine idea to potentially gain even more benefits (though probably not for networking since most NICs have their own DMA engine anyway).

Quoted 5 lines

For tx, we'll just go copyless once we plumb the destructors properly. But for rx on a shared interface it is impossible to avoid the copy. You can only choose if you want it done by the cpu or a local dma engine.

Yep, agree on both counts.

-Greg