VMXNET3 paravirtual NIC device implementation
Subsystems (2)
v7 cover VMXNET3 paravirtual NIC device implementation
NOTE: This is not a final patches for commit,
they are just for the latest fixes review. This set of patches still misses test for the device.
We didn't succeed to find any guide or sample for the king of tests required (packets transmission). If someone can provide a pointer to the relevant information we'll be very grateful.
This set of patches implements VMWare VMXNET3 paravirtual NIC device. The device supports of all the device features including offload capabilties, VLANs and etc.
The device is tested on different OSes:
Fedora 15
Ubuntu 10.4
Centos 6.2
Windows 2008R2
Windows 2008 64bit
Windows 2008 32bit
Windows 2003 64bit
Windows 2003 32bitChanges in V7:
Issues reported by Michael S. Tsirkin reviewed and mostly fixed:
File vmware_utils.h:
...
Review
Quoted 7 lines
(quoted diff) +7
-0+static inline void +vmw_shmem_st64(target_phys_addr_t addr, uint64_t value) +{ + VMW_SHPRN("SHMEM store64: %" PRIx64 " (value %" PRIx64 ")", addr, value); + stq_le_phys(addr, value); +} +
Pls remove these wrappers. These are just memory stores. Our codebase is too large as it is without every driver wrapping all standard calls.
[DF] Idea behind this macros is to have printout in each of them [DF] Printouts are really needed when reverse-engineering windows guest drivers [DF] Since there is no windows drivers code this is the only way to understand [DF] sequences they use
Quoted 1 line
+/* MACROS for simplification of operations on array-style registers */
UPPERCASE ABUSE
[DF] Fixed
Review
Quoted 2 lines
(quoted diff) +2
-0+#define IS_MULTIREG_ADDR(addr, base, cnt, regsize) \ + (((addr + 1) > (base)) && ((addr) < (base) + (cnt) * (regsize)))
Same as range_covers_byte(base, cnt * regsize, addr)?
[DF] Fixed, thanks
Review
Quoted 4 lines
(quoted diff) +4
-0+ +#define MULTIREG_IDX_BY_ADDR(addr, base, regsize) \ + (((addr) - (base)) / (regsize)) +
Above two macros is all that's left. No objection but it does not say what they do - want to add minimal documentation?
And please prefix with VMWARE_ or something.
[DF] Prefix added, macros documented
File vmxnet_utils.h (and related with the same problems):
...
Review
Quoted 6 lines
+ * See the COPYING file in the top-level directory.
+ *
+ */(quoted diff) +3
-0+ +#ifndef _VMXNET_UTILS_H_ +#define _VMXNET_UTILS_H_
Please do not start identifiers with _ followed by an uppercase lattters.
[DF] Fixed
...
Review
Quoted 7 lines
(quoted diff) +7
-0+ +struct eth_header { + uint8_t h_dest[ETH_ALEN]; /* destination eth addr */ + uint8_t h_source[ETH_ALEN]; /* source ether addr */ + uint16_t h_proto; /* packet type ID field */ +}; +
And fix struct definitions to
1. follow qemu coding style
[DF] It's not clear what's wrong with the coding style. Please, elaborate.
2. start with vmxnet
[DF] Since this is generic network definitions with no device specifics
[DF] I'm not sure it makes sense to add vmxnet prefix.
[DF] I'd say it makes sense to put them into some generic header files under "net" directory instead [DF] and clean up other devices that have their local definitions of the same structures. [DF] Please, advise.
I also don't really understand why are these functions split out - vmxnet is the only user, no?
[DF] Currently vmxnet3 is the only user, hovewer we have vmxnet2 implementation as well (not submitted yet) [DF] and it uses the same header
File vmxnet_pkt.h:
...
Review
Quoted 3 lines
(quoted diff) +3
-0+ +/* tx module context handle */ +typedef void *VmxnetTxPktH;
This gets you zero type safety and makes debugging impossible. Just use pointers like everyone does.
[DF] Handle type is added to hide VmxnetTxPkt structure into .c header [DF] for better encapsulation.
[DF] Should we drop it anyway?
Files vmxnet3.*:
...
Review
Quoted 3 lines
(quoted diff) +3
-0+ + if (zero_region) { + vmw_shmem_set(pa, 0, size*cell_size);
spaces around *
[DF] Fixed
...
Quoted 7 lines
+#define
vmxnet3_ring_dump(macro, ring_name, ridx, r)\
+ macro("%s#%d: base %" PRIx64 " size %lucell_size%lu gen %d next %lu",
\
+ (ring_name), (ridx),
\
+ (r)->pa, (r)->size, (r)->cell_size, (r)->gen, (r)->next)
make macros upper case
[DF] Fixed
...
Review
Quoted 5 lines
(quoted diff) +5
-0+static inline void vmxnet3_flush_shmem_changes(void) +{ + /* + * Flush shared memory changes + * Needed before transferring control to guest
what does 'transferring control to guest' mean?
[DF] Changed to
[DF] /*
[DF] * Flush shared memory changes [DF] * Needed before sending interrupt to guest to ensure [DF] * it gets consistent memory state [DF] */
...
Review
Quoted 3 lines
(quoted diff) +3
-0+ */ + smp_wmb(); +}
Don't use wrappers like this. They just hide bugs. For example it's not helpful before an interrupt in the function below.
[DF] I guess you are talking about vmxnet3_complete_packet() [DF] Strictly speaking barrier is a must because we change shared memory in [DF] vmxnet3_complete_packet()
[DF] And the wrapper is a good thing because its name explains its effect [DF] in a formal way as opposed to comments
...
Review
Quoted 2 lines
(quoted diff) +2
-0+ switch (status) { + case VMXNET3_PKT_STATUS_OK: {
don't put {} around cases: they align incorrectly
if it's too big move to a function.
[DF] Fixed
...
Review
Quoted 5 lines
(quoted diff) +5
-0+static bool +vmxnet3_send_packet(VMXNET3State *s, uint32_t qidx) +{ + size_t bytes_sent = 0; + bool res = true;
why = true? don't initialize just because.
[DF] Fixed
...
Quoted 6 lines
+/*
+ * VMWARE headers we got from Linux kernel do not fully comply QEMU coding
+ * standards in sense of types and defines used.
+ * Since we didn't want to change VMWARE code, following set of typedefs
+ * and defines needed to compile these headers with QEMU introduced.
+ */
No need for this now.
You can export headers and put them under linux-headers.
[DF] Not sure it is possible because the header as-is is not stand-alone and won't compile [DF] without changes. We extracted definitions we use from their header and dropped unused [DF] and kernel-specific stuff.
[DF} Please, advise.
...
Quoted 1 line
+ if (
VMXNET3_OM_TSO==s->offload_mode) {
Don't do Yoda style like this
[DF] "Yoda" style removed everywhere
Changes in V6:
Fixed most of problems pointed out by Michael S. Tsirkin
The only issue still open is creation of shared place
with generic network structures and functions. Currently
all generic network code introduced by VMXNET3 resides in vmxnet_utils.c/h files. It could be moved to some shared location however we believe it is a matter of separate refactoring as there are a lot of copy-pasted definitions in almost every device and code cleanup efforts requred in order to create truly shared codebase.
Implemented suggestions by Anthony Liguori
Fixed incorrect checksum caclulation for some packets in SW offloads mode
Changes in V5:
MSI-X save/load implemented in the device instead of pci bus as suggested by Michael S. Tsirkin
Patches regrouped as suggested by Paolo Bonzini
Changes in V4:
Fixed a few problems uncovered by NETIO test suit Assertion on failure to initialize MSI/MSI-X replaced with warning message and fallback to Legacy/MSI respectively
Various coding style adjustments and patch split-up as suggested by Anthony
Liguori
Live migration support added
Changes in V3:
Fixed crash when net device that is used as network fronted has no virtio HDR support.
Task offloads emulation for cases when net device that is used as network fronted has no virtio HDR support.
Changes in V2:
License text changed accoring to community suggestions Standard license header from GPLv2+ - licensed QEMU files used
Dmitry Fleytman (6):
Adding utility function net_checksum_add_cont() that allows checksum
calculation of scattered data with odd chunk sizes
Adding utility function iov_net_csum_add() for iovec checksum
calculation Adding utility function iov_rebuild() for smart
iovec copy
Adding common definitions for VMWARE devices
Adding common code for VMWARE network devices
Adding packet abstraction for VMWARE network devices Adding VMXNET3 device implementation
Makefile.objs | 1 + default-configs/pci.mak | 1 + hw/Makefile.objs | 1 + hw/pci.h | 1 + hw/vmware_utils.h | 143 +++ hw/vmxnet3.c | 2442 +++++++++++++++++++++++++++++++++++++++++++++++ hw/vmxnet3.h | 762 +++++++++++++++ hw/vmxnet_debug.h | 121 +++ hw/vmxnet_pkt.c | 776 +++++++++++++++ hw/vmxnet_pkt.h | 311 ++++++ hw/vmxnet_utils.c | 219 +++++ hw/vmxnet_utils.h | 340 +++++++ iov.c | 53 + iov.h | 13 + net/checksum.c | 13 +- net/checksum.h | 14 +- tests/Makefile | 2 +- 17 files changed, 5205 insertions(+), 8 deletions(-) create mode 100644 hw/vmware_utils.h create mode 100644 hw/vmxnet3.c create mode 100644 hw/vmxnet3.h create mode 100644 hw/vmxnet_debug.h create mode 100644 hw/vmxnet_pkt.c create mode 100644 hw/vmxnet_pkt.h create mode 100644 hw/vmxnet_utils.c create mode 100644 hw/vmxnet_utils.h
Signature · 1 line
1.7.11.7
PATCH v7 1/6 Adding utility function net_checksum_add_cont() that allows checksum calculation of scattered data with odd chunk sizes
Adding utility function net_raw_checksum() that calculates checksum of buffer given
--- net/checksum.c | 13 +++++++------ net/checksum.h | 14 +++++++++++++- 2 files changed, 20 insertions(+), 7 deletions(-)
net/checksum.c +7 -6
@@ -20,16 +20,17 @@#define PROTO_TCP 6#define PROTO_UDP 17 -uint32_t net_checksum_add (int len,uint8_t *buf) +uint32_t net_checksum_add_cont (int len,uint8_t *buf,int seq) {uint32_t sum =0 ;int i; -for (i =0 ; i < len; i++) { -if (i &1 ) - sum += (uint32_t )buf[i]; -else - sum += (uint32_t )buf[i] <<8 ; +for (i = seq; i < seq + len; i++) { +if (i &1 ) { + sum += (uint32_t )buf[i - seq]; + }else { + sum += (uint32_t )buf[i - seq] <<8 ; + } }return sum; }
net/checksum.h +13 -1
@@ -20,10 +20,22 @@#include <stdint.h> -uint32_t net_checksum_add (int len,uint8_t *buf); +uint32_t net_checksum_add_cont (int len,uint8_t *buf,int seq);uint16_t net_checksum_finish (uint32_t sum);uint16_t net_checksum_tcpudp (uint16_t length,uint16_t proto,uint8_t *addrs,uint8_t *buf);void net_checksum_calculate (uint8_t *data,int length); +static inline uint32_t +net_checksum_add (int len,uint8_t *buf) +{ +return net_checksum_add_cont (len, buf,0 ); +} + +static inline uint16_t +net_raw_checksum (uint8_t *data,int length) +{ +return net_checksum_finish (net_checksum_add (length, data)); +} +#endif /* QEMU_NET_CHECKSUM_H */
Signature · 1 line
1.7.11.7
PATCH v7 2/6 Adding utility function iov_net_csum_add() for iovec checksum calculation Adding utility function iov_rebuild() for smart iovec copy
In reply to Dmitry Fleytman--- Makefile.objs | 1 + iov.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++ iov.h | 13 +++++++++++++ tests/Makefile | 2 +- 4 files changed, 68 insertions(+), 1 deletion(-)
Makefile.objs +1 -0
@@ -43,6 +43,7 @@ coroutine-obj-$(CONFIG_WIN32) += coroutine-win32.o block-obj-y = iov.o cache-utils.o qemu-option.o module.o async.o block-obj-y += nbd.o block.o blockjob.o aes.o qemu-config.o +block-obj-y += net/checksum.o block-obj-y += thread-pool.o qemu-progress.o qemu-sockets.o uri.o notify.o block-obj-y += $(coroutine-obj-y) $(qobject-obj-y) $(version-obj-y) block-obj-$(CONFIG_POSIX) += event_notifier-posix.o aio-posix.o
iov.c +53 -0
65 lines not on this page · raw message
iov.h +13 -0
@@ -95,3 +95,16 @@ void iov_hexdump(const struct iovec *iov, const unsigned int iov_cnt,unsigned iov_copy (struct iovec *dst_iov,unsigned int dst_iov_cnt,const struct iovec *iov,unsigned int iov_cnt,size_t offset,size_t bytes); + +/** +* Copy data between scatter-gather vectors +*/ +size_t iov_rebuild (struct iovec *dest,unsigned int dest_cnt, +const struct iovec *src,const unsigned int src_cnt, +size_t src_off); + +/** +* Checksum calculation for scatter-gather vector +*/ +uint32_t iov_net_csum_add (const struct iovec *iov,const unsigned int iov_cnt, +size_t iov_off,size_t size);
tests/Makefile +1 -1
@@ -49,7 +49,7 @@ tests/check-qlist$(EXESUF): tests/check-qlist.o qlist.o qint.o tests/check-qfloat$(EXESUF): tests/check-qfloat.o qfloat.o tests/check-qjson$(EXESUF): tests/check-qjson.o $(qobject-obj-y) qemu-tool.o tests/test-coroutine$(EXESUF): tests/test-coroutine.o $(coroutine-obj-y) $(tools-obj-y) $(block-obj-y) iov.o -tests/test-iov$(EXESUF): tests/test-iov.o iov.o +tests/test-iov$(EXESUF): tests/test-iov.o iov.o net/checksum.o tests/test-qapi-types.c tests/test-qapi-types.h :\ $(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py
Signature · 1 line
1.7.11.7
Re: [Qemu-devel] [PATCH V7 2/6] Adding utility function iov_net_csum_add() for iovec checksum calculation Adding utility function iov_rebuild() for smart iovec copy
Quoted 17 lines · Dmitry Fleytman
iov.c +13
-0@@ -252,6 +253,58 @@ unsigned iov_copy(struct iovec *dst_iov, unsigned int dst_iov_cnt,return j; } +size_t iov_rebuild (struct iovec *dst,unsigned int dst_cnt, +const struct iovec *src,const unsigned int src_cnt, +size_t src_off) +{ +size_t curr_src_off =0 ; +unsigned int i, j =0 ; + +for (i =0 ; i < src_cnt; i++) { +if (src_off < (curr_src_off + src[i].iov_len)) { +if (j == dst_cnt) { +return (size_t )-1 ; + } +
iov.c unsigned iov_copy(struct iovec *dst_iov, unsigned int dst_iov_cnt, · line 278
Quoted 10 lines
iov.c +10
-0+ dst[j].iov_len = curr_src_off + src[i].iov_len - src_off; + dst[j].iov_base = src[i].iov_base + (src_off - curr_src_off); + + src_off += dst[j].iov_len; + j++; + } + curr_src_off += src[i].iov_len; + } +return j; +}
The existing iov_copy() function provides equivalent functionality. I don't think iov_rebuild() is needed.
Quoted 17 lines
iov.c +17
-0+uint32_t +iov_net_csum_add (const struct iovec *iov,const unsigned int iov_cnt, +size_t iov_off,size_t size) +{ +size_t iovec_off, buf_off; +unsigned int i; +uint32_t res =0 ; +uint32_t seq =0 ; + + iovec_off =0 ; + buf_off =0 ; +for (i =0 ; i < iov_cnt && size; i++) { +if (iov_off < (iovec_off + iov[i].iov_len)) { +size_t len =MIN ((iovec_off + iov[i].iov_len) - iov_off , size); +void *chunk_buf = iov[i].iov_base + (iov_off - iovec_off); + + res +=net_checksum_add_cont (len, chunk_buf, seq);
iov.c unsigned iov_copy(struct iovec *dst_iov, unsigned int dst_iov_cnt, · line 306
Quoted 10 lines
iov.c +10
-0+ seq += len; + + buf_off += len; + iov_off += len; + size -= len; + } + iovec_off += iov[i].iov_len; + } +return res; +}
Rename this net_checksum_add_iov() and place it in net/checksum.c, then the new dependency on net from block can be dropped.
PATCH v7 3/6 Adding common definitions for VMWARE devices
In reply to Dmitry Fleytman--- hw/vmware_utils.h | 143 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 143 insertions(+) create mode 100644 hw/vmware_utils.h
hw/vmware_utils.h new file +143 -0
@@ -0,0 +1,143 @@ +/* +* QEMU VMWARE paravirtual devices - auxiliary code +* +* Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com) +* +* Developed by Daynix Computing LTD (http://www.daynix.com) +* +* Authors: +* Dmitry Fleytman <[email protected]> +* Yan Vugenfirer <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef VMWARE_UTILS_H +#define VMWARE_UTILS_H + +#include "range.h" + +#ifndef VMW_SHPRN +#define VMW_SHPRN (fmt, ...) do {} while (0) +#endif + +/* +* Shared memory access functions with byte swap support +* Each function contains printout for reverse-engineering needs +* +*/ +static inline void +vmw_shmem_read (target_phys_addr_t addr,void *buf,int len) +{ +VMW_SHPRN ("SHMEM r: %" PRIx64", len: %d to %p" , addr, len, buf); +cpu_physical_memory_read (addr, buf, len); +} + +static inline void +vmw_shmem_write (target_phys_addr_t addr,void *buf,int len) +{ +VMW_SHPRN ("SHMEM w: %" PRIx64", len: %d to %p" , addr, len, buf); +cpu_physical_memory_write (addr, buf, len); +} + +static inline void +vmw_shmem_rw (target_phys_addr_t addr,void *buf,int len,int is_write) +{ +VMW_SHPRN ("SHMEM r/w: %" PRIx64", len: %d (to %p), is write: %d" , + addr, len, buf, is_write); + +cpu_physical_memory_rw (addr, buf, len, is_write); +} + +static inline void +vmw_shmem_set (target_phys_addr_t addr,uint8 val,int len) +{ +int i; +VMW_SHPRN ("SHMEM set: %" PRIx64", len: %d (value 0x%X)" , addr, len, val); + +for (i =0 ; i < len; i++) { +cpu_physical_memory_write (addr + i, &val,1 ); + } +} + +static inline uint32_t +vmw_shmem_ld8 (target_phys_addr_t addr) +{ +uint8_t res =ldub_phys (addr); +VMW_SHPRN ("SHMEM load8: %" PRIx64" (value 0x%X)" , addr, res); +return res; +} + +static inline void +vmw_shmem_st8 (target_phys_addr_t addr,uint8_t value) +{ +VMW_SHPRN ("SHMEM store8: %" PRIx64" (value 0x%X)" , addr, value); +stb_phys (addr, value); +} + +static inline uint32_t +vmw_shmem_ld16 (target_phys_addr_t addr) +{ +uint16_t res =lduw_le_phys (addr); +VMW_SHPRN ("SHMEM load16: %" PRIx64" (value 0x%X)" , addr, res); +return res; +} + +static inline void +vmw_shmem_st16 (target_phys_addr_t addr,uint16_t value) +{ +VMW_SHPRN ("SHMEM store16: %" PRIx64" (value 0x%X)" , addr, value); +stw_le_phys (addr, value); +} + +static inline uint32_t +vmw_shmem_ld32 (target_phys_addr_t addr) +{ +uint32_t res =ldl_le_phys (addr); +VMW_SHPRN ("SHMEM load32: %" PRIx64" (value 0x%X)" , addr, res); +return res; +} + +static inline void +vmw_shmem_st32 (target_phys_addr_t addr,uint32_t value) +{ +VMW_SHPRN ("SHMEM store32: %" PRIx64" (value 0x%X)" , addr, value); +stl_le_phys (addr, value); +} + +static inline uint64_t +vmw_shmem_ld64 (target_phys_addr_t addr) +{ +uint64_t res =ldq_le_phys (addr); +VMW_SHPRN ("SHMEM load64: %" PRIx64" (value %" PRIx64")" , addr, res); +return res; +} + +static inline void +vmw_shmem_st64 (target_phys_addr_t addr,uint64_t value) +{ +VMW_SHPRN ("SHMEM store64: %" PRIx64" (value %" PRIx64")" , addr, value); +stq_le_phys (addr, value); +} + +/* Macros for simplification of operations on array-style registers */ + +/* +* Whether <addr> lies inside of array-style register defined by <base>, +* number of elements (<cnt>) and element size (<regsize>) +* +*/ +#define VMW_IS_MULTIREG_ADDR (addr, base, cnt, regsize) \ + range_covers_byte(base, cnt * regsize, addr) + +/* +* Returns index of given register (<addr>) in array-style register defined by +* <base> and element size (<regsize>) +* +*/ +#define VMW_MULTIREG_IDX_BY_ADDR (addr, base, regsize) \ + (((addr) - (base)) / (regsize)) + +#endif
Signature · 1 line
1.7.11.7
PATCH v7 4/6 Adding common code for VMWARE network devices
In reply to Dmitry Fleytman--- hw/vmware_utils.h | 24 ++-- hw/vmxnet_debug.h | 121 +++++++++++++++++++ hw/vmxnet_utils.c | 219 +++++++++++++++++++++++++++++++++++ hw/vmxnet_utils.h | 340 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 692 insertions(+), 12 deletions(-) create mode 100644 hw/vmxnet_debug.h create mode 100644 hw/vmxnet_utils.c create mode 100644 hw/vmxnet_utils.h
hw/vmware_utils.h +12 -12
@@ -29,21 +29,21 @@* */ static inline void -vmw_shmem_read (target_phys_addr_t addr,void *buf,int len) +vmw_shmem_read (hwaddr addr,void *buf,int len) {VMW_SHPRN ("SHMEM r: %" PRIx64", len: %d to %p" , addr, len, buf);cpu_physical_memory_read (addr, buf, len); }static inline void -vmw_shmem_write (target_phys_addr_t addr,void *buf,int len) +vmw_shmem_write (hwaddr addr,void *buf,int len) {VMW_SHPRN ("SHMEM w: %" PRIx64", len: %d to %p" , addr, len, buf);cpu_physical_memory_write (addr, buf, len); }static inline void -vmw_shmem_rw (target_phys_addr_t addr,void *buf,int len,int is_write) +vmw_shmem_rw (hwaddr addr,void *buf,int len,int is_write) {VMW_SHPRN ("SHMEM r/w: %" PRIx64", len: %d (to %p), is write: %d" , addr, len, buf, is_write);
@@ -52,7 +52,7 @@ vmw_shmem_rw(target_phys_addr_t addr, void *buf, int len, int is_write) }static inline void -vmw_shmem_set (target_phys_addr_t addr,uint8 val,int len) +vmw_shmem_set (hwaddr addr,uint8 val,int len) {int i;VMW_SHPRN ("SHMEM set: %" PRIx64", len: %d (value 0x%X)" , addr, len, val);
@@ -63,7 +63,7 @@ vmw_shmem_set(target_phys_addr_t addr, uint8 val, int len) }static inline uint32_t -vmw_shmem_ld8 (target_phys_addr_t addr) +vmw_shmem_ld8 (hwaddr addr) {uint8_t res =ldub_phys (addr);VMW_SHPRN ("SHMEM load8: %" PRIx64" (value 0x%X)" , addr, res);
@@ -71,14 +71,14 @@ vmw_shmem_ld8(target_phys_addr_t addr) }static inline void -vmw_shmem_st8 (target_phys_addr_t addr,uint8_t value) +vmw_shmem_st8 (hwaddr addr,uint8_t value) {VMW_SHPRN ("SHMEM store8: %" PRIx64" (value 0x%X)" , addr, value);stb_phys (addr, value); }static inline uint32_t -vmw_shmem_ld16 (target_phys_addr_t addr) +vmw_shmem_ld16 (hwaddr addr) {uint16_t res =lduw_le_phys (addr);VMW_SHPRN ("SHMEM load16: %" PRIx64" (value 0x%X)" , addr, res);
@@ -86,14 +86,14 @@ vmw_shmem_ld16(target_phys_addr_t addr) }static inline void -vmw_shmem_st16 (target_phys_addr_t addr,uint16_t value) +vmw_shmem_st16 (hwaddr addr,uint16_t value) {VMW_SHPRN ("SHMEM store16: %" PRIx64" (value 0x%X)" , addr, value);stw_le_phys (addr, value); }static inline uint32_t -vmw_shmem_ld32 (target_phys_addr_t addr) +vmw_shmem_ld32 (hwaddr addr) {uint32_t res =ldl_le_phys (addr);VMW_SHPRN ("SHMEM load32: %" PRIx64" (value 0x%X)" , addr, res);
@@ -101,14 +101,14 @@ vmw_shmem_ld32(target_phys_addr_t addr) }static inline void -vmw_shmem_st32 (target_phys_addr_t addr,uint32_t value) +vmw_shmem_st32 (hwaddr addr,uint32_t value) {VMW_SHPRN ("SHMEM store32: %" PRIx64" (value 0x%X)" , addr, value);stl_le_phys (addr, value); }static inline uint64_t -vmw_shmem_ld64 (target_phys_addr_t addr) +vmw_shmem_ld64 (hwaddr addr) {uint64_t res =ldq_le_phys (addr);VMW_SHPRN ("SHMEM load64: %" PRIx64" (value %" PRIx64")" , addr, res);
@@ -116,7 +116,7 @@ vmw_shmem_ld64(target_phys_addr_t addr) }static inline void -vmw_shmem_st64 (target_phys_addr_t addr,uint64_t value) +vmw_shmem_st64 (hwaddr addr,uint64_t value) {VMW_SHPRN ("SHMEM store64: %" PRIx64" (value %" PRIx64")" , addr, value);stq_le_phys (addr, value);
hw/vmxnet_debug.h new file +121 -0
121 lines not on this page · raw message
hw/vmxnet_utils.c new file +219 -0
219 lines not on this page · raw message
hw/vmxnet_utils.h new file +340 -0
340 lines not on this page · raw message
Signature · 1 line
1.7.11.7
Re: [Qemu-devel] [PATCH V7 4/6] Adding common code for VMWARE network devices
Quoted 17 lines · Dmitry Fleytman
Signed-off-by Dmitry Fleytman <[email protected]>Signed-off-by Yan Vugenfirer <[email protected]>--- hw/vmware_utils.h | 24 ++-- hw/vmxnet_debug.h | 121 +++++++++++++++++++ hw/vmxnet_utils.c | 219 +++++++++++++++++++++++++++++++++++ hw/vmxnet_utils.h | 340 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 692 insertions(+), 12 deletions(-) create mode 100644 hw/vmxnet_debug.h create mode 100644 hw/vmxnet_utils.c create mode 100644 hw/vmxnet_utils.hhw/vmware_utils.h +0
-0
hw/vmware_utils.h line 36
Quoted 9 lines
hw/vmware_utils.h +1
-1@@ -29,21 +29,21 @@* */ static inline void -vmw_shmem_read (target_phys_addr_t addr,void *buf,int len) +vmw_shmem_read (hwaddr addr,void *buf,int len) {VMW_SHPRN ("SHMEM r: %" PRIx64", len: %d to %p" , addr, len, buf);cpu_physical_memory_read (addr, buf, len); }
All changes to this file should be squashed with the previous patch.
Quoted 8 lines
hw/vmware_utils.h +8
-0+/* #define VMXNET_DEBUG_CB */ +#define VMXNET_DEBUG_WARNINGS +#define VMXNET_DEBUG_ERRORS +/* #define VMXNET_DEBUG_INTERRUPTS */ +/* #define VMXNET_DEBUG_CONFIG */ +/* #define VMXNET_DEBUG_RINGS */ +/* #define VMXNET_DEBUG_PACKETS */ +/* #define VMXNET_DEBUG_SHMEM_ACCESS */
hw/vmware_utils.h
Quoted 10 lines
hw/vmware_utils.h +10
-0+ +#ifdef VMXNET_DEBUG_SHMEM_ACCESS +#define VMW_SHPRN (fmt, ...) \ + do { \ + printf("[%s][SH][%s]: " fmt "\n", VMXNET_DEVICE_NAME, __func__, \ + ## __VA_ARGS__); \ + } while (0) +#else +#define VMW_SHPRN (fmt, ...) do {} while (0) +#endif
Please use QEMU tracing. It eliminates all this boilerplate and conditional compilation. Tracing can be enabled/disabled at runtime and works with SystemTap/DTrace. See docs/tracing.txt.
Quoted 27 lines
hw/vmxnet_utils.h new file +21
-0@@ -0,0 +1,340 @@ +/* +* QEMU VMWARE VMXNET* paravirtual NICs - network auxiliary code +* +* Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com) +* +* Developed by Daynix Computing LTD (http://www.daynix.com) +* +* Authors: +* Dmitry Fleytman <[email protected]> +* Tamir Shomer <[email protected]> +* Yan Vugenfirer <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#ifndef _VMXNET_UTILS_H_ +#define _VMXNET_UTILS_H_ + +#include <sys/types.h>
hw/vmxnet_utils.h line 31
Quoted 10 lines
hw/vmxnet_utils.h new file +10
-0+#include <bswap.h> +#include <string.h> +#include <hwaddr.h> +#include <hw/virtio-net.h> + +struct eth_header { +uint8_t h_dest[ETH_ALEN ];/* destination eth addr */ +uint8_t h_source[ETH_ALEN ];/* source ether addr */ +uint16_t h_proto;/* packet type ID field */ +};
Looks like it's copy-pasted stuff from /usr/include/linux/if_*.h, /usr/include/netinet/*.h, and friends. If the system-wide headers are included names will collide for some of the macros at least.
Did you check if the slirp/ definitions can be reused?
I'd rather we import network header definitions once in a generic place into the source tree. That way vmxnet and other components don't need to redefine these structs.
PATCH v7 5/6 Adding packet abstraction for VMWARE network devices
In reply to Dmitry Fleytman--- hw/vmxnet_pkt.c | 776 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/vmxnet_pkt.h | 311 ++++++++++++++++++++++ hw/vmxnet_utils.c | 6 +- hw/vmxnet_utils.h | 6 +- 4 files changed, 1093 insertions(+), 6 deletions(-) create mode 100644 hw/vmxnet_pkt.c create mode 100644 hw/vmxnet_pkt.h
hw/vmxnet_pkt.c new file +776 -0
@@ -0,0 +1,776 @@ +/* +* QEMU VMWARE VMXNET* paravirtual NICs - packets abstractions +* +* Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com) +* +* Developed by Daynix Computing LTD (http://www.daynix.com) +* +* Authors: +* Dmitry Fleytman <[email protected]> +* Tamir Shomer <[email protected]> +* Yan Vugenfirer <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory. +* +*/ + +#include "vmxnet_pkt.h" +#include "vmxnet_utils.h" +#include "iov.h" + +#include "net/checksum.h" + +/*============================================================================= +*============================================================================= +* +* TX CODE +* +*============================================================================= +*===========================================================================*/ + +enum { +VMXNET_TX_PKT_VHDR_FRAG =0 , +VMXNET_TX_PKT_L2HDR_FRAG , +VMXNET_TX_PKT_L3HDR_FRAG , +VMXNET_TX_PKT_PL_START_FRAG +}; + +/* TX packet private context */ +typedef struct { +struct virtio_net_hdr virt_hdr; +bool has_virt_hdr; + +struct iovec *raw; +uint32_t raw_frags; +uint32_t max_raw_frags; + +struct iovec *vec; + +uint8_t l2_hdr[ETH_MAX_L2_HDR_LEN ]; + +uint32_t payload_len; + +uint32_t payload_frags; +uint32_t max_payload_frags; + +uint16_t hdr_len; +eth_pkt_types_e packet_type; +uint8_t l4proto; +}VmxnetTxPkt ; + +void vmxnet_tx_pkt_init (VmxnetTxPktH *pkt,uint32_t max_frags, +bool has_virt_hdr) +{ +VmxnetTxPkt *p =g_malloc0 (sizeof *p); + + p->vec =g_malloc ((sizeof *p->vec) * + (max_frags +VMXNET_TX_PKT_PL_START_FRAG )); + + p->raw =g_malloc ((sizeof *p->raw) * max_frags); + + p->max_payload_frags = max_frags; + p->max_raw_frags = max_frags; + p->has_virt_hdr = has_virt_hdr; + p->vec[VMXNET_TX_PKT_VHDR_FRAG ].iov_base = &p->virt_hdr; + p->vec[VMXNET_TX_PKT_VHDR_FRAG ].iov_len = + p->has_virt_hdr ?sizeof p->virt_hdr :0 ; + p->vec[VMXNET_TX_PKT_L2HDR_FRAG ].iov_base = &p->l2_hdr; + p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_base =NULL ; + p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_len =0 ; + + *pkt = p; +} + +void vmxnet_tx_pkt_uninit (VmxnetTxPktH pkt) +{ +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; + +if (p) { +if (p->vec) { +g_free (p->vec); + } + +if (p->raw) { +g_free (p->raw); + } + +g_free (p); + } +} + +void vmxnet_tx_pkt_update_ip_checksums (VmxnetTxPktH pkt) +{ +uint16_t csum; +uint32_t ph_raw_csum; +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +assert (p); +uint8_t gso_type = p->virt_hdr.gso_type & ~VIRTIO_NET_HDR_GSO_ECN ; +struct ip_header *ip_hdr; + +if (VIRTIO_NET_HDR_GSO_TCPV4 != gso_type && +VIRTIO_NET_HDR_GSO_UDP != gso_type) { +return ; + } + + ip_hdr = p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_base; + +if (p->payload_len + p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_len > +ETH_MAX_IP_DGRAM_LEN ) { +return ; + } + + ip_hdr->ip_len =cpu_to_be16 (p->payload_len + + p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_len); + +/* Calculate IP header checksum */ + ip_hdr->ip_sum =0 ; + csum =net_raw_checksum ((uint8_t *)ip_hdr, + p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_len); + ip_hdr->ip_sum =cpu_to_be16 (csum); + +/* Calculate IP pseudo header checksum */ + ph_raw_csum =eth_calc_pseudo_hdr_csum (ip_hdr, p->payload_len); + csum =cpu_to_be16 (net_checksum_finish (ph_raw_csum)); +iov_from_buf (&p->vec[VMXNET_TX_PKT_PL_START_FRAG ], p->payload_frags, + p->virt_hdr.csum_offset, &csum,sizeof (csum)); +} + +static void vmxnet_tx_pkt_calculate_hdr_len (VmxnetTxPkt *p) +{ + p->hdr_len = p->vec[VMXNET_TX_PKT_L2HDR_FRAG ].iov_len + + p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_len; +} + +static bool vmxnet_tx_pkt_parse_headers (VmxnetTxPktH pkt) +{ +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +struct iovec *l2_hdr, *l3_hdr; +size_t bytes_read; +size_t full_ip6hdr_len; +uint16_t l3_proto; + +assert (p); + + l2_hdr = &p->vec[VMXNET_TX_PKT_L2HDR_FRAG ]; + l3_hdr = &p->vec[VMXNET_TX_PKT_L3HDR_FRAG ]; + + bytes_read =iov_to_buf (p->raw, p->raw_frags,0 , l2_hdr->iov_base, +ETH_MAX_L2_HDR_LEN ); +if (bytes_read <ETH_MAX_L2_HDR_LEN ) { + l2_hdr->iov_len =0 ; +return false; + }else { + l2_hdr->iov_len =eth_get_l2_hdr_length (l2_hdr->iov_base); + } + + l3_proto =eth_get_l3_proto (l2_hdr->iov_base, l2_hdr->iov_len); + +switch (l3_proto) { +case ETH_P_IP : + l3_hdr->iov_base =g_malloc (ETH_MAX_IP4_HDR_LEN ); + + bytes_read =iov_to_buf (p->raw, p->raw_frags, l2_hdr->iov_len, + l3_hdr->iov_base,sizeof (struct ip_header )); + +if (bytes_read <sizeof (struct ip_header )) { + l3_hdr->iov_len =0 ; +return false; + } + + l3_hdr->iov_len =IP_HDR_GET_LEN (l3_hdr->iov_base); + p->l4proto = ((struct ip_header *) l3_hdr->iov_base)->ip_p; + +/* copy optional IPv4 header data */ + bytes_read =iov_to_buf (p->raw, p->raw_frags, + l2_hdr->iov_len +sizeof (struct ip_header ), + l3_hdr->iov_base +sizeof (struct ip_header ), + l3_hdr->iov_len -sizeof (struct ip_header )); +if (bytes_read < l3_hdr->iov_len -sizeof (struct ip_header )) { + l3_hdr->iov_len =0 ; +return false; + } +break ; + +case ETH_P_IPV6 : +if (!eth_parse_ipv6_hdr (p->raw, p->raw_frags, l2_hdr->iov_len, + &p->l4proto, &full_ip6hdr_len)) { + l3_hdr->iov_len =0 ; +return false; + } + + l3_hdr->iov_base =g_malloc (full_ip6hdr_len); + + bytes_read =iov_to_buf (p->raw, p->raw_frags, l2_hdr->iov_len, + l3_hdr->iov_base, full_ip6hdr_len); + +if (bytes_read < full_ip6hdr_len) { + l3_hdr->iov_len =0 ; +return false; + }else { + l3_hdr->iov_len = full_ip6hdr_len; + } +break ; + +default : + l3_hdr->iov_len =0 ; +break ; + } + +vmxnet_tx_pkt_calculate_hdr_len (p); + p->packet_type =get_eth_packet_type (l2_hdr->iov_base); +return true; +} + +static bool vmxnet_tx_pkt_rebuild_payload (VmxnetTxPktH pkt) +{ +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; + + p->payload_frags =iov_rebuild (&p->vec[VMXNET_TX_PKT_PL_START_FRAG ], + p->max_payload_frags, + p->raw, p->raw_frags, + p->hdr_len); + +if (p->payload_frags != (uint32_t )-1 ) { + p->payload_len =iov_size (&p->vec[VMXNET_TX_PKT_PL_START_FRAG ], + p->payload_frags); +return true; + }else { +return false; + } +} + +bool vmxnet_tx_pkt_parse (VmxnetTxPktH pkt) +{ +return vmxnet_tx_pkt_parse_headers (pkt) && +vmxnet_tx_pkt_rebuild_payload (pkt); +} + +struct virtio_net_hdr *vmxnet_tx_pkt_get_vhdr (VmxnetTxPktH pkt) +{ +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +assert (p); +return &p->virt_hdr; +} + +static uint8_t vmxnet_tx_pkt_get_gso_type (VmxnetTxPkt *p,bool tso_enable) +{ +uint8_t rc =VIRTIO_NET_HDR_GSO_NONE ; +uint16_t l3_proto; + + l3_proto =eth_get_l3_proto (p->vec[VMXNET_TX_PKT_L2HDR_FRAG ].iov_base, + p->vec[VMXNET_TX_PKT_L2HDR_FRAG ].iov_len); + +if (!tso_enable) { + goto func_exit; + } + + rc =eth_get_gso_type (l3_proto, p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_base, + p->l4proto); + +func_exit: +return rc; +} + +void vmxnet_tx_pkt_build_vheader (VmxnetTxPktH pkt,bool tso_enable, +bool csum_enable,uint32_t gso_size) +{ +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +struct tcp_hdr l4hdr; +assert (p); + +/* csum has to be enabled if tso is. */ +assert (csum_enable || !tso_enable); + + p->virt_hdr.gso_type =vmxnet_tx_pkt_get_gso_type (p, tso_enable); + +switch (p->virt_hdr.gso_type & ~VIRTIO_NET_HDR_GSO_ECN ) { +case VIRTIO_NET_HDR_GSO_NONE : + p->virt_hdr.hdr_len =0 ; + p->virt_hdr.gso_size =0 ; +break ; + +case VIRTIO_NET_HDR_GSO_UDP : + p->virt_hdr.gso_size =IP_FRAG_ALIGN_SIZE (gso_size); + p->virt_hdr.hdr_len = p->hdr_len +sizeof (struct udp_header ); +break ; + +case VIRTIO_NET_HDR_GSO_TCPV4 : +case VIRTIO_NET_HDR_GSO_TCPV6 : +iov_to_buf (&p->vec[VMXNET_TX_PKT_PL_START_FRAG ], p->payload_frags, +0 , &l4hdr,sizeof (l4hdr)); + p->virt_hdr.hdr_len = p->hdr_len + l4hdr.th_off *sizeof (uint32_t ); + p->virt_hdr.gso_size =IP_FRAG_ALIGN_SIZE (gso_size); +break ; + +default : +assert (false); + } + +if (csum_enable) { +switch (p->l4proto) { +case IP_PROTO_TCP : + p->virt_hdr.flags =VIRTIO_NET_HDR_F_NEEDS_CSUM ; + p->virt_hdr.csum_start = p->hdr_len; + p->virt_hdr.csum_offset = offsetof(struct tcp_hdr , th_sum); +break ; +case IP_PROTO_UDP : + p->virt_hdr.flags =VIRTIO_NET_HDR_F_NEEDS_CSUM ; + p->virt_hdr.csum_start = p->hdr_len; + p->virt_hdr.csum_offset = offsetof(struct udp_hdr , uh_sum); +break ; +default : +break ; + } + } +} + +void vmxnet_tx_pkt_setup_vlan_header (VmxnetTxPktH pkt,uint16_t vlan) +{ +bool is_new; +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +assert (p); + +eth_setup_vlan_headers (p->vec[VMXNET_TX_PKT_L2HDR_FRAG ].iov_base, + vlan, &is_new); + +/* update l2hdrlen */ +if (is_new) { + p->hdr_len +=sizeof (struct vlan_header ); + p->vec[VMXNET_TX_PKT_L2HDR_FRAG ].iov_len +=sizeof (struct vlan_header ); + } +} + +bool vmxnet_tx_pkt_add_raw_fragment (VmxnetTxPktH pkt,hwaddr pa, +size_t len) +{ +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +hwaddr mapped_len =0 ; +struct iovec *ventry; +assert (p); +assert (p->max_raw_frags > p->raw_frags); + +if (!len) { +return true; + } + + ventry = &p->raw[p->raw_frags]; + mapped_len = len; + + ventry->iov_base =cpu_physical_memory_map (pa, &mapped_len, false); + ventry->iov_len = mapped_len; + p->raw_frags += !!ventry->iov_base; + +if ((ventry->iov_base ==NULL ) || (len != mapped_len)) { +return false; + } + +return true; +} + +eth_pkt_types_e vmxnet_tx_pkt_get_packet_type (VmxnetTxPktH pkt) +{ +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +assert (p); + +return p->packet_type; +} + +size_t vmxnet_tx_pkt_get_total_len (VmxnetTxPktH pkt) +{ +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +assert (p); + +return p->hdr_len + p->payload_len; +} + +void vmxnet_tx_pkt_dump (VmxnetTxPktH pkt) +{ +#ifdef VMXNET_TX_PKT_DEBUG +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +assert (p); + +printf ("TX PKT: hdr_len: %d, pkt_type: 0x%X, l2hdr_len: %lu, " +"l3hdr_len: %lu, payload_len: %u\n" , p->hdr_len, p->packet_type, + p->vec[VMXNET_TX_PKT_L2HDR_FRAG ].iov_len, + p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_len, p->payload_len); +#endif +} + +void vmxnet_tx_pkt_reset (VmxnetTxPktH pkt) +{ +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +int i; + +/* no assert, as reset can be called before tx_pkt_init */ +if (!p) { +return ; + } + +memset (&p->virt_hdr,0 ,sizeof (p->virt_hdr)); + +if (NULL != p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_base) { +g_free (p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_base); + p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_base =NULL ; + } + +assert (p->vec); +for (i =VMXNET_TX_PKT_L2HDR_FRAG ; + i < p->payload_frags +VMXNET_TX_PKT_PL_START_FRAG ; i++) { + p->vec[i].iov_len =0 ; + } + p->payload_len =0 ; + p->payload_frags =0 ; + +assert (p->raw); +for (i =0 ; i < p->raw_frags; i++) { +assert (p->raw[i].iov_base); +cpu_physical_memory_unmap (p->raw[i].iov_base, p->raw[i].iov_len, + false, p->raw[i].iov_len); + p->raw[i].iov_len =0 ; + } + p->raw_frags =0 ; + + p->hdr_len =0 ; + p->packet_type =0 ; + p->l4proto =0 ; +} + +static void vmxnet_tx_pkt_do_sw_csum (VmxnetTxPkt *p) +{ +struct iovec *iov = &p->vec[VMXNET_TX_PKT_L2HDR_FRAG ]; +uint32_t csum_cntr; +uint16_t csum =0 ; +/* num of iovec without vhdr */ +uint32_t iov_len = p->payload_frags +VMXNET_TX_PKT_PL_START_FRAG -1 ; +uint16_t csl; +struct ip_header *iphdr; +size_t csum_offset = p->virt_hdr.csum_start + p->virt_hdr.csum_offset; + +/* Put zero to checksum field */ +iov_from_buf (iov, iov_len, csum_offset, &csum,sizeof csum); + +/* Calculate L4 TCP/UDP checksum */ + csl = p->payload_len; + +/* data checksum */ + csum_cntr =iov_net_csum_add (iov, iov_len, p->virt_hdr.csum_start, csl); +/* add pseudo header to csum */ + iphdr = p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_base; + csum_cntr +=eth_calc_pseudo_hdr_csum (iphdr, csl); + +/* Put the checksum obtained into the packet */ + csum =cpu_to_be16 (net_checksum_finish (csum_cntr)); +iov_from_buf (iov, iov_len, csum_offset, &csum,sizeof csum); +} + +enum { +VMXNET_TX_PKT_FRAGMENT_L2_HDR_POS =0 , +VMXNET_TX_PKT_FRAGMENT_L3_HDR_POS , +VMXNET_TX_PKT_FRAGMENT_HEADER_NUM +}; + +#define VMXNET_MAX_FRAG_SG_LIST (64) + +static size_t vmxnet_tx_pkt_fetch_fragment (VmxnetTxPkt *p,int *src_idx, +size_t *src_offset,struct iovec *dst,int *dst_idx) +{ +size_t fetched =0 ; +struct iovec *src = p->vec; + + *dst_idx =VMXNET_TX_PKT_FRAGMENT_HEADER_NUM ; + +while (fetched < p->virt_hdr.gso_size) { + +/* no more place in fragment iov */ +if (*dst_idx ==VMXNET_MAX_FRAG_SG_LIST ) { +break ; + } + +/* no more data in iovec */ +if (*src_idx == (p->payload_frags +VMXNET_TX_PKT_PL_START_FRAG )) { +break ; + } + + + dst[*dst_idx].iov_base = src[*src_idx].iov_base + *src_offset; + dst[*dst_idx].iov_len =MIN (src[*src_idx].iov_len - *src_offset, + p->virt_hdr.gso_size - fetched); + + *src_offset += dst[*dst_idx].iov_len; + fetched += dst[*dst_idx].iov_len; + +if (*src_offset == src[*src_idx].iov_len) { + *src_offset =0 ; + (*src_idx)++; + } + + (*dst_idx)++; + } + +return fetched; +} + +static size_t vmxnet_tx_pkt_do_sw_fragmentation (VmxnetTxPkt *p, +NetClientState *vc) +{ +struct iovec fragment[VMXNET_MAX_FRAG_SG_LIST ]; +size_t fragment_len =0 ; +bool more_frags = false; +/* some poiners for shorter code */ +void *l2_iov_base, *l3_iov_base; +size_t l2_iov_len, l3_iov_len; +int src_idx =VMXNET_TX_PKT_PL_START_FRAG , dst_idx; +size_t src_offset =0 ; +size_t bytes_sent =0 ; +size_t fragment_offset =0 ; + + l2_iov_base = p->vec[VMXNET_TX_PKT_L2HDR_FRAG ].iov_base; + l2_iov_len = p->vec[VMXNET_TX_PKT_L2HDR_FRAG ].iov_len; + l3_iov_base = p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_base; + l3_iov_len = p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_len; + +/* Copy headers */ + fragment[VMXNET_TX_PKT_FRAGMENT_L2_HDR_POS ].iov_base = l2_iov_base; + fragment[VMXNET_TX_PKT_FRAGMENT_L2_HDR_POS ].iov_len = l2_iov_len; + fragment[VMXNET_TX_PKT_FRAGMENT_L3_HDR_POS ].iov_base = l3_iov_base; + fragment[VMXNET_TX_PKT_FRAGMENT_L3_HDR_POS ].iov_len = l3_iov_len; + + +/* Put as much data as possible and send */ +do { + fragment_len =vmxnet_tx_pkt_fetch_fragment (p, &src_idx, &src_offset, + fragment, &dst_idx); + + more_frags = (fragment_offset + fragment_len < p->payload_len); + +eth_setup_ip4_fragmentation (l2_iov_base, l2_iov_len, l3_iov_base, + l3_iov_len, fragment_len, fragment_offset, more_frags); + +eth_fix_ip4_checksum (l3_iov_base, l3_iov_len); + + bytes_sent +=qemu_sendv_packet (vc, fragment, dst_idx); + + fragment_offset += fragment_len; + + }while (more_frags); + +return bytes_sent; +} + +size_t vmxnet_tx_pkt_send (VmxnetTxPktH pkt,NetClientState *vc) +{ +size_t bytes_sent =0 ; +VmxnetTxPkt *p = (VmxnetTxPkt *)pkt; +assert (p); + +if (!p->has_virt_hdr && p->virt_hdr.flags &VIRTIO_NET_HDR_F_NEEDS_CSUM ) { +vmxnet_tx_pkt_do_sw_csum (p); + } + +/* +* Since underlying infrastructure does not support IP datagrams longer +* than 64K we should drop such packets and don't even try to send +*/ +if (VIRTIO_NET_HDR_GSO_NONE != p->virt_hdr.gso_type) { +if (p->payload_len > +ETH_MAX_IP_DGRAM_LEN - p->vec[VMXNET_TX_PKT_L3HDR_FRAG ].iov_len) { +return 0 ; + } + } + +if (p->has_virt_hdr || p->virt_hdr.gso_type ==VIRTIO_NET_HDR_GSO_NONE ) { + bytes_sent =qemu_sendv_packet (vc, p->vec, + p->payload_frags +VMXNET_TX_PKT_PL_START_FRAG ); + goto func_exit; + } + + bytes_sent =vmxnet_tx_pkt_do_sw_fragmentation (p, vc); + +func_exit: +return bytes_sent; +} + +/*============================================================================= +*============================================================================= +* +* RX CODE +* +*============================================================================= +*===========================================================================*/ + +/* +* RX packet may contain up to 2 fragments - rebuilt eth header +* in case of VLAN tag stripping +* and payload received from QEMU - in any case +*/ +#define VMXNET_MAX_RX_PACKET_FRAGMENTS (2) + +typedef struct { +struct virtio_net_hdr virt_hdr; +uint8_t ehdr_buf[ETH_MAX_L2_HDR_LEN ]; +struct iovec vec[VMXNET_MAX_RX_PACKET_FRAGMENTS ]; +uint16_t vec_len; +uint32_t tot_len; +uint16_t tci; +bool vlan_stripped; +bool has_virt_hdr; +eth_pkt_types_e packet_type; + +/* Analysis results */ +bool isip4; +bool isip6; +bool isudp; +bool istcp; +}VmxnetRxPkt ; + +void vmxnet_rx_pkt_init (VmxnetRxPktH *pkt,bool has_virt_hdr) +{ +VmxnetRxPkt *p =g_malloc0 (sizeof *p); + p->has_virt_hdr = has_virt_hdr; + *pkt = p; +} + +void vmxnet_rx_pkt_uninit (VmxnetRxPktH pkt) +{ +if (pkt) { +g_free (pkt); + } +} + +struct virtio_net_hdr *vmxnet_rx_pkt_get_vhdr (VmxnetRxPktH pkt) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); +return &p->virt_hdr; +} + +void vmxnet_rx_pkt_attach_data (VmxnetRxPktH pkt,const void *data,size_t len, +bool strip_vlan) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +uint16_t tci =0 ; +uint16_t ploff; +assert (p); + p->vlan_stripped = false; + +if (strip_vlan) { + p->vlan_stripped =eth_strip_vlan (data, p->ehdr_buf, &ploff, &tci); + } + +if (p->vlan_stripped) { + p->vec[0 ].iov_base = p->ehdr_buf; + p->vec[0 ].iov_len = ploff -sizeof (struct vlan_header ); + p->vec[1 ].iov_base = (uint8_t *) data + ploff; + p->vec[1 ].iov_len = len - ploff; + p->vec_len =2 ; + p->tot_len = len - ploff +sizeof (struct eth_header ); + }else { + p->vec[0 ].iov_base = (void *)data; + p->vec[0 ].iov_len = len; + p->vec_len =1 ; + p->tot_len = len; + } + + p->tci = tci; + +eth_get_protocols (data, len, &p->isip4, &p->isip6, &p->isudp, &p->istcp); +} + +void vmxnet_rx_pkt_dump (VmxnetRxPktH pkt) +{ +#ifdef VMXNET_RX_PKT_DEBUG +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + +printf ("RX PKT: tot_len: %d, vlan_stripped: %d, vlan_tag: %d\n" , + p->tot_len, p->vlan_stripped, p->tci); +#endif +} + +void vmxnet_rx_pkt_set_packet_type (VmxnetRxPktH pkt, +eth_pkt_types_e packet_type) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + + p->packet_type = packet_type; + +} + +eth_pkt_types_e vmxnet_rx_pkt_get_packet_type (VmxnetRxPktH pkt) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + +return p->packet_type; +} + +size_t vmxnet_rx_pkt_get_total_len (VmxnetRxPktH pkt) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + +return p->tot_len; +} + +void vmxnet_rx_pkt_get_protocols (VmxnetRxPktH pkt, +bool *isip4,bool *isip6, +bool *isudp,bool *istcp) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + + *isip4 = p->isip4; + *isip6 = p->isip6; + *isudp = p->isudp; + *istcp = p->istcp; +} + +struct iovec *vmxnet_rx_pkt_get_iovec (VmxnetRxPktH pkt) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + +return p->vec; +} + +void vmxnet_rx_pkt_set_vhdr (VmxnetRxPktH pkt,struct virtio_net_hdr *vhdr) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + +memcpy (&p->virt_hdr, vhdr,sizeof p->virt_hdr); +} + +bool vmxnet_rx_pkt_is_vlan_stripped (VmxnetRxPktH pkt) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + +return p->vlan_stripped; +} + +bool vmxnet_rx_pkt_has_virt_hdr (VmxnetRxPktH pkt) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + +return p->has_virt_hdr; +} + +uint16_t vmxnet_rx_pkt_get_num_frags (VmxnetRxPktH pkt) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + +return p->vec_len; +} + +uint16_t vmxnet_rx_pkt_get_vlan_tag (VmxnetRxPktH pkt) +{ +VmxnetRxPkt *p = (VmxnetRxPkt *)pkt; +assert (p); + +return p->tci; +}
hw/vmxnet_pkt.h new file +311 -0
311 lines not on this page · raw message
hw/vmxnet_utils.c +3 -3
@@ -6,9 +6,9 @@* Developed by Daynix Computing LTD (http://www.daynix.com) * * Authors: -* Dmitry Fleytman <address@hidden> -* Tamir Shomer <address@hidden> -* Yan Vugenfirer <address@hidden> +* Dmitry Fleytman <[email protected]> +* Tamir Shomer <[email protected]> +* Yan Vugenfirer <[email protected]> * * This work is licensed under the terms of the GNU GPL, version 2 or later. * See the COPYING file in the top-level directory.
hw/vmxnet_utils.h +3 -3
@@ -15,8 +15,8 @@* */ -#ifndef _VMXNET_UTILS_H_ -#define _VMXNET_UTILS_H_ +#ifndef VMXNET_UTILS_H +#define VMXNET_UTILS_H #include <sys/types.h> #include <bswap.h>
@@ -181,7 +181,7 @@ struct tcp_hdr { (((x) == 0) || ((x) == 0xFFF))#define ETH_MAX_L2_HDR_LEN \ - (sizeof(struct eth_header) + 2*sizeof(struct vlan_header)) + (sizeof(struct eth_header) + 2 * sizeof(struct vlan_header))#define ETH_MAX_IP4_HDR_LEN (60)#define ETH_MAX_IP_DGRAM_LEN (0xFFFF)
Signature · 1 line
1.7.11.7
Re: [Qemu-devel] [PATCH V7 5/6] Adding packet abstraction for VMWARE network devices
Quoted 7 lines · Dmitry Fleytman
+/*=============================================================================
+ *=============================================================================
+ *
+ * TX CODE
+ *
+ *=============================================================================
+ *===========================================================================*/
Is this huge comment box a sign that the code should be split into a foo_tx.c and an foo_rx.c file?
Quoted 1 line
+size_t
vmxnet_tx_pkt_send(VmxnetTxPktH pkt, NetClientState *vc)
'vc' is an old name that was used for VLANClientState. The struct has since been renamed to NetClientState and the rest of QEMU uses 'nc' instead of 'vc'.
hw/vmxnet_pkt.h line 38
Quoted 2 lines
hw/vmxnet_pkt.h new file +2
-0+/* tx module context handle */ +typedef void *VmxnetTxPktH ;
Forward-declaring the struct is nicer:
typedef struct VmxnetTxPkt VmxnetTxPkt;
The definition of VmxnetTxPkt is still hidden from the caller but you avoid the void* and casting. In vmxnet_pkt.c define using:
struct VmxnetTxPkt {
...
};Quoted 7 lines
hw/vmxnet_utils.c +0
-0@@ -6,9 +6,9 @@* Developed by Daynix Computing LTD (http://www.daynix.com) *
hw/vmxnet_utils.c line 14
Quoted 10 lines
hw/vmxnet_utils.c +3
-3* Authors: -* Dmitry Fleytman <address@hidden> -* Tamir Shomer <address@hidden> -* Yan Vugenfirer <address@hidden> +* Dmitry Fleytman <[email protected]> +* Tamir Shomer <[email protected]> +* Yan Vugenfirer <[email protected]> * * This work is licensed under the terms of the GNU GPL, version 2 or later. * See the COPYING file in the top-level directory.
Please squash these fixes into the previous patch.
hw/vmxnet_utils.h
Quoted 4 lines
hw/vmxnet_utils.h +0
-0
Please squash these fixes into the previous patch.
PATCH v7 6/6 Adding VMXNET3 device implementation
In reply to Dmitry Fleytman--- default-configs/pci.mak | 1 + hw/Makefile.objs | 1 + hw/pci.h | 1 + hw/vmxnet3.c | 2442 +++++++++++++++++++++++++++++++++++++++++++++++ hw/vmxnet3.h | 762 +++++++++++++++ 5 files changed, 3207 insertions(+) create mode 100644 hw/vmxnet3.c create mode 100644 hw/vmxnet3.h
default-configs/pci.mak +1 -0
@@ -13,6 +13,7 @@ CONFIG_LSI_SCSI_PCI=y CONFIG_MEGASAS_SCSI_PCI=y CONFIG_RTL8139_PCI=y CONFIG_E1000_PCI=y +CONFIG_VMXNET3_PCI=y CONFIG_IDE_CORE=y CONFIG_IDE_QDEV=y CONFIG_IDE_PCI=y
hw/Makefile.objs +1 -0
@@ -108,6 +108,7 @@ common-obj-$(CONFIG_PCNET_PCI) += pcnet-pci.o common-obj-$(CONFIG_PCNET_COMMON) += pcnet.o common-obj-$(CONFIG_E1000_PCI) += e1000.o common-obj-$(CONFIG_RTL8139_PCI) += rtl8139.o +common-obj-$(CONFIG_VMXNET3_PCI) += vmxnet3.o vmxnet_utils.o vmxnet_pkt.o common-obj-$(CONFIG_SMC91C111) += smc91c111.o common-obj-$(CONFIG_LAN9118) += lan9118.o
hw/pci.h +1 -0
@@ -60,6 +60,7 @@#define PCI_DEVICE_ID_VMWARE_NET 0x0720#define PCI_DEVICE_ID_VMWARE_SCSI 0x0730#define PCI_DEVICE_ID_VMWARE_IDE 0x1729 +#define PCI_DEVICE_ID_VMWARE_VMXNET3 0x07B0/* Intel (0x8086) */ #define PCI_DEVICE_ID_INTEL_82551IT 0x1209
hw/vmxnet3.c new file +2442 -0
2,442 lines not on this page · raw message
hw/vmxnet3.h new file +762 -0
537 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
1.7.11.7
Part of the diff is not shown here. View the raw message.
Re: [Qemu-devel] [PATCH V7 6/6] Adding VMXNET3 device implementation
hw/vmxnet3.c line 249
Quoted 4 lines · Dmitry Fleytman
hw/vmxnet3.c new file +4
-0+/* Whether MSI-X support was installed successfully */ +uint8_t msix_used; +/* Whether MSI support was installed successfully */ +uint8_t msi_used;
These two fields should be bool.
hw/vmxnet3.c line 277
Quoted 2 lines
hw/vmxnet3.c new file +2
-0+/* Whether automatic interrupts masking enabled */ +uint8_t auto_int_masking;
bool
hw/vmxnet3.c line 491
Quoted 9 lines
hw/vmxnet3.c new file +9
-0+static inline void vmxnet3_flush_shmem_changes (void ) +{ +/* +* Flush shared memory changes +* Needed before sending interrupt to guest to ensure +* it gets consistent memory state +*/ +smp_wmb (); +}
It's useful to document why a memory barrier is being used in each instance. Therefore hiding smp_wmb() inside a wrapper function isn't great.
Also, it's suspicious that smb_wmb() is used but no other barriers are used. What about a read memory barrier when accessing shared memory written by the guest?
Re: [Qemu-devel] [PATCH V7 6/6] Adding VMXNET3 device implementation
Hello, Stefan
Thanks for the review.
I'll go over it later this week and prepare new patches.
Dmitry.
Quoted 32 lines · Stefan Hajnoczi
hw/vmxnet3.c line 249
Quoted 4 lines · Dmitry Fleytman
hw/vmxnet3.c new file +4
-0+/* Whether MSI-X support was installed successfully */ +uint8_t msix_used; +/* Whether MSI support was installed successfully */ +uint8_t msi_used;These two fields should be bool.
hw/vmxnet3.c line 277
Quoted 2 lines
hw/vmxnet3.c new file +2
-0+/* Whether automatic interrupts masking enabled */ +uint8_t auto_int_masking;bool
hw/vmxnet3.c line 491
Quoted 9 lines
hw/vmxnet3.c new file +9
-0+static inline void vmxnet3_flush_shmem_changes (void ) +{ +/* +* Flush shared memory changes +* Needed before sending interrupt to guest to ensure +* it gets consistent memory state +*/ +smp_wmb (); +}It's useful to document why a memory barrier is being used in each instance. Therefore hiding
smp_wmb()inside a wrapper function isn't great.Also, it's suspicious that
smb_wmb()is used but no other barriers are used. What about a read memory barrier when accessing shared memory written by the guest?
Signature · 7 lines
Dmitry Fleytman Technology Expert and Consultant, Daynix Computing Ltd. Cell: +972-54-2819481 Skype: dmitry.fleytman
Quoted 13 lines · Dmitry Fleytman
NOTE: This is not a final patches for commit,
they are just for the latest fixes review. This set of patches still misses test for the device.We didn't succeed to find any guide or sample for the king of tests required (packets transmission). If someone can provide a pointer to the relevant information we'll be very grateful.
This set of patches implements VMWare VMXNET3 paravirtual NIC device. The device supports of all the device features including offload capabilties, VLANs and etc.
The device is tested on different OSes:
Quoted 8 lines
Fedora 15
Ubuntu 10.4
Centos 6.2
Windows 2008R2
Windows 2008 64bit
Windows 2008 32bit
Windows 2003 64bit
Windows 2003 32bitHi Dmitry,
Just a keepalive to let you know I intend to review these patches early next week. Look interesting but I'm swamped with work right now.
Stefan
Thanks, Stefan.
Sent from my iPhone
Quoted 28 lines · Stefan Hajnoczi
Quoted 13 lines · Dmitry Fleytman
NOTE: This is not a final patches for commit,
they are just for the latest fixes review. This set of patches still misses test for the device.We didn't succeed to find any guide or sample for the king of tests required (packets transmission). If someone can provide a pointer to the relevant information we'll be very grateful.
This set of patches implements VMWare VMXNET3 paravirtual NIC device. The device supports of all the device features including offload capabilties, VLANs and etc.
The device is tested on different OSes:Quoted 8 lines
Fedora 15 Ubuntu 10.4 Centos 6.2 Windows 2008R2 Windows 2008 64bit Windows 2008 32bit Windows 2003 64bit Windows 2003 32bitHi Dmitry,
Just a keepalive to let you know I intend to review these patches early next week. Look interesting but I'm swamped with work right now.Stefan
Quoted 4 lines · Dmitry Fleytman
We didn't succeed to find any guide or sample for the king of tests required (packets transmission). If someone can provide a pointer to the relevant information we'll be very grateful.
QEMU does not have a NIC test suite. You can use manual testing of guests with vmxnet3 drivers to see if the NIC works as expected. Maybe you could even try running WHQL tests on the VMXNET3 driver inside a Windows guest - I'm not sure, I've never looked into WHQL.
The community has been developing the qtest framework which is for device-level testing. For example, you could poke hardware registers and check that the correct interrupt gets raised. However, this framework is being extended to cover busses and hardware as developers decide they want test coverage. PCI support isn't available yet but Anthony Liguori recently worked on qtest PCI code.
Hello Stefan
Yes, we tested this device with various guests (both Linux and Windows) and it also passed WHQL tests for Windows Server 2008 R2 OS.
The issue is Anthony asked us to write a basic test for the device (a packet transmission test) and we are trying to get some reference code or documentation to do this.
Anthony, Paolo, please, advise.
Thanks in advance,
Dmitry Fleytman
Quoted 19 lines · Stefan Hajnoczi
Quoted 4 lines · Dmitry Fleytman
We didn't succeed to find any guide or sample for the king of tests required (packets transmission). If someone can provide a pointer to the relevant information we'll be very grateful.
QEMU does not have a NIC test suite. You can use manual testing of guests with vmxnet3 drivers to see if the NIC works as expected. Maybe you could even try running WHQL tests on the VMXNET3 driver inside a Windows guest - I'm not sure, I've never looked into WHQL.
The community has been developing the qtest framework which is for device-level testing. For example, you could poke hardware registers and check that the correct interrupt gets raised. However, this framework is being extended to cover busses and hardware as developers decide they want test coverage. PCI support isn't available yet but Anthony Liguori recently worked on qtest PCI code.
Signature · 7 lines
Dmitry Fleytman Technology Expert and Consultant, Daynix Computing Ltd. Cell: +972-54-2819481 Skype: dmitry.fleytman
Quoted 5 lines
Yes, we tested this device with various guests (both Linux and Windows) and it also passed WHQL tests for Windows Server 2008 R2 OS. The issue is Anthony asked us to write a basic test for the device (a packet transmission test) and we are trying to get some reference code or documentation to do this.
At QEMU summit we agreed that tests would only be required after the corresponding virtio model have it (i.e. virtio-scsi for SCSI HBAs, virtio-net for NICs, virtio-blk for non-SCSI block devices). Since these haven't even been posted, nor has libqos, I believe these patches are good to go.
Dmitry is a nice guy anyway and will sure write testcases as soon as the infrastructure is in, right? :)
Paolo
Quoted 11 lines · Paolo Bonzini
Quoted 5 lines
Yes, we tested this device with various guests (both Linux and Windows) and it also passed WHQL tests for Windows Server 2008 R2 OS. The issue is Anthony asked us to write a basic test for the device (a packet transmission test) and we are trying to get some reference code or documentation to do this.
At QEMU summit we agreed that tests would only be required after the corresponding virtio model have it (i.e. virtio-scsi for SCSI HBAs, virtio-net for NICs, virtio-blk for non-SCSI block devices). Since these haven't even been posted, nor has libqos, I believe these patches are good to go.
Ack.
Regards,
Anthony Liguori
Quoted 5 lines
Dmitry is a nice guy anyway and will sure write testcases as soon as the infrastructure is in, right? :)
Paolo
Hi Paolo, Anthony
No doubts I'm a nice guy :)
We'll be glad to amend this implementation with tests as soon as infrastructure is ready.
Dmitry
Quoted 27 lines · Anthony Liguori
Quoted 3 lines · Paolo Bonzini
Quoted 3 lines
Yes, we tested this device with various guests (both Linux and Windows) and it also passed WHQL tests for Windows Server 2008 R2 OS. The issue is Anthony asked us to write a basic test for the device
(a
Quoted 8 lines
Quoted 2 lines
packet transmission test) and we are trying to get some reference code or documentation to do this.
At QEMU summit we agreed that tests would only be required after the corresponding virtio model have it (i.e. virtio-scsi for SCSI HBAs, virtio-net for NICs, virtio-blk for non-SCSI block devices). Since these haven't even been posted, nor has libqos, I believe these patches are good to go.
Ack.
Regards,
Anthony Liguori
Quoted 5 lines
Dmitry is a nice guy anyway and will sure write testcases as soon as the infrastructure is in, right? :)
Paolo
Signature · 7 lines
Dmitry Fleytman Technology Expert and Consultant, Daynix Computing Ltd. Cell: +972-54-2819481 Skype: dmitry.fleytman