Skip to contentSkip to conversation

VMXNET3 paravirtual NIC device implementation

PATCH Superseded Dmitry Fleytman 3 participants 22 messages Dec 7, 2012 – Mar 27, 2013 Networking

v8 of 14 Superseded Latest is v14 →

lore
Share & tools
lore
#0 lore raw

v8 cover VMXNET3 paravirtual NIC device implementation

This set of patches implements VMWare VMXNET3 paravirtual NIC device. The device supports of all the device features including offload capabilties, VLANs and etc.
The device is tested on different OSes:

    Fedora 15
    Ubuntu 10.4
    Centos 6.2
    Windows 2008R2
    Windows 2008 64bit
    Windows 2008 32bit
    Windows 2003 64bit
    Windows 2003 32bit

Changes in V8:

Issues reported by Stefan Hajnoczi reviewed and mostly fixed:

Review

Quoted 5 lines
(quoted diff) +5 -0
+        }
+        curr_src_off += src[i].iov_len;
+    }
+    return j;
+}

The existing iov_copy() function provides equivalent functionality. I don't think iov_rebuild() is needed.

[DF] Done. Thanks, missed it.

Review

Quoted 6 lines
(quoted diff) +6 -0
+            size -= len;
+        }
+        iovec_off += iov[i].iov_len;
+    }
+    return res;
+}

Rename this net_checksum_add_iov() and place it in net/checksum.c, then the new dependency on net from block can be dropped.

[DF] Done.

Quoted 5 lines

+vmw_shmem_read(hwaddr addr, void *buf, int len)

 {
     VMW_SHPRN("SHMEM r: %" PRIx64 ", len: %d to %p", addr, len, buf);
     cpu_physical_memory_read(addr, buf, len);

}

All changes to this file should be squashed with the previous patch.

[DF] Done

Review

Quoted 13 lines
(quoted diff) +9 -0
+#ifdef VMXNET_DEBUG_SHMEM_ACCESS
+#define VMW_SHPRN(fmt, ...)                                                  
 \
+    do {                                                                     
 \
+        printf("[%s][SH][%s]: " fmt "\n", VMXNET_DEVICE_NAME, __func__,      
 \
+            ## __VA_ARGS__);                                                 
 \
+    } while (0)
+#else
+#define VMW_SHPRN(fmt, ...) do {} while (0)
+#endif

Please use QEMU tracing. It eliminates all this boilerplate and conditional compilation. Tracing can be enabled/disabled at runtime and works with SystemTap/DTrace. See docs/tracing.txt.

[DF] We'd like to stick with compile time logic in this case becase of 2 reasons:
[DF] 1. These printouts are intended for reverse engineering/development only and there is
[DF] no need to enable them at run time
[DF] 2. There is a big number of printouts, all driver-device communication is traced,
[DF] they hit performance even on strongest x86 in case of run-time logic

Review

Quoted 5 lines
(quoted diff) +5 -0
+struct eth_header {
+    uint8_t  h_dest[ETH_ALEN];   /* destination eth addr */
+    uint8_t  h_source[ETH_ALEN]; /* source ether addr    */
+    uint16_t h_proto;            /* packet type ID field */
+};

Looks like it's copy-pasted stuff from /usr/include/linux/if_*.h, /usr/include/netinet/*.h, and friends. If the system-wide headers are included names will collide for some of the macros at least.

Did you check if the slirp/ definitions can be reused?

[DF] Yes, you are right. This is copy-pasted from different places. [DF] Slips definishing do not fully cover our needs.

I'd rather we import network header definitions once in a generic place into the source tree. That way vmxnet and other components don't need to redefine these structs.

[DF] Exaclty! Our intention is to create generic header with network definitions and make everyone use it. [DF] We can move our header to some shared place if you want, however I'd do it in parallel with cleanup [DF] of similar definitions in existing code and this is a big change that os out of scope of these patches.

Quoted 2 lines

+ *===========================================================================*/

Is this huge comment box a sign that the code should be split into a foo_tx.c and an foo_rx.c file?

[DF] As for me this file is not that big to be splitted (<800 lines), however I'll do this if you insist :)

Quoted 1 line

+size_t vmxnet_tx_pkt_send(VmxnetTxPktH pkt, NetClientState *vc)

'vc' is an old name that was used for VLANClientState. The struct has since been renamed to NetClientState and the rest of QEMU uses 'nc' instead of 'vc'.

[DF] Fixed. Thanks.

Review

Quoted 2 lines
(quoted diff) +2 -0
+/* tx module context handle */
+typedef void *VmxnetTxPktH;

Forward-declaring the struct is nicer:

typedef struct VmxnetTxPkt VmxnetTxPkt;

The definition of VmxnetTxPkt is still hidden from the caller but you avoid the void* and casting. In vmxnet_pkt.c define using:

struct VmxnetTxPkt {
    ...
};

[DF] Agree, fixed. Thanks.

hw/vmxnet_utils.h

Quoted 4 lines
hw/vmxnet_utils.h +0 -0
index 7fd9a01..fac3b7b 100644

Please squash these fixes into the previous patch.

[DF] Oops, my bad. Fixed.

hw/vmxnet_utils.h

Quoted 3 lines
hw/vmxnet_utils.h +3 -0
+        uint8_t msix_used;
+        /* Whether MSI support was installed successfully */
+        uint8_t msi_used;

These two fields should be bool.

hw/vmxnet_utils.h

Quoted 2 lines
hw/vmxnet_utils.h +2 -0
+        /* Whether automatic interrupts masking enabled */
+        uint8_t auto_int_masking;

bool

[DF] Done.

hw/vmxnet_utils.h

Quoted 9 lines
hw/vmxnet_utils.h +9 -0
+static inline void vmxnet3_flush_shmem_changes(void)
+{
+    /*
+     * Flush shared memory changes
+     * Needed before sending interrupt to guest to ensure
+     * it gets consistent memory state
+     */
+    smp_wmb();
+}

It's useful to document why a memory barrier is being used in each instance. Therefore hiding smp_wmb() inside a wrapper function isn't great.

[DF] Fixed.

Also, it's suspicious that smb_wmb() is used but no other barriers are used. What about a read memory barrier when accessing shared memory written by the guest?

[DF] VMWARE interface build a in safe way - you always read out data buffer (packet descriptor) with memcopy, [DF] and then check its last byte to see whether it was fully filled by driver.
[DF] In this case device doesn't need read barriers. Drivers need write barriers of course to secure [DF] proper order of writes.

Changes in V7:

Issues reported by Michael S. Tsirkin reviewed and mostly fixed:

File vmware_utils.h:

...

hw/vmxnet_utils.h

Quoted 7 lines
hw/vmxnet_utils.h +7 -0
+static inline void
+vmw_shmem_st64(target_phys_addr_t addr, uint64_t value)
+{
+    VMW_SHPRN("SHMEM store64: %" PRIx64 " (value %" PRIx64 ")", addr, value);
+    stq_le_phys(addr, value);
+}
+

Pls remove these wrappers. These are just memory stores. Our codebase is too large as it is without every driver wrapping all standard calls.

[DF] Idea behind this macros is to have printout in each of them [DF] Printouts are really needed when reverse-engineering windows guest drivers [DF] Since there is no windows drivers code this is the only way to understand [DF] sequences they use

Quoted 1 line

+/* MACROS for simplification of operations on array-style registers */

UPPERCASE ABUSE

[DF] Fixed

hw/vmxnet_utils.h

Quoted 2 lines
hw/vmxnet_utils.h +2 -0
+#define IS_MULTIREG_ADDR(addr, base, cnt, regsize)                 \
+    (((addr + 1) > (base)) && ((addr) < (base) + (cnt) * (regsize)))

Same as range_covers_byte(base, cnt * regsize, addr)?

[DF] Fixed, thanks

hw/vmxnet_utils.h

Quoted 4 lines
hw/vmxnet_utils.h +4 -0
+
+#define MULTIREG_IDX_BY_ADDR(addr, base, regsize)                  \
+    (((addr) - (base)) / (regsize))
+

Above two macros is all that's left. No objection but it does not say what they do - want to add minimal documentation?
And please prefix with VMWARE_ or something.

[DF] Prefix added, macros documented

File vmxnet_utils.h (and related with the same problems):

...

hw/vmxnet_utils.h

Quoted 6 lines

+ * See the COPYING file in the top-level directory.
+ *
+ */

hw/vmxnet_utils.h +3 -0
+
+#ifndef _VMXNET_UTILS_H_
+#define _VMXNET_UTILS_H_

Please do not start identifiers with _ followed by an uppercase lattters.

[DF] Fixed

...

hw/vmxnet_utils.h

Quoted 7 lines
hw/vmxnet_utils.h +7 -0
+
+struct eth_header {
+    uint8_t  h_dest[ETH_ALEN];   /* destination eth addr */
+    uint8_t  h_source[ETH_ALEN]; /* source ether addr    */
+    uint16_t h_proto;            /* packet type ID field */
+};
+

And fix struct definitions to
1. follow qemu coding style
[DF] It's not clear what's wrong with the coding style. Please, elaborate.
2. start with vmxnet
[DF] Since this is generic network definitions with no device specifics
[DF] I'm not sure it makes sense to add vmxnet prefix.
[DF] I'd say it makes sense to put them into some generic header files under "net" directory instead [DF] and clean up other devices that have their local definitions of the same structures. [DF] Please, advise.

I also don't really understand why are these functions split out - vmxnet is the only user, no?

[DF] Currently vmxnet3 is the only user, hovewer we have vmxnet2 implementation as well (not submitted yet) [DF] and it uses the same header

File vmxnet_pkt.h:

...

hw/vmxnet_utils.h

Quoted 3 lines
hw/vmxnet_utils.h +3 -0
+
+/* tx module context handle */
+typedef void *VmxnetTxPktH;

This gets you zero type safety and makes debugging impossible. Just use pointers like everyone does.

[DF] Handle type is added to hide VmxnetTxPkt structure into .c header [DF] for better encapsulation.
[DF] Should we drop it anyway?

Files vmxnet3.*:

...

hw/vmxnet_utils.h

Quoted 3 lines
hw/vmxnet_utils.h +3 -0
+
+    if (zero_region) {
+        vmw_shmem_set(pa, 0, size*cell_size);

spaces around *

[DF] Fixed

...

Quoted 7 lines

+#define vmxnet3_ring_dump(macro, ring_name, ridx, r) \
+ macro("%s#%d: base %" PRIx64 " size %lu cell_size %lu gen %d next %lu",
\
+ (ring_name), (ridx),
\
+ (r)->pa, (r)->size, (r)->cell_size, (r)->gen, (r)->next)

make macros upper case

[DF] Fixed

...

hw/vmxnet_utils.h

Quoted 5 lines
hw/vmxnet_utils.h +5 -0
+static inline void vmxnet3_flush_shmem_changes(void)
+{
+    /*
+     * Flush shared memory changes
+     * Needed before transferring control to guest

what does 'transferring control to guest' mean?
[DF] Changed to
[DF] /*

[DF]     * Flush shared memory changes
[DF]     * Needed before sending interrupt to guest to ensure
[DF]     * it gets consistent memory state
[DF]     */

...

hw/vmxnet_utils.h

Quoted 3 lines
hw/vmxnet_utils.h +3 -0
+     */
+    smp_wmb();
+}

Don't use wrappers like this. They just hide bugs. For example it's not helpful before an interrupt in the function below.

[DF] I guess you are talking about vmxnet3_complete_packet() [DF] Strictly speaking barrier is a must because we change shared memory in [DF] vmxnet3_complete_packet()

[DF] And the wrapper is a good thing because its name explains its effect [DF] in a formal way as opposed to comments

...

hw/vmxnet_utils.h

Quoted 2 lines
hw/vmxnet_utils.h +2 -0
+    switch (status) {
+    case VMXNET3_PKT_STATUS_OK: {

don't put {} around cases: they align incorrectly
if it's too big move to a function.

[DF] Fixed

...

hw/vmxnet_utils.h

Quoted 5 lines
hw/vmxnet_utils.h +5 -0
+static bool
+vmxnet3_send_packet(VMXNET3State *s, uint32_t qidx)
+{
+    size_t bytes_sent = 0;
+    bool res = true;

why = true? don't initialize just because.

[DF] Fixed

...

Quoted 6 lines

+/*
+ * VMWARE headers we got from Linux kernel do not fully comply QEMU coding
+ * standards in sense of types and defines used.
+ * Since we didn't want to change VMWARE code, following set of typedefs
+ * and defines needed to compile these headers with QEMU introduced.
+ */

No need for this now.
You can export headers and put them under linux-headers.

[DF] Not sure it is possible because the header as-is is not stand-alone and won't compile [DF] without changes. We extracted definitions we use from their header and dropped unused [DF] and kernel-specific stuff.
[DF} Please, advise.

...

Quoted 1 line

+ if (VMXNET3_OM_TSO == s->offload_mode) {

Don't do Yoda style like this

[DF] "Yoda" style removed everywhere

Changes in V6:
Fixed most of problems pointed out by Michael S. Tsirkin
The only issue still open is creation of shared place
with generic network structures and functions. Currently
all generic network code introduced by VMXNET3 resides in vmxnet_utils.c/h files. It could be moved to some shared location however we believe it is a matter of separate refactoring as there are a lot of copy-pasted definitions in almost every device and code cleanup efforts requred in order to create truly shared codebase.

Implemented suggestions by Anthony Liguori

Fixed incorrect checksum caclulation for some packets in SW offloads mode

Changes in V5:
MSI-X save/load implemented in the device instead of pci bus as suggested by Michael S. Tsirkin

Patches regrouped as suggested by Paolo Bonzini

Changes in V4:
Fixed a few problems uncovered by NETIO test suit Assertion on failure to initialize MSI/MSI-X replaced with warning message and fallback to Legacy/MSI respectively

Various coding style adjustments and patch split-up as suggested by Anthony
Liguori

Live migration support added

Changes in V3:
Fixed crash when net device that is used as network fronted has no virtio HDR support.
Task offloads emulation for cases when net device that is used as network fronted has no virtio HDR support.

Changes in V2:
License text changed accoring to community suggestions Standard license header from GPLv2+ - licensed QEMU files used

Dmitry Fleytman (5):
Adding utility function net_checksum_add_cont() that allows checksum
calculation of scattered data with odd chunk sizes
Adding utility function net_checksum_add_iov() for iovec checksum
calculation
Adding common definitions for VMWARE devices
Adding packet abstraction for VMWARE network devices Adding VMXNET3 device implementation

 default-configs/pci.mak |    1 +
 hw/Makefile.objs        |    1 +
 hw/pci.h                |    1 +
 hw/vmware_utils.h       |  143 +++
 hw/vmxnet3.c            | 2437 +++++++++++++++++++++++++++++++++++++++++++++++
 hw/vmxnet3.h            |  762 +++++++++++++++
 hw/vmxnet_debug.h       |  121 +++
 hw/vmxnet_pkt.c         |  758 +++++++++++++++
 hw/vmxnet_pkt.h         |  309 ++++++
 hw/vmxnet_utils.c       |  219 +++++
 hw/vmxnet_utils.h       |  340 +++++++
 iov.h                   |    5 +
 net/checksum.c          |   41 +-
 net/checksum.h          |   22 +-
 14 files changed, 5153 insertions(+), 7 deletions(-)
 create mode 100644 hw/vmware_utils.h
 create mode 100644 hw/vmxnet3.c
 create mode 100644 hw/vmxnet3.h
 create mode 100644 hw/vmxnet_debug.h
 create mode 100644 hw/vmxnet_pkt.c
 create mode 100644 hw/vmxnet_pkt.h
 create mode 100644 hw/vmxnet_utils.c
 create mode 100644 hw/vmxnet_utils.h
Signature · 1 line
1.7.11.7
#1 lore raw

PATCH v8 1/5 Adding utility function net_checksum_add_cont() that allows checksum calculation of scattered data with odd chunk sizes

Adding utility function net_raw_checksum() that calculates checksum of buffer given

---
 net/checksum.c | 13 +++++++------
 net/checksum.h | 14 +++++++++++++-
 2 files changed, 20 insertions(+), 7 deletions(-)
net/checksum.c +7 -6
index 9919b2e..4fa5563 100644
@@ -20,16 +20,17 @@
 #define PROTO_TCP  6
 #define PROTO_UDP 17
 
-uint32_t net_checksum_add(int len, uint8_t *buf)
+uint32_t net_checksum_add_cont(int len, uint8_t *buf, int seq)
 {
     uint32_t sum = 0;
     int i;
 
-    for (i = 0; i < len; i++) {
-	if (i & 1)
-	    sum += (uint32_t)buf[i];
-	else
-	    sum += (uint32_t)buf[i] << 8;
+    for (i = seq; i < seq + len; i++) {
+        if (i & 1) {
+            sum += (uint32_t)buf[i - seq];
+        } else {
+            sum += (uint32_t)buf[i - seq] << 8;
+        }
     }
     return sum;
 }
net/checksum.h +13 -1
index 1f05298..171924c 100644
@@ -20,10 +20,22 @@
 
 #include <stdint.h>
 
-uint32_t net_checksum_add(int len, uint8_t *buf);
+uint32_t net_checksum_add_cont(int len, uint8_t *buf, int seq);
 uint16_t net_checksum_finish(uint32_t sum);
 uint16_t net_checksum_tcpudp(uint16_t length, uint16_t proto,
                              uint8_t *addrs, uint8_t *buf);
 void net_checksum_calculate(uint8_t *data, int length);
 
+static inline uint32_t
+net_checksum_add(int len, uint8_t *buf)
+{
+    return net_checksum_add_cont(len, buf, 0);
+}
+
+static inline uint16_t
+net_raw_checksum(uint8_t *data, int length)
+{
+  return net_checksum_finish(net_checksum_add(length, data));
+}
+
 #endif /* QEMU_NET_CHECKSUM_H */
Signature · 1 line
1.7.11.7
#2 lore raw

PATCH v8 2/5 Adding utility function net_checksum_add_iov() for iovec checksum calculation

In reply to Dmitry Fleytman
---
 iov.h          |  5 +++++
 net/checksum.c | 28 ++++++++++++++++++++++++++++
 net/checksum.h |  8 ++++++++
 3 files changed, 41 insertions(+)
iov.h +5 -0
index 34c8ec9..c184a80 100644
@@ -11,6 +11,9 @@
  * the COPYING file in the top-level directory.
  */
 
+#ifndef QEMU_IOV_H
+#define QEMU_IOV_H
+
 #include "qemu-common.h"
 
 /**
@@ -95,3 +98,5 @@ void iov_hexdump(const struct iovec *iov, const unsigned int iov_cnt,
 unsigned iov_copy(struct iovec *dst_iov, unsigned int dst_iov_cnt,
                  const struct iovec *iov, unsigned int iov_cnt,
                  size_t offset, size_t bytes);
+
+#endif /* QEMU_IOV_H */
net/checksum.c +28 -0
index 4fa5563..9c813ff 100644
@@ -84,3 +84,31 @@ void net_checksum_calculate(uint8_t *data, int length)
     data[14+hlen+csum_offset]   = csum >> 8;
     data[14+hlen+csum_offset+1] = csum & 0xff;
 }
+
+uint32_t
+net_checksum_add_iov(const struct iovec *iov, const unsigned int iov_cnt,
+                     uint32_t iov_off, uint32_t size)
+{
+    size_t iovec_off, buf_off;
+    unsigned int i;
+    uint32_t res = 0;
+    uint32_t seq = 0;
+
+    iovec_off = 0;
+    buf_off = 0;
+    for (i = 0; i < iov_cnt && size; i++) {
+        if (iov_off < (iovec_off + iov[i].iov_len)) {
+            size_t len = MIN((iovec_off + iov[i].iov_len) - iov_off , size);
+            void *chunk_buf = iov[i].iov_base + (iov_off - iovec_off);
+
+            res += net_checksum_add_cont(len, chunk_buf, seq);
+            seq += len;
+
+            buf_off += len;
+            iov_off += len;
+            size -= len;
+        }
+        iovec_off += iov[i].iov_len;
+    }
+    return res;
+}
net/checksum.h +8 -0
index 171924c..e63c482 100644
@@ -19,6 +19,7 @@
 #define QEMU_NET_CHECKSUM_H
 
 #include <stdint.h>
+#include "iov.h"
 
 uint32_t net_checksum_add_cont(int len, uint8_t *buf, int seq);
 uint16_t net_checksum_finish(uint32_t sum);
@@ -38,4 +39,11 @@ net_raw_checksum(uint8_t *data, int length)
   return net_checksum_finish(net_checksum_add(length, data));
 }
 
+/**
+ * Checksum calculation for scatter-gather vector
+ */
+uint32_t net_checksum_add_iov(const struct iovec *iov,
+                              const unsigned int iov_cnt,
+                              uint32_t iov_off, uint32_t size);
+
 #endif /* QEMU_NET_CHECKSUM_H */
Signature · 1 line
1.7.11.7
#3 lore raw

Re: [Qemu-devel] [PATCH V8 2/5] Adding utility function net_checksum_add_iov() for iovec checksum calculation

iov.h

Quoted 13 lines · Dmitry Fleytman
---
 iov.h          |  5 +++++
 net/checksum.c | 28 ++++++++++++++++++++++++++++
 net/checksum.h |  8 ++++++++
 3 files changed, 41 insertions(+)
iov.h +0 -0
index 34c8ec9..c184a80 100644
@@ -11,6 +11,9 @@

This file has been moved in qemu.git/master. It is now include/qemu/iov.h.

iov.h line 16

Quoted 6 lines
iov.h +3 -0
  * the COPYING file in the top-level directory.
  */
 
+#ifndef QEMU_IOV_H
+#define QEMU_IOV_H
+

This is already present in qemu.git/master.

iov.h void iov_hexdump(const struct iovec *iov, const unsigned int iov_cnt, · line 102

Quoted 9 lines
iov.h +2 -0
 #include "qemu-common.h"
 
 /**
@@ -95,3 +98,5 @@ void iov_hexdump(const struct iovec *iov, const unsigned int iov_cnt,
 unsigned iov_copy(struct iovec *dst_iov, unsigned int dst_iov_cnt,
                  const struct iovec *iov, unsigned int iov_cnt,
                  size_t offset, size_t bytes);
+
+#endif /* QEMU_IOV_H */

Same here. Just pointing out expected rebase conflicts you'll see for the next revision of this series.

Quoted 41 lines
net/checksum.c +28 -0
index 4fa5563..9c813ff 100644
@@ -84,3 +84,31 @@ void net_checksum_calculate(uint8_t *data, int length)
     data[14+hlen+csum_offset]   = csum >> 8;
     data[14+hlen+csum_offset+1] = csum & 0xff;
 }
+
+uint32_t
+net_checksum_add_iov(const struct iovec *iov, const unsigned int iov_cnt,
+                     uint32_t iov_off, uint32_t size)
+{
+    size_t iovec_off, buf_off;
+    unsigned int i;
+    uint32_t res = 0;
+    uint32_t seq = 0;
+
+    iovec_off = 0;
+    buf_off = 0;
+    for (i = 0; i < iov_cnt && size; i++) {
+        if (iov_off < (iovec_off + iov[i].iov_len)) {
+            size_t len = MIN((iovec_off + iov[i].iov_len) - iov_off , size);
+            void *chunk_buf = iov[i].iov_base + (iov_off - iovec_off);
+
+            res += net_checksum_add_cont(len, chunk_buf, seq);
+            seq += len;
+
+            buf_off += len;
+            iov_off += len;
+            size -= len;
+        }
+        iovec_off += iov[i].iov_len;
+    }
+    return res;
+}
net/checksum.h +0 -0
index 171924c..e63c482 100644

net/checksum.h line 22

Quoted 4 lines
net/checksum.h +1 -0
@@ -19,6 +19,7 @@
 #define QEMU_NET_CHECKSUM_H
 
 #include <stdint.h>
+#include "iov.h"

iov.h is the wrong header file, you need struct iovec from qemu-common.h.

#4 lore raw

Re: [Qemu-devel] [PATCH V8 2/5] Adding utility function net_checksum_add_iov() for iovec checksum calculation

Thanks for pointing out, Stefan.
I've rebased the code and fixed these issues.

Quoted 92 lines · Stefan Hajnoczi

iov.h

Quoted 13 lines · Dmitry Fleytman
---
 iov.h          |  5 +++++
 net/checksum.c | 28 ++++++++++++++++++++++++++++
 net/checksum.h |  8 ++++++++
 3 files changed, 41 insertions(+)
iov.h +0 -0
index 34c8ec9..c184a80 100644
@@ -11,6 +11,9 @@

This file has been moved in qemu.git/master. It is now include/qemu/iov.h.

iov.h line 16

Quoted 6 lines
iov.h +3 -0
  * the COPYING file in the top-level directory.
  */
 
+#ifndef QEMU_IOV_H
+#define QEMU_IOV_H
+

This is already present in qemu.git/master.

iov.h void iov_hexdump(const struct iovec *iov, const

Quoted 4 lines
iov.h +0 -0
 #include "qemu-common.h"
 
 /**
@@ -95,3 +98,5 @@ void iov_hexdump(const struct iovec *iov, const

unsigned int iov_cnt,

iov.h void iov_hexdump(const struct iovec *iov, const unsigned int iov_cnt, · line 102

Quoted 5 lines
iov.h +2 -0
 unsigned iov_copy(struct iovec *dst_iov, unsigned int dst_iov_cnt,
                  const struct iovec *iov, unsigned int iov_cnt,
                  size_t offset, size_t bytes);
+
+#endif /* QEMU_IOV_H */

Same here. Just pointing out expected rebase conflicts you'll see for the next revision of this series.

net/checksum.c void net_checksum_calculate(uint8_t *data, int length) · line 89

Quoted 11 lines
net/checksum.c +3 -0
index 4fa5563..9c813ff 100644
@@ -84,3 +84,31 @@ void net_checksum_calculate(uint8_t *data, int length)
     data[14+hlen+csum_offset]   = csum >> 8;
     data[14+hlen+csum_offset+1] = csum & 0xff;
 }
+
+uint32_t
+net_checksum_add_iov(const struct iovec *iov, const unsigned int

iov_cnt,

net/checksum.c void net_checksum_calculate(uint8_t *data, int length)

Quoted 12 lines
net/checksum.c +12 -0
+                     uint32_t iov_off, uint32_t size)
+{
+    size_t iovec_off, buf_off;
+    unsigned int i;
+    uint32_t res = 0;
+    uint32_t seq = 0;
+
+    iovec_off = 0;
+    buf_off = 0;
+    for (i = 0; i < iov_cnt && size; i++) {
+        if (iov_off < (iovec_off + iov[i].iov_len)) {
+            size_t len = MIN((iovec_off + iov[i].iov_len) - iov_off ,

size);

Quoted 18 lines
net/checksum.c +13 -0
+            void *chunk_buf = iov[i].iov_base + (iov_off - iovec_off);
+
+            res += net_checksum_add_cont(len, chunk_buf, seq);
+            seq += len;
+
+            buf_off += len;
+            iov_off += len;
+            size -= len;
+        }
+        iovec_off += iov[i].iov_len;
+    }
+    return res;
+}
net/checksum.h +0 -0
index 171924c..e63c482 100644

net/checksum.h line 22

Quoted 4 lines
net/checksum.h +1 -0
@@ -19,6 +19,7 @@
 #define QEMU_NET_CHECKSUM_H
 
 #include <stdint.h>
+#include "iov.h"

iov.h is the wrong header file, you need struct iovec from qemu-common.h.

Signature · 7 lines
Dmitry Fleytman
Technology Expert and Consultant,

Daynix Computing Ltd.

Cell: +972-54-2819481
Skype: dmitry.fleytman
#5 lore raw

PATCH v8 3/5 Adding common definitions for VMWARE devices

In reply to Dmitry Fleytman
---
 hw/vmware_utils.h | 143 +++++++++++++++++++++++
 hw/vmxnet_debug.h | 121 +++++++++++++++++++
 hw/vmxnet_utils.c | 219 +++++++++++++++++++++++++++++++++++
 hw/vmxnet_utils.h | 340 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 823 insertions(+)
 create mode 100644 hw/vmware_utils.h
 create mode 100644 hw/vmxnet_debug.h
 create mode 100644 hw/vmxnet_utils.c
 create mode 100644 hw/vmxnet_utils.h
hw/vmware_utils.h new file +143 -0
new file mode 100644
index 0000000..7f449eb
@@ -0,0 +1,143 @@
+/*
+ * QEMU VMWARE paravirtual devices - auxiliary code
+ *
+ * Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com)
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Dmitry Fleytman <[email protected]>
+ * Yan Vugenfirer <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef VMWARE_UTILS_H
+#define VMWARE_UTILS_H
+
+#include "range.h"
+
+#ifndef VMW_SHPRN
+#define VMW_SHPRN(fmt, ...) do {} while (0)
+#endif
+
+/*
+ * Shared memory access functions with byte swap support
+ * Each function contains printout for reverse-engineering needs
+ *
+ */
+static inline void
+vmw_shmem_read(hwaddr addr, void *buf, int len)
+{
+    VMW_SHPRN("SHMEM r: %" PRIx64 ", len: %d to %p", addr, len, buf);
+    cpu_physical_memory_read(addr, buf, len);
+}
+
+static inline void
+vmw_shmem_write(hwaddr addr, void *buf, int len)
+{
+    VMW_SHPRN("SHMEM w: %" PRIx64 ", len: %d to %p", addr, len, buf);
+    cpu_physical_memory_write(addr, buf, len);
+}
+
+static inline void
+vmw_shmem_rw(hwaddr addr, void *buf, int len, int is_write)
+{
+    VMW_SHPRN("SHMEM r/w: %" PRIx64 ", len: %d (to %p), is write: %d",
+              addr, len, buf, is_write);
+
+    cpu_physical_memory_rw(addr, buf, len, is_write);
+}
+
+static inline void
+vmw_shmem_set(hwaddr addr, uint8 val, int len)
+{
+    int i;
+    VMW_SHPRN("SHMEM set: %" PRIx64 ", len: %d (value 0x%X)", addr, len, val);
+
+    for (i = 0; i < len; i++) {
+        cpu_physical_memory_write(addr + i, &val, 1);
+    }
+}
+
+static inline uint32_t
+vmw_shmem_ld8(hwaddr addr)
+{
+    uint8_t res = ldub_phys(addr);
+    VMW_SHPRN("SHMEM load8: %" PRIx64 " (value 0x%X)", addr, res);
+    return res;
+}
+
+static inline void
+vmw_shmem_st8(hwaddr addr, uint8_t value)
+{
+    VMW_SHPRN("SHMEM store8: %" PRIx64 " (value 0x%X)", addr, value);
+    stb_phys(addr, value);
+}
+
+static inline uint32_t
+vmw_shmem_ld16(hwaddr addr)
+{
+    uint16_t res = lduw_le_phys(addr);
+    VMW_SHPRN("SHMEM load16: %" PRIx64 " (value 0x%X)", addr, res);
+    return res;
+}
+
+static inline void
+vmw_shmem_st16(hwaddr addr, uint16_t value)
+{
+    VMW_SHPRN("SHMEM store16: %" PRIx64 " (value 0x%X)", addr, value);
+    stw_le_phys(addr, value);
+}
+
+static inline uint32_t
+vmw_shmem_ld32(hwaddr addr)
+{
+    uint32_t res = ldl_le_phys(addr);
+    VMW_SHPRN("SHMEM load32: %" PRIx64 " (value 0x%X)", addr, res);
+    return res;
+}
+
+static inline void
+vmw_shmem_st32(hwaddr addr, uint32_t value)
+{
+    VMW_SHPRN("SHMEM store32: %" PRIx64 " (value 0x%X)", addr, value);
+    stl_le_phys(addr, value);
+}
+
+static inline uint64_t
+vmw_shmem_ld64(hwaddr addr)
+{
+    uint64_t res = ldq_le_phys(addr);
+    VMW_SHPRN("SHMEM load64: %" PRIx64 " (value %" PRIx64 ")", addr, res);
+    return res;
+}
+
+static inline void
+vmw_shmem_st64(hwaddr addr, uint64_t value)
+{
+    VMW_SHPRN("SHMEM store64: %" PRIx64 " (value %" PRIx64 ")", addr, value);
+    stq_le_phys(addr, value);
+}
+
+/* Macros for simplification of operations on array-style registers */
+
+/*
+ * Whether <addr> lies inside of array-style register defined by <base>,
+ * number of elements (<cnt>) and element size (<regsize>)
+ *
+*/
+#define VMW_IS_MULTIREG_ADDR(addr, base, cnt, regsize)                 \
+    range_covers_byte(base, cnt * regsize, addr)
+
+/*
+ * Returns index of given register (<addr>) in array-style register defined by
+ * <base> and element size (<regsize>)
+ *
+*/
+#define VMW_MULTIREG_IDX_BY_ADDR(addr, base, regsize)                  \
+    (((addr) - (base)) / (regsize))
+
+#endif
hw/vmxnet_debug.h new file +121 -0

121 lines not on this page · raw message

hw/vmxnet_utils.c new file +219 -0

219 lines not on this page · raw message

hw/vmxnet_utils.h new file +340 -0

340 lines not on this page · raw message

Signature · 1 line
1.7.11.7
#6 lore raw

Re: [Qemu-devel] [PATCH V8 3/5] Adding common definitions for VMWARE devices

Quoted 23 lines · Dmitry Fleytman
hw/vmxnet_debug.h new file +17 -0
new file mode 100644
index 0000000..faa1431
@@ -0,0 +1,121 @@
+/*
+ * QEMU VMWARE VMXNET* paravirtual NICs - debugging facilities
+ *
+ * Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com)
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Dmitry Fleytman <[email protected]>
+ * Tamir Shomer <[email protected]>
+ * Yan Vugenfirer <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+

hw/vmxnet_debug.h line 27

Quoted 10 lines
hw/vmxnet_debug.h new file +10 -0
+#ifndef _QEMU_VMXNET_DEBUG_H
+#define _QEMU_VMXNET_DEBUG_H
+
+#ifdef VMXNET_VERSION_2
+    #define VMXNET_DEVICE_NAME "vmxnet"
+#elif defined VMXNET_VERSION_3
+    #define VMXNET_DEVICE_NAME "vmxnet3"
+#else
+    #error "VMXNET version is not defined"
+#endif

Please drop the VMXNET_VERSION_* conditional dead code, only version 3 is used.

#7 lore raw

Re: [Qemu-devel] [PATCH V8 3/5] Adding common definitions for VMWARE devices

Stefan,

We have vmxnet2 implementation ready for submission as well, so this code is not dead :)

Dmitry.

Quoted 39 lines · Stefan Hajnoczi
Quoted 9 lines · Dmitry Fleytman
hw/vmxnet_debug.h new file +3 -0
new file mode 100644
index 0000000..faa1431
@@ -0,0 +1,121 @@
+/*
+ * QEMU VMWARE VMXNET* paravirtual NICs - debugging facilities
+ *

hw/vmxnet_debug.h line 13

Quoted 10 lines
hw/vmxnet_debug.h new file +10 -0
+ * Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com)
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Dmitry Fleytman <[email protected]>
+ * Tamir Shomer <[email protected]>
+ * Yan Vugenfirer <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or

later.

Quoted 4 lines
hw/vmxnet_debug.h new file +4 -0
+ * See the COPYING file in the top-level directory.
+ *
+ */
+

hw/vmxnet_debug.h line 27

Quoted 10 lines
hw/vmxnet_debug.h new file +10 -0
+#ifndef _QEMU_VMXNET_DEBUG_H
+#define _QEMU_VMXNET_DEBUG_H
+
+#ifdef VMXNET_VERSION_2
+    #define VMXNET_DEVICE_NAME "vmxnet"
+#elif defined VMXNET_VERSION_3
+    #define VMXNET_DEVICE_NAME "vmxnet3"
+#else
+    #error "VMXNET version is not defined"
+#endif

Please drop the VMXNET_VERSION_* conditional dead code, only version 3 is used.

Signature · 7 lines
Dmitry Fleytman
Technology Expert and Consultant,

Daynix Computing Ltd.

Cell: +972-54-2819481
Skype: dmitry.fleytman
#8 lore raw

Re: [Qemu-devel] [PATCH V8 3/5] Adding common definitions for VMWARE devices

Quoted 4 lines · Dmitry Fleytman

Stefan,

We have vmxnet2 implementation ready for submission as well, so this code is not dead :)

Please add it when it's needed.

Stefan

#9 lore raw

Re: [Qemu-devel] [PATCH V8 3/5] Adding common definitions for VMWARE devices

Agree. Done.

Quoted 10 lines · Stefan Hajnoczi
Quoted 4 lines · Dmitry Fleytman

Stefan,

We have vmxnet2 implementation ready for submission as well, so this code is not dead :)

Please add it when it's needed.

Stefan

Signature · 7 lines
Dmitry Fleytman
Technology Expert and Consultant,

Daynix Computing Ltd.

Cell: +972-54-2819481
Skype: dmitry.fleytman
#10 lore raw

PATCH v8 4/5 Adding packet abstraction for VMWARE network devices

In reply to Dmitry Fleytman
---
 hw/vmxnet_pkt.c | 758 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 hw/vmxnet_pkt.h | 309 +++++++++++++++++++++++
 2 files changed, 1067 insertions(+)
 create mode 100644 hw/vmxnet_pkt.c
 create mode 100644 hw/vmxnet_pkt.h
hw/vmxnet_pkt.c new file +758 -0
new file mode 100644
index 0000000..9b501e3
@@ -0,0 +1,758 @@
+/*
+ * QEMU VMWARE VMXNET* paravirtual NICs - packets abstractions
+ *
+ * Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com)
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Dmitry Fleytman <[email protected]>
+ * Tamir Shomer <[email protected]>
+ * Yan Vugenfirer <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "vmxnet_pkt.h"
+#include "vmxnet_utils.h"
+#include "iov.h"
+
+#include "net/checksum.h"
+
+/*=============================================================================
+ *=============================================================================
+ *
+ *                            TX CODE
+ *
+ *=============================================================================
+ *===========================================================================*/
+
+enum {
+    VMXNET_TX_PKT_VHDR_FRAG = 0,
+    VMXNET_TX_PKT_L2HDR_FRAG,
+    VMXNET_TX_PKT_L3HDR_FRAG,
+    VMXNET_TX_PKT_PL_START_FRAG
+};
+
+/* TX packet private context */
+struct VmxnetTxPkt{
+    struct virtio_net_hdr virt_hdr;
+    bool has_virt_hdr;
+
+    struct iovec *raw;
+    uint32_t raw_frags;
+    uint32_t max_raw_frags;
+
+    struct iovec *vec;
+
+    uint8_t l2_hdr[ETH_MAX_L2_HDR_LEN];
+
+    uint32_t payload_len;
+
+    uint32_t payload_frags;
+    uint32_t max_payload_frags;
+
+    uint16_t hdr_len;
+    eth_pkt_types_e packet_type;
+    uint8_t l4proto;
+};
+
+void vmxnet_tx_pkt_init(struct VmxnetTxPkt **pkt, uint32_t max_frags,
+    bool has_virt_hdr)
+{
+    struct VmxnetTxPkt *p = g_malloc0(sizeof *p);
+
+    p->vec = g_malloc((sizeof *p->vec) *
+        (max_frags + VMXNET_TX_PKT_PL_START_FRAG));
+
+    p->raw = g_malloc((sizeof *p->raw) * max_frags);
+
+    p->max_payload_frags = max_frags;
+    p->max_raw_frags = max_frags;
+    p->has_virt_hdr = has_virt_hdr;
+    p->vec[VMXNET_TX_PKT_VHDR_FRAG].iov_base = &p->virt_hdr;
+    p->vec[VMXNET_TX_PKT_VHDR_FRAG].iov_len =
+        p->has_virt_hdr ? sizeof p->virt_hdr : 0;
+    p->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_base = &p->l2_hdr;
+    p->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_base = NULL;
+    p->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_len = 0;
+
+    *pkt = p;
+}
+
+void vmxnet_tx_pkt_uninit(struct VmxnetTxPkt *pkt)
+{
+    if (pkt) {
+        if (pkt->vec) {
+            g_free(pkt->vec);
+        }
+
+        if (pkt->raw) {
+            g_free(pkt->raw);
+        }
+
+        g_free(pkt);
+    }
+}
+
+void vmxnet_tx_pkt_update_ip_checksums(struct VmxnetTxPkt *pkt)
+{
+    uint16_t csum;
+    uint32_t ph_raw_csum;
+    assert(pkt);
+    uint8_t gso_type = pkt->virt_hdr.gso_type & ~VIRTIO_NET_HDR_GSO_ECN;
+    struct ip_header *ip_hdr;
+
+    if (VIRTIO_NET_HDR_GSO_TCPV4 != gso_type &&
+        VIRTIO_NET_HDR_GSO_UDP != gso_type) {
+        return;
+    }
+
+    ip_hdr = pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_base;
+
+    if (pkt->payload_len + pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_len >
+        ETH_MAX_IP_DGRAM_LEN) {
+        return;
+    }
+
+    ip_hdr->ip_len = cpu_to_be16(pkt->payload_len +
+        pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_len);
+
+    /* Calculate IP header checksum                    */
+    ip_hdr->ip_sum = 0;
+    csum = net_raw_checksum((uint8_t *)ip_hdr,
+        pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_len);
+    ip_hdr->ip_sum = cpu_to_be16(csum);
+
+    /* Calculate IP pseudo header checksum             */
+    ph_raw_csum = eth_calc_pseudo_hdr_csum(ip_hdr, pkt->payload_len);
+    csum = cpu_to_be16(net_checksum_finish(ph_raw_csum));
+    iov_from_buf(&pkt->vec[VMXNET_TX_PKT_PL_START_FRAG], pkt->payload_frags,
+                 pkt->virt_hdr.csum_offset, &csum, sizeof(csum));
+}
+
+static void vmxnet_tx_pkt_calculate_hdr_len(struct VmxnetTxPkt *pkt)
+{
+    pkt->hdr_len = pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_len +
+        pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_len;
+}
+
+static bool vmxnet_tx_pkt_parse_headers(struct VmxnetTxPkt *pkt)
+{
+    struct iovec *l2_hdr, *l3_hdr;
+    size_t bytes_read;
+    size_t full_ip6hdr_len;
+    uint16_t l3_proto;
+
+    assert(pkt);
+
+    l2_hdr = &pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG];
+    l3_hdr = &pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG];
+
+    bytes_read = iov_to_buf(pkt->raw, pkt->raw_frags, 0, l2_hdr->iov_base,
+                            ETH_MAX_L2_HDR_LEN);
+    if (bytes_read < ETH_MAX_L2_HDR_LEN) {
+        l2_hdr->iov_len = 0;
+        return false;
+    } else {
+        l2_hdr->iov_len = eth_get_l2_hdr_length(l2_hdr->iov_base);
+    }
+
+    l3_proto = eth_get_l3_proto(l2_hdr->iov_base, l2_hdr->iov_len);
+
+    switch (l3_proto) {
+    case ETH_P_IP:
+        l3_hdr->iov_base = g_malloc(ETH_MAX_IP4_HDR_LEN);
+
+        bytes_read = iov_to_buf(pkt->raw, pkt->raw_frags, l2_hdr->iov_len,
+                                l3_hdr->iov_base, sizeof(struct ip_header));
+
+        if (bytes_read < sizeof(struct ip_header)) {
+            l3_hdr->iov_len = 0;
+            return false;
+        }
+
+        l3_hdr->iov_len = IP_HDR_GET_LEN(l3_hdr->iov_base);
+        pkt->l4proto = ((struct ip_header *) l3_hdr->iov_base)->ip_p;
+
+        /* copy optional IPv4 header data */
+        bytes_read = iov_to_buf(pkt->raw, pkt->raw_frags,
+                                l2_hdr->iov_len + sizeof(struct ip_header),
+                                l3_hdr->iov_base + sizeof(struct ip_header),
+                                l3_hdr->iov_len - sizeof(struct ip_header));
+        if (bytes_read < l3_hdr->iov_len - sizeof(struct ip_header)) {
+            l3_hdr->iov_len = 0;
+            return false;
+        }
+        break;
+
+    case ETH_P_IPV6:
+        if (!eth_parse_ipv6_hdr(pkt->raw, pkt->raw_frags, l2_hdr->iov_len,
+                               &pkt->l4proto, &full_ip6hdr_len)) {
+            l3_hdr->iov_len = 0;
+            return false;
+        }
+
+        l3_hdr->iov_base = g_malloc(full_ip6hdr_len);
+
+        bytes_read = iov_to_buf(pkt->raw, pkt->raw_frags, l2_hdr->iov_len,
+                                l3_hdr->iov_base, full_ip6hdr_len);
+
+        if (bytes_read < full_ip6hdr_len) {
+            l3_hdr->iov_len = 0;
+            return false;
+        } else {
+            l3_hdr->iov_len = full_ip6hdr_len;
+        }
+        break;
+
+    default:
+        l3_hdr->iov_len = 0;
+        break;
+    }
+
+    vmxnet_tx_pkt_calculate_hdr_len(pkt);
+    pkt->packet_type = get_eth_packet_type(l2_hdr->iov_base);
+    return true;
+}
+
+static bool vmxnet_tx_pkt_rebuild_payload(struct VmxnetTxPkt *pkt)
+{
+    size_t payload_len = iov_size(pkt->raw, pkt->raw_frags) - pkt->hdr_len;
+
+    pkt->payload_frags = iov_copy(&pkt->vec[VMXNET_TX_PKT_PL_START_FRAG],
+                                pkt->max_payload_frags,
+                                pkt->raw, pkt->raw_frags,
+                                pkt->hdr_len, payload_len);
+
+    if (pkt->payload_frags != (uint32_t) -1) {
+        pkt->payload_len = payload_len;
+        return true;
+    } else {
+        return false;
+    }
+}
+
+bool vmxnet_tx_pkt_parse(struct VmxnetTxPkt *pkt)
+{
+    return vmxnet_tx_pkt_parse_headers(pkt) &&
+           vmxnet_tx_pkt_rebuild_payload(pkt);
+}
+
+struct virtio_net_hdr *vmxnet_tx_pkt_get_vhdr(struct VmxnetTxPkt *pkt)
+{
+    assert(pkt);
+    return &pkt->virt_hdr;
+}
+
+static uint8_t vmxnet_tx_pkt_get_gso_type(struct VmxnetTxPkt *pkt,
+                                          bool tso_enable)
+{
+    uint8_t rc = VIRTIO_NET_HDR_GSO_NONE;
+    uint16_t l3_proto;
+
+    l3_proto = eth_get_l3_proto(pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_base,
+        pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_len);
+
+    if (!tso_enable) {
+        goto func_exit;
+    }
+
+    rc = eth_get_gso_type(l3_proto, pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_base,
+                          pkt->l4proto);
+
+func_exit:
+    return rc;
+}
+
+void vmxnet_tx_pkt_build_vheader(struct VmxnetTxPkt *pkt, bool tso_enable,
+    bool csum_enable, uint32_t gso_size)
+{
+    struct tcp_hdr l4hdr;
+    assert(pkt);
+
+    /* csum has to be enabled if tso is. */
+    assert(csum_enable || !tso_enable);
+
+    pkt->virt_hdr.gso_type = vmxnet_tx_pkt_get_gso_type(pkt, tso_enable);
+
+    switch (pkt->virt_hdr.gso_type & ~VIRTIO_NET_HDR_GSO_ECN) {
+    case VIRTIO_NET_HDR_GSO_NONE:
+        pkt->virt_hdr.hdr_len = 0;
+        pkt->virt_hdr.gso_size = 0;
+        break;
+
+    case VIRTIO_NET_HDR_GSO_UDP:
+        pkt->virt_hdr.gso_size = IP_FRAG_ALIGN_SIZE(gso_size);
+        pkt->virt_hdr.hdr_len = pkt->hdr_len + sizeof(struct udp_header);
+        break;
+
+    case VIRTIO_NET_HDR_GSO_TCPV4:
+    case VIRTIO_NET_HDR_GSO_TCPV6:
+        iov_to_buf(&pkt->vec[VMXNET_TX_PKT_PL_START_FRAG], pkt->payload_frags,
+                   0, &l4hdr, sizeof(l4hdr));
+        pkt->virt_hdr.hdr_len = pkt->hdr_len + l4hdr.th_off * sizeof(uint32_t);
+        pkt->virt_hdr.gso_size = IP_FRAG_ALIGN_SIZE(gso_size);
+        break;
+
+    default:
+        assert(false);
+    }
+
+    if (csum_enable) {
+        switch (pkt->l4proto) {
+        case IP_PROTO_TCP:
+            pkt->virt_hdr.flags = VIRTIO_NET_HDR_F_NEEDS_CSUM;
+            pkt->virt_hdr.csum_start = pkt->hdr_len;
+            pkt->virt_hdr.csum_offset = offsetof(struct tcp_hdr, th_sum);
+            break;
+        case IP_PROTO_UDP:
+            pkt->virt_hdr.flags = VIRTIO_NET_HDR_F_NEEDS_CSUM;
+            pkt->virt_hdr.csum_start = pkt->hdr_len;
+            pkt->virt_hdr.csum_offset = offsetof(struct udp_hdr, uh_sum);
+            break;
+        default:
+            break;
+        }
+    }
+}
+
+void vmxnet_tx_pkt_setup_vlan_header(struct VmxnetTxPkt *pkt, uint16_t vlan)
+{
+    bool is_new;
+    assert(pkt);
+
+    eth_setup_vlan_headers(pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_base,
+        vlan, &is_new);
+
+    /* update l2hdrlen */
+    if (is_new) {
+        pkt->hdr_len += sizeof(struct vlan_header);
+        pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_len +=
+            sizeof(struct vlan_header);
+    }
+}
+
+bool vmxnet_tx_pkt_add_raw_fragment(struct VmxnetTxPkt *pkt, hwaddr pa,
+    size_t len)
+{
+    hwaddr mapped_len = 0;
+    struct iovec *ventry;
+    assert(pkt);
+    assert(pkt->max_raw_frags > pkt->raw_frags);
+
+    if (!len) {
+        return true;
+     }
+
+    ventry = &pkt->raw[pkt->raw_frags];
+    mapped_len = len;
+
+    ventry->iov_base = cpu_physical_memory_map(pa, &mapped_len, false);
+    ventry->iov_len = mapped_len;
+    pkt->raw_frags += !!ventry->iov_base;
+
+    if ((ventry->iov_base == NULL) || (len != mapped_len)) {
+        return false;
+    }
+
+    return true;
+}
+
+eth_pkt_types_e vmxnet_tx_pkt_get_packet_type(struct VmxnetTxPkt *pkt)
+{
+    assert(pkt);
+
+    return pkt->packet_type;
+}
+
+size_t vmxnet_tx_pkt_get_total_len(struct VmxnetTxPkt *pkt)
+{
+    assert(pkt);
+
+    return pkt->hdr_len + pkt->payload_len;
+}
+
+void vmxnet_tx_pkt_dump(struct VmxnetTxPkt *pkt)
+{
+#ifdef VMXNET_TX_PKT_DEBUG
+    assert(pkt);
+
+    printf("TX PKT: hdr_len: %d, pkt_type: 0x%X, l2hdr_len: %lu, "
+        "l3hdr_len: %lu, payload_len: %u\n", pkt->hdr_len, pkt->packet_type,
+        pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_len,
+        pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_len, pkt->payload_len);
+#endif
+}
+
+void vmxnet_tx_pkt_reset(struct VmxnetTxPkt *pkt)
+{
+    int i;
+
+    /* no assert, as reset can be called before tx_pkt_init */
+    if (!pkt) {
+        return;
+    }
+
+    memset(&pkt->virt_hdr, 0, sizeof(pkt->virt_hdr));
+
+    if (NULL != pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_base) {
+        g_free(pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_base);
+        pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_base = NULL;
+    }
+
+    assert(pkt->vec);
+    for (i = VMXNET_TX_PKT_L2HDR_FRAG;
+         i < pkt->payload_frags + VMXNET_TX_PKT_PL_START_FRAG; i++) {
+        pkt->vec[i].iov_len = 0;
+    }
+    pkt->payload_len = 0;
+    pkt->payload_frags = 0;
+
+    assert(pkt->raw);
+    for (i = 0; i < pkt->raw_frags; i++) {
+        assert(pkt->raw[i].iov_base);
+        cpu_physical_memory_unmap(pkt->raw[i].iov_base, pkt->raw[i].iov_len,
+                                  false, pkt->raw[i].iov_len);
+        pkt->raw[i].iov_len = 0;
+    }
+    pkt->raw_frags = 0;
+
+    pkt->hdr_len = 0;
+    pkt->packet_type = 0;
+    pkt->l4proto = 0;
+}
+
+static void vmxnet_tx_pkt_do_sw_csum(struct VmxnetTxPkt *pkt)
+{
+    struct iovec *iov = &pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG];
+    uint32_t csum_cntr;
+    uint16_t csum = 0;
+    /* num of iovec without vhdr */
+    uint32_t iov_len = pkt->payload_frags + VMXNET_TX_PKT_PL_START_FRAG - 1;
+    uint16_t csl;
+    struct ip_header *iphdr;
+    size_t csum_offset = pkt->virt_hdr.csum_start + pkt->virt_hdr.csum_offset;
+
+    /* Put zero to checksum field */
+    iov_from_buf(iov, iov_len, csum_offset, &csum, sizeof csum);
+
+    /* Calculate L4 TCP/UDP checksum */
+    csl = pkt->payload_len;
+
+    /* data checksum */
+    csum_cntr =
+        net_checksum_add_iov(iov, iov_len, pkt->virt_hdr.csum_start, csl);
+    /* add pseudo header to csum */
+    iphdr = pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_base;
+    csum_cntr += eth_calc_pseudo_hdr_csum(iphdr, csl);
+
+    /* Put the checksum obtained into the packet */
+    csum = cpu_to_be16(net_checksum_finish(csum_cntr));
+    iov_from_buf(iov, iov_len, csum_offset, &csum, sizeof csum);
+}
+
+enum {
+    VMXNET_TX_PKT_FRAGMENT_L2_HDR_POS = 0,
+    VMXNET_TX_PKT_FRAGMENT_L3_HDR_POS,
+    VMXNET_TX_PKT_FRAGMENT_HEADER_NUM
+};
+
+#define VMXNET_MAX_FRAG_SG_LIST (64)
+
+static size_t vmxnet_tx_pkt_fetch_fragment(struct VmxnetTxPkt *pkt,
+    int *src_idx, size_t *src_offset, struct iovec *dst, int *dst_idx)
+{
+    size_t fetched = 0;
+    struct iovec *src = pkt->vec;
+
+    *dst_idx = VMXNET_TX_PKT_FRAGMENT_HEADER_NUM;
+
+    while (fetched < pkt->virt_hdr.gso_size) {
+
+        /* no more place in fragment iov */
+        if (*dst_idx == VMXNET_MAX_FRAG_SG_LIST) {
+            break;
+        }
+
+        /* no more data in iovec */
+        if (*src_idx == (pkt->payload_frags + VMXNET_TX_PKT_PL_START_FRAG)) {
+            break;
+        }
+
+
+        dst[*dst_idx].iov_base = src[*src_idx].iov_base + *src_offset;
+        dst[*dst_idx].iov_len = MIN(src[*src_idx].iov_len - *src_offset,
+            pkt->virt_hdr.gso_size - fetched);
+
+        *src_offset += dst[*dst_idx].iov_len;
+        fetched += dst[*dst_idx].iov_len;
+
+        if (*src_offset == src[*src_idx].iov_len) {
+            *src_offset = 0;
+            (*src_idx)++;
+        }
+
+        (*dst_idx)++;
+    }
+
+    return fetched;
+}
+
+static size_t vmxnet_tx_pkt_do_sw_fragmentation(struct VmxnetTxPkt *pkt,
+    NetClientState *nc)
+{
+    struct iovec fragment[VMXNET_MAX_FRAG_SG_LIST];
+    size_t fragment_len = 0;
+    bool more_frags = false;
+    /* some poiners for shorter code */
+    void *l2_iov_base, *l3_iov_base;
+    size_t l2_iov_len, l3_iov_len;
+    int src_idx =  VMXNET_TX_PKT_PL_START_FRAG, dst_idx;
+    size_t src_offset = 0;
+    size_t bytes_sent = 0;
+    size_t fragment_offset = 0;
+
+    l2_iov_base = pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_base;
+    l2_iov_len = pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_len;
+    l3_iov_base = pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_base;
+    l3_iov_len = pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_len;
+
+    /* Copy headers */
+    fragment[VMXNET_TX_PKT_FRAGMENT_L2_HDR_POS].iov_base = l2_iov_base;
+    fragment[VMXNET_TX_PKT_FRAGMENT_L2_HDR_POS].iov_len = l2_iov_len;
+    fragment[VMXNET_TX_PKT_FRAGMENT_L3_HDR_POS].iov_base = l3_iov_base;
+    fragment[VMXNET_TX_PKT_FRAGMENT_L3_HDR_POS].iov_len = l3_iov_len;
+
+
+    /* Put as much data as possible and send */
+    do {
+        fragment_len = vmxnet_tx_pkt_fetch_fragment(pkt, &src_idx, &src_offset,
+            fragment, &dst_idx);
+
+        more_frags = (fragment_offset + fragment_len < pkt->payload_len);
+
+        eth_setup_ip4_fragmentation(l2_iov_base, l2_iov_len, l3_iov_base,
+            l3_iov_len, fragment_len, fragment_offset, more_frags);
+
+        eth_fix_ip4_checksum(l3_iov_base, l3_iov_len);
+
+        bytes_sent += qemu_sendv_packet(nc, fragment, dst_idx);
+
+        fragment_offset += fragment_len;
+
+    } while (more_frags);
+
+    return bytes_sent;
+}
+
+size_t vmxnet_tx_pkt_send(struct VmxnetTxPkt *pkt, NetClientState *nc)
+{
+    size_t bytes_sent = 0;
+    assert(pkt);
+
+    if (!pkt->has_virt_hdr &&
+        pkt->virt_hdr.flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) {
+        vmxnet_tx_pkt_do_sw_csum(pkt);
+    }
+
+    /*
+     * Since underlying infrastructure does not support IP datagrams longer
+     * than 64K we should drop such packets and don't even try to send
+     */
+    if (VIRTIO_NET_HDR_GSO_NONE != pkt->virt_hdr.gso_type) {
+        if (pkt->payload_len >
+            ETH_MAX_IP_DGRAM_LEN -
+            pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_len) {
+            return 0;
+        }
+    }
+
+    if (pkt->has_virt_hdr ||
+        pkt->virt_hdr.gso_type == VIRTIO_NET_HDR_GSO_NONE) {
+        bytes_sent = qemu_sendv_packet(nc, pkt->vec,
+            pkt->payload_frags + VMXNET_TX_PKT_PL_START_FRAG);
+        goto func_exit;
+    }
+
+    bytes_sent = vmxnet_tx_pkt_do_sw_fragmentation(pkt, nc);
+
+func_exit:
+    return bytes_sent;
+}
+
+/*=============================================================================
+ *=============================================================================
+ *
+ *                            RX CODE
+ *
+ *=============================================================================
+ *===========================================================================*/
+
+/*
+ * RX packet may contain up to 2 fragments - rebuilt eth header
+ * in case of VLAN tag stripping
+ * and payload received from QEMU - in any case
+ */
+#define VMXNET_MAX_RX_PACKET_FRAGMENTS (2)
+
+struct VmxnetRxPkt{
+    struct virtio_net_hdr virt_hdr;
+    uint8_t ehdr_buf[ETH_MAX_L2_HDR_LEN];
+    struct iovec vec[VMXNET_MAX_RX_PACKET_FRAGMENTS];
+    uint16_t vec_len;
+    uint32_t tot_len;
+    uint16_t tci;
+    bool vlan_stripped;
+    bool has_virt_hdr;
+    eth_pkt_types_e packet_type;
+
+    /* Analysis results */
+    bool isip4;
+    bool isip6;
+    bool isudp;
+    bool istcp;
+};
+
+void vmxnet_rx_pkt_init(struct VmxnetRxPkt **pkt, bool has_virt_hdr)
+{
+    struct VmxnetRxPkt *p = g_malloc0(sizeof *p);
+    p->has_virt_hdr = has_virt_hdr;
+    *pkt = p;
+}
+
+void vmxnet_rx_pkt_uninit(struct VmxnetRxPkt *pkt)
+{
+    if (pkt) {
+        g_free(pkt);
+    }
+}
+
+struct virtio_net_hdr *vmxnet_rx_pkt_get_vhdr(struct VmxnetRxPkt *pkt)
+{
+    assert(pkt);
+    return &pkt->virt_hdr;
+}
+
+void vmxnet_rx_pkt_attach_data(struct VmxnetRxPkt *pkt, const void *data,
+                               size_t len, bool strip_vlan)
+{
+    uint16_t tci = 0;
+    uint16_t ploff;
+    assert(pkt);
+    pkt->vlan_stripped = false;
+
+    if (strip_vlan) {
+        pkt->vlan_stripped = eth_strip_vlan(data, pkt->ehdr_buf, &ploff, &tci);
+    }
+
+    if (pkt->vlan_stripped) {
+        pkt->vec[0].iov_base = pkt->ehdr_buf;
+        pkt->vec[0].iov_len = ploff - sizeof(struct vlan_header);
+        pkt->vec[1].iov_base = (uint8_t *) data + ploff;
+        pkt->vec[1].iov_len = len - ploff;
+        pkt->vec_len = 2;
+        pkt->tot_len = len - ploff + sizeof(struct eth_header);
+    } else {
+        pkt->vec[0].iov_base = (void *)data;
+        pkt->vec[0].iov_len = len;
+        pkt->vec_len = 1;
+        pkt->tot_len = len;
+    }
+
+    pkt->tci = tci;
+
+    eth_get_protocols(data, len, &pkt->isip4, &pkt->isip6,
+        &pkt->isudp, &pkt->istcp);
+}
+
+void vmxnet_rx_pkt_dump(struct VmxnetRxPkt *pkt)
+{
+#ifdef VMXNET_RX_PKT_DEBUG
+    VmxnetRxPkt *pkt = (VmxnetRxPkt *)pkt;
+    assert(pkt);
+
+    printf("RX PKT: tot_len: %d, vlan_stripped: %d, vlan_tag: %d\n",
+              pkt->tot_len, pkt->vlan_stripped, pkt->tci);
+#endif
+}
+
+void vmxnet_rx_pkt_set_packet_type(struct VmxnetRxPkt *pkt,
+    eth_pkt_types_e packet_type)
+{
+    assert(pkt);
+
+    pkt->packet_type = packet_type;
+
+}
+
+eth_pkt_types_e vmxnet_rx_pkt_get_packet_type(struct VmxnetRxPkt *pkt)
+{
+    assert(pkt);
+
+    return pkt->packet_type;
+}
+
+size_t vmxnet_rx_pkt_get_total_len(struct VmxnetRxPkt *pkt)
+{
+    assert(pkt);
+
+    return pkt->tot_len;
+}
+
+void vmxnet_rx_pkt_get_protocols(struct VmxnetRxPkt *pkt,
+                                 bool *isip4, bool *isip6,
+                                 bool *isudp, bool *istcp)
+{
+    assert(pkt);
+
+    *isip4 = pkt->isip4;
+    *isip6 = pkt->isip6;
+    *isudp = pkt->isudp;
+    *istcp = pkt->istcp;
+}
+
+struct iovec *vmxnet_rx_pkt_get_iovec(struct VmxnetRxPkt *pkt)
+{
+    assert(pkt);
+
+    return pkt->vec;
+}
+
+void vmxnet_rx_pkt_set_vhdr(struct VmxnetRxPkt *pkt,
+                            struct virtio_net_hdr *vhdr)
+{
+    assert(pkt);
+
+    memcpy(&pkt->virt_hdr, vhdr, sizeof pkt->virt_hdr);
+}
+
+bool vmxnet_rx_pkt_is_vlan_stripped(struct VmxnetRxPkt *pkt)
+{
+    assert(pkt);
+
+    return pkt->vlan_stripped;
+}
+
+bool vmxnet_rx_pkt_has_virt_hdr(struct VmxnetRxPkt *pkt)
+{
+    assert(pkt);
+
+    return pkt->has_virt_hdr;
+}
+
+uint16_t vmxnet_rx_pkt_get_num_frags(struct VmxnetRxPkt *pkt)
+{
+    assert(pkt);
+
+    return pkt->vec_len;
+}
+
+uint16_t vmxnet_rx_pkt_get_vlan_tag(struct VmxnetRxPkt *pkt)
+{
+    assert(pkt);
+
+    return pkt->tci;
+}
hw/vmxnet_pkt.h new file +309 -0

309 lines not on this page · raw message

Signature · 1 line
1.7.11.7
#11 lore raw

Re: [Qemu-devel] [PATCH V8 4/5] Adding packet abstraction for VMWARE network devices

hw/vmxnet_pkt.c line 94

Quoted 10 lines · Dmitry Fleytman
hw/vmxnet_pkt.c new file +10 -0
+void vmxnet_tx_pkt_uninit(struct VmxnetTxPkt *pkt)
+{
+    if (pkt) {
+        if (pkt->vec) {
+            g_free(pkt->vec);
+        }
+
+        if (pkt->raw) {
+            g_free(pkt->raw);
+        }

g_free(NULL) is a nop so the conditional is not necessary.

Quoted 29 lines
hw/vmxnet_pkt.c new file +29 -0
+static size_t vmxnet_tx_pkt_do_sw_fragmentation(struct VmxnetTxPkt *pkt,
+    NetClientState *nc)
+{
+    struct iovec fragment[VMXNET_MAX_FRAG_SG_LIST];
+    size_t fragment_len = 0;
+    bool more_frags = false;
+    /* some poiners for shorter code */
+    void *l2_iov_base, *l3_iov_base;
+    size_t l2_iov_len, l3_iov_len;
+    int src_idx =  VMXNET_TX_PKT_PL_START_FRAG, dst_idx;
+    size_t src_offset = 0;
+    size_t bytes_sent = 0;
+    size_t fragment_offset = 0;
+
+    l2_iov_base = pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_base;
+    l2_iov_len = pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_len;
+    l3_iov_base = pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_base;
+    l3_iov_len = pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_len;
+
+    /* Copy headers */
+    fragment[VMXNET_TX_PKT_FRAGMENT_L2_HDR_POS].iov_base = l2_iov_base;
+    fragment[VMXNET_TX_PKT_FRAGMENT_L2_HDR_POS].iov_len = l2_iov_len;
+    fragment[VMXNET_TX_PKT_FRAGMENT_L3_HDR_POS].iov_base = l3_iov_base;
+    fragment[VMXNET_TX_PKT_FRAGMENT_L3_HDR_POS].iov_len = l3_iov_len;
+
+
+    /* Put as much data as possible and send */
+    do {
+        fragment_len = vmxnet_tx_pkt_fetch_fragment(pkt, &src_idx, &src_offset,

hw/vmxnet_pkt.c line 542

Quoted 10 lines
hw/vmxnet_pkt.c new file +10 -0
+            fragment, &dst_idx);
+
+        more_frags = (fragment_offset + fragment_len < pkt->payload_len);
+
+        eth_setup_ip4_fragmentation(l2_iov_base, l2_iov_len, l3_iov_base,
+            l3_iov_len, fragment_len, fragment_offset, more_frags);
+
+        eth_fix_ip4_checksum(l3_iov_base, l3_iov_len);
+
+        bytes_sent += qemu_sendv_packet(nc, fragment, dst_idx);

The return values from qemu_sendv_packet() are:

Positive - Number of bytes successfully sent
0 - Packet queued for deferred transmission
Negative - Error

Accumulating bytes_sent is not safe against negative return values and also questionable for 0 return values.

Stefan

#12 lore raw

Re: [Qemu-devel] [PATCH V8 4/5] Adding packet abstraction for VMWARE network devices

Quoted 62 lines · Stefan Hajnoczi

hw/vmxnet_pkt.c line 94

Quoted 10 lines · Dmitry Fleytman
hw/vmxnet_pkt.c new file +10 -0
+void vmxnet_tx_pkt_uninit(struct VmxnetTxPkt *pkt)
+{
+    if (pkt) {
+        if (pkt->vec) {
+            g_free(pkt->vec);
+        }
+
+        if (pkt->raw) {
+            g_free(pkt->raw);
+        }

g_free(NULL) is a nop so the conditional is not necessary.

Thanks for pointing out. Removed.

Quoted 19 lines
hw/vmxnet_pkt.c new file +19 -0
+static size_t vmxnet_tx_pkt_do_sw_fragmentation(struct VmxnetTxPkt *pkt,
+    NetClientState *nc)
+{
+    struct iovec fragment[VMXNET_MAX_FRAG_SG_LIST];
+    size_t fragment_len = 0;
+    bool more_frags = false;
+    /* some poiners for shorter code */
+    void *l2_iov_base, *l3_iov_base;
+    size_t l2_iov_len, l3_iov_len;
+    int src_idx =  VMXNET_TX_PKT_PL_START_FRAG, dst_idx;
+    size_t src_offset = 0;
+    size_t bytes_sent = 0;
+    size_t fragment_offset = 0;
+
+    l2_iov_base = pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_base;
+    l2_iov_len = pkt->vec[VMXNET_TX_PKT_L2HDR_FRAG].iov_len;
+    l3_iov_base = pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_base;
+    l3_iov_len = pkt->vec[VMXNET_TX_PKT_L3HDR_FRAG].iov_len;
+

hw/vmxnet_pkt.c

Quoted 10 lines
hw/vmxnet_pkt.c new file +10 -0
+    /* Copy headers */
+    fragment[VMXNET_TX_PKT_FRAGMENT_L2_HDR_POS].iov_base = l2_iov_base;
+    fragment[VMXNET_TX_PKT_FRAGMENT_L2_HDR_POS].iov_len = l2_iov_len;
+    fragment[VMXNET_TX_PKT_FRAGMENT_L3_HDR_POS].iov_base = l3_iov_base;
+    fragment[VMXNET_TX_PKT_FRAGMENT_L3_HDR_POS].iov_len = l3_iov_len;
+
+
+    /* Put as much data as possible and send */
+    do {
+        fragment_len = vmxnet_tx_pkt_fetch_fragment(pkt, &src_idx,

&src_offset,

hw/vmxnet_pkt.c

Quoted 3 lines
hw/vmxnet_pkt.c new file +3 -0
+            fragment, &dst_idx);
+
+        more_frags = (fragment_offset + fragment_len <

pkt->payload_len);

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+        eth_setup_ip4_fragmentation(l2_iov_base, l2_iov_len,

l3_iov_base,

hw/vmxnet_pkt.c line 542

Quoted 5 lines
hw/vmxnet_pkt.c new file +5 -0
+            l3_iov_len, fragment_len, fragment_offset, more_frags);
+
+        eth_fix_ip4_checksum(l3_iov_base, l3_iov_len);
+
+        bytes_sent += qemu_sendv_packet(nc, fragment, dst_idx);

The return values from qemu_sendv_packet() are:

Quoted 8 lines

Positive - Number of bytes successfully sent
0 - Packet queued for deferred transmission
Negative - Error

Accumulating bytes_sent is not safe against negative return values and also questionable for 0 return values.

Thanks for clarification.
Fixed.

Quoted 2 lines

Stefan

Signature · 7 lines
Dmitry Fleytman
Technology Expert and Consultant,

Daynix Computing Ltd.

Cell: +972-54-2819481
Skype: dmitry.fleytman
#13 lore raw

PATCH v8 5/5 Adding VMXNET3 device implementation

In reply to Dmitry Fleytman
---
 default-configs/pci.mak |    1 +
 hw/Makefile.objs        |    1 +
 hw/pci.h                |    1 +
 hw/vmxnet3.c            | 2437 +++++++++++++++++++++++++++++++++++++++++++++++
 hw/vmxnet3.h            |  762 +++++++++++++++
 5 files changed, 3202 insertions(+)
 create mode 100644 hw/vmxnet3.c
 create mode 100644 hw/vmxnet3.h
default-configs/pci.mak +1 -0
index ae9d1eb..cdc0cdc 100644
@@ -13,6 +13,7 @@ CONFIG_LSI_SCSI_PCI=y
 CONFIG_MEGASAS_SCSI_PCI=y
 CONFIG_RTL8139_PCI=y
 CONFIG_E1000_PCI=y
+CONFIG_VMXNET3_PCI=y
 CONFIG_IDE_CORE=y
 CONFIG_IDE_QDEV=y
 CONFIG_IDE_PCI=y
hw/Makefile.objs +1 -0
index d581d8d..1a38481 100644
@@ -111,6 +111,7 @@ common-obj-$(CONFIG_PCNET_PCI) += pcnet-pci.o
 common-obj-$(CONFIG_PCNET_COMMON) += pcnet.o
 common-obj-$(CONFIG_E1000_PCI) += e1000.o
 common-obj-$(CONFIG_RTL8139_PCI) += rtl8139.o
+common-obj-$(CONFIG_VMXNET3_PCI) += vmxnet3.o vmxnet_utils.o vmxnet_pkt.o
 
 common-obj-$(CONFIG_SMC91C111) += smc91c111.o
 common-obj-$(CONFIG_LAN9118) += lan9118.o
hw/pci.h +1 -0
index 4da0c2a..ad52b8f 100644
@@ -60,6 +60,7 @@
 #define PCI_DEVICE_ID_VMWARE_NET         0x0720
 #define PCI_DEVICE_ID_VMWARE_SCSI        0x0730
 #define PCI_DEVICE_ID_VMWARE_IDE         0x1729
+#define PCI_DEVICE_ID_VMWARE_VMXNET3     0x07B0
 
 /* Intel (0x8086) */
 #define PCI_DEVICE_ID_INTEL_82551IT      0x1209
hw/vmxnet3.c new file +2437 -0

2,437 lines not on this page · raw message

hw/vmxnet3.h new file +762 -0

542 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.7.11.7

Part of the diff is not shown here. View the raw message.

#14 lore raw

Re: [Qemu-devel] [PATCH V8 5/5] Adding VMXNET3 device implementation

hw/vmxnet3.c line 2122

Quoted 7 lines · Dmitry Fleytman
hw/vmxnet3.c new file +7 -0
+static void vmxnet3_get_ring_from_file(QEMUFile *f, Vmxnet3Ring *r)
+{
+#if HWADDR_BITS == 64
+    r->pa = qemu_get_be64(f);
+#else
+    r->pa = qemu_get_be32(f);
+#endif

hw/virtio.c always uses be64 for its hwaddr pa field. That way you avoid the #ifdefs.

hw/vmxnet3.c line 2367

Quoted 7 lines
hw/vmxnet3.c new file +7 -0
+#if HWADDR_BITS == 64
+            VMSTATE_UINT64(drv_shmem, VMXNET3State),
+            VMSTATE_UINT64(temp_shared_guest_driver_memory, VMXNET3State),
+#else
+            VMSTATE_UINT32(drv_shmem, VMXNET3State),
+            VMSTATE_UINT32(temp_shared_guest_driver_memory, VMXNET3State),
+#endif

The hardware register interface is 2 32-bit registers. Can you use uint64_t for these fields and avoid the #ifdefs?

#15 lore raw

Re: [Qemu-devel] [PATCH V8 5/5] Adding VMXNET3 device implementation

Quoted 13 lines · Stefan Hajnoczi

hw/vmxnet3.c line 2122

Quoted 7 lines · Dmitry Fleytman
hw/vmxnet3.c new file +7 -0
+static void vmxnet3_get_ring_from_file(QEMUFile *f, Vmxnet3Ring *r)
+{
+#if HWADDR_BITS == 64
+    r->pa = qemu_get_be64(f);
+#else
+    r->pa = qemu_get_be32(f);
+#endif

hw/virtio.c always uses be64 for its hwaddr pa field. That way you avoid the #ifdefs.

Good point. Done.

Quoted 4 lines

hw/vmxnet3.c

Quoted 3 lines
hw/vmxnet3.c new file +3 -0
+#if HWADDR_BITS == 64
+            VMSTATE_UINT64(drv_shmem, VMXNET3State),
+            VMSTATE_UINT64(temp_shared_guest_driver_memory,

VMXNET3State),

Quoted 9 lines

hw/vmxnet3.c

Quoted 3 lines
hw/vmxnet3.c new file +3 -0
+#else
+            VMSTATE_UINT32(drv_shmem, VMXNET3State),
+            VMSTATE_UINT32(temp_shared_guest_driver_memory,

VMXNET3State),

Quoted 1 line

+#endif

The hardware register interface is 2 32-bit registers. Can you use uint64_t for these fields and avoid the #ifdefs?

Done.

Signature · 7 lines
Dmitry Fleytman
Technology Expert and Consultant,

Daynix Computing Ltd.

Cell: +972-54-2819481
Skype: dmitry.fleytman
#16 lore raw

Re: [Qemu-devel] [PATCH V8 5/5] Adding VMXNET3 device implementation

In reply to Dmitry Fleytman
Quoted 2,529 lines · Dmitry Fleytman

---

default-configs/pci.mak |    1 +
hw/Makefile.objs        |    1 +
hw/pci.h                |    1 +
hw/vmxnet3.c            | 2437 +++++++++++++++++++++++++++++++++++++++++++++++
hw/vmxnet3.h            |  762 +++++++++++++++
5 files changed, 3202 insertions(+)

create mode 100644 hw/vmxnet3.c
create mode 100644 hw/vmxnet3.h

default-configs/pci.mak +0 -0
index ae9d1eb..cdc0cdc 100644
@@ -13,6 +13,7 @@ CONFIG_LSI_SCSI_PCI=y

CONFIG_MEGASAS_SCSI_PCI=y
CONFIG_RTL8139_PCI=y
CONFIG_E1000_PCI=y
+CONFIG_VMXNET3_PCI=y
CONFIG_IDE_CORE=y
CONFIG_IDE_QDEV=y
CONFIG_IDE_PCI=y

hw/Makefile.objs +0 -0
index d581d8d..1a38481 100644
@@ -111,6 +111,7 @@ common-obj-$(CONFIG_PCNET_PCI) += pcnet-pci.o

common-obj-$(CONFIG_PCNET_COMMON) += pcnet.o common-obj-$(CONFIG_E1000_PCI) += e1000.o common-obj-$(CONFIG_RTL8139_PCI) += rtl8139.o +common-obj-$(CONFIG_VMXNET3_PCI) += vmxnet3.o vmxnet_utils.o vmxnet_pkt.o

common-obj-$(CONFIG_SMC91C111) += smc91c111.o common-obj-$(CONFIG_LAN9118) += lan9118.o

hw/pci.h +1 -0
index 4da0c2a..ad52b8f 100644
@@ -60,6 +60,7 @@
 #define PCI_DEVICE_ID_VMWARE_NET         0x0720
 #define PCI_DEVICE_ID_VMWARE_SCSI        0x0730
 #define PCI_DEVICE_ID_VMWARE_IDE         0x1729
+#define PCI_DEVICE_ID_VMWARE_VMXNET3     0x07B0
 
 /* Intel (0x8086) */
 #define PCI_DEVICE_ID_INTEL_82551IT      0x1209
hw/vmxnet3.c new file +2437 -0
new file mode 100644
index 0000000..c81b434
@@ -0,0 +1,2437 @@
+/*
+ * QEMU VMWARE VMXNET3 paravirtual NIC
+ *
+ * Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com)
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Dmitry Fleytman <[email protected]>
+ * Tamir Shomer <[email protected]>
+ * Yan Vugenfirer <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "hw.h"
+#include "pci.h"
+#include "net.h"
+#include "virtio-net.h"
+#include "net/tap.h"
+#include "net/checksum.h"
+#include "sysemu.h"
+#include "iov.h"
+#include "bswap.h"
+#include "msix.h"
+#include "msi.h"
+
+#include "vmxnet3.h"
+#include "vmxnet_debug.h"
+#include "vmware_utils.h"
+#include "vmxnet_utils.h"
+#include "vmxnet_pkt.h"
+
+#define PCI_DEVICE_ID_VMWARE_VMXNET3_REVISION 0x1
+#define VMXNET3_MSIX_BAR_SIZE 0x2000
+
+#define VMXNET3_BAR0_IDX      (0)
+#define VMXNET3_BAR1_IDX      (1)
+#define VMXNET3_MSIX_BAR_IDX  (2)
+
+#define VMXNET3_OFF_MSIX_TABLE (0x000)
+#define VMXNET3_OFF_MSIX_PBA   (0x800)
+
+/* Link speed in Mbps should be shifted by 16 */
+#define VMXNET3_LINK_SPEED      (1000 << 16)
+
+/* Link status: 1 - up, 0 - down. */
+#define VMXNET3_LINK_STATUS_UP  0x1
+
+/* Least significant bit should be set for revision and version */
+#define VMXNET3_DEVICE_VERSION    0x1
+#define VMXNET3_DEVICE_REVISION   0x1
+
+/* Macros for rings descriptors access */
+#define VMXNET3_READ_TX_QUEUE_DESCR8(dpa, field) \
+    (vmw_shmem_ld8(dpa + offsetof(struct Vmxnet3_TxQueueDesc, field)))
+
+#define VMXNET3_WRITE_TX_QUEUE_DESCR8(dpa, field, value) \
+    (vmw_shmem_st8(dpa + offsetof(struct Vmxnet3_TxQueueDesc, field, value)))
+
+#define VMXNET3_READ_TX_QUEUE_DESCR32(dpa, field) \
+    (vmw_shmem_ld32(dpa + offsetof(struct Vmxnet3_TxQueueDesc, field)))
+
+#define VMXNET3_WRITE_TX_QUEUE_DESCR32(dpa, field, value) \
+    (vmw_shmem_st32(dpa + offsetof(struct Vmxnet3_TxQueueDesc, field), value))
+
+#define VMXNET3_READ_TX_QUEUE_DESCR64(dpa, field) \
+    (vmw_shmem_ld64(dpa + offsetof(struct Vmxnet3_TxQueueDesc, field)))
+
+#define VMXNET3_WRITE_TX_QUEUE_DESCR64(dpa, field, value) \
+    (vmw_shmem_st64(dpa + offsetof(struct Vmxnet3_TxQueueDesc, field), value))
+
+#define VMXNET3_READ_RX_QUEUE_DESCR64(dpa, field) \
+    (vmw_shmem_ld64(dpa + offsetof(struct Vmxnet3_RxQueueDesc, field)))
+
+#define VMXNET3_READ_RX_QUEUE_DESCR32(dpa, field) \
+    (vmw_shmem_ld32(dpa + offsetof(struct Vmxnet3_RxQueueDesc, field)))
+
+#define VMXNET3_WRITE_RX_QUEUE_DESCR64(dpa, field, value) \
+    (vmw_shmem_st64(dpa + offsetof(struct Vmxnet3_RxQueueDesc, field), value))
+
+#define VMXNET3_WRITE_RX_QUEUE_DESCR8(dpa, field, value) \
+    (vmw_shmem_st8(dpa + offsetof(struct Vmxnet3_RxQueueDesc, field), value))
+
+/* Macros for guest driver shared area access */
+#define VMXNET3_READ_DRV_SHARED64(shpa, field) \
+    (vmw_shmem_ld64(shpa + offsetof(struct Vmxnet3_DriverShared, field)))
+
+#define VMXNET3_READ_DRV_SHARED32(shpa, field) \
+    (vmw_shmem_ld32(shpa + offsetof(struct Vmxnet3_DriverShared, field)))
+
+#define VMXNET3_WRITE_DRV_SHARED32(shpa, field, val) \
+    (vmw_shmem_st32(shpa + offsetof(struct Vmxnet3_DriverShared, field), val))
+
+#define VMXNET3_READ_DRV_SHARED16(shpa, field) \
+    (vmw_shmem_ld16(shpa + offsetof(struct Vmxnet3_DriverShared, field)))
+
+#define VMXNET3_READ_DRV_SHARED8(shpa, field) \
+    (vmw_shmem_ld8(shpa + offsetof(struct Vmxnet3_DriverShared, field)))
+
+#define VMXNET3_READ_DRV_SHARED(shpa, field, b, l) \
+    (vmw_shmem_read(shpa + offsetof(struct Vmxnet3_DriverShared, field), b, l))
+
+/* Cyclic ring abstraction */
+typedef struct {
+    hwaddr pa;
+    size_t size;
+    size_t cell_size;
+    size_t next;
+    uint8_t gen;
+} Vmxnet3Ring;
+
+static inline void vmxnet3_ring_init(Vmxnet3Ring *ring,
+                                     hwaddr pa,
+                                     size_t size,
+                                     size_t cell_size,
+                                     bool zero_region)
+{
+    ring->pa = pa;
+    ring->size = size;
+    ring->cell_size = cell_size;
+    ring->gen = VMXNET3_INIT_GEN;
+    ring->next = 0;
+
+    if (zero_region) {
+        vmw_shmem_set(pa, 0, size * cell_size);
+    }
+}
+
+#define VMXNET3_RING_DUMP(macro, ring_name, ridx, r)                         \
+    macro("%s#%d: base %" PRIx64 " size %lu cell_size %lu gen %d next %lu",  \
+          (ring_name), (ridx),                                               \
+          (r)->pa, (r)->size, (r)->cell_size, (r)->gen, (r)->next)
+
+static inline void vmxnet3_ring_inc(Vmxnet3Ring *ring)
+{
+    if (++ring->next >= ring->size) {
+        ring->next = 0;
+        ring->gen ^= 1;
+    }
+}
+
+static inline void vmxnet3_ring_dec(Vmxnet3Ring *ring)
+{
+    if (ring->next-- == 0) {
+        ring->next = ring->size - 1;
+        ring->gen ^= 1;
+    }
+}
+
+static inline hwaddr vmxnet3_ring_curr_cell_pa(Vmxnet3Ring *ring)
+{
+    return ring->pa + ring->next * ring->cell_size;
+}
+
+static inline void vmxnet3_ring_read_curr_cell(Vmxnet3Ring *ring, void *buff)
+{
+    vmw_shmem_read(vmxnet3_ring_curr_cell_pa(ring), buff, ring->cell_size);
+}
+
+static inline void vmxnet3_ring_write_curr_cell(Vmxnet3Ring *ring, void *buff)
+{
+    vmw_shmem_write(vmxnet3_ring_curr_cell_pa(ring), buff, ring->cell_size);
+}
+
+static inline size_t vmxnet3_ring_curr_cell_idx(Vmxnet3Ring *ring)
+{
+    return ring->next;
+}
+
+static inline uint8_t vmxnet3_ring_curr_gen(Vmxnet3Ring *ring)
+{
+    return ring->gen;
+}
+
+/* Debug trace-related functions */
+static inline void
+vmxnet3_dump_tx_descr(struct Vmxnet3_TxDesc *descr)
+{
+    VMW_PKPRN("TX DESCR: "
+              "addr %" PRIx64 ", len: %d, gen: %d, rsvd: %d, "
+              "dtype: %d, ext1: %d, msscof: %d, hlen: %d, om: %d, "
+              "eop: %d, cq: %d, ext2: %d, ti: %d, tci: %d",
+              le64_to_cpu(descr->addr), descr->len, descr->gen, descr->rsvd,
+              descr->dtype, descr->ext1, descr->msscof, descr->hlen, descr->om,
+              descr->eop, descr->cq, descr->ext2, descr->ti, descr->tci);
+}
+
+static inline void
+vmxnet3_dump_virt_hdr(struct virtio_net_hdr *vhdr)
+{
+    VMW_PKPRN("VHDR: flags 0x%x, gso_type: 0x%x, hdr_len: %d, gso_size: %d, "
+              "csum_start: %d, csum_offset: %d",
+              vhdr->flags, vhdr->gso_type, vhdr->hdr_len, vhdr->gso_size,
+              vhdr->csum_start, vhdr->csum_offset);
+}
+
+static inline void
+vmxnet3_dump_rx_descr(struct Vmxnet3_RxDesc *descr)
+{
+    VMW_PKPRN("RX DESCR: addr %" PRIx64 ", len: %d, gen: %d, rsvd: %d, "
+              "dtype: %d, ext1: %d, btype: %d",
+              le64_to_cpu(descr->addr), descr->len, descr->gen,
+              descr->rsvd, descr->dtype, descr->ext1, descr->btype);
+}
+
+/* Device state and helper functions */
+#define VMXNET3_RX_RINGS_PER_QUEUE (2)
+
+typedef struct {
+    Vmxnet3Ring tx_ring;
+    Vmxnet3Ring comp_ring;
+
+    uint8_t intr_idx;
+    hwaddr tx_stats_pa;
+    struct UPT1_TxStats txq_stats;
+} Vmxnet3TxqDescr;
+
+typedef struct {
+    Vmxnet3Ring rx_ring[VMXNET3_RX_RINGS_PER_QUEUE];
+    Vmxnet3Ring comp_ring;
+    uint8_t intr_idx;
+    hwaddr rx_stats_pa;
+    struct UPT1_RxStats rxq_stats;
+} Vmxnet3RxqDescr;
+
+typedef struct {
+    bool is_masked;
+    bool is_pending;
+    bool is_asserted;
+} Vmxnet3IntState;
+
+typedef struct {
+        PCIDevice dev;
+        NICState *nic;
+        NICConf conf;
+        MemoryRegion bar0;
+        MemoryRegion bar1;
+        MemoryRegion msix_bar;
+
+        Vmxnet3RxqDescr rxq_descr[VMXNET3_DEVICE_MAX_RX_QUEUES];
+        Vmxnet3TxqDescr txq_descr[VMXNET3_DEVICE_MAX_TX_QUEUES];
+
+        /* Whether MSI-X support was installed successfully */
+        bool msix_used;
+        /* Whether MSI support was installed successfully */
+        bool msi_used;
+        hwaddr drv_shmem;
+        hwaddr temp_shared_guest_driver_memory;
+
+        uint8_t txq_num;
+
+        /* This boolean tells whether RX packet being indicated has to */
+        /* be split into head and body chunks from different RX rings  */
+        bool rx_packets_compound;
+
+        bool rx_vlan_stripping;
+        bool lro_supported;
+
+        uint8_t rxq_num;
+
+        /* Network MTU */
+        uint32_t mtu;
+
+        /* Maximum number of fragments for indicated TX packets */
+        uint32_t max_tx_frags;
+
+        /* Maximum number of fragments for indicated RX packets */
+        uint16_t max_rx_frags;
+
+        /* Index for events interrupt */
+        uint8_t event_int_idx;
+
+        /* Whether automatic interrupts masking enabled */
+        bool auto_int_masking;
+
+        bool peer_has_vhdr;
+
+        /* TX packets to QEMU interface */
+        struct VmxnetTxPkt *tx_pkt;
+        uint32_t offload_mode;
+        uint32_t cso_or_gso_size;
+        uint16_t tci;
+        bool needs_vlan;
+
+        struct VmxnetRxPkt *rx_pkt;
+
+        bool tx_sop;
+        bool skip_current_tx_pkt;
+
+        uint32_t device_active;
+        uint32_t last_command;
+
+        uint32_t link_status_and_speed;
+
+        Vmxnet3IntState interrupt_states[VMXNET3_MAX_INTRS];
+
+        uint32_t temp_mac;   /* To store the low part first */
+
+        MACAddr perm_mac;
+        uint32_t vlan_table[VMXNET3_VFT_SIZE];
+        uint32_t rx_mode;
+        MACAddr *mcast_list;
+        uint32_t mcast_list_len;
+        uint32_t mcast_list_buff_size; /* needed for live migration. */
+} VMXNET3State;
+
+/* Interrupt management */
+
+/*
+ *This function returns sign whether interrupt line is in asserted state
+ * This depends on the type of interrupt used. For INTX interrupt line will
+ * be asserted until explicit deassertion, for MSI(X) interrupt line will
+ * be deasserted automatically due to notification semantics of the MSI(X)
+ * interrupts
+ */
+static bool _vmxnet3_assert_interrupt_line(VMXNET3State *s, uint32_t int_idx)
+{
+    if (s->msix_used && msix_enabled(&s->dev)) {
+        VMW_IRPRN("Sending MSI-X notification for vector %u", int_idx);
+        msix_notify(&s->dev, int_idx);
+        return false;
+    }
+    if (s->msi_used && msi_enabled(&s->dev)) {
+        VMW_IRPRN("Sending MSI notification for vector %u", int_idx);
+        msi_notify(&s->dev, int_idx);
+        return false;
+    }
+
+    VMW_IRPRN("Asserting line for interrupt %u", int_idx);
+    qemu_set_irq(s->dev.irq[int_idx], 1);
+    return true;
+}
+
+static void _vmxnet3_deassert_interrupt_line(VMXNET3State *s, int lidx)
+{
+    /*
+     * This function should never be called for MSI(X) interrupts
+     * because deassertion never required for message interrupts
+     */
+    assert(!s->msix_used || !msix_enabled(&s->dev));
+    /*
+     * This function should never be called for MSI(X) interrupts
+     * because deassertion never required for message interrupts
+     */
+    assert(!s->msi_used || !msi_enabled(&s->dev));
+
+    VMW_IRPRN("Deasserting line for interrupt %u", lidx);
+    qemu_set_irq(s->dev.irq[lidx], 0);
+}
+
+static void vmxnet3_update_interrupt_line_state(VMXNET3State *s, int lidx)
+{
+    if (!s->interrupt_states[lidx].is_pending &&
+       s->interrupt_states[lidx].is_asserted) {
+        VMW_IRPRN("New interrupt line state for index %d is DOWN", lidx);
+        _vmxnet3_deassert_interrupt_line(s, lidx);
+        s->interrupt_states[lidx].is_asserted = false;
+        return;
+    }
+
+    if (s->interrupt_states[lidx].is_pending &&
+       !s->interrupt_states[lidx].is_masked &&
+       !s->interrupt_states[lidx].is_asserted) {
+        VMW_IRPRN("New interrupt line state for index %d is UP", lidx);
+        s->interrupt_states[lidx].is_asserted =
+            _vmxnet3_assert_interrupt_line(s, lidx);
+        s->interrupt_states[lidx].is_pending = false;
+        return;
+    }
+}
+
+static void vmxnet3_trigger_interrupt(VMXNET3State *s, int lidx)
+{
+    s->interrupt_states[lidx].is_pending = true;
+    vmxnet3_update_interrupt_line_state(s, lidx);
+
+    if (s->msix_used && msix_enabled(&s->dev) && s->auto_int_masking) {
+        goto do_automask;
+    }
+
+    if (s->msi_used && msi_enabled(&s->dev) && s->auto_int_masking) {
+        goto do_automask;
+    }
+
+    return;
+
+do_automask:
+    s->interrupt_states[lidx].is_masked = true;
+    vmxnet3_update_interrupt_line_state(s, lidx);
+}
+
+static bool vmxnet3_interrupt_asserted(VMXNET3State *s, int lidx)
+{
+    return s->interrupt_states[lidx].is_asserted;
+}
+
+static void vmxnet3_clear_interrupt(VMXNET3State *s, int int_idx)
+{
+    s->interrupt_states[int_idx].is_pending = false;
+    if (s->auto_int_masking) {
+        s->interrupt_states[int_idx].is_masked = true;
+    }
+    vmxnet3_update_interrupt_line_state(s, int_idx);
+}
+
+static void
+vmxnet3_on_interrupt_mask_changed(VMXNET3State *s, int lidx, bool is_masked)
+{
+    s->interrupt_states[lidx].is_masked = is_masked;
+    vmxnet3_update_interrupt_line_state(s, lidx);
+}
+
+static bool vmxnet3_verify_driver_magic(hwaddr dshmem)
+{
+    return (VMXNET3_READ_DRV_SHARED32(dshmem, magic) == VMXNET3_REV1_MAGIC);
+}
+
+#define VMXNET3_GET_BYTE(x, byte_num) (((x) >> (byte_num)*8) & 0xFF)
+#define VMXNET3_MAKE_BYTE(byte_num, val) \
+    (((uint32_t)((val) & 0xFF)) << (byte_num)*8)
+
+static void vmxnet3_set_variable_mac(VMXNET3State *s, uint32_t h, uint32_t l)
+{
+    s->conf.macaddr.a[0] = VMXNET3_GET_BYTE(l,  0);
+    s->conf.macaddr.a[1] = VMXNET3_GET_BYTE(l,  1);
+    s->conf.macaddr.a[2] = VMXNET3_GET_BYTE(l,  2);
+    s->conf.macaddr.a[3] = VMXNET3_GET_BYTE(l,  3);
+    s->conf.macaddr.a[4] = VMXNET3_GET_BYTE(h, 0);
+    s->conf.macaddr.a[5] = VMXNET3_GET_BYTE(h, 1);
+
+    VMW_CFPRN("Variable MAC: " VMXNET_MF, VMXNET_MA(s->conf.macaddr.a));
+
+    qemu_format_nic_info_str(&s->nic->nc, s->conf.macaddr.a);
+}
+
+static uint64_t vmxnet3_get_mac_low(MACAddr *addr)
+{
+    return VMXNET3_MAKE_BYTE(0, addr->a[0]) |
+           VMXNET3_MAKE_BYTE(1, addr->a[1]) |
+           VMXNET3_MAKE_BYTE(2, addr->a[2]) |
+           VMXNET3_MAKE_BYTE(3, addr->a[3]);
+}
+
+static uint64_t vmxnet3_get_mac_high(MACAddr *addr)
+{
+    return VMXNET3_MAKE_BYTE(0, addr->a[4]) |
+           VMXNET3_MAKE_BYTE(1, addr->a[5]);
+}
+
+static void
+vmxnet3_inc_tx_consumption_counter(VMXNET3State *s, int qidx)
+{
+    vmxnet3_ring_inc(&s->txq_descr[qidx].tx_ring);
+}
+
+static inline void
+vmxnet3_inc_rx_consumption_counter(VMXNET3State *s, int qidx, int ridx)
+{
+    vmxnet3_ring_inc(&s->rxq_descr[qidx].rx_ring[ridx]);
+}
+
+static inline void
+vmxnet3_inc_tx_completion_counter(VMXNET3State *s, int qidx)
+{
+    vmxnet3_ring_inc(&s->txq_descr[qidx].comp_ring);
+}
+
+static void
+vmxnet3_inc_rx_completion_counter(VMXNET3State *s, int qidx)
+{
+    vmxnet3_ring_inc(&s->rxq_descr[qidx].comp_ring);
+}
+
+static void
+vmxnet3_dec_rx_completion_counter(VMXNET3State *s, int qidx)
+{
+    vmxnet3_ring_dec(&s->rxq_descr[qidx].comp_ring);
+}
+
+static void vmxnet3_complete_packet(VMXNET3State *s, int qidx, uint32 tx_ridx)
+{
+    struct Vmxnet3_TxCompDesc txcq_descr;
+
+    VMXNET3_RING_DUMP(VMW_RIPRN, "TXC", qidx, &s->txq_descr[qidx].comp_ring);
+
+    txcq_descr.txdIdx = tx_ridx;
+    txcq_descr.gen = vmxnet3_ring_curr_gen(&s->txq_descr[qidx].comp_ring);
+
+    vmxnet3_ring_write_curr_cell(&s->txq_descr[qidx].comp_ring, &txcq_descr);
+
+    /* Flush changes in TX descriptor before changing the counter value */
+    smp_wmb();
+
+    vmxnet3_inc_tx_completion_counter(s, qidx);
+    vmxnet3_trigger_interrupt(s, s->txq_descr[qidx].intr_idx);
+}
+
+static bool
+vmxnet3_setup_tx_offloads(VMXNET3State *s)
+{
+    switch (s->offload_mode) {
+    case VMXNET3_OM_NONE:
+        vmxnet_tx_pkt_build_vheader(s->tx_pkt, false, false, 0);
+        break;
+
+    case VMXNET3_OM_CSUM:
+        vmxnet_tx_pkt_build_vheader(s->tx_pkt, false, true, 0);
+        VMW_PKPRN("L4 CSO requested\n");
+        break;
+
+    case VMXNET3_OM_TSO:
+        vmxnet_tx_pkt_build_vheader(s->tx_pkt, true, true,
+            s->cso_or_gso_size);
+        vmxnet_tx_pkt_update_ip_checksums(s->tx_pkt);
+        VMW_PKPRN("GSO offload requested.");
+        break;
+
+    default:
+        assert(false);
+        return false;
+    }
+
+    return true;
+}
+
+static void
+vmxnet3_tx_retrieve_metadata(VMXNET3State *s,
+                             const struct Vmxnet3_TxDesc *txd)
+{
+    s->offload_mode = txd->om;
+    s->cso_or_gso_size = txd->msscof;
+    s->tci = txd->tci;
+    s->needs_vlan = txd->ti;
+}
+
+typedef enum {
+    VMXNET3_PKT_STATUS_OK,
+    VMXNET3_PKT_STATUS_ERROR,
+    VMXNET3_PKT_STATUS_DISCARD,/* only for tx */
+    VMXNET3_PKT_STATUS_OUT_OF_BUF /* only for rx */
+} Vmxnet3PktStatus;
+
+static void
+vmxnet3_on_tx_done_update_stats(VMXNET3State *s, int qidx,
+    Vmxnet3PktStatus status)
+{
+    size_t tot_len = vmxnet_tx_pkt_get_total_len(s->tx_pkt);
+    struct UPT1_TxStats *stats = &s->txq_descr[qidx].txq_stats;
+
+    switch (status) {
+    case VMXNET3_PKT_STATUS_OK:
+        switch (vmxnet_tx_pkt_get_packet_type(s->tx_pkt)) {
+        case ETH_PKT_BCAST:
+            stats->bcastPktsTxOK++;
+            stats->bcastBytesTxOK += tot_len;
+            break;
+        case ETH_PKT_MCAST:
+            stats->mcastPktsTxOK++;
+            stats->mcastBytesTxOK += tot_len;
+            break;
+        case ETH_PKT_UCAST:
+            stats->ucastPktsTxOK++;
+            stats->ucastBytesTxOK += tot_len;
+            break;
+        default:
+            assert(false);
+        }
+
+        if (s->offload_mode == VMXNET3_OM_TSO) {
+            /*
+             * According to VMWARE headers this statistic is a number
+             * of packets after segmentation but since we don't have
+             * this information in QEMU model, the best we can do is to
+             * provide number of non-segmented packets
+             */
+            stats->TSOPktsTxOK++;
+            stats->TSOBytesTxOK += tot_len;
+        }
+        break;
+
+    case VMXNET3_PKT_STATUS_DISCARD:
+        stats->pktsTxDiscard++;
+        break;
+
+    case VMXNET3_PKT_STATUS_ERROR:
+        stats->pktsTxError++;
+        break;
+
+    default:
+        assert(false);
+    }
+}
+
+static void
+vmxnet3_on_rx_done_update_stats(VMXNET3State *s,
+                                int qidx,
+                                Vmxnet3PktStatus status)
+{
+    struct UPT1_RxStats *stats = &s->rxq_descr[qidx].rxq_stats;
+    size_t tot_len = vmxnet_rx_pkt_get_total_len(s->rx_pkt);
+
+    switch (status) {
+    case VMXNET3_PKT_STATUS_OUT_OF_BUF:
+        stats->pktsRxOutOfBuf++;
+        break;
+
+    case VMXNET3_PKT_STATUS_ERROR:
+        stats->pktsRxError++;
+        break;
+    case VMXNET3_PKT_STATUS_OK:
+        switch (vmxnet_rx_pkt_get_packet_type(s->rx_pkt)) {
+        case ETH_PKT_BCAST:
+            stats->bcastPktsRxOK++;
+            stats->bcastBytesRxOK += tot_len;
+            break;
+        case ETH_PKT_MCAST:
+            stats->mcastPktsRxOK++;
+            stats->mcastBytesRxOK += tot_len;
+            break;
+        case ETH_PKT_UCAST:
+            stats->ucastPktsRxOK++;
+            stats->ucastBytesRxOK += tot_len;
+            break;
+        default:
+            assert(false);
+        }
+
+        if (tot_len > s->mtu) {
+            stats->LROPktsRxOK++;
+            stats->LROBytesRxOK += tot_len;
+        }
+        break;
+    default:
+        assert(false);
+    }
+}
+
+static inline bool
+vmxnet3_pop_next_tx_descr(VMXNET3State *s,
+                          int qidx,
+                          struct Vmxnet3_TxDesc *txd,
+                          uint32_t *descr_idx)
+{
+    Vmxnet3Ring *ring = &s->txq_descr[qidx].tx_ring;
+
+    vmxnet3_ring_read_curr_cell(ring, txd);
+    if (txd->gen == vmxnet3_ring_curr_gen(ring)) {
+        VMXNET3_RING_DUMP(VMW_RIPRN, "TX", qidx, ring);
+        *descr_idx = vmxnet3_ring_curr_cell_idx(ring);
+        vmxnet3_inc_tx_consumption_counter(s, qidx);
+        return true;
+    }
+
+    return false;
+}
+
+static bool
+vmxnet3_send_packet(VMXNET3State *s, uint32_t qidx)
+{
+    size_t bytes_sent;
+    bool res;
+    Vmxnet3PktStatus status = VMXNET3_PKT_STATUS_OK;
+
+    res = vmxnet3_setup_tx_offloads(s);
+    if (!res) {
+        status = VMXNET3_PKT_STATUS_ERROR;
+        goto func_exit;
+    }
+
+    /* debug prints */
+    vmxnet3_dump_virt_hdr(vmxnet_tx_pkt_get_vhdr(s->tx_pkt));
+    vmxnet_tx_pkt_dump(s->tx_pkt);
+
+    bytes_sent = vmxnet_tx_pkt_send(s->tx_pkt, &s->nic->nc);
+    if (!bytes_sent) {
+        res = false;
+        status = VMXNET3_PKT_STATUS_DISCARD;
+        goto func_exit;
+    }
+
+func_exit:
+    vmxnet3_on_tx_done_update_stats(s, qidx, status);
+    return res;
+}
+
+static void vmxnet3_process_tx_queue(VMXNET3State *s, int qidx)
+{
+    struct Vmxnet3_TxDesc txd;
+    uint32_t txd_idx;
+    uint32_t data_len;
+    hwaddr data_pa;
+
+    for (;;) {
+        if (!vmxnet3_pop_next_tx_descr(s, qidx, &txd, &txd_idx)) {
+            break;
+        }
+
+        vmxnet3_dump_tx_descr(&txd);
+
+        if (!s->skip_current_tx_pkt) {
+            data_len = (txd.len > 0) ? txd.len : VMXNET3_MAX_TX_BUF_SIZE;
+            data_pa = le64_to_cpu(txd.addr);
+
+            if (!vmxnet_tx_pkt_add_raw_fragment(s->tx_pkt,
+                                                data_pa,
+                                                data_len)) {
+                s->skip_current_tx_pkt = true;
+            }
+        }
+
+        if (s->tx_sop) {
+            vmxnet3_tx_retrieve_metadata(s, &txd);
+            s->tx_sop = false;
+        }
+
+        if (txd.eop) {
+            if (!s->skip_current_tx_pkt) {
+                vmxnet_tx_pkt_parse(s->tx_pkt);
+
+                if (s->needs_vlan) {
+                    vmxnet_tx_pkt_setup_vlan_header(s->tx_pkt, s->tci);
+                }
+
+                vmxnet3_send_packet(s, qidx);
+            } else {
+                vmxnet3_on_tx_done_update_stats(s, qidx,
+                                                VMXNET3_PKT_STATUS_ERROR);
+            }
+
+            vmxnet3_complete_packet(s, qidx, txd_idx);
+            s->tx_sop = true;
+            s->skip_current_tx_pkt = false;
+            vmxnet_tx_pkt_reset(s->tx_pkt);
+        }
+    }
+}
+
+static inline void
+vmxnet3_read_next_rx_descr(VMXNET3State *s, int qidx, int ridx,
+                           struct Vmxnet3_RxDesc *dbuf, uint32_t *didx)
+{
+    Vmxnet3Ring *ring = &s->rxq_descr[qidx].rx_ring[ridx];
+    *didx = vmxnet3_ring_curr_cell_idx(ring);
+    vmxnet3_ring_read_curr_cell(ring, dbuf);
+}
+
+static inline uint8_t
+vmxnet3_get_rx_ring_gen(VMXNET3State *s, int qidx, int ridx)
+{
+    return s->rxq_descr[qidx].rx_ring[ridx].gen;
+}
+
+static inline hwaddr
+vmxnet3_pop_rxc_descr(VMXNET3State *s, int qidx, uint32_t *descr_gen)
+{
+    uint8_t ring_gen;
+    struct Vmxnet3_RxCompDesc rxcd;
+
+    hwaddr daddr =
+        vmxnet3_ring_curr_cell_pa(&s->rxq_descr[qidx].comp_ring);
+
+    cpu_physical_memory_read(daddr, &rxcd, sizeof(struct Vmxnet3_RxCompDesc));
+    ring_gen = vmxnet3_ring_curr_gen(&s->rxq_descr[qidx].comp_ring);
+
+    if (rxcd.gen != ring_gen) {
+        *descr_gen = ring_gen;
+        vmxnet3_inc_rx_completion_counter(s, qidx);
+        return daddr;
+    }
+
+    return 0;
+}
+
+static inline void
+vmxnet3_revert_rxc_descr(VMXNET3State *s, int qidx)
+{
+    vmxnet3_dec_rx_completion_counter(s, qidx);
+}
+
+#define RXQ_IDX      (0)
+#define RX_HEAD_BODY_RING (0)
+#define RX_BODY_ONLY_RING (1)
+
+static bool
+vmxnet3_get_next_head_rx_descr(VMXNET3State *s,
+                               struct Vmxnet3_RxDesc *descr_buf,
+                               uint32_t *descr_idx,
+                               uint32_t *ridx)
+{
+    for (;;) {
+        uint32_t ring_gen;
+        vmxnet3_read_next_rx_descr(s, RXQ_IDX, RX_HEAD_BODY_RING,
+                                   descr_buf, descr_idx);
+
+        /* If no more free descriptors - return */
+        ring_gen = vmxnet3_get_rx_ring_gen(s, RXQ_IDX, RX_HEAD_BODY_RING);
+        if (descr_buf->gen != ring_gen) {
+            return false;
+        }
+
+        /* Mark current descriptor as used/skipped */
+        vmxnet3_inc_rx_consumption_counter(s, RXQ_IDX, RX_HEAD_BODY_RING);
+
+        /* If this is what we are looking for - return */
+        if (descr_buf->btype == VMXNET3_RXD_BTYPE_HEAD) {
+            *ridx = RX_HEAD_BODY_RING;
+            return true;
+        }
+    }
+}
+
+static bool
+vmxnet3_get_next_body_rx_descr(VMXNET3State *s,
+                               struct Vmxnet3_RxDesc *d,
+                               uint32_t *didx,
+                               uint32_t *ridx)
+{
+    vmxnet3_read_next_rx_descr(s, RXQ_IDX, RX_HEAD_BODY_RING, d, didx);
+
+    /* Try to find corresponding descriptor in head/body ring */
+    if ((d->gen == vmxnet3_get_rx_ring_gen(s, RXQ_IDX, RX_HEAD_BODY_RING)) &&
+         (d->btype == VMXNET3_RXD_BTYPE_BODY)) {
+        vmxnet3_inc_rx_consumption_counter(s, RXQ_IDX, RX_HEAD_BODY_RING);
+        *ridx = RX_HEAD_BODY_RING;
+        return true;
+    }
+
+    /*
+     * If there is no free descriptors on head/body ring or next free
+     * descriptor is a head descriptor switch to body only ring
+     */
+    vmxnet3_read_next_rx_descr(s, RXQ_IDX, RX_BODY_ONLY_RING, d, didx);
+
+    /* If no more free descriptors - return */
+    if (d->gen == vmxnet3_get_rx_ring_gen(s, RXQ_IDX, RX_BODY_ONLY_RING)) {
+        assert(d->btype == VMXNET3_RXD_BTYPE_BODY);
+        *ridx = RX_BODY_ONLY_RING;
+        vmxnet3_inc_rx_consumption_counter(s, RXQ_IDX, RX_BODY_ONLY_RING);
+        return true;
+    }
+
+    return false;
+}
+
+static inline bool
+vmxnet3_get_next_rx_descr(VMXNET3State *s, bool is_head,
+                          struct Vmxnet3_RxDesc *descr_buf,
+                          uint32_t *descr_idx,
+                          uint32_t *ridx)
+{
+    if (is_head || !s->rx_packets_compound) {
+        return vmxnet3_get_next_head_rx_descr(s, descr_buf, descr_idx, ridx);
+    } else {
+        return vmxnet3_get_next_body_rx_descr(s, descr_buf, descr_idx, ridx);
+    }
+}
+
+static void vmxnet3_rx_update_descr(struct VmxnetRxPkt *pkt,
+    struct Vmxnet3_RxCompDesc *rxcd)
+{
+    int csum_ok, is_gso;
+    bool isip4, isip6, istcp, isudp;
+    struct virtio_net_hdr *vhdr;
+    uint8_t offload_type;
+
+    if (vmxnet_rx_pkt_is_vlan_stripped(pkt)) {
+        rxcd->ts = 1;
+        rxcd->tci = vmxnet_rx_pkt_get_vlan_tag(pkt);
+    }
+
+    if (!vmxnet_rx_pkt_has_virt_hdr(pkt)) {
+        goto nocsum;
+    }
+
+    vhdr = vmxnet_rx_pkt_get_vhdr(pkt);
+    /*
+     * Checksum is valid when lower level tell so or when lower level
+     * requires checksum offload telling that packet produced/bridged
+     * locally and did travel over network after last checksum calculation
+     * or production
+     */
+    csum_ok = VMXNET_FLAG_IS_SET(vhdr->flags, VIRTIO_NET_HDR_F_DATA_VALID) ||
+              VMXNET_FLAG_IS_SET(vhdr->flags, VIRTIO_NET_HDR_F_NEEDS_CSUM);
+
+    offload_type = vhdr->gso_type & ~VIRTIO_NET_HDR_GSO_ECN;
+    is_gso = (offload_type != VIRTIO_NET_HDR_GSO_NONE) ? 1 : 0;
+
+    if (!csum_ok && !is_gso) {
+        goto nocsum;
+    }
+
+    vmxnet_rx_pkt_get_protocols(pkt, &isip4, &isip6, &isudp, &istcp);
+    if ((!istcp && !isudp) || (!isip4 && !isip6)) {
+        goto nocsum;
+    }
+
+    rxcd->cnc = 0;
+    rxcd->v4 = isip4 ? 1 : 0;
+    rxcd->v6 = isip6 ? 1 : 0;
+    rxcd->tcp = istcp ? 1 : 0;
+    rxcd->udp = isudp ? 1 : 0;
+    rxcd->fcs = rxcd->tuc = rxcd->ipc = 1;
+    return;
+
+nocsum:
+    rxcd->cnc = 1;
+    return;
+}
+
+static void
+vmxnet3_physical_memory_writev(const struct iovec *iov,
+                               size_t start_iov_off,
+                               hwaddr target_addr,
+                               size_t bytes_to_copy)
+{
+    size_t curr_off = 0;
+    size_t copied = 0;
+
+    while (bytes_to_copy) {
+        if (start_iov_off < (curr_off + iov->iov_len)) {
+            size_t chunk_len =
+                MIN((curr_off + iov->iov_len) - start_iov_off, bytes_to_copy);
+
+            cpu_physical_memory_write(target_addr + copied,
+                                      iov->iov_base + start_iov_off - curr_off,
+                                      chunk_len);
+
+            copied += chunk_len;
+            start_iov_off += chunk_len;
+            curr_off = start_iov_off;
+            bytes_to_copy -= chunk_len;
+        } else {
+            curr_off += iov->iov_len;
+        }
+        iov++;
+    }
+}
+
+static bool
+vmxnet3_indicate_packet(VMXNET3State *s)
+{
+    struct Vmxnet3_RxDesc rxd;
+    bool is_head = true;
+    uint32_t rxd_idx;
+    uint32_t rx_ridx;
+
+    struct Vmxnet3_RxCompDesc rxcd;
+    uint32_t new_rxcd_gen = VMXNET3_INIT_GEN;
+    hwaddr new_rxcd_pa = 0;
+    hwaddr ready_rxcd_pa = 0;
+    struct iovec *data = vmxnet_rx_pkt_get_iovec(s->rx_pkt);
+    size_t bytes_copied = 0;
+    size_t bytes_left = vmxnet_rx_pkt_get_total_len(s->rx_pkt);
+    uint16_t num_frags = 0;
+    size_t chunk_size;
+
+    vmxnet_rx_pkt_dump(s->rx_pkt);
+
+    while (bytes_left > 0) {
+
+        /* cannot add more frags to packet */
+        if (num_frags == s->max_rx_frags) {
+            break;
+        }
+
+        new_rxcd_pa = vmxnet3_pop_rxc_descr(s, RXQ_IDX, &new_rxcd_gen);
+        if (!new_rxcd_pa) {
+            break;
+        }
+
+        if (!vmxnet3_get_next_rx_descr(s, is_head, &rxd, &rxd_idx, &rx_ridx)) {
+            break;
+        }
+
+        chunk_size = MIN(bytes_left, rxd.len);
+        vmxnet3_physical_memory_writev(data, bytes_copied,
+                                       le64_to_cpu(rxd.addr), chunk_size);
+        bytes_copied += chunk_size;
+        bytes_left -= chunk_size;
+
+        vmxnet3_dump_rx_descr(&rxd);
+
+        if (0 != ready_rxcd_pa) {
+            cpu_physical_memory_write(ready_rxcd_pa, &rxcd, sizeof(rxcd));
+        }
+
+        memset(&rxcd, 0, sizeof(struct Vmxnet3_RxCompDesc));
+        rxcd.rxdIdx = rxd_idx;
+        rxcd.len = chunk_size;
+        rxcd.sop = is_head;
+        rxcd.gen = new_rxcd_gen;
+        rxcd.rqID = RXQ_IDX + rx_ridx * s->rxq_num;
+
+        if (0 == bytes_left) {
+            vmxnet3_rx_update_descr(s->rx_pkt, &rxcd);
+        }
+
+        VMW_RIPRN("RX Completion descriptor: rxRing: %lu rxIdx %lu len %lu "
+                  "sop %d csum_correct %lu",
+                  (unsigned long) rx_ridx,
+                  (unsigned long) rxcd.rxdIdx,
+                  (unsigned long) rxcd.len,
+                  (int) rxcd.sop,
+                  (unsigned long) rxcd.tuc);
+
+        is_head = false;
+        ready_rxcd_pa = new_rxcd_pa;
+        new_rxcd_pa = 0;
+    }
+
+    if (0 != ready_rxcd_pa) {
+        rxcd.eop = 1;
+        rxcd.err = (0 != bytes_left);
+        cpu_physical_memory_write(ready_rxcd_pa, &rxcd, sizeof(rxcd));
+
+        /* Flush RX descriptor changes */
+        smp_wmb();
+    }
+
+    if (0 != new_rxcd_pa) {
+        vmxnet3_revert_rxc_descr(s, RXQ_IDX);
+    }
+
+    vmxnet3_trigger_interrupt(s, s->rxq_descr[RXQ_IDX].intr_idx);
+
+    if (bytes_left == 0) {
+        vmxnet3_on_rx_done_update_stats(s, RXQ_IDX, VMXNET3_PKT_STATUS_OK);
+        return true;
+    } else if (num_frags == s->max_rx_frags) {
+        vmxnet3_on_rx_done_update_stats(s, RXQ_IDX, VMXNET3_PKT_STATUS_ERROR);
+        return false;
+    } else {
+        vmxnet3_on_rx_done_update_stats(s, RXQ_IDX,
+                                        VMXNET3_PKT_STATUS_OUT_OF_BUF);
+        return false;
+    }
+}
+
+static void
+vmxnet3_io_bar0_write(void *opaque, hwaddr addr,
+                      uint64_t val, unsigned size)
+{
+    VMXNET3State *s = opaque;
+
+    if (VMW_IS_MULTIREG_ADDR(addr, VMXNET3_REG_TXPROD,
+                        VMXNET3_DEVICE_MAX_TX_QUEUES, VMXNET3_REG_ALIGN)) {
+        int tx_queue_idx =
+            VMW_MULTIREG_IDX_BY_ADDR(addr, VMXNET3_REG_TXPROD,
+                                     VMXNET3_REG_ALIGN);
+        assert(tx_queue_idx <= s->txq_num);
+        vmxnet3_process_tx_queue(s, tx_queue_idx);
+        return;
+    }
+
+    if (VMW_IS_MULTIREG_ADDR(addr, VMXNET3_REG_IMR,
+                        VMXNET3_MAX_INTRS, VMXNET3_REG_ALIGN)) {
+        int l = VMW_MULTIREG_IDX_BY_ADDR(addr, VMXNET3_REG_IMR,
+                                         VMXNET3_REG_ALIGN);
+
+        VMW_CBPRN("Interrupt mask for line %d written: 0x%" PRIx64, l, val);
+
+        vmxnet3_on_interrupt_mask_changed(s, l, val);
+        return;
+    }
+
+    if (VMW_IS_MULTIREG_ADDR(addr, VMXNET3_REG_RXPROD,
+                        VMXNET3_DEVICE_MAX_RX_QUEUES, VMXNET3_REG_ALIGN) ||
+       VMW_IS_MULTIREG_ADDR(addr, VMXNET3_REG_RXPROD2,
+                        VMXNET3_DEVICE_MAX_RX_QUEUES, VMXNET3_REG_ALIGN)) {
+        return;
+    }
+
+    VMW_WRPRN("BAR0 unknown write [%" PRIx64 "] = %" PRIx64 ", size %d",
+              (uint64_t) addr, val, size);
+}
+
+static uint64_t
+vmxnet3_io_bar0_read(void *opaque, hwaddr addr, unsigned size)
+{
+    if (VMW_IS_MULTIREG_ADDR(addr, VMXNET3_REG_IMR,
+                        VMXNET3_MAX_INTRS, VMXNET3_REG_ALIGN)) {
+        assert(false);
+    }
+
+    VMW_CBPRN("BAR0 unknown read [%" PRIx64 "], size %d", addr, size);
+    return 0;
+}
+
+static void vmxnet3_reset(VMXNET3State *s)
+{
+    VMW_CBPRN("Resetting vmxnet3...");
+
+    vmxnet_tx_pkt_reset(s->tx_pkt);
+    s->tx_sop = true;
+    s->skip_current_tx_pkt = false;
+}
+
+static void vmxnet3_deactivate_device(VMXNET3State *s)
+{
+    VMW_CBPRN("Deactivating vmxnet3...");
+    s->device_active = false;
+}
+
+static void vmxnet3_update_rx_mode(VMXNET3State *s)
+{
+    s->rx_mode = VMXNET3_READ_DRV_SHARED32(s->drv_shmem,
+                                           devRead.rxFilterConf.rxMode);
+    VMW_CFPRN("RX mode: 0x%08X", s->rx_mode);
+}
+
+static void vmxnet3_update_vlan_filters(VMXNET3State *s)
+{
+    int i;
+
+    /* Copy configuration from shared memory */
+    VMXNET3_READ_DRV_SHARED(s->drv_shmem,
+                            devRead.rxFilterConf.vfTable,
+                            s->vlan_table,
+                            sizeof(s->vlan_table));
+
+    /* Invert byte order when needed */
+    for (i = 0; i < ARRAY_SIZE(s->vlan_table); i++) {
+        s->vlan_table[i] = le32_to_cpu(s->vlan_table[i]);
+    }
+
+    /* Dump configuration for debugging purposes */
+    VMW_CFPRN("Configured VLANs:");
+    for (i = 0; i < sizeof(s->vlan_table) * 8; i++) {
+        if (VMXNET3_VFTABLE_ENTRY_IS_SET(s->vlan_table, i)) {
+            VMW_CFPRN("\tVLAN %d is present", i);
+        }
+    }
+}
+
+static void vmxnet3_update_mcast_filters(VMXNET3State *s)
+{
+    uint16_t list_bytes =
+        VMXNET3_READ_DRV_SHARED16(s->drv_shmem,
+                                  devRead.rxFilterConf.mfTableLen);
+
+    s->mcast_list_len = list_bytes / sizeof(s->mcast_list[0]);
+
+    s->mcast_list = g_realloc(s->mcast_list, list_bytes);
+    if (NULL == s->mcast_list) {
+        if (0 == s->mcast_list_len) {
+            VMW_CFPRN("Current multicast list is empty");
+        } else {
+            VMW_ERPRN("Failed to allocate multicast list of %d elements",
+                      s->mcast_list_len);
+        }
+        s->mcast_list_len = 0;
+    } else {
+        int i;
+        hwaddr mcast_list_pa =
+            VMXNET3_READ_DRV_SHARED64(s->drv_shmem,
+                                      devRead.rxFilterConf.mfTablePA);
+
+        cpu_physical_memory_read(mcast_list_pa, s->mcast_list, list_bytes);
+        VMW_CFPRN("Current multicast list len is %d:", s->mcast_list_len);
+        for (i = 0; i < s->mcast_list_len; i++) {
+            VMW_CFPRN("\t" VMXNET_MF, VMXNET_MA(s->mcast_list[i].a));
+        }
+    }
+}
+
+static void vmxnet3_setup_rx_filtering(VMXNET3State *s)
+{
+    vmxnet3_update_rx_mode(s);
+    vmxnet3_update_vlan_filters(s);
+    vmxnet3_update_mcast_filters(s);
+}
+
+static uint32_t vmxnet3_get_interrupt_config(VMXNET3State *s)
+{
+    uint32_t interrupt_mode = VMXNET3_IT_AUTO | (VMXNET3_IMM_AUTO << 2);
+    VMW_CFPRN("Interrupt config is 0x%X", interrupt_mode);
+    return interrupt_mode;
+}
+
+static void vmxnet3_fill_stats(VMXNET3State *s)
+{
+    int i;
+    for (i = 0; i < s->txq_num; i++) {
+        cpu_physical_memory_write(s->txq_descr[i].tx_stats_pa,
+                                  &s->txq_descr[i].txq_stats,
+                                  sizeof(s->txq_descr[i].txq_stats));
+    }
+
+    for (i = 0; i < s->rxq_num; i++) {
+        cpu_physical_memory_write(s->rxq_descr[i].rx_stats_pa,
+                                  &s->rxq_descr[i].rxq_stats,
+                                  sizeof(s->rxq_descr[i].rxq_stats));
+    }
+}
+
+static void vmxnet3_adjust_by_guest_type(VMXNET3State *s)
+{
+    struct Vmxnet3_GOSInfo gos;
+
+    VMXNET3_READ_DRV_SHARED(s->drv_shmem, devRead.misc.driverInfo.gos,
+                            &gos, sizeof(gos));
+    s->rx_packets_compound =
+        (gos.gosType == VMXNET3_GOS_TYPE_WIN) ? false : true;
+
+    VMW_CFPRN("Guest type specifics: RXCOMPOUND: %d", s->rx_packets_compound);
+}
+
+static void
+vmxnet3_dump_conf_descr(const char *name,
+                        struct Vmxnet3_VariableLenConfDesc *pm_descr)
+{
+    VMW_CFPRN("%s descriptor dump: Version %u, Length %u",
+              name, pm_descr->confVer, pm_descr->confLen);
+
+};
+
+static void vmxnet3_update_pm_state(VMXNET3State *s)
+{
+    struct Vmxnet3_VariableLenConfDesc pm_descr;
+
+    pm_descr.confLen =
+        VMXNET3_READ_DRV_SHARED32(s->drv_shmem, devRead.pmConfDesc.confLen);
+    pm_descr.confVer =
+        VMXNET3_READ_DRV_SHARED32(s->drv_shmem, devRead.pmConfDesc.confVer);
+    pm_descr.confPA =
+        VMXNET3_READ_DRV_SHARED64(s->drv_shmem, devRead.pmConfDesc.confPA);
+
+    vmxnet3_dump_conf_descr("PM State", &pm_descr);
+}
+
+static void vmxnet3_update_features(VMXNET3State *s)
+{
+    uint32_t guest_features;
+    int rxcso_supported;
+
+    guest_features = VMXNET3_READ_DRV_SHARED32(s->drv_shmem,
+                                               devRead.misc.uptFeatures);
+
+    rxcso_supported = VMXNET_FLAG_IS_SET(guest_features, UPT1_F_RXCSUM);
+    s->rx_vlan_stripping = VMXNET_FLAG_IS_SET(guest_features, UPT1_F_RXVLAN);
+    s->lro_supported = VMXNET_FLAG_IS_SET(guest_features, UPT1_F_LRO);
+
+    VMW_CFPRN("Features configuration: LRO: %d, RXCSUM: %d, VLANSTRIP: %d",
+              s->lro_supported, rxcso_supported,
+              s->rx_vlan_stripping);
+    if (s->peer_has_vhdr) {
+        tap_set_offload(s->nic->nc.peer,
+                        rxcso_supported,
+                        s->lro_supported,
+                        s->lro_supported,
+                        0,
+                        0);
+    }
+}
+
+static void vmxnet3_activate_device(VMXNET3State *s)
+{
+    int i;
+    static const uint32_t VMXNET3_DEF_TX_THRESHOLD = 1;
+    hwaddr qdescr_table_pa;
+    uint64_t pa;
+    uint32_t size;
+
+    /* Verify configuration consistency */
+    if (!vmxnet3_verify_driver_magic(s->drv_shmem)) {
+        VMW_ERPRN("Device configuration received from driver is invalid");
+        return;
+    }
+
+    vmxnet3_adjust_by_guest_type(s);
+    vmxnet3_update_features(s);
+    vmxnet3_update_pm_state(s);
+    vmxnet3_setup_rx_filtering(s);
+    /* Cache fields from shared memory */
+    s->mtu = VMXNET3_READ_DRV_SHARED32(s->drv_shmem, devRead.misc.mtu);
+    VMW_CFPRN("MTU is %u", s->mtu);
+
+    s->max_rx_frags =
+        VMXNET3_READ_DRV_SHARED16(s->drv_shmem, devRead.misc.maxNumRxSG);
+
+    VMW_CFPRN("Max RX fragments is %u", s->max_rx_frags);
+
+    s->event_int_idx =
+        VMXNET3_READ_DRV_SHARED8(s->drv_shmem, devRead.intrConf.eventIntrIdx);
+    VMW_CFPRN("Events interrupt line is %u", s->event_int_idx);
+
+    s->auto_int_masking =
+        VMXNET3_READ_DRV_SHARED8(s->drv_shmem, devRead.intrConf.autoMask);
+    VMW_CFPRN("Automatic interrupt masking is %d", (int)s->auto_int_masking);
+
+    s->txq_num =
+        VMXNET3_READ_DRV_SHARED8(s->drv_shmem, devRead.misc.numTxQueues);
+    s->rxq_num =
+        VMXNET3_READ_DRV_SHARED8(s->drv_shmem, devRead.misc.numRxQueues);
+
+    VMW_CFPRN("Number of TX/RX queues %u/%u", s->txq_num, s->rxq_num);
+    assert(s->txq_num <= VMXNET3_DEVICE_MAX_TX_QUEUES);
+
+    qdescr_table_pa =
+        VMXNET3_READ_DRV_SHARED64(s->drv_shmem, devRead.misc.queueDescPA);
+    VMW_CFPRN("TX queues descriptors table is at 0x%" PRIx64, qdescr_table_pa);
+
+    /*
+     * Worst-case scenario is a packet that holds all TX rings space so
+     * we calculate total size of all TX rings for max TX fragments number
+     */
+    s->max_tx_frags = 0;
+
+    /* TX queues */
+    for (i = 0; i < s->txq_num; i++) {
+        hwaddr qdescr_pa =
+            qdescr_table_pa + i * sizeof(struct Vmxnet3_TxQueueDesc);
+
+        /* Read interrupt number for this TX queue */
+        s->txq_descr[i].intr_idx =
+            VMXNET3_READ_TX_QUEUE_DESCR8(qdescr_pa, conf.intrIdx);
+
+        VMW_CFPRN("TX Queue %d interrupt: %d", i, s->txq_descr[i].intr_idx);
+
+        /* Read rings memory locations for TX queues */
+        pa = VMXNET3_READ_TX_QUEUE_DESCR64(qdescr_pa, conf.txRingBasePA);
+        size = VMXNET3_READ_TX_QUEUE_DESCR32(qdescr_pa, conf.txRingSize);
+
+        vmxnet3_ring_init(&s->txq_descr[i].tx_ring, pa, size,
+                          sizeof(struct Vmxnet3_TxDesc), false);
+        VMXNET3_RING_DUMP(VMW_CFPRN, "TX", i, &s->txq_descr[i].tx_ring);
+
+        s->max_tx_frags += size;
+
+        /* TXC ring */
+        pa = VMXNET3_READ_TX_QUEUE_DESCR64(qdescr_pa, conf.compRingBasePA);
+        size = VMXNET3_READ_TX_QUEUE_DESCR32(qdescr_pa, conf.compRingSize);
+        vmxnet3_ring_init(&s->txq_descr[i].comp_ring, pa, size,
+                          sizeof(struct Vmxnet3_TxCompDesc), true);
+        VMXNET3_RING_DUMP(VMW_CFPRN, "TXC", i, &s->txq_descr[i].comp_ring);
+
+        s->txq_descr[i].tx_stats_pa =
+            qdescr_pa + offsetof(struct Vmxnet3_TxQueueDesc, stats);
+
+        memset(&s->txq_descr[i].txq_stats, 0,
+               sizeof(s->txq_descr[i].txq_stats));
+
+        /* Fill device-managed parameters for queues */
+        VMXNET3_WRITE_TX_QUEUE_DESCR32(qdescr_pa,
+                                       ctrl.txThreshold,
+                                       VMXNET3_DEF_TX_THRESHOLD);
+    }
+
+    /* Preallocate TX packet wrapper */
+    VMW_CFPRN("Max TX fragments is %u", s->max_tx_frags);
+    vmxnet_tx_pkt_init(&s->tx_pkt, s->max_tx_frags, s->peer_has_vhdr);
+    vmxnet_rx_pkt_init(&s->rx_pkt, s->peer_has_vhdr);
+
+    /* Read rings memory locations for RX queues */
+    for (i = 0; i < s->rxq_num; i++) {
+        int j;
+        hwaddr qd_pa =
+            qdescr_table_pa + s->txq_num * sizeof(struct Vmxnet3_TxQueueDesc) +
+            i * sizeof(struct Vmxnet3_RxQueueDesc);
+
+        /* Read interrupt number for this RX queue */
+        s->rxq_descr[i].intr_idx =
+            VMXNET3_READ_TX_QUEUE_DESCR8(qd_pa, conf.intrIdx);
+
+        VMW_CFPRN("RX Queue %d interrupt: %d", i, s->rxq_descr[i].intr_idx);
+
+        /* Read rings memory locations */
+        for (j = 0; j < VMXNET3_RX_RINGS_PER_QUEUE; j++) {
+            /* RX rings */
+            pa = VMXNET3_READ_RX_QUEUE_DESCR64(qd_pa, conf.rxRingBasePA[j]);
+            size = VMXNET3_READ_RX_QUEUE_DESCR32(qd_pa, conf.rxRingSize[j]);
+            vmxnet3_ring_init(&s->rxq_descr[i].rx_ring[j], pa, size,
+                              sizeof(struct Vmxnet3_RxDesc), false);
+            VMW_CFPRN("RX queue %d:%d: Base: %" PRIx64 ", Size: %d",
+                      i, j, pa, size);
+        }
+
+        /* RXC ring */
+        pa = VMXNET3_READ_RX_QUEUE_DESCR64(qd_pa, conf.compRingBasePA);
+        size = VMXNET3_READ_RX_QUEUE_DESCR32(qd_pa, conf.compRingSize);
+        vmxnet3_ring_init(&s->rxq_descr[i].comp_ring, pa, size,
+                          sizeof(struct Vmxnet3_RxCompDesc), true);
+        VMW_CFPRN("RXC queue %d: Base: %" PRIx64 ", Size: %d", i, pa, size);
+
+        s->rxq_descr[i].rx_stats_pa =
+            qd_pa + offsetof(struct Vmxnet3_RxQueueDesc, stats);
+        memset(&s->rxq_descr[i].rxq_stats, 0,
+               sizeof(s->rxq_descr[i].rxq_stats));
+    }
+
+    /* Make sure everything is in place before device activation */
+    smp_wmb();
+
+    /* Revert MAC address to the permanent one on device reactivation */
+    memcpy(&s->conf.macaddr.a, &s->perm_mac.a, sizeof(s->perm_mac.a));
+    VMW_CFPRN("MAC address set to: " VMXNET_MF, VMXNET_MA(s->conf.macaddr.a));
+
+    s->device_active = true;
+}
+
+static void vmxnet3_handle_command(VMXNET3State *s, uint64_t cmd)
+{
+    s->last_command = cmd;
+
+    switch (cmd) {
+    case VMXNET3_CMD_GET_PERM_MAC_HI:
+        VMW_CBPRN("Set: Get upper part of permanent MAC");
+        break;
+
+    case VMXNET3_CMD_GET_PERM_MAC_LO:
+        VMW_CBPRN("Set: Get lower part of permanent MAC");
+        break;
+
+    case VMXNET3_CMD_GET_STATS:
+        VMW_CBPRN("Set: Get device statistics");
+        vmxnet3_fill_stats(s);
+        break;
+
+    case VMXNET3_CMD_ACTIVATE_DEV:
+        VMW_CBPRN("Set: Activating vmxnet3 device");
+        vmxnet3_activate_device(s);
+        break;
+
+    case VMXNET3_CMD_UPDATE_RX_MODE:
+        VMW_CBPRN("Set: Update rx mode");
+        vmxnet3_update_rx_mode(s);
+        break;
+
+    case VMXNET3_CMD_UPDATE_VLAN_FILTERS:
+        VMW_CBPRN("Set: Update VLAN filters");
+        vmxnet3_update_vlan_filters(s);
+        break;
+
+    case VMXNET3_CMD_UPDATE_MAC_FILTERS:
+        VMW_CBPRN("Set: Update MAC filters");
+        vmxnet3_update_mcast_filters(s);
+        break;
+
+    case VMXNET3_CMD_UPDATE_FEATURE:
+        VMW_CBPRN("Set: Update features");
+        vmxnet3_update_features(s);
+        break;
+
+    case VMXNET3_CMD_UPDATE_PMCFG:
+        VMW_CBPRN("Set: Update power management config");
+        vmxnet3_update_pm_state(s);
+        break;
+
+    case VMXNET3_CMD_GET_LINK:
+        VMW_CBPRN("Set: Get link");
+        break;
+
+    case VMXNET3_CMD_RESET_DEV:
+        VMW_CBPRN("Set: Reset device");
+        vmxnet3_reset(s);
+        break;
+
+    case VMXNET3_CMD_QUIESCE_DEV:
+        VMW_CBPRN("Set: VMXNET3_CMD_QUIESCE_DEV - pause the device");
+        vmxnet3_deactivate_device(s);
+        break;
+
+    case VMXNET3_CMD_GET_CONF_INTR:
+        VMW_CBPRN("Set: VMXNET3_CMD_GET_CONF_INTR - interrupt configuration");
+        break;
+
+    default:
+        VMW_CBPRN("Received unknown command: %" PRIx64, cmd);
+        break;
+    }
+}
+
+static uint64_t vmxnet3_get_command_status(VMXNET3State *s)
+{
+    uint64_t ret;
+
+    switch (s->last_command) {
+    case VMXNET3_CMD_ACTIVATE_DEV:
+        ret = (s->device_active) ? 0 : -1;
+        VMW_CFPRN("Device active: %" PRIx64, ret);
+        break;
+
+    case VMXNET3_CMD_GET_LINK:
+        ret = s->link_status_and_speed;
+        VMW_CFPRN("Link and speed: %" PRIx64, ret);
+        break;
+
+    case VMXNET3_CMD_GET_PERM_MAC_LO:
+        ret = vmxnet3_get_mac_low(&s->perm_mac);
+        break;
+
+    case VMXNET3_CMD_GET_PERM_MAC_HI:
+        ret = vmxnet3_get_mac_high(&s->perm_mac);
+        break;
+
+    case VMXNET3_CMD_GET_CONF_INTR:
+        ret = vmxnet3_get_interrupt_config(s);
+        break;
+
+    default:
+        VMW_WRPRN("Received request for unknown command: %x", s->last_command);
+        ret = -1;
+        break;
+    }
+
+    return ret;
+}
+
+static void vmxnet3_set_events(VMXNET3State *s, uint32_t val)
+{
+    uint32_t events;
+
+    VMW_CBPRN("Setting events: 0x%x", val);
+    events = VMXNET3_READ_DRV_SHARED32(s->drv_shmem, ecr) | val;
+    VMXNET3_WRITE_DRV_SHARED32(s->drv_shmem, ecr, events);
+}
+
+static void vmxnet3_ack_events(VMXNET3State *s, uint32_t val)
+{
+    uint32_t events;
+
+    VMW_CBPRN("Clearing events: 0x%x", val);
+    events = VMXNET3_READ_DRV_SHARED32(s->drv_shmem, ecr) & ~val;
+    VMXNET3_WRITE_DRV_SHARED32(s->drv_shmem, ecr, events);
+}
+
+static void
+vmxnet3_io_bar1_write(void *opaque,
+                      hwaddr addr,
+                      uint64_t val,
+                      unsigned size)
+{
+    VMXNET3State *s = opaque;
+
+    switch (addr) {
+    /* Vmxnet3 Revision Report Selection */
+    case VMXNET3_REG_VRRS:
+        VMW_CBPRN("Write BAR1 [VMXNET3_REG_VRRS] = %" PRIx64 ", size %d",
+                  val, size);
+        break;
+
+    /* UPT Version Report Selection */
+    case VMXNET3_REG_UVRS:
+        VMW_CBPRN("Write BAR1 [VMXNET3_REG_UVRS] = %" PRIx64 ", size %d",
+                  val, size);
+        break;
+
+    /* Driver Shared Address Low */
+    case VMXNET3_REG_DSAL:
+        VMW_CBPRN("Write BAR1 [VMXNET3_REG_DSAL] = %" PRIx64 ", size %d",
+                  val, size);
+        /*
+         * Guest driver will first write the low part of the shared
+         * memory address. We save it to temp variable and set the
+         * shared address only after we get the high part
+         */
+        if (0 == val) {
+            s->device_active = false;
+        }
+        s->temp_shared_guest_driver_memory = val;
+        s->drv_shmem = 0;
+        break;
+
+    /* Driver Shared Address High */
+    case VMXNET3_REG_DSAH:
+        VMW_CBPRN("Write BAR1 [VMXNET3_REG_DSAH] = %" PRIx64 ", size %d",
+                  val, size);
+        /*
+         * Set the shared memory between guest driver and device.
+         * We already should have low address part.
+         */
+        s->drv_shmem = s->temp_shared_guest_driver_memory | (val << 32);
+        break;
+
+    /* Command */
+    case VMXNET3_REG_CMD:
+        VMW_CBPRN("Write BAR1 [VMXNET3_REG_CMD] = %" PRIx64 ", size %d",
+                  val, size);
+        vmxnet3_handle_command(s, val);
+        break;
+
+    /* MAC Address Low */
+    case VMXNET3_REG_MACL:
+        VMW_CBPRN("Write BAR1 [VMXNET3_REG_MACL] = %" PRIx64 ", size %d",
+                  val, size);
+        s->temp_mac = val;
+        break;
+
+    /* MAC Address High */
+    case VMXNET3_REG_MACH:
+        VMW_CBPRN("Write BAR1 [VMXNET3_REG_MACH] = %" PRIx64 ", size %d",
+                  val, size);
+        vmxnet3_set_variable_mac(s, val, s->temp_mac);
+        break;
+
+    /* Interrupt Cause Register */
+    case VMXNET3_REG_ICR:
+        VMW_CBPRN("Write BAR1 [VMXNET3_REG_ICR] = %" PRIx64 ", size %d",
+                  val, size);
+        assert(false);
+        break;
+
+    /* Event Cause Register */
+    case VMXNET3_REG_ECR:
+        VMW_CBPRN("Write BAR1 [VMXNET3_REG_ECR] = %" PRIx64 ", size %d",
+                  val, size);
+        vmxnet3_ack_events(s, val);
+        break;
+
+    default:
+        VMW_CBPRN("Unknown Write to BAR1 [%" PRIx64 "] = %" PRIx64 ", size %d",
+                  addr, val, size);
+        break;
+    }
+}
+
+static uint64_t
+vmxnet3_io_bar1_read(void *opaque, hwaddr addr, unsigned size)
+{
+        VMXNET3State *s = opaque;
+        uint64_t ret = 0;
+
+        switch (addr) {
+        /* Vmxnet3 Revision Report Selection */
+        case VMXNET3_REG_VRRS:
+            VMW_CBPRN("Read BAR1 [VMXNET3_REG_VRRS], size %d", size);
+            ret = VMXNET3_DEVICE_REVISION;
+            break;
+
+        /* UPT Version Report Selection */
+        case VMXNET3_REG_UVRS:
+            VMW_CBPRN("Read BAR1 [VMXNET3_REG_UVRS], size %d", size);
+            ret = VMXNET3_DEVICE_VERSION;
+            break;
+
+        /* Command */
+        case VMXNET3_REG_CMD:
+            VMW_CBPRN("Read BAR1 [VMXNET3_REG_CMD], size %d", size);
+            ret = vmxnet3_get_command_status(s);
+            break;
+
+        /* MAC Address Low */
+        case VMXNET3_REG_MACL:
+            VMW_CBPRN("Read BAR1 [VMXNET3_REG_MACL], size %d", size);
+            ret = vmxnet3_get_mac_low(&s->conf.macaddr);
+            break;
+
+        /* MAC Address High */
+        case VMXNET3_REG_MACH:
+            VMW_CBPRN("Read BAR1 [VMXNET3_REG_MACH], size %d", size);
+            ret = vmxnet3_get_mac_high(&s->conf.macaddr);
+            break;
+
+        /*
+         * Interrupt Cause Register
+         * Used for legacy interrupts only so interrupt index always 0
+         */
+        case VMXNET3_REG_ICR:
+            VMW_CBPRN("Read BAR1 [VMXNET3_REG_ICR], size %d", size);
+            if (vmxnet3_interrupt_asserted(s, 0)) {
+                vmxnet3_clear_interrupt(s, 0);
+                ret = true;
+            } else {
+                ret = false;
+            }
+            break;
+
+        default:
+            VMW_CBPRN("Unknow read BAR1[%" PRIx64 "], %d bytes", addr, size);
+            break;
+        }
+
+        return ret;
+}
+
+static int
+vmxnet3_can_receive(NetClientState *nc)
+{
+    VMXNET3State *s = DO_UPCAST(NICState, nc, nc)->opaque;
+    return s->device_active &&
+           VMXNET_FLAG_IS_SET(s->link_status_and_speed, VMXNET3_LINK_STATUS_UP);
+}
+
+static inline bool
+vmxnet3_is_registered_vlan(VMXNET3State *s, const void *data)
+{
+    uint16_t vlan_tag = eth_get_pkt_tci(data) & VLAN_VID_MASK;
+    if (IS_SPECIAL_VLAN_ID(vlan_tag)) {
+        return true;
+    }
+
+    return VMXNET3_VFTABLE_ENTRY_IS_SET(s->vlan_table, vlan_tag);
+}
+
+static bool
+vmxnet3_is_allowed_mcast_group(VMXNET3State *s, const uint8_t *group_mac)
+{
+    int i;
+    for (i = 0; i < s->mcast_list_len; i++) {
+        if (!memcmp(group_mac, s->mcast_list[i].a, sizeof(s->mcast_list[i]))) {
+            return true;
+        }
+    }
+    return false;
+}
+
+static bool
+vmxnet3_rx_filter_may_indicate(VMXNET3State *s, const void *data,
+    size_t size)
+{
+    struct eth_header *ehdr = PKT_GET_ETH_HDR(data);
+
+    if (VMXNET_FLAG_IS_SET(s->rx_mode, VMXNET3_RXM_PROMISC)) {
+        return true;
+    }
+
+    if (!vmxnet3_is_registered_vlan(s, data)) {
+        return false;
+    }
+
+    switch (vmxnet_rx_pkt_get_packet_type(s->rx_pkt)) {
+    case ETH_PKT_UCAST:
+        if (!VMXNET_FLAG_IS_SET(s->rx_mode, VMXNET3_RXM_UCAST)) {
+            return false;
+        }
+        if (memcmp(s->conf.macaddr.a, ehdr->h_dest, ETH_ALEN)) {
+            return false;
+        }
+        break;
+
+    case ETH_PKT_BCAST:
+        if (!VMXNET_FLAG_IS_SET(s->rx_mode, VMXNET3_RXM_BCAST)) {
+            return false;
+        }
+        break;
+
+    case ETH_PKT_MCAST:
+        if (VMXNET_FLAG_IS_SET(s->rx_mode, VMXNET3_RXM_ALL_MULTI)) {
+            return true;
+        }
+        if (!VMXNET_FLAG_IS_SET(s->rx_mode, VMXNET3_RXM_MCAST)) {
+            return false;
+        }
+        if (!vmxnet3_is_allowed_mcast_group(s, ehdr->h_dest)) {
+            return false;
+        }
+        break;
+
+    default:
+        assert(false);
+    }
+
+    return true;
+}
+
+static ssize_t
+vmxnet3_receive(NetClientState *nc, const uint8_t *buf, size_t size)
+{
+    VMXNET3State *s = DO_UPCAST(NICState, nc, nc)->opaque;
+    size_t bytes_indicated;
+
+    if (!vmxnet3_can_receive(&s->nic->nc)) {
+        VMW_PKPRN("Cannot receive now");
+        return -1;
+    }
+
+    if (s->peer_has_vhdr) {
+        vmxnet_rx_pkt_set_vhdr(s->rx_pkt, (struct virtio_net_hdr *)buf);
+        buf += sizeof(struct virtio_net_hdr);
+        size -= sizeof(struct virtio_net_hdr);
+    }
+
+    vmxnet_rx_pkt_set_packet_type(s->rx_pkt,
+        get_eth_packet_type(PKT_GET_ETH_HDR(buf)));
+
+    if (vmxnet3_rx_filter_may_indicate(s, buf, size)) {
+        vmxnet_rx_pkt_attach_data(s->rx_pkt, buf, size, s->rx_vlan_stripping);
+        bytes_indicated = vmxnet3_indicate_packet(s) ? size : -1;
+        if (bytes_indicated < size) {
+            VMW_PKPRN("RX: %lu of %lu bytes indicated", bytes_indicated, size);
+        }
+    } else {
+        VMW_PKPRN("Packet dropped by RX filter");
+        bytes_indicated = size;
+    }
+
+    assert(size > 0);
+    assert(bytes_indicated != 0);
+    return bytes_indicated;
+}
+
+static void vmxnet3_cleanup(NetClientState *nc)
+{
+    VMXNET3State *s = DO_UPCAST(NICState, nc, nc)->opaque;
+    s->nic = NULL;
+}
+
+static void vmxnet3_set_link_status(NetClientState *nc)
+{
+    VMXNET3State *s = DO_UPCAST(NICState, nc, nc)->opaque;
+
+    if (nc->link_down) {
+        s->link_status_and_speed &= ~VMXNET3_LINK_STATUS_UP;
+    } else {
+        s->link_status_and_speed |= VMXNET3_LINK_STATUS_UP;
+    }
+
+    vmxnet3_set_events(s, VMXNET3_ECR_LINK);
+    vmxnet3_trigger_interrupt(s, s->event_int_idx);
+}
+
+static NetClientInfo net_vmxnet3_info = {
+        .type = NET_CLIENT_OPTIONS_KIND_NIC,
+        .size = sizeof(NICState),
+        .can_receive = vmxnet3_can_receive,
+        .receive = vmxnet3_receive,
+        .cleanup = vmxnet3_cleanup,
+        .link_status_changed = vmxnet3_set_link_status,
+};
+
+static bool vmxnet3_peer_has_vnet_hdr(VMXNET3State *s)
+{
+    NetClientState *peer = s->nic->nc.peer;
+
+    if ((NULL != peer)                              &&
+        (peer->info->type == NET_CLIENT_OPTIONS_KIND_TAP)   &&
+        tap_has_vnet_hdr(peer)) {
+        return true;
+    }
+
+    VMW_WRPRN("Peer has no virtio extension. Task offloads will be emulated.");
+    return false;
+}
+
+static void vmxnet3_net_uninit(VMXNET3State *s)
+{
+    if (NULL != s->mcast_list) {
+        g_free(s->mcast_list);
+    }
+
+    vmxnet_tx_pkt_reset(s->tx_pkt);
+    vmxnet_tx_pkt_uninit(s->tx_pkt);
+    vmxnet_rx_pkt_uninit(s->rx_pkt);
+}
+
+static void vmxnet3_net_init(VMXNET3State *s)
+{
+    VMW_CBPRN("vmxnet3_net_init called...");
+
+    qemu_macaddr_default_if_unset(&s->conf.macaddr);
+
+    /* Windows guest will query the address that was set on init */
+    memcpy(&s->perm_mac.a, &s->conf.macaddr.a, sizeof(s->perm_mac.a));
+
+    s->mcast_list = NULL;
+    s->mcast_list_len = 0;
+
+    s->link_status_and_speed = VMXNET3_LINK_SPEED | VMXNET3_LINK_STATUS_UP;
+
+    VMW_CFPRN("Permanent MAC: " MAC_FMT, MAC_ARG(s->perm_mac.a));
+
+    s->nic = qemu_new_nic(&net_vmxnet3_info, &s->conf,
+                          object_get_typename(OBJECT(s)),
+                          s->dev.qdev.id, s);
+
+    s->peer_has_vhdr = vmxnet3_peer_has_vnet_hdr(s);
+    s->tx_sop = true;
+    s->skip_current_tx_pkt = false;
+    s->tx_pkt = NULL;
+    s->rx_pkt = NULL;
+    s->rx_vlan_stripping = false;
+    s->lro_supported = false;
+
+    if (s->peer_has_vhdr) {
+        tap_set_vnet_hdr_len(s->nic->nc.peer, sizeof(struct virtio_net_hdr));
+        tap_using_vnet_hdr(s->nic->nc.peer, 1);
+    }
+
+    qemu_format_nic_info_str(&s->nic->nc, s->conf.macaddr.a);
+}
+
+static void
+vmxnet3_unuse_msix_vectors(VMXNET3State *s, int num_vectors)
+{
+    int i;
+    for (i = 0; i < num_vectors; i++) {
+        msix_vector_unuse(&s->dev, i);
+    }
+}
+
+static bool
+vmxnet3_use_msix_vectors(VMXNET3State *s, int num_vectors)
+{
+    int i;
+    for (i = 0; i < num_vectors; i++) {
+        int res = msix_vector_use(&s->dev, i);
+        if (0 > res) {
+            VMW_WRPRN("Failed to use MSI-X vector %d, error %d", i, res);
+            vmxnet3_unuse_msix_vectors(s, i);
+            return false;
+        }
+    }
+    return true;
+}
+
+static bool
+vmxnet3_init_msix(VMXNET3State *s)
+{
+    int res = msix_init(&s->dev, VMXNET3_MAX_INTRS,
+                        &s->msix_bar,
+                        VMXNET3_MSIX_BAR_IDX, VMXNET3_OFF_MSIX_TABLE,
+                        &s->msix_bar,
+                        VMXNET3_MSIX_BAR_IDX, VMXNET3_OFF_MSIX_PBA,
+                        0);
+
+    if (0 > res) {
+        VMW_WRPRN("Failed to initialize MSI-X, error %d", res);
+        s->msix_used = false;
+    } else {
+        if (!vmxnet3_use_msix_vectors(s, VMXNET3_MAX_INTRS)) {
+            VMW_WRPRN("Failed to use MSI-X vectors, error %d", res);
+            msix_uninit(&s->dev, &s->msix_bar, &s->msix_bar);
+            s->msix_used = false;
+        } else {
+            s->msix_used = true;
+        }
+    }
+    return s->msix_used;
+}
+
+static void
+vmxnet3_cleanup_msix(VMXNET3State *s)
+{
+    if (s->msix_used) {
+        msix_vector_unuse(&s->dev, VMXNET3_MAX_INTRS);
+        msix_uninit(&s->dev, &s->msix_bar, &s->msix_bar);
+    }
+}
+
+static bool
+vmxnet3_init_msi(VMXNET3State *s)
+{
+#define VMXNET3_MSI_NUM_VECTORS   (1)
+#define VMXNET3_MSI_OFFSET        (0x50)
+#define VMXNET3_USE_64BIT         (true)
+#define VMXNET3_PER_VECTOR_MASK   (false)
+
+    int res;
+    res = msi_init(&s->dev, VMXNET3_MSI_OFFSET, VMXNET3_MSI_NUM_VECTORS,
+                   VMXNET3_USE_64BIT, VMXNET3_PER_VECTOR_MASK);
+    if (0 > res) {
+        VMW_WRPRN("Failed to initialize MSI, error %d", res);
+        s->msi_used = false;
+    } else {
+        s->msi_used = true;
+    }
+
+    return s->msi_used;
+}
+
+static void
+vmxnet3_cleanup_msi(VMXNET3State *s)
+{
+    if (s->msi_used) {
+        msi_uninit(&s->dev);
+    }
+}
+
+static void
+vmxnet3_msix_save(QEMUFile *f, void *opaque)
+{
+    msix_save(&((VMXNET3State *)opaque)->dev, f);
+}
+
+static int
+vmxnet3_msix_load(QEMUFile *f, void *opaque, int version_id)
+{
+    msix_load(&((VMXNET3State *)opaque)->dev, f);
+    return 0;
+}
+
+static int vmxnet3_pci_init(PCIDevice *dev)
+{
+    static const MemoryRegionOps b0_ops = {
+        .read = vmxnet3_io_bar0_read,
+        .write = vmxnet3_io_bar0_write,
+        .endianness = DEVICE_LITTLE_ENDIAN,
+        .impl = {
+                .min_access_size = 4,
+                .max_access_size = 4,
+        },
+    };
+
+    static const MemoryRegionOps b1_ops = {
+        .read = vmxnet3_io_bar1_read,
+        .write = vmxnet3_io_bar1_write,
+        .endianness = DEVICE_LITTLE_ENDIAN,
+        .impl = {
+                .min_access_size = 4,
+                .max_access_size = 4,
+        },
+    };
+
+    VMXNET3State *s = DO_UPCAST(VMXNET3State, dev, dev);
+    int i;
+
+    VMW_CBPRN("Starting init...");
+
+    memory_region_init_io(&s->bar0, &b0_ops, s,
+                          "vmxnet3-b0", VMXNET3_PT_REG_SIZE);
+    pci_register_bar(&s->dev, VMXNET3_BAR0_IDX,
+                     PCI_BASE_ADDRESS_SPACE_MEMORY, &s->bar0);
+
+    memory_region_init_io(&s->bar1, &b1_ops, s,
+                          "vmxnet3-b1", VMXNET3_VD_REG_SIZE);
+    pci_register_bar(&s->dev, VMXNET3_BAR1_IDX,
+                     PCI_BASE_ADDRESS_SPACE_MEMORY, &s->bar1);
+
+    memory_region_init(&s->msix_bar, "vmxnet3-msix-bar",
+                       VMXNET3_MSIX_BAR_SIZE);
+    pci_register_bar(&s->dev, VMXNET3_MSIX_BAR_IDX,
+                     PCI_BASE_ADDRESS_SPACE_MEMORY, &s->msix_bar);
+
+    for (i = 0; i < ARRAY_SIZE(s->interrupt_states); i++) {
+        s->interrupt_states[i].is_asserted = false;
+        s->interrupt_states[i].is_pending = false;
+        s->interrupt_states[i].is_masked = true;
+    }
+
+    /* Interrupt pin A */
+    s->dev.config[PCI_INTERRUPT_PIN] = 0x01;
+
+    if (!vmxnet3_init_msix(s)) {
+        VMW_WRPRN("Failed to initialize MSI-X, configuration is inconsistent.");
+    }
+
+    if (!vmxnet3_init_msi(s)) {
+        VMW_WRPRN("Failed to initialize MSI, configuration is inconsistent.");
+    }
+
+    vmxnet3_net_init(s);
+
+    register_savevm(&dev->qdev, "vmxnet3-msix", -1, 1,
+                    vmxnet3_msix_save, vmxnet3_msix_load, s);
+
+    add_boot_device_path(s->conf.bootindex, &dev->qdev, "/ethernet-phy@0");
+
+    return 0;
+}
+
+
+static void vmxnet3_pci_uninit(PCIDevice *dev)
+{
+    VMXNET3State *s = DO_UPCAST(VMXNET3State, dev, dev);
+
+    VMW_CBPRN("Starting uninit...");
+
+    unregister_savevm(&dev->qdev, "vmxnet3-msix", s);
+
+    vmxnet3_net_uninit(s);
+
+    vmxnet3_cleanup_msix(s);
+
+    vmxnet3_cleanup_msi(s);
+
+    memory_region_destroy(&s->bar0);
+    memory_region_destroy(&s->bar1);
+    memory_region_destroy(&s->msix_bar);
+}
+
+static void vmxnet3_qdev_reset(DeviceState *dev)
+{
+    VMXNET3State *s = DO_UPCAST(VMXNET3State, dev.qdev, dev);
+    VMW_CBPRN("Starting QDEV reset...");
+    vmxnet3_reset(s);
+}
+
+static bool vmxnet3_mc_list_needed(void *opaque)
+{
+    return true;
+}
+
+static int vmxnet3_mcast_list_pre_load(void *opaque)
+{
+    VMXNET3State *s = opaque;
+
+    s->mcast_list = g_malloc(s->mcast_list_buff_size);
+
+    return 0;
+}
+
+
+static void vmxnet3_pre_save(void *opaque)
+{
+    VMXNET3State *s = opaque;
+
+    s->mcast_list_buff_size = s->mcast_list_len * sizeof(MACAddr);
+}
+
+static const VMStateDescription vmxtate_vmxnet3_mcast_list = {
+    .name = "vmxnet3/mcast_list",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .pre_load = vmxnet3_mcast_list_pre_load,
+    .fields = (VMStateField[]) {
+        VMSTATE_VBUFFER_UINT32(mcast_list, VMXNET3State, 0, NULL, 0,
+            mcast_list_buff_size),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static void vmxnet3_get_ring_from_file(QEMUFile *f, Vmxnet3Ring *r)
+{
+#if HWADDR_BITS == 64
+    r->pa = qemu_get_be64(f);
+#else
+    r->pa = qemu_get_be32(f);
+#endif
+    r->size = qemu_get_be32(f);
+    r->cell_size = qemu_get_be32(f);
+    r->next = qemu_get_be32(f);
+    r->gen = qemu_get_byte(f);
+}
+
+static void vmxnet3_put_ring_to_file(QEMUFile *f, Vmxnet3Ring *r)
+{
+#if HWADDR_BITS == 64
+    qemu_put_be64(f, r->pa);
+#else
+    qemu_put_be32(f, r->pa);
+#endif
+    qemu_put_be32(f, r->size);
+    qemu_put_be32(f, r->cell_size);
+    qemu_put_be32(f, r->next);
+    qemu_put_byte(f, r->gen);
+}
+
+static void vmxnet3_get_tx_stats_from_file(QEMUFile *f,
+    struct UPT1_TxStats *tx_stat)
+{
+   tx_stat->TSOPktsTxOK = qemu_get_be64(f);
+   tx_stat->TSOBytesTxOK = qemu_get_be64(f);
+   tx_stat->ucastPktsTxOK = qemu_get_be64(f);
+   tx_stat->ucastBytesTxOK = qemu_get_be64(f);
+   tx_stat->mcastPktsTxOK = qemu_get_be64(f);
+   tx_stat->mcastBytesTxOK = qemu_get_be64(f);
+   tx_stat->bcastPktsTxOK = qemu_get_be64(f);
+   tx_stat->bcastBytesTxOK = qemu_get_be64(f);
+   tx_stat->pktsTxError = qemu_get_be64(f);
+   tx_stat->pktsTxDiscard = qemu_get_be64(f);
+}
+
+static void vmxnet3_put_tx_stats_to_file(QEMUFile *f,
+    struct UPT1_TxStats *tx_stat)
+{
+   qemu_put_be64(f, tx_stat->TSOPktsTxOK);
+   qemu_put_be64(f, tx_stat->TSOBytesTxOK);
+   qemu_put_be64(f, tx_stat->ucastPktsTxOK);
+   qemu_put_be64(f, tx_stat->ucastBytesTxOK);
+   qemu_put_be64(f, tx_stat->mcastPktsTxOK);
+   qemu_put_be64(f, tx_stat->mcastBytesTxOK);
+   qemu_put_be64(f, tx_stat->bcastPktsTxOK);
+   qemu_put_be64(f, tx_stat->bcastBytesTxOK);
+   qemu_put_be64(f, tx_stat->pktsTxError);
+   qemu_put_be64(f, tx_stat->pktsTxDiscard);
+}
+
+static int vmxnet3_get_txq_descr(QEMUFile *f, void *pv, size_t size)
+{
+    Vmxnet3TxqDescr *r = pv;
+
+    vmxnet3_get_ring_from_file(f, &r->tx_ring);
+    vmxnet3_get_ring_from_file(f, &r->comp_ring);
+    r->intr_idx = qemu_get_byte(f);
+
+#if HWADDR_BITS == 64
+    r->tx_stats_pa = qemu_get_be64(f);
+#else
+    r->tx_stats_pa = qemu_get_be32(f);
+#endif
+
+    vmxnet3_get_tx_stats_from_file(f, &r->txq_stats);
+
+    return 0;
+}
+
+static void vmxnet3_put_txq_descr(QEMUFile *f, void *pv, size_t size)
+{
+    Vmxnet3TxqDescr *r = pv;
+
+    vmxnet3_put_ring_to_file(f, &r->tx_ring);
+    vmxnet3_put_ring_to_file(f, &r->comp_ring);
+    qemu_put_byte(f, r->intr_idx);
+#if HWADDR_BITS == 64
+    qemu_put_be64(f, r->tx_stats_pa);
+#else
+    qemu_put_be32(f, r->tx_stats_pa);
+#endif
+    vmxnet3_put_tx_stats_to_file(f, &r->txq_stats);
+}
+
+const VMStateInfo txq_descr_info = {
+    .name = "txq_descr",
+    .get = vmxnet3_get_txq_descr,
+    .put = vmxnet3_put_txq_descr
+};
+
+static void vmxnet3_get_rx_stats_from_file(QEMUFile *f,
+    struct UPT1_RxStats *rx_stat)
+{
+   rx_stat->LROPktsRxOK = qemu_get_be64(f);
+   rx_stat->LROBytesRxOK = qemu_get_be64(f);
+   rx_stat->ucastPktsRxOK = qemu_get_be64(f);
+   rx_stat->ucastBytesRxOK = qemu_get_be64(f);
+   rx_stat->mcastPktsRxOK = qemu_get_be64(f);
+   rx_stat->mcastBytesRxOK = qemu_get_be64(f);
+   rx_stat->bcastPktsRxOK = qemu_get_be64(f);
+   rx_stat->bcastBytesRxOK = qemu_get_be64(f);
+   rx_stat->pktsRxOutOfBuf = qemu_get_be64(f);
+   rx_stat->pktsRxError = qemu_get_be64(f);
+}
+
+static void vmxnet3_put_rx_stats_to_file(QEMUFile *f,
+    struct UPT1_RxStats *rx_stat)
+{
+   qemu_put_be64(f, rx_stat->LROPktsRxOK);
+   qemu_put_be64(f, rx_stat->LROBytesRxOK);
+   qemu_put_be64(f, rx_stat->ucastPktsRxOK);
+   qemu_put_be64(f, rx_stat->ucastBytesRxOK);
+   qemu_put_be64(f, rx_stat->mcastPktsRxOK);
+   qemu_put_be64(f, rx_stat->mcastBytesRxOK);
+   qemu_put_be64(f, rx_stat->bcastPktsRxOK);
+   qemu_put_be64(f, rx_stat->bcastBytesRxOK);
+   qemu_put_be64(f, rx_stat->pktsRxOutOfBuf);
+   qemu_put_be64(f, rx_stat->pktsRxError);
+}
+
+static int vmxnet3_get_rxq_descr(QEMUFile *f, void *pv, size_t size)
+{
+    Vmxnet3RxqDescr *r = pv;
+    int i;
+
+    for (i = 0; i < VMXNET3_RX_RINGS_PER_QUEUE; i++) {
+        vmxnet3_get_ring_from_file(f, &r->rx_ring[i]);
+    }
+
+    vmxnet3_get_ring_from_file(f, &r->comp_ring);
+    r->intr_idx = qemu_get_byte(f);
+#if HWADDR_BITS == 64
+    r->rx_stats_pa = qemu_get_be64(f);
+#else
+    r->rx_stats_pa = qemu_get_be32(f);
+#endif
+
+    vmxnet3_get_rx_stats_from_file(f, &r->rxq_stats);
+
+    return 0;
+}
+
+static void vmxnet3_put_rxq_descr(QEMUFile *f, void *pv, size_t size)
+{
+    Vmxnet3RxqDescr *r = pv;
+    int i;
+
+    for (i = 0; i < VMXNET3_RX_RINGS_PER_QUEUE; i++) {
+        vmxnet3_put_ring_to_file(f, &r->rx_ring[i]);
+    }
+
+    vmxnet3_put_ring_to_file(f, &r->comp_ring);
+    qemu_put_byte(f, r->intr_idx);
+#if HWADDR_BITS == 64
+    qemu_put_be64(f, r->rx_stats_pa);
+#else
+    qemu_put_be32(f, r->rx_stats_pa);
+#endif
+    vmxnet3_put_rx_stats_to_file(f, &r->rxq_stats);
+}
+
+static int vmxnet3_post_load(void *opaque, int version_id)
+{
+    VMXNET3State *s = opaque;
+
+    vmxnet_tx_pkt_init(&s->tx_pkt, s->max_tx_frags, s->peer_has_vhdr);
+    vmxnet_rx_pkt_init(&s->rx_pkt, s->peer_has_vhdr);
+
+    if (s->msix_used) {
+        if  (!vmxnet3_use_msix_vectors(s, VMXNET3_MAX_INTRS)) {
+            VMW_WRPRN("Failed to re-use MSI-X vectors");
+            msix_uninit(&s->dev, &s->msix_bar, &s->msix_bar);
+            s->msix_used = false;
+            return -1;
+        }
+    }
+
+    return 0;
+}
+
+const VMStateInfo rxq_descr_info = {
+    .name = "rxq_descr",
+    .get = vmxnet3_get_rxq_descr,
+    .put = vmxnet3_put_rxq_descr
+};
+
+static int vmxnet3_get_int_state(QEMUFile *f, void *pv, size_t size)
+{
+    Vmxnet3IntState *r = pv;
+
+    r->is_masked = qemu_get_byte(f);
+    r->is_pending = qemu_get_byte(f);
+    r->is_asserted = qemu_get_byte(f);
+
+    return 0;
+}
+
+static void vmxnet3_put_int_state(QEMUFile *f, void *pv, size_t size)
+{
+    Vmxnet3IntState *r = pv;
+
+    qemu_put_byte(f, r->is_masked);
+    qemu_put_byte(f, r->is_pending);
+    qemu_put_byte(f, r->is_asserted);
+}
+
+const VMStateInfo int_state_info = {
+    .name = "int_state",
+    .get = vmxnet3_get_int_state,
+    .put = vmxnet3_put_int_state
+};
+
+static const VMStateDescription vmstate_vmxnet3 = {
+    .name = "vmxnet3",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .pre_save = vmxnet3_pre_save,
+    .post_load = vmxnet3_post_load,
+    .fields      = (VMStateField[]) {
+            VMSTATE_PCI_DEVICE(dev, VMXNET3State),
+            VMSTATE_BOOL(rx_packets_compound, VMXNET3State),
+            VMSTATE_BOOL(rx_vlan_stripping, VMXNET3State),
+            VMSTATE_BOOL(lro_supported, VMXNET3State),
+            VMSTATE_UINT32(rx_mode, VMXNET3State),
+            VMSTATE_UINT32(mcast_list_len, VMXNET3State),
+            VMSTATE_UINT32(mcast_list_buff_size, VMXNET3State),
+            VMSTATE_UINT32_ARRAY(vlan_table, VMXNET3State, VMXNET3_VFT_SIZE),
+            VMSTATE_UINT32(mtu, VMXNET3State),
+            VMSTATE_UINT16(max_rx_frags, VMXNET3State),
+            VMSTATE_UINT32(max_tx_frags, VMXNET3State),
+            VMSTATE_UINT8(event_int_idx, VMXNET3State),
+            VMSTATE_BOOL(auto_int_masking, VMXNET3State),
+            VMSTATE_UINT8(txq_num, VMXNET3State),
+            VMSTATE_UINT8(rxq_num, VMXNET3State),
+            VMSTATE_UINT32(device_active, VMXNET3State),
+            VMSTATE_UINT32(last_command, VMXNET3State),
+            VMSTATE_UINT32(link_status_and_speed, VMXNET3State),
+            VMSTATE_UINT32(temp_mac, VMXNET3State),
+#if HWADDR_BITS == 64
+            VMSTATE_UINT64(drv_shmem, VMXNET3State),
+            VMSTATE_UINT64(temp_shared_guest_driver_memory, VMXNET3State),
+#else
+            VMSTATE_UINT32(drv_shmem, VMXNET3State),
+            VMSTATE_UINT32(temp_shared_guest_driver_memory, VMXNET3State),
+#endif
+
+            VMSTATE_ARRAY(txq_descr, VMXNET3State,
+                VMXNET3_DEVICE_MAX_TX_QUEUES, 0, txq_descr_info,
+                Vmxnet3TxqDescr),
+            VMSTATE_ARRAY(rxq_descr, VMXNET3State,
+                VMXNET3_DEVICE_MAX_RX_QUEUES, 0, rxq_descr_info,
+                Vmxnet3RxqDescr),
+            VMSTATE_ARRAY(interrupt_states, VMXNET3State, VMXNET3_MAX_INTRS,
+                0, int_state_info, Vmxnet3IntState),
+
+            VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmxtate_vmxnet3_mcast_list,
+            .needed = vmxnet3_mc_list_needed
+        },
+        {
+            /* empty element. */
+        }
+    }
+};
+
+static void
+vmxnet3_write_config(PCIDevice *pci, uint32_t addr, uint32_t val, int len)
+{
+    pci_default_write_config(pci, addr, val, len);
+    msix_write_config(pci, addr, val, len);
+    msi_write_config(pci, addr, val, len);
+}
+
+static Property vmxnet3_properties[] = {
+    DEFINE_NIC_PROPERTIES(VMXNET3State, conf),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
+static void vmxnet3_class_init(ObjectClass *class, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(class);
+    PCIDeviceClass *c = PCI_DEVICE_CLASS(class);
+
+    c->init = vmxnet3_pci_init;
+    c->exit = vmxnet3_pci_uninit;
+    c->vendor_id = PCI_VENDOR_ID_VMWARE;
+    c->device_id = PCI_DEVICE_ID_VMWARE_VMXNET3;
+    c->revision = PCI_DEVICE_ID_VMWARE_VMXNET3_REVISION;
+    c->class_id = PCI_CLASS_NETWORK_ETHERNET;
+    c->subsystem_vendor_id = PCI_VENDOR_ID_VMWARE;
+    c->subsystem_id = PCI_DEVICE_ID_VMWARE_VMXNET3;
+    c->config_write = vmxnet3_write_config,
+    dc->desc = "VMWare Paravirtualized Ethernet v3";
+    dc->reset = vmxnet3_qdev_reset;
+    dc->vmsd = &vmstate_vmxnet3;
+    dc->props = vmxnet3_properties;
+}
+
+static TypeInfo vmxnet3_info = {
+    .name          = "vmxnet3",
+    .parent        = TYPE_PCI_DEVICE,
+    .instance_size = sizeof(VMXNET3State),
+    .class_init    = vmxnet3_class_init,
+};
+
+static void vmxnet3_register_types(void)
+{
+    VMW_CBPRN("vmxnet3_register_types called...");
+    type_register_static(&vmxnet3_info);
+}
+
+type_init(vmxnet3_register_types)
hw/vmxnet3.h new file +32 -0
new file mode 100644
index 0000000..b6089ba
@@ -0,0 +1,762 @@
+/*
+ * QEMU VMWARE VMXNET3 paravirtual NIC
+ *
+ * Copyright (c) 2012 Ravello Systems LTD (http://ravellosystems.com)
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Dmitry Fleytman <[email protected]>
+ * Tamir Shomer <[email protected]>
+ * Yan Vugenfirer <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef _QEMU_VMXNET3_H
+#define _QEMU_VMXNET3_H
+
+#define VMXNET_VERSION_3
+
+#define VMXNET3_DEVICE_MAX_TX_QUEUES 8
+#define VMXNET3_DEVICE_MAX_RX_QUEUES 8   /* Keep this value as a power of 2 */
+
+/*
+ * VMWARE headers we got from Linux kernel do not fully comply QEMU coding
+ * standards in sense of types and defines used.
+ * Since we didn't want to change VMWARE code, following set of typedefs
+ * and defines needed to compile these headers with QEMU introduced.
+ */
+#define u64     uint64_t

hw/vmxnet3.h line 42

Quoted 10 lines
hw/vmxnet3.h new file +10 -0
+#define u32     uint32_t
+#define u16     uint16_t
+#define u8      uint8_t
+#define __le16  uint16_t
+#define __le32  uint32_t
+#define __le64  uint64_t
+#define __packed QEMU_PACKED
+
+#if defined(HOST_WORDS_BIGENDIAN)
+#define const_cpu_to_le64(x) bswap_64(x)

This breaks compilation on ppc.

Why the hell are you byte swapping enum values? The read function you use to fetch shared register values already swaps bytes for you. And what happened to code quality standards in QEMU? My eyes are really getting sore looking at this code...

Alex

#17 lore raw
In reply to Dmitry Fleytman

Hi Everyone

Is there any progress with these patches?
Is there a chance they will be committed any time soon?

Thanks,
Dmitry

Quoted 509 lines · Dmitry Fleytman

This set of patches implements VMWare VMXNET3 paravirtual NIC device. The device supports of all the device features including offload capabilties,
VLANs and etc.
The device is tested on different OSes:

    Fedora 15
    Ubuntu 10.4
    Centos 6.2
    Windows 2008R2
    Windows 2008 64bit
    Windows 2008 32bit
    Windows 2003 64bit
    Windows 2003 32bit

Changes in V8:

Issues reported by Stefan Hajnoczi reviewed and mostly fixed:

Review

Quoted 5 lines
(quoted diff) +5 -0
+        }
+        curr_src_off += src[i].iov_len;
+    }
+    return j;
+}

The existing iov_copy() function provides equivalent functionality. I don't think iov_rebuild() is needed.

[DF] Done. Thanks, missed it.

Review

Quoted 6 lines
(quoted diff) +6 -0
+            size -= len;
+        }
+        iovec_off += iov[i].iov_len;
+    }
+    return res;
+}

Rename this net_checksum_add_iov() and place it in net/checksum.c, then the new dependency on net from block can be dropped.

[DF] Done.

Quoted 5 lines

+vmw_shmem_read(hwaddr addr, void *buf, int len)

 {
     VMW_SHPRN("SHMEM r: %" PRIx64 ", len: %d to %p", addr, len, buf);
     cpu_physical_memory_read(addr, buf, len);

}

All changes to this file should be squashed with the previous patch.

[DF] Done

Review

Quoted 13 lines
(quoted diff) +9 -0
+#ifdef VMXNET_DEBUG_SHMEM_ACCESS
+#define VMW_SHPRN(fmt, ...)
 \
+    do {
 \
+        printf("[%s][SH][%s]: " fmt "\n", VMXNET_DEVICE_NAME, __func__,
 \
+            ## __VA_ARGS__);
 \
+    } while (0)
+#else
+#define VMW_SHPRN(fmt, ...) do {} while (0)
+#endif

Please use QEMU tracing. It eliminates all this boilerplate and conditional compilation. Tracing can be enabled/disabled at runtime and works with SystemTap/DTrace. See docs/tracing.txt.

[DF] We'd like to stick with compile time logic in this case becase of 2 reasons:
[DF] 1. These printouts are intended for reverse engineering/development only and there is
[DF] no need to enable them at run time
[DF] 2. There is a big number of printouts, all driver-device communication is traced,
[DF] they hit performance even on strongest x86 in case of run-time
logic

Review

Quoted 5 lines
(quoted diff) +5 -0
+struct eth_header {
+    uint8_t  h_dest[ETH_ALEN];   /* destination eth addr */
+    uint8_t  h_source[ETH_ALEN]; /* source ether addr    */
+    uint16_t h_proto;            /* packet type ID field */
+};

Looks like it's copy-pasted stuff from /usr/include/linux/if_*.h, /usr/include/netinet/*.h, and friends. If the system-wide headers are included names will collide for some of the macros at least.

Did you check if the slirp/ definitions can be reused?

[DF] Yes, you are right. This is copy-pasted from different places. [DF] Slips definishing do not fully cover our needs.

I'd rather we import network header definitions once in a generic place into the source tree. That way vmxnet and other components don't need to redefine these structs.

[DF] Exaclty! Our intention is to create generic header with network definitions and make everyone use it.
[DF] We can move our header to some shared place if you want, however I'd do it in parallel with cleanup
[DF] of similar definitions in existing code and this is a big change that os out of scope of these patches.

Quoted 2 lines

+

*===========================================================================*/

Is this huge comment box a sign that the code should be split into a foo_tx.c and an foo_rx.c file?

[DF] As for me this file is not that big to be splitted (<800 lines), however I'll do this if you insist :)

Quoted 1 line

+size_t vmxnet_tx_pkt_send(VmxnetTxPktH pkt, NetClientState *vc)

'vc' is an old name that was used for VLANClientState. The struct has since been renamed to NetClientState and the rest of QEMU uses 'nc' instead of 'vc'.

[DF] Fixed. Thanks.

Review

Quoted 2 lines
(quoted diff) +2 -0
+/* tx module context handle */
+typedef void *VmxnetTxPktH;

Forward-declaring the struct is nicer:

typedef struct VmxnetTxPkt VmxnetTxPkt;

The definition of VmxnetTxPkt is still hidden from the caller but you avoid the void* and casting. In vmxnet_pkt.c define using:

struct VmxnetTxPkt {
    ...
};

[DF] Agree, fixed. Thanks.

hw/vmxnet_utils.h

Quoted 4 lines
hw/vmxnet_utils.h +0 -0
index 7fd9a01..fac3b7b 100644

Please squash these fixes into the previous patch.

[DF] Oops, my bad. Fixed.

hw/vmxnet_utils.h

Quoted 3 lines
hw/vmxnet_utils.h +3 -0
+        uint8_t msix_used;
+        /* Whether MSI support was installed successfully */
+        uint8_t msi_used;

These two fields should be bool.

hw/vmxnet_utils.h

Quoted 2 lines
hw/vmxnet_utils.h +2 -0
+        /* Whether automatic interrupts masking enabled */
+        uint8_t auto_int_masking;

bool

[DF] Done.

hw/vmxnet_utils.h

Quoted 9 lines
hw/vmxnet_utils.h +9 -0
+static inline void vmxnet3_flush_shmem_changes(void)
+{
+    /*
+     * Flush shared memory changes
+     * Needed before sending interrupt to guest to ensure
+     * it gets consistent memory state
+     */
+    smp_wmb();
+}

It's useful to document why a memory barrier is being used in each instance. Therefore hiding smp_wmb() inside a wrapper function isn't great.

[DF] Fixed.

Also, it's suspicious that smb_wmb() is used but no other barriers are used. What about a read memory barrier when accessing shared memory written by the guest?

[DF] VMWARE interface build a in safe way - you always read out data buffer (packet descriptor) with memcopy,
[DF] and then check its last byte to see whether it was fully filled by driver.
[DF] In this case device doesn't need read barriers. Drivers need write barriers of course to secure
[DF] proper order of writes.

Changes in V7:

Issues reported by Michael S. Tsirkin reviewed and mostly fixed:

File vmware_utils.h:

...

hw/vmxnet_utils.h

Quoted 4 lines
hw/vmxnet_utils.h +4 -0
+static inline void
+vmw_shmem_st64(target_phys_addr_t addr, uint64_t value)
+{
+    VMW_SHPRN("SHMEM store64: %" PRIx64 " (value %" PRIx64 ")", addr,

value);

hw/vmxnet_utils.h

Quoted 3 lines
hw/vmxnet_utils.h +3 -0
+    stq_le_phys(addr, value);
+}
+

Pls remove these wrappers. These are just memory stores. Our codebase is too large as it is without every driver wrapping all standard calls.

[DF] Idea behind this macros is to have printout in each of them [DF] Printouts are really needed when reverse-engineering windows guest drivers
[DF] Since there is no windows drivers code this is the only way to understand
[DF] sequences they use

Quoted 1 line

+/* MACROS for simplification of operations on array-style registers */

UPPERCASE ABUSE

[DF] Fixed

hw/vmxnet_utils.h

Quoted 2 lines
hw/vmxnet_utils.h +2 -0
+#define IS_MULTIREG_ADDR(addr, base, cnt, regsize)                 \
+    (((addr + 1) > (base)) && ((addr) < (base) + (cnt) * (regsize)))

Same as range_covers_byte(base, cnt * regsize, addr)?

[DF] Fixed, thanks

hw/vmxnet_utils.h

Quoted 4 lines
hw/vmxnet_utils.h +4 -0
+
+#define MULTIREG_IDX_BY_ADDR(addr, base, regsize)                  \
+    (((addr) - (base)) / (regsize))
+

Above two macros is all that's left. No objection but it does not say what they do - want to add minimal documentation?
And please prefix with VMWARE_ or something.

[DF] Prefix added, macros documented

File vmxnet_utils.h (and related with the same problems):

...

hw/vmxnet_utils.h

Quoted 6 lines

+ * See the COPYING file in the top-level directory.
+ *
+ */

hw/vmxnet_utils.h +3 -0
+
+#ifndef _VMXNET_UTILS_H_
+#define _VMXNET_UTILS_H_

Please do not start identifiers with _ followed by an uppercase lattters.

[DF] Fixed

...

hw/vmxnet_utils.h

Quoted 7 lines
hw/vmxnet_utils.h +7 -0
+
+struct eth_header {
+    uint8_t  h_dest[ETH_ALEN];   /* destination eth addr */
+    uint8_t  h_source[ETH_ALEN]; /* source ether addr    */
+    uint16_t h_proto;            /* packet type ID field */
+};
+

And fix struct definitions to
1. follow qemu coding style
[DF] It's not clear what's wrong with the coding style. Please, elaborate.
2. start with vmxnet
[DF] Since this is generic network definitions with no device specifics [DF] I'm not sure it makes sense to add vmxnet prefix.
[DF] I'd say it makes sense to put them into some generic header files under "net" directory instead
[DF] and clean up other devices that have their local definitions of the same structures.
[DF] Please, advise.

I also don't really understand why are these functions split out - vmxnet is the only user, no?

[DF] Currently vmxnet3 is the only user, hovewer we have vmxnet2 implementation as well (not submitted yet)
[DF] and it uses the same header

File vmxnet_pkt.h:

...

hw/vmxnet_utils.h

Quoted 3 lines
hw/vmxnet_utils.h +3 -0
+
+/* tx module context handle */
+typedef void *VmxnetTxPktH;

This gets you zero type safety and makes debugging impossible. Just use pointers like everyone does.

[DF] Handle type is added to hide VmxnetTxPkt structure into .c header [DF] for better encapsulation.
[DF] Should we drop it anyway?

Files vmxnet3.*:

...

hw/vmxnet_utils.h

Quoted 3 lines
hw/vmxnet_utils.h +3 -0
+
+    if (zero_region) {
+        vmw_shmem_set(pa, 0, size*cell_size);

spaces around *

[DF] Fixed

...

Quoted 3 lines

+#define vmxnet3_ring_dump(macro, ring_name, ridx, r) \
+ macro("%s#%d: base %" PRIx64 " size %lu cell_size %lu gen %d next

%lu",

Quoted 4 lines

\
+ (ring_name), (ridx),
\
+ (r)->pa, (r)->size, (r)->cell_size, (r)->gen, (r)->next)

make macros upper case

[DF] Fixed

...

hw/vmxnet_utils.h

Quoted 5 lines
hw/vmxnet_utils.h +5 -0
+static inline void vmxnet3_flush_shmem_changes(void)
+{
+    /*
+     * Flush shared memory changes
+     * Needed before transferring control to guest

what does 'transferring control to guest' mean?
[DF] Changed to
[DF] /*

[DF]     * Flush shared memory changes
[DF]     * Needed before sending interrupt to guest to ensure
[DF]     * it gets consistent memory state
[DF]     */

...

hw/vmxnet_utils.h

Quoted 3 lines
hw/vmxnet_utils.h +3 -0
+     */
+    smp_wmb();
+}

Don't use wrappers like this. They just hide bugs. For example it's not helpful before an interrupt in the function below.

[DF] I guess you are talking about vmxnet3_complete_packet() [DF] Strictly speaking barrier is a must because we change shared memory in [DF] vmxnet3_complete_packet()

[DF] And the wrapper is a good thing because its name explains its effect [DF] in a formal way as opposed to comments

...

hw/vmxnet_utils.h

Quoted 2 lines
hw/vmxnet_utils.h +2 -0
+    switch (status) {
+    case VMXNET3_PKT_STATUS_OK: {

don't put {} around cases: they align incorrectly
if it's too big move to a function.

[DF] Fixed

...

hw/vmxnet_utils.h

Quoted 5 lines
hw/vmxnet_utils.h +5 -0
+static bool
+vmxnet3_send_packet(VMXNET3State *s, uint32_t qidx)
+{
+    size_t bytes_sent = 0;
+    bool res = true;

why = true? don't initialize just because.

[DF] Fixed

...

Quoted 2 lines

+/*
+ * VMWARE headers we got from Linux kernel do not fully comply QEMU

coding

Quoted 4 lines

+ * standards in sense of types and defines used.
+ * Since we didn't want to change VMWARE code, following set of typedefs
+ * and defines needed to compile these headers with QEMU introduced.
+ */

No need for this now.
You can export headers and put them under linux-headers.

[DF] Not sure it is possible because the header as-is is not stand-alone and won't compile
[DF] without changes. We extracted definitions we use from their header and dropped unused
[DF] and kernel-specific stuff.
[DF} Please, advise.

...

Quoted 1 line

+ if (VMXNET3_OM_TSO == s->offload_mode) {

Don't do Yoda style like this

[DF] "Yoda" style removed everywhere

Changes in V6:
Fixed most of problems pointed out by Michael S. Tsirkin The only issue still open is creation of shared place
with generic network structures and functions. Currently all generic network code introduced by VMXNET3 resides in vmxnet_utils.c/h files. It could be moved to some shared location
however
we believe it is a matter of separate refactoring as there are a lot of
copy-pasted
definitions in almost every device and code cleanup efforts requred in
order
to create truly shared codebase.

Implemented suggestions by Anthony Liguori

Fixed incorrect checksum caclulation for some packets in SW offloads
mode

Changes in V5:
MSI-X save/load implemented in the device instead of pci bus as suggested by Michael S. Tsirkin

Patches regrouped as suggested by Paolo Bonzini

Changes in V4:
Fixed a few problems uncovered by NETIO test suit Assertion on failure to initialize MSI/MSI-X replaced with warning message and fallback to Legacy/MSI respectively

Various coding style adjustments and patch split-up as suggested by
Anthony
Liguori

Live migration support added

Changes in V3:
Fixed crash when net device that is used as network fronted has no virtio HDR support.
Task offloads emulation for cases when net device that is used as network fronted has no virtio HDR support.

Changes in V2:
License text changed accoring to community suggestions Standard license header from GPLv2+ - licensed QEMU files used

Dmitry Fleytman (5):
Adding utility function net_checksum_add_cont() that allows checksum
calculation of scattered data with odd chunk sizes
Adding utility function net_checksum_add_iov() for iovec checksum
calculation
Adding common definitions for VMWARE devices
Adding packet abstraction for VMWARE network devices Adding VMXNET3 device implementation

 default-configs/pci.mak |    1 +
 hw/Makefile.objs        |    1 +
 hw/pci.h                |    1 +
 hw/vmware_utils.h       |  143 +++
 hw/vmxnet3.c            | 2437
+++++++++++++++++++++++++++++++++++++++++++++++
 hw/vmxnet3.h            |  762 +++++++++++++++
 hw/vmxnet_debug.h       |  121 +++
 hw/vmxnet_pkt.c         |  758 +++++++++++++++
 hw/vmxnet_pkt.h         |  309 ++++++
 hw/vmxnet_utils.c       |  219 +++++
 hw/vmxnet_utils.h       |  340 +++++++
 iov.h                   |    5 +
 net/checksum.c          |   41 +-
 net/checksum.h          |   22 +-
 14 files changed, 5153 insertions(+), 7 deletions(-)
 create mode 100644 hw/vmware_utils.h
 create mode 100644 hw/vmxnet3.c
 create mode 100644 hw/vmxnet3.h
 create mode 100644 hw/vmxnet_debug.h
 create mode 100644 hw/vmxnet_pkt.c
 create mode 100644 hw/vmxnet_pkt.h
 create mode 100644 hw/vmxnet_utils.c
 create mode 100644 hw/vmxnet_utils.h

--
1.7.11.7

Signature · 7 lines
Dmitry Fleytman
Technology Expert and Consultant,

Daynix Computing Ltd.

Cell: +972-54-2819481
Skype: dmitry.fleytman
#18 lore raw
Quoted 2 lines · Dmitry Fleytman

Is there any progress with these patches?
Is there a chance they will be committed any time soon?

Hi Dmitry,
Thanks for posting this version. I have left comments but overall it seems self-contained and ready to merge soon.

One thing to be very careful about is the migration code. An error there can mean breaking live migration between QEMU versions. I took a look and it seems fine but another pair of eyes would help here.

Michael: Would you like to take this through your pci/net tree?

Stefan

#19 lore raw

Thanks Stefan,

I'l go over your comments and submit a new version.

Dmitry

Quoted 16 lines · Stefan Hajnoczi
Quoted 2 lines · Dmitry Fleytman

Is there any progress with these patches?
Is there a chance they will be committed any time soon?

Hi Dmitry,
Thanks for posting this version. I have left comments but overall it seems self-contained and ready to merge soon.

One thing to be very careful about is the migration code. An error there can mean breaking live migration between QEMU versions. I took a look and it seems fine but another pair of eyes would help here.

Michael: Would you like to take this through your pci/net tree?

Stefan

Signature · 7 lines
Dmitry Fleytman
Technology Expert and Consultant,

Daynix Computing Ltd.

Cell: +972-54-2819481
Skype: dmitry.fleytman
#20 lore raw
In reply to Dmitry Fleytman
Quoted 16 lines · Dmitry Fleytman

Review

Quoted 5 lines
(quoted diff) +5 -0
+struct eth_header {
+    uint8_t  h_dest[ETH_ALEN];   /* destination eth addr */
+    uint8_t  h_source[ETH_ALEN]; /* source ether addr    */
+    uint16_t h_proto;            /* packet type ID field */
+};

Looks like it's copy-pasted stuff from /usr/include/linux/if_*.h, /usr/include/netinet/*.h, and friends. If the system-wide headers are included names will collide for some of the macros at least.

Did you check if the slirp/ definitions can be reused?

[DF] Yes, you are right. This is copy-pasted from different places. [DF] Slips definishing do not fully cover our needs.

Quoted 7 lines

I'd rather we import network header definitions once in a generic place into the source tree. That way vmxnet and other components don't need to redefine these structs.

[DF] Exaclty! Our intention is to create generic header with network definitions and make everyone use it. [DF] We can move our header to some shared place if you want, however I'd do it in parallel with cleanup [DF] of similar definitions in existing code and this is a big change that os out of scope of these patches.

Please put it in a generic place in include/qemu/.... Please double-check copyright headers you need when copy-pasting code from various system headers.

Quoted 7 lines
Quoted 2 lines

+ *===========================================================================*/

Is this huge comment box a sign that the code should be split into a foo_tx.c and an foo_rx.c file?

[DF] As for me this file is not that big to be splitted (<800 lines), however I'll do this if you insist :)

It came to mind because I find comment boxes ugly. They tend to be pointless "*********** PUBLIC METHODS ***********" or they show that the code should really be split. Just a pet peeve but please do split it if possible.

Quoted 16 lines

Review

Quoted 9 lines
(quoted diff) +9 -0
+static inline void vmxnet3_flush_shmem_changes(void)
+{
+    /*
+     * Flush shared memory changes
+     * Needed before sending interrupt to guest to ensure
+     * it gets consistent memory state
+     */
+    smp_wmb();
+}

It's useful to document why a memory barrier is being used in each instance. Therefore hiding smp_wmb() inside a wrapper function isn't great.

[DF] Fixed.

Quoted 8 lines

Also, it's suspicious that smb_wmb() is used but no other barriers are used. What about a read memory barrier when accessing shared memory written by the guest?

[DF] VMWARE interface build a in safe way - you always read out data buffer (packet descriptor) with memcopy, [DF] and then check its last byte to see whether it was fully filled by driver.
[DF] In this case device doesn't need read barriers. Drivers need write barriers of course to secure [DF] proper order of writes.

That's not portable, you're making assumptions about memory ordering which varies from architecture to architecture. Some of the barriers compile out on x86 because the memory model is strong.

QEMU is portable across host architectures so you need to use the read memory barrier.

Stefan

#21 lore raw
Quoted 23 lines · Stefan Hajnoczi
Quoted 16 lines · Dmitry Fleytman

Review

Quoted 5 lines
(quoted diff) +5 -0
+struct eth_header {
+    uint8_t  h_dest[ETH_ALEN];   /* destination eth addr */
+    uint8_t  h_source[ETH_ALEN]; /* source ether addr    */
+    uint16_t h_proto;            /* packet type ID field */
+};

Looks like it's copy-pasted stuff from /usr/include/linux/if_*.h, /usr/include/netinet/*.h, and friends. If the system-wide headers are included names will collide for some of the macros at least.

Did you check if the slirp/ definitions can be reused?

[DF] Yes, you are right. This is copy-pasted from different places. [DF] Slips definishing do not fully cover our needs.

Quoted 5 lines

I'd rather we import network header definitions once in a generic place into the source tree. That way vmxnet and other components don't need to redefine these structs.

[DF] Exaclty! Our intention is to create generic header with network

definitions and make everyone use it.

Quoted 10 lines
Quoted 1 line

[DF] We can move our header to some shared place if you want, however

I'd do it in parallel with cleanup

Quoted 1 line

[DF] of similar definitions in existing code and this is a big change

that os out of scope of these patches.

Please put it in a generic place in include/qemu/.... Please double-check copyright headers you need when copy-pasting code from various system headers.

Done.

Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

+

*===========================================================================*/

Quoted 4 lines

Is this huge comment box a sign that the code should be split into a foo_tx.c and an foo_rx.c file?

Quoted 9 lines
Quoted 1 line

[DF] As for me this file is not that big to be splitted (<800 lines),

however I'll do this if you insist :)

It came to mind because I find comment boxes ugly. They tend to be pointless "*********** PUBLIC METHODS ***********" or they show that the code should really be split. Just a pet peeve but please do split it if possible.

Did it.

Quoted 28 lines
Quoted 14 lines

Review

Quoted 9 lines
(quoted diff) +9 -0
+static inline void vmxnet3_flush_shmem_changes(void)
+{
+    /*
+     * Flush shared memory changes
+     * Needed before sending interrupt to guest to ensure
+     * it gets consistent memory state
+     */
+    smp_wmb();
+}

It's useful to document why a memory barrier is being used in each instance. Therefore hiding smp_wmb() inside a wrapper function isn't great.

Quoted 7 lines

[DF] Fixed.

Also, it's suspicious that smb_wmb() is used but no other barriers are used. What about a read memory barrier when accessing shared memory written by the guest?

[DF] VMWARE interface build a in safe way - you always read out data

buffer (packet descriptor) with memcopy,

Quoted 1 line

[DF] and then check its last byte to see whether it was fully filled by

driver.

Quoted 1 line

[DF] In this case device doesn't need read barriers. Drivers need write

barriers of course to secure

Quoted 1 line

[DF] proper order of writes.

Quoted 9 lines

That's not portable, you're making assumptions about memory ordering which varies from architecture to architecture. Some of the barriers compile out on x86 because the memory model is strong.

QEMU is portable across host architectures so you need to use the read memory barrier.

Stefan

Oh, I see, thanks for explanation.
Fixed.

Signature · 7 lines
Dmitry Fleytman
Technology Expert and Consultant,

Daynix Computing Ltd.

Cell: +972-54-2819481
Skype: dmitry.fleytman