virtio: put last seen used index into ring itself

Changes from v2: added padding between avail idx and flags, and changed virtio to only publish used index when callbacks are enabled.

Here's a rewrite of the original patch with a new layout. I haven't tested it yet so no idea how this performs, but I think this addresses the cache bounce issue raised by Avi. Posting for early flames/comments.

Generally, the Host end of the virtio ring doesn't need to see where Guest is up to in consuming the ring. However, to completely understand what's going on from the outside, this information must be exposed. For example, host can reduce the number of interrupts by detecting that the guest is currently handling previous buffers.

We add a feature bit so the guest can tell the host that it's writing out the current value there, if it wants to use that.

This differs from original approach in that the used index is put after avail index (they are typically written out together). To avoid cache bounces on descriptor access,
and make future extensions easier, we put the ring itself at start of page, and move the control after it.

Michael S. Tsirkin (2):
virtio: support layout with avail ring before idx
virtio: publish used idx

 drivers/net/virtio_net.c     |    2 +
 drivers/virtio/virtio_ring.c |   25 +++++++++++++++------
 include/linux/virtio_ring.h  |   50 ++++++++++++++++++++++++++++++++++++-----
 3 files changed, 64 insertions(+), 13 deletions(-)