Skip to contentSkip to conversation

vhost question

DISCUSSION Steve Glass 2 participants 5 messages Mar 22, 2012 – Mar 25, 2012 1 message not in the archive

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Just some further information concerning my earlier question concerning vhost and virtio.

I'm using virtio to implement an emulated mac80211 device in the guest. A simple network simulation will be used to control delivery of frames between guests and for this I am using the vhost approach.

A simple first-cut attempt at the tx and rx kick handlers are given below. When the guest transmits frames the vhost's TX kick handler is executed and copies the buffers onto a queue for the intended recipient(s). When the vhost's RX kick handler is run it copies the buffer from the queue and notifies the client that the buffers have been used.

The problem is that if there are no frames in the queue when the guest rx kick handler runs then it has to exit and I have to arrange that it runs again. That's done in the current prototype by having the guests poll using a timer - which is ugly and inefficient. Can I get the vhost tx kick handler to wake the appropriate vhost rx kick handler? How can I achieve this?

Many thanks,

Steve

static void
handle_rx(struct vhost_work *work)

{
        int n;
        unsigned out, in, frames;
        struct transmission *t;
        struct vhost_poll *p = container_of(work, struct vhost_poll,
work);
        struct vhost_virtqueue *vq =
                container_of(p, struct vhost_virtqueue, poll);
        struct vhost_node *node =
                container_of(vq, struct vhost_node, vqs[WLAN_VQ_RX]);
        struct vhost_dev *dev = &node->vdev;

        mutex_lock(&vq->mutex);
        vhost_disable_notify(dev, vq);
        while (!queue_empty(&node->rxq)) {
                n = vhost_get_vq_desc(dev,
                                      vq,
                                      vq->iov,
                                      ARRAY_SIZE(vq->iov),
                                      &out,
                                      &in,
                                      NULL,
                                      NULL);
                if (0 < n || n == vq->num)
                        break;
                t = queue_pop(&node->rxq);
                BUG_ON(copy_to_user(vq->iov[0].iov_base, t->buf,
t->buf_sz));
                vq->iov[0].iov_len = t->buf_sz;
                vhost_add_used(vq, n, out);
                transmission_free(t);
                ++frames;
        }
        if (frames)
                vhost_signal(dev, vq);
        vhost_enable_notify(dev, vq);
        mutex_unlock(&vq->mutex);
}

static void
handle_tx(struct vhost_work *work)

{
        int n;
        unsigned out, in;
        struct transmission *t;
        struct vhost_node *receiver;

        struct vhost_poll *p =
                container_of(work, struct vhost_poll, work);
        struct vhost_virtqueue *vq =
                container_of(p, struct vhost_virtqueue, poll);
        struct vhost_node *w =
                container_of(vq, struct vhost_node, vqs[WLAN_VQ_TX]);
        struct vhost_dev *dev = &w->vdev;

        mutex_lock(&vq->mutex);
        do {
                vhost_disable_notify(dev, vq);
                n = vhost_get_vq_desc(dev,
                                      vq,
                                      vq->iov,
                                      ARRAY_SIZE(vq->iov),
                                      &out,
                                      &in,
                                      NULL,
                                      NULL);
                while (n >= 0 && n != vq->num) {
                        receiver = net_get_receiver(w);
                        if ((receiver) && (t = transmission_alloc())) {
                                BUG_ON(copy_from_user(t->buf,
                                                      vq->iov[1].iov_base,
vq->iov[1].iov_len));
                                t->buf_sz = vq->iov[1].iov_len;
                                queue_push(&receiver->rxq, t);
                                // ToDo: kick receiver's handle_rx
                        }
                        vhost_add_used(vq, n, out);
                        n = vhost_get_vq_desc(dev,
                                              vq,
                                              vq->iov,
                                              ARRAY_SIZE(vq->iov),
                                              &out,
                                              &in,
                                              NULL,
                                              NULL);
                }
                vhost_signal(dev, vq);
        } while (vhost_enable_notify(dev, vq));
        mutex_unlock(&vq->mutex);
}

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9qhO4ACgkQW7aAm65EWy7w4wCgrzGB2Zit4rWUzMjwpJEJnIfj xDsAoLBDMj+4MVrjPS5upgDSIGOi4IzL
=Ms/+
-----END PGP SIGNATURE-----

#1 lore raw
Quoted 19 lines · Steve Glass

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Just some further information concerning my earlier question concerning vhost and virtio.

I'm using virtio to implement an emulated mac80211 device in the guest. A simple network simulation will be used to control delivery of frames between guests and for this I am using the vhost approach.

A simple first-cut attempt at the tx and rx kick handlers are given below. When the guest transmits frames the vhost's TX kick handler is executed and copies the buffers onto a queue for the intended recipient(s). When the vhost's RX kick handler is run it copies the buffer from the queue and notifies the client that the buffers have been used.

Quoted 6 lines

The problem is that if there are no frames in the queue when the guest rx kick handler runs then it has to exit and I have to arrange that it runs again. That's done in the current prototype by having the guests poll using a timer - which is ugly and inefficient. Can I get the vhost tx kick handler to wake the appropriate vhost rx kick handler? How can I achieve this?

Can you queue a tx->rx kick on the vhost work queue with vhost_work_queue()?

Stefan

A message that is not in the archive (sent off-list or not yet received): CADC0ay0ZRL79uMq72kAewOBTMCgMe1_1BLotdH9QnOwosJkTYQ@mail.gmail.com

#3 lore raw
Quoted 6 lines · Steve Glass
Quoted 5 lines · Stefan Hajnoczi

Can you queue a tx->rx kick on the vhost work queue with vhost_work_queue()?

Quoted 8 lines
Quoted 1 line

Stefan

I've been looking at this and trying to work out how to achieve that. I can see how I can enqueue some work on the vhost queue but the rx_handler needs to run in the process context of the receiving VM (or else it can't access the ring). I don't think that's true of the work queue (doesn't it run with the qemu process context?)

The vhost worker thread has the mm of the process, see use_mm(dev->mm) in drivers/vhost/vhost.c:vhost_worker(). It sounds like you need vhost worker A (tx) to schedule work for vhost worker B (rx).

Stefan

#4 lore raw
In reply to a message not in the archive
Quoted 6 lines · Steve Glass
Quoted 5 lines · Stefan Hajnoczi

Can you queue a tx->rx kick on the vhost work queue with vhost_work_queue()?

Quoted 8 lines
Quoted 1 line

Stefan

I've been looking at this and trying to work out how to achieve that. I can see how I can enqueue some work on the vhost queue but the rx_handler needs to run in the process context of the receiving VM (or else it can't access the ring). I don't think that's true of the work queue (doesn't it run with the qemu process context?)

The vhost worker thread has the mm of the process, see use_mm(dev->mm) in drivers/vhost/vhost.c:vhost_worker(). It sounds like you need vhost worker A (tx) to schedule work for vhost worker B (rx).

Stefan

#5 lore raw
In reply to a message not in the archive
Quoted 6 lines · Steve Glass
Quoted 5 lines · Stefan Hajnoczi

Can you queue a tx->rx kick on the vhost work queue with vhost_work_queue()?

Quoted 8 lines
Quoted 1 line

Stefan

I've been looking at this and trying to work out how to achieve that. I can see how I can enqueue some work on the vhost queue but the rx_handler needs to run in the process context of the receiving VM (or else it can't access the ring). I don't think that's true of the work queue (doesn't it run with the qemu process context?)

The vhost worker thread has the mm of the process, see use_mm(dev->mm) in drivers/vhost/vhost.c:vhost_worker(). It sounds like you need vhost worker A (tx) to schedule work for vhost worker B (rx).

Stefan