Skip to content

RFC virtio nvme

Ming Lin linux-nvme virtualization 1 version In review

Get series with b4
b4 am 1441904521.18716.4.camel@ssi

Versions newest first

  1. v1 RFC Latest0 of 2 patches 1 replyIn review

Patches in v1 No applies recorded yet

None of the 2 patches the cover letter announces are in this archive.

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 0/2] virtio nvme
Quoted 10 lines · Stefan Hajnoczi
Quoted 9 lines · Ming Lin

These 2 patches added virtio-nvme to kernel and qemu, basically modified from virtio-blk and nvme code.

As title said, request for your comments.

Play it in Qemu with:

(quoted diff) +0 -2
-drive file=disk.img,format=raw,if=none,id=D22 \
-device virtio-nvme-pci,drive=D22,serial=1234,num_queues=4
Quoted 6 lines
Quoted 2 lines

The goal is to have a full NVMe stack from VM guest(virtio-nvme) to host(vhost_nvme) to LIO NVMe-over-fabrics target.

Why is a virtio-nvme guest device needed? I guess there must either be NVMe-only features that you want to pass through, or you think the performance will be significantly better than virtio-blk/virtio-scsi?

It simply passes through NVMe commands.

Right now performance is poor. Performance tunning is on my todo list. It should be as good as virtio-blk/virtio-scsi.

Quoted 4 lines

At first glance it seems like the virtio_nvme guest driver is just another block driver like virtio_blk, so I'm not clear why a virtio-nvme device makes sense.

I think the future "LIO NVMe target" only speaks NVMe protocol.

Nick(CCed), could you correct me if I'm wrong?

For SCSI stack, we have:
virtio-scsi(guest)
tcm_vhost(or vhost_scsi, host)
LIO-scsi-target

For NVMe stack, we'll have similar components:
virtio-nvme(guest)
vhost_nvme(host)
LIO-NVMe-target

Quoted 13 lines
Quoted 13 lines

Now there are lots of duplicated code with linux/nvme-core.c and qemu/nvme.c. The ideal result is to have a multi level NVMe stack(similar as SCSI). So we can re-use the nvme code, for example

.-------------------------.
| NVMe device register |

  Upper level           | NVMe protocol process   |
                        |                         |
                        '-------------------------'
Quoted 8 lines
Quoted 4 lines
              .-----------.    .-----------.    .------------------.
 Lower level  |   PCIe    |    | VIRTIO    |    |NVMe over Fabrics |
              |           |    |           |    |initiator         |
              '-----------'    '-----------'    '------------------'

You mentioned LIO and SCSI. How will NVMe over Fabrics be integrated into LIO? If it is mapped to SCSI then using virtio_scsi in the guest and tcm_vhost should work.

I think it's not mapped to SCSI.

Nick, would you share more here?

Quoted 2 lines

Please also post virtio draft specifications documenting the virtio device.

I'll do this later.

Quoted 2 lines

Stefan