nvme: use __iowrite64_copy for SQs in the CMB

Use __iowrite64_copy() instead of memcpy_toio() when copying SQ entries to a SQ that's contained in the CMB.
__iowrite64_copy() writes to the MMIO in 64-bit units, whenever possible.

Signed-off-by Saar Gross <saar at annapurnalabs.com>

---
 drivers/nvme/host/pci.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

drivers/nvme/host/pci.c +1 -2

index 0248d0e..0e603f0 100644
@@ -289,7 +289,7 @@ static void __nvme_submit_cmd(struct nvme_queue *nvmeq,
        u16 tail = nvmeq->sq_tail;
 
        if (nvmeq->sq_cmds_io)
-               memcpy_toio(&nvmeq->sq_cmds_io[tail], cmd, sizeof(*cmd));
+               __iowrite64_copy(&nvmeq->sq_cmds_io[tail], cmd,
 sizeof(*cmd) / 8);
        else
                memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
 
--

2.7.4

Hi everyone,

Any comments on this patch? I didn't receive any so far.

Quoted 4 lines

Any comments on this patch? I didn't receive any so far.

Please explain why we would want to apply the patch, e.g. without it some devices may be broken, or that on a particular device you get much better performance (including numbers and an explanation of the setup).

Quoted 9 lines

Quoted 4 lines

Any comments on this patch? I didn't receive any so far.

Please explain why we would want to apply the patch, e.g. without it some devices may be broken, or that on a particular device you get much better performance (including numbers and an explanation of the setup).

Some PCIe devices have requirements on the alignment of PCIe writes into them. Would this patch help with some of those devices?

Stephen

Sorry for taking so long to reply: yes, it might help with some PCIe devices with weird alignment requirements.

The patch itself doesn't fix any broken devices (possibly other than those PCIe devices mentioned above), it only replaces the current implementation of memcpy_toio() to __iowrite64_copy() which is more suited when working with WC buffers. We've tested the patch on a CMB enabled device where we got the same performance as the original implementation, which is expected as using __iowrite64_copy() should really only make a difference with a very stressed system.

Thanks,
Saar

Quoted 15 lines · Stephen Bates

Quoted 9 lines

Quoted 4 lines

Any comments on this patch? I didn't receive any so far.

Please explain why we would want to apply the patch, e.g. without it some devices may be broken, or that on a particular device you get much better performance (including numbers and an explanation of the setup).

Some PCIe devices have requirements on the alignment of PCIe writes into them. Would this patch help with some of those devices?

Stephen

Hi again,

Any other comments?

Quoted 3 lines · Saar Gross

Hi again,

Any other comments?

I still can't see a good explanation on why we'd want/need the patch.

As already mentioned: the current implementation uses memcpy() which is generic in nature, whereas the new implementation uses explicit 64-bit writes when possible.
What else are you looking for?