Controller Memory Buffer
Jonas Pfefferle
Hi @all,
Is there any NVMe drive available which supports the optional "Controller Memory Buffer" feature introduced in NVMe 1.2?
Thanks & Regards,
Jonas
Stephen Bates
Jonas
There are some Intel drives that include a small (a few MB) CMB that supports SQS only. Both Eideticom and Everspin have NVMe devices that have CMBs that support the WDS and RDS modes as well as SQS.
I know that other vendors are pretty actively looking at this so I’d expect more NVMe SSDs with a variety of CMB modes later this year. I’d suggest pinging those companies and getting roadmap updates from them to understand their timelines.
Cheers
Stephen
Quoted 13 lines · Jonas Pfefferle
Hi @all,
Is there any NVMe drive available which supports the optional "Controller Memory Buffer" feature introduced in NVMe 1.2?
Thanks & Regards,
Jonas_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
Jonas Pfefferle
Hi Stephen,
Thanks for the hints. Do you happen to know which Intel drives these are? I have a few Intel drives: DC P3600/3700 and 900p/P4800X but neither of those devices have CMB.
I'm experimenting with Mellanox ConnectX-5 target offloading so it would be nice to see the effects of having SQ/CQ and data staging on the device. From the experiments I've done so far it looks like the latencies are higher with the offloaded target compared to SPDK or the Kernel target. I assume this is partially due to the additional PCIe round trip times.
Thanks & Cheers,
Jonas
Quoted 34 lines · On Sat, 10 Mar 2018 21:11:01 +0000 "Stephen Bates" <sbates(a)raithlin.com>…
Jonas
There are some Intel drives that include a small (a few MB) CMB that supports SQS only. Both Eideticom and Everspin have NVMe devices that have CMBs that support the WDS and RDS modes as well as SQS.
I know that other vendors are pretty actively looking at this so I’d expect more NVMe SSDs with a variety of CMB modes later this year. I’d suggest pinging those companies and getting roadmap updates from them to understand their timelines.
Cheers
Stephen
Quoted 8 lines
On Mar 9, 2018, at 9:15 AM, Jonas Pfefferle <pepperjo(a)japf.ch> wrote:
Hi @all,
Is there any NVMe drive available which supports the optional "Controller Memory Buffer" feature introduced in NVMe 1.2?
Quoted 7 lines
Thanks & Regards,
Jonas_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
Stephen Bates
Quoted 1 line
Thanks for the hints. Do you happen to know which Intel drives these are?
I don't have model numbers. Contact your Intel rep for those.
Quoted 1 line
I'm experimenting with Mellanox ConnectX-5 target offloading
Cool. We are playing with that too! In fact we have a Linux kernel fork that mashes up Max's offload driver with our p2pdma framework for CMBs here - https://github.com/lsgunth/linux/tree/max-mlnx-offload-p2pdma
Quoted 1 line
the latencies are higher
Do you have a PCIe switch or are the ConnectX-5 and NVMe SSD connected via the root complex on the (I presume Intel) CPU? If the later then I suspect some of your latency is coming from the non-performant MemWr TLP path that connects the root ports together in the root complex.
Stephen
PS Since we have completely diverged from SPDK at this point I suggest taking this off list ;-)...
Harris
On 3/11/18, 4:31 AM, "SPDK on behalf of Jonas Pfefferle" <spdk-bounces(a)lists.01.org on behalf of pepperjo(a)japf.ch> wrote:
Hi Stephen,
Thanks for the hints. Do you happen to know which Intel drives these are? I have a few Intel drives: DC P3600/3700 and 900p/P4800X but neither of those
devices have CMB.
The Intel D3600/3700 (dual-port) SSDs have the SQS-only CMB that Stephen mentioned.
-Jim