Skip to contentSkip to conversation

NVMe: Correctly handle IOCTL_SUBMIT_IO when cpus > online queues

PATCH New Sam Bradshaw 2 participants 3 messages Jul 25, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 NVMe: Correctly handle IOCTL_SUBMIT_IO when cpus > online queues

nvme_submit_io_cmd() use smp_processor_id() to pick an IO queue index. This patch fixes the case where there are more cpus from which the ioctl call can originate than online queues, which can happen when a device supports or was allocated fewer interrupt vectors than exist cpu cores.

Signed-off-by Sam Bradshaw <sbradshaw at micron.com>

---

drivers/block/nvme-core.c +3 -2
index 28aec2d..5171753 100644
@@ -979,8 +979,9 @@ int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
 int nvme_submit_io_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
 								u32 *result)
 {
-	return nvme_submit_sync_cmd(dev, smp_processor_id() + 1, cmd, result,
-							NVME_IO_TIMEOUT);
+	return nvme_submit_sync_cmd(dev, (smp_processor_id() %
+						(dev->online_queues - 1)) + 1,
+						 cmd, result, NVME_IO_TIMEOUT);
 }
 
 static int nvme_submit_admin_cmd_async(struct nvme_dev *dev,
#1 lore raw

PATCH 1/1 NVMe: Correctly handle IOCTL_SUBMIT_IO when cpus > online queues

Quoted 4 lines · Sam Bradshaw

nvme_submit_io_cmd() use smp_processor_id() to pick an IO queue index. This patch fixes the case where there are more cpus from which the ioctl call can originate than online queues, which can happen when a device supports or was allocated fewer interrupt vectors than exist cpu cores.

Oh you're right, but I think the idea was to use the queue associated with the cpu, though the code was clearly broken before.

- return nvme_submit_sync_cmd(dev, smp_processor_id() + 1, cmd, result,
+ return nvme_submit_sync_cmd(dev, this_cpu_read(*dev->io_queues), cmd, result,
NVME_IO_TIMEOUT);

Quoted 19 lines
Signed-off-by Sam Bradshaw <sbradshaw at micron.com>

---

drivers/block/nvme-core.c +3 -2
index 28aec2d..5171753 100644
@@ -979,8 +979,9 @@ int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
 int nvme_submit_io_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
 								u32 *result)
 {
-	return nvme_submit_sync_cmd(dev, smp_processor_id() + 1, cmd, result,
-							NVME_IO_TIMEOUT);
+	return nvme_submit_sync_cmd(dev, (smp_processor_id() %
+						(dev->online_queues - 1)) + 1,
+						 cmd, result, NVME_IO_TIMEOUT);
 }
 
 static int nvme_submit_admin_cmd_async(struct nvme_dev *dev,
#2 lore raw

PATCH 1/1 NVMe: Correctly handle IOCTL_SUBMIT_IO when cpus > online queues

Quoted 25 lines

-----Original Message-----
From: Keith Busch [mailto:keith.busch at intel.com]
Sent: Friday, July 25, 2014 4:41 PM
To: Sam Bradshaw (sbradshaw)
Cc: linux-nvme at lists.infradead.org
Subject: Re: [PATCH] NVMe: Correctly handle IOCTL_SUBMIT_IO when cpus > online queues

Quoted 1 line · Sam Bradshaw

nvme_submit_io_cmd() use smp_processor_id() to pick an IO queue index.

This patch

Quoted 1 line

fixes the case where there are more cpus from which the ioctl call

can originate

Quoted 1 line

than online queues, which can happen when a device supports or was

allocated fewer

Quoted 1 line

interrupt vectors than exist cpu cores.

Oh you're right, but I think the idea was to use the queue associated with the cpu, though the code was clearly broken before.

(quoted diff) +0 -1
-	return nvme_submit_sync_cmd(dev, smp_processor_id() + 1, cmd,

result,

(quoted diff) +1 -0
+	return nvme_submit_sync_cmd(dev, this_cpu_read(*dev->io_queues),
Quoted 2 lines

cmd, result,
NVME_IO_TIMEOUT);

Agreed. The above is more consistent with other queue selection paths.

Do you want to send an updated patch or should I?

-Sam