storvsc: setup 1:1 mapping between hardware queue and CPU queue
Subsystems (3)
PATCH v2 1/1 storvsc: setup 1:1 mapping between hardware queue and CPU queue
From: Long Li <[email protected]>
storvsc doesn't use a dedicated hardware queue for a given CPU queue. When issuing I/O, it selects returning CPU (hardware queue) dynamically based on vmbus channel usage across all channels.
This patch advertises num_possible_cpus() as number of hardware queues. This will have upper layer setup 1:1 mapping between hardware queue and CPU queue and avoid unnecessary locking when issuing I/O.
Changes:
v2: rely on default upper layer function to map queues. (suggested by Ming Lei <[email protected]>)
--- drivers/scsi/storvsc_drv.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/scsi/storvsc_drv.c +1 -2
@@ -1836,8 +1836,7 @@ static int storvsc_probe(struct hv_device *device,/* * Set the number of HW queues we are supporting. */ -if (stor_device->num_sc !=0 ) - host->nr_hw_queues = stor_device->num_sc +1 ; + host->nr_hw_queues =num_possible_cpus ();/* * Set the error handler work queue.
Signature · 1 line
2.17.1
From: Long Li <[email protected]> Sent: Thursday, August 22, 2019 1:42 PM
Quoted 23 lines
storvsc doesn't use a dedicated hardware queue for a given CPU queue. When issuing I/O, it selects returning CPU (hardware queue) dynamically based on vmbus channel usage across all channels.
This patch advertises
num_possible_cpus()as number of hardware queues. This will have upper layer setup 1:1 mapping between hardware queue and CPU queue and avoid unnecessary locking when issuing I/O.Changes:
v2: rely on default upper layer function to map queues. (suggested by Ming Lei <[email protected]>)Signed-off-by Long Li <[email protected]>--- drivers/scsi/storvsc_drv.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)drivers/scsi/storvsc_drv.c +0
-0
drivers/scsi/storvsc_drv.c static int storvsc_probe(struct hv_device *device, · line 1839
Quoted 6 lines
drivers/scsi/storvsc_drv.c +1
-2@@ -1836,8 +1836,7 @@ static int storvsc_probe(struct hv_device *device,/* * Set the number of HW queues we are supporting. */ -if (stor_device->num_sc !=0 ) - host->nr_hw_queues = stor_device->num_sc +1 ; + host->nr_hw_queues =num_possible_cpus ();
For a lot of the VM sizes in Azure, num_possible_cpus() is 128, even if the VM has only 4 or 8 or some other smaller number of vCPUs. So I'm wondering if you really want num_present_cpus() here instead, which would include only the vCPUs that actually exist in the VM.
Michael
Quoted 5 lines
drivers/scsi/storvsc_drv.c +0
-1/* * Set the error handler work queue. -- 2.17.1
Quoted 30 lines
Quoted 30 lines
Quoted 30 lines
Subject: RE: [Patch v2] storvsc: setup 1:1 mapping between hardware queue and CPU queue
From: Long Li <[email protected]> Sent: Thursday, August 22, 2019 1:42 PM
drivers/scsi/storvsc_drv.c static int storvsc_probe(struct hv_device
Quoted 24 lines
storvsc doesn't use a dedicated hardware queue for a given CPU queue. When issuing I/O, it selects returning CPU (hardware queue) dynamically based on vmbus channel usage across all channels.
This patch advertises
num_possible_cpus()as number of hardware queues. This will have upper layer setup 1:1 mapping between hardware queue and CPU queue and avoid unnecessary locking when issuing I/O.Changes:
v2: rely on default upper layer function to map queues. (suggested by Ming Lei
<[email protected]>)Signed-off-by Long Li <[email protected]>--- drivers/scsi/storvsc_drv.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)drivers/scsi/storvsc_drv.c +0
-0@@ -1836,8 +1836,7 @@ static int storvsc_probe(struct hv_device*device,
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
drivers/scsi/storvsc_drv.c static int storvsc_probe(struct hv_device *device, · line 1839
Quoted 6 lines
drivers/scsi/storvsc_drv.c +1
-2/* * Set the number of HW queues we are supporting. */ -if (stor_device->num_sc !=0 ) - host->nr_hw_queues = stor_device->num_sc +1 ; + host->nr_hw_queues =num_possible_cpus ();For a lot of the VM sizes in Azure,
num_possible_cpus()is 128, even if the VM has only 4 or 8 or some other smaller number of vCPUs.
So I'm wondering if you really wantnum_present_cpus()here instead, which would include only the vCPUs that actually exist in the VM.
I think reporting num_possible_cpus() doesn't do more harm or take more resources. Because block layer allocates map for all the possible CPUs.
The actual mapping is done in blk_mq_map_queues(), and it iterates all the possible CPUs. If we report num_present_cpus(), the rest of the CPUs also need to be mapped.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Michael
Quoted 5 lines
drivers/scsi/storvsc_drv.c +0
-1/* * Set the error handler work queue. -- 2.17.1
Quoted 49 lines
Quoted 49 lines
Quoted 49 lines
Subject: RE: [Patch v2] storvsc: setup 1:1 mapping between hardware queue and CPU queue
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Subject: RE: [Patch v2] storvsc: setup 1:1 mapping between hardware queue and CPU queue
From: Long Li <[email protected]> Sent: Thursday, August 22, 2019
1:42 PMQuoted 2 lines
storvsc doesn't use a dedicated hardware queue for a given CPU
queue.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
When issuing I/O, it selects returning CPU (hardware queue) dynamically based on vmbus channel usage across all channels.
This patch advertises
num_possible_cpus()as number of hardware queues. This will have upper layer setup 1:1 mapping between hardware queue and CPU queue and avoid unnecessary locking whenissuing I/O.
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
drivers/scsi/storvsc_drv.c static int storvsc_probe(struct hv_device
Quoted 17 lines
Changes:
v2: rely on default upper layer function to map queues. (suggested by Ming Lei
<[email protected]>)Signed-off-by Long Li <[email protected]>--- drivers/scsi/storvsc_drv.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/scsi/storvsc_drv.cindexb89269120a2d..dfd3b76a4f89100644drivers/scsi/storvsc_drv.c +0
-0@@ -1836,8 +1836,7 @@ static int storvsc_probe(struct hv_device*device,
drivers/scsi/storvsc_drv.c static int storvsc_probe(struct hv_device *device, · line 1839
Quoted 6 lines
drivers/scsi/storvsc_drv.c +1
-2/* * Set the number of HW queues we are supporting. */ -if (stor_device->num_sc !=0 ) - host->nr_hw_queues = stor_device->num_sc +1 ; + host->nr_hw_queues =num_possible_cpus ();For a lot of the VM sizes in Azure,
num_possible_cpus()is 128, even if the VM has only 4 or 8 or some other smaller number of vCPUs. So I'm wondering if you really wantnum_present_cpus()here instead, which would include only the vCPUs that actually exist in the VM.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
I think reporting
num_possible_cpus()doesn't do more harm or take more resources. Because block layer allocates map for all the possible CPUs.The actual mapping is done in
blk_mq_map_queues(), and it iterates all the possible CPUs. If we reportnum_present_cpus(), the rest of the CPUs also need to be mapped.
Actually I get your point, reporting num_present_cpus() will get less number of struct blk_mq_hw_ctx created. So it saves memory.
If we don't plan to support adding/onlining CPUs, we should use num_present_cpus().
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Michael
Quoted 5 lines
drivers/scsi/storvsc_drv.c +0
-1/* * Set the error handler work queue. -- 2.17.1