scsi: storvsc: Spread interrupts when picking a channel for I/O requests
Subsystems (2)
PATCH 1/1 scsi: storvsc: Spread interrupts when picking a channel for I/O requests
Update the algorithm in storvsc_do_io to look for a channel starting with the current CPU + 1 and wrap around (within the current NUMA node). This spreads VMbus interrupts more evenly across CPUs. Previous code always started with first CPU in the current NUMA node, skewing the interrupt load to that CPU.
--- drivers/scsi/storvsc_drv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/scsi/storvsc_drv.c +2 -1
@@ -1310,7 +1310,8 @@ static int storvsc_do_io(struct hv_device *device,*/ cpumask_and (&alloced_mask, &stor_device->alloced_cpus,cpumask_of_node (cpu_to_node (q_num))); -for_each_cpu (tgt_cpu, &alloced_mask) { +for_each_cpu_wrap (tgt_cpu, &alloced_mask, + outgoing_channel->target_cpu +1 ) {if (tgt_cpu != outgoing_channel->target_cpu) { outgoing_channel = stor_device->stor_chns[tgt_cpu];
Signature · 1 line
1.8.3.1
Updated/corrected two email addresses ...
Quoted 36 lines
-----Original Message-----
From: Michael Kelley (EOSG)
Sent: Wednesday, January 24, 2018 2:14 PM
To: KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: Michael Kelley (EOSG) <[email protected]>
Subject: [PATCH 1/1] scsi: storvsc: Spread interrupts when picking a channel for I/O requestsUpdate the algorithm in
storvsc_do_ioto look for a channel starting with the current CPU + 1 and wrap around (within the current NUMA node). This spreads VMbus interrupts more evenly across CPUs. Previous code always started with first CPU in the current NUMA node, skewing the interrupt load to that CPU.Signed-off-by Michael Kelley <[email protected]>--- drivers/scsi/storvsc_drv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/scsi/storvsc_drv.c +2
-2@@ -1310,7 +1310,8 @@ static int storvsc_do_io(struct hv_device *device,*/ cpumask_and (&alloced_mask, &stor_device->alloced_cpus,cpumask_of_node (cpu_to_node (q_num))); -for_each_cpu (tgt_cpu, &alloced_mask) { +for_each_cpu_wrap (tgt_cpu, &alloced_mask, + outgoing_channel->target_cpu +1 ) {if (tgt_cpu != outgoing_channel->target_cpu) { outgoing_channel = stor_device->stor_chns[tgt_cpu]; --1.8.3.1
Quoted 38 lines
Subject: RE: [PATCH 1/1] scsi: storvsc: Spread interrupts when picking a channel for I/O requests
Updated/corrected two email addresses ...
Quoted 28 lines
-----Original Message-----
From: Michael Kelley (EOSG)
Sent: Wednesday, January 24, 2018 2:14 PM
To: KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: Michael Kelley (EOSG) <[email protected]>
Subject: [PATCH 1/1] scsi: storvsc: Spread interrupts when picking a channel for I/O requestsUpdate the algorithm in
storvsc_do_ioto look for a channel starting with the current CPU + 1 and wrap around (within the current NUMA node). This spreads VMbus interrupts more evenly across CPUs. Previous code always started with first CPU in the current NUMA node, skewing the interrupt load to that CPU.Signed-off-by Michael Kelley <[email protected]>--- drivers/scsi/storvsc_drv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/scsi/storvsc_drv.c +0
-0drivers/scsi/storvsc_drv.c static int storvsc_do_io(struct hv_device *device, · line 1313
Quoted 4 lines
drivers/scsi/storvsc_drv.c +0
-0@@ -1310,7 +1310,8 @@ static int storvsc_do_io(struct hv_device *device,*/ cpumask_and(&alloced_mask, &stor_device- alloced_cpus,cpumask_of_node(cpu_to_node(q_num)));
Quoted 3 lines
drivers/scsi/storvsc_drv.c static int storvsc_do_io(struct hv_device *device, · line 1314
Quoted 3 lines
drivers/scsi/storvsc_drv.c +2
-1-for_each_cpu (tgt_cpu, &alloced_mask) { +for_each_cpu_wrap (tgt_cpu, &alloced_mask, + outgoing_channel->target_cpu +1 ) {
Does it work when target_cpu is the last CPU on the system?
Otherwise, looking good.
Quoted 6 lines
drivers/scsi/storvsc_drv.c static int storvsc_do_io(struct hv_device *device, · line 1316
Quoted 1 line
drivers/scsi/storvsc_drv.c +0
-0if (tgt_cpu != outgoing_channel->target_cpu){
Quoted 4 lines
drivers/scsi/storvsc_drv.c +0
-0outgoing_channel =stor_device->stor_chns[tgt_cpu];
--
1.8.3.1
Quoted 47 lines
From: Long Li
Sent: Wednesday, January 31, 2018 12:23 PM
To: Michael Kelley (EOSG) <[email protected]>; KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; James E . J . Bottomley <[email protected]>
Subject: RE: [PATCH 1/1] scsi: storvsc: Spread interrupts when picking a channel for I/O requestsQuoted 38 lines
Subject: RE: [PATCH 1/1] scsi: storvsc: Spread interrupts when picking a channel for I/O requests
Updated/corrected two email addresses ...
Quoted 28 lines
-----Original Message-----
From: Michael Kelley (EOSG)
Sent: Wednesday, January 24, 2018 2:14 PM
To: KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: Michael Kelley (EOSG) <[email protected]>
Subject: [PATCH 1/1] scsi: storvsc: Spread interrupts when picking a channel for I/O requestsUpdate the algorithm in
storvsc_do_ioto look for a channel starting with the current CPU + 1 and wrap around (within the current NUMA node). This spreads VMbus interrupts more evenly across CPUs. Previous code always started with first CPU in the current NUMA node, skewing the interrupt load to that CPU.Signed-off-by Michael Kelley <[email protected]>--- drivers/scsi/storvsc_drv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/scsi/storvsc_drv.c +0
-0drivers/scsi/storvsc_drv.c static int storvsc_do_io(struct hv_device *device, · line 1313
Quoted 4 lines
drivers/scsi/storvsc_drv.c +0
-0@@ -1310,7 +1310,8 @@ static int storvsc_do_io(struct hv_device *device,*/ cpumask_and(&alloced_mask, &stor_device- alloced_cpus,cpumask_of_node(cpu_to_node(q_num)));
Quoted 7 lines
Quoted 3 lines
drivers/scsi/storvsc_drv.c static int storvsc_do_io(struct hv_device *device, · line 1314
Quoted 3 lines
drivers/scsi/storvsc_drv.c +2
-1-for_each_cpu (tgt_cpu, &alloced_mask) { +for_each_cpu_wrap (tgt_cpu, &alloced_mask, + outgoing_channel->target_cpu +1 ) {Does it work when
target_cpuis the last CPU on the system?Otherwise, looking good.
Yes, it works. for_each_cpu_wrap() correctly wraps in the case where the 3rd parameter ('start') is one past the end of the mask. Arguably, we shouldn't rely on that, and should do the wrap to 0 before calling for_each_cpu_wrap().
Quoted 7 lines
Quoted 7 lines
drivers/scsi/storvsc_drv.c static int storvsc_do_io(struct hv_device *device, · line 1316
Quoted 2 lines
drivers/scsi/storvsc_drv.c +0
-0if (tgt_cpu != outgoing_channel->target_cpu){
Quoted 4 lines
drivers/scsi/storvsc_drv.c +0
-0outgoing_channel =stor_device->stor_chns[tgt_cpu];
--
1.8.3.1
_______________________________________________
devel mailing list
[email protected]
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Long Li
Reviewed-byQuoted 38 lines
Subject: RE: [PATCH 1/1] scsi: storvsc: Spread interrupts when picking a channel for I/O requests
Quoted 35 lines
From: Long Li
Sent: Wednesday, January 31, 2018 12:23 PM
To: Michael Kelley (EOSG) <[email protected]>; KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; James E . J . Bottomley <[email protected]>
Subject: RE: [PATCH 1/1] scsi: storvsc: Spread interrupts when picking a channel for I/O requestsQuoted 24 lines
Subject: RE: [PATCH 1/1] scsi: storvsc: Spread interrupts when picking a channel for I/O requests
Updated/corrected two email addresses ...
Quoted 19 lines
-----Original Message-----
From: Michael Kelley (EOSG)
Sent: Wednesday, January 24, 2018 2:14 PM
To: KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: Michael Kelley (EOSG) <[email protected]>
Subject: [PATCH 1/1] scsi: storvsc: Spread interrupts when picking a channel for I/O requestsUpdate the algorithm in
storvsc_do_ioto look for a channel starting with the current CPU + 1 and wrap around (within the current NUMA node). This spreads VMbus interrupts more evenly across CPUs. Previous code always started with first CPU in the current NUMA node, skewing the interrupt load to that CPU.Signed-off-by Michael Kelley <[email protected]>
Quoted 21 lines
Quoted 9 lines
Quoted 9 lines
drivers/scsi/storvsc_drv.c static int storvsc_do_io(struct hv_device
Quoted 9 lines
--- drivers/scsi/storvsc_drv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/drivers/scsi/storvsc_drv.cindex e07907d..f3264c4 100644drivers/scsi/storvsc_drv.c +0
-0@@ -1310,7 +1310,8 @@ static int storvsc_do_io(struct hv_device*device,
Quoted 2 lines
Quoted 2 lines
drivers/scsi/storvsc_drv.c static int storvsc_do_io(struct hv_device · line 1311
Quoted 2 lines
drivers/scsi/storvsc_drv.c +0
-0*/ cpumask_and(&alloced_mask, &stor_device-
alloced_cpus,Quoted 8 lines
Quoted 5 lines
Quoted 1 line
cpumask_of_node(cpu_to_node(q_num)));
drivers/scsi/storvsc_drv.c static int storvsc_do_io(struct hv_device *device, · line 1314
Quoted 3 lines
drivers/scsi/storvsc_drv.c +2
-1-for_each_cpu (tgt_cpu, &alloced_mask) { +for_each_cpu_wrap (tgt_cpu, &alloced_mask, + outgoing_channel->target_cpu +1 ) {Does it work when
target_cpuis the last CPU on the system?
Quoted 14 lines
Quoted 1 line
Otherwise, looking good.
Yes, it works.
for_each_cpu_wrap()correctly wraps in the case where the 3rd parameter ('start') is one past the end of the mask. Arguably, we shouldn't rely on that, and should do the wrap to 0 before callingfor_each_cpu_wrap().Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
if (
tgt_cpu!=outgoing_channel->target_cpu){
Quoted 4 lines
drivers/scsi/storvsc_drv.c +0
-1outgoing_channel = stor_device->stor_chns[tgt_cpu]; --1.8.3.1
_______________________________________________
devel mailing list
[email protected]
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Martin K. Petersen
AppliedMichael,
Quoted 5 lines
Update the algorithm in
storvsc_do_ioto look for a channel starting with the current CPU + 1 and wrap around (within the current NUMA node). This spreads VMbus interrupts more evenly across CPUs. Previous code always started with first CPU in the current NUMA node, skewing the interrupt load to that CPU.
Applied to 4.16/scsi-fixes. Thanks!
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering