RDMA/iw_cxgb4: disable delayed ack by default
Subsystems (2)
Potnuri Bharat Teja
maintainerPATCH 1/1 RDMA/iw_cxgb4: disable delayed ack by default
Receive side delayed ack mode is needed only for certain area networks/ connections. Therefore disable it by default.
--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/infiniband/hw/cxgb4/cm.c +2 -2
@@ -77,9 +77,9 @@ static int enable_ecn;module_param (enable_ecn, int,0644 );MODULE_PARM_DESC (enable_ecn,"Enable ECN (default=0/disabled)" ); -static int dack_mode =1 ; +static int dack_mode;module_param (dack_mode, int,0644 ); -MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=1)" ); +MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=0)" );uint c4iw_max_read_depth =32 ;module_param (c4iw_max_read_depth, int,0644 );
Signature · 1 line
2.24.0
Potnuri Bharat Teja
maintainerMissed rdma-next subject prefix. Resending.
-----Original Message-----
From: Potnuri Bharat Teja <[email protected]>
Sent: Wednesday, September 9, 2020 7:17 PM
To: [email protected]; [email protected]
Cc: [email protected]; Potnuri Bharat Teja <[email protected]>
Subject: [PATCH] RDMA/iw_cxgb4: disable delayed ack by default
Receive side delayed ack mode is needed only for certain area networks/ connections. Therefore disable it by default.
--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/infiniband/hw/cxgb4/cm.c +2 -3
@@ -77,9 +77,9 @@ static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); -static int dack_mode =1 ; +static int dack_mode;module_param (dack_mode, int,0644 ); -MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=1)" ); +MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=0)" );uint c4iw_max_read_depth =32 ;module_param (c4iw_max_read_depth, int,0644 ); --
2.24.0
Quoted 21 lines · Potnuri Bharat Teja
Missed rdma-next subject prefix. Resending.
-----Original Message-----
From: Potnuri Bharat Teja <[email protected]>
Sent: Wednesday, September 9, 2020 7:17 PM
To: [email protected]; [email protected]
Cc: [email protected]; Potnuri Bharat Teja <[email protected]>
Subject: [PATCH] RDMA/iw_cxgb4: disable delayed ack by defaultReceive side delayed ack mode is needed only for certain area networks/ connections. Therefore disable it by default.
Signed-off-by Potnuri Bharat Teja <[email protected]>--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/infiniband/hw/cxgb4/cm.c +0
-0
drivers/infiniband/hw/cxgb4/cm.c static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); · line 80
Quoted 6 lines
drivers/infiniband/hw/cxgb4/cm.c +2
-2@@ -77,9 +77,9 @@ static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); -static int dack_mode =1 ; +static int dack_mode;module_param (dack_mode, int,0644 ); -MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=1)" ); +MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=0)" );
Are you sure that this doesn't break user scripts?
Thanks
Quoted 5 lines
drivers/infiniband/hw/cxgb4/cm.c +0
-1uint c4iw_max_read_depth =32 ;module_param (c4iw_max_read_depth, int,0644 ); --2.24.0
Potnuri Bharat Teja
maintainerQuoted 15 lines
Quoted 15 lines
Subject: [PATCH] RDMA/iw_cxgb4: disable delayed ack by default
Receive side delayed ack mode is needed only for certain area networks/ connections. Therefore disable it by default.
Signed-off-by Potnuri Bharat Teja <[email protected]>--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/infiniband/hw/cxgb4/cm.c +0
-0@@ -77,9 +77,9 @@ static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)");
Quoted 7 lines
drivers/infiniband/hw/cxgb4/cm.c static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); · line 80
Quoted 5 lines
drivers/infiniband/hw/cxgb4/cm.c +2
-2-static int dack_mode =1 ; +static int dack_mode;module_param (dack_mode, int,0644 ); -MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=1)" ); +MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=0)" );Are you sure that this doesn't break user scripts?
Yes, I am sure. This does not interfere with user/kernel RDMA functionalities.
Thanks,
Bharat.
Quoted 8 lines
Thanks
Quoted 5 lines
drivers/infiniband/hw/cxgb4/cm.c +0
-1uint c4iw_max_read_depth =32 ;module_param (c4iw_max_read_depth, int,0644 ); --2.24.0
Quoted 15 lines · Potnuri Bharat Teja
Quoted 15 lines
Quoted 15 lines
Subject: [PATCH] RDMA/iw_cxgb4: disable delayed ack by default
Receive side delayed ack mode is needed only for certain area networks/ connections. Therefore disable it by default.
Signed-off-by Potnuri Bharat Teja <[email protected]>--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/infiniband/hw/cxgb4/cm.c +0
-0@@ -77,9 +77,9 @@ static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)");
Quoted 8 lines
Quoted 7 lines
drivers/infiniband/hw/cxgb4/cm.c static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); · line 80
Quoted 5 lines
drivers/infiniband/hw/cxgb4/cm.c +2
-2-static int dack_mode =1 ; +static int dack_mode;module_param (dack_mode, int,0644 ); -MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=1)" ); +MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=0)" );Are you sure that this doesn't break user scripts?
Yes, I am sure. This does not interfere with user/kernel RDMA functionalities.
How is it possible?
Before this change user that did "modprobe iw_cxgb4" had delayed mode enabled, after this he will need to issue "modprobe iw_cxgb4 dack_mode=1"
https://github.com/linux-rdma/rdma-core/blob/master/kernel-boot/rdma-hw-modules.rules#L12
Thanks
Quoted 11 lines
Thanks,
Bharat.Quoted 8 lines
Thanks
Quoted 5 lines
drivers/infiniband/hw/cxgb4/cm.c +0
-1uint c4iw_max_read_depth =32 ;module_param (c4iw_max_read_depth, int,0644 ); --2.24.0
Potnuri Bharat Teja
maintainerQuoted 23 lines · Leon Romanovsky
Quoted 22 lines · Potnuri Bharat Teja
Quoted 22 lines
Quoted 14 lines
Subject: [PATCH] RDMA/iw_cxgb4: disable delayed ack by default
Receive side delayed ack mode is needed only for certain area networks/ connections. Therefore disable it by default.
Signed-off-by Potnuri Bharat Teja <[email protected]>--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/infiniband/hw/cxgb4/cm.c +0
-0drivers/infiniband/hw/cxgb4/cm.c static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); · line 80
Quoted 6 lines
drivers/infiniband/hw/cxgb4/cm.c +2
-2@@ -77,9 +77,9 @@ static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); -static int dack_mode =1 ; +static int dack_mode;module_param (dack_mode, int,0644 ); -MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=1)" ); +MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=0)" );Are you sure that this doesn't break user scripts?
Quoted 6 lines
Quoted 1 line
Yes, I am sure. This does not interfere with user/kernel RDMA functionalities.
How is it possible?
Before this change user that did "modprobeiw_cxgb4" had delayed mode enabled, after this he will need to issue "modprobeiw_cxgb4dack_mode=1"
Sorry I didnt get it right earlier. Yes now if user wants delayed ack mode, user can issue "modprobe iw_cxgb4 dack_mode=1" or change it dynamically via /sys/module/<>/paramters.
Chelsio adapters for better performance in most cases needs delayed ack mode to be disabled. This change somehow got missed upstream all this while. In fewer explicit cases, user can enable delayed ack mode where it is needed.
Quoted 15 lines
https://github.com/linux-rdma/rdma-core/blob/master/kernel-boot/rdma-hw-modules.rules#L12
Thanks
Quoted 11 lines
Thanks,
Bharat.Quoted 8 lines
Thanks
Quoted 5 lines
drivers/infiniband/hw/cxgb4/cm.c +0
-1uint c4iw_max_read_depth =32 ;module_param (c4iw_max_read_depth, int,0644 ); --2.24.0
Quoted 30 lines · Potnuri Bharat Teja
Quoted 23 lines · Leon Romanovsky
Quoted 22 lines · Potnuri Bharat Teja
Quoted 22 lines
Quoted 14 lines
Subject: [PATCH] RDMA/iw_cxgb4: disable delayed ack by default
Receive side delayed ack mode is needed only for certain area networks/ connections. Therefore disable it by default.
Signed-off-by Potnuri Bharat Teja <[email protected]>--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/infiniband/hw/cxgb4/cm.c +0
-0drivers/infiniband/hw/cxgb4/cm.c static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); · line 80
Quoted 6 lines
drivers/infiniband/hw/cxgb4/cm.c +2
-2@@ -77,9 +77,9 @@ static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); -static int dack_mode =1 ; +static int dack_mode;module_param (dack_mode, int,0644 ); -MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=1)" ); +MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=0)" );Are you sure that this doesn't break user scripts?
Quoted 6 lines
Quoted 1 line
Yes, I am sure. This does not interfere with user/kernel RDMA functionalities.
How is it possible?
Before this change user that did "modprobeiw_cxgb4" had delayed mode enabled, after this he will need to issue "modprobeiw_cxgb4dack_mode=1"
Quoted 6 lines
Sorry I didnt get it right earlier. Yes now if user wants delayed ack mode, user can issue "modprobe
iw_cxgb4dack_mode=1" or change it dynamically via /sys/module/<>/paramters.
Chelsio adapters for better performance in most cases needs delayed ack mode to be disabled. This change somehow got missed upstream all this while. In fewer explicit cases, user can enable delayed ack mode where it is needed.
So why doesn't rdma-core have dack_mode=0 in kernel-boot scripts?
Thanks
Potnuri Bharat Teja
maintainerQuoted 31 lines · Leon Romanovsky
Quoted 30 lines · Potnuri Bharat Teja
Quoted 29 lines · Leon Romanovsky
Quoted 15 lines · Potnuri Bharat Teja
Quoted 15 lines
Quoted 15 lines
Subject: [PATCH] RDMA/iw_cxgb4: disable delayed ack by default
Receive side delayed ack mode is needed only for certain area networks/ connections. Therefore disable it by default.
Signed-off-by Potnuri Bharat Teja <[email protected]>--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/infiniband/hw/cxgb4/cm.c +0
-0@@ -77,9 +77,9 @@ static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)");Quoted 8 lines
Quoted 7 lines
drivers/infiniband/hw/cxgb4/cm.c static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); · line 80
Quoted 5 lines
drivers/infiniband/hw/cxgb4/cm.c +2
-2-static int dack_mode =1 ; +static int dack_mode;module_param (dack_mode, int,0644 ); -MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=1)" ); +MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=0)" );Are you sure that this doesn't break user scripts?
Yes, I am sure. This does not interfere with user/kernel RDMA functionalities.
How is it possible?
Before this change user that did "modprobeiw_cxgb4" had delayed mode enabled, after this he will need to issue "modprobeiw_cxgb4dack_mode=1"
Quoted 9 lines
Quoted 6 lines
Sorry I didnt get it right earlier. Yes now if user wants delayed ack mode, user can issue "modprobe
iw_cxgb4dack_mode=1" or change it dynamically via /sys/module/<>/paramters.
Chelsio adapters for better performance in most cases needs delayed ack mode to be disabled. This change somehow got missed upstream all this while. In fewer explicit cases, user can enable delayed ack mode where it is needed.So why doesn't rdma-core have dack_mode=0 in kernel-boot scripts?
Do you mean why dont I simply change rdma-hw-modules.rules instead?
I could do that if this is a user mode only setting. But this is needed for kernel mode too where in some cases rdma-core may not be installed.
Thanks.
Quoted 39 lines · Potnuri Bharat Teja
Quoted 38 lines · Leon Romanovsky
Quoted 36 lines · Potnuri Bharat Teja
Quoted 23 lines · Leon Romanovsky
Quoted 22 lines · Potnuri Bharat Teja
Quoted 22 lines
Quoted 14 lines
Subject: [PATCH] RDMA/iw_cxgb4: disable delayed ack by default
Receive side delayed ack mode is needed only for certain area networks/ connections. Therefore disable it by default.
Signed-off-by Potnuri Bharat Teja <[email protected]>--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/infiniband/hw/cxgb4/cm.c +0
-0drivers/infiniband/hw/cxgb4/cm.c static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); · line 80
Quoted 6 lines
drivers/infiniband/hw/cxgb4/cm.c +2
-2@@ -77,9 +77,9 @@ static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); -static int dack_mode =1 ; +static int dack_mode;module_param (dack_mode, int,0644 ); -MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=1)" ); +MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=0)" );Are you sure that this doesn't break user scripts?
Quoted 6 lines
Quoted 1 line
Yes, I am sure. This does not interfere with user/kernel RDMA functionalities.
How is it possible?
Before this change user that did "modprobeiw_cxgb4" had delayed mode enabled, after this he will need to issue "modprobeiw_cxgb4dack_mode=1"Sorry I didnt get it right earlier. Yes now if user wants delayed ack mode, user can issue "modprobe
iw_cxgb4dack_mode=1" or change it dynamically via /sys/module/<>/paramters.
Chelsio adapters for better performance in most cases needs delayed ack mode to be disabled. This change somehow got missed upstream all this while. In fewer explicit cases, user can enable delayed ack mode where it is needed.
Quoted 5 lines
Quoted 2 lines
So why doesn't rdma-core have dack_mode=0 in kernel-boot scripts?
Do you mean why dont I simply change rdma-hw-modules.rules instead?
I could do that if this is a user mode only setting. But this is needed for kernel mode too where in some cases rdma-core may not be installed.
No, I'm asking why if "dack_mode=0" is better and preferable way to operate, it was never set in rdma-core.
I'm not saying that your kernel patch is wrong, just trying to get sense on implications for the rdma-core.
Thanks
Quoted 3 lines
Thanks.
Potnuri Bharat Teja
maintainerQuoted 40 lines · Leon Romanovsky
Quoted 39 lines · Potnuri Bharat Teja
Quoted 38 lines · Leon Romanovsky
Quoted 36 lines · Potnuri Bharat Teja
Quoted 23 lines · Leon Romanovsky
Quoted 22 lines · Potnuri Bharat Teja
Quoted 22 lines
Quoted 14 lines
Subject: [PATCH] RDMA/iw_cxgb4: disable delayed ack by default
Receive side delayed ack mode is needed only for certain area networks/ connections. Therefore disable it by default.
Signed-off-by Potnuri Bharat Teja <[email protected]>--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/infiniband/hw/cxgb4/cm.c +0
-0drivers/infiniband/hw/cxgb4/cm.c static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); · line 80
Quoted 6 lines
drivers/infiniband/hw/cxgb4/cm.c +2
-2@@ -77,9 +77,9 @@ static int enable_ecn; module_param(enable_ecn, int, 0644); MODULE_PARM_DESC(enable_ecn, "Enable ECN (default=0/disabled)"); -static int dack_mode =1 ; +static int dack_mode;module_param (dack_mode, int,0644 ); -MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=1)" ); +MODULE_PARM_DESC (dack_mode,"Delayed ack mode (default=0)" );Are you sure that this doesn't break user scripts?
Quoted 6 lines
Quoted 1 line
Yes, I am sure. This does not interfere with user/kernel RDMA functionalities.
How is it possible?
Before this change user that did "modprobeiw_cxgb4" had delayed mode enabled, after this he will need to issue "modprobeiw_cxgb4dack_mode=1"Sorry I didnt get it right earlier. Yes now if user wants delayed ack mode, user can issue "modprobe
iw_cxgb4dack_mode=1" or change it dynamically via /sys/module/<>/paramters.
Chelsio adapters for better performance in most cases needs delayed ack mode to be disabled. This change somehow got missed upstream all this while. In fewer explicit cases, user can enable delayed ack mode where it is needed.
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines
So why doesn't rdma-core have dack_mode=0 in kernel-boot scripts?
Do you mean why dont I simply change rdma-hw-modules.rules instead?
I could do that if this is a user mode only setting. But this is needed for kernel mode too where in some cases rdma-core may not be installed.No, I'm asking why if "dack_mode=0" is better and preferable way to operate, it was never set in rdma-core.
Found this only recently that it was wrongly enabled by default which affects severely the performance of some of our adapters with low HW resources for larger IO sizes. I could have mentioned this in commit description but I thought it's a trivial change code wise.
Quoted 3 lines
I'm not saying that your kernel patch is wrong, just trying to get sense on implications for the rdma-core.
ok. yes rdma-hw-modules.rules provides a cleaner way. I am still used to setting module params via /sys/module/<>/parameters/*.
Thanks for the review.