RDMA/bnxt_re: Congestion control settings using debugfs hook
Subsystems (2)
Selvin Xavier
maintainerPATCH v2 1/1 RDMA/bnxt_re: Congestion control settings using debugfs hook
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Currently supporting only GEN 0 version of the parameters. Reading these files queries the firmware and report the values currently programmed. Writing to the files sends commands that update the congestion control settings
---
v1 -> v2:
Addressed Leon's comments
- rename debugfs file "g" to "run_avg_weight_g"
- Fix the indentation errors
- Remove the unnecessary error message during the read entry point
- Fix the return value
drivers/infiniband/hw/bnxt_re/bnxt_re.h | 2 + drivers/infiniband/hw/bnxt_re/debugfs.c | 212 +++++++++++++++++++++++++++++++- drivers/infiniband/hw/bnxt_re/debugfs.h | 15 +++ 3 files changed, 228 insertions(+), 1 deletion(-)
drivers/infiniband/hw/bnxt_re/bnxt_re.h +2 -0
@@ -232,6 +232,8 @@ struct bnxt_re_dev {unsigned long event_bitmap;struct bnxt_qplib_cc_param cc_param;struct workqueue_struct *dcb_wq; +struct dentry *cc_config; +struct bnxt_re_dbg_cc_config_params *cc_config_params; };#define to_bnxt_re_dev (ptr, member) \
drivers/infiniband/hw/bnxt_re/debugfs.c +211 -1
236 lines not on this page · raw message
drivers/infiniband/hw/bnxt_re/debugfs.h +15 -0
@@ -18,4 +18,19 @@ void bnxt_re_debugfs_rem_pdev(struct bnxt_re_dev *rdev);void bnxt_re_register_debugfs (void );void bnxt_re_unregister_debugfs (void ); +#define CC_CONFIG_GEN_EXT (x, y) (((x) << 16) | (y)) +#define CC_CONFIG_GEN0_EXT0 CC_CONFIG_GEN_EXT(0, 0) + +#define BNXT_RE_CC_PARAM_GEN0 __ffs(CMDQ_MODIFY_ROCE_CC_MODIFY_MASK_INACTIVITY_CP) + +struct bnxt_re_cc_param { +struct bnxt_re_dev *rdev; +struct dentry *dentry; +u32 offset; +u8 cc_gen; +}; + +struct bnxt_re_dbg_cc_config_params { +struct bnxt_re_cc_param gen0_parms[BNXT_RE_CC_PARAM_GEN0 ]; +};#endif
Signature · 1 line
2.5.5
Quoted 3 lines · Selvin Xavier
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Should something like this be in debugfs though?
bnxt_qplib_modify_cc() is just sending a firmware command, seems like this should belong to fwctl?
Additionally there may be interest in some common way to control CC for RDMA..
Jason
Selvin Xavier
maintainerQuoted 7 lines · Jason Gunthorpe
Quoted 3 lines · Selvin Xavier
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Should something like this be in debugfs though?
Since these are Broadcom specific parameters, i thought its better to be under debugfs. Also I took the reference of a similar implementation in mlx5.
Quoted 3 lines
bnxt_qplib_modify_cc()is just sending a firmware command, seems like this should belong to fwctl?
Agree. We can move to this model once fwctl is accepted. For now, it is important for us to support our customers with an immediate solution. Customers are asking for this support.
Quoted 3 lines
Additionally there may be interest in some common way to control CC for RDMA..
Do you think there are common parameters for multiple vendors here? I think enable/disable is an option.
Quoted 2 lines
Jason
Quoted 11 lines · Selvin Xavier
Quoted 7 lines · Jason Gunthorpe
Quoted 3 lines · Selvin Xavier
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Should something like this be in debugfs though?
Since these are Broadcom specific parameters, i thought its better to be under debugfs. Also I took the reference of a similar implementation in mlx5.
debugfs is disabled in a lot of deployments, it is a big part of why we are doing fwctl. If you know it works for you cases, debugfs is pretty open ended..
Quoted 5 lines
Quoted 2 lines
bnxt_qplib_modify_cc()is just sending a firmware command, seems like this should belong to fwctl?Agree. We can move to this model once fwctl is accepted. For now, it is important for us to support our customers with an immediate solution. Customers are asking for this support.
Well, fwctl can be accepted when you guys come through with an implementation :)
Quoted 5 lines
Quoted 2 lines
Additionally there may be interest in some common way to control CC for RDMA..
Do you think there are common parameters for multiple vendors here? I think enable/disable is an option.
I haven't seen much commonality here, every site seems to have their own totally different stuff right now.
Jason
Selvin Xavier
maintainerQuoted 9 lines · Jason Gunthorpe
Quoted 7 lines · Selvin Xavier
Quoted 6 lines · Jason Gunthorpe
Quoted 3 lines · Selvin Xavier
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Should something like this be in debugfs though?
Since these are Broadcom specific parameters, i thought its better to be under debugfs. Also I took the reference of a similar implementation in mlx5.
debugfs is disabled in a lot of deployments, it is a big part of why we are doing fwctl. If you know it works for you cases, debugfs is pretty open ended..
The main use case for this debugfs support is for evaluation customers and the tuning for their network. So debugfs should be okay.
Quoted 20 lines
Quoted 6 lines
Quoted 3 lines
bnxt_qplib_modify_cc()is just sending a firmware command, seems like this should belong to fwctl?Agree. We can move to this model once fwctl is accepted. For now, it is important for us to support our customers with an immediate solution. Customers are asking for this support.
Well, fwctl can be accepted when you guys come through with an implementation :)
Quoted 5 lines
Quoted 2 lines
Additionally there may be interest in some common way to control CC for RDMA..
Do you think there are common parameters for multiple vendors here? I think enable/disable is an option.
I haven't seen much commonality here, every site seems to have their own totally different stuff right now.
Jason
Quoted 15 lines · Selvin Xavier
Quoted 14 lines · Jason Gunthorpe
Quoted 11 lines · Selvin Xavier
Quoted 7 lines · Jason Gunthorpe
Quoted 3 lines · Selvin Xavier
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Should something like this be in debugfs though?
Since these are Broadcom specific parameters, i thought its better to be under debugfs. Also I took the reference of a similar implementation in mlx5.
Quoted 5 lines
Quoted 3 lines
debugfs is disabled in a lot of deployments, it is a big part of why we are doing fwctl. If you know it works for you cases, debugfs is pretty open ended..
The main use case for this debugfs support is for evaluation customers and the tuning for their network. So debugfs should be okay.
In my experience it makes no difference, if the customer is using secure boot then they are always using secure boot expect in small lab systems perhaps.
Are you certain this is useful not just "should be okay" ?
Jason
Selvin Xavier
maintainerQuoted 20 lines · Jason Gunthorpe
Quoted 18 lines · Selvin Xavier
Quoted 17 lines · Jason Gunthorpe
Quoted 11 lines · Selvin Xavier
Quoted 7 lines · Jason Gunthorpe
Quoted 3 lines · Selvin Xavier
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Should something like this be in debugfs though?
Since these are Broadcom specific parameters, i thought its better to be under debugfs. Also I took the reference of a similar implementation in mlx5.
debugfs is disabled in a lot of deployments, it is a big part of why we are doing fwctl. If you know it works for you cases, debugfs is pretty open ended..
Quoted 8 lines
Quoted 2 lines
The main use case for this debugfs support is for evaluation customers and the tuning for their network. So debugfs should be okay.
In my experience it makes no difference, if the customer is using secure boot then they are always using secure boot expect in small lab systems perhaps.
Are you certain this is useful not just "should be okay" ?
Yes. It is useful based on the request we have seen from our customers.
Quoted 2 lines
Jason
Leon Romanovsky
AppliedQuoted 10 lines · Selvin Xavier
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Currently supporting only GEN 0 version of the parameters. Reading these files queries the firmware and report the values currently programmed. Writing to the files sends commands that update the congestion control settings
[...]
Applied, thanks!
[1/1] RDMA/bnxt_re: Congestion control settings using debugfs hook
https://git.kernel.org/rdma/rdma/c/a3c71713d9548f
Best regards,
Signature · 1 line
Leon Romanovsky <[email protected]>
Quoted 18 lines · Selvin Xavier
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Currently supporting only GEN 0 version of the parameters. Reading these files queries the firmware and report the values currently programmed. Writing to the files sends commands that update the congestion control settings
Signed-off-by Selvin Xavier <[email protected]>---
v1 -> v2:
Addressed Leon's comments
- rename debugfs file "g" to "run_avg_weight_g"
- Fix the indentation errors
- Remove the unnecessary error message during the read entry point
- Fix the return value
Quoted 4 lines
drivers/infiniband/hw/bnxt_re/bnxt_re.h | 2 + drivers/infiniband/hw/bnxt_re/debugfs.c | 212 +++++++++++++++++++++++++++++++- drivers/infiniband/hw/bnxt_re/debugfs.h | 15 +++ 3 files changed, 228 insertions(+), 1 deletion(-)
We don't have strong policy of what can/can't be in debugfs, so let's take it.
Thanks
Quoted 23 lines · Selvin Xavier
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Currently supporting only GEN 0 version of the parameters. Reading these files queries the firmware and report the values currently programmed. Writing to the files sends commands that update the congestion control settings
Signed-off-by Selvin Xavier <[email protected]>---
v1 -> v2:
Addressed Leon's comments
- rename debugfs file "g" to "run_avg_weight_g"
- Fix the indentation errors
- Remove the unnecessary error message during the read entry point
- Fix the return valuedrivers/infiniband/hw/bnxt_re/bnxt_re.h | 2 + drivers/infiniband/hw/bnxt_re/debugfs.c | 212 +++++++++++++++++++++++++++++++- drivers/infiniband/hw/bnxt_re/debugfs.h | 15 +++ 3 files changed, 228 insertions(+), 1 deletion(-)
...
Quoted 17 lines
drivers/infiniband/hw/bnxt_re/debugfs.c +10
-0+static const struct file_operations bnxt_re_cc_config_ops = { + .owner =THIS_MODULE , + .open = simple_open, + .read = bnxt_re_cc_config_get, + .write = bnxt_re_cc_config_set, +}; +void bnxt_re_debugfs_add_pdev (struct bnxt_re_dev *rdev) {struct pci_dev *pdev = rdev->en_dev->pdev; +struct bnxt_re_dbg_cc_config_params *cc_params; +int i; rdev->dbg_root =debugfs_create_dir (dev_name (&pdev->dev), bnxt_re_debugfs_root); rdev->qp_debugfs =debugfs_create_dir ("QPs" , rdev->dbg_root); + rdev->cc_config =debugfs_create_dir ("cc_config" , rdev->dbg_root);
drivers/infiniband/hw/bnxt_re/debugfs.c void bnxt_re_debug_rem_qpinfo(struct bnxt_re_dev *rdev, struct bnxt_re_qp *qp) · line 325
Quoted 10 lines
drivers/infiniband/hw/bnxt_re/debugfs.c +10
-0+ + rdev->cc_config_params =kzalloc (sizeof (*cc_params),GFP_KERNEL ); + +for (i =0 ; i <BNXT_RE_CC_PARAM_GEN0 ; i++) { +struct bnxt_re_cc_param *tmp_params = &rdev->cc_config_params->gen0_parms[i]; + + tmp_params->rdev = rdev; + tmp_params->offset = i; + tmp_params->cc_gen =CC_CONFIG_GEN0_EXT0 ; + tmp_params->dentry = debugfs_create_file(bnxt_re_cc_gen0_name[i], 0400,
Write operation doesn't seem to work?
Quoted 5 lines
drivers/infiniband/hw/bnxt_re/debugfs.c +3
-0+ rdev->cc_config, tmp_params, + &bnxt_re_cc_config_ops); + } }
Selvin Xavier
maintainerQuoted 44 lines · Chengchang Tang
Quoted 23 lines · Selvin Xavier
Implements routines to set and get different settings of the congestion control. This will enable the users to modify the settings according to their network.
Currently supporting only GEN 0 version of the parameters. Reading these files queries the firmware and report the values currently programmed. Writing to the files sends commands that update the congestion control settings
Signed-off-by Selvin Xavier <[email protected]>---
v1 -> v2:
Addressed Leon's comments
- rename debugfs file "g" to "run_avg_weight_g"
- Fix the indentation errors
- Remove the unnecessary error message during the read entry point
- Fix the return valuedrivers/infiniband/hw/bnxt_re/bnxt_re.h | 2 + drivers/infiniband/hw/bnxt_re/debugfs.c | 212 +++++++++++++++++++++++++++++++- drivers/infiniband/hw/bnxt_re/debugfs.h | 15 +++ 3 files changed, 228 insertions(+), 1 deletion(-)...
Quoted 15 lines
drivers/infiniband/hw/bnxt_re/debugfs.c +9
-0+static const struct file_operations bnxt_re_cc_config_ops = { + .owner =THIS_MODULE , + .open = simple_open, + .read = bnxt_re_cc_config_get, + .write = bnxt_re_cc_config_set, +}; +void bnxt_re_debugfs_add_pdev (struct bnxt_re_dev *rdev) {struct pci_dev *pdev = rdev->en_dev->pdev; +struct bnxt_re_dbg_cc_config_params *cc_params; +int i; rdev->dbg_root =debugfs_create_dir (dev_name (&pdev->dev), bnxt_re_debugfs_root);
Quoted 14 lines
drivers/infiniband/hw/bnxt_re/debugfs.c void bnxt_re_debug_rem_qpinfo(struct bnxt_re_dev *rdev, struct bnxt_re_qp *qp) · line 325
Quoted 12 lines
rdev->qp_debugfs = debugfs_create_dir("QPs", rdev->dbg_root);drivers/infiniband/hw/bnxt_re/debugfs.c +11
-0+ rdev->cc_config =debugfs_create_dir ("cc_config" , rdev->dbg_root); + + rdev->cc_config_params =kzalloc (sizeof (*cc_params),GFP_KERNEL ); + +for (i =0 ; i <BNXT_RE_CC_PARAM_GEN0 ; i++) { +struct bnxt_re_cc_param *tmp_params = &rdev->cc_config_params->gen0_parms[i]; + + tmp_params->rdev = rdev; + tmp_params->offset = i; + tmp_params->cc_gen =CC_CONFIG_GEN0_EXT0 ; + tmp_params->dentry = debugfs_create_file(bnxt_re_cc_gen0_name[i], 0400,Write operation doesn't seem to work?
right. I made a mistake in some code cleanup done just before posting this patch.
I have a bug fix patch for this, will post it. Thanks.
Quoted 7 lines
Quoted 5 lines
drivers/infiniband/hw/bnxt_re/debugfs.c +3
-0+ rdev->cc_config, tmp_params, + &bnxt_re_cc_config_ops); + } }