implement DEVLINK_CMD_REGION_NEW
Subsystems (8)
v3 cover implement DEVLINK_CMD_REGION_NEW
This series adds support for the DEVLINK_CMD_REGION_NEW operation, used to enable userspace requesting a snapshot of a region on demand.
This can be useful to enable adding regions for a driver for which there is no trigger to create snapshots. By making this a core part of devlink, there is no need for the drivers to use a separate channel such as debugfs.
The primary intent for this kind of region is to expose device information that might be useful for diagnostics and information gathering.
The first few patches refactor regions to support a new ops structure for extending the available operations that regions can perform. This includes converting the destructor into an op from a function argument.
Next, patches refactor the snapshot id allocation to use an xarray which tracks the number of current snapshots using a given id. This is done so that id lifetime can be determined, and ids can be released when no longer in use.
Without this change, snapshot ids remain used forever, until the snapshot_id count rolled over UINT_MAX.
Finally, code to enable the previously unused DEVLINK_CMD_REGION_NEW is added. This code enforces that the snapshot id is always provided, unlike previous revisions of this series.
Finally, a patch is added to enable using this new command via the .snapshot callback in both netdevsim and the ice driver.
For the ice driver, a new "nvm-flash" region is added, which will enable read access to the NVM flash contents. The intention for this is to allow diagnostics tools to gather information about the device. By using a snapshot and gathering the NVM contents all at once, the contents can be atomic.
Links to previous discussions:
1st RFC - https://lore.kernel.org/netdev/[email protected]/
2nd RFC - https://lore.kernel.org/netdev/[email protected]/
v1 - https://lore.kernel.org/netdev/[email protected]/
v2 - https://lore.kernel.org/netdev/[email protected]/
Major changes since RFC:
* use an xarray for tracking snapshot ids, rather than an IDR
* remove support for auto-generated snapshot ids in DEVLINK_CMD_REGION_NEW
See each patch for an individual changelog per-patch
Jacob Keller (11):
devlink: prepare to support region operations
devlink: convert snapshot destructor callback to region op
devlink: trivial: fix tab in function documentation
devlink: add function to take snapshot while locked
devlink: use -ENOSPC to indicate no more room for snapshots
devlink: extract snapshot id allocation to helper function
devlink: report error once U32_MAX snapshot ids have been used
devlink: track snapshot id usage count using an xarray
devlink: implement DEVLINK_CMD_REGION_NEW
netdevsim: support taking immediate snapshot via devlink
ice: add a devlink region for dumping NVM contents
.../networking/devlink/devlink-region.rst | 8 + Documentation/networking/devlink/ice.rst | 26 ++ drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_devlink.c | 96 +++++ drivers/net/ethernet/intel/ice/ice_devlink.h | 3 + drivers/net/ethernet/intel/ice/ice_main.c | 4 + drivers/net/ethernet/mellanox/mlx4/crdump.c | 36 +- drivers/net/netdevsim/dev.c | 45 ++- include/net/devlink.h | 33 +- net/core/devlink.c | 362 +++++++++++++++--- .../drivers/net/netdevsim/devlink.sh | 10 + 11 files changed, 546 insertions(+), 79 deletions(-)
Signature · 1 line
2.24.1
PATCH v3 1/11 devlink: prepare to support region operations
Modify the devlink region code in preparation for adding new operations on regions.
Create a devlink_region_ops structure, and move the name pointer from within the devlink_region structure into the ops structure (similar to the devlink_health_reporter_ops).
This prepares the regions to enable support of additional operations in the future such as requesting snapshots, or accessing the region directly without a snapshot.
In order to re-use the constant strings in the mlx4 driver their declaration must be changed to 'const char * const' to ensure the compiler realizes that both the data and the pointer cannot change.
---
Changes since RFC
* Picked up Jiri's Reviewed-by
drivers/net/ethernet/mellanox/mlx4/crdump.c | 16 +++++++++++---- drivers/net/netdevsim/dev.c | 6 +++++- include/net/devlink.h | 16 +++++++++++---- net/core/devlink.c | 22 ++++++++++----------- 4 files changed, 40 insertions(+), 20 deletions(-)
drivers/net/ethernet/mellanox/mlx4/crdump.c +12 -4
@@ -38,8 +38,16 @@#define CR_ENABLE_BIT_OFFSET 0xF3F04#define MAX_NUM_OF_DUMPS_TO_STORE (8) -static const char *region_cr_space_str ="cr-space" ; -static const char *region_fw_health_str ="fw-health" ; +static const char *const region_cr_space_str ="cr-space" ; +static const char *const region_fw_health_str ="fw-health" ; + +static const struct devlink_region_ops region_cr_space_ops = { + .name = region_cr_space_str, +}; + +static const struct devlink_region_ops region_fw_health_ops = { + .name = region_fw_health_str, +};/* Set to true in case cr enable bit was set to true before crdump */ static bool crdump_enbale_bit_set;
@@ -205,7 +213,7 @@ int mlx4_crdump_init(struct mlx4_dev *dev)/* Create cr-space region */ crdump->region_crspace =devlink_region_create (devlink, - region_cr_space_str, + ®ion_cr_space_ops,MAX_NUM_OF_DUMPS_TO_STORE ,pci_resource_len (pdev,0 ));if (IS_ERR (crdump->region_crspace))
@@ -216,7 +224,7 @@ int mlx4_crdump_init(struct mlx4_dev *dev)/* Create fw-health region */ crdump->region_fw_health =devlink_region_create (devlink, - region_fw_health_str, + ®ion_fw_health_ops,MAX_NUM_OF_DUMPS_TO_STORE ,HEALTH_BUFFER_SIZE );if (IS_ERR (crdump->region_fw_health))
drivers/net/netdevsim/dev.c +5 -1
@@ -340,11 +340,15 @@ static void nsim_devlink_param_load_driverinit_values(struct devlink *devlink)#define NSIM_DEV_DUMMY_REGION_SNAPSHOT_MAX 16 +static const struct devlink_region_ops dummy_region_ops = { + .name ="dummy" , +}; +static int nsim_dev_dummy_region_init (struct nsim_dev *nsim_dev,struct devlink *devlink) { nsim_dev->dummy_region = -devlink_region_create (devlink,"dummy" , +devlink_region_create (devlink, &dummy_region_ops,NSIM_DEV_DUMMY_REGION_SNAPSHOT_MAX ,NSIM_DEV_DUMMY_REGION_SIZE );return PTR_ERR_OR_ZERO (nsim_dev->dummy_region);
include/net/devlink.h +12 -4
@@ -498,6 +498,14 @@ struct devlink_info_req;typedef void devlink_snapshot_data_dest_t (const void *data); +/** +* struct devlink_region_ops - Region operations +* @name: region name +*/ +struct devlink_region_ops { +const char *name; +}; +struct devlink_fmsg ;struct devlink_health_reporter ;
@@ -963,10 +971,10 @@ void devlink_port_param_value_changed(struct devlink_port *devlink_port, u32 param_id);void devlink_param_value_str_fill (union devlink_param_value *dst_val,const char *src); -struct devlink_region *devlink_region_create (struct devlink *devlink, -const char *region_name, -u32 region_max_snapshots, -u64 region_size); +struct devlink_region * +devlink_region_create (struct devlink *devlink, +const struct devlink_region_ops *ops, +u32 region_max_snapshots,u64 region_size);void devlink_region_destroy (struct devlink_region *region);u32 devlink_region_snapshot_id_get (struct devlink *devlink); int devlink_region_snapshot_create(struct devlink_region *region,
net/core/devlink.c +11 -11
@@ -344,7 +344,7 @@ devlink_sb_tc_index_get_from_info(struct devlink_sb *devlink_sb,struct devlink_region {struct devlink *devlink;struct list_head list; -const char *name; +const struct devlink_region_ops *ops;struct list_head snapshot_list;u32 max_snapshots;u32 cur_snapshots;
@@ -365,7 +365,7 @@ devlink_region_get_by_name(struct devlink *devlink, const char *region_name)struct devlink_region *region;list_for_each_entry (region, &devlink->region_list, list) -if (!strcmp (region->name, region_name)) +if (!strcmp (region->ops->name, region_name))return region;return NULL ;
@@ -3695,7 +3695,7 @@ static int devlink_nl_region_fill(struct sk_buff *msg, struct devlink *devlink,if (err) goto nla_put_failure; - err =nla_put_string (msg,DEVLINK_ATTR_REGION_NAME , region->name); + err =nla_put_string (msg,DEVLINK_ATTR_REGION_NAME , region->ops->name);if (err) goto nla_put_failure;
@@ -3741,7 +3741,7 @@ static void devlink_nl_region_notify(struct devlink_region *region, goto out_cancel_msg; err =nla_put_string (msg,DEVLINK_ATTR_REGION_NAME , - region->name); + region->ops->name);if (err) goto out_cancel_msg;
@@ -7647,21 +7647,21 @@ EXPORT_SYMBOL_GPL(devlink_param_value_str_fill);* devlink_region_create - create a new address region * * @devlink: devlink -* @region_name: region name +* @ops: region operations and name * @region_max_snapshots: Maximum supported number of snapshots for region * @region_size: size of region */ -struct devlink_region *devlink_region_create (struct devlink *devlink, -const char *region_name, -u32 region_max_snapshots, -u64 region_size) +struct devlink_region * +devlink_region_create (struct devlink *devlink, +const struct devlink_region_ops *ops, +u32 region_max_snapshots,u64 region_size) {struct devlink_region *region;int err =0 ;mutex_lock (&devlink->lock); -if (devlink_region_get_by_name (devlink, region_name)) { +if (devlink_region_get_by_name (devlink, ops->name)) { err = -EEXIST ; goto unlock; }
@@ -7674,7 +7674,7 @@ struct devlink_region *devlink_region_create(struct devlink *devlink, region->devlink = devlink; region->max_snapshots = region_max_snapshots; - region->name = region_name; + region->ops = ops; region->size = region_size;INIT_LIST_HEAD (®ion->snapshot_list);list_add_tail (®ion->list, &devlink->region_list);
Signature · 1 line
2.24.1
Re: [PATCH net-next v3 01/11] devlink: prepare to support region operations
Quoted 39 lines · Jacob Keller
Modify the devlink region code in preparation for adding new operations on regions.
Create a
devlink_region_opsstructure, and move the name pointer from within thedevlink_regionstructure into the ops structure (similar to thedevlink_health_reporter_ops).This prepares the regions to enable support of additional operations in the future such as requesting snapshots, or accessing the region directly without a snapshot.
In order to re-use the constant strings in the mlx4 driver their declaration must be changed to 'const char * const' to ensure the compiler realizes that both the data and the pointer cannot change.
Signed-off-by Jacob Keller <[email protected]>Reviewed-by Jakub Kicinski <[email protected]>Reviewed-by Jiri Pirko <[email protected]>---
Changes since RFC
* Picked up Jiri's Reviewed-bydrivers/net/ethernet/mellanox/mlx4/crdump.c | 16 +++++++++++---- drivers/net/netdevsim/dev.c | 6 +++++- include/net/devlink.h | 16 +++++++++++---- net/core/devlink.c | 22 ++++++++++----------- 4 files changed, 40 insertions(+), 20 deletions(-)drivers/net/ethernet/mellanox/mlx4/crdump.c +1
-2@@ -38,8 +38,16 @@#define CR_ENABLE_BIT_OFFSET 0xF3F04#define MAX_NUM_OF_DUMPS_TO_STORE (8) -static const char *region_cr_space_str ="cr-space" ; -static const char *region_fw_health_str ="fw-health" ; +static const char *const region_cr_space_str ="cr-space" ;
drivers/net/ethernet/mellanox/mlx4/crdump.c line 51
Quoted 10 lines
drivers/net/ethernet/mellanox/mlx4/crdump.c +9
-0+static const char *const region_fw_health_str ="fw-health" ; + +static const struct devlink_region_ops region_cr_space_ops = { + .name = region_cr_space_str, +}; + +static const struct devlink_region_ops region_fw_health_ops = { + .name = region_fw_health_str, +};
Hi, Jacob.
After pull net-next, I get below compiler errors:
drivers/net/ethernet/mellanox//mlx4/crdump.c:45:10: error: initializer element is not constant
.name = region_cr_space_str,
^
drivers/net/ethernet/mellanox//mlx4/crdump.c:45:10: note: (near initialization for ‘region_cr_space_ops.name’) drivers/net/ethernet/mellanox//mlx4/crdump.c:50:10: error: initializer element is not constant
.name = region_fw_health_str,
It seems the value of variables region_cr_space_str and region_cr_space_str is unknown during compiling phase.
Huazhong.
Quoted 153 lines
drivers/net/ethernet/mellanox/mlx4/crdump.c +2
-2/* Set to true in case cr enable bit was set to true before crdump */ static bool crdump_enbale_bit_set;@@ -205,7 +213,7 @@ int mlx4_crdump_init(struct mlx4_dev *dev)/* Create cr-space region */ crdump->region_crspace =devlink_region_create (devlink, - region_cr_space_str, + ®ion_cr_space_ops,MAX_NUM_OF_DUMPS_TO_STORE ,pci_resource_len (pdev,0 ));if (IS_ERR (crdump->region_crspace))@@ -216,7 +224,7 @@ int mlx4_crdump_init(struct mlx4_dev *dev)/* Create fw-health region */ crdump->region_fw_health =devlink_region_create (devlink, - region_fw_health_str, + ®ion_fw_health_ops,MAX_NUM_OF_DUMPS_TO_STORE ,HEALTH_BUFFER_SIZE );if (IS_ERR (crdump->region_fw_health))drivers/net/netdevsim/dev.c +5
-1@@ -340,11 +340,15 @@ static void nsim_devlink_param_load_driverinit_values(struct devlink *devlink)#define NSIM_DEV_DUMMY_REGION_SNAPSHOT_MAX 16 +static const struct devlink_region_ops dummy_region_ops = { + .name ="dummy" , +}; +static int nsim_dev_dummy_region_init (struct nsim_dev *nsim_dev,struct devlink *devlink) { nsim_dev->dummy_region = -devlink_region_create (devlink,"dummy" , +devlink_region_create (devlink, &dummy_region_ops,NSIM_DEV_DUMMY_REGION_SNAPSHOT_MAX ,NSIM_DEV_DUMMY_REGION_SIZE );return PTR_ERR_OR_ZERO (nsim_dev->dummy_region);include/net/devlink.h +12
-4@@ -498,6 +498,14 @@ struct devlink_info_req;typedef void devlink_snapshot_data_dest_t (const void *data); +/** +* struct devlink_region_ops - Region operations +* @name: region name +*/ +struct devlink_region_ops { +const char *name; +}; +struct devlink_fmsg ;struct devlink_health_reporter ;@@ -963,10 +971,10 @@ void devlink_port_param_value_changed(struct devlink_port *devlink_port, u32 param_id);void devlink_param_value_str_fill (union devlink_param_value *dst_val,const char *src); -struct devlink_region *devlink_region_create (struct devlink *devlink, -const char *region_name, -u32 region_max_snapshots, -u64 region_size); +struct devlink_region * +devlink_region_create (struct devlink *devlink, +const struct devlink_region_ops *ops, +u32 region_max_snapshots,u64 region_size);void devlink_region_destroy (struct devlink_region *region);u32 devlink_region_snapshot_id_get (struct devlink *devlink); int devlink_region_snapshot_create(struct devlink_region *region,net/core/devlink.c +11
-11@@ -344,7 +344,7 @@ devlink_sb_tc_index_get_from_info(struct devlink_sb *devlink_sb,struct devlink_region {struct devlink *devlink;struct list_head list; -const char *name; +const struct devlink_region_ops *ops;struct list_head snapshot_list;u32 max_snapshots;u32 cur_snapshots;@@ -365,7 +365,7 @@ devlink_region_get_by_name(struct devlink *devlink, const char *region_name)struct devlink_region *region;list_for_each_entry (region, &devlink->region_list, list) -if (!strcmp (region->name, region_name)) +if (!strcmp (region->ops->name, region_name))return region;return NULL ;@@ -3695,7 +3695,7 @@ static int devlink_nl_region_fill(struct sk_buff *msg, struct devlink *devlink,if (err) goto nla_put_failure; - err =nla_put_string (msg,DEVLINK_ATTR_REGION_NAME , region->name); + err =nla_put_string (msg,DEVLINK_ATTR_REGION_NAME , region->ops->name);if (err) goto nla_put_failure;@@ -3741,7 +3741,7 @@ static void devlink_nl_region_notify(struct devlink_region *region, goto out_cancel_msg; err =nla_put_string (msg,DEVLINK_ATTR_REGION_NAME , - region->name); + region->ops->name);if (err) goto out_cancel_msg;@@ -7647,21 +7647,21 @@ EXPORT_SYMBOL_GPL(devlink_param_value_str_fill);* devlink_region_create - create a new address region * * @devlink: devlink -* @region_name: region name +* @ops: region operations and name * @region_max_snapshots: Maximum supported number of snapshots for region * @region_size: size of region */ -struct devlink_region *devlink_region_create (struct devlink *devlink, -const char *region_name, -u32 region_max_snapshots, -u64 region_size) +struct devlink_region * +devlink_region_create (struct devlink *devlink, +const struct devlink_region_ops *ops, +u32 region_max_snapshots,u64 region_size) {struct devlink_region *region;int err =0 ;mutex_lock (&devlink->lock); -if (devlink_region_get_by_name (devlink, region_name)) { +if (devlink_region_get_by_name (devlink, ops->name)) { err = -EEXIST ; goto unlock; }@@ -7674,7 +7674,7 @@ struct devlink_region *devlink_region_create(struct devlink *devlink, region->devlink = devlink; region->max_snapshots = region_max_snapshots; - region->name = region_name; + region->ops = ops; region->size = region_size;INIT_LIST_HEAD (®ion->snapshot_list);list_add_tail (®ion->list, &devlink->region_list);
Re: [PATCH net-next v3 01/11] devlink: prepare to support region operations
Quoted 67 lines · tanhuazhong
Quoted 39 lines · Jacob Keller
Modify the devlink region code in preparation for adding new operations on regions.
Create a
devlink_region_opsstructure, and move the name pointer from within thedevlink_regionstructure into the ops structure (similar to thedevlink_health_reporter_ops).This prepares the regions to enable support of additional operations in the future such as requesting snapshots, or accessing the region directly without a snapshot.
In order to re-use the constant strings in the mlx4 driver their declaration must be changed to 'const char * const' to ensure the compiler realizes that both the data and the pointer cannot change.
Signed-off-by Jacob Keller <[email protected]>Reviewed-by Jakub Kicinski <[email protected]>Reviewed-by Jiri Pirko <[email protected]>---
Changes since RFC
* Picked up Jiri's Reviewed-bydrivers/net/ethernet/mellanox/mlx4/crdump.c | 16 +++++++++++---- drivers/net/netdevsim/dev.c | 6 +++++- include/net/devlink.h | 16 +++++++++++---- net/core/devlink.c | 22 ++++++++++----------- 4 files changed, 40 insertions(+), 20 deletions(-)drivers/net/ethernet/mellanox/mlx4/crdump.c +1
-2@@ -38,8 +38,16 @@#define CR_ENABLE_BIT_OFFSET 0xF3F04#define MAX_NUM_OF_DUMPS_TO_STORE (8) -static const char *region_cr_space_str ="cr-space" ; -static const char *region_fw_health_str ="fw-health" ; +static const char *const region_cr_space_str ="cr-space" ;drivers/net/ethernet/mellanox/mlx4/crdump.c line 51
Quoted 10 lines
drivers/net/ethernet/mellanox/mlx4/crdump.c +9
-0+static const char *const region_fw_health_str ="fw-health" ; + +static const struct devlink_region_ops region_cr_space_ops = { + .name = region_cr_space_str, +}; + +static const struct devlink_region_ops region_fw_health_ops = { + .name = region_fw_health_str, +};Hi, Jacob.
After pull net-next, I get below compiler errors:
drivers/net/ethernet/mellanox//mlx4/crdump.c:45:10: error: initializer element is not constant
.name =region_cr_space_str,^drivers/net/ethernet/mellanox//mlx4/crdump.c:45:10: note: (near initialization for ‘region_cr_space_ops.name’) drivers/net/ethernet/mellanox//mlx4/crdump.c:50:10: error: initializer element is not constant
.name =region_fw_health_str,
Quoted 5 lines
It seems the value of variables
region_cr_space_strandregion_cr_space_stris unknown during compiling phase.Huazhong.
What compiler are you using? I had gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1-1) and switching from "const *" to "const * const" it worked.
I'm wondering if this is only true in a more recent GCC.
We can fix it by using a macro instead, or maybe there's some other trick we can use to get the compiler to realize the variable is truly constant.
Thanks,
Jake
Re: [PATCH net-next v3 01/11] devlink: prepare to support region operations
In reply to tanhuazhongQuoted 67 lines · tanhuazhong
Quoted 39 lines · Jacob Keller
Modify the devlink region code in preparation for adding new operations on regions.
Create a
devlink_region_opsstructure, and move the name pointer from within thedevlink_regionstructure into the ops structure (similar to thedevlink_health_reporter_ops).This prepares the regions to enable support of additional operations in the future such as requesting snapshots, or accessing the region directly without a snapshot.
In order to re-use the constant strings in the mlx4 driver their declaration must be changed to 'const char * const' to ensure the compiler realizes that both the data and the pointer cannot change.
Signed-off-by Jacob Keller <[email protected]>Reviewed-by Jakub Kicinski <[email protected]>Reviewed-by Jiri Pirko <[email protected]>---
Changes since RFC
* Picked up Jiri's Reviewed-bydrivers/net/ethernet/mellanox/mlx4/crdump.c | 16 +++++++++++---- drivers/net/netdevsim/dev.c | 6 +++++- include/net/devlink.h | 16 +++++++++++---- net/core/devlink.c | 22 ++++++++++----------- 4 files changed, 40 insertions(+), 20 deletions(-)drivers/net/ethernet/mellanox/mlx4/crdump.c +1
-2@@ -38,8 +38,16 @@#define CR_ENABLE_BIT_OFFSET 0xF3F04#define MAX_NUM_OF_DUMPS_TO_STORE (8) -static const char *region_cr_space_str ="cr-space" ; -static const char *region_fw_health_str ="fw-health" ; +static const char *const region_cr_space_str ="cr-space" ;drivers/net/ethernet/mellanox/mlx4/crdump.c line 51
Quoted 10 lines
drivers/net/ethernet/mellanox/mlx4/crdump.c +9
-0+static const char *const region_fw_health_str ="fw-health" ; + +static const struct devlink_region_ops region_cr_space_ops = { + .name = region_cr_space_str, +}; + +static const struct devlink_region_ops region_fw_health_ops = { + .name = region_fw_health_str, +};Hi, Jacob.
After pull net-next, I get below compiler errors:
drivers/net/ethernet/mellanox//mlx4/crdump.c:45:10: error: initializer element is not constant
.name =region_cr_space_str,^drivers/net/ethernet/mellanox//mlx4/crdump.c:45:10: note: (near initialization for ‘region_cr_space_ops.name’) drivers/net/ethernet/mellanox//mlx4/crdump.c:50:10: error: initializer element is not constant
.name =region_fw_health_str,
Quoted 5 lines
It seems the value of variables
region_cr_space_strandregion_cr_space_stris unknown during compiling phase.Huazhong.
Based on some searching, this appears to be something where newer versions of GCC and clang can recognize that "const char * const" is safe to use as a constant expression, but older compilers cannot.
I'm going to submit a fix that uses a macro so that we avoid this breakage on slightly older compilers.
Thanks,
Jake
PATCH v3 2/11 devlink: convert snapshot destructor callback to region op
In reply to Jacob KellerIt does not makes sense that two snapshots for a given region would use different destructors. Simplify snapshot creation by adding a .destructor op for regions.
This operation will replace the data_destructor for the snapshot creation, and makes snapshot creation easier.
---
Changes since RFC:
* Picked up Jiri's Reviewed-by
drivers/net/ethernet/mellanox/mlx4/crdump.c | 6 ++++-- drivers/net/netdevsim/dev.c | 3 ++- include/net/devlink.h | 7 +++---- net/core/devlink.c | 11 +++++------ 4 files changed, 14 insertions(+), 13 deletions(-)
drivers/net/ethernet/mellanox/mlx4/crdump.c +4 -2
@@ -43,10 +43,12 @@ static const char * const region_fw_health_str = "fw-health";static const struct devlink_region_ops region_cr_space_ops = { .name = region_cr_space_str, + .destructor = &kvfree, };static const struct devlink_region_ops region_fw_health_ops = { .name = region_fw_health_str, + .destructor = &kvfree, };/* Set to true in case cr enable bit was set to true before crdump */
@@ -107,7 +109,7 @@ static void mlx4_crdump_collect_crspace(struct mlx4_dev *dev,readl (cr_space + offset); err =devlink_region_snapshot_create (crdump->region_crspace, - crspace_data, id, &kvfree); + crspace_data, id);if (err) {kvfree (crspace_data); mlx4_warn(dev, "crdump: devlink create %s snapshot id %d err %d\n",
@@ -146,7 +148,7 @@ static void mlx4_crdump_collect_fw_health(struct mlx4_dev *dev,readl (health_buf_start + offset); err =devlink_region_snapshot_create (crdump->region_fw_health, - health_data, id, &kvfree); + health_data, id);if (err) {kvfree (health_data); mlx4_warn(dev, "crdump: devlink create %s snapshot id %d err %d\n",
drivers/net/netdevsim/dev.c +2 -1
@@ -56,7 +56,7 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file, id =devlink_region_snapshot_id_get (priv_to_devlink (nsim_dev)); err =devlink_region_snapshot_create (nsim_dev->dummy_region, - dummy_data, id, kfree); + dummy_data, id);if (err) {pr_err ("Failed to create region snapshot\n" );kfree (dummy_data);
@@ -342,6 +342,7 @@ static void nsim_devlink_param_load_driverinit_values(struct devlink *devlink)static const struct devlink_region_ops dummy_region_ops = { .name ="dummy" , + .destructor = &kfree, }; static int nsim_dev_dummy_region_init(struct nsim_dev *nsim_dev,
include/net/devlink.h +3 -4
@@ -496,14 +496,14 @@ enum devlink_param_generic_id {struct devlink_region ;struct devlink_info_req ; -typedef void devlink_snapshot_data_dest_t (const void *data); -/** * struct devlink_region_ops - Region operations * @name: region name +* @destructor: callback used to free snapshot memory when deleting */ struct devlink_region_ops {const char *name; +void (*destructor)(const void *data); };struct devlink_fmsg ;
@@ -978,8 +978,7 @@ devlink_region_create(struct devlink *devlink,void devlink_region_destroy (struct devlink_region *region);u32 devlink_region_snapshot_id_get (struct devlink *devlink);int devlink_region_snapshot_create (struct devlink_region *region, -u8 *data,u32 snapshot_id, -devlink_snapshot_data_dest_t *data_destructor); +u8 *data,u32 snapshot_id);int devlink_info_serial_number_put (struct devlink_info_req *req,const char *sn); int devlink_info_driver_name_put(struct devlink_info_req *req,
net/core/devlink.c +5 -6
@@ -354,7 +354,6 @@ struct devlink_region {struct devlink_snapshot {struct list_head list;struct devlink_region *region; -devlink_snapshot_data_dest_t *data_destructor;u8 *data;u32 id; };
@@ -3775,7 +3774,7 @@ static void devlink_region_snapshot_del(struct devlink_region *region,devlink_nl_region_notify (region, snapshot,DEVLINK_CMD_REGION_DEL ); region->cur_snapshots--;list_del (&snapshot->list); - (*snapshot->data_destructor)(snapshot->data); + region->ops->destructor (snapshot->data);kfree (snapshot); }
@@ -7659,6 +7658,9 @@ devlink_region_create(struct devlink *devlink,struct devlink_region *region;int err =0 ; +if (WARN_ON (!ops) ||WARN_ON (!ops->destructor)) +return ERR_PTR (-EINVAL ); +mutex_lock (&devlink->lock);if (devlink_region_get_by_name (devlink, ops->name)) {
@@ -7745,11 +7747,9 @@ EXPORT_SYMBOL_GPL(devlink_region_snapshot_id_get);* @region: devlink region of the snapshot * @data: snapshot data * @snapshot_id: snapshot id to be created -* @data_destructor: pointer to destructor function to free data */ int devlink_region_snapshot_create (struct devlink_region *region, -u8 *data,u32 snapshot_id, -devlink_snapshot_data_dest_t *data_destructor) +u8 *data,u32 snapshot_id) {struct devlink *devlink = region->devlink;struct devlink_snapshot *snapshot;
@@ -7777,7 +7777,6 @@ int devlink_region_snapshot_create(struct devlink_region *region, snapshot->id = snapshot_id; snapshot->region = region; snapshot->data = data; - snapshot->data_destructor = data_destructor;list_add_tail (&snapshot->list, ®ion->snapshot_list);
Signature · 1 line
2.24.1
PATCH v3 3/11 devlink: trivial: fix tab in function documentation
In reply to Jacob KellerThe function documentation comment for devlink_region_snapshot_create included a literal tab character between 'future analyses' that was difficult to spot as it happened to only display as one space wide.
Fix the comment to use a space here instead of a stray tab appearing in the middle of a sentence.
---
Changes since RFC:
* Picked up Jiri's Reviewed-by
net/core/devlink.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/core/devlink.c +1 -1
@@ -7740,7 +7740,7 @@ EXPORT_SYMBOL_GPL(devlink_region_snapshot_id_get);* devlink_region_snapshot_create - create a new snapshot * This will add a new snapshot of a region. The snapshot * will be stored on the region struct and can be accessed -* from devlink. This is useful for future analyses of snapshots. +* from devlink. This is useful for future analyses of snapshots. * Multiple snapshots can be created on a region. * The @snapshot_id should be obtained using the getter function. *
Signature · 1 line
2.24.1
PATCH v3 4/11 devlink: add function to take snapshot while locked
In reply to Jacob KellerA future change is going to add a new devlink command to request a snapshot on demand. This function will want to call the devlink_region_snapshot_create function while already holding the devlink instance lock.
Extract the logic of this function into a static function prefixed by __ to indicate that it is an internal helper function. Modify the original function to be implemented in terms of the new locked function.
---
Changes since v1:
* Picked up Jakub's Reviewed-by
net/core/devlink.c | 78 ++++++++++++++++++++++++++++------------------ 1 file changed, 47 insertions(+), 31 deletions(-)
net/core/devlink.c +47 -31
@@ -3768,6 +3768,52 @@ static void devlink_nl_region_notify(struct devlink_region *region,nlmsg_free (msg); } +/** +* __devlink_region_snapshot_create - create a new snapshot +* This will add a new snapshot of a region. The snapshot +* will be stored on the region struct and can be accessed +* from devlink. This is useful for future analyses of snapshots. +* Multiple snapshots can be created on a region. +* The @snapshot_id should be obtained using the getter function. +* +* Must be called only while holding the devlink instance lock. +* +* @region: devlink region of the snapshot +* @data: snapshot data +* @snapshot_id: snapshot id to be created +*/ +static int +__devlink_region_snapshot_create (struct devlink_region *region, +u8 *data,u32 snapshot_id) +{ +struct devlink *devlink = region->devlink; +struct devlink_snapshot *snapshot; + +lockdep_assert_held (&devlink->lock); + +/* check if region can hold one more snapshot */ +if (region->cur_snapshots == region->max_snapshots) +return -ENOMEM ; + +if (devlink_region_snapshot_get_by_id (region, snapshot_id)) +return -EEXIST ; + + snapshot =kzalloc (sizeof (*snapshot),GFP_KERNEL ); +if (!snapshot) +return -ENOMEM ; + + snapshot->id = snapshot_id; + snapshot->region = region; + snapshot->data = data; + +list_add_tail (&snapshot->list, ®ion->snapshot_list); + + region->cur_snapshots++; + +devlink_nl_region_notify (region, snapshot,DEVLINK_CMD_REGION_NEW ); +return 0 ; +} +static void devlink_region_snapshot_del (struct devlink_region *region,struct devlink_snapshot *snapshot) {
@@ -7752,42 +7798,12 @@ int devlink_region_snapshot_create(struct devlink_region *region, u8 *data, u32 snapshot_id) {struct devlink *devlink = region->devlink; -struct devlink_snapshot *snapshot;int err;mutex_lock (&devlink->lock); - -/* check if region can hold one more snapshot */ -if (region->cur_snapshots == region->max_snapshots) { - err = -ENOMEM ; - goto unlock; - } - -if (devlink_region_snapshot_get_by_id (region, snapshot_id)) { - err = -EEXIST ; - goto unlock; - } - - snapshot =kzalloc (sizeof (*snapshot),GFP_KERNEL ); -if (!snapshot) { - err = -ENOMEM ; - goto unlock; - } - - snapshot->id = snapshot_id; - snapshot->region = region; - snapshot->data = data; - -list_add_tail (&snapshot->list, ®ion->snapshot_list); - - region->cur_snapshots++; - -devlink_nl_region_notify (region, snapshot,DEVLINK_CMD_REGION_NEW ); + err =__devlink_region_snapshot_create (region, data, snapshot_id);mutex_unlock (&devlink->lock); -return 0 ; -unlock: -mutex_unlock (&devlink->lock);return err; }EXPORT_SYMBOL_GPL (devlink_region_snapshot_create);
Signature · 1 line
2.24.1
PATCH v3 5/11 devlink: use -ENOSPC to indicate no more room for snapshots
In reply to Jacob KellerThe devlink_region_snapshot_create function returns -ENOMEM when the maximum number of snapshots has been reached. This is confusing because it is not an issue of being out of memory. Change this to use -ENOSPC instead.
---
Changes since v1:
* Introduced this patch
net/core/devlink.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/core/devlink.c +1 -1
@@ -3793,7 +3793,7 @@ __devlink_region_snapshot_create(struct devlink_region *region,/* check if region can hold one more snapshot */ if (region->cur_snapshots == region->max_snapshots) -return -ENOMEM ; +return -ENOSPC ;if (devlink_region_snapshot_get_by_id (region, snapshot_id))return -EEXIST ;
Signature · 1 line
2.24.1
PATCH v3 6/11 devlink: extract snapshot id allocation to helper function
In reply to Jacob KellerA future change is going to implement a new devlink command to request a snapshot on demand. As part of this, the logic for handling the snapshot ids will be refactored. To simplify the snapshot id allocation function, move it to a separate function prefixed by __. This helper function will assume the lock is held.
While no other callers will exist, it simplifies refactoring the logic because there is no need to complicate the function with gotos to handle unlocking on failure.
---
Changes since RFC:
* Added this patch to explain reasoning why the function is kept despite only having one caller.
net/core/devlink.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
net/core/devlink.c +14 -1
@@ -3768,6 +3768,19 @@ static void devlink_nl_region_notify(struct devlink_region *region,nlmsg_free (msg); } +/** +* __devlink_region_snapshot_id_get - get snapshot ID +* @devlink: devlink instance +* +* Returns a new snapshot id. Must be called while holding the +* devlink instance lock. +*/ +static u32 __devlink_region_snapshot_id_get (struct devlink *devlink) +{ +lockdep_assert_held (&devlink->lock); +return ++devlink->snapshot_id; +} +/** * __devlink_region_snapshot_create - create a new snapshot * This will add a new snapshot of a region. The snapshot
@@ -7775,7 +7788,7 @@ u32 devlink_region_snapshot_id_get(struct devlink *devlink)u32 id;mutex_lock (&devlink->lock); - id = ++devlink->snapshot_id; + id =__devlink_region_snapshot_id_get (devlink);mutex_unlock (&devlink->lock);return id;
Signature · 1 line
2.24.1
Re: [PATCH net-next v3 06/11] devlink: extract snapshot id allocation to helper function
Quoted 11 lines · Thu, Mar 26, 2020 at 07:37:13PM CET, [email protected] wrote:
A future change is going to implement a new devlink command to request a snapshot on demand. As part of this, the logic for handling the snapshot ids will be refactored. To simplify the snapshot id allocation function, move it to a separate function prefixed by
__. This helper function will assume the lock is held.While no other callers will exist, it simplifies refactoring the logic because there is no need to complicate the function with gotos to handle unlocking on failure.
Signed-off-by Jacob Keller <[email protected]>
For the 3rd time. I'm not sure why you don't took this :/
Re: [PATCH net-next v3 06/11] devlink: extract snapshot id allocation to helper function
Quoted 11 lines · Jiri Pirko
Quoted 10 lines · Thu, Mar 26, 2020 at 07:37:13PM CET, [email protected] wrote:
A future change is going to implement a new devlink command to request a snapshot on demand. As part of this, the logic for handling the snapshot ids will be refactored. To simplify the snapshot id allocation function, move it to a separate function prefixed by
__. This helper function will assume the lock is held.While no other callers will exist, it simplifies refactoring the logic because there is no need to complicate the function with gotos to handle unlocking on failure.
Quoted 6 lines
Quoted 1 line
Signed-off-by Jacob Keller <[email protected]>Reviewed-by Jiri Pirko <[email protected]>For the 3rd time. I'm not sure why you don't took this :/
Missed it while going over all the emails to take in review.
PATCH v3 7/11 devlink: report error once U32_MAX snapshot ids have been used
In reply to Jacob KellerThe devlink_snapshot_id_get() function returns a snapshot id. The snapshot id is a u32, so there is no way to indicate an error code.
A future change is going to possibly add additional cases where this function could fail. Refactor the function to return the snapshot id in an argument, so that it can return zero or an error value.
This ensures that snapshot ids cannot be confused with error values, and aids in the future refactor of snapshot id allocation management.
Because there is no current way to release previously used snapshot ids, add a simple check ensuring that an error is reported in case the snapshot_id would over flow.
---
Changes since RFC:
* Picked up Jiri's Reviewed-by
Changes since v1:
* Refactored this patch to return the id in an argument, instead of conflating id and error in the return value of the function.
* Allow up to U32_MAX instead of INT_MAX
* Removed Jiri's Reviewed-by, since this is a complete re-write
Changes since v2:
* Fix commit message to avoid incorrect statement about checking snapshot ids. This referred to a previous version of the code which has been removed.
* Fixed a build failure due to forgetting a '{' bracket which got lost in a rebase conflict.
* Picked up Jiri's Reviewed-by again
drivers/net/ethernet/mellanox/mlx4/crdump.c | 11 ++++++--- drivers/net/netdevsim/dev.c | 6 ++++- include/net/devlink.h | 2 +- net/core/devlink.c | 27 +++++++++++++++------ 4 files changed, 33 insertions(+), 13 deletions(-)
drivers/net/ethernet/mellanox/mlx4/crdump.c +8 -3
@@ -169,6 +169,7 @@ int mlx4_crdump_collect(struct mlx4_dev *dev)struct pci_dev *pdev = dev->persist->pdev;unsigned long cr_res_size;u8 __iomem *cr_space; +int err;u32 id;if (!dev->caps.health_buffer_addrs) {
@@ -189,10 +190,14 @@ int mlx4_crdump_collect(struct mlx4_dev *dev)return -ENODEV ; } -crdump_enable_crspace_access (dev, cr_space); -/* Get the available snapshot ID for the dumps */ - id =devlink_region_snapshot_id_get (devlink); + err =devlink_region_snapshot_id_get (devlink, &id); +if (err) { +mlx4_err (dev,"crdump: devlink get snapshot id err %d\n" , err); +return err; + } + +crdump_enable_crspace_access (dev, cr_space);/* Try to capture dumps */ mlx4_crdump_collect_crspace (dev, cr_space, id);
drivers/net/netdevsim/dev.c +5 -1
@@ -54,7 +54,11 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file,get_random_bytes (dummy_data,NSIM_DEV_DUMMY_REGION_SIZE ); - id =devlink_region_snapshot_id_get (priv_to_devlink (nsim_dev)); + err =devlink_region_snapshot_id_get (priv_to_devlink (nsim_dev), &id); +if (err) { +pr_err ("Failed to get snapshot id\n" ); +return err; + } err =devlink_region_snapshot_create (nsim_dev->dummy_region, dummy_data, id);if (err) {
include/net/devlink.h +1 -1
@@ -976,7 +976,7 @@ devlink_region_create(struct devlink *devlink,const struct devlink_region_ops *ops, u32 region_max_snapshots, u64 region_size);void devlink_region_destroy (struct devlink_region *region); -u32 devlink_region_snapshot_id_get (struct devlink *devlink); +int devlink_region_snapshot_id_get (struct devlink *devlink,u32 *id);int devlink_region_snapshot_create (struct devlink_region *region,u8 *data,u32 snapshot_id); int devlink_info_serial_number_put(struct devlink_info_req *req,
net/core/devlink.c +19 -8
@@ -3771,14 +3771,22 @@ static void devlink_nl_region_notify(struct devlink_region *region,/** * __devlink_region_snapshot_id_get - get snapshot ID * @devlink: devlink instance +* @id: storage to return snapshot id * -* Returns a new snapshot id. Must be called while holding the -* devlink instance lock. +* Allocates a new snapshot id. Returns zero on success, or a negative +* error on failure. Must be called while holding the devlink instance +* lock. */ -static u32 __devlink_region_snapshot_id_get (struct devlink *devlink) +static int __devlink_region_snapshot_id_get (struct devlink *devlink,u32 *id) {lockdep_assert_held (&devlink->lock); -return ++devlink->snapshot_id; + +if (devlink->snapshot_id >=U32_MAX ) +return -ENOSPC ; + + *id = ++devlink->snapshot_id; + +return 0 ; }/**
@@ -7781,17 +7789,20 @@ EXPORT_SYMBOL_GPL(devlink_region_destroy);* Driver should use the same id for multiple snapshots taken * on multiple regions at the same time/by the same trigger. * +* Returns zero on success, or a negative error code on failure. +* * @devlink: devlink +* @id: storage to return id */ -u32 devlink_region_snapshot_id_get (struct devlink *devlink) +int devlink_region_snapshot_id_get (struct devlink *devlink,u32 *id) { -u32 id; +int err;mutex_lock (&devlink->lock); - id =__devlink_region_snapshot_id_get (devlink); + err =__devlink_region_snapshot_id_get (devlink, id);mutex_unlock (&devlink->lock); -return id; +return err; }EXPORT_SYMBOL_GPL (devlink_region_snapshot_id_get);
Signature · 1 line
2.24.1
PATCH v3 8/11 devlink: track snapshot id usage count using an xarray
In reply to Jacob KellerEach snapshot created for a devlink region must have an id. These ids are supposed to be unique per "event" that caused the snapshot to be created. Drivers call devlink_region_snapshot_id_get to obtain a new id to use for a new event trigger. The id values are tracked per devlink, so that the same id number can be used if a triggering event creates multiple snapshots on different regions.
There is no mechanism for snapshot ids to ever be reused. Introduce an xarray to store the count of how many snapshots are using a given id, replacing the snapshot_id field previously used for picking the next id.
The devlink_region_snapshot_id_get() function will use xa_alloc to insert an initial value of 1 value at an available slot between 0 and U32_MAX.
The new __devlink_snapshot_id_increment() and __devlink_snapshot_id_decrement() functions will be used to track how many snapshots currently use an id.
Drivers must now call devlink_snapshot_id_put() in order to release their reference of the snapshot id after adding region snapshots.
By tracking the total number of snapshots using a given id, it is possible for the decrement() function to erase the id from the xarray when it is not in use.
With this method, a snapshot id can become reused again once all snapshots that referred to it have been deleted via DEVLINK_CMD_REGION_DEL, and the driver has finished adding snapshots.
This work also paves the way to introduce a mechanism for userspace to request a snapshot.
---
Changes since RFC:
* Rewrote to use xarray directly, instead of IDR
Changes since v1:
* Add a new devlink_region_snapshot_id_put, and have the devlink_region_snapshot_id_get start the xarray usage count at 1. This fixes a race condition where the driver might create two snapshots using the id, but the first one is freed before the second one is created. Additionally it resolves an issue where the snapshot id could be locked forever if the driver does not create a snapshot due to an error.
* Add WARN_ON to checks in __devlink_snapshot_id_increment
* Remove "if (err) return err" constructions that occur just after a return 0, as a direct return can be used instead.
* Rename goto labels to indicate the cause of failure instead of the action taken to clean up.
* Set XA_FLAGS_ALLOC so that xa_alloc can actually be used properly
* Remove the unnecessary locking around xa_destroy
Changes since v2:
* Move devlink variable assignment
* Remove a comment about placement of devlink_region_snapshot_id_put
* Renamed label to use "err_" prefix
drivers/net/ethernet/mellanox/mlx4/crdump.c | 3 + drivers/net/netdevsim/dev.c | 6 +- include/net/devlink.h | 4 +- net/core/devlink.c | 130 +++++++++++++++++++- 4 files changed, 135 insertions(+), 8 deletions(-)
drivers/net/ethernet/mellanox/mlx4/crdump.c +3 -0
@@ -203,6 +203,9 @@ int mlx4_crdump_collect(struct mlx4_dev *dev)mlx4_crdump_collect_crspace (dev, cr_space, id);mlx4_crdump_collect_fw_health (dev, cr_space, id); +/* Release reference on the snapshot id */ +devlink_region_snapshot_id_put (devlink, id); +crdump_disable_crspace_access (dev, cr_space);iounmap (cr_space);
drivers/net/netdevsim/dev.c +5 -1
@@ -44,23 +44,27 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file, size_t count, loff_t *ppos) { struct nsim_dev *nsim_dev = file->private_data; +struct devlink *devlink;void *dummy_data;int err;u32 id; + devlink =priv_to_devlink (nsim_dev); + dummy_data =kmalloc (NSIM_DEV_DUMMY_REGION_SIZE ,GFP_KERNEL );if (!dummy_data)return -ENOMEM ;get_random_bytes (dummy_data,NSIM_DEV_DUMMY_REGION_SIZE ); - err =devlink_region_snapshot_id_get (priv_to_devlink (nsim_dev), &id); + err =devlink_region_snapshot_id_get (devlink, &id);if (err) {pr_err ("Failed to get snapshot id\n" );return err; } err =devlink_region_snapshot_create (nsim_dev->dummy_region, dummy_data, id); +devlink_region_snapshot_id_put (devlink, id);if (err) {pr_err ("Failed to create region snapshot\n" );kfree (dummy_data);
include/net/devlink.h +3 -1
@@ -18,6 +18,7 @@#include <net/net_namespace.h> #include <net/flow_offload.h> #include <uapi/linux/devlink.h> +#include <linux/xarray.h> struct devlink_ops ;
@@ -29,13 +30,13 @@ struct devlink {struct list_head resource_list;struct list_head param_list;struct list_head region_list; -u32 snapshot_id;struct list_head reporter_list;struct mutex reporters_lock;/* protects reporter_list */ struct devlink_dpipe_headers *dpipe_headers;struct list_head trap_list;struct list_head trap_group_list;const struct devlink_ops *ops; +struct xarray snapshot_ids;struct device *dev;possible_net_t _net;struct mutex lock;
@@ -977,6 +978,7 @@ devlink_region_create(struct devlink *devlink, u32 region_max_snapshots, u64 region_size);void devlink_region_destroy (struct devlink_region *region);int devlink_region_snapshot_id_get (struct devlink *devlink,u32 *id); +void devlink_region_snapshot_id_put (struct devlink *devlink,u32 id);int devlink_region_snapshot_create (struct devlink_region *region,u8 *data,u32 snapshot_id); int devlink_info_serial_number_put(struct devlink_info_req *req,
net/core/devlink.c +124 -6
198 lines not on this page · raw message
Signature · 1 line
2.24.1
Re: [PATCH net-next v3 08/11] devlink: track snapshot id usage count using an xarray
Quoted 26 lines · Thu, Mar 26, 2020 at 07:37:15PM CET, [email protected] wrote:
Each snapshot created for a devlink region must have an id. These ids are supposed to be unique per "event" that caused the snapshot to be created. Drivers call
devlink_region_snapshot_id_getto obtain a new id to use for a new event trigger. The id values are tracked per devlink, so that the same id number can be used if a triggering event creates multiple snapshots on different regions.There is no mechanism for snapshot ids to ever be reused. Introduce an xarray to store the count of how many snapshots are using a given id, replacing the
snapshot_idfield previously used for picking the next id.The
devlink_region_snapshot_id_get()function will usexa_allocto insert an initial value of 1 value at an available slot between 0 andU32_MAX.The new
__devlink_snapshot_id_increment()and__devlink_snapshot_id_decrement()functions will be used to track how many snapshots currently use an id.Drivers must now call
devlink_snapshot_id_put()in order to release their reference of the snapshot id after adding region snapshots.By tracking the total number of snapshots using a given id, it is possible for the
decrement()function to erase the id from the xarray when it is not in use.
Quoted 8 lines
With this method, a snapshot id can become reused again once all snapshots that referred to it have been deleted via
DEVLINK_CMD_REGION_DEL, and the driver has finished adding snapshots.This work also paves the way to introduce a mechanism for userspace to request a snapshot.
Signed-off-by Jacob Keller <[email protected]>
PATCH v3 9/11 devlink: implement DEVLINK_CMD_REGION_NEW
In reply to Jacob KellerImplement support for the DEVLINK_CMD_REGION_NEW command for creating snapshots. This new command parallels the existing DEVLINK_CMD_REGION_DEL.
In order for DEVLINK_CMD_REGION_NEW to work for a region, the new ".snapshot" operation must be implemented in the region's ops structure.
The desired snapshot id must be provided. This helps avoid confusion on the purpose of DEVLINK_CMD_REGION_NEW, and keeps the API simpler.
The requested id will be inserted into the xarray tracking the number of snapshots using each id. If this id is already used by another snapshot on any region, an error will be returned.
---
Changes since RFC:
* Removed ability to not specify the snapshot id. Now requesting a region must always come with a snapshot id, as suggested by Jiri.
* Removed unnecessary newlines on NL_SET_ERR_MSG_MOD calls
Changes since v1:
* Add a WARN_ON to the xa_load check in devlink_snapshot_id_insert
* Remove an unnecessary "if (err) return err" construction in the insert function, as a direct return would behave the same here.
* Use ENOSPC instead of ENOMEM for when the maximum number of snapshots is reached.
* Remove an NL_SET_ERR_MSG_MOD message when snapshot_id_insert fails, as the only expected failures would be ENOMEM or would trigger a WARN_ON.
* Rename labels to specify the cause of failure rather than the action taken to cleanup.
Changes since v2:
* Renamed labels again to use "err_" prefix.
* Picked up Jiri's Reviewed-by
* Removed { } that are no longer necessary since the NL_SET_ERR_MSG_MOD call was removed in a previous version.
.../networking/devlink/devlink-region.rst | 8 ++ include/net/devlink.h | 6 ++ net/core/devlink.c | 99 +++++++++++++++++++ 3 files changed, 113 insertions(+)
Documentation/networking/devlink/devlink-region.rst +8 -0
@@ -20,6 +20,11 @@ address regions that are otherwise inaccessible to the user. Regions may also be used to provide an additional way to debug complex error states, but see also :doc:`devlink-health` +Regions may optionally support capturing a snapshot on demand via the +``DEVLINK_CMD_REGION_NEW`` netlink message. A driver wishing to allow +requested snapshots must implement the ``.snapshot`` callback for the region +in its ``devlink_region_ops`` structure. + example usage -------------
@@ -40,6 +45,9 @@ example usage # Delete a snapshot using: $ devlink region del pci/0000:00:05.0/cr-space snapshot 1 + # Request an immediate snapshot, if supported by the region + $ devlink region new pci/0000:00:05.0/cr-space snapshot 5 + # Dump a snapshot: $ devlink region dump pci/0000:00:05.0/fw-health snapshot 1 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
include/net/devlink.h +6 -0
@@ -501,10 +501,16 @@ struct devlink_info_req;* struct devlink_region_ops - Region operations * @name: region name * @destructor: callback used to free snapshot memory when deleting +* @snapshot: callback to request an immediate snapshot. On success, +* the data variable must be updated to point to the snapshot data. +* The function will be called while the devlink instance lock is +* held. */ struct devlink_region_ops {const char *name;void (*destructor)(const void *data); +int (*snapshot)(struct devlink *devlink,struct netlink_ext_ack *extack, +u8 **data); };struct devlink_fmsg ;
net/core/devlink.c +99 -0
117 lines not on this page · raw message
Signature · 1 line
2.24.1
PATCH v3 10/11 netdevsim: support taking immediate snapshot via devlink
In reply to Jacob KellerImplement the .snapshot region operation for the dummy data region. This enables a region snapshot to be taken upon request via the new DEVLINK_CMD_REGION_SNAPSHOT command.
---
Changes since v1
* remove selftests for creating a region without a snapshot id, as this functionality has been removed.
* Ran the selftests to ensure they pass now
Changes since v2
* Moved devlink variable assignment
drivers/net/netdevsim/dev.c | 28 +++++++++++++++---- .../drivers/net/netdevsim/devlink.sh | 10 +++++++ 2 files changed, 32 insertions(+), 6 deletions(-)
drivers/net/netdevsim/dev.c +22 -6
@@ -39,23 +39,38 @@ static struct dentry *nsim_dev_ddir;#define NSIM_DEV_DUMMY_REGION_SIZE (1024 * 32) +static int +nsim_dev_take_snapshot (struct devlink *devlink,struct netlink_ext_ack *extack, +u8 **data) +{ +void *dummy_data; + + dummy_data =kmalloc (NSIM_DEV_DUMMY_REGION_SIZE ,GFP_KERNEL ); +if (!dummy_data) +return -ENOMEM ; + +get_random_bytes (dummy_data,NSIM_DEV_DUMMY_REGION_SIZE ); + + *data = dummy_data; + +return 0 ; +} +static ssize_t nsim_dev_take_snapshot_write (struct file *file,const char __user *data,size_t count,loff_t *ppos) {struct nsim_dev *nsim_dev = file->private_data;struct devlink *devlink; -void *dummy_data; +u8 *dummy_data;int err;u32 id; devlink =priv_to_devlink (nsim_dev); - dummy_data =kmalloc (NSIM_DEV_DUMMY_REGION_SIZE ,GFP_KERNEL ); -if (!dummy_data) -return -ENOMEM ; - -get_random_bytes (dummy_data,NSIM_DEV_DUMMY_REGION_SIZE ); + err =nsim_dev_take_snapshot (devlink,NULL , &dummy_data); +if (err) +return err; err =devlink_region_snapshot_id_get (devlink, &id);if (err) {
@@ -351,6 +366,7 @@ static void nsim_devlink_param_load_driverinit_values(struct devlink *devlink)static const struct devlink_region_ops dummy_region_ops = { .name ="dummy" , .destructor = &kfree, + .snapshot = nsim_dev_take_snapshot, }; static int nsim_dev_dummy_region_init(struct nsim_dev *nsim_dev,
tools/testing/selftests/drivers/net/netdevsim/devlink.sh +10 -0
@@ -141,6 +141,16 @@ regions_test() check_region_snapshot_count dummy post-first-delete 2 + devlink region new $DL_HANDLE/dummy snapshot 25 + check_err $? "Failed to create a new snapshot with id 25" + + check_region_snapshot_count dummy post-first-request 3 + + devlink region del $DL_HANDLE/dummy snapshot 25 + check_err $? "Failed to delete snapshot with id 25" + + check_region_snapshot_count dummy post-second-delete 2 + log_test "regions test" }
Signature · 1 line
2.24.1
Re: [PATCH net-next v3 10/11] netdevsim: support taking immediate snapshot via devlink
Quoted 5 lines · Thu, Mar 26, 2020 at 07:37:17PM CET, [email protected] wrote:
Implement the .snapshot region operation for the dummy data region. This enables a region snapshot to be taken upon request via the new
DEVLINK_CMD_REGION_SNAPSHOTcommand.Signed-off-by Jacob Keller <[email protected]>
PATCH v3 11/11 ice: add a devlink region for dumping NVM contents
In reply to Jacob KellerAdd a devlink region for exposing the device's Non Volatime Memory flash contents.
Support the recently added .snapshot operation, enabling userspace to request a snapshot of the NVM contents via DEVLINK_CMD_REGION_NEW.
---
Changes since RFC:
* Capture the entire NVM instead of just Shadow RAM
* Use vmalloc instead of kmalloc, since the memory does not need to be physically contiguous.
* Remove the direct reading function, as this will be sent in a separate series
Changes since v1:
* use dev_err instead of dev_warn when a region fails to be created
Changes since v2:
* Removed redundant "out of memory" extack message
* Fixed up function declaration alignment to avoid checkpatch.pl warnings
Documentation/networking/devlink/ice.rst | 26 ++++++ drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_devlink.c | 96 ++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_devlink.h | 3 + drivers/net/ethernet/intel/ice/ice_main.c | 4 + 5 files changed, 131 insertions(+)
Documentation/networking/devlink/ice.rst +26 -0
@@ -69,3 +69,29 @@ The ``ice`` driver reports the following versions - The version of the DDP package that is active in the device. Note that both the name (as reported by ``fw.app.name``) and version are required to uniquely identify the package. + +Regions +======= + +The ``ice`` driver enables access to the contents of the Non Volatile Memory +flash chip via the ``nvm-flash`` region. + +Users can request an immediate capture of a snapshot via the +``DEVLINK_CMD_REGION_NEW`` + +.. code:: shell + + $ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1 + $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1 + + $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1 + 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30 + 0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8 + 0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc + 0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5 + + $ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0 + length 16 + 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30 + + $ devlink region delete pci/0000:01:00.0/nvm-flash snapshot 1
drivers/net/ethernet/intel/ice/ice.h +2 -0
@@ -351,6 +351,8 @@ struct ice_pf {/* devlink port data */ struct devlink_port devlink_port; +struct devlink_region *nvm_region; +/* OS reserved IRQ details */ struct msix_entry *msix_entries;struct ice_res_tracker *irq_tracker;
drivers/net/ethernet/intel/ice/ice_devlink.c +96 -0
99 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_devlink.h +3 -0
@@ -11,4 +11,7 @@ void ice_devlink_unregister(struct ice_pf *pf);int ice_devlink_create_port (struct ice_pf *pf);void ice_devlink_destroy_port (struct ice_pf *pf); +void ice_devlink_init_regions (struct ice_pf *pf); +void ice_devlink_destroy_regions (struct ice_pf *pf); +#endif /* _ICE_DEVLINK_H_ */
drivers/net/ethernet/intel/ice/ice_main.c +4 -0
@@ -3276,6 +3276,8 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) goto err_init_pf_unroll; } +ice_devlink_init_regions (pf); + pf->num_alloc_vsi = hw->func_caps.guar_num_vsi;if (!pf->num_alloc_vsi) { err = -EIO ;
@@ -3385,6 +3387,7 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)devm_kfree (dev, pf->vsi); err_init_pf_unroll:ice_deinit_pf (pf); +ice_devlink_destroy_regions (pf);ice_deinit_hw (hw); err_exit_unroll:ice_devlink_unregister (pf);
@@ -3427,6 +3430,7 @@ static void ice_remove(struct pci_dev *pdev)ice_vsi_free_q_vectors (pf->vsi[i]); }ice_deinit_pf (pf); +ice_devlink_destroy_regions (pf);ice_deinit_hw (&pf->hw);ice_devlink_unregister (pf);
Signature · 1 line
2.24.1
Re: [PATCH net-next v3 11/11] ice: add a devlink region for dumping NVM contents
Quoted 50 lines · Thu, Mar 26, 2020 at 07:37:18PM CET, [email protected] wrote:
Add a devlink region for exposing the device's Non Volatime Memory flash contents.
Support the recently added .snapshot operation, enabling userspace to request a snapshot of the NVM contents via
DEVLINK_CMD_REGION_NEW.Signed-off-by Jacob Keller <[email protected]>---
Changes since RFC:
* Capture the entire NVM instead of just Shadow RAM
* Use vmalloc instead of kmalloc, since the memory does not need to be physically contiguous.
* Remove the direct reading function, as this will be sent in a separate seriesChanges since v1:
* usedev_errinstead ofdev_warnwhen a region fails to be createdChanges since v2:
* Removed redundant "out of memory" extack message
* Fixed up function declaration alignment to avoid checkpatch.pl warningsDocumentation/networking/devlink/ice.rst | 26 ++++++ drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_devlink.c | 96 ++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_devlink.h | 3 + drivers/net/ethernet/intel/ice/ice_main.c | 4 + 5 files changed, 131 insertions(+)Documentation/networking/devlink/ice.rst +13
-0@@ -69,3 +69,29 @@ The ``ice`` driver reports the following versions - The version of the DDP package that is active in the device. Note that both the name (as reported by ``fw.app.name``) and version are required to uniquely identify the package. + +Regions +======= + +The ``ice`` driver enables access to the contents of the Non Volatile Memory +flash chip via the ``nvm-flash`` region. + +Users can request an immediate capture of a snapshot via the +``DEVLINK_CMD_REGION_NEW`` + +.. code:: shell + + $ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1
Documentation/networking/devlink/ice.rst The ``ice`` driver reports the following versions · line 94
Quoted 10 lines
Documentation/networking/devlink/ice.rst +10
-0+ $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1 + + $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1 + 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30 + 0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8 + 0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc + 0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5 + + $ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0 + length 16
I still think this should be one line. Anyway,
Re: [PATCH net-next v3 11/11] ice: add a devlink region for dumping NVM contents
Quoted 64 lines · Jiri Pirko
Quoted 50 lines · Thu, Mar 26, 2020 at 07:37:18PM CET, [email protected] wrote:
Add a devlink region for exposing the device's Non Volatime Memory flash contents.
Support the recently added .snapshot operation, enabling userspace to request a snapshot of the NVM contents via
DEVLINK_CMD_REGION_NEW.Signed-off-by Jacob Keller <[email protected]>---
Changes since RFC:
* Capture the entire NVM instead of just Shadow RAM
* Use vmalloc instead of kmalloc, since the memory does not need to be physically contiguous.
* Remove the direct reading function, as this will be sent in a separate seriesChanges since v1:
* usedev_errinstead ofdev_warnwhen a region fails to be createdChanges since v2:
* Removed redundant "out of memory" extack message
* Fixed up function declaration alignment to avoid checkpatch.pl warningsDocumentation/networking/devlink/ice.rst | 26 ++++++ drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_devlink.c | 96 ++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_devlink.h | 3 + drivers/net/ethernet/intel/ice/ice_main.c | 4 + 5 files changed, 131 insertions(+)Documentation/networking/devlink/ice.rst +13
-0@@ -69,3 +69,29 @@ The ``ice`` driver reports the following versions - The version of the DDP package that is active in the device. Note that both the name (as reported by ``fw.app.name``) and version are required to uniquely identify the package. + +Regions +======= + +The ``ice`` driver enables access to the contents of the Non Volatile Memory +flash chip via the ``nvm-flash`` region. + +Users can request an immediate capture of a snapshot via the +``DEVLINK_CMD_REGION_NEW`` + +.. code:: shell + + $ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1Documentation/networking/devlink/ice.rst The ``ice`` driver reports the following versions · line 94
Quoted 10 lines
Documentation/networking/devlink/ice.rst +10
-0+ $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1 + + $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1 + 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30 + 0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8 + 0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc + 0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5 + + $ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0 + length 16I still think this should be one line. Anyway,
Quoted 2 lines
Reviewed-by Jiri Pirko <[email protected]>
Ugh, I knew I was forgetting something.
Re: [PATCH net-next v3 11/11] ice: add a devlink region for dumping NVM contents
In reply to Jiri PirkoQuoted 26 lines · Jiri Pirko
Quoted 12 lines · Thu, Mar 26, 2020 at 07:37:18PM CET, [email protected] wrote:
Documentation/networking/devlink/ice.rst +12
-0+Regions +======= + +The ``ice`` driver enables access to the contents of the Non Volatile Memory +flash chip via the ``nvm-flash`` region. + +Users can request an immediate capture of a snapshot via the +``DEVLINK_CMD_REGION_NEW`` + +.. code:: shell + + $ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1Documentation/networking/devlink/ice.rst The ``ice`` driver reports the following versions · line 94
Quoted 10 lines
Documentation/networking/devlink/ice.rst +10
-0+ $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1 + + $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1 + 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30 + 0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8 + 0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc + 0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5 + + $ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0 + length 16I still think this should be one line. Anyway,
Quoted 2 lines
Reviewed-by Jiri Pirko <[email protected]>
I'm happy to send a v4 with this fix in, or to send a separate follow-up patch which cleans up all of the devlink documents to avoid this.
Dave, which would you prefer?
Thanks,
Jake
David Miller
maintainerRe: [PATCH net-next v3 11/11] ice: add a devlink region for dumping NVM contents
From: Jacob Keller <[email protected]>
Date: Thu, 26 Mar 2020 15:35:15 -0700
Quoted 4 lines
I'm happy to send a v4 with this fix in, or to send a separate follow-up patch which cleans up all of the devlink documents to avoid this.
Dave, which would you prefer?
I'm going to apply this series as-is, so please do a follow-up.
Thanks.
Re: [PATCH net-next v3 11/11] ice: add a devlink region for dumping NVM contents
Quoted 12 lines · David Miller
From: Jacob Keller <[email protected]>
Date: Thu, 26 Mar 2020 15:35:15 -0700Quoted 4 lines
I'm happy to send a v4 with this fix in, or to send a separate follow-up patch which cleans up all of the devlink documents to avoid this.
Dave, which would you prefer?
I'm going to apply this series as-is, so please do a follow-up.
Thanks.
Will do.
Regards,
Jake
David Miller
maintainerFrom: Jacob Keller <[email protected]>
Date: Thu, 26 Mar 2020 11:37:07 -0700
Quoted 2 lines
This series adds support for the
DEVLINK_CMD_REGION_NEWoperation, used to enable userspace requesting a snapshot of a region on demand.
Series applied, thanks Jacob.
Jakub Kicinski
maintainerQuoted 26 lines · On Thu, 26 Mar 2020 11:37:07 -0700 Jacob Keller wrote:
This series adds support for the
DEVLINK_CMD_REGION_NEWoperation, used to enable userspace requesting a snapshot of a region on demand.This can be useful to enable adding regions for a driver for which there is no trigger to create snapshots. By making this a core part of devlink, there is no need for the drivers to use a separate channel such as debugfs.
The primary intent for this kind of region is to expose device information that might be useful for diagnostics and information gathering.
The first few patches refactor regions to support a new ops structure for extending the available operations that regions can perform. This includes converting the destructor into an op from a function argument.
Next, patches refactor the snapshot id allocation to use an xarray which tracks the number of current snapshots using a given id. This is done so that id lifetime can be determined, and ids can be released when no longer in use.
Without this change, snapshot ids remain used forever, until the
snapshot_idcount rolled overUINT_MAX.Finally, code to enable the previously unused
DEVLINK_CMD_REGION_NEWis added. This code enforces that the snapshot id is always provided, unlike previous revisions of this series.
Quoted 8 lines
Finally, a patch is added to enable using this new command via the .snapshot callback in both netdevsim and the ice driver.
For the ice driver, a new "nvm-flash" region is added, which will enable read access to the NVM flash contents. The intention for this is to allow diagnostics tools to gather information about the device. By using a snapshot and gathering the NVM contents all at once, the contents can be atomic.
Hi Jake,
does iproute2 needs some patches to make this work?
./devlink region new netdevsim/netdevsim1/dummy snapshot_id 1 Command "new" not found
Quoted 46 lines
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Jakub Kicinski
Sent: Monday, April 27, 2020 5:27 PM
To: Keller, Jacob E <[email protected]>
Cc: [email protected]; Jiri Pirko <[email protected]>
Subject: Re: [PATCH net-next v3 00/11] implementDEVLINK_CMD_REGION_NEWQuoted 1 line · On Thu, 26 Mar 2020 11:37:07 -0700 Jacob Keller wrote:
This series adds support for the
DEVLINK_CMD_REGION_NEWoperation, usedto
Quoted 25 lines
enable userspace requesting a snapshot of a region on demand.
This can be useful to enable adding regions for a driver for which there is no trigger to create snapshots. By making this a core part of devlink, there is no need for the drivers to use a separate channel such as debugfs.
The primary intent for this kind of region is to expose device information that might be useful for diagnostics and information gathering.
The first few patches refactor regions to support a new ops structure for extending the available operations that regions can perform. This includes converting the destructor into an op from a function argument.
Next, patches refactor the snapshot id allocation to use an xarray which tracks the number of current snapshots using a given id. This is done so that id lifetime can be determined, and ids can be released when no longer in use.
Without this change, snapshot ids remain used forever, until the
snapshot_idcount rolled overUINT_MAX.Finally, code to enable the previously unused
DEVLINK_CMD_REGION_NEWis added. This code enforces that the snapshot id is always provided, unlike previous revisions of this series.Quoted 8 lines
Finally, a patch is added to enable using this new command via the .snapshot callback in both netdevsim and the ice driver.
For the ice driver, a new "nvm-flash" region is added, which will enable read access to the NVM flash contents. The intention for this is to allow diagnostics tools to gather information about the device. By using a snapshot and gathering the NVM contents all at once, the contents can be atomic.
Quoted 6 lines
Hi Jake,
does iproute2 needs some patches to make this work?
./devlink region new netdevsim/netdevsim1/dummy
snapshot_id1 Command "new" not found
Ahh, yes I think it does. I seem to have forgotten to send these, will do so now.
Thanks,
Jake