Add ability to flash modules' firmware
Subsystems (7)
v7 cover Add ability to flash modules' firmware
CMIS compliant modules such as QSFP-DD might be running a firmware that can be updated in a vendor-neutral way by exchanging messages between the host and the module as described in section 7.2.2 of revision 4.0 of the CMIS standard.
According to the CMIS standard, the firmware update process is done using a CDB commands sequence.
CDB (Command Data Block Message Communication) reads and writes are performed on memory map pages 9Fh-AFh according to the CMIS standard, section 8.12 of revision 4.0.
Add a pair of new ethtool messages that allow:
* User space to trigger firmware update of transceiver modules
* The kernel to notify user space about the progress of the process
The user interface is designed to be asynchronous in order to avoid RTNL being held for too long and to allow several modules to be updated simultaneously. The interface is designed with CMIS compliant modules in mind, but kept generic enough to accommodate future use cases, if these arise.
The kernel interface that will implement the firmware update using CDB command will include 2 layers that will be added under ethtool:
* The upper layer that will be triggered from the module layer, is cmis_ fw_update.
* The lower one is cmis_cdb.
In the future there might be more operations to implement using CDB commands. Therefore, the idea is to keep the cmis_cdb interface clean and the cmis_fw_update specific to the cdb commands handling it.
The communication between the kernel and the driver will be done using two ethtool operations that enable reading and writing the transceiver module EEPROM.
The operation ethtool_ops::get_module_eeprom_by_page, that is already implemented, will be used for reading from the EEPROM the CDB reply, e.g. reading module setting, state, etc.
The operation ethtool_ops::set_module_eeprom_by_page, that is added in the current patchset, will be used for writing to the EEPROM the CDB command such as start firmware image, run firmware image, etc.
Therefore in order for a driver to implement module flashing, that driver needs to implement the two functions mentioned above.
Patchset overview:
Patch #1-#2: Implement the EEPROM writing in mlxsw. Patch #3: Define the interface between the kernel and user space. Patch #4: Add ability to notify the flashing firmware progress. Patch #5: Veto operations during flashing.
Patch #6: Add extended compliance codes.
Patch #7: Add the cdb layer.
Patch #8: Add the fw_update layer.
Patch #9: Add ability to flash transceiver modules' firmware.
v7:
Patch #4:
* Return -ENOMEM instead of PTR_ERR(attr) on ethnl_module_fw_flash_ntf_put_err().
Patch #9:
* Fix Warning for not unlocking the spin_lock in the error flow on module_flash_fw_work_list_add().
* Avoid the fall-through on ethnl_sock_priv_destroy().
v6:
* Squash some of the last patch to patch #5 and patch #9.
Patch #3:
* Add paragraph in .rst file.
Patch #4:
* Reserve '1' more place on SKB for NUL terminator in
the error message string.
* Add more prints on error flow, re-write the printing function and add ethnl_module_fw_flash_ntf_put_err().
* Change the communication method so notification will be sent in unicast instead of multicast.
* Add new 'struct ethnl_module_fw_flash_ntf_params' that holds the relevant info for unicast communication and use it to send notification to the specific socket.
* s/nla_put_u64_64bit/nla_put_uint/
Patch #7:
* In ethtool_cmis_cdb_init(), Use 'const' for the 'params' parameter.
Patch #8:
* Add a list field to struct ethtool_module_fw_flash for module_fw_flash_work_list that will be presented in the next patch.
* Move ethtool_cmis_fw_update() cleaning to a new function that will be represented in the next patch.
* Move some of the fields in struct ethtool_module_fw_flash to a separate struct, so ethtool_cmis_fw_update() will get only the relevant parameters for it.
* Edit the relevant functions to get the relevant params for them.
* s/CMIS_MODULE_READY_MAX_DURATION_USEC/CMIS_MODULE_READY_MAX_DURATION_MSEC
Patch #9:
* Add a paragraph in the commit message.
* Rename labels in module_flash_fw_schedule().
* Add info to genl_sk_priv_*() and implement the relevant callbacks, in order to handle properly a scenario of closing the socket from user space before the work item was ended.
* Add a list the holds all the ethtool_module_fw_flash struct that corresponds to the in progress work items.
* Add a new enum for the socket types.
* Use both above to identify a flashing socket, add it to the list and when closing socket affect only the flashing type.
* Create a new function that will get the work item instead of ethtool_cmis_fw_update().
* Edit the relevant functions to get the relevant params for them.
* The new function will call the old ethtool_cmis_fw_update(), and do the cleaning, so the existence of the list should be completely isolated in module.c.
Danielle Ratson (7):
ethtool: Add an interface for flashing transceiver modules' firmware
ethtool: Add flashing transceiver modules' firmware notifications
ability
ethtool: Veto some operations during firmware flashing process
net: sfp: Add more extended compliance codes
ethtool: cmis_cdb: Add a layer for supporting CDB commands
ethtool: cmis_fw_update: add a layer for supporting firmware update
using CDB
ethtool: Add ability to flash transceiver modules' firmware
Ido Schimmel (2):
ethtool: Add ethtool operation to write to a transceiver module EEPROM
mlxsw: Implement ethtool operation to write to a transceiver module
EEPROM
Documentation/netlink/specs/ethtool.yaml | 55 ++ Documentation/networking/ethtool-netlink.rst | 70 +++ .../net/ethernet/mellanox/mlxsw/core_env.c | 57 ++ .../net/ethernet/mellanox/mlxsw/core_env.h | 6 + drivers/net/ethernet/mellanox/mlxsw/minimal.c | 15 + .../mellanox/mlxsw/spectrum_ethtool.c | 15 + include/linux/ethtool.h | 20 +- include/linux/netdevice.h | 4 +- include/linux/sfp.h | 6 + include/uapi/linux/ethtool.h | 18 + include/uapi/linux/ethtool_netlink.h | 19 + net/ethtool/Makefile | 2 +- net/ethtool/cmis.h | 124 ++++ net/ethtool/cmis_cdb.c | 581 ++++++++++++++++++ net/ethtool/cmis_fw_update.c | 399 ++++++++++++ net/ethtool/eeprom.c | 6 + net/ethtool/ioctl.c | 12 + net/ethtool/module.c | 394 ++++++++++++ net/ethtool/module_fw.h | 75 +++ net/ethtool/netlink.c | 56 ++ net/ethtool/netlink.h | 16 + tools/net/ynl/Makefile.deps | 3 +- 22 files changed, 1942 insertions(+), 11 deletions(-) create mode 100644 net/ethtool/cmis.h create mode 100644 net/ethtool/cmis_cdb.c create mode 100644 net/ethtool/cmis_fw_update.c create mode 100644 net/ethtool/module_fw.h
Signature · 1 line
2.45.0
PATCH v7 1/9 ethtool: Add ethtool operation to write to a transceiver module EEPROM
From: Ido Schimmel <[email protected]>
Ethtool can already retrieve information from a transceiver module EEPROM by invoking the ethtool_ops::get_module_eeprom_by_page operation. Add a corresponding operation that allows ethtool to write to a transceiver module EEPROM.
The new write operation is purely an in-kernel API and is not exposed to user space.
The purpose of this operation is not to enable arbitrary read / write access, but to allow the kernel to write to specific addresses as part of transceiver module firmware flashing. In the future, more functionality can be implemented on top of these read / write operations.
Adjust the comments of the 'ethtool_module_eeprom' structure as it is no longer used only for read access.
--- include/linux/ethtool.h | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
include/linux/ethtool.h +12 -8
@@ -504,17 +504,16 @@ struct ethtool_ts_stats {#define ETH_MODULE_MAX_I2C_ADDRESS 0x7f/** -* struct ethtool_module_eeprom - EEPROM dump from specified page -* @offset: Offset within the specified EEPROM page to begin read, in bytes. -* @length: Number of bytes to read. -* @page: Page number to read from. -* @bank: Page bank number to read from, if applicable by EEPROM spec. +* struct ethtool_module_eeprom - plug-in module EEPROM read / write parameters +* @offset: When @offset is 0-127, it is used as an address to the Lower Memory +* (@page must be 0). Otherwise, it is used as an address to the +* Upper Memory. +* @length: Number of bytes to read / write. +* @page: Page number. +* @bank: Bank number, if supported by EEPROM spec. * @i2c_address: I2C address of a page. Value less than 0x7f expected. Most * EEPROMs use 0x50 or 0x51. * @data: Pointer to buffer with EEPROM data of @length size. -* -* This can be used to manage pages during EEPROM dump in ethtool and pass -* required information to the driver. */ struct ethtool_module_eeprom {u32 offset;
@@ -822,6 +821,8 @@ struct ethtool_rxfh_param {* @get_module_eeprom_by_page: Get a region of plug-in module EEPROM data from * specified page. Returns a negative error code or the amount of bytes * read. +* @set_module_eeprom_by_page: Write to a region of plug-in module EEPROM, +* from kernel space only. Returns a negative error code or zero. * @get_eth_phy_stats: Query some of the IEEE 802.3 PHY statistics. * @get_eth_mac_stats: Query some of the IEEE 802.3 MAC statistics. * @get_eth_ctrl_stats: Query some of the IEEE 802.3 MAC Ctrl statistics.
@@ -956,6 +957,9 @@ struct ethtool_ops {int (*get_module_eeprom_by_page)(struct net_device *dev,const struct ethtool_module_eeprom *page,struct netlink_ext_ack *extack); +int (*set_module_eeprom_by_page)(struct net_device *dev, +const struct ethtool_module_eeprom *page, +struct netlink_ext_ack *extack);void (*get_eth_phy_stats)(struct net_device *dev,struct ethtool_eth_phy_stats *phy_stats); void (*get_eth_mac_stats)(struct net_device *dev,
Signature · 1 line
2.45.0
Re: [PATCH net-next v7 1/9] ethtool: Add ethtool operation to write to a transceiver module EEPROM
Quoted 16 lines · Danielle Ratson
From: Ido Schimmel <[email protected]>
Ethtool can already retrieve information from a transceiver module EEPROM by invoking the
ethtool_ops::get_module_eeprom_by_pageoperation. Add a corresponding operation that allows ethtool to write to a transceiver module EEPROM.The new write operation is purely an in-kernel API and is not exposed to user space.
The purpose of this operation is not to enable arbitrary read / write access, but to allow the kernel to write to specific addresses as part of transceiver module firmware flashing. In the future, more functionality can be implemented on top of these read / write operations.
Quoted 6 lines
Adjust the comments of the '
ethtool_module_eeprom' structure as it is no longer used only for read access.Signed-off-by Ido Schimmel <[email protected]>Signed-off-by Danielle Ratson <[email protected]>Reviewed-by Petr Machata <[email protected]>
Andrew
PATCH v7 2/9 mlxsw: Implement ethtool operation to write to a transceiver module EEPROM
In reply to Danielle RatsonFrom: Ido Schimmel <[email protected]>
Implement the ethtool_ops::set_module_eeprom_by_page operation to allow ethtool to write to a transceiver module EEPROM, in a similar fashion to the ethtool_ops::get_module_eeprom_by_page operation.
--- .../net/ethernet/mellanox/mlxsw/core_env.c | 57 +++++++++++++++++++ .../net/ethernet/mellanox/mlxsw/core_env.h | 6 ++ drivers/net/ethernet/mellanox/mlxsw/minimal.c | 15 +++++ .../mellanox/mlxsw/spectrum_ethtool.c | 15 +++++ 4 files changed, 93 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/core_env.c +57 -0
@@ -513,6 +513,63 @@ mlxsw_env_get_module_eeprom_by_page(struct mlxsw_core *mlxsw_core, }EXPORT_SYMBOL (mlxsw_env_get_module_eeprom_by_page); +int +mlxsw_env_set_module_eeprom_by_page (struct mlxsw_core *mlxsw_core, +u8 slot_index,u8 module, +const struct ethtool_module_eeprom *page, +struct netlink_ext_ack *extack) +{ +struct mlxsw_env *mlxsw_env =mlxsw_core_env (mlxsw_core); +u32 bytes_written =0 ; +u16 device_addr; +int err; + +if (!mlxsw_env_linecard_is_active (mlxsw_env, slot_index)) { +NL_SET_ERR_MSG_MOD (extack, +"Cannot write to EEPROM of a module on an inactive line card" ); +return -EIO ; + } + + err =mlxsw_env_validate_module_type (mlxsw_core, slot_index, module); +if (err) { +NL_SET_ERR_MSG_MOD (extack,"EEPROM is not equipped on port module type" ); +return err; + } + + device_addr = page->offset; + +while (bytes_written < page->length) { +char mcia_pl[MLXSW_REG_MCIA_LEN ]; +char eeprom_tmp[128 ] = {}; +u8 size; + + size =min_t (u8, page->length - bytes_written, + mlxsw_env->max_eeprom_len); + +mlxsw_reg_mcia_pack (mcia_pl, slot_index, module, page->page, + device_addr + bytes_written, size, + page->i2c_address); +mlxsw_reg_mcia_bank_number_set (mcia_pl, page->bank); +memcpy (eeprom_tmp, page->data + bytes_written, size); +mlxsw_reg_mcia_eeprom_memcpy_to (mcia_pl, eeprom_tmp); + + err =mlxsw_reg_write (mlxsw_core,MLXSW_REG (mcia), mcia_pl); +if (err) { +NL_SET_ERR_MSG_MOD (extack,"Failed to access module's EEPROM" ); +return err; + } + + err =mlxsw_env_mcia_status_process (mcia_pl, extack); +if (err) +return err; + + bytes_written += size; + } + +return 0 ; +} +EXPORT_SYMBOL (mlxsw_env_set_module_eeprom_by_page); +static int mlxsw_env_module_reset (struct mlxsw_core *mlxsw_core,u8 slot_index,u8 module) {
drivers/net/ethernet/mellanox/mlxsw/core_env.h +6 -0
@@ -28,6 +28,12 @@ mlxsw_env_get_module_eeprom_by_page(struct mlxsw_core *mlxsw_core,const struct ethtool_module_eeprom *page, struct netlink_ext_ack *extack); +int +mlxsw_env_set_module_eeprom_by_page (struct mlxsw_core *mlxsw_core, +u8 slot_index,u8 module, +const struct ethtool_module_eeprom *page, +struct netlink_ext_ack *extack); +int mlxsw_env_reset_module (struct net_device *netdev,struct mlxsw_core *mlxsw_core,u8 slot_index,u8 module,u32 *flags);
drivers/net/ethernet/mellanox/mlxsw/minimal.c +15 -0
@@ -140,6 +140,20 @@ mlxsw_m_get_module_eeprom_by_page(struct net_device *netdev, page, extack); } +static int +mlxsw_m_set_module_eeprom_by_page (struct net_device *netdev, +const struct ethtool_module_eeprom *page, +struct netlink_ext_ack *extack) +{ +struct mlxsw_m_port *mlxsw_m_port =netdev_priv (netdev); +struct mlxsw_core *core = mlxsw_m_port->mlxsw_m->core; + +return mlxsw_env_set_module_eeprom_by_page (core, + mlxsw_m_port->slot_index, + mlxsw_m_port->module, + page, extack); +} +static int mlxsw_m_reset (struct net_device *netdev,u32 *flags) {struct mlxsw_m_port *mlxsw_m_port =netdev_priv (netdev);
@@ -181,6 +195,7 @@ static const struct ethtool_ops mlxsw_m_port_ethtool_ops = { .get_module_info = mlxsw_m_get_module_info, .get_module_eeprom = mlxsw_m_get_module_eeprom, .get_module_eeprom_by_page = mlxsw_m_get_module_eeprom_by_page, + .set_module_eeprom_by_page = mlxsw_m_set_module_eeprom_by_page, .reset = mlxsw_m_reset, .get_module_power_mode = mlxsw_m_get_module_power_mode, .set_module_power_mode = mlxsw_m_set_module_power_mode,
drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c +15 -0
@@ -1067,6 +1067,20 @@ mlxsw_sp_get_module_eeprom_by_page(struct net_device *dev, module, page, extack); } +static int +mlxsw_sp_set_module_eeprom_by_page (struct net_device *dev, +const struct ethtool_module_eeprom *page, +struct netlink_ext_ack *extack) +{ +struct mlxsw_sp_port *mlxsw_sp_port =netdev_priv (dev); +struct mlxsw_sp *mlxsw_sp = mlxsw_sp_port->mlxsw_sp; +u8 slot_index = mlxsw_sp_port->mapping.slot_index; +u8 module = mlxsw_sp_port->mapping.module; + +return mlxsw_env_set_module_eeprom_by_page (mlxsw_sp->core, slot_index, + module, page, extack); +} +static int mlxsw_sp_get_ts_info (struct net_device *netdev,struct ethtool_ts_info *info) {
@@ -1256,6 +1270,7 @@ const struct ethtool_ops mlxsw_sp_port_ethtool_ops = { .get_module_info = mlxsw_sp_get_module_info, .get_module_eeprom = mlxsw_sp_get_module_eeprom, .get_module_eeprom_by_page = mlxsw_sp_get_module_eeprom_by_page, + .set_module_eeprom_by_page = mlxsw_sp_set_module_eeprom_by_page, .get_ts_info = mlxsw_sp_get_ts_info, .get_eth_phy_stats = mlxsw_sp_get_eth_phy_stats, .get_eth_mac_stats = mlxsw_sp_get_eth_mac_stats,
Signature · 1 line
2.45.0
Re: [PATCH net-next v7 2/9] mlxsw: Implement ethtool operation to write to a transceiver module EEPROM
Quoted 8 lines · Danielle Ratson
From: Ido Schimmel <[email protected]>
Implement the
ethtool_ops::set_module_eeprom_by_pageoperation to allow ethtool to write to a transceiver module EEPROM, in a similar fashion to theethtool_ops::get_module_eeprom_by_pageoperation.Signed-off-by Ido Schimmel <[email protected]>Reviewed-by Petr Machata <[email protected]>
Andrew
PATCH v7 3/9 ethtool: Add an interface for flashing transceiver modules' firmware
In reply to Danielle RatsonCMIS compliant modules such as QSFP-DD might be running a firmware that can be updated in a vendor-neutral way by exchanging messages between the host and the module as described in section 7.3.1 of revision 5.2 of the CMIS standard.
Add a pair of new ethtool messages that allow:
* User space to trigger firmware update of transceiver modules
* The kernel to notify user space about the progress of the process
The user interface is designed to be asynchronous in order to avoid RTNL being held for too long and to allow several modules to be updated simultaneously. The interface is designed with CMIS compliant modules in mind, but kept generic enough to accommodate future use cases, if these arise.
---
Notes:
v6:
* Add paragraph in .rst file.
v5:
* Modify tools/net/ynl/Makefile.deps so the ynl file will include the ethtool.h changes.
* u64>uint for 'total' and 'done' attrs.
* Translate the enum from ethtool_netlink.h to YAML.
v4:
* s/is composed from/consists of/.
Documentation/netlink/specs/ethtool.yaml | 55 +++++++++++++++ Documentation/networking/ethtool-netlink.rst | 70 ++++++++++++++++++++ include/uapi/linux/ethtool.h | 18 +++++ include/uapi/linux/ethtool_netlink.h | 19 ++++++ tools/net/ynl/Makefile.deps | 3 +- 5 files changed, 164 insertions(+), 1 deletion(-)
Documentation/netlink/specs/ethtool.yaml +55 -0
@@ -20,6 +20,10 @@ definitions: name: header-flags type: flags entries: [ compact-bitsets, omit-reply, stats ] + - + name: module-fw-flash-status + type: enum + entries: [ started, in_progress, completed, error ] attribute-sets: -
@@ -975,6 +979,32 @@ attribute-sets: - name: burst-tmr type: u32 + - + name: module-fw-flash + attributes: + - + name: header + type: nest + nested-attributes: header + - + name: file-name + type: string + - + name: password + type: u32 + - + name: status + type: u32 + enum: module-fw-flash-status + - + name: status-msg + type: string + - + name: done + type: uint + - + name: total + type: uint operations: enum-model: directional
@@ -1730,3 +1760,28 @@ operations: name: mm-ntf doc: Notification for change in MAC Merge configuration. notify: mm-get + - + name: module-fw-flash-act + doc: Flash transceiver module firmware. + + attribute-set: module-fw-flash + + do: + request: + attributes: + - header + - file-name + - password + - + name: module-fw-flash-ntf + doc: Notification for firmware flashing progress and status. + + attribute-set: module-fw-flash + + event: + attributes: + - header + - status + - status-msg + - done + - total
Documentation/networking/ethtool-netlink.rst +70 -0
92 lines not on this page · raw message
include/uapi/linux/ethtool.h +18 -0
@@ -877,6 +877,24 @@ enum ethtool_mm_verify_status { ETHTOOL_MM_VERIFY_STATUS_DISABLED, }; +/** +* enum ethtool_module_fw_flash_status - plug-in module firmware flashing status +* @ETHTOOL_MODULE_FW_FLASH_STATUS_STARTED: The firmware flashing process has +* started. +* @ETHTOOL_MODULE_FW_FLASH_STATUS_IN_PROGRESS: The firmware flashing process +* is in progress. +* @ETHTOOL_MODULE_FW_FLASH_STATUS_COMPLETED: The firmware flashing process was +* completed successfully. +* @ETHTOOL_MODULE_FW_FLASH_STATUS_ERROR: The firmware flashing process was +* stopped due to an error. +*/ +enum ethtool_module_fw_flash_status { +ETHTOOL_MODULE_FW_FLASH_STATUS_STARTED =1 , +ETHTOOL_MODULE_FW_FLASH_STATUS_IN_PROGRESS , +ETHTOOL_MODULE_FW_FLASH_STATUS_COMPLETED , +ETHTOOL_MODULE_FW_FLASH_STATUS_ERROR , +}; +/** * struct ethtool_gstrings - string set for data tagging * @cmd: Command number = %ETHTOOL_GSTRINGS
include/uapi/linux/ethtool_netlink.h +19 -0
@@ -57,6 +57,7 @@ enum {ETHTOOL_MSG_PLCA_GET_STATUS ,ETHTOOL_MSG_MM_GET ,ETHTOOL_MSG_MM_SET , +ETHTOOL_MSG_MODULE_FW_FLASH_ACT ,/* add new constants above here */ __ETHTOOL_MSG_USER_CNT,
@@ -109,6 +110,7 @@ enum {ETHTOOL_MSG_PLCA_NTF ,ETHTOOL_MSG_MM_GET_REPLY ,ETHTOOL_MSG_MM_NTF , +ETHTOOL_MSG_MODULE_FW_FLASH_NTF ,/* add new constants above here */ __ETHTOOL_MSG_KERNEL_CNT,
@@ -996,6 +998,23 @@ enum { ETHTOOL_A_MM_MAX = (__ETHTOOL_A_MM_CNT - 1) }; +/* MODULE_FW_FLASH */ + +enum { +ETHTOOL_A_MODULE_FW_FLASH_UNSPEC , +ETHTOOL_A_MODULE_FW_FLASH_HEADER ,/* nest - _A_HEADER_* */ +ETHTOOL_A_MODULE_FW_FLASH_FILE_NAME ,/* string */ +ETHTOOL_A_MODULE_FW_FLASH_PASSWORD ,/* u32 */ +ETHTOOL_A_MODULE_FW_FLASH_STATUS ,/* u32 */ +ETHTOOL_A_MODULE_FW_FLASH_STATUS_MSG ,/* string */ +ETHTOOL_A_MODULE_FW_FLASH_DONE ,/* uint */ +ETHTOOL_A_MODULE_FW_FLASH_TOTAL ,/* uint */ + +/* add new constants above here */ + __ETHTOOL_A_MODULE_FW_FLASH_CNT, +ETHTOOL_A_MODULE_FW_FLASH_MAX = (__ETHTOOL_A_MODULE_FW_FLASH_CNT -1 ) +}; +/* generic netlink info */ #define ETHTOOL_GENL_NAME "ethtool"#define ETHTOOL_GENL_VERSION 1
tools/net/ynl/Makefile.deps +2 -1
@@ -16,7 +16,8 @@ get_hdr_inc=-D$(1) -include $(UAPI_PATH)/linux/$(2) CFLAGS_devlink:=$(call get_hdr_inc,_LINUX_DEVLINK_H_,devlink.h) CFLAGS_dpll:=$(call get_hdr_inc,_LINUX_DPLL_H,dpll.h) -CFLAGS_ethtool:=$(call get_hdr_inc,_LINUX_ETHTOOL_NETLINK_H_,ethtool_netlink.h) +CFLAGS_ethtool:=$(call get_hdr_inc,_LINUX_ETHTOOL_H,ethtool.h) \ + $(call get_hdr_inc,_LINUX_ETHTOOL_NETLINK_H_,ethtool_netlink.h) CFLAGS_handshake:=$(call get_hdr_inc,_LINUX_HANDSHAKE_H,handshake.h) CFLAGS_mptcp_pm:=$(call get_hdr_inc,_LINUX_MPTCP_PM_H,mptcp_pm.h) CFLAGS_netdev:=$(call get_hdr_inc,_LINUX_NETDEV_H,netdev.h)
Signature · 1 line
2.45.0
Re: [PATCH net-next v7 3/9] ethtool: Add an interface for flashing transceiver modules' firmware
Quoted 11 lines · Danielle Ratson
CMIS compliant modules such as QSFP-DD might be running a firmware that can be updated in a vendor-neutral way by exchanging messages between the host and the module as described in section 7.3.1 of revision 5.2 of the CMIS standard.
Add a pair of new ethtool messages that allow:
* User space to trigger firmware update of transceiver modules
* The kernel to notify user space about the progress of the process
Quoted 8 lines
The user interface is designed to be asynchronous in order to avoid RTNL being held for too long and to allow several modules to be updated simultaneously. The interface is designed with CMIS compliant modules in mind, but kept generic enough to accommodate future use cases, if these arise.
Signed-off-by Danielle Ratson <[email protected]>Reviewed-by Petr Machata <[email protected]>
Andrew
PATCH v7 4/9 ethtool: Add flashing transceiver modules' firmware notifications ability
In reply to Danielle RatsonAdd progress notifications ability to user space while flashing modules' firmware by implementing the interface between the user space and the kernel.
---
Notes:
v7:
* Return -ENOMEM instead of PTR_ERR(attr) on ethnl_module_fw_flash_ntf_put_err()ץ
v6:
* Reserve '1' more place on SKB for NUL terminator in the error message string.
* Add more prints on error flow, re-write the printing function and add ethnl_module_fw_flash_ntf_put_err().
* Change the communication method so notification will be sent in unicast instead of multicast.
* Add new 'struct ethnl_module_fw_flash_ntf_params' that holds the relevant info for unicast communication and use it to send notification to the specific socket.
* s/nla_put_u64_64bit/nla_put_uint/
v2:
* Increase err_msg length.
v6:
* Reserve '1' more place on SKB for NUL terminator in the error message string.
* Add more prints on error flow, re-write the printing function and add ethnl_module_fw_flash_ntf_put_err().
* Change the communication method so notification will be sent in unicast instead of multicast.
* Add new 'struct ethnl_module_fw_flash_ntf_params' that holds the relevant info for unicast communication and use it to send notification to the specific socket.
* s/nla_put_u64_64bit/nla_put_uint/
v2:
* Increase err_msg length.
net/ethtool/module.c | 117 ++++++++++++++++++++++++++++++++++++++++ net/ethtool/module_fw.h | 31 +++++++++++ net/ethtool/netlink.c | 5 ++ net/ethtool/netlink.h | 1 + 4 files changed, 154 insertions(+) create mode 100644 net/ethtool/module_fw.h
net/ethtool/module.c +117 -0
@@ -5,6 +5,7 @@#include "netlink.h" #include "common.h" #include "bitset.h" +#include "module_fw.h" struct module_req_info {struct ethnl_req_info base;
@@ -158,3 +159,119 @@ const struct ethnl_request_ops ethnl_module_request_ops = { .set = ethnl_set_module, .set_ntf_cmd = ETHTOOL_MSG_MODULE_NTF, }; + +/* MODULE_FW_FLASH_NTF */ + +static int +ethnl_module_fw_flash_ntf_put_err (struct sk_buff *skb,char *err_msg, +char *sub_err_msg) +{ +int err_msg_len, sub_err_msg_len, total_len; +struct nlattr *attr; + +if (!err_msg) +return 0 ; + + err_msg_len =strlen (err_msg); + total_len = err_msg_len +2 ;/* For period and NUL. */ + +if (sub_err_msg) { + sub_err_msg_len =strlen (sub_err_msg); + total_len += sub_err_msg_len +2 ;/* For ", ". */ + } + + attr =nla_reserve (skb,ETHTOOL_A_MODULE_FW_FLASH_STATUS_MSG , + total_len); +if (!attr) +return -ENOMEM ; + +if (sub_err_msg) +sprintf (nla_data (attr),"%s, %s." , err_msg, sub_err_msg); +else +sprintf (nla_data (attr),"%s." , err_msg); + +return 0 ; +} + +static void +ethnl_module_fw_flash_ntf (struct net_device *dev, +enum ethtool_module_fw_flash_status status, +struct ethnl_module_fw_flash_ntf_params *ntf_params, +char *err_msg,char *sub_err_msg, +u64 done,u64 total) +{ +struct sk_buff *skb; +void *hdr; +int ret; + +if (ntf_params->closed_sock) +return ; + + skb =genlmsg_new (NLMSG_GOODSIZE ,GFP_KERNEL ); +if (!skb) +return ; + + hdr =ethnl_unicast_put (skb, ntf_params->portid, ntf_params->seq, +ETHTOOL_MSG_MODULE_FW_FLASH_NTF ); +if (!hdr) + goto err_skb; + + ret =ethnl_fill_reply_header (skb, dev, +ETHTOOL_A_MODULE_FW_FLASH_HEADER ); +if (ret <0 ) + goto err_skb; + +if (nla_put_u32 (skb,ETHTOOL_A_MODULE_FW_FLASH_STATUS , status)) + goto err_skb; + + ret =ethnl_module_fw_flash_ntf_put_err (skb, err_msg, sub_err_msg); +if (ret <0 ) + goto err_skb; + +if (nla_put_uint (skb,ETHTOOL_A_MODULE_FW_FLASH_DONE , done)) + goto err_skb; + +if (nla_put_uint (skb,ETHTOOL_A_MODULE_FW_FLASH_TOTAL , total)) + goto err_skb; + +genlmsg_end (skb, hdr); +genlmsg_unicast (dev_net (dev), skb, ntf_params->portid); +return ; + +err_skb: +nlmsg_free (skb); +} + +void ethnl_module_fw_flash_ntf_err (struct net_device *dev, +struct ethnl_module_fw_flash_ntf_params *params, +char *err_msg,char *sub_err_msg) +{ +ethnl_module_fw_flash_ntf (dev,ETHTOOL_MODULE_FW_FLASH_STATUS_ERROR , + params, err_msg, sub_err_msg,0 ,0 ); +} + +void +ethnl_module_fw_flash_ntf_start (struct net_device *dev, +struct ethnl_module_fw_flash_ntf_params *params) +{ +ethnl_module_fw_flash_ntf (dev,ETHTOOL_MODULE_FW_FLASH_STATUS_STARTED , + params,NULL ,NULL ,0 ,0 ); +} + +void +ethnl_module_fw_flash_ntf_complete (struct net_device *dev, +struct ethnl_module_fw_flash_ntf_params *params) +{ +ethnl_module_fw_flash_ntf (dev,ETHTOOL_MODULE_FW_FLASH_STATUS_COMPLETED , + params,NULL ,NULL ,0 ,0 ); +} + +void +ethnl_module_fw_flash_ntf_in_progress (struct net_device *dev, +struct ethnl_module_fw_flash_ntf_params *params, +u64 done,u64 total) +{ +ethnl_module_fw_flash_ntf (dev, +ETHTOOL_MODULE_FW_FLASH_STATUS_IN_PROGRESS , + params,NULL ,NULL , done, total); +}
net/ethtool/module_fw.h new file +31 -0
@@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include <uapi/linux/ethtool.h> + +/** +* struct ethnl_module_fw_flash_ntf_params - module firmware flashing +* notifications parameters +* @portid: Netlink portid of sender. +* @seq: Sequence number of sender. +* @closed_sock: Indicates whether the socket was closed from user space. +*/ +struct ethnl_module_fw_flash_ntf_params { +u32 portid; +u32 seq; +bool closed_sock; +}; + +void +ethnl_module_fw_flash_ntf_err (struct net_device *dev, +struct ethnl_module_fw_flash_ntf_params *params, +char *err_msg,char *sub_err_msg); +void +ethnl_module_fw_flash_ntf_start (struct net_device *dev, +struct ethnl_module_fw_flash_ntf_params *params); +void +ethnl_module_fw_flash_ntf_complete (struct net_device *dev, +struct ethnl_module_fw_flash_ntf_params *params); +void +ethnl_module_fw_flash_ntf_in_progress (struct net_device *dev, +struct ethnl_module_fw_flash_ntf_params *params, +u64 done,u64 total);
net/ethtool/netlink.c +5 -0
@@ -239,6 +239,11 @@ void *ethnl_bcastmsg_put(struct sk_buff *skb, u8 cmd) cmd); } +void *ethnl_unicast_put (struct sk_buff *skb,u32 portid,u32 seq,u8 cmd) +{ +return genlmsg_put (skb, portid, seq, ðtool_genl_family,0 , cmd); +} +int ethnl_multicast (struct sk_buff *skb,struct net_device *dev) { return genlmsg_multicast_netns(ðtool_genl_family, dev_net(dev), skb,
net/ethtool/netlink.h +1 -0
@@ -21,6 +21,7 @@ struct sk_buff *ethnl_reply_init(size_t payload, struct net_device *dev, u8 cmd, void **ehdrp);void *ethnl_dump_put (struct sk_buff *skb,struct netlink_callback *cb,u8 cmd);void *ethnl_bcastmsg_put (struct sk_buff *skb,u8 cmd); +void *ethnl_unicast_put (struct sk_buff *skb,u32 portid,u32 seq,u8 cmd);int ethnl_multicast (struct sk_buff *skb,struct net_device *dev);/**
Signature · 1 line
2.45.0
Re: [PATCH net-next v7 4/9] ethtool: Add flashing transceiver modules' firmware notifications ability
Quoted 6 lines · Danielle Ratson
Add progress notifications ability to user space while flashing modules' firmware by implementing the interface between the user space and the kernel.
Signed-off-by Danielle Ratson <[email protected]>Reviewed-by Petr Machata <[email protected]>
Andrew
PATCH v7 5/9 ethtool: Veto some operations during firmware flashing process
In reply to Danielle RatsonSome operations cannot be performed during the firmware flashing process.
For example:
- Port must be down during the whole flashing process to avoid packet loss while committing reset for example.
- Writing to EEPROM interrupts the flashing process, so operations like ethtool dump, module reset, get and set power mode should be vetoed.
- Split port firmware flashing should be vetoed.
In order to veto those scenarios, add a flag in 'struct net_device' that indicates when a firmware flash is taking place on the module and use it to prevent interruptions during the process.
---
Notes:
v6:
* Squash some of the vetoes from the last patch to this patch.
include/linux/netdevice.h | 4 +++- net/ethtool/eeprom.c | 6 ++++++ net/ethtool/ioctl.c | 12 ++++++++++++ net/ethtool/netlink.c | 12 ++++++++++++ 4 files changed, 33 insertions(+), 1 deletion(-)
include/linux/netdevice.h +3 -1
@@ -1989,6 +1989,8 @@ enum netdev_reg_state {* * @threaded: napi threaded mode is enabled * +* @module_fw_flash_in_progress: Module firmware flashing is in progress. +* * @net_notifier_list: List of per-net netdev notifier block * that follow this device when it is moved * to another network namespace.
@@ -2373,7 +2375,7 @@ struct net_device {bool proto_down;bool threaded; unsigned wol_enabled:1; - + unsigned module_fw_flash_in_progress:1;struct list_head net_notifier_list;#if IS_ENABLED (CONFIG_MACSEC )
net/ethtool/eeprom.c +6 -0
@@ -91,6 +91,12 @@ static int get_module_eeprom_by_page(struct net_device *dev, {const struct ethtool_ops *ops = dev->ethtool_ops; +if (dev->module_fw_flash_in_progress) { +NL_SET_ERR_MSG (extack, +"Module firmware flashing is in progress" ); +return -EBUSY ; + } +if (dev->sfp_bus)return sfp_get_module_eeprom_by_page (dev->sfp_bus, page_data, extack);
net/ethtool/ioctl.c +12 -0
@@ -658,6 +658,9 @@ static int ethtool_get_settings(struct net_device *dev, void __user *useraddr)if (!dev->ethtool_ops->get_link_ksettings)return -EOPNOTSUPP ; +if (dev->module_fw_flash_in_progress) +return -EBUSY ; +memset (&link_ksettings,0 ,sizeof (link_ksettings)); err = dev->ethtool_ops->get_link_ksettings (dev, &link_ksettings);if (err <0 )
@@ -1449,6 +1452,9 @@ static int ethtool_reset(struct net_device *dev, char __user *useraddr)if (!dev->ethtool_ops->reset)return -EOPNOTSUPP ; +if (dev->module_fw_flash_in_progress) +return -EBUSY ; +if (copy_from_user (&reset, useraddr,sizeof (reset)))return -EFAULT ;
@@ -2462,6 +2468,9 @@ int ethtool_get_module_info_call(struct net_device *dev,const struct ethtool_ops *ops = dev->ethtool_ops;struct phy_device *phydev = dev->phydev; +if (dev->module_fw_flash_in_progress) +return -EBUSY ; +if (dev->sfp_bus)return sfp_get_module_info (dev->sfp_bus, modinfo);
@@ -2499,6 +2508,9 @@ int ethtool_get_module_eeprom_call(struct net_device *dev,const struct ethtool_ops *ops = dev->ethtool_ops;struct phy_device *phydev = dev->phydev; +if (dev->module_fw_flash_in_progress) +return -EBUSY ; +if (dev->sfp_bus)return sfp_get_module_eeprom (dev->sfp_bus, ee, data);
net/ethtool/netlink.c +12 -0
@@ -765,10 +765,22 @@ static void ethnl_notify_features(struct netdev_notifier_info *info)static int ethnl_netdev_event (struct notifier_block *this,unsigned long event,void *ptr) { +struct netdev_notifier_info *info = ptr; +struct netlink_ext_ack *extack; +struct net_device *dev; + + dev =netdev_notifier_info_to_dev (info); + extack =netdev_notifier_info_to_extack (info); +switch (event) {case NETDEV_FEAT_CHANGE :ethnl_notify_features (ptr);break ; +case NETDEV_PRE_UP : +if (dev->module_fw_flash_in_progress) { +NL_SET_ERR_MSG (extack,"Can't set port up while flashing module firmware" ); +return NOTIFY_BAD ; + } }return NOTIFY_DONE ;
Signature · 1 line
2.45.0
Re: [PATCH net-next v7 5/9] ethtool: Veto some operations during firmware flashing process
Quoted 11 lines · Danielle Ratson
Some operations cannot be performed during the firmware flashing process.
For example:
- Port must be down during the whole flashing process to avoid packet loss while committing reset for example.
- Writing to EEPROM interrupts the flashing process, so operations like ethtool dump, module reset, get and set power mode should be vetoed.
Quoted 8 lines
- Split port firmware flashing should be vetoed.
In order to veto those scenarios, add a flag in 'struct
net_device' that indicates when a firmware flash is taking place on the module and use it to prevent interruptions during the process.Signed-off-by Danielle Ratson <[email protected]>Reviewed-by Petr Machata <[email protected]>
Andrew
PATCH v7 6/9 net: sfp: Add more extended compliance codes
In reply to Danielle RatsonSFF-8024 is used to define various constants re-used in several SFF SFP-related specifications.
Add SFF-8024 extended compliance code definitions for CMIS compliant modules and use them in the next patch to determine the firmware flashing work.
--- include/linux/sfp.h | 6 ++++++ 1 file changed, 6 insertions(+)
include/linux/sfp.h +6 -0
@@ -284,6 +284,12 @@ enum {SFF8024_ID_QSFP_8438 =0x0c ,SFF8024_ID_QSFP_8436_8636 =0x0d ,SFF8024_ID_QSFP28_8636 =0x11 , +SFF8024_ID_QSFP_DD =0x18 , +SFF8024_ID_OSFP =0x19 , +SFF8024_ID_DSFP =0x1B , +SFF8024_ID_QSFP_PLUS_CMIS =0x1E , +SFF8024_ID_SFP_DD_CMIS =0x1F , +SFF8024_ID_SFP_PLUS_CMIS =0x20 ,SFF8024_ENCODING_UNSPEC =0x00 , SFF8024_ENCODING_8B10B = 0x01,
Signature · 1 line
2.45.0
Re: [PATCH net-next v7 6/9] net: sfp: Add more extended compliance codes
Quoted 10 lines · Danielle Ratson
SFF-8024 is used to define various constants re-used in several SFF SFP-related specifications.
Add SFF-8024 extended compliance code definitions for CMIS compliant modules and use them in the next patch to determine the firmware flashing work.
Signed-off-by Danielle Ratson <[email protected]>Reviewed-by Petr Machata <[email protected]>Reviewed-by Russell King (Oracle) <[email protected]>
Andrew
PATCH v7 7/9 ethtool: cmis_cdb: Add a layer for supporting CDB commands
In reply to Danielle RatsonCDB (Command Data Block Message Communication) reads and writes are performed on memory map pages 9Fh-AFh according to the CMIS standard, section 8.20 of revision 5.2.
Page 9Fh is used to specify the CDB command to be executed and also provides an area for a local payload (LPL).
According to the CMIS standard, the firmware update process is done using a CDB commands sequence that will be implemented in the next patch.
The kernel interface that will implement the firmware update using CDB command will include 2 layers that will be added under ethtool:
* The upper layer that will be triggered from the module layer, is cmis_fw_update.
* The lower one is cmis_cdb.
In the future there might be more operations to implement using CDB commands. Therefore, the idea is to keep the CDB interface clean and the cmis_fw_update specific to the CDB commands handling it.
These two layers will communicate using the API the consists of three functions:
- struct ethtool_cmis_cdb *
ethtool_cmis_cdb_init(struct net_device *dev,
struct ethtool_module_fw_flash_params *params); - void ethtool_cmis_cdb_fini(struct ethtool_cmis_cdb *cdb); - int ethtool_cmis_cdb_execute_cmd(struct net_device *dev, struct ethtool_cmis_cdb_cmd_args *args);
Add the CDB layer to support initializing, finishing and executing CDB commands:
* The initialization process will include creating of an ethtool_cmis_cdb instance, querying the module CDB support, entering and validating the password from user space (CMD 0x0000) and querying the module features (CMD 0x0040).
* The finishing API will simply free the ethtool_cmis_cdb instance.
* The executing process will write the CDB command to EEPROM using set_module_eeprom_by_page() that was presented earlier, and will process the reply from EEPROM.
---
Notes:
v6:
* In ethtool_cmis_cdb_init(), Use 'const' for the params
parameter.
v5:
* Drop all the inline in cmis_cdb.c.
v4:
* Add kernel-doc for msleep_pre_rpl and err_msg.
v3:
* Use kmemdup() instead of kmalloc+memcpy.
v2:
* Define ethtool_cmis_cdb_request::epl_len to be __be16 instead
of u16.net/ethtool/Makefile | 2 +- net/ethtool/cmis.h | 117 ++++++++ net/ethtool/cmis_cdb.c | 581 ++++++++++++++++++++++++++++++++++++++++ net/ethtool/module_fw.h | 10 + 4 files changed, 709 insertions(+), 1 deletion(-) create mode 100644 net/ethtool/cmis.h create mode 100644 net/ethtool/cmis_cdb.c
net/ethtool/Makefile +1 -1
@@ -8,4 +8,4 @@ ethtool_nl-y := netlink.o bitset.o strset.o linkinfo.o linkmodes.o rss.o \ linkstate.o debug.o wol.o features.o privflags.o rings.o \ channels.o coalesce.o pause.o eee.o tsinfo.o cabletest.o \ tunnels.o fec.o eeprom.o stats.o phc_vclocks.o mm.o \ - module.o pse-pd.o plca.o mm.o + module.o cmis_cdb.o pse-pd.o plca.o mm.o
net/ethtool/cmis.h new file +117 -0
117 lines not on this page · raw message
net/ethtool/cmis_cdb.c new file +581 -0
581 lines not on this page · raw message
net/ethtool/module_fw.h +10 -0
@@ -15,6 +15,16 @@ struct ethnl_module_fw_flash_ntf_params {bool closed_sock; }; +/** +* struct ethtool_module_fw_flash_params - module firmware flashing parameters +* @password: Module password. Only valid when @pass_valid is set. +* @password_valid: Whether the module password is valid or not. +*/ +struct ethtool_module_fw_flash_params { +__be32 password; +u8 password_valid:1 ; +}; + void ethnl_module_fw_flash_ntf_err(struct net_device *dev, struct ethnl_module_fw_flash_ntf_params *params,
Signature · 1 line
2.45.0
Andrew Lunn
maintainerRe: [PATCH net-next v7 7/9] ethtool: cmis_cdb: Add a layer for supporting CDB commands
Quoted 18 lines
net/ethtool/cmis_cdb.c new file +18
-0+int ethtool_cmis_wait_for_cond (struct net_device *dev,u8 flags,u8 flag, +u16 max_duration,u32 offset, +bool (*cond_success)(u8 ),bool (*cond_fail)(u8 ), +u8 *state) +{ +const struct ethtool_ops *ops = dev->ethtool_ops; +struct ethtool_module_eeprom page_data = {0 }; +struct cmis_wait_for_cond_rpl rpl = {}; +struct netlink_ext_ack extack = {}; +unsigned long end; +int err; + +if (!(flags & flag)) +return 0 ; + +if (max_duration ==0 ) + max_duration =U16_MAX ; +
net/ethtool/cmis_cdb.c line 361
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +10
-0+ end = jiffies +msecs_to_jiffies (max_duration); + do { + ethtool_cmis_page_init(&page_data, 0, offset, sizeof(rpl)); + page_data.data = (u8 *)&rpl; + + err = ops->get_module_eeprom_by_page(dev, &page_data, &extack); + if (err < 0) { + if (extack._msg) + netdev_err(dev, "%s\n", extack._msg); + continue;
continue here is interested. Say you get -EIO because the module has been ejected. I would say that is fatal. Won't this spam the logs, as fast as the I2C bus can fail, without the 20ms sleep, for 65535 jiffies?
net/ethtool/cmis_cdb.c line 371
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +10
-0+ } + +if ((*cond_success)(rpl.state)) +return 0 ; + +if (*cond_fail && (*cond_fail)(rpl.state)) +break ; + +msleep (20 ); + }while (time_before (jiffies, end));
Please could you implement this using iopoll.h. This appears to have the usual problem. Say msleep(20) actually sleeps a lot longer, because the system is busy doing other things. time_before(jiffies, end)) is false, because of the long delay, but in fact the operation has completed without error. Yet you return EBUSY. iopoll.h gets this correct, it does one more evaluation of the condition after exiting the loop to handle this issue.
net/ethtool/cmis_cdb.c line 529
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +10
-0+static u8 cmis_cdb_calc_checksum (const void *data,size_t size) +{ +const u8 *bytes = (const u8 *)data; +u8 checksum =0 ; + +for (size_t i =0 ; i < size; i++) + checksum += bytes[i]; + +return ~checksum; +}
I expect there is already a helper do that somewhere.
Andrew
---
pw-bot: cr
RE: [PATCH net-next v7 7/9] ethtool: cmis_cdb: Add a layer for supporting CDB commands
Hi Andrew,
Thanks for reviewing the patches.
Quoted 40 lines
From: Andrew Lunn <[email protected]>
Sent: Monday, 24 June 2024 22:51
To: Danielle Ratson <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; mlxsw <[email protected]>; Ido Schimmel <[email protected]>; Petr Machata <[email protected]>
Subject: Re: [PATCH net-next v7 7/9] ethtool:cmis_cdb: Add a layer for supporting CDB commandsQuoted 14 lines
net/ethtool/cmis_cdb.c new file +14
-0+int ethtool_cmis_wait_for_cond (struct net_device *dev,u8 flags,u8 flag, +u16 max_duration,u32 offset, +bool (*cond_success)(u8 ),bool (*cond_fail)(u8 ), +u8 *state) +{ +const struct ethtool_ops *ops = dev->ethtool_ops; +struct ethtool_module_eeprom page_data = {0 }; +struct cmis_wait_for_cond_rpl rpl = {}; +struct netlink_ext_ack extack = {}; +unsigned long end; +int err; + +if (!(flags & flag)) +return 0 ;net/ethtool/cmis_cdb.c
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +10
-0+ +if (max_duration ==0 ) + max_duration =U16_MAX ; + + end = jiffies +msecs_to_jiffies (max_duration); + do { + ethtool_cmis_page_init(&page_data, 0, offset, sizeof(rpl)); + page_data.data = (u8 *)&rpl; + + err = ops->get_module_eeprom_by_page(dev, &page_data,&extack);
Quoted 8 lines
net/ethtool/cmis_cdb.c line 361
Quoted 4 lines
net/ethtool/cmis_cdb.c new file +4
-0+if (err <0 ) { +if (extack._msg) +netdev_err (dev,"%s\n" , extack._msg); +continue ;continue here is interested. Say you get
-EIObecause the module has been ejected. I would say that is fatal. Won't this spam the logs, as fast as the I2C bus can fail, without the 20ms sleep, for 65535 jiffies?
If the module is ejected from some reason, it might span the logs I guess. But it is less likely than the scenario I wanted to cover. According to SPEC 5.2:
"
7.2.5.1 Foreground Mode CDB Messaging
[...]
In foreground mode the module rejects any register ACCESS until a currently executing CDB command execution has completed.
Note: READs of the CdbStatus registers 00h:37 or 00h:38 (see Table 8-13) will also be rejected by the module. "
So in that case the module won't be able to respond and we need to wait for it to be responsive and the status to be valid.
Quoted 12 lines
net/ethtool/cmis_cdb.c line 371
Quoted 11 lines
net/ethtool/cmis_cdb.c new file +10
-0+ } + +if ((*cond_success)(rpl.state)) +return 0 ; + +if (*cond_fail && (*cond_fail)(rpl.state)) +break ; + +msleep (20 ); + }while (time_before (jiffies, end));
Quoted 7 lines
Please could you implement this using
iopoll.h. This appears to have the usual problem. Say msleep(20) actually sleeps a lot longer, because the system is busy doing other things. time_before(jiffies,
end)) is false, because of the long delay, but in fact the operation has completed without error. Yet you return EBUSY.iopoll.hgets this correct, it does one more evaluation of the condition after exiting the loop to handle this issue.
OK.
Quoted 13 lines
net/ethtool/cmis_cdb.c line 529
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +9
-0+static u8 cmis_cdb_calc_checksum (const void *data,size_t size) { +const u8 *bytes = (const u8 *)data; +u8 checksum =0 ; + +for (size_t i =0 ; i < size; i++) + checksum += bytes[i]; + +return ~checksum; +}I expect there is already a helper do that somewhere.
Quoted 1 line
Andrew
Yes it does, but actually it is an helper that occurs in specific places (for example pci_vpd_check_csum()), that i can use from here.
Quoted 3 lines
---
pw-bot: cr
RE: [PATCH net-next v7 7/9] ethtool: cmis_cdb: Add a layer for supporting CDB commands
Quoted 105 lines
From: Danielle Ratson <[email protected]>
Sent: Wednesday, 26 June 2024 9:14
To: Andrew Lunn <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; mlxsw <[email protected]>; Ido Schimmel <[email protected]>; Petr Machata <[email protected]>
Subject: RE: [PATCH net-next v7 7/9] ethtool:cmis_cdb: Add a layer for supporting CDB commandsHi Andrew,
Thanks for reviewing the patches.
Quoted 4 lines
From: Andrew Lunn <[email protected]>
Sent: Monday, 24 June 2024 22:51
To: Danielle Ratson <[email protected]>
Cc: [email protected]; [email protected];Quoted 36 lines
[email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; mlxsw <[email protected]>; Ido Schimmel <[email protected]>; Petr Machata <[email protected]>
Subject: Re: [PATCH net-next v7 7/9] ethtool:cmis_cdb: Add a layer for supporting CDB commandsQuoted 14 lines
net/ethtool/cmis_cdb.c new file +14
-0+int ethtool_cmis_wait_for_cond (struct net_device *dev,u8 flags,u8 flag, +u16 max_duration,u32 offset, +bool (*cond_success)(u8 ),bool (*cond_fail)(u8 ), +u8 *state) +{ +const struct ethtool_ops *ops = dev->ethtool_ops; +struct ethtool_module_eeprom page_data = {0 }; +struct cmis_wait_for_cond_rpl rpl = {}; +struct netlink_ext_ack extack = {}; +unsigned long end; +int err; + +if (!(flags & flag)) +return 0 ;net/ethtool/cmis_cdb.c
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +10
-0+ +if (max_duration ==0 ) + max_duration =U16_MAX ; + + end = jiffies +msecs_to_jiffies (max_duration); + do { + ethtool_cmis_page_init(&page_data, 0, offset, sizeof(rpl)); + page_data.data = (u8 *)&rpl; + + err = ops->get_module_eeprom_by_page(dev, &page_data,&extack);
Quoted 8 lines
net/ethtool/cmis_cdb.c line 361
Quoted 4 lines
net/ethtool/cmis_cdb.c new file +4
-0+if (err <0 ) { +if (extack._msg) +netdev_err (dev,"%s\n" , extack._msg); +continue ;continue here is interested. Say you get
-EIObecause the module has been ejected. I would say that is fatal. Won't this spam the logs, as fast as the I2C bus can fail, without the 20ms sleep, for 65535 jiffies?If the module is ejected from some reason, it might span the logs I guess. But it is less likely than the scenario I wanted to cover. According to SPEC 5.2:
"
7.2.5.1 Foreground Mode CDB Messaging
[...]
In foreground mode the module rejects any register ACCESS until a currently executing CDB command execution has completed.
Note: READs of the CdbStatus registers 00h:37 or 00h:38 (see Table 8-13) will also be rejected by the module.
"So in that case the module won't be able to respond and we need to wait for it to be responsive and the status to be valid.
Quoted 12 lines
net/ethtool/cmis_cdb.c line 371
Quoted 11 lines
net/ethtool/cmis_cdb.c new file +10
-0+ } + +if ((*cond_success)(rpl.state)) +return 0 ; + +if (*cond_fail && (*cond_fail)(rpl.state)) +break ; + +msleep (20 ); + }while (time_before (jiffies, end));Quoted 7 lines
Please could you implement this using
iopoll.h. This appears to have the usual problem. Say msleep(20) actually sleeps a lot longer, because the system is busy doing other things.time_before(jiffies, end)) is false, because of the long delay, but in fact the operation has completed without error. Yet you return EBUSY.iopoll.hgets this correct, it does one more evaluation of the condition after exiting the loop to handle this issue.
Quoted 1 line
OK.
Hi Andrew,
I implemented the above as you asked, but it seems to have a problem.
The iopoll functions have a sleeping parameter "sleep_us" that supposed to be equivalent to the msleep(20) if I put 20000 there. However, this parameter is defined as 'Maximum time to sleep between reads in us', so it will not always sleep 20msec as it should. This is problematic since there are modules that needs this 20msec sleep in order to be able to poll again from the module. Otherwise, these modules fail during the write FW command iterations, while polling the flag or status. Therefore, unfortunately in this case I'd rather stay with the origin code.
Thank you for all your comments,
Danielle
Quoted 22 lines
Quoted 14 lines
net/ethtool/cmis_cdb.c line 529
Quoted 11 lines
net/ethtool/cmis_cdb.c new file +9
-0+static u8 cmis_cdb_calc_checksum (const void *data,size_t size) { +const u8 *bytes = (const u8 *)data; +u8 checksum =0 ; + +for (size_t i =0 ; i < size; i++) + checksum += bytes[i]; + +return ~checksum; +}I expect there is already a helper do that somewhere.
Quoted 1 line
Andrew
Yes it does, but actually it is an helper that occurs in specific places (for example
pci_vpd_check_csum()), that i can use from here.Quoted 3 lines
---
pw-bot: cr
Andrew Lunn
maintainerRe: [PATCH net-next v7 7/9] ethtool: cmis_cdb: Add a layer for supporting CDB commands
Quoted 34 lines
Quoted 34 lines
Quoted 33 lines
Quoted 14 lines
net/ethtool/cmis_cdb.c new file +14
-0+int ethtool_cmis_wait_for_cond (struct net_device *dev,u8 flags,u8 flag, +u16 max_duration,u32 offset, +bool (*cond_success)(u8 ),bool (*cond_fail)(u8 ), +u8 *state) +{ +const struct ethtool_ops *ops = dev->ethtool_ops; +struct ethtool_module_eeprom page_data = {0 }; +struct cmis_wait_for_cond_rpl rpl = {}; +struct netlink_ext_ack extack = {}; +unsigned long end; +int err; + +if (!(flags & flag)) +return 0 ;net/ethtool/cmis_cdb.c
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +10
-0+ +if (max_duration ==0 ) + max_duration =U16_MAX ; + + end = jiffies +msecs_to_jiffies (max_duration); + do { + ethtool_cmis_page_init(&page_data, 0, offset, sizeof(rpl)); + page_data.data = (u8 *)&rpl; + + err = ops->get_module_eeprom_by_page(dev, &page_data,&extack);
net/ethtool/cmis_cdb.c line 361
Quoted 4 lines
net/ethtool/cmis_cdb.c new file +4
-0+if (err <0 ) { +if (extack._msg) +netdev_err (dev,"%s\n" , extack._msg); +continue ;continue here is interested. Say you get
-EIObecause the module has been ejected. I would say that is fatal. Won't this spam the logs, as fast as the I2C bus can fail, without the 20ms sleep, for 65535 jiffies?
Quoted 1 line
Quoted 1 line
If the module is ejected from some reason, it might span the logs I guess.
Please could you test it.
65535 jiffies is i think 655 seconds? That is probably too long to loop when the module has been ejected. Maybe replace it with HZ?
Maybe netdev_err() should become netdev_dbg()? And please add a 20ms delay before the continue.
Quoted 19 lines
Quoted 19 lines
Quoted 18 lines
net/ethtool/cmis_cdb.c line 371
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +10
-0+ } + +if ((*cond_success)(rpl.state)) +return 0 ; + +if (*cond_fail && (*cond_fail)(rpl.state)) +break ; + +msleep (20 ); + }while (time_before (jiffies, end));Please could you implement this using
iopoll.h. This appears to have the usual problem. Say msleep(20) actually sleeps a lot longer, because the system is busy doing other things.time_before(jiffies, end)) is false, because of the long delay, but in fact the operation has completed without error. Yet you return EBUSY.iopoll.hgets this correct, it does one more evaluation of the condition after exiting the loop to handle this issue.
Quoted 5 lines
Quoted 1 line
OK.
Hi Andrew,
Therefore, unfortunately in this case I'd rather stay with the origin code.
O.K. Please evaluate the condition again after the while() just so ETIMEDOUT is not returned in error.
Andrew
RE: [PATCH net-next v7 7/9] ethtool: cmis_cdb: Add a layer for supporting CDB commands
Quoted 53 lines
From: Andrew Lunn <[email protected]>
Sent: Wednesday, 26 June 2024 16:40
To: Danielle Ratson <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; mlxsw <[email protected]>; Ido Schimmel <[email protected]>; Petr Machata <[email protected]>
Subject: Re: [PATCH net-next v7 7/9] ethtool:cmis_cdb: Add a layer for supporting CDB commandsQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+int ethtool_cmis_wait_for_cond(struct
net_device*dev, u8 flags, u8flag,
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ u16 max_duration, u32 offset, + bool (*cond_success)(u8), bool(*cond_fail)(u8),
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +10
-0+ u8 *state) +{ +const struct ethtool_ops *ops = dev->ethtool_ops; +struct ethtool_module_eeprom page_data = {0 }; +struct cmis_wait_for_cond_rpl rpl = {}; +struct netlink_ext_ack extack = {}; +unsigned long end; +int err; + +if (!(flags & flag))Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
net/ethtool/cmis_cdb.c
Quoted 8 lines
net/ethtool/cmis_cdb.c new file +8
-0+return 0 ; + +if (max_duration ==0 ) + max_duration =U16_MAX ; + + end = jiffies +msecs_to_jiffies (max_duration); + do { + ethtool_cmis_page_init(&page_data, 0, offset,sizeof(rpl));
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
net/ethtool/cmis_cdb.c
Quoted 3 lines
net/ethtool/cmis_cdb.c new file +3
-0+ page_data.data = (u8 *)&rpl; + + err = ops->get_module_eeprom_by_page(dev,
&page_data,Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
&extack);
net/ethtool/cmis_cdb.c
Quoted 3 lines
net/ethtool/cmis_cdb.c new file +3
-0+if (err <0 ) { +if (extack._msg) + netdev_err(dev, "%s\n",extack._msg);
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
net/ethtool/cmis_cdb.c line 361
Quoted 1 line
net/ethtool/cmis_cdb.c new file +1
-0+continue ;continue here is interested. Say you get
-EIObecause the module has been ejected. I would say that is fatal. Won't this spam the logs, as fast as the I2C bus can fail, without the 20ms sleep, for 65535
Quoted 9 lines
jiffies?
Quoted 2 lines
Quoted 2 lines
If the module is ejected from some reason, it might span the logs I guess.
Please could you test it.
65535 jiffies is i think 655 seconds? That is probably too long to loop when the module has been ejected. Maybe replace it with HZ?
Well actually it is 65535 msec which is ~65 sec and a bit over 1 minute.
The test you are asking for is a bit complicated since I don’t have a machine physically nearby, do you find it very much important? I mean, it is not very reasonable thing to do, burning fw on a module and in the exact same time eject it.
Quoted 23 lines
Maybe
netdev_err()should becomenetdev_dbg()? And please add a 20ms delay before the continue.Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
net/ethtool/cmis_cdb.c line 371
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +10
-0+ } + +if ((*cond_success)(rpl.state)) +return 0 ; + +if (*cond_fail && (*cond_fail)(rpl.state)) +break ; + +msleep (20 ); + }while (time_before (jiffies, end));Please could you implement this using
iopoll.h. This appears to have the usual problem. Say msleep(20) actually sleeps a lot longer, because the system is busy doing other things. time_before(jiffies,
end)) is false, because of the long delay, but in fact the operation has completed without error. Yet you return EBUSY.iopoll.hgets this correct, it does one more evaluation of the condition after exiting the loop to handle this issue.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
OK.
Hi Andrew,
Therefore, unfortunately in this case I'd rather stay with the origin code.
O.K. Please evaluate the condition again after the
while()just so ETIMEDOUT is not returned in error.
Not sure I understood.
Do you want to have one more polling in the end of the loop? What could return ETIMEDOUT?
Thanks,
Danielle
Quoted 2 lines
Andrew
Andrew Lunn
maintainerRe: [PATCH net-next v7 7/9] ethtool: cmis_cdb: Add a layer for supporting CDB commands
Quoted 7 lines
Quoted 5 lines
Please could you test it.
65535 jiffies is i think 655 seconds? That is probably too long to loop when the module has been ejected. Maybe replace it with HZ?
Well actually it is 65535 msec which is ~65 sec and a bit over 1 minute.
I _think_ it depends on CONFIG_HZ, which can be 100, 250, 300 and 1000.
Quoted 5 lines
The test you are asking for is a bit complicated since I don’t have a machine physically nearby, do you find it very much important?
I mean, it is not very reasonable thing to do, burning fw on a module and in the exact same time eject it.
Shooting yourself in the foot is not a very reasonable thing to do, but the Unix philosophy is to all root to do it. Do we really want 60 to 600 seconds of the kernel spamming the log when somebody does do this?
Quoted 15 lines
Quoted 14 lines
Maybe
netdev_err()should becomenetdev_dbg()? And please add a 20ms delay before the continue.Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
net/ethtool/cmis_cdb.c new file +10
-0+ } + +if ((*cond_success)(rpl.state)) +return 0 ; + +if (*cond_fail && (*cond_fail)(rpl.state)) +break ; + +msleep (20 ); + }while (time_before (jiffies, end));
Quoted 5 lines
Quoted 2 lines
O.K. Please evaluate the condition again after the
while()just so ETIMEDOUT is not returned in error.Not sure I understood.
Do you want to have one more polling in the end of the loop? What could return ETIMEDOUT?
Consider what happens when msleep(20) actually sleeps a lot longer.
Look at the core code which gets this correct:
#define read_poll_timeout(op, val, cond, sleep_us, timeout_us, \
sleep_before_read, args...) \
({ \
u64 __timeout_us = (timeout_us); \
unsigned long __sleep_us = (sleep_us); \
ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
might_sleep_if((__sleep_us) != 0); \
if (sleep_before_read && __sleep_us) \
usleep_range((__sleep_us >> 2) + 1, __sleep_us); \
for (;;) { \
(val) = op(args); \
if (cond) \
break; \
if (__timeout_us && \
ktime_compare(ktime_get(), __timeout) > 0) { \
(val) = op(args); \
break; \
} \
if (__sleep_us) \
usleep_range((__sleep_us >> 2) + 1, __sleep_us); \
cpu_relax(); \
} \
(cond) ? 0 : -ETIMEDOUT; \
})So after breaking out of the for loop with a timeout, it evaluates the condition one more time, and uses that to decide on 0 or ETIMEDOUT. So it does not matter if usleep_range() range slept for 60 seconds, not 60ms, the exit code will be correct.
Andrew
RE: [PATCH net-next v7 7/9] ethtool: cmis_cdb: Add a layer for supporting CDB commands
Quoted 28 lines
From: Andrew Lunn <[email protected]>
Sent: Wednesday, 26 June 2024 20:43
To: Danielle Ratson <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; mlxsw <[email protected]>; Ido Schimmel <[email protected]>; Petr Machata <[email protected]>
Subject: Re: [PATCH net-next v7 7/9] ethtool:cmis_cdb: Add a layer for supporting CDB commandsQuoted 7 lines
Quoted 5 lines
Please could you test it.
65535 jiffies is i think 655 seconds? That is probably too long to loop when the module has been ejected. Maybe replace it with HZ?
Well actually it is 65535 msec which is ~65 sec and a bit over 1 minute.
I _think_ it depends on
CONFIG_HZ, which can be 100, 250, 300 and 1000.Quoted 2 lines
The test you are asking for is a bit complicated since I don’t have a machine physically nearby, do you find it very much important?
Quoted 6 lines
Quoted 2 lines
I mean, it is not very reasonable thing to do, burning fw on a module and in the exact same time eject it.
Shooting yourself in the foot is not a very reasonable thing to do, but the Unix philosophy is to all root to do it. Do we really want 60 to 600 seconds of the kernel spamming the log when somebody does do this?
Ok i checked it and using netdev_err_once() fulfill that issue. Thanks!
Quoted 51 lines
Quoted 16 lines
Quoted 15 lines
Maybe
netdev_err()should becomenetdev_dbg()? And please add a 20ms delay before the continue.Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
(quoted diff) +10
-0+ } + + if ((*cond_success)(rpl.state)) + return 0; + + if (*cond_fail && (*cond_fail)(rpl.state)) + break; + + msleep(20); + } while (time_before(jiffies, end));Quoted 5 lines
Quoted 2 lines
O.K. Please evaluate the condition again after the
while()just so ETIMEDOUT is not returned in error.Not sure I understood.
Do you want to have one more polling in the end of the loop? What couldreturn ETIMEDOUT?
Consider what happens when msleep(20) actually sleeps a lot longer.
Look at the core code which gets this correct:
#define read_poll_timeout(op, val, cond, sleep_us, timeout_us, \ sleep_before_read, args...) \ ({ \ u64 __timeout_us = (timeout_us); \ unsigned long __sleep_us = (sleep_us); \ ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \ might_sleep_if((__sleep_us) != 0); \ if (sleep_before_read && __sleep_us) \ usleep_range((__sleep_us >> 2) + 1, __sleep_us); \ for (;;) { \ (val) = op(args); \ if (cond) \ break; \ if (__timeout_us && \ ktime_compare(ktime_get(), __timeout) > 0) { \ (val) = op(args); \ break; \ } \ if (__sleep_us) \ usleep_range((__sleep_us >> 2) + 1, __sleep_us); \ cpu_relax(); \ } \ (cond) ? 0 : -ETIMEDOUT; \ })
Quoted 6 lines
So after breaking out of the for loop with a timeout, it evaluates the condition one more time, and uses that to decide on 0 or ETIMEDOUT. So it does not matter if
usleep_range()range slept for 60 seconds, not 60ms, the exit code will be correct.Andrew
Ok ill fix it, thanks.
PATCH v7 8/9 ethtool: cmis_fw_update: add a layer for supporting firmware update using CDB
In reply to Danielle RatsonAccording to the CMIS standard, the firmware update process is done using a CDB commands sequence.
Implement a work that will be triggered from the module layer in the next patch the will initiate and execute all the CDB commands in order, to eventually complete the firmware update process.
This flashing process includes, writing the firmware image, running the new firmware image and committing it after testing, so that it will run upon reset.
This work will also notify user space about the progress of the firmware update process.
---
Notes:
v6:
* Add a list field to struct ethtool_module_fw_flash for module_fw_flash_work_list that will be presented in the next patch.
* Move ethtool_cmis_fw_update() cleaning to a new function that will be represented in the next patch.
* Move some of the fields in struct ethtool_module_fw_flash to a separate struct, so ethtool_cmis_fw_update() will get only the relevant parameters for it.
* Edit the relevant functions to get the relevant params for them.
* s/CMIS_MODULE_READY_MAX_DURATION_USEC/CMIS_MODULE_READY_MAX_DURATION_MSEC.
v2:
* Decrease msleep before querying completion flag in Write FW Image command.
* Change the condition for failing when LPL is not supported.
* Re-write cmis_fw_update_write_image().
net/ethtool/Makefile | 2 +- net/ethtool/cmis.h | 7 + net/ethtool/cmis_fw_update.c | 399 +++++++++++++++++++++++++++++++++++ net/ethtool/module_fw.h | 31 +++ 4 files changed, 438 insertions(+), 1 deletion(-) create mode 100644 net/ethtool/cmis_fw_update.c
net/ethtool/Makefile +1 -1
@@ -8,4 +8,4 @@ ethtool_nl-y := netlink.o bitset.o strset.o linkinfo.o linkmodes.o rss.o \ linkstate.o debug.o wol.o features.o privflags.o rings.o \ channels.o coalesce.o pause.o eee.o tsinfo.o cabletest.o \ tunnels.o fec.o eeprom.o stats.o phc_vclocks.o mm.o \ - module.o cmis_cdb.o pse-pd.o plca.o mm.o + module.o cmis_fw_update.o cmis_cdb.o pse-pd.o plca.o mm.o
net/ethtool/cmis.h +7 -0
@@ -20,6 +20,12 @@ struct ethtool_cmis_cdb {enum ethtool_cmis_cdb_cmd_id {ETHTOOL_CMIS_CDB_CMD_QUERY_STATUS =0x0000 ,ETHTOOL_CMIS_CDB_CMD_MODULE_FEATURES =0x0040 , +ETHTOOL_CMIS_CDB_CMD_FW_MANAGMENT_FEATURES =0x0041 , +ETHTOOL_CMIS_CDB_CMD_START_FW_DOWNLOAD =0x0101 , +ETHTOOL_CMIS_CDB_CMD_WRITE_FW_BLOCK_LPL =0x0103 , +ETHTOOL_CMIS_CDB_CMD_COMPLETE_FW_DOWNLOAD =0x0107 , +ETHTOOL_CMIS_CDB_CMD_RUN_FW_IMAGE =0x0109 , +ETHTOOL_CMIS_CDB_CMD_COMMIT_FW_IMAGE =0x010A , };/**
@@ -47,6 +53,7 @@ struct ethtool_cmis_cdb_request {#define CDB_F_COMPLETION_VALID BIT(0)#define CDB_F_STATUS_VALID BIT(1) +#define CDB_F_MODULE_STATE_VALID BIT(2)/** * struct ethtool_cmis_cdb_cmd_args - CDB commands execution arguments
net/ethtool/cmis_fw_update.c new file +399 -0
399 lines not on this page · raw message
net/ethtool/module_fw.h +31 -0
@@ -25,6 +25,35 @@ struct ethtool_module_fw_flash_params { u8 password_valid:1; }; +/** +* struct ethtool_cmis_fw_update_params - CMIS firmware update specific +* parameters +* @dev: Pointer to the net_device to be flashed. +* @params: Module firmware flashing parameters. +* @ntf_params: Module firmware flashing notification parameters. +* @fw: Firmware to flash. +*/ +struct ethtool_cmis_fw_update_params { +struct net_device *dev; +struct ethtool_module_fw_flash_params params; +struct ethnl_module_fw_flash_ntf_params ntf_params; +const struct firmware *fw; +}; + +/** +* struct ethtool_module_fw_flash - module firmware flashing +* @list: List node for &module_fw_flash_work_list. +* @dev_tracker: Refcount tracker for @dev. +* @work: The flashing firmware work. +* @fw_update: CMIS firmware update specific parameters. +*/ +struct ethtool_module_fw_flash { +struct list_head list; +netdevice_tracker dev_tracker; +struct work_struct work; +struct ethtool_cmis_fw_update_params fw_update; +}; + void ethnl_module_fw_flash_ntf_err(struct net_device *dev, struct ethnl_module_fw_flash_ntf_params *params,
@@ -39,3 +68,5 @@ void ethnl_module_fw_flash_ntf_in_progress(struct net_device *dev, struct ethnl_module_fw_flash_ntf_params *params, u64 done, u64 total); + +void ethtool_cmis_fw_update (struct ethtool_cmis_fw_update_params *params);
Signature · 1 line
2.45.0
Andrew Lunn
maintainerRe: [PATCH net-next v7 8/9] ethtool: cmis_fw_update: add a layer for supporting firmware update using CDB
net/ethtool/cmis_fw_update.c line 342
Quoted 5 lines
net/ethtool/cmis_fw_update.c new file +5
-0+static int cmis_fw_update_reset (struct net_device *dev) +{ +__u32 reset_data =ETH_RESET_PHY ; + +return dev->ethtool_ops->reset (dev, &reset_data);
Is there a test somewhere that this op is actually implemented?
Maybe the next patch.
Andrew
RE: [PATCH net-next v7 8/9] ethtool: cmis_fw_update: add a layer for supporting firmware update using CDB
Quoted 18 lines
-----Original Message-----
From: Andrew Lunn <[email protected]>
Sent: Monday, 24 June 2024 22:57
To: Danielle Ratson <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; mlxsw <[email protected]>; Ido Schimmel <[email protected]>; Petr Machata <[email protected]>
Subject: Re: [PATCH net-next v7 8/9] ethtool:cmis_fw_update: add a layer for supporting firmware update using CDB
Quoted 8 lines
net/ethtool/cmis_fw_update.c line 342
Quoted 4 lines
net/ethtool/cmis_fw_update.c new file +4
-0+static int cmis_fw_update_reset (struct net_device *dev) { +__u32 reset_data =ETH_RESET_PHY ; + +return dev->ethtool_ops->reset (dev, &reset_data);Is there a test somewhere that this op is actually implemented?
Maybe the next patch.
Yes, indeed.
Quoted 2 lines
Andrew
PATCH v7 9/9 ethtool: Add ability to flash transceiver modules' firmware
In reply to Danielle RatsonAdd the ability to flash the modules' firmware by implementing the interface between the user space and the kernel.
Example from a succeeding implementation:
# ethtool --flash-module-firmware swp40 file test.bin
Transceiver module firmware flashing started for device swp40 Transceiver module firmware flashing in progress for device swp40
Progress: 99%
Transceiver module firmware flashing completed for device swp40
In addition, add infrastructure that allows modules to set socket-specific private data. This ensures that when a socket is closed from user space during the flashing process, the right socket halts sending notifications to user space until the work item is completed.
---
Notes:
v7:
* Fix Warning for not unlocking the spin_lock in the error flow on module_flash_fw_work_list_add().
* Avoid the fall-through on ethnl_sock_priv_destroy().
v6:
* Add a paragraph in the commit message.
* Rename labels in module_flash_fw_schedule().
* Add info to genl_sk_priv_*() and implement the relevant callbacks, in order to handle properly a scenario of closing the socket from user space before the work item was ended.
* Add a list the holds all the ethtool_module_fw_flash struct that corresponds to the in progress work items.
* Add a new enum for the socket types.
* Use both above to identify a flashing socket, add it to the list and when closing socket affect only the flashing type.
* Create a new function that will get the work item instead of ethtool_cmis_fw_update().
* Edit the relevant functions to get the relevant params for them.
* The new function will call the old ethtool_cmis_fw_update(), and do the cleaning, so the existence of the list should be completely isolated in module.c.
net/ethtool/module.c | 277 ++++++++++++++++++++++++++++++++++++++++ net/ethtool/module_fw.h | 3 + net/ethtool/netlink.c | 39 ++++++ net/ethtool/netlink.h | 15 +++ 4 files changed, 334 insertions(+)
net/ethtool/module.c +277 -0
@@ -1,6 +1,9 @@// SPDX-License-Identifier: GPL-2.0-only #include <linux/ethtool.h> +#include <linux/firmware.h> +#include <linux/sfp.h> +#include <net/devlink.h> #include "netlink.h" #include "common.h"
@@ -34,6 +37,12 @@ static int module_get_power_mode(struct net_device *dev,if (!ops->get_module_power_mode)return 0 ; +if (dev->module_fw_flash_in_progress) { +NL_SET_ERR_MSG (extack, +"Module firmware flashing is in progress" ); +return -EBUSY ; + } +return ops->get_module_power_mode (dev, &data->power, extack); }
@@ -110,6 +119,12 @@ ethnl_set_module_validate(struct ethnl_req_info *req_info,if (!tb[ETHTOOL_A_MODULE_POWER_MODE_POLICY ])return 0 ; +if (req_info->dev->module_fw_flash_in_progress) { +NL_SET_ERR_MSG (info->extack, +"Module firmware flashing is in progress" ); +return -EBUSY ; + } +if (!ops->get_module_power_mode || !ops->set_module_power_mode) { NL_SET_ERR_MSG_ATTR(info->extack, tb[ETHTOOL_A_MODULE_POWER_MODE_POLICY],
@@ -160,6 +175,268 @@ const struct ethnl_request_ops ethnl_module_request_ops = { .set_ntf_cmd = ETHTOOL_MSG_MODULE_NTF, }; +/* MODULE_FW_FLASH_ACT */ + +const struct nla_policy +ethnl_module_fw_flash_act_policy[ETHTOOL_A_MODULE_FW_FLASH_PASSWORD +1 ] = { + [ETHTOOL_A_MODULE_FW_FLASH_HEADER ] = +NLA_POLICY_NESTED (ethnl_header_policy), + [ETHTOOL_A_MODULE_FW_FLASH_FILE_NAME ] = { .type =NLA_NUL_STRING }, + [ETHTOOL_A_MODULE_FW_FLASH_PASSWORD ] = { .type =NLA_U32 }, +}; + +static LIST_HEAD (module_fw_flash_work_list); +static DEFINE_SPINLOCK (module_fw_flash_work_list_lock); + +static int +module_flash_fw_work_list_add (struct ethtool_module_fw_flash *module_fw, +struct genl_info *info) +{ +struct ethtool_module_fw_flash *work; + +/* First, check if already registered. */ +spin_lock (&module_fw_flash_work_list_lock); +list_for_each_entry (work, &module_fw_flash_work_list, list) { +if (work->fw_update.ntf_params.portid == info->snd_portid && + work->fw_update.dev == module_fw->fw_update.dev) { +spin_unlock (&module_fw_flash_work_list_lock); +return -EALREADY ; + } + } + +list_add_tail (&module_fw->list, &module_fw_flash_work_list); +spin_unlock (&module_fw_flash_work_list_lock); + +return 0 ; +} + +static void module_flash_fw_work_list_del (struct list_head *list) +{ +spin_lock (&module_fw_flash_work_list_lock); +list_del (list); +spin_unlock (&module_fw_flash_work_list_lock); +} + +static void module_flash_fw_work (struct work_struct *work) +{ +struct ethtool_module_fw_flash *module_fw; + + module_fw = container_of(work, struct ethtool_module_fw_flash, work); + +ethtool_cmis_fw_update (&module_fw->fw_update); + +module_flash_fw_work_list_del (&module_fw->list); + module_fw->fw_update.dev->module_fw_flash_in_progress = false; +netdev_put (module_fw->fw_update.dev, &module_fw->dev_tracker); +release_firmware (module_fw->fw_update.fw); +kfree (module_fw); +} + +#define MODULE_EEPROM_PHYS_ID_PAGE 0 +#define MODULE_EEPROM_PHYS_ID_I2C_ADDR 0x50 + +static int module_flash_fw_work_init (struct ethtool_module_fw_flash *module_fw, +struct net_device *dev, +struct netlink_ext_ack *extack) +{ +const struct ethtool_ops *ops = dev->ethtool_ops; +struct ethtool_module_eeprom page_data = {}; +u8 phys_id; +int err; + +/* Fetch the SFF-8024 Identifier Value. For all supported standards, it +* is located at I2C address 0x50, byte 0. See section 4.1 in SFF-8024, +* revision 4.9. +*/ + page_data.page =MODULE_EEPROM_PHYS_ID_PAGE ; + page_data.offset =SFP_PHYS_ID ; + page_data.length =sizeof (phys_id); + page_data.i2c_address =MODULE_EEPROM_PHYS_ID_I2C_ADDR ; + page_data.data = &phys_id; + + err = ops->get_module_eeprom_by_page (dev, &page_data, extack); +if (err <0 ) +return err; + +switch (phys_id) { +case SFF8024_ID_QSFP_DD : +case SFF8024_ID_OSFP : +case SFF8024_ID_DSFP : +case SFF8024_ID_QSFP_PLUS_CMIS : +case SFF8024_ID_SFP_DD_CMIS : +case SFF8024_ID_SFP_PLUS_CMIS : +INIT_WORK (&module_fw->work, module_flash_fw_work); +break ; +default : +NL_SET_ERR_MSG (extack, +"Module type does not support firmware flashing" ); +return -EOPNOTSUPP ; + } + +return 0 ; +} + +void ethnl_module_fw_flash_sock_destroy (struct ethnl_sock_priv *sk_priv) +{ +struct ethtool_module_fw_flash *work; + +spin_lock (&module_fw_flash_work_list_lock); +list_for_each_entry (work, &module_fw_flash_work_list, list) { +if (work->fw_update.dev == sk_priv->dev && + work->fw_update.ntf_params.portid == sk_priv->portid) { + work->fw_update.ntf_params.closed_sock = true; +break ; + } + } +spin_unlock (&module_fw_flash_work_list_lock); +} + +static int +module_flash_fw_schedule (struct net_device *dev,const char *file_name, +struct ethtool_module_fw_flash_params *params, +struct sk_buff *skb,struct genl_info *info) +{ +struct ethtool_cmis_fw_update_params *fw_update; +struct ethtool_module_fw_flash *module_fw; +int err; + + module_fw =kzalloc (sizeof (*module_fw),GFP_KERNEL ); +if (!module_fw) +return -ENOMEM ; + + fw_update = &module_fw->fw_update; + fw_update->params = *params; + err =request_firmware_direct (&fw_update->fw, + file_name, &dev->dev); +if (err) { +NL_SET_ERR_MSG (info->extack, +"Failed to request module firmware image" ); + goto err_free; + } + + err =module_flash_fw_work_init (module_fw, dev, info->extack); +if (err <0 ) + goto err_release_firmware; + + dev->module_fw_flash_in_progress = true; +netdev_hold (dev, &module_fw->dev_tracker,GFP_KERNEL ); + fw_update->dev = dev; + fw_update->ntf_params.portid = info->snd_portid; + fw_update->ntf_params.seq = info->snd_seq; + fw_update->ntf_params.closed_sock = false; + + err =ethnl_sock_priv_set (skb, dev, fw_update->ntf_params.portid, +ETHTOOL_SOCK_TYPE_MODULE_FW_FLASH ); +if (err <0 ) + goto err_release_firmware; + + err =module_flash_fw_work_list_add (module_fw, info); +if (err <0 ) + goto err_release_firmware; + +schedule_work (&module_fw->work); + +return 0 ; + +err_release_firmware: +release_firmware (fw_update->fw); +err_free: +kfree (module_fw); +return err; +} + +static int module_flash_fw (struct net_device *dev,struct nlattr **tb, +struct sk_buff *skb,struct genl_info *info) +{ +struct ethtool_module_fw_flash_params params = {}; +const char *file_name; +struct nlattr *attr; + +if (GENL_REQ_ATTR_CHECK (info,ETHTOOL_A_MODULE_FW_FLASH_FILE_NAME )) +return -EINVAL ; + + file_name =nla_data (tb[ETHTOOL_A_MODULE_FW_FLASH_FILE_NAME ]); + + attr = tb[ETHTOOL_A_MODULE_FW_FLASH_PASSWORD ]; +if (attr) { + params.password =cpu_to_be32 (nla_get_u32 (attr)); + params.password_valid = true; + } + +return module_flash_fw_schedule (dev, file_name, ¶ms, skb, info); +} + +static int ethnl_module_fw_flash_validate (struct net_device *dev, +struct netlink_ext_ack *extack) +{ +struct devlink_port *devlink_port = dev->devlink_port; +const struct ethtool_ops *ops = dev->ethtool_ops; + +if (!ops->set_module_eeprom_by_page || + !ops->get_module_eeprom_by_page) { +NL_SET_ERR_MSG (extack, +"Flashing module firmware is not supported by this device" ); +return -EOPNOTSUPP ; + } + +if (!ops->reset) { +NL_SET_ERR_MSG (extack, +"Reset module is not supported by this device, so flashing is not permitted" ); +return -EOPNOTSUPP ; + } + +if (dev->module_fw_flash_in_progress) { +NL_SET_ERR_MSG (extack,"Module firmware flashing already in progress" ); +return -EBUSY ; + } + +if (dev->flags &IFF_UP ) { +NL_SET_ERR_MSG (extack,"Netdevice is up, so flashing is not permitted" ); +return -EBUSY ; + } + +if (devlink_port && devlink_port->attrs.split) { +NL_SET_ERR_MSG (extack,"Can't perform firmware flashing on a split port" ); +return -EOPNOTSUPP ; + } + +return 0 ; +} + +int ethnl_act_module_fw_flash (struct sk_buff *skb,struct genl_info *info) +{ +struct ethnl_req_info req_info = {}; +struct nlattr **tb = info->attrs; +struct net_device *dev; +int ret; + + ret =ethnl_parse_header_dev_get (&req_info, + tb[ETHTOOL_A_MODULE_FW_FLASH_HEADER ], +genl_info_net (info), info->extack, + true); +if (ret <0 ) +return ret; + dev = req_info.dev; + +rtnl_lock (); + ret =ethnl_ops_begin (dev); +if (ret <0 ) + goto out_rtnl; + + ret =ethnl_module_fw_flash_validate (dev, info->extack); +if (ret <0 ) + goto out_rtnl; + + ret =module_flash_fw (dev, tb, skb, info); + +ethnl_ops_complete (dev); + +out_rtnl: +rtnl_unlock (); +ethnl_parse_header_dev_put (&req_info); +return ret; +} +/* MODULE_FW_FLASH_NTF */ static int
net/ethtool/module_fw.h +3 -0
@@ -1,6 +1,7 @@/* SPDX-License-Identifier: GPL-2.0-only */ #include <uapi/linux/ethtool.h> +#include "netlink.h" /** * struct ethnl_module_fw_flash_ntf_params - module firmware flashing
@@ -54,6 +55,8 @@ struct ethtool_module_fw_flash {struct ethtool_cmis_fw_update_params fw_update; }; +void ethnl_module_fw_flash_sock_destroy (struct ethnl_sock_priv *sk_priv); + void ethnl_module_fw_flash_ntf_err(struct net_device *dev, struct ethnl_module_fw_flash_ntf_params *params,
net/ethtool/netlink.c +39 -0
@@ -4,6 +4,7 @@#include <linux/ethtool_netlink.h> #include <linux/pm_runtime.h> #include "netlink.h" +#include "module_fw.h" static struct genl_family ethtool_genl_family;
@@ -30,6 +31,35 @@ const struct nla_policy ethnl_header_policy_stats[] = { ETHTOOL_FLAGS_STATS), }; +int ethnl_sock_priv_set (struct sk_buff *skb,struct net_device *dev,u32 portid, +enum ethnl_sock_type type) +{ +struct ethnl_sock_priv *sk_priv; + + sk_priv =genl_sk_priv_get (ðtool_genl_family,NETLINK_CB (skb).sk); +if (IS_ERR (sk_priv)) +return PTR_ERR (sk_priv); + + sk_priv->dev = dev; + sk_priv->portid = portid; + sk_priv->type = type; + +return 0 ; +} + +static void ethnl_sock_priv_destroy (void *priv) +{ +struct ethnl_sock_priv *sk_priv = priv; + +switch (sk_priv->type) { +case ETHTOOL_SOCK_TYPE_MODULE_FW_FLASH : +ethnl_module_fw_flash_sock_destroy (sk_priv); +break ; +default : +break ; + } +} +int ethnl_ops_begin (struct net_device *dev) {int ret;
@@ -1142,6 +1172,13 @@ static const struct genl_ops ethtool_genl_ops[] = { .policy = ethnl_mm_set_policy, .maxattr = ARRAY_SIZE(ethnl_mm_set_policy) - 1, }, + { + .cmd = ETHTOOL_MSG_MODULE_FW_FLASH_ACT, + .flags = GENL_UNS_ADMIN_PERM, + .doit = ethnl_act_module_fw_flash, + .policy = ethnl_module_fw_flash_act_policy, + .maxattr = ARRAY_SIZE(ethnl_module_fw_flash_act_policy) - 1, + }, };static const struct genl_multicast_group ethtool_nl_mcgrps[] = {
@@ -1158,6 +1195,8 @@ static struct genl_family ethtool_genl_family __ro_after_init = { .resv_start_op = ETHTOOL_MSG_MODULE_GET + 1, .mcgrps = ethtool_nl_mcgrps, .n_mcgrps = ARRAY_SIZE(ethtool_nl_mcgrps), + .sock_priv_size = sizeof(struct ethnl_sock_priv), + .sock_priv_destroy = ethnl_sock_priv_destroy, };/* module setup */
net/ethtool/netlink.h +15 -0
@@ -284,6 +284,19 @@ struct ethnl_reply_data {int ethnl_ops_begin (struct net_device *dev);void ethnl_ops_complete (struct net_device *dev); +enum ethnl_sock_type { +ETHTOOL_SOCK_TYPE_MODULE_FW_FLASH , +}; + +struct ethnl_sock_priv { +struct net_device *dev; +u32 portid; +enum ethnl_sock_type type; +}; + +int ethnl_sock_priv_set (struct sk_buff *skb,struct net_device *dev,u32 portid, +enum ethnl_sock_type type); +/** * struct ethnl_request_ops - unified handling of GET and SET requests * @request_cmd: command id for request (GET)
@@ -442,6 +455,7 @@ extern const struct nla_policy ethnl_plca_set_cfg_policy[ETHTOOL_A_PLCA_MAX + 1]extern const struct nla_policy ethnl_plca_get_status_policy[ETHTOOL_A_PLCA_HEADER +1 ];extern const struct nla_policy ethnl_mm_get_policy[ETHTOOL_A_MM_HEADER +1 ];extern const struct nla_policy ethnl_mm_set_policy[ETHTOOL_A_MM_MAX +1 ]; +extern const struct nla_policy ethnl_module_fw_flash_act_policy[ETHTOOL_A_MODULE_FW_FLASH_PASSWORD +1 ];int ethnl_set_features (struct sk_buff *skb,struct genl_info *info);int ethnl_act_cable_test (struct sk_buff *skb,struct genl_info *info);
@@ -449,6 +463,7 @@ int ethnl_act_cable_test_tdr(struct sk_buff *skb, struct genl_info *info);int ethnl_tunnel_info_doit (struct sk_buff *skb,struct genl_info *info);int ethnl_tunnel_info_start (struct netlink_callback *cb);int ethnl_tunnel_info_dumpit (struct sk_buff *skb,struct netlink_callback *cb); +int ethnl_act_module_fw_flash (struct sk_buff *skb,struct genl_info *info);extern const char stats_std_names[__ETHTOOL_STATS_CNT][ETH_GSTRING_LEN ];extern const char stats_eth_phy_names[__ETHTOOL_A_STATS_ETH_PHY_CNT][ETH_GSTRING_LEN ];
Signature · 1 line
2.45.0