Extend module EEPROM API
Subsystems (3)
v2 cover Extend module EEPROM API
Ethtool supports module EEPROM dumps via the ethtool -m <dev> command. But in current state its functionality is limited - offset and length parameters, which are used to specify a linear desired region of EEPROM data to dump, is not enough, considering emergence of complex module EEPROM layouts such as CMIS 4.0.
Moreover, CMIS 4.0 extends the amount of pages that may be accessible by introducing another parameter for page addressing - banks. Besides, currently module EEPROM is represented as a chunk of concatenated pages, where lower 128 bytes of all pages, except page 00h, are omitted. Offset and length are used to address parts of this fake linear memory. But in practice drivers, which implement get_module_info() and get_module_eeprom() ethtool ops still calculate page number and set I2C address on their own.
This series adds support in ethtool -m of dumping an arbitrary page specified by page number, bank number and I2C address. Implement netlink handler for ethtool -m in order to make such requests to the kernel
and extend CLI by adding corresponding parameters.
New command line format:
ethtool -m <dev> [hex on|off] [raw on|off] [offset N] [length N] [page N] [bank N] [i2c N]
Netlink infrastructure works on per-page basis and allows dumps of a single page at once. But in case user requests human-readable output, which currently may require more than one page, userspace can make such additional calls to kernel on demand and place pages in a linked list. It allows to get pages from cache on demand and pass them to refactored SFF decoders.
Change Log:
v1 -> v2:
- Changed offset defines to specification values.
- Added default offset value (128) if page number is specified.
- Fixed return values.
- Removed page_available()
Vladyslav Tarasiuk (4):
ethtool: Add netlink handler for getmodule (-m)
ethtool: Refactor human-readable module EEPROM output for new API
ethtool: Rename QSFP-DD identifiers to use CMIS 4.0
ethtool: Update manpages to reflect changes to getmodule (-m) command
Makefile.am | 3 +- qsfp-dd.c => cmis4.c | 220 +++++++++++---------- cmis4.h | 128 +++++++++++++ ethtool.8.in | 14 ++ ethtool.c | 4 + internal.h | 12 ++ list.h | 34 ++++ netlink/desc-ethtool.c | 13 ++ netlink/extapi.h | 2 + netlink/module-eeprom.c | 416 ++++++++++++++++++++++++++++++++++++++++ qsfp-dd.h | 29 +-- qsfp.c | 130 +++++++------ qsfp.h | 51 ++--- sff-common.c | 3 + sff-common.h | 3 +- 15 files changed, 870 insertions(+), 192 deletions(-) rename qsfp-dd.c => cmis4.c (55%) create mode 100644 cmis4.h create mode 100644 list.h create mode 100644 netlink/module-eeprom.c
Signature · 1 line
2.18.2
PATCH v2 1/4 ethtool: Add netlink handler for getmodule (-m)
From: Vladyslav Tarasiuk <[email protected]>
Implement "ethtool -m <dev>" subcommand using netlink and extend the interface for new module EEPROM standards. Currently, ethtool supports module EEPROM dumps of continuous memory regions, which are specified using a pair of parameters - offset and length. But due to emergence of new standards such as CMIS 4.0, which further extends possible addressed memory, this approach shows its limitations.
Extend command line interface in order to support dumps of arbitrary pages including CMIS 4.0-specific banked pages:
ethtool -m <dev> [page N] [bank N] [i2c N]
Command example:
# ethtool -m eth2 page 1 offset 0x80 length 0x20
Offset Values ------ ------ 0x0080: 11 00 23 80 00 00 00 00 00 00 00 08 ff 00 00 00 0x0090: 00 00 01 a0 4d 65 6c 6c 61 6e 6f 78 20 20 20 20
--- Makefile.am | 1 + ethtool.c | 4 + internal.h | 10 + list.h | 34 ++++ netlink/desc-ethtool.c | 13 ++ netlink/extapi.h | 2 + netlink/module-eeprom.c | 404 ++++++++++++++++++++++++++++++++++++++++ 7 files changed, 468 insertions(+) create mode 100644 list.h create mode 100644 netlink/module-eeprom.c
Makefile.am +1 -0
@@ -38,6 +38,7 @@ ethtool_SOURCES += \ netlink/eee.c netlink/tsinfo.c netlink/fec.c \ netlink/stats.c \ netlink/desc-ethtool.c netlink/desc-genlctrl.c \ + netlink/module-eeprom.c \ netlink/desc-rtnl.c netlink/cable_test.c netlink/tunnels.c \ uapi/linux/ethtool_netlink.h \ uapi/linux/netlink.h uapi/linux/genetlink.h \
ethtool.c +4 -0
@@ -5897,11 +5897,15 @@ static const struct option args[] = { { .opts = "-m|--dump-module-eeprom|--module-info", .func = do_getmodule, + .nlfunc = nl_getmodule, .help = "Query/Decode Module EEPROM information and optical diagnostics if available", .xhelp =" [ raw on|off ]\n" " [ hex on|off ]\n" " [ offset N ]\n" " [ length N ]\n" +" [ page N ]\n" +" [ bank N ]\n" +" [ i2c N ]\n" }, { .opts = "--show-eee",
internal.h +10 -0
@@ -216,6 +216,16 @@ static inline int ethtool_link_mode_set_bit(unsigned int nr, u32 *mask)return 0 ; } +/* Struct for managing module EEPROM pages */ +struct ethtool_module_eeprom { +u32 offset; +u32 length; +u8 page; +u8 bank; +u8 i2c_address; +u8 *data; +}; +/* Context for sub-commands */ struct cmd_context {const char *devname;/* net device name */
list.h new file +34 -0
@@ -0,0 +1,34 @@ +#ifndef ETHTOOL_LIST_H__ +#define ETHTOOL_LIST_H__ + +#include <unistd.h> + +/* Generic list utilities */ + +struct list_head { +struct list_head *next, *prev; +}; + +#define LIST_HEAD_INIT (name) { &(name), &(name) } + +static inline void list_add (struct list_head *new,struct list_head *head) +{ + head->next->prev = new; + new->next = head->next; + new->prev = head; + head->next = new; +} + +static inline void list_del (struct list_head *entry) +{ + entry->next->prev = entry->prev; + entry->prev->next = entry->next; + entry->next =NULL ; + entry->prev =NULL ; +} + +#define list_for_each_safe (pos, n, head) \ + for (pos = (head)->next, n = pos->next; pos != (head); \ + pos = n, n = pos->next) + +#endif
netlink/desc-ethtool.c +13 -0
@@ -363,6 +363,17 @@ static const struct pretty_nla_desc __stats_desc[] = { NLATTR_DESC_NESTED(ETHTOOL_A_STATS_GRP, stats_grp), }; +const struct pretty_nla_desc __module_eeprom_desc[] = { +NLATTR_DESC_INVALID (ETHTOOL_A_MODULE_EEPROM_UNSPEC ), +NLATTR_DESC_NESTED (ETHTOOL_A_MODULE_EEPROM_HEADER , header), +NLATTR_DESC_U32 (ETHTOOL_A_MODULE_EEPROM_OFFSET ), +NLATTR_DESC_U32 (ETHTOOL_A_MODULE_EEPROM_LENGTH ), +NLATTR_DESC_U8 (ETHTOOL_A_MODULE_EEPROM_PAGE ), +NLATTR_DESC_U8 (ETHTOOL_A_MODULE_EEPROM_BANK ), +NLATTR_DESC_U8 (ETHTOOL_A_MODULE_EEPROM_I2C_ADDRESS ), +NLATTR_DESC_BINARY (ETHTOOL_A_MODULE_EEPROM_DATA ) +}; +const struct pretty_nlmsg_desc ethnl_umsg_desc[] = {NLMSG_DESC_INVALID (ETHTOOL_MSG_USER_NONE ),NLMSG_DESC (ETHTOOL_MSG_STRSET_GET , strset),
@@ -396,6 +407,7 @@ const struct pretty_nlmsg_desc ethnl_umsg_desc[] = {NLMSG_DESC (ETHTOOL_MSG_FEC_GET , fec),NLMSG_DESC (ETHTOOL_MSG_FEC_SET , fec),NLMSG_DESC (ETHTOOL_MSG_STATS_GET , stats), + NLMSG_DESC(ETHTOOL_MSG_MODULE_EEPROM_GET, module_eeprom), };const unsigned int ethnl_umsg_n_desc =ARRAY_SIZE (ethnl_umsg_desc);
@@ -434,6 +446,7 @@ const struct pretty_nlmsg_desc ethnl_kmsg_desc[] = {NLMSG_DESC (ETHTOOL_MSG_FEC_GET_REPLY , fec),NLMSG_DESC (ETHTOOL_MSG_FEC_NTF , fec),NLMSG_DESC (ETHTOOL_MSG_STATS_GET_REPLY , stats), + NLMSG_DESC(ETHTOOL_MSG_MODULE_EEPROM_GET_REPLY, module_eeprom), };const unsigned int ethnl_kmsg_n_desc =ARRAY_SIZE (ethnl_kmsg_desc);
netlink/extapi.h +2 -0
@@ -44,6 +44,7 @@ int nl_sfec(struct cmd_context *ctx);bool nl_gstats_chk (struct cmd_context *ctx);int nl_gstats (struct cmd_context *ctx);int nl_monitor (struct cmd_context *ctx); +int nl_getmodule (struct cmd_context *ctx);void nl_monitor_usage (void );
@@ -97,6 +98,7 @@ static inline void nl_monitor_usage(void)#define nl_sfec NULL#define nl_gstats_chk NULL#define nl_gstats NULL +#define nl_getmodule NULL#endif /* ETHTOOL_ENABLE_NETLINK */
netlink/module-eeprom.c new file +404 -0
404 lines not on this page · raw message
Signature · 1 line
2.18.2
PATCH v2 2/4 ethtool: Refactor human-readable module EEPROM output for new API
In reply to Moshe ShemeshFrom: Vladyslav Tarasiuk <[email protected]>
Reuse existing SFF8636 and QSFP-DD infrastructures to implement EEPROM decoders, which work with paged memory. Add support for human-readable output for QSFP, QSFP28, QSFP Plus, QSFP-DD and DSFP transreceivers from netlink 'ethtool -m' handler.
--- internal.h | 2 + netlink/module-eeprom.c | 12 ++++ qsfp-dd.c | 44 +++++++++++--- qsfp-dd.h | 29 +++++---- qsfp.c | 128 +++++++++++++++++++++++----------------- qsfp.h | 51 ++++++++-------- sff-common.c | 3 + sff-common.h | 3 +- 8 files changed, 170 insertions(+), 102 deletions(-)
internal.h +2 -0
@@ -391,6 +391,8 @@ void sff8472_show_all(const __u8 *id);/* QSFP Optics diagnostics */ void sff8636_show_all (const __u8 *id,__u32 eeprom_len); +void sff8636_show_all_paged (const struct ethtool_module_eeprom *page_zero, +const struct ethtool_module_eeprom *page_three);/* FUJITSU Extended Socket network device */ int fjes_dump_regs (struct ethtool_drvinfo *info,struct ethtool_regs *regs);
netlink/module-eeprom.c +12 -0
@@ -302,6 +302,7 @@ static int decoder_prefetch(struct nl_context *nlctx) request.page =3 ;break ;case SFF8024_ID_QSFP_DD : +case SFF8024_ID_DSFP :memset (&request,0 ,sizeof (request)); request.i2c_address =ETH_I2C_ADDRESS_LOW ; request.offset =128 ;
@@ -315,13 +316,24 @@ static int decoder_prefetch(struct nl_context *nlctx)static void decoder_print (void ) { +struct ethtool_module_eeprom *page_three =cache_get (3 ,0 ,ETH_I2C_ADDRESS_LOW );struct ethtool_module_eeprom *page_zero =cache_get (0 ,0 ,ETH_I2C_ADDRESS_LOW ); +struct ethtool_module_eeprom *page_one =cache_get (1 ,0 ,ETH_I2C_ADDRESS_LOW );u8 module_id = page_zero->data[SFF8636_ID_OFFSET ];switch (module_id) {case SFF8024_ID_SFP :sff8079_show_all (page_zero->data);break ; +case SFF8024_ID_QSFP : +case SFF8024_ID_QSFP28 : +case SFF8024_ID_QSFP_PLUS : +sff8636_show_all_paged (page_zero, page_three); +break ; +case SFF8024_ID_QSFP_DD : +case SFF8024_ID_DSFP : +cmis4_show_all (page_zero, page_one); +break ;default :dump_hex (stdout, page_zero->data, page_zero->length, page_zero->offset);break ;
qsfp-dd.c +35 -9
59 lines not on this page · raw message
qsfp-dd.h +16 -13
@@ -96,30 +96,33 @@/*----------------------------------------------------------------------- * Upper Memory Page 0x01: contains advertising fields that define properties * that are unique to active modules and cable assemblies. -* RealOffset = 1 * 0x80 + LocalOffset +* GlobalOffset = 2 * 0x80 + LocalOffset */ -#define PAG01H_UPPER_OFFSET (0x01 * 0x80) +#define PAG01H_UPPER_OFFSET (0x02 * 0x80)/* Supported Link Length (Page 1) */ -#define QSFP_DD_SMF_LEN_OFFSET (PAG01H_UPPER_OFFSET + 0x84) -#define QSFP_DD_OM5_LEN_OFFSET (PAG01H_UPPER_OFFSET + 0x85) -#define QSFP_DD_OM4_LEN_OFFSET (PAG01H_UPPER_OFFSET + 0x86) -#define QSFP_DD_OM3_LEN_OFFSET (PAG01H_UPPER_OFFSET + 0x87) -#define QSFP_DD_OM2_LEN_OFFSET (PAG01H_UPPER_OFFSET + 0x88) +#define QSFP_DD_SMF_LEN_OFFSET 0x84 +#define QSFP_DD_OM5_LEN_OFFSET 0x85 +#define QSFP_DD_OM4_LEN_OFFSET 0x86 +#define QSFP_DD_OM3_LEN_OFFSET 0x87 +#define QSFP_DD_OM2_LEN_OFFSET 0x88/* Wavelength (Page 1) */ -#define QSFP_DD_NOM_WAVELENGTH_MSB (PAG01H_UPPER_OFFSET + 0x8A) -#define QSFP_DD_NOM_WAVELENGTH_LSB (PAG01H_UPPER_OFFSET + 0x8B) -#define QSFP_DD_WAVELENGTH_TOL_MSB (PAG01H_UPPER_OFFSET + 0x8C) -#define QSFP_DD_WAVELENGTH_TOL_LSB (PAG01H_UPPER_OFFSET + 0x8D) +#define QSFP_DD_NOM_WAVELENGTH_MSB 0x8A +#define QSFP_DD_NOM_WAVELENGTH_LSB 0x8B +#define QSFP_DD_WAVELENGTH_TOL_MSB 0x8C +#define QSFP_DD_WAVELENGTH_TOL_LSB 0x8D/* Signal integrity controls */ -#define QSFP_DD_SIG_INTEG_TX_OFFSET (PAG01H_UPPER_OFFSET + 0xA1) -#define QSFP_DD_SIG_INTEG_RX_OFFSET (PAG01H_UPPER_OFFSET + 0xA2) +#define QSFP_DD_SIG_INTEG_TX_OFFSET 0xA1 +#define QSFP_DD_SIG_INTEG_RX_OFFSET 0xA2#define YESNO (x) (((x) != 0) ? "Yes" : "No")#define ONOFF (x) (((x) != 0) ? "On" : "Off")void qsfp_dd_show_all (const __u8 *id); +void cmis4_show_all (const struct ethtool_module_eeprom *page_zero, +const struct ethtool_module_eeprom *page_one); +#endif /* QSFP_DD_H__ */
qsfp.c +73 -55
167 lines not on this page · raw message
qsfp.h +27 -24
59 lines not on this page · raw message
sff-common.c +3 -0
@@ -139,6 +139,9 @@ void sff8024_show_identifier(const __u8 *id, int id_offset)case SFF8024_ID_QSFP_DD :printf (" (QSFP-DD Double Density 8X Pluggable Transceiver (INF-8628))\n" );break ; +case SFF8024_ID_DSFP : +printf (" (DSFP Dual Small Form Factor Pluggable Transceiver)\n" ); +break ;default :printf (" (reserved or unknown)\n" );break ;
sff-common.h +2 -1
@@ -62,7 +62,8 @@#define SFF8024_ID_CDFP_S3 0x16#define SFF8024_ID_MICRO_QSFP 0x17#define SFF8024_ID_QSFP_DD 0x18 -#define SFF8024_ID_LAST SFF8024_ID_QSFP_DD +#define SFF8024_ID_DSFP 0x1B +#define SFF8024_ID_LAST SFF8024_ID_DSFP#define SFF8024_ID_UNALLOCATED_LAST 0x7F#define SFF8024_ID_VENDOR_START 0x80#define SFF8024_ID_VENDOR_LAST 0xFF
Signature · 1 line
2.18.2
PATCH v2 3/4 ethtool: Rename QSFP-DD identifiers to use CMIS 4.0
In reply to Moshe ShemeshFrom: Vladyslav Tarasiuk <[email protected]>
QSFP-DD and DSFP EEPROM layout complies to CMIS 4.0 specification. As DSFP support is added, there are currently two standards, which share the same infrastructure. Rename QSFP_DD and qsfp_dd occurrences to use CMIS4 or cmis4 respectively to make function names generic for any module compliant to CMIS 4.0.
--- Makefile.am | 2 +- qsfp-dd.c => cmis4.c | 210 ++++++++++++++++++++-------------------- cmis4.h | 128 ++++++++++++++++++++++++ netlink/module-eeprom.c | 2 +- qsfp.c | 2 +- 5 files changed, 236 insertions(+), 108 deletions(-) rename qsfp-dd.c => cmis4.c (56%) create mode 100644 cmis4.h
Makefile.am +1 -1
@@ -17,7 +17,7 @@ ethtool_SOURCES += \ smsc911x.c at76c50x-usb.c sfc.c stmmac.c \ sff-common.c sff-common.h sfpid.c sfpdiag.c \ ixgbevf.c tse.c vmxnet3.c qsfp.c qsfp.h fjes.c lan78xx.c \ - igc.c qsfp-dd.c qsfp-dd.h bnxt.c + igc.c cmis4.c cmis4.h bnxt.c endif if ENABLE_BASH_COMPLETION
cmis4.c renamed from qsfp-dd.c +105 -105
402 lines not on this page · raw message
cmis4.h new file +128 -0
128 lines not on this page · raw message
netlink/module-eeprom.c +1 -1
@@ -11,7 +11,7 @@#include "../sff-common.h" #include "../qsfp.h" -#include "../qsfp-dd.h" +#include "../cmis4.h" #include "../internal.h" #include "../common.h" #include "../list.h"
qsfp.c +1 -1
@@ -58,7 +58,7 @@#include "internal.h" #include "sff-common.h" #include "qsfp.h" -#include "qsfp-dd.h" +#include "cmis4.h" #define MAX_DESC_SIZE 42
Signature · 1 line
2.18.2
PATCH v2 4/4 ethtool: Update manpages to reflect changes to getmodule (-m) command
In reply to Moshe ShemeshFrom: Vladyslav Tarasiuk <[email protected]>
Add page, bank and i2c parameters and mention change in offset and length treatment if either one of new parameters is specified by the user.
--- ethtool.8.in | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
ethtool.8.in +14 -0
@@ -365,6 +365,9 @@ ethtool \- query or control network driver and hardware settings .B2 hex on off .BN offset .BN length +.BN page +.BN bank +.BN i2c .HP .B ethtool \-\-show\-priv\-flags .I devname
@@ -1173,6 +1176,17 @@ Changes the number of multi-purpose channels. Retrieves and if possible decodes the EEPROM from plugin modules, e.g SFP+, QSFP. If the driver and module support it, the optical diagnostic information is also read and decoded. +When either one of +.I page, +.I bank +or +.I i2c +parameters is specified, dumps only of a single page or its portion is +allowed. In such a case +.I offset +and +.I length +parameters are treated relatively to EEPROM page boundaries. .TP .B \-\-show\-priv\-flags Queries the specified network device for its private flags. The
Signature · 1 line
2.18.2
Re: [PATCH ethtool v2 3/4] ethtool: Rename QSFP-DD identifiers to use CMIS 4.0
In reply to Moshe ShemeshQuoted 10 lines · Moshe Shemesh
From: Vladyslav Tarasiuk <[email protected]>
QSFP-DD and DSFP EEPROM layout complies to CMIS 4.0 specification. As DSFP support is added, there are currently two standards, which share the same infrastructure. Rename
QSFP_DDandqsfp_ddoccurrences to use CMIS4 or cmis4 respectively to make function names generic for any module compliant to CMIS 4.0.Signed-off-by Vladyslav Tarasiuk <[email protected]>Reviewed-by Moshe Shemesh <[email protected]>
Quoted 9 lines
--- Makefile.am | 2 +- qsfp-dd.c => cmis4.c | 210 ++++++++++++++++++++-------------------- cmis4.h | 128 ++++++++++++++++++++++++ netlink/module-eeprom.c | 2 +- qsfp.c | 2 +- 5 files changed, 236 insertions(+), 108 deletions(-) rename qsfp-dd.c => cmis4.c (56%) create mode 100644 cmis4.h
Is there a reason to call this "cmis4" instead of just "cmis"? Revision 5.0 was published earlier this month [1] and I assume more revisions will follow.
Other standards (e.g., SFF-8024) also have multiple revisions and the revision number is only mentioned in the "revision compliance" field.
[1] http://www.qsfp-dd.com/wp-content/uploads/2021/05/CMIS5p0.pdf
Re: [PATCH ethtool v2 3/4] ethtool: Rename QSFP-DD identifiers to use CMIS 4.0
Quoted 20 lines · Ido Schimmel
Quoted 17 lines · Moshe Shemesh
From: Vladyslav Tarasiuk <[email protected]>
QSFP-DD and DSFP EEPROM layout complies to CMIS 4.0 specification. As DSFP support is added, there are currently two standards, which share the same infrastructure. Rename
QSFP_DDandqsfp_ddoccurrences to use CMIS4 or cmis4 respectively to make function names generic for any module compliant to CMIS 4.0.Signed-off-by Vladyslav Tarasiuk <[email protected]>Reviewed-by Moshe Shemesh <[email protected]>--- Makefile.am | 2 +- qsfp-dd.c => cmis4.c | 210 ++++++++++++++++++++-------------------- cmis4.h | 128 ++++++++++++++++++++++++ netlink/module-eeprom.c | 2 +- qsfp.c | 2 +- 5 files changed, 236 insertions(+), 108 deletions(-)Quoted 2 lines
rename
qsfp-dd.c=>cmis4.c(56%)
create mode 100644cmis4.h
Quoted 3 lines
Is there a reason to call this "cmis4" instead of just "cmis"? Revision 5.0 was published earlier this month [1] and I assume more revisions will follow.
We called it cmis4 as we comply here with CMIS version 4.
However, I understand your point that other eeprom module specifications only spec number is mentioned and probably we can do the same here.
Andrew, Michal, WDYT ?
Quoted 4 lines
Other standards (e.g., SFF-8024) also have multiple revisions and the revision number is only mentioned in the "revision compliance" field.
[1] http://www.qsfp-dd.com/wp-content/uploads/2021/05/CMIS5p0.pdf