Support for RollBall 10G copper SFP modules
Subsystems (4)
v2 cover Support for RollBall 10G copper SFP modules
Hi Russell,
this series should apply on linux-arm git repository, on branch net-queue.
Some internet providers are already starting to offer 2.5G copper connectivity to their users. On Turris Omnia the SFP port is capable of 2.5G speed, so we tested some copper SFP modules.
This adds support to the SFP subsystem for 10G RollBall copper modules which contain a Marvell 88X3310 PHY. By default these modules are configured in 10GKR only mode on the host interface, and also contain some bad information in EEPROM (the extended_cc byte).
The PHY in these modules is also accessed via a different I2C protocol than the standard one.
Patch 1 adds support for this different I2C MDIO bus.
Patch 2 adds support for these modules into the SFP driver. Patch 3 changes phylink code so that a PHY can be attached even though 802.3z mode is requested.
Marek
Marek Behún (3):
net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules
net: phy: sfp: add support for multigig RollBall modules
net: phylink: don't fail attaching phy on 1000base-x/2500base-x mode
drivers/net/phy/mdio-i2c.c | 196 +++++++++++++++++++++++++++++++++++-- drivers/net/phy/phylink.c | 4 +- drivers/net/phy/sfp.c | 69 +++++++++++-- 3 files changed, 250 insertions(+), 19 deletions(-)
base-commit: a32e90737c1c92653767d3c95c63c16b9b72c6c2
Signature · 1 line
2.26.2
PATCH v2 1/3 net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules
Some multigig SFPs from RollBall and Hilink do not expose functional MDIO access to the internal PHY of the SFP via I2C address 0x56 (although there seems to be read-only clause 22 access on this address).
Instead these SFPs PHY can be accessed via I2C via the SFP Enhanced Digital Diagnostic Interface - I2C address 0x51.
This extends the mdio-i2c driver so that when SFP PHY address 17 is used (which in mdio-i2c terms corresponds to I2C address 0x51), then this different protocol is used for MDIO access.
--- drivers/net/phy/mdio-i2c.c | 196 +++++++++++++++++++++++++++++++++++-- 1 file changed, 186 insertions(+), 10 deletions(-)
drivers/net/phy/mdio-i2c.c +186 -10
@@ -3,6 +3,7 @@* MDIO I2C bridge * * Copyright (C) 2015-2016 Russell King +* Copyright (C) 2020 Marek Behun * * Network PHYs can appear on I2C buses when they are part of SFP module. * This driver exposes these PHYs to the networking PHY code, allowing
@@ -17,11 +18,17 @@/* * I2C bus addresses 0x50 and 0x51 are normally an EEPROM, which is * specified to be present in SFP modules. These correspond with PHY -* addresses 16 and 17. Disallow access to these "phy" addresses. +* addresses 16 and 17. Disallow access to 0x50 "phy" address. +* Use RollBall protocol when accessing via the 0x51 address. */ static bool i2c_mii_valid_phy_id (int phy_id) { -return phy_id !=0x10 && phy_id !=0x11 ; +return phy_id !=0x10 ; +} + +static bool i2c_mii_rollball_phy_id (int phy_id) +{ +return phy_id ==0x11 ; }static unsigned int i2c_mii_phy_addr (int phy_id)
@@ -29,16 +36,13 @@ static unsigned int i2c_mii_phy_addr(int phy_id)return phy_id +0x40 ; } -static int i2c_mii_read (struct mii_bus *bus,int phy_id,int reg) +static int i2c_mii_read_default (struct mii_bus *bus,int phy_id,int reg) {struct i2c_adapter *i2c = bus->priv;struct i2c_msg msgs[2 ];u8 addr[3 ], data[2 ], *p;int bus_addr, ret; -if (!i2c_mii_valid_phy_id (phy_id)) -return 0xffff ; - p = addr;if (reg &MII_ADDR_C45 ) { *p++ =0x20 | ((reg >>16 ) &31 );
@@ -63,16 +67,13 @@ static int i2c_mii_read(struct mii_bus *bus, int phy_id, int reg)return data[0 ] <<8 | data[1 ]; } -static int i2c_mii_write (struct mii_bus *bus,int phy_id,int reg,u16 val) +static int i2c_mii_write_default (struct mii_bus *bus,int phy_id,int reg,u16 val) {struct i2c_adapter *i2c = bus->priv;struct i2c_msg msg;int ret;u8 data[5 ], *p; -if (!i2c_mii_valid_phy_id (phy_id)) -return 0 ; - p = data;if (reg &MII_ADDR_C45 ) { *p++ = (reg >>16 ) &31 ;
@@ -92,6 +93,181 @@ static int i2c_mii_write(struct mii_bus *bus, int phy_id, int reg, u16 val)return ret <0 ? ret :0 ; } +/* RollBall SFPs do not access internal PHY via I2C address 0x56, but +* instead via address 0x51, when SFP page is set to 0x03 and password to +* 0xffffffff: +* +* address size contents description +* ------- ---- -------- ----------- +* 0x80 1 CMD 0x01/0x02/0x04 for write/read/done +* 0x81 1 DEV Clause 45 device +* 0x82 2 REG Clause 45 register +* 0x84 2 VAL Register value +*/ +#define ROLLBALL_CMD_ADDR 0x80 +#define ROLLBALL_DATA_ADDR 0x81 + +#define ROLLBALL_CMD_WRITE 0x01 +#define ROLLBALL_CMD_READ 0x02 +#define ROLLBALL_CMD_DONE 0x04 + +static int i2c_rollball_mii_poll (struct mii_bus *bus,int bus_addr,u8 *buf,size_t len) +{ +struct i2c_adapter *i2c = bus->priv; +struct i2c_msg msgs[2 ]; +u8 buf0[2 ], *res; +int i, ret; + + buf0[0 ] =ROLLBALL_CMD_ADDR ; + + msgs[0 ].addr = bus_addr; + msgs[0 ].flags =0 ; + msgs[0 ].len =1 ; + msgs[0 ].buf = &buf0[0 ]; + + res = buf ? buf : &buf0[1 ]; + + msgs[1 ].addr = bus_addr; + msgs[1 ].flags =I2C_M_RD ; + msgs[1 ].len = buf ? len :1 ; + msgs[1 ].buf = res; + +/* By experiment it takes up to 70 ms to access a register for these SFPs. Sleep 20ms +* between iteratios and try 10 times. +*/ + i =10 ; +do { +msleep (20 ); + + ret =i2c_transfer (i2c, msgs,ARRAY_SIZE (msgs)); +if (ret <0 ) +return ret; +else if (ret !=ARRAY_SIZE (msgs)) +return -EIO ; + +if (*res ==ROLLBALL_CMD_DONE ) +return 0 ; + }while (i-- >0 ); + +dev_dbg (&bus->dev,"poll timed out\n" ); + +return -ETIMEDOUT ; +} + +static int i2c_rollball_mii_cmd (struct mii_bus *bus,int bus_addr,u8 cmd,u8 *data,size_t len) +{ +struct i2c_adapter *i2c = bus->priv; +struct i2c_msg msgs[2 ]; +u8 cmdbuf[2 ]; +int ret; + + msgs[0 ].addr = bus_addr; + msgs[0 ].flags =0 ; + msgs[0 ].len = len; + msgs[0 ].buf = data; + + cmdbuf[0 ] =ROLLBALL_CMD_ADDR ; + cmdbuf[1 ] = cmd; + + msgs[1 ].addr = bus_addr; + msgs[1 ].flags =0 ; + msgs[1 ].len =sizeof (cmdbuf); + msgs[1 ].buf = cmdbuf; + + ret =i2c_transfer (i2c, msgs,2 ); +if (ret <0 ) +return ret; + +return ret ==ARRAY_SIZE (msgs) ?0 : -EIO ; +} + +static int i2c_mii_read_rollball (struct mii_bus *bus,int phy_id,int reg) +{ +u8 buf[4 ], res[6 ]; +int bus_addr, ret; +u16 val; + +if (!(reg &MII_ADDR_C45 )) +return -EOPNOTSUPP ; + + buf[0 ] =ROLLBALL_DATA_ADDR ; + buf[1 ] = (reg >>16 ) &0x1f ; + buf[2 ] = (reg >>8 ) &0xff ; + buf[3 ] = reg &0xff ; + + bus_addr =i2c_mii_phy_addr (phy_id); + + ret =i2c_rollball_mii_cmd (bus, bus_addr,ROLLBALL_CMD_READ , buf,sizeof (buf)); +if (ret <0 ) +return ret; + + ret =i2c_rollball_mii_poll (bus, bus_addr, res,sizeof (res)); +if (ret == -ETIMEDOUT ) +return 0xffff ; +else if (ret <0 ) +return ret; + + val = res[4 ]; + val <<=8 ; + val |= res[5 ]; + +dev_dbg (&bus->dev,"read reg %02x:%04x = %04x\n" , (reg >>16 ) &0x1f , reg &0xffff , val); + +return val; +} + +static int i2c_mii_write_rollball (struct mii_bus *bus,int phy_id,int reg,u16 val) +{ +int bus_addr, ret; +u8 buf[6 ]; + +if (!(reg &MII_ADDR_C45 )) +return -EOPNOTSUPP ; + + buf[0 ] =ROLLBALL_DATA_ADDR ; + buf[1 ] = (reg >>16 ) &0x1f ; + buf[2 ] = (reg >>8 ) &0xff ; + buf[3 ] = reg &0xff ; + buf[4 ] = val >>8 ; + buf[5 ] = val &0xff ; + + bus_addr =i2c_mii_phy_addr (phy_id); + + ret =i2c_rollball_mii_cmd (bus, bus_addr,ROLLBALL_CMD_WRITE , buf,sizeof (buf)); +if (ret <0 ) +return ret; + + ret =i2c_rollball_mii_poll (bus, bus_addr,NULL ,0 ); +if (ret <0 ) +return ret; + +dev_dbg (&bus->dev,"write reg %02x:%04x = %04x\n" , (reg >>16 ) &0x1f , reg &0xffff , val); + +return 0 ; +} + +static int i2c_mii_read (struct mii_bus *bus,int phy_id,int reg) +{ +if (!i2c_mii_valid_phy_id (phy_id)) +return 0xffff ; + +if (i2c_mii_rollball_phy_id (phy_id)) +return i2c_mii_read_rollball (bus, phy_id, reg); +else +return i2c_mii_read_default (bus, phy_id, reg); +} + +static int i2c_mii_write (struct mii_bus *bus,int phy_id,int reg,u16 val) +{ +if (!i2c_mii_valid_phy_id (phy_id)) +return 0 ; + +if (i2c_mii_rollball_phy_id (phy_id)) +return i2c_mii_write_rollball (bus, phy_id, reg, val); +else +return i2c_mii_write_default (bus, phy_id, reg, val); +} +struct mii_bus *mdio_i2c_alloc (struct device *parent,struct i2c_adapter *i2c) {struct mii_bus *mii;
Signature · 1 line
2.26.2
PATCH v2 3/3 net: phylink: don't fail attaching phy on 1000base-x/2500base-x mode
In reply to Marek BehúnSome SFPs may contain an internal PHY which may in some cases want to connect with the host interface in 1000base-x/2500base-x mode. Do not fail if such PHY is being attached in one of these PHY interface modes.
--- drivers/net/phy/phylink.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
drivers/net/phy/phylink.c +1 -3
@@ -1017,9 +1017,7 @@ static int phylink_bringup_phy(struct phylink *pl, struct phy_device *phy,static int phylink_attach_phy (struct phylink *pl,struct phy_device *phy,phy_interface_t interface) { -if (WARN_ON (pl->cfg_link_an_mode ==MLO_AN_FIXED || - (pl->cfg_link_an_mode ==MLO_AN_INBAND && -phy_interface_mode_is_8023z (interface)))) +if (WARN_ON (pl->cfg_link_an_mode ==MLO_AN_FIXED ))return -EINVAL ;if (pl->phydev)
Signature · 1 line
2.26.2
PATCH v2 2/3 net: phy: sfp: add support for multigig RollBall modules
In reply to Marek BehúnThis adds support for multigig copper SFP modules from RollBall/Hilink. These modules have a specific way to access clause 45 registers of the internal PHY.
We also need to wait at least 25 seconds after deasserting TX disable before accessing the PHY. The code waits for 30 seconds just to be sure.
--- drivers/net/phy/sfp.c | 69 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 63 insertions(+), 6 deletions(-)
drivers/net/phy/sfp.c +63 -6
@@ -167,6 +167,7 @@ static const enum gpiod_flags gpio_flags[] = {* on board (for a copper SFP) time to initialise. */ #define T_WAIT msecs_to_jiffies(50) +#define T_WAIT_LONG_PHY msecs_to_jiffies(30000)#define T_START_UP msecs_to_jiffies(300)#define T_START_UP_BAD_GPON msecs_to_jiffies(60000)#define T_START_UP_COOLED msecs_to_jiffies(90000)
@@ -207,8 +208,11 @@ static const enum gpiod_flags gpio_flags[] = {/* SFP modules appear to always have their PHY configured for bus address * 0x56 (which with mdio-i2c, translates to a PHY address of 22). +* RollBall SFPs access phy via SFP Enhanced Digital Diagnostic Interface +* via address 0x51 (mdio-i2c will use RollBall protocol on this address). */ -#define SFP_PHY_ADDR 22 +#define SFP_PHY_ADDR 22 +#define SFP_PHY_ADDR_ROLLBALL 17struct sff_data {unsigned int gpios;
@@ -223,6 +227,7 @@ struct sfp {struct phy_device *mod_phy;const struct sff_data *type;u32 max_power_mW; +int phy_addr;unsigned int (*get_state)(struct sfp *);void (*set_state)(struct sfp *,unsigned int );
@@ -251,6 +256,7 @@ struct sfp {struct sfp_eeprom_id id;unsigned int module_power_mW;unsigned int module_t_start_up; +unsigned int module_t_wait;#if IS_ENABLED (CONFIG_HWMON )struct sfp_diag diag;
@@ -1594,7 +1600,7 @@ static int sfp_sm_probe_phy(struct sfp *sfp, bool is_c45)struct phy_device *phy;int err; - phy =get_phy_device (sfp->i2c_mii,SFP_PHY_ADDR , is_c45); + phy =get_phy_device (sfp->i2c_mii, sfp->phy_addr, is_c45);if (phy ==ERR_PTR (-ENODEV ))return PTR_ERR (phy);if (IS_ERR (phy)) {
@@ -1931,12 +1937,40 @@ static void sfp_print_module_info(struct sfp *sfp, const struct sfp_eeprom_id *i id->ext.enhopts)); } +static int sfp_rollball_init_mdio (struct sfp *sfp) +{ +u8 page, password[4 ]; +int err; + + page =3 ; + + err =sfp_write (sfp, true,SFP_PAGE , &page,1 ); +if (err !=1 ) { +dev_err (sfp->dev,"Failed to set SFP page for RollBall MDIO access: %d\n" , err); +return err; + } + + password[0 ] =0xff ; + password[1 ] =0xff ; + password[2 ] =0xff ; + password[3 ] =0xff ; + + err =sfp_write (sfp, true,0x7b , password,4 ); +if (err !=4 ) { +dev_err (sfp->dev,"Failed to write password for RollBall MDIO access; %d\n" , err); +return err; + } + +return 0 ; +} +static int sfp_sm_mod_probe (struct sfp *sfp,bool report) {/* SFP module inserted - read I2C data */ struct sfp_eeprom_id id;bool cotsworks_sfbg;bool cotsworks; +bool rollball;u8 check;int ret;
@@ -2006,6 +2040,23 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report) sfp->id = id; + sfp->phy_addr =SFP_PHY_ADDR ; + + rollball = ((!memcmp (id.base.vendor_name,"OEM " ,16 ) || + !memcmp (id.base.vendor_name,"Turris " ,16 )) && + (!memcmp (id.base.vendor_pn,"SFP-10G-T " ,16 ) || + !memcmp (id.base.vendor_pn,"RTSFP-10" ,8 ))); +if (rollball) { + sfp->phy_addr =SFP_PHY_ADDR_ROLLBALL ; + ret =sfp_rollball_init_mdio (sfp); +if (ret <0 ) +return ret; + +/* RollBall SFPs may have wrong (zero) extended compliacne code burned in EEPROM. +* To try to probe for PHY we need the correct one */ + id.base.extended_cc =SFF8024_ECC_10GBASE_T_SFI ; + } +/* Check whether we support this module */ if (!sfp->type->module_supported (&id)) { dev_err(sfp->dev,
@@ -2032,6 +2083,11 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report)else sfp->module_t_start_up =T_START_UP ; +if (rollball) + sfp->module_t_wait =T_WAIT_LONG_PHY ; +else + sfp->module_t_wait =T_WAIT ; +return 0 ; }
@@ -2225,9 +2281,10 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event)/* We need to check the TX_FAULT state, which is not defined * while TX_DISABLE is asserted. The earliest we want to do -* anything (such as probe for a PHY) is 50ms. +* anything (such as probe for a PHY) is 50ms (or more on +* specific modules). */ -sfp_sm_next (sfp,SFP_S_WAIT ,T_WAIT ); +sfp_sm_next (sfp,SFP_S_WAIT , sfp->module_t_wait);break ;case SFP_S_WAIT :
@@ -2241,8 +2298,8 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event)* indicated by TX_FAULT deasserting. */ timeout = sfp->module_t_start_up; -if (timeout >T_WAIT ) - timeout -=T_WAIT ; +if (timeout > sfp->module_t_wait) + timeout -= sfp->module_t_wait;else timeout =1 ;
Signature · 1 line
2.26.2
Andrew Lunn
maintainerRe: [PATCH russell-kings-net-queue v2 2/3] net: phy: sfp: add support for multigig RollBall modules
drivers/net/phy/sfp.c static int sfp_sm_mod_probe(struct sfp *sfp, bool report) · line 2048
Quoted 10 lines
drivers/net/phy/sfp.c +6
-0@@ -2006,6 +2040,23 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report) sfp->id = id; + sfp->phy_addr =SFP_PHY_ADDR ; + + rollball = ((!memcmp (id.base.vendor_name,"OEM " ,16 ) || + !memcmp (id.base.vendor_name,"Turris " ,16 )) && + (!memcmp (id.base.vendor_pn,"SFP-10G-T " ,16 ) || + !memcmp (id.base.vendor_pn,"RTSFP-10" ,8 )));
Are you customising the SFP, so that it has your vendor name?
Is the generic SFP OEM/SFP-10G-T, and your customized one Turris/ RTSFP-10?
Andrew
Andrew Lunn
maintainerRe: [PATCH russell-kings-net-queue v2 1/3] net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules
In reply to Marek BehúnQuoted 3 lines
This extends the mdio-i2c driver so that when SFP PHY address 17 is used (which in mdio-i2c terms corresponds to I2C address
0x51), then this different protocol is used for MDIO access.
Hi Marek
I don't see that being very scalable. What happens when the next SFP comes along which has a different protocol at address 0x51. Since you can identify the SFP via the EEPROM information, i would prefer you explicitly tell it to use the rollball protocol when instantiating the MDIO bus.
drivers/net/phy/mdio-i2c.c line 27
Quoted 11 lines
* I2C bus addresses
0x50and0x51are normally an EEPROM, which is
* specified to be present in SFP modules. These correspond with PHY
- * addresses 16 and 17. Disallow access to these "phy" addresses.
+ * addresses 16 and 17. Disallow access to0x50"phy" address.
+ * Use RollBall protocol when accessing via the0x51address. */
static booli2c_mii_valid_phy_id(int phy_id)
{drivers/net/phy/mdio-i2c.c +2
-1-return phy_id !=0x10 && phy_id !=0x11 ; +return phy_id !=0x10 ; +}
I'm not sure that is safe. It means that we will scan address 0x11 to see if there is a PHY there. And if the SFP does have diagnostics registers, that might be enough that phylib thinks there is a PHY there.
I think you only need to allow access to 0x11 if rollball protocol has been enabled.
Andrew
Re: [PATCH russell-kings-net-queue v2 2/3] net: phy: sfp: add support for multigig RollBall modules
In reply to Andrew LunnQuoted 13 lines · On Tue, 20 Oct 2020 17:51:26 +0200 Andrew Lunn <[email protected]> wrote:
drivers/net/phy/sfp.c static int sfp_sm_mod_probe(struct sfp *sfp, bool report) · line 2048
Quoted 10 lines
drivers/net/phy/sfp.c +6
-0@@ -2006,6 +2040,23 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report) sfp->id = id; + sfp->phy_addr =SFP_PHY_ADDR ; + + rollball = ((!memcmp (id.base.vendor_name,"OEM " ,16 ) || + !memcmp (id.base.vendor_name,"Turris " ,16 )) && + (!memcmp (id.base.vendor_pn,"SFP-10G-T " ,16 ) || + !memcmp (id.base.vendor_pn,"RTSFP-10" ,8 )));Are you customising the SFP, so that it has your vendor name?
Quoted 4 lines
Is the generic SFP OEM/SFP-10G-T, and your customized one Turris/ RTSFP-10?
Andrew
Hilink puts OEM/SFP-10G-T into their modules. RollBall puts OEM/RTSFP-10 and sometimes OEM/RTSFP-10G. They are rebranding these modules for us to Turris/RTSFP-10.
Marek
Re: [PATCH russell-kings-net-queue v2 1/3] net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules
In reply to Andrew LunnQuoted 11 lines · On Tue, 20 Oct 2020 18:00:39 +0200 Andrew Lunn <[email protected]> wrote:
Quoted 3 lines
This extends the mdio-i2c driver so that when SFP PHY address 17 is used (which in mdio-i2c terms corresponds to I2C address
0x51), then this different protocol is used for MDIO access.Hi Marek
I don't see that being very scalable. What happens when the next SFP comes along which has a different protocol at address
0x51. Since you can identify the SFP via the EEPROM information, i would prefer you explicitly tell it to use the rollball protocol when instantiating the MDIO bus.
At first I proposed a separate mdio bus driver for RollBall SFPs. But Russell suggested doing this instead, saying that in the future this can be changed.
Quoted 18 lines
drivers/net/phy/mdio-i2c.c line 27
Quoted 12 lines
* I2C bus addresses
0x50and0x51are normally an EEPROM, which is
* specified to be present in SFP modules. These correspond with PHY
- * addresses 16 and 17. Disallow access to these "phy" addresses.
+ * addresses 16 and 17. Disallow access to0x50"phy" address.
+ * Use RollBall protocol when accessing via the0x51address. */
static booli2c_mii_valid_phy_id(int phy_id)
{drivers/net/phy/mdio-i2c.c +2
-1-return phy_id !=0x10 && phy_id !=0x11 ; +return phy_id !=0x10 ; +}I'm not sure that is safe. It means that we will scan address
0x11to see if there is a PHY there. And if the SFP does have diagnostics registers, that might be enough that phylib thinks there is a PHY there.
Quoted 2 lines
I think you only need to allow access to
0x11if rollball protocol has been enabled.
I can do that...
Quoted 2 lines
Andrew