net: ethernet: mtk_eth_soc: various enhancements

Gesendet: Dienstag, 07. März 2023 um 16:53 Uhr
Von: "Daniel Golle" <[email protected]>

Reset the internal PCS state machine when changing interface mode. This prevents confusing the state machine when changing interface modes, e.g. from SGMII to 2500Base-X or vice-versa.

Reviewed-by Russell King (Oracle) <[email protected]>

Tested-by Bjørn Mork <[email protected]>

Signed-off-by Daniel Golle <[email protected]>

Gesendet: Dienstag, 07. März 2023 um 16:53 Uhr
Von: "Daniel Golle" <[email protected]>
Only restart auto-negotiation and write link timer if actually necessary. This prevents loosing the link in case of minor changes.

Gesendet: Dienstag, 07. März 2023 um 16:53 Uhr
Von: "Daniel Golle" <[email protected]>

After conversion to phylink_pcs the 1000Base-X and 2500Base-X modes would work only after ethtool -s eth1 autoneg off.
As ethtool autoneg and the ETHTOOL_LINK_MODE_Autoneg_BIT is supposed to control auto-negotiation on the external interface it doesn't make much sense to use it to control on-board SGMII auto-negotiation between MAC and PHY.
Set correct values to really only enable SGMII auto-negotiation when actually operating in SGMII mode. For 1000Base-X and 2500Base-X mode, enable remote-fault detection only if in-band-status is enabled. This fixes using 1000Base-X and 2500Base-X SFPs on the BananaPi R3 board and also makes it possible to use interface-mode-switching PHYs operating in either SGMII mode for 10M/100M/1000M or in 2500Base-X for 2500M mode on other boards.

After conversion to phylink_pcs the 1000Base-X and 2500Base-X modes would work only after ethtool -s eth1 autoneg off.
As ethtool autoneg and the ETHTOOL_LINK_MODE_Autoneg_BIT is supposed to control auto-negotiation on the external interface it doesn't make much sense to use it to control on-board SGMII auto-negotiation between MAC and PHY.
Set correct values to really only enable SGMII auto-negotiation when actually operating in SGMII mode. For 1000Base-X and 2500Base-X mode, enable remote-fault detection only if in-band-status is enabled. This fixes using 1000Base-X and 2500Base-X SFPs on the BananaPi R3 board and also makes it possible to use interface-mode-switching PHYs operating in either SGMII mode for 10M/100M/1000M or in 2500Base-X for 2500M mode on other boards.

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

Quoted 14 lines · Daniel Golle

After conversion to phylink_pcs the 1000Base-X and 2500Base-X modes would work only after ethtool -s eth1 autoneg off.
As ethtool autoneg and the ETHTOOL_LINK_MODE_Autoneg_BIT is supposed to control auto-negotiation on the external interface it doesn't make much sense to use it to control on-board SGMII auto-negotiation between MAC and PHY.
Set correct values to really only enable SGMII auto-negotiation when actually operating in SGMII mode. For 1000Base-X and 2500Base-X mode, enable remote-fault detection only if in-band-status is enabled. This fixes using 1000Base-X and 2500Base-X SFPs on the BananaPi R3 board and also makes it possible to use interface-mode-switching PHYs operating in either SGMII mode for 10M/100M/1000M or in 2500Base-X for 2500M mode on other boards.

Quoted 2 lines

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

NAK.

There are PHYs out there which operate in SGMII mode but do not exchange the SGMII 16-bit configuration word. The code implemented here by me was explicitly to allow such a configuration to work, which is defined as:

	SGMII *without* mode == inband

An example of this is the Broadcom 84881 PHY which can be found on SFP modules.

Quoted 25 lines · Russell King (Oracle)

Quoted 14 lines · Daniel Golle

After conversion to phylink_pcs the 1000Base-X and 2500Base-X modes would work only after ethtool -s eth1 autoneg off.
As ethtool autoneg and the ETHTOOL_LINK_MODE_Autoneg_BIT is supposed to control auto-negotiation on the external interface it doesn't make much sense to use it to control on-board SGMII auto-negotiation between MAC and PHY.
Set correct values to really only enable SGMII auto-negotiation when actually operating in SGMII mode. For 1000Base-X and 2500Base-X mode, enable remote-fault detection only if in-band-status is enabled. This fixes using 1000Base-X and 2500Base-X SFPs on the BananaPi R3 board and also makes it possible to use interface-mode-switching PHYs operating in either SGMII mode for 10M/100M/1000M or in 2500Base-X for 2500M mode on other boards.

Quoted 2 lines

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

NAK.

There are PHYs out there which operate in SGMII mode but do not exchange the SGMII 16-bit configuration word. The code implemented here by me was explicitly to allow such a configuration to work, which is defined as:

Quoted 4 lines

	SGMII *without* mode == inband

An example of this is the Broadcom 84881 PHY which can be found on SFP modules.

I also have multiple such 1000Base-T SFP modules here (finisar, AJYA), and this change doesn't touch the codepaths relevant for those. They are operating in SGMII mode, they have always been working fine.

What I'm trying to fix here is 1000Base-X and 2500Base-X mode which has been broken by introducing ETHTOOL_LINK_MODE_Autoneg_BIT as the deciding factor for in-band AN here.

Can you explain why ETHTOOL_LINK_MODE_Autoneg_BIT was used there in first place? Is my understanding of this bit controlling autoneg on the *external* interface rather than on the *system-side* interface wrong?

Quoted 31 lines · Daniel Golle

Quoted 25 lines · Russell King (Oracle)

Quoted 14 lines · Daniel Golle

After conversion to phylink_pcs the 1000Base-X and 2500Base-X modes would work only after ethtool -s eth1 autoneg off.
As ethtool autoneg and the ETHTOOL_LINK_MODE_Autoneg_BIT is supposed to control auto-negotiation on the external interface it doesn't make much sense to use it to control on-board SGMII auto-negotiation between MAC and PHY.
Set correct values to really only enable SGMII auto-negotiation when actually operating in SGMII mode. For 1000Base-X and 2500Base-X mode, enable remote-fault detection only if in-band-status is enabled. This fixes using 1000Base-X and 2500Base-X SFPs on the BananaPi R3 board and also makes it possible to use interface-mode-switching PHYs operating in either SGMII mode for 10M/100M/1000M or in 2500Base-X for 2500M mode on other boards.

Quoted 2 lines

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

NAK.

There are PHYs out there which operate in SGMII mode but do not exchange the SGMII 16-bit configuration word. The code implemented here by me was explicitly to allow such a configuration to work, which is defined as:

Quoted 4 lines

	SGMII *without* mode == inband

An example of this is the Broadcom 84881 PHY which can be found on SFP modules.

Quoted 7 lines

I also have multiple such 1000Base-T SFP modules here (finisar, AJYA), and this change doesn't touch the codepaths relevant for those. They are operating in SGMII mode, they have always been working fine.

What I'm trying to fix here is 1000Base-X and 2500Base-X mode which has been broken by introducing ETHTOOL_LINK_MODE_Autoneg_BIT as the deciding factor for in-band AN here.

... which is correct.

Quoted 3 lines

Can you explain why ETHTOOL_LINK_MODE_Autoneg_BIT was used there in first place? Is my understanding of this bit controlling autoneg on the *external* interface rather than on the *system-side* interface wrong?

Think about what 1000BASE-X is for. It's not really for internal links, it's intended by IEEE 802.3 to be the 1G *media* side protocol for 1000BASE-SX, 1000BASE-LX, 1000BASE-CX etc links.

Therefore, when being used in that case, one may wish to disable autoneg over the fibre link. Hence, turning off autoneg via ethtool *should* turn off autoneg over the fibre link. So, using ETHTOOL_LINK_MODE_Autoneg_BIT to gate 802.3z autonegotiation the correct thing to do.

If we have a PHY using 1000BASE-X, then it is at odds with the primary purpose of this protocol, especially with it comes to AN. This is why phylink used to refuse to accept PHYs when using 802.3z mode, but Marek wanted this to work, so relaxed the checks preventing such a setup working.

Quoted 56 lines · Russell King (Oracle)

Quoted 31 lines · Daniel Golle

Quoted 25 lines · Russell King (Oracle)

Quoted 14 lines · Daniel Golle

After conversion to phylink_pcs the 1000Base-X and 2500Base-X modes would work only after ethtool -s eth1 autoneg off.
As ethtool autoneg and the ETHTOOL_LINK_MODE_Autoneg_BIT is supposed to control auto-negotiation on the external interface it doesn't make much sense to use it to control on-board SGMII auto-negotiation between MAC and PHY.
Set correct values to really only enable SGMII auto-negotiation when actually operating in SGMII mode. For 1000Base-X and 2500Base-X mode, enable remote-fault detection only if in-band-status is enabled. This fixes using 1000Base-X and 2500Base-X SFPs on the BananaPi R3 board and also makes it possible to use interface-mode-switching PHYs operating in either SGMII mode for 10M/100M/1000M or in 2500Base-X for 2500M mode on other boards.

Quoted 2 lines

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

NAK.

There are PHYs out there which operate in SGMII mode but do not exchange the SGMII 16-bit configuration word. The code implemented here by me was explicitly to allow such a configuration to work, which is defined as:

Quoted 4 lines

	SGMII *without* mode == inband

An example of this is the Broadcom 84881 PHY which can be found on SFP modules.

Quoted 7 lines

I also have multiple such 1000Base-T SFP modules here (finisar, AJYA), and this change doesn't touch the codepaths relevant for those. They are operating in SGMII mode, they have always been working fine.

What I'm trying to fix here is 1000Base-X and 2500Base-X mode which has been broken by introducing ETHTOOL_LINK_MODE_Autoneg_BIT as the deciding factor for in-band AN here.

... which is correct.

Quoted 3 lines

Can you explain why ETHTOOL_LINK_MODE_Autoneg_BIT was used there in first place? Is my understanding of this bit controlling autoneg on the *external* interface rather than on the *system-side* interface wrong?

Think about what 1000BASE-X is for. It's not really for internal links, it's intended by IEEE 802.3 to be the 1G *media* side protocol for 1000BASE-SX, 1000BASE-LX, 1000BASE-CX etc links.

Therefore, when being used in that case, one may wish to disable autoneg over the fibre link. Hence, turning off autoneg via ethtool *should* turn off autoneg over the fibre link. So, using ETHTOOL_LINK_MODE_Autoneg_BIT to gate 802.3z autonegotiation the correct thing to do.

Quoted 5 lines

If we have a PHY using 1000BASE-X, then it is at odds with the primary purpose of this protocol, especially with it comes to AN. This is why phylink used to refuse to accept PHYs when using 802.3z mode, but Marek wanted this to work, so relaxed the checks preventing such a setup working.

Sadly 2500Base-X is very commonly used to connect 2500Base-T-capable PHYs or SFP modules. I also got an ATS branded 1000M/100M/10M copper SFP module which uses 1000Base-X as system-side interface, independently of the speed of the link partner on the TP interface.
All of them do not work with inband-AN enabled and a link only comes up after ethtool -s eth1 autoneg off in the current implementation, while previously they were working just fine.

I understand that there isn't really a good solution for 1000Base-X as thanks to you I now understand that SerDes autoneg just transparently ends up being autoneg on a fiber link.

2500Base-X, however, is hardly used for fiber links, but rather mostly for 2500Base-T PHYs and SFP module as well as xPON SFPs. Maybe we could at least have in-band AN disabled by default for those to get them working without requiring the user to carry out ethtool settings?

So, what I would want to do is to move the decision about whether the PCS should enable in-band into the phylink core code instead of these random decisions being made in each PCS.

At that point, we can then make the decision about whether the ethtool autoneg bit should affect whether the PCS uses inband depending on whether the PCS is effectively the media facing entity, or whether there is a PHY attached - and if there is a PHY attached, ask the PHY whether it will be using in-band or not.

This also would give a way to ensure that all PCS adopt the same behaviour.

Does that sound a reasonable approach?

Strangely, I already have some patches along those lines in my net-queue branch. See:

net: phylink: add helpers for decoding mode
net: use phylink_mode_*() helpers
net: phylink: split PCS in-band from inband mode

It's nowhere near finished though, it was just an idea back in 2021 when the problem of getting rid of differing PCS behaviours was on my mind.

Quoted 18 lines · Russell King (Oracle)

So, what I would want to do is to move the decision about whether the PCS should enable in-band into the phylink core code instead of these random decisions being made in each PCS.

At that point, we can then make the decision about whether the ethtool autoneg bit should affect whether the PCS uses inband depending on whether the PCS is effectively the media facing entity, or whether there is a PHY attached - and if there is a PHY attached, ask the PHY whether it will be using in-band or not.

This also would give a way to ensure that all PCS adopt the same behaviour.

Does that sound a reasonable approach?

Strangely, I already have some patches along those lines in my net-queue branch. See:

Quoted 7 lines

net: phylink: add helpers for decoding mode
net: use phylink_mode_*() helpers
net: phylink: split PCS in-band from inband mode

It's nowhere near finished though, it was just an idea back in 2021 when the problem of getting rid of differing PCS behaviours was on my mind.

Having looked at those patches
(http://git.armlinux.org.uk/cgit/linux-arm.git/commit/?h=net-queue&id=a632167d226cf95d92cd887b2f1678e1539833b1)
and seen the way in which they are incomplete, could you sketch here how do you see an actual pcs_validate() implementation making use of the new "mode" argument?

I'd expect there to be some logic which changes the "mode", if the PCS validation with the existing one fails... or?

Quoted 71 lines · Daniel Golle

Quoted 56 lines · Russell King (Oracle)

Quoted 31 lines · Daniel Golle

Quoted 25 lines · Russell King (Oracle)

Quoted 14 lines · Daniel Golle

After conversion to phylink_pcs the 1000Base-X and 2500Base-X modes would work only after ethtool -s eth1 autoneg off.
As ethtool autoneg and the ETHTOOL_LINK_MODE_Autoneg_BIT is supposed to control auto-negotiation on the external interface it doesn't make much sense to use it to control on-board SGMII auto-negotiation between MAC and PHY.
Set correct values to really only enable SGMII auto-negotiation when actually operating in SGMII mode. For 1000Base-X and 2500Base-X mode, enable remote-fault detection only if in-band-status is enabled. This fixes using 1000Base-X and 2500Base-X SFPs on the BananaPi R3 board and also makes it possible to use interface-mode-switching PHYs operating in either SGMII mode for 10M/100M/1000M or in 2500Base-X for 2500M mode on other boards.

Quoted 2 lines

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

NAK.

There are PHYs out there which operate in SGMII mode but do not exchange the SGMII 16-bit configuration word. The code implemented here by me was explicitly to allow such a configuration to work, which is defined as:

Quoted 4 lines

	SGMII *without* mode == inband

An example of this is the Broadcom 84881 PHY which can be found on SFP modules.

Quoted 7 lines

I also have multiple such 1000Base-T SFP modules here (finisar, AJYA), and this change doesn't touch the codepaths relevant for those. They are operating in SGMII mode, they have always been working fine.

What I'm trying to fix here is 1000Base-X and 2500Base-X mode which has been broken by introducing ETHTOOL_LINK_MODE_Autoneg_BIT as the deciding factor for in-band AN here.

... which is correct.

Quoted 3 lines

Can you explain why ETHTOOL_LINK_MODE_Autoneg_BIT was used there in first place? Is my understanding of this bit controlling autoneg on the *external* interface rather than on the *system-side* interface wrong?

Think about what 1000BASE-X is for. It's not really for internal links, it's intended by IEEE 802.3 to be the 1G *media* side protocol for 1000BASE-SX, 1000BASE-LX, 1000BASE-CX etc links.

Therefore, when being used in that case, one may wish to disable autoneg over the fibre link. Hence, turning off autoneg via ethtool *should* turn off autoneg over the fibre link. So, using ETHTOOL_LINK_MODE_Autoneg_BIT to gate 802.3z autonegotiation the correct thing to do.

Quoted 5 lines

If we have a PHY using 1000BASE-X, then it is at odds with the primary purpose of this protocol, especially with it comes to AN. This is why phylink used to refuse to accept PHYs when using 802.3z mode, but Marek wanted this to work, so relaxed the checks preventing such a setup working.

Sadly 2500Base-X is very commonly used to connect 2500Base-T-capable PHYs or SFP modules. I also got an ATS branded 1000M/100M/10M copper SFP module which uses 1000Base-X as system-side interface, independently of the speed of the link partner on the TP interface.
All of them do not work with inband-AN enabled and a link only comes up after ethtool -s eth1 autoneg off in the current implementation, while previously they were working just fine.

Quoted 8 lines

I understand that there isn't really a good solution for 1000Base-X as thanks to you I now understand that SerDes autoneg just transparently ends up being autoneg on a fiber link.

2500Base-X, however, is hardly used for fiber links, but rather mostly for 2500Base-T PHYs and SFP module as well as xPON SFPs. Maybe we could at least have in-band AN disabled by default for those to get them working without requiring the user to carry out ethtool settings?

We could _possibly_ make 2500base-X ignore the autoneg bit, but in order to do that I would want to make other changes, because this is getting absolutely stupid to have these decisions being made in each and every PCS - and have each PCS author implementing different decision making in their PCS driver.

The problem that gives is it makes phylink maintenance in hard, because it becomes impossible to predict what the effect of any change is.

It also means that plugging the same SFP module into different hardware will give different results (maybe it works, maybe it doesn't.)

So, what I would want to do is to move the decision about whether the PCS should enable in-band into the phylink core code instead of these random decisions being made in each PCS.

At that point, we can then make the decision about whether the ethtool autoneg bit should affect whether the PCS uses inband depending on whether the PCS is effectively the media facing entity, or whether there is a PHY attached - and if there is a PHY attached, ask the PHY whether it will be using in-band or not.

This also would give a way to ensure that all PCS adopt the same behaviour.

Does that sound a reasonable approach?

Strangely, I already have some patches along those lines in my net-queue branch. See:

net: phylink: add helpers for decoding mode
net: use phylink_mode_*() helpers
net: phylink: split PCS in-band from inband mode

It's nowhere near finished though, it was just an idea back in 2021 when the problem of getting rid of differing PCS behaviours was on my mind.

In general it sound reasonable. We may need more SFP qurik bits to indicate presence of a PHY on SFP modules which do not expose that PHY via i2c-mdio or otherwise let the host know about it's presence.

For my TP-LINK TL-SM410U 2500Base-T SFP this unfortunately seems to be the case, and I assume it's actually like that for most 2500Base-T as well as xPON SFPs... (xPON SFPs are usually managed via high-level protocols, even Web-UI is common there. They don't tell you much about them via I2C, I suppose to get them to work in as many SFP host devices as possible without any software changes).

FYI:
TP-LINK TL-SM410U 2500Base-T module:

sfp EE: 00000000: 03 04 07 00 00 00 00 00 00 40 00 01 1f 00 00 00 .........@...... sfp EE: 00000010: 00 00 00 00 54 50 2d 4c 49 4e 4b 20 20 20 20 20 ....TP-LINK
sfp EE: 00000020: 20 20 20 20 00 30 b5 c2 54 4c 2d 53 4d 34 31 30 .0..TL-SM410
sfp EE: 00000030: 55 20 20 20 20 20 20 20 32 2e 30 20 00 00 00 1b U 2.0 ....
sfp EE: 00000040: 00 08 01 00 80 ff ff ff 40 3d f0 0d c0 ff ff ff ........@=...... sfp EE: 00000050: c8 39 7a 08 c0 ff ff ff 50 3d f0 0d c0 ff ff ff .9z.....P=......
sfp sfp2: module TP-LINK TL-SM410U rev 2.0 sn 12260M4001782 dc 220622

And this is the ATS SFP-GE-T 10/100/1000M copper module doing rate-adaptation to 1000Base-X:

sfp sfp1: EEPROM extended structure checksum failure: 0xb0 != 0xaf

sfp EE: 00000000: 03 04 07 00 00 00 02 12 00 01 01 01 0c 00 03 00 ................ sfp EE: 00000010: 00 00 00 00 4f 45 4d 20 20 20 20 20 20 20 20 20 ....OEM
sfp EE: 00000020: 20 20 20 20 00 00 90 65 53 46 50 2d 47 45 2d 54 ...eSFP-GE-T
sfp EE: 00000030: 20 20 20 20 00 00 00 00 43 20 20 20 00 00 00 f0 ....C ....
sfp EE: 00000040: 00 12 00 00 32 31 30 37 31 30 41 30 30 31 32 37 ....210710A00127 sfp EE: 00000050: 33 39 00 00 32 31 30 37 31 30 20 20 60 00 01 af 39..210710 `...
sfp sfp1: module OEM SFP-GE-T rev C sn dc

That one already needs quirks to even work at all as TX-FAULT is not reported properly by the module, see

https://github.com/dangowrt/linux/commit/2c694bd494583f08858fabca97cfdc79de8ba089

Quoted 3 lines · Daniel Golle

In general it sound reasonable. We may need more SFP qurik bits to indicate presence of a PHY on SFP modules which do not expose that PHY via i2c-mdio or otherwise let the host know about it's presence.

That's a whole load of fun - some modules where the PHY is inaccessible will be using 1000base-X, others will be using SGMII. So yes, its likely that we may need quirks for these. We don't have quirks yet because you're the first to suggest there's a problem.

Quoted 6 lines

For my TP-LINK TL-SM410U 2500Base-T SFP this unfortunately seems to be the case, and I assume it's actually like that for most 2500Base-T as well as xPON SFPs... (xPON SFPs are usually managed via high-level protocols, even Web-UI is common there. They don't tell you much about them via I2C, I suppose to get them to work in as many SFP host devices as possible without any software changes).

xPON SFPs are a whole different ball game. For some, they auto-detect while booting and try 2500base-X or 1000base-X to see which will sync and if not they try the other. Other xPON SFPs run their host interface at a speed determined by the configuration set by the remote end. Other xPON SFPs may do something entirely different.

In many cases, their EEPROM is a full of errors - such as advertising that they're 1.2 or 1.3 Gbd while operating in 2500base-X mode.

They do weird stuff with their status pins as well, for example, some use the RX_LOS pin as a uart - which is a problem if it's e.g. tied from the cage to a switch that uses the pin to gate the link-up indication without any software control of that!

With xPON SFPs, it's just a total minefield, which lots of SFF MSA violations all over the place. They're essentially a law to themselves (this is exactly why we have the quirks infrastructure.)

Quoted 10 lines

FYI:
TP-LINK TL-SM410U 2500Base-T module:

sfp EE: 00000000: 03 04 07 00 00 00 00 00 00 40 00 01 1f 00 00 00 .........@...... sfp EE: 00000010: 00 00 00 00 54 50 2d 4c 49 4e 4b 20 20 20 20 20 ....TP-LINK
sfp EE: 00000020: 20 20 20 20 00 30 b5 c2 54 4c 2d 53 4d 34 31 30 .0..TL-SM410
sfp EE: 00000030: 55 20 20 20 20 20 20 20 32 2e 30 20 00 00 00 1b U 2.0 ....
sfp EE: 00000040: 00 08 01 00 80 ff ff ff 40 3d f0 0d c0 ff ff ff ........@=...... sfp EE: 00000050: c8 39 7a 08 c0 ff ff ff 50 3d f0 0d c0 ff ff ff .9z.....P=......
sfp sfp2: module TP-LINK TL-SM410U rev 2.0 sn 12260M4001782 dc 220622

I'm guessing this is a module with a checksum problem...

Quoted 5 lines

And this is the ATS SFP-GE-T 10/100/1000M copper module doing rate-adaptation to 1000Base-X:

sfp sfp1: EEPROM extended structure checksum failure: 0xb0 != 0xaf

Given how close that is, it looks like they used the wrong algorithm.

Quoted 7 lines

sfp EE: 00000000: 03 04 07 00 00 00 02 12 00 01 01 01 0c 00 03 00 ................ sfp EE: 00000010: 00 00 00 00 4f 45 4d 20 20 20 20 20 20 20 20 20 ....OEM
sfp EE: 00000020: 20 20 20 20 00 00 90 65 53 46 50 2d 47 45 2d 54 ...eSFP-GE-T
sfp EE: 00000030: 20 20 20 20 00 00 00 00 43 20 20 20 00 00 00 f0 ....C ....
sfp EE: 00000040: 00 12 00 00 32 31 30 37 31 30 41 30 30 31 32 37 ....210710A00127 sfp EE: 00000050: 33 39 00 00 32 31 30 37 31 30 20 20 60 00 01 af 39..210710 `...
sfp sfp1: module OEM SFP-GE-T rev C sn dc

Welcome to the wonderful world of horribly broken SFPs.

Do we know what form of rate adaption this module needs on the transmit path? Does it require the host to pace itself to the media speed (which I suspect will be unreadable if the PHY isn't accessible) or will it send pause frames?

It would be nice to add these to my database - please send me the output of ethtool -m $iface raw on > foo.bin for each module.

Quoted 4 lines

That one already needs quirks to even work at all as TX-FAULT is not reported properly by the module, see

https://github.com/dangowrt/linux/commit/2c694bd494583f08858fabca97cfdc79de8ba089

I'm guessing that's not on a kernel version that has:

73472c830eae net: sfp: add support for HALNy GPON SFP 5029be761161 net: sfp: move Huawei MA5671A fixup 275416754e9a net: sfp: move Alcatel Lucent 3FE46541AA fixup 23571c7b9643 net: sfp: move quirk handling into sfp.c 8475c4b70b04 net: sfp: re-implement soft state polling setup

which reworks how we deal with the soft/hard state signals.

I think the problem space is growing, and I fear that if we try to address all these issues in one go, we're going to end up with way too much to deal with in one go (which means poor reviews etc.)

Can we try to concentrate on fixing one problem at a time, rather than throwing a whole load of problems into the mix?

Thanks.

Signature · 2 lines

RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

Quoted 11 lines · Russell King (Oracle)

Quoted 10 lines

FYI:
TP-LINK TL-SM410U 2500Base-T module:

sfp EE: 00000000: 03 04 07 00 00 00 00 00 00 40 00 01 1f 00 00 00 .........@...... sfp EE: 00000010: 00 00 00 00 54 50 2d 4c 49 4e 4b 20 20 20 20 20 ....TP-LINK
sfp EE: 00000020: 20 20 20 20 00 30 b5 c2 54 4c 2d 53 4d 34 31 30 .0..TL-SM410
sfp EE: 00000030: 55 20 20 20 20 20 20 20 32 2e 30 20 00 00 00 1b U 2.0 ....
sfp EE: 00000040: 00 08 01 00 80 ff ff ff 40 3d f0 0d c0 ff ff ff ........@=...... sfp EE: 00000050: c8 39 7a 08 c0 ff ff ff 50 3d f0 0d c0 ff ff ff .9z.....P=......
sfp sfp2: module TP-LINK TL-SM410U rev 2.0 sn 12260M4001782 dc 220622

Quoted 1 line

I'm guessing this is a module with a checksum problem...

No, the checksum of the TL-SM410U is correct. I have patched the kernel to always dump the EEPROM, so I can share it with you.

Quoted 2 lines

It would be nice to add these to my database - please send me the output of ethtool -m $iface raw on > foo.bin for each module.

Gesendet: Mittwoch, 08. März 2023 um 16:24 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 2 lines

Quoted 2 lines

It would be nice to add these to my database - please send me the output of ethtool -m $iface raw on > foo.bin for each module.

so if you can do that for me, then I can see whether it's likely that the patches that are already in mainline will do anything to solve the workaround you've had to add for the hw signals.

Gesendet: Samstag, 11. März 2023 um 13:05 Uhr
Von: "Frank Wunderlich" <[email protected]>

Quoted 8 lines

Gesendet: Mittwoch, 08. März 2023 um 16:24 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 2 lines

Quoted 2 lines

It would be nice to add these to my database - please send me the output of ethtool -m $iface raw on > foo.bin for each module.

so if you can do that for me, then I can see whether it's likely that the patches that are already in mainline will do anything to solve the workaround you've had to add for the hw signals.

i got the 2.5G copper sfps, and tried them...they work well with the v12 (including this patch), but not in v13...

i dumped the eeprom like you mention for your database:

$ hexdump -C 2g5_sfp.bin 
00000000  03 04 07 00 01 00 00 00  00 02 00 05 19 00 00 00  |................|
00000010  1e 14 00 00 4f 45 4d 20  20 20 20 20 20 20 20 20  |....OEM         |
00000020  20 20 20 20 00 00 00 00  53 46 50 2d 32 2e 35 47  |    ....SFP-2.5G|
00000030  2d 54 20 20 20 20 20 20  31 2e 30 20 03 52 00 19  |-T      1.0 .R..|
00000040  00 1a 00 00 53 4b 32 33  30 31 31 31 30 30 30 38  |....SK2301110008|
00000050  20 20 20 20 32 33 30 31  31 30 20 20 68 f0 01 e8  |    230110  h...|
00000060  00 00 11 37 4f 7a dc ff  3d c0 6e 74 9b 7c 06 ca  |...7Oz..=.nt.|..|
00000070  e1 d0 f9 00 00 00 00 00  00 00 00 00 08 f0 fc 64  |...............d|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000100  5f 00 ce 00 5a 00 d3 00  8c a0 75 30 88 b8 79 18  |_...Z.....u0..y.|
00000110  1d 4c 01 f4 19 64 03 e8  4d f0 06 30 3d e8 06 f2  |.L...d..M..0=...|
00000120  2b d4 00 c7 27 10 00 df  00 00 00 00 00 00 00 00  |+...'...........|
00000130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000140  00 00 00 00 3f 80 00 00  00 00 00 00 01 00 00 00  |....?...........|
00000150  01 00 00 00 01 00 00 00  01 00 00 00 00 00 00 f1  |................|
00000160  29 1a 82 41 0b b8 13 88  0f a0 ff ff ff ff 80 ff  |)..A............|
00000170  00 00 ff ff 00 00 ff ff  04 ff ff ff ff ff ff 00  |................|
00000180  43 4e 53 38 54 55 54 41  41 43 33 30 2d 31 34 31  |CNS8TUTAAC30-141|
00000190  30 2d 30 34 56 30 34 20  49 fb 46 00 00 00 00 29  |0-04V04 I.F....)|
000001a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 aa aa  |................|
000001c0  47 4c 43 2d 54 20 20 20  20 20 20 20 20 20 20 20  |GLC-T           |
000001d0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 97  |               .|
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001f0  00 00 00 00 00 00 00 00  00 40 00 40 00 00 00 00  |.........@.@....|

how can we add a quirk to support this?

Quoted 43 lines

Gesendet: Samstag, 11. März 2023 um 13:05 Uhr
Von: "Frank Wunderlich" <[email protected]>

Quoted 8 lines

Gesendet: Mittwoch, 08. März 2023 um 16:24 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 2 lines

Quoted 2 lines

It would be nice to add these to my database - please send me the output of ethtool -m $iface raw on > foo.bin for each module.

so if you can do that for me, then I can see whether it's likely that the patches that are already in mainline will do anything to solve the workaround you've had to add for the hw signals.

i got the 2.5G copper sfps, and tried them...they work well with the v12 (including this patch), but not in v13...

i dumped the eeprom like you mention for your database:

$ hexdump -C 2g5_sfp.bin 
00000000  03 04 07 00 01 00 00 00  00 02 00 05 19 00 00 00  |................|
00000010  1e 14 00 00 4f 45 4d 20  20 20 20 20 20 20 20 20  |....OEM         |
00000020  20 20 20 20 00 00 00 00  53 46 50 2d 32 2e 35 47  |    ....SFP-2.5G|
00000030  2d 54 20 20 20 20 20 20  31 2e 30 20 03 52 00 19  |-T      1.0 .R..|
00000040  00 1a 00 00 53 4b 32 33  30 31 31 31 30 30 30 38  |....SK2301110008|
00000050  20 20 20 20 32 33 30 31  31 30 20 20 68 f0 01 e8  |    230110  h...|
00000060  00 00 11 37 4f 7a dc ff  3d c0 6e 74 9b 7c 06 ca  |...7Oz..=.nt.|..|
00000070  e1 d0 f9 00 00 00 00 00  00 00 00 00 08 f0 fc 64  |...............d|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000100  5f 00 ce 00 5a 00 d3 00  8c a0 75 30 88 b8 79 18  |_...Z.....u0..y.|
00000110  1d 4c 01 f4 19 64 03 e8  4d f0 06 30 3d e8 06 f2  |.L...d..M..0=...|
00000120  2b d4 00 c7 27 10 00 df  00 00 00 00 00 00 00 00  |+...'...........|
00000130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000140  00 00 00 00 3f 80 00 00  00 00 00 00 01 00 00 00  |....?...........|
00000150  01 00 00 00 01 00 00 00  01 00 00 00 00 00 00 f1  |................|
00000160  29 1a 82 41 0b b8 13 88  0f a0 ff ff ff ff 80 ff  |)..A............|
00000170  00 00 ff ff 00 00 ff ff  04 ff ff ff ff ff ff 00  |................|
00000180  43 4e 53 38 54 55 54 41  41 43 33 30 2d 31 34 31  |CNS8TUTAAC30-141|
00000190  30 2d 30 34 56 30 34 20  49 fb 46 00 00 00 00 29  |0-04V04 I.F....)|
000001a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 aa aa  |................|
000001c0  47 4c 43 2d 54 20 20 20  20 20 20 20 20 20 20 20  |GLC-T           |
000001d0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 97  |               .|
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001f0  00 00 00 00 00 00 00 00  00 40 00 40 00 00 00 00  |.........@.@....|

Quoted 1 line

how can we add a quirk to support this?

tried to add the quirk like this onto v13

drivers/net/phy/sfp.c +2 -0

@@ -403,6 +403,8 @@ static const struct sfp_quirk sfp_quirks[] = {
        SFP_QUIRK_F("OEM", "RTSFP-10G", sfp_fixup_rollball_cc),
        SFP_QUIRK_F("Turris", "RTSFP-10", sfp_fixup_rollball),
        SFP_QUIRK_F("Turris", "RTSFP-10G", sfp_fixup_rollball),
+
+       SFP_QUIRK_M("OEM", "SFP-2.5G-T", sfp_quirk_2500basex),
 };
 

but still no link...

how can i verify the quirk was applied? see only this in dmesg:

[    2.192274] sfp sfp-1: module OEM              SFP-2.5G-T       rev 1.0  sn SK2301110008     dc 230110  

also tried to force speed (module should support 100/1000/2500), but it seems i can set speed option only on gmac (eth1) and not on the sfp (phy).

i guess between mac and sfp speed is always 2500base-X and after the phy (if there is any) the link speed is maybe different.

regards Frank

Quoted 66 lines · Frank Wunderlich

Quoted 43 lines

Gesendet: Samstag, 11. März 2023 um 13:05 Uhr
Von: "Frank Wunderlich" <[email protected]>

Quoted 8 lines

Gesendet: Mittwoch, 08. März 2023 um 16:24 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 2 lines

Quoted 2 lines

It would be nice to add these to my database - please send me the output of ethtool -m $iface raw on > foo.bin for each module.

so if you can do that for me, then I can see whether it's likely that the patches that are already in mainline will do anything to solve the workaround you've had to add for the hw signals.

i got the 2.5G copper sfps, and tried them...they work well with the v12 (including this patch), but not in v13...

i dumped the eeprom like you mention for your database:

$ hexdump -C 2g5_sfp.bin 
00000000  03 04 07 00 01 00 00 00  00 02 00 05 19 00 00 00  |................|
00000010  1e 14 00 00 4f 45 4d 20  20 20 20 20 20 20 20 20  |....OEM         |
00000020  20 20 20 20 00 00 00 00  53 46 50 2d 32 2e 35 47  |    ....SFP-2.5G|
00000030  2d 54 20 20 20 20 20 20  31 2e 30 20 03 52 00 19  |-T      1.0 .R..|
00000040  00 1a 00 00 53 4b 32 33  30 31 31 31 30 30 30 38  |....SK2301110008|
00000050  20 20 20 20 32 33 30 31  31 30 20 20 68 f0 01 e8  |    230110  h...|
00000060  00 00 11 37 4f 7a dc ff  3d c0 6e 74 9b 7c 06 ca  |...7Oz..=.nt.|..|
00000070  e1 d0 f9 00 00 00 00 00  00 00 00 00 08 f0 fc 64  |...............d|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000100  5f 00 ce 00 5a 00 d3 00  8c a0 75 30 88 b8 79 18  |_...Z.....u0..y.|
00000110  1d 4c 01 f4 19 64 03 e8  4d f0 06 30 3d e8 06 f2  |.L...d..M..0=...|
00000120  2b d4 00 c7 27 10 00 df  00 00 00 00 00 00 00 00  |+...'...........|
00000130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000140  00 00 00 00 3f 80 00 00  00 00 00 00 01 00 00 00  |....?...........|
00000150  01 00 00 00 01 00 00 00  01 00 00 00 00 00 00 f1  |................|
00000160  29 1a 82 41 0b b8 13 88  0f a0 ff ff ff ff 80 ff  |)..A............|
00000170  00 00 ff ff 00 00 ff ff  04 ff ff ff ff ff ff 00  |................|
00000180  43 4e 53 38 54 55 54 41  41 43 33 30 2d 31 34 31  |CNS8TUTAAC30-141|
00000190  30 2d 30 34 56 30 34 20  49 fb 46 00 00 00 00 29  |0-04V04 I.F....)|
000001a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 aa aa  |................|
000001c0  47 4c 43 2d 54 20 20 20  20 20 20 20 20 20 20 20  |GLC-T           |
000001d0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 97  |               .|
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001f0  00 00 00 00 00 00 00 00  00 40 00 40 00 00 00 00  |.........@.@....|

Quoted 1 line

how can we add a quirk to support this?

tried to add the quirk like this onto v13

drivers/net/phy/sfp.c +2 -0

@@ -403,6 +403,8 @@ static const struct sfp_quirk sfp_quirks[] = {
        SFP_QUIRK_F("OEM", "RTSFP-10G", sfp_fixup_rollball_cc),
        SFP_QUIRK_F("Turris", "RTSFP-10", sfp_fixup_rollball),
        SFP_QUIRK_F("Turris", "RTSFP-10G", sfp_fixup_rollball),
+
+       SFP_QUIRK_M("OEM", "SFP-2.5G-T", sfp_quirk_2500basex),
 };
 

but still no link...

how can i verify the quirk was applied? see only this in dmesg:

[    2.192274] sfp sfp-1: module OEM              SFP-2.5G-T       rev 1.0  sn SK2301110008     dc 230110  

also tried to force speed (module should support 100/1000/2500), but it seems i can set speed option only on gmac (eth1) and not on the sfp (phy).

Quoted 2 lines

i guess between mac and sfp speed is always 2500base-X and after the phy (if there is any) the link speed is maybe different.

As discussed in the previous iteration of the series where I suggested to add work-arounds disabling in-band AN for 1000Base-X and 2500Base-X having those will also affect fiber transceivers which transparently pass-through the electrical SerDes signal into light. Russell explained it very well and I now agree that a good solution would be to add a new SFP quirk indicating that a SFP module got a "hidden" PHY which doesn't like in-band autonegotiation.

Quoted 67 lines · Daniel Golle

Quoted 66 lines · Frank Wunderlich

Quoted 43 lines

Gesendet: Samstag, 11. März 2023 um 13:05 Uhr
Von: "Frank Wunderlich" <[email protected]>

Quoted 8 lines

Gesendet: Mittwoch, 08. März 2023 um 16:24 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 2 lines

Quoted 2 lines

It would be nice to add these to my database - please send me the output of ethtool -m $iface raw on > foo.bin for each module.

so if you can do that for me, then I can see whether it's likely that the patches that are already in mainline will do anything to solve the workaround you've had to add for the hw signals.

i got the 2.5G copper sfps, and tried them...they work well with the v12 (including this patch), but not in v13...

i dumped the eeprom like you mention for your database:

$ hexdump -C 2g5_sfp.bin 
00000000  03 04 07 00 01 00 00 00  00 02 00 05 19 00 00 00  |................|
00000010  1e 14 00 00 4f 45 4d 20  20 20 20 20 20 20 20 20  |....OEM         |
00000020  20 20 20 20 00 00 00 00  53 46 50 2d 32 2e 35 47  |    ....SFP-2.5G|
00000030  2d 54 20 20 20 20 20 20  31 2e 30 20 03 52 00 19  |-T      1.0 .R..|
00000040  00 1a 00 00 53 4b 32 33  30 31 31 31 30 30 30 38  |....SK2301110008|
00000050  20 20 20 20 32 33 30 31  31 30 20 20 68 f0 01 e8  |    230110  h...|
00000060  00 00 11 37 4f 7a dc ff  3d c0 6e 74 9b 7c 06 ca  |...7Oz..=.nt.|..|
00000070  e1 d0 f9 00 00 00 00 00  00 00 00 00 08 f0 fc 64  |...............d|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000100  5f 00 ce 00 5a 00 d3 00  8c a0 75 30 88 b8 79 18  |_...Z.....u0..y.|
00000110  1d 4c 01 f4 19 64 03 e8  4d f0 06 30 3d e8 06 f2  |.L...d..M..0=...|
00000120  2b d4 00 c7 27 10 00 df  00 00 00 00 00 00 00 00  |+...'...........|
00000130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000140  00 00 00 00 3f 80 00 00  00 00 00 00 01 00 00 00  |....?...........|
00000150  01 00 00 00 01 00 00 00  01 00 00 00 00 00 00 f1  |................|
00000160  29 1a 82 41 0b b8 13 88  0f a0 ff ff ff ff 80 ff  |)..A............|
00000170  00 00 ff ff 00 00 ff ff  04 ff ff ff ff ff ff 00  |................|
00000180  43 4e 53 38 54 55 54 41  41 43 33 30 2d 31 34 31  |CNS8TUTAAC30-141|
00000190  30 2d 30 34 56 30 34 20  49 fb 46 00 00 00 00 29  |0-04V04 I.F....)|
000001a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 aa aa  |................|
000001c0  47 4c 43 2d 54 20 20 20  20 20 20 20 20 20 20 20  |GLC-T           |
000001d0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 97  |               .|
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001f0  00 00 00 00 00 00 00 00  00 40 00 40 00 00 00 00  |.........@.@....|

Quoted 1 line

how can we add a quirk to support this?

tried to add the quirk like this onto v13

drivers/net/phy/sfp.c +2 -0

@@ -403,6 +403,8 @@ static const struct sfp_quirk sfp_quirks[] = {
        SFP_QUIRK_F("OEM", "RTSFP-10G", sfp_fixup_rollball_cc),
        SFP_QUIRK_F("Turris", "RTSFP-10", sfp_fixup_rollball),
        SFP_QUIRK_F("Turris", "RTSFP-10G", sfp_fixup_rollball),
+
+       SFP_QUIRK_M("OEM", "SFP-2.5G-T", sfp_quirk_2500basex),
 };
 

but still no link...

how can i verify the quirk was applied? see only this in dmesg:

[    2.192274] sfp sfp-1: module OEM              SFP-2.5G-T       rev 1.0  sn SK2301110008     dc 230110  

also tried to force speed (module should support 100/1000/2500), but it seems i can set speed option only on gmac (eth1) and not on the sfp (phy).

Quoted 10 lines

Quoted 2 lines

i guess between mac and sfp speed is always 2500base-X and after the phy (if there is any) the link speed is maybe different.

As discussed in the previous iteration of the series where I suggested to add work-arounds disabling in-band AN for 1000Base-X and 2500Base-X having those will also affect fiber transceivers which transparently pass-through the electrical SerDes signal into light. Russell explained it very well and I now agree that a good solution would be to add a new SFP quirk indicating that a SFP module got a "hidden" PHY which doesn't like in-band autonegotiation.

I do not believe that fibre SFPs have hidden PHYs that disrupt the autonegotiation. What makes you think that they do?

Do you have
autonegotiation working with some fibre SFP modules but not other fibre modules?

Hi

Quoted 8 lines

Gesendet: Mittwoch, 08. März 2023 um 16:24 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 2 lines

Quoted 2 lines

It would be nice to add these to my database - please send me the output of ethtool -m $iface raw on > foo.bin for each module.

so if you can do that for me, then I can see whether it's likely that the patches that are already in mainline will do anything to solve the workaround you've had to add for the hw signals.

i got the 2.5G copper sfps, and tried them...they work well with the v12 (including this patch), but not in v13...

i dumped the eeprom like you mention for your database:

$ hexdump -C 2g5_sfp.bin 
00000000  03 04 07 00 01 00 00 00  00 02 00 05 19 00 00 00  |................|
00000010  1e 14 00 00 4f 45 4d 20  20 20 20 20 20 20 20 20  |....OEM         |
00000020  20 20 20 20 00 00 00 00  53 46 50 2d 32 2e 35 47  |    ....SFP-2.5G|
00000030  2d 54 20 20 20 20 20 20  31 2e 30 20 03 52 00 19  |-T      1.0 .R..|
00000040  00 1a 00 00 53 4b 32 33  30 31 31 31 30 30 30 38  |....SK2301110008|
00000050  20 20 20 20 32 33 30 31  31 30 20 20 68 f0 01 e8  |    230110  h...|
00000060  00 00 11 37 4f 7a dc ff  3d c0 6e 74 9b 7c 06 ca  |...7Oz..=.nt.|..|
00000070  e1 d0 f9 00 00 00 00 00  00 00 00 00 08 f0 fc 64  |...............d|
00000080  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000100  5f 00 ce 00 5a 00 d3 00  8c a0 75 30 88 b8 79 18  |_...Z.....u0..y.|
00000110  1d 4c 01 f4 19 64 03 e8  4d f0 06 30 3d e8 06 f2  |.L...d..M..0=...|
00000120  2b d4 00 c7 27 10 00 df  00 00 00 00 00 00 00 00  |+...'...........|
00000130  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000140  00 00 00 00 3f 80 00 00  00 00 00 00 01 00 00 00  |....?...........|
00000150  01 00 00 00 01 00 00 00  01 00 00 00 00 00 00 f1  |................|
00000160  29 1a 82 41 0b b8 13 88  0f a0 ff ff ff ff 80 ff  |)..A............|
00000170  00 00 ff ff 00 00 ff ff  04 ff ff ff ff ff ff 00  |................|
00000180  43 4e 53 38 54 55 54 41  41 43 33 30 2d 31 34 31  |CNS8TUTAAC30-141|
00000190  30 2d 30 34 56 30 34 20  49 fb 46 00 00 00 00 29  |0-04V04 I.F....)|
000001a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 aa aa  |................|
000001c0  47 4c 43 2d 54 20 20 20  20 20 20 20 20 20 20 20  |GLC-T           |
000001d0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 97  |               .|
000001e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001f0  00 00 00 00 00 00 00 00  00 40 00 40 00 00 00 00  |.........@.@....|

how can we add a quirk to support this?

some more information:

root@bpi-r3:~# ethtool eth1
Settings for eth1:
        Supported ports: [ FIBRE ]
        Supported link modes:   2500baseX/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  2500baseX/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 2500Mb/s
        Duplex: Full
        Auto-negotiation: on
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes
root@bpi-r3:~# ethtool -m eth1
        Identifier                                : 0x03 (SFP)
        Extended identifier                       : 0x04 (GBIC/SFP defined by 2-wire interface ID)
        Connector                                 : 0x07 (LC)
        Transceiver codes                         : 0x00 0x01 0x00 0x00 0x00 0x00 0x02 0x00 0x00
        Transceiver type                          : SONET: OC-48, short reach
        Encoding                                  : 0x05 (SONET Scrambled)
        BR, Nominal                               : 2500MBd
        Rate identifier                           : 0x00 (unspecified)
        Length (SMF,km)                           : 0km
        Length (SMF)                              : 0m
        Length (50um)                             : 300m
        Length (62.5um)                           : 200m
        Length (Copper)                           : 0m
        Length (OM3)                              : 0m
        Laser wavelength                          : 850nm
        Vendor name                               : OEM
        Vendor OUI                                : 00:00:00
        Vendor PN                                 : SFP-2.5G-T
        Vendor rev                                : 1.0
        Option values                             : 0x00 0x1a
        Option                                    : RX_LOS implemented
        Option                                    : TX_FAULT implemented
        Option                                    : TX_DISABLE implemented
        BR margin, max                            : 0%
        BR margin, min                            : 0%
        Vendor SN                                 : SK2301110008
        Date code                                 : 230110
        Optical diagnostics support               : Yes
        Laser bias current                        : 6.000 mA
        Laser output power                        : 0.5000 mW / -3.01 dBm
        Receiver signal average optical power     : 0.4000 mW / -3.98 dBm
        Module temperature                        : 28.21 degrees C / 82.78 degrees F
        Module voltage                            : 3.3067 V
        Alarm/warning flags implemented           : Yes
        Laser bias current high alarm             : Off
        Laser bias current low alarm              : Off
        Laser bias current high warning           : Off
        Laser bias current low warning            : Off
        Laser output power high alarm             : Off
        Laser output power low alarm              : Off
        Laser output power high warning           : Off
        Laser output power low warning            : Off
        Module temperature high alarm             : Off
        Module temperature low alarm              : Off
        Module temperature high warning           : Off
        Module temperature low warning            : Off
        Module voltage high alarm                 : Off
        Module voltage low alarm                  : Off
        Module voltage high warning               : Off
        Module voltage low warning                : Off
        Laser rx power high alarm                 : Off
        Laser rx power low alarm                  : Off
        Laser rx power high warning               : Off
        Laser rx power low warning                : Off
        Laser bias current high alarm threshold   : 15.000 mA
        Laser bias current low alarm threshold    : 1.000 mA
        Laser bias current high warning threshold : 13.000 mA
        Laser bias current low warning threshold  : 2.000 mA
        Laser output power high alarm threshold   : 1.9952 mW / 3.00 dBm
        Laser output power low alarm threshold    : 0.1584 mW / -8.00 dBm
        Laser output power high warning threshold : 1.5848 mW / 2.00 dBm
        Laser output power low warning threshold  : 0.1778 mW / -7.50 dBm
        Module temperature high alarm threshold   : 95.00 degrees C / 203.00 degrees F
        Module temperature low alarm threshold    : -50.00 degrees C / -58.00 degrees F
        Module temperature high warning threshold : 90.00 degrees C / 194.00 degrees F
        Module temperature low warning threshold  : -45.00 degrees C / -49.00 degrees F
        Module voltage high alarm threshold       : 3.6000 V
        Module voltage low alarm threshold        : 3.0000 V
        Module voltage high warning threshold     : 3.5000 V
        Module voltage low warning threshold      : 3.1000 V
        Laser rx power high alarm threshold       : 1.1220 mW / 0.50 dBm
        Laser rx power low alarm threshold        : 0.0199 mW / -17.01 dBm
        Laser rx power high warning threshold     : 1.0000 mW / 0.00 dBm
        Laser rx power low warning threshold      : 0.0223 mW / -16.52 dBm

i guess this sfp have a phy as it can operate in 100/1000/2500 mode like described on the module.

Quoted 4 lines

i got the 2.5G copper sfps, and tried them...they work well with the v12 (including this patch), but not in v13...

how can we add a quirk to support this?

Why does it need a quirk?

Quoted 37 lines

some more information:

root@bpi-r3:~# ethtool eth1
Settings for eth1:
        Supported ports: [ FIBRE ]
        Supported link modes:   2500baseX/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  2500baseX/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 2500Mb/s
        Duplex: Full
        Auto-negotiation: on
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes
root@bpi-r3:~# ethtool -m eth1
        Identifier                                : 0x03 (SFP)
        Extended identifier                       : 0x04 (GBIC/SFP defined by 2-wire interface ID)
        Connector                                 : 0x07 (LC)
        Transceiver codes                         : 0x00 0x01 0x00 0x00 0x00 0x00 0x02 0x00 0x00
        Transceiver type                          : SONET: OC-48, short reach
        Encoding                                  : 0x05 (SONET Scrambled)
        BR, Nominal                               : 2500MBd

        Vendor name                               : OEM
        Vendor OUI                                : 00:00:00
        Vendor PN                                 : SFP-2.5G-T
        Vendor rev                                : 1.0
        Option values                             : 0x00 0x1a

Quoted 2 lines

i guess this sfp have a phy as it can operate in 100/1000/2500 mode like described on the module.

It would help to know the kernel messages.

Am 11. März 2023 21:00:20 MEZ schrieb "Russell King (Oracle)" <[email protected]>:

Quoted 3 lines · Frank Wunderlich

Quoted 2 lines

i got the 2.5G copper sfps, and tried them...they work well with the v12 (including this patch), but not in v13...

Quoted 3 lines

Quoted 1 line

how can we add a quirk to support this?

Why does it need a quirk?

To disable the inband-mode for this 2.5g copper sfp. But have not found a way to set a flag which i can grab in phylink.

Gesendet: Samstag, 11. März 2023 um 21:30 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 9 lines · Frank Wunderlich

Am 11. März 2023 21:00:20 MEZ schrieb "Russell King (Oracle)" <[email protected]>:

Quoted 7 lines · Frank Wunderlich

Quoted 4 lines

i got the 2.5G copper sfps, and tried them...they work well with the v12 (including this patch), but not in v13...

how can we add a quirk to support this?

Why does it need a quirk?

Quoted 3 lines

To disable the inband-mode for this 2.5g copper sfp. But have not found a way to set a flag which i can grab in phylink.

We could make sfp_parse_support() set Autoneg, Pause, and Asym_Pause in "modes" at the top of that function, and then use the SFP modes quirk to clear the Autoneg bit for this SFP. Would that work for you?

Just to make it clear...the issue with the copper-sfps is no regression of this series it exists before. i only had none of them to test for until this weekend....my 1g fibre-sfp were working fine with the inband-flag.

this patch tries to fix it in mtk driver, this was rejected (as far as i understand it should be handled in phylink core instead of pcs driver). and no more in the v13, so we try to fix it another way.

whatever i do in phylink_parse_mode the link is always inband...i tried to add a new state to have the configuration not FIXED or PHY or INBAND

drivers/net/phy/phylink.c

(quoted diff) +1 -0

@@ -151,6 +151,7 @@ static const char *phylink_an_mode_str(unsigned int mode)
                [MLO_AN_PHY] = "phy",
                [MLO_AN_FIXED] = "fixed",
                [MLO_AN_INBAND] = "inband",
+               [MLO_AN_INBAND_DISABLED] = "inband disabled",
        };
 

include/linux/phylink.h

(quoted diff) +1 -0

@@ -20,6 +20,7 @@ enum {
        MLO_AN_PHY = 0, /* Conventional PHY */
        MLO_AN_FIXED,   /* Fixed-link mode */
        MLO_AN_INBAND,  /* In-band protocol */
+       MLO_AN_INBAND_DISABLED
 

is my start the right way?

Quoted 13 lines

Gesendet: Samstag, 11. März 2023 um 21:30 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 9 lines · Frank Wunderlich

Am 11. März 2023 21:00:20 MEZ schrieb "Russell King (Oracle)" <[email protected]>:

Quoted 7 lines · Frank Wunderlich

Quoted 4 lines

i got the 2.5G copper sfps, and tried them...they work well with the v12 (including this patch), but not in v13...

how can we add a quirk to support this?

Why does it need a quirk?

Quoted 7 lines

Quoted 3 lines

To disable the inband-mode for this 2.5g copper sfp. But have not found a way to set a flag which i can grab in phylink.

We could make sfp_parse_support() set Autoneg, Pause, and Asym_Pause in "modes" at the top of that function, and then use the SFP modes quirk to clear the Autoneg bit for this SFP. Would that work for you?

i already tried this (without moving the autoneg/pause to sfp_parse_support):

static void sfp_quirk_disable_autoneg(const struct sfp_eeprom_id *id,
unsigned long *modes, unsigned long *interfaces)

{
	linkmode_clear_bit(ETHTOOL_LINK_MODE_Autoneg_BIT, modes);
}

quirk was executed, but no change (no link on 2g5 sfp).

i guess you mean moving code handling the dt-property for inband-mode in phylink_parse_mode (phylink.c) to the sfp-function (drivers/net/phy/sfp-bus.c)

Quoted 23 lines · Frank Wunderlich

Quoted 13 lines

Gesendet: Samstag, 11. März 2023 um 21:30 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 9 lines · Frank Wunderlich

Am 11. März 2023 21:00:20 MEZ schrieb "Russell King (Oracle)" <[email protected]>:

Quoted 7 lines · Frank Wunderlich

Quoted 4 lines

i got the 2.5G copper sfps, and tried them...they work well with the v12 (including this patch), but not in v13...

how can we add a quirk to support this?

Why does it need a quirk?

Quoted 7 lines

Quoted 3 lines

To disable the inband-mode for this 2.5g copper sfp. But have not found a way to set a flag which i can grab in phylink.

We could make sfp_parse_support() set Autoneg, Pause, and Asym_Pause in "modes" at the top of that function, and then use the SFP modes quirk to clear the Autoneg bit for this SFP. Would that work for you?

i already tried this (without moving the autoneg/pause to sfp_parse_support):

Quoted 8 lines

static void sfp_quirk_disable_autoneg(const struct sfp_eeprom_id *id,
unsigned long *modes, unsigned long *interfaces)

{
	linkmode_clear_bit(ETHTOOL_LINK_MODE_Autoneg_BIT, modes);
}

quirk was executed, but no change (no link on 2g5 sfp).

It won't have any effect on its own - because sfp_parse_support() does this:

        if (bus->sfp_quirk && bus->sfp_quirk->modes)
                bus->sfp_quirk->modes(id, modes, interfaces);

        linkmode_or(support, support, modes);

        phylink_set(support, Autoneg);
        phylink_set(support, Pause);
        phylink_set(support, Asym_Pause);

Which means clearing Autoneg in "modes" via the modes SFP quirk will have *absolutely* *no* *effect* what so ever.

The fact that you replied having *not* followed my suggestion and then itimiating that it doesn't work is very frustrating.

Quoted 1 line

i guess you mean moving code handling the dt-property for inband-mode in phylink_parse_mode (phylink.c) to the sfp-function (drivers/net/phy/sfp-bus.c)

No.

[rest of email cut because I can't be bothered to read it after this]

Please try what I suggested. You might find that it works.

Gesendet: Montag, 13. März 2023 um 11:59 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Since describing what I wanted you to test didn't work, here's a patch instead, based upon the quirk that you provided (which is what I'd have written anyway). Add a "#define DEBUG" to the top of drivers/net/phy/phylink.c in addition to applying this patch, and please test the resulting kernel, sending me the resulting kernel messages, and also reporting whether this works or not.

Quoted 9 lines

Gesendet: Montag, 13. März 2023 um 11:59 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Since describing what I wanted you to test didn't work, here's a patch instead, based upon the quirk that you provided (which is what I'd have written anyway). Add a "#define DEBUG" to the top of drivers/net/phy/phylink.c in addition to applying this patch, and please test the resulting kernel, sending me the resulting kernel messages, and also reporting whether this works or not.

Hi

thx for the patch...sorry for misunderstanding. i thought the sfp quirk only sets a flag and i need to change something in phylink.c to do the same as done on userspace, so i tried to simulate the userspace call there only for testing.

here relevant parts of debug

[    1.990637] sfp sfp-1: module OEM              SFP-2.5G-T       rev 1.0  sn SK2301110008     dc 230110  
[    2.000147] mtk_soc_eth 15100000.ethernet eth1: optical SFP: interfaces=[mac=2-4,21-22, sfp=]

Gesendet: Montag, 13. März 2023 um 23:57 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 17 lines · Frank Wunderlich

Quoted 9 lines

Gesendet: Montag, 13. März 2023 um 11:59 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Since describing what I wanted you to test didn't work, here's a patch instead, based upon the quirk that you provided (which is what I'd have written anyway). Add a "#define DEBUG" to the top of drivers/net/phy/phylink.c in addition to applying this patch, and please test the resulting kernel, sending me the resulting kernel messages, and also reporting whether this works or not.

Hi

thx for the patch...sorry for misunderstanding. i thought the sfp quirk only sets a flag and i need to change something in phylink.c to do the same as done on userspace, so i tried to simulate the userspace call there only for testing.

here relevant parts of debug

Quoted 2 lines

[    1.990637] sfp sfp-1: module OEM              SFP-2.5G-T       rev 1.0  sn SK2301110008     dc 230110  
[    2.000147] mtk_soc_eth 15100000.ethernet eth1: optical SFP: interfaces=[mac=2-4,21-22, sfp=]

First thing... why are the SFP interfaces here empty? They should be listing at least 22 for this SFP. Looking at the full log, you have omitted:

[    2.008678] mtk_soc_eth 15100000.ethernet eth1: unsupported SFP module: no common interface modes

which seems to suggest that we need more than what I provided - and is a big pointer to why it isn't working... and I guess has been there all along.

This means that the interface configuration never gets updated, so its pointless trying to add quirks etc. Error messages are rather a key point.

So everything after this is just not relevant. Let's fix that. Here's an updated patch which sets an interface mode for this SFP and sets a link mode for it (although we use 2500baseX rather than baseT here just to test this). I'm guessing it also does rate adaption, which we will have to work out later.

Hi,

at least the error-message is gone, and interface gets up when i call ethtoo to switch off autoneg.

root@bpi-r3:~# dmesg | grep -i 'sfp\|eth1'
[    1.991838] sfp sfp-1: module OEM              SFP-2.5G-T       rev 1.0  sn SK2301110008     dc 230110  
[    2.001352] mtk_soc_eth 15100000.ethernet eth1: optical SFP: interfaces=[mac=2-4,21-22, sfp=22]
[    2.010059] mtk_soc_eth 15100000.ethernet eth1: optical SFP: chosen 2500base-x interface
[    2.018145] mtk_soc_eth 15100000.ethernet eth1: requesting link mode inband/2500base-x with support 00,00000000,00000000,0000e400
[   34.385814] mtk_soc_eth 15100000.ethernet eth1: configuring for inband/2500base-x link mode
[   34.394259] mtk_soc_eth 15100000.ethernet eth1: major config 2500base-x
[   34.400860] mtk_soc_eth 15100000.ethernet eth1: phylink_mac_config: mode=inband/2500base-x/Unknown/Unknown/none adv=00,00000000,00000000,0000e400 pause=04 link=0 an=1

root@bpi-r3:~#

root@bpi-r3:~# ethtool -s eth1 autoneg off
root@bpi-r3:~# [  131.031902] mtk_soc_eth 15100000.ethernet eth1: Link is Up - 2.5Gbps/Full - flow control off
[  131.040366] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready

full log here:
https://pastebin.com/yDC7PuM2

i see that an is still 1..maybe because of the fixed value here?

https://elixir.bootlin.com/linux/v6.3-rc1/source/drivers/net/phy/phylink.c#L3038

and yes, module seems to do rate adaption (it is labeled with 100M/1G/2.5G), i tried it on a 1G-Port and link came up (with workaround patch from daniel), traffic "works" but in tx-direction with massive retransmitts (i guess because pause-frames are ignored - pause was 00).

Gesendet: Dienstag, 14. März 2023 um 10:12 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 3 lines · Frank Wunderlich

Hi,

at least the error-message is gone, and interface gets up when i call ethtoo to switch off autoneg.

Quoted 1 line

[   34.400860] mtk_soc_eth 15100000.ethernet eth1: phylink_mac_config: mode=inband/2500base-x/Unknown/Unknown/none adv=00,00000000,00000000,0000e400 pause=04 link=0 an=1

Looking good - apart from that pesky an=1 (which isn't used by the PCS driver, and I've been thinking of killing it off anyway.) Until such time that happens, we really ought to set that correctly, which means an extra bit is needed in phylink_sfp_set_config(). However, this should not affect anything.

Quoted 8 lines

root@bpi-r3:~#

root@bpi-r3:~# ethtool -s eth1 autoneg off
root@bpi-r3:~# [  131.031902] mtk_soc_eth 15100000.ethernet eth1: Link is Up - 2.5Gbps/Full - flow control off
[  131.040366] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready

full log here:
https://pastebin.com/yDC7PuM2

Quoted 3 lines

i see that an is still 1..maybe because of the fixed value here?

https://elixir.bootlin.com/linux/v6.3-rc1/source/drivers/net/phy/phylink.c#L3038

Not sure what that line has to do with it - this is what the above points to:

        phylink_sfp_set_config(pl, MLO_AN_INBAND, pl->sfp_support, &config);

Anyway, the important thing is the Autoneg bit in the advertising mask is now zero, which is what we want. That should set the PCS to disable negotiation when in 2500baseX mode, the same as ethtool -s eth1 autoneg off.

So I think the question becomes - what was the state that ethtool was reporting before asking ethtool to set autoneg off, and why does that make a difference.

Quoted 2 lines

and yes, module seems to do rate adaption (it is labeled with 100M/1G/2.5G), i tried it on a 1G-Port and link came up (with workaround patch from daniel), traffic "works" but in tx-direction with massive retransmitts (i guess because pause-frames are ignored - pause was 00).

We'll see about addressing that later once we've got the module working at 2.5G. However, thanks for the information.

The patch below should result in ethtool reporting 2500baseT rather than 2500baseX, and that an=1 should now be an=0. Please try it, and dump the ethtool eth1 before asking for autoneg to be manually disabled, and also report the kernel messages.

Hi

Quoted 6 lines

Gesendet: Dienstag, 14. März 2023 um 10:12 Uhr
Von: "Russell King (Oracle)" <[email protected]>

Quoted 3 lines · Frank Wunderlich

Hi,

at least the error-message is gone, and interface gets up when i call ethtoo to switch off autoneg.

...

Quoted 18 lines

Quoted 1 line

[   34.400860] mtk_soc_eth 15100000.ethernet eth1: phylink_mac_config: mode=inband/2500base-x/Unknown/Unknown/none adv=00,00000000,00000000,0000e400 pause=04 link=0 an=1

Looking good - apart from that pesky an=1 (which isn't used by the PCS driver, and I've been thinking of killing it off anyway.) Until such time that happens, we really ought to set that correctly, which means an extra bit is needed in phylink_sfp_set_config(). However, this should not affect anything.

Quoted 10 lines

root@bpi-r3:~#

root@bpi-r3:~# ethtool -s eth1 autoneg off
root@bpi-r3:~# [  131.031902] mtk_soc_eth 15100000.ethernet eth1: Link is Up - 2.5Gbps/Full - flow control off
[  131.040366] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready

full log here:
https://pastebin.com/yDC7PuM2

i see that an is still 1..maybe because of the fixed value here?

Quoted 6 lines

Quoted 1 line

https://elixir.bootlin.com/linux/v6.3-rc1/source/drivers/net/phy/phylink.c#L3038

Not sure what that line has to do with it - this is what the above points to:

        phylink_sfp_set_config(pl, MLO_AN_INBAND, pl->sfp_support, &config);

MLO_AN_INBAND => may cause the an=1 and mode=inband if previously (?) disabled :)

Quoted 4 lines

The patch below should result in ethtool reporting 2500baseT rather than 2500baseX, and that an=1 should now be an=0. Please try it, and dump the ethtool eth1 before asking for autoneg to be manually disabled, and also report the kernel messages.

i see no Patch below ;)

Gesendet: Dienstag, 14. März 2023 um 11:10 Uhr
Von: "Russell King (Oracle)" <[email protected]>

For 802.3z modes, MLO_AN_INBAND with Autoneg clear in the advertising mode disables in-band negotiation. This is exactly how "ethtool -s ethX autoneg off" works.

Quoted 4 lines

Quoted 4 lines

The patch below should result in ethtool reporting 2500baseT rather than 2500baseX, and that an=1 should now be an=0. Please try it, and dump the ethtool eth1 before asking for autoneg to be manually disabled, and also report the kernel messages.

Hi

very good...do not need the manual autoneg with the last Patch :)

Quoted 6 lines

Gesendet: Dienstag, 14. März 2023 um 11:10 Uhr
Von: "Russell King (Oracle)" <[email protected]>

For 802.3z modes, MLO_AN_INBAND with Autoneg clear in the advertising mode disables in-band negotiation. This is exactly how "ethtool -s ethX autoneg off" works.

ok, this seems now correctly set.

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

The patch below should result in ethtool reporting 2500baseT rather than 2500baseX, and that an=1 should now be an=0. Please try it, and dump the ethtool eth1 before asking for autoneg to be manually disabled, and also report the kernel messages.

root@bpi-r3:~# ip link set eth1 up
[   91.624075] mtk_soc_eth 15100000.ethernet eth1: configuring for inband/2500base-x link mode
[   91.632485] mtk_soc_eth 15100000.ethernet eth1: major config 2500base-x
[   91.639094] mtk_soc_eth 15100000.ethernet eth1: phylink_mac_config: mode=inband/2500base-x/Unknown/Unknown/none adv=00,00000000,00008000,00006400 pause=00 link=0 an=0
root@bpi-r3:~# [   95.808983] mtk_soc_eth 15100000.ethernet eth1: Link is Up - Unknown/Unknown - flow control off
[   95.817706] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready

root@bpi-r3:~# ethtool eth1
Settings for eth1:
        Supported ports: [ FIBRE ]
        Supported link modes:   2500baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  2500baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Auto-negotiation: off
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes 

root@bpi-r3:~# dmesg | grep -i 'sfp\|eth1'
[    0.000000] Linux version 6.3.0-rc1-bpi-r3-sfp13 (frank@frank-G5) (aarch64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #2 SMP Tue Mar 143
[    1.658048] sfp sfp-1: Host maximum power 1.0W
[    1.663128] sfp sfp-2: Host maximum power 1.0W
[    1.812401] mtk_soc_eth 15100000.ethernet eth1: mediatek frame engine at 0xffffffc00af80000, irq 123
[    2.001796] sfp sfp-1: module OEM              SFP-2.5G-T       rev 1.0  sn SK2301110008     dc 230110
[    2.011307] mtk_soc_eth 15100000.ethernet eth1: optical SFP: interfaces=[mac=2-4,21-22, sfp=22]
[    2.020000] mtk_soc_eth 15100000.ethernet eth1: optical SFP: chosen 2500base-x interface
[    2.028080] mtk_soc_eth 15100000.ethernet eth1: requesting link mode inband/2500base-x with support 00,00000000,00008000,00006400
[   91.624075] mtk_soc_eth 15100000.ethernet eth1: configuring for inband/2500base-x link mode
[   91.632485] mtk_soc_eth 15100000.ethernet eth1: major config 2500base-x
[   91.639094] mtk_soc_eth 15100000.ethernet eth1: phylink_mac_config: mode=inband/2500base-x/Unknown/Unknown/none adv=00,00000000,00008000,00006400 pause=00 link=0 an=0
[   95.808983] mtk_soc_eth 15100000.ethernet eth1: Link is Up - Unknown/Unknown - flow control off
[   95.817706] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready

so you can see the link-up comes directly after the interface up

does the ethtool-output look like expected? i see speed/duplex is set as supported/advertised but not active

Supported link modes: 2500baseT/Full
Advertised link modes: 2500baseT/Full
vs.

        Speed: Unknown!
        Duplex: Unknown! (255) 

imho ETHTOOL_LINK_MODE_2500baseT_Full_BIT sets only the supported which intersected with the advertised from the other side maximum should be taken as actual mode...so this part seems not correctly working at the moment.

the "Supported ports: [ FIBRE ]" is also misleading for copper sfp, but imho all SFP are shown like this.

full log if needed:
https://pastebin.com/6yWe4Kyi

next step:
is it possible to have pause for rate adaption (handling rx pause frames correctly)?

Gesendet: Dienstag, 07. März 2023 um 16:54 Uhr
Von: "Daniel Golle" <[email protected]>
Link partner advertised link modes are not reported by the SerDes hardware if not operating in SGMII mode. Hence we cannot use phylink_mii_c22_pcs_decode_state() in this case.
Implement reporting link and an_complete only and use speed according to the interface mode.

Link partner advertised link modes are not reported by the SerDes hardware if not operating in SGMII mode. Hence we cannot use phylink_mii_c22_pcs_decode_state() in this case.
Implement reporting link and an_complete only and use speed according to the interface mode.

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

Quoted 6 lines

Link partner advertised link modes are not reported by the SerDes hardware if not operating in SGMII mode. Hence we cannot use phylink_mii_c22_pcs_decode_state() in this case.
Implement reporting link and an_complete only and use speed according to the interface mode.

Quoted 2 lines

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

This has been proven to work by Frank Wunderlich last October, so by making this change, you will be regressing his setup.

What are you testing against? Have you proven independently that the link partner is indeed sending a valid advertisement for the LPA register to be filled in?

Am 8. März 2023 12:38:31 MEZ schrieb "Russell King (Oracle)" <[email protected]>:

Quoted 10 lines · Daniel Golle

Quoted 8 lines

Link partner advertised link modes are not reported by the SerDes hardware if not operating in SGMII mode. Hence we cannot use phylink_mii_c22_pcs_decode_state() in this case.
Implement reporting link and an_complete only and use speed according to the interface mode.

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

Quoted 2 lines

This has been proven to work by Frank Wunderlich last October, so by making this change, you will be regressing his setup.

Hi

My tests were done with 1 kind of 1g fibre sfp as i only have these atm...have ordered some 2g5 rj54 ones,but don't have them yet. I'm not sure if they are working with/without sgmii (1000base-X) and if they have builtin phy.

Daniel have a lot more of different SFPs and some (especially 2g5) were not working after our pcs change.

Quoted 5 lines

What are you testing against? Have you proven independently that the link partner is indeed sending a valid advertisement for the LPA register to be filled in?

Quoted 18 lines · Frank Wunderlich

Am 8. März 2023 12:38:31 MEZ schrieb "Russell King (Oracle)" <[email protected]>:

Quoted 7 lines · Daniel Golle

Quoted 6 lines

Link partner advertised link modes are not reported by the SerDes hardware if not operating in SGMII mode. Hence we cannot use phylink_mii_c22_pcs_decode_state() in this case.
Implement reporting link and an_complete only and use speed according to the interface mode.

Quoted 5 lines

Quoted 2 lines

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

This has been proven to work by Frank Wunderlich last October, so by making this change, you will be regressing his setup.

Hi

My tests were done with 1 kind of 1g fibre sfp as i only have these atm...have ordered some 2g5 rj54 ones,but don't have them yet. I'm not sure if they are working with/without sgmii (1000base-X) and if they have builtin phy.

Quoted 1 line

Daniel have a lot more of different SFPs and some (especially 2g5) were not working after our pcs change.

Exactly. 1 GBit/s SFPs with built-in PHY and using SGMII are working fine before and after conversion to phylink_pcs. 1000Base-X and 2500Base-X PHYs and SFPs were broken after this.

The patch about (and the other one you already NACK'ed) fixes those codepaths which were simply not used in Frank's setup.

I have a ballpark of different SFPs and MediaTek boards with different PHYs here and tried all of them.

I have no way to tell if the SFPs and PHYs which stopped working after the phylink_pcs conversion are sending valid advertisement. The only other boards with SFP slots I got here are RealTek-based switches, and all I can say is that on an RTL8380 based 1G switch both, the SFP modules containing a PHY and operating in SGMII mode as well as the ones without a PHY exposed via i2c-mdio and operating in 1000Base-X mode are working fine with that switch, with both, stock firmware and OpenWrt running on it.

However, even should they not send valid advertisement, they are very common parts and they were working before and not after the change to phylink_pcs, for the reasons mentioned in the description of this patch.

Am 8. März 2023 12:38:31 MEZ schrieb "Russell King (Oracle)" <[email protected]>:

Quoted 10 lines · Daniel Golle

Quoted 8 lines

Link partner advertised link modes are not reported by the SerDes hardware if not operating in SGMII mode. Hence we cannot use phylink_mii_c22_pcs_decode_state() in this case.
Implement reporting link and an_complete only and use speed according to the interface mode.

Fixes 14a44ab0330d ("net: mtk_eth_soc: partially convert to phylink_pcs")

Signed-off-by Daniel Golle <[email protected]>

Quoted 2 lines

This has been proven to work by Frank Wunderlich last October, so by making this change, you will be regressing his setup.

Hi

My tests were done with 1 kind of 1g fibre sfp as i only have these atm...have ordered some 2g5 rj54 ones,but don't have them yet. I'm not sure if they are working with/without sgmii (1000base-X) and if they have builtin phy.

Gesendet: Dienstag, 07. März 2023 um 16:54 Uhr
Von: "Daniel Golle" <[email protected]>
The SGMII core found in several MediaTek SoCs is identical to what can also be found in MediaTek's MT7531 Ethernet switch IC.
As this has not always been clear, both drivers developed different implementations to deal with the PCS.
Recently Alexander Couzens pointed out this fact which lead to the development of this shared driver.

Add a dedicated driver, mostly by copying the code now found in the Ethernet driver. The now redundant code will be removed by a follow-up commit.