Skip to contentSkip to conversation

Add support for SFPs behind PHYs

PATCH Applied Russell King - ARM Linux admin 5 participants 12 messages Nov 15, 2019 – Nov 19, 2019

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v2 cover Add support for SFPs behind PHYs

This series adds partial support for SFP cages connected to PHYs, specifically optical SFPs.

We add core infrastructure to phylib for this, and arrange for minimal code in the PHY driver - currently, this is code to verify that the module is one that we can support for Marvell 10G PHYs.

v2: add yaml binding patch

 .../bindings/net/ethernet-controller.yaml          |  5 ++
 .../devicetree/bindings/net/ethernet-phy.yaml      |  5 ++
 drivers/net/phy/marvell10g.c                       | 25 +++++++-
 drivers/net/phy/phy.c                              |  7 +++
 drivers/net/phy/phy_device.c                       | 66 ++++++++++++++++++++++
 include/linux/phy.h                                | 11 ++++
 6 files changed, 118 insertions(+), 1 deletion(-)
Signature · 3 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
#1 lore raw

PATCH v2 1/3 dt-bindings: net: add ethernet controller and phy sfp property

Document the missing sfp property for ethernet controllers (which has existed for some time) which is being extended to ethernet PHYs.

---
 .../devicetree/bindings/net/ethernet-controller.yaml         | 5 +++++
 Documentation/devicetree/bindings/net/ethernet-phy.yaml      | 5 +++++
 2 files changed, 10 insertions(+)
Documentation/devicetree/bindings/net/ethernet-controller.yaml +5 -0
index 0e7c31794ae6..ac471b60ed6a 100644
@@ -121,6 +121,11 @@ title: Ethernet Controller Generic Binding
       and is useful for determining certain configuration settings
       such as flow control thresholds.
 
+  sfp:
+    $ref: /schemas/types.yaml#definitions/phandle
+    description:
+      Specifies a reference to a node representing a SFP cage.
+
   tx-fifo-depth:
     $ref: /schemas/types.yaml#definitions/uint32
     description:
Documentation/devicetree/bindings/net/ethernet-phy.yaml +5 -0
index f70f18ff821f..8927941c74bb 100644
@@ -153,6 +153,11 @@ title: Ethernet PHY Generic Binding
       Delay after the reset was deasserted in microseconds. If
       this property is missing the delay will be skipped.
 
+  sfp:
+    $ref: /schemas/types.yaml#definitions/phandle
+    description:
+      Specifies a reference to a node representing a SFP cage.
+
 required:
   - reg
 
Signature · 1 line
2.20.1
Reviewed-by on v2 1/3
#2 lore raw

Re: [PATCH net-next v2 1/3] dt-bindings: net: add ethernet controller and phy sfp property

Quoted 9 lines · Russell King

Document the missing sfp property for ethernet controllers (which has existed for some time) which is being extended to ethernet PHYs.

---
 .../devicetree/bindings/net/ethernet-controller.yaml         | 5 +++++
 Documentation/devicetree/bindings/net/ethernet-phy.yaml      | 5 +++++
 2 files changed, 10 insertions(+)
#4 lore raw

PATCH v2 2/3 net: phy: add core phylib sfp support

In reply to Russell King - ARM Linux admin

Add core phylib help for supporting SFP sockets on PHYs. This provides a mechanism to inform the SFP layer about PHY up/down events, and also unregister the SFP bus when the PHY is going away.

---
 drivers/net/phy/phy.c        |  7 ++++
 drivers/net/phy/phy_device.c | 66 ++++++++++++++++++++++++++++++++++++
 include/linux/phy.h          | 11 ++++++
 3 files changed, 84 insertions(+)
drivers/net/phy/phy.c +7 -0
index 0a314cf45408..cb9ab24ee898 100644
@@ -23,6 +23,7 @@
 #include <linux/ethtool.h>
 #include <linux/phy.h>
 #include <linux/phy_led_triggers.h>
+#include <linux/sfp.h>
 #include <linux/workqueue.h>
 #include <linux/mdio.h>
 #include <linux/io.h>
@@ -866,6 +867,9 @@ void phy_stop(struct phy_device *phydev)
 
 	mutex_lock(&phydev->lock);
 
+	if (phydev->sfp_bus)
+		sfp_upstream_stop(phydev->sfp_bus);
+
 	phydev->state = PHY_HALTED;
 
 	mutex_unlock(&phydev->lock);
@@ -900,6 +904,9 @@ void phy_start(struct phy_device *phydev)
 		goto out;
 	}
 
+	if (phydev->sfp_bus)
+		sfp_upstream_start(phydev->sfp_bus);
+
 	/* if phy was suspended, bring the physical link up again */
 	__phy_resume(phydev);
 
drivers/net/phy/phy_device.c +66 -0

90 lines not on this page · raw message

include/linux/phy.h +11 -0
index 1f9a141ccfb4..9753d9f28d73 100644
@@ -203,6 +203,8 @@ static inline const char *phy_modes(phy_interface_t interface)
 
 struct device;
 struct phylink;
+struct sfp_bus;
+struct sfp_upstream_ops;
 struct sk_buff;
 
 /*
@@ -342,6 +344,8 @@ struct phy_c45_device_ids {
  * dev_flags: Device-specific flags used by the PHY driver.
  * irq: IRQ number of the PHY's interrupt (-1 if none)
  * phy_timer: The timer for handling the state machine
+ * sfp_bus_attached: flag indicating whether the SFP bus has been attached
+ * sfp_bus: SFP bus attached to this PHY's fiber port
  * attached_dev: The attached enet driver's device instance ptr
  * adjust_link: Callback for the enet controller to respond to
  * changes in the link state.
@@ -430,6 +434,9 @@ struct phy_device {
 
 	struct mutex lock;
 
+	/* This may be modified under the rtnl lock */
+	bool sfp_bus_attached;
+	struct sfp_bus *sfp_bus;
 	struct phylink *phylink;
 	struct net_device *attached_dev;
 
@@ -1015,6 +1022,10 @@ int phy_suspend(struct phy_device *phydev);
 int phy_resume(struct phy_device *phydev);
 int __phy_resume(struct phy_device *phydev);
 int phy_loopback(struct phy_device *phydev, bool enable);
+void phy_sfp_attach(void *upstream, struct sfp_bus *bus);
+void phy_sfp_detach(void *upstream, struct sfp_bus *bus);
+int phy_sfp_probe(struct phy_device *phydev,
+	          const struct sfp_upstream_ops *ops);
 struct phy_device *phy_attach(struct net_device *dev, const char *bus_id,
 			      phy_interface_t interface);
 struct phy_device *phy_find_first(struct mii_bus *bus);
Signature · 1 line
2.20.1
maintainer Reviewed-by on v2 2/3
#5 lore raw

Re: [PATCH net-next v2 2/3] net: phy: add core phylib sfp support

Quoted 5 lines · Russell King

Add core phylib help for supporting SFP sockets on PHYs. This provides a mechanism to inform the SFP layer about PHY up/down events, and also unregister the SFP bus when the PHY is going away.

Andrew

#6 lore raw

PATCH v2 3/3 net: phy: marvell10g: add SFP+ support

In reply to Russell King - ARM Linux admin

Add support for SFP+ cages to the Marvell 10G PHY driver. This is slightly complicated by the way phylib works in that we need to use a multi-step process to attach the SFP bus, and we also need to track the phylink state machine to know when the module's transmit disable signal should change state.

With appropriate DT changes, this allows the SFP+ canges on the Macchiatobin platform to be functional.

---
 drivers/net/phy/marvell10g.c | 25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)
drivers/net/phy/marvell10g.c +24 -1
index 3b99882692e3..1bf13017d288 100644
@@ -26,6 +26,7 @@
 #include <linux/hwmon.h>
 #include <linux/marvell_phy.h>
 #include <linux/phy.h>
+#include <linux/sfp.h>
 
 #define MV_PHY_ALASKA_NBT_QUIRK_MASK	0xfffffffe
 #define MV_PHY_ALASKA_NBT_QUIRK_REV	(MARVELL_PHY_ID_88X3310 | 0xa)
@@ -206,6 +207,28 @@ static int mv3310_hwmon_probe(struct phy_device *phydev)
 }
 #endif
 
+static int mv3310_sfp_insert(void *upstream, const struct sfp_eeprom_id *id)
+{
+	struct phy_device *phydev = upstream;
+	__ETHTOOL_DECLARE_LINK_MODE_MASK(support) = { 0, };
+	phy_interface_t iface;
+
+	sfp_parse_support(phydev->sfp_bus, id, support);
+	iface = sfp_select_interface(phydev->sfp_bus, id, support);
+
+	if (iface != PHY_INTERFACE_MODE_10GKR) {
+		dev_err(&phydev->mdio.dev, "incompatible SFP module inserted\n");
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static const struct sfp_upstream_ops mv3310_sfp_ops = {
+	.attach = phy_sfp_attach,
+	.detach = phy_sfp_detach,
+	.module_insert = mv3310_sfp_insert,
+};
+
 static int mv3310_probe(struct phy_device *phydev)
 {
 	struct mv3310_priv *priv;
@@ -236,7 +259,7 @@ static int mv3310_probe(struct phy_device *phydev)
 	if (ret)
 		return ret;
 
-	return 0;
+	return phy_sfp_probe(phydev, &mv3310_sfp_ops);
 }
 
 static int mv3310_suspend(struct phy_device *phydev)
Signature · 1 line
2.20.1
maintainer
#7 lore raw

Re: [PATCH net-next v2 3/3] net: phy: marvell10g: add SFP+ support

drivers/net/phy/marvell10g.c static int mv3310_hwmon_probe(struct phy_device *phydev) · line 222

Quoted 13 lines
drivers/net/phy/marvell10g.c +13 -0
+static int mv3310_sfp_insert(void *upstream, const struct sfp_eeprom_id *id)
+{
+	struct phy_device *phydev = upstream;
+	__ETHTOOL_DECLARE_LINK_MODE_MASK(support) = { 0, };
+	phy_interface_t iface;
+
+	sfp_parse_support(phydev->sfp_bus, id, support);
+	iface = sfp_select_interface(phydev->sfp_bus, id, support);
+
+	if (iface != PHY_INTERFACE_MODE_10GKR) {
+		dev_err(&phydev->mdio.dev, "incompatible SFP module inserted\n");
+		return -EINVAL;
+	}

Hi Russell

Is it possible to put an SFP module into an SFP+ cage? sfp_select_interface() would then say 1000Base-X or 2500Base-X. The SFP+ cage has a single SERDES pair, so electrically, would it be possible to do 1000Base-X? Should mv3310_sfp_insert() be reconfiguring the PHY so the SFP side swaps to 1000Base-X?

Andrew

#8 lore raw

Re: [PATCH net-next v2 3/3] net: phy: marvell10g: add SFP+ support

Quoted 16 lines · Andrew Lunn

drivers/net/phy/marvell10g.c static int mv3310_hwmon_probe(struct phy_device *phydev) · line 222

Quoted 13 lines
drivers/net/phy/marvell10g.c +13 -0
+static int mv3310_sfp_insert(void *upstream, const struct sfp_eeprom_id *id)
+{
+	struct phy_device *phydev = upstream;
+	__ETHTOOL_DECLARE_LINK_MODE_MASK(support) = { 0, };
+	phy_interface_t iface;
+
+	sfp_parse_support(phydev->sfp_bus, id, support);
+	iface = sfp_select_interface(phydev->sfp_bus, id, support);
+
+	if (iface != PHY_INTERFACE_MODE_10GKR) {
+		dev_err(&phydev->mdio.dev, "incompatible SFP module inserted\n");
+		return -EINVAL;
+	}

Hi Russell

Quoted 5 lines

Is it possible to put an SFP module into an SFP+ cage? sfp_select_interface() would then say 1000Base-X or 2500Base-X. The SFP+ cage has a single SERDES pair, so electrically, would it be possible to do 1000Base-X? Should mv3310_sfp_insert() be reconfiguring the PHY so the SFP side swaps to 1000Base-X?

The answer is... it depends.

Some SFP+ cages have stronger pull-ups and run the I2C bus at 400kHz. SFP modules limit the pullups to 4.7k minimum and a bus speed of 100kHz.

Some SFP modules will stand the faster bus speed and the stronger pull-ups. Others will not. Others may end up with EEPROM corruption.

It is possible to reconfigure the 3310 to 1000base-X (I don't think 2500base-X is possible on the fiber port) but it requires a PHY reset. I have code to do it, but I haven't tested it - as the pullups on the board I have to test with are too strong to allow the EEPROM in SFP modules to be read.

If I get around to replacing the resistor packs, then I can test it, but I'm not going to contribute completely untested code!

So, this patch reflects what can be done with the SFP+ slots on Macchiatobin boards today.

Signature · 3 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
maintainer
#9 lore raw

Re: [PATCH net-next v2 3/3] net: phy: marvell10g: add SFP+ support

Quoted 1 line

The answer is... it depends.

Hi Russell

One issue we have had with phylink is people using the interfaces wrongly. When asking this question, i was thinking about documentation. Your answer suggests this method is not simply about the validation you are doing here, it could also be about configuration of the PHY to fit the module.

Maybe it would be good to add documentation somewhere about the range of things this call can do?

Quoted 2 lines

So, this patch reflects what can be done with the SFP+ slots on Macchiatobin boards today.

This all sounds very hardware dependent. So we are going to need some more DT properties i guess. Have you thought about this?

Maybe we need to add compatibles sff,sfp+ and sff,sff+ ? Indicate the board is capable of the higher speeds? And when sfp+/sff+ is used, maybe a boolean to indicate it is also sff/sfp compatible? sfp_select_interface() can then look at these properties and return PHY_INTERFACE_MODE_NA if the board is not capable of supporting the module?

Would it even make sense to make the PHY interface more like the MAC interface? A validate function to indicate what it is capable of? A configure function to configure its mode towards the SFP?

Andrew

#10 lore raw

Re: [PATCH net-next v2 3/3] net: phy: marvell10g: add SFP+ support

Quoted 9 lines · Andrew Lunn
Quoted 1 line

The answer is... it depends.

Hi Russell

One issue we have had with phylink is people using the interfaces wrongly. When asking this question, i was thinking about documentation. Your answer suggests this method is not simply about the validation you are doing here, it could also be about configuration of the PHY to fit the module.

Err.

Is that not what phylink_sfp_module_insert() is doing - validating that the module can be supported by the MAC and setting the MAC up for it.

The implementation for marvell10g is no different - I'm not sure why you're thinking something else is going on here.

Quoted 8 lines

Maybe it would be good to add documentation somewhere about the range of things this call can do?

Quoted 2 lines

So, this patch reflects what can be done with the SFP+ slots on Macchiatobin boards today.

This all sounds very hardware dependent. So we are going to need some more DT properties i guess. Have you thought about this?

I don't see how DT properties help.

DT properties can't stop you plugging in a SFP module into a SFP+ slot with too strong pullups and corrupting the EEPROM on the SFP module.

There's nothing that really tells you that the module is SFP or SFP+, and if we wanted to guess, we'd need to read the EEPROM... but reading the EEPROM might corrupt it - catch-22.

Quoted 3 lines

Maybe we need to add compatibles sff,sfp+ and sff,sff+ ? Indicate the board is capable of the higher speeds? And when sfp+/sff+ is used, maybe a boolean to indicate it is also sff/sfp compatible?

I've had a patch like that hanging around for a few years. I've yet to find anything that could benefit from it or use it to make some kind of decision in the code.

Quoted 7 lines

sfp_select_interface() can then look at these properties and return PHY_INTERFACE_MODE_NA if the board is not capable of supporting the module?

Would it even make sense to make the PHY interface more like the MAC interface? A validate function to indicate what it is capable of? A configure function to configure its mode towards the SFP?

I don't see how any of that helps. The problem is not whether the PHY interface mode is supported it not - on the Macchiatobin DS, the 88x3310 PHY certainly supports 10GBASE-R and 1000BASE-X via the fiber port. So it's entirely possible to configure the 3310 to drive a 1G fiber SFP.

The problem on the Macchiatobin is the I2C pull-up resistors, which either prevent a SFP module being readable or end up corrupting the SFP module EEPROM in the process of trying (and probably failing) to read it. There is nothing the kernel can do to work around that.

Signature · 3 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
maintainer Applied
#11 lore raw
In reply to Russell King - ARM Linux admin

From: Russell King - ARM Linux admin <[email protected]>
Date: Fri, 15 Nov 2019 19:53:39 +0000

Quoted 8 lines

This series adds partial support for SFP cages connected to PHYs, specifically optical SFPs.

We add core infrastructure to phylib for this, and arrange for minimal code in the PHY driver - currently, this is code to verify that the module is one that we can support for Marvell 10G PHYs.

v2: add yaml binding patch

I've applied this series to net-next.

Andrew, if you still have concerns about capability detection etc. please keep following up in this patch #3 thread.

Thanks.