Skip to contentSkip to conversation

dsa: using multi-gbps speeds on CPU port

DISCUSSION Maxime Chevallier 6 participants 13 messages May 15, 2019 – May 17, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hello everyone,

I'm working on a setup where I have a 88e6390X DSA switch connected to a CPU (an armada 8040) with 2500BaseX and RXAUI interfaces (we only use one at a time).

I'm facing a limitation with the current way to represent that link, where we use a fixed-link description in the CPU port, like this :

...

switch0: switch0@1 {
	...
	port@0 {
		reg = <0>;
		label = "cpu";
		ethernet = <&eth0>;
		phy-mode = "2500base-x";
		fixed-link {
			speed = <2500>;
			full-duplex;
		};
	};
};

...

In this scenario, the dsa core will try to create a PHY emulating the fixed-link on the DSA port side. This can't work with link speeds above 1Gbps, since we don't have any emulation for these PHYs, which would be using C45 MMDs.

We could add support to emulate these modes, but I think there were some discussions about using phylink to support these higher speed fixed-link modes, instead of using PHY emulation.

However using phylink in master DSA ports seems to be a bit tricky, since master ports don't have a dedicated net_device, and instead reference the CPU-side netdevice (if I understood correctly).

I'll be happy to help on that, but before prototyping anything, I wanted to have your thougts on this, and see if you had any plans.

Thanks,

Maxime

#1 lore raw
Quoted 5 lines · Maxime Chevallier

Hello everyone,

I'm working on a setup where I have a 88e6390X DSA switch connected to a CPU (an armada 8040) with 2500BaseX and RXAUI interfaces (we only use one at a time).

Hi Maxime

RXAUI should just work. By default, the CPU port is configured to its maximum speed, so it should be doing 10Gbps. Slowing it down to 2500BaseX is however an issue.

Quoted 24 lines

I'm facing a limitation with the current way to represent that link, where we use a fixed-link description in the CPU port, like this :

...

switch0: switch0@1 {
	...
	port@0 {
		reg = <0>;
		label = "cpu";
		ethernet = <&eth0>;
		phy-mode = "2500base-x";
		fixed-link {
			speed = <2500>;
			full-duplex;
		};
	};
};

...

In this scenario, the dsa core will try to create a PHY emulating the fixed-link on the DSA port side. This can't work with link speeds above 1Gbps, since we don't have any emulation for these PHYs, which would be using C45 MMDs.

Quoted 7 lines

We could add support to emulate these modes, but I think there were some discussions about using phylink to support these higher speed fixed-link modes, instead of using PHY emulation.

However using phylink in master DSA ports seems to be a bit tricky, since master ports don't have a dedicated net_device, and instead reference the CPU-side netdevice (if I understood correctly).

I think you are getting your terminology wrong. 'master' is eth0 in the example you gave above. CPU and DSA ports don't have netdev structures, and so any PHY used with them is not corrected to a netdev.

Quoted 2 lines

I'll be happy to help on that, but before prototyping anything, I wanted to have your thougts on this, and see if you had any plans.

There are two different issues here.

1) Is using a fixed-link on a CPU or DSA port the right way to do this?
2) Making fixed-link support > 1G.

The reason i decided to use fixed-link on CPU and DSA ports is that we already have all the code needed to configure a port, and an API to do it, the adjust_link() callback. Things have moved on since then, and we now have an additional API, .phylink_mac_config(). It might be better to directly use that. If there is a max-speed property, create a phylink_link_state structure, which has no reference to a netdev, and pass it to .phylink_mac_config().

It is just an idea, but maybe you could investigate if that would work.

On the master interface, the armada 8040, eth0, you still need something. However, if you look at phylink_parse_fixedlink(), it puts the speed etc into a phylink_link_state. It never instantiates a fixed-phy. So i think that could be expanded to support higher speeds without too much trouble. The interesting part is the IOCTL handler.

Andrew

#2 lore raw

Hi Andrew,

Quoted 4 lines · On Wed, 15 May 2019 15:27:01 +0200 Andrew Lunn <[email protected]> wrote:

I think you are getting your terminology wrong. 'master' is eth0 in the example you gave above. CPU and DSA ports don't have netdev structures, and so any PHY used with them is not corrected to a netdev.

Ah yes sorry, I'm still in the process of getting familiar with the internals of DSA :/

Quoted 16 lines
Quoted 2 lines

I'll be happy to help on that, but before prototyping anything, I wanted to have your thougts on this, and see if you had any plans.

There are two different issues here.

1) Is using a fixed-link on a CPU or DSA port the right way to do this?
2) Making fixed-link support > 1G.

The reason i decided to use fixed-link on CPU and DSA ports is that we already have all the code needed to configure a port, and an API to do it, the adjust_link() callback. Things have moved on since then, and we now have an additional API, .phylink_mac_config(). It might be better to directly use that. If there is a max-speed property, create a phylink_link_state structure, which has no reference to a netdev, and pass it to .phylink_mac_config().

Quoted 2 lines

It is just an idea, but maybe you could investigate if that would work.

Ok I see what you mean, this would allow us to get rid of the phydev built from the fixed-link, and the .adjust_link call. I'll prototype that, thanks for the hint.

Quoted 5 lines

On the master interface, the armada 8040, eth0, you still need something. However, if you look at phylink_parse_fixedlink(), it puts the speed etc into a phylink_link_state. It never instantiates a fixed-phy. So i think that could be expanded to support higher speeds without too much trouble. The interesting part is the IOCTL handler.

Yes I'm not too worried about that part, unless I missed something this shouldn't be too problematic.

Once again, thanks for your help,

Maxime

#3 lore raw
Quoted 29 lines · Maxime Chevallier

Hi Andrew,

Quoted 4 lines · On Wed, 15 May 2019 15:27:01 +0200 Andrew Lunn <[email protected]> wrote:

I think you are getting your terminology wrong. 'master' is eth0 in the example you gave above. CPU and DSA ports don't have netdev structures, and so any PHY used with them is not corrected to a netdev.

Ah yes sorry, I'm still in the process of getting familiar with the internals of DSA :/

Quoted 16 lines
Quoted 2 lines

I'll be happy to help on that, but before prototyping anything, I wanted to have your thougts on this, and see if you had any plans.

There are two different issues here.

1) Is using a fixed-link on a CPU or DSA port the right way to do this?
2) Making fixed-link support > 1G.

The reason i decided to use fixed-link on CPU and DSA ports is that we already have all the code needed to configure a port, and an API to do it, the adjust_link() callback. Things have moved on since then, and we now have an additional API, .phylink_mac_config(). It might be better to directly use that. If there is a max-speed property, create a phylink_link_state structure, which has no reference to a netdev, and pass it to .phylink_mac_config().

Quoted 6 lines
Quoted 2 lines

It is just an idea, but maybe you could investigate if that would work.

Ok I see what you mean, this would allow us to get rid of the phydev built from the fixed-link, and the .adjust_link call. I'll prototype that, thanks for the hint.

Vladimir mentioned a few weeks ago that he is considering adding support for PHYLIB and PHYLINK to run without a net_device instance, you two should probably coordinate with each other and make sure both of your requirements (which are likely the same) get addressed.

Signature · 1 line
Florian
#12 lore raw
In reply to Andrew Lunn
Quoted 5 lines · Andrew Lunn

On the master interface, the armada 8040, eth0, you still need something. However, if you look at phylink_parse_fixedlink(), it puts the speed etc into a phylink_link_state. It never instantiates a fixed-phy. So i think that could be expanded to support higher speeds without too much trouble. The interesting part is the IOCTL handler.

phylink already supports 2500 and 10G fixed-link (actually, it doesn't care too much about the speed value, just passing it through), provided phy_lookup_setting() and the MAC support the speed.

Since it doesn't bother with emulating a set of phy registers, which then would be read by phylib and translated back to a speed and duplex for the MAC to use, it's way more flexible when it comes to the old emulated-phy fixed links code.

What it doesn't do is provide an emulation of C45 PHYs for mii-tool/ mii-diag - ethtool is the way forward, and it supports ethtool for these speeds.

Signature · 3 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
#4 lore raw
In reply to Florian Fainelli
Quoted 4 lines · Florian Fainelli

Vladimir mentioned a few weeks ago that he is considering adding support for PHYLIB and PHYLINK to run without a net_device instance, you two should probably coordinate with each other and make sure both of your requirements (which are likely the same) get addressed.

I don't see how that's sane unless we just replace the "netdevice" in there with an opaque "void *" and lose the typechecking.

That then means we'd need to eradicate all the messages therein, since we can't use netdev_*() functions to print.

Then there's the patches I still have, that were rejected, and have had no progress to get SFP working on 88x3310 - I'm just not bothering to push them due to the rejection, and the lack of any ideas how to approach this problem. So we have the Macchiatobin which has now been around for quite some time with SFP+ slots that are not particularly functional with mainline kernels (but hey, I don't care, because they work for me, because I have the patches that work!)

You all know where that is, I've tried pointing it out several times...

Signature · 3 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
#5 lore raw
Quoted 21 lines · Russell King - ARM Linux admin
Quoted 4 lines · Florian Fainelli

Vladimir mentioned a few weeks ago that he is considering adding support for PHYLIB and PHYLINK to run without a net_device instance, you two should probably coordinate with each other and make sure both of your requirements (which are likely the same) get addressed.

I don't see how that's sane unless we just replace the "netdevice" in there with an opaque "void *" and lose the typechecking.

That then means we'd need to eradicate all the messages therein, since we can't use netdev_*() functions to print.

Then there's the patches I still have, that were rejected, and have had no progress to get SFP working on 88x3310 - I'm just not bothering to push them due to the rejection, and the lack of any ideas how to approach this problem. So we have the Macchiatobin which has now been around for quite some time with SFP+ slots that are not particularly functional with mainline kernels (but hey, I don't care, because they work for me, because I have the patches that work!)

Quoted 6 lines

You all know where that is, I've tried pointing it out several times...

--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up According to speedtest.net: 11.9Mbps down 500kbps up

I'm unfortunately not up to speed yet on what has been tried so far, but I do plan to start prototyping the idea soon and I'll probably find your patches then.
My basic idea is to interface a Raspberry Pi-like board to a dumb "switch evaluation board" which has only the Ethernet ports and the SPI/whatever control interface exposed. The DSA CPU/master port combo in this case would go through a Cat5 cable, which is not going to pan out very well currently because both the RPi-side PHY and the switch board-side PHY need some massaging from their respective drivers. Both PHYs are C22.
A rough idea of what I'm actually planning to do at:
https://www.spinics.net/lists/netdev/msg569087.html

-Vladimir

#6 lore raw
Quoted 7 lines

My basic idea is to interface a Raspberry Pi-like board to a dumb "switch evaluation board" which has only the Ethernet ports and the SPI/whatever control interface exposed. The DSA CPU/master port combo in this case would go through a Cat5 cable, which is not going to pan out very well currently because both the RPi-side PHY and the switch board-side PHY need some massaging from their respective drivers. Both PHYs are C22.

Hi Vladimir

There are a number of boards like this, back to back PHYs. But they all have the switch PHY strapped so they start on power on and auto-negotiate. DSA then just works.

Andrew

#7 lore raw
In reply to Florian Fainelli

Hi everyone,

Quoted 30 lines · On Wed, 15 May 2019 09:09:26 -0700 Florian Fainelli <[email protected]>…
Quoted 29 lines · Maxime Chevallier

Hi Andrew,

Quoted 4 lines · On Wed, 15 May 2019 15:27:01 +0200 Andrew Lunn <[email protected]> wrote:

I think you are getting your terminology wrong. 'master' is eth0 in the example you gave above. CPU and DSA ports don't have netdev structures, and so any PHY used with them is not corrected to a netdev.

Ah yes sorry, I'm still in the process of getting familiar with the internals of DSA :/

Quoted 16 lines
Quoted 2 lines

I'll be happy to help on that, but before prototyping anything, I wanted to have your thougts on this, and see if you had any plans.

There are two different issues here.

1) Is using a fixed-link on a CPU or DSA port the right way to do this?
2) Making fixed-link support > 1G.

The reason i decided to use fixed-link on CPU and DSA ports is that we already have all the code needed to configure a port, and an API to do it, the adjust_link() callback. Things have moved on since then, and we now have an additional API, .phylink_mac_config(). It might be better to directly use that. If there is a max-speed property, create a phylink_link_state structure, which has no reference to a netdev, and pass it to .phylink_mac_config().

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It is just an idea, but maybe you could investigate if that would work.

I've quickly prototyped and tested this solution, and besides a few tweaks that are needed on the mv88e6xxx driver side, it works fine.

I'll post an RFC with this shortly, so that you can see what it looks like.

As Russell said, there wasn't anything needed on the master interface side.

Quoted 5 lines

Vladimir mentioned a few weeks ago that he is considering adding support for PHYLIB and PHYLINK to run without a net_device instance, you two should probably coordinate with each other and make sure both of your requirements (which are likely the same) get addressed.

That would help a lot solving this issue indeed, I'll be happy to help on that, thanks for the tip !

Maxime

Signature · 3 lines
Maxime Chevallier, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com
#8 lore raw
Quoted 53 lines

Subject: Re: dsa: using multi-gbps speeds on CPU port

Hi everyone,

Quoted 30 lines · On Wed, 15 May 2019 09:09:26 -0700 Florian Fainelli <[email protected]>…
Quoted 29 lines · Maxime Chevallier

Hi Andrew,

Quoted 4 lines · On Wed, 15 May 2019 15:27:01 +0200 Andrew Lunn <[email protected]> wrote:

I think you are getting your terminology wrong. 'master' is eth0 in the example you gave above. CPU and DSA ports don't have netdev structures, and so any PHY used with them is not corrected to a netdev.

Ah yes sorry, I'm still in the process of getting familiar with the internals of DSA :/

Quoted 16 lines
Quoted 2 lines

I'll be happy to help on that, but before prototyping anything, I wanted to have your thougts on this, and see if you had any plans.

There are two different issues here.

1) Is using a fixed-link on a CPU or DSA port the right way to do this?
2) Making fixed-link support > 1G.

The reason i decided to use fixed-link on CPU and DSA ports is that we already have all the code needed to configure a port, and an API to do it, the adjust_link() callback. Things have moved on since then, and we now have an additional API, .phylink_mac_config(). It might be better to directly use that. If there is a max-speed property, create a phylink_link_state structure, which has no reference to a netdev, and pass it to .phylink_mac_config().

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It is just an idea, but maybe you could investigate if that would work.

I've quickly prototyped and tested this solution, and besides a few tweaks that are needed on the mv88e6xxx driver side, it works fine.

I'll post an RFC with this shortly, so that you can see what it looks like.

As Russell said, there wasn't anything needed on the master interface side.

Quoted 5 lines

Vladimir mentioned a few weeks ago that he is considering adding support for PHYLIB and PHYLINK to run without a net_device instance, you two should probably coordinate with each other and make sure both of your requirements (which are likely the same) get addressed.

Quoted 5 lines

That would help a lot solving this issue indeed, I'll be happy to help on that, thanks for the tip !

Maxime

Hi Maxime,

I am currently maintaining some drivers for Freescale/NXP DPAA2 Ethernet. This architecture has a management firmware that abstracts and simplifies the hardware configuration into a so called object model. DPAA2 is a little too modular and you have the concept of a network interface object (DPNI) which is completely self-contained and separate from the hardware port itself (DPMAC). You can connect DPNIs to DPMACs but also DPNIs to one another. The dpaa2-eth driver conceptually handles a DPNI object. Among other things, the management firmware presents the link state information to the DPNI object as abstract as possible (speed, duplex, up/down etc.). The firmware gathers this information from whomever the DPNI is connected to. Since the firmware can't reuse Linux PHY drivers due to incompatible licensing, we need another driver which acts as glue logic between the PHY drivers and the firmware. This is the out-of-tree dpmac driver that notifies the firmware of any external PHY events. At the end of the day, the dpaa2-eth driver gets notified of these external PHY events after the firmware itself is notified and raises an interrupt line.

To start the PHY state machine for a port, the dpmac driver must fabricate a netdevice which it does not register with the stack. One would, of course, suggest to move the PHY management directly into the dpaa2-eth driver. But the firmware's ABI is already stable and besides, it is not desirable to grant MDIO access to users of the DPNI object.

Obviously, that fake netdevice has to go before the dpmac driver sees mainline. What you guys are proposing (the phylink/netdev decoupling) would also benefit our scenario. I talked to Vladimir and we'll make sure that whatever works for us is also benefiting the DSA cpu/cascade port. Hopefully we'll have some patches early next week.

-Ioana

#9 lore raw
Quoted 59 lines · Ioana Ciornei
Quoted 53 lines

Subject: Re: dsa: using multi-gbps speeds on CPU port

Hi everyone,

Quoted 30 lines · On Wed, 15 May 2019 09:09:26 -0700 Florian Fainelli <[email protected]>…
Quoted 29 lines · Maxime Chevallier

Hi Andrew,

Quoted 4 lines · On Wed, 15 May 2019 15:27:01 +0200 Andrew Lunn <[email protected]> wrote:

I think you are getting your terminology wrong. 'master' is eth0 in the example you gave above. CPU and DSA ports don't have netdev structures, and so any PHY used with them is not corrected to a netdev.

Ah yes sorry, I'm still in the process of getting familiar with the internals of DSA :/

Quoted 16 lines
Quoted 2 lines

I'll be happy to help on that, but before prototyping anything, I wanted to have your thougts on this, and see if you had any plans.

There are two different issues here.

1) Is using a fixed-link on a CPU or DSA port the right way to do this?
2) Making fixed-link support > 1G.

The reason i decided to use fixed-link on CPU and DSA ports is that we already have all the code needed to configure a port, and an API to do it, the adjust_link() callback. Things have moved on since then, and we now have an additional API, .phylink_mac_config(). It might be better to directly use that. If there is a max-speed property, create a phylink_link_state structure, which has no reference to a netdev, and pass it to .phylink_mac_config().

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It is just an idea, but maybe you could investigate if that would work.

I've quickly prototyped and tested this solution, and besides a few tweaks that are needed on the mv88e6xxx driver side, it works fine.

I'll post an RFC with this shortly, so that you can see what it looks like.

As Russell said, there wasn't anything needed on the master interface side.

Quoted 5 lines

Vladimir mentioned a few weeks ago that he is considering adding support for PHYLIB and PHYLINK to run without a net_device instance, you two should probably coordinate with each other and make sure both of your requirements (which are likely the same) get addressed.

Quoted 5 lines

That would help a lot solving this issue indeed, I'll be happy to help on that, thanks for the tip !

Maxime

Quoted 7 lines

Hi Maxime,

I am currently maintaining some drivers for Freescale/NXP DPAA2 Ethernet. This architecture has a management firmware that abstracts and simplifies the hardware configuration into a so called object model. DPAA2 is a little too modular and you have the concept of a network interface object (DPNI) which is completely self-contained and separate from the hardware port itself (DPMAC). You can connect DPNIs to DPMACs but also DPNIs to one another. The dpaa2-eth driver conceptually handles a DPNI object. Among other things, the management firmware presents the link state information to the DPNI object as abstract as possible (speed, duplex, up/down etc.). The firmware gathers this information from whomever the DPNI is connected to. Since the firmware can't reuse Linux PHY drivers due to incompatible licensing, we need another driver which acts as glue logic between the PHY drivers and the firmware. This is the out-of-tree dpmac driver that notifies the firmware of any external PHY events. At the end of the day, the dpaa2-eth driver gets notified of these external PHY events after the firmware itself is notified and raises an interrupt line.

To start the PHY state machine for a port, the dpmac driver must fabricate a netdevice which it does not register with the stack. One would, of course, suggest to move the PHY management directly into the dpaa2-eth driver. But the firmware's ABI is already stable and besides, it is not desirable to grant MDIO access to users of the DPNI object.

Obviously, that fake netdevice has to go before the dpmac driver sees mainline. What you guys are proposing (the phylink/netdev decoupling) would also benefit our scenario. I talked to Vladimir and we'll make sure that whatever works for us is also benefiting the DSA cpu/cascade port. Hopefully we'll have some patches early next week.

For SFP, I've already removed much of the netdev bits from that layer, but I don't see any way to really get rid of it from phylink - we need access to the netdev state there to know what the carrier state is for the netdev (phylink tracks that state and manages the carrier state on behalf of the MAC driver.)

Signature · 3 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
#10 lore raw
Quoted 66 lines · Russell King - ARM Linux admin
Quoted 65 lines · Ioana Ciornei
Quoted 53 lines

Subject: Re: dsa: using multi-gbps speeds on CPU port

Hi everyone,

Quoted 30 lines · On Wed, 15 May 2019 09:09:26 -0700 Florian Fainelli <[email protected]>…
Quoted 29 lines · Maxime Chevallier

Hi Andrew,

Quoted 4 lines · On Wed, 15 May 2019 15:27:01 +0200 Andrew Lunn <[email protected]> wrote:

I think you are getting your terminology wrong. 'master' is eth0 in the example you gave above. CPU and DSA ports don't have netdev structures, and so any PHY used with them is not corrected to a netdev.

Ah yes sorry, I'm still in the process of getting familiar with the internals of DSA :/

Quoted 16 lines
Quoted 2 lines

I'll be happy to help on that, but before prototyping anything, I wanted to have your thougts on this, and see if you had any plans.

There are two different issues here.

1) Is using a fixed-link on a CPU or DSA port the right way to do this?
2) Making fixed-link support > 1G.

The reason i decided to use fixed-link on CPU and DSA ports is that we already have all the code needed to configure a port, and an API to do it, the adjust_link() callback. Things have moved on since then, and we now have an additional API, .phylink_mac_config(). It might be better to directly use that. If there is a max-speed property, create a phylink_link_state structure, which has no reference to a netdev, and pass it to .phylink_mac_config().

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It is just an idea, but maybe you could investigate if that would work.

I've quickly prototyped and tested this solution, and besides a few tweaks that are needed on the mv88e6xxx driver side, it works fine.

I'll post an RFC with this shortly, so that you can see what it looks like.

As Russell said, there wasn't anything needed on the master interface side.

Quoted 5 lines

Vladimir mentioned a few weeks ago that he is considering adding support for PHYLIB and PHYLINK to run without a net_device instance, you two should probably coordinate with each other and make sure both of your requirements (which are likely the same) get addressed.

Quoted 5 lines

That would help a lot solving this issue indeed, I'll be happy to help on that, thanks for the tip !

Maxime

Hi Maxime,

I am currently maintaining some drivers for Freescale/NXP DPAA2 Ethernet. This architecture has a management firmware that abstracts and simplifies the hardware configuration into a so called object model. DPAA2 is a little too modular and you have the concept of a network interface object (DPNI) which is completely self-contained and separate from the hardware port itself (DPMAC). You can connect DPNIs to DPMACs but also DPNIs to one another. The dpaa2-eth driver conceptually handles a DPNI object. Among other things, the management firmware presents the link state information to the DPNI object as abstract as possible (speed, duplex, up/down etc.). The firmware gathers this information from whomever the DPNI is connected to. Since the firmware can't reuse Linux PHY drivers due to incompatible licensing, we need another driver which acts as glue logic between the PHY drivers and the firmware. This is the out-of-tree dpmac driver that notifies the firmware of any external PHY events. At the end of the day, the dpaa2-eth driver gets notified of these external PHY events after the firmware itself is notified and raises an interrupt line.

To start the PHY state machine for a port, the dpmac driver must fabricate a netdevice which it does not register with the stack. One would, of course, suggest to move the PHY management directly into the dpaa2-eth driver. But the firmware's ABI is already stable and besides, it is not desirable to grant MDIO access to users of the DPNI object.

Quoted 7 lines
Quoted 1 line

Obviously, that fake netdevice has to go before the dpmac driver sees mainline. What you guys are proposing (the phylink/netdev decoupling) would also benefit our scenario. I talked to Vladimir and we'll make sure that whatever works for us is also benefiting the DSA cpu/cascade port. Hopefully we'll have some patches early next week.

For SFP, I've already removed much of the netdev bits from that layer, but I don't see any way to really get rid of it from phylink - we need access to the netdev state there to know what the carrier state is for the netdev (phylink tracks that state and manages the carrier state on behalf of the MAC driver.)

We can make that a callback that is optional in case you want to use a PHYLINK instance without a backing net_device. If you pass a valid net_device pointer, then we default to netif_carrier_ok(), else the caller of phylink_create() (which would have to be renamed, or exposed with an additional argument, say phylink_create_cb()) needs to provide it.

Signature · 1 line
Florian
#11 lore raw
Quoted 75 lines · Florian Fainelli
Quoted 66 lines · Russell King - ARM Linux admin
Quoted 65 lines · Ioana Ciornei
Quoted 53 lines

Subject: Re: dsa: using multi-gbps speeds on CPU port

Hi everyone,

Quoted 30 lines · On Wed, 15 May 2019 09:09:26 -0700 Florian Fainelli <[email protected]>…
Quoted 29 lines · Maxime Chevallier

Hi Andrew,

Quoted 4 lines · On Wed, 15 May 2019 15:27:01 +0200 Andrew Lunn <[email protected]> wrote:

I think you are getting your terminology wrong. 'master' is eth0 in the example you gave above. CPU and DSA ports don't have netdev structures, and so any PHY used with them is not corrected to a netdev.

Ah yes sorry, I'm still in the process of getting familiar with the internals of DSA :/

Quoted 16 lines
Quoted 2 lines

I'll be happy to help on that, but before prototyping anything, I wanted to have your thougts on this, and see if you had any plans.

There are two different issues here.

1) Is using a fixed-link on a CPU or DSA port the right way to do this?
2) Making fixed-link support > 1G.

The reason i decided to use fixed-link on CPU and DSA ports is that we already have all the code needed to configure a port, and an API to do it, the adjust_link() callback. Things have moved on since then, and we now have an additional API, .phylink_mac_config(). It might be better to directly use that. If there is a max-speed property, create a phylink_link_state structure, which has no reference to a netdev, and pass it to .phylink_mac_config().

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It is just an idea, but maybe you could investigate if that would work.

I've quickly prototyped and tested this solution, and besides a few tweaks that are needed on the mv88e6xxx driver side, it works fine.

I'll post an RFC with this shortly, so that you can see what it looks like.

As Russell said, there wasn't anything needed on the master interface side.

Quoted 5 lines

Vladimir mentioned a few weeks ago that he is considering adding support for PHYLIB and PHYLINK to run without a net_device instance, you two should probably coordinate with each other and make sure both of your requirements (which are likely the same) get addressed.

Quoted 5 lines

That would help a lot solving this issue indeed, I'll be happy to help on that, thanks for the tip !

Maxime

Hi Maxime,

I am currently maintaining some drivers for Freescale/NXP DPAA2 Ethernet. This architecture has a management firmware that abstracts and simplifies the hardware configuration into a so called object model. DPAA2 is a little too modular and you have the concept of a network interface object (DPNI) which is completely self-contained and separate from the hardware port itself (DPMAC). You can connect DPNIs to DPMACs but also DPNIs to one another. The dpaa2-eth driver conceptually handles a DPNI object. Among other things, the management firmware presents the link state information to the DPNI object as abstract as possible (speed, duplex, up/down etc.). The firmware gathers this information from whomever the DPNI is connected to. Since the firmware can't reuse Linux PHY drivers due to incompatible licensing, we need another driver which acts as glue logic between the PHY drivers and the firmware. This is the out-of-tree dpmac driver that notifies the firmware of any external PHY events. At the end of the day, the dpaa2-eth driver gets notified of these external PHY events after the firmware itself is notified and raises an interrupt line.

To start the PHY state machine for a port, the dpmac driver must fabricate a netdevice which it does not register with the stack. One would, of course, suggest to move the PHY management directly into the dpaa2-eth driver. But the firmware's ABI is already stable and besides, it is not desirable to grant MDIO access to users of the DPNI object.

Quoted 7 lines
Quoted 1 line

Obviously, that fake netdevice has to go before the dpmac driver sees mainline. What you guys are proposing (the phylink/netdev decoupling) would also benefit our scenario. I talked to Vladimir and we'll make sure that whatever works for us is also benefiting the DSA cpu/cascade port. Hopefully we'll have some patches early next week.

For SFP, I've already removed much of the netdev bits from that layer, but I don't see any way to really get rid of it from phylink - we need access to the netdev state there to know what the carrier state is for the netdev (phylink tracks that state and manages the carrier state on behalf of the MAC driver.)

Quoted 5 lines

We can make that a callback that is optional in case you want to use a PHYLINK instance without a backing net_device. If you pass a valid net_device pointer, then we default to netif_carrier_ok(), else the caller of phylink_create() (which would have to be renamed, or exposed with an additional argument, say phylink_create_cb()) needs to provide it.

You'll need:
- a callback to set the carrier state
- a callback to get the current carrier state

at the very least.

Then there's all the netdev printing functions that would have to be converted to something (which really ought to keep staying which Linux interface they're referring to). Given that we are called before the netdev has a name, and using the device struct isn't sufficient, I don't see an easy solution to that. We need it, especially as we have boards with multiple ethernet devices using this, so the information that phylink prints has to be meaningful. Using the parent device is _painful_ so doesn't hack it - and we have situations where we have multiple ethernet devices for one parent device.

Signature · 3 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up