Skip to contentSkip to conversation

ethtool pause mode clarifications

DISCUSSION Russell King - ARM Linux admin 2 participants 3 messages Dec 13, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

Please can someone explain the ethtool pause mode settings? The man page isn't particularly clear, it says:

-A --pause
Changes the pause parameters of the specified Ethernet device.

autoneg on|off
Specifies whether pause autonegotiation should be enabled.

rx on|off
Specifies whether RX pause should be enabled.

tx on|off
Specifies whether TX pause should be enabled.

"autoneg" states whether pause autonegotiation should be enabled, but how is this possible, when pause autonegotiation happens as part of the rest of the autonegotiation as a matter of course, and the only control we have at the PHY is the value of the pause and asym pause bits?

The alternative interpretation is that "autoneg" controls whether the local end interprets the result of the pause negotiation or not. In other words, if:

autoneg off rx on tx on

that causes pause modes to still be advertised to the partner, but the results of pause negotiation to be overridden by forcing enablement of pause reception and transmission.

Then there's the whole issue of translating something like:

autoneg on rx on tx off

to the pause mode advertisement. phylib translates this to a local advertisement of pause=1 asym pause=1. Looking at the table in 802.3,

Local device Link partner
Pause AsymDir Pause AsymDir Result

  1     1       1     X     TX + RX - but ethtool wanted RX only
  1     1       0     1     This would give RX only

however, ethtool has no control over the link partner's advertisement, so despite requesting "autoneg on rx on tx off" this may be no different from "autoneg on rx on tx on".

However, there's another way to handle that - which is to treat autoneg as a mask for the autonegotiation result:

	if (!ethtool_pause_autoneg || (local_adv & partner_adv & PAUSE)) {
		tx_pause = 1;
		rx_pause = 1;
	} else if (local_adv & partner_adv & ASYM_PAUSE) {
		tx_pause = partner_adv & PAUSE;
		rx_pause = local_adv & PAUSE;
	} else {
		tx_pause = 0;
		rx_pause = 0;
	}

	tx_pause &= ethtool_pause_tx;
	rx_pause &= ethtool_pause_rx;

In other words, treating the ethtool rx/tx pause parameters as permissive bits for the result of negotiation or force bits when pause autoneg is disabled.

Then comes the pause advertisement. phylib uses this:

	pause_advertisement = rx;
	asym_pause_advertisement = rx ^ tx;

So, if we were to request rx=1 tx=1, we end up advertising pause but no asym pause. That gives us only two possible resolutions to pause
negotiation: TX+RX pause enabled at both ends, or TX+RX pause disabled at both ends. There is no resolution that leads to symmetric pause. If what I suggest above is adopted for pause mode resolution, then a more correct implementation would be:

	pause_advertisement = rx;
	asym_pause_advertisement = rx | tx;

So, would it be possible to clarify what these settings mean in the ethtool man page please?

Thanks.

Signature · 3 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
#1 lore raw
Quoted 14 lines · Russell King - ARM Linux admin

Hi,

Please can someone explain the ethtool pause mode settings? The man page isn't particularly clear, it says:

-A --pause
Changes the pause parameters of the specified Ethernet device.

autoneg on|off
Specifies whether pause autonegotiation should be enabled.

rx on|off
Specifies whether RX pause should be enabled.

Quoted 8 lines

tx on|off
Specifies whether TX pause should be enabled.

"autoneg" states whether pause autonegotiation should be enabled, but how is this possible, when pause autonegotiation happens as part of the rest of the autonegotiation as a matter of course, and the only control we have at the PHY is the value of the pause and asym pause bits?

Hi Russell

Yah, this is not clear. How i've interpreted it is:

autoneg on:

The driver should validate rx and tx with its capabilities, and then tell the PHY what to advertise, kick off an auto-neg, and wait for the result to program the MAC with the negotiated value.

If autoneg in general is off, return an error.

autoneg off:

Forget about the PHY, program the MAC directly, and potentially shoot yourself in the foot. But it can be useful it auto-neg in general is off, or there is no PHY.

Quoted 2 lines

So, would it be possible to clarify what these settings mean in the ethtool man page please?

I suspect the first step would be to survey current implementations and find out what is the most popular interpretation of this text. Then expand the document, and maybe list some of the alternative meanings which are currently in use?

Clearly, the more of this we can handle in phylink/phylib, the more uniform it will be. But there is also a trend at the moment for firmware to control the PHY, and it seems like a few MAC driver writers have no idea what their firmware is actually doing with the PHY for things like this.

Andrew

#2 lore raw
Quoted 28 lines · Andrew Lunn
Quoted 14 lines · Russell King - ARM Linux admin

Hi,

Please can someone explain the ethtool pause mode settings? The man page isn't particularly clear, it says:

-A --pause
Changes the pause parameters of the specified Ethernet device.

autoneg on|off
Specifies whether pause autonegotiation should be enabled.

rx on|off
Specifies whether RX pause should be enabled.

Quoted 8 lines

tx on|off
Specifies whether TX pause should be enabled.

"autoneg" states whether pause autonegotiation should be enabled, but how is this possible, when pause autonegotiation happens as part of the rest of the autonegotiation as a matter of course, and the only control we have at the PHY is the value of the pause and asym pause bits?

Hi Russell

Yah, this is not clear. How i've interpreted it is:

Quoted 7 lines

autoneg on:

The driver should validate rx and tx with its capabilities, and then tell the PHY what to advertise, kick off an auto-neg, and wait for the result to program the MAC with the negotiated value.

If autoneg in general is off, return an error.

That's similar to how I interpreted it - the ethtool uapi header describes it this way:

* struct ethtool_pauseparam - Ethernet pause (flow control) parameters
* @cmd: Command number = %ETHTOOL_GPAUSEPARAM or %ETHTOOL_SPAUSEPARAM
* @autoneg: Flag to enable autonegotiation of pause frame use
* @rx_pause: Flag to enable reception of pause frames
* @tx_pause: Flag to enable transmission of pause frames
*
* Drivers should reject a non-zero setting of @autoneg when
* autoneogotiation is disabled (or not supported) for the link. *
* If the link is autonegotiated, drivers should use
* mii_advertise_flowctrl() or similar code to set the advertised
* pause frame capabilities based on the @rx_pause and @tx_pause flags,
* even if @autoneg is zero. They should also allow the advertised
* pause frame capabilities to be controlled directly through the
* advertising field of &struct ethtool_cmd.
*
* If @autoneg is non-zero, the MAC is configured to send and/or
* receive pause frames according to the result of autonegotiation.
* Otherwise, it is configured directly based on the @rx_pause and
* @tx_pause flags.

but my point is that this is perverse and counter-intuitive.

ethtool -A ethN autoneg on rx on tx on

will advertise PAUSE but no ASYM PAUSE, which will only allow pause resolutions to:
RX+TX enabled
RX+TX disabled

depending on the link partner.

ethtool -A ethN autoneg on rx on tx off

will advertise PAUSE and ASYM PAUSE, allowing pause resolutions to:
RX+TX enabled
RX enabled TX disabled
RX+TX disabled

This seems utterly wrong to me.

Then we've got the issue that the advertisement can also be changed via ethtool -s (as explicitly stated above), which means if we store the ethtool rx/tx status separately, it will get out of sync with what is being advertised. If we try to update the rx/tx enablement status from the advertisement, then we fall into a very similar trap - if ASYM_PAUSE=1 PAUSE=1 that could be RX=1 TX=0 or RX=1 TX=1. It is completely ambiguous.

Quoted 5 lines

autoneg off:

Forget about the PHY, program the MAC directly, and potentially shoot yourself in the foot. But it can be useful it auto-neg in general is off, or there is no PHY.

Yep, that's how I interpret it - autoneg off => manual control of the MAC.

Quoted 7 lines
Quoted 2 lines

So, would it be possible to clarify what these settings mean in the ethtool man page please?

I suspect the first step would be to survey current implementations and find out what is the most popular interpretation of this text. Then expand the document, and maybe list some of the alternative meanings which are currently in use?

I've looked at a number of implementations, and the result of the vagueness of specification is that we have every driver doing something different.

Some refuse set_pauseparam if autoneg=1.

Some set the PHY advertisement based on rx_pause/tx_pause in a way that seems popular, but leads to the perverse outcome I highlight above.

Some report the MAC enablement state via get_pauseparam rx_pause / tx_pause - which means get_pauseparam followed by set_pauseparam with no changes is not a no-op.

Some require pause autoneg to match the autoneg state in ksettings, and then use tx_pause/rx_pause an "abilities mask" (which isn't clear what effect it has.)

Some disappears off into firmware, so there's no clue there.

Some have a boolean pause state, which they update if tx_pause doesn't match, else if rx_pause doesn't match, else return an error if autoneg is on (which gives really odd behaviour.) See myri10ge_set_pauseparam() for that one.

skge_set_pauseparam() is another variation on randomness. If you change autoneg, you switch between none and symmetric mode. Otherwise what you end up with depends on the rx/tx_pause values.

And then when you search the Internet, you very quickly come across lots of people who struggle to cope with all these random driver specific behaviours. The whole thing seems to be one big trainwreck of a mess - which I suspect comes down to the total lack of clear specification about how this interface is supposed to work.

Quoted 5 lines

Clearly, the more of this we can handle in phylink/phylib, the more uniform it will be. But there is also a trend at the moment for firmware to control the PHY, and it seems like a few MAC driver writers have no idea what their firmware is actually doing with the PHY for things like this.

Yes, and I have nothing good to say about that; firmware leads to brokeness or restrictive implementations that are impossible to work-around. I'm pretty sure that for one of the platforms I have here, it's downfall is going to be restrictive vendor firmware.

Signature · 3 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up