Skip to contentSkip to conversation

net/mlx5e: Expose physical received bits counters to ethtool

RFC PATCH In review Kenta Akagi 2 participants 6 messages Jan 12 – Jan 28

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover net/mlx5e: Expose physical received bits counters to ethtool

Hi,

I would like to measure the cable BER on ConnectX.

According to the documentation[1][2], there are counters that can be used for this purpose: rx_corrected_bits_phy, rx_pcs_symbol_err_phy and rx_bits_phy. However, rx_bits_phy does not show up in ethtool statistics.

This patch exposes the PPCNT phy_received_bits as rx_bits_phy.

On a ConnectX-5 with 25Gbase connection, it works as expected.

On the other hand, although I have not verified it, in an 800Gbps environment rx_bits_phy would likely overflow after about 124 days. Since I cannot judge whether this is acceptable, I am posting this as an RFC first.

[1] commit 8ce3b586faa4 ("net/mlx5: Add counter information to mlx5
driver documentation")
[2] https://docs.kernel.org/networking/device_drivers/ethernet/mellanox/mlx5/counters.html

Kenta Akagi (1):
net/mlx5e: Expose physical received bits counters to ethtool

 drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 1 +
 1 file changed, 1 insertion(+)
Signature · 1 line
2.50.1
#1 lore raw

RFC PATCH 1/1 net/mlx5e: Expose physical received bits counters to ethtool

rx_bits_phy is documented but not shown in ethtool --statistics. Make this value available via ethtool.

rx_bits_phy is needed to calculate the Bit Error Ratio from rx_pcs_symbol_err_phy or rx_corrected_bits_phy. The existing rx_bytes_phy cannot be used for this calculation as it appears to be a counter above the PHY layer in the strict sense (i.e. traffic based).

---
 drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 1 +
 1 file changed, 1 insertion(+)
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +1 -0
index a2802cfc9b98..e167355daad5 100644
@@ -1226,6 +1226,7 @@ static const struct counter_desc pport_phy_layer_cntrs_stats_desc[] = {
 	MLX5_BYTE_OFF(ppcnt_reg, \
 		      counter_set.phys_layer_statistical_cntrs.c##_high)
 static const struct counter_desc pport_phy_statistical_stats_desc[] = {
+	{ "rx_bits_phy", PPORT_PHY_STATISTICAL_OFF(phy_received_bits) },
 	{ "rx_pcs_symbol_err_phy", PPORT_PHY_STATISTICAL_OFF(phy_symbol_errors) },
 	{ "rx_corrected_bits_phy", PPORT_PHY_STATISTICAL_OFF(phy_corrected_bits) },
 };
Signature · 1 line
2.50.1
#2 lore raw
In reply to Kenta Akagi
Quoted 12 lines · Kenta Akagi

Hi,

I would like to measure the cable BER on ConnectX.

According to the documentation[1][2], there are counters that can be used for this purpose: rx_corrected_bits_phy, rx_pcs_symbol_err_phy and rx_bits_phy. However, rx_bits_phy does not show up in ethtool statistics.

This patch exposes the PPCNT phy_received_bits as rx_bits_phy.

Quoted 7 lines

On a ConnectX-5 with 25Gbase connection, it works as expected.

On the other hand, although I have not verified it, in an 800Gbps environment rx_bits_phy would likely overflow after about 124 days. Since I cannot judge whether this is acceptable, I am posting this as an RFC first.

Hi,

This is a 64-bits counter so no overflow is expected.

Quoted 11 lines

[1] commit 8ce3b586faa4 ("net/mlx5: Add counter information to mlx5
driver documentation")
[2] https://docs.kernel.org/networking/device_drivers/ethernet/mellanox/mlx5/counters.html

Kenta Akagi (1):
net/mlx5e: Expose physical received bits counters to ethtool

  drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 1 +
  1 file changed, 1 insertion(+)
#3 lore raw
Quoted 23 lines · Tariq Toukan
Quoted 12 lines · Kenta Akagi

Hi,

I would like to measure the cable BER on ConnectX.

According to the documentation[1][2], there are counters that can be used for this purpose: rx_corrected_bits_phy, rx_pcs_symbol_err_phy and rx_bits_phy. However, rx_bits_phy does not show up in ethtool statistics.

This patch exposes the PPCNT phy_received_bits as rx_bits_phy.

Quoted 7 lines

On a ConnectX-5 with 25Gbase connection, it works as expected.

On the other hand, although I have not verified it, in an 800Gbps environment rx_bits_phy would likely overflow after about 124 days. Since I cannot judge whether this is acceptable, I am posting this as an RFC first.

Quoted 4 lines

Hi,

This is a 64-bits counter so no overflow is expected.

Sorry, ignore my comment, your numbers make sense. Maybe it's ~248 days, but same idea.

Quoted 12 lines
Quoted 11 lines

[1] commit 8ce3b586faa4 ("net/mlx5: Add counter information to mlx5
     driver documentation")
[2] https://docs.kernel.org/networking/device_drivers/ethernet/ mellanox/mlx5/counters.html

Kenta Akagi (1):
   net/mlx5e: Expose physical received bits counters to ethtool

  drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 1 +

  1 file changed, 1 insertion(+)
#4 lore raw
Quoted 26 lines · Tariq Toukan
Quoted 23 lines · Tariq Toukan
Quoted 19 lines · Kenta Akagi

Hi,

I would like to measure the cable BER on ConnectX.

According to the documentation[1][2], there are counters that can be used for this purpose: rx_corrected_bits_phy, rx_pcs_symbol_err_phy and rx_bits_phy. However, rx_bits_phy does not show up in ethtool statistics.

This patch exposes the PPCNT phy_received_bits as rx_bits_phy.

On a ConnectX-5 with 25Gbase connection, it works as expected.

On the other hand, although I have not verified it, in an 800Gbps environment rx_bits_phy would likely overflow after about 124 days. Since I cannot judge whether this is acceptable, I am posting this as an RFC first.

Quoted 8 lines
Quoted 4 lines

Hi,

This is a 64-bits counter so no overflow is expected.

Sorry, ignore my comment, your numbers make sense. Maybe it's ~248 days, but same idea.

Hi, thank you for checking.

Ah, it seems I didn't realize it was unsigned, and I also forgot to include the expression. Sorry about that.
Yes - at 800 Gbps, 0xFFFFFFFFFFFFFFFF / (800 * (2^30) * 86400) = 248.55 days, so it will overflow.

In practice, is it possible to expose this as a statistic via ethtool? Or is there some other value that could be exposed for BER calculation - e.g., a register that indicates the elapsed seconds since link-up?

Thanks.

Quoted 14 lines
Quoted 12 lines
Quoted 11 lines

[1] commit 8ce3b586faa4 ("net/mlx5: Add counter information to mlx5
     driver documentation")
[2] https://docs.kernel.org/networking/device_drivers/ethernet/ mellanox/mlx5/counters.html

Kenta Akagi (1):
   net/mlx5e: Expose physical received bits counters to ethtool

  drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 1 +

  1 file changed, 1 insertion(+)
#5 lore raw
Quoted 43 lines · Kenta Akagi
Quoted 26 lines · Tariq Toukan
Quoted 23 lines · Tariq Toukan
Quoted 19 lines · Kenta Akagi

Hi,

I would like to measure the cable BER on ConnectX.

According to the documentation[1][2], there are counters that can be used for this purpose: rx_corrected_bits_phy, rx_pcs_symbol_err_phy and rx_bits_phy. However, rx_bits_phy does not show up in ethtool statistics.

This patch exposes the PPCNT phy_received_bits as rx_bits_phy.

On a ConnectX-5 with 25Gbase connection, it works as expected.

On the other hand, although I have not verified it, in an 800Gbps environment rx_bits_phy would likely overflow after about 124 days. Since I cannot judge whether this is acceptable, I am posting this as an RFC first.

Quoted 8 lines
Quoted 4 lines

Hi,

This is a 64-bits counter so no overflow is expected.

Sorry, ignore my comment, your numbers make sense. Maybe it's ~248 days, but same idea.

Hi, thank you for checking.

Ah, it seems I didn't realize it was unsigned, and I also forgot to include the expression. Sorry about that.
Yes - at 800 Gbps, 0xFFFFFFFFFFFFFFFF / (800 * (2^30) * 86400) = 248.55 days, so it will overflow.

Quoted 6 lines

In practice, is it possible to expose this as a statistic via ethtool? Or is there some other value that could be exposed for BER calculation - e.g., a register that indicates the elapsed seconds since link-up?

Thanks.

Hi Kenta,

You will find "FEC histogram" feature useful.
It splits the errors into ranges (bins), and plots a counter for each bin. There's also a bin for 0-errors, which is what you're looking for.

Example output:

# ./tools/net/ynl/pyynl/cli.py --spec Documentation/netlink/specs/ethtool.yaml --do fec-get --json '{"header":{"dev-index": 5, "flags": 4}}'
{'active': 50,
'auto': 1,
'header': {'dev-index': 5, 'dev-name': 'eth3'},
'modes': {'bits': {}, 'nomask': True, 'size': 125},
'stats': {'corr-bits': [2810],
           'corrected': [2123],
           'hist': [{'bin-high': 0, 'bin-low': 0, 'bin-val': 235826196705},
                    {'bin-high': 1, 'bin-low': 1, 'bin-val': 1436},
                    {'bin-high': 2, 'bin-low': 2, 'bin-val': 687},
                    {'bin-high': 3, 'bin-low': 3, 'bin-val': 0},
                    {'bin-high': 4, 'bin-low': 4, 'bin-val': 0},
                    {'bin-high': 5, 'bin-low': 5, 'bin-val': 0},
                    {'bin-high': 6, 'bin-low': 6, 'bin-val': 0},
                    {'bin-high': 7, 'bin-low': 7, 'bin-val': 0},
                    {'bin-high': 8, 'bin-low': 8, 'bin-val': 0},
                    {'bin-high': 9, 'bin-low': 9, 'bin-val': 0},
                    {'bin-high': 10, 'bin-low': 10, 'bin-val': 0},
                    {'bin-high': 11, 'bin-low': 11, 'bin-val': 0},
                    {'bin-high': 12, 'bin-low': 12, 'bin-val': 0},
                    {'bin-high': 13, 'bin-low': 13, 'bin-val': 0},
                    {'bin-high': 14, 'bin-low': 14, 'bin-val': 0},
                    {'bin-high': 15, 'bin-low': 15, 'bin-val': 0}],
           'uncorr': [0]}}