net/mlx5e: Expose physical received bits counters to ethtool
Subsystems (5)
cover net/mlx5e: Expose physical received bits counters to ethtool
Hi,
I would like to measure the cable BER on ConnectX.
According to the documentation[1][2], there are counters that can be used for this purpose: rx_corrected_bits_phy, rx_pcs_symbol_err_phy and rx_bits_phy. However, rx_bits_phy does not show up in ethtool statistics.
This patch exposes the PPCNT phy_received_bits as rx_bits_phy.
On a ConnectX-5 with 25Gbase connection, it works as expected.
On the other hand, although I have not verified it, in an 800Gbps environment rx_bits_phy would likely overflow after about 124 days. Since I cannot judge whether this is acceptable, I am posting this as an RFC first.
[1] commit 8ce3b586faa4 ("net/mlx5: Add counter information to mlx5
driver documentation")
[2] https://docs.kernel.org/networking/device_drivers/ethernet/mellanox/mlx5/counters.html
Kenta Akagi (1):
net/mlx5e: Expose physical received bits counters to ethtool
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 1 + 1 file changed, 1 insertion(+)
Signature · 1 line
2.50.1
RFC PATCH 1/1 net/mlx5e: Expose physical received bits counters to ethtool
rx_bits_phy is documented but not shown in ethtool --statistics. Make this value available via ethtool.
rx_bits_phy is needed to calculate the Bit Error Ratio from rx_pcs_symbol_err_phy or rx_corrected_bits_phy. The existing rx_bytes_phy cannot be used for this calculation as it appears to be a counter above the PHY layer in the strict sense (i.e. traffic based).
--- drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +1 -0
@@ -1226,6 +1226,7 @@ static const struct counter_desc pport_phy_layer_cntrs_stats_desc[] = {MLX5_BYTE_OFF (ppcnt_reg, \ counter_set.phys_layer_statistical_cntrs.c##_high)static const struct counter_desc pport_phy_statistical_stats_desc[] = { + {"rx_bits_phy" ,PPORT_PHY_STATISTICAL_OFF (phy_received_bits) }, {"rx_pcs_symbol_err_phy" ,PPORT_PHY_STATISTICAL_OFF (phy_symbol_errors) }, {"rx_corrected_bits_phy" ,PPORT_PHY_STATISTICAL_OFF (phy_corrected_bits) }, };
Signature · 1 line
2.50.1
Quoted 12 lines · Kenta Akagi
Hi,
I would like to measure the cable BER on ConnectX.
According to the documentation[1][2], there are counters that can be used for this purpose:
rx_corrected_bits_phy,rx_pcs_symbol_err_phyandrx_bits_phy. However,rx_bits_phydoes not show up in ethtool statistics.This patch exposes the PPCNT
phy_received_bitsasrx_bits_phy.
Quoted 7 lines
On a ConnectX-5 with 25Gbase connection, it works as expected.
On the other hand, although I have not verified it, in an 800Gbps environment
rx_bits_phywould likely overflow after about 124 days. Since I cannot judge whether this is acceptable, I am posting this as an RFC first.
Hi,
This is a 64-bits counter so no overflow is expected.
Quoted 11 lines
[1] commit
8ce3b586faa4("net/mlx5: Add counter information to mlx5
driver documentation")
[2] https://docs.kernel.org/networking/device_drivers/ethernet/mellanox/mlx5/counters.htmlKenta Akagi (1):
net/mlx5e: Expose physical received bits counters to ethtooldrivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 1 + 1 file changed, 1 insertion(+)
Quoted 23 lines · Tariq Toukan
Quoted 12 lines · Kenta Akagi
Hi,
I would like to measure the cable BER on ConnectX.
According to the documentation[1][2], there are counters that can be used for this purpose:
rx_corrected_bits_phy,rx_pcs_symbol_err_phyandrx_bits_phy. However,rx_bits_phydoes not show up in ethtool statistics.This patch exposes the PPCNT
phy_received_bitsasrx_bits_phy.Quoted 7 lines
On a ConnectX-5 with 25Gbase connection, it works as expected.
On the other hand, although I have not verified it, in an 800Gbps environment
rx_bits_phywould likely overflow after about 124 days. Since I cannot judge whether this is acceptable, I am posting this as an RFC first.
Quoted 4 lines
Hi,
This is a 64-bits counter so no overflow is expected.
Sorry, ignore my comment, your numbers make sense. Maybe it's ~248 days, but same idea.
Quoted 12 lines
Quoted 11 lines
[1] commit
8ce3b586faa4("net/mlx5: Add counter information to mlx5
driver documentation")
[2] https://docs.kernel.org/networking/device_drivers/ethernet/ mellanox/mlx5/counters.htmlKenta Akagi (1):
net/mlx5e: Expose physical received bits counters to ethtool
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c| 1 +1 file changed, 1 insertion(+)
Quoted 26 lines · Tariq Toukan
Quoted 23 lines · Tariq Toukan
Quoted 19 lines · Kenta Akagi
Hi,
I would like to measure the cable BER on ConnectX.
According to the documentation[1][2], there are counters that can be used for this purpose:
rx_corrected_bits_phy,rx_pcs_symbol_err_phyandrx_bits_phy. However,rx_bits_phydoes not show up in ethtool statistics.This patch exposes the PPCNT
phy_received_bitsasrx_bits_phy.On a ConnectX-5 with 25Gbase connection, it works as expected.
On the other hand, although I have not verified it, in an 800Gbps environment
rx_bits_phywould likely overflow after about 124 days. Since I cannot judge whether this is acceptable, I am posting this as an RFC first.
Quoted 8 lines
Quoted 4 lines
Hi,
This is a 64-bits counter so no overflow is expected.
Sorry, ignore my comment, your numbers make sense. Maybe it's ~248 days, but same idea.
Hi, thank you for checking.
Ah, it seems I didn't realize it was unsigned, and I also forgot to include the expression. Sorry about that.
Yes - at 800 Gbps, 0xFFFFFFFFFFFFFFFF / (800 * (2^30) * 86400) = 248.55 days, so it will overflow.
In practice, is it possible to expose this as a statistic via ethtool? Or is there some other value that could be exposed for BER calculation - e.g., a register that indicates the elapsed seconds since link-up?
Thanks.
Quoted 14 lines
Quoted 12 lines
Quoted 11 lines
[1] commit
8ce3b586faa4("net/mlx5: Add counter information to mlx5
driver documentation")
[2] https://docs.kernel.org/networking/device_drivers/ethernet/ mellanox/mlx5/counters.htmlKenta Akagi (1):
net/mlx5e: Expose physical received bits counters to ethtool
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c| 1 +1 file changed, 1 insertion(+)
Quoted 43 lines · Kenta Akagi
Quoted 26 lines · Tariq Toukan
Quoted 23 lines · Tariq Toukan
Quoted 19 lines · Kenta Akagi
Hi,
I would like to measure the cable BER on ConnectX.
According to the documentation[1][2], there are counters that can be used for this purpose:
rx_corrected_bits_phy,rx_pcs_symbol_err_phyandrx_bits_phy. However,rx_bits_phydoes not show up in ethtool statistics.This patch exposes the PPCNT
phy_received_bitsasrx_bits_phy.On a ConnectX-5 with 25Gbase connection, it works as expected.
On the other hand, although I have not verified it, in an 800Gbps environment
rx_bits_phywould likely overflow after about 124 days. Since I cannot judge whether this is acceptable, I am posting this as an RFC first.Quoted 8 lines
Quoted 4 lines
Hi,
This is a 64-bits counter so no overflow is expected.
Sorry, ignore my comment, your numbers make sense. Maybe it's ~248 days, but same idea.
Hi, thank you for checking.
Ah, it seems I didn't realize it was unsigned, and I also forgot to include the expression. Sorry about that.
Yes - at 800 Gbps,0xFFFFFFFFFFFFFFFF/ (800 * (2^30) * 86400) = 248.55 days, so it will overflow.
Quoted 6 lines
In practice, is it possible to expose this as a statistic via ethtool? Or is there some other value that could be exposed for BER calculation - e.g., a register that indicates the elapsed seconds since link-up?
Thanks.
Hi Kenta,
You will find "FEC histogram" feature useful.
It splits the errors into ranges (bins), and plots a counter for each bin. There's also a bin for 0-errors, which is what you're looking for.
Example output:
# ./tools/net/ynl/pyynl/cli.py --spec Documentation/netlink/specs/ethtool.yaml --do fec-get --json '{"header":{"dev-index": 5, "flags": 4}}'
{'active': 50,
'auto': 1,
'header': {'dev-index': 5, 'dev-name': 'eth3'},
'modes': {'bits': {}, 'nomask': True, 'size': 125},
'stats': {'corr-bits': [2810],
'corrected': [2123],
'hist': [{'bin-high': 0, 'bin-low': 0, 'bin-val': 235826196705},
{'bin-high': 1, 'bin-low': 1, 'bin-val': 1436},
{'bin-high': 2, 'bin-low': 2, 'bin-val': 687},
{'bin-high': 3, 'bin-low': 3, 'bin-val': 0},
{'bin-high': 4, 'bin-low': 4, 'bin-val': 0},
{'bin-high': 5, 'bin-low': 5, 'bin-val': 0},
{'bin-high': 6, 'bin-low': 6, 'bin-val': 0},
{'bin-high': 7, 'bin-low': 7, 'bin-val': 0},
{'bin-high': 8, 'bin-low': 8, 'bin-val': 0},
{'bin-high': 9, 'bin-low': 9, 'bin-val': 0},
{'bin-high': 10, 'bin-low': 10, 'bin-val': 0},
{'bin-high': 11, 'bin-low': 11, 'bin-val': 0},
{'bin-high': 12, 'bin-low': 12, 'bin-val': 0},
{'bin-high': 13, 'bin-low': 13, 'bin-val': 0},
{'bin-high': 14, 'bin-low': 14, 'bin-val': 0},
{'bin-high': 15, 'bin-low': 15, 'bin-val': 0}],
'uncorr': [0]}}