Accumulate statistics over reset
Subsystems (2)
v3 cover Accumulate statistics over reset
Resets happen with or without user interaction. For example, incidents such as TX hang or a reconfiguration of parameters will result in a reset. During reset, hardware and software statistics were set to zero. This created an issue for the user where a reset happens in the background, statistics set to zero, and the user checks statistics expecting them to be populated.
Accumulate statistics over reset so the above doesn't happen.
Change is divided in two patches:
1. Accumulate HW and netdev statistics over reset
2. Accumulate ring statistics over reset
Benjamin Mikailenko (2):
ice: Accumulate HW and Netdev statistics over reset
ice: Accumulate ring statistics over reset
drivers/net/ethernet/intel/ice/ice.h | 7 + drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 3 + drivers/net/ethernet/intel/ice/ice_ethtool.c | 12 +- drivers/net/ethernet/intel/ice/ice_lib.c | 263 +++++++++++++++++- drivers/net/ethernet/intel/ice/ice_main.c | 98 +++++-- drivers/net/ethernet/intel/ice/ice_repr.c | 10 +- drivers/net/ethernet/intel/ice/ice_txrx.c | 40 ++- drivers/net/ethernet/intel/ice/ice_txrx.h | 18 +- drivers/net/ethernet/intel/ice/ice_txrx_lib.c | 2 +- drivers/net/ethernet/intel/ice/ice_xsk.c | 25 +- 10 files changed, 414 insertions(+), 64 deletions(-)
Signature · 6 lines
2.34.3 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v3 1/2 ice: Accumulate HW and Netdev statistics over reset
Resets happen with or without user interaction. For example, incidents such as TX hang or a reconfiguration of parameters will result in a reset. During reset, hardware and software statistics were set to zero. This created an issue for the user where a reset happens in the background, statistics set to zero, and the user checks statistics expecting them to be populated.
To ensure this doesn't happen, keep accumulating stats over reset.
1. Remove function calls which reset hardware and netdev statistics.
2. Do not rollover statistics in ice_stat_update40 during reset.
--- drivers/net/ethernet/intel/ice/ice.h | 1 + drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 3 ++ drivers/net/ethernet/intel/ice/ice_lib.c | 7 +++++ drivers/net/ethernet/intel/ice/ice_main.c | 30 +++++++++++++++++--- 4 files changed, 37 insertions(+), 4 deletions(-)
drivers/net/ethernet/intel/ice/ice.h +1 -0
@@ -358,6 +358,7 @@ struct ice_vsi {/* VSI stats */ struct rtnl_link_stats64 net_stats; +struct rtnl_link_stats64 net_stats_prev;struct ice_eth_stats eth_stats;struct ice_eth_stats eth_stats_prev;
drivers/net/ethernet/intel/ice/ice_dcb_lib.c +3 -0
@@ -874,6 +874,9 @@ void ice_update_dcb_stats(struct ice_pf *pf) prev_ps = &pf->stats_prev; cur_ps = &pf->stats; +if (ice_is_reset_in_progress (pf->state)) + pf->stat_prev_loaded = false; +for (i =0 ; i <8 ; i++) { ice_stat_update32(hw, GLPRT_PXOFFRXC(port, i), pf->stat_prev_loaded,
drivers/net/ethernet/intel/ice/ice_lib.c +7 -0
@@ -1795,11 +1795,15 @@ void ice_update_eth_stats(struct ice_vsi *vsi) {struct ice_eth_stats *prev_es, *cur_es;struct ice_hw *hw = &vsi->back->hw; +struct ice_pf *pf = vsi->back;u16 vsi_num = vsi->vsi_num;/* HW absolute index of a VSI */ prev_es = &vsi->eth_stats_prev; cur_es = &vsi->eth_stats; +if (ice_is_reset_in_progress (pf->state)) + vsi->stat_offsets_loaded = false; +ice_stat_update40 (hw,GLV_GORCL (vsi_num), vsi->stat_offsets_loaded, &prev_es->rx_bytes, &cur_es->rx_bytes);
@@ -3279,6 +3283,8 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi) goto err_vectors;ice_vsi_map_rings_to_vectors (vsi); + + vsi->stat_offsets_loaded = false;if (ice_is_xdp_ena_vsi (vsi)) { ret =ice_vsi_determine_xdp_res (vsi);if (ret)
@@ -3315,6 +3321,7 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi)if (ret) goto err_vectors; + vsi->stat_offsets_loaded = false;break ;case ICE_VSI_CHNL :if (test_bit (ICE_FLAG_RSS_ENA , pf->flags)) {
drivers/net/ethernet/intel/ice/ice_main.c +26 -4
@@ -6487,6 +6487,7 @@ ice_update_vsi_tx_ring_stats(struct ice_vsi *vsi,*/ static void ice_update_vsi_ring_stats (struct ice_vsi *vsi) { +struct rtnl_link_stats64 *net_stats, *stats_prev;struct rtnl_link_stats64 *vsi_stats;u64 pkts, bytes;int i;
@@ -6526,10 +6527,28 @@ static void ice_update_vsi_ring_stats(struct ice_vsi *vsi)rcu_read_unlock (); - vsi->net_stats.tx_packets = vsi_stats->tx_packets; - vsi->net_stats.tx_bytes = vsi_stats->tx_bytes; - vsi->net_stats.rx_packets = vsi_stats->rx_packets; - vsi->net_stats.rx_bytes = vsi_stats->rx_bytes; + net_stats = &vsi->net_stats; + stats_prev = &vsi->net_stats_prev; + +/* clear prev counters after reset */ +if (vsi_stats->tx_packets < stats_prev->tx_packets || + vsi_stats->rx_packets < stats_prev->rx_packets) { + stats_prev->tx_packets =0 ; + stats_prev->tx_bytes =0 ; + stats_prev->rx_packets =0 ; + stats_prev->rx_bytes =0 ; + } + +/* update netdev counters */ + net_stats->tx_packets += vsi_stats->tx_packets - stats_prev->tx_packets; + net_stats->tx_bytes += vsi_stats->tx_bytes - stats_prev->tx_bytes; + net_stats->rx_packets += vsi_stats->rx_packets - stats_prev->rx_packets; + net_stats->rx_bytes += vsi_stats->rx_bytes - stats_prev->rx_bytes; + + stats_prev->tx_packets = vsi_stats->tx_packets; + stats_prev->tx_bytes = vsi_stats->tx_bytes; + stats_prev->rx_packets = vsi_stats->rx_packets; + stats_prev->rx_bytes = vsi_stats->rx_bytes;kfree (vsi_stats); }
@@ -6591,6 +6610,9 @@ void ice_update_pf_stats(struct ice_pf *pf) prev_ps = &pf->stats_prev; cur_ps = &pf->stats; +if (ice_is_reset_in_progress (pf->state)) + pf->stat_prev_loaded = false; +ice_stat_update40 (hw,GLPRT_GORCL (port), pf->stat_prev_loaded, &prev_ps->eth.rx_bytes, &cur_ps->eth.rx_bytes);
Signature · 6 lines
2.34.3 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [net-next, v3 1/2] ice: Accumulate HW and Netdev statistics over reset
Dear Benjamin,
Thank you for the patch.
Am 27.10.22 um 17:24 schrieb Benjamin Mikailenko:
Quoted 2 lines
Resets happen with or without user interaction. For example, incidents such as TX hang or a reconfiguration of parameters will result in a reset.
Are these resets logged somewhere?
Quoted 6 lines
During reset, hardware and software statistics were set to zero. This created an issue for the user where a reset happens in the background, statistics set to zero, and the user checks statistics expecting them to be populated.
To ensure this doesn't happen, keep accumulating stats over reset.
Are there users, which expect the statistics to be set to 0 after a reset?
Also, rtnl_link_stats64 (include/uapi/linux/if_link.h) contains more fields than rx/tx packets/bytes. Do they need to be adapted too?
Quoted 5 lines
1. Remove function calls which reset hardware and netdev statistics.
2. Do not rollover statistics inice_stat_update40during reset.Signed-off-by Benjamin Mikailenko <[email protected]>---
Do you have an overview, what changed between v1 and v3?
Quoted 85 lines
drivers/net/ethernet/intel/ice/ice.h | 1 + drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 3 ++ drivers/net/ethernet/intel/ice/ice_lib.c | 7 +++++ drivers/net/ethernet/intel/ice/ice_main.c | 30 +++++++++++++++++--- 4 files changed, 37 insertions(+), 4 deletions(-)drivers/net/ethernet/intel/ice/ice.h +1
-0@@ -358,6 +358,7 @@ struct ice_vsi {/* VSI stats */ struct rtnl_link_stats64 net_stats; +struct rtnl_link_stats64 net_stats_prev;struct ice_eth_stats eth_stats;struct ice_eth_stats eth_stats_prev;drivers/net/ethernet/intel/ice/ice_dcb_lib.c +3
-0@@ -874,6 +874,9 @@ void ice_update_dcb_stats(struct ice_pf *pf) prev_ps = &pf->stats_prev; cur_ps = &pf->stats; +if (ice_is_reset_in_progress (pf->state)) + pf->stat_prev_loaded = false; +for (i =0 ; i <8 ; i++) { ice_stat_update32(hw, GLPRT_PXOFFRXC(port, i), pf->stat_prev_loaded,drivers/net/ethernet/intel/ice/ice_lib.c +7
-0@@ -1795,11 +1795,15 @@ void ice_update_eth_stats(struct ice_vsi *vsi) {struct ice_eth_stats *prev_es, *cur_es;struct ice_hw *hw = &vsi->back->hw; +struct ice_pf *pf = vsi->back;u16 vsi_num = vsi->vsi_num;/* HW absolute index of a VSI */ prev_es = &vsi->eth_stats_prev; cur_es = &vsi->eth_stats; +if (ice_is_reset_in_progress (pf->state)) + vsi->stat_offsets_loaded = false; +ice_stat_update40 (hw,GLV_GORCL (vsi_num), vsi->stat_offsets_loaded, &prev_es->rx_bytes, &cur_es->rx_bytes);@@ -3279,6 +3283,8 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi) goto err_vectors;ice_vsi_map_rings_to_vectors (vsi); + + vsi->stat_offsets_loaded = false;if (ice_is_xdp_ena_vsi (vsi)) { ret =ice_vsi_determine_xdp_res (vsi);if (ret)@@ -3315,6 +3321,7 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi)if (ret) goto err_vectors; + vsi->stat_offsets_loaded = false;break ;case ICE_VSI_CHNL :if (test_bit (ICE_FLAG_RSS_ENA , pf->flags)) {drivers/net/ethernet/intel/ice/ice_main.c +1
-0@@ -6487,6 +6487,7 @@ ice_update_vsi_tx_ring_stats(struct ice_vsi *vsi,*/ static void ice_update_vsi_ring_stats (struct ice_vsi *vsi) { +struct rtnl_link_stats64 *net_stats, *stats_prev;struct rtnl_link_stats64 *vsi_stats;u64 pkts, bytes;int i;@@ -6526,10 +6527,28 @@ static void ice_update_vsi_ring_stats(struct ice_vsi *vsi)rcu_read_unlock ();
drivers/net/ethernet/intel/ice/ice_main.c static void ice_update_vsi_ring_stats(struct ice_vsi *vsi) · line 6535
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_main.c +6
-4- vsi->net_stats.tx_packets = vsi_stats->tx_packets; - vsi->net_stats.tx_bytes = vsi_stats->tx_bytes; - vsi->net_stats.rx_packets = vsi_stats->rx_packets; - vsi->net_stats.rx_bytes = vsi_stats->rx_bytes; + net_stats = &vsi->net_stats; + stats_prev = &vsi->net_stats_prev; + +/* clear prev counters after reset */ +if (vsi_stats->tx_packets < stats_prev->tx_packets || + vsi_stats->rx_packets < stats_prev->rx_packets) {
Why not unconditionally clear it?
Quoted 20 lines
drivers/net/ethernet/intel/ice/ice_main.c +16
-0+ stats_prev->tx_packets =0 ; + stats_prev->tx_bytes =0 ; + stats_prev->rx_packets =0 ; + stats_prev->rx_bytes =0 ; + } + +/* update netdev counters */ + net_stats->tx_packets += vsi_stats->tx_packets - stats_prev->tx_packets; + net_stats->tx_bytes += vsi_stats->tx_bytes - stats_prev->tx_bytes; + net_stats->rx_packets += vsi_stats->rx_packets - stats_prev->rx_packets; + net_stats->rx_bytes += vsi_stats->rx_bytes - stats_prev->rx_bytes; + + stats_prev->tx_packets = vsi_stats->tx_packets; + stats_prev->tx_bytes = vsi_stats->tx_bytes; + stats_prev->rx_packets = vsi_stats->rx_packets; + stats_prev->rx_bytes = vsi_stats->rx_bytes;kfree (vsi_stats); }
drivers/net/ethernet/intel/ice/ice_main.c void ice_update_pf_stats(struct ice_pf *pf) · line 6618
Quoted 9 lines
drivers/net/ethernet/intel/ice/ice_main.c +3
-0@@ -6591,6 +6610,9 @@ void ice_update_pf_stats(struct ice_pf *pf) prev_ps = &pf->stats_prev; cur_ps = &pf->stats; +if (ice_is_reset_in_progress (pf->state)) + pf->stat_prev_loaded = false; +ice_stat_update40 (hw,GLPRT_GORCL (port), pf->stat_prev_loaded, &prev_ps->eth.rx_bytes, &cur_ps->eth.rx_bytes);
Kind regards,
Paul
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [net-next, v3 1/2] ice: Accumulate HW and Netdev statistics over reset
Quoted 5 lines · Paul Menzel
Dear Benjamin,
Thank you for the patch.
Hello Paul,
Thanks for the review!
I apologize for the prolonged response. I wanted to reach out to other developers before responding.
Quoted 6 lines
Am 27.10.22 um 17:24 schrieb Benjamin Mikailenko:
Quoted 2 lines
Resets happen with or without user interaction. For example, incidents such as TX hang or a reconfiguration of parameters will result in a reset.
Are these resets logged somewhere?
Dmesg will show a reset occurred.
The ICE driver also maintains a counter of how many resets were performed. But this information isn't currently reported to the user.
Quoted 9 lines
Quoted 6 lines
During reset, hardware and software statistics were set to zero. This created an issue for the user where a reset happens in the background, statistics set to zero, and the user checks statistics expecting them to be populated.
To ensure this doesn't happen, keep accumulating stats over reset.
Are there users, which expect the statistics to be set to 0 after a reset?
There's currently no functionality for a user to purposefully initiate a standalone reset. The only time a reset will occur is as result of another process. Since the ability isn't possible, I don't see why a current user would expect statistics to set to zero over reset. If they did want to reset statistics unloading/reloading the driver will reset statistics.
Quoted 2 lines
Also,
rtnl_link_stats64(include/uapi/linux/if_link.h) contains more fields than rx/tx packets/bytes. Do they need to be adapted too?
The other fields in rtnl_link_stats64 aren't currently used.
Quoted 8 lines
Quoted 5 lines
1. Remove function calls which reset hardware and netdev statistics.
2. Do not rollover statistics inice_stat_update40during reset.Signed-off-by Benjamin Mikailenko <[email protected]>---
Do you have an overview, what changed between v1 and v3?
Adding this to the next version (v4):
v2: Fixed styling
v3: Allocate statistic structures when XDP is enabled
v4: Added revision notes
Quoted 120 lines
Quoted 75 lines
drivers/net/ethernet/intel/ice/ice.h | 1 +
drivers/net/ethernet/intel/ice/ice_dcb_lib.c| 3 ++ drivers/net/ethernet/intel/ice/ice_lib.c | 7 +++++ drivers/net/ethernet/intel/ice/ice_main.c | 30 +++++++++++++++++---4 files changed, 37 insertions(+), 4 deletions(-)drivers/net/ethernet/intel/ice/ice.h +1
-0@@ -358,6 +358,7 @@ struct ice_vsi { /* VSI stats */ struct rtnl_link_stats64 net_stats; + struct rtnl_link_stats64 net_stats_prev; struct ice_eth_stats eth_stats; struct ice_eth_stats eth_stats_prev;diff --git
a/drivers/net/ethernet/intel/ice/ice_dcb_lib.cb/drivers/net/ethernet/intel/ice/ice_dcb_lib.cindex5e6d168bac48..4f3a848f4e99100644drivers/net/ethernet/intel/ice/ice_dcb_lib.c +2
-0@@ -874,6 +874,9 @@ void ice_update_dcb_stats(struct ice_pf *pf) prev_ps = &pf->stats_prev; cur_ps = &pf->stats; + if (ice_is_reset_in_progress(pf->state)) + pf->stat_prev_loaded = false; + for (i = 0; i < 8; i++) { ice_stat_update32(hw, GLPRT_PXOFFRXC(port, i), pf->stat_prev_loaded,drivers/net/ethernet/intel/ice/ice_lib.c +1
-0@@ -1795,11 +1795,15 @@ void ice_update_eth_stats(struct ice_vsi *vsi) { struct ice_eth_stats *prev_es, *cur_es; struct ice_hw *hw = &vsi->back->hw; + struct ice_pf *pf = vsi->back;u16
vsi_num= vsi->vsi_num; /* HW absolute index of a VSI */prev_es=&vsi->eth_stats_prev;
cur_es=&vsi->eth_stats;
+ if (ice_is_reset_in_progress(pf->state))
+vsi->stat_offsets_loaded= false;
+
ice_stat_update40(hw, GLV_GORCL(vsi_num), vsi->stat_offsets_loaded, &prev_es->rx_bytes, &cur_es->rx_bytes);
@@ -3279,6 +3283,8 @@ intice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi)gotoerr_vectors;
ice_vsi_map_rings_to_vectors(vsi);
+
+vsi->stat_offsets_loaded= false;
if (ice_is_xdp_ena_vsi(vsi)) {
ret =ice_vsi_determine_xdp_res(vsi);
if (ret)drivers/net/ethernet/intel/ice/ice_lib.c +0
-0@@ -3315,6 +3321,7 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi) if (ret) goto err_vectors; + vsi->stat_offsets_loaded = false; break;case
ICE_VSI_CHNL:
if (test_bit(ICE_FLAG_RSS_ENA, pf->flags)) {drivers/net/ethernet/intel/ice/ice_main.c +1
-0@@ -6487,6 +6487,7 @@ ice_update_vsi_tx_ring_stats(struct ice_vsi *vsi,*/ static void ice_update_vsi_ring_stats(struct ice_vsi *vsi) { + struct rtnl_link_stats64 *net_stats, *stats_prev; struct rtnl_link_stats64 *vsi_stats; u64 pkts, bytes; int i;@@ -6526,10 +6527,28 @@ static void ice_update_vsi_ring_stats(struct ice_vsi *vsi) rcu_read_unlock();drivers/net/ethernet/intel/ice/ice_main.c static void ice_update_vsi_ring_stats(struct ice_vsi *vsi) · line 6534
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_main.c +6
-3- vsi->net_stats.tx_packets = vsi_stats->tx_packets; - vsi->net_stats.tx_bytes = vsi_stats->tx_bytes; - vsi->net_stats.rx_packets = vsi_stats->rx_packets; - vsi->net_stats.rx_bytes = vsi_stats->rx_bytes; + net_stats = &vsi->net_stats; + stats_prev = &vsi->net_stats_prev; + + /* clear prev counters after reset */ + if (vsi_stats->tx_packets < stats_prev->tx_packets || + vsi_stats->rx_packets < stats_prev->rx_packets) {Why not unconditionally clear it?
Quoted 19 lines
drivers/net/ethernet/intel/ice/ice_main.c +16
-0+ stats_prev->tx_packets = 0; + stats_prev->tx_bytes = 0; + stats_prev->rx_packets = 0; + stats_prev->rx_bytes = 0; + } + + /* update netdev counters */ + net_stats->tx_packets += vsi_stats->tx_packets - stats_prev->tx_packets; + net_stats->tx_bytes += vsi_stats->tx_bytes - stats_prev->tx_bytes; + net_stats->rx_packets += vsi_stats->rx_packets - stats_prev->rx_packets; + net_stats->rx_bytes += vsi_stats->rx_bytes - stats_prev->rx_bytes; + + stats_prev->tx_packets = vsi_stats->tx_packets; + stats_prev->tx_bytes = vsi_stats->tx_bytes; + stats_prev->rx_packets = vsi_stats->rx_packets; + stats_prev->rx_bytes = vsi_stats->rx_bytes; kfree(vsi_stats); }drivers/net/ethernet/intel/ice/ice_main.c void ice_update_pf_stats(struct ice_pf *pf) · line 6617
Quoted 8 lines
drivers/net/ethernet/intel/ice/ice_main.c +2
-0@@ -6591,6 +6610,9 @@ void ice_update_pf_stats(struct ice_pf *pf) prev_ps = &pf->stats_prev; cur_ps = &pf->stats; + if (ice_is_reset_in_progress(pf->state)) + pf->stat_prev_loaded = false; + ice_stat_update40(hw, GLPRT_GORCL(port), pf->stat_prev_loaded, &prev_ps->eth.rx_bytes, &cur_ps->eth.rx_bytes);Kind regards,
Quoted 1 line
Paul
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [net-next, v3 1/2] ice: Accumulate HW and Netdev statistics over reset
Quoted 52 lines · Benjamin Mikailenko
Quoted 5 lines · Paul Menzel
Dear Benjamin,
Thank you for the patch.
Hello Paul,
Thanks for the review!
I apologize for the prolonged response. I wanted to reach out to other developers before responding.Quoted 6 lines
Am 27.10.22 um 17:24 schrieb Benjamin Mikailenko:
Quoted 2 lines
Resets happen with or without user interaction. For example, incidents such as TX hang or a reconfiguration of parameters will result in a reset.
Are these resets logged somewhere?
Dmesg will show a reset occurred.
The ICE driver also maintains a counter of how many resets were performed. But this information isn't currently reported to the user.Quoted 9 lines
Quoted 6 lines
During reset, hardware and software statistics were set to zero. This created an issue for the user where a reset happens in the background, statistics set to zero, and the user checks statistics expecting them to be populated.
To ensure this doesn't happen, keep accumulating stats over reset.
Are there users, which expect the statistics to be set to 0 after a reset?
There's currently no functionality for a user to purposefully initiate a standalone reset. The only time a reset will occur is as result of another process. Since the ability isn't possible, I don't see why a current user would expect statistics to set to zero over reset. If they did want to reset statistics unloading/reloading the driver will reset statistics.
Quoted 2 lines
Also,
rtnl_link_stats64(include/uapi/linux/if_link.h) contains more fields than rx/tx packets/bytes. Do they need to be adapted too?The other fields in
rtnl_link_stats64aren't currently used.Quoted 6 lines
Quoted 5 lines
1. Remove function calls which reset hardware and netdev statistics.
2. Do not rollover statistics inice_stat_update40during reset.Signed-off-by Benjamin Mikailenko <[email protected]>---
Quoted 96 lines
Quoted 2 lines
Do you have an overview, what changed between v1 and v3?
Adding this to the next version (v4):
v2: Fixed styling
v3: Allocate statistic structures when XDP is enabled
v4: Added revision notesQuoted 88 lines
Quoted 75 lines
drivers/net/ethernet/intel/ice/ice.h | 1 +
drivers/net/ethernet/intel/ice/ice_dcb_lib.c| 3 ++ drivers/net/ethernet/intel/ice/ice_lib.c | 7 +++++ drivers/net/ethernet/intel/ice/ice_main.c | 30 +++++++++++++++++---4 files changed, 37 insertions(+), 4 deletions(-)drivers/net/ethernet/intel/ice/ice.h +1
-0@@ -358,6 +358,7 @@ struct ice_vsi { /* VSI stats */ struct rtnl_link_stats64 net_stats; + struct rtnl_link_stats64 net_stats_prev; struct ice_eth_stats eth_stats; struct ice_eth_stats eth_stats_prev;diff --git
a/drivers/net/ethernet/intel/ice/ice_dcb_lib.cb/drivers/net/ethernet/intel/ice/ice_dcb_lib.cindex5e6d168bac48..4f3a848f4e99100644drivers/net/ethernet/intel/ice/ice_dcb_lib.c +2
-0@@ -874,6 +874,9 @@ void ice_update_dcb_stats(struct ice_pf *pf) prev_ps = &pf->stats_prev; cur_ps = &pf->stats; + if (ice_is_reset_in_progress(pf->state)) + pf->stat_prev_loaded = false; + for (i = 0; i < 8; i++) { ice_stat_update32(hw, GLPRT_PXOFFRXC(port, i), pf->stat_prev_loaded,drivers/net/ethernet/intel/ice/ice_lib.c +1
-0@@ -1795,11 +1795,15 @@ void ice_update_eth_stats(struct ice_vsi *vsi) { struct ice_eth_stats *prev_es, *cur_es; struct ice_hw *hw = &vsi->back->hw; + struct ice_pf *pf = vsi->back;u16
vsi_num= vsi->vsi_num; /* HW absolute index of a VSI */prev_es=&vsi->eth_stats_prev;
cur_es=&vsi->eth_stats;
+ if (ice_is_reset_in_progress(pf->state))
+vsi->stat_offsets_loaded= false;
+
ice_stat_update40(hw, GLV_GORCL(vsi_num), vsi->stat_offsets_loaded, &prev_es->rx_bytes, &cur_es->rx_bytes);
@@ -3279,6 +3283,8 @@ intice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi)gotoerr_vectors;
ice_vsi_map_rings_to_vectors(vsi);
+
+vsi->stat_offsets_loaded= false;
if (ice_is_xdp_ena_vsi(vsi)) {
ret =ice_vsi_determine_xdp_res(vsi);
if (ret)drivers/net/ethernet/intel/ice/ice_lib.c +0
-0@@ -3315,6 +3321,7 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi) if (ret) goto err_vectors; + vsi->stat_offsets_loaded = false; break;case
ICE_VSI_CHNL:
if (test_bit(ICE_FLAG_RSS_ENA, pf->flags)) {drivers/net/ethernet/intel/ice/ice_main.c +1
-0@@ -6487,6 +6487,7 @@ ice_update_vsi_tx_ring_stats(struct ice_vsi *vsi,*/ static void ice_update_vsi_ring_stats(struct ice_vsi *vsi) { + struct rtnl_link_stats64 *net_stats, *stats_prev; struct rtnl_link_stats64 *vsi_stats; u64 pkts, bytes; int i;@@ -6526,10 +6527,28 @@ static void ice_update_vsi_ring_stats(struct ice_vsi *vsi) rcu_read_unlock();drivers/net/ethernet/intel/ice/ice_main.c static void ice_update_vsi_ring_stats(struct ice_vsi *vsi) · line 6534
Quoted 10 lines
drivers/net/ethernet/intel/ice/ice_main.c +6
-3- vsi->net_stats.tx_packets = vsi_stats->tx_packets; - vsi->net_stats.tx_bytes = vsi_stats->tx_bytes; - vsi->net_stats.rx_packets = vsi_stats->rx_packets; - vsi->net_stats.rx_bytes = vsi_stats->rx_bytes; + net_stats = &vsi->net_stats; + stats_prev = &vsi->net_stats_prev; + + /* clear prev counters after reset */ + if (vsi_stats->tx_packets < stats_prev->tx_packets || + vsi_stats->rx_packets < stats_prev->rx_packets) {Why not unconditionally clear it?
Missed this question -
We need to maintain a delta and increment net_stats based on the delta. That's because vsi_stats are zeroed out by the hardware.
Quoted 32 lines
Quoted 32 lines
Quoted 19 lines
drivers/net/ethernet/intel/ice/ice_main.c +16
-0+ stats_prev->tx_packets = 0; + stats_prev->tx_bytes = 0; + stats_prev->rx_packets = 0; + stats_prev->rx_bytes = 0; + } + + /* update netdev counters */ + net_stats->tx_packets += vsi_stats->tx_packets - stats_prev->tx_packets; + net_stats->tx_bytes += vsi_stats->tx_bytes - stats_prev->tx_bytes; + net_stats->rx_packets += vsi_stats->rx_packets - stats_prev->rx_packets; + net_stats->rx_bytes += vsi_stats->rx_bytes - stats_prev->rx_bytes; + + stats_prev->tx_packets = vsi_stats->tx_packets; + stats_prev->tx_bytes = vsi_stats->tx_bytes; + stats_prev->rx_packets = vsi_stats->rx_packets; + stats_prev->rx_bytes = vsi_stats->rx_bytes; kfree(vsi_stats); }drivers/net/ethernet/intel/ice/ice_main.c void ice_update_pf_stats(struct ice_pf *pf) · line 6617
Quoted 8 lines
drivers/net/ethernet/intel/ice/ice_main.c +2
-0@@ -6591,6 +6610,9 @@ void ice_update_pf_stats(struct ice_pf *pf) prev_ps = &pf->stats_prev; cur_ps = &pf->stats; + if (ice_is_reset_in_progress(pf->state)) + pf->stat_prev_loaded = false; + ice_stat_update40(hw, GLPRT_GORCL(port), pf->stat_prev_loaded, &prev_ps->eth.rx_bytes, &cur_ps->eth.rx_bytes);Kind regards,
Quoted 1 line
Quoted 1 line
Paul
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [net-next, v3 1/2] ice: Accumulate HW and Netdev statistics over reset
In reply to Benjamin MikailenkoQuoted 22 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Benjamin Mikailenko
Sent: Thursday, October 27, 2022 8:54 PM
To: [email protected]
Cc: Mikailenko, Benjamin <[email protected]>
Subject: [Intel-wired-lan] [net-next, v3 1/2] ice: Accumulate HW and Netdev statistics over resetResets happen with or without user interaction. For example, incidents such as TX hang or a reconfiguration of parameters will result in a reset. During reset, hardware and software statistics were set to zero. This created an issue for the user where a reset happens in the background, statistics set to zero, and the user checks statistics expecting them to be populated.
To ensure this doesn't happen, keep accumulating stats over reset.
1. Remove function calls which reset hardware and netdev statistics.
2. Do not rollover statistics inice_stat_update40during reset.
Quoted 8 lines
Signed-off-by Benjamin Mikailenko <[email protected]>--- drivers/net/ethernet/intel/ice/ice.h | 1 + drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 3 ++ drivers/net/ethernet/intel/ice/ice_lib.c | 7 +++++ drivers/net/ethernet/intel/ice/ice_main.c | 30 +++++++++++++++++--- 4 files changed, 37 insertions(+), 4 deletions(-)
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v3 2/2 ice: Accumulate ring statistics over reset
In reply to Benjamin MikailenkoResets may occur with or without user interaction. For example, a TX hang or reconfiguration of parameters will result in a reset. During reset, the VSI is freed, freeing any statistics structures inside as well. This would create an issue for the user where a reset happens in the background, statistics set to zero, and the user checks ring statistics expecting them to be populated.
To ensure this doesn't happen, accumulate ring statistics over reset.
Define a new ring statistics structure, ice_ring_stats. The new structure lives in the VSI's parent, preserving ring statistics when VSI is freed.
1. Define a new structure vsi_ring_stats in the PF scope
2. Allocate/free stats only during probe, unload, or change in ring size
3. Replace previous ring statistics functionality with new structure
--- drivers/net/ethernet/intel/ice/ice.h | 6 + drivers/net/ethernet/intel/ice/ice_ethtool.c | 12 +- drivers/net/ethernet/intel/ice/ice_lib.c | 256 +++++++++++++++++- drivers/net/ethernet/intel/ice/ice_main.c | 68 ++++- drivers/net/ethernet/intel/ice/ice_repr.c | 10 +- drivers/net/ethernet/intel/ice/ice_txrx.c | 40 ++- drivers/net/ethernet/intel/ice/ice_txrx.h | 18 +- drivers/net/ethernet/intel/ice/ice_txrx_lib.c | 2 +- drivers/net/ethernet/intel/ice/ice_xsk.c | 25 +- 9 files changed, 377 insertions(+), 60 deletions(-)
drivers/net/ethernet/intel/ice/ice.h +6 -0
@@ -305,6 +305,11 @@ enum ice_vsi_state {ICE_VSI_STATE_NBITS /* must be last */ }; +struct ice_vsi_stats { +struct ice_ring_stats **tx_ring_stats;/* Tx ring stats array */ +struct ice_ring_stats **rx_ring_stats;/* Rx ring stats array */ +}; +/* struct that defines a VSI, associated with a dev */ struct ice_vsi {struct net_device *netdev;
@@ -526,6 +531,7 @@ struct ice_pf {u16 ctrl_vsi_idx;/* control VSI index in pf->vsi array */ struct ice_vsi **vsi;/* VSIs created by the driver */ +struct ice_vsi_stats **vsi_stats;struct ice_sw *first_sw;/* first switch created by firmware */ u16 eswitch_mode;/* current mode of eswitch */ struct ice_vfs vfs;
drivers/net/ethernet/intel/ice/ice_ethtool.c +6 -6
@@ -1375,9 +1375,9 @@ __ice_get_ethtool_stats(struct net_device *netdev,ice_for_each_alloc_txq (vsi, j) { tx_ring =READ_ONCE (vsi->tx_rings[j]); -if (tx_ring) { - data[i++] = tx_ring->stats.pkts; - data[i++] = tx_ring->stats.bytes; +if (tx_ring && tx_ring->ring_stats) { + data[i++] = tx_ring->ring_stats->stats.pkts; + data[i++] = tx_ring->ring_stats->stats.bytes; }else { data[i++] =0 ; data[i++] =0 ;
@@ -1386,9 +1386,9 @@ __ice_get_ethtool_stats(struct net_device *netdev,ice_for_each_alloc_rxq (vsi, j) { rx_ring =READ_ONCE (vsi->rx_rings[j]); -if (rx_ring) { - data[i++] = rx_ring->stats.pkts; - data[i++] = rx_ring->stats.bytes; +if (rx_ring && rx_ring->ring_stats) { + data[i++] = rx_ring->ring_stats->stats.pkts; + data[i++] = rx_ring->ring_stats->stats.bytes; }else { data[i++] =0 ; data[i++] =0 ;
drivers/net/ethernet/intel/ice/ice_lib.c +248 -8
355 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_main.c +54 -14
157 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_repr.c +6 -4
@@ -163,18 +163,20 @@ ice_repr_sp_stats64(const struct net_device *dev,u64 pkts, bytes; tx_ring = np->vsi->tx_rings[vf_id]; -ice_fetch_u64_stats_per_ring (&tx_ring->syncp, tx_ring->stats, +ice_fetch_u64_stats_per_ring (&tx_ring->ring_stats->syncp, + tx_ring->ring_stats->stats, &pkts, &bytes); stats->rx_packets = pkts; stats->rx_bytes = bytes; rx_ring = np->vsi->rx_rings[vf_id]; -ice_fetch_u64_stats_per_ring (&rx_ring->syncp, rx_ring->stats, +ice_fetch_u64_stats_per_ring (&rx_ring->ring_stats->syncp, + rx_ring->ring_stats->stats, &pkts, &bytes); stats->tx_packets = pkts; stats->tx_bytes = bytes; - stats->tx_dropped = rx_ring->rx_stats.alloc_page_failed + - rx_ring->rx_stats.alloc_buf_failed; + stats->tx_dropped = rx_ring->ring_stats->rx_stats.alloc_page_failed + + rx_ring->ring_stats->rx_stats.alloc_buf_failed;return 0 ; }
drivers/net/ethernet/intel/ice/ice_txrx.c +26 -14
@@ -325,7 +325,7 @@ static bool ice_clean_tx_irq(struct ice_tx_ring *tx_ring, int napi_budget)if (netif_tx_queue_stopped (txring_txq (tx_ring)) && !test_bit (ICE_VSI_DOWN , vsi->state)) {netif_tx_wake_queue (txring_txq (tx_ring)); - ++tx_ring->tx_stats.restart_q; + ++tx_ring->ring_stats->tx_stats.restart_q; } }
@@ -367,7 +367,7 @@ int ice_setup_tx_ring(struct ice_tx_ring *tx_ring) tx_ring->next_to_use =0 ; tx_ring->next_to_clean =0 ; - tx_ring->tx_stats.prev_pkt =-1 ; + tx_ring->ring_stats->tx_stats.prev_pkt =-1 ;return 0 ; err:
@@ -667,7 +667,7 @@ ice_alloc_mapped_page(struct ice_rx_ring *rx_ring, struct ice_rx_buf *bi)/* alloc new page for storage */ page =dev_alloc_pages (ice_rx_pg_order (rx_ring));if (unlikely (!page)) { - rx_ring->rx_stats.alloc_page_failed++; + rx_ring->ring_stats->rx_stats.alloc_page_failed++;return false; }
@@ -680,7 +680,7 @@ ice_alloc_mapped_page(struct ice_rx_ring *rx_ring, struct ice_rx_buf *bi)*/ if (dma_mapping_error (rx_ring->dev, dma)) {__free_pages (page,ice_rx_pg_order (rx_ring)); - rx_ring->rx_stats.alloc_page_failed++; + rx_ring->ring_stats->rx_stats.alloc_page_failed++;return false; }
@@ -1091,7 +1091,7 @@ ice_is_non_eop(struct ice_rx_ring *rx_ring, union ice_32b_rx_flex_desc *rx_desc)if (likely (ice_test_staterr (rx_desc->wb.status_error0,ICE_RXD_EOF )))return false; - rx_ring->rx_stats.non_eop_descs++; + rx_ring->ring_stats->rx_stats.non_eop_descs++;return true; }
@@ -1222,7 +1222,7 @@ int ice_clean_rx_irq(struct ice_rx_ring *rx_ring, int budget) }/* exit if we failed to retrieve a buffer */ if (!skb) { - rx_ring->rx_stats.alloc_buf_failed++; + rx_ring->ring_stats->rx_stats.alloc_buf_failed++;if (rx_buf) rx_buf->pagecnt_bias++;break ;
@@ -1275,7 +1275,9 @@ int ice_clean_rx_irq(struct ice_rx_ring *rx_ring, int budget)ice_finalize_xdp_rx (xdp_ring, xdp_xmit); rx_ring->skb = skb; -ice_update_rx_ring_stats (rx_ring, total_rx_pkts, total_rx_bytes); +if (rx_ring->ring_stats) +ice_update_rx_ring_stats (rx_ring, total_rx_pkts, + total_rx_bytes);/* guarantee a trip back through this routine if there was a failure */ return failure ? budget : (int )total_rx_pkts;
@@ -1292,15 +1294,25 @@ static void __ice_update_sample(struct ice_q_vector *q_vector,struct ice_tx_ring *tx_ring;ice_for_each_tx_ring (tx_ring, *rc) { - packets += tx_ring->stats.pkts; - bytes += tx_ring->stats.bytes; +struct ice_ring_stats *ring_stats; + + ring_stats = tx_ring->ring_stats; +if (!ring_stats) +continue ; + packets += ring_stats->stats.pkts; + bytes += ring_stats->stats.bytes; } }else {struct ice_rx_ring *rx_ring;ice_for_each_rx_ring (rx_ring, *rc) { - packets += rx_ring->stats.pkts; - bytes += rx_ring->stats.bytes; +struct ice_ring_stats *ring_stats; + + ring_stats = rx_ring->ring_stats; +if (!ring_stats) +continue ; + packets += ring_stats->stats.pkts; + bytes += ring_stats->stats.bytes; } }
@@ -1549,7 +1561,7 @@ static int __ice_maybe_stop_tx(struct ice_tx_ring *tx_ring, unsigned int size)/* A reprieve! - use start_queue because it doesn't call schedule */ netif_tx_start_queue (txring_txq (tx_ring)); - ++tx_ring->tx_stats.restart_q; + ++tx_ring->ring_stats->tx_stats.restart_q;return 0 ; }
@@ -2293,7 +2305,7 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct ice_tx_ring *tx_ring)if (__skb_linearize (skb)) goto out_drop; count =ice_txd_use_count (skb->len); - tx_ring->tx_stats.tx_linearize++; + tx_ring->ring_stats->tx_stats.tx_linearize++; }/* need: 1 descriptor per page * PAGE_SIZE/ICE_MAX_DATA_PER_TXD,
@@ -2304,7 +2316,7 @@ ice_xmit_frame_ring(struct sk_buff *skb, struct ice_tx_ring *tx_ring)*/ if (ice_maybe_stop_tx (tx_ring, count +ICE_DESCS_PER_CACHE_LINE +ICE_DESCS_FOR_CTX_DESC )) { - tx_ring->tx_stats.tx_busy++; + tx_ring->ring_stats->tx_stats.tx_busy++;return NETDEV_TX_BUSY ; }
drivers/net/ethernet/intel/ice/ice_txrx.h +12 -6
@@ -191,6 +191,16 @@ struct ice_rxq_stats {u64 alloc_buf_failed; }; +struct ice_ring_stats { +struct rcu_head rcu;/* to avoid race on free */ +struct ice_q_stats stats; +struct u64_stats_sync syncp; +union { +struct ice_txq_stats tx_stats; +struct ice_rxq_stats rx_stats; + }; +}; +enum ice_ring_state_t {ICE_TX_XPS_INIT_DONE ,ICE_TX_NBITS ,
@@ -283,9 +293,7 @@ struct ice_rx_ring {u16 rx_buf_len;/* stats structs */ -struct ice_rxq_stats rx_stats; -struct ice_q_stats stats; -struct u64_stats_sync syncp; +struct ice_ring_stats *ring_stats;struct rcu_head rcu;/* to avoid race on free */ /* CL4 - 3rd cacheline starts here */
@@ -325,10 +333,8 @@ struct ice_tx_ring {u16 count;/* Number of descriptors */ u16 q_index;/* Queue number of ring */ /* stats structs */ -struct ice_txq_stats tx_stats; +struct ice_ring_stats *ring_stats;/* CL3 - 3rd cacheline starts here */ -struct ice_q_stats stats; -struct u64_stats_sync syncp;struct rcu_head rcu;/* to avoid race on free */ DECLARE_BITMAP (xps_state,ICE_TX_NBITS );/* XPS Config State */ struct ice_channel *ch;
drivers/net/ethernet/intel/ice/ice_txrx_lib.c +1 -1
@@ -285,7 +285,7 @@ int ice_xmit_xdp_ring(void *data, u16 size, struct ice_tx_ring *xdp_ring)ice_clean_xdp_irq (xdp_ring);if (!unlikely (ICE_DESC_UNUSED (xdp_ring))) { - xdp_ring->tx_stats.tx_busy++; + xdp_ring->ring_stats->tx_stats.tx_busy++;return ICE_XDP_CONSUMED ; }
drivers/net/ethernet/intel/ice/ice_xsk.c +18 -7
@@ -24,13 +24,24 @@ static struct xdp_buff **ice_xdp_buf(struct ice_rx_ring *rx_ring, u32 idx)*/ static void ice_qp_reset_stats (struct ice_vsi *vsi,u16 q_idx) { -memset (&vsi->rx_rings[q_idx]->rx_stats,0 , -sizeof (vsi->rx_rings[q_idx]->rx_stats)); -memset (&vsi->tx_rings[q_idx]->stats,0 , -sizeof (vsi->tx_rings[q_idx]->stats)); +struct ice_vsi_stats *vsi_stat; +struct ice_pf *pf; + + pf = vsi->back; +if (!pf->vsi_stats) +return ; + + vsi_stat = pf->vsi_stats[vsi->idx]; +if (!vsi_stat) +return ; + +memset (&vsi_stat->rx_ring_stats[q_idx]->rx_stats,0 , +sizeof (vsi_stat->rx_ring_stats[q_idx]->rx_stats)); +memset (&vsi_stat->tx_ring_stats[q_idx]->stats,0 , +sizeof (vsi_stat->tx_ring_stats[q_idx]->stats));if (ice_is_xdp_ena_vsi (vsi)) -memset (&vsi->xdp_rings[q_idx]->stats,0 , -sizeof (vsi->xdp_rings[q_idx]->stats)); +memset (&vsi->xdp_rings[q_idx]->ring_stats->stats,0 , +sizeof (vsi->xdp_rings[q_idx]->ring_stats->stats)); }/**
@@ -722,7 +733,7 @@ int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, int budget)/* XDP_PASS path */ skb =ice_construct_skb_zc (rx_ring, xdp);if (!skb) { - rx_ring->rx_stats.alloc_buf_failed++; + rx_ring->ring_stats->rx_stats.alloc_buf_failed++;break ; }
Signature · 6 lines
2.34.3 _______________________________________________ Intel-wired-lan mailing list [email protected] https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Re: [Intel-wired-lan] [net-next, v3 2/2] ice: Accumulate ring statistics over reset
Quoted 27 lines
-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Benjamin Mikailenko
Sent: Thursday, October 27, 2022 8:54 PM
To: [email protected]
Cc: Mikailenko, Benjamin <[email protected]>
Subject: [Intel-wired-lan] [net-next, v3 2/2] ice: Accumulate ring statistics over resetResets may occur with or without user interaction. For example, a TX hang or reconfiguration of parameters will result in a reset. During reset, the VSI is freed, freeing any statistics structures inside as well. This would create an issue for the user where a reset happens in the background, statistics set to zero, and the user checks ring statistics expecting them to be populated.
To ensure this doesn't happen, accumulate ring statistics over reset.
Define a new ring statistics structure,
ice_ring_stats. The new structure lives in the VSI's parent, preserving ring statistics when VSI is freed.1. Define a new structure
vsi_ring_statsin the PF scope 2. Allocate/free stats only during probe, unload, or change in ring size 3. Replace previous ring statistics functionality with new structureSigned-off-by Benjamin Mikailenko <[email protected]>
Quoted 12 lines
--- drivers/net/ethernet/intel/ice/ice.h | 6 + drivers/net/ethernet/intel/ice/ice_ethtool.c | 12 +- drivers/net/ethernet/intel/ice/ice_lib.c | 256 +++++++++++++++++- drivers/net/ethernet/intel/ice/ice_main.c | 68 ++++- drivers/net/ethernet/intel/ice/ice_repr.c | 10 +- drivers/net/ethernet/intel/ice/ice_txrx.c | 40 ++- drivers/net/ethernet/intel/ice/ice_txrx.h | 18 +- drivers/net/ethernet/intel/ice/ice_txrx_lib.c | 2 +- drivers/net/ethernet/intel/ice/ice_xsk.c | 25 +- 9 files changed, 377 insertions(+), 60 deletions(-)
_______________________________________________
Intel-wired-lan mailing list
[email protected]
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan