Skip to contentSkip to conversation

igc bug fixes related to qbv_count usage

PATCH In review Faizal Rahim 5 participants 16 messages Jul 7, 2024 – Aug 1, 2024

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v2 cover igc bug fixes related to qbv_count usage

These igc bug fixes are sent as a patch series because:

1. The two patches below remove the reliance on using the qbv_count field. "igc: Fix qbv_config_change_errors logics"
"igc: Fix reset adapter logics when tx mode change"

qbv_count field will be removed in future patch via iwl-next.

2. The patch "igc: Fix qbv tx latency by setting gtxoffset" reuse the function igc_tsn_will_tx_mode_change() which was created in the patch:
"igc: Fix reset adapter logics when tx mode change"

v1: https://patchwork.kernel.org/project/netdevbpf/cover/[email protected]/

Changelog:
v1 -> v2
- Instead of casting to bool, use !! (Simon)
- Simplify new functions created. Instead of if.. return true, else return false, use single return. (Simon)
- Remove patch "igc: Remove unused qbv_coun" from this series which is targeting to iwl-net. This patch will be sent to iwl-next. (Simon)

Faizal Rahim (3):
igc: Fix qbv_config_change_errors logics
igc: Fix reset adapter logics when tx mode change
igc: Fix qbv tx latency by setting gtxoffset

 drivers/net/ethernet/intel/igc/igc_main.c |  8 +++--
 drivers/net/ethernet/intel/igc/igc_tsn.c  | 41 ++++++++++++++++-------
 drivers/net/ethernet/intel/igc/igc_tsn.h  |  1 +
 3 files changed, 36 insertions(+), 14 deletions(-)

--
2.25.1

#1 lore raw

PATCH v2 1/3 igc: Fix qbv_config_change_errors logics

When user issues these cmds:
1. Either a) or b)
a) mqprio with hardware offload disabled
b) taprio with txtime-assist feature enabled
2. etf
3. tc qdisc delete
4. taprio with base time in the past

At step 4, qbv_config_change_errors wrongly increased by 1.

Excerpt from IEEE 802.1Q-2018 8.6.9.3.1:
"If AdminBaseTime specifies a time in the past, and the current schedule is running, then: Increment ConfigChangeError counter"

qbv_config_change_errors should only increase if base time is in the past and no taprio is active. In user perspective, taprio was not active when first triggered at step 4. However, i225/6 reuses qbv for etf, so qbv is enabled with a dummy schedule at step 2 where it enters igc_tsn_enable_offload() and qbv_count got incremented to 1. At step 4, it enters igc_tsn_enable_offload() again, qbv_count is incremented to 2. Because taprio is running, tc_setup_type is TC_SETUP_QDISC_ETF and qbv_count > 1, qbv_config_change_errors value got incremented.

This issue happens due to reliance on qbv_count field where a non-zero value indicates that taprio is running. But qbv_count increases regardless if taprio is triggered by user or by other tsn feature. It does not align with qbv_config_change_errors expectation where it is only concerned with taprio triggered by user.

Fixing this by relocating the qbv_config_change_errors logic to igc_save_qbv_schedule(), eliminating reliance on qbv_count and its inaccuracies from i225/6's multiple uses of qbv feature for other TSN features.

The new function created: igc_tsn_is_taprio_activated_by_user() uses taprio_offload_enable field to indicate that the current running taprio was triggered by user, instead of triggered by non-qbv feature like etf.

Fixes ae4fe4698300 ("igc: Add qbv_config_change_errors counter")
---
 drivers/net/ethernet/intel/igc/igc_main.c |  8 ++++++--
 drivers/net/ethernet/intel/igc/igc_tsn.c  | 16 ++++++++--------
 drivers/net/ethernet/intel/igc/igc_tsn.h  |  1 +
 3 files changed, 15 insertions(+), 10 deletions(-)
drivers/net/ethernet/intel/igc/igc_main.c +6 -2
index 305e05294a26..0f8a5ad940ec 100644
@@ -6334,12 +6334,16 @@ static int igc_save_qbv_schedule(struct igc_adapter *adapter,
 	if (!validate_schedule(adapter, qopt))
 		return -EINVAL;
 
+	igc_ptp_read(adapter, &now);
+
+	if (igc_tsn_is_taprio_activated_by_user(adapter) &&
+	    is_base_time_past(qopt->base_time, &now))
+		adapter->qbv_config_change_errors++;
+
 	adapter->cycle_time = qopt->cycle_time;
 	adapter->base_time = qopt->base_time;
 	adapter->taprio_offload_enable = true;
 
-	igc_ptp_read(adapter, &now);
-
 	for (n = 0; n < qopt->num_entries; n++) {
 		struct tc_taprio_sched_entry *e = &qopt->entries[n];
 
drivers/net/ethernet/intel/igc/igc_tsn.c +8 -8
index 22cefb1eeedf..f6eaa288926e 100644
@@ -78,6 +78,14 @@ void igc_tsn_adjust_txtime_offset(struct igc_adapter *adapter)
 	wr32(IGC_GTXOFFSET, txoffset);
 }
 
+bool igc_tsn_is_taprio_activated_by_user(struct igc_adapter *adapter)
+{
+	struct igc_hw *hw = &adapter->hw;
+
+	return (rd32(IGC_BASET_H) || rd32(IGC_BASET_L)) &&
+		adapter->taprio_offload_enable;
+}
+
 /* Returns the TSN specific registers to their default values after
  * the adapter is reset.
  */
@@ -262,14 +270,6 @@ static int igc_tsn_enable_offload(struct igc_adapter *adapter)
 		s64 n = div64_s64(ktime_sub_ns(systim, base_time), cycle);
 
 		base_time = ktime_add_ns(base_time, (n + 1) * cycle);
-
-		/* Increase the counter if scheduling into the past while
-		 * Gate Control List (GCL) is running.
-		 */
-		if ((rd32(IGC_BASET_H) || rd32(IGC_BASET_L)) &&
-		    (adapter->tc_setup_type == TC_SETUP_QDISC_TAPRIO) &&
-		    (adapter->qbv_count > 1))
-			adapter->qbv_config_change_errors++;
 	} else {
 		if (igc_is_device_id_i226(hw)) {
 			ktime_t adjust_time, expires_time;
drivers/net/ethernet/intel/igc/igc_tsn.h +1 -0
index b53e6af560b7..98ec845a86bf 100644
@@ -7,5 +7,6 @@
 int igc_tsn_offload_apply(struct igc_adapter *adapter);
 int igc_tsn_reset(struct igc_adapter *adapter);
 void igc_tsn_adjust_txtime_offset(struct igc_adapter *adapter);
+bool igc_tsn_is_taprio_activated_by_user(struct igc_adapter *adapter);
 
 #endif /* _IGC_BASE_H */
Signature · 1 line
2.25.1
#2 lore raw

Re: [Intel-wired-lan] [PATCH iwl-net v2 1/3] igc: Fix qbv_config_change_errors logics

FYI: kernel test robot notices the stable kernel rule is not satisfied.

Hi,

Thanks for your patch.

FYI: kernel test robot notices the stable kernel rule is not satisfied.

The check is based on https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html#option-1

Rule: add the tag "Cc: [email protected]" in the sign-off area to have the patch automatically included in the stable tree.
Subject: [Intel-wired-lan] [PATCH iwl-net v2 1/3] igc: Fix qbv_config_change_errors logics

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Acked-by on v2 1/3
#3 lore raw

Re: [PATCH iwl-net v2 1/3] igc: Fix qbv_config_change_errors logics

In reply to Faizal Rahim
Quoted 34 lines · Faizal Rahim

When user issues these cmds:
1. Either a) or b)
a) mqprio with hardware offload disabled
b) taprio with txtime-assist feature enabled
2. etf
3. tc qdisc delete
4. taprio with base time in the past

At step 4, qbv_config_change_errors wrongly increased by 1.

Excerpt from IEEE 802.1Q-2018 8.6.9.3.1:
"If AdminBaseTime specifies a time in the past, and the current schedule is running, then: Increment ConfigChangeError counter"

qbv_config_change_errors should only increase if base time is in the past and no taprio is active. In user perspective, taprio was not active when first triggered at step 4. However, i225/6 reuses qbv for etf, so qbv is enabled with a dummy schedule at step 2 where it enters igc_tsn_enable_offload() and qbv_count got incremented to 1. At step 4, it enters igc_tsn_enable_offload() again, qbv_count is incremented to 2. Because taprio is running, tc_setup_type is TC_SETUP_QDISC_ETF and qbv_count > 1, qbv_config_change_errors value got incremented.

This issue happens due to reliance on qbv_count field where a non-zero value indicates that taprio is running. But qbv_count increases regardless if taprio is triggered by user or by other tsn feature. It does not align with qbv_config_change_errors expectation where it is only concerned with taprio triggered by user.

Fixing this by relocating the qbv_config_change_errors logic to igc_save_qbv_schedule(), eliminating reliance on qbv_count and its inaccuracies from i225/6's multiple uses of qbv feature for other TSN features.

Quoted 8 lines

The new function created: igc_tsn_is_taprio_activated_by_user() uses taprio_offload_enable field to indicate that the current running taprio was triggered by user, instead of triggered by non-qbv feature like etf.

Fixes ae4fe4698300 ("igc: Add qbv_config_change_errors counter")

---

Looks good:

Cheers,

Signature · 1 line
Vinicius
Tested-by on v2 1/3
#4 lore raw

Re: [Intel-wired-lan] [PATCH iwl-net v2 1/3] igc: Fix qbv_config_change_errors logics

In reply to Faizal Rahim
Quoted 42 lines · Faizal Rahim

When user issues these cmds:
1. Either a) or b)
a) mqprio with hardware offload disabled
b) taprio with txtime-assist feature enabled
2. etf
3. tc qdisc delete
4. taprio with base time in the past

At step 4, qbv_config_change_errors wrongly increased by 1.

Excerpt from IEEE 802.1Q-2018 8.6.9.3.1:
"If AdminBaseTime specifies a time in the past, and the current schedule is running, then: Increment ConfigChangeError counter"

qbv_config_change_errors should only increase if base time is in the past and no taprio is active. In user perspective, taprio was not active when first triggered at step 4. However, i225/6 reuses qbv for etf, so qbv is enabled with a dummy schedule at step 2 where it enters igc_tsn_enable_offload() and qbv_count got incremented to 1. At step 4, it enters igc_tsn_enable_offload() again, qbv_count is incremented to 2. Because taprio is running, tc_setup_type is TC_SETUP_QDISC_ETF and qbv_count > 1, qbv_config_change_errors value got incremented.

This issue happens due to reliance on qbv_count field where a non-zero value indicates that taprio is running. But qbv_count increases regardless if taprio is triggered by user or by other tsn feature. It does not align with qbv_config_change_errors expectation where it is only concerned with taprio triggered by user.

Fixing this by relocating the qbv_config_change_errors logic to igc_save_qbv_schedule(), eliminating reliance on qbv_count and its inaccuracies from i225/6's multiple uses of qbv feature for other TSN features.

The new function created: igc_tsn_is_taprio_activated_by_user() uses taprio_offload_enable field to indicate that the current running taprio was triggered by user, instead of triggered by non-qbv feature like etf.

Fixes ae4fe4698300 ("igc: Add qbv_config_change_errors counter")
Quoted 6 lines
---
  drivers/net/ethernet/intel/igc/igc_main.c |  8 ++++++--
  drivers/net/ethernet/intel/igc/igc_tsn.c  | 16 ++++++++--------
  drivers/net/ethernet/intel/igc/igc_tsn.h  |  1 +
  3 files changed, 15 insertions(+), 10 deletions(-)
#5 lore raw

PATCH v2 2/3 igc: Fix reset adapter logics when tx mode change

In reply to Faizal Rahim

Following the "igc: Fix TX Hang issue when QBV Gate is close" changes, remaining issues with the reset adapter logic in igc_tsn_offload_apply() have been observed:

1. The reset adapter logics for i225 and i226 differ, although they should be the same according to the guidelines in I225/6 HW Design Section 7.5.2.1 on software initialization during tx mode changes.
2. The i225 resets adapter every time, even though tx mode doesn't change. This occurs solely based on the condition igc_is_device_id_i225() when calling schedule_work().
3. i226 doesn't reset adapter for tsn->legacy tx mode changes. It only resets adapter for legacy->tsn tx mode transitions.
4. qbv_count introduced in the patch is actually not needed; in this context, a non-zero value of qbv_count is used to indicate if tx mode was unconditionally set to tsn in igc_tsn_enable_offload(). This could be replaced by checking the existing register IGC_TQAVCTRL_TRANSMIT_MODE_TSN bit.

This patch resolves all issues and enters schedule_work() to reset the adapter only when changing tx mode. It also removes reliance on qbv_count.

qbv_count field will be removed in a future patch.

Test ran:

1. Verify reset adapter behaviour in i225/6:
a) Enrol a new GCL

      Reset adapter observed (tx mode change legacy->tsn)
   b) Enrol a new GCL without deleting qdisc
      No reset adapter observed (tx mode remain tsn->tsn)
   c) Delete qdisc
      Reset adapter observed (tx mode change tsn->legacy)

2. Tested scenario from "igc: Fix TX Hang issue when QBV Gate is closed" to confirm it remains resolved.

Fixes 175c241288c0 ("igc: Fix TX Hang issue when QBV Gate is closed")
---
 drivers/net/ethernet/intel/igc/igc_tsn.c | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/igc/igc_tsn.c +20 -3
index f6eaa288926e..9fafe275f30f 100644
@@ -49,6 +49,13 @@ static unsigned int igc_tsn_new_flags(struct igc_adapter *adapter)
 	return new_flags;
 }
 
+static bool igc_tsn_is_tx_mode_in_tsn(struct igc_adapter *adapter)
+{
+	struct igc_hw *hw = &adapter->hw;
+
+	return !!(rd32(IGC_TQAVCTRL) & IGC_TQAVCTRL_TRANSMIT_MODE_TSN);
+}
+
 void igc_tsn_adjust_txtime_offset(struct igc_adapter *adapter)
 {
 	struct igc_hw *hw = &adapter->hw;
@@ -331,15 +338,25 @@ int igc_tsn_reset(struct igc_adapter *adapter)
 	return err;
 }
 
+static bool igc_tsn_will_tx_mode_change(struct igc_adapter *adapter)
+{
+	bool any_tsn_enabled = (bool)(igc_tsn_new_flags(adapter) &
+			       IGC_FLAG_TSN_ANY_ENABLED);
+
+	return (any_tsn_enabled && !igc_tsn_is_tx_mode_in_tsn(adapter)) ||
+	       (!any_tsn_enabled && igc_tsn_is_tx_mode_in_tsn(adapter));
+}
+
 int igc_tsn_offload_apply(struct igc_adapter *adapter)
 {
 	struct igc_hw *hw = &adapter->hw;
 
-	/* Per I225/6 HW Design Section 7.5.2.1, transmit mode
-	 * cannot be changed dynamically. Require reset the adapter.
+	/* Per I225/6 HW Design Section 7.5.2.1 guideline, if tx mode change
+	 * from legacy->tsn or tsn->legacy, then reset adapter is needed.
 	 */
 	if (netif_running(adapter->netdev) &&
-	    (igc_is_device_id_i225(hw) || !adapter->qbv_count)) {
+	    (igc_is_device_id_i225(hw) || igc_is_device_id_i226(hw)) &&
+	     igc_tsn_will_tx_mode_change(adapter)) {
 		schedule_work(&adapter->reset_task);
 		return 0;
 	}
Signature · 1 line
2.25.1
Acked-by on v2 2/3
#6 lore raw

Re: [PATCH iwl-net v2 2/3] igc: Fix reset adapter logics when tx mode change

Quoted 33 lines · Faizal Rahim

Following the "igc: Fix TX Hang issue when QBV Gate is close" changes, remaining issues with the reset adapter logic in igc_tsn_offload_apply() have been observed:

1. The reset adapter logics for i225 and i226 differ, although they should be the same according to the guidelines in I225/6 HW Design Section 7.5.2.1 on software initialization during tx mode changes.
2. The i225 resets adapter every time, even though tx mode doesn't change. This occurs solely based on the condition igc_is_device_id_i225() when calling schedule_work().
3. i226 doesn't reset adapter for tsn->legacy tx mode changes. It only resets adapter for legacy->tsn tx mode transitions.
4. qbv_count introduced in the patch is actually not needed; in this context, a non-zero value of qbv_count is used to indicate if tx mode was unconditionally set to tsn in igc_tsn_enable_offload(). This could be replaced by checking the existing register IGC_TQAVCTRL_TRANSMIT_MODE_TSN bit.

This patch resolves all issues and enters schedule_work() to reset the adapter only when changing tx mode. It also removes reliance on qbv_count.

qbv_count field will be removed in a future patch.

Test ran:

1. Verify reset adapter behaviour in i225/6:
a) Enrol a new GCL

      Reset adapter observed (tx mode change legacy->tsn)
   b) Enrol a new GCL without deleting qdisc
      No reset adapter observed (tx mode remain tsn->tsn)
   c) Delete qdisc
      Reset adapter observed (tx mode change tsn->legacy)
Quoted 7 lines

2. Tested scenario from "igc: Fix TX Hang issue when QBV Gate is closed" to confirm it remains resolved.

Fixes 175c241288c0 ("igc: Fix TX Hang issue when QBV Gate is closed")

---

There were a quite a few bugs, some of them my fault, on this part of the code, changing between the modes in the hardware.

So I would like some confirmation that ETF offloading/LaunchTime was also tested with this change. Just to be sure.

But code-wise, looks good:

Cheers,

Signature · 1 line
Vinicius
#7 lore raw

Re: [PATCH iwl-net v2 2/3] igc: Fix reset adapter logics when tx mode change

Hi Vinicius,

Quoted 49 lines · Vinicius Costa Gomes
Quoted 33 lines · Faizal Rahim

Following the "igc: Fix TX Hang issue when QBV Gate is close" changes, remaining issues with the reset adapter logic in igc_tsn_offload_apply() have been observed:

1. The reset adapter logics for i225 and i226 differ, although they should be the same according to the guidelines in I225/6 HW Design Section 7.5.2.1 on software initialization during tx mode changes.
2. The i225 resets adapter every time, even though tx mode doesn't change. This occurs solely based on the condition igc_is_device_id_i225() when calling schedule_work().
3. i226 doesn't reset adapter for tsn->legacy tx mode changes. It only

    resets adapter for legacy->tsn tx mode transitions.

4. qbv_count introduced in the patch is actually not needed; in this context, a non-zero value of qbv_count is used to indicate if tx mode was unconditionally set to tsn in igc_tsn_enable_offload(). This could be replaced by checking the existing register IGC_TQAVCTRL_TRANSMIT_MODE_TSN bit.

This patch resolves all issues and enters schedule_work() to reset the adapter only when changing tx mode. It also removes reliance on qbv_count.

qbv_count field will be removed in a future patch.

Test ran:

1. Verify reset adapter behaviour in i225/6:
a) Enrol a new GCL
Reset adapter observed (tx mode change legacy->tsn)
b) Enrol a new GCL without deleting qdisc
No reset adapter observed (tx mode remain tsn->tsn)
c) Delete qdisc
Reset adapter observed (tx mode change tsn->legacy)

Quoted 7 lines

2. Tested scenario from "igc: Fix TX Hang issue when QBV Gate is closed" to confirm it remains resolved.

Fixes 175c241288c0 ("igc: Fix TX Hang issue when QBV Gate is closed")

---

There were a quite a few bugs, some of them my fault, on this part of the code, changing between the modes in the hardware.

So I would like some confirmation that ETF offloading/LaunchTime was also tested with this change. Just to be sure.

Quoted 6 lines

But code-wise, looks good:

Cheers,

Tested etf with offload, looks like working correctly.

1. mqprio
tc qdisc add dev enp1s0 handle 100: parent root mqprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \
queues 1@0 1@1 2@2 \
hw 0

No reset adapter observed.

2. etf with offload
tc qdisc replace dev enp1s0 parent 100:1 etf \
clockid CLOCK_TAI delta 300000 offload

Reset adapter observed (tx mode legacy -> tsn).

3. delete qdisc
tc qdisc delete dev enp1s0 parent root handle 100

Reset adapter observed (tx mode tsn -> legacy).

#8 lore raw

Re: [PATCH iwl-net v2 2/3] igc: Fix reset adapter logics when tx mode change

Quoted 74 lines · Faizal

Hi Vinicius,

Quoted 49 lines · Vinicius Costa Gomes
Quoted 33 lines · Faizal Rahim

Following the "igc: Fix TX Hang issue when QBV Gate is close" changes, remaining issues with the reset adapter logic in igc_tsn_offload_apply() have been observed:

1. The reset adapter logics for i225 and i226 differ, although they should be the same according to the guidelines in I225/6 HW Design Section 7.5.2.1 on software initialization during tx mode changes.
2. The i225 resets adapter every time, even though tx mode doesn't change. This occurs solely based on the condition igc_is_device_id_i225() when calling schedule_work().
3. i226 doesn't reset adapter for tsn->legacy tx mode changes. It only

    resets adapter for legacy->tsn tx mode transitions.

4. qbv_count introduced in the patch is actually not needed; in this context, a non-zero value of qbv_count is used to indicate if tx mode was unconditionally set to tsn in igc_tsn_enable_offload(). This could be replaced by checking the existing register IGC_TQAVCTRL_TRANSMIT_MODE_TSN bit.

This patch resolves all issues and enters schedule_work() to reset the adapter only when changing tx mode. It also removes reliance on qbv_count.

qbv_count field will be removed in a future patch.

Test ran:

1. Verify reset adapter behaviour in i225/6:
a) Enrol a new GCL
Reset adapter observed (tx mode change legacy->tsn)
b) Enrol a new GCL without deleting qdisc
No reset adapter observed (tx mode remain tsn->tsn)
c) Delete qdisc
Reset adapter observed (tx mode change tsn->legacy)

Quoted 7 lines

2. Tested scenario from "igc: Fix TX Hang issue when QBV Gate is closed" to confirm it remains resolved.

Fixes 175c241288c0 ("igc: Fix TX Hang issue when QBV Gate is closed")

---

There were a quite a few bugs, some of them my fault, on this part of the code, changing between the modes in the hardware.

So I would like some confirmation that ETF offloading/LaunchTime was also tested with this change. Just to be sure.

Quoted 6 lines

But code-wise, looks good:

Cheers,

Tested etf with offload, looks like working correctly.

1. mqprio
tc qdisc add dev enp1s0 handle 100: parent root mqprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \
queues 1@0 1@1 2@2 \
hw 0

No reset adapter observed.

2. etf with offload
tc qdisc replace dev enp1s0 parent 100:1 etf \
clockid CLOCK_TAI delta 300000 offload

Quoted 7 lines

Reset adapter observed (tx mode legacy -> tsn).

3. delete qdisc
tc qdisc delete dev enp1s0 parent root handle 100

Reset adapter observed (tx mode tsn -> legacy).

That no unexpected resets are happening, is good.

But what I had in mind was some functional tests that ETF is working. I guess that's the only way of knowing that it's still working. Sorry that I wasn't clear about that.

Cheers,

Signature · 1 line
Vinicius
#9 lore raw

Re: [PATCH iwl-net v2 2/3] igc: Fix reset adapter logics when tx mode change

Quoted 84 lines · Vinicius Costa Gomes
Quoted 74 lines · Faizal

Hi Vinicius,

Quoted 49 lines · Vinicius Costa Gomes
Quoted 33 lines · Faizal Rahim

Following the "igc: Fix TX Hang issue when QBV Gate is close" changes, remaining issues with the reset adapter logic in igc_tsn_offload_apply() have been observed:

1. The reset adapter logics for i225 and i226 differ, although they should be the same according to the guidelines in I225/6 HW Design Section 7.5.2.1 on software initialization during tx mode changes.
2. The i225 resets adapter every time, even though tx mode doesn't change. This occurs solely based on the condition igc_is_device_id_i225() when calling schedule_work().
3. i226 doesn't reset adapter for tsn->legacy tx mode changes. It only

     resets adapter for legacy->tsn tx mode transitions.

4. qbv_count introduced in the patch is actually not needed; in this context, a non-zero value of qbv_count is used to indicate if tx mode was unconditionally set to tsn in igc_tsn_enable_offload(). This could be replaced by checking the existing register IGC_TQAVCTRL_TRANSMIT_MODE_TSN bit.

This patch resolves all issues and enters schedule_work() to reset the adapter only when changing tx mode. It also removes reliance on qbv_count.

qbv_count field will be removed in a future patch.

Test ran:

1. Verify reset adapter behaviour in i225/6:
a) Enrol a new GCL
Reset adapter observed (tx mode change legacy->tsn)
b) Enrol a new GCL without deleting qdisc
No reset adapter observed (tx mode remain tsn->tsn)
c) Delete qdisc
Reset adapter observed (tx mode change tsn->legacy)

Quoted 7 lines

2. Tested scenario from "igc: Fix TX Hang issue when QBV Gate is closed" to confirm it remains resolved.

Fixes 175c241288c0 ("igc: Fix TX Hang issue when QBV Gate is closed")

---

There were a quite a few bugs, some of them my fault, on this part of the code, changing between the modes in the hardware.

So I would like some confirmation that ETF offloading/LaunchTime was also tested with this change. Just to be sure.

Quoted 6 lines

But code-wise, looks good:

Cheers,

Tested etf with offload, looks like working correctly.

1. mqprio
tc qdisc add dev enp1s0 handle 100: parent root mqprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \
queues 1@0 1@1 2@2 \
hw 0

No reset adapter observed.

2. etf with offload
tc qdisc replace dev enp1s0 parent 100:1 etf \
clockid CLOCK_TAI delta 300000 offload

Quoted 7 lines

Reset adapter observed (tx mode legacy -> tsn).

3. delete qdisc
tc qdisc delete dev enp1s0 parent root handle 100

Reset adapter observed (tx mode tsn -> legacy).

Quoted 8 lines

That no unexpected resets are happening, is good.

But what I had in mind was some functional tests that ETF is working. I guess that's the only way of knowing that it's still working. Sorry that I wasn't clear about that.

Cheers,

My bad.

Just tested ETF functionality and it is working.

1. On Tx Board
a) mqprio
tc qdisc add dev enp1s0 handle 100: parent root mqprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \
queues 1@0 1@1 2@2 \
hw 0
b) etf with offload
tc qdisc replace dev enp1s0 parent 100:1 etf \
clockid CLOCK_TAI delta 300000 offload
c) use UDP TAI app to send packets where tx timestamp is set to current_time + 1ms for each packet.

2. On Rx Board
a) Checked .pcap log. Observed that interval duration between each rx packet is 1ms

Thanks for your help.

#10 lore raw

Re: [PATCH iwl-net v2 2/3] igc: Fix reset adapter logics when tx mode change

Hi,

Quoted 92 lines · Faizal
Quoted 91 lines · Vinicius Costa Gomes
Quoted 74 lines · Faizal

Hi Vinicius,

Quoted 49 lines · Vinicius Costa Gomes
Quoted 33 lines · Faizal Rahim

Following the "igc: Fix TX Hang issue when QBV Gate is close" changes, remaining issues with the reset adapter logic in igc_tsn_offload_apply() have been observed:

1. The reset adapter logics for i225 and i226 differ, although they should be the same according to the guidelines in I225/6 HW Design Section 7.5.2.1 on software initialization during tx mode changes.
2. The i225 resets adapter every time, even though tx mode doesn't change. This occurs solely based on the condition igc_is_device_id_i225() when calling schedule_work().
3. i226 doesn't reset adapter for tsn->legacy tx mode changes. It only

     resets adapter for legacy->tsn tx mode transitions.

4. qbv_count introduced in the patch is actually not needed; in this context, a non-zero value of qbv_count is used to indicate if tx mode was unconditionally set to tsn in igc_tsn_enable_offload(). This could be replaced by checking the existing register IGC_TQAVCTRL_TRANSMIT_MODE_TSN bit.

This patch resolves all issues and enters schedule_work() to reset the adapter only when changing tx mode. It also removes reliance on qbv_count.

qbv_count field will be removed in a future patch.

Test ran:

1. Verify reset adapter behaviour in i225/6:
a) Enrol a new GCL
Reset adapter observed (tx mode change legacy->tsn)
b) Enrol a new GCL without deleting qdisc
No reset adapter observed (tx mode remain tsn->tsn)
c) Delete qdisc
Reset adapter observed (tx mode change tsn->legacy)

Quoted 7 lines

2. Tested scenario from "igc: Fix TX Hang issue when QBV Gate is closed" to confirm it remains resolved.

Fixes 175c241288c0 ("igc: Fix TX Hang issue when QBV Gate is closed")

---

There were a quite a few bugs, some of them my fault, on this part of the code, changing between the modes in the hardware.

So I would like some confirmation that ETF offloading/LaunchTime was also tested with this change. Just to be sure.

Quoted 6 lines

But code-wise, looks good:

Cheers,

Tested etf with offload, looks like working correctly.

1. mqprio
tc qdisc add dev enp1s0 handle 100: parent root mqprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \
queues 1@0 1@1 2@2 \
hw 0

No reset adapter observed.

2. etf with offload
tc qdisc replace dev enp1s0 parent 100:1 etf \
clockid CLOCK_TAI delta 300000 offload

Quoted 7 lines

Reset adapter observed (tx mode legacy -> tsn).

3. delete qdisc
tc qdisc delete dev enp1s0 parent root handle 100

Reset adapter observed (tx mode tsn -> legacy).

That no unexpected resets are happening, is good.

But what I had in mind was some functional tests that ETF is working. I guess that's the only way of knowing that it's still working. Sorry that I wasn't clear about that.

Quoted 6 lines
Quoted 1 line

Cheers,

My bad.

Just tested ETF functionality and it is working.

Awesome. Thanks for the confirmation:

Cheers,

Signature · 1 line
Vinicius
#11 lore raw

RE: [Intel-wired-lan] [PATCH iwl-net v2 2/3] igc: Fix reset adapter logics when tx mode change

Quoted 106 lines · Vinicius Costa Gomes

From: Vinicius Costa Gomes <[email protected]> Hi,

Quoted 8 lines · Faizal
Quoted 7 lines · Vinicius Costa Gomes
Quoted 5 lines · Faizal

Hi Vinicius,

Quoted 2 lines · Vinicius Costa Gomes

Faizal Rahim <[email protected]> writes:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Following the "igc: Fix TX Hang issue when QBV Gate is close" changes, remaining issues with the reset adapter logic in igc_tsn_offload_apply() have been observed:

1. The reset adapter logics for i225 and i226 differ, although they should be the same according to the guidelines in I225/6 HW Design Section 7.5.2.1 on software initialization during tx mode changes.
2. The i225 resets adapter every time, even though tx mode doesn't

change.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

This occurs solely based on the condition igc_is_device_id_i225() when calling schedule_work().
3. i226 doesn't reset adapter for tsn->legacy tx mode changes. It only

     resets adapter for legacy->tsn tx mode transitions.

4. qbv_count introduced in the patch is actually not needed; in this context, a non-zero value of qbv_count is used to indicate if tx mode was unconditionally set to tsn in igc_tsn_enable_offload(). This could be replaced by checking the existing register IGC_TQAVCTRL_TRANSMIT_MODE_TSN bit.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

This patch resolves all issues and enters schedule_work() to reset the adapter only when changing tx mode. It also removes reliance on

qbv_count.

Quoted 64 lines
Quoted 64 lines
Quoted 49 lines
Quoted 27 lines
Quoted 13 lines

qbv_count field will be removed in a future patch.

Test ran:

1. Verify reset adapter behaviour in i225/6:
a) Enrol a new GCL
Reset adapter observed (tx mode change legacy->tsn)
b) Enrol a new GCL without deleting qdisc
No reset adapter observed (tx mode remain tsn->tsn)
c) Delete qdisc
Reset adapter observed (tx mode change tsn->legacy)

Quoted 7 lines

2. Tested scenario from "igc: Fix TX Hang issue when QBV Gate is closed" to confirm it remains resolved.

Fixes 175c241288c0 ("igc: Fix TX Hang issue when QBV Gate is closed")

---

There were a quite a few bugs, some of them my fault, on this part of the code, changing between the modes in the hardware.

So I would like some confirmation that ETF offloading/LaunchTime was also tested with this change. Just to be sure.

Quoted 6 lines

But code-wise, looks good:

Cheers,

Tested etf with offload, looks like working correctly.

1. mqprio
tc qdisc add dev enp1s0 handle 100: parent root mqprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \
queues 1@0 1@1 2@2 \
hw 0

No reset adapter observed.

2. etf with offload
tc qdisc replace dev enp1s0 parent 100:1 etf \
clockid CLOCK_TAI delta 300000 offload

Quoted 7 lines

Reset adapter observed (tx mode legacy -> tsn).

3. delete qdisc
tc qdisc delete dev enp1s0 parent root handle 100

Reset adapter observed (tx mode tsn -> legacy).

That no unexpected resets are happening, is good.

But what I had in mind was some functional tests that ETF is working. I guess that's the only way of knowing that it's still working. Sorry that I wasn't clear about that.

Quoted 6 lines
Quoted 1 line

Cheers,

My bad.

Just tested ETF functionality and it is working.

Quoted 8 lines

Awesome. Thanks for the confirmation:

Cheers,
--
Vinicius

Hello Vinicius and Abdul,
(I wanted to reply to the original email from Abdul, but I subscribed too late, so I'm replying to the last message. Apologies in advance)

This patch is very useful for us in L-Acoustics.
According to Milan/AVB specification, we must use CBS for streaming audio.

Until this patch, we could not change the CBS configuration at runtime for i225. For instance, adding or removing streams would cause the card to reset and drop all streams.

To be precise, we submit a netlink request via tc change command every time a stream is added/removed.

We were using a home-made patch that I planned to submit, but I forgot..

I tested this patch, and it is working as expected for CBS runtime changes. Thank you!

Tested-by on v2 2/3
#12 lore raw

Re: [Intel-wired-lan] [PATCH iwl-net v2 2/3] igc: Fix reset adapter logics when tx mode change

In reply to Faizal Rahim
Quoted 36 lines · Faizal Rahim

Following the "igc: Fix TX Hang issue when QBV Gate is close" changes, remaining issues with the reset adapter logic in igc_tsn_offload_apply() have been observed:

1. The reset adapter logics for i225 and i226 differ, although they should be the same according to the guidelines in I225/6 HW Design Section 7.5.2.1 on software initialization during tx mode changes.
2. The i225 resets adapter every time, even though tx mode doesn't change. This occurs solely based on the condition igc_is_device_id_i225() when calling schedule_work().
3. i226 doesn't reset adapter for tsn->legacy tx mode changes. It only

    resets adapter for legacy->tsn tx mode transitions.

4. qbv_count introduced in the patch is actually not needed; in this context, a non-zero value of qbv_count is used to indicate if tx mode was unconditionally set to tsn in igc_tsn_enable_offload(). This could be replaced by checking the existing register IGC_TQAVCTRL_TRANSMIT_MODE_TSN bit.

This patch resolves all issues and enters schedule_work() to reset the adapter only when changing tx mode. It also removes reliance on qbv_count.

qbv_count field will be removed in a future patch.

Test ran:

1. Verify reset adapter behaviour in i225/6:
a) Enrol a new GCL
Reset adapter observed (tx mode change legacy->tsn)
b) Enrol a new GCL without deleting qdisc
No reset adapter observed (tx mode remain tsn->tsn)
c) Delete qdisc
Reset adapter observed (tx mode change tsn->legacy)

2. Tested scenario from "igc: Fix TX Hang issue when QBV Gate is closed" to confirm it remains resolved.

Quoted 9 lines
Fixes 175c241288c0 ("igc: Fix TX Hang issue when QBV Gate is closed")
---
  drivers/net/ethernet/intel/igc/igc_tsn.c | 23 ++++++++++++++++++++---
  1 file changed, 20 insertions(+), 3 deletions(-)
#13 lore raw

PATCH v2 3/3 igc: Fix qbv tx latency by setting gtxoffset

In reply to Faizal Rahim

A large tx latency issue was discovered during testing when only QBV was enabled. The issue occurs because gtxoffset was not set when QBV is active, it was only set when launch time is active.

The patch "igc: Correct the launchtime offset" only sets gtxoffset when the launchtime_enable field is set by the user. Enabling launchtime_enable ultimately sets the register IGC_TXQCTL_QUEUE_MODE_LAUNCHT (referred to as LaunchT in the SW user manual).

Section 7.5.2.6 of the IGC i225/6 SW User Manual Rev 1.2.4 states:
"The latency between transmission scheduling (launch time) and the time the packet is transmitted to the network is listed in Table 7-61."

However, the patch misinterprets the phrase "launch time" in that section by assuming it specifically refers to the LaunchT register, whereas it actually denotes the generic term for when a packet is released from the internal buffer to the MAC transmit logic.

This launch time, as per that section, also implicitly refers to the QBV gate open time, where a packet waits in the buffer for the QBV gate to open. Therefore, latency applies whenever QBV is in use. TSN features such as QBU and QAV reuse QBV, making the latency universal to TSN features.

Discussed with i226 HW owner (Shalev, Avi) and we were in agreement that the term "launch time" used in Section 7.5.2.6 is not clear and can be easily misinterpreted. Avi will update this section to:
"When TQAVCTRL.TRANSMIT_MODE = TSN, the latency between transmission scheduling and the time the packet is transmitted to the network is listed in Table 7-61."

Fix this issue by using igc_tsn_is_tx_mode_in_tsn() as a condition to write to gtxoffset, aligning with the newly updated SW User Manual.

Tested:
1. Enrol taprio on talker board
base-time 0
cycle-time 1000000
flags 0x2
index 0 cmd S gatemask 0x1 interval1
index 0 cmd S gatemask 0x1 interval2

Note:
interval1 = interval for a 64 bytes packet to go through interval2 = cycle-time - interval1

2. Take tcpdump on listener board

3. Use udp tai app on talker to send packets to listener

4. Check the timestamp on listener via wireshark

Test Result:
100 Mbps: 113 ~193 ns
1000 Mbps: 52 ~ 84 ns
2500 Mbps: 95 ~ 223 ns

Note that the test result is similar to the patch "igc: Correct the launchtime offset".

Fixes 790835fcc0cb ("igc: Correct the launchtime offset")
---
 drivers/net/ethernet/intel/igc/igc_tsn.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/intel/igc/igc_tsn.c +1 -1
index 9fafe275f30f..efe13a9350ca 100644
@@ -61,7 +61,7 @@ void igc_tsn_adjust_txtime_offset(struct igc_adapter *adapter)
 	struct igc_hw *hw = &adapter->hw;
 	u16 txoffset;
 
-	if (!is_any_launchtime(adapter))
+	if (!igc_tsn_is_tx_mode_in_tsn(adapter))
 		return;
 
 	switch (adapter->link_speed) {
Signature · 1 line
2.25.1
Acked-by on v2 3/3
#14 lore raw

Re: [PATCH iwl-net v2 3/3] igc: Fix qbv tx latency by setting gtxoffset

Quoted 56 lines · Faizal Rahim

A large tx latency issue was discovered during testing when only QBV was enabled. The issue occurs because gtxoffset was not set when QBV is active, it was only set when launch time is active.

The patch "igc: Correct the launchtime offset" only sets gtxoffset when the launchtime_enable field is set by the user. Enabling launchtime_enable ultimately sets the register IGC_TXQCTL_QUEUE_MODE_LAUNCHT (referred to as LaunchT in the SW user manual).

Section 7.5.2.6 of the IGC i225/6 SW User Manual Rev 1.2.4 states:
"The latency between transmission scheduling (launch time) and the time the packet is transmitted to the network is listed in Table 7-61."

However, the patch misinterprets the phrase "launch time" in that section by assuming it specifically refers to the LaunchT register, whereas it actually denotes the generic term for when a packet is released from the internal buffer to the MAC transmit logic.

This launch time, as per that section, also implicitly refers to the QBV gate open time, where a packet waits in the buffer for the QBV gate to open. Therefore, latency applies whenever QBV is in use. TSN features such as QBU and QAV reuse QBV, making the latency universal to TSN features.

Discussed with i226 HW owner (Shalev, Avi) and we were in agreement that the term "launch time" used in Section 7.5.2.6 is not clear and can be easily misinterpreted. Avi will update this section to:
"When TQAVCTRL.TRANSMIT_MODE = TSN, the latency between transmission scheduling and the time the packet is transmitted to the network is listed in Table 7-61."

Fix this issue by using igc_tsn_is_tx_mode_in_tsn() as a condition to write to gtxoffset, aligning with the newly updated SW User Manual.

Tested:
1. Enrol taprio on talker board
base-time 0
cycle-time 1000000
flags 0x2
index 0 cmd S gatemask 0x1 interval1
index 0 cmd S gatemask 0x1 interval2

Note:
interval1 = interval for a 64 bytes packet to go through interval2 = cycle-time - interval1

2. Take tcpdump on listener board

3. Use udp tai app on talker to send packets to listener

4. Check the timestamp on listener via wireshark

Test Result:
100 Mbps: 113 ~193 ns
1000 Mbps: 52 ~ 84 ns
2500 Mbps: 95 ~ 223 ns

Quoted 7 lines

Note that the test result is similar to the patch "igc: Correct the launchtime offset".

Fixes 790835fcc0cb ("igc: Correct the launchtime offset")

---

Good catch.

Quoted 19 lines
 drivers/net/ethernet/intel/igc/igc_tsn.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/intel/igc/igc_tsn.c +1 -1
index 9fafe275f30f..efe13a9350ca 100644
@@ -61,7 +61,7 @@ void igc_tsn_adjust_txtime_offset(struct igc_adapter *adapter)
 	struct igc_hw *hw = &adapter->hw;
 	u16 txoffset;
 
-	if (!is_any_launchtime(adapter))
+	if (!igc_tsn_is_tx_mode_in_tsn(adapter))
 		return;
 
 	switch (adapter->link_speed) {
Signature · 1 line
2.25.1

Cheers,

Signature · 1 line
Vinicius
Tested-by on v2 3/3
#15 lore raw

Re: [Intel-wired-lan] [PATCH iwl-net v2 3/3] igc: Fix qbv tx latency by setting gtxoffset

In reply to Faizal Rahim
Quoted 59 lines · Faizal Rahim

A large tx latency issue was discovered during testing when only QBV was enabled. The issue occurs because gtxoffset was not set when QBV is active, it was only set when launch time is active.

The patch "igc: Correct the launchtime offset" only sets gtxoffset when the launchtime_enable field is set by the user. Enabling launchtime_enable ultimately sets the register IGC_TXQCTL_QUEUE_MODE_LAUNCHT (referred to as LaunchT in the SW user manual).

Section 7.5.2.6 of the IGC i225/6 SW User Manual Rev 1.2.4 states:
"The latency between transmission scheduling (launch time) and the time the packet is transmitted to the network is listed in Table 7-61."

However, the patch misinterprets the phrase "launch time" in that section by assuming it specifically refers to the LaunchT register, whereas it actually denotes the generic term for when a packet is released from the internal buffer to the MAC transmit logic.

This launch time, as per that section, also implicitly refers to the QBV gate open time, where a packet waits in the buffer for the QBV gate to open. Therefore, latency applies whenever QBV is in use. TSN features such as QBU and QAV reuse QBV, making the latency universal to TSN features.

Discussed with i226 HW owner (Shalev, Avi) and we were in agreement that the term "launch time" used in Section 7.5.2.6 is not clear and can be easily misinterpreted. Avi will update this section to:
"When TQAVCTRL.TRANSMIT_MODE = TSN, the latency between transmission scheduling and the time the packet is transmitted to the network is listed in Table 7-61."

Fix this issue by using igc_tsn_is_tx_mode_in_tsn() as a condition to write to gtxoffset, aligning with the newly updated SW User Manual.

Tested:
1. Enrol taprio on talker board

    base-time 0
    cycle-time 1000000
    flags 0x2
    index 0 cmd S gatemask 0x1 interval1
    index 0 cmd S gatemask 0x1 interval2

    Note:
    interval1 = interval for a 64 bytes packet to go through
    interval2 = cycle-time - interval1

2. Take tcpdump on listener board

3. Use udp tai app on talker to send packets to listener

4. Check the timestamp on listener via wireshark

Test Result:
100 Mbps: 113 ~193 ns
1000 Mbps: 52 ~ 84 ns
2500 Mbps: 95 ~ 223 ns

Note that the test result is similar to the patch "igc: Correct the launchtime offset".

Quoted 8 lines
Fixes 790835fcc0cb ("igc: Correct the launchtime offset")
---
  drivers/net/ethernet/intel/igc/igc_tsn.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)