Skip to contentSkip to conversation

More on ath10k_flush.

DISCUSSION Ben Greear 2 participants 4 messages Apr 1, 2014 – Apr 2, 2014 Qualcomm Atheros ATH10K wireless driver

lore
Share & tools
lore

Ben Greear

author
#0 lore raw

So, I tried adding lots of debug to the ath10k_flush, and I think I might have found the issue.

My test case is: create 32 stations, start TCP on all of them, then reset (effectively ifdown;ifup) the station vifs.

I get lots of spews because the ath10k_flush call thinks it is hanging. But, based on the debug, I think the problem is that the tx logic is continuing to accept packets while the flush is trying to happen.

If my debugging stats are correct, the NIC managed to send around 4000 frames during the flush attempt, and the ath10k driver processed lots of tx completions as well. But there are still lots of pkts pending tx and the tx queue was never empty.

ath10k: failed to flush transmit queue (skip 0 ar-state 1 pending_tx 1009 pre-pending-tx: 1045): 0
ath10k: pre: htc-send-tot: 3423 htt-tx 2545397 tx-compl 2544355 mgt-tx 0 mgt-compl 0
ath10k: post: htc-send-tot: 3424 htt-tx 2630981 tx-compl 2630156 mgt-tx 0 mgt-compl 0
ath10k: pre: hw-queued: 124395 hw-reaped: 124394
ath10k: post: hw-queued: 128418 hw-reaped: 128417

Do we need to somehow shut down all tx logic during the ath10k_flush so that (other?) stations cannot transmit?

Thanks,
Ben

Signature · 8 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#1 lore raw
Quoted 16 lines · Ben Greear

So, I tried adding lots of debug to the ath10k_flush, and I think I might have found the issue.

My test case is: create 32 stations, start TCP on all of them, then reset (effectively ifdown;ifup) the station vifs.

I get lots of spews because the ath10k_flush call thinks it is hanging. But, based on the debug, I think the problem is that the tx logic is continuing to accept packets while the flush is trying to happen.

If my debugging stats are correct, the NIC managed to send around 4000 frames during the flush attempt, and the ath10k driver processed lots of tx completions as well. But there are still lots of pkts pending tx and the tx queue was never empty.

Quoted 5 lines

ath10k: failed to flush transmit queue (skip 0 ar-state 1 pending_tx 1009 pre-pending-tx: 1045): 0
ath10k: pre: htc-send-tot: 3423 htt-tx 2545397 tx-compl 2544355 mgt-tx 0 mgt-compl 0
ath10k: post: htc-send-tot: 3424 htt-tx 2630981 tx-compl 2630156 mgt-tx 0 mgt-compl 0
ath10k: pre: hw-queued: 124395 hw-reaped: 124394
ath10k: post: hw-queued: 128418 hw-reaped: 128417

What do these prints reflect exactly? What base upstream ath10k commit are you on?

Quoted 2 lines

Do we need to somehow shut down all tx logic during the ath10k_flush so that (other?) stations cannot transmit?

This seems strange. drv_flush() is called with all tx queues being stopped in mac80211 with IEEE80211_QUEUE_STOP_REASON_FLUSH. I guess you can still receive a few frames (due to SMP with many interfaces), but that's it (at most num_cpus-1 I guess). You shouldn't be seeing 4000 frames being queued while you flush.

Michał

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

author
#2 lore raw
Quoted 27 lines · Michal Kazior
Quoted 16 lines · Ben Greear

So, I tried adding lots of debug to the ath10k_flush, and I think I might have found the issue.

My test case is: create 32 stations, start TCP on all of them, then reset (effectively ifdown;ifup) the station vifs.

I get lots of spews because the ath10k_flush call thinks it is hanging. But, based on the debug, I think the problem is that the tx logic is continuing to accept packets while the flush is trying to happen.

If my debugging stats are correct, the NIC managed to send around 4000 frames during the flush attempt, and the ath10k driver processed lots of tx completions as well. But there are still lots of pkts pending tx and the tx queue was never empty.

Quoted 5 lines

ath10k: failed to flush transmit queue (skip 0 ar-state 1 pending_tx 1009 pre-pending-tx: 1045): 0
ath10k: pre: htc-send-tot: 3423 htt-tx 2545397 tx-compl 2544355 mgt-tx 0 mgt-compl 0
ath10k: post: htc-send-tot: 3424 htt-tx 2630981 tx-compl 2630156 mgt-tx 0 mgt-compl 0
ath10k: pre: hw-queued: 124395 hw-reaped: 124394
ath10k: post: hw-queued: 128418 hw-reaped: 128417

What do these prints reflect exactly? What base upstream ath10k commit are you on?

Quoted 8 lines
Quoted 2 lines

Do we need to somehow shut down all tx logic during the ath10k_flush so that (other?) stations cannot transmit?

This seems strange. drv_flush() is called with all tx queues being stopped in mac80211 with IEEE80211_QUEUE_STOP_REASON_FLUSH. I guess you can still receive a few frames (due to SMP with many interfaces), but that's it (at most num_cpus-1 I guess). You shouldn't be seeing 4000 frames being queued while you flush.

Subsequent debugging makes me think the extra packets are not coming from the upper stack. Possibly there are ath10k<->firmware management messages being queued or something like that?

I'm out of office today, will post the debug patch when I get a chance.

Thanks,
Ben

Quoted 4 lines

Michał

Signature · 8 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#3 lore raw
Quoted 2 lines · Ben Greear
Quoted 1 line · Michal Kazior

On 1 April 2014 02:24, Ben Greear <[email protected]> wrote:

[...]

Quoted 11 lines
Quoted 9 lines
Quoted 2 lines

Do we need to somehow shut down all tx logic during the ath10k_flush so that (other?) stations cannot transmit?

This seems strange. drv_flush() is called with all tx queues being stopped in mac80211 with IEEE80211_QUEUE_STOP_REASON_FLUSH. I guess you can still receive a few frames (due to SMP with many interfaces), but that's it (at most num_cpus-1 I guess). You shouldn't be seeing 4000 frames being queued while you flush.

Quoted 4 lines

Subsequent debugging makes me think the extra packets are not coming from the upper stack. Possibly there are ath10k<->firmware management messages
being queued or something like that?

ath10k_flush() waits only for HTT Tx. It doesn't even wait for management Tx (it probably should..).

There's a limit to the number of outstanding HTT Tx requests - it is 1424 (+n_cpus - 1). So it is possible to see at most this much frames being completed while waiting for a flush.

Michał

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k