Skip to contentSkip to conversation

Reporting firmware stats to ethtool

DISCUSSION Ben Greear 4 participants 12 messages Aug 7, 2014 – Aug 9, 2014 Qualcomm Atheros ATH10K wireless driver

lore
Share & tools
lore

Ben Greear

author
#0 lore raw

I'm working on a patch to report the stats seen in debugfs/...ath10k/fw_stats as ethtool stats, somewhat similar to how ath9k does it.

I notice that my user-space tool is reporting huge numbers because the stats are reset to zero when firmware restarts, and so my tool thinks the stats wrapped.

I can fix my tool easily enough, but I first wanted to see if anyone had strong feelings about keeping the stats from resetting to zero by storing history and calculating diffs in the driver.

I think my preference is to punt this to user-space, but if someone feels otherwise, please let me know sooner than later.

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#1 lore raw
Quoted 7 lines · Ben Greear

I'm working on a patch to report the stats seen in debugfs/...ath10k/fw_stats as ethtool stats, somewhat similar to how ath9k does it.

I notice that my user-space tool is reporting huge numbers because the stats are reset to zero when firmware restarts, and so my tool thinks the stats wrapped.

Quoted 6 lines

I can fix my tool easily enough, but I first wanted to see if anyone had strong feelings about keeping the stats from resetting to zero by storing history and calculating diffs in the driver.

I think my preference is to punt this to user-space, but if someone feels otherwise, please let me know sooner than later.

I also prefer to have this in user space, but how does user space know when the stats have been zeroed?

Signature · 1 line
Kalle Valo
author
#2 lore raw
Quoted 13 lines · Kalle Valo
Quoted 11 lines · Ben Greear

I'm working on a patch to report the stats seen in debugfs/...ath10k/fw_stats as ethtool stats, somewhat similar to how ath9k does it.

I notice that my user-space tool is reporting huge numbers because the stats are reset to zero when firmware restarts, and so my tool thinks the stats wrapped.

I can fix my tool easily enough, but I first wanted to see if anyone had strong feelings about keeping the stats from resetting to zero by storing history and calculating diffs in the driver.

Quoted 5 lines
Quoted 2 lines

I think my preference is to punt this to user-space, but if someone feels otherwise, please let me know sooner than later.

I also prefer to have this in user space, but how does user space know when the stats have been zeroed?

Poll often enough that it cannot increment more than 2 billion (or other large number) between polls, and then if polled value is less than previous (and difference is > 2 billion), we know we had a reset and not a wrap.

User-space stats will not be perfect in the case of firmware resets, or resets after the 'large number', but nothing is going to make it perfect, and in practice, this seems good enough.

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#3 lore raw
Quoted 17 lines · Ben Greear
Quoted 16 lines · Kalle Valo
Quoted 13 lines · Ben Greear

I'm working on a patch to report the stats seen in debugfs/...ath10k/fw_stats as ethtool stats, somewhat similar to how ath9k does it.

I notice that my user-space tool is reporting huge numbers because the stats are reset to zero when firmware restarts, and so my tool thinks the stats wrapped.

I can fix my tool easily enough, but I first wanted to see if anyone had strong feelings about keeping the stats from resetting to zero by storing history and calculating diffs in the driver.

I think my preference is to punt this to user-space, but if someone feels otherwise, please let me know sooner than later.

Quoted 6 lines
Quoted 2 lines

I also prefer to have this in user space, but how does user space know when the stats have been zeroed?

Poll often enough that it cannot increment more than 2 billion (or other large number) between polls, and then if polled value is less than previous (and difference is > 2 billion), we know we had a reset and not a wrap.

I do kind of prefer 64 bit counters in the general case. Nuke it from orbit, it's the only way to be sure.

Quoted 16 lines

User-space stats will not be perfect in the case of firmware resets, or resets after the 'large number', but nothing is going to make it perfect, and in practice, this seems good enough.

Thanks,
Ben

--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

Signature · 3 lines
Dave Täht

NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
author
#4 lore raw
Quoted 21 lines · Dave Taht
Quoted 20 lines · Ben Greear
Quoted 18 lines · Kalle Valo
Quoted 7 lines · Ben Greear

I'm working on a patch to report the stats seen in debugfs/...ath10k/fw_stats as ethtool stats, somewhat similar to how ath9k does it.

I notice that my user-space tool is reporting huge numbers because the stats are reset to zero when firmware restarts, and so my tool thinks the stats wrapped.

Quoted 6 lines

I can fix my tool easily enough, but I first wanted to see if anyone had strong feelings about keeping the stats from resetting to zero by storing history and calculating diffs in the driver.

I think my preference is to punt this to user-space, but if someone feels otherwise, please let me know sooner than later.

I also prefer to have this in user space, but how does user space know when the stats have been zeroed?

Quoted 6 lines
Quoted 3 lines

Poll often enough that it cannot increment more than 2 billion (or other large number) between polls, and then if polled value is less than previous (and difference is > 2 billion), we know we had a reset and not a wrap.

I do kind of prefer 64 bit counters in the general case. Nuke it from orbit, it's the only way to be sure.

It's 64-bit to user-space, but that means nothing because firmware uses 32-bit (or even 16 bit in some cases, probably) internally. A great deal of counters are the same, so be very careful when trying to keep long term counters grabbed from firmware/drivers/hardware.

And, stations come and go when you re-associate, so all sorts of wifi counters reset themselves all the time...

Thanks,
Ben

Quoted 20 lines
Quoted 17 lines

User-space stats will not be perfect in the case of firmware resets, or resets after the 'large number', but nothing is going to make it perfect, and in practice, this seems good enough.

Thanks,
Ben

--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#5 lore raw
Quoted 29 lines · Ben Greear
Quoted 21 lines · Dave Taht
Quoted 20 lines · Ben Greear
Quoted 18 lines · Kalle Valo
Quoted 7 lines · Ben Greear

I'm working on a patch to report the stats seen in debugfs/...ath10k/fw_stats as ethtool stats, somewhat similar to how ath9k does it.

I notice that my user-space tool is reporting huge numbers because the stats are reset to zero when firmware restarts, and so my tool thinks the stats wrapped.

Quoted 6 lines

I can fix my tool easily enough, but I first wanted to see if anyone had strong feelings about keeping the stats from resetting to zero by storing history and calculating diffs in the driver.

I think my preference is to punt this to user-space, but if someone feels otherwise, please let me know sooner than later.

I also prefer to have this in user space, but how does user space know when the stats have been zeroed?

Quoted 6 lines
Quoted 3 lines

Poll often enough that it cannot increment more than 2 billion (or other large number) between polls, and then if polled value is less than previous (and difference is > 2 billion), we know we had a reset and not a wrap.

I do kind of prefer 64 bit counters in the general case. Nuke it from orbit, it's the only way to be sure.

Quoted 7 lines

It's 64-bit to user-space, but that means nothing because firmware uses 32-bit (or even 16 bit in some cases, probably) internally. A great deal of counters are the same, so be very careful when trying to keep long term counters grabbed from firmware/drivers/hardware.

And, stations come and go when you re-associate, so all sorts of wifi counters reset themselves all the time...

Does ath driver notify mac80211 about firmware restart, ie. through ieee80211_restart_hw(). If only user-space could get that info.

Gr. AvS

Quoted 25 lines

Thanks,
Ben

Quoted 20 lines
Quoted 17 lines

User-space stats will not be perfect in the case of firmware resets, or resets after the 'large number', but nothing is going to make it perfect, and in practice, this seems good enough.

Thanks,
Ben

--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

author
#6 lore raw
Quoted 35 lines · Arend van Spriel
Quoted 34 lines · Ben Greear
Quoted 21 lines · Dave Taht
Quoted 20 lines · Ben Greear
Quoted 18 lines · Kalle Valo
Quoted 7 lines · Ben Greear

I'm working on a patch to report the stats seen in debugfs/...ath10k/fw_stats as ethtool stats, somewhat similar to how ath9k does it.

I notice that my user-space tool is reporting huge numbers because the stats are reset to zero when firmware restarts, and so my tool thinks the stats wrapped.

Quoted 6 lines

I can fix my tool easily enough, but I first wanted to see if anyone had strong feelings about keeping the stats from resetting to zero by storing history and calculating diffs in the driver.

I think my preference is to punt this to user-space, but if someone feels otherwise, please let me know sooner than later.

I also prefer to have this in user space, but how does user space know when the stats have been zeroed?

Quoted 6 lines
Quoted 3 lines

Poll often enough that it cannot increment more than 2 billion (or other large number) between polls, and then if polled value is less than previous (and difference is > 2 billion), we know we had a reset and not a wrap.

I do kind of prefer 64 bit counters in the general case. Nuke it from orbit, it's the only way to be sure.

It's 64-bit to user-space, but that means nothing because firmware uses 32-bit (or even 16 bit in some cases, probably) internally. A great deal of counters are the same, so be very careful when trying to keep long term counters grabbed from firmware/drivers/hardware.

Quoted 4 lines
Quoted 2 lines

And, stations come and go when you re-associate, so all sorts of wifi counters reset themselves all the time...

Does ath driver notify mac80211 about firmware restart, ie. through ieee80211_restart_hw(). If only user-space could get that info.

We should soon have a udev event on firmware crash (and a way to read that crash log out of debugfs). A simple restart (w/out crash) might not be so interesting?

Might be nice to have a fw restart and fw crash counter in ethtool stats in case something wants to poll that instead of dealing with uevents. I will add those counters.

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#7 lore raw
Quoted 4 lines · Ben Greear
Quoted 2 lines

Does ath driver notify mac80211 about firmware restart, ie. through ieee80211_restart_hw(). If only user-space could get that info.

We should soon have a udev event on firmware crash

Who should send the event, the driver or mac80211? I'm not very familiar with udev stuff, what kind of rules there are for udev events?

Quoted 1 line

A simple restart (w/out crash) might not be so interesting?

Yeah, we should not report that. ath10k stops the firmware on interface down and starts it again on interface up which is business as usual.

Quoted 3 lines

Might be nice to have a fw restart and fw crash counter in ethtool stats in case something wants to poll that instead of dealing with uevents. I will add those counters.

Yeah, those would be great to have.

Signature · 6 lines
Kalle Valo

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#8 lore raw
In reply to Arend van Spriel
Quoted 8 lines · Arend van Spriel
Quoted 8 lines
Quoted 2 lines

I do kind of prefer 64 bit counters in the general case. Nuke it from orbit, it's the only way to be sure.

It's 64-bit to user-space, but that means nothing because firmware uses 32-bit (or even 16 bit in some cases, probably) internally. A great deal of counters are the same, so be very careful when trying to keep long term counters grabbed from firmware/drivers/hardware.

Quoted 5 lines
Quoted 2 lines

And, stations come and go when you re-associate, so all sorts of wifi counters reset themselves all the time...

Does ath driver notify mac80211 about firmware restart, ie. through ieee80211_restart_hw().

ath10k does use ieee80211_restart_hw().

Quoted 1 line

If only user-space could get that info.

Yeah, that would be nice to have for ath10k firmware crash dump functionality. And doesn't Android also need something similar?

Signature · 1 line
Kalle Valo
#9 lore raw
Quoted 16 lines · Kalle Valo
Quoted 8 lines · Arend van Spriel
Quoted 8 lines
Quoted 2 lines

I do kind of prefer 64 bit counters in the general case. Nuke it from orbit, it's the only way to be sure.

It's 64-bit to user-space, but that means nothing because firmware uses 32-bit (or even 16 bit in some cases, probably) internally. A great deal of counters are the same, so be very careful when trying to keep long term counters grabbed from firmware/drivers/hardware.

Quoted 5 lines
Quoted 2 lines

And, stations come and go when you re-associate, so all sorts of wifi counters reset themselves all the time...

Does ath driver notify mac80211 about firmware restart, ie. through ieee80211_restart_hw().

Quoted 6 lines

ath10k does use ieee80211_restart_hw().

Quoted 1 line

If only user-space could get that info.

Yeah, that would be nice to have for ath10k firmware crash dump functionality. And doesn't Android also need something similar?

Probably. bcmdhd seems to send a "firmware hang" event up to wpa_supp, which probably ends up in the android wifi framework through the control interface. Currently, this is a driver private event handled by wpa_supplicant_lib, but it seems trivial to me to add a nl80211 event to trigger that.

I am not sure what infrastructure your "ath10k firmware crash dump" is going to use. I have seen similar thing from Marvell recently [1] which relies on udev and ethtool to do the work. I guess aligning the solutions is why this topic is listed for the wireless breakout session at kernel summit in Chicago.

Regards,
Arend

[1] http://www.spinics.net/lists/linux-wireless/msg123943.html

#10 lore raw
Quoted 10 lines · Arend van Spriel
Quoted 9 lines
Quoted 2 lines

Does ath driver notify mac80211 about firmware restart, ie. through ieee80211_restart_hw().

ath10k does use ieee80211_restart_hw().

Quoted 1 line

If only user-space could get that info.

Yeah, that would be nice to have for ath10k firmware crash dump functionality. And doesn't Android also need something similar?

Quoted 8 lines

Probably. bcmdhd seems to send a "firmware hang" event up to wpa_supp, which probably ends up in the android wifi framework through the control interface. Currently, this is a driver private event handled by wpa_supplicant_lib, but it seems trivial to me to add a nl80211 event to trigger that.

I am not sure what infrastructure your "ath10k firmware crash dump" is going to use.

Currently we are planning to just provide a debugfs file to retrieve the latest dump.

Quoted 4 lines

I have seen similar thing from Marvell recently [1] which relies on udev and ethtool to do the work. I guess aligning the solutions is why this topic is listed for the wireless breakout session at kernel summit in Chicago.

Yeah, we did talk about that a bit in the mailing list few months back.

Signature · 1 line
Kalle Valo
#11 lore raw
In reply to Ben Greear
Quoted 17 lines · Ben Greear
Quoted 16 lines · Kalle Valo
Quoted 13 lines · Ben Greear

I'm working on a patch to report the stats seen in debugfs/...ath10k/fw_stats as ethtool stats, somewhat similar to how ath9k does it.

I notice that my user-space tool is reporting huge numbers because the stats are reset to zero when firmware restarts, and so my tool thinks the stats wrapped.

I can fix my tool easily enough, but I first wanted to see if anyone had strong feelings about keeping the stats from resetting to zero by storing history and calculating diffs in the driver.

I think my preference is to punt this to user-space, but if someone feels otherwise, please let me know sooner than later.

Quoted 7 lines
Quoted 2 lines

I also prefer to have this in user space, but how does user space know when the stats have been zeroed?

Poll often enough that it cannot increment more than 2 billion (or other large number) between polls, and then if polled value is less than previous (and difference is > 2 billion), we know we had a reset and not a wrap.

That's not very reliable as "less than previous" assumption is not always true.

Quoted 3 lines

User-space stats will not be perfect in the case of firmware resets, or resets after the 'large number', but nothing is going to make it perfect, and in practice, this seems good enough.

Yeah, and we can always fix this later if there's a sudden need to get reliable numbers.

Signature · 1 line
Kalle Valo