Skip to contentSkip to conversation

htt_rx questions in 4.0 kernel.

DISCUSSION Ben Greear 3 participants 9 messages May 5, 2015 – May 7, 2015 Qualcomm Atheros ATH10K wireless driver

lore
Share & tools
lore

Ben Greear

author
#0 lore raw

I am slowly tracking down problems with receiving data frames on 3.19 and higher kernels, when using my rx-sw-crypt patch and CT firmware.

I think the main thing my patch is changing in this case is that the frames are received in raw mode with this patch enabled.

Hard to do a bisect, because the rx logic changed so much and I have to apply fairly tricky patches to make my feature work, but 3.18 and earlier seem to work fine, and 3.19 and 4.0 does not.

Latest clue appears to be that ARP frames are received fine (data contents looks right), but the stack fails the is_data() check.

While poking at this further, I noticed that ath10k_htt_rx_h_undecap defines 'hdr' and assigns it, but never uses it.

I also do not see the first_hdr used when decapping raw frames.

Maybe that is part of the problem?

Thanks,
Ben

Signature · 8 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#1 lore raw
Quoted 11 lines · Ben Greear

I am slowly tracking down problems with receiving data frames on 3.19 and higher kernels, when using my rx-sw-crypt patch and CT firmware.

I think the main thing my patch is changing in this case is that the frames are received in raw mode with this patch enabled.

Hard to do a bisect, because the rx logic changed so much and I have to apply fairly tricky patches to make my feature work, but 3.18 and earlier seem to work fine, and 3.19 and 4.0 does not.

Quoted 5 lines

Latest clue appears to be that ARP frames are received fine (data contents looks right), but the stack fails the is_data() check.

While poking at this further, I noticed that ath10k_htt_rx_h_undecap defines 'hdr' and assigns it, but never uses it.

It does use it:

https://github.com/kvalo/ath/blob/master/drivers/net/wireless/ath/ath10k/htt_rx.c#L1028
https://github.com/kvalo/ath/blob/master/drivers/net/wireless/ath/ath10k/htt_rx.c#L1034
https://github.com/kvalo/ath/blob/master/drivers/net/wireless/ath/ath10k/htt_rx.c#L1039

Quoted 2 lines

I also do not see the first_hdr used when decapping raw frames.

first_hdr isn't necessary for raw frame processing. Let me explain why:

When you receive A-MSDU you have two options: you either get it raw, or non-raw.

When you get non-raw each subframe is delivered in a separate MSDU buffer on HTT. Each contains an rx descriptor. However only first MSDU in that list contains a complete decapped 802.11 header info. That's why first_hdr is needed.

When you get raw there's no MSDU-based splitting. If a raw frame doesn't fit into a single HTT buffer you get chained MSDUs. In that case only first buffer contains Rx descriptor. Others are just data continuation and have no Rx descriptor.

Ergo first_hdr doesn't make sense for raw as it is the hdr anyway.

Quoted 1 line

Maybe that is part of the problem?

Hmm..

Did you try to compare ARP hexdump when it is popped from HTT Rx ring and when it is given to mac80211? Is the protected-bit set correctly? Are Rx flags set correctly so mac80211 treats the frame properly? Maybe the way you're hooking your swcrypto is not correct anymore, i.e. there's an implicit assumption in your swcrypto logic which isn't true anymore?

Michał

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

author
#2 lore raw
Quoted 15 lines · Michal Kazior
Quoted 14 lines · Ben Greear

I am slowly tracking down problems with receiving data frames on 3.19 and higher kernels, when using my rx-sw-crypt patch and CT firmware.

I think the main thing my patch is changing in this case is that the frames are received in raw mode with this patch enabled.

Hard to do a bisect, because the rx logic changed so much and I have to apply fairly tricky patches to make my feature work, but 3.18 and earlier seem to work fine, and 3.19 and 4.0 does not.

Latest clue appears to be that ARP frames are received fine (data contents looks right), but the stack fails the is_data() check.

Quoted 8 lines
Quoted 2 lines

While poking at this further, I noticed that ath10k_htt_rx_h_undecap defines 'hdr' and assigns it, but never uses it.

It does use it:

https://github.com/kvalo/ath/blob/master/drivers/net/wireless/ath/ath10k/htt_rx.c#L1028
https://github.com/kvalo/ath/blob/master/drivers/net/wireless/ath/ath10k/htt_rx.c#L1034
https://github.com/kvalo/ath/blob/master/drivers/net/wireless/ath/ath10k/htt_rx.c#L1039

I am talking about the ath10k_htt_rx_h_undecap() method.

Quoted 24 lines
Quoted 4 lines

I also do not see the first_hdr used when decapping raw frames.

first_hdr isn't necessary for raw frame processing. Let me explain why:

When you receive A-MSDU you have two options: you either get it raw, or non-raw.

When you get non-raw each subframe is delivered in a separate MSDU buffer on HTT. Each contains an rx descriptor. However only first MSDU in that list contains a complete decapped 802.11 header info. That's why first_hdr is needed.

When you get raw there's no MSDU-based splitting. If a raw frame doesn't fit into a single HTT buffer you get chained MSDUs. In that case only first buffer contains Rx descriptor. Others are just data continuation and have no Rx descriptor.

Ergo first_hdr doesn't make sense for raw as it is the hdr anyway.

Quoted 1 line

Maybe that is part of the problem?

Quoted 8 lines

Hmm..

Did you try to compare ARP hexdump when it is popped from HTT Rx ring and when it is given to mac80211? Is the protected-bit set correctly? Are Rx flags set correctly so mac80211 treats the frame properly? Maybe the way you're hooking your swcrypto is not correct anymore, i.e. there's an implicit assumption in your swcrypto logic which isn't true anymore?

I see same problem with OPEN frames, no encryption needed, and from what I can tell, the driver receives the exact same ARP data that the sniffer sees (and which wireshark shows as legit).

Lots of frames work, but also many have a bit or two flipped in the first bit of the packet, and sometimes it gets into this state where it appears the header is invalid or maybe stale.

It seems when the stale header problem happens, the receiver stays stuck for often quite a while (many seconds), but then later it will start working again.

I'll keep debugging...

Thanks,
Ben

Quoted 4 lines

Michał

Signature · 7 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
author
#3 lore raw
Quoted 67 lines · Ben Greear
Quoted 15 lines · Michal Kazior
Quoted 14 lines · Ben Greear

I am slowly tracking down problems with receiving data frames on 3.19 and higher kernels, when using my rx-sw-crypt patch and CT firmware.

I think the main thing my patch is changing in this case is that the frames are received in raw mode with this patch enabled.

Hard to do a bisect, because the rx logic changed so much and I have to apply fairly tricky patches to make my feature work, but 3.18 and earlier seem to work fine, and 3.19 and 4.0 does not.

Latest clue appears to be that ARP frames are received fine (data contents looks right), but the stack fails the is_data() check.

Quoted 8 lines
Quoted 2 lines

While poking at this further, I noticed that ath10k_htt_rx_h_undecap defines 'hdr' and assigns it, but never uses it.

It does use it:

https://github.com/kvalo/ath/blob/master/drivers/net/wireless/ath/ath10k/htt_rx.c#L1028
https://github.com/kvalo/ath/blob/master/drivers/net/wireless/ath/ath10k/htt_rx.c#L1034
https://github.com/kvalo/ath/blob/master/drivers/net/wireless/ath/ath10k/htt_rx.c#L1039

I am talking about the ath10k_htt_rx_h_undecap() method.

Quoted 24 lines
Quoted 4 lines

I also do not see the first_hdr used when decapping raw frames.

first_hdr isn't necessary for raw frame processing. Let me explain why:

When you receive A-MSDU you have two options: you either get it raw, or non-raw.

When you get non-raw each subframe is delivered in a separate MSDU buffer on HTT. Each contains an rx descriptor. However only first MSDU in that list contains a complete decapped 802.11 header info. That's why first_hdr is needed.

When you get raw there's no MSDU-based splitting. If a raw frame doesn't fit into a single HTT buffer you get chained MSDUs. In that case only first buffer contains Rx descriptor. Others are just data continuation and have no Rx descriptor.

Ergo first_hdr doesn't make sense for raw as it is the hdr anyway.

Quoted 1 line

Maybe that is part of the problem?

Quoted 8 lines

Hmm..

Did you try to compare ARP hexdump when it is popped from HTT Rx ring and when it is given to mac80211? Is the protected-bit set correctly? Are Rx flags set correctly so mac80211 treats the frame properly? Maybe the way you're hooking your swcrypto is not correct anymore, i.e. there's an implicit assumption in your swcrypto logic which isn't true anymore?

I see same problem with OPEN frames, no encryption needed, and from what I can tell, the driver receives the exact same ARP data that the sniffer sees (and which wireshark shows as legit).

Quoted 8 lines

Lots of frames work, but also many have a bit or two flipped in the first bit of the packet, and sometimes it gets into this state where it appears the header is invalid or maybe stale.

It seems when the stale header problem happens, the receiver stays stuck for often quite a while (many seconds), but then later it will start working again.

I'll keep debugging...

So, it looks like my problem is that I was passing frames with CRC errors up the stack (and flagging them as not having CRC errors).

Looks like that can cause some fairly strange behaviour.

With that fixed (to properly flag pkts as having FCS errors), then it is running much better. The nice part is that I do not need to make any hacks to the rx logic now...it 'just works'. Only hack now is to configure my firmware properly to disable any attempt at decrypting rx'd pkts (and put it into raw-rx mode).

Thanks,
Ben

Signature · 8 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#4 lore raw

changing hw address of ath10k card

In reply to Ben Greear

Hello

I recently stumbled across a problem in ath10k. while developing on embedded wireless routers
i found a device which uses a ar988x card using a card embedded board eeprom. the problem is there is no way to override the cards hardware/mac address, which is required on that device since it uses just a fake mac on that board data. is there any way/trick available to deal with that problem?

Sebastian

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

#5 lore raw

Re: changing hw address of ath10k card

Quoted 1 line · Sebastian Gottschall

Hello

Hi,

Quoted 6 lines

I recently stumbled across a problem in ath10k. while developing on embedded wireless routers
i found a device which uses a ar988x card using a card embedded board eeprom. the problem is there is no way to override
the cards hardware/mac address, which is required on that device since it uses just a fake mac on that board data.

If device contains valid EEPROM data then both calibration and mac address is read using OTP. In that case a mere fake board template binary is used by the driver (the default board.bin).

If device contains no valid EEPROM, which is the case for many routers out in the wild e.g. TP-Link Archer C5/C7, you must provide a complete board data. On routers this data is often stored on NAND/Flash partitions. OpenWRT does handle this and extracts calibration data file. If you fail to provide that and attempt to use the default board.bin you'll get hardcoded mac and Tx/Rx performance will be very poor most likely.

If ath10k assigns 00:03:07:12:34:56 mac address to the interface then you've most likely provided a fake board template and the device doesn't have a valid EEPROM data. Are you using skip_otp module parameter by any chance or other hacks/custom patches on ath10k?

Quoted 1 line

is there any way/trick available to deal with that problem?

You can always change the mac address via ip or ifconfig commands after interface is created but that's probably not what you really want.

Michał

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

#6 lore raw

Re: changing hw address of ath10k card

Am 06.05.2015 um 13:06 schrieb Michal Kazior:

Quoted 11 lines · Sebastian Gottschall
Quoted 1 line

Hello

Hi,

Quoted 6 lines

I recently stumbled across a problem in ath10k. while developing on embedded wireless routers
i found a device which uses a ar988x card using a card embedded board eeprom. the problem is there is no way to override
the cards hardware/mac address, which is required on that device since it uses just a fake mac on that board data.

Quoted 3 lines

If device contains valid EEPROM data then both calibration and mac address is read using OTP. In that case a mere fake board template binary is used by the driver (the default board.bin).

the problem is that the board.bin is never used. i modified the board.bin to change the mac address, but it shows no effect

Quoted 8 lines

If device contains no valid EEPROM, which is the case for many routers out in the wild e.g. TP-Link Archer C5/C7, you must provide a complete board data. On routers this data is often stored on NAND/Flash partitions. OpenWRT does handle this and extracts calibration data file. If you fail to provide that and attempt to use the default board.bin you'll get hardcoded mac and Tx/Rx performance will be very poor most likely.

i'm aware of that. i handle that correct for such devices.
but the dlink DIR862 and DIR866 using full equiped ar988x cards with valid eeprom

Quoted 5 lines

If ath10k assigns 00:03:07:12:34:56 mac address to the interface then you've most likely provided a fake board template and the device doesn't have a valid EEPROM data. Are you using skip_otp module parameter by any chance or other hacks/custom patches on ath10k?

i just use the openwrt patch which allows to override invalid otp data using the board.bin. this works for device likes the archer c7. but board.bin is not used for valid otp data.

this is this one here

drivers/net/wireless/ath/ath10k/core.c +0 -0
@@ -387,9 +387,14 @@ static int ath10k_download_and_run_otp(s
 
        ath10k_dbg(ar, ATH10K_DBG_BOOT, "boot otp execute result %d\n", 

result);

- if (!skip_otp && result != 0) {
- ath10k_err(ar, "otp calibration failed: %d", result);
- return -EINVAL;
+ if (!skip_otp) {
+ if (result == 2) {
+ ath10k_warn(ar, "otp stream is empty, using board.bin contents");
+ return 0;
+ } else if (result != 0) {
+ ath10k_err(ar, "otp calibration failed: %d", result);
+ return -EINVAL;
+ }

Quoted 6 lines
Quoted 3 lines

is there any way/trick available to deal with that problem?

You can always change the mac address via ip or ifconfig commands after interface is created but that's probably not what you really want.

if i can trust that bssid (which i dont believe) etc. internal handling will change as well, this might work as workaround. but its a nasty way to handle it

Sebastian

Quoted 4 lines

Michał

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

#7 lore raw

Re: changing hw address of ath10k card

Quoted 18 lines · Sebastian Gottschall

Am 06.05.2015 um 13:06 schrieb Michal Kazior:

Quoted 17 lines
Quoted 2 lines · Sebastian Gottschall

Hello

Hi,

Quoted 7 lines

I recently stumbled across a problem in ath10k. while developing on embedded
wireless routers
i found a device which uses a ar988x card using a card embedded board eeprom. the problem is there is no way to override
the cards hardware/mac address, which is required on that device since it uses just a fake mac on that board data.

Quoted 6 lines
Quoted 3 lines

If device contains valid EEPROM data then both calibration and mac address is read using OTP. In that case a mere fake board template binary is used by the driver (the default board.bin).

the problem is that the board.bin is never used. i modified the board.bin to change the mac address, but it shows no effect

Hmm.. I guess this implies OTP has overwritten the board template provided by ath10k - which is normal. This means there's at least some valid EEPROM data on the device.

Quoted 18 lines
Quoted 7 lines

If device contains no valid EEPROM, which is the case for many routers out in the wild e.g. TP-Link Archer C5/C7, you must provide a complete board data. On routers this data is often stored on NAND/Flash partitions. OpenWRT does handle this and extracts calibration data file. If you fail to provide that and attempt to use the default board.bin you'll get hardcoded mac and Tx/Rx performance will be very poor most likely.

i'm aware of that. i handle that correct for such devices.
but the dlink DIR862 and DIR866 using full equiped ar988x cards with valid eeprom

Quoted 6 lines

If ath10k assigns 00:03:07:12:34:56 mac address to the interface then you've most likely provided a fake board template and the device doesn't have a valid EEPROM data. Are you using skip_otp module parameter by any chance or other hacks/custom patches on ath10k?

Quoted 6 lines

i just use the openwrt patch which allows to override invalid otp data using the board.bin.
this works for device likes the archer c7.

but board.bin is not used for
valid otp data.

It is used. The mere fact that it's overwritten is a proof of that. OTP by design does overwrite provided board data in device memory since it uses it only as a template.

The template mac may suggest that calibration data isn't complete and mac address is stored out-of-band somewhere else - no idea where though.

Another problem may be OTP program itself. What firmware did you try to run with ath10k and the device you have? 10.1? 10.2? 10.2.4? Perhaps OTP doesn't recognize the particular device board sold in these D-Link routers.

You could try extracting qca988x firmware from the proprietary router software blob and use that with ath10k to see if it's OTP/firmware related problem. You can check other threads on this mailing list to see how to assemble ath10k firmware blobs.

Michał

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

#8 lore raw

Re: changing hw address of ath10k card

Am 06.05.2015 um 13:32 schrieb Michal Kazior:

Quoted 15 lines · Sebastian Gottschall
Quoted 14 lines

Am 06.05.2015 um 13:06 schrieb Michal Kazior:

Quoted 13 lines
Quoted 1 line · Sebastian Gottschall

Hello

Hi,

Quoted 7 lines

I recently stumbled across a problem in ath10k. while developing on embedded
wireless routers
i found a device which uses a ar988x card using a card embedded board eeprom. the problem is there is no way to override
the cards hardware/mac address, which is required on that device since it uses just a fake mac on that board data.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

If device contains valid EEPROM data then both calibration and mac address is read using OTP. In that case a mere fake board template binary is used by the driver (the default board.bin).

the problem is that the board.bin is never used. i modified the board.bin to change the mac address, but it shows no effect

Hmm.. I guess this implies OTP has overwritten the board template provided by ath10k - which is normal. This means there's at least some valid EEPROM data on the device.

thats what i said. in original LSDK there is a function to change the hw address. but the ath10k firmware doesnt seem to provide such a command beside this a setting for changing ack timing would be usefull which is possible with lsdk as well. otherwise long range device (i got some request for porting such devices) are not able to establish any connection.
the ack timing register is combined with cts and rts timing within the card own registers (i found that out while reviewing some old firmware sourcecode)

Quoted 26 lines
Quoted 17 lines
Quoted 9 lines

If device contains no valid EEPROM, which is the case for many routers out in the wild e.g. TP-Link Archer C5/C7, you must provide a complete board data. On routers this data is often stored on NAND/Flash partitions. OpenWRT does handle this and extracts calibration data file. If you fail to provide that and attempt to use the default board.bin you'll get hardcoded mac and Tx/Rx performance will be very poor most likely.

i'm aware of that. i handle that correct for such devices.
but the dlink DIR862 and DIR866 using full equiped ar988x cards with valid eeprom

Quoted 5 lines

If ath10k assigns 00:03:07:12:34:56 mac address to the interface then you've most likely provided a fake board template and the device doesn't have a valid EEPROM data. Are you using skip_otp module parameter by any chance or other hacks/custom patches on ath10k?

Quoted 5 lines

i just use the openwrt patch which allows to override invalid otp data using the board.bin.
this works for device likes the archer c7.
but board.bin is not used for
valid otp data.

It is used. The mere fact that it's overwritten is a proof of that. OTP by design does overwrite provided board data in device memory since it uses it only as a template.

Quoted 8 lines

The template mac may suggest that calibration data isn't complete and mac address is stored out-of-band somewhere else - no idea where though.

Another problem may be OTP program itself. What firmware did you try to run with ath10k and the device you have? 10.1? 10.2? 10.2.4? Perhaps OTP doesn't recognize the particular device board sold in these D-Link routers.

latest firmware-5.bin so 10.2.4

Quoted 5 lines

You could try extracting qca988x firmware from the proprietary router software blob and use that with ath10k to see if it's OTP/firmware related problem. You can check other threads on this mailing list to see how to assemble ath10k firmware blobs.

i may try this, but i dont know if its compatible with ath10k. it has no the same format as ath10k, so i need to regenerate something like that and the api which is used, is unknown

Quoted 4 lines

Michał

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k