Skip to contentSkip to conversation

Anyone doing WiFi throughput tests?

ath9k-devel Ben Greear 10 participants 49 messages May 26, 2012 – Jun 20, 2012

lore
Share & tools
lore

Ben Greear

author
#0 lore raw

We've been doing some tests using Atheros stations and various APs. The max throughput we've seen so far is about 237Mbps (received UDP payload on the stations). (Open-Air, AP about 5 feet away, 3x3 MIMO, HT40, 5Ghz, etc).

We are still running lots of different permutations, but I am interested if anyone else has any numbers to share (official or otherwise).

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#1 lore raw
Quoted 6 lines · Ben Greear

We've been doing some tests using Atheros stations and various APs. The max throughput we've seen so far is about 237Mbps (received UDP payload on the stations). (Open-Air, AP about 5 feet away, 3x3 MIMO, HT40, 5Ghz, etc).

We are still running lots of different permutations, but I am interested if anyone else has any numbers to share (official or otherwise).

Which card/chip ?

With a XB112 card (3x3) in HT40 mode, 5Ghz, TX throughput (TCP) can reach 290 Mbps. Sample iperf run:

[  3] 28.0-29.0 sec  34.1 MBytes   286 Mbits/sec
[  3] 29.0-30.0 sec  34.6 MBytes   290 Mbits/sec
[  3] 30.0-31.0 sec  34.8 MBytes   292 Mbits/sec
[  3] 31.0-32.0 sec  34.8 MBytes   292 Mbits/sec
[  3] 32.0-33.0 sec  34.9 MBytes   293 Mbits/sec

Sujith

#2 lore raw

Am 26.05.2012 um 05:24 schrieb Sujith Manoharan <[email protected]>:

Quoted 10 lines
Quoted 6 lines · Ben Greear

We've been doing some tests using Atheros stations and various APs. The max throughput we've seen so far is about 237Mbps (received UDP payload on the stations). (Open-Air, AP about 5 feet away, 3x3 MIMO, HT40, 5Ghz, etc).

We are still running lots of different permutations, but I am interested if anyone else has any numbers to share (official or otherwise).

Which card/chip ?

Quoted 8 lines

With a XB112 card (3x3) in HT40 mode, 5Ghz, TX throughput (TCP) can reach 290 Mbps. Sample iperf run:

[  3] 28.0-29.0 sec  34.1 MBytes   286 Mbits/sec
[  3] 29.0-30.0 sec  34.6 MBytes   290 Mbits/sec
[  3] 30.0-31.0 sec  34.8 MBytes   292 Mbits/sec
[  3] 31.0-32.0 sec  34.8 MBytes   292 Mbits/sec
[  3] 32.0-33.0 sec  34.9 MBytes   293 Mbits/sec

Do you use ath9k for that purpouse? I was thinking our driver is not able to operate MIMO. Some changes that i missed?

JoeSemler

Quoted 6 lines

Sujith
_______________________________________________
ath9k-devel mailing list
[email protected]
https://lists.ath9k.org/mailman/listinfo/ath9k-devel

#3 lore raw
Quoted 11 lines · Joe Semler
Quoted 8 lines

With a XB112 card (3x3) in HT40 mode, 5Ghz, TX throughput (TCP) can reach 290 Mbps. Sample iperf run:

[  3] 28.0-29.0 sec  34.1 MBytes   286 Mbits/sec
[  3] 29.0-30.0 sec  34.6 MBytes   290 Mbits/sec
[  3] 30.0-31.0 sec  34.8 MBytes   292 Mbits/sec
[  3] 31.0-32.0 sec  34.8 MBytes   292 Mbits/sec
[  3] 32.0-33.0 sec  34.9 MBytes   293 Mbits/sec

Do you use ath9k for that purpouse? I was thinking our driver is not able to operate MIMO. Some changes that i missed?

Yes, those numbers are with ath9k, with latest wireless-testing.

Sujith

#4 lore raw

2012/5/26 Sujith Manoharan <[email protected]>

Quoted 11 lines
Quoted 1 line · Joe Semler
Quoted 1 line

With a XB112 card (3x3) in HT40 mode, 5Ghz, TX throughput (TCP) can

reach 290 Mbps.

Quoted 8 lines
Quoted 7 lines

Sample iperf run:

[  3] 28.0-29.0 sec  34.1 MBytes   286 Mbits/sec
[  3] 29.0-30.0 sec  34.6 MBytes   290 Mbits/sec
[  3] 30.0-31.0 sec  34.8 MBytes   292 Mbits/sec
[  3] 31.0-32.0 sec  34.8 MBytes   292 Mbits/sec
[  3] 32.0-33.0 sec  34.9 MBytes   293 Mbits/sec
Quoted 6 lines
Quoted 1 line

Do you use ath9k for that purpouse? I was thinking our driver is not

able to operate MIMO.

Quoted 1 line

Some changes that i missed?

Yes, those numbers are with ath9k, with latest wireless-testing.

OK, tnx Sujith, but that brings me to my next question acc. 2x2 or 3x3 on OpenWRT-based devices like NanoStation M or AirBridge M.
Up to now only 1x2 or 1x3 was possible on this devices. Do we have changings there too?

Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ath9k.org/pipermail/ath9k-devel/attachments/20120526/0412732a/attachment.html

#5 lore raw
Quoted 3 lines · Josef Semler

OK, tnx Sujith, but that brings me to my next question acc. 2x2 or 3x3 on OpenWRT-based devices like NanoStation M or AirBridge M. Up to now only 1x2 or 1x3 was possible on this devices. Do we have changings there too?

Hm, am not sure about these devices. But, the number of streams is determined based on the chip, and there is nothing that can be done to increase it. If you have access, load the driver with the parameter 'debug=0x200' and check dmesg - this should show the number of TX/RX streams.

Sujith

#6 lore raw
In reply to Josef Semler
Quoted 20 lines · Josef Semler

2012/5/26 Sujith Manoharan <[email protected] <mailto:[email protected]>>

Joe Semler wrote:
> > With a XB112 card (3x3) in HT40 mode, 5Ghz, TX throughput (TCP) can reach 290 Mbps.
> > Sample iperf run:
> >
> > [ 3] 28.0-29.0 sec 34.1 MBytes 286 Mbits/sec
> > [ 3] 29.0-30.0 sec 34.6 MBytes 290 Mbits/sec
> > [ 3] 30.0-31.0 sec 34.8 MBytes 292 Mbits/sec
> > [ 3] 31.0-32.0 sec 34.8 MBytes 292 Mbits/sec
> > [ 3] 32.0-33.0 sec 34.9 MBytes 293 Mbits/sec
>
> Do you use ath9k for that purpouse? I was thinking our driver is not able to operate MIMO.
> Some changes that i missed?

Quoted 6 lines

Yes, those numbers are with ath9k, with latest wireless-testing.

OK, tnx Sujith, but that brings me to my next question acc. 2x2 or 3x3 on OpenWRT-based devices like NanoStation M or AirBridge M. Up to now only 1x2 or 1x3 was possible on this devices. Do we have changings there too?

Why do you think that only 1x2 or 1x3 was possible on these devices? 2x2 and 3x3 has been working just fine for a long time now, and it's enabled by default where supported by the hardware.

- Felix

#7 lore raw

Hi Felix,

Am 26.05.2012 um 14:35 schrieb Felix Fietkau <[email protected]>:

Quoted 9 lines
Quoted 6 lines

OK, tnx Sujith, but that brings me to my next question acc. 2x2 or 3x3 on OpenWRT-based devices like NanoStation M or AirBridge M. Up to now only 1x2 or 1x3 was possible on this devices. Do we have changings there too?

Why do you think that only 1x2 or 1x3 was possible on these devices? 2x2 and 3x3 has been working just fine for a long time now, and it's enabled by default where supported by the hardware

Pickend this Info from The List that devices with openwrt are just transmitting at 1 stream but rec. at both. How can I change the settings? Also with uci or only with the iw-commands? Tnx 4 support.

Joe

Quoted 2 lines

- Felix

#8 lore raw

Hi,

If the device EEPROM states that it's a 2x2/3x3 device, it'll TX on two/three streams.

If it's something like an AR9281 where it's physically a 1T2R stream device, that's what it'll transmit/receive on by default.

If it's something like an AR9285 where it's physically a 1T1R stream device with antenna diversity, that's also what you get.

If the device EEPROM lies though, well, that's a problem. :-)

Adrian

#9 lore raw

2012/5/26 Adrian Chadd <[email protected]>

Quoted 5 lines

Hi,

If the device EEPROM states that it's a 2x2/3x3 device, it'll TX on two/three streams.

Quoted 8 lines

If it's something like an AR9281 where it's physically a 1T2R stream device, that's what it'll transmit/receive on by default.

If it's something like an AR9285 where it's physically a 1T1R stream device with antenna diversity, that's also what you get.

If the device EEPROM lies though, well, that's a problem. :-)

Installed OpenWRT f?r ubnt-nano on a Nanobridge. Well, NanoBridge should be 2x2 MIMO, but the device shows me AR7240 rev.2
Is this a a EEPROM-lie and how can I correct it?

Joe

Quoted 2 lines

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ath9k.org/pipermail/ath9k-devel/attachments/20120529/470a7475/attachment.htm

#10 lore raw
Quoted 12 lines · Josef Semler

2012/5/26 Adrian Chadd <adrian at freebsd.org <mailto:[email protected]>>

Hi,

If the device EEPROM states that it's a 2x2/3x3 device, it'll TX on two/three streams.

If it's something like an AR9281 where it's physically a 1T2R stream device, that's what it'll transmit/receive on by default.

Quoted 8 lines

If it's something like an AR9285 where it's physically a 1T1R stream device with antenna diversity, that's also what you get.

If the device EEPROM lies though, well, that's a problem. :-)

Installed OpenWRT f?r ubnt-nano on a Nanobridge. Well, NanoBridge should be 2x2 MIMO, but the device shows me AR7240 rev.2
Is this a a EEPROM-lie and how can I correct it?

AR7240 is the CPU, not the wifi chip.

- Felix

#11 lore raw
In reply to Joe Semler
Quoted 11 lines · Joe Semler

Hi Felix,

Am 26.05.2012 um 14:35 schrieb Felix Fietkau <[email protected]>:

Quoted 6 lines
Quoted 6 lines

OK, tnx Sujith, but that brings me to my next question acc. 2x2 or 3x3 on OpenWRT-based devices like NanoStation M or AirBridge M. Up to now only 1x2 or 1x3 was possible on this devices. Do we have changings there too?

Quoted 6 lines
Quoted 3 lines

Why do you think that only 1x2 or 1x3 was possible on these devices? 2x2 and 3x3 has been working just fine for a long time now, and it's enabled by default where supported by the hardware

Pickend this Info from The List that devices with openwrt are just transmitting at 1 stream but rec. at both. How can I change the settings? Also with uci or only with the iw-commands?

You don't need to change anything, just use the defaults.
I have no idea what post from the list you're referring to, but here's a piece of advice: It's not unheard of that sometimes people put wrong things on the internet :)

- Felix

author
#12 lore raw
In reply to Sujith Manoharan
Quoted 9 lines · Sujith Manoharan
Quoted 6 lines · Ben Greear

We've been doing some tests using Atheros stations and various APs. The max throughput we've seen so far is about 237Mbps (received UDP payload on the stations). (Open-Air, AP about 5 feet away, 3x3 MIMO, HT40, 5Ghz, etc).

We are still running lots of different permutations, but I am interested if anyone else has any numbers to share (official or otherwise).

Which card/chip ?

We're using WPEA-127N. We have a dual-core Atom for one system, and a quad-core i7 CPU (and two wifi NICs) in another system.. So far, the Atom with single NIC is benchmarking better in some tests. We were only using one of the NICs in the i7 for testing, but maybe there is still some interference or maybe we just had bad antenna placement or something.

We've used various Asus and Netgear APs...haven't done throughput tests with home-grown APs running Atheros NICs recently, but will do so next week.

Quoted 1 line

With a XB112 card (3x3) in HT40 mode, 5Ghz, TX throughput (TCP) can reach 290 Mbps.

What chipset or brand/model is this? I see that XB112 mentioned in the WPEA-127N description, but maybe that is just a form-factor description?

Quoted 7 lines

Sample iperf run:

[  3] 28.0-29.0 sec  34.1 MBytes   286 Mbits/sec
[  3] 29.0-30.0 sec  34.6 MBytes   290 Mbits/sec
[  3] 30.0-31.0 sec  34.8 MBytes   292 Mbits/sec
[  3] 31.0-32.0 sec  34.8 MBytes   292 Mbits/sec
[  3] 32.0-33.0 sec  34.9 MBytes   293 Mbits/sec

Can you offer any additional details on how you tested this? You are using the same NIC for both AP and station?

Open-air connection?

How far away is AP?

I would like to try to reproduce this result, as it is significantly better than what I'm seeing.

Are you doing any special AP tuning, like using short-guard-intervals or similar? Care to post the wpa_supplicant and hostapd config files?

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#13 lore raw
Quoted 5 lines · Ben Greear

We're using WPEA-127N. We have a dual-core Atom for one system, and a quad-core i7 CPU (and two wifi NICs) in another system.. So far, the Atom with single NIC is benchmarking better in some tests. We were only using one of the NICs in the i7 for testing, but maybe there is still some interference or maybe we just had bad antenna placement or something.

Fiddling with antenna position/placement does help.

Quoted 2 lines

What chipset or brand/model is this? I see that XB112 mentioned in the WPEA-127N description, but maybe that is just a form-factor description?

XB112 is the board and I am using an engineering sample. The WLAN chipset is AR9380.

Quoted 5 lines

Can you offer any additional details on how you tested this? You are using the same NIC for both AP and station?

Open-air connection?

Quoted 7 lines

How far away is AP?

I would like to try to reproduce this result, as it is significantly better than what I'm seeing.

Are you doing any special AP tuning, like using short-guard-intervals or similar? Care to post the wpa_supplicant and hostapd config files?

I am using a DB120 as AP which has dual radio - AR9340/AR9300. The setup is standard:

STA --------------> AP -------------------> CONSOLE
wlan/OTA gig-ethernet

The AP runs an internal BSP/SDK build and is positioned about 4-5 feet from the Station. As for the HT parameters, HT40 and short-GI is enabled in both bands.

Sujith

#14 lore raw
Quoted 10 lines · Sujith Manoharan

I am using a DB120 as AP which has dual radio - AR9340/AR9300. The setup is standard:

STA --------------> AP -------------------> CONSOLE
wlan/OTA gig-ethernet

The AP runs an internal BSP/SDK build and is positioned about 4-5 feet from the Station. As for the HT parameters, HT40 and short-GI is enabled in both bands.

With UDP, these are the numbers I get:

Server: [ iperf -i1 -s -u -l 8K ]

[  3] 29.0-30.0 sec  38.9 MBytes   327 Mbits/sec   0.246 ms   10/ 4993 (0.2%)
[  3] 30.0-31.0 sec  38.6 MBytes   323 Mbits/sec   0.260 ms   34/ 4970 (0.68%)
[  3] 31.0-32.0 sec  39.3 MBytes   330 Mbits/sec   0.212 ms   14/ 5045 (0.28%)
[  3] 32.0-33.0 sec  37.2 MBytes   312 Mbits/sec   0.229 ms   15/ 4777 (0.31%)
[  3] 33.0-34.0 sec  39.0 MBytes   327 Mbits/sec   0.253 ms   13/ 5002 (0.26%)

Client: [ iperf -i1 -c 172.16.0.10 -t 10000 -u -b 400M -l 8K ]

[  3] 29.0-30.0 sec  39.0 MBytes   327 Mbits/sec
[  3] 30.0-31.0 sec  38.9 MBytes   326 Mbits/sec
[  3] 31.0-32.0 sec  39.4 MBytes   330 Mbits/sec
[  3] 32.0-33.0 sec  37.3 MBytes   313 Mbits/sec
[  3] 33.0-34.0 sec  39.1 MBytes   328 Mbits/sec

Sujith

author
#15 lore raw
Quoted 3 lines · Sujith Manoharan
Quoted 2 lines · Sujith Manoharan

I am using a DB120 as AP which has dual radio - AR9340/AR9300. The setup is standard:

Do you have a link to anyone selling this AP, or is it just an engineering sample as well?

I didn't have much luck finding it in google... Is there a more specific part number or manufacturer available?

I can't even find marketing type pages for AR9340, search results come up empty at www.atheros.com :P

Quoted 18 lines
Quoted 7 lines

STA --------------> AP -------------------> CONSOLE
wlan/OTA gig-ethernet

The AP runs an internal BSP/SDK build and is positioned about 4-5 feet from the Station. As for the HT parameters, HT40 and short-GI is enabled in both bands.

With UDP, these are the numbers I get:

Server: [ iperf -i1 -s -u -l 8K ]

[  3] 29.0-30.0 sec  38.9 MBytes   327 Mbits/sec   0.246 ms   10/ 4993 (0.2%)
[  3] 30.0-31.0 sec  38.6 MBytes   323 Mbits/sec   0.260 ms   34/ 4970 (0.68%)
[  3] 31.0-32.0 sec  39.3 MBytes   330 Mbits/sec   0.212 ms   14/ 5045 (0.28%)
[  3] 32.0-33.0 sec  37.2 MBytes   312 Mbits/sec   0.229 ms   15/ 4777 (0.31%)
[  3] 33.0-34.0 sec  39.0 MBytes   327 Mbits/sec   0.253 ms   13/ 5002 (0.26%)
Quoted 7 lines

Client: [ iperf -i1 -c 172.16.0.10 -t 10000 -u -b 400M -l 8K ]

[  3] 29.0-30.0 sec  39.0 MBytes   327 Mbits/sec
[  3] 30.0-31.0 sec  38.9 MBytes   326 Mbits/sec
[  3] 31.0-32.0 sec  39.4 MBytes   330 Mbits/sec
[  3] 32.0-33.0 sec  37.3 MBytes   313 Mbits/sec
[  3] 33.0-34.0 sec  39.1 MBytes   328 Mbits/sec

Those are nice results! I'll try WPEA-127N as AP and STA and see what I get.

If you have any suggestions for other commercially available NICs or APs to try, please let me know.

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#16 lore raw
Quoted 1 line · Ben Greear

I can't even find marketing type pages for AR9340

..

Quoted 2 lines

If you have any suggestions for other commercially available NICs or APs to try

Tests which noone outside Atheros can replicate are obviously not relevant to anyone outside Atheros.

On the other hand Atheros developers obviously do not work with hardware which is already on the market. Those projects are finished already.

Every hardware vendor who is using open source works exactly the same way, the problem is obviously the gap between users and Atheros developers when the code is less than perfect when hardware goes to market at the end of each project.

//Peter

#17 lore raw
Quoted 6 lines · Peter Stuge

Tests which noone outside Atheros can replicate are obviously not relevant to anyone outside Atheros.

On the other hand Atheros developers obviously do not work with hardware which is already on the market. Those projects are finished already.

The hardware we're talking about is a reference design, which is what we in development tend to work with to begin with.

Customers do weird things to our stuff at times. :-)

iperf is a perfectly fine tool to test performance with. That's why sujith was using it.

adrian

#18 lore raw
In reply to Ben Greear
Quoted 2 lines · Ben Greear

Do you have a link to anyone selling this AP, or is it just an engineering sample as well?

The board is an engineering sample but (a) customers and developers can ask for them under NDA, (b) yes, openwrt developers have done this and have the hardware, and (c) the chip is shipping as far as I'm aware.

It's "just" a MIPS74k + AR9340 (3x3) + on-board AR9580 I think. I'll have to double check. The wireless NIC side of things is pretty standard though, so any Osprey NIC will be fine.

Quoted 7 lines

I can't even find marketing type pages for AR9340, search results come up empty at www.atheros.com :P

Quoted 2 lines

[ ?3] 33.0-34.0 sec ?39.0 MBytes ? 327 Mbits/sec ? 0.253 ms ? 13/ 5002 (0.26%)

Those are nice results! ?I'll try WPEA-127N as AP and STA and see what I get.

That's what you should be getting. I'd be really surprised if you didn't. :-)

nbd has said he'll look into it, so between Sujith and Felix I think we're well covered for now.

Thanks for bringing this to everyone's attention Ben!

Adrian

#19 lore raw
Quoted 22 lines · Adrian Chadd
Quoted 2 lines · Ben Greear

Do you have a link to anyone selling this AP, or is it just an engineering sample as well?

The board is an engineering sample but (a) customers and developers can ask for them under NDA, (b) yes, openwrt developers have done this and have the hardware, and (c) the chip is shipping as far as I'm aware.

It's "just" a MIPS74k + AR9340 (3x3) + on-board AR9580 I think. I'll have to double check. The wireless NIC side of things is pretty standard though, so any Osprey NIC will be fine.

Quoted 7 lines

I can't even find marketing type pages for AR9340, search results come up empty at www.atheros.com :P

Quoted 2 lines

[ 3] 33.0-34.0 sec 39.0 MBytes 327 Mbits/sec 0.253 ms 13/ 5002
(0.26%)

Those are nice results! I'll try WPEA-127N as AP and STA and see what I get.

Quoted 6 lines

That's what you should be getting. I'd be really surprised if you didn't. :-)

nbd has said he'll look into it, so between Sujith and Felix I think we're well covered for now.

Thanks for bringing this to everyone's attention Ben!

I already committed some performance fixes in OpenWrt that bring UDP performance over a WDS link between an AR9344+AR9380 (Atheros DB120) and an AR7242+AR9380 (TP-Link TL-WR2543ND) up to about 310-320 Mbit/s, when doing Tx on the DB120 side. Reverse direction is slower, because AR7242 isn't as powerful as the AR9344.

These fixes do not touch ath9k, I only tweaked the network stack to avoid redundant copying/reallocation of skb data.

- Felix

#20 lore raw

Sweet. Thanks a lot for chasing that up.

What's the oprofile output look like for the AR7242 when doing that?

Adrian

#21 lore raw
Quoted 3 lines · Adrian Chadd

Sweet. Thanks a lot for chasing that up.

What's the oprofile output look like for the AR7242 when doing that?

Haven't run oprofile on the AR7242 yet, but here's what 350Mbits/s UDP ethernet-rx <-> wifi-tx traffic (bridged) looks like on AR9344:

CPU: MIPS 74K, speed 417 MHz (estimated)
Counted CYCLES events (0-0 Cycles) with a unit mask of 0x00 (No unit mask) count 10000

samples  %        app name                 symbol name
203986    6.8165  vmlinux                  ag71xx_poll
139395    4.6581  vmlinux                  __do_softirq
119259    3.9852  mac80211.ko              invoke_tx_handlers
112990    3.7757  vmlinux                  ring_buffer_consume
89195     2.9806  mac80211.ko              ieee80211_tx_status
69185     2.3119  vmlinux                  __copy_user
67684     2.2618  vmlinux                  __bzero
67014     2.2394  ath9k.ko                 ath_txq_schedule
65597     2.1920  vmlinux                  __slab_alloc.isra.60.constprop.63
65375     2.1846  vmlinux                  eth_type_trans
63594     2.1251  vmlinux                  r4k_dma_cache_inv
61121     2.0425  mac80211.ko              ieee80211_subif_start_xmit
50797     1.6975  ath9k_hw.ko              ar9003_set_txdesc
49942     1.6689  vmlinux                  __rmemcpy
48397     1.6173  ath9k.ko                 ath_tx_complete
46697     1.5605  vmlinux                  skb_release_data
45083     1.5065  vmlinux                  __netif_receive_skb
43928     1.4679  ath9k.ko                 ath_tx_complete_aggr.isra.21
43289     1.4466  vmlinux                  vlan_untag
41766     1.3957  mac80211.ko              minstrel_ht_set_rate
41154     1.3752  ath9k.ko                 ath_tx_setup_buffer.isra.18
41136     1.3746  vmlinux                  pfifo_fast_dequeue
40500     1.3534  vmlinux                  __slab_free.isra.58
39498     1.3199  vmlinux                  __qdisc_run
39278     1.3125  ath9k.ko                 ath_tx_start
38789     1.2962  vmlinux                  r4k_dma_cache_wback_inv
38497     1.2864  ath9k.ko                 ath_tx_fill_desc
37308     1.2467  vmlinux                  kfree
36235     1.2109  vmlinux                  br_handle_frame
35178     1.1755  vmlinux                  skb_put
34532     1.1539  vmlinux                  __kmalloc_track_caller
31535     1.0538  vmlinux                  kmem_cache_alloc
31045     1.0374  vmlinux                  dev_queue_xmit
30953     1.0343  vmlinux                  vlan_do_receive
30151     1.0075  mac80211.ko              __ieee80211_tx
30132     1.0069  vmlinux                  put_cpu_partial
29625     0.9900  vmlinux                  mips_dma_map_page
28214     0.9428  mac80211.ko              ieee80211_tx_prepare
27059     0.9042  vmlinux                  br_fdb_update
25845     0.8637  oprofile.ko              add_event_entry
25230     0.8431  vmlinux                  br_handle_frame_finish
24652     0.8238  vmlinux                  net_rx_action
22232     0.7429  mac80211.ko              rate_control_get_rate
21840     0.7298  mac80211.ko              minstrel_ht_get_rate
21591     0.7215  mac80211.ko              ccmp_encrypt_skb
20313     0.6788  ath9k.ko                 ath9k_ps_restore
20055     0.6702  vmlinux                  r4k_wait_irqoff
19867     0.6639  vmlinux                  dev_hard_start_xmit
19220     0.6423  vmlinux                  __br_fdb_get
18673     0.6240  vmlinux                  ksize
17769     0.5938  vmlinux                  __alloc_skb
17687     0.5910  vmlinux                  kmem_cache_free
17605     0.5883  mac80211.ko              minstrel_ht_tx_status
17112     0.5718  mac80211.ko              ieee80211_select_queue
16926     0.5656  ath9k.ko                 ath_tx_complete_buf
16310     0.5450  vmlinux                  pfifo_fast_enqueue
16065     0.5368  vmlinux                  local_bh_enable
15586     0.5208  ath9k.ko                 ath_tx_update_baw.isra.14
14449     0.4828  vmlinux                  skb_push
14097     0.4711  oprofile.ko              sync_buffer
13755     0.4596  vmlinux                  napi_complete
12793     0.4275  vmlinux                  mips_dma_unmap_page
12228     0.4086  vmlinux                  br_dev_queue_push_xmit
12212     0.4081  vmlinux                  br_forward
12149     0.4060  vmlinux                  __netdev_alloc_skb
11250     0.3759  vmlinux                  rcu_bh_qs
11153     0.3727  vmlinux                  atomic64_add_return
11123     0.3717  ath9k.ko                 ath9k_tx
11026     0.3685  ath9k.ko                 ath_debug_stat_tx
10763     0.3597  oprofile.ko              op_cpu_buffer_read_entry
author
#22 lore raw
In reply to Adrian Chadd
Quoted 5 lines · Adrian Chadd
Quoted 2 lines · Ben Greear

Do you have a link to anyone selling this AP, or is it just an engineering sample as well?

Quoted 8 lines

The board is an engineering sample but (a) customers and developers can ask for them under NDA, (b) yes, openwrt developers have done this and have the hardware, and (c) the chip is shipping as far as I'm aware.

It's "just" a MIPS74k + AR9340 (3x3) + on-board AR9580 I think. I'll have to double check. The wireless NIC side of things is pretty standard though, so any Osprey NIC will be fine.

Ok, I think I understand. But, "Osprey" returns no useful results that I can find, so it must be some internal code name for a project.

If you can be more specific about exactly what chipsets/NICs this includes it would be useful for those of us without NDAs or other internal Atheros/Qualcomm connections.

As far as shipping chipsets go (perhaps from the list below)
http://www.qca.qualcomm.com/technology/technology.php?nav1=47

What is considered the top-end Atheros chip for fastest/best 3x3 MIMO speeds?

Maybe 9390 (EW-DNXA-H1 for instance?) is expected to be better than 9380?

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#23 lore raw

Ah, Osprey is AR93xx series stuff. Well, not strictly speaking, but that "family."

I'd have to go and check what the current state of all the latest 11n chips are. Luis, Sujith and Felix would know better than I; I'm still stuck in the land of AR92xx in FreeBSD (at least for the next few weeks.)

Also, FreeBSD on 2x2 (AR9160) hostap:

TCP sta -> hostap:

[SUM] 0.0-10.0 sec 178 MBytes 149 Mbits/sec

UDP sta -> hostap:

[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0-10.1 sec 274 MBytes 228 Mbits/sec 0.135 ms 17045/212291 (8%)

TCP is around the expected spot, but I've seen it around 160MBit in the past. I think I messed up something with BAR/TX scheduling. The UDP RX is likely another scheduling issue; I seem to be occasionally overflowing the RX descriptor list.

In any case, I'll fix it up and report back, just as a comparison point.

Adrian

#24 lore raw
Quoted 7 lines · Adrian Chadd

Ah, Osprey is AR93xx series stuff. Well, not strictly speaking, but that "family."

I'd have to go and check what the current state of all the latest 11n chips are. Luis, Sujith and Felix would know better than I; I'm still stuck in the land of AR92xx in FreeBSD (at least for the next few weeks.)

ath9k has good support for the AR9003 family - AR9380 based devices, that is. And the numbers that I posted were with a XB112, which uses AR9380.

Quoted 10 lines

Also, FreeBSD on 2x2 (AR9160) hostap:

TCP sta -> hostap:

[SUM] 0.0-10.0 sec 178 MBytes 149 Mbits/sec

UDP sta -> hostap:

Quoted 7 lines

[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0-10.1 sec 274 MBytes 228 Mbits/sec 0.135 ms 17045/212291 (8%)

TCP is around the expected spot, but I've seen it around 160MBit in the past. I think I messed up something with BAR/TX scheduling. The UDP RX is likely another scheduling issue; I seem to be occasionally overflowing the RX descriptor list.

Well, 228 Mbps is much more reasonable. :)

Sujith

#25 lore raw

Heh. 228MBit is more reasonable, but I'm getting it up around 235/240MBit at the present moment. I'm hitting queue overruns though, so I think I've messed something up in the driver, or someone broke the scheduler in weird/wonderful ways.

I'll let you know when it's at 250MBit again. :)

Adrian

#26 lore raw

.. yup, I got it to 250MBit UDP. :-)

It turns out that (at least in FreeBSD-9), the scheduler and sleep state behaviour when doing adaptive power save/sleep state (ie, adaptive CPU speed changes, going into C2) is enough to negatively impact my iperf and ath/net80211 taskqueue scheduling.

When I nail it back up to C1, fixed speed - everything is perfectly fine.

I'll go and do some further digging into this, but it's good to see that I can squeeze decently high throughput out of the 2x2 NICs.

Adrian

author
#27 lore raw
Quoted 11 lines · Adrian Chadd

.. yup, I got it to 250MBit UDP. :-)

It turns out that (at least in FreeBSD-9), the scheduler and sleep state behaviour when doing adaptive power save/sleep state (ie, adaptive CPU speed changes, going into C2) is enough to negatively impact my iperf and ath/net80211 taskqueue scheduling.

When I nail it back up to C1, fixed speed - everything is perfectly fine.

I'll go and do some further digging into this, but it's good to see that I can squeeze decently high throughput out of the 2x2 NICs.

I'm getting right at 250Mbps of UDP payload received when
using a 2x2 AR9382 NIC (WPEA-121N) in a Lenovo X220i (with hacked white-listed BIOS). AP is a 3x3 AR9380 NIC (WPEA_127N) in Atom based network appliance.

Open-air connection, about 3 feet apart. This is in the 'download' direction: Wired to Station. HT-40 on 5Ghz.

The rates bounce around a bit...down to 240Mbps or so for a bit, then back to 250Mbps. Might be some other interference around as this is a relatively noisy environment...

Upload speed seems to be a constant 243Mbps..at least at this moment.

Kernel is 3.3.7+.

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
author
#28 lore raw
In reply to Ben Greear
Quoted 11 lines · Ben Greear
Quoted 9 lines · Sujith Manoharan

With UDP, these are the numbers I get:

Server: [ iperf -i1 -s -u -l 8K ]

[  3] 29.0-30.0 sec  38.9 MBytes   327 Mbits/sec   0.246 ms   10/ 4993 (0.2%)
[  3] 30.0-31.0 sec  38.6 MBytes   323 Mbits/sec   0.260 ms   34/ 4970 (0.68%)
[  3] 31.0-32.0 sec  39.3 MBytes   330 Mbits/sec   0.212 ms   14/ 5045 (0.28%)
[  3] 32.0-33.0 sec  37.2 MBytes   312 Mbits/sec   0.229 ms   15/ 4777 (0.31%)
[  3] 33.0-34.0 sec  39.0 MBytes   327 Mbits/sec   0.253 ms   13/ 5002 (0.26%)

We started testing with two AR9380 NICs today (one AP, the other STA). I applied Felix's skb optimization patch, and the ath9k memleak fix patch on top of 3.3.7+.

When we generate traffic using a modified version of pktgen, the STA interface transmits at around 310Mbps for a minute or two, but then the system dies of OOM (and maybe worse..having trouble getting useful serial console log). It died much faster before Felix's two patches were applied.

I disabled all of our network related buffer adjustments (ie, no longer increasing tcp_wmem, etc), and it still crashes.

The system has 2GB RAM, but it is 32-bit kernel, so not all is available to the networking code... That said, the OOM killer kills VNC and such.

Anyway, I'll try some memleak debugging to see if I can find any leaks. It seems to me that we should not actually OOM just by trying to transmit too fast on a station interface :P

PS. If anyone knows how to make minicom ignore page refreshes so that it doesn't obscure the last bit of the crash log when BIOS starts up again, please let me know!

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#29 lore raw
Quoted 6 lines · Ben Greear
Quoted 1 line · Ben Greear

On 05/26/2012 09:39 AM, Sujith Manoharan wrote:

We started testing with two AR9380 NICs today (one AP, the other STA). I applied Felix's skb optimization patch, and the ath9k memleak fix patch on top of 3.3.7+.

Quoted 8 lines

The system has 2GB RAM, but it is 32-bit kernel, so not all is available to the networking code... That said, the OOM killer kills VNC and such.

Anyway, I'll try some memleak debugging to see if I can find any leaks. It seems to me that we should not actually OOM just by trying to transmit too fast on a station interface :P

well, there's that:
http://comments.gmane.org/gmane.linux.drivers.ath9k.devel/8233

It might not fix the bug, but it can save you time to confirm that is not related to this particular skb leak.

Regards,
Chr

author
#30 lore raw
Quoted 15 lines · Christian Lamparter
Quoted 6 lines · Ben Greear
Quoted 1 line · Ben Greear

On 05/26/2012 09:39 AM, Sujith Manoharan wrote:

We started testing with two AR9380 NICs today (one AP, the other STA). I applied Felix's skb optimization patch, and the ath9k memleak fix patch on top of 3.3.7+.

Quoted 8 lines

The system has 2GB RAM, but it is 32-bit kernel, so not all is available to the networking code... That said, the OOM killer kills VNC and such.

Anyway, I'll try some memleak debugging to see if I can find any leaks. It seems to me that we should not actually OOM just by trying to transmit too fast on a station interface :P

Quoted 5 lines

well, there's that:
http://comments.gmane.org/gmane.linux.drivers.ath9k.devel/8233

It might not fix the bug, but it can save you time to confirm that is not related to this particular skb leak.

Ok, I will try this.

The leak this might fix is some ack logic up in mac80211?

Seems like we could also use some atomic counters to keep track of number of outstanding mac80211-alocated skbs with this and be worried if this number grows and grows?

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
author
#31 lore raw
In reply to Christian Lamparter
Quoted 15 lines · Christian Lamparter
Quoted 6 lines · Ben Greear
Quoted 1 line · Ben Greear

On 05/26/2012 09:39 AM, Sujith Manoharan wrote:

We started testing with two AR9380 NICs today (one AP, the other STA). I applied Felix's skb optimization patch, and the ath9k memleak fix patch on top of 3.3.7+.

Quoted 8 lines

The system has 2GB RAM, but it is 32-bit kernel, so not all is available to the networking code... That said, the OOM killer kills VNC and such.

Anyway, I'll try some memleak debugging to see if I can find any leaks. It seems to me that we should not actually OOM just by trying to transmit too fast on a station interface :P

Quoted 5 lines

well, there's that:
http://comments.gmane.org/gmane.linux.drivers.ath9k.devel/8233

It might not fix the bug, but it can save you time to confirm that is not related to this particular skb leak.

I ported this to 3.3.7+ and applied it to my kernel trees. It has tested out fine so far, though it did not actually fix the problem I was having. That was not a real leak, just always-growing pending queue length, probably due to some issue with our version of pktgen.

It is mostly a port-by-hand type of thing since there are lots of conflicts. Let me know if you'd like me to post my version (and plz confirm your signed-off-by).

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#32 lore raw
Quoted 34 lines · Ben Greear
Quoted 20 lines · Christian Lamparter
Quoted 9 lines · Ben Greear
Quoted 2 lines · Ben Greear

On 05/26/2012 09:39 AM, Sujith Manoharan wrote:

We started testing with two AR9380 NICs today (one AP, the other STA). I applied Felix's skb optimization patch, and the ath9k memleak fix patch on top of 3.3.7+.

Quoted 8 lines

The system has 2GB RAM, but it is 32-bit kernel, so not all is available to the networking code... ?That said, the OOM killer kills VNC and such.

Anyway, I'll try some memleak debugging to see if I can find any leaks. ?It seems to me that we should not actually OOM just by trying to transmit too fast on a station interface :P

Quoted 5 lines

well, there's that:
http://comments.gmane.org/gmane.linux.drivers.ath9k.devel/8233

It might not fix the bug, but it can save you time to confirm that is not related to this particular skb leak.

I ported this to 3.3.7+ and applied it to my kernel trees. ?It has tested out fine so far, though it did not actually fix the problem I was having. ?That was not a real leak, just always-growing pending queue length, probably due to some issue with our version of pktgen.

Quoted 4 lines

It is mostly a port-by-hand type of thing since there are lots of conflicts. ?Let me know if you'd like me to post my version (and plz confirm your signed-off-by).

please! (and cc cerowrt-devel at lists.bufferbloat.net)

I'm hoping this string of patches will have some bearing on my own

(while I'm trying to not write a line of code for a while, others on my
list are struggling with this)

Quoted 13 lines

Thanks,
Ben

--
Ben Greear <[email protected]>
Candela Technologies Inc ?http://www.candelatech.com

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in the body of a message to majordomo at vger.kernel.org
More majordomo info at ?http://vger.kernel.org/majordomo-info.html

Signature · 4 lines
Dave T?ht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://www.bufferbloat.net
author
#33 lore raw
Quoted 40 lines · Dave Taht
Quoted 34 lines · Ben Greear
Quoted 20 lines · Christian Lamparter
Quoted 9 lines · Ben Greear
Quoted 2 lines · Ben Greear

On 05/26/2012 09:39 AM, Sujith Manoharan wrote:

We started testing with two AR9380 NICs today (one AP, the other STA). I applied Felix's skb optimization patch, and the ath9k memleak fix patch on top of 3.3.7+.

Quoted 8 lines

The system has 2GB RAM, but it is 32-bit kernel, so not all is available to the networking code... That said, the OOM killer kills VNC and such.

Anyway, I'll try some memleak debugging to see if I can find any leaks. It seems to me that we should not actually OOM just by trying to transmit too fast on a station interface :P

Quoted 5 lines

well, there's that:
http://comments.gmane.org/gmane.linux.drivers.ath9k.devel/8233

It might not fix the bug, but it can save you time to confirm that is not related to this particular skb leak.

I ported this to 3.3.7+ and applied it to my kernel trees. It has tested out fine so far, though it did not actually fix the problem I was having. That was not a real leak, just always-growing pending queue length, probably due to some issue with our version of pktgen.

Quoted 4 lines

It is mostly a port-by-hand type of thing since there are lots of conflicts. Let me know if you'd like me to post my version (and plz confirm your signed-off-by).

Quoted 7 lines

please! (and cc cerowrt-devel at lists.bufferbloat.net)

I'm hoping this string of patches will have some bearing on my own

(while I'm trying to not write a line of code for a while, others on my
list are struggling with this)

For your bug, do you get any warnings on a serial console?

What does 'top' show? Ie, why is the load so high? Just flogging the kernel with pkts shouldn't explode the load.

Maybe processes are blocked trying to take a lock..maybe a networking lock?

Tried enabling lockdep in this scenario, and maybe the hard/soft deadlock detection logic?

If you back off the traffic, does the system recover? If so, maybe your CPU just can't handle the load....

If you think the mac80211 pending queues are backing up, cat out /debug/ieee*/phy*/queues

That was the symptom I saw today with pktgen, but I think that is probably more the fault of pktgen and may not be an issue with more normal traffic flow.

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#34 lore raw
In reply to Ben Greear
Quoted 14 lines · Ben Greear

We started testing with two AR9380 NICs today (one AP, the other STA). I applied Felix's skb optimization patch, and the ath9k memleak fix patch on top of 3.3.7+.

When we generate traffic using a modified version of pktgen, the STA interface transmits at around 310Mbps for a minute or two, but then the system dies of OOM (and maybe worse..having trouble getting useful serial console log). It died much faster before Felix's two patches were applied.

I disabled all of our network related buffer adjustments (ie, no longer increasing tcp_wmem, etc), and it still crashes.

Quoted 8 lines

The system has 2GB RAM, but it is 32-bit kernel, so not all is available to the networking code... That said, the OOM killer kills VNC and such.

Anyway, I'll try some memleak debugging to see if I can find any leaks. It seems to me that we should not actually OOM just by trying to transmit too fast on a station interface :P

In my setup, UDP RX (AP->STA) locks up the station hard and eventually I get this:

[  292.093359] BUG: soft lockup - CPU#0 stuck for 23s! [syslog-ng:403]
[  292.093555] irq event stamp: 12065793
[  292.093560] hardirqs last  enabled at (12065792): [<ffffffff81490855>] _raw_spin_unlock_irqrestore+0x65/0x80
[  292.093579] hardirqs last disabled at (12065793): [<ffffffff814922ea>] apic_timer_interrupt+0x6a/0x80
[  292.093591] softirqs last  enabled at (4271834): [<ffffffff81059507>] __do_softirq+0x137/0x240
[  292.093605] softirqs last disabled at (4273355): [<ffffffff81492cec>] call_softirq+0x1c/0x30
[  292.093617] CPU 0 
[  292.093826] Pid: 403, comm: syslog-ng Tainted: G           O 3.4.0-wl #21 LENOVO 7661GN4/7661GN4
[  292.093841] RIP: 0010:[<ffffffff8149085a>]  [<ffffffff8149085a>] _raw_spin_unlock_irqrestore+0x6a/0x80
[  292.093855] RSP: 0018:ffff88007d403c90  EFLAGS: 00000282
[  292.093862] RAX: ffff880037526dd0 RBX: 0000000000000000 RCX: 0000000000000002
[  292.093869] RDX: 0000000000000002 RSI: ffff880037527428 RDI: 0000000000000282
[  292.093876] RBP: ffff88007d403ca0 R08: ffff8800755f5048 R09: 0000000000000005
[  292.093883] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88007d403c08
[  292.093889] R13: ffffffff814922ef R14: ffff88007d403ca0 R15: ffffffff81a65c80
[  292.093897] FS:  00007f6c0dc5db00(0000) GS:ffff88007d400000(0000) knlGS:0000000000000000
[  292.093905] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  292.093911] CR2: 00007f35a1fbe1c3 CR3: 0000000075490000 CR4: 00000000000007f0
[  292.093918] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  292.093925] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  292.093933] Process syslog-ng (pid: 403, threadinfo ffff8800789de000, task ffff880037526dd0)
[  292.093939] Stack:
[  292.093943]  ffff8800755f5000 0000000000000282 ffff88007d403cc0 ffffffff81270aa2
[  292.093960]  00000000000007b6 0000000000000040 ffff88007d403d10 ffffffff812720fe
[  292.093975]  0000000000000000 0000000001000000 ffff88007d403d10 ffff880075155700
[  292.093990] Call Trace:
[  292.093995]  <IRQ> 
[  292.094006]  [<ffffffff81270aa2>] dma_entry_alloc+0x62/0x90
[  292.094017]  [<ffffffff812720fe>] debug_dma_map_page+0x7e/0x140
[  292.094037]  [<ffffffffa06b3669>] ath_rx_tasklet+0x969/0x1f10 [ath9k]
[  292.094056]  [<ffffffffa06b0054>] ath9k_tasklet+0x104/0x180 [ath9k]
[  292.094068]  [<ffffffff8105a073>] tasklet_action+0x83/0x110
[  292.094079]  [<ffffffff81059498>] __do_softirq+0xc8/0x240
[  292.094091]  [<ffffffff81492cec>] call_softirq+0x1c/0x30
[  292.094102]  [<ffffffff81016855>] do_softirq+0xa5/0xe0
[  292.094113]  [<ffffffff81059986>] irq_exit+0xa6/0xe0
[  292.094124]  [<ffffffff81493583>] do_IRQ+0x63/0xe0
[  292.094134]  [<ffffffff81490cef>] common_interrupt+0x6f/0x6f
[  292.094140]  <EOI> 
[  292.094150]  [<ffffffff810aeafc>] ? mark_held_locks+0x8c/0x110
[  292.094162]  [<ffffffff8149085a>] ? _raw_spin_unlock_irqrestore+0x6a/0x80
[  292.094173]  [<ffffffff8107fd23>] __wake_up+0x53/0x70
[  292.094196]  [<ffffffffa022af3c>] jbd2_journal_stop+0x24c/0x2c0 [jbd2]
[  292.094246]  [<ffffffffa0275788>] __ext4_journal_stop+0x78/0xa0 [ext4]
[  292.094278]  [<ffffffffa025a5ad>] ext4_da_write_end+0xad/0x390 [ext4]
[  292.094293]  [<ffffffff81116d03>] generic_file_buffered_write+0x1a3/0x2c0
[  292.094309]  [<ffffffff8148dba7>] ? mutex_lock_nested+0x2f7/0x3d0
[  292.094321]  [<ffffffff81118c4a>] __generic_file_aio_write+0x22a/0x440
[  292.094334]  [<ffffffff81118ed3>] generic_file_aio_write+0x73/0xe0
[  292.094363]  [<ffffffffa025136f>] ext4_file_write+0xaf/0x270 [ext4]
[  292.094374]  [<ffffffff810adcf0>] ? lock_release_non_nested+0xa0/0x2e0
[  292.094403]  [<ffffffffa02512c0>] ? ext4_file_mmap+0x60/0x60 [ext4]
[  292.094415]  [<ffffffff8117c472>] do_sync_readv_writev+0xd2/0x110
[  292.094427]  [<ffffffff8113bd23>] ? might_fault+0x53/0xb0
[  292.094438]  [<ffffffff8120aa0c>] ? security_file_permission+0x2c/0xb0
[  292.094449]  [<ffffffff8117bb91>] ? rw_verify_area+0x61/0xf0
[  292.094460]  [<ffffffff8117c75b>] do_readv_writev+0xdb/0x1f0
[  292.094470]  [<ffffffff810adcf0>] ? lock_release_non_nested+0xa0/0x2e0
[  292.094481]  [<ffffffff8113bd23>] ? might_fault+0x53/0xb0
[  292.094491]  [<ffffffff814917d5>] ? sysret_check+0x22/0x5d
[  292.094502]  [<ffffffff8117c8a5>] vfs_writev+0x35/0x60
[  292.094511]  [<ffffffff8117ca3a>] sys_writev+0x4a/0xc0
[  292.094523]  [<ffffffff814917a9>] system_call_fastpath+0x16/0x1b
[  292.094530] Code: ff 65 48 8b 04 25 70 c8 00 00 83 a8 44 e0 ff ff 01 48 8b 80 38 e0 ff ff a8 08 75 17 5b 41 5c 5d c3 e8 bb e4 c1 ff 48 89 df 57 9d <66> 66 90 66 90 90 eb ce e8 79 e9 ff ff eb e2 0f 1f 80 00 00 00 

Or, this is spewed in the log:

[ 1934.719823] INFO: rcu_preempt self-detected stall on CPU { 1}  (t=18000 jiffies)
[ 1934.719830] sending NMI to all CPUs:
[ 1934.719830] NMI backtrace for cpu 1
[ 1934.719830] CPU 1 
[ 1934.719830] Pid: 0, comm: swapper/1 Tainted: G           O 3.4.0-wl #21 LENOVO 7661GN4/7661GN4
[ 1934.719830] RIP: 0010:[<ffffffff8125e302>]  [<ffffffff8125e302>] __const_udelay+0x12/0x40
[ 1934.719830] RSP: 0018:ffff88007d503600  EFLAGS: 00000006
[ 1934.719830] RAX: 0000000000000046 RBX: 0000000000002710 RCX: 000000000000f7f6
[ 1934.719830] RDX: 00000000005b54ce RSI: 0000000000000002 RDI: 0000000000418958
[ 1934.719830] RBP: ffff88007d503600 R08: 0000000000000002 R09: 0000000000000000
[ 1934.719830] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff81a397c0
[ 1934.719830] R13: ffffffff81a39880 R14: ffff88007d50e7c0 R15: 000001c27649776b
[ 1934.719830] FS:  0000000000000000(0000) GS:ffff88007d500000(0000) knlGS:0000000000000000
[ 1934.719830] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 1934.719830] CR2: 00007ffbf709408a CR3: 0000000001a0b000 CR4: 00000000000007e0
[ 1934.719830] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1934.719830] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 1934.719830] Process swapper/1 (pid: 0, threadinfo ffff880079840000, task ffff880079832f10)
[ 1934.719830] Stack:
[ 1934.719830]  ffff88007d503620 ffffffff8103698a 0000000000000002 ffff88007d50ec80
[ 1934.719830]  ffff88007d503670 ffffffff810e0bf3 0000000000000001 ffff880079832f10
[ 1934.719830]  ffff88007d503690 0000000000000001 0000000000000001 0000000000000000
[ 1934.719830] Call Trace:
[ 1934.719830]  <IRQ> 
[ 1934.719830]  [<ffffffff8103698a>] arch_trigger_all_cpu_backtrace+0x6a/0x90
[ 1934.719830]  [<ffffffff810e0bf3>] __rcu_pending+0x193/0x4e0
[ 1934.719830]  [<ffffffff810e0fba>] rcu_pending+0x7a/0x90
[ 1934.719830]  [<ffffffff810e20c9>] rcu_check_callbacks+0xd9/0x220
[ 1934.719830]  [<ffffffff81062ad8>] update_process_times+0x48/0x90
[ 1934.719830]  [<ffffffff810a79e4>] tick_sched_timer+0x64/0xc0
[ 1934.719830]  [<ffffffff810799d7>] __run_hrtimer+0x77/0x270
[ 1934.719830]  [<ffffffff810a7980>] ? tick_nohz_handler+0x110/0x110
[ 1934.719830]  [<ffffffff8107a6f7>] hrtimer_interrupt+0xd7/0x1f0
[ 1934.719830]  [<ffffffff81493669>] smp_apic_timer_interrupt+0x69/0x99
[ 1934.719830]  [<ffffffff814922ef>] apic_timer_interrupt+0x6f/0x80
[ 1934.719830]  [<ffffffff810a9eaa>] ? lock_is_held+0xaa/0xd0
[ 1934.719830]  [<ffffffff8138e00c>] skb_dst_set_noref+0x4c/0x80
[ 1934.719830]  [<ffffffff813b826e>] ip_route_input_common+0xbce/0xf20
[ 1934.719830]  [<ffffffff813b76a0>] ? ip_route_output_flow+0x70/0x70
[ 1934.719830]  [<ffffffff813ba922>] ip_rcv_finish+0x3d2/0x570
[ 1934.719830]  [<ffffffff813bb254>] ip_rcv+0x214/0x2e0
[ 1934.719830]  [<ffffffff81384836>] __netif_receive_skb+0x5d6/0x690
[ 1934.719830]  [<ffffffff81384351>] ? __netif_receive_skb+0xf1/0x690
[ 1934.719830]  [<ffffffff8135aa07>] ? led_trigger_event+0x27/0x90
[ 1934.719830]  [<ffffffff81384f53>] netif_receive_skb+0x33/0x110
[ 1934.719830]  [<ffffffffa0405d92>] ? ieee80211_data_to_8023+0x182/0x440 [cfg80211]
[ 1934.719830]  [<ffffffffa05b5c53>] ieee80211_deliver_skb.isra.23+0xa3/0x200 [mac80211]
[ 1934.719830]  [<ffffffffa05b6b2b>] ieee80211_rx_handlers+0xd7b/0x2430 [mac80211]
[ 1934.719830]  [<ffffffff810aec2c>] ? trace_hardirqs_on_caller+0xac/0x190
[ 1934.719830]  [<ffffffff810aed1d>] ? trace_hardirqs_on+0xd/0x10
[ 1934.719830]  [<ffffffff81490867>] ? _raw_spin_unlock_irqrestore+0x77/0x80
[ 1934.719830]  [<ffffffffa05b8331>] ieee80211_prepare_and_rx_handle+0x151/0xa70 [mac80211]
[ 1934.719830]  [<ffffffffa05b8f8d>] ieee80211_rx+0x33d/0x8d0 [mac80211]
[ 1934.719830]  [<ffffffffa05b8cf6>] ? ieee80211_rx+0xa6/0x8d0 [mac80211]
[ 1934.719830]  [<ffffffff810aeafc>] ? mark_held_locks+0x8c/0x110
[ 1934.719830]  [<ffffffffa073e994>] ath_rx_tasklet+0xd24/0x1f10 [ath9k]
[ 1934.719830]  [<ffffffffa073b014>] ath9k_tasklet+0xe4/0x160 [ath9k]
[ 1934.719830]  [<ffffffff8105a073>] tasklet_action+0x83/0x110
[ 1934.719830]  [<ffffffff81059498>] __do_softirq+0xc8/0x240
[ 1934.719830]  [<ffffffff81492cec>] call_softirq+0x1c/0x30
[ 1934.719830]  [<ffffffff81016855>] do_softirq+0xa5/0xe0
[ 1934.719830]  [<ffffffff81059986>] irq_exit+0xa6/0xe0
[ 1934.719830]  [<ffffffff81493583>] do_IRQ+0x63/0xe0
[ 1934.719830]  [<ffffffff81490cef>] common_interrupt+0x6f/0x6f
[ 1934.719830]  <EOI> 
[ 1934.719830]  [<ffffffff8101ce63>] ? native_sched_clock+0x13/0x80
[ 1934.719830]  [<ffffffffa032d332>] ? acpi_idle_enter_bm+0x26b/0x2b2 [processor]
[ 1934.719830]  [<ffffffffa032d32e>] ? acpi_idle_enter_bm+0x267/0x2b2 [processor]
[ 1934.719830]  [<ffffffff8135848e>] cpuidle_enter+0x1e/0x20
[ 1934.719830]  [<ffffffff81358ad6>] cpuidle_idle_call+0xa6/0x340
[ 1934.719830]  [<ffffffff8101eee5>] cpu_idle+0xc5/0x140
[ 1934.719830]  [<ffffffff8147d376>] start_secondary+0x208/0x20f
[ 1934.719830] Code: 66 90 ff 15 39 6f 80 00 5d c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 66 66 66 66 90 65 48 8b 14 25 98 3c 01 00 <48> 8d 0c 92 48 8d 04 bd 00 00 00 00 48 89 ca 48 c1 e2 04 48 29 

I am trying to see which option in the 'kernel hacking' section causes this, because using my distro's stock kernel doesn't have any trouble in transmitting/receiving large amounts of data. (Probably CONFIG_DEBUG_ATOMIC_SLEEP).

Quoted 3 lines

PS. If anyone knows how to make minicom ignore page refreshes so that it doesn't obscure the last bit of the crash log when BIOS starts up again, please let me know!

There's a capture option in minicom.

Sujith

#35 lore raw
In reply to Ben Greear
Quoted 11 lines · On Saturday 26 May 2012 17:37:50 Ben Greear wrote:

We're using WPEA-127N. We have a dual-core Atom for one system, and a quad-core i7 CPU (and two wifi NICs) in another system.. So far, the Atom with single NIC is benchmarking better in some tests. We were only using one of the NICs in the i7 for testing, but maybe there is still some interference or maybe we just had bad antenna placement or something.

We've used various Asus and Netgear APs...haven't done throughput tests with home-grown APs running Atheros NICs recently, but will do so next week.

Quoted 5 lines
Quoted 1 line

With a XB112 card (3x3) in HT40 mode, 5Ghz, TX throughput (TCP) can reach 290 Mbps.

What chipset or brand/model is this? I see that XB112 mentioned in the WPEA-127N description, but maybe that is just a form-factor description?

Well, Atheros made some nice presentations about that:
<http://wenku.baidu.com/view/ac0523f57c1cfad6195fa7f4.html>
take a look at slide 12. It has 11n technology (2x2 vs 3x3 ZF vs ML vs CC vs LDPC vs (TxBF) vs range vs tcp throughput all on one diagramm)

Regards,
Chr

#36 lore raw
Quoted 7 lines · Christian Lamparter

Well, Atheros made some nice presentations about that:
<http://wenku.baidu.com/view/ac0523f57c1cfad6195fa7f4.html>
take a look at slide 12. It has 11n technology (2x2 vs 3x3 ZF vs ML vs CC vs LDPC vs (TxBF) vs range vs tcp throughput all on one diagramm)

Well, that's a really helpful document to have as reference for our throughput testing. Sadly, the second source Google knows
(http://wenku.it168.com/d_000141980.shtml) also requires to create an account for downloading the PDF, which I fail after I missed to learn Chinese in time ;(

QCA folks, any chance to post an official download link to these slides?

Cheers, Zefir

#37 lore raw
Quoted 1 line · Zefir Kurtisi

QCA folks, any chance to post an official download link to these slides?

I expect there will never be any such link. Maybe you could get it for yourself if you sign an NDA. It's very secret stuff after all.

//Peter

#38 lore raw
In reply to Ben Greear
Quoted 7 lines · Ben Greear

We've been doing some tests using Atheros stations and various APs. ?The max throughput
we've seen so far is about 237Mbps (received UDP payload on the stations). (Open-Air, AP about 5 feet away, 3x3 MIMO, HT40, 5Ghz, etc).

We are still running lots of different permutations, but I am interested if anyone else has any numbers to share (official or otherwise).

FWIW, FreeBSD was getting 270MBit/sec one-way UDP and 150MBit one-way TCP out of AR9160/AR9280's late last year. I should re-run those tests again now that I've fixed a bunch of things and see if I've regressed. Those are 2T2R devices w/ a Routerstation Pro (AR7161) as the hostap.

At that stage I was maxing out the AR7161 CPU quite badly and filling up all kinds of TX/RX paths, to the point that beacon transmission stopped being reliable. But I haven't really sat down and run performance measurements on MIPS so it's quite possible there's some inefficiencies in FreeBSD that I can work around (to reduce the CPU overhead, I was happy with the throughput.. :)

Sujith, would you mind testing out ath9k/openwrt on a DB120 and see if you can get the same results as our internal LSDK builds?

Thanks,

Adrian

#39 lore raw
Quoted 6 lines · Adrian Chadd
Quoted 5 lines · Ben Greear

We've been doing some tests using Atheros stations and various APs. ?The max throughput
we've seen so far is about 237Mbps (received UDP payload on the stations). (Open-Air, AP about 5 feet away, 3x3 MIMO, HT40, 5Ghz, etc).

Quoted 7 lines
Quoted 2 lines

We are still running lots of different permutations, but I am interested if anyone else has any numbers to share (official or otherwise).

FWIW, FreeBSD was getting 270MBit/sec one-way UDP and 150MBit one-way TCP out of AR9160/AR9280's late last year. I should re-run those tests again now that I've fixed a bunch of things and see if I've regressed. Those are 2T2R devices w/ a Routerstation Pro (AR7161) as the hostap.

270 Mbps with a 2-stream device ? That seems abnormally high. :)

Quoted 9 lines

At that stage I was maxing out the AR7161 CPU quite badly and filling up all kinds of TX/RX paths, to the point that beacon transmission stopped being reliable. But I haven't really sat down and run performance measurements on MIPS so it's quite possible there's some inefficiencies in FreeBSD that I can work around (to reduce the CPU overhead, I was happy with the throughput.. :)

Sujith, would you mind testing out ath9k/openwrt on a DB120 and see if you can get the same results as our internal LSDK builds?

Odd. I get low numbers with a DB120 running OpenWRT (git HEAD).

TCP TX - ~260 Mbps.
TCP RX - ~160 Mbps.

UDP RX - ~240 Mbps.
UDP TX - iperf borks and doesn't display anything.

The load seems to be a bit high (average: 1.92, 1.06, 0.61 ) and the console becomes laggy.

Maybe

#40 lore raw
Quoted 5 lines · Sujith Manoharan

Odd. I get low numbers with a DB120 running OpenWRT (git HEAD).

TCP TX - ~260 Mbps.
TCP RX - ~160 Mbps.

Quoted 7 lines

UDP RX - ~240 Mbps.
UDP TX - iperf borks and doesn't display anything.

The load seems to be a bit high (average: 1.92, 1.06, 0.61 ) and the console becomes laggy.

Maybe

Huh.

Now that I have added "(setq vm-confirm-mail-send t)" to my .emacs, I'll finish the email. :)

Maybe support for DB120 is not complete yet - but am not sure.

Sujith

author
#41 lore raw
Quoted 8 lines · Sujith Manoharan
Quoted 7 lines · Sujith Manoharan

Odd. I get low numbers with a DB120 running OpenWRT (git HEAD).

TCP TX - ~260 Mbps.
TCP RX - ~160 Mbps.

UDP RX - ~240 Mbps.
UDP TX - iperf borks and doesn't display anything.

Maybe NAT won't let it through, if this is coming downstream?

For that matter, from what perspective is the TX or RX?

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#42 lore raw
Quoted 11 lines · Ben Greear
Quoted 8 lines · Sujith Manoharan
Quoted 7 lines · Sujith Manoharan

Odd. I get low numbers with a DB120 running OpenWRT (git HEAD).

TCP TX - ~260 Mbps.
TCP RX - ~160 Mbps.

UDP RX - ~240 Mbps.
UDP TX - iperf borks and doesn't display anything.

Maybe NAT won't let it through, if this is coming downstream?

Ah yes, OpenWRT has default iptables rules - I'll check.

Quoted 1 line

For that matter, from what perspective is the TX or RX?

I was using the ath9k station as the test unit, so:

STA->AP is TX.
AP->STA is RX.

Sujith

#43 lore raw
In reply to Sujith Manoharan
Quoted 12 lines · Sujith Manoharan
Quoted 11 lines · Sujith Manoharan

Odd. I get low numbers with a DB120 running OpenWRT (git HEAD).

TCP TX - ~260 Mbps.
TCP RX - ~160 Mbps.

UDP RX - ~240 Mbps.
UDP TX - iperf borks and doesn't display anything.

The load seems to be a bit high (average: 1.92, 1.06, 0.61 ) and the console becomes laggy.

Quoted 8 lines
Quoted 1 line

Maybe

Huh.

Now that I have added "(setq vm-confirm-mail-send t)" to my .emacs, I'll finish the email. :)

Maybe support for DB120 is not complete yet - but am not sure.

I will look into it.

- Felix

#45 lore raw
In reply to Sujith Manoharan
Quoted 6 lines · Sujith Manoharan
Quoted 4 lines

FWIW, FreeBSD was getting 270MBit/sec one-way UDP and 150MBit one-way TCP out of AR9160/AR9280's late last year. I should re-run those tests again now that I've fixed a bunch of things and see if I've regressed. Those are 2T2R devices w/ a Routerstation Pro (AR7161) as the hostap.

270 Mbps with a 2-stream device ? That seems abnormally high. :)

The 40MHz/MCS15/SGI PHY rate is "300MBit", so I was expecting to get around 250MBit. That was pretty easy. 270MBit was kind of scary and showed all kinds of broken corner cases.

I'll go and try FreeBSD-HEAD again with all the lock debugging disabled and post results. It's quite possible that I'm mis-remembering. :)

Adrian

#46 lore raw
In reply to Adrian Chadd

To all,

Quoted 5 lines · Adrian

FWIW, FreeBSD was getting 270MBit/sec one-way UDP and 150MBit one-way TCP out of AR9160/AR9280's late last year. I should re-run those tests again now that I've fixed a bunch of things and see if I've regressed. Those are 2T2R devices w/ a Routerstation Pro (AR7161) as the hostap.

Quoted 8 lines

At that stage I was maxing out the AR7161 CPU quite badly and filling up all kinds of TX/RX paths, to the point that beacon transmission stopped being reliable. But I haven't really sat down and run performance measurements on MIPS so it's quite possible there's some inefficiencies in FreeBSD that I can work around (to reduce the CPU overhead, I was happy with the throughput..

I have been running iperf tests on some AR9160/AR9280's (Ubiquity AR9220) on Routerstation's (AR7161) running openwrt. In adhoc mode I am getting 100Mbps UDP one-way and 63Mbps TCP one-way. Obviously this is quite a difference from Adrian's measurements.

Checking the various /proc/net/ files I discovered that the sender is having a lot of SndBufErrors.

Can anyone tell me how to track down the source of these errors/drops? I have CONFIG_ATH9K_DEBUG enabled. How can I see into the various queues/buffers in the network stack to see where the problem resides? Any other advice on tracking down this problem?

Thanks in advance,

-Stephen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ath9k.org/pipermail/ath9k-devel/attachments/20120615/cf8415c0/attachment.htm

#47 lore raw
Quoted 4 lines · Stephen Donecker

I have been running iperf tests on some AR9160/AR9280's (Ubiquity AR9220) on Routerstation's (AR7161) running openwrt. In adhoc mode I am getting 100Mbps UDP one-way and 63Mbps TCP one-way. Obviously this is quite a difference from Adrian's measurements.

Eek! Yes, the hardware and NIC is capable of so uch more than that.

Quoted 2 lines

Checking the various /proc/net/ files I discovered that the sender is having a lot of SndBufErrors.

That's a good starting point.

Quoted 4 lines

Can anyone tell me how to track down the source of these errors/drops? I have CONFIG_ATH9K_DEBUG enabled. How can I see into the various queues/buffers in the network stack to see where the problem resides? Any other advice on tracking down this problem?

Hi,

I'd suggest opening up a PR at the kernel.org bugzilla and adding as much debugging information as you can.

Thanks,

Adrian

#48 lore raw

Adrian,

Quoted 3 lines
Quoted 2 lines

Checking the various /proc/net/ files I discovered that the sender is having a lot of SndBufErrors.

That's a good starting point.

I was able to eliminate the SndbuffErrors in /proc/net/snmp by increasing the ifconfig txqueuelen. However I am still only getting 100Mbps UDP one-way.

-Stephen