Skip to contentSkip to conversation

ath10k: improve throughout of tcp/udp TX/RX of sdio

PATCH Applied Wen Gong 4 participants 27 messages Sep 5, 2019 – Oct 14, 2019 Qualcomm Atheros ATH10K wireless driver

v5 of 10 Applied Latest is v10 →

lore
Share & tools
lore

Wen Gong

author
#0 lore raw

v5 cover ath10k: improve throughout of tcp/udp TX/RX of sdio

The bottleneck of throughout on sdio chip is the bus bandwidth, to the patches are all to increase the use ratio of sdio bus.

udp-rx udp-tx tcp-rx tcp-tx

without patches(Mbps)  320        180       170       151
with patches(Mbps)     450        410       400       320

These patches only affect sdio bus chip, explanation is mentioned in each patch's commit log.

Alagu Sankar (1):
ath10k: enable RX bundle receive for sdio
v2: fix incorrect skb tail of rx bundle in ath10k_sdio_mbox_rx_process_packet
v3: change some code style
split fix incorrect skb tail of rx bundle to patch "adjust skb length in ath10k_sdio_mbox_rx_packet"
v4: add err handle in ath10k_sdio_mbox_rx_fetch_bundle
v5: no change

Nicolas Boichat (1):
ath10k: adjust skb length in ath10k_sdio_mbox_rx_packet
v2: no this patch
v3: new added
v4: change commit log
v5: no change

Wen Gong (6):
ath10k: change max RX bundle size from 8 to 32 for sdio
v2: change macro HTC_GET_BUNDLE_COUNT
v3: change some code style
v4: add macro ATH10K_HTC_FLAG_BUNDLE_MASK
v5: no change

ath10k: add workqueue for RX path of sdio
v2: no change
v3: change some code style
v4: no change
v5: no change

ath10k: disable TX complete indication of htt for sdio
v2: change some code style
v3: change some code style
v4: no change
v5: no change

ath10k: add htt TX bundle for sdio
v2: no change
v3: change some code style
v4: no change
v5: change ath10k_htc_setup_tx_req to add check bundle_tx to forbidden init 2 times

ath10k: enable alt data of TX path for sdio
v2: no change
v3: change some code style
v4: add macro ATH10K_HTC_MSG_READY_EXT_ALT_DATA_MASK
v5: no change

ath10k: enable napi on RX path for sdio
v2: no change
v3: change some code style
v4: change some code style
v5: move skb_queue_head_init(&ar->htt.rx_indication_head) from ath10k_htt_connect to ath10k_core_create to forbidden init 2 times

 drivers/net/wireless/ath/ath10k/core.c   |  44 +++-
 drivers/net/wireless/ath/ath10k/core.h   |   4 +-
 drivers/net/wireless/ath/ath10k/hif.h    |   9 +
 drivers/net/wireless/ath/ath10k/htc.c    | 378 ++++++++++++++++++++++++++++---
 drivers/net/wireless/ath/ath10k/htc.h    |  49 +++-
 drivers/net/wireless/ath/ath10k/htt.c    |  13 ++
 drivers/net/wireless/ath/ath10k/htt.h    |  20 +-
 drivers/net/wireless/ath/ath10k/htt_rx.c |  80 ++++++-
 drivers/net/wireless/ath/ath10k/htt_tx.c |  38 +++-
 drivers/net/wireless/ath/ath10k/hw.h     |   2 +-
 drivers/net/wireless/ath/ath10k/sdio.c   | 281 ++++++++++++++++++++---
 drivers/net/wireless/ath/ath10k/sdio.h   |  31 ++-
 12 files changed, 852 insertions(+), 97 deletions(-)
Signature · 7 lines
1.9.1


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
author
#1 lore raw

PATCH v5 1/8 ath10k: adjust skb length in ath10k_sdio_mbox_rx_packet

From: Nicolas Boichat <[email protected]>

When the FW bundles multiple packets, pkt->act_len may be incorrect as it refers to the first packet only (however, the FW will only bundle packets that fit into the same pkt->alloc_len).

Before this patch, the skb length would be set (incorrectly) to pkt->act_len in ath10k_sdio_mbox_rx_packet, and then later manually adjusted in ath10k_sdio_mbox_rx_process_packet.

The first problem is that ath10k_sdio_mbox_rx_process_packet does not use proper skb_put commands to adjust the length (it directly changes skb->len), so we end up with a mismatch between skb->head + skb->tail and skb->data + skb->len. This is quite serious, and causes corruptions in the TCP stack, as the stack tries to coalesce packets, and relies on skb->tail being correct (that is, skb_tail_pointer must point to the first byte_after_ the data).

Instead of re-adjusting the size in ath10k_sdio_mbox_rx_process_packet, this moves the code to ath10k_sdio_mbox_rx_packet, and also add a bounds check, as skb_put would crash the kernel if not enough space is available.

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

Fixes 8530b4e7b22bc3b ("ath10k: sdio: set skb len for all rx packets")

---
v2: no this patch
v3: new added
v4: change commit log
v5: no change

 drivers/net/wireless/ath/ath10k/sdio.c | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)
drivers/net/wireless/ath/ath10k/sdio.c +19 -8
index 8ed4fbd..1127e44 100644
@@ -381,16 +381,11 @@ static int ath10k_sdio_mbox_rx_process_packet(struct ath10k *ar,
 	struct ath10k_htc_hdr *htc_hdr = (struct ath10k_htc_hdr *)skb->data;
 	bool trailer_present = htc_hdr->flags & ATH10K_HTC_FLAG_TRAILER_PRESENT;
 	enum ath10k_htc_ep_id eid;
-	u16 payload_len;
 	u8 *trailer;
 	int ret;
 
-	payload_len = le16_to_cpu(htc_hdr->len);
-	skb->len = payload_len + sizeof(struct ath10k_htc_hdr);
-
 	if (trailer_present) {
-		trailer = skb->data + sizeof(*htc_hdr) +
-			  payload_len - htc_hdr->trailer_len;
+		trailer = skb->data + skb->len - htc_hdr->trailer_len;
 
 		eid = pipe_id_to_eid(htc_hdr->eid);
 
@@ -632,13 +627,29 @@ static int ath10k_sdio_mbox_rx_packet(struct ath10k *ar,
 {
 	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
 	struct sk_buff *skb = pkt->skb;
+	struct ath10k_htc_hdr *htc_hdr;
 	int ret;
 
 	ret = ath10k_sdio_readsb(ar, ar_sdio->mbox_info.htc_addr,
 				 skb->data, pkt->alloc_len);
+
+	if (!ret) {
+		/* Update actual length. The original length may be incorrect,
+		 * as the FW will bundle multiple packets as long as their sizes
+		 * fit within the same aligned length (pkt->alloc_len).
+		 */
+		htc_hdr = (struct ath10k_htc_hdr *)skb->data;
+		pkt->act_len = le16_to_cpu(htc_hdr->len) + sizeof(*htc_hdr);
+		if (pkt->act_len <= pkt->alloc_len) {
+			skb_put(skb, pkt->act_len);
+		} else {
+			ath10k_warn(ar, "rx packet too large (%zu > %zu)\n",
+				    pkt->act_len, pkt->alloc_len);
+			ret = -EMSGSIZE;
+		}
+	}
+
 	pkt->status = ret;
-	if (!ret)
-		skb_put(skb, pkt->act_len);
 
 	return ret;
 }
Signature · 7 lines
1.9.1


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#2 lore raw

Re: [PATCH v5 1/8] ath10k: adjust skb length in ath10k_sdio_mbox_rx_packet

Quoted 29 lines · Wen Gong

From: Nicolas Boichat <[email protected]>

When the FW bundles multiple packets, pkt->act_len may be incorrect as it refers to the first packet only (however, the FW will only bundle packets that fit into the same pkt->alloc_len).

Before this patch, the skb length would be set (incorrectly) to pkt->act_len in ath10k_sdio_mbox_rx_packet, and then later manually adjusted in ath10k_sdio_mbox_rx_process_packet.

The first problem is that ath10k_sdio_mbox_rx_process_packet does not use proper skb_put commands to adjust the length (it directly changes skb->len), so we end up with a mismatch between skb->head + skb->tail and skb->data + skb->len. This is quite serious, and causes corruptions in the TCP stack, as the stack tries to coalesce packets, and relies on skb->tail being correct (that is, skb_tail_pointer must point to the first byte_after_ the data).

Instead of re-adjusting the size in ath10k_sdio_mbox_rx_process_packet, this moves the code to ath10k_sdio_mbox_rx_packet, and also add a bounds check, as skb_put would crash the kernel if not enough space is available.

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

Fixes 8530b4e7b22bc3b ("ath10k: sdio: set skb len for all rx packets")

[...]

Quoted 22 lines
drivers/net/wireless/ath/ath10k/sdio.c +14 -0
@@ -632,13 +627,29 @@ static int ath10k_sdio_mbox_rx_packet(struct ath10k *ar,
 {
 	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
 	struct sk_buff *skb = pkt->skb;
+	struct ath10k_htc_hdr *htc_hdr;
 	int ret;
 
 	ret = ath10k_sdio_readsb(ar, ar_sdio->mbox_info.htc_addr,
 				 skb->data, pkt->alloc_len);
+
+	if (!ret) {
+		/* Update actual length. The original length may be incorrect,
+		 * as the FW will bundle multiple packets as long as their sizes
+		 * fit within the same aligned length (pkt->alloc_len).
+		 */
+		htc_hdr = (struct ath10k_htc_hdr *)skb->data;
+		pkt->act_len = le16_to_cpu(htc_hdr->len) + sizeof(*htc_hdr);
+		if (pkt->act_len <= pkt->alloc_len) {
+			skb_put(skb, pkt->act_len);
+		} else {
+			ath10k_warn(ar, "rx packet too large (%zu > %zu)\n",
+				    pkt->act_len, pkt->alloc_len);

drivers/net/wireless/ath/ath10k/sdio.c static int ath10k_sdio_mbox_rx_packet(struct ath10k *ar, · line 655

Quoted 10 lines
drivers/net/wireless/ath/ath10k/sdio.c +4 -2
+			ret = -EMSGSIZE;
+		}
+	}
+
 	pkt->status = ret;
-	if (!ret)
-		skb_put(skb, pkt->act_len);
 
 	return ret;
 }

I simplified the error handling in this function to follow the ath10k style, please check carefully that I didn't create any bugs:

https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/commit/?h=pending&id=6e4d2cc5

Signature · 6 lines
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#3 lore raw

Re: [PATCH v5 1/8] ath10k: adjust skb length in ath10k_sdio_mbox_rx_packet

Quoted 10 lines · Kalle Valo
Quoted 1 line · Wen Gong

From: Nicolas Boichat <[email protected]>

[...]

I simplified the error handling in this function to follow the ath10k style, please check carefully that I didn't create any bugs:

https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/commit/?h=pending&id=6e4d2cc5

Thanks, that looks right.

Quoted 2 lines

--
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

#4 lore raw

Re: [PATCH v5 1/8] ath10k: adjust skb length in ath10k_sdio_mbox_rx_packet

In reply to Wen Gong
Quoted 21 lines · Wen Gong

When the FW bundles multiple packets, pkt->act_len may be incorrect as it refers to the first packet only (however, the FW will only bundle packets that fit into the same pkt->alloc_len).

Before this patch, the skb length would be set (incorrectly) to pkt->act_len in ath10k_sdio_mbox_rx_packet, and then later manually adjusted in ath10k_sdio_mbox_rx_process_packet.

The first problem is that ath10k_sdio_mbox_rx_process_packet does not use proper skb_put commands to adjust the length (it directly changes skb->len), so we end up with a mismatch between skb->head + skb->tail and skb->data + skb->len. This is quite serious, and causes corruptions in the TCP stack, as the stack tries to coalesce packets, and relies on skb->tail being correct (that is, skb_tail_pointer must point to the first byte_after_ the data).

Instead of re-adjusting the size in ath10k_sdio_mbox_rx_process_packet, this moves the code to ath10k_sdio_mbox_rx_packet, and also add a bounds check, as skb_put would crash the kernel if not enough space is available.

Quoted 7 lines

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

Fixes 8530b4e7b22bc3b ("ath10k: sdio: set skb len for all rx packets")

Patch applied to ath-next branch of ath.git, thanks.

b7139960832e ath10k: adjust skb length in ath10k_sdio_mbox_rx_packet

Signature · 9 lines
https://patchwork.kernel.org/patch/11132657/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#5 lore raw

Re: [PATCH v5 1/8] ath10k: adjust skb length in ath10k_sdio_mbox_rx_packet

In reply to Wen Gong
Quoted 21 lines · Wen Gong

When the FW bundles multiple packets, pkt->act_len may be incorrect as it refers to the first packet only (however, the FW will only bundle packets that fit into the same pkt->alloc_len).

Before this patch, the skb length would be set (incorrectly) to pkt->act_len in ath10k_sdio_mbox_rx_packet, and then later manually adjusted in ath10k_sdio_mbox_rx_process_packet.

The first problem is that ath10k_sdio_mbox_rx_process_packet does not use proper skb_put commands to adjust the length (it directly changes skb->len), so we end up with a mismatch between skb->head + skb->tail and skb->data + skb->len. This is quite serious, and causes corruptions in the TCP stack, as the stack tries to coalesce packets, and relies on skb->tail being correct (that is, skb_tail_pointer must point to the first byte_after_ the data).

Instead of re-adjusting the size in ath10k_sdio_mbox_rx_process_packet, this moves the code to ath10k_sdio_mbox_rx_packet, and also add a bounds check, as skb_put would crash the kernel if not enough space is available.

Quoted 7 lines

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

Fixes 8530b4e7b22bc3b ("ath10k: sdio: set skb len for all rx packets")

Patch applied to ath-next branch of ath.git, thanks.

b7139960832e ath10k: adjust skb length in ath10k_sdio_mbox_rx_packet

Signature · 3 lines
https://patchwork.kernel.org/patch/11132657/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
author
#6 lore raw

PATCH v5 2/8 ath10k: enable RX bundle receive for sdio

In reply to Wen Gong

From: Alagu Sankar <[email protected]>

The existing implementation of initiating multiple sdio transfers for receive bundling is slowing down the receive speed. Combining the transfers using a bundle method would be ideal.

The transmission utilization ratio for sdio bus for small packet is slow, because the space and time cost for sdio bus is same for large length packet and small length packet. So the speed of data for large length packet is higher than small length.

Test result of different length of data:
data packet(byte) cost time(us) calculated rate(Mbps)

      256               28                73
      512               33               124
     1024               35               234
     1792               45               318
    14336              168               682
    28672              333               688
    57344              660               695

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

---
v2: fix incorrect skb tail of rx bundle in ath10k_sdio_mbox_rx_process_packet
v3: change some code style
split fix incorrect skb tail of rx bundle to patch "adjust skb length in ath10k_sdio_mbox_rx_packet"
v4: add err handle in ath10k_sdio_mbox_rx_fetch_bundle
v5: no change

 drivers/net/wireless/ath/ath10k/sdio.c | 108 ++++++++++++++++++++++-----------
 drivers/net/wireless/ath/ath10k/sdio.h |   7 ++-
 2 files changed, 76 insertions(+), 39 deletions(-)
drivers/net/wireless/ath/ath10k/sdio.c +71 -37
index 1127e44..d7ffcd5 100644
@@ -24,6 +24,9 @@
 #include "trace.h"
 #include "sdio.h"
 
+#define ATH10K_SDIO_DMA_BUF_SIZE	(32 * 1024)
+#define ATH10K_SDIO_VSG_BUF_SIZE	(32 * 1024)
+
 /* inlined helper functions */
 
 static inline int ath10k_sdio_calc_txrx_padded_len(struct ath10k_sdio *ar_sdio,
@@ -484,11 +487,11 @@ static int ath10k_sdio_mbox_rx_process_packets(struct ath10k *ar,
 	return ret;
 }
 
-static int ath10k_sdio_mbox_alloc_pkt_bundle(struct ath10k *ar,
-					     struct ath10k_sdio_rx_data *rx_pkts,
-					     struct ath10k_htc_hdr *htc_hdr,
-					     size_t full_len, size_t act_len,
-					     size_t *bndl_cnt)
+static int ath10k_sdio_mbox_alloc_bundle(struct ath10k *ar,
+					 struct ath10k_sdio_rx_data *rx_pkts,
+					 struct ath10k_htc_hdr *htc_hdr,
+					 size_t full_len, size_t act_len,
+					 size_t *bndl_cnt)
 {
 	int ret, i;
 
@@ -529,6 +532,7 @@ static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar,
 	size_t full_len, act_len;
 	bool last_in_bundle;
 	int ret, i;
+	int pkt_cnt = 0;
 
 	if (n_lookaheads > ATH10K_SDIO_MAX_RX_MSGS) {
 		ath10k_warn(ar,
@@ -572,20 +576,22 @@ static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar,
 			 */
 			size_t bndl_cnt;
 
-			ret = ath10k_sdio_mbox_alloc_pkt_bundle(ar,
-								&ar_sdio->rx_pkts[i],
-								htc_hdr,
-								full_len,
-								act_len,
-								&bndl_cnt);
+			struct ath10k_sdio_rx_data *rx_pkts =
+				&ar_sdio->rx_pkts[pkt_cnt];
+
+			ret = ath10k_sdio_mbox_alloc_bundle(ar,
+							    rx_pkts,
+							    htc_hdr,
+							    full_len,
+							    act_len,
+							    &bndl_cnt);
 
 			if (ret) {
 				ath10k_warn(ar, "alloc_bundle error %d\n", ret);
 				goto err;
 			}
 
-			n_lookaheads += bndl_cnt;
-			i += bndl_cnt;
+			pkt_cnt += bndl_cnt;
 			/*Next buffer will be the last in the bundle */
 			last_in_bundle = true;
 		}
@@ -597,7 +603,7 @@ static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar,
 		if (htc_hdr->flags & ATH10K_HTC_FLAGS_RECV_1MORE_BLOCK)
 			full_len += ATH10K_HIF_MBOX_BLOCK_SIZE;
 
-		ret = ath10k_sdio_mbox_alloc_rx_pkt(&ar_sdio->rx_pkts[i],
+		ret = ath10k_sdio_mbox_alloc_rx_pkt(&ar_sdio->rx_pkts[pkt_cnt],
 						    act_len,
 						    full_len,
 						    last_in_bundle,
@@ -606,9 +612,10 @@ static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar,
 			ath10k_warn(ar, "alloc_rx_pkt error %d\n", ret);
 			goto err;
 		}
+		pkt_cnt++;
 	}
 
-	ar_sdio->n_rx_pkts = i;
+	ar_sdio->n_rx_pkts = pkt_cnt;
 
 	return 0;
 
@@ -622,10 +629,10 @@ static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar,
 	return ret;
 }
 
-static int ath10k_sdio_mbox_rx_packet(struct ath10k *ar,
-				      struct ath10k_sdio_rx_data *pkt)
+static int ath10k_sdio_mbox_rx_fetch(struct ath10k *ar)
 {
 	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
+	struct ath10k_sdio_rx_data *pkt = &ar_sdio->rx_pkts[0];
 	struct sk_buff *skb = pkt->skb;
 	struct ath10k_htc_hdr *htc_hdr;
 	int ret;
@@ -633,46 +640,64 @@ static int ath10k_sdio_mbox_rx_packet(struct ath10k *ar,
 	ret = ath10k_sdio_readsb(ar, ar_sdio->mbox_info.htc_addr,
 				 skb->data, pkt->alloc_len);
 
-	if (!ret) {
-		/* Update actual length. The original length may be incorrect,
-		 * as the FW will bundle multiple packets as long as their sizes
-		 * fit within the same aligned length (pkt->alloc_len).
-		 */
-		htc_hdr = (struct ath10k_htc_hdr *)skb->data;
-		pkt->act_len = le16_to_cpu(htc_hdr->len) + sizeof(*htc_hdr);
-		if (pkt->act_len <= pkt->alloc_len) {
-			skb_put(skb, pkt->act_len);
-		} else {
-			ath10k_warn(ar, "rx packet too large (%zu > %zu)\n",
-				    pkt->act_len, pkt->alloc_len);
-			ret = -EMSGSIZE;
-		}
+	if (ret) {
+		ar_sdio->n_rx_pkts = 0;
+		ath10k_sdio_mbox_free_rx_pkt(pkt);
+		return ret;
 	}
 
+	htc_hdr = (struct ath10k_htc_hdr *)skb->data;
+	pkt->act_len = le16_to_cpu(htc_hdr->len) + sizeof(*htc_hdr);
 	pkt->status = ret;
+	skb_put(skb, pkt->act_len);
 
 	return ret;
 }
 
-static int ath10k_sdio_mbox_rx_fetch(struct ath10k *ar)
+static int ath10k_sdio_mbox_rx_fetch_bundle(struct ath10k *ar)
 {
 	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
+	struct ath10k_sdio_rx_data *pkt;
+	struct ath10k_htc_hdr *htc_hdr;
 	int ret, i;
+	u32 pkt_offset, virt_pkt_len;
 
+	virt_pkt_len = 0;
 	for (i = 0; i < ar_sdio->n_rx_pkts; i++) {
-		ret = ath10k_sdio_mbox_rx_packet(ar,
-						 &ar_sdio->rx_pkts[i]);
+		virt_pkt_len += ar_sdio->rx_pkts[i].alloc_len;
+	}
+
+	if (virt_pkt_len < ATH10K_SDIO_DMA_BUF_SIZE) {
+		ret = ath10k_sdio_readsb(ar, ar_sdio->mbox_info.htc_addr,
+					 ar_sdio->vsg_buffer, virt_pkt_len);
 		if (ret)
 			goto err;
+	} else {
+		ath10k_err(ar, "size exceeding limit %d\n", virt_pkt_len);
+		ret = -ENOMEM;
+		goto err;
+	}
+
+	pkt_offset = 0;
+	for (i = 0; i < ar_sdio->n_rx_pkts; i++) {
+		pkt = &ar_sdio->rx_pkts[i];
+		htc_hdr = (struct ath10k_htc_hdr *)(ar_sdio->vsg_buffer + pkt_offset);
+		pkt->act_len = le16_to_cpu(htc_hdr->len) + sizeof(*htc_hdr);
+
+		skb_put_data(pkt->skb, htc_hdr, pkt->act_len);
+		pkt->status = 0;
+		pkt_offset += pkt->alloc_len;
 	}
 
 	return 0;
 
 err:
 	/* Free all packets that was not successfully fetched. */
-	for (; i < ar_sdio->n_rx_pkts; i++)
+	for (i = 0; i < ar_sdio->n_rx_pkts; i++)
 		ath10k_sdio_mbox_free_rx_pkt(&ar_sdio->rx_pkts[i]);
 
+	ar_sdio->n_rx_pkts = 0;
+
 	return ret;
 }
 
@@ -715,7 +740,10 @@ static int ath10k_sdio_mbox_rxmsg_pending_handler(struct ath10k *ar,
 			 */
 			*done = false;
 
-		ret = ath10k_sdio_mbox_rx_fetch(ar);
+		if (ar_sdio->n_rx_pkts > 1)
+			ret = ath10k_sdio_mbox_rx_fetch_bundle(ar);
+		else
+			ret = ath10k_sdio_mbox_rx_fetch(ar);
 
 		/* Process fetched packets. This will potentially update
 		 * n_lookaheads depending on if the packets contain lookahead
@@ -1123,7 +1151,7 @@ static int ath10k_sdio_bmi_get_rx_lookahead(struct ath10k *ar)
 					 MBOX_HOST_INT_STATUS_ADDRESS,
 					 &rx_word);
 		if (ret) {
-			ath10k_warn(ar, "unable to read RX_LOOKAHEAD_VALID: %d\n", ret);
+			ath10k_warn(ar, "unable to read rx_lookahd: %d\n", ret);
 			return ret;
 		}
 
@@ -2018,6 +2046,12 @@ static int ath10k_sdio_probe(struct sdio_func *func,
 		goto err_core_destroy;
 	}
 
+	ar_sdio->vsg_buffer = devm_kmalloc(ar->dev, ATH10K_SDIO_VSG_BUF_SIZE, GFP_KERNEL);
+	if (!ar_sdio->vsg_buffer) {
+		ret = -ENOMEM;
+		goto err_core_destroy;
+	}
+
 	ar_sdio->irq_data.irq_en_reg =
 		devm_kzalloc(ar->dev, sizeof(struct ath10k_sdio_irq_enable_regs),
 			     GFP_KERNEL);
drivers/net/wireless/ath/ath10k/sdio.h +5 -2
index b8c7ac0..4896eca 100644
@@ -138,8 +138,8 @@ struct ath10k_sdio_irq_proc_regs {
 	u8 rx_lookahead_valid;
 	u8 host_int_status2;
 	u8 gmbox_rx_avail;
-	__le32 rx_lookahead[2];
-	__le32 rx_gmbox_lookahead_alias[2];
+	__le32 rx_lookahead[2 * ATH10K_HIF_MBOX_NUM_MAX];
+	__le32 int_status_enable;
 };
 
 struct ath10k_sdio_irq_enable_regs {
@@ -196,6 +196,9 @@ struct ath10k_sdio {
 	struct ath10k *ar;
 	struct ath10k_sdio_irq_data irq_data;
 
+	/* temporary buffer for sdio read */
+	u8 *vsg_buffer;
+
 	/* temporary buffer for BMI requests */
 	u8 *bmi_buf;
 
Signature · 7 lines
1.9.1


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#7 lore raw

Re: [PATCH v5 2/8] ath10k: enable RX bundle receive for sdio

Quoted 26 lines · Wen Gong

From: Alagu Sankar <[email protected]>

The existing implementation of initiating multiple sdio transfers for receive bundling is slowing down the receive speed. Combining the transfers using a bundle method would be ideal.

The transmission utilization ratio for sdio bus for small packet is slow, because the space and time cost for sdio bus is same for large length packet and small length packet. So the speed of data for large length packet is higher than small length.

Test result of different length of data:
data packet(byte) cost time(us) calculated rate(Mbps)

      256               28                73
      512               33               124
     1024               35               234
     1792               45               318
    14336              168               682
    28672              333               688
    57344              660               695

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

[...]

drivers/net/wireless/ath/ath10k/sdio.c line 28

Quoted 8 lines
drivers/net/wireless/ath/ath10k/sdio.c +2 -0
@@ -24,6 +24,9 @@
 #include "trace.h"
 #include "sdio.h"
 
+#define ATH10K_SDIO_DMA_BUF_SIZE	(32 * 1024)
+#define ATH10K_SDIO_VSG_BUF_SIZE	(32 * 1024)

Why two defines? Seems error prone to me and using the latter should be enough.

Quoted 10 lines
drivers/net/wireless/ath/ath10k/sdio.c +1 -0
@@ -529,6 +532,7 @@ static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar,
 	size_t full_len, act_len;
 	bool last_in_bundle;
 	int ret, i;
+	int pkt_cnt = 0;
 
 	if (n_lookaheads > ATH10K_SDIO_MAX_RX_MSGS) {
 		ath10k_warn(ar,
@@ -572,20 +576,22 @@ static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar,
 			 */

drivers/net/wireless/ath/ath10k/sdio.c static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar, · line 580

Quoted 10 lines
drivers/net/wireless/ath/ath10k/sdio.c +2 -6
 			size_t bndl_cnt;
 
-			ret = ath10k_sdio_mbox_alloc_pkt_bundle(ar,
-								&ar_sdio->rx_pkts[i],
-								htc_hdr,
-								full_len,
-								act_len,
-								&bndl_cnt);
+			struct ath10k_sdio_rx_data *rx_pkts =
+				&ar_sdio->rx_pkts[pkt_cnt];

You need to declare rx_pkts in the beginning of the block, not mixed within the code.

drivers/net/wireless/ath/ath10k/sdio.c static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar, · line 615

Quoted 5 lines
drivers/net/wireless/ath/ath10k/sdio.c +1 -0
@@ -606,9 +612,10 @@ static int ath10k_sdio_mbox_rx_alloc(struct ath10k *ar,
 			ath10k_warn(ar, "alloc_rx_pkt error %d\n", ret);
 			goto err;
 		}
+		pkt_cnt++;

Empty line before 'pkt_cnt++'.

Quoted 16 lines
drivers/net/wireless/ath/ath10k/sdio.c +8 -3
-static int ath10k_sdio_mbox_rx_fetch(struct ath10k *ar)
+static int ath10k_sdio_mbox_rx_fetch_bundle(struct ath10k *ar)
 {
 	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
+	struct ath10k_sdio_rx_data *pkt;
+	struct ath10k_htc_hdr *htc_hdr;
 	int ret, i;
+	u32 pkt_offset, virt_pkt_len;
 
+	virt_pkt_len = 0;
 	for (i = 0; i < ar_sdio->n_rx_pkts; i++) {
-		ret = ath10k_sdio_mbox_rx_packet(ar,
-						 &ar_sdio->rx_pkts[i]);
+		virt_pkt_len += ar_sdio->rx_pkts[i].alloc_len;
+	}
+

drivers/net/wireless/ath/ath10k/sdio.c static int ath10k_sdio_mbox_rx_packet(struct ath10k *ar, · line 679

Quoted 10 lines
drivers/net/wireless/ath/ath10k/sdio.c +8 -0
+	if (virt_pkt_len < ATH10K_SDIO_DMA_BUF_SIZE) {
+		ret = ath10k_sdio_readsb(ar, ar_sdio->mbox_info.htc_addr,
+					 ar_sdio->vsg_buffer, virt_pkt_len);
 		if (ret)
 			goto err;
+	} else {
+		ath10k_err(ar, "size exceeding limit %d\n", virt_pkt_len);
+		ret = -ENOMEM;
+		goto err;
+	}

Use common error handling style, ath10k_warn() and -E2BIG:

if (virt_pkt_len >= ATH10K_SDIO_DMA_BUF_SIZE) {
	ath10k_err(ar, "size exceeding limit %d\n", virt_pkt_len);
	ret = -E2BIG;
	goto err;
}

ret = ath10k_sdio_readsb(ar, ar_sdio->mbox_info.htc_addr,

			 ar_sdio->vsg_buffer, virt_pkt_len);

if (ret) {
ath10k_warn("failed to do foo: %d", ret)
goto err;
}

drivers/net/wireless/ath/ath10k/sdio.c static int ath10k_sdio_bmi_get_rx_lookahead(struct ath10k *ar) · line 1154

Quoted 6 lines
drivers/net/wireless/ath/ath10k/sdio.c +1 -1
@@ -1123,7 +1151,7 @@ static int ath10k_sdio_bmi_get_rx_lookahead(struct ath10k *ar)
 					 MBOX_HOST_INT_STATUS_ADDRESS,
 					 &rx_word);
 		if (ret) {
-			ath10k_warn(ar, "unable to read RX_LOOKAHEAD_VALID: %d\n", ret);
+			ath10k_warn(ar, "unable to read rx_lookahd: %d\n", ret);

Looks like an unnecessary change?

drivers/net/wireless/ath/ath10k/sdio.c struct ath10k_sdio { · line 200

Quoted 6 lines
drivers/net/wireless/ath/ath10k/sdio.c +2 -0
@@ -196,6 +196,9 @@ struct ath10k_sdio {
 	struct ath10k *ar;
 	struct ath10k_sdio_irq_data irq_data;
 
+	/* temporary buffer for sdio read */
+	u8 *vsg_buffer;

So how is vsg_buffer protected? You should document that here.

Signature · 6 lines
Kalle Valo

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
author
#8 lore raw

PATCH v5 3/8 ath10k: change max RX bundle size from 8 to 32 for sdio

In reply to Wen Gong

The max bundle size support by firmware is 32, change it from 8 to 32 will help performance. This results in significant performance improvement on RX path.

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

---
v2: change macro HTC_GET_BUNDLE_COUNT
v3: change some code style
v4: add macro ATH10K_HTC_FLAG_BUNDLE_MASK
v5: no change

 drivers/net/wireless/ath/ath10k/htc.h  | 12 +++++++++---
 drivers/net/wireless/ath/ath10k/sdio.c |  6 +++---
 drivers/net/wireless/ath/ath10k/sdio.h |  4 ++--
 3 files changed, 14 insertions(+), 8 deletions(-)
drivers/net/wireless/ath/ath10k/htc.h +9 -3
index f55d3ca..7055156 100644
@@ -39,7 +39,7 @@
  * 4-byte aligned.
  */
 
-#define HTC_HOST_MAX_MSG_PER_RX_BUNDLE        8
+#define HTC_HOST_MAX_MSG_PER_RX_BUNDLE        32
 
 enum ath10k_htc_tx_flags {
 	ATH10K_HTC_FLAG_NEED_CREDIT_UPDATE = 0x01,
@@ -48,10 +48,16 @@ enum ath10k_htc_tx_flags {
 
 enum ath10k_htc_rx_flags {
 	ATH10K_HTC_FLAGS_RECV_1MORE_BLOCK = 0x01,
-	ATH10K_HTC_FLAG_TRAILER_PRESENT = 0x02,
-	ATH10K_HTC_FLAG_BUNDLE_MASK     = 0xF0
+	ATH10K_HTC_FLAG_TRAILER_PRESENT = 0x02
 };
 
+#define ATH10K_HTC_FLAG_BUNDLE_MASK 0xF0
+#define ATH10K_HTC_BUNDLE_EXTRA_MASK GENMASK(3, 2)
+
+#define ATH10K_HTC_GET_BUNDLE_COUNT(flags) \
+	    (FIELD_GET(ATH10K_HTC_FLAG_BUNDLE_MASK, (flags)) +  \
+	    (FIELD_GET(ATH10K_HTC_BUNDLE_EXTRA_MASK, (flags)) << 4))
+
 struct ath10k_htc_hdr {
 	u8 eid; /* @enum ath10k_htc_ep_id */
 	u8 flags; /* @enum ath10k_htc_tx_flags, ath10k_htc_rx_flags */
drivers/net/wireless/ath/ath10k/sdio.c +3 -3
index d7ffcd5..714b0b1 100644
@@ -24,8 +24,8 @@
 #include "trace.h"
 #include "sdio.h"
 
-#define ATH10K_SDIO_DMA_BUF_SIZE	(32 * 1024)
-#define ATH10K_SDIO_VSG_BUF_SIZE	(32 * 1024)
+#define ATH10K_SDIO_DMA_BUF_SIZE	(64 * 1024)
+#define ATH10K_SDIO_VSG_BUF_SIZE	(64 * 1024)
 
 /* inlined helper functions */
 
@@ -495,7 +495,7 @@ static int ath10k_sdio_mbox_alloc_bundle(struct ath10k *ar,
 {
 	int ret, i;
 
-	*bndl_cnt = FIELD_GET(ATH10K_HTC_FLAG_BUNDLE_MASK, htc_hdr->flags);
+	*bndl_cnt = ATH10K_HTC_GET_BUNDLE_COUNT(htc_hdr->flags);
 
 	if (*bndl_cnt > HTC_HOST_MAX_MSG_PER_RX_BUNDLE) {
 		ath10k_warn(ar,
drivers/net/wireless/ath/ath10k/sdio.h +2 -2
index 4896eca..3ca76c7 100644
@@ -89,10 +89,10 @@
  * to the maximum value (HTC_HOST_MAX_MSG_PER_RX_BUNDLE).
  *
  * in this case the driver must allocate
- * (HTC_HOST_MAX_MSG_PER_RX_BUNDLE * HTC_HOST_MAX_MSG_PER_RX_BUNDLE) skb's.
+ * (HTC_HOST_MAX_MSG_PER_RX_BUNDLE * 2) skb's.
  */
 #define ATH10K_SDIO_MAX_RX_MSGS \
-	(HTC_HOST_MAX_MSG_PER_RX_BUNDLE * HTC_HOST_MAX_MSG_PER_RX_BUNDLE)
+	(HTC_HOST_MAX_MSG_PER_RX_BUNDLE * 2)
 
 #define ATH10K_FIFO_TIMEOUT_AND_CHIP_CONTROL   0x00000868u
 #define ATH10K_FIFO_TIMEOUT_AND_CHIP_CONTROL_DISABLE_SLEEP_OFF 0xFFFEFFFF
Signature · 7 lines
1.9.1


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#9 lore raw

Re: [PATCH v5 3/8] ath10k: change max RX bundle size from 8 to 32 for sdio

Quoted 8 lines · Wen Gong

The max bundle size support by firmware is 32, change it from 8 to 32 will help performance. This results in significant performance improvement on RX path.

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

[...]

Quoted 16 lines
drivers/net/wireless/ath/ath10k/htc.h +1 -2
@@ -39,7 +39,7 @@
  * 4-byte aligned.
  */
 
-#define HTC_HOST_MAX_MSG_PER_RX_BUNDLE        8
+#define HTC_HOST_MAX_MSG_PER_RX_BUNDLE        32
 
 enum ath10k_htc_tx_flags {
 	ATH10K_HTC_FLAG_NEED_CREDIT_UPDATE = 0x01,
@@ -48,10 +48,16 @@ enum ath10k_htc_tx_flags {
 
 enum ath10k_htc_rx_flags {
 	ATH10K_HTC_FLAGS_RECV_1MORE_BLOCK = 0x01,
-	ATH10K_HTC_FLAG_TRAILER_PRESENT = 0x02,

drivers/net/wireless/ath/ath10k/htc.h enum ath10k_htc_tx_flags { · line 59

Quoted 10 lines
drivers/net/wireless/ath/ath10k/htc.h +7 -1
-	ATH10K_HTC_FLAG_BUNDLE_MASK     = 0xF0
+	ATH10K_HTC_FLAG_TRAILER_PRESENT = 0x02
 };
 
+#define ATH10K_HTC_FLAG_BUNDLE_MASK 0xF0
+#define ATH10K_HTC_BUNDLE_EXTRA_MASK GENMASK(3, 2)
+
+#define ATH10K_HTC_GET_BUNDLE_COUNT(flags) \
+	    (FIELD_GET(ATH10K_HTC_FLAG_BUNDLE_MASK, (flags)) +  \
+	    (FIELD_GET(ATH10K_HTC_BUNDLE_EXTRA_MASK, (flags)) << 4))

When using FIELD_GET() there should be no need to use '<<', the main id of FIELD_GET() and BITMASK() is that there's no need do any manual shifting. So I suspect something is wrong here, but didn't have time to investigate.

Signature · 6 lines
Kalle Valo

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
author
#10 lore raw

Re: [PATCH v5 3/8] ath10k: change max RX bundle size from 8 to 32 for sdio

Quoted 12 lines · Kalle Valo

drivers/net/wireless/ath/ath10k/htc.h enum ath10k_htc_tx_flags { · line 59

Quoted 7 lines
drivers/net/wireless/ath/ath10k/htc.h +6 -0
+#define ATH10K_HTC_FLAG_BUNDLE_MASK 0xF0
+#define ATH10K_HTC_BUNDLE_EXTRA_MASK GENMASK(3, 2)
+
+#define ATH10K_HTC_GET_BUNDLE_COUNT(flags) \
+	    (FIELD_GET(ATH10K_HTC_FLAG_BUNDLE_MASK, (flags)) +  \
+	    (FIELD_GET(ATH10K_HTC_BUNDLE_EXTRA_MASK, (flags)) << 4))

When using FIELD_GET() there should be no need to use '<<', the main id of FIELD_GET() and BITMASK() is that there's no need do any manual shifting. So I suspect something is wrong here, but didn't have time to investigate.

the value is like this:
the u8 flags:
b7 b6 b5 b4 b3 b2 b1 b0
the count= 0Bb7b6b5b4 + 0Bb3b2 << 4 FIELD_GET is to >>, not <<
so it need to manual shift for the "0Bb3b2 << 4"

_______________________________________________
ath10k mailing list [email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

author
#11 lore raw

PATCH v5 4/8 ath10k: add workqueue for RX path of sdio

In reply to Wen Gong

The thread of read rx message by sdio bus from firmware is synchronous, it will cost much time for process the left part of rx message which includes indicate the rx packet to uppper net stack. It will reduce the time of read from sdio.

This patch move the indication to a workqueue, it results in significant performance improvement on RX path.

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

---
v2: no change
v3: change some code style
v4: no change
v5: no change

 drivers/net/wireless/ath/ath10k/sdio.c | 116 ++++++++++++++++++++++++++++++++-
 drivers/net/wireless/ath/ath10k/sdio.h |  20 ++++++
 2 files changed, 133 insertions(+), 3 deletions(-)
drivers/net/wireless/ath/ath10k/sdio.c +113 -3
index 714b0b1..3b808a3 100644
@@ -412,6 +412,66 @@ static int ath10k_sdio_mbox_rx_process_packet(struct ath10k *ar,
 	return 0;
 }
 
+static struct ath10k_sdio_rx_request *ath10k_sdio_alloc_rx_req(struct ath10k *ar)
+{
+	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
+	struct ath10k_sdio_rx_request *rx_req;
+
+	spin_lock_bh(&ar_sdio->rx_lock);
+
+	if (list_empty(&ar_sdio->rx_req_freeq)) {
+		rx_req = NULL;
+		ath10k_dbg(ar, ATH10K_DBG_SDIO, "rx_req alloc fail\n");
+		goto out;
+	}
+
+	rx_req = list_first_entry(&ar_sdio->rx_req_freeq,
+				  struct ath10k_sdio_rx_request, list);
+	list_del(&rx_req->list);
+
+out:
+	spin_unlock_bh(&ar_sdio->rx_lock);
+	return rx_req;
+}
+
+static void ath10k_sdio_free_rx_req(struct ath10k *ar,
+				    struct ath10k_sdio_rx_request *rx_req)
+{
+	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
+
+	memset(rx_req, 0, sizeof(*rx_req));
+
+	spin_lock_bh(&ar_sdio->rx_lock);
+	list_add_tail(&rx_req->list, &ar_sdio->rx_req_freeq);
+	spin_unlock_bh(&ar_sdio->rx_lock);
+}
+
+static int ath10k_sdio_prep_async_rx_req(struct ath10k *ar,
+					 struct sk_buff *skb,
+					 struct ath10k_htc_ep *ep)
+{
+	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
+	struct ath10k_sdio_rx_request *rx_req;
+
+	/* Allocate a rx request for the message and queue it on the
+	 * SDIO rx workqueue.
+	 */
+	rx_req = ath10k_sdio_alloc_rx_req(ar);
+	if (!rx_req) {
+		ath10k_warn(ar, "unable to allocate rx request for async request\n");
+		return -ENOMEM;
+	}
+
+	rx_req->skb = skb;
+	rx_req->ep = ep;
+
+	spin_lock_bh(&ar_sdio->wr_async_lock_rx);
+	list_add_tail(&rx_req->list, &ar_sdio->wr_asyncq_rx);
+	spin_unlock_bh(&ar_sdio->wr_async_lock_rx);
+
+	return 0;
+}
+
 static int ath10k_sdio_mbox_rx_process_packets(struct ath10k *ar,
 					       u32 lookaheads[],
 					       int *n_lookahead)
@@ -465,10 +525,16 @@ static int ath10k_sdio_mbox_rx_process_packets(struct ath10k *ar,
 		if (ret)
 			goto out;
 
-		if (!pkt->trailer_only)
-			ep->ep_ops.ep_rx_complete(ar_sdio->ar, pkt->skb);
-		else
+		if (!pkt->trailer_only) {
+			ret = ath10k_sdio_prep_async_rx_req(ar, pkt->skb, ep);
+			if (ret)
+				kfree_skb(pkt->skb);
+			else
+				queue_work(ar->workqueue_aux,
+					   &ar_sdio->wr_async_work_rx);
+		} else {
 			kfree_skb(pkt->skb);
+		}
 
 		/* The RX complete handler now owns the skb...*/
 		pkt->skb = NULL;
@@ -1319,6 +1385,26 @@ static void __ath10k_sdio_write_async(struct ath10k *ar,
 	ath10k_sdio_free_bus_req(ar, req);
 }
 
+static void ath10k_rx_indication_async_work(struct work_struct *work)
+{
+	struct ath10k_sdio *ar_sdio = container_of(work, struct ath10k_sdio,
+						   wr_async_work_rx);
+	struct ath10k *ar = ar_sdio->ar;
+	struct ath10k_sdio_rx_request *req, *tmp_req;
+
+	spin_lock_bh(&ar_sdio->wr_async_lock_rx);
+
+	list_for_each_entry_safe(req, tmp_req, &ar_sdio->wr_asyncq_rx, list) {
+		list_del(&req->list);
+		spin_unlock_bh(&ar_sdio->wr_async_lock_rx);
+		req->ep->ep_ops.ep_rx_complete(ar, req->skb);
+		ath10k_sdio_free_rx_req(ar, req);
+		spin_lock_bh(&ar_sdio->wr_async_lock_rx);
+	}
+
+	spin_unlock_bh(&ar_sdio->wr_async_lock_rx);
+}
+
 static void ath10k_sdio_write_async_work(struct work_struct *work)
 {
 	struct ath10k_sdio *ar_sdio = container_of(work, struct ath10k_sdio,
@@ -1807,10 +1893,24 @@ static void ath10k_sdio_irq_disable(struct ath10k *ar)
 static void ath10k_sdio_hif_stop(struct ath10k *ar)
 {
 	struct ath10k_sdio_bus_request *req, *tmp_req;
+	struct ath10k_sdio_rx_request *rx_req, *tmp_rx_req;
 	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
 
 	ath10k_sdio_irq_disable(ar);
 
+	cancel_work_sync(&ar_sdio->wr_async_work_rx);
+
+	spin_lock_bh(&ar_sdio->wr_async_lock_rx);
+
+	/* Free all RX requests that have not been handled */
+	list_for_each_entry_safe(rx_req, tmp_rx_req, &ar_sdio->wr_asyncq_rx, list) {
+		list_del(&rx_req->list);
+		rx_req->ep->ep_ops.ep_rx_complete(ar, rx_req->skb);
+		ath10k_sdio_free_rx_req(ar, rx_req);
+	}
+
+	spin_unlock_bh(&ar_sdio->wr_async_lock_rx);
+
 	cancel_work_sync(&ar_sdio->wr_async_work);
 
 	spin_lock_bh(&ar_sdio->wr_async_lock);
@@ -2089,6 +2189,16 @@ static int ath10k_sdio_probe(struct sdio_func *func,
 	for (i = 0; i < ATH10K_SDIO_BUS_REQUEST_MAX_NUM; i++)
 		ath10k_sdio_free_bus_req(ar, &ar_sdio->bus_req[i]);
 
+	spin_lock_init(&ar_sdio->rx_lock);
+	spin_lock_init(&ar_sdio->wr_async_lock_rx);
+	INIT_LIST_HEAD(&ar_sdio->rx_req_freeq);
+	INIT_LIST_HEAD(&ar_sdio->wr_asyncq_rx);
+
+	INIT_WORK(&ar_sdio->wr_async_work_rx, ath10k_rx_indication_async_work);
+
+	for (i = 0; i < ARRAY_SIZE(ar_sdio->rx_req); i++)
+		ath10k_sdio_free_rx_req(ar, &ar_sdio->rx_req[i]);
+
 	dev_id_base = FIELD_GET(QCA_MANUFACTURER_ID_BASE, id->device);
 	switch (dev_id_base) {
 	case QCA_MANUFACTURER_ID_AR6005_BASE:
drivers/net/wireless/ath/ath10k/sdio.h +20 -0
index 3ca76c7..9bb438e 100644
@@ -38,6 +38,7 @@
 
 #define ATH10K_HIF_MBOX_NUM_MAX                 4
 #define ATH10K_SDIO_BUS_REQUEST_MAX_NUM         64
+#define ATH10K_SDIO_RX_REQUEST_MAX_NUM          1024
 
 #define ATH10K_SDIO_HIF_COMMUNICATION_TIMEOUT_HZ (100 * HZ)
 
@@ -98,6 +99,12 @@
 #define ATH10K_FIFO_TIMEOUT_AND_CHIP_CONTROL_DISABLE_SLEEP_OFF 0xFFFEFFFF
 #define ATH10K_FIFO_TIMEOUT_AND_CHIP_CONTROL_DISABLE_SLEEP_ON 0x10000
 
+struct ath10k_sdio_rx_request {
+	struct list_head list;
+	struct sk_buff *skb;
+	struct ath10k_htc_ep *ep;
+};
+
 struct ath10k_sdio_bus_request {
 	struct list_head list;
 
@@ -187,9 +194,17 @@ struct ath10k_sdio {
 	struct ath10k_sdio_bus_request bus_req[ATH10K_SDIO_BUS_REQUEST_MAX_NUM];
 	/* free list of bus requests */
 	struct list_head bus_req_freeq;
+
 	/* protects access to bus_req_freeq */
 	spinlock_t lock;
 
+	/* available rx requests */
+	struct ath10k_sdio_rx_request rx_req[ATH10K_SDIO_RX_REQUEST_MAX_NUM];
+	/* free list of rx requests */
+	struct list_head rx_req_freeq;
+	/* protects access to rx_req_freeq */
+	spinlock_t rx_lock;
+
 	struct ath10k_sdio_rx_data rx_pkts[ATH10K_SDIO_MAX_RX_MSGS];
 	size_t n_rx_pkts;
 
@@ -209,6 +224,11 @@ struct ath10k_sdio {
 	struct list_head wr_asyncq;
 	/* protects access to wr_asyncq */
 	spinlock_t wr_async_lock;
+
+	struct work_struct wr_async_work_rx;
+	struct list_head wr_asyncq_rx;
+	/* protects access to wr_asyncq_rx */
+	spinlock_t wr_async_lock_rx;
 };
 
 static inline struct ath10k_sdio *ath10k_sdio_priv(struct ath10k *ar)
Signature · 7 lines
1.9.1


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#12 lore raw

Re: [PATCH v5 4/8] ath10k: add workqueue for RX path of sdio

Quoted 4 lines · Wen Gong

The thread of read rx message by sdio bus from firmware is synchronous, it will cost much time for process the left part of rx message which includes indicate the rx packet to uppper net stack. It will reduce the time of read from sdio.

This paragraph is hard to read.

Quoted 2 lines

This patch move the indication to a workqueue, it results in significant performance improvement on RX path.

How much is "significant"? Some numbers would make it a lot easier to understand the importance of the changes.

Quoted 4 lines

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

[...]

Quoted 11 lines
drivers/net/wireless/ath/ath10k/sdio.c +11 -0
+static struct ath10k_sdio_rx_request *ath10k_sdio_alloc_rx_req(struct ath10k *ar)
+{
+	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
+	struct ath10k_sdio_rx_request *rx_req;
+
+	spin_lock_bh(&ar_sdio->rx_lock);
+
+	if (list_empty(&ar_sdio->rx_req_freeq)) {
+		rx_req = NULL;
+		ath10k_dbg(ar, ATH10K_DBG_SDIO, "rx_req alloc fail\n");
+		goto out;

drivers/net/wireless/ath/ath10k/sdio.c static int ath10k_sdio_mbox_rx_process_packet(struct ath10k *ar, · line 435

Quoted 10 lines
drivers/net/wireless/ath/ath10k/sdio.c +10 -0
+	}
+
+	rx_req = list_first_entry(&ar_sdio->rx_req_freeq,
+				  struct ath10k_sdio_rx_request, list);
+	list_del(&rx_req->list);
+
+out:
+	spin_unlock_bh(&ar_sdio->rx_lock);
+	return rx_req;
+}

The name of the function is "alloc" but it does not allocate anything?

drivers/net/wireless/ath/ath10k/sdio.c static int ath10k_sdio_mbox_rx_process_packet(struct ath10k *ar, · line 447

Quoted 11 lines
drivers/net/wireless/ath/ath10k/sdio.c +11 -0
+static void ath10k_sdio_free_rx_req(struct ath10k *ar,
+				    struct ath10k_sdio_rx_request *rx_req)
+{
+	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
+
+	memset(rx_req, 0, sizeof(*rx_req));
+
+	spin_lock_bh(&ar_sdio->rx_lock);
+	list_add_tail(&rx_req->list, &ar_sdio->rx_req_freeq);
+	spin_unlock_bh(&ar_sdio->rx_lock);
+}

And this neither frees anything. IMHO that's quite misleading naming. Maybe call _get() and _put()? Or maybe there's some more common naming in kernel?

Quoted 15 lines
drivers/net/wireless/ath/ath10k/sdio.c +15 -0
+static int ath10k_sdio_prep_async_rx_req(struct ath10k *ar,
+					 struct sk_buff *skb,
+					 struct ath10k_htc_ep *ep)
+{
+	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
+	struct ath10k_sdio_rx_request *rx_req;
+
+	/* Allocate a rx request for the message and queue it on the
+	 * SDIO rx workqueue.
+	 */
+	rx_req = ath10k_sdio_alloc_rx_req(ar);
+	if (!rx_req) {
+		ath10k_warn(ar, "unable to allocate rx request for async request\n");
+		return -ENOMEM;
+	}

drivers/net/wireless/ath/ath10k/sdio.c static int ath10k_sdio_mbox_rx_process_packet(struct ath10k *ar, · line 473

Quoted 10 lines
drivers/net/wireless/ath/ath10k/sdio.c +10 -0
+
+	rx_req->skb = skb;
+	rx_req->ep = ep;
+
+	spin_lock_bh(&ar_sdio->wr_async_lock_rx);
+	list_add_tail(&rx_req->list, &ar_sdio->wr_asyncq_rx);
+	spin_unlock_bh(&ar_sdio->wr_async_lock_rx);
+
+	return 0;
+}

Is there enough room in struct ath10k_skb_cb so that you could add struct ath10k_htc_ep there? That way struct ath10k_sdio_rx_request would be useless and you could just use skb_queue_*() functions, which would make this patch a lot simpler.

drivers/net/wireless/ath/ath10k/sdio.h struct ath10k_sdio { · line 232

Quoted 10 lines
drivers/net/wireless/ath/ath10k/sdio.h +5 -0
@@ -209,6 +224,11 @@ struct ath10k_sdio {
 	struct list_head wr_asyncq;
 	/* protects access to wr_asyncq */
 	spinlock_t wr_async_lock;
+
+	struct work_struct wr_async_work_rx;
+	struct list_head wr_asyncq_rx;
+	/* protects access to wr_asyncq_rx */
+	spinlock_t wr_async_lock_rx;
 };

I think the naming is now confusing. I'm guessing "wr_async_lock" means "write asynchronous lock"? So what is wr_asyncq_rx then? It would a lot simpler if we have tx_lock and rx_lock, or something like that.

Naturally all cleanup for wr_async_lock needs to be done in a separate patch.

Signature · 6 lines
Kalle Valo

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
author
#13 lore raw

PATCH v5 5/8 ath10k: disable TX complete indication of htt for sdio

In reply to Wen Gong

Tx complete message from firmware cost bus bandwidth of sdio, and bus bandwidth is the bollteneck of throughput, it will effect the bandwidth occupancy of data packet of TX and RX.

This patch disable TX complete indication from firmware for htt data packet, it results in significant performance improvement on TX path.

The downside of this patch is ath10k will not know the TX status of the data packet for poor signal situation. Although upper network stack or application layer have retry mechanism, the retry will be later than ath10k get the TX fail status if not disable TX complete.

This patch only effect sdio chip, it will not effect PCI, SNOC etc.

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

---
v2: change some code style
v3: change some code style
v4: no change
v5: no change

 drivers/net/wireless/ath/ath10k/core.c   |  9 ++++++++-
 drivers/net/wireless/ath/ath10k/hif.h    |  9 +++++++++
 drivers/net/wireless/ath/ath10k/htc.c    | 10 ++++++++++
 drivers/net/wireless/ath/ath10k/htc.h    |  3 +++
 drivers/net/wireless/ath/ath10k/htt.c    |  5 +++++
 drivers/net/wireless/ath/ath10k/htt.h    | 13 +++++++++++-
 drivers/net/wireless/ath/ath10k/htt_rx.c | 34 +++++++++++++++++++++++++++++++-
 drivers/net/wireless/ath/ath10k/htt_tx.c | 31 +++++++++++++++++++++++++++++
 drivers/net/wireless/ath/ath10k/hw.h     |  2 +-
 drivers/net/wireless/ath/ath10k/sdio.c   | 23 +++++++++++++++++++++
 10 files changed, 135 insertions(+), 4 deletions(-)
drivers/net/wireless/ath/ath10k/core.c +8 -1
index dc45d16..d56dcbd 100644
@@ -30,6 +30,7 @@
 
 static unsigned int ath10k_cryptmode_param;
 static bool uart_print;
+static bool disable_tx_comp = true;
 static bool skip_otp;
 static bool rawmode;
 static bool fw_diag_log;
@@ -41,6 +42,9 @@
 module_param_named(debug_mask, ath10k_debug_mask, uint, 0644);
 module_param_named(cryptmode, ath10k_cryptmode_param, uint, 0644);
 module_param(uart_print, bool, 0644);
+
+/* If upper layer need the TX complete status, it can enable tx complete */
+module_param(disable_tx_comp, bool, 0644);
 module_param(skip_otp, bool, 0644);
 module_param(rawmode, bool, 0644);
 module_param(fw_diag_log, bool, 0644);
@@ -688,7 +692,10 @@ static void ath10k_init_sdio(struct ath10k *ar, enum ath10k_firmware_mode mode)
 	/* Data transfer is not initiated, when reduced Tx completion
 	 * is used for SDIO. disable it until fixed
 	 */
-	param &= ~HI_ACS_FLAGS_SDIO_REDUCE_TX_COMPL_SET;
+	if (disable_tx_comp)
+		param |= HI_ACS_FLAGS_SDIO_REDUCE_TX_COMPL_SET;
+	else
+		param &= ~HI_ACS_FLAGS_SDIO_REDUCE_TX_COMPL_SET;
 
 	/* Alternate credit size of 1544 as used by SDIO firmware is
 	 * not big enough for mac80211 / native wifi frames. disable it
drivers/net/wireless/ath/ath10k/hif.h +9 -0
index 496ee34..0dd8973 100644
@@ -56,6 +56,8 @@ struct ath10k_hif_ops {
 
 	int (*swap_mailbox)(struct ath10k *ar);
 
+	int (*get_htt_tx_complete)(struct ath10k *ar);
+
 	int (*map_service_to_pipe)(struct ath10k *ar, u16 service_id,
 				   u8 *ul_pipe, u8 *dl_pipe);
 
@@ -144,6 +146,13 @@ static inline int ath10k_hif_swap_mailbox(struct ath10k *ar)
 	return 0;
 }
 
+static inline int ath10k_hif_get_htt_tx_complete(struct ath10k *ar)
+{
+	if (ar->hif.ops->get_htt_tx_complete)
+		return ar->hif.ops->get_htt_tx_complete(ar);
+	return 0;
+}
+
 static inline int ath10k_hif_map_service_to_pipe(struct ath10k *ar,
 						 u16 service_id,
 						 u8 *ul_pipe, u8 *dl_pipe)
drivers/net/wireless/ath/ath10k/htc.c +10 -0
index 1d4d1a1..4c6cdc2 100644
@@ -660,6 +660,16 @@ int ath10k_htc_wait_target(struct ath10k_htc *htc)
 	return 0;
 }
 
+void ath10k_htc_change_tx_credit_flow(struct ath10k_htc *htc,
+				      enum ath10k_htc_ep_id eid,
+				      bool enable)
+{
+	struct ath10k *ar = htc->ar;
+	struct ath10k_htc_ep *ep = &ar->htc.endpoint[eid];
+
+	ep->tx_credit_flow_enabled = enable;
+}
+
 int ath10k_htc_connect_service(struct ath10k_htc *htc,
 			       struct ath10k_htc_svc_conn_req *conn_req,
 			       struct ath10k_htc_svc_conn_resp *conn_resp)
drivers/net/wireless/ath/ath10k/htc.h +3 -0
index 7055156..d61eb6c0 100644
@@ -373,6 +373,9 @@ struct ath10k_htc {
 int ath10k_htc_connect_service(struct ath10k_htc *htc,
 			       struct ath10k_htc_svc_conn_req  *conn_req,
 			       struct ath10k_htc_svc_conn_resp *conn_resp);
+void ath10k_htc_change_tx_credit_flow(struct ath10k_htc *htc,
+				      enum ath10k_htc_ep_id eid,
+				      bool enable);
 int ath10k_htc_send(struct ath10k_htc *htc, enum ath10k_htc_ep_id eid,
 		    struct sk_buff *packet);
 struct sk_buff *ath10k_htc_alloc_skb(struct ath10k *ar, int size);
drivers/net/wireless/ath/ath10k/htt.c +5 -0
index 7b75200..4354bf2 100644
@@ -10,6 +10,7 @@
 #include "htt.h"
 #include "core.h"
 #include "debug.h"
+#include "hif.h"
 
 static const enum htt_t2h_msg_type htt_main_t2h_msg_types[] = {
 	[HTT_MAIN_T2H_MSG_TYPE_VERSION_CONF] = HTT_T2H_MSG_TYPE_VERSION_CONF,
@@ -153,6 +154,10 @@ int ath10k_htt_connect(struct ath10k_htt *htt)
 
 	htt->eid = conn_resp.eid;
 
+	htt->disable_tx_comp = ath10k_hif_get_htt_tx_complete(htt->ar);
+	if (htt->disable_tx_comp)
+		ath10k_htc_change_tx_credit_flow(&htt->ar->htc, htt->eid, true);
+
 	return 0;
 }
 
drivers/net/wireless/ath/ath10k/htt.h +12 -1
index 30c0800..889bf9f 100644
@@ -150,9 +150,19 @@ enum htt_data_tx_desc_flags1 {
 	HTT_DATA_TX_DESC_FLAGS1_MORE_IN_BATCH    = 1 << 12,
 	HTT_DATA_TX_DESC_FLAGS1_CKSUM_L3_OFFLOAD = 1 << 13,
 	HTT_DATA_TX_DESC_FLAGS1_CKSUM_L4_OFFLOAD = 1 << 14,
-	HTT_DATA_TX_DESC_FLAGS1_RSVD1            = 1 << 15
+	HTT_DATA_TX_DESC_FLAGS1_TX_COMPLETE      = 1 << 15
 };
 
+#define HTT_TX_CREDIT_DELTA_ABS_M      0xffff0000
+#define HTT_TX_CREDIT_DELTA_ABS_S      16
+#define HTT_TX_CREDIT_DELTA_ABS_GET(word) \
+	    (((word) & HTT_TX_CREDIT_DELTA_ABS_M) >> HTT_TX_CREDIT_DELTA_ABS_S)
+
+#define HTT_TX_CREDIT_SIGN_BIT_M       0x00000100
+#define HTT_TX_CREDIT_SIGN_BIT_S       8
+#define HTT_TX_CREDIT_SIGN_BIT_GET(word) \
+	    (((word) & HTT_TX_CREDIT_SIGN_BIT_M) >> HTT_TX_CREDIT_SIGN_BIT_S)
+
 enum htt_data_tx_ext_tid {
 	HTT_DATA_TX_EXT_TID_NON_QOS_MCAST_BCAST = 16,
 	HTT_DATA_TX_EXT_TID_MGMT                = 17,
@@ -2019,6 +2029,7 @@ struct ath10k_htt {
 	bool tx_mem_allocated;
 	const struct ath10k_htt_tx_ops *tx_ops;
 	const struct ath10k_htt_rx_ops *rx_ops;
+	bool disable_tx_comp;
 };
 
 struct ath10k_htt_tx_ops {
drivers/net/wireless/ath/ath10k/htt_rx.c +33 -1
index 83a7fb6..5bf3179 100644
@@ -3691,6 +3691,9 @@ bool ath10k_htt_t2h_msg_handler(struct ath10k *ar, struct sk_buff *skb)
 	}
 	case HTT_T2H_MSG_TYPE_MGMT_TX_COMPLETION: {
 		struct htt_tx_done tx_done = {};
+		struct ath10k_htt *htt = &ar->htt;
+		struct ath10k_htc *htc = &ar->htc;
+		struct ath10k_htc_ep *ep = &ar->htc.endpoint[htt->eid];
 		int status = __le32_to_cpu(resp->mgmt_tx_completion.status);
 		int info = __le32_to_cpu(resp->mgmt_tx_completion.info);
 
@@ -3716,6 +3719,12 @@ bool ath10k_htt_t2h_msg_handler(struct ath10k *ar, struct sk_buff *skb)
 			break;
 		}
 
+		if (htt->disable_tx_comp) {
+			spin_lock_bh(&htc->tx_lock);
+			ep->tx_credits++;
+			spin_unlock_bh(&htc->tx_lock);
+		}
+
 		status = ath10k_txrx_tx_unref(htt, &tx_done);
 		if (!status) {
 			spin_lock_bh(&htt->tx_lock);
@@ -3790,8 +3799,31 @@ bool ath10k_htt_t2h_msg_handler(struct ath10k *ar, struct sk_buff *skb)
 		skb_queue_tail(&htt->rx_in_ord_compl_q, skb);
 		return false;
 	}
-	case HTT_T2H_MSG_TYPE_TX_CREDIT_UPDATE_IND:
+	case HTT_T2H_MSG_TYPE_TX_CREDIT_UPDATE_IND: {
+		struct ath10k_htt *htt = &ar->htt;
+		struct ath10k_htc *htc = &ar->htc;
+		struct ath10k_htc_ep *ep = &ar->htc.endpoint[htt->eid];
+		u32 msg_word = __le32_to_cpu(*(__le32 *)resp);
+		int htt_credit_delta;
+
+		htt_credit_delta = HTT_TX_CREDIT_DELTA_ABS_GET(msg_word);
+		if (HTT_TX_CREDIT_SIGN_BIT_GET(msg_word))
+			htt_credit_delta = -htt_credit_delta;
+
+		ath10k_dbg(ar, ATH10K_DBG_HTT,
+			   "credit update: delta:%d\n",
+			   htt_credit_delta);
+
+		if (htt->disable_tx_comp) {
+			spin_lock_bh(&htc->tx_lock);
+			ep->tx_credits += htt_credit_delta;
+			spin_unlock_bh(&htc->tx_lock);
+			ath10k_dbg(ar, ATH10K_DBG_HTT,
+				   "credit total:%d\n",
+				   ep->tx_credits);
+		}
 		break;
+	}
 	case HTT_T2H_MSG_TYPE_CHAN_CHANGE: {
 		u32 phymode = __le32_to_cpu(resp->chan_change.phymode);
 		u32 freq = __le32_to_cpu(resp->chan_change.freq);
drivers/net/wireless/ath/ath10k/htt_tx.c +31 -0
index 2ef717f1..7eb753a8 100644
@@ -543,7 +543,35 @@ void ath10k_htt_tx_free(struct ath10k_htt *htt)
 
 void ath10k_htt_htc_tx_complete(struct ath10k *ar, struct sk_buff *skb)
 {
+	struct ath10k_htt *htt = &ar->htt;
+	struct htt_tx_done tx_done = {0};
+	struct htt_cmd_hdr *htt_hdr;
+	struct htt_data_tx_desc *desc_hdr;
+	u16 flags1;
+
 	dev_kfree_skb_any(skb);
+
+	if (!htt->disable_tx_comp)
+		return;
+
+	htt_hdr = (struct htt_cmd_hdr *)skb->data;
+	if (htt_hdr->msg_type != HTT_H2T_MSG_TYPE_TX_FRM)
+		return;
+
+	desc_hdr = (struct htt_data_tx_desc *)
+		(skb->data + sizeof(*htt_hdr));
+	flags1 = __le16_to_cpu(desc_hdr->flags1);
+
+	ath10k_dbg(ar, ATH10K_DBG_HTT,
+		   "htt tx complete msdu id:%u ,flags1:%x\n",
+		   __le16_to_cpu(desc_hdr->id), flags1);
+
+	if (flags1 & HTT_DATA_TX_DESC_FLAGS1_TX_COMPLETE)
+		return;
+
+	tx_done.status = HTT_TX_COMPL_STATE_ACK;
+	tx_done.msdu_id = __le16_to_cpu(desc_hdr->id);
+	ath10k_txrx_tx_unref(&ar->htt, &tx_done);
 }
 
 void ath10k_htt_hif_tx_complete(struct ath10k *ar, struct sk_buff *skb)
@@ -1261,6 +1289,9 @@ static int ath10k_htt_tx_hl(struct ath10k_htt *htt, enum ath10k_hw_txrx_mode txm
 		flags0 |= SM(ATH10K_HW_TXRX_MGMT,
 			     HTT_DATA_TX_DESC_FLAGS0_PKT_TYPE);
 		flags0 |= HTT_DATA_TX_DESC_FLAGS0_MAC_HDR_PRESENT;
+
+		if (htt->disable_tx_comp)
+			flags1 |= HTT_DATA_TX_DESC_FLAGS1_TX_COMPLETE;
 		break;
 	}
 
drivers/net/wireless/ath/ath10k/hw.h +1 -1
index 2ae57c1..6349665 100644
@@ -759,7 +759,7 @@ struct ath10k_hw_ops {
 #define TARGET_TLV_NUM_TDLS_VDEVS		1
 #define TARGET_TLV_NUM_TIDS			((TARGET_TLV_NUM_PEERS) * 2)
 #define TARGET_TLV_NUM_MSDU_DESC		(1024 + 32)
-#define TARGET_TLV_NUM_MSDU_DESC_HL		64
+#define TARGET_TLV_NUM_MSDU_DESC_HL		1024
 #define TARGET_TLV_NUM_WOW_PATTERNS		22
 #define TARGET_TLV_MGMT_NUM_MSDU_DESC		(50)
 
drivers/net/wireless/ath/ath10k/sdio.c +23 -0
index 3b808a3..e01ab63 100644
@@ -1786,6 +1786,28 @@ static int ath10k_sdio_hif_swap_mailbox(struct ath10k *ar)
 	return 0;
 }
 
+static int ath10k_sdio_get_htt_tx_complete(struct ath10k *ar)
+{
+	u32 addr, val;
+	int ret;
+
+	addr = host_interest_item_address(HI_ITEM(hi_acs_flags));
+
+	ret = ath10k_sdio_hif_diag_read32(ar, addr, &val);
+	if (ret) {
+		ath10k_warn(ar,
+			    "unable to read hi_acs_flags for htt tx comple : %d\n", ret);
+		return ret;
+	}
+
+	ret = (val & HI_ACS_FLAGS_SDIO_REDUCE_TX_COMPL_FW_ACK);
+
+	ath10k_dbg(ar, ATH10K_DBG_SDIO, "sdio reduce tx complete fw%sack\n",
+		   ret ? " " : " not ");
+
+	return ret;
+}
+
 /* HIF start/stop */
 
 static int ath10k_sdio_hif_start(struct ath10k *ar)
@@ -2069,6 +2091,7 @@ static void ath10k_sdio_hif_send_complete_check(struct ath10k *ar,
 	.start			= ath10k_sdio_hif_start,
 	.stop			= ath10k_sdio_hif_stop,
 	.swap_mailbox		= ath10k_sdio_hif_swap_mailbox,
+	.get_htt_tx_complete	= ath10k_sdio_get_htt_tx_complete,
 	.map_service_to_pipe	= ath10k_sdio_hif_map_service_to_pipe,
 	.get_default_pipe	= ath10k_sdio_hif_get_default_pipe,
 	.send_complete_check	= ath10k_sdio_hif_send_complete_check,
Signature · 7 lines
1.9.1


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#14 lore raw

Re: [PATCH v5 5/8] ath10k: disable TX complete indication of htt for sdio

Quoted 3 lines · Wen Gong

Tx complete message from firmware cost bus bandwidth of sdio, and bus bandwidth is the bollteneck of throughput, it will effect the bandwidth occupancy of data packet of TX and RX.

TBH I'm not enthuastic about this, this feels an ugly hack. And it adds yet another module parameter which I detest.

Quoted 2 lines

This patch disable TX complete indication from firmware for htt data packet, it results in significant performance improvement on TX path.

So how much does this feature improve through exactly? Do you have any numbers?

Quoted 4 lines

The downside of this patch is ath10k will not know the TX status of the data packet for poor signal situation. Although upper network stack or application layer have retry mechanism, the retry will be later than ath10k get the TX fail status if not disable TX complete.

I don't understand this description. What's the difference in practise from user's point of view?

But I think this patch should dropped from the patchset and revisited after rest of the patches are applied.

Signature · 6 lines
Kalle Valo

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
author
#15 lore raw

PATCH v5 6/8 ath10k: add htt TX bundle for sdio

In reply to Wen Gong

The transmission utilization ratio for sdio bus for small packet is slow, because the space and time cost for sdio bus is same for large length packet and small length packet. So the speed of data for large length packet is higher than small length.

Test result of different length of data:

data packet(byte) cost time(us) calculated rate(Mbps)

      256               28                73
      512               33               124
     1024               35               234
     1792               45               318
    14336              168               682
    28672              333               688
    57344              660               695

This patch change the TX packet from single packet to a large length bundle packet, max size is 32, it results in significant performance improvement on TX path.

This patch only effect sdio chip, it will not effect PCI, SNOC etc. It only enable bundle for sdio chip.

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

---
v2: no change
v3: change some code style
v4: no change
v5: change ath10k_htc_setup_tx_req to add check bundle_tx to forbidden init 2 times

 drivers/net/wireless/ath/ath10k/core.c   |  14 +-
 drivers/net/wireless/ath/ath10k/core.h   |   4 +-
 drivers/net/wireless/ath/ath10k/htc.c    | 356 ++++++++++++++++++++++++++++---
 drivers/net/wireless/ath/ath10k/htc.h    |  21 +-
 drivers/net/wireless/ath/ath10k/htt.c    |   8 +
 drivers/net/wireless/ath/ath10k/htt.h    |   4 +
 drivers/net/wireless/ath/ath10k/htt_rx.c |   1 +
 drivers/net/wireless/ath/ath10k/htt_tx.c |   7 +-
 8 files changed, 378 insertions(+), 37 deletions(-)
drivers/net/wireless/ath/ath10k/core.c +11 -3
index d56dcbd..9841533 100644
@@ -3195,6 +3195,11 @@ struct ath10k *ath10k_core_create(size_t priv_size, struct device *dev,
 	if (!ar->workqueue_aux)
 		goto err_free_wq;
 
+	ar->workqueue_tx_complete =
+		create_singlethread_workqueue("ath10k_tx_complete_wq");
+	if (!ar->workqueue_tx_complete)
+		goto err_free_aux_wq;
+
 	mutex_init(&ar->conf_mutex);
 	mutex_init(&ar->dump_mutex);
 	spin_lock_init(&ar->data_lock);
@@ -3220,7 +3225,7 @@ struct ath10k *ath10k_core_create(size_t priv_size, struct device *dev,
 
 	ret = ath10k_coredump_create(ar);
 	if (ret)
-		goto err_free_aux_wq;
+		goto err_free_tx_complete;
 
 	ret = ath10k_debug_create(ar);
 	if (ret)
@@ -3230,12 +3235,12 @@ struct ath10k *ath10k_core_create(size_t priv_size, struct device *dev,
 
 err_free_coredump:
 	ath10k_coredump_destroy(ar);
-
+err_free_tx_complete:
+	destroy_workqueue(ar->workqueue_tx_complete);
 err_free_aux_wq:
 	destroy_workqueue(ar->workqueue_aux);
 err_free_wq:
 	destroy_workqueue(ar->workqueue);
-
 err_free_mac:
 	ath10k_mac_destroy(ar);
 
@@ -3251,6 +3256,9 @@ void ath10k_core_destroy(struct ath10k *ar)
 	flush_workqueue(ar->workqueue_aux);
 	destroy_workqueue(ar->workqueue_aux);
 
+	flush_workqueue(ar->workqueue_tx_complete);
+	destroy_workqueue(ar->workqueue_tx_complete);
+
 	ath10k_debug_destroy(ar);
 	ath10k_coredump_destroy(ar);
 	ath10k_htt_tx_destroy(&ar->htt);
drivers/net/wireless/ath/ath10k/core.h +3 -1
index 4d7db07..be9eb37 100644
@@ -1079,7 +1079,7 @@ struct ath10k {
 	struct workqueue_struct *workqueue;
 	/* Auxiliary workqueue */
 	struct workqueue_struct *workqueue_aux;
-
+	struct workqueue_struct *workqueue_tx_complete;
 	/* prevents concurrent FW reconfiguration */
 	struct mutex conf_mutex;
 
@@ -1120,6 +1120,8 @@ struct ath10k {
 
 	struct work_struct register_work;
 	struct work_struct restart_work;
+	struct work_struct bundle_tx_work;
+	struct work_struct tx_complete_work;
 
 	/* cycle count is reported twice for each visited channel during scan.
 	 * access protected by data_lock
drivers/net/wireless/ath/ath10k/htc.c +326 -30

421 lines not on this page · raw message

drivers/net/wireless/ath/ath10k/htc.h +19 -2
index d61eb6c0..ec4957a 100644
@@ -40,6 +40,9 @@
  */
 
 #define HTC_HOST_MAX_MSG_PER_RX_BUNDLE        32
+#define HTC_HOST_MAX_MSG_PER_TX_BUNDLE        32
+#define HTC_HOST_MIN_MSG_PER_TX_BUNDLE        2
+#define HTC_HOST_MIN_CREDIT_PER_TX_BUNDLE     2
 
 enum ath10k_htc_tx_flags {
 	ATH10K_HTC_FLAG_NEED_CREDIT_UPDATE = 0x01,
@@ -70,8 +73,14 @@ struct ath10k_htc_hdr {
 		u8 seq_no; /* for tx */
 		u8 control_byte1;
 	} __packed;
-	u8 pad0;
-	u8 pad1;
+	union {
+		__le16 pad_len;
+		struct {
+			u8 pad0;
+			u8 pad1;
+		} __packed;
+	} __packed;
+
 } __packed __aligned(4);
 
 enum ath10k_ath10k_htc_msg_id {
@@ -340,7 +349,12 @@ struct ath10k_htc_ep {
 
 	u8 seq_no; /* for debugging */
 	int tx_credits;
+	int tx_credit_size;
 	bool tx_credit_flow_enabled;
+	bool bundle_tx;
+	struct sk_buff_head tx_req_head;
+	struct sk_buff_head tx_complete_head;
+
 };
 
 struct ath10k_htc_svc_tx_credits {
@@ -369,6 +383,7 @@ struct ath10k_htc {
 
 int ath10k_htc_init(struct ath10k *ar);
 int ath10k_htc_wait_target(struct ath10k_htc *htc);
+void ath10k_htc_setup_tx_req(struct ath10k_htc_ep *ep);
 int ath10k_htc_start(struct ath10k_htc *htc);
 int ath10k_htc_connect_service(struct ath10k_htc *htc,
 			       struct ath10k_htc_svc_conn_req  *conn_req,
@@ -378,6 +393,8 @@ void ath10k_htc_change_tx_credit_flow(struct ath10k_htc *htc,
 				      bool enable);
 int ath10k_htc_send(struct ath10k_htc *htc, enum ath10k_htc_ep_id eid,
 		    struct sk_buff *packet);
+int ath10k_htc_send_hl(struct ath10k_htc *htc, enum ath10k_htc_ep_id eid,
+		       struct sk_buff *packet);
 struct sk_buff *ath10k_htc_alloc_skb(struct ath10k *ar, int size);
 void ath10k_htc_tx_completion_handler(struct ath10k *ar, struct sk_buff *skb);
 void ath10k_htc_rx_completion_handler(struct ath10k *ar, struct sk_buff *skb);
drivers/net/wireless/ath/ath10k/htt.c +8 -0
index 4354bf2..127b4e4 100644
@@ -135,6 +135,8 @@ int ath10k_htt_connect(struct ath10k_htt *htt)
 {
 	struct ath10k_htc_svc_conn_req conn_req;
 	struct ath10k_htc_svc_conn_resp conn_resp;
+	struct ath10k *ar = htt->ar;
+	struct ath10k_htc_ep *ep;
 	int status;
 
 	memset(&conn_req, 0, sizeof(conn_req));
@@ -142,6 +144,7 @@ int ath10k_htt_connect(struct ath10k_htt *htt)
 
 	conn_req.ep_ops.ep_tx_complete = ath10k_htt_htc_tx_complete;
 	conn_req.ep_ops.ep_rx_complete = ath10k_htt_htc_t2h_msg_handler;
+	conn_req.ep_ops.ep_tx_credits = ath10k_htt_op_ep_tx_credits;
 
 	/* connect to control service */
 	conn_req.service_id = ATH10K_HTC_SVC_ID_HTT_DATA_MSG;
@@ -154,6 +157,11 @@ int ath10k_htt_connect(struct ath10k_htt *htt)
 
 	htt->eid = conn_resp.eid;
 
+	if (ar->bus_param.dev_type == ATH10K_DEV_TYPE_HL) {
+		ep = &ar->htc.endpoint[htt->eid];
+		ath10k_htc_setup_tx_req(ep);
+	}
+
 	htt->disable_tx_comp = ath10k_hif_get_htt_tx_complete(htt->ar);
 	if (htt->disable_tx_comp)
 		ath10k_htc_change_tx_credit_flow(&htt->ar->htc, htt->eid, true);
drivers/net/wireless/ath/ath10k/htt.h +4 -0
index 889bf9f..4851a2e 100644
@@ -2030,6 +2030,9 @@ struct ath10k_htt {
 	const struct ath10k_htt_tx_ops *tx_ops;
 	const struct ath10k_htt_rx_ops *rx_ops;
 	bool disable_tx_comp;
+	bool bundle_tx;
+	struct sk_buff_head tx_req_head;
+	struct sk_buff_head tx_complete_head;
 };
 
 struct ath10k_htt_tx_ops {
@@ -2276,6 +2279,7 @@ int ath10k_htt_tx_fetch_resp(struct ath10k *ar,
 			     __le16 fetch_seq_num,
 			     struct htt_tx_fetch_record *records,
 			     size_t num_records);
+void ath10k_htt_op_ep_tx_credits(struct ath10k *ar);
 
 void ath10k_htt_tx_txq_update(struct ieee80211_hw *hw,
 			      struct ieee80211_txq *txq);
drivers/net/wireless/ath/ath10k/htt_rx.c +1 -0
index 5bf3179..09825b1 100644
@@ -3821,6 +3821,7 @@ bool ath10k_htt_t2h_msg_handler(struct ath10k *ar, struct sk_buff *skb)
 			ath10k_dbg(ar, ATH10K_DBG_HTT,
 				   "credit total:%d\n",
 				   ep->tx_credits);
+			ep->ep_ops.ep_tx_credits(htc->ar);
 		}
 		break;
 	}
drivers/net/wireless/ath/ath10k/htt_tx.c +6 -1
index 7eb753a8..e619b33 100644
@@ -541,6 +541,11 @@ void ath10k_htt_tx_free(struct ath10k_htt *htt)
 	ath10k_htt_tx_destroy(htt);
 }
 
+void ath10k_htt_op_ep_tx_credits(struct ath10k *ar)
+{
+	queue_work(ar->workqueue, &ar->bundle_tx_work);
+}
+
 void ath10k_htt_htc_tx_complete(struct ath10k *ar, struct sk_buff *skb)
 {
 	struct ath10k_htt *htt = &ar->htt;
@@ -1357,7 +1362,7 @@ static int ath10k_htt_tx_hl(struct ath10k_htt *htt, enum ath10k_hw_txrx_mode txm
 	 */
 	tx_desc->peerid = __cpu_to_le32(HTT_INVALID_PEERID);
 
-	res = ath10k_htc_send(&htt->ar->htc, htt->eid, msdu);
+	res = ath10k_htc_send_hl(&htt->ar->htc, htt->eid, msdu);
 
 out:
 	return res;
Signature · 7 lines
1.9.1


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
author
#16 lore raw

PATCH v5 7/8 ath10k: enable alt data of TX path for sdio

In reply to Wen Gong

The default credit size is 1792 bytes, but the IP mtu is 1500 bytes, then it has about 290 bytes's waste for each data packet on sdio transfer path for TX bundle, it will reduce the transmission utilization ratio for data packet.

This patch enable the small credit size in firmware, firmware will use the new credit size 1556 bytes, it will increase the transmission utilization ratio for data packet on TX patch. It results in significant performance improvement on TX path.

This patch only effect sdio chip, it will not effect PCI, SNOC etc.

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00017-QCARMSWP-1.

---
v2: no change
v3: change some code style
v4: add macro ATH10K_HTC_MSG_READY_EXT_ALT_DATA_MASK
v5: no change

 drivers/net/wireless/ath/ath10k/core.c | 19 +++++++++++++++++--
 drivers/net/wireless/ath/ath10k/htc.c  | 12 ++++++++++--
 drivers/net/wireless/ath/ath10k/htc.h  | 13 +++++++++++--
 3 files changed, 38 insertions(+), 6 deletions(-)
drivers/net/wireless/ath/ath10k/core.c +17 -2
index 9841533..9fd07c7 100644
@@ -31,6 +31,7 @@
 static unsigned int ath10k_cryptmode_param;
 static bool uart_print;
 static bool disable_tx_comp = true;
+static bool alt_data = true;
 static bool skip_otp;
 static bool rawmode;
 static bool fw_diag_log;
@@ -45,6 +46,15 @@
 
 /* If upper layer need the TX complete status, it can enable tx complete */
 module_param(disable_tx_comp, bool, 0644);
+
+/* alt_data is only used for sdio chip, for previous version of firmware, its
+ * alt data size is 1544 which is not enough for native wifi, so it need to
+ * alt_data for the firmware.
+ * If the firmware has changed alt data size to 1556, then it can enable
+ * alt_data for the firmware.
+ * alt_data will not effect PCI, SNOC etc.
+ */
+module_param(alt_data, bool, 0644);
 module_param(skip_otp, bool, 0644);
 module_param(rawmode, bool, 0644);
 module_param(fw_diag_log, bool, 0644);
@@ -698,9 +708,14 @@ static void ath10k_init_sdio(struct ath10k *ar, enum ath10k_firmware_mode mode)
 		param &= ~HI_ACS_FLAGS_SDIO_REDUCE_TX_COMPL_SET;
 
 	/* Alternate credit size of 1544 as used by SDIO firmware is
-	 * not big enough for mac80211 / native wifi frames. disable it
+	 * not big enough for mac80211 / native wifi frames. disable it.
+	 * If alternate credit size of 1556 as used by SDIO firmware is
+	 * big enough for mac80211 / native wifi frames. enable it
 	 */
-	param &= ~HI_ACS_FLAGS_ALT_DATA_CREDIT_SIZE;
+	if (alt_data && mode == ATH10K_FIRMWARE_MODE_NORMAL)
+		param |= HI_ACS_FLAGS_ALT_DATA_CREDIT_SIZE;
+	else
+		param &= ~HI_ACS_FLAGS_ALT_DATA_CREDIT_SIZE;
 
 	if (mode == ATH10K_FIRMWARE_MODE_UTF)
 		param &= ~HI_ACS_FLAGS_SDIO_SWAP_MAILBOX_SET;
drivers/net/wireless/ath/ath10k/htc.c +10 -2
index 3e551e4..3bf5722 100644
@@ -941,12 +941,16 @@ int ath10k_htc_wait_target(struct ath10k_htc *htc)
 	 */
 	if (htc->control_resp_len >=
 	    sizeof(msg->hdr) + sizeof(msg->ready_ext)) {
+		htc->alt_data_credit_size =
+			__le16_to_cpu(msg->ready_ext.reserved) &
+			ATH10K_HTC_MSG_READY_EXT_ALT_DATA_MASK;
 		htc->max_msgs_per_htc_bundle =
 			min_t(u8, msg->ready_ext.max_msgs_per_htc_bundle,
 			      HTC_HOST_MAX_MSG_PER_RX_BUNDLE);
 		ath10k_dbg(ar, ATH10K_DBG_HTC,
-			   "Extended ready message. RX bundle size: %d\n",
-			   htc->max_msgs_per_htc_bundle);
+			   "Extended ready message. RX bundle size: %d, alt size:%d\n",
+			   htc->max_msgs_per_htc_bundle,
+			   htc->alt_data_credit_size);
 	}
 
 	INIT_WORK(&ar->bundle_tx_work, ath10k_htc_bundle_tx_work);
@@ -1098,6 +1102,10 @@ int ath10k_htc_connect_service(struct ath10k_htc *htc,
 	ep->tx_credits = tx_alloc;
 	ep->tx_credit_size = htc->target_credit_size;
 
+	if (conn_req->service_id == ATH10K_HTC_SVC_ID_HTT_DATA_MSG &&
+	    htc->alt_data_credit_size != 0)
+		ep->tx_credit_size = htc->alt_data_credit_size;
+
 	/* copy all the callbacks */
 	ep->ep_ops = conn_req->ep_ops;
 
drivers/net/wireless/ath/ath10k/htc.h +11 -2
index ec4957a..5c21ec7 100644
@@ -109,6 +109,8 @@ enum ath10k_htc_conn_flags {
 #define ATH10K_HTC_CONN_FLAGS_RECV_ALLOC_LSB  8
 };
 
+#define ATH10K_HTC_MSG_READY_EXT_ALT_DATA_MASK 0xFFF
+
 enum ath10k_htc_conn_svc_status {
 	ATH10K_HTC_CONN_SVC_STATUS_SUCCESS      = 0,
 	ATH10K_HTC_CONN_SVC_STATUS_NOT_FOUND    = 1,
@@ -141,8 +143,14 @@ struct ath10k_htc_ready_extended {
 	struct ath10k_htc_ready base;
 	u8 htc_version; /* @enum ath10k_htc_version */
 	u8 max_msgs_per_htc_bundle;
-	u8 pad0;
-	u8 pad1;
+	union {
+		__le16 reserved;
+		struct {
+			u8 pad0;
+			u8 pad1;
+		} __packed;
+	} __packed;
+
 } __packed;
 
 struct ath10k_htc_conn_svc {
@@ -379,6 +387,7 @@ struct ath10k_htc {
 	int total_transmit_credits;
 	int target_credit_size;
 	u8 max_msgs_per_htc_bundle;
+	int alt_data_credit_size;
 };
 
 int ath10k_htc_init(struct ath10k *ar);
Signature · 7 lines
1.9.1


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
author
#17 lore raw

PATCH v5 8/8 ath10k: enable napi on RX path for sdio

In reply to Wen Gong

For tcp RX, the quantity of tcp acks to remote is 1/2 of the quantity of tcp data from remote, then it will have many small length packets on TX path of sdio bus, then it reduce the RX packets's bandwidth of tcp.

This patch enable napi on RX path, then the RX packet of tcp will not feed to tcp stack immeditely from mac80211 since GRO is enabled by default, it will feed to tcp stack after napi complete, if rx bundle is enabled, then it will feed to tcp stack one time for each bundle of RX. For example, RX bundle size is 32, then tcp stack will receive one large length packet, its length is neary 1500*32, then tcp stack will send a tcp ack for this large packet, this will reduce the tcp acks ratio from 1/2 to 1/32. This results in significant performance improvement for tcp RX.

Tcp rx throughout is 240Mbps without this patch, and it arrive 390Mbps with this patch. The cpu usage has no obvious difference with and without NAPI.

call stack for each RX packet on GRO path:
(skb length is about 1500 bytes)
skb_gro_receive ([kernel.kallsyms]) tcp4_gro_receive ([kernel.kallsyms]) inet_gro_receive ([kernel.kallsyms]) dev_gro_receive ([kernel.kallsyms]) napi_gro_receive ([kernel.kallsyms]) ieee80211_deliver_skb ([mac80211]) ieee80211_rx_handlers ([mac80211]) ieee80211_prepare_and_rx_handle ([mac80211]) ieee80211_rx_napi ([mac80211]) ath10k_htt_rx_proc_rx_ind_hl ([ath10k_core]) ath10k_htt_rx_pktlog_completion_handler ([ath10k_core]) ath10k_sdio_napi_poll ([ath10k_sdio]) net_rx_action ([kernel.kallsyms]) softirqentry_text_start ([kernel.kallsyms]) do_softirq ([kernel.kallsyms])

call stack for napi complete and send tcp ack from tcp stack:
(skb length is about 1500*32 bytes)
_tcp_ack_snd_check ([kernel.kallsyms]) tcp_v4_do_rcv ([kernel.kallsyms])
tcp_v4_rcv ([kernel.kallsyms]) local_deliver_finish ([kernel.kallsyms]) ip_local_deliver ([kernel.kallsyms])
ip_rcv_finish ([kernel.kallsyms])
ip_rcv ([kernel.kallsyms])
netif_receive_skb_core ([kernel.kallsyms]) netif_receive_skb_one_core([kernel.kallsyms]) netif_receive_skb ([kernel.kallsyms]) netif_receive_skb_internal ([kernel.kallsyms]) napi_gro_complete ([kernel.kallsyms]) napi_gro_flush ([kernel.kallsyms]) napi_complete_done ([kernel.kallsyms]) ath10k_sdio_napi_poll ([ath10k_sdio]) net_rx_action ([kernel.kallsyms]) __softirqentry_text_start ([kernel.kallsyms]) do_softirq ([kernel.kallsyms])

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

---
v2: no change
v3: change some code style
v4: change some code style
v5: move skb_queue_head_init(&ar->htt.rx_indication_head) from ath10k_htt_connect to ath10k_core_create to forbidden init 2 times

 drivers/net/wireless/ath/ath10k/core.c   |  2 ++
 drivers/net/wireless/ath/ath10k/htt.h    |  3 +++
 drivers/net/wireless/ath/ath10k/htt_rx.c | 45 ++++++++++++++++++++++++++------
 drivers/net/wireless/ath/ath10k/sdio.c   | 33 +++++++++++++++++++++++
 4 files changed, 75 insertions(+), 8 deletions(-)
drivers/net/wireless/ath/ath10k/core.c +2 -0
index 9fd07c7..aff7812 100644
@@ -3224,6 +3224,8 @@ struct ath10k *ath10k_core_create(size_t priv_size, struct device *dev,
 	init_waitqueue_head(&ar->htt.empty_tx_wq);
 	init_waitqueue_head(&ar->wmi.tx_credits_wq);
 
+	skb_queue_head_init(&ar->htt.rx_indication_head);
+
 	init_completion(&ar->offchan_tx_completed);
 	INIT_WORK(&ar->offchan_tx_work, ath10k_offchan_tx_work);
 	skb_queue_head_init(&ar->offchan_tx_queue);
drivers/net/wireless/ath/ath10k/htt.h +3 -0
index 4851a2e..462a25b 100644
@@ -1879,6 +1879,8 @@ struct ath10k_htt {
 	struct ath10k *ar;
 	enum ath10k_htc_ep_id eid;
 
+	struct sk_buff_head rx_indication_head;
+
 	u8 target_version_major;
 	u8 target_version_minor;
 	struct completion target_version_received;
@@ -2298,6 +2300,7 @@ int ath10k_htt_tx_mgmt_inc_pending(struct ath10k_htt *htt, bool is_mgmt,
 void ath10k_htt_rx_pktlog_completion_handler(struct ath10k *ar,
 					     struct sk_buff *skb);
 int ath10k_htt_txrx_compl_task(struct ath10k *ar, int budget);
+int ath10k_htt_rx_hl_indication(struct ath10k *ar, int budget);
 void ath10k_htt_set_tx_ops(struct ath10k_htt *htt);
 void ath10k_htt_set_rx_ops(struct ath10k_htt *htt);
 #endif
drivers/net/wireless/ath/ath10k/htt_rx.c +37 -8

64 lines not on this page · raw message

drivers/net/wireless/ath/ath10k/sdio.c +33 -0
index e01ab63..7432d1c 100644
@@ -1402,6 +1402,9 @@ static void ath10k_rx_indication_async_work(struct work_struct *work)
 		spin_lock_bh(&ar_sdio->wr_async_lock_rx);
 	}
 
+	if (test_bit(ATH10K_FLAG_CORE_REGISTERED, &ar->dev_flags))
+		napi_schedule(&ar->napi);
+
 	spin_unlock_bh(&ar_sdio->wr_async_lock_rx);
 }
 
@@ -1815,6 +1818,8 @@ static int ath10k_sdio_hif_start(struct ath10k *ar)
 	struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar);
 	int ret;
 
+	napi_enable(&ar->napi);
+
 	/* Sleep 20 ms before HIF interrupts are disabled.
 	 * This will give target plenty of time to process the BMI done
 	 * request before interrupts are disabled.
@@ -1953,6 +1958,9 @@ static void ath10k_sdio_hif_stop(struct ath10k *ar)
 	}
 
 	spin_unlock_bh(&ar_sdio->wr_async_lock);
+
+	napi_synchronize(&ar->napi);
+	napi_disable(&ar->napi);
 }
 
 #ifdef CONFIG_PM
@@ -2129,6 +2137,26 @@ static SIMPLE_DEV_PM_OPS(ath10k_sdio_pm_ops, ath10k_sdio_pm_suspend,
 
 #endif /* CONFIG_PM_SLEEP */
 
+static int ath10k_sdio_napi_poll(struct napi_struct *ctx, int budget)
+{
+	struct ath10k *ar = container_of(ctx, struct ath10k, napi);
+	int done;
+
+	done = ath10k_htt_rx_hl_indication(ar, budget);
+	ath10k_dbg(ar, ATH10K_DBG_SDIO, "napi poll: done: %d, budget:%d\n", done, budget);
+
+	if (done < budget)
+		napi_complete_done(ctx, done);
+
+	return done;
+}
+
+void ath10k_sdio_init_napi(struct ath10k *ar)
+{
+	netif_napi_add(&ar->napi_dev, &ar->napi, ath10k_sdio_napi_poll,
+		       ATH10K_NAPI_BUDGET);
+}
+
 static int ath10k_sdio_probe(struct sdio_func *func,
 			     const struct sdio_device_id *id)
 {
@@ -2154,6 +2182,8 @@ static int ath10k_sdio_probe(struct sdio_func *func,
 		return -ENOMEM;
 	}
 
+	ath10k_sdio_init_napi(ar);
+
 	ath10k_dbg(ar, ATH10K_DBG_BOOT,
 		   "sdio new func %d vendor 0x%x device 0x%x block 0x%x/0x%x\n",
 		   func->num, func->vendor, func->device,
@@ -2274,6 +2304,9 @@ static void ath10k_sdio_remove(struct sdio_func *func)
 		   func->num, func->vendor, func->device);
 
 	ath10k_core_unregister(ar);
+
+	netif_napi_del(&ar->napi);
+
 	ath10k_core_destroy(ar);
 
 	flush_workqueue(ar_sdio->workqueue);
Signature · 7 lines
1.9.1


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#18 lore raw

Re: [PATCH v5 8/8] ath10k: enable napi on RX path for sdio

Quoted 62 lines · Wen Gong

For tcp RX, the quantity of tcp acks to remote is 1/2 of the quantity of tcp data from remote, then it will have many small length packets on TX path of sdio bus, then it reduce the RX packets's bandwidth of tcp.

This patch enable napi on RX path, then the RX packet of tcp will not feed to tcp stack immeditely from mac80211 since GRO is enabled by default, it will feed to tcp stack after napi complete, if rx bundle is enabled, then it will feed to tcp stack one time for each bundle of RX. For example, RX bundle size is 32, then tcp stack will receive one large length packet, its length is neary 1500*32, then tcp stack will send a tcp ack for this large packet, this will reduce the tcp acks ratio from 1/2 to 1/32. This results in significant performance improvement for tcp RX.

Tcp rx throughout is 240Mbps without this patch, and it arrive 390Mbps with this patch. The cpu usage has no obvious difference with and without NAPI.

call stack for each RX packet on GRO path:
(skb length is about 1500 bytes)
skb_gro_receive ([kernel.kallsyms]) tcp4_gro_receive ([kernel.kallsyms]) inet_gro_receive ([kernel.kallsyms]) dev_gro_receive ([kernel.kallsyms]) napi_gro_receive ([kernel.kallsyms]) ieee80211_deliver_skb ([mac80211]) ieee80211_rx_handlers ([mac80211]) ieee80211_prepare_and_rx_handle ([mac80211]) ieee80211_rx_napi ([mac80211]) ath10k_htt_rx_proc_rx_ind_hl ([ath10k_core]) ath10k_htt_rx_pktlog_completion_handler ([ath10k_core]) ath10k_sdio_napi_poll ([ath10k_sdio]) net_rx_action ([kernel.kallsyms]) softirqentry_text_start ([kernel.kallsyms]) do_softirq ([kernel.kallsyms])

call stack for napi complete and send tcp ack from tcp stack:
(skb length is about 1500*32 bytes)
_tcp_ack_snd_check ([kernel.kallsyms]) tcp_v4_do_rcv ([kernel.kallsyms])
tcp_v4_rcv ([kernel.kallsyms]) local_deliver_finish ([kernel.kallsyms]) ip_local_deliver ([kernel.kallsyms])
ip_rcv_finish ([kernel.kallsyms])
ip_rcv ([kernel.kallsyms])
netif_receive_skb_core ([kernel.kallsyms]) netif_receive_skb_one_core([kernel.kallsyms]) netif_receive_skb ([kernel.kallsyms]) netif_receive_skb_internal ([kernel.kallsyms]) napi_gro_complete ([kernel.kallsyms]) napi_gro_flush ([kernel.kallsyms]) napi_complete_done ([kernel.kallsyms]) ath10k_sdio_napi_poll ([ath10k_sdio]) net_rx_action ([kernel.kallsyms]) __softirqentry_text_start ([kernel.kallsyms]) do_softirq ([kernel.kallsyms])

Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.

[...]

drivers/net/wireless/ath/ath10k/htt_rx.c static bool ath10k_htt_rx_proc_rx_ind_hl(struct ath10k_htt *htt, · line 2266

Quoted 8 lines
drivers/net/wireless/ath/ath10k/htt_rx.c +1 -1
@@ -2263,7 +2263,7 @@ static bool ath10k_htt_rx_proc_rx_ind_hl(struct ath10k_htt *htt,
 	if (mpdu_ranges->mpdu_range_status == HTT_RX_IND_MPDU_STATUS_TKIP_MIC_ERR)
 		rx_status->flag |= RX_FLAG_MMIC_ERROR;
 
-	ieee80211_rx_ni(ar->hw, skb);
+	ieee80211_rx_napi(ar->hw, NULL, skb, &ar->napi);

Doesn't this break USB?

Signature · 1 line
Kalle Valo
#19 lore raw
In reply to Wen Gong
Quoted 9 lines · Wen Gong

The bottleneck of throughout on sdio chip is the bus bandwidth, to the patches are all to increase the use ratio of sdio bus.

udp-rx udp-tx tcp-rx tcp-tx

without patches(Mbps)  320        180       170       151
with patches(Mbps)     450        410       400       320

These patches only affect sdio bus chip, explanation is mentioned in each patch's commit log.

I tried to apply patches 2-8, patch 2 had a conflict due to my changes and patch 8 didn't apply at all. Also I saw few warnings with the patches I was able to test:

$ ath10k-check
drivers/net/wireless/ath/ath10k/htc.c: In function 'ath10k_htc_bundle_tx_work':
drivers/net/wireless/ath/ath10k/htc.c:796:24: warning: variable 'eid' set but not used [-Wunused-but-set-variable]
drivers/net/wireless/ath/ath10k/sdio.c:2169:6: warning: no previous prototype for 'ath10k_sdio_check_fw_reg' [-Wmissing-prototypes]
drivers/net/wireless/ath/ath10k/sdio.c: In function 'ath10k_sdio_check_fw_reg':
drivers/net/wireless/ath/ath10k/sdio.c:2171:6: warning: variable 'ret' set but not used [-Wunused-but-set-variable]
drivers/net/wireless/ath/ath10k/sdio.c: In function 'ath10k_sdio_fw_crashed_dump':
drivers/net/wireless/ath/ath10k/sdio.c:2434:17: warning: too many arguments for format [-Wformat-extra-args]
drivers/net/wireless/ath/ath10k/sdio.c:2169:6: warning: symbol 'ath10k_sdio_check_fw_reg' was not declared. Should it be static?
drivers/net/wireless/ath/ath10k/sdio.c: In function 'ath10k_sdio_fw_crashed_dump':
drivers/net/wireless/ath/ath10k/sdio.c:2434:17: warning: too many arguments for format [-Wformat-extra-args]
drivers/net/wireless/ath/ath10k/sdio.c:734: braces {} are not necessary for single statement blocks
drivers/net/wireless/ath/ath10k/sdio.c:969: braces {} are not necessary for single statement blocks

But please don't send a new version until I have provided review comments.

Signature · 6 lines
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#20 lore raw
Quoted 29 lines · Kalle Valo
Quoted 9 lines · Wen Gong

The bottleneck of throughout on sdio chip is the bus bandwidth, to the patches are all to increase the use ratio of sdio bus.

udp-rx udp-tx tcp-rx tcp-tx

without patches(Mbps)  320        180       170       151
with patches(Mbps)     450        410       400       320

These patches only affect sdio bus chip, explanation is mentioned in each patch's commit log.

I tried to apply patches 2-8, patch 2 had a conflict due to my changes and patch 8 didn't apply at all. Also I saw few warnings with the patches I was able to test:

$ ath10k-check
drivers/net/wireless/ath/ath10k/htc.c: In function 'ath10k_htc_bundle_tx_work':
drivers/net/wireless/ath/ath10k/htc.c:796:24: warning: variable 'eid'
set but not used [-Wunused-but-set-variable]
drivers/net/wireless/ath/ath10k/sdio.c:2169:6: warning: no previous
prototype for 'ath10k_sdio_check_fw_reg' [-Wmissing-prototypes]
drivers/net/wireless/ath/ath10k/sdio.c: In function 'ath10k_sdio_check_fw_reg':
drivers/net/wireless/ath/ath10k/sdio.c:2171:6: warning: variable 'ret'
set but not used [-Wunused-but-set-variable]
drivers/net/wireless/ath/ath10k/sdio.c: In function 'ath10k_sdio_fw_crashed_dump':
drivers/net/wireless/ath/ath10k/sdio.c:2434:17: warning: too many
arguments for format [-Wformat-extra-args]
drivers/net/wireless/ath/ath10k/sdio.c:2169:6: warning: symbol
Quoted 8 lines
'ath10k_sdio_check_fw_reg' was not declared. Should it be static?
drivers/net/wireless/ath/ath10k/sdio.c: In function 'ath10k_sdio_fw_crashed_dump':
drivers/net/wireless/ath/ath10k/sdio.c:2434:17: warning: too many
arguments for format [-Wformat-extra-args]
drivers/net/wireless/ath/ath10k/sdio.c:734: braces {} are not
necessary for single statement blocks
drivers/net/wireless/ath/ath10k/sdio.c:969: braces {} are not
necessary for single statement blocks

Actually some of the warnings are from another patch.

Signature · 1 line
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
#21 lore raw
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

The bottleneck of throughout on sdio chip is the bus bandwidth, to the patches are all to increase the use ratio of sdio bus.

I tried to apply patches 2-8, patch 2 had a conflict due to my changes and patch 8 didn't apply at all. Also I saw few warnings with the patches I was able to test:

HI kalle,

i see some warning is from patch "ath10k: add fw coredump for sdio when firmware assert" and this patch also have change in sdio.c, so maybe it will have conflict with the 8 patches.

patch 8 didn't apply at all, is it means each change of the patch is conflict?

I used command to check each patch.
perl ~/opensource/checkpatch.pl --strict --no-tree --max-line-length=90 --show-types --ignore CONST_STRUCT ./*

I found it not check Wunused-but-set-variable.

#22 lore raw
Quoted 7 lines · Wen Gong
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

The bottleneck of throughout on sdio chip is the bus bandwidth, to the patches are all to increase the use ratio of sdio bus.

I tried to apply patches 2-8, patch 2 had a conflict due to my changes and patch 8 didn't apply at all. Also I saw few warnings with the patches I was able to test:

Quoted 8 lines

Hi kalle,

i see some warning is from patch "ath10k: add fw coredump for sdio when firmware assert" and this patch also have change in sdio.c, so maybe it will have
conflict with the 8 patches.

patch 8 didn't apply at all, is it means each change of the patch is conflict?

Patches 1-7 applied fine, but patch 8 didn't apply. I didn't investigate what was the conflict.

Quoted 5 lines

I used command to check each patch.
perl ~/opensource/checkpatch.pl --strict --no-tree --max-line-length=90 --show-types --ignore CONST_STRUCT ./*

I found it not check Wunused-but-set-variable.

checkpatch only checks style issues, unused-but-set-variable is a warning from GCC.

I use ath10k-check script to check all ath10k patches:

https://wireless.wiki.kernel.org/en/users/drivers/ath10k/codingstyle#checking_code

And I use latest GCC and sparse with that. crosstool is a simple way to install a relatively new version of GCC for kernel compilation:

https://mirrors.edge.kernel.org/pub/tools/crosstool/

Signature · 6 lines
Kalle Valo

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Applied
#23 lore raw
In reply to Wen Gong
Quoted 9 lines · Wen Gong

The bottleneck of throughout on sdio chip is the bus bandwidth, to the patches are all to increase the use ratio of sdio bus.

udp-rx udp-tx tcp-rx tcp-tx

without patches(Mbps)  320        180       170       151
with patches(Mbps)     450        410       400       320

These patches only affect sdio bus chip, explanation is mentioned in each patch's commit log.

Below is my summary of the patchset. I recommend splitting these into smaller sets, makes it a lot easier to review and apply. And please send only one or two patchsets at a time.

[PATCH v5 1/8] ath10k: adjust skb length in ath10k_sdio_mbox_rx_packet

Applied.

Patchset 1:

[PATCH v5 2/8] ath10k: enable RX bundle receive for sdio [PATCH v5 3/8] ath10k: change max RX bundle size from 8 to 32 for sdio

Reasonal but needs some cleanup.

Patchset 2:

[PATCH v5 4/8] ath10k: add workqueue for RX path of sdio

Is really another thread needed? We already have one for SDIO.

[PATCH v5 6/8] ath10k: add htt TX bundle for sdio

And again a new thread so that we would have three threads for SDIO? I'm not convinced about that.

Patchset 3:

[PATCH v5 7/8] ath10k: enable alt data of TX path for sdio

Again another module parameter?

[PATCH v5 8/8] ath10k: enable napi on RX path for sdio

Seems reasonable, but worried about breaking USB.

Patchset 4:

[PATCH v5 5/8] ath10k: disable TX complete indication of htt for sdio

Quite hackish and I need numbers how much it really improves throughput

Signature · 1 line
Kalle Valo
author
#24 lore raw
Quoted 18 lines · Kalle Valo
Quoted 9 lines · Wen Gong

The bottleneck of throughout on sdio chip is the bus bandwidth, to the patches are all to increase the use ratio of sdio bus.

udp-rx udp-tx tcp-rx tcp-tx

without patches(Mbps)  320        180       170       151
with patches(Mbps)     450        410       400       320

These patches only affect sdio bus chip, explanation is mentioned in each patch's commit log.

Below is my summary of the patchset. I recommend splitting these into smaller sets, makes it a lot easier to review and apply. And please send only one or two patchsets at a time.

[PATCH v5 1/8] ath10k: adjust skb length in ath10k_sdio_mbox_rx_packet

Quoted 8 lines

Applied.

Patchset 1:

[PATCH v5 2/8] ath10k: enable RX bundle receive for sdio [PATCH v5 3/8] ath10k: change max RX bundle size from 8 to 32 for sdio

Reasonal but needs some cleanup.

[PATCH v5 2/8] I will use sk_buff_head to replace the ath10k_sdio_rx_request, then it will be simple [PATCH v5 3/8] FIELD_GET is to >>, not <<, so << still need by my understand

Quoted 6 lines

Patchset 2:

[PATCH v5 4/8] ath10k: add workqueue for RX path of sdio

Is really another thread needed? We already have one for SDIO.

the SDIO thread is used for async tx, this queue is for RX, and it will improve udp rx from 200Mbps to 400Mbps. And it used the workqueue_aux of ar, not new created. this patch is better to put to Patchset 1, it helps RX, so it should put together with the [PATCH v5 2/8] ath10k: enable RX bundle receive for sdio

Quoted 5 lines

[PATCH v5 6/8] ath10k: add htt TX bundle for sdio

And again a new thread so that we would have three threads for SDIO? I'm not convinced about that.

The thread is for tx complete indication and the thread only used for tx bundle, if it does not have
heavy traffic, then it will not bundle, then the thread will not run.
for bundled tx, after bundled, it has max 32 packets in each bundle, the tx complete for the 32 packets will cost much time, if not give the tx complete task to the thread, then it will much delay the bundle of the next packets, then it will drop throughput.

Quoted 6 lines

Patchset 3:

[PATCH v5 7/8] ath10k: enable alt data of TX path for sdio

Again another module parameter?

the alt_data could be removed

Quoted 4 lines

[PATCH v5 8/8] ath10k: enable napi on RX path for sdio

Seems reasonable, but worried about breaking USB.

it can change to check napi enabled, if not, will use old ieee80211_rx_ni in ath10k_htt_rx_proc_rx_ind_hl

Quoted 6 lines

Patchset 4:

[PATCH v5 5/8] ath10k: disable TX complete indication of htt for sdio

Quite hackish and I need numbers how much it really improves throughput

it will improve throughput,
for udp tx, it can arrive to 400Mbps, if remove this patch, it will drop to 130M. it not only remove the tx complete message's bus bandwidth of sdio, and it also has a improvement in firmware's tx path's logic, it will change the logic
of tx simple both in firmware and ath10k and faster the tx circle.
And the paramter disable_tx_comp can be removed.
this patch shoud be put in Patchset 2, it help TX, so it is better to put together with the [PATCH v5 6/8] ath10k: add htt TX bundle for sdio.

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

author
#25 lore raw
Quoted 2 lines · Wen Gong
Quoted 1 line · Kalle Valo

Wen Gong <[email protected]> writes:

patch v6 ath10k: improve throughout of RX of sdio has sent new patch v6 only have 3 patches, the left patches will also sent later

Alagu Sankar (1):
ath10k: enable RX bundle receive for sdio

Wen Gong (2):
ath10k: change max RX bundle size from 8 to 32 for sdio
ath10k: add workqueue for RX path of sdio

https://patchwork.kernel.org/patch/11160247/
https://patchwork.kernel.org/patch/11160245/
https://patchwork.kernel.org/patch/11160241/

#26 lore raw
Quoted 19 lines

-----Original Message-----
From: ath10k <[email protected]> On Behalf Of Wen Gong
Sent: Thursday, September 26, 2019 10:33 AM
To: [email protected]
Cc: [email protected]; [email protected]
Subject: [EXT] Re: [PATCH v5 0/8] ath10k: improve throughout of tcp/udp TX/RX of sdio

Quoted 2 lines · Wen Gong
Quoted 1 line · Kalle Valo

Wen Gong <[email protected]> writes:

patch v6 ath10k: improve throughout of RX of sdio has sent new patch v6 only have 3 patches, the left patches will also sent later

Alagu Sankar (1):
ath10k: enable RX bundle receive for sdio

Quoted 8 lines

Wen Gong (2):
ath10k: change max RX bundle size from 8 to 32 for sdio
ath10k: add workqueue for RX path of sdio

https://patchwork.kernel.org/patch/11160247/
https://patchwork.kernel.org/patch/11160245/
https://patchwork.kernel.org/patch/11160241/

Left 4 patches sent v6:
https://patchwork.kernel.org/patch/11188393/
https://patchwork.kernel.org/patch/11188403/
https://patchwork.kernel.org/patch/11188405/
https://patchwork.kernel.org/patch/11188407/

Quoted 4 lines

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k