Skip to contentSkip to conversation

wifi: brcmfmac: pcie: S3 and deep sleep on the T2 Mac chips

PATCH New Andre Eikmeyer 1 participant 6 messages Nov 11 Broadcom BRCM80211 IEEE802.11 wireless drivers

lore
Share & tools
lore
#0 lore raw

v2 cover wifi: brcmfmac: pcie: S3 and deep sleep on the T2 Mac chips

This goes on top of Michael Reeves' BCM4388 series [1], which also changes how brcmfmac talks to the BCM4364 and BCM4377 of the T2 Macs. These patches are follow-ups to it.

Michael's Patch 5 moves the mailbox to the control ring when the firmware does not set BRCMF_PCIE_SHARED_USE_MAILBOX. On the BCM4377 of the T2 Macs that makes D3 work at all, since this firmware doesn't read the TCM mailbox the driver used before. But that caused some other issues as an aftermath.

Patches 1 and 2 fix two things in that path. D0_INFORM on resume fails while the PCIe state is still DOWN, which I reported on patch 5 [2]. And with host ready as a capability, the host ready doorbell at init halts the BCM4364 firmware.

Patch 3 probes again after a resume through the platform firmware, since the chip loses its firmware in S3. Patch 4 stands on its own and drops a tx status wait on flowring delete that the BCM4377 firmware doesn't satisfy.

Patch 5 is new in v2. The BCM4364 and BCM4377 firmware support inband deep sleep, but the driver only advertises that it has no out-of-band device wake, which leaves deep sleep disabled. The firmware then stays awake and its latency tolerance holds the package out of PC10. With patch 5 the firmware enters deep sleep at idle and the package reaches PC10 on both machines. It also relies on the mailbox over the control ring from Michael's patch 5.

The Bluetooth function of the BCM4377 gets the same resume handling in "Bluetooth: hci_bcm4377: resume after ACPI S3" on linux-bluetooth.

Successfully tested on the BCM4377 (MacBookAir9,1), BCM4364B2 (MacBookPro15,1), and on the BCM4364B3 (MacBookPro16,1).

v1 [3] has not received any feedback yet. Without these patches the T2 Mac chips do not survive S3 on top of Michael's series.

Changes in v2:
- Add patch 5, inband device wake and deep sleep
- Patches 1 to 4 are unchanged

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/20261106153612.2aadef2a@kait2en-mbp15-1/
[3] https://lore.kernel.org/all/[email protected]/

Andre Eikmeyer (5):
wifi: brcmfmac: pcie: set the state before D0_INFORM on resume
wifi: brcmfmac: pcie: skip the host ready doorbell at init once it is
a capability
wifi: brcmfmac: pcie: probe again after a resume through firmware
wifi: brcmfmac: msgbuf: send the flowring delete without waiting for
tx status
wifi: brcmfmac: pcie: support inband device wake and deep sleep

 .../broadcom/brcm80211/brcmfmac/commonring.c  |  14 +
 .../broadcom/brcm80211/brcmfmac/commonring.h  |   5 +
 .../broadcom/brcm80211/brcmfmac/msgbuf.c      |  23 +-
 .../broadcom/brcm80211/brcmfmac/pcie.c        | 259 +++++++++++++++++-
 4 files changed, 282 insertions(+), 19 deletions(-)

base-commit: 991a73f4909f1348ffb4b38765cdb522a775e4c2
prerequisite-patch-id: e3427132df0f9740b79d9b3f84bc14c25f787514
prerequisite-patch-id: 1a0aae8b057854813ca6d1f20d3bf153a7b761ec
prerequisite-patch-id: 3453364de54d0d786a6126fca792567f13aa3679
prerequisite-patch-id: d40a127ef83ad7d898ca7c3d643b45b829051477
prerequisite-patch-id: 5bd50905bb42d138eb6fda7d5ad795615348ecdd

#1 lore raw

PATCH v2 1/5 wifi: brcmfmac: pcie: set the state before D0_INFORM on resume

With the mailbox on the control ring, D0_INFORM goes through the ring write path, which refuses to write while devinfo->state is DOWN. brcmf_pcie_pm_leave_D3() sets the state only after D0_INFORM, so every hot resume fails and falls back to remove and re-probe:

brcmf_pcie_send_mb_data: Failed to send H2D mailbox data (-5)

Set the state before D0_INFORM and back to DOWN when the hot resume fails.

---
 drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c +2 -1
index 4de157d18ff..231a2fb89f0 100644
@@ -2838,10 +2838,10 @@ static int brcmf_pcie_pm_leave_D3(struct device *dev)
 	/* Check if device is still up and running, if so we are ready */
 	if (brcmf_pcie_read_pcie32(devinfo, devinfo->reginfo->intmask) != 0) {
 		brcmf_dbg(PCIE, "Try to wakeup device....\n");
+		devinfo->state = BRCMFMAC_PCIE_STATE_UP;
 		if (brcmf_pcie_send_mb_data(devinfo, BRCMF_H2D_HOST_D0_INFORM))
 			goto cleanup;
 		brcmf_dbg(PCIE, "Hot resume, continue....\n");
-		devinfo->state = BRCMFMAC_PCIE_STATE_UP;
 		brcmf_pcie_select_core(devinfo, BCMA_CORE_PCIE2);
 		brcmf_bus_change_state(bus, BRCMF_BUS_UP);
 		brcmf_pcie_intr_enable(devinfo);
@@ -2851,6 +2851,7 @@ static int brcmf_pcie_pm_leave_D3(struct device *dev)
 	}
 
 cleanup:
+	devinfo->state = BRCMFMAC_PCIE_STATE_DOWN;
 	brcmf_chip_detach(devinfo->ci);
 	devinfo->ci = NULL;
 	pdev = devinfo->pdev;
#2 lore raw

PATCH v2 2/5 wifi: brcmfmac: pcie: skip the host ready doorbell at init once it is a capability

In reply to Andre Eikmeyer

With host ready in the host capabilities, the BCM4364 firmware halts on the doorbell that brcmf_pcie_preinit() rings, before it answers the first ioctl. The B3 of the MacBookPro16,1 does so on every load, the B2 of the MacBookPro15,1 on some:

brcmf_pcie_handle_mb_data D2H_MB_DATA: FW HALT
brcmf_attach: dongle is not responding: err=-5

The BCM4377 stays silent without the capability, so it stays. Once host ready is a capability the firmware takes the doorbell as the exit from D3, where Broadcom's DHD driver rings it, after D0_INFORM. Ring it at init only when host ready is not a capability.

---
 .../net/wireless/broadcom/brcm80211/brcmfmac/pcie.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c +11 -2
index 231a2fb89f0..fc0f23477eb 100644
@@ -325,6 +325,7 @@ struct brcmf_pcie_shared_info {
 	dma_addr_t ringupd_dmahandle;
 	u8 version;
 	bool mb_via_ctl;
+	bool hostrdy;
 };
 
 #define BRCMF_OTP_MAX_PARAM_LEN 16
@@ -1599,7 +1600,13 @@ static int brcmf_pcie_preinit(struct device *dev)
 	brcmf_dbg(PCIE, "Enter\n");
 
 	brcmf_pcie_intr_enable(buspub->devinfo);
-	brcmf_pcie_hostready(buspub->devinfo);
+	/*
+	 * Once the host capabilities enable host ready, the firmware takes the
+	 * doorbell as the exit from D3 only. The BCM4364 firmware halts on it
+	 * here.
+	 */
+	if (!buspub->devinfo->shared.hostrdy)
+		brcmf_pcie_hostready(buspub->devinfo);
 
 	return 0;
 }
@@ -1806,8 +1813,10 @@ brcmf_pcie_init_share_ram_info(struct brcmf_pciedev_info *devinfo,
 			!(shared->flags & BRCMF_PCIE_SHARED_USE_MAILBOX);
 
 		host_cap = shared->version;
-		if (shared->flags & BRCMF_PCIE_SHARED_HOSTRDY_DB1)
+		if (shared->flags & BRCMF_PCIE_SHARED_HOSTRDY_DB1) {
 			host_cap |= BRCMF_HOSTCAP_H2D_ENABLE_HOSTRDY;
+			shared->hostrdy = true;
+		}
 		if (shared->flags & BRCMF_PCIE_SHARED_DAR)
 			host_cap |= BRCMF_HOSTCAP_H2D_DAR;
 		host_cap |= BRCMF_HOSTCAP_DS_NO_OOB_DW;
#3 lore raw

PATCH v2 3/5 wifi: brcmfmac: pcie: probe again after a resume through firmware

In reply to Andre Eikmeyer

After ACPI S3 the chip comes back without its firmware, but the interrupt mask can read back nonzero, so brcmf_pcie_pm_leave_D3() takes the hot resume path and every command times out afterwards. Seen on the BCM4377 and BCM4364 of the T2 Macs.

When the platform firmware handled the resume and WoWL is off, go straight to the existing remove and re-probe path.

---
 .../net/wireless/broadcom/brcm80211/brcmfmac/pcie.c    | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c +10 -0
index fc0f23477eb..faa1e16ed4f 100644
@@ -7,6 +7,7 @@
 #include <linux/module.h>
 #include <linux/firmware.h>
 #include <linux/pci.h>
+#include <linux/suspend.h>
 #include <linux/vmalloc.h>
 #include <linux/delay.h>
 #include <linux/interrupt.h>
@@ -2844,6 +2845,15 @@ static int brcmf_pcie_pm_leave_D3(struct device *dev)
 	devinfo = bus->bus_priv.pcie->devinfo;
 	brcmf_dbg(PCIE, "Enter, dev=%p, bus=%p\n", dev, bus);
 
+	/*
+	 * Without WoWL the device is powered off when the platform firmware
+	 * handles the system sleep, as with ACPI S3, and comes back without
+	 * its firmware. Its registers can still read back nonzero, so probe
+	 * it again.
+	 */
+	if (pm_resume_via_firmware() && !devinfo->wowl_enabled)
+		goto cleanup;
+
 	/* Check if device is still up and running, if so we are ready */
 	if (brcmf_pcie_read_pcie32(devinfo, devinfo->reginfo->intmask) != 0) {
 		brcmf_dbg(PCIE, "Try to wakeup device....\n");
#4 lore raw

PATCH v2 4/5 wifi: brcmfmac: msgbuf: send the flowring delete without waiting for tx status

In reply to Andre Eikmeyer

The BCM4377 firmware returns the tx status of queued packets only when it gets the flowring delete request. brcmf_msgbuf_delete_flowring() waits up to 100 ms for them before it sends the request, so every disconnect logs a timeout, and the cleared counter goes below zero when the status arrives.

The race the wait was added for in commit 0fa24196e425 ("wifi: brcmfmac:
fix continuous 802.1x tx pending timeout error") is covered by RING_CLOSING since commit 09be7546a602 ("wifi: brcmfmac: fix scheduling while atomic issue when deleting flowring"), and late tx status still releases the packet. Send the delete right away and reset the counter when the ring is created.

---
 .../broadcom/brcm80211/brcmfmac/msgbuf.c      | 23 ++++++++-----------
 1 file changed, 10 insertions(+), 13 deletions(-)
drivers/net/wireless/broadcom/brcm80211/brcmfmac/msgbuf.c +10 -13
index 8e5c697e7f1..a9ccf647f9a 100644
@@ -73,7 +73,6 @@
 #define BRCMF_MSGBUF_TRICKLE_TXWORKER_THRS	32
 #define BRCMF_MSGBUF_UPDATE_RX_PTR_THRS		48
 
-#define BRCMF_MAX_TXSTATUS_WAIT_RETRIES		10
 
 struct msgbuf_common_hdr {
 	u8				msgtype;
@@ -645,6 +644,7 @@ brcmf_msgbuf_flowring_create_worker(struct brcmf_msgbuf *msgbuf,
 	brcmf_commonring_config(msgbuf->flowrings[flowid],
 				BRCMF_H2D_TXFLOWRING_MAX_ITEM,
 				BRCMF_H2D_TXFLOWRING_ITEMSIZE, dma_buf);
+	atomic_set(&msgbuf->flowrings[flowid]->outstanding_tx, 0);
 
 	commonring = msgbuf->commonrings[BRCMF_H2D_MSGRING_CONTROL_SUBMIT];
 	brcmf_commonring_lock(commonring);
@@ -1330,6 +1330,10 @@ brcmf_msgbuf_process_flow_ring_delete_response(struct brcmf_msgbuf *msgbuf,
 	}
 	brcmf_dbg(MSGBUF, "Flowring %d Delete response status %d\n", flowid,
 		  status);
+	if (atomic_read(&msgbuf->flowrings[flowid]->outstanding_tx))
+		brcmf_dbg(MSGBUF, "Flowring %d deleted with %d tx status outstanding\n",
+			  flowid,
+			  atomic_read(&msgbuf->flowrings[flowid]->outstanding_tx));
 
 	brcmf_msgbuf_remove_flowring(msgbuf, flowid);
 }
@@ -1471,23 +1475,16 @@ void brcmf_msgbuf_delete_flowring(struct brcmf_pub *drvr, u16 flowid)
 	void *ret_ptr;
 	u8 ifidx;
 	int err;
-	int retry = BRCMF_MAX_TXSTATUS_WAIT_RETRIES;
 
-	/* make sure it is not in txflow */
+	/*
+	 * Make sure txflow posts nothing more to the ring. The firmware returns
+	 * the tx status of the posted packets when it deletes the ring, before
+	 * it sends the delete response.
+	 */
 	brcmf_commonring_lock(commonring_del);
 	flow->rings[flowid]->status = RING_CLOSING;
 	brcmf_commonring_unlock(commonring_del);
 
-	/* wait for commonring txflow finished */
-	while (retry && atomic_read(&commonring_del->outstanding_tx)) {
-		usleep_range(5000, 10000);
-		retry--;
-	}
-	if (!retry) {
-		brcmf_err("timed out waiting for txstatus\n");
-		atomic_set(&commonring_del->outstanding_tx, 0);
-	}
-
 	/* no need to submit if firmware can not be reached */
 	if (drvr->bus_if->state != BRCMF_BUS_UP) {
 		brcmf_dbg(MSGBUF, "bus down, flowring will be removed\n");
#5 lore raw

PATCH v2 5/5 wifi: brcmfmac: pcie: support inband device wake and deep sleep

In reply to Andre Eikmeyer

The BCM4364 and BCM4377 firmware report inband deep sleep in the PCIe shared flags. The driver only advertises that it has no out-of-band device wake, which leaves deep sleep disabled. The firmware then stays awake and keeps its latency tolerance short enough to hold the package out of PC10.

Advertise inband device wake when the firmware supports it and follow the handshake of the firmware. The device asks for deep sleep while device wake is deasserted, the host acknowledges it once it has nothing to send, asserts device wake before it uses the rings again and waits for the exit note, and deasserts device wake after 200 ms without ring writes. Hooks around the ring lock cover every host to device ring writer. A system suspend asserts device wake before D3_INFORM and acknowledges no deep sleep request until the resume.

Tested on a MacBookPro15,1 (BCM4364) and a MacBookAir9,1 (BCM4377). The firmware enters and leaves deep sleep about 40 times a minute at idle with Wi-Fi associated, traffic and downloads work, and the package reaches PC10 without ignoring the latency tolerance of the Wi-Fi port.

---
 .../broadcom/brcm80211/brcmfmac/commonring.c  |  14 ++
 .../broadcom/brcm80211/brcmfmac/commonring.h  |   5 +
 .../broadcom/brcm80211/brcmfmac/pcie.c        | 233 +++++++++++++++++-
 3 files changed, 249 insertions(+), 3 deletions(-)
drivers/net/wireless/broadcom/brcm80211/brcmfmac/commonring.c +14 -0
index e44236cb210..881505b992c 100644
@@ -28,6 +28,16 @@ void brcmf_commonring_register_cb(struct brcmf_commonring *commonring,
 }
 
 
+/* Both run outside the ring lock and may sleep */
+void brcmf_commonring_register_pm_cb(struct brcmf_commonring *commonring,
+				     void (*cr_wake)(void *ctx),
+				     void (*cr_idle)(void *ctx))
+{
+	commonring->cr_wake = cr_wake;
+	commonring->cr_idle = cr_idle;
+}
+
+
 void brcmf_commonring_config(struct brcmf_commonring *commonring, u16 depth,
 			     u16 item_len, void *buf_addr)
 {
@@ -53,6 +63,8 @@ void brcmf_commonring_lock(struct brcmf_commonring *commonring)
 {
 	unsigned long flags;
 
+	if (commonring->cr_wake)
+		commonring->cr_wake(commonring->cr_ctx);
 	spin_lock_irqsave(&commonring->lock, flags);
 	commonring->flags = flags;
 }
@@ -62,6 +74,8 @@ void brcmf_commonring_unlock(struct brcmf_commonring *commonring)
 		__releases(&commonring->lock)
 {
 	spin_unlock_irqrestore(&commonring->lock, commonring->flags);
+	if (commonring->cr_idle)
+		commonring->cr_idle(commonring->cr_ctx);
 }
 
 
drivers/net/wireless/broadcom/brcm80211/brcmfmac/commonring.h +5 -0
index 7fb11f4823e..a27be6d7b17 100644
@@ -20,6 +20,8 @@ struct brcmf_commonring {
 	int (*cr_update_wptr)(void *ctx);
 	int (*cr_write_rptr)(void *ctx);
 	int (*cr_write_wptr)(void *ctx);
+	void (*cr_wake)(void *ctx);
+	void (*cr_idle)(void *ctx);
 
 	void *cr_ctx;
 
@@ -38,6 +40,9 @@ void brcmf_commonring_register_cb(struct brcmf_commonring *commonring,
 				  int (*cr_update_wptr)(void *ctx),
 				  int (*cr_write_rptr)(void *ctx),
 				  int (*cr_write_wptr)(void *ctx), void *ctx);
+void brcmf_commonring_register_pm_cb(struct brcmf_commonring *commonring,
+				     void (*cr_wake)(void *ctx),
+				     void (*cr_idle)(void *ctx));
 void brcmf_commonring_config(struct brcmf_commonring *commonring, u16 depth,
 			     u16 item_len, void *buf_addr);
 void brcmf_commonring_lock(struct brcmf_commonring *commonring);
drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c +230 -3

317 lines not on this page · raw message