Skip to contentSkip to conversation

net: wwan: t7xx: PCIe driver for MediaTek M.2 modem

PATCH Superseded Ricardo Martinez 3 participants 22 messages May 6, 2022

Subsystems (5)

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore

Ricardo Martinez

author reviewer
#0 lore raw

v7 cover net: wwan: t7xx: PCIe driver for MediaTek M.2 modem

t7xx is the PCIe host device driver for Intel 5G 5000 M.2 solution which is based on MediaTek's T700 modem to provide WWAN connectivity. The driver uses the WWAN framework infrastructure to create the following control ports and network interfaces:
* /dev/wwan0mbim0 - Interface conforming to the MBIM protocol. Applications like libmbim [1] or Modem Manager [2] from v1.16 onwards with [3][4] can use it to enable data communication towards WWAN.
* /dev/wwan0at0 - Interface that supports AT commands.
* wwan0 - Primary network interface for IP traffic.

The main blocks in t7xx driver are:
* PCIe layer - Implements probe, removal, and power management callbacks.
* Port-proxy - Provides a common interface to interact with different types of ports such as WWAN ports.
* Modem control & status monitor - Implements the entry point for modem initialization, reset and exit, as well as exception handling.
* CLDMA (Control Layer DMA) - Manages the HW used by the port layer to send control messages to the modem using MediaTek's CCCI (Cross-Core Communication Interface) protocol.
* DPMAIF (Data Plane Modem AP Interface) - Controls the HW that provides uplink and downlink queues for the data path. The data exchange takes place using circular buffers to share data buffer addresses and metadata to describe the packets.
* MHCCIF (Modem Host Cross-Core Interface) - Provides interrupt channels for bidirectional event notification such as handshake, exception, PM and port enumeration.

The compilation of the t7xx driver is enabled by the CONFIG_MTK_T7XX config option which depends on CONFIG_WWAN.
This driver was originally developed by MediaTek. Intel adapted t7xx to the WWAN framework, optimized and refactored the driver source code in close collaboration with MediaTek. This will enable getting the t7xx driver on the Approved Vendor List for interested OEM's and ODM's productization plans with Intel 5G 5000 M.2 solution.

List of contributors:
Amir Hanania <[email protected]>
Andriy Shevchenko <[email protected]> Chandrashekar Devegowda <[email protected]> Dinesh Sharma <[email protected]>
Eliot Lee <[email protected]>
Haijun Liu <[email protected]>
M Chetan Kumar <[email protected]>
Mika Westerberg <[email protected]> Moises Veleta <[email protected]> Pierre-louis Bossart <[email protected]> Chiranjeevi Rapolu <[email protected]> Ricardo Martinez <[email protected]> Madhusmita Sahu <[email protected]> Muralidharan Sethuraman <[email protected]> Soumya Prakash Mishra <[email protected]> Sreehari Kancharla <[email protected]> Suresh Nagaraj <[email protected]>

[1] https://www.freedesktop.org/software/libmbim/
[2] https://www.freedesktop.org/software/ModemManager/
[3] https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/merge_requests/582
[4] https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/merge_requests/523

V7:
- Delete unused macros.
- Avoid duplicated calls to le32_to_cpu().
- Fix 'out of bounds' compilation error.
- Rename port_number to port_count.
- Remove '!!' when the destination variable is a boolean.
- Remove unneeded spinlock around rx_length_th.
- Remove common field from union inside dpmaif_drb struct.
- Use 'goto' for the exit flow in t7xx_pci_enable_sleep().
- Merge CLDMA tgpd and rgpd structs.
- Add comments to clarify skb consumption by ports.
- Introduce skb_data_area_size() helper.
- Declare the port config array as constant.
- Update CLDMA_JUMBO_BUFF_SZ definition when ccci_header
is introduced by port-proxy patch.
- Update Reviewed-by tags.
- Simplify t7xx_dpmaif_tx_send_skb() and make t7xx_dpmaif_add_skb_to_ring() report the tx queue full state early.

V6:
- Remove unneeded initializations and bit masks.
- Remove t7xx_common.h file.
- Add comment to circular linking in GPD list.
- Use min instead of min_t.
- Use int for local indexes instead of short or char.
- Update the commit message in CLDMA patch about dependencies on core patch.
- Add space between contributor name and email address.
- Rename registers with double negatives
e.g. DIS_ASPM_LOWPWR_CLR_0 -> ENABLE_ASPM_LOWPWR.
- Fix a race condition in pci sleep resource locking.
- Initialize interrupts with t7xx_pcie_mac_set_int() instead of 'clear'.
- Remove duplicate spin_lock_init(&md->exp_lock).
- Remove .ndo_select_queue callback due to singular TX queue.
- Remove call to deprecated netif_rx_any_context().
- Fix include guard name in t7xx_hif_dpmaif.h.
- Remove unused q_num parameter in DPMAIF functions.
- Do not serialize the drb_wr_idx write in t7xx_dpmaif_add_skb_to_ring() and the read from t7xx_txq_drb_wr_available().
- Fix potential leak in t7xx_dpmaif_add_skb_to_ring().
- Unionize:
DRB structs: msg and pd.
PIT structs: msg and pd.
- Replace list_head & spinlock with skb_buff_head in dpmaif_tx_queue.
- Remove rx_length_th check in TX WWAN port flow.
- Remove wwan_remove_port() from the critical section in WWAN port uninit.
- Use skb_end_pointer() to avoid conditional compilation.
- Simplify the loop in t7xx_port_ctrl_tx() by checking the buffer offset instead of calculating the number of required packets.
- Remove the code for unused channel PORT_CH_STATUS_RX.
- Remove bit flags from ports. Ports can check chan_enable instead of the PORT_F_RX_ALLOW_DROP flag.
- Use INVALID_SEQ_NUM to identify the first seq number.
- Rename port_static to port_conf and ports_private to ports.
- Implement t7xx_port_send_skb() and t7xx_port_send_ctl_skb() in a layered approach to reduce duplicated code and simplify the CCCI header handling.
- Move wwan_port_rx() call from port-proxy to WWAN port.
- Rename t7xx_port_recv_skb() to t7xx_port_enqueue_skb().
- Move control message parsing logic from port-proxy to control port, preserve the endianness when parsing the message and make port-proxy export a function to enable/disable ports.
- Use flexible arrays for:
port-proxy ports.
payload data in t7xx_fsm_event, port_msg, and mtk_runtime_feature.

v5:
- Update Intel's copyright years to 2021-2022.
- Remove circular dependency between DPMAIF HW (07) and HIF (08).
- Keep separate patches for CLDMA (02) and Core (03)
but improve the code split by decoupling CLDMA from
modem ops and cleaning up t7xx_common.h.
- Rename ID_CLDMA0/ID_CLDMA1 to CLDMA_ID_AP/CLDMA_ID_MD.
- Consistently use CLDMA's ring_lock to protect tr_ring.
- Free resources first and then print messages.
- Implement suggested name changes.
- Do not explicitly include dev_printk.h.
- Remove redundant dev_err()s.
- Fix possible memory leak during probe.
- Remove infrastructure for legacy interrupts.
- Remove unused macros and variables, including those that
can be replaced with constants.
- Remove PCIE_MAC_MSIX_MSK_SET macro which is duplicated code.
- Refactor __t7xx_pci_pm_suspend() for clarity.
- Refactor t7xx_cldma_rx_ring_init() and t7xx_cldma_tx_ring_init().
- Do not use & for function callbacks.
- Declare a structure to access skb->cb[] data.
- Use skb_put_data instead of memcpy.
- No need to use kfree_sensitive.
- Use dev_kfree_skb() instead of kfree_skb().
- Refactor t7xx_prepare_device_rt_data() to remove potential leaks, avoid unneeded memset and keep rt_data and packet_size updates inside the same 'if' block.
- Set port's rx_length_th back to 0 during uninit.
- Remove unneeded 'blocking' parameter from t7xx_cldma_send_skb().
- Return -EIO in t7xx_cldma_send_skb() if the queue is inactive.
- Refactor t7xx_cldma_qs_are_active() to use pci_device_is_present().
- Simplify t7xx_cldma_stop_q() and rename it to t7xx_cldma_stop_all_qs().
- Fix potential leaks in t7xx_cldma_init().
- Improve error handling in fsm_append_event and fsm_routine_starting().
- Propagate return codes from fsm_append_cmd() and t7xx_fsm_append_event().
- Refactor fsm_wait_for_event() to avoid unnecessary sleep.
- Create the WWAN ports and net device only after the modem is in the ready state.
- Refactor t7xx_port_proxy_recv_skb() and port_recv_skb().
- Rename t7xx_port_check_rx_seq_num() as t7xx_port_next_rx_seq_num() and fix the seq_num logic to handle overflows.
- Declare seq_nums as u16 instead of short.
- Use unsigned int for local indexes.
- Use min_t instead of the ternary operator.
- Refactor the loop in t7xx_dpmaif_rx_data_collect() to avoid a dead condition check.
- Use a bitmap (bat_bitmap) instead of an array to keep track of the DRB status. Used in t7xx_dpmaif_avail_pkt_bat_cnt().
- Refactor t7xx_dpmaif_tx_send_skb() to protect tx_submit_skb_cnt with spinlock and remove the misleading tx_drb_available variable.
- Consolidate bit operations before endianness conversion.
- Use C bit fields in dpmaif_drb_skb struct which is not HW related.
- Add back the que_started check in t7xx_select_tx_queue().
- Create a helper function to get the DRB count.
- Simplify the use of 'usage' during t7xx_ccmni_close().
- Enforce CCMNI MTU selection with BUILD_BUG_ON() instead of a comment.
- Remove t7xx_ccmni_ctrl->capability parameter which remains constant.

v4:
- Implement list_prev_entry_circular() and list_next_entry_circular() macros.
- Remove inline from all c files.
- Define ioread32_poll_timeout_atomic() helper macro.
- Fix return code for WWAN port tx op.
- Allow AT commands fragmentation same as MBIM commands.
- Introduce t7xx_common.h file in the first patch.
- Rename functions and variables as suggested in v3.
- Reduce code duplication by creating fsm_wait_for_event() helper function.
- Remove unneeded dev_err in t7xx_fsm_clr_event().
- Remove unused variable last_state from struct t7xx_fsm_ctl.
- Remove unused variable txq_select_times from struct dpmaif_ctrl.
- Replace ETXTBSY with EBUSY.
- Refactor t7xx_dpmaif_rx_buf_alloc() to remove an unneeded allocation.
- Fix potential leak at t7xx_dpmaif_rx_frag_alloc().
- Simplify return value handling at t7xx_dpmaif_rx_start().
- Add a helper to handle the common part of CCCI header initialization.
- Make sure interrupts are enabled during PM resume.
- Add a parameter to t7xx_fsm_append_cmd() to tell if it is in interrupt context.

v3:
- Avoid unneeded ping-pong changes between patches.
- Use t7xx_ prefix in functions.
- Use t7xx_ prefix in generic structs where mtk_ or ccci prefix was used.
- Update Authors/Contributors header.
- Remove skb pools used for control path.
- Remove skb pools used for RX data path.
- Do not use dedicated TX queue for ACK-only packets.
- Remove __packed attribute from GPD structs.
- Remove the infrastructure for test and debug ports.
- Use the skb control buffer to store metadata.
- Get the IP packet type from RX PIT.
- Merge variable declaration and simple assignments.
- Use preferred coding patterns.
- Remove global variables.
- Declare HW facing structure members as little endian.
- Rename goto tags to describe what is going to be done.
- Do not use variable length arrays.
- Remove unneeded blank lines source code and kdoc headers.
- Use C99 initialization format for port-proxy ports.
- Clean up comments.
- Review included headers.
- Better use of 100 column limit.
- Remove unneeded mb() in CLDMA.
- Remove unneeded spin locks and atomics.
- Handle read_poll_timeout error.
- Use dev_err_ratelimited() where required.
- Fix resource leak when requesting IRQs.
- Use generic DEFAULT_TX_QUEUE_LEN instead custom macro.
- Use ETH_DATA_LEN instead of defining WWAN_DEFAULT_MTU.
- Use sizeof() instead of defines when the size of structures is required.
- Remove unneeded code from netdev:
No need to configure HW address length
No need to implement .ndo_change_mtu
Remove random address generation
- Code simplifications by using kernel provided functions and macros such as:

    module_pci_driver
    PTR_ERR_OR_ZERO
    for_each_set_bit
    pci_device_is_present
    skb_queue_purge
    list_prev_entry
    __ffs64

v2:
- Replace pdev->driver->name with dev_driver_string(&pdev->dev).
- Replace random_ether_addr() with eth_random_addr().
- Update kernel-doc comment for enum data_policy.
- Indicate the driver is 'Supported' instead of 'Maintained'.
- Fix the Signed-of-by and Co-developed-by tags in the patches.
- Added authors and contributors in the top comment of the src files.

Ricardo Martinez (14):
list: Add list_next_entry_circular() and list_prev_entry_circular()
net: skb: introduce skb_data_area_size()
net: wwan: t7xx: Add control DMA interface
net: wwan: t7xx: Add core components
net: wwan: t7xx: Add port proxy infrastructure
net: wwan: t7xx: Add control port
net: wwan: t7xx: Add AT and MBIM WWAN ports
net: wwan: t7xx: Data path HW layer
net: wwan: t7xx: Add data path interface
net: wwan: t7xx: Add WWAN network interface
net: wwan: t7xx: Introduce power management
net: wwan: t7xx: Runtime PM
net: wwan: t7xx: Device deep sleep lock/unlock
net: wwan: t7xx: Add maintainers and documentation

 .../networking/device_drivers/wwan/index.rst  |    1 +
 .../networking/device_drivers/wwan/t7xx.rst   |  120 ++
 MAINTAINERS                                   |   11 +
 drivers/net/wwan/Kconfig                      |   14 +
 drivers/net/wwan/Makefile                     |    1 +
 drivers/net/wwan/t7xx/Makefile                |   20 +
 drivers/net/wwan/t7xx/t7xx_cldma.c            |  281 ++++
 drivers/net/wwan/t7xx/t7xx_cldma.h            |  180 +++
 drivers/net/wwan/t7xx/t7xx_dpmaif.c           | 1283 ++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_dpmaif.h           |  179 +++
 drivers/net/wwan/t7xx/t7xx_hif_cldma.c        | 1340 +++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_hif_cldma.h        |  127 ++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif.c       |  574 +++++++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif.h       |  206 +++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c    | 1245 +++++++++++++++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.h    |  116 ++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c    |  683 +++++++++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.h    |   78 +
 drivers/net/wwan/t7xx/t7xx_mhccif.c           |  122 ++
 drivers/net/wwan/t7xx/t7xx_mhccif.h           |   37 +
 drivers/net/wwan/t7xx/t7xx_modem_ops.c        |  727 +++++++++
 drivers/net/wwan/t7xx/t7xx_modem_ops.h        |   88 ++
 drivers/net/wwan/t7xx/t7xx_netdev.c           |  423 ++++++
 drivers/net/wwan/t7xx/t7xx_netdev.h           |   55 +
 drivers/net/wwan/t7xx/t7xx_pci.c              |  761 ++++++++++
 drivers/net/wwan/t7xx/t7xx_pci.h              |  120 ++
 drivers/net/wwan/t7xx/t7xx_pcie_mac.c         |  262 ++++
 drivers/net/wwan/t7xx/t7xx_pcie_mac.h         |   31 +
 drivers/net/wwan/t7xx/t7xx_port.h             |  135 ++
 drivers/net/wwan/t7xx/t7xx_port_ctrl_msg.c    |  273 ++++
 drivers/net/wwan/t7xx/t7xx_port_proxy.c       |  512 +++++++
 drivers/net/wwan/t7xx/t7xx_port_proxy.h       |   98 ++
 drivers/net/wwan/t7xx/t7xx_port_wwan.c        |  176 +++
 drivers/net/wwan/t7xx/t7xx_reg.h              |  350 +++++
 drivers/net/wwan/t7xx/t7xx_state_monitor.c    |  550 +++++++
 drivers/net/wwan/t7xx/t7xx_state_monitor.h    |  135 ++
 include/linux/list.h                          |   26 +
 include/linux/skbuff.h                        |    5 +
 38 files changed, 11345 insertions(+)
 create mode 100644 Documentation/networking/device_drivers/wwan/t7xx.rst
 create mode 100644 drivers/net/wwan/t7xx/Makefile
 create mode 100644 drivers/net/wwan/t7xx/t7xx_cldma.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_cldma.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_dpmaif.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_dpmaif.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_cldma.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_cldma.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_mhccif.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_mhccif.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_modem_ops.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_modem_ops.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_netdev.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_netdev.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_pci.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_pci.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_pcie_mac.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_pcie_mac.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_port.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_port_ctrl_msg.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_port_proxy.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_port_proxy.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_port_wwan.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_reg.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_state_monitor.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_state_monitor.h
Signature · 1 line
2.25.1
author reviewer
#1 lore raw

PATCH v7 1/14 list: Add list_next_entry_circular() and list_prev_entry_circular()

Add macros to get the next or previous entries and wraparound if needed. For example, calling list_next_entry_circular() on the last element should return the first element in the list.

---
 include/linux/list.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
include/linux/list.h +26 -0
index dd6c2041d09c..c147eeb2d39d 100644
@@ -563,6 +563,19 @@ static inline void list_splice_tail_init(struct list_head *list,
 #define list_next_entry(pos, member) \
 	list_entry((pos)->member.next, typeof(*(pos)), member)
 
+/**
+ * list_next_entry_circular - get the next element in list
+ * @pos:	the type * to cursor.
+ * @head:	the list head to take the element from.
+ * @member:	the name of the list_head within the struct.
+ *
+ * Wraparound if pos is the last element (return the first element).
+ * Note, that list is expected to be not empty.
+ */
+#define list_next_entry_circular(pos, head, member) \
+	(list_is_last(&(pos)->member, head) ? \
+	list_first_entry(head, typeof(*(pos)), member) : list_next_entry(pos, member))
+
 /**
  * list_prev_entry - get the prev element in list
  * @pos:	the type * to cursor
@@ -571,6 +584,19 @@ static inline void list_splice_tail_init(struct list_head *list,
 #define list_prev_entry(pos, member) \
 	list_entry((pos)->member.prev, typeof(*(pos)), member)
 
+/**
+ * list_prev_entry_circular - get the prev element in list
+ * @pos:	the type * to cursor.
+ * @head:	the list head to take the element from.
+ * @member:	the name of the list_head within the struct.
+ *
+ * Wraparound if pos is the first element (return the last element).
+ * Note, that list is expected to be not empty.
+ */
+#define list_prev_entry_circular(pos, head, member) \
+	(list_is_first(&(pos)->member, head) ? \
+	list_last_entry(head, typeof(*(pos)), member) : list_prev_entry(pos, member))
+
 /**
  * list_for_each	-	iterate over a list
  * @pos:	the &struct list_head to use as a loop cursor.
Signature · 1 line
2.25.1
author reviewer
#2 lore raw

PATCH v7 2/14 net: skb: introduce skb_data_area_size()

In reply to Ricardo Martinez

Helper to calculate the linear data space in the skb.

---
 include/linux/skbuff.h | 5 +++++
 1 file changed, 5 insertions(+)
include/linux/skbuff.h +5 -0
index 3270cb72e4d8..bd667c0129ad 100644
@@ -1665,6 +1665,11 @@ static inline void skb_set_end_offset(struct sk_buff *skb, unsigned int offset)
 }
 #endif
 
+static inline unsigned int skb_data_area_size(struct sk_buff *skb)
+{
+	return skb_end_pointer(skb) - skb->data;
+}
+
 struct ubuf_info *msg_zerocopy_alloc(struct sock *sk, size_t size);
 struct ubuf_info *msg_zerocopy_realloc(struct sock *sk, size_t size,
 				       struct ubuf_info *uarg);
Signature · 1 line
2.25.1
author reviewer
#4 lore raw

PATCH v7 3/14 net: wwan: t7xx: Add control DMA interface

In reply to Ricardo Martinez

From: Haijun Liu <[email protected]>

Cross Layer DMA (CLDMA) Hardware interface (HIF) enables the control path of Host-Modem data transfers. CLDMA HIF layer provides a common interface to the Port Layer.

CLDMA manages 8 independent RX/TX physical channels with data flow control in HW queues. CLDMA uses ring buffers of General Packet Descriptors (GPD) for TX/RX. GPDs can represent multiple or single data buffers (DB).

CLDMA HIF initializes GPD rings, registers ISR handlers for CLDMA interrupts, and initializes CLDMA HW registers.

CLDMA TX flow:
1. Port Layer write
2. Get DB address
3. Configure GPD
4. Triggering processing via HW register write

CLDMA RX flow:
1. CLDMA HW sends a RX "done" to host
2. Driver starts thread to safely read GPD
3. DB is sent to Port layer
4. Create a new buffer for GPD ring

Note: This patch does not enable compilation since it has dependencies such as t7xx_pcie_mac_clear_int()/t7xx_pcie_mac_set_int() and struct t7xx_pci_dev which are added by the core patch.

---
 drivers/net/wwan/t7xx/t7xx_cldma.c     |  281 ++++++
 drivers/net/wwan/t7xx/t7xx_cldma.h     |  180 ++++
 drivers/net/wwan/t7xx/t7xx_hif_cldma.c | 1192 ++++++++++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_hif_cldma.h |  126 +++
 drivers/net/wwan/t7xx/t7xx_reg.h       |   33 +
 5 files changed, 1812 insertions(+)
 create mode 100644 drivers/net/wwan/t7xx/t7xx_cldma.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_cldma.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_cldma.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_cldma.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_reg.h
drivers/net/wwan/t7xx/t7xx_cldma.c new file +281 -0
new file mode 100644
index 000000000000..9f43f256db1d
@@ -0,0 +1,281 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2021, MediaTek Inc.
+ * Copyright (c) 2021-2022, Intel Corporation.
+ *
+ * Authors:
+ *  Haijun Liu <[email protected]>
+ *  Moises Veleta <[email protected]>
+ *  Ricardo Martinez <[email protected]>
+ *
+ * Contributors:
+ *  Amir Hanania <[email protected]>
+ *  Andy Shevchenko <[email protected]>
+ *  Eliot Lee <[email protected]>
+ *  Sreehari Kancharla <[email protected]>
+ */
+
+#include <linux/bits.h>
+#include <linux/delay.h>
+#include <linux/io.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/types.h>
+
+#include "t7xx_cldma.h"
+
+#define ADDR_SIZE	8
+
+void t7xx_cldma_clear_ip_busy(struct t7xx_cldma_hw *hw_info)
+{
+	u32 val;
+
+	val = ioread32(hw_info->ap_pdn_base + REG_CLDMA_IP_BUSY);
+	val |= IP_BUSY_WAKEUP;
+	iowrite32(val, hw_info->ap_pdn_base + REG_CLDMA_IP_BUSY);
+}
+
+/**
+ * t7xx_cldma_hw_restore() - Restore CLDMA HW registers.
+ * @hw_info: Pointer to struct t7xx_cldma_hw.
+ *
+ * Restore HW after resume. Writes uplink configuration for CLDMA HW.
+ */
+void t7xx_cldma_hw_restore(struct t7xx_cldma_hw *hw_info)
+{
+	u32 ul_cfg;
+
+	ul_cfg = ioread32(hw_info->ap_pdn_base + REG_CLDMA_UL_CFG);
+	ul_cfg &= ~UL_CFG_BIT_MODE_MASK;
+
+	if (hw_info->hw_mode == MODE_BIT_64)
+		ul_cfg |= UL_CFG_BIT_MODE_64;
+	else if (hw_info->hw_mode == MODE_BIT_40)
+		ul_cfg |= UL_CFG_BIT_MODE_40;
+	else if (hw_info->hw_mode == MODE_BIT_36)
+		ul_cfg |= UL_CFG_BIT_MODE_36;
+
+	iowrite32(ul_cfg, hw_info->ap_pdn_base + REG_CLDMA_UL_CFG);
+	/* Disable TX and RX invalid address check */
+	iowrite32(UL_MEM_CHECK_DIS, hw_info->ap_pdn_base + REG_CLDMA_UL_MEM);
+	iowrite32(DL_MEM_CHECK_DIS, hw_info->ap_pdn_base + REG_CLDMA_DL_MEM);
+}
+
+void t7xx_cldma_hw_start_queue(struct t7xx_cldma_hw *hw_info, unsigned int qno,
+			       enum mtk_txrx tx_rx)
+{
+	void __iomem *reg;
+	u32 val;
+
+	reg = tx_rx == MTK_RX ? hw_info->ap_pdn_base + REG_CLDMA_DL_START_CMD :
+				hw_info->ap_pdn_base + REG_CLDMA_UL_START_CMD;
+	val = qno == CLDMA_ALL_Q ? CLDMA_ALL_Q : BIT(qno);
+	iowrite32(val, reg);
+}
+
+void t7xx_cldma_hw_start(struct t7xx_cldma_hw *hw_info)
+{
+	/* Enable the TX & RX interrupts */
+	iowrite32(TXRX_STATUS_BITMASK, hw_info->ap_pdn_base + REG_CLDMA_L2TIMCR0);
+	iowrite32(TXRX_STATUS_BITMASK, hw_info->ap_ao_base + REG_CLDMA_L2RIMCR0);
+	/* Enable the empty queue interrupt */
+	iowrite32(EMPTY_STATUS_BITMASK, hw_info->ap_pdn_base + REG_CLDMA_L2TIMCR0);
+	iowrite32(EMPTY_STATUS_BITMASK, hw_info->ap_ao_base + REG_CLDMA_L2RIMCR0);
+}
+
+void t7xx_cldma_hw_reset(void __iomem *ao_base)
+{
+	u32 val;
+
+	val = ioread32(ao_base + REG_INFRA_RST2_SET);
+	val |= RST2_PMIC_SW_RST_SET;
+	iowrite32(val, ao_base + REG_INFRA_RST2_SET);
+	val = ioread32(ao_base + REG_INFRA_RST4_SET);
+	val |= RST4_CLDMA1_SW_RST_SET;
+	iowrite32(val, ao_base + REG_INFRA_RST4_SET);
+	udelay(1);
+
+	val = ioread32(ao_base + REG_INFRA_RST4_CLR);
+	val |= RST4_CLDMA1_SW_RST_CLR;
+	iowrite32(val, ao_base + REG_INFRA_RST4_CLR);
+	val = ioread32(ao_base + REG_INFRA_RST2_CLR);
+	val |= RST2_PMIC_SW_RST_CLR;
+	iowrite32(val, ao_base + REG_INFRA_RST2_CLR);
+}
+
+bool t7xx_cldma_tx_addr_is_set(struct t7xx_cldma_hw *hw_info, unsigned int qno)
+{
+	u32 offset = REG_CLDMA_UL_START_ADDRL_0 + qno * ADDR_SIZE;
+
+	return ioread64(hw_info->ap_pdn_base + offset);
+}
+
+void t7xx_cldma_hw_set_start_addr(struct t7xx_cldma_hw *hw_info, unsigned int qno, u64 address,
+				  enum mtk_txrx tx_rx)
+{
+	u32 offset = qno * ADDR_SIZE;
+	void __iomem *reg;
+
+	reg = tx_rx == MTK_RX ? hw_info->ap_ao_base + REG_CLDMA_DL_START_ADDRL_0 :
+				hw_info->ap_pdn_base + REG_CLDMA_UL_START_ADDRL_0;
+	iowrite64(address, reg + offset);
+}
+
+void t7xx_cldma_hw_resume_queue(struct t7xx_cldma_hw *hw_info, unsigned int qno,
+				enum mtk_txrx tx_rx)
+{
+	void __iomem *base = hw_info->ap_pdn_base;
+
+	if (tx_rx == MTK_RX)
+		iowrite32(BIT(qno), base + REG_CLDMA_DL_RESUME_CMD);
+	else
+		iowrite32(BIT(qno), base + REG_CLDMA_UL_RESUME_CMD);
+}
+
+unsigned int t7xx_cldma_hw_queue_status(struct t7xx_cldma_hw *hw_info, unsigned int qno,
+					enum mtk_txrx tx_rx)
+{
+	void __iomem *reg;
+	u32 mask, val;
+
+	mask = qno == CLDMA_ALL_Q ? CLDMA_ALL_Q : BIT(qno);
+	reg = tx_rx == MTK_RX ? hw_info->ap_ao_base + REG_CLDMA_DL_STATUS :
+				hw_info->ap_pdn_base + REG_CLDMA_UL_STATUS;
+	val = ioread32(reg);
+
+	return val & mask;
+}
+
+void t7xx_cldma_hw_tx_done(struct t7xx_cldma_hw *hw_info, unsigned int bitmask)
+{
+	unsigned int ch_id;
+
+	ch_id = ioread32(hw_info->ap_pdn_base + REG_CLDMA_L2TISAR0);
+	ch_id &= bitmask;
+	/* Clear the ch IDs in the TX interrupt status register */
+	iowrite32(ch_id, hw_info->ap_pdn_base + REG_CLDMA_L2TISAR0);
+	ioread32(hw_info->ap_pdn_base + REG_CLDMA_L2TISAR0);
+}
+
+void t7xx_cldma_hw_rx_done(struct t7xx_cldma_hw *hw_info, unsigned int bitmask)
+{
+	unsigned int ch_id;
+
+	ch_id = ioread32(hw_info->ap_pdn_base + REG_CLDMA_L2RISAR0);
+	ch_id &= bitmask;
+	/* Clear the ch IDs in the RX interrupt status register */
+	iowrite32(ch_id, hw_info->ap_pdn_base + REG_CLDMA_L2RISAR0);
+	ioread32(hw_info->ap_pdn_base + REG_CLDMA_L2RISAR0);
+}
+
+unsigned int t7xx_cldma_hw_int_status(struct t7xx_cldma_hw *hw_info, unsigned int bitmask,
+				      enum mtk_txrx tx_rx)
+{
+	void __iomem *reg;
+	u32 val;
+
+	reg = tx_rx == MTK_RX ? hw_info->ap_pdn_base + REG_CLDMA_L2RISAR0 :
+				hw_info->ap_pdn_base + REG_CLDMA_L2TISAR0;
+	val = ioread32(reg);
+	return val & bitmask;
+}
+
+void t7xx_cldma_hw_irq_dis_txrx(struct t7xx_cldma_hw *hw_info, unsigned int qno,
+				enum mtk_txrx tx_rx)
+{
+	void __iomem *reg;
+	u32 val;
+
+	reg = tx_rx == MTK_RX ? hw_info->ap_ao_base + REG_CLDMA_L2RIMSR0 :
+				hw_info->ap_pdn_base + REG_CLDMA_L2TIMSR0;
+	val = qno == CLDMA_ALL_Q ? CLDMA_ALL_Q : BIT(qno);
+	iowrite32(val, reg);
+}
+
+void t7xx_cldma_hw_irq_dis_eq(struct t7xx_cldma_hw *hw_info, unsigned int qno, enum mtk_txrx tx_rx)
+{
+	void __iomem *reg;
+	u32 val;
+
+	reg = tx_rx == MTK_RX ? hw_info->ap_ao_base + REG_CLDMA_L2RIMSR0 :
+				hw_info->ap_pdn_base + REG_CLDMA_L2TIMSR0;
+	val = qno == CLDMA_ALL_Q ? CLDMA_ALL_Q : BIT(qno);
+	iowrite32(val << EQ_STA_BIT_OFFSET, reg);
+}
+
+void t7xx_cldma_hw_irq_en_txrx(struct t7xx_cldma_hw *hw_info, unsigned int qno,
+			       enum mtk_txrx tx_rx)
+{
+	void __iomem *reg;
+	u32 val;
+
+	reg = tx_rx == MTK_RX ? hw_info->ap_ao_base + REG_CLDMA_L2RIMCR0 :
+				hw_info->ap_pdn_base + REG_CLDMA_L2TIMCR0;
+	val = qno == CLDMA_ALL_Q ? CLDMA_ALL_Q : BIT(qno);
+	iowrite32(val, reg);
+}
+
+void t7xx_cldma_hw_irq_en_eq(struct t7xx_cldma_hw *hw_info, unsigned int qno, enum mtk_txrx tx_rx)
+{
+	void __iomem *reg;
+	u32 val;
+
+	reg = tx_rx == MTK_RX ? hw_info->ap_ao_base + REG_CLDMA_L2RIMCR0 :
+				hw_info->ap_pdn_base + REG_CLDMA_L2TIMCR0;
+	val = qno == CLDMA_ALL_Q ? CLDMA_ALL_Q : BIT(qno);
+	iowrite32(val << EQ_STA_BIT_OFFSET, reg);
+}
+
+/**
+ * t7xx_cldma_hw_init() - Initialize CLDMA HW.
+ * @hw_info: Pointer to struct t7xx_cldma_hw.
+ *
+ * Write uplink and downlink configuration to CLDMA HW.
+ */
+void t7xx_cldma_hw_init(struct t7xx_cldma_hw *hw_info)
+{
+	u32 ul_cfg, dl_cfg;
+
+	ul_cfg = ioread32(hw_info->ap_pdn_base + REG_CLDMA_UL_CFG);
+	dl_cfg = ioread32(hw_info->ap_ao_base + REG_CLDMA_DL_CFG);
+	/* Configure the DRAM address mode */
+	ul_cfg &= ~UL_CFG_BIT_MODE_MASK;
+	dl_cfg &= ~DL_CFG_BIT_MODE_MASK;
+
+	if (hw_info->hw_mode == MODE_BIT_64) {
+		ul_cfg |= UL_CFG_BIT_MODE_64;
+		dl_cfg |= DL_CFG_BIT_MODE_64;
+	} else if (hw_info->hw_mode == MODE_BIT_40) {
+		ul_cfg |= UL_CFG_BIT_MODE_40;
+		dl_cfg |= DL_CFG_BIT_MODE_40;
+	} else if (hw_info->hw_mode == MODE_BIT_36) {
+		ul_cfg |= UL_CFG_BIT_MODE_36;
+		dl_cfg |= DL_CFG_BIT_MODE_36;
+	}
+
+	iowrite32(ul_cfg, hw_info->ap_pdn_base + REG_CLDMA_UL_CFG);
+	dl_cfg |= DL_CFG_UP_HW_LAST;
+	iowrite32(dl_cfg, hw_info->ap_ao_base + REG_CLDMA_DL_CFG);
+	iowrite32(0, hw_info->ap_ao_base + REG_CLDMA_INT_MASK);
+	iowrite32(BUSY_MASK_MD, hw_info->ap_ao_base + REG_CLDMA_BUSY_MASK);
+	iowrite32(UL_MEM_CHECK_DIS, hw_info->ap_pdn_base + REG_CLDMA_UL_MEM);
+	iowrite32(DL_MEM_CHECK_DIS, hw_info->ap_pdn_base + REG_CLDMA_DL_MEM);
+}
+
+void t7xx_cldma_hw_stop_all_qs(struct t7xx_cldma_hw *hw_info, enum mtk_txrx tx_rx)
+{
+	void __iomem *reg;
+
+	reg = tx_rx == MTK_RX ? hw_info->ap_pdn_base + REG_CLDMA_DL_STOP_CMD :
+				hw_info->ap_pdn_base + REG_CLDMA_UL_STOP_CMD;
+	iowrite32(CLDMA_ALL_Q, reg);
+}
+
+void t7xx_cldma_hw_stop(struct t7xx_cldma_hw *hw_info, enum mtk_txrx tx_rx)
+{
+	void __iomem *reg;
+
+	reg = tx_rx == MTK_RX ? hw_info->ap_ao_base + REG_CLDMA_L2RIMSR0 :
+				hw_info->ap_pdn_base + REG_CLDMA_L2TIMSR0;
+	iowrite32(TXRX_STATUS_BITMASK, reg);
+	iowrite32(EMPTY_STATUS_BITMASK, reg);
+}
drivers/net/wwan/t7xx/t7xx_cldma.h new file +180 -0

180 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_hif_cldma.c new file +1192 -0

1,192 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_hif_cldma.h new file +126 -0

126 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_reg.h new file +33 -0
new file mode 100644
index 000000000000..7dc6c77a59e3
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only
+ *
+ * Copyright (c) 2021, MediaTek Inc.
+ * Copyright (c) 2021-2022, Intel Corporation.
+ *
+ * Authors:
+ *  Haijun Liu <[email protected]>
+ *  Chiranjeevi Rapolu <[email protected]>
+ *
+ * Contributors:
+ *  Amir Hanania <[email protected]>
+ *  Andy Shevchenko <[email protected]>
+ *  Eliot Lee <[email protected]>
+ *  Moises Veleta <[email protected]>
+ *  Ricardo Martinez <[email protected]>
+ *  Sreehari Kancharla <[email protected]>
+ */
+
+#ifndef __T7XX_REG_H__
+#define __T7XX_REG_H__
+
+enum t7xx_int {
+	DPMAIF_INT,
+	CLDMA0_INT,
+	CLDMA1_INT,
+	CLDMA2_INT,
+	MHCCIF_INT,
+	DPMAIF2_INT,
+	SAP_RGU_INT,
+	CLDMA3_INT,
+};
+
+#endif /* __T7XX_REG_H__ */
Signature · 1 line
2.25.1
maintainer Reviewed-by on v7 3/14
#5 lore raw

Re: [PATCH net-next v7 03/14] net: wwan: t7xx: Add control DMA interface

Quoted 28 lines · Ricardo Martinez

Cross Layer DMA (CLDMA) Hardware interface (HIF) enables the control path of Host-Modem data transfers. CLDMA HIF layer provides a common interface to the Port Layer.

CLDMA manages 8 independent RX/TX physical channels with data flow control in HW queues. CLDMA uses ring buffers of General Packet Descriptors (GPD) for TX/RX. GPDs can represent multiple or single data buffers (DB).

CLDMA HIF initializes GPD rings, registers ISR handlers for CLDMA interrupts, and initializes CLDMA HW registers.

CLDMA TX flow:
1. Port Layer write
2. Get DB address
3. Configure GPD
4. Triggering processing via HW register write

CLDMA RX flow:
1. CLDMA HW sends a RX "done" to host
2. Driver starts thread to safely read GPD
3. DB is sent to Port layer
4. Create a new buffer for GPD ring

Note: This patch does not enable compilation since it has dependencies such as t7xx_pcie_mac_clear_int()/t7xx_pcie_mac_set_int() and struct t7xx_pci_dev which are added by the core patch.

Quoted 6 lines
author reviewer
#6 lore raw

PATCH v7 4/14 net: wwan: t7xx: Add core components

In reply to Ricardo Martinez

From: Haijun Liu <[email protected]>

Registers the t7xx device driver with the kernel. Setup all the core
components: PCIe layer, Modem Host Cross Core Interface (MHCCIF), modem control operations, modem state machine, and build infrastructure.

* PCIe layer code implements driver probe and removal.
* MHCCIF provides interrupt channels to communicate events such as handshake, PM and port enumeration.
* Modem control implements the entry point for modem init, reset and exit.
* The modem status monitor is a state machine used by modem control to complete initialization and stop. It is used also to propagate exception events reported by other components.

---
 drivers/net/wwan/Kconfig                   |  14 +
 drivers/net/wwan/Makefile                  |   1 +
 drivers/net/wwan/t7xx/Makefile             |  12 +
 drivers/net/wwan/t7xx/t7xx_mhccif.c        | 102 ++++
 drivers/net/wwan/t7xx/t7xx_mhccif.h        |  37 ++
 drivers/net/wwan/t7xx/t7xx_modem_ops.c     | 498 +++++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_modem_ops.h     |  85 ++++
 drivers/net/wwan/t7xx/t7xx_pci.c           | 225 +++++++++
 drivers/net/wwan/t7xx/t7xx_pci.h           |  64 +++
 drivers/net/wwan/t7xx/t7xx_pcie_mac.c      | 262 ++++++++++
 drivers/net/wwan/t7xx/t7xx_pcie_mac.h      |  31 ++
 drivers/net/wwan/t7xx/t7xx_reg.h           | 104 ++++
 drivers/net/wwan/t7xx/t7xx_state_monitor.c | 540 +++++++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_state_monitor.h | 133 +++++
 14 files changed, 2108 insertions(+)
 create mode 100644 drivers/net/wwan/t7xx/Makefile
 create mode 100644 drivers/net/wwan/t7xx/t7xx_mhccif.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_mhccif.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_modem_ops.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_modem_ops.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_pci.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_pci.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_pcie_mac.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_pcie_mac.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_state_monitor.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_state_monitor.h
drivers/net/wwan/Kconfig +14 -0
index 609fd4a2c865..3486ffe94ac4 100644
@@ -105,6 +105,20 @@ config IOSM
 
 	  If unsure, say N.
 
+config MTK_T7XX
+	tristate "MediaTek PCIe 5G WWAN modem T7xx device"
+	depends on PCI
+	help
+	  Enables MediaTek PCIe based 5G WWAN modem (T7xx series) device.
+	  Adapts WWAN framework and provides network interface like wwan0
+	  and tty interfaces like wwan0at0 (AT protocol), wwan0mbim0
+	  (MBIM protocol), etc.
+
+	  To compile this driver as a module, choose M here: the module will be
+	  called mtk_t7xx.
+
+	  If unsure, say N.
+
 endif # WWAN
 
 endmenu
drivers/net/wwan/Makefile +1 -0
index e722650bebea..3960c0ae2445 100644
@@ -13,3 +13,4 @@ obj-$(CONFIG_MHI_WWAN_MBIM) += mhi_wwan_mbim.o
 obj-$(CONFIG_QCOM_BAM_DMUX) += qcom_bam_dmux.o
 obj-$(CONFIG_RPMSG_WWAN_CTRL) += rpmsg_wwan_ctrl.o
 obj-$(CONFIG_IOSM) += iosm/
+obj-$(CONFIG_MTK_T7XX) += t7xx/
drivers/net/wwan/t7xx/Makefile new file +12 -0
new file mode 100644
index 000000000000..6a49013bc343
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+ccflags-y += -Werror
+
+obj-${CONFIG_MTK_T7XX} := mtk_t7xx.o
+mtk_t7xx-y:=	t7xx_pci.o \
+		t7xx_pcie_mac.o \
+		t7xx_mhccif.o \
+		t7xx_state_monitor.o  \
+		t7xx_modem_ops.o \
+		t7xx_cldma.o \
+		t7xx_hif_cldma.o  \
drivers/net/wwan/t7xx/t7xx_mhccif.c new file +102 -0

102 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_mhccif.h new file +37 -0
new file mode 100644
index 000000000000..209b386bc088
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0-only
+ *
+ * Copyright (c) 2021, MediaTek Inc.
+ * Copyright (c) 2021-2022, Intel Corporation.
+ *
+ * Authors:
+ *  Haijun Liu <[email protected]>
+ *  Sreehari Kancharla <[email protected]>
+ *
+ * Contributors:
+ *  Amir Hanania <[email protected]>
+ *  Ricardo Martinez <[email protected]>
+ */
+
+#ifndef __T7XX_MHCCIF_H__
+#define __T7XX_MHCCIF_H__
+
+#include <linux/types.h>
+
+#include "t7xx_pci.h"
+#include "t7xx_reg.h"
+
+#define D2H_SW_INT_MASK (D2H_INT_EXCEPTION_INIT |		\
+			 D2H_INT_EXCEPTION_INIT_DONE |		\
+			 D2H_INT_EXCEPTION_CLEARQ_DONE |	\
+			 D2H_INT_EXCEPTION_ALLQ_RESET |		\
+			 D2H_INT_PORT_ENUM |			\
+			 D2H_INT_ASYNC_MD_HK)
+
+void t7xx_mhccif_mask_set(struct t7xx_pci_dev *t7xx_dev, u32 val);
+void t7xx_mhccif_mask_clr(struct t7xx_pci_dev *t7xx_dev, u32 val);
+u32 t7xx_mhccif_mask_get(struct t7xx_pci_dev *t7xx_dev);
+void t7xx_mhccif_init(struct t7xx_pci_dev *t7xx_dev);
+u32 t7xx_mhccif_read_sw_int_sts(struct t7xx_pci_dev *t7xx_dev);
+void t7xx_mhccif_h2d_swint_trigger(struct t7xx_pci_dev *t7xx_dev, u32 channel);
+
+#endif /*__T7XX_MHCCIF_H__ */
drivers/net/wwan/t7xx/t7xx_modem_ops.c new file +498 -0

498 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_modem_ops.h new file +85 -0

85 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_pci.c new file +225 -0

225 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_pci.h new file +64 -0

64 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_pcie_mac.c new file +262 -0

262 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_pcie_mac.h new file +31 -0
new file mode 100644
index 000000000000..50336fa7e8c2
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0-only
+ *
+ * Copyright (c) 2021, MediaTek Inc.
+ * Copyright (c) 2021-2022, Intel Corporation.
+ *
+ * Authors:
+ *  Haijun Liu <[email protected]>
+ *  Sreehari Kancharla <[email protected]>
+ *
+ * Contributors:
+ *  Moises Veleta <[email protected]>
+ *  Ricardo Martinez <[email protected]>
+ */
+
+#ifndef __T7XX_PCIE_MAC_H__
+#define __T7XX_PCIE_MAC_H__
+
+#include "t7xx_pci.h"
+#include "t7xx_reg.h"
+
+#define IREG_BASE(t7xx_dev)	((t7xx_dev)->base_addr.pcie_mac_ireg_base)
+
+void t7xx_pcie_mac_interrupts_en(struct t7xx_pci_dev *t7xx_dev);
+void t7xx_pcie_mac_interrupts_dis(struct t7xx_pci_dev *t7xx_dev);
+void t7xx_pcie_mac_atr_init(struct t7xx_pci_dev *t7xx_dev);
+void t7xx_pcie_mac_clear_int(struct t7xx_pci_dev *t7xx_dev, enum t7xx_int int_type);
+void t7xx_pcie_mac_set_int(struct t7xx_pci_dev *t7xx_dev, enum t7xx_int int_type);
+void t7xx_pcie_mac_clear_int_status(struct t7xx_pci_dev *t7xx_dev, enum t7xx_int int_type);
+void t7xx_pcie_set_mac_msix_cfg(struct t7xx_pci_dev *t7xx_dev, unsigned int irq_count);
+
+#endif /* __T7XX_PCIE_MAC_H__ */
drivers/net/wwan/t7xx/t7xx_reg.h +104 -0

110 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_state_monitor.c new file +540 -0

540 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_state_monitor.h new file +133 -0

133 lines not on this page · raw message

Signature · 1 line
2.25.1
author reviewer
#7 lore raw

PATCH v7 5/14 net: wwan: t7xx: Add port proxy infrastructure

In reply to Ricardo Martinez

From: Haijun Liu <[email protected]>

Port-proxy provides a common interface to interact with different types of ports. Ports export their configuration via struct t7xx_port and operate as defined by struct port_ops.

---
 drivers/net/wwan/t7xx/Makefile             |   1 +
 drivers/net/wwan/t7xx/t7xx_hif_cldma.c     |   3 +-
 drivers/net/wwan/t7xx/t7xx_modem_ops.c     |  14 +-
 drivers/net/wwan/t7xx/t7xx_port.h          | 132 ++++++
 drivers/net/wwan/t7xx/t7xx_port_proxy.c    | 452 +++++++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_port_proxy.h    |  72 ++++
 drivers/net/wwan/t7xx/t7xx_state_monitor.c |   5 +
 7 files changed, 677 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/wwan/t7xx/t7xx_port.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_port_proxy.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_port_proxy.h
drivers/net/wwan/t7xx/Makefile +1 -0
index 6a49013bc343..99f9ca3b4b51 100644
@@ -10,3 +10,4 @@ mtk_t7xx-y:=	t7xx_pci.o \
 		t7xx_modem_ops.o \
 		t7xx_cldma.o \
 		t7xx_hif_cldma.o  \
+		t7xx_port_proxy.o  \
drivers/net/wwan/t7xx/t7xx_hif_cldma.c +2 -1
index c756b1d0b519..7a8c09f7435a 100644
@@ -46,6 +46,7 @@
 #include "t7xx_mhccif.h"
 #include "t7xx_pci.h"
 #include "t7xx_pcie_mac.h"
+#include "t7xx_port_proxy.h"
 #include "t7xx_reg.h"
 #include "t7xx_state_monitor.h"
 
@@ -55,7 +56,7 @@
 #define CHECK_Q_STOP_TIMEOUT_US		1000000
 #define CHECK_Q_STOP_STEP_US		10000
 
-#define CLDMA_JUMBO_BUFF_SZ		64528	/* 63kB + CCCI header */
+#define CLDMA_JUMBO_BUFF_SZ		(63 * 1024 + sizeof(struct ccci_header))
 
 static void md_cd_queue_struct_reset(struct cldma_queue *queue, struct cldma_ctrl *md_ctrl,
 				     enum mtk_txrx tx_rx, unsigned int index)
drivers/net/wwan/t7xx/t7xx_modem_ops.c +13 -1
index 30292e3f1c37..d1d3384608be 100644
@@ -34,6 +34,8 @@
 #include "t7xx_modem_ops.h"
 #include "t7xx_pci.h"
 #include "t7xx_pcie_mac.h"
+#include "t7xx_port.h"
+#include "t7xx_port_proxy.h"
 #include "t7xx_reg.h"
 #include "t7xx_state_monitor.h"
 
@@ -273,6 +275,7 @@ static void t7xx_md_exception(struct t7xx_modem *md, enum hif_ex_stage stage)
 	if (stage == HIF_EX_CLEARQ_DONE) {
 		/* Give DHL time to flush data */
 		msleep(PORT_RESET_DELAY_MS);
+		t7xx_port_proxy_reset(md->port_prox);
 	}
 
 	t7xx_cldma_exception(md->md_ctrl[CLDMA_ID_MD], stage);
@@ -426,6 +429,7 @@ int t7xx_md_reset(struct t7xx_pci_dev *t7xx_dev)
 	md->exp_id = 0;
 	t7xx_fsm_reset(md);
 	t7xx_cldma_reset(md->md_ctrl[CLDMA_ID_MD]);
+	t7xx_port_proxy_reset(md->port_prox);
 	md->md_init_finish = true;
 	return 0;
 }
@@ -462,14 +466,21 @@ int t7xx_md_init(struct t7xx_pci_dev *t7xx_dev)
 	if (ret)
 		goto err_uninit_fsm;
 
+	ret = t7xx_port_proxy_init(md);
+	if (ret)
+		goto err_uninit_md_cldma;
+
 	ret = t7xx_fsm_append_cmd(md->fsm_ctl, FSM_CMD_START, 0);
 	if (ret) /* fsm_uninit flushes cmd queue */
-		goto err_uninit_md_cldma;
+		goto err_uninit_proxy;
 
 	t7xx_md_sys_sw_init(t7xx_dev);
 	md->md_init_finish = true;
 	return 0;
 
+err_uninit_proxy:
+	t7xx_port_proxy_uninit(md->port_prox);
+
 err_uninit_md_cldma:
 	t7xx_cldma_exit(md->md_ctrl[CLDMA_ID_MD]);
 
@@ -492,6 +503,7 @@ void t7xx_md_exit(struct t7xx_pci_dev *t7xx_dev)
 		return;
 
 	t7xx_fsm_append_cmd(md->fsm_ctl, FSM_CMD_PRE_STOP, FSM_CMD_FLAG_WAIT_FOR_COMPLETION);
+	t7xx_port_proxy_uninit(md->port_prox);
 	t7xx_cldma_exit(md->md_ctrl[CLDMA_ID_MD]);
 	t7xx_fsm_uninit(md);
 	destroy_workqueue(md->handshake_wq);
drivers/net/wwan/t7xx/t7xx_port.h new file +132 -0

132 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_port_proxy.c new file +452 -0

452 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_port_proxy.h new file +72 -0

72 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_state_monitor.c +5 -0
index 4b2e6913c45e..30f3739f1d83 100644
@@ -37,6 +37,7 @@
 #include "t7xx_modem_ops.h"
 #include "t7xx_pci.h"
 #include "t7xx_pcie_mac.h"
+#include "t7xx_port_proxy.h"
 #include "t7xx_reg.h"
 #include "t7xx_state_monitor.h"
 
@@ -90,6 +91,9 @@ static void fsm_state_notify(struct t7xx_modem *md, enum md_state state)
 void t7xx_fsm_broadcast_state(struct t7xx_fsm_ctl *ctl, enum md_state state)
 {
 	ctl->md_state = state;
+
+	/* Update to port first, otherwise sending message on HS2 may fail */
+	t7xx_port_proxy_md_status_notify(ctl->md->port_prox, state);
 	fsm_state_notify(ctl->md, state);
 }
 
@@ -258,6 +262,7 @@ static void t7xx_fsm_broadcast_ready_state(struct t7xx_fsm_ctl *ctl)
 	ctl->md_state = MD_STATE_READY;
 
 	fsm_state_notify(ctl->md, MD_STATE_READY);
+	t7xx_port_proxy_md_status_notify(ctl->md->port_prox, MD_STATE_READY);
 }
 
 static void fsm_routine_ready(struct t7xx_fsm_ctl *ctl)
Signature · 1 line
2.25.1
maintainer Reviewed-by on v7 5/14
#8 lore raw

Re: [PATCH net-next v7 05/14] net: wwan: t7xx: Add port proxy infrastructure

Quoted 11 lines · Ricardo Martinez

Port-proxy provides a common interface to interact with different types of ports. Ports export their configuration via struct t7xx_port and operate as defined by struct port_ops.

author reviewer
#9 lore raw

PATCH v7 6/14 net: wwan: t7xx: Add control port

In reply to Ricardo Martinez

From: Haijun Liu <[email protected]>

Control Port implements driver control messages such as modem-host handshaking, controls port enumeration, and handles exception messages.

The handshaking process between the driver and the modem happens during the init sequence. The process involves the exchange of a list of supported runtime features to make sure that modem and host are ready to provide proper feature lists including port enumeration. Further features can be enabled and controlled in this handshaking process.

---
 drivers/net/wwan/t7xx/Makefile             |   1 +
 drivers/net/wwan/t7xx/t7xx_modem_ops.c     | 214 +++++++++++++++-
 drivers/net/wwan/t7xx/t7xx_modem_ops.h     |   3 +
 drivers/net/wwan/t7xx/t7xx_port.h          |   3 +
 drivers/net/wwan/t7xx/t7xx_port_ctrl_msg.c | 273 +++++++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_port_proxy.c    |  40 +++
 drivers/net/wwan/t7xx/t7xx_port_proxy.h    |  25 ++
 drivers/net/wwan/t7xx/t7xx_state_monitor.c |   3 +
 drivers/net/wwan/t7xx/t7xx_state_monitor.h |   2 +
 9 files changed, 561 insertions(+), 3 deletions(-)
 create mode 100644 drivers/net/wwan/t7xx/t7xx_port_ctrl_msg.c
drivers/net/wwan/t7xx/Makefile +1 -0
index 99f9ca3b4b51..63e1c67b82b5 100644
@@ -11,3 +11,4 @@ mtk_t7xx-y:=	t7xx_pci.o \
 		t7xx_cldma.o \
 		t7xx_hif_cldma.o  \
 		t7xx_port_proxy.o  \
+		t7xx_port_ctrl_msg.o \
drivers/net/wwan/t7xx/t7xx_modem_ops.c +211 -3

263 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_modem_ops.h +3 -0
index 1fc286fa1a14..7469ed636ae8 100644
@@ -57,6 +57,9 @@ enum md_event_id {
 
 struct t7xx_sys_info {
 	bool				ready;
+	bool				handshake_ongoing;
+	u8				feature_set[FEATURE_COUNT];
+	struct t7xx_port		*ctl_port;
 };
 
 struct t7xx_modem {
drivers/net/wwan/t7xx/t7xx_port.h +3 -0
index 0f3ffba5f9c2..dc4133eb433a 100644
@@ -125,8 +125,11 @@ struct t7xx_port {
 };
 
 struct sk_buff *t7xx_port_alloc_skb(int payload);
+struct sk_buff *t7xx_ctrl_alloc_skb(int payload);
 int t7xx_port_enqueue_skb(struct t7xx_port *port, struct sk_buff *skb);
 int t7xx_port_send_skb(struct t7xx_port *port, struct sk_buff *skb, unsigned int pkt_header,
 		       unsigned int ex_msg);
+int t7xx_port_send_ctl_skb(struct t7xx_port *port, struct sk_buff *skb, unsigned int msg,
+			   unsigned int ex_msg);
 
 #endif /* __T7XX_PORT_H__ */
drivers/net/wwan/t7xx/t7xx_port_ctrl_msg.c new file +273 -0

273 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_port_proxy.c +40 -0
index be4171a1d384..55f932d2c126 100644
@@ -49,6 +49,15 @@
 	     i++, (p) = &(proxy)->ports[i])
 
 static const struct t7xx_port_conf t7xx_md_port_conf[] = {
+	{
+		.tx_ch = PORT_CH_CONTROL_TX,
+		.rx_ch = PORT_CH_CONTROL_RX,
+		.txq_index = Q_IDX_CTRL,
+		.rxq_index = Q_IDX_CTRL,
+		.path_id = CLDMA_ID_MD,
+		.ops = &ctl_port_ops,
+		.name = "t7xx_ctrl",
+	},
 };
 
 static struct t7xx_port *t7xx_proxy_get_port_by_ch(struct port_proxy *port_prox, enum port_ch ch)
@@ -129,6 +138,16 @@ struct sk_buff *t7xx_port_alloc_skb(int payload)
 	return skb;
 }
 
+struct sk_buff *t7xx_ctrl_alloc_skb(int payload)
+{
+	struct sk_buff *skb = t7xx_port_alloc_skb(payload + sizeof(struct ctrl_msg_header));
+
+	if (skb)
+		skb_reserve(skb, sizeof(struct ctrl_msg_header));
+
+	return skb;
+}
+
 /**
  * t7xx_port_enqueue_skb() - Enqueue the received skb into the port's rx_skb_list.
  * @port: port context.
@@ -194,6 +213,24 @@ static int t7xx_port_send_ccci_skb(struct t7xx_port *port, struct sk_buff *skb,
 	return 0;
 }
 
+int t7xx_port_send_ctl_skb(struct t7xx_port *port, struct sk_buff *skb, unsigned int msg,
+			   unsigned int ex_msg)
+{
+	struct ctrl_msg_header *ctrl_msg_h;
+	unsigned int msg_len = skb->len;
+	u32 pkt_header = 0;
+
+	ctrl_msg_h = skb_push(skb, sizeof(*ctrl_msg_h));
+	ctrl_msg_h->ctrl_msg_id = cpu_to_le32(msg);
+	ctrl_msg_h->ex_msg = cpu_to_le32(ex_msg);
+	ctrl_msg_h->data_length = cpu_to_le32(msg_len);
+
+	if (!msg_len)
+		pkt_header = CCCI_HEADER_NO_DATA;
+
+	return t7xx_port_send_ccci_skb(port, skb, pkt_header, ex_msg);
+}
+
 int t7xx_port_send_skb(struct t7xx_port *port, struct sk_buff *skb, unsigned int pkt_header,
 		       unsigned int ex_msg)
 {
@@ -359,6 +396,9 @@ static void t7xx_proxy_init_all_ports(struct t7xx_modem *md)
 
 		t7xx_port_struct_init(port);
 
+		if (port_conf->tx_ch == PORT_CH_CONTROL_TX)
+			md->core_md.ctl_port = port;
+
 		port->t7xx_dev = md->t7xx_dev;
 		port->dev = &md->t7xx_dev->pdev->dev;
 		spin_lock_init(&port->port_update_lock);
drivers/net/wwan/t7xx/t7xx_port_proxy.h +25 -0
index e3ef269f190e..23f6af0ba3eb 100644
@@ -53,6 +53,27 @@ struct ccci_header {
 #define CCCI_H_SEQ_FLD		GENMASK(30, 16)
 #define CCCI_H_CHN_FLD		GENMASK(15, 0)
 
+struct ctrl_msg_header {
+	__le32	ctrl_msg_id;
+	__le32	ex_msg;
+	__le32	data_length;
+};
+
+/* Control identification numbers for AP<->MD messages  */
+#define CTL_ID_HS1_MSG		0x0
+#define CTL_ID_HS2_MSG		0x1
+#define CTL_ID_HS3_MSG		0x2
+#define CTL_ID_MD_EX		0x4
+#define CTL_ID_DRV_VER_ERROR	0x5
+#define CTL_ID_MD_EX_ACK	0x6
+#define CTL_ID_MD_EX_PASS	0x8
+#define CTL_ID_PORT_ENUM	0x9
+
+/* Modem exception check identification code - "EXCP" */
+#define MD_EX_CHK_ID		0x45584350
+/* Modem exception check acknowledge identification code - "EREC" */
+#define MD_EX_CHK_ACK_ID	0x45524543
+
 #define PORT_INFO_RSRVD		GENMASK(31, 16)
 #define PORT_INFO_ENFLG		BIT(15)
 #define PORT_INFO_CH_ID		GENMASK(14, 0)
@@ -62,10 +83,14 @@ struct ccci_header {
 #define PORT_ENUM_TAIL_PATTERN	0xa5a5a5a5
 #define PORT_ENUM_VER_MISMATCH	0x00657272
 
+/* Port operations mapping */
+extern struct port_ops ctl_port_ops;
+
 void t7xx_port_proxy_reset(struct port_proxy *port_prox);
 void t7xx_port_proxy_uninit(struct port_proxy *port_prox);
 int t7xx_port_proxy_init(struct t7xx_modem *md);
 void t7xx_port_proxy_md_status_notify(struct port_proxy *port_prox, unsigned int state);
+int t7xx_port_enum_msg_handler(struct t7xx_modem *md, void *msg);
 int t7xx_port_proxy_chl_enable_disable(struct port_proxy *port_prox, unsigned int ch_id,
 				       bool en_flag);
 
drivers/net/wwan/t7xx/t7xx_state_monitor.c +3 -0
index 30f3739f1d83..d841f5051af8 100644
@@ -293,6 +293,9 @@ static int fsm_routine_starting(struct t7xx_fsm_ctl *ctl)
 
 	if (!md->core_md.ready) {
 		dev_err(dev, "MD handshake timeout\n");
+		if (md->core_md.handshake_ongoing)
+			t7xx_fsm_append_event(ctl, FSM_EVENT_MD_HS2_EXIT, NULL, 0);
+
 		fsm_routine_exception(ctl, NULL, EXCEPTION_HS_TIMEOUT);
 		return -ETIMEDOUT;
 	}
drivers/net/wwan/t7xx/t7xx_state_monitor.h +2 -0
index a6041a51809e..b1af0259d4c5 100644
@@ -37,9 +37,11 @@ enum t7xx_fsm_state {
 
 enum t7xx_fsm_event_state {
 	FSM_EVENT_INVALID,
+	FSM_EVENT_MD_HS2,
 	FSM_EVENT_MD_EX,
 	FSM_EVENT_MD_EX_REC_OK,
 	FSM_EVENT_MD_EX_PASS,
+	FSM_EVENT_MD_HS2_EXIT,
 	FSM_EVENT_MAX
 };
 
Signature · 1 line
2.25.1
author reviewer
#10 lore raw

PATCH v7 7/14 net: wwan: t7xx: Add AT and MBIM WWAN ports

In reply to Ricardo Martinez

From: Chandrashekar Devegowda <[email protected]>

Adds AT and MBIM ports to the port proxy infrastructure.
The initialization method is responsible for creating the corresponding ports using the WWAN framework infrastructure. The implemented WWAN port operations are start, stop, and TX.

---
 drivers/net/wwan/t7xx/Makefile          |   1 +
 drivers/net/wwan/t7xx/t7xx_port_proxy.c |  20 +++
 drivers/net/wwan/t7xx/t7xx_port_proxy.h |   1 +
 drivers/net/wwan/t7xx/t7xx_port_wwan.c  | 176 ++++++++++++++++++++++++
 4 files changed, 198 insertions(+)
 create mode 100644 drivers/net/wwan/t7xx/t7xx_port_wwan.c
drivers/net/wwan/t7xx/Makefile +1 -0
index 63e1c67b82b5..9eec2e2472fb 100644
@@ -12,3 +12,4 @@ mtk_t7xx-y:=	t7xx_pci.o \
 		t7xx_hif_cldma.o  \
 		t7xx_port_proxy.o  \
 		t7xx_port_ctrl_msg.o \
+		t7xx_port_wwan.o \
drivers/net/wwan/t7xx/t7xx_port_proxy.c +20 -0
index 55f932d2c126..7d2c0e81e33d 100644
@@ -50,6 +50,26 @@
 
 static const struct t7xx_port_conf t7xx_md_port_conf[] = {
 	{
+		.tx_ch = PORT_CH_UART2_TX,
+		.rx_ch = PORT_CH_UART2_RX,
+		.txq_index = Q_IDX_AT_CMD,
+		.rxq_index = Q_IDX_AT_CMD,
+		.txq_exp_index = 0xff,
+		.rxq_exp_index = 0xff,
+		.path_id = CLDMA_ID_MD,
+		.ops = &wwan_sub_port_ops,
+		.name = "AT",
+		.port_type = WWAN_PORT_AT,
+	}, {
+		.tx_ch = PORT_CH_MBIM_TX,
+		.rx_ch = PORT_CH_MBIM_RX,
+		.txq_index = Q_IDX_MBIM,
+		.rxq_index = Q_IDX_MBIM,
+		.path_id = CLDMA_ID_MD,
+		.ops = &wwan_sub_port_ops,
+		.name = "MBIM",
+		.port_type = WWAN_PORT_MBIM,
+	}, {
 		.tx_ch = PORT_CH_CONTROL_TX,
 		.rx_ch = PORT_CH_CONTROL_RX,
 		.txq_index = Q_IDX_CTRL,
drivers/net/wwan/t7xx/t7xx_port_proxy.h +1 -0
index 23f6af0ba3eb..bc1ff5c6c700 100644
@@ -84,6 +84,7 @@ struct ctrl_msg_header {
 #define PORT_ENUM_VER_MISMATCH	0x00657272
 
 /* Port operations mapping */
+extern struct port_ops wwan_sub_port_ops;
 extern struct port_ops ctl_port_ops;
 
 void t7xx_port_proxy_reset(struct port_proxy *port_prox);
drivers/net/wwan/t7xx/t7xx_port_wwan.c new file +176 -0

176 lines not on this page · raw message

Signature · 1 line
2.25.1
Reviewed-by on v7 7/14
#11 lore raw

Re: [PATCH net-next v7 07/14] net: wwan: t7xx: Add AT and MBIM WWAN ports

Quoted 7 lines · Ricardo Martinez

From: Chandrashekar Devegowda <[email protected]>

Adds AT and MBIM ports to the port proxy infrastructure.
The initialization method is responsible for creating the corresponding ports using the WWAN framework infrastructure. The implemented WWAN port operations are start, stop, and TX.

Quoted 5 lines
Signature · 1 line
 i.
author reviewer
#12 lore raw

PATCH v7 8/14 net: wwan: t7xx: Data path HW layer

In reply to Ricardo Martinez

From: Haijun Liu <[email protected]>

Data Path Modem AP Interface (DPMAIF) HW layer provides HW abstraction for the upper layer (DPMAIF HIF). It implements functions to do the HW configuration, TX/RX control and interrupt handling.

---
 drivers/net/wwan/t7xx/t7xx_dpmaif.c | 1283 +++++++++++++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_dpmaif.h |  179 ++++
 drivers/net/wwan/t7xx/t7xx_reg.h    |  213 +++++
 3 files changed, 1675 insertions(+)
 create mode 100644 drivers/net/wwan/t7xx/t7xx_dpmaif.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_dpmaif.h
drivers/net/wwan/t7xx/t7xx_dpmaif.c new file +1283 -0
new file mode 100644
index 000000000000..c8bf6929af51
@@ -0,0 +1,1283 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2021, MediaTek Inc.
+ * Copyright (c) 2021-2022, Intel Corporation.
+ *
+ * Authors:
+ *  Amir Hanania <[email protected]>
+ *  Haijun Liu <[email protected]>
+ *  Moises Veleta <[email protected]>
+ *  Ricardo Martinez <[email protected]>
+ *
+ * Contributors:
+ *  Andy Shevchenko <[email protected]>
+ *  Chiranjeevi Rapolu <[email protected]>
+ *  Eliot Lee <[email protected]>
+ *  Sreehari Kancharla <[email protected]>
+ */
+
+#include <linux/bits.h>
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/delay.h>
+#include <linux/dev_printk.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/types.h>
+
+#include "t7xx_dpmaif.h"
+#include "t7xx_reg.h"
+
+#define ioread32_poll_timeout_atomic(addr, val, cond, delay_us, timeout_us) \
+	readx_poll_timeout_atomic(ioread32, addr, val, cond, delay_us, timeout_us)
+
+static int t7xx_dpmaif_init_intr(struct dpmaif_hw_info *hw_info)
+{
+	struct dpmaif_isr_en_mask *isr_en_msk = &hw_info->isr_en_mask;
+	u32 value, ul_intr_enable, dl_intr_enable;
+	int ret;
+
+	ul_intr_enable = DP_UL_INT_ERR_MSK | DP_UL_INT_QDONE_MSK;
+	isr_en_msk->ap_ul_l2intr_en_msk = ul_intr_enable;
+	iowrite32(DPMAIF_AP_ALL_L2TISAR0_MASK, hw_info->pcie_base + DPMAIF_AP_L2TISAR0);
+
+	/* Set interrupt enable mask */
+	iowrite32(ul_intr_enable, hw_info->pcie_base + DPMAIF_AO_UL_AP_L2TIMCR0);
+	iowrite32(~ul_intr_enable, hw_info->pcie_base + DPMAIF_AO_UL_AP_L2TIMSR0);
+
+	/* Check mask status */
+	ret = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_AO_UL_AP_L2TIMR0,
+					   value, (value & ul_intr_enable) != ul_intr_enable, 0,
+					   DPMAIF_CHECK_INIT_TIMEOUT_US);
+	if (ret)
+		return ret;
+
+	dl_intr_enable = DP_DL_INT_PITCNT_LEN_ERR | DP_DL_INT_BATCNT_LEN_ERR;
+	isr_en_msk->ap_dl_l2intr_err_en_msk = dl_intr_enable;
+	ul_intr_enable = DPMAIF_DL_INT_DLQ0_QDONE | DPMAIF_DL_INT_DLQ0_PITCNT_LEN |
+		    DPMAIF_DL_INT_DLQ1_QDONE | DPMAIF_DL_INT_DLQ1_PITCNT_LEN;
+	isr_en_msk->ap_ul_l2intr_en_msk = ul_intr_enable;
+	iowrite32(DPMAIF_AP_APDL_ALL_L2TISAR0_MASK, hw_info->pcie_base + DPMAIF_AP_APDL_L2TISAR0);
+
+	/* Set DL ISR PD enable mask */
+	iowrite32(~ul_intr_enable, hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMSR0);
+	ret = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMR0,
+					   value, (value & ul_intr_enable) != ul_intr_enable, 0,
+					   DPMAIF_CHECK_INIT_TIMEOUT_US);
+	if (ret)
+		return ret;
+
+	isr_en_msk->ap_udl_ip_busy_en_msk = DPMAIF_UDL_IP_BUSY;
+	iowrite32(DPMAIF_AP_IP_BUSY_MASK, hw_info->pcie_base + DPMAIF_AP_IP_BUSY);
+	iowrite32(isr_en_msk->ap_udl_ip_busy_en_msk,
+		  hw_info->pcie_base + DPMAIF_AO_AP_DLUL_IP_BUSY_MASK);
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_UL_AP_L1TIMR0);
+	value |= DPMAIF_DL_INT_Q2APTOP | DPMAIF_DL_INT_Q2TOQ1;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_UL_AP_L1TIMR0);
+	iowrite32(DPMA_HPC_ALL_INT_MASK, hw_info->pcie_base + DPMAIF_HPC_INTR_MASK);
+
+	return 0;
+}
+
+static void t7xx_dpmaif_mask_ulq_intr(struct dpmaif_hw_info *hw_info, unsigned int q_num)
+{
+	struct dpmaif_isr_en_mask *isr_en_msk;
+	u32 value, ul_int_que_done;
+	int ret;
+
+	isr_en_msk = &hw_info->isr_en_mask;
+	ul_int_que_done = BIT(q_num + DP_UL_INT_DONE_OFFSET) & DP_UL_INT_QDONE_MSK;
+	isr_en_msk->ap_ul_l2intr_en_msk &= ~ul_int_que_done;
+	iowrite32(ul_int_que_done, hw_info->pcie_base + DPMAIF_AO_UL_AP_L2TIMSR0);
+
+	ret = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_AO_UL_AP_L2TIMR0,
+					   value, (value & ul_int_que_done) == ul_int_que_done, 0,
+					   DPMAIF_CHECK_TIMEOUT_US);
+	if (ret)
+		dev_err(hw_info->dev,
+			"Could not mask the UL interrupt. DPMAIF_AO_UL_AP_L2TIMR0 is 0x%x\n",
+			value);
+}
+
+void t7xx_dpmaif_unmask_ulq_intr(struct dpmaif_hw_info *hw_info, unsigned int q_num)
+{
+	struct dpmaif_isr_en_mask *isr_en_msk;
+	u32 value, ul_int_que_done;
+	int ret;
+
+	isr_en_msk = &hw_info->isr_en_mask;
+	ul_int_que_done = BIT(q_num + DP_UL_INT_DONE_OFFSET) & DP_UL_INT_QDONE_MSK;
+	isr_en_msk->ap_ul_l2intr_en_msk |= ul_int_que_done;
+	iowrite32(ul_int_que_done, hw_info->pcie_base + DPMAIF_AO_UL_AP_L2TIMCR0);
+
+	ret = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_AO_UL_AP_L2TIMR0,
+					   value, (value & ul_int_que_done) != ul_int_que_done, 0,
+					   DPMAIF_CHECK_TIMEOUT_US);
+	if (ret)
+		dev_err(hw_info->dev,
+			"Could not unmask the UL interrupt. DPMAIF_AO_UL_AP_L2TIMR0 is 0x%x\n",
+			value);
+}
+
+void t7xx_dpmaif_dl_unmask_batcnt_len_err_intr(struct dpmaif_hw_info *hw_info)
+{
+	hw_info->isr_en_mask.ap_dl_l2intr_en_msk |= DP_DL_INT_BATCNT_LEN_ERR;
+	iowrite32(DP_DL_INT_BATCNT_LEN_ERR, hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMCR0);
+}
+
+void t7xx_dpmaif_dl_unmask_pitcnt_len_err_intr(struct dpmaif_hw_info *hw_info)
+{
+	hw_info->isr_en_mask.ap_dl_l2intr_en_msk |= DP_DL_INT_PITCNT_LEN_ERR;
+	iowrite32(DP_DL_INT_PITCNT_LEN_ERR, hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMCR0);
+}
+
+static u32 t7xx_update_dlq_intr(struct dpmaif_hw_info *hw_info, u32 q_done)
+{
+	u32 value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_UL_AP_L2TIMR0);
+	iowrite32(q_done, hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMSR0);
+	return value;
+}
+
+static int t7xx_mask_dlq_intr(struct dpmaif_hw_info *hw_info, unsigned int qno)
+{
+	u32 value, q_done;
+	int ret;
+
+	q_done = qno == DPF_RX_QNO0 ? DPMAIF_DL_INT_DLQ0_QDONE : DPMAIF_DL_INT_DLQ1_QDONE;
+	iowrite32(q_done, hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMSR0);
+
+	ret = read_poll_timeout_atomic(t7xx_update_dlq_intr, value, value & q_done,
+				       0, DPMAIF_CHECK_TIMEOUT_US, false, hw_info, q_done);
+	if (ret) {
+		dev_err(hw_info->dev,
+			"Could not mask the DL interrupt. DPMAIF_AO_UL_AP_L2TIMR0 is 0x%x\n",
+			value);
+		return -ETIMEDOUT;
+	}
+
+	hw_info->isr_en_mask.ap_dl_l2intr_en_msk &= ~q_done;
+	return 0;
+}
+
+void t7xx_dpmaif_dlq_unmask_rx_done(struct dpmaif_hw_info *hw_info, unsigned int qno)
+{
+	u32 mask;
+
+	mask = qno == DPF_RX_QNO0 ? DPMAIF_DL_INT_DLQ0_QDONE : DPMAIF_DL_INT_DLQ1_QDONE;
+	iowrite32(mask, hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMCR0);
+	hw_info->isr_en_mask.ap_dl_l2intr_en_msk |= mask;
+}
+
+void t7xx_dpmaif_clr_ip_busy_sts(struct dpmaif_hw_info *hw_info)
+{
+	u32 ip_busy_sts;
+
+	ip_busy_sts = ioread32(hw_info->pcie_base + DPMAIF_AP_IP_BUSY);
+	iowrite32(ip_busy_sts, hw_info->pcie_base + DPMAIF_AP_IP_BUSY);
+}
+
+static void t7xx_dpmaif_dlq_mask_rx_pitcnt_len_err_intr(struct dpmaif_hw_info *hw_info,
+							unsigned int qno)
+{
+	if (qno == DPF_RX_QNO0)
+		iowrite32(DPMAIF_DL_INT_DLQ0_PITCNT_LEN,
+			  hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMSR0);
+	else
+		iowrite32(DPMAIF_DL_INT_DLQ1_PITCNT_LEN,
+			  hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMSR0);
+}
+
+void t7xx_dpmaif_dlq_unmask_pitcnt_len_err_intr(struct dpmaif_hw_info *hw_info,
+						unsigned int qno)
+{
+	if (qno == DPF_RX_QNO0)
+		iowrite32(DPMAIF_DL_INT_DLQ0_PITCNT_LEN,
+			  hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMCR0);
+	else
+		iowrite32(DPMAIF_DL_INT_DLQ1_PITCNT_LEN,
+			  hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMCR0);
+}
+
+void t7xx_dpmaif_ul_clr_all_intr(struct dpmaif_hw_info *hw_info)
+{
+	iowrite32(DPMAIF_AP_ALL_L2TISAR0_MASK, hw_info->pcie_base + DPMAIF_AP_L2TISAR0);
+}
+
+void t7xx_dpmaif_dl_clr_all_intr(struct dpmaif_hw_info *hw_info)
+{
+	iowrite32(DPMAIF_AP_APDL_ALL_L2TISAR0_MASK, hw_info->pcie_base + DPMAIF_AP_APDL_L2TISAR0);
+}
+
+static void t7xx_dpmaif_set_intr_para(struct dpmaif_hw_intr_st_para *para,
+				      enum dpmaif_hw_intr_type intr_type, unsigned int intr_queue)
+{
+	para->intr_types[para->intr_cnt] = intr_type;
+	para->intr_queues[para->intr_cnt] = intr_queue;
+	para->intr_cnt++;
+}
+
+/* The para->intr_cnt counter is set to zero before this function is called.
+ * It does not check for overflow as there is no risk of overflowing intr_types or intr_queues.
+ */
+static void t7xx_dpmaif_hw_check_tx_intr(struct dpmaif_hw_info *hw_info,
+					 unsigned int intr_status,
+					 struct dpmaif_hw_intr_st_para *para)
+{
+	unsigned long value;
+
+	value = FIELD_GET(DP_UL_INT_QDONE_MSK, intr_status);
+	if (value) {
+		unsigned int index;
+
+		t7xx_dpmaif_set_intr_para(para, DPF_INTR_UL_DONE, value);
+
+		for_each_set_bit(index, &value, DPMAIF_TXQ_NUM)
+			t7xx_dpmaif_mask_ulq_intr(hw_info, index);
+	}
+
+	value = FIELD_GET(DP_UL_INT_EMPTY_MSK, intr_status);
+	if (value)
+		t7xx_dpmaif_set_intr_para(para, DPF_INTR_UL_DRB_EMPTY, value);
+
+	value = FIELD_GET(DP_UL_INT_MD_NOTREADY_MSK, intr_status);
+	if (value)
+		t7xx_dpmaif_set_intr_para(para, DPF_INTR_UL_MD_NOTREADY, value);
+
+	value = FIELD_GET(DP_UL_INT_MD_PWR_NOTREADY_MSK, intr_status);
+	if (value)
+		t7xx_dpmaif_set_intr_para(para, DPF_INTR_UL_MD_PWR_NOTREADY, value);
+
+	value = FIELD_GET(DP_UL_INT_ERR_MSK, intr_status);
+	if (value)
+		t7xx_dpmaif_set_intr_para(para, DPF_INTR_UL_LEN_ERR, value);
+
+	/* Clear interrupt status */
+	iowrite32(intr_status, hw_info->pcie_base + DPMAIF_AP_L2TISAR0);
+}
+
+/* The para->intr_cnt counter is set to zero before this function is called.
+ * It does not check for overflow as there is no risk of overflowing intr_types or intr_queues.
+ */
+static void t7xx_dpmaif_hw_check_rx_intr(struct dpmaif_hw_info *hw_info,
+					 unsigned int intr_status,
+					 struct dpmaif_hw_intr_st_para *para, int qno)
+{
+	if (qno == DPF_RX_QNO_DFT) {
+		if (intr_status & DP_DL_INT_SKB_LEN_ERR)
+			t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_SKB_LEN_ERR, DPF_RX_QNO_DFT);
+
+		if (intr_status & DP_DL_INT_BATCNT_LEN_ERR) {
+			t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_BATCNT_LEN_ERR, DPF_RX_QNO_DFT);
+			hw_info->isr_en_mask.ap_dl_l2intr_en_msk &= ~DP_DL_INT_BATCNT_LEN_ERR;
+			iowrite32(DP_DL_INT_BATCNT_LEN_ERR,
+				  hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMSR0);
+		}
+
+		if (intr_status & DP_DL_INT_PITCNT_LEN_ERR) {
+			t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_PITCNT_LEN_ERR, DPF_RX_QNO_DFT);
+			hw_info->isr_en_mask.ap_dl_l2intr_en_msk &= ~DP_DL_INT_PITCNT_LEN_ERR;
+			iowrite32(DP_DL_INT_PITCNT_LEN_ERR,
+				  hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMSR0);
+		}
+
+		if (intr_status & DP_DL_INT_PKT_EMPTY_MSK)
+			t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_PKT_EMPTY_SET, DPF_RX_QNO_DFT);
+
+		if (intr_status & DP_DL_INT_FRG_EMPTY_MSK)
+			t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_FRG_EMPTY_SET, DPF_RX_QNO_DFT);
+
+		if (intr_status & DP_DL_INT_MTU_ERR_MSK)
+			t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_MTU_ERR, DPF_RX_QNO_DFT);
+
+		if (intr_status & DP_DL_INT_FRG_LEN_ERR_MSK)
+			t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_FRGCNT_LEN_ERR, DPF_RX_QNO_DFT);
+
+		if (intr_status & DP_DL_INT_Q0_PITCNT_LEN_ERR) {
+			t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_Q0_PITCNT_LEN_ERR, BIT(qno));
+			t7xx_dpmaif_dlq_mask_rx_pitcnt_len_err_intr(hw_info, qno);
+		}
+
+		if (intr_status & DP_DL_INT_HPC_ENT_TYPE_ERR)
+			t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_HPC_ENT_TYPE_ERR,
+						  DPF_RX_QNO_DFT);
+
+		if (intr_status & DP_DL_INT_Q0_DONE) {
+			/* Mask RX done interrupt immediately after it occurs, do not clear
+			 * the interrupt if the mask operation fails.
+			 */
+			if (!t7xx_mask_dlq_intr(hw_info, qno))
+				t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_Q0_DONE, BIT(qno));
+			else
+				intr_status &= ~DP_DL_INT_Q0_DONE;
+		}
+	} else {
+		if (intr_status & DP_DL_INT_Q1_PITCNT_LEN_ERR) {
+			t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_Q1_PITCNT_LEN_ERR, BIT(qno));
+			t7xx_dpmaif_dlq_mask_rx_pitcnt_len_err_intr(hw_info, qno);
+		}
+
+		if (intr_status & DP_DL_INT_Q1_DONE) {
+			if (!t7xx_mask_dlq_intr(hw_info, qno))
+				t7xx_dpmaif_set_intr_para(para, DPF_INTR_DL_Q1_DONE, BIT(qno));
+			else
+				intr_status &= ~DP_DL_INT_Q1_DONE;
+		}
+	}
+
+	intr_status |= DP_DL_INT_BATCNT_LEN_ERR;
+	/* Clear interrupt status */
+	iowrite32(intr_status, hw_info->pcie_base + DPMAIF_AP_APDL_L2TISAR0);
+}
+
+/**
+ * t7xx_dpmaif_hw_get_intr_cnt() - Reads interrupt status and count from HW.
+ * @hw_info: Pointer to struct hw_info.
+ * @para: Pointer to struct dpmaif_hw_intr_st_para.
+ * @qno: Queue number.
+ *
+ * Reads RX/TX interrupt status from HW and clears UL/DL status as needed.
+ *
+ * Return: Interrupt count.
+ */
+int t7xx_dpmaif_hw_get_intr_cnt(struct dpmaif_hw_info *hw_info,
+				struct dpmaif_hw_intr_st_para *para, int qno)
+{
+	u32 rx_intr_status, tx_intr_status = 0;
+	u32 rx_intr_qdone, tx_intr_qdone = 0;
+
+	rx_intr_status = ioread32(hw_info->pcie_base + DPMAIF_AP_APDL_L2TISAR0);
+	rx_intr_qdone = ioread32(hw_info->pcie_base + DPMAIF_AO_UL_APDL_L2TIMR0);
+
+	/* TX interrupt status */
+	if (qno == DPF_RX_QNO_DFT) {
+		/* All ULQ and DLQ0 interrupts use the same source no need to check ULQ interrupts
+		 * when a DLQ1 interrupt has occurred.
+		 */
+		tx_intr_status = ioread32(hw_info->pcie_base + DPMAIF_AP_L2TISAR0);
+		tx_intr_qdone = ioread32(hw_info->pcie_base + DPMAIF_AO_UL_AP_L2TIMR0);
+	}
+
+	t7xx_dpmaif_clr_ip_busy_sts(hw_info);
+
+	if (qno == DPF_RX_QNO_DFT) {
+		/* Do not schedule bottom half again or clear UL interrupt status when we
+		 * have already masked it.
+		 */
+		tx_intr_status &= ~tx_intr_qdone;
+		if (tx_intr_status)
+			t7xx_dpmaif_hw_check_tx_intr(hw_info, tx_intr_status, para);
+	}
+
+	if (rx_intr_status) {
+		if (qno == DPF_RX_QNO0) {
+			rx_intr_status &= DP_DL_Q0_STATUS_MASK;
+			if (rx_intr_qdone & DPMAIF_DL_INT_DLQ0_QDONE)
+				/* Do not schedule bottom half again or clear DL
+				 * queue done interrupt status when we have already masked it.
+				 */
+				rx_intr_status &= ~DP_DL_INT_Q0_DONE;
+		} else {
+			rx_intr_status &= DP_DL_Q1_STATUS_MASK;
+			if (rx_intr_qdone & DPMAIF_DL_INT_DLQ1_QDONE)
+				rx_intr_status &= ~DP_DL_INT_Q1_DONE;
+		}
+
+		if (rx_intr_status)
+			t7xx_dpmaif_hw_check_rx_intr(hw_info, rx_intr_status, para, qno);
+	}
+
+	return para->intr_cnt;
+}
+
+static int t7xx_dpmaif_sram_init(struct dpmaif_hw_info *hw_info)
+{
+	u32 value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AP_MEM_CLR);
+	value |= DPMAIF_MEM_CLR;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AP_MEM_CLR);
+
+	return ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_AP_MEM_CLR,
+					    value, !(value & DPMAIF_MEM_CLR), 0,
+					    DPMAIF_CHECK_INIT_TIMEOUT_US);
+}
+
+static void t7xx_dpmaif_hw_reset(struct dpmaif_hw_info *hw_info)
+{
+	iowrite32(DPMAIF_AP_AO_RST_BIT, hw_info->pcie_base + DPMAIF_AP_AO_RGU_ASSERT);
+	udelay(2);
+	iowrite32(DPMAIF_AP_RST_BIT, hw_info->pcie_base + DPMAIF_AP_RGU_ASSERT);
+	udelay(2);
+	iowrite32(DPMAIF_AP_AO_RST_BIT, hw_info->pcie_base + DPMAIF_AP_AO_RGU_DEASSERT);
+	udelay(2);
+	iowrite32(DPMAIF_AP_RST_BIT, hw_info->pcie_base + DPMAIF_AP_RGU_DEASSERT);
+	udelay(2);
+}
+
+static int t7xx_dpmaif_hw_config(struct dpmaif_hw_info *hw_info)
+{
+	u32 ap_port_mode;
+	int ret;
+
+	t7xx_dpmaif_hw_reset(hw_info);
+
+	ret = t7xx_dpmaif_sram_init(hw_info);
+	if (ret)
+		return ret;
+
+	ap_port_mode = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_THRES);
+	ap_port_mode |= DPMAIF_PORT_MODE_PCIE;
+	iowrite32(ap_port_mode, hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_THRES);
+	iowrite32(DPMAIF_CG_EN, hw_info->pcie_base + DPMAIF_AP_CG_EN);
+	return 0;
+}
+
+static void t7xx_dpmaif_pcie_dpmaif_sign(struct dpmaif_hw_info *hw_info)
+{
+	iowrite32(DPMAIF_PCIE_MODE_SET_VALUE, hw_info->pcie_base + DPMAIF_UL_RESERVE_AO_RW);
+}
+
+static void t7xx_dpmaif_dl_performance(struct dpmaif_hw_info *hw_info)
+{
+	u32 enable_bat_cache, enable_pit_burst;
+
+	enable_bat_cache = ioread32(hw_info->pcie_base + DPMAIF_DL_BAT_INIT_CON1);
+	enable_bat_cache |= DPMAIF_DL_BAT_CACHE_PRI;
+	iowrite32(enable_bat_cache, hw_info->pcie_base + DPMAIF_DL_BAT_INIT_CON1);
+
+	enable_pit_burst = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_THRES);
+	enable_pit_burst |= DPMAIF_DL_BURST_PIT_EN;
+	iowrite32(enable_pit_burst, hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_THRES);
+}
+
+ /* DPMAIF DL DLQ part HW setting */
+
+static void t7xx_dpmaif_hw_hpc_cntl_set(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = DPMAIF_HPC_DLQ_PATH_MODE | DPMAIF_HPC_ADD_MODE_DF << 2;
+	value |= DPMAIF_HASH_PRIME_DF << 4;
+	value |= DPMAIF_HPC_TOTAL_NUM << 8;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_HPC_CNTL);
+}
+
+static void t7xx_dpmaif_hw_agg_cfg_set(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = DPMAIF_AGG_MAX_LEN_DF | DPMAIF_AGG_TBL_ENT_NUM_DF << 16;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_DLQ_AGG_CFG);
+}
+
+static void t7xx_dpmaif_hw_hash_bit_choose_set(struct dpmaif_hw_info *hw_info)
+{
+	iowrite32(DPMAIF_DLQ_HASH_BIT_CHOOSE_DF,
+		  hw_info->pcie_base + DPMAIF_AO_DL_DLQPIT_INIT_CON5);
+}
+
+static void t7xx_dpmaif_hw_mid_pit_timeout_thres_set(struct dpmaif_hw_info *hw_info)
+{
+	iowrite32(DPMAIF_MID_TIMEOUT_THRES_DF, hw_info->pcie_base + DPMAIF_AO_DL_DLQPIT_TIMEOUT0);
+}
+
+static void t7xx_dpmaif_hw_dlq_timeout_thres_set(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value, i;
+
+	/* Each register holds two DLQ threshold timeout values */
+	for (i = 0; i < DPMAIF_HPC_MAX_TOTAL_NUM / 2; i++) {
+		value = FIELD_PREP(DPMAIF_DLQ_LOW_TIMEOUT_THRES_MKS, DPMAIF_DLQ_TIMEOUT_THRES_DF);
+		value |= FIELD_PREP(DPMAIF_DLQ_HIGH_TIMEOUT_THRES_MSK,
+				    DPMAIF_DLQ_TIMEOUT_THRES_DF);
+		iowrite32(value,
+			  hw_info->pcie_base + DPMAIF_AO_DL_DLQPIT_TIMEOUT1 + sizeof(u32) * i);
+	}
+}
+
+static void t7xx_dpmaif_hw_dlq_start_prs_thres_set(struct dpmaif_hw_info *hw_info)
+{
+	iowrite32(DPMAIF_DLQ_PRS_THRES_DF, hw_info->pcie_base + DPMAIF_AO_DL_DLQPIT_TRIG_THRES);
+}
+
+static void t7xx_dpmaif_dl_dlq_hpc_hw_init(struct dpmaif_hw_info *hw_info)
+{
+	t7xx_dpmaif_hw_hpc_cntl_set(hw_info);
+	t7xx_dpmaif_hw_agg_cfg_set(hw_info);
+	t7xx_dpmaif_hw_hash_bit_choose_set(hw_info);
+	t7xx_dpmaif_hw_mid_pit_timeout_thres_set(hw_info);
+	t7xx_dpmaif_hw_dlq_timeout_thres_set(hw_info);
+	t7xx_dpmaif_hw_dlq_start_prs_thres_set(hw_info);
+}
+
+static int t7xx_dpmaif_dl_bat_init_done(struct dpmaif_hw_info *hw_info, bool frg_en)
+{
+	u32 value, dl_bat_init = 0;
+	int ret;
+
+	if (frg_en)
+		dl_bat_init = DPMAIF_DL_BAT_FRG_INIT;
+
+	dl_bat_init |= DPMAIF_DL_BAT_INIT_ALLSET;
+	dl_bat_init |= DPMAIF_DL_BAT_INIT_EN;
+
+	ret = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_DL_BAT_INIT,
+					   value, !(value & DPMAIF_DL_BAT_INIT_NOT_READY), 0,
+					   DPMAIF_CHECK_INIT_TIMEOUT_US);
+	if (ret) {
+		dev_err(hw_info->dev, "Data plane modem DL BAT is not ready\n");
+		return ret;
+	}
+
+	iowrite32(dl_bat_init, hw_info->pcie_base + DPMAIF_DL_BAT_INIT);
+
+	ret = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_DL_BAT_INIT,
+					   value, !(value & DPMAIF_DL_BAT_INIT_NOT_READY), 0,
+					   DPMAIF_CHECK_INIT_TIMEOUT_US);
+	if (ret)
+		dev_err(hw_info->dev, "Data plane modem DL BAT initialization failed\n");
+
+	return ret;
+}
+
+static void t7xx_dpmaif_dl_set_bat_base_addr(struct dpmaif_hw_info *hw_info,
+					     dma_addr_t addr)
+{
+	iowrite32(lower_32_bits(addr), hw_info->pcie_base + DPMAIF_DL_BAT_INIT_CON0);
+	iowrite32(upper_32_bits(addr), hw_info->pcie_base + DPMAIF_DL_BAT_INIT_CON3);
+}
+
+static void t7xx_dpmaif_dl_set_bat_size(struct dpmaif_hw_info *hw_info, unsigned int size)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_DL_BAT_INIT_CON1);
+	value &= ~DPMAIF_BAT_SIZE_MSK;
+	value |= size & DPMAIF_BAT_SIZE_MSK;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_DL_BAT_INIT_CON1);
+}
+
+static void t7xx_dpmaif_dl_bat_en(struct dpmaif_hw_info *hw_info, bool enable)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_DL_BAT_INIT_CON1);
+
+	if (enable)
+		value |= DPMAIF_BAT_EN_MSK;
+	else
+		value &= ~DPMAIF_BAT_EN_MSK;
+
+	iowrite32(value, hw_info->pcie_base + DPMAIF_DL_BAT_INIT_CON1);
+}
+
+static void t7xx_dpmaif_dl_set_ao_bid_maxcnt(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON0);
+	value &= ~DPMAIF_BAT_BID_MAXCNT_MSK;
+	value |= FIELD_PREP(DPMAIF_BAT_BID_MAXCNT_MSK, DPMAIF_HW_PKT_BIDCNT);
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON0);
+}
+
+static void t7xx_dpmaif_dl_set_ao_mtu(struct dpmaif_hw_info *hw_info)
+{
+	iowrite32(DPMAIF_HW_MTU_SIZE, hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON1);
+}
+
+static void t7xx_dpmaif_dl_set_ao_pit_chknum(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON2);
+	value &= ~DPMAIF_PIT_CHK_NUM_MSK;
+	value |= FIELD_PREP(DPMAIF_PIT_CHK_NUM_MSK, DPMAIF_HW_CHK_PIT_NUM);
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON2);
+}
+
+static void t7xx_dpmaif_dl_set_ao_remain_minsz(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON0);
+	value &= ~DPMAIF_BAT_REMAIN_MINSZ_MSK;
+	value |= FIELD_PREP(DPMAIF_BAT_REMAIN_MINSZ_MSK,
+			    DPMAIF_HW_BAT_REMAIN / DPMAIF_BAT_REMAIN_SZ_BASE);
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON0);
+}
+
+static void t7xx_dpmaif_dl_set_ao_bat_bufsz(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON2);
+	value &= ~DPMAIF_BAT_BUF_SZ_MSK;
+	value |= FIELD_PREP(DPMAIF_BAT_BUF_SZ_MSK,
+			    DPMAIF_HW_BAT_PKTBUF / DPMAIF_BAT_BUFFER_SZ_BASE);
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON2);
+}
+
+static void t7xx_dpmaif_dl_set_ao_bat_rsv_length(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON2);
+	value &= ~DPMAIF_BAT_RSV_LEN_MSK;
+	value |= DPMAIF_HW_BAT_RSVLEN;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_PKTINFO_CON2);
+}
+
+static void t7xx_dpmaif_dl_set_pkt_alignment(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_THRES);
+	value &= ~DPMAIF_PKT_ALIGN_MSK;
+	value |= DPMAIF_PKT_ALIGN_EN;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_THRES);
+}
+
+static void t7xx_dpmaif_dl_set_pkt_checksum(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_THRES);
+	value |= DPMAIF_DL_PKT_CHECKSUM_EN;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_THRES);
+}
+
+static void t7xx_dpmaif_dl_set_ao_frg_check_thres(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_FRG_THRES);
+	value &= ~DPMAIF_FRG_CHECK_THRES_MSK;
+	value |= DPMAIF_HW_CHK_FRG_NUM;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_FRG_THRES);
+}
+
+static void t7xx_dpmaif_dl_set_ao_frg_bufsz(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_FRG_THRES);
+	value &= ~DPMAIF_FRG_BUF_SZ_MSK;
+	value |= FIELD_PREP(DPMAIF_FRG_BUF_SZ_MSK,
+			    DPMAIF_HW_FRG_PKTBUF / DPMAIF_FRG_BUFFER_SZ_BASE);
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_FRG_THRES);
+}
+
+static void t7xx_dpmaif_dl_frg_ao_en(struct dpmaif_hw_info *hw_info, bool enable)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_FRG_THRES);
+
+	if (enable)
+		value |= DPMAIF_FRG_EN_MSK;
+	else
+		value &= ~DPMAIF_FRG_EN_MSK;
+
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_FRG_THRES);
+}
+
+static void t7xx_dpmaif_dl_set_ao_bat_check_thres(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_THRES);
+	value &= ~DPMAIF_BAT_CHECK_THRES_MSK;
+	value |= FIELD_PREP(DPMAIF_BAT_CHECK_THRES_MSK, DPMAIF_HW_CHK_BAT_NUM);
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_RDY_CHK_THRES);
+}
+
+static void t7xx_dpmaif_dl_set_pit_seqnum(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_PIT_SEQ_END);
+	value &= ~DPMAIF_DL_PIT_SEQ_MSK;
+	value |= DPMAIF_DL_PIT_SEQ_VALUE;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_DL_PIT_SEQ_END);
+}
+
+static void t7xx_dpmaif_dl_set_dlq_pit_base_addr(struct dpmaif_hw_info *hw_info,
+						 dma_addr_t addr)
+{
+	iowrite32(lower_32_bits(addr), hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT_CON0);
+	iowrite32(upper_32_bits(addr), hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT_CON4);
+}
+
+static void t7xx_dpmaif_dl_set_dlq_pit_size(struct dpmaif_hw_info *hw_info, unsigned int size)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT_CON1);
+	value &= ~DPMAIF_PIT_SIZE_MSK;
+	value |= size & DPMAIF_PIT_SIZE_MSK;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT_CON1);
+	iowrite32(0, hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT_CON2);
+	iowrite32(0, hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT_CON3);
+	iowrite32(0, hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT_CON5);
+	iowrite32(0, hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT_CON6);
+}
+
+static void t7xx_dpmaif_dl_dlq_pit_en(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT_CON3);
+	value |= DPMAIF_DLQPIT_EN_MSK;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT_CON3);
+}
+
+static void t7xx_dpmaif_dl_dlq_pit_init_done(struct dpmaif_hw_info *hw_info,
+					     unsigned int pit_idx)
+{
+	unsigned int dl_pit_init;
+	int timeout;
+	u32 value;
+
+	dl_pit_init = DPMAIF_DL_PIT_INIT_ALLSET;
+	dl_pit_init |= (pit_idx << DPMAIF_DLQPIT_CHAN_OFS);
+	dl_pit_init |= DPMAIF_DL_PIT_INIT_EN;
+
+	timeout = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT,
+					       value, !(value & DPMAIF_DL_PIT_INIT_NOT_READY),
+					       DPMAIF_CHECK_DELAY_US,
+					       DPMAIF_CHECK_INIT_TIMEOUT_US);
+	if (timeout) {
+		dev_err(hw_info->dev, "Data plane modem DL PIT is not ready\n");
+		return;
+	}
+
+	iowrite32(dl_pit_init, hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT);
+	timeout = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_DL_DLQPIT_INIT,
+					       value, !(value & DPMAIF_DL_PIT_INIT_NOT_READY),
+					       DPMAIF_CHECK_DELAY_US,
+					       DPMAIF_CHECK_INIT_TIMEOUT_US);
+	if (timeout)
+		dev_err(hw_info->dev, "Data plane modem DL PIT initialization failed\n");
+}
+
+static void t7xx_dpmaif_config_dlq_pit_hw(struct dpmaif_hw_info *hw_info, unsigned int q_num,
+					  struct dpmaif_dl *dl_que)
+{
+	t7xx_dpmaif_dl_set_dlq_pit_base_addr(hw_info, dl_que->pit_base);
+	t7xx_dpmaif_dl_set_dlq_pit_size(hw_info, dl_que->pit_size_cnt);
+	t7xx_dpmaif_dl_dlq_pit_en(hw_info);
+	t7xx_dpmaif_dl_dlq_pit_init_done(hw_info, q_num);
+}
+
+static void t7xx_dpmaif_config_all_dlq_hw(struct dpmaif_hw_info *hw_info)
+{
+	int i;
+
+	for (i = 0; i < DPMAIF_RXQ_NUM; i++)
+		t7xx_dpmaif_config_dlq_pit_hw(hw_info, i, &hw_info->dl_que[i]);
+}
+
+static void t7xx_dpmaif_dl_all_q_en(struct dpmaif_hw_info *hw_info, bool enable)
+{
+	u32 dl_bat_init, value;
+	int timeout;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_DL_BAT_INIT_CON1);
+
+	if (enable)
+		value |= DPMAIF_BAT_EN_MSK;
+	else
+		value &= ~DPMAIF_BAT_EN_MSK;
+
+	iowrite32(value, hw_info->pcie_base + DPMAIF_DL_BAT_INIT_CON1);
+	dl_bat_init = DPMAIF_DL_BAT_INIT_ONLY_ENABLE_BIT;
+	dl_bat_init |= DPMAIF_DL_BAT_INIT_EN;
+
+	timeout = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_DL_BAT_INIT,
+					       value, !(value & DPMAIF_DL_BAT_INIT_NOT_READY), 0,
+					       DPMAIF_CHECK_TIMEOUT_US);
+	if (timeout)
+		dev_err(hw_info->dev, "Timeout updating BAT setting to HW\n");
+
+	iowrite32(dl_bat_init, hw_info->pcie_base + DPMAIF_DL_BAT_INIT);
+	timeout = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_DL_BAT_INIT,
+					       value, !(value & DPMAIF_DL_BAT_INIT_NOT_READY), 0,
+					       DPMAIF_CHECK_TIMEOUT_US);
+	if (timeout)
+		dev_err(hw_info->dev, "Data plane modem DL BAT is not ready\n");
+}
+
+static int t7xx_dpmaif_config_dlq_hw(struct dpmaif_hw_info *hw_info)
+{
+	struct dpmaif_dl *dl_que;
+	int ret;
+
+	t7xx_dpmaif_dl_dlq_hpc_hw_init(hw_info);
+
+	dl_que = &hw_info->dl_que[0]; /* All queues share one BAT/frag BAT table */
+	if (!dl_que->que_started)
+		return -EBUSY;
+
+	t7xx_dpmaif_dl_set_ao_remain_minsz(hw_info);
+	t7xx_dpmaif_dl_set_ao_bat_bufsz(hw_info);
+	t7xx_dpmaif_dl_set_ao_frg_bufsz(hw_info);
+	t7xx_dpmaif_dl_set_ao_bat_rsv_length(hw_info);
+	t7xx_dpmaif_dl_set_ao_bid_maxcnt(hw_info);
+	t7xx_dpmaif_dl_set_pkt_alignment(hw_info);
+	t7xx_dpmaif_dl_set_pit_seqnum(hw_info);
+	t7xx_dpmaif_dl_set_ao_mtu(hw_info);
+	t7xx_dpmaif_dl_set_ao_pit_chknum(hw_info);
+	t7xx_dpmaif_dl_set_ao_bat_check_thres(hw_info);
+	t7xx_dpmaif_dl_set_ao_frg_check_thres(hw_info);
+	t7xx_dpmaif_dl_frg_ao_en(hw_info, true);
+
+	t7xx_dpmaif_dl_set_bat_base_addr(hw_info, dl_que->frg_base);
+	t7xx_dpmaif_dl_set_bat_size(hw_info, dl_que->frg_size_cnt);
+	t7xx_dpmaif_dl_bat_en(hw_info, true);
+
+	ret = t7xx_dpmaif_dl_bat_init_done(hw_info, true);
+	if (ret)
+		return ret;
+
+	t7xx_dpmaif_dl_set_bat_base_addr(hw_info, dl_que->bat_base);
+	t7xx_dpmaif_dl_set_bat_size(hw_info, dl_que->bat_size_cnt);
+	t7xx_dpmaif_dl_bat_en(hw_info, false);
+
+	ret = t7xx_dpmaif_dl_bat_init_done(hw_info, false);
+	if (ret)
+		return ret;
+
+	/* Init PIT (two PIT table) */
+	t7xx_dpmaif_config_all_dlq_hw(hw_info);
+	t7xx_dpmaif_dl_all_q_en(hw_info, true);
+	t7xx_dpmaif_dl_set_pkt_checksum(hw_info);
+	return 0;
+}
+
+static void t7xx_dpmaif_ul_update_drb_size(struct dpmaif_hw_info *hw_info,
+					   unsigned int q_num, unsigned int size)
+{
+	unsigned int value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_UL_DRBSIZE_ADDRH_n(q_num));
+	value &= ~DPMAIF_DRB_SIZE_MSK;
+	value |= size & DPMAIF_DRB_SIZE_MSK;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_UL_DRBSIZE_ADDRH_n(q_num));
+}
+
+static void t7xx_dpmaif_ul_update_drb_base_addr(struct dpmaif_hw_info *hw_info,
+						unsigned int q_num, dma_addr_t addr)
+{
+	iowrite32(lower_32_bits(addr), hw_info->pcie_base + DPMAIF_ULQSAR_n(q_num));
+	iowrite32(upper_32_bits(addr), hw_info->pcie_base + DPMAIF_UL_DRB_ADDRH_n(q_num));
+}
+
+static void t7xx_dpmaif_ul_rdy_en(struct dpmaif_hw_info *hw_info,
+				  unsigned int q_num, bool ready)
+{
+	u32 value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_UL_CHNL_ARB0);
+
+	if (ready)
+		value |= BIT(q_num);
+	else
+		value &= ~BIT(q_num);
+
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_UL_CHNL_ARB0);
+}
+
+static void t7xx_dpmaif_ul_arb_en(struct dpmaif_hw_info *hw_info,
+				  unsigned int q_num, bool enable)
+{
+	u32 value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_UL_CHNL_ARB0);
+
+	if (enable)
+		value |= BIT(q_num + 8);
+	else
+		value &= ~BIT(q_num + 8);
+
+	iowrite32(value, hw_info->pcie_base + DPMAIF_AO_UL_CHNL_ARB0);
+}
+
+static void t7xx_dpmaif_config_ulq_hw(struct dpmaif_hw_info *hw_info)
+{
+	struct dpmaif_ul *ul_que;
+	int i;
+
+	for (i = 0; i < DPMAIF_TXQ_NUM; i++) {
+		ul_que = &hw_info->ul_que[i];
+		if (ul_que->que_started) {
+			t7xx_dpmaif_ul_update_drb_size(hw_info, i, ul_que->drb_size_cnt *
+						       DPMAIF_UL_DRB_SIZE_WORD);
+			t7xx_dpmaif_ul_update_drb_base_addr(hw_info, i, ul_que->drb_base);
+			t7xx_dpmaif_ul_rdy_en(hw_info, i, true);
+			t7xx_dpmaif_ul_arb_en(hw_info, i, true);
+		} else {
+			t7xx_dpmaif_ul_arb_en(hw_info, i, false);
+		}
+	}
+}
+
+static int t7xx_dpmaif_hw_init_done(struct dpmaif_hw_info *hw_info)
+{
+	u32 ap_cfg;
+	int ret;
+
+	ap_cfg = ioread32(hw_info->pcie_base + DPMAIF_AP_OVERWRITE_CFG);
+	ap_cfg |= DPMAIF_SRAM_SYNC;
+	iowrite32(ap_cfg, hw_info->pcie_base + DPMAIF_AP_OVERWRITE_CFG);
+
+	ret = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_AP_OVERWRITE_CFG,
+					   ap_cfg, !(ap_cfg & DPMAIF_SRAM_SYNC), 0,
+					   DPMAIF_CHECK_TIMEOUT_US);
+	if (ret)
+		return ret;
+
+	iowrite32(DPMAIF_UL_INIT_DONE, hw_info->pcie_base + DPMAIF_AO_UL_INIT_SET);
+	iowrite32(DPMAIF_DL_INIT_DONE, hw_info->pcie_base + DPMAIF_AO_DL_INIT_SET);
+	return 0;
+}
+
+static bool t7xx_dpmaif_dl_idle_check(struct dpmaif_hw_info *hw_info)
+{
+	u32 dpmaif_dl_is_busy = ioread32(hw_info->pcie_base + DPMAIF_DL_CHK_BUSY);
+
+	return !(dpmaif_dl_is_busy & DPMAIF_DL_IDLE_STS);
+}
+
+static void t7xx_dpmaif_ul_all_q_en(struct dpmaif_hw_info *hw_info, bool enable)
+{
+	u32 ul_arb_en = ioread32(hw_info->pcie_base + DPMAIF_AO_UL_CHNL_ARB0);
+
+	if (enable)
+		ul_arb_en |= DPMAIF_UL_ALL_QUE_ARB_EN;
+	else
+		ul_arb_en &= ~DPMAIF_UL_ALL_QUE_ARB_EN;
+
+	iowrite32(ul_arb_en, hw_info->pcie_base + DPMAIF_AO_UL_CHNL_ARB0);
+}
+
+static bool t7xx_dpmaif_ul_idle_check(struct dpmaif_hw_info *hw_info)
+{
+	u32 dpmaif_ul_is_busy = ioread32(hw_info->pcie_base + DPMAIF_UL_CHK_BUSY);
+
+	return !(dpmaif_ul_is_busy & DPMAIF_UL_IDLE_STS);
+}
+
+void t7xx_dpmaif_ul_update_hw_drb_cnt(struct dpmaif_hw_info *hw_info, unsigned int q_num,
+				      unsigned int drb_entry_cnt)
+{
+	u32 ul_update, value;
+	int err;
+
+	ul_update = drb_entry_cnt & DPMAIF_UL_ADD_COUNT_MASK;
+	ul_update |= DPMAIF_UL_ADD_UPDATE;
+
+	err = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_ULQ_ADD_DESC_CH_n(q_num),
+					   value, !(value & DPMAIF_UL_ADD_NOT_READY), 0,
+					   DPMAIF_CHECK_TIMEOUT_US);
+	if (err) {
+		dev_err(hw_info->dev, "UL add is not ready\n");
+		return;
+	}
+
+	iowrite32(ul_update, hw_info->pcie_base + DPMAIF_ULQ_ADD_DESC_CH_n(q_num));
+
+	err = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_ULQ_ADD_DESC_CH_n(q_num),
+					   value, !(value & DPMAIF_UL_ADD_NOT_READY), 0,
+					   DPMAIF_CHECK_TIMEOUT_US);
+	if (err)
+		dev_err(hw_info->dev, "Timeout updating UL add\n");
+}
+
+unsigned int t7xx_dpmaif_ul_get_rd_idx(struct dpmaif_hw_info *hw_info, unsigned int q_num)
+{
+	unsigned int value = ioread32(hw_info->pcie_base + DPMAIF_ULQ_STA0_n(q_num));
+
+	return FIELD_GET(DPMAIF_UL_DRB_RIDX_MSK, value) / DPMAIF_UL_DRB_SIZE_WORD;
+}
+
+int t7xx_dpmaif_dlq_add_pit_remain_cnt(struct dpmaif_hw_info *hw_info, unsigned int dlq_pit_idx,
+				       unsigned int pit_remain_cnt)
+{
+	u32 dl_update, value;
+	int ret;
+
+	dl_update = pit_remain_cnt & DPMAIF_PIT_REM_CNT_MSK;
+	dl_update |= DPMAIF_DL_ADD_UPDATE | (dlq_pit_idx << DPMAIF_ADD_DLQ_PIT_CHAN_OFS);
+
+	ret = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_DL_DLQPIT_ADD,
+					   value, !(value & DPMAIF_DL_ADD_NOT_READY), 0,
+					   DPMAIF_CHECK_TIMEOUT_US);
+	if (ret) {
+		dev_err(hw_info->dev, "Data plane modem is not ready to add dlq\n");
+		return ret;
+	}
+
+	iowrite32(dl_update, hw_info->pcie_base + DPMAIF_DL_DLQPIT_ADD);
+
+	ret = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_DL_DLQPIT_ADD,
+					   value, !(value & DPMAIF_DL_ADD_NOT_READY), 0,
+					   DPMAIF_CHECK_TIMEOUT_US);
+	if (ret) {
+		dev_err(hw_info->dev, "Data plane modem add dlq failed\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+unsigned int t7xx_dpmaif_dl_dlq_pit_get_wr_idx(struct dpmaif_hw_info *hw_info,
+					       unsigned int dlq_pit_idx)
+{
+	u32 value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_DLQ_WR_IDX +
+			 dlq_pit_idx * DLQ_PIT_IDX_SIZE);
+	return value & DPMAIF_DL_RD_WR_IDX_MSK;
+}
+
+static bool t7xx_dl_add_timedout(struct dpmaif_hw_info *hw_info)
+{
+	u32 value;
+	int ret;
+
+	ret = ioread32_poll_timeout_atomic(hw_info->pcie_base + DPMAIF_DL_BAT_ADD,
+					   value, !(value & DPMAIF_DL_ADD_NOT_READY), 0,
+					   DPMAIF_CHECK_TIMEOUT_US);
+	return ret;
+}
+
+int t7xx_dpmaif_dl_snd_hw_bat_cnt(struct dpmaif_hw_info *hw_info, unsigned int bat_entry_cnt)
+{
+	unsigned int value;
+
+	if (t7xx_dl_add_timedout(hw_info)) {
+		dev_err(hw_info->dev, "DL add BAT not ready\n");
+		return -EBUSY;
+	}
+
+	value = bat_entry_cnt & DPMAIF_DL_ADD_COUNT_MASK;
+	value |= DPMAIF_DL_ADD_UPDATE;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_DL_BAT_ADD);
+
+	if (t7xx_dl_add_timedout(hw_info)) {
+		dev_err(hw_info->dev, "DL add BAT timeout\n");
+		return -EBUSY;
+	}
+
+	return 0;
+}
+
+unsigned int t7xx_dpmaif_dl_get_bat_rd_idx(struct dpmaif_hw_info *hw_info, unsigned int q_num)
+{
+	u32 value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_BAT_RD_IDX);
+	return value & DPMAIF_DL_RD_WR_IDX_MSK;
+}
+
+unsigned int t7xx_dpmaif_dl_get_bat_wr_idx(struct dpmaif_hw_info *hw_info, unsigned int q_num)
+{
+	u32 value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_BAT_WR_IDX);
+	return value & DPMAIF_DL_RD_WR_IDX_MSK;
+}
+
+int t7xx_dpmaif_dl_snd_hw_frg_cnt(struct dpmaif_hw_info *hw_info, unsigned int frg_entry_cnt)
+{
+	unsigned int value;
+
+	if (t7xx_dl_add_timedout(hw_info)) {
+		dev_err(hw_info->dev, "Data plane modem is not ready to add frag DLQ\n");
+		return -EBUSY;
+	}
+
+	value = frg_entry_cnt & DPMAIF_DL_ADD_COUNT_MASK;
+	value |= DPMAIF_DL_FRG_ADD_UPDATE | DPMAIF_DL_ADD_UPDATE;
+	iowrite32(value, hw_info->pcie_base + DPMAIF_DL_BAT_ADD);
+
+	if (t7xx_dl_add_timedout(hw_info)) {
+		dev_err(hw_info->dev, "Data plane modem add frag DLQ failed");
+		return -EBUSY;
+	}
+
+	return 0;
+}
+
+unsigned int t7xx_dpmaif_dl_get_frg_rd_idx(struct dpmaif_hw_info *hw_info, unsigned int q_num)
+{
+	u32 value;
+
+	value = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_FRGBAT_RD_IDX);
+	return value & DPMAIF_DL_RD_WR_IDX_MSK;
+}
+
+static void t7xx_dpmaif_set_queue_property(struct dpmaif_hw_info *hw_info,
+					   struct dpmaif_hw_params *init_para)
+{
+	struct dpmaif_dl *dl_que;
+	struct dpmaif_ul *ul_que;
+	int i;
+
+	for (i = 0; i < DPMAIF_RXQ_NUM; i++) {
+		dl_que = &hw_info->dl_que[i];
+		dl_que->bat_base = init_para->pkt_bat_base_addr[i];
+		dl_que->bat_size_cnt = init_para->pkt_bat_size_cnt[i];
+		dl_que->pit_base = init_para->pit_base_addr[i];
+		dl_que->pit_size_cnt = init_para->pit_size_cnt[i];
+		dl_que->frg_base = init_para->frg_bat_base_addr[i];
+		dl_que->frg_size_cnt = init_para->frg_bat_size_cnt[i];
+		dl_que->que_started = true;
+	}
+
+	for (i = 0; i < DPMAIF_TXQ_NUM; i++) {
+		ul_que = &hw_info->ul_que[i];
+		ul_que->drb_base = init_para->drb_base_addr[i];
+		ul_que->drb_size_cnt = init_para->drb_size_cnt[i];
+		ul_que->que_started = true;
+	}
+}
+
+/**
+ * t7xx_dpmaif_hw_stop_all_txq() - Stop all TX queues.
+ * @hw_info: Pointer to struct hw_info.
+ *
+ * Disable HW UL queues. Checks busy UL queues to go to idle
+ * with an attempt count of 1000000.
+ *
+ * Return:
+ * * 0			- Success
+ * * -ETIMEDOUT		- Timed out checking busy queues
+ */
+int t7xx_dpmaif_hw_stop_all_txq(struct dpmaif_hw_info *hw_info)
+{
+	int count = 0;
+
+	t7xx_dpmaif_ul_all_q_en(hw_info, false);
+	while (t7xx_dpmaif_ul_idle_check(hw_info)) {
+		if (++count >= DPMAIF_MAX_CHECK_COUNT) {
+			dev_err(hw_info->dev, "Failed to stop TX, status: 0x%x\n",
+				ioread32(hw_info->pcie_base + DPMAIF_UL_CHK_BUSY));
+			return -ETIMEDOUT;
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * t7xx_dpmaif_hw_stop_all_rxq() - Stop all RX queues.
+ * @hw_info: Pointer to struct hw_info.
+ *
+ * Disable HW DL queue. Checks busy UL queues to go to idle
+ * with an attempt count of 1000000.
+ * Check that HW PIT write index equals read index with the same
+ * attempt count.
+ *
+ * Return:
+ * * 0			- Success.
+ * * -ETIMEDOUT		- Timed out checking busy queues.
+ */
+int t7xx_dpmaif_hw_stop_all_rxq(struct dpmaif_hw_info *hw_info)
+{
+	unsigned int wr_idx, rd_idx;
+	int count = 0;
+
+	t7xx_dpmaif_dl_all_q_en(hw_info, false);
+	while (t7xx_dpmaif_dl_idle_check(hw_info)) {
+		if (++count >= DPMAIF_MAX_CHECK_COUNT) {
+			dev_err(hw_info->dev, "Failed to stop RX, status: 0x%x\n",
+				ioread32(hw_info->pcie_base + DPMAIF_DL_CHK_BUSY));
+			return -ETIMEDOUT;
+		}
+	}
+
+	/* Check middle PIT sync done */
+	count = 0;
+	do {
+		wr_idx = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_PIT_WR_IDX);
+		wr_idx &= DPMAIF_DL_RD_WR_IDX_MSK;
+		rd_idx = ioread32(hw_info->pcie_base + DPMAIF_AO_DL_PIT_RD_IDX);
+		rd_idx &= DPMAIF_DL_RD_WR_IDX_MSK;
+
+		if (wr_idx == rd_idx)
+			return 0;
+	} while (++count < DPMAIF_MAX_CHECK_COUNT);
+
+	dev_err(hw_info->dev, "Check middle PIT sync fail\n");
+	return -ETIMEDOUT;
+}
+
+void t7xx_dpmaif_start_hw(struct dpmaif_hw_info *hw_info)
+{
+	t7xx_dpmaif_ul_all_q_en(hw_info, true);
+	t7xx_dpmaif_dl_all_q_en(hw_info, true);
+}
+
+/**
+ * t7xx_dpmaif_hw_init() - Initialize HW data path API.
+ * @hw_info: Pointer to struct hw_info.
+ * @init_param: Pointer to struct dpmaif_hw_params.
+ *
+ * Configures port mode, clock config, HW interrupt initialization, and HW queue.
+ *
+ * Return:
+ * * 0		- Success.
+ * * -ERROR	- Error code from failure sub-initializations.
+ */
+int t7xx_dpmaif_hw_init(struct dpmaif_hw_info *hw_info, struct dpmaif_hw_params *init_param)
+{
+	int ret;
+
+	ret = t7xx_dpmaif_hw_config(hw_info);
+	if (ret) {
+		dev_err(hw_info->dev, "DPMAIF HW config failed\n");
+		return ret;
+	}
+
+	ret = t7xx_dpmaif_init_intr(hw_info);
+	if (ret) {
+		dev_err(hw_info->dev, "DPMAIF HW interrupts init failed\n");
+		return ret;
+	}
+
+	t7xx_dpmaif_set_queue_property(hw_info, init_param);
+	t7xx_dpmaif_pcie_dpmaif_sign(hw_info);
+	t7xx_dpmaif_dl_performance(hw_info);
+
+	ret = t7xx_dpmaif_config_dlq_hw(hw_info);
+	if (ret) {
+		dev_err(hw_info->dev, "DPMAIF HW dlq config failed\n");
+		return ret;
+	}
+
+	t7xx_dpmaif_config_ulq_hw(hw_info);
+
+	ret = t7xx_dpmaif_hw_init_done(hw_info);
+	if (ret)
+		dev_err(hw_info->dev, "DPMAIF HW queue init failed\n");
+
+	return ret;
+}
+
+bool t7xx_dpmaif_ul_clr_done(struct dpmaif_hw_info *hw_info, unsigned int qno)
+{
+	u32 intr_status;
+
+	intr_status = ioread32(hw_info->pcie_base + DPMAIF_AP_L2TISAR0);
+	intr_status &= BIT(DP_UL_INT_DONE_OFFSET + qno);
+	if (intr_status) {
+		iowrite32(intr_status, hw_info->pcie_base + DPMAIF_AP_L2TISAR0);
+		return true;
+	}
+
+	return false;
+}
drivers/net/wwan/t7xx/t7xx_dpmaif.h new file +179 -0

179 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_reg.h +213 -0

217 lines not on this page · raw message

Signature · 1 line
2.25.1
author reviewer
#13 lore raw

PATCH v7 9/14 net: wwan: t7xx: Add data path interface

In reply to Ricardo Martinez

From: Haijun Liu <[email protected]>

Data Path Modem AP Interface (DPMAIF) HIF layer provides methods for initialization, ISR, control and event handling of TX/RX flows.

DPMAIF TX
Exposes the 'dmpaif_tx_send_skb' function which can be used by the network device to transmit packets.
The uplink data management uses a Descriptor Ring Buffer (DRB). First DRB entry is a message type that will be followed by 1 or more normal DRB entries. Message type DRB will hold the skb information and each normal DRB entry holds a pointer to the skb payload.

DPMAIF RX
The downlink buffer management uses Buffer Address Table (BAT) and Packet Information Table (PIT) rings.
The BAT ring holds the address of skb data buffer for the HW to use, while the PIT contains metadata about a whole network packet including a reference to the BAT entry holding the data buffer address. The driver reads the PIT and BAT entries written by the modem, when reaching a threshold, the driver will reload the PIT and BAT rings.

---
 drivers/net/wwan/t7xx/Makefile             |    4 +
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif.c    |  484 ++++++++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif.h    |  205 ++++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c | 1220 ++++++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.h |  116 ++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c |  651 +++++++++++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.h |   78 ++
 7 files changed, 2758 insertions(+)
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.h
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.h
drivers/net/wwan/t7xx/Makefile +4 -0
index 9eec2e2472fb..04a9ba50dc14 100644
@@ -13,3 +13,7 @@ mtk_t7xx-y:=	t7xx_pci.o \
 		t7xx_port_proxy.o  \
 		t7xx_port_ctrl_msg.o \
 		t7xx_port_wwan.o \
+		t7xx_hif_dpmaif.o  \
+		t7xx_hif_dpmaif_tx.o \
+		t7xx_hif_dpmaif_rx.o  \
+		t7xx_dpmaif.o \
drivers/net/wwan/t7xx/t7xx_hif_dpmaif.c new file +484 -0

484 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_hif_dpmaif.h new file +205 -0

205 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c new file +1220 -0

1,220 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.h new file +116 -0

116 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c new file +651 -0

651 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.h new file +78 -0

78 lines not on this page · raw message

Signature · 1 line
2.25.1
Reviewed-by on v7 9/14
#14 lore raw

Re: [PATCH net-next v7 09/14] net: wwan: t7xx: Add data path interface

Quoted 22 lines · Ricardo Martinez

From: Haijun Liu <[email protected]>

Data Path Modem AP Interface (DPMAIF) HIF layer provides methods for initialization, ISR, control and event handling of TX/RX flows.

DPMAIF TX
Exposes the 'dmpaif_tx_send_skb' function which can be used by the network device to transmit packets.
The uplink data management uses a Descriptor Ring Buffer (DRB). First DRB entry is a message type that will be followed by 1 or more normal DRB entries. Message type DRB will hold the skb information and each normal DRB entry holds a pointer to the skb payload.

DPMAIF RX
The downlink buffer management uses Buffer Address Table (BAT) and Packet Information Table (PIT) rings.
The BAT ring holds the address of skb data buffer for the HW to use, while the PIT contains metadata about a whole network packet including a reference to the BAT entry holding the data buffer address. The driver reads the PIT and BAT entries written by the modem, when reaching a threshold, the driver will reload the PIT and BAT rings.

Quoted 6 lines
Signature · 1 line
 i.
author reviewer
#15 lore raw

PATCH v7 10/14 net: wwan: t7xx: Add WWAN network interface

In reply to Ricardo Martinez

From: Haijun Liu <[email protected]>

Creates the Cross Core Modem Network Interface (CCMNI) which implements the wwan_ops for registration with the WWAN framework, CCMNI also implements the net_device_ops functions used by the network device. Network device operations include open, close, start transmission, TX timeout and change MTU.

---
 drivers/net/wwan/t7xx/Makefile         |   1 +
 drivers/net/wwan/t7xx/t7xx_modem_ops.c |  11 +-
 drivers/net/wwan/t7xx/t7xx_netdev.c    | 423 +++++++++++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_netdev.h    |  55 ++++
 4 files changed, 489 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/wwan/t7xx/t7xx_netdev.c
 create mode 100644 drivers/net/wwan/t7xx/t7xx_netdev.h
drivers/net/wwan/t7xx/Makefile +1 -0
index 04a9ba50dc14..dc6a7d682c15 100644
@@ -17,3 +17,4 @@ mtk_t7xx-y:=	t7xx_pci.o \
 		t7xx_hif_dpmaif_tx.o \
 		t7xx_hif_dpmaif_rx.o  \
 		t7xx_dpmaif.o \
+		t7xx_netdev.o
drivers/net/wwan/t7xx/t7xx_modem_ops.c +10 -1
index a1b60173d89d..1056ad9bf34f 100644
@@ -35,6 +35,7 @@
 #include "t7xx_hif_cldma.h"
 #include "t7xx_mhccif.h"
 #include "t7xx_modem_ops.h"
+#include "t7xx_netdev.h"
 #include "t7xx_pci.h"
 #include "t7xx_pcie_mac.h"
 #include "t7xx_port.h"
@@ -670,10 +671,14 @@ int t7xx_md_init(struct t7xx_pci_dev *t7xx_dev)
 	if (ret)
 		goto err_destroy_hswq;
 
-	ret = t7xx_cldma_init(md->md_ctrl[CLDMA_ID_MD]);
+	ret = t7xx_ccmni_init(t7xx_dev);
 	if (ret)
 		goto err_uninit_fsm;
 
+	ret = t7xx_cldma_init(md->md_ctrl[CLDMA_ID_MD]);
+	if (ret)
+		goto err_uninit_ccmni;
+
 	ret = t7xx_port_proxy_init(md);
 	if (ret)
 		goto err_uninit_md_cldma;
@@ -692,6 +697,9 @@ int t7xx_md_init(struct t7xx_pci_dev *t7xx_dev)
 err_uninit_md_cldma:
 	t7xx_cldma_exit(md->md_ctrl[CLDMA_ID_MD]);
 
+err_uninit_ccmni:
+	t7xx_ccmni_exit(t7xx_dev);
+
 err_uninit_fsm:
 	t7xx_fsm_uninit(md);
 
@@ -713,6 +721,7 @@ void t7xx_md_exit(struct t7xx_pci_dev *t7xx_dev)
 	t7xx_fsm_append_cmd(md->fsm_ctl, FSM_CMD_PRE_STOP, FSM_CMD_FLAG_WAIT_FOR_COMPLETION);
 	t7xx_port_proxy_uninit(md->port_prox);
 	t7xx_cldma_exit(md->md_ctrl[CLDMA_ID_MD]);
+	t7xx_ccmni_exit(t7xx_dev);
 	t7xx_fsm_uninit(md);
 	destroy_workqueue(md->handshake_wq);
 }
drivers/net/wwan/t7xx/t7xx_netdev.c new file +423 -0

423 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_netdev.h new file +55 -0

55 lines not on this page · raw message

Signature · 1 line
2.25.1
author reviewer
#16 lore raw

PATCH v7 11/14 net: wwan: t7xx: Introduce power management

In reply to Ricardo Martinez

From: Haijun Liu <[email protected]>

Implements suspend, resumes, freeze, thaw, poweroff, and restore dev_pm_ops callbacks.

From the host point of view, the t7xx driver is one entity. But, the device has several modules that need to be addressed in different ways during power management (PM) flows.
The driver uses the term 'PM entities' to refer to the 2 DPMA and 2 CLDMA HW blocks that need to be managed during PM flows. When a dev_pm_ops function is called, the PM entities list is iterated and the matching function is called for each entry in the list.

---
 drivers/net/wwan/t7xx/t7xx_hif_cldma.c     | 123 +++++-
 drivers/net/wwan/t7xx/t7xx_hif_cldma.h     |   1 +
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif.c    |  90 +++++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif.h    |   1 +
 drivers/net/wwan/t7xx/t7xx_mhccif.c        |  17 +
 drivers/net/wwan/t7xx/t7xx_pci.c           | 421 +++++++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_pci.h           |  46 +++
 drivers/net/wwan/t7xx/t7xx_state_monitor.c |   2 +
 8 files changed, 700 insertions(+), 1 deletion(-)
drivers/net/wwan/t7xx/t7xx_hif_cldma.c +122 -1
index 7a8c09f7435a..cbe5ea4495e0 100644
@@ -1076,6 +1076,120 @@ int t7xx_cldma_alloc(enum cldma_id hif_id, struct t7xx_pci_dev *t7xx_dev)
 	return 0;
 }
 
+static void t7xx_cldma_resume_early(struct t7xx_pci_dev *t7xx_dev, void *entity_param)
+{
+	struct cldma_ctrl *md_ctrl = entity_param;
+	struct t7xx_cldma_hw *hw_info;
+	unsigned long flags;
+	int qno_t;
+
+	hw_info = &md_ctrl->hw_info;
+
+	spin_lock_irqsave(&md_ctrl->cldma_lock, flags);
+	t7xx_cldma_hw_restore(hw_info);
+	for (qno_t = 0; qno_t < CLDMA_TXQ_NUM; qno_t++) {
+		t7xx_cldma_hw_set_start_addr(hw_info, qno_t, md_ctrl->txq[qno_t].tx_next->gpd_addr,
+					     MTK_TX);
+		t7xx_cldma_hw_set_start_addr(hw_info, qno_t, md_ctrl->rxq[qno_t].tr_done->gpd_addr,
+					     MTK_RX);
+	}
+	t7xx_cldma_enable_irq(md_ctrl);
+	t7xx_cldma_hw_start_queue(hw_info, CLDMA_ALL_Q, MTK_RX);
+	md_ctrl->rxq_active |= TXRX_STATUS_BITMASK;
+	t7xx_cldma_hw_irq_en_eq(hw_info, CLDMA_ALL_Q, MTK_RX);
+	t7xx_cldma_hw_irq_en_txrx(hw_info, CLDMA_ALL_Q, MTK_RX);
+	spin_unlock_irqrestore(&md_ctrl->cldma_lock, flags);
+}
+
+static int t7xx_cldma_resume(struct t7xx_pci_dev *t7xx_dev, void *entity_param)
+{
+	struct cldma_ctrl *md_ctrl = entity_param;
+	unsigned long flags;
+
+	spin_lock_irqsave(&md_ctrl->cldma_lock, flags);
+	md_ctrl->txq_active |= TXRX_STATUS_BITMASK;
+	t7xx_cldma_hw_irq_en_txrx(&md_ctrl->hw_info, CLDMA_ALL_Q, MTK_TX);
+	t7xx_cldma_hw_irq_en_eq(&md_ctrl->hw_info, CLDMA_ALL_Q, MTK_TX);
+	spin_unlock_irqrestore(&md_ctrl->cldma_lock, flags);
+
+	if (md_ctrl->hif_id == CLDMA_ID_MD)
+		t7xx_mhccif_mask_clr(t7xx_dev, D2H_SW_INT_MASK);
+
+	return 0;
+}
+
+static void t7xx_cldma_suspend_late(struct t7xx_pci_dev *t7xx_dev, void *entity_param)
+{
+	struct cldma_ctrl *md_ctrl = entity_param;
+	struct t7xx_cldma_hw *hw_info;
+	unsigned long flags;
+
+	hw_info = &md_ctrl->hw_info;
+
+	spin_lock_irqsave(&md_ctrl->cldma_lock, flags);
+	t7xx_cldma_hw_irq_dis_eq(hw_info, CLDMA_ALL_Q, MTK_RX);
+	t7xx_cldma_hw_irq_dis_txrx(hw_info, CLDMA_ALL_Q, MTK_RX);
+	md_ctrl->rxq_active &= ~TXRX_STATUS_BITMASK;
+	t7xx_cldma_hw_stop_all_qs(hw_info, MTK_RX);
+	t7xx_cldma_clear_ip_busy(hw_info);
+	t7xx_cldma_disable_irq(md_ctrl);
+	spin_unlock_irqrestore(&md_ctrl->cldma_lock, flags);
+}
+
+static int t7xx_cldma_suspend(struct t7xx_pci_dev *t7xx_dev, void *entity_param)
+{
+	struct cldma_ctrl *md_ctrl = entity_param;
+	struct t7xx_cldma_hw *hw_info;
+	unsigned long flags;
+
+	if (md_ctrl->hif_id == CLDMA_ID_MD)
+		t7xx_mhccif_mask_set(t7xx_dev, D2H_SW_INT_MASK);
+
+	hw_info = &md_ctrl->hw_info;
+
+	spin_lock_irqsave(&md_ctrl->cldma_lock, flags);
+	t7xx_cldma_hw_irq_dis_eq(hw_info, CLDMA_ALL_Q, MTK_TX);
+	t7xx_cldma_hw_irq_dis_txrx(hw_info, CLDMA_ALL_Q, MTK_TX);
+	md_ctrl->txq_active &= ~TXRX_STATUS_BITMASK;
+	t7xx_cldma_hw_stop_all_qs(hw_info, MTK_TX);
+	md_ctrl->txq_started = 0;
+	spin_unlock_irqrestore(&md_ctrl->cldma_lock, flags);
+
+	return 0;
+}
+
+static int t7xx_cldma_pm_init(struct cldma_ctrl *md_ctrl)
+{
+	md_ctrl->pm_entity = kzalloc(sizeof(*md_ctrl->pm_entity), GFP_KERNEL);
+	if (!md_ctrl->pm_entity)
+		return -ENOMEM;
+
+	md_ctrl->pm_entity->entity_param = md_ctrl;
+
+	if (md_ctrl->hif_id == CLDMA_ID_MD)
+		md_ctrl->pm_entity->id = PM_ENTITY_ID_CTRL1;
+	else
+		md_ctrl->pm_entity->id = PM_ENTITY_ID_CTRL2;
+
+	md_ctrl->pm_entity->suspend = t7xx_cldma_suspend;
+	md_ctrl->pm_entity->suspend_late = t7xx_cldma_suspend_late;
+	md_ctrl->pm_entity->resume = t7xx_cldma_resume;
+	md_ctrl->pm_entity->resume_early = t7xx_cldma_resume_early;
+
+	return t7xx_pci_pm_entity_register(md_ctrl->t7xx_dev, md_ctrl->pm_entity);
+}
+
+static int t7xx_cldma_pm_uninit(struct cldma_ctrl *md_ctrl)
+{
+	if (!md_ctrl->pm_entity)
+		return -EINVAL;
+
+	t7xx_pci_pm_entity_unregister(md_ctrl->t7xx_dev, md_ctrl->pm_entity);
+	kfree(md_ctrl->pm_entity);
+	md_ctrl->pm_entity = NULL;
+	return 0;
+}
+
 void t7xx_cldma_hif_hw_init(struct cldma_ctrl *md_ctrl)
 {
 	struct t7xx_cldma_hw *hw_info = &md_ctrl->hw_info;
@@ -1126,6 +1240,7 @@ static void t7xx_cldma_destroy_wqs(struct cldma_ctrl *md_ctrl)
  * t7xx_cldma_init() - Initialize CLDMA.
  * @md_ctrl: CLDMA context structure.
  *
+ * Allocate and initialize device power management entity.
  * Initialize HIF TX/RX queue structure.
  * Register CLDMA callback ISR with PCIe driver.
  *
@@ -1136,12 +1251,16 @@ static void t7xx_cldma_destroy_wqs(struct cldma_ctrl *md_ctrl)
 int t7xx_cldma_init(struct cldma_ctrl *md_ctrl)
 {
 	struct t7xx_cldma_hw *hw_info = &md_ctrl->hw_info;
-	int i;
+	int ret, i;
 
 	md_ctrl->txq_active = 0;
 	md_ctrl->rxq_active = 0;
 	md_ctrl->is_late_init = false;
 
+	ret = t7xx_cldma_pm_init(md_ctrl);
+	if (ret)
+		return ret;
+
 	spin_lock_init(&md_ctrl->cldma_lock);
 
 	for (i = 0; i < CLDMA_TXQ_NUM; i++) {
@@ -1176,6 +1295,7 @@ int t7xx_cldma_init(struct cldma_ctrl *md_ctrl)
 
 err_workqueue:
 	t7xx_cldma_destroy_wqs(md_ctrl);
+	t7xx_cldma_pm_uninit(md_ctrl);
 	return -ENOMEM;
 }
 
@@ -1190,4 +1310,5 @@ void t7xx_cldma_exit(struct cldma_ctrl *md_ctrl)
 	t7xx_cldma_stop(md_ctrl);
 	t7xx_cldma_late_release(md_ctrl);
 	t7xx_cldma_destroy_wqs(md_ctrl);
+	t7xx_cldma_pm_uninit(md_ctrl);
 }
drivers/net/wwan/t7xx/t7xx_hif_cldma.h +1 -0
index deb239e4f803..47a35e552da7 100644
@@ -98,6 +98,7 @@ struct cldma_ctrl {
 	struct dma_pool *gpd_dmapool;
 	struct cldma_ring tx_ring[CLDMA_TXQ_NUM];
 	struct cldma_ring rx_ring[CLDMA_RXQ_NUM];
+	struct md_pm_entity *pm_entity;
 	struct t7xx_cldma_hw hw_info;
 	bool is_late_init;
 	int (*recv_skb)(struct cldma_queue *queue, struct sk_buff *skb);
drivers/net/wwan/t7xx/t7xx_hif_dpmaif.c +90 -0

113 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_hif_dpmaif.h +1 -0
index 1d5de7ceeeca..1225ca0ed51e 100644
@@ -174,6 +174,7 @@ struct dpmaif_callbacks {
 struct dpmaif_ctrl {
 	struct device			*dev;
 	struct t7xx_pci_dev		*t7xx_dev;
+	struct md_pm_entity		dpmaif_pm_entity;
 	enum dpmaif_state		state;
 	bool				dpmaif_sw_init_done;
 	struct dpmaif_hw_info		hw_info;
drivers/net/wwan/t7xx/t7xx_mhccif.c +17 -0
index 5826e2d0b5a8..4bb452f5ccff 100644
@@ -24,6 +24,11 @@
 #include "t7xx_pcie_mac.h"
 #include "t7xx_reg.h"
 
+#define D2H_INT_SR_ACK		(D2H_INT_SUSPEND_ACK |		\
+				 D2H_INT_RESUME_ACK |		\
+				 D2H_INT_SUSPEND_ACK_AP |	\
+				 D2H_INT_RESUME_ACK_AP)
+
 static void t7xx_mhccif_clear_interrupts(struct t7xx_pci_dev *t7xx_dev, u32 mask)
 {
 	void __iomem *mhccif_pbase = t7xx_dev->base_addr.mhccif_rc_base;
@@ -53,6 +58,18 @@ static irqreturn_t t7xx_mhccif_isr_thread(int irq, void *data)
 	}
 
 	t7xx_mhccif_clear_interrupts(t7xx_dev, int_status);
+
+	if (int_status & D2H_INT_SR_ACK)
+		complete(&t7xx_dev->pm_sr_ack);
+
+	iowrite32(T7XX_L1_BIT(1), IREG_BASE(t7xx_dev) + ENABLE_ASPM_LOWPWR);
+
+	int_status = t7xx_mhccif_read_sw_int_sts(t7xx_dev);
+	if (!int_status) {
+		val = T7XX_L1_1_BIT(1) | T7XX_L1_2_BIT(1);
+		iowrite32(val, IREG_BASE(t7xx_dev) + ENABLE_ASPM_LOWPWR);
+	}
+
 	t7xx_pcie_mac_set_int(t7xx_dev, MHCCIF_INT);
 	return IRQ_HANDLED;
 }
drivers/net/wwan/t7xx/t7xx_pci.c +421 -0

456 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_pci.h +46 -0

65 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_state_monitor.c +2 -0
index d841f5051af8..0bcca08ff2bd 100644
@@ -188,6 +188,7 @@ static void fsm_routine_exception(struct t7xx_fsm_ctl *ctl, struct t7xx_fsm_comm
 	case EXCEPTION_EVENT:
 		dev_err(dev, "Exception event\n");
 		t7xx_fsm_broadcast_state(ctl, MD_STATE_EXCEPTION);
+		t7xx_pci_pm_exp_detected(ctl->md->t7xx_dev);
 		t7xx_md_exception_handshake(ctl->md);
 
 		fsm_wait_for_event(ctl, FSM_EVENT_MD_EX_REC_OK, FSM_EVENT_MD_EX,
@@ -300,6 +301,7 @@ static int fsm_routine_starting(struct t7xx_fsm_ctl *ctl)
 		return -ETIMEDOUT;
 	}
 
+	t7xx_pci_pm_init_late(md->t7xx_dev);
 	fsm_routine_ready(ctl);
 	return 0;
 }
Signature · 1 line
2.25.1
author reviewer
#17 lore raw

PATCH v7 12/14 net: wwan: t7xx: Runtime PM

In reply to Ricardo Martinez

From: Haijun Liu <[email protected]>

Enables runtime power management callbacks including runtime_suspend and runtime_resume. Autosuspend is used to prevent overhead by frequent wake-ups.

---
 drivers/net/wwan/t7xx/t7xx_hif_cldma.c     | 14 ++++++++++++++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c | 17 +++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c | 15 +++++++++++++++
 drivers/net/wwan/t7xx/t7xx_pci.c           | 22 ++++++++++++++++++++++
 4 files changed, 68 insertions(+)
drivers/net/wwan/t7xx/t7xx_hif_cldma.c +14 -0
index cbe5ea4495e0..90306eb9858b 100644
@@ -33,6 +33,7 @@
 #include <linux/list.h>
 #include <linux/netdevice.h>
 #include <linux/pci.h>
+#include <linux/pm_runtime.h>
 #include <linux/sched.h>
 #include <linux/skbuff.h>
 #include <linux/slab.h>
@@ -251,6 +252,8 @@ static void t7xx_cldma_rx_done(struct work_struct *work)
 	t7xx_cldma_clear_ip_busy(&md_ctrl->hw_info);
 	t7xx_cldma_hw_irq_en_txrx(&md_ctrl->hw_info, queue->index, MTK_RX);
 	t7xx_cldma_hw_irq_en_eq(&md_ctrl->hw_info, queue->index, MTK_RX);
+	pm_runtime_mark_last_busy(md_ctrl->dev);
+	pm_runtime_put_autosuspend(md_ctrl->dev);
 }
 
 static int t7xx_cldma_gpd_tx_collect(struct cldma_queue *queue)
@@ -360,6 +363,9 @@ static void t7xx_cldma_tx_done(struct work_struct *work)
 		t7xx_cldma_hw_irq_en_txrx(hw_info, queue->index, MTK_TX);
 	}
 	spin_unlock_irqrestore(&md_ctrl->cldma_lock, flags);
+
+	pm_runtime_mark_last_busy(md_ctrl->dev);
+	pm_runtime_put_autosuspend(md_ctrl->dev);
 }
 
 static void t7xx_cldma_ring_free(struct cldma_ctrl *md_ctrl,
@@ -568,6 +574,7 @@ static void t7xx_cldma_irq_work_cb(struct cldma_ctrl *md_ctrl)
 		if (l2_tx_int & (TXRX_STATUS_BITMASK | EMPTY_STATUS_BITMASK)) {
 			for_each_set_bit(i, &l2_tx_int, L2_INT_BIT_COUNT) {
 				if (i < CLDMA_TXQ_NUM) {
+					pm_runtime_get(md_ctrl->dev);
 					t7xx_cldma_hw_irq_dis_eq(hw_info, i, MTK_TX);
 					t7xx_cldma_hw_irq_dis_txrx(hw_info, i, MTK_TX);
 					queue_work(md_ctrl->txq[i].worker,
@@ -592,6 +599,7 @@ static void t7xx_cldma_irq_work_cb(struct cldma_ctrl *md_ctrl)
 		if (l2_rx_int & (TXRX_STATUS_BITMASK | EMPTY_STATUS_BITMASK)) {
 			l2_rx_int |= l2_rx_int >> CLDMA_RXQ_NUM;
 			for_each_set_bit(i, &l2_rx_int, CLDMA_RXQ_NUM) {
+				pm_runtime_get(md_ctrl->dev);
 				t7xx_cldma_hw_irq_dis_eq(hw_info, i, MTK_RX);
 				t7xx_cldma_hw_irq_dis_txrx(hw_info, i, MTK_RX);
 				queue_work(md_ctrl->rxq[i].worker, &md_ctrl->rxq[i].cldma_work);
@@ -922,6 +930,10 @@ int t7xx_cldma_send_skb(struct cldma_ctrl *md_ctrl, int qno, struct sk_buff *skb
 	if (qno >= CLDMA_TXQ_NUM)
 		return -EINVAL;
 
+	ret = pm_runtime_resume_and_get(md_ctrl->dev);
+	if (ret < 0 && ret != -EACCES)
+		return ret;
+
 	queue = &md_ctrl->txq[qno];
 
 	spin_lock_irqsave(&md_ctrl->cldma_lock, flags);
@@ -965,6 +977,8 @@ int t7xx_cldma_send_skb(struct cldma_ctrl *md_ctrl, int qno, struct sk_buff *skb
 	} while (!ret);
 
 allow_sleep:
+	pm_runtime_mark_last_busy(md_ctrl->dev);
+	pm_runtime_put_autosuspend(md_ctrl->dev);
 	return ret;
 }
 
drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c +17 -0
index d110f7edf56b..5f25555eb4a4 100644
@@ -32,6 +32,7 @@
 #include <linux/minmax.h>
 #include <linux/mm.h>
 #include <linux/netdevice.h>
+#include <linux/pm_runtime.h>
 #include <linux/sched.h>
 #include <linux/skbuff.h>
 #include <linux/slab.h>
@@ -910,6 +911,7 @@ static void t7xx_dpmaif_rxq_work(struct work_struct *work)
 {
 	struct dpmaif_rx_queue *rxq = container_of(work, struct dpmaif_rx_queue, dpmaif_rxq_work);
 	struct dpmaif_ctrl *dpmaif_ctrl = rxq->dpmaif_ctrl;
+	int ret;
 
 	atomic_set(&rxq->rx_processing, 1);
 	/* Ensure rx_processing is changed to 1 before actually begin RX flow */
@@ -921,7 +923,14 @@ static void t7xx_dpmaif_rxq_work(struct work_struct *work)
 		return;
 	}
 
+	ret = pm_runtime_resume_and_get(dpmaif_ctrl->dev);
+	if (ret < 0 && ret != -EACCES)
+		return;
+
 	t7xx_dpmaif_do_rx(dpmaif_ctrl, rxq);
+
+	pm_runtime_mark_last_busy(dpmaif_ctrl->dev);
+	pm_runtime_put_autosuspend(dpmaif_ctrl->dev);
 	atomic_set(&rxq->rx_processing, 0);
 }
 
@@ -1123,11 +1132,19 @@ static void t7xx_dpmaif_bat_release_work(struct work_struct *work)
 {
 	struct dpmaif_ctrl *dpmaif_ctrl = container_of(work, struct dpmaif_ctrl, bat_release_work);
 	struct dpmaif_rx_queue *rxq;
+	int ret;
+
+	ret = pm_runtime_resume_and_get(dpmaif_ctrl->dev);
+	if (ret < 0 && ret != -EACCES)
+		return;
 
 	/* ALL RXQ use one BAT table, so choose DPF_RX_QNO_DFT */
 	rxq = &dpmaif_ctrl->rxq[DPF_RX_QNO_DFT];
 	t7xx_dpmaif_bat_release_and_add(rxq);
 	t7xx_dpmaif_frag_bat_release_and_add(rxq);
+
+	pm_runtime_mark_last_busy(dpmaif_ctrl->dev);
+	pm_runtime_put_autosuspend(dpmaif_ctrl->dev);
 }
 
 int t7xx_dpmaif_bat_rel_wq_alloc(struct dpmaif_ctrl *dpmaif_ctrl)
drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c +15 -0
index e8435a871842..a5ac9dfaecfd 100644
@@ -28,6 +28,7 @@
 #include <linux/list.h>
 #include <linux/minmax.h>
 #include <linux/netdevice.h>
+#include <linux/pm_runtime.h>
 #include <linux/sched.h>
 #include <linux/spinlock.h>
 #include <linux/skbuff.h>
@@ -161,6 +162,10 @@ static void t7xx_dpmaif_tx_done(struct work_struct *work)
 	struct dpmaif_hw_info *hw_info;
 	int ret;
 
+	ret = pm_runtime_resume_and_get(dpmaif_ctrl->dev);
+	if (ret < 0 && ret != -EACCES)
+		return;
+
 	hw_info = &dpmaif_ctrl->hw_info;
 	ret = t7xx_dpmaif_tx_release(dpmaif_ctrl, txq->index, txq->drb_size_cnt);
 	if (ret == -EAGAIN ||
@@ -174,6 +179,9 @@ static void t7xx_dpmaif_tx_done(struct work_struct *work)
 		t7xx_dpmaif_clr_ip_busy_sts(hw_info);
 		t7xx_dpmaif_unmask_ulq_intr(hw_info, txq->index);
 	}
+
+	pm_runtime_mark_last_busy(dpmaif_ctrl->dev);
+	pm_runtime_put_autosuspend(dpmaif_ctrl->dev);
 }
 
 static void t7xx_setup_msg_drb(struct dpmaif_ctrl *dpmaif_ctrl, unsigned int q_num,
@@ -423,6 +431,7 @@ static void t7xx_do_tx_hw_push(struct dpmaif_ctrl *dpmaif_ctrl)
 static int t7xx_dpmaif_tx_hw_push_thread(void *arg)
 {
 	struct dpmaif_ctrl *dpmaif_ctrl = arg;
+	int ret;
 
 	while (!kthread_should_stop()) {
 		if (t7xx_tx_lists_are_all_empty(dpmaif_ctrl) ||
@@ -437,7 +446,13 @@ static int t7xx_dpmaif_tx_hw_push_thread(void *arg)
 				break;
 		}
 
+		ret = pm_runtime_resume_and_get(dpmaif_ctrl->dev);
+		if (ret < 0 && ret != -EACCES)
+			return ret;
+
 		t7xx_do_tx_hw_push(dpmaif_ctrl);
+		pm_runtime_mark_last_busy(dpmaif_ctrl->dev);
+		pm_runtime_put_autosuspend(dpmaif_ctrl->dev);
 	}
 
 	return 0;
drivers/net/wwan/t7xx/t7xx_pci.c +22 -0
index 564147664af0..400c11f7b31e 100644
@@ -31,6 +31,7 @@
 #include <linux/mutex.h>
 #include <linux/pci.h>
 #include <linux/pm.h>
+#include <linux/pm_runtime.h>
 #include <linux/pm_wakeup.h>
 
 #include "t7xx_mhccif.h"
@@ -44,6 +45,7 @@
 #define T7XX_PCI_EREG_BASE		2
 
 #define PM_ACK_TIMEOUT_MS		1500
+#define PM_AUTOSUSPEND_MS		20000
 #define PM_RESOURCE_POLL_TIMEOUT_US	10000
 #define PM_RESOURCE_POLL_STEP_US	100
 
@@ -82,6 +84,8 @@ static int t7xx_pci_pm_init(struct t7xx_pci_dev *t7xx_dev)
 				DPM_FLAG_NO_DIRECT_COMPLETE);
 
 	iowrite32(T7XX_L1_BIT(0), IREG_BASE(t7xx_dev) + DISABLE_ASPM_LOWPWR);
+	pm_runtime_set_autosuspend_delay(&pdev->dev, PM_AUTOSUSPEND_MS);
+	pm_runtime_use_autosuspend(&pdev->dev);
 
 	return t7xx_wait_pm_config(t7xx_dev);
 }
@@ -96,6 +100,8 @@ void t7xx_pci_pm_init_late(struct t7xx_pci_dev *t7xx_dev)
 			     D2H_INT_RESUME_ACK_AP);
 	iowrite32(T7XX_L1_BIT(0), IREG_BASE(t7xx_dev) + ENABLE_ASPM_LOWPWR);
 	atomic_set(&t7xx_dev->md_pm_state, MTK_PM_RESUMED);
+
+	pm_runtime_put_noidle(&t7xx_dev->pdev->dev);
 }
 
 static int t7xx_pci_pm_reinit(struct t7xx_pci_dev *t7xx_dev)
@@ -104,6 +110,9 @@ static int t7xx_pci_pm_reinit(struct t7xx_pci_dev *t7xx_dev)
 	 * so just roll back PM setting to the init setting.
 	 */
 	atomic_set(&t7xx_dev->md_pm_state, MTK_PM_INIT);
+
+	pm_runtime_get_noresume(&t7xx_dev->pdev->dev);
+
 	iowrite32(T7XX_L1_BIT(0), IREG_BASE(t7xx_dev) + DISABLE_ASPM_LOWPWR);
 	return t7xx_wait_pm_config(t7xx_dev);
 }
@@ -403,6 +412,7 @@ static int __t7xx_pci_pm_resume(struct pci_dev *pdev, bool state_check)
 	t7xx_dev->rgu_pci_irq_en = true;
 	t7xx_pcie_mac_set_int(t7xx_dev, SAP_RGU_INT);
 	iowrite32(T7XX_L1_BIT(0), IREG_BASE(t7xx_dev) + ENABLE_ASPM_LOWPWR);
+	pm_runtime_mark_last_busy(&pdev->dev);
 	atomic_set(&t7xx_dev->md_pm_state, MTK_PM_RESUMED);
 
 	return ret;
@@ -439,6 +449,16 @@ static int t7xx_pci_pm_thaw(struct device *dev)
 	return __t7xx_pci_pm_resume(to_pci_dev(dev), false);
 }
 
+static int t7xx_pci_pm_runtime_suspend(struct device *dev)
+{
+	return __t7xx_pci_pm_suspend(to_pci_dev(dev));
+}
+
+static int t7xx_pci_pm_runtime_resume(struct device *dev)
+{
+	return __t7xx_pci_pm_resume(to_pci_dev(dev), true);
+}
+
 static const struct dev_pm_ops t7xx_pci_pm_ops = {
 	.suspend = t7xx_pci_pm_suspend,
 	.resume = t7xx_pci_pm_resume,
@@ -448,6 +468,8 @@ static const struct dev_pm_ops t7xx_pci_pm_ops = {
 	.poweroff = t7xx_pci_pm_suspend,
 	.restore = t7xx_pci_pm_resume,
 	.restore_noirq = t7xx_pci_pm_resume_noirq,
+	.runtime_suspend = t7xx_pci_pm_runtime_suspend,
+	.runtime_resume = t7xx_pci_pm_runtime_resume
 };
 
 static int t7xx_request_irq(struct pci_dev *pdev)
Signature · 1 line
2.25.1
author reviewer
#18 lore raw

PATCH v7 13/14 net: wwan: t7xx: Device deep sleep lock/unlock

In reply to Ricardo Martinez

From: Haijun Liu <[email protected]>

Introduce the mechanism to lock/unlock the device 'deep sleep' mode. When the PCIe link state is L1.2 or L2, the host side still can keep the device is in D0 state from the host side point of view. At the same time, if the device's 'deep sleep' mode is unlocked, the device will go to 'deep sleep' while it is still in D0 state on the host side.

---
 drivers/net/wwan/t7xx/t7xx_hif_cldma.c     | 12 +++
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c | 14 +++-
 drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c | 41 +++++++---
 drivers/net/wwan/t7xx/t7xx_mhccif.c        |  3 +
 drivers/net/wwan/t7xx/t7xx_pci.c           | 93 ++++++++++++++++++++++
 drivers/net/wwan/t7xx/t7xx_pci.h           | 10 +++
 6 files changed, 158 insertions(+), 15 deletions(-)
drivers/net/wwan/t7xx/t7xx_hif_cldma.c +12 -0
index 90306eb9858b..46066dcd2607 100644
@@ -934,6 +934,7 @@ int t7xx_cldma_send_skb(struct cldma_ctrl *md_ctrl, int qno, struct sk_buff *skb
 	if (ret < 0 && ret != -EACCES)
 		return ret;
 
+	t7xx_pci_disable_sleep(md_ctrl->t7xx_dev);
 	queue = &md_ctrl->txq[qno];
 
 	spin_lock_irqsave(&md_ctrl->cldma_lock, flags);
@@ -955,6 +956,11 @@ int t7xx_cldma_send_skb(struct cldma_ctrl *md_ctrl, int qno, struct sk_buff *skb
 			queue->tx_next = list_next_entry_circular(tx_req, gpd_ring, entry);
 			spin_unlock_irqrestore(&queue->ring_lock, flags);
 
+			if (!t7xx_pci_sleep_disable_complete(md_ctrl->t7xx_dev)) {
+				ret = -ETIMEDOUT;
+				break;
+			}
+
 			/* Protect the access to the modem for queues operations (resume/start)
 			 * which access shared locations by all the queues.
 			 * cldma_lock is independent of ring_lock which is per queue.
@@ -967,6 +973,11 @@ int t7xx_cldma_send_skb(struct cldma_ctrl *md_ctrl, int qno, struct sk_buff *skb
 		}
 		spin_unlock_irqrestore(&queue->ring_lock, flags);
 
+		if (!t7xx_pci_sleep_disable_complete(md_ctrl->t7xx_dev)) {
+			ret = -ETIMEDOUT;
+			break;
+		}
+
 		if (!t7xx_cldma_hw_queue_status(&md_ctrl->hw_info, qno, MTK_TX)) {
 			spin_lock_irqsave(&md_ctrl->cldma_lock, flags);
 			t7xx_cldma_hw_resume_queue(&md_ctrl->hw_info, qno, MTK_TX);
@@ -977,6 +988,7 @@ int t7xx_cldma_send_skb(struct cldma_ctrl *md_ctrl, int qno, struct sk_buff *skb
 	} while (!ret);
 
 allow_sleep:
+	t7xx_pci_enable_sleep(md_ctrl->t7xx_dev);
 	pm_runtime_mark_last_busy(md_ctrl->dev);
 	pm_runtime_put_autosuspend(md_ctrl->dev);
 	return ret;
drivers/net/wwan/t7xx/t7xx_hif_dpmaif_rx.c +11 -3
index 5f25555eb4a4..35a8a0d7c1ee 100644
@@ -927,8 +927,11 @@ static void t7xx_dpmaif_rxq_work(struct work_struct *work)
 	if (ret < 0 && ret != -EACCES)
 		return;
 
-	t7xx_dpmaif_do_rx(dpmaif_ctrl, rxq);
+	t7xx_pci_disable_sleep(dpmaif_ctrl->t7xx_dev);
+	if (t7xx_pci_sleep_disable_complete(dpmaif_ctrl->t7xx_dev))
+		t7xx_dpmaif_do_rx(dpmaif_ctrl, rxq);
 
+	t7xx_pci_enable_sleep(dpmaif_ctrl->t7xx_dev);
 	pm_runtime_mark_last_busy(dpmaif_ctrl->dev);
 	pm_runtime_put_autosuspend(dpmaif_ctrl->dev);
 	atomic_set(&rxq->rx_processing, 0);
@@ -1138,11 +1141,16 @@ static void t7xx_dpmaif_bat_release_work(struct work_struct *work)
 	if (ret < 0 && ret != -EACCES)
 		return;
 
+	t7xx_pci_disable_sleep(dpmaif_ctrl->t7xx_dev);
+
 	/* ALL RXQ use one BAT table, so choose DPF_RX_QNO_DFT */
 	rxq = &dpmaif_ctrl->rxq[DPF_RX_QNO_DFT];
-	t7xx_dpmaif_bat_release_and_add(rxq);
-	t7xx_dpmaif_frag_bat_release_and_add(rxq);
+	if (t7xx_pci_sleep_disable_complete(dpmaif_ctrl->t7xx_dev)) {
+		t7xx_dpmaif_bat_release_and_add(rxq);
+		t7xx_dpmaif_frag_bat_release_and_add(rxq);
+	}
 
+	t7xx_pci_enable_sleep(dpmaif_ctrl->t7xx_dev);
 	pm_runtime_mark_last_busy(dpmaif_ctrl->dev);
 	pm_runtime_put_autosuspend(dpmaif_ctrl->dev);
 }
drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c +29 -12

68 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_mhccif.c +3 -0
index 4bb452f5ccff..3ee18d46f8d2 100644
@@ -59,6 +59,9 @@ static irqreturn_t t7xx_mhccif_isr_thread(int irq, void *data)
 
 	t7xx_mhccif_clear_interrupts(t7xx_dev, int_status);
 
+	if (int_status & D2H_INT_DS_LOCK_ACK)
+		complete_all(&t7xx_dev->sleep_lock_acquire);
+
 	if (int_status & D2H_INT_SR_ACK)
 		complete(&t7xx_dev->pm_sr_ack);
 
drivers/net/wwan/t7xx/t7xx_pci.c +93 -0

129 lines not on this page · raw message

drivers/net/wwan/t7xx/t7xx_pci.h +10 -0
index f51fc5a1301f..50b37056ce5a 100644
@@ -21,6 +21,7 @@
 #include <linux/irqreturn.h>
 #include <linux/mutex.h>
 #include <linux/pci.h>
+#include <linux/spinlock.h>
 #include <linux/types.h>
 
 #include "t7xx_reg.h"
@@ -55,6 +56,9 @@ typedef irqreturn_t (*t7xx_intr_callback)(int irq, void *param);
  * @md_pm_entity_mtx: protects md_pm_entities list
  * @pm_sr_ack: ack from the device when went to sleep or woke up
  * @md_pm_state: state for resume/suspend
+ * @md_pm_lock: protects PCIe sleep lock
+ * @sleep_disable_count: PCIe L1.2 lock counter
+ * @sleep_lock_acquire: indicates that sleep has been disabled
  */
 struct t7xx_pci_dev {
 	t7xx_intr_callback	intr_handler[EXT_INT_NUM];
@@ -71,6 +75,9 @@ struct t7xx_pci_dev {
 	struct mutex		md_pm_entity_mtx;	/* Protects MD PM entities list */
 	struct completion	pm_sr_ack;
 	atomic_t		md_pm_state;
+	spinlock_t		md_pm_lock;		/* Protects PCI resource lock */
+	unsigned int		sleep_disable_count;
+	struct completion	sleep_lock_acquire;
 };
 
 enum t7xx_pm_id {
@@ -102,6 +109,9 @@ struct md_pm_entity {
 	void			*entity_param;
 };
 
+void t7xx_pci_disable_sleep(struct t7xx_pci_dev *t7xx_dev);
+void t7xx_pci_enable_sleep(struct t7xx_pci_dev *t7xx_dev);
+int t7xx_pci_sleep_disable_complete(struct t7xx_pci_dev *t7xx_dev);
 int t7xx_pci_pm_entity_register(struct t7xx_pci_dev *t7xx_dev, struct md_pm_entity *pm_entity);
 int t7xx_pci_pm_entity_unregister(struct t7xx_pci_dev *t7xx_dev, struct md_pm_entity *pm_entity);
 void t7xx_pci_pm_init_late(struct t7xx_pci_dev *t7xx_dev);
Signature · 1 line
2.25.1
author reviewer
#19 lore raw

PATCH v7 14/14 net: wwan: t7xx: Add maintainers and documentation

In reply to Ricardo Martinez

Adds maintainers and documentation for MediaTek t7xx 5G WWAN modem device driver.

---
 .../networking/device_drivers/wwan/index.rst  |   1 +
 .../networking/device_drivers/wwan/t7xx.rst   | 120 ++++++++++++++++++
 MAINTAINERS                                   |  11 ++
 3 files changed, 132 insertions(+)
 create mode 100644 Documentation/networking/device_drivers/wwan/t7xx.rst
Documentation/networking/device_drivers/wwan/index.rst +1 -0
index 1cb8c7371401..370d8264d5dc 100644
@@ -9,6 +9,7 @@ Contents:
    :maxdepth: 2
 
    iosm
+   t7xx
 
 .. only::  subproject and html
 
Documentation/networking/device_drivers/wwan/t7xx.rst new file +120 -0

120 lines not on this page · raw message

MAINTAINERS +11 -0
index fa6896e8b2d8..41bc2f3ae716 100644
@@ -12485,6 +12485,17 @@ S:	Maintained
 F:	drivers/net/dsa/mt7530.*
 F:	net/dsa/tag_mtk.c
 
+MEDIATEK T7XX 5G WWAN MODEM DRIVER
+M:	Chandrashekar Devegowda <[email protected]>
+M:	Intel Corporation <[email protected]>
+R:	Chiranjeevi Rapolu <[email protected]>
+R:	Liu Haijun <[email protected]>
+R:	M Chetan Kumar <[email protected]>
+R:	Ricardo Martinez <[email protected]>
+L:	[email protected]
+S:	Supported
+F:	drivers/net/wwan/t7xx/
+
 MEDIATEK USB3 DRD IP DRIVER
 M:	Chunfeng Yun <[email protected]>
 L:	[email protected]
Signature · 1 line
2.25.1
maintainer
#20 lore raw
In reply to Ricardo Martinez

Hello Ricardo,

Quoted 54 lines · Ricardo Martinez

t7xx is the PCIe host device driver for Intel 5G 5000 M.2 solution which is based on MediaTek's T700 modem to provide WWAN connectivity. The driver uses the WWAN framework infrastructure to create the following control ports and network interfaces:
* /dev/wwan0mbim0 - Interface conforming to the MBIM protocol. Applications like libmbim [1] or Modem Manager [2] from v1.16 onwards with [3][4] can use it to enable data communication towards WWAN.
* /dev/wwan0at0 - Interface that supports AT commands.
* wwan0 - Primary network interface for IP traffic.

The main blocks in t7xx driver are:
* PCIe layer - Implements probe, removal, and power management callbacks.
* Port-proxy - Provides a common interface to interact with different types of ports such as WWAN ports.
* Modem control & status monitor - Implements the entry point for modem initialization, reset and exit, as well as exception handling.
* CLDMA (Control Layer DMA) - Manages the HW used by the port layer to send control messages to the modem using MediaTek's CCCI (Cross-Core Communication Interface) protocol.
* DPMAIF (Data Plane Modem AP Interface) - Controls the HW that provides uplink and downlink queues for the data path. The data exchange takes place using circular buffers to share data buffer addresses and metadata to describe the packets.
* MHCCIF (Modem Host Cross-Core Interface) - Provides interrupt channels for bidirectional event notification such as handshake, exception, PM and port enumeration.

The compilation of the t7xx driver is enabled by the CONFIG_MTK_T7XX config option which depends on CONFIG_WWAN.
This driver was originally developed by MediaTek. Intel adapted t7xx to the WWAN framework, optimized and refactored the driver source code in close collaboration with MediaTek. This will enable getting the t7xx driver on the Approved Vendor List for interested OEM's and ODM's productization plans with Intel 5G 5000 M.2 solution.

List of contributors:
Amir Hanania <[email protected]>
Andriy Shevchenko <[email protected]> Chandrashekar Devegowda <[email protected]> Dinesh Sharma <[email protected]>
Eliot Lee <[email protected]>
Haijun Liu <[email protected]>
M Chetan Kumar <[email protected]>
Mika Westerberg <[email protected]> Moises Veleta <[email protected]> Pierre-louis Bossart <[email protected]> Chiranjeevi Rapolu <[email protected]> Ricardo Martinez <[email protected]> Madhusmita Sahu <[email protected]> Muralidharan Sethuraman <[email protected]> Soumya Prakash Mishra <[email protected]> Sreehari Kancharla <[email protected]> Suresh Nagaraj <[email protected]>

Quoted 4 lines

[1] https://www.freedesktop.org/software/libmbim/
[2] https://www.freedesktop.org/software/ModemManager/
[3] https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/merge_requests/582
[4] https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/merge_requests/523

Now the driver looks really nice. Good job!

Signature · 1 line
Sergey
author reviewer
#21 lore raw
Quoted 58 lines · Sergey Ryazanov

Hello Ricardo,

Quoted 54 lines · Ricardo Martinez

t7xx is the PCIe host device driver for Intel 5G 5000 M.2 solution which is based on MediaTek's T700 modem to provide WWAN connectivity. The driver uses the WWAN framework infrastructure to create the following control ports and network interfaces:
* /dev/wwan0mbim0 - Interface conforming to the MBIM protocol. Applications like libmbim [1] or Modem Manager [2] from v1.16 onwards with [3][4] can use it to enable data communication towards WWAN.
* /dev/wwan0at0 - Interface that supports AT commands.
* wwan0 - Primary network interface for IP traffic.

The main blocks in t7xx driver are:
* PCIe layer - Implements probe, removal, and power management callbacks.
* Port-proxy - Provides a common interface to interact with different types of ports such as WWAN ports.
* Modem control & status monitor - Implements the entry point for modem initialization, reset and exit, as well as exception handling.
* CLDMA (Control Layer DMA) - Manages the HW used by the port layer to send control messages to the modem using MediaTek's CCCI (Cross-Core Communication Interface) protocol.
* DPMAIF (Data Plane Modem AP Interface) - Controls the HW that provides uplink and downlink queues for the data path. The data exchange takes place using circular buffers to share data buffer addresses and metadata to describe the packets.
* MHCCIF (Modem Host Cross-Core Interface) - Provides interrupt channels for bidirectional event notification such as handshake, exception, PM and port enumeration.

The compilation of the t7xx driver is enabled by the CONFIG_MTK_T7XX config option which depends on CONFIG_WWAN.
This driver was originally developed by MediaTek. Intel adapted t7xx to the WWAN framework, optimized and refactored the driver source code in close collaboration with MediaTek. This will enable getting the t7xx driver on the Approved Vendor List for interested OEM's and ODM's productization plans with Intel 5G 5000 M.2 solution.

List of contributors:
Amir Hanania <[email protected]>
Andriy Shevchenko <[email protected]> Chandrashekar Devegowda <[email protected]> Dinesh Sharma <[email protected]>
Eliot Lee <[email protected]>
Haijun Liu <[email protected]>
M Chetan Kumar <[email protected]>
Mika Westerberg <[email protected]> Moises Veleta <[email protected]> Pierre-louis Bossart <[email protected]> Chiranjeevi Rapolu <[email protected]> Ricardo Martinez <[email protected]> Madhusmita Sahu <[email protected]> Muralidharan Sethuraman <[email protected]> Soumya Prakash Mishra <[email protected]> Sreehari Kancharla <[email protected]> Suresh Nagaraj <[email protected]>

Quoted 5 lines
Quoted 4 lines

[1] https://www.freedesktop.org/software/libmbim/
[2] https://www.freedesktop.org/software/ModemManager/
[3] https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/merge_requests/582
[4] https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/merge_requests/523

Now the driver looks really nice. Good job!

Thanks Sergey.