net: introduce Qualcomm IPA driver
Subsystems (7)
cover net: introduce Qualcomm IPA driver
This series presents the driver for the Qualcomm IP Accelerator (IPA). The IPA is a component present in some Qualcomm SoCs that allows network functions such as aggregation, filtering, routing, and NAT to be performed without active involvement of the main application processor (AP).
Initially, these advanced features are disabled; the IPA driver simply provides a network interface that makes the modem's LTE network available to the AP. In addition, only support for the IPA found in the Qualcomm SDM845 SoC is provided.
This code is derived from a driver developed internally by Qualcomm. A version of the original source can be seen here:
https://source.codeaurora.org/quic/la/kernel/msm-4.9/tree
in the "drivers/platform/msm/ipa" directory. Many were involved in developing this, but the following individuals deserve explicit acknowledgement for their substantial contributions:
Abhishek Choubey
Ady Abraham
Chaitanya Pratapa
David Arinzon
Ghanim Fodi
Gidon Studinski
Ravi Gummadidala
Shihuan Liu
Skylar Chang
A version of this code was posted in November 2018 as an RFC.
https://lore.kernel.org/lkml/[email protected]/
Fixes addressing all feedback received have been implemented. It has undergone considerable further rework since that time, and most of the "future work" described then has now been completed.
This code (including its dependencies) is available in buildable form here, based on kernel v5.1:
remote: ssh://[email protected]/people/alex.elder/linux.git
branch: ipa-v1_kernel-v5.1
f5d4a676a981 arm64: defconfig: enable build of IPA code
-Alex
Alex Elder (18):
bitfield.h: add FIELD_MAX() and field_max()
soc: qcom: create "include/soc/qcom/rmnet.h"
dt-bindings: soc: qcom: add IPA bindings
soc: qcom: ipa: main code
soc: qcom: ipa: configuration data
soc: qcom: ipa: clocking, interrupts, and memory
soc: qcom: ipa: GSI headers
soc: qcom: ipa: the generic software interface
soc: qcom: ipa: GSI transactions
soc: qcom: ipa: IPA interface to GSI
soc: qcom: ipa: IPA endpoints
soc: qcom: ipa: immediate commands
soc: qcom: ipa: IPA network device and microcontroller
soc: qcom: ipa: AP/modem communications
soc: qcom: ipa: support build of IPA code
MAINTAINERS: add entry for the Qualcomm IPA driver
arm64: dts: sdm845: add IPA information
arm64: defconfig: enable build of IPA code
.../devicetree/bindings/net/qcom,ipa.txt | 164 ++ MAINTAINERS | 6 + arch/arm64/boot/dts/qcom/sdm845.dtsi | 51 + arch/arm64/configs/defconfig | 1 + drivers/net/Kconfig | 2 + drivers/net/Makefile | 1 + .../ethernet/qualcomm/rmnet/rmnet_handlers.c | 1 + .../net/ethernet/qualcomm/rmnet/rmnet_map.h | 24 - .../qualcomm/rmnet/rmnet_map_command.c | 1 + .../ethernet/qualcomm/rmnet/rmnet_map_data.c | 1 + .../net/ethernet/qualcomm/rmnet/rmnet_vnd.c | 1 + drivers/net/ipa/Kconfig | 16 + drivers/net/ipa/Makefile | 7 + drivers/net/ipa/gsi.c | 1741 +++++++++++++++++ drivers/net/ipa/gsi.h | 241 +++ drivers/net/ipa/gsi_private.h | 147 ++ drivers/net/ipa/gsi_reg.h | 376 ++++ drivers/net/ipa/gsi_trans.c | 604 ++++++ drivers/net/ipa/gsi_trans.h | 106 + drivers/net/ipa/ipa.h | 131 ++ drivers/net/ipa/ipa_clock.c | 291 +++ drivers/net/ipa/ipa_clock.h | 52 + drivers/net/ipa/ipa_cmd.c | 372 ++++ drivers/net/ipa/ipa_cmd.h | 116 ++ drivers/net/ipa/ipa_data-sdm845.c | 245 +++ drivers/net/ipa/ipa_data.h | 267 +++ drivers/net/ipa/ipa_endpoint.c | 1253 ++++++++++++ drivers/net/ipa/ipa_endpoint.h | 96 + drivers/net/ipa/ipa_gsi.c | 48 + drivers/net/ipa/ipa_gsi.h | 49 + drivers/net/ipa/ipa_interrupt.c | 279 +++ drivers/net/ipa/ipa_interrupt.h | 53 + drivers/net/ipa/ipa_main.c | 920 +++++++++ drivers/net/ipa/ipa_mem.c | 237 +++ drivers/net/ipa/ipa_mem.h | 82 + drivers/net/ipa/ipa_netdev.c | 250 +++ drivers/net/ipa/ipa_netdev.h | 24 + drivers/net/ipa/ipa_qmi.c | 399 ++++ drivers/net/ipa/ipa_qmi.h | 35 + drivers/net/ipa/ipa_qmi_msg.c | 583 ++++++ drivers/net/ipa/ipa_qmi_msg.h | 238 +++ drivers/net/ipa/ipa_reg.h | 279 +++ drivers/net/ipa/ipa_smp2p.c | 304 +++ drivers/net/ipa/ipa_smp2p.h | 47 + drivers/net/ipa/ipa_uc.c | 208 ++ drivers/net/ipa/ipa_uc.h | 32 + include/linux/bitfield.h | 14 + include/soc/qcom/rmnet.h | 38 + 48 files changed, 10409 insertions(+), 24 deletions(-) create mode 100644 Documentation/devicetree/bindings/net/qcom,ipa.txt create mode 100644 drivers/net/ipa/Kconfig create mode 100644 drivers/net/ipa/Makefile create mode 100644 drivers/net/ipa/gsi.c create mode 100644 drivers/net/ipa/gsi.h create mode 100644 drivers/net/ipa/gsi_private.h create mode 100644 drivers/net/ipa/gsi_reg.h create mode 100644 drivers/net/ipa/gsi_trans.c create mode 100644 drivers/net/ipa/gsi_trans.h create mode 100644 drivers/net/ipa/ipa.h create mode 100644 drivers/net/ipa/ipa_clock.c create mode 100644 drivers/net/ipa/ipa_clock.h create mode 100644 drivers/net/ipa/ipa_cmd.c create mode 100644 drivers/net/ipa/ipa_cmd.h create mode 100644 drivers/net/ipa/ipa_data-sdm845.c create mode 100644 drivers/net/ipa/ipa_data.h create mode 100644 drivers/net/ipa/ipa_endpoint.c create mode 100644 drivers/net/ipa/ipa_endpoint.h create mode 100644 drivers/net/ipa/ipa_gsi.c create mode 100644 drivers/net/ipa/ipa_gsi.h create mode 100644 drivers/net/ipa/ipa_interrupt.c create mode 100644 drivers/net/ipa/ipa_interrupt.h create mode 100644 drivers/net/ipa/ipa_main.c create mode 100644 drivers/net/ipa/ipa_mem.c create mode 100644 drivers/net/ipa/ipa_mem.h create mode 100644 drivers/net/ipa/ipa_netdev.c create mode 100644 drivers/net/ipa/ipa_netdev.h create mode 100644 drivers/net/ipa/ipa_qmi.c create mode 100644 drivers/net/ipa/ipa_qmi.h create mode 100644 drivers/net/ipa/ipa_qmi_msg.c create mode 100644 drivers/net/ipa/ipa_qmi_msg.h create mode 100644 drivers/net/ipa/ipa_reg.h create mode 100644 drivers/net/ipa/ipa_smp2p.c create mode 100644 drivers/net/ipa/ipa_smp2p.h create mode 100644 drivers/net/ipa/ipa_uc.c create mode 100644 drivers/net/ipa/ipa_uc.h create mode 100644 include/soc/qcom/rmnet.h
Signature · 1 line
2.20.1
PATCH 1/18 bitfield.h: add FIELD_MAX() and field_max()
Define FIELD_MAX(), which supplies the maximum value that can be represented by a field value. Define field_max() as well, to go along with the lower-case forms of the field mask functions.
--- include/linux/bitfield.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
include/linux/bitfield.h +14 -0
@@ -63,6 +63,19 @@ (1ULL << __bf_shf(_mask))); \ }) +/** +* FIELD_MAX() - produce the maximum value representable by a field +* @_mask: shifted mask defining the field's length and position +* +* FIELD_MAX() returns the maximum value that can be held in the field +* specified by @_mask. +*/ +#define FIELD_MAX (_mask) \ + ({ \ + __BF_FIELD_CHECK(_mask, 0ULL, 0ULL, "FIELD_MAX: "); \ + (typeof(_mask))((_mask) >> __bf_shf(_mask)); \ + }) +/** * FIELD_FIT() - check if value fits in the field * @_mask: shifted mask defining the field's length and position
@@ -118,6 +131,7 @@ static __always_inline u64 field_mask(u64 field) {return field /field_multiplier (field); } +#define field_max (field) ((typeof(field))field_mask(field))#define ____MAKE_OP (type,base,to,from) \ static __always_inline __##type type##_encode_bits(base v, base field) \ { \
Signature · 1 line
2.20.1
Re: [PATCH 01/18] bitfield.h: add FIELD_MAX() and field_max()
Quoted 5 lines · Alex Elder
Define
FIELD_MAX(), which supplies the maximum value that can be represented by a field value. Definefield_max()as well, to go along with the lower-case forms of the field mask functions.Signed-off-by Alex Elder <[email protected]>
Via which tree is this going? I assume I do not have take it unless someone says otherwise.
Signature · 1 line
Kalle Valo
Re: [PATCH 01/18] bitfield.h: add FIELD_MAX() and field_max()
Quoted 10 lines · Kalle Valo
Quoted 5 lines · Alex Elder
Define
FIELD_MAX(), which supplies the maximum value that can be represented by a field value. Definefield_max()as well, to go along with the lower-case forms of the field mask functions.Signed-off-by Alex Elder <[email protected]>Via which tree is this going? I assume I do not have take it unless someone says otherwise.
Sorry about that, perhaps I should have posted it separately.
I don't have an answer, but we could avoid having to coordinate if it went together with the rest.
-Alex
Re: [PATCH 01/18] bitfield.h: add FIELD_MAX() and field_max()
Quoted 9 lines · Alex Elder
Quoted 8 lines · Kalle Valo
Quoted 5 lines · Alex Elder
Define
FIELD_MAX(), which supplies the maximum value that can be represented by a field value. Definefield_max()as well, to go along with the lower-case forms of the field mask functions.Signed-off-by Alex Elder <[email protected]>
Quoted 7 lines
Quoted 2 lines
Via which tree is this going? I assume I do not have take it unless someone says otherwise.
Sorry about that, perhaps I should have posted it separately.
I don't have an answer, but we could avoid having to coordinate if it went together with the rest.
It's unlikely to conflict, and I don't think anyone really thinks that the file is "theirs" (being basically standalone), so I think you should just take it with whatever code that needs it.
johannes
PATCH 2/18 soc: qcom: create "include/soc/qcom/rmnet.h"
In reply to Alex ElderThe IPA driver requires some (but not all) symbols defined in "drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h". Create a new public header file "include/soc/qcom/rmnet.h" and move the needed definitions there.
--- .../ethernet/qualcomm/rmnet/rmnet_handlers.c | 1 + .../net/ethernet/qualcomm/rmnet/rmnet_map.h | 24 ------------ .../qualcomm/rmnet/rmnet_map_command.c | 1 + .../ethernet/qualcomm/rmnet/rmnet_map_data.c | 1 + .../net/ethernet/qualcomm/rmnet/rmnet_vnd.c | 1 + include/soc/qcom/rmnet.h | 38 +++++++++++++++++++ 6 files changed, 42 insertions(+), 24 deletions(-) create mode 100644 include/soc/qcom/rmnet.h
drivers/net/ethernet/qualcomm/rmnet/rmnet_handlers.c +1 -0
@@ -17,6 +17,7 @@#include <linux/netdev_features.h> #include <linux/if_arp.h> #include <net/sock.h> +#include <soc/qcom/rmnet.h> #include "rmnet_private.h" #include "rmnet_config.h" #include "rmnet_vnd.h"
drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h +0 -24
@@ -39,30 +39,6 @@ enum rmnet_map_commands { RMNET_MAP_COMMAND_ENUM_LENGTH }; -struct rmnet_map_header { -u8 pad_len:6 ; -u8 reserved_bit:1 ; -u8 cd_bit:1 ; -u8 mux_id; -__be16 pkt_len; -}__aligned(1) ; - -struct rmnet_map_dl_csum_trailer { -u8 reserved1; -u8 valid:1 ; -u8 reserved2:7 ; -u16 csum_start_offset; -u16 csum_length; -__be16 csum_value; -}__aligned(1) ; - -struct rmnet_map_ul_csum_header { -__be16 csum_start_offset; -u16 csum_insert_offset:14 ; -u16 udp_ip4_ind:1 ; -u16 csum_enabled:1 ; -}__aligned(1) ; -#define RMNET_MAP_GET_MUX_ID (Y ) (((struct rmnet_map_header *) \ (Y)->data)->mux_id)#define RMNET_MAP_GET_CD_BIT (Y ) (((struct rmnet_map_header *) \
drivers/net/ethernet/qualcomm/rmnet/rmnet_map_command.c +1 -0
@@ -11,6 +11,7 @@*/ #include <linux/netdevice.h> +#include <soc/qcom/rmnet.h> #include "rmnet_config.h" #include "rmnet_map.h" #include "rmnet_private.h"
drivers/net/ethernet/qualcomm/rmnet/rmnet_map_data.c +1 -0
@@ -17,6 +17,7 @@#include <linux/ip.h> #include <linux/ipv6.h> #include <net/ip6_checksum.h> +#include <soc/qcom/rmnet.h> #include "rmnet_config.h" #include "rmnet_map.h" #include "rmnet_private.h"
drivers/net/ethernet/qualcomm/rmnet/rmnet_vnd.c +1 -0
@@ -17,6 +17,7 @@#include <linux/etherdevice.h> #include <linux/if_arp.h> #include <net/pkt_sched.h> +#include <soc/qcom/rmnet.h> #include "rmnet_config.h" #include "rmnet_handlers.h" #include "rmnet_private.h"
include/soc/qcom/rmnet.h new file +38 -0
@@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ +#ifndef _SOC_QCOM_RMNET_H_ +#define _SOC_QCOM_RMNET_H_ + +#include <linux/types.h> + +/* Header structure that precedes packets in ETH_P_MAP protocol */ +struct rmnet_map_header { +u8 pad_len :6 ; +u8 reserved_bit :1 ; +u8 cd_bit :1 ; +u8 mux_id; +__be16 pkt_len; +}__aligned(1) ; + +/* Checksum offload metadata header for outbound packets*/ +struct rmnet_map_ul_csum_header { +__be16 csum_start_offset; +u16 csum_insert_offset :14 ; +u16 udp_ip4_ind :1 ; +u16 csum_enabled :1 ; +}__aligned(1) ; + +/* Checksum offload metadata trailer for inbound packets */ +struct rmnet_map_dl_csum_trailer { +u8 reserved1; +u8 valid :1 ; +u8 reserved2 :7 ; +u16 csum_start_offset; +u16 csum_length; +__be16 csum_value; +}__aligned(1) ; + +#endif /* _SOC_QCOM_RMNET_H_ */
Signature · 1 line
2.20.1
Re: [PATCH 02/18] soc: qcom: create "include/soc/qcom/rmnet.h"
Quoted 1 line · Alex Elder
include/soc/qcom/rmnet.h
Should this file be added to the MAINTAINERS file
update in patch 16/18 ?
Re: [PATCH 02/18] soc: qcom: create "include/soc/qcom/rmnet.h"
Quoted 5 lines · Joe Perches
Quoted 1 line · Alex Elder
include/soc/qcom/rmnet.hShould this file be added to the
MAINTAINERSfile
update in patch 16/18 ?
Sure, that's a good point. I'll add it when I submit a v2. Thank you.
-Alex
Re: [PATCH 02/18] soc: qcom: create "include/soc/qcom/rmnet.h"
In reply to Alex ElderQuoted 14 lines · Alex Elder
include/soc/qcom/rmnet.h new file +8
-0@@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ +#ifndef _SOC_QCOM_RMNET_H_ +#define _SOC_QCOM_RMNET_H_ +
include/soc/qcom/rmnet.h line 18
Quoted 10 lines
include/soc/qcom/rmnet.h new file +10
-0+#include <linux/types.h> + +/* Header structure that precedes packets in ETH_P_MAP protocol */ +struct rmnet_map_header { +u8 pad_len :6 ; +u8 reserved_bit :1 ; +u8 cd_bit :1 ; +u8 mux_id; +__be16 pkt_len; +}__aligned(1) ;
If we move this into include/soc/, I want the structure to be portable, and avoid the bit fields. Please use mask/shift operations or the include/linux/bits.h macros instead to make this work with big-endian kernels.
Arnd
Re: [PATCH 02/18] soc: qcom: create "include/soc/qcom/rmnet.h"
Quoted 23 lines · Arnd Bergmann
Quoted 21 lines · Alex Elder
include/soc/qcom/rmnet.h new file +15
-0@@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ +#ifndef _SOC_QCOM_RMNET_H_ +#define _SOC_QCOM_RMNET_H_ + +#include <linux/types.h> + +/* Header structure that precedes packets in ETH_P_MAP protocol */ +struct rmnet_map_header { + u8 pad_len : 6; + u8 reserved_bit : 1; + u8 cd_bit : 1;
Quoted 8 lines
include/soc/qcom/rmnet.h line 18
Quoted 3 lines
include/soc/qcom/rmnet.h new file +3
-0+u8 mux_id; +__be16 pkt_len; +}__aligned(1) ;If we move this into
include/soc/, I want the structure to be portable, and avoid the bit fields. Please use mask/shift operations or theinclude/linux/bits.hmacros instead to make this work with big-endian kernels.
Sure, I'll do that. I did that everywhere else in the driver, but here I just tried to preserve the original code as I moved it. I will update at least these structures, and all existing code (plus the IPA code) to use fields masks.
-Alex
Quoted 3 lines
Arnd
Re: [PATCH 02/18] soc: qcom: create "include/soc/qcom/rmnet.h"
Quoted 19 lines
Quoted 10 lines
Quoted 10 lines
include/soc/qcom/rmnet.h new file +10
-0+#ifndef _SOC_QCOM_RMNET_H_ +#define _SOC_QCOM_RMNET_H_ + +#include <linux/types.h> + +/* Header structure that precedes packets in ETH_P_MAP protocol */ +struct rmnet_map_header { + u8 pad_len : 6; + u8 reserved_bit : 1; + u8 cd_bit : 1;Quoted 8 lines
include/soc/qcom/rmnet.h line 18
Quoted 3 lines
include/soc/qcom/rmnet.h new file +3
-0+u8 mux_id; +__be16 pkt_len; +}__aligned(1) ;If we move this into
include/soc/, I want the structure to be portable, and avoid the bit fields. Please use mask/shift operations or theinclude/linux/bits.hmacros instead to make this work with big-endian kernels.
Quoted 9 lines
Sure, I'll do that. I did that everywhere else in the driver, but here I just tried to preserve the original code as I moved it. I will update at least these structures, and all existing code (plus the IPA code) to use fields masks.
-Alex
Quoted 3 lines
Arnd
Hi Alex
Could we instead have the rmnet header definition in include/linux/if_rmnet.h
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [PATCH 02/18] soc: qcom: create "include/soc/qcom/rmnet.h"
On 5/15/19 8:09 PM, Subash Abhinov Kasiviswanathan wrote:
. . .
Quoted 4 lines
Hi Alex
Could we instead have the rmnet header definition in
include/linux/if_rmnet.h
I have no objection to that, but I don't actually know what the criteria are for putting a file in that directory.
Glancing at other "if_*" files there it seems sensible, but because I don't know, I'd like to have a little better justification.
Can you provide a good explanation about why these definitions belong in "include/linux/if_rmnet.h" instead of "include/soc/qcom/rmnet.h"?
Thanks.
-Alex
Re: [PATCH 02/18] soc: qcom: create "include/soc/qcom/rmnet.h"
Quoted 14 lines · Alex Elder
On 5/15/19 8:09 PM, Subash Abhinov Kasiviswanathan wrote:
. . .Quoted 4 lines
Hi Alex
Could we instead have the rmnet header definition in
include/linux/if_rmnet.hI have no objection to that, but I don't actually know what the criteria are for putting a file in that directory.
Glancing at other "if_*" files there it seems sensible, but because I don't know, I'd like to have a little better justification.
Quoted 7 lines
Can you provide a good explanation about why these definitions belong in "
include/linux/if_rmnet.h" instead of "include/soc/qcom/rmnet.h"?Thanks.
-Alex
rmnet was designed similar to vlan / macvlan / ipvlan / bridge. These drivers support creation of virtual netdevices, define custom rtnl_link_ops, expose netlink attributes to uapi via if_link.h and register rx_handlers.
They expose some common structs and helpers via if_vlan.h / if_macvlan.h / if_bridge.h. I would prefer rmnet to use if_rmnet.h similar to them.
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Re: [PATCH 02/18] soc: qcom: create "include/soc/qcom/rmnet.h"
Quoted 1 line · Subash Abhinov Kasiviswanathan
On 2019-05-17 11:27, Alex Elder wrote:
. . .
Quoted 8 lines
Quoted 7 lines
Can you provide a good explanation about why these definitions belong in "
include/linux/if_rmnet.h" instead of "include/soc/qcom/rmnet.h"?Thanks.
-Alex
Quoted 8 lines
rmnet was designed similar to vlan / macvlan / ipvlan / bridge. These drivers support creation of virtual netdevices, define custom
rtnl_link_ops, expose netlink attributes to uapi viaif_link.hand registerrx_handlers.They expose some common structs and helpers via
if_vlan.h/if_macvlan.h/if_bridge.h. I would prefer rmnet to useif_rmnet.hsimilar to them.
OK, I will name the file "include/linux/if_rmnet.h" as you suggest. It will still only define the three structures that I need in the IPA driver; I won't expose anything else from the rmnet_data driver.
I will mention now that, to facilitate addressing Arnd's concerns about the portability of using C bit-fields in these structures, I made a set of other changes (including a bug fix in one of
the structure definitions). As a preview, here are the subject lines for that series:
net: qualcomm: rmnet: fix struct rmnet_map_header
net: qualcomm: rmnet: kill RMNET_MAP_GET_*() accessor macros
net: qualcomm: rmnet: use field masks instead of C bit-fields
net: qualcomm: rmnet: don't use C bit-fields in rmnet checksum header
net: qualcomm: rmnet: don't use C bit-fields in rmnet checksum trailer
soc: qcom: ipa: get rid of a variable in rmnet_map_ipv4_ul_csum_header()
net: create "include/linux/if_rmnet.h"
I will be posting that as a separate series now and will have the IPA driver series mention a dependence on that.
-Alex
PATCH 3/18 dt-bindings: soc: qcom: add IPA bindings
In reply to Alex ElderAdd the binding definitions for the "qcom,ipa" device tree node.
--- .../devicetree/bindings/net/qcom,ipa.txt | 164 ++++++++++++++++++ 1 file changed, 164 insertions(+) create mode 100644 Documentation/devicetree/bindings/net/qcom,ipa.txt
Documentation/devicetree/bindings/net/qcom,ipa.txt new file +164 -0
@@ -0,0 +1,164 @@ +Qualcomm IP Accelerator (IPA) + +This binding describes the Qualcomm IPA. The IPA is capable of offloading +certain network processing tasks (e.g. filtering, routing, and NAT) from +the main processor. + +The IPA sits between multiple independent "execution environments," +including the Application Processor (AP) and the modem. The IPA presents +a Generic Software Interface (GSI) to each execution environment. +The GSI is an integral part of the IPA, but it is logically isolated +and has a distinct interrupt and a separately-defined address space. + + -------- --------- + | | | | + | AP +<---. .----+ Modem | + | +--. | | .->+ | + | | | | | | | | + -------- | | | | --------- + v | v | + --+-+---+-+-- + | GSI | + |-----------| + | | + | IPA | + | | + ------------- + +See also: + bindings/interrupt-controller/interrupts.txt + bindings/interconnect/interconnect.txt + bindings/soc/qcom/qcom,smp2p.txt + bindings/reserved-memory/reserved-memory.txt + bindings/clock/clock-bindings.txt + +All properties except "modem-init" defined below are required. + +- compatible: + Must be "qcom,sdm845-ipa". + +- modem-init: + This Boolean property is optional. If present, it indicates that + the modem is responsible for performing early IPA initialization, + including loading and validating firwmare used by the GSI. This + early initialization is performed by Trust Zone otherwise. + +- reg: + Resources specifying the physical address spaces of the IPA and GSI. + +- reg-names: + The names of the two address space ranges defined by the "reg" + property. Must be: + "ipa-reg" + "ipa-shared" + "gsi" + +- interrupts: + Specifies the IRQs used by the IPA. Four interrupts are required, + specifying: the IPA IRQ; the GSI IRQ; the clock query interrupt + from the modem; and the "ready for setup" interrupt from the modem. + The first two are hardware IRQs; the third and fourth are SMP2P + input interrupts. + +- interrupt-names: + The names of the interrupts defined by the "interrupts-extended" + property. Must be: + "ipa" + "gsi" + "ipa-clock-query" + "ipa-setup-ready" + +- clocks: + Resource that defines the IPA core clock. + +- clock-names: + The name used for the IPA core clock. Must be "core". + +- interconnects: + Specifies the interconnects used by the IPA. Three interconnects + are required, specifying: the path from the IPA to memory; from + IPA to internal (SoC resident) memory; and between the AP subsystem + and IPA for register access. + +- interconnect-names: + The names of the interconnects defined by the "interconnects" + property. Must be: + "memory" + "imem" + "config" + +- qcom,smem-states + The state bits used for SMP2P output. Two states must be specified. + The first indicates whether the value in the second bit is valid + (1 means valid). The second, if valid, defines whether the IPA + clock is enabled (1 means enabled). + +- qcom,smem-state-names + The names of the state bits used for SMP2P output. Must be: + "ipa-clock-enabled-valid" + "ipa-clock-enabled" + +- memory-region + A phandle for a reserved memory area that holds the firmware passed + to Trust Zone for authentication. (Note, this is required + only when Trust Zone performs early initialization; that is, + it is required if "modem-init" is not defined.) + += EXAMPLE + +The following example represents the IPA present in the SDM845 SoC. It +shows portions of the "modem-smp2p" node to indicate its relationship +with the interrupts and SMEM states used by the IPA. + + smp2p-mpss { + compatible = "qcom,smp2p"; + . . . + ipa_smp2p_out: ipa-ap-to-modem { + qcom,entry-name = "ipa"; + #qcom,smem-state-cells = <1>; + }; + + ipa_smp2p_in: ipa-modem-to-ap { + qcom,entry-name = "ipa"; + interrupt-controller; + #interrupt-cells = <2>; + }; + }; + + ipa@1e40000 { + compatible = "qcom,sdm845-ipa"; + + modem-init; + + reg = <0 0x1e40000 0 0x7000>, + <0 0x1e47000 0 0x2000>, + <0 0x1e04000 0 0x2c000>; + reg-names = "ipa-reg", + "ipa-shared"; + "gsi"; + + interrupts-extended = <&intc 0 311 IRQ_TYPE_EDGE_RISING>, + <&intc 0 432 IRQ_TYPE_LEVEL_HIGH>, + <&ipa_smp2p_in 0 IRQ_TYPE_EDGE_RISING>, + <&ipa_smp2p_in 1 IRQ_TYPE_EDGE_RISING>; + interrupt-names = "ipa", + "gsi", + "ipa-clock-query", + "ipa-setup-ready"; + + clocks = <&rpmhcc RPMH_IPA_CLK>; + clock-names = "core"; + + interconnects = + <&rsc_hlos MASTER_IPA &rsc_hlos SLAVE_EBI1>, + <&rsc_hlos MASTER_IPA &rsc_hlos SLAVE_IMEM>, + <&rsc_hlos MASTER_APPSS_PROC &rsc_hlos SLAVE_IPA_CFG>; + interconnect-names = "memory", + "imem", + "config"; + + qcom,smem-states = <&ipa_smp2p_out 0>, + <&ipa_smp2p_out 1>; + qcom,smem-state-names = "ipa-clock-enabled-valid", + "ipa-clock-enabled"; + };
Signature · 1 line
2.20.1
Re: [PATCH 03/18] dt-bindings: soc: qcom: add IPA bindings
Documentation/devicetree/bindings/net/qcom,ipa.txt
Quoted 14 lines · Alex Elder
Add the binding definitions for the "qcom,ipa" device tree node.
Signed-off-by Alex Elder <[email protected]>--- .../devicetree/bindings/net/qcom,ipa.txt | 164 ++++++++++++++++++ 1 file changed, 164 insertions(+) create mode 100644 Documentation/devicetree/bindings/net/qcom,ipa.txtDocumentation/devicetree/bindings/net/qcom,ipa.txt new file +0
-0
For new bindings, we should use the yaml format so we can verify the device tree files against the binding.
Documentation/devicetree/bindings/net/qcom,ipa.txt line 54
Quoted 10 lines
Documentation/devicetree/bindings/net/qcom,ipa.txt new file +10
-0+ +- reg: + Resources specifying the physical address spaces of the IPA and GSI. + +- reg-names: + The names of the two address space ranges defined by the "reg" + property. Must be: + "ipa-reg" + "ipa-shared" + "gsi"
Those are three, not two ;-)
Arnd
Re: [PATCH 03/18] dt-bindings: soc: qcom: add IPA bindings
Quoted 10 lines · Arnd Bergmann
Quoted 9 lines · Alex Elder
Add the binding definitions for the "qcom,ipa" device tree node.
Signed-off-by Alex Elder <[email protected]>--- .../devicetree/bindings/net/qcom,ipa.txt | 164 ++++++++++++++++++ 1 file changed, 164 insertions(+) create mode 100644 Documentation/devicetree/bindings/net/qcom,ipa.txt
Quoted 8 lines
Documentation/devicetree/bindings/net/qcom,ipa.txt
Quoted 5 lines
Documentation/devicetree/bindings/net/qcom,ipa.txt new file +0
-0For new bindings, we should use the yaml format so we can verify the device tree files against the binding.
OK. I didn't realize that was upstream yet. I will convert.
Quoted 4 lines
Quoted 4 lines
Documentation/devicetree/bindings/net/qcom,ipa.txt new file +4
-0+ +- reg: + Resources specifying the physical address spaces of the IPA and GSI. +
Quoted 8 lines
Documentation/devicetree/bindings/net/qcom,ipa.txt line 54
Quoted 6 lines
+- reg-names:
Documentation/devicetree/bindings/net/qcom,ipa.txt new file +5
-0+ The names of the two address space ranges defined by the "reg" + property. Must be: + "ipa-reg" + "ipa-shared" + "gsi"Those are three, not two ;-)
Oops! I added one recently and I guess I missed that. Thanks for catching it.
-Alex
Quoted 3 lines
Arnd
Re: [PATCH 03/18] dt-bindings: soc: qcom: add IPA bindings
Quoted 21 lines · Alex Elder
Quoted 10 lines · Arnd Bergmann
Quoted 9 lines · Alex Elder
Add the binding definitions for the "qcom,ipa" device tree node.
Signed-off-by Alex Elder <[email protected]>--- .../devicetree/bindings/net/qcom,ipa.txt | 164 ++++++++++++++++++ 1 file changed, 164 insertions(+) create mode 100644 Documentation/devicetree/bindings/net/qcom,ipa.txtQuoted 8 lines
Documentation/devicetree/bindings/net/qcom,ipa.txt
Quoted 5 lines
Documentation/devicetree/bindings/net/qcom,ipa.txt new file +0
-0For new bindings, we should use the yaml format so we can verify the device tree files against the binding.
Quoted 1 line
OK. I didn't realize that was upstream yet. I will convert.
Not required yet, but it puts the maintainer in a good mood. :)
As does CCing the DT list.
Rob
Re: [PATCH 03/18] dt-bindings: soc: qcom: add IPA bindings
Quoted 4 lines · Rob Herring
Quoted 1 line
OK. I didn't realize that was upstream yet. I will convert.
Not required yet, but it puts the maintainer in a good mood. :)
As does CCing the DT list.
I'll convert it to YAML *and* CC the DT list next time, to avoid triggering a crabby maintainer.
-Alex
PATCH 4/18 soc: qcom: ipa: main code
In reply to Alex ElderThis patch includes three source files that represent some basic "main program" code for the IPA driver. They are:
- "ipa.h" defines the top-level IPA structure which represents an IPA device throughout the code.
- "ipa_main.c" contains the platform driver probe function, along with some general code used during initialization.
- "ipa_reg.h" defines the offsets of the 32-bit registers used for the IPA device, along with masks that define the position and width of fields less than 32 bits located within these registers.
Each file includes some documentation that provides a little more overview of how the code is organized and used.
--- drivers/net/ipa/ipa.h | 131 ++++++ drivers/net/ipa/ipa_main.c | 920 +++++++++++++++++++++++++++++++++++++ drivers/net/ipa/ipa_reg.h | 279 +++++++++++ 3 files changed, 1330 insertions(+) create mode 100644 drivers/net/ipa/ipa.h create mode 100644 drivers/net/ipa/ipa_main.c create mode 100644 drivers/net/ipa/ipa_reg.h
drivers/net/ipa/ipa.h new file +131 -0
@@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ +#ifndef _IPA_H_ +#define _IPA_H_ + +#include <linux/types.h> +#include <linux/device.h> +#include <linux/notifier.h> +#include <linux/pm_wakeup.h> + +#include "gsi.h" +#include "ipa_qmi.h" +#include "ipa_endpoint.h" +#include "ipa_interrupt.h" + +struct clk ; +struct icc_path ; +struct net_device ; +struct platform_device ; + +struct ipa_clock ; +struct ipa_smp2p ; +struct ipa_interrupt ; + +/** +* struct ipa - IPA information +* @gsi: Embedded GSI structure +* @pdev: Platform device +* @smp2p: SMP2P information +* @clock: IPA clocking information +* @suspend_ref: Whether clock reference preventing suspend taken +* @route_virt: Virtual address of routing table +* @route_addr: DMA address for routing table +* @filter_virt: Virtual address of filter table +* @filter_addr: DMA address for filter table +* @interrupt: IPA Interrupt information +* @uc_loaded: Non-zero when microcontroller has reported it's ready +* @ipa_phys: Physical address of IPA memory space +* @ipa_virt: Virtual address for IPA memory space +* @reg_virt: Virtual address used for IPA register access +* @shared_phys: Physical address of memory space shared with modem +* @shared_virt: Virtual address of memory space shared with modem +* @shared_offset: Additional offset used for shared memory +* @wakeup: Wakeup source information +* @filter_support: Bit mask indicating endpoints that support filtering +* @initialized: Bit mask indicating endpoints initialized +* @set_up: Bit mask indicating endpoints set up +* @enabled: Bit mask indicating endpoints enabled +* @suspended: Bit mask indicating endpoints suspended +* @endpoint: Array of endpoint information +* @endpoint_map: Mapping of GSI channel to IPA endpoint information +* @command_endpoint: Endpoint used for command TX +* @default_endpoint: Endpoint used for default route RX +* @modem_netdev: Network device structure used for modem +* @setup_complete: Flag indicating whether setup stage has completed +* @qmi: QMI information +*/ +struct ipa { +struct gsi gsi; +struct platform_device *pdev; +struct ipa_smp2p *smp2p; +struct ipa_clock *clock; +atomic_t suspend_ref; + +void *route_virt; +dma_addr_t route_addr; +void *filter_virt; +dma_addr_t filter_addr; + +struct ipa_interrupt *interrupt; +u32 uc_loaded; + +phys_addr_t reg_phys; +void __iomem *reg_virt; +phys_addr_t shared_phys; +void *shared_virt; +u32 shared_offset; + +struct wakeup_source wakeup; + +/* Bit masks indicating endpoint state */ +u32 filter_support; +u32 initialized; +u32 set_up; +u32 enabled; +u32 suspended; + +struct ipa_endpoint endpoint[IPA_ENDPOINT_MAX ]; +struct ipa_endpoint *endpoint_map[GSI_CHANNEL_MAX ]; +struct ipa_endpoint *command_endpoint;/* TX */ +struct ipa_endpoint *default_endpoint;/* Default route RX */ + +struct net_device *modem_netdev; +u32 setup_complete; + +struct ipa_qmi qmi; +}; + +/** +* ipa_setup() - Perform IPA setup +* @ipa: IPA pointer +* +* IPA initialization is broken into stages: init; config; setup; and +* sometimes enable. (These have inverses exit, deconfig, teardown, and +* disable.) Activities performed at the init stage can be done without +* requiring any access to hardware. For IPA, activities performed at the +* config stage require the IPA clock to be running, because they involve +* access to IPA registers. The setup stage is performed only after the +* GSI hardware is ready (more on this below). And finally IPA endpoints +* can be enabled once they're successfully set up. +* +* This function, @ipa_setup(), starts the setup stage. +* +* In order for the GSI hardware to be functional it needs firmware to be +* loaded (in addition to some other low-level initialization). This early +* GSI initialization can be done either by Trust Zone or by the modem. If +* it's done by Trust Zone, the AP loads the GSI firmware and supplies it to +* Trust Zone to verify and install. The AP knows when this completes, and +* whether it was successful. In this case the AP proceeds to setup once it +* knows GSI is ready. +* +* If the modem performs early GSI initialization, the AP needs to know when +* this has occurred. An SMP2P interrupt is used for this purpose, and +* receipt of that interrupt triggers the call to ipa_setup(). +*/ +int ipa_setup (struct ipa *ipa); + +#endif /* _IPA_H_ */
drivers/net/ipa/ipa_main.c new file +920 -0
920 lines not on this page · raw message
drivers/net/ipa/ipa_reg.h new file +279 -0
279 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 5/18 soc: qcom: ipa: configuration data
In reply to Alex ElderThis patch defines configuration data that is used to specify some of the details of IPA hardware supported by the driver. It is built as Device Tree match data, discovered at boot time. Initially the driver only supports the Qualcomm SDM845 SoC.
--- drivers/net/ipa/ipa_data-sdm845.c | 245 +++++++++++++++++++++++++++ drivers/net/ipa/ipa_data.h | 267 ++++++++++++++++++++++++++++++ 2 files changed, 512 insertions(+) create mode 100644 drivers/net/ipa/ipa_data-sdm845.c create mode 100644 drivers/net/ipa/ipa_data.h
drivers/net/ipa/ipa_data-sdm845.c new file +245 -0
@@ -0,0 +1,245 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#include <linux/log2.h> + +#include "gsi.h" +#include "ipa_data.h" +#include "ipa_endpoint.h" + +/* Differentiate Boolean from numerical options */ +#define NO 0 +#define YES 1 + +/* Endpoint configuration for the SDM845 SoC. */ +static const struct gsi_ipa_endpoint_data gsi_ipa_endpoint_data[] = { + { + .ee_id =GSI_EE_AP , + .channel_id =4 , + .endpoint_id =IPA_ENDPOINT_AP_COMMAND_TX , + .toward_ipa =YES , + .channel = { + .tlv_count =20 , + .wrr_priority =YES , + .tre_count =256 , + .event_count =512 , + }, + .endpoint = { + .seq_type =IPA_SEQ_DMA_ONLY , + .config = { + .dma_mode =YES , + .dma_endpoint =IPA_ENDPOINT_AP_LAN_RX , + }, + }, + }, + { + .ee_id =GSI_EE_AP , + .channel_id =5 , + .endpoint_id =IPA_ENDPOINT_AP_LAN_RX , + .toward_ipa =NO , + .channel = { + .tlv_count =8 , + .tre_count =256 , + .event_count =256 , + }, + .endpoint = { + .seq_type =IPA_SEQ_INVALID , + .config = { + .checksum =YES , + .aggregation =YES , + .status_enable =YES , + .rx = { + .pad_align =ilog2 (sizeof (u32)), + }, + }, + }, + }, + { + .ee_id =GSI_EE_AP , + .channel_id =3 , + .endpoint_id =IPA_ENDPOINT_AP_MODEM_TX , + .toward_ipa =YES , + .channel = { + .tlv_count =16 , + .tre_count =512 , + .event_count =512 , + }, + .endpoint = { + .support_flt =YES , + .seq_type = +IPA_SEQ_2ND_PKT_PROCESS_PASS_NO_DEC_UCP , + .config = { + .checksum =YES , + .qmap =YES , + .status_enable =YES , + .tx = { + .delay =YES , + .status_endpoint = +IPA_ENDPOINT_MODEM_AP_RX , + }, + }, + }, + }, + { + .ee_id =GSI_EE_AP , + .channel_id =6 , + .endpoint_id =IPA_ENDPOINT_AP_MODEM_RX , + .toward_ipa =NO , + .channel = { + .tlv_count =8 , + .tre_count =256 , + .event_count =256 , + }, + .endpoint = { + .seq_type =IPA_SEQ_INVALID , + .config = { + .checksum =YES , + .qmap =YES , + .aggregation =YES , + .rx = { + .aggr_close_eof =YES , + }, + }, + }, + }, + { + .ee_id =GSI_EE_MODEM , + .channel_id =1 , + .endpoint_id =IPA_ENDPOINT_MODEM_COMMAND_TX , + .toward_ipa =YES , + .endpoint = { + .seq_type =IPA_SEQ_PKT_PROCESS_NO_DEC_UCP , + }, + }, + { + .ee_id =GSI_EE_MODEM , + .channel_id =0 , + .endpoint_id =IPA_ENDPOINT_MODEM_LAN_TX , + .toward_ipa =YES , + .endpoint = { + .support_flt =YES , + }, + }, + { + .ee_id =GSI_EE_MODEM , + .channel_id =3 , + .endpoint_id =IPA_ENDPOINT_MODEM_LAN_RX , + .toward_ipa =NO , + }, + { + .ee_id =GSI_EE_MODEM , + .channel_id =4 , + .endpoint_id =IPA_ENDPOINT_MODEM_AP_TX , + .toward_ipa =YES , + .endpoint = { + .support_flt =YES , + }, + }, + { + .ee_id =GSI_EE_MODEM , + .channel_id =2 , + .endpoint_id =IPA_ENDPOINT_MODEM_AP_RX , + .toward_ipa =NO , + }, +}; + +static const struct ipa_resource_src ipa_resource_src[] = { + { + .type =IPA_RESOURCE_TYPE_SRC_PKT_CONTEXTS , + .limits[IPA_RESOURCE_GROUP_LWA_DL ] = { + .min =1 , + .max =63 , + }, + .limits[IPA_RESOURCE_GROUP_UL_DL ] = { + .min =1 , + .max =63 , + }, + }, + { + .type =IPA_RESOURCE_TYPE_SRC_DESCRIPTOR_LISTS , + .limits[IPA_RESOURCE_GROUP_LWA_DL ] = { + .min =10 , + .max =10 , + }, + .limits[IPA_RESOURCE_GROUP_UL_DL ] = { + .min =10 , + .max =10 , + }, + }, + { + .type =IPA_RESOURCE_TYPE_SRC_DESCRIPTOR_BUFF , + .limits[IPA_RESOURCE_GROUP_LWA_DL ] = { + .min =12 , + .max =12 , + }, + .limits[IPA_RESOURCE_GROUP_UL_DL ] = { + .min =14 , + .max =14 , + }, + }, + { + .type =IPA_RESOURCE_TYPE_SRC_HPS_DMARS , + .limits[IPA_RESOURCE_GROUP_LWA_DL ] = { + .min =0 , + .max =63 , + }, + .limits[IPA_RESOURCE_GROUP_UL_DL ] = { + .min =0 , + .max =63 , + }, + }, + { + .type =IPA_RESOURCE_TYPE_SRC_ACK_ENTRIES , + .limits[IPA_RESOURCE_GROUP_LWA_DL ] = { + .min =14 , + .max =14 , + }, + .limits[IPA_RESOURCE_GROUP_UL_DL ] = { + .min =20 , + .max =20 , + }, + }, +}; + +static const struct ipa_resource_dst ipa_resource_dst[] = { + { + .type =IPA_RESOURCE_TYPE_DST_DATA_SECTORS , + .limits[IPA_RESOURCE_GROUP_LWA_DL ] = { + .min =4 , + .max =4 , + }, + .limits[IPA_RESOURCE_GROUP_UL_DL ] = { + .min =4 , + .max =4 , + }, + }, + { + .type =IPA_RESOURCE_TYPE_DST_DPS_DMARS , + .limits[IPA_RESOURCE_GROUP_LWA_DL ] = { + .min =2 , + .max =63 , + }, + .limits[IPA_RESOURCE_GROUP_UL_DL ] = { + .min =1 , + .max =63 , + }, + }, +}; + +/* Resource configuration for the SDM845 SoC. */ +static const struct ipa_resource_data ipa_resource_data = { + .resource_src = ipa_resource_src, + .resource_src_count =ARRAY_SIZE (ipa_resource_src), + .resource_dst = ipa_resource_dst, + .resource_dst_count =ARRAY_SIZE (ipa_resource_dst), +}; + +/* Configuration data for the SDM845 SoC. */ +const struct ipa_data ipa_data_sdm845 = { + .endpoint_data = gsi_ipa_endpoint_data, + .endpoint_data_count =ARRAY_SIZE (gsi_ipa_endpoint_data), + .resource_data = &ipa_resource_data, +};
drivers/net/ipa/ipa_data.h new file +267 -0
267 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 6/18 soc: qcom: ipa: clocking, interrupts, and memory
In reply to Alex ElderThis patch incorporates three source files (and their headers). They're grouped into one patch mainly for the purpose of making the number and size of patches in this series somewhat reasonable.
- "ipa_clock.c" and "ipa_clock.h" implement clocking for the IPA device. The IPA has a single core clock managed by the common clock framework. In addition, the IPA has three buses whose bandwidth is managed by the Linux interconnect framework. At this time the core clock and all three buses are either on or off; we don't yet do any more fine-grained management than that. The core clock and interconnects are enabled and disabled as a unit, using a unified clock-like abstraction, ipa_clock_get()/ipa_clock_put().
- "ipa_interrupt.c" and "ipa_interrupt.h" implement IPA interrupts. There are two hardware IRQs used by the IPA driver (the other is the GSI interrupt, described in a separate patch). Several types of interrupt are handled by the IPA IRQ handler; these are not part of data/fast path.
- The IPA has a region of local memory that is accessible by the AP (and modem). Within that region are areas with certain defined purposes. "ipa_mem.c" and "ipa_mem.h" define those regions, and implement their initialization.
--- drivers/net/ipa/ipa_clock.c | 291 ++++++++++++++++++++++++++++++++ drivers/net/ipa/ipa_clock.h | 52 ++++++ drivers/net/ipa/ipa_interrupt.c | 279 ++++++++++++++++++++++++++++++ drivers/net/ipa/ipa_interrupt.h | 53 ++++++ drivers/net/ipa/ipa_mem.c | 237 ++++++++++++++++++++++++++ drivers/net/ipa/ipa_mem.h | 82 +++++++++ 6 files changed, 994 insertions(+) create mode 100644 drivers/net/ipa/ipa_clock.c create mode 100644 drivers/net/ipa/ipa_clock.h create mode 100644 drivers/net/ipa/ipa_interrupt.c create mode 100644 drivers/net/ipa/ipa_interrupt.h create mode 100644 drivers/net/ipa/ipa_mem.c create mode 100644 drivers/net/ipa/ipa_mem.h
drivers/net/ipa/ipa_clock.c new file +291 -0
@@ -0,0 +1,291 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ + +#include <linux/atomic.h> +#include <linux/mutex.h> +#include <linux/clk.h> +#include <linux/device.h> +#include <linux/interconnect.h> + +#include "ipa.h" +#include "ipa_clock.h" +#include "ipa_netdev.h" + +/** +* DOC: IPA Clocking +* +* The "IPA Clock" manages both the IPA core clock and the interconnects +* (buses) the IPA depends on as a single logical entity. A reference count +* is incremented by "get" operations and decremented by "put" operations. +* Transitions of that count from 0 to 1 result in the clock and interconnects +* being enabled, and transitions of the count from 1 to 0 cause them to be +* disabled. We currently operate the core clock at a fixed clock rate, and +* all buses at a fixed average and peak bandwidth. As more advanced IPA +* features are enabled, we can will better use of clock and bus scaling. +* +* An IPA clock reference must be held for any access to IPA hardware. +*/ + +#define IPA_CORE_CLOCK_RATE (75UL * 1000 * 1000)/* Hz */ + +/* Interconnect path bandwidths (each times 1000 bytes per second) */ +#define IPA_MEMORY_AVG (80 * 1000)/* 80 MBps */ +#define IPA_MEMORY_PEAK (600 * 1000) + +#define IPA_IMEM_AVG (80 * 1000) +#define IPA_IMEM_PEAK (350 * 1000) + +#define IPA_CONFIG_AVG (40 * 1000) +#define IPA_CONFIG_PEAK (40 * 1000) + +/** +* struct ipa_clock - IPA clocking information +* @core: IPA core clock +* @memory_path: Memory interconnect +* @imem_path: Internal memory interconnect +* @config_path: Configuration space interconnect +* @mutex; Protects clock enable/disable +* @count: Clocking reference count +*/ +struct ipa_clock { +struct ipa *ipa; +atomic_t count; +struct mutex mutex;/* protects clock enable/disable */ +struct clk *core; +struct icc_path *memory_path; +struct icc_path *imem_path; +struct icc_path *config_path; +}; + +/* Initialize interconnects required for IPA operation */ +static int ipa_interconnect_init (struct ipa_clock *clock,struct device *dev) +{ +struct icc_path *path; + + path =of_icc_get (dev,"memory" ); +if (IS_ERR (path)) + goto err_return; + clock->memory_path = path; + + path =of_icc_get (dev,"imem" ); +if (IS_ERR (path)) + goto err_memory_path_put; + clock->imem_path = path; + + path =of_icc_get (dev,"config" ); +if (IS_ERR (path)) + goto err_imem_path_put; + clock->config_path = path; + +return 0 ; + +err_imem_path_put: +icc_put (clock->imem_path); +err_memory_path_put: +icc_put (clock->memory_path); +err_return: + +return PTR_ERR (path); +} + +/* Inverse of ipa_interconnect_init() */ +static void ipa_interconnect_exit (struct ipa_clock *clock) +{ +icc_put (clock->config_path); +icc_put (clock->imem_path); +icc_put (clock->memory_path); +} + +/* Currently we only use one bandwidth level, so just "enable" interconnects */ +static int ipa_interconnect_enable (struct ipa_clock *clock) +{ +int ret; + + ret =icc_set_bw (clock->memory_path,IPA_MEMORY_AVG ,IPA_MEMORY_PEAK ); +if (ret) +return ret; + + ret =icc_set_bw (clock->imem_path,IPA_IMEM_AVG ,IPA_IMEM_PEAK ); +if (ret) + goto err_disable_memory_path; + + ret =icc_set_bw (clock->config_path,IPA_CONFIG_AVG ,IPA_CONFIG_PEAK ); +if (ret) + goto err_disable_imem_path; + +return 0 ; + +err_disable_imem_path: + (void )icc_set_bw (clock->imem_path,0 ,0 ); +err_disable_memory_path: + (void )icc_set_bw (clock->memory_path,0 ,0 ); + +return ret; +} + +/* To disable an interconnect, we just its bandwidth to 0 */ +static int ipa_interconnect_disable (struct ipa_clock *clock) +{ +int ret; + + ret =icc_set_bw (clock->memory_path,0 ,0 ); +if (ret) +return ret; + + ret =icc_set_bw (clock->imem_path,0 ,0 ); +if (ret) + goto err_reenable_memory_path; + + ret =icc_set_bw (clock->config_path,0 ,0 ); +if (ret) + goto err_reenable_imem_path; + +return 0 ; + +err_reenable_imem_path: + (void )icc_set_bw (clock->imem_path,IPA_IMEM_AVG ,IPA_IMEM_PEAK ); +err_reenable_memory_path: + (void )icc_set_bw (clock->memory_path,IPA_MEMORY_AVG ,IPA_MEMORY_PEAK ); + +return ret; +} + +/* Turn on IPA clocks, including interconnects */ +static int ipa_clock_enable (struct ipa_clock *clock) +{ +int ret; + + ret =ipa_interconnect_enable (clock); +if (ret) +return ret; + + ret =clk_prepare_enable (clock->core); +if (ret) +ipa_interconnect_disable (clock); + +return ret; +} + +/* Inverse of ipa_clock_enable() */ +static void ipa_clock_disable (struct ipa_clock *clock) +{ +clk_disable_unprepare (clock->core); + (void )ipa_interconnect_disable (clock); +} + +/* Get an IPA clock reference, but only if the reference count is +* already non-zero. Returns true if the additional reference was +* added successfully, or false otherwise. +*/ +bool ipa_clock_get_additional (struct ipa_clock *clock) +{ +return !!atomic_inc_not_zero (&clock->count); +} + +/* Get an IPA clock reference. If the reference count is non-zero, it is +* incremented and return is immediate. Otherwise it is checked again +* under protection of the mutex, and enable clocks and resume RX endpoints +* before returning. For the first reference, the count is intentionally +* not incremented until after these activities are complete. +*/ +void ipa_clock_get (struct ipa_clock *clock) +{ +/* If the clock is running, just bump the reference count */ +if (ipa_clock_get_additional (clock)) +return ; + +/* Otherwise get the mutex and check again */ +mutex_lock (&clock->mutex); + +/* A reference might have been added before we got the mutex. */ +if (!ipa_clock_get_additional (clock)) { +int ret; + + ret =ipa_clock_enable (clock); +if (!WARN (ret,"error %d enabling IPA clock\n" , ret)) { +struct ipa *ipa = clock->ipa; + +if (ipa->default_endpoint) +ipa_endpoint_resume (ipa->default_endpoint); + +if (ipa->modem_netdev) +ipa_netdev_resume (ipa->modem_netdev); + +atomic_inc (&clock->count); + } + } + +mutex_unlock (&clock->mutex); +} + +/* Attempt to remove an IPA clock reference. If this represents +* the last reference, suspend endpoints and disable the clock +* (and interconnects) under protection of a mutex. +*/ +void ipa_clock_put (struct ipa_clock *clock) +{ +/* If this is not the last reference there's nothing more to do */ +if (!atomic_dec_and_mutex_lock (&clock->count, &clock->mutex)) +return ; + +if (clock->ipa->modem_netdev) +ipa_netdev_suspend (clock->ipa->modem_netdev); + +if (clock->ipa->default_endpoint) +ipa_endpoint_suspend (clock->ipa->default_endpoint); + +ipa_clock_disable (clock); + +mutex_unlock (&clock->mutex); +} + +/* Initialize IPA clocking */ +struct ipa_clock *ipa_clock_init (struct ipa *ipa) +{ +struct device *dev = &ipa->pdev->dev; +struct ipa_clock *clock; +int ret; + + clock =kzalloc (sizeof (*clock),GFP_KERNEL ); +if (!clock) +return ERR_PTR (-ENOMEM ); + + clock->ipa = ipa; + clock->core =clk_get (dev,"core" ); +if (IS_ERR (clock->core)) { + ret =PTR_ERR (clock->core); + goto err_free_clock; + } + + ret =clk_set_rate (clock->core,IPA_CORE_CLOCK_RATE ); +if (ret) + goto err_clk_put; + + ret =ipa_interconnect_init (clock, dev); +if (ret) + goto err_clk_put; + +mutex_init (&clock->mutex); +atomic_set (&clock->count,0 ); + +return clock; + +err_clk_put: +clk_put (clock->core); +err_free_clock: +kfree (clock); + +return ERR_PTR (ret); +} + +/* Inverse of ipa_clock_init() */ +void ipa_clock_exit (struct ipa_clock *clock) +{ +mutex_destroy (&clock->mutex); +ipa_interconnect_exit (clock); +clk_put (clock->core); +kfree (clock); +}
drivers/net/ipa/ipa_clock.h new file +52 -0
52 lines not on this page · raw message
drivers/net/ipa/ipa_interrupt.c new file +279 -0
279 lines not on this page · raw message
drivers/net/ipa/ipa_interrupt.h new file +53 -0
53 lines not on this page · raw message
drivers/net/ipa/ipa_mem.c new file +237 -0
237 lines not on this page · raw message
drivers/net/ipa/ipa_mem.h new file +82 -0
82 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 7/18 soc: qcom: ipa: GSI headers
In reply to Alex ElderThe Generic Software Interface is a layer of the IPA driver that abstracts the underlying hardware. The next patch includes the main code for GSI (including some additional documentation). This patch just includes three GSI header files.
- "gsi.h" is the top-level GSI header file. There is one of these associated with the IPA structure; in fact, it is embedded within the IPA structure. (Were it not embedded this way, many of the definitions structures defined here could be private to GSI code.) The main abstraction implemented by the GSI code is the channel, and this header exposes several operations that can be performed on a GSI channel.
- "gsi_private.h" exposes some definitions that are intended to be private, used only by the main GSI code and the GSI transaction code (defined in an upcoming patch).
- Like "ipa_reg.h", "gsi_reg.h" defines the offsets of the 32-bit registers used by the GSI layer, along with masks that define the position and width of fields less than 32 bits located within these registers.
--- drivers/net/ipa/gsi.h | 241 ++++++++++++++++++++++ drivers/net/ipa/gsi_private.h | 147 +++++++++++++ drivers/net/ipa/gsi_reg.h | 376 ++++++++++++++++++++++++++++++++++ 3 files changed, 764 insertions(+) create mode 100644 drivers/net/ipa/gsi.h create mode 100644 drivers/net/ipa/gsi_private.h create mode 100644 drivers/net/ipa/gsi_reg.h
drivers/net/ipa/gsi.h new file +241 -0
@@ -0,0 +1,241 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* Copyright (c) 2015-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ +#ifndef _GSI_H_ +#define _GSI_H_ + +#include <linux/types.h> +#include <linux/spinlock.h> +#include <linux/mutex.h> +#include <linux/completion.h> +#include <linux/platform_device.h> +#include <linux/netdevice.h> + +#define GSI_CHANNEL_MAX 14 +#define GSI_EVT_RING_MAX 10 + +struct device ; +struct scatterlist ; +struct platform_device ; + +struct gsi ; +struct gsi_trans ; +struct gsi_channel_data ; +struct gsi_ipa_endpoint_data ; + +/* Execution environment IDs */ +enum gsi_ee_id { +GSI_EE_AP =0 , +GSI_EE_MODEM =1 , +GSI_EE_UC =2 , +GSI_EE_TZ =3 , +}; + +/* Channel operation statistics, aggregated across all channels */ +struct gsi_channel_stats { +u64 allocate; +u64 start; +u64 stop; +u64 reset; +u64 free; +}; + +struct gsi_ring { +spinlock_t spinlock;/* protects wp, rp updates */ +void *virt; +dma_addr_t addr; +size_t size; +u32 base;/* low 32 bits of addr */ +u32 wp; +u32 wp_local; +u32 rp_local; +u32 end;/* offset past last element */ +}; + +struct gsi_trans_info { +struct gsi_trans **map;/* TRE -> transaction map */ +u32 pool_count;/* # transactions in the pool */ +struct gsi_trans *pool;/* transaction allocation pool */ +u32 pool_free;/* next free transaction in pool */ +u32 sg_pool_count;/* # SGs in the allocation pool */ +struct scatterlist *sg_pool;/* SG allocation pool */ +u32 sg_pool_free;/* next free SG pool entry */ + +spinlock_t spinlock;/* protects updates to the rest */ +u32 tre_avail;/* unallocated TREs in ring */ +struct list_head alloc;/* allocated, not committed */ +struct list_head pending;/* committed, awaiting completion */ +struct list_head complete;/* completed, awaiting poll */ +struct list_head polled;/* returned by gsi_channel_poll_one() */ +}; + +/* Hardware values signifying the state of a channel */ +enum gsi_channel_state { +GSI_CHANNEL_STATE_NOT_ALLOCATED =0x0 , +GSI_CHANNEL_STATE_ALLOCATED =0x1 , +GSI_CHANNEL_STATE_STARTED =0x2 , +GSI_CHANNEL_STATE_STOPPED =0x3 , +GSI_CHANNEL_STATE_STOP_IN_PROC =0x4 , +GSI_CHANNEL_STATE_ERROR =0xf , +}; + +/* We only care about channels between IPA and AP */ +struct gsi_channel { +struct gsi *gsi; +u32 toward_ipa;/* 0: IPA->AP; 1: AP->IPA */ + +const struct gsi_channel_data *data;/* initialization data */ + +struct completion completion;/* signals channel state changes */ +enum gsi_channel_state state; + +struct gsi_ring tre_ring; +u32 evt_ring_id; + +u64 byte_count;/* total # bytes transferred */ +u64 trans_count;/* total # transactions */ +u64 doorbell_byte_count;/* TX byte_count at last doorbell */ +u64 doorbell_trans_count;/* TX trans_count at last doorbell */ + +struct gsi_trans_info trans_info; + +struct napi_struct napi; +}; + +/* Hardware values signifying the state of an event ring */ +enum gsi_evt_ring_state { +GSI_EVT_RING_STATE_NOT_ALLOCATED =0x0 , +GSI_EVT_RING_STATE_ALLOCATED =0x1 , +GSI_EVT_RING_STATE_ERROR =0xf , +}; + +struct gsi_evt_ring { +struct gsi_channel *channel; +struct completion completion;/* signals event ring state changes */ +enum gsi_evt_ring_state state; +struct gsi_ring ring; +}; + +struct gsi { +struct device *dev;/* Same as IPA device */ +struct net_device dummy_dev;/* needed for NAPI */ +void __iomem *virt; +u32 irq; +u32 irq_wake_enabled;/* 1: irq wake was enabled */ +struct gsi_channel channel[GSI_CHANNEL_MAX ]; +struct gsi_channel_stats channel_stats; +struct gsi_evt_ring evt_ring[GSI_EVT_RING_MAX ]; +u32 event_bitmap; +u32 event_enable_bitmap; +spinlock_t spinlock;/* global register updates */ +struct mutex mutex;/* protects commands, programming */ +}; + +/** +* gsi_setup() - Set up the GSI subsystem +* @gsi: Address of GSI structure embedded in an IPA structure +* +* @Return: 0 if successful, or a negative error code +* +* Performs initialization that must wait until the GSI hardware is +* ready (including firmware loaded). +*/ +int gsi_setup (struct gsi *gsi); + +/** +* gsi_teardown() - Tear down GSI subsystem +* @gsi: GSI address previously passed to a successful gsi_setup() call +*/ +void gsi_teardown (struct gsi *gsi); + +/** +* gsi_channel_trans_max() - Channel maximum number of transactions +* @gsi: GSI pointer +* @channel_id: Channel whose limit is to be returned +* +* @Return: The maximum number of pending transactions on the channel +*/ +u32 gsi_channel_trans_max (struct gsi *gsi,u32 channel_id); + +/** +* gsi_channel_trans_tre_max() - Return the maximum TREs per transaction +* @gsi: GSI pointer +* @channel_id: Channel whose limit is to be returned +* +* @Return: The maximum TRE count per transaction on the channel +*/ +u32 gsi_channel_trans_tre_max (struct gsi *gsi,u32 channel_id); + +/** +* gsi_channel_trans_quiesce() - Wait for channel transactions to complete +* @gsi: GSI pointer +* @channel_id: Channel to quiesce +* +* Wait for all of a channel's currently-allocated transactions to +* be committed, complete, and be freed. +* +* NOTE: Assumes no new transactions will be issued before it returns. +*/ +void gsi_channel_trans_quiesce (struct gsi *gsi,u32 channel_id); + +/** +* gsi_channel_start() - Make a GSI channel operational +* @gsi: GSI pointer +* @channel_id: Channel to start +* +* @Return: 0 if successful, or a negative error code +*/ +int gsi_channel_start (struct gsi *gsi,u32 channel_id); + +/** +* gsi_channel_stop() - Stop an operational GSI channel +* @gsi: GSI pointer returned by gsi_setup() +* @channel_id: Channel to stop +* +* @Return: 0 if successful, or a negative error code +*/ +int gsi_channel_stop (struct gsi *gsi,u32 channel_id); + +/** +* gsi_channel_reset() - Reset a GSI channel +* @gsi: GSI pointer +* @channel_id: Channel to be reset +* +* @Return: 0 if successful, or a negative error code +* +* GSI hardware relinquishes ownership of all pending receive buffer +* transactions as a result of reset. They will be completed with +* result code -ECANCELED. +*/ +int gsi_channel_reset (struct gsi *gsi,u32 channel_id); + +/** +* gsi_channel_config() - Configure a GSI channel +* @gsi: GSI pointer +* @channel_id: Channel to be configured +* @doorbell_enable: Whether to enable hardware doorbell engine +*/ +void gsi_channel_config (struct gsi *gsi,u32 channel_id,bool doorbell_enable); + +/** +* gsi_init() - Initialize the GSI subsystem +* @gsi: Address of GSI structure embedded in an IPA structure +* @pdev: IPA platform device +* +* @Return: 0 if successful, or a negative error code +* +* Early stage initialization of the GSI subsystem, performing tasks +* that can be done before the GSI hardware is ready to use. +*/ +int gsi_init (struct gsi *gsi,struct platform_device *pdev,u32 data_count, +const struct gsi_ipa_endpoint_data *data); + +/** +* gsi_exit() - Exit the GSI subsystem +* @gsi: GSI address previously passed to a successful gsi_init() call +*/ +void gsi_exit (struct gsi *gsi); + +#endif /* _GSI_H_ */
drivers/net/ipa/gsi_private.h new file +147 -0
147 lines not on this page · raw message
drivers/net/ipa/gsi_reg.h new file +376 -0
376 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 8/18 soc: qcom: ipa: the generic software interface
In reply to Alex ElderThis patch includes "gsi.c", which implements the generic software interface (GSI) for IPA. The generic software interface abstracts channels, which provide a means of transferring data either from the AP to the IPA, or from the IPA to the AP. A ring buffer of "transfer elements" (TREs) is used to describe data transfers to perform. The AP writes a doorbell register associated with a channel to let it know it has added new entries (for an AP->IPA channel) or has finished processing entries (for an IPA->AP channel).
Each channel also has an event ring buffer, used by the IPA to communicate information about events related to a channel (for example, the completion of TREs). The IPA writes its own doorbell register, which triggers an interrupt on the AP, to signal that new event information has arrived.
--- drivers/net/ipa/gsi.c | 1741 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 1741 insertions(+) create mode 100644 drivers/net/ipa/gsi.c
drivers/net/ipa/gsi.c new file +1741 -0
@@ -0,0 +1,1741 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (c) 2015-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ + +#include <linux/types.h> +#include <linux/bits.h> +#include <linux/bitfield.h> +#include <linux/spinlock.h> +#include <linux/mutex.h> +#include <linux/completion.h> +#include <linux/io.h> +#include <linux/bug.h> +#include <linux/interrupt.h> +#include <linux/platform_device.h> +#include <linux/netdevice.h> + +#include "gsi.h" +#include "gsi_reg.h" +#include "gsi_private.h" +#include "gsi_trans.h" +#include "ipa_gsi.h" +#include "ipa_data.h" + +/** +* DOC: The IPA Generic Software Interface +* +* The generic software interface (GSI) is an integral component of the IPA, +* providing a well-defined communication layer between the AP subsystem +* and the IPA core. The modem uses the GSI layer as well. +* +* -------- --------- +* | | | | +* | AP +<---. .----+ Modem | +* | +--. | | .->+ | +* | | | | | | | | +* -------- | | | | --------- +* v | v | +* --+-+---+-+-- +* | GSI | +* |-----------| +* | | +* | IPA | +* | | +* ------------- +* +* In the above diagram, the AP and Modem represent "execution environments" +* (EEs), which are independent operating environments that use the IPA for +* data transfer. +* +* Each EE uses a set of unidirectional GSI "channels," which allow transfer +* of data to or from the IPA. A channel is implemented as a ring buffer, +* with a DRAM-resident array of "transfer elements" (TREs) available to +* describe transfers to or from other EEs through the IPA. A transfer +* element can also contain an immediate command, requesting the IPA perform +* actions other than data transfer. +* +* Each TRE refers to a block of data--also located DRAM. After writing one +* or more TREs to a channel, the writer (either the IPA or an EE) writes a +* doorbell register to inform the receiving side how many elements have +* been written. Writing to a doorbell register triggers an interrupt on +* the receiver. +* +* Each channel has a GSI "event ring" associated with it. An event ring +* is implemented very much like a channel ring, but is always directed from +* the IPA to an EE. The IPA notifies an EE (such as the AP) about channel +* events by adding an entry to the event ring associated with the channel; +* when it writes the event ring's doorbell register the EE is interrupted. +* Each entry in an event ring contains a pointer to the channel TRE whose +* completion the event represents. +* +* Each TRE in a channel ring has a set of flags. One flag indicates whether +* the completion of the transfer operation generates an entry (and possibly +* an interrupt) in the channel's event ring. Oother flags allow transfer +* elements to be chained together, forming a single logical transaction. +* TRE flags are used to control whether and when interrupts are generated +* to signal completion of channel transfers. +* +* Elements in channel and event rings are completed (or consumed) strictly +* in order. Completion of one entry implies the completion of all preceding +* entries. A single completion interrupt can communicate the completion of +* many transfers. +* +* Note that all GSI registers are little-endian, which is the assumed +* endianness of I/O space accesses. The accessor functions perform byte +* swapping if needed (i.e., for a big endian CPU). +*/ + +/* Delay period for interrupt moderation (in 32KHz IPA timer ticks) */ +#define IPA_GSI_EVT_RING_INT_MODT (32 * 1)/* 1ms under 32KHz clock */ + +#define GSI_CMD_TIMEOUT 5/* seconds */ + +#define GSI_MHI_ER_START 10/* First reserved event number */ +#define GSI_MHI_ER_END 16/* Last reserved event number */ + +#define GSI_RESET_WA_MIN_SLEEP 1000/* microseconds */ +#define GSI_RESET_WA_MAX_SLEEP 2000/* microseconds */ + +#define GSI_ISR_MAX_ITER 50 + +/* Hardware values from the error log register error code field */ +enum gsi_err_code { +GSI_INVALID_TRE_ERR =0x1 , +GSI_OUT_OF_BUFFERS_ERR =0x2 , +GSI_OUT_OF_RESOURCES_ERR =0x3 , +GSI_UNSUPPORTED_INTER_EE_OP_ERR =0x4 , +GSI_EVT_RING_EMPTY_ERR =0x5 , +GSI_NON_ALLOCATED_EVT_ACCESS_ERR =0x6 , +GSI_HWO_1_ERR =0x8 , +}; + +/* Hardware values from the error log register error type field */ +enum gsi_err_type { +GSI_ERR_TYPE_GLOB =0x1 , +GSI_ERR_TYPE_CHAN =0x2 , +GSI_ERR_TYPE_EVT =0x3 , +}; + +/* Fields in an error log register at GSI_ERROR_LOG_OFFSET */ +#define GSI_LOG_ERR_ARG3_FMASK GENMASK(3, 0) +#define GSI_LOG_ERR_ARG2_FMASK GENMASK(7, 4) +#define GSI_LOG_ERR_ARG1_FMASK GENMASK(11, 8) +#define GSI_LOG_ERR_CODE_FMASK GENMASK(15, 12) +#define GSI_LOG_ERR_VIRT_IDX_FMASK GENMASK(23, 19) +#define GSI_LOG_ERR_TYPE_FMASK GENMASK(27, 24) +#define GSI_LOG_ERR_EE_FMASK GENMASK(31, 28) + +/* Hardware values used when programming an event ring */ +enum gsi_evt_chtype { +GSI_EVT_CHTYPE_MHI_EV =0x0 , +GSI_EVT_CHTYPE_XHCI_EV =0x1 , +GSI_EVT_CHTYPE_GPI_EV =0x2 , +GSI_EVT_CHTYPE_XDCI_EV =0x3 , +}; + +/* Hardware values used when programming a channel */ +enum gsi_channel_protocol { +GSI_CHANNEL_PROTOCOL_MHI =0x0 , +GSI_CHANNEL_PROTOCOL_XHCI =0x1 , +GSI_CHANNEL_PROTOCOL_GPI =0x2 , +GSI_CHANNEL_PROTOCOL_XDCI =0x3 , +}; + +/* Hardware values representing an event ring immediate command opcode */ +enum gsi_evt_ch_cmd_opcode { +GSI_EVT_ALLOCATE =0x0 , +GSI_EVT_RESET =0x9 , +GSI_EVT_DE_ALLOC =0xa , +}; + +/* Hardware values representing a channel immediate command opcode */ +enum gsi_ch_cmd_opcode { +GSI_CH_ALLOCATE =0x0 , +GSI_CH_START =0x1 , +GSI_CH_STOP =0x2 , +GSI_CH_RESET =0x9 , +GSI_CH_DE_ALLOC =0xa , +GSI_CH_DB_STOP =0xb , +}; + +/** gsi_gpi_channel_scratch - GPI protocol scratch register +* +* @max_outstanding_tre: +* Defines the maximum number of TREs allowed in a single transaction +* on a channel (in Bytes). This determines the amount of prefetch +* performed by the hardware. We configure this to equal the size of +* the TLV FIFO for the channel. +* @outstanding_threshold: +* Defines the threshold (in Bytes) determining when the sequencer +* should update the channel doorbell. We configure this to equal +* the size of two TREs. +*/ +struct gsi_gpi_channel_scratch { +u64 rsvd1; +u16 rsvd2; +u16 max_outstanding_tre; +u16 rsvd3; +u16 outstanding_threshold; +} __packed; + +/** gsi_channel_scratch - channel scratch configuration area +* +* The exact interpretation of this register is protocol-specific. +* We only use GPI channels; see struct gsi_gpi_channel_scratch, above. +*/ +union gsi_channel_scratch { +struct gsi_gpi_channel_scratch gpi; +struct { +u32 word1; +u32 word2; +u32 word3; +u32 word4; + } data; +} __packed; + +/* Enable or disable an event interrupt */ +static void +_gsi_irq_control_event (struct gsi *gsi,u32 evt_ring_id,bool enable) +{ +u32 mask =BIT (evt_ring_id); +u32 val; + +if (enable) + gsi->event_enable_bitmap |= mask; +else + gsi->event_enable_bitmap &= ~mask; + + val = gsi->event_enable_bitmap; +iowrite32 (val, gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET ); +} + +static void gsi_irq_enable_event (struct gsi *gsi,u32 evt_ring_id) +{ +_gsi_irq_control_event (gsi, evt_ring_id, true); +} + +static void gsi_irq_disable_event (struct gsi *gsi,u32 evt_ring_id) +{ +_gsi_irq_control_event (gsi, evt_ring_id, false); +} + +/* Enable or disable all interrupt types */ +static void _gsi_irq_control_all (struct gsi *gsi,bool enable) +{ +u32 val; + +/* Inter EE commands / interrupt are no supported. */ + val = enable ?GSI_CNTXT_TYPE_IRQ_MSK_ALL :0 ; +iowrite32 (val, gsi->virt +GSI_CNTXT_TYPE_IRQ_MSK_OFFSET ); + + val = enable ?GENMASK (GSI_CHANNEL_MAX -1 ,0 ) :0 ; +iowrite32 (val, gsi->virt +GSI_CNTXT_SRC_CH_IRQ_MSK_OFFSET ); + + val = enable ?GENMASK (GSI_EVT_RING_MAX -1 ,0 ) :0 ; +iowrite32 (val, gsi->virt +GSI_CNTXT_SRC_EV_CH_IRQ_MSK_OFFSET ); + +/* IEOB interrupts are managed individually */ + val = enable ? gsi->event_enable_bitmap :0 ; +iowrite32 (val, gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET ); + + val = enable ?GSI_CNTXT_GLOB_IRQ_ALL :0 ; +iowrite32 (val, gsi->virt +GSI_CNTXT_GLOB_IRQ_EN_OFFSET ); + +/* Never enable GSI_BREAK_POINT */ + val = enable ?GSI_CNTXT_GSI_IRQ_ALL & ~EN_BREAK_POINT_FMASK :0 ; +iowrite32 (val, gsi->virt +GSI_CNTXT_GSI_IRQ_EN_OFFSET ); +} + +static void gsi_irq_disable_all (struct gsi *gsi) +{ +_gsi_irq_control_all (gsi, false); +} + +static void gsi_irq_enable_all (struct gsi *gsi) +{ +_gsi_irq_control_all (gsi, true); +} + +/* Return the channel id associated with a given channel */ +u32 gsi_channel_id (struct gsi_channel *channel) +{ +return channel - &channel->gsi->channel[0 ]; +} + +/* Return the hardware's notion of the current state of a channel */ +static enum gsi_channel_state gsi_channel_state (struct gsi_channel *channel) +{ +u32 channel_id =gsi_channel_id (channel); +struct gsi *gsi = channel->gsi; +u32 val; + + val =ioread32 (gsi->virt +GSI_CH_C_CNTXT_0_OFFSET (channel_id)); + +return u32_get_bits (val,CHSTATE_FMASK ); +} + +/* Return the hardware's notion of the current state of an event ring */ +static enum gsi_evt_ring_state +gsi_evt_ring_state (struct gsi *gsi,u32 evt_ring_id) +{ +u32 val =ioread32 (gsi->virt +GSI_EV_CH_E_CNTXT_0_OFFSET (evt_ring_id)); + +return u32_get_bits (val,EV_CHSTATE_FMASK ); +} + +/* Channel control interrupt handler */ +static void gsi_isr_chan_ctrl (struct gsi *gsi) +{ +u32 channel_mask; + + channel_mask =ioread32 (gsi->virt +GSI_CNTXT_SRC_CH_IRQ_OFFSET ); +iowrite32 (channel_mask, gsi->virt +GSI_CNTXT_SRC_CH_IRQ_CLR_OFFSET ); + +while (channel_mask) { +u32 channel_id =__ffs (channel_mask); +struct gsi_channel *channel; + + channel_mask ^=BIT (channel_id); + + channel = &gsi->channel[channel_id]; + channel->state =gsi_channel_state (channel); + +complete (&channel->completion); + } +} + +static void gsi_isr_evt_ctrl (struct gsi *gsi) +{ +u32 evt_mask; + + evt_mask =ioread32 (gsi->virt +GSI_CNTXT_SRC_EV_CH_IRQ_OFFSET ); +iowrite32 (evt_mask, gsi->virt +GSI_CNTXT_SRC_EV_CH_IRQ_CLR_OFFSET ); + +while (evt_mask) { +u32 evt_ring_id =__ffs (evt_mask); +struct gsi_evt_ring *evt_ring; + + evt_mask ^=BIT (evt_ring_id); + + evt_ring = &gsi->evt_ring[evt_ring_id]; + evt_ring->state =gsi_evt_ring_state (gsi, evt_ring_id); + +complete (&evt_ring->completion); + } +} + +static void +gsi_isr_glob_chan_err (struct gsi *gsi,u32 err_ee,u32 channel_id,u32 code) +{ +if (code ==GSI_OUT_OF_RESOURCES_ERR ) { +dev_err (gsi->dev,"channel %u out of resources\n" , channel_id); +complete (&gsi->channel[channel_id].completion); +return ; + } + +/* Report, but otherwise ignore all other error codes */ +WARN (true,"channel %u global error ee 0x%08x code 0x%08x\n" , + channel_id, err_ee, code); +} + +static void +gsi_isr_glob_evt_err (struct gsi *gsi,u32 err_ee,u32 evt_ring_id,u32 code) +{ +if (code ==GSI_OUT_OF_RESOURCES_ERR ) { +struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id]; +u32 channel_id =gsi_channel_id (evt_ring->channel); + +complete (&evt_ring->completion); +dev_err (gsi->dev,"evt_ring for channel %u out of resources\n" , + channel_id); +return ; + } + +/* Report, but otherwise ignore all other error codes */ +WARN (true,"event ring 0x%08x global error ee %u code 0x%08x\n" , + evt_ring_id, err_ee, code); +} + +static void gsi_isr_glob_err (struct gsi *gsi) +{ +enum gsi_err_type type; +enum gsi_err_code code; +u32 which; +u32 val; +u32 ee; + +/* Get the logged error, then reinitialize the log */ + val =ioread32 (gsi->virt +GSI_ERROR_LOG_OFFSET ); +iowrite32 (0 , gsi->virt +GSI_ERROR_LOG_OFFSET ); +iowrite32 (~0 , gsi->virt +GSI_ERROR_LOG_CLR_OFFSET ); + + ee =u32_get_bits (val,GSI_LOG_ERR_EE_FMASK ); + which =u32_get_bits (val,GSI_LOG_ERR_VIRT_IDX_FMASK ); + type =u32_get_bits (val,GSI_LOG_ERR_TYPE_FMASK ); + code =u32_get_bits (val,GSI_LOG_ERR_CODE_FMASK ); + +if (type ==GSI_ERR_TYPE_CHAN ) +gsi_isr_glob_chan_err (gsi, ee, which, code); +else if (type ==GSI_ERR_TYPE_EVT ) +gsi_isr_glob_evt_err (gsi, ee, which, code); +else /* type GSI_ERR_TYPE_GLOB should be fatal */ +WARN (true,"unexpected global error 0x%08x\n" , type); +} + +static void gsi_isr_glob_ee (struct gsi *gsi) +{ +u32 val; + + val =ioread32 (gsi->virt +GSI_CNTXT_GLOB_IRQ_STTS_OFFSET ); + +if (val &ERROR_INT_FMASK ) +gsi_isr_glob_err (gsi); + +iowrite32 (val, gsi->virt +GSI_CNTXT_GLOB_IRQ_CLR_OFFSET ); + + val ^=ERROR_INT_FMASK ; + +if (val &EN_GP_INT1_FMASK ) +dev_err (gsi->dev,"unexpected global INT1\n" ); + val ^=EN_GP_INT1_FMASK ; + +WARN (val,"unexpected global interrupt 0x%08x\n" , val); +} + +/* Returns true if the interrupt state (enabled or not) changed */ +static bool gsi_channel_intr (struct gsi_channel *channel,bool enable) +{ +u32 evt_ring_id = channel->evt_ring_id; +struct gsi *gsi = channel->gsi; +u32 mask =BIT (evt_ring_id); +unsigned long flags; +bool different; +u32 enabled; + +spin_lock_irqsave (&gsi->spinlock, flags); + + enabled = gsi->event_enable_bitmap & mask; + different = enable == !enabled; + +if (different) { +if (enabled) +gsi_irq_disable_event (channel->gsi, evt_ring_id); +else +gsi_irq_enable_event (channel->gsi, evt_ring_id); + } + +spin_unlock_irqrestore (&gsi->spinlock, flags); + +return different; +} + +/* This function is almost always called in interrupt context, +* meaning the interrupt is enabled. The request to disable +* the interrupt here will therefore "succeed", that is, it +* will disable an enabled interrupt. +* +* However, this function is also called when cancelling pending +* transactions, and when that occurs it's possible interrupts are +* already disabled. For that reason we only schedule NAPI if we +* actually caused interrupts to become disabled. +*/ +void gsi_event_handle (struct gsi *gsi,u32 evt_ring_id) +{ +struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id]; +struct gsi_channel *channel = evt_ring->channel; + +if (gsi_channel_intr (channel, false)) +napi_schedule (&channel->napi); +} + +static void gsi_isr_ieob (struct gsi *gsi) +{ +u32 evt_mask; + + evt_mask =ioread32 (gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_OFFSET ); + evt_mask &=ioread32 (gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET ); +iowrite32 (evt_mask, gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_CLR_OFFSET ); + +while (evt_mask) { +u32 evt_ring_id =__ffs (evt_mask); + + evt_mask ^=BIT (evt_ring_id); + +gsi_event_handle (gsi, evt_ring_id); + } +} + +static void gsi_isr_inter_ee_chan_ctrl (struct gsi *gsi) +{ +u32 channel_mask; + + channel_mask =ioread32 (gsi->virt +GSI_INTER_EE_SRC_CH_IRQ_OFFSET ); +iowrite32 (channel_mask, gsi->virt +GSI_INTER_EE_SRC_CH_IRQ_CLR_OFFSET ); + +while (channel_mask) { +u32 channel_id =__ffs (channel_mask); + +/* not currently expected */ +dev_err (gsi->dev,"ch %u inter-EE interrupt\n" , channel_id); + channel_mask ^=BIT (channel_id); + } +} + +static void gsi_isr_inter_ee_evt_ctrl (struct gsi *gsi) +{ +u32 evt_mask; + + evt_mask =ioread32 (gsi->virt +GSI_INTER_EE_SRC_EV_CH_IRQ_OFFSET ); +iowrite32 (evt_mask, gsi->virt +GSI_INTER_EE_SRC_EV_CH_IRQ_CLR_OFFSET ); + +while (evt_mask) { +u32 evt_ring_id =__ffs (evt_mask); + +/* not currently expected */ +dev_err (gsi->dev,"evt %u inter-EE interrupt\n" , evt_ring_id); + evt_mask ^=BIT (evt_ring_id); + } +} + +static void gsi_isr_general (struct gsi *gsi) +{ +u32 val; + + val =ioread32 (gsi->virt +GSI_CNTXT_GSI_IRQ_STTS_OFFSET ); +iowrite32 (val, gsi->virt +GSI_CNTXT_GSI_IRQ_CLR_OFFSET ); + +if (val &CLR_BREAK_POINT_FMASK ) +dev_err (gsi->dev,"breakpoint!\n" ); + val ^=CLR_BREAK_POINT_FMASK ; + +WARN (val,"unexpected general interrupt 0x%08x\n" , val); +} + +/** +* gsi_isr() - Top level GSI interrupt service routine +* @irq: Interrupt number (ignored) +* @dev_id: Device id pointer supplied to request_irq() +* +* This is the main handler function registered for the GSI IRQ. The +* GSI pointer is supplied as the "device id" value when the handler +* is registered, and is provided here. Each type of interrupt has a +* separate handler function that is called from here. +*/ +static irqreturn_t gsi_isr (int irq,void *dev_id) +{ +struct gsi *gsi = dev_id; +u32 intr_mask; +u32 cnt =0 ; + +while ((intr_mask =ioread32 (gsi->virt +GSI_CNTXT_TYPE_IRQ_OFFSET ))) { +/* intr_mask contains bitmask of pending GSI interrupts */ +do { +u32 gsi_intr =BIT (__ffs (intr_mask)); + + intr_mask ^= gsi_intr; + +switch (gsi_intr) { +case CH_CTRL_FMASK : +gsi_isr_chan_ctrl (gsi); +break ; +case EV_CTRL_FMASK : +gsi_isr_evt_ctrl (gsi); +break ; +case GLOB_EE_FMASK : +gsi_isr_glob_ee (gsi); +break ; +case IEOB_FMASK : +gsi_isr_ieob (gsi); +break ; +case INTER_EE_CH_CTRL_FMASK : +gsi_isr_inter_ee_chan_ctrl (gsi); +break ; +case INTER_EE_EV_CTRL_FMASK : +gsi_isr_inter_ee_evt_ctrl (gsi); +break ; +case GENERAL_FMASK : +gsi_isr_general (gsi); +break ; +default : +WARN (true,"%s: unrecognized type 0x%08x\n" , + __func__, gsi_intr); +break ; + } + }while (intr_mask); + +if (WARN (++cnt >GSI_ISR_MAX_ITER ,"interrupt flood\n" )) +break ; + } + +return IRQ_HANDLED ; +} + +/* Return the virtual address associated with a 32-bit ring offset */ +void *gsi_ring_virt (struct gsi_ring *ring,u32 offset) +{ +return ring->virt + (offset - ring->base); +} + +/* Return the ring index of a 32-bit ring offset */ +u32 ring_index (struct gsi_ring *ring,u32 offset) +{ +/* Code assumes channel and event ring elements are the same size */ +BUILD_BUG_ON (sizeof (struct gsi_tre ) != +sizeof (struct gsi_xfer_compl_evt )); + +return (offset - ring->base) /sizeof (struct gsi_tre ); +} + +/* Return the 32-bit ring offset that precedes the one at the given offset */ +static u32 ring_prev (struct gsi_ring *ring,u32 offset) +{ +if (offset == ring->base) + offset = ring->end; + +return offset -sizeof (struct gsi_tre ); +} + +/* Advance a ring's local write pointer by the given number of slots */ +void gsi_ring_wp_local_add (struct gsi_ring *ring,u32 val) +{ + ring->wp_local += val *sizeof (struct gsi_tre ); +if (ring->wp_local >= ring->end) + ring->wp_local -= ring->size; +} + +/* Advance a ring's local read pointer by the given number of slots */ +static void gsi_ring_rp_local_add (struct gsi_ring *ring,u32 val) +{ + ring->rp_local += val *sizeof (struct gsi_tre ); +if (ring->rp_local == ring->end) + ring->rp_local -= ring->size; +} + +static void __gsi_evt_tx_update (struct gsi_evt_ring *evt_ring,u32 rp) +{ +struct gsi_channel *channel = evt_ring->channel; +struct gsi_ring *ring = &evt_ring->ring; +struct gsi_xfer_compl_evt *evt; +struct gsi_trans *first_trans; +struct gsi_trans *last_trans; +u32 trans_count; +u32 byte_count; +u32 tre_offset; +u32 tre_index; + +/* Get the first (oldest) un-processed event */ + evt =gsi_ring_virt (ring, ring->rp_local); +/* Get the TRE offset from that, and its associated transaction */ + tre_offset =le64_to_cpu (evt->xfer_ptr) &GENMASK (31 ,0 ); + tre_index =ring_index (&channel->tre_ring, tre_offset); + first_trans =gsi_channel_trans_mapped (channel, tre_index); + +/* Get the last (newest) un-processed event */ + evt =gsi_ring_virt (ring,ring_prev (ring, rp)); +/* Get the TRE offset from that, and its associated transaction */ + tre_offset =le64_to_cpu (evt->xfer_ptr) &GENMASK (31 ,0 ); + tre_index =ring_index (&channel->tre_ring, tre_offset); + last_trans =gsi_channel_trans_mapped (channel, tre_index); + +/* Report the total number of transactions and bytes that have +* been transferred, *including* the last one. +*/ + trans_count = last_trans->trans_count - first_trans->trans_count +1 ; + byte_count = last_trans->byte_count - first_trans->byte_count; + byte_count += last_trans->len; + +ipa_gsi_channel_tx_completed (channel->gsi,gsi_channel_id (channel), + trans_count, byte_count); +} + +/** +* __gsi_evt_rx_update() - Record lengths of received data +* @evt_ring: Event ring associated with channel that received packets +* @ep: Last event in the ring associated with a completed request +* +* Events for RX channels contain the actual number of bytes received into +* the buffer. Every event has a transaction associated with it, and here +* we update each transaction's result code to record the received length. +* +* This function is called whenever we learn that the GSI hardware has filled +* new events since the last time we checked. We need to update transaction +* lengths for events starting at the ring's rp_local up to (and including) +* the ring offset supplied as an argument. +* +* Events are sequential within the event ring, and transactions are +* sequential within the transaction pool. We compute the first event's +* transaction pointer; the next event's transaction will just next one in +* the transaction pool. +* +* Note that @rp always points to an element *within* the event ring. +*/ +static void __gsi_evt_rx_update (struct gsi_evt_ring *evt_ring,u32 rp) +{ +struct gsi_channel *channel = evt_ring->channel; +struct gsi_ring *ring = &evt_ring->ring; +struct gsi_xfer_compl_evt *evt_last; +struct gsi_xfer_compl_evt *evt_end; +struct gsi_trans_info *trans_info; +struct gsi_xfer_compl_evt *evt; +struct gsi_trans *trans_end; +struct gsi_trans *trans; +u32 byte_count =0 ; +u32 tre_offset; +u32 tre_index; + +/* Start with the first un-processed event */ + evt =gsi_ring_virt (ring, ring->rp_local); + evt_last =gsi_ring_virt (ring, rp); + evt_end =gsi_ring_virt (ring, ring->end); + +/* Event xfer_ptr records the TRE it's associated with */ + tre_offset =le64_to_cpu (evt->xfer_ptr) &GENMASK (31 ,0 ); + tre_index =ring_index (&channel->tre_ring, tre_offset); +/* Get the transaction mapped to the first unprocessed event */ + trans =gsi_channel_trans_mapped (channel, tre_index); + trans_info = &channel->trans_info; + trans_end = &trans_info->pool[trans_info->pool_count]; + +do { + trans->len =__le16_to_cpu (evt->len); + trans->result =__le16_to_cpu (evt->len); + byte_count += trans->result; +if (++evt == evt_end) + evt =gsi_ring_virt (&evt_ring->ring, ring->base); +if (++trans == trans_end) + trans = &trans_info->pool[0 ]; + }while (evt != evt_last); + +/* We record RX bytes when they are received */ + channel->byte_count += byte_count; + channel->trans_count++; +} + +static void +gsi_evt_ring_doorbell (struct gsi *gsi,u32 evt_ring_id) +{ +struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id]; +u32 val; + +/* We only need to write the lower 32 bits */ + val = evt_ring->ring.wp_local; +iowrite32 (val, gsi->virt +GSI_EV_CH_E_DOORBELL_0_OFFSET (evt_ring_id)); +} + +static u32 gsi_channel_max (struct gsi *gsi) +{ +u32 val =ioread32 (gsi->virt +GSI_GSI_HW_PARAM_2_OFFSET ); + +return u32_get_bits (val,NUM_CH_PER_EE_FMASK ); +} + +static u32 gsi_evt_ring_max (struct gsi *gsi) +{ +u32 val =ioread32 (gsi->virt +GSI_GSI_HW_PARAM_2_OFFSET ); + +return u32_get_bits (val,NUM_EV_PER_EE_FMASK ); +} + +/* Issue a GSI command by writing a value to a register, then wait +* for completion to be signaled. Returns true if successful or +* false if a timeout occurred. +*/ +static void +gsi_command (struct gsi *gsi,u32 reg,u32 val,struct completion *completion) +{ +unsigned long ret; + +reinit_completion (completion); + +iowrite32 (val, gsi->virt + reg); + ret =wait_for_completion_timeout (completion,GSI_CMD_TIMEOUT *HZ ); +WARN (!ret,"%s timeout reg 0x%08x val 0x%08x\n" , __func__, reg, val); +} + +/* Issue an event ring command and wait for it to complete */ +static void evt_ring_command (struct gsi *gsi,u32 evt_ring_id, +enum gsi_evt_ch_cmd_opcode op) +{ +struct completion *completion = &gsi->evt_ring[evt_ring_id].completion; +u32 val =0 ; + + val |=u32_encode_bits (evt_ring_id,EV_CHID_FMASK ); + val |=u32_encode_bits (op,EV_OPCODE_FMASK ); + +gsi_command (gsi,GSI_EV_CH_CMD_OFFSET , val, completion); +} + +/* Issue a channel command and wait for it to complete */ +static void +gsi_channel_command (struct gsi_channel *channel,enum gsi_ch_cmd_opcode op) +{ +u32 channel_id =gsi_channel_id (channel); +u32 val =0 ; + + val |=u32_encode_bits (channel_id,CH_CHID_FMASK ); + val |=u32_encode_bits (op,CH_OPCODE_FMASK ); + +gsi_command (channel->gsi,GSI_CH_CMD_OFFSET , val, &channel->completion); +} + +static int gsi_ring_alloc (struct gsi *gsi,struct gsi_ring *ring,u32 count) +{ +size_t size =roundup_pow_of_two (count *sizeof (struct gsi_tre )); +dma_addr_t addr; + +/* Hardware requires a power-of-2 ring size (and alignment) */ + ring->virt =dma_alloc_coherent (gsi->dev, size, &addr,GFP_KERNEL ); +if (!ring->virt) +return -ENOMEM ; + ring->addr = addr; + ring->base = addr &GENMASK (31 ,0 ); + ring->size = size; + ring->end = ring->base + size; +spin_lock_init (&ring->spinlock); + +return 0 ; +} + +static void gsi_ring_free (struct gsi *gsi,struct gsi_ring *ring) +{ +dma_free_coherent (gsi->dev, ring->size, ring->virt, ring->addr); +memset (ring,0 ,sizeof (*ring)); +} + +static void gsi_evt_ring_prime (struct gsi *gsi,u32 evt_ring_id) +{ +struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id]; +struct gsi_ring *ring = &evt_ring->ring; +unsigned long flags; + +spin_lock_irqsave (&ring->spinlock, flags); + +memset (ring->virt,0 , ring->size); +/* Point the write pointer at the last element */ + ring->wp_local =ring_prev (ring, ring->base); +gsi_evt_ring_doorbell (gsi, evt_ring_id); + +spin_unlock_irqrestore (&ring->spinlock, flags); +} + +static void gsi_evt_ring_program (struct gsi *gsi,u32 evt_ring_id) +{ +struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id]; +u32 val =0 ; + +BUILD_BUG_ON (sizeof (struct gsi_xfer_compl_evt ) > +field_max (EV_ELEMENT_SIZE_FMASK )); + + val |=u32_encode_bits (GSI_EVT_CHTYPE_GPI_EV ,EV_CHTYPE_FMASK ); + val |=EV_INTYPE_FMASK ; + val |=u32_encode_bits (sizeof (struct gsi_xfer_compl_evt ), +EV_ELEMENT_SIZE_FMASK ); +iowrite32 (val, gsi->virt +GSI_EV_CH_E_CNTXT_0_OFFSET (evt_ring_id)); + + val =u32_encode_bits (evt_ring->ring.size,EV_R_LENGTH_FMASK ); +iowrite32 (val, gsi->virt +GSI_EV_CH_E_CNTXT_1_OFFSET (evt_ring_id)); + +/* The context 2 and 3 registers store the low-order and +* high-order 32 bits of the address of the event ring, +* respectively. +*/ + val = evt_ring->ring.base; +iowrite32 (val, gsi->virt +GSI_EV_CH_E_CNTXT_2_OFFSET (evt_ring_id)); + + val = evt_ring->ring.addr >>32 ; +iowrite32 (val, gsi->virt +GSI_EV_CH_E_CNTXT_3_OFFSET (evt_ring_id)); + +/* Enable interrupt moderation by setting the moderation delay */ + val =u32_encode_bits (IPA_GSI_EVT_RING_INT_MODT ,MODT_FMASK ); + val |=u32_encode_bits (1 ,MODC_FMASK );/* comes from channel */ +iowrite32 (val, gsi->virt +GSI_EV_CH_E_CNTXT_8_OFFSET (evt_ring_id)); + +/* No MSI write data, and MSI address high and low address is 0 */ +iowrite32 (0 , gsi->virt +GSI_EV_CH_E_CNTXT_9_OFFSET (evt_ring_id)); +iowrite32 (0 , gsi->virt +GSI_EV_CH_E_CNTXT_10_OFFSET (evt_ring_id)); +iowrite32 (0 , gsi->virt +GSI_EV_CH_E_CNTXT_11_OFFSET (evt_ring_id)); + +/* We don't need to get event read pointer updates */ +iowrite32 (0 , gsi->virt +GSI_EV_CH_E_CNTXT_12_OFFSET (evt_ring_id)); +iowrite32 (0 , gsi->virt +GSI_EV_CH_E_CNTXT_13_OFFSET (evt_ring_id)); +} + +static void gsi_ring_init (struct gsi_ring *ring) +{ + ring->wp = ring->base; + ring->wp_local = ring->base; + ring->rp_local = ring->base; +} + +static void gsi_evt_ring_scratch_zero (struct gsi *gsi,u32 evt_ring_id) +{ +iowrite32 (0 , gsi->virt +GSI_EV_CH_E_SCRATCH_0_OFFSET (evt_ring_id)); +iowrite32 (0 , gsi->virt +GSI_EV_CH_E_SCRATCH_1_OFFSET (evt_ring_id)); +} + +static int gsi_evt_ring_alloc_hw (struct gsi *gsi,u32 evt_ring_id) +{ +struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id]; +unsigned long flags; +u32 val; + +evt_ring_command (gsi, evt_ring_id,GSI_EVT_ALLOCATE ); + +if (evt_ring->state !=GSI_EVT_RING_STATE_ALLOCATED ) { +dev_err (gsi->dev,"evt_ring_id %u allocation bad state %u\n" , + evt_ring_id, evt_ring->state); +return -EIO ; + } + +gsi_evt_ring_program (gsi, evt_ring_id); +gsi_ring_init (&evt_ring->ring); +gsi_evt_ring_prime (gsi, evt_ring_id); + +spin_lock_irqsave (&gsi->spinlock, flags); + +/* Enable the event interrupt (clear it first in case pending) */ + val =BIT (evt_ring_id); +iowrite32 (val, gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_CLR_OFFSET ); +gsi_irq_enable_event (gsi, evt_ring_id); + +spin_unlock_irqrestore (&gsi->spinlock, flags); + +return 0 ; +} + +static void gsi_evt_ring_free_hw (struct gsi *gsi,u32 evt_ring_id) +{ +struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id]; +unsigned long flags; + +spin_lock_irqsave (&gsi->spinlock, flags); + +/* Disable the event interrupt */ +gsi_irq_disable_event (gsi, evt_ring_id); + +spin_unlock_irqrestore (&gsi->spinlock, flags); + +evt_ring_command (gsi, evt_ring_id,GSI_EVT_RESET ); + +gsi_evt_ring_program (gsi, evt_ring_id); +gsi_ring_init (&evt_ring->ring); +gsi_evt_ring_scratch_zero (gsi, evt_ring_id); +gsi_evt_ring_prime (gsi, evt_ring_id); + +evt_ring_command (gsi, evt_ring_id,GSI_EVT_DE_ALLOC ); +} + +static int gsi_evt_ring_id_alloc (struct gsi *gsi) +{ +u32 evt_ring_id; + +if (gsi->event_bitmap == ~0U ) +return -ENOSPC ; + + evt_ring_id =ffz (gsi->event_bitmap); + gsi->event_bitmap |=BIT (evt_ring_id); + +return (int )evt_ring_id; +} + +static void gsi_evt_ring_id_free (struct gsi *gsi,u32 evt_ring_id) +{ + gsi->event_bitmap &= ~BIT (evt_ring_id); +} + +void gsi_channel_doorbell (struct gsi_channel *channel) +{ +u32 channel_id =gsi_channel_id (channel); +struct gsi *gsi = channel->gsi; +u32 val; + + channel->tre_ring.wp = channel->tre_ring.wp_local; + +/* We only need to write the lower 32 bits */ + val = channel->tre_ring.wp_local; +iowrite32 (val, gsi->virt +GSI_CH_C_DOORBELL_0_OFFSET (channel_id)); +} + +static void __gsi_evt_ring_update (struct gsi *gsi,u32 evt_ring_id) +{ +struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id]; +u32 offset =GSI_EV_CH_E_CNTXT_4_OFFSET (evt_ring_id); +struct gsi_channel *channel = evt_ring->channel; +struct gsi_ring *tre_ring = &channel->tre_ring; +struct gsi_ring *ring = &evt_ring->ring; +u32 rp =ioread32 (gsi->virt + offset); +struct gsi_xfer_compl_evt *evt; +struct gsi_trans *trans; +u32 tre_offset; +u32 tre_index; +u32 rp_last; + +/* If we have nothing new to process we're done */ +if (ring->rp_local == rp) +return ; + +/* Extract information from the newly-completed events. For TX +* channels, report the number of transferred bytes they represent. +* For RX channels, update each transaction with the number of bytes +* actually received. +*/ +if (channel->toward_ipa) +__gsi_evt_tx_update (evt_ring, rp); +else +__gsi_evt_rx_update (evt_ring, rp); + +/* Get the TRE pointer from the latest completion event, and get +* the transaction associated with that. Move all new transactions +* up to and including that one to the completed list. +*/ + rp_last =ring_prev (ring, rp); + evt =gsi_ring_virt (ring, rp_last); + tre_offset =le64_to_cpu (evt->xfer_ptr) &GENMASK (31 ,0 ); + tre_index =ring_index (tre_ring, tre_offset); + trans =gsi_channel_trans_mapped (channel, tre_index); +gsi_trans_move_complete (trans); + +/* We need nothing more from these TREs, so consume them */ + tre_ring->rp_local = tre_offset; +gsi_ring_rp_local_add (tre_ring,1 ); + +/* Record that we're caught up on these events, and give the +* completed ones back to the hardware for reuse. +*/ + ring->rp_local = rp; + ring->wp_local = rp_last; +gsi_evt_ring_doorbell (channel->gsi, channel->evt_ring_id); +} + +/* Consult hardware, move any newly completed transactions to completed list */ +static void gsi_channel_update (struct gsi_channel *channel) +{ +struct gsi_evt_ring *evt_ring; +unsigned long flags; + + evt_ring = &channel->gsi->evt_ring[channel->evt_ring_id]; + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +__gsi_evt_ring_update (channel->gsi, channel->evt_ring_id); + +spin_unlock_irqrestore (&evt_ring->ring.spinlock, flags); +} + +/** +* gsi_channel_poll_one() - Return a single completed transaction on a channel +* @channel: Channel to be polled +* +* @Return: Transaction pointer, or null if none are available +* +* This function returns the first entry on a channel's completed +* transaction list. If that list is empty, the hardware is consulted +* to determine whether any new transactions have completed. If so, +* they're moved to the completed list and the new first entry is +* returned. If there are no more completed transactions, a null +* pointer is returned. +*/ +static struct gsi_trans *gsi_channel_poll_one (struct gsi_channel *channel) +{ +struct gsi_trans *trans; + +/* Get the first transaction from the completed list */ + trans =gsi_channel_trans_complete (channel); +if (!trans) { +/* List is empty; see if there's more to do */ +gsi_channel_update (channel); + trans =gsi_channel_trans_complete (channel); + } + +if (trans) +gsi_trans_move_polled (trans); + +return trans; +} + +/** +* gsi_channel_poll() - NAPI poll function for a channel +* @napi: NAPI structure for the channel +* @budget: Budget supplied by NAPI core + +* @channel_id: Channel to be reset +* +* @Return: Number of items polled (<= budget) +* +* Single transactions completed by hardware are polled until either +* the budget is exhausted, or there are no more. Each transaction +* polled is passed to gsi_trans_complete(), to perform remaining +* completion processing and retire/free the transaction. +*/ +static int gsi_channel_poll (struct napi_struct *napi,int budget) +{ +struct gsi_channel *channel; +int count =0 ; + + channel = container_of(napi, struct gsi_channel, napi); +while (count < budget) { +struct gsi_trans *trans; + + trans =gsi_channel_poll_one (channel); +if (!trans) +break ; +gsi_trans_complete (trans); + } + +if (count < budget) { +napi_complete (&channel->napi); + (void )gsi_channel_intr (channel, true); + } + +return count; +} + +/* The event bitmap represents which event ids are available for +* allocation. Set bits are not available, clear bits can be used. +* This function initializes the map so all events supported by the +* hardware are available, then precludes any reserved events from +* being allocated. +*/ +static u32 gsi_event_bitmap_init (u32 evt_ring_max) +{ +u32 event_bitmap =GENMASK (BITS_PER_LONG -1 , evt_ring_max); + +return event_bitmap |GENMASK (GSI_MHI_ER_END ,GSI_MHI_ER_START ); +} + +/* Setup function for event rings */ +static int gsi_evt_ring_setup (struct gsi *gsi) +{ +u32 evt_ring_max; +u32 evt_ring_id; + + evt_ring_max =gsi_evt_ring_max (gsi); +dev_dbg (gsi->dev,"evt_ring_max %u\n" , evt_ring_max); +if (evt_ring_max !=GSI_EVT_RING_MAX ) +return -EIO ; + +for (evt_ring_id =0 ; evt_ring_id <GSI_EVT_RING_MAX ; evt_ring_id++) { +struct gsi_evt_ring *evt_ring = &gsi->evt_ring[evt_ring_id]; + + evt_ring->state =gsi_evt_ring_state (gsi, evt_ring_id); +if (evt_ring->state !=GSI_EVT_RING_STATE_NOT_ALLOCATED ) +return -EIO ; + } + +/* Enable all event interrupts */ +gsi_irq_enable_all (gsi); + +return 0 ; +} + +/* Inverse of gsi_evt_ring_setup() */ +static void gsi_evt_ring_teardown (struct gsi *gsi) +{ +gsi_irq_disable_all (gsi); +} + +static void gsi_channel_scratch_write (struct gsi_channel *channel) +{ +u32 channel_id =gsi_channel_id (channel); +struct gsi_gpi_channel_scratch *gpi; +union gsi_channel_scratch scr = { }; +struct gsi *gsi = channel->gsi; +u32 val; + +/* See comments above definition of gsi_gpi_channel_scratch */ + gpi = &scr.gpi; + gpi->max_outstanding_tre = channel->data->tlv_count * +sizeof (struct gsi_tre ); + gpi->outstanding_threshold =2 *sizeof (struct gsi_tre ); + + val = scr.data.word1; +iowrite32 (val, gsi->virt +GSI_CH_C_SCRATCH_0_OFFSET (channel_id)); + + val = scr.data.word2; +iowrite32 (val, gsi->virt +GSI_CH_C_SCRATCH_1_OFFSET (channel_id)); + + val = scr.data.word3; +iowrite32 (val, gsi->virt +GSI_CH_C_SCRATCH_2_OFFSET (channel_id)); + +/* We must preserve the upper 16 bits of the last scratch +* register. The next sequence assumes those bits remain +* unchanged between the read and the write. +*/ + val =ioread32 (gsi->virt +GSI_CH_C_SCRATCH_3_OFFSET (channel_id)); + val = (scr.data.word4 &GENMASK (31 ,16 )) | (val &GENMASK (15 ,0 )); +iowrite32 (val, gsi->virt +GSI_CH_C_SCRATCH_3_OFFSET (channel_id)); +} + +static void gsi_channel_program (struct gsi_channel *channel,bool doorbell) +{ +u32 channel_id =gsi_channel_id (channel); +struct gsi *gsi = channel->gsi; +u32 wrr_weight =0 ; +u32 val =0 ; + +BUILD_BUG_ON (sizeof (struct gsi_tre ) >field_max (ELEMENT_SIZE_FMASK )); + + val |=u32_encode_bits (GSI_CHANNEL_PROTOCOL_GPI ,CHTYPE_PROTOCOL_FMASK ); +if (channel->toward_ipa) + val |=CHTYPE_DIR_FMASK ; + val |=u32_encode_bits (channel->evt_ring_id,ERINDEX_FMASK ); + val |=u32_encode_bits (sizeof (struct gsi_tre ),ELEMENT_SIZE_FMASK ); +iowrite32 (val, gsi->virt +GSI_CH_C_CNTXT_0_OFFSET (channel_id)); + + val =u32_encode_bits (channel->tre_ring.size,R_LENGTH_FMASK ); +iowrite32 (val, gsi->virt +GSI_CH_C_CNTXT_1_OFFSET (channel_id)); + +/* The context 2 and 3 registers store the low-order and +* high-order 32 bits of the address of the channel ring, +* respectively. +*/ + val = channel->tre_ring.addr &GENMASK (31 ,0 ); +iowrite32 (val, gsi->virt +GSI_CH_C_CNTXT_2_OFFSET (channel_id)); + + val = channel->tre_ring.addr >>32 ; +iowrite32 (val, gsi->virt +GSI_CH_C_CNTXT_3_OFFSET (channel_id)); + +if (channel->data->wrr_priority) + wrr_weight =field_max (WRR_WEIGHT_FMASK ); + val =u32_encode_bits (wrr_weight,WRR_WEIGHT_FMASK ); + +/* Max prefetch is 1 segment (do not set MAX_PREFETCH_FMASK) */ +if (doorbell) + val |=USE_DB_ENG_FMASK ; +iowrite32 (val, gsi->virt +GSI_CH_C_QOS_OFFSET (channel_id)); +} + +static void +__gsi_channel_config (struct gsi_channel *channel,bool doorbell_enable) +{ +gsi_channel_program (channel, doorbell_enable); +gsi_ring_init (&channel->tre_ring); +gsi_channel_scratch_write (channel); +} + +void gsi_channel_config (struct gsi *gsi,u32 channel_id,bool doorbell_enable) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; + +mutex_lock (&gsi->mutex); + +__gsi_channel_config (channel, doorbell_enable); + +mutex_unlock (&gsi->mutex); +} + +/* Setup function for a single channel */ +static int gsi_channel_setup_one (struct gsi_channel *channel) +{ +struct gsi *gsi = channel->gsi; +int ret; + +if (!gsi) +return 0 ;/* Ignore uninitialized channels */ + + channel->state =gsi_channel_state (channel); +if (channel->state !=GSI_CHANNEL_STATE_NOT_ALLOCATED ) +return -EIO ; + +mutex_lock (&gsi->mutex); + + ret =gsi_evt_ring_alloc_hw (gsi, channel->evt_ring_id); +if (ret) { +mutex_unlock (&gsi->mutex); + +return ret; + } + +gsi_channel_command (channel,GSI_CH_ALLOCATE ); + ret = channel->state ==GSI_CHANNEL_STATE_ALLOCATED ?0 : -EIO ; +if (ret) { +gsi_evt_ring_free_hw (gsi, channel->evt_ring_id); +mutex_unlock (&gsi->mutex); + +return ret; + } + +__gsi_channel_config (channel, true); + +mutex_unlock (&gsi->mutex); + + gsi->channel_stats.allocate++; + +if (channel->toward_ipa) +netif_tx_napi_add (&gsi->dummy_dev, &channel->napi, + gsi_channel_poll,NAPI_POLL_WEIGHT ); +else +netif_napi_add (&gsi->dummy_dev, &channel->napi, + gsi_channel_poll,NAPI_POLL_WEIGHT ); + +return 0 ; +} + +/* Inverse of gsi_channel_setup_one() */ +static void gsi_channel_teardown_one (struct gsi_channel *channel) +{ +struct gsi *gsi = channel->gsi; + +if (!gsi) +return ; + +netif_napi_del (&channel->napi); + +mutex_lock (&gsi->mutex); + +gsi_channel_command (channel,GSI_CH_DE_ALLOC ); + + gsi->channel_stats.free++; + +gsi_evt_ring_free_hw (gsi, channel->evt_ring_id); + +mutex_unlock (&gsi->mutex); + +gsi_channel_trans_exit (channel); +} + +/* Setup function for channels */ +static int gsi_channel_setup (struct gsi *gsi) +{ +u32 channel_max; +u32 channel_id; +int ret; + + channel_max =gsi_channel_max (gsi); +dev_dbg (gsi->dev,"channel_max %u\n" , channel_max); +if (channel_max !=GSI_CHANNEL_MAX ) +return -EIO ; + + ret =gsi_evt_ring_setup (gsi); +if (ret) +return ret; + +for (channel_id =0 ; channel_id <GSI_CHANNEL_MAX ; channel_id++) { + ret =gsi_channel_setup_one (&gsi->channel[channel_id]); +if (ret) + goto err_unwind; + } + +return 0 ; + +err_unwind: +while (channel_id--) +gsi_channel_teardown_one (&gsi->channel[channel_id]); +gsi_evt_ring_teardown (gsi); + +return ret; +} + +/* Inverse of gsi_channel_setup() */ +static void gsi_channel_teardown (struct gsi *gsi) +{ +u32 channel_id; + +for (channel_id =0 ; channel_id <GSI_CHANNEL_MAX ; channel_id++) { +struct gsi_channel *channel = &gsi->channel[channel_id]; + +gsi_channel_teardown_one (channel); + } + +gsi_evt_ring_teardown (gsi); +} + +/* Setup function for GSI. GSI firmware must be loaded and initialized */ +int gsi_setup (struct gsi *gsi) +{ +u32 val; + +/* Here is where we first touch the GSI hardware */ + val =ioread32 (gsi->virt +GSI_GSI_STATUS_OFFSET ); +if (!(val &ENABLED_FMASK )) { +dev_err (gsi->dev,"GSI has not been enabled\n" ); +return -EIO ; + } + +/* Initialize the error log */ +iowrite32 (0 , gsi->virt +GSI_ERROR_LOG_OFFSET ); + +/* Writing 1 indicates IRQ interrupts; 0 would be MSI */ +iowrite32 (1 , gsi->virt +GSI_CNTXT_INTSET_OFFSET ); + +return gsi_channel_setup (gsi); +} + +/* Inverse of gsi_setup() */ +void gsi_teardown (struct gsi *gsi) +{ +gsi_channel_teardown (gsi); +} + +/* Initialize a channel's event ring */ +static int gsi_channel_evt_ring_init (struct gsi_channel *channel) +{ +struct gsi *gsi = channel->gsi; +struct gsi_evt_ring *evt_ring; +int ret; + + ret =gsi_evt_ring_id_alloc (gsi); +if (ret <0 ) +return ret; + channel->evt_ring_id = ret; + + evt_ring = &gsi->evt_ring[channel->evt_ring_id]; + evt_ring->channel = channel; + + ret =gsi_ring_alloc (gsi, &evt_ring->ring, channel->data->event_count); +if (ret) + goto err_free_evt_ring_id; + +return 0 ; + +err_free_evt_ring_id: +gsi_evt_ring_id_free (gsi, channel->evt_ring_id); + +return ret; +} + +/* Inverse of gsi_channel_evt_ring_init() */ +static void gsi_channel_evt_ring_exit (struct gsi_channel *channel) +{ +struct gsi *gsi = channel->gsi; +struct gsi_evt_ring *evt_ring; + + evt_ring = &gsi->evt_ring[channel->evt_ring_id]; +gsi_ring_free (gsi, &evt_ring->ring); + +gsi_evt_ring_id_free (gsi, channel->evt_ring_id); +} + +/* Init function for event rings */ +static void gsi_evt_ring_init (struct gsi *gsi) +{ +u32 evt_ring_id; + +BUILD_BUG_ON (GSI_EVT_RING_MAX >=BITS_PER_LONG ); + + gsi->event_bitmap =gsi_event_bitmap_init (GSI_EVT_RING_MAX ); + gsi->event_enable_bitmap =0 ; +for (evt_ring_id =0 ; evt_ring_id <GSI_EVT_RING_MAX ; evt_ring_id++) +init_completion (&gsi->evt_ring[evt_ring_id].completion); +} + +/* Inverse of gsi_evt_ring_init() */ +static void gsi_evt_ring_exit (struct gsi *gsi) +{ +/* Nothing to do */ +} + +/* Init function for a single channel */ +static int +gsi_channel_init_one (struct gsi *gsi,const struct gsi_ipa_endpoint_data *data) +{ +struct gsi_channel *channel; +int ret; + +if (data->ee_id !=GSI_EE_AP ) +return 0 ;/* Ignore non-AP channels */ + +if (data->channel_id >=GSI_CHANNEL_MAX ) +return -EIO ; + channel = &gsi->channel[data->channel_id]; + + channel->gsi = gsi; + channel->toward_ipa = data->toward_ipa; + channel->data = &data->channel; + +init_completion (&channel->completion); + + ret =gsi_channel_evt_ring_init (channel); +if (ret) +return ret; + + ret =gsi_ring_alloc (gsi, &channel->tre_ring, channel->data->tre_count); +if (ret) + goto err_channel_evt_ring_exit; + + ret =gsi_channel_trans_init (channel); +if (ret) + goto err_ring_free; + +return 0 ; + +err_ring_free: +gsi_ring_free (gsi, &channel->tre_ring); +err_channel_evt_ring_exit: +gsi_channel_evt_ring_exit (channel); + +return ret; +} + +/* Inverse of gsi_channel_init_one() */ +static void gsi_channel_exit_one (struct gsi_channel *channel) +{ +gsi_channel_trans_exit (channel); +gsi_ring_free (channel->gsi, &channel->tre_ring); +gsi_channel_evt_ring_exit (channel); +} + +/* Init function for channels */ +static int gsi_channel_init (struct gsi *gsi,u32 data_count, +const struct gsi_ipa_endpoint_data *data) +{ +int ret =0 ; +u32 i; + +gsi_evt_ring_init (gsi); +for (i =0 ; i < data_count; i++) { + ret =gsi_channel_init_one (gsi, &data[i]); +if (ret) +break ; + } + +return ret; +} + +/* Inverse of gsi_channel_init() */ +static void gsi_channel_exit (struct gsi *gsi) +{ +u32 channel_id; + +for (channel_id =0 ; channel_id <GSI_CHANNEL_MAX ; channel_id++) +gsi_channel_exit_one (&gsi->channel[channel_id]); +gsi_evt_ring_exit (gsi); +} + +/* Init function for GSI. GSI hardware does not need to be "ready" */ +int gsi_init (struct gsi *gsi,struct platform_device *pdev,u32 data_count, +const struct gsi_ipa_endpoint_data *data) +{ +struct resource *res; +resource_size_t size; +unsigned int irq; +int ret; + + gsi->dev = &pdev->dev; +init_dummy_netdev (&gsi->dummy_dev); + +/* Get GSI memory range and map it */ + res =platform_get_resource_byname (pdev,IORESOURCE_MEM ,"gsi" ); +if (!res) +return -ENXIO ; + + size =resource_size (res); +if (res->start >U32_MAX || size >U32_MAX - res->start) +return -EINVAL ; + + gsi->virt =ioremap_nocache (res->start, size); +if (!gsi->virt) +return -ENOMEM ; + + ret =platform_get_irq_byname (pdev,"gsi" ); +if (ret <0 ) + goto err_unmap_virt; + irq = ret; + + ret =request_irq (irq, gsi_isr,0 ,"gsi" , gsi); +if (ret) + goto err_unmap_virt; + gsi->irq = irq; + + ret =enable_irq_wake (gsi->irq); +if (ret) +dev_err (gsi->dev,"error %d enabling gsi wake irq\n" , ret); + gsi->irq_wake_enabled = ret ?0 :1 ; + +spin_lock_init (&gsi->spinlock); +mutex_init (&gsi->mutex); + + ret =gsi_channel_init (gsi, data_count, data); +if (ret) + goto err_mutex_destroy; + +return 0 ; + +err_mutex_destroy: +mutex_destroy (&gsi->mutex); +if (gsi->irq_wake_enabled) + (void )disable_irq_wake (gsi->irq); +free_irq (gsi->irq, gsi); +err_unmap_virt: +iounmap (gsi->virt); + +return ret; +} + +/* Inverse of gsi_init() */ +void gsi_exit (struct gsi *gsi) +{ +gsi_channel_exit (gsi); + +mutex_destroy (&gsi->mutex); +if (gsi->irq_wake_enabled) + (void )disable_irq_wake (gsi->irq); +free_irq (gsi->irq, gsi); +iounmap (gsi->virt); +} + +/* Returns the maximum number of pending transactions on a channel */ +u32 gsi_channel_trans_max (struct gsi *gsi,u32 channel_id) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; + +return channel->data->tre_count; +} + +/* Returns the maximum number of TREs in a single transaction for a channel */ +u32 gsi_channel_trans_tre_max (struct gsi *gsi,u32 channel_id) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; + +return channel->data->tlv_count; +} + +/* Wait for all transaction activity on a channel to complete */ +void gsi_channel_trans_quiesce (struct gsi *gsi,u32 channel_id) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; +struct gsi_trans_info *trans_info; +struct gsi_trans *trans =NULL ; +struct gsi_evt_ring *evt_ring; +struct list_head *list; +unsigned long flags; + + trans_info = &channel->trans_info; + evt_ring = &channel->gsi->evt_ring[channel->evt_ring_id]; + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +/* Find the last list to which a transaction was added */ +if (!list_empty (&trans_info->alloc)) + list = &trans_info->alloc; +else if (!list_empty (&trans_info->pending)) + list = &trans_info->pending; +else if (!list_empty (&trans_info->complete)) + list = &trans_info->complete; +else if (!list_empty (&trans_info->polled)) + list = &trans_info->polled; +else + list =NULL ; + +if (list) { +struct gsi_trans *trans; + +/* The last entry on this list is the last one allocated. +* Grab a reference so we can wait for it. +*/ + trans = list_last_entry(list, struct gsi_trans, links); +refcount_inc (&trans->refcount); + } + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +/* If there is one, wait for it to complete */ +if (trans) { +wait_for_completion (&trans->completion); +gsi_trans_free (trans); + } +} + +/* Make a channel operational */ +int gsi_channel_start (struct gsi *gsi,u32 channel_id) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; + +if (channel->state !=GSI_CHANNEL_STATE_ALLOCATED && + channel->state !=GSI_CHANNEL_STATE_STOP_IN_PROC && + channel->state !=GSI_CHANNEL_STATE_STOPPED ) { +dev_err (gsi->dev,"channel %u bad state %u\n" , channel_id, + (u32 )channel->state); +return -ENOTSUPP ; + } + +napi_enable (&channel->napi); + +mutex_lock (&gsi->mutex); + +gsi_channel_command (channel,GSI_CH_START ); + +mutex_unlock (&gsi->mutex); + + gsi->channel_stats.start++; + +return 0 ; +} + +/* Stop an operational channel */ +int gsi_channel_stop (struct gsi *gsi,u32 channel_id) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; +int ret; + +if (channel->state ==GSI_CHANNEL_STATE_STOPPED ) +return 0 ; + +if (channel->state !=GSI_CHANNEL_STATE_STARTED && + channel->state !=GSI_CHANNEL_STATE_STOP_IN_PROC && + channel->state !=GSI_CHANNEL_STATE_ERROR ) { +dev_err (gsi->dev,"channel %u bad state %u\n" , channel_id, + (u32 )channel->state); +return -ENOTSUPP ; + } + +gsi_channel_trans_quiesce (gsi, channel_id); + +mutex_lock (&gsi->mutex); + +gsi_channel_command (channel,GSI_CH_STOP ); + +mutex_unlock (&gsi->mutex); + +if (channel->state ==GSI_CHANNEL_STATE_STOPPED ) + ret =0 ; +else if (channel->state ==GSI_CHANNEL_STATE_STOP_IN_PROC ) + ret = -EAGAIN ; +else + ret = -EIO ; + + gsi->channel_stats.stop++; + +if (!ret) +napi_disable (&channel->napi); + +return ret; +} + +/* Reset a GSI channel */ +int gsi_channel_reset (struct gsi *gsi,u32 channel_id) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; + +if (channel->state !=GSI_CHANNEL_STATE_STOPPED ) { +dev_err (gsi->dev,"channel %u bad state %u\n" , channel_id, + (u32 )channel->state); +return -ENOTSUPP ; + } + +/* In case the reset follows stop, need to wait 1 msec */ +usleep_range (USEC_PER_MSEC ,2 *USEC_PER_MSEC ); + +mutex_lock (&gsi->mutex); + +gsi_channel_command (channel,GSI_CH_RESET ); + +/* workaround: reset RX channels again */ +if (!channel->toward_ipa) { +usleep_range (USEC_PER_MSEC ,2 *USEC_PER_MSEC ); +gsi_channel_command (channel,GSI_CH_RESET ); + } + +__gsi_channel_config (channel, true); + +/* Cancel pending transactions before the channel is started again */ +gsi_channel_trans_cancel_pending (channel); + +mutex_unlock (&gsi->mutex); + + gsi->channel_stats.reset++; + +return 0 ; +}
Signature · 1 line
2.20.1
Re: [PATCH 08/18] soc: qcom: ipa: the generic software interface
Quoted 24 lines · Alex Elder
drivers/net/ipa/gsi.c new file +24
-0+/** gsi_gpi_channel_scratch - GPI protocol scratch register +* +* @max_outstanding_tre: +* Defines the maximum number of TREs allowed in a single transaction +* on a channel (in Bytes). This determines the amount of prefetch +* performed by the hardware. We configure this to equal the size of +* the TLV FIFO for the channel. +* @outstanding_threshold: +* Defines the threshold (in Bytes) determining when the sequencer +* should update the channel doorbell. We configure this to equal +* the size of two TREs. +*/ +struct gsi_gpi_channel_scratch { +u64 rsvd1; +u16 rsvd2; +u16 max_outstanding_tre; +u16 rsvd3; +u16 outstanding_threshold; +} __packed; + +/** gsi_channel_scratch - channel scratch configuration area +* +* The exact interpretation of this register is protocol-specific. +* We only use GPI channels; see struct gsi_gpi_channel_scratch, above.
drivers/net/ipa/gsi.c line 196
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+*/ +union gsi_channel_scratch { +struct gsi_gpi_channel_scratch gpi; +struct { +u32 word1; +u32 word2; +u32 word3; +u32 word4; + } data; +} __packed;
What are the exact alignment requirements on these structures, do you ever need to have them on odd addresses? If not, please remove the __packed, or add __aligned() with the actual alignment, e.g. __aligned(4), to let the compiler create better code and avoid bytewise accesses.
drivers/net/ipa/gsi.c line 1517
Quoted 11 lines
drivers/net/ipa/gsi.c new file +11
-0+/* Init function for GSI. GSI hardware does not need to be "ready" */ +int gsi_init (struct gsi *gsi,struct platform_device *pdev,u32 data_count, +const struct gsi_ipa_endpoint_data *data) +{ +struct resource *res; +resource_size_t size; +unsigned int irq; +int ret; + + gsi->dev = &pdev->dev; +init_dummy_netdev (&gsi->dummy_dev);
Can you add a comment here to explain what the 'dummy' device is needed for?
drivers/net/ipa/gsi.c line 1530
Quoted 12 lines
drivers/net/ipa/gsi.c new file +12
-0+/* Get GSI memory range and map it */ + res =platform_get_resource_byname (pdev,IORESOURCE_MEM ,"gsi" ); +if (!res) +return -ENXIO ; + + size =resource_size (res); +if (res->start >U32_MAX || size >U32_MAX - res->start) +return -EINVAL ; + + gsi->virt =ioremap_nocache (res->start, size); +if (!gsi->virt) +return -ENOMEM ;
The _nocache() postfix is not needed here, and I find it a bit confusing, just use plain ioremap, or maybe even devm_platform_ioremap_resource() to save the platform_get_resource_byname().
drivers/net/ipa/gsi.c line 1548
Quoted 12 lines
drivers/net/ipa/gsi.c new file +12
-0+ ret =request_irq (irq, gsi_isr,0 ,"gsi" , gsi); +if (ret) + goto err_unmap_virt; + gsi->irq = irq; + + ret =enable_irq_wake (gsi->irq); +if (ret) +dev_err (gsi->dev,"error %d enabling gsi wake irq\n" , ret); + gsi->irq_wake_enabled = ret ?0 :1 ; + +spin_lock_init (&gsi->spinlock); +mutex_init (&gsi->mutex);
This looks a bit dangerous if you can ever get to the point of having a pending interrupt. before the structure is fully initialized. This can probably not happen in practice, but it's better to request the interrupts last to be on the safe side.
Quoted 32 lines
drivers/net/ipa/gsi.c new file +32
-0+/* Wait for all transaction activity on a channel to complete */ +void gsi_channel_trans_quiesce (struct gsi *gsi,u32 channel_id) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; +struct gsi_trans_info *trans_info; +struct gsi_trans *trans =NULL ; +struct gsi_evt_ring *evt_ring; +struct list_head *list; +unsigned long flags; + + trans_info = &channel->trans_info; + evt_ring = &channel->gsi->evt_ring[channel->evt_ring_id]; + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +/* Find the last list to which a transaction was added */ +if (!list_empty (&trans_info->alloc)) + list = &trans_info->alloc; +else if (!list_empty (&trans_info->pending)) + list = &trans_info->pending; +else if (!list_empty (&trans_info->complete)) + list = &trans_info->complete; +else if (!list_empty (&trans_info->polled)) + list = &trans_info->polled; +else + list =NULL ; + +if (list) { +struct gsi_trans *trans; + +/* The last entry on this list is the last one allocated. +* Grab a reference so we can wait for it.
drivers/net/ipa/gsi.c line 1636
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+*/ + trans = list_last_entry(list, struct gsi_trans, links); +refcount_inc (&trans->refcount); + } + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +/* If there is one, wait for it to complete */ +if (trans) { +wait_for_completion (&trans->completion);
Since you are waiting here, you clearly can't be called from interrupt context, or with interrupts disabled, so it's clearer to use spin_lock_irq() instead of spin_lock_irqsave().
I generally try to avoid the _irqsave versions altogether, unless it is really needed for a function that is called both from irq-disabled and irq-enabled context.
Arnd
Re: [PATCH 08/18] soc: qcom: ipa: the generic software interface
Quoted 34 lines · Arnd Bergmann
Quoted 32 lines · Alex Elder
drivers/net/ipa/gsi.c new file +32
-0+/** gsi_gpi_channel_scratch - GPI protocol scratch register +* +* @max_outstanding_tre: +* Defines the maximum number of TREs allowed in a single transaction +* on a channel (in Bytes). This determines the amount of prefetch +* performed by the hardware. We configure this to equal the size of +* the TLV FIFO for the channel. +* @outstanding_threshold: +* Defines the threshold (in Bytes) determining when the sequencer +* should update the channel doorbell. We configure this to equal +* the size of two TREs. +*/ +struct gsi_gpi_channel_scratch { +u64 rsvd1; +u16 rsvd2; +u16 max_outstanding_tre; +u16 rsvd3; +u16 outstanding_threshold; +} __packed; + +/** gsi_channel_scratch - channel scratch configuration area +* +* The exact interpretation of this register is protocol-specific. +* We only use GPI channels; see struct gsi_gpi_channel_scratch, above. +*/ +union gsi_channel_scratch { +struct gsi_gpi_channel_scratch gpi; +struct { +u32 word1; +u32 word2; +u32 word3; +u32 word4;
Quoted 8 lines
drivers/net/ipa/gsi.c line 196
Quoted 2 lines
drivers/net/ipa/gsi.c new file +2
-0+ } data; +} __packed;What are the exact alignment requirements on these structures, do you ever need to have them on odd addresses? If not, please remove the
__packed, or add__aligned()with the actual alignment, e.g.__aligned(4), to let the compiler create better code and avoid bytewise accesses.
Honestly I don't know but I would guess they've actually got alignment requirements consistent with C standard... Many, many structures had the __packed attribute attached in the original code. I removed most but apparently not all. I will remove the __packed here, and will scan through the rest of the code for other similar instances and will remove those if appropriate as well.
Quoted 6 lines
Quoted 6 lines
drivers/net/ipa/gsi.c new file +6
-0+/* Init function for GSI. GSI hardware does not need to be "ready" */ +int gsi_init (struct gsi *gsi,struct platform_device *pdev,u32 data_count, +const struct gsi_ipa_endpoint_data *data) +{ +struct resource *res; +resource_size_t size;
Quoted 8 lines
drivers/net/ipa/gsi.c line 1517
Quoted 5 lines
drivers/net/ipa/gsi.c new file +5
-0+unsigned int irq; +int ret; + + gsi->dev = &pdev->dev; +init_dummy_netdev (&gsi->dummy_dev);Can you add a comment here to explain what the 'dummy' device is needed for?
Yes, good idea.
FYI it's needed because the GSI code is not a "real" network device (that, where needed, is implemented in "ipa_netdev.c", two logical layers up), but in order to use NAPI there needs to be one.
Quoted 9 lines
Quoted 9 lines
drivers/net/ipa/gsi.c new file +9
-0+/* Get GSI memory range and map it */ + res =platform_get_resource_byname (pdev,IORESOURCE_MEM ,"gsi" ); +if (!res) +return -ENXIO ; + + size =resource_size (res); +if (res->start >U32_MAX || size >U32_MAX - res->start) +return -EINVAL ; +
Quoted 8 lines
drivers/net/ipa/gsi.c line 1530
Quoted 3 lines
drivers/net/ipa/gsi.c new file +3
-0+ gsi->virt =ioremap_nocache (res->start, size); +if (!gsi->virt) +return -ENOMEM ;The
_nocache()postfix is not needed here, and I find it a bit confusing, just use plain ioremap, or maybe evendevm_platform_ioremap_resource()to save theplatform_get_resource_byname().
OK good idea. This was in the original code and I neglected to chase this down. Thank you for catching it.
Quoted 9 lines
Quoted 9 lines
drivers/net/ipa/gsi.c new file +9
-0+ ret =request_irq (irq, gsi_isr,0 ,"gsi" , gsi); +if (ret) + goto err_unmap_virt; + gsi->irq = irq; + + ret =enable_irq_wake (gsi->irq); +if (ret) +dev_err (gsi->dev,"error %d enabling gsi wake irq\n" , ret); + gsi->irq_wake_enabled = ret ?0 :1 ;
Quoted 8 lines
drivers/net/ipa/gsi.c line 1548
Quoted 3 lines
drivers/net/ipa/gsi.c new file +3
-0+ +spin_lock_init (&gsi->spinlock); +mutex_init (&gsi->mutex);This looks a bit dangerous if you can ever get to the point of having a pending interrupt. before the structure is fully initialized. This can probably not happen in practice, but it's better to request the interrupts last to be on the safe side.
Understood. I'll fix that.
Quoted 47 lines
Quoted 32 lines
drivers/net/ipa/gsi.c new file +32
-0+/* Wait for all transaction activity on a channel to complete */ +void gsi_channel_trans_quiesce (struct gsi *gsi,u32 channel_id) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; +struct gsi_trans_info *trans_info; +struct gsi_trans *trans =NULL ; +struct gsi_evt_ring *evt_ring; +struct list_head *list; +unsigned long flags; + + trans_info = &channel->trans_info; + evt_ring = &channel->gsi->evt_ring[channel->evt_ring_id]; + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +/* Find the last list to which a transaction was added */ +if (!list_empty (&trans_info->alloc)) + list = &trans_info->alloc; +else if (!list_empty (&trans_info->pending)) + list = &trans_info->pending; +else if (!list_empty (&trans_info->complete)) + list = &trans_info->complete; +else if (!list_empty (&trans_info->polled)) + list = &trans_info->polled; +else + list =NULL ; + +if (list) { +struct gsi_trans *trans; + +/* The last entry on this list is the last one allocated. +* Grab a reference so we can wait for it. drivers/net/ipa/gsi.c line 1636
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+*/ + trans = list_last_entry(list, struct gsi_trans, links); +refcount_inc (&trans->refcount); + } + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +/* If there is one, wait for it to complete */ +if (trans) { +wait_for_completion (&trans->completion);Since you are waiting here, you clearly can't be called from interrupt context, or with interrupts disabled, so it's clearer to use
spin_lock_irq()instead ofspin_lock_irqsave().
Quoted 3 lines
I generally try to avoid the
_irqsaveversions altogether, unless it is really needed for a function that is called both from irq-disabled and irq-enabled context.
OK. And I appreciate what your saying here because I do prefer code that communicates more about the context in ways like you describe.
Thanks you.
-Alex
Quoted 3 lines
Arnd
Re: [PATCH 08/18] soc: qcom: ipa: the generic software interface
Quoted 50 lines · Alex Elder
Quoted 47 lines · Arnd Bergmann
Quoted 32 lines
drivers/net/ipa/gsi.c new file +32
-0+/* Wait for all transaction activity on a channel to complete */ +void gsi_channel_trans_quiesce (struct gsi *gsi,u32 channel_id) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; +struct gsi_trans_info *trans_info; +struct gsi_trans *trans =NULL ; +struct gsi_evt_ring *evt_ring; +struct list_head *list; +unsigned long flags; + + trans_info = &channel->trans_info; + evt_ring = &channel->gsi->evt_ring[channel->evt_ring_id]; + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +/* Find the last list to which a transaction was added */ +if (!list_empty (&trans_info->alloc)) + list = &trans_info->alloc; +else if (!list_empty (&trans_info->pending)) + list = &trans_info->pending; +else if (!list_empty (&trans_info->complete)) + list = &trans_info->complete; +else if (!list_empty (&trans_info->polled)) + list = &trans_info->polled; +else + list =NULL ; + +if (list) { +struct gsi_trans *trans; + +/* The last entry on this list is the last one allocated. +* Grab a reference so we can wait for it. drivers/net/ipa/gsi.c line 1636
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+*/ + trans = list_last_entry(list, struct gsi_trans, links); +refcount_inc (&trans->refcount); + } + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +/* If there is one, wait for it to complete */ +if (trans) { +wait_for_completion (&trans->completion);Since you are waiting here, you clearly can't be called from interrupt context, or with interrupts disabled, so it's clearer to use
spin_lock_irq()instead ofspin_lock_irqsave().
Quoted 7 lines
Quoted 3 lines
I generally try to avoid the
_irqsaveversions altogether, unless it is really needed for a function that is called both from irq-disabled and irq-enabled context.OK. And I appreciate what your saying here because I do prefer code that communicates more about the context in ways like you describe.
Right, also reading the status of the irq-enable flag can be expensive on some CPUs, so spin_lock_irqsave() ends up much more slower than spin_lock() or spin_lock_irq(). Not sure if it makes a huge difference on this particular platform, but it's better not to have to worry about it.
Arnd
Re: [PATCH 08/18] soc: qcom: ipa: the generic software interface
In reply to Alex ElderOn Sun, May 12, 2019 at 3:25 AM Alex Elder <[email protected]> wrote:
The per-event interrupt handling seems to be more complex than necessary:
Quoted 9 lines
drivers/net/ipa/gsi.c new file +9
-0+/* Enable or disable an event interrupt */ +static void +_gsi_irq_control_event (struct gsi *gsi,u32 evt_ring_id,bool enable) +{ +u32 mask =BIT (evt_ring_id); +u32 val; + +if (enable) + gsi->event_enable_bitmap |= mask;
drivers/net/ipa/gsi.c line 216
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+else + gsi->event_enable_bitmap &= ~mask; + + val = gsi->event_enable_bitmap; +iowrite32 (val, gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET ); +} + +static void gsi_irq_enable_event (struct gsi *gsi,u32 evt_ring_id) +{ +_gsi_irq_control_event (gsi, evt_ring_id, true);
You maintain a bitmap here of the enabled-state, and have to use a spinlock to ensure that the two are in sync.
Quoted 16 lines
drivers/net/ipa/gsi.c new file +16
-0+/* Returns true if the interrupt state (enabled or not) changed */ +static bool gsi_channel_intr (struct gsi_channel *channel,bool enable) +{ +u32 evt_ring_id = channel->evt_ring_id; +struct gsi *gsi = channel->gsi; +u32 mask =BIT (evt_ring_id); +unsigned long flags; +bool different; +u32 enabled; + +spin_lock_irqsave (&gsi->spinlock, flags); + + enabled = gsi->event_enable_bitmap & mask; + different = enable == !enabled; + +if (different) {
drivers/net/ipa/gsi.c line 432
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+if (enabled) +gsi_irq_disable_event (channel->gsi, evt_ring_id); +else +gsi_irq_enable_event (channel->gsi, evt_ring_id); + } + +spin_unlock_irqrestore (&gsi->spinlock, flags); + +return different; +}
This gets called for each active channel, so you repeatedly have to get the spinlock and read the irq-enabled state for it.
Quoted 6 lines
drivers/net/ipa/gsi.c new file +6
-0+static void gsi_isr_ieob (struct gsi *gsi) +{ +u32 evt_mask; + + evt_mask =ioread32 (gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_OFFSET ); + evt_mask &=ioread32 (gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET );
drivers/net/ipa/gsi.c line 468
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+iowrite32 (evt_mask, gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_CLR_OFFSET ); + +while (evt_mask) { +u32 evt_ring_id =__ffs (evt_mask); + + evt_mask ^=BIT (evt_ring_id); + +gsi_event_handle (gsi, evt_ring_id); + } +}
However, you start out by clearing all bits here.
Why not skip the clearing and and leave the interrupts enabled, while moving the GSI_CNTXT_SRC_IEOB_IRQ_CLR_OFFSET write (for a single channel that was completed) to the end of gsi_channel_poll()?
Something like
static void gsi_isr_ieob(struct gsi *gsi)
{
u32 evt_mask;
evt_mask = ioread32(gsi->virt + GSI_CNTXT_SRC_IEOB_IRQ_OFFSET);
while (evt_mask) {
u32 evt_ring_id = __ffs(evt_mask);
evt_mask ^= BIT(evt_ring_id);
napi_schedule(gsi->evt_ring[evt_ring_id].channel.napi);
}
}I also removed the GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET
read here, as that is probably more expensive than calling napi_schedule() for a channel that is already scheduled. Most of the time, I'd expect the interrupt to only signal a single channel anyway.
Arnd
Re: [PATCH 08/18] soc: qcom: ipa: the generic software interface
Quoted 4 lines · Arnd Bergmann
On Sun, May 12, 2019 at 3:25 AM Alex Elder <[email protected]> wrote:
The per-event interrupt handling seems to be more complex than necessary:
I just noticed this message. I'll take another look at this whole interrupt control mechanism and will try to streamline it along the lines of what you describe.
Thanks.
-Alex
Quoted 100 lines
Quoted 10 lines
drivers/net/ipa/gsi.c new file +9
-0+/* Enable or disable an event interrupt */ +static void +_gsi_irq_control_event (struct gsi *gsi,u32 evt_ring_id,bool enable) +{ +u32 mask =BIT (evt_ring_id); +u32 val; + +if (enable) + gsi->event_enable_bitmap |= mask;drivers/net/ipa/gsi.c line 216
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+else + gsi->event_enable_bitmap &= ~mask; + + val = gsi->event_enable_bitmap; +iowrite32 (val, gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET ); +} + +static void gsi_irq_enable_event (struct gsi *gsi,u32 evt_ring_id) +{ +_gsi_irq_control_event (gsi, evt_ring_id, true);You maintain a bitmap here of the enabled-state, and have to use a spinlock to ensure that the two are in sync.
Quoted 16 lines
drivers/net/ipa/gsi.c new file +16
-0+/* Returns true if the interrupt state (enabled or not) changed */ +static bool gsi_channel_intr (struct gsi_channel *channel,bool enable) +{ +u32 evt_ring_id = channel->evt_ring_id; +struct gsi *gsi = channel->gsi; +u32 mask =BIT (evt_ring_id); +unsigned long flags; +bool different; +u32 enabled; + +spin_lock_irqsave (&gsi->spinlock, flags); + + enabled = gsi->event_enable_bitmap & mask; + different = enable == !enabled; + +if (different) {drivers/net/ipa/gsi.c line 432
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+if (enabled) +gsi_irq_disable_event (channel->gsi, evt_ring_id); +else +gsi_irq_enable_event (channel->gsi, evt_ring_id); + } + +spin_unlock_irqrestore (&gsi->spinlock, flags); + +return different; +}This gets called for each active channel, so you repeatedly have to get the spinlock and read the irq-enabled state for it.
Quoted 6 lines
drivers/net/ipa/gsi.c new file +6
-0+static void gsi_isr_ieob (struct gsi *gsi) +{ +u32 evt_mask; + + evt_mask =ioread32 (gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_OFFSET ); + evt_mask &=ioread32 (gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET );drivers/net/ipa/gsi.c line 468
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+iowrite32 (evt_mask, gsi->virt +GSI_CNTXT_SRC_IEOB_IRQ_CLR_OFFSET ); + +while (evt_mask) { +u32 evt_ring_id =__ffs (evt_mask); + + evt_mask ^=BIT (evt_ring_id); + +gsi_event_handle (gsi, evt_ring_id); + } +}However, you start out by clearing all bits here.
Why not skip the clearing and and leave the interrupts enabled, while moving the
GSI_CNTXT_SRC_IEOB_IRQ_CLR_OFFSETwrite (for a single channel that was completed) to the end ofgsi_channel_poll()?Something like
static void
gsi_isr_ieob(struct gsi *gsi){ u32 evt_mask; evt_mask = ioread32(gsi->virt + GSI_CNTXT_SRC_IEOB_IRQ_OFFSET); while (evt_mask) { u32 evt_ring_id = __ffs(evt_mask); evt_mask ^= BIT(evt_ring_id); napi_schedule(gsi->evt_ring[evt_ring_id].channel.napi); } }I also removed the
GSI_CNTXT_SRC_IEOB_IRQ_MSK_OFFSET
read here, as that is probably more expensive than callingnapi_schedule()for a channel that is already scheduled. Most of the time, I'd expect the interrupt to only signal a single channel anyway.Arnd
Re: [PATCH 08/18] soc: qcom: ipa: the generic software interface
In reply to Alex ElderQuoted 7 lines · Alex Elder
drivers/net/ipa/gsi.c new file +7
-0+static int gsi_ring_alloc (struct gsi *gsi,struct gsi_ring *ring,u32 count) +{ +size_t size =roundup_pow_of_two (count *sizeof (struct gsi_tre )); +dma_addr_t addr; + +/* Hardware requires a power-of-2 ring size (and alignment) */ + ring->virt =dma_alloc_coherent (gsi->dev, size, &addr,GFP_KERNEL );
drivers/net/ipa/gsi.c line 799
Quoted 10 lines
drivers/net/ipa/gsi.c new file +10
-0+if (!ring->virt) +return -ENOMEM ; + ring->addr = addr; + ring->base = addr &GENMASK (31 ,0 ); + ring->size = size; + ring->end = ring->base + size; +spin_lock_init (&ring->spinlock); + +return 0 ; +}
Another comment for this patch: dma_alloc_coherent() does not guarantee alignment of the requested buffer as implied by the comment. In many configurations, it /is/ naturally aligned because the buffer comes from alloc_pages(), but you can't really be sure.
I suspect it's actually only broken when the buffer spans a 4GB boundary (and updating the lower 32 bit in the register gives a wrong pointer), which is unlikely but will happen at some point according to Murphy's law. If you just need the dma_addr_t to not cross a 4GB boundary, the easiest solution would be to use GFP_DMA32, which gives you a buffer that is mapped to the first 4GB bus address space (not necessarily the first 4GB of RAM if you have an iommu).
If you manually align the ring buffer, it should be fine too, though I have to say that the way the driver does pointer arithmetic on 32-bit integers seems rather fragile as well.
A nicer way to deal with ring buffers in general is to only ever use a 32-bit index number stored in an atomic_t, use atomic_inc_return() to advance the index and then mask the number when turning it into an index. With that, you should also be able to avoid the shared spinlock. Moving the rp and wp into separate cache lines further reduces the coherency traffic by avoiding concurrent writes on the same line.
Arnd
PATCH 9/18 soc: qcom: ipa: GSI transactions
In reply to Alex ElderThis patch implements GSI transactions. A GSI transaction is a structure that represents a single request (consisting of one or more TREs) sent to the GSI hardware. The last TRE in a transaction includes a flag requesting that the GSI interrupt the AP to notify that it has completed.
TREs are executed and completed strictly in order. For this reason, the completion of a single TRE implies that all previous TREs (in particular all of those "earlier" in a transaction) have completed.
Whenever there is a need to send a request (a set of TREs) to the IPA, a GSI transaction is allocated, specifying the number of TREs that will be required. Details of the request (e.g. transfer offsets and length) are represented by in a Linux scatterlist array that is incorporated in the transaction structure.
Once "filled," the transaction is committed. The GSI transaction layer performs all needed mapping (and unmapping) for DMA, and issues the request to the hardware. When the hardware signals that the request has completed, a function in the IPA layer is called, allowing for cleanup or followup activity to be performed before the transaction is freed.
--- drivers/net/ipa/gsi_trans.c | 604 ++++++++++++++++++++++++++++++++++++ drivers/net/ipa/gsi_trans.h | 106 +++++++ 2 files changed, 710 insertions(+) create mode 100644 drivers/net/ipa/gsi_trans.c create mode 100644 drivers/net/ipa/gsi_trans.h
drivers/net/ipa/gsi_trans.c new file +604 -0
@@ -0,0 +1,604 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#include <linux/types.h> +#include <linux/bits.h> +#include <linux/bitfield.h> +#include <linux/refcount.h> +#include <linux/scatterlist.h> + +#include "gsi.h" +#include "gsi_private.h" +#include "gsi_trans.h" +#include "ipa_gsi.h" +#include "ipa_data.h" +#include "ipa_cmd.h" + +/** +* DOC: GSI Transactions +* +* A GSI transaction abstracts the behavior of a GSI channel by representing +* everything about a related group of data transfers in a single structure. +* Most details of interaction with the GSI hardware are managed by the GSI +* transaction core, allowing users to simply describe transfers to be +* performed and optionally supply a callback function to run once the set +* of transfers has been completed. +* +* To perform a data transfer (or a related set of them), a user of the GSI +* transaction interface allocates a transaction, indicating the number of +* TREs required (one per data transfer). If sufficient TREs are available, +* they are reserved for use in the transaction and the allocation succeeds. +* This way exhaustion of the available TREs in a channel ring is detected +* as early as possible. All resources required to complete a transaction +* are allocated at transaction allocation time. +* +* Transfers performed as part of a transaction are represented in an array +* of Linux scatterlist structures. This array is allocated with the +* transaction, and its entries must be initialized using standard +* scatterlist functions (such as sg_init_one() or skb_to_sgvec()). The +* user must supply the total number of bytes represented by all transfers +* in the transaction. +* +* Once a transaction has been prepared, it is committed. The GSI transaction +* layer is responsible for DMA mapping (and unmapping) memory described in +* the transaction's scatterlist array. The only way committing a transaction +* fails is if this DMA mapping step returns an error. Otherwise, ownership +* of the entire transaction is transferred to the GSI transaction core. The +* GSI transaction code formats the content of the scatterlist array into the +* channel ring buffer and informs the hardware that new TREs are available +* to process. +* +* The last TRE in each transaction is marked to interrupt the AP when the +* GSI hardware has completed it. Because transfers described by TREs are +* performed strictly in order, signaling the completion of just the last +* TRE in the transaction is sufficient to indicate the full transaction +* is complete. +* +* When a transaction is complete, ipa_gsi_trans_complete() is called by the +* GSI code into the IPA layer, allowing it to perform any final cleanup +* required before the transaction is freed. +* +* It is possible to await the completion of a transaction; only immediate +* commands currently use this functionality. +*/ + +/* gsi_tre->flags mask values (in CPU byte order) */ +#define GSI_TRE_FLAGS_CHAIN_FMASK GENMASK(0, 0) +#define GSI_TRE_FLAGS_IEOB_FMASK GENMASK(8, 8) +#define GSI_TRE_FLAGS_IEOT_FMASK GENMASK(9, 9) +#define GSI_TRE_FLAGS_BEI_FMASK GENMASK(10, 10) +#define GSI_TRE_FLAGS_TYPE_FMASK GENMASK(23, 16) + +/* Hardware values representing a transfer element type */ +enum gsi_tre_type { +GSI_RE_XFER =0x2 , +GSI_RE_IMMD_CMD =0x3 , +GSI_RE_NOP =0x4 , +}; + +/* Map a given ring entry index to the transaction associated with it */ +static void gsi_channel_trans_map (struct gsi_channel *channel,u32 index, +struct gsi_trans *trans) +{ + channel->trans_info.map[index] = trans; +} + +/* Return the transaction mapped to a given ring entry */ +struct gsi_trans * +gsi_channel_trans_mapped (struct gsi_channel *channel,u32 index) +{ +return channel->trans_info.map[index]; +} + +/* Return the oldest completed transaction for a channel (or null) */ +struct gsi_trans *gsi_channel_trans_complete (struct gsi_channel *channel) +{ +return list_first_entry_or_null (&channel->trans_info.complete, + struct gsi_trans, links); +} + +/* Move a transaction from the allocated list to the pending list */ +static void gsi_trans_move_pending (struct gsi_trans *trans) +{ +struct gsi_channel *channel = &trans->gsi->channel[trans->channel_id]; +struct gsi_trans_info *trans_info = &channel->trans_info; +unsigned long flags; + +spin_lock_irqsave (&trans_info->spinlock, flags); + +list_move_tail (&trans->links, &trans_info->pending); + +spin_unlock_irqrestore (&trans_info->spinlock, flags); +} + +/* Move a transaction and all of its predecessors from the pending list +* to the completed list. +*/ +void gsi_trans_move_complete (struct gsi_trans *trans) +{ +struct gsi_channel *channel = &trans->gsi->channel[trans->channel_id]; +struct gsi_trans_info *trans_info; +struct list_head list; +unsigned long flags; + + trans_info = &channel->trans_info; + +spin_lock_irqsave (&trans_info->spinlock, flags); + +/* Move this transaction and all predecessors to completed list */ +list_cut_position (&list, &trans_info->pending, &trans->links); +list_splice_tail (&list, &trans_info->complete); + +spin_unlock_irqrestore (&trans_info->spinlock, flags); +} + +/* Move a transaction from the completed list to the polled list */ +void gsi_trans_move_polled (struct gsi_trans *trans) +{ +struct gsi_channel *channel = &trans->gsi->channel[trans->channel_id]; +struct gsi_trans_info *trans_info = &channel->trans_info; +unsigned long flags; + +spin_lock_irqsave (&trans_info->spinlock, flags); + +list_move_tail (&trans->links, &trans_info->polled); + +spin_unlock_irqrestore (&trans_info->spinlock, flags); +} + +/* Allocate a GSI transaction on a channel */ +struct gsi_trans * +gsi_channel_trans_alloc (struct gsi *gsi,u32 channel_id,u32 tre_count) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; +struct gsi_trans_info *trans_info = &channel->trans_info; +struct gsi_trans *trans =NULL ; +unsigned long flags; + +/* Caller should know the limit is gsi_channel_trans_max() */ +if (WARN_ON (tre_count > channel->data->tlv_count)) +return NULL ; + +spin_lock_irqsave (&trans_info->spinlock, flags); + +if (trans_info->tre_avail >= tre_count) { +u32 avail; + +/* Allocate the transaction */ +if (trans_info->pool_free == trans_info->pool_count) + trans_info->pool_free =0 ; + trans = &trans_info->pool[trans_info->pool_free++]; + +/* Allocate the scatter/gather entries it will use. If +* what's needed would cross the end-of-pool boundary, +* allocate them from the beginning. +*/ + avail = trans_info->sg_pool_count - trans_info->sg_pool_free; +if (tre_count > avail) + trans_info->sg_pool_free =0 ; + trans->sgl = &trans_info->sg_pool[trans_info->sg_pool_free]; + trans->sgc = tre_count; + trans_info->sg_pool_free += tre_count; + +/* We reserve the TREs now, but consume them at commit time */ + trans_info->tre_avail -= tre_count; + +list_add_tail (&trans->links, &trans_info->alloc); + } + +spin_unlock_irqrestore (&trans_info->spinlock, flags); + +if (!trans) +return NULL ; + + trans->gsi = gsi; + trans->channel_id = channel_id; +refcount_set (&trans->refcount,1 ); + trans->tre_count = tre_count; +init_completion (&trans->completion); + +/* We're reusing, so make sure all fields are reinitialized */ + trans->dev = gsi->dev; + trans->result =0 ;/* Success assumed unless overwritten */ + trans->data =NULL ; + +return trans; +} + +/* Free a previously-allocated transaction (used only in case of error) */ +void gsi_trans_free (struct gsi_trans *trans) +{ +struct gsi_trans_info *trans_info; +struct gsi_channel *channel; +unsigned long flags; + +if (!refcount_dec_and_test (&trans->refcount)) +return ; + + channel = &trans->gsi->channel[trans->channel_id]; + trans_info = &channel->trans_info; + +spin_lock_irqsave (&trans_info->spinlock, flags); + +list_del (&trans->links); + trans_info->tre_avail += trans->tre_count; + +spin_unlock_irqrestore (&trans_info->spinlock, flags); +} + +/* Compute the length/opcode value to use for a TRE */ +static __le16 gsi_tre_len_opcode (enum ipa_cmd_opcode opcode,u32 len) +{ +return opcode ==IPA_CMD_NONE ?cpu_to_le16 ((u16 )len) + :cpu_to_le16 ((u16 )opcode); +} + +/* Compute the flags value to use for a given TRE */ +static __le32 gsi_tre_flags (bool last_tre,bool bei,enum ipa_cmd_opcode opcode) +{ +enum gsi_tre_type tre_type; +u32 tre_flags; + + tre_type = opcode ==IPA_CMD_NONE ?GSI_RE_XFER :GSI_RE_IMMD_CMD ; + tre_flags =u32_encode_bits (tre_type,GSI_TRE_FLAGS_TYPE_FMASK ); + +/* Last TRE contains interrupt flags */ +if (last_tre) { +/* All transactions end in a transfer completion interrupt */ + tre_flags |=GSI_TRE_FLAGS_IEOT_FMASK ; +/* Don't interrupt when outbound commands are acknowledged */ +if (bei) + tre_flags |=GSI_TRE_FLAGS_BEI_FMASK ; + }else {/* All others indicate there's more to come */ + tre_flags |=GSI_TRE_FLAGS_CHAIN_FMASK ; + } + +return cpu_to_le32 (tre_flags); +} + +static void gsi_trans_tre_fill (struct gsi_tre *dest_tre,dma_addr_t addr, +u32 len,bool last_tre,bool bei, +enum ipa_cmd_opcode opcode) +{ +struct gsi_tre tre; + + tre.addr =cpu_to_le64 (addr); + tre.len_opcode =gsi_tre_len_opcode (opcode, len); + tre.reserved =0 ; + tre.flags =gsi_tre_flags (last_tre, bei, opcode); + + *dest_tre = tre;/* Write TRE as a single (16-byte) unit */ +} + +int gsi_trans_read_byte (struct gsi *gsi,u32 channel_id,dma_addr_t addr) +{ +struct gsi_channel *channel = &gsi->channel[channel_id]; +struct gsi_trans_info *trans_info; +struct gsi_evt_ring *evt_ring; +struct gsi_tre *dest_tre; +bool exhausted = false; +unsigned long flags; +u32 wp_local; + +/* assert(!channel->toward_ipa); */ + + trans_info = &channel->trans_info; +spin_lock_irqsave (&trans_info->spinlock, flags); + +if (trans_info->tre_avail) + trans_info->tre_avail--; +else + exhausted = true; + +spin_unlock_irqrestore (&trans_info->spinlock, flags); + +if (exhausted) +return -EBUSY ; + + evt_ring = &gsi->evt_ring[channel->evt_ring_id]; +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + + wp_local = channel->tre_ring.wp_local; +if (wp_local == channel->tre_ring.end) + wp_local = channel->tre_ring.base; + dest_tre =gsi_ring_virt (&channel->tre_ring, wp_local); + +gsi_trans_tre_fill (dest_tre, addr,1 , true, false,IPA_CMD_NONE ); + +gsi_ring_wp_local_add (&channel->tre_ring,1 ); + +gsi_channel_doorbell (channel); + +spin_unlock_irqrestore (&evt_ring->ring.spinlock, flags); + +return 0 ; +} + +/** +* __gsi_trans_commit() - Common GSI transaction commit code +* @trans: Transaction to commit +* @opcode: Immediate command opcode, or IPA_CMD_NONE +* @ring_db: Whether to tell the hardware about these queued transfers +* +* @Return: 0 if successful, or a negative error code +* +* Maps the transactions's scatterlist array for DMA, and returns -ENOMEM +* if that fails. Formats channel ring TRE entries based on the content of +* the scatterlist. Records the transaction pointer in the map entry for +* the last ring entry used for the transaction so it can be recovered when +* it completes, and moves the transaction to the pending list. Updates the +* channel ring pointer and optionally rings the doorbell. +*/ +static int __gsi_trans_commit (struct gsi_trans *trans, +enum ipa_cmd_opcode opcode,bool ring_db) +{ +struct gsi_channel *channel = &trans->gsi->channel[trans->channel_id]; +enum dma_data_direction direction; +bool bei = channel->toward_ipa; +struct gsi_evt_ring *evt_ring; +struct scatterlist *sg; +struct gsi_tre *dest_tre; +u32 queued_trans_count; +u32 queued_byte_count; +unsigned long flags; +u32 byte_count =0 ; +u32 wp_local; +u32 index; +u32 avail; +int ret; +u32 i; + + direction = channel->toward_ipa ?DMA_TO_DEVICE :DMA_FROM_DEVICE ; + ret =dma_map_sg (trans->dev, trans->sgl, trans->sgc, direction); +if (!ret) +return -ENOMEM ; + + evt_ring = &channel->gsi->evt_ring[channel->evt_ring_id]; + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +/* We'll consume the entries available at the end of the ring, +* switching to the beginning to finish if necessary. +*/ + wp_local = channel->tre_ring.wp_local; + dest_tre =gsi_ring_virt (&channel->tre_ring, wp_local); + + avail = (channel->tre_ring.end - wp_local) /sizeof (*dest_tre); + +for_each_sg (trans->sgl, sg, trans->sgc, i) { +bool last_tre = i == trans->tre_count -1 ; +dma_addr_t addr =sg_dma_address (sg); +u32 len =sg_dma_len (sg); + + byte_count += len; +if (!avail--) + dest_tre =gsi_ring_virt (&channel->tre_ring, + channel->tre_ring.base); + +gsi_trans_tre_fill (dest_tre, addr, len, last_tre, bei, opcode); + dest_tre++; + } + +if (channel->toward_ipa) { +/* We record TX bytes when they are sent */ + trans->len = byte_count; + trans->trans_count = channel->trans_count; + trans->byte_count = channel->byte_count; + channel->trans_count++; + channel->byte_count += byte_count; + } + +/* Advance the write pointer; record info for last used element */ +gsi_ring_wp_local_add (&channel->tre_ring, trans->tre_count -1 ); + index =ring_index (&channel->tre_ring, channel->tre_ring.wp_local); +gsi_channel_trans_map (channel, index, trans); +gsi_ring_wp_local_add (&channel->tre_ring,1 ); + +gsi_trans_move_pending (trans); + +/* Ring doorbell if requested, or if all TREs are allocated */ +if (ring_db || !channel->trans_info.tre_avail) { +/* Report what we're handing off to hardware for TX channels */ +if (channel->toward_ipa) { + queued_trans_count = channel->trans_count - + channel->doorbell_trans_count; + queued_byte_count = channel->byte_count - + channel->doorbell_byte_count; + channel->doorbell_trans_count = channel->trans_count; + channel->doorbell_byte_count = channel->byte_count; + +ipa_gsi_channel_tx_queued (trans->gsi, +gsi_channel_id (channel), + queued_trans_count, + queued_byte_count); + } + +gsi_channel_doorbell (channel); + } + +spin_unlock_irqrestore (&evt_ring->ring.spinlock, flags); + +return 0 ; +} + +/* Commit a GSI transaction */ +int gsi_trans_commit (struct gsi_trans *trans,bool ring_db) +{ +return __gsi_trans_commit (trans,IPA_CMD_NONE , ring_db); +} + +/* Commit a GSI command transaction and wait for it to complete */ +int gsi_trans_commit_command (struct gsi_trans *trans, +enum ipa_cmd_opcode opcode) +{ +int ret; + +refcount_inc (&trans->refcount); + + ret =__gsi_trans_commit (trans, opcode, true); +if (ret) + goto out_free_trans; + +wait_for_completion (&trans->completion); + +out_free_trans: +gsi_trans_free (trans); + +return ret; +} + +/* Commit a GSI command transaction, wait for it to complete, with timeout */ +int gsi_trans_commit_command_timeout (struct gsi_trans *trans, +enum ipa_cmd_opcode opcode, +unsigned long timeout) +{ +unsigned long timeout_jiffies =msecs_to_jiffies (timeout); +unsigned long remaining; +int ret; + +refcount_inc (&trans->refcount); + + ret =__gsi_trans_commit (trans, opcode, true); +if (ret) + goto out_free_trans; + + remaining =wait_for_completion_timeout (&trans->completion, + timeout_jiffies); +out_free_trans: +gsi_trans_free (trans); + +return ret ? ret : remaining ?0 : -ETIMEDOUT ; +} + +/* Return a channel's next completed transaction (or NULL) */ +void gsi_trans_complete (struct gsi_trans *trans) +{ +struct gsi_channel *channel = &trans->gsi->channel[trans->channel_id]; +enum dma_data_direction direction; + + direction = channel->toward_ipa ?DMA_TO_DEVICE :DMA_FROM_DEVICE ; + +dma_unmap_sg (trans->dev, trans->sgl, trans->sgc, direction); + +ipa_gsi_trans_complete (trans); + +complete (&trans->completion); + +gsi_trans_free (trans); +} + +/* Cancel a channel's pending transactions */ +void gsi_channel_trans_cancel_pending (struct gsi_channel *channel) +{ +struct gsi_trans_info *trans_info = &channel->trans_info; +u32 evt_ring_id = channel->evt_ring_id; +struct gsi *gsi = channel->gsi; +struct gsi_evt_ring *evt_ring; +struct gsi_trans *trans; +unsigned long flags; + + evt_ring = &gsi->evt_ring[evt_ring_id]; + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +list_for_each_entry (trans, &trans_info->pending, links) + trans->result = -ECANCELED ; + +list_splice_tail_init (&trans_info->pending, &trans_info->complete); + +spin_unlock_irqrestore (&evt_ring->ring.spinlock, flags); + +spin_lock_irqsave (&gsi->spinlock, flags); + +if (gsi->event_enable_bitmap &BIT (evt_ring_id)) +gsi_event_handle (gsi, evt_ring_id); + +spin_unlock_irqrestore (&gsi->spinlock, flags); +} + +/* Initialize a channel's GSI transaction info */ +int gsi_channel_trans_init (struct gsi_channel *channel) +{ +struct gsi_trans_info *trans_info = &channel->trans_info; +u32 tre_count = channel->data->tre_count; + + trans_info->map =kcalloc (tre_count,sizeof (*trans_info->map), +GFP_KERNEL ); +if (!trans_info->map) +return -ENOMEM ; + +/* We will never need more transactions than there are TRE +* entries in the transfer ring. For that reason, we can +* preallocate an array of (at least) that many transactions, +* and use a single free index to determine the next one +* available for allocation. +*/ + trans_info->pool_count = tre_count; + trans_info->pool =kcalloc (trans_info->pool_count, +sizeof (*trans_info->pool),GFP_KERNEL ); +if (!trans_info->pool) + goto err_free_map; +/* If we get extra memory from the allocator, use it */ + trans_info->pool_count = +ksize (trans_info->pool) /sizeof (*trans_info->pool); + trans_info->pool_free =0 ; + +/* While transactions are allocated one at a time, a transaction +* can have multiple TREs. The number of TRE entries in a single +* transaction is limited by the number of TLV FIFO entries the +* channel has. We reserve TREs when a transaction is allocated, +* but we don't actually use them until the transaction is +* committed. +* +* A transaction uses a scatterlist array to represent the data +* transfers implemented by the transaction. Each scatterliest +* element is used to fill a single TRE when the transaction is +* committed. As a result, we need the same number of scatterlist +* elements as there are TREs in the transfer ring, and we can +* preallocate them in a pool. +* +* If we allocate a few (tlv_count - 1) extra entries in our pool +* we can always satisfy requests without ever worrying about +* straddling the end of the array. If there aren't enough +* entries starting at the free index, we just allocate free +* entries from the beginning of the pool. +*/ + trans_info->sg_pool_count = tre_count + channel->data->tlv_count -1 ; + trans_info->sg_pool =kcalloc (trans_info->sg_pool_count, +sizeof (*trans_info->sg_pool),GFP_KERNEL ); +if (!trans_info->sg_pool) + goto err_free_pool; +/* Use any extra memory we get from the allocator */ + trans_info->sg_pool_count = +ksize (trans_info->sg_pool) /sizeof (*trans_info->sg_pool); + trans_info->sg_pool_free =0 ; + +spin_lock_init (&trans_info->spinlock); + trans_info->tre_avail = tre_count;/* maximum active */ +INIT_LIST_HEAD (&trans_info->alloc); +INIT_LIST_HEAD (&trans_info->pending); +INIT_LIST_HEAD (&trans_info->complete); +INIT_LIST_HEAD (&trans_info->polled); + +return 0 ; + +err_free_pool: +kfree (trans_info->pool); +err_free_map: +kfree (trans_info->map); + +return -ENOMEM ; +} + +/* Inverse of gsi_channel_trans_init() */ +void gsi_channel_trans_exit (struct gsi_channel *channel) +{ +struct gsi_trans_info *trans_info = &channel->trans_info; + +kfree (trans_info->sg_pool); +kfree (trans_info->pool); +kfree (trans_info->map); +}
drivers/net/ipa/gsi_trans.h new file +106 -0
106 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
drivers/net/ipa/gsi_trans.c line 274
Quoted 13 lines
drivers/net/ipa/gsi_trans.c new file +13
-0+static void gsi_trans_tre_fill (struct gsi_tre *dest_tre,dma_addr_t addr, +u32 len,bool last_tre,bool bei, +enum ipa_cmd_opcode opcode) +{ +struct gsi_tre tre; + + tre.addr =cpu_to_le64 (addr); + tre.len_opcode =gsi_tre_len_opcode (opcode, len); + tre.reserved =0 ; + tre.flags =gsi_tre_flags (last_tre, bei, opcode); + + *dest_tre = tre;/* Write TRE as a single (16-byte) unit */ +}
Have you checked that the atomic write is actually what happens here, but looking at the compiler output? You might need to add a 'volatile' qualifier to the dest_tre argument so the temporary structure doesn't get optimized away here.
Quoted 18 lines
drivers/net/ipa/gsi_trans.c new file +18
-0+/* Cancel a channel's pending transactions */ +void gsi_channel_trans_cancel_pending (struct gsi_channel *channel) +{ +struct gsi_trans_info *trans_info = &channel->trans_info; +u32 evt_ring_id = channel->evt_ring_id; +struct gsi *gsi = channel->gsi; +struct gsi_evt_ring *evt_ring; +struct gsi_trans *trans; +unsigned long flags; + + evt_ring = &gsi->evt_ring[evt_ring_id]; + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +list_for_each_entry (trans, &trans_info->pending, links) + trans->result = -ECANCELED ; + +list_splice_tail_init (&trans_info->pending, &trans_info->complete);
drivers/net/ipa/gsi_trans.c line 520
Quoted 10 lines
drivers/net/ipa/gsi_trans.c new file +10
-0+ +spin_unlock_irqrestore (&evt_ring->ring.spinlock, flags); + +spin_lock_irqsave (&gsi->spinlock, flags); + +if (gsi->event_enable_bitmap &BIT (evt_ring_id)) +gsi_event_handle (gsi, evt_ring_id); + +spin_unlock_irqrestore (&gsi->spinlock, flags); +}
That is a lot of irqsave()/irqrestore() operations. Do you actually call all of these functions from hardirq context?
Arnd
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 10 lines · Arnd Bergmann
Quoted 10 lines
drivers/net/ipa/gsi_trans.c new file +10
-0+static void gsi_trans_tre_fill (struct gsi_tre *dest_tre,dma_addr_t addr, +u32 len,bool last_tre,bool bei, +enum ipa_cmd_opcode opcode) +{ +struct gsi_tre tre; + + tre.addr =cpu_to_le64 (addr); + tre.len_opcode =gsi_tre_len_opcode (opcode, len); + tre.reserved =0 ; + tre.flags =gsi_tre_flags (last_tre, bei, opcode);
Quoted 8 lines
drivers/net/ipa/gsi_trans.c line 274
Quoted 3 lines
drivers/net/ipa/gsi_trans.c new file +3
-0+ + *dest_tre = tre;/* Write TRE as a single (16-byte) unit */ +}Have you checked that the atomic write is actually what happens here, but looking at the compiler output? You might need to add a 'volatile' qualifier to the
dest_treargument so the temporary structure doesn't get optimized away here.
No, and I really should have checked, since I'm assuming that's what will happen. I will check, and may well add the volatile regardless.
Quoted 23 lines
Quoted 23 lines
drivers/net/ipa/gsi_trans.c new file +23
-0+/* Cancel a channel's pending transactions */ +void gsi_channel_trans_cancel_pending (struct gsi_channel *channel) +{ +struct gsi_trans_info *trans_info = &channel->trans_info; +u32 evt_ring_id = channel->evt_ring_id; +struct gsi *gsi = channel->gsi; +struct gsi_evt_ring *evt_ring; +struct gsi_trans *trans; +unsigned long flags; + + evt_ring = &gsi->evt_ring[evt_ring_id]; + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +list_for_each_entry (trans, &trans_info->pending, links) + trans->result = -ECANCELED ; + +list_splice_tail_init (&trans_info->pending, &trans_info->complete); + +spin_unlock_irqrestore (&evt_ring->ring.spinlock, flags); + +spin_lock_irqsave (&gsi->spinlock, flags); +
Quoted 8 lines
drivers/net/ipa/gsi_trans.c line 520
Quoted 5 lines
drivers/net/ipa/gsi_trans.c new file +5
-0+if (gsi->event_enable_bitmap &BIT (evt_ring_id)) +gsi_event_handle (gsi, evt_ring_id); + +spin_unlock_irqrestore (&gsi->spinlock, flags); +}That is a lot of irqsave()/irqrestore() operations. Do you actually call all of these functions from hardirq context?
The transaction list is definitely updated in IRQ context, but I think it is no longer updated in hardirq context (the softirq was a recent change). This particular function is definitely not called in a hardirq context, so I can remove the irqsave/irqrestore.
I'll survey my spinlock use throughout the driver and will remove any irqsave/irqrestore used in non-hardirq contexts.
Thanks.
-Alex
Quoted 2 lines
Arnd
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 33 lines · Alex Elder
Quoted 23 lines · Arnd Bergmann
Quoted 23 lines
drivers/net/ipa/gsi_trans.c new file +23
-0+/* Cancel a channel's pending transactions */ +void gsi_channel_trans_cancel_pending (struct gsi_channel *channel) +{ +struct gsi_trans_info *trans_info = &channel->trans_info; +u32 evt_ring_id = channel->evt_ring_id; +struct gsi *gsi = channel->gsi; +struct gsi_evt_ring *evt_ring; +struct gsi_trans *trans; +unsigned long flags; + + evt_ring = &gsi->evt_ring[evt_ring_id]; + +spin_lock_irqsave (&evt_ring->ring.spinlock, flags); + +list_for_each_entry (trans, &trans_info->pending, links) + trans->result = -ECANCELED ; + +list_splice_tail_init (&trans_info->pending, &trans_info->complete); + +spin_unlock_irqrestore (&evt_ring->ring.spinlock, flags); + +spin_lock_irqsave (&gsi->spinlock, flags); +Quoted 8 lines
drivers/net/ipa/gsi_trans.c line 520
Quoted 5 lines
drivers/net/ipa/gsi_trans.c new file +5
-0+if (gsi->event_enable_bitmap &BIT (evt_ring_id)) +gsi_event_handle (gsi, evt_ring_id); + +spin_unlock_irqrestore (&gsi->spinlock, flags); +}That is a lot of irqsave()/irqrestore() operations. Do you actually call all of these functions from hardirq context?
Quoted 5 lines
The transaction list is definitely updated in IRQ context, but I think it is no longer updated in hardirq context (the softirq was a recent change). This particular function is definitely not called in a hardirq context, so I can remove the irqsave/irqrestore.
If you want to protect against concurrent softirqs, you still need spin_lock_bh(), which is cheaper than spin_lock_irqsave() but still requires writing to the shared cache line for the atomic update of the lock.
Quoted 2 lines
I'll survey my spinlock use throughout the driver and will remove any irqsave/irqrestore used in non-hardirq contexts.
Ok. I actually hope that most of the spinlocks can be removed from the data path entirely. I just replied on the ring.spinlock, which I think can go away and be replaced either with two atomic_t values (rp_local and wp_local only; 'wp' appears to be unused), or even just an smp_rmb()/ smp_wmb() pair for each access. The gsi register spinlock can probably be avoided as well if we stop disabling and renabling the interrupts as I suggested.
gsi_trans_info->spinlock is harder to get rid of unfortunately, as that would require changing the way you do the doubly linked lists.
Arnd
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
In reply to Arnd BergmannQuoted 9 lines · Arnd Bergmann
Quoted 9 lines
drivers/net/ipa/gsi_trans.c new file +9
-0+static void gsi_trans_tre_fill (struct gsi_tre *dest_tre,dma_addr_t addr, +u32 len,bool last_tre,bool bei, +enum ipa_cmd_opcode opcode) +{ +struct gsi_tre tre; + + tre.addr =cpu_to_le64 (addr); + tre.len_opcode =gsi_tre_len_opcode (opcode, len); + tre.reserved =0 ;
Quoted 8 lines
drivers/net/ipa/gsi_trans.c line 274
Quoted 4 lines
drivers/net/ipa/gsi_trans.c new file +4
-0+ tre.flags =gsi_tre_flags (last_tre, bei, opcode); + + *dest_tre = tre;/* Write TRE as a single (16-byte) unit */ +}Have you checked that the atomic write is actually what happens here, but looking at the compiler output? You might need to add a 'volatile' qualifier to the
dest_treargument so the temporary structure doesn't get optimized away here.
Currently, the assignment *does* become a "stp" instruction. But I don't know that we can *force* the compiler to write it as a pair of registers, so I'll soften the comment with "Attempt to write" or something similar.
To my knowledge, adding a volatile qualifier only prevents the compiler from performing funny optimizations, but that has no effect on whether the 128-bit assignment is made as a single unit. Do you know otherwise?
-Alex
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 25 lines · Alex Elder
Quoted 9 lines · Arnd Bergmann
Quoted 9 lines
drivers/net/ipa/gsi_trans.c new file +9
-0+static void gsi_trans_tre_fill (struct gsi_tre *dest_tre,dma_addr_t addr, +u32 len,bool last_tre,bool bei, +enum ipa_cmd_opcode opcode) +{ +struct gsi_tre tre; + + tre.addr =cpu_to_le64 (addr); + tre.len_opcode =gsi_tre_len_opcode (opcode, len); + tre.reserved =0 ;Quoted 8 lines
drivers/net/ipa/gsi_trans.c line 274
Quoted 4 lines
drivers/net/ipa/gsi_trans.c new file +4
-0+ tre.flags =gsi_tre_flags (last_tre, bei, opcode); + + *dest_tre = tre;/* Write TRE as a single (16-byte) unit */ +}Have you checked that the atomic write is actually what happens here, but looking at the compiler output? You might need to add a 'volatile' qualifier to the
dest_treargument so the temporary structure doesn't get optimized away here.Currently, the assignment *does* become a "stp" instruction. But I don't know that we can *force* the compiler to write it as a pair of registers, so I'll soften the comment with "Attempt to write" or something similar.
Quoted 4 lines
To my knowledge, adding a volatile qualifier only prevents the compiler from performing funny optimizations, but that has no effect on whether the 128-bit assignment is made as a single unit. Do you know otherwise?
I don't think it you can force the 128-bit assignment to be atomic, but marking 'dest_tre' should serve to prevent a specific optimization that replaces the function with
dest_tre->addr = ...
dest_tre->len_opcode = ...
dest_tre->reserved = ...
dest_tre->flags = ...which it might find more efficient than the stp and is equivalent when the pointer is not marked volatile. We also have the WRITE_ONCE() macro that can help prevent this, but it does not work reliably beyond 64 bit assignments.
Arnd
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 40 lines · Arnd Bergmann
Quoted 25 lines · Alex Elder
Quoted 9 lines · Arnd Bergmann
Quoted 9 lines
drivers/net/ipa/gsi_trans.c new file +9
-0+static void gsi_trans_tre_fill (struct gsi_tre *dest_tre,dma_addr_t addr, +u32 len,bool last_tre,bool bei, +enum ipa_cmd_opcode opcode) +{ +struct gsi_tre tre; + + tre.addr =cpu_to_le64 (addr); + tre.len_opcode =gsi_tre_len_opcode (opcode, len); + tre.reserved =0 ;Quoted 8 lines
drivers/net/ipa/gsi_trans.c line 274
Quoted 4 lines
drivers/net/ipa/gsi_trans.c new file +4
-0+ tre.flags =gsi_tre_flags (last_tre, bei, opcode); + + *dest_tre = tre;/* Write TRE as a single (16-byte) unit */ +}Have you checked that the atomic write is actually what happens here, but looking at the compiler output? You might need to add a 'volatile' qualifier to the
dest_treargument so the temporary structure doesn't get optimized away here.Currently, the assignment *does* become a "stp" instruction. But I don't know that we can *force* the compiler to write it as a pair of registers, so I'll soften the comment with "Attempt to write" or something similar.
Quoted 4 lines
To my knowledge, adding a volatile qualifier only prevents the compiler from performing funny optimizations, but that has no effect on whether the 128-bit assignment is made as a single unit. Do you know otherwise?
I don't think it you can force the 128-bit assignment to be atomic, but marking '
dest_tre' should serve to prevent a specific optimization that replaces the function withdest_tre->addr = ... dest_tre->len_opcode = ... dest_tre->reserved = ... dest_tre->flags = ...
Quoted 4 lines
which it might find more efficient than the stp and is equivalent when the pointer is not marked volatile. We also have the
WRITE_ONCE()macro that can help prevent this, but it does not work reliably beyond 64 bit assignments.
OK, I'll mark it volatile to avoid that potential result. Thanks.
-Alex
Quoted 3 lines
Arnd
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 36 lines · Alex Elder
Quoted 35 lines · Arnd Bergmann
Quoted 21 lines · Alex Elder
Quoted 9 lines · Arnd Bergmann
Quoted 9 lines
+static void gsi_trans_tre_fill(struct
gsi_tre*dest_tre,dma_addr_taddr, + u32 len, boollast_tre, bool bei, + enumipa_cmd_opcodeopcode) +{ + structgsi_tretre; + +
tre.addr =cpu_to_le64(addr); + tre.len_opcode =gsi_tre_len_opcode(opcode, len); + tre.reserved = 0; +
tre.flags =gsi_tre_flags(last_tre, bei, opcode); + +*dest_tre= tre; /* Write TRE as a single (16-byte)
unit */ +}Quoted 4 lines
Have you checked that the atomic write is actually what happens here, but looking at the compiler output? You might need to add a 'volatile' qualifier to the
dest_treargument so the temporary structure doesn't get optimized away here.Currently, the assignment *does* become a "stp" instruction. But I don't know that we can *force* the compiler to write it as a pair of registers, so I'll soften the comment with "Attempt to write" or something similar.
Quoted 4 lines
To my knowledge, adding a volatile qualifier only prevents the compiler from performing funny optimizations, but that has no effect on whether the 128-bit assignment is made as a single unit. Do you know otherwise?
I don't think it you can force the 128-bit assignment to be atomic, but marking '
dest_tre' should serve to prevent a specific optimization that replaces the function with
dest_tre->addr= ...dest_tre->len_opcode= ...dest_tre->reserved= ...dest_tre->flags= ...
Quoted 6 lines
Quoted 4 lines
which it might find more efficient than the stp and is equivalent when the pointer is not marked volatile. We also have the
WRITE_ONCE()macro that can help prevent this, but it does not work reliably beyond 64 bit assignments.OK, I'll mark it volatile to avoid that potential result.
OK I got interesting results so I wanted to report back.
The way it is currently written (no volatile qualifier) is the *only* way I get a 16-byte store instruction.
Specifically, with dest_tre having type (struct gsi_tre *):
*dest_tre = tre;
Produces this:
4ec: a9002013 stp x19, x8, [x0]
I attempted to make the assigned-to pointer volatile:
*(volatile struct gsi_tre *)dest_tre = tre;
But that apparently is interpreted as "assign things in the destination in exactly the way they were assigned above into the "tre" structure." Because it produced this:
748: f9000348 str x8, [x26] 74c: 7940c3e8 ldrh w8, [sp, #96] 750: 79001348 strh w8, [x26, #8] 754: 7940bbe8 ldrh w8, [sp, #92] 758: 79001748 strh w8, [x26, #10] 75c: b9405be8 ldr w8, [sp, #88] 760: b9000f48 str w8, [x26, #12]
From there I went back and changed the type of the dest_tre pointer parameter to (volatile struct gsi_tre *), and changed the the type of the values passed through that argument in the two callers to also have the volatile qualifier. This way there was no need to use a cast in the left-hand side. That too produced a string of separate assignments, not a single 128-bit one.
I also attempted this:
*dest_tre = *(volatile struct gsi_tre *)&tre;
And even this:
*(volatile struct gsi_tre *)dest_tre = *(volatile struct gsi_tre *)&tre;But neither produced a single "stp" instruction; all produced the same sequence of instructions above.
So it seems that I must *not* apply a volatile qualifier, because doing so restricts the compiler from making the single instruction optimization.
If I've missed something and you have another suggestion for me to try let me know and I'll try it.
-Alex
Quoted 8 lines
Thanks.
-Alex
Quoted 3 lines
Arnd
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 7 lines · Alex Elder
Quoted 2 lines · Alex Elder
Quoted 1 line · Arnd Bergmann
On Fri, May 17, 2019 at 8:08 PM Alex Elder <[email protected]>
So it seems that I must *not* apply a volatile qualifier, because doing so restricts the compiler from making the single instruction optimization.
Right, I guess that makes sense.
Quoted 2 lines
If I've missed something and you have another suggestion for me to try let me know and I'll try it.
A memcpy() might do the right thing as well. Another idea would be a cast to __int128 like
#ifdef CONFIG_ARCH_SUPPORTS_INT128
typedef __int128 tre128_t;
#else
typedef struct { __u64 a; __u64 b; } tre128_t;
#else
static inline void set_tre(struct gsi_tre *dest_tre, struct gs_tre *src_tre)
{
*(volatile tre128_t *)dest_tre = *(tre128_t *)src_tre;
}Arnd
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 9 lines · Arnd Bergmann
Quoted 7 lines · Alex Elder
Quoted 2 lines · Alex Elder
Quoted 1 line · Arnd Bergmann
On Fri, May 17, 2019 at 8:08 PM Alex Elder <[email protected]>
So it seems that I must *not* apply a volatile qualifier, because doing so restricts the compiler from making the single instruction optimization.
Quoted 6 lines
Right, I guess that makes sense.
Quoted 2 lines
If I've missed something and you have another suggestion for me to try let me know and I'll try it.
A
memcpy()might do the right thing as well. Another idea would
I find memcpy() does the right thing.
Quoted 1 line
be a cast to
__int128like
I find that my environment supports 128 bit integers. But...
Quoted 10 lines
#ifdef
CONFIG_ARCH_SUPPORTS_INT128
typedef__int128tre128_t;
#else
typedef struct {__u64a;__u64b; }tre128_t;
#elsestatic inline void
set_tre(struct gsi_tre *dest_tre, struct gs_tre *src_tre){ *(volatile tre128_t *)dest_tre = *(tre128_t *)src_tre; }
...this produces two 8-bit assignments. Could it be because it's implemented as two 64-bit values? I think so. Dropping the volatile qualifier produces a single "stp" instruction.
The only other thing I thought I could do to encourage the compiler to do the right thing is define the type (or variables) to have 128-bit alignment. And doing that for the original simple assignment didn't change the (desirable) outcome, but I don't think it's really necessary in this case, considering the single instruction uses two 64-bit registers.
I'm going to leave it as it was originally; it's the simplest:
*dest_tre = tre;
I added a comment about structuring the code this way with the intention of getting the single instruction. If a different compiler produces different result
-Alex
Quoted 3 lines
Arnd
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 30 lines · Alex Elder
Quoted 9 lines · Arnd Bergmann
Quoted 7 lines · Alex Elder
Quoted 2 lines · Alex Elder
Quoted 1 line · Arnd Bergmann
On Fri, May 17, 2019 at 8:08 PM Alex Elder <[email protected]>
So it seems that I must *not* apply a volatile qualifier, because doing so restricts the compiler from making the single instruction optimization.
Quoted 6 lines
Right, I guess that makes sense.
Quoted 2 lines
If I've missed something and you have another suggestion for me to try let me know and I'll try it.
A
memcpy()might do the right thing as well. Another idea wouldI find
memcpy()does the right thing.Quoted 1 line
be a cast to
__int128likeI find that my environment supports 128 bit integers. But...
Quoted 6 lines
#ifdef
CONFIG_ARCH_SUPPORTS_INT128
typedef__int128tre128_t;
#else
typedef struct {__u64a;__u64b; }tre128_t;
#else
Quoted 7 lines
Quoted 4 lines
static inline void
set_tre(struct gsi_tre *dest_tre, struct gs_tre *src_tre){ *(volatile tre128_t *)dest_tre = *(tre128_t *)src_tre; }...this produces two 8-bit assignments. Could it be because it's implemented as two 64-bit values? I think so. Dropping the volatile qualifier produces a single "stp" instruction.
I have no idea how two 8-bit assignments could do that, it sounds like a serious gcc bug, unless you mean two 8-byte assignments, which would be within the range of expected behavior. If it's actually 8-bit stores, please open a bug against gcc with a minimized test case.
Quoted 8 lines
The only other thing I thought I could do to encourage the compiler to do the right thing is define the type (or variables) to have 128-bit alignment. And doing that for the original simple assignment didn't change the (desirable) outcome, but I don't think it's really necessary in this case, considering the single instruction uses two 64-bit registers.
Quoted 6 lines
I'm going to leave it as it was originally; it's the simplest:
*dest_tre = tre;I added a comment about structuring the code this way with the intention of getting the single instruction. If a different compiler produces different result.
Ok, that's probably the best we can do then.
Arnd
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 5 lines · Arnd Bergmann
I have no idea how two 8-bit assignments could do that, it sounds like a serious gcc bug, unless you mean two 8-byte assignments, which would be within the range of expected behavior. If it's actually 8-bit stores, please open a bug against gcc with a minimized test case.
Sorry, it's 8 *byte* assignments, not 8 bit. -Alex
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 9 lines · Alex Elder
Quoted 5 lines · Arnd Bergmann
I have no idea how two 8-bit assignments could do that, it sounds like a serious gcc bug, unless you mean two 8-byte assignments, which would be within the range of expected behavior. If it's actually 8-bit stores, please open a bug against gcc with a minimized test case.
Sorry, it's 8 *byte* assignments, not 8 bit. -Alex
Is it important to the hardware that you're writing all 128 bits of this in an atomic manner? My understanding is that while you may get different behaviors using various combinations of volatile/aligned/packed, this is way deep in the compiler's "free to do whatever I want" territory. If the hardware's going to misbehave if you don't get an atomic 128-bit write, then I don't think this has been nailed down enough, since I think Clang or even a different version of GCC is within its right to split the writes up differently.
Is filling out the TRE touching memory that the hardware is also watching at the same time? Usually when the hardware cares about the contents of a struct, there's a particular (smaller) field that can be written out atomically. I remember USB having these structs that needed to be filled out, but the hardware wouldn't actually slurp them up until the smaller "token" part was non-zero. If the hardware is scanning this struct, it might be safer to do it in two steps: 1) flush out the filled out struct except for the field with the "go" bit (which you'd have zeroed), then 2) set the field containing the "go" bit.
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 9 lines · Evan Green
Quoted 8 lines · Alex Elder
Quoted 5 lines · Arnd Bergmann
I have no idea how two 8-bit assignments could do that, it sounds like a serious gcc bug, unless you mean two 8-byte assignments, which would be within the range of expected behavior. If it's actually 8-bit stores, please open a bug against gcc with a minimized test case.
Quoted 3 lines
Quoted 1 line
Sorry, it's 8 *byte* assignments, not 8 bit. -Alex
Is it important to the hardware that you're writing all 128 bits of
No, it is not important in the ways you are describing.
We're just geeking out over how to get optimal performance. A single 128-bit write is nicer than two 64-bit writes, or more smaller writes.
The hardware won't touch the TRE until the doorbell gets rung telling it that it is permitted to do so. The doorbell is an I/O write, which implies a memory barrier, so the entire TRE will be up-to-date in memory regardless of whether we write it 128 bits or 8 bits at a time.
-Alex
Quoted 19 lines
this in an atomic manner? My understanding is that while you may get different behaviors using various combinations of volatile/aligned/packed, this is way deep in the compiler's "free to do whatever I want" territory. If the hardware's going to misbehave if you don't get an atomic 128-bit write, then I don't think this has been nailed down enough, since I think Clang or even a different version of GCC is within its right to split the writes up differently.
Is filling out the TRE touching memory that the hardware is also watching at the same time? Usually when the hardware cares about the contents of a struct, there's a particular (smaller) field that can be written out atomically. I remember USB having these structs that needed to be filled out, but the hardware wouldn't actually slurp them up until the smaller "token" part was non-zero. If the hardware is scanning this struct, it might be safer to do it in two steps: 1) flush out the filled out struct except for the field with the "go" bit (which you'd have zeroed), then 2) set the field containing the "go" bit.
Re: [PATCH 09/18] soc: qcom: ipa: GSI transactions
Quoted 21 lines · Alex Elder
Quoted 9 lines · Evan Green
Quoted 8 lines · Alex Elder
Quoted 5 lines · Arnd Bergmann
I have no idea how two 8-bit assignments could do that, it sounds like a serious gcc bug, unless you mean two 8-byte assignments, which would be within the range of expected behavior. If it's actually 8-bit stores, please open a bug against gcc with a minimized test case.
Quoted 3 lines
Quoted 1 line
Sorry, it's 8 *byte* assignments, not 8 bit. -Alex
Is it important to the hardware that you're writing all 128 bits of
No, it is not important in the ways you are describing.
We're just geeking out over how to get optimal performance. A single 128-bit write is nicer than two 64-bit writes, or more smaller writes.
Quoted 6 lines
The hardware won't touch the TRE until the doorbell gets rung telling it that it is permitted to do so. The doorbell is an I/O write, which implies a memory barrier, so the entire TRE will be up-to-date in memory regardless of whether we write it 128 bits or 8 bits at a time.
Ah, understood. Carry on!
PATCH 10/18 soc: qcom: ipa: IPA interface to GSI
In reply to Alex ElderThis patch provides interface functions supplied by the IPA layer that are called from the GSI layer. One function is called when a GSI transaction has completed. The others allow the GSI layer to inform the IPA layer when transactions have been supplied to hardware, and when the hardware has indicated transactions have completed.
--- drivers/net/ipa/ipa_gsi.c | 48 ++++++++++++++++++++++++++++++++++++++ drivers/net/ipa/ipa_gsi.h | 49 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 97 insertions(+) create mode 100644 drivers/net/ipa/ipa_gsi.c create mode 100644 drivers/net/ipa/ipa_gsi.h
drivers/net/ipa/ipa_gsi.c new file +48 -0
@@ -0,0 +1,48 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#include <linux/types.h> + +#include "gsi_trans.h" +#include "ipa.h" +#include "ipa_endpoint.h" + +void ipa_gsi_trans_complete (struct gsi_trans *trans) +{ + struct ipa *ipa = container_of(trans->gsi, struct ipa, gsi); +struct ipa_endpoint *endpoint; + + endpoint = ipa->endpoint_map[trans->channel_id]; +if (endpoint == ipa->command_endpoint) +return ;/* Nothing to do for commands */ + +if (endpoint->toward_ipa) +ipa_endpoint_skb_tx_callback (trans); +else +ipa_endpoint_rx_callback (trans); +} + +void ipa_gsi_channel_tx_queued (struct gsi *gsi,u32 channel_id,u32 count, +u32 byte_count) +{ + struct ipa *ipa = container_of(gsi, struct ipa, gsi); +struct ipa_endpoint *endpoint; + + endpoint = ipa->endpoint_map[channel_id]; +if (endpoint->netdev) +netdev_sent_queue (endpoint->netdev, byte_count); +} + +void ipa_gsi_channel_tx_completed (struct gsi *gsi,u32 channel_id,u32 count, +u32 byte_count) +{ + struct ipa *ipa = container_of(gsi, struct ipa, gsi); +struct ipa_endpoint *endpoint; + + endpoint = ipa->endpoint_map[channel_id]; +if (endpoint->netdev) +netdev_completed_queue (endpoint->netdev, count, byte_count); +}
drivers/net/ipa/ipa_gsi.h new file +49 -0
49 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 11/18 soc: qcom: ipa: IPA endpoints
In reply to Alex ElderThis patch includes the code implementing an IPA endpoint. This is the primary abstraction implemented by the IPA. An endpoint is one end of a network connection between two entities physically connected to the IPA. Specifically, the AP and the modem implement endpoints, and an (AP endpoint, modem endpoint) pair implements the transfer of network data in one direction between the AP and modem.
Endpoints are built on top of GSI channels, but IPA endpoints represent the higher-level functionality that the IPA provides. Data can be sent through a GSI channel, but it is the IPA endpoint that represents what is on the "other end" to receive that data. Other functionality, including aggregation, checksum offload and (at some future date) IP routing and filtering are all associated with the IPA endpoint.
--- drivers/net/ipa/ipa_endpoint.c | 1253 ++++++++++++++++++++++++++++++++ drivers/net/ipa/ipa_endpoint.h | 96 +++ 2 files changed, 1349 insertions(+) create mode 100644 drivers/net/ipa/ipa_endpoint.c create mode 100644 drivers/net/ipa/ipa_endpoint.h
drivers/net/ipa/ipa_endpoint.c new file +1253 -0
@@ -0,0 +1,1253 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#include <linux/types.h> +#include <linux/device.h> +#include <linux/slab.h> +#include <linux/bitfield.h> +#include <soc/qcom/rmnet.h> + +#include "gsi.h" +#include "gsi_trans.h" +#include "ipa.h" +#include "ipa_data.h" +#include "ipa_endpoint.h" +#include "ipa_cmd.h" +#include "ipa_mem.h" +#include "ipa_netdev.h" + +#define atomic_dec_not_zero (v) atomic_add_unless((v), -1, 0) + +#define IPA_REPLENISH_BATCH 16 + +#define IPA_RX_BUFFER_SIZE (PAGE_SIZE << IPA_RX_BUFFER_ORDER) +#define IPA_RX_BUFFER_ORDER 1/* 8KB endpoint RX buffers (2 pages) */ + +/* The amount of RX buffer space consumed by standard skb overhead */ +#define IPA_RX_BUFFER_OVERHEAD (PAGE_SIZE - SKB_MAX_ORDER(NET_SKB_PAD, 0)) + +#define IPA_ENDPOINT_STOP_RETRY_MAX 10 +#define IPA_ENDPOINT_STOP_RX_SIZE 1/* bytes */ + +#define IPA_ENDPOINT_RESET_AGGR_RETRY_MAX 3 +#define IPA_AGGR_TIME_LIMIT_DEFAULT 1/* milliseconds */ + +/** enum ipa_status_opcode - status element opcode hardware values */ +enum ipa_status_opcode { +IPA_STATUS_OPCODE_PACKET =0x01 , +IPA_STATUS_OPCODE_NEW_FRAG_RULE =0x02 , +IPA_STATUS_OPCODE_DROPPED_PACKET =0x04 , +IPA_STATUS_OPCODE_SUSPENDED_PACKET =0x08 , +IPA_STATUS_OPCODE_LOG =0x10 , +IPA_STATUS_OPCODE_DCMP =0x20 , +IPA_STATUS_OPCODE_PACKET_2ND_PASS =0x40 , +}; + +/** enum ipa_status_exception - status element exception type */ +enum ipa_status_exception { +IPA_STATUS_EXCEPTION_NONE , +IPA_STATUS_EXCEPTION_DEAGGR , +IPA_STATUS_EXCEPTION_IPTYPE , +IPA_STATUS_EXCEPTION_PACKET_LENGTH , +IPA_STATUS_EXCEPTION_PACKET_THRESHOLD , +IPA_STATUS_EXCEPTION_FRAG_RULE_MISS , +IPA_STATUS_EXCEPTION_SW_FILT , +IPA_STATUS_EXCEPTION_NAT , +IPA_STATUS_EXCEPTION_IPV6CT , +IPA_STATUS_EXCEPTION_MAX , +}; + +/** +* struct ipa_status - Abstracted IPA status element +* @opcode: Status element type +* @exception: The first exception that took place +* @pkt_len: Payload length +* @dst_endpoint: Destination endpoint +* @metadata: 32-bit metadata value used by packet +* @rt_miss: Flag; if 1, indicates there was a routing rule miss +* +* Note that the hardware status element supplies additional information +* that is currently unused. +*/ +struct ipa_status { +enum ipa_status_opcode opcode; +enum ipa_status_exception exception; +u32 pkt_len; +u32 dst_endpoint; +u32 metadata; +u32 rt_miss; +}; + +/* Field masks for struct ipa_status_raw structure fields */ + +#define IPA_STATUS_SRC_IDX_FMASK GENMASK(4, 0) + +#define IPA_STATUS_DST_IDX_FMASK GENMASK(4, 0) + +#define IPA_STATUS_FLAGS1_FLT_LOCAL_FMASK GENMASK(0, 0) +#define IPA_STATUS_FLAGS1_FLT_HASH_FMASK GENMASK(1, 1) +#define IPA_STATUS_FLAGS1_FLT_GLOBAL_FMASK GENMASK(2, 2) +#define IPA_STATUS_FLAGS1_FLT_RET_HDR_FMASK GENMASK(3, 3) +#define IPA_STATUS_FLAGS1_FLT_RULE_ID_FMASK GENMASK(13, 4) +#define IPA_STATUS_FLAGS1_RT_LOCAL_FMASK GENMASK(14, 14) +#define IPA_STATUS_FLAGS1_RT_HASH_FMASK GENMASK(15, 15) +#define IPA_STATUS_FLAGS1_UCP_FMASK GENMASK(16, 16) +#define IPA_STATUS_FLAGS1_RT_TBL_IDX_FMASK GENMASK(21, 17) +#define IPA_STATUS_FLAGS1_RT_RULE_ID_FMASK GENMASK(31, 22) + +#define IPA_STATUS_FLAGS2_NAT_HIT_FMASK GENMASK_ULL(0, 0) +#define IPA_STATUS_FLAGS2_NAT_ENTRY_IDX_FMASK GENMASK_ULL(13, 1) +#define IPA_STATUS_FLAGS2_NAT_TYPE_FMASK GENMASK_ULL(15, 14) +#define IPA_STATUS_FLAGS2_TAG_INFO_FMASK GENMASK_ULL(63, 16) + +#define IPA_STATUS_FLAGS3_SEQ_NUM_FMASK GENMASK(7, 0) +#define IPA_STATUS_FLAGS3_TOD_CTR_FMASK GENMASK(31, 8) + +#define IPA_STATUS_FLAGS4_HDR_LOCAL_FMASK GENMASK(0, 0) +#define IPA_STATUS_FLAGS4_HDR_OFFSET_FMASK GENMASK(10, 1) +#define IPA_STATUS_FLAGS4_FRAG_HIT_FMASK GENMASK(11, 11) +#define IPA_STATUS_FLAGS4_FRAG_RULE_FMASK GENMASK(15, 12) +#define IPA_STATUS_FLAGS4_HW_SPECIFIC_FMASK GENMASK(31, 16) + +/* Status element provided by hardware */ +struct ipa_status_raw { +u8 opcode; +u8 exception; +u16 mask; +u16 pkt_len; +u8 endp_src_idx;/* Only bottom 5 bits valid */ +u8 endp_dst_idx;/* Only bottom 5 bits valid */ +u32 metadata; +u32 flags1; +u64 flags2; +u32 flags3; +u32 flags4; +}; + +static void ipa_endpoint_replenish (struct ipa_endpoint *endpoint); + +/* suspend_delay represents suspend for RX, delay for TX endpoints */ +bool ipa_endpoint_init_ctrl (struct ipa_endpoint *endpoint,bool suspend_delay) +{ +u32 offset =IPA_REG_ENDP_INIT_CTRL_N_OFFSET (endpoint->endpoint_id); +u32 mask; +u32 val; + + mask = endpoint->toward_ipa ?ENDP_DELAY_FMASK :ENDP_SUSPEND_FMASK ; + + val =ioread32 (endpoint->ipa->reg_virt + offset); +if (suspend_delay == !!(val & mask)) +return false;/* Already set to desired state */ + + val ^= mask; +iowrite32 (val, endpoint->ipa->reg_virt + offset); + +return true; +} + +static void ipa_endpoint_init_cfg (struct ipa_endpoint *endpoint) +{ +u32 offset =IPA_REG_ENDP_INIT_CFG_N_OFFSET (endpoint->endpoint_id); +u32 val =0 ; + +/* FRAG_OFFLOAD_EN is 0 */ +if (endpoint->data->config.checksum) { +if (endpoint->toward_ipa) { +u32 checksum_offset; + + val |=u32_encode_bits (IPA_CS_OFFLOAD_UL , +CS_OFFLOAD_EN_FMASK ); +/* Checksum header offset is in 4-byte units */ + checksum_offset =sizeof (struct rmnet_map_header ); + checksum_offset /=sizeof (u32); + val |=u32_encode_bits (checksum_offset, +CS_METADATA_HDR_OFFSET_FMASK ); + }else { + val |=u32_encode_bits (IPA_CS_OFFLOAD_DL , +CS_OFFLOAD_EN_FMASK ); + } + }else { + val |=u32_encode_bits (IPA_CS_OFFLOAD_NONE , +CS_OFFLOAD_EN_FMASK ); + } +/* CS_GEN_QMB_MASTER_SEL is 0 */ + +iowrite32 (val, endpoint->ipa->reg_virt + offset); +} + +static void ipa_endpoint_init_hdr (struct ipa_endpoint *endpoint) +{ +u32 offset =IPA_REG_ENDP_INIT_HDR_N_OFFSET (endpoint->endpoint_id); +u32 val =0 ; + +if (endpoint->data->config.qmap) { +size_t header_size =sizeof (struct rmnet_map_header ); + +if (endpoint->toward_ipa && endpoint->data->config.checksum) + header_size +=sizeof (struct rmnet_map_ul_csum_header ); + + val |=u32_encode_bits (header_size,HDR_LEN_FMASK ); +/* metadata is the 4 byte rmnet_map header itself */ + val |=HDR_OFST_METADATA_VALID_FMASK ; + val |=u32_encode_bits (0 ,HDR_OFST_METADATA_FMASK ); +/* HDR_ADDITIONAL_CONST_LEN is 0; (IPA->AP only) */ +if (!endpoint->toward_ipa) { +u32 size_offset = offsetof(struct rmnet_map_header , + pkt_len); + + val |=HDR_OFST_PKT_SIZE_VALID_FMASK ; + val |=u32_encode_bits (size_offset, +HDR_OFST_PKT_SIZE_FMASK ); + } +/* HDR_A5_MUX is 0 */ +/* HDR_LEN_INC_DEAGG_HDR is 0 */ +/* HDR_METADATA_REG_VALID is 0; (AP->IPA only) */ + } + +iowrite32 (val, endpoint->ipa->reg_virt + offset); +} + +static void ipa_endpoint_init_hdr_ext (struct ipa_endpoint *endpoint) +{ +u32 offset =IPA_REG_ENDP_INIT_HDR_EXT_N_OFFSET (endpoint->endpoint_id); +u32 pad_align = endpoint->data->config.rx.pad_align; +u32 val =0 ; + + val |=HDR_ENDIANNESS_FMASK ;/* big endian */ + val |=HDR_TOTAL_LEN_OR_PAD_VALID_FMASK ; +/* HDR_TOTAL_LEN_OR_PAD is 0 (pad, not total_len) */ +/* HDR_PAYLOAD_LEN_INC_PADDING is 0 */ +/* HDR_TOTAL_LEN_OR_PAD_OFFSET is 0 */ +if (!endpoint->toward_ipa) + val |=u32_encode_bits (pad_align,HDR_PAD_TO_ALIGNMENT_FMASK ); + +iowrite32 (val, endpoint->ipa->reg_virt + offset); +} + +/** +* Generate a metadata mask value that will select only the mux_id +* field in an rmnet_map header structure. The mux_id is at offset +* 1 byte from the beginning of the structure, but the metadata +* value is treated as a 4-byte unit. So this mask must be computed +* with endianness in mind. Note that ipa_endpoint_init_hdr_metadata_mask() +* will convert this value to the proper byte order. +* +* Marked __always_inline because this is really computing a +* constant value. +*/ +static __always_inline __be32 ipa_rmnet_mux_id_metadata_mask(void) +{ +size_t mux_id_offset = offsetof(struct rmnet_map_header , mux_id); +u32 mux_id_mask =0 ; +u8 *bytes; + + bytes = (u8 *)&mux_id_mask; + bytes[mux_id_offset] =0xff ;/* mux_id is 1 byte */ + +return cpu_to_be32 (mux_id_mask); +} + +static void ipa_endpoint_init_hdr_metadata_mask (struct ipa_endpoint *endpoint) +{ +u32 endpoint_id = endpoint->endpoint_id; +u32 val =0 ; +u32 offset; + + offset =IPA_REG_ENDP_INIT_HDR_METADATA_MASK_N_OFFSET (endpoint_id); + +if (!endpoint->toward_ipa && endpoint->data->config.qmap) + val =ipa_rmnet_mux_id_metadata_mask (); + +iowrite32 (val, endpoint->ipa->reg_virt + offset); +} + +/* Compute the aggregation size value to use for a given buffer size */ +static u32 ipa_aggr_size_kb (u32 rx_buffer_size) +{ +BUILD_BUG_ON (IPA_RX_BUFFER_SIZE > +field_max (AGGR_BYTE_LIMIT_FMASK ) *SZ_1K + +IPA_MTU +IPA_RX_BUFFER_OVERHEAD ); + +/* Because we don't have the "hard byte limit" enabled, we +* need to make sure there's enough space in the buffer to +* receive a complete MTU (plus normal skb overhead) beyond +* the aggregated size limit we specify. +*/ + rx_buffer_size -=IPA_MTU +IPA_RX_BUFFER_OVERHEAD ; + +return rx_buffer_size /SZ_1K ; +} + +static void ipa_endpoint_init_aggr (struct ipa_endpoint *endpoint) +{ +const struct ipa_endpoint_config_data *config = &endpoint->data->config; +u32 offset =IPA_REG_ENDP_INIT_AGGR_N_OFFSET (endpoint->endpoint_id); +u32 val =0 ; + +if (config->aggregation) { +if (!endpoint->toward_ipa) { +u32 aggr_size =ipa_aggr_size_kb (IPA_RX_BUFFER_SIZE ); + + val |=u32_encode_bits (IPA_ENABLE_AGGR ,AGGR_EN_FMASK ); + val |=u32_encode_bits (IPA_GENERIC ,AGGR_TYPE_FMASK ); + val |=u32_encode_bits (aggr_size, +AGGR_BYTE_LIMIT_FMASK ); + val |=u32_encode_bits (IPA_AGGR_TIME_LIMIT_DEFAULT , +AGGR_TIME_LIMIT_FMASK ); + val |=u32_encode_bits (0 ,AGGR_PKT_LIMIT_FMASK ); +if (config->rx.aggr_close_eof) + val |=AGGR_SW_EOF_ACTIVE_FMASK ; +/* AGGR_HARD_BYTE_LIMIT_ENABLE is 0 */ + }else { + val |=u32_encode_bits (IPA_ENABLE_DEAGGR , +AGGR_EN_FMASK ); + val |=u32_encode_bits (IPA_QCMAP ,AGGR_TYPE_FMASK ); +/* other fields ignored */ + } +/* AGGR_FORCE_CLOSE is 0 */ + }else { + val |=u32_encode_bits (IPA_BYPASS_AGGR ,AGGR_EN_FMASK ); +/* other fields ignored */ + } + +iowrite32 (val, endpoint->ipa->reg_virt + offset); +} + +static void ipa_endpoint_init_mode (struct ipa_endpoint *endpoint) +{ +u32 offset =IPA_REG_ENDP_INIT_MODE_N_OFFSET (endpoint->endpoint_id); +u32 val =0 ; + +if (endpoint->toward_ipa && endpoint->data->config.dma_mode) { +u32 dma_endpoint_id = endpoint->data->config.dma_endpoint; + + val |=u32_encode_bits (IPA_DMA ,MODE_FMASK ); + val |=u32_encode_bits (dma_endpoint_id,DEST_PIPE_INDEX_FMASK ); + }else { + val |=u32_encode_bits (IPA_BASIC ,MODE_FMASK ); + } +/* Other bitfields unspecified (and 0) */ + +iowrite32 (val, endpoint->ipa->reg_virt + offset); +} + +static void ipa_endpoint_init_deaggr (struct ipa_endpoint *endpoint) +{ +u32 offset =IPA_REG_ENDP_INIT_DEAGGR_N_OFFSET (endpoint->endpoint_id); +u32 val =0 ; + +/* DEAGGR_HDR_LEN is 0 */ +/* PACKET_OFFSET_VALID is 0 */ +/* PACKET_OFFSET_LOCATION is ignored (not valid) */ +/* MAX_PACKET_LEN is 0 (not enforced) */ + +iowrite32 (val, endpoint->ipa->reg_virt + offset); +} + +static void ipa_endpoint_init_seq (struct ipa_endpoint *endpoint) +{ +u32 offset =IPA_REG_ENDP_INIT_SEQ_N_OFFSET (endpoint->endpoint_id); +u32 seq_type = endpoint->data->seq_type; +u32 val =0 ; + + val |=u32_encode_bits (seq_type &0xf ,HPS_SEQ_TYPE_FMASK ); + val |=u32_encode_bits ((seq_type >>4 ) &0xf ,DPS_SEQ_TYPE_FMASK ); +/* HPS_REP_SEQ_TYPE is 0 */ +/* DPS_REP_SEQ_TYPE is 0 */ + +iowrite32 (val, endpoint->ipa->reg_virt + offset); +} + +/* Complete transaction initiated in ipa_endpoint_skb_tx() */ +void ipa_endpoint_skb_tx_callback (struct gsi_trans *trans) +{ +struct sk_buff *skb = trans->data; + +dev_kfree_skb_any (skb); +} + +/** +* ipa_endpoint_skb_tx() - Transmit a socket buffer +* @endpoint: Endpoint pointer +* @skb: Socket buffer to send +* +* Returns: 0 if successful, or a negative error code +*/ +int ipa_endpoint_skb_tx (struct ipa_endpoint *endpoint,struct sk_buff *skb) +{ +struct gsi_trans *trans; +bool doorbell; +u32 nr_frags; +int ret; + +/* Make sure source endpoint's TLV FIFO has enough entries to +* hold the linear portion of the skb and all its fragments. +* If not, see if we can linearize it before giving up. +*/ + nr_frags =skb_shinfo (skb)->nr_frags; +if (1 + nr_frags > endpoint->trans_tre_max) { +if (skb_linearize (skb)) +return -ENOMEM ; + nr_frags =0 ; + } + + trans =gsi_channel_trans_alloc (&endpoint->ipa->gsi, + endpoint->channel_id, nr_frags +1 ); +if (!trans) +return -EBUSY ; + trans->data = skb; + + ret =skb_to_sgvec (skb, trans->sgl,0 , skb->len); +if (ret <0 ) + goto err_trans_free; + trans->sgc = ret; + +/* doorbell = __netdev_sent_queue(skb->dev, ->len, ->xmit_more); */ + doorbell = !skb->xmit_more; + ret =gsi_trans_commit (trans, doorbell); +if (ret) + goto err_trans_free; +return 0 ; + +err_trans_free: +gsi_trans_free (trans); + +return -ENOMEM ; +} + +static void ipa_endpoint_status (struct ipa_endpoint *endpoint) +{ +const struct ipa_endpoint_config_data *config = &endpoint->data->config; +enum ipa_endpoint_id endpoint_id = endpoint->endpoint_id; +u32 val =0 ; +u32 offset; + + offset =IPA_REG_ENDP_STATUS_N_OFFSET (endpoint_id); + +if (endpoint->data->config.status_enable) { + val |=STATUS_EN_FMASK ; +if (endpoint->toward_ipa) { +u32 status_endpoint_id = config->tx.status_endpoint; + + val |=u32_encode_bits (status_endpoint_id, +STATUS_ENDP_FMASK ); + } +/* STATUS_LOCATION is 0 (status element precedes packet) */ +/* STATUS_PKT_SUPPRESS_FMASK */ + } + +iowrite32 (val, endpoint->ipa->reg_virt + offset); +} + +static void ipa_endpoint_skb_copy (struct ipa_endpoint *endpoint, +void *data,u32 len,u32 extra) +{ +struct sk_buff *skb; + + skb =__dev_alloc_skb (len,GFP_ATOMIC ); +if (skb) { +skb_put (skb, len); +memcpy (skb->data, data, len); + skb->truesize += extra; + } + +/* Now receive it, or drop it if there's no netdev */ +if (endpoint->netdev) +ipa_netdev_skb_rx (endpoint->netdev, skb); +else if (skb) +dev_kfree_skb_any (skb); +} + +static void ipa_endpoint_skb_build (struct ipa_endpoint *endpoint, +struct page *page,u32 len) +{ +struct sk_buff *skb; + +/* assert(len <= SKB_WITH_OVERHEAD(IPA_RX_BUFFER_SIZE-NET_SKB_PAD)); */ + skb =build_skb (page_address (page),IPA_RX_BUFFER_SIZE ); +if (skb) { +/* Reserve the headroom and account for the data */ +skb_reserve (skb,NET_SKB_PAD ); +skb_put (skb, len); + } + +/* Now receive it, or drop it if there's no netdev */ +if (endpoint->netdev) +ipa_netdev_skb_rx (endpoint->netdev, skb); +else if (skb) +dev_kfree_skb_any (skb); + +/* If no socket buffer took the pages, free them */ +if (!skb) +__free_pages (page,IPA_RX_BUFFER_ORDER ); +} + +/* Maps an exception type returned in a ipa_status_raw structure +* to the ipa_status_exception value that represents it in +* the exception field of a ipa_status structure. Returns +* IPA_STATUS_EXCEPTION_MAX for an unrecognized value. +*/ +static enum ipa_status_exception exception_map (u8 exception,bool is_ipv6) +{ +switch (exception) { +case 0x00 :return IPA_STATUS_EXCEPTION_NONE ; +case 0x01 :return IPA_STATUS_EXCEPTION_DEAGGR ; +case 0x04 :return IPA_STATUS_EXCEPTION_IPTYPE ; +case 0x08 :return IPA_STATUS_EXCEPTION_PACKET_LENGTH ; +case 0x10 :return IPA_STATUS_EXCEPTION_FRAG_RULE_MISS ; +case 0x20 :return IPA_STATUS_EXCEPTION_SW_FILT ; +case 0x40 :return is_ipv6 ?IPA_STATUS_EXCEPTION_IPV6CT + :IPA_STATUS_EXCEPTION_NAT ; +default :return IPA_STATUS_EXCEPTION_MAX ; + } +} + +/* A rule miss is indicated as an all-1's value in the rt_rule_id +* or flt_rule_id field of the ipa_status structure. +*/ +static bool ipa_rule_miss_id (u32 id) +{ +return id ==field_max (IPA_STATUS_FLAGS1_RT_RULE_ID_FMASK ); +} + +size_t ipa_status_parse (struct ipa_status *status,void *data,u32 count) +{ +const struct ipa_status_raw *status_raw = data; +bool is_ipv6; +u32 val; + +BUILD_BUG_ON (sizeof (*status_raw) %4 ); +if (WARN_ON (count <sizeof (*status_raw))) +return 0 ; + + status->opcode = status_raw->opcode; + is_ipv6 = status_raw->mask &BIT (7 ) ? false : true; + status->exception =exception_map (status_raw->exception, is_ipv6); + status->pkt_len = status_raw->pkt_len; + val =u32_get_bits (status_raw->endp_dst_idx,IPA_STATUS_DST_IDX_FMASK ); + status->dst_endpoint = val; + status->metadata = status_raw->metadata; + val =u32_get_bits (status_raw->flags1, +IPA_STATUS_FLAGS1_RT_RULE_ID_FMASK ); + status->rt_miss =ipa_rule_miss_id (val) ?1 :0 ; + +return sizeof (*status_raw); +} + +/* The format of a packet status element is the same for several status +* types (opcodes). The NEW_FRAG_RULE, LOG, DCMP (decompression) types +* aren't currently supported +*/ +static bool ipa_status_format_packet (enum ipa_status_opcode opcode) +{ +switch (opcode) { +case IPA_STATUS_OPCODE_PACKET : +case IPA_STATUS_OPCODE_DROPPED_PACKET : +case IPA_STATUS_OPCODE_SUSPENDED_PACKET : +case IPA_STATUS_OPCODE_PACKET_2ND_PASS : +return true; +default : +return false; + } +} + +static bool ipa_endpoint_status_skip (struct ipa_endpoint *endpoint, +struct ipa_status *status) +{ +if (!ipa_status_format_packet (status->opcode)) +return true; +if (!status->pkt_len) +return true; +if (status->dst_endpoint != endpoint->endpoint_id) +return true; + +return false;/* Don't skip this packet, process it */ +} + +static void ipa_endpoint_status_parse (struct ipa_endpoint *endpoint, +struct page *page,u32 total_len) +{ +void *data =page_address (page) +NET_SKB_PAD ; +u32 unused =IPA_RX_BUFFER_SIZE - total_len; +u32 resid = total_len; + +while (resid) { +struct ipa_status status; +bool drop_packet = false; +size_t status_size; +u32 align; +u32 len; + + status_size =ipa_status_parse (&status, data, resid); + +/* Skip over status packets that lack packet data */ +if (ipa_endpoint_status_skip (endpoint, &status)) { + data += status_size; + resid -= status_size; +continue ; + } + +/* Packet data follows the status structure. Unless +* the packet failed to match a routing rule, or it +* had a deaggregation exception, we'll consume it. +*/ +if (status.exception ==IPA_STATUS_EXCEPTION_NONE ) { +if (status.rt_miss) + drop_packet = true; + }else if (status.exception ==IPA_STATUS_EXCEPTION_DEAGGR ) { + drop_packet = true; + } + +/* Compute the amount of buffer space consumed by the +* packet, including the status element. If the hardware +* is configured to pad packet data to an aligned boundary, +* account for that. And if checksum offload is is enabled +* a trailer containing computed checksum information will +* be appended. +*/ + align = endpoint->data->config.rx.pad_align ? :1 ; + len = status_size +ALIGN (status.pkt_len, align); +if (endpoint->data->config.checksum) + len +=sizeof (struct rmnet_map_dl_csum_trailer ); + +/* Charge the new packet with a proportional fraction of +* the unused space in the original receive buffer. +* XXX Charge a proportion of the *whole* receive buffer? +*/ +if (!drop_packet) { +u32 extra = unused * len / total_len; +void *data2 = data + status_size; +u32 len2 = status.pkt_len; + +/* Client receives only packet data (no status) */ +ipa_endpoint_skb_copy (endpoint, data2, len2, extra); + } + +/* Consume status and the full packet it describes */ + data += len; + resid -= len; + } + +__free_pages (page,IPA_RX_BUFFER_ORDER ); +} + +/* Complete transaction initiated in ipa_endpoint_replenish_one() */ +void ipa_endpoint_rx_callback (struct gsi_trans *trans) +{ +struct page *page = trans->data; +struct ipa_endpoint *endpoint; +struct ipa *ipa; +u32 len; + + ipa = container_of(trans->gsi, struct ipa, gsi); + endpoint = ipa->endpoint_map[trans->channel_id]; + +atomic_inc (&endpoint->replenish_backlog); +ipa_endpoint_replenish (endpoint); + +if (trans->result == -ECANCELED ) { +__free_pages (page,IPA_RX_BUFFER_ORDER ); +return ; + } + +/* Record the actual received length and build a socket buffer */ +/* assert(trans_result > 0); */ + len = trans->result; + +if (endpoint->data->config.status_enable) +ipa_endpoint_status_parse (endpoint, page, len); +else +ipa_endpoint_skb_build (endpoint, page, len); +} + +static int ipa_endpoint_replenish_one (struct ipa_endpoint *endpoint) +{ +struct gsi_trans *trans; +bool doorbell = false; +struct page *page; +u32 offset; +u32 len; + + page =dev_alloc_pages (IPA_RX_BUFFER_ORDER ); +if (!page) +return false; + offset =NET_SKB_PAD ; + len =IPA_RX_BUFFER_SIZE - offset; + + trans =gsi_channel_trans_alloc (&endpoint->ipa->gsi, + endpoint->channel_id,1 ); +if (!trans) + goto err_page_free; + trans->data = page; + +/* Set up and map a scatterlist entry representing the buffer */ +sg_init_table (trans->sgl, trans->sgc); +sg_set_page (trans->sgl, page, len, offset); + +if (++endpoint->replenish_ready ==IPA_REPLENISH_BATCH ) { + doorbell = true; + endpoint->replenish_ready =0 ; + } + +if (!gsi_trans_commit (trans, doorbell)) +return 0 ; + +err_page_free: +__free_pages (page,IPA_RX_BUFFER_ORDER ); + +return -ENOMEM ; +} + +/** +* ipa_endpoint_replenish() - Replenish the Rx packets cache. +* +* Allocate RX packet wrapper structures with maximal socket buffers +* for an endpoint. These are supplied to the hardware, which fills +* them with incoming data. +*/ +static void ipa_endpoint_replenish (struct ipa_endpoint *endpoint) +{ +u32 backlog; + +while (atomic_dec_not_zero (&endpoint->replenish_backlog)) +if (ipa_endpoint_replenish_one (endpoint)) + goto try_again_later; + +return ; + +try_again_later: +/* The last one didn't succeed, so fix the backlog */ + backlog =atomic_inc_return (&endpoint->replenish_backlog); + +/* Whenever a receive buffer transaction completes we'll try to +* replenish again. It's unlikely, but if we fail to supply even +* one buffer, nothing will trigger another replenish attempt. +* If this happens, schedule work to try again. +*/ +if (backlog == endpoint->replenish_max) +schedule_delayed_work (&endpoint->replenish_work, +msecs_to_jiffies (1 )); +} + +static void ipa_endpoint_replenish_work (struct work_struct *work) +{ +struct delayed_work *dwork =to_delayed_work (work); +struct ipa_endpoint *endpoint; + + endpoint = container_of(dwork, struct ipa_endpoint, replenish_work); + +ipa_endpoint_replenish (endpoint); +} + +static bool ipa_endpoint_set_up (struct ipa_endpoint *endpoint) +{ +struct ipa *ipa = endpoint->ipa; + +return ipa && (ipa->set_up &BIT (endpoint->endpoint_id)); +} + +static void ipa_endpoint_default_route_set (struct ipa *ipa, +enum ipa_endpoint_id endpoint_id) +{ +u32 val; + +/* ROUTE_DIS is 0 */ + val =u32_encode_bits (endpoint_id,ROUTE_DEF_PIPE_FMASK ); + val |=ROUTE_DEF_HDR_TABLE_FMASK ; + val |=u32_encode_bits (0 ,ROUTE_DEF_HDR_OFST_FMASK ); + val |=u32_encode_bits (endpoint_id,ROUTE_FRAG_DEF_PIPE_FMASK ); + val |=ROUTE_DEF_RETAIN_HDR_FMASK ; + +iowrite32 (val, ipa->reg_virt +IPA_REG_ROUTE_OFFSET ); +} +/** +* ipa_endpoint_default_route_init() - Configure IPA default route +* @ipa: IPA pointer +* @client: Client to which exceptions should be directed +*/ +void ipa_endpoint_default_route_setup (struct ipa_endpoint *endpoint) +{ +ipa_endpoint_default_route_set (endpoint->ipa, endpoint->endpoint_id); +} + +/** +* ipa_endpoint_default_route_teardown() - +* Inverse of ipa_endpoint_default_route_setup() +* @ipa: IPA pointer +*/ +void ipa_endpoint_default_route_teardown (struct ipa_endpoint *endpoint) +{ +ipa_endpoint_default_route_set (endpoint->ipa,0 ); +} + +/** +* ipa_endpoint_stop()- Stops a GSI channel in IPA +* @client: Client whose endpoint should be stopped +* +* This function implements the sequence to stop a GSI channel +* in IPA. This function returns when the channel is is STOP state. +* +* Return value: 0 on success, negative otherwise +*/ +int ipa_endpoint_stop (struct ipa_endpoint *endpoint) +{ +struct device *dev = &endpoint->ipa->pdev->dev; +size_t size =IPA_ENDPOINT_STOP_RX_SIZE ; +struct gsi *gsi = &endpoint->ipa->gsi; +void *virt =NULL ; +dma_addr_t addr; +int ret; +int i; + +/* An RX endpoint might not stop right away. In that case we issue +* a small (1-byte) DMA command, delay for a bit (1-2 milliseconds), +* and try again. Allocate the DMA buffer in case this is needed. +*/ +if (!endpoint->toward_ipa) { + virt =dma_alloc_coherent (dev, size, &addr,GFP_KERNEL ); +if (!virt) +return -ENOMEM ; + } + +for (i =0 ; i <IPA_ENDPOINT_STOP_RETRY_MAX ; i++) { + ret =gsi_channel_stop (gsi, endpoint->channel_id); +if (ret != -EAGAIN ) +break ; + +if (endpoint->toward_ipa) +continue ; + +/* Send a 1 byte 32-bit DMA task and try again after a delay */ + ret =ipa_cmd_dma_task_32 (endpoint->ipa, size, addr); +if (ret) +break ; + +usleep_range (USEC_PER_MSEC ,2 *USEC_PER_MSEC ); + } +if (i >=IPA_ENDPOINT_STOP_RETRY_MAX ) + ret = -EIO ; + +if (!endpoint->toward_ipa) +dma_free_coherent (dev, size, virt, addr); + +return ret; +} + +bool ipa_endpoint_enabled (struct ipa_endpoint *endpoint) +{ +return !!(endpoint->ipa->enabled &BIT (endpoint->endpoint_id)); +} + +int ipa_endpoint_enable_one (struct ipa_endpoint *endpoint) +{ +struct ipa *ipa = endpoint->ipa; +int ret; + +if (WARN_ON (!ipa_endpoint_set_up (endpoint))) +return -EINVAL ; + + ret =gsi_channel_start (&ipa->gsi, endpoint->channel_id); +if (ret) +return ret; + +ipa_interrupt_suspend_enable (ipa->interrupt, endpoint->endpoint_id); + +if (!endpoint->toward_ipa) +ipa_endpoint_replenish (endpoint); + + ipa->enabled |=BIT (endpoint->endpoint_id); + +return 0 ; +} + +void ipa_endpoint_disable_one (struct ipa_endpoint *endpoint) +{ +struct ipa *ipa = endpoint->ipa; +int ret; + +if (WARN_ON (!ipa_endpoint_enabled (endpoint))) +return ; + +ipa_interrupt_suspend_disable (ipa->interrupt, endpoint->endpoint_id); + + ret =ipa_endpoint_stop (endpoint); +WARN (ret,"error %d attempting to stop endpoint %u\n" , ret, + endpoint->endpoint_id); + +if (!ret) + endpoint->ipa->enabled &= ~BIT (endpoint->endpoint_id); +} + +static bool ipa_endpoint_aggr_active (struct ipa_endpoint *endpoint) +{ +u32 mask =BIT (endpoint->endpoint_id); +struct ipa *ipa = endpoint->ipa; +u32 val; + + val =ioread32 (ipa->reg_virt +IPA_REG_STATE_AGGR_ACTIVE_OFFSET ); + +return !!(val & mask); +} + +static void ipa_endpoint_force_close (struct ipa_endpoint *endpoint) +{ +u32 mask =BIT (endpoint->endpoint_id); +struct ipa *ipa = endpoint->ipa; +u32 val; + + val =u32_encode_bits (mask,PIPE_BITMAP_FMASK ); +iowrite32 (val, ipa->reg_virt +IPA_REG_AGGR_FORCE_CLOSE_OFFSET ); +} + +/** +* ipa_endpoint_reset_rx_aggr() - Reset RX endpoint with aggregation active +* @endpoint: Endpoint to be reset +* +* If aggregation is active on an RX endpoint when a reset is performed +* on its underlying GSI channel, a special sequence of actions must be +* taken to ensure the IPA pipeline is properly cleared. +* +* Return: 0 if successful, or a negative error code +*/ +static int ipa_endpoint_reset_rx_aggr (struct ipa_endpoint *endpoint) +{ +struct device *dev = &endpoint->ipa->pdev->dev; +struct ipa *ipa = endpoint->ipa; +bool endpoint_suspended = false; +struct gsi *gsi = &ipa->gsi; +dma_addr_t addr; +u32 len =1 ; +void *virt; +int ret; +int i; + + virt =kzalloc (len,GFP_KERNEL ); +if (!virt) +return -ENOMEM ; + + addr =dma_map_single (dev, virt, len,DMA_FROM_DEVICE ); +if (dma_mapping_error (dev, addr)) { + ret = -ENOMEM ; + goto out_free_virt; + } + +/* Force close aggregation before issuing the reset */ +ipa_endpoint_force_close (endpoint); + + ret =gsi_channel_reset (gsi, endpoint->channel_id); +if (ret) + goto out_unmap_addr; + +/* Reconfigure the channel with the doorbell engine disabled, +* and poll until we know aggregation isn't active any more. +*/ +gsi_channel_config (gsi, endpoint->channel_id, false); + +if (ipa_endpoint_init_ctrl (endpoint, false)) + endpoint_suspended = true; + +/* Start channel and do a 1 byte read */ + ret =gsi_channel_start (gsi, endpoint->channel_id); +if (ret) + goto out_suspend_again; + + ret =gsi_trans_read_byte (gsi, endpoint->channel_id, addr); +if (ret) + goto err_stop_channel; + +/* Wait for aggregation to be closed on the channel */ +for (i =0 ; i <IPA_ENDPOINT_RESET_AGGR_RETRY_MAX ; i++) { +if (!ipa_endpoint_aggr_active (endpoint)) +break ; +usleep_range (USEC_PER_MSEC ,2 *USEC_PER_MSEC ); + } +WARN_ON (ipa_endpoint_aggr_active (endpoint)); + + ret =ipa_endpoint_stop (endpoint); +if (ret) + goto out_suspend_again; + +/* Finally, reset the channel again, and sleep for 1 millisecond +* to complete the channel reset sequence. +* +* Finish by suspending the channel again (if necessary) and +* re-enabling its doorbell engine. +*/ + ret =gsi_channel_reset (gsi, endpoint->channel_id); + +usleep_range (USEC_PER_MSEC ,2 *USEC_PER_MSEC ); + + goto out_suspend_again; + +err_stop_channel: +ipa_endpoint_stop (endpoint); +out_suspend_again: +if (endpoint_suspended) + (void )ipa_endpoint_init_ctrl (endpoint, true); + +/* Reconfigure the channel, with doorbell engine enabled again */ +gsi_channel_config (gsi, endpoint->channel_id, true); +out_unmap_addr: +dma_unmap_single (dev, addr, len,DMA_FROM_DEVICE ); +out_free_virt: +kfree (virt); + +return ret; +} + +static void ipa_endpoint_reset (struct ipa_endpoint *endpoint) +{ +u32 channel_id = endpoint->channel_id; +struct ipa *ipa = endpoint->ipa; +struct gsi *gsi = &ipa->gsi; +int ret; + +/* For TX endpoints, or RX endpoints without aggregation active, +* we only need to reset the underlying GSI channel. +*/ +if (!endpoint->toward_ipa && endpoint->data->config.aggregation) { +if (ipa_endpoint_aggr_active (endpoint)) + ret =ipa_endpoint_reset_rx_aggr (endpoint); +else + ret =gsi_channel_reset (gsi, channel_id); + }else { + ret =gsi_channel_reset (gsi, channel_id); + } +WARN (ret,"error %d attempting to reset channel %u\n" , ret, + endpoint->channel_id); +} + +static bool ipa_endpoint_suspended (struct ipa_endpoint *endpoint) +{ +return !!(endpoint->ipa->suspended &BIT (endpoint->endpoint_id)); +} + +/** +* ipa_endpoint_suspend_aggr() - Emulate suspend interrupt +* @endpoint_id: Endpoint on which to emulate a suspend +* +* Emulate suspend IPA interrupt to unsuspend an endpoint suspended +* with an open aggregation frame. This is to work around a hardware +* issue where the suspend interrupt will not be generated when it +* should be. +*/ +static void ipa_endpoint_suspend_aggr (struct ipa_endpoint *endpoint) +{ +struct ipa *ipa = endpoint->ipa; + +/* Nothing to do if the endpoint doesn't have aggregation open */ +if (!ipa_endpoint_aggr_active (endpoint)) +return ; + +/* Force close aggregation */ +ipa_endpoint_force_close (endpoint); + +ipa_interrupt_simulate_suspend (ipa->interrupt); +} + +/* Suspend an RX endpoint */ +void ipa_endpoint_suspend (struct ipa_endpoint *endpoint) +{ +struct gsi *gsi = &endpoint->ipa->gsi; + +if (!ipa_endpoint_enabled (endpoint)) +return ; + +if (!ipa_endpoint_init_ctrl (endpoint, true)) +return ; + +/* Due to a hardware bug, a client suspended with an open +* aggregation frame will not generate a SUSPEND IPA interrupt. +* We work around this by force-closing the aggregation frame, +* then simulating the arrival of such an interrupt. +*/ +if (endpoint->data->config.aggregation) +ipa_endpoint_suspend_aggr (endpoint); + +gsi_channel_trans_quiesce (gsi, endpoint->channel_id); + + endpoint->ipa->suspended |=BIT (endpoint->endpoint_id); +} + +/* Resume a suspended RX endpoint */ +void ipa_endpoint_resume (struct ipa_endpoint *endpoint) +{ +if (!ipa_endpoint_suspended (endpoint)) +return ; + +if (!ipa_endpoint_init_ctrl (endpoint, false)) +return ; + + endpoint->ipa->suspended &= ~BIT (endpoint->endpoint_id); +} + +static void ipa_endpoint_program (struct ipa_endpoint *endpoint) +{ +if (endpoint->toward_ipa) { +bool delay_mode = !!endpoint->data->config.tx.delay; + + (void )ipa_endpoint_init_ctrl (endpoint, delay_mode); +ipa_endpoint_init_hdr_ext (endpoint); +ipa_endpoint_init_aggr (endpoint); +ipa_endpoint_init_deaggr (endpoint); +ipa_endpoint_init_seq (endpoint); + }else { + (void )ipa_endpoint_init_ctrl (endpoint, false); +ipa_endpoint_init_hdr_ext (endpoint); +ipa_endpoint_init_aggr (endpoint); + } +ipa_endpoint_init_cfg (endpoint); +ipa_endpoint_init_hdr (endpoint); +ipa_endpoint_init_hdr_metadata_mask (endpoint); +ipa_endpoint_init_mode (endpoint); +ipa_endpoint_status (endpoint); +} + +static void ipa_endpoint_setup_one (struct ipa_endpoint *endpoint) +{ +struct gsi *gsi = &endpoint->ipa->gsi; +u32 channel_id = endpoint->channel_id; + +/* Only AP endpoints get configured */ +if (endpoint->ee_id !=GSI_EE_AP ) +return ; + + endpoint->trans_tre_max =gsi_channel_trans_tre_max (gsi, channel_id); +if (!endpoint->toward_ipa) { + endpoint->replenish_max = +gsi_channel_trans_max (gsi, channel_id); +atomic_set (&endpoint->replenish_backlog, + endpoint->replenish_max); +INIT_DELAYED_WORK (&endpoint->replenish_work, + ipa_endpoint_replenish_work); + } + +ipa_endpoint_program (endpoint); + + endpoint->ipa->set_up |=BIT (endpoint->endpoint_id); +} + +static void ipa_endpoint_teardown_one (struct ipa_endpoint *endpoint) +{ +if (!endpoint->toward_ipa) +cancel_delayed_work_sync (&endpoint->replenish_work); + +ipa_endpoint_reset (endpoint); + + endpoint->ipa->set_up &= ~BIT (endpoint->endpoint_id); +} + +void ipa_endpoint_setup (struct ipa *ipa) +{ +u32 initialized = ipa->initialized; + + ipa->set_up =0 ; +while (initialized) { +enum ipa_endpoint_id endpoint_id =__ffs (initialized); + + initialized ^=BIT (endpoint_id); + +ipa_endpoint_setup_one (&ipa->endpoint[endpoint_id]); + } +} + +void ipa_endpoint_teardown (struct ipa *ipa) +{ +u32 set_up = ipa->set_up; + +while (set_up) { +enum ipa_endpoint_id endpoint_id =__fls (set_up); + + set_up ^=BIT (endpoint_id); + +ipa_endpoint_teardown_one (&ipa->endpoint[endpoint_id]); + } +} + +static int ipa_endpoint_init_one (struct ipa *ipa, +const struct gsi_ipa_endpoint_data *data) +{ +struct ipa_endpoint *endpoint; + +if (data->endpoint_id >=IPA_ENDPOINT_MAX ) +return -EIO ; + endpoint = &ipa->endpoint[data->endpoint_id]; + +if (data->ee_id ==GSI_EE_AP ) + ipa->endpoint_map[data->channel_id] = endpoint; + + endpoint->ipa = ipa; + endpoint->ee_id = data->ee_id; + endpoint->channel_id = data->channel_id; + endpoint->endpoint_id = data->endpoint_id; + endpoint->toward_ipa = data->toward_ipa; + endpoint->data = &data->endpoint; + +if (endpoint->data->support_flt) + ipa->filter_support |=BIT (endpoint->endpoint_id); + + ipa->initialized |=BIT (endpoint->endpoint_id); + +return 0 ; +} + +void ipa_endpoint_exit_one (struct ipa_endpoint *endpoint) +{ + endpoint->ipa->initialized &= ~BIT (endpoint->endpoint_id); +} + +int ipa_endpoint_init (struct ipa *ipa,u32 data_count, +const struct gsi_ipa_endpoint_data *data) +{ +u32 initialized; +int ret; +u32 i; + + ipa->initialized =0 ; + + ipa->filter_support =0 ; +for (i =0 ; i < data_count; i++) { + ret =ipa_endpoint_init_one (ipa, &data[i]); +if (ret) + goto err_endpoint_unwind; + } +dev_dbg (&ipa->pdev->dev,"initialized 0x%08x\n" , ipa->initialized); + +/* Verify the bitmap of endpoints that support filtering. */ +dev_dbg (&ipa->pdev->dev,"filter_support 0x%08x\n" , + ipa->filter_support); +if (!ipa->filter_support) + goto err_endpoint_unwind; +if (hweight32 (ipa->filter_support) >IPA_SMEM_FLT_COUNT ) + goto err_endpoint_unwind; + +return 0 ; + +err_endpoint_unwind: + initialized = ipa->initialized; +while (initialized) { +enum ipa_endpoint_id endpoint_id =__fls (initialized); + + initialized ^=BIT (endpoint_id); + +ipa_endpoint_exit_one (&ipa->endpoint[endpoint_id]); + } + +return ret; +} + +void ipa_endpoint_exit (struct ipa *ipa) +{ +u32 initialized = ipa->initialized; + +while (initialized) { +enum ipa_endpoint_id endpoint_id =__fls (initialized); + + initialized ^=BIT (endpoint_id); + +ipa_endpoint_exit_one (&ipa->endpoint[endpoint_id]); + } +}
drivers/net/ipa/ipa_endpoint.h new file +96 -0
96 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 12/18 soc: qcom: ipa: immediate commands
In reply to Alex ElderOne TX endpoint (per EE) is used for issuing immediate commands to the IPA. These commands request activites beyond simple data transfers to be done by the IPA hardware. For example, the IPA is able to manage routing packets among endpoints, and immediate commands are used to configure tables used for that routing.
Immediate commands are built on top of GSI transactions. They are different from normal transfers (in that they use a special endpoint, and their "payload" is interpreted differently), so separate functions are used to issue immediate command transactions.
--- drivers/net/ipa/ipa_cmd.c | 372 ++++++++++++++++++++++++++++++++++++++ drivers/net/ipa/ipa_cmd.h | 116 ++++++++++++ 2 files changed, 488 insertions(+) create mode 100644 drivers/net/ipa/ipa_cmd.c create mode 100644 drivers/net/ipa/ipa_cmd.h
drivers/net/ipa/ipa_cmd.c new file +372 -0
@@ -0,0 +1,372 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2019 Linaro Ltd. +*/ + +#include <linux/types.h> +#include <linux/device.h> +#include <linux/slab.h> +#include <linux/bitfield.h> + +#include "gsi.h" +#include "gsi_trans.h" +#include "ipa.h" +#include "ipa_endpoint.h" +#include "ipa_cmd.h" +#include "ipa_mem.h" + +/** +* DOC: IPA Immediate Commands +* +* The AP command TX endpoint is used to issue immediate commands to the IPA. +* An immediate command is generally used to request the IPA do something +* other than data transfer to another endpoint. +* +* Immediate commands are represented by GSI transactions just like other +* transfer requests, represented by a single GSI TRE. Each immediate +* command has a well-defined format, having a payload of a known length. +* This allows the transfer element's length field to be used to hold an +* immediate command's opcode. The payload for a command resides in DRAM +* and is described by a single scatterlist entry in its transaction. +* Commands do not require a transaction completion callback. To commit +* an immediate command transaction, either gsi_trans_commit_command() or +* gsi_trans_commit_command_timeout() is used. +*/ + +#define IPA_GSI_DMA_TASK_TIMEOUT 15/* milliseconds */ + +/** +* __ipa_cmd_timeout() - Send an immediate command with timeout +* @ipa: IPA structure +* @opcode: Immediate command opcode (must not be IPA_CMD_NONE) +* payload: Pointer to command payload +* size: Size of payload +* @timeout: Milliseconds to wait for completion (0 waits indefinitely) +* +* This common function implements ipa_cmd() and ipa_cmd_timeout(). It +* allocates, initializes, and commits a transaction for the immediate +* command. The transaction is committed using gsi_trans_commit_command(), +* or if a non-zero timeout is supplied, gsi_trans_commit_command_timeout(). +* +* Return: 0 if successful, or a negative error code +*/ +static int __ipa_cmd_timeout (struct ipa *ipa,enum ipa_cmd_opcode opcode, +void *payload,size_t size,u32 timeout) +{ +struct ipa_endpoint *endpoint = ipa->command_endpoint; +struct gsi_trans *trans; +int ret; + +/* assert(opcode != IPA_CMD_NONE) */ + trans =gsi_channel_trans_alloc (&ipa->gsi, endpoint->channel_id,1 ); +if (!trans) +return -EBUSY ; + +sg_init_one (trans->sgl, payload, size); + +if (timeout) + ret =gsi_trans_commit_command_timeout (trans, opcode, timeout); +else + ret =gsi_trans_commit_command (trans, opcode); +if (ret) + goto err_trans_free; + +return 0 ; + +err_trans_free: +gsi_trans_free (trans); + +return ret; +} + +static int +ipa_cmd (struct ipa *ipa,enum ipa_cmd_opcode opcode,void *payload,size_t size) +{ +return __ipa_cmd_timeout (ipa, opcode, payload, size,0 ); +} + +static int ipa_cmd_timeout (struct ipa *ipa,enum ipa_cmd_opcode opcode, +void *payload,size_t size) +{ +return __ipa_cmd_timeout (ipa, opcode, payload, size, +IPA_GSI_DMA_TASK_TIMEOUT ); +} + +/* Field masks for ipa_imm_cmd_hw_hdr_init_local structure fields */ +#define IPA_CMD_HDR_INIT_FLAGS_TABLE_SIZE_FMASK GENMASK(11, 0) +#define IPA_CMD_HDR_INIT_FLAGS_HDR_ADDR_FMASK GENMASK(27, 12) + +struct ipa_imm_cmd_hw_hdr_init_local { +u64 hdr_table_addr; +u32 flags; +u32 pad;/* XXX Needed? */ +}; + +/* Initialize header space in IPA local memory */ +int ipa_cmd_hdr_init_local (struct ipa *ipa,u32 offset,u32 size) +{ +struct ipa_imm_cmd_hw_hdr_init_local *payload; +struct device *dev = &ipa->pdev->dev; +dma_addr_t addr; +void *virt; +u32 flags; +u32 max; +int ret; + +/* Note: size *can* be zero in this case */ +if (size >field_max (IPA_CMD_HDR_INIT_FLAGS_TABLE_SIZE_FMASK )) +return -EINVAL ; + + max =field_max (IPA_CMD_HDR_INIT_FLAGS_HDR_ADDR_FMASK ); +if (offset > max || ipa->shared_offset > max - offset) +return -EINVAL ; + offset += ipa->shared_offset; + +/* A zero-filled buffer of the right size is all that's required */ + virt =dma_alloc_coherent (dev, size, &addr,GFP_KERNEL ); +if (!virt) +return -ENOMEM ; + + payload =kzalloc (sizeof (*payload),GFP_KERNEL ); +if (!payload) { + ret = -ENOMEM ; + goto out_dma_free; + } + + payload->hdr_table_addr = addr; + flags =u32_encode_bits (size,IPA_CMD_HDR_INIT_FLAGS_TABLE_SIZE_FMASK ); + flags |=u32_encode_bits (offset,IPA_CMD_HDR_INIT_FLAGS_HDR_ADDR_FMASK ); + payload->flags = flags; + + ret =ipa_cmd (ipa,IPA_CMD_HDR_INIT_LOCAL , payload,sizeof (*payload)); + +kfree (payload); +out_dma_free: +dma_free_coherent (dev, size, virt, addr); + +return ret; +} + +enum ipahal_pipeline_clear_option { +IPAHAL_HPS_CLEAR =0 , +IPAHAL_SRC_GRP_CLEAR =1 , +IPAHAL_FULL_PIPELINE_CLEAR =2 , +}; + +/* Field masks for ipa_imm_cmd_hw_dma_shared_mem structure fields */ +#define IPA_CMD_DMA_SHARED_FLAGS_DIRECTION_FMASK GENMASK(0, 0) +#define IPA_CMD_DMA_SHARED_FLAGS_SKIP_CLEAR_FMASK GENMASK(1, 1) +#define IPA_CMD_DMA_SHARED_FLAGS_CLEAR_OPTIONS_FMASK GENMASK(3, 2) + +struct ipa_imm_cmd_hw_dma_shared_mem { +u16 sw_rsvd; +u16 size; +u16 local_addr; +u16 flags; +u64 system_addr; +}; + +/* Use a DMA command to zero a block of memory */ +int ipa_cmd_smem_dma_zero (struct ipa *ipa,u32 offset,u32 size) +{ +struct ipa_imm_cmd_hw_dma_shared_mem *payload; +struct device *dev = &ipa->pdev->dev; +dma_addr_t addr; +void *virt; +u32 val; +int ret; + +/* size must be non-zero, and must fit in a 16 bit field */ +if (!size || size >U16_MAX ) +return -EINVAL ; + +/* offset must fit in a 16 bit local_addr field */ +if (offset >U16_MAX || ipa->shared_offset >U16_MAX - offset) +return -EINVAL ; + offset += ipa->shared_offset; + +/* A zero-filled buffer of the right size is all that's required */ + virt =dma_alloc_coherent (dev, size, &addr,GFP_KERNEL ); +if (!virt) +return -ENOMEM ; + + payload =kzalloc (sizeof (*payload),GFP_KERNEL ); +if (!payload) { + ret = -ENOMEM ; + goto out_dma_free; + } + + payload->size = size; + payload->local_addr = offset; +/* direction: 0 = write to IPA; skip clear: 0 = don't wait */ + val =u16_encode_bits (IPAHAL_HPS_CLEAR , +IPA_CMD_DMA_SHARED_FLAGS_CLEAR_OPTIONS_FMASK ); + payload->flags = val; + payload->system_addr = addr; + + ret =ipa_cmd (ipa,IPA_CMD_DMA_SHARED_MEM , payload,sizeof (*payload)); + +kfree (payload); +out_dma_free: +dma_free_coherent (dev, size, virt, addr); + +return ret; +} + +/* Field masks for ipa_imm_cmd_hw_ip_fltrt_init structure fields */ +#define IPA_CMD_IP_FLTRT_FLAGS_HASH_SIZE_FMASK GENMASK_ULL(11, 0) +#define IPA_CMD_IP_FLTRT_FLAGS_HASH_ADDR_FMASK GENMASK_ULL(27, 12) +#define IPA_CMD_IP_FLTRT_FLAGS_NHASH_SIZE_FMASK GENMASK_ULL(39, 28) +#define IPA_CMD_IP_FLTRT_FLAGS_NHASH_ADDR_FMASK GENMASK_ULL(55, 40) + +struct ipa_imm_cmd_hw_ip_fltrt_init { +u64 hash_rules_addr; +u64 flags; +u64 nhash_rules_addr; +}; + +/* Configure a routing or filter table, for IPv4 or IPv6 */ +static int ipa_cmd_table_config (struct ipa *ipa,enum ipa_cmd_opcode opcode, +dma_addr_t addr,size_t size,u32 hash_offset, +u32 nhash_offset) +{ +struct ipa_imm_cmd_hw_ip_fltrt_init *payload; +u64 val; +u32 max; +int ret; + +if (size >field_max (IPA_CMD_IP_FLTRT_FLAGS_HASH_SIZE_FMASK )) +return -EINVAL ; +if (size >field_max (IPA_CMD_IP_FLTRT_FLAGS_NHASH_SIZE_FMASK )) +return -EINVAL ; + + max =field_max (IPA_CMD_IP_FLTRT_FLAGS_HASH_ADDR_FMASK ); +if (hash_offset > max || ipa->shared_offset > max - hash_offset) +return -EINVAL ; + hash_offset += ipa->shared_offset; + + max =field_max (IPA_CMD_IP_FLTRT_FLAGS_NHASH_ADDR_FMASK ); +if (nhash_offset > max || ipa->shared_offset > max - nhash_offset) +return -EINVAL ; + nhash_offset += ipa->shared_offset; + + payload =kzalloc (sizeof (*payload),GFP_KERNEL ); +if (!payload) +return -ENOMEM ; + + payload->hash_rules_addr = addr; + val =u64_encode_bits (size,IPA_CMD_IP_FLTRT_FLAGS_HASH_SIZE_FMASK ); + val |=u64_encode_bits (hash_offset, +IPA_CMD_IP_FLTRT_FLAGS_HASH_ADDR_FMASK ); + val |=u64_encode_bits (size,IPA_CMD_IP_FLTRT_FLAGS_NHASH_SIZE_FMASK ); + val |=u64_encode_bits (nhash_offset, +IPA_CMD_IP_FLTRT_FLAGS_NHASH_ADDR_FMASK ); + payload->flags = val; + payload->nhash_rules_addr = addr; + + ret =ipa_cmd (ipa, opcode, payload,sizeof (*payload)); + +kfree (payload); + +return ret; +} + +/* Configure IPv4 routing table */ +int ipa_cmd_route_config_ipv4 (struct ipa *ipa,size_t size) +{ +enum ipa_cmd_opcode opcode =IPA_CMD_IP_V4_ROUTING_INIT ; +u32 nhash_offset =IPA_SMEM_V4_RT_NHASH_OFFSET ; +u32 hash_offset =IPA_SMEM_V4_RT_HASH_OFFSET ; +dma_addr_t addr = ipa->route_addr; + +return ipa_cmd_table_config (ipa, opcode, addr, size, hash_offset, + nhash_offset); +} + +/* Configure IPv6 routing table */ +int ipa_cmd_route_config_ipv6 (struct ipa *ipa,size_t size) +{ +enum ipa_cmd_opcode opcode =IPA_CMD_IP_V6_ROUTING_INIT ; +u32 nhash_offset =IPA_SMEM_V6_RT_NHASH_OFFSET ; +u32 hash_offset =IPA_SMEM_V6_RT_HASH_OFFSET ; +dma_addr_t addr = ipa->route_addr; + +return ipa_cmd_table_config (ipa, opcode, addr, size, hash_offset, + nhash_offset); +} + +/* Configure IPv4 filter table */ +int ipa_cmd_filter_config_ipv4 (struct ipa *ipa,size_t size) +{ +enum ipa_cmd_opcode opcode =IPA_CMD_IP_V4_FILTER_INIT ; +u32 nhash_offset =IPA_SMEM_V4_FLT_NHASH_OFFSET ; +u32 hash_offset =IPA_SMEM_V4_FLT_HASH_OFFSET ; +dma_addr_t addr = ipa->filter_addr; + +return ipa_cmd_table_config (ipa, opcode, addr, size, hash_offset, + nhash_offset); +} + +/* Configure IPv6 filter table */ +int ipa_cmd_filter_config_ipv6 (struct ipa *ipa,size_t size) +{ +enum ipa_cmd_opcode opcode =IPA_CMD_IP_V6_FILTER_INIT ; +u32 nhash_offset =IPA_SMEM_V6_FLT_NHASH_OFFSET ; +u32 hash_offset =IPA_SMEM_V6_FLT_HASH_OFFSET ; +dma_addr_t addr = ipa->filter_addr; + +return ipa_cmd_table_config (ipa, opcode, addr, size, hash_offset, + nhash_offset); +} + +/* Field masks for ipa_imm_cmd_hw_dma_task_32b_addr structure fields */ +#define IPA_CMD_DMA32_TASK_SW_RSVD_FMASK GENMASK(10, 0) +#define IPA_CMD_DMA32_TASK_CMPLT_FMASK GENMASK(11, 11) +#define IPA_CMD_DMA32_TASK_EOF_FMASK GENMASK(12, 12) +#define IPA_CMD_DMA32_TASK_FLSH_FMASK GENMASK(13, 13) +#define IPA_CMD_DMA32_TASK_LOCK_FMASK GENMASK(14, 14) +#define IPA_CMD_DMA32_TASK_UNLOCK_FMASK GENMASK(15, 15) +#define IPA_CMD_DMA32_SIZE1_FMASK GENMASK(31, 16) +#define IPA_CMD_DMA32_PACKET_SIZE_FMASK GENMASK(15, 0) + +struct ipa_imm_cmd_hw_dma_task_32b_addr { +u32 size1_flags; +u32 addr1; +u32 packet_size; +u32 rsvd; +}; + +/* Use a 32-bit DMA command to zero a block of memory */ +int ipa_cmd_dma_task_32 (struct ipa *ipa,size_t size,dma_addr_t addr) +{ +struct ipa_imm_cmd_hw_dma_task_32b_addr *payload; +u32 size1_flags; +int ret; + +if (size >field_max (IPA_CMD_DMA32_SIZE1_FMASK )) +return -EINVAL ; +if (size >field_max (IPA_CMD_DMA32_PACKET_SIZE_FMASK )) +return -EINVAL ; + + payload =kzalloc (sizeof (*payload),GFP_KERNEL ); +if (!payload) +return -ENOMEM ; + +/* complete: 0 = don't interrupt; eof: 0 = don't assert eot */ + size1_flags =IPA_CMD_DMA32_TASK_FLSH_FMASK ; +/* lock: 0 = don't lock endpoint; unlock: 0 = don't unlock */ + size1_flags |=u32_encode_bits (size,IPA_CMD_DMA32_SIZE1_FMASK ); + + payload->size1_flags = size1_flags; + payload->addr1 = addr; + payload->packet_size = +u32_encode_bits (size,IPA_CMD_DMA32_PACKET_SIZE_FMASK ); + + ret =ipa_cmd_timeout (ipa,IPA_CMD_DMA_TASK_32B_ADDR , payload, +sizeof (payload)); + +kfree (payload); + +return ret; +}
drivers/net/ipa/ipa_cmd.h new file +116 -0
116 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH 12/18] soc: qcom: ipa: immediate commands
Quoted 34 lines · Alex Elder
drivers/net/ipa/ipa_cmd.c new file +34
-0+/* Initialize header space in IPA local memory */ +int ipa_cmd_hdr_init_local (struct ipa *ipa,u32 offset,u32 size) +{ +struct ipa_imm_cmd_hw_hdr_init_local *payload; +struct device *dev = &ipa->pdev->dev; +dma_addr_t addr; +void *virt; +u32 flags; +u32 max; +int ret; + +/* Note: size *can* be zero in this case */ +if (size >field_max (IPA_CMD_HDR_INIT_FLAGS_TABLE_SIZE_FMASK )) +return -EINVAL ; + + max =field_max (IPA_CMD_HDR_INIT_FLAGS_HDR_ADDR_FMASK ); +if (offset > max || ipa->shared_offset > max - offset) +return -EINVAL ; + offset += ipa->shared_offset; + +/* A zero-filled buffer of the right size is all that's required */ + virt =dma_alloc_coherent (dev, size, &addr,GFP_KERNEL ); +if (!virt) +return -ENOMEM ; + + payload =kzalloc (sizeof (*payload),GFP_KERNEL ); +if (!payload) { + ret = -ENOMEM ; + goto out_dma_free; + } + + payload->hdr_table_addr = addr; + flags =u32_encode_bits (size,IPA_CMD_HDR_INIT_FLAGS_TABLE_SIZE_FMASK ); + flags |=u32_encode_bits (offset,IPA_CMD_HDR_INIT_FLAGS_HDR_ADDR_FMASK );
drivers/net/ipa/ipa_cmd.c line 149
Quoted 10 lines
drivers/net/ipa/ipa_cmd.c new file +10
-0+ payload->flags = flags; + + ret =ipa_cmd (ipa,IPA_CMD_HDR_INIT_LOCAL , payload,sizeof (*payload)); + +kfree (payload); +out_dma_free: +dma_free_coherent (dev, size, virt, addr); + +return ret; +}
This looks rather strange. I think I looked at it before and you explained it, but I have since forgotten what you do it for, so I assume everyone else that tries to understand this will have problems too.
The issue I see is that you do an expensive dma_alloc_coherent() but then never actually use the pointer returned by it, only the dma address that cannot be turned back into a virtual address in order to access the data in it.
If you can't actually use payload->hdr_table_addr, why even allocate it here?
Arnd
Re: [PATCH 12/18] soc: qcom: ipa: immediate commands
Quoted 42 lines · Arnd Bergmann
Quoted 40 lines · Alex Elder
drivers/net/ipa/ipa_cmd.c new file +40
-0+/* Initialize header space in IPA local memory */ +int ipa_cmd_hdr_init_local (struct ipa *ipa,u32 offset,u32 size) +{ +struct ipa_imm_cmd_hw_hdr_init_local *payload; +struct device *dev = &ipa->pdev->dev; +dma_addr_t addr; +void *virt; +u32 flags; +u32 max; +int ret; + +/* Note: size *can* be zero in this case */ +if (size >field_max (IPA_CMD_HDR_INIT_FLAGS_TABLE_SIZE_FMASK )) +return -EINVAL ; + + max =field_max (IPA_CMD_HDR_INIT_FLAGS_HDR_ADDR_FMASK ); +if (offset > max || ipa->shared_offset > max - offset) +return -EINVAL ; + offset += ipa->shared_offset; + +/* A zero-filled buffer of the right size is all that's required */ + virt =dma_alloc_coherent (dev, size, &addr,GFP_KERNEL ); +if (!virt) +return -ENOMEM ; + + payload =kzalloc (sizeof (*payload),GFP_KERNEL ); +if (!payload) { + ret = -ENOMEM ; + goto out_dma_free; + } + + payload->hdr_table_addr = addr; + flags =u32_encode_bits (size,IPA_CMD_HDR_INIT_FLAGS_TABLE_SIZE_FMASK ); + flags |=u32_encode_bits (offset,IPA_CMD_HDR_INIT_FLAGS_HDR_ADDR_FMASK ); + payload->flags = flags; + + ret =ipa_cmd (ipa,IPA_CMD_HDR_INIT_LOCAL , payload,sizeof (*payload)); + +kfree (payload); +out_dma_free:
Quoted 8 lines
drivers/net/ipa/ipa_cmd.c line 149
Quoted 4 lines
drivers/net/ipa/ipa_cmd.c new file +4
-0+dma_free_coherent (dev, size, virt, addr); + +return ret; +}This looks rather strange. I think I looked at it before and you explained it, but I have since forgotten what you do it for, so I assume everyone else that tries to understand this will have problems too.
This is a bug. I think I misunderstood why you were puzzled before. Now I get it. I need to save that DMA address and not free it at the end of the function (except on error).
Here's what I think happened. There are two parts of initializing these tables. One part tells the hardware where the table is located. Another part zeroes the contents of those tables. (The zeroing part could be accomplished when the table is allocated, but there are cases where they have to be zeroed again without needing to tell the hardware so we need to at least be able to do that independently.)
I think I was assuming this was the function that did the zeroing, and I thought that adding the comment about "all we need is a zero-filled buffer" addressed what you thought should be made clearer.
I will definitely fix this, and I'm glad you repeated it so I was forced to take another look.
If I again misunderstand your point, please let me know.
-Alex
Quoted 10 lines
The issue I see is that you do an expensive
dma_alloc_coherent()but then never actually use the pointer returned by it, only the dma address that cannot be turned back into a virtual address in order to access the data in it.If you can't actually use
payload->hdr_table_addr, why even allocate it here?Arnd
Re: [PATCH 12/18] soc: qcom: ipa: immediate commands
Quoted 52 lines · Alex Elder
Quoted 42 lines · Arnd Bergmann
Quoted 40 lines · Alex Elder
drivers/net/ipa/ipa_cmd.c new file +40
-0+/* Initialize header space in IPA local memory */ +int ipa_cmd_hdr_init_local (struct ipa *ipa,u32 offset,u32 size) +{ +struct ipa_imm_cmd_hw_hdr_init_local *payload; +struct device *dev = &ipa->pdev->dev; +dma_addr_t addr; +void *virt; +u32 flags; +u32 max; +int ret; + +/* Note: size *can* be zero in this case */ +if (size >field_max (IPA_CMD_HDR_INIT_FLAGS_TABLE_SIZE_FMASK )) +return -EINVAL ; + + max =field_max (IPA_CMD_HDR_INIT_FLAGS_HDR_ADDR_FMASK ); +if (offset > max || ipa->shared_offset > max - offset) +return -EINVAL ; + offset += ipa->shared_offset; + +/* A zero-filled buffer of the right size is all that's required */ + virt =dma_alloc_coherent (dev, size, &addr,GFP_KERNEL ); +if (!virt) +return -ENOMEM ; + + payload =kzalloc (sizeof (*payload),GFP_KERNEL ); +if (!payload) { + ret = -ENOMEM ; + goto out_dma_free; + } + + payload->hdr_table_addr = addr; + flags =u32_encode_bits (size,IPA_CMD_HDR_INIT_FLAGS_TABLE_SIZE_FMASK ); + flags |=u32_encode_bits (offset,IPA_CMD_HDR_INIT_FLAGS_HDR_ADDR_FMASK ); + payload->flags = flags; + + ret =ipa_cmd (ipa,IPA_CMD_HDR_INIT_LOCAL , payload,sizeof (*payload)); + +kfree (payload); +out_dma_free:Quoted 8 lines
drivers/net/ipa/ipa_cmd.c line 149
Quoted 4 lines
drivers/net/ipa/ipa_cmd.c new file +4
-0+dma_free_coherent (dev, size, virt, addr); + +return ret; +}This looks rather strange. I think I looked at it before and you explained it, but I have since forgotten what you do it for, so I assume everyone else that tries to understand this will have problems too.
Quoted 4 lines
This is a bug. I think I misunderstood why you were puzzled before. Now I get it. I need to save that DMA address and not free it at the end of the function (except on error).
OK, now I'm going to correct myself. I hope I don't make any mistakes here because things are confused enough...
Part of what I described previously is still true, namely there are tables that need to be initialized (i.e., the IPA needs to be told where they reside), and there is a separate step is available to zero the content of the tables.
But there really is no need for the AP to hang onto this DMA memory after this immediate command has been issued. I will add comments in the code to make it less surprising.
But here's a summary of why.
I think there are two things at play that make it confusing.
The first thing is that these "header tables" are actually located in a region of shared memory ("smem") that is local to the IPA (not the AP). The the IPA_CMD_HDR_INIT_LOCAL immediate command is meant to:
1) define the header table location in IPA local memory
2) define the header table size
3) provide a buffer used to fill the table with its initial contents
The location and size are encoded in the flags field of the payload (offset and size).
The initial contents are filled via DMA from a buffer in main memory, whose DMA address is supplied in the hdr_table_addr parameter in the payload. The initial contents we supply are all zero. So this is why we need to allocate DMA memory.
The second thing is that this is an instance where the AP is responsible for performing some initialization of resources it may not "own" thereafter. The IPA hardware owns this table, even though the AP needs to tell it where it sits in IPA local memory. The AP is able to copy (using DMA) content into that table, but doing so involves a DMA transfer.
More advanced features of the IPA would make more use of this header table, but those features not yet supported so this initialization (and a subsequent, seemingly redundant zeroing) is all we do.
Does that make sense?
-Alex
Quoted 32 lines
Here's what I think happened. There are two parts of initializing these tables. One part tells the hardware where the table is located. Another part zeroes the contents of those tables. (The zeroing part could be accomplished when the table is allocated, but there are cases where they have to be zeroed again without needing to tell the hardware so we need to at least be able to do that independently.)
I think I was assuming this was the function that did the zeroing, and I thought that adding the comment about "all we need is a zero-filled buffer" addressed what you thought should be made clearer.
I will definitely fix this, and I'm glad you repeated it so I was forced to take another look.
If I again misunderstand your point, please let me know.
-Alex
Quoted 10 lines
The issue I see is that you do an expensive
dma_alloc_coherent()but then never actually use the pointer returned by it, only the dma address that cannot be turned back into a virtual address in order to access the data in it.If you can't actually use
payload->hdr_table_addr, why even allocate it here?Arnd
Re: [PATCH 12/18] soc: qcom: ipa: immediate commands
Quoted 54 lines · Alex Elder
Quoted 10 lines · Alex Elder
Quoted 4 lines · Arnd Bergmann
This looks rather strange. I think I looked at it before and you explained it, but I have since forgotten what you do it for, so I assume everyone else that tries to understand this will have problems too.
This is a bug. I think I misunderstood why you were puzzled before. Now I get it. I need to save that DMA address and not free it at the end of the function (except on error).
OK, now I'm going to correct myself. I hope I don't make any mistakes here because things are confused enough...
Part of what I described previously is still true, namely there are tables that need to be initialized (i.e., the IPA needs to be told where they reside), and there is a separate step is available to zero the content of the tables.
But there really is no need for the AP to hang onto this DMA memory after this immediate command has been issued. I will add comments in the code to make it less surprising.
But here's a summary of why.
I think there are two things at play that make it confusing.
The first thing is that these "header tables" are actually located in a region of shared memory ("smem") that is local to the IPA (not the AP). The the
IPA_CMD_HDR_INIT_LOCALimmediate command is meant to:
1) define the header table location in IPA local memory
2) define the header table size
3) provide a buffer used to fill the table with its initial contentsThe location and size are encoded in the flags field of the payload (offset and size).
The initial contents are filled via DMA from a buffer in main memory, whose DMA address is supplied in the
hdr_table_addrparameter in the payload. The initial contents we supply are all zero. So this is why we need to allocate DMA memory.The second thing is that this is an instance where the AP is responsible for performing some initialization of resources it may not "own" thereafter. The IPA hardware owns this table, even though the AP needs to tell it where it sits in IPA local memory. The AP is able to copy (using DMA) content into that table, but doing so involves a DMA transfer.
Quoted 6 lines
More advanced features of the IPA would make more use of this header table, but those features not yet supported so this initialization (and a subsequent, seemingly redundant zeroing) is all we do.
Does that make sense?
Ok, that sounds reasonable, yes. I'm not sure if dma_alloc_coherent() guarantees zero-initialization though, so if that is required, you may have to add a memset().
Arnd
Re: [PATCH 12/18] soc: qcom: ipa: immediate commands
Quoted 4 lines · Arnd Bergmann
Ok, that sounds reasonable, yes. I'm not sure if
dma_alloc_coherent()guarantees zero-initialization though, so if that is required, you may have to add amemset().
I had dma_zalloc_coherent() originally but I think Christoph Hellwig posted something recently that removed that function, because dma_alloc_coherent() always zeroes the underlying memory.
+Christoph, who might be able to explain or confirm
-Alex
Re: [PATCH 12/18] soc: qcom: ipa: immediate commands
Quoted 11 lines · Alex Elder
Quoted 4 lines · Arnd Bergmann
Ok, that sounds reasonable, yes. I'm not sure if
dma_alloc_coherent()guarantees zero-initialization though, so if that is required, you may have to add amemset().I had
dma_zalloc_coherent()originally but I think Christoph Hellwig posted something recently that removed that function, becausedma_alloc_coherent()always zeroes the underlying memory.+Christoph, who might be able to explain or confirm
dma_alloc_coherent always zeroes the returned memory, which is why dma_zalloc_coherent has been removed.
PATCH 13/18 soc: qcom: ipa: IPA network device and microcontroller
In reply to Alex ElderThis patch includes the code that implements a Linux network device, using one TX and one RX IPA endpoint. It is used to implement the network device representing the modem and its connection to wireless networks. There are only a few things that are really modem-specific though, and they aren't clearly called out here. Such distinctions will be made clearer if we wish to support a network device for anything other than the modem.
Sort of unrelated, this patch also includes the code supporting the microcontroller CPU present on the IPA. The microcontroller can be used to implement special handling of packets, but at this time we don't support that. Still, it is a component that needs to be initialized, and in the event of a crash we need to do some synchronization between the AP and the microcontroller.
--- drivers/net/ipa/ipa_netdev.c | 250 +++++++++++++++++++++++++++++++++++ drivers/net/ipa/ipa_netdev.h | 24 ++++ drivers/net/ipa/ipa_uc.c | 208 +++++++++++++++++++++++++++++ drivers/net/ipa/ipa_uc.h | 32 +++++ 4 files changed, 514 insertions(+) create mode 100644 drivers/net/ipa/ipa_netdev.c create mode 100644 drivers/net/ipa/ipa_netdev.h create mode 100644 drivers/net/ipa/ipa_uc.c create mode 100644 drivers/net/ipa/ipa_uc.h
drivers/net/ipa/ipa_netdev.c new file +250 -0
@@ -0,0 +1,250 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (c) 2014-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ + +/* Modem Transport Network Driver. */ + +#include <linux/errno.h> +#include <linux/if_arp.h> +#include <linux/netdevice.h> +#include <linux/skbuff.h> +#include <soc/qcom/rmnet.h> + +#include "ipa.h" +#include "ipa_data.h" +#include "ipa_endpoint.h" +#include "ipa_mem.h" +#include "ipa_netdev.h" +#include "ipa_qmi.h" + +#define IPA_NETDEV_NAME "rmnet_ipa%d" + +#define TAILROOM 0/* for padding by mux layer */ + +#define IPA_NETDEV_TIMEOUT 10/* seconds */ + +/** struct ipa_priv - IPA network device private data */ +struct ipa_priv { +struct ipa_endpoint *tx_endpoint; +struct ipa_endpoint *rx_endpoint; +}; + +/** ipa_netdev_open() - Opens the modem network interface */ +static int ipa_netdev_open (struct net_device *netdev) +{ +struct ipa_priv *priv =netdev_priv (netdev); +int ret; + + ret =ipa_endpoint_enable_one (priv->rx_endpoint); +if (ret) +return ret; + ret =ipa_endpoint_enable_one (priv->tx_endpoint); +if (ret) + goto err_disable_rx; + +netif_start_queue (netdev); + +return 0 ; + +err_disable_rx: +ipa_endpoint_disable_one (priv->rx_endpoint); + +return ret; +} + +/** ipa_netdev_stop() - Stops the modem network interface. */ +static int ipa_netdev_stop (struct net_device *netdev) +{ +struct ipa_priv *priv =netdev_priv (netdev); + +netif_stop_queue (netdev); + +ipa_endpoint_disable_one (priv->tx_endpoint); +ipa_endpoint_disable_one (priv->rx_endpoint); + +return 0 ; +} + +/** ipa_netdev_xmit() - Transmits an skb. +* +* @skb: skb to be transmitted +* @dev: network device +* +* Return codes: +* NETDEV_TX_OK: Success +* NETDEV_TX_BUSY: Error while transmitting the skb. Try again later +*/ +static int ipa_netdev_xmit (struct sk_buff *skb,struct net_device *netdev) +{ +struct net_device_stats *stats = &netdev->stats; +struct ipa_priv *priv =netdev_priv (netdev); +struct ipa_endpoint *endpoint; +u32 skb_len = skb->len; + +if (!skb_len) + goto err_drop; + + endpoint = priv->tx_endpoint; +if (endpoint->data->config.qmap && skb->protocol !=htons (ETH_P_MAP )) + goto err_drop; + +if (ipa_endpoint_skb_tx (endpoint, skb)) +return NETDEV_TX_BUSY ; + + stats->tx_packets++; + stats->tx_bytes += skb_len; + +return NETDEV_TX_OK ; + +err_drop: +dev_kfree_skb_any (skb); + stats->tx_dropped++; + +return NETDEV_TX_OK ; +} + +void ipa_netdev_skb_rx (struct net_device *netdev,struct sk_buff *skb) +{ +struct net_device_stats *stats = &netdev->stats; + +if (skb) { + skb->dev = netdev; + skb->protocol =htons (ETH_P_MAP ); + stats->rx_packets++; + stats->rx_bytes += skb->len; + + (void )netif_receive_skb (skb); + }else { + stats->rx_dropped++; + } +} + +static const struct net_device_ops ipa_netdev_ops = { + .ndo_open = ipa_netdev_open, + .ndo_stop = ipa_netdev_stop, + .ndo_start_xmit = ipa_netdev_xmit, +}; + +/** netdev_setup() - netdev setup function */ +static void netdev_setup (struct net_device *netdev) +{ + netdev->netdev_ops = &ipa_netdev_ops; +ether_setup (netdev); +/* No header ops (override value set by ether_setup()) */ + netdev->header_ops =NULL ; + netdev->type =ARPHRD_RAWIP ; + netdev->hard_header_len =0 ; + netdev->max_mtu =IPA_MTU ; + netdev->mtu = netdev->max_mtu; + netdev->addr_len =0 ; + netdev->flags &= ~(IFF_BROADCAST |IFF_MULTICAST ); +/* The endpoint is configured for QMAP */ + netdev->needed_headroom =sizeof (struct rmnet_map_header ); + netdev->needed_tailroom =TAILROOM ; + netdev->watchdog_timeo =IPA_NETDEV_TIMEOUT *HZ ; + netdev->hw_features =NETIF_F_SG ; +} + +/** ipa_netdev_suspend() - suspend callback for runtime_pm +* @dev: pointer to device +* +* This callback will be invoked by the runtime_pm framework when an AP suspend +* operation is invoked, usually by pressing a suspend button. +* +* Returns -EAGAIN to runtime_pm framework in case there are pending packets +* in the Tx queue. This will postpone the suspend operation until all the +* pending packets will be transmitted. +* +* In case there are no packets to send, releases the WWAN0_PROD entity. +* As an outcome, the number of IPA active clients should be decremented +* until IPA clocks can be gated. +*/ +void ipa_netdev_suspend (struct net_device *netdev) +{ +struct ipa_priv *priv =netdev_priv (netdev); + +netif_stop_queue (netdev); + +ipa_endpoint_suspend (priv->rx_endpoint); +} + +/** ipa_netdev_resume() - resume callback for runtime_pm +* @dev: pointer to device +* +* This callback will be invoked by the runtime_pm framework when an AP resume +* operation is invoked. +* +* Enables the network interface queue and returns success to the +* runtime_pm framework. +*/ +void ipa_netdev_resume (struct net_device *netdev) +{ +struct ipa_priv *priv =netdev_priv (netdev); + +ipa_endpoint_resume (priv->rx_endpoint); + +netif_wake_queue (netdev); +} + +struct net_device *ipa_netdev_setup (struct ipa *ipa, +struct ipa_endpoint *rx_endpoint, +struct ipa_endpoint *tx_endpoint) +{ +struct net_device *netdev; +struct ipa_priv *priv; +int ret; + +/* Zero modem shared memory before we begin */ + ret =ipa_smem_zero_modem (ipa); +if (ret) +return ERR_PTR (ret); + +/* Start QMI communication with the modem */ + ret =ipa_qmi_setup (ipa); +if (ret) +return ERR_PTR (ret); + + netdev =alloc_netdev (sizeof (struct ipa_priv ),IPA_NETDEV_NAME , +NET_NAME_UNKNOWN , netdev_setup); +if (!netdev) { + ret = -ENOMEM ; + goto err_qmi_exit; + } + + rx_endpoint->netdev = netdev; + tx_endpoint->netdev = netdev; + + priv =netdev_priv (netdev); + priv->rx_endpoint = rx_endpoint; + priv->tx_endpoint = tx_endpoint; + + ret =register_netdev (netdev); +if (ret) + goto err_free_netdev; + +return netdev; + +err_free_netdev: +free_netdev (netdev); +err_qmi_exit: +ipa_qmi_teardown (ipa); + +return ERR_PTR (ret); +} + +void ipa_netdev_teardown (struct net_device *netdev) +{ +struct ipa_priv *priv =netdev_priv (netdev); +struct ipa *ipa = priv->tx_endpoint->ipa; + +if (!netif_queue_stopped (netdev)) + (void )ipa_netdev_stop (netdev); + +unregister_netdev (netdev); + +free_netdev (netdev); + +ipa_qmi_teardown (ipa); +}
drivers/net/ipa/ipa_netdev.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ +#ifndef _IPA_NETDEV_H_ +#define _IPA_NETDEV_H_ + +struct ipa ; +struct ipa_endpoint ; +struct net_device ; +struct sk_buff ; + +struct net_device *ipa_netdev_setup (struct ipa *ipa, +struct ipa_endpoint *rx_endpoint, +struct ipa_endpoint *tx_endpoint); +void ipa_netdev_teardown (struct net_device *netdev); + +void ipa_netdev_skb_rx (struct net_device *netdev,struct sk_buff *skb); + +void ipa_netdev_suspend (struct net_device *netdev); +void ipa_netdev_resume (struct net_device *netdev); + +#endif /* _IPA_NETDEV_H_ */
drivers/net/ipa/ipa_uc.c new file +208 -0
208 lines not on this page · raw message
drivers/net/ipa/ipa_uc.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* Copyright (c) 2012-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2019 Linaro Ltd. +*/ +#ifndef _IPA_UC_H_ +#define _IPA_UC_H_ + +struct ipa ; + +/** +* ipa_uc_setup() - set up the IPA microcontroller subsystem +* @ipa: IPA pointer +*/ +void ipa_uc_setup (struct ipa *ipa); + +/** +* ipa_uc_teardown() - inverse of ipa_uc_setup() +* @ipa: IPA pointer +*/ +void ipa_uc_teardown (struct ipa *ipa); + +/** +* ipa_uc_panic_notifier() +* @ipa: IPA pointer +* +* Notifier function called when the system crashes, to inform the +* microcontroller of the event. +*/ +void ipa_uc_panic_notifier (struct ipa *ipa); + +#endif /* _IPA_UC_H_ */
Signature · 1 line
2.20.1
Re: [PATCH 13/18] soc: qcom: ipa: IPA network device and microcontroller
Quoted 8 lines · Alex Elder
This patch includes the code that implements a Linux network device, using one TX and one RX IPA endpoint. It is used to implement the network device representing the modem and its connection to wireless networks. There are only a few things that are really modem-specific though, and they aren't clearly called out here. Such distinctions will be made clearer if we wish to support a network device for anything other than the modem.
This does not seem to do much at all, as far as I can see it's a fairly small abstraction between the linux netdev layer and the actual implementation. Could you just merge this file into whichever file it interacts with most closely, and open-code the wrappers there?
Arnd
Re: [PATCH 13/18] soc: qcom: ipa: IPA network device and microcontroller
Quoted 10 lines · Arnd Bergmann
Quoted 8 lines · Alex Elder
This patch includes the code that implements a Linux network device, using one TX and one RX IPA endpoint. It is used to implement the network device representing the modem and its connection to wireless networks. There are only a few things that are really modem-specific though, and they aren't clearly called out here. Such distinctions will be made clearer if we wish to support a network device for anything other than the modem.
Quoted 4 lines
This does not seem to do much at all, as far as I can see it's a fairly small abstraction between the linux netdev layer and the actual implementation. Could you just merge this file into whichever file it interacts with most closely, and open-code the wrappers there?
This used to be a bigger file, containing IOCTLs for configuring the endpoints used.
It is logically separate from endpoints, because not all endpoints are attached to network devices. The IPA command TX endpoint isn't associated with a network device, and the default route RX endpoint isn't either.
In addition, the modem can crash and be restarted independent of all other endpoints. I haven't added proper handling for that yet, though (just the ipa_ssr_*() stubs), and I thought maybe in finishing that I might find keeping it separated in this way would better fit how that would work out. The presence of the "rmnet_ipa0" network device essentially represents the presence of a functioning modem.
That being said, in the process of trying to streamline the data path, I *did* add a netdev pointer to the ipa_endpoint structure, so I've already blurred the line between these layers.
So I will try to do as you suggest, and this code will most likely end up in "ipa_endpoint.c".
-Alex
Quoted 2 lines
Arnd
PATCH 14/18 soc: qcom: ipa: AP/modem communications
In reply to Alex ElderThis patch implements two forms of out-of-band communication between the AP and modem.
- QMI is a mechanism that allows clients running on the AP interact with services running on the modem (and vice-versa). The AP IPA driver uses QMI to communicate with the corresponding IPA driver resident on the modem, to agree on parameters used with the IPA hardware and to ensure both sides are ready before entering operational mode.
- SMP2P is a more primitive mechanism available for the modem and AP to communicate with each other. It provides a means for either the AP or modem to interrupt the other, and furthermore, to provide 32 bits worth of information. The IPA driver uses SMP2P to tell the modem what the state of the IPA clock was in the event of a crash. This allows the modem to safely access the IPA hardware (or avoid doing so) when a crash occurs, for example, to access information within the IPA hardware.
--- drivers/net/ipa/ipa_qmi.c | 399 +++++++++++++++++++++++ drivers/net/ipa/ipa_qmi.h | 35 ++ drivers/net/ipa/ipa_qmi_msg.c | 583 ++++++++++++++++++++++++++++++++++ drivers/net/ipa/ipa_qmi_msg.h | 238 ++++++++++++++ drivers/net/ipa/ipa_smp2p.c | 304 ++++++++++++++++++ drivers/net/ipa/ipa_smp2p.h | 47 +++ 6 files changed, 1606 insertions(+) create mode 100644 drivers/net/ipa/ipa_qmi.c create mode 100644 drivers/net/ipa/ipa_qmi.h create mode 100644 drivers/net/ipa/ipa_qmi_msg.c create mode 100644 drivers/net/ipa/ipa_qmi_msg.h create mode 100644 drivers/net/ipa/ipa_smp2p.c create mode 100644 drivers/net/ipa/ipa_smp2p.h
drivers/net/ipa/ipa_qmi.c new file +399 -0
@@ -0,0 +1,399 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ + +#include <linux/types.h> +#include <linux/string.h> +#include <linux/slab.h> +#include <linux/qrtr.h> +#include <linux/soc/qcom/qmi.h> + +#include "ipa.h" +#include "ipa_endpoint.h" +#include "ipa_mem.h" +#include "ipa_qmi_msg.h" + +#define QMI_INIT_DRIVER_TIMEOUT 60000/* A minute in milliseconds */ + +/* The AP and modem perform a "handshake" at initialization time to ensure +* each side knows the other side is ready. Two QMI handles (endpoints) are +* used for this; one provides service on the modem for AP requests, and the +* other is on the AP to service modem requests (and to supply an indication +* from the AP). +* +* The QMI service on the modem expects to receive an INIT_DRIVER request from +* the AP, which contains parameters used by the modem during initialization. +* The AP sends this request using the client handle as soon as it is knows +* the modem side service is available. The modem responds to this request +* immediately. +* +* When the modem learns the AP service is available, it is able to +* communicate its status to the AP. The modem uses this to tell +* the AP when it is ready to receive an indication, sending an +* INDICATION_REGISTER request to the handle served by the AP. This +* is independent of the modem's initialization of its driver. +* +* When the modem has completed the driver initialization requested by the +* AP, it sends a DRIVER_INIT_COMPLETE request to the AP. This request +* could arrive at the AP either before or after the INDICATION_REGISTER +* request. +* +* The final step in the handshake occurs after the AP has received both +* requests from the modem. The AP completes the handshake by sending an +* INIT_COMPLETE_IND indication message to the modem. +*/ + +#define IPA_HOST_SERVICE_SVC_ID 0x31 +#define IPA_HOST_SVC_VERS 1 +#define IPA_HOST_SERVICE_INS_ID 1 + +#define IPA_MODEM_SERVICE_SVC_ID 0x31 +#define IPA_MODEM_SERVICE_INS_ID 2 +#define IPA_MODEM_SVC_VERS 1 + +/* Send an INIT_COMPLETE_IND indication message to the modem */ +static int ipa_send_master_driver_init_complete_ind (struct qmi_handle *qmi, +struct sockaddr_qrtr *sq) +{ +struct ipa_init_complete_ind ind = { }; + + ind.status.result =QMI_RESULT_SUCCESS_V01 ; + ind.status.error =QMI_ERR_NONE_V01 ; + +return qmi_send_indication (qmi, sq,IPA_QMI_INIT_COMPLETE_IND , +IPA_QMI_INIT_COMPLETE_IND_SZ , + ipa_init_complete_ind_ei, &ind); +} + +/* This function is called to determine whether to complete the handshake by +* sending an INIT_COMPLETE_IND indication message to the modem. The +* "init_driver" parameter is false when we've received an INDICATION_REGISTER +* request message from the modem, or true when we've received the response +* from the INIT_DRIVER request message we send. If this function decides the +* message should be sent, it calls ipa_send_master_driver_init_complete_ind() +* to send it. +*/ +static void ipa_handshake_complete (struct qmi_handle *qmi, +struct sockaddr_qrtr *sq,bool init_driver) +{ +struct ipa *ipa; +bool send_it; +int ret; + +if (init_driver) { + ipa = container_of(qmi, struct ipa, qmi.client_handle); + ipa->qmi.init_driver_response_received = true; + send_it = !!ipa->qmi.indication_register_received; + }else { + ipa = container_of(qmi, struct ipa, qmi.server_handle); + ipa->qmi.indication_register_received =1 ; + send_it = !!ipa->qmi.init_driver_response_received; + } +if (!send_it) +return ; + + ret =ipa_send_master_driver_init_complete_ind (qmi, sq); +WARN (ret,"error %d sending init complete indication\n" , ret); +} + +/* Callback function to handle an INDICATION_REGISTER request message from the +* modem. This informs the AP that the modem is now ready to receive the +* INIT_COMPLETE_IND indication message. +*/ +static void ipa_indication_register_fn (struct qmi_handle *qmi, +struct sockaddr_qrtr *sq, +struct qmi_txn *txn, +const void *decoded) +{ +struct ipa_indication_register_rsp rsp = { }; +int ret; + + rsp.rsp.result =QMI_RESULT_SUCCESS_V01 ; + rsp.rsp.error =QMI_ERR_NONE_V01 ; + + ret =qmi_send_response (qmi, sq, txn,IPA_QMI_INDICATION_REGISTER , +IPA_QMI_INDICATION_REGISTER_RSP_SZ , + ipa_indication_register_rsp_ei, &rsp); +if (!WARN (ret,"error %d sending response\n" , ret)) +ipa_handshake_complete (qmi, sq, false); +} + +/* Callback function to handle a DRIVER_INIT_COMPLETE request message from the +* modem. This informs the AP that the modem has completed the initializion +* of its driver. +*/ +static void ipa_driver_init_complete_fn (struct qmi_handle *qmi, +struct sockaddr_qrtr *sq, +struct qmi_txn *txn, +const void *decoded) +{ +struct ipa_driver_init_complete_rsp rsp = { }; +int ret; + + rsp.rsp.result =QMI_RESULT_SUCCESS_V01 ; + rsp.rsp.error =QMI_ERR_NONE_V01 ; + + ret =qmi_send_response (qmi, sq, txn,IPA_QMI_DRIVER_INIT_COMPLETE , +IPA_QMI_DRIVER_INIT_COMPLETE_RSP_SZ , + ipa_driver_init_complete_rsp_ei, &rsp); + +WARN (ret,"error %d sending response\n" , ret); +} + +/* The server handles two request message types sent by the modem. */ +static struct qmi_msg_handler ipa_server_msg_handlers[] = { + { + .type =QMI_REQUEST , + .msg_id =IPA_QMI_INDICATION_REGISTER , + .ei = ipa_indication_register_req_ei, + .decoded_size =IPA_QMI_INDICATION_REGISTER_REQ_SZ , + .fn = ipa_indication_register_fn, + }, + { + .type =QMI_REQUEST , + .msg_id =IPA_QMI_DRIVER_INIT_COMPLETE , + .ei = ipa_driver_init_complete_req_ei, + .decoded_size =IPA_QMI_DRIVER_INIT_COMPLETE_REQ_SZ , + .fn = ipa_driver_init_complete_fn, + }, +}; + +/* Callback function to handle an IPA_QMI_INIT_DRIVER response message from +* the modem. This only acknowledges that the modem received the request. +* The modem will eventually report that it has completed its modem +* initialization by sending a IPA_QMI_DRIVER_INIT_COMPLETE request. +*/ +static void ipa_init_driver_rsp_fn (struct qmi_handle *qmi, +struct sockaddr_qrtr *sq, +struct qmi_txn *txn, +const void *decoded) +{ + txn->result =0 ;/* IPA_QMI_INIT_DRIVER request was successful */ +complete (&txn->completion); + +ipa_handshake_complete (qmi, sq, true); +} + +/* The client handles one response message type sent by the modem. */ +static struct qmi_msg_handler ipa_client_msg_handlers[] = { + { + .type =QMI_RESPONSE , + .msg_id =IPA_QMI_INIT_DRIVER , + .ei = ipa_init_modem_driver_rsp_ei, + .decoded_size =IPA_QMI_INIT_DRIVER_RSP_SZ , + .fn = ipa_init_driver_rsp_fn, + }, +}; + +/* Return a pointer to an init modem driver request structure, which contains +* configuration parameters for the modem. The modem may be started multiple +* times, but generally these parameters don't change so we can reuse the +* request structure once it's initialized. The only exception is the +* skip_uc_load field, which will be set only after the microcontroller has +* reported it has completed its initialization. +*/ +static const struct ipa_init_modem_driver_req * +init_modem_driver_req (struct ipa_qmi *ipa_qmi) +{ + struct ipa *ipa = container_of(ipa_qmi, struct ipa, qmi); +static struct ipa_init_modem_driver_req req; + +/* This is not the first boot if the microcontroller is loaded */ + req.skip_uc_load = ipa->uc_loaded; + req.skip_uc_load_valid = true; + +/* We only have to initialize most of it once */ +if (req.platform_type_valid) +return &req; + + req.platform_type_valid = true; + req.platform_type =IPA_QMI_PLATFORM_TYPE_MSM_ANDROID ; + + req.hdr_tbl_info_valid =IPA_SMEM_MODEM_HDR_SIZE ?1 :0 ; + req.hdr_tbl_info.start = ipa_qmi->base +IPA_SMEM_MODEM_HDR_OFFSET ; + req.hdr_tbl_info.end = req.hdr_tbl_info.start + +IPA_SMEM_MODEM_HDR_SIZE -1 ; + + req.v4_route_tbl_info_valid = true; + req.v4_route_tbl_info.start = + ipa_qmi->base +IPA_SMEM_V4_RT_NHASH_OFFSET ; + req.v4_route_tbl_info.count =IPA_SMEM_MODEM_RT_COUNT ; + + req.v6_route_tbl_info_valid = true; + req.v6_route_tbl_info.start = + ipa_qmi->base +IPA_SMEM_V6_RT_NHASH_OFFSET ; + req.v6_route_tbl_info.count =IPA_SMEM_MODEM_RT_COUNT ; + + req.v4_filter_tbl_start_valid = true; + req.v4_filter_tbl_start = ipa_qmi->base +IPA_SMEM_V4_FLT_NHASH_OFFSET ; + + req.v6_filter_tbl_start_valid = true; + req.v6_filter_tbl_start = ipa_qmi->base +IPA_SMEM_V6_FLT_NHASH_OFFSET ; + + req.modem_mem_info_valid =IPA_SMEM_MODEM_SIZE ?1 :0 ; + req.modem_mem_info.start = ipa_qmi->base +IPA_SMEM_MODEM_OFFSET ; + req.modem_mem_info.size =IPA_SMEM_MODEM_SIZE ; + + req.ctrl_comm_dest_end_pt_valid = true; + req.ctrl_comm_dest_end_pt =IPA_ENDPOINT_AP_MODEM_RX ; + + req.hdr_proc_ctx_tbl_info_valid = +IPA_SMEM_MODEM_HDR_PROC_CTX_SIZE ?1 :0 ; + req.hdr_proc_ctx_tbl_info.start = + ipa_qmi->base +IPA_SMEM_MODEM_HDR_PROC_CTX_OFFSET ; + req.hdr_proc_ctx_tbl_info.end = req.hdr_proc_ctx_tbl_info.start + +IPA_SMEM_MODEM_HDR_PROC_CTX_SIZE -1 ; + + req.v4_hash_route_tbl_info_valid = true; + req.v4_hash_route_tbl_info.start = + ipa_qmi->base +IPA_SMEM_V4_RT_HASH_OFFSET ; + req.v4_hash_route_tbl_info.count =IPA_SMEM_MODEM_RT_COUNT ; + + req.v6_hash_route_tbl_info_valid = true; + req.v6_hash_route_tbl_info.start = + ipa_qmi->base +IPA_SMEM_V6_RT_HASH_OFFSET ; + req.v6_hash_route_tbl_info.count =IPA_SMEM_MODEM_RT_COUNT ; + + req.v4_hash_filter_tbl_start_valid = true; + req.v4_hash_filter_tbl_start = + ipa_qmi->base +IPA_SMEM_V4_FLT_HASH_OFFSET ; + + req.v6_hash_filter_tbl_start_valid = true; + req.v6_hash_filter_tbl_start = + ipa_qmi->base +IPA_SMEM_V6_FLT_HASH_OFFSET ; + +return &req; +} + +/* The modem service we requested is now available via the client handle. +* Send an INIT_DRIVER request to the modem. +*/ +static int +ipa_client_new_server (struct qmi_handle *qmi,struct qmi_service *svc) +{ +const struct ipa_init_modem_driver_req *req; +struct ipa *ipa; +struct sockaddr_qrtr sq; +struct qmi_txn *txn; +int ret; + + ipa = container_of(qmi, struct ipa, qmi.client_handle); + req =init_modem_driver_req (&ipa->qmi); + + txn =kzalloc (sizeof (*txn),GFP_KERNEL ); +if (!txn) +return -ENOMEM ; + + ret =qmi_txn_init (qmi, txn,NULL ,NULL ); +if (ret) { +kfree (txn); +return ret; + } + + sq.sq_family =AF_QIPCRTR ; + sq.sq_node = svc->node; + sq.sq_port = svc->port; + + ret =qmi_send_request (qmi, &sq, txn,IPA_QMI_INIT_DRIVER , +IPA_QMI_INIT_DRIVER_REQ_SZ , + ipa_init_modem_driver_req_ei, req); +if (!ret) + ret =qmi_txn_wait (txn,MAX_SCHEDULE_TIMEOUT ); +if (ret) +qmi_txn_cancel (txn); +kfree (txn); + +return ret; +} + +/* The only callback we supply for the client handle is notification that the +* service on the modem has become available. +*/ +static struct qmi_ops ipa_client_ops = { + .new_server = ipa_client_new_server, +}; + +static int ipa_qmi_initialize (struct ipa *ipa) +{ +struct ipa_qmi *ipa_qmi = &ipa->qmi; +int ret; + +/* The only handle operation that might be interesting for the server +* would be del_client, to find out when the modem side client has +* disappeared. But other than reporting the event, we wouldn't do +* anything about that. So we just pass a null pointer for its handle +* operations. All the real work is done by the message handlers. +*/ + ret =qmi_handle_init (&ipa_qmi->server_handle, +IPA_QMI_SERVER_MAX_RCV_SZ ,NULL , + ipa_server_msg_handlers); +if (ret) +return ret; + + ret =qmi_add_server (&ipa_qmi->server_handle,IPA_HOST_SERVICE_SVC_ID , +IPA_HOST_SVC_VERS ,IPA_HOST_SERVICE_INS_ID ); +if (ret) + goto err_release_server_handle; + +/* The client handle is only used for sending an INIT_DRIVER request +* to the modem, and receiving its response message. +*/ + ret =qmi_handle_init (&ipa_qmi->client_handle, +IPA_QMI_CLIENT_MAX_RCV_SZ , &ipa_client_ops, + ipa_client_msg_handlers); +if (ret) + goto err_release_server_handle; + + ret =qmi_add_lookup (&ipa_qmi->client_handle,IPA_MODEM_SERVICE_SVC_ID , +IPA_MODEM_SVC_VERS ,IPA_MODEM_SERVICE_INS_ID ); +if (ret) + goto err_release_client_handle; + +/* All QMI offsets are relative to the start of IPA shared memory */ + ipa_qmi->base = ipa->shared_offset; + ipa_qmi->initialized =1 ; + +return 0 ; + +err_release_client_handle: +/* Releasing the handle also removes registered lookups */ +qmi_handle_release (&ipa_qmi->client_handle); +memset (&ipa_qmi->client_handle,0 ,sizeof (ipa_qmi->client_handle)); +err_release_server_handle: +/* Releasing the handle also removes registered services */ +qmi_handle_release (&ipa_qmi->server_handle); +memset (&ipa_qmi->server_handle,0 ,sizeof (ipa_qmi->server_handle)); + +return ret; +} + +/* This is called by ipa_netdev_setup(). We can be informed via remoteproc +* that the modem has shut down, in which case this function will be called +* again to prepare for it coming back up again. +*/ +int ipa_qmi_setup (struct ipa *ipa) +{ + ipa->qmi.init_driver_response_received =0 ; + ipa->qmi.indication_register_received =0 ; + +if (!ipa->qmi.initialized) +return ipa_qmi_initialize (ipa); + +return 0 ; +} + +void ipa_qmi_teardown (struct ipa *ipa) +{ +if (!ipa->qmi.initialized) +return ; + +qmi_handle_release (&ipa->qmi.client_handle); +memset (&ipa->qmi.client_handle,0 ,sizeof (ipa->qmi.client_handle)); + +qmi_handle_release (&ipa->qmi.server_handle); +memset (&ipa->qmi.server_handle,0 ,sizeof (ipa->qmi.server_handle)); + + ipa->qmi.initialized =0 ; +}
drivers/net/ipa/ipa_qmi.h new file +35 -0
@@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* Copyright (c) 2018, The Linux Foundation. All rights reserved. +* Copyright (C) 2018-2019 Linaro Ltd. +*/ +#ifndef _IPA_QMI_H_ +#define _IPA_QMI_H_ + +#include <linux/types.h> +#include <linux/soc/qcom/qmi.h> + +struct ipa ; + +/** +* struct ipa_qmi - QMI state associated with an IPA +* @initialized - whether QMI initialization has completed +* @client_handle - used to send an QMI requests to the modem +* @server_handle - used to handle QMI requests from the modem +* @indication_register_received - tracks modem request receipt +* @init_driver_response_received - tracks modem response receipt +*/ +struct ipa_qmi { +u32 initialized; +u32 base; +struct qmi_handle client_handle; +struct qmi_handle server_handle; +u32 indication_register_received; +u32 init_driver_response_received; + +}; + +int ipa_qmi_setup (struct ipa *ipa); +void ipa_qmi_teardown (struct ipa *ipa); + +#endif /* !_IPA_QMI_H_ */
drivers/net/ipa/ipa_qmi_msg.c new file +583 -0
583 lines not on this page · raw message
drivers/net/ipa/ipa_qmi_msg.h new file +238 -0
238 lines not on this page · raw message
drivers/net/ipa/ipa_smp2p.c new file +304 -0
304 lines not on this page · raw message
drivers/net/ipa/ipa_smp2p.h new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 15/18 soc: qcom: ipa: support build of IPA code
In reply to Alex ElderAdd build and Kconfig support for the Qualcomm IPA driver.
--- drivers/net/Kconfig | 2 ++ drivers/net/Makefile | 1 + drivers/net/ipa/Kconfig | 16 ++++++++++++++++ drivers/net/ipa/Makefile | 7 +++++++ 4 files changed, 26 insertions(+) create mode 100644 drivers/net/ipa/Kconfig create mode 100644 drivers/net/ipa/Makefile
drivers/net/Kconfig +2 -0
@@ -387,6 +387,8 @@ source "drivers/net/fddi/Kconfig" source "drivers/net/hippi/Kconfig" +source "drivers/net/ipa/Kconfig" + config NET_SB1000 tristate "General Instruments Surfboard 1000" depends on PNP
drivers/net/Makefile +1 -0
@@ -45,6 +45,7 @@ obj-$(CONFIG_ETHERNET) += ethernet/ obj-$(CONFIG_FDDI) += fddi/ obj-$(CONFIG_HIPPI) += hippi/ obj-$(CONFIG_HAMRADIO) += hamradio/ +obj-$(CONFIG_IPA) += ipa/ obj-$(CONFIG_PLIP) += plip/ obj-$(CONFIG_PPP) += ppp/ obj-$(CONFIG_PPP_ASYNC) += ppp/
drivers/net/ipa/Kconfig new file +16 -0
@@ -0,0 +1,16 @@ +config IPA + tristate "Qualcomm IPA support" + depends on NET + select QCOM_QMI_HELPERS + select QCOM_MDT_LOADER + default n + help + Choose Y here to include support for the Qualcomm IP Accelerator + (IPA), a hardware block present in some Qualcomm SoCs. The IPA + is a programmable protocol processor that is capable of generic + hardware handling of IP packets, including routing, filtering, + and NAT. Currently the IPA driver supports only basic transport + of network traffic between the AP and modem, on the Qualcomm + SDM845 SoC. + + If unsure, say N.
drivers/net/ipa/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +obj-$(CONFIG_IPA) += ipa.o + +ipa-y := ipa_main.o ipa_clock.o ipa_mem.o \ + ipa_interrupt.o gsi.o gsi_trans.o \ + ipa_gsi.o ipa_smp2p.o ipa_uc.o \ + ipa_endpoint.o ipa_cmd.o ipa_netdev.o \ + ipa_qmi.o ipa_qmi_msg.o ipa_data-sdm845.o
Signature · 1 line
2.20.1
PATCH 16/18 MAINTAINERS: add entry for the Qualcomm IPA driver
In reply to Alex ElderAdd an entry in the MAINTAINERS file for the Qualcomm IPA driver
--- MAINTAINERS | 6 ++++++ 1 file changed, 6 insertions(+)
MAINTAINERS +6 -0
@@ -12694,6 +12694,12 @@ L: [email protected] (moderated for non-subscribers) S: Supported F: sound/soc/qcom/ +QCOM IPA DRIVER +M: Alex Elder <[email protected]> +L: [email protected] +S: Supported +F: drivers/net/ipa/ + QEMU MACHINE EMULATOR AND VIRTUALIZER SUPPORT M: Gabriel Somlo <[email protected]> M: "Michael S. Tsirkin" <[email protected]>
Signature · 1 line
2.20.1
PATCH 17/18 arm64: dts: sdm845: add IPA information
In reply to Alex ElderAdd IPA-related nodes and definitions to "sdm845.dtsi".
--- arch/arm64/boot/dts/qcom/sdm845.dtsi | 51 ++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+)
arch/arm64/boot/dts/qcom/sdm845.dtsi +51 -0
@@ -18,6 +18,7 @@ #include <dt-bindings/soc/qcom,rpmh-rsc.h> #include <dt-bindings/clock/qcom,gcc-sdm845.h> #include <dt-bindings/thermal/thermal.h> +#include <dt-bindings/interconnect/qcom,sdm845.h> / { interrupt-parent = <&intc>;
@@ -342,6 +343,17 @@ interrupt-controller; #interrupt-cells = <2>; }; + + ipa_smp2p_out: ipa-ap-to-modem { + qcom,entry-name = "ipa"; + #qcom,smem-state-cells = <1>; + }; + + ipa_smp2p_in: ipa-modem-to-ap { + qcom,entry-name = "ipa"; + interrupt-controller; + #interrupt-cells = <2>; + }; }; smp2p-slpi {
@@ -1090,6 +1102,45 @@ }; }; + ipa@1e40000 { + compatible = "qcom,sdm845-ipa"; + + modem-init; + + reg = <0 0x1e40000 0 0x7000>, + <0 0x1e47000 0 0x2000>, + <0 0x1e04000 0 0x2c000>; + reg-names = "ipa-reg", + "ipa-shared", + "gsi"; + + interrupts-extended = + <&intc 0 311 IRQ_TYPE_EDGE_RISING>, + <&intc 0 432 IRQ_TYPE_LEVEL_HIGH>, + <&ipa_smp2p_in 0 IRQ_TYPE_EDGE_RISING>, + <&ipa_smp2p_in 1 IRQ_TYPE_EDGE_RISING>; + interrupt-names = "ipa", + "gsi", + "ipa-clock-query", + "ipa-setup-ready"; + + clocks = <&rpmhcc RPMH_IPA_CLK>; + clock-names = "core"; + + interconnects = + <&rsc_hlos MASTER_IPA &rsc_hlos SLAVE_EBI1>, + <&rsc_hlos MASTER_IPA &rsc_hlos SLAVE_IMEM>, + <&rsc_hlos MASTER_APPSS_PROC &rsc_hlos SLAVE_IPA_CFG>; + interconnect-names = "memory", + "imem", + "config"; + + qcom,smem-states = <&ipa_smp2p_out 0>, + <&ipa_smp2p_out 1>; + qcom,smem-state-names = "ipa-clock-enabled-valid", + "ipa-clock-enabled"; + }; + tcsr_mutex_regs: syscon@1f40000 { compatible = "syscon"; reg = <0 0x01f40000 0 0x40000>;
Signature · 1 line
2.20.1
PATCH 18/18 arm64: defconfig: enable build of IPA code
In reply to Alex ElderAdd CONFIG_IPA to the 64-bit Arm defconfig.
--- arch/arm64/configs/defconfig | 1 + 1 file changed, 1 insertion(+)
arch/arm64/configs/defconfig +1 -0
@@ -268,6 +268,7 @@ CONFIG_SMSC911X=y CONFIG_SNI_AVE=y CONFIG_SNI_NETSEC=y CONFIG_STMMAC_ETH=m +CONFIG_IPA=y CONFIG_MDIO_BUS_MUX_MMIOREG=y CONFIG_AT803X_PHY=m CONFIG_MARVELL_PHY=m
Signature · 1 line
2.20.1
Re: [PATCH 18/18] arm64: defconfig: enable build of IPA code
arch/arm64/configs/defconfig CONFIG_SMSC911X=y · line 274
Quoted 12 lines · Alex Elder
arch/arm64/configs/defconfig +1
-0@@ -268,6 +268,7 @@ CONFIG_SMSC911X=y CONFIG_SNI_AVE=y CONFIG_SNI_NETSEC=y CONFIG_STMMAC_ETH=m +CONFIG_IPA=y CONFIG_MDIO_BUS_MUX_MMIOREG=y CONFIG_AT803X_PHY=m CONFIG_MARVELL_PHY=m
Since the device is not needed for booting, please make this CONFIG_IPA=m instead to keep the kernel image a little smaller.
Arnd
Re: [PATCH 18/18] arm64: defconfig: enable build of IPA code
Quoted 18 lines · Arnd Bergmann
arch/arm64/configs/defconfig CONFIG_SMSC911X=y · line 274
Quoted 12 lines · Alex Elder
arch/arm64/configs/defconfig +1
-0@@ -268,6 +268,7 @@ CONFIG_SMSC911X=y CONFIG_SNI_AVE=y CONFIG_SNI_NETSEC=y CONFIG_STMMAC_ETH=m +CONFIG_IPA=y CONFIG_MDIO_BUS_MUX_MMIOREG=y CONFIG_AT803X_PHY=m CONFIG_MARVELL_PHY=mSince the device is not needed for booting, please make this
CONFIG_IPA=minstead to keep the kernel image a little smaller.
Quoted 2 lines
Arnd
Oops, yes, that was my intention but I forgot to fix that
before I sent it out. This code works as a module, but
in order to make the whole system allow the module to be both removed and re-inserted safely, I need some work to be done on the modem end and that's beyond my direct control. I have been testing with it as a kernel built-in driver in the mean time.
In any case, it is my intention to have it be normally built as a module and I will ensure that when I send out future revisions of this series.
-Alex
Quoted 6 lines · Alex Elder
A version of this code was posted in November 2018 as an RFC.
https://lore.kernel.org/lkml/[email protected]/
Fixes addressing all feedback received have been implemented. It has undergone considerable further rework since that time, and most of the "future work" described then has now been completed.
I think this has turned out really well now. I've gone through the patches today and not found any real show-stoppers, but replied with a couple of minor things I noticed.
I think it's probably worth rearranging the rx and tx code to avoid the spinlocks, which would be the main optimization I can still think of to reduce the coherency traffic.
Arnd
Quoted 8 lines · Arnd Bergmann
Quoted 6 lines · Alex Elder
A version of this code was posted in November 2018 as an RFC.
https://lore.kernel.org/lkml/[email protected]/
Fixes addressing all feedback received have been implemented. It has undergone considerable further rework since that time, and most of the "future work" described then has now been completed.
Quoted 7 lines
I think this has turned out really well now. I've gone through the patches today and not found any real show-stoppers, but replied with a couple of minor things I noticed.
I think it's probably worth rearranging the rx and tx code to avoid the spinlocks, which would be the main optimization I can still think of to reduce the coherency traffic.
Arnd I appreciate your review *so* much. You clearly committed a significant block of time to this and your comments were very insightful and focused. I concur with everything you pointed out and will address all of your concerns when I send out v2 of the series.
Thank you for your careful review.
-Alex
Quoted 3 lines
Arnd