Skip to content

RFC Implement NTB Controller using multiple PCI

Kishon Vijay Abraham I linux-devicetree linux-doc linux-pci lkml ntb 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest21 patches 7 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/21dt-bindings: PCI: Endpoint: Add DT bindings for PCI EPF BusDocumentation/devicetree/bindings/pci/endpoint/pci-epf-bus.txt+27 -00 replies
2/21dt-bindings: PCI: Endpoint: Add DT bindings for PCI EPF DeviceDocumentation/devicetree/bindings/pci/endpoint/pci-epf.txt+28 -02 replies
3/21dt-bindings: PCI: Endpoint: Add DT bindings for PCI EPF NTB DeviceDocumentation/devicetree/bindings/pci/endpoint/pci-epf-ntb.txt+31 -00 replies
4/21Documentation: PCI: Add specification for the *PCI NTB* function deviceDocumentation/PCI/endpoint/pci-test-ntb.txt+315 -01 reply
5/21PCI: endpoint: Add API to get reference to EPC from device-treedrivers/pci/endpoint/pci-epc-core.c include/linux/pci-epc.h+64 -11 reply
6/21PCI: endpoint: Add API to create EPF device from device treedrivers/pci/endpoint/pci-epf-core.c include/linux/pci-epf.h+74 -00 replies
7/21PCI: endpoint: Add "pci-epf-bus" driverdrivers/pci/endpoint/Makefile drivers/pci/endpoint/pci-epf-bus.c+56 -10 replies
8/21PCI: endpoint: Make *_get_first_free_bar() take into account 64 bit BARdrivers/pci/endpoint/pci-epc-core.c+10 -20 replies
9/21PCI: endpoint: Add helper API to get the 'next' unreserved BARdrivers/pci/endpoint/pci-epc-core.c include/linux/pci-epc.h+24 -40 replies
10/21PCI: endpoint: Make pci_epf_driver ops optionaldrivers/pci/endpoint/pci-epf-core.c+3 -50 replies
11/21PCI: endpoint: Add helper API to populate header with values from DTdrivers/pci/endpoint/pci-epc-core.c include/linux/pci-epc.h+25 -00 replies
12/21PCI: endpoint: Add support to associate secondary EPC with EPFdrivers/pci/endpoint/functions/pci-epf-test.c drivers/pci/endpoint/pci-ep-cfs.c drivers/pci/endpoint/pci-epc-core.c +3 more+135 -340 replies
13/21PCI: endpoint: Add pci_epc_ops to map MSI irqdrivers/pci/endpoint/pci-epc-core.c include/linux/pci-epc.h+47 -00 replies
14/21PCI: cadence: Implement ->msi_map_irq() opsdrivers/pci/controller/pcie-cadence-ep.c+59 -00 replies
15/21PCI: endpoint: Remove unused pci_epf_match_device()drivers/pci/endpoint/pci-epf-core.c include/linux/pci-epf.h+0 -180 replies
16/21PCI: endpoint: Fix missing mutex_unlock in error casedrivers/pci/endpoint/pci-epc-core.c+8 -30 replies
17/21PCI: endpoint: *_free_bar() to return error codes on failuredrivers/pci/endpoint/pci-epc-core.c include/linux/pci-epc.h+7 -90 replies
18/21PCI: endpoint: Add EP function driver to provide NTB functionalitydrivers/pci/endpoint/functions/Kconfig drivers/pci/endpoint/functions/Makefile drivers/pci/endpoint/functions/pci-epf-ntb.c+1156 -00 replies
19/21PCI: Add TI J721E device to pci idsinclude/linux/pci_ids.h+1 -01 reply
20/21NTB: Add support for EPF PCI-Express Non-Transparent Bridgedrivers/ntb/hw/Kconfig drivers/ntb/hw/Makefile drivers/ntb/hw/epf/Kconfig +2 more+656 -01 reply
21/21NTB: tool: Enable the NTB/PCIe link on the local or remote side of bridgedrivers/ntb/test/ntb_tool.c+1 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 00/21] Implement NTB Controller using multiple PCI

This series is sent as RFC since this series is dependent on
[1] (cannot be merged before that series) and to get early review comments.

I'll also split this series into smaller chunk when I post the next revision.

This series is about implementing SW defined NTB using multiple endpoint instances. This series has been tested using 2 endpoint instances in J7 connected to two DRA7 boards.

This was presented in Linux Plumbers Conference. The presentation can be found @ [2]

This series:
*) Add support to define endpoint function using device tree *) Add a specification for implementing NTB controller using
multiple endpoint instances.
*) Add a NTB endpoint function driver and a NTB host side PCI
driver that follows the specification.
*) Add support in PCIe endpoint core to support secondary
interface.
*) Add a device tree overlay file to configure J7 as NTB

The test setup is something like below

   +-------------+                                   +-------------+
   |             |                                   |             |
   |    DRA72    |                                   |    DRA76    |
   |             |                                   |             |
   +------^------+                                   +------^------+
          |                                                 |
          |                                                 |
+---------|-------------------------------------------------|---------+
|  +------v------+                                   +------v------+  |
|  |             |                                   |             |  |
|  |     EP      |                                   |     EP      |  |
|  | CONTROLLER1 |                                   | CONTROLLER2 |  |
|  |             <----------------------------------->             |  |
|  |             |                                   |             |  |
|  |             |                                   |             |  |
|  |             |                 J7                |             |  |
|  |             |  (Configured using NTB Function)  |             |  |
|  +-------------+                                   +-------------+  |
+---------------------------------------------------------------------+

Here DRA72 and DRA76 could be replaced with *any* PCI host.

EP side (J7):
=============

In the kernel:

        cd /sys/kernel/config/pci_ep/
        echo 1 > controllers/d800000.pcie-ep/start
        echo 1 > controllers/d000000.pcie-ep/start

RC side (DRA7):
===============

        echo 0000:01:00.0 > /sys/bus/pci/devices/0000\:01\:00.0/driver/unbind
        echo 0000:01:00.0 > /sys/bus/pci/drivers/ntb_hw_epf/bind
        modprobe ntb_transport
        modprobe ntb_netdev

On each of the hosts Ethernet Interface will be created.

Provide an IP address to each of the hosts:
HOST1 (dra72):
ifconfig eth2 192.168.1.2 up

HOST2 (dra76):
ifconfig eth2 192.168.1.1 up

Once this is done standard network utilities like ping or iperf can be used.

root@dra7xx-evm:~# iperf -c 192.168.1.2
------------------------------------------------------------
Client connecting to 192.168.1.2, TCP port 5001
TCP window size: 2.50 MByte (default)
------------------------------------------------------------
[  3] local 192.168.1.1 port 60814 connected with 192.168.1.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   705 MBytes   591 Mbits/sec

[1] -> http://lore.kernel.org/r/[email protected]
[2] -> https://www.linuxplumbersconf.org/event/4/contributions/395/attachments/284/481/Implementing_NTB_Controller_Using_PCIe_Endpoint_-_final.pdf

Kishon Vijay Abraham I (21):
dt-bindings: PCI: Endpoint: Add DT bindings for PCI EPF Bus
dt-bindings: PCI: Endpoint: Add DT bindings for PCI EPF Device
dt-bindings: PCI: Endpoint: Add DT bindings for PCI EPF NTB Device
Documentation: PCI: Add specification for the *PCI NTB* function
device
PCI: endpoint: Add API to get reference to EPC from device-tree
PCI: endpoint: Add API to create EPF device from device tree
PCI: endpoint: Add "pci-epf-bus" driver
PCI: endpoint: Make *_get_first_free_bar() take into account 64 bit
BAR
PCI: endpoint: Add helper API to get the 'next' unreserved BAR
PCI: endpoint: Make pci_epf_driver ops optional
PCI: endpoint: Add helper API to populate header with values from DT
PCI: endpoint: Add support to associate secondary EPC with EPF
PCI: endpoint: Add pci_epc_ops to map MSI irq
PCI: cadence: Implement ->msi_map_irq() ops
PCI: endpoint: Remove unused pci_epf_match_device()
PCI: endpoint: Fix missing mutex_unlock in error case
PCI: endpoint: *_free_bar() to return error codes on failure
PCI: endpoint: Add EP function driver to provide NTB functionality
PCI: Add TI J721E device to pci ids
NTB: Add support for EPF PCI-Express Non-Transparent Bridge
NTB: tool: Enable the NTB/PCIe link on the local or remote side of
bridge

 Documentation/PCI/endpoint/pci-test-ntb.txt   |  315 +++++
 .../bindings/pci/endpoint/pci-epf-bus.txt     |   27 +
 .../bindings/pci/endpoint/pci-epf-ntb.txt     |   31 +
 .../bindings/pci/endpoint/pci-epf.txt         |   28 +
 drivers/ntb/hw/Kconfig                        |    1 +
 drivers/ntb/hw/Makefile                       |    1 +
 drivers/ntb/hw/epf/Kconfig                    |    5 +
 drivers/ntb/hw/epf/Makefile                   |    1 +
 drivers/ntb/hw/epf/ntb_hw_epf.c               |  648 ++++++++++
 drivers/ntb/test/ntb_tool.c                   |    1 +
 drivers/pci/controller/pcie-cadence-ep.c      |   59 +
 drivers/pci/endpoint/Makefile                 |    3 +-
 drivers/pci/endpoint/functions/Kconfig        |   12 +
 drivers/pci/endpoint/functions/Makefile       |    1 +
 drivers/pci/endpoint/functions/pci-epf-ntb.c  | 1143 +++++++++++++++++
 drivers/pci/endpoint/functions/pci-epf-test.c |   12 +-
 drivers/pci/endpoint/pci-ep-cfs.c             |    6 +-
 drivers/pci/endpoint/pci-epc-core.c           |  221 +++-
 drivers/pci/endpoint/pci-epf-bus.c            |   54 +
 drivers/pci/endpoint/pci-epf-core.c           |  133 +-
 include/linux/pci-epc.h                       |   42 +-
 include/linux/pci-epf.h                       |   35 +-
 include/linux/pci_ids.h                       |    1 +
 23 files changed, 2715 insertions(+), 65 deletions(-)
 create mode 100644 Documentation/PCI/endpoint/pci-test-ntb.txt
 create mode 100644 Documentation/devicetree/bindings/pci/endpoint/pci-epf-bus.txt
 create mode 100644 Documentation/devicetree/bindings/pci/endpoint/pci-epf-ntb.txt
 create mode 100644 Documentation/devicetree/bindings/pci/endpoint/pci-epf.txt
 create mode 100644 drivers/ntb/hw/epf/Kconfig
 create mode 100644 drivers/ntb/hw/epf/Makefile
 create mode 100644 drivers/ntb/hw/epf/ntb_hw_epf.c
 create mode 100644 drivers/pci/endpoint/functions/pci-epf-ntb.c
 create mode 100644 drivers/pci/endpoint/pci-epf-bus.c
Signature · 1 line
2.17.1