Implement NTB Controller using multiple PCI EP
Subsystems (7)
v7 cover Implement NTB Controller using multiple PCI EP
This series is about implementing SW defined Non-Transparent Bridge (NTB) using multiple endpoint (EP) instances. This series has been tested using 2 endpoint instances in J7 connected to J7 board on one end and DRA7 board on the other end. However there is nothing platform specific for the NTB functionality.
This was presented in Linux Plumbers Conference. Link to presentation and video can be found @ [1]
RFC patch series can be found @ [2]
v1 patch series can be found @ [3]
v2 patch series can be found @ [4]
v3 patch series can be found @ [5]
v4 patch series can be found @ [6]
v5 patch series can be found @ [7]
v6 patch series can be found @ [8]
Changes from v6:
1) Fixed issues when multiple NTB devices are creating using multiple functions
2) Fixed issue with writing scratchpad register
3) Created a video demo @ [9]
Changes from v5:
1) Fixed a formatting issue in Kconfig pointed out by Randy
2) Checked for Error or Null in pci_epc_add_epf()
Changes from v4:
1) Fixed error condition checks in pci_epc_add_epf()
Changes from v3:
1) Fixed Documentation edits suggested by Randy Dunlap <[email protected]>
Changes from v2:
1) Add support for the user to create sub-directory of 'EPF Device' directory (for endpoint function specific configuration using configfs).
2) Add documentation for NTB specific attributes in configfs
3) Check for PCI_CLASS_MEMORY_RAM (PCIe class) before binding ntb_hw_epf driver
4) Other documentation fixes
Changes from v1:
1) As per Rob's comment, removed support for creating NTB function device from DT
2) Add support to create NTB EPF device using configfs (added support in configfs to associate primary and secondary EPC with EPF.
Changes from RFC:
1) Converted the DT binding patches to YAML schema and merged the DT binding patches together
2) NTB documentation is converted to .rst
3) One HOST can now interrupt the other HOST using MSI-X interrupts
4) Added support for teardown of memory window and doorbell configuration
5) Add support to provide support 64-bit memory window size from DT
[1] -> https://linuxplumbersconf.org/event/4/contributions/395/
[2] -> http://lore.kernel.org/r/[email protected]
[3] -> http://lore.kernel.org/r/[email protected]
[4] -> http://lore.kernel.org/r/[email protected]
[5] -> http://lore.kernel.org/r/[email protected]
[6] -> http://lore.kernel.org/r/[email protected]
[7] -> http://lore.kernel.org/r/[email protected]
[8] -> http://lore.kernel.org/r/[email protected]
[9] -> https://youtu.be/dLKKxrg5-rY
Kishon Vijay Abraham I (18):
Documentation: PCI: Add specification for the *PCI NTB* function
device
PCI: endpoint: Make *_get_first_free_bar() take into account 64 bit
BAR
PCI: endpoint: Add helper API to get the 'next' unreserved BAR
PCI: endpoint: Make *_free_bar() to return error codes on failure
PCI: endpoint: Remove unused pci_epf_match_device()
PCI: endpoint: Add support to associate secondary EPC with EPF
PCI: endpoint: Add support in configfs to associate two EPCs with EPF
PCI: endpoint: Add pci_epc_ops to map MSI irq
PCI: endpoint: Add pci_epf_ops for epf drivers to expose function
specific attrs
PCI: endpoint: Allow user to create sub-directory of 'EPF Device'
directory
PCI: cadence: Implement ->msi_map_irq() ops
PCI: cadence: Configure LM_EP_FUNC_CFG based on epc->function_num_map
PCI: endpoint: Add EP function driver to provide NTB functionality
PCI: Add TI J721E device to pci ids
NTB: Add support for EPF PCI-Express Non-Transparent Bridge
NTB: tool: Enable the NTB/PCIe link on the local or remote side of
bridge
Documentation: PCI: Add configfs binding documentation for pci-ntb
endpoint function
Documentation: PCI: Add userguide for PCI endpoint NTB function
.../PCI/endpoint/function/binding/pci-ntb.rst | 38 + Documentation/PCI/endpoint/index.rst | 3 + .../PCI/endpoint/pci-endpoint-cfs.rst | 10 + .../PCI/endpoint/pci-ntb-function.rst | 351 +++ Documentation/PCI/endpoint/pci-ntb-howto.rst | 160 ++ drivers/misc/pci_endpoint_test.c | 1 - drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 + drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++ drivers/ntb/test/ntb_tool.c | 1 + .../pci/controller/cadence/pcie-cadence-ep.c | 60 +- drivers/pci/endpoint/functions/Kconfig | 12 + drivers/pci/endpoint/functions/Makefile | 1 + drivers/pci/endpoint/functions/pci-epf-ntb.c | 2114 +++++++++++++++++ drivers/pci/endpoint/functions/pci-epf-test.c | 13 +- drivers/pci/endpoint/pci-ep-cfs.c | 176 +- drivers/pci/endpoint/pci-epc-core.c | 130 +- drivers/pci/endpoint/pci-epf-core.c | 105 +- include/linux/pci-epc.h | 39 +- include/linux/pci-epf.h | 28 +- include/linux/pci_ids.h | 1 + 23 files changed, 3934 insertions(+), 73 deletions(-) create mode 100644 Documentation/PCI/endpoint/function/binding/pci-ntb.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-function.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-howto.rst create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644 drivers/ntb/hw/epf/Makefile create mode 100644 drivers/ntb/hw/epf/ntb_hw_epf.c create mode 100644 drivers/pci/endpoint/functions/pci-epf-ntb.c
Signature · 1 line
2.17.1
PATCH v7 1/18 Documentation: PCI: Add specification for the *PCI NTB* function device
Add specification for the *PCI NTB* function device. The endpoint function driver and the host PCI driver should be created based on this specification.
--- Documentation/PCI/endpoint/index.rst | 1 + .../PCI/endpoint/pci-ntb-function.rst | 351 ++++++++++++++++++ 2 files changed, 352 insertions(+) create mode 100644 Documentation/PCI/endpoint/pci-ntb-function.rst
Documentation/PCI/endpoint/index.rst +1 -0
@@ -11,5 +11,6 @@ PCI Endpoint Framework pci-endpoint-cfs pci-test-function pci-test-howto + pci-ntb-function function/binding/pci-test
Documentation/PCI/endpoint/pci-ntb-function.rst new file +351 -0
351 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v7 2/18 PCI: endpoint: Make *_get_first_free_bar() take into account 64 bit BAR
In reply to Kishon Vijay Abraham Ipci_epc_get_first_free_bar() uses only "reserved_bar" member in epc_features to get the first unreserved BAR. However if the reserved BAR is also a 64-bit BAR, then the next BAR shouldn't be returned (since 64-bit BAR uses two BARs).
Make pci_epc_get_first_free_bar() take into account 64 bit BAR while returning the first free unreserved BAR.
--- drivers/pci/endpoint/pci-epc-core.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
drivers/pci/endpoint/pci-epc-core.c +10 -2
@@ -93,12 +93,20 @@ EXPORT_SYMBOL_GPL(pci_epc_get);unsigned int pci_epc_get_first_free_bar (const struct pci_epc_features *epc_features) { -int free_bar; +unsigned long free_bar;if (!epc_features)return 0 ; - free_bar =ffz (epc_features->reserved_bar); +/* Find if the reserved BAR is also a 64-bit BAR */ + free_bar = epc_features->reserved_bar & epc_features->bar_fixed_64bit; + +/* Set the adjacent bit if the reserved BAR is also a 64-bit BAR */ + free_bar <<=1 ; + free_bar |= epc_features->reserved_bar; + +/* Now find the free BAR */ + free_bar =ffz (free_bar);if (free_bar >5 )return 0 ;
Signature · 1 line
2.17.1
PATCH v7 3/18 PCI: endpoint: Add helper API to get the 'next' unreserved BAR
In reply to Kishon Vijay Abraham IAdd an API to get the next unreserved BAR starting from a given BAR number that can be used by the endpoint function.
--- drivers/pci/endpoint/pci-epc-core.c | 26 ++++++++++++++++++++++---- include/linux/pci-epc.h | 2 ++ 2 files changed, 24 insertions(+), 4 deletions(-)
drivers/pci/endpoint/pci-epc-core.c +22 -4
@@ -87,17 +87,36 @@ EXPORT_SYMBOL_GPL(pci_epc_get);* pci_epc_get_first_free_bar() - helper to get first unreserved BAR * @epc_features: pci_epc_features structure that holds the reserved bar bitmap * -* Invoke to get the first unreserved BAR that can be used for endpoint +* Invoke to get the first unreserved BAR that can be used by the endpoint * function. For any incorrect value in reserved_bar return '0'. */ unsigned int pci_epc_get_first_free_bar (const struct pci_epc_features *epc_features) +{ +return pci_epc_get_next_free_bar (epc_features,BAR_0 ); +} +EXPORT_SYMBOL_GPL (pci_epc_get_first_free_bar); + +/** +* pci_epc_get_next_free_bar() - helper to get unreserved BAR starting from @bar +* @epc_features: pci_epc_features structure that holds the reserved bar bitmap +* @bar: the starting BAR number from where unreserved BAR should be searched +* +* Invoke to get the next unreserved BAR starting from @bar that can be used +* for endpoint function. For any incorrect value in reserved_bar return '0'. +*/ +unsigned int pci_epc_get_next_free_bar (const struct pci_epc_features + *epc_features,enum pci_barno bar) {unsigned long free_bar;if (!epc_features)return 0 ; +/* If 'bar - 1' is a 64-bit BAR, move to the next BAR */ +if ((epc_features->bar_fixed_64bit <<1 ) &1 << bar) + bar++; +/* Find if the reserved BAR is also a 64-bit BAR */ free_bar = epc_features->reserved_bar & epc_features->bar_fixed_64bit;
@@ -105,14 +124,13 @@ unsigned int pci_epc_get_first_free_bar(const struct pci_epc_features free_bar <<=1 ; free_bar |= epc_features->reserved_bar; -/* Now find the free BAR */ - free_bar =ffz (free_bar); + free_bar =find_next_zero_bit (&free_bar,6 , bar);if (free_bar >5 )return 0 ;return free_bar; } -EXPORT_SYMBOL_GPL (pci_epc_get_first_free_bar); +EXPORT_SYMBOL_GPL (pci_epc_get_next_free_bar);/** * pci_epc_get_features() - get the features supported by EPC
include/linux/pci-epc.h +2 -0
@@ -203,6 +203,8 @@ const struct pci_epc_features *pci_epc_get_features(struct pci_epc *epc, u8 func_no);unsigned int pci_epc_get_first_free_bar (const struct pci_epc_features *epc_features); +unsigned int pci_epc_get_next_free_bar (const struct pci_epc_features + *epc_features,enum pci_barno bar);struct pci_epc *pci_epc_get (const char *epc_name);void pci_epc_put (struct pci_epc *epc);
Signature · 1 line
2.17.1
PATCH v7 4/18 PCI: endpoint: Make *_free_bar() to return error codes on failure
In reply to Kishon Vijay Abraham IModify pci_epc_get_next_free_bar() and pci_epc_get_first_free_bar() to return error values if there are no free BARs available.
--- drivers/pci/endpoint/functions/pci-epf-test.c | 2 ++ drivers/pci/endpoint/pci-epc-core.c | 12 ++++++------ include/linux/pci-epc.h | 8 ++++---- include/linux/pci-epf.h | 1 + 4 files changed, 13 insertions(+), 10 deletions(-)
drivers/pci/endpoint/functions/pci-epf-test.c +2 -0
@@ -834,6 +834,8 @@ static int pci_epf_test_bind(struct pci_epf *epf) linkup_notifier = epc_features->linkup_notifier; core_init_notifier = epc_features->core_init_notifier; test_reg_bar =pci_epc_get_first_free_bar (epc_features); +if (test_reg_bar <0 ) +return -EINVAL ;pci_epf_configure_bar (epf, epc_features); }
drivers/pci/endpoint/pci-epc-core.c +6 -6
@@ -90,8 +90,8 @@ EXPORT_SYMBOL_GPL(pci_epc_get);* Invoke to get the first unreserved BAR that can be used by the endpoint * function. For any incorrect value in reserved_bar return '0'. */ -unsigned int pci_epc_get_first_free_bar (const struct pci_epc_features - *epc_features) +enum pci_barno +pci_epc_get_first_free_bar (const struct pci_epc_features *epc_features) {return pci_epc_get_next_free_bar (epc_features,BAR_0 ); }
@@ -105,13 +105,13 @@ EXPORT_SYMBOL_GPL(pci_epc_get_first_free_bar);* Invoke to get the next unreserved BAR starting from @bar that can be used * for endpoint function. For any incorrect value in reserved_bar return '0'. */ -unsigned int pci_epc_get_next_free_bar (const struct pci_epc_features - *epc_features,enum pci_barno bar) +enum pci_barno pci_epc_get_next_free_bar (const struct pci_epc_features + *epc_features,enum pci_barno bar) {unsigned long free_bar;if (!epc_features) -return 0 ; +return BAR_0 ;/* If 'bar - 1' is a 64-bit BAR, move to the next BAR */ if ((epc_features->bar_fixed_64bit <<1 ) &1 << bar)
@@ -126,7 +126,7 @@ unsigned int pci_epc_get_next_free_bar(const struct pci_epc_features free_bar =find_next_zero_bit (&free_bar,6 , bar);if (free_bar >5 ) -return 0 ; +return NO_BAR ;return free_bar; }
include/linux/pci-epc.h +4 -4
@@ -201,10 +201,10 @@ int pci_epc_start(struct pci_epc *epc);void pci_epc_stop (struct pci_epc *epc);const struct pci_epc_features *pci_epc_get_features (struct pci_epc *epc,u8 func_no); -unsigned int pci_epc_get_first_free_bar (const struct pci_epc_features - *epc_features); -unsigned int pci_epc_get_next_free_bar (const struct pci_epc_features - *epc_features,enum pci_barno bar); +enum pci_barno +pci_epc_get_first_free_bar (const struct pci_epc_features *epc_features); +enum pci_barno pci_epc_get_next_free_bar (const struct pci_epc_features + *epc_features,enum pci_barno bar);struct pci_epc *pci_epc_get (const char *epc_name);void pci_epc_put (struct pci_epc *epc);
include/linux/pci-epf.h +1 -0
@@ -21,6 +21,7 @@ enum pci_notify_event { };enum pci_barno { +NO_BAR =-1 ,BAR_0 ,BAR_1 ,BAR_2 ,
Signature · 1 line
2.17.1
PATCH v7 5/18 PCI: endpoint: Remove unused pci_epf_match_device()
In reply to Kishon Vijay Abraham IRemove unused pci_epf_match_device() function added in pci-epf-core.c
--- drivers/pci/endpoint/pci-epf-core.c | 16 ---------------- include/linux/pci-epf.h | 2 -- 2 files changed, 18 deletions(-)
drivers/pci/endpoint/pci-epf-core.c +0 -16
@@ -282,22 +282,6 @@ struct pci_epf *pci_epf_create(const char *name) }EXPORT_SYMBOL_GPL (pci_epf_create); -const struct pci_epf_device_id * -pci_epf_match_device (const struct pci_epf_device_id *id,struct pci_epf *epf) -{ -if (!id || !epf) -return NULL ; - -while (*id->name) { -if (strcmp (epf->name, id->name) ==0 ) -return id; - id++; - } - -return NULL ; -} -EXPORT_SYMBOL_GPL (pci_epf_match_device); -static void pci_epf_dev_release (struct device *dev) {struct pci_epf *epf =to_pci_epf (dev);
include/linux/pci-epf.h +0 -2
@@ -165,8 +165,6 @@ static inline void *epf_get_drvdata(struct pci_epf *epf)return dev_get_drvdata (&epf->dev); } -const struct pci_epf_device_id * -pci_epf_match_device (const struct pci_epf_device_id *id,struct pci_epf *epf);struct pci_epf *pci_epf_create (const char *name);void pci_epf_destroy (struct pci_epf *epf); int __pci_epf_register_driver(struct pci_epf_driver *driver,
Signature · 1 line
2.17.1
PATCH v7 6/18 PCI: endpoint: Add support to associate secondary EPC with EPF
In reply to Kishon Vijay Abraham IIn the case of standard endpoint functions, only one endpoint controller (EPC) will be associated with an endpoint function (EPF). However for providing NTB (non transparent bridge) functionality, two EPCs should be associated with a single EPF. Add support to associate secondary EPC with EPF. This is in preparation for adding NTB endpoint function driver.
--- drivers/pci/endpoint/functions/pci-epf-test.c | 11 ++-- drivers/pci/endpoint/pci-ep-cfs.c | 6 +- drivers/pci/endpoint/pci-epc-core.c | 47 +++++++++++---- drivers/pci/endpoint/pci-epf-core.c | 57 +++++++++++++------ include/linux/pci-epc.h | 25 +++++++- include/linux/pci-epf.h | 17 +++++- 6 files changed, 125 insertions(+), 38 deletions(-)
drivers/pci/endpoint/functions/pci-epf-test.c +7 -4
@@ -619,7 +619,8 @@ static void pci_epf_test_unbind(struct pci_epf *epf)if (epf_test->reg[bar]) {pci_epc_clear_bar (epc, epf->func_no, epf_bar); -pci_epf_free_space (epf, epf_test->reg[bar], bar); +pci_epf_free_space (epf, epf_test->reg[bar], bar, +PRIMARY_INTERFACE ); } } }
@@ -651,7 +652,8 @@ static int pci_epf_test_set_bar(struct pci_epf *epf) ret =pci_epc_set_bar (epc, epf->func_no, epf_bar);if (ret) { -pci_epf_free_space (epf, epf_test->reg[bar], bar); +pci_epf_free_space (epf, epf_test->reg[bar], bar, +PRIMARY_INTERFACE );dev_err (dev,"Failed to set BAR%d\n" , bar);if (bar == test_reg_bar)return ret;
@@ -771,7 +773,7 @@ static int pci_epf_test_alloc_space(struct pci_epf *epf) } base =pci_epf_alloc_space (epf, test_reg_size, test_reg_bar, - epc_features->align); + epc_features->align,PRIMARY_INTERFACE );if (!base) {dev_err (dev,"Failed to allocated register space\n" );return -ENOMEM ;
@@ -789,7 +791,8 @@ static int pci_epf_test_alloc_space(struct pci_epf *epf)continue ; base =pci_epf_alloc_space (epf, bar_size[bar], bar, - epc_features->align); + epc_features->align, +PRIMARY_INTERFACE );if (!base)dev_err (dev,"Failed to allocate space for BAR%d\n" , bar);
drivers/pci/endpoint/pci-ep-cfs.c +3 -3
@@ -94,13 +94,13 @@ static int pci_epc_epf_link(struct config_item *epc_item,struct pci_epc *epc = epc_group->epc;struct pci_epf *epf = epf_group->epf; - ret =pci_epc_add_epf (epc, epf); + ret =pci_epc_add_epf (epc, epf,PRIMARY_INTERFACE );if (ret)return ret; ret =pci_epf_bind (epf);if (ret) { -pci_epc_remove_epf (epc, epf); +pci_epc_remove_epf (epc, epf,PRIMARY_INTERFACE );return ret; }
@@ -120,7 +120,7 @@ static void pci_epc_epf_unlink(struct config_item *epc_item, epc = epc_group->epc; epf = epf_group->epf;pci_epf_unbind (epf); -pci_epc_remove_epf (epc, epf); +pci_epc_remove_epf (epc, epf,PRIMARY_INTERFACE ); }static struct configfs_item_operations pci_epc_item_ops = {
drivers/pci/endpoint/pci-epc-core.c +36 -11
82 lines not on this page · raw message
drivers/pci/endpoint/pci-epf-core.c +41 -16
97 lines not on this page · raw message
include/linux/pci-epc.h +23 -2
@@ -13,6 +13,12 @@struct pci_epc ; +enum pci_epc_interface_type { +UNKNOWN_INTERFACE =-1 , +PRIMARY_INTERFACE , +SECONDARY_INTERFACE , +}; +enum pci_epc_irq_type {PCI_EPC_IRQ_UNKNOWN ,PCI_EPC_IRQ_LEGACY ,
@@ -20,6 +26,19 @@ enum pci_epc_irq_type { PCI_EPC_IRQ_MSIX, }; +static inline const char * +pci_epc_interface_string (enum pci_epc_interface_type type) +{ +switch (type) { +case PRIMARY_INTERFACE : +return "primary" ; +case SECONDARY_INTERFACE : +return "secondary" ; +default : +return "UNKNOWN interface" ; + } +} +/** * struct pci_epc_ops - set of function pointers for performing EPC operations * @write_header: ops to populate configuration space header
@@ -175,10 +194,12 @@ __pci_epc_create(struct device *dev, const struct pci_epc_ops *ops, struct module *owner);void devm_pci_epc_destroy (struct device *dev,struct pci_epc *epc);void pci_epc_destroy (struct pci_epc *epc); -int pci_epc_add_epf (struct pci_epc *epc,struct pci_epf *epf); +int pci_epc_add_epf (struct pci_epc *epc,struct pci_epf *epf, +enum pci_epc_interface_type type);void pci_epc_linkup (struct pci_epc *epc);void pci_epc_init_notify (struct pci_epc *epc); -void pci_epc_remove_epf (struct pci_epc *epc,struct pci_epf *epf); +void pci_epc_remove_epf (struct pci_epc *epc,struct pci_epf *epf, +enum pci_epc_interface_type type);int pci_epc_write_header (struct pci_epc *epc,u8 func_no,struct pci_epf_header *hdr); int pci_epc_set_bar(struct pci_epc *epc, u8 func_no,
include/linux/pci-epf.h +15 -2
@@ -14,6 +14,7 @@#include <linux/pci.h> struct pci_epf ; +enum pci_epc_interface_type ;enum pci_notify_event {CORE_INIT ,
@@ -119,6 +120,11 @@ struct pci_epf_bar {* @list: to add pci_epf as a list of PCI endpoint functions to pci_epc * @nb: notifier block to notify EPF of any EPC events (like linkup) * @lock: mutex to protect pci_epf_ops +* @sec_epc: the secondary EPC device to which this EPF device is bound +* @sec_epc_list: to add pci_epf as list of PCI endpoint functions to secondary +* EPC device +* @sec_epc_bar: represents the BAR of EPF device associated with secondary EPC +* @sec_epc_func_no: unique (physical) function number within the secondary EPC */ struct pci_epf {struct device dev;
@@ -135,6 +141,12 @@ struct pci_epf {struct notifier_block nb;/* mutex to protect against concurrent access of pci_epf_ops */ struct mutex lock; + +/* Below members are to attach secondary EPC to an endpoint function */ +struct pci_epc *sec_epc; +struct list_head sec_epc_list; +struct pci_epf_bar sec_epc_bar[6 ]; +u8 sec_epc_func_no; };/**
@@ -171,8 +183,9 @@ int __pci_epf_register_driver(struct pci_epf_driver *driver, struct module *owner);void pci_epf_unregister_driver (struct pci_epf_driver *driver);void *pci_epf_alloc_space (struct pci_epf *epf,size_t size,enum pci_barno bar, -size_t align); -void pci_epf_free_space (struct pci_epf *epf,void *addr,enum pci_barno bar); +size_t align,enum pci_epc_interface_type type); +void pci_epf_free_space (struct pci_epf *epf,void *addr,enum pci_barno bar, +enum pci_epc_interface_type type);int pci_epf_bind (struct pci_epf *epf);void pci_epf_unbind (struct pci_epf *epf);#endif /* __LINUX_PCI_EPF_H */
Signature · 1 line
2.17.1
PATCH v7 7/18 PCI: endpoint: Add support in configfs to associate two EPCs with EPF
In reply to Kishon Vijay Abraham INow that PCI endpoint core supports to add secondary endpoint controller (EPC) with endpoint function (EPF), Add support in configfs to associate two EPCs with EPF. This creates "primary" and "secondary" directory inside the directory created by users for EPF device. Users have to add a symlink of endpoint controller (pci_ep/controllers/) to "primary" or "secondary" directory to bind EPF to primary and secondary EPF interfaces respectively. Existing method of linking directory representing EPF device to directory representing EPC device to associate a single EPC device with a EPF device will continue to work.
--- .../PCI/endpoint/pci-endpoint-cfs.rst | 10 ++ drivers/pci/endpoint/pci-ep-cfs.c | 147 ++++++++++++++++++ 2 files changed, 157 insertions(+)
Documentation/PCI/endpoint/pci-endpoint-cfs.rst +10 -0
@@ -68,6 +68,16 @@ created) ... subsys_vendor_id ... subsys_id ... interrupt_pin + ... primary/ + ... <Symlink EPC Device1>/ + ... secondary/ + ... <Symlink EPC Device2>/ + +If an EPF device has to be associated with 2 EPCs (like in the case of +Non-transparent bridge), symlink of endpoint controller connected to primary +interface should be added in 'primary' directory and symlink of endpoint +controller connected to secondary interface should be added in 'secondary' +directory. EPC Device ==========
drivers/pci/endpoint/pci-ep-cfs.c +147 -0
171 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v7 8/18 PCI: endpoint: Add pci_epc_ops to map MSI irq
In reply to Kishon Vijay Abraham IAdd pci_epc_ops to map physical address to MSI address and return MSI data. The physical address is an address in the outbound region. This is required to implement doorbell functionality of NTB (non transparent bridge) wherein EPC on either side of the interface (primary and secondary) can directly write to the physical address (in outbound region) of the other interface to ring doorbell using MSI.
--- drivers/pci/endpoint/pci-epc-core.c | 41 +++++++++++++++++++++++++++++ include/linux/pci-epc.h | 8 ++++++ 2 files changed, 49 insertions(+)
drivers/pci/endpoint/pci-epc-core.c +41 -0
@@ -230,6 +230,47 @@ int pci_epc_raise_irq(struct pci_epc *epc, u8 func_no, }EXPORT_SYMBOL_GPL (pci_epc_raise_irq); +/** +* pci_epc_map_msi_irq() - Map physical address to MSI address and return +* MSI data +* @epc: the EPC device which has the MSI capability +* @func_no: the physical endpoint function number in the EPC device +* @phys_addr: the physical address of the outbound region +* @interrupt_num: the MSI interrupt number +* @entry_size: Size of Outbound address region for each interrupt +* @msi_data: the data that should be written in order to raise MSI interrupt +* with interrupt number as 'interrupt num' +* @msi_addr_offset: Offset of MSI address from the aligned outbound address +* to which the MSI address is mapped +* +* Invoke to map physical address to MSI address and return MSI data. The +* physical address should be an address in the outbound region. This is +* required to implement doorbell functionality of NTB wherein EPC on either +* side of the interface (primary and secondary) can directly write to the +* physical address (in outbound region) of the other interface to ring +* doorbell. +*/ +int pci_epc_map_msi_irq (struct pci_epc *epc,u8 func_no,phys_addr_t phys_addr, +u8 interrupt_num,u32 entry_size,u32 *msi_data, +u32 *msi_addr_offset) +{ +int ret; + +if (IS_ERR_OR_NULL (epc)) +return -EINVAL ; + +if (!epc->ops->map_msi_irq) +return -EINVAL ; + +mutex_lock (&epc->lock); + ret = epc->ops->map_msi_irq (epc, func_no, phys_addr, interrupt_num, + entry_size, msi_data, msi_addr_offset); +mutex_unlock (&epc->lock); + +return ret; +} +EXPORT_SYMBOL_GPL (pci_epc_map_msi_irq); +/** * pci_epc_get_msi() - get the number of MSI interrupt numbers allocated * @epc: the EPC device to which MSI interrupts was requested
include/linux/pci-epc.h +8 -0
@@ -55,6 +55,7 @@ pci_epc_interface_string(enum pci_epc_interface_type type)* @get_msix: ops to get the number of MSI-X interrupts allocated by the RC * from the MSI-X capability register * @raise_irq: ops to raise a legacy, MSI or MSI-X interrupt +* @map_msi_irq: ops to map physical address to MSI address and return MSI data * @start: ops to start the PCI link * @stop: ops to stop the PCI link * @owner: the module owner containing the ops
@@ -77,6 +78,10 @@ struct pci_epc_ops {int (*get_msix)(struct pci_epc *epc,u8 func_no);int (*raise_irq)(struct pci_epc *epc,u8 func_no,enum pci_epc_irq_type type,u16 interrupt_num); +int (*map_msi_irq)(struct pci_epc *epc,u8 func_no, +phys_addr_t phys_addr,u8 interrupt_num, +u32 entry_size,u32 *msi_data, +u32 *msi_addr_offset);int (*start)(struct pci_epc *epc);void (*stop)(struct pci_epc *epc); const struct pci_epc_features* (*get_features)(struct pci_epc *epc,
@@ -216,6 +221,9 @@ int pci_epc_get_msi(struct pci_epc *epc, u8 func_no);int pci_epc_set_msix (struct pci_epc *epc,u8 func_no,u16 interrupts,enum pci_barno ,u32 offset);int pci_epc_get_msix (struct pci_epc *epc,u8 func_no); +int pci_epc_map_msi_irq (struct pci_epc *epc,u8 func_no, +phys_addr_t phys_addr,u8 interrupt_num, +u32 entry_size,u32 *msi_data,u32 *msi_addr_offset);int pci_epc_raise_irq (struct pci_epc *epc,u8 func_no,enum pci_epc_irq_type type,u16 interrupt_num);int pci_epc_start (struct pci_epc *epc);
Signature · 1 line
2.17.1
PATCH v7 9/18 PCI: endpoint: Add pci_epf_ops for epf drivers to expose function specific attrs
In reply to Kishon Vijay Abraham IIn addition to the attributes that are generic across function drivers documented in Documentation/PCI/endpoint/pci-endpoint-cfs.rst, there could be function specific attributes that has to be exposed by the function driver to be configured by the user. Add ->add_cfs() in pci_epf_ops to be populated by the function driver if it has to expose any function specific attributes and pci_epf_type_add_cfs() to be invoked by pci-ep-cfs.c when sub-directory to main function directory is created.
--- drivers/pci/endpoint/pci-epf-core.c | 32 +++++++++++++++++++++++++++++ include/linux/pci-epf.h | 5 +++++ 2 files changed, 37 insertions(+)
drivers/pci/endpoint/pci-epf-core.c +32 -0
@@ -20,6 +20,38 @@ static DEFINE_MUTEX(pci_epf_mutex);static struct bus_type pci_epf_bus_type;static const struct device_type pci_epf_type; +/** +* pci_epf_type_add_cfs() - Help function drivers to expose function specific +* attributes in configfs +* @epf: the EPF device that has to be configured using configfs +* @group: the parent configfs group (corresponding to entries in +* pci_epf_device_id) +* +* Invoke to expose function specific attributes in configfs. If the function +* driver does not have anything to expose (attributes configured by user), +* return NULL. +*/ +struct config_group *pci_epf_type_add_cfs (struct pci_epf *epf, +struct config_group *group) +{ +struct config_group *epf_type_group; + +if (!epf->driver) { +dev_err (&epf->dev,"epf device not bound to driver\n" ); +return NULL ; + } + +if (!epf->driver->ops->add_cfs) +return NULL ; + +mutex_lock (&epf->lock); + epf_type_group = epf->driver->ops->add_cfs (epf, group); +mutex_unlock (&epf->lock); + +return epf_type_group; +} +EXPORT_SYMBOL_GPL (pci_epf_type_add_cfs); +/** * pci_epf_unbind() - Notify the function driver that the binding between the * EPF device and EPC device has been lost
include/linux/pci-epf.h +5 -0
@@ -62,10 +62,13 @@ struct pci_epf_header {* @bind: ops to perform when a EPC device has been bound to EPF device * @unbind: ops to perform when a binding has been lost between a EPC device * and EPF device +* @add_cfs: ops to initialize function specific configfs attributes */ struct pci_epf_ops {int (*bind)(struct pci_epf *epf);void (*unbind)(struct pci_epf *epf); +struct config_group *(*add_cfs)(struct pci_epf *epf, +struct config_group *group); };/**
@@ -188,4 +191,6 @@ void pci_epf_free_space(struct pci_epf *epf, void *addr, enum pci_barno bar, enum pci_epc_interface_type type);int pci_epf_bind (struct pci_epf *epf);void pci_epf_unbind (struct pci_epf *epf); +struct config_group *pci_epf_type_add_cfs (struct pci_epf *epf, +struct config_group *group);#endif /* __LINUX_PCI_EPF_H */
Signature · 1 line
2.17.1
PATCH v7 10/18 PCI: endpoint: Allow user to create sub-directory of 'EPF Device' directory
In reply to Kishon Vijay Abraham IDocumentation/PCI/endpoint/pci-endpoint-cfs.rst explains how a user has to create a directory in-order to create a 'EPF Device' that can be configured/probed by 'EPF Driver'.
Allow user to create a sub-directory of 'EPF Device' directory for any function specific attributes that has to be exposed to the user.
--- drivers/pci/endpoint/pci-ep-cfs.c | 23 +++++++++++++++++++++++ include/linux/pci-epf.h | 3 +++ 2 files changed, 26 insertions(+)
drivers/pci/endpoint/pci-ep-cfs.c +23 -0
@@ -490,7 +490,29 @@ static struct configfs_item_operations pci_epf_ops = { .release = pci_epf_release, }; +static struct config_group *pci_epf_type_make (struct config_group *group, +const char *name) +{ +struct pci_epf_group *epf_group =to_pci_epf_group (&group->cg_item); +struct config_group *epf_type_group; + + epf_type_group =pci_epf_type_add_cfs (epf_group->epf, group); +return epf_type_group; +} + +static void pci_epf_type_drop (struct config_group *group, +struct config_item *item) +{ +config_item_put (item); +} + +static struct configfs_group_operations pci_epf_type_group_ops = { + .make_group = &pci_epf_type_make, + .drop_item = &pci_epf_type_drop, +}; +static const struct config_item_type pci_epf_type = { + .ct_group_ops = &pci_epf_type_group_ops, .ct_item_ops = &pci_epf_ops, .ct_attrs = pci_epf_attrs, .ct_owner =THIS_MODULE ,
@@ -553,6 +575,7 @@ static struct config_group *pci_epf_make(struct config_group *group, goto free_name; } + epf->group = &epf_group->group; epf_group->epf = epf;kfree (epf_name);
include/linux/pci-epf.h +3 -0
@@ -9,6 +9,7 @@#ifndef __LINUX_PCI_EPF_H#define __LINUX_PCI_EPF_H +#include <linux/configfs.h> #include <linux/device.h> #include <linux/mod_devicetable.h> #include <linux/pci.h>
@@ -128,6 +129,7 @@ struct pci_epf_bar {* EPC device * @sec_epc_bar: represents the BAR of EPF device associated with secondary EPC * @sec_epc_func_no: unique (physical) function number within the secondary EPC +* @group: configfs group associated with the EPF device */ struct pci_epf {struct device dev;
@@ -150,6 +152,7 @@ struct pci_epf {struct list_head sec_epc_list;struct pci_epf_bar sec_epc_bar[6 ];u8 sec_epc_func_no; +struct config_group *group; };/**
Signature · 1 line
2.17.1
PATCH v7 11/18 PCI: cadence: Implement ->msi_map_irq() ops
In reply to Kishon Vijay Abraham IImplement ->msi_map_irq() ops in order to map physical address to MSI address and return MSI data.
--- .../pci/controller/cadence/pcie-cadence-ep.c | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+)
drivers/pci/controller/cadence/pcie-cadence-ep.c +53 -0
@@ -383,6 +383,57 @@ static int cdns_pcie_ep_send_msi_irq(struct cdns_pcie_ep *ep, u8 fn,return 0 ; } +static int cdns_pcie_ep_map_msi_irq (struct pci_epc *epc,u8 fn, +phys_addr_t addr,u8 interrupt_num, +u32 entry_size,u32 *msi_data, +u32 *msi_addr_offset) +{ +struct cdns_pcie_ep *ep =epc_get_drvdata (epc); +u32 cap =CDNS_PCIE_EP_FUNC_MSI_CAP_OFFSET ; +struct cdns_pcie *pcie = &ep->pcie; +u64 pci_addr, pci_addr_mask =0xff ; +u16 flags, mme, data, data_mask; +u8 msi_count; +int ret; +int i; + +/* Check whether the MSI feature has been enabled by the PCI host. */ + flags =cdns_pcie_ep_fn_readw (pcie, fn, cap +PCI_MSI_FLAGS ); +if (!(flags &PCI_MSI_FLAGS_ENABLE )) +return -EINVAL ; + +/* Get the number of enabled MSIs */ + mme = (flags &PCI_MSI_FLAGS_QSIZE ) >>4 ; + msi_count =1 << mme; +if (!interrupt_num || interrupt_num > msi_count) +return -EINVAL ; + +/* Compute the data value to be written. */ + data_mask = msi_count -1 ; + data =cdns_pcie_ep_fn_readw (pcie, fn, cap +PCI_MSI_DATA_64 ); + data = data & ~data_mask; + +/* Get the PCI address where to write the data into. */ + pci_addr =cdns_pcie_ep_fn_readl (pcie, fn, cap +PCI_MSI_ADDRESS_HI ); + pci_addr <<=32 ; + pci_addr |=cdns_pcie_ep_fn_readl (pcie, fn, cap +PCI_MSI_ADDRESS_LO ); + pci_addr &=GENMASK_ULL (63 ,2 ); + +for (i =0 ; i < interrupt_num; i++) { + ret =cdns_pcie_ep_map_addr (epc, fn, addr, + pci_addr & ~pci_addr_mask, + entry_size); +if (ret) +return ret; + addr = addr + entry_size; + } + + *msi_data = data; + *msi_addr_offset = pci_addr & pci_addr_mask; + +return 0 ; +} +static int cdns_pcie_ep_send_msix_irq (struct cdns_pcie_ep *ep,u8 fn,u16 interrupt_num) {
@@ -482,6 +533,7 @@ static const struct pci_epc_features cdns_pcie_epc_features = { .linkup_notifier = false, .msi_capable = true, .msix_capable = true, + .align = 256, }; static const struct pci_epc_features*
@@ -501,6 +553,7 @@ static const struct pci_epc_ops cdns_pcie_epc_ops = { .set_msix = cdns_pcie_ep_set_msix, .get_msix = cdns_pcie_ep_get_msix, .raise_irq = cdns_pcie_ep_raise_irq, + .map_msi_irq = cdns_pcie_ep_map_msi_irq, .start = cdns_pcie_ep_start, .get_features = cdns_pcie_ep_get_features, };
Signature · 1 line
2.17.1
PATCH v7 12/18 PCI: cadence: Configure LM_EP_FUNC_CFG based on epc->function_num_map
In reply to Kishon Vijay Abraham IThe number of functions supported by the endpoint controller is configured in LM_EP_FUNC_CFG based on func_no member of struct pci_epf. Now that an endpoint function can be associated with two endpoint controllers (primary and secondary), just using func_no will not suffice as that will take into account only if the endpoint controller is associated with the primary interface of endpoint function. Instead use epc->function_num_map which will already have the configured functions information (irrespective of whether the endpoint controller is associated with primary or secondary interface).
--- drivers/pci/controller/cadence/pcie-cadence-ep.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
drivers/pci/controller/cadence/pcie-cadence-ep.c +1 -6
@@ -507,18 +507,13 @@ static int cdns_pcie_ep_start(struct pci_epc *epc)struct cdns_pcie_ep *ep =epc_get_drvdata (epc);struct cdns_pcie *pcie = &ep->pcie;struct device *dev = pcie->dev; -struct pci_epf *epf; -u32 cfg;int ret;/* * BIT(0) is hardwired to 1, hence function 0 is always enabled * and can't be disabled anyway. */ - cfg =BIT (0 ); -list_for_each_entry (epf, &epc->pci_epf, list) - cfg |=BIT (epf->func_no); -cdns_pcie_writel (pcie,CDNS_PCIE_LM_EP_FUNC_CFG , cfg); +cdns_pcie_writel (pcie,CDNS_PCIE_LM_EP_FUNC_CFG , epc->function_num_map); ret =cdns_pcie_start_link (pcie);if (ret) {
Signature · 1 line
2.17.1
PATCH v7 13/18 PCI: endpoint: Add EP function driver to provide NTB functionality
In reply to Kishon Vijay Abraham IAdd a new endpoint function driver to provide NTB functionality using multiple PCIe endpoint instances.
--- drivers/pci/endpoint/functions/Kconfig | 12 + drivers/pci/endpoint/functions/Makefile | 1 + drivers/pci/endpoint/functions/pci-epf-ntb.c | 2114 ++++++++++++++++++ 3 files changed, 2127 insertions(+) create mode 100644 drivers/pci/endpoint/functions/pci-epf-ntb.c
drivers/pci/endpoint/functions/Kconfig +12 -0
@@ -12,3 +12,15 @@ config PCI_EPF_TEST for PCI Endpoint. If in doubt, say "N" to disable Endpoint test driver. + +config PCI_EPF_NTB + tristate "PCI Endpoint NTB driver" + depends on PCI_ENDPOINT + help + Select this configuration option to enable the NTB driver + for PCI Endpoint. NTB driver implements NTB controller + functionality using multiple PCIe endpoint instances. It + can support NTB endpoint function devices created using + device tree. + + If in doubt, say "N" to disable Endpoint NTB driver.
drivers/pci/endpoint/functions/Makefile +1 -0
@@ -4,3 +4,4 @@ # obj-$(CONFIG_PCI_EPF_TEST) += pci-epf-test.o +obj-$(CONFIG_PCI_EPF_NTB) += pci-epf-ntb.o
drivers/pci/endpoint/functions/pci-epf-ntb.c new file +2114 -0
2,114 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v7 14/18 PCI: Add TI J721E device to pci ids
In reply to Kishon Vijay Abraham IAdd TI J721E device to the pci id database. Since this device has a configurable PCIe endpoint, it could be used with different drivers.
--- drivers/misc/pci_endpoint_test.c | 1 - include/linux/pci_ids.h | 1 + 2 files changed, 1 insertion(+), 1 deletion(-)
drivers/misc/pci_endpoint_test.c +0 -1
@@ -68,7 +68,6 @@#define PCI_ENDPOINT_TEST_FLAGS 0x2c#define FLAG_USE_DMA BIT(0) -#define PCI_DEVICE_ID_TI_J721E 0xb00d#define PCI_DEVICE_ID_TI_AM654 0xb00c#define is_am654_pci_dev (pdev) \
include/linux/pci_ids.h +1 -0
@@ -880,6 +880,7 @@#define PCI_DEVICE_ID_TI_X620 0xac8d#define PCI_DEVICE_ID_TI_X420 0xac8e#define PCI_DEVICE_ID_TI_XX20_FM 0xac8f +#define PCI_DEVICE_ID_TI_J721E 0xb00d#define PCI_DEVICE_ID_TI_DRA74x 0xb500#define PCI_DEVICE_ID_TI_DRA72x 0xb501
Signature · 1 line
2.17.1
PATCH v7 15/18 NTB: Add support for EPF PCI-Express Non-Transparent Bridge
In reply to Kishon Vijay Abraham IAdd support for EPF PCI-Express Non-Transparent Bridge (NTB) device. This driver is platform independent and could be used by any platform which have multiple PCIe endpoint instances configured using the pci-epf-ntb driver. The driver connnects to the standard NTB sub-system interface. The EPF NTB device has configurable number of memory windows (Max 4), configurable number of doorbell (Max 32), and configurable number of scratch-pad registers.
--- drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 + drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++++++++++++++++++++++++++++ 5 files changed, 764 insertions(+) create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644 drivers/ntb/hw/epf/Makefile create mode 100644 drivers/ntb/hw/epf/ntb_hw_epf.c
drivers/ntb/hw/Kconfig +1 -0
@@ -2,4 +2,5 @@ source "drivers/ntb/hw/amd/Kconfig" source "drivers/ntb/hw/idt/Kconfig" source "drivers/ntb/hw/intel/Kconfig" +source "drivers/ntb/hw/epf/Kconfig" source "drivers/ntb/hw/mscc/Kconfig"
drivers/ntb/hw/Makefile +1 -0
@@ -2,4 +2,5 @@ obj-$(CONFIG_NTB_AMD) += amd/ obj-$(CONFIG_NTB_IDT) += idt/ obj-$(CONFIG_NTB_INTEL) += intel/ +obj-$(CONFIG_NTB_EPF) += epf/ obj-$(CONFIG_NTB_SWITCHTEC) += mscc/
drivers/ntb/hw/epf/Kconfig new file +6 -0
@@ -0,0 +1,6 @@ +config NTB_EPF + tristate "Generic EPF Non-Transparent Bridge support" + depends on m + help + This driver supports EPF NTB on configurable endpoint. + If unsure, say N.
drivers/ntb/hw/epf/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-$(CONFIG_NTB_EPF) += ntb_hw_epf.o
drivers/ntb/hw/epf/ntb_hw_epf.c new file +755 -0
755 lines not on this page · raw message
Signature · 1 line
2.17.1
RE: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Hi Kishon,
Quoted 69 lines
Subject: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- Transparent Bridge
From: Kishon Vijay Abraham I <[email protected]>
Add support for EPF PCI-Express Non-Transparent Bridge (NTB) device. This driver is platform independent and could be used by any platform which have multiple PCIe endpoint instances configured using the pci-epf-ntb driver. The driver connnects to the standard NTB sub-system interface. The EPF NTB device has configurable number of memory windows (Max 4), configurable number of doorbell (Max 32), and configurable number of scratch-pad registers.
Signed-off-by Kishon Vijay Abraham I <[email protected]>--- drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 + drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++++++++++++++++++++++++++++ 5 files changed, 764 insertions(+) create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644
drivers/ntb/hw/epf/Makefilecreate mode 100644drivers/ntb/hw/epf/ntb_hw_epf.cdrivers/ntb/hw/Kconfig index renamed from drivers/ntb/hw/Kconfig +0
-0
e77c587060ff..c325be526b80100644drivers/ntb/hw/Kconfig +1
-0@@ -2,4 +2,5 @@ source "drivers/ntb/hw/amd/Kconfig" source "drivers/ntb/hw/idt/Kconfig" source "drivers/ntb/hw/intel/Kconfig" +source "drivers/ntb/hw/epf/Kconfig" source "drivers/ntb/hw/mscc/Kconfig"drivers/ntb/hw/Makefile index renamed from drivers/ntb/hw/Makefile +0
-0
4714d6238845..223ca592b5f9100644drivers/ntb/hw/Makefile +1
-0@@ -2,4 +2,5 @@ obj-$(CONFIG_NTB_AMD) += amd/ obj-$(CONFIG_NTB_IDT) += idt/ obj-$(CONFIG_NTB_INTEL) += intel/ +obj-$(CONFIG_NTB_EPF) += epf/ obj-$(CONFIG_NTB_SWITCHTEC) += mscc/drivers/ntb/hw/epf/Kconfig new renamed from drivers/ntb/hw/epf/Kconfig +0
-0file mode 100644 index 000000000000..6197d1aab344
drivers/ntb/hw/epf/Kconfig new file +6
-0@@ -0,0 +1,6 @@ +config NTB_EPF + tristate "Generic EPF Non-Transparent Bridge support" + depends on m + help + This driver supports EPF NTB on configurable endpoint. + If unsure, say N.drivers/ntb/hw/epf/Makefile new renamed from drivers/ntb/hw/epf/Makefile +0
-0file mode 100644 index 000000000000..2f560a422bc6
drivers/ntb/hw/epf/Makefile new file +1
-0@@ -0,0 +1 @@ +obj-$(CONFIG_NTB_EPF) += ntb_hw_epf.o(quoted diff) +0
-0
b/drivers/ntb/hw/epf/ntb_hw_epf.cnew file mode 100644 index 000000000000..0a144987851adrivers/ntb/hw/epf/ntb_hw_epf.c new file +0
-0@@ -0,0 +1,755 @@
......
Quoted 26 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +25
-0+static int ntb_epf_init_pci (struct ntb_epf_dev *ndev, +struct pci_dev *pdev) +{ +struct device *dev = ndev->dev; +int ret; + +pci_set_drvdata (pdev, ndev); + + ret =pci_enable_device (pdev); +if (ret) { +dev_err (dev,"Cannot enable PCI device\n" ); + goto err_pci_enable; + } + + ret =pci_request_regions (pdev,"ntb" ); +if (ret) { +dev_err (dev,"Cannot obtain PCI resources\n" ); + goto err_pci_regions; + } + +pci_set_master (pdev); + + ret =dma_set_mask_and_coherent (dev,DMA_BIT_MASK (64 )); +if (ret) { + ret = dma_set_mask_and_coherent(dev,
DMA_BIT_MASK(32));
drivers/ntb/hw/epf/ntb_hw_epf.c line 596
Quoted 8 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +8
-0+if (ret) { +dev_err (dev,"Cannot set DMA mask\n" ); + goto err_dma_mask; + } +dev_warn (&pdev->dev,"Cannot DMA highmem\n" ); + } + + ndev->ctrl_reg =pci_iomap (pdev,0 ,0 );
The second parameter of pci_iomap should be ndev->ctrl_reg_bar instead of the hardcode value 0, right?
drivers/ntb/hw/epf/ntb_hw_epf.c line 602
Quoted 6 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +6
-0+if (!ndev->ctrl_reg) { + ret = -EIO ; + goto err_dma_mask; + } + + ndev->peer_spad_reg =pci_iomap (pdev,1 ,0 );
pci_iomap(pdev, ndev->peer_spad_reg_bar, 0);
drivers/ntb/hw/epf/ntb_hw_epf.c line 608
Quoted 6 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +6
-0+if (!ndev->peer_spad_reg) { + ret = -EIO ; + goto err_dma_mask; + } + + ndev->db_reg =pci_iomap (pdev,2 ,0 );
pci_iomap(pdev, ndev->db_reg_bar, 0);
Best Regards
Sherry
Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Hi Sherry,
Quoted 118 lines · Sherry Sun
Hi Kishon,
Quoted 69 lines
Subject: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- Transparent Bridge
From: Kishon Vijay Abraham I <[email protected]>
Add support for EPF PCI-Express Non-Transparent Bridge (NTB) device. This driver is platform independent and could be used by any platform which have multiple PCIe endpoint instances configured using the pci-epf-ntb driver. The driver connnects to the standard NTB sub-system interface. The EPF NTB device has configurable number of memory windows (Max 4), configurable number of doorbell (Max 32), and configurable number of scratch-pad registers.
Signed-off-by Kishon Vijay Abraham I <[email protected]>--- drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 + drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++++++++++++++++++++++++++++ 5 files changed, 764 insertions(+) create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644
drivers/ntb/hw/epf/Makefilecreate mode 100644drivers/ntb/hw/epf/ntb_hw_epf.cdrivers/ntb/hw/Kconfig index renamed from drivers/ntb/hw/Kconfig +0
-0
e77c587060ff..c325be526b80100644drivers/ntb/hw/Kconfig +1
-0@@ -2,4 +2,5 @@ source "drivers/ntb/hw/amd/Kconfig" source "drivers/ntb/hw/idt/Kconfig" source "drivers/ntb/hw/intel/Kconfig" +source "drivers/ntb/hw/epf/Kconfig" source "drivers/ntb/hw/mscc/Kconfig"drivers/ntb/hw/Makefile index renamed from drivers/ntb/hw/Makefile +0
-0
4714d6238845..223ca592b5f9100644drivers/ntb/hw/Makefile +1
-0@@ -2,4 +2,5 @@ obj-$(CONFIG_NTB_AMD) += amd/ obj-$(CONFIG_NTB_IDT) += idt/ obj-$(CONFIG_NTB_INTEL) += intel/ +obj-$(CONFIG_NTB_EPF) += epf/ obj-$(CONFIG_NTB_SWITCHTEC) += mscc/drivers/ntb/hw/epf/Kconfig new renamed from drivers/ntb/hw/epf/Kconfig +0
-0file mode 100644 index 000000000000..6197d1aab344
drivers/ntb/hw/epf/Kconfig new file +6
-0@@ -0,0 +1,6 @@ +config NTB_EPF + tristate "Generic EPF Non-Transparent Bridge support" + depends on m + help + This driver supports EPF NTB on configurable endpoint. + If unsure, say N.drivers/ntb/hw/epf/Makefile new renamed from drivers/ntb/hw/epf/Makefile +0
-0file mode 100644 index 000000000000..2f560a422bc6
drivers/ntb/hw/epf/Makefile new file +1
-0@@ -0,0 +1 @@ +obj-$(CONFIG_NTB_EPF) += ntb_hw_epf.o(quoted diff) +0
-0
b/drivers/ntb/hw/epf/ntb_hw_epf.cnew file mode 100644 index 000000000000..0a144987851adrivers/ntb/hw/epf/ntb_hw_epf.c new file +0
-0@@ -0,0 +1,755 @@......
Quoted 26 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +25
-0+static int ntb_epf_init_pci (struct ntb_epf_dev *ndev, +struct pci_dev *pdev) +{ +struct device *dev = ndev->dev; +int ret; + +pci_set_drvdata (pdev, ndev); + + ret =pci_enable_device (pdev); +if (ret) { +dev_err (dev,"Cannot enable PCI device\n" ); + goto err_pci_enable; + } + + ret =pci_request_regions (pdev,"ntb" ); +if (ret) { +dev_err (dev,"Cannot obtain PCI resources\n" ); + goto err_pci_regions; + } + +pci_set_master (pdev); + + ret =dma_set_mask_and_coherent (dev,DMA_BIT_MASK (64 )); +if (ret) { + ret = dma_set_mask_and_coherent(dev,
DMA_BIT_MASK(32));drivers/ntb/hw/epf/ntb_hw_epf.c line 596
Quoted 8 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +8
-0+if (ret) { +dev_err (dev,"Cannot set DMA mask\n" ); + goto err_dma_mask; + } +dev_warn (&pdev->dev,"Cannot DMA highmem\n" ); + } + + ndev->ctrl_reg =pci_iomap (pdev,0 ,0 );The second parameter of
pci_iomapshould bendev->ctrl_reg_barinstead of the hardcode value 0, right?drivers/ntb/hw/epf/ntb_hw_epf.c line 602
Quoted 6 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +6
-0+if (!ndev->ctrl_reg) { + ret = -EIO ; + goto err_dma_mask; + } + + ndev->peer_spad_reg =pci_iomap (pdev,1 ,0 );
pci_iomap(pdev, ndev->peer_spad_reg_bar, 0);
Quoted 8 lines
drivers/ntb/hw/epf/ntb_hw_epf.c line 608
Quoted 6 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +6
-0+if (!ndev->peer_spad_reg) { + ret = -EIO ; + goto err_dma_mask; + } + + ndev->db_reg =pci_iomap (pdev,2 ,0 );
pci_iomap(pdev, ndev->db_reg_bar, 0);
Good catch. Will fix it and send. Thank you for reviewing.
Regards,
Kishon
RE: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Hi Kishon,
Quoted 136 lines
Subject: Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- Transparent Bridge
Hi Sherry,
Quoted 10 lines · Sherry Sun
Hi Kishon,
Quoted 8 lines
Subject: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- Transparent Bridge
From: Kishon Vijay Abraham I <[email protected]>
Add support for EPF PCI-Express Non-Transparent Bridge (NTB) device. This driver is platform independent and could be used by any platform which have multiple PCIe endpoint instances configured using the pci-epf-
ntb driver.
Quoted 91 lines
Quoted 62 lines
The driver connnects to the standard NTB sub-system interface. The EPF NTB device has configurable number of memory windows (Max 4), configurable number of doorbell (Max 32), and configurable number of scratch-pad registers.
Signed-off-by Kishon Vijay Abraham I <[email protected]>--- drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 + drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++++++++++++++++++++++++++++ 5 files changed, 764 insertions(+) create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644
drivers/ntb/hw/epf/Makefilecreate mode 100644drivers/ntb/hw/epf/ntb_hw_epf.cdrivers/ntb/hw/Kconfig index renamed from drivers/ntb/hw/Kconfig +0
-0
e77c587060ff..c325be526b80100644drivers/ntb/hw/Kconfig +1
-0@@ -2,4 +2,5 @@ source "drivers/ntb/hw/amd/Kconfig" source "drivers/ntb/hw/idt/Kconfig" source "drivers/ntb/hw/intel/Kconfig" +source "drivers/ntb/hw/epf/Kconfig" source "drivers/ntb/hw/mscc/Kconfig"drivers/ntb/hw/Makefile index renamed from drivers/ntb/hw/Makefile +0
-0
4714d6238845..223ca592b5f9100644drivers/ntb/hw/Makefile +1
-0@@ -2,4 +2,5 @@ obj-$(CONFIG_NTB_AMD) += amd/ obj-$(CONFIG_NTB_IDT) += idt/ obj-$(CONFIG_NTB_INTEL) += intel/ +obj-$(CONFIG_NTB_EPF) += epf/ obj-$(CONFIG_NTB_SWITCHTEC) += mscc/ diff --gita/drivers/ntb/hw/epf/Kconfig b/drivers/ntb/hw/epf/Kconfig new file mode 100644 index 000000000000..6197d1aab344
drivers/ntb/hw/epf/Kconfig new file +6
-0@@ -0,0 +1,6 @@ +config NTB_EPF + tristate "Generic EPF Non-Transparent Bridge support" + depends on m + help + This driver supports EPF NTB on configurable endpoint. + If unsure, say N.(quoted diff) +0
-0b/drivers/ntb/hw/epf/Makefile new file mode 100644 index 000000000000..2f560a422bc6
drivers/ntb/hw/epf/Makefile new file +1
-0@@ -0,0 +1 @@ +obj-$(CONFIG_NTB_EPF) += ntb_hw_epf.o(quoted diff) +0
-0
b/drivers/ntb/hw/epf/ntb_hw_epf.cnew file mode 100644 index 000000000000..0a144987851adrivers/ntb/hw/epf/ntb_hw_epf.c new file +0
-0@@ -0,0 +1,755 @@......
Quoted 28 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +25
-0+static int ntb_epf_init_pci (struct ntb_epf_dev *ndev, +struct pci_dev *pdev) +{ +struct device *dev = ndev->dev; +int ret; + +pci_set_drvdata (pdev, ndev); + + ret =pci_enable_device (pdev); +if (ret) { +dev_err (dev,"Cannot enable PCI device\n" ); + goto err_pci_enable; + } + + ret =pci_request_regions (pdev,"ntb" ); +if (ret) { +dev_err (dev,"Cannot obtain PCI resources\n" ); + goto err_pci_regions; + } + +pci_set_master (pdev); + + ret =dma_set_mask_and_coherent (dev,DMA_BIT_MASK (64 )); +if (ret) { + ret = dma_set_mask_and_coherent(dev,
DMA_BIT_MASK(32));drivers/ntb/hw/epf/ntb_hw_epf.c new file +2
-0+if (ret) { +dev_err (dev,"Cannot set DMA mask\n" );Quoted 8 lines
drivers/ntb/hw/epf/ntb_hw_epf.c line 596
Quoted 6 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +6
-0+ goto err_dma_mask; + } +dev_warn (&pdev->dev,"Cannot DMA highmem\n" ); + } + + ndev->ctrl_reg =pci_iomap (pdev,0 ,0 );The second parameter of
pci_iomapshould bendev->ctrl_reg_barinsteadof the hardcode value 0, right?
Quoted 10 lines
drivers/ntb/hw/epf/ntb_hw_epf.c line 602
Quoted 7 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +6
-0+if (!ndev->ctrl_reg) { + ret = -EIO ; + goto err_dma_mask; + } + + ndev->peer_spad_reg =pci_iomap (pdev,1 ,0 );
pci_iomap(pdev, ndev->peer_spad_reg_bar, 0);Quoted 8 lines
drivers/ntb/hw/epf/ntb_hw_epf.c line 608
Quoted 6 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +6
-0+if (!ndev->peer_spad_reg) { + ret = -EIO ; + goto err_dma_mask; + } + + ndev->db_reg =pci_iomap (pdev,2 ,0 );
pci_iomap(pdev, ndev->db_reg_bar, 0);
Quoted 1 line
Good catch. Will fix it and send. Thank you for reviewing.
You're welcome.
By the way, since I've studied VOP(virtio over pcie) before, and only recently learned about NTB related code. I have some questions about NTB.
For NTB, in order to make two(or more) different systems to communicate with each other, seems at least three boards are required(two host boards and one board with multiple EP instances as bridge), right?
But for VOP, only two boards are needed(one board as host and one board as card) to realize the communication between the two systems, so my question is what are the advantages of using NTB? Because I think the architecture of NTB seems more complicated. Many thanks!
Best regards
Sherry
Quoted 3 lines
Regards,
Kishon
Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Hi Sherry,
Quoted 140 lines · Sherry Sun
Hi Kishon,
Quoted 136 lines
Subject: Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- Transparent Bridge
Hi Sherry,
Quoted 10 lines · Sherry Sun
Hi Kishon,
Quoted 8 lines
Subject: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- Transparent Bridge
From: Kishon Vijay Abraham I <[email protected]>
Add support for EPF PCI-Express Non-Transparent Bridge (NTB) device. This driver is platform independent and could be used by any platform which have multiple PCIe endpoint instances configured using the pci-epf-
ntb driver.
Quoted 91 lines
Quoted 62 lines
The driver connnects to the standard NTB sub-system interface. The EPF NTB device has configurable number of memory windows (Max 4), configurable number of doorbell (Max 32), and configurable number of scratch-pad registers.
Signed-off-by Kishon Vijay Abraham I <[email protected]>--- drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 + drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++++++++++++++++++++++++++++ 5 files changed, 764 insertions(+) create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644
drivers/ntb/hw/epf/Makefilecreate mode 100644drivers/ntb/hw/epf/ntb_hw_epf.cdrivers/ntb/hw/Kconfig index renamed from drivers/ntb/hw/Kconfig +0
-0
e77c587060ff..c325be526b80100644drivers/ntb/hw/Kconfig +1
-0@@ -2,4 +2,5 @@ source "drivers/ntb/hw/amd/Kconfig" source "drivers/ntb/hw/idt/Kconfig" source "drivers/ntb/hw/intel/Kconfig" +source "drivers/ntb/hw/epf/Kconfig" source "drivers/ntb/hw/mscc/Kconfig"drivers/ntb/hw/Makefile index renamed from drivers/ntb/hw/Makefile +0
-0
4714d6238845..223ca592b5f9100644drivers/ntb/hw/Makefile +1
-0@@ -2,4 +2,5 @@ obj-$(CONFIG_NTB_AMD) += amd/ obj-$(CONFIG_NTB_IDT) += idt/ obj-$(CONFIG_NTB_INTEL) += intel/ +obj-$(CONFIG_NTB_EPF) += epf/ obj-$(CONFIG_NTB_SWITCHTEC) += mscc/ diff --gita/drivers/ntb/hw/epf/Kconfig b/drivers/ntb/hw/epf/Kconfig new file mode 100644 index 000000000000..6197d1aab344
drivers/ntb/hw/epf/Kconfig new file +6
-0@@ -0,0 +1,6 @@ +config NTB_EPF + tristate "Generic EPF Non-Transparent Bridge support" + depends on m + help + This driver supports EPF NTB on configurable endpoint. + If unsure, say N.(quoted diff) +0
-0b/drivers/ntb/hw/epf/Makefile new file mode 100644 index 000000000000..2f560a422bc6
drivers/ntb/hw/epf/Makefile new file +1
-0@@ -0,0 +1 @@ +obj-$(CONFIG_NTB_EPF) += ntb_hw_epf.o(quoted diff) +0
-0
b/drivers/ntb/hw/epf/ntb_hw_epf.cnew file mode 100644 index 000000000000..0a144987851adrivers/ntb/hw/epf/ntb_hw_epf.c new file +0
-0@@ -0,0 +1,755 @@......
Quoted 28 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +25
-0+static int ntb_epf_init_pci (struct ntb_epf_dev *ndev, +struct pci_dev *pdev) +{ +struct device *dev = ndev->dev; +int ret; + +pci_set_drvdata (pdev, ndev); + + ret =pci_enable_device (pdev); +if (ret) { +dev_err (dev,"Cannot enable PCI device\n" ); + goto err_pci_enable; + } + + ret =pci_request_regions (pdev,"ntb" ); +if (ret) { +dev_err (dev,"Cannot obtain PCI resources\n" ); + goto err_pci_regions; + } + +pci_set_master (pdev); + + ret =dma_set_mask_and_coherent (dev,DMA_BIT_MASK (64 )); +if (ret) { + ret = dma_set_mask_and_coherent(dev,
DMA_BIT_MASK(32));drivers/ntb/hw/epf/ntb_hw_epf.c new file +2
-0+if (ret) { +dev_err (dev,"Cannot set DMA mask\n" );Quoted 8 lines
drivers/ntb/hw/epf/ntb_hw_epf.c line 596
Quoted 6 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +6
-0+ goto err_dma_mask; + } +dev_warn (&pdev->dev,"Cannot DMA highmem\n" ); + } + + ndev->ctrl_reg =pci_iomap (pdev,0 ,0 );The second parameter of
pci_iomapshould bendev->ctrl_reg_barinsteadof the hardcode value 0, right?
Quoted 10 lines
drivers/ntb/hw/epf/ntb_hw_epf.c line 602
Quoted 7 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +6
-0+if (!ndev->ctrl_reg) { + ret = -EIO ; + goto err_dma_mask; + } + + ndev->peer_spad_reg =pci_iomap (pdev,1 ,0 );
pci_iomap(pdev, ndev->peer_spad_reg_bar, 0);Quoted 8 lines
drivers/ntb/hw/epf/ntb_hw_epf.c line 608
Quoted 6 lines
drivers/ntb/hw/epf/ntb_hw_epf.c new file +6
-0+if (!ndev->peer_spad_reg) { + ret = -EIO ; + goto err_dma_mask; + } + + ndev->db_reg =pci_iomap (pdev,2 ,0 );
pci_iomap(pdev, ndev->db_reg_bar, 0);Quoted 1 line
Good catch. Will fix it and send. Thank you for reviewing.
Quoted 7 lines
You're welcome.
By the way, since I've studied VOP(virtio over pcie) before, and only recently learned about NTB related code. I have some questions about NTB.For NTB, in order to make two(or more) different systems to communicate with each other, seems at least three boards are required(two host boards and one board with multiple EP instances as bridge), right?
right, this series is about creating NTB bridge by configuring multiple EP instances in an SoC, however there are also dedicated HW NTB switches (internally they might as well use multiple EP instances).
Quoted 2 lines
But for VOP, only two boards are needed(one board as host and one board as card) to realize the communication between the two systems, so my question is what are the advantages of using NTB?
NTB is a bridge that facilitates communication between two different systems. So it by itself will not be source or sink of any data unlike a normal EP to RP system (or the VOP) which will be source or sink of data.
Quoted 1 line
Because I think the architecture of NTB seems more complicated. Many thanks!
yeah, I think it enables a different use case all together. Consider you have two x86 HOST PCs (having RP) and they have to be communicate using PCIe. NTB can be used in such cases for the two x86 PCs to communicate with each other over PCIe, which wouldn't be possible without NTB.
Regards,
Kishon
Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Quoted 9 lines · Kishon Vijay Abraham I
Quoted 2 lines · Sherry Sun
But for VOP, only two boards are needed(one board as host and one board as card) to realize the communication between the two systems, so my question is what are the advantages of using NTB?
NTB is a bridge that facilitates communication between two different systems. So it by itself will not be source or sink of any data unlike a normal EP to RP system (or the VOP) which will be source or sink of data.
Quoted 6 lines
Quoted 1 line
Because I think the architecture of NTB seems more complicated. Many thanks!
yeah, I think it enables a different use case all together. Consider you have two x86 HOST PCs (having RP) and they have to be communicate using PCIe. NTB can be used in such cases for the two x86 PCs to communicate with each other over PCIe, which wouldn't be possible without NTB.
I think for VOP, we should have an abstraction that can work on either NTB or directly on the endpoint framework but provide an interface that then lets you create logical devices the same way.
Doing VOP based on NTB plus the new NTB_EPF driver would also work and just move the abstraction somewhere else, but I guess it would complicate setting it up for those users that only care about the simpler endpoint case.
Arnd
Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Hi Sherry, Arnd,
Quoted 17 lines · Arnd Bergmann
Quoted 9 lines · Kishon Vijay Abraham I
Quoted 2 lines · Sherry Sun
But for VOP, only two boards are needed(one board as host and one board as card) to realize the communication between the two systems, so my question is what are the advantages of using NTB?
NTB is a bridge that facilitates communication between two different systems. So it by itself will not be source or sink of any data unlike a normal EP to RP system (or the VOP) which will be source or sink of data.
Quoted 6 lines
Quoted 1 line
Because I think the architecture of NTB seems more complicated. Many thanks!
yeah, I think it enables a different use case all together. Consider you have two x86 HOST PCs (having RP) and they have to be communicate using PCIe. NTB can be used in such cases for the two x86 PCs to communicate with each other over PCIe, which wouldn't be possible without NTB.
Quoted 8 lines
I think for VOP, we should have an abstraction that can work on either NTB or directly on the endpoint framework but provide an interface that then lets you create logical devices the same way.
Doing VOP based on NTB plus the new
NTB_EPFdriver would also work and just move the abstraction somewhere else, but I guess it would complicate setting it up for those users that only care about the simpler endpoint case.
I'm not sure if you've got a chance to look at [1], where I added support for RP<->EP system both running Linux, with EP configured using Linux EP framework (as well as HOST ports connected to NTB switch, patches 20 and 21, that uses the Linux NTB framework) to communicate using virtio over PCIe.
The cover-letter [1] shows a picture of the two use cases supported in that series.
[1] -> http://lore.kernel.org/r/[email protected]
Thank You,
Kishon
RE: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Hi Kishon,
Quoted 37 lines
Subject: Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- Transparent Bridge
Hi Sherry, Arnd,
Quoted 1 line · Arnd Bergmann
On Tue, Nov 10, 2020 at 3:20 PM Kishon Vijay Abraham I <[email protected]>
Quoted 4 lines · wrote:
Quoted 4 lines · Sherry Sun
Quoted 3 lines
But for VOP, only two boards are needed(one board as host and one board as card) to realize the communication between the two systems,
so my question is what are the advantages of using NTB?
Quoted 4 lines
Quoted 4 lines
NTB is a bridge that facilitates communication between two different systems. So it by itself will not be source or sink of any data unlike a normal EP to RP system (or the VOP) which will be source or sink
of data.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Because I think the architecture of NTB seems more complicated. Many
thanks!
Quoted 5 lines
Quoted 5 lines
yeah, I think it enables a different use case all together. Consider you have two x86 HOST PCs (having RP) and they have to be communicate using PCIe. NTB can be used in such cases for the two x86 PCs to communicate with each other over PCIe, which wouldn't be possible
without NTB.
Quoted 9 lines
I think for VOP, we should have an abstraction that can work on either NTB or directly on the endpoint framework but provide an interface that then lets you create logical devices the same way.
Doing VOP based on NTB plus the new
NTB_EPFdriver would also work and just move the abstraction somewhere else, but I guess it would complicate setting it up for those users that only care about the simpler endpoint case.
Quoted 6 lines
I'm not sure if you've got a chance to look at [1], where I added support for RP<->EP system both running Linux, with EP configured using Linux EP framework (as well as HOST ports connected to NTB switch, patches 20 and 21, that uses the Linux NTB framework) to communicate using virtio over PCIe.
I saw your patches at [1], here you take a rpmsg as an example to communicate between two SoCs using PCIe RC<->EP and HOST1-NTB-HOST2 for different usercases. The VOP code works under the PCIe RC<->EP framework, which means that we can also make VOP works under the Linux NTB framework, just like the rpmsg way you did here, right?
Best regards
Sherry
Quoted 15 lines
The cover-letter [1] shows a picture of the two use cases supported in that series.
[1] ->
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flore.ke
rnel.org%2Fr%2F20200702082143.25259-1- kishon%40ti.com&data=04%7C01%7Csherry.sun%40nxp.com%7C5d8b7 3a4b72947bea65d08d8858f5091%7C686ea1d3bc2b4c6fa92cd99c5c301635%7 C0%7C0%7C637406197865119992%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C100 0&sdata=iRrBvQ9xjoOUYU%2FDidMLZZpW6XuU4ITVXFDA%2B%2F4rJFU %3D&reserved=0Thank You,
Kishon
Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Hi Sherry,
Quoted 46 lines · Sherry Sun
Hi Kishon,
Quoted 37 lines
Subject: Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- Transparent Bridge
Hi Sherry, Arnd,
Quoted 1 line · Arnd Bergmann
On Tue, Nov 10, 2020 at 3:20 PM Kishon Vijay Abraham I <[email protected]>
Quoted 4 lines · wrote:
Quoted 4 lines · Sherry Sun
Quoted 3 lines
But for VOP, only two boards are needed(one board as host and one board as card) to realize the communication between the two systems,
so my question is what are the advantages of using NTB?
Quoted 4 lines
Quoted 4 lines
NTB is a bridge that facilitates communication between two different systems. So it by itself will not be source or sink of any data unlike a normal EP to RP system (or the VOP) which will be source or sink
of data.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Because I think the architecture of NTB seems more complicated. Many
thanks!
Quoted 5 lines
Quoted 5 lines
yeah, I think it enables a different use case all together. Consider you have two x86 HOST PCs (having RP) and they have to be communicate using PCIe. NTB can be used in such cases for the two x86 PCs to communicate with each other over PCIe, which wouldn't be possible
without NTB.
Quoted 9 lines
I think for VOP, we should have an abstraction that can work on either NTB or directly on the endpoint framework but provide an interface that then lets you create logical devices the same way.
Doing VOP based on NTB plus the new
NTB_EPFdriver would also work and just move the abstraction somewhere else, but I guess it would complicate setting it up for those users that only care about the simpler endpoint case.Quoted 6 lines
I'm not sure if you've got a chance to look at [1], where I added support for RP<->EP system both running Linux, with EP configured using Linux EP framework (as well as HOST ports connected to NTB switch, patches 20 and 21, that uses the Linux NTB framework) to communicate using virtio over PCIe.
Quoted 4 lines
I saw your patches at [1], here you take a rpmsg as an example to communicate between two SoCs using PCIe RC<->EP and HOST1-NTB-HOST2 for different usercases. The VOP code works under the PCIe RC<->EP framework, which means that we can also make VOP works under the Linux NTB framework, just like the rpmsg way you did here, right?
Does VOP really work with EP framework? At-least whatever is in upstream doesn't seem to indicate so.
The NTB framework lets one host with RP port to communicate with another host with RP port.
The EP Framework lets one device with EP port to communicate with a host with RP port.
Rest of the trick should be how you tie them together.
PCIe framework creates "pci_device" for each of the devices it enumerates. NTB framework works on this pci_device to communicate with the remote host using PCIe bridge. The remote host will use NTB framework as well.
So depends on what interfaces VOP device provides you can use either NTB framework or EP framework. If it's going to connect two different devices in turn creating pci_device on each of the systems, then you can use NTB framework.
Regards
Kishon
RE: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Hi Kishon,
Quoted 59 lines
Subject: Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- Transparent Bridge
Hi Sherry,
Quoted 51 lines · Sherry Sun
Hi Kishon,
Quoted 42 lines
Subject: Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- Transparent Bridge
Hi Sherry, Arnd,
Quoted 2 lines · Arnd Bergmann
On Tue, Nov 10, 2020 at 3:20 PM Kishon Vijay Abraham I <[email protected]>
Quoted 5 lines · wrote:
Quoted 5 lines · Sherry Sun
Quoted 4 lines
But for VOP, only two boards are needed(one board as host and one board as card) to realize the communication between the two systems,
so my question is what are the advantages of using NTB?
Quoted 5 lines
Quoted 5 lines
NTB is a bridge that facilitates communication between two different systems. So it by itself will not be source or sink of any data unlike a normal EP to RP system (or the VOP) which will be source or sink
of data.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Because I think the architecture of NTB seems more complicated. Many
thanks!
Quoted 6 lines
Quoted 6 lines
yeah, I think it enables a different use case all together. Consider you have two x86 HOST PCs (having RP) and they have to be communicate using PCIe. NTB can be used in such cases for the two x86 PCs to communicate with each other over PCIe, which wouldn't be possible
without NTB.
Quoted 9 lines
I think for VOP, we should have an abstraction that can work on either NTB or directly on the endpoint framework but provide an interface that then lets you create logical devices the same way.
Doing VOP based on NTB plus the new
NTB_EPFdriver would also work and just move the abstraction somewhere else, but I guess it would complicate setting it up for those users that only care about the simpler endpoint case.Quoted 6 lines
I'm not sure if you've got a chance to look at [1], where I added support for RP<->EP system both running Linux, with EP configured using Linux EP framework (as well as HOST ports connected to NTB switch, patches 20 and 21, that uses the Linux NTB framework) to communicate using virtio over PCIe.
Quoted 2 lines
I saw your patches at [1], here you take a rpmsg as an example to communicate between two SoCs using PCIe RC<->EP and HOST1-NTB-
Quoted 8 lines
HOST2 for different usercases.
Quoted 2 lines
The VOP code works under the PCIe RC<->EP framework, which means that we can also make VOP works under the Linux NTB framework, just like the
rpmsg way you did here, right?
Does VOP really work with EP framework? At-least whatever is in upstream doesn't seem to indicate so.
We did write a pci_epf driver to support VOP, looks like pci-epf-test.c, and it works well.
So certainly VOP can work with EP framework.
But it's a pity that the VOP related codes has been deleted before we send the pci_epf_vop driver patches to upstream.
Quoted 8 lines
The NTB framework lets one host with RP port to communicate with another host with RP port.
The EP Framework lets one device with EP port to communicate with a host with RP port.
Rest of the trick should be how you tie them together.
Quoted 9 lines
PCIe framework creates "
pci_device" for each of the devices it enumerates. NTB framework works on thispci_deviceto communicate with the remote host using PCIe bridge. The remote host will use NTB framework as well.So depends on what interfaces VOP device provides you can use either NTB framework or EP framework. If it's going to connect two different devices in turn creating
pci_deviceon each of the systems, then you can use NTB framework.
Thanks for your detailed explanation! It is clear.
I think maybe VOP is suitable for the basic PCIe framework instead of NTB.
Best regards
Sherry
Quoted 2 lines
Regards
Kishon
Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
In reply to Kishon Vijay Abraham IQuoted 33 lines · Kishon Vijay Abraham I
Quoted 17 lines · Arnd Bergmann
Quoted 9 lines · Kishon Vijay Abraham I
Quoted 2 lines · Sherry Sun
But for VOP, only two boards are needed(one board as host and one board as card) to realize the communication between the two systems, so my question is what are the advantages of using NTB?
NTB is a bridge that facilitates communication between two different systems. So it by itself will not be source or sink of any data unlike a normal EP to RP system (or the VOP) which will be source or sink of data.
Quoted 6 lines
Quoted 1 line
Because I think the architecture of NTB seems more complicated. Many thanks!
yeah, I think it enables a different use case all together. Consider you have two x86 HOST PCs (having RP) and they have to be communicate using PCIe. NTB can be used in such cases for the two x86 PCs to communicate with each other over PCIe, which wouldn't be possible without NTB.
Quoted 8 lines
I think for VOP, we should have an abstraction that can work on either NTB or directly on the endpoint framework but provide an interface that then lets you create logical devices the same way.
Doing VOP based on NTB plus the new
NTB_EPFdriver would also work and just move the abstraction somewhere else, but I guess it would complicate setting it up for those users that only care about the simpler endpoint case.I'm not sure if you've got a chance to look at [1], where I added support for RP<->EP system both running Linux, with EP configured using Linux EP framework (as well as HOST ports connected to NTB switch, patches 20 and 21, that uses the Linux NTB framework) to communicate using virtio over PCIe.
Quoted 4 lines
The cover-letter [1] shows a picture of the two use cases supported in that series.
No, I missed, that, thanks for pointing me to it!
This looks very promising indeed, I need to read up on the whole discussion there. I also see your slides at [1] that help do explain some of it. I have one fundamental question that I can't figure out from the description, maybe you can help me here:
How is the configuration managed, taking the EP case as an example? Your UseCase1 example sounds like the system that owns the EP hardware is the one that turns the EP into a vhost device, and creates a vhost-rpmsg device on top, while the RC side would probe the pci-vhost and then detect a virtio-rpmsg device to talk to. Can it also do the opposite, so you end up with e.g. a virtio-net device on the EP side and vhost-net on the RC?
Arnd
Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Hi Arnd,
Quoted 46 lines · Arnd Bergmann
Quoted 33 lines · Kishon Vijay Abraham I
Quoted 17 lines · Arnd Bergmann
Quoted 9 lines · Kishon Vijay Abraham I
Quoted 2 lines · Sherry Sun
But for VOP, only two boards are needed(one board as host and one board as card) to realize the communication between the two systems, so my question is what are the advantages of using NTB?
NTB is a bridge that facilitates communication between two different systems. So it by itself will not be source or sink of any data unlike a normal EP to RP system (or the VOP) which will be source or sink of data.
Quoted 6 lines
Quoted 1 line
Because I think the architecture of NTB seems more complicated. Many thanks!
yeah, I think it enables a different use case all together. Consider you have two x86 HOST PCs (having RP) and they have to be communicate using PCIe. NTB can be used in such cases for the two x86 PCs to communicate with each other over PCIe, which wouldn't be possible without NTB.
Quoted 8 lines
I think for VOP, we should have an abstraction that can work on either NTB or directly on the endpoint framework but provide an interface that then lets you create logical devices the same way.
Doing VOP based on NTB plus the new
NTB_EPFdriver would also work and just move the abstraction somewhere else, but I guess it would complicate setting it up for those users that only care about the simpler endpoint case.I'm not sure if you've got a chance to look at [1], where I added support for RP<->EP system both running Linux, with EP configured using Linux EP framework (as well as HOST ports connected to NTB switch, patches 20 and 21, that uses the Linux NTB framework) to communicate using virtio over PCIe.
Quoted 4 lines
The cover-letter [1] shows a picture of the two use cases supported in that series.
No, I missed, that, thanks for pointing me to it!
This looks very promising indeed, I need to read up on the whole discussion there. I also see your slides at [1] that help do explain some of it. I have one fundamental question that I can't figure out from the description, maybe you can help me here:
Quoted 5 lines
How is the configuration managed, taking the EP case as an example? Your UseCase1 example sounds like the system that owns the EP hardware is the one that turns the EP into a vhost device, and creates a vhost-rpmsg device on top, while the RC side would probe the pci-vhost and then detect a virtio-rpmsg device to talk to.
That's correct. Slide no 9 in [1] should give the layering details.
Quoted 2 lines
Can it also do the opposite, so you end up with e.g. a virtio-net device on the EP side and vhost-net on the RC?
Unfortunately no. Again referring slide 9 in [1], we only have vhost-pci-epf on the EP side which only creates a "vhost_dev" to deal with vhost side of things. For doing the opposite, we'd need to create virtio-pci-epf for EP side that interacts with core virtio (and also the corresponding vhost back end on PCI host).
Thanks
Kishon
Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Quoted 15 lines · Kishon Vijay Abraham I
Quoted 11 lines · Arnd Bergmann
This looks very promising indeed, I need to read up on the whole discussion there. I also see your slides at [1] that help do explain some of it. I have one fundamental question that I can't figure out from the description, maybe you can help me here:
How is the configuration managed, taking the EP case as an example? Your UseCase1 example sounds like the system that owns the EP hardware is the one that turns the EP into a vhost device, and creates a vhost-rpmsg device on top, while the RC side would probe the pci-vhost and then detect a virtio-rpmsg device to talk to.
That's correct. Slide no 9 in [1] should give the layering details.
Quoted 8 lines
Quoted 2 lines
Can it also do the opposite, so you end up with e.g. a virtio-net device on the EP side and vhost-net on the RC?
Unfortunately no. Again referring slide 9 in [1], we only have vhost-pci-epf on the EP side which only creates a "
vhost_dev" to deal with vhost side of things. For doing the opposite, we'd need to create virtio-pci-epf for EP side that interacts with core virtio (and also the corresponding vhost back end on PCI host).
Ok, I see. So I think this is the opposite of drivers/misc/mic and the bluefield driver were using, so we would probably end up needing both.
Then again, I guess the NTB driver would give us the functionality for free, if it shows a symmetric link?
Arnd
Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge
Hi Arnd,
Quoted 25 lines · Arnd Bergmann
Quoted 15 lines · Kishon Vijay Abraham I
Quoted 11 lines · Arnd Bergmann
This looks very promising indeed, I need to read up on the whole discussion there. I also see your slides at [1] that help do explain some of it. I have one fundamental question that I can't figure out from the description, maybe you can help me here:
How is the configuration managed, taking the EP case as an example? Your UseCase1 example sounds like the system that owns the EP hardware is the one that turns the EP into a vhost device, and creates a vhost-rpmsg device on top, while the RC side would probe the pci-vhost and then detect a virtio-rpmsg device to talk to.
That's correct. Slide no 9 in [1] should give the layering details.
Quoted 8 lines
Quoted 2 lines
Can it also do the opposite, so you end up with e.g. a virtio-net device on the EP side and vhost-net on the RC?
Unfortunately no. Again referring slide 9 in [1], we only have vhost-pci-epf on the EP side which only creates a "
vhost_dev" to deal with vhost side of things. For doing the opposite, we'd need to create virtio-pci-epf for EP side that interacts with core virtio (and also the corresponding vhost back end on PCI host).
Quoted 6 lines
Ok, I see. So I think this is the opposite of
drivers/misc/micand the bluefield driver were using, so we would probably end up needing both.Then again, I guess the NTB driver would give us the functionality for free, if it shows a symmetric link?
Right, NTB driver would need "pci_dev" on both sides of the link. But that would also mean we cannot use pci EP framework which actually uses "pci_epf".
Thanks
Kishon
PATCH v7 16/18 NTB: tool: Enable the NTB/PCIe link on the local or remote side of bridge
In reply to Kishon Vijay Abraham IInvoke ntb_link_enable() to enable the NTB/PCIe link on the local or remote side of the bridge.
--- drivers/ntb/test/ntb_tool.c | 1 + 1 file changed, 1 insertion(+)
drivers/ntb/test/ntb_tool.c +1 -0
@@ -1638,6 +1638,7 @@ static int tool_probe(struct ntb_client *self, struct ntb_dev *ntb)tool_setup_dbgfs (tc); +ntb_link_enable (ntb,NTB_SPEED_AUTO ,NTB_WIDTH_AUTO );return 0 ; err_clear_mws:
Signature · 1 line
2.17.1
PATCH v7 17/18 Documentation: PCI: Add configfs binding documentation for pci-ntb endpoint function
In reply to Kishon Vijay Abraham IAdd binding documentation for pci-ntb endpoint function that helps in adding and configuring pci-ntb endpoint function.
--- .../PCI/endpoint/function/binding/pci-ntb.rst | 38 +++++++++++++++++++ Documentation/PCI/endpoint/index.rst | 1 + 2 files changed, 39 insertions(+) create mode 100644 Documentation/PCI/endpoint/function/binding/pci-ntb.rst
Documentation/PCI/endpoint/function/binding/pci-ntb.rst new file +38 -0
@@ -0,0 +1,38 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================== +PCI NTB Endpoint Function +========================== + +1) Create a subdirectory to pci_epf_ntb directory in configfs. + +Standard EPF Configurable Fields: + +================ =========================================================== +vendorid should be 0x104c +deviceid should be 0xb00d for TI's J721E SoC +revid don't care +progif_code don't care +subclass_code should be 0x00 +baseclass_code should be 0x5 +cache_line_size don't care +subsys_vendor_id don't care +subsys_id don't care +interrupt_pin don't care +msi_interrupts don't care +msix_interrupts don't care +================ =========================================================== + +2) Create a subdirectory to directory created in 1 + +NTB EPF specific configurable fields: + +================ =========================================================== +db_count Number of doorbells; default = 4 +mw1 size of memory window1 +mw2 size of memory window2 +mw3 size of memory window3 +mw4 size of memory window4 +num_mws Number of memory windows; max = 4 +spad_count Number of scratchpad registers; default = 64 +================ ===========================================================
Documentation/PCI/endpoint/index.rst +1 -0
@@ -14,3 +14,4 @@ PCI Endpoint Framework pci-ntb-function function/binding/pci-test + function/binding/pci-ntb
Signature · 1 line
2.17.1
PATCH v7 18/18 Documentation: PCI: Add userguide for PCI endpoint NTB function
In reply to Kishon Vijay Abraham IAdd documentation to help users use pci-epf-ntb function driver and existing host side NTB infrastructure for NTB functionality.
--- Documentation/PCI/endpoint/index.rst | 1 + Documentation/PCI/endpoint/pci-ntb-howto.rst | 160 +++++++++++++++++++ 2 files changed, 161 insertions(+) create mode 100644 Documentation/PCI/endpoint/pci-ntb-howto.rst
Documentation/PCI/endpoint/index.rst +1 -0
@@ -12,6 +12,7 @@ PCI Endpoint Framework pci-test-function pci-test-howto pci-ntb-function + pci-ntb-howto function/binding/pci-test function/binding/pci-ntb
Documentation/PCI/endpoint/pci-ntb-howto.rst new file +160 -0
160 lines not on this page · raw message
Signature · 1 line
2.17.1
Hi Jon Mason, Allen Hubbe, Dave Jiang,
Quoted 5 lines · Kishon Vijay Abraham I
This series is about implementing SW defined Non-Transparent Bridge (NTB) using multiple endpoint (EP) instances. This series has been tested using 2 endpoint instances in J7 connected to J7 board on one end and DRA7 board on the other end. However there is nothing platform specific for the NTB functionality.
This series has two patches that adds to drivers/ntb/ directory. [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge and [PATCH v7 16/18] NTB: tool: Enable the NTB/PCIe link on the local or remote side of bridge.
If you can review and Ack the above patches, Lorenzo can queue it along with the rest of the series.
Thanks for your help in advance.
Best Regards,
Kishon
Quoted 121 lines
This was presented in Linux Plumbers Conference. Link to presentation and video can be found @ [1]
RFC patch series can be found @ [2]
v1 patch series can be found @ [3]
v2 patch series can be found @ [4]
v3 patch series can be found @ [5]
v4 patch series can be found @ [6]
v5 patch series can be found @ [7]
v6 patch series can be found @ [8]Changes from v6:
1) Fixed issues when multiple NTB devices are creating using multiple functions
2) Fixed issue with writing scratchpad register
3) Created a video demo @ [9]Changes from v5:
1) Fixed a formatting issue inKconfigpointed out by Randy
2) Checked for Error or Null inpci_epc_add_epf()Changes from v4:
1) Fixed error condition checks inpci_epc_add_epf()Changes from v3:
1) Fixed Documentation edits suggested by Randy Dunlap <[email protected]>Changes from v2:
1) Add support for the user to create sub-directory of 'EPF Device' directory (for endpoint function specific configuration using configfs).
2) Add documentation for NTB specific attributes in configfs
3) Check forPCI_CLASS_MEMORY_RAM(PCIe class) before bindingntb_hw_epfdriver
4) Other documentation fixesChanges from v1:
1) As per Rob's comment, removed support for creating NTB function device from DT
2) Add support to create NTB EPF device using configfs (added support in configfs to associate primary and secondary EPC with EPF.Changes from RFC:
1) Converted the DT binding patches to YAML schema and merged the DT binding patches together
2) NTB documentation is converted to .rst
3) One HOST can now interrupt the other HOST using MSI-X interrupts
4) Added support for teardown of memory window and doorbell configuration
5) Add support to provide support 64-bit memory window size from DT[1] -> https://linuxplumbersconf.org/event/4/contributions/395/
[2] -> http://lore.kernel.org/r/[email protected]
[3] -> http://lore.kernel.org/r/[email protected]
[4] -> http://lore.kernel.org/r/[email protected]
[5] -> http://lore.kernel.org/r/[email protected]
[6] -> http://lore.kernel.org/r/[email protected]
[7] -> http://lore.kernel.org/r/[email protected]
[8] -> http://lore.kernel.org/r/[email protected]
[9] -> https://youtu.be/dLKKxrg5-rYKishon Vijay Abraham I (18):
Documentation: PCI: Add specification for the *PCI NTB* function
device
PCI: endpoint: Make*_get_first_free_bar()take into account 64 bit
BAR
PCI: endpoint: Add helper API to get the 'next' unreserved BAR
PCI: endpoint: Make*_free_bar()to return error codes on failure
PCI: endpoint: Remove unusedpci_epf_match_device()
PCI: endpoint: Add support to associate secondary EPC with EPF
PCI: endpoint: Add support in configfs to associate two EPCs with EPF
PCI: endpoint: Addpci_epc_opsto map MSI irq
PCI: endpoint: Addpci_epf_opsfor epf drivers to expose function
specific attrs
PCI: endpoint: Allow user to create sub-directory of 'EPF Device'
directory
PCI: cadence: Implement ->msi_map_irq() ops
PCI: cadence: ConfigureLM_EP_FUNC_CFGbased onepc->function_num_map
PCI: endpoint: Add EP function driver to provide NTB functionality
PCI: Add TI J721E device to pci ids
NTB: Add support for EPF PCI-Express Non-Transparent Bridge
NTB: tool: Enable the NTB/PCIe link on the local or remote side of
bridge
Documentation: PCI: Add configfs binding documentation for pci-ntb
endpoint function
Documentation: PCI: Add userguide for PCI endpoint NTB function.../PCI/endpoint/function/binding/pci-ntb.rst | 38 + Documentation/PCI/endpoint/index.rst | 3 + .../PCI/endpoint/pci-endpoint-cfs.rst | 10 + .../PCI/endpoint/pci-ntb-function.rst | 351 +++ Documentation/PCI/endpoint/pci-ntb-howto.rst | 160 ++ drivers/misc/pci_endpoint_test.c | 1 - drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 + drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++ drivers/ntb/test/ntb_tool.c | 1 + .../pci/controller/cadence/pcie-cadence-ep.c | 60 +- drivers/pci/endpoint/functions/Kconfig | 12 + drivers/pci/endpoint/functions/Makefile | 1 + drivers/pci/endpoint/functions/pci-epf-ntb.c | 2114 +++++++++++++++++ drivers/pci/endpoint/functions/pci-epf-test.c | 13 +- drivers/pci/endpoint/pci-ep-cfs.c | 176 +- drivers/pci/endpoint/pci-epc-core.c | 130 +- drivers/pci/endpoint/pci-epf-core.c | 105 +- include/linux/pci-epc.h | 39 +- include/linux/pci-epf.h | 28 +- include/linux/pci_ids.h | 1 + 23 files changed, 3934 insertions(+), 73 deletions(-) create mode 100644 Documentation/PCI/endpoint/function/binding/pci-ntb.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-function.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-howto.rst create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644 drivers/ntb/hw/epf/Makefile create mode 100644 drivers/ntb/hw/epf/ntb_hw_epf.c create mode 100644 drivers/pci/endpoint/functions/pci-epf-ntb.c
Hi,
Quoted 14 lines · Kishon Vijay Abraham I
Hi Jon Mason, Allen Hubbe, Dave Jiang,
Quoted 5 lines · Kishon Vijay Abraham I
This series is about implementing SW defined Non-Transparent Bridge (NTB) using multiple endpoint (EP) instances. This series has been tested using 2 endpoint instances in J7 connected to J7 board on one end and DRA7 board on the other end. However there is nothing platform specific for the NTB functionality.
This series has two patches that adds to
drivers/ntb/directory. [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge and [PATCH v7 16/18] NTB: tool: Enable the NTB/PCIe link on the local or remote side of bridge.
Quoted 4 lines
If you can review and Ack the above patches, Lorenzo can queue it along with the rest of the series.
Thanks for your help in advance.
Gentle ping on this series.
Thanks
Kishon
Quoted 125 lines
Best Regards,
KishonQuoted 121 lines
This was presented in Linux Plumbers Conference. Link to presentation and video can be found @ [1]
RFC patch series can be found @ [2]
v1 patch series can be found @ [3]
v2 patch series can be found @ [4]
v3 patch series can be found @ [5]
v4 patch series can be found @ [6]
v5 patch series can be found @ [7]
v6 patch series can be found @ [8]Changes from v6:
1) Fixed issues when multiple NTB devices are creating using multiple functions
2) Fixed issue with writing scratchpad register
3) Created a video demo @ [9]Changes from v5:
1) Fixed a formatting issue inKconfigpointed out by Randy
2) Checked for Error or Null inpci_epc_add_epf()Changes from v4:
1) Fixed error condition checks inpci_epc_add_epf()Changes from v3:
1) Fixed Documentation edits suggested by Randy Dunlap <[email protected]>Changes from v2:
1) Add support for the user to create sub-directory of 'EPF Device' directory (for endpoint function specific configuration using configfs).
2) Add documentation for NTB specific attributes in configfs
3) Check forPCI_CLASS_MEMORY_RAM(PCIe class) before bindingntb_hw_epfdriver
4) Other documentation fixesChanges from v1:
1) As per Rob's comment, removed support for creating NTB function device from DT
2) Add support to create NTB EPF device using configfs (added support in configfs to associate primary and secondary EPC with EPF.Changes from RFC:
1) Converted the DT binding patches to YAML schema and merged the DT binding patches together
2) NTB documentation is converted to .rst
3) One HOST can now interrupt the other HOST using MSI-X interrupts
4) Added support for teardown of memory window and doorbell configuration
5) Add support to provide support 64-bit memory window size from DT[1] -> https://linuxplumbersconf.org/event/4/contributions/395/
[2] -> http://lore.kernel.org/r/[email protected]
[3] -> http://lore.kernel.org/r/[email protected]
[4] -> http://lore.kernel.org/r/[email protected]
[5] -> http://lore.kernel.org/r/[email protected]
[6] -> http://lore.kernel.org/r/[email protected]
[7] -> http://lore.kernel.org/r/[email protected]
[8] -> http://lore.kernel.org/r/[email protected]
[9] -> https://youtu.be/dLKKxrg5-rYKishon Vijay Abraham I (18):
Documentation: PCI: Add specification for the *PCI NTB* function
device
PCI: endpoint: Make*_get_first_free_bar()take into account 64 bit
BAR
PCI: endpoint: Add helper API to get the 'next' unreserved BAR
PCI: endpoint: Make*_free_bar()to return error codes on failure
PCI: endpoint: Remove unusedpci_epf_match_device()
PCI: endpoint: Add support to associate secondary EPC with EPF
PCI: endpoint: Add support in configfs to associate two EPCs with EPF
PCI: endpoint: Addpci_epc_opsto map MSI irq
PCI: endpoint: Addpci_epf_opsfor epf drivers to expose function
specific attrs
PCI: endpoint: Allow user to create sub-directory of 'EPF Device'
directory
PCI: cadence: Implement ->msi_map_irq() ops
PCI: cadence: ConfigureLM_EP_FUNC_CFGbased onepc->function_num_map
PCI: endpoint: Add EP function driver to provide NTB functionality
PCI: Add TI J721E device to pci ids
NTB: Add support for EPF PCI-Express Non-Transparent Bridge
NTB: tool: Enable the NTB/PCIe link on the local or remote side of
bridge
Documentation: PCI: Add configfs binding documentation for pci-ntb
endpoint function
Documentation: PCI: Add userguide for PCI endpoint NTB function.../PCI/endpoint/function/binding/pci-ntb.rst | 38 + Documentation/PCI/endpoint/index.rst | 3 + .../PCI/endpoint/pci-endpoint-cfs.rst | 10 + .../PCI/endpoint/pci-ntb-function.rst | 351 +++ Documentation/PCI/endpoint/pci-ntb-howto.rst | 160 ++ drivers/misc/pci_endpoint_test.c | 1 - drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 + drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++ drivers/ntb/test/ntb_tool.c | 1 + .../pci/controller/cadence/pcie-cadence-ep.c | 60 +- drivers/pci/endpoint/functions/Kconfig | 12 + drivers/pci/endpoint/functions/Makefile | 1 + drivers/pci/endpoint/functions/pci-epf-ntb.c | 2114 +++++++++++++++++ drivers/pci/endpoint/functions/pci-epf-test.c | 13 +- drivers/pci/endpoint/pci-ep-cfs.c | 176 +- drivers/pci/endpoint/pci-epc-core.c | 130 +- drivers/pci/endpoint/pci-epf-core.c | 105 +- include/linux/pci-epc.h | 39 +- include/linux/pci-epf.h | 28 +- include/linux/pci_ids.h | 1 + 23 files changed, 3934 insertions(+), 73 deletions(-) create mode 100644 Documentation/PCI/endpoint/function/binding/pci-ntb.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-function.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-howto.rst create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644 drivers/ntb/hw/epf/Makefile create mode 100644 drivers/ntb/hw/epf/ntb_hw_epf.c create mode 100644 drivers/pci/endpoint/functions/pci-epf-ntb.c
Quoted 17 lines · Kishon Vijay Abraham I
Hi,
Quoted 14 lines · Kishon Vijay Abraham I
Hi Jon Mason, Allen Hubbe, Dave Jiang,
Quoted 5 lines · Kishon Vijay Abraham I
This series is about implementing SW defined Non-Transparent Bridge (NTB) using multiple endpoint (EP) instances. This series has been tested using 2 endpoint instances in J7 connected to J7 board on one end and DRA7 board on the other end. However there is nothing platform specific for the NTB functionality.
This series has two patches that adds to
drivers/ntb/directory. [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge and [PATCH v7 16/18] NTB: tool: Enable the NTB/PCIe link on the local or remote side of bridge.
Quoted 6 lines
Quoted 4 lines
If you can review and Ack the above patches, Lorenzo can queue it along with the rest of the series.
Thanks for your help in advance.
Gentle ping on this series.
I am not queueing any more patches for this merge window - we postpone this series to v5.11 and in the interim it would be good to define some possible users.
Thanks,
Lorenzo
Quoted 127 lines
Thanks
KishonQuoted 125 lines
Best Regards,
KishonQuoted 121 lines
This was presented in Linux Plumbers Conference. Link to presentation and video can be found @ [1]
RFC patch series can be found @ [2]
v1 patch series can be found @ [3]
v2 patch series can be found @ [4]
v3 patch series can be found @ [5]
v4 patch series can be found @ [6]
v5 patch series can be found @ [7]
v6 patch series can be found @ [8]Changes from v6:
1) Fixed issues when multiple NTB devices are creating using multiple functions
2) Fixed issue with writing scratchpad register
3) Created a video demo @ [9]Changes from v5:
1) Fixed a formatting issue inKconfigpointed out by Randy
2) Checked for Error or Null inpci_epc_add_epf()Changes from v4:
1) Fixed error condition checks inpci_epc_add_epf()Changes from v3:
1) Fixed Documentation edits suggested by Randy Dunlap <[email protected]>Changes from v2:
1) Add support for the user to create sub-directory of 'EPF Device' directory (for endpoint function specific configuration using configfs).
2) Add documentation for NTB specific attributes in configfs
3) Check forPCI_CLASS_MEMORY_RAM(PCIe class) before bindingntb_hw_epfdriver
4) Other documentation fixesChanges from v1:
1) As per Rob's comment, removed support for creating NTB function device from DT
2) Add support to create NTB EPF device using configfs (added support in configfs to associate primary and secondary EPC with EPF.Changes from RFC:
1) Converted the DT binding patches to YAML schema and merged the DT binding patches together
2) NTB documentation is converted to .rst
3) One HOST can now interrupt the other HOST using MSI-X interrupts
4) Added support for teardown of memory window and doorbell configuration
5) Add support to provide support 64-bit memory window size from DT[1] -> https://linuxplumbersconf.org/event/4/contributions/395/
[2] -> http://lore.kernel.org/r/[email protected]
[3] -> http://lore.kernel.org/r/[email protected]
[4] -> http://lore.kernel.org/r/[email protected]
[5] -> http://lore.kernel.org/r/[email protected]
[6] -> http://lore.kernel.org/r/[email protected]
[7] -> http://lore.kernel.org/r/[email protected]
[8] -> http://lore.kernel.org/r/[email protected]
[9] -> https://youtu.be/dLKKxrg5-rYKishon Vijay Abraham I (18):
Documentation: PCI: Add specification for the *PCI NTB* function
device
PCI: endpoint: Make*_get_first_free_bar()take into account 64 bit
BAR
PCI: endpoint: Add helper API to get the 'next' unreserved BAR
PCI: endpoint: Make*_free_bar()to return error codes on failure
PCI: endpoint: Remove unusedpci_epf_match_device()
PCI: endpoint: Add support to associate secondary EPC with EPF
PCI: endpoint: Add support in configfs to associate two EPCs with EPF
PCI: endpoint: Addpci_epc_opsto map MSI irq
PCI: endpoint: Addpci_epf_opsfor epf drivers to expose function
specific attrs
PCI: endpoint: Allow user to create sub-directory of 'EPF Device'
directory
PCI: cadence: Implement ->msi_map_irq() ops
PCI: cadence: ConfigureLM_EP_FUNC_CFGbased onepc->function_num_map
PCI: endpoint: Add EP function driver to provide NTB functionality
PCI: Add TI J721E device to pci ids
NTB: Add support for EPF PCI-Express Non-Transparent Bridge
NTB: tool: Enable the NTB/PCIe link on the local or remote side of
bridge
Documentation: PCI: Add configfs binding documentation for pci-ntb
endpoint function
Documentation: PCI: Add userguide for PCI endpoint NTB function.../PCI/endpoint/function/binding/pci-ntb.rst | 38 + Documentation/PCI/endpoint/index.rst | 3 + .../PCI/endpoint/pci-endpoint-cfs.rst | 10 + .../PCI/endpoint/pci-ntb-function.rst | 351 +++ Documentation/PCI/endpoint/pci-ntb-howto.rst | 160 ++ drivers/misc/pci_endpoint_test.c | 1 - drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 + drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++ drivers/ntb/test/ntb_tool.c | 1 + .../pci/controller/cadence/pcie-cadence-ep.c | 60 +- drivers/pci/endpoint/functions/Kconfig | 12 + drivers/pci/endpoint/functions/Makefile | 1 + drivers/pci/endpoint/functions/pci-epf-ntb.c | 2114 +++++++++++++++++ drivers/pci/endpoint/functions/pci-epf-test.c | 13 +- drivers/pci/endpoint/pci-ep-cfs.c | 176 +- drivers/pci/endpoint/pci-epc-core.c | 130 +- drivers/pci/endpoint/pci-epf-core.c | 105 +- include/linux/pci-epc.h | 39 +- include/linux/pci-epf.h | 28 +- include/linux/pci_ids.h | 1 + 23 files changed, 3934 insertions(+), 73 deletions(-) create mode 100644 Documentation/PCI/endpoint/function/binding/pci-ntb.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-function.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-howto.rst create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644 drivers/ntb/hw/epf/Makefile create mode 100644 drivers/ntb/hw/epf/ntb_hw_epf.c create mode 100644 drivers/pci/endpoint/functions/pci-epf-ntb.c
+Alan
Hi Jon Mason, Allen Hubbe, Dave Jiang,
Quoted 13 lines · Lorenzo Pieralisi
Quoted 12 lines · Kishon Vijay Abraham I
Hi,
Quoted 9 lines · Kishon Vijay Abraham I
Hi Jon Mason, Allen Hubbe, Dave Jiang,
Quoted 5 lines · Kishon Vijay Abraham I
This series is about implementing SW defined Non-Transparent Bridge (NTB) using multiple endpoint (EP) instances. This series has been tested using 2 endpoint instances in J7 connected to J7 board on one end and DRA7 board on the other end. However there is nothing platform specific for the NTB functionality.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
This series has two patches that adds to
drivers/ntb/directory. [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge and [PATCH v7 16/18] NTB: tool: Enable the NTB/PCIe link on the local or remote side of bridge.If you can review and Ack the above patches, Lorenzo can queue it along with the rest of the series.
Would you be able to review and Ack the NTB parts of this series?
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Thanks for your help in advance.
Gentle ping on this series.
I am not queueing any more patches for this merge window - we postpone this series to v5.11 and in the interim it would be good to define some possible users.
Alan, Do you have a system where you can test this series? It only needs two endpoint instances on a single system.
Thanks
Kishon
Quoted 131 lines
Thanks,
LorenzoQuoted 127 lines
Thanks
KishonQuoted 125 lines
Best Regards,
KishonQuoted 121 lines
This was presented in Linux Plumbers Conference. Link to presentation and video can be found @ [1]
RFC patch series can be found @ [2]
v1 patch series can be found @ [3]
v2 patch series can be found @ [4]
v3 patch series can be found @ [5]
v4 patch series can be found @ [6]
v5 patch series can be found @ [7]
v6 patch series can be found @ [8]Changes from v6:
1) Fixed issues when multiple NTB devices are creating using multiple functions
2) Fixed issue with writing scratchpad register
3) Created a video demo @ [9]Changes from v5:
1) Fixed a formatting issue inKconfigpointed out by Randy
2) Checked for Error or Null inpci_epc_add_epf()Changes from v4:
1) Fixed error condition checks inpci_epc_add_epf()Changes from v3:
1) Fixed Documentation edits suggested by Randy Dunlap <[email protected]>Changes from v2:
1) Add support for the user to create sub-directory of 'EPF Device' directory (for endpoint function specific configuration using configfs).
2) Add documentation for NTB specific attributes in configfs
3) Check forPCI_CLASS_MEMORY_RAM(PCIe class) before bindingntb_hw_epfdriver
4) Other documentation fixesChanges from v1:
1) As per Rob's comment, removed support for creating NTB function device from DT
2) Add support to create NTB EPF device using configfs (added support in configfs to associate primary and secondary EPC with EPF.Changes from RFC:
1) Converted the DT binding patches to YAML schema and merged the DT binding patches together
2) NTB documentation is converted to .rst
3) One HOST can now interrupt the other HOST using MSI-X interrupts
4) Added support for teardown of memory window and doorbell configuration
5) Add support to provide support 64-bit memory window size from DT[1] -> https://linuxplumbersconf.org/event/4/contributions/395/
[2] -> http://lore.kernel.org/r/[email protected]
[3] -> http://lore.kernel.org/r/[email protected]
[4] -> http://lore.kernel.org/r/[email protected]
[5] -> http://lore.kernel.org/r/[email protected]
[6] -> http://lore.kernel.org/r/[email protected]
[7] -> http://lore.kernel.org/r/[email protected]
[8] -> http://lore.kernel.org/r/[email protected]
[9] -> https://youtu.be/dLKKxrg5-rYKishon Vijay Abraham I (18):
Documentation: PCI: Add specification for the *PCI NTB* function
device
PCI: endpoint: Make*_get_first_free_bar()take into account 64 bit
BAR
PCI: endpoint: Add helper API to get the 'next' unreserved BAR
PCI: endpoint: Make*_free_bar()to return error codes on failure
PCI: endpoint: Remove unusedpci_epf_match_device()
PCI: endpoint: Add support to associate secondary EPC with EPF
PCI: endpoint: Add support in configfs to associate two EPCs with EPF
PCI: endpoint: Addpci_epc_opsto map MSI irq
PCI: endpoint: Addpci_epf_opsfor epf drivers to expose function
specific attrs
PCI: endpoint: Allow user to create sub-directory of 'EPF Device'
directory
PCI: cadence: Implement ->msi_map_irq() ops
PCI: cadence: ConfigureLM_EP_FUNC_CFGbased onepc->function_num_map
PCI: endpoint: Add EP function driver to provide NTB functionality
PCI: Add TI J721E device to pci ids
NTB: Add support for EPF PCI-Express Non-Transparent Bridge
NTB: tool: Enable the NTB/PCIe link on the local or remote side of
bridge
Documentation: PCI: Add configfs binding documentation for pci-ntb
endpoint function
Documentation: PCI: Add userguide for PCI endpoint NTB function.../PCI/endpoint/function/binding/pci-ntb.rst | 38 + Documentation/PCI/endpoint/index.rst | 3 + .../PCI/endpoint/pci-endpoint-cfs.rst | 10 + .../PCI/endpoint/pci-ntb-function.rst | 351 +++ Documentation/PCI/endpoint/pci-ntb-howto.rst | 160 ++ drivers/misc/pci_endpoint_test.c | 1 - drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 + drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++ drivers/ntb/test/ntb_tool.c | 1 + .../pci/controller/cadence/pcie-cadence-ep.c | 60 +- drivers/pci/endpoint/functions/Kconfig | 12 + drivers/pci/endpoint/functions/Makefile | 1 + drivers/pci/endpoint/functions/pci-epf-ntb.c | 2114 +++++++++++++++++ drivers/pci/endpoint/functions/pci-epf-test.c | 13 +- drivers/pci/endpoint/pci-ep-cfs.c | 176 +- drivers/pci/endpoint/pci-epc-core.c | 130 +- drivers/pci/endpoint/pci-epf-core.c | 105 +- include/linux/pci-epc.h | 39 +- include/linux/pci-epf.h | 28 +- include/linux/pci_ids.h | 1 + 23 files changed, 3934 insertions(+), 73 deletions(-) create mode 100644 Documentation/PCI/endpoint/function/binding/pci-ntb.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-function.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-howto.rst create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644 drivers/ntb/hw/epf/Makefile create mode 100644 drivers/ntb/hw/epf/ntb_hw_epf.c create mode 100644 drivers/pci/endpoint/functions/pci-epf-ntb.c
Quoted 40 lines · Kishon Vijay Abraham I
+Alan
Hi Jon Mason, Allen Hubbe, Dave Jiang,
Quoted 8 lines · Lorenzo Pieralisi
Quoted 7 lines · Kishon Vijay Abraham I
Hi,
Quoted 4 lines · Kishon Vijay Abraham I
Hi Jon Mason, Allen Hubbe, Dave Jiang,
Quoted 1 line · Kishon Vijay Abraham I
This series is about implementing SW defined Non-Transparent Bridge
(NTB)
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
using multiple endpoint (EP) instances. This series has been tested
using
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
2 endpoint instances in J7 connected to J7 board on one end and DRA7
board
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
on the other end. However there is nothing platform specific for the
NTB
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
functionality.
This series has two patches that adds to
drivers/ntb/directory. [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non-Transparent Bridge and [PATCH v7 16/18] NTB: tool: Enable the NTB/PCIe link on the local or remote side of bridge.If you can review and Ack the above patches, Lorenzo can queue it along with the rest of the series.
Would you be able to review and Ack the NTB parts of this series?
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Thanks for your help in advance.
Gentle ping on this series.
I am not queueing any more patches for this merge window - we postpone this series to v5.11 and in the interim it would be good to define some possible users.
Quoted 6 lines
Alan, Do you have a system where you can test this series? It only needs two endpoint instances on a single system.
Thanks
Kishon
Kishon, I have no suitable systems. I wonder if two Nvidia Jetson Xavier NX boards can be connected back to back by ethernet and plugged into x86 system pcie slots for this. I believe they have the dwc pcie controller with driver support for endpoint mode but I personally don't have any. The FriendlyElec NanoPi M4 boards also supports endpoint mode with the Rockchip RK3399 pcie controller but I have not found the endpoint driver suppor for that.
Regards, Alan
Quoted 139 lines
Quoted 37 lines
Thanks,
LorenzoQuoted 33 lines
Thanks
KishonQuoted 31 lines
Best Regards,
KishonQuoted 27 lines
This was presented in Linux Plumbers Conference. Link to presentation and video can be found @ [1]
RFC patch series can be found @ [2]
v1 patch series can be found @ [3]
v2 patch series can be found @ [4]
v3 patch series can be found @ [5]
v4 patch series can be found @ [6]
v5 patch series can be found @ [7]
v6 patch series can be found @ [8]Changes from v6:
1) Fixed issues when multiple NTB devices are creating using multiple functions
2) Fixed issue with writing scratchpad register
3) Created a video demo @ [9]Changes from v5:
1) Fixed a formatting issue inKconfigpointed out by Randy
2) Checked for Error or Null inpci_epc_add_epf()Changes from v4:
1) Fixed error condition checks inpci_epc_add_epf()Changes from v3:
1) Fixed Documentation edits suggested by Randy Dunlap <Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Changes from v2:
1) Add support for the user to create sub-directory of 'EPF Device' directory (for endpoint function specific configuration using configfs).
2) Add documentation for NTB specific attributes in configfs
3) Check forPCI_CLASS_MEMORY_RAM(PCIe class) before binding
ntb_hw_epfQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
driver
4) Other documentation fixesChanges from v1:
1) As per Rob's comment, removed support for creating NTB function device from DT
2) Add support to create NTB EPF device using configfs (added supportin
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
configfs to associate primary and secondary EPC with EPF.
Changes from RFC:
1) Converted the DT binding patches to YAML schema and merged the DT binding patches together
2) NTB documentation is converted to .rst
3) One HOST can now interrupt the other HOST using MSI-X interrupts
4) Added support for teardown of memory window and doorbell configuration
5) Add support to provide support 64-bit memory window size from DT[1] -> https://linuxplumbersconf.org/event/4/contributions/395/
[2] -> http://lore.kernel.org/r/[email protected]
[3] -> http://lore.kernel.org/r/[email protected]
[4] -> http://lore.kernel.org/r/[email protected]
[5] -> http://lore.kernel.org/r/[email protected]
[6] -> http://lore.kernel.org/r/[email protected]
[7] -> http://lore.kernel.org/r/[email protected]
[8] -> http://lore.kernel.org/r/[email protected]
[9] -> https://youtu.be/dLKKxrg5-rYQuoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Kishon Vijay Abraham I (18):
Documentation: PCI: Add specification for the *PCI NTB* function
device
PCI: endpoint: Make*_get_first_free_bar()take into account 64 bit
BAR
PCI: endpoint: Add helper API to get the 'next' unreserved BAR
PCI: endpoint: Make*_free_bar()to return error codes on failure
PCI: endpoint: Remove unusedpci_epf_match_device()
PCI: endpoint: Add support to associate secondary EPC with EPF
PCI: endpoint: Add support in configfs to associate two EPCs withEPF
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
PCI: endpoint: Add
pci_epc_opsto map MSI irq
PCI: endpoint: Addpci_epf_opsfor epf drivers to expose function
specific attrs
PCI: endpoint: Allow user to create sub-directory of 'EPF Device'
directory
PCI: cadence: Implement ->msi_map_irq() ops
PCI: cadence: ConfigureLM_EP_FUNC_CFGbased on
epc->function_num_mapQuoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
PCI: endpoint: Add EP function driver to provide NTB functionality
PCI: Add TI J721E device to pci ids
NTB: Add support for EPF PCI-Express Non-Transparent Bridge
NTB: tool: Enable the NTB/PCIe link on the local or remote side of
bridge
Documentation: PCI: Add configfs binding documentation for pci-ntb
endpoint function
Documentation: PCI: Add userguide for PCI endpoint NTB functionQuoted 16 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
.../PCI/endpoint/function/binding/pci-ntb.rst | 38 + Documentation/PCI/endpoint/index.rst | 3 + .../PCI/endpoint/pci-endpoint-cfs.rst | 10 + .../PCI/endpoint/pci-ntb-function.rst | 351 +++ Documentation/PCI/endpoint/pci-ntb-howto.rst | 160 ++ drivers/misc/pci_endpoint_test.c | 1 - drivers/ntb/hw/Kconfig | 1 + drivers/ntb/hw/Makefile | 1 +Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/ntb/hw/epf/Kconfig | 6 + drivers/ntb/hw/epf/Makefile | 1 + drivers/ntb/hw/epf/ntb_hw_epf.c | 755 ++++++ drivers/ntb/test/ntb_tool.c | 1 + .../pci/controller/cadence/pcie-cadence-ep.c | 60 +- drivers/pci/endpoint/functions/Kconfig | 12 + drivers/pci/endpoint/functions/Makefile | 1 + drivers/pci/endpoint/functions/pci-epf-ntb.c | 2114+++++++++++++++++
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
drivers/pci/endpoint/functions/pci-epf-test.c | 13 +- drivers/pci/endpoint/pci-ep-cfs.c | 176 +- drivers/pci/endpoint/pci-epc-core.c | 130 +- drivers/pci/endpoint/pci-epf-core.c | 105 +- include/linux/pci-epc.h | 39 +- include/linux/pci-epf.h | 28 +- include/linux/pci_ids.h | 1 + 23 files changed, 3934 insertions(+), 73 deletions(-) create mode 100644
Documentation/PCI/endpoint/function/binding/pci-ntb.rstQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
create mode 100644 Documentation/PCI/endpoint/pci-ntb-function.rst create mode 100644 Documentation/PCI/endpoint/pci-ntb-howto.rst create mode 100644 drivers/ntb/hw/epf/Kconfig create mode 100644 drivers/ntb/hw/epf/Makefile create mode 100644 drivers/ntb/hw/epf/ntb_hw_epf.c create mode 100644 drivers/pci/endpoint/functions/pci-epf-ntb.c