IB/mad: Add support for Intel Omni-Path Architecture (OPA) MAD processing.
Subsystems (7)
v4 cover IB/mad: Add support for Intel Omni-Path Architecture (OPA) MAD processing.
From: Ira Weiny <[email protected]>
The following patch series modifies the kernel MAD processing (ib_mad/ib_umad) and related interfaces to send and receive Intel Omni-Path Architecture MADs on devices which support them.
In addition to supporting some IBTA management classes, OPA devices use MADs with lengths up to 2K. These "jumbo" MADs increase the performance of management traffic.
To distinguish IBTA MADs from OPA MADs a new Base Version is introduced. The new format shares the same common header with IBTA MADs which allows us to share most of the MAD processing code when dealing with the new Base Version.
The patch series is broken into 3 main areas.
1) Add the ability for devices to indicate MAD size. modify the MAD code to use this MAD size
2) Enhance the interface to the device agents to support larger and variable length MADs.
3) Add capability bit to indicate support for OPA MADs
4) Add support for creating and processing OPA MADs
Changes for V4:
Rebased to latest Rolands for-next branch (3.19-rc4) Fixed compile issue in ehca driver found with 0-day build.
Ira Weiny (19):
IB/mad: Rename is_data_mad to is_rmpp_data_mad
IB/core: Cache device attributes for use by upper level drivers IB/mad: Change validate_mad signature to take ib_mad_hdr rather than
ib_mad
IB/mad: Change ib_response_mad signature to take ib_mad_hdr rather
than ib_mad
IB/mad: Change cast in rcv_has_same_class
IB/core: Add max_mad_size to ib_device_attr
IB/mad: Convert ib_mad_private allocations from kmem_cache to kmalloc IB/mad: Add helper function for smi_handle_dr_smp_send IB/mad: Add helper function for smi_handle_dr_smp_recv IB/mad: Add helper function for smi_check_forward_dr_smp IB/mad: Add helper function for SMI processing
IB/mad: Add MAD size parameters to process_mad
IB/mad: Add base version parameter to ib_create_send_mad IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag IB/mad: Create jumbo_mad data structures
IB/mad: Add Intel Omni-Path Architecture defines
IB/mad: Implement support for Intel Omni-Path Architecture base
version MADs in ib_create_send_mad
IB/mad: Implement Intel Omni-Path Architecture SMP processing IB/mad: Implement Intel Omni-Path Architecture MAD processing
drivers/infiniband/core/agent.c | 26 +- drivers/infiniband/core/agent.h | 3 +- drivers/infiniband/core/cm.c | 6 +- drivers/infiniband/core/device.c | 2 + drivers/infiniband/core/mad.c | 519 ++++++++++++++++++--------- drivers/infiniband/core/mad_priv.h | 7 +- drivers/infiniband/core/mad_rmpp.c | 144 ++++---- drivers/infiniband/core/opa_smi.h | 78 ++++ drivers/infiniband/core/sa_query.c | 3 +- drivers/infiniband/core/smi.c | 231 ++++++++---- drivers/infiniband/core/smi.h | 6 + drivers/infiniband/core/sysfs.c | 5 +- drivers/infiniband/core/user_mad.c | 38 +- drivers/infiniband/hw/amso1100/c2_provider.c | 5 +- drivers/infiniband/hw/amso1100/c2_rnic.c | 1 + drivers/infiniband/hw/cxgb3/iwch_provider.c | 6 +- drivers/infiniband/hw/cxgb4/provider.c | 8 +- drivers/infiniband/hw/ehca/ehca_hca.c | 3 + drivers/infiniband/hw/ehca/ehca_iverbs.h | 4 +- drivers/infiniband/hw/ehca/ehca_sqp.c | 8 +- drivers/infiniband/hw/ipath/ipath_mad.c | 8 +- drivers/infiniband/hw/ipath/ipath_verbs.c | 1 + drivers/infiniband/hw/ipath/ipath_verbs.h | 3 +- drivers/infiniband/hw/mlx4/mad.c | 12 +- drivers/infiniband/hw/mlx4/main.c | 1 + drivers/infiniband/hw/mlx4/mlx4_ib.h | 3 +- drivers/infiniband/hw/mlx5/mad.c | 8 +- drivers/infiniband/hw/mlx5/main.c | 1 + drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +- drivers/infiniband/hw/mthca/mthca_dev.h | 4 +- drivers/infiniband/hw/mthca/mthca_mad.c | 12 +- drivers/infiniband/hw/mthca/mthca_provider.c | 2 + drivers/infiniband/hw/nes/nes_verbs.c | 4 +- drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 3 +- drivers/infiniband/hw/ocrdma/ocrdma_ah.h | 3 +- drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 1 + drivers/infiniband/hw/qib/qib_iba7322.c | 3 +- drivers/infiniband/hw/qib/qib_mad.c | 11 +- drivers/infiniband/hw/qib/qib_verbs.c | 1 + drivers/infiniband/hw/qib/qib_verbs.h | 3 +- drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 2 + drivers/infiniband/ulp/srpt/ib_srpt.c | 3 +- include/rdma/ib_mad.h | 40 ++- include/rdma/ib_verbs.h | 15 +- include/rdma/opa_smi.h | 106 ++++++ 45 files changed, 999 insertions(+), 357 deletions(-) create mode 100644 drivers/infiniband/core/opa_smi.h create mode 100644 include/rdma/opa_smi.h
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-1-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
PATCH v4 1/19 IB/mad: Rename is_data_mad to is_rmpp_data_mad
From: Ira Weiny <[email protected]>
is_rmpp_data_mad is more descriptive for this function.
--- drivers/infiniband/core/mad.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
drivers/infiniband/core/mad.c +4 -3
@@ -1734,7 +1734,7 @@ out:return valid; } -static int is_data_mad (struct ib_mad_agent_private *mad_agent_priv, +static int is_rmpp_data_mad (struct ib_mad_agent_private *mad_agent_priv,struct ib_mad_hdr *mad_hdr) {struct ib_rmpp_mad *rmpp_mad;
@@ -1836,7 +1836,7 @@ ib_find_send_mad(struct ib_mad_agent_private *mad_agent_priv,* been notified that the send has completed */ list_for_each_entry (wr, &mad_agent_priv->send_list, agent_list) { - if (is_data_mad(mad_agent_priv, wr->send_buf.mad) && + if (is_rmpp_data_mad(mad_agent_priv, wr->send_buf.mad) && wr->tid == mad->mad_hdr.tid && wr->timeout && rcv_has_same_class(wr, wc) &&
@@ -2411,7 +2411,8 @@ find_send_wr(struct ib_mad_agent_private *mad_agent_priv,list_for_each_entry (mad_send_wr, &mad_agent_priv->send_list, agent_list) { -if (is_data_mad (mad_agent_priv, mad_send_wr->send_buf.mad) && +if (is_rmpp_data_mad (mad_agent_priv, + mad_send_wr->send_buf.mad) && &mad_send_wr->send_buf == send_buf)return mad_send_wr; }
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 2/19 IB/core: Cache device attributes for use by upper level drivers
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
Upper level drivers can access these cached device attributes rather than caching them on their own.
--- drivers/infiniband/core/device.c | 2 ++ include/rdma/ib_verbs.h | 1 + 2 files changed, 3 insertions(+)
drivers/infiniband/core/device.c +2 -0
@@ -294,6 +294,8 @@ int ib_register_device(struct ib_device *device,spin_lock_init (&device->event_handler_lock);spin_lock_init (&device->client_data_lock); + device->query_device (device, &device->cached_dev_attrs); + ret =read_port_table_lengths (device);if (ret) { printk(KERN_WARNING "Couldn't create table lengths cache for device %s\n",
include/rdma/ib_verbs.h +1 -0
@@ -1675,6 +1675,7 @@ struct ib_device {u32 local_dma_lkey;u8 node_type;u8 phys_port_cnt; +struct ib_device_attr cached_dev_attrs; };struct ib_client {
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-3-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
RE: [PATCH v4 02/19] IB/core: Cache device attributes for use by upper level drivers
include/rdma/ib_verbs.h struct ib_device { · line 1679
Quoted 10 lines
include/rdma/ib_verbs.h +1
-0@@ -1675,6 +1675,7 @@ struct ib_device {u32 local_dma_lkey;u8 node_type;u8 phys_port_cnt; +struct ib_device_attr cached_dev_attrs; };
Looking at the device attributes, I think all of the values are static for a given device. If this is indeed the case, then I would just remove the word 'cached' from the field name. Cached makes me think of the values dynamically changing, and if that's the case, then this patch isn't sufficient.
Alternatively, if there's only a few values that ULPs need, maybe just store those. --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBD574-P5GAC/[email protected]
Re: [PATCH v4 02/19] IB/core: Cache device attributes for use by upper level drivers
Quoted 4 lines · Sean
Quoted 4 lines
include/rdma/ib_verbs.h +0
-0
Quoted 9 lines
include/rdma/ib_verbs.h struct ib_device { · line 1679
Quoted 6 lines
include/rdma/ib_verbs.h +1
-0@@ -1675,6 +1675,7 @@ struct ib_device {u32 local_dma_lkey;u8 node_type;u8 phys_port_cnt; +struct ib_device_attr cached_dev_attrs; };Looking at the device attributes, I think all of the values are static for a given device.
Yes
Quoted 2 lines
If this is indeed the case, then I would just remove the word 'cached' from the field name. Cached makes me think of the values dynamically changing, and if that's the case, then this patch isn't sufficient.
I understand your point and I originally called this "attributes" but it was suggested to call it cached_dev_attrs.
https://www.mail-archive.com/linux-rdma%40vger.kernel.org/msg22486.html
I can change it back if we are all agreed.
Roland?
Quoted 1 line
Alternatively, if there's only a few values that ULPs need, maybe just store those.
This patch was suggested because there are too many of the flags needed by ULPs.
A quote from Or:
"I find it very annoying that upper level drivers replicate in different ways elements from the IB device attributes returned by ib_query_device. I met that in multiple drivers and upcoming designs for which I do code review. Are you up to come up with a patch that caches the device attributes on the device structure? if not, I can do that.. and have your code to see it."
https://www.mail-archive.com/linux-rdma%40vger.kernel.org/msg22011.html
Roland is having such a patch even acceptable or would you prefer to have a query_device call by each ULP?
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20150406221044.GA433-W4f6Xiosr+yv7QzWx2u06xL4W9x8LtSr@public.gmane.org
RE: [PATCH v4 02/19] IB/core: Cache device attributes for use by upper level drivers
Quoted 7 lines
I understand your point and I originally called this "attributes" but it was
suggested to call itcached_dev_attrs.https://www.mail-archive.com/linux-rdma%40vger.kernel.org/msg22486.html
I can change it back if we are all agreed.
I'll disagree with Or on this. Unless some of these values can change dynamically, these are the attributes.
This leads me to, is there a reason to keep the per device query_attribute() call? (This would not be part of this series.) Are there attributes being returned from the kernel to user space that fall outside of the defined attribute area?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBDE3E-P5GAC/[email protected]
Re: [PATCH v4 02/19] IB/core: Cache device attributes for use by upper level drivers
Quoted 9 lines · Sean
Quoted 7 lines
I understand your point and I originally called this "attributes" but it was
suggested to call itcached_dev_attrs.https://www.mail-archive.com/linux-rdma%40vger.kernel.org/msg22486.html
I can change it back if we are all agreed.
I'll disagree with Or on this. Unless some of these values can change dynamically, these are the attributes.
Or
Any comment?
Personally, this patch was not originally part of my series. I had a ib_query_device call which Or objected to "yet another caching" of a device attribute.
With the discussions on "management helpers" I probably don't need this at all. So, if I can remove it I will and then we can debate its merits separate from the OPA changes.
Ira
Quoted 3 lines
This leads me to, is there a reason to keep the per device
query_attribute()call? (This would not be part of this series.) Are there attributes being returned from the kernel to user space that fall outside of the defined attribute area?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 3/19 IB/mad: Change validate_mad signature to take ib_mad_hdr rather than ib_mad
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
validate_mad only needs access to the MAD header and can be used for both IB and Jumbo MADs.
--- drivers/infiniband/core/mad.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/infiniband/core/mad.c +6 -6
@@ -1708,20 +1708,20 @@ out:return mad_agent; } -static int validate_mad (struct ib_mad *mad,u32 qp_num) +static int validate_mad (struct ib_mad_hdr *mad_hdr,u32 qp_num) {int valid =0 ;/* Make sure MAD base version is understood */ -if (mad->mad_hdr.base_version !=IB_MGMT_BASE_VERSION ) { +if (mad_hdr->base_version !=IB_MGMT_BASE_VERSION ) {pr_err ("MAD received with unsupported base version %d\n" , - mad->mad_hdr.base_version); + mad_hdr->base_version); goto out; }/* Filter SMI packets sent to other than QP0 */ -if ((mad->mad_hdr.mgmt_class ==IB_MGMT_CLASS_SUBN_LID_ROUTED ) || - (mad->mad_hdr.mgmt_class ==IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE )) { +if ((mad_hdr->mgmt_class ==IB_MGMT_CLASS_SUBN_LID_ROUTED ) || + (mad_hdr->mgmt_class ==IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE )) {if (qp_num ==0 ) valid =1 ; }else {
@@ -1979,7 +1979,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv,snoop_recv (qp_info, &recv->header.recv_wc,IB_MAD_SNOOP_RECVS );/* Validate MAD */ -if (!validate_mad (&recv->mad.mad, qp_info->qp->qp_num)) +if (!validate_mad (&recv->mad.mad.mad_hdr, qp_info->qp->qp_num)) goto out; response =kmem_cache_alloc (ib_mad_cache,GFP_KERNEL );
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-4-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Hefty, Sean
Reviewed-byRE: [PATCH v4 03/19] IB/mad: Change validate_mad signature to take ib_mad_hdr rather than ib_mad
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 4/19 IB/mad: Change ib_response_mad signature to take ib_mad_hdr rather than ib_mad
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
ib_response_mad only needs access to the MAD header and can be used for both IB and Jumbo MADs.
--- drivers/infiniband/core/mad.c | 20 ++++++++++---------- drivers/infiniband/core/user_mad.c | 6 +++--- include/rdma/ib_mad.h | 2 +- 3 files changed, 14 insertions(+), 14 deletions(-)
drivers/infiniband/core/mad.c +10 -10
@@ -179,12 +179,12 @@ static int is_vendor_method_in_use(return 0 ; } -int ib_response_mad (struct ib_mad *mad) +int ib_response_mad (struct ib_mad_hdr *hdr) { -return ((mad->mad_hdr.method &IB_MGMT_METHOD_RESP ) || - (mad->mad_hdr.method ==IB_MGMT_METHOD_TRAP_REPRESS ) || - ((mad->mad_hdr.mgmt_class ==IB_MGMT_CLASS_BM ) && - (mad->mad_hdr.attr_mod &IB_BM_ATTR_MOD_RESP ))); +return ((hdr->method &IB_MGMT_METHOD_RESP ) || + (hdr->method ==IB_MGMT_METHOD_TRAP_REPRESS ) || + ((hdr->mgmt_class ==IB_MGMT_CLASS_BM ) && + (hdr->attr_mod &IB_BM_ATTR_MOD_RESP ))); }EXPORT_SYMBOL (ib_response_mad);
@@ -791,7 +791,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,switch (ret) {case IB_MAD_RESULT_SUCCESS |IB_MAD_RESULT_REPLY : -if (ib_response_mad (&mad_priv->mad.mad) && +if (ib_response_mad (&mad_priv->mad.mad.mad_hdr) && mad_agent_priv->agent.recv_handler) { local->mad_priv = mad_priv; local->recv_mad_agent = mad_agent_priv;
@@ -1628,7 +1628,7 @@ find_mad_agent(struct ib_mad_port_private *port_priv,unsigned long flags;spin_lock_irqsave (&port_priv->reg_lock, flags); -if (ib_response_mad (mad)) { +if (ib_response_mad (&mad->mad_hdr)) {u32 hi_tid;struct ib_mad_agent_private *entry;
@@ -1765,8 +1765,8 @@ static inline int rcv_has_same_gid(struct ib_mad_agent_private *mad_agent_priv,u8 port_num = mad_agent_priv->agent.port_num;u8 lmc; - send_resp =ib_response_mad ((struct ib_mad *)wr->send_buf.mad); - rcv_resp =ib_response_mad (rwc->recv_buf.mad); + send_resp =ib_response_mad ((struct ib_mad_hdr *)wr->send_buf.mad); + rcv_resp =ib_response_mad (&rwc->recv_buf.mad->mad_hdr);if (send_resp == rcv_resp)/* both requests, or both responses. GIDs different */
@@ -1879,7 +1879,7 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, }/* Complete corresponding request */ -if (ib_response_mad (mad_recv_wc->recv_buf.mad)) { +if (ib_response_mad (&mad_recv_wc->recv_buf.mad->mad_hdr)) {spin_lock_irqsave (&mad_agent_priv->lock, flags); mad_send_wr =ib_find_send_mad (mad_agent_priv, mad_recv_wc);if (!mad_send_wr) {
drivers/infiniband/core/user_mad.c +3 -3
@@ -426,11 +426,11 @@ static int is_duplicate(struct ib_umad_file *file,* the same TID, reject the second as a duplicate. This is more * restrictive than required by the spec. */ -if (!ib_response_mad ((struct ib_mad *) hdr)) { -if (!ib_response_mad ((struct ib_mad *) sent_hdr)) +if (!ib_response_mad (hdr)) { +if (!ib_response_mad (sent_hdr))return 1 ;continue ; - }else if (!ib_response_mad ((struct ib_mad *) sent_hdr)) + }else if (!ib_response_mad (sent_hdr))continue ;if (same_destination (&packet->mad.hdr, &sent_packet->mad.hdr))
include/rdma/ib_mad.h +1 -1
@@ -263,7 +263,7 @@ struct ib_mad_send_buf {* ib_response_mad - Returns if the specified MAD has been generated in * response to a sent request or trap. */ -int ib_response_mad (struct ib_mad *mad); +int ib_response_mad (struct ib_mad_hdr *hdr);/** * ib_get_rmpp_resptime - Returns the RMPP response time.
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-5-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Hefty, Sean
Reviewed-byRE: [PATCH v4 04/19] IB/mad: Change ib_response_mad signature to take ib_mad_hdr rather than ib_mad
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 5/19 IB/mad: Change cast in rcv_has_same_class
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
rcv_has_same_class only needs access to the MAD header and can be used for both IB and Jumbo MADs.
--- drivers/infiniband/core/mad.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/core/mad.c +1 -1
@@ -1750,7 +1750,7 @@ static int is_rmpp_data_mad(struct ib_mad_agent_private *mad_agent_priv,static inline int rcv_has_same_class (struct ib_mad_send_wr_private *wr,struct ib_mad_recv_wc *rwc) { -return ((struct ib_mad *)(wr->send_buf.mad))->mad_hdr.mgmt_class == +return ((struct ib_mad_hdr *)(wr->send_buf.mad))->mgmt_class == rwc->recv_buf.mad->mad_hdr.mgmt_class; }
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 6/19 IB/core: Add max_mad_size to ib_device_attr
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
Change all IB drivers to report the max MAD size.
Add check to verify that all devices support at least IB_MGMT_MAD_SIZE
---
Changes from V3:
Fix ehca compile found with 0-day build
drivers/infiniband/core/mad.c | 6 ++++++ drivers/infiniband/hw/amso1100/c2_rnic.c | 1 + drivers/infiniband/hw/cxgb3/iwch_provider.c | 1 + drivers/infiniband/hw/cxgb4/provider.c | 1 + drivers/infiniband/hw/ehca/ehca_hca.c | 3 +++ drivers/infiniband/hw/ipath/ipath_verbs.c | 1 + drivers/infiniband/hw/mlx4/main.c | 1 + drivers/infiniband/hw/mlx5/main.c | 1 + drivers/infiniband/hw/mthca/mthca_provider.c | 2 ++ drivers/infiniband/hw/nes/nes_verbs.c | 1 + drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 1 + drivers/infiniband/hw/qib/qib_verbs.c | 1 + drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 2 ++ include/rdma/ib_mad.h | 1 + include/rdma/ib_verbs.h | 1 + 15 files changed, 24 insertions(+)
drivers/infiniband/core/mad.c +6 -0
@@ -2924,6 +2924,12 @@ static int ib_mad_port_open(struct ib_device *device,char name[sizeof "ib_mad123" ];int has_smi; +if (device->cached_dev_attrs.max_mad_size <IB_MGMT_MAD_SIZE ) { +dev_err (&device->dev,"Min MAD size for device is %u\n" , +IB_MGMT_MAD_SIZE ); +return -EFAULT ; + } +/* Create new device info */ port_priv =kzalloc (sizeof *port_priv,GFP_KERNEL );if (!port_priv) {
drivers/infiniband/hw/amso1100/c2_rnic.c +1 -0
@@ -197,6 +197,7 @@ static int c2_rnic_query(struct c2_dev *c2dev, struct ib_device_attr *props) props->max_srq_sge =0 ; props->max_pkeys =0 ; props->local_ca_ack_delay =0 ; + props->max_mad_size =IB_MGMT_MAD_SIZE ; bail2:vq_repbuf_free (c2dev, reply);
drivers/infiniband/hw/cxgb3/iwch_provider.c +1 -0
@@ -1174,6 +1174,7 @@ static int iwch_query_device(struct ib_device *ibdev, props->max_pd = dev->attr.max_pds; props->local_ca_ack_delay =0 ; props->max_fast_reg_page_list_len =T3_MAX_FASTREG_DEPTH ; + props->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }
drivers/infiniband/hw/cxgb4/provider.c +1 -0
@@ -332,6 +332,7 @@ static int c4iw_query_device(struct ib_device *ibdev, props->max_pd =T4_MAX_NUM_PD ; props->local_ca_ack_delay =0 ; props->max_fast_reg_page_list_len =t4_max_fr_depth (use_dsgl); + props->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }
drivers/infiniband/hw/ehca/ehca_hca.c +3 -0
@@ -40,6 +40,7 @@*/ #include <linux/gfp.h> +#include <rdma/ib_mad.h> #include "ehca_tools.h" #include "ehca_iverbs.h"
@@ -133,6 +134,8 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props)if (rblock->hca_cap_indicators & cap_mapping[i +1 ]) props->device_cap_flags |= cap_mapping[i]; + props->max_mad_size =IB_MGMT_MAD_SIZE ; + query_device1:ehca_free_fw_ctrlblock (rblock);
drivers/infiniband/hw/ipath/ipath_verbs.c +1 -0
@@ -1538,6 +1538,7 @@ static int ipath_query_device(struct ib_device *ibdev, props->max_mcast_qp_attach = ib_ipath_max_mcast_qp_attached; props->max_total_mcast_qp_attach = props->max_mcast_qp_attach * props->max_mcast_grp; + props->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }
drivers/infiniband/hw/mlx4/main.c +1 -0
@@ -229,6 +229,7 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->max_total_mcast_qp_attach = props->max_mcast_qp_attach * props->max_mcast_grp; props->max_map_per_fmr = dev->dev->caps.max_fmr_maps; + props->max_mad_size =IB_MGMT_MAD_SIZE ; out:kfree (in_mad);
drivers/infiniband/hw/mlx5/main.c +1 -0
@@ -243,6 +243,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev, props->max_total_mcast_qp_attach = props->max_mcast_qp_attach * props->max_mcast_grp; props->max_map_per_fmr =INT_MAX ;/* no limit in ConnectIB */ + props->max_mad_size =IB_MGMT_MAD_SIZE ;#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING if (dev->mdev->caps.gen.flags &MLX5_DEV_CAP_FLAG_ON_DMND_PG )
drivers/infiniband/hw/mthca/mthca_provider.c +2 -0
@@ -123,6 +123,8 @@ static int mthca_query_device(struct ib_device *ibdev, props->max_map_per_fmr = (1 << (32 -ilog2 (mdev->limits.num_mpts))) -1 ; + props->max_mad_size =IB_MGMT_MAD_SIZE ; + err =0 ; out:kfree (in_mad);
drivers/infiniband/hw/nes/nes_verbs.c +1 -0
@@ -555,6 +555,7 @@ static int nes_query_device(struct ib_device *ibdev, struct ib_device_attr *prop props->max_qp_init_rd_atom = props->max_qp_rd_atom; props->atomic_cap =IB_ATOMIC_NONE ; props->max_map_per_fmr =1 ; + props->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c +1 -0
@@ -103,6 +103,7 @@ int ocrdma_query_device(struct ib_device *ibdev, struct ib_device_attr *attr) attr->local_ca_ack_delay = dev->attr.local_ca_ack_delay; attr->max_fast_reg_page_list_len = dev->attr.max_pages_per_frmr; attr->max_pkeys =1 ; + attr->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }
drivers/infiniband/hw/qib/qib_verbs.c +1 -0
@@ -1591,6 +1591,7 @@ static int qib_query_device(struct ib_device *ibdev, props->max_mcast_qp_attach = ib_qib_max_mcast_qp_attached; props->max_total_mcast_qp_attach = props->max_mcast_qp_attach * props->max_mcast_grp; + props->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }
drivers/infiniband/hw/usnic/usnic_ib_verbs.c +2 -0
@@ -22,6 +22,7 @@#include <rdma/ib_user_verbs.h> #include <rdma/ib_addr.h> +#include <rdma/ib_mad.h> #include "usnic_abi.h" #include "usnic_ib.h"
@@ -296,6 +297,7 @@ int usnic_ib_query_device(struct ib_device *ibdev, props->max_mcast_qp_attach =0 ; props->max_total_mcast_qp_attach =0 ; props->max_map_per_fmr =0 ; + props->max_mad_size =IB_MGMT_MAD_SIZE ;/* Owned by Userspace * max_qp_wr, max_sge, max_sge_rd, max_cqe */ mutex_unlock (&us_ibdev->usdev_lock);
include/rdma/ib_mad.h +1 -0
@@ -135,6 +135,7 @@ enum {IB_MGMT_SA_DATA =200 ,IB_MGMT_DEVICE_HDR =64 ,IB_MGMT_DEVICE_DATA =192 , + IB_MGMT_MAD_SIZE = IB_MGMT_MAD_HDR + IB_MGMT_MAD_DATA, }; struct ib_mad_hdr {
include/rdma/ib_verbs.h +1 -0
@@ -210,6 +210,7 @@ struct ib_device_attr {int sig_prot_cap;int sig_guard_cap;struct ib_odp_caps odp_caps; +u32 max_mad_size; };enum ib_mtu {
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-7-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v4 06/19] IB/core: Add max_mad_size to ib_device_attr
Quoted 34 lines · On Wed, 2015-02-04 at 18:29 -0500, [email protected]…
From: Ira Weiny <[email protected]>
Change all IB drivers to report the max MAD size.
Add check to verify that all devices support at leastIB_MGMT_MAD_SIZESigned-off-by Ira Weiny <[email protected]>---
Changes from V3:
Fix ehca compile found with 0-day builddrivers/infiniband/core/mad.c | 6 ++++++ drivers/infiniband/hw/amso1100/c2_rnic.c | 1 + drivers/infiniband/hw/cxgb3/iwch_provider.c | 1 + drivers/infiniband/hw/cxgb4/provider.c | 1 + drivers/infiniband/hw/ehca/ehca_hca.c | 3 +++ drivers/infiniband/hw/ipath/ipath_verbs.c | 1 + drivers/infiniband/hw/mlx4/main.c | 1 + drivers/infiniband/hw/mlx5/main.c | 1 + drivers/infiniband/hw/mthca/mthca_provider.c | 2 ++ drivers/infiniband/hw/nes/nes_verbs.c | 1 + drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 1 + drivers/infiniband/hw/qib/qib_verbs.c | 1 + drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 2 ++ include/rdma/ib_mad.h | 1 + include/rdma/ib_verbs.h | 1 + 15 files changed, 24 insertions(+)drivers/infiniband/core/mad.c +0
-0
drivers/infiniband/core/mad.c static int ib_mad_port_open(struct ib_device *device, · line 2932
Quoted 9 lines
drivers/infiniband/core/mad.c +6
-0@@ -2924,6 +2924,12 @@ static int ib_mad_port_open(struct ib_device *device,char name[sizeof "ib_mad123" ];int has_smi; +if (device->cached_dev_attrs.max_mad_size <IB_MGMT_MAD_SIZE ) { +dev_err (&device->dev,"Min MAD size for device is %u\n" , +IB_MGMT_MAD_SIZE ); +return -EFAULT ; + } +
The printk message here is not very informative and it qualifies as an error. Someone reading that for the first time in the dmesg output and wondering why their device isn't working will be confused if they don't know about the mad size changes you are making here. Something like "max supported MAD size (%u) < min required by ib_mad (%u), ignoring dev \n"
Quoted 189 lines
drivers/infiniband/core/mad.c +0
-0/* Create new device info */ port_priv =kzalloc (sizeof *port_priv,GFP_KERNEL );if (!port_priv) {drivers/infiniband/hw/amso1100/c2_rnic.c +1
-0@@ -197,6 +197,7 @@ static int c2_rnic_query(struct c2_dev *c2dev, struct ib_device_attr *props) props->max_srq_sge =0 ; props->max_pkeys =0 ; props->local_ca_ack_delay =0 ; + props->max_mad_size =IB_MGMT_MAD_SIZE ; bail2:vq_repbuf_free (c2dev, reply);drivers/infiniband/hw/cxgb3/iwch_provider.c +1
-0@@ -1174,6 +1174,7 @@ static int iwch_query_device(struct ib_device *ibdev, props->max_pd = dev->attr.max_pds; props->local_ca_ack_delay =0 ; props->max_fast_reg_page_list_len =T3_MAX_FASTREG_DEPTH ; + props->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }drivers/infiniband/hw/cxgb4/provider.c +1
-0@@ -332,6 +332,7 @@ static int c4iw_query_device(struct ib_device *ibdev, props->max_pd =T4_MAX_NUM_PD ; props->local_ca_ack_delay =0 ; props->max_fast_reg_page_list_len =t4_max_fr_depth (use_dsgl); + props->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }drivers/infiniband/hw/ehca/ehca_hca.c +3
-0@@ -40,6 +40,7 @@*/ #include <linux/gfp.h> +#include <rdma/ib_mad.h> #include "ehca_tools.h" #include "ehca_iverbs.h" @@ -133,6 +134,8 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props)if (rblock->hca_cap_indicators & cap_mapping[i +1 ]) props->device_cap_flags |= cap_mapping[i]; + props->max_mad_size =IB_MGMT_MAD_SIZE ; + query_device1:ehca_free_fw_ctrlblock (rblock);drivers/infiniband/hw/ipath/ipath_verbs.c +1
-0@@ -1538,6 +1538,7 @@ static int ipath_query_device(struct ib_device *ibdev, props->max_mcast_qp_attach = ib_ipath_max_mcast_qp_attached; props->max_total_mcast_qp_attach = props->max_mcast_qp_attach * props->max_mcast_grp; + props->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }drivers/infiniband/hw/mlx4/main.c +1
-0@@ -229,6 +229,7 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->max_total_mcast_qp_attach = props->max_mcast_qp_attach * props->max_mcast_grp; props->max_map_per_fmr = dev->dev->caps.max_fmr_maps; + props->max_mad_size =IB_MGMT_MAD_SIZE ; out:kfree (in_mad);drivers/infiniband/hw/mlx5/main.c +1
-0@@ -243,6 +243,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev, props->max_total_mcast_qp_attach = props->max_mcast_qp_attach * props->max_mcast_grp; props->max_map_per_fmr =INT_MAX ;/* no limit in ConnectIB */ + props->max_mad_size =IB_MGMT_MAD_SIZE ;#ifdef CONFIG_INFINIBAND_ON_DEMAND_PAGING if (dev->mdev->caps.gen.flags &MLX5_DEV_CAP_FLAG_ON_DMND_PG )drivers/infiniband/hw/mthca/mthca_provider.c +2
-0@@ -123,6 +123,8 @@ static int mthca_query_device(struct ib_device *ibdev, props->max_map_per_fmr = (1 << (32 -ilog2 (mdev->limits.num_mpts))) -1 ; + props->max_mad_size =IB_MGMT_MAD_SIZE ; + err =0 ; out:kfree (in_mad);drivers/infiniband/hw/nes/nes_verbs.c +1
-0@@ -555,6 +555,7 @@ static int nes_query_device(struct ib_device *ibdev, struct ib_device_attr *prop props->max_qp_init_rd_atom = props->max_qp_rd_atom; props->atomic_cap =IB_ATOMIC_NONE ; props->max_map_per_fmr =1 ; + props->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }drivers/infiniband/hw/ocrdma/ocrdma_verbs.c +1
-0@@ -103,6 +103,7 @@ int ocrdma_query_device(struct ib_device *ibdev, struct ib_device_attr *attr) attr->local_ca_ack_delay = dev->attr.local_ca_ack_delay; attr->max_fast_reg_page_list_len = dev->attr.max_pages_per_frmr; attr->max_pkeys =1 ; + attr->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }drivers/infiniband/hw/qib/qib_verbs.c +1
-0@@ -1591,6 +1591,7 @@ static int qib_query_device(struct ib_device *ibdev, props->max_mcast_qp_attach = ib_qib_max_mcast_qp_attached; props->max_total_mcast_qp_attach = props->max_mcast_qp_attach * props->max_mcast_grp; + props->max_mad_size =IB_MGMT_MAD_SIZE ;return 0 ; }drivers/infiniband/hw/usnic/usnic_ib_verbs.c +2
-0@@ -22,6 +22,7 @@#include <rdma/ib_user_verbs.h> #include <rdma/ib_addr.h> +#include <rdma/ib_mad.h> #include "usnic_abi.h" #include "usnic_ib.h" @@ -296,6 +297,7 @@ int usnic_ib_query_device(struct ib_device *ibdev, props->max_mcast_qp_attach =0 ; props->max_total_mcast_qp_attach =0 ; props->max_map_per_fmr =0 ; + props->max_mad_size =IB_MGMT_MAD_SIZE ;/* Owned by Userspace * max_qp_wr, max_sge, max_sge_rd, max_cqe */ mutex_unlock (&us_ibdev->usdev_lock);include/rdma/ib_mad.h +1
-0@@ -135,6 +135,7 @@ enum {IB_MGMT_SA_DATA =200 ,IB_MGMT_DEVICE_HDR =64 ,IB_MGMT_DEVICE_DATA =192 , + IB_MGMT_MAD_SIZE = IB_MGMT_MAD_HDR + IB_MGMT_MAD_DATA, }; struct ib_mad_hdr {include/rdma/ib_verbs.h +1
-0@@ -210,6 +210,7 @@ struct ib_device_attr {int sig_prot_cap;int sig_guard_cap;struct ib_odp_caps odp_caps; +u32 max_mad_size; };enum ib_mtu {
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
A message that is not in the archive (sent off-list or not yet received): [email protected]
RE: [PATCH v4 06/19] IB/core: Add max_mad_size to ib_device_attr
PiA+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2luZmluaWJhbmQvY29yZS9tYWQuYw0KPiA+IGIvZHJp dmVycy9pbmZpbmliYW5kL2NvcmUvbWFkLmMgaW5kZXggODE5Yjc5NC4uYTZhMzNjZiAxMDA2NDQN Cj4gPiAtLS0gYS9kcml2ZXJzL2luZmluaWJhbmQvY29yZS9tYWQuYw0KPiA+ICsrKyBiL2RyaXZl cnMvaW5maW5pYmFuZC9jb3JlL21hZC5jDQo+ID4gQEAgLTI5MjQsNiArMjkyNCwxMiBAQCBzdGF0 aWMgaW50IGliX21hZF9wb3J0X29wZW4oc3RydWN0IGliX2RldmljZQ0KPiAqZGV2aWNlLA0KPiA+ ICAJY2hhciBuYW1lW3NpemVvZiAiaWJfbWFkMTIzIl07DQo+ID4gIAlpbnQgaGFzX3NtaTsNCj4g Pg0KPiA+ICsJaWYgKGRldmljZS0+Y2FjaGVkX2Rldl9hdHRycy5tYXhfbWFkX3NpemUgPCBJQl9N R01UX01BRF9TSVpFKSB7DQo+ID4gKwkJZGV2X2VycigmZGV2aWNlLT5kZXYsICJNaW4gTUFEIHNp emUgZm9yIGRldmljZSBpcyAldVxuIiwNCj4gPiArCQkJSUJfTUdNVF9NQURfU0laRSk7DQo+ID4g KwkJcmV0dXJuIC1FRkFVTFQ7DQo+ID4gKwl9DQo+ID4gKw0KPiANCj4gVGhlIHByaW50ayBtZXNz YWdlIGhlcmUgaXMgbm90IHZlcnkgaW5mb3JtYXRpdmUgYW5kIGl0IHF1YWxpZmllcyBhcyBhbiBl cnJvci4NCj4gU29tZW9uZSByZWFkaW5nIHRoYXQgZm9yIHRoZSBmaXJzdCB0aW1lIGluIHRoZSBk bWVzZyBvdXRwdXQgYW5kIHdvbmRlcmluZw0KPiB3aHkgdGhlaXIgZGV2aWNlIGlzbid0IHdvcmtp bmcgd2lsbCBiZSBjb25mdXNlZCBpZiB0aGV5IGRvbid0IGtub3cgYWJvdXQgdGhlDQo+IG1hZCBz aXplIGNoYW5nZXMgeW91IGFyZSBtYWtpbmcgaGVyZS4gIFNvbWV0aGluZyBsaWtlICJtYXggc3Vw cG9ydGVkIE1BRA0KPiBzaXplICgldSkgPCBtaW4gcmVxdWlyZWQgYnkgaWJfbWFkICgldSksIGln bm9yaW5nIGRldiBcbiINCg0KR29vZCBzdWdnZXN0aW9uLg0KDQpGaXhlZCBmb3IgdjUgd2l0aCB0 aGlzIG1lc3NhZ2UuDQoNCisgICAgICAgICAgICAgICBkZXZfZXJyKCZkZXZpY2UtPmRldiwNCisg ICAgICAgICAgICAgICAgICAgICAgICJNYXggc3VwcG9ydGVkIE1BRCBzaXplICgldSkgPCBtaW4g cmVxdWlyZWQgYnkgaWJfbWFkICgldSksIGlnbm9yaW5nIGRldmljZSAoJXMpXG4iLA0KKyAgICAg ICAgICAgICAgICAgICAgICAgZGV2aWNlLT5jYWNoZWRfZGV2X2F0dHJzLm1heF9tYWRfc2l6ZSwN CisgICAgICAgICAgICAgICAgICAgICAgIElCX01HTVRfTUFEX1NJWkUsIGRldmljZS0+bmFtZSk7 DQoNCg0KSXJhDQoNCg==
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 2807E5FD2F6FDA4886F6618EAC48510E0CC3ECE5-8k97q/[email protected]
Re: [PATCH v4 06/19] IB/core: Add max_mad_size to ib_device_attr
Quoted 25 lines · Ira
Quoted 8 lines
Review static int ib_mad_port_open(struct ib_device
Quoted 4 lines
(quoted diff) +0
-0
b/drivers/infiniband/core/mad.cindex 819b794..a6a33cf 100644 +++b/drivers/infiniband/core/mad.c(quoted diff) +0
-0@@ -2924,6 +2924,12 @@ static int ib_mad_port_open(struct ib_device*device,
Quoted 3 lines
drivers/infiniband/core/mad.c +0
-0char name[sizeof "ib_mad123" ];int has_smi;Quoted 12 lines
drivers/infiniband/core/mad.c static int ib_mad_port_open(struct ib_device *device, · line 2932
Quoted 6 lines
drivers/infiniband/core/mad.c +6
-0+if (device->cached_dev_attrs.max_mad_size <IB_MGMT_MAD_SIZE ) { +dev_err (&device->dev,"Min MAD size for device is %u\n" , +IB_MGMT_MAD_SIZE ); +return -EFAULT ; + } +The printk message here is not very informative and it qualifies as an error. Someone reading that for the first time in the dmesg output and wondering why their device isn't working will be confused if they don't know about the mad size changes you are making here. Something like "max supported MAD size (%u) < min required by
ib_mad(%u), ignoring dev \n"Good suggestion.
Fixed for v5 with this message.
Review
Quoted 4 lines
(quoted diff) +4
-0+ dev_err(&device->dev, + "Max supported MAD size (%u) < min required by ib_mad (%u), ignoring device (%s)\n", + device->cached_dev_attrs.max_mad_size, + IB_MGMT_MAD_SIZE, device->name);
It also seems redundant since the only call to ib_mad_port_open is:
if (ib_mad_port_open(device, i)) {
printk(KERN_ERR PFX "Couldn't open %s port %d\n",
device->name, i);So, why does this particular error deserve a special double error print? I assume it is basically impossible to hit?
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20150225182308.GA14580-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org
RE: [PATCH v4 06/19] IB/core: Add max_mad_size to ib_device_attr
Quoted 9 lines
Review
Quoted 5 lines
Fixed for v5 with this message.
(quoted diff) +2
-0+ dev_err(&device->dev, + "Max supported MAD size (%u) < min required by ib_mad(%u), ignoring device (%s)\n",
Review
Quoted 2 lines
(quoted diff) +2
-0+ device->cached_dev_attrs.max_mad_size, + IB_MGMT_MAD_SIZE, device->name);
Quoted 8 lines
It also seems redundant since the only call to
ib_mad_port_openis:if (ib_mad_port_open(device, i)) { printk(KERN_ERR PFX "Couldn't open %s port %d\n", device->name, i);So, why does this particular error deserve a special double error print? I assume it is basically impossible to hit?
This does indicate a coding error. Generally I prefer details of why the device could not open the port. But if the community feels this is redundant or "not possible" I can drop the hunk.
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 2807E5FD2F6FDA4886F6618EAC48510E0CC3ED5B-8k97q/[email protected]
Re: [PATCH v4 06/19] IB/core: Add max_mad_size to ib_device_attr
Quoted 15 lines · Ira
Quoted 15 lines
Review
Quoted 5 lines
Fixed for v5 with this message.
(quoted diff) +2
-0+ dev_err(&device->dev, + "Max supported MAD size (%u) < min required by ib_mad(%u), ignoring device (%s)\n",
Review
Quoted 2 lines
(quoted diff) +2
-0+ device->cached_dev_attrs.max_mad_size, + IB_MGMT_MAD_SIZE, device->name);It also seems redundant since the only call to
ib_mad_port_openis:if (ib_mad_port_open(device, i)) { printk(KERN_ERR PFX "Couldn't open %s port %d\n", device->name, i);
Quoted 6 lines
Quoted 2 lines
So, why does this particular error deserve a special double error print? I assume it is basically impossible to hit?
This does indicate a coding error. Generally I prefer details of why the device could not open the port. But if the community feels this is redundant or "not possible" I can drop the hunk.
Internal logic errors are handled with WARN_ON/BUG/etc.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 7/19 IB/mad: Convert ib_mad_private allocations from kmem_cache to kmalloc
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
Use the new max_mad_size specified by devices for the allocations and DMA maps.
kmalloc is more flexible to support devices with different sized MADs and research and testing showed that the current use of kmem_cache does not provide performance benefits over kmalloc.
--- drivers/infiniband/core/mad.c | 73 ++++++++++++++++++------------------------- 1 file changed, 30 insertions(+), 43 deletions(-)
drivers/infiniband/core/mad.c +30 -43
@@ -59,8 +59,6 @@ MODULE_PARM_DESC(send_queue_size, "Size of send queue in number of work requestsmodule_param_named (recv_queue_size, mad_recvq_size, int,0444 );MODULE_PARM_DESC (recv_queue_size,"Size of receive queue in number of work requests" ); -static struct kmem_cache *ib_mad_cache; -static struct list_head ib_mad_port_list;static u32 ib_mad_client_id =0 ;
@@ -717,6 +715,13 @@ static void build_smp_wc(struct ib_qp *qp, wc->port_num = port_num; } +static struct ib_mad_private *alloc_mad_priv (struct ib_device *dev) +{ +return (kmalloc (sizeof (struct ib_mad_private_header ) + +sizeof (struct ib_grh ) + + dev->cached_dev_attrs.max_mad_size,GFP_ATOMIC )); +} +/* * Return 0 if SMP is to be sent * Return 1 if SMP was consumed locally (whether or not solicited)
@@ -771,7 +776,8 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv, } local->mad_priv =NULL ; local->recv_mad_agent =NULL ; - mad_priv =kmem_cache_alloc (ib_mad_cache,GFP_ATOMIC ); + + mad_priv =alloc_mad_priv (mad_agent_priv->agent.device);if (!mad_priv) { ret = -ENOMEM ;dev_err (&device->dev,"No memory for local response MAD\n" );
@@ -801,10 +807,10 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,*/ atomic_inc (&mad_agent_priv->refcount); }else -kmem_cache_free (ib_mad_cache, mad_priv); +kfree (mad_priv);break ;case IB_MAD_RESULT_SUCCESS |IB_MAD_RESULT_CONSUMED : -kmem_cache_free (ib_mad_cache, mad_priv); +kfree (mad_priv);break ;case IB_MAD_RESULT_SUCCESS :/* Treat like an incoming receive MAD */
@@ -820,14 +826,14 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,* No receiving agent so drop packet and * generate send completion. */ -kmem_cache_free (ib_mad_cache, mad_priv); +kfree (mad_priv);break ; } local->mad_priv = mad_priv; local->recv_mad_agent = recv_mad_agent;break ;default : -kmem_cache_free (ib_mad_cache, mad_priv); +kfree (mad_priv);kfree (local); ret = -EINVAL ; goto out;
@@ -1237,7 +1243,7 @@ void ib_free_recv_mad(struct ib_mad_recv_wc *mad_recv_wc) recv_wc); priv = container_of(mad_priv_hdr, struct ib_mad_private, header); -kmem_cache_free (ib_mad_cache, priv); +kfree (priv); } }EXPORT_SYMBOL (ib_free_recv_mad);
@@ -1924,6 +1930,11 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, } } +static size_t mad_recv_buf_size (struct ib_device *dev) +{ +return (sizeof (struct ib_grh ) + dev->cached_dev_attrs.max_mad_size); +} +static bool generate_unmatched_resp (struct ib_mad_private *recv,struct ib_mad_private *response) {
@@ -1964,8 +1975,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, recv = container_of(mad_priv_hdr, struct ib_mad_private, header);ib_dma_unmap_single (port_priv->device, recv->header.mapping, -sizeof (struct ib_mad_private ) - -sizeof (struct ib_mad_private_header ), +mad_recv_buf_size (port_priv->device),DMA_FROM_DEVICE );/* Setup MAD receive work completion from "normal" work completion */
@@ -1982,7 +1992,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv,if (!validate_mad (&recv->mad.mad.mad_hdr, qp_info->qp->qp_num)) goto out; - response =kmem_cache_alloc (ib_mad_cache,GFP_KERNEL ); + response =alloc_mad_priv (port_priv->device);if (!response) {dev_err (&port_priv->device->dev,"ib_mad_recv_done_handler no memory for response buffer\n" );
@@ -2075,7 +2085,7 @@ out:if (response) {ib_mad_post_receive_mads (qp_info, response);if (recv) -kmem_cache_free (ib_mad_cache, recv); +kfree (recv); }else ib_mad_post_receive_mads (qp_info, recv); }
@@ -2535,7 +2545,7 @@ local_send_completion:spin_lock_irqsave (&mad_agent_priv->lock, flags);atomic_dec (&mad_agent_priv->refcount);if (free_mad) -kmem_cache_free (ib_mad_cache, local->mad_priv); +kfree (local->mad_priv);kfree (local); }spin_unlock_irqrestore (&mad_agent_priv->lock, flags);
@@ -2664,7 +2674,7 @@ static int ib_mad_post_receive_mads(struct ib_mad_qp_info *qp_info, mad_priv = mad; mad =NULL ; }else { - mad_priv =kmem_cache_alloc (ib_mad_cache,GFP_KERNEL ); + mad_priv =alloc_mad_priv (qp_info->port_priv->device);if (!mad_priv) {dev_err (&qp_info->port_priv->device->dev,"No memory for receive buffer\n" );
@@ -2674,8 +2684,7 @@ static int ib_mad_post_receive_mads(struct ib_mad_qp_info *qp_info, } sg_list.addr =ib_dma_map_single (qp_info->port_priv->device, &mad_priv->grh, -sizeof *mad_priv - -sizeof mad_priv->header, +mad_recv_buf_size (qp_info->port_priv->device),DMA_FROM_DEVICE );if (unlikely (ib_dma_mapping_error (qp_info->port_priv->device, sg_list.addr))) {
@@ -2699,10 +2708,9 @@ static int ib_mad_post_receive_mads(struct ib_mad_qp_info *qp_info,spin_unlock_irqrestore (&recv_queue->lock, flags);ib_dma_unmap_single (qp_info->port_priv->device, mad_priv->header.mapping, -sizeof *mad_priv - -sizeof mad_priv->header, +mad_recv_buf_size (qp_info->port_priv->device),DMA_FROM_DEVICE ); -kmem_cache_free (ib_mad_cache, mad_priv); +kfree (mad_priv);dev_err (&qp_info->port_priv->device->dev,"ib_post_recv failed: %d\n" , ret);break ;
@@ -2739,10 +2747,9 @@ static void cleanup_recv_queue(struct ib_mad_qp_info *qp_info)ib_dma_unmap_single (qp_info->port_priv->device, recv->header.mapping, -sizeof (struct ib_mad_private ) - -sizeof (struct ib_mad_private_header ), +mad_recv_buf_size (qp_info->port_priv->device),DMA_FROM_DEVICE ); -kmem_cache_free (ib_mad_cache, recv); +kfree (recv); } qp_info->recv_queue.count =0 ;
@@ -3138,45 +3145,25 @@ static struct ib_client mad_client = {static int __init ib_mad_init_module (void ) { -int ret; - mad_recvq_size =min (mad_recvq_size,IB_MAD_QP_MAX_SIZE ); mad_recvq_size =max (mad_recvq_size,IB_MAD_QP_MIN_SIZE ); mad_sendq_size =min (mad_sendq_size,IB_MAD_QP_MAX_SIZE ); mad_sendq_size =max (mad_sendq_size,IB_MAD_QP_MIN_SIZE ); - ib_mad_cache =kmem_cache_create ("ib_mad" , -sizeof (struct ib_mad_private ), -0 , -SLAB_HWCACHE_ALIGN , -NULL ); -if (!ib_mad_cache) { -pr_err ("Couldn't create ib_mad cache\n" ); - ret = -ENOMEM ; - goto error1; - } -INIT_LIST_HEAD (&ib_mad_port_list);if (ib_register_client (&mad_client)) {pr_err ("Couldn't register ib_mad client\n" ); - ret = -EINVAL ; - goto error2; +return (-EINVAL ); }return 0 ; - -error2: -kmem_cache_destroy (ib_mad_cache); -error1: -return ret; } static void __exit ib_mad_cleanup_module(void) {ib_unregister_client (&mad_client); -kmem_cache_destroy (ib_mad_cache); }module_init (ib_mad_init_module);
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-8-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v4 07/19] IB/mad: Convert ib_mad_private allocations from kmem_cache to kmalloc
Quoted 28 lines · On Wed, 2015-02-04 at 18:29 -0500, [email protected]…
From: Ira Weiny <[email protected]>
Use the new
max_mad_sizespecified by devices for the allocations and DMA maps.kmalloc is more flexible to support devices with different sized MADs and research and testing showed that the current use of
kmem_cachedoes not provide performance benefits over kmalloc.Signed-off-by Ira Weiny <[email protected]>--- drivers/infiniband/core/mad.c | 73 ++++++++++++++++++------------------------- 1 file changed, 30 insertions(+), 43 deletions(-)drivers/infiniband/core/mad.c +0
-2@@ -59,8 +59,6 @@ MODULE_PARM_DESC(send_queue_size, "Size of send queue in number of work requestsmodule_param_named (recv_queue_size, mad_recvq_size, int,0444 );MODULE_PARM_DESC (recv_queue_size,"Size of receive queue in number of work requests" ); -static struct kmem_cache *ib_mad_cache; -static struct list_head ib_mad_port_list;static u32 ib_mad_client_id =0 ;
drivers/infiniband/core/mad.c static void build_smp_wc(struct ib_qp *qp, · line 722
Quoted 8 lines
drivers/infiniband/core/mad.c +5
-0@@ -717,6 +715,13 @@ static void build_smp_wc(struct ib_qp *qp, wc->port_num = port_num; } +static struct ib_mad_private *alloc_mad_priv (struct ib_device *dev) +{ +return (kmalloc (sizeof (struct ib_mad_private_header ) + +sizeof (struct ib_grh ) + + dev->cached_dev_attrs.max_mad_size,GFP_ATOMIC ));
Ouch! GFP_ATOMIC? I thought that generally all of the mad processing was done from workqueue context where sleeping is allowed? In the two places where you removed kmem_cache_alloc() calls and replaced it with calls to this code, they both used GFP_KERNEL and now you have switched it to GFP_ATOMIC. If there isn't a good reason for this, it should be switched back to GFP_KERNEL.
Quoted 195 lines
drivers/infiniband/core/mad.c +25
-41+} +/* * Return 0 if SMP is to be sent * Return 1 if SMP was consumed locally (whether or not solicited) @@ -771,7 +776,8 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv, } local->mad_priv =NULL ; local->recv_mad_agent =NULL ; - mad_priv =kmem_cache_alloc (ib_mad_cache,GFP_ATOMIC ); + + mad_priv =alloc_mad_priv (mad_agent_priv->agent.device);if (!mad_priv) { ret = -ENOMEM ;dev_err (&device->dev,"No memory for local response MAD\n" );@@ -801,10 +807,10 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,*/ atomic_inc (&mad_agent_priv->refcount); }else -kmem_cache_free (ib_mad_cache, mad_priv); +kfree (mad_priv);break ;case IB_MAD_RESULT_SUCCESS |IB_MAD_RESULT_CONSUMED : -kmem_cache_free (ib_mad_cache, mad_priv); +kfree (mad_priv);break ;case IB_MAD_RESULT_SUCCESS :/* Treat like an incoming receive MAD */ @@ -820,14 +826,14 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,* No receiving agent so drop packet and * generate send completion. */ -kmem_cache_free (ib_mad_cache, mad_priv); +kfree (mad_priv);break ; } local->mad_priv = mad_priv; local->recv_mad_agent = recv_mad_agent;break ;default : -kmem_cache_free (ib_mad_cache, mad_priv); +kfree (mad_priv);kfree (local); ret = -EINVAL ; goto out;@@ -1237,7 +1243,7 @@ void ib_free_recv_mad(struct ib_mad_recv_wc *mad_recv_wc) recv_wc); priv = container_of(mad_priv_hdr, struct ib_mad_private, header); -kmem_cache_free (ib_mad_cache, priv); +kfree (priv); } }EXPORT_SYMBOL (ib_free_recv_mad);@@ -1924,6 +1930,11 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, } } +static size_t mad_recv_buf_size (struct ib_device *dev) +{ +return (sizeof (struct ib_grh ) + dev->cached_dev_attrs.max_mad_size); +} +static bool generate_unmatched_resp (struct ib_mad_private *recv,struct ib_mad_private *response) {@@ -1964,8 +1975,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, recv = container_of(mad_priv_hdr, struct ib_mad_private, header);ib_dma_unmap_single (port_priv->device, recv->header.mapping, -sizeof (struct ib_mad_private ) - -sizeof (struct ib_mad_private_header ), +mad_recv_buf_size (port_priv->device),DMA_FROM_DEVICE );/* Setup MAD receive work completion from "normal" work completion */ @@ -1982,7 +1992,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv,if (!validate_mad (&recv->mad.mad.mad_hdr, qp_info->qp->qp_num)) goto out; - response =kmem_cache_alloc (ib_mad_cache,GFP_KERNEL ); + response =alloc_mad_priv (port_priv->device);if (!response) {dev_err (&port_priv->device->dev,"ib_mad_recv_done_handler no memory for response buffer\n" );@@ -2075,7 +2085,7 @@ out:if (response) {ib_mad_post_receive_mads (qp_info, response);if (recv) -kmem_cache_free (ib_mad_cache, recv); +kfree (recv); }else ib_mad_post_receive_mads (qp_info, recv); }@@ -2535,7 +2545,7 @@ local_send_completion:spin_lock_irqsave (&mad_agent_priv->lock, flags);atomic_dec (&mad_agent_priv->refcount);if (free_mad) -kmem_cache_free (ib_mad_cache, local->mad_priv); +kfree (local->mad_priv);kfree (local); }spin_unlock_irqrestore (&mad_agent_priv->lock, flags);@@ -2664,7 +2674,7 @@ static int ib_mad_post_receive_mads(struct ib_mad_qp_info *qp_info, mad_priv = mad; mad =NULL ; }else { - mad_priv =kmem_cache_alloc (ib_mad_cache,GFP_KERNEL ); + mad_priv =alloc_mad_priv (qp_info->port_priv->device);if (!mad_priv) {dev_err (&qp_info->port_priv->device->dev,"No memory for receive buffer\n" );@@ -2674,8 +2684,7 @@ static int ib_mad_post_receive_mads(struct ib_mad_qp_info *qp_info, } sg_list.addr =ib_dma_map_single (qp_info->port_priv->device, &mad_priv->grh, -sizeof *mad_priv - -sizeof mad_priv->header, +mad_recv_buf_size (qp_info->port_priv->device),DMA_FROM_DEVICE );if (unlikely (ib_dma_mapping_error (qp_info->port_priv->device, sg_list.addr))) {@@ -2699,10 +2708,9 @@ static int ib_mad_post_receive_mads(struct ib_mad_qp_info *qp_info,spin_unlock_irqrestore (&recv_queue->lock, flags);ib_dma_unmap_single (qp_info->port_priv->device, mad_priv->header.mapping, -sizeof *mad_priv - -sizeof mad_priv->header, +mad_recv_buf_size (qp_info->port_priv->device),DMA_FROM_DEVICE ); -kmem_cache_free (ib_mad_cache, mad_priv); +kfree (mad_priv);dev_err (&qp_info->port_priv->device->dev,"ib_post_recv failed: %d\n" , ret);break ;@@ -2739,10 +2747,9 @@ static void cleanup_recv_queue(struct ib_mad_qp_info *qp_info)ib_dma_unmap_single (qp_info->port_priv->device, recv->header.mapping, -sizeof (struct ib_mad_private ) - -sizeof (struct ib_mad_private_header ), +mad_recv_buf_size (qp_info->port_priv->device),DMA_FROM_DEVICE ); -kmem_cache_free (ib_mad_cache, recv); +kfree (recv); } qp_info->recv_queue.count =0 ;@@ -3138,45 +3145,25 @@ static struct ib_client mad_client = {static int __init ib_mad_init_module (void ) { -int ret; - mad_recvq_size =min (mad_recvq_size,IB_MAD_QP_MAX_SIZE ); mad_recvq_size =max (mad_recvq_size,IB_MAD_QP_MIN_SIZE ); mad_sendq_size =min (mad_sendq_size,IB_MAD_QP_MAX_SIZE ); mad_sendq_size =max (mad_sendq_size,IB_MAD_QP_MIN_SIZE ); - ib_mad_cache =kmem_cache_create ("ib_mad" , -sizeof (struct ib_mad_private ), -0 , -SLAB_HWCACHE_ALIGN , -NULL ); -if (!ib_mad_cache) { -pr_err ("Couldn't create ib_mad cache\n" ); - ret = -ENOMEM ; - goto error1; - } -INIT_LIST_HEAD (&ib_mad_port_list);if (ib_register_client (&mad_client)) {pr_err ("Couldn't register ib_mad client\n" ); - ret = -EINVAL ; - goto error2; +return (-EINVAL ); }return 0 ; - -error2: -kmem_cache_destroy (ib_mad_cache); -error1: -return ret; } static void __exit ib_mad_cleanup_module(void) {ib_unregister_client (&mad_client); -kmem_cache_destroy (ib_mad_cache); }module_init (ib_mad_init_module);
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
A message that is not in the archive (sent off-list or not yet received): [email protected]
RE: [PATCH v4 07/19] IB/mad: Convert ib_mad_private allocations from kmem_cache to kmalloc
PiA+DQo+ID4gK3N0YXRpYyBzdHJ1Y3QgaWJfbWFkX3ByaXZhdGUgKmFsbG9jX21hZF9wcml2KHN0 cnVjdCBpYl9kZXZpY2UgKmRldikgew0KPiA+ICsJcmV0dXJuIChrbWFsbG9jKHNpemVvZihzdHJ1 Y3QgaWJfbWFkX3ByaXZhdGVfaGVhZGVyKSArDQo+ID4gKwkJCXNpemVvZihzdHJ1Y3QgaWJfZ3Jo KSArDQo+ID4gKwkJCWRldi0+Y2FjaGVkX2Rldl9hdHRycy5tYXhfbWFkX3NpemUsDQo+IEdGUF9B VE9NSUMpKTsNCj4gDQo+IE91Y2ghICBHRlBfQVRPTUlDPyAgSSB0aG91Z2h0IHRoYXQgZ2VuZXJh bGx5IGFsbCBvZiB0aGUgbWFkIHByb2Nlc3Npbmcgd2FzDQo+IGRvbmUgZnJvbSB3b3JrcXVldWUg Y29udGV4dCB3aGVyZSBzbGVlcGluZyBpcyBhbGxvd2VkPyAgSW4gdGhlIHR3byBwbGFjZXMNCj4g d2hlcmUgeW91IHJlbW92ZWQga21lbV9jYWNoZV9hbGxvYygpIGNhbGxzIGFuZCByZXBsYWNlZCBp dCB3aXRoIGNhbGxzIHRvIHRoaXMNCj4gY29kZSwgdGhleSBib3RoIHVzZWQgR0ZQX0tFUk5FTCBh bmQgbm93IHlvdSBoYXZlIHN3aXRjaGVkIGl0IHRvDQo+IEdGUF9BVE9NSUMuICBJZiB0aGVyZSBp c24ndCBhIGdvb2QgcmVhc29uIGZvciB0aGlzLCBpdCBzaG91bGQgYmUgc3dpdGNoZWQgYmFjaw0K PiB0byBHRlBfS0VSTkVMLg0KDQpUaGUgb3JpZ2luYWwga21lbV9jYWNoZV9hbGxvY3MgYXJlIGFj dHVhbGx5IGJvdGggR0ZQX0FUT01JQyAoMSB1c2FnZSwgc2VlIGJlbG93KSBhbmQgR0ZQX0tFUk5F TCAodGhlIDIgdXNhZ2VzIHlvdSByZWZlcmVuY2UpLg0KDQpNeSBiYWQgZm9yIG5vdCBtYWtpbmcg dGhpcyBzcGVjaWZpYyB0byB0aGUgYWxsb2NhdGlvbi4NCg0KSSB3aWxsIHJlc2VhcmNoIHRoZSBv cmlnaW5hbCBHRlBfQVRPTUlDIHVzYWdlIGFuZCBpZiBpdCBpcyBuZWNlc3NhcnkgaGF2ZSB0aGlz IGZ1bmN0aW9uIHRha2UgZ2ZwX3QuICBPdGhlcndpc2UgaWYgd2UgY2FuIGdldCBhd2F5IHdpdGgg R0ZQX0tFUk5FTCBJIGFncmVlIHRoYXQgd291bGQgYmUgYmVzdC4NCg0KPiANCj4gPiArfQ0KPiA+ ICsNCj4gPiAgLyoNCj4gPiAgICogUmV0dXJuIDAgaWYgU01QIGlzIHRvIGJlIHNlbnQNCj4gPiAg ICogUmV0dXJuIDEgaWYgU01QIHdhcyBjb25zdW1lZCBsb2NhbGx5ICh3aGV0aGVyIG9yIG5vdCBz b2xpY2l0ZWQpIEBADQo+ID4gLTc3MSw3ICs3NzYsOCBAQCBzdGF0aWMgaW50IGhhbmRsZV9vdXRn b2luZ19kcl9zbXAoc3RydWN0DQo+IGliX21hZF9hZ2VudF9wcml2YXRlICptYWRfYWdlbnRfcHJp diwNCj4gPiAgCX0NCj4gPiAgCWxvY2FsLT5tYWRfcHJpdiA9IE5VTEw7DQo+ID4gIAlsb2NhbC0+ cmVjdl9tYWRfYWdlbnQgPSBOVUxMOw0KPiA+IC0JbWFkX3ByaXYgPSBrbWVtX2NhY2hlX2FsbG9j KGliX21hZF9jYWNoZSwgR0ZQX0FUT01JQyk7DQoNCk9yaWdpbmFsIHVzYWdlIGhlcmUuLi4gDQoN ClRoYW5rcywNCklyYQ0KDQo+ID4gKw0KPiA+ICsJbWFkX3ByaXYgPSBhbGxvY19tYWRfcHJpdiht YWRfYWdlbnRfcHJpdi0+YWdlbnQuZGV2aWNlKTsNCj4gPiAgCWlmICghbWFkX3ByaXYpIHsNCj4g PiAgCQlyZXQgPSAtRU5PTUVNOw0KPiA+ICAJCWRldl9lcnIoJmRldmljZS0+ZGV2LCAiTm8gbWVt b3J5IGZvciBsb2NhbCByZXNwb25zZQ0KPiBNQURcbiIpOyBAQA0KDQo=
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 8/19 IB/mad: Add helper function for smi_handle_dr_smp_send
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
This helper function will be used for processing both IB and OPA SMP sends.
--- drivers/infiniband/core/smi.c | 81 +++++++++++++++++++++++++------------------ 1 file changed, 47 insertions(+), 34 deletions(-)
drivers/infiniband/core/smi.c +47 -34
@@ -39,84 +39,81 @@#include <rdma/ib_smi.h> #include "smi.h" -/* -* Fixup a directed route SMP for sending -* Return 0 if the SMP should be discarded -*/ -enum smi_action smi_handle_dr_smp_send (struct ib_smp *smp, -u8 node_type,int port_num) +static inline +enum smi_action __smi_handle_dr_smp_send (u8 node_type,int port_num, +u8 *hop_ptr,u8 hop_cnt, +u8 *initial_path, +u8 *return_path, +u8 direction, +int dr_dlid_is_permissive, +int dr_slid_is_permissive) { -u8 hop_ptr, hop_cnt; - - hop_ptr = smp->hop_ptr; - hop_cnt = smp->hop_cnt; -/* See section 14.2.2.2, Vol 1 IB spec */ /* C14-6 -- valid hop_cnt values are from 0 to 63 */ if (hop_cnt >=IB_SMP_MAX_PATH_HOPS )return IB_SMI_DISCARD ; -if (!ib_get_smp_direction (smp)) { +if (!direction) {/* C14-9:1 */ -if (hop_cnt && hop_ptr ==0 ) { - smp->hop_ptr++; -return (smp->initial_path[smp->hop_ptr] == +if (hop_cnt && *hop_ptr ==0 ) { + (*hop_ptr)++; +return (initial_path[*hop_ptr] == port_num ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-9:2 */ -if (hop_ptr && hop_ptr < hop_cnt) { +if (*hop_ptr && *hop_ptr < hop_cnt) {if (node_type !=RDMA_NODE_IB_SWITCH )return IB_SMI_DISCARD ; -/* smp->return_path set when received */ - smp->hop_ptr++; -return (smp->initial_path[smp->hop_ptr] == +/* return_path set when received */ + (*hop_ptr)++; +return (initial_path[*hop_ptr] == port_num ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-9:3 -- We're at the end of the DR segment of path */ -if (hop_ptr == hop_cnt) { -/* smp->return_path set when received */ - smp->hop_ptr++; +if (*hop_ptr == hop_cnt) { +/* return_path set when received */ + (*hop_ptr)++;return (node_type ==RDMA_NODE_IB_SWITCH || - smp->dr_dlid ==IB_LID_PERMISSIVE ? + dr_dlid_is_permissive ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-9:4 -- hop_ptr = hop_cnt + 1 -> give to SMA/SM */ /* C14-9:5 -- Fail unreasonable hop pointer */ -return (hop_ptr == hop_cnt +1 ?IB_SMI_HANDLE :IB_SMI_DISCARD ); +return (*hop_ptr == hop_cnt +1 ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }else {/* C14-13:1 */ -if (hop_cnt && hop_ptr == hop_cnt +1 ) { - smp->hop_ptr--; -return (smp->return_path[smp->hop_ptr] == +if (hop_cnt && *hop_ptr == hop_cnt +1 ) { + (*hop_ptr)--; +return (return_path[*hop_ptr] == port_num ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-13:2 */ -if (2 <= hop_ptr && hop_ptr <= hop_cnt) { +if (2 <= *hop_ptr && *hop_ptr <= hop_cnt) {if (node_type !=RDMA_NODE_IB_SWITCH )return IB_SMI_DISCARD ; - smp->hop_ptr--; -return (smp->return_path[smp->hop_ptr] == + (*hop_ptr)--; +return (return_path[*hop_ptr] == port_num ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-13:3 -- at the end of the DR segment of path */ -if (hop_ptr ==1 ) { - smp->hop_ptr--; +if (*hop_ptr ==1 ) { + (*hop_ptr)--;/* C14-13:3 -- SMPs destined for SM shouldn't be here */ return (node_type ==RDMA_NODE_IB_SWITCH || - smp->dr_slid ==IB_LID_PERMISSIVE ? + dr_slid_is_permissive ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-13:4 -- hop_ptr = 0 -> should have gone to SM */ -if (hop_ptr ==0 ) +if (*hop_ptr ==0 )return IB_SMI_HANDLE ;/* C14-13:5 -- Check for unreasonable hop pointer */
@@ -125,6 +122,22 @@ enum smi_action smi_handle_dr_smp_send(struct ib_smp *smp, }/* +* Fixup a directed route SMP for sending +* Return 0 if the SMP should be discarded +*/ +enum smi_action smi_handle_dr_smp_send (struct ib_smp *smp, +u8 node_type,int port_num) +{ +return __smi_handle_dr_smp_send (node_type, port_num, + &smp->hop_ptr, smp->hop_cnt, + smp->initial_path, + smp->return_path, +ib_get_smp_direction (smp), + smp->dr_dlid ==IB_LID_PERMISSIVE , + smp->dr_slid ==IB_LID_PERMISSIVE ); +} + +/* * Adjust information for a received SMP * Return 0 if the SMP should be dropped */
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-9-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Hefty, Sean
Reviewed-byRE: [PATCH v4 08/19] IB/mad: Add helper function for smi_handle_dr_smp_send
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 9/19 IB/mad: Add helper function for smi_handle_dr_smp_recv
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
This helper function will be used for processing both IB and OPA SMP recvs.
--- drivers/infiniband/core/smi.c | 80 +++++++++++++++++++++++++------------------ 1 file changed, 47 insertions(+), 33 deletions(-)
drivers/infiniband/core/smi.c +47 -33
@@ -137,91 +137,105 @@ enum smi_action smi_handle_dr_smp_send(struct ib_smp *smp, smp->dr_slid == IB_LID_PERMISSIVE); } -/* -* Adjust information for a received SMP -* Return 0 if the SMP should be dropped -*/ -enum smi_action smi_handle_dr_smp_recv (struct ib_smp *smp,u8 node_type, -int port_num,int phys_port_cnt) +static inline +enum smi_action __smi_handle_dr_smp_recv (u8 node_type,int port_num, +int phys_port_cnt, +u8 *hop_ptr,u8 hop_cnt, +u8 *initial_path, +u8 *return_path, +u8 direction, +int dr_dlid_is_permissive, +int dr_slid_is_permissive) { -u8 hop_ptr, hop_cnt; - - hop_ptr = smp->hop_ptr; - hop_cnt = smp->hop_cnt; -/* See section 14.2.2.2, Vol 1 IB spec */ /* C14-6 -- valid hop_cnt values are from 0 to 63 */ if (hop_cnt >=IB_SMP_MAX_PATH_HOPS )return IB_SMI_DISCARD ; -if (!ib_get_smp_direction (smp)) { +if (!direction) {/* C14-9:1 -- sender should have incremented hop_ptr */ -if (hop_cnt && hop_ptr ==0 ) +if (hop_cnt && *hop_ptr ==0 )return IB_SMI_DISCARD ;/* C14-9:2 -- intermediate hop */ -if (hop_ptr && hop_ptr < hop_cnt) { +if (*hop_ptr && *hop_ptr < hop_cnt) {if (node_type !=RDMA_NODE_IB_SWITCH )return IB_SMI_DISCARD ; - smp->return_path[hop_ptr] = port_num; -/* smp->hop_ptr updated when sending */ -return (smp->initial_path[hop_ptr+1 ] <= phys_port_cnt ? + return_path[*hop_ptr] = port_num; +/* hop_ptr updated when sending */ +return (initial_path[*hop_ptr+1 ] <= phys_port_cnt ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-9:3 -- We're at the end of the DR segment of path */ -if (hop_ptr == hop_cnt) { +if (*hop_ptr == hop_cnt) {if (hop_cnt) - smp->return_path[hop_ptr] = port_num; -/* smp->hop_ptr updated when sending */ + return_path[*hop_ptr] = port_num; +/* hop_ptr updated when sending */ return (node_type ==RDMA_NODE_IB_SWITCH || - smp->dr_dlid ==IB_LID_PERMISSIVE ? + dr_dlid_is_permissive ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-9:4 -- hop_ptr = hop_cnt + 1 -> give to SMA/SM */ /* C14-9:5 -- fail unreasonable hop pointer */ -return (hop_ptr == hop_cnt +1 ?IB_SMI_HANDLE :IB_SMI_DISCARD ); +return (*hop_ptr == hop_cnt +1 ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }else {/* C14-13:1 */ -if (hop_cnt && hop_ptr == hop_cnt +1 ) { - smp->hop_ptr--; -return (smp->return_path[smp->hop_ptr] == +if (hop_cnt && *hop_ptr == hop_cnt +1 ) { + (*hop_ptr)--; +return (return_path[*hop_ptr] == port_num ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-13:2 */ -if (2 <= hop_ptr && hop_ptr <= hop_cnt) { +if (2 <= *hop_ptr && *hop_ptr <= hop_cnt) {if (node_type !=RDMA_NODE_IB_SWITCH )return IB_SMI_DISCARD ; -/* smp->hop_ptr updated when sending */ -return (smp->return_path[hop_ptr-1 ] <= phys_port_cnt ? +/* hop_ptr updated when sending */ +return (return_path[*hop_ptr-1 ] <= phys_port_cnt ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-13:3 -- We're at the end of the DR segment of path */ -if (hop_ptr ==1 ) { -if (smp->dr_slid ==IB_LID_PERMISSIVE ) { +if (*hop_ptr ==1 ) { +if (dr_slid_is_permissive) {/* giving SMP to SM - update hop_ptr */ - smp->hop_ptr--; + (*hop_ptr)--;return IB_SMI_HANDLE ; } -/* smp->hop_ptr updated when sending */ +/* hop_ptr updated when sending */ return (node_type ==RDMA_NODE_IB_SWITCH ?IB_SMI_HANDLE :IB_SMI_DISCARD ); }/* C14-13:4 -- hop_ptr = 0 -> give to SM */ /* C14-13:5 -- Check for unreasonable hop pointer */ -return (hop_ptr ==0 ?IB_SMI_HANDLE :IB_SMI_DISCARD ); +return (*hop_ptr ==0 ?IB_SMI_HANDLE :IB_SMI_DISCARD ); } } +/* +* Adjust information for a received SMP +* Return 0 if the SMP should be dropped +*/ +enum smi_action smi_handle_dr_smp_recv (struct ib_smp *smp,u8 node_type, +int port_num,int phys_port_cnt) +{ +return __smi_handle_dr_smp_recv (node_type, port_num, phys_port_cnt, + &smp->hop_ptr, smp->hop_cnt, + smp->initial_path, + smp->return_path, +ib_get_smp_direction (smp), + smp->dr_dlid ==IB_LID_PERMISSIVE , + smp->dr_slid ==IB_LID_PERMISSIVE ); +} +enum smi_forward_action smi_check_forward_dr_smp (struct ib_smp *smp) {u8 hop_ptr, hop_cnt;
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 10/19 IB/mad: Add helper function for smi_check_forward_dr_smp
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
This helper function will be used for processing both IB and OPA SMPs.
--- drivers/infiniband/core/smi.c | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-)
drivers/infiniband/core/smi.c +17 -9
@@ -236,21 +236,20 @@ enum smi_action smi_handle_dr_smp_recv(struct ib_smp *smp, u8 node_type, smp->dr_slid == IB_LID_PERMISSIVE); } -enum smi_forward_action smi_check_forward_dr_smp (struct ib_smp *smp) +static inline +enum smi_forward_action __smi_check_forward_dr_smp (u8 hop_ptr,u8 hop_cnt, +u8 direction, +int dr_dlid_is_permissive, +int dr_slid_is_permissive) { -u8 hop_ptr, hop_cnt; - - hop_ptr = smp->hop_ptr; - hop_cnt = smp->hop_cnt; - -if (!ib_get_smp_direction (smp)) { +if (!direction) {/* C14-9:2 -- intermediate hop */ if (hop_ptr && hop_ptr < hop_cnt)return IB_SMI_FORWARD ;/* C14-9:3 -- at the end of the DR segment of path */ if (hop_ptr == hop_cnt) -return (smp->dr_dlid ==IB_LID_PERMISSIVE ? +return (dr_dlid_is_permissive ?IB_SMI_SEND :IB_SMI_LOCAL );/* C14-9:4 -- hop_ptr = hop_cnt + 1 -> give to SMA/SM */
@@ -263,10 +262,19 @@ enum smi_forward_action smi_check_forward_dr_smp(struct ib_smp *smp)/* C14-13:3 -- at the end of the DR segment of path */ if (hop_ptr ==1 ) -return (smp->dr_slid !=IB_LID_PERMISSIVE ? +return (dr_slid_is_permissive ?IB_SMI_SEND :IB_SMI_LOCAL ); }return IB_SMI_LOCAL ; + +} + +enum smi_forward_action smi_check_forward_dr_smp (struct ib_smp *smp) +{ +return __smi_check_forward_dr_smp (smp->hop_ptr, smp->hop_cnt, +ib_get_smp_direction (smp), + smp->dr_dlid ==IB_LID_PERMISSIVE , + smp->dr_slid !=IB_LID_PERMISSIVE ); }/*
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 11/19 IB/mad: Add helper function for SMI processing
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
This helper function will be used for processing both IB and OPA SMPs.
--- drivers/infiniband/core/mad.c | 85 +++++++++++++++++++++++++------------------ 1 file changed, 49 insertions(+), 36 deletions(-)
drivers/infiniband/core/mad.c +49 -36
@@ -1930,6 +1930,52 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, } } +enum smi_action handle_ib_smi (struct ib_mad_port_private *port_priv, +struct ib_mad_qp_info *qp_info, +struct ib_wc *wc, +int port_num, +struct ib_mad_private *recv, +struct ib_mad_private *response) +{ +enum smi_forward_action retsmi; + +if (smi_handle_dr_smp_recv (&recv->mad.smp, + port_priv->device->node_type, + port_num, + port_priv->device->phys_port_cnt) == +IB_SMI_DISCARD ) +return IB_SMI_DISCARD ; + + retsmi =smi_check_forward_dr_smp (&recv->mad.smp); +if (retsmi ==IB_SMI_LOCAL ) +return IB_SMI_HANDLE ; + +if (retsmi ==IB_SMI_SEND ) {/* don't forward */ +if (smi_handle_dr_smp_send (&recv->mad.smp, + port_priv->device->node_type, + port_num) ==IB_SMI_DISCARD ) +return IB_SMI_DISCARD ; + +if (smi_check_local_smp (&recv->mad.smp, port_priv->device) ==IB_SMI_DISCARD ) +return IB_SMI_DISCARD ; + }else if (port_priv->device->node_type ==RDMA_NODE_IB_SWITCH ) { +/* forward case for switches */ +memcpy (response, recv,sizeof (*response)); + response->header.recv_wc.wc = &response->header.wc; + response->header.recv_wc.recv_buf.mad = &response->mad.mad; + response->header.recv_wc.recv_buf.grh = &response->grh; + +agent_send_response (&response->mad.mad, + &response->grh, wc, + port_priv->device, +smi_get_fwd_port (&recv->mad.smp), + qp_info->qp->qp_num); + +return IB_SMI_DISCARD ; + } +return IB_SMI_HANDLE ; +} +static size_t mad_recv_buf_size (struct ib_device *dev) {return (sizeof (struct ib_grh ) + dev->cached_dev_attrs.max_mad_size);
@@ -2006,45 +2052,12 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv,if (recv->mad.mad.mad_hdr.mgmt_class ==IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE ) { -enum smi_forward_action retsmi; - -if (smi_handle_dr_smp_recv (&recv->mad.smp, - port_priv->device->node_type, - port_num, - port_priv->device->phys_port_cnt) == -IB_SMI_DISCARD ) +if (handle_ib_smi (port_priv, qp_info, wc, port_num, recv, + response) + ==IB_SMI_DISCARD ) goto out; - - retsmi =smi_check_forward_dr_smp (&recv->mad.smp); -if (retsmi ==IB_SMI_LOCAL ) - goto local; - -if (retsmi ==IB_SMI_SEND ) {/* don't forward */ -if (smi_handle_dr_smp_send (&recv->mad.smp, - port_priv->device->node_type, - port_num) ==IB_SMI_DISCARD ) - goto out; - -if (smi_check_local_smp (&recv->mad.smp, port_priv->device) ==IB_SMI_DISCARD ) - goto out; - }else if (port_priv->device->node_type ==RDMA_NODE_IB_SWITCH ) { -/* forward case for switches */ -memcpy (response, recv,sizeof (*response)); - response->header.recv_wc.wc = &response->header.wc; - response->header.recv_wc.recv_buf.mad = &response->mad.mad; - response->header.recv_wc.recv_buf.grh = &response->grh; - -agent_send_response (&response->mad.mad, - &response->grh, wc, - port_priv->device, -smi_get_fwd_port (&recv->mad.smp), - qp_info->qp->qp_num); - - goto out; - } } -local:/* Give driver "right of first refusal" on incoming MAD */ if (port_priv->device->process_mad) { ret = port_priv->device->process_mad(port_priv->device, 0,
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 12/19 IB/mad: Add MAD size parameters to process_mad
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
In support of variable length MADs add in and out MAD size parameters to the process_mad call.
The out MAD size parameter is passed by reference such that it can be updated by the agent to indicate the proper response length to be sent by the MAD stack.
The in and out MAD parameters are made generic by specifying them as ib_mad_hdr.
Drivers are modified as needed.
--- drivers/infiniband/core/mad.c | 30 ++++++++++++++++++---------- drivers/infiniband/core/sysfs.c | 5 ++++- drivers/infiniband/hw/amso1100/c2_provider.c | 5 ++++- drivers/infiniband/hw/cxgb3/iwch_provider.c | 5 ++++- drivers/infiniband/hw/cxgb4/provider.c | 7 +++++-- drivers/infiniband/hw/ehca/ehca_iverbs.h | 4 ++-- drivers/infiniband/hw/ehca/ehca_sqp.c | 8 +++++++- drivers/infiniband/hw/ipath/ipath_mad.c | 8 +++++++- drivers/infiniband/hw/ipath/ipath_verbs.h | 3 ++- drivers/infiniband/hw/mlx4/mad.c | 9 ++++++++- drivers/infiniband/hw/mlx4/mlx4_ib.h | 3 ++- drivers/infiniband/hw/mlx5/mad.c | 8 +++++++- drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 ++- drivers/infiniband/hw/mthca/mthca_dev.h | 4 ++-- drivers/infiniband/hw/mthca/mthca_mad.c | 9 +++++++-- drivers/infiniband/hw/nes/nes_verbs.c | 3 ++- drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 3 ++- drivers/infiniband/hw/ocrdma/ocrdma_ah.h | 3 ++- drivers/infiniband/hw/qib/qib_mad.c | 8 +++++++- drivers/infiniband/hw/qib/qib_verbs.h | 3 ++- include/rdma/ib_verbs.h | 8 +++++--- 21 files changed, 103 insertions(+), 36 deletions(-)
drivers/infiniband/core/mad.c +20 -10
@@ -715,11 +715,12 @@ static void build_smp_wc(struct ib_qp *qp, wc->port_num = port_num; } -static struct ib_mad_private *alloc_mad_priv (struct ib_device *dev) +static struct ib_mad_private *alloc_mad_priv (struct ib_device *dev, +size_t *mad_size) { + *mad_size = dev->cached_dev_attrs.max_mad_size;return (kmalloc (sizeof (struct ib_mad_private_header ) + -sizeof (struct ib_grh ) + - dev->cached_dev_attrs.max_mad_size,GFP_ATOMIC )); +sizeof (struct ib_grh ) + *mad_size,GFP_ATOMIC )); }/*
@@ -741,6 +742,8 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,u8 port_num;struct ib_wc mad_wc;struct ib_send_wr *send_wr = &mad_send_wr->send_wr; +size_t in_mad_size = mad_agent_priv->agent.device->cached_dev_attrs.max_mad_size; +size_t out_mad_size;if (device->node_type ==RDMA_NODE_IB_SWITCH && smp->mgmt_class ==IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE )
@@ -777,7 +780,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv, local->mad_priv =NULL ; local->recv_mad_agent =NULL ; - mad_priv =alloc_mad_priv (mad_agent_priv->agent.device); + mad_priv =alloc_mad_priv (mad_agent_priv->agent.device, &out_mad_size);if (!mad_priv) { ret = -ENOMEM ;dev_err (&device->dev,"No memory for local response MAD\n" );
@@ -792,8 +795,9 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,/* No GRH for DR SMP */ ret = device->process_mad (device,0 , port_num, &mad_wc,NULL , - (struct ib_mad *)smp, - (struct ib_mad *)&mad_priv->mad); + (struct ib_mad_hdr *)smp, in_mad_size, + (struct ib_mad_hdr *)&mad_priv->mad, + &out_mad_size);switch (ret) {case IB_MAD_RESULT_SUCCESS |IB_MAD_RESULT_REPLY :
@@ -2011,6 +2015,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv,struct ib_mad_agent_private *mad_agent;int port_num;int ret =IB_MAD_RESULT_SUCCESS ; +size_t resp_mad_size; mad_list = (struct ib_mad_list_head *)(unsigned long )wc->wr_id; qp_info = mad_list->mad_queue->qp_info;
@@ -2038,7 +2043,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv,if (!validate_mad (&recv->mad.mad.mad_hdr, qp_info->qp->qp_num)) goto out; - response =alloc_mad_priv (port_priv->device); + response =alloc_mad_priv (port_priv->device, &resp_mad_size);if (!response) {dev_err (&port_priv->device->dev,"ib_mad_recv_done_handler no memory for response buffer\n" );
@@ -2063,8 +2068,10 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, ret = port_priv->device->process_mad (port_priv->device,0 , port_priv->port_num, wc, &recv->grh, - &recv->mad.mad, - &response->mad.mad); + (struct ib_mad_hdr *)&recv->mad.mad, + port_priv->device->cached_dev_attrs.max_mad_size, + (struct ib_mad_hdr *)&response->mad.mad, + &resp_mad_size);if (ret &IB_MAD_RESULT_SUCCESS ) {if (ret &IB_MAD_RESULT_CONSUMED ) goto out;
@@ -2687,7 +2694,10 @@ static int ib_mad_post_receive_mads(struct ib_mad_qp_info *qp_info, mad_priv = mad; mad =NULL ; }else { - mad_priv =alloc_mad_priv (qp_info->port_priv->device); +size_t mad_size; + + mad_priv =alloc_mad_priv (qp_info->port_priv->device, + &mad_size);if (!mad_priv) {dev_err (&qp_info->port_priv->device->dev,"No memory for receive buffer\n" );
drivers/infiniband/core/sysfs.c +4 -1
@@ -326,6 +326,7 @@ static ssize_t show_pma_counter(struct ib_port *p, struct port_attribute *attr,int width = (tab_attr->index >>16 ) &0xff ;struct ib_mad *in_mad =NULL ;struct ib_mad *out_mad =NULL ; +size_t out_mad_size =sizeof (*out_mad);ssize_t ret;if (!p->ibdev->process_mad)
@@ -347,7 +348,9 @@ static ssize_t show_pma_counter(struct ib_port *p, struct port_attribute *attr, in_mad->data[41 ] = p->port_num;/* PortSelect field */ if ((p->ibdev->process_mad (p->ibdev,IB_MAD_IGNORE_MKEY , - p->port_num,NULL ,NULL , in_mad, out_mad) & + p->port_num,NULL ,NULL , + (struct ib_mad_hdr *)in_mad,sizeof (*in_mad), + (struct ib_mad_hdr *)out_mad, &out_mad_size) & (IB_MAD_RESULT_SUCCESS |IB_MAD_RESULT_REPLY )) != (IB_MAD_RESULT_SUCCESS |IB_MAD_RESULT_REPLY )) { ret = -EINVAL ;
drivers/infiniband/hw/amso1100/c2_provider.c +4 -1
@@ -584,7 +584,10 @@ static int c2_process_mad(struct ib_device *ibdev,u8 port_num, struct ib_wc *in_wc, struct ib_grh *in_grh, - struct ib_mad *in_mad, struct ib_mad *out_mad) + struct ib_mad_hdr *in_mad, + size_t in_mad_size, + struct ib_mad_hdr *out_mad, + size_t *out_mad_size) { pr_debug("%s:%u\n", __func__, __LINE__);return -ENOSYS ;
drivers/infiniband/hw/cxgb3/iwch_provider.c +4 -1
@@ -87,7 +87,10 @@ static int iwch_process_mad(struct ib_device *ibdev,u8 port_num, struct ib_wc *in_wc, struct ib_grh *in_grh, - struct ib_mad *in_mad, struct ib_mad *out_mad) + struct ib_mad_hdr *in_mad, + size_t in_mad_size, + struct ib_mad_hdr *out_mad, + size_t *out_mad_size) { return -ENOSYS; }
drivers/infiniband/hw/cxgb4/provider.c +5 -2
@@ -81,8 +81,11 @@ static int c4iw_multicast_detach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)static int c4iw_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num,struct ib_wc *in_wc, -struct ib_grh *in_grh,struct ib_mad *in_mad, -struct ib_mad *out_mad) +struct ib_grh *in_grh, +struct ib_mad_hdr *in_mad, +size_t in_mad_size, +struct ib_mad_hdr *out_mad, +size_t *out_mad_size) {return -ENOSYS ; }
drivers/infiniband/hw/ehca/ehca_iverbs.h +2 -2
@@ -189,8 +189,8 @@ int ehca_mmap(struct ib_ucontext *context, struct vm_area_struct *vma);int ehca_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num,struct ib_wc *in_wc,struct ib_grh *in_grh, -struct ib_mad *in_mad, -struct ib_mad *out_mad); +struct ib_mad_hdr *in,size_t in_mad_size, +struct ib_mad_hdr *out,size_t *out_mad_size);void ehca_poll_eqs (unsigned long data);
drivers/infiniband/hw/ehca/ehca_sqp.c +7 -1
@@ -218,9 +218,15 @@ perf_reply:int ehca_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num,struct ib_wc *in_wc,struct ib_grh *in_grh, -struct ib_mad *in_mad,struct ib_mad *out_mad) +struct ib_mad_hdr *in,size_t in_mad_size, +struct ib_mad_hdr *out,size_t *out_mad_size) {int ret; +struct ib_mad *in_mad = (struct ib_mad *)in; +struct ib_mad *out_mad = (struct ib_mad *)out; + +if (in_mad_size !=sizeof (*in_mad) || *out_mad_size !=sizeof (*out_mad)) +return IB_MAD_RESULT_FAILURE ;if (!port_num || port_num > ibdev->phys_port_cnt || !in_wc)return IB_MAD_RESULT_FAILURE ;
drivers/infiniband/hw/ipath/ipath_mad.c +7 -1
@@ -1491,9 +1491,15 @@ bail:*/ int ipath_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num,struct ib_wc *in_wc,struct ib_grh *in_grh, -struct ib_mad *in_mad,struct ib_mad *out_mad) +struct ib_mad_hdr *in,size_t in_mad_size, +struct ib_mad_hdr *out,size_t *out_mad_size) {int ret; +struct ib_mad *in_mad = (struct ib_mad *)in; +struct ib_mad *out_mad = (struct ib_mad *)out; + +if (in_mad_size !=sizeof (*in_mad) || *out_mad_size !=sizeof (*out_mad)) +return IB_MAD_RESULT_FAILURE ;switch (in_mad->mad_hdr.mgmt_class) {case IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE :
drivers/infiniband/hw/ipath/ipath_verbs.h +2 -1
@@ -703,7 +703,8 @@ int ipath_process_mad(struct ib_device *ibdev,u8 port_num, struct ib_wc *in_wc, struct ib_grh *in_grh, - struct ib_mad *in_mad, struct ib_mad *out_mad); + struct ib_mad_hdr *in, size_t in_mad_size, + struct ib_mad_hdr *out, size_t *out_mad_size);/* * Compare the lower 24 bits of the two values.
drivers/infiniband/hw/mlx4/mad.c +8 -1
@@ -855,8 +855,15 @@ static int iboe_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,int mlx4_ib_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num,struct ib_wc *in_wc,struct ib_grh *in_grh, -struct ib_mad *in_mad,struct ib_mad *out_mad) +struct ib_mad_hdr *in,size_t in_mad_size, +struct ib_mad_hdr *out,size_t *out_mad_size) { +struct ib_mad *in_mad = (struct ib_mad *)in; +struct ib_mad *out_mad = (struct ib_mad *)out; + +if (in_mad_size !=sizeof (*in_mad) || *out_mad_size !=sizeof (*out_mad)) +return IB_MAD_RESULT_FAILURE ; +switch (rdma_port_get_link_layer (ibdev, port_num)) {case IB_LINK_LAYER_INFINIBAND : return ib_process_mad(ibdev, mad_flags, port_num, in_wc,
drivers/infiniband/hw/mlx4/mlx4_ib.h +2 -1
@@ -690,7 +690,8 @@ int mlx4_MAD_IFC(struct mlx4_ib_dev *dev, int mad_ifc_flags, void *in_mad, void *response_mad);int mlx4_ib_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num,struct ib_wc *in_wc,struct ib_grh *in_grh, -struct ib_mad *in_mad,struct ib_mad *out_mad); +struct ib_mad_hdr *in,size_t in_mad_size, +struct ib_mad_hdr *out,size_t *out_mad_size);int mlx4_ib_mad_init (struct mlx4_ib_dev *dev);void mlx4_ib_mad_cleanup (struct mlx4_ib_dev *dev);
drivers/infiniband/hw/mlx5/mad.c +7 -1
@@ -59,10 +59,16 @@ int mlx5_MAD_IFC(struct mlx5_ib_dev *dev, int ignore_mkey, int ignore_bkey,int mlx5_ib_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num,struct ib_wc *in_wc,struct ib_grh *in_grh, -struct ib_mad *in_mad,struct ib_mad *out_mad) +struct ib_mad_hdr *in,size_t in_mad_size, +struct ib_mad_hdr *out,size_t *out_mad_size) {u16 slid;int err; +struct ib_mad *in_mad = (struct ib_mad *)in; +struct ib_mad *out_mad = (struct ib_mad *)out; + +if (in_mad_size !=sizeof (*in_mad) || *out_mad_size !=sizeof (*out_mad)) +return IB_MAD_RESULT_FAILURE ; slid = in_wc ? in_wc->slid :be16_to_cpu (IB_LID_PERMISSIVE );
drivers/infiniband/hw/mlx5/mlx5_ib.h +2 -1
@@ -589,7 +589,8 @@ int mlx5_ib_unmap_fmr(struct list_head *fmr_list);int mlx5_ib_fmr_dealloc (struct ib_fmr *ibfmr);int mlx5_ib_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num,struct ib_wc *in_wc,struct ib_grh *in_grh, -struct ib_mad *in_mad,struct ib_mad *out_mad); +struct ib_mad_hdr *in,size_t in_mad_size, +struct ib_mad_hdr *out,size_t *out_mad_size);struct ib_xrcd *mlx5_ib_alloc_xrcd (struct ib_device *ibdev,struct ib_ucontext *context,struct ib_udata *udata);
drivers/infiniband/hw/mthca/mthca_dev.h +2 -2
@@ -578,8 +578,8 @@ int mthca_process_mad(struct ib_device *ibdev,u8 port_num, struct ib_wc *in_wc, struct ib_grh *in_grh, - struct ib_mad *in_mad, - struct ib_mad *out_mad); + struct ib_mad_hdr *in, size_t in_mad_size, + struct ib_mad_hdr *out, size_t *out_mad_size);int mthca_create_agents (struct mthca_dev *dev);void mthca_free_agents (struct mthca_dev *dev);
drivers/infiniband/hw/mthca/mthca_mad.c +7 -2
@@ -197,13 +197,18 @@ int mthca_process_mad(struct ib_device *ibdev,u8 port_num, struct ib_wc *in_wc, struct ib_grh *in_grh, - struct ib_mad *in_mad, - struct ib_mad *out_mad) + struct ib_mad_hdr *in, size_t in_mad_size, + struct ib_mad_hdr *out, size_t *out_mad_size) { int err;u16 slid = in_wc ? in_wc->slid :be16_to_cpu (IB_LID_PERMISSIVE );u16 prev_lid =0 ;struct ib_port_attr pattr; +struct ib_mad *in_mad = (struct ib_mad *)in; +struct ib_mad *out_mad = (struct ib_mad *)out; + +if (in_mad_size !=sizeof (*in_mad) || *out_mad_size !=sizeof (*out_mad)) +return IB_MAD_RESULT_FAILURE ;/* Forward locally generated traps to the SM */ if (in_mad->mad_hdr.method == IB_MGMT_METHOD_TRAP &&
drivers/infiniband/hw/nes/nes_verbs.c +2 -1
@@ -3224,7 +3224,8 @@ static int nes_multicast_detach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)*/ static int nes_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num,struct ib_wc *in_wc,struct ib_grh *in_grh, -struct ib_mad *in_mad,struct ib_mad *out_mad) +struct ib_mad_hdr *in,size_t in_mad_size, +struct ib_mad_hdr *out,size_t *out_mad_size) {nes_debug (NES_DBG_INIT ,"\n" );return -ENOSYS ;
drivers/infiniband/hw/ocrdma/ocrdma_ah.c +2 -1
@@ -189,7 +189,8 @@ int ocrdma_process_mad(struct ib_device *ibdev,u8 port_num, struct ib_wc *in_wc, struct ib_grh *in_grh, - struct ib_mad *in_mad, struct ib_mad *out_mad) + struct ib_mad_hdr *in, size_t in_mad_size, + struct ib_mad_hdr *out, size_t *out_mad_size) { return IB_MAD_RESULT_SUCCESS; }
drivers/infiniband/hw/ocrdma/ocrdma_ah.h +2 -1
@@ -38,5 +38,6 @@ int ocrdma_process_mad(struct ib_device *,u8 port_num, struct ib_wc *in_wc, struct ib_grh *in_grh, - struct ib_mad *in_mad, struct ib_mad *out_mad); + struct ib_mad_hdr *in, size_t in_mad_size, + struct ib_mad_hdr *out, size_t *out_mad_size);#endif /* __OCRDMA_AH_H__ */
drivers/infiniband/hw/qib/qib_mad.c +7 -1
@@ -2401,11 +2401,17 @@ bail:*/ int qib_process_mad (struct ib_device *ibdev,int mad_flags,u8 port,struct ib_wc *in_wc,struct ib_grh *in_grh, -struct ib_mad *in_mad,struct ib_mad *out_mad) +struct ib_mad_hdr *in,size_t in_mad_size, +struct ib_mad_hdr *out,size_t *out_mad_size) {int ret;struct qib_ibport *ibp =to_iport (ibdev, port);struct qib_pportdata *ppd =ppd_from_ibp (ibp); +struct ib_mad *in_mad = (struct ib_mad *)in; +struct ib_mad *out_mad = (struct ib_mad *)out; + +if (in_mad_size !=sizeof (*in_mad) || *out_mad_size !=sizeof (*out_mad)) +return IB_MAD_RESULT_FAILURE ;switch (in_mad->mad_hdr.mgmt_class) {case IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE :
drivers/infiniband/hw/qib/qib_verbs.h +2 -1
@@ -873,7 +873,8 @@ void qib_sys_guid_chg(struct qib_ibport *ibp);void qib_node_desc_chg (struct qib_ibport *ibp);int qib_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num,struct ib_wc *in_wc,struct ib_grh *in_grh, -struct ib_mad *in_mad,struct ib_mad *out_mad); +struct ib_mad_hdr *in,size_t in_mad_size, +struct ib_mad_hdr *out,size_t *out_mad_size);int qib_create_agents (struct qib_ibdev *dev);void qib_free_agents (struct qib_ibdev *dev);
include/rdma/ib_verbs.h +5 -3
@@ -1408,7 +1408,7 @@ struct ib_flow {struct ib_uobject *uobject; }; -struct ib_mad ; +struct ib_mad_hdr ;struct ib_grh ;enum ib_process_mad_flags {
@@ -1640,8 +1640,10 @@ struct ib_device {u8 port_num, struct ib_wc *in_wc, struct ib_grh *in_grh, - struct ib_mad *in_mad, - struct ib_mad *out_mad); + struct ib_mad_hdr *in_mad, + size_t in_mad_size, + struct ib_mad_hdr *out_mad, + size_t *out_mad_size);struct ib_xrcd * (*alloc_xrcd)(struct ib_device *device,struct ib_ucontext *ucontext,struct ib_udata *udata);
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-13-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
RE: [PATCH v4 12/19] IB/mad: Add MAD size parameters to process_mad
drivers/infiniband/core/mad.c static void build_smp_wc(struct ib_qp *qp, · line 721
Quoted 5 lines
drivers/infiniband/core/mad.c +3
-1-static struct ib_mad_private *alloc_mad_priv (struct ib_device *dev) +static struct ib_mad_private *alloc_mad_priv (struct ib_device *dev, +size_t *mad_size) { + *mad_size = dev->cached_dev_attrs.max_mad_size;
Why does this function return the value that the caller can just read from the device?
Actually, it's odd for an alloc() call to return how much it allocated, rather than taking that as input.
Quoted 30 lines
drivers/infiniband/core/mad.c +3
-2return (kmalloc (sizeof (struct ib_mad_private_header ) + -sizeof (struct ib_grh ) + - dev->cached_dev_attrs.max_mad_size,GFP_ATOMIC )); +sizeof (struct ib_grh ) + *mad_size,GFP_ATOMIC )); }/* @@ -741,6 +742,8 @@ static int handle_outgoing_dr_smp(structib_mad_agent_private *mad_agent_priv, u8 port_num;struct ib_wc mad_wc;struct ib_send_wr *send_wr = &mad_send_wr->send_wr; +size_t in_mad_size = mad_agent_priv->agent.device- >cached_dev_attrs.max_mad_size; +size_t out_mad_size;drivers/infiniband/core/mad.c +1
-1if (device->node_type ==RDMA_NODE_IB_SWITCH && smp->mgmt_class ==IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE )@@ -777,7 +780,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv, local->mad_priv =NULL ; local->recv_mad_agent =NULL ; - mad_priv =alloc_mad_priv (mad_agent_priv->agent.device); + mad_priv =alloc_mad_priv (mad_agent_priv->agent.device, &out_mad_size);if (!mad_priv) {drivers/infiniband/core/mad.c +0
-0ret = -ENOMEM ;dev_err (&device->dev,"No memory for local response MAD\n" );
drivers/infiniband/core/mad.c static int handle_outgoing_dr_smp(struct · line 801
Quoted 9 lines
drivers/infiniband/core/mad.c +3
-2@@ -792,8 +795,9 @@ static int handle_outgoing_dr_smp(structib_mad_agent_private *mad_agent_priv,/* No GRH for DR SMP */ ret = device->process_mad (device,0 , port_num, &mad_wc,NULL , - (struct ib_mad *)smp, - (struct ib_mad *)&mad_priv->mad); + (struct ib_mad_hdr *)smp, in_mad_size, + (struct ib_mad_hdr *)&mad_priv->mad, + &out_mad_size);
Rather than calling device->process_mad() directly, would it be better to call a common function? So we can avoid adding:
Review
Quoted 6 lines
drivers/infiniband/hw/ehca/ehca_sqp.c +4
-0+struct ib_mad *in_mad = (struct ib_mad *)in; +struct ib_mad *out_mad = (struct ib_mad *)out; + + if (in_mad_size != sizeof(*in_mad) || *out_mad_size !=
sizeof(*out_mad))(quoted diff) +1
-0+ return IB_MAD_RESULT_FAILURE;
to existing drivers?
Quoted 24 lines
drivers/infiniband/core/mad.c +1
-0switch (ret) {case IB_MAD_RESULT_SUCCESS |IB_MAD_RESULT_REPLY :@@ -2011,6 +2015,7 @@ static void ib_mad_recv_done_handler(structib_mad_port_private *port_priv, struct ib_mad_agent_private *mad_agent;int port_num;int ret =IB_MAD_RESULT_SUCCESS ; +size_t resp_mad_size; mad_list = (struct ib_mad_list_head *)(unsigned long )wc->wr_id;drivers/infiniband/core/mad.c +1
-1qp_info = mad_list->mad_queue->qp_info;@@ -2038,7 +2043,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, if (!validate_mad(&recv->mad.mad.mad_hdr, qp_info->qp->qp_num)) goto out; - response =alloc_mad_priv (port_priv->device); + response =alloc_mad_priv (port_priv->device, &resp_mad_size);if (!response) { dev_err(&port_priv->device->dev,"
ib_mad_recv_done_handlerno memory for response
buffer\n");
drivers/infiniband/core/mad.c static void ib_mad_recv_done_handler(struct · line 2075
Quoted 10 lines
drivers/infiniband/core/mad.c +2
-2@@ -2063,8 +2068,10 @@ static void ib_mad_recv_done_handler(structib_mad_port_private *port_priv, ret = port_priv->device->process_mad(port_priv->device, 0, port_priv->port_num, wc, &recv->grh, - &recv->mad.mad, - &response->mad.mad); + (struct ib_mad_hdr *)&recv- >mad.mad, + port_priv->device- >cached_dev_attrs.max_mad_size,
This is the size of the allocated buffer. Something based on wc.byte_len seems like a better option.
Quoted 18 lines
(quoted diff) +1
-0+ (struct ib_mad_hdr *)&response-Quoted 1 line
mad.mad,
drivers/infiniband/core/mad.c +5
-1+ &resp_mad_size);if (ret &IB_MAD_RESULT_SUCCESS ) {if (ret &IB_MAD_RESULT_CONSUMED ) goto out;@@ -2687,7 +2694,10 @@ static int ib_mad_post_receive_mads(structib_mad_qp_info *qp_info, mad_priv = mad; mad =NULL ; }else { - mad_priv =alloc_mad_priv (qp_info->port_priv->device); +size_t mad_size; + + mad_priv =alloc_mad_priv (qp_info->port_priv->device, + &mad_size);if (!mad_priv) { dev_err(&qp_info->port_priv->device->dev,
Quoted 1 line
"No memory for receive buffer\n");
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBD692-P5GAC/[email protected]
Re: [PATCH v4 12/19] IB/mad: Add MAD size parameters to process_mad
Quoted 9 lines · Sean
drivers/infiniband/core/mad.c static void build_smp_wc(struct ib_qp *qp, · line 721
Quoted 5 lines
drivers/infiniband/core/mad.c +3
-1-static struct ib_mad_private *alloc_mad_priv (struct ib_device *dev) +static struct ib_mad_private *alloc_mad_priv (struct ib_device *dev, +size_t *mad_size) { + *mad_size = dev->cached_dev_attrs.max_mad_size;Why does this function return the value that the caller can just read from the device?
Actually, it's odd for an
alloc()call to return how much it allocated, rather than taking that as input.
True.
This was done just for the convenience of the callers. The previous patch used ib_device to determine the size I was just trying to keep that semantic with the function "helping" the caller...
I'll change this to:
static struct ib_mad_private *alloc_mad_priv(size_t mad_size, gfp_t flags);
which also addresses Dougs comments regarding GFP_ATOMIC.
Quoted 43 lines
Quoted 31 lines
drivers/infiniband/core/mad.c +3
-2return (kmalloc (sizeof (struct ib_mad_private_header ) + -sizeof (struct ib_grh ) + - dev->cached_dev_attrs.max_mad_size,GFP_ATOMIC )); +sizeof (struct ib_grh ) + *mad_size,GFP_ATOMIC )); }/* @@ -741,6 +742,8 @@ static int handle_outgoing_dr_smp(structib_mad_agent_private *mad_agent_priv, u8 port_num;struct ib_wc mad_wc;struct ib_send_wr *send_wr = &mad_send_wr->send_wr; +size_t in_mad_size = mad_agent_priv->agent.device- >cached_dev_attrs.max_mad_size; +size_t out_mad_size;drivers/infiniband/core/mad.c +1
-1if (device->node_type ==RDMA_NODE_IB_SWITCH && smp->mgmt_class ==IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE )@@ -777,7 +780,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv, local->mad_priv =NULL ; local->recv_mad_agent =NULL ; - mad_priv =alloc_mad_priv (mad_agent_priv->agent.device); + mad_priv =alloc_mad_priv (mad_agent_priv->agent.device, &out_mad_size);if (!mad_priv) {drivers/infiniband/core/mad.c +0
-0ret = -ENOMEM ;dev_err (&device->dev,"No memory for local response MAD\n" );drivers/infiniband/core/mad.c static int handle_outgoing_dr_smp(struct · line 801
Quoted 9 lines
drivers/infiniband/core/mad.c +3
-2@@ -792,8 +795,9 @@ static int handle_outgoing_dr_smp(structib_mad_agent_private *mad_agent_priv,/* No GRH for DR SMP */ ret = device->process_mad (device,0 , port_num, &mad_wc,NULL , - (struct ib_mad *)smp, - (struct ib_mad *)&mad_priv->mad); + (struct ib_mad_hdr *)smp, in_mad_size, + (struct ib_mad_hdr *)&mad_priv->mad, + &out_mad_size);Rather than calling
device->process_mad()directly, would it be better to call a common function? So we can avoid adding:
Quoted 8 lines
Review
Quoted 6 lines
drivers/infiniband/hw/ehca/ehca_sqp.c +4
-0+struct ib_mad *in_mad = (struct ib_mad *)in; +struct ib_mad *out_mad = (struct ib_mad *)out; + + if (in_mad_size != sizeof(*in_mad) || *out_mad_size !=
sizeof(*out_mad))(quoted diff) +1
-0+ return IB_MAD_RESULT_FAILURE;to existing drivers?
No. The checks need to be done by the devices. IB devices expect exactly 256 bytes, OPA devices expect 2K, while some drivers don't care at all (don't support MADs).
However, upon further reflection these checks indicate a programming error in which the device is indicating the wrong mad size or there is a bug in the MAD module. Therefore these should be WARN_ON.
Quoted 33 lines
Quoted 33 lines
drivers/infiniband/core/mad.c +1
-0switch (ret) {case IB_MAD_RESULT_SUCCESS |IB_MAD_RESULT_REPLY :@@ -2011,6 +2015,7 @@ static void ib_mad_recv_done_handler(structib_mad_port_private *port_priv, struct ib_mad_agent_private *mad_agent;int port_num;int ret =IB_MAD_RESULT_SUCCESS ; +size_t resp_mad_size; mad_list = (struct ib_mad_list_head *)(unsigned long )wc->wr_id;drivers/infiniband/core/mad.c +1
-1qp_info = mad_list->mad_queue->qp_info;@@ -2038,7 +2043,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, if (!validate_mad(&recv->mad.mad.mad_hdr, qp_info->qp->qp_num)) goto out; - response =alloc_mad_priv (port_priv->device); + response =alloc_mad_priv (port_priv->device, &resp_mad_size);if (!response) { dev_err(&port_priv->device->dev,"
ib_mad_recv_done_handlerno memory for response
buffer\n");drivers/infiniband/core/mad.c +1
-2@@ -2063,8 +2068,10 @@ static void ib_mad_recv_done_handler(structib_mad_port_private *port_priv, ret = port_priv->device->process_mad(port_priv->device, 0, port_priv->port_num, wc, &recv->grh, - &recv->mad.mad, - &response->mad.mad); + (struct ib_mad_hdr *)&recv-
Quoted 5 lines
Review static void ib_mad_recv_done_handler(struct · line 2075
Quoted 3 lines
(quoted diff) +1
-0>mad.mad, + port_priv->device- >cached_dev_attrs.max_mad_size,This is the size of the allocated buffer.
Yes
Quoted 1 line
Something based on wc.byte_len seems like a better option.
The IB process_mad calls currently have no size checks and expect the full IB MAD... I did not want to break that semantic and risk introducing bugs in those drivers.
While it seems byte_len should be sufficient I can't test for all devices.
Furthermore for OPA devices there is no issue with passing the full buffer size. So I prefer to leave this as is.
Ira
Quoted 13 lines
Quoted 13 lines
(quoted diff) +1
-0+ (struct ib_mad_hdr *)&response-Quoted 1 line
mad.mad,
drivers/infiniband/core/mad.c +1
-0+ &resp_mad_size);if (ret &IB_MAD_RESULT_SUCCESS ) {if (ret &IB_MAD_RESULT_CONSUMED ) goto out;@@ -2687,7 +2694,10 @@ static int ib_mad_post_receive_mads(structib_mad_qp_info *qp_info, mad_priv = mad; mad =NULL ; }else {
Quoted 8 lines
Quoted 8 lines
drivers/infiniband/core/mad.c +4
-1- mad_priv =alloc_mad_priv (qp_info->port_priv->device); +size_t mad_size; + + mad_priv =alloc_mad_priv (qp_info->port_priv->device, + &mad_size);if (!mad_priv) { dev_err(&qp_info->port_priv->device->dev,"No memory for receive buffer\n");
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20150407172303.GB433-W4f6Xiosr+yv7QzWx2u06xL4W9x8LtSr@public.gmane.org
RE: [PATCH v4 12/19] IB/mad: Add MAD size parameters to process_mad
Quoted 12 lines
Quoted 1 line
Rather than calling
device->process_mad()directly, would it be betterto call a common function? So we can avoid adding:
Quoted 9 lines
Review
Quoted 7 lines
drivers/infiniband/hw/ehca/ehca_sqp.c +4
-0+struct ib_mad *in_mad = (struct ib_mad *)in; +struct ib_mad *out_mad = (struct ib_mad *)out; + + if (in_mad_size != sizeof(*in_mad) || *out_mad_size !=
sizeof(*out_mad))(quoted diff) +1
-0+ return IB_MAD_RESULT_FAILURE;to existing drivers?
Quoted 4 lines
No. The checks need to be done by the devices. IB devices expect exactly 256
bytes, OPA devices expect 2K, while some drivers don't care at all (don't support MADs).
The checks are per device, but that doesn't mean that every driver has to do the same check.
ib_process_mad(...)
{
if (ib_device(...))
insert check here
else if (opa_device(...))
insert some other check here
else
hit caller on nose
dev->process_mad(...)
}I'm fine either way. It's more a matter of how much trust is given to the other kernel components.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBE224-P5GAC/[email protected]
Re: [PATCH v4 12/19] IB/mad: Add MAD size parameters to process_mad
Quoted 19 lines · Sean
Quoted 12 lines
Quoted 1 line
Rather than calling
device->process_mad()directly, would it be betterto call a common function? So we can avoid adding:
Quoted 9 lines
Review
Quoted 7 lines
(quoted diff) +4
-0+ struct ib_mad *in_mad = (struct ib_mad *)in; + struct ib_mad *out_mad = (struct ib_mad *)out; + + if (in_mad_size != sizeof(*in_mad) || *out_mad_size !=
sizeof(*out_mad))(quoted diff) +1
-0+ return IB_MAD_RESULT_FAILURE;to existing drivers?
Quoted 4 lines
No. The checks need to be done by the devices. IB devices expect exactly 256
bytes, OPA devices expect 2K, while some drivers don't care at all (don't support MADs).The checks are per device, but that doesn't mean that every driver has to do the same check.
Quoted 8 lines
ib_process_mad(...)
{
if (ib_device(...))
insert check here
else if (opa_device(...))
insert some other check here
else
hit caller on nose
See this is another Oddity in the stack. Those devices which are neither OPA nor IB (iWarp or RoCE) have some support for MADs so we can't just "hit the caller on the nose.
Quoted 4 lines
dev->process_mad(...) }I'm fine either way. It's more a matter of how much trust is given to the other kernel components.
Indeed, I debated if I should add such checks at all. In the end I have very little trust so I did... ;-)
However, to better reflect the nature of the error I've changed them all to WARN_ONs.
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 13/19 IB/mad: Add base version parameter to ib_create_send_mad
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
In preparation to support the new OPA MAD Base version, add a base version parameter to ib_create_send_mad and set it to IB_MGMT_BASE_VERSION for current users.
Definition of the new base version and it's processing will occur in later patches.
--- drivers/infiniband/core/agent.c | 3 ++- drivers/infiniband/core/cm.c | 6 ++++-- drivers/infiniband/core/mad.c | 3 ++- drivers/infiniband/core/mad_rmpp.c | 6 ++++-- drivers/infiniband/core/sa_query.c | 3 ++- drivers/infiniband/core/user_mad.c | 3 ++- drivers/infiniband/hw/mlx4/mad.c | 3 ++- drivers/infiniband/hw/mthca/mthca_mad.c | 3 ++- drivers/infiniband/hw/qib/qib_iba7322.c | 3 ++- drivers/infiniband/hw/qib/qib_mad.c | 3 ++- drivers/infiniband/ulp/srpt/ib_srpt.c | 3 ++- include/rdma/ib_mad.h | 4 +++- 12 files changed, 29 insertions(+), 14 deletions(-)
drivers/infiniband/core/agent.c +2 -1
@@ -108,7 +108,8 @@ void agent_send_response(struct ib_mad *mad, struct ib_grh *grh, send_buf =ib_create_send_mad (agent, wc->src_qp, wc->pkey_index,0 ,IB_MGMT_MAD_HDR ,IB_MGMT_MAD_DATA , -GFP_KERNEL ); +GFP_KERNEL , +IB_MGMT_BASE_VERSION );if (IS_ERR (send_buf)) {dev_err (&device->dev,"ib_create_send_mad error\n" ); goto err1;
drivers/infiniband/core/cm.c +4 -2
@@ -267,7 +267,8 @@ static int cm_alloc_msg(struct cm_id_private *cm_id_priv, m =ib_create_send_mad (mad_agent, cm_id_priv->id.remote_cm_qpn, cm_id_priv->av.pkey_index,0 ,IB_MGMT_MAD_HDR ,IB_MGMT_MAD_DATA , -GFP_ATOMIC ); +GFP_ATOMIC , +IB_MGMT_BASE_VERSION );if (IS_ERR (m)) {ib_destroy_ah (ah);return PTR_ERR (m);
@@ -297,7 +298,8 @@ static int cm_alloc_response_msg(struct cm_port *port, m =ib_create_send_mad (port->mad_agent,1 , mad_recv_wc->wc->pkey_index,0 ,IB_MGMT_MAD_HDR ,IB_MGMT_MAD_DATA , -GFP_ATOMIC ); +GFP_ATOMIC , +IB_MGMT_BASE_VERSION );if (IS_ERR (m)) {ib_destroy_ah (ah);return PTR_ERR (m);
drivers/infiniband/core/mad.c +2 -1
@@ -930,7 +930,8 @@ struct ib_mad_send_buf * ib_create_send_mad(struct ib_mad_agent *mad_agent, u32 remote_qpn, u16 pkey_index, int rmpp_active, int hdr_len, int data_len, - gfp_t gfp_mask) + gfp_t gfp_mask, + u8 base_version) { struct ib_mad_agent_private *mad_agent_priv;struct ib_mad_send_wr_private *mad_send_wr;
drivers/infiniband/core/mad_rmpp.c +4 -2
@@ -139,7 +139,8 @@ static void ack_recv(struct mad_rmpp_recv *rmpp_recv, hdr_len =ib_get_mad_data_offset (recv_wc->recv_buf.mad->mad_hdr.mgmt_class); msg =ib_create_send_mad (&rmpp_recv->agent->agent, recv_wc->wc->src_qp, recv_wc->wc->pkey_index,1 , hdr_len, -0 ,GFP_KERNEL ); +0 ,GFP_KERNEL , +IB_MGMT_BASE_VERSION );if (IS_ERR (msg))return ;
@@ -165,7 +166,8 @@ static struct ib_mad_send_buf *alloc_response_msg(struct ib_mad_agent *agent, hdr_len =ib_get_mad_data_offset (recv_wc->recv_buf.mad->mad_hdr.mgmt_class); msg =ib_create_send_mad (agent, recv_wc->wc->src_qp, recv_wc->wc->pkey_index,1 , - hdr_len,0 ,GFP_KERNEL ); + hdr_len,0 ,GFP_KERNEL , +IB_MGMT_BASE_VERSION );if (IS_ERR (msg))ib_destroy_ah (ah);else {
drivers/infiniband/core/sa_query.c +2 -1
@@ -583,7 +583,8 @@ static int alloc_mad(struct ib_sa_query *query, gfp_t gfp_mask) query->mad_buf =ib_create_send_mad (query->port->agent,1 , query->sm_ah->pkey_index,0 ,IB_MGMT_SA_HDR ,IB_MGMT_SA_DATA , - gfp_mask); + gfp_mask, +IB_MGMT_BASE_VERSION );if (IS_ERR (query->mad_buf)) {kref_put (&query->sm_ah->ref, free_sm_ah);return -ENOMEM ;
drivers/infiniband/core/user_mad.c +2 -1
@@ -521,7 +521,8 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf, packet->msg =ib_create_send_mad (agent,be32_to_cpu (packet->mad.hdr.qpn), packet->mad.hdr.pkey_index, rmpp_active, - hdr_len, data_len,GFP_KERNEL ); + hdr_len, data_len,GFP_KERNEL , +IB_MGMT_BASE_VERSION );if (IS_ERR (packet->msg)) { ret =PTR_ERR (packet->msg); goto err_ah;
drivers/infiniband/hw/mlx4/mad.c +2 -1
@@ -358,7 +358,8 @@ static void forward_trap(struct mlx4_ib_dev *dev, u8 port_num, struct ib_mad *maif (agent) { send_buf =ib_create_send_mad (agent, qpn,0 ,0 ,IB_MGMT_MAD_HDR , -IB_MGMT_MAD_DATA ,GFP_ATOMIC ); +IB_MGMT_MAD_DATA ,GFP_ATOMIC , +IB_MGMT_BASE_VERSION );if (IS_ERR (send_buf))return ;/*
drivers/infiniband/hw/mthca/mthca_mad.c +2 -1
@@ -170,7 +170,8 @@ static void forward_trap(struct mthca_dev *dev,if (agent) { send_buf =ib_create_send_mad (agent, qpn,0 ,0 ,IB_MGMT_MAD_HDR , -IB_MGMT_MAD_DATA ,GFP_ATOMIC ); +IB_MGMT_MAD_DATA ,GFP_ATOMIC , +IB_MGMT_BASE_VERSION );if (IS_ERR (send_buf))return ;/*
drivers/infiniband/hw/qib/qib_iba7322.c +2 -1
@@ -5488,7 +5488,8 @@ static void try_7322_ipg(struct qib_pportdata *ppd) goto retry; send_buf =ib_create_send_mad (agent,0 ,0 ,0 ,IB_MGMT_MAD_HDR , -IB_MGMT_MAD_DATA ,GFP_ATOMIC ); +IB_MGMT_MAD_DATA ,GFP_ATOMIC , +IB_MGMT_BASE_VERSION );if (IS_ERR (send_buf)) goto retry;
drivers/infiniband/hw/qib/qib_mad.c +2 -1
@@ -83,7 +83,8 @@ static void qib_send_trap(struct qib_ibport *ibp, void *data, unsigned len)return ; send_buf =ib_create_send_mad (agent,0 ,0 ,0 ,IB_MGMT_MAD_HDR , -IB_MGMT_MAD_DATA ,GFP_ATOMIC ); +IB_MGMT_MAD_DATA ,GFP_ATOMIC , +IB_MGMT_BASE_VERSION );if (IS_ERR (send_buf))return ;
drivers/infiniband/ulp/srpt/ib_srpt.c +2 -1
@@ -477,7 +477,8 @@ static void srpt_mad_recv_handler(struct ib_mad_agent *mad_agent, rsp =ib_create_send_mad (mad_agent, mad_wc->wc->src_qp, mad_wc->wc->pkey_index,0 ,IB_MGMT_DEVICE_HDR ,IB_MGMT_DEVICE_DATA , -GFP_KERNEL ); +GFP_KERNEL , +IB_MGMT_BASE_VERSION );if (IS_ERR (rsp)) goto err_rsp;
include/rdma/ib_mad.h +3 -1
@@ -619,6 +619,7 @@ int ib_process_mad_wc(struct ib_mad_agent *mad_agent,* automatically adjust the allocated buffer size to account for any * additional padding that may be necessary. * @gfp_mask: GFP mask used for the memory allocation. +* @base_version: Base Version of this MAD * * This routine allocates a MAD for sending. The returned MAD send buffer * will reference a data buffer usable for sending a MAD, along
@@ -634,7 +635,8 @@ struct ib_mad_send_buf *ib_create_send_mad(struct ib_mad_agent *mad_agent, u32 remote_qpn, u16 pkey_index, int rmpp_active, int hdr_len, int data_len, - gfp_t gfp_mask); + gfp_t gfp_mask, + u8 base_version);/** * ib_is_mad_class_rmpp - returns whether given management class
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 14/19 IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
OPA MADs share a common header with IBTA MADs but with a different base version and an extended length. These "jumbo" MADs increase the performance of management traffic.
Sharing a common header with IBTA MADs allows us to share most of the MAD processing code when dealing with OPA MADs in addition to supporting some IBTA MADs on OPA devices.
Add a device capability flag to indicate OPA MAD support on the device.
--- include/rdma/ib_verbs.h | 5 +++++ 1 file changed, 5 insertions(+)
include/rdma/ib_verbs.h +5 -0
@@ -128,6 +128,10 @@ enum ib_device_cap_flags { IB_DEVICE_ON_DEMAND_PAGING = (1<<31), }; +enum ib_device_cap_flags2 { +IB_DEVICE_OPA_MAD_SUPPORT =1 +}; +enum ib_signature_prot_cap {IB_PROT_T10DIF_TYPE_1 =1 ,IB_PROT_T10DIF_TYPE_2 =1 <<1 ,
@@ -210,6 +214,7 @@ struct ib_device_attr {int sig_prot_cap;int sig_guard_cap;struct ib_odp_caps odp_caps; +u64 device_cap_flags2;u32 max_mad_size; };
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-15-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 34 lines · On 2/4/2015 6:29 PM, [email protected] wrote:
From: Ira Weiny <[email protected]>
OPA MADs share a common header with IBTA MADs but with a different base version and an extended length. These "jumbo" MADs increase the performance of management traffic.
Sharing a common header with IBTA MADs allows us to share most of the MAD processing code when dealing with OPA MADs in addition to supporting some IBTA MADs on OPA devices.
Add a device capability flag to indicate OPA MAD support on the device.
Signed-off-by Ira Weiny <[email protected]>--- include/rdma/ib_verbs.h | 5 +++++ 1 file changed, 5 insertions(+)include/rdma/ib_verbs.h +4
-0@@ -128,6 +128,10 @@ enum ib_device_cap_flags { IB_DEVICE_ON_DEMAND_PAGING = (1<<31), }; +enum ib_device_cap_flags2 { +IB_DEVICE_OPA_MAD_SUPPORT =1 +}; +enum ib_signature_prot_cap {IB_PROT_T10DIF_TYPE_1 =1 ,IB_PROT_T10DIF_TYPE_2 =1 <<1 ,
include/rdma/ib_verbs.h struct ib_device_attr { · line 220
Quoted 7 lines
include/rdma/ib_verbs.h +1
-0@@ -210,6 +214,7 @@ struct ib_device_attr {int sig_prot_cap;int sig_guard_cap;struct ib_odp_caps odp_caps; +u64 device_cap_flags2;u32 max_mad_size; };
Why is OPA support determined via a device capability flag ? What are the tradeoffs for doing it this way versus the other choices that have been used in the past for other RDMA technologies like RoCE, iWARP, usNIC, … ?
-- Hal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 54D52589.8020305-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 38 lines · On 2/4/2015 6:29 PM, [email protected] wrote:
Quoted 37 lines
From: Ira Weiny <[email protected]>
OPA MADs share a common header with IBTA MADs but with a different base version and an extended length. These "jumbo" MADs increase the performance of management traffic.
Sharing a common header with IBTA MADs allows us to share most of the MAD processing code when dealing with OPA MADs in addition to supporting some IBTA MADs on OPA devices.
Add a device capability flag to indicate OPA MAD support on the device.
Signed-off-by Ira Weiny <[email protected]>--- include/rdma/ib_verbs.h | 5 +++++ 1 file changed, 5 insertions(+)include/rdma/ib_verbs.h index renamed from include/rdma/ib_verbs.h +0
-03ab4033..2614233 100644
include/rdma/ib_verbs.h +4
-0@@ -128,6 +128,10 @@ enum ib_device_cap_flags { IB_DEVICE_ON_DEMAND_PAGING = (1<<31), }; +enum ib_device_cap_flags2 { +IB_DEVICE_OPA_MAD_SUPPORT =1 +}; +enum ib_signature_prot_cap {IB_PROT_T10DIF_TYPE_1 =1 ,IB_PROT_T10DIF_TYPE_2 =1 <<1 ,@@ -210,6 +214,7 @@ struct ib_device_attr {int sig_prot_cap;int sig_guard_cap;struct ib_odp_caps odp_caps;
Quoted 8 lines
include/rdma/ib_verbs.h struct ib_device_attr { · line 220
Quoted 4 lines
include/rdma/ib_verbs.h +1
-0+u64 device_cap_flags2;u32 max_mad_size; };Why is OPA support determined via a device capability flag ? What are the tradeoffs for doing it this way versus the other choices that have been used in the past for other RDMA technologies like RoCE, iWARP, usNIC, ... ?
None of those technologies use the MAD stack for Subnet Management. Other MAD support is very limited (ie IB compatible PMA queries on the local port only).
Do you have a suggestion for alternatives?
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 2807E5FD2F6FDA4886F6618EAC48510E0CC244A8-8k97q/[email protected]
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 47 lines · Ira
Quoted 38 lines · On 2/4/2015 6:29 PM, [email protected] wrote:
Quoted 37 lines
From: Ira Weiny <[email protected]>
OPA MADs share a common header with IBTA MADs but with a different base version and an extended length. These "jumbo" MADs increase the performance of management traffic.
Sharing a common header with IBTA MADs allows us to share most of the MAD processing code when dealing with OPA MADs in addition to supporting some IBTA MADs on OPA devices.
Add a device capability flag to indicate OPA MAD support on the device.
Signed-off-by Ira Weiny <[email protected]>--- include/rdma/ib_verbs.h | 5 +++++ 1 file changed, 5 insertions(+)include/rdma/ib_verbs.h index renamed from include/rdma/ib_verbs.h +0
-03ab4033..2614233 100644
include/rdma/ib_verbs.h +4
-0@@ -128,6 +128,10 @@ enum ib_device_cap_flags { IB_DEVICE_ON_DEMAND_PAGING = (1<<31), }; +enum ib_device_cap_flags2 { +IB_DEVICE_OPA_MAD_SUPPORT =1 +}; +enum ib_signature_prot_cap {IB_PROT_T10DIF_TYPE_1 =1 ,IB_PROT_T10DIF_TYPE_2 =1 <<1 ,@@ -210,6 +214,7 @@ struct ib_device_attr {int sig_prot_cap;int sig_guard_cap;struct ib_odp_caps odp_caps;Quoted 8 lines
include/rdma/ib_verbs.h struct ib_device_attr { · line 220
Quoted 4 lines
include/rdma/ib_verbs.h +1
-0+u64 device_cap_flags2;u32 max_mad_size; };Why is OPA support determined via a device capability flag ? What are the tradeoffs for doing it this way versus the other choices that have been used in the past for other RDMA technologies like RoCE, iWARP, usNIC, ... ?
Quoted 3 lines
None of those technologies use the MAD stack for Subnet Management. Other MAD support is very limited (ie IB compatible PMA queries on the local port only).
Do you have a suggestion for alternatives?
The desire to leverage the IB MAD infrastructure for OPA is understood but the current approach represents OPA as a device capability which does not seem appropriate because OPA is clearly a different type of RDMA technology than IB.
-- Hal
Quoted 3 lines
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 54DCB1E9.7010309-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 34 lines
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
include/rdma/ib_verbs.h index renamed from include/rdma/ib_verbs.h +0
-03ab4033..2614233 100644
include/rdma/ib_verbs.h +4
-0@@ -128,6 +128,10 @@ enum ib_device_cap_flags { IB_DEVICE_ON_DEMAND_PAGING = (1<<31), }; +enum ib_device_cap_flags2 { +IB_DEVICE_OPA_MAD_SUPPORT =1 +}; +enum ib_signature_prot_cap {IB_PROT_T10DIF_TYPE_1 =1 ,IB_PROT_T10DIF_TYPE_2 =1 <<1 ,@@ -210,6 +214,7 @@ struct ib_device_attr {int sig_prot_cap;int sig_guard_cap;struct ib_odp_caps odp_caps;Quoted 8 lines
Quoted 8 lines
include/rdma/ib_verbs.h struct ib_device_attr { · line 220
Quoted 4 lines
include/rdma/ib_verbs.h +1
-0+u64 device_cap_flags2;u32 max_mad_size; };Why is OPA support determined via a device capability flag ? What are the tradeoffs for doing it this way versus the other choices that have been used in the past for other RDMA technologies like RoCE, iWARP,
usNIC, ... ?
Quoted 2 lines
None of those technologies use the MAD stack for Subnet Management.
Other MAD support is very limited (ie IB compatible PMA queries on the local port only).
Quoted 1 line
Quoted 7 lines
Quoted 1 line
Do you have a suggestion for alternatives?
The desire to leverage the IB MAD infrastructure for OPA is understood but the current approach represents OPA as a device capability which does not seem appropriate because OPA is clearly a different type of RDMA technology than IB.
While it is a different type of technology, standard verbs[*] remains 100% compatible. Unlike other verbs technologies user space software does not need any knowledge that the underlying device is not IB. For example, PR (and SA) queries, CM, rdmacm, and verbs calls themselves are all 100% IB compatible.
Therefore, to address your initial question regarding tradeoffs I believe this method is the least invasive to the code as well as removing any potential performance penalties to core verbs.
Ira
[*] We don't support some of the extensions particularly those which have been most recently introduced. And we would like to make our own extensions in the form of higher MTU availability, but the patch is not yet ready to be submitted upstream.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 2807E5FD2F6FDA4886F6618EAC48510E0CC29020-8k97q/[email protected]
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 36 lines · Ira
Quoted 36 lines
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
include/rdma/ib_verbs.h index renamed from include/rdma/ib_verbs.h +0
-03ab4033..2614233 100644
include/rdma/ib_verbs.h +4
-0@@ -128,6 +128,10 @@ enum ib_device_cap_flags { IB_DEVICE_ON_DEMAND_PAGING = (1<<31), }; +enum ib_device_cap_flags2 { +IB_DEVICE_OPA_MAD_SUPPORT =1 +}; +enum ib_signature_prot_cap {IB_PROT_T10DIF_TYPE_1 =1 ,IB_PROT_T10DIF_TYPE_2 =1 <<1 ,@@ -210,6 +214,7 @@ struct ib_device_attr {int sig_prot_cap;int sig_guard_cap;struct ib_odp_caps odp_caps;Quoted 8 lines
Quoted 8 lines
include/rdma/ib_verbs.h struct ib_device_attr { · line 220
Quoted 4 lines
include/rdma/ib_verbs.h +1
-0+u64 device_cap_flags2;u32 max_mad_size; };Why is OPA support determined via a device capability flag ? What are the tradeoffs for doing it this way versus the other choices that have been used in the past for other RDMA technologies like RoCE, iWARP,
usNIC, ... ?
Quoted 2 lines
None of those technologies use the MAD stack for Subnet Management.
Other MAD support is very limited (ie IB compatible PMA queries on the local port only).
Quoted 2 lines
Do you have a suggestion for alternatives?
Quoted 7 lines
Quoted 5 lines
The desire to leverage the IB MAD infrastructure for OPA is understood but the current approach represents OPA as a device capability which does not seem appropriate because OPA is clearly a different type of RDMA technology than IB.
While it is a different type of technology, standard verbs[*] remains 100% compatible. Unlike other verbs technologies user space software does not need any knowledge that the underlying device is not IB. For example, PR (and SA) queries, CM, rdmacm, and verbs calls themselves are all 100% IB compatible.
Even if OPA is 100% standard verbs compatible which it does not appear to be, that does not make OPA an extra capability of an IBA device. While it is a primary goal of the RDMA stack to have a common verbs API for various RDMA interconnects, each one is properly represented to allow it’s unique characteristics to be exposed.
Quoted 5 lines
Therefore, to address your initial question regarding tradeoffs I believe this method is the least invasive to the code as well as removing any potential performance penalties to core verbs.
Ira
[*] We don't support some of the extensions particularly those which have been most recently introduced. And we would like to make our own extensions in the form of higher MTU availability, but the patch is not yet ready to be submitted upstream.
There appear to be a number of things that are not exposed by the current patch set which will be needed in subsequent patches. It would be better to see the complete picture so it can be reviewed as a whole.
-- Hal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 54EB7756.7070407-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 2 lines
Do you have a suggestion for alternatives?
The desire to leverage the IB MAD infrastructure for OPA is understood but the current approach represents OPA as a device capability which does not seem appropriate because OPA is clearly a different type of RDMA technology than IB.
Quoted 7 lines
Quoted 1 line
While it is a different type of technology, standard verbs[*] remains 100%
compatible. Unlike other verbs technologies user space software does not need any knowledge that the underlying device is not IB. For example, PR (and SA) queries, CM, rdmacm, and verbs calls themselves are all 100% IB compatible.
Even if OPA is 100% standard verbs compatible which it does not appear to be, that does not make OPA an extra capability of an IBA device.
I don't want to make it an extra capability of an IBA device. I want to make it an extra capability of a "verbs" device.
Quoted 3 lines
While it is a primary goal of the RDMA stack to have a common verbs API for various RDMA interconnects, each one is properly represented to allow it's unique characteristics to be exposed.
The difference here is that we have maintained IB Verbs compatibility where other RDMA technologies did not. We have tested many Verbs applications (both kernel and user space) and they function _without_ _modification_.
Despite this compatibility we are still having this discussion.
I can think of no other way to signal the MAD capability to the MAD stack which will preserve the verbs compatibility in the same way.
Quoted 7 lines
Quoted 2 lines
Therefore, to address your initial question regarding tradeoffs I believe this
method is the least invasive to the code as well as removing any potential performance penalties to core verbs.
Quoted 3 lines
Ira
Quoted 8 lines
Quoted 1 line
[*] We don't support some of the extensions particularly those which have
been most recently introduced. And we would like to make our own extensions in the form of higher MTU availability, but the patch is not yet ready to be submitted upstream.
There appear to be a number of things that are not exposed by the current patch set which will be needed in subsequent patches. It would be better to see the complete picture so it can be reviewed as a whole.
Is there something in particular you would like to see? There are no other patches required in the core modules for verbs applications to function. The MTU patch only improves verbs performance.
Ira
Quoted 6 lines
-- Hal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 2807E5FD2F6FDA4886F6618EAC48510E0CC3D330-8k97q/[email protected]
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 10 lines · Ira
Quoted 10 lines
Quoted 10 lines
Quoted 8 lines
Quoted 2 lines
Do you have a suggestion for alternatives?
The desire to leverage the IB MAD infrastructure for OPA is understood but the current approach represents OPA as a device capability which does not seem appropriate because OPA is clearly a different type of RDMA technology than IB.
While it is a different type of technology, standard verbs[*] remains 100%
Quoted 8 lines
Quoted 6 lines
compatible. Unlike other verbs technologies user space software does not need any knowledge that the underlying device is not IB. For example, PR (and SA) queries, CM, rdmacm, and verbs calls themselves are all 100% IB compatible.
Even if OPA is 100% standard verbs compatible which it does not appear to be, that does not make OPA an extra capability of an IBA device.
I don't want to make it an extra capability of an IBA device. I want to make it an extra capability of a "verbs" device.
And this, friends, is why it's bad to make both a link layer and an user space API with the exact same name ;-). Anyway, I get your point Ira and it makes sense to me. However, I also get Hal's point. Our track record on this particular issue is a bit wonky though.
First we had InfiniBand.
Then came iWARP, and we used the transport type to differentiate it from an actual InfiniBand device, but left the underlying link layer listed as InfiniBand. Then came RoCE, and we listed its transport type as InfiniBand, but changed the link layer to Ethernet. Which left us in the oxymoronic position that even though iWARP was over Ethernet, the tools said it was over InfiniBand, while RoCE was the only thing that listed Ethernet as the link layer. We later fixed that up with some hacks in tools to keep users from being confused and filing bugs.
Maybe this represents an opportunity to straighten some of this mess out. If I remember correctly, this is the matrix of technologies today:
Technology LinkLayer Transport
InfiniBand InfiniBand InfiniBand Verbs
iWARP InfiniBand iWARP Verbs (subset of IBV, with
specific connection establishment requirements that don't exist with IBV)
RoCE Ethernet InfiniBand Verbs (but with different
addressing because of the different link layer)
OPA ? InfiniBand Verbs
It makes me wonder if we shouldn't make this matrix more accurate:
Technology LinkLayer Transport
InfiniBand InfiniBand InfiniBand Verbs
iWARP Ethernet iWARP Verbs
RoCE Ethernet RoCE-v1 or RoCE-v2
OPA ? OPA Verbs
With this sort of setup, the core ib_mad/ib_umad code would simply check the verbs type to see what support it can enable. For IBV it would be the existing support, for OPAV it would be the additional jumbo support.
I'm not sure how much we might expect a change like this to break existing software though, so maybe staightening this mess out is a non-starter.
Quoted 9 lines
Quoted 3 lines
While it is a primary goal of the RDMA stack to have a common verbs API for various RDMA interconnects, each one is properly represented to allow it's unique characteristics to be exposed.
The difference here is that we have maintained IB Verbs compatibility where other RDMA technologies did not. We have tested many Verbs applications (both kernel and user space) and they function _without_ _modification_.
Despite this compatibility we are still having this discussion.
I can think of no other way to signal the MAD capability to the MAD stack which will preserve the verbs compatibility in the same way.
See above. Define a new transport type, OPAVerbs, that is a superset of IBV and enable jumbo support when OPAV is the transport on the link.
Quoted 30 lines
Quoted 7 lines
Quoted 2 lines
Therefore, to address your initial question regarding tradeoffs I believe this
method is the least invasive to the code as well as removing any potential performance penalties to core verbs.
Quoted 3 lines
Ira
Quoted 8 lines
Quoted 1 line
[*] We don't support some of the extensions particularly those which have
been most recently introduced. And we would like to make our own extensions in the form of higher MTU availability, but the patch is not yet ready to be submitted upstream.
There appear to be a number of things that are not exposed by the current patch set which will be needed in subsequent patches. It would be better to see the complete picture so it can be reviewed as a whole.
Is there something in particular you would like to see? There are no other patches required in the core modules for verbs applications to function. The MTU patch only improves verbs performance.
Ira
Quoted 6 lines
-- Hal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
A message that is not in the archive (sent off-list or not yet received): [email protected]
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
RG91ZywNCg0KWW91IGhhdmUgZ2l2ZW4gbWUgYSBsb3QgdG8gdGhpbmsgYWJvdXQuLi4gIENvbW1l bnRzIGJlbG93Li4uDQoNCj4gPiA+ID4NCj4gPiA+ID4gV2hpbGUgaXQgaXMgYSBkaWZmZXJlbnQg dHlwZSBvZiB0ZWNobm9sb2d5LCBzdGFuZGFyZCB2ZXJic1sqXQ0KPiA+ID4gPiByZW1haW5zIDEw MCUNCj4gPiA+IGNvbXBhdGlibGUuICBVbmxpa2Ugb3RoZXIgdmVyYnMgdGVjaG5vbG9naWVzIHVz ZXIgc3BhY2Ugc29mdHdhcmUNCj4gPiA+IGRvZXMgbm90IG5lZWQgYW55IGtub3dsZWRnZSB0aGF0 IHRoZSB1bmRlcmx5aW5nIGRldmljZSBpcyBub3QgSUIuDQo+ID4gPiBGb3IgZXhhbXBsZSwgUFIg KGFuZCBTQSkgcXVlcmllcywgQ00sIHJkbWFjbSwgYW5kIHZlcmJzIGNhbGxzIHRoZW1zZWx2ZXMN Cj4gYXJlIGFsbCAxMDAlIElCIGNvbXBhdGlibGUuDQo+ID4gPg0KPiA+ID4gRXZlbiBpZiBPUEEg aXMgMTAwJSBzdGFuZGFyZCB2ZXJicyBjb21wYXRpYmxlIHdoaWNoIGl0IGRvZXMgbm90DQo+ID4g PiBhcHBlYXIgdG8gYmUsIHRoYXQgZG9lcyBub3QgbWFrZSBPUEEgYW4gZXh0cmEgY2FwYWJpbGl0 eSBvZiBhbiBJQkEgZGV2aWNlLg0KPiA+DQo+ID4gSSBkb24ndCB3YW50IHRvIG1ha2UgaXQgYW4g ZXh0cmEgY2FwYWJpbGl0eSBvZiBhbiBJQkEgZGV2aWNlLiAgSSB3YW50IHRvIG1ha2UgaXQNCj4g YW4gZXh0cmEgY2FwYWJpbGl0eSBvZiBhICJ2ZXJicyIgZGV2aWNlLg0KPiANCj4gQW5kIHRoaXMs IGZyaWVuZHMsIGlzIHdoeSBpdCdzIGJhZCB0byBtYWtlIGJvdGggYSBsaW5rIGxheWVyIGFuZCBh biB1c2VyIHNwYWNlIEFQSQ0KPiB3aXRoIHRoZSBleGFjdCBzYW1lIG5hbWUgOy0pLiAgQW55d2F5 LCBJIGdldCB5b3VyIHBvaW50IElyYSBhbmQgaXQgbWFrZXMgc2Vuc2UNCj4gdG8gbWUuICBIb3dl dmVyLCBJIGFsc28gZ2V0IEhhbCdzIHBvaW50LiAgT3VyIHRyYWNrIHJlY29yZCBvbiB0aGlzIHBh cnRpY3VsYXINCj4gaXNzdWUgaXMgYSBiaXQgd29ua3kgdGhvdWdoLg0KDQpUaGFua3MgZm9yIGxh eWluZyB0aGlzIG91dC4gIEkgdG9vIHVuZGVyc3RhbmQgSGFscyBwb2ludC4NCg0KPiANCj4gRmly c3Qgd2UgaGFkIEluZmluaUJhbmQuDQo+IA0KPiBUaGVuIGNhbWUgaVdBUlAsIGFuZCB3ZSB1c2Vk IHRoZSB0cmFuc3BvcnQgdHlwZSB0byBkaWZmZXJlbnRpYXRlIGl0IGZyb20gYW4NCj4gYWN0dWFs IEluZmluaUJhbmQgZGV2aWNlLCBidXQgbGVmdCB0aGUgdW5kZXJseWluZyBsaW5rIGxheWVyIGxp c3RlZCBhcyBJbmZpbmlCYW5kLg0KPiBUaGVuIGNhbWUgUm9DRSwgYW5kIHdlIGxpc3RlZCBpdHMg dHJhbnNwb3J0IHR5cGUgYXMgSW5maW5pQmFuZCwgYnV0IGNoYW5nZWQNCj4gdGhlIGxpbmsgbGF5 ZXIgdG8gRXRoZXJuZXQuICBXaGljaCBsZWZ0IHVzIGluIHRoZSBveHltb3JvbmljIHBvc2l0aW9u IHRoYXQgZXZlbg0KPiB0aG91Z2ggaVdBUlAgd2FzIG92ZXIgRXRoZXJuZXQsIHRoZSB0b29scyBz YWlkIGl0IHdhcyBvdmVyIEluZmluaUJhbmQsIHdoaWxlDQo+IFJvQ0Ugd2FzIHRoZSBvbmx5IHRo aW5nIHRoYXQgbGlzdGVkIEV0aGVybmV0IGFzIHRoZSBsaW5rIGxheWVyLiAgV2UgbGF0ZXIgZml4 ZWQNCj4gdGhhdCB1cCB3aXRoIHNvbWUgaGFja3MgaW4gdG9vbHMgdG8ga2VlcCB1c2VycyBmcm9t IGJlaW5nIGNvbmZ1c2VkIGFuZCBmaWxpbmcNCj4gYnVncy4NCj4gDQo+IE1heWJlIHRoaXMgcmVw cmVzZW50cyBhbiBvcHBvcnR1bml0eSB0byBzdHJhaWdodGVuIHNvbWUgb2YgdGhpcyBtZXNzIG91 dC4gIElmIEkNCj4gcmVtZW1iZXIgY29ycmVjdGx5LCB0aGlzIGlzIHRoZSBtYXRyaXggb2YgdGVj aG5vbG9naWVzIHRvZGF5Og0KPiANCj4gVGVjaG5vbG9neQlMaW5rTGF5ZXIJVHJhbnNwb3J0DQo+ IA0KPiBJbmZpbmlCYW5kCUluZmluaUJhbmQJSW5maW5pQmFuZCBWZXJicw0KPiBpV0FSUAkJSW5m aW5pQmFuZAlpV0FSUCBWZXJicyAoc3Vic2V0IG9mIElCViwgd2l0aA0KPiAJCQkJc3BlY2lmaWMg Y29ubmVjdGlvbiBlc3RhYmxpc2htZW50DQo+IAkJCQlyZXF1aXJlbWVudHMgdGhhdCBkb24ndCBl eGlzdCB3aXRoIElCVikNCj4gUm9DRQkJRXRoZXJuZXQJSW5maW5pQmFuZCBWZXJicyAoYnV0IHdp dGggZGlmZmVyZW50DQo+IAkJCQlhZGRyZXNzaW5nIGJlY2F1c2Ugb2YgdGhlIGRpZmZlcmVudA0K PiAJCQkJbGluayBsYXllcikNCj4gT1BBCQk/CQlJbmZpbmlCYW5kIFZlcmJzDQoNCkkgdGhpbmsg dGhpcyBpcyBfcmVsYXRpdmVseV8gYWNjdXJhdGUuICBUaGUgb25lIGV4Y2VwdGlvbiBpcyB3aXRo IHRoZSB2YXJpb3VzIElCIHZlcmJzIGV4dGVuc2lvbnMgd2hpY2ggaGF2ZSBiZWVuIGludHJvZHVj ZWQuICBXaGlsZSBtb3N0IGFyZSBiZWluZyBwdXNoZWQgaW50byB0aGUgc3BlYyBub3QgYWxsIG9m IHRoZW0gYXJlIGluIHRoZSBzcGVjIHByaW9yIHRvIGJlaW5nIGluIHRoZSBrZXJuZWwuICBJdCBt YWtlcyBrZWVwaW5nIHVwIHdpdGggd2hhdCAiSUIgVmVyYnMiIHJlYWxseSBpcyBkaWZmaWN1bHQu DQoNCk1pbmQgeW91IEknbSBub3Qgb3Bwb3NpbmcgaGF2aW5nIElCIFZlcmJzIGJlIGZsZXhpYmxl LiAgQnV0IEkgdGhpbmsgd2UgY2FuIGFjY3VyYXRlbHkgaGF2ZSBtdWx0aXBsZSB1bmRlcmx5aW5n IHRlY2hub2xvZ2llcyB3aGljaCBzdXBwb3J0IElCIFZlcmJzIHdpdGggdmFyaW91cyBleHRlbnNp b25zLg0KDQo+IA0KPiBJdCBtYWtlcyBtZSB3b25kZXIgaWYgd2Ugc2hvdWxkbid0IG1ha2UgdGhp cyBtYXRyaXggbW9yZSBhY2N1cmF0ZToNCj4gDQo+IFRlY2hub2xvZ3kJTGlua0xheWVyCVRyYW5z cG9ydA0KPiANCj4gSW5maW5pQmFuZAlJbmZpbmlCYW5kCUluZmluaUJhbmQgVmVyYnMNCj4gaVdB UlAJCUV0aGVybmV0CWlXQVJQIFZlcmJzDQo+IFJvQ0UJCUV0aGVybmV0CVJvQ0UtdjEgb3IgUm9D RS12Mg0KPiBPUEEJCT8JCU9QQSBWZXJicw0KPiANCj4gV2l0aCB0aGlzIHNvcnQgb2Ygc2V0dXAs IHRoZSBjb3JlIGliX21hZC9pYl91bWFkIGNvZGUgd291bGQgc2ltcGx5IGNoZWNrIHRoZQ0KPiB2 ZXJicyB0eXBlIHRvIHNlZSB3aGF0IHN1cHBvcnQgaXQgY2FuIGVuYWJsZS4gIEZvciBJQlYgaXQg d291bGQgYmUgdGhlIGV4aXN0aW5nDQo+IHN1cHBvcnQsIGZvciBPUEFWIGl0IHdvdWxkIGJlIHRo ZSBhZGRpdGlvbmFsIGp1bWJvIHN1cHBvcnQuDQoNCk9QQSwgdG8gYmUgY29tcGF0aWJsZSB3aXRo IElCIFZlcmJzLCB1c2VzIHRoZSBzYW1lIG5vZGUgdHlwZXMgYXMgSW5maW5pQmFuZCB2ZXJicyAo MSA9PSBDQSwgMiA9PSBTd2l0Y2gpLiAgQXMgc3VjaCBpdCByZXR1cm5zIHRoZSBzYW1lIFRyYW5z cG9ydCB0eXBlLg0KDQo+IA0KPiBJJ20gbm90IHN1cmUgaG93IG11Y2ggd2UgbWlnaHQgZXhwZWN0 IGEgY2hhbmdlIGxpa2UgdGhpcyB0byBicmVhayBleGlzdGluZw0KPiBzb2Z0d2FyZSB0aG91Z2gs IHNvIG1heWJlIHN0YWlnaHRlbmluZyB0aGlzIG1lc3Mgb3V0IGlzIGEgbm9uLXN0YXJ0ZXIuDQoN CkkgdGhpbmsgdGhpcyBpcyBnb2luZyB0byBicmVhayBxdWl0ZSBhIGJpdC4gIEkgaGF2ZSBwcm90 b3R5cGVkIHNldHRpbmcgT1BBIGRldmljZXMgdG8gIk9QQSBMaW5rIExheWVyIiBhbmQgdGhlIHBl cmZ0ZXN0IHRvb2xzIGp1c3QgZmFsbCBvdmVyLiAgQW55IGNoYW5nZXMgdG8gdGhlIExpbmsgbGF5 ZXIgb3IgdGhlIHRyYW5zcG9ydCB0eXBlcyB3aWxsIHJlcXVpcmUgYSB0cmFuc2l0aW9uIHBlcmlv ZCBmb3IgVUxQcy4NCg0KPiANCj4gPiA+IFdoaWxlIGl0IGlzIGEgcHJpbWFyeSBnb2FsIG9mIHRo ZSBSRE1BIHN0YWNrIHRvIGhhdmUgYSBjb21tb24gdmVyYnMNCj4gPiA+IEFQSSBmb3IgdmFyaW91 cyBSRE1BIGludGVyY29ubmVjdHMsIGVhY2ggb25lIGlzIHByb3Blcmx5IHJlcHJlc2VudGVkDQo+ ID4gPiB0byBhbGxvdyBpdCdzIHVuaXF1ZSBjaGFyYWN0ZXJpc3RpY3MgdG8gYmUgZXhwb3NlZC4N Cj4gPg0KPiA+IFRoZSBkaWZmZXJlbmNlIGhlcmUgaXMgdGhhdCB3ZSBoYXZlIG1haW50YWluZWQg SUIgVmVyYnMgY29tcGF0aWJpbGl0eSB3aGVyZQ0KPiBvdGhlciBSRE1BIHRlY2hub2xvZ2llcyBk aWQgbm90LiAgV2UgaGF2ZSB0ZXN0ZWQgbWFueSBWZXJicyBhcHBsaWNhdGlvbnMNCj4gKGJvdGgg a2VybmVsIGFuZCB1c2VyIHNwYWNlKSBhbmQgdGhleSBmdW5jdGlvbiBfd2l0aG91dF8gX21vZGlm aWNhdGlvbl8uDQo+ID4NCj4gPiBEZXNwaXRlIHRoaXMgY29tcGF0aWJpbGl0eSB3ZSBhcmUgc3Rp bGwgaGF2aW5nIHRoaXMgZGlzY3Vzc2lvbi4NCj4gPg0KPiA+IEkgY2FuIHRoaW5rIG9mIG5vIG90 aGVyIHdheSB0byBzaWduYWwgdGhlIE1BRCBjYXBhYmlsaXR5IHRvIHRoZSBNQUQgc3RhY2sNCj4g d2hpY2ggd2lsbCBwcmVzZXJ2ZSB0aGUgdmVyYnMgY29tcGF0aWJpbGl0eSBpbiB0aGUgc2FtZSB3 YXkuDQo+IA0KPiBTZWUgYWJvdmUuICBEZWZpbmUgYSBuZXcgdHJhbnNwb3J0IHR5cGUsIE9QQVZl cmJzLCB0aGF0IGlzIGEgc3VwZXJzZXQgb2YgSUJWDQo+IGFuZCBlbmFibGUganVtYm8gc3VwcG9y dCB3aGVuIE9QQVYgaXMgdGhlIHRyYW5zcG9ydCBvbiB0aGUgbGluay4NCg0KQnV0IHRoZSB0cmFu c3BvcnQgdHlwZSBpcyBub3QgY2hhbmdpbmcuICBPbmNlIGFnYWluIHdlIGFyZSBhdHRlbXB0aW5n IHRvIGJlIGNvbXBsZXRlbHkgdmVyYnMgY29tcGF0aWJsZS4gIEZyb20gdGhlIE1BRCBzdGFjayBQ T1YgdGhlIHZlcmJzIGNhbGxzIGluIHRoZSBrZXJuZWwgYXJlIG5vdCBkaWZmZXJlbnQuDQoNCldv dWxkIGl0IGJlIGFjY2VwdGFibGUgaWYgdGhlIHJlc3VsdCBvZiBteSBwYXRjaCBzZXJpZXMgd2Fz Og0KDQpJbmZpbmlCYW5kCUluZmluaUJhbmQJSW5maW5pQmFuZCBWZXJicw0KaVdBUlAJCUluZmlu aUJhbmQJaVdBUlAgVmVyYnMgKHN1YnNldCBvZiBJQlYsIHdpdGgNCgkJCQlzcGVjaWZpYyBjb25u ZWN0aW9uIGVzdGFibGlzaG1lbnQNCgkJCQlyZXF1aXJlbWVudHMgdGhhdCBkb24ndCBleGlzdCB3 aXRoIElCVikNClJvQ0UJCUV0aGVybmV0CUluZmluaUJhbmQgVmVyYnMgKGJ1dCB3aXRoIGRpZmZl cmVudA0KCQkJCWFkZHJlc3NpbmcgYmVjYXVzZSBvZiB0aGUgZGlmZmVyZW50DQoJCQkJbGluayBs YXllcikNCk9QQQkJT1BBCQlJbmZpbmlCYW5kIFZlcmJzDQoNCkFuZCB0aGUgTUFEIHN0YWNrIGxv b2tlZCBhdCB0aGUgbGluayBsYXllciB0byBzZWUgdGhlIGRpZmZlcmVuY2U/DQoNCklyYQ0KDQo= --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 2807E5FD2F6FDA4886F6618EAC48510E0CC4F18C-8k97q/[email protected]
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
PiBJbmZpbmlCYW5kCUluZmluaUJhbmQJSW5maW5pQmFuZCBWZXJicw0KPiBpV0FSUAkJSW5maW5p QmFuZAlpV0FSUCBWZXJicyAoc3Vic2V0IG9mIElCViwgd2l0aA0KPiAJCQkJc3BlY2lmaWMgY29u bmVjdGlvbiBlc3RhYmxpc2htZW50DQo+IAkJCQlyZXF1aXJlbWVudHMgdGhhdCBkb24ndCBleGlz dCB3aXRoIElCVikNCj4gUm9DRQkJRXRoZXJuZXQJSW5maW5pQmFuZCBWZXJicyAoYnV0IHdpdGgg ZGlmZmVyZW50DQo+IAkJCQlhZGRyZXNzaW5nIGJlY2F1c2Ugb2YgdGhlIGRpZmZlcmVudA0KPiAJ CQkJbGluayBsYXllcikNCj4gT1BBCQlPUEEJCUluZmluaUJhbmQgVmVyYnMNCg0KVmVyYnMgaXMg YW4gaW50ZXJmYWNlIGRlZmluaXRpb24gdG8gaGFyZHdhcmUgdGhhdCBoYXMgYmVlbiB0d2lzdGVk IHRvIGJlIGEgc29mdHdhcmUgQVBJIGFuZCBleHRlbmRlZCB0byBleHBvc2UgdmVuZG9yLXNwZWNp ZmljIGltcGxlbWVudGF0aW9uICdmZWF0dXJlcycgYXMgZXh0ZW5zaW9ucy4gIEl0IGlzIG5vdCBh IHRyYW5zcG9ydC4NCg0KVGhlIGRldmljZSBjYXBhYmlsaXR5IGJpdHMgc2VlbXMgdG8gaGF2ZSBl dm9sdmVkIHRvIG1lYW46IHZlbmRvciBBIGltcGxlbWVudGVkIHNvbWUgcmFuZG9tICdmZWF0dXJl JyBpbiB0aGVpciBoYXJkd2FyZSBhbmQgd2FudHMgYWxsIGFwcGxpY2F0aW9ucyB0byBub3cgY2hl Y2sgZm9yIHRoaXMgJ2ZlYXR1cmUnIGFuZCBjaGFuZ2UgdGhlaXIgY29kZSB0byB1c2UgaXQuICBC YXNpY2FsbHksIHdoYXQgZ2V0cyBkZWZpbmVkIGFzIGEgZGV2aWNlIGNhcCBpcyByYXRoZXIgYXJi aXRyYXJ5Lg0KDQotIFNlYW4NCg==
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A8237399E6F06F-8oqHQFITsIHTXloPLtfHfbfspsVTdybXVpNB7YpNyf8@public.gmane.org
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
PiANCj4gPiBJbmZpbmlCYW5kCUluZmluaUJhbmQJSW5maW5pQmFuZCBWZXJicw0KPiA+IGlXQVJQ CQlJbmZpbmlCYW5kCWlXQVJQIFZlcmJzIChzdWJzZXQgb2YgSUJWLCB3aXRoDQo+ID4gCQkJCXNw ZWNpZmljIGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudA0KPiA+IAkJCQlyZXF1aXJlbWVudHMgdGhh dCBkb24ndCBleGlzdCB3aXRoIElCVikNCj4gPiBSb0NFCQlFdGhlcm5ldAlJbmZpbmlCYW5kIFZl cmJzIChidXQgd2l0aCBkaWZmZXJlbnQNCj4gPiAJCQkJYWRkcmVzc2luZyBiZWNhdXNlIG9mIHRo ZSBkaWZmZXJlbnQNCj4gPiAJCQkJbGluayBsYXllcikNCj4gPiBPUEEJCU9QQQkJSW5maW5pQmFu ZCBWZXJicw0KPiANCj4gVmVyYnMgaXMgYW4gaW50ZXJmYWNlIGRlZmluaXRpb24gdG8gaGFyZHdh cmUgdGhhdCBoYXMgYmVlbiB0d2lzdGVkIHRvIGJlIGENCj4gc29mdHdhcmUgQVBJIGFuZCBleHRl bmRlZCB0byBleHBvc2UgdmVuZG9yLXNwZWNpZmljIGltcGxlbWVudGF0aW9uICdmZWF0dXJlcycN Cj4gYXMgZXh0ZW5zaW9ucy4gIEl0IGlzIG5vdCBhIHRyYW5zcG9ydC4NCj4gDQo+IFRoZSBkZXZp Y2UgY2FwYWJpbGl0eSBiaXRzIHNlZW1zIHRvIGhhdmUgZXZvbHZlZCB0byBtZWFuOiB2ZW5kb3Ig QQ0KPiBpbXBsZW1lbnRlZCBzb21lIHJhbmRvbSAnZmVhdHVyZScgaW4gdGhlaXIgaGFyZHdhcmUg YW5kIHdhbnRzIGFsbA0KPiBhcHBsaWNhdGlvbnMgdG8gbm93IGNoZWNrIGZvciB0aGlzICdmZWF0 dXJlJyBhbmQgY2hhbmdlIHRoZWlyIGNvZGUgdG8gdXNlIGl0Lg0KPiBCYXNpY2FsbHksIHdoYXQg Z2V0cyBkZWZpbmVkIGFzIGEgZGV2aWNlIGNhcCBpcyByYXRoZXIgYXJiaXRyYXJ5Lg0KPiANCg0K VGhpcyB3YXMgdGhlIHBvaW50IEkgd2FzIHRyeWluZyB0byBtYWtlIGFuZCB0aGUgcmVhc29uIHRo ZSBPUEEgTUFEIHN1cHBvcnQgd2FzIGltcGxlbWVudGVkIGFzIGEgZGV2aWNlIGNhcGFiaWxpdHku DQoNCklyYQ0KDQo=
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 2807E5FD2F6FDA4886F6618EAC48510E0CC4F50B-8k97q/[email protected]
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 14 lines · Ira
Quoted 14 lines
Quoted 9 lines
InfiniBand InfiniBand InfiniBand Verbs
iWARP InfiniBand iWARP Verbs (subset of IBV, with
specific connection establishment requirements that don't exist with IBV)
RoCE Ethernet InfiniBand Verbs (but with different
addressing because of the different link layer)
OPA OPA InfiniBand VerbsVerbs is an interface definition to hardware that has been twisted to be a software API and extended to expose vendor-specific implementation 'features' as extensions. It is not a transport.
Quoted 7 lines
Quoted 5 lines
The device capability bits seems to have evolved to mean: vendor A implemented some random 'feature' in their hardware and wants all applications to now check for this 'feature' and change their code to use it. Basically, what gets defined as a device cap is rather arbitrary.
This was the point I was trying to make and the reason the OPA MAD support was implemented as a device capability.
The proposed device capability stands for a change that is way more drastic than just a vendor extension. This is a device running a completely different wire protocol which does not interoperate with the IB device that is impersonating.
Also, it does not come under the same jurisdication as IB. It is entirely possible that IBTA could make some change in the future where OPA can no longer masquerade as IB.
OPA must also be identifiable by any verbs or management application. To do that, it should be properly represented in node type, transport type, and link layer as all the other RDMA technologies have been regardless of the changes needed. All the other RDMA technologies have gone through this process.
-- Hal
Quoted 2 lines
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 54FE0F16.5090905-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
PiBPbiAzLzQvMjAxNSAxMTo0MSBBTSwgV2VpbnksIElyYSB3cm90ZToNCj4gPj4NCj4gPj4+IElu ZmluaUJhbmQJSW5maW5pQmFuZAlJbmZpbmlCYW5kIFZlcmJzDQo+ID4+PiBpV0FSUAkJSW5maW5p QmFuZAlpV0FSUCBWZXJicyAoc3Vic2V0IG9mIElCViwgd2l0aA0KPiA+Pj4gCQkJCXNwZWNpZmlj IGNvbm5lY3Rpb24gZXN0YWJsaXNobWVudA0KPiA+Pj4gCQkJCXJlcXVpcmVtZW50cyB0aGF0IGRv bid0IGV4aXN0IHdpdGggSUJWKQ0KPiA+Pj4gUm9DRQkJRXRoZXJuZXQJSW5maW5pQmFuZCBWZXJi cyAoYnV0IHdpdGggZGlmZmVyZW50DQo+ID4+PiAJCQkJYWRkcmVzc2luZyBiZWNhdXNlIG9mIHRo ZSBkaWZmZXJlbnQNCj4gPj4+IAkJCQlsaW5rIGxheWVyKQ0KPiA+Pj4gT1BBCQlPUEEJCUluZmlu aUJhbmQgVmVyYnMNCj4gPj4NCj4gPj4gVmVyYnMgaXMgYW4gaW50ZXJmYWNlIGRlZmluaXRpb24g dG8gaGFyZHdhcmUgdGhhdCBoYXMgYmVlbiB0d2lzdGVkIHRvDQo+ID4+IGJlIGEgc29mdHdhcmUg QVBJIGFuZCBleHRlbmRlZCB0byBleHBvc2UgdmVuZG9yLXNwZWNpZmljIGltcGxlbWVudGF0aW9u DQo+ICdmZWF0dXJlcycNCj4gPj4gYXMgZXh0ZW5zaW9ucy4gIEl0IGlzIG5vdCBhIHRyYW5zcG9y dC4NCj4gPj4NCj4gPj4gVGhlIGRldmljZSBjYXBhYmlsaXR5IGJpdHMgc2VlbXMgdG8gaGF2ZSBl dm9sdmVkIHRvIG1lYW46IHZlbmRvciBBDQo+ID4+IGltcGxlbWVudGVkIHNvbWUgcmFuZG9tICdm ZWF0dXJlJyBpbiB0aGVpciBoYXJkd2FyZSBhbmQgd2FudHMgYWxsDQo+ID4+IGFwcGxpY2F0aW9u cyB0byBub3cgY2hlY2sgZm9yIHRoaXMgJ2ZlYXR1cmUnIGFuZCBjaGFuZ2UgdGhlaXIgY29kZSB0 byB1c2UgaXQuDQo+ID4+IEJhc2ljYWxseSwgd2hhdCBnZXRzIGRlZmluZWQgYXMgYSBkZXZpY2Ug Y2FwIGlzIHJhdGhlciBhcmJpdHJhcnkuDQo+ID4+DQo+ID4NCj4gPiBUaGlzIHdhcyB0aGUgcG9p bnQgSSB3YXMgdHJ5aW5nIHRvIG1ha2UgYW5kIHRoZSByZWFzb24gdGhlIE9QQSBNQUQgc3VwcG9y dA0KPiB3YXMgaW1wbGVtZW50ZWQgYXMgYSBkZXZpY2UgY2FwYWJpbGl0eS4NCj4gDQo+IFRoZSBw cm9wb3NlZCBkZXZpY2UgY2FwYWJpbGl0eSBzdGFuZHMgZm9yIGEgY2hhbmdlIHRoYXQgaXMgd2F5 IG1vcmUgZHJhc3RpYw0KPiB0aGFuIGp1c3QgYSB2ZW5kb3IgZXh0ZW5zaW9uLiBUaGlzIGlzIGEg ZGV2aWNlIHJ1bm5pbmcgYSBjb21wbGV0ZWx5IGRpZmZlcmVudA0KPiB3aXJlIHByb3RvY29sIHdo aWNoIGRvZXMgbm90IGludGVyb3BlcmF0ZSB3aXRoIHRoZSBJQiBkZXZpY2UgdGhhdCBpcw0KPiBp bXBlcnNvbmF0aW5nLg0KPiANCj4gQWxzbywgaXQgZG9lcyBub3QgY29tZSB1bmRlciB0aGUgc2Ft ZSBqdXJpc2RpY2F0aW9uIGFzIElCLiBJdCBpcyBlbnRpcmVseSBwb3NzaWJsZQ0KPiB0aGF0IElC VEEgY291bGQgbWFrZSBzb21lIGNoYW5nZSBpbiB0aGUgZnV0dXJlIHdoZXJlIE9QQSBjYW4gbm8g bG9uZ2VyDQo+IG1hc3F1ZXJhZGUgYXMgSUIuDQo+IA0KPiBPUEEgbXVzdCBhbHNvIGJlIGlkZW50 aWZpYWJsZSBieSBhbnkgdmVyYnMgDQoNCkkgZGlzYWdyZWUuICBTb2Z0d2FyZSBhcHBsaWNhdGlv bnMgcnVubmluZyBJUCBkb24ndCBuZWVkIHRvIGtub3cgdGhleSBhcmUgcnVubmluZyBvdmVyIElC IHZzIEV0aGVybmV0PyAgV2h5IHdvdWxkIHRoaXMgaGF2ZSB0byBiZSB0cnVlIGZvciBWZXJicz8N CgkNCnBlcmZ0ZXN0LCBsaWJpYnZlcmJzIChleGFtcGxlIGFwcHMpLCBtdmFwaWNoICh2ZXJicyks IG9wZW5tcGkgKHZlcmJzKSwgaWJhY20gKGxpYnJkbWFjbSBiYXNlZCBhcHBsaWNhdGlvbnMpLCBz cnAsIGFuZCBpcG9pYiBhbGwgcnVuIHdpdGhvdXQgYW55IGtub3dsZWRnZSBvZiB0aGUgbGluayBi ZWluZyBPUEEuICBXaGlsZSBub3QgYW4gZXhoYXVzdGl2ZSBsaXN0IG9mIHZlcmJzIGFwcGxpY2F0 aW9ucywgdGhpcyBpcyBhIHByZXR0eSBnb29kIHNhbXBsaW5nLiAgV2UgaGF2ZSBzcGVjaWZpY2Fs bHkgZGVzaWduZWQgT1BBIHRvIHN1cHBvcnQgdGhlc2UgYXBwbGljYXRpb25zIHdpdGhvdXQgbW9k aWZpY2F0aW9ucy4NCg0KPiBvciBtYW5hZ2VtZW50IGFwcGxpY2F0aW9uLiANCg0KQWdyZWVkLCBi dXQgX29ubHlfIHdoZW4gdGFsa2luZyB0byB0aGUgaGFyZHdhcmUuICBPdGhlciBNQUQgaW50ZXJm YWNlcyBhcmUgSUIgY29tcGF0aWJsZS4NCg0KVGhlIGlkZWEgb2YgdGhlIG9yaWdpbmFsIHNlcmll cyB3YXMgdG8gY2hlY2sgdGhlIGRldmljZSBjYXBhYmlsaXR5IGJpdHMgKGluIGtlcm5lbCBhbmQg aW4gdXNlcnNwYWNlKSBmb3IgU00gZGlhZ25vc3RpYyBhbmQgZGlhZ25vc3RpYyB0b29sIHN1cHBv cnQuICBBcyBjdXJyZW50bHkgc3VibWl0dGVkIEkgd291bGQgbmVlZCB0byBhZGQgYSBrZXJuZWwg QUJJIHRvIGdldCB0aGUgZXh0ZW5kZWQgY2FwYWJpbGl0eSBiaXRzIChiZWNhdXNlIHRoZSBvcmln aW5hbHMgd2VyZSBleGhhdXN0ZWQpLiAgSSBoYXZlIGJlZW4gd2FpdGluZyBmb3IgdGhpcyBkaXNj dXNzaW9uIHRvIHNldHRsZSBiZWZvcmUgZ29pbmcgdGhyb3VnaCB0aGF0IGVmZm9ydC4NCgkNCj4g VG8gZG8NCj4gdGhhdCwgaXQgc2hvdWxkIGJlIHByb3Blcmx5IHJlcHJlc2VudGVkIGluIG5vZGUg dHlwZSwgdHJhbnNwb3J0IHR5cGUsIGFuZCBsaW5rDQo+IGxheWVyIGFzIGFsbCB0aGUgb3RoZXIg UkRNQSB0ZWNobm9sb2dpZXMgaGF2ZSBiZWVuIHJlZ2FyZGxlc3Mgb2YgdGhlIGNoYW5nZXMNCj4g bmVlZGVkLiBBbGwgdGhlIG90aGVyIFJETUEgdGVjaG5vbG9naWVzIGhhdmUgZ29uZSB0aHJvdWdo IHRoaXMgcHJvY2Vzcy4NCj4gDQoNCkFzIEkgc3VnZ2VzdGVkIGFib3ZlOiBXb3VsZCBzZXR0aW5n IHRoZSBMaW5rIExheWVyIHRvIGEgbmV3IHZhbHVlIChpZS4gSUJfTElOS19MQVlFUl9PTU5JX1BB VEhfQVJDSCkgd2hpbGUgbWFpbnRhaW5pbmcgdGhlIFRyYW5zcG9ydCBhcyBJbmZpbmlCYW5kIFZl cmJzIGJlIHNhdGlzZmFjdG9yeT8NCg0KV2hpbGUgdGhpcyBicmVha3MgYXQgbGVhc3Qgc29tZSBv ZiB0aGUgZXhhbXBsZXMgSSBsaXN0IGFib3ZlIEkgYmVsaWV2ZSBJIGhhdmUgd29ya2VkIG91dCBh IHdheSB0byBwaGFzZSBpbiB0aGlzIHN1cHBvcnQgdGhyb3VnaCBvdXIgcHJvdmlkZXIgbGlicmFy eS4NCg0KDQpUbyBhZGRyZXNzIHlvdXIgY29tbWVudHMgZnJvbSB0aGUgb3RoZXIgZm9yayBvZiB0 aGlzIHRocmVhZDoNCg0KPiA+IFRoZSAzMiBiaXQgTElEcyBpbiB0aGUgU01QIGFyZSBkZXNpZ25l ZCBmb3IgZnV0dXJlIGV4cGFuc2lvbi4gIEN1cnJlbnRseSBPUEENCj4gZG9lcyBub3Qgc3VwcG9y dCA+IDE2IGJpdCBMSURzLg0KPiANCj4gSXQncyBub3QganVzdCB2ZXJicyAoc3RydWN0dXJlcyBh bmQgQVBJcykgYnV0IGFsc28gQ00gYW5kIGFueSBvdGhlciBwbGFjZSB3aGVyZQ0KPiBMSUQgaXMg dXNlZCB3aGljaCBhcmUgbnVtZXJvdXMuDQo+IA0KPiBJcyBhIG5ldyB2ZXJicyBjb21pbmcgZm9y IHRoaXMgPw0KDQpJIGNhbid0IGd1YXJhbnRlZSB0aGF0IG5vIGNoYW5nZXMgd2lsbCBiZSByZXF1 aXJlZCBpbiB0aGUgZnV0dXJlLiAgQnV0IHJpZ2h0IG5vdyB0aGUgYW5zd2VyIGlzICJObyIgYmVj YXVzZSB3ZSBzcGVjaWZpY2FsbHkgaW1wbGVtZW50ZWQgT1BBIHRvIGJlICJJbmZpbmlCYW5kIFZl cmJzIi4NCg0KPiBIb3cgd2lsbCBjb21wYXRpYmlsaXR5IGJlIGRlYWx0IHdpdGggPw0KDQpDb21w YXRpYmlsaXR5IGlzIHByb3ZpZGVkIGJ5IHByZXNlbnRpbmcgaWRlbnRpY2FsIEluZmluaUJhbmQg VmVyYnMgaW50ZXJmYWNlcyAoa2VybmVsIGFiaSwgU0EgcHJvdG9jb2xzLCBldGMpIHRvIHRoZSB1 c2VyLg0KDQo+IA0KPiBUaGlzIGlzIGFsc28gYW5vdGhlciBleGFtcGxlIHdoeSBhIG1vcmUgY29t cGxldGUgcGljdHVyZSBvZiBPUEEgaXMgbmVlZGVkLg0KPiANCj4gLS0gSGFsDQoNCg0KQXQgdGhp cyBtb21lbnQgT1BBIHByb3ZpZGVzIFVMUCBjb21wYXRpYmlsaXR5IGZvciBhbGwgSUIgVmVyYnMg YXBwbGljYXRpb25zIHdpdGggdGhlIGV4Y2VwdGlvbiBvZiBtYW5hZ2VtZW50IHNvZnR3YXJlIHdo aWNoIHRhbGtzIGRpcmVjdGx5IHRvIHRoZSBTTUEgYW5kIFBNQS4gIEFsbCBVTFAgaW50ZXJhY3Rp b25zIHdpdGggbWFuYWdlbWVudCAoUGF0aCBSZWNvcmQsIENNLCBNdWx0aWNhc3Qgam9pbnMsIGV0 YykgYXJlIHN0aWxsIElCIGZvcm1hdHRlZCBhbmQgY29tcGF0aWJsZS4gIFdlIGhhdmUgc3BlY2lm aWNhbGx5IGRlc2lnbmVkIHRoaXMgaW50ZXJvcGVyYWJpbGl0eSB3aXRoIE9GQS4NCg0KVG8gaGVs cCBpbGx1c3RyYXRlIG15IHBvaW50IEkgaGF2ZSBpbmNsdWRlZCAyIHBhdGNoZXMgYmVsb3cuICBU aGUgZmlyc3QgZGVmaW5lcyBhIG5ldyBMaW5rIExheWVyOyBJQl9MSU5LX0xBWUVSX09NTklfUEFU SF9BUkNIIGFuZCBtb2RpZmllcyBhbGwgdGhlIGtlcm5lbCBpbnRlcmZhY2VzIHdoaWNoIGxvb2sg YXQgbGluayBsYXllci4gIFRoaXMgaGFzIGJlZW4gbWluaW1hbGx5IHRlc3RlZCB3aXRoIElQb0lC IGFuZCB0aGUgcGVyZnRlc3QgdG9vbHMgKHRoZSBtb2RpZmljYXRpb24gb2Ygd2hpY2ggaXMgaW5j bHVkZWQgYXMgdGhlIDJuZCBwYXRjaCkuICBUaGUgZW50aXJlIHBhdGNoIGJvaWxzIGRvd24gdG8g Y2hhbmdpbmcgImlmIChJbmZpbmlCYW5kKSIgdG8gImlmIChJbmZpbmlCYW5kIHx8IE9QQSkiLiAg SU1PIHRoaXMgaXMgcmF0aGVyIGluZWZmaWNpZW50IGJ1dCBpZiB0aGlzIGlzIG1vcmUgYWNjZXB0 YWJsZSB0byB0aGUgY29tbXVuaXR5IEkgYW0gd2lsbGluZyB0byBpbnZlc3RpZ2F0ZSB0aGlzIGZ1 cnRoZXIuDQoNCklyYQ0KDQoNCg0KRnJvbSA5ZjA5YmU5MjU3NjIwNGIzZWFkNzFmNzE0ZmMyMzEx MTBmMDNiZmY2IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQ0KRnJvbTogSXJhIFdlaW55IDxpcmEu d2VpbnlAaW50ZWwuY29tPg0KRGF0ZTogV2VkLCAzIERlYyAyMDE0IDIwOjAxOjA5IC0wNTAwDQpT dWJqZWN0OiBbUEFUQ0hdIFdJUDogSUIvY29yZTogQWRkIElCX0xJTktfTEFZRVJfT01OSV9QQVRI X0FSQ0gNCg0KVGhpcyBPUEEgTGluayBMYXllciBpcyAxMDAlIGNvbXBhdGlibGUgd2l0aCBJbmZp bmlCYW5kIFZlcmJzIHNvZnR3YXJlLg0KLS0tDQogZHJpdmVycy9pbmZpbmliYW5kL2NvcmUvYWdl bnQuYyAgICAgICAgICAgfCAgNCArKystDQogZHJpdmVycy9pbmZpbmliYW5kL2NvcmUvY21hLmMg ICAgICAgICAgICAgfCAzOSArKysrKysrKysrKysrKysrKysrKysrKy0tLS0tLS0tDQogZHJpdmVy cy9pbmZpbmliYW5kL2NvcmUvbWFkLmMgICAgICAgICAgICAgfCAgNCArKystDQogZHJpdmVycy9p bmZpbmliYW5kL2NvcmUvbXVsdGljYXN0LmMgICAgICAgfCAxNiArKysrKysrKy0tLS0tDQogZHJp dmVycy9pbmZpbmliYW5kL2NvcmUvc2FfcXVlcnkuYyAgICAgICAgfCAyMiArKysrKysrKysrKy0t LS0tLQ0KIGRyaXZlcnMvaW5maW5pYmFuZC9jb3JlL3N5c2ZzLmMgICAgICAgICAgIHwgIDIgKysN CiBkcml2ZXJzL2luZmluaWJhbmQvY29yZS91Y21hLmMgICAgICAgICAgICB8ICAxICsNCiBkcml2 ZXJzL2luZmluaWJhbmQvdWxwL2lwb2liL2lwb2liX21haW4uYyB8ICAzICsrLQ0KIGluY2x1ZGUv cmRtYS9pYl92ZXJicy5oICAgICAgICAgICAgICAgICAgIHwgIDEgKw0KIDkgZmlsZXMgY2hhbmdl ZCwgNjYgaW5zZXJ0aW9ucygrKSwgMjYgZGVsZXRpb25zKC0pDQoNCmRpZmYgLS1naXQgYS9kcml2 ZXJzL2luZmluaWJhbmQvY29yZS9hZ2VudC5jIGIvZHJpdmVycy9pbmZpbmliYW5kL2NvcmUvYWdl bnQuYw0KaW5kZXggZjZkMjk2MS4uMGIxZTdlZSAxMDA2NDQNCi0tLSBhL2RyaXZlcnMvaW5maW5p YmFuZC9jb3JlL2FnZW50LmMNCisrKyBiL2RyaXZlcnMvaW5maW5pYmFuZC9jb3JlL2FnZW50LmMN CkBAIC0xNDcsNiArMTQ3LDcgQEAgaW50IGliX2FnZW50X3BvcnRfb3BlbihzdHJ1Y3QgaWJfZGV2 aWNlICpkZXZpY2UsIGludCBwb3J0X251bSkNCiAJc3RydWN0IGliX2FnZW50X3BvcnRfcHJpdmF0 ZSAqcG9ydF9wcml2Ow0KIAl1bnNpZ25lZCBsb25nIGZsYWdzOw0KIAlpbnQgcmV0Ow0KKwllbnVt IHJkbWFfbGlua19sYXllciBsbDsNCiANCiAJLyogQ3JlYXRlIG5ldyBkZXZpY2UgaW5mbyAqLw0K IAlwb3J0X3ByaXYgPSBremFsbG9jKHNpemVvZiAqcG9ydF9wcml2LCBHRlBfS0VSTkVMKTsNCkBA IC0xNTYsNyArMTU3LDggQEAgaW50IGliX2FnZW50X3BvcnRfb3BlbihzdHJ1Y3QgaWJfZGV2aWNl ICpkZXZpY2UsIGludCBwb3J0X251bSkNCiAJCWdvdG8gZXJyb3IxOw0KIAl9DQogDQotCWlmIChy ZG1hX3BvcnRfZ2V0X2xpbmtfbGF5ZXIoZGV2aWNlLCBwb3J0X251bSkgPT0gSUJfTElOS19MQVlF Ul9JTkZJTklCQU5EKSB7DQorCWxsID0gcmRtYV9wb3J0X2dldF9saW5rX2xheWVyKGRldmljZSwg cG9ydF9udW0pOw0KKwlpZiAobGwgPT0gSUJfTElOS19MQVlFUl9JTkZJTklCQU5EIHx8IGxsID09 IElCX0xJTktfTEFZRVJfT01OSV9QQVRIX0FSQ0gpIHsNCiAJCS8qIE9idGFpbiBzZW5kIG9ubHkg TUFEIGFnZW50IGZvciBTTUkgUVAgKi8NCiAJCXBvcnRfcHJpdi0+YWdlbnRbMF0gPSBpYl9yZWdp c3Rlcl9tYWRfYWdlbnQoZGV2aWNlLCBwb3J0X251bSwNCiAJCQkJCQkJICAgIElCX1FQVF9TTUks IE5VTEwsIDAsDQpkaWZmIC0tZ2l0IGEvZHJpdmVycy9pbmZpbmliYW5kL2NvcmUvY21hLmMgYi9k cml2ZXJzL2luZmluaWJhbmQvY29yZS9jbWEuYw0KaW5kZXggZDU3MDAzMC4uZDE2NTg2YyAxMDA2 NDQNCi0tLSBhL2RyaXZlcnMvaW5maW5pYmFuZC9jb3JlL2NtYS5jDQorKysgYi9kcml2ZXJzL2lu ZmluaWJhbmQvY29yZS9jbWEuYw0KQEAgLTM0OSw2ICszNDksMTUgQEAgc3RhdGljIGludCBjbWFf dHJhbnNsYXRlX2FkZHIoc3RydWN0IHNvY2thZGRyICphZGRyLCBzdHJ1Y3QgcmRtYV9kZXZfYWRk ciAqZGV2X2ENCiAJcmV0dXJuIHJldDsNCiB9DQogDQorc3RhdGljIGlubGluZSBpbnQgbGxfbWF0 Y2hlc19kZXZfdHlwZShlbnVtIHJkbWFfbGlua19sYXllciBsbCwNCisJCQkJICAgICAgdW5zaWdu ZWQgc2hvcnQgZGV2X3R5cGUpDQorew0KKwlyZXR1cm4gKChkZXZfdHlwZSA9PSBBUlBIUkRfSU5G SU5JQkFORCAmJg0KKwkJKGxsID09IElCX0xJTktfTEFZRVJfSU5GSU5JQkFORCB8fCBsbCA9PSBJ Ql9MSU5LX0xBWUVSX09NTklfUEFUSF9BUkNIKSkNCisJCXx8DQorCQkoZGV2X3R5cGUgIT0gQVJQ SFJEX0lORklOSUJBTkQgJiYgbGwgPT0gSUJfTElOS19MQVlFUl9FVEhFUk5FVCkpOw0KK30NCisN CiBzdGF0aWMgaW50IGNtYV9hY3F1aXJlX2RldihzdHJ1Y3QgcmRtYV9pZF9wcml2YXRlICppZF9w cml2LA0KIAkJCSAgIHN0cnVjdCByZG1hX2lkX3ByaXZhdGUgKmxpc3Rlbl9pZF9wcml2KQ0KIHsN CkBAIC0zNTcsMTEgKzM2Niw5IEBAIHN0YXRpYyBpbnQgY21hX2FjcXVpcmVfZGV2KHN0cnVjdCBy ZG1hX2lkX3ByaXZhdGUgKmlkX3ByaXYsDQogCXVuaW9uIGliX2dpZCBnaWQsIGlib2VfZ2lkOw0K IAlpbnQgcmV0ID0gLUVOT0RFVjsNCiAJdTggcG9ydCwgZm91bmRfcG9ydDsNCi0JZW51bSByZG1h X2xpbmtfbGF5ZXIgZGV2X2xsID0gZGV2X2FkZHItPmRldl90eXBlID09IEFSUEhSRF9JTkZJTklC QU5EID8NCi0JCUlCX0xJTktfTEFZRVJfSU5GSU5JQkFORCA6IElCX0xJTktfTEFZRVJfRVRIRVJO RVQ7DQorCXVuc2lnbmVkIHNob3J0IGRldl90eXBlID0gZGV2X2FkZHItPmRldl90eXBlOw0KIA0K LQlpZiAoZGV2X2xsICE9IElCX0xJTktfTEFZRVJfSU5GSU5JQkFORCAmJg0KLQkgICAgaWRfcHJp di0+aWQucHMgPT0gUkRNQV9QU19JUE9JQikNCisJaWYgKGRldl90eXBlICE9IEFSUEhSRF9JTkZJ TklCQU5EICYmIGlkX3ByaXYtPmlkLnBzID09IFJETUFfUFNfSVBPSUIpDQogCQlyZXR1cm4gLUVJ TlZBTDsNCiANCiAJbXV0ZXhfbG9jaygmbG9jayk7DQpAQCAtMzcwLDkgKzM3NywxMSBAQCBzdGF0 aWMgaW50IGNtYV9hY3F1aXJlX2RldihzdHJ1Y3QgcmRtYV9pZF9wcml2YXRlICppZF9wcml2LA0K IA0KIAltZW1jcHkoJmdpZCwgZGV2X2FkZHItPnNyY19kZXZfYWRkciArDQogCSAgICAgICByZG1h X2FkZHJfZ2lkX29mZnNldChkZXZfYWRkciksIHNpemVvZiBnaWQpOw0KKw0KIAlpZiAobGlzdGVu X2lkX3ByaXYgJiYNCi0JICAgIHJkbWFfcG9ydF9nZXRfbGlua19sYXllcihsaXN0ZW5faWRfcHJp di0+aWQuZGV2aWNlLA0KLQkJCQkgICAgIGxpc3Rlbl9pZF9wcml2LT5pZC5wb3J0X251bSkgPT0g ZGV2X2xsKSB7DQorCSAgICBsbF9tYXRjaGVzX2Rldl90eXBlKHJkbWFfcG9ydF9nZXRfbGlua19s YXllcihsaXN0ZW5faWRfcHJpdi0+aWQuZGV2aWNlLA0KKwkJCQkJCQkgbGlzdGVuX2lkX3ByaXYt PmlkLnBvcnRfbnVtKSwNCisJCQkJZGV2X3R5cGUpKSB7DQogCQljbWFfZGV2ID0gbGlzdGVuX2lk X3ByaXYtPmNtYV9kZXY7DQogCQlwb3J0ID0gbGlzdGVuX2lkX3ByaXYtPmlkLnBvcnRfbnVtOw0K IAkJaWYgKHJkbWFfbm9kZV9nZXRfdHJhbnNwb3J0KGNtYV9kZXYtPmRldmljZS0+bm9kZV90eXBl KSA9PSBSRE1BX1RSQU5TUE9SVF9JQiAmJg0KQEAgLTM5NCw3ICs0MDMsOCBAQCBzdGF0aWMgaW50 IGNtYV9hY3F1aXJlX2RldihzdHJ1Y3QgcmRtYV9pZF9wcml2YXRlICppZF9wcml2LA0KIAkJCSAg ICBsaXN0ZW5faWRfcHJpdi0+Y21hX2RldiA9PSBjbWFfZGV2ICYmDQogCQkJICAgIGxpc3Rlbl9p ZF9wcml2LT5pZC5wb3J0X251bSA9PSBwb3J0KQ0KIAkJCQljb250aW51ZTsNCi0JCQlpZiAocmRt YV9wb3J0X2dldF9saW5rX2xheWVyKGNtYV9kZXYtPmRldmljZSwgcG9ydCkgPT0gZGV2X2xsKSB7 DQorCQkJaWYgKGxsX21hdGNoZXNfZGV2X3R5cGUocmRtYV9wb3J0X2dldF9saW5rX2xheWVyKGNt YV9kZXYtPmRldmljZSwgcG9ydCksDQorCQkJCQkJZGV2X3R5cGUpKSB7DQogCQkJCWlmIChyZG1h X25vZGVfZ2V0X3RyYW5zcG9ydChjbWFfZGV2LT5kZXZpY2UtPm5vZGVfdHlwZSkgPT0gUkRNQV9U UkFOU1BPUlRfSUIgJiYNCiAJCQkJICAgIHJkbWFfcG9ydF9nZXRfbGlua19sYXllcihjbWFfZGV2 LT5kZXZpY2UsIHBvcnQpID09IElCX0xJTktfTEFZRVJfRVRIRVJORVQpDQogCQkJCQlyZXQgPSBp Yl9maW5kX2NhY2hlZF9naWQoY21hX2Rldi0+ZGV2aWNlLCAmaWJvZV9naWQsICZmb3VuZF9wb3J0 LCBOVUxMKTsNCkBAIC02OTksOSArNzA5LDExIEBAIHN0YXRpYyBpbnQgY21hX2liX2luaXRfcXBf YXR0cihzdHJ1Y3QgcmRtYV9pZF9wcml2YXRlICppZF9wcml2LA0KIAlzdHJ1Y3QgcmRtYV9kZXZf YWRkciAqZGV2X2FkZHIgPSAmaWRfcHJpdi0+aWQucm91dGUuYWRkci5kZXZfYWRkcjsNCiAJaW50 IHJldDsNCiAJdTE2IHBrZXk7DQorCWVudW0gcmRtYV9saW5rX2xheWVyIGxsOw0KKw0KKwlsbCA9 IHJkbWFfcG9ydF9nZXRfbGlua19sYXllcihpZF9wcml2LT5pZC5kZXZpY2UsIGlkX3ByaXYtPmlk LnBvcnRfbnVtKTsNCiANCi0JaWYgKHJkbWFfcG9ydF9nZXRfbGlua19sYXllcihpZF9wcml2LT5p ZC5kZXZpY2UsIGlkX3ByaXYtPmlkLnBvcnRfbnVtKSA9PQ0KLQkgICAgSUJfTElOS19MQVlFUl9J TkZJTklCQU5EKQ0KKwlpZiAobGwgPT0gSUJfTElOS19MQVlFUl9JTkZJTklCQU5EIHx8IGxsID09 IElCX0xJTktfTEFZRVJfT01OSV9QQVRIX0FSQ0gpDQogCQlwa2V5ID0gaWJfYWRkcl9nZXRfcGtl eShkZXZfYWRkcik7DQogCWVsc2UNCiAJCXBrZXkgPSAweGZmZmY7DQpAQCAtOTMwLDYgKzk0Miw3 IEBAIHN0YXRpYyB2b2lkIGNtYV9jYW5jZWxfcm91dGUoc3RydWN0IHJkbWFfaWRfcHJpdmF0ZSAq aWRfcHJpdikNCiB7DQogCXN3aXRjaCAocmRtYV9wb3J0X2dldF9saW5rX2xheWVyKGlkX3ByaXYt PmlkLmRldmljZSwgaWRfcHJpdi0+aWQucG9ydF9udW0pKSB7DQogCWNhc2UgSUJfTElOS19MQVlF Ul9JTkZJTklCQU5EOg0KKwljYXNlIElCX0xJTktfTEFZRVJfT01OSV9QQVRIX0FSQ0g6DQogCQlp ZiAoaWRfcHJpdi0+cXVlcnkpDQogCQkJaWJfc2FfY2FuY2VsX3F1ZXJ5KGlkX3ByaXYtPnF1ZXJ5 X2lkLCBpZF9wcml2LT5xdWVyeSk7DQogCQlicmVhazsNCkBAIC0xMDA4LDYgKzEwMjEsNyBAQCBz dGF0aWMgdm9pZCBjbWFfbGVhdmVfbWNfZ3JvdXBzKHN0cnVjdCByZG1hX2lkX3ByaXZhdGUgKmlk X3ByaXYpDQogCQlsaXN0X2RlbCgmbWMtPmxpc3QpOw0KIAkJc3dpdGNoIChyZG1hX3BvcnRfZ2V0 X2xpbmtfbGF5ZXIoaWRfcHJpdi0+Y21hX2Rldi0+ZGV2aWNlLCBpZF9wcml2LT5pZC5wb3J0X251 bSkpIHsNCiAJCWNhc2UgSUJfTElOS19MQVlFUl9JTkZJTklCQU5EOg0KKwkJY2FzZSBJQl9MSU5L X0xBWUVSX09NTklfUEFUSF9BUkNIOg0KIAkJCWliX3NhX2ZyZWVfbXVsdGljYXN0KG1jLT5tdWx0 aWNhc3QuaWIpOw0KIAkJCWtmcmVlKG1jKTsNCiAJCQlicmVhazsNCkBAIC0xOTcxLDYgKzE5ODUs NyBAQCBpbnQgcmRtYV9yZXNvbHZlX3JvdXRlKHN0cnVjdCByZG1hX2NtX2lkICppZCwgaW50IHRp bWVvdXRfbXMpDQogCWNhc2UgUkRNQV9UUkFOU1BPUlRfSUI6DQogCQlzd2l0Y2ggKHJkbWFfcG9y dF9nZXRfbGlua19sYXllcihpZC0+ZGV2aWNlLCBpZC0+cG9ydF9udW0pKSB7DQogCQljYXNlIElC X0xJTktfTEFZRVJfSU5GSU5JQkFORDoNCisJCWNhc2UgSUJfTElOS19MQVlFUl9PTU5JX1BBVEhf QVJDSDoNCiAJCQlyZXQgPSBjbWFfcmVzb2x2ZV9pYl9yb3V0ZShpZF9wcml2LCB0aW1lb3V0X21z KTsNCiAJCQlicmVhazsNCiAJCWNhc2UgSUJfTElOS19MQVlFUl9FVEhFUk5FVDoNCkBAIC0yMDIz LDYgKzIwMzgsNyBAQCBzdGF0aWMgaW50IGNtYV9iaW5kX2xvb3BiYWNrKHN0cnVjdCByZG1hX2lk X3ByaXZhdGUgKmlkX3ByaXYpDQogCXUxNiBwa2V5Ow0KIAlpbnQgcmV0Ow0KIAl1OCBwOw0KKwll bnVtIHJkbWFfbGlua19sYXllciBsbDsNCiANCiAJY21hX2RldiA9IE5VTEw7DQogCW11dGV4X2xv Y2soJmxvY2spOw0KQEAgLTIwNTksOCArMjA3NSw5IEBAIHBvcnRfZm91bmQ6DQogCWlmIChyZXQp DQogCQlnb3RvIG91dDsNCiANCisJbGwgPSByZG1hX3BvcnRfZ2V0X2xpbmtfbGF5ZXIoY21hX2Rl di0+ZGV2aWNlLCBwKTsNCiAJaWRfcHJpdi0+aWQucm91dGUuYWRkci5kZXZfYWRkci5kZXZfdHlw ZSA9DQotCQkocmRtYV9wb3J0X2dldF9saW5rX2xheWVyKGNtYV9kZXYtPmRldmljZSwgcCkgPT0g SUJfTElOS19MQVlFUl9JTkZJTklCQU5EKSA/DQorCQkobGwgPT0gSUJfTElOS19MQVlFUl9JTkZJ TklCQU5EIHx8IGxsID09IElCX0xJTktfTEFZRVJfT01OSV9QQVRIX0FSQ0gpID8NCiAJCUFSUEhS RF9JTkZJTklCQU5EIDogQVJQSFJEX0VUSEVSOw0KIA0KIAlyZG1hX2FkZHJfc2V0X3NnaWQoJmlk X3ByaXYtPmlkLnJvdXRlLmFkZHIuZGV2X2FkZHIsICZnaWQpOw0KQEAgLTMzNjQsNiArMzM4MSw3 IEBAIGludCByZG1hX2pvaW5fbXVsdGljYXN0KHN0cnVjdCByZG1hX2NtX2lkICppZCwgc3RydWN0 IHNvY2thZGRyICphZGRyLA0KIAljYXNlIFJETUFfVFJBTlNQT1JUX0lCOg0KIAkJc3dpdGNoIChy ZG1hX3BvcnRfZ2V0X2xpbmtfbGF5ZXIoaWQtPmRldmljZSwgaWQtPnBvcnRfbnVtKSkgew0KIAkJ Y2FzZSBJQl9MSU5LX0xBWUVSX0lORklOSUJBTkQ6DQorCQljYXNlIElCX0xJTktfTEFZRVJfT01O SV9QQVRIX0FSQ0g6DQogCQkJcmV0ID0gY21hX2pvaW5faWJfbXVsdGljYXN0KGlkX3ByaXYsIG1j KTsNCiAJCQlicmVhazsNCiAJCWNhc2UgSUJfTElOS19MQVlFUl9FVEhFUk5FVDoNCkBAIC0zNDA4 LDYgKzM0MjYsNyBAQCB2b2lkIHJkbWFfbGVhdmVfbXVsdGljYXN0KHN0cnVjdCByZG1hX2NtX2lk ICppZCwgc3RydWN0IHNvY2thZGRyICphZGRyKQ0KIAkJCWlmIChyZG1hX25vZGVfZ2V0X3RyYW5z cG9ydChpZF9wcml2LT5jbWFfZGV2LT5kZXZpY2UtPm5vZGVfdHlwZSkgPT0gUkRNQV9UUkFOU1BP UlRfSUIpIHsNCiAJCQkJc3dpdGNoIChyZG1hX3BvcnRfZ2V0X2xpbmtfbGF5ZXIoaWQtPmRldmlj ZSwgaWQtPnBvcnRfbnVtKSkgew0KIAkJCQljYXNlIElCX0xJTktfTEFZRVJfSU5GSU5JQkFORDoN CisJCQkJY2FzZSBJQl9MSU5LX0xBWUVSX09NTklfUEFUSF9BUkNIOg0KIAkJCQkJaWJfc2FfZnJl ZV9tdWx0aWNhc3QobWMtPm11bHRpY2FzdC5pYik7DQogCQkJCQlrZnJlZShtYyk7DQogCQkJCQli cmVhazsNCmRpZmYgLS1naXQgYS9kcml2ZXJzL2luZmluaWJhbmQvY29yZS9tYWQuYyBiL2RyaXZl cnMvaW5maW5pYmFuZC9jb3JlL21hZC5jDQppbmRleCA3NGMzMGY0Li40MTAwMzEyIDEwMDY0NA0K LS0tIGEvZHJpdmVycy9pbmZpbmliYW5kL2NvcmUvbWFkLmMNCisrKyBiL2RyaXZlcnMvaW5maW5p YmFuZC9jb3JlL21hZC5jDQpAQCAtMjkyMiw2ICsyOTIyLDcgQEAgc3RhdGljIGludCBpYl9tYWRf cG9ydF9vcGVuKHN0cnVjdCBpYl9kZXZpY2UgKmRldmljZSwNCiAJdW5zaWduZWQgbG9uZyBmbGFn czsNCiAJY2hhciBuYW1lW3NpemVvZiAiaWJfbWFkMTIzIl07DQogCWludCBoYXNfc21pOw0KKwll bnVtIHJkbWFfbGlua19sYXllciBsbDsNCiANCiAJLyogQ3JlYXRlIG5ldyBkZXZpY2UgaW5mbyAq Lw0KIAlwb3J0X3ByaXYgPSBremFsbG9jKHNpemVvZiAqcG9ydF9wcml2LCBHRlBfS0VSTkVMKTsN CkBAIC0yOTM4LDcgKzI5MzksOCBAQCBzdGF0aWMgaW50IGliX21hZF9wb3J0X29wZW4oc3RydWN0 IGliX2RldmljZSAqZGV2aWNlLA0KIAlpbml0X21hZF9xcChwb3J0X3ByaXYsICZwb3J0X3ByaXYt PnFwX2luZm9bMV0pOw0KIA0KIAljcV9zaXplID0gbWFkX3NlbmRxX3NpemUgKyBtYWRfcmVjdnFf c2l6ZTsNCi0JaGFzX3NtaSA9IHJkbWFfcG9ydF9nZXRfbGlua19sYXllcihkZXZpY2UsIHBvcnRf bnVtKSA9PSBJQl9MSU5LX0xBWUVSX0lORklOSUJBTkQ7DQorCWxsID0gcmRtYV9wb3J0X2dldF9s aW5rX2xheWVyKGRldmljZSwgcG9ydF9udW0pOw0KKwloYXNfc21pID0gKGxsID09IElCX0xJTktf TEFZRVJfSU5GSU5JQkFORCB8fCBsbCA9PSBJQl9MSU5LX0xBWUVSX09NTklfUEFUSF9BUkNIKTsN CiAJaWYgKGhhc19zbWkpDQogCQljcV9zaXplICo9IDI7DQogDQpkaWZmIC0tZ2l0IGEvZHJpdmVy cy9pbmZpbmliYW5kL2NvcmUvbXVsdGljYXN0LmMgYi9kcml2ZXJzL2luZmluaWJhbmQvY29yZS9t dWx0aWNhc3QuYw0KaW5kZXggZmExN2I1NS4uNWJjZTFhNTggMTAwNjQ0DQotLS0gYS9kcml2ZXJz L2luZmluaWJhbmQvY29yZS9tdWx0aWNhc3QuYw0KKysrIGIvZHJpdmVycy9pbmZpbmliYW5kL2Nv cmUvbXVsdGljYXN0LmMNCkBAIC03NzgsMTAgKzc3OCwxMiBAQCBzdGF0aWMgdm9pZCBtY2FzdF9l dmVudF9oYW5kbGVyKHN0cnVjdCBpYl9ldmVudF9oYW5kbGVyICpoYW5kbGVyLA0KIHsNCiAJc3Ry dWN0IG1jYXN0X2RldmljZSAqZGV2Ow0KIAlpbnQgaW5kZXg7DQorCWVudW0gcmRtYV9saW5rX2xh eWVyIGxsOw0KIA0KIAlkZXYgPSBjb250YWluZXJfb2YoaGFuZGxlciwgc3RydWN0IG1jYXN0X2Rl dmljZSwgZXZlbnRfaGFuZGxlcik7DQotCWlmIChyZG1hX3BvcnRfZ2V0X2xpbmtfbGF5ZXIoZGV2 LT5kZXZpY2UsIGV2ZW50LT5lbGVtZW50LnBvcnRfbnVtKSAhPQ0KLQkgICAgSUJfTElOS19MQVlF Ul9JTkZJTklCQU5EKQ0KKw0KKwlsbCA9IHJkbWFfcG9ydF9nZXRfbGlua19sYXllcihkZXYtPmRl dmljZSwgZXZlbnQtPmVsZW1lbnQucG9ydF9udW0pOw0KKwlpZiAobGwgIT0gSUJfTElOS19MQVlF Ul9JTkZJTklCQU5EICYmIGxsICE9IElCX0xJTktfTEFZRVJfT01OSV9QQVRIX0FSQ0gpDQogCQly ZXR1cm47DQogDQogCWluZGV4ID0gZXZlbnQtPmVsZW1lbnQucG9ydF9udW0gLSBkZXYtPnN0YXJ0 X3BvcnQ7DQpAQCAtODI0LDggKzgyNiw5IEBAIHN0YXRpYyB2b2lkIG1jYXN0X2FkZF9vbmUoc3Ry dWN0IGliX2RldmljZSAqZGV2aWNlKQ0KIAl9DQogDQogCWZvciAoaSA9IDA7IGkgPD0gZGV2LT5l bmRfcG9ydCAtIGRldi0+c3RhcnRfcG9ydDsgaSsrKSB7DQotCQlpZiAocmRtYV9wb3J0X2dldF9s aW5rX2xheWVyKGRldmljZSwgZGV2LT5zdGFydF9wb3J0ICsgaSkgIT0NCi0JCSAgICBJQl9MSU5L X0xBWUVSX0lORklOSUJBTkQpDQorCQllbnVtIHJkbWFfbGlua19sYXllciBsbCA9IHJkbWFfcG9y dF9nZXRfbGlua19sYXllcihkZXZpY2UsDQorCQkJCQkJCQkgICBkZXYtPnN0YXJ0X3BvcnQgKyBp KTsNCisJCWlmIChsbCAhPSBJQl9MSU5LX0xBWUVSX0lORklOSUJBTkQgJiYgbGwgIT0gSUJfTElO S19MQVlFUl9PTU5JX1BBVEhfQVJDSCkNCiAJCQljb250aW51ZTsNCiAJCXBvcnQgPSAmZGV2LT5w b3J0W2ldOw0KIAkJcG9ydC0+ZGV2ID0gZGV2Ow0KQEAgLTg2Myw4ICs4NjYsOSBAQCBzdGF0aWMg dm9pZCBtY2FzdF9yZW1vdmVfb25lKHN0cnVjdCBpYl9kZXZpY2UgKmRldmljZSkNCiAJZmx1c2hf d29ya3F1ZXVlKG1jYXN0X3dxKTsNCiANCiAJZm9yIChpID0gMDsgaSA8PSBkZXYtPmVuZF9wb3J0 IC0gZGV2LT5zdGFydF9wb3J0OyBpKyspIHsNCi0JCWlmIChyZG1hX3BvcnRfZ2V0X2xpbmtfbGF5 ZXIoZGV2aWNlLCBkZXYtPnN0YXJ0X3BvcnQgKyBpKSA9PQ0KLQkJICAgIElCX0xJTktfTEFZRVJf SU5GSU5JQkFORCkgew0KKwkJZW51bSByZG1hX2xpbmtfbGF5ZXIgbGwgPSByZG1hX3BvcnRfZ2V0 X2xpbmtfbGF5ZXIoZGV2aWNlLA0KKwkJCQkJCQkJICAgZGV2LT5zdGFydF9wb3J0ICsgaSk7DQor CQlpZiAobGwgPT0gSUJfTElOS19MQVlFUl9JTkZJTklCQU5EIHx8IGxsID09IElCX0xJTktfTEFZ RVJfT01OSV9QQVRIX0FSQ0gpIHsNCiAJCQlwb3J0ID0gJmRldi0+cG9ydFtpXTsNCiAJCQlkZXJl Zl9wb3J0KHBvcnQpOw0KIAkJCXdhaXRfZm9yX2NvbXBsZXRpb24oJnBvcnQtPmNvbXApOw0KZGlm ZiAtLWdpdCBhL2RyaXZlcnMvaW5maW5pYmFuZC9jb3JlL3NhX3F1ZXJ5LmMgYi9kcml2ZXJzL2lu ZmluaWJhbmQvY29yZS9zYV9xdWVyeS5jDQppbmRleCBjMzhmMDMwLi45MGVkYTEyIDEwMDY0NA0K LS0tIGEvZHJpdmVycy9pbmZpbmliYW5kL2NvcmUvc2FfcXVlcnkuYw0KKysrIGIvZHJpdmVycy9p bmZpbmliYW5kL2NvcmUvc2FfcXVlcnkuYw0KQEAgLTQ1MCw3ICs0NTAsOCBAQCBzdGF0aWMgdm9p ZCBpYl9zYV9ldmVudChzdHJ1Y3QgaWJfZXZlbnRfaGFuZGxlciAqaGFuZGxlciwgc3RydWN0IGli X2V2ZW50ICpldmVudA0KIAkJc3RydWN0IGliX3NhX3BvcnQgKnBvcnQgPQ0KIAkJCSZzYV9kZXYt PnBvcnRbZXZlbnQtPmVsZW1lbnQucG9ydF9udW0gLSBzYV9kZXYtPnN0YXJ0X3BvcnRdOw0KIA0K LQkJaWYgKHJkbWFfcG9ydF9nZXRfbGlua19sYXllcihoYW5kbGVyLT5kZXZpY2UsIHBvcnQtPnBv cnRfbnVtKSAhPSBJQl9MSU5LX0xBWUVSX0lORklOSUJBTkQpDQorCQllbnVtIHJkbWFfbGlua19s YXllciBsbCA9IHJkbWFfcG9ydF9nZXRfbGlua19sYXllcihoYW5kbGVyLT5kZXZpY2UsIHBvcnQt PnBvcnRfbnVtKTsNCisJCWlmIChsbCAhPSBJQl9MSU5LX0xBWUVSX0lORklOSUJBTkQgJiYgbGwg IT0gSUJfTElOS19MQVlFUl9PTU5JX1BBVEhfQVJDSCkNCiAJCQlyZXR1cm47DQogDQogCQlzcGlu X2xvY2tfaXJxc2F2ZSgmcG9ydC0+YWhfbG9jaywgZmxhZ3MpOw0KQEAgLTExNTMsNiArMTE1NCw3 IEBAIHN0YXRpYyB2b2lkIGliX3NhX2FkZF9vbmUoc3RydWN0IGliX2RldmljZSAqZGV2aWNlKQ0K IHsNCiAJc3RydWN0IGliX3NhX2RldmljZSAqc2FfZGV2Ow0KIAlpbnQgcywgZSwgaTsNCisJZW51 bSByZG1hX2xpbmtfbGF5ZXIgbGw7DQogDQogCWlmIChyZG1hX25vZGVfZ2V0X3RyYW5zcG9ydChk ZXZpY2UtPm5vZGVfdHlwZSkgIT0gUkRNQV9UUkFOU1BPUlRfSUIpDQogCQlyZXR1cm47DQpAQCAt MTE3NSw3ICsxMTc3LDggQEAgc3RhdGljIHZvaWQgaWJfc2FfYWRkX29uZShzdHJ1Y3QgaWJfZGV2 aWNlICpkZXZpY2UpDQogDQogCWZvciAoaSA9IDA7IGkgPD0gZSAtIHM7ICsraSkgew0KIAkJc3Bp bl9sb2NrX2luaXQoJnNhX2Rldi0+cG9ydFtpXS5haF9sb2NrKTsNCi0JCWlmIChyZG1hX3BvcnRf Z2V0X2xpbmtfbGF5ZXIoZGV2aWNlLCBpICsgMSkgIT0gSUJfTElOS19MQVlFUl9JTkZJTklCQU5E KQ0KKwkJbGwgPSByZG1hX3BvcnRfZ2V0X2xpbmtfbGF5ZXIoZGV2aWNlLCBpICsgMSk7DQorCQlp ZiAobGwgIT0gSUJfTElOS19MQVlFUl9JTkZJTklCQU5EICYmIGxsICE9IElCX0xJTktfTEFZRVJf T01OSV9QQVRIX0FSQ0gpDQogCQkJY29udGludWU7DQogDQogCQlzYV9kZXYtPnBvcnRbaV0uc21f YWggICAgPSBOVUxMOw0KQEAgLTEyMDQsMTYgKzEyMDcsMjAgQEAgc3RhdGljIHZvaWQgaWJfc2Ff YWRkX29uZShzdHJ1Y3QgaWJfZGV2aWNlICpkZXZpY2UpDQogCWlmIChpYl9yZWdpc3Rlcl9ldmVu dF9oYW5kbGVyKCZzYV9kZXYtPmV2ZW50X2hhbmRsZXIpKQ0KIAkJZ290byBlcnI7DQogDQotCWZv ciAoaSA9IDA7IGkgPD0gZSAtIHM7ICsraSkNCi0JCWlmIChyZG1hX3BvcnRfZ2V0X2xpbmtfbGF5 ZXIoZGV2aWNlLCBpICsgMSkgPT0gSUJfTElOS19MQVlFUl9JTkZJTklCQU5EKQ0KKwlmb3IgKGkg PSAwOyBpIDw9IGUgLSBzOyArK2kpIHsNCisJCWxsID0gcmRtYV9wb3J0X2dldF9saW5rX2xheWVy KGRldmljZSwgaSArIDEpOw0KKwkJaWYgKGxsID09IElCX0xJTktfTEFZRVJfSU5GSU5JQkFORCB8 fCBsbCA9PSBJQl9MSU5LX0xBWUVSX09NTklfUEFUSF9BUkNIKQ0KIAkJCXVwZGF0ZV9zbV9haCgm c2FfZGV2LT5wb3J0W2ldLnVwZGF0ZV90YXNrKTsNCisJfQ0KIA0KIAlyZXR1cm47DQogDQogZXJy Og0KLQl3aGlsZSAoLS1pID49IDApDQotCQlpZiAocmRtYV9wb3J0X2dldF9saW5rX2xheWVyKGRl dmljZSwgaSArIDEpID09IElCX0xJTktfTEFZRVJfSU5GSU5JQkFORCkNCisJd2hpbGUgKC0taSA+ PSAwKSB7DQorCQlsbCA9IHJkbWFfcG9ydF9nZXRfbGlua19sYXllcihkZXZpY2UsIGkgKyAxKTsN CisJCWlmIChsbCA9PSBJQl9MSU5LX0xBWUVSX0lORklOSUJBTkQgfHwgbGwgPT0gSUJfTElOS19M QVlFUl9PTU5JX1BBVEhfQVJDSCkNCiAJCQlpYl91bnJlZ2lzdGVyX21hZF9hZ2VudChzYV9kZXYt PnBvcnRbaV0uYWdlbnQpOw0KKwl9DQogDQogCWtmcmVlKHNhX2Rldik7DQogDQpAQCAtMTIzMyw3 ICsxMjQwLDggQEAgc3RhdGljIHZvaWQgaWJfc2FfcmVtb3ZlX29uZShzdHJ1Y3QgaWJfZGV2aWNl ICpkZXZpY2UpDQogCWZsdXNoX3dvcmtxdWV1ZShpYl93cSk7DQogDQogCWZvciAoaSA9IDA7IGkg PD0gc2FfZGV2LT5lbmRfcG9ydCAtIHNhX2Rldi0+c3RhcnRfcG9ydDsgKytpKSB7DQotCQlpZiAo cmRtYV9wb3J0X2dldF9saW5rX2xheWVyKGRldmljZSwgaSArIDEpID09IElCX0xJTktfTEFZRVJf SU5GSU5JQkFORCkgew0KKwkJZW51bSByZG1hX2xpbmtfbGF5ZXIgbGwgPSByZG1hX3BvcnRfZ2V0 X2xpbmtfbGF5ZXIoZGV2aWNlLCBpICsgMSk7DQorCQlpZiAobGwgPT0gSUJfTElOS19MQVlFUl9J TkZJTklCQU5EIHx8IGxsID09IElCX0xJTktfTEFZRVJfT01OSV9QQVRIX0FSQ0gpIHsNCiAJCQlp Yl91bnJlZ2lzdGVyX21hZF9hZ2VudChzYV9kZXYtPnBvcnRbaV0uYWdlbnQpOw0KIAkJCWlmIChz YV9kZXYtPnBvcnRbaV0uc21fYWgpDQogCQkJCWtyZWZfcHV0KCZzYV9kZXYtPnBvcnRbaV0uc21f YWgtPnJlZiwgZnJlZV9zbV9haCk7DQpkaWZmIC0tZ2l0IGEvZHJpdmVycy9pbmZpbmliYW5kL2Nv cmUvc3lzZnMuYyBiL2RyaXZlcnMvaW5maW5pYmFuZC9jb3JlL3N5c2ZzLmMNCmluZGV4IGNiZDAz ODMuLjY2YjAxZjQgMTAwNjQ0DQotLS0gYS9kcml2ZXJzL2luZmluaWJhbmQvY29yZS9zeXNmcy5j DQorKysgYi9kcml2ZXJzL2luZmluaWJhbmQvY29yZS9zeXNmcy5jDQpAQCAtMjUzLDYgKzI1Myw4 IEBAIHN0YXRpYyBzc2l6ZV90IGxpbmtfbGF5ZXJfc2hvdyhzdHJ1Y3QgaWJfcG9ydCAqcCwgc3Ry dWN0IHBvcnRfYXR0cmlidXRlICp1bnVzZWQsDQogCQlyZXR1cm4gc3ByaW50ZihidWYsICIlc1xu IiwgIkluZmluaUJhbmQiKTsNCiAJY2FzZSBJQl9MSU5LX0xBWUVSX0VUSEVSTkVUOg0KIAkJcmV0 dXJuIHNwcmludGYoYnVmLCAiJXNcbiIsICJFdGhlcm5ldCIpOw0KKwljYXNlIElCX0xJTktfTEFZ RVJfT01OSV9QQVRIX0FSQ0g6DQorCQlyZXR1cm4gc3ByaW50ZihidWYsICIlc1xuIiwgIk9tbmlQ YXRoQXJjaCIpOw0KIAlkZWZhdWx0Og0KIAkJcmV0dXJuIHNwcmludGYoYnVmLCAiJXNcbiIsICJV bmtub3duIik7DQogCX0NCmRpZmYgLS1naXQgYS9kcml2ZXJzL2luZmluaWJhbmQvY29yZS91Y21h LmMgYi9kcml2ZXJzL2luZmluaWJhbmQvY29yZS91Y21hLmMNCmluZGV4IDQ1ZDY3ZTkuLjUwMmUy ZTggMTAwNjQ0DQotLS0gYS9kcml2ZXJzL2luZmluaWJhbmQvY29yZS91Y21hLmMNCisrKyBiL2Ry aXZlcnMvaW5maW5pYmFuZC9jb3JlL3VjbWEuYw0KQEAgLTcyNyw2ICs3MjcsNyBAQCBzdGF0aWMg c3NpemVfdCB1Y21hX3F1ZXJ5X3JvdXRlKHN0cnVjdCB1Y21hX2ZpbGUgKmZpbGUsDQogCQlzd2l0 Y2ggKHJkbWFfcG9ydF9nZXRfbGlua19sYXllcihjdHgtPmNtX2lkLT5kZXZpY2UsDQogCQkJY3R4 LT5jbV9pZC0+cG9ydF9udW0pKSB7DQogCQljYXNlIElCX0xJTktfTEFZRVJfSU5GSU5JQkFORDoN CisJCWNhc2UgSUJfTElOS19MQVlFUl9PTU5JX1BBVEhfQVJDSDoNCiAJCQl1Y21hX2NvcHlfaWJf cm91dGUoJnJlc3AsICZjdHgtPmNtX2lkLT5yb3V0ZSk7DQogCQkJYnJlYWs7DQogCQljYXNlIElC X0xJTktfTEFZRVJfRVRIRVJORVQ6DQpkaWZmIC0tZ2l0IGEvZHJpdmVycy9pbmZpbmliYW5kL3Vs cC9pcG9pYi9pcG9pYl9tYWluLmMgYi9kcml2ZXJzL2luZmluaWJhbmQvdWxwL2lwb2liL2lwb2li X21haW4uYw0KaW5kZXggNThiNWFhMy4uNWM1MTg2NiAxMDA2NDQNCi0tLSBhL2RyaXZlcnMvaW5m aW5pYmFuZC91bHAvaXBvaWIvaXBvaWJfbWFpbi5jDQorKysgYi9kcml2ZXJzL2luZmluaWJhbmQv dWxwL2lwb2liL2lwb2liX21haW4uYw0KQEAgLTE2NzMsNyArMTY3Myw4IEBAIHN0YXRpYyB2b2lk IGlwb2liX2FkZF9vbmUoc3RydWN0IGliX2RldmljZSAqZGV2aWNlKQ0KIAl9DQogDQogCWZvciAo cCA9IHM7IHAgPD0gZTsgKytwKSB7DQotCQlpZiAocmRtYV9wb3J0X2dldF9saW5rX2xheWVyKGRl dmljZSwgcCkgIT0gSUJfTElOS19MQVlFUl9JTkZJTklCQU5EKQ0KKwkJZW51bSByZG1hX2xpbmtf bGF5ZXIgbGwgPSByZG1hX3BvcnRfZ2V0X2xpbmtfbGF5ZXIoZGV2aWNlLCBwKTsNCisJCWlmIChs bCAhPSBJQl9MSU5LX0xBWUVSX0lORklOSUJBTkQgJiYgbGwgIT0gSUJfTElOS19MQVlFUl9PTU5J X1BBVEhfQVJDSCkNCiAJCQljb250aW51ZTsNCiAJCWRldiA9IGlwb2liX2FkZF9wb3J0KCJpYiVk IiwgZGV2aWNlLCBwKTsNCiAJCWlmICghSVNfRVJSKGRldikpIHsNCmRpZmYgLS1naXQgYS9pbmNs dWRlL3JkbWEvaWJfdmVyYnMuaCBiL2luY2x1ZGUvcmRtYS9pYl92ZXJicy5oDQppbmRleCA2NTk5 NGExLi42YTE1MDg4IDEwMDY0NA0KLS0tIGEvaW5jbHVkZS9yZG1hL2liX3ZlcmJzLmgNCisrKyBi L2luY2x1ZGUvcmRtYS9pYl92ZXJicy5oDQpAQCAtODgsNiArODgsNyBAQCBlbnVtIHJkbWFfbGlu a19sYXllciB7DQogCUlCX0xJTktfTEFZRVJfVU5TUEVDSUZJRUQsDQogCUlCX0xJTktfTEFZRVJf SU5GSU5JQkFORCwNCiAJSUJfTElOS19MQVlFUl9FVEhFUk5FVCwNCisJSUJfTElOS19MQVlFUl9P TU5JX1BBVEhfQVJDSCwNCiB9Ow0KIA0KIGVudW0gaWJfZGV2aWNlX2NhcF9mbGFncyB7DQotLSAN CjEuOC4yDQoNCg0KDQoNCg0KRnJvbSAyOGM1YzdlNDRiODdjNGUzZTI5NjM0ZmQzNzhkYTQ4NzE0 MDFjYmNkIE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQ0KRnJvbTogSXJhIFdlaW55IDxpcmEud2Vp bnlAaW50ZWwuY29tPg0KRGF0ZTogV2VkLCAxMSBNYXIgMjAxNSAwMjoyODoxNCAtMDQwMA0KU3Vi amVjdDogW1BBVENIXSBwZXJmdGVzdDogV0lQIGFkZCBPUEEgTGluayBMYXllciBzdXBwb3J0DQoN Ci0tLQ0KIHNyYy9wZXJmdGVzdF9wYXJhbWV0ZXJzLmMgfCAzICsrKw0KIDEgZmlsZSBjaGFuZ2Vk LCAzIGluc2VydGlvbnMoKykNCg0KZGlmZiAtLWdpdCBhL3NyYy9wZXJmdGVzdF9wYXJhbWV0ZXJz LmMgYi9zcmMvcGVyZnRlc3RfcGFyYW1ldGVycy5jDQppbmRleCBmYzQwODhhLi45MjhiYTk2IDEw MDc1NQ0KLS0tIGEvc3JjL3BlcmZ0ZXN0X3BhcmFtZXRlcnMuYw0KKysrIGIvc3JjL3BlcmZ0ZXN0 X3BhcmFtZXRlcnMuYw0KQEAgLTk4NSw2ICs5ODUsNyBAQCBjb25zdCBjaGFyICp0cmFuc3BvcnRf c3RyKGVudW0gaWJ2X3RyYW5zcG9ydF90eXBlIHR5cGUpDQogLyoqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq Kg0KICAqDQogICoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKi8NCisjZGVmaW5lIElCVl9MSU5LX0xBWUVS X09QQSAoSUJWX0xJTktfTEFZRVJfRVRIRVJORVQrMSkNCiBjb25zdCBjaGFyICpsaW5rX2xheWVy X3N0cih1aW50OF90IGxpbmtfbGF5ZXIpDQogew0KIAlzd2l0Y2ggKGxpbmtfbGF5ZXIpIHsNCkBA IC05OTQsNiArOTk1LDggQEAgY29uc3QgY2hhciAqbGlua19sYXllcl9zdHIodWludDhfdCBsaW5r X2xheWVyKQ0KIAkJCXJldHVybiAiSUIiOw0KIAkJY2FzZSBJQlZfTElOS19MQVlFUl9FVEhFUk5F VDoNCiAJCQlyZXR1cm4gIkV0aGVybmV0IjsNCisJCWNhc2UgSUJWX0xJTktfTEFZRVJfT1BBOg0K KwkJCXJldHVybiAiT1BBIjsNCiAJCSNpZmRlZiBIQVZFX1NDSUYNCiAJCWNhc2UgSUJWX0xJTktf TEFZRVJfU0NJRjoNCiAJCQlyZXR1cm4gIlNDSUYiOw0KLS0gDQoxLjguMg0KDQoNCg0K --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 2807E5FD2F6FDA4886F6618EAC48510E0CC5C11A-8k97q/[email protected]
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
PiA+IG9yIG1hbmFnZW1lbnQgYXBwbGljYXRpb24uDQo+IA0KPiBBZ3JlZWQsIGJ1dCBfb25seV8g d2hlbiB0YWxraW5nIHRvIHRoZSBoYXJkd2FyZS4gIE90aGVyIE1BRCBpbnRlcmZhY2VzIGFyZQ0K PiBJQiBjb21wYXRpYmxlLg0KDQpNYXliZSB0aGlzIGlzIHdoYXQgSXJhIGlzIHRoaW5raW5nLCBh bmQganVzdCBub3QgZXhwbGFpbmluZyB2ZXJ5IHdlbGwuICBCdXQgaXQgbWFrZXMgc2Vuc2UgdG8g bWUgdG8gdXNlIG1hbmFnZW1lbnQgc3BlY2lmaWMgZmllbGRzL2F0dHJpYnV0ZXMvZmxhZ3MgZm9y IHRoZSAqbWFuYWdlbWVudCogcGllY2VzLCByYXRoZXIgdGhhbiB1c2luZyB0aGUgbGluayBhbmQv b3IgdHJhbnNwb3J0IGxheWVyIHByb3RvY29scyBhcyBhIHByb3h5LiAgTWFuYWdlbWVudCByZWxh dGVkIGNvZGUgc2hvdWxkIHJlYWxseSBicmFuY2ggYmFzZWQgb24gdGhhdC4NCg0KVGhlIGludHJv ZHVjdGlvbiBvZiBuZXcgT1BBIGxpbmsgYW5kIHRyYW5zcG9ydCBwcm90b2NvbHMsIGFuZCBmaXhp bmcgd2hhdCdzIHRoZXJlIGZvciBpV2FycCwgY2FuIHRoZW4gYmUgYWRkcmVzc2VkIHNlcGFyYXRl bHkuDQoNCkkgZG9uJ3QgaGF2ZSBhbnkgdGhvdWdodHMgZm9yIHdoYXQgdGhlIG1hbmFnZW1lbnQg c3BlY2lmaWMgZmllbGRzL2F0dHJpYnV0ZS9mbGFncyBzaG91bGQgYmUgLS0gd2hldGhlciBuZXcg ZmllbGRzIGFyZSBhZGRlZCB0byB0aGUgZGV2aWNlIGF0dHJpYnV0ZXMsIGEgbWFuYWdlbWVudCBm bGFnIGlzIGRlZmluZWQsIGV0Yy4NCg==
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A8237399E8106A-P5GAC/[email protected]
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
PiBCdXQgaXQgbWFrZXMgc2Vuc2UgdG8gbWUgdG8gdXNlIG1hbmFnZW1lbnQgc3BlY2lmaWMNCj4g ZmllbGRzL2F0dHJpYnV0ZXMvZmxhZ3MgZm9yIHRoZSAqbWFuYWdlbWVudCogcGllY2VzLCByYXRo ZXIgdGhhbiB1c2luZyB0aGUNCj4gbGluayBhbmQvb3IgdHJhbnNwb3J0IGxheWVyIHByb3RvY29s cyBhcyBhIHByb3h5LiAgTWFuYWdlbWVudCByZWxhdGVkIGNvZGUNCj4gc2hvdWxkIHJlYWxseSBi cmFuY2ggYmFzZWQgb24gdGhhdC4NCg0KQXMgYSBwcm9wb3NhbCwgd2UgY291bGQgYWRkIGEgbmV3 IGZpZWxkIHRvIHRoZSBrZXJuZWwgcG9ydCBhdHRyaWJ1dGUgc3RydWN0dXJlLiAgVGhlIGZpZWxk IHdvdWxkIGJlIGEgYml0bWFzayBvZiBtYW5hZ2VtZW50IGNhcGFiaWxpdGllcy9wcm90b2NvbHM6 DQoNCklCX01HTVRfUFJPVE9fU00gLSBzdXBwb3J0cyBJQiBTTVBzDQpJQl9NR01UX1BST1RPX1NB IC0gc3VwcG9ydHMgSUIgU0EgTUFEcw0KSUJfTUdNVF9QUk9UT19HUyAtIHN1cHBvcnRzIElCIEdT SSBNQURzIChlLmcuIENNLCBQTSwgLi4uKQ0KSUJfTUdNVF9QUk9UT19PUEFfU00gLSBzdXBwb3J0 cyBPUEEgU01QcyAob3Igd2hhdGV2ZXIgdGhleSBhcmUgY2FsbGVkKQ0KSUJfTUdNVF9QUk9UT19P UEFfR1MgLSBzdXBwb3J0cyBPUEEgR1MgTUFEcyAob3Igd2hhdGV2ZXIgaXMgc3VwcG9ydGVkKSAN Cg0KSWYgdGhlICpHUyBmbGFncyBhcmUgbm90IHN1ZmZpY2llbnQgdG8gZGlzdGluZ3Vpc2ggYmV0 d2VlbiBNQURzIHN1cHBvcnRlZCBvdmVyIElCIGFuZCBSb0NFLCBpdCBjYW4gYmUgZnVydGhlciBk aXZpZGVkIChpLmUuIENNLCBQTSwgQk0sIERNLCBldGMuKS4NCg0KVGhpcyB3b3VsZCBwcm92aWRl IGEgZGlyZWN0IG1hcHBpbmcgb2Ygd2hpY2ggbWFuYWdlbWVudCBwcm90b2NvbHMgYXJlIHN1cHBv cnRlZCBmb3IgYSBnaXZlbiBwb3J0LCByYXRoZXIgdGhhbiBpdCBiZWluZyBpbmZlcnJlZCBieSB0 aGUgbGluay90cmFuc3BvcnQgZmllbGRzLCB3aGljaCBzaG91bGQgcmVhbGx5IGJlIGluZGVwZW5k ZW50LiAgSXQgd291bGQgYWxzbyBhbGxvdyBmb3Igc2ltcGxlIGNoZWNrcyBieSB0aGUgY29yZSBs YXllci4NCg0KSWYgd2Ugd2FudCB0aGUgY29kZSB0byBiZSBtb3JlIGdlbmVyaWMsIGFkZGl0aW9u YWwgZmllbGQocykgY291bGQgYmUgYWRkZWQsIHN1Y2ggYXMgbWFkX3NpemUsIHNvIHRoYXQgYW55 IHNpemUgb2YgbWFuYWdlbWVudCBkYXRhZ3JhbSBpcyBzdXBwb3J0ZWQuICBUaGlzIHdvdWxkIGJl IHVzZWQgaW5zdGVhZCBvZiBpbmZlcnJpbmcgdGhlIHNpemUgYmFzZWQgb24gdGhlIHN1cHBvcnRl ZCBwcm90b2NvbC4NCg0KLSBTZWFuIA0K
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A8237399E818C6-P5GAC/[email protected]
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Hi, folks
I've done a draft (very rough draft...) according to my understanding on Sean's proposal.
The implementation is to allow device setup the management flags during ib_query_port() (amso1100 as eg), and later we could use the flags to check the capability.
For new capability/proto, like OPA, device could setup new flag IB_MGMT_PROTO_OPA during query_port() callback, and some helper like rdma_mgmt_cap_opa() can be used for management branch.
How do you think about this?
Regards,
Michael Wang
drivers/infiniband/core/cma.c +1 -2
@@ -375,8 +375,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_mgmt_cap_iboe (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else
@@ -395,8 +394,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port)continue ; if (rdma_port_get_link_layer(cma_dev->device, port) ==
dev_ll) {
- if (rdma_node_get_transport(cma_dev->device->node_type) == RDMA_TRANSPORT_IB &&
- rdma_port_get_link_layer(cma_dev->device, port) == IB_LINK_LAYER_ETHERNET)
+ if (rdma_mgmt_cap_iboe(cma_dev->device, port))
ret = ib_find_cached_gid(cma_dev->device,
&iboe_gid, &found_port, NULL);
else
ret = ib_find_cached_gid(cma_dev->device, &gid, &found_port, NULL);
drivers/infiniband/core/mad.c +2 -2
@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ; + has_smi =rdma_mgmt_cap_smi (device, port_num);if (has_smi) cq_size *=2 ;
@@ -3057,7 +3057,7 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_mgmt_cap_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {
drivers/infiniband/core/verbs.c +20 -0
@@ -146,6 +146,26 @@ enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_ }EXPORT_SYMBOL (rdma_port_get_link_layer); +int rdma_port_default_mgmt_flags (struct ib_device *device,u8 port_num) +{ +int mgmt_flags =0 ; +enum rdma_transport_type tp = +rdma_node_get_transport (device->node_type); +enum rdma_link_layer ll = +rdma_port_get_link_layer (device, port_num); + +if (tp ==RDMA_TRANSPORT_IB ) { + mgmt_flags |=IB_MGMT_PROTO_IB ; +if (ll ==IB_LINK_LAYER_INFINIBAND ) { + mgmt_flags |=IB_MGMT_PROTO_SMI ; + mgmt_flags |=IB_MGMT_PROTO_IBOE ; + } + } + +return mgmt_flags; +} +EXPORT_SYMBOL (rdma_port_default_mgmt_flags); +/* Protection domains */ struct ib_pd *ib_alloc_pd (struct ib_device *device)
drivers/infiniband/hw/amso1100/c2_provider.c +3 -0
@@ -96,6 +96,9 @@ static int c2_query_port(struct ib_device *ibdev, props->active_width =1 ; props->active_speed =IB_SPEED_SDR ; +/* Makeup flags here, by default or on your own */ + props->mgmt_flags =rdma_port_default_mgmt_flags (ibdev, port); +return 0 ; }
include/rdma/ib_verbs.h +34 -0
@@ -90,6 +90,13 @@ enum rdma_link_layer { IB_LINK_LAYER_ETHERNET, }; +enum rdma_mgmt_flag { +IB_MGMT_PROTO_IB , +IB_MGMT_PROTO_SMI , +IB_MGMT_PROTO_IBOE , +/* More Here*/ +}; +enum ib_device_cap_flags {IB_DEVICE_RESIZE_MAX_WR =1 ,IB_DEVICE_BAD_PKEY_CNTR = (1 <<1 ),
@@ -352,6 +359,7 @@ struct ib_port_attr {enum ib_mtu active_mtu;int gid_tbl_len;u32 port_cap_flags; +u32 mgmt_flags;u32 max_msg_sz;u32 bad_pkey_cntr;u32 qkey_viol_cntr;
@@ -1743,6 +1751,32 @@ int ib_query_port(struct ib_device *device,enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num); +int rdma_port_default_mgmt_flags (struct ib_device *device,u8 port_num); + +static inline int rdma_mgmt_cap (struct ib_device *device,u8 port_num) +{ +struct ib_port_attr port_attr; +memset (&port_attr,0 ,sizeof port_attr); +ib_query_port (device, port_num, &port_attr); +return port_attr.mgmt_flags; +} + +static inline int rdma_mgmt_cap_ib (struct ib_device *device) +{ +u8 port_num = device->node_type ==RDMA_NODE_IB_SWITCH ?0 :1 ; +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_IB ; +} + +static inline int rdma_mgmt_cap_smi (struct ib_device *device,u8 port_num) +{ +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_SMI ; +} + +static inline int rdma_mgmt_cap_iboe (struct ib_device *device,u8 port_num) +{ +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_IBOE ; +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Quoted 12 lines · Sean
Quoted 4 lines
But it makes sense to me to use management specific fields/attributes/flags for the *management* pieces, rather than using the link and/or transport layer protocols as a proxy. Management related code should really branch based on that.
As a proposal, we could add a new field to the kernel port attribute structure. The field would be a bitmask of management capabilities/protocols:
IB_MGMT_PROTO_SM- supports IB SMPsIB_MGMT_PROTO_SA- supports IB SA MADsIB_MGMT_PROTO_GS- supports IB GSI MADs (e.g. CM, PM, ...)IB_MGMT_PROTO_OPA_SM- supports OPA SMPs (or whatever they are called)IB_MGMT_PROTO_OPA_GS- supports OPA GS MADs (or whatever is supported)
Quoted 8 lines
If the *GS flags are not sufficient to distinguish between MADs supported over IB and RoCE, it can be further divided (i.e. CM, PM, BM, DM, etc.).
This would provide a direct mapping of which management protocols are supported for a given port, rather than it being inferred by the link/transport fields, which should really be independent. It would also allow for simple checks by the core layer.
If we want the code to be more generic, additional field(s) could be added, such as
mad_size, so that any size of management datagram is supported. This would be used instead of inferring the size based on the supported protocol.- Sean
N�����r��y���b�X��ǧv�^�){.n�+����{��ٚ�{ay�ʇڙ�,j��f���h���z��w������j:+v���w�j�m��������zZ+�����ݢj"��!tml=
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
In reply to a message not in the archiveHi, folks
I've done a draft (very rough draft...) according to my understanding on Sean's proposal.
The implementation is to allow device setup the management flags during ib_query_port() (amso1100 as eg), and later we could use the flags to check the capability.
For new capability/proto, like OPA, device could setup new flag IB_MGMT_PROTO_OPA during query_port() callback, and some helper like rdma_mgmt_cap_opa() can be used for management branch.
How do you think about this?
Regards,
Michael Wang
drivers/infiniband/core/cma.c +1 -2
@@ -375,8 +375,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_mgmt_cap_iboe (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else
@@ -395,8 +394,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port)continue ; if (rdma_port_get_link_layer(cma_dev->device, port) ==
dev_ll) {
- if (rdma_node_get_transport(cma_dev->device->node_type) == RDMA_TRANSPORT_IB &&
- rdma_port_get_link_layer(cma_dev->device, port) == IB_LINK_LAYER_ETHERNET)
+ if (rdma_mgmt_cap_iboe(cma_dev->device, port))
ret = ib_find_cached_gid(cma_dev->device,
&iboe_gid, &found_port, NULL);
else
ret = ib_find_cached_gid(cma_dev->device, &gid, &found_port, NULL);
drivers/infiniband/core/mad.c +2 -2
@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ; + has_smi =rdma_mgmt_cap_smi (device, port_num);if (has_smi) cq_size *=2 ;
@@ -3057,7 +3057,7 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_mgmt_cap_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {
drivers/infiniband/core/verbs.c +20 -0
@@ -146,6 +146,26 @@ enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_ }EXPORT_SYMBOL (rdma_port_get_link_layer); +int rdma_port_default_mgmt_flags (struct ib_device *device,u8 port_num) +{ +int mgmt_flags =0 ; +enum rdma_transport_type tp = +rdma_node_get_transport (device->node_type); +enum rdma_link_layer ll = +rdma_port_get_link_layer (device, port_num); + +if (tp ==RDMA_TRANSPORT_IB ) { + mgmt_flags |=IB_MGMT_PROTO_IB ; +if (ll ==IB_LINK_LAYER_INFINIBAND ) { + mgmt_flags |=IB_MGMT_PROTO_SMI ; + mgmt_flags |=IB_MGMT_PROTO_IBOE ; + } + } + +return mgmt_flags; +} +EXPORT_SYMBOL (rdma_port_default_mgmt_flags); +/* Protection domains */ struct ib_pd *ib_alloc_pd (struct ib_device *device)
drivers/infiniband/hw/amso1100/c2_provider.c +3 -0
@@ -96,6 +96,9 @@ static int c2_query_port(struct ib_device *ibdev, props->active_width =1 ; props->active_speed =IB_SPEED_SDR ; +/* Makeup flags here, by default or on your own */ + props->mgmt_flags =rdma_port_default_mgmt_flags (ibdev, port); +return 0 ; }
include/rdma/ib_verbs.h +34 -0
@@ -90,6 +90,13 @@ enum rdma_link_layer { IB_LINK_LAYER_ETHERNET, }; +enum rdma_mgmt_flag { +IB_MGMT_PROTO_IB , +IB_MGMT_PROTO_SMI , +IB_MGMT_PROTO_IBOE , +/* More Here*/ +}; +enum ib_device_cap_flags {IB_DEVICE_RESIZE_MAX_WR =1 ,IB_DEVICE_BAD_PKEY_CNTR = (1 <<1 ),
@@ -352,6 +359,7 @@ struct ib_port_attr {enum ib_mtu active_mtu;int gid_tbl_len;u32 port_cap_flags; +u32 mgmt_flags;u32 max_msg_sz;u32 bad_pkey_cntr;u32 qkey_viol_cntr;
@@ -1743,6 +1751,32 @@ int ib_query_port(struct ib_device *device,enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num); +int rdma_port_default_mgmt_flags (struct ib_device *device,u8 port_num); + +static inline int rdma_mgmt_cap (struct ib_device *device,u8 port_num) +{ +struct ib_port_attr port_attr; +memset (&port_attr,0 ,sizeof port_attr); +ib_query_port (device, port_num, &port_attr); +return port_attr.mgmt_flags; +} + +static inline int rdma_mgmt_cap_ib (struct ib_device *device) +{ +u8 port_num = device->node_type ==RDMA_NODE_IB_SWITCH ?0 :1 ; +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_IB ; +} + +static inline int rdma_mgmt_cap_smi (struct ib_device *device,u8 port_num) +{ +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_SMI ; +} + +static inline int rdma_mgmt_cap_iboe (struct ib_device *device,u8 port_num) +{ +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_IBOE ; +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Quoted 12 lines · Sean
Quoted 4 lines
But it makes sense to me to use management specific fields/attributes/flags for the *management* pieces, rather than using the link and/or transport layer protocols as a proxy. Management related code should really branch based on that.
As a proposal, we could add a new field to the kernel port attribute structure. The field would be a bitmask of management capabilities/protocols:
IB_MGMT_PROTO_SM- supports IB SMPsIB_MGMT_PROTO_SA- supports IB SA MADsIB_MGMT_PROTO_GS- supports IB GSI MADs (e.g. CM, PM, ...)IB_MGMT_PROTO_OPA_SM- supports OPA SMPs (or whatever they are called)IB_MGMT_PROTO_OPA_GS- supports OPA GS MADs (or whatever is supported)
Quoted 8 lines
If the *GS flags are not sufficient to distinguish between MADs supported over IB and RoCE, it can be further divided (i.e. CM, PM, BM, DM, etc.).
This would provide a direct mapping of which management protocols are supported for a given port, rather than it being inferred by the link/transport fields, which should really be independent. It would also allow for simple checks by the core layer.
If we want the code to be more generic, additional field(s) could be added, such as
mad_size, so that any size of management datagram is supported. This would be used instead of inferring the size based on the supported protocol.- Sean
N�����r��y���b�X��ǧv�^�)Þº{.n�+����{��Ù�{ay�ÊÚ�,j��f���h���z��w������j:+v���w�j�m��������zZ+�����ݢj"��!tml=
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): 20150320235748.GA22703-W4f6Xiosr+yv7QzWx2u06xL4W9x8LtSr@public.gmane.org
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
My apologies to those who are duplicated here. This did not make it to the mailing list due to mail configuration issues.
From [email protected] Fri Mar 20 18:55:29 2015
Quoted 10 lines
Hi, folks
I've done a draft (very rough draft...) according to my understanding on Sean's proposal.
The implementation is to allow device setup the management flags during
ib_query_port()(amso1100 as eg), and later we could use the flags to check the capability.
Quoted 5 lines
For new capability/proto, like OPA, device could setup new flag
IB_MGMT_PROTO_OPAduringquery_port()callback, and some helper likerdma_mgmt_cap_opa()can be used for management branch.How do you think about this?
This is not saving us anything... See below.
Quoted 16 lines
Regards,
Michael Wangdrivers/infiniband/core/cma.c +0
-1@@ -375,8 +375,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; - if (rdma_node_get_transport(cma_dev->device->node_type) ==
Quoted 4 lines
RDMA_TRANSPORT_IB&&
-rdma_port_get_link_layer(cma_dev->device, port)==IB_LINK_LAYER_ETHERNET)
+ if (rdma_mgmt_cap_iboe(cma_dev->device, port))
This is still indicating a specific technology "iboe" rather than the specific management capabilities the port has.
Also this if statement does not seem to have anything to do with the management support. Here the iboe_gid is a different format and needs to be processed differently from the gid.
Quoted 53 lines
ret = ib_find_cached_gid(cma_dev->device, &iboe_gid, &found_port, NULL); elsedrivers/infiniband/core/cma.c +0
-1@@ -395,8 +394,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port) continue;if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { - if (rdma_node_get_transport(cma_dev->device->node_type)==
RDMA_TRANSPORT_IB&&
-rdma_port_get_link_layer(cma_dev->device, port)==IB_LINK_LAYER_ETHERNET)
+ if (rdma_mgmt_cap_iboe(cma_dev->device, port))ret = ib_find_cached_gid(cma_dev->device, &iboe_gid, &found_port, NULL); else ret = ib_find_cached_gid(cma_dev->device, &gid,
&found_port, NULL);drivers/infiniband/core/mad.c +2
-2@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ; + has_smi =rdma_mgmt_cap_smi (device, port_num);if (has_smi) cq_size *=2 ;@@ -3057,7 +3057,7 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_mgmt_cap_ib (device))return ;if (device->node_type == RDMA_NODE_IB_SWITCH) {drivers/infiniband/core/verbs.c +0
-0@@ -146,6 +146,26 @@ enum rdma_link_layerrdma_port_get_link_layer(struct
ib_device*device, u8port_
}
EXPORT_SYMBOL(rdma_port_get_link_layer);
drivers/infiniband/core/verbs.c enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_ · line 155
Quoted 7 lines
drivers/infiniband/core/verbs.c +7
-0+int rdma_port_default_mgmt_flags (struct ib_device *device,u8 port_num) +{ +int mgmt_flags =0 ; +enum rdma_transport_type tp = +rdma_node_get_transport (device->node_type); +enum rdma_link_layer ll = +rdma_port_get_link_layer (device, port_num);
This does not separate the management capabilities from the transport and link layer like Sean was advocating. This is just refactoring the current implementation with the use of additional flags.
Quoted 36 lines
drivers/infiniband/core/verbs.c +13
-0+ +if (tp ==RDMA_TRANSPORT_IB ) { + mgmt_flags |=IB_MGMT_PROTO_IB ; +if (ll ==IB_LINK_LAYER_INFINIBAND ) { + mgmt_flags |=IB_MGMT_PROTO_SMI ; + mgmt_flags |=IB_MGMT_PROTO_IBOE ; + } + } + +return mgmt_flags; +} +EXPORT_SYMBOL (rdma_port_default_mgmt_flags); +/* Protection domains */ struct ib_pd *ib_alloc_pd (struct ib_device *device)drivers/infiniband/hw/amso1100/c2_provider.c +3
-0@@ -96,6 +96,9 @@ static int c2_query_port(struct ib_device *ibdev, props->active_width =1 ; props->active_speed =IB_SPEED_SDR ; +/* Makeup flags here, by default or on your own */ + props->mgmt_flags =rdma_port_default_mgmt_flags (ibdev, port); +return 0 ; }include/rdma/ib_verbs.h +0
-0
include/rdma/ib_verbs.h enum rdma_link_layer { · line 96
Quoted 7 lines
include/rdma/ib_verbs.h +4
-0@@ -90,6 +90,13 @@ enum rdma_link_layer { IB_LINK_LAYER_ETHERNET, }; +enum rdma_mgmt_flag { +IB_MGMT_PROTO_IB , +IB_MGMT_PROTO_SMI , +IB_MGMT_PROTO_IBOE ,
IB and IBoE are not management protocols.
Quoted 15 lines
include/rdma/ib_verbs.h +4
-0+/* More Here*/ +}; +enum ib_device_cap_flags {IB_DEVICE_RESIZE_MAX_WR =1 ,IB_DEVICE_BAD_PKEY_CNTR = (1 <<1 ),@@ -352,6 +359,7 @@ struct ib_port_attr {enum ib_mtu active_mtu;int gid_tbl_len;u32 port_cap_flags; +u32 mgmt_flags;u32 max_msg_sz;u32 bad_pkey_cntr;u32 qkey_viol_cntr;
include/rdma/ib_verbs.h int ib_query_port(struct ib_device *device, · line 1754
Quoted 4 lines
include/rdma/ib_verbs.h +1
-0@@ -1743,6 +1751,32 @@ int ib_query_port(struct ib_device *device,enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num); +int rdma_port_default_mgmt_flags (struct ib_device *device,u8 port_num);
This should return u32.
I think I would rather see your other patch go in to clean up the code a bit and work this issue separately.
Ira
Quoted 49 lines
include/rdma/ib_verbs.h +25
-0+ +static inline int rdma_mgmt_cap (struct ib_device *device,u8 port_num) +{ +struct ib_port_attr port_attr; +memset (&port_attr,0 ,sizeof port_attr); +ib_query_port (device, port_num, &port_attr); +return port_attr.mgmt_flags; +} + +static inline int rdma_mgmt_cap_ib (struct ib_device *device) +{ +u8 port_num = device->node_type ==RDMA_NODE_IB_SWITCH ?0 :1 ; +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_IB ; +} + +static inline int rdma_mgmt_cap_smi (struct ib_device *device,u8 port_num) +{ +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_SMI ; +} + +static inline int rdma_mgmt_cap_iboe (struct ib_device *device,u8 port_num) +{ +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_IBOE ; +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);Quoted 18 lines · Sean
Quoted 4 lines
But it makes sense to me to use management specific fields/attributes/flags for the *management* pieces, rather than using the link and/or transport layer protocols as a proxy. Management related code should really branch based on that.
As a proposal, we could add a new field to the kernel port attribute structure. The field would be a bitmask of management capabilities/protocols:
IB_MGMT_PROTO_SM- supports IB SMPsIB_MGMT_PROTO_SA- supports IB SA MADsIB_MGMT_PROTO_GS- supports IB GSI MADs (e.g. CM, PM, ...)IB_MGMT_PROTO_OPA_SM- supports OPA SMPs (or whatever they are called)IB_MGMT_PROTO_OPA_GS- supports OPA GS MADs (or whatever is supported)If the *GS flags are not sufficient to distinguish between MADs supported over IB and RoCE, it can be further divided (i.e. CM, PM, BM, DM, etc.).
This would provide a direct mapping of which management protocols are supported for a given port, rather than it being inferred by the link/transport fields, which should really be independent. It would also allow for simple checks by the core layer.
If we want the code to be more generic, additional field(s) could be added, such as
mad_size, so that any size of management datagram is supported. This would be used instead of inferring the size based on the supported protocol.
Quoted 8 lines
Quoted 2 lines
- Sean
N�����r��y���b�X��ǧv�^�)Þº{.n�+����{��Ùs�{ay�Ê?ÚT�,j��f���h���z��w������j:+v���w�j�m��������zZ+�����ݢj"��!tml=--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20150321000541.GA24717-W4f6Xiosr+yv7QzWx2u06xL4W9x8LtSr@public.gmane.org
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Hello, Ira
Thanks for the reply :-)
Quoted 5 lines · ira.weiny
My apologies to those who are duplicated here. This did not make it to the mailing list due to mail configuration issues.
From [email protected] Fri Mar 20 18:55:29 2015
[snip]
Quoted 6 lines
Quoted 5 lines
- if (
rdma_node_get_transport(cma_dev->device->node_type)==RDMA_TRANSPORT_IB&&
-rdma_port_get_link_layer(cma_dev->device, port)==IB_LINK_LAYER_ETHERNET)
+ if (rdma_mgmt_cap_iboe(cma_dev->device, port))
Quoted 6 lines
This is still indicating a specific technology "iboe" rather than the specific management capabilities the port has.
Also this if statement does not seem to have anything to do with the management support. Here the
iboe_gidis a different format and needs to be processed differently from the gid.
Agree, the mgmt cap here is more close to concept 'feature' rather than 'protocol'.
The purpose is to help branch the management, like when vendor declare his device support ibeo format, mad can get the hint by checking the port attribute then branch the management path, which currently is inferred from transport type and link layer type, not told by the vendor.
Quoted 4 lines
Quoted 4 lines
ret = ib_find_cached_gid(cma_dev->device, &iboe_gid, &found_port, NULL); else
[snip]
Quoted 8 lines
drivers/infiniband/core/verbs.c enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_ · line 155
Quoted 4 lines
drivers/infiniband/core/verbs.c +4
-0+enum rdma_transport_type tp = +rdma_node_get_transport (device->node_type); +enum rdma_link_layer ll = +rdma_port_get_link_layer (device, port_num);This does not separate the management capabilities from the transport and link layer like Sean was advocating. This is just refactoring the current implementation with the use of additional flags.
Yes, it's currently copy the logical we are using to branch the management, however, if the vendor could provide his own setup code here, we don't really need to infer anymore but following the indicator from vendor, this 'default' method is for transition.
Sean, could you please give more hint or details on what you'd rather like to have? I think I got some misunderstanding on your suggestion...
Quoted 2 lines
Quoted 2 lines
+
[snip]
Quoted 7 lines
include/rdma/ib_verbs.h enum rdma_link_layer { · line 96
Quoted 5 lines
include/rdma/ib_verbs.h +4
-0+enum rdma_mgmt_flag { +IB_MGMT_PROTO_IB , +IB_MGMT_PROTO_SMI , +IB_MGMT_PROTO_IBOE ,IB and IBoE are not management protocols.
May be it should be 'feature' rather than 'proto'?
We have management branch for iboe although it's not really a protocol (or does it belong to any protocol specially?), also we have too much places to check if device got infiniband feature, especially at the beginning of initialization, I really want to have some way to simplify these check, or make them more like a formal mechanism.
Quoted 1 line
[snip]
Quoted 4 lines
Quoted 2 lines
+int
rdma_port_default_mgmt_flags(struct ib_device *device, u8 port_num);This should return u32.
Yeah.. that's why I call it a draft ;-)
Actually the flags are even not defined in bit but integer :-P just try to introduce the idea see if it's close to Sean's proposal.
Quoted 3 lines
I think I would rather see your other patch go in to clean up the code a bit and work this issue separately.
Agree, it seems like not a simple work, me too would rather to cleanup the code of inferring firstly, put them together into some public helper, waiting for adapt to some new mechanism.
Sean, what's your opinion?
Regards,
Michael Wang
Quoted 64 lines
Ira
Quoted 57 lines
include/rdma/ib_verbs.h +25
-0+ +static inline int rdma_mgmt_cap (struct ib_device *device,u8 port_num) +{ +struct ib_port_attr port_attr; +memset (&port_attr,0 ,sizeof port_attr); +ib_query_port (device, port_num, &port_attr); +return port_attr.mgmt_flags; +} + +static inline int rdma_mgmt_cap_ib (struct ib_device *device) +{ +u8 port_num = device->node_type ==RDMA_NODE_IB_SWITCH ?0 :1 ; +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_IB ; +} + +static inline int rdma_mgmt_cap_smi (struct ib_device *device,u8 port_num) +{ +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_SMI ; +} + +static inline int rdma_mgmt_cap_iboe (struct ib_device *device,u8 port_num) +{ +return rdma_mgmt_cap (device, port_num) &IB_MGMT_PROTO_IBOE ; +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);Quoted 12 lines · Sean
Quoted 4 lines
But it makes sense to me to use management specific fields/attributes/flags for the *management* pieces, rather than using the link and/or transport layer protocols as a proxy. Management related code should really branch based on that.
As a proposal, we could add a new field to the kernel port attribute structure. The field would be a bitmask of management capabilities/protocols:
IB_MGMT_PROTO_SM- supports IB SMPsIB_MGMT_PROTO_SA- supports IB SA MADsIB_MGMT_PROTO_GS- supports IB GSI MADs (e.g. CM, PM, ...)IB_MGMT_PROTO_OPA_SM- supports OPA SMPs (or whatever they are called)IB_MGMT_PROTO_OPA_GS- supports OPA GS MADs (or whatever is supported)Quoted 8 lines
If the *GS flags are not sufficient to distinguish between MADs supported over IB and RoCE, it can be further divided (i.e. CM, PM, BM, DM, etc.).
This would provide a direct mapping of which management protocols are supported for a given port, rather than it being inferred by the link/transport fields, which should really be independent. It would also allow for simple checks by the core layer.
If we want the code to be more generic, additional field(s) could be added, such as
mad_size, so that any size of management datagram is supported. This would be used instead of inferring the size based on the supported protocol.- Sean
N�����r��y���b�X��ǧv�^�)Þº{.n�+����{��Ùs�{ay� Ê?ÚT�,j ��f���h���z� �w��� ���j:+v���w�j�m���� ����zZ+�����ݢj"��!tml=--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): CAJuTgQUsZ34F-dKpsmW+5=axDWb93pA43LZ-qKbEjqyu-RUUmg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
PiBZZXMsIGl0J3MgY3VycmVudGx5IGNvcHkgdGhlIGxvZ2ljYWwgd2UgYXJlIHVzaW5nIHRvIGJy YW5jaCB0aGUNCj4gbWFuYWdlbWVudCwgaG93ZXZlciwgaWYgdGhlIHZlbmRvciBjb3VsZCBwcm92 aWRlIGhpcyBvd24gc2V0dXAgY29kZQ0KPiBoZXJlLCB3ZSBkb24ndCByZWFsbHkgbmVlZCB0byBp bmZlciBhbnltb3JlIGJ1dCBmb2xsb3dpbmcgdGhlDQo+IGluZGljYXRvciBmcm9tIHZlbmRvciwg dGhpcyAnZGVmYXVsdCcgbWV0aG9kIGlzIGZvciB0cmFuc2l0aW9uLg0KPiANCj4gU2VhbiwgY291 bGQgeW91IHBsZWFzZSBnaXZlIG1vcmUgaGludCBvciBkZXRhaWxzIG9uIHdoYXQgeW91J2QgcmF0 aGVyDQo+IGxpa2UgdG8gaGF2ZT8gSSB0aGluayBJIGdvdCBzb21lIG1pc3VuZGVyc3RhbmRpbmcg b24geW91cg0KPiBzdWdnZXN0aW9uLi4uDQoNClRvIHJlc3RhdGUgbXkgc3VnZ2VzdGluZywgSSB3 YXMgdGhpbmtpbmcgb2YgZGVmaW5pbmcgc29tZXRoaW5nIGxpa2UgdGhpczoNCg0KZW51bSB7DQoJ SUJfTUdNVF9QUk9UT19TTSA9ICgxIDw8IDApLCAgIC8qIHN1cHBvcnRzIElCIFNNUHMgKi8NCglJ Ql9NR01UX1BST1RPX1NBID0gKDEgPDwgMSksICAgLyogc3VwcG9ydHMgSUIgU0EgTUFEcyAqLw0K CUlCX01HTVRfUFJPVE9fR1MgPSAoMSA8PCAyKSwgICAvKiBzdXBwb3J0cyBJQiBHU0kgTUFEcyAo ZS5nLiBQTSwgLi4uKSAqLw0KCUlCX01HTVRfUFJPVE9fQ00gPSAoMSA8PCAzKSwgICAvKiBJQiBD TSBjYWxsZWQgb3V0IHNlcGFyYXRlbHkgKi8NCglJQl9NR01UX1BST1RPX0lXX0NNID0gKDEgPDwg NCksLyogaVdhcnAgQ00gKi8NCgkvKiBPUEEgY2FuIGRlZmluZSBuZXcgdmFsdWVzIGhlcmUgKi8N Cn07DQoNCnN0cnVjdCBpYl9wb3J0X2F0dHIgew0KCS4uLg0KCXUzMgltZ210X3Byb3RvOyAgLyog Yml0bWFzayBvZiBzdXBwb3J0ZWQgcHJvdG9jb2xzICovDQp9Ow0KDQpJIGFtIG5vdCBmYW1pbGlh ciBlbm91Z2ggd2l0aCBSb0NFIChJQm9FKSB0byBrbm93IG9mZiB0aGUgdG9wIG9mIG15IGhlYWQg aWYgdGhpcyBicmVha2Rvd24gd29ya3MgYXMgSSBkZWZpbmVkIGl0LCBvciBpZiBJQl9NR01UX1BS T1RPX0dTIG5lZWRzIHRvIGJlIHNlcGFyYXRlZCBpbnRvIG1vcmUgbWdtdCBjbGFzc2VzLiAgKEhh bCBvciBJcmEgbWlnaHQuKSAgSSBzZXBhcmF0ZWQgb3V0IHRoZSBDTSBjbGFzcywgYXMgdGhlIHJk bWEgY20gaGFzIGNoZWNrcyB3aGVyZSBpdCB3YW50cyB0byBkaXN0aW5ndWlzaCBiZXR3ZWVuIHdo aWNoIENNIHByb3RvY29sIHRvIGV4ZWN1dGUgKElCIG9yIGlXYXJwKS4NCg0KVGhpcyBjaGFuZ2Ug d291bGQgYmUgbGltaXRlZCB0byBtYW5hZ2VtZW50IGNoZWNrcyBvbmx5LiAgVGhlcmUgbWF5IHN0 aWxsIGJlIHBsYWNlcyBpbiB0aGUgY29kZSB3aGVyZSB0aGUgbGluayBhbmQgdHJhbnNwb3J0IGNo ZWNrcyB3b3VsZCBjb250aW51ZSB0byBleGlzdC4gIEFnYWluLCB0aGlzIGlzIGp1c3QgYSBzdWdn ZXN0aW9uLiAgV2l0aG91dCBhY3R1YWxseSBpbXBsZW1lbnRpbmcgdGhlIHBhdGNoLCBJIGRvbid0 IGtub3cgaWYgdGhpcyB3b3VsZCBzaW1wbGlmeSB0aGluZ3MuICBUaGUgY2hlY2tzIGluIHRoZSBy ZG1hIGNtLCBpbiBwYXJ0aWN1bGFyLCBhcmUgbWVzc3kuDQoNCi0gU2Vhbg0K
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A8237399E82E58-P5GAC/[email protected]
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 12 lines · Sean
Quoted 1 line
[snip]
To restate my suggesting, I was thinking of defining something like this:
enum { IB_MGMT_PROTO_SM = (1 << 0), /* supports IB SMPs */ IB_MGMT_PROTO_SA = (1 << 1), /* supports IB SA MADs */ IB_MGMT_PROTO_GS = (1 << 2), /* supports IB GSI MADs (e.g. PM, ...) */ IB_MGMT_PROTO_CM = (1 << 3), /* IB CM called out separately */ IB_MGMT_PROTO_IW_CM = (1 << 4),/* iWarp CM */ /* OPA can define new values here */ };
Quoted 4 lines
struct ib_port_attr {
...
u32 mgmt_proto; /* bitmask of supported protocols */
};Thanks for the restate, Sean :) seems like your proposal is also to ask vendor setup 'mgmt_proto' during ib_query_port(), correct?
Quoted 2 lines
I am not familiar enough with RoCE (IBoE) to know off the top of my head if this breakdown works as I defined it, or if
IB_MGMT_PROTO_GSneeds to be separated into more mgmt classes. (Hal or Ira might.) I separated out the CM class, as the rdma cm has checks where it wants to distinguish between which CM protocol to execute (IB or iWarp).
Maybe we can apply this thought to CM part firstly?
Quoted 2 lines
This change would be limited to management checks only. There may still be places in the code where the link and transport checks would continue to exist. Again, this is just a suggestion. Without actually implementing the patch, I don't know if this would simplify things. The checks in the rdma cm, in particular, are messy.
I think it's time to make a formal patch now and discuss the problem in a separate thread, I'll still use the mechanism in draft and apply these flags, let's see if it satisfied peoples ;-)
Regards,
Michael Wang
Quoted 1 line
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Hi, Sean
Quoted 14 lines · Michael Wang
Quoted 13 lines · Sean
Quoted 1 line
[snip]
To restate my suggesting, I was thinking of defining something like this:
enum { IB_MGMT_PROTO_SM = (1 << 0), /* supports IB SMPs */ IB_MGMT_PROTO_SA = (1 << 1), /* supports IB SA MADs */ IB_MGMT_PROTO_GS = (1 << 2), /* supports IB GSI MADs (e.g. PM, ...) */ IB_MGMT_PROTO_CM = (1 << 3), /* IB CM called out separately */ IB_MGMT_PROTO_IW_CM = (1 << 4),/* iWarp CM */ /* OPA can define new values here */ };
Quoted 7 lines
Quoted 4 lines
struct
ib_port_attr{
...
u32mgmt_proto; /* bitmask of supported protocols */
};Thanks for the restate, Sean :) seems like your proposal is also to ask vendor setup '
mgmt_proto' duringib_query_port(), correct?
I think we got one problem here, if we rely on ib_query_port() to setup mgmt flags each time, the performance may suffered, since some implementation of query_port() is really expensive, like hardware communication (mlx4/5) and mutex protection (usnic)...
And also I found that the current implementation doesn't match the idea very well, for example CM, I haven't found any scene to check whether a specified port support CM or not (correct me please), mostly only check the device rather than it's port, SM is checking the port, however since we already verified transport layer at beginning, just check link layer sounds not that bad...
Thus I think using flags may not very helpful to the current implementation, may be use some helper to refine the code is more applicable?
I'll send out the patch later with just helpers, we can discuss in that thread see if there is any better solutions ;-)
Regards,
Michael Wang
Quoted 6 lines
Quoted 3 lines
I am not familiar enough with RoCE (IBoE) to know off the top of my head if this breakdown works as I defined it, or if
IB_MGMT_PROTO_GSneeds to be separated into more mgmt classes. (Hal or Ira might.) I separated out the CM class, as the rdma cm has checks where it wants to distinguish between which CM protocol to execute (IB or iWarp).Maybe we can apply this thought to CM part firstly?
Quoted 1 line
Quoted 10 lines
Quoted 1 line
This change would be limited to management checks only. There may still be places in the code where the link and transport checks would continue to exist. Again, this is just a suggestion. Without actually implementing the patch, I don't know if this would simplify things. The checks in the rdma cm, in particular, are messy.
I think it's time to make a formal patch now and discuss the problem in a separate thread, I'll still use the mechanism in draft and apply these flags, let's see if it satisfied peoples ;-)
Regards,
Michael WangQuoted 1 line
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 27 lines · Michael Wang
Hi, Sean
Quoted 15 lines · Michael Wang
Quoted 14 lines · Sean
Quoted 1 line
[snip]
To restate my suggesting, I was thinking of defining something like this:
enum { IB_MGMT_PROTO_SM = (1 << 0), /* supports IB SMPs */ IB_MGMT_PROTO_SA = (1 << 1), /* supports IB SA MADs */ IB_MGMT_PROTO_GS = (1 << 2), /* supports IB GSI MADs (e.g. PM,...) */
IB_MGMT_PROTO_CM= (1 << 3), /* IB CM called out separately */
IB_MGMT_PROTO_IW_CM= (1 << 4),/* iWarp CM */
/* OPA can define new values here */
};Quoted 8 lines
Quoted 4 lines
struct
ib_port_attr{
...
u32mgmt_proto; /* bitmask of supported protocols */
};Thanks for the restate, Sean :) seems like your proposal is also to ask vendor
setup 'mgmt_proto' duringib_query_port(), correct?
Quoted 5 lines
I think we got one problem here, if we rely on
ib_query_port()to setup mgmt flags
each time, the performance may suffered, since some implementation ofquery_port()is really expensive, like hardware communication (mlx4/5) and mutex protection (usnic)...
This is why my OPA patch series included a cache of the device attributes.
https://www.mail-archive.com/linux-rdma%40vger.kernel.org/msg22827.html
This was suggested by Or and incorporated rather than having the MAD stack issue ib_query_device. The same could be done for port attributes.
That said I would like to see the next version of your patch series. We have had a lot of churn on that thread and I think you have a lot of things you were going to incorporate. :-D
Thanks!
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
In reply to a message not in the archiveQuoted 4 lines · Ira
I think this is going to break quite a bit. I have prototyped setting OPA devices to "OPA Link Layer" and the perftest tools just fall over. Any changes to the Link layer or the transport types will require a transition period for ULPs.
How do the perftest tools work with OPA in the first place? OPA seems to have 32 bit lids. Do you mean it 'works' as long as the lid is < 16 bits? Same general point about all of verbs, lots of 'uint16_t lid' in the interfaces?
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20150306174729.GE22375-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org
RE: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 11 lines · Ira
Quoted 5 lines
I think this is going to break quite a bit. I have prototyped setting OPA devices to "OPA Link Layer" and the perftest tools just fall over. Any changes to the Link layer or the transport types will require a transition period for ULPs.
How do the perftest tools work with OPA in the first place? OPA seems to have 32 bit lids. Do you mean it 'works' as long as the lid is < 16 bits? Same general point about all of verbs, lots of '
uint16_tlid' in the interfaces?
The 32 bit LIDs in the SMP are designed for future expansion. Currently OPA does not support > 16 bit LIDs.
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 2807E5FD2F6FDA4886F6618EAC48510E0CC53437-8k97q/[email protected]
Re: [PATCH v4 14/19] IB/core: Add IB_DEVICE_OPA_MAD_SUPPORT device cap flag
Quoted 8 lines · Ira
Quoted 8 lines · Ira
Quoted 5 lines
I think this is going to break quite a bit. I have prototyped setting OPA devices to "OPA Link Layer" and the perftest tools just fall over. Any changes to the Link layer or the transport types will require a transition period for ULPs.
Quoted 5 lines
Quoted 3 lines
How do the perftest tools work with OPA in the first place? OPA seems to have 32 bit lids. Do you mean it 'works' as long as the lid is < 16 bits? Same general point about all of verbs, lots of '
uint16_tlid' in the interfaces?The 32 bit LIDs in the SMP are designed for future expansion. Currently OPA does not support > 16 bit LIDs.
It's not just verbs (structures and APIs) but also CM and any other place where LID is used which are numerous.
Is a new verbs coming for this ? How will compatibility be dealt with ?
This is also another example why a more complete picture of OPA is needed.
-- Hal
Quoted 4 lines
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 15/19 IB/mad: Create jumbo_mad data structures
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
Define jumbo_mad and jumbo_rmpp_mad.
Jumbo MAD structures are 2K versions of ib_mad and ib_rmpp_mad structures. Currently only OPA base version MADs are of this type.
Create an RMPP Base header to share between ib_rmpp_mad and jumbo_rmpp_mad
Update existing code to use the new structures.
--- drivers/infiniband/core/mad.c | 18 +++--- drivers/infiniband/core/mad_priv.h | 2 + drivers/infiniband/core/mad_rmpp.c | 120 ++++++++++++++++++------------------- drivers/infiniband/core/user_mad.c | 16 ++--- include/rdma/ib_mad.h | 26 +++++++- 5 files changed, 103 insertions(+), 79 deletions(-)
drivers/infiniband/core/mad.c +9 -9
@@ -883,7 +883,7 @@ static int alloc_send_rmpp_list(struct ib_mad_send_wr_private *send_wr, gfp_t gfp_mask) {struct ib_mad_send_buf *send_buf = &send_wr->send_buf; -struct ib_rmpp_mad *rmpp_mad = send_buf->mad; +struct ib_rmpp_base *rmpp_base = send_buf->mad;struct ib_rmpp_segment *seg =NULL ;int left, seg_size, pad;
@@ -909,10 +909,10 @@ static int alloc_send_rmpp_list(struct ib_mad_send_wr_private *send_wr,if (pad)memset (seg->data + seg_size - pad,0 , pad); - rmpp_mad->rmpp_hdr.rmpp_version = send_wr->mad_agent_priv-> + rmpp_base->rmpp_hdr.rmpp_version = send_wr->mad_agent_priv-> agent.rmpp_version; - rmpp_mad->rmpp_hdr.rmpp_type =IB_MGMT_RMPP_TYPE_DATA ; -ib_set_rmpp_flags (&rmpp_mad->rmpp_hdr,IB_MGMT_RMPP_FLAG_ACTIVE ); + rmpp_base->rmpp_hdr.rmpp_type =IB_MGMT_RMPP_TYPE_DATA ; +ib_set_rmpp_flags (&rmpp_base->rmpp_hdr,IB_MGMT_RMPP_FLAG_ACTIVE ); send_wr->cur_seg =container_of (send_wr->rmpp_list.next, struct ib_rmpp_segment, list);
@@ -1748,14 +1748,14 @@ out:static int is_rmpp_data_mad (struct ib_mad_agent_private *mad_agent_priv,struct ib_mad_hdr *mad_hdr) { -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base; - rmpp_mad = (struct ib_rmpp_mad *)mad_hdr; + rmpp_base = (struct ib_rmpp_base *)mad_hdr;return !mad_agent_priv->agent.rmpp_version || !ib_mad_kernel_rmpp_agent (&mad_agent_priv->agent) || - !(ib_get_rmpp_flags (&rmpp_mad->rmpp_hdr) & + !(ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ) || - (rmpp_mad->rmpp_hdr.rmpp_type ==IB_MGMT_RMPP_TYPE_DATA ); + (rmpp_base->rmpp_hdr.rmpp_type ==IB_MGMT_RMPP_TYPE_DATA ); } static inline int rcv_has_same_class(struct ib_mad_send_wr_private *wr,
@@ -1897,7 +1897,7 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv,spin_unlock_irqrestore (&mad_agent_priv->lock, flags);if (!ib_mad_kernel_rmpp_agent (&mad_agent_priv->agent) &&ib_is_mad_class_rmpp (mad_recv_wc->recv_buf.mad->mad_hdr.mgmt_class) - && (ib_get_rmpp_flags (&((struct ib_rmpp_mad *)mad_recv_wc->recv_buf.mad)->rmpp_hdr) + && (ib_get_rmpp_flags (&((struct ib_rmpp_base *)mad_recv_wc->recv_buf.mad)->rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE )) {/* user rmpp is in effect * and this is an active RMPP MAD
drivers/infiniband/core/mad_priv.h +2 -0
@@ -80,6 +80,8 @@ struct ib_mad_private {struct ib_mad mad;struct ib_rmpp_mad rmpp_mad;struct ib_smp smp; +struct jumbo_mad jumbo_mad; +struct jumbo_rmpp_mad jumbo_rmpp_mad; } mad; } __attribute__ ((packed));
drivers/infiniband/core/mad_rmpp.c +60 -60
276 lines not on this page · raw message
drivers/infiniband/core/user_mad.c +8 -8
@@ -448,7 +448,7 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf,struct ib_mad_agent *agent;struct ib_ah_attr ah_attr;struct ib_ah *ah; -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base;__be64 *tid;int ret, data_len, hdr_len, copy_offset, rmpp_active;
@@ -504,13 +504,13 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf, goto err_up; } - rmpp_mad = (struct ib_rmpp_mad *) packet->mad.data; - hdr_len =ib_get_mad_data_offset (rmpp_mad->mad_hdr.mgmt_class); + rmpp_base = (struct ib_rmpp_base *) packet->mad.data; + hdr_len =ib_get_mad_data_offset (rmpp_base->mad_hdr.mgmt_class); -if (ib_is_mad_class_rmpp (rmpp_mad->mad_hdr.mgmt_class) +if (ib_is_mad_class_rmpp (rmpp_base->mad_hdr.mgmt_class) &&ib_mad_kernel_rmpp_agent (agent)) { copy_offset =IB_MGMT_RMPP_HDR ; - rmpp_active =ib_get_rmpp_flags (&rmpp_mad->rmpp_hdr) & + rmpp_active =ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ; }else { copy_offset =IB_MGMT_MAD_HDR ;
@@ -558,12 +558,12 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf, tid = &((struct ib_mad_hdr *) packet->msg->mad)->tid; *tid =cpu_to_be64 (((u64 ) agent->hi_tid) <<32 | (be64_to_cpup (tid) &0xffffffff )); - rmpp_mad->mad_hdr.tid = *tid; + rmpp_base->mad_hdr.tid = *tid; }if (!ib_mad_kernel_rmpp_agent (agent) - &&ib_is_mad_class_rmpp (rmpp_mad->mad_hdr.mgmt_class) - && (ib_get_rmpp_flags (&rmpp_mad->rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE )) { + &&ib_is_mad_class_rmpp (rmpp_base->mad_hdr.mgmt_class) + && (ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE )) {spin_lock_irq (&file->send_lock);list_add_tail (&packet->list, &file->send_list);spin_unlock_irq (&file->send_lock);
include/rdma/ib_mad.h +24 -2
@@ -136,6 +136,11 @@ enum {IB_MGMT_DEVICE_HDR =64 ,IB_MGMT_DEVICE_DATA =192 ,IB_MGMT_MAD_SIZE =IB_MGMT_MAD_HDR +IB_MGMT_MAD_DATA , +JUMBO_MGMT_MAD_HDR =IB_MGMT_MAD_HDR , +JUMBO_MGMT_MAD_DATA =2024 , +JUMBO_MGMT_RMPP_HDR =IB_MGMT_RMPP_HDR , +JUMBO_MGMT_RMPP_DATA =2012 , + JUMBO_MGMT_MAD_SIZE = JUMBO_MGMT_MAD_HDR + JUMBO_MGMT_MAD_DATA, }; struct ib_mad_hdr {
@@ -182,12 +187,26 @@ struct ib_mad {u8 data[IB_MGMT_MAD_DATA ]; }; -struct ib_rmpp_mad { +struct jumbo_mad { +struct ib_mad_hdr mad_hdr; +u8 data[JUMBO_MGMT_MAD_DATA ]; +}; + +struct ib_rmpp_base {struct ib_mad_hdr mad_hdr;struct ib_rmpp_hdr rmpp_hdr; +} __packed; + +struct ib_rmpp_mad { +struct ib_rmpp_base base;u8 data[IB_MGMT_RMPP_DATA ]; }; +struct jumbo_rmpp_mad { +struct ib_rmpp_base base; +u8 data[JUMBO_MGMT_RMPP_DATA ]; +}; +struct ib_sa_mad {struct ib_mad_hdr mad_hdr;struct ib_rmpp_hdr rmpp_hdr;
@@ -402,7 +421,10 @@ struct ib_mad_send_wc {struct ib_mad_recv_buf {struct list_head list;struct ib_grh *grh; -struct ib_mad *mad; +union { +struct ib_mad *mad; +struct jumbo_mad *jumbo_mad; + }; };/**
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-16-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
RE: [PATCH v4 15/19] IB/mad: Create jumbo_mad data structures
Quoted 1 line
Define
jumbo_madandjumbo_rmpp_mad.
I would just use 'opa_mad' in place of 'jumbo_mad'. Jumbo sounds like a marketing term or elephant name.
Quoted 484 lines
Jumbo MAD structures are 2K versions of
ib_madandib_rmpp_madstructures. Currently only OPA base version MADs are of this type.Create an RMPP Base header to share between
ib_rmpp_madandjumbo_rmpp_madUpdate existing code to use the new structures.
Signed-off-by Ira Weiny <[email protected]>--- drivers/infiniband/core/mad.c | 18 +++--- drivers/infiniband/core/mad_priv.h | 2 + drivers/infiniband/core/mad_rmpp.c | 120 ++++++++++++++++++-------------- ----- drivers/infiniband/core/user_mad.c | 16 ++--- include/rdma/ib_mad.h | 26 +++++++- 5 files changed, 103 insertions(+), 79 deletions(-)drivers/infiniband/core/mad.c +4
-4@@ -883,7 +883,7 @@ static int alloc_send_rmpp_list(struct ib_mad_send_wr_private *send_wr, gfp_t gfp_mask) {struct ib_mad_send_buf *send_buf = &send_wr->send_buf; -struct ib_rmpp_mad *rmpp_mad = send_buf->mad; +struct ib_rmpp_base *rmpp_base = send_buf->mad;struct ib_rmpp_segment *seg =NULL ;int left, seg_size, pad;@@ -909,10 +909,10 @@ static int alloc_send_rmpp_list(struct ib_mad_send_wr_private *send_wr, if (pad) memset(seg->data + seg_size - pad, 0, pad); - rmpp_mad->rmpp_hdr.rmpp_version = send_wr->mad_agent_priv-> + rmpp_base->rmpp_hdr.rmpp_version = send_wr->mad_agent_priv-> agent.rmpp_version; - rmpp_mad->rmpp_hdr.rmpp_type =IB_MGMT_RMPP_TYPE_DATA ; -ib_set_rmpp_flags (&rmpp_mad->rmpp_hdr,IB_MGMT_RMPP_FLAG_ACTIVE ); + rmpp_base->rmpp_hdr.rmpp_type =IB_MGMT_RMPP_TYPE_DATA ; +ib_set_rmpp_flags (&rmpp_base->rmpp_hdr,IB_MGMT_RMPP_FLAG_ACTIVE ); send_wr->cur_seg = container_of(send_wr->rmpp_list.next,drivers/infiniband/core/mad.c +5
-5struct ib_rmpp_segment, list);@@ -1748,14 +1748,14 @@ out:static int is_rmpp_data_mad (struct ib_mad_agent_private *mad_agent_priv,struct ib_mad_hdr *mad_hdr) { -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base; - rmpp_mad = (struct ib_rmpp_mad *)mad_hdr; + rmpp_base = (struct ib_rmpp_base *)mad_hdr;return !mad_agent_priv->agent.rmpp_version || !ib_mad_kernel_rmpp_agent (&mad_agent_priv->agent) || - !(ib_get_rmpp_flags (&rmpp_mad->rmpp_hdr) & + !(ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ) || - (rmpp_mad->rmpp_hdr.rmpp_type ==IB_MGMT_RMPP_TYPE_DATA ); + (rmpp_base->rmpp_hdr.rmpp_type ==IB_MGMT_RMPP_TYPE_DATA ); } static inline int rcv_has_same_class(struct ib_mad_send_wr_private *wr,@@ -1897,7 +1897,7 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv,spin_unlock_irqrestore (&mad_agent_priv->lock, flags);if (!ib_mad_kernel_rmpp_agent (&mad_agent_priv->agent) &&ib_is_mad_class_rmpp (mad_recv_wc->recv_buf.mad- >mad_hdr.mgmt_class) - && (ib_get_rmpp_flags(&((struct ib_rmpp_mad *)mad_recv_wc->recv_buf.mad)->rmpp_hdr) + && (ib_get_rmpp_flags(&((struct ib_rmpp_base*)mad_recv_wc->recv_buf.mad)->rmpp_hdr)
drivers/infiniband/core/mad.c +0
-0& IB_MGMT_RMPP_FLAG_ACTIVE)) {/* user rmpp is in effect * and this is an active RMPP MAD drivers/infiniband/core/mad_priv.h +2
-0@@ -80,6 +80,8 @@ struct ib_mad_private {struct ib_mad mad;struct ib_rmpp_mad rmpp_mad;struct ib_smp smp; +struct jumbo_mad jumbo_mad; +struct jumbo_rmpp_mad jumbo_rmpp_mad; } mad; } __attribute__ ((packed));drivers/infiniband/core/mad_rmpp.c +2
-2@@ -111,10 +111,10 @@ void ib_cancel_rmpp_recvs(struct ib_mad_agent_private *agent) }static void format_ack (struct ib_mad_send_buf *msg, -struct ib_rmpp_mad *data, +struct ib_rmpp_base *data,struct mad_rmpp_recv *rmpp_recv) { -struct ib_rmpp_mad *ack = msg->mad; +struct ib_rmpp_base *ack = msg->mad;unsigned long flags;drivers/infiniband/core/mad_rmpp.c +1
-1memcpy (ack, &data->mad_hdr, msg->hdr_len);@@ -144,7 +144,7 @@ static void ack_recv(struct mad_rmpp_recv *rmpp_recv,if (IS_ERR (msg))return ; -format_ack (msg, (struct ib_rmpp_mad *) recv_wc->recv_buf.mad, rmpp_recv); +format_ack (msg, (struct ib_rmpp_base *) recv_wc->recv_buf.mad, rmpp_recv); msg->ah = rmpp_recv->ah;drivers/infiniband/core/mad_rmpp.c +7
-7ret =ib_post_send_mad (msg,NULL );if (ret)@@ -182,20 +182,20 @@ static void ack_ds_ack(struct ib_mad_agent_private *agent, struct ib_mad_recv_wc *recv_wc) {struct ib_mad_send_buf *msg; -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base;int ret; msg =alloc_response_msg (&agent->agent, recv_wc);if (IS_ERR (msg))return ; - rmpp_mad = msg->mad; -memcpy (rmpp_mad, recv_wc->recv_buf.mad, msg->hdr_len); + rmpp_base = msg->mad; +memcpy (rmpp_base, recv_wc->recv_buf.mad, msg->hdr_len); - rmpp_mad->mad_hdr.method ^=IB_MGMT_METHOD_RESP ; -ib_set_rmpp_flags (&rmpp_mad->rmpp_hdr,IB_MGMT_RMPP_FLAG_ACTIVE ); - rmpp_mad->rmpp_hdr.seg_num =0 ; - rmpp_mad->rmpp_hdr.paylen_newwin =cpu_to_be32 (1 ); + rmpp_base->mad_hdr.method ^=IB_MGMT_METHOD_RESP ; +ib_set_rmpp_flags (&rmpp_base->rmpp_hdr,IB_MGMT_RMPP_FLAG_ACTIVE ); + rmpp_base->rmpp_hdr.seg_num =0 ; + rmpp_base->rmpp_hdr.paylen_newwin =cpu_to_be32 (1 ); ret =ib_post_send_mad (msg,NULL );drivers/infiniband/core/mad_rmpp.c +10
-10if (ret) {@@ -215,23 +215,23 @@ static void nack_recv(struct ib_mad_agent_private *agent, struct ib_mad_recv_wc *recv_wc, u8 rmpp_status) {struct ib_mad_send_buf *msg; -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base;int ret; msg =alloc_response_msg (&agent->agent, recv_wc);if (IS_ERR (msg))return ; - rmpp_mad = msg->mad; -memcpy (rmpp_mad, recv_wc->recv_buf.mad, msg->hdr_len); + rmpp_base = msg->mad; +memcpy (rmpp_base, recv_wc->recv_buf.mad, msg->hdr_len); - rmpp_mad->mad_hdr.method ^=IB_MGMT_METHOD_RESP ; - rmpp_mad->rmpp_hdr.rmpp_version =IB_MGMT_RMPP_VERSION ; - rmpp_mad->rmpp_hdr.rmpp_type =IB_MGMT_RMPP_TYPE_ABORT ; -ib_set_rmpp_flags (&rmpp_mad->rmpp_hdr,IB_MGMT_RMPP_FLAG_ACTIVE ); - rmpp_mad->rmpp_hdr.rmpp_status = rmpp_status; - rmpp_mad->rmpp_hdr.seg_num =0 ; - rmpp_mad->rmpp_hdr.paylen_newwin =0 ; + rmpp_base->mad_hdr.method ^=IB_MGMT_METHOD_RESP ; + rmpp_base->rmpp_hdr.rmpp_version =IB_MGMT_RMPP_VERSION ; + rmpp_base->rmpp_hdr.rmpp_type =IB_MGMT_RMPP_TYPE_ABORT ; +ib_set_rmpp_flags (&rmpp_base->rmpp_hdr,IB_MGMT_RMPP_FLAG_ACTIVE ); + rmpp_base->rmpp_hdr.rmpp_status = rmpp_status; + rmpp_base->rmpp_hdr.seg_num =0 ; + rmpp_base->rmpp_hdr.paylen_newwin =0 ; ret =ib_post_send_mad (msg,NULL );drivers/infiniband/core/mad_rmpp.c +6
-6if (ret) {@@ -373,18 +373,18 @@ insert_rmpp_recv(struct ib_mad_agent_private *agent,static inline int get_last_flag (struct ib_mad_recv_buf *seg) { -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base; - rmpp_mad = (struct ib_rmpp_mad *) seg->mad; -return ib_get_rmpp_flags (&rmpp_mad->rmpp_hdr) &IB_MGMT_RMPP_FLAG_LAST ; + rmpp_base = (struct ib_rmpp_base *) seg->mad; +return ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) &IB_MGMT_RMPP_FLAG_LAST ; }static inline int get_seg_num (struct ib_mad_recv_buf *seg) { -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base; - rmpp_mad = (struct ib_rmpp_mad *) seg->mad; -return be32_to_cpu (rmpp_mad->rmpp_hdr.seg_num); + rmpp_base = (struct ib_rmpp_base *) seg->mad; +return be32_to_cpu (rmpp_base->rmpp_hdr.seg_num); }static inline struct
ib_mad_recv_buf* get_next_seg(structlist_head*rmpp_list,drivers/infiniband/core/mad_rmpp.c +2
-2@@ -436,9 +436,9 @@ static inline int get_mad_len(struct mad_rmpp_recv *rmpp_recv) rmpp_mad = (struct ib_rmpp_mad *)rmpp_recv->cur_seg_buf->mad; - hdr_size =ib_get_mad_data_offset (rmpp_mad->mad_hdr.mgmt_class); + hdr_size =ib_get_mad_data_offset (rmpp_mad- >base.mad_hdr.mgmt_class); data_size =sizeof (struct ib_rmpp_mad ) - hdr_size; - pad =IB_MGMT_RMPP_DATA -be32_to_cpu (rmpp_mad- >rmpp_hdr.paylen_newwin); + pad = IB_MGMT_RMPP_DATA - be32_to_cpu(rmpp_mad-Quoted 1 line
base.rmpp_hdr.paylen_newwin);
drivers/infiniband/core/mad_rmpp.c +4
-4if (pad >IB_MGMT_RMPP_DATA || pad <0 ) pad =0 ;@@ -567,20 +567,20 @@ static int send_next_seg(struct ib_mad_send_wr_private *mad_send_wr) u32 paylen = 0; rmpp_mad = mad_send_wr->send_buf.mad; -ib_set_rmpp_flags (&rmpp_mad->rmpp_hdr,IB_MGMT_RMPP_FLAG_ACTIVE ); - rmpp_mad->rmpp_hdr.seg_num =cpu_to_be32 (++mad_send_wr->seg_num); +ib_set_rmpp_flags (&rmpp_mad->base.rmpp_hdr,IB_MGMT_RMPP_FLAG_ACTIVE ); + rmpp_mad->base.rmpp_hdr.seg_num =cpu_to_be32 (++mad_send_wr- >seg_num);if (mad_send_wr->seg_num ==1 ) { - rmpp_mad->rmpp_hdr.rmpp_rtime_flags |=IB_MGMT_RMPP_FLAG_FIRST ; + rmpp_mad->base.rmpp_hdr.rmpp_rtime_flags |=IB_MGMT_RMPP_FLAG_FIRST ; paylen = mad_send_wr->send_buf.seg_count *IB_MGMT_RMPP_DATA - mad_send_wr->pad; }if (mad_send_wr->seg_num == mad_send_wr->send_buf.seg_count) { - rmpp_mad->rmpp_hdr.rmpp_rtime_flags |=IB_MGMT_RMPP_FLAG_LAST ; + rmpp_mad->base.rmpp_hdr.rmpp_rtime_flags |=IB_MGMT_RMPP_FLAG_LAST ;drivers/infiniband/core/mad_rmpp.c +6
-6paylen =IB_MGMT_RMPP_DATA - mad_send_wr->pad; } - rmpp_mad->rmpp_hdr.paylen_newwin =cpu_to_be32 (paylen); + rmpp_mad->base.rmpp_hdr.paylen_newwin =cpu_to_be32 (paylen);/* 2 seconds for an ACK until we can find the packet lifetime */ timeout = mad_send_wr->send_buf.timeout_ms;@@ -644,19 +644,19 @@ static void process_rmpp_ack(structib_mad_agent_private *agent, struct ib_mad_recv_wc *mad_recv_wc) { struct ib_mad_send_wr_private *mad_send_wr; -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base;unsigned long flags;int seg_num, newwin, ret; - rmpp_mad = (struct ib_rmpp_mad *)mad_recv_wc->recv_buf.mad; -if (rmpp_mad->rmpp_hdr.rmpp_status) { + rmpp_base = (struct ib_rmpp_base *)mad_recv_wc->recv_buf.mad; +if (rmpp_base->rmpp_hdr.rmpp_status) {abort_send (agent, mad_recv_wc,IB_MGMT_RMPP_STATUS_BAD_STATUS );nack_recv (agent, mad_recv_wc,IB_MGMT_RMPP_STATUS_BAD_STATUS );return ; } - seg_num =be32_to_cpu (rmpp_mad->rmpp_hdr.seg_num); - newwin =be32_to_cpu (rmpp_mad->rmpp_hdr.paylen_newwin); + seg_num =be32_to_cpu (rmpp_base->rmpp_hdr.seg_num); + newwin =be32_to_cpu (rmpp_base->rmpp_hdr.paylen_newwin);if (newwin < seg_num) {drivers/infiniband/core/mad_rmpp.c +1
-1abort_send (agent, mad_recv_wc,IB_MGMT_RMPP_STATUS_W2S );nack_recv (agent, mad_recv_wc,IB_MGMT_RMPP_STATUS_W2S );@@ -741,7 +741,7 @@ process_rmpp_data(struct ib_mad_agent_private *agent,struct ib_rmpp_hdr *rmpp_hdr;u8 rmpp_status; - rmpp_hdr = &((struct ib_rmpp_mad *)mad_recv_wc->recv_buf.mad)- >rmpp_hdr; + rmpp_hdr = &((struct ib_rmpp_base *)mad_recv_wc->recv_buf.mad)- >rmpp_hdr;drivers/infiniband/core/mad_rmpp.c +6
-7if (rmpp_hdr->rmpp_status) { rmpp_status =IB_MGMT_RMPP_STATUS_BAD_STATUS ;@@ -770,30 +770,30 @@ bad:static void process_rmpp_stop (struct ib_mad_agent_private *agent,struct ib_mad_recv_wc *mad_recv_wc) { -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base; - rmpp_mad = (struct ib_rmpp_mad *)mad_recv_wc->recv_buf.mad; + rmpp_base = (struct ib_rmpp_base *)mad_recv_wc->recv_buf.mad; -if (rmpp_mad->rmpp_hdr.rmpp_status !=IB_MGMT_RMPP_STATUS_RESX ) { +if (rmpp_base->rmpp_hdr.rmpp_status !=IB_MGMT_RMPP_STATUS_RESX ) {abort_send (agent, mad_recv_wc,IB_MGMT_RMPP_STATUS_BAD_STATUS );nack_recv (agent, mad_recv_wc,IB_MGMT_RMPP_STATUS_BAD_STATUS ); }else -abort_send (agent, mad_recv_wc, rmpp_mad- >rmpp_hdr.rmpp_status); +abort_send (agent, mad_recv_wc, rmpp_base- >rmpp_hdr.rmpp_status); }static void process_rmpp_abort (struct ib_mad_agent_private *agent,struct ib_mad_recv_wc *mad_recv_wc) { -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base; - rmpp_mad = (struct ib_rmpp_mad *)mad_recv_wc->recv_buf.mad; + rmpp_base = (struct ib_rmpp_base *)mad_recv_wc->recv_buf.mad; - if (rmpp_mad->rmpp_hdr.rmpp_status < IB_MGMT_RMPP_STATUS_ABORT_MIN||
drivers/infiniband/core/mad_rmpp.c +0
-1- rmpp_mad->rmpp_hdr.rmpp_status > IB_MGMT_RMPP_STATUS_ABORT_MAX){
drivers/infiniband/core/mad_rmpp.c +1
-0+ if (rmpp_base->rmpp_hdr.rmpp_status < IB_MGMT_RMPP_STATUS_ABORT_MIN||
drivers/infiniband/core/mad_rmpp.c +1
-0+ rmpp_base->rmpp_hdr.rmpp_status > IB_MGMT_RMPP_STATUS_ABORT_MAX){
abort_send(agent,mad_recv_wc,IB_MGMT_RMPP_STATUS_BAD_STATUS);drivers/infiniband/core/mad_rmpp.c +0
-1nack_recv (agent, mad_recv_wc,IB_MGMT_RMPP_STATUS_BAD_STATUS ); }else - abort_send(agent, mad_recv_wc, rmpp_mad-Quoted 1 line
rmpp_hdr.rmpp_status);
drivers/infiniband/core/mad_rmpp.c +1
-0+ abort_send(agent, mad_recv_wc, rmpp_base-Quoted 1 line
rmpp_hdr.rmpp_status);
}
struct
ib_mad_recv_wc*drivers/infiniband/core/mad_rmpp.c +3
-3@@ -803,16 +803,16 @@ ib_process_rmpp_recv_wc(struct ib_mad_agent_private *agent, struct ib_rmpp_mad *rmpp_mad; rmpp_mad = (struct ib_rmpp_mad *)mad_recv_wc->recv_buf.mad; -if (!(rmpp_mad->rmpp_hdr.rmpp_rtime_flags & IB_MGMT_RMPP_FLAG_ACTIVE)) +if (!(rmpp_mad->base.rmpp_hdr.rmpp_rtime_flags &IB_MGMT_RMPP_FLAG_ACTIVE ))return mad_recv_wc; -if (rmpp_mad->rmpp_hdr.rmpp_version !=IB_MGMT_RMPP_VERSION ) { +if (rmpp_mad->base.rmpp_hdr.rmpp_version !=IB_MGMT_RMPP_VERSION ) {abort_send (agent, mad_recv_wc,IB_MGMT_RMPP_STATUS_UNV );nack_recv (agent, mad_recv_wc,IB_MGMT_RMPP_STATUS_UNV ); goto out; } -switch (rmpp_mad->rmpp_hdr.rmpp_type) { +switch (rmpp_mad->base.rmpp_hdr.rmpp_type) {drivers/infiniband/core/mad_rmpp.c +2
-2case IB_MGMT_RMPP_TYPE_DATA :return process_rmpp_data (agent, mad_recv_wc);case IB_MGMT_RMPP_TYPE_ACK :@@ -873,11 +873,11 @@ int ib_send_rmpp_mad(struct ib_mad_send_wr_private *mad_send_wr)int ret; rmpp_mad = mad_send_wr->send_buf.mad; -if (!(ib_get_rmpp_flags (&rmpp_mad->rmpp_hdr) & +if (!(ib_get_rmpp_flags (&rmpp_mad->base.rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ))return IB_RMPP_RESULT_UNHANDLED ; -if (rmpp_mad->rmpp_hdr.rmpp_type !=IB_MGMT_RMPP_TYPE_DATA ) { +if (rmpp_mad->base.rmpp_hdr.rmpp_type !=IB_MGMT_RMPP_TYPE_DATA ) { mad_send_wr->seg_num =1 ;return IB_RMPP_RESULT_INTERNAL ;drivers/infiniband/core/mad_rmpp.c +4
-4}@@ -895,15 +895,15 @@ int ib_send_rmpp_mad(struct ib_mad_send_wr_private *mad_send_wr)int ib_process_rmpp_send_wc (struct ib_mad_send_wr_private *mad_send_wr,struct ib_mad_send_wc *mad_send_wc) { -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base;int ret; - rmpp_mad = mad_send_wr->send_buf.mad; -if (!(ib_get_rmpp_flags (&rmpp_mad->rmpp_hdr) & + rmpp_base = mad_send_wr->send_buf.mad; +if (!(ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ))return IB_RMPP_RESULT_UNHANDLED ;/* RMPP not active */ -if (rmpp_mad->rmpp_hdr.rmpp_type !=IB_MGMT_RMPP_TYPE_DATA ) +if (rmpp_base->rmpp_hdr.rmpp_type !=IB_MGMT_RMPP_TYPE_DATA )return IB_RMPP_RESULT_INTERNAL ;/* ACK, STOP, or ABORT */ drivers/infiniband/core/mad_rmpp.c +3
-3if (mad_send_wc->status != IB_WC_SUCCESS ||@@ -933,11 +933,11 @@ int ib_process_rmpp_send_wc(structib_mad_send_wr_private *mad_send_wr, int ib_retry_rmpp(struct ib_mad_send_wr_private *mad_send_wr) { - struct ib_rmpp_mad *rmpp_mad; + struct ib_rmpp_base *rmpp_base;int ret; - rmpp_mad = mad_send_wr->send_buf.mad; -if (!(ib_get_rmpp_flags (&rmpp_mad->rmpp_hdr) & + rmpp_base = mad_send_wr->send_buf.mad; +if (!(ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ))return IB_RMPP_RESULT_UNHANDLED ;/* RMPP not active */ drivers/infiniband/core/user_mad.c +5
-5@@ -448,7 +448,7 @@ static ssize_t ib_umad_write(struct file *filp, constchar __user *buf, struct ib_mad_agent *agent;struct ib_ah_attr ah_attr;struct ib_ah *ah; -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base;__be64 *tid;int ret, data_len, hdr_len, copy_offset, rmpp_active;@@ -504,13 +504,13 @@ static ssize_t ib_umad_write(struct file *filp,const char __user *buf, goto err_up; } - rmpp_mad = (struct ib_rmpp_mad *) packet->mad.data; - hdr_len =ib_get_mad_data_offset (rmpp_mad->mad_hdr.mgmt_class); + rmpp_base = (struct ib_rmpp_base *) packet->mad.data; + hdr_len =ib_get_mad_data_offset (rmpp_base->mad_hdr.mgmt_class); -if (ib_is_mad_class_rmpp (rmpp_mad->mad_hdr.mgmt_class) +if (ib_is_mad_class_rmpp (rmpp_base->mad_hdr.mgmt_class) &&ib_mad_kernel_rmpp_agent (agent)) { copy_offset =IB_MGMT_RMPP_HDR ; - rmpp_active =ib_get_rmpp_flags (&rmpp_mad->rmpp_hdr) & + rmpp_active =ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ; }else {drivers/infiniband/core/user_mad.c +3
-3copy_offset =IB_MGMT_MAD_HDR ;@@ -558,12 +558,12 @@ static ssize_t ib_umad_write(struct file *filp,const char __user *buf, tid = &((struct ib_mad_hdr *) packet->msg->mad)->tid; *tid =cpu_to_be64 (((u64 ) agent->hi_tid) <<32 | (be64_to_cpup (tid) &0xffffffff )); - rmpp_mad->mad_hdr.tid = *tid; + rmpp_base->mad_hdr.tid = *tid; }if (!ib_mad_kernel_rmpp_agent (agent) - &&ib_is_mad_class_rmpp (rmpp_mad->mad_hdr.mgmt_class) - && (ib_get_rmpp_flags (&rmpp_mad->rmpp_hdr) & IB_MGMT_RMPP_FLAG_ACTIVE)) { + &&ib_is_mad_class_rmpp (rmpp_base->mad_hdr.mgmt_class) + && (ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) & IB_MGMT_RMPP_FLAG_ACTIVE)) {drivers/infiniband/core/user_mad.c +0
-0spin_lock_irq (&file->send_lock);list_add_tail (&packet->list, &file->send_list);spin_unlock_irq (&file->send_lock);include/rdma/ib_mad.h +0
-0
include/rdma/ib_mad.h enum { · line 143
Quoted 8 lines
include/rdma/ib_mad.h +5
-0@@ -136,6 +136,11 @@ enum {IB_MGMT_DEVICE_HDR =64 ,IB_MGMT_DEVICE_DATA =192 ,IB_MGMT_MAD_SIZE =IB_MGMT_MAD_HDR +IB_MGMT_MAD_DATA , +JUMBO_MGMT_MAD_HDR =IB_MGMT_MAD_HDR , +JUMBO_MGMT_MAD_DATA =2024 , +JUMBO_MGMT_RMPP_HDR =IB_MGMT_RMPP_HDR , +JUMBO_MGMT_RMPP_DATA =2012 , + JUMBO_MGMT_MAD_SIZE = JUMBO_MGMT_MAD_HDR + JUMBO_MGMT_MAD_DATA,
Keep the "IB_" prefix, or add a new "OPA_" prefix.
Quoted 17 lines
include/rdma/ib_mad.h +7
-1};struct ib_mad_hdr {@@ -182,12 +187,26 @@ struct ib_mad {u8 data[IB_MGMT_MAD_DATA ]; }; -struct ib_rmpp_mad { +struct jumbo_mad { +struct ib_mad_hdr mad_hdr; +u8 data[JUMBO_MGMT_MAD_DATA ]; +}; + +struct ib_rmpp_base {struct ib_mad_hdr mad_hdr;struct ib_rmpp_hdr rmpp_hdr; +} __packed;
include/rdma/ib_mad.h struct ib_mad { · line 208
Quoted 10 lines
include/rdma/ib_mad.h +7
-0+ +struct ib_rmpp_mad { +struct ib_rmpp_base base;u8 data[IB_MGMT_RMPP_DATA ]; }; +struct jumbo_rmpp_mad { +struct ib_rmpp_base base; +u8 data[JUMBO_MGMT_RMPP_DATA ]; +};
Please separate this patch into 2 changes. One that adds and updates ib_rmpp_base, with the second one defining ib_opa_mad & ib_opa_rmpp_mad (or whatever prefix is chosen). --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBD6C8-P5GAC/[email protected]
RE: [PATCH v4 15/19] IB/mad: Create jumbo_mad data structures
Quoted 11 lines
include/rdma/ib_mad.h enum { · line 141
Quoted 11 lines
include/rdma/ib_mad.h +3
-0@@ -136,6 +136,11 @@ enum {IB_MGMT_DEVICE_HDR =64 ,IB_MGMT_DEVICE_DATA =192 ,IB_MGMT_MAD_SIZE =IB_MGMT_MAD_HDR +IB_MGMT_MAD_DATA , +JUMBO_MGMT_MAD_HDR =IB_MGMT_MAD_HDR , +JUMBO_MGMT_MAD_DATA =2024 , + JUMBO_MGMT_RMPP_HDR = IB_MGMT_RMPP_HDR,
Examining a later patch in this series highlighted that JUMBO_MGMT_MAD_HDR and JUMBO_MGMT_RMPP_HDR are the same size as the corresponding IB_MGMT_* defines. This seems unnecessary. --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 15/19] IB/mad: Create jumbo_mad data structures
In reply to a message not in the archiveQuoted 3 lines · Sean
Quoted 1 line
Define
jumbo_madandjumbo_rmpp_mad.I would just use '
opa_mad' in place of 'jumbo_mad'. Jumbo sounds like a marketing term or elephant name.
Done in v5.
Quoted 11 lines
Quoted 11 lines
Jumbo MAD structures are 2K versions of
ib_madandib_rmpp_madstructures. Currently only OPA base version MADs are of this type.Create an RMPP Base header to share between
ib_rmpp_madandjumbo_rmpp_madUpdate existing code to use the new structures.
Signed-off-by Ira Weiny <[email protected]>---
[snip]
Quoted 7 lines
Quoted 7 lines
include/rdma/ib_mad.h +0
-0@@ -136,6 +136,11 @@ enum {IB_MGMT_DEVICE_HDR =64 , IB_MGMT_DEVICE_DATA = 192,
Quoted 8 lines
include/rdma/ib_mad.h enum { · line 143
Quoted 6 lines
include/rdma/ib_mad.h +5
-0IB_MGMT_MAD_SIZE =IB_MGMT_MAD_HDR +IB_MGMT_MAD_DATA , +JUMBO_MGMT_MAD_HDR =IB_MGMT_MAD_HDR , +JUMBO_MGMT_MAD_DATA =2024 , +JUMBO_MGMT_RMPP_HDR =IB_MGMT_RMPP_HDR , +JUMBO_MGMT_RMPP_DATA =2012 , + JUMBO_MGMT_MAD_SIZE = JUMBO_MGMT_MAD_HDR + JUMBO_MGMT_MAD_DATA,Keep the "
IB_" prefix, or add a new "OPA_" prefix.
I'll change JUMBO to OPA to match the "jumbo" to "opa" change.
Integrating this and your follow up comment v5 of this hunk now reads:
@@ -136,6 +136,9 @@ enum {
IB_MGMT_DEVICE_HDR = 64,
IB_MGMT_DEVICE_DATA = 192,
IB_MGMT_MAD_SIZE = IB_MGMT_MAD_HDR + IB_MGMT_MAD_DATA,
+ OPA_MGMT_MAD_DATA = 2024,
+ OPA_MGMT_RMPP_DATA = 2012,
+ OPA_MGMT_MAD_SIZE = IB_MGMT_MAD_HDR + OPA_MGMT_MAD_DATA,};
Quoted 24 lines
Quoted 24 lines
include/rdma/ib_mad.h +10
-1};struct ib_mad_hdr {@@ -182,12 +187,26 @@ struct ib_mad {u8 data[IB_MGMT_MAD_DATA ]; }; -struct ib_rmpp_mad { +struct jumbo_mad { +struct ib_mad_hdr mad_hdr; +u8 data[JUMBO_MGMT_MAD_DATA ]; +}; + +struct ib_rmpp_base {struct ib_mad_hdr mad_hdr;struct ib_rmpp_hdr rmpp_hdr; +} __packed; + +struct ib_rmpp_mad { +struct ib_rmpp_base base;u8 data[IB_MGMT_RMPP_DATA ]; };
Quoted 6 lines
include/rdma/ib_mad.h struct ib_mad { · line 208
Quoted 4 lines
include/rdma/ib_mad.h +4
-0+struct jumbo_rmpp_mad { +struct ib_rmpp_base base; +u8 data[JUMBO_MGMT_RMPP_DATA ]; +};Please separate this patch into 2 changes. One that adds and updates
ib_rmpp_base, with the second one definingib_opa_mad&ib_opa_rmpp_mad(or whatever prefix is chosen).
Done in v5.
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 16/19 IB/mad: Add Intel Omni-Path Architecture defines
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
OPA_SMP_CLASS_VERSION -- Defined at 0x80OPA_MGMT_BASE_VERSION -- Defined at 0x80
Increase max management version to accommodate OPA
--- drivers/infiniband/core/mad_priv.h | 2 +- include/rdma/ib_mad.h | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-)
drivers/infiniband/core/mad_priv.h +1 -1
@@ -56,7 +56,7 @@/* Registration table sizes */ #define MAX_MGMT_CLASS 80 -#define MAX_MGMT_VERSION 8 +#define MAX_MGMT_VERSION 0x83#define MAX_MGMT_OUI 8#define MAX_MGMT_VENDOR_RANGE2 (IB_MGMT_CLASS_VENDOR_RANGE2_END - \ IB_MGMT_CLASS_VENDOR_RANGE2_START + 1)
include/rdma/ib_mad.h +4 -1
@@ -42,8 +42,11 @@#include <rdma/ib_verbs.h> #include <uapi/rdma/ib_user_mad.h> -/* Management base version */ +/* Management base versions */ #define IB_MGMT_BASE_VERSION 1 +#define OPA_MGMT_BASE_VERSION 0x80 + +#define OPA_SMP_CLASS_VERSION 0x80/* Management classes */ #define IB_MGMT_CLASS_SUBN_LID_ROUTED 0x01
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-17-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
RE: [PATCH v4 16/19] IB/mad: Add Intel Omni-Path Architecture defines
drivers/infiniband/core/mad_priv.h line 59
Quoted 4 lines
/* Registration table sizes */
#defineMAX_MGMT_CLASS80drivers/infiniband/core/mad_priv.h +1
-1-#define MAX_MGMT_VERSION 8 +#define MAX_MGMT_VERSION 0x83
It's unfortunate that this results in a big jump in used versions. Mad_priv.h defines this:
struct ib_mad_port_private {
...
struct ib_mad_mgmt_version_table version[MAX_MGMT_VERSION];struct ib_mad_mgmt_version_table {
struct ib_mad_mgmt_class_table *class;
struct ib_mad_mgmt_vendor_class_table *vendor;
};This ends up allocating about 2K of data per port of NULL pointers. Not a huge deal, but still.
I don't have a great fix here. Maybe the version[] array can be the necessary size, with some sort of simple mapping function from version to the index? --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBD708-P5GAC/[email protected]
Re: [PATCH v4 16/19] IB/mad: Add Intel Omni-Path Architecture defines
Quoted 6 lines · Sean
drivers/infiniband/core/mad_priv.h line 59
Quoted 4 lines
/* Registration table sizes */
#defineMAX_MGMT_CLASS80drivers/infiniband/core/mad_priv.h +1
-1-#define MAX_MGMT_VERSION 8 +#define MAX_MGMT_VERSION 0x83It's unfortunate that this results in a big jump in used versions.
Mad_priv.hdefines this:
It is unfortunate.
Quoted 11 lines
struct ib_mad_port_private { ... struct ib_mad_mgmt_version_table version[MAX_MGMT_VERSION];struct ib_mad_mgmt_version_table { struct ib_mad_mgmt_class_table *class; struct ib_mad_mgmt_vendor_class_table *vendor; };This ends up allocating about 2K of data per port of NULL pointers. Not a huge deal, but still.
I agree this is not ideal but this is not a large amount of space. Nor is this something which is dynamically being allocated.
Quoted 2 lines
I don't have a great fix here. Maybe the version[] array can be the necessary size, with some sort of simple mapping function from version to the index?
I did not have a great fix either. Hence the current implementation.
Frankly I don't know of many systems that have more than a few ports and at 2K each this does not seem like a big deal.
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 17/19 IB/mad: Implement support for Intel Omni-Path Architecture base version MADs in ib_create_send_mad
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
If the device supports OPA MADs process the OPA base version.
Set MAD size and sg lengths as appropriate Split RMPP MADs as appropriate
--- drivers/infiniband/core/mad.c | 38 ++++++++++++++++++++++++++++---------- 1 file changed, 28 insertions(+), 10 deletions(-)
drivers/infiniband/core/mad.c +28 -10
@@ -857,11 +857,11 @@ out:return ret; } -static int get_pad_size (int hdr_len,int data_len) +static int get_pad_size (int hdr_len,int data_len,size_t mad_size) {int seg_size, pad; - seg_size =sizeof (struct ib_mad ) - hdr_len; + seg_size = mad_size - hdr_len;if (data_len && seg_size) { pad = seg_size - data_len % seg_size;return pad == seg_size ?0 : pad;
@@ -880,14 +880,14 @@ static void free_send_rmpp_list(struct ib_mad_send_wr_private *mad_send_wr) }static int alloc_send_rmpp_list (struct ib_mad_send_wr_private *send_wr, -gfp_t gfp_mask) +size_t mad_size,gfp_t gfp_mask) {struct ib_mad_send_buf *send_buf = &send_wr->send_buf;struct ib_rmpp_base *rmpp_base = send_buf->mad;struct ib_rmpp_segment *seg =NULL ;int left, seg_size, pad; - send_buf->seg_size =sizeof (struct ib_mad ) - send_buf->hdr_len; + send_buf->seg_size = mad_size - send_buf->hdr_len; seg_size = send_buf->seg_size; pad = send_wr->pad;
@@ -937,20 +937,31 @@ struct ib_mad_send_buf * ib_create_send_mad(struct ib_mad_agent *mad_agent,struct ib_mad_send_wr_private *mad_send_wr;int pad, message_size, ret, size;void *buf; +size_t mad_size; +int opa; mad_agent_priv = container_of(mad_agent, struct ib_mad_agent_private, agent); - pad =get_pad_size (hdr_len, data_len); + + opa = mad_agent_priv->agent.device->cached_dev_attrs.device_cap_flags2 & +IB_DEVICE_OPA_MAD_SUPPORT ; + +if (opa && base_version ==OPA_MGMT_BASE_VERSION ) + mad_size =sizeof (struct jumbo_mad ); +else + mad_size =sizeof (struct ib_mad ); + + pad =get_pad_size (hdr_len, data_len, mad_size); message_size = hdr_len + data_len + pad;if (ib_mad_kernel_rmpp_agent (mad_agent)) { -if (!rmpp_active && message_size >sizeof (struct ib_mad )) +if (!rmpp_active && message_size > mad_size)return ERR_PTR (-EINVAL ); }else -if (rmpp_active || message_size >sizeof (struct ib_mad )) +if (rmpp_active || message_size > mad_size)return ERR_PTR (-EINVAL ); - size = rmpp_active ? hdr_len :sizeof (struct ib_mad ); + size = rmpp_active ? hdr_len : mad_size; buf =kzalloc (sizeof *mad_send_wr + size, gfp_mask);if (!buf)return ERR_PTR (-ENOMEM );
@@ -965,7 +976,14 @@ struct ib_mad_send_buf * ib_create_send_mad(struct ib_mad_agent *mad_agent, mad_send_wr->mad_agent_priv = mad_agent_priv; mad_send_wr->sg_list[0 ].length = hdr_len; mad_send_wr->sg_list[0 ].lkey = mad_agent->mr->lkey; - mad_send_wr->sg_list[1 ].length =sizeof (struct ib_mad ) - hdr_len; + +/* OPA MADs don't have to be the full 2048 bytes */ +if (opa && base_version ==OPA_MGMT_BASE_VERSION && + data_len < mad_size - hdr_len) + mad_send_wr->sg_list[1 ].length = data_len; +else + mad_send_wr->sg_list[1 ].length = mad_size - hdr_len; + mad_send_wr->sg_list[1 ].lkey = mad_agent->mr->lkey; mad_send_wr->send_wr.wr_id = (unsigned long ) mad_send_wr;
@@ -978,7 +996,7 @@ struct ib_mad_send_buf * ib_create_send_mad(struct ib_mad_agent *mad_agent, mad_send_wr->send_wr.wr.ud.pkey_index = pkey_index;if (rmpp_active) { - ret =alloc_send_rmpp_list (mad_send_wr, gfp_mask); + ret =alloc_send_rmpp_list (mad_send_wr, mad_size, gfp_mask);if (ret) {kfree (buf);return ERR_PTR (ret);
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-18-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
RE: [PATCH v4 17/19] IB/mad: Implement support for Intel Omni-Path Architecture base version MADs in ib_create_send_mad
Quoted 15 lines
drivers/infiniband/core/mad.c +4
-1@@ -937,20 +937,31 @@ struct ib_mad_send_buf * ib_create_send_mad(structib_mad_agent *mad_agent, struct ib_mad_send_wr_private *mad_send_wr;int pad, message_size, ret, size;void *buf; +size_t mad_size; +int opa; mad_agent_priv = container_of(mad_agent, struct ib_mad_agent_private, agent); - pad =get_pad_size (hdr_len, data_len); + + opa = mad_agent_priv->agent.device-Quoted 1 line
cached_dev_attrs.device_cap_flags2 &
drivers/infiniband/core/mad.c struct ib_mad_send_buf * ib_create_send_mad(struct ib_mad_agent *mad_agent, · line 952
Quoted 6 lines
drivers/infiniband/core/mad.c +6
-0+IB_DEVICE_OPA_MAD_SUPPORT ; + +if (opa && base_version ==OPA_MGMT_BASE_VERSION ) + mad_size =sizeof (struct jumbo_mad ); +else + mad_size =sizeof (struct ib_mad );
Didn't an earlier patch make is possible to read the mad_size directly from the device?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBD722-P5GAC/[email protected]
Re: [PATCH v4 17/19] IB/mad: Implement support for Intel Omni-Path Architecture base version MADs in ib_create_send_mad
Quoted 15 lines · Sean
Quoted 15 lines
drivers/infiniband/core/mad.c +4
-1@@ -937,20 +937,31 @@ struct ib_mad_send_buf * ib_create_send_mad(structib_mad_agent *mad_agent, struct ib_mad_send_wr_private *mad_send_wr;int pad, message_size, ret, size;void *buf; +size_t mad_size; +int opa; mad_agent_priv = container_of(mad_agent, struct ib_mad_agent_private, agent); - pad =get_pad_size (hdr_len, data_len); + + opa = mad_agent_priv->agent.device-Quoted 1 line
cached_dev_attrs.device_cap_flags2 &
Quoted 9 lines
drivers/infiniband/core/mad.c struct ib_mad_send_buf * ib_create_send_mad(struct ib_mad_agent *mad_agent, · line 952
Quoted 6 lines
drivers/infiniband/core/mad.c +6
-0+IB_DEVICE_OPA_MAD_SUPPORT ; + +if (opa && base_version ==OPA_MGMT_BASE_VERSION ) + mad_size =sizeof (struct jumbo_mad ); +else + mad_size =sizeof (struct ib_mad );Didn't an earlier patch make is possible to read the
mad_sizedirectly from the device?
Not exactly, that patch specified the "max_mad_size" the device supported.
OPA devices support IB MADs. If the IB Base Version is specified on an OPA device the IB MAD size is to be used. This maintains compatibility with any software issuing IB MADs.
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20150408213612.GG433-W4f6Xiosr+yv7QzWx2u06xL4W9x8LtSr@public.gmane.org
RE: [PATCH v4 17/19] IB/mad: Implement support for Intel Omni-Path Architecture base version MADs in ib_create_send_mad
Quoted 11 lines
Quoted 1 line
Didn't an earlier patch make is possible to read the
mad_sizedirectlyfrom the device?
Quoted 1 line
Not exactly, that patch specified the "
max_mad_size" the device supported.OPA devices support IB MADs. If the IB Base Version is specified on an OPA
device the IB MAD size is to be used. This maintains compatibility with any
software issuing IB MADs.
Got it - thanks
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 18/19 IB/mad: Implement Intel Omni-Path Architecture SMP processing
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
Define the new OPA SMP format, create support functions for this format using the previously defined helper functions as appropriate.
--- drivers/infiniband/core/mad_priv.h | 2 + drivers/infiniband/core/opa_smi.h | 78 +++++++++++++++++++++++++++ drivers/infiniband/core/smi.c | 54 +++++++++++++++++++ drivers/infiniband/core/smi.h | 6 +++ include/rdma/opa_smi.h | 106 +++++++++++++++++++++++++++++++++++++ 5 files changed, 246 insertions(+) create mode 100644 drivers/infiniband/core/opa_smi.h create mode 100644 include/rdma/opa_smi.h
drivers/infiniband/core/mad_priv.h +2 -0
@@ -41,6 +41,7 @@#include <linux/workqueue.h> #include <rdma/ib_mad.h> #include <rdma/ib_smi.h> +#include <rdma/opa_smi.h> #define IB_MAD_QPS_CORE 2/* Always QP0 and QP1 as a minimum */
@@ -82,6 +83,7 @@ struct ib_mad_private {struct ib_smp smp;struct jumbo_mad jumbo_mad;struct jumbo_rmpp_mad jumbo_rmpp_mad; +struct opa_smp opa_smp; } mad; } __attribute__ ((packed));
drivers/infiniband/core/opa_smi.h new file +78 -0
78 lines not on this page · raw message
drivers/infiniband/core/smi.c +54 -0
87 lines not on this page · raw message
drivers/infiniband/core/smi.h +6 -0
@@ -62,6 +62,9 @@ extern enum smi_action smi_handle_dr_smp_send(struct ib_smp *smp,* Return IB_SMI_HANDLE if the SMP should be handled by the local SMA/SM * via process_mad */ +/* NOTE: This is called on opa_smp's don't check fields which are not common +* between ib_smp and opa_smp +*/ static inline enum smi_action smi_check_local_smp (struct ib_smp *smp,struct ib_device *device) {
@@ -77,6 +80,9 @@ static inline enum smi_action smi_check_local_smp(struct ib_smp *smp,* Return IB_SMI_HANDLE if the SMP should be handled by the local SMA/SM * via process_mad */ +/* NOTE: This is called on opa_smp's don't check fields which are not common +* between ib_smp and opa_smp +*/ static inline enum smi_action smi_check_local_returning_smp (struct ib_smp *smp,struct ib_device *device) {
include/rdma/opa_smi.h new file +106 -0
106 lines not on this page · raw message
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-19-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
RE: [PATCH v4 18/19] IB/mad: Implement Intel Omni-Path Architecture SMP processing
drivers/infiniband/core/smi.c enum smi_action smi_handle_dr_smp_recv(struct ib_smp · line 258
Quoted 8 lines
drivers/infiniband/core/smi.c +3
-0@@ -236,6 +252,24 @@ enum smi_action smi_handle_dr_smp_recv(struct ib_smp *smp, u8 node_type, smp->dr_slid == IB_LID_PERMISSIVE); } +/* +* Adjust information for a received SMP +* Return 0 if the SMP should be dropped
The function returns an enum. The comment of returning 0 is misleading. The entire comment seems unnecessary.
Quoted 66 lines
drivers/infiniband/core/smi.c +15
-0+*/ +enum smi_action opa_smi_handle_dr_smp_recv (struct opa_smp *smp,u8 node_type, +int port_num,int phys_port_cnt) +{ +return __smi_handle_dr_smp_recv (node_type, port_num, phys_port_cnt, + &smp->hop_ptr, smp->hop_cnt, + smp->route.dr.initial_path, + smp->route.dr.return_path, +opa_get_smp_direction (smp), + smp->route.dr.dr_dlid == +OPA_LID_PERMISSIVE , + smp->route.dr.dr_slid == +OPA_LID_PERMISSIVE ); +} + static inlineenum
smi_forward_action__smi_check_forward_dr_smp(u8hop_ptr, u8hop_cnt,drivers/infiniband/core/smi.c +10
-0u8 direction,@@ -277,6 +311,16 @@ enum smi_forward_action smi_check_forward_dr_smp(struct ib_smp *smp) smp->dr_slid != IB_LID_PERMISSIVE); } +enum smi_forward_action opa_smi_check_forward_dr_smp (struct opa_smp *smp) +{ +return __smi_check_forward_dr_smp (smp->hop_ptr, smp->hop_cnt, +opa_get_smp_direction (smp), + smp->route.dr.dr_dlid == +OPA_LID_PERMISSIVE , + smp->route.dr.dr_slid == +OPA_LID_PERMISSIVE ); +} +/* * Return the forwarding port number from initial_path for outgoing SMP and
* fromreturn_pathfor returning SMP(quoted diff) +0
-0@@ -286,3 +330,13 @@ int smi_get_fwd_port(struct ib_smp *smp) return (!ib_get_smp_direction(smp) ? smp->initial_path[smp-Quoted 1 line
hop_ptr+1] :
drivers/infiniband/core/smi.c +3
-0smp->return_path[smp->hop_ptr-1]); } + +/* +* Return the forwarding port number from initial_path for outgoing SMP and
+ * fromreturn_pathfor returning SMP
+ */drivers/infiniband/core/smi.c +3
-0+int opa_smi_get_fwd_port (struct opa_smp *smp) +{ + return !opa_get_smp_direction(smp) ? smp->route.dr.initial_path[smp-Quoted 1 line
hop_ptr+1] :
drivers/infiniband/core/smi.c +2
-0+ smp->route.dr.return_path[smp->hop_ptr-1 ]; +}drivers/infiniband/core/smi.h +1
-0@@ -62,6 +62,9 @@ extern enum smi_action smi_handle_dr_smp_send(structib_smp *smp, * Return IB_SMI_HANDLE if the SMP should be handled by the local SMA/SM * via process_mad */ +/* NOTE: This is called on opa_smp's don't check fields which are not
Quoted 3 lines
common
+ * betweenib_smpandopa_smp
+ */
This comment suggests that the function is not correct for OPA. ?
Quoted 9 lines
static inline enum
smi_actionsmi_check_local_smp(structib_smp*smp,drivers/infiniband/core/smi.h +1
-0struct ib_device *device) {@@ -77,6 +80,9 @@ static inline enum smi_action smi_check_local_smp(structib_smp *smp, * Return IB_SMI_HANDLE if the SMP should be handled by the local SMA/SM * via process_mad */ +/* NOTE: This is called on opa_smp's don't check fields which are not
Quoted 3 lines
common
+ * betweenib_smpandopa_smp
+ */
Same comment
Quoted 4 lines
static inline enum
smi_actionsmi_check_local_returning_smp(structib_smp
*smp,
structib_device*device)
{
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBD742-P5GAC/[email protected]
Re: [PATCH v4 18/19] IB/mad: Implement Intel Omni-Path Architecture SMP processing
Quoted 10 lines · Sean
drivers/infiniband/core/smi.c enum smi_action smi_handle_dr_smp_recv(struct ib_smp · line 258
Quoted 8 lines
drivers/infiniband/core/smi.c +3
-0@@ -236,6 +252,24 @@ enum smi_action smi_handle_dr_smp_recv(struct ib_smp *smp, u8 node_type, smp->dr_slid == IB_LID_PERMISSIVE); } +/* +* Adjust information for a received SMP +* Return 0 if the SMP should be dropped The function returns an enum. The comment of returning 0 is misleading. The entire comment seems unnecessary.
Sorry, I just copied the same comment from the IB function.
/*
* Adjust information for a received SMP
* Return 0 if the SMP should be dropped
*/
enum smi_action smi_handle_dr_smp_recv(struct ib_smp *smp, u8 node_type,
int port_num, int phys_port_cnt)
I'll add a patch which cleans up those original comments and then update those in this series.
Quoted 64 lines
Quoted 64 lines
drivers/infiniband/core/smi.c +15
-0+*/ +enum smi_action opa_smi_handle_dr_smp_recv (struct opa_smp *smp,u8 node_type, +int port_num,int phys_port_cnt) +{ +return __smi_handle_dr_smp_recv (node_type, port_num, phys_port_cnt, + &smp->hop_ptr, smp->hop_cnt, + smp->route.dr.initial_path, + smp->route.dr.return_path, +opa_get_smp_direction (smp), + smp->route.dr.dr_dlid == +OPA_LID_PERMISSIVE , + smp->route.dr.dr_slid == +OPA_LID_PERMISSIVE ); +} + static inlineenum
smi_forward_action__smi_check_forward_dr_smp(u8hop_ptr, u8hop_cnt,drivers/infiniband/core/smi.c +10
-0u8 direction,@@ -277,6 +311,16 @@ enum smi_forward_action smi_check_forward_dr_smp(struct ib_smp *smp) smp->dr_slid != IB_LID_PERMISSIVE); } +enum smi_forward_action opa_smi_check_forward_dr_smp (struct opa_smp *smp) +{ +return __smi_check_forward_dr_smp (smp->hop_ptr, smp->hop_cnt, +opa_get_smp_direction (smp), + smp->route.dr.dr_dlid == +OPA_LID_PERMISSIVE , + smp->route.dr.dr_slid == +OPA_LID_PERMISSIVE ); +} +/* * Return the forwarding port number from initial_path for outgoing SMP and
* fromreturn_pathfor returning SMP(quoted diff) +0
-0@@ -286,3 +330,13 @@ int smi_get_fwd_port(struct ib_smp *smp) return (!ib_get_smp_direction(smp) ? smp->initial_path[smp-Quoted 1 line
hop_ptr+1] :
drivers/infiniband/core/smi.c +3
-0smp->return_path[smp->hop_ptr-1]); } + +/* +* Return the forwarding port number from initial_path for outgoing SMP and
+ * fromreturn_pathfor returning SMP
+ */drivers/infiniband/core/smi.c +3
-0+int opa_smi_get_fwd_port (struct opa_smp *smp) +{ + return !opa_get_smp_direction(smp) ? smp->route.dr.initial_path[smp-Quoted 1 line
hop_ptr+1] :
drivers/infiniband/core/smi.c +2
-0+ smp->route.dr.return_path[smp->hop_ptr-1 ]; +}drivers/infiniband/core/smi.h +0
-0@@ -62,6 +62,9 @@ extern enum smi_action smi_handle_dr_smp_send(structib_smp *smp, * Return IB_SMI_HANDLE if the SMP should be handled by the local SMA/SM
Quoted 8 lines
Quoted 6 lines
drivers/infiniband/core/smi.h +1
-0* via process_mad */ +/* NOTE: This is called on opa_smp's don't check fields which are not common
+ * betweenib_smpandopa_smp
+ */This comment suggests that the function is not correct for OPA. ?
This was a mistake left over from an early version of the patches. OPA versions are in opa_smi.h. Those should be used.
Removed comment and fixed handle_outgoing_dr_smp.
Quoted 15 lines
Quoted 10 lines
static inline enum
smi_actionsmi_check_local_smp(structib_smp*smp,drivers/infiniband/core/smi.h +1
-0struct ib_device *device) {@@ -77,6 +80,9 @@ static inline enum smi_action smi_check_local_smp(structib_smp *smp, * Return IB_SMI_HANDLE if the SMP should be handled by the local SMA/SM * via process_mad */ +/* NOTE: This is called on opa_smp's don't check fields which are not Quoted 3 lines
common
+ * betweenib_smpandopa_smp
+ */Same comment
Same fix.
Ira
Quoted 5 lines
Quoted 5 lines
static inline enum
smi_actionsmi_check_local_returning_smp(structib_smp
*smp,
structib_device*device)
{
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 19/19 IB/mad: Implement Intel Omni-Path Architecture MAD processing
In reply to a message not in the archiveFrom: Ira Weiny <[email protected]>
For devices which support OPA MADs
OPA SMP packets must carry a valid pkey
process wc.pkey_index returned by agents for response.
Handle variable length OPA MADs by:
* Adjusting the 'fake' WC for locally routed SMP's to represent the proper incoming byte_len
* out_mad_size is used from the local HCA agents
1) when sending agent responses on the wire
2) when passing responses through the local_completions function
NOTE: wc.byte_len includes the GRH length and therefore is different from the
in_mad_size specified to the local HCA agents. out_mad_size should _not_ include the GRH length as it is added by the verbs layer and is not part of MAD processing.
--- drivers/infiniband/core/agent.c | 27 +++- drivers/infiniband/core/agent.h | 3 +- drivers/infiniband/core/mad.c | 251 ++++++++++++++++++++++++++++++------- drivers/infiniband/core/mad_priv.h | 1 + drivers/infiniband/core/mad_rmpp.c | 32 +++-- drivers/infiniband/core/user_mad.c | 35 +++--- include/rdma/ib_mad.h | 2 + 7 files changed, 276 insertions(+), 75 deletions(-)
drivers/infiniband/core/agent.c +22 -5
@@ -80,13 +80,17 @@ ib_get_agent_port(struct ib_device *device, int port_num)void agent_send_response (struct ib_mad *mad,struct ib_grh *grh,struct ib_wc *wc,struct ib_device *device, -int port_num,int qpn) +int port_num,int qpn,u32 resp_mad_len, +int opa) {struct ib_agent_port_private *port_priv;struct ib_mad_agent *agent;struct ib_mad_send_buf *send_buf;struct ib_ah *ah; +size_t data_len; +size_t hdr_len;struct ib_mad_send_wr_private *mad_send_wr; +u8 base_version;if (device->node_type ==RDMA_NODE_IB_SWITCH ) port_priv =ib_get_agent_port (device,0 );
@@ -106,16 +110,29 @@ void agent_send_response(struct ib_mad *mad, struct ib_grh *grh,return ; } +/* base version determines MAD size */ + base_version = mad->mad_hdr.base_version; +if (opa && base_version ==OPA_MGMT_BASE_VERSION ) { + data_len = resp_mad_len -JUMBO_MGMT_MAD_HDR ; + hdr_len =JUMBO_MGMT_MAD_HDR ; + }else { + data_len =IB_MGMT_MAD_DATA ; + hdr_len =IB_MGMT_MAD_HDR ; + } + send_buf =ib_create_send_mad (agent, wc->src_qp, wc->pkey_index,0 , -IB_MGMT_MAD_HDR ,IB_MGMT_MAD_DATA , -GFP_KERNEL , -IB_MGMT_BASE_VERSION ); + hdr_len, data_len,GFP_KERNEL , + base_version);if (IS_ERR (send_buf)) {dev_err (&device->dev,"ib_create_send_mad error\n" ); goto err1; } -memcpy (send_buf->mad, mad,sizeof *mad); +if (opa && base_version ==OPA_MGMT_BASE_VERSION ) +memcpy (send_buf->mad, mad,JUMBO_MGMT_MAD_HDR + data_len); +else +memcpy (send_buf->mad, mad,sizeof (*mad)); + send_buf->ah = ah;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {
drivers/infiniband/core/agent.h +2 -1
@@ -46,6 +46,7 @@ extern int ib_agent_port_close(struct ib_device *device, int port_num);extern void agent_send_response (struct ib_mad *mad,struct ib_grh *grh,struct ib_wc *wc,struct ib_device *device, -int port_num,int qpn); +int port_num,int qpn,u32 resp_mad_len, +int opa);#endif /* __AGENT_H_ */
drivers/infiniband/core/mad.c +207 -44
448 lines not on this page · raw message
drivers/infiniband/core/mad_priv.h +1 -0
@@ -154,6 +154,7 @@ struct ib_mad_local_private {struct ib_mad_private *mad_priv;struct ib_mad_agent_private *recv_mad_agent;struct ib_mad_send_wr_private *mad_send_wr; +size_t return_wc_byte_len; };struct ib_mad_mgmt_method_table {
drivers/infiniband/core/mad_rmpp.c +22 -10
@@ -1,6 +1,7 @@/* * Copyright (c) 2005 Intel Inc. All rights reserved. * Copyright (c) 2005-2006 Voltaire, Inc. All rights reserved. +* Copyright (c) 2014 Intel Corporation. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU
@@ -67,6 +68,7 @@ struct mad_rmpp_recv {u8 mgmt_class;u8 class_version;u8 method; +u8 base_version; };static inline void deref_rmpp_recv (struct mad_rmpp_recv *rmpp_recv)
@@ -318,6 +320,7 @@ create_rmpp_recv(struct ib_mad_agent_private *agent, rmpp_recv->mgmt_class = mad_hdr->mgmt_class; rmpp_recv->class_version = mad_hdr->class_version; rmpp_recv->method = mad_hdr->method; + rmpp_recv->base_version = mad_hdr->base_version;return rmpp_recv; error:kfree (rmpp_recv);
@@ -431,16 +434,25 @@ static void update_seg_num(struct mad_rmpp_recv *rmpp_recv,static inline int get_mad_len (struct mad_rmpp_recv *rmpp_recv) { -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base;int hdr_size, data_size, pad; +int opa = rmpp_recv->agent->qp_info->qp->device->cached_dev_attrs.device_cap_flags2 & +IB_DEVICE_OPA_MAD_SUPPORT ; - rmpp_mad = (struct ib_rmpp_mad *)rmpp_recv->cur_seg_buf->mad; + rmpp_base = (struct ib_rmpp_base *)rmpp_recv->cur_seg_buf->mad; - hdr_size =ib_get_mad_data_offset (rmpp_mad->base.mad_hdr.mgmt_class); - data_size =sizeof (struct ib_rmpp_mad ) - hdr_size; - pad =IB_MGMT_RMPP_DATA -be32_to_cpu (rmpp_mad->base.rmpp_hdr.paylen_newwin); -if (pad >IB_MGMT_RMPP_DATA || pad <0 ) - pad =0 ; + hdr_size =ib_get_mad_data_offset (rmpp_base->mad_hdr.mgmt_class); +if (opa && rmpp_recv->base_version ==OPA_MGMT_BASE_VERSION ) { + data_size =sizeof (struct jumbo_rmpp_mad ) - hdr_size; + pad =JUMBO_MGMT_RMPP_DATA -be32_to_cpu (rmpp_base->rmpp_hdr.paylen_newwin); +if (pad >JUMBO_MGMT_RMPP_DATA || pad <0 ) + pad =0 ; + }else { + data_size =sizeof (struct ib_rmpp_mad ) - hdr_size; + pad =IB_MGMT_RMPP_DATA -be32_to_cpu (rmpp_base->rmpp_hdr.paylen_newwin); +if (pad >IB_MGMT_RMPP_DATA || pad <0 ) + pad =0 ; + }return hdr_size + rmpp_recv->seg_num * data_size - pad; }
@@ -933,11 +945,11 @@ int ib_process_rmpp_send_wc(struct ib_mad_send_wr_private *mad_send_wr,int ib_retry_rmpp (struct ib_mad_send_wr_private *mad_send_wr) { -struct ib_rmpp_base *rmpp_base; +struct ib_rmpp_mad *rmpp_mad;int ret; - rmpp_base = mad_send_wr->send_buf.mad; -if (!(ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) & + rmpp_mad = mad_send_wr->send_buf.mad; +if (!(ib_get_rmpp_flags (&rmpp_mad->base.rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ))return IB_RMPP_RESULT_UNHANDLED ;/* RMPP not active */
drivers/infiniband/core/user_mad.c +20 -15
@@ -263,20 +263,23 @@ static ssize_t copy_recv_mad(struct ib_umad_file *file, char __user *buf, {struct ib_mad_recv_buf *recv_buf;int left, seg_payload, offset, max_seg_payload; +size_t seg_size; -/* We need enough room to copy the first (or only) MAD segment. */ recv_buf = &packet->recv_wc->recv_buf; -if ((packet->length <=sizeof (*recv_buf->mad) && + seg_size = packet->recv_wc->mad_seg_size; + +/* We need enough room to copy the first (or only) MAD segment. */ +if ((packet->length <= seg_size && count <hdr_size (file) + packet->length) || - (packet->length >sizeof (*recv_buf->mad) && - count <hdr_size (file) +sizeof (*recv_buf->mad))) + (packet->length > seg_size && + count <hdr_size (file) + seg_size))return -EINVAL ;if (copy_to_user (buf, &packet->mad,hdr_size (file)))return -EFAULT ; buf +=hdr_size (file); - seg_payload =min_t (int, packet->length,sizeof (*recv_buf->mad)); + seg_payload =min_t (int, packet->length, seg_size);if (copy_to_user (buf, recv_buf->mad, seg_payload))return -EFAULT ;
@@ -293,7 +296,7 @@ static ssize_t copy_recv_mad(struct ib_umad_file *file, char __user *buf,return -ENOSPC ; } offset =ib_get_mad_data_offset (recv_buf->mad->mad_hdr.mgmt_class); - max_seg_payload =sizeof (struct ib_mad ) - offset; + max_seg_payload = seg_size - offset;for (left = packet->length - seg_payload, buf += seg_payload; left; left -= seg_payload, buf += seg_payload) {
@@ -448,9 +451,10 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf,struct ib_mad_agent *agent;struct ib_ah_attr ah_attr;struct ib_ah *ah; -struct ib_rmpp_base *rmpp_base; +struct ib_rmpp_mad *rmpp_mad;__be64 *tid;int ret, data_len, hdr_len, copy_offset, rmpp_active; +u8 base_version;if (count <hdr_size (file) +IB_MGMT_RMPP_HDR )return -EINVAL ;
@@ -504,25 +508,26 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf, goto err_up; } - rmpp_base = (struct ib_rmpp_base *) packet->mad.data; - hdr_len =ib_get_mad_data_offset (rmpp_base->mad_hdr.mgmt_class); + rmpp_mad = (struct ib_rmpp_mad *) packet->mad.data; + hdr_len =ib_get_mad_data_offset (rmpp_mad->base.mad_hdr.mgmt_class); -if (ib_is_mad_class_rmpp (rmpp_base->mad_hdr.mgmt_class) +if (ib_is_mad_class_rmpp (rmpp_mad->base.mad_hdr.mgmt_class) &&ib_mad_kernel_rmpp_agent (agent)) { copy_offset =IB_MGMT_RMPP_HDR ; - rmpp_active =ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) & + rmpp_active =ib_get_rmpp_flags (&rmpp_mad->base.rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ; }else { copy_offset =IB_MGMT_MAD_HDR ; rmpp_active =0 ; } + base_version = ((struct ib_mad_hdr *)&packet->mad.data)->base_version; data_len = count -hdr_size (file) - hdr_len; packet->msg =ib_create_send_mad (agent,be32_to_cpu (packet->mad.hdr.qpn), packet->mad.hdr.pkey_index, rmpp_active, hdr_len, data_len,GFP_KERNEL , -IB_MGMT_BASE_VERSION ); + base_version);if (IS_ERR (packet->msg)) { ret =PTR_ERR (packet->msg); goto err_ah;
@@ -558,12 +563,12 @@ static ssize_t ib_umad_write(struct file *filp, const char __user *buf, tid = &((struct ib_mad_hdr *) packet->msg->mad)->tid; *tid =cpu_to_be64 (((u64 ) agent->hi_tid) <<32 | (be64_to_cpup (tid) &0xffffffff )); - rmpp_base->mad_hdr.tid = *tid; + rmpp_mad->base.mad_hdr.tid = *tid; }if (!ib_mad_kernel_rmpp_agent (agent) - &&ib_is_mad_class_rmpp (rmpp_base->mad_hdr.mgmt_class) - && (ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE )) { + &&ib_is_mad_class_rmpp (rmpp_mad->base.mad_hdr.mgmt_class) + && (ib_get_rmpp_flags (&rmpp_mad->base.rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE )) {spin_lock_irq (&file->send_lock);list_add_tail (&packet->list, &file->send_list);spin_unlock_irq (&file->send_lock);
include/rdma/ib_mad.h +2 -0
@@ -436,6 +436,7 @@ struct ib_mad_recv_buf {* @recv_buf: Specifies the location of the received data buffer(s). * @rmpp_list: Specifies a list of RMPP reassembled received MAD buffers. * @mad_len: The length of the received MAD, without duplicated headers. +* @mad_seg_size: The size of individual MAD segments * * For received response, the wr_id contains a pointer to the ib_mad_send_buf * for the corresponding send request.
@@ -445,6 +446,7 @@ struct ib_mad_recv_wc {struct ib_mad_recv_buf recv_buf;struct list_head rmpp_list;int mad_len; +size_t mad_seg_size; };/**
Signature · 6 lines
1.8.2 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1423092585-26692-20-git-send-email-ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
RE: [PATCH v4 19/19] IB/mad: Implement Intel Omni-Path Architecture MAD processing
drivers/infiniband/core/agent.c ib_get_agent_port(struct ib_device *device, int · line 85
Quoted 13 lines
drivers/infiniband/core/agent.c +2
-1@@ -80,13 +80,17 @@ ib_get_agent_port(struct ib_device *device, int port_num)void agent_send_response (struct ib_mad *mad,struct ib_grh *grh,struct ib_wc *wc,struct ib_device *device, -int port_num,int qpn) +int port_num,int qpn,u32 resp_mad_len, +int opa)
Can't OPA support be determined by looking at the device structure?
Quoted 16 lines
drivers/infiniband/core/agent.c +3
-0{struct ib_agent_port_private *port_priv;struct ib_mad_agent *agent;struct ib_mad_send_buf *send_buf;struct ib_ah *ah; +size_t data_len; +size_t hdr_len;struct ib_mad_send_wr_private *mad_send_wr; +u8 base_version;if (device->node_type ==RDMA_NODE_IB_SWITCH )drivers/infiniband/core/agent.c +0
-0port_priv =ib_get_agent_port (device,0 );@@ -106,16 +110,29 @@ void agent_send_response(struct ib_mad *mad, structib_grh *grh, return; }
drivers/infiniband/core/agent.c void agent_send_response(struct ib_mad *mad, struct · line 122
Quoted 10 lines
drivers/infiniband/core/agent.c +9
-0+/* base version determines MAD size */ + base_version = mad->mad_hdr.base_version; +if (opa && base_version ==OPA_MGMT_BASE_VERSION ) { + data_len = resp_mad_len -JUMBO_MGMT_MAD_HDR ; + hdr_len =JUMBO_MGMT_MAD_HDR ; + }else { + data_len =IB_MGMT_MAD_DATA ; + hdr_len =IB_MGMT_MAD_HDR ; + }
I _think_ this can be simplified to:
hdr_len = IB_MGMT_MAD_HDR; data_len = resp_mad_len - hdr_len;
IB should set resp_mad_len = 256 in all cases.
Quoted 7 lines
drivers/infiniband/core/agent.c +3
-3+ send_buf =ib_create_send_mad (agent, wc->src_qp, wc->pkey_index,0 , -IB_MGMT_MAD_HDR ,IB_MGMT_MAD_DATA , -GFP_KERNEL , -IB_MGMT_BASE_VERSION ); + hdr_len, data_len,GFP_KERNEL , + base_version);
drivers/infiniband/core/agent.c void agent_send_response(struct ib_mad *mad, struct ib_grh *grh, · line 134
Quoted 10 lines
drivers/infiniband/core/agent.c +4
-1if (IS_ERR (send_buf)) {dev_err (&device->dev,"ib_create_send_mad error\n" ); goto err1; } -memcpy (send_buf->mad, mad,sizeof *mad); +if (opa && base_version ==OPA_MGMT_BASE_VERSION ) +memcpy (send_buf->mad, mad,JUMBO_MGMT_MAD_HDR + data_len); +else +memcpy (send_buf->mad, mad,sizeof (*mad));
And this may be able to be simplified to:
memcpy(send_buf->mad, mad, resp_mad_len);
Quoted 64 lines
drivers/infiniband/core/agent.c +1
-0+ send_buf->ah = ah;drivers/infiniband/core/agent.c +0
-0if (device->node_type ==RDMA_NODE_IB_SWITCH ) {drivers/infiniband/core/agent.h +2
-1@@ -46,6 +46,7 @@ extern int ib_agent_port_close(struct ib_device *device, int port_num);extern void agent_send_response (struct ib_mad *mad,struct ib_grh *grh,struct ib_wc *wc,struct ib_device *device, -int port_num,int qpn); +int port_num,int qpn,u32 resp_mad_len, +int opa);#endif /*
__AGENT_H_*/drivers/infiniband/core/mad.c +3
-0@@ -3,6 +3,7 @@* Copyright (c) 2005 Intel Corporation. All rights reserved. * Copyright (c) 2005 Mellanox Technologies Ltd. All rights reserved. * Copyright (c) 2009 HNR Consulting. All rights reserved. +* Copyright (c) 2014 Intel Corporation. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU @@ -44,6 +45,7 @@#include "mad_priv.h" #include "mad_rmpp.h" #include "smi.h" +#include "opa_smi.h" #include "agent.h" MODULE_LICENSE ("Dual BSD/GPL" );@@ -733,6 +735,7 @@ static int handle_outgoing_dr_smp(structib_mad_agent_private *mad_agent_priv, { int ret = 0;struct ib_smp *smp = mad_send_wr->send_buf.mad; +struct opa_smp *opa_smp = (struct opa_smp *)smp;unsigned long flags;struct ib_mad_local_private *local;drivers/infiniband/core/mad.c +2
-0struct ib_mad_private *mad_priv;@@ -744,6 +747,9 @@ static int handle_outgoing_dr_smp(structib_mad_agent_private *mad_agent_priv, struct ib_send_wr *send_wr = &mad_send_wr->send_wr;size_t in_mad_size = mad_agent_priv->agent.device- >cached_dev_attrs.max_mad_size;size_t out_mad_size; +u16 drslid; + int opa = mad_agent_priv->qp_info->qp->device-Quoted 1 line
cached_dev_attrs.device_cap_flags2 &
drivers/infiniband/core/mad.c +1
-0+IB_DEVICE_OPA_MAD_SUPPORT ;if (device->node_type ==RDMA_NODE_IB_SWITCH && smp->mgmt_class ==IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE )@@ -757,13 +763,36 @@ static int handle_outgoing_dr_smp(structib_mad_agent_private *mad_agent_priv, * If we are at the start of the LID routed part, don't update the
drivers/infiniband/core/mad.c static int handle_outgoing_dr_smp(struct · line 767
Quoted 10 lines
drivers/infiniband/core/mad.c +1
-7* hop_ptr or hop_cnt. See section 14.2.2, Vol 1 IB spec. */ -if ((ib_get_smp_direction (smp) ? smp->dr_dlid : smp->dr_slid) == -IB_LID_PERMISSIVE && -smi_handle_dr_smp_send (smp, device->node_type, port_num) == -IB_SMI_DISCARD ) { - ret = -EINVAL ; -dev_err (&device->dev,"Invalid directed route\n" ); - goto out; +if (opa && smp->class_version ==OPA_SMP_CLASS_VERSION ) {
There are several places where this sort of check is made. IMO, this check should only require looking at the MAD, not the MAD + the device attributes that the MAD will be transferred on. I would actually prefer to see this as:
if (smp->class_version == OPA_SMP_CLASS_VERSION)
That check is sufficient. There is no conflict with IB MADs, and it's needlessly complicates the code to assume that the IBTA is going to someday define another 128 class versions in such a way that those versions will not require any other changes to the code.
Quoted 81 lines
drivers/infiniband/core/mad.c +3
-0+u32 opa_drslid; +if ((opa_get_smp_direction (opa_smp) + ? opa_smp->route.dr.dr_dlid : opa_smp->route.dr.dr_slid)==
drivers/infiniband/core/mad.c +18
-0+ OPA_LID_PERMISSIVE && + opa_smi_handle_dr_smp_send(opa_smp, device->node_type, + port_num) == IB_SMI_DISCARD) { + ret = -EINVAL ; +dev_err (&device->dev,"OPA Invalid directed route\n" ); + goto out; + } + opa_drslid =be32_to_cpu (opa_smp->route.dr.dr_slid); +if (opa_drslid !=OPA_LID_PERMISSIVE && + opa_drslid &0xffff0000 ) { + ret = -EINVAL ; +dev_err (&device->dev,"OPA Invalid dr_slid 0x%x\n" , + opa_drslid); + goto out; + } + drslid = (u16)(opa_drslid &0x0000ffff ); + }else { + if ((ib_get_smp_direction(smp) ? smp->dr_dlid : smp->dr_slid)==
drivers/infiniband/core/mad.c +2
-0+IB_LID_PERMISSIVE && +smi_handle_dr_smp_send (smp, device->node_type, port_num)==
drivers/infiniband/core/mad.c +13
-1+ IB_SMI_DISCARD) { + ret = -EINVAL ; +dev_err (&device->dev,"Invalid directed route\n" ); + goto out; + } + drslid =be16_to_cpu (smp->dr_slid); }/* Check to post send on QP or process locally */ @@ -789,10 +818,16 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv, }build_smp_wc (mad_agent_priv->agent.qp, - send_wr->wr_id,be16_to_cpu (smp->dr_slid), + send_wr->wr_id, drslid, send_wr->wr.ud.pkey_index, send_wr->wr.ud.port_num, &mad_wc); +if (opa && smp->base_version ==OPA_MGMT_BASE_VERSION ) { + mad_wc.byte_len = mad_send_wr->send_buf.hdr_len + + mad_send_wr->send_buf.data_len + +sizeof (struct ib_grh ); + } +/* No GRH for DR SMP */ ret = device->process_mad(device, 0, port_num, &mad_wc, NULL,drivers/infiniband/core/mad.c +4
-1(struct ib_mad_hdr *)smp, in_mad_size,@@ -821,7 +856,10 @@ static int handle_outgoing_dr_smp(structib_mad_agent_private *mad_agent_priv, port_priv =ib_get_mad_port (mad_agent_priv->agent.device, mad_agent_priv->agent.port_num);if (port_priv) { -memcpy (&mad_priv->mad.mad, smp,sizeof (struct ib_mad )); +if (opa && smp->base_version ==OPA_MGMT_BASE_VERSION ) +memcpy (&mad_priv->mad.mad, smp,sizeof (struct jumbo_mad )); +else +memcpy (&mad_priv->mad.mad, smp,sizeof (struct ib_mad ));drivers/infiniband/core/mad.c +2
-0recv_mad_agent =find_mad_agent (port_priv, &mad_priv->mad.mad); }@@ -844,6 +882,8 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv, } local->mad_send_wr = mad_send_wr; + local->mad_send_wr->send_wr.wr.ud.pkey_index = mad_wc.pkey_index; + local->return_wc_byte_len = out_mad_size;/* Reference MAD agent until send side of local completion handled */ drivers/infiniband/core/mad.c +0
-0atomic_inc (&mad_agent_priv->refcount);/* Queue local completion to local list */
drivers/infiniband/core/mad.c out: · line 1782
Quoted 7 lines
drivers/infiniband/core/mad.c +3
-1@@ -1737,14 +1777,18 @@ out:return mad_agent; } -static int validate_mad (struct ib_mad_hdr *mad_hdr,u32 qp_num) +static int validate_mad (struct ib_mad_hdr *mad_hdr, +struct ib_mad_qp_info *qp_info, +int opa)
I'm not a fan of having an 'opa' integer passed around to a bunch of functions. This can be determined through the qp_info parameter.
drivers/infiniband/core/mad.c out: · line 1785
Quoted 3 lines
drivers/infiniband/core/mad.c +1
-0{int valid =0 ; +u32 qp_num = qp_info->qp->qp_num;
Am I missing where this is used?
Quoted 53 lines
drivers/infiniband/core/mad.c +8
-7/* Make sure MAD base version is understood */ -if (mad_hdr->base_version !=IB_MGMT_BASE_VERSION ) { -pr_err ("MAD received with unsupported base version %d\n" , - mad_hdr->base_version); +if (mad_hdr->base_version !=IB_MGMT_BASE_VERSION && + (!opa || mad_hdr->base_version !=OPA_MGMT_BASE_VERSION )) { +pr_err ("MAD received with unsupported base version %d %s\n" , + mad_hdr->base_version, opa ?"(opa)" :"" ); goto out; }@@ -1844,18 +1888,18 @@ ib_find_send_mad(struct ib_mad_agent_private *mad_agent_priv, struct ib_mad_recv_wc *wc) {struct ib_mad_send_wr_private *wr; -struct ib_mad *mad; +struct ib_mad_hdr *mad_hdr; - mad = (struct ib_mad *)wc->recv_buf.mad; + mad_hdr = (struct ib_mad_hdr *)wc->recv_buf.mad;list_for_each_entry (wr, &mad_agent_priv->wait_list, agent_list) { -if ((wr->tid == mad->mad_hdr.tid) && +if ((wr->tid == mad_hdr->tid) &&rcv_has_same_class (wr, wc) &&/* * Don't check GID for direct routed MADs. * These might have permissive LIDs. */ - (is_direct (wc->recv_buf.mad->mad_hdr.mgmt_class) || + (is_direct (mad_hdr->mgmt_class) ||rcv_has_same_gid (mad_agent_priv, wr, wc)))return (wr->status ==IB_WC_SUCCESS ) ? wr :NULL ;drivers/infiniband/core/mad.c +2
-2}@@ -1866,14 +1910,14 @@ ib_find_send_mad(struct ib_mad_agent_private*mad_agent_priv, */ list_for_each_entry (wr, &mad_agent_priv->send_list, agent_list) {if (is_rmpp_data_mad (mad_agent_priv, wr->send_buf.mad) && - wr->tid == mad->mad_hdr.tid && + wr->tid == mad_hdr->tid && wr->timeout &&rcv_has_same_class (wr, wc) &&/* * Don't check GID for direct routed MADs. * These might have permissive LIDs. */ - (is_direct (wc->recv_buf.mad->mad_hdr.mgmt_class) || + (is_direct (mad_hdr->mgmt_class) ||rcv_has_same_gid (mad_agent_priv, wr, wc)))/* Verify request has not been canceled */
Quoted 1 line
return (
wr->status==IB_WC_SUCCESS) ? wr : NULL;
The updates to the two functions above can be pulled out into a separate commit.
Quoted 44 lines
drivers/infiniband/core/mad.c +1
-1@@ -1889,7 +1933,7 @@ void ib_mark_mad_done(struct ib_mad_send_wr_private *mad_send_wr) &mad_send_wr->mad_agent_priv->done_list); } -static void ib_mad_complete_recv (struct ib_mad_agent_private *mad_agent_priv, +void ib_mad_complete_recv (struct ib_mad_agent_private *mad_agent_priv,struct ib_mad_recv_wc *mad_recv_wc){
drivers/infiniband/core/mad.c +3
-1struct ib_mad_send_wr_private *mad_send_wr;@@ -1992,7 +2036,9 @@ enum smi_action handle_ib_smi(struct ib_mad_port_private *port_priv, &response->grh, wc, port_priv->device,smi_get_fwd_port (&recv->mad.smp), - qp_info->qp->qp_num); + qp_info->qp->qp_num, +sizeof (struct ib_mad ), + 0);return IB_SMI_DISCARD ;drivers/infiniband/core/mad.c +3
-1}@@ -2005,7 +2051,9 @@ static size_t mad_recv_buf_size(struct ib_device *dev) }static bool generate_unmatched_resp (struct ib_mad_private *recv, -struct ib_mad_private *response) +struct ib_mad_private *response, +size_t *resp_len, +int opa) { if (recv->mad.mad.mad_hdr.method == IB_MGMT_METHOD_GET ||drivers/infiniband/core/mad.c +2
-0recv->mad.mad.mad_hdr.method == IB_MGMT_METHOD_SET) {@@ -2019,29 +2067,103 @@ static bool generate_unmatched_resp(struct ib_mad_private *recv, if (recv->mad.mad.mad_hdr.mgmt_class ==IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE ) response->mad.mad.mad_hdr.status |= IB_SMP_DIRECTION; +if (opa && recv->mad.mad.mad_hdr.base_version ==OPA_MGMT_BASE_VERSION ) { + if (recv->mad.mad.mad_hdr.mgmt_class ==
drivers/infiniband/core/mad.c static bool generate_unmatched_resp(struct · line 2084
Quoted 10 lines
drivers/infiniband/core/mad.c +9
-0+IB_MGMT_CLASS_SUBN_LID_ROUTED || + recv->mad.mad.mad_hdr.mgmt_class == + IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE) + *resp_len =opa_get_smp_header_size ( + (struct opa_smp *)&recv- >mad.smp); + else + *resp_len =sizeof (struct ib_mad_hdr ); + } +
A local variable mad_hdr = &recv->mad.mad.mad_hdr may help with the readability of this function.
Quoted 115 lines
drivers/infiniband/core/mad.c +62
-0return true; } else {return false; } } + +static enum smi_action +handle_opa_smi (struct ib_mad_port_private *port_priv, +struct ib_mad_qp_info *qp_info, +struct ib_wc *wc, +int port_num, +struct ib_mad_private *recv, +struct ib_mad_private *response) +{ +enum smi_forward_action retsmi; + +if (opa_smi_handle_dr_smp_recv (&recv->mad.opa_smp, + port_priv->device->node_type, + port_num, + port_priv->device->phys_port_cnt) == +IB_SMI_DISCARD ) +return IB_SMI_DISCARD ; + + retsmi =opa_smi_check_forward_dr_smp (&recv->mad.opa_smp); +if (retsmi ==IB_SMI_LOCAL ) +return IB_SMI_HANDLE ; + +if (retsmi ==IB_SMI_SEND ) {/* don't forward */ +if (opa_smi_handle_dr_smp_send (&recv->mad.opa_smp, + port_priv->device->node_type, + port_num) ==IB_SMI_DISCARD ) +return IB_SMI_DISCARD ; + +if (opa_smi_check_local_smp (&recv->mad.opa_smp, port_priv- >device) == IB_SMI_DISCARD) +return IB_SMI_DISCARD ; + + }else if (port_priv->device->node_type ==RDMA_NODE_IB_SWITCH ) { +/* forward case for switches */ +memcpy (response, recv,sizeof (*response)); + response->header.recv_wc.wc = &response->header.wc; + response->header.recv_wc.recv_buf.jumbo_mad = &response- >mad.jumbo_mad; + response->header.recv_wc.recv_buf.grh = &response->grh; + +agent_send_response ((struct ib_mad *)&response->mad.mad, + &response->grh, wc, + port_priv->device, +opa_smi_get_fwd_port (&recv->mad.opa_smp), + qp_info->qp->qp_num, + recv->header.wc.byte_len, +1 ); + +return IB_SMI_DISCARD ; + } + +return IB_SMI_HANDLE ; +} + +static enum smi_action +handle_smi(struct ib_mad_port_private *port_priv, + struct ib_mad_qp_info *qp_info, + struct ib_wc *wc, + int port_num, + struct ib_mad_private *recv, + struct ib_mad_private *response, + int opa) +{ + if (opa && recv->mad.mad.mad_hdr.base_version ==
OPA_MGMT_BASE_VERSION&&drivers/infiniband/core/mad.c +2
-0+ recv->mad.mad.mad_hdr.class_version == OPA_SMI_CLASS_VERSION) + return handle_opa_smi(port_priv, qp_info, wc, port_num, recv,response);
drivers/infiniband/core/mad.c +2
-0+ + return handle_ib_smi(port_priv, qp_info, wc, port_num, recv,response);
drivers/infiniband/core/mad.c +2
-0+} + static void ib_mad_recv_done_handler(struct ib_mad_port_private
*port_priv,drivers/infiniband/core/mad.c +7
-11- struct ib_wc *wc) +struct ib_wc *wc, + struct ib_mad_private_header *mad_priv_hdr, + struct ib_mad_qp_info *qp_info) { -struct ib_mad_qp_info *qp_info; -struct ib_mad_private_header *mad_priv_hdr; struct ib_mad_private *recv, *response = NULL; -struct ib_mad_list_head *mad_list;struct ib_mad_agent_private *mad_agent;int port_num;int ret =IB_MAD_RESULT_SUCCESS ;size_t resp_mad_size; +int opa = qp_info->qp->device->cached_dev_attrs.device_cap_flags2 & +IB_DEVICE_OPA_MAD_SUPPORT ; - mad_list = (struct ib_mad_list_head *)(unsigned long )wc->wr_id; - qp_info = mad_list->mad_queue->qp_info; -dequeue_mad (mad_list); - - mad_priv_hdr = container_of(mad_list, struct ib_mad_private_header, - mad_list); recv = container_of(mad_priv_hdr, struct ib_mad_private, header); ib_dma_unmap_single(port_priv->device, recv->header.mapping,@@ -2051,7 +2173,13 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv,/* Setup MAD receive work completion from "normal" work completion */ recv->header.wc = *wc; recv->header.recv_wc.wc = &recv->header.wc; - recv->header.recv_wc.mad_len =sizeof (struct ib_mad ); +if (opa && recv->mad.mad.mad_hdr.base_version ==OPA_MGMT_BASE_VERSION ) { + recv->header.recv_wc.mad_len = wc->byte_len - sizeof(struct
Quoted 1 line
ib_grh);
Can this logic to set mad_len be used for both OPA and IB?
Quoted 128 lines
drivers/infiniband/core/mad.c +6
-1+ recv->header.recv_wc.mad_seg_size =sizeof (struct jumbo_mad ); + }else { + recv->header.recv_wc.mad_len =sizeof (struct ib_mad ); + recv->header.recv_wc.mad_seg_size =sizeof (struct ib_mad ); + } recv->header.recv_wc.recv_buf.mad = &recv->mad.mad; recv->header.recv_wc.recv_buf.grh = &recv->grh;@@ -2059,7 +2187,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv,snoop_recv (qp_info, &recv->header.recv_wc,IB_MAD_SNOOP_RECVS );/* Validate MAD */ -if (!validate_mad (&recv->mad.mad.mad_hdr, qp_info->qp->qp_num)) +if (!validate_mad (&recv->mad.mad.mad_hdr, qp_info, opa)) goto out;drivers/infiniband/core/mad.c +1
-2response =alloc_mad_priv (port_priv->device, &resp_mad_size);@@ -2076,8 +2204,7 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, if (recv->mad.mad.mad_hdr.mgmt_class == IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE) { -if (handle_ib_smi (port_priv, qp_info, wc, port_num, recv, - response) +if (handle_smi (port_priv, qp_info, wc, port_num, recv, response, opa) ==IB_SMI_DISCARD )drivers/infiniband/core/mad.c +3
-1goto out; }@@ -2099,7 +2226,9 @@ static void ib_mad_recv_done_handler(struct ib_mad_port_private *port_priv, &recv->grh, wc, port_priv->device, port_num, - qp_info->qp->qp_num); + qp_info->qp->qp_num, + resp_mad_size, + opa); goto out; }drivers/infiniband/core/mad.c +5
-2}@@ -2114,9 +2243,12 @@ static void ib_mad_recv_done_handler(structib_mad_port_private *port_priv, */ recv =NULL ; }else if ((ret &IB_MAD_RESULT_SUCCESS ) && -generate_unmatched_resp (recv, response)) { +generate_unmatched_resp (recv, response, &resp_mad_size, opa)) {agent_send_response (&response->mad.mad, &recv->grh, wc, - port_priv->device, port_num, qp_info->qp- >qp_num); + port_priv->device, port_num, + qp_info->qp->qp_num, + resp_mad_size, + opa);drivers/infiniband/core/mad.c +17
-0} out:@@ -2381,6 +2513,23 @@ static void mad_error_handler(struct ib_mad_port_private *port_priv, } } +static void ib_mad_recv_mad (struct ib_mad_port_private *port_priv, +struct ib_wc *wc) +{ +struct ib_mad_qp_info *qp_info; +struct ib_mad_list_head *mad_list; +struct ib_mad_private_header *mad_priv_hdr; + + mad_list = (struct ib_mad_list_head *)(unsigned long )wc->wr_id; + qp_info = mad_list->mad_queue->qp_info; +dequeue_mad (mad_list); + + mad_priv_hdr = container_of(mad_list, struct ib_mad_private_header, + mad_list); + +ib_mad_recv_done_handler (port_priv, wc, mad_priv_hdr, qp_info); +} +/* * IB MAD completion callback */
drivers/infiniband/core/mad.c +1
-1@@ -2399,7 +2548,7 @@ static void ib_mad_completion_handler(struct work_struct *work) ib_mad_send_done_handler(port_priv, &wc);break ;case IB_WC_RECV : -ib_mad_recv_done_handler (port_priv, &wc); +ib_mad_recv_mad (port_priv, &wc);break ;default :drivers/infiniband/core/mad.c +2
-0BUG_ON (1 );@@ -2518,10 +2667,14 @@ static void local_completions(struct work_struct *work)int free_mad;struct ib_wc wc;struct ib_mad_send_wc mad_send_wc; +int opa; mad_agent_priv = container_of(work, struct ib_mad_agent_private, local_work); + opa = mad_agent_priv->qp_info->qp->device-Quoted 1 line
cached_dev_attrs.device_cap_flags2 &
drivers/infiniband/core/mad.c +3
-0+IB_DEVICE_OPA_MAD_SUPPORT ; +spin_lock_irqsave (&mad_agent_priv->lock, flags);while (!list_empty (&mad_agent_priv->local_list)) { local = list_entry(mad_agent_priv->local_list.next,@@ -2531,6 +2684,7 @@ static void local_completions(struct work_struct *work)spin_unlock_irqrestore (&mad_agent_priv->lock, flags); free_mad =0 ;if (local->mad_priv) { +u8 base_version; recv_mad_agent = local->recv_mad_agent;if (!recv_mad_agent) {drivers/infiniband/core/mad.c +0
-1dev_err(&mad_agent_priv->agent.device->dev,@@ -2546,11 +2700,20 @@ static void local_completions(struct work_struct *work) build_smp_wc(recv_mad_agent->agent.qp, (unsigned long) local->mad_send_wr, be16_to_cpu(IB_LID_PERMISSIVE), -0 , recv_mad_agent->agent.port_num, &wc);
drivers/infiniband/core/mad.c static void local_completions(struct work_struct · line 2711
Quoted 10 lines
drivers/infiniband/core/mad.c +5
-2+ local->mad_send_wr->send_wr.wr.ud.pkey_index, + recv_mad_agent->agent.port_num, &wc); local->mad_priv->header.recv_wc.wc = &wc; - local->mad_priv->header.recv_wc.mad_len = -sizeof (struct ib_mad ); + + base_version = local->mad_priv- >mad.mad.mad_hdr.base_version; +if (opa && base_version ==OPA_MGMT_BASE_VERSION ) {
Okay, how about having something like this?
int is_opa_mad(struct ib_mad_private *mad_priv)
that returns true if the MAD is a new OPA MAD
drivers/infiniband/core/mad.c static void local_completions(struct work_struct · line 2713
Quoted 2 lines
drivers/infiniband/core/mad.c +1
-0+ local->mad_priv->header.recv_wc.mad_len = local- >return_wc_byte_len;
The mad_len calculation seems like it should be the same in all cases.
Quoted 50 lines
drivers/infiniband/core/mad.c +1
-0+ local->mad_priv->header.recv_wc.mad_seg_size =
sizeof(struct jumbo_mad);drivers/infiniband/core/mad.c +2
-0+ }else { + local->mad_priv->header.recv_wc.mad_len =
sizeof(struct ib_mad);drivers/infiniband/core/mad.c +1
-0+ local->mad_priv->header.recv_wc.mad_seg_size =
sizeof(struct ib_mad);drivers/infiniband/core/mad.c +2
-0+ } + INIT_LIST_HEAD(&local->mad_priv-Quoted 1 line
header.recv_wc.rmpp_list);
drivers/infiniband/core/mad.c +1
-1list_add (&local->mad_priv->header.recv_wc.recv_buf.list, &local->mad_priv->header.recv_wc.rmpp_list);@@ -2699,7 +2862,7 @@ static int ib_mad_post_receive_mads(structib_mad_qp_info *qp_info, struct ib_mad_queue *recv_queue = &qp_info->recv_queue;/* Initialize common scatter list fields */ - sg_list.length =sizeof *mad_priv -sizeof mad_priv->header; + sg_list.length =mad_recv_buf_size (qp_info->port_priv->device); sg_list.lkey = (*qp_info->port_priv->mr).lkey;drivers/infiniband/core/mad.c +0
-0/* Initialize common receive WR fields */ drivers/infiniband/core/mad_priv.h +1
-0@@ -154,6 +154,7 @@ struct ib_mad_local_private {struct ib_mad_private *mad_priv;struct ib_mad_agent_private *recv_mad_agent;struct ib_mad_send_wr_private *mad_send_wr; +size_t return_wc_byte_len; };struct ib_mad_mgmt_method_table {drivers/infiniband/core/mad_rmpp.c +1
-0@@ -1,6 +1,7 @@/* * Copyright (c) 2005 Intel Inc. All rights reserved. * Copyright (c) 2005-2006 Voltaire, Inc. All rights reserved. +* Copyright (c) 2014 Intel Corporation. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU
drivers/infiniband/core/mad_rmpp.c struct mad_rmpp_recv { · line 71
Quoted 4 lines
drivers/infiniband/core/mad_rmpp.c +1
-0@@ -67,6 +68,7 @@ struct mad_rmpp_recv {u8 mgmt_class;u8 class_version;u8 method; +u8 base_version;
You're not really caring about the base version, right? You really just want to know if this is an OPA MAD.
Quoted 202 lines
drivers/infiniband/core/mad_rmpp.c +3
-1};static inline void deref_rmpp_recv (struct mad_rmpp_recv *rmpp_recv)@@ -318,6 +320,7 @@ create_rmpp_recv(struct ib_mad_agent_private *agent, rmpp_recv->mgmt_class = mad_hdr->mgmt_class; rmpp_recv->class_version = mad_hdr->class_version; rmpp_recv->method = mad_hdr->method; + rmpp_recv->base_version = mad_hdr->base_version;return rmpp_recv; error:kfree (rmpp_recv);@@ -431,16 +434,25 @@ static void update_seg_num(struct mad_rmpp_recv *rmpp_recv, static inline int get_mad_len(struct mad_rmpp_recv *rmpp_recv) { -struct ib_rmpp_mad *rmpp_mad; +struct ib_rmpp_base *rmpp_base;int hdr_size, data_size, pad; + int opa = rmpp_recv->agent->qp_info->qp->device-Quoted 1 line
cached_dev_attrs.device_cap_flags2 &
drivers/infiniband/core/mad_rmpp.c +2
-2+IB_DEVICE_OPA_MAD_SUPPORT ; - rmpp_mad = (struct ib_rmpp_mad *)rmpp_recv->cur_seg_buf->mad; + rmpp_base = (struct ib_rmpp_base *)rmpp_recv->cur_seg_buf->mad; - hdr_size = ib_get_mad_data_offset(rmpp_mad-Quoted 1 line
base.mad_hdr.mgmt_class);
drivers/infiniband/core/mad_rmpp.c +0
-2- data_size =sizeof (struct ib_rmpp_mad ) - hdr_size; - pad = IB_MGMT_RMPP_DATA - be32_to_cpu(rmpp_mad-Quoted 1 line
base.rmpp_hdr.paylen_newwin);
drivers/infiniband/core/mad_rmpp.c +4
-2-if (pad >IB_MGMT_RMPP_DATA || pad <0 ) - pad =0 ; + hdr_size =ib_get_mad_data_offset (rmpp_base->mad_hdr.mgmt_class); +if (opa && rmpp_recv->base_version ==OPA_MGMT_BASE_VERSION ) { + data_size =sizeof (struct jumbo_rmpp_mad ) - hdr_size; + pad = JUMBO_MGMT_RMPP_DATA - be32_to_cpu(rmpp_base-Quoted 1 line
rmpp_hdr.paylen_newwin);
drivers/infiniband/core/mad_rmpp.c +5
-0+if (pad >JUMBO_MGMT_RMPP_DATA || pad <0 ) + pad =0 ; + }else { + data_size =sizeof (struct ib_rmpp_mad ) - hdr_size; + pad = IB_MGMT_RMPP_DATA - be32_to_cpu(rmpp_base-Quoted 1 line
rmpp_hdr.paylen_newwin);
drivers/infiniband/core/mad_rmpp.c +6
-3+if (pad >IB_MGMT_RMPP_DATA || pad <0 ) + pad =0 ; + }return hdr_size + rmpp_recv->seg_num * data_size - pad; }@@ -933,11 +945,11 @@ int ib_process_rmpp_send_wc(structib_mad_send_wr_private *mad_send_wr, int ib_retry_rmpp(struct ib_mad_send_wr_private *mad_send_wr) { - struct ib_rmpp_base *rmpp_base; + struct ib_rmpp_mad *rmpp_mad;int ret; - rmpp_base = mad_send_wr->send_buf.mad; -if (!(ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) & + rmpp_mad = mad_send_wr->send_buf.mad; +if (!(ib_get_rmpp_flags (&rmpp_mad->base.rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ))return IB_RMPP_RESULT_UNHANDLED ;/* RMPP not active */ drivers/infiniband/core/user_mad.c +9
-6@@ -263,20 +263,23 @@ static ssize_t copy_recv_mad(struct ib_umad_file *file, char __user *buf, {struct ib_mad_recv_buf *recv_buf;int left, seg_payload, offset, max_seg_payload; +size_t seg_size; -/* We need enough room to copy the first (or only) MAD segment. */ recv_buf = &packet->recv_wc->recv_buf; -if ((packet->length <=sizeof (*recv_buf->mad) && + seg_size = packet->recv_wc->mad_seg_size; + +/* We need enough room to copy the first (or only) MAD segment. */ +if ((packet->length <= seg_size && count <hdr_size (file) + packet->length) || - (packet->length >sizeof (*recv_buf->mad) && - count <hdr_size (file) +sizeof (*recv_buf->mad))) + (packet->length > seg_size && + count <hdr_size (file) + seg_size))return -EINVAL ;if (copy_to_user (buf, &packet->mad,hdr_size (file)))return -EFAULT ; buf +=hdr_size (file); - seg_payload =min_t (int, packet->length,sizeof (*recv_buf->mad)); + seg_payload =min_t (int, packet->length, seg_size);if (copy_to_user (buf, recv_buf->mad, seg_payload))return -EFAULT ;@@ -293,7 +296,7 @@ static ssize_t copy_recv_mad(struct ib_umad_file *file, char__user *buf, return -ENOSPC ; } offset =ib_get_mad_data_offset (recv_buf->mad- >mad_hdr.mgmt_class); - max_seg_payload =sizeof (struct ib_mad ) - offset; + max_seg_payload = seg_size - offset;drivers/infiniband/core/user_mad.c +2
-1for (left = packet->length - seg_payload, buf += seg_payload; left; left -= seg_payload, buf += seg_payload) {@@ -448,9 +451,10 @@ static ssize_t ib_umad_write(struct file *filp, constchar __user *buf, struct ib_mad_agent *agent;struct ib_ah_attr ah_attr;struct ib_ah *ah; -struct ib_rmpp_base *rmpp_base; +struct ib_rmpp_mad *rmpp_mad;__be64 *tid;int ret, data_len, hdr_len, copy_offset, rmpp_active; +u8 base_version;if (count <hdr_size (file) +IB_MGMT_RMPP_HDR )drivers/infiniband/core/user_mad.c +6
-5return -EINVAL ;@@ -504,25 +508,26 @@ static ssize_t ib_umad_write(struct file *filp,const char __user *buf, goto err_up; } - rmpp_base = (struct ib_rmpp_base *) packet->mad.data; - hdr_len =ib_get_mad_data_offset (rmpp_base->mad_hdr.mgmt_class); + rmpp_mad = (struct ib_rmpp_mad *) packet->mad.data; + hdr_len =ib_get_mad_data_offset (rmpp_mad->base.mad_hdr.mgmt_class); -if (ib_is_mad_class_rmpp (rmpp_base->mad_hdr.mgmt_class) +if (ib_is_mad_class_rmpp (rmpp_mad->base.mad_hdr.mgmt_class) &&ib_mad_kernel_rmpp_agent (agent)) { copy_offset =IB_MGMT_RMPP_HDR ; - rmpp_active =ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) & + rmpp_active =ib_get_rmpp_flags (&rmpp_mad->base.rmpp_hdr) &IB_MGMT_RMPP_FLAG_ACTIVE ; }else { copy_offset =IB_MGMT_MAD_HDR ; rmpp_active =0 ; } + base_version = ((struct ib_mad_hdr *)&packet->mad.data)- >base_version; data_len = count -hdr_size (file) - hdr_len; packet->msg =ib_create_send_mad (agent,be32_to_cpu (packet->mad.hdr.qpn), packet->mad.hdr.pkey_index, rmpp_active, hdr_len, data_len,GFP_KERNEL , -IB_MGMT_BASE_VERSION ); + base_version);if (IS_ERR (packet->msg)) {drivers/infiniband/core/user_mad.c +3
-3ret =PTR_ERR (packet->msg); goto err_ah;@@ -558,12 +563,12 @@ static ssize_t ib_umad_write(struct file *filp,const char __user *buf, tid = &((struct ib_mad_hdr *) packet->msg->mad)->tid; *tid =cpu_to_be64 (((u64 ) agent->hi_tid) <<32 | (be64_to_cpup (tid) &0xffffffff )); - rmpp_base->mad_hdr.tid = *tid; + rmpp_mad->base.mad_hdr.tid = *tid; }if (!ib_mad_kernel_rmpp_agent (agent) - &&ib_is_mad_class_rmpp (rmpp_base->mad_hdr.mgmt_class) - && (ib_get_rmpp_flags (&rmpp_base->rmpp_hdr) & IB_MGMT_RMPP_FLAG_ACTIVE)) { + &&ib_is_mad_class_rmpp (rmpp_mad->base.mad_hdr.mgmt_class) + && (ib_get_rmpp_flags (&rmpp_mad->base.rmpp_hdr) & IB_MGMT_RMPP_FLAG_ACTIVE)) {drivers/infiniband/core/user_mad.c +0
-0spin_lock_irq (&file->send_lock);list_add_tail (&packet->list, &file->send_list);spin_unlock_irq (&file->send_lock);include/rdma/ib_mad.h +1
-0@@ -436,6 +436,7 @@ struct ib_mad_recv_buf {* @recv_buf: Specifies the location of the received data buffer(s). * @rmpp_list: Specifies a list of RMPP reassembled received MAD buffers. * @mad_len: The length of the received MAD, without duplicated headers. +* @mad_seg_size: The size of individual MAD segments * * For received response, the wr_id contains a pointer to the
ib_mad_send_buf
* for the corresponding send request.include/rdma/ib_mad.h +1
-1@@ -445,6 +446,7 @@ struct ib_mad_recv_wc {struct ib_mad_recv_buf recv_buf;struct list_head rmpp_list;int mad_len; +size_t mad_seg_size; };/** -- 1.8.2
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBD9DE-P5GAC/[email protected]
Re: [PATCH v4 19/19] IB/mad: Implement Intel Omni-Path Architecture MAD processing
Quoted 7 lines · Sean
drivers/infiniband/core/mad.c static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv, · line 766
Quoted 1 line
drivers/infiniband/core/mad.c +1
-0+if (opa && smp->class_version ==OPA_SMP_CLASS_VERSION ) {There are several places where this sort of check is made. IMO, this check should only require looking at the MAD, not the MAD + the device attributes that the MAD will be transferred on. I would actually prefer to see this as:
Quoted 6 lines
if (smp->class_version == OPA_SMP_CLASS_VERSION)That check is sufficient. There is no conflict with IB MADs, and it's needlessly complicates the code to assume that the IBTA is going to someday define another 128 class versions in such a way that those versions will not require any other changes to the code.
Hal asked for this, and I agree. It is just lazy not to check the underlying device type for this stuff - they are different number spaces, administered by different bodies with no apparent coordination.
The IBA is pretty clear what should happen to process an unsupported class version and and adding OPA shouldn't suddenly make the IB side non-conformant, however aesthetically unpleasing the code may be.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20150407022954.GA7531-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org
RE: [PATCH v4 19/19] IB/mad: Implement Intel Omni-Path Architecture MAD processing
Quoted 8 lines
Hal asked for this, and I agree. It is just lazy not to check the underlying device type for this stuff - they are different number spaces, administered by different bodies with no apparent coordination.
The IBA is pretty clear what should happen to process an unsupported class version and and adding OPA shouldn't suddenly make the IB side non-conformant, however aesthetically unpleasing the code may be.
Are there checks to ensure that MADs not supported by RoCE aren't processed on RoCE ports? --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBE150-P5GAC/[email protected]
Re: [PATCH v4 19/19] IB/mad: Implement Intel Omni-Path Architecture MAD processing
Quoted 11 lines · Sean
Quoted 8 lines
Hal asked for this, and I agree. It is just lazy not to check the underlying device type for this stuff - they are different number spaces, administered by different bodies with no apparent coordination.
The IBA is pretty clear what should happen to process an unsupported class version and and adding OPA shouldn't suddenly make the IB side non-conformant, however aesthetically unpleasing the code may be.
Are there checks to ensure that MADs not supported by RoCE aren't processed on RoCE ports?
RoCE isn't different, it uses the same numbering space and rules as IB.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20150407171659.GA15634-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org
RE: [PATCH v4 19/19] IB/mad: Implement Intel Omni-Path Architecture MAD processing
Quoted 10 lines
Quoted 1 line
Quoted 1 line
The IBA is pretty clear what should happen to process an unsupported
class
Quoted 5 lines
Quoted 2 lines
version and and adding OPA shouldn't suddenly make the IB side non-conformant, however aesthetically unpleasing the code may be.
Are there checks to ensure that MADs not supported by RoCE aren't processed on RoCE ports?
RoCE isn't different, it uses the same numbering space and rules as IB.
I was agreeing with the point made about non-compliance. RoCE only supports a subset of IB management, and AFAIK, there are no checks to validate MADs received over a RoCE port. Those should be added. --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBE1EB-P5GAC/[email protected]
Re: [PATCH v4 19/19] IB/mad: Implement Intel Omni-Path Architecture MAD processing
Quoted 8 lines · Sean
Quoted 8 lines
Quoted 1 line
Quoted 1 line
The IBA is pretty clear what should happen to process an unsupported
class
Quoted 5 lines
Quoted 2 lines
version and and adding OPA shouldn't suddenly make the IB side non-conformant, however aesthetically unpleasing the code may be.
Are there checks to ensure that MADs not supported by RoCE aren't processed on RoCE ports?
Quoted 6 lines
Quoted 2 lines
RoCE isn't different, it uses the same numbering space and rules as IB.
I was agreeing with the point made about non-compliance. RoCE only supports a subset of IB management, and AFAIK, there are no checks to validate MADs received over a RoCE port. Those should be added.
RoCE follows the base IB spec pretty well, except that there is no QP0 (or SMPs, or SMI, etc). There is already code that avoids creating QP0 MAD stuff and the SMI for RoCE. A RoCE driver should never deliver a QP0 packet to the mad layer.
There are already various checks that tie SMP's to QP0, on tx and also in validate_mad. Eg validate_mad already refuses every SMP on RoCE since qp_num can never be 0.
So I think we are good already. Every difference not caught by the missing QP0 is handled by not registering certain agents for RoCE, which looks OK already since I think most GSI's are supported.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v4 19/19] IB/mad: Implement Intel Omni-Path Architecture MAD processing
In reply to Jason GunthorpeQuoted 7 lines
Quoted 1 line
The IBA is pretty clear what should happen to process an unsupported
class
Quoted 2 lines
version and and adding OPA shouldn't suddenly make the IB side non-conformant, however aesthetically unpleasing the code may be.
Are there checks to ensure that MADs not supported by RoCE aren't processed on RoCE ports?
For the receive side, we need to enhance the validate_mad(). A similar check could be added to the send side, if we don't trust the senders.
Mgmt_class checks are everywhere in the code. Pairing each with device checks for IB, RoCE, or OPA would be unwieldy.
IMO, a MAD should be self-identifying. E.g. update the mad private structure to indicate what sort of MAD it is -- is_smp(), is_gmp(), is_rmpp(), is_opa(), is_ib(), much like the device checks are being updated. --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A82373A8FBE17D-P5GAC/[email protected]
Re: [PATCH v4 19/19] IB/mad: Implement Intel Omni-Path Architecture MAD processing
Quoted 4 lines · Sean
IMO, a MAD should be self-identifying. E.g. update the mad private structure to indicate what sort of MAD it is --
is_smp(),is_gmp(),is_rmpp(),is_opa(),is_ib(), much like the device checks are being updated.
decording into the mad private structure during initial validation seems like a good idea.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20150407171920.GB15634-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org
Re: [PATCH v4 19/19] IB/mad: Implement Intel Omni-Path Architecture MAD processing
Quoted 3 lines · Jason Gunthorpe
Quoted 1 line · Sean
IMO, a MAD should be self-identifying.
A "MAD" is self-identifying (contains versions and classes).
IBoE CM MADs are a sub-set of the IB MADs. As are the PMA MADs which those devices support. So the MAD stack need not perform any checks on those MADs.
Additionally, the MAD stack simply checks if QP0 is supported and fails any post to it, regardless of MAD content. And by definition the MAD stack will never get anything from QP0.
Quoted 7 lines
Quoted 4 lines
E.g. update the mad private
structure to indicate what sort of MAD it is --is_smp(),is_gmp(),is_rmpp(),is_opa(),is_ib(), much like the device checks are being updated.decording into the mad private structure during initial validation seems like a good idea.
I think what you mean is that the MAD needs to self-identify the "MAD space" (OPA vs IB) in which it resides. I agree this would be nice.
However, I need to work through the full implications of this.
Ira
Quoted 2 lines
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 11 lines · On 2/4/2015 6:29 PM, [email protected] wrote:
From: Ira Weiny <[email protected]>
The following patch series modifies the kernel MAD processing (ib_mad/ib_umad) and related interfaces to send and receive Intel Omni-Path Architecture MADs on devices which support them.
In addition to supporting some IBTA management classes, OPA devices use MADs with lengths up to 2K. These "jumbo" MADs increase the performance of management traffic.
To distinguish IBTA MADs from OPA MADs a new Base Version is introduced.
With your recent changes, I don't think that statement above is strictly true any longer. While OPA does use a different base version for it's jumbo MADs, aren't OPA MADs distinguished from IBTA MADs by the new OPA MAD device capability bit ?
Quoted 88 lines
The
new format shares the same common header with IBTA MADs which allows us to share most of the MAD processing code when dealing with the new Base Version.The patch series is broken into 3 main areas.
1) Add the ability for devices to indicate MAD size. modify the MAD code to use this MAD size
2) Enhance the interface to the device agents to support larger and variable length MADs.
3) Add capability bit to indicate support for OPA MADs
4) Add support for creating and processing OPA MADs
Changes for V4:
Rebased to latest Rolands for-next branch (3.19-rc4) Fixed compile issue in ehca driver found with 0-day build.
Ira Weiny (19):
IB/mad: Renameis_data_madtois_rmpp_data_mad
IB/core: Cache device attributes for use by upper level drivers IB/mad: Changevalidate_madsignature to takeib_mad_hdrrather than
ib_mad
IB/mad: Changeib_response_madsignature to takeib_mad_hdrrather
thanib_mad
IB/mad: Change cast inrcv_has_same_class
IB/core: Addmax_mad_sizetoib_device_attr
IB/mad: Convertib_mad_privateallocations fromkmem_cacheto kmalloc IB/mad: Add helper function forsmi_handle_dr_smp_sendIB/mad: Add helper function forsmi_handle_dr_smp_recvIB/mad: Add helper function forsmi_check_forward_dr_smpIB/mad: Add helper function for SMI processing
IB/mad: Add MAD size parameters toprocess_mad
IB/mad: Add base version parameter toib_create_send_madIB/core: AddIB_DEVICE_OPA_MAD_SUPPORTdevice cap flag IB/mad: Createjumbo_maddata structures
IB/mad: Add Intel Omni-Path Architecture defines
IB/mad: Implement support for Intel Omni-Path Architecture base
version MADs inib_create_send_mad
IB/mad: Implement Intel Omni-Path Architecture SMP processing IB/mad: Implement Intel Omni-Path Architecture MAD processingdrivers/infiniband/core/agent.c | 26 +- drivers/infiniband/core/agent.h | 3 +- drivers/infiniband/core/cm.c | 6 +- drivers/infiniband/core/device.c | 2 + drivers/infiniband/core/mad.c | 519 ++++++++++++++++++--------- drivers/infiniband/core/mad_priv.h | 7 +- drivers/infiniband/core/mad_rmpp.c | 144 ++++---- drivers/infiniband/core/opa_smi.h | 78 ++++ drivers/infiniband/core/sa_query.c | 3 +- drivers/infiniband/core/smi.c | 231 ++++++++---- drivers/infiniband/core/smi.h | 6 + drivers/infiniband/core/sysfs.c | 5 +- drivers/infiniband/core/user_mad.c | 38 +- drivers/infiniband/hw/amso1100/c2_provider.c | 5 +- drivers/infiniband/hw/amso1100/c2_rnic.c | 1 + drivers/infiniband/hw/cxgb3/iwch_provider.c | 6 +- drivers/infiniband/hw/cxgb4/provider.c | 8 +- drivers/infiniband/hw/ehca/ehca_hca.c | 3 + drivers/infiniband/hw/ehca/ehca_iverbs.h | 4 +- drivers/infiniband/hw/ehca/ehca_sqp.c | 8 +- drivers/infiniband/hw/ipath/ipath_mad.c | 8 +- drivers/infiniband/hw/ipath/ipath_verbs.c | 1 + drivers/infiniband/hw/ipath/ipath_verbs.h | 3 +- drivers/infiniband/hw/mlx4/mad.c | 12 +- drivers/infiniband/hw/mlx4/main.c | 1 + drivers/infiniband/hw/mlx4/mlx4_ib.h | 3 +- drivers/infiniband/hw/mlx5/mad.c | 8 +- drivers/infiniband/hw/mlx5/main.c | 1 + drivers/infiniband/hw/mlx5/mlx5_ib.h | 3 +- drivers/infiniband/hw/mthca/mthca_dev.h | 4 +- drivers/infiniband/hw/mthca/mthca_mad.c | 12 +- drivers/infiniband/hw/mthca/mthca_provider.c | 2 + drivers/infiniband/hw/nes/nes_verbs.c | 4 +- drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 3 +- drivers/infiniband/hw/ocrdma/ocrdma_ah.h | 3 +- drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 1 + drivers/infiniband/hw/qib/qib_iba7322.c | 3 +- drivers/infiniband/hw/qib/qib_mad.c | 11 +- drivers/infiniband/hw/qib/qib_verbs.c | 1 + drivers/infiniband/hw/qib/qib_verbs.h | 3 +-
Quoted 8 lines
drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 2 + drivers/infiniband/ulp/srpt/ib_srpt.c | 3 +- include/rdma/ib_mad.h | 40 ++- include/rdma/ib_verbs.h | 15 +- include/rdma/opa_smi.h | 106 ++++++ 45 files changed, 999 insertions(+), 357 deletions(-) create mode 100644 drivers/infiniband/core/opa_smi.h create mode 100644 include/rdma/opa_smi.h
What performance tests were run in terms of IBTA MADs ?
-- Hal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 54D52562.5050408-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org
Quoted 12 lines · On 2/4/2015 6:29 PM, [email protected] wrote:
Quoted 10 lines
From: Ira Weiny <[email protected]>
The following patch series modifies the kernel MAD processing (ib_mad/ib_umad) and related interfaces to send and receive Intel Omni-Path Architecture MADs on devices which support them.
In addition to supporting some IBTA management classes, OPA devices use MADs with lengths up to 2K. These "jumbo" MADs increase the performance of management traffic.
Quoted 7 lines
Quoted 1 line
To distinguish IBTA MADs from OPA MADs a new Base Version is introduced.
With your recent changes, I don't think that statement above is strictly true any longer. While OPA does use a different base version for it's jumbo MADs, aren't OPA MADs distinguished from IBTA MADs by the new OPA MAD device capability bit ?
True.
However, OPA MADs with a base version of 0x1 are compatible with and therefore can be processed by the same code as IBTA MADs.
If I need to respin the series I will update the comment.
Quoted 2 lines
What performance tests were run in terms of IBTA MADs ?
Sorry for the delay. As there have been a couple of versions of this series since I ran those tests in December I took the time to re-run these tests.
OpenSM (sweep time) and infiniband-diag (iblinkinfo and saquery) were timed on my small cluster with no noticeable change in performance. But this is not the best test as I only have 6 or so nodes on 2 switches.
For example iblinkinfo runs very quickly:
[root@phcppriv12 OPENIB_FF]# time iblinkinfo > /dev/null
real 0m0.072s user 0m0.002s sys 0m0.041s
The better test we have at this small scale are a couple of tools (closed source) which send SMA and PMA packets as rapidly as possible.
Those showed no difference in performance.
For example I ran these tools with 3 different kernels 1) "stock roland", 2) the series in question up to the kmalloc patch, and final 3) then the full OPA series.
Here is a summary of the results:
Roland for-next (ecb7b12) up to kmalloc patch full OPA patch set
SMA 21072 21324 21381
SMA rcv 17139 17329 17303
PMA 24159.4 24401.2 24166.5
PMA rcv 24159.4 24401.2 24166.5
NOTE: The results shown above are specifically shown without units as I am not allowed to publish performance numbers. However larger numbers equal better performance.
The numbers are quite repeatable and as you can see are all close to each other from kernel to kernel.
The remote node for these tests was running the stock RHEL7 kernel. All software and hardware was held constant except for the kernel patches in question.
Ira
Quoted 2 lines
-- Hal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html