Enable checksum offload capability reporting

A new filed csum_cap is added to both ib_query_device. It contains two members:
eth_csum_cap and ib_csum_cap, indicates checksum capability of Ethernet and Infiniband link layer respectively for different QP types.

Current checksum caps use the following enum members:
- IB_CSUM_SUPPORT_UD: device supports validation/calculation of csum for UD QP.
- IB_CSUM_SUPPORT_RAW: device supports validation/calculation of csum for raw QP.

A new filed csum_cap is added to both ib_query_device. It contains two members:
eth_csum_cap and ib_csum_cap, indicates checksum capability of Ethernet and Infiniband link layer respectively for different QP types.

Current checksum caps use the following enum members:
- IB_CSUM_SUPPORT_UD: device supports validation/calculation of csum for UD QP.
- IB_CSUM_SUPPORT_RAW: device supports validation/calculation of csum for raw QP.

A new filed csum_cap is added to both ib_query_device. It contains two members:
eth_csum_cap and ib_csum_cap, indicates checksum capability of Ethernet and Infiniband link layer respectively for different QP types.

Current checksum caps use the following enum members:
- IB_CSUM_SUPPORT_UD: device supports validation/calculation of csum for UD QP.
- IB_CSUM_SUPPORT_RAW: device supports validation/calculation of csum for raw QP.

Signed-off-by Bodong Wang <bodong-VPRAkNaXOzVWk0Htik3J/[email protected]>

---
 include/rdma/ib_verbs.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

include/rdma/ib_verbs.h +0 -0

index b0f898e..94dbaee 100644

include/rdma/ib_verbs.h +5 -0

 struct ib_device_attr {
 	u64			fw_ver;
 	__be64			sys_image_guid;

@@ -229,6 +234,7 @@ struct ib_device_attr {
 	struct ib_odp_caps	odp_caps;
 	uint64_t		timestamp_mask;
 	uint64_t		hca_core_clock; /* in KHZ */
+	struct ib_csum_cap_per_link csum_cap;
 };
 
 enum ib_mtu {

@@ -868,6 +874,10 @@ enum ib_qp_create_flags {
 	IB_QP_CREATE_RESERVED_END		= 1 << 31,
 };
 
+enum ib_csum_cap_flags {
+	IB_CSUM_SUPPORT_UD	= (1 << IB_QPT_UD),
+	IB_CSUM_SUPPORT_RAW	= (1 << IB_QPT_RAW_PACKET),
+};
 
 /*
  * Note: users may not call ib_close_qp or ib_destroy_qp from the event_handler

Signed-off-by Bodong Wang <bodong-VPRAkNaXOzVWk0Htik3J/[email protected]>

---
 drivers/infiniband/hw/mlx4/main.c | 3 +++
 1 file changed, 3 insertions(+)

drivers/infiniband/hw/mlx4/main.c +0 -0

index 8be6db8..a70ca6a 100644

Quoted 10 lines · Bodong Wang

Signed-off-by Bodong Wang <bodong-VPRAkNaXOzVWk0Htik3J/[email protected]>

---
 drivers/infiniband/hw/mlx4/main.c | 3 +++
 1 file changed, 3 insertions(+)

drivers/infiniband/hw/mlx4/main.c +0 -0

index 8be6db8..a70ca6a 100644

drivers/infiniband/hw/mlx4/main.c static int mlx4_ib_query_device(struct ib_device *ibdev, · line 222

Quoted 6 lines

drivers/infiniband/hw/mlx4/main.c +3 -0

@@ -217,6 +217,9 @@ static int mlx4_ib_query_device(struct ib_device *ibdev,
              props->device_cap_flags |= IB_DEVICE_MANAGED_FLOW_STEERING;
      }
 
+     props->csum_cap.eth_csum_cap |= IB_CSUM_SUPPORT_RAW;
+     props->csum_cap.ib_csum_cap |= IB_CSUM_SUPPORT_UD;
+

This patch highlights something I didn't think about on the previous patch. Why separate eth/ib if you have per QP flags? The QP denotes the ib/eth relationship without the need to separate it into two different caps. In other words, you can never have an IB qp type on eth because the only eth QP types we support other than RAW are all RDMA and

not IP. Really, there's enough spare bits in ib_device_cap_flags that you could do away with the new caps entirely. Right now, we support UD (which we already have a flag for), we can add two flags (for RAW and RC) and that should cover all of the foreseeable options as that would allow us to extend IP CSUM support to cover connected mode and cover all of the current options. I don't see us doing IP traffic in any other situation, so I thing that should suffice. Bits 25 and 26 could be used for the two new bits. Then you just need to extend the bits to user space.

Quoted 23 lines · Doug Ledford

Quoted 10 lines · Bodong Wang

Signed-off-by Bodong Wang <bodong-VPRAkNaXOzVWk0Htik3J/[email protected]>

---
 drivers/infiniband/hw/mlx4/main.c | 3 +++
 1 file changed, 3 insertions(+)

drivers/infiniband/hw/mlx4/main.c +0 -0

index 8be6db8..a70ca6a 100644

drivers/infiniband/hw/mlx4/main.c static int mlx4_ib_query_device(struct ib_device *ibdev, · line 222

Quoted 6 lines

drivers/infiniband/hw/mlx4/main.c +3 -0

@@ -217,6 +217,9 @@ static int mlx4_ib_query_device(struct ib_device *ibdev,
              props->device_cap_flags |= IB_DEVICE_MANAGED_FLOW_STEERING;
      }
 
+     props->csum_cap.eth_csum_cap |= IB_CSUM_SUPPORT_RAW;
+     props->csum_cap.ib_csum_cap |= IB_CSUM_SUPPORT_UD;
+

This patch highlights something I didn't think about on the previous patch. Why separate eth/ib if you have per QP flags? The QP denotes the ib/eth relationship without the need to separate it into two different caps. In other words, you can never have an IB qp type on eth because the only eth QP types we support other than RAW are all RDMA and

Quoted 8 lines

not IP. Really, there's enough spare bits in ib_device_cap_flags that you could do away with the new caps entirely. Right now, we support UD (which we already have a flag for), we can add two flags (for RAW and RC) and that should cover all of the foreseeable options as that would allow us to extend IP CSUM support to cover connected mode and cover all of the current options. I don't see us doing IP traffic in any other situation, so I thing that should suffice. Bits 25 and 26 could be used for the two new bits. Then you just need to extend the bits to user space.

Doug,

The vendor may support the offload for a certain QP type only over certain link.

E.g mlx4 support checksum for UD QPs only over IB but not over Eth,

checksum for RC QPs isn't supported,

and RAW_PACKET QPs
are available anyway only for Eth links.

But if this is what you think
needs to be done, I guess we can do that.

Here's the only matrix of IP checksumming that makes sense:

1) UD over IB (because it is one of the supported IPoIB types)
2) RC over IB (same as above)
3) Raw ETH over Eth (because IP over Eth makes sense and is a common type of packet to send on Raw Eth, but Raw Eth will never be sent on IB as it isn't supported there at all)

Anything else would require adding more Raw ETH QPs elsewhere, or expanding the IPoIB spec to include more connection types.

Quoted 2 lines

Really, there's enough spare bits in ib_device_cap_flags that you could do away with the new caps entirely.