IB/core: Add support for enhanced atomic operations

- Add a new IB_WR_ATOMIC_MASKED_CMP_AND_SWP and IB_WR_ATOMIC_MASKED_FETCH_AND_ADD send opcodes that can be used to mark a "masked atomic compare and swap" and "masked atomic fetch and add" work request correspondingly.
- Add IB_DEVICE_MASKED_ATOMIC capability bit.
- Add mask fields to atomic struct of ib_send_wr
- Add new opcodes to ib_wc_opcode

---
 include/rdma/ib_verbs.h |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

index 09509ed..338cff1 100644
@@ -105,6 +105,7 @@ enum ib_device_cap_flags {
 	IB_DEVICE_UD_TSO		= (1<<19),
 	IB_DEVICE_MEM_MGT_EXTENSIONS	= (1<<21),
 	IB_DEVICE_BLOCK_MULTICAST_LOOPBACK = (1<<22),
+	IB_DEVICE_MASKED_ATOMIC		= (1<<23),
 };
 
 enum ib_atomic_cap {

@@ -467,6 +468,8 @@ enum ib_wc_opcode {
 	IB_WC_LSO,
 	IB_WC_LOCAL_INV,
 	IB_WC_FAST_REG_MR,
+	IB_WC_MASKED_COMP_SWAP,
+	IB_WC_MASKED_FETCH_ADD,
 /*
  * Set value of IB_WC_RECV so consumers can test if a completion is a
  * receive by testing (opcode & IB_WC_RECV).

@@ -689,6 +692,8 @@ enum ib_wr_opcode {
 	IB_WR_RDMA_READ_WITH_INV,
 	IB_WR_LOCAL_INV,
 	IB_WR_FAST_REG_MR,
+	IB_WR_ATOMIC_MASKED_CMP_AND_SWP,
+	IB_WR_ATOMIC_MASKED_FETCH_AND_ADD,
 };
 
 enum ib_send_flags {

@@ -732,6 +737,8 @@ struct ib_send_wr {
 			u64	compare_add;
 			u64	swap;
 			u32	rkey;
+			u64	compare_add_mask;
+			u64	swap_mask;
 		} atomic;
 		struct {
 			struct ib_ah *ah;

1.6.6.GIT

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

I think I'd be OK adding this for 2.6.34, if there were some case made that this would be useful for someone working on code that used this. So what case can be made for adding this to 2.6.34 without an in-tree user?

The description of these operations from the mlx4 patch really belongs here.

Roland Dreier  <[email protected]>
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Support needs to be there before anyone can use it :)

Here is a git tree of our atomics work so far:
http://www.openfabrics.org/git/?p=~agrover/ofed_1_5/linux-2.6.git;a=shortlog;h=future/20100203

Still in-development, but it should be ready very soon. We will be the first in-kernel user of atomics, as well as masked atomics. This tree does not support masked atomics yet because it is based on ofed 1.5.1. When 1.5.2 is officially released, I'll rebase, add mask support, and plan on pushing to mainline and ofed 1.6 as soon as it opens.

Regards -- Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

May be I missed something, but how do you guys intend to reflect this new functionality vs. the capabilities? A new atomic_enhanced_cap ?

The reason for asking is that the IB ordinary atomic repertoire fits nicely with those of PCIe Gen 3, so one could possible assume new HCAs supporting PCIe Gen3 to possess the ATOMIC_GLOB capability for atomic_cap. But I do not see that happening to the proposed Enhanced IB Atomics.

Thanks, Håkon

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html