Skip to contentSkip to conversation

Different signatures to ibv_open_xrcd - branch issue?

DISCUSSION Shlomo Pongratz 3 participants 8 messages Mar 25, 2013 – Apr 4, 2013 6 messages not in the archive InfiniBand

lore
Share & tools
lore

Shlomo Pongratz

author
#0 lore raw

Hi Sean,

In libibverbs/include/infiniband/verbs.h the API "ibv_open_xrcd" is defined with the following signature;
static inline struct ibv_xrcd *
ibv_open_xrcd(struct ibv_context *context, struct ibv_xrcd_init_attr *xrcd_init_attr)
And libibverbs/examples/xsrq_pingpong.c the call is O.K. however in librdmacm/examples/rdma_xserver.c I see a different call sequence i.e. "attr.ext.xrc.xrcd = ibv_open_xrcd(srq_id->verbs, -1, 0);" My git branch for libivvers is based on origin/xrc2 and for librdmacm is based on origin/xrc.

Is any of my branches wrong?

best regards,

S.P.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 51502C30.2000806-VPRAkNaXOzVWk0Htik3J/[email protected]

#2 lore raw
Quoted 11 lines

In libibverbs/include/infiniband/verbs.h the API "ibv_open_xrcd" is defined with the following signature;
static inline struct ibv_xrcd *
ibv_open_xrcd(struct ibv_context *context, struct ibv_xrcd_init_attr *xrcd_init_attr)
And libibverbs/examples/xsrq_pingpong.c the call is O.K. however in librdmacm/examples/rdma_xserver.c I see a different call sequence i.e. "attr.ext.xrc.xrcd = ibv_open_xrcd(srq_id->verbs, -1, 0);" My git branch for libivvers is based on origin/xrc2 and for librdmacm is based on origin/xrc.

Quoted 1 line

Is any of my branches wrong?

The librdmacm changes are against an older version of the changes. The libibverbs has the correct API, which incorporates the extension mechanism that's been discussed. Once libibverbs has been accepted, the librdmacm can be updated. (The librdmacm code is surrounded by a #define, which disables it from the build.)

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A823736F367AB8-P5GAC/[email protected]

author
#4 lore raw
Quoted 11 lines · Sean
Quoted 11 lines

In libibverbs/include/infiniband/verbs.h the API "ibv_open_xrcd" is defined with the following signature;
static inline struct ibv_xrcd *
ibv_open_xrcd(struct ibv_context *context, struct ibv_xrcd_init_attr *xrcd_init_attr)
And libibverbs/examples/xsrq_pingpong.c the call is O.K. however in librdmacm/examples/rdma_xserver.c I see a different call sequence i.e. "attr.ext.xrc.xrcd = ibv_open_xrcd(srq_id->verbs, -1, 0);" My git branch for libivvers is based on origin/xrc2 and for librdmacm is based on origin/xrc.

Quoted 4 lines
Quoted 1 line

Is any of my branches wrong?

The librdmacm changes are against an older version of the changes. The libibverbs has the correct API, which incorporates the extension mechanism that's been discussed. Once libibverbs has been accepted, the librdmacm can be updated. (The librdmacm code is surrounded by a #define, which disables it from the build.)

- Sean

I understand that currently I can't do RDMA using XRC, is that the case?
How complicated are the required modifications needed to make librdmacm complies to the libibverbs changes?

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 515173F2.4030506-VPRAkNaXOzVWk0Htik3J/[email protected]

#6 lore raw
Quoted 3 lines

I understand that currently I can't do RDMA using XRC, is that the case? How complicated are the required modifications needed to make librdmacm complies to the libibverbs changes?

The changes needed for the librdmacm to support XRC aren't huge. The complication is getting XRC merged with libibverbs. --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A823736F36825D-P5GAC/[email protected]

author
#8 lore raw
Quoted 4 lines · Sean
Quoted 3 lines

I understand that currently I can't do RDMA using XRC, is that the case? How complicated are the required modifications needed to make librdmacm complies to the libibverbs changes?

The changes needed for the librdmacm to support XRC aren't huge. The complication is getting XRC merged with libibverbs.

I think I missed something here, can I use your branch xrc2 of libibverbs for testing purpose, or there are still things to merge there? And for librdmacm, I'm looking forward to try it with a real application the moment you'll have a branch witch is compatible with libibverbs.

S.P.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 51520D43.2060006-VPRAkNaXOzVWk0Htik3J/[email protected]

#10 lore raw
Quoted 2 lines

I think I missed something here, can I use your branch xrc2 of libibverbs for testing purpose, or there are still things to merge there?

The xrc2 branch is usable. I'm about 10-15 minutes away from posting v5 of those patches.

Quoted 2 lines

And for librdmacm, I'm looking forward to try it with a real application the moment you'll have a branch witch is compatible with libibverbs.

The librdmacm needs to be updated to match the latest libibverbs. If the libibverbs changes are ever merged upstream, I'll update the librdmacm at that point.

Until then, the only sample app for XRC is ibv_xsrq_pingpong.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 1828884A29C6694DAF28B7E6B8A823736F368566-P5GAC/[email protected]

#12 lore raw

Hi,

Hefty, Sean wrote on Tue, Mar 26, 2013 :

Quoted 4 lines
Quoted 2 lines

I think I missed something here, can I use your branch xrc2 of libibverbs for testing purpose, or there are still things to merge there?

The xrc2 branch is usable. I'm about 10-15 minutes away from posting v5 of those patches.

Sorry for hijacking this thread, but I was actually running into some troubles with libibverbs-1.1.4 and a recent kernel, using ibv_open_xrc_domain... Which is basically failing with an invalid argument when writing to the uverb0 char device.

So I guess the "commands" (IB_USER_VERBS_CMD_OPEN_XRCD vs IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN) changed since 2.6.32 and I would need to use this libibverbs branch for now? (doesn't look like there is any xrcd at all in 1.1.6 anyway)
What I don't get is that the abi_version did not change at all, if the commands changed, shouldn't the kernel abi version be updated as well? (None of the four abi_version I could find (ib_verbs, ib_mad, rdma_cm, uverbs0 in the device) changed at all)

This looks like it's impossible to get the same library to fully work for multiple kernel versions, while all the other basic features do work quite well independantly.

Regards,

Signature · 5 lines
Dominique Martinet
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#13 lore raw

PiBTbyBJIGd1ZXNzIHRoZSAiY29tbWFuZHMiIChJQl9VU0VSX1ZFUkJTX0NNRF9PUEVOX1hSQ0Qg dnMNCj4gSUJfVVNFUl9WRVJCU19DTURfT1BFTl9YUkNfRE9NQUlOKSBjaGFuZ2VkIHNpbmNlIDIu Ni4zMiBhbmQgSSB3b3VsZCBuZWVkDQo+IHRvIHVzZSB0aGlzIGxpYmlidmVyYnMgYnJhbmNoIGZv ciBub3c/IChkb2Vzbid0IGxvb2sgbGlrZSB0aGVyZSBpcyBhbnkNCj4geHJjZCBhdCBhbGwgaW4g MS4xLjYgYW55d2F5KQ0KPiBXaGF0IEkgZG9uJ3QgZ2V0IGlzIHRoYXQgdGhlIGFiaV92ZXJzaW9u IGRpZCBub3QgY2hhbmdlIGF0IGFsbCwgaWYgdGhlDQo+IGNvbW1hbmRzIGNoYW5nZWQsIHNob3Vs ZG4ndCB0aGUga2VybmVsIGFiaSB2ZXJzaW9uIGJlIHVwZGF0ZWQgYXMgd2VsbD8NCj4gKE5vbmUg b2YgdGhlIGZvdXIgYWJpX3ZlcnNpb24gSSBjb3VsZCBmaW5kIChpYl92ZXJicywgaWJfbWFkLCBy ZG1hX2NtLA0KPiB1dmVyYnMwIGluIHRoZSBkZXZpY2UpIGNoYW5nZWQgYXQgYWxsKQ0KDQpUaGUg KnVwc3RyZWFtKiBrZXJuZWwgQUJJIGhhcyBub3QgY2hhbmdlZC4gIE9GRUQgcHJvdmlkZWQga2Vy bmVsIHBhdGNoZXMgdGhhdCBtb2RpZmllZCBwcmV2aW91cyBrZXJuZWxzLiAgVGhlIEFCSSB0aGF0 IE9GRUQgcGF0Y2hlZCBpbiBkaWZmZXJlZCBmcm9tIHdoYXQgd2FzIGZpbmFsbHkgbWVyZ2VkIHVw c3RyZWFtLiAgSSBleHBlY3QgdGhhdCB0aGUgdXBzdHJlYW0gbGliaWJ2ZXJicyB3aWxsIHdvcmsg YWdhaW5zdCBhbGwgdXBzdHJlYW0ga2VybmVscywgd2l0aCBmdWxsIGJpbmFyeSBjb21wYXRpYmls aXR5IGZvciBleGlzdGluZyBhcHBsaWNhdGlvbnMuICBZb3VyIGJpbmFyeSBjb21wYXRpYmlsaXR5 IHN1cHBvcnQgd2l0aCBPRkVEIHdpbGwgdmFyeS4NCg0KLSBTZWFuDQo=
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html