Skip to contentSkip to conversation

Mellanox implementation for atomic operations

DISCUSSION lihaidong 3 participants 3 messages May 10, 2010 – Jun 16, 2010 3 messages not in the archive InfiniBand

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

lihaidong

author
#1 lore raw

Hi,
I have a question about atomic operations.
According to IB specification o10-48, all atomic operation request made to the same HCA, referencing the same physical memory are serialized with respect to each other. I know this should be complied with if HCA supports atomic operations, right?
According to IB specification o10-49, all atomic operations requests that referencing the same physical memory are serialized with respect to each other. This means that atomic operations performed by processors should serialized with atomic operations performed by HCAs, too, if they were referencing the same physical memory.
I want to know whether Mellanox implementation for atomic operations comply with o10-49 or not.
if not ,to what extent it comply with the rule?
I also was intrested in how this rule is complied with by others vendors?

2010-05-10
lihaidong

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw

Hi.

Quoted 4 lines · lihaidong

Hi,
I have a question about atomic operations.
According to IB specification o10-48, all atomic operation request made to the same HCA, referencing the same physical memory are serialized with respect to each other. I know this should be complied with if HCA supports atomic operations, right?

Right.

Quoted 2 lines

According to IB specification o10-49, all atomic operations requests that referencing the same physical memory are serialized with respect to each other. This means that atomic operations performed by processors should serialized with atomic operations performed by HCAs, too, if they were referencing the same physical memory.

So far so good.

Quoted 5 lines

I want to know whether Mellanox implementation for atomic operations comply with o10-49 or not. if not ,to what extent it comply with the rule?
I also was intrested in how this rule is complied with by others vendors?

Let give you a general answer:
The struct ibv_device_attr contains the atomic_cap attribute, this attribute defines the atomicity
level that the HCA support (None, only within the HCA, between all HCAs (global)).

I think that your code should check this attribute (This way your code will support all vendors HCAs).

As much as i know, atomic operations are only supported within one HCA.

I hope that this answer helped you ..
Dotan

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#5 lore raw

The ib_qib driver supports atomic IB operations and they are global since it does it in the host software instead of PCIe bus transactions which don't have global atomic support (yet).

Quoted 27 lines · Dotan Barak

Hi.

Quoted 4 lines · lihaidong

Hi,
I have a question about atomic operations.
According to IB specification o10-48, all atomic operation request made to the same HCA, referencing the same physical memory are serialized with respect to each other. I know this should be complied with if HCA supports atomic operations, right?

Right.

Quoted 2 lines

According to IB specification o10-49, all atomic operations requests that referencing the same physical memory are serialized with respect to each other. This means that atomic operations performed by processors should serialized with atomic operations performed by HCAs, too, if they were referencing the same physical memory.

So far so good.

Quoted 5 lines

I want to know whether Mellanox implementation for atomic operations comply with o10-49 or not. if not ,to what extent it comply with the rule?
I also was intrested in how this rule is complied with by others vendors?

Let give you a general answer:
The struct ibv_device_attr contains the atomic_cap attribute, this attribute defines the atomicity
level that the HCA support (None, only within the HCA, between all HCAs (global)).

I think that your code should check this attribute (This way your code will support all vendors HCAs).

As much as i know, atomic operations are only supported within one HCA.

Quoted 7 lines

I hope that this answer helped you ..
Dotan

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html