WRITE/SEND to multiple destinations
Hello everyone,
As a newbie to the world of InfiniBand programming,
I would appreciate your helps on my somewhat general and basic question.
Assume that I have a server, and N clients (N > 1).
I would like my server to set all clients' memory buffers to some value V. How would I go about this problem?
So far, I've managed to establish 1-1 connections using READ/WRITE/SEND/RECEIVE. So I guess the core of my question is:
how 1-to-N connections are established in InfiniBand, and
if, in this regard, there is any difference between
RDMA verbs (READ/WRITE) vs. SEND/RECEIVE?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
You can do SEND/RECV over Unreliable Datagram multicast which could be beneficial in your situation (i.e., better than 1-1 connections). You can't use multicast for READs or WRITEs.
--Anuj
Quoted 14 lines · floaterions
Hello everyone,
As a newbie to the world of InfiniBand programming,
I would appreciate your helps on my somewhat general and basic question.Assume that I have a server, and N clients (N > 1).
I would like my server to set all clients' memory buffers to some value V. How would I go about this problem?
So far, I've managed to establish 1-1 connections using READ/WRITE/SEND/RECEIVE. So I guess the core of my question is:
how 1-to-N connections are established in InfiniBand, and
if, in this regard, there is any difference between
RDMA verbs (READ/WRITE) vs. SEND/RECEIVE?
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
For the RC transport, the answer is yes. You definitely need one queue pair per client at the server, although it could be possible for the QPs at the server to share an event channel.
But UD transport is also pretty "reliable" as packet loss in any InfiniBand transport is extremely rare. A rather high ballpark is one in a billion.
--Anuj
Quoted 34 lines · Just Floaterions
Thank you for your help. So the question is, for reliable connections, do I need separate event channel/queue-pairs for each connection?
Quoted 28 lines · Anuj Kalia
You can do SEND/RECV over Unreliable Datagram multicast which could be beneficial in your situation (i.e., better than 1-1 connections). You can't use multicast for READs or WRITEs.
--Anuj
Quoted 20 lines · floaterions
Hello everyone,
As a newbie to the world of InfiniBand programming,
I would appreciate your helps on my somewhat general and basic question.Assume that I have a server, and N clients (N > 1).
I would like my server to set all clients' memory buffers to some value V.
How would I go about this problem?
So far, I've managed to establish 1-1 connections using READ/WRITE/SEND/RECEIVE.
So I guess the core of my question is:
how 1-to-N connections are established in InfiniBand, and if, in this regard, there is any difference between
RDMA verbs (READ/WRITE) vs. SEND/RECEIVE?--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Stupid me. I totally forgot about the XRC transport:
http://mvapich.cse.ohio-state.edu/static/media/publications/abstract/koop-cluster08.pdf.
--Anuj
Quoted 4 lines · Anuj Kalia
For the RC transport, the answer is yes. You definitely need one queue pair per client at the server, although it could be possible for the QPs at the server to share an event channel.
Quoted 41 lines
But UD transport is also pretty "reliable" as packet loss in any InfiniBand transport is extremely rare. A rather high ballpark is one in a billion.
--Anuj
Quoted 34 lines · Just Floaterions
Thank you for your help. So the question is, for reliable connections, do I need separate event channel/queue-pairs for each connection?
Quoted 28 lines · Anuj Kalia
You can do SEND/RECV over Unreliable Datagram multicast which could be beneficial in your situation (i.e., better than 1-1 connections). You can't use multicast for READs or WRITEs.
--Anuj
Quoted 20 lines · floaterions
Hello everyone,
As a newbie to the world of InfiniBand programming,
I would appreciate your helps on my somewhat general and basic question.Assume that I have a server, and N clients (N > 1).
I would like my server to set all clients' memory buffers to some value V.
How would I go about this problem?
So far, I've managed to establish 1-1 connections using READ/WRITE/SEND/RECEIVE.
So I guess the core of my question is:
how 1-to-N connections are established in InfiniBand, and if, in this regard, there is any difference between
RDMA verbs (READ/WRITE) vs. SEND/RECEIVE?--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html