Skip to contentSkip to conversation

verbs: Add RDMA write RC pingpong test

PATCH In review Animesh Kishore 2 participants 5 messages May 9, 2023 – May 10, 2023 InfiniBand

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 verbs: Add RDMA write RC pingpong test

- The test pingpongs data between server and client instance using RC QPs with RDMA write BTH opcode.
- For RDMA write, there's no completion at responder. Hence, we send a sideband ACK(using socket) from requester side on completion. This indicates to responder that it has received data.

Check available test arguments and help:
./build/bin/ibv_rc_wr_pingpong -h

e.g.
Run server instance:
./build/bin/ibv_rc_wr_pingpong -g 0 -d <ib_dev> -c -s 8192

Run client instance:
./build/bin/ibv_rc_wr_pingpong -g 0 -d <ib_dev> -c -s 8192 <server IP>

---
 debian/ibverbs-utils.install         |   2 +
 libibverbs/examples/CMakeLists.txt   |   3 +
 libibverbs/examples/rc_wr_pingpong.c | 782 +++++++++++++++++++++++++++
 libibverbs/man/CMakeLists.txt        |   1 +
 libibverbs/man/ibv_rc_wr_pingpong.1  |  63 +++
 5 files changed, 851 insertions(+)
 create mode 100644 libibverbs/examples/rc_wr_pingpong.c
 create mode 100644 libibverbs/man/ibv_rc_wr_pingpong.1
debian/ibverbs-utils.install +2 -0
index 170b8d26..3d77aa0f 100644
@@ -2,6 +2,7 @@ usr/bin/ibv_asyncwatch
 usr/bin/ibv_devices
 usr/bin/ibv_devinfo
 usr/bin/ibv_rc_pingpong
+usr/bin/ibv_rc_wr_pingpong
 usr/bin/ibv_srq_pingpong
 usr/bin/ibv_uc_pingpong
 usr/bin/ibv_ud_pingpong
@@ -10,6 +11,7 @@ usr/share/man/man1/ibv_asyncwatch.1
 usr/share/man/man1/ibv_devices.1
 usr/share/man/man1/ibv_devinfo.1
 usr/share/man/man1/ibv_rc_pingpong.1
+usr/share/man/man1/ibv_rc_wr_pingpong.1
 usr/share/man/man1/ibv_srq_pingpong.1
 usr/share/man/man1/ibv_uc_pingpong.1
 usr/share/man/man1/ibv_ud_pingpong.1
libibverbs/examples/CMakeLists.txt +3 -0
index dc4c4978..031faf0e 100644
@@ -26,3 +26,6 @@ target_link_libraries(ibv_ud_pingpong LINK_PRIVATE ibverbs ibverbs_tools)
 
 rdma_executable(ibv_xsrq_pingpong xsrq_pingpong.c)
 target_link_libraries(ibv_xsrq_pingpong LINK_PRIVATE ibverbs ibverbs_tools)
+
+rdma_executable(ibv_rc_wr_pingpong rc_wr_pingpong.c)
+target_link_libraries(ibv_rc_wr_pingpong LINK_PRIVATE ibverbs ibverbs_tools)
libibverbs/examples/rc_wr_pingpong.c new file +782 -0

782 lines not on this page · raw message

libibverbs/man/CMakeLists.txt +1 -0
index 843d7f0a..40fa64cf 100644
@@ -70,6 +70,7 @@ rdma_man_pages(
   ibv_rate_to_mbps.3.md
   ibv_rate_to_mult.3.md
   ibv_rc_pingpong.1
+  ibv_rc_wr_pingpong.1
   ibv_read_counters.3.md
   ibv_reg_mr.3
   ibv_req_notify_cq.3.md
libibverbs/man/ibv_rc_wr_pingpong.1 new file +63 -0

63 lines not on this page · raw message

Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/1 verbs: Add RDMA write RC pingpong test

- The test pingpongs data between server and client instance using RC QPs with RDMA write BTH opcode.
- For RDMA write, there's no completion at responder. Hence, we send a sideband ACK(using socket) from requester side on completion. This indicates to responder that it has received data.

Check available test arguments and help:
./build/bin/ibv_rc_wr_pingpong -h

e.g.
Run server instance:
./build/bin/ibv_rc_wr_pingpong -g 0 -d <ib_dev> -c -s 8192

Run client instance:
./build/bin/ibv_rc_wr_pingpong -g 0 -d <ib_dev> -c -s 8192 <server IP>

---
 debian/ibverbs-utils.install         |   2 +
 libibverbs/examples/CMakeLists.txt   |   3 +
 libibverbs/examples/rc_wr_pingpong.c | 782 +++++++++++++++++++++++++++
 libibverbs/man/CMakeLists.txt        |   1 +
 libibverbs/man/ibv_rc_wr_pingpong.1  |  63 +++
 5 files changed, 851 insertions(+)
 create mode 100644 libibverbs/examples/rc_wr_pingpong.c
 create mode 100644 libibverbs/man/ibv_rc_wr_pingpong.1
debian/ibverbs-utils.install +2 -0
index 170b8d26..3d77aa0f 100644
@@ -2,6 +2,7 @@ usr/bin/ibv_asyncwatch
 usr/bin/ibv_devices
 usr/bin/ibv_devinfo
 usr/bin/ibv_rc_pingpong
+usr/bin/ibv_rc_wr_pingpong
 usr/bin/ibv_srq_pingpong
 usr/bin/ibv_uc_pingpong
 usr/bin/ibv_ud_pingpong
@@ -10,6 +11,7 @@ usr/share/man/man1/ibv_asyncwatch.1
 usr/share/man/man1/ibv_devices.1
 usr/share/man/man1/ibv_devinfo.1
 usr/share/man/man1/ibv_rc_pingpong.1
+usr/share/man/man1/ibv_rc_wr_pingpong.1
 usr/share/man/man1/ibv_srq_pingpong.1
 usr/share/man/man1/ibv_uc_pingpong.1
 usr/share/man/man1/ibv_ud_pingpong.1
libibverbs/examples/CMakeLists.txt +3 -0
index dc4c4978..031faf0e 100644
@@ -26,3 +26,6 @@ target_link_libraries(ibv_ud_pingpong LINK_PRIVATE ibverbs ibverbs_tools)
 
 rdma_executable(ibv_xsrq_pingpong xsrq_pingpong.c)
 target_link_libraries(ibv_xsrq_pingpong LINK_PRIVATE ibverbs ibverbs_tools)
+
+rdma_executable(ibv_rc_wr_pingpong rc_wr_pingpong.c)
+target_link_libraries(ibv_rc_wr_pingpong LINK_PRIVATE ibverbs ibverbs_tools)
libibverbs/examples/rc_wr_pingpong.c new file +782 -0

782 lines not on this page · raw message

libibverbs/man/CMakeLists.txt +1 -0
index 843d7f0a..40fa64cf 100644
@@ -70,6 +70,7 @@ rdma_man_pages(
   ibv_rate_to_mbps.3.md
   ibv_rate_to_mult.3.md
   ibv_rc_pingpong.1
+  ibv_rc_wr_pingpong.1
   ibv_read_counters.3.md
   ibv_reg_mr.3
   ibv_req_notify_cq.3.md
libibverbs/man/ibv_rc_wr_pingpong.1 new file +63 -0

63 lines not on this page · raw message

Signature · 1 line
2.25.1
#2 lore raw
In reply to Animesh Kishore
Quoted 18 lines · Animesh Kishore

- The test pingpongs data between server and client instance using RC QPs with RDMA write BTH opcode.
- For RDMA write, there's no completion at responder. Hence, we send a sideband ACK(using socket) from requester side on completion. This indicates to responder that it has received data.

Check available test arguments and help:
./build/bin/ibv_rc_wr_pingpong -h

e.g.
Run server instance:
./build/bin/ibv_rc_wr_pingpong -g 0 -d <ib_dev> -c -s 8192

Run client instance:
./build/bin/ibv_rc_wr_pingpong -g 0 -d <ib_dev> -c -s 8192 <server IP>

Quoted 9 lines
---
 debian/ibverbs-utils.install         |   2 +
 libibverbs/examples/CMakeLists.txt   |   3 +
 libibverbs/examples/rc_wr_pingpong.c | 782 +++++++++++++++++++++++++++
 libibverbs/man/CMakeLists.txt        |   1 +
 libibverbs/man/ibv_rc_wr_pingpong.1  |  63 +++
 5 files changed, 851 insertions(+)
 create mode 100644 libibverbs/examples/rc_wr_pingpong.c
 create mode 100644 libibverbs/man/ibv_rc_wr_pingpong.1

Like I said in relevant PR https://github.com/linux-rdma/rdma-core/pull/1325#issuecomment-1531194836
This new ibv_rc_wr_pingpong is unlikely to be merged.

Thanks

#3 lore raw
Quoted 30 lines · Leon Romanovsky
Quoted 18 lines · Animesh Kishore

- The test pingpongs data between server and client instance using RC QPs with RDMA write BTH opcode.
- For RDMA write, there's no completion at responder. Hence, we send a sideband ACK(using socket) from requester side on completion. This indicates to responder that it has received data.

Check available test arguments and help:
./build/bin/ibv_rc_wr_pingpong -h

e.g.
Run server instance:
./build/bin/ibv_rc_wr_pingpong -g 0 -d <ib_dev> -c -s 8192

Run client instance:
./build/bin/ibv_rc_wr_pingpong -g 0 -d <ib_dev> -c -s 8192 <server IP>

Quoted 9 lines
---
 debian/ibverbs-utils.install         |   2 +
 libibverbs/examples/CMakeLists.txt   |   3 +
 libibverbs/examples/rc_wr_pingpong.c | 782 +++++++++++++++++++++++++++
 libibverbs/man/CMakeLists.txt        |   1 +
 libibverbs/man/ibv_rc_wr_pingpong.1  |  63 +++
 5 files changed, 851 insertions(+)
 create mode 100644 libibverbs/examples/rc_wr_pingpong.c
 create mode 100644 libibverbs/man/ibv_rc_wr_pingpong.1
Quoted 4 lines

Like I said in relevant PR https://github.com/linux-rdma/rdma-core/pull/1325#issuecomment-1531194836
This new ibv_rc_wr_pingpong is unlikely to be merged.

Thanks

Hi Leon,

Do you suggest to wait for more comments or close the PR at once ?

I think it's helpful for new users for below reasons.
Helps demonstrate complete flow and API usages. Works as a reference to build IBverbs production applications which are typically in C/C++.

Also note, it's an overkill to extend existing example/rc_pingpong.c (uses send/recv) to support RDMA write.

Thanks
Animesh

#4 lore raw
Quoted 31 lines · Animesh Kishore
Quoted 30 lines · Leon Romanovsky
Quoted 27 lines · Animesh Kishore

- The test pingpongs data between server and client instance using RC QPs with RDMA write BTH opcode.
- For RDMA write, there's no completion at responder. Hence, we send a sideband ACK(using socket) from requester side on completion. This indicates to responder that it has received data.

Check available test arguments and help:
./build/bin/ibv_rc_wr_pingpong -h

e.g.
Run server instance:
./build/bin/ibv_rc_wr_pingpong -g 0 -d <ib_dev> -c -s 8192

Run client instance:
./build/bin/ibv_rc_wr_pingpong -g 0 -d <ib_dev> -c -s 8192 <server IP>

---
 debian/ibverbs-utils.install         |   2 +
 libibverbs/examples/CMakeLists.txt   |   3 +
 libibverbs/examples/rc_wr_pingpong.c | 782 +++++++++++++++++++++++++++
 libibverbs/man/CMakeLists.txt        |   1 +
 libibverbs/man/ibv_rc_wr_pingpong.1  |  63 +++
 5 files changed, 851 insertions(+)
 create mode 100644 libibverbs/examples/rc_wr_pingpong.c
 create mode 100644 libibverbs/man/ibv_rc_wr_pingpong.1
Quoted 8 lines
Quoted 4 lines

Like I said in relevant PR https://github.com/linux-rdma/rdma-core/pull/1325#issuecomment-1531194836
This new ibv_rc_wr_pingpong is unlikely to be merged.

Thanks

Hi Leon,

Do you suggest to wait for more comments or close the PR at once ?

You need to find more people who think that such application must be included in rdma-core.

Quoted 7 lines

I think it's helpful for new users for below reasons.
Helps demonstrate complete flow and API usages. Works as a reference to build IBverbs production applications which are typically in C/C++.

Also note, it's an overkill to extend existing example/rc_pingpong.c (uses send/recv) to support RDMA write.

It is unlikely to be accepted either.

Quoted 3 lines

Thanks
Animesh