Skip to contentSkip to conversation

Github RXE CI support

DISCUSSION zhenwei pi 2 participants 2 messages Jun 8, 2025 – Jun 12, 2025 InfiniBand

lore
Share & tools
lore

zhenwei pi

author
#0 lore raw

Hi,

On July 2024, I attempted to contribute the new feature of Valkey Over RDMA to the Valkey community, which allows Valkey to communicate using RDMA and achieved significant performance improvements and latency reductions. Valkey is a popular KV database with a large number of users, so relevant CI testing is necessary. However, the virtual machine used by GitHub does not support RXE (compilation option is not turned on), so we need to compile an out of tree RXE driver in a timely manner and install it into CI's virtual machine, and then run the automated testing program. At that time, in order to support this feature as soon as possible, I copied the RXE code from Linux based on the kernel version of GitHub virtual machine and successfully ran it. Later, with the upgrade of the GitHub virtual machine kernel version, it was also upgraded once.

Link of my RXE: https://github.com/pizhenwei/rxe.git
Github CI steps example(Valkey: https://github.com/valkey-io/valkey.git, CI config: .github/workflows/ci.yml):
steps:
- name: clone-rxe-kmod
run: |
mkdir -p tests/rdma/rxe
git clone https://github.com/pizhenwei/rxe.git tests/rdma/rxe
make -C tests/rdma/rxe
- name: clear-kernel-log
run: sudo dmesg -c > /dev/null
- name: test
run: sudo ./runtest-rdma --install-rxe

Because Github CI virtual machine does not allow to run 'insmod -f tests/rdma/rxe/rdma_rxe.ko' directly, it's installed by 'sudo ./runtest-rdma --install-rxe'.

With the increasing popularity of RDMA, more and more services may require automated testing based on RXE. I hope to have community support for the corresponding official version, rather than using multiple personally supported versions(like my RXE).

Looking forward to the response from the RXE community!

#1 lore raw
Quoted 17 lines · zhenwei pi

Hi,

On July 2024, I attempted to contribute the new feature of Valkey Over RDMA to the Valkey community, which allows Valkey to communicate using RDMA and achieved significant performance improvements and latency reductions. Valkey is a popular KV database with a large number of users, so relevant CI testing is necessary. However, the virtual machine used by GitHub does not support RXE (compilation option is not turned on), so we need to compile an out of tree RXE driver in a timely manner and install it into CI's virtual machine, and then run the automated testing program. At that time, in order to support this feature as soon as possible, I copied the RXE code from Linux based on the kernel version of GitHub virtual machine and successfully ran it. Later, with the upgrade of the GitHub virtual machine kernel version, it was also upgraded once.

Link of my RXE: https://github.com/pizhenwei/rxe.git
Github CI steps example(Valkey: https://github.com/valkey-io/valkey.git, CI config: .github/workflows/ci.yml):
steps:
- name: clone-rxe-kmod
run: |
mkdir -p tests/rdma/rxe
git clone https://github.com/pizhenwei/rxe.git tests/rdma/rxe
make -C tests/rdma/rxe
- name: clear-kernel-log
run: sudo dmesg -c > /dev/null
- name: test
run: sudo ./runtest-rdma --install-rxe

Quoted 3 lines

Because Github CI virtual machine does not allow to run 'insmod -f tests/rdma/rxe/rdma_rxe.ko' directly, it's installed by 'sudo ./runtest-rdma --install-rxe'.

With the increasing popularity of RDMA, more and more services may require automated testing based on RXE. I hope to have community support for the corresponding official version, rather than using multiple personally supported versions(like my RXE).

There is no such thing like "official supported version" for in-tree kernel modules. Use latest from rdma-rc branch.

Thanks

Quoted 3 lines

Looking forward to the response from the RXE community!