Github RXE CI support
Hi,
On July 2024, I attempted to contribute the new feature of Valkey Over RDMA to the Valkey community, which allows Valkey to communicate using RDMA and achieved significant performance improvements and latency reductions. Valkey is a popular KV database with a large number of users, so relevant CI testing is necessary. However, the virtual machine used by GitHub does not support RXE (compilation option is not turned on), so we need to compile an out of tree RXE driver in a timely manner and install it into CI's virtual machine, and then run the automated testing program. At that time, in order to support this feature as soon as possible, I copied the RXE code from Linux based on the kernel version of GitHub virtual machine and successfully ran it. Later, with the upgrade of the GitHub virtual machine kernel version, it was also upgraded once.
Link of my RXE: https://github.com/pizhenwei/rxe.git
Github CI steps example(Valkey: https://github.com/valkey-io/valkey.git, CI config: .github/workflows/ci.yml):
steps:
- name: clone-rxe-kmod
run: |
mkdir -p tests/rdma/rxe
git clone https://github.com/pizhenwei/rxe.git tests/rdma/rxe
make -C tests/rdma/rxe
- name: clear-kernel-log
run: sudo dmesg -c > /dev/null
- name: test
run: sudo ./runtest-rdma --install-rxe
Because Github CI virtual machine does not allow to run 'insmod -f tests/rdma/rxe/rdma_rxe.ko' directly, it's installed by 'sudo ./runtest-rdma --install-rxe'.
With the increasing popularity of RDMA, more and more services may require automated testing based on RXE. I hope to have community support for the corresponding official version, rather than using multiple personally supported versions(like my RXE).
Looking forward to the response from the RXE community!
Quoted 17 lines · zhenwei pi
Hi,
On July 2024, I attempted to contribute the new feature of Valkey Over RDMA to the Valkey community, which allows Valkey to communicate using RDMA and achieved significant performance improvements and latency reductions. Valkey is a popular KV database with a large number of users, so relevant CI testing is necessary. However, the virtual machine used by GitHub does not support RXE (compilation option is not turned on), so we need to compile an out of tree RXE driver in a timely manner and install it into CI's virtual machine, and then run the automated testing program. At that time, in order to support this feature as soon as possible, I copied the RXE code from Linux based on the kernel version of GitHub virtual machine and successfully ran it. Later, with the upgrade of the GitHub virtual machine kernel version, it was also upgraded once.
Link of my RXE: https://github.com/pizhenwei/rxe.git
Github CI stepsexample(Valkey: https://github.com/valkey-io/valkey.git, CI config: .github/workflows/ci.yml):
steps:
- name: clone-rxe-kmod
run: |
mkdir -p tests/rdma/rxe
git clone https://github.com/pizhenwei/rxe.git tests/rdma/rxe
make -C tests/rdma/rxe
- name: clear-kernel-log
run: sudo dmesg -c >/dev/null
- name: test
run: sudo ./runtest-rdma --install-rxe
Quoted 3 lines
Because Github CI virtual machine does not allow to run 'insmod -f
tests/rdma/rxe/rdma_rxe.ko' directly, it's installed by 'sudo ./runtest-rdma --install-rxe'.With the increasing popularity of RDMA, more and more services may require automated testing based on RXE. I hope to have community support for the corresponding official version, rather than using multiple personally supported versions(like my RXE).
There is no such thing like "official supported version" for in-tree kernel modules. Use latest from rdma-rc branch.
Thanks
Quoted 3 lines
Looking forward to the response from the RXE community!