Skip to contentSkip to conversation

perftest: add a ROCm build to CI

PATCH In review Stephen Bates 3 participants 4 messages Nov 6 – Nov 16 InfiniBand

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 perftest: add a ROCm build to CI

CI builds only the CUDA configuration. The two existing workflows pass no --enable-rocm, so src/rocm_memory.c and everything behind HAVE_ROCM and HAVE_ROCM_DMABUF is never compiled on a pull request, and a change that breaks the ROCm backend goes green.

Add an Ubuntu 24.04 job building --enable-rocm --enable-rocm-dmabuf, alongside the two CUDA jobs and triggered the same way. It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The -full and -complete tags, and an apt install of the amdrocm-core metapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen and the BLAS and FFT libraries. A compile check touches none of it. rocm/dev-ubuntu-24.04:7.2.4 is about 1.1 GB against 8.3 GB for the smallest 10.1 image, and still ships hipcc and the HIP and HSA headers needed to compile the ROCm backend.

7.2.4 rather than 10.1 because AMD publishes slim tags only through 7.2.x; every 10.x tag is -full. What this job needs is a toolchain that compiles the sources, which that image provides. Moving to a newer ROCm is a one-line change to the container tag if a slim 10.x image appears, or if something here starts depending on a newer HIP API.

This is a single host-only build; there are no HIP kernel sources in the project yet, so there is nothing to compile for a specific device target. When device code and --with-rocm-arch support are introduced this job can gain an architecture matrix at that point.

Like the CUDA jobs this builds and installs only. There is no RDMA device in a runner, and the project has no test suite.

Verified by running this workflow in a fork of the repository. The job
passed: configure accepts every option, and make and make install succeed warning-free.

---

Changes since v2:
Drop the architecture matrix; collapse to a single host-only build job. Update the commit message to match what the workflow actually does. An architecture matrix will be added when HIP kernel sources and --with-rocm-arch support are introduced.

 .github/workflows/ubuntu24_04_rocm7_2.yaml | 72 ++++++++++++++++++++++
 1 file changed, 72 insertions(+)
 create mode 100644 .github/workflows/ubuntu24_04_rocm7_2.yaml
.github/workflows/ubuntu24_04_rocm7_2.yaml new file +72 -0
new file mode 100644
index 0000000..e8f3ea5
@@ -0,0 +1,72 @@
+name: Build and Test perftest on Ubuntu 24.04 with ROCm
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    branches:
+      - master
+  workflow_dispatch:
+
+jobs:
+  build:
+    runs-on: ubuntu-24.04
+
+    # AMD's slim ROCm image. The -full and -complete tags carry the whole ML
+    # stack -- Composable Kernel alone is over 2 GB -- which a compile check
+    # never touches. This one is about 1.1 GB and still ships hipcc and the
+    # HIP and HSA headers needed to compile the ROCm backend.
+    container: rocm/dev-ubuntu-24.04:7.2.4
+
+    name: ROCm
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      # The job runs as root inside the container, so no sudo here.
+      - name: Install build dependencies
+        run: |
+          export DEBIAN_FRONTEND=noninteractive
+          apt-get update
+          apt-get install -y \
+            build-essential \
+            autoconf \
+            automake \
+            libtool \
+            pkg-config \
+            libibverbs-dev \
+            librdmacm-dev \
+            libibumad-dev \
+            libpci-dev
+
+      - name: Locate ROCm
+        run: |
+          # The image ships /opt/rocm as a symlink to a versioned prefix.
+          # configure needs the include path, so resolve it rather than
+          # assuming either spelling.
+          ROCM_PATH=$(ls -d /opt/rocm* 2>/dev/null | sort -V | tail -1)
+          if [ -z "$ROCM_PATH" ]; then
+            echo "No ROCm installation found under /opt" >&2
+            ls -la /opt || true
+            exit 1
+          fi
+          echo "Using ROCm at $ROCM_PATH"
+          "$ROCM_PATH/bin/hipcc" --version | head -2 || hipcc --version | head -2
+          echo "ROCM_PATH=$ROCM_PATH" >> $GITHUB_ENV
+          echo "$ROCM_PATH/bin" >> $GITHUB_PATH
+
+      - name: Run autogen.sh
+        run: ./autogen.sh
+
+      - name: Configure the build
+        run: |
+          ./configure --enable-rocm --enable-rocm-dmabuf \
+                      --with-rocm="$ROCM_PATH"
+
+      - name: Build perftest
+        run: make
+
+      - name: Install perftest
+        run: make install
Signature · 1 line
2.53.0
#1 lore raw
Sashiko has reviewed this patch and found no issues.
Quoted 15 lines

CI builds only the CUDA configuration. The two existing workflows pass no --enable-rocm, so src/rocm_memory.c and everything behind HAVE_ROCM and HAVE_ROCM_DMABUF is never compiled on a pull request, and a change that breaks the ROCm backend goes green.

Add an Ubuntu 24.04 job building --enable-rocm --enable-rocm-dmabuf, alongside the two CUDA jobs and triggered the same way. It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The -full and -complete tags, and an apt install of the amdrocm-core metapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen and the BLAS and FFT libraries. A compile check touches none of it. rocm/dev-ubuntu-24.04:7.2.4 is about 1.1 GB against 8.3 GB for the smallest 10.1 image, and still ships hipcc and the HIP and HSA headers needed to compile the ROCm backend.

Quoted 6 lines

[ ... ]

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#2 lore raw
In reply to Stephen Bates
Quoted 29 lines · Stephen Bates

CI builds only the CUDA configuration. The two existing workflows pass no --enable-rocm, so src/rocm_memory.c and everything behind HAVE_ROCM and HAVE_ROCM_DMABUF is never compiled on a pull request, and a change that breaks the ROCm backend goes green.

Add an Ubuntu 24.04 job building --enable-rocm --enable-rocm-dmabuf, alongside the two CUDA jobs and triggered the same way. It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The -full and -complete tags, and an apt install of the amdrocm-core metapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen and the BLAS and FFT libraries. A compile check touches none of it. rocm/dev-ubuntu-24.04:7.2.4 is about 1.1 GB against 8.3 GB for the smallest 10.1 image, and still ships hipcc and the HIP and HSA headers needed to compile the ROCm backend.

7.2.4 rather than 10.1 because AMD publishes slim tags only through 7.2.x; every 10.x tag is -full. What this job needs is a toolchain that compiles the sources, which that image provides. Moving to a newer ROCm is a one-line change to the container tag if a slim 10.x image appears, or if something here starts depending on a newer HIP API.

This is a single host-only build; there are no HIP kernel sources in the project yet, so there is nothing to compile for a specific device target. When device code and --with-rocm-arch support are introduced this job can gain an architecture matrix at that point.

Like the CUDA jobs this builds and installs only. There is no RDMA device in a runner, and the project has no test suite.

Quoted 8 lines

Verified by running this workflow in a fork of the repository. The job
passed: configure accepts every option, and make and make install succeed warning-free.

Stephen,

Thanks for bringing this to the mailing list. It brought to my attention that the perftest repository had been neglected.

Please open a PR against the perftest repository, and I'll start cleaning things up there.

Thanks

Quoted 94 lines

---

Changes since v2:
Drop the architecture matrix; collapse to a single host-only build job. Update the commit message to match what the workflow actually does. An architecture matrix will be added when HIP kernel sources and --with-rocm-arch support are introduced.

 .github/workflows/ubuntu24_04_rocm7_2.yaml | 72 ++++++++++++++++++++++
 1 file changed, 72 insertions(+)
 create mode 100644 .github/workflows/ubuntu24_04_rocm7_2.yaml
.github/workflows/ubuntu24_04_rocm7_2.yaml new file +72 -0
new file mode 100644
index 0000000..e8f3ea5
@@ -0,0 +1,72 @@
+name: Build and Test perftest on Ubuntu 24.04 with ROCm
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    branches:
+      - master
+  workflow_dispatch:
+
+jobs:
+  build:
+    runs-on: ubuntu-24.04
+
+    # AMD's slim ROCm image. The -full and -complete tags carry the whole ML
+    # stack -- Composable Kernel alone is over 2 GB -- which a compile check
+    # never touches. This one is about 1.1 GB and still ships hipcc and the
+    # HIP and HSA headers needed to compile the ROCm backend.
+    container: rocm/dev-ubuntu-24.04:7.2.4
+
+    name: ROCm
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      # The job runs as root inside the container, so no sudo here.
+      - name: Install build dependencies
+        run: |
+          export DEBIAN_FRONTEND=noninteractive
+          apt-get update
+          apt-get install -y \
+            build-essential \
+            autoconf \
+            automake \
+            libtool \
+            pkg-config \
+            libibverbs-dev \
+            librdmacm-dev \
+            libibumad-dev \
+            libpci-dev
+
+      - name: Locate ROCm
+        run: |
+          # The image ships /opt/rocm as a symlink to a versioned prefix.
+          # configure needs the include path, so resolve it rather than
+          # assuming either spelling.
+          ROCM_PATH=$(ls -d /opt/rocm* 2>/dev/null | sort -V | tail -1)
+          if [ -z "$ROCM_PATH" ]; then
+            echo "No ROCm installation found under /opt" >&2
+            ls -la /opt || true
+            exit 1
+          fi
+          echo "Using ROCm at $ROCM_PATH"
+          "$ROCM_PATH/bin/hipcc" --version | head -2 || hipcc --version | head -2
+          echo "ROCM_PATH=$ROCM_PATH" >> $GITHUB_ENV
+          echo "$ROCM_PATH/bin" >> $GITHUB_PATH
+
+      - name: Run autogen.sh
+        run: ./autogen.sh
+
+      - name: Configure the build
+        run: |
+          ./configure --enable-rocm --enable-rocm-dmabuf \
+                      --with-rocm="$ROCM_PATH"
+
+      - name: Build perftest
+        run: make
+
+      - name: Install perftest
+        run: make install
Signature · 1 line
2.53.0
#3 lore raw
Quoted 4 lines · Leon Romanovsky

On Fri, Nov 06, 2026 at 04:48:21PM -0600, Stephen Bates wrote:

Stephen,

Quoted 8 lines

Thanks for bringing this to the mailing list. It brought to my attention that the perftest repository had been neglected.

Please open a PR against the perftest repository, and I'll start cleaning things up there.

Thanks

Leon

Thanks. The PR can be found here:

https://github.com/linux-rdma/perftest/pull/419

Stephen