Skip to contentSkip to conversation

perftest: add a ROCm build to CI

PATCH Superseded Stephen Bates 2 participants 2 messages Nov 4 InfiniBand

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 perftest: add a ROCm build to CI

CI builds only the CUDA configuration. The two existing workflows pass no --enable-rocm, so src/rocm_memory.c and everything behind HAVE_ROCM and HAVE_ROCM_DMABUF is never compiled on a pull request, and a change that breaks the ROCm backend goes green.

Add an Ubuntu 24.04 job building --enable-rocm --enable-rocm-dmabuf, alongside the two CUDA jobs and triggered the same way, as a matrix over gfx950 (Instinct MI355X, CDNA4) and gfx1250 (Instinct MI455X, CDNA5). Those two were picked to cover consecutive Instinct generations: HIP sources that compile correctly for one generation may have microarchitecture-specific issues on the next, and testing both catches that. fail-fast is off: a failure is far more likely to be specific to one generation than common to both, and cancelling the other leg hides exactly the comparison that would say so.

It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The -full and -complete tags, and an apt install of the amdrocm-core metapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen and the BLAS and FFT libraries. A compile check touches none of it. rocm/dev-ubuntu-24.04:7.2.4 is about 1.1 GB against 8.3 GB for the smallest 10.1 image, and still ships hipcc, the HIP and HSA headers, and device support for both matrix targets -- llc -mcpu=help lists gfx950 and gfx1250, and both compile.

7.2.4 rather than 10.1 because AMD publishes slim tags only through 7.2.x; every 10.x tag is -full. What this job needs is a toolchain that compiles the sources for the targets in the matrix, which that image provides. Moving to a newer ROCm is a one-line change to the container tag if a slim 10.x image appears, or if something here starts depending on a newer HIP API.

Like the CUDA jobs this builds and installs only. There is no RDMA device in a runner, and the project has no test suite.

Verified by running this workflow in a fork of the repository across both matrix architectures. Both legs passed: configure accepts every option, and make and make install succeed warning-free.

---

Changes since v1:
- Add a comment on the matrix explaining that rocm_arch is not yet passed to configure; --with-rocm-arch only exists once HIP kernel sources are added.

 .github/workflows/ubuntu24_04_rocm7_2.yaml | 70 ++++++++++++++++++++++
 1 file changed, 70 insertions(+)
 create mode 100644 .github/workflows/ubuntu24_04_rocm7_2.yaml
.github/workflows/ubuntu24_04_rocm7_2.yaml new file +70 -0
new file mode 100644
index 0000000..7079037
@@ -0,0 +1,70 @@
+name: Build and Test perftest on Ubuntu 24.04 with ROCm
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    branches:
+      - master
+  workflow_dispatch:
+
+jobs:
+  build:
+    runs-on: ubuntu-24.04
+
+    # AMD's slim ROCm image. The -full and -complete tags carry the whole ML
+    # stack -- Composable Kernel alone is over 2 GB -- which a compile check
+    # never touches. This one is about 1.1 GB and still ships hipcc, the HIP
+    # and HSA headers, and device support for every target below.
+    container: rocm/dev-ubuntu-24.04:7.2.4
+
+    strategy:
+      fail-fast: false
+      matrix:
+        # rocm_arch is not yet passed to configure: --with-rocm-arch only
+        # exists once HIP kernel sources are added. Both legs build identically
+        # until then; the matrix is in place so the workflow does not need
+        # amending when device code arrives.
+        rocm_arch: [gfx950, gfx1250]
+
+    name: ROCm (${{ matrix.rocm_arch }})
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      # The job runs as root inside the container, so no sudo here.
+      - name: Install build dependencies
+        run: |
+          export DEBIAN_FRONTEND=noninteractive
+          apt-get update
+          apt-get install -y \
+            build-essential \
+            autoconf \
+            automake \
+            libtool \
+            pkg-config \
+            libibverbs-dev \
+            librdmacm-dev \
+            libibumad-dev \
+            libpci-dev
+
+      - name: Set up ROCm environment
+        run: |
+          echo "ROCM_PATH=/opt/rocm" >> $GITHUB_ENV
+          echo "/opt/rocm/bin" >> $GITHUB_PATH
+
+      - name: Run autogen.sh
+        run: ./autogen.sh
+
+      - name: Configure the build
+        run: |
+          ./configure --enable-rocm --enable-rocm-dmabuf \
+                      --with-rocm="$ROCM_PATH"
+
+      - name: Build perftest
+        run: make
+
+      - name: Install perftest
+        run: make install
Signature · 1 line
2.53.0
#1 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] perftest: unused matrix.rocm_arch in CI workflow and commit message mismatch --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perftest: unused matrix.rocm_arch in CI workflow and commit message mismatch --

commit f80ce07d3d5d2a3f4581f598c3afcf67e5214052
Author: Stephen Bates <[email protected]>

perftest: add a ROCm build to CI

This commit adds a GitHub Actions workflow on Ubuntu 24.04 to build and install perftest with ROCm and ROCm DMA-BUF support enabled inside AMD's slim ROCm container image.

Quoted 47 lines

perftest: add a ROCm build to CI

CI builds only the CUDA configuration. The two existing workflows pass no --enable-rocm, so src/rocm_memory.c and everything behind HAVE_ROCM and HAVE_ROCM_DMABUF is never compiled on a pull request, and a change that breaks the ROCm backend goes green.

Add an Ubuntu 24.04 job building --enable-rocm --enable-rocm-dmabuf, alongside the two CUDA jobs and triggered the same way, as a matrix over gfx950 (Instinct MI355X, CDNA4) and gfx1250 (Instinct MI455X, CDNA5). Those two were picked to cover consecutive Instinct generations: HIP sources that compile correctly for one generation may have microarchitecture-specific issues on the next, and testing both catches that. fail-fast is off: a failure is far more likely to be specific to one generation than common to both, and cancelling the other leg hides exactly the comparison that would say so.

It runs in AMD's slim ROCm image rather than installing ROCm on the runner. The -full and -complete tags, and an apt install of the amdrocm-core metapackages, all pull the whole ML stack: Composable Kernel alone is over 2 GB, before rocSHMEM, MIOpen and the BLAS and FFT libraries. A compile check touches none of it. rocm/dev-ubuntu-24.04:7.2.4 is about 1.1 GB against 8.3 GB for the smallest 10.1 image, and still ships hipcc, the HIP and HSA headers, and device support for both matrix targets -- llc -mcpu=help lists gfx950 and gfx1250, and both compile.

7.2.4 rather than 10.1 because AMD publishes slim tags only through 7.2.x; every 10.x tag is -full. What this job needs is a toolchain that compiles the sources for the targets in the matrix, which that image provides. Moving to a newer ROCm is a one-line change to the container tag if a slim 10.x image appears, or if something here starts depending on a newer HIP API.

Like the CUDA jobs this builds and installs only. There is no RDMA device in a runner, and the project has no test suite.

Verified by running this workflow in a fork of the repository across both matrix architectures. Both legs passed: configure accepts every option, and make and make install succeed warning-free.

.github/workflows/ubuntu24_04_rocm7_2.yaml new file +0 -0
new file mode 100644
index 0000000000000..70790377d85a0
@@ -0,0 +1,70 @@

[ ... ]

Quoted 10 lines
.github/workflows/ubuntu24_04_rocm7_2.yaml new file +10 -0
+    strategy:
+      fail-fast: false
+      matrix:
+        # rocm_arch is not yet passed to configure: --with-rocm-arch only
+        # exists once HIP kernel sources are added. Both legs build identically
+        # until then; the matrix is in place so the workflow does not need
+        # amending when device code arrives.
+        rocm_arch: [gfx950, gfx1250]
+
+    name: ROCm (${{ matrix.rocm_arch }})

[ ... ]

.github/workflows/ubuntu24_04_rocm7_2.yaml line 64

Quoted 4 lines
.github/workflows/ubuntu24_04_rocm7_2.yaml new file +4 -0
+      - name: Configure the build
+        run: |
+          ./configure --enable-rocm --enable-rocm-dmabuf \
+                      --with-rocm="$ROCM_PATH"

[Severity: Medium]
As reported by Sashiko on v1 [1], in .github/workflows/ubuntu24_04_rocm7_2.yaml, jobs.build.strategy.matrix defines rocm_arch: [gfx950, gfx1250], but matrix.rocm_arch is only interpolated into the job display name and is never passed to the environment, ./configure, or make in jobs.build.steps.

The new comment in jobs.build.strategy.matrix notes:

        # rocm_arch is not yet passed to configure: --with-rocm-arch only
        # exists once HIP kernel sources are added. Both legs build identically
        # until then; the matrix is in place so the workflow does not need
        # amending when device code arrives.

However, because the "Configure the build" step in jobs.build.steps invokes:

          ./configure --enable-rocm --enable-rocm-dmabuf \
                      --with-rocm="$ROCM_PATH"

without passing matrix.rocm_arch, won't this workflow file still need to be amended anyway when --with-rocm-arch and device code arrive? In the meantime, every push and pull request against master pulls the container image twice, installs all packages twice, and runs two identical host-only builds.

In addition, does the commit message still match what this workflow actually does? The commit message states that the job builds a matrix over gfx950 and gfx1250 to catch microarchitecture-specific HIP compilation issues across consecutive Instinct generations, that fail-fast is off because a failure is likely specific to one generation, and that both targets compile in this job, whereas the workflow currently compiles neither target architecture.

Could the workflow run a single build job until HIP kernel sources and --with-rocm-arch support are introduced, and could the commit message be updated to match?

[1] https://lore.kernel.org/r/[email protected]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1