Skip to contentSkip to conversation

Checking and CI improvements

PATCH In review Nicholas Piggin 3 participants 10 messages Jul 26, 2024 – Aug 12, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Checking and CI improvements

Here's some assorted fixes and improvements to static checking and CI.

checkpatch support is a big one and you could consider it RFC for now. We may want to customise it somewhat (but IMO fewer customisations the easier to maintain the script if we can agree on close to kernel style). It's nice because it gets DCO, license headers, etc., not just code style.

Here is a pipeline with this series applied:

https://gitlab.com/npiggin/kvm-unit-tests/-/pipelines/1388888286

Thanks,
Nick

Nicholas Piggin (6):
gitlab-ci: fix CentOS mirror list
arm: Fix kerneldoc
gitlab-ci: upgrade to CentOS 8
gitlab-ci: Move check-kerneldoc test out of the centos test
gitlab-ci: add a shellcheck test
checkpatch support

 .gitlab-ci.yml         |   83 +-
 arm/fpu.c              |   24 +-
 scripts/check-patch.py |   61 +
 scripts/checkpatch.pl  | 7839 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 7986 insertions(+), 21 deletions(-)
 create mode 100755 scripts/check-patch.py
 create mode 100755 scripts/checkpatch.pl
Signature · 1 line
2.45.2
#1 lore raw

PATCH 1/6 gitlab-ci: fix CentOS mirror list

The CentOS repository URLs have changed, so the centos7 job no longer runs. Apply a recipe from the internet that gets it going again.

---
 .gitlab-ci.yml | 4 ++++
 1 file changed, 4 insertions(+)
.gitlab-ci.yml +4 -0
index e0eb85a94..823f03c3e 100644
@@ -300,6 +300,10 @@ build-centos7:
  extends: .outoftree_template
  image: centos:7
  before_script:
+# CentOS mirrors have changed, these sed scripts fixes the repos.
+ - sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo
+ - sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo
+ - sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
  - yum update -y
  - yum install -y make python qemu-kvm gcc
  script:
Signature · 1 line
2.45.2
#2 lore raw

PATCH 2/6 arm: Fix kerneldoc

In reply to Nicholas Piggin

Some invalid kerneldoc comments crept in while centos ci job was down.

Cc Subhasish Ghosh <[email protected]>
Cc Joey Gouly <[email protected]>
Cc Suzuki K Poulose <[email protected]>
Fixes d47d370c8f ("arm: Add test for FPU/SIMD context save/restore")
---
 arm/fpu.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)
arm/fpu.c +12 -12
index 39413fc3e..edbd9a946 100644
@@ -212,8 +212,8 @@ static void nr_cpu_check(int nr)
 		report_abort("At least %d cpus required", nr);
 }
 
-/**
- * @brief check if the FPU/SIMD/SVE register contents are the same as
+/*
+ * check if the FPU/SIMD/SVE register contents are the same as
  * the input data provided.
  */
 static uint32_t __fpuregs_testall(uint64_t *indata, int sve)
@@ -256,8 +256,8 @@ static uint32_t __fpuregs_testall(uint64_t *indata, int sve)
 	return result;
 }
 
-/**
- * @brief writes randomly sampled data into the FPU/SIMD registers.
+/*
+ * Write randomly sampled data into the FPU/SIMD registers.
  */
 static void __fpuregs_writeall_random(uint64_t **indata, int sve)
 {
@@ -315,9 +315,9 @@ static void sveregs_testall_run(void *data)
 	       "SVE register save/restore mask: 0x%x", result);
 }
 
-/**
- * @brief This test uses two CPUs to test FPU/SIMD save/restore
- * @details CPU1 writes random data into FPU/SIMD registers,
+/*
+ * This test uses two CPUs to test FPU/SIMD save/restore
+ * CPU1 writes random data into FPU/SIMD registers,
  * CPU0 corrupts/overwrites the data and finally CPU1 checks
  * if the data remains unchanged in its context.
  */
@@ -344,9 +344,9 @@ static void fpuregs_context_switch_cpu1(int sve)
 	free(indata_local);
 }
 
-/**
- * @brief This test uses two CPUs to test FPU/SIMD save/restore
- * @details CPU0 writes random data into FPU/SIMD registers,
+/*
+ * This test uses two CPUs to test FPU/SIMD save/restore
+ * CPU0 writes random data into FPU/SIMD registers,
  * CPU1 corrupts/overwrites the data and finally CPU0 checks if
  * the data remains unchanged in its context.
  */
@@ -374,7 +374,7 @@ static void fpuregs_context_switch_cpu0(int sve)
 	free(indata_local);
 }
 
-/**
+/*
  * Checks if during context switch, FPU/SIMD registers
  * are saved/restored.
  */
@@ -384,7 +384,7 @@ static void fpuregs_context_switch(void)
 	fpuregs_context_switch_cpu1(0);
 }
 
-/**
+/*
  * Checks if during context switch, SVE registers
  * are saved/restored.
  */
Signature · 1 line
2.45.2
Reviewed-by on v1 2/6
#3 lore raw

Re: [kvm-unit-tests PATCH 2/6] arm: Fix kerneldoc

Quoted 7 lines · Nicholas Piggin

Some invalid kerneldoc comments crept in while centos ci job was down.

Cc Subhasish Ghosh <[email protected]>
Cc Joey Gouly <[email protected]>
Cc Suzuki K Poulose <[email protected]>
Fixes d47d370c8f ("arm: Add test for FPU/SIMD context save/restore")

Thanks for the fix !

Quoted 74 lines
---
  arm/fpu.c | 24 ++++++++++++------------
  1 file changed, 12 insertions(+), 12 deletions(-)
arm/fpu.c +12 -12
index 39413fc3e..edbd9a946 100644
@@ -212,8 +212,8 @@ static void nr_cpu_check(int nr)
  		report_abort("At least %d cpus required", nr);
  }
 
-/**
- * @brief check if the FPU/SIMD/SVE register contents are the same as
+/*
+ * check if the FPU/SIMD/SVE register contents are the same as
   * the input data provided.
   */
  static uint32_t __fpuregs_testall(uint64_t *indata, int sve)
@@ -256,8 +256,8 @@ static uint32_t __fpuregs_testall(uint64_t *indata, int sve)
  	return result;
  }
 
-/**
- * @brief writes randomly sampled data into the FPU/SIMD registers.
+/*
+ * Write randomly sampled data into the FPU/SIMD registers.
   */
  static void __fpuregs_writeall_random(uint64_t **indata, int sve)
  {
@@ -315,9 +315,9 @@ static void sveregs_testall_run(void *data)
  	       "SVE register save/restore mask: 0x%x", result);
  }
 
-/**
- * @brief This test uses two CPUs to test FPU/SIMD save/restore
- * @details CPU1 writes random data into FPU/SIMD registers,
+/*
+ * This test uses two CPUs to test FPU/SIMD save/restore
+ * CPU1 writes random data into FPU/SIMD registers,
   * CPU0 corrupts/overwrites the data and finally CPU1 checks
   * if the data remains unchanged in its context.
   */
@@ -344,9 +344,9 @@ static void fpuregs_context_switch_cpu1(int sve)
  	free(indata_local);
  }
 
-/**
- * @brief This test uses two CPUs to test FPU/SIMD save/restore
- * @details CPU0 writes random data into FPU/SIMD registers,
+/*
+ * This test uses two CPUs to test FPU/SIMD save/restore
+ * CPU0 writes random data into FPU/SIMD registers,
   * CPU1 corrupts/overwrites the data and finally CPU0 checks if
   * the data remains unchanged in its context.
   */
@@ -374,7 +374,7 @@ static void fpuregs_context_switch_cpu0(int sve)
  	free(indata_local);
  }
 
-/**
+/*
   * Checks if during context switch, FPU/SIMD registers
   * are saved/restored.
   */
@@ -384,7 +384,7 @@ static void fpuregs_context_switch(void)
  	fpuregs_context_switch_cpu1(0);
  }
 
-/**
+/*
   * Checks if during context switch, SVE registers
   * are saved/restored.
   */
#4 lore raw

PATCH 3/6 gitlab-ci: upgrade to CentOS 8

In reply to Nicholas Piggin

CentOS 7 reached EOL at the end of June 2024. Upgrade to CentOS 8. The mirror URL adjustment script still seems to be required.

This brings across some of the x86-64 tests that had been enabled on the fedora build as well.

---
 .gitlab-ci.yml | 40 +++++++++++++++++++++++++++++++++-------
 1 file changed, 33 insertions(+), 7 deletions(-)
.gitlab-ci.yml +33 -7
index 823f03c3e..2d048b11d 100644
@@ -296,26 +296,52 @@ build-clang:
       | tee results.txt
  - grep -q PASS results.txt && ! grep -q FAIL results.txt
 
-build-centos7:
+build-centos8:
  extends: .outoftree_template
- image: centos:7
+ image: centos:8
  before_script:
 # CentOS mirrors have changed, these sed scripts fixes the repos.
  - sed -i s/mirror.centos.org/vault.centos.org/g /etc/yum.repos.d/*.repo
  - sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo
  - sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
  - yum update -y
- - yum install -y make python qemu-kvm gcc
+ - yum install -y make python39 qemu-kvm gcc
  script:
  - mkdir build
  - cd build
  - ../configure --arch=x86_64 --disable-pretty-print-stacks
  - make -j2
  - ACCEL=tcg ./run_tests.sh
-     vmexit_cpuid vmexit_mov_from_cr8 vmexit_mov_to_cr8 vmexit_inl_pmtimer
-     vmexit_ple_round_robin vmexit_tscdeadline vmexit_tscdeadline_immed port80
-     setjmp sieve tsc rmap_chain umip
-     | tee results.txt
+      eventinj
+      intel_iommu
+      ioapic-split
+      memory
+      pks
+      pku
+      port80
+      rdpru
+      rmap_chain
+      setjmp
+      sieve
+      smap
+      smptest
+      smptest3
+      syscall
+      tsc
+      umip
+      vmexit_cpuid
+      vmexit_cr0_wp
+      vmexit_cr4_pge
+      vmexit_inl_pmtimer
+      vmexit_ipi
+      vmexit_ipi_halt
+      vmexit_mov_from_cr8
+      vmexit_mov_to_cr8
+      vmexit_ple_round_robin
+      vmexit_tscdeadline
+      vmexit_tscdeadline_immed
+      | tee results.txt
+ - if grep -q FAIL results.txt ; then exit 1 ; fi
  - grep -q PASS results.txt && ! grep -q FAIL results.txt
  - make -s check-kerneldoc 2>&1 | tee docwarnings.txt
  - test -z `cat docwarnings.txt`
Signature · 1 line
2.45.2
#5 lore raw

PATCH 4/6 gitlab-ci: Move check-kerneldoc test out of the centos test

In reply to Nicholas Piggin

This creates a new lightweight (alpine) kerneldoc check, based on the QEMU static checking jobs.

The aim is to decouple the test build and run jobs on different platforms from the static checker images, so one can be adjusted without affecting the other.

---
 .gitlab-ci.yml | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
.gitlab-ci.yml +14 -3
index 2d048b11d..0f72cefe7 100644
@@ -305,7 +305,7 @@ build-centos8:
  - sed -i s/^#.*baseurl=http/baseurl=http/g /etc/yum.repos.d/*.repo
  - sed -i s/^mirrorlist=http/#mirrorlist=http/g /etc/yum.repos.d/*.repo
  - yum update -y
- - yum install -y make python39 qemu-kvm gcc
+ - yum install -y make qemu-kvm gcc
  script:
  - mkdir build
  - cd build
@@ -343,8 +343,6 @@ build-centos8:
       | tee results.txt
  - if grep -q FAIL results.txt ; then exit 1 ; fi
  - grep -q PASS results.txt && ! grep -q FAIL results.txt
- - make -s check-kerneldoc 2>&1 | tee docwarnings.txt
- - test -z `cat docwarnings.txt`
 
 # Cirrus-CI provides containers with macOS and Linux with KVM enabled,
 # so we can test some scenarios there that are not possible with the
@@ -418,3 +416,16 @@ s390x-kvm:
  only:
   variables:
    - $S390X_Z15_RUNNER_AVAILABLE
+
+check-kerneldoc:
+  extends: .intree_template
+  stage: build
+  image: python:3.10-alpine
+  needs: []
+  script:
+    - ./configure --arch=x86_64
+    - make -s check-kerneldoc 2>&1 | tee docwarnings.txt
+    - test -z `cat docwarnings.txt`
+  before_script:
+    - apk -U add bash make perl gcc
+  allow_failure: true
Signature · 1 line
2.45.2
#6 lore raw

PATCH 5/6 gitlab-ci: add a shellcheck test

In reply to Nicholas Piggin

Now that we have some shellcheck support, might as well run it in CI.

---
 .gitlab-ci.yml | 13 +++++++++++++
 1 file changed, 13 insertions(+)
.gitlab-ci.yml +13 -0
index 0f72cefe7..2d55c6dcf 100644
@@ -429,3 +429,16 @@ check-kerneldoc:
   before_script:
     - apk -U add bash make perl gcc
   allow_failure: true
+
+check-shellcheck:
+  extends: .intree_template
+  stage: build
+  image: python:3.10-alpine
+  needs: []
+  script:
+    - ./configure --arch=x86_64
+    - make -s shellcheck 2>&1 | tee shellcheckwarnings.txt
+    - test -z `cat shellcheckwarnings.txt`
+  before_script:
+    - apk -U add bash make gcc shellcheck
+  allow_failure: true
Signature · 1 line
2.45.2
#7 lore raw

PATCH 6/6 checkpatch support

In reply to Nicholas Piggin

This brings checkpatch.pl almost unchanged from linux.git, and adds it to gitlab-ci using QEMU's check-patch.py script.

For now, warnings are not treated as errors until we get a feeling for what warnings are useful.

---
 .gitlab-ci.yml         |   11 +
 scripts/check-patch.py |   61 +
 scripts/checkpatch.pl  | 7839 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 7911 insertions(+)
 create mode 100755 scripts/check-patch.py
 create mode 100755 scripts/checkpatch.pl
.gitlab-ci.yml +11 -0
index 2d55c6dcf..cae3e7484 100644
@@ -442,3 +442,14 @@ check-shellcheck:
   before_script:
     - apk -U add bash make gcc shellcheck
   allow_failure: true
+
+check-patch:
+  extends: .intree_template
+  stage: build
+  image: python:3.10-alpine
+  needs: []
+  script:
+    - scripts/check-patch.py
+  before_script:
+    - apk -U add git perl
+  allow_failure: true
scripts/check-patch.py new file +61 -0

61 lines not on this page · raw message

scripts/checkpatch.pl new file +7839 -0

2,925 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.45.2

Part of the diff is not shown here. View the raw message.

Acked-by on the v1 series
#8 lore raw
In reply to Nicholas Piggin
Quoted 24 lines · Nicholas Piggin

Here's some assorted fixes and improvements to static checking and CI.

checkpatch support is a big one and you could consider it RFC for now. We may want to customise it somewhat (but IMO fewer customisations the easier to maintain the script if we can agree on close to kernel style). It's nice because it gets DCO, license headers, etc., not just code style.

Here is a pipeline with this series applied:

https://gitlab.com/npiggin/kvm-unit-tests/-/pipelines/1388888286

Thanks,
Nick

Nicholas Piggin (6):
gitlab-ci: fix CentOS mirror list
arm: Fix kerneldoc
gitlab-ci: upgrade to CentOS 8
gitlab-ci: Move check-kerneldoc test out of the centos test
gitlab-ci: add a shellcheck test
checkpatch support

Quoted 11 lines
 .gitlab-ci.yml         |   83 +-
 arm/fpu.c              |   24 +-
 scripts/check-patch.py |   61 +
 scripts/checkpatch.pl  | 7839 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 7986 insertions(+), 21 deletions(-)
 create mode 100755 scripts/check-patch.py
 create mode 100755 scripts/checkpatch.pl
Signature · 1 line
2.45.2

For the series,

Thanks,
drew

#9 lore raw
In reply to Nicholas Piggin
Quoted 24 lines · Nicholas Piggin

Here's some assorted fixes and improvements to static checking and CI.

checkpatch support is a big one and you could consider it RFC for now. We may want to customise it somewhat (but IMO fewer customisations the easier to maintain the script if we can agree on close to kernel style). It's nice because it gets DCO, license headers, etc., not just code style.

Here is a pipeline with this series applied:

https://gitlab.com/npiggin/kvm-unit-tests/-/pipelines/1388888286

Thanks,
Nick

Nicholas Piggin (6):
gitlab-ci: fix CentOS mirror list
arm: Fix kerneldoc
gitlab-ci: upgrade to CentOS 8
gitlab-ci: Move check-kerneldoc test out of the centos test
gitlab-ci: add a shellcheck test
checkpatch support

Quoted 11 lines
 .gitlab-ci.yml         |   83 +-
 arm/fpu.c              |   24 +-
 scripts/check-patch.py |   61 +
 scripts/checkpatch.pl  | 7839 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 7986 insertions(+), 21 deletions(-)
 create mode 100755 scripts/check-patch.py
 create mode 100755 scripts/checkpatch.pl
Signature · 1 line
2.45.2

Merged

Thanks,
drew