Skip to contentSkip to conversation

Add new build target 'check-spelling'

RFC PATCH In review Stefan Weil via 4 participants 7 messages Oct 31, 2022

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 Add new build target 'check-spelling'

make check-spelling can now be used to get a list of spelling errors. It uses the latest version of codespell, a spell checker implemented in Python.

---

This RFC can already be used for manual tests, but still reports false positives, mostly because some variable names are interpreted as words. These words can either be ignored in the check, or in some cases the code might be changed to use different variable names.

The check currently only skips a few directories and files, so for example checked out submodules are also checked.

The rule can be extended to allow user provided ignore and skip lists, for example by introducing Makefile variables CODESPELL_SKIP=userfile or CODESPELL_IGNORE=userfile. A limited check could be implemented by providing a base directory CODESPELL_START=basedirectory, for example CODESPELL_START=docs.

Regards,
Stefan

 tests/Makefile.include       | 10 ++++++++++
 tests/codespell/README.rst   | 18 ++++++++++++++++++
 tests/codespell/exclude-file |  3 +++
 tests/codespell/ignore-words | 19 +++++++++++++++++++
 tests/requirements.txt       |  1 +
 5 files changed, 51 insertions(+)
 create mode 100644 tests/codespell/README.rst
 create mode 100644 tests/codespell/exclude-file
 create mode 100644 tests/codespell/ignore-words
tests/Makefile.include +10 -0
index 9422ddaece..b9daeda932 100644
@@ -155,6 +155,16 @@ check-acceptance-deprecated-warning:
 
 check-acceptance: check-acceptance-deprecated-warning | check-avocado
 
+.PHONY: check-spelling
+CODESPELL_DIR=tests/codespell
+check-spelling: check-venv
+	source $(TESTS_VENV_DIR)/bin/activate && \
+	cd "$(SRC_PATH)" && \
+	codespell -s . \
+	  --exclude-file=$(CODESPELL_DIR)/exclude-file \
+	  --ignore-words=$(CODESPELL_DIR)/ignore-words \
+	  --skip="./.git,./bin,./build,./linux-headers,*.patch,nohup.out"
+
 # Consolidated targets
 
 .PHONY: check check-clean get-vm-images
tests/codespell/README.rst new file +18 -0
new file mode 100644
index 0000000000..67e070d631
@@ -0,0 +1,18 @@
+=============================
+Check spelling with codespell
+=============================
+
+`make check-spelling` can be used to get a list of spelling errors.
+It reports files with spelling errors and a summary of all misspelled words.
+The report is generated by the latest version of codespell, a spell checker
+implemented in Python.
+
+See https://github.com/codespell-project/codespell for more information.
+
+Some file patterns are excluded from the check.
+
+In addition tests/codespell includes several files which are used to
+suppress certain false positives in the codespell report.
+
+exclude-file - complete lines which should be ignored
+ignore-words - list of words which should be ignored
tests/codespell/exclude-file new file +3 -0
new file mode 100644
index 0000000000..57de81a4eb
@@ -0,0 +1,3 @@
+ * VAS controller.
+number generator daemon such as the one found in the vhost-device crate of
+introspection.  The latter can conceivably confuse clients, so tread
tests/codespell/ignore-words new file +19 -0
new file mode 100644
index 0000000000..4d336a2f44
@@ -0,0 +1,19 @@
+buid
+busses
+dout
+falt
+fpr
+hace
+hax
+hda
+nd
+ot
+pard
+parm
+ptd
+ser
+som
+synopsys
+te
+toke
+ue
tests/requirements.txt +1 -0
index 0ba561b6bd..dd44e6768f 100644
@@ -4,3 +4,4 @@
 # Note that qemu.git/python/ is always implicitly installed.
 avocado-framework==88.1
 pycdlib==1.11.0
+codespell
Signature · 1 line
2.30.2
#1 lore raw
Quoted 40 lines · Stefan Weil

make check-spelling can now be used to get a list of spelling errors. It uses the latest version of codespell, a spell checker implemented in Python.

---

This RFC can already be used for manual tests, but still reports false positives, mostly because some variable names are interpreted as words. These words can either be ignored in the check, or in some cases the code might be changed to use different variable names.

The check currently only skips a few directories and files, so for example checked out submodules are also checked.

The rule can be extended to allow user provided ignore and skip lists, for example by introducing Makefile variables CODESPELL_SKIP=userfile or CODESPELL_IGNORE=userfile. A limited check could be implemented by providing a base directory CODESPELL_START=basedirectory, for example CODESPELL_START=docs.

Regards,
Stefan

  tests/Makefile.include       | 10 ++++++++++
  tests/codespell/README.rst   | 18 ++++++++++++++++++
  tests/codespell/exclude-file |  3 +++
  tests/codespell/ignore-words | 19 +++++++++++++++++++
  tests/requirements.txt       |  1 +
  5 files changed, 51 insertions(+)

create mode 100644 tests/codespell/README.rst
create mode 100644 tests/codespell/exclude-file
create mode 100644 tests/codespell/ignore-words

tests/Makefile.include +0 -0
index 9422ddaece..b9daeda932 100644
@@ -155,6 +155,16 @@ check-acceptance-deprecated-warning:
 
  check-acceptance: check-acceptance-deprecated-warning | check-avocado

tests/Makefile.include check-acceptance-deprecated-warning: · line 166

Quoted 10 lines
tests/Makefile.include +9 -0
 
+.PHONY: check-spelling
+CODESPELL_DIR=tests/codespell
+check-spelling: check-venv
+	source $(TESTS_VENV_DIR)/bin/activate && \
+	cd "$(SRC_PATH)" && \
+	codespell -s . \
+	  --exclude-file=$(CODESPELL_DIR)/exclude-file \
+	  --ignore-words=$(CODESPELL_DIR)/ignore-words \
+	  --skip="./.git,./bin,./build,./linux-headers,*.patch,nohup.out"

I like the idea, but I think it's unlikely that we can make this work for the whole source tree any time soon. So maybe it makes more sense to start with some few directories first (e.g. docs/ ) and then the maintainers can opt-in by cleaning up their directories first and then by adding their directories to this target here?

Thomas

#2 lore raw

Am 31.10.22 um 08:52 schrieb Thomas Huth:

Quoted 23 lines · Stefan Weil
Quoted 22 lines

make check-spelling can now be used to get a list of spelling errors. It uses the latest version of codespell, a spell checker implemented in Python.

---

This RFC can already be used for manual tests, but still reports false positives, mostly because some variable names are interpreted as words. These words can either be ignored in the check, or in some cases the code might be changed to use different variable names.

The check currently only skips a few directories and files, so for example checked out submodules are also checked.

The rule can be extended to allow user provided ignore and skip lists, for example by introducing Makefile variables CODESPELL_SKIP=userfile or CODESPELL_IGNORE=userfile. A limited check could be implemented by providing a base directory CODESPELL_START=basedirectory, for example CODESPELL_START=docs.

Regards,
Stefan

[...]

Quoted 3 lines
Quoted 1 line

I like the idea, but I think it's unlikely that we can make this work for the whole source tree any time soon. So maybe it makes more sense to start with some few directories first (e.g. docs/ ) and then the maintainers can opt-in by cleaning up their directories first and then by adding their directories to this target here?

 Thomas

Even without implementing CODESPELL_START as described above, the script can already be used and integrated into CI scripts.

It takes about 60 seconds to check the whole source tree including submodules on my (slow) virtual machine.

The resulting output has about 20000 lines or 1272 KiB. It can be filtered for relevant parts of the source tree or used for a summary.

Sample script: grep "^[.]" spellcheck.log | sed s/^..// | sed 's/\/.*//' | sed s/:.*// | sort | uniq -c

This produces a summary for the top level hierarchy of files and directories:

      3 accel
      1 audio
      1 backends
     77 block
      7 block.c
     20 bsd-user
    386 capstone
     12 chardev
      1 configure
      8 contrib
      6 crypto
     64 disas
     32 docs
     31 dtc
      8 fpu
      1 gdbstub
      1 gdb-xml
      1 .github
    537 hw
      7 inc
    114 include
      1 libdecnumber
     33 linux-user
      1 MAINTAINERS
    150 meson
      6 meson.build
     16 migration
      1 nbd
      5 net
     12 pc-bios
      7 python
      3 qapi
      2 qemu
      5 qemu-options.hx
     22 qga
  14175 roms
     43 scripts
      3 semihosting
     18 slirp
      2 softmmu
     59 subprojects
    504 target
      6 tcg
      3 test.rb
    175 tests
      6 tools
     20 ui
      8 util

It shows that "roms" contributes by far the most typos. Omitting it would reduce the required time to 22 seconds and the number of typos found (2947 lines in output) very much.

"capstone" (which has no entry in MAINTAINERS), "target" and "hw" also contribute more than 300 hits each, therefore cc'ing Richard.

Stefan

#3 lore raw
Quoted 90 lines · Stefan Weil

Am 31.10.22 um 08:52 schrieb Thomas Huth:

Quoted 23 lines · Stefan Weil
Quoted 22 lines

make check-spelling can now be used to get a list of spelling errors. It uses the latest version of codespell, a spell checker implemented in Python.

---

This RFC can already be used for manual tests, but still reports false positives, mostly because some variable names are interpreted as words. These words can either be ignored in the check, or in some cases the code might be changed to use different variable names.

The check currently only skips a few directories and files, so for example checked out submodules are also checked.

The rule can be extended to allow user provided ignore and skip lists, for example by introducing Makefile variables CODESPELL_SKIP=userfile or CODESPELL_IGNORE=userfile. A limited check could be implemented by providing a base directory CODESPELL_START=basedirectory, for example CODESPELL_START=docs.

Regards,
Stefan

[...]

Quoted 3 lines
Quoted 1 line

I like the idea, but I think it's unlikely that we can make this work for the whole source tree any time soon. So maybe it makes more sense to start with some few directories first (e.g. docs/ ) and then the maintainers can opt-in by cleaning up their directories first and then by adding their directories to this target here?

 Thomas

Even without implementing CODESPELL_START as described above, the script can already be used and integrated into CI scripts.

It takes about 60 seconds to check the whole source tree including submodules on my (slow) virtual machine.

The resulting output has about 20000 lines or 1272 KiB. It can be filtered for relevant parts of the source tree or used for a summary.

Sample script: grep "^[.]" spellcheck.log | sed s/^..// | sed 's/\/.*//' | sed s/:.*// | sort | uniq -c

This produces a summary for the top level hierarchy of files and directories:

      3 accel
      1 audio
      1 backends
     77 block
      7 block.c
     20 bsd-user
    386 capstone
     12 chardev
      1 configure
      8 contrib
      6 crypto
     64 disas
     32 docs
     31 dtc
      8 fpu
      1 gdbstub
      1 gdb-xml
      1 .github
    537 hw
      7 inc
    114 include
      1 libdecnumber
     33 linux-user
      1 MAINTAINERS
    150 meson
      6 meson.build
     16 migration
      1 nbd
      5 net
     12 pc-bios
      7 python
      3 qapi
      2 qemu
      5 qemu-options.hx
     22 qga
  14175 roms
     43 scripts
      3 semihosting
     18 slirp
      2 softmmu
     59 subprojects
    504 target
      6 tcg
      3 test.rb
    175 tests
      6 tools
     20 ui
      8 util

Quoted 1 line

It shows that "roms" contributes by far the most typos. Omitting it would reduce the required time to 22 seconds and the number of typos found (2947 lines in output) very much.

"roms" mostly consists of third-party submodules that we do not have direct control of. I think this should definitely be omitted.

Quoted 1 line

"capstone" (which has no entry in MAINTAINERS)

That's likely because it has been a submodule that has been removed a while ago. "rm -rf capstone" should solve that issue on your local buildtree ;-)

(yes, that's another nuisance of submodules - the checked out files don't go away when the submodule gets removed)

Thomas

#4 lore raw
In reply to Stefan Weil via
Quoted 35 lines · Stefan Weil via

Am 31.10.22 um 08:52 schrieb Thomas Huth:

Quoted 26 lines · Stefan Weil
Quoted 25 lines

make check-spelling can now be used to get a list of spelling errors. It uses the latest version of codespell, a spell checker implemented in Python.

---

This RFC can already be used for manual tests, but still reports false positives, mostly because some variable names are interpreted as words. These words can either be ignored in the check, or in some cases the code
might be changed to use different variable names.

The check currently only skips a few directories and files, so for example
checked out submodules are also checked.

The rule can be extended to allow user provided ignore and skip lists, for example by introducing Makefile variables CODESPELL_SKIP=userfile or CODESPELL_IGNORE=userfile. A limited check could be implemented by providing a base directory CODESPELL_START=basedirectory, for example CODESPELL_START=docs.

Regards,
Stefan

[...]

Quoted 6 lines
Quoted 5 lines

I like the idea, but I think it's unlikely that we can make this work for the whole source tree any time soon. So maybe it makes more sense to start with some few directories first (e.g. docs/ ) and then the maintainers can opt-in by cleaning up their directories first and then by adding their directories to this target here?

Quoted 5 lines
Quoted 1 line

 Thomas

Even without implementing CODESPELL_START as described above, the script can already be used and integrated into CI scripts.

To get most value from CI, we strongly prefer the test to be a clear pass/fail.

We do have some jobs that are marked non-gating, since they have false failures and need manual inspection of results. The effect is those jobs are largely ignored by everyone, so not really of significant benefit.

So I'd agree with Thomas about starting with a config that can get a clear pass/fail, and expanding from there if we can't get the full-tree clean from the start.

Quoted 3 lines

It shows that "roms" contributes by far the most typos. Omitting it would reduce the required time to 22 seconds and the number of typos found (2947 lines in output) very much.

We should not look at 'roms' at all since it is just a place for git submodulees, and build system integration. No interesting code lives there.

Quoted 2 lines

"capstone" (which has no entry in MAINTAINERS), "target" and "hw" also contribute more than 300 hits each, therefore cc'ing Richard.

We should completely ignoring 'capstone' and any other git submodule as those are 3rd party codebases we don't maintain ourselves.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#5 lore raw
In reply to Stefan Weil via
Quoted 29 lines · Stefan Weil via

make check-spelling can now be used to get a list of spelling errors. It uses the latest version of codespell, a spell checker implemented in Python.

---

This RFC can already be used for manual tests, but still reports false positives, mostly because some variable names are interpreted as words. These words can either be ignored in the check, or in some cases the code might be changed to use different variable names.

The check currently only skips a few directories and files, so for example checked out submodules are also checked.

The rule can be extended to allow user provided ignore and skip lists, for example by introducing Makefile variables CODESPELL_SKIP=userfile or CODESPELL_IGNORE=userfile. A limited check could be implemented by providing a base directory CODESPELL_START=basedirectory, for example CODESPELL_START=docs.

Regards,
Stefan

  tests/Makefile.include       | 10 ++++++++++
  tests/codespell/README.rst   | 18 ++++++++++++++++++
  tests/codespell/exclude-file |  3 +++
  tests/codespell/ignore-words | 19 +++++++++++++++++++
  tests/requirements.txt       |  1 +
  5 files changed, 51 insertions(+)
Quoted 3 lines

create mode 100644 tests/codespell/README.rst
create mode 100644 tests/codespell/exclude-file
create mode 100644 tests/codespell/ignore-words

Just wondering about this list...

tests/codespell/ignore-words line 1

Quoted 3 lines

+++ b/tests/codespell/ignore-words

tests/codespell/ignore-words new file +1 -0
@@ -0,0 +1,19 @@
+buid

What is 'buid'? PPC-specific apparently.

tests/codespell/ignore-words line 9

Quoted 8 lines
tests/codespell/ignore-words new file +8 -0
+busses
+dout
+falt
+fpr
+hace
+hax
+hda
+nd

Apparently 'NIC info'...

tests/codespell/ignore-words line 10

Quoted 1 line
tests/codespell/ignore-words new file +1 -0
+ot

Is 'ot' MemOp?

tests/codespell/ignore-words line 17

Quoted 7 lines
tests/codespell/ignore-words new file +7 -0
+pard
+parm
+ptd
+ser
+som
+synopsys
+te

Is that 'target endianness'?

tests/codespell/ignore-words line 18

Quoted 1 line
tests/codespell/ignore-words new file +1 -0
+toke

Where is 'toke'?

tests/codespell/ignore-words line 19

Quoted 1 line
tests/codespell/ignore-words new file +1 -0
+ue

Where is 'ue'?

#6 lore raw

Below I added some examples for the words which are currently ignored by codespell.

Am 31.10.22 um 16:40 schrieb Philippe Mathieu-Daudé:

Quoted 34 lines · Stefan Weil via
Quoted 28 lines

make check-spelling can now be used to get a list of spelling errors. It uses the latest version of codespell, a spell checker implemented in Python.

---

This RFC can already be used for manual tests, but still reports false positives, mostly because some variable names are interpreted as words. These words can either be ignored in the check, or in some cases the code might be changed to use different variable names.

The check currently only skips a few directories and files, so for example checked out submodules are also checked.

The rule can be extended to allow user provided ignore and skip lists, for example by introducing Makefile variables CODESPELL_SKIP=userfile or CODESPELL_IGNORE=userfile. A limited check could be implemented by providing a base directory CODESPELL_START=basedirectory, for example CODESPELL_START=docs.

Regards,
Stefan

  tests/Makefile.include       | 10 ++++++++++ tests/codespell/README.rst   | 18 ++++++++++++++++++ tests/codespell/exclude-file |  3 +++ tests/codespell/ignore-words | 19 +++++++++++++++++++ tests/requirements.txt       |  1 +

Quoted 4 lines
  5 files changed, 51 insertions(+)

  create mode 100644 tests/codespell/README.rst
  create mode 100644 tests/codespell/exclude-file
  create mode 100644 tests/codespell/ignore-words

Quoted 7 lines

Just wondering about this list...

tests/codespell/ignore-words line 1

Quoted 3 lines

+++ b/tests/codespell/ignore-words

tests/codespell/ignore-words new file +1 -0
@@ -0,0 +1,19 @@
+buid

What is 'buid'? PPC-specific apparently.

hw/ppc/spapr_pci.c:SpaprPhbState *spapr_pci_find_phb(SpaprMachineState *spapr, uint64_t buid) include/hw/ppc/xics.h: * We currently only support one BUID which is our interrupt base [...]

Quoted 10 lines

tests/codespell/ignore-words line 9

Quoted 8 lines
tests/codespell/ignore-words new file +8 -0
+busses
+dout
+falt
+fpr
+hace
+hax
+hda
+nd

Apparently 'NIC info'...

hw/arm/aspeed.c:    NICInfo *nd = &nd_table[0]; hw/display/macfb.c:    NubusDevice *nd = NUBUS_DEVICE(s); [...]

Quoted 3 lines

tests/codespell/ignore-words line 10

Quoted 1 line
tests/codespell/ignore-words new file +1 -0
+ot

Is 'ot' MemOp?

target/i386/tcg/decode-new.c.inc:static bool decode_op_size(DisasContext *s, X86OpEntry *e, X86OpSize size, MemOp *ot) [...]

Quoted 9 lines

tests/codespell/ignore-words line 17

Quoted 7 lines
tests/codespell/ignore-words new file +7 -0
+pard
+parm
+ptd
+ser
+som
+synopsys
+te

Is that 'target endianness'?

accel/tcg/cputlb.c: * @te: pointer to CPUTLBEntry
hw/audio/cs4231a.c:#define TE (1 << 6)
[...]

Quoted 3 lines

tests/codespell/ignore-words line 18

Quoted 1 line
tests/codespell/ignore-words new file +1 -0
+toke

Where is 'toke'?

This one is no longer needed. It was used in the old capstone code which I still had in my local sources.

Quoted 2 lines

tests/codespell/ignore-words line 19

Quoted 1 line
tests/codespell/ignore-words new file +1 -0
+ue

Where is 'ue'?

tests/tcg/i386/test-i386-fp-exceptions.c:#define UE (1 << 4) tests/unit/test-keyval.c:    qdict = keyval_parse("val,,ue", "implied", NULL, &err); [...]

I simply had added some examples of "words" which occurred often and which were reported by codespell as typos. These "typos" occur at least 10 times (list produced with grep "^[a-z]" codespell.log | sort -n +1):

statics      10
regiser      11
usig         11
inh          12
tne          12
overriden    13
inactivate   15
upto         15
hsa          16
useable      17
daa          18
crate        21
endianess    22
olt          22
sring        23
vill         25
keypairs     35
gir          46
sav          47
asign       120
inflight    191

Some of them are real typos, others like aSign or statics are variable names and should be ignored, too.

Stefan