docs: Document list of undocumented commands
v3 cover docs: Document list of undocumented commands
This patchset adds a script to the buildsystem which determines the lists of grub commands implemented in the source code and the list of grub commands documented in grub.info and documents the differences.
For that matter, the script will autogenerate a list of the undocumented commands to be included in grub.info, so that the undocumented commands are at least listed there so users have a point to start researching. If the command is at least mentioned somewhere withing grub.texi, that will be noted.
The two commands which were documented in grub.info but not implemented in the source code are removed from grub.texi in separate patches in the patchset.
For conflict free merging, this patchset requires my gitignore patchset as it adds a gitignore pattern for the file it generates.
The main differences to the previous versions ('' and 'v2) of this patchset are:
* Added usage comments to the check-commands.py script
* Use more descriptive variable names in check-commands.py
* If a command is mentioned somewhere in grub.texi, we now add a remark about that to the autogenerated "this command is undocumented" documentation stub.
Hans Ulrich Niedermann (3):
docs: Add grub command documentation checks
docs: Remove docs for non-existing pxe_unload command
docs: Remove docs for non-existing uppermem command
.gitignore | 1 + docs/Makefile.am | 16 +++- docs/check-commands.py | 205 +++++++++++++++++++++++++++++++++++++++++ docs/grub.texi | 28 +----- 4 files changed, 223 insertions(+), 27 deletions(-) create mode 100644 docs/check-commands.py
Signature · 1 line
2.25.3
PATCH v3 1/3 docs: Add grub command documentation checks
This adds the docs/check-commands.py script which checks whether grub commands implemented in the git source tree are documented in docs/grub.texi and vice versa.
During a standard "make" command, BUILT_SOURCES makes sure that docs/undocumented-commands.texi will be created (if it does not exist yet) or updated (if it differs from what the script would generate) or be left alone otherwise.
If you run "make grub.info" in the docs/ subdirectory, the BUILT_SOURCES trick to create or update undocumented-commands.texi will not work (this is an Automake limitation). So if you want to avoid the "all" make target, you need to explictly run "make update-undoc" first.
The generated docs/undocumented-commands.texi will document each otherwise undocumented command by describing that it is undocumented, and by adding that it is mentioned somewhere else in the info page if that happens to be the case.
The build time requirements of the make target's build rule are already required by other parts of the grub buildsystem:
* $(CMP)
compare two files by content
* $(PYTHON)
check-commands.py has been written to run on Python 2 and 3 (tested with Python 2.7 and Python 3.7)
The docs/check-commands.py script runs fast enough to not perceivably slow down a "make" on the grub source tree. The script basically reads and more or less greps through docs/grub.texi and all *.c source files, and that happens very quickly.
--- .gitignore | 1 + docs/Makefile.am | 16 +++- docs/check-commands.py | 205 +++++++++++++++++++++++++++++++++++++++++ docs/grub.texi | 4 + 4 files changed, 223 insertions(+), 3 deletions(-) create mode 100644 docs/check-commands.py
.gitignore +1 -0
@@ -93,6 +93,7 @@ widthspec.bin /docs/*.info-[0-9]* /docs/stamp-1 /docs/stamp-vti +/docs/undocumented-commands.texi /docs/version-dev.texi /docs/version.texi /ehci_test
docs/Makefile.am +13 -3
@@ -2,8 +2,18 @@ AUTOMAKE_OPTIONS = subdir-objects # AM_MAKEINFOFLAGS = --no-split --no-validate info_TEXINFOS = grub.texi grub-dev.texi -grub_TEXINFOS = fdl.texi - -EXTRA_DIST = font_char_metrics.png font_char_metrics.txt +grub_TEXINFOS = fdl.texi undocumented-commands.texi +EXTRA_DIST = check-commands.py font_char_metrics.png font_char_metrics.txt +CLEANFILES = undocumented-commands.texi +BUILT_SOURCES = update-undoc +update-undoc: + $(PYTHON) check-commands.py > undocumented-commands.texi.tmp + @if test -f $(srcdir)/undocumented-commands.texi && $(CMP) undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \ + echo "Not updating undocumented-commands.texi: is up to date"; \ + rm -f undocumented-commands.texi.tmp; \ + else \ + echo "Updating undocumented-commands.texi"; \ + mv -f undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; \ + fi
docs/check-commands.py new file +205 -0
205 lines not on this page · raw message
docs/grub.texi +4 -0
@@ -3764,6 +3764,7 @@ shell}. * General commands:: * Command-line and menu entry commands:: * Networking commands:: +* Undocumented commands:: @end menu
@@ -5636,6 +5637,9 @@ is given, use default list of servers. @end deffn +@include undocumented-commands.texi + + @node Internationalisation @chapter Internationalisation
Signature · 1 line
2.25.3
Re: [PATCH 1/3 v3] docs: Add grub command documentation checks
Quoted 15 lines · On Fri, 24 Apr 2020 15:50:54 +0200 Hans Ulrich Niedermann…
This adds the
docs/check-commands.pyscript which checks whether grub commands implemented in the git source tree are documented in docs/grub.texi and vice versa.During a standard "make" command,
BUILT_SOURCESmakes sure that docs/undocumented-commands.texi will be created (if it does not exist yet) or updated (if it differs from what the script would generate) or be left alone otherwise.If you run "make grub.info" in the docs/ subdirectory, the
BUILT_SOURCEStrick to create or update undocumented-commands.texi will not work (this is an Automake limitation). So if you want to avoid the "all" make target, you need to explictly run "make update-undoc" first.
Quoted 4 lines
The generated docs/undocumented-commands.texi will document each otherwise undocumented command by describing that it is undocumented, and by adding that it is mentioned somewhere else in the info page if that happens to be the case.
The large RFC patchset for TXT boot just made me notice one potential issue with this way of hooking that script into the build process:
Nobody will notice during patch reviews when a patch changes the set of commands in the source code without changing the grub.texi documentation accordingly.
The following are the technical details about why, and how we could force missing documentation changes to show up during patch reviews.
What this patch does (requiring manual code/doc review)
=======================================================
As of this patch, the docs/check-commands.py script is called and docs/undocumented-commands.texi is generated from scratch, and there are no checks on the actual results:
* Every "make" build will generate a new docs/undocumented-commands.texi file which will then be included from docs/grub.texi to be part of the generated grub info page.
(This file should be in $(builddir), not in $(srcdir), but the
older and current build rules for info pages only work in $(srcdir), so docs/undocumented-commands.texi is created in $(srcdir) to stay compatible with this inherited weirdness.)
* If a patch adds or removes commands in the source code, a different docs/undocumented-commands.texi will be generated to reflect that.
However, if nobody happens to notice that the patch changes the set of implemented commands, nobody will notice that documentation updates would be in order.
So patch reviews will not directly show that commands have been added or removed from the codebase and that the documentation should perhaps be adapted to reflect those changes.
What this could be changed to: Forced grub command doc review
=============================================================
If the grub project wants enforce that every single patch which changes the set of implemented commands to also change the set of commands documented in docs/grub.texi accordingly, an alternative approach enforcing this would be the following:
* Add docs/undocumented-commands.texi to the git repository.
* Still have every "make" build update the
docs/undocumented-commands.texi in the $(srcdir).
* Now, if a patch (set) adds adds or removes grub commands in the source code, the generated docs/undocumented-commands.texi will be different from the docs/undocumented-command.texi from the git repo and thus this difference will show up in patches.
This of course assumes that someone will at some time run "make" after applying the respective patch. I would expect a "make" run to be a part of all test processes, so I would expect this assumption to hold.
* If the patch review process results in the decision that the patch should actually add undocumented grub commands to or remove documented grub commands from the source code, then the changes to docs/undocumented-commands.texi can be committed along with the change to the source code.
However, if the patch review process results in the decision that the source code and the documentation should remain in sync, the further divergence will show up here, and a documentation update to docs/grub.texi to reflect the changes in the source code can be asked for in a revised version of the patch.
* However, this forces changes to docs/grub.texi to be in the same patch as the changes to the grub source code.
If you want it to be possible to have a patch set which contains first a patch with the code change and then another patch with the documentation update, this will result in a lot of ugliness with "make" changing git tracked files to different content. In that case, I would strongly advise to stay with the manual doc review process.
Should there be any patches changing doc/check-commands.py which generates doc/undocumented-commands.texi, these generated changes must be contained the same patch for the same reason.
Conclusion
==========
Which of these two methods is preferable for grub is for other people to decide. Both are valid models. I just needed to present that these two alternative methods exist and can be chosen from.
Quoted 312 lines
The build time requirements of the make target's build rule are already required by other parts of the grub buildsystem:
* $(CMP)
compare two files by content* $(PYTHON)
check-commands.pyhas been written to run on Python 2 and 3 (tested with Python 2.7 and Python 3.7)The
docs/check-commands.pyscript runs fast enough to not perceivably slow down a "make" on the grub source tree. The script basically reads and more or less greps through docs/grub.texi and all *.c source files, and that happens very quickly.Signed-off-by Hans Ulrich Niedermann <[email protected]>--- .gitignore | 1 + docs/Makefile.am | 16 +++- docs/check-commands.py | 205 +++++++++++++++++++++++++++++++++++++++++ docs/grub.texi | 4 + 4 files changed, 223 insertions(+), 3 deletions(-) create mode 100644 docs/check-commands.py.gitignore +1
-0@@ -93,6 +93,7 @@ widthspec.bin /docs/*.info-[0-9]* /docs/stamp-1 /docs/stamp-vti +/docs/undocumented-commands.texi /docs/version-dev.texi /docs/version.texi /ehci_testdocs/Makefile.am +2
-3@@ -2,8 +2,18 @@ AUTOMAKE_OPTIONS = subdir-objects # AM_MAKEINFOFLAGS = --no-split --no-validate info_TEXINFOS = grub.texi grub-dev.texi -grub_TEXINFOS = fdl.texi - -EXTRA_DIST = font_char_metrics.png font_char_metrics.txt +grub_TEXINFOS = fdl.texi undocumented-commands.texi +EXTRA_DIST = check-commands.py font_char_metrics.pngfont_char_metrics.txt
docs/Makefile.am +5
-0+CLEANFILES = undocumented-commands.texi +BUILT_SOURCES = update-undoc +update-undoc: + $(PYTHON) check-commands.py > undocumented-commands.texi.tmp + @if test -f $(srcdir)/undocumented-commands.texi && $(CMP)undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \
docs/Makefile.am +1
-0+ echo "Not updating undocumented-commands.texi: is upto date"; \
docs/Makefile.am +4
-0+ rm -f undocumented-commands.texi.tmp; \ + else \ + echo "Updating undocumented-commands.texi"; \ + mv -f undocumented-commands.texi.tmp$(srcdir)/undocumented-commands.texi; \
docs/Makefile.am +1
-0+ fidocs/check-commands.py new file +3
-0@@ -0,0 +1,205 @@ +#!/usr/bin/env python +# +# check-commands.py - compare commands defined in grub.texi the *.csources +#
docs/check-commands.py new file +4
-0+# Usage: +# check-commands.py > undocumented-commands.texi.new +# +# There are no command line parameters. The locations of grub.texiand +# the *.c source files are determined relative to the location of the +# script file. The only output happens on stdout, in texinfo format +# for inclusione into grub.texi via '@include'.
docs/check-commands.py new file +17
-0+ +from __future__ import print_function + +import os +import re + +import sys + + +class CommandChecker: + + def __init__(self): + srcdir, self.prog = os.path.split(__file__) + self.top_srcdir = os.path.dirname(os.path.abspath(srcdir)) + + def read_texi_text_commands(self, texi_filename): + texi_pathname = os.path.join(self.top_srcdir, 'docs',
texi_filename)docs/check-commands.py new file +10
-0+ command_re = re.compile(r'@command\{([a-zA-Z0-9_-]+)\}') + commands = set() + with open(texi_pathname) as texi: + for line in texi.readlines(): + for m in command_re.finditer(line): + commands.add(m[1]) + return commands + + def read_texi_command_menu(self, texi_filename): + texi_pathname = os.path.join(self.top_srcdir, 'docs',
texi_filename)docs/check-commands.py new file +5
-0+ menuentry_re = re.compile(r'\* ([^:]+)::') + commands_node_re = re.compile(r'^@node .+ commands$') + commands = set() + + # Avoid using enum for the state machine state (forcompatibility)
docs/check-commands.py new file +31
-0+ waiting_for_node = True + waiting_for_menu = False + collecting_commands = False + + with open(texi_pathname) as texi: + for line in texi.readlines(): + line = line.strip() + if line.startswith('@comment'): + continue + if waiting_for_node: + if commands_node_re.match(line): + waiting_for_node = False + waiting_for_menu = True + continue + if waiting_for_menu: + if line == '@menu': + waiting_for_menu = False + collecting_commands = True + continue + if collecting_commands: + if line == '@end menu': + collecting_commands = False + waiting_for_node = True + continue + m = menuentry_re.match(line) + commands.add(m.group(1)) + return commands + + def read_src_commands(self): + top = os.path.join(self.top_srcdir, 'grub-core') + register_re =re.compile(r'grub_register_(command|command_p1|extcmd)\s*\("([a-z0-9A-Z_\[]+)",')
docs/check-commands.py new file +28
-0+ commands = set() + for dirpath, dirnames, filenames in os.walk(top): + for filename in filenames: + filepath = os.path.join(dirpath, filename) + filepath_ext = os.path.splitext(filepath)[1] + if filepath_ext == '.c': + with open(filepath) as cfile: + for line in cfile.readlines(): + for m in register_re.finditer(line): + commands.add(m.group(2)) + return commands + + +def write_undocumented_commands(undocumented_commands, + mentioned_commands): + print("""\ +@node Undocumented commands +@section The list of undocumented commands +""") + + if not undocumented_commands: + print(""" +There appear to be no undocumented commands at this time. +""") + return + + print("""\ +These commands are implemented in the grub software but still needto be +documented and sorted into categories.
+""")docs/check-commands.py new file +2
-0+ + sorted_undocumented_commands =sorted(list(undocumented_commands)) +
+ # Fit the longest command into the format string for the menu
entriesdocs/check-commands.py new file +9
-0+ maxlen_str = sorted(list(undocumented_commands), + key=lambda cmd: len(cmd), + reverse=True)[0] + fmt = '* %%-%ds %%s' % (2+len(maxlen_str)) + + print("@menu") + for cmd in sorted_undocumented_commands: + if cmd in mentioned_commands: + print(fmt % ("%s::" % cmd, "Undocumented command (butmentioned somewhere)"))
docs/check-commands.py new file +10
-0+ else: + print(fmt % ("%s::" % cmd, "Undocumented command")) + print("@end menu") + print() + + for cmd in sorted_undocumented_commands: + print("@node %s" % cmd) + print("@subsection %s" % cmd) + print() + print("The grub command @command{%s} has not been documentedproperly yet." % cmd)
docs/check-commands.py new file +21
-0+ if cmd in mentioned_commands: + print(""" +However, the @command{%s} command @emph{is} mentioned +somewhere within this info file, so you might still find some +interesting information there.""" % cmd) + print() + + +def print_set(st): + for n, item in enumerate(sorted(list(st)), start=1): + print("@comment", " %d." % n, item) + + +def main(): + cc = CommandChecker() + + print("""\ +@comment Automatically generated by the %s script. +@comment Do not modify this generated file. +@comment +@comment If you want to document some of the commands listed below,feel +@comment free to copy over some of the @nodes and @subsections below +@comment into the grub.texi file proper and re-generate this file. +@comment""" % cc.prog)
docs/check-commands.py new file +6
-0+ + # This gives a few good hints about what commands are at least + # mentioned somewhere which can be helpful for finding out more + # about commands. + texi_text_commands = cc.read_texi_text_commands('grub.texi') + print("@comment", "Commands mentioned somewhere in the grub.texitext:")
docs/check-commands.py new file +4
-0+ print_set(texi_text_commands) + + texi_node_commands = cc.read_texi_command_menu('grub.texi') + # print("@comment", "Commands documented in grub.texinode/subsection:")
docs/check-commands.py new file +10
-0+ # print_set(texi_node_commands) + + src_commands = cc.read_src_commands() + # print("@comment", "Commands registered in grub source:") + # print_set(src_commands) + + node_without_src = texi_node_commands - src_commands + if node_without_src: + print("@comment", + "Cmds documented in grub.texi node/subsection but notregistered in source:")
docs/check-commands.py new file +12
-0+ print_set(node_without_src) + print("@comment") + else: + print("""\ +@comment There appear to be no commands documented in a grub.texi +@comment node/subsection but not registered in grub source. +@comment""") + + src_without_node = src_commands - texi_node_commands + if src_without_node: + print("@comment", + "Cmds registered in source but not documented ingrub.texi node/subsection:")
docs/check-commands.py new file +12
-0+ print_set(src_without_node) + else: + print("""\ +@comment All commands registered in the source appear to have been +@comment documented in a grub.texi node/subsection. Congratulations! +@comment""") + + print() + + write_undocumented_commands(src_without_node, texi_text_commands) + + # Once grub.texi actually documents all commands, we canuncomment
docs/check-commands.py new file +3
-0+ # this and actually fail if the set of implemented commands and + # the set of documented commands differ in any way. + # if ((len(node_without_src) > 0) or (len(src_without_node) >0)):
docs/check-commands.py new file +7
-0+ # sys.exit(1) + + sys.exit(0) + + +if __name__ == '__main__': + main()docs/grub.texi +4
-0@@ -3764,6 +3764,7 @@ shell}. * General commands:: * Command-line and menu entry commands:: * Networking commands:: +* Undocumented commands:: @end menu@@ -5636,6 +5637,9 @@ is given, use default list of servers. @end deffn +@include undocumented-commands.texi + + @node Internationalisation @chapter Internationalisation
PATCH v3 2/3 docs: Remove docs for non-existing pxe_unload command
In reply to Hans Ulrich NiedermannRemove the documentation of the pxe_unload command from the docs/grub.texi file.
The pxe_unload command is not implemented in the grub source at this time at all. It appears to have been removed in commit 71a78acb0648d3d73c95ab0f021f907499aacc0 (from 2011-07-05).
--- docs/grub.texi | 11 ----------- 1 file changed, 11 deletions(-)
docs/grub.texi +0 -11
@@ -4012,7 +4012,6 @@ you forget a command, you can run the command @command{help} * password_pbkdf2:: Set a hashed password * play:: Play a tune * probe:: Retrieve device info -* pxe_unload:: Unload the PXE environment * rdmsr:: Read values from model-specific registers * read:: Read user input * reboot:: Reboot your computer
@@ -4863,16 +4862,6 @@ The option @option{--part-uuid} is currently only implemented for GPT-formatted @end deffn -@node pxe_unload -@subsection pxe_unload - -@deffn Command pxe_unload -Unload the PXE environment (@pxref{Network}). - -This command is only available on PC BIOS systems. -@end deffn - - @node rdmsr @subsection rdmsr
Signature · 1 line
2.25.3
PATCH v3 3/3 docs: Remove docs for non-existing uppermem command
In reply to Hans Ulrich NiedermannRemove all documentation of and mentions of the uppermem command from the docs/grub.texi file.
The uppermem command is not implemented in the GRUB source at all and appears to not have been implemented despite there having been plans to do so.
To reduce user confusion, this even removes the paragraph describing how GRUB's uppermem command was supposed to complement the Linux kernel's mem= parameter.
--- docs/grub.texi | 13 ------------- 1 file changed, 13 deletions(-)
docs/grub.texi +0 -13
@@ -1114,12 +1114,6 @@ grub> @kbd{initrd16 /initrd} Finally, run the command @command{boot} (@pxref{boot}). @end enumerate -@strong{Caution:} If you use an initrd and specify the @samp{mem=} -option to the kernel to let it use less than actual memory size, you -will also have to specify the same memory size to GRUB. To let GRUB know -the size, run the command @command{uppermem} @emph{before} loading the -kernel. @xref{uppermem}, for more information. - @node NetBSD @subsection NetBSD
@@ -4031,7 +4025,6 @@ you forget a command, you can run the command @command{help} * true:: Do nothing, successfully * trust:: Add public key to list of trusted keys * unset:: Unset an environment variable -* uppermem:: Set the upper memory size @comment * vbeinfo:: List available video modes * verify_detached:: Verify detached digital signature * videoinfo:: List available video modes
@@ -5349,12 +5342,6 @@ Unset the environment variable @var{envvar}. @end deffn -@node uppermem -@subsection uppermem - -This command is not yet implemented for GRUB 2, although it is planned. - - @ignore @node vbeinfo @subsection vbeinfo
Signature · 1 line
2.25.3