docs: Document list of undocumented commands
v4 cover docs: Document list of undocumented commands
This patchset adds a script to the buildsystem which determines the lists of GRUB commands implemented in the source code and the list of GRUB commands documented in grub.info and documents the differences.
For that matter, the script will autogenerate a list of the undocumented commands to be included in grub.info, so that the undocumented commands are at least listed there so users have a point to start researching. If the command is at least mentioned somewhere withing grub.texi, that will be noted.
The two commands pxe_unload and uppermem which were documented in grub.info but not implemented in the source code are removed from grub.texi in separate patches in the patchset.
The long term goal of this is to reduce the number of undocumented GRUB commands to zero and to keep it there.
For conflict free merging, this patchset requires my gitignore patchset as it adds a gitignore pattern for the file it generates.
Hans Ulrich Niedermann (3):
docs: Add grub command documentation checks
docs: Remove docs for non-existing pxe_unload command
docs: Remove docs for non-existing uppermem command
.gitignore | 1 + docs/Makefile.am | 18 +++- docs/check-commands.py | 205 +++++++++++++++++++++++++++++++++++++++++ docs/grub.texi | 28 +----- 4 files changed, 225 insertions(+), 27 deletions(-) create mode 100644 docs/check-commands.py
Interdiff:
docs/Makefile.am +3 -1
@@ -6,10 +6,12 @@ grub_TEXINFOS = fdl.texi undocumented-commands.texi EXTRA_DIST = check-commands.py font_char_metrics.png font_char_metrics.txt +# This has only been tested to work without the Automake +# info-in-builddir option. CLEANFILES = undocumented-commands.texi BUILT_SOURCES = update-undoc update-undoc: - $(PYTHON) check-commands.py > undocumented-commands.texi.tmp + $(PYTHON) $(srcdir)/check-commands.py > undocumented-commands.texi.tmp @if test -f $(srcdir)/undocumented-commands.texi && $(CMP) undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \ echo "Not updating undocumented-commands.texi: is up to date"; \ rm -f undocumented-commands.texi.tmp; \
Signature · 1 line
2.26.2
PATCH v4 1/3 docs: Add grub command documentation checks
This adds the docs/check-commands.py script which checks whether grub commands implemented in the git source tree are documented in docs/grub.texi and vice versa.
During a standard "make" command, BUILT_SOURCES makes sure that docs/undocumented-commands.texi will be created (if it does not exist yet) or updated (if it differs from what the script would generate). Otherwise, the existing file is left alone.
With docs/undocumented-commands.texi being shipped in dist tarballs alongside grub.info, building GRUB from a dist tarball will still work without rebuilding grub.info page unless an actual change has happened which causes a different docs/undocumented-commands.texi to be generated.
If you run "make grub.info" in the docs/ subdirectory, the BUILT_SOURCES trick to create or update undocumented-commands.texi will not work (this is an Automake limitation). So if you want to avoid the "all" make target, you need to explictly run "make update-undoc" first.
The generated docs/undocumented-commands.texi file will document each otherwise undocumented command by describing that it is undocumented, and by adding that it is mentioned somewhere else in the info page if that happens to be the case.
The build time requirements of the make target's build rule are already required by other parts of the GRUB buildsystem:
* $(CMP)
compare two files by content
* $(PYTHON)
check-commands.py has been written to run on Python 2 and 3 (tested with Python 2.7 and Python 3.7)
* texinfo tools
When building from a git source tree, texinfo is required to rebuild docs/grub.info. When building from a dist tarball, texinfo is not required. This is standard Automake behaviour which we do not change.
However, when building from a dist tarball if you ever run "make clean", then the texinfo toolchain will be required as docs/undocumented-commands.texi will be rebuilt and thus a rebuild of docs/grub.info will be triggered.
I do not expect distribution package builds from dist tarball to use "make clean", so I do not expect this new requirement for the texinfo tools to have much of an impact.
The docs/check-commands.py script runs fast enough to not perceivably slow down a "make" on the grub source tree. The script basically reads and more or less greps through docs/grub.texi and all *.c source files, and that happens very quickly.
create mode 100644 docs/check-commands.py
.gitignore +1 -0
@@ -96,6 +96,7 @@ widthspec.bin /docs/*.info-[0-9]* /docs/stamp-1 /docs/stamp-vti +/docs/undocumented-commands.texi /docs/version-dev.texi /docs/version.texi /ehci_test
docs/Makefile.am +15 -3
@@ -2,8 +2,20 @@ AUTOMAKE_OPTIONS = subdir-objects # AM_MAKEINFOFLAGS = --no-split --no-validate info_TEXINFOS = grub.texi grub-dev.texi -grub_TEXINFOS = fdl.texi - -EXTRA_DIST = font_char_metrics.png font_char_metrics.txt +grub_TEXINFOS = fdl.texi undocumented-commands.texi +EXTRA_DIST = check-commands.py font_char_metrics.png font_char_metrics.txt +# This has only been tested to work without the Automake +# info-in-builddir option. +CLEANFILES = undocumented-commands.texi +BUILT_SOURCES = update-undoc +update-undoc: + $(PYTHON) $(srcdir)/check-commands.py > undocumented-commands.texi.tmp + @if test -f $(srcdir)/undocumented-commands.texi && $(CMP) undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \ + echo "Not updating undocumented-commands.texi: is up to date"; \ + rm -f undocumented-commands.texi.tmp; \ + else \ + echo "Updating undocumented-commands.texi"; \ + mv -f undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; \ + fi
docs/check-commands.py new file +205 -0
205 lines not on this page · raw message
docs/grub.texi +4 -0
@@ -3764,6 +3764,7 @@ shell}. * General commands:: * Command-line and menu entry commands:: * Networking commands:: +* Undocumented commands:: @end menu
@@ -5636,6 +5637,9 @@ is given, use default list of servers. @end deffn +@include undocumented-commands.texi + + @node Internationalisation @chapter Internationalisation
Signature · 1 line
2.26.2
Re: [PATCH v4 1/3] docs: Add grub command documentation checks
11-May-20 1:12 AM, Hans Ulrich Niedermann пишет:
Quoted 333 lines
This adds the
docs/check-commands.pyscript which checks whether grub commands implemented in the git source tree are documented in docs/grub.texi and vice versa.During a standard "make" command,
BUILT_SOURCESmakes sure that docs/undocumented-commands.texi will be created (if it does not exist yet) or updated (if it differs from what the script would generate). Otherwise, the existing file is left alone.With docs/undocumented-commands.texi being shipped in dist tarballs alongside grub.info, building GRUB from a dist tarball will still work without rebuilding grub.info page unless an actual change has happened which causes a different docs/undocumented-commands.texi to be generated.
If you run "make grub.info" in the docs/ subdirectory, the
BUILT_SOURCEStrick to create or update undocumented-commands.texi will not work (this is an Automake limitation). So if you want to avoid the "all" make target, you need to explictly run "make update-undoc" first.The generated docs/undocumented-commands.texi file will document each otherwise undocumented command by describing that it is undocumented, and by adding that it is mentioned somewhere else in the info page if that happens to be the case.
The build time requirements of the make target's build rule are already required by other parts of the GRUB buildsystem:
* $(CMP)
compare two files by content* $(PYTHON)
check-commands.pyhas been written to run on Python 2 and 3 (tested with Python 2.7 and Python 3.7)* texinfo tools
When building from a git source tree, texinfo is required to rebuild docs/grub.info. When building from a dist tarball, texinfo is not required. This is standard Automake behaviour which we do not change.However, when building from a dist tarball if you ever run "make clean", then the texinfo toolchain will be required as docs/undocumented-commands.texi will be rebuilt and thus a rebuild of docs/grub.info will be triggered.
I do not expect distribution package builds from dist tarball to use "make clean", so I do not expect this new requirement for the texinfo tools to have much of an impact.
The
docs/check-commands.pyscript runs fast enough to not perceivably slow down a "make" on the grub source tree. The script basically reads and more or less greps through docs/grub.texi and all *.c source files, and that happens very quickly.Signed-off-by Hans Ulrich Niedermann <[email protected]>create mode 100644 docs/check-commands.py.gitignore +1
-0@@ -96,6 +96,7 @@ widthspec.bin /docs/*.info-[0-9]* /docs/stamp-1 /docs/stamp-vti +/docs/undocumented-commands.texi /docs/version-dev.texi /docs/version.texi /ehci_testdocs/Makefile.am +15
-3@@ -2,8 +2,20 @@ AUTOMAKE_OPTIONS = subdir-objects # AM_MAKEINFOFLAGS = --no-split --no-validate info_TEXINFOS = grub.texi grub-dev.texi -grub_TEXINFOS = fdl.texi - -EXTRA_DIST = font_char_metrics.png font_char_metrics.txt +grub_TEXINFOS = fdl.texi undocumented-commands.texi +EXTRA_DIST = check-commands.py font_char_metrics.png font_char_metrics.txt +# This has only been tested to work without the Automake +# info-in-builddir option. +CLEANFILES = undocumented-commands.texi +BUILT_SOURCES = update-undoc +update-undoc: + $(PYTHON) $(srcdir)/check-commands.py > undocumented-commands.texi.tmp + @if test -f $(srcdir)/undocumented-commands.texi && $(CMP) undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; then \ + echo "Not updating undocumented-commands.texi: is up to date"; \ + rm -f undocumented-commands.texi.tmp; \ + else \ + echo "Updating undocumented-commands.texi"; \ + mv -f undocumented-commands.texi.tmp $(srcdir)/undocumented-commands.texi; \ + fidocs/check-commands.py new file +205
-0@@ -0,0 +1,205 @@ +#!/usr/bin/env python +# +# check-commands.py - compare commands defined in grub.texi the *.c sources +# +# Usage: +# check-commands.py > undocumented-commands.texi.new +# +# There are no command line parameters. The locations of grub.texi and +# the *.c source files are determined relative to the location of the +# script file. The only output happens on stdout, in texinfo format +# for inclusione into grub.texi via '@include'. + +from __future__ import print_function + +import os +import re + +import sys + + +class CommandChecker: + + def __init__(self): + srcdir, self.prog = os.path.split(__file__) + self.top_srcdir = os.path.dirname(os.path.abspath(srcdir)) + + def read_texi_text_commands(self, texi_filename): + texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename) + command_re = re.compile(r'@command\{([a-zA-Z0-9_-]+)\}') + commands = set() + with open(texi_pathname) as texi: + for line in texi.readlines(): + for m in command_re.finditer(line): + commands.add(m[1]) + return commands + + def read_texi_command_menu(self, texi_filename): + texi_pathname = os.path.join(self.top_srcdir, 'docs', texi_filename) + menuentry_re = re.compile(r'\* ([^:]+)::') + commands_node_re = re.compile(r'^@node .+ commands$') + commands = set() + + # Avoid using enum for the state machine state (for compatibility) + waiting_for_node = True + waiting_for_menu = False + collecting_commands = False + + with open(texi_pathname) as texi: + for line in texi.readlines(): + line = line.strip() + if line.startswith('@comment'): + continue + if waiting_for_node: + if commands_node_re.match(line): + waiting_for_node = False + waiting_for_menu = True + continue + if waiting_for_menu: + if line == '@menu': + waiting_for_menu = False + collecting_commands = True + continue + if collecting_commands: + if line == '@end menu': + collecting_commands = False + waiting_for_node = True + continue + m = menuentry_re.match(line) + commands.add(m.group(1)) + return commands + + def read_src_commands(self): + top = os.path.join(self.top_srcdir, 'grub-core') + register_re = re.compile(r'grub_register_(command|command_p1|extcmd)\s*\("([a-z0-9A-Z_\[]+)",') + commands = set() + for dirpath, dirnames, filenames in os.walk(top): + for filename in filenames: + filepath = os.path.join(dirpath, filename) + filepath_ext = os.path.splitext(filepath)[1] + if filepath_ext == '.c': + with open(filepath) as cfile: + for line in cfile.readlines(): + for m in register_re.finditer(line): + commands.add(m.group(2)) + return commands + + +def write_undocumented_commands(undocumented_commands, + mentioned_commands): + print("""\ +@node Undocumented commands +@section The list of undocumented commands +""") + + if not undocumented_commands: + print(""" +There appear to be no undocumented commands at this time. +""") + return + + print("""\ +These commands are implemented in the grub software but still need to be +documented and sorted into categories. +""") + + sorted_undocumented_commands = sorted(list(undocumented_commands)) + + # Fit the longest command into the format string for the menu entries + maxlen_str = sorted(list(undocumented_commands), + key=lambda cmd: len(cmd), + reverse=True)[0] + fmt = '* %%-%ds %%s' % (2+len(maxlen_str)) + + print("@menu") + for cmd in sorted_undocumented_commands: + if cmd in mentioned_commands: + print(fmt % ("%s::" % cmd, "Undocumented command (but mentioned somewhere)")) + else: + print(fmt % ("%s::" % cmd, "Undocumented command")) + print("@end menu") + print() + + for cmd in sorted_undocumented_commands: + print("@node %s" % cmd) + print("@subsection %s" % cmd) + print() + print("The grub command @command{%s} has not been documented properly yet." % cmd) + if cmd in mentioned_commands: + print(""" +However, the @command{%s} command @emph{is} mentioned +somewhere within this info file, so you might still find some +interesting information there.""" % cmd) + print() + + +def print_set(st): + for n, item in enumerate(sorted(list(st)), start=1): + print("@comment", " %d." % n, item) + + +def main(): + cc = CommandChecker() + + print("""\ +@comment Automatically generated by the %s script. +@comment Do not modify this generated file. +@comment +@comment If you want to document some of the commands listed below, feel +@comment free to copy over some of the @nodes and @subsections below +@comment into the grub.texi file proper and re-generate this file. +@comment""" % cc.prog) + + # This gives a few good hints about what commands are at least + # mentioned somewhere which can be helpful for finding out more + # about commands. + texi_text_commands = cc.read_texi_text_commands('grub.texi') + print("@comment", "Commands mentioned somewhere in the grub.texi text:") + print_set(texi_text_commands) + + texi_node_commands = cc.read_texi_command_menu('grub.texi') + # print("@comment", "Commands documented in grub.texi node/subsection:") + # print_set(texi_node_commands) + + src_commands = cc.read_src_commands() + # print("@comment", "Commands registered in grub source:") + # print_set(src_commands) + + node_without_src = texi_node_commands - src_commands + if node_without_src: + print("@comment", + "Cmds documented in grub.texi node/subsection but not registered in source:") + print_set(node_without_src) + print("@comment") + else: + print("""\ +@comment There appear to be no commands documented in a grub.texi +@comment node/subsection but not registered in grub source. +@comment""") + + src_without_node = src_commands - texi_node_commands + if src_without_node: + print("@comment", + "Cmds registered in source but not documented in grub.texi node/subsection:") + print_set(src_without_node) + else: + print("""\ +@comment All commands registered in the source appear to have been +@comment documented in a grub.texi node/subsection. Congratulations! +@comment""") + + print() + + write_undocumented_commands(src_without_node, texi_text_commands) + + # Once grub.texi actually documents all commands, we can uncomment + # this and actually fail if the set of implemented commands and + # the set of documented commands differ in any way. + # if ((len(node_without_src) > 0) or (len(src_without_node) > 0)): + # sys.exit(1) + + sys.exit(0) + + +if __name__ == '__main__': + main()docs/grub.texi +4
-0@@ -3764,6 +3764,7 @@ shell}. * General commands:: * Command-line and menu entry commands:: * Networking commands:: +* Undocumented commands:: @end menu@@ -5636,6 +5637,9 @@ is given, use default list of servers. @end deffn +@include undocumented-commands.texi + + @node Internationalisation @chapter Internationalisation
PATCH v4 2/3 docs: Remove docs for non-existing pxe_unload command
In reply to Hans Ulrich NiedermannRemove the documentation of the pxe_unload command from the docs/grub.texi file.
The pxe_unload command is not implemented in the grub source at this time at all. It appears to have been removed in commit 71a78acb0648d3d73c95ab0f021f907499aacc0 (from 2011-07-05).
docs/grub.texi +0 -11
@@ -4012,7 +4012,6 @@ you forget a command, you can run the command @command{help} * password_pbkdf2:: Set a hashed password * play:: Play a tune * probe:: Retrieve device info -* pxe_unload:: Unload the PXE environment * rdmsr:: Read values from model-specific registers * read:: Read user input * reboot:: Reboot your computer
@@ -4863,16 +4862,6 @@ The option @option{--part-uuid} is currently only implemented for GPT-formatted @end deffn -@node pxe_unload -@subsection pxe_unload - -@deffn Command pxe_unload -Unload the PXE environment (@pxref{Network}). - -This command is only available on PC BIOS systems. -@end deffn - - @node rdmsr @subsection rdmsr
Signature · 1 line
2.26.2
Re: [PATCH v4 2/3] docs: Remove docs for non-existing pxe_unload command
Quoted 6 lines · Hans Ulrich Niedermann
Remove the documentation of the
pxe_unloadcommand from the docs/grub.texi file.The
pxe_unloadcommand is not implemented in the grub source at this time at all. It appears to have been removed in commit71a78acb0648d3d73c95ab0f021f907499aacc0(from 2011-07-05).
It seems to me you though about commit 671a78acb (cleanup pxe and efi network release). I will fix it before push...
Quoted 1 line
Signed-off-by Hans Ulrich Niedermann <[email protected]>
Daniel
PATCH v4 3/3 docs: Remove docs for non-existing uppermem command
In reply to Hans Ulrich NiedermannRemove all documentation of and mentions of the uppermem command from the docs/grub.texi file.
The uppermem command is not implemented in the GRUB source at all and appears to never have been implemented despite former plans to add an uppermem command.
To reduce user confusion, this even removes the paragraph describing how GRUB's uppermem command was supposed to complement the Linux kernel's mem= parameter.
docs/grub.texi +0 -13
@@ -1114,12 +1114,6 @@ grub> @kbd{initrd16 /initrd} Finally, run the command @command{boot} (@pxref{boot}). @end enumerate -@strong{Caution:} If you use an initrd and specify the @samp{mem=} -option to the kernel to let it use less than actual memory size, you -will also have to specify the same memory size to GRUB. To let GRUB know -the size, run the command @command{uppermem} @emph{before} loading the -kernel. @xref{uppermem}, for more information. - @node NetBSD @subsection NetBSD
@@ -4031,7 +4025,6 @@ you forget a command, you can run the command @command{help} * true:: Do nothing, successfully * trust:: Add public key to list of trusted keys * unset:: Unset an environment variable -* uppermem:: Set the upper memory size @comment * vbeinfo:: List available video modes * verify_detached:: Verify detached digital signature * videoinfo:: List available video modes
@@ -5349,12 +5342,6 @@ Unset the environment variable @var{envvar}. @end deffn -@node uppermem -@subsection uppermem - -This command is not yet implemented for GRUB 2, although it is planned. - - @ignore @node vbeinfo @subsection vbeinfo
Signature · 1 line
2.26.2
Re: [PATCH v4 3/3] docs: Remove docs for non-existing uppermem command
Quoted 7 lines · Hans Ulrich Niedermann
Remove all documentation of and mentions of the uppermem command from the docs/grub.texi file.
The uppermem command is not implemented in the GRUB source at all and appears to never have been implemented despite former plans to add an uppermem command.
Quoted 5 lines
To reduce user confusion, this even removes the paragraph describing how GRUB's uppermem command was supposed to complement the Linux kernel's mem= parameter.
Signed-off-by Hans Ulrich Niedermann <[email protected]>
Daniel