Skip to contentSkip to conversation

docs: Fix building with Sphinx 3.2

PATCH In review Peter Maydell 4 participants 11 messages Oct 30, 2020 – Nov 2, 2020

lore
Share & tools
lore
#0 lore raw

cover docs: Fix building with Sphinx 3.2

This patchseries fixes some issues with building our docs with Sphinx 3.2:
* kerneldoc was using the 'c:function' directive for both functions and macros, but Sphinx 3.2 wants 'c:macro' for macros and complains if the argument to 'c:function' isn't parseable as a function declaration
* qemu-option-trace.rst.inc's use of option:: provokes a warning in Sphinx 3.2

We fix the first by making kerneldoc output the right directive depending on the Sphinx version it's working with, and fix the second by just dropping our usage of option:: in favour of a simple definition-list markup.

This does mean our kernel-doc gets another patch that makes it diverge a little from the kernel's version, but we already have one of those (commit 152d1967f650f67b7e). I do want to try to upstream these to the kernel, but that will require more work I suspect since the kernel makes much more extensive use of kernel-doc and probably also has other issues when building with newer Sphinxes. For the moment I would like us to release QEMU 5.2 with docs that build with all the Sphinxes we know about.

Tested by building with Sphinx 1.6.1, 2.0, 2.4, 3.0 and 3.2.

thanks
-- PMM

Peter Maydell (2):
scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments qemu-option-trace.rst.inc: Don't use option:: markup

 docs/qemu-option-trace.rst.inc |  6 +++---
 scripts/kernel-doc             | 18 +++++++++++++++++-
 2 files changed, 20 insertions(+), 4 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/2 scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments

The kerneldoc script currently emits Sphinx markup for a macro with arguments that uses the c:function directive. This is correct for Sphinx versions earlier than Sphinx 3, where c:macro doesn't allow documentation of macros with arguments and c:function is not picky about the syntax of what it is passed. However, in Sphinx 3 the c:macro directive was enhanced to support macros with arguments, and c:function was made more picky about what syntax it accepted.

When kerneldoc is told that it needs to produce output for Sphinx 3 or later, make it emit c:function only for functions and c:macro for macros with arguments. We assume that anything with a return type is a function and anything without is a macro.

This fixes the Sphinx error:

/home/petmay01/linaro/qemu-from-laptop/qemu/docs/../include/qom/object.h:155:Error in declarator If declarator-id with parameters (e.g., 'void f(int arg)'):
Invalid C declaration: Expected identifier in nested name. [error at 25]
DECLARE_INSTANCE_CHECKER ( InstanceType, OBJ_NAME, TYPENAME) -------------------------^
If parenthesis in noptr-declarator (e.g., 'void (*f(int arg))(double)'):
Error in declarator or parameters
Invalid C declaration: Expecting "(" in parameters. [error at 39]
DECLARE_INSTANCE_CHECKER ( InstanceType, OBJ_NAME, TYPENAME) ---------------------------------------^

---
 scripts/kernel-doc | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)
scripts/kernel-doc +17 -1
index 0ff62bb6a2d..4fbaaa05e38 100755
@@ -839,7 +839,23 @@ sub output_function_rst(%) {
 	output_highlight_rst($args{'purpose'});
 	$start = "\n\n**Syntax**\n\n  ``";
     } else {
-	print ".. c:function:: ";
+        if ((split(/\./, $sphinx_version))[0] >= 3) {
+            # Sphinx 3 and later distinguish macros and functions and
+            # complain if you use c:function with something that's not
+            # syntactically valid as a function declaration.
+            # We assume that anything with a return type is a function
+            # and anything without is a macro.
+            if ($args{'functiontype'} ne "") {
+                print ".. c:function:: ";
+            } else {
+                print ".. c:macro:: ";
+            }
+        } else {
+            # Older Sphinx don't support documenting macros that take
+            # arguments with c:macro, and don't complain about the use
+            # of c:function for this.
+            print ".. c:function:: ";
+        }
     }
     if ($args{'functiontype'} ne "") {
 	$start .= $args{'functiontype'} . " " . $args{'function'} . " (";
Signature · 1 line
2.20.1
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments

Quoted 26 lines · Peter Maydell

The kerneldoc script currently emits Sphinx markup for a macro with arguments that uses the c:function directive. This is correct for Sphinx versions earlier than Sphinx 3, where c:macro doesn't allow documentation of macros with arguments and c:function is not picky about the syntax of what it is passed. However, in Sphinx 3 the c:macro directive was enhanced to support macros with arguments, and c:function was made more picky about what syntax it accepted.

When kerneldoc is told that it needs to produce output for Sphinx 3 or later, make it emit c:function only for functions and c:macro for macros with arguments. We assume that anything with a return type is a function and anything without is a macro.

This fixes the Sphinx error:

/home/petmay01/linaro/qemu-from-laptop/qemu/docs/../include/qom/object.h:155:Error in declarator If declarator-id with parameters (e.g., 'void f(int arg)'):
Invalid C declaration: Expected identifier in nested name. [error at 25]
DECLARE_INSTANCE_CHECKER ( InstanceType, OBJ_NAME, TYPENAME) -------------------------^
If parenthesis in noptr-declarator (e.g., 'void (*f(int arg))(double)'):
Error in declarator or parameters
Invalid C declaration: Expecting "(" in parameters. [error at 39]
DECLARE_INSTANCE_CHECKER ( InstanceType, OBJ_NAME, TYPENAME) ---------------------------------------^

Quoted 4 lines
---
 scripts/kernel-doc | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Tested-by on v1 1/2
#3 lore raw

Re: [PATCH 1/2] scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments

In reply to Peter Maydell
Quoted 27 lines · Peter Maydell

The kerneldoc script currently emits Sphinx markup for a macro with arguments that uses the c:function directive. This is correct for Sphinx versions earlier than Sphinx 3, where c:macro doesn't allow documentation of macros with arguments and c:function is not picky about the syntax of what it is passed. However, in Sphinx 3 the c:macro directive was enhanced to support macros with arguments, and c:function was made more picky about what syntax it accepted.

When kerneldoc is told that it needs to produce output for Sphinx 3 or later, make it emit c:function only for functions and c:macro for macros with arguments. We assume that anything with a return type is a function and anything without is a macro.

This fixes the Sphinx error:

/home/petmay01/linaro/qemu-from-laptop/qemu/docs/../include/qom/object.h:155:Error in declarator If declarator-id with parameters (e.g., 'void f(int arg)'):
Invalid C declaration: Expected identifier in nested name. [error at 25]
DECLARE_INSTANCE_CHECKER ( InstanceType, OBJ_NAME, TYPENAME) -------------------------^
If parenthesis in noptr-declarator (e.g., 'void (*f(int arg))(double)'):
Error in declarator or parameters
Invalid C declaration: Expecting "(" in parameters. [error at 39]
DECLARE_INSTANCE_CHECKER ( InstanceType, OBJ_NAME, TYPENAME) ---------------------------------------^

Quoted 4 lines
---
 scripts/kernel-doc | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)
#4 lore raw

PATCH 2/2 qemu-option-trace.rst.inc: Don't use option:: markup

In reply to Peter Maydell

Sphinx 3.2 is pickier than earlier versions about the option:: markup, and complains about our usage in qemu-option-trace.rst:

../../docs/qemu-option-trace.rst.inc:4:Malformed option description
'[enable=]PATTERN', should look like "opt", "-opt args", "--opt args", "/opt args" or "+opt args"

In this file, we're really trying to document the different parts of the top-level --trace option, which qemu-nbd.rst and qemu-img.rst have already introduced with an option:: markup. So it's not right to use option:: here anyway. Switch to a different markup (definition lists) which gives about the same formatted output.

(Unlike option::, this markup doesn't produce index entries; but at the moment we don't do anything much with indexes anyway, and in any case I think it doesn't make much sense to have individual index entries for the sub-parts of the --trace option.)

---
 docs/qemu-option-trace.rst.inc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
docs/qemu-option-trace.rst.inc +3 -3
index 7e09773a9c5..d7acbe67f73 100644
@@ -1,7 +1,7 @@
 
 Specify tracing options.
 
-.. option:: [enable=]PATTERN
+``[enable=]PATTERN``
 
   Immediately enable events matching *PATTERN*
   (either event name or a globbing pattern).  This option is only
@@ -11,7 +11,7 @@ Specify tracing options.
 
   Use :option:`-trace help` to print a list of names of trace points.
 
-.. option:: events=FILE
+``events=FILE``
 
   Immediately enable events listed in *FILE*.
   The file must contain one event name (as listed in the ``trace-events-all``
@@ -19,7 +19,7 @@ Specify tracing options.
   available if QEMU has been compiled with the ``simple``, ``log`` or
   ``ftrace`` tracing backend.
 
-.. option:: file=FILE
+``file=FILE``
 
   Log output traces to *FILE*.
   This option is only available if QEMU has been compiled with
Signature · 1 line
2.20.1
Reviewed-by on v1 2/2
#5 lore raw

Re: [PATCH 2/2] qemu-option-trace.rst.inc: Don't use option:: markup

Quoted 18 lines · Peter Maydell

Sphinx 3.2 is pickier than earlier versions about the option:: markup, and complains about our usage in qemu-option-trace.rst:

../../docs/qemu-option-trace.rst.inc:4:Malformed option description
'[enable=]PATTERN', should look like "opt", "-opt args", "--opt args", "/opt args" or "+opt args"

In this file, we're really trying to document the different parts of the top-level --trace option, which qemu-nbd.rst and qemu-img.rst have already introduced with an option:: markup. So it's not right to use option:: here anyway. Switch to a different markup (definition lists) which gives about the same formatted output.

(Unlike option::, this markup doesn't produce index entries; but at the moment we don't do anything much with indexes anyway, and in any case I think it doesn't make much sense to have individual index entries for the sub-parts of the --trace option.)

Quoted 4 lines
---
 docs/qemu-option-trace.rst.inc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Tested-by on v1 2/2
#6 lore raw

Re: [PATCH 2/2] qemu-option-trace.rst.inc: Don't use option:: markup

In reply to Peter Maydell
Quoted 19 lines · Peter Maydell

Sphinx 3.2 is pickier than earlier versions about the option:: markup, and complains about our usage in qemu-option-trace.rst:

../../docs/qemu-option-trace.rst.inc:4:Malformed option description
'[enable=]PATTERN', should look like "opt", "-opt args", "--opt args", "/opt args" or "+opt args"

In this file, we're really trying to document the different parts of the top-level --trace option, which qemu-nbd.rst and qemu-img.rst have already introduced with an option:: markup. So it's not right to use option:: here anyway. Switch to a different markup (definition lists) which gives about the same formatted output.

(Unlike option::, this markup doesn't produce index entries; but at the moment we don't do anything much with indexes anyway, and in any case I think it doesn't make much sense to have individual index entries for the sub-parts of the --trace option.)

Quoted 4 lines
---
 docs/qemu-option-trace.rst.inc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
#7 lore raw
In reply to Peter Maydell
Quoted 10 lines · Peter Maydell

This does mean our kernel-doc gets another patch that makes it diverge a little from the kernel's version, but we already have one of those (commit 152d1967f650f67b7e). I do want to try to upstream these to the kernel, but that will require more work I suspect since the kernel makes much more extensive use of kernel-doc and probably also has other issues when building with newer Sphinxes. For the moment I would like us to release QEMU 5.2 with docs that build with all the Sphinxes we know about.

FWIW I've sent to Linux our other two local patches, and if you are okay with it I can also do the sync in the other direction before 5.2 (the plan was to do it afterwards).

Paolo

#8 lore raw
Quoted 13 lines · Paolo Bonzini
Quoted 10 lines · Peter Maydell

This does mean our kernel-doc gets another patch that makes it diverge a little from the kernel's version, but we already have one of those (commit 152d1967f650f67b7e). I do want to try to upstream these to the kernel, but that will require more work I suspect since the kernel makes much more extensive use of kernel-doc and probably also has other issues when building with newer Sphinxes. For the moment I would like us to release QEMU 5.2 with docs that build with all the Sphinxes we know about.

Quoted 3 lines

FWIW I've sent to Linux our other two local patches, and if you are okay with it I can also do the sync in the other direction before 5.2 (the plan was to do it afterwards).

Probably safer to sync afterwards. Do you have a link to the patches you've sent in the Linux direction ?

thanks
-- PMM

#9 lore raw
Quoted 15 lines · Peter Maydell
Quoted 13 lines · Paolo Bonzini
Quoted 10 lines · Peter Maydell

This does mean our kernel-doc gets another patch that makes it diverge a little from the kernel's version, but we already have one of those (commit 152d1967f650f67b7e). I do want to try to upstream these to the kernel, but that will require more work I suspect since the kernel makes much more extensive use of kernel-doc and probably also has other issues when building with newer Sphinxes. For the moment I would like us to release QEMU 5.2 with docs that build with all the Sphinxes we know about.

Quoted 6 lines
Quoted 3 lines

FWIW I've sent to Linux our other two local patches, and if you are okay with it I can also do the sync in the other direction before 5.2 (the plan was to do it afterwards).

Probably safer to sync afterwards. Do you have a link to the patches you've sent in the Linux direction ?

Having actually looked at the current state of the kernel's kernel-doc script I see somebody has already done the necessary updates for Sphinx 3 compatibility. So we have two choices for 5.2:
* take this patch 1 as a minimal fix
* do the sync of the kernel's version of the script

I'll have a look at how painful or otherwise the sync is, I guess. We only use the kerneldoc stuff in the 'devel' manual which is not really userfacing anyway.

thanks
-- PMM

#10 lore raw
Quoted 10 lines · Peter Maydell

Having actually looked at the current state of the kernel's kernel-doc script I see somebody has already done the necessary updates for Sphinx 3 compatibility. So we have two choices for 5.2:
* take this patch 1 as a minimal fix
* do the sync of the kernel's version of the script

I'll have a look at how painful or otherwise the sync is, I guess. We only use the kerneldoc stuff in the 'devel' manual which is not really userfacing anyway.

Oh nice! I had a slightly older Linux checkout and it wasn't there yet.
I think you can go ahead with these two patches, I'll take care of the
sync for 6.0.

Here are our two patches.

https://lore.kernel.org/linux-doc/[email protected]/T/#t