Skip to contentSkip to conversation

Auto-generate maintainer profile entries

RFC PATCH In review Mauro Carvalho Chehab 3 participants 11 messages Apr 14 – Apr 15

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover Auto-generate maintainer profile entries

Hi Dan/Jon,

This small patch series change the way maintainer entry profile links are added to the documentation. Instead of having an entry for each of them at an ReST file, get them from MAINTAINERS content.

That should likely make easier to maintain, as there will be a single point to place all such profiles.

I made this as an RFC. The goal is mostly to be a start of discussions about how this is implemented.

Also, it should be noticed that I'm not incorporating the diff content from Dan's sugggestion, as it was just an e-mail reply without a proper patch title/description/SoB.

Some points on this RFC:

1. some P: entries are links to web pages. The current approach ignores them;

2. the current logic doesn't use glob. So, if one would add an entry like:

P: Documentation/foo/profiles-*.rst

it will generate an entry like "../foo/profiles-*".

This probably works, as toc trees accept glob.

3. entries are placed at the order they occur at MAINTAINERS file (but duplication is properly handled);

4. as Randy mentioned, if an entry there is inside another TOC using numeration, those entries will have numeration as well;

5. the approach I took on patch 1 was a little bit lazy, as it ends processing MAINTAINERS two times, and there are some code duplication on different classes to handle path. I opted to do this way to minimize the differences, but it makes sense to clean the code later on newer versions of this series or after applying it;

6. patches 2 and 3 can be applied independently of this approach. They just add two missing "P:" entries to MAINTAINERS.

Mauro Carvalho Chehab (4):
docs: maintainers_include: auto-generate maintainer profile TOC
MAINTAINERS: add an entry for media maintainers profile
MAINTAINERS: add maintainer-tip.rst to X86
docs: auto-generate maintainer entry profile links

 .../maintainer/maintainer-entry-profile.rst   | 17 +---
 .../process/maintainer-handbooks.rst          | 10 +-
 Documentation/sphinx/maintainers_include.py   | 93 +++++++++++++++----
 MAINTAINERS                                   |  2 +
 4 files changed, 81 insertions(+), 41 deletions(-)
Signature · 1 line
2.52.0
#1 lore raw

RFC PATCH 1/4 docs: maintainers_include: auto-generate maintainer profile TOC

Add a feature to allow auto-generating media entry profiles from the corresponding field inside MAINTAINERS file(s).

---
 Documentation/sphinx/maintainers_include.py | 93 +++++++++++++++++----
 1 file changed, 76 insertions(+), 17 deletions(-)
Documentation/sphinx/maintainers_include.py +76 -17
index 519ad18685b2..1dac83bf1a65 100755
@@ -21,6 +21,8 @@ import sys
 import re
 import os.path
 
+from textwrap import indent
+
 from docutils import statemachine
 from docutils.parsers.rst import Directive
 from docutils.parsers.rst.directives.misc import Include
@@ -30,20 +32,11 @@ def ErrorString(exc):  # Shamelessly stolen from docutils
 
 __version__  = '1.0'
 
-def setup(app):
-    app.add_directive("maintainers-include", MaintainersInclude)
-    return dict(
-        version = __version__,
-        parallel_read_safe = True,
-        parallel_write_safe = True
-    )
+class MaintainersParser:
+    """Parse MAINTAINERS file(s) content"""
 
-class MaintainersInclude(Include):
-    """MaintainersInclude (``maintainers-include``) directive"""
-    required_arguments = 0
-
-    def parse_maintainers(self, path):
-        """Parse all the MAINTAINERS lines into ReST for human-readability"""
+    def __init__(self, base_path, path):
+        self.profiles = list()
 
         result = list()
         result.append(".. _maintainers:")
@@ -78,6 +71,12 @@ class MaintainersInclude(Include):
             # Drop needless input whitespace.
             line = line.rstrip()
 
+            match = re.match(r"P:\s*(Documentation/\S+)\.rst", line)
+            if match:
+                fname = os.path.relpath(match.group(1), base_path)
+                if fname not in self.profiles:
+                    self.profiles.append(fname)
+
             # Linkify all non-wildcard refs to ReST files in Documentation/.
             pat = r'(Documentation/([^\s\?\*]*)\.rst)'
             m = re.search(pat, line)
@@ -165,12 +164,23 @@ class MaintainersInclude(Include):
             for separated in field_content.split('\n'):
                 result.append(separated)
 
-        output = "\n".join(result)
+        self.output = "\n".join(result)
+
+        # Create a TOC class
+
+class MaintainersInclude(Include):
+    """MaintainersInclude (``maintainers-include``) directive"""
+    required_arguments = 0
+
+    def emit(self, base_path, path):
+        """Parse all the MAINTAINERS lines into ReST for human-readability"""
+
+        output = MaintainersParser(base_path, path).output
+
         # For debugging the pre-rendered results...
         #print(output, file=open("/tmp/MAINTAINERS.rst", "w"))
 
-        self.state_machine.insert_input(
-          statemachine.string2lines(output), path)
+        self.state_machine.insert_input(statemachine.string2lines(output), path)
 
     def run(self):
         """Include the MAINTAINERS file as part of this reST file."""
@@ -186,12 +196,61 @@ class MaintainersInclude(Include):
 
         # Append "MAINTAINERS"
         path = os.path.join(path, "MAINTAINERS")
+        base_path = os.path.dirname(self.state.document.document.current_source)
 
         try:
             self.state.document.settings.record_dependencies.add(path)
-            lines = self.parse_maintainers(path)
+            lines = self.emit(base_path, path)
         except IOError as error:
             raise self.severe('Problems with "%s" directive path:\n%s.' %
                       (self.name, ErrorString(error)))
 
         return []
+
+class MaintainersProfile(Include):
+    required_arguments = 0
+
+    def emit(self, base_path, path):
+        """Parse all the MAINTAINERS lines looking for profile entries"""
+
+        profiles = MaintainersParser(base_path, path).profiles
+
+        output  = ".. toctree::\n"
+        output += "   :maxdepth: 2\n\n"
+        output += indent("\n".join(profiles), "   ")
+
+        self.state_machine.insert_input(statemachine.string2lines(output), path)
+
+    def run(self):
+        """Include the MAINTAINERS file as part of this reST file."""
+        if not self.state.document.settings.file_insertion_enabled:
+            raise self.warning('"%s" directive disabled.' % self.name)
+
+        # Walk up source path directories to find Documentation/../
+        path = self.state_machine.document.attributes['source']
+        path = os.path.realpath(path)
+        tail = path
+        while tail != "Documentation" and tail != "":
+            (path, tail) = os.path.split(path)
+
+        # Append "MAINTAINERS"
+        path = os.path.join(path, "MAINTAINERS")
+        base_path = os.path.dirname(self.state.document.document.current_source)
+
+        try:
+            self.state.document.settings.record_dependencies.add(path)
+            lines = self.emit(base_path, path)
+        except IOError as error:
+            raise self.severe('Problems with "%s" directive path:\n%s.' %
+                      (self.name, ErrorString(error)))
+
+        return []
+
+def setup(app):
+    app.add_directive("maintainers-include", MaintainersInclude)
+    app.add_directive("maintainers-profile-toc", MaintainersProfile)
+    return dict(
+        version = __version__,
+        parallel_read_safe = True,
+        parallel_write_safe = True
+    )
Signature · 1 line
2.52.0
Acked-by on v1 1/4
#2 lore raw

Re: [PATCH RFC 1/4] docs: maintainers_include: auto-generate maintainer profile TOC

Quoted 6 lines · Mauro Carvalho Chehab

Add a feature to allow auto-generating media entry profiles from the corresponding field inside MAINTAINERS file(s).

Nice!

#3 lore raw

RFC PATCH 2/4 MAINTAINERS: add an entry for media maintainers profile

In reply to Mauro Carvalho Chehab

While media has a maintainers entry profile, its entry is missing at MAINTAINERS.

---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)
MAINTAINERS +1 -0
index f0b106a4dd96..620219e48f98 100644
@@ -16115,6 +16115,7 @@ S:	Maintained
 W:	https://linuxtv.org
 Q:	http://patchwork.kernel.org/project/linux-media/list/
 T:	git git://linuxtv.org/media.git
+P:	Documentation/driver-api/media/maintainer-entry-profile.rst
 F:	Documentation/admin-guide/media/
 F:	Documentation/devicetree/bindings/media/
 F:	Documentation/driver-api/media/
Signature · 1 line
2.52.0
reviewer Acked-by on v1 2/4
#4 lore raw

Re: [PATCH RFC 2/4] MAINTAINERS: add an entry for media maintainers profile

Quoted 4 lines · Mauro Carvalho Chehab

While media has a maintainers entry profile, its entry is missing at MAINTAINERS.

Quoted 16 lines
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)
MAINTAINERS +1 -0
index f0b106a4dd96..620219e48f98 100644
@@ -16115,6 +16115,7 @@ S:	Maintained
 W:	https://linuxtv.org
 Q:	http://patchwork.kernel.org/project/linux-media/list/
 T:	git git://linuxtv.org/media.git
+P:	Documentation/driver-api/media/maintainer-entry-profile.rst
 F:	Documentation/admin-guide/media/
 F:	Documentation/devicetree/bindings/media/
 F:	Documentation/driver-api/media/
Signature · 1 line
~Randy
#5 lore raw

RFC PATCH 3/4 MAINTAINERS: add maintainer-tip.rst to X86

In reply to Mauro Carvalho Chehab

While the maintainer's profile for tip is there, it is not at X86 maintainer's entry.

---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)
MAINTAINERS +1 -0
index 620219e48f98..a85fcae5f56e 100644
@@ -28560,6 +28560,7 @@ M:	Ingo Molnar <[email protected]>
 M:	Borislav Petkov <[email protected]>
 M:	Dave Hansen <[email protected]>
 M:	[email protected]
+P:	Documentation/process/maintainer-tip.rst
 R:	"H. Peter Anvin" <[email protected]>
 L:	[email protected]
 S:	Maintained
Signature · 1 line
2.52.0
reviewer Acked-by on v1 3/4
#6 lore raw

Re: [PATCH RFC 3/4] MAINTAINERS: add maintainer-tip.rst to X86

Quoted 4 lines · Mauro Carvalho Chehab

While the maintainer's profile for tip is there, it is not at X86 maintainer's entry.

Quoted 16 lines
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)
MAINTAINERS +1 -0
index 620219e48f98..a85fcae5f56e 100644
@@ -28560,6 +28560,7 @@ M:	Ingo Molnar <[email protected]>
 M:	Borislav Petkov <[email protected]>
 M:	Dave Hansen <[email protected]>
 M:	[email protected]
+P:	Documentation/process/maintainer-tip.rst
 R:	"H. Peter Anvin" <[email protected]>
 L:	[email protected]
 S:	Maintained
Signature · 1 line
~Randy
#8 lore raw

RFC PATCH 4/4 docs: auto-generate maintainer entry profile links

In reply to Mauro Carvalho Chehab

Instead of manually creating a TOC tree for them, use the new tag to auto-generate its TOC.

---
 .../maintainer/maintainer-entry-profile.rst     | 17 ++---------------
 Documentation/process/maintainer-handbooks.rst  | 10 +---------
 2 files changed, 3 insertions(+), 24 deletions(-)
Documentation/maintainer/maintainer-entry-profile.rst +2 -15
index 6020d188e13d..48ecabd4ce13 100644
@@ -98,18 +98,5 @@ Existing profiles
 For now, existing maintainer profiles are listed here; we will likely want
 to do something different in the near future.
 
-.. toctree::
-   :maxdepth: 1
-
-   ../doc-guide/maintainer-profile
-   ../nvdimm/maintainer-entry-profile
-   ../arch/riscv/patch-acceptance
-   ../process/maintainer-soc
-   ../process/maintainer-soc-clean-dts
-   ../driver-api/media/maintainer-entry-profile
-   ../process/maintainer-netdev
-   ../driver-api/vfio-pci-device-specific-driver-acceptance
-   ../nvme/feature-and-quirk-policy
-   ../filesystems/nfs/nfsd-maintainer-entry-profile
-   ../filesystems/xfs/xfs-maintainer-entry-profile
-   ../mm/damon/maintainer-profile
+See Documentation/process/maintainer-handbooks.rst for subsystem-specific
+profiles.
Documentation/process/maintainer-handbooks.rst +1 -9
index 3d72ad25fc6a..d3d74c719018 100644
@@ -9,12 +9,4 @@ which is supplementary to the general development process handbook
 
 Contents:
 
-.. toctree::
-   :numbered:
-   :maxdepth: 2
-
-   maintainer-netdev
-   maintainer-soc
-   maintainer-soc-clean-dts
-   maintainer-tip
-   maintainer-kvm-x86
+.. maintainers-profile-toc::
Signature · 1 line
2.52.0
reviewer
#9 lore raw

Re: [PATCH RFC 4/4] docs: auto-generate maintainer entry profile links

Quoted 42 lines · Mauro Carvalho Chehab

Instead of manually creating a TOC tree for them, use the new tag to auto-generate its TOC.

---
 .../maintainer/maintainer-entry-profile.rst     | 17 ++---------------
 Documentation/process/maintainer-handbooks.rst  | 10 +---------
 2 files changed, 3 insertions(+), 24 deletions(-)
Documentation/maintainer/maintainer-entry-profile.rst +2 -15
index 6020d188e13d..48ecabd4ce13 100644
@@ -98,18 +98,5 @@ Existing profiles
 For now, existing maintainer profiles are listed here; we will likely want
 to do something different in the near future.
 
-.. toctree::
-   :maxdepth: 1
-
-   ../doc-guide/maintainer-profile
-   ../nvdimm/maintainer-entry-profile
-   ../arch/riscv/patch-acceptance
-   ../process/maintainer-soc
-   ../process/maintainer-soc-clean-dts
-   ../driver-api/media/maintainer-entry-profile
-   ../process/maintainer-netdev
-   ../driver-api/vfio-pci-device-specific-driver-acceptance
-   ../nvme/feature-and-quirk-policy
-   ../filesystems/nfs/nfsd-maintainer-entry-profile
-   ../filesystems/xfs/xfs-maintainer-entry-profile
-   ../mm/damon/maintainer-profile
+See Documentation/process/maintainer-handbooks.rst for subsystem-specific
+profiles.
Documentation/process/maintainer-handbooks.rst +0 -0
index 3d72ad25fc6a..d3d74c719018 100644
@@ -9,12 +9,4 @@ which is supplementary to the general development process handbook
 
 Contents:
 

Documentation/process/maintainer-handbooks.rst which is supplementary to the general development process handbook · line 12

Quoted 10 lines
Documentation/process/maintainer-handbooks.rst +1 -9
-.. toctree::
-   :numbered:
-   :maxdepth: 2
-
-   maintainer-netdev
-   maintainer-soc
-   maintainer-soc-clean-dts
-   maintainer-tip
-   maintainer-kvm-x86
+.. maintainers-profile-toc::

It appears that the maintainer profile entries should be listed here, following "Contents:". Is that correct?
I see nothing following "Contents:" except for the page footer.

Signature · 1 line
~Randy
#10 lore raw

Re: [PATCH RFC 4/4] docs: auto-generate maintainer entry profile links

In reply to Mauro Carvalho Chehab
Quoted 14 lines · Mauro Carvalho Chehab

Instead of manually creating a TOC tree for them, use the new tag to auto-generate its TOC.

---
 .../maintainer/maintainer-entry-profile.rst     | 17 ++---------------
 Documentation/process/maintainer-handbooks.rst  | 10 +---------
 2 files changed, 3 insertions(+), 24 deletions(-)
Documentation/maintainer/maintainer-entry-profile.rst +0 -0
index 6020d188e13d..48ecabd4ce13 100644

Documentation/maintainer/maintainer-entry-profile.rst Existing profiles · line 99

Quoted 2 lines
Documentation/maintainer/maintainer-entry-profile.rst +0 -0
@@ -98,18 +98,5 @@ Existing profiles
 For now, existing maintainer profiles are listed here; we will likely want
 to do something different in the near future.

Given the "near future" is now, I would say go ahead and delete this.

Documentation/process/maintainer-handbooks.rst which is supplementary to the general development process handbook

Quoted 22 lines
Documentation/maintainer/maintainer-entry-profile.rst +2 -15
-.. toctree::
-   :maxdepth: 1
-
-   ../doc-guide/maintainer-profile
-   ../nvdimm/maintainer-entry-profile
-   ../arch/riscv/patch-acceptance
-   ../process/maintainer-soc
-   ../process/maintainer-soc-clean-dts
-   ../driver-api/media/maintainer-entry-profile
-   ../process/maintainer-netdev
-   ../driver-api/vfio-pci-device-specific-driver-acceptance
-   ../nvme/feature-and-quirk-policy
-   ../filesystems/nfs/nfsd-maintainer-entry-profile
-   ../filesystems/xfs/xfs-maintainer-entry-profile
-   ../mm/damon/maintainer-profile
+See Documentation/process/maintainer-handbooks.rst for subsystem-specific
+profiles.
Documentation/process/maintainer-handbooks.rst +0 -0
index 3d72ad25fc6a..d3d74c719018 100644
@@ -9,12 +9,4 @@ which is supplementary to the general development process handbook

Given this whole thing started with a question about why there are 2 files you can go ahead and fold in:

---
For developers, see below for all the known subsystem specific guides. If the subsystem you are contributing to does not have a guide listed here, it is fair to seek clarification of questions raised in Documentation/maintainer/maintainer-entry-profile.rst.

For maintainers, consider documenting additional requirements and expectations if submissions routinely overlook specific submission criteria. See Documentation/maintainer/maintainer-entry-profile.rst and the P: tag in MAINTAINERS.
---

...suggestion with a:

...if you want me to submit that separately, just holler.