Documentation issues
The docs pull for 4.13 will include the conversion of the last DocBook template files, thanks mostly to Mauro. That's the good news; the bad news is that I get to explain a lot of merge conflicts to Linus, most of which result from other trees reaching into Documentation/ and changing files that have been converted.
I can continue in this mode, but I do wonder if there's a better way out there somewhere. So I think there would be value in a session on the maintenance of "subsystems" that don't fit neatly into the kernel source-tree hierarchy.
We could also talk about the state of the RST conversion and whether/how we'd like it to continue. Perhaps we would rather, as Peter recently suggested, "just delete all that nonsense and go back to 80 column 7bit ASCII"? In general, what do the maintainers want from the documentation subsystem, and how can we make it easy to continue improving our docs?
jon
Quoted 11 lines · Jonathan Corbet
The docs pull for 4.13 will include the conversion of the last DocBook template files, thanks mostly to Mauro. That's the good news; the bad news is that I get to explain a lot of merge conflicts to Linus, most of which result from other trees reaching into
Documentation/and changing files that have been converted.I can continue in this mode, but I do wonder if there's a better way out there somewhere. So I think there would be value in a session on the maintenance of "subsystems" that don't fit neatly into the kernel source-tree hierarchy.
Quoted 5 lines
We could also talk about the state of the RST conversion and whether/how we'd like it to continue. Perhaps we would rather, as Peter recently suggested, "just delete all that nonsense and go back to 80 column 7bit ASCII"? In general, what do the maintainers want from the documentation subsystem, and how can we make it easy to continue improving our docs?
For one, I'm going to continue converting PM documentation to RST and I would not welcome deleting the existing .txt one until that is complete.
Same goes for ACPI, pretty much.
Also I don't expect too many documentation-related merge conflicts in the PM case going forward.
Thanks,
Rafael
Quoted 18 lines · Rafael J. Wysocki
Quoted 11 lines · Jonathan Corbet
The docs pull for 4.13 will include the conversion of the last DocBook template files, thanks mostly to Mauro. That's the good news; the bad news is that I get to explain a lot of merge conflicts to Linus, most of which result from other trees reaching into
Documentation/and changing files that have been converted.I can continue in this mode, but I do wonder if there's a better way out there somewhere. So I think there would be value in a session on the maintenance of "subsystems" that don't fit neatly into the kernel source-tree hierarchy.
Quoted 5 lines
We could also talk about the state of the RST conversion and whether/how we'd like it to continue. Perhaps we would rather, as Peter recently suggested, "just delete all that nonsense and go back to 80 column 7bit ASCII"? In general, what do the maintainers want from the documentation subsystem, and how can we make it easy to continue improving our docs?
Quoted 7 lines
For one, I'm going to continue converting PM documentation to RST and I would not welcome deleting the existing .txt one until that is complete.
Same goes for ACPI, pretty much.
Also I don't expect too many documentation-related merge conflicts in the PM case going forward.
I should probably add that I find RST much more appealing than "80 column 7bit ASCII" at least because of how it allows cross-references to be expressed if nothing else.
Thanks,
Rafael
Em Fri, 23 Jun 2017 12:39:36 -0600
Jonathan Corbet <[email protected]> escreveu:
Quoted 10 lines
The docs pull for 4.13 will include the conversion of the last DocBook template files, thanks mostly to Mauro. That's the good news; the bad news is that I get to explain a lot of merge conflicts to Linus, most of which result from other trees reaching into
Documentation/and changing files that have been converted.I can continue in this mode, but I do wonder if there's a better way out there somewhere. So I think there would be value in a session on the maintenance of "subsystems" that don't fit neatly into the kernel source-tree hierarchy.
Unfortunately, conflicts are unavoidable for such kind of conversion, as git won't identify it as a rename.
Quoted 5 lines
We could also talk about the state of the RST conversion and whether/how we'd like it to continue. Perhaps we would rather, as Peter recently suggested, "just delete all that nonsense and go back to 80 column 7bit ASCII"? In general, what do the maintainers want from the documentation subsystem, and how can we make it easy to continue improving our docs?
From my side, going to plain ASCII is a very bad idea.
After converting hundreds of text files to ReST, the big problem we have with text files is that they don't use the same notation. It is a big mess. Even before the ReST conversion, what we had is:
Some files there doesn't even have titles
Some files use markdown
Some files use MediaWiki notation
Some files use ReST
Some files use their own notation.
I guess the great benefit of adopting ReST (with would have been achieved if we had adopted any other notation) is that now we have a documentation style.
Just placing all documents in the same style is a huge improvement from what we had before.
Being able of grouping the documents into books is another big advantage. It is now clearer where some document should be placed.
Also, several text editors (both TUI and GUI) are able to recognize ReST format and use colors for highlights.
So, there are improvements even for those that don't care about fancy html/pdf books, as the documentation will look more coherent and better organized.
In the specific case of media, converting to plain ASCII is a *huge* regression, and some tables simply don't fit at a 80 columns requirement. In summary, We have several tables there to describe bit fields on a 64-bits word (for example, do describe how images are encoded). For such tables, we would require at least 210+ characters per line. So, converting them to plain ASCII simply makes them a lot worse to read.
Also, as our uAPI documentation was always using an enriched text language (they were originally written in LaTeX and PDF, back in the nineties), converting them to plain ASCII actually means to rewrite the entire thing.
Even the conversion to ReST required us to change the format of several tables (as we were using a lot to have tables inside tables, and ReST doesn't support it). Thankfully, Markus found a way to do such conversion via script. Yet, we had lots of manual work to fix stuff after that.
So, I think we should continue the conversion.
-
From my PoV, after the next merge window, we shouldn't expect more conflicts related to DocBook conversion.
However, we'll still have conflicts for the files that are under Documentation/*.txt, as there are stuff there from random subsystems, although get_maintainers usually point them all to the docs subsystem.
I recently took the effort of sending patches converting all those text patches to ReST (except for logo.txt). I opted to not rename them to ReST nor add them on any existing book[1], in order to avoid merge conflicts.
[1] I added a patch that creates an "unsorted" book at the end of series meant just to allow testing the conversion.
IMHO, the next step would be to merge those patches by the end of the next merge window (in order to avoid most conflicts).
Then, I would move those files to subdirectories, renaming them to ReST and adding on some index.rst file.
Thanks,
Mauro
Quoted 35 lines · Mauro Carvalho Chehab
Em Fri, 23 Jun 2017 12:39:36 -0600
Jonathan Corbet <[email protected]> escreveu:Quoted 10 lines
The docs pull for 4.13 will include the conversion of the last DocBook template files, thanks mostly to Mauro. That's the good news; the bad news is that I get to explain a lot of merge conflicts to Linus, most of which result from other trees reaching into
Documentation/and changing files that have been converted.I can continue in this mode, but I do wonder if there's a better way out there somewhere. So I think there would be value in a session on the maintenance of "subsystems" that don't fit neatly into the kernel source-tree hierarchy.
Unfortunately, conflicts are unavoidable for such kind of conversion, as git won't identify it as a rename.
Quoted 5 lines
We could also talk about the state of the RST conversion and whether/how we'd like it to continue. Perhaps we would rather, as Peter recently suggested, "just delete all that nonsense and go back to 80 column 7bit ASCII"? In general, what do the maintainers want from the documentation subsystem, and how can we make it easy to continue improving our docs?
From my side, going to plain ASCII is a very bad idea.
After converting hundreds of text files to ReST, the big problem we have with text files is that they don't use the same notation. It is a big mess. Even before the ReST conversion, what we had is:
Some files there doesn't even have titles
Some files use markdown
Some files use MediaWiki notation
Some files use ReST
Some files use their own notation.
Quoted 6 lines
I guess the great benefit of adopting ReST (with would have been achieved if we had adopted any other notation) is that now we have a documentation style.
Just placing all documents in the same style is a huge improvement from what we had before.
Agreed.
Quoted 2 lines
Being able of grouping the documents into books is another big advantage. It is now clearer where some document should be placed.
I'm not so sure about this one.
I sometimes find it somewhat hard to split things to make them fall nicely into one of the prescribed "bins".
I guess it gets easier when writing documentation from scratch.
Quoted 2 lines
Also, several text editors (both TUI and GUI) are able to recognize ReST format and use colors for highlights.
Honestly, I've had a mixed experience with that. In some cases I wish they had not done that. :-)
Quoted 3 lines
So, there are improvements even for those that don't care about fancy html/pdf books, as the documentation will look more coherent and better organized.
I actually think that being able to generate HTML output with active cross-reference links is a *huge* advantage as it makes it way easier to browse it (plus the fact that it is automatically made available in HTML by kernel.org).
Quoted 13 lines
In the specific case of media, converting to plain ASCII is a *huge* regression, and some tables simply don't fit at a 80 columns requirement. In summary, We have several tables there to describe bit fields on a 64-bits word (for example, do describe how images are encoded). For such tables, we would require at least 210+ characters per line. So, converting them to plain ASCII simply makes them a lot worse to read.
Also, as our uAPI documentation was always using an enriched text language (they were originally written in LaTeX and PDF, back in the nineties), converting them to plain ASCII actually means to rewrite the entire thing.
Quoted 7 lines
Even the conversion to ReST required us to change the format of several tables (as we were using a lot to have tables inside tables, and ReST doesn't support it). Thankfully, Markus found a way to do such conversion via script. Yet, we had lots of manual work to fix stuff after that.
So, I think we should continue the conversion.
I agree and, as I said, I'm going to do that for the PM and ACPI documentation.
At the same time I think it would be good to take that as an opportinity to improve the *content* of the documentation files as well, so not just convert them, but also make them more up to date etc in the process.
Quoted 14 lines
-
From my PoV, after the next merge window, we shouldn't expect more conflicts related to DocBook conversion.
However, we'll still have conflicts for the files that are under
Documentation/*.txt, as there are stuff there from random subsystems, althoughget_maintainersusually point them all to the docs subsystem.I recently took the effort of sending patches converting all those text patches to ReST (except for logo.txt). I opted to not rename them to ReST nor add them on any existing book[1], in order to avoid merge conflicts.
Quoted 8 lines
[1] I added a patch that creates an "unsorted" book at the end of series meant just to allow testing the conversion.
IMHO, the next step would be to merge those patches by the end of the next merge window (in order to avoid most conflicts).
Then, I would move those files to subdirectories, renaming them to ReST and adding on some
index.rstfile.
One comment on that.
There are pieces of .txt documentation falling into the "well-knows source of information" category, with many references to them all over the Web. kernel-parameters.txt is probably the most spectacular example here, but there are others.
Let us not move or rename these, please, or at least put symbolic links in place to point to the new locations or similar, such that the existing WWW links pointing to the documentation at kernel.org still work going forward.
And if we have moved or renamed them already, can we possibly make these links work again somehow?
Thanks,
Rafael
Em Sat, 24 Jun 2017 14:20:40 +0200
"Rafael J. Wysocki" <[email protected]> escreveu:
Quoted 5 lines · Mauro Carvalho Chehab
Quoted 3 lines
Em Fri, 23 Jun 2017 12:39:36 -0600
Jonathan Corbet <[email protected]> escreveu:
Quoted 7 lines
Quoted 2 lines
Being able of grouping the documents into books is another big advantage. It is now clearer where some document should be placed.
I'm not so sure about this one.
I sometimes find it somewhat hard to split things to make them fall nicely into one of the prescribed "bins".
Yeah, one of the problems with existing documentation is that, sometimes, the same document describes both userspace-relevant info and kernelspace APIs.
While sometimes it makes sense to split those two, on other cases, it doesn't. That's specially true on documentation for an specific driver. For example the cpia2 driver documentation:
https://www.kernel.org/doc/html/latest/media/v4l-drivers/cpia2.html
contains a mix of user's information and programmers information. I found lots of similar cases on the documentation for input drivers.
I guess one of the discussions we need to have is with regards to how to better organize things.
Anyway, that forces us to think about better organizing the stuff we have, so, IMHO, it is a good thing.
Quoted 1 line
I guess it gets easier when writing documentation from scratch.
Yes.
Quoted 6 lines
Quoted 3 lines
Also, several text editors (both TUI and GUI) are able to recognize ReST format and use colors for highlights.
Honestly, I've had a mixed experience with that. In some cases I wish they had not done that. :-)
:-)
Yeah, there are bugs on highlights on some tools. Yet, I like it, because, when visualizing docs in text mode, it does emphasis to things like bold, italics, etc.
Quoted 9 lines
Quoted 4 lines
So, there are improvements even for those that don't care about fancy html/pdf books, as the documentation will look more coherent and better organized.
I actually think that being able to generate HTML output with active cross-reference links is a *huge* advantage as it makes it way easier to browse it (plus the fact that it is automatically made available in HTML by kernel.org).
Yes, that's a huge advantage. It also allows me reply with URLs when complaining about someone not doing the right thing, instead of writing otherwise longer explanations.
Quoted 19 lines
Quoted 19 lines
In the specific case of media, converting to plain ASCII is a *huge* regression, and some tables simply don't fit at a 80 columns requirement. In summary, We have several tables there to describe bit fields on a 64-bits word (for example, do describe how images are encoded). For such tables, we would require at least 210+ characters per line. So, converting them to plain ASCII simply makes them a lot worse to read.
Also, as our uAPI documentation was always using an enriched text language (they were originally written in LaTeX and PDF, back in the nineties), converting them to plain ASCII actually means to rewrite the entire thing.
Even the conversion to ReST required us to change the format of several tables (as we were using a lot to have tables inside tables, and ReST doesn't support it). Thankfully, Markus found a way to do such conversion via script. Yet, we had lots of manual work to fix stuff after that.
Quoted 7 lines
Quoted 1 line
So, I think we should continue the conversion.
I agree and, as I said, I'm going to do that for the PM and ACPI documentation.
At the same time I think it would be good to take that as an opportinity to improve the *content* of the documentation files as well, so not just convert them, but also make them more up to date etc in the process.
Agreed. From my experience of converting documents outside my main area of domain, a lot of documents are getting more attention in the conversion process, with attract patches for fixing broken stuff on them.
Quoted 30 lines
Quoted 14 lines
-
From my PoV, after the next merge window, we shouldn't expect more conflicts related to DocBook conversion.
However, we'll still have conflicts for the files that are under
Documentation/*.txt, as there are stuff there from random subsystems, althoughget_maintainersusually point them all to the docs subsystem.I recently took the effort of sending patches converting all those text patches to ReST (except for logo.txt). I opted to not rename them to ReST nor add them on any existing book[1], in order to avoid merge conflicts.
Quoted 8 lines
[1] I added a patch that creates an "unsorted" book at the end of series meant just to allow testing the conversion.
IMHO, the next step would be to merge those patches by the end of the next merge window (in order to avoid most conflicts).
Then, I would move those files to subdirectories, renaming them to ReST and adding on some
index.rstfile.One comment on that.
There are pieces of .txt documentation falling into the "well-knows source of information" category, with many references to them all over the Web. kernel-parameters.txt is probably the most spectacular example here, but there are others.
Quoted 6 lines
Let us not move or rename these, please, or at least put symbolic links in place to point to the new locations or similar, such that the existing WWW links pointing to the documentation at kernel.org still work going forward.
And if we have moved or renamed them already, can we possibly make these links work again somehow?
Agreed. We discussed in the past about two alternatives for those "well known" documents:
1) write a small text on the old file pointing to the new location;
2) use symlink.
Right now, we're actually mixing (1) and (2). IMHO, we should either do (1) or (2).
My personal preference is for (1), as it force people to update cross-references to the right place, but I'm OK with (2) too.
A separate matter is what are those "well known" documents. I bet different Kernel developers will point to different sets of documents they consider to be the ones that require some sort of symlinks ;-)
Thanks,
Mauro
Quoted 15 lines · Mauro Carvalho Chehab
Quoted 11 lines
There are pieces of .txt documentation falling into the "well-knows source of information" category, with many references to them all over the Web. kernel-parameters.txt is probably the most spectacular example here, but there are others.
Let us not move or rename these, please, or at least put symbolic links in place to point to the new locations or similar, such that the existing WWW links pointing to the documentation at kernel.org still work going forward.
And if we have moved or renamed them already, can we possibly make these links work again somehow?
Agreed. We discussed in the past about two alternatives for those "well known" documents:
Quoted 6 lines
1) write a small text on the old file pointing to the new location;
2) use symlink.Right now, we're actually mixing (1) and (2). IMHO, we should either do (1) or (2).
Unfortunately option (3) has also been applied to some of the files:
$ ll Documentation/kernel-parameters.txt ls: cannot access 'Documentation/kernel-parameters.txt': No such file or directory
I wasn't sure whether this was intentional or not. But if not, I'll happily send a patch that introduces a symlink.
Signature · 2 lines
Jiri Kosina SUSE Labs
Em Sun, 25 Jun 2017 22:56:07 +0200 (CEST)
Jiri Kosina <[email protected]> escreveu:
Quoted 24 lines · Mauro Carvalho Chehab
Quoted 16 lines
Quoted 6 lines
There are pieces of .txt documentation falling into the "well-knows source of information" category, with many references to them all over the Web. kernel-parameters.txt is probably the most spectacular example here, but there are others.
Quoted 6 lines
Let us not move or rename these, please, or at least put symbolic links in place to point to the new locations or similar, such that the existing WWW links pointing to the documentation at kernel.org still work going forward.
And if we have moved or renamed them already, can we possibly make these links work again somehow?
Agreed. We discussed in the past about two alternatives for those "well known" documents:
Quoted 6 lines
1) write a small text on the old file pointing to the new location;
2) use symlink.Right now, we're actually mixing (1) and (2). IMHO, we should either do (1) or (2).
Quoted 7 lines
Unfortunately option (3) has also been applied to some of the files:
$ ll Documentation/kernel-parameters.txt ls: cannot access 'Documentation/kernel-parameters.txt': No such file or directoryI wasn't sure whether this was intentional or not. But if not, I'll happily send a patch that introduces a symlink.
It was not intentional in the sense of "hiding" where it went. The idea is to keep the number of such references "minimum", in order to avoid bloating the Documents/ with lots of (1) or (2).
So, the reason why there's currently no cross reference for it is just because nobody decided to put it at the list of "well known" docs that would require a cross-reference of type (1) or (2).
Thanks,
Mauro
Quoted 35 lines · Mauro Carvalho Chehab
Em Sun, 25 Jun 2017 22:56:07 +0200 (CEST)
Jiri Kosina <[email protected]> escreveu:Quoted 24 lines · Mauro Carvalho Chehab
Quoted 16 lines
Quoted 6 lines
There are pieces of .txt documentation falling into the "well-knows source of information" category, with many references to them all over the Web. kernel-parameters.txt is probably the most spectacular example here, but there are others.
Quoted 6 lines
Let us not move or rename these, please, or at least put symbolic links in place to point to the new locations or similar, such that the existing WWW links pointing to the documentation at kernel.org still work going forward.
And if we have moved or renamed them already, can we possibly make these links work again somehow?
Agreed. We discussed in the past about two alternatives for those "well known" documents:
Quoted 6 lines
1) write a small text on the old file pointing to the new location;
2) use symlink.Right now, we're actually mixing (1) and (2). IMHO, we should either do (1) or (2).
Quoted 7 lines
Unfortunately option (3) has also been applied to some of the files:
$ ll Documentation/kernel-parameters.txt ls: cannot access 'Documentation/kernel-parameters.txt': No such file or directoryI wasn't sure whether this was intentional or not. But if not, I'll happily send a patch that introduces a symlink.
Quoted 7 lines
It was not intentional in the sense of "hiding" where it went. The idea is to keep the number of such references "minimum", in order to avoid bloating the Documents/ with lots of (1) or (2).
So, the reason why there's currently no cross reference for it is just because nobody decided to put it at the list of "well known" docs that would require a cross-reference of type (1) or (2).
Which I'm reading as "please send a patch if you care" frankly. :-)
Thanks,
Rafael
Quoted 25 lines · Jiri Kosina
Quoted 15 lines · Mauro Carvalho Chehab
Quoted 11 lines
There are pieces of .txt documentation falling into the "well-knows source of information" category, with many references to them all over the Web. kernel-parameters.txt is probably the most spectacular example here, but there are others.
Let us not move or rename these, please, or at least put symbolic links in place to point to the new locations or similar, such that the existing WWW links pointing to the documentation at kernel.org still work going forward.
And if we have moved or renamed them already, can we possibly make these links work again somehow?
Agreed. We discussed in the past about two alternatives for those "well known" documents:
Quoted 6 lines
1) write a small text on the old file pointing to the new location;
2) use symlink.Right now, we're actually mixing (1) and (2). IMHO, we should either do (1) or (2).
Quoted 7 lines
Unfortunately option (3) has also been applied to some of the files:
$ ll Documentation/kernel-parameters.txt ls: cannot access 'Documentation/kernel-parameters.txt': No such file or directoryI wasn't sure whether this was intentional or not. But if not, I'll happily send a patch that introduces a symlink.
If we do symlinks, wouldn't it be cleaner to separate the old doc directory from the new doc directory? That is, Documentation/* keeps the old txt or symlinks while the ReST is put in another directory, say, docs/* as originally suggested.
thanks,
Takashi
Quoted 32 lines · Takashi Iwai
Quoted 30 lines · Jiri Kosina
Quoted 15 lines · Mauro Carvalho Chehab
Quoted 11 lines
There are pieces of .txt documentation falling into the "well-knows source of information" category, with many references to them all over the Web. kernel-parameters.txt is probably the most spectacular example here, but there are others.
Let us not move or rename these, please, or at least put symbolic links in place to point to the new locations or similar, such that the existing WWW links pointing to the documentation at kernel.org still work going forward.
And if we have moved or renamed them already, can we possibly make these links work again somehow?
Agreed. We discussed in the past about two alternatives for those "well known" documents:
Quoted 6 lines
1) write a small text on the old file pointing to the new location;
2) use symlink.Right now, we're actually mixing (1) and (2). IMHO, we should either do (1) or (2).
Unfortunately option (3) has also been applied to some of the files:
$ ll Documentation/kernel-parameters.txt ls: cannot access 'Documentation/kernel-parameters.txt': No such file or directory
Quoted 7 lines
Quoted 2 lines
I wasn't sure whether this was intentional or not. But if not, I'll happily send a patch that introduces a symlink.
If we do symlinks, wouldn't it be cleaner to separate the old doc directory from the new doc directory? That is,
Documentation/*keeps the old txt or symlinks while the ReST is put in another directory, say, docs/* as originally suggested.
That actually sounds like a good idea to me. :-)
Question is if it's not too late to do that now that we have all that stuff under Documentation/.
Thanks,
Rafael
Quoted 41 lines · Rafael J. Wysocki
Quoted 32 lines · Takashi Iwai
Quoted 30 lines · Jiri Kosina
Quoted 15 lines · Mauro Carvalho Chehab
Quoted 11 lines
There are pieces of .txt documentation falling into the "well-knows source of information" category, with many references to them all over the Web. kernel-parameters.txt is probably the most spectacular example here, but there are others.
Let us not move or rename these, please, or at least put symbolic links in place to point to the new locations or similar, such that the existing WWW links pointing to the documentation at kernel.org still work going forward.
And if we have moved or renamed them already, can we possibly make these links work again somehow?
Agreed. We discussed in the past about two alternatives for those "well known" documents:
Quoted 6 lines
1) write a small text on the old file pointing to the
new location;2) use symlink.
Right now, we're actually mixing (1) and (2). IMHO, we should either do (1) or (2).
Unfortunately option (3) has also been applied to some of the files:
$ ll Documentation/kernel-parameters.txt ls: cannot access 'Documentation/kernel-parameters.txt': No such file or directoryQuoted 7 lines
Quoted 2 lines
I wasn't sure whether this was intentional or not. But if not, I'll happily send a patch that introduces a symlink.
If we do symlinks, wouldn't it be cleaner to separate the old doc directory from the new doc directory? That is,
Documentation/*keeps the old txt or symlinks while the ReST is put in another directory, say, docs/* as originally suggested.
Quoted 4 lines
That actually sounds like a good idea to me. :-)
Question is if it's not too late to do that now that we have all that stuff under
Documentation/.
I think it's a bit late, at least from more documentation-active susbsystems like gpu/. The docbook->rst switch was pains, dealing once more with piles of renames isn't really something I'd like to do. And one of the main selling points of .rst was that it integrates much more smoothly into our existing pile of .txt docs, whereas the DocBook/ subdir was always a bit a ghetto. Forcing another split seems ill-advised to me.
-Daniel
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
Em Tue, 27 Jun 2017 10:41:27 +0200
Daniel Vetter <[email protected]> escreveu:
Quoted 47 lines · Rafael J. Wysocki
Quoted 41 lines · Takashi Iwai
Quoted 32 lines
Quoted 30 lines · Jiri Kosina
Quoted 15 lines · Mauro Carvalho Chehab
Quoted 11 lines
There are pieces of .txt documentation falling into the "well-knows source of information" category, with many references to them all over the Web. kernel-parameters.txt is probably the most spectacular example here, but there are others.
Let us not move or rename these, please, or at least put symbolic links in place to point to the new locations or similar, such that the existing WWW links pointing to the documentation at kernel.org still work going forward.
And if we have moved or renamed them already, can we possibly make these links work again somehow?
Agreed. We discussed in the past about two alternatives for those "well known" documents:
Quoted 6 lines
1) write a small text on the old file pointing to the
new location;2) use symlink.
Right now, we're actually mixing (1) and (2). IMHO, we should either do (1) or (2).
Unfortunately option (3) has also been applied to some of the files:
$ ll Documentation/kernel-parameters.txt ls: cannot access 'Documentation/kernel-parameters.txt': No such file or directoryQuoted 7 lines
Quoted 2 lines
I wasn't sure whether this was intentional or not. But if not, I'll happily send a patch that introduces a symlink.
If we do symlinks, wouldn't it be cleaner to separate the old doc directory from the new doc directory? That is,
Documentation/*keeps the old txt or symlinks while the ReST is put in another directory, say, docs/* as originally suggested.Quoted 4 lines
That actually sounds like a good idea to me. :-)
Question is if it's not too late to do that now that we have all that stuff under
Documentation/.
Quoted 7 lines
I think it's a bit late, at least from more documentation-active susbsystems like gpu/. The
docbook->rstswitch was pains, dealing once more with piles of renames isn't really something I'd like to do. And one of the main selling points of .rst was that it integrates much more smoothly into our existing pile of .txt docs, whereas the DocBook/ subdir was always a bit a ghetto. Forcing another split seems ill-advised to me.
Yeah, while I would love that the docs dir would be just docs/, I also think it is a bit late. Well, renaming patches are actually handled nice by git (a way nicer than docbook->rst conversion), and shouldn't hurt that much.
Still, all references by filename will require changes due to the directory change, both inside the Kernel and on website URLs). So, we may end needing to have another set of symlinks.
Thanks,
Mauro
Quoted 7 lines · On Sun, 25 Jun 2017 22:56:07 +0200 (CEST) Jiri Kosina <[email protected]> wrote:
Unfortunately option (3) has also been applied to some of the files:
$ ll Documentation/kernel-parameters.txt ls: cannot access 'Documentation/kernel-parameters.txt': No such file or directoryI wasn't sure whether this was intentional or not. But if not, I'll happily send a patch that introduces a symlink.
Worries about moving well-known files are why I delayed some of this stuff a bit so I could bring it up at conferences and the kernel summit. It only proceeded after I didn't get any real pushback in those settings.
In general, we move files around in the kernel tree all the time. We don't normally leave symlinks behind; indeed, we never do. Instead, we figure out where the file moved to and get on with life. Are documentation files somehow different, needing different rules in this regard? I wouldn't mind some clarity on that point.
My hope (cue inspirational music here) is that, someday, it will be possible to type "ls Documentation" and get back a reasonably sized listing that is easy to explore. Let's just say that's not the situation at the moment. Getting there will certainly involve moving files around; replacing them with symlinks or pointer files would, to a real extent, defeat the purpose.
jon
Quoted 18 lines
-----Original Message-----
From: Jonathan Corbet on Monday, June 26, 2017 3:18 PMQuoted 4 lines · On Sun, 25 Jun 2017 22:56:07 +0200 (CEST) Jiri Kosina <[email protected]> wrote:
Unfortunately option (3) has also been applied to some of the files:
$ ll Documentation/kernel-parameters.txt ls: cannot access 'Documentation/kernel-parameters.txt': No suchfile or directory
Quoted 3 lines
I wasn't sure whether this was intentional or not. But if not, I'll happily send a patch that introduces a symlink.
Worries about moving well-known files are why I delayed some of this stuff a bit so I could bring it up at conferences and the kernel summit. It only proceeded after I didn't get any real pushback in those settings.
Quoted 5 lines
In general, we move files around in the kernel tree all the time. We don't normally leave symlinks behind; indeed, we never do. Instead, we figure out where the file moved to and get on with life. Are documentation files somehow different, needing different rules in this regard? I wouldn't mind some clarity on that point.
My 2 cents is that files in Documentation are much more often linked to or referenced by things outside the kernel tree, than source code is. A quick perusal of elinux.org shows a fair number of references to text files under Documentation, some with direct links to online git trees. Documentation/SubmittingPatches is a good example.
This one has a "This file has moved..." line in its content, which I believe suffices for getting people to the right place. IMHO that's better than a symlink, at least for web references.
Quoted 7 lines
My hope (cue inspirational music here) is that, someday, it will be possible to type "ls Documentation" and get back a reasonably sized listing that is easy to explore. Let's just say that's not the situation at the moment. Getting there will certainly involve moving files around; replacing them with symlinks or pointer files would, to a real extent, defeat the purpose.
I like the suggestion by Takashi Iwai to separate the new docs from the old, putting symlinks and leaving legacy (not-yet-converted) docs in Documentation, and new stuff in 'docs/*'. I think that the new docs have not been 'pinned' yet by external references, and could still be moved around (in the short term), without much pain. Then "ls docs" could produce the nice listing, while "ls Documentation" would show ugly symlinks, placeholders, and legacy junk.
-- Tim
Hi Jon,
Quoted 14 lines · Jonathan Corbet
Quoted 7 lines · On Sun, 25 Jun 2017 22:56:07 +0200 (CEST) Jiri Kosina <[email protected]> wrote:
Unfortunately option (3) has also been applied to some of the files:
$ ll Documentation/kernel-parameters.txt ls: cannot access 'Documentation/kernel-parameters.txt': No such file or directoryI wasn't sure whether this was intentional or not. But if not, I'll happily send a patch that introduces a symlink.
Worries about moving well-known files are why I delayed some of this stuff a bit so I could bring it up at conferences and the kernel summit. It only proceeded after I didn't get any real pushback in those settings.
Quoted 5 lines
In general, we move files around in the kernel tree all the time. We don't normally leave symlinks behind; indeed, we never do. Instead, we figure out where the file moved to and get on with life. Are documentation files somehow different, needing different rules in this regard? I wouldn't mind some clarity on that point.
Documentation files are special in that they (some of them) are much more likely to be linked from an external web page, and may be read by mere mortals.
My gut feeling says this is mostly limited to SubmittingPatches and CodingStyle, which are (coincidently?) the only two files in the top level using CamelCase.
Source files are mostly used by developers, who don't rely on external links, and who are used to work with the source file hierarchy on a regular basis.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
A message that is not in the archive (sent off-list or not yet received): CAFhKne8ZttmqWYJToCw9EDywzeMdp=eiFpoZ+O5xrzmKnpA09Q@mail.gmail.com
A message that is not in the archive (sent off-list or not yet received): CAFhKne_W-EbjUd_Cm4kyBHrVK6K9r8Ss3gY0ogO1nztbQZYBEg@mail.gmail.com
I find documentation comes in the following bins:
1. User documentation (possible sub-split into "how to use this from kernel space" and "how to use this from user space")
2. Information for someone who's interested in modifying the code. Possibly including architectural considerations (eg locking), performance, ideas for future improvement, etc.
3. Random swearing and abuse
I think the second and third categories of documentation should be kept out of the kernel books and left as plain comments by the code.
On 24 Jun 2017 9:41 am, "Mauro Carvalho Chehab" <[email protected]> wrote:
Yeah, one of the problems with existing documentation is that, sometimes, the same document describes both userspace-relevant info and kernelspace APIs.
Quoted 8 lines · Matthew Wilcox
I find documentation comes in the following bins:
1. User documentation (possible sub-split into "how to use this from kernel space" and "how to use this from user space")
2. Information for someone who's interested in modifying the code. Possibly including architectural considerations (eg locking), performance, ideas for future improvement, etc.
3. Random swearing and abuse
There also is information on various frameworks that driver writers are expected to use and honestly various mixtures of that with 1. above.
Mutual exclusion mechanisms also need to be documented properly IMO and so on.
Quoted 9 lines
I think the second and third categories of documentation should be kept out of the kernel books and left as plain comments by the code.
On 24 Jun 2017 9:41 am, "Mauro Carvalho Chehab" <[email protected]> wrote:
Yeah, one of the problems with existing documentation is that, sometimes, the same document describes both userspace-relevant info and kernelspace APIs.
Right.
But, alas, those things happen to be related, especially when framework-provided sysfs attributes and similar come into play. With the current layout of things it sometimes is hard to avoid documenting the same thing in two different places, risking that the two descriptions of the same thing will diverge in the future eventually.
Moreover, does debugfs fall into "user documentation" or "developer documentation", for example?
And generally documentation on how to diagnose problems for that matter?
Em Sun, 25 Jun 2017 23:42:20 +0200
"Rafael J. Wysocki" <[email protected]> escreveu:
Quoted 25 lines · Matthew Wilcox
Quoted 8 lines
I find documentation comes in the following bins:
1. User documentation (possible sub-split into "how to use this from kernel space" and "how to use this from user space")
2. Information for someone who's interested in modifying the code. Possibly including architectural considerations (eg locking), performance, ideas for future improvement, etc.
3. Random swearing and abuseThere also is information on various frameworks that driver writers are expected to use and honestly various mixtures of that with 1. above.
Mutual exclusion mechanisms also need to be documented properly IMO and so on.
Quoted 9 lines
I think the second and third categories of documentation should be kept out of the kernel books and left as plain comments by the code.
On 24 Jun 2017 9:41 am, "Mauro Carvalho Chehab" <[email protected]> wrote:
Yeah, one of the problems with existing documentation is that, sometimes, the same document describes both userspace-relevant info and kernelspace APIs.
Quoted 8 lines
Right.
But, alas, those things happen to be related, especially when framework-provided sysfs attributes and similar come into play. With the current layout of things it sometimes is hard to avoid documenting the same thing in two different places, risking that the two descriptions of the same thing will diverge in the future eventually.
Well, sysfs attribute description is currently a little messy, IMHO. We have (or should have) all of them described under Documentation/ABI/, but it is not uncommon to have them described on some other random places. As things get bitrot, we end by having different descriptions on each place.
I also suspect that some sysfs descriptions are only outside Documentation/ABI/ (although I never actually tried to seek for such issue).
Quoted 4 lines
Moreover, does debugfs fall into "user documentation" or "developer documentation", for example?
And generally documentation on how to diagnose problems for that matter?
The same thing for sysfs applies to debugfs.
IMHO, debugfs and information about how to diagnose problems belong to "user documentation" / "how to use this from user space" (using Matthew's classification), in the sense that an advanced admin may use them, in order to properly address some issue that will eventually generate a bug report (either for Kernel or to some application/library).
Thanks,
Mauro
Quoted 43 lines · Mauro Carvalho Chehab
Em Sun, 25 Jun 2017 23:42:20 +0200
"Rafael J. Wysocki" <[email protected]> escreveu:Quoted 25 lines · Matthew Wilcox
Quoted 8 lines
I find documentation comes in the following bins:
1. User documentation (possible sub-split into "how to use this from kernel space" and "how to use this from user space")
2. Information for someone who's interested in modifying the code. Possibly including architectural considerations (eg locking), performance, ideas for future improvement, etc.
3. Random swearing and abuseThere also is information on various frameworks that driver writers are expected to use and honestly various mixtures of that with 1. above.
Mutual exclusion mechanisms also need to be documented properly IMO and so on.
Quoted 9 lines
I think the second and third categories of documentation should be kept out of the kernel books and left as plain comments by the code.
On 24 Jun 2017 9:41 am, "Mauro Carvalho Chehab" <[email protected]> wrote:
Yeah, one of the problems with existing documentation is that, sometimes, the same document describes both userspace-relevant info and kernelspace APIs.
Quoted 8 lines
Right.
But, alas, those things happen to be related, especially when framework-provided sysfs attributes and similar come into play. With the current layout of things it sometimes is hard to avoid documenting the same thing in two different places, risking that the two descriptions of the same thing will diverge in the future eventually.
Well, sysfs attribute description is currently a little messy, IMHO. We have (or should have) all of them described under
Documentation/ABI/, but it is not uncommon to have them described on some other random places. As things get bitrot, we end by having different descriptions on each place.
Quoted 3 lines
I also suspect that some sysfs descriptions are only outside
Documentation/ABI/(although I never actually tried to seek for such issue).
That would be my guess too, although I don't remember any specific examples readily.
My point is that some frameworks expose things via sysfs and drivers are expected to provide callbacks invoked from there or similar. The information on what is expected to be provided falls under the "driver API", but it is strictly related to the "admin guide" part as well.
Quoted 5 lines
Quoted 4 lines
Moreover, does debugfs fall into "user documentation" or "developer documentation", for example?
And generally documentation on how to diagnose problems for that matter?
Quoted 8 lines
The same thing for sysfs applies to debugfs.
IMHO, debugfs and information about how to diagnose problems belong to "user documentation" / "how to use this from user space" (using Matthew's classification), in the sense that an advanced admin may use them, in order to properly address some issue that will eventually generate a bug report (either for Kernel or to some application/library).
It also is part of driver developer's toolkit, however, like unit tests and such.
Thanks,
Rafael
Em Sun, 25 Jun 2017 17:32:31 -0400
Matthew Wilcox <[email protected]> escreveu:
Quoted 11 lines
I find documentation comes in the following bins:
1. User documentation (possible sub-split into "how to use this from kernel space" and "how to use this from user space")
2. Information for someone who's interested in modifying the code. Possibly including architectural considerations (eg locking), performance, ideas for future improvement, etc.
3. Random swearing and abuseI think the second and third categories of documentation should be kept out of the kernel books and left as plain comments by the code.
Having comments together the code is good, but sometimes it is not enough.
Just as an example, last week I received two patch series, from different developers, that didn't get right how to collect quality measurements from digital TV frontends. The functions they use are documented via kernel-doc, and they looked on other drivers as examples. Still, that was not not clear enough, as both developers didn't quite get some things.
So, I had to write a few patches to actually explain how a frontend driver is expected to collect statistics:
https://patchwork.linuxtv.org/patch/42081/
And, today, I complemented with two additional texts:
https://patchwork.linuxtv.org/patch/42115/ https://patchwork.linuxtv.org/patch/42116/
As those explanations complement the kernel-doc macros from dvb_frontend.h, they need cross references, in order for the driver developer to better understand how things are connected, and to avoid the need of having to explain everything that it is already documented via kernel-doc tags, as can be seeing at:
https://www.infradead.org/~mchehab/kernel_docs/media/kapi/dtv-core.html#digital-tv-frontend-kabi
That's just today's example. There are plenty of cases where we need to add images and complex tables, etc that just don't fit well as plain text.
Thanks,
Mauro
Quoted 3 lines · On Sat, 24 Jun 2017 14:20:40 +0200 "Rafael J. Wysocki" <[email protected]>…
At the same time I think it would be good to take that as an opportinity to improve the *content* of the documentation files as well, so not just convert them, but also make them more up to date etc in the process.
This is certainly an area of ongoing concern. There has been a fair amount of energy put into the transition, which is great, but if it comes down to "make a bunch of crufty old stuff look better with cross-references to other crufty old stuff", it's not that big a win.
My hope, I guess, is that making the documentation more coherent and approachable will inspire more people to also work to make it better.
jon
Quoted 4 lines · Jonathan Corbet
I can continue in this mode, but I do wonder if there's a better way out there somewhere. So I think there would be value in a session on the maintenance of "subsystems" that don't fit neatly into the kernel source-tree hierarchy.
One thing I think it'd be good to do is work out a sensible way of keeping the maintainers of the relevant subsystems in the loop on things
- a combination of the mechanics and how much effort we want to put into making sure people have at least seen things. For example it looks like the regulator DocBook got converted to RST without me being aware of it (it seems I did get copied on one mail at some point but I can't have read it and there don't seem to have been any resends), that case isn't a problem itself but it seems like an area where things could end up not running smoothly.
I know I rely fairly heavily on subject lines to filter what I'm looking at since some of my subsystems mean that I get copied on lots of random stuff that's of at most passing relevance to me which ends up being an issue with things like DT bindings sometimes (I think this is also part of what went wrong with the regulator conversion) but that doesn't scale since the subject lines can't easily be useful for multiple subsystems simultaneously. I'm not sure what else to do though, we basically only have the subject lines and CC lists to work with here.
Quoted 17 lines · Mark Brown
Quoted 4 lines · Jonathan Corbet
I can continue in this mode, but I do wonder if there's a better way out there somewhere. So I think there would be value in a session on the maintenance of "subsystems" that don't fit neatly into the kernel source-tree hierarchy.
One thing I think it'd be good to do is work out a sensible way of keeping the maintainers of the relevant subsystems in the loop on things
- a combination of the mechanics and how much effort we want to put into making sure people have at least seen things. For example it looks like the regulator DocBook got converted to RST without me being aware of it (it seems I did get copied on one mail at some point but I can't have read it and there don't seem to have been any resends), that case isn't a problem itself but it seems like an area where things could end up not running smoothly.
Quoted 8 lines
I know I rely fairly heavily on subject lines to filter what I'm looking at since some of my subsystems mean that I get copied on lots of random stuff that's of at most passing relevance to me which ends up being an issue with things like DT bindings sometimes (I think this is also part of what went wrong with the regulator conversion) but that doesn't scale since the subject lines can't easily be useful for multiple subsystems simultaneously. I'm not sure what else to do though, we basically only have the subject lines and CC lists to work with here.
Looking at MAINTAINERS, there's relatively few entries for files under Documentation/, and most are for individual files (about 450). None are for one of the remaining DocBook files still in 4.12 afaics. I think we can fix this problem by making sure that a) docs are sorted usefully into directories (e.g. we put all the gpu stuff into Documentation/gpu/) and b) there's MAINTAINERS entries for all of them (which atm doesn't seem to be the case for a lot of stuff).
But I think the lack of clear maintainer responsibilities for docs was something Jon already brough up at the previous KS, and we didn't really get anywhere with it.
-Daniel
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
Quoted 7 lines · Daniel Vetter
Looking at
MAINTAINERS, there's relatively few entries for files underDocumentation/, and most are for individual files (about 450). None are for one of the remaining DocBook files still in 4.12 afaics. I think we can fix this problem by making sure that a) docs are sorted usefully into directories (e.g. we put all the gpu stuff intoDocumentation/gpu/) and b) there'sMAINTAINERSentries for all of them (which atm doesn't seem to be the case for a lot of stuff).
Yeah, there's directories there for my subsystems but I have to confess I'd not checked the broader picture recently.
Quoted 3 lines
But I think the lack of clear maintainer responsibilities for docs was something Jon already brough up at the previous KS, and we didn't really get anywhere with it.
Is that a case of people actively getting in the way somehow or more a case that we're not able to generate enough interest for people to actually work on improving the docs (which is partly about us showing that we care of course)?