Help needed with man page conversions to asciidoc?
Hello Karel,
Am Mo., 8. März 2021 um 11:53 Uhr schrieb Karel Zak <[email protected]>:
Quoted 3 lines
Oh, it's like reading a dream -- sure, go ahead :-)
OK. Sometimes dreams become true ;) I know about the peculiarities of the ~50 years old *roff formats, and I'm happy to help a project to get rid of this odd stuff...
Quoted 4 lines
It would be probably good to move this discussion to [email protected] upstream mailing list to make it possible to others to contribute with ideas and objections.
I like the idea with translated man pages maintained in the upstream tree, keep it all in one place is the only way how to keep it up to date. If I good understand then the result will be another template on translationproject.org to maintain man pages strings in po, right? What about in-tree .po files? I guess it should be separate from normal po files, po-docs/ directory?
Yes, it needs a separate .po file. It's not OK to mix UI and doc translations. Well, it's theoretically possible to merge both .pot files in one to force translators to work on both UI and man pages. But I think the UI .pot file with more than 4000 gettext messages is big enough, especially for newbies. So don't let's blow it up to more than 10000.
I propose the following structure:
The asciidoc man pages will be stored where the *roff man pages reside for ages: in their respective subdirectories (sys-utils, schedutils, whatever). For the special asciidoc files footer.adoc, translation.adoc and asciidoc.conf we need an extra directory, let's name it "man-common".
For the man page translations, we can also use man-common. It's a good place for files like po4a.conf or the TP sync script (if desired), and the subdirectory "po".
Quoted 1 line
For now (as the first step) for me more important is ASCII to man pages conversion with some usable result. The translations should be the next logical step.
Yes, although I'm sure I need to always have a look at the localizabililty while converting the man pages. Once all is OK, I need some more time anyway to import existing .po files and provide some help to the TP folks. As far as I can see, we have translations of util-linux man pages in German, French, Spanish and Polish in manpages-l10n. Some other teams, like Japanese, or even the abandoned Hungarian and Turkish projects, also have such files. Sadly there are no Czech translations known to me :(
Quoted 2 lines
Is there any possibility to use some "style" template for conversion to the man pages (for example the same header/footer for all pages, generic sections like AVAILABILITY, etc.)?
Yes, see the attached tarball for an example footer.adoc. I could extend it with a "Reporting bugs" section.
Quoted 1 line
Don't worry about Autotools, I'll do this task. All I need is some command-line examples of how to do the conversion.
Oh, very nice... Although I also use Autotools in manpages-l10n (but haven't written all the Makefile* and related stuff), my knowledge is poor, and I'm happy after each release without problems;)
Karel, I've already forked the util-linux project at Github; maybe it is useful to create a branch for the Asciidoc stuff in your main repo, which I can check out into the forked repo? This would be probably the most painless way for a merge request.
Best Regards,
Mario
Karel Zak
maintainerQuoted 8 lines · Mario Blättermann
Quoted 8 lines
I like the idea with translated man pages maintained in the upstream tree, keep it all in one place is the only way how to keep it up to date. If I good understand then the result will be another template on translationproject.org to maintain man pages strings in po, right? What about in-tree .po files? I guess it should be separate from normal po files, po-docs/ directory?
Quoted 6 lines
Yes, it needs a separate .po file. It's not OK to mix UI and doc translations. Well, it's theoretically possible to merge both .pot files in one to force translators to work on both UI and man pages. But I think the UI .pot file with more than 4000 gettext messages is big enough, especially for newbies. So don't let's blow it up to more than 10000.
I agree.
Quoted 7 lines
I propose the following structure:
The asciidoc man pages will be stored where the *roff man pages reside for ages: in their respective subdirectories (sys-utils, schedutils, whatever). For the special asciidoc files footer.adoc, translation.adoc and asciidoc.conf we need an extra directory, let's name it "man-common".
OK.
Quoted 3 lines
For the man page translations, we can also use man-common. It's a good place for files like po4a.conf or the TP sync script (if desired), and the subdirectory "po".
OK. For scripts, see also tools/ directory where we have some checkers, simple script to commit po/ changes to git (git-tp-sync) etc.
...
Quoted 6 lines
Quoted 4 lines
Is there any possibility to use some "style" template for conversion to the man pages (for example the same header/footer for all pages, generic sections like AVAILABILITY, etc.)?
Yes, see the attached tarball for an example footer.adoc. I could extend it with a "Reporting bugs" section.
I see, it looks pretty cool and readable.
Quoted 11 lines
Quoted 2 lines
Don't worry about Autotools, I'll do this task. All I need is some command-line examples of how to do the conversion.
Oh, very nice... Although I also use Autotools in manpages-l10n (but haven't written all the Makefile* and related stuff), my knowledge is poor, and I'm happy after each release without problems;)
Karel, I've already forked the util-linux project at Github; maybe it is useful to create a branch for the Asciidoc stuff in your main repo, which I can check out into the forked repo? This would be probably the most painless way for a merge request.
OK, I have created a new branch "topic/asciidoc" on github.
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Hello,
Am Di., 9. März 2021 um 09:49 Uhr schrieb Karel Zak <[email protected]>:
Quoted 54 lines · Mario Blättermann
Quoted 8 lines
Quoted 8 lines
I like the idea with translated man pages maintained in the upstream tree, keep it all in one place is the only way how to keep it up to date. If I good understand then the result will be another template on translationproject.org to maintain man pages strings in po, right? What about in-tree .po files? I guess it should be separate from normal po files, po-docs/ directory?
Quoted 6 lines
Yes, it needs a separate .po file. It's not OK to mix UI and doc translations. Well, it's theoretically possible to merge both .pot files in one to force translators to work on both UI and man pages. But I think the UI .pot file with more than 4000 gettext messages is big enough, especially for newbies. So don't let's blow it up to more than 10000.
I agree.
Quoted 7 lines
I propose the following structure:
The asciidoc man pages will be stored where the *roff man pages reside for ages: in their respective subdirectories (sys-utils, schedutils, whatever). For the special asciidoc files footer.adoc, translation.adoc and asciidoc.conf we need an extra directory, let's name it "man-common".
OK.
Quoted 3 lines
For the man page translations, we can also use man-common. It's a good place for files like po4a.conf or the TP sync script (if desired), and the subdirectory "po".
OK. For scripts, see also
tools/directory where we have some checkers, simple script to commit po/ changes to git (git-tp-sync) etc....
Quoted 6 lines
Quoted 4 lines
Is there any possibility to use some "style" template for conversion to the man pages (for example the same header/footer for all pages, generic sections like AVAILABILITY, etc.)?
Yes, see the attached tarball for an example footer.adoc. I could extend it with a "Reporting bugs" section.
I see, it looks pretty cool and readable.
Quoted 7 lines
Quoted 2 lines
Don't worry about Autotools, I'll do this task. All I need is some command-line examples of how to do the conversion.
Oh, very nice... Although I also use Autotools in manpages-l10n (but haven't written all the Makefile* and related stuff), my knowledge is poor, and I'm happy after each release without problems;)
Quoted 7 lines
Quoted 4 lines
Karel, I've already forked the util-linux project at Github; maybe it is useful to create a branch for the Asciidoc stuff in your main repo, which I can check out into the forked repo? This would be probably the most painless way for a merge request.
OK, I have created a new branch "topic/asciidoc" on github.
Just FYI: I'm finished with the import, but I stumbled upon some peculiarity in Asciidoc. Many of the current util-linux man pages use "mailto:" links as follows:
.MT [email protected]
John Doe
.ME
This leads to properly clickable links in the online man page collections of certain Linux distributions, such as Archlinux [1] or Debian [2]. But the *roff conversion engine of Asciidoctor produces a slightly different code:
.MTO "johndoe\(atexample.com" "John Doe" ""
Which renders to ...
John Doe <[email protected]>
... in the HTML output generated by man2html which is used by the mentioned online collections. The link is not clickable. Because I consider the online collections as the second most important way to view man pages, a non-clickable "mailto:" link would be a step back, which I don't want. I've posted this issue in the Asciidoctor forum [3]. Maybe it is possible to create the desired custom code using a macro... I hope so. Or the Asciidoctor developers change it upstream - which would be the better solution.
BTW, the same applies to weblinks, but first let's find out how to get proper mail links.
[1] https://man.archlinux.org/
[2] https://manpages.debian.org/
[3] https://discuss.asciidoctor.org/How-to-change-rendering-of-links-in-man-pages-tt8575.html
Best Regards,
Mario
Hello,
Am Fr., 19. März 2021 um 23:50 Uhr schrieb Mario Blättermann <[email protected]>:
Quoted 94 lines
Hello,
Am Di., 9. März 2021 um 09:49 Uhr schrieb Karel Zak <[email protected]>:
Quoted 54 lines · Mario Blättermann
Quoted 8 lines
Quoted 8 lines
I like the idea with translated man pages maintained in the upstream tree, keep it all in one place is the only way how to keep it up to date. If I good understand then the result will be another template on translationproject.org to maintain man pages strings in po, right? What about in-tree .po files? I guess it should be separate from normal po files, po-docs/ directory?
Quoted 6 lines
Yes, it needs a separate .po file. It's not OK to mix UI and doc translations. Well, it's theoretically possible to merge both .pot files in one to force translators to work on both UI and man pages. But I think the UI .pot file with more than 4000 gettext messages is big enough, especially for newbies. So don't let's blow it up to more than 10000.
I agree.
Quoted 7 lines
I propose the following structure:
The asciidoc man pages will be stored where the *roff man pages reside for ages: in their respective subdirectories (sys-utils, schedutils, whatever). For the special asciidoc files footer.adoc, translation.adoc and asciidoc.conf we need an extra directory, let's name it "man-common".
OK.
Quoted 3 lines
For the man page translations, we can also use man-common. It's a good place for files like po4a.conf or the TP sync script (if desired), and the subdirectory "po".
OK. For scripts, see also
tools/directory where we have some checkers, simple script to commit po/ changes to git (git-tp-sync) etc....
Quoted 6 lines
Quoted 4 lines
Is there any possibility to use some "style" template for conversion to the man pages (for example the same header/footer for all pages, generic sections like AVAILABILITY, etc.)?
Yes, see the attached tarball for an example footer.adoc. I could extend it with a "Reporting bugs" section.
I see, it looks pretty cool and readable.
Quoted 7 lines
Quoted 2 lines
Don't worry about Autotools, I'll do this task. All I need is some command-line examples of how to do the conversion.
Oh, very nice... Although I also use Autotools in manpages-l10n (but haven't written all the Makefile* and related stuff), my knowledge is poor, and I'm happy after each release without problems;)
Quoted 7 lines
Quoted 4 lines
Karel, I've already forked the util-linux project at Github; maybe it is useful to create a branch for the Asciidoc stuff in your main repo, which I can check out into the forked repo? This would be probably the most painless way for a merge request.
OK, I have created a new branch "topic/asciidoc" on github.
Just FYI: I'm finished with the import, but I stumbled upon some peculiarity in Asciidoc. Many of the current util-linux man pages use "mailto:" links as follows:
.MT [email protected]
John Doe
.METhis leads to properly clickable links in the online man page collections of certain Linux distributions, such as Archlinux [1] or Debian [2]. But the *roff conversion engine of Asciidoctor produces a slightly different code:
.MTO "johndoe\(atexample.com" "John Doe" ""
Which renders to ...
John Doe <[email protected]>
... in the HTML output generated by man2html which is used by the mentioned online collections. The link is not clickable. Because I consider the online collections as the second most important way to view man pages, a non-clickable "mailto:" link would be a step back, which I don't want. I've posted this issue in the Asciidoctor forum [3]. Maybe it is possible to create the desired custom code using a macro... I hope so. Or the Asciidoctor developers change it upstream - which would be the better solution.
Quoted 7 lines
BTW, the same applies to weblinks, but first let's find out how to get proper mail links.
[1] https://man.archlinux.org/
[2] https://manpages.debian.org/
[3] https://discuss.asciidoctor.org/How-to-change-rendering-of-links-in-man-pages-tt8575.html
Just found another problem: After applying the Makefile patches [1] and running "autogen.sh" and "configure", the command »make gen-man« fails:
[mariob@t450 util-linux-mariobl]$ make gen-man
Makefile:15938: warning: overriding recipe for target 'misc-utils/uuidd.8' Makefile:15927: warning: ignoring old recipe for target 'misc-utils/uuidd.8' Makefile:15938: warning: overriding recipe for target 'misc-utils/getopt.1' Makefile:15927: warning: ignoring old recipe for target 'misc-utils/getopt.1'
GEN lib/terminal-colors.d.5/bin/sh: line 3: lib/terminal-colors.d.5.adoc: Permission denied
make: *** [Makefile:15940: lib/terminal-colors.d.5] Error 126
Don't know what permissions are really needed here:
[mariob@t450 util-linux-mariobl]$ ls -l lib/terminal-colors.d.5.adoc -rw-r--r-- 1 mariob mariob 4947 Mar 20 16:34 lib/terminal-colors.d.5.adoc
I cannot imagine that lib/terminal-colors.d.5.adoc needs to be executable... However, after chmod +x, the error message is even stranger than before:
[mariob@t450 util-linux-mariobl]$ make gen-man
Makefile:15938: warning: overriding recipe for target 'misc-utils/uuidd.8' Makefile:15927: warning: ignoring old recipe for target 'misc-utils/uuidd.8' Makefile:15938: warning: overriding recipe for target 'misc-utils/getopt.1' Makefile:15927: warning: ignoring old recipe for target 'misc-utils/getopt.1'
GEN lib/terminal-colors.d.5lib/terminal-colors.d.5.adoc: line 1: //po4a:: No such file or directory lib/terminal-colors.d.5.adoc: line 2: ////: Is a directory lib/terminal-colors.d.5.adoc: line 3: terminal-colors.d.5: command not found lib/terminal-colors.d.5.adoc: line 4: syntax error near unexpected token `newline'lib/terminal-colors.d.5.adoc: line 4: `Copyright 2014 Ondrej Oprala <ooprala@red hat.com>'
make: *** [Makefile:15940: lib/terminal-colors.d.5] Error 2
Obviously the script cannot handle the comment lines at all.
I noticed the following line in the Makefile generated by the configure script:
asciidoc_man_cmd = $(ASCIIDOCTOR)
This seems to be the pure "asciidoctor" command without any options, as far as I can tell, given my poor knowledge about GNU Autotools.
[1] https://github.com/mariobl/util-linux/commit/34455501de74c405b56daa438ee3e839e92fbc17
Best Regards,
Mario
Karel Zak
maintainerQuoted 4 lines · Mario Blättermann
Just found another problem: After applying the
Makefilepatches [1] and running "autogen.sh" and "configure", the command »make gen-man« fails:
Quoted 8 lines
[mariob@t450 util-linux-mariobl]$ make gen-man
Makefile:15938: warning: overriding recipe for target 'misc-utils/uuidd.8' Makefile:15927: warning: ignoring old recipe for target 'misc-utils/uuidd.8' Makefile:15938: warning: overriding recipe for target 'misc-utils/getopt.1' Makefile:15927: warning: ignoring old recipe for target 'misc-utils/getopt.1'
GENlib/terminal-colors.d.5/bin/sh: line 3:lib/terminal-colors.d.5.adoc: Permission denied
make: *** [Makefile:15940:lib/terminal-colors.d.5] Error 126
It sounds like $asciidoc_man_cmd is empty. This is my mistake.
Quoted 1 line
[1] https://github.com/mariobl/util-linux/commit/34455501de74c405b56daa438ee3e839e92fbc17
You also need to define $ASCIIDOCTOR variable in ./configure.ac. Please add the patch below. I have probably forgot it in my suggestion.
Karel
configure.ac +1 -0
@@ -232,6 +232,7 @@ PKG_INSTALLDIR(['${usrlib_execdir}/pkgconfig']) GTK_DOC_CHECK([1.10]) AC_PATH_PROG([XSLTPROC], [xsltproc]) +AC_PATH_PROG([ASCIIDOCTOR], [asciidoctor]) linux_os=no bsd_os=no
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Hello Karel,
Am Di., 23. März 2021 um 10:27 Uhr schrieb Karel Zak <[email protected]>:
Quoted 32 lines · Mario Blättermann
Quoted 4 lines
Just found another problem: After applying the
Makefilepatches [1] and running "autogen.sh" and "configure", the command »make gen-man« fails:Quoted 8 lines
[mariob@t450 util-linux-mariobl]$ make gen-man
Makefile:15938: warning: overriding recipe for target 'misc-utils/uuidd.8' Makefile:15927: warning: ignoring old recipe for target 'misc-utils/uuidd.8' Makefile:15938: warning: overriding recipe for target 'misc-utils/getopt.1' Makefile:15927: warning: ignoring old recipe for target 'misc-utils/getopt.1'
GENlib/terminal-colors.d.5/bin/sh: line 3:lib/terminal-colors.d.5.adoc: Permission denied
make: *** [Makefile:15940:lib/terminal-colors.d.5] Error 126It sounds like $asciidoc_man_cmd is empty. This is my mistake.
Quoted 1 line
[1] https://github.com/mariobl/util-linux/commit/34455501de74c405b56daa438ee3e839e92fbc17
You also need to define $ASCIIDOCTOR variable in ./configure.ac. Please add the patch below. I have probably forgot it in my suggestion.
Karel
configure.ac +0
-0
configure.ac PKG_INSTALLDIR(['${usrlib_execdir}/pkgconfig']) · line 238
Quoted 7 lines
configure.ac +1
-0@@ -232,6 +232,7 @@ PKG_INSTALLDIR(['${usrlib_execdir}/pkgconfig']) GTK_DOC_CHECK([1.10]) AC_PATH_PROG([XSLTPROC], [xsltproc]) +AC_PATH_PROG([ASCIIDOCTOR], [asciidoctor]) linux_os=no bsd_os=no
It works until it reaches the first man page link:
GEN lib/terminal-colors.d.5
GEN libuuid/man/uuid.3 GEN libuuid/man/uuid_clear.3 GEN libuuid/man/uuid_compare.3 GEN libuuid/man/uuid_copy.3 GEN libuuid/man/uuid_generate.3 GEN libuuid/man/uuid_is_null.3 GEN libuuid/man/uuid_parse.3 GEN libuuid/man/uuid_time.3 GEN libuuid/man/uuid_unparse.3 GEN libuuid/man/uuid_generate_random.3
asciidoctor: FAILED: input file ./libuuid/man/uuid_generate_random.3.adoc is mis sing
BTW, it also produces HTML versions of the man pages, which are not in .gitignore. Is it intended to also install HTML versions?
Best Regards,
Mario
Hello Karel,
Am Di., 23. März 2021 um 21:07 Uhr schrieb Mario Blättermann <[email protected]>:
Quoted 15 lines
[...]
It works until it reaches the first man page link:GEN
lib/terminal-colors.d.5GEN libuuid/man/uuid.3 GEN libuuid/man/uuid_clear.3 GEN libuuid/man/uuid_compare.3 GEN libuuid/man/uuid_copy.3 GEN libuuid/man/uuid_generate.3 GEN libuuid/man/uuid_is_null.3 GEN libuuid/man/uuid_parse.3 GEN libuuid/man/uuid_time.3 GEN libuuid/man/uuid_unparse.3 GEN libuuid/man/uuid_generate_random.3
Quoted 6 lines
asciidoctor: FAILED: input file ./libuuid/man/uuid_generate_random.3.adoc is mis sing
BTW, it also produces HTML versions of the man pages, which are not in .gitignore. Is it intended to also install HTML versions?
I found the problem, at least partially. A backslash was missing in Makefile.am, and now "make gen-man" creates *roff man pages, as expected. However, it stops here:
GEN sys-utils/swapoff.8
asciidoctor: FAILED: input file ./sys-utils/swapoff.8.adoc is missing
make: *** [Makefile:15942: sys-utils/swapoff.8] Fehler 1
But I'm wondering why it doesn't complain about the other symlinks in libuuid etc, and it didn't stop until the sys-utils directory has been reached.
Best Regards,
Mario
Karel Zak
maintainerQuoted 3 lines · Mario Blättermann
I found the problem, at least partially. A backslash was missing in Makefile.am, and now "make gen-man" creates *roff man pages, as expected. However, it stops here:
OK.
Quoted 7 lines
GEN sys-utils/swapoff.8
asciidoctor: FAILED: input file ./sys-utils/swapoff.8.adoc is missing
make: *** [Makefile:15942: sys-utils/swapoff.8] Fehler 1But I'm wondering why it doesn't complain about the other symlinks in libuuid etc, and it didn't stop until the sys-utils directory has been reached.
It seems asciidoctor generates also the man page links (see below for more details). It's safe to remove the man page link files from the tree at all.
The issue is that we have the link pages in the dist_man_MANS= and 'make' tries to verify the link file *before* the original man page. It means
dist_man_MANS += sys-utils/swapon.8 sys-utils/swapoff.8
is no problem, but:
dist_man_MANS += sys-utils/swapoff.8 sys-utils/swapon.8
ends with error ;-)
I'll send you pull request to your topic/asciidoc do fix this (and some others minor) issue.
Karel
$ git checkout -f $ make clean $ git status On branch ad nothing to commit, working tree clean
make the man page (V=1 for verbose output):
$ make libuuid/man/uuid_generate.3 V=1
/usr/bin/mkdir -p $(dirname libuuid/man/uuid_generate.3)
srcdir=''; \
test -f ./libuuid/man/uuid_generate.3.adoc || srcdir=./; \
/usr/bin/asciidoctor -b manpage -a 'VERSION=2.36.689-bdda0' -a 'ADJTIME_PATH=/etc/adjtime' ${srcdir}libuuid/man/uuid_generate.3.adocnow see that asciidoctor also updated the uuid_generate_*.3 man pages:
$ git diff
libuuid/man/uuid_generate_random.3 +1 -1
@@ -1 +1 @@ -.so man3/uuid_generate.3 +.so uuid_generate.3
libuuid/man/uuid_generate_time.3 +1 -1
@@ -1 +1 @@ -.so man3/uuid_generate.3 +.so uuid_generate.3
libuuid/man/uuid_generate_time_safe.3 +1 -1
@@ -1 +1 @@ -.so man3/uuid_generate.3 +.so uuid_generate.3
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Karel Zak
maintainerQuoted 6 lines · Karel Zak
The issue is that we have the link pages in the dist_man_MANS= and 'make' tries to verify the link file *before* the original man page. It means
dist_man_MANS+= sys-utils/swapon.8 sys-utils/swapoff.8
Quoted 8 lines
is no problem, but:
dist_man_MANS+= sys-utils/swapoff.8 sys-utils/swapon.8ends with error ;-)
I'll send you pull request to your topic/asciidoc do fix this (and some others minor) issue.
Note that fix it by "right order" in $dist_man_MANS is a poor solution, because for "make -j" it does not follow the order. The solution we need is split man pages and man page links to two variables. I'll do fix it.
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com