Skip to contentSkip to conversation

Documentation/git-diff-tree.txt: fix formatting

PATCH Superseded SZEDER Gábor 4 participants 25 messages Mar 24, 2019 – Apr 3, 2019 Documentation

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 Documentation/git-diff-tree.txt: fix formatting

Some more recent versions of Asciidoctor issue the following warning while building the documentation:

ASCIIDOC git-diff-tree.xml
asciidoctor: WARNING: diff-format.txt: line 2: unterminated listing block

This highlights an issue where the "Raw output format" header is not rendered as a header, and the rest of the document is rendered in monospace. This is not caused by 'diff-format.txt' in itself, but rather by 'git-diff-tree.txt' including 'pretty-formats.txt' and 'diff-format.txt' on subsequent lines, while the former happens to end with monospace-formatted example commands.

Fix this by inserting an empty line between the two include::
directives.

The page rendered with AsciiDoc doesn't have this formatting issue.

---
 Documentation/git-diff-tree.txt | 1 +
 1 file changed, 1 insertion(+)
Documentation/git-diff-tree.txt +1 -0
index 24f32e8c54..5c8a2a5e97 100644
@@ -118,6 +118,7 @@ include::pretty-options.txt[]
 
 
 include::pretty-formats.txt[]
+
 include::diff-format.txt[]
 
 GIT
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#1 lore raw

PATCH 2/3 Documentation/technical/api-config.txt: fix formatting

Some more recent versions of Asciidoctor issue the following warning while building the documentation:

ASCIIDOC technical/api-config.html
asciidoctor: WARNING: api-config.txt: line 232: unterminated listing block

This highlight an issue where the length of the '----' lines surrounding a code example don't match, and the rest of the document is rendered in monospace.

Fix this by making sure that the length of those lines match.

The page rendered with AsciiDoc doesn't have this formatting issue.

---
 Documentation/technical/api-config.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/technical/api-config.txt +1 -1
index fa39ac9d71..7d20716c32 100644
@@ -229,7 +229,7 @@ A `config_set` can be used to construct an in-memory cache for
 config-like files that the caller specifies (i.e., files like `.gitmodules`,
 `~/.gitconfig` etc.). For example,
 
----------------------------------------
+----------------------------------------
 struct config_set gm_config;
 git_configset_init(&gm_config);
 int b;
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#2 lore raw

PATCH 3/3 Documentation/technical/protocol-v2.txt: fix formatting

In reply to SZEDER Gábor

Some more recent versions of Asciidoctor issue the following warning while building the documentation:

ASCIIDOC technical/protocol-v2.html
asciidoctor: WARNING: protocol-v2.txt: line 38: unterminated listing block

This highlights an issue where the 'Initial Client Request' header is not rendered as a header but in monospace. I'm not sure what exactly causes this issue and why it's an issue only with this particular header, but all headers in 'protocol-v2.txt' are written like this:

Initial Client Request
------------------------

i.e. the header itself is indented by a space, and the "underline"

Dropping that indentation and making the length of the underline match the length of the header apparently fixes this issue.

While at it, adjust all other headers 'protocol-v2.txt' as well, to match the style we use everywhere else.

The page rendered with AsciiDoc doesn't have this formatting issue.

---
 Documentation/technical/protocol-v2.txt | 52 ++++++++++++-------------
 1 file changed, 26 insertions(+), 26 deletions(-)
Documentation/technical/protocol-v2.txt +26 -26
index ead85ce35c..03264c7d9a 100644
@@ -1,5 +1,5 @@
- Git Wire Protocol, Version 2
-==============================
+Git Wire Protocol, Version 2
+============================
 
 This document presents a specification for a version 2 of Git's wire
 protocol.  Protocol v2 will improve upon v1 in the following ways:
@@ -22,8 +22,8 @@ will be commands which a client can request be executed.  Once a command
 has completed, a client can reuse the connection and request that other
 commands be executed.
 
- Packet-Line Framing
----------------------
+Packet-Line Framing
+-------------------
 
 All communication is done using packet-line framing, just as in v1.  See
 `Documentation/technical/pack-protocol.txt` and
@@ -34,8 +34,8 @@ In protocol v2 these special packets will have the following semantics:
   * '0000' Flush Packet (flush-pkt) - indicates the end of a message
   * '0001' Delimiter Packet (delim-pkt) - separates sections of a message
 
- Initial Client Request
-------------------------
+Initial Client Request
+----------------------
 
 In general a client can request to speak protocol v2 by sending
 `version=2` through the respective side-channel for the transport being
@@ -43,22 +43,22 @@ used which inevitably sets `GIT_PROTOCOL`.  More information can be
 found in `pack-protocol.txt` and `http-protocol.txt`.  In all cases the
 response from the server is the capability advertisement.
 
- Git Transport
-~~~~~~~~~~~~~~~
+Git Transport
+~~~~~~~~~~~~~
 
 When using the git:// transport, you can request to use protocol v2 by
 sending "version=2" as an extra parameter:
 
    003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0
 
- SSH and File Transport
-~~~~~~~~~~~~~~~~~~~~~~~~
+SSH and File Transport
+~~~~~~~~~~~~~~~~~~~~~~
 
 When using either the ssh:// or file:// transport, the GIT_PROTOCOL
 environment variable must be set explicitly to include "version=2".
 
- HTTP Transport
-~~~~~~~~~~~~~~~~
+HTTP Transport
+~~~~~~~~~~~~~~
 
 When using the http:// or https:// transport a client makes a "smart"
 info/refs request as described in `http-protocol.txt` and requests that
@@ -79,8 +79,8 @@ A v2 server would reply:
 Subsequent requests are then made directly to the service
 `$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack).
 
- Capability Advertisement
---------------------------
+Capability Advertisement
+------------------------
 
 A server which decides to communicate (based on a request from a client)
 using protocol version 2, notifies the client by sending a version string
@@ -101,8 +101,8 @@ to be executed by the client.
     key = 1*(ALPHA | DIGIT | "-_")
     value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;")
 
- Command Request
------------------
+Command Request
+---------------
 
 After receiving the capability advertisement, a client can then issue a
 request to select the command it wants with any particular capabilities
@@ -137,8 +137,8 @@ command be executed or can terminate the connection.  A client may
 optionally send an empty request consisting of just a flush-pkt to
 indicate that no more requests will be made.
 
- Capabilities
---------------
+Capabilities
+------------
 
 There are two different types of capabilities: normal capabilities,
 which can be used to to convey information or alter the behavior of a
@@ -153,8 +153,8 @@ management on the server side in order to function correctly.  This
 permits simple round-robin load-balancing on the server side, without
 needing to worry about state management.
 
- agent
-~~~~~~~
+agent
+~~~~~
 
 The server can advertise the `agent` capability with a value `X` (in the
 form `agent=X`) to notify the client that the server is running version
@@ -168,8 +168,8 @@ printable ASCII characters except space (i.e., the byte range 32 < x <
 and debugging purposes, and MUST NOT be used to programmatically assume
 the presence or absence of particular features.
 
- ls-refs
-~~~~~~~~~
+ls-refs
+~~~~~~~
 
 `ls-refs` is the command used to request a reference advertisement in v2.
 Unlike the current reference advertisement, ls-refs takes in arguments
@@ -199,8 +199,8 @@ The output of ls-refs is as follows:
     symref = "symref-target:" symref-target
     peeled = "peeled:" obj-id
 
- fetch
-~~~~~~~
+fetch
+~~~~~
 
 `fetch` is the command used to fetch a packfile in v2.  It can be looked
 at as a modified version of the v1 fetch where the ref-advertisement is
@@ -444,8 +444,8 @@ header.
 		2 - progress messages
 		3 - fatal error message just before stream aborts
 
- server-option
-~~~~~~~~~~~~~~~
+server-option
+~~~~~~~~~~~~~
 
 If advertised, indicates that any number of server specific options can be
 included in a request.  This is done by sending each option as a
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#3 lore raw

Re: [PATCH 3/3] Documentation/technical/protocol-v2.txt: fix formatting

Quoted 11 lines · SZEDER Gábor

Some more recent versions of Asciidoctor issue the following warning while building the documentation:

ASCIIDOC technical/protocol-v2.html
asciidoctor: WARNING: protocol-v2.txt: line 38: unterminated listing block

This highlights an issue where the 'Initial Client Request' header is not rendered as a header but in monospace. I'm not sure what exactly causes this issue and why it's an issue only with this particular header, but all headers in 'protocol-v2.txt' are written like this:

Quoted 4 lines

Initial Client Request
------------------------

i.e. the header itself is indented by a space, and the "underline"

Hrm, I don't know where the rest of the sentence went, probably some unnoticed mishap during rebase/reword... Anyway, it should read:

i.e. the header itself is indented by a space, and the "underline" is two characters longer than the header.

Quoted 156 lines

Dropping that indentation and making the length of the underline match the length of the header apparently fixes this issue.

While at it, adjust all other headers 'protocol-v2.txt' as well, to match the style we use everywhere else.

The page rendered with AsciiDoc doesn't have this formatting issue.

---
 Documentation/technical/protocol-v2.txt | 52 ++++++++++++-------------
 1 file changed, 26 insertions(+), 26 deletions(-)
Documentation/technical/protocol-v2.txt +26 -26
index ead85ce35c..03264c7d9a 100644
@@ -1,5 +1,5 @@
- Git Wire Protocol, Version 2
-==============================
+Git Wire Protocol, Version 2
+============================
 
 This document presents a specification for a version 2 of Git's wire
 protocol.  Protocol v2 will improve upon v1 in the following ways:
@@ -22,8 +22,8 @@ will be commands which a client can request be executed.  Once a command
 has completed, a client can reuse the connection and request that other
 commands be executed.
 
- Packet-Line Framing
----------------------
+Packet-Line Framing
+-------------------
 
 All communication is done using packet-line framing, just as in v1.  See
 `Documentation/technical/pack-protocol.txt` and
@@ -34,8 +34,8 @@ In protocol v2 these special packets will have the following semantics:
   * '0000' Flush Packet (flush-pkt) - indicates the end of a message
   * '0001' Delimiter Packet (delim-pkt) - separates sections of a message
 
- Initial Client Request
-------------------------
+Initial Client Request
+----------------------
 
 In general a client can request to speak protocol v2 by sending
 `version=2` through the respective side-channel for the transport being
@@ -43,22 +43,22 @@ used which inevitably sets `GIT_PROTOCOL`.  More information can be
 found in `pack-protocol.txt` and `http-protocol.txt`.  In all cases the
 response from the server is the capability advertisement.
 
- Git Transport
-~~~~~~~~~~~~~~~
+Git Transport
+~~~~~~~~~~~~~
 
 When using the git:// transport, you can request to use protocol v2 by
 sending "version=2" as an extra parameter:
 
    003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0
 
- SSH and File Transport
-~~~~~~~~~~~~~~~~~~~~~~~~
+SSH and File Transport
+~~~~~~~~~~~~~~~~~~~~~~
 
 When using either the ssh:// or file:// transport, the GIT_PROTOCOL
 environment variable must be set explicitly to include "version=2".
 
- HTTP Transport
-~~~~~~~~~~~~~~~~
+HTTP Transport
+~~~~~~~~~~~~~~
 
 When using the http:// or https:// transport a client makes a "smart"
 info/refs request as described in `http-protocol.txt` and requests that
@@ -79,8 +79,8 @@ A v2 server would reply:
 Subsequent requests are then made directly to the service
 `$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack).
 
- Capability Advertisement
---------------------------
+Capability Advertisement
+------------------------
 
 A server which decides to communicate (based on a request from a client)
 using protocol version 2, notifies the client by sending a version string
@@ -101,8 +101,8 @@ to be executed by the client.
     key = 1*(ALPHA | DIGIT | "-_")
     value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;")
 
- Command Request
------------------
+Command Request
+---------------
 
 After receiving the capability advertisement, a client can then issue a
 request to select the command it wants with any particular capabilities
@@ -137,8 +137,8 @@ command be executed or can terminate the connection.  A client may
 optionally send an empty request consisting of just a flush-pkt to
 indicate that no more requests will be made.
 
- Capabilities
---------------
+Capabilities
+------------
 
 There are two different types of capabilities: normal capabilities,
 which can be used to to convey information or alter the behavior of a
@@ -153,8 +153,8 @@ management on the server side in order to function correctly.  This
 permits simple round-robin load-balancing on the server side, without
 needing to worry about state management.
 
- agent
-~~~~~~~
+agent
+~~~~~
 
 The server can advertise the `agent` capability with a value `X` (in the
 form `agent=X`) to notify the client that the server is running version
@@ -168,8 +168,8 @@ printable ASCII characters except space (i.e., the byte range 32 < x <
 and debugging purposes, and MUST NOT be used to programmatically assume
 the presence or absence of particular features.
 
- ls-refs
-~~~~~~~~~
+ls-refs
+~~~~~~~
 
 `ls-refs` is the command used to request a reference advertisement in v2.
 Unlike the current reference advertisement, ls-refs takes in arguments
@@ -199,8 +199,8 @@ The output of ls-refs is as follows:
     symref = "symref-target:" symref-target
     peeled = "peeled:" obj-id
 
- fetch
-~~~~~~~
+fetch
+~~~~~
 
 `fetch` is the command used to fetch a packfile in v2.  It can be looked
 at as a modified version of the v1 fetch where the ref-advertisement is
@@ -444,8 +444,8 @@ header.
 		2 - progress messages
 		3 - fatal error message just before stream aborts
 
- server-option
-~~~~~~~~~~~~~~~
+server-option
+~~~~~~~~~~~~~
 
 If advertised, indicates that any number of server specific options can be
 included in a request.  This is done by sending each option as a
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#4 lore raw

v2 cover Asciidoctor-related formatting and CI fixes

In reply to SZEDER Gábor

The first three patches fix formatting issues with Asciidoctor; they are the same patches that I sent earlier today, except some commit message updates (now they mention the Asciidoctor version which started to issue the warning, and I fixed that missing half-sentence in one of the commit messages).

The last three patches are a small cleanup and fixes to the documentation CI build jobs; notably patch 5 un-breaks the documentation build job on Travis CI in the era of Asciidoctor v2.0.0 [1], and patch 5 fixes some forever-broken checks.

The formatting and CI fixes are not really related, but the check fixed in patch 6 would choke without the formatting fixes silencing warnings from Asciidoctor, that's why I send them in the same patch series.

[1] https://public-inbox.org/git/[email protected]/

SZEDER Gábor (6):
Documentation/git-diff-tree.txt: fix formatting Documentation/technical/api-config.txt: fix formatting Documentation/technical/protocol-v2.txt: fix formatting
ci: install Asciidoctor in 'ci/install-dependencies.sh'
ci: stick with Asciidoctor v1.5.8 for now
ci: fix AsciiDoc/Asciidoctor stderr check in the documentation build
job

 Documentation/git-diff-tree.txt         |  1 +
 Documentation/technical/api-config.txt  |  2 +-
 Documentation/technical/protocol-v2.txt | 52 ++++++++++++-------------
 ci/install-dependencies.sh              |  3 ++
 ci/test-documentation.sh                | 15 ++++---
 5 files changed, 38 insertions(+), 35 deletions(-)
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#5 lore raw

PATCH v2 1/6 Documentation/git-diff-tree.txt: fix formatting

Asciidoctor versions v1.5.7 or later print the following warning while building the documentation:

ASCIIDOC git-diff-tree.xml
asciidoctor: WARNING: diff-format.txt: line 2: unterminated listing block

This highlights an issue (even with older Asciidoctor versions) where the "Raw output format" header is not rendered as a header, and the rest of the document is rendered in monospace. This is not caused by 'diff-format.txt' in itself, but rather by 'git-diff-tree.txt' including 'pretty-formats.txt' and 'diff-format.txt' on subsequent lines, while the former happens to end with monospace-formatted example commands.

Fix this by inserting an empty line between the two include::
directives.

The page rendered with AsciiDoc doesn't have this formatting issue.

---
 Documentation/git-diff-tree.txt | 1 +
 1 file changed, 1 insertion(+)
Documentation/git-diff-tree.txt +1 -0
index 24f32e8c54..5c8a2a5e97 100644
@@ -118,6 +118,7 @@ include::pretty-options.txt[]
 
 
 include::pretty-formats.txt[]
+
 include::diff-format.txt[]
 
 GIT
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#6 lore raw

PATCH v2 2/6 Documentation/technical/api-config.txt: fix formatting

In reply to SZEDER Gábor

Asciidoctor versions v1.5.7 or later print the following warning while building the documentation:

ASCIIDOC technical/api-config.html
asciidoctor: WARNING: api-config.txt: line 232: unterminated listing block

This highlight an issue (even with older Asciidoctor versions) where the length of the '----' lines surrounding a code example don't match, and the rest of the document is rendered in monospace.

Fix this by making sure that the length of those lines match.

The page rendered with AsciiDoc doesn't have this formatting issue.

---
 Documentation/technical/api-config.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/technical/api-config.txt +1 -1
index fa39ac9d71..7d20716c32 100644
@@ -229,7 +229,7 @@ A `config_set` can be used to construct an in-memory cache for
 config-like files that the caller specifies (i.e., files like `.gitmodules`,
 `~/.gitconfig` etc.). For example,
 
----------------------------------------
+----------------------------------------
 struct config_set gm_config;
 git_configset_init(&gm_config);
 int b;
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#7 lore raw

PATCH v2 3/6 Documentation/technical/protocol-v2.txt: fix formatting

In reply to SZEDER Gábor

Asciidoctor versions v1.5.7 or later print the following warning while building the documentation:

ASCIIDOC technical/protocol-v2.html
asciidoctor: WARNING: protocol-v2.txt: line 38: unterminated listing block

This highlights an issue (even with older Asciidoctor versions) where the 'Initial Client Request' header is not rendered as a header but in monospace. I'm not sure what exactly causes this issue and why it's an issue only with this particular header, but all headers in 'protocol-v2.txt' are written like this:

Initial Client Request
------------------------

i.e. the header itself is indented by a space, and the "underline" is two characters longer than the header.

Dropping that indentation and making the length of the underline match the length of the header apparently fixes this issue.

While at it, adjust all other headers 'protocol-v2.txt' as well, to match the style we use everywhere else.

The page rendered with AsciiDoc doesn't have this formatting issue.

---
 Documentation/technical/protocol-v2.txt | 52 ++++++++++++-------------
 1 file changed, 26 insertions(+), 26 deletions(-)
Documentation/technical/protocol-v2.txt +26 -26
index ead85ce35c..03264c7d9a 100644
@@ -1,5 +1,5 @@
- Git Wire Protocol, Version 2
-==============================
+Git Wire Protocol, Version 2
+============================
 
 This document presents a specification for a version 2 of Git's wire
 protocol.  Protocol v2 will improve upon v1 in the following ways:
@@ -22,8 +22,8 @@ will be commands which a client can request be executed.  Once a command
 has completed, a client can reuse the connection and request that other
 commands be executed.
 
- Packet-Line Framing
----------------------
+Packet-Line Framing
+-------------------
 
 All communication is done using packet-line framing, just as in v1.  See
 `Documentation/technical/pack-protocol.txt` and
@@ -34,8 +34,8 @@ In protocol v2 these special packets will have the following semantics:
   * '0000' Flush Packet (flush-pkt) - indicates the end of a message
   * '0001' Delimiter Packet (delim-pkt) - separates sections of a message
 
- Initial Client Request
-------------------------
+Initial Client Request
+----------------------
 
 In general a client can request to speak protocol v2 by sending
 `version=2` through the respective side-channel for the transport being
@@ -43,22 +43,22 @@ used which inevitably sets `GIT_PROTOCOL`.  More information can be
 found in `pack-protocol.txt` and `http-protocol.txt`.  In all cases the
 response from the server is the capability advertisement.
 
- Git Transport
-~~~~~~~~~~~~~~~
+Git Transport
+~~~~~~~~~~~~~
 
 When using the git:// transport, you can request to use protocol v2 by
 sending "version=2" as an extra parameter:
 
    003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0
 
- SSH and File Transport
-~~~~~~~~~~~~~~~~~~~~~~~~
+SSH and File Transport
+~~~~~~~~~~~~~~~~~~~~~~
 
 When using either the ssh:// or file:// transport, the GIT_PROTOCOL
 environment variable must be set explicitly to include "version=2".
 
- HTTP Transport
-~~~~~~~~~~~~~~~~
+HTTP Transport
+~~~~~~~~~~~~~~
 
 When using the http:// or https:// transport a client makes a "smart"
 info/refs request as described in `http-protocol.txt` and requests that
@@ -79,8 +79,8 @@ A v2 server would reply:
 Subsequent requests are then made directly to the service
 `$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack).
 
- Capability Advertisement
---------------------------
+Capability Advertisement
+------------------------
 
 A server which decides to communicate (based on a request from a client)
 using protocol version 2, notifies the client by sending a version string
@@ -101,8 +101,8 @@ to be executed by the client.
     key = 1*(ALPHA | DIGIT | "-_")
     value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;")
 
- Command Request
------------------
+Command Request
+---------------
 
 After receiving the capability advertisement, a client can then issue a
 request to select the command it wants with any particular capabilities
@@ -137,8 +137,8 @@ command be executed or can terminate the connection.  A client may
 optionally send an empty request consisting of just a flush-pkt to
 indicate that no more requests will be made.
 
- Capabilities
---------------
+Capabilities
+------------
 
 There are two different types of capabilities: normal capabilities,
 which can be used to to convey information or alter the behavior of a
@@ -153,8 +153,8 @@ management on the server side in order to function correctly.  This
 permits simple round-robin load-balancing on the server side, without
 needing to worry about state management.
 
- agent
-~~~~~~~
+agent
+~~~~~
 
 The server can advertise the `agent` capability with a value `X` (in the
 form `agent=X`) to notify the client that the server is running version
@@ -168,8 +168,8 @@ printable ASCII characters except space (i.e., the byte range 32 < x <
 and debugging purposes, and MUST NOT be used to programmatically assume
 the presence or absence of particular features.
 
- ls-refs
-~~~~~~~~~
+ls-refs
+~~~~~~~
 
 `ls-refs` is the command used to request a reference advertisement in v2.
 Unlike the current reference advertisement, ls-refs takes in arguments
@@ -199,8 +199,8 @@ The output of ls-refs is as follows:
     symref = "symref-target:" symref-target
     peeled = "peeled:" obj-id
 
- fetch
-~~~~~~~
+fetch
+~~~~~
 
 `fetch` is the command used to fetch a packfile in v2.  It can be looked
 at as a modified version of the v1 fetch where the ref-advertisement is
@@ -444,8 +444,8 @@ header.
 		2 - progress messages
 		3 - fatal error message just before stream aborts
 
- server-option
-~~~~~~~~~~~~~~~
+server-option
+~~~~~~~~~~~~~
 
 If advertised, indicates that any number of server specific options can be
 included in a request.  This is done by sending each option as a
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#8 lore raw

PATCH v2 4/6 ci: install Asciidoctor in 'ci/install-dependencies.sh'

In reply to SZEDER Gábor

When our '.travis.yml' was split into several 'ci/*' scripts [1], the installation of the 'asciidoctor' gem somehow ended up in 'ci/test-documentation.sh'.

Install it in 'ci/install-dependencies.sh', where we install everything else.

[1] 657343a602 (travis-ci: move Travis CI code into dedicated scripts,

    2017-09-10)
---
 ci/install-dependencies.sh | 3 +++
 ci/test-documentation.sh   | 3 ---
 2 files changed, 3 insertions(+), 3 deletions(-)
ci/install-dependencies.sh +3 -0
index d64667fcbf..76ec308965 100755
@@ -54,6 +54,9 @@ StaticAnalysis)
 Documentation)
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto
+
+	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
+	gem install asciidoctor
 	;;
 esac
 
ci/test-documentation.sh +0 -3
index be3b7d376a..8f91f48c81 100755
@@ -5,9 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
-gem install asciidoctor
-
 make check-builtins
 make check-docs
 
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#13 lore raw

PATCH v2 6/6 ci: fix AsciiDoc/Asciidoctor stderr check in the documentation build job

In reply to SZEDER Gábor

In our 'ci/*' build scripts we rely on 'set -e' aborting the build job when a command exits with error, while in 'ci/test-documentation.sh' we tried to check the emptiness of AsciiDoc's and Asciidoctor's standard error with '! test -s stderr.log'. Unfortunately, the combination of the two doesn't work as intended, because, according to POSIX [1]:

"The -e setting shall be ignored when executing [...] a pipeline beginning with the ! reserved word" [2]

Watch and learn:

  $ cat e.sh
  #!/bin/sh
  set -ex
  echo unexpected >file
  ! test -s file
  test ! -s file
  echo "should not reach this"
  $ ./e.sh
  + echo unexpected
  + test -s file
  + test ! -s file
  $

This is why we haven't noticed the warnings from Asciidoctor that were fixed in the first patches of this patch series, though some of them were already there in the build of v2.18.0-rc0 [3].

Check the emptiness of that file with 'test ! -s' instead, which works properly with 'set -e'. Furthermore, dump the contents of that file to the log for our convenience, so if it were to unexpectedly end up being non-empty, then we wouldn't have to scroll through all that long build log looking for warnings, but could see them right away near the end of the log.

And one more thing: we build the docs with Asciidoctor right after a 'make clean', meaning that 'make doc' starts with running 'GIT-VERSION-GEN', which in turn prints the version to its standard error. This version then goes to 'stderr.log' as well, and a 'sed' command is supposed to remove it to prevent it from interfering with that (previously defunct) emptiness check. Alas, this command doesn't work as intended, either, because it leaves the file to be checked intact, but the defunct emptiness check hid this issue, too... This patch deals with this as well.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#set

[2] POSIX doesn't discuss the meaning of '! cmd' in case of simple commands, but it defines that "A pipeline is a sequence of one or more commands separated by the control operator '|'", so apparently a simple command is considered as pipeline as well.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_02

[3] https://travis-ci.org/git/git/jobs/385932007#L1463

---
 ci/test-documentation.sh | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
ci/test-documentation.sh +7 -5
index 8f91f48c81..bd4a65e933 100755
@@ -10,7 +10,8 @@ make check-docs
 
 # Build docs with AsciiDoc
 make doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
-! test -s stderr.log
+cat stderr.log
+test ! -s stderr.log
 test -s Documentation/git.html
 test -s Documentation/git.xml
 test -s Documentation/git.1
@@ -21,13 +22,14 @@ check_unignored_build_artifacts
 
 # Build docs with AsciiDoctor
 make clean
-make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
-sed '/^GIT_VERSION = / d' stderr.log
-! test -s stderr.log
+make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
+sed '/^GIT_VERSION = / d' stderr.raw >stderr.log
+cat stderr.log
+test ! -s stderr.log
 test -s Documentation/git.html
 grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
-rm -f stdout.log stderr.log
+rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
 
 save_good_tree
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#12 lore raw

PATCH v2 5/6 ci: stick with Asciidoctor v1.5.8 for now

In reply to SZEDER Gábor

The release of Asciidoctor v2.0.0 two days ago broke our documentation build job on Travis CI, where we 'gem install asciidoctor', which always brings us the latest and (supposedly) greatest. Alas, we are not ready for that just yet, because it removed support for DocBook 4.5, and we have been requiring that particular DocBook version to build 'user-manual.xml' with Asciidoctor, resulting in:

ASCIIDOC user-manual.xml
asciidoctor: FAILED: missing converter for backend 'docbook45'. Processing aborted.

    Use --trace for backtrace

make[1]: *** [user-manual.xml] Error 1

Unfortunately, we can't simply switch to DocBook 5 right away, as doing so leads to validation errors from 'xmlto', and working around those leads to yet another errors... [1]

So let's stick with Asciidoctor v1.5.8 (latest stable release before v2.0.0) in our documentation build job on Travis CI for now, until we figure out how to deal with the fallout from Asciidoctor v2.0.0.

[1] https://public-inbox.org/git/[email protected]/

---
 ci/install-dependencies.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
ci/install-dependencies.sh +1 -1
index 76ec308965..52a44c690a 100755
@@ -56,7 +56,7 @@ Documentation)
 	sudo apt-get -q -y install asciidoc xmlto
 
 	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
-	gem install asciidoctor
+	gem install --version 1.5.8 asciidoctor
 	;;
 esac
 
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#9 lore raw

Re: [PATCH v2 4/6] ci: install Asciidoctor in 'ci/install-dependencies.sh'

In reply to SZEDER Gábor

Hi,

I like the rest of the patch series, but this patch, I am not so sure about...

Quoted 6 lines · SZEDER Gábor

When our '.travis.yml' was split into several 'ci/*' scripts [1], the installation of the 'asciidoctor' gem somehow ended up in 'ci/test-documentation.sh'.

Install it in 'ci/install-dependencies.sh', where we install everything else.

The big difference you introduce is that asciidoctor is now installed with every job, not only with the Documentation job that actually uses it.

Even if it affects me very little (because I don't pay much attention to Travis, it's been too flakey for me, and it does not test our Windows side, and it is too slow), I'd rather install asciidoctor really only when needed.

So I'd like to recommend to drop this patch from the series.

Thanks,
Dscho

Quoted 42 lines

[1] 657343a602 (travis-ci: move Travis CI code into dedicated scripts,

    2017-09-10)
---
 ci/install-dependencies.sh | 3 +++
 ci/test-documentation.sh   | 3 ---
 2 files changed, 3 insertions(+), 3 deletions(-)
ci/install-dependencies.sh +3 -0
index d64667fcbf..76ec308965 100755
@@ -54,6 +54,9 @@ StaticAnalysis)
 Documentation)
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto
+
+	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
+	gem install asciidoctor
 	;;
 esac
 
ci/test-documentation.sh +0 -4
index be3b7d376a..8f91f48c81 100755
@@ -5,9 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
-gem install asciidoctor
-
 make check-builtins
 make check-docs
 
--

2.21.0.539.g07239c3a71.dirty

#10 lore raw

Re: [PATCH v2 4/6] ci: install Asciidoctor in 'ci/install-dependencies.sh'

Quoted 11 lines · Johannes Schindelin

Hi,

I like the rest of the patch series, but this patch, I am not so sure about...

Quoted 4 lines · SZEDER Gábor

When our '.travis.yml' was split into several 'ci/*' scripts [1], the installation of the 'asciidoctor' gem somehow ended up in 'ci/test-documentation.sh'.

Quoted 5 lines
Quoted 2 lines

Install it in 'ci/install-dependencies.sh', where we install everything else.

The big difference you introduce is that asciidoctor is now installed with every job, not only with the Documentation job that actually uses it.

It is only installed in the 'Documentation' build job.

Quoted 20 lines

Even if it affects me very little (because I don't pay much attention to Travis, it's been too flakey for me, and it does not test our Windows side, and it is too slow), I'd rather install asciidoctor really only when needed.

So I'd like to recommend to drop this patch from the series.

Thanks,
Dscho

Quoted 10 lines

[1] 657343a602 (travis-ci: move Travis CI code into dedicated scripts,

    2017-09-10)
---
 ci/install-dependencies.sh | 3 +++
 ci/test-documentation.sh   | 3 ---
 2 files changed, 3 insertions(+), 3 deletions(-)
Quoted 6 lines

ci/install-dependencies.sh StaticAnalysis) · line 54

Quoted 6 lines
ci/install-dependencies.sh +0 -0
index d64667fcbf..76ec308965 100755
@@ -54,6 +54,9 @@ StaticAnalysis)
 Documentation)
     ^^^^^^^^^^^^^^
Quoted 26 lines
Quoted 26 lines
ci/install-dependencies.sh +3 -0
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto
+
+	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
+	gem install asciidoctor
 	;;
 esac
 
ci/test-documentation.sh +0 -4
index be3b7d376a..8f91f48c81 100755
@@ -5,9 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
-gem install asciidoctor
-
 make check-builtins
 make check-docs
 
--

2.21.0.539.g07239c3a71.dirty

#11 lore raw

Re: [PATCH v2 4/6] ci: install Asciidoctor in 'ci/install-dependencies.sh'

Hi Gábor,

Quoted 11 lines · SZEDER Gábor
Quoted 9 lines · Johannes Schindelin

I like the rest of the patch series, but this patch, I am not so sure about...

Quoted 4 lines · SZEDER Gábor

When our '.travis.yml' was split into several 'ci/*' scripts [1], the installation of the 'asciidoctor' gem somehow ended up in 'ci/test-documentation.sh'.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Install it in 'ci/install-dependencies.sh', where we install everything else.

The big difference you introduce is that asciidoctor is now installed with every job, not only with the Documentation job that actually uses it.

It is only installed in the 'Documentation' build job.

Oy, you're right!

Quoted 9 lines
Quoted 9 lines

Even if it affects me very little (because I don't pay much attention to Travis, it's been too flakey for me, and it does not test our Windows side, and it is too slow), I'd rather install asciidoctor really only when needed.

So I'd like to recommend to drop this patch from the series.

[...]

Quoted 8 lines
Quoted 6 lines

ci/install-dependencies.sh StaticAnalysis) · line 54

Quoted 6 lines
ci/install-dependencies.sh +0 -0
index d64667fcbf..76ec308965 100755
@@ -54,6 +54,9 @@ StaticAnalysis)
 Documentation)
     ^^^^^^^^^^^^^^

To be honest, I think it was kind of easy to miss that. Maybe the commit message could talk about this, so that future Dscho will have a helpful hint when looking at the commit history and stumbling over this move?

If you could add that hint, I would be most grateful.

In any case, based on what you taught me I retract my objection to this patch.

Thanks,
Dscho

Quoted 29 lines
Quoted 27 lines
Quoted 27 lines
ci/install-dependencies.sh +3 -0
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto
+
+	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
+	gem install asciidoctor
 	;;
 esac
 
ci/test-documentation.sh +0 -4
index be3b7d376a..8f91f48c81 100755
@@ -5,9 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
-gem install asciidoctor
-
 make check-builtins
 make check-docs
 
--

2.21.0.539.g07239c3a71.dirty

#14 lore raw

Re: [PATCH v2 0/6] Asciidoctor-related formatting and CI fixes

In reply to SZEDER Gábor
Quoted 11 lines · SZEDER Gábor

The first three patches fix formatting issues with Asciidoctor; they are the same patches that I sent earlier today, except some commit message updates (now they mention the Asciidoctor version which started to issue the warning, and I fixed that missing half-sentence in one of the commit messages).

The last three patches are a small cleanup and fixes to the documentation CI build jobs; notably patch 5 un-breaks the documentation build job on Travis CI in the era of Asciidoctor v2.0.0 [1], and patch 5 fixes some forever-broken checks.

Quoted 4 lines

The formatting and CI fixes are not really related, but the check fixed in patch 6 would choke without the formatting fixes silencing warnings from Asciidoctor, that's why I send them in the same patch series.

Thanks, these all look good to me (I do agree with Dscho that the "Documentation" subtlety in patch 4 might be worth calling out in the commit message).

-Peff

#15 lore raw

v3 cover Asciidoctor-related formatting and CI fixes

In reply to SZEDER Gábor

The first three patches fix formatting issues with Asciidoctor, while the last three patches are a small cleanup and fixes to the documentation CI build jobs; notably patch 5 un-breaks the documentation build job on Travis CI in the era of Asciidoctor v2.0.0 [1], and patch 6 fixes some forever-broken checks.

Changes since v2:

- The fixed stderr check in patch 6 is triggered when merged with 'pu', in particular with commit 9a71722b4d (Doc: auto-detect changed build flags, 2019-03-17) in 'ma/doc-diff-doc-vs-doctor-comparison'. The changes to patch 6 deal with that.

- Mention in patch 4's commit message that Asciidoctor is still only installed in the Documentation build job.

[1] https://public-inbox.org/git/[email protected]/

SZEDER Gábor (6):
Documentation/git-diff-tree.txt: fix formatting Documentation/technical/api-config.txt: fix formatting Documentation/technical/protocol-v2.txt: fix formatting
ci: install Asciidoctor in 'ci/install-dependencies.sh'
ci: stick with Asciidoctor v1.5.8 for now
ci: fix AsciiDoc/Asciidoctor stderr check in the documentation build
job

 Documentation/git-diff-tree.txt         |  1 +
 Documentation/technical/api-config.txt  |  2 +-
 Documentation/technical/protocol-v2.txt | 52 ++++++++++++-------------
 ci/install-dependencies.sh              |  3 ++
 ci/test-documentation.sh                | 24 +++++++-----
 5 files changed, 46 insertions(+), 36 deletions(-)

Range-diff:

1:  8026f62876 = 1:  8026f62876 Documentation/git-diff-tree.txt: fix formatting
2:  fd19cf4b24 = 2:  fd19cf4b24 Documentation/technical/api-config.txt: fix formatting
3:  638dcd64e9 = 3:  638dcd64e9 Documentation/technical/protocol-v2.txt: fix formatting
4:  6f8c6ff398 ! 4:  2e94e2b7b6 ci: install Asciidoctor in 'ci/install-dependencies.sh'

@@ -6,8 +6,9 @@
installation of the 'asciidoctor' gem somehow ended up in 'ci/test-documentation.sh'.

- Install it in 'ci/install-dependencies.sh', where we install
- everything else.
+ Install it in 'ci/install-dependencies.sh', where we install other
+ dependencies of the Documentation build job as well (asciidoc,
+ xmlto).

[1] 657343a602 (travis-ci: move Travis CI code into dedicated scripts, 2017-09-10)

5:  b9eb371f81 = 5:  95476591dd ci: stick with Asciidoctor v1.5.8 for now
6:  ba2a005a05 ! 6:  04ee6831fd ci: fix AsciiDoc/Asciidoctor stderr check in the documentation build job

@@ -2,51 +2,62 @@

ci: fix AsciiDoc/Asciidoctor stderr check in the documentation build job

- In our 'ci/*' build scripts we rely on 'set -e' aborting the build job
- when a command exits with error, while in 'ci/test-documentation.sh'
- we tried to check the emptiness of AsciiDoc's and Asciidoctor's
- standard error with '! test -s stderr.log'. Unfortunately, the
- combination of the two doesn't work as intended, because, according to
- POSIX [1]:
+ In 'ci/test-documentation.sh' we save the standard error of 'make
+ doc', and, in an attempt to make sure that neither AsciiDoc nor
+ Asciidoctor printed any warnings, we check the emptiness of the
+ resulting file with '! test -s stderr.log'. This check has never
+ actually worked, because in our 'ci/*' build scripts we rely on 'set
+ -e' aborting the build job when a command exits with error, and,
+ unfortunately, the combination of the two doesn't work as intended.
+ According to POSIX [1]:

"The -e setting shall be ignored when executing [...] a pipeline beginning with the ! reserved word" [2]

Watch and learn:

- $ cat e.sh
- #!/bin/sh
- set -ex
- echo unexpected >file
- ! test -s file
- test ! -s file
- echo "should not reach this"
- $ ./e.sh
- + echo unexpected
- + test -s file
- + test ! -s file
- $
+ $ echo unexpected >file
+ $ ( set -e; ! test -s file ; echo "should not reach this" ) ; echo $?
+ should not reach this
+ 0

This is why we haven't noticed the warnings from Asciidoctor that were fixed in the first patches of this patch series, though some of them were already there in the build of v2.18.0-rc0 [3].

Check the emptiness of that file with 'test ! -s' instead, which works
- properly with 'set -e'. Furthermore, dump the contents of that file
- to the log for our convenience, so if it were to unexpectedly end up
- being non-empty, then we wouldn't have to scroll through all that long
- build log looking for warnings, but could see them right away near the
- end of the log.
-
- And one more thing: we build the docs with Asciidoctor right after a
- 'make clean', meaning that 'make doc' starts with running
- 'GIT-VERSION-GEN', which in turn prints the version to its standard
- error. This version then goes to 'stderr.log' as well, and a 'sed'
- command is supposed to remove it to prevent it from interfering with
- that (previously defunct) emptiness check. Alas, this command doesn't
- work as intended, either, because it leaves the file to be checked
- intact, but the defunct emptiness check hid this issue, too... This
- patch deals with this as well.
+ properly with 'set -e':
+
+ $ ( set -e; test ! -s file ; echo "should not reach this" ) ; echo $?
+ 1
+
+ Furthermore, dump the contents of that file to the log for our
+ convenience, so if it were to unexpectedly end up being non-empty,
+ then we wouldn't have to scroll through all that long build log
+ looking for warnings, but could see them right away near the end of
+ the log.
+
+ Note that we are only really interested in the standard error of
+ AsciiDoc and Asciidoctor, but by saving the stderr of 'make doc' we
+ also save any error output from the make rules. Currently there is
+ only one such line: we build the docs with Asciidoctor right after a
+ 'make clean', meaning that 'make USE_ASCIIDOCTOR=1 doc' always starts
+ with running 'GIT-VERSION-GEN', which in turn prints the version to
+ stderr. A 'sed' command was supposed to remove this version line to
+ prevent it from triggering that (previously defunct) emptiness check,
+ but, unfortunately, this command doesn't work as intended, either,
+ because it leaves the file to be checked intact, but that defunct
+ emptiness check hid this issue, too... Furthermore, in the near
+ future there will be an other line on stderr, because commit
+ 9a71722b4d (Doc: auto-detect changed build flags, 2019-03-17) in the
+ currently cooking branch 'ma/doc-diff-doc-vs-doctor-comparison' will
+ print "* new asciidoc flags" at the beginning of both 'make doc'
+ invokations.
+
+ Extend that 'sed' command to remove this line, too, wrap it in a
+ helper function so the output of both 'make doc' is filtered the same
+ way, and change its invokation to actually write the logfile to be
+ checked.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#set

@@ -66,15 +77,32 @@
+++ b/ci/test-documentation.sh
@@

+ . ${0%/*}/lib.sh
+
++filter_log () {
++ sed -e '/^GIT_VERSION = /d' \
++ -e '/^ \* new asciidoc flags$/d' \
++ "$1"
++}
++
+ make check-builtins
+ make check-docs
+
# Build docs with AsciiDoc
- make doc > >(tee stdout.log) 2> >(tee stderr.log >&2) +-make doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
-! test -s stderr.log
-+cat stderr.log
++make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
++cat stderr.raw
++filter_log stderr.raw >stderr.log
+test ! -s stderr.log
test -s Documentation/git.html
test -s Documentation/git.xml
test -s Documentation/git.1
-@@
+ grep '<meta name="generator" content="AsciiDoc ' Documentation/git.html +
+-rm -f stdout.log stderr.log
++rm -f stdout.log stderr.log stderr.raw
+ check_unignored_build_artifacts

# Build docs with AsciiDoctor
make clean
@@ -82,8 +110,8 @@
-sed '/^GIT_VERSION = / d' stderr.log
-! test -s stderr.log
+make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
-+sed '/^GIT_VERSION = / d' stderr.raw >stderr.log
-+cat stderr.log
++cat stderr.raw
++filter_log stderr.raw >stderr.log
+test ! -s stderr.log
test -s Documentation/git.html
grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html

Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#16 lore raw

PATCH v3 1/6 Documentation/git-diff-tree.txt: fix formatting

Asciidoctor versions v1.5.7 or later print the following warning while building the documentation:

ASCIIDOC git-diff-tree.xml
asciidoctor: WARNING: diff-format.txt: line 2: unterminated listing block

This highlights an issue (even with older Asciidoctor versions) where the "Raw output format" header is not rendered as a header, and the rest of the document is rendered in monospace. This is not caused by 'diff-format.txt' in itself, but rather by 'git-diff-tree.txt' including 'pretty-formats.txt' and 'diff-format.txt' on subsequent lines, while the former happens to end with monospace-formatted example commands.

Fix this by inserting an empty line between the two include::
directives.

The page rendered with AsciiDoc doesn't have this formatting issue.

---
 Documentation/git-diff-tree.txt | 1 +
 1 file changed, 1 insertion(+)
Documentation/git-diff-tree.txt +1 -0
index 24f32e8c54..5c8a2a5e97 100644
@@ -118,6 +118,7 @@ include::pretty-options.txt[]
 
 
 include::pretty-formats.txt[]
+
 include::diff-format.txt[]
 
 GIT
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#17 lore raw

PATCH v3 2/6 Documentation/technical/api-config.txt: fix formatting

In reply to SZEDER Gábor

Asciidoctor versions v1.5.7 or later print the following warning while building the documentation:

ASCIIDOC technical/api-config.html
asciidoctor: WARNING: api-config.txt: line 232: unterminated listing block

This highlight an issue (even with older Asciidoctor versions) where the length of the '----' lines surrounding a code example don't match, and the rest of the document is rendered in monospace.

Fix this by making sure that the length of those lines match.

The page rendered with AsciiDoc doesn't have this formatting issue.

---
 Documentation/technical/api-config.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/technical/api-config.txt +1 -1
index fa39ac9d71..7d20716c32 100644
@@ -229,7 +229,7 @@ A `config_set` can be used to construct an in-memory cache for
 config-like files that the caller specifies (i.e., files like `.gitmodules`,
 `~/.gitconfig` etc.). For example,
 
----------------------------------------
+----------------------------------------
 struct config_set gm_config;
 git_configset_init(&gm_config);
 int b;
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#19 lore raw

PATCH v3 4/6 ci: install Asciidoctor in 'ci/install-dependencies.sh'

In reply to SZEDER Gábor

When our '.travis.yml' was split into several 'ci/*' scripts [1], the installation of the 'asciidoctor' gem somehow ended up in 'ci/test-documentation.sh'.

Install it in 'ci/install-dependencies.sh', where we install other dependencies of the Documentation build job as well (asciidoc, xmlto).

[1] 657343a602 (travis-ci: move Travis CI code into dedicated scripts,

    2017-09-10)
---
 ci/install-dependencies.sh | 3 +++
 ci/test-documentation.sh   | 3 ---
 2 files changed, 3 insertions(+), 3 deletions(-)
ci/install-dependencies.sh +3 -0
index d64667fcbf..76ec308965 100755
@@ -54,6 +54,9 @@ StaticAnalysis)
 Documentation)
 	sudo apt-get -q update
 	sudo apt-get -q -y install asciidoc xmlto
+
+	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
+	gem install asciidoctor
 	;;
 esac
 
ci/test-documentation.sh +0 -3
index be3b7d376a..8f91f48c81 100755
@@ -5,9 +5,6 @@
 
 . ${0%/*}/lib.sh
 
-test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
-gem install asciidoctor
-
 make check-builtins
 make check-docs
 
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#18 lore raw

PATCH v3 3/6 Documentation/technical/protocol-v2.txt: fix formatting

In reply to SZEDER Gábor

Asciidoctor versions v1.5.7 or later print the following warning while building the documentation:

ASCIIDOC technical/protocol-v2.html
asciidoctor: WARNING: protocol-v2.txt: line 38: unterminated listing block

This highlights an issue (even with older Asciidoctor versions) where the 'Initial Client Request' header is not rendered as a header but in monospace. I'm not sure what exactly causes this issue and why it's an issue only with this particular header, but all headers in 'protocol-v2.txt' are written like this:

Initial Client Request
------------------------

i.e. the header itself is indented by a space, and the "underline" is two characters longer than the header.

Dropping that indentation and making the length of the underline match the length of the header apparently fixes this issue.

While at it, adjust all other headers 'protocol-v2.txt' as well, to match the style we use everywhere else.

The page rendered with AsciiDoc doesn't have this formatting issue.

---
 Documentation/technical/protocol-v2.txt | 52 ++++++++++++-------------
 1 file changed, 26 insertions(+), 26 deletions(-)
Documentation/technical/protocol-v2.txt +26 -26
index ead85ce35c..03264c7d9a 100644
@@ -1,5 +1,5 @@
- Git Wire Protocol, Version 2
-==============================
+Git Wire Protocol, Version 2
+============================
 
 This document presents a specification for a version 2 of Git's wire
 protocol.  Protocol v2 will improve upon v1 in the following ways:
@@ -22,8 +22,8 @@ will be commands which a client can request be executed.  Once a command
 has completed, a client can reuse the connection and request that other
 commands be executed.
 
- Packet-Line Framing
----------------------
+Packet-Line Framing
+-------------------
 
 All communication is done using packet-line framing, just as in v1.  See
 `Documentation/technical/pack-protocol.txt` and
@@ -34,8 +34,8 @@ In protocol v2 these special packets will have the following semantics:
   * '0000' Flush Packet (flush-pkt) - indicates the end of a message
   * '0001' Delimiter Packet (delim-pkt) - separates sections of a message
 
- Initial Client Request
-------------------------
+Initial Client Request
+----------------------
 
 In general a client can request to speak protocol v2 by sending
 `version=2` through the respective side-channel for the transport being
@@ -43,22 +43,22 @@ used which inevitably sets `GIT_PROTOCOL`.  More information can be
 found in `pack-protocol.txt` and `http-protocol.txt`.  In all cases the
 response from the server is the capability advertisement.
 
- Git Transport
-~~~~~~~~~~~~~~~
+Git Transport
+~~~~~~~~~~~~~
 
 When using the git:// transport, you can request to use protocol v2 by
 sending "version=2" as an extra parameter:
 
    003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0
 
- SSH and File Transport
-~~~~~~~~~~~~~~~~~~~~~~~~
+SSH and File Transport
+~~~~~~~~~~~~~~~~~~~~~~
 
 When using either the ssh:// or file:// transport, the GIT_PROTOCOL
 environment variable must be set explicitly to include "version=2".
 
- HTTP Transport
-~~~~~~~~~~~~~~~~
+HTTP Transport
+~~~~~~~~~~~~~~
 
 When using the http:// or https:// transport a client makes a "smart"
 info/refs request as described in `http-protocol.txt` and requests that
@@ -79,8 +79,8 @@ A v2 server would reply:
 Subsequent requests are then made directly to the service
 `$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack).
 
- Capability Advertisement
---------------------------
+Capability Advertisement
+------------------------
 
 A server which decides to communicate (based on a request from a client)
 using protocol version 2, notifies the client by sending a version string
@@ -101,8 +101,8 @@ to be executed by the client.
     key = 1*(ALPHA | DIGIT | "-_")
     value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;")
 
- Command Request
------------------
+Command Request
+---------------
 
 After receiving the capability advertisement, a client can then issue a
 request to select the command it wants with any particular capabilities
@@ -137,8 +137,8 @@ command be executed or can terminate the connection.  A client may
 optionally send an empty request consisting of just a flush-pkt to
 indicate that no more requests will be made.
 
- Capabilities
---------------
+Capabilities
+------------
 
 There are two different types of capabilities: normal capabilities,
 which can be used to to convey information or alter the behavior of a
@@ -153,8 +153,8 @@ management on the server side in order to function correctly.  This
 permits simple round-robin load-balancing on the server side, without
 needing to worry about state management.
 
- agent
-~~~~~~~
+agent
+~~~~~
 
 The server can advertise the `agent` capability with a value `X` (in the
 form `agent=X`) to notify the client that the server is running version
@@ -168,8 +168,8 @@ printable ASCII characters except space (i.e., the byte range 32 < x <
 and debugging purposes, and MUST NOT be used to programmatically assume
 the presence or absence of particular features.
 
- ls-refs
-~~~~~~~~~
+ls-refs
+~~~~~~~
 
 `ls-refs` is the command used to request a reference advertisement in v2.
 Unlike the current reference advertisement, ls-refs takes in arguments
@@ -199,8 +199,8 @@ The output of ls-refs is as follows:
     symref = "symref-target:" symref-target
     peeled = "peeled:" obj-id
 
- fetch
-~~~~~~~
+fetch
+~~~~~
 
 `fetch` is the command used to fetch a packfile in v2.  It can be looked
 at as a modified version of the v1 fetch where the ref-advertisement is
@@ -444,8 +444,8 @@ header.
 		2 - progress messages
 		3 - fatal error message just before stream aborts
 
- server-option
-~~~~~~~~~~~~~~~
+server-option
+~~~~~~~~~~~~~
 
 If advertised, indicates that any number of server specific options can be
 included in a request.  This is done by sending each option as a
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#20 lore raw

PATCH v3 5/6 ci: stick with Asciidoctor v1.5.8 for now

In reply to SZEDER Gábor

The release of Asciidoctor v2.0.0 two days ago broke our documentation build job on Travis CI, where we 'gem install asciidoctor', which always brings us the latest and (supposedly) greatest. Alas, we are not ready for that just yet, because it removed support for DocBook 4.5, and we have been requiring that particular DocBook version to build 'user-manual.xml' with Asciidoctor, resulting in:

ASCIIDOC user-manual.xml
asciidoctor: FAILED: missing converter for backend 'docbook45'. Processing aborted.

    Use --trace for backtrace

make[1]: *** [user-manual.xml] Error 1

Unfortunately, we can't simply switch to DocBook 5 right away, as doing so leads to validation errors from 'xmlto', and working around those leads to yet another errors... [1]

So let's stick with Asciidoctor v1.5.8 (latest stable release before v2.0.0) in our documentation build job on Travis CI for now, until we figure out how to deal with the fallout from Asciidoctor v2.0.0.

[1] https://public-inbox.org/git/[email protected]/

---
 ci/install-dependencies.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
ci/install-dependencies.sh +1 -1
index 76ec308965..52a44c690a 100755
@@ -56,7 +56,7 @@ Documentation)
 	sudo apt-get -q -y install asciidoc xmlto
 
 	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
-	gem install asciidoctor
+	gem install --version 1.5.8 asciidoctor
 	;;
 esac
 
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#23 lore raw

PATCH v3 6/6 ci: fix AsciiDoc/Asciidoctor stderr check in the documentation build job

In reply to SZEDER Gábor

In 'ci/test-documentation.sh' we save the standard error of 'make doc', and, in an attempt to make sure that neither AsciiDoc nor Asciidoctor printed any warnings, we check the emptiness of the resulting file with '! test -s stderr.log'. This check has never actually worked, because in our 'ci/*' build scripts we rely on 'set -e' aborting the build job when a command exits with error, and, unfortunately, the combination of the two doesn't work as intended. According to POSIX [1]:

"The -e setting shall be ignored when executing [...] a pipeline beginning with the ! reserved word" [2]

Watch and learn:

  $ echo unexpected >file
  $ ( set -e; ! test -s file ; echo "should not reach this" ) ; echo $?
  should not reach this
  0

This is why we haven't noticed the warnings from Asciidoctor that were fixed in the first patches of this patch series, though some of them were already there in the build of v2.18.0-rc0 [3].

Check the emptiness of that file with 'test ! -s' instead, which works properly with 'set -e':

  $ ( set -e; test ! -s file ; echo "should not reach this" ) ; echo $?
  1

Furthermore, dump the contents of that file to the log for our convenience, so if it were to unexpectedly end up being non-empty, then we wouldn't have to scroll through all that long build log looking for warnings, but could see them right away near the end of the log.

Note that we are only really interested in the standard error of AsciiDoc and Asciidoctor, but by saving the stderr of 'make doc' we also save any error output from the make rules. Currently there is only one such line: we build the docs with Asciidoctor right after a 'make clean', meaning that 'make USE_ASCIIDOCTOR=1 doc' always starts with running 'GIT-VERSION-GEN', which in turn prints the version to stderr. A 'sed' command was supposed to remove this version line to prevent it from triggering that (previously defunct) emptiness check, but, unfortunately, this command doesn't work as intended, either, because it leaves the file to be checked intact, but that defunct emptiness check hid this issue, too... Furthermore, in the near future there will be an other line on stderr, because commit 9a71722b4d (Doc: auto-detect changed build flags, 2019-03-17) in the currently cooking branch 'ma/doc-diff-doc-vs-doctor-comparison' will print "* new asciidoc flags" at the beginning of both 'make doc' invokations.

Extend that 'sed' command to remove this line, too, wrap it in a helper function so the output of both 'make doc' is filtered the same way, and change its invokation to actually write the logfile to be checked.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#set

[2] POSIX doesn't discuss the meaning of '! cmd' in case of simple commands, but it defines that "A pipeline is a sequence of one or more commands separated by the control operator '|'", so apparently a simple command is considered as pipeline as well.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_02

[3] https://travis-ci.org/git/git/jobs/385932007#L1463

---
 ci/test-documentation.sh | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)
ci/test-documentation.sh +16 -7
index 8f91f48c81..d49089832d 100755
@@ -5,29 +5,38 @@
 
 . ${0%/*}/lib.sh
 
+filter_log () {
+	sed -e '/^GIT_VERSION = /d' \
+	    -e '/^    \* new asciidoc flags$/d' \
+	    "$1"
+}
+
 make check-builtins
 make check-docs
 
 # Build docs with AsciiDoc
-make doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
-! test -s stderr.log
+make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
+cat stderr.raw
+filter_log stderr.raw >stderr.log
+test ! -s stderr.log
 test -s Documentation/git.html
 test -s Documentation/git.xml
 test -s Documentation/git.1
 grep '<meta name="generator" content="AsciiDoc ' Documentation/git.html
 
-rm -f stdout.log stderr.log
+rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
 
 # Build docs with AsciiDoctor
 make clean
-make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
-sed '/^GIT_VERSION = / d' stderr.log
-! test -s stderr.log
+make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
+cat stderr.raw
+filter_log stderr.raw >stderr.log
+test ! -s stderr.log
 test -s Documentation/git.html
 grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html
 
-rm -f stdout.log stderr.log
+rm -f stdout.log stderr.log stderr.raw
 check_unignored_build_artifacts
 
 save_good_tree
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#24 lore raw

Re: [PATCH v3 0/6] Asciidoctor-related formatting and CI fixes

In reply to SZEDER Gábor

Hi,

Quoted 14 lines · SZEDER Gábor

[...]
- Mention in patch 4's commit message that Asciidoctor is still only installed in the Documentation build job.
[...]

Range-diff:

1:  8026f62876 = 1:  8026f62876 Documentation/git-diff-tree.txt: fix formatting
2:  fd19cf4b24 = 2:  fd19cf4b24 Documentation/technical/api-config.txt: fix formatting
3:  638dcd64e9 = 3:  638dcd64e9 Documentation/technical/protocol-v2.txt: fix formatting
4:  6f8c6ff398 ! 4:  2e94e2b7b6 ci: install Asciidoctor in 'ci/install-dependencies.sh'

@@ -6,8 +6,9 @@
installation of the 'asciidoctor' gem somehow ended up in 'ci/test-documentation.sh'.

Quoted 5 lines

- Install it in 'ci/install-dependencies.sh', where we install
- everything else.
+ Install it in 'ci/install-dependencies.sh', where we install other
+ dependencies of the Documentation build job as well (asciidoc,
+ xmlto).

I would have wished for something that more explicitly said that it still *only* installs it for the Documentation job.

But I can live with the current wording.

Quoted 38 lines

[1] 657343a602 (travis-ci: move Travis CI code into dedicated scripts, 2017-09-10)
[...]

@@ -66,15 +77,32 @@
+++ b/ci/test-documentation.sh
@@

+ . ${0%/*}/lib.sh
+
++filter_log () {
++ sed -e '/^GIT_VERSION = /d' \
++ -e '/^ \* new asciidoc flags$/d' \
++ "$1"
++}
++
+ make check-builtins
+ make check-docs
+
# Build docs with AsciiDoc
- make doc > >(tee stdout.log) 2> >(tee stderr.log >&2) +-make doc > >(tee stdout.log) 2> >(tee stderr.log >&2)
-! test -s stderr.log
-+cat stderr.log
++make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
++cat stderr.raw
++filter_log stderr.raw >stderr.log
+test ! -s stderr.log
test -s Documentation/git.html
test -s Documentation/git.xml
test -s Documentation/git.1
-@@
+ grep '<meta name="generator" content="AsciiDoc ' Documentation/git.html +
+-rm -f stdout.log stderr.log
++rm -f stdout.log stderr.log stderr.raw
+ check_unignored_build_artifacts

Quoted 13 lines

# Build docs with AsciiDoctor
make clean
@@ -82,8 +110,8 @@
-sed '/^GIT_VERSION = / d' stderr.log
-! test -s stderr.log
+make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)
-+sed '/^GIT_VERSION = / d' stderr.raw >stderr.log
-+cat stderr.log
++cat stderr.raw
++filter_log stderr.raw >stderr.log
+test ! -s stderr.log
test -s Documentation/git.html
grep '<meta name="generator" content="Asciidoctor ' Documentation/git.html

Wow. Subtle. And a bit hard to read without color ;-) But from what I understand, this does the right thing.

So from my side, this patch series is good to go!

Thanks,
Dscho

#21 lore raw

PATCH v2 5/6 ci: stick with Asciidoctor v1.5.8 for now

In reply to SZEDER Gábor
Quoted 1 line · SZEDER Gábor

The release of Asciidoctor v2.0.0 two days ago broke our documentation

Well, what happened "two days ago" when I sent v2 is now seven days ago... Let's just say "recent" instead.

--- >8 ---

Subject: ci: stick with Asciidoctor v1.5.8 for now

The recent release of Asciidoctor v2.0.0 broke our documentation build job on Travis CI, where we 'gem install asciidoctor', which always brings us the latest and (supposedly) greatest. Alas, we are not ready for that just yet, because it removed support for DocBook 4.5, and we have been requiring that particular DocBook version to build 'user-manual.xml' with Asciidoctor, resulting in:

ASCIIDOC user-manual.xml
asciidoctor: FAILED: missing converter for backend 'docbook45'. Processing aborted.

    Use --trace for backtrace

make[1]: *** [user-manual.xml] Error 1

Unfortunately, we can't simply switch to DocBook 5 right away, as doing so leads to validation errors from 'xmlto', and working around those leads to yet another errors... [1]

So let's stick with Asciidoctor v1.5.8 (latest stable release before v2.0.0) in our documentation build job on Travis CI for now, until we figure out how to deal with the fallout from Asciidoctor v2.0.0.

[1] https://public-inbox.org/git/[email protected]/

---
 ci/install-dependencies.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
ci/install-dependencies.sh +1 -1
index 76ec308965..52a44c690a 100755
@@ -56,7 +56,7 @@ Documentation)
 	sudo apt-get -q -y install asciidoc xmlto
 
 	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
-	gem install asciidoctor
+	gem install --version 1.5.8 asciidoctor
 	;;
 esac
 
Signature · 1 line
2.21.0.539.g07239c3a71.dirty
#22 lore raw

Re: [PATCH v3.1 5/6] ci: stick with Asciidoctor v1.5.8 for now

Hi Junio

Quoted 11 lines · SZEDER Gábor
Quoted 1 line · SZEDER Gábor

The release of Asciidoctor v2.0.0 two days ago broke our documentation

Well, what happened "two days ago" when I sent v2 is now seven days ago... Let's just say "recent" instead.

--- >8 ---

Subject: ci: stick with Asciidoctor v1.5.8 for now

This was picked up as 28216d13f4 ("ci: stick with Asciidoctor v1.5.8 for now", 2019-03-29) as part of sg/asciidoctor-in-ci. Actually, all of the above is included, self-quote, scissors and all.
Martin