Skip to content

Sideband the whole fetch v2 response

Jonathan Tan git 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest4 patches 3 repliesIn review
  2. v24 patches 11 repliesSuperseded
  3. v14 patches 12 repliesSuperseded

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4pkt-line: introduce struct packet_writerpkt-line.c pkt-line.h upload-pack.c+115 -580 replies
2/4sideband: reverse its dependency on pkt-linepkt-line.c pkt-line.h sideband.c +1 more+141 -950 replies
3/4{fetch,upload}-pack: sideband v2 fetch responseDocumentation/technical/protocol-v2.txt fetch-pack.c pkt-line.c +4 more+78 -130 replies
4/4tests: define GIT_TEST_SIDEBAND_ALLfetch-pack.c t/README t/lib-httpd/apache.conf +4 more+21 -102 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.

Read the v3 thread

Cover letter [PATCH v3 0/4] Sideband the whole fetch v2 response

Like previous versions, this is on origin/ms/packet-err-check.

First of all, thanks to those who noticed the issue with t5409. I have merged sg/stress-test and ran:

    make DEVELOPER=1 && (cd t && sh ./t5409-col*.sh --stress)

with no issues.

demultiplex_sideband() now requires its caller to maintain a strbuf, since progress messages can be split across packets. I have also changed the API of demultiplex_sideband() to hopefully make things clearer.

As for keepalive packets, I think it's best if we state that the remote should use 0005\2 instead of 0005\1, and have made the change accordingly. This does not need to be treated specially by the client. (I shouldn't have a problem updating my CDN offloading patches [1] to use 0005\2 before the packfile starts and 0005\1 after the packfile starts.)

[1] Patches 5-8 of https://public-inbox.org/git/[email protected]/

Jonathan Tan (4):
pkt-line: introduce struct packet_writer
sideband: reverse its dependency on pkt-line
{fetch,upload}-pack: sideband v2 fetch response
tests: define GIT_TEST_SIDEBAND_ALL

 Documentation/technical/protocol-v2.txt |  10 ++
 fetch-pack.c                            |  13 +-
 pkt-line.c                              | 107 ++++++++++++--
 pkt-line.h                              |  34 +++++
 sideband.c                              | 178 ++++++++++++------------
 sideband.h                              |  25 +++-
 t/README                                |   5 +
 t/lib-httpd/apache.conf                 |   1 +
 t/t5537-fetch-shallow.sh                |   3 +-
 t/t5701-git-serve.sh                    |   2 +-
 t/t5702-protocol-v2.sh                  |   4 +-
 upload-pack.c                           | 131 ++++++++++-------
 12 files changed, 346 insertions(+), 167 deletions(-)

Interdiff against v2:

Documentation/technical/protocol-v2.txt +2 -2
index 1b0633f59f..39b40c0dc1 100644
@@ -314,8 +314,8 @@ included in the client's request:
 	Instruct the server to send the whole response multiplexed, not just
 	the packfile section. All non-flush and non-delim PKT-LINE in the
 	response (not only in the packfile section) will then start with a byte
-	indicating its sideband (1, 2, or 3), and the server may send "0005\1"
-	(a PKT-LINE of sideband 1 with no payload) as a keepalive packet.
+	indicating its sideband (1, 2, or 3), and the server may send "0005\2"
+	(a PKT-LINE of sideband 2 with no payload) as a keepalive packet.
 
 The response of `fetch` is broken into a number of sections separated by
 delimiter packets (0001), with each section beginning with its section
pkt-line.c +17 -31

84 lines not on this page · raw message

sideband.c +32 -33

115 lines not on this page · raw message

sideband.h +18 -11
index f75c4fde2a..227740a58e 100644
@@ -1,21 +1,28 @@
 #ifndef SIDEBAND_H
 #define SIDEBAND_H
 
-#define SIDEBAND_PROTOCOL_ERROR -2
-#define SIDEBAND_REMOTE_ERROR -1
-#define SIDEBAND_FLUSH 0
-#define SIDEBAND_PRIMARY 1
-#define SIDEBAND_PROGRESS 2
+enum sideband_type {
+	SIDEBAND_PROTOCOL_ERROR = -2,
+	SIDEBAND_REMOTE_ERROR = -1,
+	SIDEBAND_FLUSH = 0,
+	SIDEBAND_PRIMARY = 1
+};
 
 /*
- * Inspects a multiplexed packet read from the remote and returns which
- * sideband it is for.
+ * Inspects a multiplexed packet read from the remote. If this packet is a
+ * progress packet and thus should not be processed by the caller, returns 0.
+ * Otherwise, returns 1, releases scratch, and sets sideband_type.
  *
- * If SIDEBAND_PROTOCOL_ERROR, SIDEBAND_REMOTE_ERROR, or SIDEBAND_PROGRESS,
- * also prints a message (or the formatted contents of the notice in the case
- * of SIDEBAND_PROGRESS) to stderr.
+ * If this packet is SIDEBAND_PROTOCOL_ERROR, SIDEBAND_REMOTE_ERROR, or a
+ * progress packet, also prints a message to stderr.
+ *
+ * scratch must be a struct strbuf allocated by the caller. It is used to store
+ * progress messages split across multiple packets.
  */
-int demultiplex_sideband(const char *me, char *buf, int len, int die_on_error);
+int demultiplex_sideband(const char *me, char *buf, int len,
+			 int die_on_error,
+			 struct strbuf *scratch,
+			 enum sideband_type *sideband_type);
 
 void send_sideband(int fd, int band, const char *data, ssize_t sz, int packet_max);
 
Signature · 1 line
2.19.0.271.gfe8321ec05.dirty