optionally send server-options when using v2

PATCH In review Brandon Williams 1 participant 5 messages Apr 23, 2018 Documentation

Share & tools

Brandon Williams

author

2018-04-23 22:46 UTC #0 lore raw

cover optionally send server-options when using v2

Building on top of protocol version 2 this series adds the ability to optionally send server specific options when using protocol v2. This resembles the "push-options" feature except server options are sent as capability lines during a command request allowing for all current and future commands to benefit from sending arbitrary server options (and not requiring that sending server specific options be re-implemented for each and every command that may want to make use of them in the future).

These options can be provided by the user via the command line by giving "-o <option>" or "--server-option=<option>" to either ls-remote or fetch.

Command request example:

	command=fetch
	server-option=hello
	server-option=world
	0001
	want A
	want B
	have X
	have Y
	0000

These options are only transmitted to the remote end when communicating using protocol version 2.

Brandon Williams (3):
serve: introduce the server-option capability
ls-remote: send server options when using protocol v2
fetch: send server options when using protocol v2

 Documentation/fetch-options.txt         |  8 +++++++
 Documentation/git-ls-remote.txt         |  8 +++++++
 Documentation/technical/protocol-v2.txt | 10 ++++++++
 builtin/fetch.c                         |  5 ++++
 builtin/ls-remote.c                     |  4 ++++
 connect.c                               |  9 ++++++-
 fetch-pack.c                            |  7 ++++++
 fetch-pack.h                            |  1 +
 remote.h                                |  4 +++-
 serve.c                                 |  1 +
 t/t5701-git-serve.sh                    | 21 ++++++++++++++++
 t/t5702-protocol-v2.sh                  | 32 +++++++++++++++++++++++++
 transport.c                             |  3 ++-
 transport.h                             |  6 +++++
 14 files changed, 116 insertions(+), 3 deletions(-)

Signature · 1 line

2.17.0.484.g0c8726318c-goog

Brandon Williams

author

2018-04-23 22:46 UTC #1 lore raw

PATCH 1/3 serve: introduce the server-option capability

Introduce the "server-option" capability to protocol version 2. This enables future clients the ability to send server specific options in command requests when using protocol version 2.

Signed-off-by Brandon Williams <[email protected]>

---
 Documentation/technical/protocol-v2.txt | 10 ++++++++++
 serve.c                                 |  1 +
 t/t5701-git-serve.sh                    | 21 +++++++++++++++++++++
 3 files changed, 32 insertions(+)

Documentation/technical/protocol-v2.txt +10 -0

index 136179d7d..d7b6f38e0 100644
@@ -393,3 +393,13 @@ header.
 		1 - pack data
 		2 - progress messages
 		3 - fatal error message just before stream aborts
+
+ server-option
+~~~~~~~~~~~~~~~
+
+If advertised, indicates that any number of server specific options can be
+included in a request.  This is done by sending each option as a
+"server-option=<option>" capability line in the capability-list section of
+a request.
+
+The provided options must not contain a NUL or LF character.

serve.c +1 -0

index a5a7b2f7d..bda085f09 100644
@@ -56,6 +56,7 @@ static struct protocol_capability capabilities[] = {
 	{ "agent", agent_advertise, NULL },
 	{ "ls-refs", always_advertise, ls_refs },
 	{ "fetch", upload_pack_advertise, upload_pack_v2 },
+	{ "server-option", always_advertise, NULL },
 };
 
 static void advertise_capabilities(void)

t/t5701-git-serve.sh +21 -0

index 72d7bc562..011a5796d 100755
@@ -10,6 +10,7 @@ test_expect_success 'test capability advertisement' '
 	agent=git/$(git version | cut -d" " -f3)
 	ls-refs
 	fetch=shallow
+	server-option
 	0000
 	EOF

@@ -173,4 +174,24 @@ test_expect_success 'symrefs parameter' '
 	test_cmp actual expect
 '
 
+test_expect_success 'sending server-options' '
+	test-pkt-line pack >in <<-EOF &&
+	command=ls-refs
+	server-option=hello
+	server-option=world
+	0001
+	ref-prefix HEAD
+	0000
+	EOF
+
+	cat >expect <<-EOF &&
+	$(git rev-parse HEAD) HEAD
+	0000
+	EOF
+
+	git serve --stateless-rpc <in >out &&
+	test-pkt-line unpack <out >actual &&
+	test_cmp actual expect
+'
+
 test_done

Signature · 1 line

2.17.0.484.g0c8726318c-goog

Brandon Williams

author

2018-04-23 22:46 UTC #2 lore raw

PATCH 2/3 ls-remote: send server options when using protocol v2

In reply to Brandon Williams

Teach ls-remote to optionally accept server options by specifying them on the cmdline via '-o' or '--server-option'. These server options are sent to the remote end when querying for the remote end's refs using protocol version 2.

If communicating using a protocol other than v2 the provided options are ignored and not sent to the remote end.

Signed-off-by Brandon Williams <[email protected]>

---
 Documentation/git-ls-remote.txt |  8 ++++++++
 builtin/ls-remote.c             |  4 ++++
 connect.c                       |  9 ++++++++-
 remote.h                        |  4 +++-
 t/t5702-protocol-v2.sh          | 16 ++++++++++++++++
 transport.c                     |  2 +-
 transport.h                     |  6 ++++++
 7 files changed, 46 insertions(+), 3 deletions(-)

Documentation/git-ls-remote.txt +8 -0

index 5f2628c8f..e5defb1b2 100644
@@ -60,6 +60,14 @@ OPTIONS
 	upload-pack only shows the symref HEAD, so it will be the only
 	one shown by ls-remote.
 
+-o <option>::
+--server-option=<option>::
+	Transmit the given string to the server when communicating using
+	protocol version 2.  The given string must not contain a NUL or LF
+	character.
+	When multiple `--server-option=<option>` are given, they are all
+	sent to the other side in the order listed on the command line.
+
 <repository>::
 	The "remote" repository to query.  This parameter can be
 	either a URL or the name of a remote (see the GIT URLS and

builtin/ls-remote.c +4 -0

index 380c18027..3150bfb92 100644
@@ -45,6 +45,7 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)
 	const char *uploadpack = NULL;
 	const char **pattern = NULL;
 	struct argv_array ref_prefixes = ARGV_ARRAY_INIT;
+	struct string_list server_options = STRING_LIST_INIT_DUP;
 
 	struct remote *remote;
 	struct transport *transport;

@@ -67,6 +68,7 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)
 			      2, PARSE_OPT_NOCOMPLETE),
 		OPT_BOOL(0, "symref", &show_symref_target,
 			 N_("show underlying ref in addition to the object pointed by it")),
+		OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
 		OPT_END()
 	};

@@ -107,6 +109,8 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix)
 	transport = transport_get(remote, NULL);
 	if (uploadpack != NULL)
 		transport_set_option(transport, TRANS_OPT_UPLOADPACK, uploadpack);
+	if (server_options.nr)
+		transport->server_options = &server_options;
 
 	ref = transport_get_remote_refs(transport, &ref_prefixes);
 	if (transport_disconnect(transport))

connect.c +8 -1

index 54971166a..3000768c7 100644
@@ -408,7 +408,8 @@ static int process_ref_v2(const char *line, struct ref ***list)
 
 struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,
 			     struct ref **list, int for_push,
-			     const struct argv_array *ref_prefixes)
+			     const struct argv_array *ref_prefixes,
+			     const struct string_list *server_options)
 {
 	int i;
 	*list = NULL;

@@ -419,6 +420,12 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,
 	if (server_supports_v2("agent", 0))
 		packet_write_fmt(fd_out, "agent=%s", git_user_agent_sanitized());
 
+	if (server_options && server_options->nr &&
+	    server_supports_v2("server-option", 1))
+		for (i = 0; i < server_options->nr; i++)
+			packet_write_fmt(fd_out, "server-option=%s",
+					 server_options->items[i].string);
+
 	packet_delim(fd_out);
 	/* When pushing we don't want to request the peeled tags */
 	if (!for_push)

remote.h +3 -1

index 2b3180f94..93dd97e25 100644
@@ -153,6 +153,7 @@ void free_refs(struct ref *ref);
 struct oid_array;
 struct packet_reader;
 struct argv_array;
+struct string_list;
 extern struct ref **get_remote_heads(struct packet_reader *reader,
 				     struct ref **list, unsigned int flags,
 				     struct oid_array *extra_have,

@@ -161,7 +162,8 @@ extern struct ref **get_remote_heads(struct packet_reader *reader,
 /* Used for protocol v2 in order to retrieve refs from a remote */
 extern struct ref **get_remote_refs(int fd_out, struct packet_reader *reader,
 				    struct ref **list, int for_push,
-				    const struct argv_array *ref_prefixes);
+				    const struct argv_array *ref_prefixes,
+				    const struct string_list *server_options);
 
 int resolve_remote_symref(struct ref *ref, struct ref *list);
 int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid);

t/t5702-protocol-v2.sh +16 -0

index 56f7c3c32..71ef1aee1 100755
@@ -154,6 +154,22 @@ test_expect_success 'ref advertisment is filtered with ls-remote using protocol
 	test_cmp actual expect
 '
 
+test_expect_success 'server-options are sent when using ls-remote' '
+	test_when_finished "rm -f log" &&
+
+	GIT_TRACE_PACKET="$(pwd)/log" git -c protocol.version=2 \
+		ls-remote -o hello -o world "file://$(pwd)/file_parent" master >actual &&
+
+	cat >expect <<-EOF &&
+	$(git -C file_parent rev-parse refs/heads/master)$(printf "\t")refs/heads/master
+	EOF
+
+	test_cmp actual expect &&
+	grep "server-option=hello" log &&
+	grep "server-option=world" log
+'
+
+
 test_expect_success 'clone with file:// using protocol v2' '
 	test_when_finished "rm -f log" &&

transport.c +1 -1

index 4d8beaaab..42fd468f3 100644
@@ -218,7 +218,7 @@ static struct ref *get_refs_via_connect(struct transport *transport, int for_pus
 	switch (data->version) {
 	case protocol_v2:
 		get_remote_refs(data->fd[1], &reader, &refs, for_push,
-				ref_prefixes);
+				ref_prefixes, transport->server_options);
 		break;
 	case protocol_v1:
 	case protocol_v0:

transport.h +6 -0

index e783cfa07..73a7be3c8 100644
@@ -71,6 +71,12 @@ struct transport {
 	 */
 	const struct string_list *push_options;
 
+	/*
+	 * These strings will be passed to the remote side on each command
+	 * request, if both sides support the server-option capability.
+	 */
+	const struct string_list *server_options;
+
 	char *pack_lockfile;
 	signed verbose : 3;
 	/**

Signature · 1 line

2.17.0.484.g0c8726318c-goog

Brandon Williams

author

2018-04-23 22:46 UTC #3 lore raw

PATCH 3/3 fetch: send server options when using protocol v2

In reply to Brandon Williams

Teach fetch to optionally accept server options by specifying them on the cmdline via '-o' or '--server-option'. These server options are sent to the remote end when performing a fetch communicating using protocol version 2.

If communicating using a protocol other than v2 the provided options are ignored and not sent to the remote end.

Signed-off-by Brandon Williams <[email protected]>

---
 Documentation/fetch-options.txt |  8 ++++++++
 builtin/fetch.c                 |  5 +++++
 fetch-pack.c                    |  7 +++++++
 fetch-pack.h                    |  1 +
 t/t5702-protocol-v2.sh          | 16 ++++++++++++++++
 transport.c                     |  1 +
 6 files changed, 38 insertions(+)

Documentation/fetch-options.txt +8 -0

index 8631e365f..97d3217df 100644
@@ -188,6 +188,14 @@ endif::git-pull[]
 	is specified. This flag forces progress status even if the
 	standard error stream is not directed to a terminal.
 
+-o <option>::
+--server-option=<option>::
+	Transmit the given string to the server when communicating using
+	protocol version 2.  The given string must not contain a NUL or LF
+	character.
+	When multiple `--server-option=<option>` are given, they are all
+	sent to the other side in the order listed on the command line.
+
 -4::
 --ipv4::
 	Use IPv4 addresses only, ignoring IPv6 addresses.

builtin/fetch.c +5 -0

index 7ee83ac0f..5a6f6b2dc 100644
@@ -62,6 +62,7 @@ static int shown_url = 0;
 static int refmap_alloc, refmap_nr;
 static const char **refmap_array;
 static struct list_objects_filter_options filter_options;
+static struct string_list server_options = STRING_LIST_INIT_DUP;
 
 static int git_fetch_config(const char *k, const char *v, void *cb)
 {

@@ -170,6 +171,7 @@ static struct option builtin_fetch_options[] = {
 		 N_("accept refs that update .git/shallow")),
 	{ OPTION_CALLBACK, 0, "refmap", NULL, N_("refmap"),
 	  N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg },
+	OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
 	OPT_SET_INT('4', "ipv4", &family, N_("use IPv4 addresses only"),
 			TRANSPORT_FAMILY_IPV4),
 	OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"),

@@ -1417,6 +1419,9 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int pru
 		}
 	}
 
+	if (server_options.nr)
+		gtransport->server_options = &server_options;
+
 	sigchain_push_common(unlock_pack_on_signal);
 	atexit(unlock_pack);
 	refspec = parse_fetch_refspec(ref_nr, refs);

fetch-pack.c +7 -0

index 216d1368b..199eb8a1d 100644
@@ -1174,6 +1174,13 @@ static int send_fetch_request(int fd_out, const struct fetch_pack_args *args,
 		packet_buf_write(&req_buf, "command=fetch");
 	if (server_supports_v2("agent", 0))
 		packet_buf_write(&req_buf, "agent=%s", git_user_agent_sanitized());
+	if (args->server_options && args->server_options->nr &&
+	    server_supports_v2("server-option", 1)) {
+		int i;
+		for (i = 0; i < args->server_options->nr; i++)
+			packet_write_fmt(fd_out, "server-option=%s",
+					 args->server_options->items[i].string);
+	}
 
 	packet_buf_delim(&req_buf);
 	if (args->use_thin_pack)

fetch-pack.h +1 -0

index 667024a76..f4ba851c6 100644
@@ -15,6 +15,7 @@ struct fetch_pack_args {
 	const char *deepen_since;
 	const struct string_list *deepen_not;
 	struct list_objects_filter_options filter_options;
+	const struct string_list *server_options;
 	unsigned deepen_relative:1;
 	unsigned quiet:1;
 	unsigned keep_pack:1;

t/t5702-protocol-v2.sh +16 -0

index 71ef1aee1..dbfd0691c 100755
@@ -217,6 +217,22 @@ test_expect_success 'ref advertisment is filtered during fetch using protocol v2
 	! grep "refs/tags/three" log
 '
 
+test_expect_success 'server-options are sent when fetching' '
+	test_when_finished "rm -f log" &&
+
+	test_commit -C file_parent four &&
+
+	GIT_TRACE_PACKET="$(pwd)/log" git -C file_child -c protocol.version=2 \
+		fetch -o hello -o world origin master &&
+
+	git -C file_child log -1 --format=%s origin/master >actual &&
+	git -C file_parent log -1 --format=%s >expect &&
+	test_cmp expect actual &&
+
+	grep "server-option=hello" log &&
+	grep "server-option=world" log
+'
+
 # Test protocol v2 with 'http://' transport
 #
 . "$TEST_DIRECTORY"/lib-httpd.sh

transport.c +1 -0

index 42fd468f3..b088bcd49 100644
@@ -266,6 +266,7 @@ static int fetch_refs_via_pack(struct transport *transport,
 	args.no_dependents = data->options.no_dependents;
 	args.filter_options = data->options.filter_options;
 	args.stateless_rpc = transport->stateless_rpc;
+	args.server_options = transport->server_options;
 
 	if (!data->got_remote_heads)
 		refs_tmp = get_refs_via_connect(transport, 0, NULL);

Signature · 1 line

2.17.0.484.g0c8726318c-goog

Brandon Williams

author

2018-04-23 22:47 UTC #4 lore raw

In reply to Brandon Williams

Quoted 24 lines · Brandon Williams

Building on top of protocol version 2 this series adds the ability to optionally send server specific options when using protocol v2. This resembles the "push-options" feature except server options are sent as capability lines during a command request allowing for all current and future commands to benefit from sending arbitrary server options (and not requiring that sending server specific options be re-implemented for each and every command that may want to make use of them in the future).
These options can be provided by the user via the command line by giving "-o <option>" or "--server-option=<option>" to either ls-remote or fetch.
Command request example:
	command=fetch
	server-option=hello
	server-option=world
	0001
	want A
	want B
	have X
	have Y
	0000

Quoted 2 lines

These options are only transmitted to the remote end when communicating using protocol version 2.

Forgot to mention that this series is based on current upstream master (fe0a9eaf3) and a merge of origin/bw/protocol-v2.

Quoted 25 lines

Brandon Williams (3):
serve: introduce the server-option capability
ls-remote: send server options when using protocol v2
fetch: send server options when using protocol v2

 Documentation/fetch-options.txt         |  8 +++++++
 Documentation/git-ls-remote.txt         |  8 +++++++
 Documentation/technical/protocol-v2.txt | 10 ++++++++
 builtin/fetch.c                         |  5 ++++
 builtin/ls-remote.c                     |  4 ++++
 connect.c                               |  9 ++++++-
 fetch-pack.c                            |  7 ++++++
 fetch-pack.h                            |  1 +
 remote.h                                |  4 +++-
 serve.c                                 |  1 +
 t/t5701-git-serve.sh                    | 21 ++++++++++++++++
 t/t5702-protocol-v2.sh                  | 32 +++++++++++++++++++++++++
 transport.c                             |  3 ++-
 transport.h                             |  6 +++++
 14 files changed, 116 insertions(+), 3 deletions(-)

Signature · 1 line

2.17.0.484.g0c8726318c-goog

Signature · 1 line

Brandon Williams