http authentication via prompts (with correct line lengths)

Currently git over http only works with a .netrc file which required that you store your password on the file system in plaintext. This commit adds to configuration options for http for a username and an optional password. If a http.username is set, then the .netrc file is ignored and the username is used instead. If a http.password is set, then that is used as well, otherwise the user is prompted for their password.

With the old .netrc working, this patch provides backwards compatibility while adding a more secure option for users whose http password may be sensitive (such as if its a domain controller password) and do not wish to have it on the filesystem.

---
 Documentation/config.txt                           |    7 +++
 Documentation/howto/setup-git-server-over-http.txt |   38 ++++++++++++++++--
 http.c                                             |   41 ++++++++++++++++++-
 http.h                                             |    2 +
 4 files changed, 81 insertions(+), 7 deletions(-)

index f5152c5..821bf48 100644
@@ -920,6 +920,13 @@ help.autocorrect::
 	value is 0 - the command will be just shown but not executed.
 	This is the default.
 
+http.username, http.password:
+    The username and password for http authentication. http.username is
+    required, http.password is optional. If supplied, the .netrc file will
+    be ignored. If a password is not supplied, git will prompt for it.
+    Be careful when configuring a password as it will be stored in plain text
+    on the filesystem.
+
 http.proxy::
 	Override the HTTP proxy, normally configured using the 'http_proxy'
 	environment variable (see linkgit:curl[1]).  This can be overridden

index 622ee5c..462a9d4 100644
@@ -189,8 +189,19 @@ Make sure that you have HTTP support, i.e. your git was built with
 libcurl (version more recent than 7.10). The command 'git http-push' with
 no argument should display a usage message.
 
-Then, add the following to your $HOME/.netrc (you can do without, but will be
-asked to input your password a _lot_ of times):
+There are 2 ways to authenticate with git http, netrc and via the git config.
+The netrc option requires that you put the username and password for the connection
+in $HOME/.netrc. The configuration method allows you to specify a username and
+optionally a password. If the password is not supplied then git will prompt you
+for the password. The downside to the netrc method is that you must have your
+username and password in plaintext on the filesystem, albeit in a protected file.
+If the username/password combo is a sensitive one, you may wish to use the
+git config method. The downside of the config method is that you will be prompted
+for your password every time you push or pull to the remote repository.
+
+Using netrc:
+
+Using your favourite ext editor, add the following to your $HOME/.netrc:
 
     machine <servername>
     login <username>

@@ -204,7 +215,7 @@ instead of the server name.
 
 To check whether all is OK, do:
 
-   curl --netrc --location -v http://<username>@<servername>/my-new-repo.git/HEAD
+   curl --netrc --location -v http://<servername>/my-new-repo.git/HEAD
 
 ...this should give something like 'ref: refs/heads/master', which is
 the content of the file HEAD on the server.

@@ -213,12 +224,31 @@ Now, add the remote in your existing repository which contains the project
 you want to export:
 
    $ git-config remote.upload.url \
-       http://<username>@<servername>/my-new-repo.git/
+       http://<servername>/my-new-repo.git/
 
 It is important to put the last '/'; Without it, the server will send
 a redirect which git-http-push does not (yet) understand, and git-http-push
 will repeat the request infinitely.
 
+Using git config:
+
+curl --user <username>:<password> --location -v http://<servername>/my-new-repo.git/HEAD
+
+...this should give something like 'ref: refs/heads/master', which is
+the content of the file HEAD on the server.
+
+Now, add the remote in your existing repository which contains the project
+you want to export:
+
+   $ git-config remote.upload.url \
+       http://<servername>/my-new-repo.git/
+
+Also, add in your username with:
+   $ git-config http.username <username>
+
+And optionally your password (you will be prompted for it if you do not):
+   $ git-config http.password <password>
+
 
 Step 4: make the initial push
 -----------------------------

index 905b462..71320d1 100644
@@ -5,6 +5,8 @@
 
 #include <curl/curl.h>
 #include <curl/easy.h>
+#include <termios.h>
+#include <stdio.h>
 
 #include "strbuf.h"
 #include "remote.h"

1.6.1.2

It appears that none of the issues I raised in my response to your earlier round was addressed in this patch, except for the line rewrapping of the proposed commit log message.

Junio,
Just spent about 30 minutes replying to your points until the last one
made most moot. I agree that putting the info into the url will fix the bug, which I have never seen (see #3 below), and make the howto easier to read. So a few things I wanted to discuss or ask for help on:

1) Note that I'm not a C guy so:

I'm not sure what you mean here. Should I just declare them as:
static const char *curl_http_password; ?

Also do you mean that during after the patch phase they get changed to:
static const char *curl_http_password = NULL; ?

Or do you mean that I can send in a patch to fix other static variables (not mine) which are being initialized to NULL?

2) Being that I'm not a big C guy, I'm not sure the best way to go about parsing the username out of the URL to pull it into a variable to pass to CURLOPT_USERPASS. Any advice from the community would be greatly appreciated.

3) From my experience with curl, many of the options do
not work the same across versions or platforms. For example, the new CURLOPT_USERNAME/PASSWORD options worked fine in 7.19.4 on cygwin but not on FC9, which is why I used the older USERPWD. Also, my curl never prompted me for the password when I supplied a username in the URL which is what prompted me to do this patch in the first place. As such, I think it is better to pull the username & password prompting logic into git make this stable and fix the bug.

4) I'm not really impressed that file permissions actually make the .netrc file a secure option. However, it's already in there and would break backwards compatibility to take it out. I also realize that there is a need for automated builds to be able to pull the source. So I would like to add a nice warning section to the http docs explaining the repercussions of using it.

Thanks for the help,
Mike

Yeah, a preparatory patch to clean things up, like the one I sent out earlier this evening, was what I meant.

I am sort of a C guy, but I am by no means a libcurl person. A quick and dirty patch is attached, which is partly based on yours, but is stripped of version dependency and also I suspect it handles only the http-walker side. It is on top of the two clean-up patch I sent this evening.

It hasn't seen any test, but I just ran this once:

    $ git clone http://[email protected]/test-repo.git/

from a repository that requires authentication but I have no .netrc and no http.password configuration; I was asked for the password once, of course.

Heh, 7.19.4 was only released on a few days ago if I am reading its download page correctly.

The version of libcurl on my box is 7.18.something, and it does not seem to ask for password when the URL has only username but not colon-password. I also expected it to ask for password when $HOME/.netrc has login but not password for a given machine, but that does not seem to happen either. Perhaps the version is too old.

I agree with the first two sentences and am not happy with http.password because of it.

---
 http.c |   60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 60 insertions(+), 0 deletions(-)

index f4f0bf6..3d5caa6 100644
@@ -25,6 +25,7 @@ static long curl_low_speed_limit = -1;
 static long curl_low_speed_time = -1;
 static int curl_ftp_no_epsv;
 static const char *curl_http_proxy;
+static char *user_name, *user_pass;
 
 static struct curl_slist *pragma_header;
 

@@ -135,6 +136,20 @@ static int http_options(const char *var, const char *value, void *cb)
 	return git_default_config(var, value, cb);
 }
 
+static void init_curl_http_auth(CURL *result)
+{
+	if (!user_name)
+		curl_easy_setopt(result, CURLOPT_NETRC, CURL_NETRC_OPTIONAL);
+	else {
+		struct strbuf up = STRBUF_INIT;
+		if (!user_pass)
+			user_pass = xstrdup(getpass("Password: "));
+		strbuf_addf(&up, "%s:%s", user_name, user_pass);
+		curl_easy_setopt(result, CURLOPT_USERPWD,
+				 strbuf_detach(&up, NULL));
+	}
+}
+
 static CURL *get_curl_handle(void)
 {
 	CURL *result = curl_easy_init();

@@ -153,6 +168,8 @@ static CURL *get_curl_handle(void)
 	curl_easy_setopt(result, CURLOPT_NETRC, CURL_NETRC_OPTIONAL);
 #endif
 
+	init_curl_http_auth(result);
+
 	if (ssl_cert != NULL)
 		curl_easy_setopt(result, CURLOPT_SSLCERT, ssl_cert);
 #if LIBCURL_VERSION_NUM >= 0x070902

@@ -190,6 +207,46 @@ static CURL *get_curl_handle(void)
 	return result;
 }
 
+static void http_auth_init(const char *url)
+{
+	char *at, *colon, *cp, *slash;
+	int len;
+
+	cp = strstr(url, "://");
+	if (!cp)
+		return;
+
+	/*
+	 * Ok, the URL looks like "proto://something".  Which one?
+	 * "proto://<user>:<pass>@<host>/...",
+	 * "proto://<user>@<host>/...", or just
+	 * "proto://<host>/..."?
+	 */
+	cp += 3;
+	at = strchr(cp, '@');
+	colon = strchr(cp, ':');
+	slash = strchrnul(cp, '/');
+	if (!at || slash <= at)
+		return; /* No credentials */
+	if (!colon || at <= colon) {
+		/* Only username */
+		len = at - cp;
+		user_name = xmalloc(len + 1);
+		memcpy(user_name, cp, len);
+		user_name[len] = '\0';
+		user_pass = NULL;
+	} else {
+		len = colon - cp;
+		user_name = xmalloc(len + 1);
+		memcpy(user_name, cp, len);
+		user_name[len] = '\0';
+		len = at - (colon + 1);
+		user_pass = xmalloc(len + 1);
+		memcpy(user_pass, colon + 1, len);
+		user_pass[len] = '\0';
+	}
+}
+
 void http_init(struct remote *remote)
 {
 	char *low_speed_limit;

@@ -252,6 +309,9 @@ void http_init(struct remote *remote)
 	if (getenv("GIT_CURL_FTP_NO_EPSV"))
 		curl_ftp_no_epsv = 1;
 
+	if (remote && remote->url && remote->url[0])
+		http_auth_init(remote->url[0]);
+
 #ifndef NO_CURL_EASY_DUPHANDLE
 	curl_default = get_curl_handle();
 #endif

No, that's entirely expected. libcurl has no "prompt the user if no password was given" logic but instead delegates that work to the application.

There was once functionality for this (removed in October 2003) but it was broken and violated internal guidelines so we cut out and threw that code away.

More recently there have been people interested in re-implementing this "the right way" but so far it hasn't been made and thus the application is left to perform this task.

 / daniel.haxx.se

It's always nice to find _the_ area expert around ;-)

I somehow misread the description on CURLOPT_NETRC that appears in
http://curl.haxx.se/libcurl/c/curl_easy_setopt.html:

libcurl uses a user name (and supplied or prompted password) supplied with CURLOPT_USERPWD in preference to any of the options controlled by this parameter.

especially the "or prompted password" part to mean that unless supplied to the library by the caller the library would prompt the user and obtain the password.

Thanks for clarification.

2009/3/10 Junio C Hamano <[email protected]>:

May I suggest perhaps require a libcurl version that is no older than three years or something like that?

Perhaps this list can serve as some help:

http://curl.haxx.se/docs/releases.html

(spoiler: libcurl 7.9.3 is more than seven years old!)

 / daniel.haxx.se

2009/3/12 Daniel Stenberg <[email protected]>:

It might be a plan 8-) Though I was thinking technically in terms of features we think git needs. Though doubtless there are several security fixes it would be beneficial to keep up to date with.

And still the release IBM package for AIX [1]. 8-(

The summary of automatic builds (http://curl.haxx.se/auto/) is very nicely presented. Is that custom code?

Thanks for curl, even the old versions!

Mike

[1] http://www-03.ibm.com/systems/power/software/aix/linux/toolbox/alpha.html

Right, but if you set a common lowest denominator first you know what features to expect to be there _at least_, then there might of course be a set of additional ones brought by newer versions. It would reduce the amount of conditionals in the code and what-if-this-is-used scenarios (in the code and in support/docs). It also reduces the risks of git getting odd problems due to very old libcurl bugs.

However, someone who's building/getting git might also be able to build/get a newer libcurl.

The code is custom (perl) but present in the curl CVS repo for the web site and could probably fairly easy be adapted for other purposes/projects.

In the curl project we provide scripts for distributed automatic tests and then we have a central server that receives the reports by mail and the automatic summary script displays the status of those tests on that page.

 / daniel.haxx.se

Yeah, I did this as "How about doing it this way without adding a band-aid configuration options" demonstration, and meant to clean it up (rather, meant to wait for the original submitter to clean-up) before moving it forward, but I forgot. Sorry about that.

How does this look?

http://curl.haxx.se/libcurl/c/curl_easy_setopt.html seems to say "added in 7.X.Y" for some options but does say when CURLOPT_USERPWD was added, so I am assuming it was available even in very early versions...

-- >8 --
From 750d9305009a0f3fd14c0b5c5e62ae1eb2b18fda Mon Sep 17 00:00:00 2001
From: Junio C Hamano <[email protected]>
Date: Thu, 12 Mar 2009 22:34:43 -0700
Subject: [PATCH] http.c: CURLOPT_NETRC_OPTIONAL is not available in ancient versions of cURL

Besides, we have already called easy_setopt with the option before coming to this function if it was available, so there is no need to repeat it here.

---
 http.c |    4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

index b8f947e..2fc55d6 100644
@@ -138,9 +138,7 @@ static int http_options(const char *var, const char *value, void *cb)
 
 static void init_curl_http_auth(CURL *result)
 {
-	if (!user_name)
-		curl_easy_setopt(result, CURLOPT_NETRC, CURL_NETRC_OPTIONAL);
-	else {
+	if (user_name) {
 		struct strbuf up = STRBUF_INIT;
 		if (!user_pass)
 			user_pass = xstrdup(getpass("Password: "));

1.6.2.249.g770a0

I was going to try and clean this up this weekend or early next week. I'm also trying to encourage open source submissions at work and was using this as an example patch to get people going (we need the fix to use git). So I do plan finishing this, just have to do it when I have time.

Daniel, thanks for the link, I had been wondering what was introduced when in curl.

-Mike

-Mike Gaffney (http://rdocul.us)

Thanks.

Yes it was.

Driven by use cases such as this, I also recently produced the "symbols-in-versions" document in the libcurl tree which should help apps to know what should works when:

http://cool.haxx.se/cvs.cgi/curl/docs/libcurl/symbols-in-versions?rev=HEAD&content-type=text/vnd.viewcvs-markup

 / daniel.haxx.se

2009/3/13 Junio C Hamano <[email protected]>:

This patch fixes the build breakage for me, thanks. If I can find a combination of AIX + working gcc + correct 32bit / non-broken 64bit libraries + necessary Gnu tools + ancient curl + Apache2 in this maze of twisty turny servers (all different) I'll give the http server tests a whirl too.

2009/3/13 Daniel Stenberg <[email protected]>:

Very helpful, thanks.

Junio, if I check all the unprotected CURL* options against this list, would that give us our absolute minimum supported version? If so, would it then be ok to remove any unnecessary ifdefs for lower versions if they exist?

Mike

Yeah, I wish we new about it much earlier. Thanks, Daniel.

Sounds like a good plan. Please get the ball rolling.

Hi,

AFAICT my concerns were not addressed either: misleading subject unless the patch is split into two, remote specific config variable instead of global one, security issues.

Ciao,
Dscho

I guess it makes sense to split the config out into two patches. I wanted both to help with automated builds, and as it's a read only account I wasn't worried about someone reading the password. I'm not very impressed with the permissions on the .netrc file actually providing security so I can see not allowing the password in the config either. In my system at work, we have shared machines but all developers have root access, so file permissions don't really secure anything for us. It's also why we can't really use keys (there is no way to enforce that a key is secured afaik).

I wanted to do a remote specific config as well but a global works well in many environments where your push repo is under http as you don't keep having to configure it. I also couldn't see a good way to do a remote specific config without changing the remote struct (which seemd like putting specific in a general). I would love some advice on this and where to put it.

I can see your security points but I would argue that if that's what we are worried about then we should not allow the netrc file at all. I added notes in the config documentation about this. I'm open to discussion on this point.

-Mike Gaffney (http://rdocul.us)

Hi,

I guess, too, because it has been asked for. I guess that since nobody contradicted that wish, it would make sense, I guess.

If Git were written for you, for that very specific setup, then yes, I can see that one does not need to care about storing passwords in plaintext files _there_.

However, in addition to you, Git was written for some others, too.

And $HOME/.netrc is a well established paradigm, many programs check the permissions and flatly refuse to run with a big red warning if the permissions are not set restrictively. So there is definitely a big, huge, vast difference between storing passwords in $HOME/.netrc and storing them in .git/config.

Again, happily the Git team decided that in addition to you, we want to support other users. For example us.

And we _do_ work on computers where only trustworthy people have root access.

IMHO in this case, "works well" does not mean the same as "makes sense" at all.

Again, Git was written for other people, too.

It should not be necessary to say more, but here I go: on two projects I have to push to multiple HTTP servers, and I do have different passwords there.

However, I am pretty convinced that it is a good idea to have the passwords in $HOME/.netrc where they belong instead of in a config where it is all too easy to fsck up the permissions.

BTW that is another reason (in addition to it just being good style, separating different issues into different patches) why I want you to split the patch: to reject something insecure (storing passwords in config) and to accept the secure part (reading passwords interactively from the console).

Umm. Into the remote struct?

See above.

Oh, so you mean you will address my concerns? That's good, as I am looking forward to your answers to them.

Ciao,
Dscho

Johannes,
Your points make sense, thank you for clarifying, it helps
me understand what the underlying concerns are. The attitude doesn't really help.
Does Junio's counter solution where git would prompt for the
password if the username contained a url solve your concerns with unsecure config variables? The patch would be just a bugfix to current functionality.
Also, libcurl does not warn you that you have an insecure netrc file.
Just tried it with 7.19.4 on cygwin and FC9.

Thanks for the feedback,
Mike