Adapt to new semantics of remote-helper "import" command
Git 1.7.7 (commit 9504bc9d, "transport-helper: change import semantics", 2011-07-16) incompatibly changed the interface of the "import" capability.
Before, git would always send a single import command, which the remote helper would respond to with a fast-import stream, terminated by end of file, meaning there was no way to fetch multiple refs in one connection. Nowadays, git instead sends a sequence of import lines:
import refs/heads/foo
import refs/heads/bar
terminated by a blank line. The helper is to respond with a fast-import stream terminated by the "done" command and process further commands until another blank line indicates the end of the command stream.
---
Hi Simon and Gabriel,
Here's a rough patch against git://github.com/lelutin/git-remote-bzr.git
master.
Without this patch, whenever I try to use "git clone bzr::<something>", after doing all the work it removes the resulting repo and exits with status 141 (SIGPIPE). Maybe the transport-helper should mask SIGPIPE when writing the final newline to avoid that.
I'd have prefered to write a patch for remote-bzr that works with older versions of git fast-import, too, but it wasn't obvious how. Hints welcome.
BTW, would you mind if I sent a patch to include git-remote-bzr in git.git under contrib/?
Thanks for git remote-bzr! I'd be happy for any thoughts you have.
Ciao,
Jonathan
[1] http://thread.gmane.org/gmane.comp.version-control.git/176002/focus=176606
README.rst | 2 +- git-remote-bzr | 33 ++++++++++++++++++++++++++++++--- 2 files changed, 31 insertions(+), 4 deletions(-)
README.rst +1 -1
@@ -34,7 +34,7 @@ Relevant bug reports Requirements ------------ -- git 1.6.6 or later +- git 1.7.7 or later - python 2.5 + - bzr 2.x - bzr-fastimport
git-remote-bzr +30 -3
@@ -49,7 +49,7 @@ def do_list(repo, args): print # end list -def do_import(repo, args): +def import_one_ref(repo, args): """Import a fast-import stream that is exported from Bazaar.""" if len(args) != 1: die("Import needs exactly one ref")
@@ -65,6 +65,23 @@ def do_import(repo, args): if bzrp.wait(): die("'bzr fast-export' returned unexpectedly with code %d", bzrp.returncode) + print "done" + + +def do_import(repo,args): + import_one_ref(repo, args) + + cmdline = True + while cmdline: + cmdline = next_command() + if not cmdline: + # Return to main processing loop + return True + cmd = cmdline.pop(0) + if cmd != "import": + warn("Unexpected command %s during import" % cmd) + return False + import_one_ref(repo, cmdline) def do_push(repo, args):
@@ -123,8 +140,8 @@ def sanitize(value): return value -def read_one_line(repo): - """Read and process one command.""" +def next_command(): + """Read one command.""" line = sys.stdin.readline() cmdline = line
@@ -138,6 +155,16 @@ def read_one_line(repo): # Blank line means we're about to quit return False + return cmdline + + +def read_one_line(repo): + """Read and process one command.""" + cmdline = next_command() + + if not cmdline: + return False + cmd = cmdline.pop(0) debug("Got command '%s' with args '%s'", cmd, ' '.join(cmdline))
Signature · 1 line
1.7.9.rc2
Quoted 27 lines · Jonathan Nieder
Git 1.7.7 (commit
9504bc9d, "transport-helper: change import semantics", 2011-07-16) incompatibly changed the interface of the "import" capability.Before, git would always send a single import command, which the remote helper would respond to with a fast-import stream, terminated by end of file, meaning there was no way to fetch multiple refs in one connection. Nowadays, git instead sends a sequence of import lines:
import refs/heads/foo
import refs/heads/barterminated by a blank line. The helper is to respond with a fast-import stream terminated by the "done" command and process further commands until another blank line indicates the end of the command stream.
---
Hi Simon and Gabriel,Here's a rough patch against
git://github.com/lelutin/git-remote-bzr.git
master.Without this patch, whenever I try to use "git clone bzr::<something>", after doing all the work it removes the resulting repo and exits with status 141 (SIGPIPE). Maybe the transport-helper should mask SIGPIPE when writing the final newline to avoid that.
Quoted 6 lines
I'd have prefered to write a patch for remote-bzr that works with older versions of git fast-import, too, but it wasn't obvious how. Hints welcome.
BTW, would you mind if I sent a patch to include git-remote-bzr in git.git under contrib/?
Please note that the bzr-git package, which provides git integration for bzr and vice versa, also includes a 'git-remote-bzr' command. Apart from the 'import' command, it includes experimental implementations of 'fetch' and push as well.
It would be nice to consolidate the efforts, or at the very least prevent name clashes.
Cheers,
Jelmer
Quoted 7 lines · Jelmer Vernooij
Quoted 2 lines · Jonathan Nieder
BTW, would you mind if I sent a patch to include git-remote-bzr in git.git under contrib/?
Please note that the bzr-git package, which provides git integration for bzr and vice versa, also includes a 'git-remote-bzr' command.
That's good to hear. Then there should be no need for git.git to have its own helper.
Unfortunately, when I try to clone any repo, at the last step I get
Traceback (most recent call last):map 6/7
File "/usr/lib/git-core/git-remote-bzr", line 220, in <module>
commands[argv[0]](argv, shortname, remote_dir)
File "/usr/lib/git-core/git-remote-bzr", line 89, in cmd_list
for ref, git_sha1 in refs.as_dict().iteritems():
File "/usr/lib/python2.7/dist-packages/dulwich/repo.py", line 196, in as_dict
ret[key] = self[("%s/%s" % (base, key)).strip("/")]
File "/usr/lib/python2.7/dist-packages/dulwich/repo.py", line 267, in __getitem__
_, sha = self._follow(name) File "/usr/lib/python2.7/dist-packages/dulwich/repo.py", line 249, in _follow
contents = self.read_ref(refname)
File "/usr/lib/python2.7/dist-packages/dulwich/repo.py", line 225, in read_ref
contents = self.read_loose_ref(refname)
File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/git/refs.py", line 129, in read_loose_ref
tag_name = ref_to_tag_name(ref)
File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/git/refs.py", line 89, in ref_to_tag_name
raise ValueError("unable to map ref %s back to tag name" % ref)
ValueError: unable to map ref refs/heads back to tag name
Will file a bug.
Hello,
Quoted 4 lines · Jonathan Nieder
Hi Simon and Gabriel,
Here's a rough patch against
git://github.com/lelutin/git-remote-bzr.git
master.
great! thanks for your help.
I must admit that this project never got to completion and is now getting quite the low priority for my part. I'm no longer working for the company that was using mainly Bazaar as their VCS, and I've been a happy git-only user for some time now.
So, I don't have the same incentive to complete the project as I had before.
But I'd be happy to see this get to a point where it's working fine.
Quoted 8 lines
Without this patch, whenever I try to use "git clone bzr::<something>", after doing all the work it removes the resulting repo and exits with status 141 (SIGPIPE). Maybe the transport-helper should mask SIGPIPE when writing the final newline to avoid that.
I'd have prefered to write a patch for remote-bzr that works with older versions of git fast-import, too, but it wasn't obvious how. Hints welcome.
hmm.. I can wait some time to see if some ideas come out around this, and commit your patch as-is if there are no comments/reworks.
Quoted 2 lines
BTW, would you mind if I sent a patch to include git-remote-bzr in git.git under contrib/?
absolutely not, that'd be great actually :)
I didn't do that up to now, though, since I bumped into so much bugs that I couldn't work out -- some very bad performance issues, and problems with handling mark files with bzr-fastimport.
Quoted 1 line
Thanks for git remote-bzr! I'd be happy for any thoughts you have.
The idea behind git-remote-bzr was to be able to interact with Bazaar from within your git repository, i.e. to expose remote branches that you can pull from and push to using the default git commands, without having to learn to use yet another tool since the remote-helper would be interfacing with the tool for you.
I have dived for a short period of time into bzrlib, the python library behind Bazaar, to see how much work it would take to put together a simplified fast-import client.. and .... wechrk!
It was a huge maze of version-dependant code (the API undergoes big changes frequently, and backwards compatibility is maintained pretty far) mixed with a 40-foot-deep class hierarchy. So I quickly gave up on that idea..
IIRC, at the time I started work on this, Simon and I used bzr-fastimport[1] because we were able to get farther with this tool. I haven't used bzr-git[2] that much, though, so I can't comment too extensively on it. But I would guess that it is maintained more frequently than bzr-fastimport is, so it could be a better choice for the backend fast-import client.
I would guess that Jelmer would be happy to help out with interfacing with bzr-git.
Signature · 1 line
Gabriel Filion