git svn clone a non-standard repository

Hi,

I'm trying to create a git repo of the Dojo Toolkit, which has a quite non-standard layout. There are 4 modules to the current Dojo project that I care about (and some others in the repo that I'm not interested in). The problem is, the trunk code is split out into <modulename>/trunk, while tags are in tags/<tagname>/<modulename> and branches are in branches/<branchname>/<modulename>. e.g:

moduleA/trunk <- contains trunk development of moduleA
moduleA/tags <- empty
moduleA/branches <- empty
moduleB/trunk
moduleB/tags
moduleB/branches
moduleC/trunk
moduleC/tags
moduleC/branches
moduleD/trunk
moduleD/tags
moduleD/branches
moduleE/trunk <- I don't care about this one...
trunk/ <- contains ancient version, not actual trunk
tags/1.0.0/moduleA <- contains tagged version of moduleA tags/1.0.0/moduleB <- contains tagged version of moduleB tags/1.0.0/moduleC <- contains tagged version of moduleC tags/1.0.0/moduleD <- contains tagged version of moduleD tags/1.0.1/moduleA
tags/1.0.1/moduleB
...

So I'd like to set up a git repo that tracks this SVN repository, and allows me to see:
moduleA/
moduleB/
moduleC/
moduleD/
... in my checkout, whether I'm on trunk or a tag.

What's the best way to set this up?

I've started with "git svn clone http://path/to/svn -T moduleA/trunk -t tags -b branches", and it's been sucking down branches for a couple days now, still not done. Can I set up moduleB/moduleC/moduleD as additional remotes in this same repository, and end up with the desired result? Was thinking I would add additional svn sections to .git/config, and then git svn fetch -- will this work, or is there a better way?

Thanks,

John Locke
"Open Source Solutions for Small Business Problems"
published by Charles River Media, June 2004
http://www.freelock.com

John Locke venit, vidit, dixit 24.06.2008 20:32:

All of those module?/{tags,branches} are empty, right? I assume branches follow the pattern below for tags.

So you want one git repo, reflecting the modules simply by different subdirs? I'm afraid that raises similar problems like those I asked about in my (yet unanswered) post

http://permalink.gmane.org/gmane.comp.version-control.git/85861

This should give you all "modules" (as subsdirs) for all branches and tags already. svn branch "branchname" becomes git remote branch "branchname", svn tag "tagname" becomes git remote branch "tags/tagname". Note that your new git branch trunk will contain the contents of "moduleA" only.

If I understand your layout (and "desires") correctly then trunk will be your only problem: you want to map "module?/trunk" to subdir "module?" of branch "trunk". This is more or less the problem I'm facing. Regular expressions (other than the ones for branches and tags) in git svn config would solve this.

Michael

Hi, Michael,

Yup.

I saw your post, and flagged it to see if anybody answered it, yes it does seem similar.

That's exactly what happened.

I found in the docs a section on subtrees, and with a bit of trickery, I think I'm getting what I want, though it's not simple to update. What I've done so far is change .git/config to:

[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[svn-remote "svn"]
url = http://svn.dojotoolkit.org/src
branches = branches/*:refs/remotes/* tags = tags/*:refs/remotes/tags/*
[svn-remote "dojo"]
url = http://svn.dojotoolkit.org/src
fetch = dojo/trunk:refs/remotes/dojo-trunk
[svn-remote "dijit"]
url = http://svn.dojotoolkit.org/src
fetch = dijit/trunk:refs/remotes/dijit-trunk
[svn-remote "dojox"]
url = http://svn.dojotoolkit.org/src
fetch = dojox/trunk:refs/remotes/dojox-trunk
[svn-remote "util"]
url = http://svn.dojotoolkit.org/src
fetch = util/trunk:refs/remotes/util-trunk

... dojo, dijit, dojox, and util correspond to moduleA-D in my previous example. So I have "svn" for the branches, and separate SVN sections for the trunk of each module. Then I used git svn fetch [dojo|dijit|dojox|util] to suck down those as individual branches.

Now the tricky part: I checked out a new "work" branch, and deleted everything in the working copy. Then, 4 steps out of the howto, adjusted to pull the particular branch from the current repository (instead of an outside one):

git remote add -t dojo -f dojo ./
git merge -s ours --no-commit dojo-trunk git read-tree --prefix=dojo/ -u dojo-trunk git commit -m "merge dojo into subdirectory"

... repeat for the other modules, and I ended up with my "work" branch containing the full trunk for all 4 modules.

Now for tracking the original project. According to the howto, you use this syntax to pull down changes:

git pull -s subtree Bproject master
... when I try git pull -s subtree ./ dojo, I get "Already up to date", even though I know there are new changes. Still need to mess around with git svn rebase, I guess, to get the updates working correctly.

Cheers,

John Locke
"Open Source Solutions for Small Business Problems"
published by Charles River Media, June 2004
http://www.freelock.com

Sorry to respond to my own post, but this section looks wrong:

... since I added the remotes as svn-remote sections directly in git-config, I skipped that first line.

I'm going to write this all up on my blog when I get it working correctly...

Cheers,

John Locke
"Open Source Solutions for Small Business Problems"
published by Charles River Media, June 2004
http://www.freelock.com

Okay. I think I've closed the loop, and have this updating successfully, if not necessarily easily.

To update, we need a local branch for each submodule:

git checkout -b dojo dojo-trunk <- create local branch and switch to it (only create it the first time through)

git svn rebase dojo-trunk <- do this to update each local branch from the remote svn repository version--must have corresponding branch checked out.

... now that we've merged the remote changes to a local branch, we can update our work branch from each module's branch:

git checkout work
git pull -s subtree ./ dojo
git pull -s subtree ./ dijit
....

John Locke
"Open Source Solutions for Small Business Problems"
published by Charles River Media, June 2004
http://www.freelock.com