Problem with git-cvsimport

Hello,

using git-cvsimport (1.5.3.4), it dies with

Update guidance-common/src/java/com/jentro/manager/guidance/common/servlet/IconServlet.java:
2104 bytes
Tree ID 01cb84cbee2e70a712459be6601b993603eed5bd
Parent ID dcd8dc76f4638d1994165070c9813202992d546a
Committed patch 3775 (bmw +0000 2004-10-14 11:10:43)
Commit ID 53c68066f71651b057884e1101cda3967070724d
Fetching guidance-common/src/java/com/jentro/manager/guidance/common/serverapi/GuidanceException.java
v 1.14.4.2
Update guidance-common/src/java/com/jentro/manager/guidance/common/serverapi/GuidanceException.java:
3718 bytes
Tree ID 886268190ac2cb28b5f1e6cdb309054bcb8fa38e
Parent ID 53c68066f71651b057884e1101cda3967070724d
Merge parent branch: master
fatal: Not a valid object name master
Use of uninitialized value in chomp at /usr/bin/git-cvsimport line 766.
Use of uninitialized value in pattern match (m//) at
/usr/bin/git-cvsimport line 527.
Use of uninitialized value in concatenation (.) or string at /usr/bin/git-cvsimport line 767.
Cannot get commit id ():

What can I do to avoid this problem?

Cheers,

Thomas

I've had some similar problem. I've converted two big old repositories by first converting to SVN using:

cvs2svn -s myrepo-svn /path/to/cvsmodule git-svnimport -i -u -C /path/to-git file://myrepo-svn

Worked like a charm

Cheers --- Jan

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I would actually plug using cvs2svn to convert directly to git. See this thread for Michael's original announcement.

http://marc.info/?l=git&m=118592701426175&w=2

I'm in the process of drafting Samba's own git repos from the CVS and SVN history (http://gitweb.samba.org/).

cheers, jerry
=====================================================================
Samba ------- http://www.samba.org
Centeris ----------- http://www.centeris.com
"What man is a man who does not make the world better?" --Balian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHC4BJIR7qMdg1EfYRAlQ4AKDctlXFv0kcT51sA6P99qjVrPJ+MgCfWkCB wPSf6l06UIlz0HERasHbryg=
=zSSf
-----END PGP SIGNATURE-----

Seconded! I've tried git-cvsimport, parsecvs, fromcvs, and cvs2svn on my employer's many large CVS modules, and cvs2svn is the only one that has never mangled an import.

That said, it is a work in progress, so there are some caveats:

* Setting up the direct conversion to git is more work than it should (and probably will, eventually) be.

* There is no support for incremental importing, and git-cvsimport _will_ mess up your git repository sooner or later if you try to use it for subsequent incremental imports.

* Tags each get a branch with a single commit, with the actual tag pointing to that commit. This makes it harder than necessary to figure out what the history looks like; gitk's default view won't show any tags, for example, since it only shows the master branch and not the single-commit tag branches.

* Branches all get a useless commit at their branch point. All branches from the main branch appear to be merged from the vendor branch (ie, the useless commit has the vendor branch as an extra parent), which might make sense to someone who knows what the vendor branch is for, but makes no sense to me. This combined with the previous point makes "gitk --all" look needlessly spaghetti-like if you have a slightly complicated CVS history.

To sum up, cvs2svn gets the important stuff right, but has some sharp corners you need to watch so you don't put an eye out.

Eyvind Bernhardsen

I'm glad this worked.

I just fixed this in cvs2svn trunk r4213. Now it reuses a single branch called 'refs/heads/TAG.FIXUP' whenever it needs to make a tag fixup branch, and it resets that branch when done. (Resetting the tag fixup branch changes it to 0000000000000000000000000000000000000000 but doesn't really delete it; I don't know the ramifications of that but at least it doesn't appear in gitk output any more.)

I assume that the "useless commit" that you are referring to is the one with log message "This commit was manufactured by cvs2svn to create branch 'BRANCH'." Is that correct?

I'm not a git expert, so I don't know whether these commits are in fact useless. But let me explain the reason I put them in and you can tell me whether it is nonsense.

When you branch a file in CVS, CVS notes that the branch exists in the file but doesn't record an author, log message, or timestamp. The contents of a file just after it is branched are exactly the same as the contents on the parent branch. Moreover, different files can be added to a branch from different parent branches.

The intended purpose of the "useless commit" in the git output of cvs2svn is to record the fact that a branch was created, to record exactly which files exist on the branch at the time it was created, and to record the source branches of the file on the branch.

I imagine that *if* a branch is created with a single parent branch, and *if* each and every file from the parent branch is added to the new branch, then it is possible that the "useless commit" could be omitted.
But this decision would require information that cvs2svn doesn't
currently have at that stage of the conversion, and keeping the necessary records would be quite expensive.

But in the general case, it doesn't seem to me that the commits are really useless. Am I wrong? If so, please tell me what should be changed in the git-fast-import data that is output when a branch is created (e.g., for the main-cvsrepos in the cvs2svn test suite).

Regarding the superfluous vendorbranch parent: vendor branches are an obscure CVS feature for tracking upstream sources. The file contents on the vendorbranch are typically exactly the same as that on trunk, and if a branch is created while the vendorbranch is active, CVS doesn't record whether the branch's parent was trunk or the vendorbranch. Haven't yet built the heuristics into cvs2svn to make this decision more intelligently, so sometimes "vendorbranch" is listed as a branch parent when it could be omitted.

Thanks for the feedback!

Michael

[1] http://www.kernel.org/pub/software/scm/git/docs/git-fast-import.html

On 10. okt.. 2007, at 04.34, Michael Haggerty wrote:

[...]

Thanks for the response! I should have explained that my goal is to convince my coworkers that git is worth the effort to learn, and IMO, making imported git repositories look as "clean" as possible to someone familiar with the CVS repository is an important part of that. My suggestions and complaints should be seen in that light :)

Great! I'll try this out at the first opportunity.

The main problem I have with the current branched-tag behaviour is that nothing exists on the branch to show that a tag is branched off, so no tags are shown at all when running gitk on a single branch. A git newbie who is familiar with the CVS repository will probably just assume that tags haven't been imported, which is unfortunate.

Tagging the branch point is an obvious solution to that problem, but I wonder if branching for tag fixups could be made optional. Instead of representing a tag with differences to the commit it's closest to like this:

    o--*--o--...
        \--t

(where "*" represents the closest commit on the branch, and "t" represents the fixup commit), might the following be possible and/or desirable?

    o--*--t--r--o--...

"r" being a commit that reverts the changes in "t".

It might just be me, but I think of a tag as being "on" a branch even when it has been messed with (sliding, omitting files, etc), and a linear history preserves that illusion.

Yes. And I'm regretting my choice of words already. "Unexpected commit" would have been better.

[...]

I know barely enough about git (or CVS, for that matter) to be dangerous, but your explanation makes sense to me.

Right; that is how I think of a branch, even if CVS doesn't. To me, the extra commit is useless because it should be identical to the branch point commit on the parent branch. I don't think having an extra commit there is a huge problem, though.

[...]

I don't think I've ever seen the vendor branch used for anything but the initial import (creation) of a repository, and then only because it is a required parameter to "cvs import". Is there any chance you could add an option to ignore the vendor branch?

Thanks for making cvs2svn the best CVS-to-git conversion tool :) Now if it would only support incremental importing...

Eyvind Bernhardsen

On 10/10/07, Eyvind Bernhardsen <[email protected]> wrote:
...

Michael,

I second this question: is there any chance incremental importing will be implemented in cvs2svn?

I've not used cvs2svn much and when I did it was for svn not git; but given that git-cvsimport is known to mess up your git repo (as Eyvind pointed out earlier) there doesn't appear to be any reliable tools to allow for incrementally importing from cvs to git.

Are others using a tool for reliably importing from cvs to git?

After reading the fairly recent "cvs2svn conversion directly to git ready for experimentation" thread it is clear that its doable but hasn't been done (seeing as you were looking for volunteers to do it).

Sorry for the noise,
Mike

tisdag 30 oktober 2007 skrev Mike Snitzer:

I use fromcvs which is *very* fast, and quite memory conservative compared to the others and seems reliable so far (six months). It probably breaks on exotic variants of branches though, but I don't have those / don't care about them.

Do not push into the same repo fromcvs works on. I don't understand why, but I pushed once and *poof* the conversion went bad.

Drawbacks, more dependencies and access to the rcs files is required and tags are not converted.

-- robin

I actually use fromcvs for a few repositories, and actually started using it on repositories where cvsps (and git-cvsimport) fail.

Most projects have "rsyncs" or "tarballs" of their CVS repository available, making fromcvs possible on most of them. And CVS tags, well they are about as good as CVS $Keywords$.

Unfortunately, no, there is not much chance that I will implement this.
I wouldn't be interested in a works-most-of-the-time solution, and a
reliable solution would take weeks to implement.

If somebody else wants to implement this feature, I would be happy to help him get started, answer questions, discuss the design, etc. Or if somebody wants to sponsor the work, I might be able to justify working on it myself. But otherwise, I'm afraid it is unlikely to happen.

That's because it is quite a tricky problem, especially since CVS allows history to be changed retroactively; for example,

- shift a tag to a different file revision

- add an existing tag to a new file or remove it from an old file

- delete ("obsolete") old revisions

- change files from vendor branches to main line of development

- even nastier server-side repository manipulations like deleting an RCS file, renaming a file, etc.

These things really happen in the topsy-turvy CVS world; indeed, they are a part of many organizations' standard workflow.

cvs2svn uses repository-wide information in the heuristics that it uses to determine changesets, choose branch parents, fix clock skew, etc. Therefore the naive approach of running a full conversion a second time and just skipping over the revisions that were handled during the first conversion would not even begin to work. (I believe that this is the approach of cvsps, which uses mostly local information to determine changesets.)

I think the correct approach would involve recording the "frontier" of the CVS repository, then at the next incremental conversion:

1. compare the current CVS repository to the recorded information

2. emit "fixup" changesets to reflect any CVS changes that happened behind the previous "frontier".

3. emit changesets to reflect CVS changes beyond the frontier.

It is step 2 that is IMO the trickiest because it is so open-ended, and modern SCMs don't allow all of the corresponding operations in any straightforward way. Presumably one would have to prohibit some of the nastier CVS tricks and abort the incremental conversion if any are detected.

Furthermore, for many use-cases of incremental conversion the conversion would have to run quickly. Therefore, the incremental conversion code should be written with a strong emphasis on achieving good performance.

Michael