Skip to contentSkip to conversation

How to fork a file (git cp ?)

DISCUSSION Mikhail T. 8 participants 19 messages Jun 15, 2016

lore
Share & tools
lore

Mikhail T.

author
#0 lore raw

I need to add a new thing to our project. The thing will be similar to what already exists. I'd like to "derive" the new files from the existing ones -- without altering them and by preserving the change-history.

This is not a separate branch -- the "forked" files will co-exist. I'd call this git-cp (analogous to git-mv), but it does not exist...

Any ideas? Thanks!

-mi

#1 lore raw

Am 04.05.2011 19:56, schrieb Mikhail T.:

Quoted 34 lines

I need to add a new thing to our project. The thing will be similar to what already exists. I'd like to "derive" the new files from the existing ones -- without altering them and by preserving the change-history.

This is not a separate branch -- the "forked" files will co-exist. I'd call this git-cp (analogous to git-mv), but it does not exist...

Any ideas? Thanks!

   -mi
#2 lore raw
In reply to Mikhail T.

(Sorry for the botched previous post - fat fingers)

Am 04.05.2011 19:56, schrieb Mikhail T.:

Quoted 4 lines

I need to add a new thing to our project. The thing will be similar to what already exists. I'd like to "derive" the new files from the existing ones -- without altering them and by preserving the change-history.

You cannot. Git does not have such a thing as "copy-with-preserved-history".

You just cp the file and git add it. But you will not be able to follow a history of the file.

-- Hannes

#3 lore raw

----- Original Message -----

Quoted 11 lines

From: "Johannes Sixt" <[email protected]>
Sent: Wednesday, May 4, 2011 2:22:13 PM
Subject: Re: How to fork a file (git cp ?)

Am 04.05.2011 19:56, schrieb Mikhail T.:

Quoted 5 lines

I need to add a new thing to our project. The thing will be similar to
what already exists. I'd like to "derive" the new files from the existing ones -- without altering them and by preserving the change-history.

Quoted 5 lines

You cannot. Git does not have such a thing as
"copy-with-preserved-history".

You just cp the file and git add it. But you will not be able to follow a history of the file.

Log (and other commands) can search for copies while traversing history:

-C[<n>]
--find-copies[=<n>]
Detect copies as well as renames. See also --find-copies-harder. If n is specified, it has the same meaning as for -M<n>.

--find-copies-harder
For performance reasons, by default, -C option finds copies only if the original file of the copy was modified in the same changeset. This flag makes the command inspect unmodified files as candidates for the source of copy. This is a very expensive operation for large projects, so use it with caution. Giving more than one -C option has the same effect.

But as I discovered a few weeks ago, the existing merge strategies don't understand copies (recursive can follow a rename, but if two files pass rename detection, I think the one with the higher similarity index wins).

Thanks,
Stephen

author
#4 lore raw
In reply to Johannes Sixt
Quoted 2 lines · Johannes Sixt

You just cp the file and git add it. But you will not be able to follow a history of the file.

Thank you for the information...

So, is this something worth adding to the wishlist, or was it omitted on purpose (and which purpose was that, then)?

Yours,

-mi

#5 lore raw
Quoted 9 lines · Mikhail T.
Quoted 3 lines · Johannes Sixt

You just cp the file and git add it. But you will not be able to follow a history of the file.

Thank you for the information...

So, is this something worth adding to the wishlist, or was it omitted on purpose (and which purpose was that, then)?

Oh yes, that was intentional. This is easily one of the most debated "features" of Git, especially in the early days of Git when almost all SCM systems did it "the CVS way", by tracking the history of single files. Instead, Git tracks snapshots of the whole tree and focuses on the whole content instead of single files. Renames are tracked by detecting removal/adding of files, which can be detected later, for example using "git log --follow". The reason for this is mostly speed issues, and most of the time the history of a single file is not interesting in a project, but changes in the file tree as a whole.

From the FAQ at <https://git.wiki.kernel.org/index.php/GitFaq>:

Git has to interoperate with a lot of different workflows, for example some changes can come from patches, where rename information may not be available. Relying on explicit rename tracking makes it impossible to merge two trees that have done exactly the same thing, except one did it as a patch (create/delete) and one did it using some other heuristic.

On a second note, tracking renames is really just a special case of tracking how content moves in the tree. In some cases, you may instead be interested in querying when a function was added or moved to a different file. By only relying on the ability to recreate this information when needed, Git aims to provide a more flexible way to track how your tree is changing.

However, this does not mean that Git has no support for renames. The diff machinery in Git has support for automatically detecting renames, this is turned on by the '-M' switch to the git-diff-* family of commands. The rename detection machinery is used by git-log(1) and git-whatchanged(1), so for example, 'git log -M' will give the commit history with rename information. Git also supports a limited form of merging across renames. The two tools for assigning blame, git-blame(1) and git-annotate(1) both use the automatic rename detection code to track renames.

As a very special case, 'git log' version 1.5.3 and later has '--follow' option that allows you to follow renames when given a single path.

Git has a rename command git mv, but that is just for convenience. The effect is indistinguishable from removing the file and adding another with different name and the same content.

This mail from Linus explains the issue in more detail and colour:
<http://permalink.gmane.org/gmane.comp.version-control.git/217>.

Regards,
Øyvind

#6 lore raw
In reply to Johannes Sixt
Quoted 7 lines · Johannes Sixt

Am 04.05.2011 19:56, schrieb Mikhail T.:

Quoted 4 lines

I need to add a new thing to our project. The thing will be similar to what already exists. I'd like to "derive" the new files from the existing ones -- without altering them and by preserving the change-history.

You cannot. Git does not have such a thing as "copy-with-preserved-history".

Well, if you come from the mindset that a "file" has an identity (hence there is a distinction between "This file used to be called A and at one point was renamed to B which is the name we see today" and "Some time ago somebody created a file B with the same contents as A and then removed A at the same time"), "copy" would not make much sense. What identity does a new file B gets when you create it by copying from A?

The same identity, or a different one? What happens when you later refactor the redundant part from these two files to create a common third file C? What identity does C have?

Quoted 2 lines

You just cp the file and git add it. But you will not be able to follow a history of the file.

Correct. You cannot follow a history of _the file_, as there is no such thing.

You can still follow the history of contents, though. If you did a refactor like the one in the above example, "blame -L <range>" would follow the contents just fine. The command is a 80% satisfactory implementation of Linus's grand vision expressed in one of the most important message in the git mailing list archive:

    http://thread.gmane.org/gmane.comp.version-control.git/27/focus=217
author
#7 lore raw
Quoted 6 lines · Junio C Hamano

Well, if you come from the mindset that a "file" has an identity (hence there is a distinction between "This file used to be called A and at one point was renamed to B which is the name we see today" and "Some time ago somebody created a file B with the same contents as A and then removed A at the same time"), "copy" would not make much sense. What identity does a new file B gets when you create it by copying from A?

What I want is to signify something like: "This code was obtained from that in file A."

"copy" -- of an individual file -- makes just as much sense as "move" (rename).

-mi

#8 lore raw
Quoted 9 lines · Mikhail T.
Quoted 6 lines · Junio C Hamano

Well, if you come from the mindset that a "file" has an identity (hence there is a distinction between "This file used to be called A and at one point was renamed to B which is the name we see today" and "Some time ago somebody created a file B with the same contents as A and then removed A at the same time"), "copy" would not make much sense. What identity does a new file B gets when you create it by copying from A?

What I want is to signify something like: "This code was obtained from that in file A."

I think that is what exactly "blame -C -C" gives you.

author
#9 lore raw
Quoted 1 line · Junio C Hamano

I think that is what exactly "blame -C -C" gives you.

For that to be useful, one has to suspect, the file was derived by copying something else... Simple "git log" will not suggest that -- unless the commit message, that adds the new copy of a file points to it...

Quoted 2 lines · Piotr Krukowiecki

Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for
"cp old new ; rm old; add new"

git-mv preserves the old's change-history in new, so it is more than the above, is not it?

Quoted 1 line

then there should be a git-cp as a shortcut for

Yes...

-mi

#10 lore raw
Quoted 6 lines · Mikhail T.
Quoted 1 line · Junio C Hamano

I think that is what exactly "blame -C -C" gives you.

For that to be useful, one has to suspect, the file was derived by copying something else... Simple "git log" will not suggest that -- unless the commit message, that adds the new copy of a file points to it...

I think "git log --follow" will do what you want; it uses FIND_COPIES_HARDER.

Quoted 5 lines · Piotr Krukowiecki
Quoted 2 lines

Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for
"cp old new ; rm old; add new"

git-mv preserves the old's change-history in new, so it is more than the above, is not it?

No. The renames are detected at the time of viewing, not at the time of commit.

-Peff

#11 lore raw
In reply to Mikhail T.
Quoted 7 lines · Mikhail T.
Quoted 2 lines · Junio C Hamano

I think that is what exactly "blame -C -C" gives you.

For that to be useful, one has to suspect, the file was derived by copying something else... Simple "git log" will not suggest that -- unless the commit message, that adds the new copy of a file points to it...

Maybe it should be the default (performance issues?)

Quoted 7 lines · Piotr Krukowiecki
Quoted 3 lines

Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for "cp old new ; rm old; add new"

git-mv preserves the old's change-history in new, so it is more than the above, is not it?

It's the same IMO: log with --follow will follow both "copies" and "renames".

BTW, I don't understand why 'status' shows renames but not copies:

$ cp f fcp && git add fcp && git status
# Changes to be committed:
#	new file:   fcp
$ mv f fmv && git add fmv && git rm f && git status
# Changes to be committed:
#	renamed:    f -> fmv

I would expect sth like "copied: f -> fcp".
Not sure what about one file copied to multiple files, but I suppose renames have the same problem.

It should not be a problem performance-wise...

Quoted 3 lines
Quoted 1 line

then there should be a git-cp as a shortcut for

Yes...

Signature · 1 line
Piotr Krukowiecki
#12 lore raw
Quoted 7 lines · Piotr Krukowiecki
Quoted 5 lines
Quoted 1 line

I think that is what exactly "blame -C -C" gives you.

For that to be useful, one has to suspect, the file was derived by copying something else... Simple "git log" will not suggest that -- unless the commit message, that adds the new copy of a file points to it...

Maybe it should be the default (performance issues?)

Performance is part of it, but also the fact that "--follow" has some limitations. For example, you can't use it with arbitrary pathspecs. I hope to fix that at some point. There was a GSoC proposal, but it didn't get selected; I'm hoping to work on it myself sometime this summer.

Quoted 11 lines

BTW, I don't understand why 'status' shows renames but not copies:

$ cp f fcp && git add fcp && git status
# Changes to be committed:
#	new file:   fcp
$ mv f fmv && git add fmv && git rm f && git status
# Changes to be committed:
#	renamed:    f -> fmv

I would expect sth like "copied: f -> fcp".

Yeah, we probably should do copy detection. Even weirder, we seem to do rename detection for what's to be committed, but not for what's in the worktree.

If you want to do a patch, the changes would go in wt-status.c, in the functions:

wt_status_collect_changes_worktree
wt_status_collect_changes_index

Quoted 1 line

It should not be a problem performance-wise...

For people running "git status" manually, no. But something like FIND_COPIES_HARDER may be expensive on a big tree for people who use "git status" output as part of their shell prompt. So probably it should be configurable.

-Peff

#13 lore raw
In reply to Piotr Krukowiecki
Quoted 6 lines · Piotr Krukowiecki

BTW, I don't understand why 'status' shows renames but not copies:

$ cp f fcp && git add fcp && git status
# Changes to be committed:
#       new file:   fcp
Quoted 7 lines
$ mv f fmv && git add fmv && git rm f && git status
# Changes to be committed:
#       renamed:    f -> fmv

I would expect sth like "copied: f -> fcp".
Not sure what about one file copied to multiple files, but I suppose renames have the same problem.

I mean similar problem - you can delete one or more identical files and add one or more new files with the same content.

Quoted 1 line

It should not be a problem performance-wise...

Signature · 1 line
Piotr Krukowiecki
#16 lore raw
In reply to Junio C Hamano
Quoted 9 lines · Junio C Hamano
Quoted 7 lines · Mikhail T.
Quoted 6 lines · Junio C Hamano

Well, if you come from the mindset that a "file" has an identity (hence there is a distinction between "This file used to be called A and at one point was renamed to B which is the name we see today" and "Some time ago somebody created a file B with the same contents as A and then removed A at the same time"), "copy" would not make much sense.  What identity does a new file B gets when you create it by copying from A?

Quoted 4 lines
Quoted 2 lines

What I want is to signify something like: "This code was obtained from that in file A."

I think that is what exactly "blame -C -C" gives you.

Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for
"cp old new ; rm old; add new"
then there should be a git-cp as a shortcut for
"cp old new; add new"

Just for convenience (and symmetry with git-mv).

He did write:

Quoted 1 line

"copy" -- of an individual file -- makes just as much sense as "move" (rename).

Signature · 1 line
Piotr Krukowiecki
#17 lore raw
Quoted 4 lines · Piotr Krukowiecki

Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for
"cp old new ; rm old; add new"
then there should be a git-cp as a shortcut for
"cp old new; add new"

Copying and then futzing with a copy is a bad discipline to begin with. git already has a reputation of having too many commands. I am not thrilled about the idea of making things worse by adding an unnecessary command, and especially one that encourages a bad workflow.

#18 lore raw
Quoted 11 lines · Junio C Hamano
Quoted 4 lines · Piotr Krukowiecki

Maybe Mikhail wanted to say that if there's a git-mv as a shortcut for "cp old new ; rm old; add new"
then there should be a git-cp as a shortcut for
  "cp old new; add new"

Copying and then futzing with a copy is a bad discipline to begin with. git already has a reputation of having too many commands.  I am not thrilled about the idea of making things worse by adding an unnecessary command, and especially one that encourages a bad workflow.

I'm not saying I want to have git-cp, but could you explain what is wrong with "cp old new; add new" (why is that bad workflow? How come adding a copy of a file can be a workflow at all?)

Signature · 1 line
Piotr Krukowiecki
#14 lore raw
In reply to Piotr Krukowiecki
Quoted 10 lines · Piotr Krukowiecki
Quoted 7 lines · Mikhail T.
Quoted 2 lines · Junio C Hamano

I think that is what exactly "blame -C -C" gives you.

For that to be useful, one has to suspect, the file was derived by copying something else... Simple "git log" will not suggest that -- unless the commit message, that adds the new copy of a file points to it...

Maybe it should be the default (performance issues?)

...

Quoted 1 line

BTW, I don't understand why 'status' shows renames but not copies:

Rename detection compares new files against deleted files. Copy detection compares new files against every file in the tree, which is usually much more costly.

--Pete

#15 lore raw
Quoted 3 lines · Pete Harlan

Rename detection compares new files against deleted files. Copy detection compares new files against every file in the tree, which is usually much more costly.

Almost correct. Copy detection usually compares new paths with paths modified in the same commit. "Every file in the tree" is done only when you ask to find copies harder.