Selective fetch possible?
Hi everybody,
Is it possible to git-fetch only a portion of the tree of the specified repository, say, fetch only one directory or a subset of files matching some regular expression? This is currently
- to my knowledge - only possible via wget iff the GIT repository has gitweb enabled. But that's just a workaround.
Thanks for the great instrument that GIT is.
Best regards.
PS: please CC me in replies, because I'm not subscribed to git ML.
Signature · 12 lines
Filippo Zangheri GPG key ID: 0xE1D879FA Key fingerprint: 816B CE57 D43C 0A47 EF35 3378 EA5F A72A E1D8 79FA Key server: pgp.mit.edu -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GE d- s+:- a-- C++ UL+++ P+ L+++ E-- W+ N* o-- K- w--- O-- M-- V- PS++ PE+ Y+ PGP++ t 5-- X++ R* tv b+ DI-- D---- G-- e++ h-- r++ z* ------END GEEK CODE BLOCK------
Quoted 5 lines · Filippo Zangheri
Is it possible to git-fetch only a portion of the tree of the specified repository, say, fetch only one directory or a subset of files matching some regular expression? This is currently
- to my knowledge - only possible via wget iff the GIT repository has gitweb enabled. But that's just a workaround.
No.
You can use a shallow clone to fetch only X commits back into history on any branch, and you can also manually configure the fetch specification in .git/config to only fetch specific branches, but you must fetch the entire tree to get any of the files in it.
If the repository is available by git:// protocol you may be able to use git-archive to obtain a tarfile for just the directory you want (service has to be enabled on the remote side) but that is just a raw UNIX tar; there is no Git repository and no ability to commit/fetch/push/diff/apply/log/etc.
Signature · 1 line
Shawn.
Quoted 15 lines · Shawn O. Pearce
Quoted 6 lines · Filippo Zangheri
Is it possible to git-fetch only a portion of the tree of the specified repository, say, fetch only one directory or a subset of files matching some regular expression? This is currently
- to my knowledge - only possible via wget iff the GIT repository has gitweb enabled. But that's just a workaround.No.
You can use a shallow clone to fetch only X commits back into history on any branch, and you can also manually configure the fetch specification in .git/config to only fetch specific branches, but you must fetch the entire tree to get any of the files in it.
Quoted 5 lines
If the repository is available by
git://protocol you may be able to use git-archive to obtain a tarfile for just the directory you want (service has to be enabled on the remote side) but that is just a raw UNIX tar; there is no Git repository and no ability to commit/fetch/push/diff/apply/log/etc.
Note that what you wanted is, I guess, something called partial checkout or subtree checkout. This feature appears now and then in feature requests; lately Nguyen Thai Ngoc Duy (pclouds) offered to do this in "on subtree checkout" thread:
http://thread.gmane.org/gmane.comp.version-control.git/74915
The problem is twofold, as far as I understand it. First, what to do if there is merge conflicts outside checked out (selected) directory? Second, how to make repository contain only relevant objects: git in many places assumes full connectivity, and that if it has an object it hass all objects depending on it.
Signature · 3 lines
Jakub Narebski Poland ShadeHawk on #git
Quoted 21 lines · Jakub Narebski
Quoted 13 lines · Shawn O. Pearce
Quoted 5 lines · Filippo Zangheri
Is it possible to git-fetch only a portion of the tree of the specified repository, say, fetch only one directory or a subset of files matching some regular expression? This is currently
- to my knowledge - only possible via wget iff the GIT repository has gitweb enabled. But that's just a workaround.No.
You can use a shallow clone to fetch only X commits back into history on any branch, and you can also manually configure the fetch specification in .git/config to only fetch specific branches, but you must fetch the entire tree to get any of the files in it.
Quoted 5 lines
If the repository is available by
git://protocol you may be able to use git-archive to obtain a tarfile for just the directory you want (service has to be enabled on the remote side) but that is just a raw UNIX tar; there is no Git repository and no ability to commit/fetch/push/diff/apply/log/etc.
Quoted 5 lines
Note that what you wanted is, I guess, something called partial checkout or subtree checkout. This feature appears now and then in feature requests; lately Nguyen Thai Ngoc Duy (pclouds) offered to do this in "on subtree checkout" thread:
http://thread.gmane.org/gmane.comp.version-control.git/74915
I still believe that it could be done fairly easily by simply recording the SHA1's of the files and directories that are *not* checked out somewhere in the .git directory, and just reusing those when checking the working tree. i.e. rather than stat-ing a tree that was never checked out, get the known SHA1 for that tree from where it was recorded on checkout.
Quoted 2 lines
The problem is twofold, as far as I understand it. First, what to do if there is merge conflicts outside checked out (selected) directory?
This is something that has been repeated many times, and I fail to see how it can be an issue. How can there be a conflict in a directory that is not, and never has been, checked out, and therefore cannot have been modified?
The only possibility that I can see is if the directory has been renamed elsewhere, but in that case, it *is* effectively checked out (just with a different directory name).
Quoted 4 lines
Second, how to make repository contain only relevant objects: git in many places assumes full connectivity, and that if it has an object it hass all objects depending on it.
Yes, this is the big problem as I see it.
Rogan
Quoted 8 lines · Rogan Dawes
Quoted 6 lines · Jakub Narebski
Quoted 4 lines · Shawn O. Pearce
Quoted 3 lines · Filippo Zangheri
Is it possible to git-fetch only a portion of the tree of the specified repository, say, fetch only one directory or a subset of files matching some regular expression?
Quoted 7 lines
Quoted 2 lines
The problem is twofold, as far as I understand it. First, what to do if there is merge conflicts outside checked out (selected) directory?
This is something that has been repeated many times, and I fail to see how it can be an issue. How can there be a conflict in a directory that is not, and never has been, checked out, and therefore cannot have been modified?
Given two branches:
code
docs
and the code people checkout the "src/" subdirectory and the docs people checkout the "Documentation/" subdirectory, and they *only* every work in that subdirectory, things are fine.
Until one day some developer also checks out "Documentation/" and fixes something in the documentation as part of the same commit that makes a code change. The push this to the code branch.
Someday in the future a documentation writer merges the code branch over to the docs branch, "just keeping it current".
Now there arises a possiblity of a merge conflict in a part of the tree that you do not have checked out.
If you want to say "don't ever modify stuff outside of your branch's purpose" then why aren't you just using submodules (one for docs and one for code) and using a supermodule to tie everything together into a "release package"?
Quoted 6 lines
Quoted 4 lines
Second, how to make repository contain only relevant objects: git in many places assumes full connectivity, and that if it has an object it hass all objects depending on it.
Yes, this is the big problem as I see it.
This is easy enough that if the above problem could be resolved sufficiently to the git gurus' satisfaction you would be able to get some advice on how to solve it. Its not difficult, just damn annoying. We already do it (to some extent) with grafts and shallow clones.
Signature · 1 line
Shawn.
Quoted 28 lines · Shawn O. Pearce
Quoted 10 lines · Rogan Dawes
Quoted 8 lines · Jakub Narebski
Quoted 4 lines · Shawn O. Pearce
Quoted 3 lines · Filippo Zangheri
Is it possible to git-fetch only a portion of the tree of the specified repository, say, fetch only one directory or a subset of files matching some regular expression?
The problem is twofold, as far as I understand it. First, what to do if there is merge conflicts outside checked out (selected) directory?
This is something that has been repeated many times, and I fail to see how it can be an issue. How can there be a conflict in a directory that is not, and never has been, checked out, and therefore cannot have been modified?
Given two branches:
code
docsand the code people checkout the "src/" subdirectory and the docs people checkout the "
Documentation/" subdirectory, and they *only* every work in that subdirectory, things are fine.Until one day some developer also checks out "
Documentation/" and fixes something in the documentation as part of the same commit that makes a code change. The push this to the code branch.Someday in the future a documentation writer merges the code branch over to the docs branch, "just keeping it current".
Quoted 8 lines
Now there arises a possiblity of a merge conflict in a part of the tree that you do not have checked out.
If you want to say "don't ever modify stuff outside of your branch's purpose" then why aren't you just using submodules (one for docs and one for code) and using a supermodule to tie everything together into a "release package"?
Ok, fair enough. Thanks for the example.
I think that one should not *expect* to be able to complete merges with only a partial checkout, though. It *may* work in cases where there are no conflicts, but I think it would be a perfectly valid error path to fail if there is a conflicting merge in a part of the tree that has not been checked out.
So, for a user working on partial trees, they would be able to modify their partial tree, and check in their changes, but merges would have to be done by someone with a complete checkout. For the given examples where partial trees make sense (documentation workers), this seems like a reasonable compromise.
Quoted 11 lines
Quoted 5 lines
Quoted 4 lines
Second, how to make repository contain only relevant objects: git in many places assumes full connectivity, and that if it has an object it hass all objects depending on it.
Yes, this is the big problem as I see it.
This is easy enough that if the above problem could be resolved sufficiently to the git gurus' satisfaction you would be able to get some advice on how to solve it. Its not difficult, just damn annoying. We already do it (to some extent) with grafts and shallow clones.
How's my suggestion above?
Rogan
Rogan Dawes ha scritto:
(...)
Quoted 11 lines
I think that one should not *expect* to be able to complete merges with only a partial checkout, though. It *may* work in cases where there are no conflicts, but I think it would be a perfectly valid error path to fail if there is a conflicting merge in a part of the tree that has not been checked out.
So, for a user working on partial trees, they would be able to modify their partial tree, and check in their changes, but merges would have to be done by someone with a complete checkout. For the given examples where partial trees make sense (documentation workers), this seems like a reasonable compromise.
I think this is what every reasonable developer should have in mind when he's working on just a project subset :). And I also think, this is not a valid reason for forbidding/not implementing such a partial (or subtree) checkout.
Signature · 12 lines
Filippo Zangheri GPG key ID: 0xE1D879FA Key fingerprint: 816B CE57 D43C 0A47 EF35 3378 EA5F A72A E1D8 79FA Key server: pgp.mit.edu -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GE d- s+:- a-- C++ UL+++ P+ L+++ E-- W+ N* o-- K- w--- O-- M-- V- PS++ PE+ Y+ PGP++ t 5-- X++ R* tv b+ DI-- D---- G-- e++ h-- r++ z* ------END GEEK CODE BLOCK------