Can you do this with GIT?

Hi All,

I hope this is the right forum, it's all I could find. Sincere apologies in advance if I in the wrong place.

I set up a source control system for the company around CVS, but GIT has some very attractive features and I'd like to migrate if it can do some other things that we need.

We produce variations based on a (mostly) common codebase. In CVS I set up "environment" modules for each platform, then when you are working on that platform, you simply check out the correct environment and build. Only the needed code and tools are exposed in that environment (this is important as clients must NOT see each other's code and most customers have some customization). I do this by defining and renaming modules in the CVSROOT modules file.

Does GIT support anything like this? Or another way to acheive the same end?

Thanks muchly, in advance, Colin

View this message in context: http://www.nabble.com/Can-you-do-this-with-GIT--tf4159350.html#a11834063
Sent from the git mailing list archive at Nabble.com.

this sounds very close to what i'm trying to achieve at the moment.

i couldn;t find any ready way to do it.... but i've hacked up a few preliminary thoughts on how this might work in git. comments and feedback are welcome :-)

the basic idea is to import each snapshot of each vendor release as a tagged root commit, then push all these tagged root commits into a central "library" repository. each tagged root commit stands alone and functions as a cvs module.

library repos are set up as remotes under git, and a simple script will import a set of snapshots into a brand new project as a baseline for customization.

the three shell scripts, and the two control files used by them, are included here:

==> vendor-tracking.sh <==
###########################################################
#
# Import a vendor snapshot into a "library" repository. #
# To import a vendor snapshot, unzip the snapshot into

# a new folder.  Then go into the folder and type
#
#  xxx-import <library> <tag>
#
# This will create an empty git repo in the current
# snapshot folder and create a root commit in 
# it which holds the snapshot.  This root commit will be
# tagged as "tag" and then pushed into the "library" repo.
#
# No error checking is done.
#
###########################################################
xxx-import ()
{(
	repo=$1
	tag=$2

export GIT_DIR=/tmp/$$.$tag

rm -rf $GIT_DIR
mkdir -p $GIT_DIR

git init
git add .
git commit -s -m "Import $tag"
git tag $tag master
git push $repo refs/tags/$tag:refs/tags/$tag

rm -rf $GIT_DIR
)}

###########################################################
#
# Use a vendor release in the current project. #
# To use a copy of some vendor snapshot in the current # project, go to the top level of your current project,

# then type
#
#  xxx-use <library> <tag> <path>
#
# The tagged snapshot will be fetched from the library
# and deposited into the <path> subdirectory, then committed
# to the project.
# 
# No error checking is done.
#
###########################################################
xxx-use ()
{(
	repo=$1
	tag=$2
	path=$3

git fetch $repo refs/tags/$tag:refs/tags/$tag git read-tree --prefix=$path/ $tag git commit -s -m "Use $tag as $path"
)}

###########################################################
#
# Create a new project from a set of vendor components. #
# To start a new project with a known array of vendor # components, create a remotes file describing the libraries

# and an manifest file describing the components and how they
# will be used. Then, in a new folder, type
#
#  xxx-start <remotes> <manifest>
#
# This will create a new Git project in the current folder,
# add a remote for each library repository, then proceed to
# fetch and commit each vendor component into the new project.
# The new project is now ready to be customized.
# 
# No error checking is done.
#
###########################################################
xxx-start ()
{(
	remotes=$1
	manifest=$2

git init

cat $remotes |
while read name url
do
git remote add $name $url
done

cat $manifest |
while read path remote tag
do
xxx-use $remote $tag $path
done

git checkout -f master
)}

==> remotes <==
lib1 ssh://[email protected]/home/me/work/library
lib2 //filesrv/git/library

==> manifest <==
path/to/A lib1 vendorA/componentA/version1
path/to/B lib2 vendorB/componentB/version2

i would prefer to import and tag raw tree objects, since i get some of my snapshots as pre-assembled collections of (modified copies of) several components. it would therefore be convenient to have several tags per snapshot pointing at the subtrees corresponding to each component. however, fetch and push seem to prefer commits over trees....

as it stands, the vendor snapshots have no "history" to them. each snapshot stands in isolation. there was an interesting thread a few days ago about importing ancient kernel history as a series of trees, then "stitching together" a "synthetic history" from a collection of trees. i would see something like that as the basis for "vendor tracking branches", which could be restitched if an older snapshot is received "out of order".

hope it helps
ray

Hi,

You might be interested in http://git.or.cz: there is a lot of useful information to find; amongst others what is the right forum...

So yes, this is the right forum.

I would use branches for that. A base branch with the common code, and the customisations in all the branches, which merge from the base branch.

If you have an interesting change in a custom branch, you can percolate that back into the base branch, by checking out that base branch and cherry-picking the commit you want to have.

Of course, you can automate this merging (or even rebasing, if you are interested in keeping your customisations nice and tidy) with a script. With a git alias even.

Hth,
Dscho

this would break down if there were client-specific modules in the base branch, though... how could those be hidden from the other clients?

ray

Hi,

Umm. Don't put the client-specific modules in the base branch, then? The base branch is the common code, the code that every client may look at. Nothing else.

Maybe I did not get the whole picture... do you want your _clients_ to access your main repo with Git?

Ciao,
Dscho

yes, i don;t think there's any other way to do it with branches (and just be careful not to merge the private branches back! :-)

not in my case, anyway... although if you define "client" to include "subcontractor", then yes, i'd be interested in going down that road... sometimes there are pieces which they aren't licensed to see, but if we could somehow track their work in a separate repo and then easily merge it back into the fully licensed tree, that would have great value.

from the original message, i also keyed on "Only the needed code and tools are exposed in that environment". this is something i'd like to see too... i'm working in a software eco-system with hundreds of standalone components, and would like the ability to pick and choose a small handful of those for any given project, without pulling in reams of history on the stuff i'm not using...

thanks
ray

Depending on how big the individual pieces are, you can do it either with branches or subprojects.

If the "mostly common" codebase means that 90% of it really ends up being common, and what you have is a few config things and perhaps a library or two that either shows up or not, git branches would work really well.

And branches are totally independent of each other, so it's quite possible to have even "secret" things in one branch, and it would never get exposed to anybody else. The biggest worry you'd need to make sure of is that you don't want to merge such private branches directly with each other - since then the merged branch would see everything that was in the history of the other branch, even if the merge itself ended up not picking the "secret stuff".

So with branches that have private information, you'd have to have one "common" branch that doesn't have any of the private stuff at all, and do the general development on that, and then you can merge that common stuff into all the individual private branches (and if you want to go the other way, moving something from a private branch to the "common area", you'd have to cherry-pick just that particular detail into the common branch).

If you have bigger building blocks, and some of the whole building blocks need to be private, you'd probably want to use the subproject support, which would make for more of a CVS "modules" kind of thing and in that sense perhaps closer to what you do now, but I suspect that just using branches would actually end up being much simpler for you. It would depend mostly on how big those building blocks that you need to keep separate are.

Linus