New SCM and commit list
Hi Linus !
Do you intend to continue posting "commited" patches to a mailing list like bk scripts did to bk-commits-head@vger ? As I said a while ago, I find this very useful, especially with the actual patch included in the commit message (which isn't the case with most other projects CVS commit lists, and I find that annoying).
If yes, then I would appreciate if you could either keep the same list, or if you want to change the list name, keep the subscriber list so those of us who actually archive it don't miss anything ;)
Thanks !
Regards,
Ben.
Quoted 6 lines · Benjamin Herrenschmidt
Do you intend to continue posting "commited" patches to a mailing list like bk scripts did to bk-commits-head@vger ? As I said a while ago, I find this very useful, especially with the actual patch included in the commit message (which isn't the case with most other projects CVS commit lists, and I find that annoying).
Absolutely. GIT isn't quite at the point where I can start using it yet, though.
I could actually start committing patches, but I want to make sure that I can also do automated simple merges, so that there is any _point_ to doing this in the first place. My plan is to not be very good at merging (in particular, I don't see GIT resolving renames _at_all_), but my hope is that the people who used to merge with me using BK might be able to still do so using GIT, as long as we try actively to be very careful.
Quoted 3 lines
If yes, then I would appreciate if you could either keep the same list, or if you want to change the list name, keep the subscriber list so those of us who actually archive it don't miss anything ;)
I didn't even set up the list. I think it's Bottomley. I'm cc'ing him just so that he sees the message, but I don't actually expect him to do anything about it. I'm not even ready to start _testing_ real merges yet. But I hope that I can get non-conflicting merges done fairly soon, and maybe I can con James or Jeff or somebody to try out GIT then...
Linus
Quoted 10 lines · Linus Torvalds
Quoted 3 lines · Benjamin Herrenschmidt
If yes, then I would appreciate if you could either keep the same list, or if you want to change the list name, keep the subscriber list so those of us who actually archive it don't miss anything ;)
I didn't even set up the list. I think it's Bottomley. I'm cc'ing him just so that he sees the message, but I don't actually expect him to do anything about it. I'm not even ready to start _testing_ real merges yet. But I hope that I can get non-conflicting merges done fairly soon, and maybe I can con James or Jeff or somebody to try out GIT then...
Not guilty. If I remember correctly, the list was set up by the vger list maintainers (davem and company). It was tied to a trigger in one of your trees (which I think Larry did). It shouldn't be too difficult to add to git ... it just means traversing all the added patches on a merge and sending out mail.
I can try out your source control tools ... I have some rc fixes ready ... when you're ready to try out merges...
James
Quoted 12 lines · James Bottomley
Quoted 10 lines · Linus Torvalds
Quoted 3 lines · Benjamin Herrenschmidt
If yes, then I would appreciate if you could either keep the same list, or if you want to change the list name, keep the subscriber list so those of us who actually archive it don't miss anything ;)
I didn't even set up the list. I think it's Bottomley. I'm cc'ing him just so that he sees the message, but I don't actually expect him to do anything about it. I'm not even ready to start _testing_ real merges yet. But I hope that I can get non-conflicting merges done fairly soon, and maybe I can con James or Jeff or somebody to try out GIT then...
Quoted 8 lines
Not guilty. If I remember correctly, the list was set up by the vger list maintainers (davem and company). It was tied to a trigger in one of your trees (which I think Larry did). It shouldn't be too difficult to add to git ... it just means traversing all the added patches on a merge and sending out mail.
I can try out your source control tools ... I have some rc fixes ready ... when you're ready to try out merges...
I have some rc fixes too, let us know when you are ready to accept them, and what format you want them in.
I have a feeling that the kernel.org mirror system is just going to _love_ us using it to store temporary git trees :)
thanks,
greg k-h
Quoted 3 lines · Greg KH
I have a feeling that the kernel.org mirror system is just going to _love_ us using it to store temporary git trees :)
I don't think kernel.org mirrors the private home directories, so it you do _temporary_ trees, just make them readable in your private home directory rather than in /pub/linux/kernel/people. For people with kernel.org accounts, we can use that as the "bkbits.net" thing.
For really public hosting, we need to find some other approach.
Linus
Quoted 4 lines · Linus Torvalds
I don't think kernel.org mirrors the private home directories, so it you do _temporary_ trees, just make them readable in your private home directory rather than in /pub/linux/kernel/people. For people with kernel.org accounts, we can use that as the "bkbits.net" thing.
It's also going to be a slight problem for those of us who don't have a kernel.org account...although I think the hosting I use on the parisc website might actually be outside the HP firewall, so I can probably beg for it to run any protocol you need (like rsync).
James
Quoted 10 lines · James Bottomley
Quoted 4 lines · Linus Torvalds
I don't think kernel.org mirrors the private home directories, so it you do _temporary_ trees, just make them readable in your private home directory rather than in /pub/linux/kernel/people. For people with kernel.org accounts, we can use that as the "bkbits.net" thing.
It's also going to be a slight problem for those of us who don't have a kernel.org account...although I think the hosting I use on the parisc website might actually be outside the HP firewall, so I can probably beg for it to run any protocol you need (like rsync).
rsync also runs over ssh so if you can ssh in you can rsync to it
Followup to: <[email protected]> By author: Linus Torvalds <[email protected]> In newsgroup: linux.dev.kernel
Quoted 6 lines · Greg KH
Quoted 3 lines
I have a feeling that the kernel.org mirror system is just going to _love_ us using it to store temporary git trees :)
Quoted 7 lines
I don't think kernel.org mirrors the private home directories, so it you do _temporary_ trees, just make them readable in your private home directory rather than in /pub/linux/kernel/people. For people with kernel.org accounts, we can use that as the "bkbits.net" thing.
For really public hosting, we need to find some other approach.
It's also pretty trivial to set up an additional /pub hierarchy, like the current /pub/scm, which is up to individual mirrors to pick up or not to pick up. We only require /pub/linux and /pub/software to be mirrored.
-hpa
Quoted 7 lines · Linus Torvalds
Quoted 3 lines · Benjamin Herrenschmidt
If yes, then I would appreciate if you could either keep the same list, or if you want to change the list name, keep the subscriber list so those of us who actually archive it don't miss anything ;)
I didn't even set up the list. I think it's Bottomley. I'm cc'ing him just so that he sees the message, but I don't actually expect him to do anything about it. I'm not even ready to start _testing_ real merges yet.
When you think kernel.org and BitKeeper, think either me or David Woodhouse. :)
DaveM / Matti(?) manage the lists ([email protected]), but largely just create them on request from others, and make sure they continue to work.
Quoted 1 line
But I hope that I can get non-conflicting merges done fairly soon, and maybe I can con James or Jeff or somebody to try out GIT then...
I don't mind being a guinea pig as long as someone else does the hard work of finding a new way to merge :)
Jeff
Quoted 6 lines · Jeff Garzik
Quoted 3 lines
But I hope that I can get non-conflicting merges done fairly soon, and maybe I can con James or Jeff or somebody to try out GIT then...
I don't mind being a guinea pig as long as someone else does the hard work of finding a new way to merge :)
So I can tell you what merges are going to be like, just to prepare you.
First, the good news: I think we can make the workflow look like bk, ie pretty much like "git pull" and "git push". And for well-behaved stuff (ie minimal changes to the same files on both sides) it will even be fast. I think.
Then the bad news: the merge algorithm is going to suck. It's going to be just plain 3-way merge, the same RCS/CVS thing you've seen before. With no understanding of renames etc. I'll try to find the best parent to base the merge off of, although early testers may have to tell the piece of crud what the most recent common parent was.
So anything that got modified in just one tree obviously merges to that version. Any file that got modified in two trees will end up just being passed to the "merge" program. See "man merge" and "man diff3". The merger gets to fix up any conflicts by hand.
Quite frankly, that means that we really want to avoid any "exciting" merges with GIT. Maybe somebody can come up with something smarter. Eventually. Don't count on it, at least not in the near future.
The good news is that it's not like a three-way file merge is any worse than many people are used to. The bad news is that BK is just a hell of a lot better. So anybody who has been depending heavily on BK merges (and hey, the beauty of them is that you often don't even _know_ that you are depending on them) will be a bit bummed by the "Welcome back to the 1980's" message from a three-way merge.
Linus
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Jeff Garzik
Quoted 2 lines
But I hope that I can get non-conflicting merges done fairly soon, and maybe I can con James or Jeff or somebody to try out GIT then...
I don't mind being a guinea pig as long as someone else does the hard work of finding a new way to merge :)
Quoted 6 lines
So I can tell you what merges are going to be like, just to prepare you.
First, the good news: I think we can make the workflow look like bk, ie pretty much like "git pull" and "git push". And for well-behaved stuff (ie minimal changes to the same files on both sides) it will even be fast. I think.
If you can stick something meaningful in a simple text file, overwritten after each merge completes, similar to the BitKeeper/csets-in file, it should be trivial to write a wrapper for the basic merge tool that calls a trigger after each merge and uses csets-in to generate diffs and email them out.
Signature · 2 lines
Ryan Anderson sometimes Pug Majere
Quoted 2 lines · Linus Torvalds
Then the bad news: the merge algorithm is going to suck. It's going to be just plain 3-way merge, the same RCS/CVS thing you've seen before. With no
Actually 3-way merge is not that bad. It's definitely better than ClearCase's merge (I always fall back to RCS merge if ClearCase cannot resolve a merge automatically).
Quoted 3 lines
understanding of renames etc. I'll try to find the best parent to base the merge off of, although early testers may have to tell the piece of crud what the most recent common parent was.
Yep, finding the best parent is the important part :-)
I guess 3-way merge got a bad name because CVS always uses the branch point as the parent, which fails miserably for any but the first merge after the branch.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Quoted 10 lines · Linus Torvalds
Then the bad news: the merge algorithm is going to suck. It's going to be just plain 3-way merge, the same RCS/CVS thing you've seen before. With no understanding of renames etc. I'll try to find the best parent to base the merge off of, although early testers may have to tell the piece of crud what the most recent common parent was.
So anything that got modified in just one tree obviously merges to that version. Any file that got modified in two trees will end up just being passed to the "merge" program. See "man merge" and "man diff3". The merger gets to fix up any conflicts by hand.
at that point Chris Mason's "rej" tool is pretty nifty:
ftp://ftp.suse.com/pub/people/mason/rej/rej-0.13.tar.gz
it gets the trivial rejects right, and is pretty powerful to quickly cycle through the nontrivial ones too. It shows the old and new code side by side too, etc.
(There is no fully automatic mode in where it would not bother the user with the really trivial rejects - but it has an automatic mode where you basically have to do nothing - maybe a fully automatic one could be added that would resolve low-risk rejects?)
it's really easy to use (but then again i'm a vim user, so i'm biased), just try it on a random .rej file you have ("rej -a kernel/sched.c.rej" or whatever).
Ingo
Quoted 6 lines · Ingo Molnar
Quoted 4 lines · Linus Torvalds
So anything that got modified in just one tree obviously merges to that version. Any file that got modified in two trees will end up just being passed to the "merge" program. See "man merge" and "man diff3". The merger gets to fix up any conflicts by hand.
Quoted 9 lines
at that point Chris Mason's "rej" tool is pretty nifty:
ftp://ftp.suse.com/pub/people/mason/rej/rej-0.13.tar.gz
(There is no fully automatic mode in where it would not bother the user with the really trivial rejects - but it has an automatic mode where you basically have to do nothing - maybe a fully automatic one could be added that would resolve low-risk rejects?)
rej -M skips the merge program, so rej -a -M will give you something like this:
coffee:/local/linux.p # rej -a -M drivers/ide/ide.c.rej
drivers/ide/ide.c: 1 matched, 0 conflicts remain
But I would want to go over the bit that calculates the conflicts remaining more carefully if people plan on trusting this ;) It'll run on unified diffs too, although it will be slower then patch since the assumption is the quick and easy placement patch does has already failed. (that's easy enough to fix though).
Quoted 3 lines
it's really easy to use (but then again i'm a vim user, so i'm biased), just try it on a random .rej file you have ("rej -a
kernel/sched.c.rej" or whatever).
you can rej -m kdiff3|meld|tkdiff or any program that does a side by side comparison of two files. (export REJMERGE=foo sets the diff prog as well)
I use rej frequently to merge patches in here, but that is mostly because there is no easy way to get the common ancestor and parent revision of the patches I'm merging.
With that info in hand, kdiff3 is pretty nice. You would have to spoon feed it the renames, but it should have most of the other features you're looking for, including the 'no gui if all conflicts are auto-solvable'
-chris
Quoted 8 lines · Chris Mason
rej -M skips the merge program, so rej -a -M will give you something like this:
coffee:/local/linux.p # rej -a -M
drivers/ide/ide.c.rej
drivers/ide/ide.c: 1 matched, 0 conflicts remainBut I would want to go over the bit that calculates the conflicts remaining more carefully if people plan on trusting this ;)
Ok, looks like this should be safe. I changed -q to skip the gui compare when rej thinks it has resolved all the conflicts correctly. With rej 0.14 (just uploaded now) this should do what you want:
rej -q -a foo.rej
Download site is here: ftp://ftp.suse.com/pub/people/mason/rej/
Please let me know if you find patches where rej is doing the wrong thing.
-chris
Quoted 21 lines · Linus Torvalds
Quoted 6 lines · Jeff Garzik
Quoted 3 lines
But I hope that I can get non-conflicting merges done fairly soon, and maybe I can con James or Jeff or somebody to try out GIT then...
I don't mind being a guinea pig as long as someone else does the hard work of finding a new way to merge :)
So I can tell you what merges are going to be like, just to prepare you.
First, the good news: I think we can make the workflow look like bk, ie pretty much like "git pull" and "git push". And for well-behaved stuff (ie minimal changes to the same files on both sides) it will even be fast. I think.
Then the bad news: the merge algorithm is going to suck. It's going to be just plain 3-way merge, the same RCS/CVS thing you've seen before. With no understanding of renames etc. I'll try to find the best parent to base the merge off of, although early testers may have to tell the piece of crud what the most recent common parent was.
Quoted 4 lines
So anything that got modified in just one tree obviously merges to that version. Any file that got modified in two trees will end up just being passed to the "merge" program. See "man merge" and "man diff3". The merger gets to fix up any conflicts by hand.
If merge took trees instead of single files, and had some way of detecting renames (or it got additional information about the differences between files), would that give BK-quality performance? Or does BK also support cases like:
orig ---> first ---> first-merge -
| / \ |------> second - -> final | \ / |------> third ---> third-merge -
where the final merge requires, for complete cleanliness, a comparison of more than 3 states (since some changes will have orig as the common ancestor and some will have second).
Does this happen in real life? It seems like sane development processes wouldn't have multiple mainline-candidate patch sets including the same patches, if for no other reason than that, should the merge fail, nobody with any clue about the original patches would be anywhere nearby. It seems better to throw something back to someone to rebase their diffs.
Otherwise, the problem seems to boil down to finding the common ancestor well, getting trees instead of files to merge, and improving merge until it handles all of the tractible cases.
-Daniel
*This .sig left intentionally blank*
Quoted 3 lines · Daniel Barkalow
If merge took trees instead of single files, and had some way of detecting renames (or it got additional information about the differences between files), would that give BK-quality performance? Or does BK also support
I wrote a script to do merges on a tree (so far without rename detection, though ;-) a long time ago, and still use it every time Linus or Marcelo release a new version.
Look at `mergetree' on http://linux-m68k-cvs.ubb.ca/~geert/
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Quoted 4 lines · Linus Torvalds
So anything that got modified in just one tree obviously merges to that version. Any file that got modified in two trees will end up just being passed to the "merge" program. See "man merge" and "man diff3". The merger gets to fix up any conflicts by hand.
"merge" does a better job than "diff3" since it can resolve the conflicts caused by similar changes to a "parent" file (this is available in both BK and GNU Arch). This is useful when you try to merge 2 branches that both include a patch which is not under the revision control. It also solves the conflicts caused by cherry-picking changes (just need to find the last consecutive common changeset as the common ancestor).
Signature · 1 line
Catalin
Quoted 1 line
"merge" does a better job than "diff3" since it can resolve the
The merge command I know of is part of Tichy's RCS tools, and calls diff3, and has no inherent superior abilities.
Is this the merge command you have in mind here?
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373, 1.925.600.0401Quoted 4 lines · Paul Jackson
Quoted 1 line
"merge" does a better job than "diff3" since it can resolve the
The merge command I know of is part of Tichy's RCS tools, and calls diff3, and has no inherent superior abilities.
You are right, I missed some diff3 options. It looks like "diff3 -mE" generates the same output as "merge" (i.e. solving the identical changes in the derived files). Sorry for the noise :-)
Signature · 1 line
Catalin
Quoted 9 lines · Benjamin Herrenschmidt
Do you intend to continue posting "commited" patches to a mailing list like bk scripts did to bk-commits-head@vger ? As I said a while ago, I find this very useful, especially with the actual patch included in the commit message (which isn't the case with most other projects CVS commit lists, and I find that annoying).
If yes, then I would appreciate if you could either keep the same list, or if you want to change the list name, keep the subscriber list so those of us who actually archive it don't miss anything ;)
The commits lists currently only accept posts from dwmw2@hera, I believe. That can relatively easily be changed if the mail is going to come from somewhere else.
I did ask Linus to let me know as soon as possible when he starts to commit patches, so we can come up with a way to keep the list fed. Since he thinks I'm James, however, I suspect that part of the message didn't get through. Perhaps he was just distracted by the Britishness?
Signature · 1 line
dwmw2