Wow! Is memory ever cheap!
I'm confused by the "lets not use ECC and use bk" talk.
My understanding is suns big machines stopped using ecc and they started to have "random" problems running big-iron applications that took them a while to figure out (and a lot of bad press) and can only be rectified in the big cycle (this was last year so its probably solved now).
I thought one of the primary reasons to have ecc is to catch wierd things before they become catostrophic...and at least know WHY weirdness is happening...
marty
Quoted 1 line · Marty Leisner
I'm confused by the "lets not use ECC and use bk" talk.
I'll take a pass at unconfusing you, I can see how you might be. I wish I had never mentioned BK, that was never the point. End to end was the point, BK was just an example and now I'm getting accused of bringing up the whole thread as a BK advertisement. Which completely misses the point. Please go read
which is a text version of the paper I mentioned before. The basic message of the paper is that it really doesn't help much to have things like ECC unless you can be sure that 100% of the rest of your system has similar checks.
The point was made again, but apparently missed here, when I pointed out that Linux's disk subsystem passes up bad data when it knows there may be a problem. ECC will not help you in this case, the data was bad before it hit memory. So now you have carefully error corrected BAD DATA. See the point? ECC doesn't help unless every other component is equally careful; those components include software and hardware. You can fix that chunk of software and then I'll go find a rogue disk controller that breaks the datapath, there are plenty to choose from.
Just to make sure you understand: I think ECC is a fine thing. If I'm running systems with no other integrity checks, I'll take ECC and like it. However, having ECC does not mean that I trust that my data is safe, that is most certainly not a true statement. The bus, the disks, the disk controller, the disk driver, the buffer cache, etc, can all corrupt
the data. Oh, yeah, let's not forget NFS. I have seen each and every
one of those things corrupt data.
As to the BitKeeper stuff, those of you who think this is a BitKeeper discussion are off wacking in the weeds. The point isn't that BitKeeper is good because it has integrity checks, the point is that integrity
checks are a good thing. Period. BitKeeper was just an example.
If there was a Linux filesystem that had built in integrity checks (and I knew about it, for all I know there is one), then I would have used that as the example. I used BitKeeper as an example because I know it and I can point to numerous cases where it exposed problems that ECC would not have caught. Ask Dave Miller about the mmap/read sparc linux cache aliasing bug that BK exposed, that one was nasty.
Let's review: ECC is nice, but it doesn't solve all data corruption problems. Applications which do their own end to end data integrity checks will catch many more error cases than what ECC catches. My efforts in this thread had nothing to do with BitKeeper, they were trying to get people to realize that end to end is good, and ECC isn't end to end.
Examples of end to end applications, which I should have thought of sooner, are the md5sums on ftp.kernel.org, the integrity checks in rpms, crcs in cpio. I'm sure you can think of lots of others, this is an old problem.
Quoted 1 line
My understanding is suns big machines stopped using ecc and they
The SUN problem was a cache problem and there is no way that I believe that SUN would turn of ECC in the cache. There are good reasons for not doing so. If you think through the end to end argument, you will see that you have no way to do checks on the data path into/out of the processor. If that part of the datapath is not checked then no amount of checking elsewhere does any good, the processor can be corrupting your data and never know it. If SUN was so stupid as to remove this, then it is a dramatically different place. I heard that there was a bug in the cache controller, I never heard that they had removed ECC. If you really want to know I can ask, I know at least one of the guys who works on that stuff there.
Signature · 2 lines
--- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Quoted 4 lines · Larry McVoy
which is a text version of the paper I mentioned before. The basic message of the paper is that it really doesn't help much to have things like ECC unless you can be sure that 100% of the rest of your system has similar checks.
UDMA has crc, scsi has parity, pci has (i think) parity, tcpip has crc, your cpu l1 and l2 have ecc...
Looks like similar checks are already there.
-Dan
Quoted 8 lines · Larry McVoy
Just to make sure you understand: I think ECC is a fine thing. If I'm running systems with no other integrity checks, I'll take ECC and like it. However, having ECC does not mean that I trust that my data is safe, that is most certainly not a true statement. The bus, the disks, the disk controller, the disk driver, the buffer cache, etc, can all corrupt
the data. Oh, yeah, let's not forget NFS. I have seen each and every
one of those things corrupt data.
This is an interesting observation of a truth that was well known in the second generation computers of the 1950s and 1960s. I first worked at John Hancock... they had a bunch of 7074 machines. All those systems made use of programmed checksums in each tape block and in each full file. The reason was that those machines did not have ECC... they did have parity checking if I remember right. With IBM's third generation computers (S/360s) and probably other manufacturers, ECC became a standard feature. Parity checking was added through different data paths such as channel memory, buffer memory, etc. There was so much protection added that the programmed checksums became superfluous.
There were still odd moments. I remember working on an Amdahl computer problem where some internal data paths... where the contents of one register moved to an internal storage area... and the path did not have parity. There was a machine fault... the path was electrically open, so the contents of the register always became zero. But since it wasn't parity checked, there was no machine check. I remember another problem on the IBM 3033. Cosmic rays (really) caused one bit errors in channel memory. That was parity but not ECC so you got a weird channel check. Back at the diagnosis ranch, the board looked good. It was only when someone noticed that the rate of such problems was proportional to the height above sea level that the light bulb went on.
The lesson is that when paths are not checked, hardware or software, data being held or transformed can change. Old lesson but a good one to know.
john alvord
Quoted 12 lines · Larry McVoy
Quoted 1 line · Marty Leisner
My understanding is suns big machines stopped using ecc and they
The SUN problem was a cache problem and there is no way that I believe that SUN would turn of ECC in the cache. There are good reasons for not doing so. If you think through the end to end argument, you will see that you have no way to do checks on the data path into/out of the processor. If that part of the datapath is not checked then no amount of checking elsewhere does any good, the processor can be corrupting your data and never know it. If SUN was so stupid as to remove this, then it is a dramatically different place. I heard that there was a bug in the cache controller, I never heard that they had removed ECC.
There are issues with error detection/correction/recovery with different designs of L1 and L2 caches. There's a good paper:
IBM S/390 storage hierarchy - G5 and G6 performance considerations IBM Journal of Research and Development
Vol 43 No. 5/6
available at
http://www.research.ibm.com/journal/rd/435/jackson.html
which covers IBM's choice of L1 and L2 design for S/390. The section on "S/390 reliability and performance implications" is relevant here. In particular, they use a solution which isn't best from the performance point of view but ensures you don't discover "too late" about an error. I heard a rumour (now I get to the unsubstantiated part :-) that Sun chose a higher-performing design for their cache subsystem but which has a nastier failure mode in the case of cache errors.
--Malcolm
Signature · 3 lines
Malcolm Beattie <[email protected]> Unix Systems Programmer Oxford University Computing Services
Quoted 8 lines · Dan Hollis
Quoted 4 lines · Larry McVoy
which is a text version of the paper I mentioned before. The basic message of the paper is that it really doesn't help much to have things like ECC unless you can be sure that 100% of the rest of your system has similar checks.
UDMA has crc, scsi has parity, pci has (i think) parity, tcpip has crc, your cpu l1 and l2 have ecc...
SCSI Ultra-160 has CRC.
PCI has parity (btw, you think right), but only a few drivers make sure PCI parity checking is enabled. On the other hand, a PCI parity error should be considered as extremally serious and the system should be stopped when such happens.
Btw, it seems (read at the pci list) that the original PCI hadn't parity. After all, PCI had been designed for PC machines... :)
Quoted 4 lines
Looks like similar checks are already there.
-Dan
Gérard.
Quoted 10 lines · On Wed, 9 May 2001, [ISO-8859-1] G�rard Roudier wrote:
Quoted 6 lines · Dan Hollis
Quoted 4 lines · Larry McVoy
which is a text version of the paper I mentioned before. The basic message of the paper is that it really doesn't help much to have things like ECC unless you can be sure that 100% of the rest of your system has similar checks.
Quoted 7 lines
Quoted 2 lines
UDMA has crc, scsi has parity, pci has (i think) parity, tcpip has crc, your cpu l1 and l2 have ecc...
SCSI Ultra-160 has CRC.
PCI has parity (btw, you think right), but only a few drivers make sure PCI parity checking is enabled. On the other hand, a PCI parity error
Sun's panic if they get SERR or PERR.
Quoted 18 lines
should be considered as extremally serious and the system should be stopped when such happens.
Btw, it seems (read at the pci list) that the original PCI hadn't parity. After all, PCI had been designed for PC machines... :)
Quoted 4 lines
Looks like similar checks are already there.
-Dan
G�rard.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 4 lines · Larry McVoy
Let's review: ECC is nice, but it doesn't solve all data corruption problems. Applications which do their own end to end data integrity checks will catch many more error cases than what ECC catches.
I think you have a wrong idea why the ECC is there. ECC deals with the inherit shortcommings of DRAM.
DRAMs are not perfect. They have a probability to lose a bit. Normally this probability is low enough to live with it. Lets say you have a system with 1MByte and let's say the probability for a single bit error is around 1 error in 100 years. Good enough. Now put 1GByte in the system. You'll get a probability of 10 errors per year. Maybe good enough for a Windows box but not acceptable for your server. So you put in ECC to bring this probability back into reasonable numbers. ECC can correct the single bit errors. You only have to deal with double bit errors. Chance for them is much much lower.
Sure, it doesn't solve all data corruption problems - only simple errors in DRAMs. But it makes systems with huge amount of RAM staying up alive much longer. And btw, your integrity checks over data will not protect against a corrupted kernel or application...
Ciao, ET.
PS: Just let your app run long enough. I'm sure it will detect a checksum error some day ;-)
Followup to: <[email protected]> By author: Edgar Toernig <[email protected]> In newsgroup: linux.dev.kernel
Quoted 4 lines
I think you have a wrong idea why the ECC is there. ECC deals with the inherit shortcommings of DRAM.
Quoted 11 lines
DRAMs are not perfect. They have a probability to lose a bit. Normally this probability is low enough to live with it. Lets say you have a system with 1MByte and let's say the probability for a single bit error is around 1 error in 100 years. Good enough. Now put 1GByte in the system. You'll get a probability of 10 errors per year. Maybe good enough for a Windows box but not acceptable for your server. So you put in ECC to bring this probability back into reasonable numbers. ECC can correct the single bit errors. You only have to deal with double bit errors. Chance for them is much much lower.
Yes, ECC, unlike parity, is a technique for reducing the error rate, with the side benefit of intercepting an error when it happens.
I am not disagreeing with Larry that integrity checks are a Good Thing[TM], and in general are a hallmark of good engineering. However, they are not a replacement for ECC for the purpose of driving the failure rate down into an acceptable probability range.
It is of course a very nice thing that DRAM prices have come down into the range where buying them in gigabyte quantities are reasonable :)
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt
Re: Linux/Pro [was Re: Coding style - a non-issue]
In reply to a message not in the archiveHorst von Brand <[email protected]> writes:
Hi,
Quoted 5 lines
Quoted 3 lines
Nor will you find any match with 4 or 8 CPU systems, except in very rare cases. Yet changes go into the system for 8 way and 16 way performance. That's a mistake.
And you are proposing a fork for handling exactly this? I don't get it...
[Ah, and I've sworn that I won't participate in this thread again...]
99.99% of your userbase have one or two processors. 99.999% of your userbase have less than 8 processors.
It simply doesn't make sense to design something for 128+ Processors if noone uses it. Especially if it puts a penalty on the user base with one or two processors. Then such a design is actively hurting your user base. And this is nonsense. That's why the design goes 1->2->4->8. We got SMP once a core developer got a sponsored SMP board. Not because Linus designed his OS with "I want to scale to lots of processors I can only dream of". Along that road lies madness (and maybe GNU Hurd ;-) ).
I actually have five (at a customers site) eight-way machines. Unfortunately now they all run Windows 2000 (Datacenter Edition (TM)). But if the "eight-way" support would hurt the "two way" machines, I'd advocate to put that code at least under a compile switch.
Actually I do like the idea in Larrys' paper. It doesn't hurt beyond eight way, because the next eight processors are their own "system". Just because it doesn't hurt the smaller boxes (much).
Regards
Henning
Signature · 5 lines
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- Geschaeftsfuehrer INTERMETA - Gesellschaft fuer Mehrwertdienste mbH [email protected] Am Schwabachgrund 22 Fon.: 09131 / 50654-0 [email protected] D-91054 Buckenhof Fax.: 09131 / 50654-20
Over 4-way systems considered harmful :-)
I'm going to weigh in here in favor of limiting effort on SMP development by the core Linux team to systems with 4 processors and under. And not just because I'd like to see those developers freed up to work on my M-Audio Delta 66 :-). The economics of massively parallel MIMD machines just aren't there. Sure, the military guys would *love* to have a petaflop engine, but they're gonna build 'em anyway and quite probably not bother to contribute their kernel source on this mailing list. *Commercial* applications for supercomputers of this level are few and far between. I'm happy with my GFlop-level UP Athlon Thunderbird. And if Moore's Law (or the AMD equivalent :-) still holds, in 12 months I'll have something twice as fast (I've had it for six months already :-).
--
Take Your Trading to the Next Level!
M. Edward Borasky, Meta-Trading Coach
[email protected]
http://www.meta-trading-coach.com
http://groups.yahoo.com/group/meta-trading-coach
Re: Over 4-way systems considered harmful :-)
Quoted 11 lines
I'm going to weigh in here in favor of limiting effort on SMP development by the core Linux team to systems with 4 processors and under. And not just because I'd like to see those developers freed up to work on my M-Audio Delta 66 :-). The economics of massively parallel MIMD machines just aren't there. Sure, the military guys would *love* to have a petaflop engine, but they're gonna build 'em anyway and quite probably not bother to contribute their kernel source on this mailing list. *Commercial* applications for supercomputers of this level are few and far between. I'm happy with my GFlop-level UP Athlon Thunderbird. And if Moore's Law (or the AMD equivalent :-) still holds, in 12 months I'll have something twice as fast (I've had it for six months already :-).
Two things.
1) If a company (say, IBM) pays people to work on 8 / 16 way scalability because that's what they want out of Linux, then stopping development on that isn't going to get effort redirected to fixing your soundcard (yes, I realise you were being flippant, but the point's the same), the headcount is just going to disappear. AKA your choice isn't "patches for 8 way scalablilty, or patches for subsystem X that you're more interested in", your choice is "patches for 8-way scalabity, or no patches". Provided that those patches don't break anything else, you still win overall by getting them.
2) Working on scalability for 8 / 16 way machines will show up races, performance problems et al that exist on 2 / 4 way machines but don't show up as often, or as obviously. I have a 16 way box that shows up races in the Linux kernel that might take you years to find on a 2 way.
What I'm trying to say is that you still win. Not as much as maybe you'd like, but, hey, it's work you're getting for free, so don't complain too much about it. The maintainers are very good at beating the message into us that we can't make small systems any worse performing whilst making the big systems better.
Martin.
Re: Over 4-way systems considered harmful :-)
In reply to M. Edward Boraskyfolx,
although i agree with the sentiment expressed below, i beg to differ...
Quoted 4 lines · M. Edward Borasky
Sure, the military guys would *love* to have a petaflop engine, but they're gonna build 'em anyway and quite probably not bother to contribute their kernel source on this mailing list. *Commercial* applications for supercomputers of this level are few and far between.
the CPlant (http://www.cs.sandia.gov/cplant/) system software (currently built on top of Linux 2.2 but it's working now on 2.4 as well) is Open Source (GPL). CPlant is built by Sandia National Labs (which could be interpreted as "the military guys" is is currently one of the largest Linux-based supercomputers in the world. The source for it is publicly available and gives some interesting insight to what happens when you try to scale beyond the "traditional" 32 or 64 processor cluster.
Additionally, the same systems software is being used in at least one big commercial application (for processing genetic information).
my point here is not that building huge SMP machine makes sense (obviously, for reasons that have been repeatedly reexplained by others including Larry McVoy, it doesn't). my point is just that many parts of the national security aparatus are playing well in the Linux kernel development space these days and it's foolish to write them off.
todd underwood, vp & cto
oso grande technologies, inc.
[email protected]
"Those who give up essential liberties for temporary safety deserve neither liberty nor safety." - Benjamin Franklin
RE: Over 4-way systems considered harmful :-)
Quoted 29 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Todd Underwood
Sent: Tuesday, December 04, 2001 12:49 PM
To: M. Edward Borasky
Cc: [email protected]
Subject: Re: Over 4-way systems considered harmful :-)folx,
although i agree with the sentiment expressed below, i beg to differ...
the CPlant (http://www.cs.sandia.gov/cplant/) system software (currently built on top of Linux 2.2 but it's working now on 2.4 as well) is Open Source (GPL). CPlant is built by Sandia National Labs (which could be interpreted as "the military guys" is is currently one of the largest Linux-based supercomputers in the world. The source for it is publicly available and gives some interesting insight to what happens when you try to scale beyond the "traditional" 32 or 64 processor cluster.
Additionally, the same systems software is being used in at least one big commercial application (for processing genetic information).
my point here is not that building huge SMP machine makes sense (obviously, for reasons that have been repeatedly reexplained by others including Larry McVoy, it doesn't). my point is just that many parts of the national security aparatus are playing well in the Linux kernel development space these days and it's foolish to write them off.
Quoted 6 lines
todd underwood, vp & cto
oso grande technologies, inc.
[email protected]"Those who give up essential liberties for temporary safety deserve neither liberty nor safety." - Benjamin Franklin
Yes ... but ... it is interesting how quickly the priorities and policies of the National Laboratories can be "re-focused" with the change of the person residing at 1600 Pennsylvania Avenue and "other factors" :-). Been there ... done that ... got the scars :-). In short: "Moore's Law: good. Amdahl's Law:
bad" :-)
--
Take Your Trading to the Next Level!
M. Edward Borasky, Meta-Trading Coach
[email protected]
http://www.meta-trading-coach.com
http://groups.yahoo.com/group/meta-trading-coach
RE: Over 4-way systems considered harmful :-)
In reply to Martin J. BlighQuoted 9 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Martin J. Bligh
Sent: Tuesday, December 04, 2001 9:41 AM
To: M. Edward Borasky; [email protected]
Subject: Re: Over 4-way systems considered harmful :-)Two things.
Quoted 11 lines
1) If a company (say, IBM) pays people to work on 8 / 16 way scalability because that's what they want out of Linux, then stopping development on that isn't going to get effort redirected to fixing your soundcard (yes,
I realise you were being flippant, but the point's the same), the headcount
is just going to disappear. AKA your choice isn't "patches for 8 way scalablilty, or patches for subsystem X that you're more interested in", your choice is "patches for 8-way scalabity, or no patches". Provided that those patches don't break anything else, you still win overall by getting them.
I don't see how this is a win for me. And it is a win for IBM only if it gives them some advantage in serving their customers. I can certainly *conceive* of workloads bursty enough to justify an 8-processor server, but do they exist in the real world? And if they do, is a single 8-processor server better than a pair of 4-processor servers when you take graceful handling of faults into account? IBM has been building high-availability systems for *decades*, preferring to field *slightly* slower but *significantly* more reliable gear, which, legend has it, no one has ever been fired for purchasing. :-)
Quoted 4 lines
2) Working on scalability for 8 / 16 way machines will show up races, performance problems et al that exist on 2 / 4 way machines but don't show up as often, or as obviously. I have a 16 way box that shows up races in the Linux kernel that might take you years to find on a 2 way.
Perhaps effort should be placed into software development processes and tools that deny race conditions the right to be born, rather than depending on testing on a 16-processor system to find them expeditiously :-). And there is a whole discipline of software performance engineering to build performance in from the start. Advances like that would be a *huge* win for the Linux community, given our (relative) freedom from corporate-world limitations like deadlines, sales quotas, programmer salaries, and full-color brochures.
Quoted 5 lines
What I'm trying to say is that you still win. Not as much as maybe you'd like, but, hey, it's work you're getting for free, so don't complain too much about it. The maintainers are very good at beating the message into us that we can't make small systems any worse performing whilst making the big systems better.
No, but we can release partly-baked VM schemes that have a shelf life on the order of days :-). Seriously, though, I don't like stepping on someone else's dream, especially since *my* dream -- a GFLOP dedicated to computer music -- has been fulfilled for about $1500 US. When I bought that machine, I thought I was going to need two processors -- one to run the OS and service the keyboard, mouse and monitor and a second dedicated to generating audio samples in real time. I had no idea how powerful these chips were until I started shopping around. And when I loaded the Atlas linear algebra library up on my Athlon and saw the speeds it was getting, I was in shock for almost a week!
However, as I said in another post: "Moore's Law: good. Amdahl's Law: bad." I guess the new generation has to discover Amdahl's Law for itself, and *this* distinguished but elderly scientist is eager to be proven wrong :-). --
Take Your Trading to the Next Level!
M. Edward Borasky, Meta-Trading Coach
[email protected]
http://www.meta-trading-coach.com
http://groups.yahoo.com/group/meta-trading-coach
RE: Over 4-way systems considered harmful :-)
Quoted 9 lines
I don't see how this is a win for me. And it is a win for IBM only if it gives them some advantage in serving their customers. I can certainly *conceive* of workloads bursty enough to justify an 8-processor server, but do they exist in the real world? And if they do, is a single 8-processor server better than a pair of 4-processor servers when you take graceful handling of faults into account? IBM has been building high-availability systems for *decades*, preferring to field *slightly* slower but *significantly* more reliable gear, which, legend has it, no one has ever been fired for purchasing. :-)
We (Sequent, now bought by IBM) sold 64 processor servers to people who needed them (the main market was large databases) so, yes, there is definitely a market for larger systems. It's cheaper to build a big processor farm out of a Beowulf style cluster, but it's not always easy / possible to split up your application over something like that. If you're generating fractals, it's easy, for other applications, it's not.
Quoted 3 lines
Perhaps effort should be placed into software development processes and tools that deny race conditions the right to be born, rather than depending on testing on a 16-processor system to find them expeditiously :-). And
I wish you good fortune, sir. When you've acheived that, come back and tell me, and we'll stop testing stuff ;-) It's always better to not code bugs in the first place, but that's not the world we live in ...
Quoted 5 lines
there is a whole discipline of software performance engineering to build performance in from the start. Advances like that would be a *huge* win for the Linux community, given our (relative) freedom from corporate-world limitations like deadlines, sales quotas, programmer salaries, and full-color brochures.
I'm all for encouraging good programming practices. In a way that's actually harder in the diverse Linux community with hundreds, nay, thousands of engineers putting code in. But we should still try - keeping the infrastructure simple helps, for example. Trying to bug fix badly designed / written code is pissing into the wind.
M.
Re: Over 4-way systems considered harmful :-)
In reply to M. Edward BoraskyHi!
Quoted 28 lines
Quoted 11 lines
I'm going to weigh in here in favor of limiting effort on SMP development by the core Linux team to systems with 4 processors and under. And not just because I'd like to see those developers freed up to work on my M-Audio Delta 66 :-). The economics of massively parallel MIMD machines just aren't there. Sure, the military guys would *love* to have a petaflop engine, but they're gonna build 'em anyway and quite probably not bother to contribute their kernel source on this mailing list. *Commercial* applications for supercomputers of this level are few and far between. I'm happy with my GFlop-level UP Athlon Thunderbird. And if Moore's Law (or the AMD equivalent :-) still holds, in 12 months I'll have something twice as fast (I've had it for six months already :-).
Two things.
1) If a company (say, IBM) pays people to work on 8 / 16 way scalability because that's what they want out of Linux, then stopping development on that isn't going to get effort redirected to fixing your soundcard (yes, I realise you were being flippant, but the point's the same), the headcount is just going to disappear. AKA your choice isn't "patches for 8 way scalablilty, or patches for subsystem X that you're more interested in", your choice is "patches for 8-way scalabity, or no patches". Provided that those patches don't break anything else, you still win overall by getting them.
2) Working on scalability for 8 / 16 way machines will show up races, performance problems et al that exist on 2 / 4 way machines but don't show up as often, or as obviously. I have a 16 way box that shows up races in the Linux kernel that might take you years to find on a 2 way.
Quoted 5 lines
What I'm trying to say is that you still win. Not as much as maybe you'd like, but, hey, it's work you're getting for free, so don't complain too much about it. The maintainers are very good at beating the message into us that we can't make small systems any worse performing whilst making the big systems better.
Making it perform better, while not hurting up, and *not making code messier* is good thing. Messiness of code might be as importnat as performance, or even more important.
Pavel
Signature · 2 lines
"I do not steal MS software. It is not worth it."
-- Pavel KankovskyRe: Over 4-way systems considered harmful :-)
In reply to Martin J. BlighQuoted 1 line · M. Edward Borasky
I don't see how this is a win for me. And it is a win for IBM only if it
I think what you don't seem to realize is that most developers don't develop because they think it's good for you, or anyone else for that matter. Every developer has his own motivation, but I doubt that there are a great majority of them that do it just to please others.
We don't do it for the "greater good", we do it because "it fixes this bug that keeps hard locking my kernel whenever I play track 2 of the new cd that came out", or "I guess nobody has written a driver for my new seti@home-on-pci card processing board, looks like if I want it to work, I'll have to write one".
I'm not trying to sham the greater good mentality, but I'm just trying to tell you that most developers are primarily in it for themselves.
Quoted 3 lines
Perhaps effort should be placed into software development processes and tools that deny race conditions the right to be born, rather than depending on testing on a 16-processor system to find them expeditiously :-). And
You have gcc, no? write some tools. I like doing driver development, I'll keep doing that until I need a tool to help me out with something, and then I'll write one. Maybe I'll get lucky and someone else has already written what I need.
Matthew Fredrickson
Re: A modest proposal -- We need a patch penguin
In reply to a message not in the archiveQuoted 13 lines · Larry McVoy
Quoted 11 lines · Keith Owens
That sounds almost like what I was looking for, with two differences.
(1) Implement the collapsed set so bk records that it is equivalent to the individual patchsets. Only record that information in my tree. I need the detailed history of what changes went into the collapsed set, nobody else does.
(2) Somebody else creates a change against the collapsed set and I pull that change. bk notices that the change is again a collapsed set for which I have local detail. The external change becomes a branch off the last detailed patch in the collapsed set.
Quoted 5 lines
This is certainly possible to do. However, unless you are willing to fund this development, we aren't going to do it. We will pick up the costs of making changes that you want if and only if we have commercial customers who want (or are likely to want) the same thing. Nothing personal, it's a business and we make tradeoffs like that all the time.
I wonder how your commercial customers develop code then. Either each programmer futzes around in his/her own tree, and then creates a patch (or some such) for everybody to see (then I don't see the point of source control as a help to the individual developer), or everybody sees all the backtracking going on everywhere (in which case the repository is a mostly useless mess AFAICS).
Signature · 1 line
Horst von Brand http://counter.li.org # 22616
Re: A modest proposal -- We need a patch penguin
Quoted 6 lines · Horst von Brand
I wonder how your commercial customers develop code then. Either each programmer futzes around in his/her own tree, and then creates a patch (or some such) for everybody to see (then I don't see the point of source control as a help to the individual developer), or everybody sees all the backtracking going on everywhere (in which case the repository is a mostly useless mess AFAICS).
If the object is to minimise confusion by not showing back-tracked changes, why not simply allow the user to mark changesets with a "visibility":
1) hidden, for stuff which shouldn't be seen by default, like backed out changes, etc..
2) small, individual development steps to achieve a new feature
3) normal, the normal commits
4) major (tagged versions ?)
This way the user can select how detailed the overview of the versions should be.
Also, when viewing a changeset/version of a certain priority, bitkeeper could optionally "fold in" the hidden changesets between the last changeset and the one the user wants to view.
Would this be a workable scheme ?
(keeps the bitkeeper repository intact, can reduce the confusion)
regards,
Rik
Signature · 4 lines
"Linux holds advantages over the single-vendor commercial OS"
-- Microsoft's "Competing with Linux" document
http://www.surriel.com/ http://distro.conectiva.com/2.4.16 cannot connect to www.sun.com
Hello all,
For some reason after I upgraded to 2.4.16, I cannot connect to
www.sun.com anymore. It also happens on some other sites. Anyone know what might be the problem? I have no problem using 2.4.7.
TIA.
- Joe
Re: 2.4.16 cannot connect to www.sun.com
Try echo 0 > /proc/sys/net/ipv4/tcp_ecn
I dont know, but ecn can prevent you from reaching some locations on the net.. could be that 2.4.17 turns it on by default.
-- Chris
Quoted 16 lines · Joe Wong
Hello all,
For some reason after I upgraded to 2.4.16, I cannot connect to
www.sun.com anymore. It also happens on some other sites. Anyone know what might be the problem? I have no problem using 2.4.7.TIA.
- Joe
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.16 cannot connect to www.sun.com
In reply to Joe WongQuoted 4 lines · [email protected] said:
For some reason after I upgraded to 2.4.16, I cannot connect to
www.sun.com anymore. It also happens on some other sites. Anyone know what might be the problem? I have no problem using 2.4.7.
http://www.tux.org/lkml/#s14-2
--
dwmw2
Re: A modest proposal -- We need a patch penguin
In reply to Horst von BrandQuoted 6 lines · Horst von Brand
Quoted 5 lines
This is certainly possible to do. However, unless you are willing to fund this development, we aren't going to do it. We will pick up the costs of making changes that you want if and only if we have commercial customers who want (or are likely to want) the same thing. Nothing personal, it's a business and we make tradeoffs like that all the time.
Quoted 6 lines
I wonder how your commercial customers develop code then. Either each programmer futzes around in his/her own tree, and then creates a patch (or some such) for everybody to see (then I don't see the point of source control as a help to the individual developer), or everybody sees all the backtracking going on everywhere (in which case the repository is a mostly useless mess AFAICS).
You are presupposing that all the developers are checking in many bad changes and only one good change. And that all the bad changes are obscuring the good ones. That a correct statement of your beliefs?
If so, what you are describing is called "hacking" in the negative sense of the word, and what my customers do is called "programming". It's quite rare to see the sort of mess that you described, it happens, but it is rare. I don'tknow how else to explain it, but it is not the norm in the professional world to try a zillion different approaches and revision control each and every one.
The norm is:
clone a repository
edit the files
modify/compile/debug until it works
check in
push the patch up the shared repository
I'm really at a loss as to why that shouldn't be the norm here as well.
Signature · 2 lines
--- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Re: A modest proposal -- We need a patch penguin
In reply to Horst von BrandQuoted 4 lines · Rik van Riel
On Fri, 1 Feb 2002, Horst von Brand wrote:
If the object is to minimise confusion by not showing back-tracked changes, why not simply allow the user to mark changesets with a "visibility":
That's what LODs do. You can do all your work in your "branch", when you are ready, you do a branch-to-branch pull which collapses the view of all your changesets down to one in the other view.
I'd love it if you could get Linus to buy into this as an acceptable answer.
I do agree that there are times when you really want to collapse a pile of changes into one and I'm willing to write that code if it becomes agreed that it is widely useful. It's maintaining both versions of the changes, the collapsed and uncollapsed, that I don't want to do. That would be a nightmare in the source base and I don't believe there is substantial real benefit. Either the changes are valuable or they aren't. If they are valuable enough that you want to save them then you should let the rest of the world see them. If they aren't, then they aren't. I'm sure you can find cases that don't match that view but I'm equally sure they are a very small percentage.
Signature · 2 lines
--- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Re: A modest proposal -- We need a patch penguin
In reply to Larry McVoyFrom: Larry McVoy <[email protected]>
Quoted 7 lines
If so, what you are describing is called "hacking" in the negative sense of the word, and what my customers do is called "programming". It's quite rare to see the sort of mess that you described, it happens, but it is rare. I don'tknow how else to explain it, but it is not the norm in the professional world to try a zillion different approaches and revision control each and every one.
Quoted 6 lines
The norm is:
clone a repository
edit the files
modify/compile/debug until it works
check in
push the patch up the shared repository
Or they create do a line of development in a repository with commits and then determine that it wasn't working. No problem throw it away and start over from a clean copy. Since the repositories are distributed, private branches just disappear if you don't like them.
-Wayne
Re: A modest proposal -- We need a patch penguin
In reply to Larry McVoyQuoted 4 lines · Larry McVoy
I do agree that there are times when you really want to collapse a pile of changes into one and I'm willing to write that code if it becomes agreed that it is widely useful. It's maintaining both versions of the changes, the collapsed and uncollapsed, that I don't want to do.
Hand waving solutions without seeing the bk code ...
Don't maintain both versions of the changes.
Everybody except the person with the uncollapsed set only sees the collapsed set, no problem there.
The person with the uncollapsed set only has one version of the changes, in the uncollapsed set.
Identify each externally visible patchset with a unique id, you probably already do this.
Define a meta patchset entry which maps the uncollapsed line of patches to the collapsed set. That is, don't duplicate the changes, add a mapping instead.
The meta entry maps the externally visible patchset to the internal set by listing just the start and end point of the LOD. These are the entries that would be given to export patch.
Add a global repository option, show/hide detail. The default is show detail, the current behaviour.
For show detail, you cannot use meta patchset entries. Everything is visible to the rest of the world.
For hide detail, you must define meta entries for what you want to be visible, the rest of the world can only see what you expose. That is, you cannot publish some detail entries and some meta entries, you must choose one or the other at the repository level.
Meta sets are single level, no collapse within collapse. A meta set is externally visible, to collapse a meta set into a meta-meta set is equivalent to rewriting the distributed bk history, don't allow that.
No duplication of changes. No rewriting of bk history. Users who want to hide their detail changes and only expose the result can do so. Users who only check in working (as opposed to hacking) code are not affected. Users who want the extra flexibility incur the cost of defining meta sets before they can publish anything.
Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to Horst von BrandQuoted 20 lines · Horst von Brand
Quoted 13 lines · Larry McVoy
Quoted 11 lines · Keith Owens
That sounds almost like what I was looking for, with two differences.
(1) Implement the collapsed set so bk records that it is equivalent to the individual patchsets. Only record that information in my tree. I need the detailed history of what changes went into the collapsed set, nobody else does.
(2) Somebody else creates a change against the collapsed set and I pull that change. bk notices that the change is again a collapsed set for which I have local detail. The external change becomes a branch off the last detailed patch in the collapsed set.
Quoted 5 lines
This is certainly possible to do. However, unless you are willing to fund this development, we aren't going to do it. We will pick up the costs of making changes that you want if and only if we have commercial customers who want (or are likely to want) the same thing. Nothing personal, it's a business and we make tradeoffs like that all the time.
Quoted 6 lines
I wonder how your commercial customers develop code then. Either each programmer futzes around in his/her own tree, and then creates a patch (or some such) for everybody to see (then I don't see the point of source control as a help to the individual developer), or everybody sees all the backtracking going on everywhere (in which case the repository is a mostly useless mess AFAICS).
Speaking from my experience on OS/2 back at IBM (I.E. about as big as it gets), they simply don't mind having amazing amounts of cruft in the database dating back to at least the last time they switched source control systems.
Under those circumstances, looking at the code history becomes a major archaeological expedition, and you either still have the original implementors around who have it all in their heads and don't NEED the database, or you have people who just look at the code and try to guess what it means, possibly asking colleagues about particularly confusing bits.
IBM's source control system always struck me as a graveyard of old code more than anything else. Lots of the really old stuff was stored offline on tapes and CDs filed lockers nobody ever opened, with backup copies at some big "we have a vault beneath NORAD" data warehousing company.
That's my impression, anyway. (A few years out of date now.) My experience with the source control system was that it DID have the complete revision history in it going back to the 1980's, but it was far more work than it was worth to try to mine through it to find something unless you were specifically looking to place blame and prove something wasn't YOUR fault. Nobody really ever had time to actually go through it for any other reason, we had far too much new stuff backlogged. (And yeah a lot of changes were made where some old timer would pipe up afterwards "we tried that five years ago, and it didn't work for the same reason you just noticed". But this was the kind of state that was usefully kept in people's heads, not in the source control system.)
Now in an open source, the source control system with history might be a useful educational resource. (Non-commercial developers don't have to hit the ground running the way commercial ones do.) But too much granularity would definitely diminish that usefulness. Flood people with low signal and high noise, and only archaeologists will ever care about it.
A system that maintained obscene amounts of granularity but HID it (showing you only diffs between release versions unless you specifically asked so see more detail) would be a distinct improvement over a system that forces every query be an archaeological dig. But since 99% of the people do NOT want to go on an archaeological dig, and since for most things only the most active 10% of the developers even care about the -pre releases and only have time to track/sync with the main releases...
Maintaining the "reverted before it left the original implementors tree" state at ALL is clearly more trouble than it's worth. If the original IMPLEMENTOR is likely to delete that after they ship, nobody else should EVER care unless they're writing some sort of biography of that developer or simply engaged in hero-worship.
I.E. yes, I think we honestly do want an easy way to limit the granularity of propogated diffs. We can do this right now by exporting to patch and re-importing, but it seems that if we do, then bitkeeper's sync mechanism becomes a problem to be worked around. I'd say this instance of all-out-war between what developers are trying to do and what bitkeeper is trying to do highlights a design gap in bitkeeper.
Just my opinion, your mileage may vary. :)
Rob
Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to Horst von BrandQuoted 6 lines · Larry McVoy
You are presupposing that all the developers are checking in many bad changes and only one good change. And that all the bad changes are obscuring the good ones. That a correct statement of your beliefs?
If so, what you are describing is called "hacking" in the negative sense of the word, and what my customers do is called "programming".
Designing while coding is not a bad thing. It's often considerably more efficient than spending a bunch of time up front coming up with an ivory tower design that doesn't work in practice. Very few battle plans survive the first engagement with the enemy. (It's nice to HAVE one. But if you can't adapt when you're in the thick of things, you're in trouble.)
Quoted 2 lines
It's quite rare to see the sort of mess that you described, it happens, but it is rare.
In your user base, sure. Because your tool can't handle it.
Please don't confuse effect with cause...
Quoted 3 lines
I don'tknow how else to explain it, but it is not the norm in the professional world to try a zillion different approaches and revision control each and every one.
The distinction here is "should bitkeeper be of use to the company", or "should bitkeeper be of use to the individual developer". Bitkeeper is aimed at only serving ONE of those two niches right now (as I'll explain in a moment), and one thing to realise is that the development teams at companies are made up of individual developers who might find bitkeeper significantly more useful if it met more of their needs.
Quoted 7 lines
The norm is:
clone a repository
edit the files
modify/compile/debug until it works
check in
push the patch up the shared repository
I'm really at a loss as to why that shouldn't be the norm here as well.
You'll notice that bitkeeper is totally useless between the clone and the check in, in your model. It can't really be used DURING development, only before and after.
You may not notice this so much in the Fortune 500. I've worked for some really big companies. Your commercial developers are often implementing to a spec that was argued over in meetings, with donuts, for weeks before an "implementor" was allowed near a keyboard. This IS a different development model from the way open source generally works. :) (And the bigger the company, the more likely they are to ensure their design is perfected before they ever actually try to implement a prototype and find out the whole theory doesn't really work in practice. And you wonder how anybody could ever manage to spend a billion dollars a year writing software and still ship crap. But that's a seperate argument. Neither method is inherently superior, and most real world development is a blance between up-front design and implementation evolution. Go too far in either direction and you get chaos or stagnation.)
As for a simple example of when your model above breaks down, a lot of developers who use things like emacs have their source control system as part of their integrated development environment. When they "save and compile", it's checked into the RCS (often with a terse three or four word comment that's intended as a label to jog the developer's memory). For one thing, this is a nice backup against stupid things like fat-fingering "gcc blah.c -o blah.c", which I personally did first thing in the morning (before the caffiene could kick in) a few days ago and set myself back just about a whole day because my laptop does NOT have any variant of source control on it. (Screaming is certainly a quick and aerobic way to wake up, I will admit that. I wouldn't recommend it though.)
The real reason people go with full-blown source control for their backups is so that when they ARE doing modify/compile/debug, they can easily back up to what they had an hour ago if they find out they went down a culdesac (as happens a lot: not always "I couldn't make this work if I kept going" but "wait, there's an easier way to do this"). Of course not all changes in direction are clean reverts. Sometimes you want to save part of your work and change the other parts yourself, and it's easier to fix up manually and check the fixup in than reimplement just to look like you knew what you were doing all along. And sometimes if it's a bug fix inserting a bunch of printfs and test code while actually fixing a multi-stage bug, they don't bother rolling back to remove the printfs. Once they've fixed the bug, it's easier to delete the instrumentation and false tries rather than roll back to a clean version and recreate the entire (possibly long and complicated) fix against the fresh tree.)
The problem is, if they use bitkeeper (with a temporary respository), all these temporary commits (debugging tries saved in case they want to roll back during development) get propagated into the main repository when they do a merge. They can't tell it "done, okay, squash this into one atomic change to check in somewhere else, with the whole change history as maybe one comment".
What you're saying is that people who use the source control tools in a way you do not expect them to are bad programmers. A stylistic difference does not equal stupidity.
People sometimes really do want to be able to squash between checkpoints, which might just be a case of squashing a scratch repository used to create the checkpoint. (If the scratch repository is deleted in the process, then the "merge granular and squashed versions" thing doesn't really arise, so possibly people just want to be able to do their development in a temporary repository which they squash and delete when they issue a release. Not a 100% solution, it means the original developer can't keep the original highly granular version around without potentially confusing bitkeeper, but it's probably an acceptable solution in 80-90% of cases, and is probably doable with a script rather than any real change to bitkeeper.)
You do not seem to be used to implementors doing any real designing, and you do not seem to be used to your source control system being used during the throes of minute-by-minute development. (Once again, you seem to be arguing against feedback from the community, by saying "you don't really want to do that, your process is bad and should be adapted to my tools". I hope this is just another miscommunication. We're trying to say "your tools aren't any good at what we're trying to use them for, but they could be." If you don't want your tool to serve a certain group of users, fine. Your tools may serve an existing niche fairly well, but if you'd like to expand to a broader audience listening to feedback would be helpful.)
Perhaps some of this is just a documentation issue. If we can explain what we want but can't easily figure out how to get bitkeeper to do it, some kind of tutorial or prepackaged scripts might be a good idea...
Rob
Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to Rik van RielQuoted 19 lines · Rik van Riel
Quoted 6 lines · Horst von Brand
I wonder how your commercial customers develop code then. Either each programmer futzes around in his/her own tree, and then creates a patch (or some such) for everybody to see (then I don't see the point of source control as a help to the individual developer), or everybody sees all the backtracking going on everywhere (in which case the repository is a mostly useless mess AFAICS).
If the object is to minimise confusion by not showing back-tracked changes, why not simply allow the user to mark changesets with a "visibility":
1) hidden, for stuff which shouldn't be seen by default, like backed out changes, etc..
2) small, individual development steps to achieve a new feature
3) normal, the normal commits
4) major (tagged versions ?)
Quoted 2 lines
This way the user can select how detailed the overview of the versions should be.
A workaround, not a fix.
First of all, not everybody's got a 100 gigabyte drive, cable modem, and Athlon 900 yet. You're talking about potentially taking accidental cruft from everybody who uses bitkeeper in 5 years. Some countries still pay for data by the byte, and big servers like kernel.org still care about bandwidth issues a lot.
Yeah data storage and transfer, and the memory and CPU power to churn through it, is getting cheaper all the time. But we're talking about expending resources shoveling information that even the original developer considers completely pointless to maintain and propogate. Your bitkeeper repositories could become enormous. The amount of proagated state would at least by multiplied by the number of developers working on the tree who use bitkeeper, meaning spreading the use of bitkeeper would have distinct downsides. The result is that maintainers/lieutenants/linus would almost certainly want to take a clean patch rather than a bitkeeper cruftball, on the size and cleanliness issue alone.
Secondly, it makes Linus's code review job a LOT harder to have unnecessary data in his change sets. And of course you could say "Linus would never have to look at that info", but you'd be wrong. Stupid example: Somebody patches a file to include a copy of decss (or encryption code, or the copyrighted ramblings of the lawsuit-happy cult of scientology) and then adds another patch to revert it before making a small fix to the file. The bitkeeper change now includes legally questionable code in its back-history, a hot potato we probably REALLY don't want to be involved with.
You don't have to worry about malicious use to see a problem. For-profit intellectual property companies generally want to be really clear about what code they export. Think about proprietary drivers that get "cleaned up" before being released under the GPL, with the removal code licensed from third parties, or patented, or still-proprietary code. If they can't collapse changesets, they simply can't use bitkeeper change sets when communicating source code with anyone outside their organization. Ever. End of story. Who knows what code might slip through otherwise? They have to audit the entire revision history rather than just the patch they mean to send. That's a nightmare. The lawyers would never approve ANYTHING for release, except as a patch file.
So Linus wouldn't want it, for-profit companies wouldn't want to give it to Linus, developers on the end of a 56k modem wouldn't want it, and it could potentially cost sites kernel.org a lot of extra bandwidth and data storage charges. This is in exchange for a the possibility that some future archeologist or grad student might want to mine the history for anecdotes to put in the biography of some developer.
(I'm writing a history book, by the way. I know all about having 10x as much detail as you are ever going to use, because I -AM- the researcher. Go visit the charles babbage institute at the university of minnesota sometime. It's great, but it's not actually all that useful unless you really are looking for something specific...)
Another random anecdote on the subject...
http://groups.google.com/groups?selm=789312483snz%40unseen.demon.co.uk
Quoted 6 lines
Also, when viewing a changeset/version of a certain priority, bitkeeper could optionally "fold in" the hidden changesets between the last changeset and the one the user wants to view.
Would this be a workable scheme ?
Depends on your definition of "workable". It's certainly an improvement, but it's still not a fix for the fundamental design assumption in bitkeeper. When I do NOT want to propagate my internal state to the rest of the world, I seem to have no choice in the matter except by NOT using bitkeeper to transfer the data.
I agree that the ability to at least hide stuff would be an improvement. And it should be easily implementable in combination with a checkpoint/tagging mechanism (it's just a view issue), but it's still working around a fault in bitkeeper that it has no real infrastructure for collapsing together change sets in a way that intentionally loses information when that really is what you want to do. The most it could do is hide the problem.
Quoted 2 lines
(keeps the bitkeeper repository intact, can reduce the confusion)
Yeah. But the point is that people often honestly do not want to keep the bitkeeper repository intact.
Think about partitioning into different repositories, to do development in different branches. If people don't want to be able to partition their data, bitkeeper wouldn't have the concept of multiple repositories on the same machine, would it? You can partition data when you store it, but not partition it when you propagate it. (The concept here is "this info does not leave my tree". This is a concept people really do want.)
It's possible this could be handled by doing ALL new development work in a seperate repository, and then ONLY exporting patches from that repository and never doing bitkeeper repository merges. But it's still working around a design flaw in bitkeeper. The way bitkeeper is designed forces you to retain cruft you don't want, and actively tries to prevent you from being able to get rid of any of it. I.E. there are things you can do with "diff" and "patch", fairly easily, which are not possible to do without bypassing bitkeeper.
I can see the mindset that went into that, but I disagree with it.
Quoted 3 lines
regards,
Rik
Rob
(P.S. You know, If larry gave us the code to bitkeeper, I'm sure we could add the features we wanted.... :)
(P.P.S. Yeah, I know the story on that one, just tweaking his nose...)
Re: Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to a message not in the archiveQuoted 6 lines · Rob Landley
The problem is, if they use bitkeeper (with a temporary respository), all these temporary commits (debugging tries saved in case they want to roll back during development) get propagated into the main repository when they do a merge. They can't tell it "done, okay, squash this into one atomic change to check in somewhere else, with the whole change history as maybe one comment".
Something like:
bk start-temporary-fork
[hack hack hack]
bk commit
[hack hack hack]
bk revert
[hack hack hack]
bk commit
[hack hack hack]
bk commit
bk fold-back-into-main-tree-as-one-atomic-update
?
Charles
Signature · 4 lines
----------------------------------------------------------------------- Charles Cazabon <[email protected]> GPL'ed software available at: http://www.qcc.sk.ca/~charlesc/software/ -----------------------------------------------------------------------
Re: Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to Horst von BrandQuoted 11 lines · Rob Landley
Quoted 7 lines
The norm is:
clone a repository
edit the files
modify/compile/debug until it works
check in
push the patch up the shared repository
I'm really at a loss as to why that shouldn't be the norm here as well.You'll notice that bitkeeper is totally useless between the clone and the check in, in your model. It can't really be used DURING development, only before and after.
What nonsense. Go read the docs.
Quoted 5 lines
As for a simple example of when your model above breaks down, a lot of developers who use things like emacs have their source control system as part of their integrated development environment. When they "save and compile", it's checked into the RCS (often with a terse three or four word comment that's intended as a label to jog the developer's memory).
Hey genuis. Download BK, install it, get into emacs, and type the key strokes you just described. What happens? I'll let you in on a little secret, smart boy, it does exactly what it should do. Works just like it were RCS or SCCS. It was carefully designed behave exactly like SCCS so that emacs/make/patch/etc all just know how to use it. Try patching a BK repository w/ patch(1) and you'll see
retrieve file from revision system with lock?
Just works. Fits neatly with the tools that we all know and love.
So explain to me again how it is that the tool is "totally useless" when it works *exactly* the way you say it should?
Signature · 2 lines
--- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Re: Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to Horst von BrandQuoted 11 lines · Charles Cazabon
Quoted 6 lines · Rob Landley
The problem is, if they use bitkeeper (with a temporary respository), all these temporary commits (debugging tries saved in case they want to roll back during development) get propagated into the main repository when they do a merge. They can't tell it "done, okay, squash this into one atomic change to check in somewhere else, with the whole change history as maybe one comment".
Something like:
bk start-temporary-fork
bk clone main temporary-fork
Quoted 4 lines
[hack hack hack]
bk commit
[hack hack hack]
bk revert
bk fix -c
Quoted 5 lines
[hack hack hack]
bk commit
[hack hack hack]
bk commit
bk fold-back-into-main-tree-as-one-atomic-update
bk push
All exists, works as described, no changes necessary.
Signature · 2 lines
--- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Re: Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to Horst von BrandQuoted 4 lines · Rob Landley
That's my impression, anyway. (A few years out of date now.) My experience with the source control system was that it DID have the complete revision history in it going back to the 1980's, but it was far more work than it was worth to try to mine through it to find something ...
bk revtool
double click on the last rev, brings up an annotated listing, click on a line, brings up the checkin comments for that line, double click on a line, brings up a changeset viewer showing the entire patch which contained that line.
Now tell me, Rob, wouldn't you find that useful? I find it useful, I find it extremely useful, I can fix bugs a factor of 10 (at least) faster with it than without. Not having it is like not having ctags, how the hell do you find anything?
Quoted 1 line
specifically looking to place blame and prove something wasn't YOUR fault.
Only losers need to go look to place blame. Programmers fix bugs. This information dramatically improves your ability to fix bugs. If you programmed more and talked less, you might understand this concept.
Quoted 2 lines
I.E. yes, I think we honestly do want an easy way to limit the granularity of propogated diffs.
So suppose I'm joe developer and I make 10 changesets to add some feature. Somewhere in there I caused a problem. The changesets have been collapsed so I now have to wade through all of them to figure out what the problem is. If I had all of them, I could
bk clone while true do make test_for_bug || break bk undo -fr+ done
which quickly walks through all of them and finds me exactly the one that caused the problem.
The more I collapse, the harder this is. But wait, you say, if BK lety me do it both ways, the original developer has all this detail. Great, that means we took a system that could let the whole world figure out the problem and we made it so only one guy can do it fast. Not real smart.
I'm sympathetic to the "wait, I made this changeset and then I found another bug" problem and we have a command that lets you reedit a changeset. And we're talking with Linus about making that even more flexible. But the rules are you can only change history on things you haven't propogated.
If you want to help make BitKeeper better, it would be a lot more productive if you used it and figured out how it really works instead of critizing how you think it works. It's a lot closer to what you want than you think it is, and it is much closer than any other system in the world.
Signature · 2 lines
--- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Re: Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to Rob LandleyQuoted 2 lines · Rob Landley
(
P.S. You know, If larry gave us the code to bitkeeper, I'm sure we could add the features we wanted.... :)
Why would he need to "give you the source" when you can just download it from bitkeeper.com ?
Are you really this much of a lazy bum that you prefer badmouthing Larry over searching bitkeeper.com for five minutes and grabbing the bitkeeper source ? ;)
regards,
Rik
Signature · 4 lines
"Linux holds advantages over the single-vendor commercial OS"
-- Microsoft's "Competing with Linux" document
http://www.surriel.com/ http://distro.conectiva.com/Re: Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to a message not in the archiveQuoted 9 lines · Larry McVoy
bk clone main temporary-fork
Quoted 3 lines
[hack hack hack]
bk commit
[hack hack hack]bk fix -c
Quoted 8 lines
Quoted 4 lines
[hack hack hack]
bk commit
[hack hack hack]
bk commitbk push
All exists, works as described, no changes necessary.
But will the changes which were committed and then reverted from the temporary tree show up in the main tree after the "push"? There should be no evidence that they ever took place, so as not to clutter Linus' tree with changes a developer made and had no intention of sending to Linus.
Charles
Signature · 4 lines
----------------------------------------------------------------------- Charles Cazabon <[email protected]> GPL'ed software available at: http://www.qcc.sk.ca/~charlesc/software/ -----------------------------------------------------------------------
Re: Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to Rob LandleyQuoted 6 lines · Rob Landley
Quoted 4 lines · Larry McVoy
You are presupposing that all the developers are checking in many bad changes and only one good change. And that all the bad changes are obscuring the good ones. That a correct statement of your beliefs?
Quoted 8 lines
Quoted 2 lines
If so, what you are describing is called "hacking" in the negative sense of the word, and what my customers do is called "programming".
Designing while coding is not a bad thing. It's often considerably more efficient than spending a bunch of time up front coming up with an ivory tower design that doesn't work in practice. Very few battle plans survive the first engagement with the enemy. (It's nice to HAVE one. But if you can't adapt when you're in the thick of things, you're in trouble.)
You're confusing designing with "coming up with the most inflexible idea possible".
I believe this says more about your experience (or skills) with designing than about the practice of coming up with a design before you start implementing.
regards,
Rik
Signature · 4 lines
"Linux holds advantages over the single-vendor commercial OS"
-- Microsoft's "Competing with Linux" document
http://www.surriel.com/ http://distro.conectiva.com/Re: Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to Rik van RielQuoted 3 lines · Rik van Riel
Are you really this much of a lazy bum that you prefer badmouthing Larry over searching bitkeeper.com for five minutes and grabbing the bitkeeper source ? ;)
Actually, I got as far as the questionaire and mailback to get the registration required to download anything from bitmoover before putting it on my "to do" list, and went and looked at the online docs instead. (I move back to Austin soon, I'm kind of swamped...)
I honestly didn't know the source was available. I vaguely remembered that if bitkeeper were to go out of business the source code would become available (under the GPL) after a few months, which kind of implied that wasn't the case now.
As for bitkeeper's website, it doesn't mention source code on the main page, it doesn't mention any so on the downloads page or the questionaire following it, the "free software" link from the downloads page mentions lmbench, bitcluster, and webroff but -not- bitkeeper, and the reference manual doesn't have a mention of source code (if it does, I missed it) until you get to the licensing page (documention->reference manual->licensing) which has "the bitkeeper license" that says portions of it (two libraries and the installer) are available as GPL code, but the rest would only be released as GPL 180 days after bitkeeper goes out of businesss).
That actually took a little more than five minutes. They I went and did something else... :)
Looking at the license again, section 2B does seem to imply it's possible to get non-GPL source code from bitmover (although it doesn't say it's not purchased source code).
But looking looking at the license again, this section under "licensee obligations" is certainly interesting:
Quoted 4 lines
(a) Maintaining Open Logging Feature: You hereby warrant
that you will not take any action to disable or oth- erwise interfere with the Open Logging feature of the BitKeeper Software. You hereby warrant that you
Quoted 8 lines
will take any necessary actions to ensure that the BitKeeper Software successfully transmits the Meta- data to an Open Logging server within 72 hours of the creation of said Metadata. By transmitting the Metadata to an Open Logging server, You hereby grant BitMover, or any other operator of an Open Logging server, permission to republish the Metadata sent by the BitKeeper Software to the Open Logging server.
Meaning if I use my laptop offline for more than three days, I could be in violation of the bitkeeper license. And under section 4.2, you can pay to have that requirement removed.
Section 3C also mentions there's ANOTHER license included in the download (a sort of terms of use) which, if you don't comply with it, seems to imply your access to bitkeeper's public servers could be revoked... (How does this interact with the "bitkeeper must be able to log your activities" requirement? I don't think I want to know...)
I think I'll stick with diff/patch/rcs for now, thanks.
Quoted 3 lines
regards,
Rik
Rob
Re: Bitkeeper change granularity (was Re: A modest proposal -- We need a patch penguin)
In reply to Horst von BrandQuoted 1 line · Larry McVoy
Only losers need to go look to place blame. Programmers fix bugs. This
And this from the guy who loves to blame others for their bugs in public (... because we did so at SUN) and thought that it improves "team morale".
Isn't this a losers' behaviour?
Regards
Henning
Signature · 5 lines
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- Geschaeftsfuehrer INTERMETA - Gesellschaft fuer Mehrwertdienste mbH [email protected] Am Schwabachgrund 22 Fon.: 09131 / 50654-0 [email protected] D-91054 Buckenhof Fax.: 09131 / 50654-20