Linux 2.4 and BitKeeper

Hi,

I've started using BitKeeper to control Linux 2.4 source code.

My latest tree can be found at linux24.bkbits.net.

Can you plz tell me (us) what the bk clone command is?

I tried:

bk clone bk://linux24.bkbits.net//linux-2.4

and

bk clone bk://linux24.bkbits.net///linux-2.4

But it does not work.

Thanks,
Ben

Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

From: Ben Greear <[email protected]>
Date: Wed, 13 Mar 2002 23:33:27 -0700

I tried:

     bk clone bk://linux24.bkbits.net//linux-2.4

and

     bk clone bk://linux24.bkbits.net///linux-2.4

But it does not work.

Try a single /, ie:

     bk clone bk://linux24.bkbits.net/linux-2.4

I know this works because I've cloned a tree using that already :-)

Franks a lot,
David S. Miller
[email protected]

[marcelo@plucky tmp]$ bk clone bk://linux24.bkbits.net/linux-2.4/ SCCS/s.COPYING
...

Works just fine here.

Hi, Linus & Marcelo agreed that the right place for this is

	bk://linux.bkbits.net/linux-2.4

and I just put it there, let me know if that doesn't work.

Also, if you have a linux-2.5 BK tree, you can save yourself a lot of bandwidth by doing the following:

	bk clone -rv2.4.18-pre8 linux-2.5 linux-2.4
	cd linux-2.4
	bk parent bk://linux.bkbits.net/linux-2.4
	bk pull

That will get you back to the baseline you should already have and then just update your tree with what Marcelo added recently.

You don't have to do that, and for those of you with fast DSL lines you can skip, I don't care, but if you are trying to get a 2.4 tree over a modem, this is much much faster.

---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Just tried it:

$ bk clone -rv2.4.18-pre8 linux-2.5 linux-2.4
remote: ERROR-rev v2.4.18-pre8 doesn't exist

no, that's not a problem for me
-alex

Whoops, sorry, I thought I had checked that. OK, try

	bk clone -rv2.4.0 linux-2.5 linux-2.4

and then try the pull.

---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

That's much better. If someone haven't started yet figurring out what revision label to use for cloning, better stick to the Larry's one (later labels may produce conflicts in Makefile :)

I did a clone with this. However, I see no files, only directories. The files do seem to be in the SCCS directories, but I don't know how to make them appear in their normal place.

The command I ran was:
bk clone bk://linux24.bkbits.net/linux-2.4

Here is what the resulting linux-2.4 directory looked like:

[greear@grok bk]$ cd linux-2.4/
[greear@grok linux-2.4]$ ls -alt
total 64

drwxrwxr-x    6 greear   greear       4096 Mar 14 11:03 BitKeeper
drwxrwxr-x   16 greear   greear       4096 Mar 14 11:03 .
drwxrwxr-x    5 greear   greear       4096 Mar 14 11:03 scripts
drwxrwxr-x   29 greear   greear       4096 Mar 14 11:03 net
drwxrwxr-x    3 greear   greear       4096 Mar 14 11:03 lib
drwxrwxr-x    3 greear   greear       4096 Mar 14 11:03 mm
drwxrwxr-x   24 greear   greear       4096 Mar 14 11:03 include
drwxrwxr-x    3 greear   greear       4096 Mar 14 11:03 init
drwxrwxr-x    3 greear   greear       4096 Mar 14 11:03 ipc
drwxrwxr-x    3 greear   greear       4096 Mar 14 11:03 kernel
drwxrwxr-x   46 greear   greear       4096 Mar 14 11:02 fs
drwxrwxr-x   40 greear   greear       4096 Mar 14 11:01 drivers
drwxrwxr-x   17 greear   greear       4096 Mar 14 10:58 arch
drwxrwxr-x   29 greear   greear       4096 Mar 14 10:57 Documentation
drwxrwxr-x    2 greear   greear       4096 Mar 14 10:57 SCCS
drwxrwxr-x    3 greear   greear       4096 Mar 14 10:57 ..

[greear@grok linux-2.4]$

What am I missing?

Thanks,
Ben

Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Uh that is how BK works. The files are stored. Try

bk -r co

to get all the files. Omit the `-r' to check out only the current directory.

There are some decent tutorials and such on bitmovers[1] page.

[1] http://www.bitmover.com

Robert Love

Type 'make config'. Make is clever enough to get the Makefile from SCCS for you. Add the missing dependencies to the Makefile so that make will fetch stuff like scripts/Configure before trying to run it, etc.

Making it get all the Config.in files by parsing arch/$(ARCH)/config.in is a fairly trivial task... but you do need to explicitly check out all the include directories, because we don't know how to deal with that yet. With kbuild-2.4, make dep won't work very well, but kbuild-2.5 ought to be OK with everything but the include files, I think.

Or you could just check it all out beforehand with bk -r co.

--
dwmw2

Has anyone done this and made it work? It would save a lot of disk space and performance if someone were to so.

---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

I fixed up the dependencies on stuff in scripts/ and all the Config.in files, so I could take a clean tree and run make config.

The kbuild-2.4 make dep didn't find any C files so didn't do much - kbuild-2.5 would be a more useful base for such a game. I ignored the lack of dependencies and went on to 'make vmlinux', at which point it started trying to include header files from /usr/include/linux because they weren't present in the kernel tree and we don't build with -nostdinc.

Extracting the information about what include files we need to get from SCCS is a difficult problem.

--
dwmw2

IIRC, make *config should be doable, but make dep should require quite a bit of work (scripts/mkdep.c).

Stelian.

Stelian Pop <[email protected]>
Alcove - http://www.alcove.com

In article <[email protected]>,

Hey, the _sane_ way to do it is to not have all those crappy SCCS dependencies in all the tools, but to simply make a bk work area be a real file tree!

Larry, your argument that there are tools that are SCCS-aware is just not sane. For each tool that is SCCS-aware, I will name a hundred that are not, and that you're not going to fix. The only sane way to make _everything_ bitkeeper-aware is to keep the tree checked out and to keep the bitkeeper files somewhere else.

Right now simple things like command-line completion and

find . -name '*.[chS]' | xargs grep xxxx

do not work, because they either don't find files or they find the wrong ones (the internal bitkeeper files etc).

I'd much rather have a separate working area, ie if my repository is under ~/BK/repository/kernel/linux-2.5, then the checked out tree would be under ~/BK/repository/kernel/linux-2.5/workarea, and I would just have a simple symbolic link from ~/v2.5 to the workarea (and never even _see_ the BitKeeper files unless I thought I needed to).

None of this "special tools for normal actions" crap.

Linus

I always check out my trees with "bk -r co -q" precisely because of command-line completion. Anything that saves my poor hands from further pain is a good thing... :) And similar to your example, I switched from my preferred method of search, grep -r, to /usr/bin/find, just so I would (and had to) add "grep -v SCCS" in the middle of the xargs pipe.

Jeff

In article <[email protected]>,

That's fine for the command line completion, but it doesn't solve the generic problem. No tree-based thing works unchanged.

The fact is, mixing up the revision control directories and the working area is a design mistake, and one that BK perpetuated due to Larry's infatuation with the fact that "make" and "patch" already know how SCCS works.

(It should be noted that this design mistake is also one of the stumbling blocks for ever improving the BK databases. It limits your viability in the long run, which is why I'm trying to prod Larry into fixing it).

Linus

That is *a* sane way to do it and I'm fine with that. But just because you like things to work that way does not mean we will disallow the other way. I *like* the way SCCS works, I can do a "make clobber", which nukes all my .o's and does a "bk clean" and do a "ls" to see what crud I have left. I personally like going to a tree that has nothing but subdirectories in it, typing making, and having it do the right thing.

Neither I nor anyone else will force this mode of working on you. Not now, not ever. You, in turn, get to respect that some people like this mode and accept that it is going to remain one of the ways you can use BK forever.

I want to be really really clear on this. I'm not saying that because some tools know this, that's how it should be. I'm saying that I, and at least some other people, find that mode of operation useful. It's not going away.

That said, watch the changes that we are making and you'll see that we keep enhancing / bugfixing the ability to run your tree in checkout:get or checkout:edit mode, one of our customers has made some changes which seem to actually respect timestamps in an error free way so that you can run checkout:edit and not suffer the performance problems and also not lose data because of NFS/SMB timestamp issues. All that stuff is actively being hacked on right now.

In addition, our bug database is going to *force* us to offer what you want as an option. LINUS, READ THIS PART. The bug database uses a file as a container for each bug. That means that a query is as slow as an integrity check, and that obviously doesn't scale. While we can, and will, add indices to the bug database to address this, we are also addressing it by making a version of the software that stores the s.files in what amounts to an "ar" archive, so you can mmap that once and be done with it (as with everything, this is a simplification, there are actually multiple ar archives which are searched in most recent to least recent order so you don't rewrite the whole 170MB repository to make a one line fix).

All this means is that the decision to make the bug database a BK repository is exactly the right one from your point of view, Linus. It means we have to have a mode where there are no s.files, no SCCS directories, and it all still works. And since we're reasonable people, we'll all you to decide on a repository by repository basis what mode you want, so I can have my precious SCCS directories and make behaviour, and you can get rid of your awful SCCS directories and insane make behaviour.

Try

bk -r grep xxxx

One gotcha, and we'll fix this now that I think of it, is that this only greps the revision history.

---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

But you could do that even more easily if your work-area was separate.

If you do a "ls" and you get nothing back, you know you didn't have any crud.

Having a separate work-area doesn't mean you _have_ to check the stuff out.

And you can even get this behaviour too, if you were to just expand the (few) tools that know about SCCS to know about BK too.

Note that it's a lot easier to expand tools that already know about revision control than it is to expand tools that have never known about it. Just grep for SCCS and make it also look for a BK tree a few directories up.

But the thing is that if you do it my way, you _can_ have it both ways.

If you do it your way, you cannot.

Oh, I agree - I already pointed out to Jeff that if you use the SCCS approach, you can never change your database, which is going to force you eventually to come around ;)

Note that there is another way to do all this too: it would be quite nice (and probably not too hard) to create a filesystem that exports a BK archive, so that you could do something like

mount -o ro -tbk /home/BK/repository/xxxx xxx

and at least for the read-only case it should be pretty easy to make an intermezzo frontend to bk (ie the kernel part is already done, just the user-land server would be needed).

(The read-only one is the easy one to make, the read-write thing is a bit more involved, but potentially quite interesting)

Oh, I've tried exactly that, and it doesn't work at all for a few reasons.

Try

bk -r grep torvalds

in a linux tree, for a flavour of one of the problems. And for a flavour of another, one of the things I used to do was

	em $(find . -name '*.[chS]' | xargs grep -l '<\vfs_readdir\>')

which still works, but now I need to remember about not getting SCCS and BitKeeper directories, which is a pain, and make sure it's all checked out (which is where the filesystem support could be really nice).

Note that the filesystem approach might make your migration path easier:
you could keep the existing BK database structure (used by the server and legacy commands) while migrating out one tool at a time to the "separate tree" layout. Maybe.

Linus

By "your way" you mean "the current way", right? In which case, the future option I described is OK, right? Or am I still missing something?

I already did this a long time ago, and how the files are stored is completely orthogonal. I used the user level NFS server and you could do a

	mount -o ro,rev=v.2.5.5 -tnfs /home/BK/repository/xxxx v2.5.5

and it worked just fine. I personally hate this because there is no way that I have ever seen to make filesystem semantics == SCM semantics. It turns into a hack for read/write. If it made you happy to do this for read only, hey, there's a nice newbie project.

Whoops, sorry, try it with -Ur, the -U says "user files only, skip the BK crud"

bk -Ur grep torvalds
CREDITS 1.1 E: [email protected]
README 1.1 them to me ([email protected]), and possibly to any other

SubmittingDrivers       1.1             <[email protected]>.
SubmittingPatches       1.1     Linux kernel.  His e-mail address is [email protected].  He gets
oops-tracing.txt        1.1     From: Linus Torvalds <[email protected]>

CREDITS 1.1 Linus Torvalds <[email protected]>

---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

No, the future option looks fine. I suspect that when you go for a real integrated database, you'll automatically get all the features I like.

The thing is, I think that the "work area" really is overrated.

There's no reason to have a work area at all if you just had:
- read-only filesystem for grep/make (autogenerated)
- separate commands for editing

I don't find it depressing at all to have to use "bk editor" to edit a file. I just aliased that one, and I'm all done. That's why I don't think that the filesystem interface should have revisions or writeability: once you get into those things, the filesystem abstractions simply aren't valid any more.

But reading the current contents (as opposed to reading some revision) of the thing _is_ a perfectly valid thing to do, where the FS interfaces do actually map perfectly.

That's better. It doesn't fix the pipe example, though. You're missing the -l option to grep etc..

Linus

Tried clearcase?

Well, it sort of works. If you use emacs, you're set because emacs can convert the file from read only to read/write in VC mode, which is something we'll have to update when we go to the new format.

If you use vim and ctags, it sucks because I haven't yet taught vim how to go from a read only revision controlled file to a read/write file. It would be way cool if some vim genius out there showed me how to do that, I know it is possible, vim has the hooks, I simply don't have the time to go figure it out.

Sure, that part works fine. But the write part is a lot more dicey. And there has to be some way to get to the revision history for all the tools that want that.

It's absolutely possible to do all this stuff through the file system and a collection of smart tools, that's more or less what clearcase is. But it sucks rocks. One of our biggest selling points is that we *don't* work like clearcase. ClearCase has shown that it is a performance, stability, and maintainence nightmare to integrate your SCM with the filesystem. It means your SCM system is now OS specific. *You* may not care that we run on AIX/Windows/Whatever, but we have to do that to survive and make enough money to keep making BK better and it actually shakes out lots of bugs.

Ask the ClearCase guys if they would choose the BK way or the ClearCase way, given a second chance, and I'll bet you the smarts would do BK.

Yeah, point taken. I need a --grep-options=<your favorites here> or something. I need the same thing for bk diffs.

---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

I'm certainly not a "vim genius", but somehow I managed to write some vim autocmds that do this ;-)

You can get the vim script from

http://www.myipv6.de/vim/extensions/bk.vim

Simply source it from your .vimrc. I tested it with vim 6.0 only, although it should also work with prior versions.

On open, it tries to checkout a file from bitkeeper if it isn't already checked out (doing "bk get" if you open it readonly and "bk edit" otherwise), and it "bk edit"s the file if you start making changes to a readonly bitkeeper controlled file.

Unfortunately, vim doesn't trigger the FileChangedRO autocmd if you do a ":set readonly!" to go from readonly to read/write, so it doesn't handle this case (AFAIK there is no way to intercept this command).

Andreas

       Andreas Ferber - dev/consulting GmbH - Bielefeld, FRG
     ---------------------------------------------------------
         +49 521 1365800 - [email protected] - www.devcon.net

Well, you shouldn't be going from read-write to readonly in this way anyways, so I don't think it is a problem.

Cheers, Andreas
--
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert

Here's the deal. I know you guys all think that I'm a genius and everything, but I'm actually dumb as a board. The "design mistake" was made so that I could have BK generate pure SCCS files and test that I did the same thing as a known working tool, ATT SCCS. By doing that, I easily saved myself a year of design. Making interleaved deltas work is hard for me (we have Rick here now and he's forgotten more about this stuff than I'll ever know, but we didn't have him when I wrote the SCCS compat weave).

At this point, I trust our implementation of the weave more than I trust the ATT one, and ours handles several cases that theirs doesn't, so I'm a lot less concerned about that compatibility.

And we know that we can get better performance, and dramatically reduce fragmentation, by sticking all the files in one big file, and we've known this for a long time. We're gonna do it, you're gonna love, it's less filling, it tastes great. There is only so many things that we can do at once and this is on our short list, but it isn't at the top. Keep that in mind as you push us to make enhancements, there is no free lunch, so prioritize.

I'm gonna hack at least make & patch to know about the new format and work the way they do now. So I can have your cake and eat it too. If I can't get the FSF to take the changes, we'll just ship 'em, we ship diff & patch already, so it's not so hard to alias make='bk make'.

---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

While you're in there, is there any way I can have an option to have the 'shouting' SCCS become .SCCS or something, so a normal listing just shows the files I'm interested in?

Daniel

It seems like once all of the bk information is kept in one file per repository that the one file can be named something like .bk.db or something like that and thus fix the problem you're having...

Thu Mar 14 21:52:33 CET 2002 ide-clean-22

- Apply more patches from Vojtech Pavlik for the handling of host chip setup. Hopefully they are settled now.

- Kill unused CONFIG_BLK_DEV_MODES

- Push register addressing down in to task_vlb_sync.

- Make the taskfile parsing stuff actually readable. This is compressing the code by an incredible amount. We use just one function doing the whole scanning right now. This should make sure that the IRQ handler used by a particular command is always right. I didn't introduce typos hopefully here.

- Don't call ide_handler_parser as argument for do_taskfile() any longer. We have killed this function by coalescing it's functionality with ide_cmd_type_parser() anyway.

- Kill unused SLC90E66 code, which Vojtech apparently missed in his patch.

- sync up with 2.5.7-pre2

Once again the actual patch is rather big mostly due to the removal of some default configuration variables which are not used anylonger. So time for the next patch stage.

Ahh, that did fix the problem. It must be configured differently where I work, because I do not have to do the bk -r co command there.

I went though them once...looks like I need a refresher course!

Thanks,
Ben

Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Ben Greear <[email protected]> [02/03/14 13:46]:

You probably have
checkout: get
in your BitKeeper/etc/config file

If it was not enough that "bleeding-edge" required a pint of blood but now we get BitKeeper access to the latest and greatest. Cool! Maybe I should be on the regular shipment list from the local blood bank. Keep up the good work. I certainly appreciate it.

Stephen