Making diff(1) of linux kernels faster
Hi Paul !
congratulations for this improvement, it seems really interesting. BTW, I personnaly use hard links between kernels to make the effective data set smaller, and I'd
like to explain here how I proceed since there are often people who seem completely amazed by this method which I learned here on LKML a few years ago :
# cd /usr/src # tar Ixf anydir/linux-2.4.12.tar.bz2 # cp -dRflp linux linux-2.4.12
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
this way, only dir entries are duplicated, so very little overhead
# (cd linux && bzcat anydir/patch-2.4.13pre1.bz2|patch -Np1)
# cp -dRflp linux linux-2.4.13pre1
Quoted 1 line
Quoted 1 line
Quoted 1 line
now, only file affected by the patch are
duplicated
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
then, you can work inside linux dir, and construct your patches very quickly since a few files effectively differ from your new tree and old
ones.
Be very careful not to modify a multi-linked file, or it will be damaged in all trees and won't be seen by diff. your editor must unlink before saving.
I hope it will help someone as it has helped me for a while now. I nearly always have sub-second diffs, even with not-so-much RAM.
Cheers,
Willy
___________________________________________________________
Un nouveau Nokia Game commence.
Allez sur http://fr.yahoo.com/nokiagame avant le 3 novembre pour participer à cette aventure tous médias.
=?iso-8859-1?q?willy=20tarreau?= <[email protected]> said:
[...]
Quoted 3 lines
Be very careful not to modify a multi-linked file, or it will be damaged in all trees and won't be seen by diff. your editor must unlink before saving.
Most don't. ed(1), vi(1) and emacs(1) are careful tro write to the very same file. jed(1) is the only outlier I'm aware of...
Signature · 2 lines
Horst von Brand [email protected] Casilla 9G, Vin~a del Mar, Chile +56 32 672616
Quoted 7 lines · Horst von Brand
Quoted 3 lines · =?iso-8859-1?q?willy=20tarreau?= <[email protected]> said:
Be very careful not to modify a multi-linked file, or it will be damaged in all trees and won't be seen by diff. your editor must unlink before saving.
Most don't. ed(1), vi(1) and emacs(1) are careful tro write to the very same file. jed(1) is the only outlier I'm aware of...
emacs does mv file file~ before saving file so the edited file will not be linked byt the backup file will be. You can stop it doing this by setting backup-by-copying-when-linked.
$ echo "Test1" >> test1 $ ln test1 test2 $ emacs test2 # modify and save the file $ ls -i test* 2491498 test1 2491500 test2 2491498 test2~
However as Horst von Brand noted if you try this with (standard) vi you will edit the linked file which is the same behaviour as if you set backup-by-copying-when-linked in emacs.
Signature · 2 lines
Nick Craig-Wood [email protected]
Quoted 11 lines · Nick Craig-Wood
Date: Thu, 18 Oct 2001 09:02:22 +0100
From: Nick Craig-Wood <[email protected]>
To: [email protected]
Cc: Paul Gortmaker <[email protected]>, [email protected],
willy tarreau <[email protected]>
Subject: Re: Making diff(1) of linux kernels fasterQuoted 3 lines · Horst von Brand
Quoted 2 lines · =?iso-8859-1?q?willy=20tarreau?= <[email protected]> said:
Be very careful not to modify a multi-linked file, or it will be damaged in all trees and won't be seen by
To be sure it is not possible to modify original tree files, I do chown -R root.root original_tree
before copying it (via cp -lR) to new one, which will be modified with whatever tools by me, logged in as a regular user. For those having root access to a box this might be a useful way of preventing accidents ... (this of course also assumes sane file permissions)
[...]
Quoted 9 lines
Quoted 4 lines
Quoted 1 line
diff. your editor must unlink before saving.
Most don't. ed(1), vi(1) and emacs(1) are careful tro write to the very same file. jed(1) is the only outlier I'm aware of...
emacs does mv file file~ before saving file so the edited file will not be linked byt the backup file will be. You can stop it doing this by setting backup-by-copying-when-linked.
Best regards,
Wojtek
--------------------
Wojtek Pilorz
[email protected]
Thursday, October 18, 2001, 11:55:41 AM,
Quoted 2 lines · Wojtek Pilorz
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Be very careful not to modify a multi-linked file, or it will be damaged in all trees and won't be seen by
WP> To be sure it is not possible to modify original tree files, I do WP> chown -R root.root original_tree
WP> before copying it (via cp -lR) to new one, which will be modified with WP> whatever tools by me, logged in as a regular user. For those having root WP> access to a box this might be a useful way of preventing accidents ... WP> (this of course also assumes sane file permissions)
WP> [...]
Everytime I see this 'hardlinked kernel tree' technique explained, I think about true COW fs where duplicate files are physically the same single file but users don't care about that - COW magic... No hardlink/vi/emacs/... tricks will be needed...
Signature · 2 lines
Best regards, vda mailto:[email protected]