Skip to contentSkip to conversation

Making diff(1) of linux kernels faster

DISCUSSION willy tarreau 5 participants 5 messages Oct 17, 2001 – Oct 18, 2001

lore
Share & tools
lore
#0 lore raw

Hi Paul !

congratulations for this improvement, it seems really interesting. BTW, I personnaly use hard links between kernels to make the effective data set smaller, and I'd
like to explain here how I proceed since there are often people who seem completely amazed by this method which I learned here on LKML a few years ago :

# cd /usr/src
# tar Ixf anydir/linux-2.4.12.tar.bz2
# cp -dRflp linux linux-2.4.12
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

this way, only dir entries are duplicated, so very little overhead

# (cd linux && bzcat anydir/patch-2.4.13pre1.bz2|patch -Np1)

# cp -dRflp linux linux-2.4.13pre1
Quoted 1 line
Quoted 1 line
Quoted 1 line

now, only file affected by the patch are

duplicated

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

then, you can work inside linux dir, and construct your patches very quickly since a few files effectively differ from your new tree and old

ones.

Be very careful not to modify a multi-linked file, or it will be damaged in all trees and won't be seen by diff. your editor must unlink before saving.

I hope it will help someone as it has helped me for a while now. I nearly always have sub-second diffs, even with not-so-much RAM.

Cheers,
Willy

___________________________________________________________
Un nouveau Nokia Game commence.
Allez sur http://fr.yahoo.com/nokiagame avant le 3 novembre pour participer à cette aventure tous médias.

#1 lore raw

=?iso-8859-1?q?willy=20tarreau?= <[email protected]> said:

[...]

Quoted 3 lines

Be very careful not to modify a multi-linked file, or it will be damaged in all trees and won't be seen by diff. your editor must unlink before saving.

Most don't. ed(1), vi(1) and emacs(1) are careful tro write to the very same file. jed(1) is the only outlier I'm aware of...

Signature · 2 lines
Horst von Brand                             [email protected]
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616
#2 lore raw
Quoted 7 lines · Horst von Brand
Quoted 3 lines · =?iso-8859-1?q?willy=20tarreau?= <[email protected]> said:

Be very careful not to modify a multi-linked file, or it will be damaged in all trees and won't be seen by diff. your editor must unlink before saving.

Most don't. ed(1), vi(1) and emacs(1) are careful tro write to the very same file. jed(1) is the only outlier I'm aware of...

emacs does mv file file~ before saving file so the edited file will not be linked byt the backup file will be. You can stop it doing this by setting backup-by-copying-when-linked.

$ echo "Test1" >> test1
$ ln test1 test2
$ emacs test2 # modify and save the file
$ ls -i test*
2491498 test1  2491500 test2  2491498 test2~

However as Horst von Brand noted if you try this with (standard) vi you will edit the linked file which is the same behaviour as if you set backup-by-copying-when-linked in emacs.

Signature · 2 lines
Nick Craig-Wood
[email protected]
#3 lore raw
Quoted 11 lines · Nick Craig-Wood

Date: Thu, 18 Oct 2001 09:02:22 +0100
From: Nick Craig-Wood <[email protected]>
To: [email protected]
Cc: Paul Gortmaker <[email protected]>, [email protected],
willy tarreau <[email protected]>
Subject: Re: Making diff(1) of linux kernels faster

Quoted 3 lines · Horst von Brand
Quoted 2 lines · =?iso-8859-1?q?willy=20tarreau?= <[email protected]> said:

Be very careful not to modify a multi-linked file, or it will be damaged in all trees and won't be seen by

To be sure it is not possible to modify original tree files, I do chown -R root.root original_tree

before copying it (via cp -lR) to new one, which will be modified with whatever tools by me, logged in as a regular user. For those having root access to a box this might be a useful way of preventing accidents ... (this of course also assumes sane file permissions)
[...]

Quoted 9 lines
Quoted 4 lines
Quoted 1 line

diff. your editor must unlink before saving.

Most don't. ed(1), vi(1) and emacs(1) are careful tro write to the very same file. jed(1) is the only outlier I'm aware of...

emacs does mv file file~ before saving file so the edited file will not be linked byt the backup file will be. You can stop it doing this by setting backup-by-copying-when-linked.

Best regards,

Wojtek

--------------------
Wojtek Pilorz
[email protected]

#4 lore raw

Thursday, October 18, 2001, 11:55:41 AM,

Quoted 2 lines · Wojtek Pilorz
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Be very careful not to modify a multi-linked file, or it will be damaged in all trees and won't be seen by

WP> To be sure it is not possible to modify original tree files, I do WP> chown -R root.root original_tree

WP> before copying it (via cp -lR) to new one, which will be modified with WP> whatever tools by me, logged in as a regular user. For those having root WP> access to a box this might be a useful way of preventing accidents ... WP> (this of course also assumes sane file permissions)
WP> [...]

Everytime I see this 'hardlinked kernel tree' technique explained, I think about true COW fs where duplicate files are physically the same single file but users don't care about that - COW magic... No hardlink/vi/emacs/... tricks will be needed...

Signature · 2 lines
Best regards, vda
mailto:[email protected]