Skip to contentSkip to conversation

kernel.bkbits.net off the air

DISCUSSION Larry McVoy 11 participants 37 messages Nov 7, 2003 – Nov 14, 2003

lore
Share & tools
lore
#0 lore raw

As many of you have figured out, I took kernel.bkbits.net (aka bk.kernel.org, cvs.kernel.org, and svn.kernel.org) of the air yesterday due to the breakin that attempted to add a trojan horse to the kernel source.

I took it down after talking with Linus and Dave about it, the point was to shut down the disk drive so that we can go do forensics on it after the fact and see what we can figure out. Maybe someone can track down who caused the problem.

This means someone has to go down to the colo with a new disk and do an install and we've been too busy to do this. Would anyone object if this wasn't done until this weekend? We're pretty booked up here with other work. Last I checked only about 6 IP addresses where using the CVS server, I've never checked on the SVN server (Ben? You have any idea?).

Signature · 2 lines
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm
#1 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 5 lines · Larry McVoy

This means someone has to go down to the colo with a new disk and do an install and we've been too busy to do this. Would anyone object if this wasn't done until this weekend? We're pretty booked up here with other work. Last I checked only about 6 IP addresses where using the CVS server, I've never checked on the SVN server (Ben? You have any idea?).

5 IP addresses, I switched to bk yesterday night ;)

- --
Clemens Schwaighofer - IT Engineer & System Administration
==========================================================
Tequila Japan, 6-17-2 Ginza Chuo-ku, Tokyo 104-8167, JAPAN
Tel: +81-(0)3-3545-7703 Fax: +81-(0)3-3545-7343
http://www.tequila.jp
==========================================================
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQE/q2uIjBz/yQjBxz8RAqozAJ4z8DQsfWuudcl2rHrxMi7nCO7kggCgoO43 dAMfTKS/ag3o81aqavLNzAk=
=71I2
-----END PGP SIGNATURE-----

#2 lore raw
In reply to Larry McVoy
Followup to:  <[email protected]>
By author:    Larry McVoy <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 10 lines

As many of you have figured out, I took kernel.bkbits.net (aka bk.kernel.org, cvs.kernel.org, and svn.kernel.org) of the air yesterday due to the breakin that attempted to add a trojan horse to the kernel source.

I took it down after talking with Linus and Dave about it, the point was to shut down the disk drive so that we can go do forensics on it after the fact and see what we can figure out. Maybe someone can track down who caused the problem.

Quoted 6 lines

This means someone has to go down to the colo with a new disk and do an install and we've been too busy to do this. Would anyone object if this wasn't done until this weekend? We're pretty booked up here with other work. Last I checked only about 6 IP addresses where using the CVS server, I've never checked on the SVN server (Ben? You have any idea?).

That doesn't include anyone who uses the mirrored repository on the main kernel.org machines. Doubt it's a big deal with the timing, though.

-hpa

Signature · 4 lines
<[email protected]> at work, <[email protected]> in private!
If you send me mail in HTML format I will assume it's spam.
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64
#3 lore raw
Quoted 2 lines · H. Peter Anvin

That doesn't include anyone who uses the mirrored repository on the main kernel.org machines.

Last I checked, kernel.org isn't offering pserver access, just ftp. If you want to take over the CVS access just say the word.

Signature · 2 lines
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm
#4 lore raw
Quoted 6 lines · Larry McVoy
Quoted 2 lines · H. Peter Anvin

That doesn't include anyone who uses the mirrored repository on the main kernel.org machines.

Last I checked, kernel.org isn't offering pserver access, just ftp. If you want to take over the CVS access just say the word.

It is faster for me to use rsync on the CVS root locally, and then use the local repository instead. Rsync is better than CVS when it comes to syncs. Cvsps expecially, really wants a local repository when you start playing heavily with -g.

- Davide

#5 lore raw
Quoted 6 lines · Davide Libenzi
Quoted 4 lines · Larry McVoy
Quoted 2 lines · H. Peter Anvin

That doesn't include anyone who uses the mirrored repository on the main kernel.org machines.

Quoted 7 lines
Quoted 2 lines

Last I checked, kernel.org isn't offering pserver access, just ftp. If you want to take over the CVS access just say the word.

It is faster for me to use rsync on the CVS root locally, and then use the local repository instead. Rsync is better than CVS when it comes to syncs. Cvsps expecially, really wants a local repository when you start playing heavily with -g.

If that's the preferred interface then maybe we should shut down the pserver completely and let people rsync it from kernel.org.

Signature · 2 lines
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm
#6 lore raw
Quoted 2 lines · Larry McVoy

If that's the preferred interface then maybe we should shut down the pserver completely and let people rsync it from kernel.org.

Larry, that's just my way. Don't make me blame by others to have you shut down the pserver :-)

- Davide

#7 lore raw
In reply to Davide Libenzi
Quoted 6 lines · Davide Libenzi
Quoted 4 lines · Larry McVoy
Quoted 2 lines · H. Peter Anvin

That doesn't include anyone who uses the mirrored repository on the main kernel.org machines.

Quoted 7 lines
Quoted 2 lines

Last I checked, kernel.org isn't offering pserver access, just ftp. If you want to take over the CVS access just say the word.

It is faster for me to use rsync on the CVS root locally, and then use the local repository instead. Rsync is better than CVS when it comes to syncs. Cvsps expecially, really wants a local repository when you start playing heavily with -g.

same here, however the rsync export on kernel.org is lacking a two sequence number locking that would allow us to checkout a coherent copy of the cvs repository. Currently it works by luck.

if we add the two sequence numbers to the rsync on kernel.org, I believe shutting down the pserver is ok.

#8 lore raw
Quoted 3 lines · Andrea Arcangeli

same here, however the rsync export on kernel.org is lacking a two sequence number locking that would allow us to checkout a coherent copy of the cvs repository. Currently it works by luck.

Peter, how the current rsync BKCVS root is updated at kernel.org? Is coherency guaranteed in some way?

- Davide

#9 lore raw
Quoted 8 lines · Davide Libenzi
Quoted 3 lines · Andrea Arcangeli

same here, however the rsync export on kernel.org is lacking a two sequence number locking that would allow us to checkout a coherent copy of the cvs repository. Currently it works by luck.

Peter, how the current rsync BKCVS root is updated at kernel.org? Is coherency guaranteed in some way?

our rsync isn't going to be coherency guaranteed anyways, no matter how the updates shows up on kernel.org, rsync can't even hang on per-file locks, sure it can't be coherent as a whole.

The best way to fix this isn't to add locking to rsync, but to add two files inside or outside the tree, each one is a sequence number, so you fetch file1 first, then you rsync and you fetch file2, then you compare them. If they're the same, your rsync copy is coherent. It's the same locking we introduced with vgettimeofday.

Ideally rsync could learn to check the sequence numbers by itself but I don't mind a bit of scripting outside of rsync.

#10 lore raw
Quoted 11 lines · Andrea Arcangeli

the updates shows up on kernel.org, rsync can't even hang on per-file locks, sure it can't be coherent as a whole.

The best way to fix this isn't to add locking to rsync, but to add two files inside or outside the tree, each one is a sequence number, so you fetch file1 first, then you rsync and you fetch file2, then you compare them. If they're the same, your rsync copy is coherent. It's the same locking we introduced with vgettimeofday.

Ideally rsync could learn to check the sequence numbers by itself but I don't mind a bit of scripting outside of rsync.

Wouldn't a simpler "stop-rsync -> update-root -> start-rsync" work? If you'll hit an update you will get a error from your local rsync, that will let you know to retry the operation.

- Davide

#11 lore raw
Quoted 12 lines · Davide Libenzi
Quoted 9 lines · Andrea Arcangeli

the updates shows up on kernel.org, rsync can't even hang on per-file locks, sure it can't be coherent as a whole.

The best way to fix this isn't to add locking to rsync, but to add two files inside or outside the tree, each one is a sequence number, so you fetch file1 first, then you rsync and you fetch file2, then you compare them. If they're the same, your rsync copy is coherent. It's the same locking we introduced with vgettimeofday.

Quoted 5 lines
Quoted 2 lines

Ideally rsync could learn to check the sequence numbers by itself but I don't mind a bit of scripting outside of rsync.

Wouldn't a simpler "stop-rsync -> update-root -> start-rsync" work? If you'll hit an update you will get a error from your local rsync, that will let you know to retry the operation.

Part of the problem is that there are multiple steps in the rsync chain, some of which can't be stopped in this way.

The sequence number idea looks sensible to me. Larry, would it be too much work to have the cvs repository generator generate these files?

-hpa

#12 lore raw
Quoted 13 lines · H. Peter Anvin
Quoted 6 lines
Quoted 6 lines

The best way to fix this isn't to add locking to rsync, but to add two files inside or outside the tree, each one is a sequence number, so you fetch file1 first, then you rsync and you fetch file2, then you compare them. If they're the same, your rsync copy is coherent. It's the same locking we introduced with vgettimeofday.

Quoted 6 lines
Quoted 2 lines

Ideally rsync could learn to check the sequence numbers by itself but I don't mind a bit of scripting outside of rsync.

Wouldn't a simpler "stop-rsync -> update-root -> start-rsync" work? If you'll hit an update you will get a error from your local rsync, that will let you know to retry the operation.

Quoted 5 lines

Part of the problem is that there are multiple steps in the rsync chain, some of which can't be stopped in this way.

The sequence number idea looks sensible to me. Larry, would it be too much work to have the cvs repository generator generate these files?

So the update of the rsync repo should do something like:

update file1
update repo
update file2

Isn't it? I do not understand how this guarantee coherency:

Kernel.org Me
get file1 (old value)
update file1 get repo-file1 (old value)
update repo-file1
...
update repo-fileJ
... get repo-fileJ (new value)
update repo-fileN get file2 (old value)
update file2

- Davide

#13 lore raw
Quoted 29 lines · Davide Libenzi
Quoted 13 lines · H. Peter Anvin
Quoted 6 lines
Quoted 6 lines

The best way to fix this isn't to add locking to rsync, but to add two files inside or outside the tree, each one is a sequence number, so you fetch file1 first, then you rsync and you fetch file2, then you compare them. If they're the same, your rsync copy is coherent. It's the same locking we introduced with vgettimeofday.

Quoted 6 lines
Quoted 2 lines

Ideally rsync could learn to check the sequence numbers by itself but I don't mind a bit of scripting outside of rsync.

Wouldn't a simpler "stop-rsync -> update-root -> start-rsync" work? If you'll hit an update you will get a error from your local rsync, that will let you know to retry the operation.

Quoted 5 lines

Part of the problem is that there are multiple steps in the rsync chain, some of which can't be stopped in this way.

The sequence number idea looks sensible to me. Larry, would it be too much work to have the cvs repository generator generate these files?

So the update of the rsync repo should do something like:

update file1
update repo
update file2

Isn't it? I do not understand how this guarantee coherency:

Quoted 9 lines

Kernel.org Me
get file1 (old value)
update file1 get repo-file1 (old value)
update repo-file1
...
update repo-fileJ
... get repo-fileJ (new value)
update repo-fileN get file2 (old value)
update file2

you must pick file2 before file1:

	you:

	do
		get file2
		get repo-file1-j
		get file1
	while file2 != file1 && sleep 10
#14 lore raw
Quoted 9 lines · Andrea Arcangeli

you must pick file2 before file1:

	you:

	do
		get file2
		get repo-file1-j
		get file1
	while file2 != file1 && sleep 10

Yes, this closes ;)

- Davide

#15 lore raw
In reply to Andrea Arcangeli
Quoted 11 lines · Andrea Arcangeli

you must pick file2 before file1:

	you:

	do
		get file2
		get repo-file1-j
		get file1
	while file2 != file1 && sleep 10

Okay... I'm starting to think the sequencing requirements on these files may be hard to maintain across multiple levels of rsync... but perhaps I'm wrong, in particular if 'file2' sorts hierachially-lexically last and 'file1' first...

-hpa

#16 lore raw
Quoted 13 lines · H. Peter Anvin
Quoted 11 lines · Andrea Arcangeli

you must pick file2 before file1:

	you:

	do
		get file2
		get repo-file1-j
		get file1
	while file2 != file1 && sleep 10
Quoted 4 lines

Okay... I'm starting to think the sequencing requirements on these files may be hard to maintain across multiple levels of rsync... but perhaps I'm wrong, in particular if 'file2' sorts hierachially-lexically last and 'file1' first...

Doing something like:

rsync file2
rsync repo
rsync file1

should work, doesn't it?

- Davide

#17 lore raw
In reply to H. Peter Anvin
Quoted 13 lines · H. Peter Anvin
Quoted 11 lines · Andrea Arcangeli

you must pick file2 before file1:

	you:

	do
		get file2
		get repo-file1-j
		get file1
	while file2 != file1 && sleep 10
Quoted 4 lines

Okay... I'm starting to think the sequencing requirements on these files may be hard to maintain across multiple levels of rsync... but perhaps I'm wrong, in particular if 'file2' sorts hierachially-lexically last and 'file1' first...

we've to start rsync three times to get them in order, 3 tcp connections, there's no way to specify the order in the rsync command line, infact those two sequence files can be as well outside the tree and we can fetch temporarily in a /tmp/ directory or similar. However we can probably hack the rsync client to be able to specify the two sequence numbers on the command line.

It maybe also cleaner to use a slightly more complicated but more compact algorithm, this would make a potential new rsync command line option cleaner since only 1 sequence file would need to be specified:

	do {
		seq = fetch(sequence-file);
		if (seq & 1)
			break;
		rsync
		if (seq != fetch(sequence-file))
			seq = 1;
	} while (seq & 1 && sleep 10 /* ideally exponential backoff */)

this way only 1 sequence-file is needed for each repository that we want to checkout. the server side only has to increase twice the same file before and after each update of the repository, so the server side is even simpler (with the only additional requirement that the sequence number has to start "even"), only the client side is a bit more complicated.

#18 lore raw
Quoted 21 lines · Andrea Arcangeli

we've to start rsync three times to get them in order, 3 tcp connections, there's no way to specify the order in the rsync command line, infact those two sequence files can be as well outside the tree and we can fetch temporarily in a /tmp/ directory or similar. However we can probably hack the rsync client to be able to specify the two sequence numbers on the command line.

It maybe also cleaner to use a slightly more complicated but more compact algorithm, this would make a potential new rsync command line option cleaner since only 1 sequence file would need to be specified:

	do {
		seq = fetch(sequence-file);
		if (seq & 1)
			break;
		rsync
		if (seq != fetch(sequence-file))
			seq = 1;
	} while (seq & 1 && sleep 10 /* ideally exponential backoff */)
Quoted 6 lines

this way only 1 sequence-file is needed for each repository that we want to checkout. the server side only has to increase twice the same file before and after each update of the repository, so the server side is even simpler (with the only additional requirement that the sequence number has to start "even"), only the client side is a bit more complicated.

Good grief. This is messy as hell, and really interferes rather badly with the whole kernel.org mirror setup.

I guess the "best" solution is to use LVM atomic snapshots, and only allow rsync off the atomic snapshot. That way any particular rsync session would always be consistent. That's a *HUGE* amount of work, though, and still doesn't solve the mirrors issue -- I don't control what the mirrors run. On the other hand, I don't know how many mirror sites actually mirror /pub/scm since it's not a requirement.

-hpa

#19 lore raw
Quoted 2 lines · H. Peter Anvin

Good grief. This is messy as hell, and really interferes rather badly with the whole kernel.org mirror setup.

allowing coherent checkouts from the mirrors too is an interesting matter, I believe I've a solution for it, but it obviously requires a special script to spread the mirror of /pub/scm. However pserver is also not being mirrored right now, and rsync is still more efficient (and it carries all the info, not just a certain local copy), so even w/o mirror-coherency it would be better.

Quoted 6 lines

I guess the "best" solution is to use LVM atomic snapshots, and only allow rsync off the atomic snapshot. That way any particular rsync session would always be consistent. That's a *HUGE* amount of work, though, and still doesn't solve the mirrors issue -- I don't control what the mirrors run. On the other hand, I don't know how many mirror sites actually mirror /pub/scm since it's not a requirement.

I'm unsure how you can leave an rsync running on the old snapshot and the new forked off ones running in the new snapshot. as for the mirror issues it should be possible to make it work like this:

1) increase file1 on the mirror
2) read file2 on the master and store it on the userspace stack
3) copy the tree from master to mirror
4) read file1 on the master and compare it with file2 on the stack
5) if they're different goto 2 after a delay
6) increase file2 on the mirror

basically those sequence numbers should not be copied, they should be completely separated, each server exporting the tree will have its own sequence numbers. ideally the master could be at sequence number 1000 and the mirror could be at sequence number 100, depends on the frequency of the mirror sync and the age of the mirror. the mirror could fetch multiple updats at once and its sequence number would advance slower, or it could sync more frequently than there are updates on the server and its sequence number would advance more quickly than the master.

(for simplicity I'm using the two sequence number model, but clearly the above can be easily converted to the single sequence number model, and infact that's preferable since having a single sequence number is cleaner from a filesystem maintainance point of view, both models are functionally equivalent)

#20 lore raw

Please pardon my late intrusion into this discussion of what appears to be CVS mirror coherency. The impression I get, which may be wrong, is that one or both of these problems is happening:

- A user is pulling from one CVS mirror, but that mirror is in the process of being updated from the BK source, so the user gets some new files and some old ones.

- A user is pulling from more than one CVS mirror at the same time, and not all mirrors are at the same revision level.

Either way, and I would not be surprised if someone else had suggested this already, but being a graphics person, let me suggest a common technique for dealing with this problem: double-buffering.

Let's assume a combination of the two above cases, because the general solution applies to all cases.

To begin with, all mirrors serve from the "front buffer", all of which are known to be at the same revision level for all files. While serving the front buffers, the "back buffers" are being updated.

At a prescribed time, all servers go off-line (to deal with asynchronicity between clocks), swap the front and back buffers, and then go back online.

The synchronization problem can be dealt with either by having everyone have their clocks set relatively close but go offline for a few seconds just in case there is some drift, or there can be a master server which signals the others when to swap buffers.

#21 lore raw
In reply to H. Peter Anvin
Quoted 6 lines · H. Peter Anvin

I guess the "best" solution is to use LVM atomic snapshots, and only allow rsync off the atomic snapshot. That way any particular rsync session would always be consistent. That's a *HUGE* amount of work, though, and still doesn't solve the mirrors issue -- I don't control what the mirrors run. On the other hand, I don't know how many mirror sites actually mirror /pub/scm since it's not a requirement.

BTW, is rsync.kernel.org::pub/scm/linux/kernel/bkcvs currently being fed with new data? I don't get any updates.

- Davide

#22 lore raw
Quoted 11 lines · Davide Libenzi
Quoted 6 lines · H. Peter Anvin

I guess the "best" solution is to use LVM atomic snapshots, and only allow rsync off the atomic snapshot. That way any particular rsync session would always be consistent. That's a *HUGE* amount of work, though, and still doesn't solve the mirrors issue -- I don't control what the mirrors run. On the other hand, I don't know how many mirror sites actually mirror /pub/scm since it's not a requirement.

Quoted 2 lines

BTW, is rsync.kernel.org::pub/scm/linux/kernel/bkcvs currently being fed with new data? I don't get any updates.

It should be... I'll look at it in a bit.

-hpa

#23 lore raw
In reply to H. Peter Anvin
Quoted 32 lines · H. Peter Anvin
Quoted 21 lines · Andrea Arcangeli

we've to start rsync three times to get them in order, 3 tcp connections, there's no way to specify the order in the rsync command line, infact those two sequence files can be as well outside the tree and we can fetch temporarily in a /tmp/ directory or similar. However we can probably hack the rsync client to be able to specify the two sequence numbers on the command line.

It maybe also cleaner to use a slightly more complicated but more compact algorithm, this would make a potential new rsync command line option cleaner since only 1 sequence file would need to be specified:

	do {
		seq = fetch(sequence-file);
		if (seq & 1)
			break;
		rsync
		if (seq != fetch(sequence-file))
			seq = 1;
	} while (seq & 1 && sleep 10 /* ideally exponential backoff */)
Quoted 6 lines

this way only 1 sequence-file is needed for each repository that we want to checkout. the server side only has to increase twice the same file before and after each update of the repository, so the server side is even simpler (with the only additional requirement that the sequence number has to start "even"), only the client side is a bit more complicated.

Good grief. This is messy as hell, and really interferes rather badly with the whole kernel.org mirror setup.

Quoted 6 lines

I guess the "best" solution is to use LVM atomic snapshots, and only allow rsync off the atomic snapshot. That way any particular rsync session would always be consistent. That's a *HUGE* amount of work, though, and still doesn't solve the mirrors issue -- I don't control what the mirrors run. On the other hand, I don't know how many mirror sites actually mirror /pub/scm since it's not a requirement.

The LVM snapshots would provide the guarantee of consistancy which would be nice for those who aren't setting it up as part of their infrastructure but it isn't all that messy. The issue came up recently on the rsync list of syncing CVS repositories. The discussion concluded with:

If the history file (or another single file) is enough:

starthist=ls -l $CVSROOT/CVSROOT/history curhist=""

while [ "$Starthist" != "$curhist" ]
do
starthist=ls -l $CVSROOT/CVSROOT/history rsync .....
curhist=ls -l $CVSROOT/CVSROOT/history
done

If not you can test the directory.

ls -l $CVSROOT/CVSROOT >$TMPFILE.start
touch $TMPFILE.test

until diff -q $TMPFILE.start $TMPFILE.test >/dev/null do
ls -l $CVSROOT/CVSROOT >$TMPFILE.start rsync .....
ls -l $CVSROOT/CVSROOT >$TMPFILE.test
done
rm -f $TMPFILE.start $TMPFILE.test

Signature · 5 lines
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		[email protected]

		Remember Cernan and Schmitt
#24 lore raw
Quoted 11 lines · jw schultz

If the history file (or another single file) is enough:

starthist=ls -l $CVSROOT/CVSROOT/history curhist=""

while [ "$Starthist" != "$curhist" ]
do
starthist=ls -l $CVSROOT/CVSROOT/history rsync .....
curhist=ls -l $CVSROOT/CVSROOT/history
done

BK2CVS does not compile $CVSROOT/CVSROOT/history, but the second one should work:

Quoted 5 lines

If not you can test the directory.

ls -l $CVSROOT/CVSROOT >$TMPFILE.start
touch $TMPFILE.test

Quoted 7 lines

until diff -q $TMPFILE.start $TMPFILE.test >/dev/null do
ls -l $CVSROOT/CVSROOT >$TMPFILE.start rsync .....
ls -l $CVSROOT/CVSROOT >$TMPFILE.test
done
rm -f $TMPFILE.start $TMPFILE.test

- Davide

#25 lore raw
In reply to jw schultz
Quoted 5 lines · jw schultz

If not you can test the directory.

ls -l $CVSROOT/CVSROOT >$TMPFILE.start
touch $TMPFILE.test

Quoted 7 lines

until diff -q $TMPFILE.start $TMPFILE.test >/dev/null do
ls -l $CVSROOT/CVSROOT >$TMPFILE.start rsync .....
ls -l $CVSROOT/CVSROOT >$TMPFILE.test
done
rm -f $TMPFILE.start $TMPFILE.test

It does not work either. If CVSROOT files are updated at the end, you can still fetch some new files and be able to fetch the old CVSROOT before the update process will be able to do it. I believe that the LOCK file idea should work pretty nicely. The update process goes like:

create LOCK
update repo
remove LOCK

While the client can simply:

do
rsync
while exist LOCK

- Davide

#26 lore raw
Quoted 24 lines · Davide Libenzi
Quoted 5 lines · jw schultz

If not you can test the directory.

ls -l $CVSROOT/CVSROOT >$TMPFILE.start
touch $TMPFILE.test

Quoted 7 lines

until diff -q $TMPFILE.start $TMPFILE.test >/dev/null do
ls -l $CVSROOT/CVSROOT >$TMPFILE.start rsync .....
ls -l $CVSROOT/CVSROOT >$TMPFILE.test
done
rm -f $TMPFILE.start $TMPFILE.test

It does not work either. If CVSROOT files are updated at the end, you can still fetch some new files and be able to fetch the old CVSROOT before the update process will be able to do it. I believe that the LOCK file idea should work pretty nicely. The update process goes like:

create LOCK
update repo
remove LOCK

Quoted 5 lines

While the client can simply:

do
rsync
while exist LOCK

that can't work either, the client can't take the lock, rsync is a readonly thing. the create lock update repo, remove lock will all three run during the client rsync. the "exist LOCK" will never notice.

unless you want to use snapshots etc.. the seq lock is the only viable way to checkout coherent with rsync.

hope this isn't going too offtopic for l-k though.

#27 lore raw
In reply to Andrea Arcangeli
Quoted 13 lines · Andrea Arcangeli

It maybe also cleaner to use a slightly more complicated but more compact algorithm, this would make a potential new rsync command line option cleaner since only 1 sequence file would need to be specified:

	do {
		seq = fetch(sequence-file);
		if (seq & 1)
			break;
		rsync
		if (seq != fetch(sequence-file))
			seq = 1;
	} while (seq & 1 && sleep 10 /* ideally exponential backoff */)
Quoted 5 lines

this way only 1 sequence-file is needed for each repository that we want to checkout. the server side only has to increase twice the same file before and after each update of the repository, so the server side is even simpler (with the only additional requirement that the sequence number has to start "even"), only the client side is a bit more complicated.

For transparency, I would change the file contents to "updating" during an update, instead of the even-odd thing. I think this will make it more obvious to people how to use it properly.

Andrew

#28 lore raw
Quoted 3 lines · Andrew Pimlott

For transparency, I would change the file contents to "updating" during an update, instead of the even-odd thing. I think this will make it more obvious to people how to use it properly.

we've more solutions now. The file contents to "updating" would work too but I believe it would be by far the most complicated, the md5sum has the advantage of valiating the file contents too and it only requires 1 file update to be atomic, no matter how the upload of the data paylod happens, so I tend to like it most even if it only works probabilitsically (but it's sure safe enough).

However I understand Larry has no real interest in helping us to rsync a known to be coherent copy of the repository (very understandable from his own business standpoint), so I guess this is all wasted time, and we've to live with an heuristic like:

       1:rsync
	sleep 60
	rsync -> if something changed goto 1

I doubt Peter can provide the coherency guarantee with the md5sum on his side, unless it's Peter fetching the update of the scm data, and not the other way around.

#29 lore raw
Quoted 3 lines · Andrea Arcangeli

However I understand Larry has no real interest in helping us to rsync a known to be coherent copy of the repository (very understandable from his own business standpoint), so I guess this is all wasted time,

Rsync coherence is your problem, there is nothing I can do about that, I don't admin kernel.org. Peter has a login on kernel.bkbits.net and we can coordinate so he gets a coherent snapshot. It's trivial, he could sync the data to kernel.org at 2PM and we update the data at 2AM, I tend to think that there won't be any coherency problems.

Signature · 2 lines
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm
#30 lore raw
Quoted 5 lines · Larry McVoy

Rsync coherence is your problem, there is nothing I can do about that, I don't admin kernel.org. Peter has a login on kernel.bkbits.net and we can coordinate so he gets a coherent snapshot. It's trivial, he could sync the data to kernel.org at 2PM and we update the data at 2AM, I tend to think that there won't be any coherency problems.

if we know rsync.kernel.org is getting the update at 2PM we can safely fetch it from kernel.org at 2AM. It's not very robust and it would be hard to enforce it with the mirrors, but it should work and it's much better than nothing ;).

#31 lore raw
In reply to Andrea Arcangeli
Quoted 5 lines · Andrea Arcangeli

I doubt Peter can provide the coherency guarantee with the md5sum on his side, unless it's Peter fetching the update of the scm data, and not the other way around.

It is, actually... Larry runs the bkcvs site, but I have a login on that machine and do everything from there.

-hpa

#32 lore raw
In reply to Davide Libenzi
Quoted 8 lines · Davide Libenzi

So the update of the rsync repo should do something like:

update file1
update repo
update file2

Isn't it? I do not understand how this guarantee coherency:

Quoted 9 lines

Kernel.org Me
get file1 (old value)
update file1 get repo-file1 (old value)
update repo-file1
...
update repo-fileJ
... get repo-fileJ (new value)
update repo-fileN get file2 (old value)
update file2

The kernel.org side goes

update file2
update repo
update file1

Signature · 2 lines
David Roundy
http://civet.berkeley.edu/droundy/
#33 lore raw
In reply to Davide Libenzi
Quoted 8 lines · Davide Libenzi

So the update of the rsync repo should do something like:

update file1
update repo
update file2

Isn't it? I do not understand how this guarantee coherency:

Quoted 9 lines

Kernel.org Me
get file1 (old value)
update file1 get repo-file1 (old value)
update repo-file1
...
update repo-fileJ
... get repo-fileJ (new value)
update repo-fileN get file2 (old value)
update file2

The source (kernel.org or bkbits) would update file2 first.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#34 lore raw
In reply to Larry McVoy
Quoted 8 lines · Larry McVoy
Quoted 2 lines · H. Peter Anvin

That doesn't include anyone who uses the mirrored repository on the main kernel.org machines.

Last I checked, kernel.org isn't offering pserver access, just ftp. If you want to take over the CVS access just say the word.

No, we don't do the pserver access, but some people get the whole repository through the mirror. The current division seems to work well, so I don't see any reason to change it.

-hpa

#35 lore raw
Quoted 6 lines · H. Peter Anvin
Quoted 5 lines · Larry McVoy
Quoted 2 lines · H. Peter Anvin

That doesn't include anyone who uses the mirrored repository on the main kernel.org machines.

Quoted 7 lines
Quoted 3 lines

Last I checked, kernel.org isn't offering pserver access, just ftp. If you want to take over the CVS access just say the word.

No, we don't do the pserver access, but some people get the whole repository through the mirror. The current division seems to work well, so I don't see any reason to change it.

Except for the higher likelihood of pserver being an exploit vector.

Signature · 1 line
Matt Mackall : http://www.selenic.com : Linux development and consulting
#36 lore raw
Quoted 3 lines · Matt Mackall

Except for the higher likelihood of pserver being an exploit vector.

True... for that it would be nice to have the pserver run in an isolated environment (separate server, UML, ...) on a copy of the tree. That's more work, though...

-hpa