Skip to contentSkip to conversation

Minutes from OSDL talk at LSE call today

DISCUSSION Hanna Linder 5 participants 9 messages Dec 4, 2003 – Dec 9, 2003

lore
Share & tools
lore
#0 lore raw

LSE Call minutes from Dec 3, 2003

Zwane Mwaikambo - OSDL experiences

Generally working with the OSDL has been great.

In order to use the OSDL machines you have to sign up to be an associate and register a project. Then wait for OSDL to review it. Zwane just wanted a large marchine to work on the irq subsystem. He needed access to the NUMA-Q machines to test for regressions.

Most of his interactions have been with Christine who is fairly responsive to email and has fixed the few problems that have popped up.

Zwane has had some trouble using the PLM (Patch Lifetime Manager). Cliff is going to talk to him about using it. Zwane has been doing something similar manually.

The STP enables the engineer to save time by not having to wait inbetween each step. It automates much of the testing process.

Marcelo Tosatti - Also uses the OSDL machines. He said it has been very easy to use. They are the only high mem systems he has available now. He uses them to stress test stuff and sometimes test 2.6 on it.

Zwane said he likes the cross compilation capabilities recently added. Cliff said that was a direct result of comments from OLS this year.

Zwane said one downside of STP and Tinderbox is the huge amount of data that comes out of it and it is hard to parse. He would prefer more minimal info saying if the kernel booted or not and the some of the errors. Cliff told him to try HackBench the most minimal of tests the STP runs.

The turn around time from patch submission to results on an idle system is 1.5 hours. Due to the reimaging of the entire system between runs. Cliff is going to look into making that faster.

Bill Irwin suggested using nfs for the root partition then the host system can feed it nfs and could cleanup and check using md5checksums which would be much faster than reimaging. Cliff said most users wont use nfs on root and it may effect the benchmark results. Bill said most benchmarks arent run on root anyway. Cliff said it is a good idea and worth looking into.

Cliff White - Kernel Tinderbox

History- Came from Brazil. Christian Reis of Asynch Open Source who worked on the Mozilla Tinderbox asked Marcelo about doing a Kernel Tinderbox. Marcelo told Christian to go talk to the OSDL people and the rest is history.

The Mozilla Tinderbox is based on CVS and can do fancy things with triggers. The kernel one is not as fancy because they are still working out issues with BK.

Basically, the client runs in a loop and every 15 minutes wakes up and checks bkbits. If there are any changes within thos 15 minutes the new kernel is downloaded and compiled with John Cherry's comp regress script, which is exhaustive. The best way to see the results is to go to http://developer.osdl.org and click on the tinderbox link. or here (http://tinderbox.osdl.org/showbuilds.pl?tree=linux2.5-bk)

Intel has contributed a 32 and 64 bit client. OSDL is looking to get other architectures added to it (hint hint nudge nudge, especially Power right now).

Marcelo asked if it also boots the kernel. Cliff, no it just compiles at this point. Working on a client that will boot but the STP is best for booting an unknown kernel generally. Cliff is looking at using STP as a client of Tinderbox.

They are not doing mm or other trees as they only work off bkbits right now.

Cliff asked if anyone has Power hardware they could really use one. It doesnt have to be onsite, just need access to a machine. Zwane said they could use cross compilation to test Power stuff instead since they dont boot.

If anyone has a desire to tweak the client is available off Cliffs page on osdl: http://developer.osdl.org/cliffw/

------

minutes compiled by [email protected]

#1 lore raw
Quoted 3 lines · Hanna Linder

The Mozilla Tinderbox is based on CVS and can do fancy things with triggers. The kernel one is not as fancy because they are still working out issues with BK.

If we could get a list of these issues we'll try and see what we can do to help. My response has been a bit spotty lately, I've needed to take some personal time, so pinging [email protected] is more likely to get you help.

Signature · 2 lines
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm
#2 lore raw
Quoted 9 lines · On Wed, 3 Dec 2003 19:35:35 -0800 Larry McVoy <[email protected]> wrote:
Quoted 3 lines · Hanna Linder

The Mozilla Tinderbox is based on CVS and can do fancy things with triggers. The kernel one is not as fancy because they are still working out issues with BK.

If we could get a list of these issues we'll try and see what we can do to help. My response has been a bit spotty lately, I've needed to take some personal time, so pinging [email protected] is more likely to get you help.

We've exchanged some email with [email protected], and they've been a great help. Really, there are two things.

The first is triggers. The Mozilla tinderbox is driven by triggers from CVS commits. I believe that triggers are resevered for the commercial version of BK.

However, unlike CVS, BK has a nice way of asking the remote repository if new changes exist, so we really don't need a trigger to tell us when to start a build.
Instead, we run the timestamp column on a cron, so it is constantly incrementing. ( Mozilla only increments their timestamp column when a trigger is recieved )

The change from trigger-driven to time-driven timestamps meant we didn't try to create a link between timestamp->source commit. Again, since Mozilla is trigger driven, they have this link.

So, the second piece is linking either the time-stamp column or the build machine column directly to a Web-based view of the code. BkWeb already provides the view.

The main issue here is finding the proper syntax for the bkweb url so we get all of the changesets included in the commit. We've recieved a few examples from your support people, and we're using one currently.

cliffw

Quoted 15 lines
Signature · 13 lines
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm


-------------------------------------------------------
This SF.net email is sponsored by OSDN's Audience Survey.
Help shape OSDN's sites and tell us what you think. Take this
five minute survey and you could win a $250 Gift Certificate.
http://www.wrgsurveys.com/2003/osdntech03.php?site=8
_______________________________________________
Lse-tech mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/lse-tech
Signature · 2 lines
The church is near, but the road is icy.
The bar is far, but i will walk carefully. - Russian proverb
#3 lore raw
Quoted 1 line · * cliff white ([email protected]) wrote:

I believe that triggers are resevered for the commercial version of BK.

I don't think this is the case (I've been using them ;-). But, from what I can tell, you'd need your trigger in the main bk repo for the kernel...

thanks,
-chris

Signature · 1 line
Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net
#4 lore raw
In reply to cliff white
Quoted 13 lines · cliff white
Quoted 9 lines · On Wed, 3 Dec 2003 19:35:35 -0800 Larry McVoy <[email protected]> wrote:
Quoted 3 lines · Hanna Linder

The Mozilla Tinderbox is based on CVS and can do fancy things with triggers. The kernel one is not as fancy because they are still working out issues with BK.

If we could get a list of these issues we'll try and see what we can do to help. My response has been a bit spotty lately, I've needed to take some personal time, so pinging [email protected] is more likely to get you help.

Quoted 6 lines

We've exchanged some email with [email protected], and they've been a great help. Really, there are two things.

The first is triggers. The Mozilla tinderbox is driven by triggers from CVS commits. I believe that triggers are resevered for the commercial version of BK.

That's not true. Trigger support is identical in both versions.

Quoted 3 lines

However, unlike CVS, BK has a nice way of asking the remote repository if new changes exist, so we really don't need a trigger to tell us when to start a build.

Right. Your problem is deciding which trees you want to track. There is Linus/Marcelo trees but there are probably another 200+ trees of the kernel on bkbits.net and who knows how many elsewhere (we've counted over 10,000 before we stopped counting). Obviously you don't want to track all of those but some of them might be interesting.

Quoted 3 lines

The main issue here is finding the proper syntax for the bkweb url so we get all of the changesets included in the commit. We've recieved a few examples from your support people, and we're using one currently.

So are there any open issues? The call implied there were.

Signature · 2 lines
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm
#5 lore raw
Quoted 33 lines · On Thu, 4 Dec 2003 15:44:54 -0800 Larry McVoy <[email protected]> wrote:
Quoted 13 lines · cliff white
Quoted 9 lines · On Wed, 3 Dec 2003 19:35:35 -0800 Larry McVoy <[email protected]> wrote:
Quoted 3 lines · Hanna Linder

The Mozilla Tinderbox is based on CVS and can do fancy things with triggers. The kernel one is not as fancy because they are still working out issues with BK.

If we could get a list of these issues we'll try and see what we can do to help. My response has been a bit spotty lately, I've needed to take some personal time, so pinging [email protected] is more likely to get you help.

Quoted 6 lines

We've exchanged some email with [email protected], and they've been a great help. Really, there are two things.

The first is triggers. The Mozilla tinderbox is driven by triggers from CVS commits. I believe that triggers are resevered for the commercial version of BK.

That's not true. Trigger support is identical in both versions.

Quoted 3 lines

However, unlike CVS, BK has a nice way of asking the remote repository if new changes exist, so we really don't need a trigger to tell us when to start a build.

Right. Your problem is deciding which trees you want to track. There is Linus/Marcelo trees but there are probably another 200+ trees of the kernel on bkbits.net and who knows how many elsewhere (we've counted over 10,000 before we stopped counting). Obviously you don't want to track all of those but some of them might be interesting.

Quoted 5 lines
Quoted 3 lines

The main issue here is finding the proper syntax for the bkweb url so we get all of the changesets included in the commit. We've recieved a few examples from your support people, and we're using one currently.

So are there any open issues? The call implied there were.

I don't think there are any open issues for BK. The tinderbox crew still has some work to do.
cliffw

Quoted 4 lines
Signature · 2 lines
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm
Signature · 2 lines
The church is near, but the road is icy.
The bar is far, but i will walk carefully. - Russian proverb
#7 lore raw
Quoted 11 lines · Zwane Mwaikambo
Quoted 5 lines · Larry McVoy
Quoted 3 lines

The first is triggers. The Mozilla tinderbox is driven by triggers from CVS commits. I believe that triggers are resevered for the commercial version of BK.

That's not true. Trigger support is identical in both versions.

Perhaps the FAQ may need updating then;

http://www.bitkeeper.com/Documentation.FAQS.Event.html

Indeed.

It's worth pointing out that triggers in open source trees are quite a bit more dangerous than in controlled environment. Carl-Daniel's boss made quite a fuss over the fact that triggers are just programs that are run and can be used to cause all sorts of problems if people were malicious.

I've toyed with the idea of disabling triggers in openlogging trees because of this. I'm neutral on the topic, it's not like triggers are some huge money maker that we need to reserve for the commercial version. If the general feeling is that triggers are useful and people will take responsibility for policing their own repos then we'll leave them in. On the other hand, if someone putting a nasty trigger into your tree somehow becomes the fault of BitMover because we provided the infrastructure then out they go in the next release.

Signature · 2 lines
---
Larry McVoy              lm at bitmover.com          http://www.bitmover.com/lm
#8 lore raw
Quoted 5 lines · Larry McVoy

It's worth pointing out that triggers in open source trees are quite a bit more dangerous than in controlled environment. Carl-Daniel's boss made quite a fuss over the fact that triggers are just programs that are run and can be used to cause all sorts of problems if people were malicious.

Quoted 8 lines

I've toyed with the idea of disabling triggers in openlogging trees because of this. I'm neutral on the topic, it's not like triggers are some huge money maker that we need to reserve for the commercial version. If the general feeling is that triggers are useful and people will take responsibility for policing their own repos then we'll leave them in. On the other hand, if someone putting a nasty trigger into your tree somehow becomes the fault of BitMover because we provided the infrastructure then out they go in the next release.

I personally find them very useful, perhaps we should just stick with the other methods of disabling said functionality.

Thanks