Skip to contentSkip to conversation

Recruitment (Reviewers, Testers, Maintainers, Hobbyists)

Ksummit-discuss Peter Huewe 54 participants 359 messages Jul 7, 2015 – Aug 5, 2015 Page 2 of 2

lore
Share & tools
lore
#266 lore raw
In reply to Peter Hüwe
Quoted 18 lines · Peter Hüwe

Am Mittwoch, 8. Juli 2015, 04:03:04 schrieb Krzysztof Kozłowski:

Quoted 6 lines

Before doing some work there is always a cause, an answer to "why I am doing this"? Employer may pay for my commits but would he pay for reviewing time? That is his decision and it would be difficult to change policies inside companies.

Quoted 8 lines

Other reason for doing open source work may be the fame. Being recognizable, getting better job offers, doing tasks which are sensible and meaningful for someone. Currently probably most of the fame goes to authors and maintainers. For example in the form of git log --author/committer= or LWN articles about statistics.

How to get more reviews from such people (when employer does not pay for it)? Give them fame! :)

Exactly! This is also what Rafael wrote in the other mail:

Quoted 8 lines
Quoted 3 lines

Most of the time there's a little to no recognition for doing that work and, quite frankly, writing code is more rewarding than that for the majority of people anyway.

So changing our fame-statistics from commits to reviews and tested by might change the situation a bit. -> The next LWN stats and coverage should probably focus on the reviewed-by /
tested-by stats. People love to be on some "top 10" lists - and also they can show something like that to their bosses.

I think we also need to encourage multiple sign offs from one person
on drivers sometimes. Often you'll get a 'I tested this and also
reviewed it' but as I'm the author / maintainer of the original driver I'll Ack it. Perhaps these cases should have all 3 tags in the commit.

Quoted 3 lines

"I've been a kernel reviewer and tester" -- meh, who cares
"I've been a top 100 kernel reviewer and tester over the last X releases" -- give him a raise/the job (esp. if kernel is not the core competency of the company :)

I'm very luck in IIO in that I have a core set of very good reviewers with (I think) a mix of paid and volunteers. As a volunteer Maintainer I couldn't cope with the rate of patches without them!

One thing that happens fairly often is that I get a very good initial review in then the reviewer moves on to other patches. I suspect this is because they are focusing on where they can do the most productive work. Sometimes I go to the effort of chasing them up for an ack / reviewed by, but often they get no recognition at all.

Note I tend to do an additional review (often v3 or later by the time I get to them) but these guys have done a lot of the leg work. One of my reviewers specializes in very detailed reviews slightly after I have applied the patches, but that's another story :)

So how to give these incredibly helpful people more recognition? Do other maintainers add reviewed-by tags sometimes without the reviewer specifically giving them? The docs have always said these should indicate that the reviewer is happy with the result. In this case the reviewer may not have looked at the result, but contributed earlier in the process.

Does anyone else actually get these sorts of reviews?

Quoted 14 lines

Another thing I noticed over the last few years (also in corporate), people get really motivated by memorabilia - "tokens of appreciation".
E.g. I constantly wear my Google T-Shirt which I received for a contribution with such proud and so often that it is almost faded --- but still everytime I look at it I have a good feeling.

--> Maybe LF can organize something?
"Here is a small token of appreciation (t-shirt, cup) for spending countless hours on reviewing and testing stuff in the Linux kernel -- keep up the good work"

Quoted 7 lines
Quoted 3 lines

The only way to address this problem I can see is to recognize reviewers *much* more than we tend to do and not just "encourage" them, because that's way insufficient.

Yes again!

What I definitely would also recommend is to organize some 'get togethers',
like a miniconf/minisummit at the next conference near you -- and where you grab a beer _together_ with the reviewers / testers afterwards (and maybe the maintainer can pay). This also helps as forms of appreciation.

I'm particularly bad at this. Have only ever actually met one of my regular reviewers (out of 6+). Note that for some subsystems the conference attendance is actually pretty limited / random and chances of a actually encountering many of the reviewers is lowish.

Guys who put in a few hours a week are the bread and butter or reviewing for my area, but that may well be all the time they have as much as they'd like to travel to conferences!

Jonathan

#267 lore raw

Hey,

Quoted 15 lines

I'm very luck in IIO in that I have a core set of very good reviewers with (I think) a mix of paid and volunteers. As a volunteer Maintainer I couldn't cope with the rate of patches without them!

One thing that happens fairly often is that I get a very good initial review in then the reviewer moves on to other patches. I suspect this is because they are focusing on where they can do the most productive work. Sometimes I go to the effort of chasing them up for an ack / reviewed by, but often they get no recognition at all.

Note I tend to do an additional review (often v3 or later by the time I get to them) but these guys have done a lot of the leg work. One of my reviewers specializes in very detailed reviews slightly after I have applied the patches, but that's another story :)

Quoted 7 lines

So how to give these incredibly helpful people more recognition? Do other maintainers add reviewed-by tags sometimes without the reviewer
specifically giving them? The docs have always said these should indicate that the reviewer is happy with the result. In this case the reviewer may not have looked at the result, but contributed earlier in the process.

Actually I do, at least if I have the feeling that it would be in the reviewers intention. Credit where credit is due.

If I'm unsure I mail the reviewers and ask them (or via IRC)

Quoted 1 line

Does anyone else actually get these sorts of reviews?

Yes happens a lot - v1-5 get a lot of reviews , v8 gets none except my own :)
But the important stuff is usually dealt with in v1-3. so I usually add the reviewed-by tags if nothing fundamentally has changed So the Situation is much likr in iio
(Yes in TPM we usually need v5 or more:)

Peter

Signature · 1 line
Sent from my mobile
#268 lore raw
In reply to Peter Hüwe
Quoted 4 lines · On Wed, 8 Jul 2015 09:04:58 +0200 Peter Hüwe <[email protected]> wrote:

So changing our fame-statistics from commits to reviews and tested by might change the situation a bit.
-> The next LWN stats and coverage should probably focus on the reviewed-by / tested-by stats.

I do occasionally report those numbers with the rest. The reason it doesn't happen very often is simple: it's a small minority of patches that carry such tags. As such, I don't think the numbers actually reflect the reviewing and testing going on; they just reflect the placement of the tags. Among other things, that makes the system relatively easy to game, and, in the past, I've seen that happening.

Should anybody be curious, I'll append the latest, hot-off-the-repo numbers for 4.2.

jon

[12,099 non-merge changesets total when I ran this]

Developers with the most reviews (total 2597)

Ian Abbott                 149 (5.7%)
Borislav Petkov            148 (5.7%)
Hannes Reinecke            100 (3.9%)
Christoph Hellwig           73 (2.8%)
H Hartley Sweeten           62 (2.4%)
Christian König             57 (2.2%)
Tomas Henzl                 54 (2.1%)
Alex Deucher                51 (2.0%)
David Gibson                49 (1.9%)
Scott Teel                  42 (1.6%)

Developers with the most test credits (total 775)

Joerg Roedel                35 (4.5%)
Keita Kobayashi             35 (4.5%)
Krishneil Singh             31 (4.0%)
Arnaldo Carvalho de Melo    30 (3.9%)
Ira Weiny                   23 (3.0%)
Doug Ledford                23 (3.0%)
Aaron Brown                 21 (2.7%)
Alex Ng                     21 (2.7%)
Javier Martinez Canillas    19 (2.5%)

Developers who gave the most tested-by credits (total 775)

Michael Wang                46 (5.9%)
Joerg Roedel                42 (5.4%)
Kuninori Morimoto           36 (4.6%)
Jiang Liu                   35 (4.5%)
Mel Gorman                  31 (4.0%)
Vitaly Kuznetsov            21 (2.7%)
Andre Przywara              20 (2.6%)
Marek Szyprowski            19 (2.5%)
Lv Zheng                    17 (2.2%)
Don Skidmore                17 (2.2%)

Developers with the most report credits (total 467)

Fengguang Wu                65 (13.9%)
Dan Carpenter               26 (5.6%)
Russell King                21 (4.5%)
Ingo Molnar                 10 (2.1%)
Stephen Rothwell             9 (1.9%)
Christoph Hellwig            5 (1.1%)
Kevin Hilman                 5 (1.1%)
Sergey Senozhatsky           5 (1.1%)
Jim Davis                    5 (1.1%)
Arnaldo Carvalho de Melo     4 (0.9%)

Developers who gave the most report credits (total 467)

Thomas Gleixner             30 (6.4%)
Paul E. McKenney            14 (3.0%)
Lv Zheng                    12 (2.6%)
Eric Dumazet                 8 (1.7%)
Paul Gortmaker               8 (1.7%)
Peter Zijlstra               8 (1.7%)
Takashi Iwai                 7 (1.5%)
Stephen Boyd                 7 (1.5%)
Aleksey Makarov              7 (1.5%)
Filipe Manana                6 (1.3%)
#269 lore raw
In reply to Peter Hüwe
Quoted 24 lines · Peter Hüwe

Am Mittwoch, 8. Juli 2015, 04:03:04 schrieb Krzysztof Kozłowski:

Quoted 6 lines

Before doing some work there is always a cause, an answer to "why I am doing this"? Employer may pay for my commits but would he pay for reviewing time? That is his decision and it would be difficult to change policies inside companies.

Quoted 8 lines

Other reason for doing open source work may be the fame. Being recognizable, getting better job offers, doing tasks which are sensible and meaningful for someone. Currently probably most of the fame goes to authors and maintainers. For example in the form of git log --author/committer= or LWN articles about statistics.

How to get more reviews from such people (when employer does not pay for it)? Give them fame! :)

Exactly!
This is also what Rafael wrote in the other mail:

Quoted 3 lines

Most of the time there's a little to no recognition for doing that work and, quite frankly, writing code is more rewarding than that for the majority of people anyway.

Quoted 6 lines

So changing our fame-statistics from commits to reviews and tested by might change the situation a bit.
-> The next LWN stats and coverage should probably focus on the reviewed-by / tested-by stats.
People love to be on some "top 10" lists - and also they can show something like that to their bosses.

gack. see below.

Quoted 1 line

"I've been a kernel reviewer and tester" -- meh, who cares

Show me a concrete example where that has been expressed. I've had the exact opposite experience. Typically I've had to tone down the expectations of others.

Quoted 3 lines

"I've been a top 100 kernel reviewer and tester over the last X releases" -- give him a raise/the job (esp. if kernel is not the core competency of the company :)

Gah! Please don't add gamification into this. It attracts the wrong sorts of people. We already have a large swath of society that want a cookie and a pat on the head for doing the most mundane things.

Plus these systems built with good intentions always go awry. "Hey! You didn't add a tag for me because I spotted a typo!" <stomps and huffs> Or, "My boss / HR gave me less of a raise because my stats dropped this quarter, wtf?"

This is a path we do *not* want to walk down.

Quoted 4 lines

--> Maybe LF can organize something?
"Here is a small token of appreciation (t-shirt, cup) for spending countless hours on reviewing and testing stuff in the Linux kernel -- keep up the good work"

How about "Here's an invite to the KS, because we value your opinion and would like you to take part in the process." ;-)

Quoted 4 lines
Quoted 3 lines

The only way to address this problem I can see is to recognize reviewers *much* more than we tend to do and not just "encourage" them, because that's way insufficient.

Yes again!

Disagree. But I've said my piece above.

Quoted 4 lines

What I definitely would also recommend is to organize some 'get togethers', like a miniconf/minisummit at the next conference near you -- and where you grab a beer _together_ with the reviewers / testers afterwards (and maybe the maintainer can pay).

Enhancing the community effect is something I can get on board with. Buying a few rounds of drinks goes a long way. Getting the folks together in meat-space is the hard ($$) part.

thx,

Jason.

#270 lore raw
Quoted 8 lines · Jason Cooper
Quoted 1 line · Peter Hüwe

"I've been a kernel reviewer and tester" -- meh, who cares

Show me a concrete example where that has been expressed. I've had the exact opposite experience. Typically I've had to tone down the expectations of others.

Lucky you.

In my experience, code reviews often don't get much attention by project managers responsible for commercial projects. If anything, quite the opposite. The Linux kernel development culture is quite different.

Linux kernel:
"Thanks a lot for reviewing my code and making it better".
Commercial:
"Why do your reviews always take such a long time ? You are holding up the release!"

Granted, that doesn't happen everywhere, but I have seen it more than once, and I would content that this kind of culture is pretty widely spread.

Anyone entering such an environment might actually encounter negative reactions to "I am a kernel reviewer and tester". Of course, one might argue that joining an affected company might not be a good idea in the first place, but not everyone has that much flexibility or choice.

I am not saying that it would be a bad idea to give code reviewers more credit; quite the contrary. However, I think it would be wrong to assume or expect that giving reviewers more credit would improve their chances for employment. It would, however, result in reviewers feeling recognized for themselves, which is a good thing.

Why does it matter ? I had a discussion a while ago with the founder of one of the web sites providing product reviews. I suggested to give product reviewers some kind of monetary reward for their reviews, eg part of the advertising revenue created by the site. Feedback I got was that reviewers typically don't care about monetary rewards, but they do care about recognition and about their standing in the community.

Guenter

#271 lore raw
Quoted 10 lines · Guenter Roeck
Quoted 8 lines · Jason Cooper
Quoted 1 line · Peter Hüwe

"I've been a kernel reviewer and tester" -- meh, who cares

Show me a concrete example where that has been expressed. I've had the exact opposite experience. Typically I've had to tone down the expectations of others.

Lucky you.

Well, my point wasn't to brag, but I think you knew that.

I'd hope that my experience isn't isolated, and if it is, why?

Quoted 9 lines

In my experience, code reviews often don't get much attention by project managers responsible for commercial projects. If anything, quite the opposite. The Linux kernel development culture is quite different.

Linux kernel:
"Thanks a lot for reviewing my code and making it better".
Commercial:
"Why do your reviews always take such a long time ? You are holding up the release!"

These are responses to reviews during employment. I was referring to, perhaps not clearly, the hiring process.

Quoted 2 lines

Granted, that doesn't happen everywhere, but I have seen it more than once, and I would content that this kind of culture is pretty widely spread.

wrt actual code reviews, I agree.

Quoted 3 lines

Of course, one might
argue that joining an affected company might not be a good idea in the first place, but not everyone has that much flexibility or choice.

When interviewing potential engineers, folks who demonstrate regular, real contributions to open source projects move up a notch or two in my book. Because the applicant has then demonstrated an ability to work with others, at a distance, for the good of the project.

iow, working in open source projects demonstrates you know how to telecommute. For companies looking to hire engineers into telecommuting positions, open source contributors are much lower risk.

What percentage of the overall market is that? I dunno. But devs wanting to live in odd places have a better shot of finding jobs by becoming regular contributors to open source.

Quoted 5 lines

I am not saying that it would be a bad idea to give code reviewers more credit; quite the contrary. However, I think it would be wrong to assume or expect that giving reviewers more credit would improve their chances for employment. It would, however, result in reviewers feeling recognized for themselves, which is a good thing.

I think we can get more results for our efforts by bringing folks in to community events. Buy beers and so forth. I'll never forget the evening in San Diego drinking blended single-malt (?!) with Olof, Tony and others.

If I enjoy working with someone, I take them out for beers, spend *time* with them. This works a lot better that adding a bullet to an eval.

Quoted 6 lines

Why does it matter ? I had a discussion a while ago with the founder of one of the web sites providing product reviews. I suggested to give product reviewers some kind of monetary reward for their reviews, eg part of the advertising revenue created by the site. Feedback I got was that reviewers typically don't care about monetary rewards, but they do care about recognition and about their standing in the community.

Fully agree. And it argues my point, things which can be counted and accrued (money, tokens, Reviewed-by [1], etc) rarely have the desired affect. At the end of the day, there's no substitute for making genuine gestures. Invites to KS or other events, dinner, drinks, socialization.

The problem with doing that is it costs money. Who pays? Does the 17 year old in Poland have the money to fly to $event?

If we agree that:

a) increasing the opportunities for trial run contributors/reviewers
b) developing community by getting together regularly
c) collecting statistics to determine program efficacy

Are good targets, then I'd like to discuss putting together a program/proposal that we could take to LF. Something quantifiable, e.g. company X can 'sponsor' 5 newcomers over the next year for $Y. It includes, per newcomer, a travel allowance for one trip and a small sum for a piece of gear. Gear issuance is determined by the co-maintainer/established kernel dev working with the newcomer.

Events could be small, 2 day regional affairs, or 1 big annual get together. Or both. If we go with small and regional, we would need to include the cost of flying the relevant established devs to the event. Regional events could be hosted by Linux-friendly companies to reduce cost.

thx,

Jason.

[1] It's worth noting that I view the tags as a *responsibility*, not a reward. It informs a bug hunter which devs had a hand it creation / merging of the blamed commit. Overloading the tags has been discussed before, with much resistance.

#272 lore raw
In reply to Jason Cooper

Hey Jason

Quoted 7 lines
Quoted 4 lines

--> Maybe LF can organize something?
"Here is a small token of appreciation (t-shirt, cup) for spending countless hours on reviewing and testing stuff in the Linux kernel -- keep up the good work"

How about "Here's an invite to the KS, because we value your opinion and would like you to take part in the process." ;-)

Personally I think that's also a valuable approach - but since KS space is limited maybe not so feasible. T-Shirts etc are easier :)

Peter

#273 lore raw
Quoted 7 lines · Peter Huewe

Hey Jason

Quoted 4 lines
Quoted 4 lines

--> Maybe LF can organize something?
"Here is a small token of appreciation (t-shirt, cup) for spending countless hours on reviewing and testing stuff in the Linux kernel -- keep up the good work"

Quoted 5 lines
Quoted 2 lines

How about "Here's an invite to the KS, because we value your opinion and would like you to take part in the process." ;-)

Personally I think that's also a valuable approach - but since KS space is limited maybe not so feasible. T-Shirts etc are easier :)

Right, later on I said (or in another mail) "KS or other event".

T-shirts are a given. Coffee mugs, thumbdrives that are modifiable a-la bad-usb. It's all good. ;-)

The chromecasts that were given out two years ago directly resulted in mach-berlin/ ... So maybe we should be giving out HW that *isn't* supported by mainline.

thx,

Jason.

#274 lore raw
Quoted 4 lines

The chromecasts that were given out two years ago directly resulted in mach-berlin/ ...
So maybe we should be giving out HW that *isn't*
supported by mainline.

+1 - also a nice idea. :)

Thanks,
Peter

#275 lore raw
In reply to Peter Hüwe
Quoted 5 lines · Peter Hüwe

Am Mittwoch, 8. Juli 2015, 04:03:04 schrieb Krzysztof Kozłowski:

Quoted 3 lines

The only way to address this problem I can see is to recognize reviewers *much* more than we tend to do and not just "encourage" them, because that's way insufficient.

Quoted 7 lines

Yes again!

What I definitely would also recommend is to organize some 'get togethers', like a miniconf/minisummit at the next conference near you -- and where you grab a beer _together_ with the reviewers / testers afterwards (and maybe the maintainer can pay).
This also helps as forms of appreciation.

I think that this was a major purpose served by Kernel Summit when it was founded. Over time KS grew along with the importance of the kernel until we started trying to cap KS attendance in the "about 100" range.

Various mini-summits, workshops, and such have been instituted over time. At least part of the goal of providing such outlets is to include a broader swath of the kernel community. I believe that was part (perhaps most) of the motivation for the KS format change to include workshops over the past several years as well. Do you think that we need even more such events? Or just bigger ones?

John

Signature · 2 lines
John W. Linville		Someday the world will need a hero, and you
[email protected]			might be all we have.  Be ready.
#276 lore raw
In reply to Peter Hüwe
Quoted 50 lines · On Wednesday 08 July 2015 09:04:58 Peter Hüwe wrote:

Am Mittwoch, 8. Juli 2015, 04:03:04 schrieb Krzysztof Kozłowski:

Quoted 5 lines

Before doing some work there is always a cause, an answer to "why I am doing this"? Employer may pay for my commits but would he pay for reviewing time? That is his decision and it would be difficult to change policies inside companies.

Quoted 8 lines

Other reason for doing open source work may be the fame. Being recognizable, getting better job offers, doing tasks which are sensible and meaningful for someone. Currently probably most of the fame goes to authors and maintainers. For example in the form of git log --author/committer= or LWN articles about statistics.

How to get more reviews from such people (when employer does not pay for it)? Give them fame! :)

Exactly!

This is also what Rafael wrote in the other mail:

Quoted 3 lines

Most of the time there's a little to no recognition for doing that work and, quite frankly, writing code is more rewarding than that for the majority of people anyway.

So changing our fame-statistics from commits to reviews and tested by might change the situation a bit.
-> The next LWN stats and coverage should probably focus on the reviewed-by / tested-by stats.
People love to be on some "top 10" lists - and also they can show something like that to their bosses.

"I've been a kernel reviewer and tester" -- meh, who cares
"I've been a top 100 kernel reviewer and tester over the last X releases" -- give him a raise/the job (esp. if kernel is not the core competency of the company :)

Another thing I noticed over the last few years (also in corporate), people get really motivated by memorabilia - "tokens of appreciation". E.g. I constantly wear my Google T-Shirt which I received for a contribution with such proud and so often that it is almost faded --- but still everytime I look at it I have a good feeling.

--> Maybe LF can organize something?
"Here is a small token of appreciation (t-shirt, cup) for spending countless hours on reviewing and testing stuff in the Linux kernel -- keep up the good work"

Quoted 3 lines

The only way to address this problem I can see is to recognize reviewers *much* more than we tend to do and not just "encourage" them, because that's way insufficient.

Quoted 7 lines

Yes again!

What I definitely would also recommend is to organize some 'get togethers', like a miniconf/minisummit at the next conference near you -- and where you grab a beer _together_ with the reviewers / testers afterwards (and maybe the maintainer can pay).
This also helps as forms of appreciation.

While real life meetings are invaluable, let's not forgot that they also have a major drawback: not everybody can attend them. In a very distributed development environment like the Linux kernel contributors come from many different horizons, and not all of them can afford to attend conferences (or sometimes just don't want to, for various reasons). If we focus too much on a groups of contributors who can meet in real life we'll alienate the rest of the "online" crowd, and risk losing contributors who will feel left out.

Signature · 3 lines
Regards,

Laurent Pinchart
#277 lore raw
In reply to Josh Boyer
Quoted 4 lines · Josh Boyer

You could make a Reviewed-by tag required before a patch can be included in a submaintainer's tree. At least some maintainers seem to (arbitrarily?) require this at times. However, if you do that then it would likely slow down development quite a bit.

It doesn't seem to have slowed down the rate of change for the subsystems that currently require this, so why do you think it would?

Quoted 2 lines

Then Greg might cry because he wouldn't get to show pretty graphs at conferences about how fast the rate of change is in the kernel.

I don't have any graphs showing rate of change. I have a few "raw" numbers that I talk about, but that's just to scare people. Even if we went back to 2.5 development speeds (2.5 changes per hour), that's enough to scare people for my needs :)

Quoted 2 lines

(I would love to see a graph comparing rate of change to rate of regressions/bugs, but then people would have to know the latter.)

Want to start tracking that? We've needed someone to do that work for quite some time now, the fact that we've gotten by without it either means that no one sees the value in funding such a position and/or it's not really something that anyone cares about...

thanks,

greg k-h

#278 lore raw
Quoted 8 lines · Greg KH
Quoted 4 lines · Josh Boyer

You could make a Reviewed-by tag required before a patch can be included in a submaintainer's tree. At least some maintainers seem to (arbitrarily?) require this at times. However, if you do that then it would likely slow down development quite a bit.

It doesn't seem to have slowed down the rate of change for the subsystems that currently require this, so why do you think it would?

That's SCSI and who else?

I think it improved SCSI a lot and got more people involved. Before there were some companies that didn't reply at all when you emailed them fixes for their driver. They just left everything for James. Adding the rule that every patch needed two reviews probably sped things up.

regards,
dan carpenter

#279 lore raw
In reply to Greg KH
Quoted 9 lines · Greg KH
Quoted 4 lines · Josh Boyer

You could make a Reviewed-by tag required before a patch can be included in a submaintainer's tree. At least some maintainers seem to (arbitrarily?) require this at times. However, if you do that then it would likely slow down development quite a bit.

It doesn't seem to have slowed down the rate of change for the subsystems that currently require this, so why do you think it would?

I think there's probably a cause and effect thing going on there - it's more likely that a subsystem will start to impose such requirements if they are confident that it is practical to get reviews of sufficient quality in. I'd guess at least some of the time it's a formalisation of existing practice rather than a completely new requirement.

Quoted 7 lines
Quoted 2 lines

(I would love to see a graph comparing rate of change to rate of regressions/bugs, but then people would have to know the latter.)

Want to start tracking that? We've needed someone to do that work for quite some time now, the fact that we've gotten by without it either means that no one sees the value in funding such a position and/or it's not really something that anyone cares about...

Well, first we'd need to find a way of counting the bugs and regressions.

#280 lore raw
In reply to Greg KH
Quoted 8 lines · Greg KH
Quoted 4 lines · Josh Boyer

You could make a Reviewed-by tag required before a patch can be included in a submaintainer's tree. At least some maintainers seem to (arbitrarily?) require this at times. However, if you do that then it would likely slow down development quite a bit.

It doesn't seem to have slowed down the rate of change for the subsystems that currently require this, so why do you think it would?

Probably because such subsystems have always required it? If it were a new imposition on a subsystem, I suspect it would certainly impact things, at least at first.

Quoted 7 lines
Quoted 2 lines

Then Greg might cry because he wouldn't get to show pretty graphs at conferences about how fast the rate of change is in the kernel.

I don't have any graphs showing rate of change. I have a few "raw" numbers that I talk about, but that's just to scare people. Even if we went back to 2.5 development speeds (2.5 changes per hour), that's enough to scare people for my needs :)

Heh.

Quoted 7 lines
Quoted 2 lines

(I would love to see a graph comparing rate of change to rate of regressions/bugs, but then people would have to know the latter.)

Want to start tracking that? We've needed someone to do that work for quite some time now, the fact that we've gotten by without it either means that no one sees the value in funding such a position and/or it's not really something that anyone cares about...

I think you're wrong on both counts there, sorry.

In order to track this well, you need data from users. And therein lies one of the problems. The majority of users don't use kernel.org kernels. They use distro kernels. The distros have data and tools to help track bugs and regressions, but upstream is somewhat loathe to look at anything that starts with b and ends with zilla. Why? Because the data coming from users is often utterly junk. You get a kernel splat and a "I don't know why this happened." And frankly, I don't expect them to know why it happened either. Particularly when you have subsystems that are using WARN_ON as a fixme comment and sprinkling them all over the damn place.

So you have users that don't/are afraid to talk to upstream, distro maintainers trying to play middle men and being overwhelmed, and upstream who is very responsive when contacted directly but because they use self-built kernels the request is usually "bisect". End users can't bisect without hand-holding, so the distro maintainers are left doing that too. The learning curve for creating a good bug report is pretty steep, and the ability to help debug it is even steeper.

Now that isn't to say that data doesn't exist. As I alluded to, the various bugzillas are full of regressions and bugs. But there is no aggregation of that data across instances, and everything is terrible. I literally spent a year doing very little other than bug triage and random fixing. Fedora's bug count for the kernel went from over 1000 bugs to just under 500. That sounds great, but it really isn't. The net result wasn't 500 bugfixes. I don't have 500 commits in the kernel, nor 500 Reviewed-by or Tested-by or Reported-by tags. It was literally cleanup of bugzilla, not kernel issues. (To be sure, some issues were fixes for the users, but those were rare.) And this was with Fedora, which sticks very very closely to the latest upstream release.

Ignoring the bug reporting pipeline problem for a minute, there is other data as well. The new Fixes: tag could easily be scripted to count as a bug/regression. Except it is fairly new, and isn't as widely used as one would like yet. That is pretty trivial to count though and I don't think it would paint an accurate picture at all.

I intended this to be short, but it wasn't. Hopefully it didn't come off as a big rant. I simply think the problem is a lot more complicated that "no one sees value or no one cares." It is really signal to noise, a huge backlog, and figuring out how to get the end users working with the developers without needed a distro translator. Or getting the developers working the other way, but when has that ever happened in the software industry?

josh

#281 lore raw
Quoted 11 lines · On Wed, 8 Jul 2015 07:43:25 -0400 Josh Boyer <[email protected]> wrote:

In order to track this well, you need data from users. And therein lies one of the problems. The majority of users don't use kernel.org kernels. They use distro kernels. The distros have data and tools to help track bugs and regressions, but upstream is somewhat loathe to look at anything that starts with b and ends with zilla. Why? Because the data coming from users is often utterly junk. You get a kernel splat and a "I don't know why this happened." And frankly, I don't expect them to know why it happened either. Particularly when you have subsystems that are using WARN_ON as a fixme comment and sprinkling them all over the damn place.

Just a note. I hate the use of WARN_ON()s in this case. I bitch quite loudly when I stumble across them (and I do often), because my ktest scripts I use to test my own patches will fail if a WARN_ON() is triggered.

At least if my test boxes have the hardware that does this, it wont last long, as I'm pretty good at bitching ;-)

-- Steve

#282 lore raw
Quoted 15 lines · Steven Rostedt
Quoted 11 lines · On Wed, 8 Jul 2015 07:43:25 -0400 Josh Boyer <[email protected]> wrote:

In order to track this well, you need data from users. And therein lies one of the problems. The majority of users don't use kernel.org kernels. They use distro kernels. The distros have data and tools to help track bugs and regressions, but upstream is somewhat loathe to look at anything that starts with b and ends with zilla. Why? Because the data coming from users is often utterly junk. You get a kernel splat and a "I don't know why this happened." And frankly, I don't expect them to know why it happened either. Particularly when you have subsystems that are using WARN_ON as a fixme comment and sprinkling them all over the damn place.

Quoted 7 lines

Just a note. I hate the use of WARN_ON()s in this case. I bitch quite loudly when I stumble across them (and I do often), because my ktest scripts I use to test my own patches will fail if a WARN_ON() is triggered.

At least if my test boxes have the hardware that does this, it wont last long, as I'm pretty good at bitching ;-)

Are you test boxes headless? Because i915 is littered with WARN and WARN_ON, to the point where we carry a patch in Fedora to quiet most of the state machine related ones. If you look at retrace and filter out all the proprietary oopses, i915 is always most of the top 10 kernel issues Fedora sees.

https://retrace.fedoraproject.org/faf/problems/?component_names=&associate=__None&daterange=2015-06-24%3A2015-07-08&exclude_taintflags=7&exclude_taintflags=9&exclude_taintflags=5&bug_filter=None&function_names=&binary_names=&source_file_names=&since_version=&since_release=&to_version=&to_release=#

josh

#283 lore raw
Quoted 8 lines · On Wed, 8 Jul 2015 15:11:32 -0400 Josh Boyer <[email protected]> wrote:
Quoted 2 lines

At least if my test boxes have the hardware that does this, it wont last long, as I'm pretty good at bitching ;-)

Are you test boxes headless? Because i915 is littered with WARN and WARN_ON, to the point where we carry a patch in Fedora to quiet most of the state machine related ones. If you look at retrace and filter out all the proprietary oopses, i915 is always most of the top 10 kernel issues Fedora sees.

Sadly, my bitching skills didn't work well here:

https://lkml.org/lkml/2015/5/7/867

Maybe I let it slide. ktest allows you to run PRE_BUILD scripts, and what I've gotten use to doing was adding:

PRE_BUILD = patch -p1 < fix.patch
POST_BUILD = git reset --hard

Where that fix.patch would include silencing those damn warnings.

As I was under the impression that this was unique to my test box (which happens to be a development machine).

Quoted 2 lines

https://retrace.fedoraproject.org/faf/problems/?component_names=&associate=__None&daterange=2015-06-24%3A2015-07-08&exclude_taintflags=7&exclude_taintflags=9&exclude_taintflags=5&bug_filter=None&function_names=&binary_names=&source_file_names=&since_version=&since_release=&to_version=&to_release=#

Maybe I should tone my bitching skills again. This time with some backup ;-)

-- Steve

#284 lore raw
In reply to Josh Boyer
Quoted 10 lines · Josh Boyer
Quoted 3 lines

The only way to address this problem I can see is to recognize reviewers *much* more than we tend to do and not just "encourage" them, because that's way insufficient.

You could make a Reviewed-by tag required before a patch can be included in a submaintainer's tree. At least some maintainers seem to (arbitrarily?) require this at times. However, if you do that then it would likely slow down development quite a bit. Then Greg might cry because he wouldn't get to show pretty graphs at conferences about how fast the rate of change is in the kernel.

It's far too common to send out a patch series, receive lots of valuable comments and suggestions, adapt according to feedback, and send out again (repeat a few cycles). Suddenly the patches looks perfect, no more comments, but also no Acked-by or Reviewed-By. Just silence.

Requiring a Reviewed-by tag is not a solution, unless there's a real incentive for people to add such tags. Finding such an incentive can be difficult, and may turn out to make matters worse (cfr. patent officers paid per patent granted).

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#285 lore raw
In reply to Rafael J. Wysocki
Quoted 18 lines · Rafael J. Wysocki
Quoted 10 lines · Peter Huewe

Hi,

In order to continue our traditions I would like to propose again the topic of recruitment, but this time not only limiting to the hobbyists market.

We are definitely short on reviewers and thus have mostly overloaded maintainers.
For testers it's usually even worse - how many patches are actually tested? Judging from what I read on LKML not that many.

Quoted 6 lines

So we should definitely discuss:
- how can we encourage hobbyists to become regular contributors -- how to keep people interested, the drop-out rates are huge.
- encourage regular contributors to become reviewers and testers
- reviewers to become co-maintainers and finally maintainers (once the original maintainer is used up or moves up/on)

Quoted 4 lines

Good topic.

Unfortunately, there are not too many incentives for people to become code reviewers or testers, or at least to spend more time reviewing patches.

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Quoted 7 lines

Most of the time there's a little to no recognition for doing that work and, quite frankly, writing code is more rewarding than that for the majority of people anyway.

The only way to address this problem I can see is to recognize reviewers *much* more than we tend to do and not just "encourage" them, because that's way insufficient.

What other incentives or recognition mechanisms would you propose?

James

#286 lore raw
Quoted 19 lines · James Bottomley
Quoted 18 lines · Rafael J. Wysocki
Quoted 8 lines · Peter Huewe

Hi,

In order to continue our traditions I would like to propose again the topic of recruitment, but this time not only limiting to the hobbyists market.

We are definitely short on reviewers and thus have mostly overloaded maintainers. For testers it's usually even worse - how many patches are actually tested? Judging from what I read on LKML not that many.

Quoted 6 lines

So we should definitely discuss:
- how can we encourage hobbyists to become regular contributors -- how to keep people interested, the drop-out rates are huge.
- encourage regular contributors to become reviewers and testers
- reviewers to become co-maintainers and finally maintainers (once the original maintainer is used up or moves up/on)

Good topic.

Quoted 8 lines
Quoted 2 lines

Unfortunately, there are not too many incentives for people to become code reviewers or testers, or at least to spend more time reviewing patches.

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Sounds like a good idea to me, though it would only effect a tiny
percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

Quoted 10 lines
Quoted 8 lines

Most of the time there's a little to no recognition for doing that work and, quite frankly, writing code is more rewarding than that for the majority of people anyway.

The only way to address this problem I can see is to recognize reviewers *much* more than we tend to do and not just "encourage" them, because that's way insufficient.

What other incentives or recognition mechanisms would you propose?

Again, it's not much an an incentive (and has disadvantages) but explicitly acknowledging reviewers for more areas in MAINTAINERS might give them more of a warm fuzzy feeling! (keeping these entries up to date is also important - another nightmare for Maintainers as they don't want anyone to feel not acknowledged as they weren't included in the 'official' reviewers list).

They do then get personally emailed lots and lots of patches as a result but they clearly like that sort of thing :)

So to me it seems like it's the small stuff that just gives a warm fuzzy feeling that may make all the difference.

Jonathan

Quoted 8 lines

James

_______________________________________________
Ksummit-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

#287 lore raw

I totally agree with Jonathan.
The warm fuzzy feeling makes the difference.
Beginning with thank you emails and seasonal greetings, but also being listed in MAINTAINERS creates a feeling of belonging. So this is something we already have improved... But still a long way to go!

I also think that an invite to something like KS is usually seen as a reward, since it is an invite only event with the big penguins.

Peter

Signature · 1 line
Sent from my mobile
#288 lore raw
In reply to [email protected]
Quoted 11 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I was thinking of writing a perl script that would read my LKML archive and somewhat intelligently looking at people who replied to patches, that also has snippets of the patch in the reply, and counting them. I think that would be a more accurate use of real reviewers than just the Reviewed-by tag.

-- Steve

#289 lore raw
Quoted 12 lines · On Wednesday 08 July 2015 14:53:15 Steven Rostedt wrote:
Quoted 10 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

Quoted 8 lines

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I was thinking of writing a perl script that would read my LKML archive and somewhat intelligently looking at people who replied to patches, that also has snippets of the patch in the reply, and counting them. I think that would be a more accurate use of real reviewers than just the Reviewed-by tag.

Reviewed-by or Acked-by metrics are unfortunately very easy to game. If we could make them more robust, we could publish statistics as we do for commits authorship (http://remword.com/kps_result/ for instance) and start mentioning people's names in kernel development reports. That might not be much, but it could help reviewers feeling valued for their work.

Signature · 3 lines
Regards,

Laurent Pinchart
#290 lore raw
Quoted 1 line · On Wed, 08 Jul 2015 22:56:38 +0300 Laurent Pinchart…

Reviewed-by or Acked-by metrics are unfortunately very easy to game. If we

I'm not worried about Acked-by, as that (to me anyway) is just a maintainer telling other maintainers that they are fine with the change, and they are OK with it going in via another tree.

I also sometimes give an Acked-by, as a "I took a quick look, and it looks good to me". I only add a Reviewed-by tag if I took enough effort to understand every part of the patch as if I wrote it myself.

-- Steve

#291 lore raw
Quoted 5 lines · Steven Rostedt
Quoted 1 line · On Wed, 08 Jul 2015 22:56:38 +0300 Laurent Pinchart…

Reviewed-by or Acked-by metrics are unfortunately very easy to game. If we

Quoted 7 lines

I'm not worried about Acked-by, as that (to me anyway) is just a maintainer telling other maintainers that they are fine with the change, and they are OK with it going in via another tree.

I also sometimes give an Acked-by, as a "I took a quick look, and it looks good to me". I only add a Reviewed-by tag if I took enough effort to understand every part of the patch as if I wrote it myself.

I do that too and that's my understanding of what the tag is for.

However, some people treat it as a this-patch-looks-good-to-me tag.

But regardless of what tags you use, if you say something along the lines of:

- I like A (because of X).
- I don't like B (because of Y).
- C will break things because of Z.

in your review, every maintainer will be grateful for that. I'd like people doing that kind of stuff to be recognized in some special way, because that's really really helpful (and we need it).

Honestly, I'm not sure how to make that happen.

Thanks,
Rafael

#292 lore raw
Quoted 10 lines · Rafael J. Wysocki
Quoted 9 lines · Steven Rostedt
Quoted 1 line · On Wed, 08 Jul 2015 22:56:38 +0300 Laurent Pinchart…

Reviewed-by or Acked-by metrics are unfortunately very easy to game. If we

I'm not worried about Acked-by, as that (to me anyway) is just a maintainer telling other maintainers that they are fine with the change, and they are OK with it going in via another tree.

Quoted 5 lines
Quoted 3 lines

I also sometimes give an Acked-by, as a "I took a quick look, and it looks good to me". I only add a Reviewed-by tag if I took enough effort to understand every part of the patch as if I wrote it myself.

I do that too and that's my understanding of what the tag is for.

That seems to dilute the value of Acked-by. Acked-by requires a review, although the example statement of "yep, looks good to me" in SubmittingPatches is a lot less stringent than the "Reviewer's statement of oversight" that is attached to the Reviewed-by tag.

"I took a quick look" is not what I would like to encourage for reviews.

Would the "Cc:" tag be a more appropriate tag in this case? (Described in the same section as Acked-by in SubmittingPatches.) Or maybe SubmittingPatches needs to change to match reality?

SubmittingPatches:

Acked-by: is not as formal as Signed-off-by:. It is a record that the acker has at least reviewed the patch and has indicated acceptance. Hence patch mergers will sometimes manually convert an acker's "yep, looks good to me" into an Acked-by: (but note that it is usually better to ask for an explicit ack).

Then the next paragraph also allows a subsystem maintainer to acknowledge just that subsystem's part of a multiple subsystem patch.

Quoted 9 lines

However, some people treat it as a this-patch-looks-good-to-me tag.

But regardless of what tags you use, if you say something along the lines of:

- I like A (because of X).
- I don't like B (because of Y).
- C will break things because of Z.

Quoted 8 lines

in your review, every maintainer will be grateful for that. I'd like people doing that kind of stuff to be recognized in some special way, because that's really really helpful (and we need it).

Honestly, I'm not sure how to make that happen.

Thanks,
Rafael

- Frank

Reviewed-by Edsel Murphy
#293 lore raw
In reply to Steven Rostedt
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Quoted 9 lines
Quoted 5 lines

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

Except for the following, they are always reliable and can be trusted.

Seriously, it does happen that I send Reviewed-by: or Acked-by: feedback if a patch is just fine as-is. What do you expect the reviewer to do in such a case ?

Thanks,
Guenter

#294 lore raw
Quoted 21 lines · Guenter Roeck
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Quoted 9 lines
Quoted 5 lines

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

Quoted 7 lines

Except for the following, they are always reliable and can be trusted.

Seriously, it does happen that I send Reviewed-by: or Acked-by: feedback if a patch is just fine as-is. What do you expect the reviewer to do in such a case ?

There's almost always something you can say.

Even if it's a trivial patch, eg. a spelling fix, as the reviewer you should be confirming that only the spelling fix happened, ie. no other changes slipped into the diff. And so you can say that.

If it's more complex than a spelling fix then there's usually something you can comment on.

There might be times when all you can say is "Yep, logic looks right" which might seem redundant, but personally I'd prefer to see that than just a plain Reviewed-by.

cheers

#295 lore raw
Quoted 12 lines · Michael Ellerman
Quoted 11 lines · Guenter Roeck
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I don't expect my Reviewed-by tag with no extra comments to carry much weight if I send it to a maintainer who does not know me.

But if I have a history of good reviews to a specific maintainer, then why should I have to add a message that says: Yes, I really, really did review the patch. I truly mean that the patch "has been reviewed and found acceptable according to the Reviewer's Statement" as listed in SubmittingPatches.

And I read Steve's qualification of "don't trust ... _much_" as being consistent with what I am saying, so I'm fine with that. The point I want to make is that a Reviewed-by tag without comments should not always be assumed to be without meaning or value.

Quoted 11 lines
Quoted 9 lines
Quoted 1 line

Except for the following, they are always reliable and can be trusted.

Seriously, it does happen that I send Reviewed-by: or Acked-by: feedback if a patch is just fine as-is. What do you expect the reviewer to do in such a case ?

There's almost always something you can say.

And there is a time to not nit-pick a patch to death.

If my comment will not result in a significantly better patch, then I am wasting the patch submitter's time, and the time of everyone else on the mail list that will have to read my comment, and potentially have to read and review a new spin of the patch.

Reviewed-by does not mean that I believe the patch is perfect and could not possibly be improved. From the "Reviewer's statement of oversight":

(c) While there may be things that could be improved with this submission, I believe that it is, at this time, (1) a worthwhile modification to the kernel, and (2) free of known issues which would argue against its inclusion.

Quoted 11 lines

Even if it's a trivial patch, eg. a spelling fix, as the reviewer you should be confirming that only the spelling fix happened, ie. no other changes slipped into the diff. And so you can say that.

If it's more complex than a spelling fix then there's usually something you can comment on.

There might be times when all you can say is "Yep, logic looks right" which might seem redundant, but personally I'd prefer to see that than just a plain Reviewed-by.

"Yep, logic looks right" actually seems like a useful comment, because it tells me that the reviewer probably does not understand the full context, but the code is self consistent and does not have any obvious problems. So I will not think that the review is complete.

-Frank

#296 lore raw
Quoted 15 lines · Frank Rowand
Quoted 5 lines · Michael Ellerman
Quoted 4 lines · Guenter Roeck
Quoted 2 lines · Steven Rostedt

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I don't expect my Reviewed-by tag with no extra comments to carry much weight if I send it to a maintainer who does not know me.

But if I have a history of good reviews to a specific maintainer, then why should I have to add a message that says: Yes, I really, really did review the patch. I truly mean that the patch "has been reviewed and found acceptable according to the Reviewer's Statement" as listed in SubmittingPatches.

Quoted 4 lines

And I read Steve's qualification of "don't trust ... _much_" as being consistent with what I am saying, so I'm fine with that. The point I want to make is that a Reviewed-by tag without comments should not always be assumed to be without meaning or value.

Indeed, and when I'm the one dealing with applying the patches it's actually helpful to not have extra text that needs reading and thinking about when applying things if I trust whoever did the review. Past history as a reviewer is much more important than any verbiage around the review and review that consists of a simple tag takes seconds to read.

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

Seriously, it does happen that I send Reviewed-by: or Acked-by: feedback if a patch is just fine as-is. What do you expect the reviewer to do in such a case ?

Quoted 8 lines
Quoted 1 line

There's almost always something you can say.

And there is a time to not nit-pick a patch to death.

If my comment will not result in a significantly better patch, then I am wasting the patch submitter's time, and the time of everyone else on the mail list that will have to read my comment, and potentially have to read and review a new spin of the patch.

Very much so.

#297 lore raw
In reply to Frank Rowand
Quoted 30 lines · Frank Rowand
Quoted 12 lines · Michael Ellerman
Quoted 11 lines · Guenter Roeck
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I don't expect my Reviewed-by tag with no extra comments to carry much weight if I send it to a maintainer who does not know me.

But if I have a history of good reviews to a specific maintainer, then why should I have to add a message that says: Yes, I really, really did review the patch. I truly mean that the patch "has been reviewed and found acceptable according to the Reviewer's Statement" as listed in SubmittingPatches.

Quoted 5 lines

And I read Steve's qualification of "don't trust ... _much_" as being consistent with what I am saying, so I'm fine with that. The point I want to make is that a Reviewed-by tag without comments should not always be assumed to be without meaning or value.

Absolutely agree.

It looks like we have yet another set of diverging maintainer expectations. Some maintainers will expect me to provide an extra comment, which I'll have to phrase carefully to avoid it being misinterpreted as "I just glanced at the code and didn't find an obvious issue with it". Others will get annoyed at me providing the extra comment.

Guenter

#298 lore raw
Quoted 36 lines · Guenter Roeck
Quoted 30 lines · Frank Rowand
Quoted 12 lines · Michael Ellerman
Quoted 11 lines · Guenter Roeck
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I don't expect my Reviewed-by tag with no extra comments to carry much weight if I send it to a maintainer who does not know me.

But if I have a history of good reviews to a specific maintainer, then why should I have to add a message that says: Yes, I really, really did review the patch. I truly mean that the patch "has been reviewed and found acceptable according to the Reviewer's Statement" as listed in SubmittingPatches.

Quoted 5 lines

And I read Steve's qualification of "don't trust ... _much_" as being consistent with what I am saying, so I'm fine with that. The point I want to make is that a Reviewed-by tag without comments should not always be assumed to be without meaning or value.

Quoted 7 lines

Absolutely agree.

It looks like we have yet another set of diverging maintainer expectations. Some maintainers will expect me to provide an extra comment, which I'll have to phrase carefully to avoid it being misinterpreted as "I just glanced at the code and didn't find an obvious issue with it". Others will get annoyed at me providing the extra comment.

Why would a couple lines of context be any harder to deal with than all the meta-data that comes along with an email including a Reviewed-by?

Signature · 2 lines
Darren Hart
Intel Open Source Technology Center
#299 lore raw

No matter what we ask people to type, the system can be gamed so long as we (a) use automated systems, and (b) we're transparent what the automated system uses as a signal about what's a valid review versus a garbage review. This is basically the same problem that Google has when trying to deal with SEO folks who are trying to game the search algorithm, and it's not really a soluble problem unless you have a lot of people constantly working on refining the system and adjusting in response to humans who are trying to attack the system.

And if there are companies who are using reviews, or signed-off-by counts as a basis of performance reviews, humans *will* be incentivized to game the system, regardless of what the kernel summit program committee might be doing to use these systems.

(Although I will say there is one person who I've consistently downgraded *because* it's obvious s/he has been sending lots of trivial patches, and while that person may (or may not) have changed, it's still human nature that I assume that this person's scores are inflated. And this is true of whoever sends huge number of checkpatch or coccinelle generated or inspired patches. So people should be aware that attempts to game the system can backfire, when people start imposing informal "manual penalties" to their evaluation of that particular person. Which is exactly what happens in the Search Engine Optimization world, by the way....)

- Ted

#300 lore raw
Quoted 9 lines · Theodore Ts'o

No matter what we ask people to type, the system can be gamed so long as we (a) use automated systems, and (b) we're transparent what the automated system uses as a signal about what's a valid review versus a garbage review. This is basically the same problem that Google has when trying to deal with SEO folks who are trying to game the search algorithm, and it's not really a soluble problem unless you have a lot of people constantly working on refining the system and adjusting in response to humans who are trying to attack the system.

Quoted 5 lines

And if there are companies who are using reviews, or signed-off-by counts as a basis of performance reviews, humans *will* be incentivized to game the system, regardless of what the kernel summit program committee might be doing to use these systems.

While this may be correct in some circumstances, there are also companies which do just the opposite, especially when it comes to reviews. At least for my part I still have to encounter a company where "provides excellent and thorough code reviews" can be found as positive statement in a performance review.

Earlier it was discussed how to improve the recognition of reviewers. Your comments seems to suggest the opposite, and may actually discourage reviewers. Why should I review Linux kernel code if that is seen by some as me trying to "game" the system ?

I used to submit some more or less random cleanup patches into various areas of the kernel, just for fun, because I liked doing it, and because I needed some distraction. I stopped doing it after I realized that this may be seen as "gaming the system". Nowadays I only submit such random patches to fix build errors or real bugs. Would I continue to do that if I get accused of trying to game the system ? Probably not.

I am not much if at all concerned with people gaming the system to improve their standing, whereever that may be. I am much more concerned that we may drive people off by accusing them to just submit patches or reviews to game the system. At the end this is our loss, not theirs. If their work results in better code, who cares what their motivation might be ? And who are we to judge their motivation ?

Quoted 11 lines

(Although I will say there is one person who I've consistently downgraded *because* it's obvious s/he has been sending lots of trivial patches, and while that person may (or may not) have changed, it's still human nature that I assume that this person's scores are inflated. And this is true of whoever sends huge number of checkpatch or coccinelle generated or inspired patches. So people should be aware that attempts to game the system can backfire, when people start imposing informal "manual penalties" to their evaluation of that particular person. Which is exactly what happens in the Search Engine Optimization world, by the way....)

That may be true for some people, but at the same time I think statements like the above might discourage people who just like cleaning up code for fun. There are several of those working on cleaning up the Linux kernel, and I truly appreciate their efforts.

Guenter

Reviewed-by John Q. Random
#301 lore raw
Quoted 5 lines · Guenter Roeck

Earlier it was discussed how to improve the recognition of reviewers. Your comments seems to suggest the opposite, and may actually discourage reviewers. Why should I review Linux kernel code if that is seen by some as me trying to "game" the system ?

So I were designing an initial system that automatically scored reviewers, I'd be looking to see, from a holistic point of view, how many reviews were zero-length:

... and nothing else,

.... versus how many reviews had specific comments on various different portions of the patch. If possible, the automated system would try to distinguish between comments that were just pointing out whitespace issues (which would be a slight positive) with comments that point out genuine design issues (this will be really hard to do in an automated fashion, but a very sophisticated nueral network[1] mgiht be able to hack it).

I might also try using some kind scheme that counted the number of words in a review (stripping out lines of patch or commit description that the review was a reply to), etc. But of course, if it was public knowledge that the system was just stripping out the original e-mail, and then just doing a "wc -w", then people would game the system by adding list of random words at the bottom of the review.

And, of course, I'd have the system give a huge negative score if a commit that got a "LGTM" positive review caused a bug that required the patch to be reverted. *That* signal, at least, would be hard to game, and would hopefuly encourage people to actually take time reviewing a commit, and not blindly slapping a reviewed-by on a commit they don't understand.

You see? It's not that reviews in and of themselves are attempts to game the system ---- just so long as they are genuine reviews. If there is evidence that the reviews are issued within seconds of the original patch going out, with a Reviewed-by: line and nothing else, what would *you* think about the quality of that review?

Quoted 4 lines

That may be true for some people, but at the same time I think statements like the above might discourage people who just like cleaning up code for fun. There are several of those working on cleaning up the Linux kernel, and I truly appreciate their efforts.

Sure, but that's not the people who (in my opinion as a program committee member) should be attendingt he Kernel Summit, where we want people who are genuinely clueful about technical and policy issues, and not people like (for example) Nick Krause.

Regards,

- Ted

[1] http://googleresearch.blogspot.com/2015/06/inceptionism-going-deeper-into-neural.html

#302 lore raw
Quoted 15 lines · Theodore Ts'o
Quoted 5 lines · Guenter Roeck

Earlier it was discussed how to improve the recognition of reviewers. Your comments seems to suggest the opposite, and may actually discourage reviewers. Why should I review Linux kernel code if that is seen by some as me trying to "game" the system ?

So I were designing an initial system that automatically scored reviewers, I'd be looking to see, from a holistic point of view, how many reviews were zero-length:

... and nothing else,

Quoted 7 lines

.... versus how many reviews had specific comments on various different portions of the patch. If possible, the automated system would try to distinguish between comments that were just pointing out whitespace issues (which would be a slight positive) with comments that point out genuine design issues (this will be really hard to do in an automated fashion, but a very sophisticated nueral network[1] mgiht be able to hack it).

That assumes the patch actually has issues. To use the reviews I do on RCU patches as an example, in a patch series, I might reply to a few patches with "here are some issues; with those fixed, Reviewed-by...", and then reply to the remaining unproblematic patches (individually or in aggregate) with just the Reviewed-by.

- Josh Triplett

#303 lore raw
Quoted 6 lines · On Thu, Jul 09, 2015 at 04:13:52PM -0700, [email protected] wrote:

That assumes the patch actually has issues. To use the reviews I do on RCU patches as an example, in a patch series, I might reply to a few patches with "here are some issues; with those fixed, Reviewed-by...", and then reply to the remaining unproblematic patches (individually or in aggregate) with just the Reviewed-by.

Right, that's why I talked about doing this on a holistic way. It's true that individual patches, and maybe even all of the patches in a patch series, might be *perfect*. But presumably that won't be true for **all** of the patches you review.

This is why creating a system to do patch reviewer ranking is so complicated. You need to do this by looking at the entire corpus of patches reviewed by the reviewer, so (for example) you can find out which patch reviewers are allowing bad patches to get through to Linus by giving a thumbs up to a patch that needed to be reverted.

(At $WORK, if someone creates a CL that causes a massive failure, it's not just the engineer who submitted the CL who is at fault, but also the reviewers who gave the CL a positive review --- and that's as it should be. Of course we also ask if there is something about the development and regression test environment that might be a systematic cause of problems, but if someone is consistently giving LGTM's without doing enough due diligence, that's a issue that ultimately need to be addressed by the engineer's manager. The question is how to deal with this kind of failure mode in an open source, volunteer world.)

- Ted

#304 lore raw
In reply to [email protected]
Quoted 6 lines · On Thu, 9 Jul 2015 16:13:52 -0700 [email protected] wrote:

That assumes the patch actually has issues. To use the reviews I do on RCU patches as an example, in a patch series, I might reply to a few patches with "here are some issues; with those fixed, Reviewed-by...", and then reply to the remaining unproblematic patches (individually or in aggregate) with just the Reviewed-by.

Josh, I read your reviews. Yes, you have a bunch of single "Reviewed-by" tags, but you also have a lot of emails with substantially significant comments. What Ted is saying, is to see how many significant replies one has. In fact, I would say if we were to automate this, each significant reply (one with actual comments), would increase the weight that a single "Reviewed-by" email would carry. That way people like yourself would have more weight attached to emails with single "Reviewed-by" than others that don't have many emails with actual comments attached.

Anyway, for consideration to KS, the top reviewers that an automated system would produce, would only get the program committee's eyes looking at it. It by no way guarantees an invite. No matter what automated system is in place, at least for KS, there will always be humans involved in analyzing that data.

-- Steve

#305 lore raw
Quoted 7 lines · Steven Rostedt
Quoted 5 lines · On Thu, 9 Jul 2015 16:13:52 -0700 [email protected] wrote:

That assumes the patch actually has issues. To use the reviews I do on RCU patches as an example, in a patch series, I might reply to a few patches with "here are some issues; with those fixed, Reviewed-by...", and then reply to the remaining unproblematic patches (individually or in aggregate) with just the Reviewed-by.

Quoted 8 lines

Josh, I read your reviews. Yes, you have a bunch of single "Reviewed-by" tags, but you also have a lot of emails with substantially significant comments. What Ted is saying, is to see how many significant replies one has. In fact, I would say if we were to automate this, each significant reply (one with actual comments), would increase the weight that a single "Reviewed-by" email would carry. That way people like yourself would have more weight attached to emails with single "Reviewed-by" than others that don't have many emails with actual comments attached.

Ah, I see what you're getting at now. I thought you were trying to figure out the value of a "Reviewed-by" on an individual patch. But you're right, a review doesn't carry much weight if your reviews *never* find issues; that would imply you're either chery-picking easy patches to review or rubber-stamping bad patches.

(And conversely, a Reviewed-by on a patch that turns out to be wrong shouldn't carry any more stigma than submitting such a patch yourself. Everybody has submitted a brown-paper-bag at some point; as long as there's not a pattern of it, that's not a critical issue with reviewing.)

- Josh Triplett

#306 lore raw
In reply to Steven Rostedt
Quoted 10 lines · Steven Rostedt
Quoted 5 lines · On Thu, 9 Jul 2015 16:13:52 -0700 [email protected] wrote:

That assumes the patch actually has issues. To use the reviews I do on RCU patches as an example, in a patch series, I might reply to a few patches with "here are some issues; with those fixed, Reviewed-by...", and then reply to the remaining unproblematic patches (individually or in aggregate) with just the Reviewed-by.

Quoted 8 lines

Josh, I read your reviews. Yes, you have a bunch of single "Reviewed-by" tags, but you also have a lot of emails with substantially significant comments. What Ted is saying, is to see how many significant replies one has. In fact, I would say if we were to automate this, each significant reply (one with actual comments), would increase the weight that a single "Reviewed-by" email would carry. That way people like yourself would have more weight attached to emails with single "Reviewed-by" than others that don't have many emails with actual comments attached.

Just to state the obvious, I greatly value Josh's reviews as well, especially the ones where he points out a bug. Much easier than finding them the hard way! ;-)

Thanx, Paul

#307 lore raw
In reply to Theodore Ts'o
Quoted 37 lines · Theodore Ts'o
Quoted 5 lines · Guenter Roeck

Earlier it was discussed how to improve the recognition of reviewers. Your comments seems to suggest the opposite, and may actually discourage reviewers. Why should I review Linux kernel code if that is seen by some as me trying to "game" the system ?

So I were designing an initial system that automatically scored reviewers, I'd be looking to see, from a holistic point of view, how many reviews were zero-length:

... and nothing else,

.... versus how many reviews had specific comments on various different portions of the patch. If possible, the automated system would try to distinguish between comments that were just pointing out whitespace issues (which would be a slight positive) with comments that point out genuine design issues (this will be really hard to do in an automated fashion, but a very sophisticated nueral network[1] mgiht be able to hack it).

I might also try using some kind scheme that counted the number of words in a review (stripping out lines of patch or commit description that the review was a reply to), etc. But of course, if it was public knowledge that the system was just stripping out the original e-mail, and then just doing a "wc -w", then people would game the system by adding list of random words at the bottom of the review.

And, of course, I'd have the system give a huge negative score if a commit that got a "LGTM" positive review caused a bug that required the patch to be reverted. *That* signal, at least, would be hard to game, and would hopefuly encourage people to actually take time reviewing a commit, and not blindly slapping a reviewed-by on a commit they don't understand.

Quoted 6 lines

You see? It's not that reviews in and of themselves are attempts to game the system ---- just so long as they are genuine reviews. If there is evidence that the reviews are issued within seconds of the original patch going out, with a Reviewed-by: line and nothing else, what would *you* think about the quality of that review?

Agreed. On the other side, is gaming really a problem with kernel code reviews ? Sure, a search engine provider will have to look out for abuse patterns, but for code reviews I am not sure if it is worth the effort. I would suspect that it is much more likely that the simple "wc -w" approach should provide you with worthy candidates for the KS. Since you are not dealing with hundreds or thousands of candidates, I'd assume you'll do a hand screening anyway, and quickly identify any gamers. I'd be quite surprised if there are any, though.

Quoted 10 lines
Quoted 4 lines

That may be true for some people, but at the same time I think statements like the above might discourage people who just like cleaning up code for fun. There are several of those working on cleaning up the Linux kernel, and I truly appreciate their efforts.

Sure, but that's not the people who (in my opinion as a program committee member) should be attendingt he Kernel Summit, where we want people who are genuinely clueful about technical and policy issues, and not people like (for example) Nick Krause.

Nick is such an outlier that I really hope he isn't used as a baseline to set any kind of policy. But how do you evaluate someone like, say, Axel Lin, who makes excellent contributions all over the place ?

Thanks,
Guenter

#308 lore raw
Quoted 8 lines · Guenter Roeck

Agreed. On the other side, is gaming really a problem with kernel code reviews ? Sure, a search engine provider will have to look out for abuse patterns, but for code reviews I am not sure if it is worth the effort. I would suspect that it is much more likely that the simple "wc -w" approach should provide you with worthy candidates for the KS. Since you are not dealing with hundreds or thousands of candidates, I'd assume you'll do a hand screening anyway, and quickly identify any gamers. I'd be quite surprised if there are any, though.

I think the concern is more long term - if we're trying to reward reviewers in order to encourage review then probably now we're fine but if we do it in a way that's gameable will that end up having the effect that we want?

#309 lore raw
In reply to Guenter Roeck
Quoted 44 lines · Guenter Roeck
Quoted 37 lines · Theodore Ts'o
Quoted 5 lines · Guenter Roeck

Earlier it was discussed how to improve the recognition of reviewers. Your comments seems to suggest the opposite, and may actually discourage reviewers. Why should I review Linux kernel code if that is seen by some as me trying to "game" the system ?

So I were designing an initial system that automatically scored reviewers, I'd be looking to see, from a holistic point of view, how many reviews were zero-length:

... and nothing else,

.... versus how many reviews had specific comments on various different portions of the patch. If possible, the automated system would try to distinguish between comments that were just pointing out whitespace issues (which would be a slight positive) with comments that point out genuine design issues (this will be really hard to do in an automated fashion, but a very sophisticated nueral network[1] mgiht be able to hack it).

I might also try using some kind scheme that counted the number of words in a review (stripping out lines of patch or commit description that the review was a reply to), etc. But of course, if it was public knowledge that the system was just stripping out the original e-mail, and then just doing a "wc -w", then people would game the system by adding list of random words at the bottom of the review.

And, of course, I'd have the system give a huge negative score if a commit that got a "LGTM" positive review caused a bug that required the patch to be reverted. *That* signal, at least, would be hard to game, and would hopefuly encourage people to actually take time reviewing a commit, and not blindly slapping a reviewed-by on a commit they don't understand.

Quoted 6 lines

You see? It's not that reviews in and of themselves are attempts to game the system ---- just so long as they are genuine reviews. If there is evidence that the reviews are issued within seconds of the original patch going out, with a Reviewed-by: line and nothing else, what would *you* think about the quality of that review?

Quoted 8 lines

Agreed. On the other side, is gaming really a problem with kernel code reviews ? Sure, a search engine provider will have to look out for abuse patterns, but for code reviews I am not sure if it is worth the effort. I would suspect that it is much more likely that the simple "wc -w" approach should provide you with worthy candidates for the KS. Since you are not dealing with hundreds or thousands of candidates, I'd assume you'll do a hand screening anyway, and quickly identify any gamers. I'd be quite surprised if there are any, though.

I've personally seen it, and I don't think I'm alone. It seems to follow a pattern of:

- Manager/HR thinks counting tags is a useful metric (#!@$ laziness).
- tag-count becomes an evaluation item.
- Pay raises are affected.
- patch submitters do the obvious.
- maintainers weep and die a little inside.

The easy ones to spot are multiple-S-o-bs. I've actually been told "No, he didn't write any code, I was just trying to help him out."

The hard ones are when a patch was developed by a company, say a new driver. On the first patch submission, there's a Reviewed-by tag for someone I've never seen on the list. He *could* be the HW engineer that designed the IP block, or he could be the likable guy that got a bad eval last time around. We don't know.

I keep those tags for one simple reason. If a bisect lands on that commit, he'll be in the list of emails for trying to figure out what went wrong. imo, it's better to include him to help diagnose a problem than to play catch-up "Who else remembers this commit?", then Cc/Fwd/Bounce during triage. Which comes back to my "Tags are a responsibility, not a reward."

As has been expressed elsewhere, it also depends on who's doing the patch submitting. There's certainly an element of trust to all of this. Unfamiliar submitters with red flags get asked awkward questions.

thx,

Jason.

#310 lore raw
Quoted 9 lines · Jason Cooper
Quoted 8 lines

Agreed. On the other side, is gaming really a problem with kernel code reviews ? Sure, a search engine provider will have to look out for abuse patterns, but for code reviews I am not sure if it is worth the effort. I would suspect that it is much more likely that the simple "wc -w" approach should provide you with worthy candidates for the KS. Since you are not dealing with hundreds or thousands of candidates, I'd assume you'll do a hand screening anyway, and quickly identify any gamers. I'd be quite surprised if there are any, though.

Quoted 9 lines

I've personally seen it, and I don't think I'm alone. It seems to follow a pattern of:

- Manager/HR thinks counting tags is a useful metric (#!@$ laziness).
- tag-count becomes an evaluation item.
- Pay raises are affected.
- patch submitters do the obvious.
- maintainers weep and die a little inside.

Sigh :-(. Guess I never had the pleasure of working for any of those companies, and the areas of the kernel I care about may be too obscure to get much attention by the gamers.

Quoted 3 lines

The easy ones to spot are multiple-S-o-bs. I've actually been told "No, he didn't write any code, I was just trying to help him out."

Multiple S-o-b's don't always mean gaming, though. For example, my company's workflow requires me to sign off upstream patches, not to get annother S-o-b with my name on it, but to certify that the patch does not accidentially publish any company IP (and, if it does, it is my fault, not the fault of the person who wrote the code).

Guenter

#311 lore raw
Quoted 23 lines · Guenter
Quoted 9 lines · Jason Cooper
Quoted 8 lines

Agreed. On the other side, is gaming really a problem with kernel code reviews ? Sure, a search engine provider will have to look out for abuse patterns, but for code reviews I am not sure if it is worth the effort. I would suspect that it is much more likely that the simple "wc -w" approach should provide you with worthy candidates for the KS. Since you are not dealing with hundreds or thousands of candidates, I'd assume you'll do a hand screening anyway, and quickly identify any gamers. I'd be quite surprised if there are any, though.

Quoted 9 lines

I've personally seen it, and I don't think I'm alone. It seems to follow a pattern of:

- Manager/HR thinks counting tags is a useful metric (#!@$ laziness).
- tag-count becomes an evaluation item.
- Pay raises are affected.
- patch submitters do the obvious.
- maintainers weep and die a little inside.

Sigh :-(. Guess I never had the pleasure of working for any of those companies, and the areas of the kernel I care about may be too obscure to get much attention by the gamers.

Quoted 8 lines
Quoted 3 lines

The easy ones to spot are multiple-S-o-bs. I've actually been told "No, he didn't write any code, I was just trying to help him out."

Multiple S-o-b's don't always mean gaming, though. For example, my company's workflow requires me to sign off upstream patches, not to get annother S-o-b with my name on it, but to certify that the patch does not accidentially publish any company IP (and, if it does, it is my fault, not the fault of the person who wrote the code).

There is a danger to having people interpret the s-o-b tag differently than what it originally was intended for, such confusion deserves serious attention and my hope is that if folks detect these misuses they can try to educate folks on it. The s-o-b tag means the person is certifying under the Developer Certificate or Origin (DCO) the patch in question. That's it. The practical gains of such a leight weight development tool to use something like the s-o-b combined with the huge legal merit behind it has convinced us to generalize the DCO and encourage people outside of Linux to use it:

http://developercertificate.org/

There was coverage over this on lwn:

https://lwn.net/Articles/592503/

I've also written about the importance here:

http://www.do-not-panic.com/2014/02/developer-certificate-of-origin.html

The list of users is outdated and I cannot keep up counting folks using it, but the more, quite recently the docker project announced it was embracing it for instance (although it seems they modified it mildly, WTF(?)):

https://blog.docker.com/2014/01/docker-code-contributions-require-developer-certificate-of-origin/

Because of its legal value, and because we all stand to gain from this legal value then as a commnity at large we should try to correc innappropriate uses of the s-o-b and educate folks on it.

I'll stay out of the conversation of other tag's, only to second Julia's sentiments

Luis

#312 lore raw

Hi Luis,

Quoted 7 lines · Luis R. Rodriguez
Quoted 6 lines
Quoted 1 line

Multiple S-o-b's don't always mean gaming, though. For example, my company's workflow requires me to sign off upstream patches, not to get annother S-o-b with my name on it, but to certify that the patch does not accidentially publish any company IP (and, if it does, it is my fault, not the fault of the person who wrote the code).

Quoted 10 lines

There is a danger to having people interpret the s-o-b tag differently than what it originally was intended for, such confusion deserves serious attention and my hope is that if folks detect these misuses they can try to educate folks on it. The s-o-b tag means the person is certifying under the Developer Certificate or Origin (DCO) the patch in question. That's it. The practical gains of such a leight weight development tool to use something like the s-o-b combined with the huge legal merit behind it has convinced us to generalize the DCO and encourage people outside of Linux to use it:

Not really sure if I understand you correctly. Are you saying that you consider the above to be a mis-use of s-o-b ?

By co-signing a patch I do (co-)certify it under DCO; I am well aware of that. That it may have an additional context doesn't change that, or limit its value. I can well imagine that other companies have a similar approval flow, and I don't really understand why that would or could be a problem. Can you educate me ?

Thanks,
Guenter

#313 lore raw
Quoted 20 lines · Guenter

Hi Luis,

Quoted 7 lines · Luis R. Rodriguez
Quoted 6 lines
Quoted 1 line

Multiple S-o-b's don't always mean gaming, though. For example, my company's workflow requires me to sign off upstream patches, not to get annother S-o-b with my name on it, but to certify that the patch does not accidentially publish any company IP (and, if it does, it is my fault, not the fault of the person who wrote the code).

Quoted 10 lines

There is a danger to having people interpret the s-o-b tag differently than what it originally was intended for, such confusion deserves serious attention and my hope is that if folks detect these misuses they can try to educate folks on it. The s-o-b tag means the person is certifying under the Developer Certificate or Origin (DCO) the patch in question. That's it. The practical gains of such a leight weight development tool to use something like the s-o-b combined with the huge legal merit behind it has convinced us to generalize the DCO and encourage people outside of Linux to use it:

Quoted 2 lines

Not really sure if I understand you correctly. Are you saying that you consider the above to be a mis-use of s-o-b ?

Only if folks disregard or not aware of the DCO.

Quoted 3 lines

By co-signing a patch I do (co-)certify it under DCO; I am well aware of that. That it may have an additional context doesn't change that, or limit its value.

Sure.

Quoted 3 lines

I can well imagine that other companies have a
similar approval flow, and I don't really understand why that would or could be a problem. Can you educate me ?

The concern is when folks document SOB practices which do not address first and foremost that the SOB *is* primarily for the DCO, now if companies or groups like to have their own other internal ammendments for it, that's fine, its just that for patches that go into Linux the SOB just is meant to implicate the DCO. Nothing more.

Luis

#314 lore raw
In reply to Guenter
Quoted 19 lines · Guenter
Quoted 9 lines · Jason Cooper
Quoted 8 lines

Agreed. On the other side, is gaming really a problem with kernel code reviews ? Sure, a search engine provider will have to look out for abuse patterns, but for code reviews I am not sure if it is worth the effort. I would suspect that it is much more likely that the simple "wc -w" approach should provide you with worthy candidates for the KS. Since you are not dealing with hundreds or thousands of candidates, I'd assume you'll do a hand screening anyway, and quickly identify any gamers. I'd be quite surprised if there are any, though.

Quoted 9 lines

I've personally seen it, and I don't think I'm alone. It seems to follow a pattern of:

- Manager/HR thinks counting tags is a useful metric (#!@$ laziness).
- tag-count becomes an evaluation item.
- Pay raises are affected.
- patch submitters do the obvious.
- maintainers weep and die a little inside.

Quoted 8 lines

Sigh :-(. Guess I never had the pleasure of working for any of those companies, and the areas of the kernel I care about may be too obscure to get much attention by the gamers.

Quoted 3 lines

The easy ones to spot are multiple-S-o-bs. I've actually been told "No, he didn't write any code, I was just trying to help him out."

Multiple S-o-b's don't always mean gaming, though.

Agreed, if two people edited a patch, or there's one person approved to cleanup/email LKML, etc. Then multiple-SoB makes sense wrt DCO. My wording could have been better. I had a specific incident in mind, but that doesn't make a rule.

thx,

Jason.

#315 lore raw
In reply to Jason Cooper
Quoted 11 lines · Jason Cooper

I've personally seen it, and I don't think I'm alone. It seems to follow a pattern of:

- Manager/HR thinks counting tags is a useful metric (#!@$ laziness).
- tag-count becomes an evaluation item.
- Pay raises are affected.
- patch submitters do the obvious.
- maintainers weep and die a little inside.

The easy ones to spot are multiple-S-o-bs. I've actually been told "No, he didn't write any code, I was just trying to help him out."

On the flip side, I've submitted patches with two Signed-off-by tags specifically because both people actually wrote the code and are agreeing to the DCO (and more generally claiming responsibility/blame for the patch), and gotten complaints from maintainers that it "doesn't reflect the chain of git trees the patch went through" or similar. Which seems ridiculous.

- Josh Triplett

#316 lore raw
Quoted 10 lines · On Fri, Jul 10, 2015 at 02:00:40PM -0700, [email protected] wrote:
Quoted 9 lines · Jason Cooper

I've personally seen it, and I don't think I'm alone. It seems to follow a pattern of:

- Manager/HR thinks counting tags is a useful metric (#!@$ laziness).
- tag-count becomes an evaluation item.
- Pay raises are affected.
- patch submitters do the obvious.
- maintainers weep and die a little inside.

Quoted 7 lines
Quoted 2 lines

The easy ones to spot are multiple-S-o-bs. I've actually been told "No, he didn't write any code, I was just trying to help him out."

On the flip side, I've submitted patches with two Signed-off-by tags specifically because both people actually wrote the code and are agreeing to the DCO (and more generally claiming responsibility/blame for the patch),

This seems perfectly sane to me.

Quoted 3 lines

and gotten complaints from maintainers that it "doesn't reflect the chain of git trees the patch went through" or similar. Which seems ridiculous.

Agreed.

thx,

Jason.

#317 lore raw
In reply to Darren Hart
Quoted 39 lines · Darren Hart
Quoted 38 lines · Guenter Roeck
Quoted 30 lines · Frank Rowand
Quoted 12 lines · Michael Ellerman
Quoted 11 lines · Guenter Roeck
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I don't expect my Reviewed-by tag with no extra comments to carry much weight if I send it to a maintainer who does not know me.

But if I have a history of good reviews to a specific maintainer, then why should I have to add a message that says: Yes, I really, really did review the patch. I truly mean that the patch "has been reviewed and found acceptable according to the Reviewer's Statement" as listed in SubmittingPatches.

Quoted 5 lines

And I read Steve's qualification of "don't trust ... _much_" as being consistent with what I am saying, so I'm fine with that. The point I want to make is that a Reviewed-by tag without comments should not always be assumed to be without meaning or value.

Absolutely agree.

Quoted 9 lines
Quoted 5 lines

It looks like we have yet another set of diverging maintainer expectations. Some maintainers will expect me to provide an extra comment, which I'll have to phrase carefully to avoid it being misinterpreted as "I just glanced at the code and didn't find an obvious issue with it". Others will get annoyed at me providing the extra comment.

Why would a couple lines of context be any harder to deal with than all the meta-data that comes along with an email including a Reviewed-by?

No, but that isn't the point. I use patchwork, which takes care of automatically adding all tags and removing the clutter around it, so I don't really care one way or another.

I neither expect reviewers to provide additional comments, nor do I mind if they do provide such comments. I see comments as beneficial on complex reviews, or if a reviewer had an earlier concern which was addressed by feedback from the submitter but did not result in code changes. Overall comments may be either positive or neutral to me, depending on the context. But a Reviewed-by: tag does not have less value to me just because there is no comment associated with it.

I _do_ expect reviewers to understand the statement they are making by providing a Reviewed-by: tag, just as I expect patch submitters to understand the statement they are making with their Signed-off: tag.

The point I am making, which has been confirmed by this e-mail exchange, is that reviewers have to be careful of yet another detail when providing feedback on a patch. Some, like you and Stephen, may expect me to provide some feedback around a Reviewed-by: tag, while others may get annoyed at me for providing such additional feedback.

Guenter

#318 lore raw
Quoted 66 lines · Guenter Roeck
Quoted 39 lines · Darren Hart
Quoted 38 lines · Guenter Roeck
Quoted 30 lines · Frank Rowand
Quoted 12 lines · Michael Ellerman
Quoted 11 lines · Guenter Roeck
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I don't expect my Reviewed-by tag with no extra comments to carry much weight if I send it to a maintainer who does not know me.

But if I have a history of good reviews to a specific maintainer, then why should I have to add a message that says: Yes, I really, really did review the patch. I truly mean that the patch "has been reviewed and found acceptable according to the Reviewer's Statement" as listed in SubmittingPatches.

Quoted 5 lines

And I read Steve's qualification of "don't trust ... _much_" as being consistent with what I am saying, so I'm fine with that. The point I want to make is that a Reviewed-by tag without comments should not always be assumed to be without meaning or value.

Absolutely agree.

Quoted 9 lines
Quoted 5 lines

It looks like we have yet another set of diverging maintainer expectations. Some maintainers will expect me to provide an extra comment, which I'll have to phrase carefully to avoid it being misinterpreted as "I just glanced at the code and didn't find an obvious issue with it". Others will get annoyed at me providing the extra comment.

Why would a couple lines of context be any harder to deal with than all the meta-data that comes along with an email including a Reviewed-by?

No, but that isn't the point. I use patchwork, which takes care of automatically adding all tags and removing the clutter around it, so I don't really care one way or another.

I neither expect reviewers to provide additional comments, nor do I mind if they do provide such comments. I see comments as beneficial on complex reviews, or if a reviewer had an earlier concern which was addressed by feedback from the submitter but did not result in code changes. Overall comments may be either positive or neutral to me, depending on the context. But a Reviewed-by: tag does not have less value to me just because there is no comment associated with it.

I _do_ expect reviewers to understand the statement they are making by providing a Reviewed-by: tag, just as I expect patch submitters to understand the statement they are making with their Signed-off: tag.

Quoted 8 lines

The point I am making, which has been confirmed by this e-mail exchange, is that reviewers have to be careful of yet another detail when providing feedback on a patch. Some, like you and Stephen, may expect me to provide some feedback around a Reviewed-by: tag, while others may get annoyed at me for providing such additional feedback.

Guenter

I do agree that a plain reviewed-by from an established contributor adds value without additional comment.

Stepping back a bit though, We are talking about Recruitment, which implies new people. In my opinion, encouraging new people to provide context is a good idea.

Signature · 2 lines
Darren Hart
Intel Open Source Technology Center
#319 lore raw
In reply to Darren Hart
Quoted 7 lines · Darren Hart
Quoted 5 lines · Guenter Roeck

It looks like we have yet another set of diverging maintainer expectations. Some maintainers will expect me to provide an extra comment, which I'll have to phrase carefully to avoid it being misinterpreted as "I just glanced at the code and didn't find an obvious issue with it". Others will get annoyed at me providing the extra comment.

I guess I'm one of those you're thinking about here - annoyed is very strong, it's definitely not an active problem. If it's saying something out of the ordinary then it's adding value but a lot of patches really are just pretty routine.

Quoted 2 lines

Why would a couple lines of context be any harder to deal with than all the meta-data that comes along with an email including a Reviewed-by?

Personally it's the difference between one line of new text that can be read by pattern matching and new text which requires looking at the actual words. It's not the end of the world (and I'd go so far as to say it's actually a good thing if it's adding something) but in cases where I've got a big stack of things I've already looked at where I'm waiting for acks it does help make things a bit smoother.

#320 lore raw
In reply to Michael Ellerman
Quoted 36 lines · Michael Ellerman
Quoted 21 lines · Guenter Roeck
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Quoted 9 lines
Quoted 5 lines

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

Quoted 7 lines

Except for the following, they are always reliable and can be trusted.

Seriously, it does happen that I send Reviewed-by: or Acked-by: feedback if a patch is just fine as-is. What do you expect the reviewer to do in such a case ?

There's almost always something you can say.

Even if it's a trivial patch, eg. a spelling fix, as the reviewer you should be confirming that only the spelling fix happened, ie. no other changes slipped into the diff. And so you can say that.

Quoted 6 lines

If it's more complex than a spelling fix then there's usually something you can comment on.

There might be times when all you can say is "Yep, logic looks right" which might seem redundant, but personally I'd prefer to see that than just a plain Reviewed-by.

Agreed. I have this same conversation about commit messages. I don't care if it's a whitespace fix, if it is worth patching, building, testing, and submitting, it is worth writing a sentence about why you did it and what it's for. The same applies to a review. What did you confirm? Did you build it? Run checkpatch or some other static analysis? I think I'm also guilty of a one-line review now and then, but I'll be sure to include detail in the future.

Signature · 2 lines
Darren Hart
Intel Open Source Technology Center
#321 lore raw
Quoted 7 lines · On Thu, 9 Jul 2015 12:44:56 -0700 Darren Hart <[email protected]> wrote:

Agreed. I have this same conversation about commit messages. I don't care if it's a whitespace fix, if it is worth patching, building, testing, and submitting, it is worth writing a sentence about why you did it and what it's for. The same applies to a review. What did you confirm? Did you build it? Run checkpatch or some other static analysis? I think I'm also guilty of a one-line review now and then, but I'll be sure to include detail in the future.

Very good point.

Again, not talking about regular people that a maintainer knows well. But a Reviewed-by tag could at least also add what the reviewer did to add it. "I examined the entire patch, and found nothing wrong with it", is an acceptable comment. So is, "I looked at the patch and even built and booted it. Looks good".

Again, I have a few people I ask to review patches, and just them replying with "Reviewed-by" is good enough for me. Because I know them well and know what they usually do when they do a review. I don't need them to be constantly telling me what they did.

-- Steve

#322 lore raw
In reply to Steven Rostedt
Quoted 16 lines · Steven Rostedt
Quoted 11 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

Quoted 8 lines

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I was thinking of writing a perl script that would read my LKML archive and somewhat intelligently looking at people who replied to patches, that also has snippets of the patch in the reply, and counting them. I think that would be a more accurate use of real reviewers than just the Reviewed-by tag.

Agreed. Unless there is commentary accompanying the review, it doesn't add a lot. While the nod of approval builds incremental confidence, the true measure of a review is a resulting change to the patch for the better. Of course, a good review takes the same amount of time regardless of if a change is required.

Signature · 2 lines
Darren Hart
Intel Open Source Technology Center
#323 lore raw
Quoted 20 lines · Darren Hart
Quoted 19 lines · Steven Rostedt
Quoted 11 lines · On Wed, 08 Jul 2015 09:00:32 +0100 [email protected] wrote:
Quoted 5 lines

We can alter that somewhat. We used to run a Maintainers lottery for the kernel summit ... we could instead offer places based on the number of Reviewed-by: tags ... we have all the machinery to calculate that. I know an invitation to the kernel summit isn't a huge incentive, but it's a useful one.

Sounds like a good idea to me, though it would only effect a tiny percentage of our reviewers. I suppose publishing a short list of the top n% of reviewers from which the lottery runs might give some recognition.

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

Quoted 8 lines
Quoted 5 lines

I was thinking of writing a perl script that would read my LKML archive and somewhat intelligently looking at people who replied to patches, that also has snippets of the patch in the reply, and counting them. I think that would be a more accurate use of real reviewers than just the Reviewed-by tag.

Agreed. Unless there is commentary accompanying the review, it doesn't add a lot.

No, that is not always true. If I see a naked "reviewed-by" from a person who's been working on the subsystem quite a bit and shown a good judgement it is enough for me. I do not need them to find something to nitpick over so that there is "meat" to the review.

Quoted 3 lines

While the nod of approval builds incremental confidence, the true measure of a review is a resulting change to the patch for the better. Of course, a good review takes the same amount of time regardless of if a change is required.

Thanks.

Signature · 1 line
Dmitry
#324 lore raw
Quoted 7 lines · On Thu, 9 Jul 2015 13:21:52 -0700 Dmitry Torokhov <[email protected]>…
Quoted 2 lines

Agreed. Unless there is commentary accompanying the review, it doesn't add a lot.

No, that is not always true. If I see a naked "reviewed-by" from a person who's been working on the subsystem quite a bit and shown a good judgement it is enough for me. I do not need them to find something to nitpick over so that there is "meat" to the review.

And this is where the problem lies. Every review by tag is different, and it really matters from who it's from. I've had people send me a "Reviewed-by" tag that I had no idea who it is from, and with no commentary. That to me is worthless.

But I've sent patches out and have asked people like Masami Hiramatsu to review it, and if he returns just a "Reviewed-by" with no commentary, I take that as a positive, and that he didn't find anything wrong with what I wrote. And I gladly add that tag as it has meaning to me.

It really does come down to the maintainer. They should only add a Reviewed-by tag from someone they trust, or from someone that has given them constructive criticisms for the patch. I really don't know how strict maintainers of other subsystems are with regard to adding those tags.

I personally don't add that tag unless I have done an extensive review of the patch. I don't want some stupid bug come back to a patch that has my Reviewed-by tag on it. But I have reviewed lots of patches where I haven't put enough effort into adding a RB tag. Perhaps I should just add "Looks-fine-to-me-by:" tag, stating I did a light review but not a thorough one. ;-)

-- Steve

#325 lore raw
In reply to Dmitry Torokhov
Quoted 5 lines · Dmitry Torokhov

No, that is not always true. If I see a naked "reviewed-by" from a person who's been working on the subsystem quite a bit and shown a good judgement it is enough for me. I do not need them to find something to nitpick over so that there is "meat" to the review.

Absolutely. If I'm looking at a patch for ext4, and I see a bare
Reviewed-by: from Jan Kara, I treat that very differently compared to a Reviewed-by: coming from someone like Nick Krause.

The challenge is if we are using a scheme that uses some kind of automated counting of Reviewed-by lines, how do we make the system smart enough so it counts the former, but not the latter? Or worse, *encourages* more of the latter, which just adds more noise which actually increases the load on Maintainers, not decreases.

(Which is also my objection to people sending patches generated from "checkpatch --file" runs. Maybe it's fine in for staging code, but for other subsystems, it basically just means there are more patches that require review by someone clueful --- since sometimes "cleanup" patches from trolls actually introduce bugs.)

- Ted

#326 lore raw
Quoted 11 lines · Theodore Ts'o
Quoted 5 lines · Dmitry Torokhov

No, that is not always true. If I see a naked "reviewed-by" from a person who's been working on the subsystem quite a bit and shown a good judgement it is enough for me. I do not need them to find something to nitpick over so that there is "meat" to the review.

Absolutely. If I'm looking at a patch for ext4, and I see a bare
Reviewed-by: from Jan Kara, I treat that very differently compared to a Reviewed-by: coming from someone like Nick Krause.

Quoted 5 lines

The challenge is if we are using a scheme that uses some kind of automated counting of Reviewed-by lines, how do we make the system smart enough so it counts the former, but not the latter? Or worse, *encourages* more of the latter, which just adds more noise which actually increases the load on Maintainers, not decreases.

The Reviewed-by: tags are added to commits by people who ckeck them in, ie. maintainers. They should be responsible for using those tags wisely.

If the maintainers exercise enough due dilligence in that area, as they do with the SoB tags, it actually should be OK to collect and publish Reviewed-by:
statistics too.

Quoted 5 lines

(Which is also my objection to people sending patches generated from "checkpatch --file" runs. Maybe it's fine in for staging code, but for other subsystems, it basically just means there are more patches that require review by someone clueful --- since sometimes "cleanup" patches from trolls actually introduce bugs.)

Or mindless cleanups for that matter. I completely agree here.

Thanks,
Rafael

#327 lore raw
In reply to Theodore Ts'o
Quoted 11 lines · Theodore Ts'o
Quoted 5 lines · Dmitry Torokhov

No, that is not always true. If I see a naked "reviewed-by" from a person who's been working on the subsystem quite a bit and shown a good judgement it is enough for me. I do not need them to find something to nitpick over so that there is "meat" to the review.

Absolutely. If I'm looking at a patch for ext4, and I see a bare
Reviewed-by: from Jan Kara, I treat that very differently compared to a Reviewed-by: coming from someone like Nick Krause.

Quoted 5 lines

The challenge is if we are using a scheme that uses some kind of automated counting of Reviewed-by lines, how do we make the system smart enough so it counts the former, but not the latter? Or worse, *encourages* more of the latter, which just adds more noise which actually increases the load on Maintainers, not decreases.

I suppose the scan should read the tags from what is committed, not from LKML, then the maintainers are responsible to include only meaningful tags in the commit history? You then of course run the risk of putting someone off by ignoring their work - and I'd rather err in favor of the reviewer (false positive).

Signature · 2 lines
Darren Hart
Intel Open Source Technology Center
#328 lore raw
In reply to Theodore Ts'o
Quoted 7 lines · Theodore Ts'o
Quoted 5 lines · Dmitry Torokhov

No, that is not always true. If I see a naked "reviewed-by" from a person who's been working on the subsystem quite a bit and shown a good judgement it is enough for me. I do not need them to find something to nitpick over so that there is "meat" to the review.

Quoted 7 lines

Absolutely. If I'm looking at a patch for ext4, and I see a bare
Reviewed-by: from Jan Kara, I treat that very differently compared to a Reviewed-by: coming from someone like Nick Krause.

The challenge is if we are using a scheme that uses some kind of automated counting of Reviewed-by lines, how do we make the system smart enough so it counts the former, but not the latter?

I guess it's possible using various features of what the person has done in the past to devise a machine learning algorithm that could make this distinction.

julia

Quoted 16 lines

Or worse,
*encourages* more of the latter, which just adds more noise which actually increases the load on Maintainers, not decreases.

(Which is also my objection to people sending patches generated from "checkpatch --file" runs. Maybe it's fine in for staging code, but for other subsystems, it basically just means there are more patches that require review by someone clueful --- since sometimes "cleanup" patches from trolls actually introduce bugs.)

- Ted
_______________________________________________
Ksummit-discuss mailing list [email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

#329 lore raw
In reply to Steven Rostedt
Quoted 8 lines · Steven Rostedt

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I was thinking of writing a perl script that would read my LKML archive and somewhat intelligently looking at people who replied to patches, that also has snippets of the patch in the reply, and counting them. I think that would be a more accurate use of real reviewers than just the Reviewed-by tag.

I'm relatively new to lkml so my perceptions might be skewed. But it seems to me that, for non-trivial patches, the Reviewed-by tag is pretty much useless as a metric. From what I can tell, the biggest problems with relying on Reviewed-by to get meaningful statistics are:

1. It's self-reported by the reviewer.

Maybe a reviewer gave some hugely valuable feedback in versions 1-4 of the patch, but wasn't around to provide the Reviewed-by tag for the final v6.

Or maybe they're just focused on the review, and forget or don't care to ask for credit for it at the time by adding a tag.

Or maybe they're a maintainer who used Signed-off-by instead (but Signed-off-by doesn't necessarily imply a review).

Also, as others have mentioned, it creates the ability to game the system by saying you've reviewed a patch when you really haven't.

2. It's too broad of a statement. It requires the reviewer to make an official certification that they fully understand and approve of the *entire* patch. That's certainly a useful statement, but:

A reviewer might review only part of the patch, and provide some useful comments for just the part of the patch they reviewed. But they don't want to take responsibility for saying, to quote Steven, "I took enough effort to understand every part of the patch as if I wrote it myself."

Or maybe they didn't carefully review the patch, but instead added something valuable to the discussion which improved the patch in a future version.

So it denies credit to all the other people who provided useful comments along the way.

Review comments such as "here's a problem with your code" or "have you thought about doing this instead?" can often be more useful a comment than "your code looks good". Maybe it would be a good idea to acknowledge those people who give valuable feedback, in all its forms. Maybe a new tag would be useful? Something which:

a) is reported by the patch author and/or maintainer, since they're in the best position to keep track of useful comments across versions;

and

b) means something like "I'm grateful to this person for giving some valuable feedback".

Signature · 1 line
Josh
#330 lore raw
Quoted 15 lines · Josh Poimboeuf
Quoted 8 lines · Steven Rostedt

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I was thinking of writing a perl script that would read my LKML archive and somewhat intelligently looking at people who replied to patches, that also has snippets of the patch in the reply, and counting them. I think that would be a more accurate use of real reviewers than just the Reviewed-by tag.

I'm relatively new to lkml so my perceptions might be skewed. But it seems to me that, for non-trivial patches, the Reviewed-by tag is pretty much useless as a metric. From what I can tell, the biggest problems with relying on Reviewed-by to get meaningful statistics are:

Quoted 5 lines

1. It's self-reported by the reviewer.

Maybe a reviewer gave some hugely valuable feedback in versions 1-4 of the patch, but wasn't around to provide the Reviewed-by tag for the final v6.

Normally good maintainers pick up ack/review tags along the way when they pickup the final patch.

Quoted 6 lines

Or maybe they're just focused on the review, and forget or don't care to ask for credit for it at the time by adding a tag.

Or maybe they're a maintainer who used Signed-off-by instead (but Signed-off-by doesn't necessarily imply a review).

At least at some level any patches picked up by a maintainer should at least be compile tested and looked at any obvious issues -- when the maintainer trusts other devs/submaintainers that did the more thorough review process.

Quoted 3 lines

Also, as others have mentioned, it creates the ability to game the system by saying you've reviewed a patch when you really haven't.

The system can always be gamed.

Quoted 11 lines

2. It's too broad of a statement. It requires the reviewer to make an official certification that they fully understand and approve of the *entire* patch. That's certainly a useful statement, but:

A reviewer might review only part of the patch, and provide some useful comments for just the part of the patch they reviewed. But they don't want to take responsibility for saying, to quote Steven, "I took enough effort to understand every part of the patch as if I wrote it myself."

Quoted 6 lines

Or maybe they didn't carefully review the patch, but instead added something valuable to the discussion which improved the patch in a future version.

So it denies credit to all the other people who provided useful comments along the way.

Mentioning/thanking others that helped in any meaningful way is always a good idea.

Quoted 7 lines

Review comments such as "here's a problem with your code" or "have you thought about doing this instead?" can often be more useful a comment than "your code looks good". Maybe it would be a good idea to acknowledge those people who give valuable feedback, in all its forms. Maybe a new tag would be useful? Something which:

Quoted 7 lines

a) is reported by the patch author and/or maintainer, since they're in the best position to keep track of useful comments across versions;

and

b) means something like "I'm grateful to this person for giving some valuable feedback".

A tag for this? Nah.

Thanks,
Davidlohr

#331 lore raw
Quoted 16 lines · Davidlohr Bueso
Quoted 15 lines · Josh Poimboeuf
Quoted 8 lines · Steven Rostedt

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I was thinking of writing a perl script that would read my LKML archive and somewhat intelligently looking at people who replied to patches, that also has snippets of the patch in the reply, and counting them. I think that would be a more accurate use of real reviewers than just the Reviewed-by tag.

I'm relatively new to lkml so my perceptions might be skewed. But it seems to me that, for non-trivial patches, the Reviewed-by tag is pretty much useless as a metric. From what I can tell, the biggest problems with relying on Reviewed-by to get meaningful statistics are:

Quoted 8 lines
Quoted 5 lines

1. It's self-reported by the reviewer.

Maybe a reviewer gave some hugely valuable feedback in versions 1-4 of the patch, but wasn't around to provide the Reviewed-by tag for the final v6.

Normally good maintainers pick up ack/review tags along the way when they pickup the final patch.

A maintainer can't pick up the tag if the reviewer doesn't post it.

In this example the reviewer wouldn't have posted Reviewed-by because, despite the fact that they did a thorough review, the patch still had issues in its earlier versions. The reviewer did a lot of work but didn't get credit for it.

Quoted 10 lines
Quoted 5 lines

Or maybe they're just focused on the review, and forget or don't care to ask for credit for it at the time by adding a tag.

Or maybe they're a maintainer who used Signed-off-by instead (but Signed-off-by doesn't necessarily imply a review).

At least at some level any patches picked up by a maintainer should at least be compile tested and looked at any obvious issues -- when the maintainer trusts other devs/submaintainers that did the more thorough review process.

Right, as I said, Signed-off-by doesn't necessarily imply Reviewed-by.

So any review done by a maintainer who only adds Signed-off-by goes uncredited.

Quoted 20 lines
Quoted 2 lines

Also, as others have mentioned, it creates the ability to game the system by saying you've reviewed a patch when you really haven't.

The system can always be gamed.

Quoted 15 lines

2. It's too broad of a statement. It requires the reviewer to make an official certification that they fully understand and approve of the *entire* patch. That's certainly a useful statement, but:

A reviewer might review only part of the patch, and provide some useful comments for just the part of the patch they reviewed. But they don't want to take responsibility for saying, to quote Steven, "I took enough effort to understand every part of the patch as if I wrote it myself."

Or maybe they didn't carefully review the patch, but instead added something valuable to the discussion which improved the patch in a future version.

Quoted 5 lines
Quoted 2 lines

So it denies credit to all the other people who provided useful comments along the way.

Mentioning/thanking others that helped in any meaningful way is always a good idea.

Sure, but if you only thank them informally then their contribution can't be quantified, which was one of the proposed goals of this discussion.

Quoted 19 lines
Quoted 6 lines

Review comments such as "here's a problem with your code" or "have you thought about doing this instead?" can often be more useful a comment than "your code looks good". Maybe it would be a good idea to acknowledge those people who give valuable feedback, in all its forms. Maybe a new tag would be useful? Something which:

Quoted 7 lines

a) is reported by the patch author and/or maintainer, since they're in the best position to keep track of useful comments across versions;

and

b) means something like "I'm grateful to this person for giving some valuable feedback".

A tag for this? Nah.

Thanks,
Davidlohr

Signature · 1 line
Josh
#332 lore raw
Quoted 26 lines · Josh Poimboeuf
Quoted 16 lines · Davidlohr Bueso
Quoted 15 lines · Josh Poimboeuf
Quoted 8 lines · Steven Rostedt

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I was thinking of writing a perl script that would read my LKML archive and somewhat intelligently looking at people who replied to patches, that also has snippets of the patch in the reply, and counting them. I think that would be a more accurate use of real reviewers than just the Reviewed-by tag.

I'm relatively new to lkml so my perceptions might be skewed. But it seems to me that, for non-trivial patches, the Reviewed-by tag is pretty much useless as a metric. From what I can tell, the biggest problems with relying on Reviewed-by to get meaningful statistics are:

Quoted 8 lines
Quoted 5 lines

1. It's self-reported by the reviewer.

Maybe a reviewer gave some hugely valuable feedback in versions 1-4 of the patch, but wasn't around to provide the Reviewed-by tag for the final v6.

Normally good maintainers pick up ack/review tags along the way when they pickup the final patch.

Quoted 6 lines

A maintainer can't pick up the tag if the reviewer doesn't post it.

In this example the reviewer wouldn't have posted Reviewed-by because, despite the fact that they did a thorough review, the patch still had issues in its earlier versions. The reviewer did a lot of work but didn't get credit for it.

And that's not always a bad thing. Sometimes I review the previous version of a patch, and then see that tag applied to a later version that is substantially different from the version I reviewed; that makes me uncomfortable. I care less about getting credit for a review and more about not having my Reviewed-by tag attached to something I didn't review. (Obviously, this is a fuzzy thing; for instance, if three people note issues and offer a Reviewed-by with those issues fixed, fixing all the issues and applying all three Reviewed-by tags seems reasonable.)

- Josh Triplett

#333 lore raw
Quoted 34 lines · Josh Triplett
Quoted 26 lines · Josh Poimboeuf
Quoted 16 lines · Davidlohr Bueso
Quoted 15 lines · Josh Poimboeuf
Quoted 8 lines · Steven Rostedt

I personally don't trust a Reviewed-by tag much, as I sometimes see them appear without any comments.

I was thinking of writing a perl script that would read my LKML archive and somewhat intelligently looking at people who replied to patches, that also has snippets of the patch in the reply, and counting them. I think that would be a more accurate use of real reviewers than just the Reviewed-by tag.

I'm relatively new to lkml so my perceptions might be skewed. But it seems to me that, for non-trivial patches, the Reviewed-by tag is pretty much useless as a metric. From what I can tell, the biggest problems with relying on Reviewed-by to get meaningful statistics are:

Quoted 8 lines
Quoted 5 lines

1. It's self-reported by the reviewer.

Maybe a reviewer gave some hugely valuable feedback in versions 1-4 of the patch, but wasn't around to provide the Reviewed-by tag for the final v6.

Normally good maintainers pick up ack/review tags along the way when they pickup the final patch.

Quoted 6 lines

A maintainer can't pick up the tag if the reviewer doesn't post it.

In this example the reviewer wouldn't have posted Reviewed-by because, despite the fact that they did a thorough review, the patch still had issues in its earlier versions. The reviewer did a lot of work but didn't get credit for it.

Quoted 6 lines

And that's not always a bad thing. Sometimes I review the previous version of a patch, and then see that tag applied to a later version that is substantially different from the version I reviewed; that makes me uncomfortable. I care less about getting credit for a review and more about not having my Reviewed-by tag attached to something I didn't review.

If our goal is to quantify the work of good reviewers, so that we can shine a light on them and hopefully motivate them to do more review, then I'd argue that having a reviewer doing a lot of work and not getting credit for it *is* a bad thing.

I agree that the Reviewed-by tag doesn't apply here. It communicates patch approval, rather than review helpfulness. That's why I proposed a new tag from the patch author and/or maintainer.

Signature · 1 line
Josh
#334 lore raw
Quoted 3 lines · Josh Poimboeuf

I agree that the Reviewed-by tag doesn't apply here. It communicates patch approval, rather than review helpfulness. That's why I proposed a new tag from the patch author and/or maintainer.

I've lobbied for:

With-Fixes-from: xxx

It would only be given for actual bug fixes and not for complaining about the subject line, white space or spelling mistakes.

regards,
dan carpenter

#335 lore raw
Quoted 11 lines · Dan Carpenter
Quoted 3 lines · Josh Poimboeuf

I agree that the Reviewed-by tag doesn't apply here. It communicates patch approval, rather than review helpfulness. That's why I proposed a new tag from the patch author and/or maintainer.

I've lobbied for:

With-Fixes-from: xxx

It would only be given for actual bug fixes and not for complaining about the subject line, white space or spelling mistakes.

This seems like overengineering to me. I usually give a credit in the changelog if I find someone's contribution big enough. Also a persistent reviewer will eventually get his credit via Reviewed-by :). So I don't think yet another tag is really necessary.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#336 lore raw
Quoted 10 lines · Jan Kara
Quoted 9 lines · Dan Carpenter
Quoted 3 lines · Josh Poimboeuf

I agree that the Reviewed-by tag doesn't apply here. It communicates patch approval, rather than review helpfulness. That's why I proposed a new tag from the patch author and/or maintainer.

I've lobbied for:

With-Fixes-from: xxx

Quoted 7 lines
Quoted 2 lines

It would only be given for actual bug fixes and not for complaining about the subject line, white space or spelling mistakes.

This seems like overengineering to me. I usually give a credit in the changelog if I find someone's contribution big enough. Also a persistent reviewer will eventually get his credit via Reviewed-by :). So I don't think yet another tag is really necessary.

I could get as many reviewed-by tags as I wanted... I basically only write reviewed-by tags as a way of being nice to people sending patches.

Lots of people give credit for fixes in the changlog like but it's more common to leave it out. Also it's not searchable.

This happened hours ago in staging. Someone sent a buggy patch and reviewers spotted the bug but no one got credit.

http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072662.html
http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072668.html

Imagine how special Patrick would feel if we gave him a nice "With-fix-from: Patrick Farrell <[email protected]>" tag. :) Also the other rule would be that only the first person to report the bug gets the credit (Sorry, Sudip). The next version of the patch had a process problem which Sudip noticed as well but that still doesn't earn a with-fix-from tag.

regards,
dan carpenter

#337 lore raw
Quoted 37 lines · Dan Carpenter
Quoted 3 lines · Jan Kara
Quoted 2 lines · Dan Carpenter
Quoted 1 line · Josh Poimboeuf

I agree that the Reviewed-by tag doesn't apply here. It

communicates

Quoted 1 line
Quoted 1 line
Quoted 1 line

patch approval, rather than review helpfulness. That's why I

proposed a

Quoted 10 lines
Quoted 8 lines
Quoted 1 line

new tag from the patch author and/or maintainer.

I've lobbied for:

With-Fixes-from: xxx

It would only be given for actual bug fixes and not for complaining about the subject line, white space or spelling mistakes.

This seems like overengineering to me. I usually give a credit in the

changelog

Quoted 1 line

if I find someone's contribution big enough. Also a persistent

reviewer will

Quoted 1 line

eventually get his credit via Reviewed-by :). So I don't think yet

another

Quoted 1 line

tag is really necessary.

I could get as many reviewed-by tags as I wanted... I basically only write reviewed-by tags as a way of being nice to people sending patches.

Lots of people give credit for fixes in the changlog like but it's more common to leave it out. Also it's not searchable.

This happened hours ago in staging. Someone sent a buggy patch and reviewers spotted the bug but no one got credit.

http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072662.html
http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072668.html

Quoted 7 lines

Imagine how special Patrick would feel if we gave him a nice "With-fix-from: Patrick Farrell <[email protected]>" tag. :) Also the other
rule would be that only the first person to report the bug gets the credit (Sorry, Sudip). The next version of the patch had a process problem which Sudip noticed as well but that still doesn't earn a with-fix-from tag.

I like this idea in principle. Not entirely sure if it will work perfectly in practice.

Likely that in high revision count series some earlier work will not get credit...
Unless patch authors get to propose such tags as suggestions to the maintainer. Preferably with description in the change log! Guessing I am not the only maintainer who doesn't keep up with all the versions of some patches other than a quick controversy check!

Quoted 8 lines

regards,
dan carpenter

_______________________________________________
Ksummit-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
#338 lore raw
In reply to Dan Carpenter
Quoted 7 lines · Dan Carpenter
Quoted 2 lines · Jan Kara
Quoted 1 line · Dan Carpenter

On Sun, Jul 12, 2015 at 07:28:23AM -0500, Josh Poimboeuf wrote:

This happened hours ago in staging. Someone sent a buggy patch and reviewers spotted the bug but no one got credit.

Quoted 7 lines

http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072662.html
http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072668.html

Imagine how special Patrick would feel if we gave him a nice "With-fix-from: Patrick Farrell <[email protected]>" tag. :) Also the other rule would be that only the first person to report the bug gets the credit (Sorry, Sudip).

Doesn't matter. I am a volunteer and contributing here is not part of my dayjob. I do it because I like doing it.

Quoted 3 lines

The next version of the patch had a process
problem which Sudip noticed as well but that still doesn't earn a with-fix-from tag.

Then will the first reviewer always get the credit? Suppose a hypothetial situation where v1 is just having a bug like the one you mentioned, reviewer X points that out so he gets "With-fix-from:", but v2 has a more serious problem and results in build failure and reviewer Y points that out and v3 is the final patch that is accepted. So now who will get "With-fix-from:" ?

regards
sudip

#339 lore raw
Quoted 23 lines · Sudip Mukherjee
Quoted 8 lines · Dan Carpenter
Quoted 2 lines · Jan Kara
Quoted 1 line · Dan Carpenter

On Sun, Jul 12, 2015 at 07:28:23AM -0500, Josh Poimboeuf wrote:

This happened hours ago in staging. Someone sent a buggy patch and reviewers spotted the bug but no one got credit.

http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072662.html

Quoted 1 line

http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072668.html

Quoted 3 lines

Imagine how special Patrick would feel if we gave him a nice "With-fix-from: Patrick Farrell <[email protected]>" tag. :) Also the

other

Quoted 2 lines

rule would be that only the first person to report the bug gets the credit (Sorry, Sudip).

Doesn't matter. I am a volunteer and contributing here is not part of my dayjob. I do it because I like doing it.

Quoted 3 lines

The next version of the patch had a process
problem which Sudip noticed as well but that still doesn't earn a with-fix-from tag.

Quoted 9 lines

Then will the first reviewer always get the credit? Suppose a hypothetial
situation where v1 is just having a bug like the one you mentioned, reviewer X points that out so he gets "With-fix-from:", but v2 has a more
serious problem and results in build failure and reviewer Y points that out
and v3 is the final patch that is accepted. So now who will get "With-fix-from:" ?

Both surely. Different bug, different tag (though only one per bug finder)

Quoted 3 lines

regards
sudip

Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
#340 lore raw
In reply to Sudip Mukherjee
Quoted 9 lines · Sudip Mukherjee
Quoted 3 lines

The next version of the patch had a process
problem which Sudip noticed as well but that still doesn't earn a with-fix-from tag.

Then will the first reviewer always get the credit? Suppose a hypothetial situation where v1 is just having a bug like the one you mentioned, reviewer X points that out so he gets "With-fix-from:", but v2 has a more serious problem and results in build failure and reviewer Y points that out and v3 is the final patch that is accepted. So now who will get "With-fix-from:" ?

No no. Build fixes don't count... Only runtime bugs. But, yes, if there are two bugs you give credit for both.

regards,
dan carpenter

#341 lore raw
In reply to Sudip Mukherjee
Quoted 20 lines · Sudip Mukherjee
Quoted 7 lines · Dan Carpenter
Quoted 2 lines · Jan Kara
Quoted 1 line · Dan Carpenter

On Sun, Jul 12, 2015 at 07:28:23AM -0500, Josh Poimboeuf wrote:

This happened hours ago in staging. Someone sent a buggy patch and reviewers spotted the bug but no one got credit.

Quoted 7 lines

http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072662.html
http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072668.html

Imagine how special Patrick would feel if we gave him a nice "With-fix-from: Patrick Farrell <[email protected]>" tag. :) Also the other rule would be that only the first person to report the bug gets the credit (Sorry, Sudip).

Doesn't matter. I am a volunteer and contributing here is not part of my dayjob. I do it because I like doing it.

Quoted 3 lines

The next version of the patch had a process
problem which Sudip noticed as well but that still doesn't earn a with-fix-from tag.

Quoted 6 lines

Then will the first reviewer always get the credit? Suppose a hypothetial situation where v1 is just having a bug like the one you mentioned, reviewer X points that out so he gets "With-fix-from:", but v2 has a more serious problem and results in build failure and reviewer Y points that out and v3 is the final patch that is accepted. So now who will get "With-fix-from:" ?

Or even more important, how in the world will the maintainer be adding these tags in a simple manner? It's a manual thing for me to do today, which is a pain, and given that I have no "state" of previous patches remembering what happened in previous versions, I don't know how I would be expected to keep track of all of this.

So whatever scheme people come up with, please remember that it has to be easy to actually implement...

thanks,

greg k-h

#342 lore raw
Quoted 28 lines · Greg KH
Quoted 20 lines · Sudip Mukherjee
Quoted 7 lines · Dan Carpenter
Quoted 2 lines · Jan Kara
Quoted 1 line · Dan Carpenter

On Sun, Jul 12, 2015 at 07:28:23AM -0500, Josh Poimboeuf wrote:

This happened hours ago in staging. Someone sent a buggy patch and reviewers spotted the bug but no one got credit.

Quoted 7 lines

http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072662.html
http://driverdev.linuxdriverproject.org/pipermail/driverdev-devel/2015-July/072668.html

Imagine how special Patrick would feel if we gave him a nice "With-fix-from: Patrick Farrell <[email protected]>" tag. :) Also the other rule would be that only the first person to report the bug gets the credit (Sorry, Sudip).

Doesn't matter. I am a volunteer and contributing here is not part of my dayjob. I do it because I like doing it.

Quoted 3 lines

The next version of the patch had a process
problem which Sudip noticed as well but that still doesn't earn a with-fix-from tag.

Quoted 6 lines

Then will the first reviewer always get the credit? Suppose a hypothetial situation where v1 is just having a bug like the one you mentioned, reviewer X points that out so he gets "With-fix-from:", but v2 has a more serious problem and results in build failure and reviewer Y points that out and v3 is the final patch that is accepted. So now who will get "With-fix-from:" ?

Quoted 8 lines

Or even more important, how in the world will the maintainer be adding these tags in a simple manner? It's a manual thing for me to do today, which is a pain, and given that I have no "state" of previous patches remembering what happened in previous versions, I don't know how I would be expected to keep track of all of this.

So whatever scheme people come up with, please remember that it has to be easy to actually implement...

I think it would need to be up to the patch author, not the maintainer, to add the tag. The patch author already tracks the patch state from version to version. And the author is in the best position to know which comments are helpful.

Also I'd suggest something a little broader than "With-fix-from". Not all useful comments are "fixes". For example, an insightful question or comment can sometimes result in big changes in the next version of the patch.

Signature · 1 line
Josh
#343 lore raw
Quoted 4 lines · Josh Poimboeuf

Also I'd suggest something a little broader than "With-fix-from". Not all useful comments are "fixes". For example, an insightful question or comment can sometimes result in big changes in the next version of the patch.

I wouldn't want it to be too broad. Sending complaints about style issues is its own reward. Painting the woodshed etc.

regards,
dan carpenter

#344 lore raw
In reply to Greg KH
Quoted 8 lines · Greg KH

Or even more important, how in the world will the maintainer be adding these tags in a simple manner? It's a manual thing for me to do today, which is a pain, and given that I have no "state" of previous patches remembering what happened in previous versions, I don't know how I would be expected to keep track of all of this.

So whatever scheme people come up with, please remember that it has to be easy to actually implement...

It's like Reported-by: the patch sender adds it when they write the patch (re-write it in this case).

regards,
dan carpenter

#345 lore raw
In reply to Josh Poimboeuf
Quoted 5 lines · On Sat, 11 Jul 2015 22:48:24 -0500 Josh Poimboeuf <[email protected]> wrote:

Right, as I said, Signed-off-by doesn't necessarily imply Reviewed-by.

So any review done by a maintainer who only adds Signed-off-by goes uncredited.

A Signed-off-by sure as well better mean a reviewed by! It holds a much heavier meaning to the patch than a Reviewed-by does.

A Signed-off-by means that you are responsible for that patch. Even if you don't necessarily understand the patch (like a high level maintainer pulling in patches for low level hardware that the maintainer doesn't fully understand), you should review it as much as you can before adding an SoB to it. Otherwise you are not doing your job.

And regardless, there's more weight for "credit" to Signed-off-by's than for Reviewed-by's anyway. So much so, that my own scripts for analysis ignores all other tags for anyone with a Signed-off-by attached to a commit.

-- Steve

#346 lore raw
In reply to Peter Huewe
Quoted 4 lines · Peter Huewe

Hi,

In order to continue our traditions I would like to propose again the topic of recruitment, but this time not only limiting to the hobbyists market.

Agreed. I have a few fresh thoughts on this area. Not necessarily good ones, but at least fresh. :-P

Quoted 2 lines

We are definitely short on reviewers and thus have mostly overloaded maintainers.

Ack.

Quoted 6 lines

For testers it's usually even worse - how many patches are actually tested? Judging from what I read on LKML not that many.

So we should definitely discuss:
- how can we encourage hobbyists to become regular contributors -- how to keep people interested, the drop-out rates are huge.

Here we need to have the correct mindset. Kernel development is hard, detailed work. It's very rewarding, but simply put, most people aren't cut out to do it. I view the dropout rate as a *good* thing. It's a _selection_ process more than a education/training process.

With most of the hard jobs in life, take a look at the training/education program, and you'll see it: 80% drop out rate? That's selection. Kernel work is one of those 'hard jobs'.

This is important to realize because it changes how we view recruitment. We shouldn't be trying to keep everybody we recruit. Rather, we should be giving more people trial runs and see how they work out as they learn the process.

iow, if an 80% drop out rate gives us the caliber of dev we need for the long term health of the community, then it's a numbers game. Say we saw 40 new people last year, which turned into 8 regular contributors. Now we want to double that. We can lower the standard to get 16 out of 40, yuck. Or, we can outreach to 80 for trial runs, and get 16.

The discussion, I think, would revolve around: Am I smoking crack, or does everyone agree with this view? How can we create slots for newcomers to contribute real changes? Where do we reach out to for candidates? And, how do we quantify success or failure at some point in the future?

Quoted 3 lines

- encourage regular contributors to become reviewers and testers
- reviewers to become co-maintainers and finally maintainers (once the original maintainer is used up or moves up/on)

heh. 'used up' :-)

Quoted 7 lines

From the 4.1 kernel report by jon corbet:
"over 60% of the changes going into this kernel passed through the hands of developers working for just five companies. This concentration reflects a simple fact: while many companies are willing to support developers working on specific tasks, the number of companies supporting subsystem maintainers is far smaller. Subsystem maintainership is also, increasingly, not a job for volunteer developers.."

Well, I disagree. Not because I think Jon is wrong, but because we need to bring in new blood *somewhere*. co-maintainer/reviewer is one of many possibilities. If we don't have enough slots for newcomers (especially on a trial basis), that's our fault.

I think it's also worth noting how many of those kernel developers started out as hobbyist/volunteers. The truth is, once you've been around a few cycles as a hobbyist/volunteer, job offers come out of nowhere. So a statistic I'd love to see is the percentage of conversions.

Quoted 3 lines

-> How do we get companies to actively sponsor subsystem maintainership - if only at driver or subsubsystem level.
e.g.: I unfortunately failed at that with my company :/

meh, companies are fickle. I think it's enough for them to be paying a salary of a kernel dev/maintainer. It's a cost they can quantify the benefits of. If a company does a Crazy Ivan [1], then the kernel dev goes to work someplace else. iow, it's the human that matters and can be relied on, not the company.

Quoted 3 lines

Also I would be interested in:
- What are the effects of the Eudyptula Challenge? how many people are still actively contributing (more than whitespace changes)

If the Eudyptula folks have statistics, I'd love to see them.

Quoted 2 lines

Nominations:
Jason Cooper (auto-nominated), last years 'speaker'

I think it was the year before last, I couldn't make it last year. :-(

thx,

Jason.

[1] https://en.wikipedia.org/wiki/Crazy_Ivan (rapid, unexpected course changes)

#347 lore raw

Hey Jason,

Quoted 21 lines
Quoted 6 lines

For testers it's usually even worse - how many patches are actually tested? Judging from what I read on LKML not that many.

So we should definitely discuss:
- how can we encourage hobbyists to become regular contributors -- how to keep people interested, the drop-out rates are huge.

Here we need to have the correct mindset. Kernel development is hard, detailed work. It's very rewarding, but simply put, most people aren't cut out to do it. I view the dropout rate as a *good* thing. It's a _selection_ process more than a education/training process.

With most of the hard jobs in life, take a look at the training/education program, and you'll see it: 80% drop out rate? That's selection. Kernel work is one of those 'hard jobs'.

This is important to realize because it changes how we view recruitment. We shouldn't be trying to keep everybody we recruit. Rather, we should be giving more people trial runs and see how they work out as they learn the process.

Quoted 5 lines

iow, if an 80% drop out rate gives us the caliber of dev we need for the long term health of the community, then it's a numbers game. Say we saw 40 new people last year, which turned into 8 regular contributors. Now we want to double that. We can lower the standard to get 16 out of 40, yuck. Or, we can outreach to 80 for trial runs, and get 16.

I think that's an interesting take on the topic - although I'm not 100% whether I agree with everything. I agree that our goal is not to lower the standards, and also using more "trial runs".

However high standards should not be the reason to drive people away -- and especially not the reason not to keep good people interested.

Not only the bad people drop out, I've seen quite a lot of good devs vanish for good - and these should be the ones we also should try to keep - especially since I'm not sure whether we can allow such high drop out rates over a long time.

Quoted 4 lines

... but because we need
to bring in new blood *somewhere*. co-maintainer/reviewer is one of many possibilities. If we don't have enough slots for newcomers (especially on a trial basis), that's our fault.

Agreed.

Interesting discussion about this topic - thanks for your opinion!

Thanks,
Peter

#348 lore raw
Quoted 27 lines · Peter Huewe
Quoted 21 lines
Quoted 6 lines

For testers it's usually even worse - how many patches are actually tested? Judging from what I read on LKML not that many.

So we should definitely discuss:
- how can we encourage hobbyists to become regular contributors -- how to keep people interested, the drop-out rates are huge.

Here we need to have the correct mindset. Kernel development is hard, detailed work. It's very rewarding, but simply put, most people aren't cut out to do it. I view the dropout rate as a *good* thing. It's a _selection_ process more than a education/training process.

With most of the hard jobs in life, take a look at the training/education program, and you'll see it: 80% drop out rate? That's selection. Kernel work is one of those 'hard jobs'.

This is important to realize because it changes how we view recruitment. We shouldn't be trying to keep everybody we recruit. Rather, we should be giving more people trial runs and see how they work out as they learn the process.

Quoted 5 lines

iow, if an 80% drop out rate gives us the caliber of dev we need for the long term health of the community, then it's a numbers game. Say we saw 40 new people last year, which turned into 8 regular contributors. Now we want to double that. We can lower the standard to get 16 out of 40, yuck. Or, we can outreach to 80 for trial runs, and get 16.

Quoted 6 lines

I think that's an interesting take on the topic - although I'm not 100% whether I agree with everything. I agree that our goal is not to lower the standards, and also using more "trial runs".

However high standards should not be the reason to drive people away -- and especially not the reason not to keep good people interested.

Sure, I'm not suggesting anything like testing or anything formal. Merely that everyone understand there's an attrition rate, and that's *ok*.

Recruitment, imo, isn't about trying to keep everyone that tries. Rather it's about opening the doors and providing real opportunities for more people to contribute.

Quoted 4 lines

Not only the bad people drop out, I've seen quite a lot of good devs vanish for good - and these should be the ones we also should try to keep - especially since I'm not sure whether we can allow such high drop out rates over a long time.

I'd love to hear some specific examples, links to email threads so we can quantify this. I suspect a lot of it is: "I scratched the itch, and didn't have anything else I wanted to add. Then daily life took me away."

Which is the hard part about qualified people. They're busy. :-)

Perhaps this can help direct the outreach portion. Should we focus on students who have stretches of downtime? That certainly has pluses and minuses.

thx,

Jason.

#349 lore raw
Quoted 35 lines · Jason Cooper
Quoted 27 lines · Peter Huewe
Quoted 21 lines
Quoted 6 lines

For testers it's usually even worse - how many patches are actually tested? Judging from what I read on LKML not that many.

So we should definitely discuss:
- how can we encourage hobbyists to become regular contributors -- how to keep people interested, the drop-out rates are huge.

Here we need to have the correct mindset. Kernel development is hard, detailed work. It's very rewarding, but simply put, most people aren't cut out to do it. I view the dropout rate as a *good* thing. It's a _selection_ process more than a education/training process.

With most of the hard jobs in life, take a look at the training/education program, and you'll see it: 80% drop out rate? That's selection. Kernel work is one of those 'hard jobs'.

This is important to realize because it changes how we view recruitment. We shouldn't be trying to keep everybody we recruit. Rather, we should be giving more people trial runs and see how they work out as they learn the process.

Quoted 5 lines

iow, if an 80% drop out rate gives us the caliber of dev we need for the long term health of the community, then it's a numbers game. Say we saw 40 new people last year, which turned into 8 regular contributors. Now we want to double that. We can lower the standard to get 16 out of 40, yuck. Or, we can outreach to 80 for trial runs, and get 16.

Quoted 6 lines

I think that's an interesting take on the topic - although I'm not 100% whether I agree with everything. I agree that our goal is not to lower the standards, and also using more "trial runs".

However high standards should not be the reason to drive people away -- and especially not the reason not to keep good people interested.

Quoted 7 lines

Sure, I'm not suggesting anything like testing or anything formal. Merely that everyone understand there's an attrition rate, and that's *ok*.

Recruitment, imo, isn't about trying to keep everyone that tries. Rather it's about opening the doors and providing real opportunities for more people to contribute.

The opportunities are there, but in addition to the opportunities themselves people need some incentives to use them. It's all about their time which they can spend in many ways, some of them more rewarding than others.

If they don't have an incentive, an opportunity itself may not be sufficient.

Thanks,
Rafael

#350 lore raw
Quoted 45 lines · On Thursday 09 July 2015 01:09:00 Rafael J. Wysocki wrote:
Quoted 36 lines · Jason Cooper
Quoted 28 lines · Peter Huewe
Quoted 21 lines
Quoted 6 lines

For testers it's usually even worse - how many patches are actually tested? Judging from what I read on LKML not that many.

So we should definitely discuss:
- how can we encourage hobbyists to become regular contributors -- how to keep people interested, the drop-out rates are huge.

Here we need to have the correct mindset. Kernel development is hard, detailed work. It's very rewarding, but simply put, most people aren't cut out to do it. I view the dropout rate as a *good* thing. It's a _selection_ process more than a education/training process.

With most of the hard jobs in life, take a look at the training/education program, and you'll see it: 80% drop out rate? That's selection. Kernel work is one of those 'hard jobs'.

This is important to realize because it changes how we view recruitment. We shouldn't be trying to keep everybody we recruit. Rather, we should be giving more people trial runs and see how they work out as they learn the process.

Quoted 6 lines

iow, if an 80% drop out rate gives us the caliber of dev we need for the long term health of the community, then it's a numbers game. Say we saw 40 new people last year, which turned into 8 regular contributors. Now we want to double that. We can lower the standard to get 16 out of 40, yuck. Or, we can outreach to 80 for trial runs, and get 16.

Quoted 6 lines

I think that's an interesting take on the topic - although I'm not 100% whether I agree with everything. I agree that our goal is not to lower the standards, and also using more "trial runs".

However high standards should not be the reason to drive people away -- and especially not the reason not to keep good people interested.

Quoted 7 lines

Sure, I'm not suggesting anything like testing or anything formal. Merely that everyone understand there's an attrition rate, and that's *ok*.

Recruitment, imo, isn't about trying to keep everyone that tries. Rather it's about opening the doors and providing real opportunities for more people to contribute.

Quoted 6 lines

The opportunities are there, but in addition to the opportunities themselves people need some incentives to use them. It's all about their time which they can spend in many ways, some of them more rewarding than others.

If they don't have an incentive, an opportunity itself may not be sufficient.

We also need to ensure that the opportunities can be identified as such by potential newcomers. As core kernel contributors it's easy for us to see the myriad of places where a newcomer could help, but it's much harder to do so when you have no experience with kernel development. Projects such as kernel janitors can help in that regard, but might not be enough.

There's also the issue that the kernel (and its developers) is often seen as a scary beast. I've heard too many programmers saying that contributing to the Linux kernel is too difficult, without having given it a try. We are seen as experts by the outside world. There's nothing wrong with that as such, but I believe we should work on broadcasting the message that newcomers should still give it a try.

Signature · 3 lines
Regards,

Laurent Pinchart
#351 lore raw
In reply to Jason Cooper
Quoted 7 lines · Jason Cooper
Quoted 4 lines · Peter Huewe

Not only the bad people drop out, I've seen quite a lot of good devs vanish for good - and these should be the ones we also should try to keep - especially since I'm not sure whether we can allow such high drop out rates over a long time.

Quoted 6 lines

I'd love to hear some specific examples, links to email threads so we can quantify this. I suspect a lot of it is: "I scratched the itch, and didn't have anything else I wanted to add. Then daily life took me away."

Which is the hard part about qualified people. They're busy. :-)

Hopefully I'm not one of the "bad people", and I don't reallly consider myself a "drop out". But, I am someone that recently (i.e. since the last KS) extracted himself from a maintainer role. It seems like I should have something to add here...

I was the wireless maintainer for roughly seven years. My situation may be a bit unique in that I was always more of a "Linux guy" than a "wireless guy", and most of the contributors were bigger experts in the technology itself than I ever was. That was fine for a while, but over time that became less and less comfortable for me. I've never really heard anyone else express that sentiment, so this is probably not a widespread concern...

The bigger concern was that while I was wrangling everyone else's wireless patches, I had less and less time to do useful work elsewhere in the kernel. I definitely have heard other maintainers express similar complaints, so this seems like a relatively common concern. It would be good to find and promote maintainer organizations within subsystems that are less likely to monopolize the mainainer's development time. Previously we have had discussions of how the TIP tree is run, but I'm not sure that works well in every case. Are there other working models for this?

I guess I'm suggesting the opposite of a "professional maintainer". Some people thrive at being the center of a subsystem, as I did for some time with wireless. But burnout is a problem, and I think we can limit some of that if somehow we can encourage less expansive roles for individual maintainers.

John

Signature · 2 lines
John W. Linville		Someday the world will need a hero, and you
[email protected]			might be all we have.  Be ready.
#352 lore raw
Quoted 7 lines · John W. Linville

I guess I'm suggesting the opposite of a "professional maintainer". Some people thrive at being the center of a subsystem, as I did for some time with wireless. But burnout is a problem, and I think we can limit some of that if somehow we can encourage less expansive roles for individual maintainers.

I hear you.

However, having individual maintainers with a high level of responsibility and sense of ownership is one of the reasons why the Linux kernel development model works as well as it does.

In a corporate environment, there is often no sense of ownership in a specific piece of code. Quite often there _is_ no owner. As a result, engineers don't feel the need to keep the code clean. They need to make a change, they make it, they move on. The code gets more and more messy and buggy over time, and no one really cares.

Whatever we change, we need to make sure this doesn't happen with the Linux kernel, or it will fall apart quickly.

Guenter

#353 lore raw
Quoted 15 lines · Guenter Roeck
Quoted 7 lines · John W. Linville

I guess I'm suggesting the opposite of a "professional maintainer". Some people thrive at being the center of a subsystem, as I did for some time with wireless. But burnout is a problem, and I think we can limit some of that if somehow we can encourage less expansive roles for individual maintainers.

I hear you.

However, having individual maintainers with a high level of responsibility and sense of ownership is one of the reasons why the Linux kernel development model works as well as it does.

Quoted 8 lines

In a corporate environment, there is often no sense of ownership in a specific piece of code. Quite often there _is_ no owner. As a result, engineers don't feel the need to keep the code clean. They need to make a change, they make it, they move on. The code gets more and more messy and buggy over time, and no one really cares.

Whatever we change, we need to make sure this doesn't happen with the Linux kernel, or it will fall apart quickly.

That is a good point, and I agree in principle. I would _not_ want to see any sort of "everyone can commit" model. Still, I think there might be ways to spread the maintainership load a bit wider.

I guess the suggestion becomes a deeper maintainer hierarchy where maintainers only pull from sub-maintainers, and sub-maintainers handle individual patches and smaller pull requests from contributors. This is essentially the model used for wireless. That introduces the problem of increased patch merge latency, for which I have no good suggestions...

John

Signature · 2 lines
John W. Linville		Someday the world will need a hero, and you
[email protected]			might be all we have.  Be ready.
#354 lore raw
Quoted 6 lines · John W. Linville

I guess the suggestion becomes a deeper maintainer hierarchy where maintainers only pull from sub-maintainers, and sub-maintainers handle individual patches and smaller pull requests from contributors. This is essentially the model used for wireless. That introduces the problem of increased patch merge latency, for which I have no good suggestions...

We haven't found that to be a problem in mvebu/arm-soc. It does increase the lead time a bit. We stop accepting patches for the coming merge window around -rc5/6 or so. Judicious use of linux-next gives sub-trees good coverage before landing in arm-soc.

I wouldn't advise adding another layer to that tree though. Rather, as I said in my other email, co-maintaining with lead rotation works well.

thx,

Jason.

#355 lore raw
In reply to John W. Linville
Quoted 9 lines · John W. Linville

The bigger concern was that while I was wrangling everyone else's wireless patches, I had less and less time to do useful work elsewhere in the kernel. I definitely have heard other maintainers express similar complaints, so this seems like a relatively common concern. It would be good to find and promote maintainer organizations within subsystems that are less likely to monopolize the mainainer's development time. Previously we have had discussions of how the TIP tree is run, but I'm not sure that works well in every case. Are there other working models for this?

arm-soc is ran a bit like TIP, but not identically. It's usually Arnd and I who handle most of it, with Kevin Kilman stepping in in particular when Arnd goes on paternity leave.

On a whole, I can say that Arnd seems to have time left to deal with other parts of the kernel and posting patches, since he spends most or all of his work on upstream stuff while it's mostly a side activity for me.

One thing that we're doing that's quite helpful is that we hand off and mostly time-slice our maintainership, which means you get to be off from it for a while every now and then. That's different from TIP where they instead tend to have areas of the kernel that each co-maintainer focuses on. The last couple of months I've been overly busy with work and life and I've stepped away shortly, which our model makes fairly easy to do. I'm re-engaging now though, since Arnd is out on paternity leave again so me and Kevin are looking after things through the fall.

We also don't engage all that much with developers directly -- mostly new lead developers/maintainers on new platforms and people upstreaming new SoC families. Most of the day to day work is delegated to per-vendor maintainers below us and we work with them on the back end and merge their branches.

I think we've talked about how we handle our work in the past, but I'd be happy to elaborate more (here over email or elsewhere) if anyone wants to learn more about what we've found to work pretty well for us.

-Olof

#356 lore raw
In reply to John W. Linville
Quoted 20 lines · John W. Linville
Quoted 7 lines · Jason Cooper
Quoted 4 lines · Peter Huewe

Not only the bad people drop out, I've seen quite a lot of good devs vanish for good - and these should be the ones we also should try to keep - especially since I'm not sure whether we can allow such high drop out rates over a long time.

Quoted 6 lines

I'd love to hear some specific examples, links to email threads so we can quantify this. I suspect a lot of it is: "I scratched the itch, and didn't have anything else I wanted to add. Then daily life took me away."

Which is the hard part about qualified people. They're busy. :-)

Hopefully I'm not one of the "bad people", and I don't reallly consider myself a "drop out". But, I am someone that recently (i.e. since the last KS) extracted himself from a maintainer role. It seems like I should have something to add here...

Quoted 7 lines

I was the wireless maintainer for roughly seven years. My situation may be a bit unique in that I was always more of a "Linux guy" than a "wireless guy", and most of the contributors were bigger experts in the technology itself than I ever was. That was fine for a while, but over time that became less and less comfortable for me. I've never really heard anyone else express that sentiment, so this is probably not a widespread concern...

I've certainly found this to be the case for me where my little corner of the kernel, platform-drivers-x86, is basically an integration point for many other subsystems, such as rfkill, backlight, input, etc. with a lot of ACPI thrown in the mix. I also never have the hardware to be able to do any testing myself. I'm 100% dependent on my submitters for any testing beyond build and static analysis.

Quoted 8 lines

The bigger concern was that while I was wrangling everyone else's wireless patches, I had less and less time to do useful work elsewhere in the kernel. I definitely have heard other maintainers express similar complaints, so this seems like a relatively common concern. It would be good to find and promote maintainer organizations within subsystems that are less likely to monopolize the mainainer's development time.

For me, the concern is that I'm basically acting as a pro-bono proxy for various vendors who don't maintain drivers for the products. So speaking of recruitment, we could use the vendors to get more involved with the drivers for their platforms. Some, Dell and Toshiba most notably, have been providing more documentation, fortunately.

Quoted 9 lines

Previously we have had discussions of how the
TIP tree is run, but I'm not sure that works well in every case. Are there other working models for this?

I guess I'm suggesting the opposite of a "professional maintainer". Some people thrive at being the center of a subsystem, as I did for some time with wireless. But burnout is a problem, and I think we can limit some of that if somehow we can encourage less expansive roles for individual maintainers.

Something along these lines is probably relevant to a number of subsystems where the professional maintainer is problematic since some to all of the content is of no interest to the employer (unless it's the LF).

Signature · 2 lines
Darren Hart
Intel Open Source Technology Center
#357 lore raw
In reply to John W. Linville
Quoted 13 lines · John W. Linville
Quoted 12 lines · Jason Cooper
Quoted 4 lines · Peter Huewe

Not only the bad people drop out, I've seen quite a lot of good devs vanish for good - and these should be the ones we also should try to keep - especially since I'm not sure whether we can allow such high drop out rates over a long time.

I'd love to hear some specific examples, links to email threads so we can quantify this. I suspect a lot of it is: "I scratched the itch, and didn't have anything else I wanted to add. Then daily life took me away."

Quoted 4 lines
Quoted 1 line

Which is the hard part about qualified people. They're busy. :-)

Hopefully I'm not one of the "bad people", and I don't reallly consider myself a "drop out".

Absolutely not. It was a sad day when I heard you were stepping down.

fwiw, I reached out to someone who I felt was capable to see if he could help. Unfortunately, he had some PhD-thesis thingy going on. :( Back to the 'qualified people are busy' ...

Our greatest mistake is if we don't learn from your experience and make the system better.

Quoted 11 lines

But, I am someone that recently
(i.e. since the last KS) extracted himself from a maintainer role. It seems like I should have something to add here...

I was the wireless maintainer for roughly seven years. My situation may be a bit unique in that I was always more of a "Linux guy" than a "wireless guy", and most of the contributors were bigger experts in the technology itself than I ever was. That was fine for a while, but over time that became less and less comfortable for me. I've never really heard anyone else express that sentiment, so this is probably not a widespread concern...

I certainly don't have a lot of the hardware people are submitting patches for, ergo ...

Quoted 9 lines

The bigger concern was that while I was wrangling everyone else's wireless patches, I had less and less time to do useful work elsewhere in the kernel. I definitely have heard other maintainers express similar complaints, so this seems like a relatively common concern. It would be good to find and promote maintainer organizations within subsystems that are less likely to monopolize the mainainer's development time. Previously we have had discussions of how the TIP tree is run, but I'm not sure that works well in every case. Are there other working models for this?

Olof has already mentioned how arm-soc is handled. As a submitter of pull requests to them for years, I can vouch that it works quite well. Additionally, mach-mvebu is co-maintained by myself and three others (there are multiple SoCs in there, we each have our focus).

Since I started a new job (hence less time for kernel work), we've rotated out who takes the lead for each cycle. It's a *huge* relief to know it's not up to me *every* cycle.

Similar experience with Thomas and I on irqchip.

tl;dr: co-maintaining works, works well, and helps avoid burnout. It also opens the door for part-time involvement.

Quoted 5 lines

I guess I'm suggesting the opposite of a "professional maintainer". Some people thrive at being the center of a subsystem, as I did for some time with wireless. But burnout is a problem, and I think we can limit some of that if somehow we can encourage less expansive roles for individual maintainers.

Fully agree.

thx,

Jason.

#358 lore raw
In reply to Peter Huewe
Quoted 3 lines · Peter Huewe

-> How do we get companies to actively sponsor subsystem maintainership - if only at driver or subsubsystem level.
e.g.: I unfortunately failed at that with my company :/

One thing I found worked really well when I was working for a silicon vendor was using employing maintainers as evidence of good software support for shiny new features on Linux, both internally and externally. That's obviously somewhat situation specific but it's one route.