Skip to contentSkip to conversation

DT, maintainership, development process

Ksummit-2013-discuss Rafael J. Wysocki 10 participants 14 messages Jul 29, 2013 – Aug 1, 2013 4 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): 20130729102452.GA2682@zurbaran

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 5 lines · Jiri Kosina
Quoted 3 lines · Samuel Ortiz

- Developement process: Jon Corbet's article about our changelog raised one interesting question: How can we better track how single SOB patches are getting merged ?

That actually is simple enough.

Check out the Linus' master branch and do

$ git log --ancestry-path --merges <commit>..

and read the log from the bottom to the top.

Quoted 11 lines
Quoted 5 lines

Should maintainers add their SOB when
they pull from subsystems maintainers (If they can) ? This is something I care about from both my MFD and NFC maintainer perspectives as I both pull from MFD sub maintainers and am John Linville's NFC sub maintainer.

This is difficult ... I think that in some sense merging from subsystem maintainer tree can be viewed as an "implicit" Signoff, although this hasn't been formalized anywhere.

I am afraid there is no way git could handle this easily without rebasing.

No, there's none, as you'll always change the commit hash by adding tags to it.

Thanks,
Rafael

#3 lore raw
Quoted 5 lines · On Mon, 29 Jul 2013 15:10:33 +0200 "Rafael J. Wysocki" <[email protected]> wrote:

That actually is simple enough.

Check out the Linus' master branch and do

$ git log --ancestry-path --merges <commit>..

So I picked a recent, single-signoff patch mostly at random:

	8ade62857ef77bdf639185410fbcd811aa700cb2

That "git log" command gives me 156 intervening commits involving at least a dozen networking trees, along with virtio, parisc, blackfin, x86, ... Even if one stops looking at the first merge performed by Linus, that's 47 to look at. Did that patch really pass through all those peoples' hands?

Plus, of course, this assumes there's no fast-forward merges in the mix.

Quoted 5 lines

From your other message:

And what about trusting maintainers? If Linus trusts them enough to pull from them, why can't everybody else trust them enough to assume that they don't do bad things on purpose?

I hope you're not reading such thoughts into what *I* wrote. But most anybody who works on code occasionally does bad things by mistake, that's why we have a review process.

jon

#4 lore raw
Quoted 9 lines · Jonathan Corbet
Quoted 5 lines · On Mon, 29 Jul 2013 15:10:33 +0200 "Rafael J. Wysocki" <[email protected]> wrote:

That actually is simple enough.

Check out the Linus' master branch and do

$ git log --ancestry-path --merges <commit>..
Quoted 8 lines

So I picked a recent, single-signoff patch mostly at random:

	8ade62857ef77bdf639185410fbcd811aa700cb2

That "git log" command gives me 156 intervening commits involving at least a dozen networking trees, along with virtio, parisc, blackfin, x86, ... Even if one stops looking at the first merge performed by Linus, that's 47 to look at. Did that patch really pass through all those peoples' hands?

Try this:

$ git describe --contains 8ade62857ef77bdf6391
v3.11-rc1~16^2~103^2^2~5^2~25

Looks scary, but lets deconstruct it right to left a chunk at a time v3.11-rc1 ~16 ^2 ~103 ^2 ^2 ~5 ^2 ~25

So, your commit is 25 commits back in the 2nd branch of the merge described by v3.11-rc1~16^2~103^2^2~5

git show v3.11-rc1~16^2~103^2^2~5
commit 76015021aaaa288bb4dc870258e0acc4d178c950
Merge: f7a01ca fded313
Author: John W. Linville <[email protected]>
Date: Mon Jun 10 14:51:17 2013 -0400

Merge branch 'for-john' of git://git.kernel.org/pub/scm/linux/kernel/git/iwlwifi/iwlwifi-next

which is by definition, 5 commits back from 2nd branch of the merge v3.11-rc1~16^2~103^2

$ git show v3.11-rc1~16^2~103^2
commit 3899ba90a4ab2f3cab8e0f91a76c14ff131c8293
Merge: 45203a3 8b3e7be
Author: John W. Linville <[email protected]>
Date:   Tue Jun 11 14:48:32 2013 -0400

Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-next into for-davem

and that merge was merged into the merge v3.11-rc1~16^2~103

git show v3.11-rc1~16^2~103
commit 4a2e667ac15edd19b02321bc030acb3ebeb22ab6
Merge: 8f728d7 3899ba9
Author: David S. Miller <[email protected]>
Date: Wed Jun 12 14:23:41 2013 -0700

Merge branch 'for-davem' of git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-next

which means the above merge was 103 commits back from the last (last = latest/newest) commit on the 2nd arm in merge v3.11-rc1~16 and so on.

All the data is there, regardless of whether a maintainer manually manipulated the patch and added a SOB line or not.

Paul.
--

Quoted 20 lines

Plus, of course, this assumes there's no fast-forward merges in the mix.

Quoted 5 lines

From your other message:

And what about trusting maintainers? If Linus trusts them enough to pull from them, why can't everybody else trust them enough to assume that they don't do bad things on purpose?

I hope you're not reading such thoughts into what *I* wrote. But most anybody who works on code occasionally does bad things by mistake, that's why we have a review process.

jon
_______________________________________________
Ksummit-2013-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-2013-discuss

#5 lore raw
Quoted 53 lines · Paul Gortmaker
Quoted 6 lines
Quoted 5 lines

That actually is simple enough.

Check out the Linus' master branch and do

$ git log --ancestry-path --merges <commit>..
Quoted 8 lines

So I picked a recent, single-signoff patch mostly at random:

	8ade62857ef77bdf639185410fbcd811aa700cb2

That "git log" command gives me 156 intervening commits involving at least a dozen networking trees, along with virtio, parisc, blackfin, x86, ... Even if one stops looking at the first merge performed by Linus, that's 47 to look at. Did that patch really pass through all those peoples' hands?

Try this:

$ git describe --contains 8ade62857ef77bdf6391
v3.11-rc1~16^2~103^2^2~5^2~25

Looks scary, but lets deconstruct it right to left a chunk at a time v3.11-rc1 ~16 ^2 ~103 ^2 ^2 ~5 ^2 ~25

So, your commit is 25 commits back in the 2nd branch of the merge described by v3.11-rc1~16^2~103^2^2~5

git show v3.11-rc1~16^2~103^2^2~5
commit 76015021aaaa288bb4dc870258e0acc4d178c950
Merge: f7a01ca fded313
Author: John W. Linville <[email protected]>
Date: Mon Jun 10 14:51:17 2013 -0400

Merge branch 'for-john' of git://git.kernel.org/pub/scm/linux/kernel/git/iwlwifi/iwlwifi-next

which is by definition, 5 commits back from 2nd branch of the merge v3.11-rc1~16^2~103^2

$ git show v3.11-rc1~16^2~103^2
commit 3899ba90a4ab2f3cab8e0f91a76c14ff131c8293
Merge: 45203a3 8b3e7be
Author: John W. Linville <[email protected]>
Date:   Tue Jun 11 14:48:32 2013 -0400

Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-next into for-davem

and that merge was merged into the merge v3.11-rc1~16^2~103

git show v3.11-rc1~16^2~103
commit 4a2e667ac15edd19b02321bc030acb3ebeb22ab6
Merge: 8f728d7 3899ba9
Author: David S. Miller <[email protected]>
Date: Wed Jun 12 14:23:41 2013 -0700

Quoted 8 lines

Merge branch 'for-davem' of git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-next

which means the above merge was 103 commits back from the last (last = latest/newest) commit on the 2nd arm in merge v3.11-rc1~16 and so on.

All the data is there, regardless of whether a maintainer manually manipulated the patch and added a SOB line or not.

Alternatively (and that's what I am using) you fire up gitk, and you see all the information almost instantly.

Signature · 2 lines
Jiri Kosina
SUSE Labs
#6 lore raw
In reply to Jonathan Corbet
Quoted 11 lines · Jonathan Corbet
Quoted 5 lines · On Mon, 29 Jul 2013 15:10:33 +0200 "Rafael J. Wysocki" <[email protected]> wrote:

That actually is simple enough.

Check out the Linus' master branch and do

$ git log --ancestry-path --merges <commit>..

So I picked a recent, single-signoff patch mostly at random:

Quoted 8 lines
	8ade62857ef77bdf639185410fbcd811aa700cb2

That "git log" command gives me 156 intervening commits involving at least a dozen networking trees, along with virtio, parisc, blackfin, x86, ... Even if one stops looking at the first merge performed by Linus, that's 47 to look at. Did that patch really pass through all those peoples' hands?

Plus, of course, this assumes there's no fast-forward merges in the mix.

Well, let's just say that the "git describe --contains" method described by Paul (http://marc.info/?l=linux-kernel&m=137511549202238&w=4) is better and using gitk also works, as noted by Jiri
(http://marc.info/?l=linux-kernel&m=137511562702289&w=4), but the point is that the information is there and the effort needed to extract it is not overwhelming.

What SOBs are useful for is to track the origins of a given patch before it enters a git repository, but once that's happened, all of the history is recorded by git.

Quoted 7 lines

From your other message:

Quoted 3 lines

And what about trusting maintainers? If Linus trusts them enough to pull from them, why can't everybody else trust them enough to assume that they don't do bad things on purpose?

I hope you're not reading such thoughts into what *I* wrote.

No, I'm not.

And in fact this last paragraph of my previous message sounds much stronger than I wanted it to. I didn't mean things like adding bad code to the kernel in a sneaky way, which in my opinion is out of the question. I meant things like making changes with the maintainer's own SOB without posting them for public review beforehand, which would be an abuse of the process even if the changes themselves were useful and correct.

Quoted 2 lines

But most anybody who works on code occasionally does bad things by mistake, that's why we have a review process.

Of course.

But that has nothing to do with how many SOB tags there are in the given commit log. There are submitters whose patches already have a couple of them when they are first posted.

A single SOB tag usually means that the committer himself is the author of the change set and I don't see why this should be regarded as a bad thing in principle. Yes, it is technically possible for maintainers to "cheat", for example by making unreviewed changes and pushing them upstream with their own SOBs even without any linux-next testing, but they can do damage in some other ways too if they are irresponsible.

We generally don't record information about what mailing lists the given patch was submitted and how much time the maintainer waited for comments before applying that patch. I suppose we possibly could record it, but then I'm not sure how useful that will be in general. It definitely would mean more work for maintainers and it's not like they don't have enough of that already. Moreover, perhaps we can simply expect maintainers not to abuse the process?

I guess my point is that the fact that there are commits with one SOB tag only doesn't have to mean that we have a problem of any sort and it even doesn't have to indicate the existence of such a problem.

Commits that have never been in linux-next are much more problematic in my opinion.

Thanks,
Rafael

Signature · 2 lines
I speak only for myself.
Rafael J. Wysocki, Intel Open Source Technology Center.
#7 lore raw
Quoted 14 lines · Rafael J. Wysocki

A single SOB tag usually means that the committer himself is the author of the change set and I don't see why this should be regarded as a bad thing in principle. Yes, it is technically possible for maintainers to "cheat", for example by making unreviewed changes and pushing them upstream with their own SOBs even without any linux-next testing, but they can do damage in some other ways too if they are irresponsible.

We generally don't record information about what mailing lists the given patch was submitted and how much time the maintainer waited for comments before applying that patch. I suppose we possibly could record it, but then I'm not sure how useful that will be in general. It definitely would mean more work for maintainers and it's not like they don't have enough of that already. Moreover, perhaps we can simply expect maintainers not to abuse the process?

Quoted 6 lines

I guess my point is that the fact that there are commits with one SOB tag only doesn't have to mean that we have a problem of any sort and it even doesn't have to indicate the existence of such a problem.

Commits that have never been in linux-next are much more problematic in my opinion.

And we still have (too many of) them...

Once in a while, I do find suspicious commits that
(a) weren't in -next,
(b) weren't in my email archive (not unreasonable, as I'm not
subscribed to all
Linux mailing lists ;-),
(c) are not to be found by Google, which means they may not have been posted for public review at all (are there Linux mailing lists that do not have a web archive?).

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#8 lore raw
Quoted 24 lines · Geert Uytterhoeven
Quoted 14 lines · Rafael J. Wysocki

A single SOB tag usually means that the committer himself is the author of the change set and I don't see why this should be regarded as a bad thing in principle. Yes, it is technically possible for maintainers to "cheat", for example by making unreviewed changes and pushing them upstream with their own SOBs even without any linux-next testing, but they can do damage in some other ways too if they are irresponsible.

We generally don't record information about what mailing lists the given patch was submitted and how much time the maintainer waited for comments before applying that patch. I suppose we possibly could record it, but then I'm not sure how useful that will be in general. It definitely would mean more work for maintainers and it's not like they don't have enough of that already. Moreover, perhaps we can simply expect maintainers not to abuse the process?

Quoted 6 lines

I guess my point is that the fact that there are commits with one SOB tag only doesn't have to mean that we have a problem of any sort and it even doesn't have to indicate the existence of such a problem.

Commits that have never been in linux-next are much more problematic in my opinion.

And we still have (too many of) them...

Quoted 8 lines

Once in a while, I do find suspicious commits that
(a) weren't in -next,
(b) weren't in my email archive (not unreasonable, as I'm not
subscribed to all
Linux mailing lists ;-),
(c) are not to be found by Google, which means they may not have been posted for public review at all (are there Linux mailing lists that do not have a web archive?).

I wonder if it's feasible to publish a list of such commits once in a while?

It doesn't have to be complete, I guess.

Rafael

A message that is not in the archive (sent off-list or not yet received): 20130729113526.GB3604@zurbaran

A message that is not in the archive (sent off-list or not yet received): [email protected]

#11 lore raw
Quoted 26 lines · Jason Cooper
Quoted 25 lines · Samuel Ortiz
Quoted 15 lines · Jason Cooper
Quoted 14 lines · Jiri Kosina
Quoted 7 lines · Samuel Ortiz

- Developement process: Jon Corbet's article about our changelog raised one interesting question: How can we better track how single SOB patches are getting merged ? Should maintainers add their SOB when they pull from subsystems maintainers (If they can) ? This is something I care about from both my MFD and NFC maintainer perspectives as I both pull from MFD sub maintainers and am John Linville's NFC sub maintainer.

This is difficult ... I think that in some sense merging from subsystem maintainer tree can be viewed as an "implicit" Signoff, although this hasn't been formalized anywhere.

Quoted 6 lines
Quoted 1 line

I am afraid there is no way git could handle this easily without rebasing.

And that breaks workflows where eg sub-maintainer and maintainer both contribute to linux-next. commit-id's _must_ match. As an example, our mvebu tree and arm-soc both contribute to linux-next and arm-soc pulls our branches.

Yes, I'm not saying we should rebase, don't get me wrong.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Perhaps the merge commit is the place for this?

That would be a first step, but that would obviously not fix the single SOB patches issue.

Well, is this that maintainers aren't adding a SoB when they apply patches from others? Or, that maintainers are applying their own patches, potentially without review?

The purported issue seems to be that some maintainers may apply their own patches without review and then those commits are difficult to track.

I honestly don't think they are difficult to track as I said here:

http://marc.info/?l=linux-kernel&m=137510284529196&w=4

Whether or not those patches are applied without review is difficult to establish even if they are posted to mailing lists before applying.

That said we have the same issue with commits with just two SOB tags if a maintainer applies a patch that nobody has responded to. Are they going to be regarded as "suspicious" too now?

And what about trusting maintainers? If Linus trusts them enough to pull from them, why can't everybody else trust them enough to assume that they don't do bad things on purpose?

Rafael

Signature · 2 lines
I speak only for myself.
Rafael J. Wysocki, Intel Open Source Technology Center.
#12 lore raw
Quoted 7 lines · Rafael J. Wysocki

That said we have the same issue with commits with just two SOB tags if a maintainer applies a patch that nobody has responded to. Are they going to be regarded as "suspicious" too now?

And what about trusting maintainers? If Linus trusts them enough to pull from them, why can't everybody else trust them enough to assume that they don't do bad things on purpose?

Not just Linus -- it's 'turtles all the way down' here. As someone else suggested, a Singed-off-by in the merge commit should suffice here. Although, I haven't always made a habit of adding S-o-b to merge commits either...

John

Signature · 2 lines
John W. Linville		Someday the world will need a hero, and you
[email protected]			might be all we have.  Be ready.
#13 lore raw
Quoted 6 lines · John W. Linville
Quoted 4 lines · Rafael J. Wysocki

That said we have the same issue with commits with just two SOB tags if a maintainer applies a patch that nobody has responded to. Are they going to be regarded as "suspicious" too now?

Quoted 8 lines
Quoted 3 lines

And what about trusting maintainers? If Linus trusts them enough to pull from them, why can't everybody else trust them enough to assume that they don't do bad things on purpose?

Not just Linus -- it's 'turtles all the way down' here. As someone else suggested, a Singed-off-by in the merge commit should suffice here. Although, I haven't always made a habit of adding S-o-b to merge commits either...

Even then, you are the author of the merge commit. So the original question of tracking a 'chain-of-custody' from submitter to Linus' tree is still answerable. Even if there is only a single SoB in the patch.

thx,

Jason.

#14 lore raw
In reply to John W. Linville
Quoted 6 lines · John W. Linville
Quoted 4 lines · Rafael J. Wysocki

That said we have the same issue with commits with just two SOB tags if a maintainer applies a patch that nobody has responded to. Are they going to be regarded as "suspicious" too now?

Quoted 8 lines
Quoted 3 lines

And what about trusting maintainers? If Linus trusts them enough to pull from them, why can't everybody else trust them enough to assume that they don't do bad things on purpose?

Not just Linus -- it's 'turtles all the way down' here. As someone else suggested, a Singed-off-by in the merge commit should suffice here. Although, I haven't always made a habit of adding S-o-b to merge commits either...

An SOB in the merge doesn't provide any additional information that can't be retrieved from git, unless you use a different e-mail address for the sign-off. :-)

Rafael

#15 lore raw
Quoted 11 lines · Rafael J. Wysocki
Quoted 10 lines · John W. Linville
Quoted 7 lines · Rafael J. Wysocki

That said we have the same issue with commits with just two SOB tags if a maintainer applies a patch that nobody has responded to. Are they going to be regarded as "suspicious" too now?

And what about trusting maintainers? If Linus trusts them enough to pull from them, why can't everybody else trust them enough to assume that they don't do bad things on purpose?

Quoted 9 lines
Quoted 4 lines

Not just Linus -- it's 'turtles all the way down' here. As someone else suggested, a Singed-off-by in the merge commit should suffice here. Although, I haven't always made a habit of adding S-o-b to merge commits either...

An SOB in the merge doesn't provide any additional information that can't be retrieved from git, unless you use a different e-mail address for the sign-off. :-)

Watch out for fast forward merges. Ideally I guess maintainers really should disable fast forwards and PGP-sign their merge commits...

-hpa

#16 lore raw
Quoted 17 lines · H. Peter Anvin
Quoted 16 lines · Rafael J. Wysocki
Quoted 14 lines · John W. Linville
Quoted 7 lines · Rafael J. Wysocki

That said we have the same issue with commits with just two SOB tags if a maintainer applies a patch that nobody has responded to. Are they going to be regarded as "suspicious" too now?

And what about trusting maintainers? If Linus trusts them enough to pull from them, why can't everybody else trust them enough to assume that they don't do bad things on purpose?

Not just Linus -- it's 'turtles all the way down' here. As someone else suggested, a Singed-off-by in the merge commit should suffice here. Although, I haven't always made a habit of adding S-o-b to merge commits either...

Quoted 7 lines
Quoted 4 lines

An SOB in the merge doesn't provide any additional information that can't be retrieved from git, unless you use a different e-mail address for the sign-off. :-)

Watch out for fast forward merges. Ideally I guess maintainers really should disable fast forwards and PGP-sign their merge commits...

If you always pull from a signed tag, this isn't a problem.

James

#17 lore raw
In reply to H. Peter Anvin
Quoted 17 lines · H. Peter Anvin
Quoted 16 lines · Rafael J. Wysocki
Quoted 14 lines · John W. Linville
Quoted 7 lines · Rafael J. Wysocki

That said we have the same issue with commits with just two SOB tags if a maintainer applies a patch that nobody has responded to. Are they going to be regarded as "suspicious" too now?

And what about trusting maintainers? If Linus trusts them enough to pull from them, why can't everybody else trust them enough to assume that they don't do bad things on purpose?

Not just Linus -- it's 'turtles all the way down' here. As someone else suggested, a Singed-off-by in the merge commit should suffice here. Although, I haven't always made a habit of adding S-o-b to merge commits either...

Quoted 7 lines
Quoted 4 lines

An SOB in the merge doesn't provide any additional information that can't be retrieved from git, unless you use a different e-mail address for the sign-off. :-)

Watch out for fast forward merges. Ideally I guess maintainers really should disable fast forwards and PGP-sign their merge commits...

If you submit a signed pull request, that covers all of the history in your tree up to the top commit you're requesting to pull.

And I don't do fast forward merges.

Thanks,
Rafael