Skip to contentSkip to conversation

Maintainer's Summit Agenda Planning

Ksummit-discuss Theodore Ts'o 19 participants 42 messages Oct 5, 2017 – Oct 31, 2017

lore
Share & tools
lore
#0 lore raw

This is a rough draft of a proposed agenda for the Maintainer's Summit.

I've fliled in most of the slots with those topics that seemed to make the most amount of sense as topics for us to talk about, with some room intentionally blank sponts.

Please comment about topics you think we should include in the agenda; or if you think there are topics that should not be included, please also let us know.

Thanks!!

- Ted

Maintainers Summit -- Thursday

8:00 am Breakfast
9:00 am Agenda Bashing
9:30 am Improve regression tracking
10:00 am Bash the kernel maintainers
10:30 am Android Kernel Issues (drivers, long-term stable) 11:00 am What is Linus Happy/Unhappy about?
11:30 am
12:00 am
12:30 pm Lunch

Appendix: Other topics that were brought up
-------------------------------------------

Documentation
Bug reporting feedback loop
Driver and/or module versions
Developing across multiple areas of the kernel
ABI feature gates
tracepoints without user space interfaces (EBPF)

Appendix: Maintainer's Summit Attendees
---------------------------------------

(Does not include the sponsored attendees; I don't have most of those names yet.)

Andrew Morton
Arnd Bergmann
Ben Hutchings
Chris Mason
Dan Williams
Fengguang Wu
Greg KH
H. Peter Anvin
James Bottomley
Jens Axboe
Jiri Kosina
Jonathan Corbet
Kees Cook
Laura Abbott
Linus Torvalds
Martin Petersen
Michael Ellerman
Rom Lemarchand
Sean Paul
Shuah Khan
Stephen Rothwell
Takashi Iwai
Ted Ts'o
Thorsten Leemhuis

#1 lore raw
Quoted 13 lines · CEST Theodore Ts'o

This is a rough draft of a proposed agenda for the Maintainer's Summit.

I've fliled in most of the slots with those topics that seemed to make the most amount of sense as topics for us to talk about, with some room intentionally blank sponts.

Please comment about topics you think we should include in the agenda; or if you think there are topics that should not be included, please also let us know.

Thanks!!

Quoted 7 lines

- Ted

Maintainers Summit -- Thursday

8:00 am Breakfast
9:00 am Agenda Bashing
9:30 am Improve regression tracking

Well, to be honest I really would like to participate in this discussion.

It really is relevant to all maintainers IMO.

Quoted 8 lines

10:00 am Bash the kernel maintainers
10:30 am Android Kernel Issues (drivers, long-term stable) 11:00 am What is Linus Happy/Unhappy about?
11:30 am
12:00 am
12:30 pm Lunch

Quoted 5 lines

Appendix: Other topics that were brought up
-------------------------------------------

Documentation
Bug reporting feedback loop

Similarly for the two topics above.

Quoted 2 lines

Driver and/or module versions
Developing across multiple areas of the kernel

And same here.

Quoted 2 lines

ABI feature gates
tracepoints without user space interfaces (EBPF)

Thanks,
Rafael

#2 lore raw
In reply to Theodore Ts'o
Quoted 3 lines · Theodore Ts'o

Please comment about topics you think we should include in the agenda; or if you think there are topics that should not be included, please also let us know.

[ ... snip ... ]

Quoted 13 lines

Maintainers Summit -- Thursday

8:00 am Breakfast
9:00 am Agenda Bashing
9:30 am Improve regression tracking
10:00 am Bash the kernel maintainers
10:30 am Android Kernel Issues (drivers, long-term stable) 11:00 am What is Linus Happy/Unhappy about?
11:30 am
12:00 am
12:30 pm Lunch

Quoted 2 lines

Appendix: Other topics that were brought up
-------------------------------------------

[... snip ... ]

If we are voting about the topics to fill 11:30 and 12:00 slots, my choice definitely would be (the first one even more so as especially major distros will apparently have representatives there):

Quoted 2 lines

Bug reporting feedback loop
ABI feature gates

Thanks,

Signature · 2 lines
Jiri Kosina
SUSE Labs
#3 lore raw
In reply to Theodore Ts'o
Quoted 29 lines · Theodore Ts'o

This is a rough draft of a proposed agenda for the Maintainer's Summit.

I've fliled in most of the slots with those topics that seemed to make the most amount of sense as topics for us to talk about, with some room intentionally blank sponts.

Please comment about topics you think we should include in the agenda; or if you think there are topics that should not be included, please also let us know.

Thanks!!

- Ted

Maintainers Summit -- Thursday

8:00 am Breakfast
9:00 am Agenda Bashing
9:30 am Improve regression tracking
10:00 am Bash the kernel maintainers
10:30 am Android Kernel Issues (drivers, long-term stable) 11:00 am What is Linus Happy/Unhappy about?
11:30 am
12:00 am
12:30 pm Lunch

Quoted 5 lines

Appendix: Other topics that were brought up
-------------------------------------------

Documentation
Bug reporting feedback loop

I think this one (bug reporting) can be combined with regression tracking session. In practice, they are tied closely.

Other than that,

Quoted 2 lines

Developing across multiple areas of the kernel
ABI feature gates

... these two are my vote to be filled in the empty slots.

thanks,

Takashi

Quoted 41 lines

tracepoints without user space interfaces (EBPF)

Appendix: Maintainer's Summit Attendees
---------------------------------------

(Does not include the sponsored attendees; I don't have most of those names yet.)

Andrew Morton
Arnd Bergmann
Ben Hutchings
Chris Mason
Dan Williams
Fengguang Wu
Greg KH
H. Peter Anvin
James Bottomley
Jens Axboe
Jiri Kosina
Jonathan Corbet
Kees Cook
Laura Abbott
Linus Torvalds
Martin Petersen
Michael Ellerman
Rom Lemarchand
Sean Paul
Shuah Khan
Stephen Rothwell
Takashi Iwai
Ted Ts'o
Thorsten Leemhuis

_______________________________________________
Ksummit-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

#4 lore raw
In reply to Theodore Ts'o
Quoted 9 lines · Theodore Ts'o

Appendix: Other topics that were brought up
-------------------------------------------

Documentation
Bug reporting feedback loop
Driver and/or module versions
Developing across multiple areas of the kernel
ABI feature gates
tracepoints without user space interfaces (EBPF)

I've got a couple of extra possibilities

1) git tree script alignment.  Most of us send out some type of your patch was added and your patch was dropped email notifications.  We usually automate it through a private tree git commit or update script. Should we standardise these (because if we do, we could have kernel.org do it for us instead of running our own infrastructure).

2) Trivial patches (again). OpenStack has recently started to become annoyed by these

http://lists.openstack.org/pipermail/openstack-dev/2017-September/thread.html#122472

I thought about our process around the trivial tree, but it hasn't been updated in the last few releases, so effectively we no longer use it. So is what we're currently doing (variable standards by tree) OK, or should we have a more concerted trivial policy?

3) Tasteful Rebasing. We've come around (finally) to the conclusion that rebasing isn't always bad.  My own opinion is that rebasing to fix issues in patches (particularly those marked for stable) so they can backport cleanly and atomically.  There's also less of a consensus that rebasing to clean up history is a reasonably good thing (provided it's not done just before requesting a pull).  However, we have a divergence of opinions not just on whether we should rebase, but what constitutes a tasteful rebase.  Just telling people, particularly would be new maintainers, that it's a judgement call always is unhelpful, we could do with putting together some more detailed guidance (assuming we can agree on it).

James

#5 lore raw
Quoted 29 lines · James Bottomley
Quoted 9 lines · Theodore Ts'o

Appendix: Other topics that were brought up
-------------------------------------------

Documentation
Bug reporting feedback loop
Driver and/or module versions
Developing across multiple areas of the kernel
ABI feature gates
tracepoints without user space interfaces (EBPF)

I've got a couple of extra possibilities

1) git tree script alignment.  Most of us send out some type of your patch was added and your patch was dropped email notifications.  We usually automate it through a private tree git commit or update script. Should we standardise these (because if we do, we could have kernel.org do it for us instead of running our own infrastructure).

2) Trivial patches (again). OpenStack has recently started to become annoyed by these

http://lists.openstack.org/pipermail/openstack-dev/2017-September/thread.html#122472

I thought about our process around the trivial tree, but it hasn't been updated in the last few releases, so effectively we no longer use it. So is what we're currently doing (variable standards by tree) OK, or should we have a more concerted trivial policy?

Quoted 11 lines

3) Tasteful Rebasing. We've come around (finally) to the conclusion that rebasing isn't always bad.  My own opinion is that rebasing to fix issues in patches (particularly those marked for stable) so they can backport cleanly and atomically.  There's also less of a consensus that rebasing to clean up history is a reasonably good thing (provided it's not done just before requesting a pull).  However, we have a divergence of opinions not just on whether we should rebase, but what constitutes a tasteful rebase.  Just telling people, particularly would be new maintainers, that it's a judgement call always is unhelpful, we could do with putting together some more detailed guidance (assuming we can agree on it).

Maintainers seem to have a lot of tribal knowledge, which is self-taught, or passed on by word of mouth, or learned the hard way when they make a mistake: branch management, merge conflicts, rebasing, pull requests, cross-tree changes, release candidate timing, co-maintainer processes, etc.

I think it would be a good idea to have a Maintainer's Guide which tries to document a lot of this knowledge. It would help new maintainers learn the ropes, and would also help drive consensus for maintainer's best practices. It could document the typical processes of a maintainer, and policy guidelines like some of the above topics.

Signature · 1 line
Josh
#6 lore raw
Quoted 5 lines · On Fri, 6 Oct 2017 11:26:21 -0500 Josh Poimboeuf <[email protected]> wrote:

I think it would be a good idea to have a Maintainer's Guide which tries to document a lot of this knowledge. It would help new maintainers learn the ropes, and would also help drive consensus for maintainer's best practices. It could document the typical processes of a maintainer, and policy guidelines like some of the above topics.

Strangely enough, this is a conversation that has been popping up in other contexts too. We may see an initial attempt before too long.

The tricky part, of course, is finding a way to document the consensus on best practices without trying to "drive" it too hard.

My own thought is that a good starting place might be a "how to avoid getting your pull request flamed" document, since there is some semblance of a consensus there and it's a place where people often make mistakes.

jon

#7 lore raw
Quoted 11 lines · Jonathan Corbet
Quoted 5 lines · On Fri, 6 Oct 2017 11:26:21 -0500 Josh Poimboeuf <[email protected]> wrote:

I think it would be a good idea to have a Maintainer's Guide which tries to document a lot of this knowledge. It would help new maintainers learn the ropes, and would also help drive consensus for maintainer's best practices. It could document the typical processes of a maintainer, and policy guidelines like some of the above topics.

Strangely enough, this is a conversation that has been popping up in other contexts too. We may see an initial attempt before too long.

Ah, nice.

Quoted 2 lines

The tricky part, of course, is finding a way to document the consensus on best practices without trying to "drive" it too hard.

We somehow manage to get consensus on millions of lines of code, you'd think we could figure out how to agree on a small process document ;-)

But seriously, I think even parts which are disagreed upon, or which are up to the maintainer's judgement, can be documented as such.

I wonder if a maintainers mailing list would help for such discussions, if we don't already have one.

Quoted 3 lines

My own thought is that a good starting place might be a "how to avoid getting your pull request flamed" document, since there is some semblance of a consensus there and it's a place where people often make mistakes.

Agreed!

Signature · 1 line
Josh
#8 lore raw
In reply to Jonathan Corbet
Quoted 14 lines · Jonathan Corbet
Quoted 7 lines · On Fri, 6 Oct 2017 11:26:21 -0500 Josh Poimboeuf <[email protected]> wrote:

I think it would be a good idea to have a Maintainer's Guide which tries to document a lot of this knowledge.  It would help new maintainers learn the ropes, and would also help drive consensus for maintainer's best practices.  It could document the typical processes of a maintainer, and policy guidelines like some of the above topics.

Strangely enough, this is a conversation that has been popping up in other contexts too.  We may see an initial attempt before too long.

Quoted 7 lines

The tricky part, of course, is finding a way to document the consensus on best practices without trying to "drive" it too hard.

My own thought is that a good starting place might be a "how to avoid getting your pull request flamed" document, since there is some semblance of a consensus there and it's a place where people often make mistakes.

Actually, I'd argue this is the most arcane area.  Accepting a pull request represents the expression of a trust relationship and it's not entirely well documented how to form that.  The mechanics of what should be in it and how it should be split vary by puller.  For instance, Linus' requirements are reasonably well documented in git- request-pull, but other trees have varying requirements.  Why he might flame you varies from too many merge window patches in a non-merge window to to many merge points in the tree or "unclean" history.

James

#9 lore raw
Quoted 23 lines · James Bottomley
Quoted 14 lines · Jonathan Corbet
Quoted 7 lines · On Fri, 6 Oct 2017 11:26:21 -0500 Josh Poimboeuf <[email protected]> wrote:

I think it would be a good idea to have a Maintainer's Guide which tries to document a lot of this knowledge.  It would help new maintainers learn the ropes, and would also help drive consensus for maintainer's best practices.  It could document the typical processes of a maintainer, and policy guidelines like some of the above topics.

Strangely enough, this is a conversation that has been popping up in other contexts too.  We may see an initial attempt before too long.

Quoted 7 lines

The tricky part, of course, is finding a way to document the consensus on best practices without trying to "drive" it too hard.

My own thought is that a good starting place might be a "how to avoid getting your pull request flamed" document, since there is some semblance of a consensus there and it's a place where people often make mistakes.

Quoted 8 lines

Actually, I'd argue this is the most arcane area.  Accepting a pull request represents the expression of a trust relationship and it's not entirely well documented how to form that.  The mechanics of what should be in it and how it should be split vary by puller.  For instance, Linus' requirements are reasonably well documented in git- request-pull, but other trees have varying requirements.  Why he might flame you varies from too many merge window patches in a non-merge window to to many merge points in the tree or "unclean" history.

Each thing you just said is a good example of something which should be documented. In fact I think each sentence could be expanded into a paragraph or section in the document.

Signature · 1 line
Josh
#10 lore raw
In reply to Jonathan Corbet
Quoted 2 lines · Jonathan Corbet

The tricky part, of course, is finding a way to document the consensus on best practices without trying to "drive" it too hard.

I put some two very technical, not too subjective design patterns in Documentation/driver-model/design-patterns.txt

I have found myself on the verge of sending a patch adding Rusty Russell's API design manifesto somewhere there
http://sweng.the-davies.net/Home/rustys-api-design-manifesto

The problem is mostly that I don't always feel what the consensus is. With some subsystem maintainers happily agree to disagree on so many things like inverse-christmas tree includes or u_int8_t vs u8 it's easy to be discouraged.

I guess we should just be more bold? Who knows, maybe noone even gets upset.

Apart from that I guess Documentation/process/* need to be updated with best practices? Maybe that was the actual initial question. I especially like management-style.rst, that doc kicks ass.

Yours,
Linus Walleij

#11 lore raw
In reply to James Bottomley
Quoted 5 lines · James Bottomley

1) git tree script alignment.  Most of us send out some type of your patch was added and your patch was dropped email notifications.  We usually automate it through a private tree git commit or update script. Should we standardise these (because if we do, we could have kernel.org do it for us instead of running our own infrastructure).

This has been talked about in the past on the list - there were concernes about monoculture making any problems with the scripts much worse and just there being enough differences in workflow to cause problems.

#12 lore raw
In reply to James Bottomley
Quoted 7 lines · James Bottomley

2) Trivial patches (again). OpenStack has recently started to become annoyed by these

http://lists.openstack.org/pipermail/openstack-dev/2017-September/thread.html#122472

I thought about our process around the trivial tree, but it hasn't been updated in the last few releases, so effectively we no longer use it.

This has been caused solely by me being buried in other things, and there was always something else that overrode trivial tree in priority.

Quoted 2 lines

 So is what we're currently doing (variable standards by tree) OK, or should we have a more concerted trivial policy?

My original plan was to revive trivial tree for 4.15, as there are quite a few patches in the queue (that still apply). But if it's generally considered annoying (although I am pretty sure we don't suffer from what's in the openstack thread), I don't object and can ditch it completely.

The thing is that such patches will keep coming anyway, and I think they have the value (although the priority is really lower than other changes of course). I still believe that having greppable comments, for example, is nice to have.

Signature · 2 lines
Jiri Kosina
SUSE Labs
#13 lore raw
Quoted 18 lines · Jiri Kosina
Quoted 10 lines · James Bottomley

2) Trivial patches (again). OpenStack has recently started to become annoyed by these

http://lists.openstack.org/pipermail/openstack-dev/2017-September/t
hread.html#122472

I thought about our process around the trivial tree, but it hasn't been updated in the last few releases, so effectively we no longer use it.

This has been caused solely by me being buried in other things, and there was always something else that overrode trivial tree in priority.

Quoted 1 line
Quoted 8 lines
Quoted 2 lines

 So is what we're currently doing (variable standards by tree) OK, or should we have a more concerted trivial policy?

My original plan was to revive trivial tree for 4.15, as there are quite a few patches in the queue (that still apply). But if it's generally considered annoying (although I am pretty sure we don't suffer from what's in the openstack thread), I don't object and can ditch it completely.

Well, their objection was core (by which they mean Maintainer) review and merge time, plus the interference to higher priority patches caused by mismerges.  I was actually thinking a trivial tree might help them because it would offload all the mismerges and review and they only need do a real merge just before release stabilisation.

Quoted 4 lines

The thing is that such patches will keep coming anyway, and I think they have the value (although the priority is really lower than other changes of course). I still believe that having greppable comments, for example, is nice to have.

Well, we tend to veto changes to old drivers in various subsystems anyway (having been burned by the this is only a trivial change and then finding out six months later that the driver is broken).

Trivial changes seem to fall broadly into three categories:

1. spelling fixes
2. whitespace changes (I ran checkpatch.pl on your file and it returned these issues).
3. pattern imposition (we now have a new API for this so lets change all prior open coded incarnations)
4. trivial pattern fixes (things like replacing if(x) kfree(x); with

      kfree(x);)

I don't want to open the whole spelling/whitespace can of worms, but perhaps we could have a more meaningful discussion about the pattern based issues ... for instance if we agree it's useful and coccinelle can do it, then tree wide replacement at -rc1 might be a better solution than ad-hoc application of hundreds of patches.

James

#14 lore raw
Quoted 51 lines · James Bottomley
Quoted 18 lines · Jiri Kosina
Quoted 10 lines · James Bottomley

2) Trivial patches (again). OpenStack has recently started to become annoyed by these

http://lists.openstack.org/pipermail/openstack-dev/2017-September/t
hread.html#122472

I thought about our process around the trivial tree, but it hasn't been updated in the last few releases, so effectively we no longer use it.

This has been caused solely by me being buried in other things, and there was always something else that overrode trivial tree in priority.

Quoted 1 line
Quoted 8 lines
Quoted 2 lines

 So is what we're currently doing (variable standards by tree) OK, or should we have a more concerted trivial policy?

My original plan was to revive trivial tree for 4.15, as there are quite a few patches in the queue (that still apply). But if it's generally considered annoying (although I am pretty sure we don't suffer from what's in the openstack thread), I don't object and can ditch it completely.

Well, their objection was core (by which they mean Maintainer) review and merge time, plus the interference to higher priority patches caused by mismerges.  I was actually thinking a trivial tree might help them because it would offload all the mismerges and review and they only need do a real merge just before release stabilisation.

Quoted 4 lines

The thing is that such patches will keep coming anyway, and I think they have the value (although the priority is really lower than other changes of course). I still believe that having greppable comments, for example, is nice to have.

Well, we tend to veto changes to old drivers in various subsystems anyway (having been burned by the this is only a trivial change and then finding out six months later that the driver is broken).

Trivial changes seem to fall broadly into three categories:

1. spelling fixes
2. whitespace changes (I ran checkpatch.pl on your file and it returned these issues).
3. pattern imposition (we now have a new API for this so lets change all prior open coded incarnations)
4. trivial pattern fixes (things like replacing if(x) kfree(x); with

Quoted 7 lines
      kfree(x);)

I don't want to open the whole spelling/whitespace can of worms, but perhaps we could have a more meaningful discussion about the pattern based issues ... for instance if we agree it's useful and coccinelle can do it, then tree wide replacement at -rc1 might be a better solution than ad-hoc application of hundreds of patches.

Do you suggest one big patch, that goes to who? Or lots of little patches that go out at once to the individual maintainers of the affected code?

Both are doable.

julia

Quoted 8 lines

James

_______________________________________________
Ksummit-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

#15 lore raw
Quoted 72 lines · Julia Lawall
Quoted 63 lines · James Bottomley
Quoted 22 lines · Jiri Kosina
Quoted 5 lines · James Bottomley

2) Trivial patches (again). OpenStack has recently started to become annoyed by these

Quoted 7 lines

http://lists.openstack.org/pipermail/openstack-dev/2017-Septemb
er/t
hread.html#122472

I thought about our process around the trivial tree, but it hasn't been updated in the last few releases, so effectively we no longer use it.

This has been caused solely by me being buried in other things, and there was always something else that overrode trivial tree in priority.

Quoted 2 lines
Quoted 8 lines
Quoted 2 lines

 So is what we're currently doing (variable standards by tree) OK, or should we have a more concerted trivial policy?

My original plan was to revive trivial tree for 4.15, as there are quite a few patches in the queue (that still apply). But if it's generally considered annoying (although I am pretty sure we don't suffer from what's in the openstack thread), I don't object and can ditch it completely.

Well, their objection was core (by which they mean Maintainer) review and merge time, plus the interference to higher priority patches caused by mismerges.  I was actually thinking a trivial tree might help them because it would offload all the mismerges and review and they only need do a real merge just before release stabilisation.

Quoted 5 lines

The thing is that such patches will keep coming anyway, and I think they have the value (although the priority is really lower than other changes of course). I still believe that having greppable comments, for example, is nice to have.

Well, we tend to veto changes to old drivers in various subsystems anyway (having been burned by the this is only a trivial change and then finding out six months later that the driver is broken).

Trivial changes seem to fall broadly into three categories:

   1. spelling fixes
   2. whitespace changes (I ran checkpatch.pl on your file and it returned
      these issues).
   3. pattern imposition (we now have a new API for this so lets change
      all prior open coded incarnations)
   4. trivial pattern fixes (things like replacing if(x) kfree(x); with
      kfree(x);)

Quoted 5 lines

I don't want to open the whole spelling/whitespace can of worms, but perhaps we could have a more meaningful discussion about the pattern based issues ... for instance if we agree it's useful and coccinelle can do it, then tree wide replacement at -rc1 might be a better solution than ad-hoc application of hundreds of patches.

Quoted 3 lines

Do you suggest one big patch, that goes to who?  Or lots of little patches that go out at once to the individual maintainers of the affected code?

I was actually thinking we validate the script and if there are no problems, apply it at -rc1 ... so effectively one big patch.  What actually gets applied (script or big patch) would be up to Linus ... he's the one that makes the -rc1 changes.  However, if we agree a curation path for the script, the application could be done by Jíři as a pull from the trivial tree (unless Linus wanted to do it himself or someone else wanted to manage this).

James

#16 lore raw
Quoted 1 line · James Bottomley

I was actually thinking we validate the script

That's the hard part.
Completeness and correctness are non trivial.

Quoted 1 line

and if there are no problems, apply it at -rc1

I also think that might be the best mechanism.

Quoted 4 lines

However, if we agree a
curation path for the script, the application could be done by Jíři as a pull from the trivial tree (unless Linus wanted to do it himself or someone else wanted to manage this).

Perhaps a kbuild robot variant could be utilized here.

Many changes should produce no object code differences and validation of the correctness of the changes might be better done using compilation comparison tools.

#17 lore raw
In reply to James Bottomley
Quoted 7 lines · James Bottomley
Quoted 3 lines · Julia Lawall

Do you suggest one big patch, that goes to who?  Or lots of little patches that go out at once to the individual maintainers of the affected code?

I was actually thinking we validate the script and if there are no problems, apply it at -rc1 ... so effectively one big patch.

By -rc1 we (drm in general, drm/i915 in particular) will already have accumulated easily 4-5 weeks' worth of commits for the *next* merge window. Applying treewide stuff to Linus' tree at -rc1 forces a backmerge and potentially conflicts galore I'm not particularly thrilled about. I'd much rather see the patches to our driver come through our tree, and we generally deal with them without a fuss.

BR,
Jani.

Signature · 1 line
Jani Nikula, Intel Open Source Technology Center
#18 lore raw
Quoted 9 lines · Jani Nikula
Quoted 7 lines · James Bottomley
Quoted 3 lines · Julia Lawall

Do you suggest one big patch, that goes to who? Or lots of little patches that go out at once to the individual maintainers of the affected code?

I was actually thinking we validate the script and if there are no problems, apply it at -rc1 ... so effectively one big patch.

Quoted 6 lines

By -rc1 we (drm in general, drm/i915 in particular) will already have accumulated easily 4-5 weeks' worth of commits for the *next* merge window. Applying treewide stuff to Linus' tree at -rc1 forces a backmerge and potentially conflicts galore I'm not particularly thrilled about. I'd much rather see the patches to our driver come through our tree, and we generally deal with them without a fuss.

And it's not just the development that's already queued, there's tons of patch series in developer trees that need to be rebased too. Doing treewide stuff at -rc1 is as disruptive as any other time for developers overall, it only optimizes the disruption in Linus repo.

The other reason I'm not too fond of doing treewide stuff as one huge patch is that those cocci patches are perfect newbie fodder for their first patch. Of course once a refactor has tappererd of it's good to do one final push, but not always and consistently. -Daniel

Signature · 3 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
#19 lore raw
In reply to Jani Nikula
Quoted 10 lines · Jani Nikula
Quoted 8 lines · James Bottomley
Quoted 5 lines · Julia Lawall

Do you suggest one big patch, that goes to who?  Or lots of little
patches that go out at once to the individual maintainers of the affected code?

Quoted 7 lines
Quoted 2 lines

I was actually thinking we validate the script and if there are no problems, apply it at -rc1 ... so effectively one big patch.

By -rc1 we (drm in general, drm/i915 in particular) will already have accumulated easily 4-5 weeks' worth of commits for the *next* merge window. Applying treewide stuff to Linus' tree at -rc1 forces a backmerge and potentially conflicts galore

If we're applying a semantic patch script (and we've verified it works well enough to use the script on the -rc1 main tree), couldn't you simply apply it to your tree at the same time?

James

Quoted 8 lines

I'm not particularly thrilled about. I'd much rather see the patches
to our driver come through our tree, and we generally deal with them without a fuss.

BR,
Jani.

#20 lore raw
Quoted 14 lines · James Bottomley
Quoted 13 lines · Jani Nikula
Quoted 10 lines · James Bottomley
Quoted 5 lines · Julia Lawall

Do you suggest one big patch, that goes to who?  Or lots of little
patches that go out at once to the individual maintainers of the affected code?

I was actually thinking we validate the script and if there are no problems, apply it at -rc1 ... so effectively one big patch.

Quoted 8 lines
Quoted 4 lines

By -rc1 we (drm in general, drm/i915 in particular) will already have accumulated easily 4-5 weeks' worth of commits for the *next* merge window. Applying treewide stuff to Linus' tree at -rc1 forces a backmerge and potentially conflicts galore

If we're applying a semantic patch script (and we've verified it works well enough to use the script on the -rc1 main tree), couldn't you simply apply it to your tree at the same time?

If we did, the fixes would show up in a later kernel release. Which is just fine for us. In other words, just let subsystems and drivers handle this as they see fit?

BR,
Jani.

Signature · 1 line
Jani Nikula, Intel Open Source Technology Center
#21 lore raw
Quoted 20 lines · Jani Nikula
Quoted 19 lines · James Bottomley
Quoted 17 lines · Jani Nikula
Quoted 10 lines · James Bottomley
Quoted 5 lines · Julia Lawall

Do you suggest one big patch, that goes to who?  Or lots of little
patches that go out at once to the individual maintainers of the affected code?

I was actually thinking we validate the script and if there are no problems, apply it at -rc1 ... so effectively one big patch.

By -rc1 we (drm in general, drm/i915 in particular) will already have accumulated easily 4-5 weeks' worth of commits for the *next* merge window. Applying treewide stuff to Linus' tree at -rc1 forces a backmerge and potentially conflicts galore

Quoted 7 lines
Quoted 3 lines

If we're applying a semantic patch script (and we've verified it works well enough to use the script on the -rc1 main tree), couldn't you simply apply it to your tree at the same time?

If we did, the fixes would show up in a later kernel release. Which is just fine for us. In other words, just let subsystems and drivers handle this as they see fit?

Scheduling and acceptance rates are the issue.

Also some scripted patches require complete treewide application to allow things like API changes.

#22 lore raw
Quoted 25 lines · Joe Perches
Quoted 24 lines · Jani Nikula
Quoted 22 lines · James Bottomley
Quoted 10 lines · Jani Nikula
Quoted 8 lines · James Bottomley
Quoted 5 lines · Julia Lawall

Do you suggest one big patch, that goes to who?  Or lots of little
patches that go out at once to the individual maintainers of the affected code?

Quoted 7 lines
Quoted 2 lines

I was actually thinking we validate the script and if there are no problems, apply it at -rc1 ... so effectively one big patch.

By -rc1 we (drm in general, drm/i915 in particular) will already have accumulated easily 4-5 weeks' worth of commits for the *next* merge window. Applying treewide stuff to Linus' tree at -rc1 forces a backmerge and potentially conflicts galore

If we're applying a semantic patch script (and we've verified it works well enough to use the script on the -rc1 main tree), couldn't you simply apply it to your tree at the same time?

Quoted 8 lines
Quoted 3 lines

If we did, the fixes would show up in a later kernel release. Which is just fine for us. In other words, just let subsystems and drivers handle this as they see fit?

Scheduling and acceptance rates are the issue.

Also some scripted patches require complete treewide application to allow things like API changes.

As described in https://lwn.net/Articles/735468/ we have a pretty extensive and growing CI system in place. We don't apply a single patch without a pre-merge green light from CI, no exceptions. I take issue with applying any patches to our driver that didn't go through our CI first, let alone bypassing our repositories.

If an API change requires a flag day treewide change in a 15M+ line hierarchically developed codebase, you're just plain doing it wrong.

Please just let subsystems and drivers handle this as they see fit, and queue changes via their trees.

BR,
Jani.

Signature · 1 line
Jani Nikula, Intel Open Source Technology Center
#23 lore raw
Quoted 35 lines · Jani Nikula
Quoted 25 lines · Joe Perches
Quoted 24 lines · Jani Nikula
Quoted 22 lines · James Bottomley
Quoted 10 lines · Jani Nikula
Quoted 8 lines · James Bottomley
Quoted 5 lines · Julia Lawall

Do you suggest one big patch, that goes to who?  Or lots of little
patches that go out at once to the individual maintainers of the affected code?

Quoted 7 lines
Quoted 2 lines

I was actually thinking we validate the script and if there are no problems, apply it at -rc1 ... so effectively one big patch.

By -rc1 we (drm in general, drm/i915 in particular) will already have accumulated easily 4-5 weeks' worth of commits for the *next* merge window. Applying treewide stuff to Linus' tree at -rc1 forces a backmerge and potentially conflicts galore

If we're applying a semantic patch script (and we've verified it works well enough to use the script on the -rc1 main tree), couldn't you simply apply it to your tree at the same time?

Quoted 8 lines
Quoted 3 lines

If we did, the fixes would show up in a later kernel release. Which is just fine for us. In other words, just let subsystems and drivers handle this as they see fit?

Scheduling and acceptance rates are the issue.

Also some scripted patches require complete treewide application to allow things like API changes.

Quoted 8 lines

As described in https://lwn.net/Articles/735468/ we have a pretty extensive and growing CI system in place. We don't apply a single patch without a pre-merge green light from CI, no exceptions. I take issue with applying any patches to our driver that didn't go through our CI first, let alone bypassing our repositories.

If an API change requires a flag day treewide change in a 15M+ line hierarchically developed codebase, you're just plain doing it wrong.

THe size of the codebase is not particularly relevant and that's simply not always possible.

#24 lore raw
Quoted 42 lines · Joe Perches
Quoted 41 lines · Jani Nikula
Quoted 25 lines · Joe Perches
Quoted 24 lines · Jani Nikula
Quoted 22 lines · James Bottomley
Quoted 10 lines · Jani Nikula
Quoted 8 lines · James Bottomley
Quoted 5 lines · Julia Lawall

Do you suggest one big patch, that goes to who?  Or lots of little
patches that go out at once to the individual maintainers of the affected code?

Quoted 7 lines
Quoted 2 lines

I was actually thinking we validate the script and if there are no problems, apply it at -rc1 ... so effectively one big patch.

By -rc1 we (drm in general, drm/i915 in particular) will already have accumulated easily 4-5 weeks' worth of commits for the *next* merge window. Applying treewide stuff to Linus' tree at -rc1 forces a backmerge and potentially conflicts galore

If we're applying a semantic patch script (and we've verified it works well enough to use the script on the -rc1 main tree), couldn't you simply apply it to your tree at the same time?

Quoted 8 lines
Quoted 3 lines

If we did, the fixes would show up in a later kernel release. Which is just fine for us. In other words, just let subsystems and drivers handle this as they see fit?

Scheduling and acceptance rates are the issue.

Also some scripted patches require complete treewide application to allow things like API changes.

As described in https://lwn.net/Articles/735468/ we have a pretty extensive and growing CI system in place. We don't apply a single patch without a pre-merge green light from CI, no exceptions. I take issue with applying any patches to our driver that didn't go through our CI first, let alone bypassing our repositories.

Quoted 4 lines
Quoted 2 lines

If an API change requires a flag day treewide change in a 15M+ line hierarchically developed codebase, you're just plain doing it wrong.

THe size of the codebase is not particularly relevant and

Sure it is. It translates to the amount of people and subsystems and the pain you're causing them.

Quoted 1 line

that's simply not always possible.

That something is not always possible is not a reason to change the usual process to cater for the rare outlier case. Making it easier to do treewide changes encourages people to do just that, and discourages them from finding the ways to not cause trouble to subsystems.

BR,
Jani.

Signature · 1 line
Jani Nikula, Intel Open Source Technology Center
#25 lore raw
In reply to James Bottomley
Quoted 10 lines · James Bottomley
Quoted 4 lines · Jani Nikula

By -rc1 we (drm in general, drm/i915 in particular) will already have accumulated easily 4-5 weeks' worth of commits for the *next* merge window. Applying treewide stuff to Linus' tree at -rc1 forces a backmerge and potentially conflicts galore

If we're applying a semantic patch script (and we've verified it works well enough to use the script on the -rc1 main tree), couldn't you simply apply it to your tree at the same time?

git doesn't do the best job figuring out that the same change was made in two different commits so you still end up with the conflicts. It's much easier if we can just get changes in through the maintainer tree.

#26 lore raw
In reply to James Bottomley
Quoted 3 lines · James Bottomley

it's useful and coccinelle
can do it, then tree wide replacement at -rc1 might be a better solution than ad-hoc application of hundreds of patches.

Patches that span subsystems are always painful and, even if submitted to all subsystems simultaneously, are frequently only partially applied.

Scripting is useful. Coccinelle is a very good tool, but it is not the only scripted patch generator.

A preferred mechanism to get scripted patches applied tree-wide would be beneficial.

#27 lore raw
In reply to James Bottomley
Quoted 2 lines · James Bottomley

Trivial changes seem to fall broadly into three categories:

[ ... snip ... ]

Quoted 2 lines

2. whitespace changes (I ran checkpatch.pl on your file and it returned these issues).

Just to be clear on this -- I'm not taking such patches through trivial.git.

Quoted 5 lines

I don't want to open the whole spelling/whitespace can of worms, but perhaps we could have a more meaningful discussion about the pattern based issues ... for instance if we agree it's useful and coccinelle can do it, then tree wide replacement at -rc1 might be a better solution than ad-hoc application of hundreds of patches.

Agreed, and I think Linus did this several times in the past and has no issues with doing so.

Thanks,

Signature · 2 lines
Jiri Kosina
SUSE Labs
#28 lore raw
In reply to James Bottomley
Quoted 8 lines · James Bottomley
Quoted 3 lines · Theodore Ts'o

Appendix: Other topics that were brought up
[...]
Developing across multiple areas of the kernel

I've got a couple of extra possibilities
[...]
2) Trivial patches (again).

Given that the "trivial patches" topic's discussion ended up boiling down to a discussion about developing across multiple areas of the kernel, maybe we should make space for a "tree-wide changes" discussion? Even after the earlier thread about it, I tripped all over this in the last couple months while doing timer conversions, so I would at least have some more strong opinions on the subject. ;)

-Kees

Signature · 2 lines
Kees Cook
Pixel Security
#29 lore raw
Quoted 11 lines · Kees Cook
Quoted 8 lines · James Bottomley
Quoted 3 lines · Theodore Ts'o

Appendix: Other topics that were brought up
[...]
Developing across multiple areas of the kernel

I've got a couple of extra possibilities
[...]
2) Trivial patches (again).

Quoted 6 lines

Given that the "trivial patches" topic's discussion ended up boiling down to a discussion about developing across multiple areas of the kernel, maybe we should make space for a "tree-wide changes" discussion? Even after the earlier thread about it, I tripped all over this in the last couple months while doing timer conversions, so I would at least have some more strong opinions on the subject. ;)

It's a ripe area (like months old limburger cheese) for discussion.

There's currently no good way to do tree-wide changes.

#30 lore raw
Quoted 19 lines · Joe Perches
Quoted 11 lines · Kees Cook
Quoted 8 lines · James Bottomley
Quoted 3 lines · Theodore Ts'o

Appendix: Other topics that were brought up
[...]
Developing across multiple areas of the kernel

I've got a couple of extra possibilities
[...]
2) Trivial patches (again).

Quoted 6 lines

Given that the "trivial patches" topic's discussion ended up boiling down to a discussion about developing across multiple areas of the kernel, maybe we should make space for a "tree-wide changes" discussion? Even after the earlier thread about it, I tripped all over this in the last couple months while doing timer conversions, so I would at least have some more strong opinions on the subject. ;)

Quoted 3 lines

It's a ripe area (like months old limburger cheese) for discussion.

There's currently no good way to do tree-wide changes.

Some things stand out for me:

1) I would like a standard way to distinguish patch submissions between "please ack this (it's going into my tree)" and "please apply this to your tree." I have tried post-"---"-line notes, cover letter notes, etc, and maintainers still miss it. It can sometimes be very disruptive (to both me and the maintainer) to have a maintainer take a patch out of the middle of a series that was intending to land via a different tree. Would "[ACK-PLEASE][PATCH]" be sufficient? Or "[MY-TREE]" or something?

2) When you have a 200+ patch series, it is outrageously difficult to figure out where to send things. The MAINTAINERS file is at best an approximation. I use a subset of the get_maintainer output plus my own parsing of MAINTAINERS to try to organize patches. The results tend to be somewhat okay, but there are still bouncing addresses, or MIA maintainers. And then a patch isn't met with silence, it might get met with an "Ack", but no attention from a committer. Having a representation of the "tree" of maintainership would be much more helpful. In other words, for every section with an "M:" line, also something "U:" ("upstream"?) that indicates either another section or a person that is the "upstream" for that maintainer. This would allow for a sane set of "Cc"s not based on git log guessing, and provide an obvious "escalation" path in the face of silence (or uncommitted Acks).

3) Maintainers (sanely) balk at getting a massive dump of patches all at once. It would be nice to document "batch limits" in the MAINTAINERS file too. (For example, davem asked me after I sent him 60 patches in a single day if I could please limit the batch size to 12 between commits (i.e. only send the next batch after the prior batch has been applied/processed). "H:"ow many at once, maybe?

4) I've taken from the example of akpm and -tip commits to include "Cc:" lines before my S-o-b. This helps me record who should be notified about patches when emailing them, but it is a bit bulky. What are best practices here?

5) When sending a large series, it's infeasible to either repeat the massive cover letter (or patch #1) description in every follow-up patch, or to Cc everyone to the 0/N cover letter. Suggestions from some people have been, "I don't care, CC me to everything" and "OMG, don't CC me about all these other subsystem patches I don't care about". My understanding was that everyone should be subscribed to lkml, and it acts as the common thread archive, so when a maintainer gets a few patches out of a /N series, they can trivially go look at the 0/N patch for more context. However, I've regularly gotten "please CC me on the 0/N patch" which puts me back to square one. What should be the definitive way to deal with this? (And if there's no one way, do we have to include CC preferences in MAINTAINERS on a per-person basis?)

6) Dealing with silence. This is not unique to other patch submissions, but silence can block an API conversion, or feature coverage. I've ultimately just used my best judgement and carried dependent patches in my own tree if they go ignored for too long. It seems to be implicitly okay for some contributors to do this (and I tenuously count myself in that list now), but I think it'd be nice to have some kind of more well defined hierarchy of escalation (see #2) that doesn't end in silence or a catch-22 (e.g. while I tend to include akpm in my escalation list, akpm has (correctly) wanted VFS changes to go through Al, but I only went to akpm because Al was busy, so then I'm stuck without a way forward). Now, I realize this is somewhat "by design" (see #3): if a maintainer is overloaded, we don't want things going unreviewed into the kernel. I think we need a bottleneck-detection process, so we can more quickly deal with maintainer silence. That said, it's not like people are falling over themselves to offer to share maintainership of areas that get saturated, so ... I don't have even a straw-man suggestion for this one...

7) As seen in this topic thread, some maintainers absolutely do not want stuff landing from "outside" their tree. This doesn't seem feasible at all, and we do regular treewide conversions at -rc1. It would be nice to declare, explicitly, "you need to deal with external changes to your tree when you merge -rc1". I thought this was already an implicit understanding. (Though in fact, sfr suggested to me that maintainer trees should be based at least on -rc2, and I've seen some maintainers continue to merge -rc releases, which I thought created ugly merge commit hell when sending a pull request to Linus later, but now I've digressed.)

8) Whatever the results of this, I'd really like to get _something_ documented as an adjunct to the SubmittingPatches document. Maybe named TreewideChanges or MultiSubsystemChanges or something. I'm happy to DO this documentation, I just want to have consensus on the ways to do things, and then I can point maintainers to the document to explain why I did something the way I did.

That's all that jumps to mind at the moment...

-Kees

Signature · 2 lines
Kees Cook
Pixel Security
#31 lore raw
Quoted 26 lines · Kees Cook
Quoted 19 lines · Joe Perches
Quoted 11 lines · Kees Cook
Quoted 8 lines · James Bottomley
Quoted 3 lines · Theodore Ts'o

Appendix: Other topics that were brought up
[...]
Developing across multiple areas of the kernel

I've got a couple of extra possibilities
[...]
2) Trivial patches (again).

Quoted 6 lines

Given that the "trivial patches" topic's discussion ended up boiling down to a discussion about developing across multiple areas of the kernel, maybe we should make space for a "tree-wide changes" discussion? Even after the earlier thread about it, I tripped all over this in the last couple months while doing timer conversions, so I would at least have some more strong opinions on the subject. ;)

Quoted 3 lines

It's a ripe area (like months old limburger cheese) for discussion.

There's currently no good way to do tree-wide changes.

Some things stand out for me:

Quoted 8 lines

1) I would like a standard way to distinguish patch submissions between "please ack this (it's going into my tree)" and "please apply this to your tree." I have tried post-"---"-line notes, cover letter notes, etc, and maintainers still miss it. It can sometimes be very disruptive (to both me and the maintainer) to have a maintainer take a patch out of the middle of a series that was intending to land via a different tree. Would "[ACK-PLEASE][PATCH]" be sufficient? Or "[MY-TREE]" or something?

Nothing is going into my tree, since I don't have one. Most changes I do are independent, so I hope that the recipient of the patch will take it. I only send such patches to the maintainers of the patch, with the cover letter CCd to some superset of all relevant mailing lists. I don't really know what to do with dependent patches. Sending all patches to the union of all maintainers can lead to a huge CC list. In that case, I would have to hope that someone who step up to pick up the patch, perhaps the person who is maintaining the dependency part, or when someone asked for the change, the person whoc asked for it in the first place.

Quoted 15 lines

2) When you have a 200+ patch series, it is outrageously difficult to figure out where to send things. The MAINTAINERS file is at best an approximation. I use a subset of the get_maintainer output plus my own parsing of MAINTAINERS to try to organize patches. The results tend to be somewhat okay, but there are still bouncing addresses, or MIA maintainers. And then a patch isn't met with silence, it might get met with an "Ack", but no attention from a committer. Having a representation of the "tree" of maintainership would be much more helpful. In other words, for every section with an "M:" line, also something "U:" ("upstream"?) that indicates either another section or a person that is the "upstream" for that maintainer. This would allow for a sane set of "Cc"s not based on git log guessing, and provide an obvious "escalation" path in the face of silence (or uncommitted Acks).

I send things to maintainers and mailing lists only. My hypothesis is that the things affected by treewide canges are typically not things that other developers feel a strong ownership of.

Quoted 6 lines

8) Whatever the results of this, I'd really like to get _something_ documented as an adjunct to the SubmittingPatches document. Maybe named TreewideChanges or MultiSubsystemChanges or something. I'm happy to DO this documentation, I just want to have consensus on the ways to do things, and then I can point maintainers to the document to explain why I did something the way I did.

Documentation would indeed be very helpful.

Another question is how a patch series should be cut up? Some people have complained about it being cut up by file, if the changes are all going into the same tree. And of course there are complaints if files from two trees are mixed into a single patch. I normally cut them up by unique set of maintainers, but sometimes quite different files get put into a single patch, or files that are very similar get split between different patches just because there is one extra maintainer on one of them. Would it be better to follow the T: entry in MAINTAINERS, if there is one? That information doesn't seem to be complete.

julia

#32 lore raw
Quoted 38 lines · Julia Lawall
Quoted 26 lines · Kees Cook
Quoted 19 lines · Joe Perches
Quoted 11 lines · Kees Cook
Quoted 8 lines · James Bottomley
Quoted 3 lines · Theodore Ts'o

Appendix: Other topics that were brought up
[...]
Developing across multiple areas of the kernel

I've got a couple of extra possibilities
[...]
2) Trivial patches (again).

Quoted 6 lines

Given that the "trivial patches" topic's discussion ended up boiling down to a discussion about developing across multiple areas of the kernel, maybe we should make space for a "tree-wide changes" discussion? Even after the earlier thread about it, I tripped all over this in the last couple months while doing timer conversions, so I would at least have some more strong opinions on the subject. ;)

Quoted 3 lines

It's a ripe area (like months old limburger cheese) for discussion.

There's currently no good way to do tree-wide changes.

Some things stand out for me:

Quoted 8 lines

1) I would like a standard way to distinguish patch submissions between "please ack this (it's going into my tree)" and "please apply this to your tree." I have tried post-"---"-line notes, cover letter notes, etc, and maintainers still miss it. It can sometimes be very disruptive (to both me and the maintainer) to have a maintainer take a patch out of the middle of a series that was intending to land via a different tree. Would "[ACK-PLEASE][PATCH]" be sufficient? Or "[MY-TREE]" or something?

Quoted 1 line

Nothing is going into my tree, since I don't have one.

Me too.

Quoted 2 lines

Most changes I do
are independent, so I hope that the recipient of the patch will take it.

And generally I will only send such a patch series once.

Quoted 7 lines

I only send such patches to the maintainers of the patch, with the cover letter CCd to some superset of all relevant mailing lists. I don't really know what to do with dependent patches. Sending all patches to the union of all maintainers can lead to a huge CC list. In that case, I would have to hope that someone who step up to pick up the patch, perhaps the person who is maintaining the dependency part, or when someone asked for the change, the person whoc asked for it in the first place.

I generally send treewide patches by second-level directory, third if it's drivers/net/

Quoted 2 lines
Quoted 2 lines

2) When you have a 200+ patch series, it is outrageously difficult to figure out where to send things.

More like impossible.

Quoted 4 lines
Quoted 4 lines

This would allow
for a sane set of "Cc"s not based on git log guessing, and provide an obvious "escalation" path in the face of silence (or uncommitted Acks).

More likely a treewide maintainer for the obvious/trivial but acceptable would help more.

Quoted 3 lines

I send things to maintainers and mailing lists only. My hypothesis is that the things affected by treewide canges are typically not things that other developers feel a strong ownership of.

Unfortunately, that's also the class of patches that no one cares much about.

Quoted 9 lines
Quoted 6 lines

8) Whatever the results of this, I'd really like to get _something_ documented as an adjunct to the SubmittingPatches document. Maybe named TreewideChanges or MultiSubsystemChanges or something. I'm happy to DO this documentation, I just want to have consensus on the ways to do things, and then I can point maintainers to the document to explain why I did something the way I did.

Documentation would indeed be very helpful.

Quoted 9 lines

Another question is how a patch series should be cut up? Some people have complained about it being cut up by file, if the changes are all going into the same tree. And of course there are complaints if files from two trees are mixed into a single patch. I normally cut them up by unique set of maintainers, but sometimes quite different files get put into a single patch, or files that are very similar get split between different patches just because there is one extra maintainer on one of them. Would it be better to follow the T: entry in MAINTAINERS, if there is one? That information doesn't seem to be complete.

It's not and it's also incomplete when overlap of ownership occurs.

#33 lore raw
Quoted 39 lines · Joe Perches
Quoted 38 lines · Julia Lawall
Quoted 34 lines · Kees Cook
Quoted 19 lines · Joe Perches
Quoted 11 lines · Kees Cook
Quoted 8 lines · James Bottomley
Quoted 3 lines · Theodore Ts'o

Appendix: Other topics that were brought up
[...]
Developing across multiple areas of the kernel

I've got a couple of extra possibilities
[...]
2) Trivial patches (again).

Quoted 6 lines

Given that the "trivial patches" topic's discussion ended up boiling down to a discussion about developing across multiple areas of the kernel, maybe we should make space for a "tree-wide changes" discussion? Even after the earlier thread about it, I tripped all over this in the last couple months while doing timer conversions, so I would at least have some more strong opinions on the subject. ;)

Quoted 3 lines

It's a ripe area (like months old limburger cheese) for discussion.

There's currently no good way to do tree-wide changes.

Some things stand out for me:

1) I would like a standard way to distinguish patch submissions between "please ack this (it's going into my tree)" and "please apply this to your tree." I have tried post-"---"-line notes, cover letter notes, etc, and maintainers still miss it. It can sometimes be very disruptive (to both me and the maintainer) to have a maintainer take a patch out of the middle of a series that was intending to land via a different tree. Would "[ACK-PLEASE][PATCH]" be sufficient? Or "[MY-TREE]" or something?

Quoted 8 lines
Quoted 1 line

Nothing is going into my tree, since I don't have one.

Me too.

Quoted 2 lines

Most changes I do
are independent, so I hope that the recipient of the patch will take it.

And generally I will only send such a patch series once.

Likewise.

julia

Quoted 54 lines
Quoted 8 lines

I only send such patches to the maintainers of the patch, with the cover letter CCd to some superset of all relevant mailing lists. I don't really know what to do with dependent patches. Sending all patches to the union of all maintainers can lead to a huge CC list. In that case, I would have to hope that someone who step up to pick up the patch, perhaps the person who is maintaining the dependency part, or when someone asked for the change, the person whoc asked for it in the first place.

I generally send treewide patches by second-level directory, third if it's drivers/net/

Quoted 2 lines
Quoted 2 lines

2) When you have a 200+ patch series, it is outrageously difficult to figure out where to send things.

More like impossible.

Quoted 4 lines
Quoted 4 lines

This would allow
for a sane set of "Cc"s not based on git log guessing, and provide an obvious "escalation" path in the face of silence (or uncommitted Acks).

More likely a treewide maintainer for the obvious/trivial but acceptable would help more.

Quoted 3 lines

I send things to maintainers and mailing lists only. My hypothesis is that the things affected by treewide canges are typically not things that other developers feel a strong ownership of.

Unfortunately, that's also the class of patches that no one cares much about.

Quoted 9 lines
Quoted 6 lines

8) Whatever the results of this, I'd really like to get _something_ documented as an adjunct to the SubmittingPatches document. Maybe named TreewideChanges or MultiSubsystemChanges or something. I'm happy to DO this documentation, I just want to have consensus on the ways to do things, and then I can point maintainers to the document to explain why I did something the way I did.

Documentation would indeed be very helpful.

Quoted 9 lines

Another question is how a patch series should be cut up? Some people have complained about it being cut up by file, if the changes are all going into the same tree. And of course there are complaints if files from two trees are mixed into a single patch. I normally cut them up by unique set of maintainers, but sometimes quite different files get put into a single patch, or files that are very similar get split between different patches just because there is one extra maintainer on one of them. Would it be better to follow the T: entry in MAINTAINERS, if there is one? That information doesn't seem to be complete.

It's not and it's also incomplete when overlap of ownership occurs.

#34 lore raw
In reply to Kees Cook

Kees,

Quoted 6 lines

3) Maintainers (sanely) balk at getting a massive dump of patches all at once. It would be nice to document "batch limits" in the MAINTAINERS file too. (For example, davem asked me after I sent him 60 patches in a single day if I could please limit the batch size to 12 between commits (i.e. only send the next batch after the prior batch has been applied/processed). "H:"ow many at once, maybe?

I generally think 10-15 patches are reasonable. But it also depends on the patch size and the nature of the changes.

For me, the deciding factors are:

1. What's the point of this series?

2. Can I complete review of this series within 30 minutes before my next concall.

3. Can I complete review of this series without my brain turning into complete mush.

Quoted 5 lines

5) When sending a large series, it's infeasible to either repeat the massive cover letter [...] My understanding was that everyone should be subscribed to lkml, and it acts as the common thread archive, so when a maintainer gets a few patches out of a /N series, they can trivially go look at the 0/N patch for more context.

First of all, as a subsystem maintainer I rely heavily on other people to review patches. Either individual driver maintainers or people with a vested interest in SCSI (enterprise distro developers).

It's virtually impossible to entice people to review patches in the first place. Forcing people to go switch from their email context to go peruse the lkml archives in a web browser to decide whether a patch series is worth reviewing in the first place is a non-starter.

People have really short attention spans. Realistically, reviews only happen when people see the mail in their inbox and they have N minutes to spare. After that point, your opportunity is essentially lost.

If a reviewer has a particular interest in a file or area, they may tag it for later review. And then after a few days of working on something else they come back and realize they have no time this week and delete the series from their inbox so they don't feel bad about it over the weekend.

As a subsystem maintainer, the burden then falls upon me. And if something has been collecting dust in patchwork for several weeks, there is absolutely no chance that I or anybody else will get to it. I'm reasonably sure that I'm the only person that ever visits the SCSI patchwork with the intent to clear out the queue.

Consequently, the only way to revive interest is to resubmit. With a comprehensive cover letter (which, incidentally, it would be nice if patchwork would capture).

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
#35 lore raw
Quoted 3 lines · Martin K. Petersen

Consequently, the only way to revive interest is to resubmit. With a comprehensive cover letter (which, incidentally, it would be nice if patchwork would capture).

Oh, yes! I have repeatedly wanted this too (and maybe a way to select the entire series from the 0/N to make a bundle, instead of clicking each patch).

-Kees

Signature · 2 lines
Kees Cook
Pixel Security
#36 lore raw

Kees,

Quoted 3 lines

Oh, yes! I have repeatedly wanted this too (and maybe a way to select the entire series from the 0/N to make a bundle, instead of clicking each patch).

I believe the latter is in the pipeline. But it can't happen soon enough...

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
#37 lore raw
In reply to Kees Cook
Quoted 25 lines · Kees Cook
Quoted 19 lines · Joe Perches
Quoted 11 lines · Kees Cook
Quoted 8 lines · James Bottomley
Quoted 3 lines · Theodore Ts'o

Appendix: Other topics that were brought up
[...]
Developing across multiple areas of the kernel

I've got a couple of extra possibilities
[...]
2) Trivial patches (again).

Quoted 6 lines

Given that the "trivial patches" topic's discussion ended up boiling down to a discussion about developing across multiple areas of the kernel, maybe we should make space for a "tree-wide changes" discussion? Even after the earlier thread about it, I tripped all over this in the last couple months while doing timer conversions, so I would at least have some more strong opinions on the subject. ;)

Quoted 3 lines

It's a ripe area (like months old limburger cheese) for discussion.

There's currently no good way to do tree-wide changes.

Some things stand out for me:

< snip >

Quoted 6 lines

3) Maintainers (sanely) balk at getting a massive dump of patches all at once. It would be nice to document "batch limits" in the MAINTAINERS file too. (For example, davem asked me after I sent him 60 patches in a single day if I could please limit the batch size to 12 between commits (i.e. only send the next batch after the prior batch has been applied/processed). "H:"ow many at once, maybe?

< snip >

But patches sent to a list are not just for the maintainer. They are also for anyone else who may be affected or who may care to review the patches. Those other people have their own schedule issues.

If you hold off sending a second patch set this week, then send it next week (when the maintainer once again has some bandwith), it is likely that someone other than the maintainer has time to look at that patche set this week, but may have no free time to do so next week.

-Frank

#38 lore raw
In reply to Kees Cook
Quoted 8 lines · Kees Cook

with an "Ack", but no attention from a committer. Having a representation of the "tree" of maintainership would be much more helpful. In other words, for every section with an "M:" line, also something "U:" ("upstream"?) that indicates either another section or a person that is the "upstream" for that maintainer. This would allow for a sane set of "Cc"s not based on git log guessing, and provide an obvious "escalation" path in the face of silence (or uncommitted Acks).

IME if you use all matching MAINTAINERS entries rather than the most exact one you'll usually find the upstream (though there are exceptions, including things like arm-soc which just isn't in MAINTAINERS at all).

Quoted 7 lines

7) As seen in this topic thread, some maintainers absolutely do not want stuff landing from "outside" their tree. This doesn't seem feasible at all, and we do regular treewide conversions at -rc1. It would be nice to declare, explicitly, "you need to deal with external changes to your tree when you merge -rc1". I thought this was already an implicit understanding. (Though in fact, sfr suggested to me that maintainer trees should be based at least on -rc2, and I've seen some

I think like a lot of these things this is a tradeoffs thing - sometimes everything will have to go in at once and we'll have to deal with it but it's better to at least try to coordinate with people's trees, if that's not possible it's annoying but understandable. It's more likely to be a problem when things just materialize without any warning.

#39 lore raw
In reply to Kees Cook

Hi Kees,

Quoted 25 lines · On Wednesday, 25 October 2017 03:54:47 EEST Kees Cook wrote:
Quoted 18 lines · Joe Perches
Quoted 10 lines · Kees Cook
Quoted 8 lines · James Bottomley
Quoted 3 lines · Theodore Ts'o

Appendix: Other topics that were brought up
[...]
Developing across multiple areas of the kernel

I've got a couple of extra possibilities
[...]
2) Trivial patches (again).

Quoted 6 lines

Given that the "trivial patches" topic's discussion ended up boiling down to a discussion about developing across multiple areas of the kernel, maybe we should make space for a "tree-wide changes" discussion? Even after the earlier thread about it, I tripped all over this in the last couple months while doing timer conversions, so I would at least have some more strong opinions on the subject. ;)

Quoted 3 lines

It's a ripe area (like months old limburger cheese) for discussion.

There's currently no good way to do tree-wide changes.

Some things stand out for me:

Quoted 8 lines

1) I would like a standard way to distinguish patch submissions between "please ack this (it's going into my tree)" and "please apply this to your tree." I have tried post-"---"-line notes, cover letter notes, etc, and maintainers still miss it. It can sometimes be very disruptive (to both me and the maintainer) to have a maintainer take a patch out of the middle of a series that was intending to land via a different tree. Would "[ACK-PLEASE][PATCH]" be sufficient? Or "[MY-TREE]" or something?

When I receive patches from such series, I often assume (and often incorrectly it seems after reading this thread) that the submitter will want to merge the whole series through a single tree. I try to always ask when sending my ack whether I should take the patch in my tree or if it will be merged through another tree. This wastes bandwidth, and would be pretty painful for you if you had to tell 200 times maintainers to take the patches. I think documenting the expected default upstream path for tree-wide changes would help here.

[snip]

Signature · 3 lines
Regards,

Laurent Pinchart
#40 lore raw
In reply to Kees Cook
Quoted 26 lines · Kees Cook
Quoted 19 lines · Joe Perches
Quoted 11 lines · Kees Cook
Quoted 8 lines · James Bottomley
Quoted 3 lines · Theodore Ts'o

Appendix: Other topics that were brought up
[...]
Developing across multiple areas of the kernel

I've got a couple of extra possibilities
[...]
2) Trivial patches (again).

Quoted 6 lines

Given that the "trivial patches" topic's discussion ended up boiling down to a discussion about developing across multiple areas of the kernel, maybe we should make space for a "tree-wide changes" discussion? Even after the earlier thread about it, I tripped all over this in the last couple months while doing timer conversions, so I would at least have some more strong opinions on the subject. ;)

Quoted 3 lines

It's a ripe area (like months old limburger cheese) for discussion.

There's currently no good way to do tree-wide changes.

Some things stand out for me:

Quoted 8 lines

1) I would like a standard way to distinguish patch submissions between "please ack this (it's going into my tree)" and "please apply this to your tree." I have tried post-"---"-line notes, cover letter notes, etc, and maintainers still miss it. It can sometimes be very disruptive (to both me and the maintainer) to have a maintainer take a patch out of the middle of a series that was intending to land via a different tree. Would "[ACK-PLEASE][PATCH]" be sufficient? Or "[MY-TREE]" or something?

I've used To vs. Cc to distinguish that, but that seems to be not explicit enough.

Perhaps a "Needs-Acked-by:" tag. That would have the advantage of being stored in the commit rather than having to be added when sending. It's also easy for the person needing to ack it to filter for it. The more we can automate the process from having a git branch of commits to sending mails, the less variation we have and the easier it will be for new people.

Quoted 6 lines

2) When you have a 200+ patch series, it is outrageously difficult to figure out where to send things. The MAINTAINERS file is at best an approximation. I use a subset of the get_maintainer output plus my own parsing of MAINTAINERS to try to organize patches. The results tend to be somewhat okay, but there are still bouncing addresses, or MIA maintainers. And then a patch isn't met with silence, it might get met

Quoted 8 lines

with an "Ack", but no attention from a committer. Having a representation of the "tree" of maintainership would be much more helpful. In other words, for every section with an "M:" line, also something "U:" ("upstream"?) that indicates either another section or a person that is the "upstream" for that maintainer. This would allow for a sane set of "Cc"s not based on git log guessing, and provide an obvious "escalation" path in the face of silence (or uncommitted Acks).

I think distinguishing between subsystem maintainers and maintainers of individual things (e.g. a driver) would be good. I think that's what you are saying. Ideally, that distinction would make it to the Cc list somehow. I usually add Cc's from get_maintainers.pl to commits, but then by the time I'm sending things I may not know who in the list has the upstream tree.

The git log guessing is pretty useless for the purpose of Cc'ing people and should be off by default IMO. I've touched things tree wide a number of times and now get Cc'ed on things I've touched once 3 years ago.

Rob

#41 lore raw
Quoted 36 lines · Rob Herring
Quoted 26 lines · Kees Cook
Quoted 19 lines · Joe Perches
Quoted 11 lines · Kees Cook
Quoted 8 lines · James Bottomley
Quoted 3 lines · Theodore Ts'o

Appendix: Other topics that were brought up
[...]
Developing across multiple areas of the kernel

I've got a couple of extra possibilities
[...]
2) Trivial patches (again).

Quoted 6 lines

Given that the "trivial patches" topic's discussion ended up boiling down to a discussion about developing across multiple areas of the kernel, maybe we should make space for a "tree-wide changes" discussion? Even after the earlier thread about it, I tripped all over this in the last couple months while doing timer conversions, so I would at least have some more strong opinions on the subject. ;)

Quoted 3 lines

It's a ripe area (like months old limburger cheese) for discussion.

There's currently no good way to do tree-wide changes.

Some things stand out for me:

Quoted 8 lines

1) I would like a standard way to distinguish patch submissions between "please ack this (it's going into my tree)" and "please apply this to your tree." I have tried post-"---"-line notes, cover letter notes, etc, and maintainers still miss it. It can sometimes be very disruptive (to both me and the maintainer) to have a maintainer take a patch out of the middle of a series that was intending to land via a different tree. Would "[ACK-PLEASE][PATCH]" be sufficient? Or "[MY-TREE]" or something?

Quoted 2 lines

I've used To vs. Cc to distinguish that, but that seems to be not explicit enough.

Yeah, same for me.

Quoted 6 lines

Perhaps a "Needs-Acked-by:" tag. That would have the advantage of being stored in the commit rather than having to be added when sending. It's also easy for the person needing to ack it to filter for it. The more we can automate the process from having a git branch of commits to sending mails, the less variation we have and the easier it will be for new people.

I tend to be faced with not knowing which person I need for an Ack. :P

The idea proposed at Kernel Summit was to add a new subject tag "Request for Ack", as:

[RFACK][PATCH] subsystem: title...

Quoted 15 lines
Quoted 6 lines

2) When you have a 200+ patch series, it is outrageously difficult to figure out where to send things. The MAINTAINERS file is at best an approximation. I use a subset of the get_maintainer output plus my own parsing of MAINTAINERS to try to organize patches. The results tend to be somewhat okay, but there are still bouncing addresses, or MIA maintainers. And then a patch isn't met with silence, it might get met

Quoted 8 lines

with an "Ack", but no attention from a committer. Having a representation of the "tree" of maintainership would be much more helpful. In other words, for every section with an "M:" line, also something "U:" ("upstream"?) that indicates either another section or a person that is the "upstream" for that maintainer. This would allow for a sane set of "Cc"s not based on git log guessing, and provide an obvious "escalation" path in the face of silence (or uncommitted Acks).

Quoted 3 lines

I think distinguishing between subsystem maintainers and maintainers of individual things (e.g. a driver) would be good. I think that's what you are saying. Ideally, that distinction would make it to the Cc

Yeah, exactly. And time times (wireless comes to mind) you have a couple levels of maintainers of the "individual thing" maintainers.

Quoted 8 lines

list somehow. I usually add Cc's from get_maintainers.pl to commits, but then by the time I'm sending things I may not know who in the list has the upstream tree.

The git log guessing is pretty useless for the purpose of Cc'ing people and should be off by default IMO. I've touched things tree wide a number of times and now get Cc'ed on things I've touched once 3 years ago.

Yeah, I fear my inbox after timer conversions start landing...

-Kees

Signature · 2 lines
Kees Cook
Pixel Security