Documentation/process: Add Linux Kernel Contribution Maturity Model
Theodore Ts'o
maintainerPATCH 1/1 Documentation/process: Add Linux Kernel Contribution Maturity Model
As a follow-up to a discussion at the 2021 Maintainer's Summit on the topic of maintainer recruitment and retention, the TAB took on the task of creating a document which to help companies and other organizations to grow in their ability to engage with the Linux Kernel development community, using the Maturity Model[2] framework.
The goal is to encourage, in a management-friendly way, companies to allow their engineers to contribute with the upstream Linux Kernel development community, so we can grow the "talent pipeline" for contributors to become respected leaders, and eventually kernel maintainers.
[1] https://lwn.net/Articles/870581/
[2] https://en.wikipedia.org/wiki/Maturity_model
--- .../process/contribution-maturity-model.rst | 111 ++++++++++++++++++ Documentation/process/index.rst | 1 + 2 files changed, 112 insertions(+) create mode 100644 Documentation/process/contribution-maturity-model.rst
Documentation/process/contribution-maturity-model.rst new file +111 -0
@@ -0,0 +1,111 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. _contribution_maturity_model: + +======================================== +Linux Kernel Contribution Maturity Model +======================================== + + +Background +========== + +As a part of the 2021 Linux Kernel Maintainers’ Summit, there was a +`discussion <https://lwn.net/Articles/870581/>`_ about the challenges in +recruiting kernel maintainers as well as maintainer succession. Some of +the conclusions from that discussion included that companies which are a +part of the Linux Kernel community need to allow engineers to be +maintainers as part of their job, so they can grow into becoming +respected leaders and eventually, kernel maintainers. To support a +strong talent pipeline, developers need to allowed and encouraged to +take on upstream contributions such as reviewing other people’s patches, +refactoring kernel infrastructure, and writing documentation. + +To that end, the Linux Foundation Technical Advisory Board (TAB) +proposes this Linux Kernel Contribution Maturity Model. These common +expectations for upstream community engagement aim to increase the +influence of individual developers, increase the collaboration of +organizations, and improve the overall health of the Linux Kernel +ecosystem. + +The TAB urges organizations to continuously evaluate their Open Source +maturity model and commit to improvements to align with this model. To +be effective, this evaluation should incorporate feedback from across +the organization, including management and developers at all seniority +levels. In the spirit of Open Source, we encourage organizations to +publish their evaluations and plans to improve their engagement with the +upstream community. + +Level 0 +======= + +* Software Engineers are not allowed to contribute patches to the Linux + kernel. + + +Level 1 +======= + +* Software Engineers are allowed to contribute patches to the Linux + kernel, either as part of their job responsibilities or on their own + time. + +Level 2 +======= + +* Software Engineers are expected to contribute to the Linux Kernel as + part of their job responsibilities. +* Software Engineers will be supported to attend Linux-related + conferences as a part of their job. +* A Software Engineer’s upstream code contributions shall be considered + in promotion and performance reviews. + +Level 3 +======= + +* Software Engineers are expected to review patches (including patches + authored by engineers from other companies) as part of their job + responsibilities +* Contributing presentations or papers to Linux-related conferences are + considered part of an engineer’s work. +* A Software Engineer’s community contributions shall be considered in + promotion and performance reviews. +* Organizations shall regularly report metrics of their open source + contributions and track these metrics over time. These metrics may be + published only internally within the organization, or at the + organization’s discretion, some or all may be published externally. + Metrics that are strongly suggested include: + + * The number of upstream kernel contributions by engineer, team, and + organization (e.g., all people reporting up to a manager, director, + or VP). + * The percentage of kernel developers who have made upstream + contributions relative to the total kernel developers in the + organization. + * The time interval between kernels used in the organization’s servers + and/or products, and the publication date of the upstream kernel + upon which the internal kernel is based. + * The number of out-of-tree commits present in internal kernels. + +Level 4 +======= + +* Software Engineers are encouraged to spend a portion of their work + time focused on Upstream Work, which is defined as reviewing patches + or papers, improving core project infrastructure such as writing or + maintaining tests, upstream tech debt reduction, writing + documentation, etc. +* Software Engineers are supported in helping to organize Linux-related + conferences. +* Organizations shall consider community member feedback in official + performance reviews. + +Level 5 +======= + +* Upstream kernel development is considered a formal job position, with + at least a third of the engineer’s time spent doing Upstream Work. +* Organizations shall actively seek out community member feedback as a + factor in official performance reviews. +* Organizations shall regularly report internally on the ratio of + Upstream Work to work focused on directly pursuing business goals.
Documentation/process/index.rst +1 -0
@@ -50,6 +50,7 @@ Other guides to the community that are of interest to most developers are: embargoed-hardware-issues maintainers researcher-guidelines + contribution-maturity-model These are some overall technical guides that have been put here for now for lack of a better place.
Signature · 1 line
2.31.0
Documentation/process/contribution-maturity-model.rst line 81
Quoted 3 lines · Theodore Ts'o
Documentation/process/contribution-maturity-model.rst new file +3
-0+ * The number of upstream kernel contributions by engineer, team, and + organization (e.g., all people reporting up to a manager, director, + or VP).
This is the only one that gave me pause. Seems eminently gameable by trash variable-name-tweaking patches, unnecessary commit splitting etc. I'm not going to name names but I'm seeing some of this as it is. And if people decide to count patches posted as opposed to commits, oh my ... I don't think we should make people focus on the number of contributions as opposed to their quality, and incorrect incentives tend to promote incorrect behaviour.
Fundamentally I presume companies will evaluate contributions based on how important the given functionality is for the company, so I'm not sure we need to get involved here. Maybe just add this at the end:
+ The metrics can be measured by team or + organization (e.g., all people reporting up to a manager, director, + or VP).
Or simply drop this part.
Besides this:
Signature · 1 line
MST
Quoted 5 lines
The goal is to encourage, in a management-friendly way, companies to allow their engineers to contribute with the upstream Linux Kernel development community, so we can grow the "talent pipeline" for contributors to become respected leaders, and eventually kernel maintainers.
These are noble goals. Also the bullet list is short, formal, and to the point. This is nice.
I kind of side with Michael Tsirkin's point that quantitative measures of performance can be harmful or give a false impression of control. Consider this example:
linux$ git log --oneline --author=Walleij |wc -l
4301
linux$ git log --oneline --author=McKenney |wc -l
3346
There is something about some code in the kernel being more "core" than others that needs to be taken into account, albeit it is maybe an elusive concept. There is a hierarchy from contributing syntactic changes to coding style to contributing and maintaining say RCU.
That can be judged by peers and is addressed in other points in the document so what about just dropping this paragraph?
Then there is this:
Documentation/process/contribution-maturity-model.rst
Quoted 3 lines
Documentation/process/contribution-maturity-model.rst new file +3
-0+* Software Engineers are encouraged to spend a portion of their work + time focused on Upstream Work, which is defined as reviewing patches + or papers,
Papers! If we are linking ourselves to the Scientific Community (and I see that as a good thing) we should certainly drop a few words about it. It's a place of (ideally) meritocracy and formal review of contributions.
Here is a real nice paper:
https://kernel.dk/systor13-final18.pdf
as a community we should probably produce more like this.
Or maybe you meant to say "mails". I don't know.
On a side note I would say that this document creates a really serious cognitive dissonance with the document Documentation/process/management-style.rst
so just for the fun of it I think this document should reference that latter document as a guideline for how Software Engineers will be evaluated in the community when they take on . It is a nice nod to the nature of conflicting interests and worldviews we need to deal with.
Yours,
Linus Walleij
Quoted 18 lines · Linus Walleij
Quoted 5 lines
The goal is to encourage, in a management-friendly way, companies to allow their engineers to contribute with the upstream Linux Kernel development community, so we can grow the "talent pipeline" for contributors to become respected leaders, and eventually kernel maintainers.
These are noble goals. Also the bullet list is short, formal, and to the point. This is nice.
I kind of side with Michael Tsirkin's point that quantitative measures of performance can be harmful or give a false impression of control. Consider this example:
linux$ git log --oneline --author=Walleij |wc -l
4301
linux$ git log --oneline --author=McKenney |wc -l
3346
Quoted 5 lines
There is something about some code in the kernel being more "core" than others that needs to be taken into account, albeit it is maybe an elusive concept. There is a hierarchy from contributing syntactic changes to coding style to contributing and maintaining say RCU.
Don't forget reputation produced as a result of contributions from the company as a whole. For example, cdjrlc.com is known for noisy "redundant word"-stripping patches and ignoring critical community reviews; ZTE with "atypical" corporate email setup for which Greg dislike; and Intel which (as you may know) has positive legacy to the community with its high-quality contributions.
Thanks.
Signature · 1 line
An old man doll... just what I always wanted! - Clara
Documentation/process/contribution-maturity-model.rst line 43
Quoted 2 lines · Theodore Ts'o
Documentation/process/contribution-maturity-model.rst new file +2
-0+* Software Engineers are not allowed to contribute patches to the Linux + kernel.
Software engineers turned product manager or support desk people?
Documentation/process/contribution-maturity-model.rst line 87
Quoted 3 lines
Documentation/process/contribution-maturity-model.rst new file +3
-0+ * The time interval between kernels used in the organization’s servers + and/or products, and the publication date of the upstream kernel + upon which the internal kernel is based.
I.e. the time for rebasing internal kernel against upstream one? For example rebasing against several stable minor releases (x.y.z) is easier than forward porting out-of-tree commits into new upstream release (x.y).
Thanks.
Signature · 1 line
An old man doll... just what I always wanted! - Clara
Theodore Ts'o
maintainerQuoted 5 lines · Bagas Sanjaya
Documentation/process/contribution-maturity-model.rst line 43
Quoted 2 lines · Theodore Ts'o
Documentation/process/contribution-maturity-model.rst new file +2
-0+* Software Engineers are not allowed to contribute patches to the Linux + kernel.Software engineers turned product manager or support desk people?
Actually, what we were thinking of companies that were paranoid about the GPL. For example, there is a particular major hardware supplier used by many in the Linux ecosystem where the senior management tends to be mostly composed of patent lawyers....
(In the United States, most companies have employment contracts which means that all intellectual output, either on or off work. Which means that some companies can prohibit their employees from contributing anything to any open source project, even if the work is done after hours on their own time. So while "Level 0" may seem horrific, there are software engineers in the US who are constrained in this way. If we can encourage those companies to move from Level 0 to Level 1, that's at least a step in the right direction.)
Quoted 8 lines
Documentation/process/contribution-maturity-model.rst line 87
Quoted 3 lines
Documentation/process/contribution-maturity-model.rst new file +3
-0+ * The time interval between kernels used in the organization’s servers + and/or products, and the publication date of the upstream kernel + upon which the internal kernel is based.I.e. the time for rebasing internal kernel against upstream one? For example rebasing against several stable minor releases (x.y.z) is easier than forward porting out-of-tree commits into new upstream release (x.y).
It's not necessarily the time to rebase the kernel. It might take less than, say, six years to rebase a product kernel based on 4.9 to 6.1. But the company might decide not to expend the engineering resources to do the work, but instead to continue shipping product or using a data center kernel based on an antediluvian upstream release. After all, there are many companies using RHEL/CentOS 7, whose kernel is based on 3.10. Yeah, with of tons of backports and (hopefully) all of the necessary security patches, but is that really the best way to run a railroad?
But, yes, if your point is that this metric doesn't distinguish between an organization which is regularly keeping itself with the very latest 4.9.336 kernel, as opposed to reguarly jumping to the most recent LTS kernel (i.e., every year or every other years going from 5.4 to 5.10 to 5.15 to 6.1), yes that's true.
However, it may be that it's better to encourage companies to adopt an "upstream-first" development model. I attempted to capture this in the metric, "number of out-of-tree commits present in internal kernels".
- Ted
Hello,
Quoted 24 lines · Theodore Ts'o
As a follow-up to a discussion at the 2021 Maintainer's Summit on the topic of maintainer recruitment and retention, the TAB took on the task of creating a document which to help companies and other organizations to grow in their ability to engage with the Linux Kernel development community, using the Maturity Model[2] framework.
The goal is to encourage, in a management-friendly way, companies to allow their engineers to contribute with the upstream Linux Kernel development community, so we can grow the "talent pipeline" for contributors to become respected leaders, and eventually kernel maintainers.
[1] https://lwn.net/Articles/870581/
[2] https://en.wikipedia.org/wiki/Maturity_modelSigned-off-by Theodore Ts'o <[email protected]>Co-developed-by Kees Cook <[email protected]>Signed-off-by Kees Cook <[email protected]>Co-developed-by Dan Williams <[email protected]>Signed-off-by Dan Williams <[email protected]>Acked-by Jakub Kicinski <[email protected]>Acked-by Dave Hansen <[email protected]>---
[snip]
Documentation/process/contribution-maturity-model.rst line 37
Quoted 7 lines
Documentation/process/contribution-maturity-model.rst new file +7
-0+The TAB urges organizations to continuously evaluate their Open Source +maturity model and commit to improvements to align with this model. To +be effective, this evaluation should incorporate feedback from across +the organization, including management and developers at all seniority +levels. In the spirit of Open Source, we encourage organizations to +publish their evaluations and plans to improve their engagement with the +upstream community.
You probably need to provide a word or two about who is the intended recipient of this document, whether this is intended to be utilized by software developers, their management chain, or both.
Quoted 12 lines
Documentation/process/contribution-maturity-model.rst new file +12
-0+ +Level 0 +======= + +* Software Engineers are not allowed to contribute patches to the Linux + kernel. + + +Level 1 +======= + +* Software Engineers are allowed to contribute patches to the Linux
Documentation/process/contribution-maturity-model.rst line 59
Quoted 10 lines
Documentation/process/contribution-maturity-model.rst new file +10
-0+ kernel, either as part of their job responsibilities or on their own + time. + +Level 2 +======= + +* Software Engineers are expected to contribute to the Linux Kernel as + part of their job responsibilities. +* Software Engineers will be supported to attend Linux-related + conferences as a part of their job.
This is not something IMHO that qualifies for Level 2, but rather for Level 3. It is reasonably simple and easy for companies to support their engineers contributing to the kernel, it is a whole different thing to financially support them to attend conference(s).
Documentation/process/contribution-maturity-model.rst line 68
Quoted 9 lines
Documentation/process/contribution-maturity-model.rst new file +9
-0+* A Software Engineer’s upstream code contributions shall be considered + in promotion and performance reviews. + +Level 3 +======= + +* Software Engineers are expected to review patches (including patches + authored by engineers from other companies) as part of their job + responsibilities
Would substitute expected with highly encouraged here.
Documentation/process/contribution-maturity-model.rst line 70
Quoted 2 lines
Documentation/process/contribution-maturity-model.rst new file +2
-0+* Contributing presentations or papers to Linux-related conferences are + considered part of an engineer’s work.
Again, that seems more of a Level 4 item to me.
Quoted 7 lines
Documentation/process/contribution-maturity-model.rst new file +7
-0+* A Software Engineer’s community contributions shall be considered in + promotion and performance reviews. +* Organizations shall regularly report metrics of their open source + contributions and track these metrics over time. These metrics may be + published only internally within the organization, or at the + organization’s discretion, some or all may be published externally. + Metrics that are strongly suggested include:
Documentation/process/contribution-maturity-model.rst line 87
Quoted 10 lines
Documentation/process/contribution-maturity-model.rst new file +10
-0+ + * The number of upstream kernel contributions by engineer, team, and + organization (e.g., all people reporting up to a manager, director, + or VP). + * The percentage of kernel developers who have made upstream + contributions relative to the total kernel developers in the + organization. + * The time interval between kernels used in the organization’s servers + and/or products, and the publication date of the upstream kernel + upon which the internal kernel is based.
That part is a bit difficult to generalize, so as a broad definition of metrics to track, it seems appropriate as a possible example. In practice lots of people have to work within the bounds of the kernel they are provided with, and they may not always entirely control the contribution process behind as much as the code getting in. It seems to me that we have usually following categories:
- everything lands upstream and/or working upstream first which is the case with Intel, AMD, Meta, Oracle, etc.
- maintaining a major distribution kernel: Android, RedHat, SUSE, Oracle UKE with as much upstream code downstream as downstream code reaching upstream eventually
- working with a well maintained downstream kernel tree based upon a LTS version from a vendor where most SoCs companies fall these days hopefully
- working with a not so major distribution kernel in the likes of OpenWrt etc.
- unmaintained kernel
Quoted 1 line
+ * The number of out-of-tree commits present in internal kernels.
The metrics that an organization must provide versus the level a given engineer is at is independent from one another, or rather, you may want to create different levels, one for engineers which is what you have, and one for organizations.
So maybe there needs to be 4 levels for an organization:
0 - no support/awareness/will to engage
1 - mild engagement with the community in one form of another, including maintenance of metrics
2 - organization is largely supportive of upstream work and does so with pouring some occasional resources to attend/organize conferences, albeit not at the "gold" sponsor level 3 - organization heavily leans on upstream work for its products and is a regular sponsor at major events
Documentation/process/contribution-maturity-model.rst line 99
Quoted 11 lines
Documentation/process/contribution-maturity-model.rst new file +11
-0+ +Level 4 +======= + +* Software Engineers are encouraged to spend a portion of their work + time focused on Upstream Work, which is defined as reviewing patches + or papers, improving core project infrastructure such as writing or + maintaining tests, upstream tech debt reduction, writing + documentation, etc. +* Software Engineers are supported in helping to organize Linux-related + conferences.
Likewise, this seems to be more of a Level 5 aspect rather than Level 4.
Documentation/process/contribution-maturity-model.rst line 103
Quoted 4 lines
Documentation/process/contribution-maturity-model.rst new file +4
-0+* Organizations shall consider community member feedback in official + performance reviews. + +Level 5
And I would make this essentially Level 6.
Quoted 20 lines
+=======
Documentation/process/contribution-maturity-model.rst new file +7
-0+ +* Upstream kernel development is considered a formal job position, with + at least a third of the engineer’s time spent doing Upstream Work. +* Organizations shall actively seek out community member feedback as a + factor in official performance reviews. +* Organizations shall regularly report internally on the ratio of + Upstream Work to work focused on directly pursuing business goals.Documentation/process/index.rst +1
-0@@ -50,6 +50,7 @@ Other guides to the community that are of interest to most developers are: embargoed-hardware-issues maintainers researcher-guidelines + contribution-maturity-model These are some overall technical guides that have been put here for now for lack of a better place.
Signature · 1 line
Florian
Jonathan Corbet
maintainerQuoted 6 lines · Theodore Ts'o
As a follow-up to a discussion at the 2021 Maintainer's Summit on the topic of maintainer recruitment and retention, the TAB took on the task of creating a document which to help companies and other organizations to grow in their ability to engage with the Linux Kernel development community, using the Maturity Model[2] framework.
Quoted 8 lines
The goal is to encourage, in a management-friendly way, companies to allow their engineers to contribute with the upstream Linux Kernel development community, so we can grow the "talent pipeline" for contributors to become respected leaders, and eventually kernel maintainers.
[1] https://lwn.net/Articles/870581/
[2] https://en.wikipedia.org/wiki/Maturity_model
So there were a few review comments on this document... Ted, were you planning a new revision or is it your feeling that it's ready to be merged in this form?
Thanks,
jon